Evalueserve
  • About
  • Blog
  • Careers
  • mind+machine™
Contact Us
  • Analytics
      • Analytics Overview
      • Marketing & Digital Analytics
      • Sales Analytics
      • Supply Chain & Operations Analytics
      • Pricing & Revenue Management
      • Customer Insights
      • IP & Innovation Analytics
      • Risk & Compliance Analytics
  • Knowledge Services
      • Marketing, Sales & Strategy
      • Marketing and Sales Enablement
      • Strategy Research Support
      • Procurement
      • Procurement Intelligence
      • Innovation Intelligence & IP Solutions
      • IP and R&D Intelligence
      • IP and R&D Search
      • IP Prosecution Services
      • (Eco)Toxicology Consulting Service
      • Professional Services
      • Rapid Research Support
      • Knowledge Management & Transformation
      • Market and Strategic Intelligence
      • Design and Production Support
      • Deals, M&A, and Corporate Finance Support
      • Scaling Data Assets
      • Financial Services
      • Risk & Compliance
      • Investment Banking Support
      • Corporate Banking Support
      • Commercial Banking Support
      • Global Markets Research
      • Index & Quant Solutions
      • Investment Management
      • Life Sciences & Healthcare
      • Biopharmaceuticals, Medical Products & Healthcare
      • Datacrest
  • Service Design Consulting
  • Digital Platforms
      • Insightsfirst
      • Researchstream
      • Spreadsmart
      • Insightloupe
      • Usecasehub
      • Datacrest
      • Pitchready
      • Researchscaler
      • Navigator
      • Publishwise
      • AnalyticsApps
      • Compsbuilder
  • Industries
      • Financial Services
      • Corporate & Investment Banking

      • Capital Markets

      • Investment Management

      • Retail & Commercial Banking

      • Risk & Compliance

      • Manufacturing
      • Automotive

      • Consumer Packaged Goods

      • Engineering

      • Energy

      • Chemicals

      • Life Sciences & Healthcare
      • Bio-Pharmaceuticals

      • Medical Products

      • Healthcare

      • Technology Media & Telecommunications
      • Telecommunications & Media

      • Hardware & Software

      • The Internet of Things

      • Professional Services
      • Consultancy

      • Legal

      • Taxation, Auditing & Accountancy

      • Services
      • Retail

      • Freight & Logistics

      • Utilities

      • Non-Profit Organizations

  • Resources
      • Case Studies
      • Blog
      • White Papers
      • Evalueserve in the News
      • Industry Insights
      • COVID-19 Disruption Insights
      • Events
  • Analytics
      • Analytics Overview
      • Marketing & Digital Analytics
      • Sales Analytics
      • Supply Chain & Operations Analytics
      • Pricing & Revenue Management
      • Customer Insights
      • IP & Innovation Analytics
      • Risk & Compliance Analytics
  • Knowledge Services
      • Marketing, Sales & Strategy
      • Marketing and Sales Enablement
      • Strategy Research Support
      • Procurement
      • Procurement Intelligence
      • Innovation Intelligence & IP Solutions
      • IP and R&D Intelligence
      • IP and R&D Search
      • IP Prosecution Services
      • (Eco)Toxicology Consulting Service
      • Professional Services
      • Rapid Research Support
      • Knowledge Management & Transformation
      • Market and Strategic Intelligence
      • Design and Production Support
      • Deals, M&A, and Corporate Finance Support
      • Scaling Data Assets
      • Financial Services
      • Risk & Compliance
      • Investment Banking Support
      • Corporate Banking Support
      • Commercial Banking Support
      • Global Markets Research
      • Index & Quant Solutions
      • Investment Management
      • Life Sciences & Healthcare
      • Biopharmaceuticals, Medical Products & Healthcare
      • Datacrest
  • Service Design Consulting
  • Digital Platforms
      • Insightsfirst
      • Researchstream
      • Spreadsmart
      • Insightloupe
      • Usecasehub
      • Datacrest
      • Pitchready
      • Researchscaler
      • Navigator
      • Publishwise
      • AnalyticsApps
      • Compsbuilder
  • Industries
      • Financial Services
      • Corporate & Investment Banking
      • Capital Markets
      • Investment Management
      • Retail & Commercial Banking
      • Risk & Compliance
      • Manufacturing
      • Automotive
      • Consumer Packaged Goods
      • Engineering
      • Energy
      • Chemicals
      • Life Sciences & Healthcare
      • Bio-Pharmaceuticals
      • Medical Products
      • Healthcare
      • Technology Media & Telecommunications
      • Telecommunications & Media
      • Hardware & Software
      • The Internet of Things
      • Professional Services
      • Consultancy
      • Legal
      • Taxation, Auditing & Accountancy
      • Services
      • Retail
      • Freight & Logistics
      • Utilities
      • Non-Profit Organizations
  • Resources
      • Case Studies
      • Blog
      • White Papers
      • Evalueserve in the News
      • Industry Insights
      • COVID-19 Disruption Insights
      • Events
  • Home > 
  • Evalueserve Blog > 
  • IPR&D Information Adventurers Blog > 
  • The Evalueserve Search Quality Index (SQI) – a tool to control, document and improve Patent Search Quality
IPR&D Information Adventurers Blog

The Evalueserve Search Quality Index (SQI) – a tool to control, document and improve Patent Search Quality

Jul 03, 2018 • Urs Dommann & Ashutosh Pande

In economics, market price is usually a good indicator of what value suppliers and buyers allocate to specific goods or services, other things being equal. If you prefer the taste of an exotic pear, you pay more. When a glut of apples is available, you pay less. The same laws of quality begetting more cost, supply and demand, commodity versus rarity, apply to IP or R&D searches (however, in a slightly different way). Let’s consider the typical pricing of patent search, patent landscapes, innovation scouting, and see what we can refer from this to our discussion about quality. In the following four simple cases the ‘searcher’ is a patent counsel/patent attorney.

Type of search  Market price Additional search costs (excluding attorney analysis work following the search)
Quick novelty search on the internet  Free Plus labor cost of patent counsel to screen and shortlist relevant results (e.g. 300 USD/hour)
Quick novelty search on paid patent database  50 USD Plus labor cost of patent counsel to screen and shortlist relevant results (e.g. 300 USD/hour)
Novelty search done by a service provider or in-house search professional  300 – 1,500 USD  0
 Validity Search done by a service provider or in-house search professional – used for a post-grant opposition  2,500 –50,000 USD  0
Freedom to Operate study  2,500 –50,000 USD  0

 

There are many other types of search or intelligence that we can discuss here, such as innovation scouting or competitive intelligence, but for the benefit of a simple analysis these cases are sufficient.

Value correlates with recall and precision

We believe that the value of IP search services – the market price – of each of the scenarios above is directly linked to customer expectations of recall and precision (concepts we have introduced in our last post).

  • For use cases where low recall is sufficient, market forces have determined a price of – let’s say – 500-1,500 USD.
  • For use cases where high recall is required, market forces have determined a far higher price. This reflects the costs/resources you must put into ensuring higher recall – something that we have discussed before, particularly in our blog on parallel search.
  • For use cases where low precision is acceptable, the price tends toward zero. For confirmation of the fact that this is becoming a virtually-free commodity, look at the development of patent analytics tools market where we see clear signals of a buyers’ market and automated solutions.

Your time and resources – like apples and pears – are perishable!

A second economic concept is especially relevant to this discussion – opportunity cost (or the cost of the lost opportunity to do something more valuable with your perishable time). The more time the end user needs to spend on the search, the higher is the opportunity cost. From an economic perspective, it doesn’t make sense for patent attorneys to do their own (lower value) prior art searches, as their hourly rates are so high. It is comparable to taking a train to the countryside, and picking your own apples!

What about quality of the resulting insights?

This leads to a third consideration of quality that is often neglected – we could call it quality of insights, or output; or the handover to the next step in the process. A good search will not only be defined by recall and precision, but also by the quality of insights and the means of handover to the end users. Ultimately, this links to the opportunity cost. In the restaurant search example in our previous post, you’d expect a search engine to sort the results in an ordered manner either based on ratings, or distance. This sorting creates insight quality. Similarly, in Evaluserve IPR&D search studies (novelty, invalidity or FTO) you would expect a sorted and annoted list, or a way to navigate the result set to arrive at the most useful result for your requirements. Or very simple a reliable recommendation. All these techniques can be construed as Insight Quality.

Search quality – can it be measured?

Adding this last ‘insight’ we now have the basis for an economic framework for further discussion about search quality.

It is clear that SQI (Search Quality Index) is directly proportional to R(Recall), P(Precision), and I(Insight quality). And it is obvious the Expected Search Quality is different for different use cases, so this will be relative and not absolute.

Below we present an initial, highly-simplified, formula for search quality index (SQI). We (Ashutosh and Urs and others) have debated if such a simplified formula is actually still correct, however we feel it is acceptable model for a first discussion, and that we plan to deepen over the next few months.

SQI = (RA/RE + PA/PE + IA/IE)/3;

with RA,E = Recall actual and expected, PA,E = Precision actual and expected and IA,E = Insights actual and expected.

We can see that SQI can still be 1 (or 100% quality) for cases where only low recall, low accuracy or low insights are expected. SQI will be < 1 when expected levels are not met (= search quality will be bad), and SQI will be > 1 if expected levels are exceeded (while search quality is very good; too many resources were put into the search, the search was too expensive).

What are your boundaries?

While we can clearly measure P (it is quite simple to define what the percentage of relevant documents is in the search output), it seems impossible or at least very difficult to measure R and I. However, R and I are probably the main drivers of search quality for many use cases and probably also the ‘most expensive’ factors, so we will spend quite a bit of time in upcoming posts to explore the factors that influence R and I. And economics will again be helpful to us, this time to measure R!

Going back to the restaurant example and applying the formula, we realize that usually we would be happy with just a very good restaurant in the city, so expected recall could be low while you want high accuracy and quality of insights. So probably in this case, it is better to take one of two approaches: Do a quick search on the internet and take a call or ask a local.

In this spirit, next time you visit either New Delhi or Zurich send us an e-mail and we’ll send you our top-3 recommendations for authentic restaurants. Both Ashutosh and Urs can help you have a great evening out in our cities!

TALK TO AN EXPERT
  • Urs Dommann

    Urs Dommann

    Business Unit Head,
    IP and R&D Solutions

  • Urs Dommann, who heads up IP and R&D Solutions at Evalueserve, is passionate about improving knowledge processes and data science to optimize quality in patent search, and is a regular speaker on this topic at IP Service World. Urs and his team created the Information Adventures blog after recognizing that moving through the complex world of data and information in IP or R&D is like venturing high into the mountains or other seemingly uncharted territory – something he loves to do in the Swiss Alps near Zurich where he lives. He’s enjoying using the blog to share theories, insights about IP and R&D, and starting a conversation as a basis of exploring – together with clients – new paths to success.

  • Ashutosh Pande

    Ashutosh Pande

    Vice President,
    IP and R&D Solutions

  • Ashutosh Pande is an expert in intellectual property, mathematics, data analytics, and data science. He’s the Vice President at Evalueserve’s IP and R&D Solutions responsible for doing and managing research around IP and R&D research services, aimed at improving their outputs for customers. He’s realized that to find the simplicity in anything, you just need to continually break down the complex ‘big picture’ into ever-simpler component parts, and search out the commonalities. Via the Information Adventurers blog – and in collaboration it’s with readers – Ashutosh looks forward to honing his own thought processes about challenges in IP and R&D research, and creating knowledge that will impact the industry.

Don’t hesitate, get in touch with Urs & Ashutosh.

Latest posts

  • Blog Post

    5 Emerging Trends in Competitive Intelligence

    Discover More
  • Blog Post

    Harnessing the Impending Mining Super-cycle

    Discover More

Follow us on

  • Service Design Consulting

    • Service Design Consulting Overview
  • Knowledge Services

    • Knowledge Services Overview
    • Marketing, Sales, & Strategy
    • Innovation Intelligence & IP Solutions
    • Professional Services
    • Financial Services
    • Life Sciences & Healthcare
  • Analytics

    • Analytics Overview
    • Marketing & Digital Analytics
    • Sales Analytics
    • Supply Chain & Operations Analytics
    • Pricing & Revenue Management
    • Customer Insights
    • IP & Innovation Analytics
    • Risk & Compliance Analytics
  • Digital Platforms

    • Digital Platforms Overview
    • Datacrest
    • Pitchready
    • Researchstream
    • Spreadsmart
    • Usecasehub
    • Insightloupe
    • Insightsfirst
    • Researchscaler
    • Navigator
    • Publishwise
    • AnalyticsApps
    • Compsbuilder
  • Industries

    • Financial Services
    • Manufacturing
    • Life Sciences & Healthcare
    • Technology Media & Telecommunications
    • Professional Services
    • Services
  • Resources

    • Blog
    • Case Studies
    • White Papers
    • Industry Insights
    • Events
    • Competitive Intelligence
    • Financial Spreading
    • Demand Forecasting
  • About Evalueserve

    • About Us
    • Evalueserve Team
    • mind+machine™
    • Stories That Connect Us
  • Careers at Evalueserve

    • Careers
  • Press Center

    • News
  • Contact Evalueserve

    • Talk to one of our experts
    • Office locations
© 2021 Evalueserve. All rights reserved.
  • Imprint
  • Privacy Policy
  • Cookie Policy
  • Terms of Use
  • Corporate Governance
  • Site Map
  • Email Marketing Guidelines