In a recent post on Innovation Intelligence, we discussed to keep up with today’s increasing pace of innovation. Delivering intelligence has now gone beyond a simplified process of gathering data and mapping information. Organizations still need intelligence covering one-to-one solutions that achieve specific goals. But perhaps more importantly, they must also answer complex sets of questions with outcomes that dynamically change depending on the business context and strategy across R&D, product development, competitor intelligence, or innovation management functions in an organization.
Here, we will dig deeper into various innovation intelligence use cases and explore aspects that allow us to develop a framework for measuring use case quality.
Understanding the components of innovation intelligence
You’ll remember Evalueserve’s search quality index (SQI), where we use R, P and I in an empirical formula to understand the quality of a search:
SQI = ((Ra/Re + Pa/Pe) + Ia/Ie)/3
However, when we want to determine the quality of intelligence projects we must look at the three components of recall (R), precision (P) and insights (I) in a different light.
In our post on the patent search process, we discussed how a project’s objectives can translate into different search queries and how recall and precision govern the optimal search strategy for different use cases. When discussing innovation, intelligence, recall and precision are concepts that govern the knowledge object for each search. Insights are derived from these knowledge objects. Patents, technical literature, product details, internet news, etc. are just a few examples of knowledge objects for which R and P matter. In an intelligence study, although recall and precision are important, the quality of intelligence depends on the insights derived from analyzing the knowledge objects. Accordingly, Ra and Pa in the above formula must be reasonably high for most of the of innovation intelligence use cases – otherwise incorrect data can lead to wrong conclusions.
To discuss intelligence quality, we look at a couple of use cases below and consider what else is needed to deliver great intelligence for these use cases.
Quality of insights
Just as we broke down the knowledge objects into logical variables of recall and precision, we can now start to look at insight and its sub-elements that influence the overall quality and utility of the intelligence.
1. Improving insight quality with high Solution recall to capture all possible problem solutions
When we discuss innovation intelligence, the intelligence we seek is either to find a new product category, technology that solves an existing problem, research program, etc. The comprehensiveness of these answers determines the delivered quality of intelligence.
Note that this recall of concepts/technologies is different from the document recall discussed in earlier blogs on search recall.
|
Objective |
Establish what is needed for delivering Intelligence. |
|
Innovation Scouting |
High recall for all concepts/technologies that answer the problem statement. Very high recall of knowledge objects (patents/ literature/ products) is not necessary. |
|
High recall for all competitors’ R&D programs and anticipation of future threats. Very high recall of knowledge objects is necessary. |
We can describe this type of recall as solution recall – a solution is the answer that we are looking for as the output of an intelligence project; and recall is comprehensively capturing all information around the solution to make the right inferences, and accordingly decide the next steps.
2. Meta information that goes beyond the obvious data points
In another post, we discussed that tagging can improve/facilitate retrieval and lead to high recall of data as the basis for better intelligence. We know as a general principle that the taxonomy and the depth of tagging influences the quality of insights. However, the more information we add beyond traditional knowledge objects enhances our ability to derive more meaningful insights. Let’s look at the examples below to understand this concept better.
For example, in a scouting exercise we should also evaluate additional dimensions such as if the solution is market ready, easy to implement, cost effective, etc., to help in benchmarking the concepts. Similarly, in a competitive intelligence use case: translating tagging of knowledge objects into competitor research programs, key technology platforms and technologies on which the competition is working; followed by benchmarking the threat from such activities; together will lead to a more meaningful intelligence.
We can consider these additional dimensions to be meta information. The more meta information we can generate the higher is our ability to deliver better, richer intelligence.
3. Meeting the technology landscape needs of the end customer
Innovation intelligence is an extremely broad term, it can mean generating leads for ideation, understanding competitor moves, scouting for new technologies, to name a few. The output of such studies can be used for– war gaming, scenario analysis, feeding the innovation funnel, decision making, etc. Another factor is that study output is consumed by different stakeholders at different levels within the organization.
The results of the study therefore need to be customized to suit the end use and the end user.
The formula for understanding the quality of intelligence delivered could therefore be
IQI = (Ra/Re + Pa/Pe) + (SRa/SRe + MIa/MIe + DFa/DFe)
Where SR = Solution recall, MI = Meta Information and DF = Deliverable Framework
In upcoming blogs in this series, we will use the above formula and deep dive into few live examples of intelligence to test the above hypothesis and explore the possibilities. Stay tuned!
