Solving for shortcomings in AI and the Jobs-to-be-Done framework. 

The future of needs or jobs analysis is AI.

But not just any kind of AI.  

We’re presented with a visual graph of pain points around shampoo. The X axis represents market size, while the Y axis represents the price consumers are willing to pay. 

First, we notice that the analysis comes from over 1700 threads of conversations across product reviews posted around 441 products. This gives us a sense of the sheer amount of data MotivBase's AI Anthropologist is able to crunch through to identify and cluster pain points to enable 'jobs' discovery. 

Next, we click on scalp (one of the pain points in the top right quadrant of the pain point map) to understand this need further. 

We notice that it offers an average price potential that is 59% over the average price in the category, and a covers almost half of the baby shampoo market size. This tells us that this pain point represents significant value in the category, offering both the ability to attain volume, and deliver a higher priced (higher margined) product.

We take a look at the snippets of verbatim (under Phrases) to confirm what this need space really refers to in the eyes of the consumer. We determine that there are two major jobs around scalp. The first one refers to solving the problem of dry scalp for children. The second refers to adults using baby shampoo as a way to reduce their own issues with dry or sensitive scalp. We might here discover something a bit unexpected in terms of the use of baby shampoo for adults and choose to explore it further. 

By exploring the contextually relevant topics around this pain point, we can further discover details about the job that baby shampoo performs for adults around their scalp and begin to write a job specification in the following manner: 

“People are looking for shampoos that contain gentler ingredients that can help them deal with itchy scalp, dandruff, and general scalp dryness. They're trying baby shampoo as a possible, even unexpected solution although the most popular baby shampoo brand seems to have mixed results in solving this job for adults.”

As you can see in this example, we have entered a new era in how we think about leveraging the jobs framework. No longer are we limited in the size of our data pool, nor are we missing the nuance or context that takes us, as innovators, beyond what people are saying to understand why it is so important to them. 


But it also allows us to: 


  1. Quantify needs, and prioritize the right technical and brand-level investments.

  2. Expedite our innovation lifecycle by identifying jobs-to-be-done in minutes rather than months, and assign a numeric value to these ‘jobs’.

  3. Better understand unspoken motivations of the consumers driving a particular need state or a group of needs.

  4. Segment the marketplace based on needs rather than attitudes or demographics. That is, our algorithm instantly delivers a needs-based segmentation for any product or category or topic.


While we are in the early days of inserting this solution into a number of Fortune 1000 companies, we are already seeing how this capability is transforming how they focus their innovation efforts. Because when it comes to opportunity in innovation, size matters. This marriage between the social sciences, and data is presenting the clearest picture of the potential that exists in the marketplace. We can now go beyond what a consumer “needs” to get to the consumer’s most dominant Jobs-to-be-done.


When Harvard Business School Professor and renowned business thinker Dr. Clayton Christensen’s released his Disruptive Innovation Theory he suggested that companies make themselves susceptible to disruption when they pursue ‘sustaining innovations’ (the products and services that worked for them in the past).


Christensen believed that by following the path most consumers take and rationally doing what’s always worked, corporations allow smaller, more agile companies to create products that open up the market to a new and more valuable consumer. 
One critical solution to this problem, was Dr. Christensen’s Jobs-to-be-Done (JTBD) framework (JTBD). 

The JTBD framework served as an extremely useful toolset to accurately pinpoint the underlying problem that a product needed to solve. It also transformed how innovation professionals looked at understanding the consumer perspective by identifying the problem, pain-point or unmet need first, before assuming the role a product plays in the hearts and minds of the consumer. 

To simplify, people buy products and services to get a “job” done. That is, when a consumer buys a drill bit, the JTBD framework teaches us that the consumer is really buying the hole and possibly the accuracy with which they can create that hole. 
The power of the JTBD framework has been largely embraced by innovation and insight teams across the Fortune 1000. If you need to understand the consumer’s unmet needs, the rule of thumb is that you need to stop examining the product first, and instead look to uncover the fundamental ‘job’ that your key consumer is trying to solve for. 

By making the job, rather than the product or the customer, the unit of analysis, the industry has made it possible for companies to achieve predictable growth. 
Unfortunately, as companies look to further move from a “product-first” approach to an “unmet needs-first” approach, they continue to make one critical error. 


They look to traditional market research methods to validate the customer’s needs. And they ignore new tools that can more accurately identify, but more importantly quantify the size of the opportunity and how much consumers will pay to hire a product that will solve the job. 

Here is the issue.

Often, Consumer Insight and Innovation teams will leverage social science-based methodologies like ethnographic research to be the foundation for jobs work. But while immersive ethnographies can provide rich insight into consumers’ unmet needs, they pose a large challenge.


First, they are costly and time consuming. 

Second, they also often rely on a sample size that is relatively small, so it is difficult to know if you have analyzed a representative sample of the population. This makes it very difficult to run any kind of analysis to predict if the findings are growing or shrinking.

Lastly, they can be skewed by Availability Bias. An availability heuristic is a mental shortcut that relies on immediate examples that come to mind. When you are trying to make a decision or infer something, a number of related events or situations immediately spring to the forefront of your thoughts. And because they spring to you immediately, you assume they’re right. This causes availability bias and is a problem with most traditional ethnographic research (with small N samples), as well as research that relies on asking consumers why they do what they do. 


This is why many organizations are looking to Artificial Intelligence (AI) solutions to provide insight on larger samples of consumers. By leveraging machine learning the N Sample can be vastly increased, cost and timing can be reduced, and the bias can be reduced.

But of course, this isn’t as easy as it sounds. Technology by itself or AI by itself cannot solve the problem and give businesses accurate predictions of outcomes. This is because most traditional big data and AI tools leverage data sources and methods of data collection that are not nearly complex enough to capture signals that can indicate the job or “pain point” the consumer is trying to solve. 


Until now.

MotivBase features a technology that leverages Big Data from consumer reviews, and an AI anthropology engine to identify and quantify culture’s most dominant unmet needs.  

The power of this, is that it solves for all three of the shortcomings identified above. 

First, it leverages an AI Anthropologist that understands context, and is therefore able to uncover consumers' needs within a context (defined by the user of technology). As people inadvertently reveal their unspoken needs while reviewing products online, our engine can match and map the semantic tone to the underlying attitudes, values, motivations and fears driving them.

Or in other words, it doesn’t just identify the “job” or consumer pain point. It pushes us beyond what the consumer is saying, to understand the emotional and social job at play, that can be critical in knowing how to solve the problem for the consumer.

But secondly, it performs this analysis on thousands upon thousands of customers and millions upon millions of reviews. The N sample is immense. 

By creating a system that forces us to look for patterns amongst thousands not tens of consumers, our methodology forces MotivBase to eliminate the availability bias, and deliver a more accurate representation of the jobs that are most dominant in culture. 

Lastly, we leverage the marriage of social science theory and big data to identify two critical pieces of information that help us assign value to a job or a pain point: 


  • We run calculations to determine the relative size of the population that a job is relevant to. 

  • We identify the price threshold to know how much a consumer is willing to pay and what jobs they are willing to pay a premium to solve. 

Let’s use the following example to illustrate this. 

Let’s say we want to conduct a quick analysis on baby shampoo and look for the jobs parents are trying to solve in relation to shampoo for their infants and toddlers. 

We begin with a quick search for baby shampoo: 


Copyright © 2018 MotivIndex Inc. For our privacy policy, click here.