Can research be done only using traditional means? I do not think so, and I have been brainstorming how can we use search data to understand consumers’ intent.
What people say and think can be uncovered by interviews, which requires them to have explicit knowledge of their attitudinal behaviour. We can go a bit deeper by observing what they do and use and project their behaviour accordingly.
What’s difficult is to know how they know, feel and dream about the context. There are various techniques to do so, one of which requires using pictures and words to project their feelings and needs. We may be successful to uncover their tacit knowledge (knowledge that can act upon, but cannot readily express in words), but quite unlikely to discover their latent needs. Latent needs are needs that people are not aware yet. Traditional research may have its limitation and time taken to determine such needs.
Even when traditional research academics fought to say that it is possible, there is still a gap between 1) attitudes towards that behaviour, 2) intended behaviours and 3) real actions.
Let’s look at some readily available data such as search data.
Search (Google, Bing, etc.) has been the cornerstone of how a consumer stays in front of the computer/mobile to look for something they need. According to research from Google, there are only three primary purposes for search. 1) Search for Information, 2) Search for Navigation (to other sites), 3) Search for Transaction (buy or sell). So, if that constitutes the universe of search, analysing what words people use and type into the search box could reveal much into their expected behaviour and back forecast to their attitudinal behaviours. It is a preliminary thought from me, which like all research needs validation and fine-tuning.
Let me give a concrete example to illustrate my thinking.
In 1980-2000s, the age old research that most textbook use is for a supermarket to place diapers and beer near to each other or near the counter before checkout. Research has shown that Dad is the one who bought the diapers for the children and by putting beer in the line of sight, there is a higher probability of him buying beer. That is the crux of cross-selling and upsell. To identify other products for cross-selling or up-sell, a company will need to hire an agency to build the complex statistical model or Artificial Model, do all the testing and finally decide the possible cross-sell candidates.
Even though it sounds logical, I see two shortfalls.
1) Data is incomplete – IT only paints what has already happened. We use what has happened to project what will occur with significant confidence, but it today’s world, what has already happened may not happen again. Even when we take into account structural breaks, academically breaks are always found when it has happened. In layman terms, who will ever foresee the rise of Uber, Facebook, IOT and others and what’s TNBT (the next big thing)
2) Data is incomplete – IT only includes internal data (we can attempt to join some research or external data, but let’s face it is not easy nor practical). How do we adopt a greenfield approach to reach out to those who is not yet our customer? How do we know his / her intent? Social data provides some interests domain or demo data for targeting, but it still begs the question, what the TNBT and needs from our non-customers. Traditional research and management consultancy will size the market/needs for you, but it is based on sampling and may not tell you how to find these new customers.
Back to the beer example, if we can analyse the term “Where to buy beer?” on the web and trend it against similar search items, we could uncover new territories to explore instead of sticking to our paradigm. In statistics, we are wary of “spurious correlations”, which means two items that were not related could show high correlation numbers. However, in innovation and creativity, we are a fan of “Spuriousness” as that will give new territories for our brains to string two unrelated concepts together. Usually, that may result in unexpected opportunities & results.
As both a professional statistician and innovator, I am of the view that depending on the context. In this case, I will go for the latter, just explore seemingly unrelated results and that will give a more reliable projection in our global context now.
Results – as shown in the picture, pizzas and beer may be a territory to explore. You can even shift the charts by months or weeks to ascertain whether those who search where to eat pizzas may be a leading indicator for where to buy beer, and much more way fo looking at the data.
Cost – $0 as these are publicly available data.
Time Taken – 5 mins.
PS: This is just the beginning, imagine what we can achieve looking in brands, value propositions, and so much more. I am now compiling a set of new territories to explore for different categories and industries. Let me know if you would like to add on the list to explore.