Times are changing fast. We conducted market surveys in every key markets in Asia, in North America, and in Europe. As CEO of a market research agency, I worked with some of the largest luxury and wine brands in the world. With my peers we’ve all kept talking about it, but when is it going to happen.
Consumer trends are changing faster than ever, Consumer packaged goods (CPG) companies are losing sales to small, agile competitors. These small competitors are fast and close to the ground: they can spot trends before the big brands and bring new products to the market faster than anyone. In fact, many of these companies are dedicated to just following these trends.
Unfortunately, a large part of the problem for them is their market research. CPGs lost an estimated $2.3 billion on failed innovation in 2019 alone. Traditional research methods can’t give these companies what they need to respond in today’s dynamic marketplace. Companies either bet on the wrong trend or cannot act fast enough to take advantage of what they know.
Traditional market research is just not getting the job done in today’s world
Market research is experiencing a paradigm change. Born into the post-war economy, market research was an appeal to science to bring rigor to understanding what customers wanted and what they would pay for.
Marketers continued use of methods such as questionnaires, interviews and focus groups for the last 80 years pays testament to traditional research’s usefulness. However, instead of helping them win, the same methods that worked for large companies, even five years ago, are now starting to cost these companies sales and market share.
The 6 big problems of traditional market research:
- Overstatement bias (the ‘say/do’ gap)
- Limited number of questions you can ask
- Respondent engagement/quality
- Slow
- Expensive
- Ability to look forward and predict
Market research has always struggled with these problems, but until recently, companies had no other options to get the answers they needed.
How digitization and technology is transforming market research
The market research industry has not ignored the advent of the digital age. ESOMAR’s latest Global Research Report shows that spend on digital market research surpassed traditional methods back in 2018.
Simply put, digital market research companies can be divided into two groups:
- Tech-enabled market research (e.g. online survey tools)
- Tech-driven market research (e.g. Big Data and AI)
What is tech-enabled market research exactly? This is research performed by firms that apply more cost efficient, digital methods to gather the same data as before that used to require an army of researchers.
Tech-enabled market research allows companies to speed up their traditional market research and to do it at a much lower cost (two of the six problems). However, tech-enabled market research doesn’t crack the biggest research problems in today’s fast-moving world.
It doesn’t solve the claimed vs actual behavior issue (i.e. the say/do gap). It still relies on prompted questions to an unmotivated sample of respondents which means it will rarely uncover something you didn’t already know about (the unknowns). And it won’t allow you to predict future behavior with any degree of statistical accuracy.
How tech-driven market research can deliver success
Instead of tech-enabled firms that rely on old methods, let’s talk about tech-driven market research that’s new and creating something transformative.
With the advent of the internet, smart phones, and advancement in data collection, there is more data available on customers, their habits, and their preferences than ever before. Tech-driven market research utilizes and unlocks the value of these behavioral data sources.
For example, UK market research company Streetbees is a tech-driven market research company that captures verbatim from 3.5M ‘bees’ across the globe who record product choices and consumption behavior on their mobile phones. Streetbees’ AI and Natural Language Processing technology analyses and structures this vast collection of text and images into powerful consumer intelligence. Brands use it to gain a deeper understanding of natural, in-the-moment behavior to identify how they can better serve their customers’ needs.
Another example is analyzing social media conversation trends to anticipate changes in consumer behavior and attitude. Every day, millions of us share our ideas, needs, wants, frustrations, and desires on social media platforms. For almost any consumer related topic that you can think of, somebody somewhere will be discussing and posting about it.
By analyzing this vast and rich data source we can identify known and unknown consumer topics and understand how and why they are trending. Furthermore, by training data science models on historical social media data, we can scientifically predict which of today’s emerging trends will sustain growth and most impact consumer consumption and purchase behavior in the future. This is incredibly useful information if you’re trying to create new product innovation ideas.
What people say now is only important if it can tell you what they will buy next
Tech-driven market research’s combination of new data sources and data science techniques, mean that for the first time companies can overcome the original critical flaws in market research. And critically, progress from looking only at past behavior in the rear-view mirror, to looking forward with statistical confidence about what’s on the horizon.
For example, PepsiCo uses an AI-driven tool called Tastewise to analyze millions of menu data points and understand what food and cuisines people are eating. They combine this with Black Swan’s trend prediction technology: Trendscope, to prioritize which emerging Beverage and Snacking trends are predicted to tip into the mainstream. Ideas created through this process are then rapidly tested and optimized through Zappi’s advanced new product concept testing platform.
Used alone, each of these tools offers a powerful, tech-driven solution that creates better answers to familiar questions. But integrated together, they combine to create an advantaged capability that sets PepsiCo apart from competitors in how they conceive new ideas and bring new products to market.
Looking at the CPG industry, BCG and Google saw that companies could improve their earnings by over 10% using AI and advanced analytics.
Why do you need tech-driven market research?
Tech-driven market research already seems like an extreme paradigm shift, but this is just the first drop in the bucket. The application of Big Data and AI powered tools is not only going to mature, it is going to accelerate and spin-off into multiple use-cases and applications.
Can tech-driven research firms develop AI sophisticated enough to offer accurate, long term predictions of market size that can automate your entire new product innovation process? Can they offer a successful ‘Supply Chain as a Service’ outsourcing platform with accuracy in long term supply chain, and output with zero waste?
Blockchain doesn’t solve a problem that the insight industries have — It will disrupt us completely. Precisely because it’s not about us, but the people who we collect data from.
3 ways Blockchain will revolutionize market research
The use of blockchain technology outside cryptocurrency is expected to rise by over 60% according to a report published by Reuters. Potential applications range from identity management to online shopping.
Blockchain will revolutionize the way the insight industries work. Not because we need to change the way they work or because it will create an advantage for us or even make our jobs easier… Blockchain doesn’t solve a problem that we have. Blockchain will completely disrupt our industries because it changes the way data is owned, stored and shared.
It will disrupt us so completely, precisely because it’s not about us, but it’s about the people who we collect the data from. It’s a technology created for them to help them solve their problem. The most disruptive technologies are those that create a revolution from the bottom-up.
Every entity from commercial to non-profit will feel the impact of a mass Blockchain adoption but as data is at the literal route of the business of the insight, our industries will be affected even more so. Here are just a few considerations…
Big Data — Whilst the big data revolution has only just begun, Blockchain WILL disrupt it. At the heart of big data’s success is the explosion of readily available passive behavioral data; precisely the thing that Blockchain is set to manage.
Surveys — If surveys do survive they will have to adapt. Firstly, they will have to connect to the Blockchain data available — participants will not have the patience to answer questions about their devices or their use of them when they can give you access to their Blockchain records.
Questionnaire — Blockchain with cryptocurrency alignment will incentivize all participants to contribute in a meaningful and honest way, as dishonesty will affect their Blockchain reputation.
It’s Anyone’s Game — The large data warehouses will no longer have control, and neither will the client in terms of customer data. Every insight agency, regardless of size, will have equal access to exactly the same personal and behavioral data via the Blockchain.
A blockchain is a digital distributed ledger, maintained by a network of multiple computing machines. It stores data in the form of blocks that are cryptographically secured and immutable. Unique features of blockchain: Decentralized, Trustless, Transparent, Immutability.
1. Improved data authenticity
Analytics and data are crucial to their day-to-day operations. Of course, the whole thing only works if the data is authentic and accurate in the first place. Otherwise, no real value could be generated.
With blockchain, all the pieces of data that go in are not only accurate, but also virtually 100% tamper-resistant. And, if anyone is somehow able to alter anything, not only will the entire chain be affected, but all the relevant parties on the network will be alerted as well. Considering how the technology works, however, even finding the specific data to modify would be practically impossible to do unless you are authorized.
2. Better access to related data
In the same way search engines use structured data to better make sense of websites, market researchers would also be able to give more meaning to specific pieces of data if they came with more related data as well. Blockchain technology can enable this data linking together so that individuals and organizations no longer have to reach out to several different sources to compile and analyze the data.
3. Ability to bypass ad platforms by using Smart Contracts
A smart contract is simply a blockchain-powered collection of rules that automatically get enforced once specific conditions are met. It is typically used for financial transactions like insurance claims processing and online commerce.
Interestingly, it also has a huge potential to revolutionize the way the world does marketing because it allows marketers, for instance, to buy data directly from customers, instead of accessing this data through an ad platform (like Google’s or Facebook’s), and automatically compensate them based on the quality and amount that they provide.
This not only gives consumers power over what they share with marketers — which promotes trust — but also provides them with an actual incentive to do so. Needless to say, this ultimately ensures that marketers only get quality leads for every penny they spend. Additionally, by giving power over their data access to the consumers themselves, personal privacy is respected.
The blockchain tech may be promising, but it’s still too young. If blockchain is so good, then why hasn’t it taken over the business world yet? Well, the technology is still in its infancy. Moreover, blockchain adoption requires many users actively using it in order to be effective. Setting up the relevant frameworks to enable such a novel technology will take time, but the rewards, to both businesses and consumers, will be immense.
Source: Forbes
