The customer base shrank, revenues fell, and prospects told sales reps repeatedly that the product was losing relevance. Not true, said the executives. It was only a misperception, they claimed – industry-sponsored studies indicated product usage was steady, even increasing a little in some areas! But the studies were wrong. The decline was real.
Why such a huge disconnect between the survey results and market reality? Simple. For the most part, they surveyed consumers with landlines, publicly listed phone numbers, who were at home during weekdays, and had the time and willingness to take a long survey. This method has been used for decades, and could be reliably extrapolated over the general population. But the world has changed. The increasingly older consumers willing to answer the phone in the example above still comprise an important constituency – but they are not and should not be construed as the average consumer. This traditional survey bias was also a factor in the 2012 Presidential election polling. Skewed survey results masked problems with candidates’ outreach operations and produced nasty surprises on election night.
In the case above, older consumers were still using the product at the same rate. But younger consumers weren’t. Younger consumers are also less likely to have landlines, have publicly listed phone numbers, spend weekdays at home, and agree to take a long survey. That’s why the surveys (which were otherwise conducted with scientifically accepted methodology) let the executives keep believing the decline was temporary. They told their sales reps to defend the product by explaining to prospects that their perception about product relevance was wrong – they had the surveys to prove it. That did not turn out well.
When the real world indicators don’t look anything like the marketing research, defending the research is usually not a winning strategy. Conduct the research so it WILL unearth the worrying trends and inconvenient truths about your products. At least you can develop a roadmap to deal with them before your competitors do.