What do customers want? Firms spend millions of dollars on market research trying to answer that question. Market-research experiments traditionally have employed conjoint analysis, a method that relies on statistics and psychology, to estimate the importance of specific product features and ask such questions as which would a customer value more, having a wider TV screen or saving $500?
The problem with this approach is that firms almost always need to know how customers value different combinations of features, rather than simply understanding the absolute value derived from each feature. “A company wants to know whether a customer values HDTV [high-definition television] more than a large screen,” Professor Olivier Toubia explains. “Would a customer pay more for HDTV even if the size of the screen isn’t bigger than their old TV? It’s not the features per se they are interested in, it’s the relative value of each.”
Toubia and John Hauser of the Massachusetts Institute of Technology wanted to design experiments that produced better results, which could help market researchers and their clients address the limitations of standard conjoint analysis. “Some decisions and combinations are more critical than others. If you just apply standard market-research methods, you may get precise estimates of preferences for features, but estimates for the combinations of those features will not be as precise,” Toubia says.
The researchers created a theoretical model and ran simulations showing that experiments designed to look at product features in different combinations would produce more precise estimates of consumer’s preferences than those looking at each individual feature as its own distinct parameter. The method included additional criteria for measuring managerial efficiency, taking into account that certain managerial decisions about how to market a product are weighted differently. For example, it may be more important to a marketing department to identify a tactic that can be executed quickly than one that is cheap to implement.
Modest improvements in the efficiency of experiments can yield dramatic reductions in the cost of conducting market research. “In the pharmaceutical industry, you sometimes have to screen 50 people to get one respondent, and the cost is high — about $450 per person. If you cut the number of people you need to survey in order to get reasonable estimates, you save a lot of money,” Toubia says. Toubia and Hauser’s results suggest that efficiency increases of up to 30 percent are possible.
Because their method considers the different decisions marketing managers will need to make, Toubia and Hauser’s work might also help market-research firms to tailor experiments more precisely to the goals of their clients and to consider how the results of each experiment will be used, a practice that is not the norm in the field.
“Experiments are typically designed without consideration of the end goal. But the end goal should be taken into account in the design phase. Are some criteria more important than others? Are some combinations more important than others? The more critical features should receive more extensive attention than others,” Toubia stresses. “How you intend to use the results will inform how you are going to design the experiment.”
The researchers hope that their work will change the way market research is conducted. Until their method is adopted broadly, Toubia says that any firm hiring a market-research consultant should make sure the consultant will tailor the service to the firm’s needs. “Market-research consultants should ask how a firm will use the results of a study before they design it,” Toubia says, “as opposed to using some off-the-shelf approach that doesn’t take into account the context behind why the firm is undertaking the research.”
Olivier Toubia is the David W. Zalaznick Associate Professor of Business in the Marketing Division at Columbia Business School.
Read the Research
"On Managerially Efficient Experimental Designs"