When a company undertakes a product test, there is a large upfront investment on their part to produce enough test product and get the product safely to their supplier. The return on this investment should be plentiful in accurate and actionable insights. Many research firms claim they conduct product testing, but buyer beware. If every step of the testing process is not handled with the utmost care, at best, poorly designed and executed product testing can provide minimal learning and insights and at worst, information leading to potentially disastrous results.
A well-executed and analyzed product test can provide a wealth of insights beyond the basic key evaluative and diagnostic measures and offer a greater understanding of what consumers think about the product, who is the best target, what motivates consumers to like or dislike it, and how the product could be improved.
Experience
Executing exceptional product testing requires experience and know-how. Careless handling of the product, poorly designed questionnaires, and limited analysis of data can bring narrow or misguided answers to key research questions. Look for a company that has a wide variety of experiences and solid examples of their work. Every product test has its own quirks and intricacies. Knowing how to handle any issues when they arise is critical to success.
Control of the Product
There are two ways suppliers can handle shipping product. Suppliers can use an outside fulfillment center, or they can receive, pack, and ship the product in-house using their own employees, trained in packing and shipping.
We believe that hand-packing each product in-house is the best approach. This ensures full control of the product guaranteeing that damaged product isn’t sent out, products are labeled correctly and safely packed to prevent damage during shipping, the right product goes to the right respondent, and proper instructions are included in each package.
This approach also enables proper follow-up on any product that is lost in transit providing tracking and replacement as needed. Questions are asked in each survey to ensure the correct product was received and was in perfect condition. When there are problems or complaints, the respondent can speak directly with our team
Questionnaire Design and Analysis
As with any study, a poorly designed questionnaire can derail the insights. When we design a questionnaire, we are planning for the types of analyses we will and potentially may want to do on the data. Below are a few techniques to consider.
Penalty-Reward Analysis
How much of a reward do you get by getting a ‘dimension’ of a product or concept right and how much of a penalty do you get when you get it wrong? The reward and the penalty are not necessarily equal. For example, if a new cookie product tastes good, satisfaction goes up, and if it tastes bad, satisfaction goes down. Maybe interesting shaped cookies, like dogs or Santa Claus, increase purchase intent, but not having the shapes, doesn’t necessarily decrease purchase intent. Penalty/Reward analysis allows us to quantify negative and positive impacts associated with product features on overall liking, satisfaction, or purchase intent.
Driver Pathways
We use an approach called Driver Pathways which helps find the patterns and connections in the data. Using a combination of questions from the survey we can help understand what drives repeat purchase intent.
Including open-ended questions in this analysis allows us to find areas we may have missed in the questionnaire that may be very important for repeat purchasing. A company that has been in a category for years and years may think they know the right questions to ask. Sometimes being so close to the category means not seeing it from the outside perspective. We sometimes see an entire branch of the Driver Pathways network that leads directly into purchase intent comprised of all open-ended questions. This says that some important dimensions have been missed and finding those missed drivers can ensure they are evaluated in future testing.
Text ScienceTM
Our firm incorporates a self-probing method of administering open-ends combined with our proprietary Text ScienceTM analysis. This delivers more text-per-respondent and greater insights into how consumers feel about a new product and why they feel that way.
Our method uses natural language understanding to transform text data into a visual map of the words and phrases used by respondents, revealing not only their relative importance but also how strongly they’re associated with each other. In product tests, this results in descriptions of sensory dimensions and product benefits that are expressed in consumers’ own language, honest and unfiltered—allowing you to message your customers using the category terminology they prefer.
Sensation
Our firm also creates a simulator based on sensory and packaging dimensions to enable clients to understand the impact on repeat purchasing or overall liking when various dimensions are changed. Changing one dimension can impact another. With our simulator, we are able to take these interactions into account.
If you just ask about the amount of candy pieces in a cookie, for instance, most consumers will say they want more candy pieces. But what if the impact of the candy pieces makes the texture of the cookie harder and consumers don’t like a harder texture? The simulator can vary both dimensions to reveal the impact on purchase intent. This helps test potential product modifications without making and testing all the formulations.
While the back end of the simulator is very sophisticated, the simulators our clients receive are user-friendly. Our clients can adjust different sensory ratings to see how each affects purchase intent.
Our volumetric forecasting with the Sensation Simulator can be used to see how much more sales would be obtained with various formulation changes. Then the client can decide if the investment to change the product is worth it.
Making the Right Choice
How are you deciding on who is going to perform your product testing? MAi Research has spent the last four decades ensuring our product testing is accurate and predictive. Let’s make your product successful by doing product testing RIGHT! Come see how you can make it happen!
