How To Contextualize Your Research Through Benchmarking

Among the many ways to leverage quantilope’s Insights Automation Platform, equipped with drag & drop pre-programmed question types and thirteen advanced methods, is using the platform to benchmark metrics in your category.

What is benchmarking?

Benchmarking, simply put, is a way of evaluating or checking a market research metric with comparison to an identified standard, so that brands don’t have to make decisions based purely on single study percentages. Benchmarking standards are often elements such as ‘industry bests’ or ‘best practices’ - such as a standard of product quality, the time it takes to accomplish a task, or commonly accepted costs. The process of benchmarking includes measuring a unique business metric and comparing it back to the identified benchmark metric.

Back to Table of Contents

Why is benchmarking important?

Benchmarking provides a way to contextualize results, helping brands understand how and where they need to make changes to a current product to improve that product’s performance against competitors in their industry. Additionally, any newly introduced products can also be compared back to benchmark metrics as a means of understanding their initial performance.

Without benchmarking, brands are less informed on how their product portfolio stacks up in the eyes of consumers. As such, brands could be wasting valuable time and dollars creating/promoting a product that is underperforming in comparison to the industry benchmark for that product or category.

Back to Table of Contents

What can I use benchmarking for?

Benchmarking can be used anytime a brand is looking to test a concept iteratively. Some of the most common benchmarking use cases include, but are not limited to:

Concept testing
Claims testing
Package testing
Territory testing
Position testing
Logos/designs testing
Advertising testing

Back to Table of Contents

Setting up a benchmarking study with quantilope

Setting up a benchmarking study with quantilope is simple - as are all studies on the platform. Benchmarking studies leverage two of quantilope’s features: an advanced, automated A/B Test and brand tracking capabilities. To set up your own benchmark study, follow the steps below:

Set up a new study on quantilope’s platform, and drag & drop an ‘A/B Monadic Test’ into it. Each cell of the A/B test should align with the products you are hoping to test.
Turn on the ‘tracking’ feature in your product and create your first wave of the project. This wave will be labeled as the ‘Benchmark’ wave.
Once you’ve set up your first wave and captured the desired completes (ensuring you’re capturing the appropriate target audience for your product), set up a second wave of the study and label this new wave as ‘New Concept(s)’.
With your ‘New Concept’ wave set up, add any new concepts you’d like to test as new cells in the A/B Test. Double check that the current wave is ‘New Concept’ and no longer the ‘Benchmark’ wave - then toggle off the Benchmark cells and toggle on the New Concept cells. Don’t change any of the questions in your ‘New Concept’ wave, as keeping the study consistent ensures you can make an accurate comparison back to the benchmark results.
Upon finalizing the set-up, set your ‘New Concept’ wave live and start collecting responses for these new concept cells. Your sample size and composition should be the same as your ‘Benchmark’ wave.

Back to Table of Contents

Analyzing benchmark data

Once you’ve collected all desired completes in your ‘New Concept’ wave, you can begin to analyze the data in comparison to your benchmark data.

You can opt to compare your new concept data against any single benchmark metric, or, across the average scores for all benchmark products. For the former, simply hide the cells you’re not interested in comparing against to focus in on the cells that you are. For the latter, set up a custom variable that splits your data into Benchmark respondents and New Concept respondents (those who were part of each wave, respectively). With this custom variable, you can dive into any metric from your study and ‘split’ the data by each of these groups to see how they compare and contrast.

It is recommended to re-test the benchmark once a year to ensure that the benchmark data is updated and relevant for the current market space.

Back to Table of Contents

Why should you benchmark your data?

In summary, benchmarking is an advantageous way to stay ahead of the competition and in line with industry expectations. Having a set of data points to reference back to provides more confidence and clarity in your brand's performance - whether that be related to a concept, packaging, messaging, or something else that you've benchmarked in the past. It also gives confidence to stakeholders that their team's budgets are being put to good use and generating the results they expect.

Check out an example of a benchmarking study with quantilope’s Cookie Benchmarking Dashboard to understand Oreo’s performance in the category.

To learn more on how to run a benchmarking study with quantilope, get in touch below.