Other inputs

Initial Intent

Script

Storyboard

Basics for successful start

FAQs

Storyboard Video

Brand Proposition

Product Concept

Packaging Concept

Radio

Video

Our benchmarks

What is Overall score?

Other metrics

Brand Asset

Ad Testing

Brand Identity

What is Need, and how do we measure it?

What is Emotion, and how do we measure it?

What is Ad Impact?

Sample & data collection

What is Brand, and how do we measure it?

Getting started

Types of campaign tests

Types of brand tests

Types of product tests

What are the key metrics behind successful ads?

Methodology

Brand Tracking

Account management

<h2>Ad Testing</h2><h3>Inputs</h3><ol><li>Can multiple campaign variants be tested within one test? Each video must be tested separately because each variant is evaluated using a different sample, which is then A/B tested against the others. This method ensures reliable data and makes it easy to compare campaigns. As a result, each variant functions as its own independent test. &nbsp;</li><li>Can I add one or two questions to the questionnaire? We do not recommend adding your own questions, as the questionnaire is automatically prepared for optimal results.&nbsp; &nbsp;</li><li>Can we include our old campaign for comparison? We can certainly re-evaluate your old campaign. However, we can only measure metrics related to branding, needs, and emotions, which we’ll analyze in detail to show exactly what worked and what didn’t. The challenge lies with the impact test—after a longer period, the ad’s impact can no longer be accurately captured. A campaign’s effect would likely be much lower, or even negligible, compared to its performance a week after it ended. This could lead to misleading results, which we aim to avoid. &nbsp;</li><li>When does it make sense to test concepts?<ul><li>You need help deciding which concept to choose.</li><li>You’re unsure if your idea will resonate with people.</li><li>You want specific insights to improve the concept.&nbsp;</li><li>You’re uncertain if people are understanding the intended message.</li><li>You want to ensure the concept won’t offend your target audience. &nbsp;</li></ul></li><li>Should we include an (illustrative) image with our concept? Your concept should include an image you plan to use in subsequent communication or one that effectively conveys the intended idea. It’s important for people to clearly understand the message the concept communicates. If the images are too unrealistic, there’s a risk that people will focus on evaluating the image itself rather than what the concept is offering. For more details on concept tests, see&nbsp;<a href="https://help.behaviolabs.com/en/article/initial-intent">here</a>. &nbsp;</li><li>Can you test a website? The Behavio platform is not designed for UX testing. However, we can evaluate first impressions and determine whether the website is overall understandable. This can be done using a video walkthrough of the website, with results presented as curves, or by analyzing the website’s visuals.</li></ol><h3>Methodology</h3><ol><li>Why don’t you just let people pick one of the concepts instead of doing an A/B test? Allowing people to choose between concepts leads to analytical thinking and less accurate results. A/B testing captures quick, intuitive responses that better reflect real consumer behavior. When faced with multiple options, people often make random choices, skewing the metrics and obscuring how elements like key messages or visuals perform. In everyday life, such as on the street, people typically see only one visual at a time. &nbsp;</li><li>Why is the statistical error 4% sometimes and 6% at other times when the sample size is the same? Even with the same sample size, the statistical error varies depending on the measured values. Generally, the statistical error is smaller at very low or very high values, and largest around 50%. For instance, with a sample of 500 respondents, the statistical error is up to 2% for values between 1% and 5%, while for values near 50%, the error exceeds 4%. &nbsp;</li><li>How do you predict the attention with a heatmap? For predicting attention—whether in videos or static visuals—we use the UNISAL model, developed by researchers at the University of Oxford. Built on Google’s MobileNetV2 visual model, it excels at analyzing both static formats and videos. The model was trained on datasets derived from eye-tracking data across 20,000 experiments, where participants viewed videos and images, all aimed at understanding what truly captures attention.</li></ol><h3>Results interpretation</h3><ol><li>How do you calculate the Overall score? When measuring the ad in the platform, we analyze three KPIs: branding, need, and emotion. These are compared against benchmarks from other tests, which vary depending on the ad’s development stage—from the initial idea to the final spot. The overall rating is the average of these indicators, reflecting how your ad or concept performs relative to others. For more details on the Overall score, see&nbsp;<a href="https://help.behaviolabs.com/en/article/what-is-overall-score">here</a>. &nbsp;</li><li>What does the curve height indicate, and how is it calculated?<ul><li>The height of the emotion curve indicates how many people responded positively to the ad at any given moment using a “smiling” emoticon.&nbsp;</li><li>The height of the brand curve indicates how many people spontaneously recognized the brand at any given moment.&nbsp;</li><li>The height of the need curve indicates how many people recalled the product category at any given moment. &nbsp;</li></ul></li><li>How do you handle typos in brand recall? Usually, when coding brand recall, we also account for typos. If the term is similar to the brand name, we include it in the brand recall, though each case is evaluated individually. However, if the client requires the brand name to be written perfectly, we do not count typos, and brand recall is considered only if there is a 100% match with the brand’s name.</li></ol><h3>Benchmarking</h3><ol><li>How do your benchmarks work? Our benchmarks compare dozens of campaigns measured over the past year, with separate benchmarks for each ad development stage. For example, we compare only storyboards or only final video campaigns.&nbsp;To ensure relevance, we use benchmarks no older than two years, allowing us to compare concepts with current campaigns that customers encounter (known as marketing noise). For more details on our benchmarks, see <a href="https://help.behaviolabs.com/en/article/our-benchmarks">here</a>. &nbsp;</li><li>How accurate is your ‘reach’ metric compared to post-buy data? Our reach metric is rigorous, modeling real-life conditions to assess the true impact of the campaign—whether a random 5-second segment of the ad sticks in viewers’ minds. In contrast, post-buy data from media agencies indicates that, on platforms like TV, the ad was shown to the target group, but they weren’t necessarily engaged or paying attention (e.g., they didn’t have to be sitting with a phone).</li></ol><h2>Brand Tracking</h2><h3>General</h3><ol><li>Is brand tracking usable for B2B? Our brand tracking works well for B2B if we can capture it in our panel. If your clients are entrepreneurs, we can do this easily. However, if they are senior executives, our classic brand tracking may not be the best solution. &nbsp;</li><li>Is brand tracking usable for employer branding? It depends on the target group. If you want to view your brand through the lens of all employees, that’s no problem. We can also target more specific groups, like IT specialists from certain regions in the US. However, IT specialists earning over $200,000 per year won’t be captured by the panel. &nbsp;</li><li>Can you measure a brand that is strong in specific regions but unknown in others? This depends on the brand’s overall penetration. For example, if a renowned restaurant is known by just five percent of the population, brand tracking makes sense as the restaurant aims to attract people from all over the state.</li></ol><h3>Data collection</h3><ol><li>Does Behavio work in markets other than the Czech Republic? Yes, we are primarily operating in Europe (excluding Russia) and the USA, but we can also handle measurements in Asia and the Middle East. So we’re ready to measure audiences almost anywhere! &nbsp;</li><li>What socio-demographic data do you provide in your report? Currently, we provide basic socio-demographic data on selected groups as part of brand tracking. If desired, you can integrate this data with the Czech Atlas tool (for Czech clients only) to access 900 additional characteristics about your clients and competitors. However, this is outside the scope of the standard packages (approximately CZK 10,000 per selected group). &nbsp;</li><li>Are you able to guarantee that my respondents will not repeat themselves, even if I want to measure 500 respondents per month? Yes, we guarantee it. The exact timing depends on the survey size and number of waves, but we follow the industry standards, which recommend a minimum six-month gap between surveys of the same respondents. We go a step further by re-surveying the same respondent only after at least one year.</li></ol><h3>Inputs</h3><ol><li>How do I determine the importance of each attribute/association (i.e. gluten-free, great taste) being tracked? Ideally, start with Market Tracking to determine the importance of each attribute/association, then incorporate it into brand tracking for long-term monitoring. For a deeper dive into your needs, we recommend our strategic research service.</li></ol><h3>Methodology</h3><ol><li>What is implicit association testing? What are its advantages? Implicit association testing leverages the brain’s natural associative process, making it a highly accurate behavioral method. Comparing two items side by side and observing reaction times is simpler and more precise than traditional scaling. This method focuses on responses within the 0.5-2.5 second interval, filtering out mechanical responders and those who rely on rational reasoning. &nbsp;</li><li>Why not use NPS? NPS is a surface-level metric that can complement research but shouldn’t be the primary KPI. It lacks context and doesn’t reliably predict future behavior or provide specific guidance on improving the customer experience. Extreme responses can skew results, and neutral ratings are excluded. Therefore, NPS should be supplemented with other research methods for a fuller understanding of the customer experience. &nbsp;</li><li>What is consideration? Why don't we measure it? At Behavio, we don’t typically measure brand consideration because it often doesn’t reflect actual purchasing behavior and isn’t applicable to all industries. People tend to select all brands they know and don’t have negative associations with, which doesn’t accurately represent real consideration or purchasing intent. &nbsp;</li><li>How do you identify barriers? Are they specific to each brand, or just general barriers to purchase? You can set specific barriers as associations. For example, if you want to measure how much your fast food brand is associated with junk food, you can track that through associations. Spontaneous barriers will also emerge in brand tracking.</li></ol>

This article covers:<ul><li><a href="https://help.behaviolabs.com/en/article/our-benchmarks#how-do-we-use-benchmarks?">How do we use Benchmarks?</a></li><li><a href="https://help.behaviolabs.com/en/article/our-benchmarks#what-do-the-colors-mean?">What do the colors mean?</a></li><li><a href="https://help.behaviolabs.com/en/article/our-benchmarks#what-do-the-numbers-in-circles-mean?">What do the numbers in circles mean?</a></li><li><a href="https://help.behaviolabs.com/en/article/our-benchmarks#where-do-the-benchmarks-come-from?">Where do the benchmarks come from?</a></li><li><a href="https://help.behaviolabs.com/en/article/our-benchmarks#benchmarks-are-important-but-tricky">Benchmarks are important but tricky</a></li></ul><h2>How do we use Benchmarks?</h2>Benchmarks – knowing if I’m better or worse than someone else – are much more exciting than the original numbers behind them.It sounds alright when 50 % of the sample react with positive emotion to your creative. But what if that’s a “usual” score for ads in general? Or what if most ads arouse much better emotion? Is 50 % good enough? That’s why the metrics in our reports always have additional information to them - how well each result matches up against other campaigns (benchmarks).We show:<ol><li>how good your result is compared to this benchmark (green, yellow or red color) and&nbsp;</li><li>what the average for the metric is.</li></ol><h2>What do the colors mean?</h2>To give you a clear sense of whether your score is high, average, or low, we use three colors with the following thresholds:Result is green,&nbsp;if it scored in the top third of all campaigns in this metric. In other words, the result is better than in most other campaigns - at least 67 % of all tested campaigns score worse.<blockquote>Example 1: Let’s have a look at the metric of “positive emotion” in Amazon’s “Holiday deals” campaign. 80 % people react with positive emotion to the ad. Green color of the barchart tells you that this score is well above average - that it is among the top third of all tested campaigns in this metric. Additionally, we can see what the average of positive emotion among all tested campaigns is: 50 %. <img src="https://cdn-assets.productfruits.com/d60770af-298d-42bb-ac37-6d515c95a48e"></blockquote>Result is yellow, if the score is around average of all tested campaigns. It belongs in the third of all campaigns, which are around the average score. In other words, the result is somewhere between 33rd and 67th percentile of all tested campaigns.<blockquote>Example 2: When we look at the metric of “category recall” in Amazon’s “Holiday deals”, we see a yellow barchart. Apparently, 52 % people recognize the category in the ad, and that’s just an average score when we compare it to all tested campaigns. To be exact, the average is 48 %.&nbsp;<figure class="image"><img src="https://cdn-assets.productfruits.com/b72141ea-4043-453e-a7ba-23bcfb785bcf"></figure></blockquote>Result is red,&nbsp;if it is among the lowest scoring campaigns in this metric. It falls into the remaining third of all campaigns. Or mathematically speaking, it os among the 0th to 33rd percentile.<blockquote>Example 3: 35 % of audience recognize the Amazon brand in “Holiday deals” commercial. The red-colored barchart tells us, that this score apparently falls into the lowest third of campaigns in brand recall.<figure class="image"><img src="https://cdn-assets.productfruits.com/b9fe6897-18db-4e1f-875f-e31484742781"></figure></blockquote><h2>What do the numbers in circles mean?</h2>The circles focus on the KPIs of the report’s chapters. They tell you how well your ad scored in those key metrics compared to all tested campaigns.For example, the circle for Branding doesn’t tell you the percentage of how many people recognized the brand. You’ll see that in detail in the report. The circle informs you about how much better your branding is vs. all other campaigns, based on your result in brand recall. The number is the KPI’s percentile.&nbsp;Basically, it tells you how many commercials you beat in that metric. After a quick look at the circles, you know how well your ad scored compared to other campaigns without being overwhelmed with detailed results.<blockquote>Example 1: Looking at Amazon’s “Holiday deals” again, we see that this ad nailed it in emotion - it’s better than 99 % of all ads! However, at branding, Amazon beat only 23 % of all ads. At Need and Ad impact, “Holiday deals” scored averagely. So, at this moment we don’t see the exact numbers hidden behind the success or bust (how many people reacted with positive emotion, recognized the brand etc.). But we see the crucial comparison of the ad to all tested campaigns.<figure class="image"><img src="https://cdn-assets.productfruits.com/2923d070-4a71-431d-af97-1b13837f0fd9"></figure></blockquote>At the beginning of each chapter, you see the circle and percentile again. Just below that in the Insight box and in Key metric, you can read the detailed result now. That is the KPI’s result in terms of how many people answered that way in the survey. When we compare that number to benchmarks (all tested campaigns), we get the metric’s percentile - how many campaigns this score beat.<blockquote>Example 2: Regarding Amazon’s “Holiday deals” commercial, we see that at Emotion, the campaign is better than 99 % of all ads. Moreover, we now get to know the key metric’s actual result - that 80 % of the audience reacted positively to the ad.<figure class="image"><img src="https://cdn-assets.productfruits.com/f090cffa-44ca-4130-a950-0d963d1c6fdc"></figure></blockquote><h2>Where do the benchmarks come from?</h2>In the text above, we repeatedly say “all tested campaigns”: That’s our database of benchmarks collected over the course of the last few years, stacked with data from hundreds of creatives. That way we are comparing your score with an up to date creatives rather than a body of historical material. We test hundreds of ads a year to have robust enough data to benchmark your ad to.<h2>Benchmarks are important but tricky</h2><ul><li>Industry standard is seldom the right benchmark.</li><li>For most metrics, your industry is irrelevant. You have to cut through current advertising noise including ads from other industries.</li><li>For some metrics, you have to benchmark against concrete competitors.</li></ul>Not surprisingly, benchmarks are very popular and vendors in the market research category are trying to market themselves accordingly: "We have 200,000 benchmarks in 3,000 product categories!" Good for them, but here is what you should know about benchmarks.&nbsp;Better less but relevantFamously, in 1936, a Literary Digest magazine polled 2 million of its readers to predict Roosevelt was going to lose the election. Roosevelt won, by far. There was another poll, who – having 20 times smaller sample size – got the election results right. It was made by George Gallup (unknown at the time, the biggest brand in market research today) and marked a new era in statistics. A sample of 1 000 people who correctly represent the target population by income and other socio-demographics, is enough. You just don’t need more. And it’s far far better than a 50-thousand or 1-million sample whose structure slightly differs from the population. Similarly, it’s much better to have several hundreds of good benchmarks than hundreds of thousands bad ones – representing irrelevant ads or out-dated metrics. &nbsp;Now – what are the good benchmarks? What should the ads you are comparing yourself to represent? In most cases, the answer is: current advertising noise. To cut through, benchmark against current advertising noise!It’s tempting to compare yourself to your own industry, your direct competitors. You’re supposed to beat these guys after all. But here is the problem: the brains of the people scrolling through their smartphones or watching TV don’t care about industries. And your primary goal is to get their attention – to cut through the advertising noise whether it’s for cars, dating app or panda food. Imagine you are in an industry which really sucks in advertising. You can easily be the best in many metrics and still get zero attention. Automotive advertising, for example, tends to be quite conservative. A car driving through somewhere, with a brand mentioned in a pack-shot. Lower effectiveness, lower return on ad spend. Always look outside your industry box, or some future Tesla of your industry will.A good benchmark is therefore a random sample of all ads that have recently been on-air. It's better to have a hundred of them than thousands of commercials that are dominated by some industry or are ten years old.Ironically, marketing thousands of benchmarks could be a dangerous lock-in for traditional vendors. To avoid losing their thousands-benchmark assets, they are forced to keep many out-dated metrics.Building brand memory structures: when your only option is being the bestSo far, we were talking about getting attention. But getting attention is only a first step in advertising. The main principle of mind-winning advertising is creating the fastest neural path between a particular customer need and a particular brand. For example: I want to buy something online → Amazon.&nbsp;And that’s where benchmarks are getting really tricky. Because only the winner counts. The human brain simply chooses the most mentally available brand. It’s irrelevant whether you are third or fourth best. The only relevant metric is: for how many people is my brand the fastest one. Industry average is again not much useful here. This time, you wanna be much more specific and know what particular brands are the winners for different groups of people or for different needs at your market.&nbsp;So, let’s sum it up: either you have to cut through current advertising noise including ads from other industries, or you want to have data about your concrete competitors. In both cases, industry average is not what you need.&nbsp;

<h2>What is Overall score?</h2>This metric shows overall potential of a creative to be successful and have impact. Thus it is used in reports for concept tests, video pre-tests and any creatives which have not been aired in population yet.There are ways to boost or hinder the ad’s eventual performance when it gets out to the audience: through the generosity of budget used on media spends, the selection of media mix and channels used and so forth. Simply put, future success is not completely guaranteed just by achieving high Overall score. What this metric shows you is the following: how the creative itself is able to contribute to your communication goals.<h2>How do we calculate Overall score?</h2>Overall score is the average calculated from the KPIs of Brand, Need and Emotion (average of their percentile scores). In detail, it is the mean calculated from total brand recall, category recall and total positive emotion, the three chapters’ key metrics.<h2>How to achieve best score?</h2>The higher the values of key metrics in Brand, Need and Emotion chapters, the higher resulting Overall score. Therefore, you must put your best effort into getting branding, need display and emotion right.

Go back to platform

<script async src="https://www.googletagmanager.com/gtag/js?id=G-3SFPFBF1HG"></script>
<script>
 window.dataLayer = window.dataLayer || [];
 function gtag(){dataLayer.push(arguments);}
 gtag('js', new Date());

 gtag('config', 'G-3SFPFBF1HG');
</script>

Was this article helpful?

You recently requested to reset your password for your account. Use the button below to reset it. This password reset is only valid for the next 24 hours.

If you’re having trouble with the button above, copy and paste the URL below into your web browser.

Please check your email for instructions to reset your password.

Check your email

Enter your email address and we will send you instructions to reset your password.

Please enter a new password below.

Set your password