You can test ad creative before you spend on it in about a week, with 20 to 30 people from your target audience. Show each person two or three creatives for a few seconds, ask what they remember and what they think the ad is offering, then ask which one they would act on and why. Keep the creative that people both understand and want, fix what they misread, and then put a small live budget behind the top two to confirm with real clicks. The pre-test tells you why an ad works or fails. The live spend tells you whether it does.
This guide is for founders, small marketing teams and agencies without a research budget. It covers what a pre-test can and cannot tell you, a six-step method, the questions to ask, how to read the answers and how to pair it with a small live test.
Why the creative is the part worth testing
Most of what an ad achieves comes from the creative, not the media plan.
- Nielsen Catalina Solutions analysed about 500 campaigns from 2016 to early 2017 and attributed 47% of advertising's sales contribution to the creative, ahead of reach (22%), brand (15%), targeting (9%), recency (5%) and context (2%) (Nielsen, 2017).
- For digital campaigns specifically, Nielsen put the creative's share of sales lift at 56%, against 30% for media (MarketingCharts summary of Nielsen data).
- Ipsos ASI found that creative quality explains about three quarters of the variation in ad recall between ads (The Drum, reporting Ipsos ASI).
And people judge fast. In a study by Gitte Lindgaard and colleagues, ratings of a web page's visual appeal after a 50 millisecond glimpse closely matched ratings made after a longer look (Behaviour and Information Technology, 2006). An ad in a feed gets roughly that kind of glance. If the message does not land in the first second or two, the rest of the ad rarely gets a chance.
So the cheapest money in a campaign is the money spent finding out which creative people understand before you pay to show it to thousands of them.
What a pre-test can and cannot tell you
A small qualitative pre-test is good at:
- Comprehension: do people get what is being offered, and for whom?
- Recall: after a short look, do they remember the brand and the main claim?
- Reactions and objections: what feels off, confusing, cheap or not for them, in their words.
- Preference with reasons: which version they would act on and why.
It is not good at predicting click-through rates, cost per acquisition or how an ad performs at scale in a real feed, next to other ads, with a real algorithm deciding who sees it. Twenty-five people cannot tell you the difference between a 1.1% and a 1.4% click rate. That is why the method ends with a small live spend.
The six-step method
- Pick two or three creatives that differ on one thing. Same offer, different hook. Or same image, different headline. If they differ on everything, you will not know what made the winner win.
- Decide who you need to hear from. People who match the audience you will target, not your team or your most loyal customers. Include a few who use a competitor.
- Show each creative briefly, then hide it. Three to five seconds for a static ad or a thumbnail is close to a scroll. Then ask what they remember, before they can look again.
- Ask what it is offering and who it is for. Compare their answer to the one message you meant the ad to carry.
- Show them side by side and ask which they would act on. Then ask why, and what would make the other one work.
- Confirm with a small live test. Put a modest budget behind the top two in your ad platform's split test and let behaviour decide between them.
How many people
For the qualitative part, 20 to 30 people is usually enough to see whether the message lands and to hear the main objections. Comprehension problems show up fast: if 8 of 25 people think your accounting tool is a bank, you do not need 500 more to know the headline is the problem. Our concept testing methodology guide covers when you need a bigger sample.
Show the versions in a fair order
The first creative anyone sees gets an unfair advantage or disadvantage. Change the order for different people. The simplest way is two copies of the same study with the creatives in opposite order, each sent to half your audience.
The questions to ask
Ask in this order, so later questions do not contaminate earlier ones:
Right after the timed view
- What do you remember from what you just saw?
- What brand was it for?
- What do you think they were offering?
With the creative in view
- Who do you think this is for? Is that you?
- What, if anything, is confusing or hard to believe?
- What would you expect to happen if you clicked?
Comparing versions
- Which of these would you be most likely to click, and why?
- What would make the other one work better for you?
- If you saw this in your feed today, what would you do?
How to read the answers
You are looking for one creative that clears three bars:
- The message lands. Most people describe the offer roughly the way you would. If a third or more get it wrong, fix the copy before anything else.
- The brand is remembered. If people recall the joke but not the name, the ad is entertaining for someone else.
- People want it, for the right reason. A version people prefer because the photo is nice, while misreading the offer, is a trap.
If no version clears all three, do not pick the least bad. Fix the problem people named and test again. That second round is still far cheaper than a week of spend on a creative nobody understands.
Then write down what you learned in one line per creative ("Version B: people remember the price, think it is for agencies, not freelancers"). That line is worth more to the next campaign than the winner itself.
Pair it with a small live test
Once you have one or two creatives that people understand and want, run them in a small split test on the platform you will use. Keep the audience, budget and placement the same, change only the creative, and set the length and budget before you start so you are not tempted to stop at the first good day. The pre-test explains the result; the live test confirms it. When they disagree (people loved B, but A gets the clicks), go back to the reasons. Often the winning ad is getting clicks from people who misread it, which shows up later as poor conversion.
Mistakes that waste the test
- Testing finished ads only. Rough mockups test the idea for much less effort. Polish the one that wins.
- Asking "do you like it?" Liking does not predict action. Ask what they remember, what it offers and what they would do.
- Showing the ad for too long. A minute of study is not how anyone sees an ad. Time the first look.
- Testing with colleagues. They know the offer already. Recruit people who do not.
- Letting the preference vote decide. A version can win the vote and lose the message. Read the reasons.
How to run it in Koji
In Koji you can run the whole qualitative part as an AI interview, in text or voice.
- Add an image to a question. Attach one image (PNG, JPEG or WebP, up to 4 MB) to any interview question: the ad, a thumbnail, a landing page. Participants see it with the question, and Koji knows what the image shows, so its follow-up questions are about the ad.
- Use a timed view for the first look. Set the image to show once, full screen, for 3, 5, 10 or 15 seconds, then hide it. Koji then asks what they remember and what they think, which is the five-second test in step 3. Our 5-second test guide covers that method in depth.
- Ask the comparison as a choice question with shuffled options, then let Koji follow up on the reason.
- Reach the right people. Share the link with your own list, or recruit from Koji's panel by country and audience.
Timed images work in text and voice interviews, not phone calls. Uploaded images are resized, stripped of metadata and stored privately. Before you send the link, run a preview to check the timing and the follow-ups.
The report groups what people remembered and misread, with the quotes behind each theme. For more on method, see concept testing with AI interviews and the older ad testing survey guide. For tools that score ads at scale, see our comparison of ad testing software.
Test your next ad before it goes live
Pick the two creatives you are about to launch, attach each to a question with a 5-second timed view, add the recall and comparison questions above and send the link to 25 people who match your audience. You will know by the end of the week which one people understand, and why. Start free on Koji and set up your first creative test, then put your launch budget behind the version that passes.
