How to Run Ad Testing and Creative Testing Surveys That Maximize ROI

The average company wastes 26% of its advertising budget on ineffective creative. That's not a targeting problem or a media buying problem. It's a creative testing problem. Most ads launch based on internal consensus ("the CMO liked this version") rather than audience validation.

Ad testing and creative testing surveys evaluate advertising concepts, messaging, and assets before they go live. The goal is simple: spend your budget on creative that works, not creative that your team likes.

But traditional ad testing has a problem. When you show someone an ad in a survey and ask "Do you like this?", you get claimed preference, not genuine reaction. People rationalize their feelings, give socially acceptable answers, and struggle to articulate why one creative resonates more than another.

Koji's conversational approach gets past claimed preference to genuine reaction by probing into emotional responses, associations, and behavioral intent in real-time.

Types of Creative Testing

Concept Testing

Evaluate rough concepts (descriptions, storyboards, wireframes) before production. This is the highest-leverage testing because you catch problems before spending on production.

Copy Testing

Evaluate specific headlines, body copy, and messaging frameworks. Tests whether the words communicate the intended message and drive the desired action.

Visual Testing

Evaluate design, imagery, color, and layout. Tests whether visual elements attract attention, communicate brand, and support the message.

Full Creative Testing

Evaluate finished ads (video, display, social, print). Tests the complete execution against campaign objectives.

A/B Preference Testing

Compare two or more creative options head-to-head. Determines which version is most effective with the target audience.

Building Ad Testing Studies with Koji

Concept Testing Study

Setup: Share the concept (image, video, or description) before the interview starts, or describe it in the first question.

Q1: First Reaction (Open-ended) "You just saw [concept]. What's your immediate reaction?"

• Probing depth: 2
• AI instruction: "Capture the gut reaction before rationalization kicks in. Don't ask if they like it. Ask what they noticed first, how it made them feel, what they thought it was about."

Q2: Message Comprehension (Open-ended) "In your own words, what is this ad trying to tell you?"

• Probing depth: 1
• Tests whether the intended message lands

Q3: Emotional Response (Scale, 1-5 on multiple dimensions) Rate how this concept makes you feel:

• Interested: 1-5
• Trustworthy: 1-5
• Relevant to me: 1-5
• Memorable: 1-5

Q4: Brand Association (Open-ended) "What kind of brand would run this ad? What do you think about them?"

• Probing depth: 1
• Tests brand fit

Q5: Behavioral Intent (Scale, 1-5) "How likely would this ad be to make you want to learn more?"

• Probing: "What would you do next if you saw this ad?"

Q6: Improvement (Open-ended) "If you could change one thing about this concept, what would it be?"

• Probing depth: 2

A/B Comparison Study

Setup: Show both options (A and B) with neutral labels.

Q1: Preference (Single Choice) "Which of these two ads do you prefer?"

• Options: Ad A / Ad B / No preference
• Probing depth: 3
• AI instruction: "This is the most important probing. WHY do they prefer one over the other? What specific element? Color, message, imagery, tone? Push past I just like it better."

Q2: Per-Ad Reactions (Open-ended) "What did you notice first about each ad?"

• Probing: AI compares specific reactions to each

Q3: Message Clarity (Open-ended) "Which ad communicated its message more clearly? Why?"

• Tests comprehension difference

Q4: Action Intent (Single Choice) "Which ad would you be more likely to click on/respond to?"

• May differ from preference (people sometimes prefer the "pretty" ad but would click the "direct" one)

Message Testing Study

Q1: Headline Reaction (Open-ended) "Read this headline: [headline]. What does it make you think?"

• Test 3-5 headlines in sequence
• Probing on each: "How relevant is this to you? Why?"

Q2: Headline Ranking (Ranking) "Rank these headlines from most to least compelling:"

• Probing on top and bottom: "What makes the top one strongest?"

Q3: Value Proposition (Open-ended) "Based on [message], what problem do you think this product solves?"

• Tests whether the value proposition communicates clearly

Q4: Credibility (Scale, 1-5) "How believable is this message?"

• Probing on low scores: "What feels unbelievable or exaggerated?"

Creative Testing Best Practices

Test with your actual audience

Internal team opinions are biased by proximity, politics, and personal taste. Test with people who match your target audience profile. Koji can recruit from your customer base or use qualification questions to screen respondents.

Test concepts, not just finished work

The earlier you test, the cheaper changes are. A concept test before production can save more than a creative test after production.

Test one variable at a time

When comparing two ads that differ in headline, imagery, AND color, you can't know which variable drove the preference. Isolate variables for clean learnings.

Capture emotional reactions first

"What's your gut reaction?" should always come before "Do you like it?" The gut reaction is more predictive of real-world behavior than rational evaluation.

Don't over-test

Testing 20 concepts creates paralysis. Test 2-4 options maximum per study. Eliminate clearly weak options internally, then validate the finalists with research.

Measure comprehension separately from preference

An ad can be well-liked but poorly understood, or clear but unengaging. Test both dimensions independently.

Why Koji Is Ideal for Creative Testing

Traditional ad testing tools (Zappi, System1, Qualtrics) use structured surveys with rating batteries. Koji adds conversational depth:

• Genuine reactions captured through open-ended conversation before rationalization
• Specific feedback on what works and what doesn't (not just a preference score)
• Emotional probing that reveals why one creative resonates and another doesn't
• Fast turnaround with results in hours, not weeks
• Scale to test across multiple audience segments simultaneously
• Voice interviews that capture tone, enthusiasm, and hesitation
• Cost efficiency that makes testing every campaign affordable, not just the big ones
• Multi-language testing for global campaigns (30+ languages)

Every dollar spent on creative testing saves multiple dollars in media waste. Koji makes testing fast, deep, and affordable enough to test every campaign.