Paid Media & Measurement

Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.

Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.

Myth:Last-click attribution is a fair scorecard.

Evidence:Large-scale field experiments show branded search and retargeting capture credit for conversions that would have happened anyway; last-click can overstate paid value by an order of magnitude.

Myth:TV's ROI estimates justify current spend levels.

Evidence:Estimating elasticities and ROI across 288 brands, Shapiro, Hitsch & Tuchman find ad elasticities far smaller than the published literature suggests, negative ROI at the margin for more than 80% of brands, and positive overall ROI for only about a third.

Myth:If ads don't move this week's sales, they didn't work.

Evidence:Across 751 short-term and 402 long-term elasticities from 56 studies (1960-2008), the average short-term advertising elasticity is 0.12 - but the mean long-term elasticity is 0.24, double the short-term, through carryover.

Myth:Outspending competitors on media weight lifts sales.

Evidence:Across 389 BehaviorScan split-cable TV experiments, increasing budget relative to competitors did not increase sales in general - changing brand, copy, or media strategy did. Standard recall and persuasion pretests did not predict which ads would move sales.

Myth:Platform-reported ROAS is sufficient evidence.

Evidence:Geo-based holdout experiments give unbiased incrementality estimates without user-level tracking and are the most practical causal method for most advertisers.

Myth:Display ad ROI shows up in online conversions.

Evidence:A randomized experiment with 1.6 million Yahoo! users found display ads profitably lifted a major retailer's purchases by 5% - but 93% of the increase happened in brick-and-mortar stores, and 78% came from customers who never clicked the ads.

Myth:A statistically significant lift tells you whether to ship the treatment.

Evidence:A significance test answers whether the experimental means differ, not whether deployment will pay off under future uncertainty. Across 552 advertising experiments, a distribution-aware decision rule substantially reduced regret versus conventional hypothesis testing.

Myth:High frequency against a narrow audience maximises impact.

Evidence:Broad reach with modest frequency consistently outperforms narrow high-frequency plans; marginal response per additional exposure declines quickly.

Myth:With enough user-level data, models can recover causal lift without experiments.

Evidence:Across 663 Facebook RCTs described by 5,000+ features, state-of-the-art observational methods (double/debiased ML, propensity matching) miss the experimental lift by a median 62-115% depending on funnel stage - larger than the median lift itself.

Myth:Cut ad spend first when a downturn hits.

Evidence:The IPA's recession evidence review finds brands that maintain share of voice through a downturn recover faster and gain share over cutters - cutting spend mortgages the recovery.

Myth:Valid lift measurement requires expensive PSA control ads or full platform blackouts.

Evidence:Ghost ads - logging the impression a control user would have been served, without serving or paying for it - reproduce RCT-grade measurement at a fraction of the cost of PSA controls, and are precise enough to have become standard practice at Google and for brands like Duracell and Nissan.

Myth:Better targeting compensates for average creative.

Evidence:In platform meta-analyses creative explains far more outcome variance than incremental targeting precision, especially in auction environments that already optimise delivery.

Myth:Marketing mix modelling replaces experiments.

Evidence:MMM and experiments are complements: experiments calibrate priors, MMM allocates across the whole mix. Neither alone is sufficient.

Myth:Retail media ROAS proves the channel works.

Evidence:Most retail-media ROAS counts sales that would have happened anyway; even incremental ROAS is so methodology-sensitive that Ovative, Albertsons Media Collective and Kellogg faculty built a framework just to standardize it.

Myth:Creative quality alone determines effectiveness.

Evidence:Comparing 4,863 effectiveness award cases (2011-2019), the largest business effects came from 'creatively committed' campaigns: more media spend, more channels, and longer durations behind one creative platform - captured in the Creative Effectiveness Ladder.

Myth:More GRPs fix weak TV performance.

Evidence:Across 288 brands in many categories, TV ad elasticities come out far smaller than the published literature suggests, and marginal ROI is negative for more than 80% of brands.

Myth:Retargeting is the most efficient spend available.

Evidence:Controlled experiments repeatedly find retargeting incrementality far below reported ROAS, because it reaches users already on a purchase path.

Myth:Media mix modeling is a six-figure enterprise tool.

Evidence:Google open-sourced its MMM (Meridian, January 2025) with Bayesian calibration from experiments built in, and Meta's Robyn is also free - credible MMM now costs analyst time, not a vendor contract.

Myth:Rotate creative constantly - ads wear out fast.

Evidence:The ARF's evidence review finds true creative wear-out is less common than assumed: repeated exposure more often builds memory (wear-in) than fatigue, and marketers tire of their ads long before consumers do.

Myth:The programmatic supply chain is basically efficient.

Evidence:Independent audits find a substantial share of programmatic spend disappears into unattributable fees and low-quality inventory before reaching a working impression.

Myth:Losing tracking data only hurts measurement, not performance.

Evidence:Using 3.3 million survey responses across 9,596 display campaigns, Goldfarb & Tucker found display ads became far less effective at shifting purchase intent after the EU Privacy Directive restricted data-driven targeting - with the biggest losses on general-content sites and for small, static formats.

Myth:A safe, neutral ad just works a little less hard.

Evidence:Modelling IPA cases with System1 emotional-response data, 'dull' low-emotion TV ads need far more media to match effective creative: roughly 10m pounds extra per UK campaign, scaling to an estimated $228bn of extra spend across US TV to match the market-share growth of the most impactful spots.

Myth:CTV attention is no better than feed scrolling.

Evidence:In the first industry-endorsed attention study, streaming video ads held almost 80% attentive viewing with minimal drop-off over time - 123% more attentive viewing than scrollable social (Amplified x Video Futures Collective, 2025).

Myth:The more personal an AI-generated ad looks, the better it will work.

Evidence:In a 100-person within-subject study across four products, moderately personalized AI imagery earned the best ad and product attitudes. At high personalization, rising creepiness outweighed the relevance benefit and neutralized gains across ad attitude, product attitude, and purchase intent.

Myth:The more personalized the retargeted ad, the better it performs.

Evidence:In a field experiment with an online travel firm, dynamic ads showing the exact product browsed were on average LESS effective than generic brand ads; specificity only wins once browsing behavior shows preferences have narrowed (e.g. the shopper has started visiting review sites).

Myth:Platform-run lift studies are objective evidence.

Evidence:Platform-run studies use marketer-invisible control construction and frequently disagree with independent geo holdouts, biasing toward the platform.

Myth:Counting how often an AI answer mentions you measures GEO business impact.

Evidence:A new causal GEO measurement framework shows why raw mentions are only one input: business impact also depends on query volume, each engine's share of use, whether people notice the mention, and the response under alternative treatment sequences.

Myth:Concentrated burst campaigns are the efficient pattern.

Evidence:Continuous or near-continuous presence generally beats short heavy bursts for established brands, because buying occurs continuously across the category.

Myth:Your ad spend only builds demand for you.

Evidence:Randomized field experiments on a restaurant-search platform show ads also lift sales for non-advertised competitors; spillovers concentrate on same-cuisine, highly rated rivals and are largest when ad intensity is low.

Myth:Spread the marketing budget across every driver of satisfaction.

Evidence:Modelling customer equity directly, Rust, Lemon & Zeithaml show reallocating spend to the highest-leverage driver of customer equity beats across-the-board investment - in their airline application, the concentrated strategy maximized return on marketing.

Myth:Geo-experiments and MMM answer different questions and cannot be fused.

Evidence:A structural estimation approach recovers the complete MMM parameter set - adstock decay and saturation - directly from geo-experiments, giving the model causal calibration instead of observational guesswork.

Myth:TV is unmeasurable brand-building with no short-term effect to capture.

Evidence:In two-minute windows around 1,224 commercials across 20 brands and $3.4B in spend, TV ads produce immediate, measurable lifts in site traffic and transactions.

Myth:Aggressive keyword blocklists protect the brand at no cost.

Evidence:Broad blocklists defund quality journalism inventory and shrink reach with little measured brand-safety benefit; contextual controls perform better.

Myth:Bidding on your own brand name is always wasted money.

Evidence:Bing field experiments across thousands of brands show brand ads add a small but real 1-4% click lift when no competitor bids, and become defensive when rivals bid to siphon the traffic.

Myth:MMM and lift tests are competing truth sources you reconcile by gut.

Evidence:Google Research lays out how lift-test results should enter a Bayesian MMM as calibrated priors, with an explicit method for choosing which priors to tune; experiments become the model's anchor, not a parallel scorecard.