Operations

Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.

Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.

Myth:Last-click attribution is a fair scorecard.

Evidence:Large-scale field experiments show branded search and retargeting capture credit for conversions that would have happened anyway; last-click can overstate paid value by an order of magnitude.

Myth:TV's ROI estimates justify current spend levels.

Evidence:Estimating elasticities and ROI across 288 brands, Shapiro, Hitsch & Tuchman find ad elasticities far smaller than the published literature suggests, negative ROI at the margin for more than 80% of brands, and positive overall ROI for only about a third.

Myth:If ads don't move this week's sales, they didn't work.

Evidence:Across 751 short-term and 402 long-term elasticities from 56 studies (1960-2008), the average short-term advertising elasticity is 0.12 - but the mean long-term elasticity is 0.24, double the short-term, through carryover.

Myth:Outspending competitors on media weight lifts sales.

Evidence:Across 389 BehaviorScan split-cable TV experiments, increasing budget relative to competitors did not increase sales in general - changing brand, copy, or media strategy did. Standard recall and persuasion pretests did not predict which ads would move sales.