Measurement

Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.

Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.

Myth:TV's ROI estimates justify current spend levels.

Evidence:Estimating elasticities and ROI across 288 brands, Shapiro, Hitsch & Tuchman find ad elasticities far smaller than the published literature suggests, negative ROI at the margin for more than 80% of brands, and positive overall ROI for only about a third.

Myth:Display ad ROI shows up in online conversions.

Evidence:A randomized experiment with 1.6 million Yahoo! users found display ads profitably lifted a major retailer's purchases by 5% - but 93% of the increase happened in brick-and-mortar stores, and 78% came from customers who never clicked the ads.

Myth:With enough user-level data, models can recover causal lift without experiments.

Evidence:Across 663 Facebook RCTs described by 5,000+ features, state-of-the-art observational methods (double/debiased ML, propensity matching) miss the experimental lift by a median 62-115% depending on funnel stage - larger than the median lift itself.

Myth:More GRPs fix weak TV performance.

Evidence:Across 288 brands in many categories, TV ad elasticities come out far smaller than the published literature suggests, and marginal ROI is negative for more than 80% of brands.