Paid Media

Incrementality

Measurement

Primary Research
Effort: high
Volume: medium

Myth:Platform-reported ROAS is sufficient evidence.

Evidence:Geo-based holdout experiments give unbiased incrementality estimates without user-level tracking and are the most practical causal method for most advertisers.

The Nuance

Geo tests need adequate market count and spend separation to reach usable power.

The Receipt

Estimating Ad Effectiveness using Geo Experiments

Vaver & Koehler, Google Research · 2011 · Primary Research

Impact8.9/10
Consensus8/10
Evidence84/100

Channels: paid · analytics

Related Cribs

Incrementality

Measurement

Myth:Your ad spend only builds demand for you.

Evidence:Randomized field experiments on a restaurant-search platform show ads also lift sales for non-advertised competitors; spillovers concentrate on same-cuisine, highly rated rivals and are largest when ad intensity is low.

Impact7.8/10
Consensus7.2/10

Incrementality

Measurement

Myth:Bidding on your own brand name is always wasted money.

Evidence:Bing field experiments across thousands of brands show brand ads add a small but real 1-4% click lift when no competitor bids, and become defensive when rivals bid to siphon the traffic.

Impact7.2/10
Consensus7.5/10

The Validity Gap

Measurement

Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.

Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.

Impact9.4/10
Consensus8.8/10

Attribution

Measurement

Myth:Last-click attribution is a fair scorecard.

Evidence:Large-scale field experiments show branded search and retargeting capture credit for conversions that would have happened anyway; last-click can overstate paid value by an order of magnitude.

Impact9.3/10
Consensus8.4/10

Crib of the Week

One crib in your inbox every Monday. No spam, unsubscribe anytime.