MMM
Myth:Marketing mix modelling replaces experiments.
Evidence:MMM and experiments are complements: experiments calibrate priors, MMM allocates across the whole mix. Neither alone is sufficient.
Uncalibrated MMM is highly sensitive to specification choices.
Jin et al., Google Research · 2017 · Primary Research
Channels: paid · analytics
MMM
Myth:Geo-experiments and MMM answer different questions and cannot be fused.
Evidence:A structural estimation approach recovers the complete MMM parameter set - adstock decay and saturation - directly from geo-experiments, giving the model causal calibration instead of observational guesswork.
MMM
Myth:MMM and lift tests are competing truth sources you reconcile by gut.
Evidence:Google Research lays out how lift-test results should enter a Bayesian MMM as calibrated priors, with an explicit method for choosing which priors to tune; experiments become the model's anchor, not a parallel scorecard.
The Validity Gap
Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.
Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.
Attribution
Myth:Last-click attribution is a fair scorecard.
Evidence:Large-scale field experiments show branded search and retargeting capture credit for conversions that would have happened anyway; last-click can overstate paid value by an order of magnitude.
The Validity Gap
Myth:TV's ROI estimates justify current spend levels.
Evidence:Estimating elasticities and ROI across 288 brands, Shapiro, Hitsch & Tuchman find ad elasticities far smaller than the published literature suggests, negative ROI at the margin for more than 80% of brands, and positive overall ROI for only about a third.
One crib in your inbox every Monday. No spam, unsubscribe anytime.