AI Shopping
Also filed underGEO
Myth:AI shopping agents always find the best unit price regardless of how product information is presented.
Evidence:In a controlled coffee-choice study across eight LLMs, a vague best-deal goal plus an artificial $10 cost per attribute lookup cut the lowest-unit-price choice rate from 89% to 29%. With an explicit lowest-price-per-ounce goal, the rate fell from 94% to 83%. Missing price and weight details, rather than arithmetic alone, shaped the choices.
This September 2026 preprint reports 16,400 simulated sessions across its studies, not real shoppers or live purchases. The quoted rates are from Study 1, not the full sample. The lookup charge is an experimental constraint, not a real storefront fee, and optimality is defined as lowest price per ounce.
Davood Wadi & Yu Ma, McGill University / arXiv · 2026 · Academic
Channels: ecommerce · content
The Validity Gap
Myth:A big enough A/B test or attribution platform can pin down each campaign's ROI.
Evidence:Across 25 large field experiments with major U.S. retailers and brokerages, the median confidence interval on ad ROI was over 100 percentage points wide; individual-level sales are so volatile (coefficient of variation ~10) that an informative test often needs 10M+ person-weeks.
Checkout
Myth:Long checkouts are fine if the product is good.
Evidence:Average large-site checkouts ask for far more fields than necessary; trimming to the minimum viable set is one of the highest-yield conversion changes available.
The Validity Gap
Myth:Display ad ROI shows up in online conversions.
Evidence:A randomized experiment with 1.6 million Yahoo! users found display ads profitably lifted a major retailer's purchases by 5% - but 93% of the increase happened in brick-and-mortar stores, and 78% came from customers who never clicked the ads.
Checkout
Myth:Forcing account creation builds your CRM.
Evidence:Mandatory account creation is a top-cited abandonment driver; guest checkout with a post-purchase account offer captures both revenue and the record.
Retail Media
Myth:Retail media ROAS proves the channel works.
Evidence:Most retail-media ROAS counts sales that would have happened anyway; even incremental ROAS is so methodology-sensitive that Ovative, Albertsons Media Collective and Kellogg faculty built a framework just to standardize it.
One crib in your inbox every Monday. No spam, unsubscribe anytime.