
Incrementality testing: What geo and holdout lift tests measure, and where they fall short
What geo lift and holdout incrementality tests actually measure, where each falls short, and where always-on causal measurement fills the gap.

See the mechanism behind causal AI's replacement of MMM, MTA, and forecasting models for enterprise revenue decisions, and where each still falls short.

Most enterprises run three separate models to answer one question: what's actually driving revenue. Marketing mix modeling gives finance one number. Multi-touch attribution gives marketing another. Standard machine learning forecasts a third, and none of them agree, because none can represent what would have happened under a different decision.
The architecture is the constraint: every regression-based model, however much data you feed it, stops short of what a revenue decision actually needs, a credible counterfactual. If you need the fuller mechanism behind causal AI first, that's covered elsewhere; here, we go straight to where MMM, MTA, and standard ML forecasting each break, and how causal AI replaces them.
Marketing mix modeling has served marketers well for decades, and at the aggregate level it remains a reasonable planning tool. The trouble starts the moment you ask a revenue decision question instead of a planning one: MMM runs on ordinary least squares regression, which estimates the marginal effect of one variable at a time, so once two channels move together, the model cannot separate their effects.
Multi-touch attribution earns its keep at the channel level, giving marketers real granularity on individual touchpoints. It breaks the same way MMM does once channels interact, because attribution assigns credit to what a customer touched, not to what caused them to buy.
Standard machine learning forecasting is genuinely good at pattern detection, spotting trends a human would miss. None of that helps once you ask it what would happen under a decision it has never seen: a forecasting model has no mechanism for representing a counterfactual.
You know you have this problem when your CFO's revenue forecast and your CMO's marketing plan are each internally consistent, and still don't agree.
Causal AI handles confounders and counterfactuals in commercial environments by building a structural model of the variables that jointly affect an outcome. It then estimates what would have happened to revenue if one input — a price change or a media investment, for instance — changed and nothing else did. A confounder is any variable, like seasonality or competitor activity, that affects both the input and the outcome, skewing a correlation-based read.
Multicollinearity, variables that correlate with each other, does the real damage here. Ordinary least squares regression assumes each variable's marginal effect can be isolated; once two inputs correlate strongly, that assumption fails and the coefficients become unstable. Correlation-based machine learning is "flawed, at best" for anticipating the impact of choices, per MIT Sloan Management Review.
Causal AI sidesteps the instability by modeling the full causal graph instead of a single regression equation, holding every other driver constant while asking what one decision would change. See FOUNT for the underlying architecture.
MMM, MTA, and standard ML forecasting sit side by side in most enterprise stacks, and each answers a narrower question than a revenue decision requires. For the detailed product comparison, see POEM365 vs MMM vs MTA. But here's what each method answers, where it breaks, and what causal AI does instead.
Traditional method | What it answers | Where it breaks for revenue decisions | What causal AI does instead |
|---|---|---|---|
Marketing mix modeling | Aggregate channel contribution to sales over a planning period | Cannot isolate one variable's effect once channels correlate | Estimates each channel's incremental effect holding while every other driver constant |
Multi-touch attribution | Which touchpoints a customer interacted with before converting | Assigns credit to correlation with the touch, not causation of the purchase | Measures causal incrementality: the effect net of what would have happened anyway |
Standard ML forecasting | What's likely to happen next, based on historical patterns | Has no mechanism to model a decision it has not seen before | Simulates the outcome of a specific decision before you make it |
MMM has earned its keep for aggregate strategic planning: a reasonable way to size a media budget across a planning cycle, and most marketing organizations still lean on it for exactly that.
Where it breaks is the moment two channels move together, what we call the Gordian Knot of Multicollinearity. Ordinary least squares regression estimates the marginal effect of one variable holding every other constant. Increase paid search and paid social spend in the same quarter, and the regression cannot tell you which one drove the lift, because the two variables move together in the data.
The model does not fail loudly. It produces a coefficient that looks precise and is not, and a revenue decision built on that coefficient inherits the error. A causal model handles the same correlated channels by estimating each one's incremental effect directly, without assuming they move independently.
MTA does real work at the channel level, giving marketers granularity that aggregate models cannot: which touchpoints a specific customer interacted with before converting.
Where it breaks for a revenue decision is the gap between correlation and causation. Attribution assigns credit based on what a customer touched on their way to a purchase, not on what caused them to buy. A customer who saw a retargeting ad right before checkout gets credited for the sale, even if they had already decided to buy before the ad ran.
Scale that error across a media budget, and you get a model that reliably tells you where customers were, and unreliably tells you what to fund next. Causal AI measures causal incrementality instead: the lift a channel produces, net of what would have happened without it, which is the number a revenue decision actually needs.
Standard ML forecasting is genuinely useful for pattern detection. Fed enough historical data, it spots seasonality, trend, and demand signals a simpler model would miss.
Where it breaks is when you ask it about a decision it has never seen. A forecasting model, however sophisticated, is trained to project patterns forward. It has no built-in way to represent what would happen if you changed price, launched in a new region, or cut a media channel, because none of that appears in its training data as a controlled experiment.
It can tell you demand is trending up. It cannot tell you why, or what happens to that trend if you intervene. Causal AI closes that gap by modeling the causal structure behind the pattern, simulating a decision before you make it, not just extending a line.
Consider an enterprise CPG brand weighing a national price increase on a core product line ahead of peak season. Marketing wants to hold price and lean on promotional media to protect volume; finance wants the increase to protect margin. Both teams have a model, and both models say they are right.
The MMM shows historical price elasticity supporting the increase without meaningful volume loss. The forecasting model, trained on last year's seasonal pattern, projects volume growth regardless of price. Neither model can say what happens to volume, share, and margin together if the price moves and a competitor reacts.
A causal model treats the decision as an intervention. It holds competitor pricing, seasonality, and promotional spend constant as separate causal inputs, then estimates what happens to volume and margin specifically because of the price change, net of everything else moving at once.
That's what a revenue decision needs: an estimate of what happens to this business, this quarter, if this lever moves. The team can compare that estimate against the do-nothing case before committing.
Once you can see the causal effect of a decision before you make it, the operating model changes. Incremental ROI, the profit a decision caused net of what would have happened anyway, becomes the headline number instead of one metric among many. Causal contribution by channel replaces share of attributed conversions as the measure that marketing reports against finance.
This is where Causal Clarity earns its place: the piece of Enterprise Decision AI that shows what's actually driving the business, so the CFO's forecast and the CMO's plan read the same causal model instead of two disagreeing ones. Growth Planning builds the plan itself on that same causal read, one plan across the business instead of one per function.
The operating shift is less about new tools and more about which model gets the final word. When finance, marketing, and operations run decisions through the same causal structure, disagreement becomes a question the model can answer.
The difference between causal AI and traditional AI is mechanism: causal AI models cause and effect directly instead of detecting patterns in historical data. Traditional AI, including most machine learning, predicts what's likely to happen based on correlation. Causal AI asks what would happen under a specific decision, and attributes an outcome to its actual cause.
Causal AI enhances predictive analytics in that it adds a layer prediction cannot provide alone: the ability to test a specific decision against a counterfactual before you commit to it. It does not discard prediction. Prediction still spots trends and demand signals well.
Causal AI complements business intelligence instead of replacing it. BI describes what's happening across the business; causal AI adds the decision layer, explaining why and what a specific choice would change. See decision intelligence versus business intelligence for where each fits.
A causal model needs data most enterprises already collect: historical spend, pricing, distribution, competitive activity, and outcome data like sales, at a granularity that captures how these variables move together. What matters more than volume is coverage: enough variation in each input to separate one variable's effect from another's.
No. Standard machine learning, including most predictive AI, is built to detect correlation, not causation. Causal AI is a distinct technical framing: not attribution, not pattern-matching, causation. It draws on the same AI toolkit but applies causal inference to answer what a decision would change.
Every enterprise we talk to runs into the same wall eventually: the models keep disagreeing, and no one can prove which one is right. The fix is the layer that resolves the other three: one causal structure giving finance, marketing, and operations the same answer.
That's what Enterprise Decision AI is built to do, and DATA POEM built POEM365 to deliver it. If you want to see what a causal read looks like against your own revenue decisions, get in touch and we'll walk through one with you.
Let's talk about how we can help you grow your business.

What geo lift and holdout incrementality tests actually measure, where each falls short, and where always-on causal measurement fills the gap.

Predictive analytics tells you what will happen; causal AI tells you what to do about it. The methodology difference, and when each one fits.

AI agents execute marketing decisions at scale. But can they own them? Why LLM agents recommend but don't decide, and what causal reasoning changes.

We define the Large Causal Model, and how it differs from three academic uses of the term and from a standard large language model.