
Incrementality testing: What geo and holdout lift tests measure, and where they fall short
What geo lift and holdout incrementality tests actually measure, where each falls short, and where always-on causal measurement fills the gap.

Predictive analytics tells you what will happen; causal AI tells you what to do about it. The methodology difference, and when each one fits.

Every quarter I sit with executives who've done everything the AI playbook asked. They have the models, the pipelines, the forecasts. What most of them can't tell me is what any of it changed.
They aren't outliers. Only 5% of companies are generating substantial value from AI, and 60% are laggards, according to BCG's The Widening AI Value Gap. The models predict well, yet the businesses running them still can't connect a prediction to a decision that moved the number.
That gap isn't a data problem or a tuning problem; it's a methodology problem. Predictive analytics and causal AI answer two different questions, and confusing them is quietly expensive. Here's where the difference sits, and how to tell which one a decision needs.
The two methodologies answer different questions with different machinery, and the contrast is easiest to read side by side.
Predictive analytics | Causal AI | |
|---|---|---|
The question it answers | "What is most likely to happen next?" | "What will happen if we act, and why?" |
How it works | Learns statistical patterns and correlations in historical data | Builds an explicit model of cause and effect, then reasons over interventions |
Can it answer "what if we intervene?" | No; it estimates what tends to follow, without telling you what an action would cause | Yes; it estimates the effect of a decision, net of everything else |
Does it hold when conditions change? | Degrades when the world moves away from its training data | More stable, because it represents the causes underneath the data, which change less than surface patterns |
What it is the right tool for | Forecasting and ranking under stable conditions | High-stakes decisions where you need to know what to change |
Predictive analytics tells you what is likely to happen; causal AI tells you what your decision will cause. Correlation-based models degrade when conditions shift, because they encode past patterns and never the mechanism beneath them. Choosing between the two starts with the question you are actually asking, and both have a place.
Causal AI and machine learning answer different questions. Machine learning finds correlations to predict what is likely to happen next, while causal AI models cause and effect to estimate what an intervention will do. Prediction describes the world; causation lets you change it.
Machine learning has earned its place. Give it enough history and it will spot patterns a human analyst would miss, and it will rank, score, and forecast at a scale no team can match by hand.
The limit is structural. A correlation-based model learns that two things move together; it cannot tell you whether one causes the other or whether a third factor drives both. As TechTarget's reference definition of causal AI puts it, generative AI and large language models are "limited to recognizing and analyzing correlations," while causal AI "digs deeper than other AI in search of cause-and-effect relationships among data."
That distinction decides what each method is good for. When you need to change an outcome, a model that only ranks what is likely leaves you guessing about which lever to pull.
Machine learning learns from what it observes, and observation alone cannot separate cause from coincidence. A correlation-based model sees that two things move together, but not whether one drives the other or a shared factor drives both. Without a model of cause and effect, it cannot say what an intervention would do.
Judea Pearl's ladder of causation, set out in The Book of Why (2018), sorts questions into three rungs.
The first rung is association: what does seeing X tell me about Y? This is where correlation-based machine learning lives, reading patterns in data it has already observed.
The second rung is intervention: what happens to Y if I do X? The third is the counterfactual: what would have happened if I had acted differently? A model trained only to spot patterns cannot climb past the first rung.
The consequence is concrete. Such a model can tell you that customers who received a discount bought more; it cannot tell you whether the discount caused the purchase or simply reached people already about to buy. In a February 2025 analysis, MIT Sloan Management Review calls correlation-based machine learning "flawed, at best" for anticipating the impact of a choice.
Reliable prediction and causal understanding turn out to be the same problem. In their 2024 ICLR paper on causal world models, Google DeepMind researchers showed that any agent able to generalize across a wide range of changing conditions must have learned an approximate causal model.
The mechanism is intuitive once you see it. A purely correlational model is fitted to one set of conditions; when the market shifts — a new pricing regime, a demand shock, a competitor's move — the correlations it learned stop holding, and its predictions drift.
A model that has captured the underlying causes carries the part that stays constant when surface conditions change. That is why dependable generalization is not a bonus you add to prediction; it needs a representation of cause and effect. Our complete guide to causal AI walks through what that means for the enterprise stack.
The limitations of correlation-based business models come from one structural blind spot: they describe what has happened without explaining why. That makes them unreliable guides for decisions. They confuse coincidence with cause, miss the effect of hidden factors, break when conditions change, cannot answer what-if questions, and give you no way to test a decision before you commit to it.
Five limitations recur across these models:
The clearest case I know is Zillow. The company's Zestimate is a home-price prediction model, and in 2021 Zillow built a home-buying business, Zillow Offers, on top of it, purchasing houses at model-set prices to resell them.
Then the housing market moved faster than the conditions the model had learned from. Zillow found itself buying homes at prices it could not recover, and in November 2021 it wound down Zillow Offers and cut about a quarter of its workforce. Its CEO said the unpredictability in forecasting home prices "far exceeds what we anticipated."
The tempting diagnosis is that Zillow needed a better prediction model. The real lesson is quieter: prediction was the wrong basis for an irreversible buying decision, because a model tuned to yesterday's market cannot tell you what a purchase will be worth once you have changed the market by making it.
Prediction is not the weaker tool; it is a different tool, and often it is exactly the right one.
A correlation-based model is a sound choice when three conditions hold: the environment is stable, the decision is low-stakes or easily reversed, and you need a forecast or a ranking. Demand forecasting in a steady category, fraud flags on transactions, lead scoring, churn prediction so a team knows where to look — these are prediction problems. A well-built model handles them well.
The danger is not prediction. It is prediction deployed as if it were causation, using a "what is likely" model to answer a "what should we do" question.
A recent preprint research paper benchmarking causal against correlation-based models for predictive maintenance makes the point without taking sides. Across 10,000 machines, a random forest delivered the highest raw cost saving; a causal model came close on cost and additionally explained why each machine was likely to fail.
The test is always the question you are asking. If you need to know what is likely, predict. If you need to know what your decision will cause, prediction alone will not get you there, and you need causal inference.
When a call is consequential and hard to reverse, causal AI changes the decision: a pricing move, a budget reallocation, a product launch, a capital commitment. Here you need the counterfactual, what happens if we do nothing and what happens if we act, before the money is committed.
Explainability stops being optional at this altitude. Gartner's March 2026 analysis of public-sector AI warns that regulated industries and governments "cannot rely on opaque 'black box' systems for consequential decisions," and a model that can show its reasoning is worth more than one that only hands you a number.
This is the problem DATA POEM exists to solve, in the category we call Enterprise Decision AI. POEM365, our Large Causal Model, is built on a causal architecture we call FOUNT: instead of learning which things have moved together, it builds an explicit model of cause and effect across the business. That lets it estimate the incremental effect of a decision, net of everything else happening at the same time, and show the reasoning behind the number.
Because the model is causal, you can test a decision before you make it. Scenario simulation runs the intervention and returns its likely effect and the drivers behind it, so the counterfactual becomes something you examine before committing, not a bet you place and hope comes good.
Causal inference is the mathematical framework: the methods for estimating cause and effect from data, developed largely by Judea Pearl and others. Causal AI's what you get when those methods are built into a system that can model, simulate, and reason over interventions at the scale of a business. One is the theory, the other the applied technology.
Yes, causal answers are possible without an experiment. Randomized experiments are the cleanest route, but they're often impossible, slow, or unethical. Causal inference supplies methods — controlling for confounders, using natural experiments, modeling how the data was generated — to estimate causal effects from observational data when a controlled test is off the table.
Predictive analytics needs enough clean history to learn stable patterns. Causal AI needs that history plus structure: knowledge of which factors could influence which, so it can separate a cause from a confounder. More data alone doesn't create causal understanding; the model also needs a representation of how the variables relate.
In finance, causal AI estimates the effect of a decision — a price change, a credit-policy shift, a budget reallocation — where forecasting only tells you the likely outcome. It matters most where choices are consequential and must be defended. That's why regulated teams favor models whose reasoning can be inspected.
The mistake is rarely choosing prediction over causation. It is starting from the model at all, asking what the data can predict before asking what the decision in front of you actually requires.
So start with the decision. If the question is what is likely, a predictive model will serve you well. If the question is what to do, and what that choice will cause, you need a model built for cause and effect.
That is the shift worth making: from analytics that describe the business to analytics that can tell you what a decision will do. If you want to see that in practice, Causal Clarity lets you simulate a decision and see its likely effect before you commit — starting with the call you are weighing now. Get in touch, and we’ll show you how.

Founder & CEO
Founder Bharath Gaddam had a clear diagnosis: the problem wasn't data or talent, it was architecture. Correlation-based models were never going to cut it for the complexity of enterprise growth. The industry wasn't under-resourced. It was fundamentally mis-built.
Let's talk about how we can help you grow your business.

What geo lift and holdout incrementality tests actually measure, where each falls short, and where always-on causal measurement fills the gap.

See the mechanism behind causal AI's replacement of MMM, MTA, and forecasting models for enterprise revenue decisions, and where each still falls short.

AI agents execute marketing decisions at scale. But can they own them? Why LLM agents recommend but don't decide, and what causal reasoning changes.

We define the Large Causal Model, and how it differs from three academic uses of the term and from a standard large language model.