Incrementality Testing on a Small Budget: The Geo Holdout Method (2026)

platform-reported roas tells you what the platform could match, not what it caused. the only way to know what a channel causes is to turn it off somewhere and watch what happens. that used to need a vendor, a data science team, and a budget most brands under $5 million do not have. in 2026 the tooling is free and open source, and the method fits a spreadsheet. this guide is the small-budget version.

---

what a geo holdout actually is

a geo holdout is an experiment where you stop (or change) advertising in a set of geographic markets and keep it running everywhere else. the markets where you stopped are the treatment group. the rest of the country is used to build a synthetic control, a weighted blend of untouched markets that tracked the treatment markets' history before the test. the difference between what the synthetic control predicts and what the treatment markets actually did during the test is the lift.

three things most guides get wrong:

1. it is not a platform lift study. meta's conversion lift and google's conversion lift split users inside the platform. a geo holdout splits your own sales data by region. it works for any channel, including ones with no lift product, and it measures orders from your store or crm, not platform conversions.
2. it does not need a vendor. meta open-sourced geolift in 2021, "an end-to-end geo-experimental methodology based on synthetic control methods used to measure the true incremental effect (lift) of ad campaign" (geolift, github). google launched meridian geox globally in september 2026, "a privacy-durable and publisher agnostic incrementality solution that measures online, offline, and omnichannel sales" (google ads, meridian geox announcement). both are free.
3. the output is a multiplier, not a verdict. the point is not "does meta work." the point is "meta's in-platform roas overstates or understates incremental roas by x," so you can correct every number the platform gives you from then on.

---

the fundamental shift: from attribution to causation, at a price a small brand can pay

before: incrementality was for brands spending seven figures a channel. google's user-based conversion lift needed budgets that, by google's own account, approached six figures. geo tests were sold as managed services. everyone else ran on platform attribution and hoped.

after:

• google cut the conversion lift based on users threshold to "$5,000 usd" and "at least 1,000 observed conversions," with studies "as short as 7 days" though google recommends "a minimum of 14 days" (google ads help, set up conversion lift based on users).
• geolift ships with power calculators that tell you, before you spend anything, which markets to hold out and for how long to detect a given lift.
• meridian geox does the same with time-based regression and feeds the result straight into meridian, google's open-source marketing mix model.
• meta shipped incremental attribution, a bidding and reporting setting trained on its own lift experiments.

in plain english: the experiment moved from a vendor engagement to a quarterly routine any brand with regional sales data can run.

---

what this broke in existing accounts

1. budgets set on platform roas

haus's analysis of 640 meta incrementality tests found meta drove "on average ~19% lift to brands' primary kpi," and that 7-day click attribution "under-reports by 15% on average" for meta overall (haus, the meta report). that is the channel average across advertisers spending an average of $14 million a year on meta. the spread inside it is what hurts: the same dataset shows advantage+ "over-reported relative to manual campaigns by 12 percentage points on average," and "58% of brands saw higher iroas on manual campaigns than advantage+." budgets set on advantage+ reported roas were set on the most flattering number in the account.

2. retargeting and brand search that looked like the best campaigns

campaigns that reach people who were already going to buy have the highest reported roas and the lowest incremental roas. accounts that scaled them on reported roas scaled the least incremental spend in the account.

3. tests that ended before the purchase cycle did

geolift's own guidance: "a good rule of thumb when deciding duration is to make sure that the test period can contain at least one full purchase cycle" (geolift walkthrough). a 7-day test on a product with a 3-week consideration window measures nothing.

4. tests read before the post-period

ads stop and orders keep coming for a while. haus's meta tests average 18.6 days of treatment plus an 8.8-day post-treatment window, and the advantage+ finding flipped over that window: "9% better than manual campaigns at the experiment midpoint but 12% worse by the end." a test read at the midpoint gives the wrong answer.

5. too little signal in the test markets

the constraint is conversions in the treatment geos, not spend. a test on markets that produce 40 orders a week cannot detect a 10% lift in two weeks. the power analysis exists to tell you this before you run it.

6. leakage

test geos targeted by ip while orders ship to a billing address, mobile users on carrier networks resolving to the wrong city, and other channels quietly changing budgets during the test. each one adds noise the synthetic control cannot explain.

7. pausing the whole account instead of the channel

a holdout answers one question about one channel. pausing three channels in the test geos at once answers none of them.

---

the data

independent datasets first, platform figures second.

• haus, 640 meta incrementality tests since the start of 2024: average treatment 18.6 days plus 8.8 days post-treatment; meta drove roughly 19% lift to the primary kpi on average; 96% of account-wide studies detected significant lift by the midpoint; 77 of the 100 highest-lift experiments on the platform were meta tests; advantage+ over-reported by 12 points relative to manual and 58% of brands saw higher iroas on manual. (haus, the meta report)
• haus, meta incremental attribution versus standard attribution: tests from july 2024 to june 2025 showed a "pooled geo-mean of 0.80x," meaning standard attribution outperformed. tests from july 2025 to june 2026 showed "1.26x (range 1.11x to 1.43x)," with dtc-only brands at 1.38x and omnichannel at 1.02x (haus, is meta's incremental attribution outperforming standard attribution). the setting went from worse to better in a year.
• stella, 225 geo-based tests from august 2024 to december 2025: median incremental roas of 2.31x, with 88.4% of tests reaching statistical significance (stella, 2025 dtc incrementality benchmarks).
• geolift's reference walkthrough uses 90 days of daily geo-level history, tests 10 and 15-day treatments, considers 2 to 5 test markets out of 40 candidates, runs at alpha 0.1, and reports minimum detectable effects of roughly 4% to 10% depending on the markets chosen. market selection scores candidates on pre-test fit (correlation above 0.91 with the treatment series in the example) and on the cost per incremental conversion the test would need. (geolift walkthrough)
• google conversion lift based on users: $5,000 minimum, 1,000 observed conversions, confidence selectable from 50% to 95%, 7-day minimum, 14 days recommended, one lift study per campaign at a time. geography-based conversion lift supports search, shopping, performance max, app, and offline conversions, and its budget "tends to be higher than users-based studies" (google ads help, comparing lift types).
• meta, january 2026: advertisers using incremental attribution saw a 24% increase in conversions versus the standard attribution model, self-reported (meta newsroom).

the honest read: the median geo test says paid media is incremental, and it says the platform number is wrong by a margin that differs per brand, per channel, and per campaign type. only your own test tells you your margin.

---

what changed in 2026 specifically

september 2026: meridian geox left beta and is available globally, open source, with colab notebooks and a technical whitepaper at developers.google.com/meridian/geox. results calibrate meridian mmm directly. this is the first vendor-grade geo methodology from google that a small brand can run without an account team.

july 2025 to june 2026: meta's incremental attribution flipped from underperforming to outperforming standard attribution in haus's pooled tests. if you tested it in 2024 and turned it off, the 2026 setting is a different product.

ongoing: google's conversion lift lives in the experiments section of google ads, and geography-based studies now cover search, shopping, and performance max, which user-based studies do not. for search-heavy accounts, geo is the only lift product that covers the main spend.

unchanged: the math. synthetic control, a power analysis before you commit, at least one purchase cycle, and a post-period.

---

what actually works now

1. pick one kpi from the source of truth

orders, revenue, or qualified leads from the store, the crm, or the payment processor. never a platform conversion. the whole point is to measure something the platform did not count.

2. build the daily geo-level history

90 days minimum of daily kpi by market. for the us, dma or state; for other countries, region or city. use shipping or billing region, and use the same field the platforms can target, because the exclusion has to match the measurement.

3. run the power analysis before choosing anything

load the history into geolift (r) or meridian geox (python, colab) and ask: for 2, 3, 4, and 5 test markets and for treatment lengths that cover a full purchase cycle, what is the minimum detectable effect? choose the smallest design whose detectable effect is comfortably below the lift you expect. if the expected lift is 15% and the best design detects 20%, you cannot run this test yet: extend the duration, add markets, or test a bigger channel.

4. choose holdout or scale-up

holdout (go dark) is the cleanest read and costs you the revenue those markets would have made from the channel. scale-up (raise spend 50% or more in test markets) protects revenue but measures marginal return, not total. small budgets usually get a cleaner answer from a holdout on a channel that is at least a quarter of spend.

5. implement the exclusion at the geo level, not the campaign level

exclude the test markets in every campaign on the channel under test. do not pause campaigns, do not touch other channels, do not launch creative during the test. record the start date, the markets, and the spend in those markets the week before. spend in test markets should drop to near zero within 48 hours; if it does not, the exclusion is leaking.

6. run the full window, then the post-period

treatment for at least one purchase cycle. then a post-period of roughly half the treatment length with the exclusion still in place, so lagged orders land. do not read the result until the post-period ends. haus's data shows the answer changes between the midpoint and the end.

7. compute the three numbers

• incremental conversions: observed minus synthetic-control prediction, summed over treatment plus post-period.
• incremental roas: incremental revenue divided by the spend that was removed from the test markets.
• the multiplier: incremental roas divided by the platform-reported roas for the same channel and period. this is the number you keep.

8. apply the multiplier and schedule the next test

correct targets: if the multiplier on meta is 0.6, a reported 3.0 roas is an incremental 1.8, and the target roas in the platform needs to reflect that. rebalance budget toward channels with higher incremental return. re-test the biggest channel each quarter and rotate through the others, because the platforms change and the multiplier drifts. when a channel qualifies for google's $5,000 user-based lift study, run it alongside the geo test as a second opinion, not a substitute.

---

what to stop doing immediately

• stop setting budgets or targets on platform-reported roas without a multiplier.
• stop scaling retargeting and brand campaigns on reported roas.
• stop running tests shorter than one purchase cycle.
• stop reading results before the post-period closes.
• stop pausing campaigns as the holdout mechanism. exclude geos.
• stop changing anything else on the account during the test.
• stop treating one test as permanent. re-test quarterly.

---

the orbit agent instructions

{
  "guide": "incrementality-testing-geo-holdout",
  "agent_tasks": {
    "assess_geo_test_feasibility": {
      "check": "for each channel, compute share of total spend, weekly conversions from the store or crm by region for the last 90 days, and whether regional targeting and regional sales data use the same geography field",
      "flag": "no daily regional kpi history, fewer than 90 days of it, a channel under 15 percent of spend proposed for testing, or a mismatch between targetable geography and sales geography",
      "recommend": "build the daily regional kpi export first and test the largest channel; if history is short, start collecting now and schedule the test for 90 days out"
    },
    "prepare_power_analysis_inputs": {
      "check": "assemble the 90-day daily kpi by market, estimate the purchase cycle from median time from first visit to order, and estimate cost per incremental conversion from current spend and conversions",
      "flag": "purchase cycle longer than 21 days with a proposed test under 28 days, or estimated minimum detectable effect above the lift the team expects",
      "recommend": "extend the treatment period to cover a full purchase cycle plus a post-period of half that length, and add test markets until the detectable effect is comfortably below the expected lift"
    },
    "verify_geo_exclusions_live": {
      "check": "48 hours after the start date, read spend by region on the channel under test and confirm every active campaign excludes the treatment markets",
      "flag": "any campaign on the channel still serving in a treatment market, or treatment-market spend above 5 percent of its pre-test level",
      "recommend": "add the exclusion to the missing campaigns, note the leakage window, and extend the test by the number of leaked days"
    },
    "monitor_test_integrity": {
      "check": "daily during treatment and post-period, compare budgets, bid strategies, creative launches, and geo settings on every channel with their pre-test values",
      "flag": "any change on any channel in the treatment or control markets during the test window",
      "recommend": "revert the change if possible, otherwise log it and exclude the affected days from the readout"
    },
    "compute_incremental_readout": {
      "check": "after the post-period closes, compute incremental conversions against the synthetic control, incremental roas against removed spend, the confidence interval, and the multiplier against the platform-reported roas for the same channel and window",
      "flag": "confidence interval spanning zero, a post-period shorter than a third of the treatment period, or a readout requested before the post-period ends",
      "recommend": "report lift, incremental roas, and the multiplier with the interval; if not significant, report the detectable effect and propose a longer or larger follow-up rather than a conclusion"
    },
    "apply_multiplier_to_targets": {
      "check": "read the current target roas or target cpa and budget share for the tested channel and compute the corrected values using the multiplier",
      "flag": "targets or budget shares that still reflect platform-reported roas 14 days after a completed readout, or a multiplier older than 120 days",
      "recommend": "propose corrected targets and reallocation for approval, and schedule the next test on the largest untested channel"
    }
  }
}

---

the bottom line

a geo holdout is the cheapest honest measurement in marketing: regional sales data you already have, a free power calculator, 2 to 5 markets, one purchase cycle, a post-period, and a multiplier you apply to every platform number afterward.

the median test says your ads work. the same test says the platform is wrong about how much. run it once a quarter and you will be the only person in the room who knows by how much.

---

built by orbit - the performance marketing intelligence layer. questions? jose@orbitllm.com