When your Growth Advisor hands you an experiment design, it includes a test market assignment, a minimum required budget, a duration, a power curve, and instructions for setting up the campaigns. This article walks through what's inside each of those pieces, why it's built the way it is, and what it means for the decisions you'll make during and after the test.

It covers geo-based experiments specifically. For other test types (platform conversion lift studies, user-level holdouts, time-based holdouts) or help deciding which test type is right, start with the Experiment Design FAQs.

If your Growth Advisor sends multiple new designs in your dashboard at once, the quickest way to review them is to compare the same five fields across each design: the hypothesis, the target metric, the target lift (or minimum detectable effect), the recommended duration, and the budget or spend change required in the test markets. A good approval conversation is usually less about which design sounds most exciting and more about which one is powered to answer a real budget decision with an affordable level of risk.

What is a geo experiment?

A geo experiment measures the incremental impact of a marketing channel by running different treatments across different geographic markets at the same time. Some markets get more of the channel (or less of it); others don't. Because the two groups are running concurrently, external factors like seasonality and macro shifts affect both sides equally, and the difference between them reflects what the channel actually caused.

The two most common forms are a holdout (spend is cut in some markets) and a pulse up (spend is increased in some markets). Both are analyzed the same way.

Glossary

A few terms come up repeatedly in experiment designs. This is the short version of each:

Before we can design your test

A few things need to be in place before a clean experiment design is possible. Your Growth Advisor will check for each of these and flag anything that's missing.

• Historical data in your target metric. The matching algorithm needs at least a few months of geography-level data in the metric you want to measure, whether that's signups, pipeline, or purchases. Thin or inconsistent data limits how well test and control markets can be matched.

• Enough eligible markets. Geo tests generally work best with 10 or more markets on each side. Businesses that operate in only a handful of geographies can sometimes still run a test, but the design tends to be tighter and may require a larger budget to reach statistical confidence.

• Geographic independence. The design works best when what happens in one market doesn't strongly affect another. For digital channels this is usually fine; for out-of-home, linear TV, and some audio buys, spillover across adjacent markets needs to be handled in the design.

• Feasible channel. Some channels are harder to measure with geo experiments. Organic and non-geo-targeted channels generally can't be tested this way. If the channel you want to test falls in this category, your GA will propose an alternative approach.

If any of these aren't in place yet, that's not the end of the road. Your GA will tell you specifically what's blocking the design and what it would take to make it viable.

Test and Control Market Selection

Paramark uses a statistical matching approach to pick test and control markets. The goal is straightforward: the two groups should look as similar as possible in the pre-period, so that any difference during the experiment window can be attributed to the channel rather than to the markets being inherently different.

The matching algorithm evaluates:

• Historical performance of your target metric

• Demographics and market characteristics

• Seasonal patterns

• External factors

and ultimately selects combinations where test and control track closely before the experiment starts.

Assignments are typically made at the state level, which provides a clean boundary for geo targeting on most ad platforms. DMA-level assignments are used for channels like linear TV where DMAs are the natural targeting unit.

When DMA-based designs make sense

A DMA-based design is usually chosen when the channel is planned, bought, or reported at the DMA level rather than the state level. This is common for linear TV and can also apply when a marketer's internal planning process, sales calendar, or media partner structures the campaign around DMAs.

The principle is the same as any other geo experiment: Paramark still needs treatment and control groups that match well in the pre-period. The difference is simply the geographic unit being assigned. In practice, DMA-based tests work best when:

• the buying platform or channel naturally operates in DMAs

• the target metric has enough volume at the DMA level to support matching and detection

• spillover risk is manageable, especially for broader-reach media

If state-level volume is already clean and operationally simple, state assignments are often preferred. But when the real-world execution unit is the DMA, designing the test around DMAs can make the setup more faithful to how the media actually runs.

Synthetic control

After the experiment runs, analysis uses a synthetic control: a weighted blend of control markets designed to mirror the historical behavior of the test markets as closely as possible. Rather than comparing test against a single average of all control markets, the synthetic control gives heavier weight to the specific markets that best match the test group in the pre-period.

This matters because it improves the precision of the comparison. The tighter the match in the pre-period, the more confidently we can attribute any difference in the test period to the channel being measured. Synthetic control methodology is well-established in causal inference research and is the standard for geo experiment analysis at Paramark.

Your analysis readout will include the weights used in the synthetic control, so you can see which markets contributed most to the counterfactual estimate.

Lift Required and Budget

How much you need to spend depends on two things: the size of the effect you want to detect (the lift required) and how much volume your target metric produces in each market.

How to think about Lift Required

The lift required is the smallest lift the experiment is designed to confidently observe. Anything less is natural variance. Common targets at Paramark sit in the 5-10% range, but the right lift for your test depends on a few factors:

• Historical conversion volume & scale by geography. Ultimately your experiment needs sufficient volume in both test and control geos to detect lift during the treatment period.

• Business decision threshold. If a 3% lift would change your budget decision, aim to detect 3%. If you only care about lifts of 10% or more, target 10%.

• Confidence and probability of detecting effect. We typically aim for a 80+% probability of detecting effect (if any), which drives the budget volume.

• Cost vs. precision. A smaller lift requires more spend or more time. Your GA will explain the tradeoffs so you can pick a point that's both detectable and affordable.

A good experiment design matches the lift required to a decision you'll actually make. Designing for a 2% MDE when you wouldn't reallocate budget unless the lift were over 8% is expensive and unnecessary.

Statistical confidence

Paramark typically designs for 80% statistical power at a standard significance threshold, which is the common convention in experimental marketing measurement. Some customers with high-stakes decisions design for 90% or higher. Your GA will confirm the target with you during the design conversation.

A B2B example: DMA-level tests using qualified leads

For B2B advertisers, a practical primary metric is often qualified leads rather than pipeline or revenue. Qualified leads usually respond faster to media changes, show up at higher volume, and are less distorted by sales-process noise. That makes them a better fit for geo-based incrementality testing.

DMA-level designs are especially useful when the channel is bought or planned at the DMA level, or when state-level grouping would blur how the campaign is actually executed. In those cases, Paramark may recommend building a test cell that represents roughly 30–40% of total target-metric volume, then using historical daily DMA data to choose treatment and control markets and simulate the detectable lift.

A typical design conversation for this kind of test covers four questions:

• Do we have enough historical DMA-level signal? Daily data on spend and the primary metric are needed to build and validate the design.

• Is the target metric stable enough? If weekday/weekend swings are large, smoothing techniques like a 7-day rolling average can help make the signal more usable.

• What lift is worth designing for? Many teams target something like a 7–10% detectable lift, depending on the business decision the test is meant to inform.

• What incremental budget would that require? Once the target metric and target lift are set, the design can translate that into required incremental volume and budget.

This is one reason GAs often encourage customers to align on the target metric and target lift together: the statistic is not just a reporting choice. It directly determines whether the test is feasible, affordable, and decision-useful.

If the required budget is too high

It's common for an initial design to come back with a budget that's larger than you want to commit. Several levers can bring it down:

• Loosen the lift required. Targeting a less lift will require less budget.

• Extend the duration. Additional weeks of data means more signal, which may reduce the weekly spend required.

• Reduce the cell sizes. Smaller test cells cost less but produce a less precise read. Your GA will show you what the revised design looks like.

• Choose a different channel or test type. If the math doesn't work at all, a platform conversion lift study or a different channel may be more feasible.

If you're weighing whether more budget can shorten the timeline, see Can I speed up an experiment by spending more?

What if the design was built before my channel setup changed?

That is a normal reason to revisit a design before launch. A geo experiment is powered against the channel setup that existed when the design was created, so meaningful changes in baseline spend, campaign structure, targeting, or search-term routing can change what the test can realistically detect.

Two checks matter most:

• Has BAU spend changed enough to affect the treatment size? If the original design assumed a much higher or lower baseline, the same incremental budget may now represent a different percentage change in the test markets. That does not automatically invalidate the design, but it can change the minimum detectable lift, the weekly budget recommendation, or the duration required.

• Has the channel mix been cleaned up enough to change the historical baseline? If brand and non-brand traffic are now mapped more cleanly, or the campaign structure materially changed, the older pre-period may no longer reflect how the channel behaves today. In that case, the safest next step is usually to let a short clean post-change baseline accumulate, then refresh the design using the updated operating conditions.

A practical rule of thumb: if the change is operationally minor, you can often keep the existing design and just sense-check the budget math. If the change affects how spend is distributed, how the channel targets, or how the target metric is generated, it's usually worth refreshing the design before launch so the read is tied to the version of the channel you are actually about to run.

What if the budget is larger than the design assumed?

This is one of the most common reasons to revisit a design before launch. If the original experiment was sized around one spend level and the planned investment grows meaningfully afterward, the main question is not just whether the experiment is still possible. It is whether the experiment still answers the same business question cleanly.

Three checks usually matter most:

• Did the size of the treatment change materially? If the channel budget is now much larger than what the design assumed, the incremental effect being created in the test markets may also be larger. That can change the appropriate test structure, the budget split, or the duration required.

• Are multiple channels now being bundled into one decision? If the original design was for one channel or one clean holdout, and the updated plan now includes meaningful CTV, audio, out-of-home, paid social, or search changes at the same time, you may no longer be testing a single-variable question. In that case, the cleanest next step is often to narrow the question or redesign the test as a multi-cell or integrated-channel experiment.

• Will the target metric still move in the same timeframe? Adding more top-of-funnel budget can change not only spend levels but also the expected timing of impact. That may affect how long the test should run and how much cooldown is needed before analysis.

A good rule: if the post-design budget increase is large enough that you would describe the launch plan differently in a stakeholder meeting, it is usually large enough to sanity-check or refresh the design with Paramark before launch.

For example, if a holdout was initially designed around a modest media plan and the final launch now includes substantially more CTV, podcast, out-of-home, or additional digital support, do not assume the original budget guidance or detectable-lift threshold transfers unchanged. The safest move is to share the revised spend plan and ask whether the existing design still reflects the version of the campaign you are actually about to run.

Special considerations for Google Performance Max and other cross-country geo tests

Performance Max and similar campaign types can be good candidates for geo experimentation, but they usually require more setup discipline than a standard state-based search test. The design still relies on the same principles — clean test and control separation, stable campaign execution, and a readable target metric — but a few extra constraints often matter in practice.

• Country boundaries need to be explicit. If the campaign is serving across multiple countries, Paramark will usually define the eligible test geography first and exclude non-test countries or other overlapping regions from the experimental setup. The goal is to prevent spend from leaking into geographies that are outside the design and muddying the read.

• The target metric needs daily geography-level history. For a state-based design, we typically need daily state-level outcome data for a meaningful historical window before the test starts. Without that, it is much harder to build a strong synthetic control or run a credible power analysis.

• Campaign structure affects whether the design stays interpretable. If Performance Max is bundled together with other initiatives, shares budgets across too many objectives, or is being actively restructured, the cleanest next step is often to stabilize the setup before launch or refresh the design once the new structure has enough baseline history behind it.

• Budget guidance should reflect the actual version of the campaign you plan to run. Because campaign automation can change delivery patterns quickly, the recommended test budget and duration should be tied to the current operating setup, not to an older version of the campaign that has since changed materially.

In short: for channels like Performance Max, the right question is not just whether a geo test is possible. It is whether the geographic boundaries, target-metric data, and campaign structure are clean enough that the final result will be decision-useful. If those pieces are in place, Paramark can usually design the test the same way it would any other geo-based incrementality experiment.

How to read a power curve

Your design comes with a power curve: a visual representation of the results of our test cell selection. In practice it's a chart that shows, for each possible size of effect, how likely the experiment is to detect it.

The x-axis represents the percentage lift simulated. The y-axis represents the experiment's statistical power at that lift: the probability of detecting an effect of that size if it really exists (loosely confidence).

A practical way to read the curve:

• Find the horizontal line at 80% power (the conventional threshold for a well-powered experiment).

• Trace it across until it intersects the curve.

• Drop down to the x-axis. That value is your lift required for this design.

So if the 80% line crosses the curve at 6%, your design is powered to detect a 6% lift or larger. Smaller lifts than that may still be real but would be harder to distinguish from noise.

Comparing two designs

When your GA shows multiple design options (for instance, "option A has a larger budget but a lower lift required; option B is cheaper but only detects larger effects"), compare the power curves. The design whose curve rises fastest and crosses the 80% threshold at the lower x-axis value is the more sensitive design, but it will cost more or take longer. Pick the design whose lift required and budget matches the decision you want to make.

A note on centering

Occasionally a power curve is not perfectly centered at zero on the x-axis. This reflects minor bias in the underlying time series model and is usually not a concern for small offsets.

Duration

Paramark will recommend a minimum duration for your experiment. The duration of your experiment is set to give the test enough time to produce a confident result, without running longer than necessary. It's determined by the same modeling process that produces the budget: a longer window generates more signal, so shorter tests require larger spend differentials and longer tests can work with smaller ones.

Typical durations:

• Bottom-funnel channels (paid search, performance social): 4-6 weeks

• Mid-funnel channels (programmatic display, paid social): 6-8 weeks

• Top-funnel channels (CTV, linear TV, audio): 8-12 weeks

Paramark aims to balance minimizing the required time to learn with how confident our results will be.

What to expect from an OTT / CTV experiment design

OTT / CTV tests usually sit on the slower, more operationally sensitive end of the spectrum. Because they are top-funnel channels, they often need more time in market before the target metric moves enough to read cleanly. That is why OTT designs often come with longer recommended durations and more attention to cooldown than lower-funnel paid search or paid social tests.

They are also more likely to use DMA-based market assignment rather than state-based assignment, because OTT buying and reporting often happen at the DMA level. When that is true, the most important question is not whether the design looks different from a search or social test — it should. The right question is whether the geographic unit in the design matches how the media will actually be bought and held stable during the test.

Finally, OTT tests require extra care around spillover and mid-flight changes. Broad-reach media can leak across neighboring markets more easily than tightly geo-targeted digital channels, and creative, flighting, or budget changes made mid-test can make the result harder to interpret. If your OTT setup changed materially after the design was produced, ask your GA to sanity-check whether the design still reflects the version of the campaign you are about to run.

At the start of the experiment, your campaigns will go through a short ramp-up or learning period (typically 3-7 days for digital channels, longer for newly launched campaigns or large creative or budget shifts) while the ad platforms calibrate delivery and bidding. Performance during this window doesn't reflect steady-state delivery and is included in the test window by default. To minimize its impact, we recommend pre-staging campaigns where possible, launching on a Monday morning rather than mid-week, and avoiding launches the day before major holidays, pricing changes, or other business events that could distort early signal.

After the experiment window ends, a short cooldown period (typically 1-2 weeks) allows lagged conversions from the test period to land before analysis. Your final readout comes 1-2 weeks after cooldown ends.

Experiment instructions

Paramark will provide instructions on how to execute the experiment that ensure we are maintaining consistent spend in non-test states and the appropriate treatment in the test states.

What Paramark does vs. what you do

Experiment design is a partnership. Here's the division of responsibility:

During the experiment

Once the experiment launches, the most important thing is consistency. The test and control markets need to run as designed for the full duration, and anything that unexpectedly changes conditions in either group can compromise the read.

A few guardrails:

• Don't change the campaigns being tested. No major creative swaps, targeting changes, or bid adjustments mid-flight beyond what was agreed in the design.

• Keep control markets stable. This is the baseline. Spend changes, new campaigns in related channels, or anything that could affect the target metric in control markets can contaminate the comparison.

• Flag unexpected events. If something happens in the business during the test (a PR event, a product launch, a macro disruption), let your GA know so it can be factored into the analysis.

If you have extra budget arrive during the experiment or want to redeploy the budget you're holding back from a holdout, the specific options are covered in Managing budget during a live experiment.

Reading the results

After cooldown, your GA delivers an analysis readout. It includes:

• The estimated lift (percentage and absolute)

• The confidence level you should have in the result

• A recommended action

The readout is designed to answer one question: did this channel drive incremental lift at the scale tested?

Three common outcomes:

• Significant positive lift. The channel is incremental at the spend level tested. Next step is usually to discuss whether scaling further is warranted and whether a follow-up test at a different spend level makes sense.

• No detectable lift. The channel did not drive measurable incremental value at the tested spend level during this window. This is a real answer, not a failure, and it usually reshapes budget decisions. See What a no lift result means for a fuller discussion.

• Inconclusive. The observed results did not exhibit a consistent pattern against the controls, or the controls do not fit well enough to draw conclusive results.

Common tradeoffs and limitations

Geo experiments are powerful, but a few limitations are worth naming up front.

Spillover between markets. Some ad exposures cross market boundaries (a user in a control state sees an ad while traveling, or a brand campaign running nationally reaches both groups). For digital channels this is usually small enough to be absorbed by the design. For CTV, linear TV, audio, and out-of-home, we handle it by buffering market assignments or using DMAs.

Affordable Lift Required versus useful Lift Required. Sometimes the MDE the customer's business requires (say, detecting a 10% lift) isn't affordable at the customer's current spend. In those cases we work with you to choose between a larger MDE target (detectable at a lower cost) and a different test structure altogether.

When geo isn't the right tool. Organic channels, channels without reliable geo targeting, very small channels, and customers operating in too few markets may need a different measurement approach. Platform-native conversion lift studies, time-based holdouts, and MMM all have a role alongside geo experiments.

Channels that influence each other. If two channels target overlapping audiences and one is being tested, the other is usually held stable to avoid confounding. Your GA will flag when this matters for your specific design.

A simple worked example

To make the pieces concrete, here's a stylized example of how a design comes together.

A B2B software customer wants to know whether LinkedIn paid is driving incremental pipeline. They spend roughly $180k/month on LinkedIn across the US, and the business has meaningful pipeline volume in 28 states.

• Hypothesis: LinkedIn paid is driving pipeline, and we want to know whether increasing spend by 50% produces more pipeline or whether it's oversaturated.

• Test type: Pulse up (increase LinkedIn spend by 50% in test markets, keep flat in control markets).

• Target metric: Qualified pipeline generated.

• Pre-period matching: 12 months of state-level pipeline data. Algorithm assigns 14 states to test, 14 to control, with balance statistics showing test and control tracking within 4% in the pre-period.

• Budget: Additional spend required in the test cell is $135k over the 8-week test window, producing a design powered to detect a 7% lift in pipeline at 80% statistical power.

• Duration: 8 weeks, with a 2-week cooldown for lagging pipeline conversions.

• Power curve: Crosses 80% at 7% lift, reaches 95% at 12% lift.

• Instructions: Duplicate the LinkedIn campaign, assign location targeting to the 14 test states, raise the budget by 50% in the duplicated campaign, maintain original campaign spend in control states.

At the end of 10 weeks (8-week test + 2-week cooldown), the synthetic control analysis produces a readout showing the pipeline lift, confidence interval, and decision recommendation.

Want to learn more about experimentation? Check out other Paramark experimentation articles here