When Paramark designs a geo experiment, one of the first things customers want to understand is how their markets were divided into test and control groups. It can look like an arbitrary list of states — but the assignment is the result of a statistical matching process designed to make the experiment as clean and readable as possible.

Why market selection matters

The goal of a geo experiment is to measure what happens in test markets compared to what would have happened if the channel hadn't run (or had run at a different spend level). For that comparison to be valid, the test and control groups need to look as similar as possible before the experiment starts.

If test markets are systematically different from control markets — different seasonal patterns, different demographic composition, different baseline conversion rates — then any difference observed during the experiment is hard to attribute to the channel alone. It could just be that the markets were different to begin with.

Market selection is how we prevent that problem.

How the matching works

Paramark's market assignment algorithm looks at the historical behavior of your target metric — the outcome you're measuring, like signups or purchases — as well as demographics and market characteristics, seasonal patterns and external factors across all eligible geographies. It identifies which combination of control markets most closely mirrors the trajectory of the test markets in the period before the experiment.

The output is a set of test and control groups where both groups were moving in parallel historically. If they were moving in parallel before, any divergence during the experiment window can be attributed to the channel being tested rather than underlying market differences.

Markets are typically assigned at the state level, which provides a clean boundary for geographic ad targeting on most platforms. DMA-level assignments are sometimes used for channels like linear TV where DMAs are the natural targeting unit.

What to expect from your experiment design

Your design will include balance statistics showing how similar the test and control groups are in the pre-period. The closer the two groups track historically, the cleaner the experiment read will be.

If you have specific geographic constraints — markets where you don't operate, regions with regulatory restrictions, or areas where you're running conflicting campaigns — share those with your Growth Advisor before the design is finalized. Those constraints can usually be incorporated, though narrower constraints mean fewer eligible markets, which may require a larger budget to achieve the same statistical confidence.

Common questions

Why is my biggest market in control rather than test?

Larger markets are sometimes more valuable as control markets because their size makes them better statistical anchors. Placement depends on the matching algorithm's output, not market size alone. If you want to understand the specific assignment, your Growth Advisor can walk you through the balance statistics.

Can we adjust the market list?

Yes. If you have specific market constraints, ideally you flag before the experiment design; but we ultimately need to create an experiment design that is mathematically strong and yet makes sense for your business. The Growth Advisor will show you the balance statistics before and after the change so you can see if it meaningfully affects the design quality.

Can I choose my own holdout regions or business-defined market groups?

Sometimes, yes — but only if those regions still produce a credible experiment. Customers often come in with a business-driven proposal such as holding out a specific region, combining neighboring states, or testing a region that maps to internal sales ownership. Those constraints can absolutely be considered, but they are not automatically the statistically best assignment.

When we evaluate a customer-defined holdout, we check three things: whether the platform can target the proposed regions cleanly, whether the target metric has enough volume in those regions, and whether the remaining control markets still create a strong pre-period match. If any of those break, we may recommend revising the regional split, changing the geographic unit, or redesigning the test around a different set of markets.

The practical rule is simple: business preferences can shape the design, but they cannot come at the expense of a clean separation between test and control or a readable final result.

What if I want to minimize the number of holdout states or DMAs but still reach statistical significance?

There is no fixed minimum like "3 states" or "5 DMAs" that guarantees significance across every test. The right holdout size depends on the volume of your primary metric in those geographies, how noisy that metric is historically, the channel being tested, and the size of the effect you need to detect.

In practice, Paramark usually starts by sizing the test cell to represent a meaningful share of the target metric rather than aiming for a specific count of markets. For many state-level geo tests, that often means a test cell representing roughly 10-40% of national target metric volume, provided the remaining markets still create a strong control group. A smaller number of large states can sometimes work better than a larger number of small states, and the reverse is also true. What matters is detectable signal, not the raw count of markets.

When customers prefer fewer markets, we evaluate several candidate state or DMA groupings and compare the tradeoffs: required lift, control quality, operational feasibility, and expected budget. If the smaller holdout is too underpowered, we will usually recommend one of four changes: add more markets, extend the test duration, increase the treatment size, or switch to a different measurement approach.

How do you decide which specific states or DMAs go into the holdout?

We do not pick holdout markets just by size or intuition. We evaluate historical target-metric behavior, look for combinations that can be matched well by the remaining controls, and account for business constraints such as geo targeting limitations, conflicting campaigns, or regulatory exclusions. State-level designs are generally preferred on most digital platforms, while DMA-level designs are more common when the channel itself is planned and bought at the DMA level, such as linear TV.

The practical takeaway is: if you want the fewest possible holdout markets, the answer is not a universal market count. It is the smallest grouping that still gives you a credible control match and a realistic chance of detecting the lift you care about.

What if we need to exclude markets after the first design is drafted?

That is a normal reason to revise a geo experiment before launch. Customers sometimes discover late-breaking constraints such as overlapping brand campaigns, out-of-home placements, platform targeting limitations, regulatory concerns, or operational issues in specific states or DMAs. A common example is realizing that a planned holdout overlaps with brand activity like billboards or other awareness media that would contaminate the comparison. When that happens, Paramark can rerun the design with those markets removed from eligibility.

The tradeoff is that every exclusion reduces the pool of usable controls or test markets, so the revised design may require a different market mix, a larger budget, a longer test, or a lower-powered read. The right next step is usually to compare the original and revised balance statistics, then decide whether the updated design is still strong enough to run.

What if our target markets have weak latent demand or very limited baseline volume?

This is one of the most common reasons a geo design looks technically possible on paper but still is not the right launch plan yet. Geo experiments work best when the selected test markets already have enough underlying demand and measurement volume for a spend change to produce a readable signal. If the baseline is too thin, the test may require either a much larger spend delta, a longer runtime, or a broader market set than the team initially expects.

In practice, this means market selection is not only about pre-period matching. It is also about whether the chosen markets give the experiment a realistic chance to detect lift. A market can be a clean statistical match and still be a poor test candidate if there is not enough target-metric volume to measure movement credibly.

When latent demand looks weak, Paramark will usually evaluate a few alternatives: shifting the test toward stronger-demand markets, expanding the eligible geo set, extending the duration, increasing the treatment intensity, or postponing the launch until the channel or metric has more scale. The right answer depends on the tradeoff between confidence, budget, and operational complexity — but the principle is consistent: prioritize a design that can actually generate signal, not just one that looks neat geographically.

Want to learn more about experimentation? Check out other Paramark experimentation articles here