Once an experiment is running, budget questions come up fast. You're holding back spend in control markets and wondering if that money has to sit idle. Or an extra budget allocation lands and you need somewhere to put it without wrecking the test you've worked hard to set up. These are good questions, and the answers are specific.

The core principle behind all of them is the same: an experiment works by comparing what happens in test markets to what happens in control markets.

Anything that changes the conditions in either set of markets in a way that wasn't planned can compromise the read. The goal is to find ways to use budget that genuinely don't affect those conditions.

If you're running a holdout: can the held-back budget go somewhere else?

Yes. The budget you've held back in control markets doesn't need to sit idle for the duration of the experiment. But where it goes matters.

• The safest option is a channel or campaign with no geographic overlap with your experiment. If your holdout is pausing Google Search in a specific set of states, putting the held-back budget into a national campaign, or into a channel that doesn't use geographic targeting, is clean. The experiment is measuring one channel in specific markets. Spending elsewhere doesn't affect that measurement.

• Another option is a multi-cell experiment. If you have sufficient markets and target metric volume, a multi-cell experiment with a holdout, pulse up and control could be a safe option that avoids contaminating the hold out experiment.

• If you need to keep the spend in the same channel, the only valid approach is to increase spend in both test and control markets proportionally at the same time. This keeps the ratio between cells intact. It’s not perfect as the experiment no longer measures the incrementality at the desired spend levels, but instead the incrementality at the new higher spend levels. It’s not the same learning, but if there’s a tradeoff that must be made, this is the path to consider. That said, your Growth Advisor should confirm this before any changes are made.

• What not to do: Put the held-back budget back into the control markets only, or into a campaign that targets those same control markets in a related channel. Either of these changes the conditions in your control group, which is what the experiment uses as its baseline. Once the control group is contaminated, the result becomes unreliable and usually unrecoverable.

If extra budget arrives mid-experiment: where can it go?

The same logic applies. The question to ask is whether the new spend could affect your target metric in the markets being measured.

• Safest options:

  • a channel or campaign that doesn't touch your experiment's markets. If the new budget goes into geographies or targeting segments that have nothing to do with your test or control states, the experiment is unaffected.

  • a multi-cell experiment. If a a multi-cell experiment is viable, this controls for both a holdout, control group and pulse up to get a clean read on the impact of the holdout (original experiment) while also adding a pulse up experiment as part of the experiment.

• Workable with major caveats:

  • the same channel, with a proportional increase to both cells. If the new budget needs to go into the channel you're already testing, it can — but only if the increase is applied to both test and control markets at the same rate, and only after aligning with your Growth Advisor on the new spend targets for each cell. If you increase spend at different rates across cells, or only in one cell, the read will reflect both the channel's incrementality and the unplanned spend change mixed together, and those two things can't be separated in analysis. Again this is not the ideal state and should only be pursued if there’s no avoiding this and/or the business risk outweighs clean experiment learnings.

• Not safe:

  • adding spend in a channel that reaches the same audiences in the same markets. If you're holding out YouTube in certain states and the new budget would go into a CTV campaign targeting those same states, you've introduced a correlated signal into your measurement. Both channels are upper-funnel video reaching similar audiences. The control markets are supposed to reflect what would have happened without YouTube. Adding CTV spend there breaks that assumption.

A simple test: does the channel you're considering reach the same people, in the same places, in a way that could move the same metric? If yes to all three, it's a contamination risk.

Can you offset a holdout by adding spend to test markets?

Not within a standard two-cell experiment. If you cut spend in control and add spend in test, you no longer have a control group. You have two different treatment conditions, and there's nothing to compare them against.

If you want to run a holdout in some markets and increase spend in others at the same time, that's a three-cell design: a holdout cell, a pulse-up cell, and a business-as-usual control cell. Paramark can design this, but it requires more markets and higher target metric volume to produce confident results in all three cells. It's worth planning at the start of an experiment cycle rather than trying to retrofit it after a test has already launched.

What if I want to pulse down Google non-brand search and pulse up Demand Gen at the same time?

This is the same three-cell logic in a more specific form. If you reduce one Google program while increasing another, you are no longer asking a simple yes-or-no question about one channel. You are testing a budget reallocation decision across two channel conditions at once.

That can be a valid experiment, but it should usually be designed upfront as a multi-cell geo test with: (1) a holdout or reduced-spend cell for the channel you want to pulse down, (2) a reallocation cell where the source channel is reduced and the destination channel is increased, and (3) a business-as-usual control cell. Without that control, it becomes hard to separate whether the result came from cutting the first channel, adding the second channel, or both together.

The practical takeaway: if the business question is "should we move budget from Google non-brand search into Demand Gen?" bring that exact reallocation question to your Growth Advisor before launch so the design, market split, and spend levels are set to answer it cleanly.

If the experiment ends but the channel keeps running: should you keep the blackout states in place?

Usually, no. Once the planned experiment window is over, the blackout states have done their job. If you keep them in place while continuing the rest of the campaign nationally, you're no longer preserving a clean experimental control — you're just withholding delivery in a subset of markets and creating avoidable business risk.

The better default is to return those states to normal delivery once the test window ends, unless your Growth Advisor has explicitly designed a longer holdout, an extended read window, or a follow-on experiment.

• Keep the blackout only if the experiment is still actively running. That includes the agreed test period and any setup your Growth Advisor told you was required to preserve the read.

• Remove the blackout after the experiment ends if the goal is business-as-usual continuation. At that point, resuming delivery across the held-out states is typically the right operational move.

• If you want to keep learning while continuing flights, don't improvise. Ask your Growth Advisor whether the right next step is an extension, a fresh geo design, or a different follow-on test. Continuing a partial blackout without a defined design usually doesn't produce a clean new read.

A simple rule: blackout states are a test construct, not a long-term media strategy. Keep them only for as long as the experiment design requires.

What if extra budget also changes what you want to learn about a channel?

Sometimes the budget increase isn't just about spending more safely during a live test. It also creates a new learning question — for example, when you now want to run a pulse-up on a channel that was tested previously, or when the channel has changed enough that the old result may no longer be the right decision input.

In that case, don't assume the prior result fully transfers. If the spending level, format mix, audience strategy, or campaign objective has materially changed, the cleanest next step is usually to design a fresh experiment around the new decision. A past result is still useful context, but it may not answer the current question.

• Good candidates for a fresh pulse-up test: you've added a meaningful amount of new budget, you're reconsidering a channel's scaling headroom, or the channel setup has changed in a way that could affect performance.

• Examples of meaningful changes: a different mix of placements or ad formats, a different audience strategy, a different optimization goal, or a new campaign structure compared with the earlier test.

• What to avoid: treating a prior experiment as a blanket green light for a new budget decision when the operating conditions are no longer similar enough.

The practical takeaway: if the business question has shifted from can we protect the current experiment? to should we scale this channel under today's setup?, ask your Growth Advisor whether the right answer is a fresh pulse-up design rather than an in-flight spend change.

What if the channel you planned to test doesn't launch on time?

If the experiment has not started yet and the launch channel slips, treat that as a design change rather than an in-flight budget tweak. The safest move is to pause and re-confirm the setup with your Growth Advisor before reallocating the planned budget.

• If the delayed channel is the one being tested, don't assume you can simply move that planned spend into a similar channel and preserve the same learning. A Roku CTV launch slipping and shifting that budget into existing CTV partners, for example, changes the delivery mix, inventory, and likely audience exposure. That is a different test condition.

• If the timing shift is driven by creative changes, trafficking delays, or campaign build work, the same rule applies. A one-week push to finish creative or finalize setup may feel operationally small, but it still changes the actual launch window, the media conditions around the test, and sometimes the post-test window (for example, pushing the flight dates out a week and moving the post-test week with it). Confirm the revised start date, end date, and any cooldown or post-test window with your Growth Advisor before treating the experiment as unchanged.

• If you still want to learn about the original channel, the right answer is usually to move the launch window and re-run the design assumptions against the new timing.

• If the business needs the budget deployed anyway, you can often redirect it into business-as-usual channels — but that should be treated as an operational decision, not as a substitute for the planned experiment on the delayed channel.

• What to avoid: treating a delayed launch plus a fallback reallocation as if it answers the same measurement question. If the channel, inventory source, market exposure, or timing window changes, the experiment question has changed too.

A simple rule: if the test has not started, protect the learning plan before protecting the calendar. A short delay is usually easier to recover from than launching the wrong design and getting an answer you can't trust.

What if you want to run two related experiments in the same window?

Usually, don't — at least not if both tests could move the same outcome in the same markets at the same time.

If you're planning a new audio test while a CTV test is also launching, or vice versa, the default should be to separate them in time unless the design was explicitly built to measure both together. Even if they are different channels, they can still influence the same upper-funnel metric and the same downstream outcomes. Once both are changing at once, it becomes much harder to tell whether the observed lift came from audio, CTV, or the combination.

• Safest option: run the tests sequentially. Finish the first experiment, let the read come back, then launch the second test in a new window.

• Possible but more complex: design a multi-cell test upfront if the real business question is about two channels or two launch decisions at once. That requires enough markets, enough target metric volume, and a design that preserves a true business-as-usual comparison cell.

• What to avoid: improvising two overlapping tests because the calendar is tight. If both channels reach overlapping audiences or are expected to move the same KPI, overlapping launches usually create attribution ambiguity rather than better learning.

A simple rule: if you would struggle to explain which channel caused the result before the test starts, the design probably needs to be simplified before launch.

The practical rule

Before making any spend change while an experiment is live, check with your Growth Advisor first. The question is always the same: does this change affect the target metric in the test or control markets? If you're not sure, the answer is to ask before acting. A spend change that contaminates an experiment can't always be corrected in analysis, and it may mean the full test window needs to be run again.

What if I want to run a one-variable pulse-down test on Google non-brand search, but I'm worried about business risk?

That is usually the cleaner design. If your main question is whether reducing Google non-brand search hurts the business, test that variable on its own first rather than combining it with a simultaneous pulse-up in another channel. A single-variable holdout or pulse-down gives you a much clearer read on what the search reduction actually causes.

If the business risk feels meaningful, build the risk-management plan into the experiment before launch:

• Pre-stage the treatment campaigns first. If you already know which treatment geos you'll use, launch those campaigns at baseline spend before the test starts so they can move through platform learning. Then begin the pulse-down once delivery is stable.

• Define the kill switch in advance. Don't wait until performance feels uncomfortable mid-test. Before launch, agree on the exact metrics, thresholds, and observation window that would trigger a stop. The threshold should reflect meaningful business protection, not normal week-to-week noise.

• Use scheduled checkpoints carefully. It's reasonable to pre-plan one or two operational check-ins during the test to confirm campaigns are pacing, geo targeting is correct, and no severe business issue is emerging. But treat those as guardrail reviews, not as opportunities to re-interpret the lift result early. If you want to make a stop/go decision at a checkpoint, define the rule before launch.

• Keep the rest of the setup stable. Once the pulse-down begins, avoid changing other major variables in the test or control markets. Otherwise you may protect the short-term business risk but lose the ability to trust the result.

Use guardrail inputs that reflect the business risk you actually care about. Before launch, identify the operational metric you would watch if the pulse-down went too far — for example started-onboarding volume, qualified leads, applications, or another early conversion signal. Bring that metric at the geo level into the setup conversation so the kill switch thresholds are tied to a real business guardrail rather than a vague sense that performance feels off.

The practical takeaway: if the first question is "how much downside do we risk by reducing non-brand search?" start with the one-variable pulse-down, pre-stage the treatment campaigns, define the kill switch and checkpoints upfront, and align on the guardrail metric you would use to stop the test if needed. If that test comes back clean, a second experiment can then ask where reallocated budget should go.

Want to learn more about experimentation? Check out other Paramark experimentation articles here