GTM experiments

Start with one uncertainty and one decision

Write the decision the test must support: pursue a segment, change a message, offer onboarding help, or use a channel. Use interviews or a manual pilot to discover objections and delivery problems. To estimate whether a change causes better results, use a randomized comparison when enough accounts are available.

• State audience, treatment, comparison, proposed mechanism, and business action.
• Changing segment, channel, and offer together tests the package, not each component.
• Choose the stable assignment unit. Account-level assignment often matters when contacts share a workspace or buying process.
• Document how subsidiaries, repeated contacts, and concurrent campaigns are handled.

Define how the experiment will be measured

Define which accounts qualify, when they enter, what outcome counts, and how long each account is observed. Compare all eligible accounts as originally assigned. Track which accounts actually received the treatment and investigate missing or uneven delivery.

• Choose one main outcome and limits for customer complaints, unwanted contact, and staff effort.
• Define activation as a specific success event, such as completing the first real workflow. State the event and the time allowed to reach it.
• Choose the effect worth detecting, sample and duration, outcome-maturation time, and analysis rule.
• Set harm stops in advance. Do not stop when a favorable number appears; use a valid sequential method if ongoing decisions are required.

Assign the test at the buying-account level

When contacts share a buying decision or product workspace, keep them in the same treatment group. Otherwise one account can receive both messages, offers, or onboarding experiences. Company-level assignment is a documented option for B2B experiments; choose it because the treatment can spill across contacts, not simply because more IDs are available.

• Choose a stable buying-account ID and document parent, subsidiary, and workspace treatment.
• Assign once before outreach or exposure. Keep later contacts attached to the account’s original group.
• Record assigned, contacted or exposed, and outcome-eligible accounts separately. Investigate delivery or tracking failures without quietly removing those accounts from the assigned population.

Source: Statsig: company-level experiment assignment

Account for small samples and large accounts

In B2B, a few large accounts can dominate a revenue measure. Check important pre-test differences such as segment, company size, source, and prior activity before launch. Where needed, balance groups using a planned stratification or blocking approach and use analysis consistent with that design.

Do not rebalance the groups after seeing results. A diagnostic pilot can uncover objections or delivery issues without establishing a causal revenue lift.

• Choose the effect worth detecting and an attainable observation period before launch.
• Use a mature outcome window. Accounts assigned later need the same time to convert.
• If the number of independent buying accounts is too small, use interviews or a manual pilot to refine the proposition, and keep the effectiveness claim open.

Source: Statsig: balancing low-volume B2B experiments

Measure the business transition being tested

For an outbound message test, a positive reply is an early response. An accepted qualified opportunity is a later business transition. Choose the primary outcome for the decision and retain the intermediate counts so you can see where the difference arose.

• Show the numerator, denominator, and observation window beside each rate.
• Report absolute percentage-point differences as well as any relative change. A zero baseline does not support an ordinary relative percentage lift.
• Check assignment balance, duplicate IDs, missing exposure events, and concurrent campaign changes before interpreting a result.

Keep the experiment record small

Keep the hypothesis, assignment, measurement plan, results, and decision in one experiment record.

• Hypothesis and intended business decision.
• Audience, assignment unit, exclusions, allocation, and the difference between treatments.
• Primary measure, guardrails, follow-up window, sample, dates, analysis plan, and effort budget.
• Owner, observed counts and rates, uncertainty, telemetry issues, concurrent changes, and conclusion.
• If the sample is too small or tracking failed, report the result as inconclusive. Fix the measurement problem or collect more comparable observations before deciding.

Avoid attractive but unsupported conclusions

A before-and-after comparison may reflect seasonality, account mix, or extra staff effort. Excluding assigned accounts that never received the treatment can also bias the comparison. Report activation results first; measure later revenue before claiming a revenue effect.

• Report activation against all assigned eligible accounts under the agreed analysis rule.
• Show contact-harm counts and rates, along with human minutes per assigned account.
• Keep delayed outcomes and incomplete follow-up visible.
• State the observed counts, the uncertainty in the estimate, and whether to stop, revise, or expand the test.

Record the decision without overstating the result

Keep the original hypothesis, assigned accounts, intervention, dates, observed counts, uncertainty, cost, and decision in one record. Later pipeline and revenue can be appended as they mature.