By Mehdi HassanWhat is an AB test hypothesis?
An AB test hypothesis is a written prediction of what a specific change will do to a specific metric, and why. It's the reason a test exists. It turns "let's try a new headline" into a claim you can prove or disprove, and it tells everyone in advance what result would count as success.
The "why" is what separates a useful hypothesis from a guess. If you can't point to something you saw, such as a recording, a support ticket or a pattern in sales calls, you're testing a hunch, and a hunch that wins teaches you very little. This template is part of our guide to A/B testing landing pages, and it plugs into every other step there.
What is the A/B test hypothesis template?
The template has six parts, and the first one, the observation, is required. Fill it in before anyone designs or builds a variant.
Part | Prompt | What good looks like |
|---|---|---|
Observation | Because we observed... | A specific finding with its source and date |
Change | We believe that changing... | One clear change, described precisely enough to build |
Audience | For... | Who sees the test: page, device, traffic source |
Prediction | Will cause... | Direction and the minimum lift worth detecting |
Measurement | Measured by... | One primary metric, plus guardrails, and the planned sample |
Decision | If it wins / loses / is flat, we will... | The action for each outcome, agreed in advance |
Copy-ready hypothesis template
Because we observed [specific evidence, with source and date],
we believe that [one change to the page]
for [audience: page, device, traffic source]
will cause [primary metric] to [increase or decrease] by at least [minimum detectable effect].
We'll measure it with [primary metric], check [guardrail metrics] for harm, and run until [planned sample per variant], about [number of weeks].
If it wins, we will [ship and next step]. If it loses, we will [learning and next step]. If it's flat, we will [next step].
How do you write the "because we observed" part?
Write the observation as a specific, sourced finding: what you saw, where and when. It should be something another person could check. "Visitors are confused" isn't an observation; "in 20 recordings from mobile paid traffic last week, most visitors scrolled past the hero without stopping" is.
- Analytics: drop-off at a step, low click-through on a CTA, a device or traffic source that converts far worse.
- Session recordings and heatmaps: hesitation, rage clicks, ignored sections, scrolling past key content.
- Sales and support: questions that come up again and again, objections raised on calls.
- User research: interview quotes, five-second test answers, survey responses.
- Audit findings: issues from a structured review. Our CRO audit process is a good source of evidence to start from.
What do good test hypothesis examples look like?
Good test hypothesis examples are specific enough that a designer could build the variant and an analyst could judge the result without asking a single question. The examples below are hypothetical, written to show the shape, not real results.
Headline (message problem)
Because five-second tests showed most participants couldn't say what the product does, we believe replacing the category headline with an outcome headline naming the user, for all desktop visitors, will increase trial starts by at least 20%, measured over the planned sample.
Offer (path problem)
Because sales notes show small teams asking for a trial on demo calls, we believe making "Start free" the primary CTA for visitors from companies under 50 people will increase activated sign-ups by at least 25%, with demo requests from larger companies as a guardrail.
Proof (trust problem)
Because recordings show visitors pausing at the form and scrolling back up to the logos, we believe adding one customer quote from a similar company beside the form will increase form submissions by at least 15%.
Pricing (clarity problem)
Because support tickets repeatedly ask what counts as a seat, we believe adding a three-question billing FAQ under the plans will increase plan selections by at least 15%, with refund requests as a guardrail.
Notice that each one names the evidence, one change, the audience and a minimum lift. The minimum lift matters because it sets the sample size: our guide to A/B test sample size shows how a 15% target needs far more traffic than a 25% one. For more starting points, our ranked A/B test ideas are grouped by the kind of problem they solve.
How do you know if a CRO hypothesis is strong?
A CRO hypothesis is strong when it's grounded in evidence, isolates one change, predicts a measurable result and leads to a decision whatever happens. Run every draft through the checklist below before it goes into the testing backlog.
- Evidence: does it start with a specific, sourced observation?
- One change: could the result be explained by anything other than the change you named?
- Mechanism: does the "why" make sense? Would the change plausibly fix the problem you observed?
- Measurable prediction: is there a direction, a primary metric and a minimum lift?
- Feasible sample: can you reach the required sample in a few weeks?
- Decision ready: have you written what you'll do for a win, a loss and a flat result?
How should you prioritize hypotheses in a backlog?
Prioritize hypotheses by the strength of the evidence, the plausible size of the effect and the cost to build and run them. A simple score on those three, reviewed every couple of weeks, keeps the backlog honest and stops the loudest opinion from winning.
Pros
- Scoring makes trade-offs visible and keeps decisions grounded in evidence.
- Evidence strength rewards research, so the team keeps doing it.
- A shared backlog shows stakeholders why their idea is or isn't next.
Cons
- Scores are still judgement calls, and people can game them.
- Cheap, low-impact tests can crowd out bigger, harder ones if effort is weighted too heavily.
- A backlog needs an owner, or it turns into a wish list.
Many of the strongest hypotheses come from the message itself. If research shows visitors don't understand the offer, the fix often starts in the value proposition or the messaging framework before it reaches the page. And once a hypothesis is approved, our guide to A/B testing mistakes covers the traps that can still ruin a well-designed test.
