Experiments
An experiment tests one thing against another with real traffic and tells you which one makes more money. You test a single page slot inside a funnel — the paywall, the welcome screen, an upsell — or a whole funnel against another, split live visitors between the arms, and let the platform call a winner by ARPU: revenue per visitor.
Two surfaces do the work, and they don’t overlap:
- The editor is where you author the alternative. A variant is a real page you write.
- The Experiments dashboard is where you run and read the test — every experiment in the project, its plain-English verdict, the lift, and the one button that makes the winner live.
A variant is a whole page
When you test a page, you author a genuine sibling page. pages/paywall@b.tsx is a variant of the paywall slot — the @ in the filename is what marks it. It is a real, editable, previewable page, exactly like paywall.tsx. You open it in the editor, change the headline, swap the layout, rewrite the offer — whatever you want the alternative to be.
There is no per-field magic. A variant isn’t a set of overrides layered on top of the original, and there’s no experimentValue() helper to sprinkle through your code. The unit of a test is a whole page. That keeps the mental model dead simple: arm a is the current page, arm b is the page you wrote next to it, and a visitor sees one or the other in full.
Because a variant is just a page, authoring it is a natural hand-off to the AI assistant or a connected agent. Start the test to create the sibling, then ask — “rewrite the paywall variant to lead with the annual discount” — and the agent edits pages/paywall@b.tsx like any other page. You frame the test; the agent writes the alternative.
[IMAGE OF the paywall and paywall@b variant open side by side in the editor with different headlines]
The experiment owns its variant page
This is the rule that makes experiments safe to run without babysitting: while a test is live, the experiment owns its variant page.
- You can’t delete
pages/paywall@b.tsxwhile the test is running or paused. - You can’t publish a release that drops it.
The only way the variant page goes away is by ending the test — promote the winner or discard it. Ending the test is the single removal path.
That one guarantee is why there are no broken experiments to reconcile. Old versions of this warned you about missing variant pages, “orphaned” variants, and releases that would strip a running test’s page out from under it. None of that exists anymore. The platform holds the variant page in place for the whole life of the test, in your draft and in every release that goes live, so a split can never point at a page that isn’t there.
The trade-off is deliberate: a running test is a commitment. If you want that variant page gone — to delete it, or to ship a release without it — conclude the experiment first. Promote a winner or discard the test, and the page is yours to change again.
Starting a test
Start the test on a page
In the Flow view, hover a page node and click the flask button (“Test this page”). The platform copies the current slot page into a variant — pages/<slot>@b.tsx — and creates a DRAFT experiment with two arms at 50/50: arm a (the control, which is the original slot page) and arm b (the new sibling).
Author the variant
You land in the variant’s code. Make it different — that’s the whole point. Preview it, iterate, hand it to an agent. Arms are labeled a–d; add up to 4 while the experiment is a DRAFT, and the weights rebalance evenly as you go.
Send it live
Start the test from the Experiments dashboard. The split goes live on your current release at the weights you set, and from that moment new visitors are routed between the arms. The platform keeps every arm’s page in the live release for as long as the test runs.
Once a test is running, weights are locked. To change the split, reweight — that starts a new version: version N’s numbers are archived, counters restart from zero, and the new split applies only to visitors who arrive after it. Reweighting is the honest way to change a running test; the old data doesn’t get silently blended with the new split.
[IMAGE OF the flask button revealed on a page node in Flow view]
The Experiments dashboard
Every test in the project lives in one place: the Experiments area. It’s split into Active and Concluded. Each row names the test, shows its arms, and — the part you actually read — a plain-English Verdict. When a test is ready to call, a nudge surfaces on your Overview so you don’t have to go looking for it.
Whole-funnel tests show up here too, alongside page tests, read the same way. (You set a funnel-vs-funnel test up from the campaign that routes the funnels; the dashboard reports every experiment in one list regardless of where it was created.)
[IMAGE OF the Experiments dashboard list with several tests, each showing a verdict pill]
Reading the verdict
The verdict is a sentence, not a spreadsheet. It walks up a ladder as data comes in:
| Verdict | What it means |
|---|---|
| Warming up | Just started — barely any data. Don’t read anything into the numbers yet. |
| Collecting | Traffic is flowing and payers are landing, but no arm has separated from the pack. |
| Leaning B | Arm b is ahead, but not confidently enough to call. Keep it running. |
| Ready: B wins | The winner is called. You get the ARPU lift and the probability behind it. Safe to promote. |
| No difference | The arms are effectively tied. The change didn’t move revenue — keep the simpler one. |
| Guardrail blocked | An arm is winning on ARPU but hurting something you can’t trade away (see below). It won’t be crowned. |
| Promoted | The test is concluded and the winner is live. |
Alongside the verdict you get the ARPU lift (how much more revenue per visitor the leader earns), the probability it really wins, and — when the answer is still “not yet” — the gates that explain why we can’t call it.
Why we can’t call it yet
A verdict isn’t withheld to be coy; each gate is a specific thing that has to be true before a number is trustworthy:
- Enough runtime. At least a week, and it has to span a weekend — buyers behave differently on Saturday than on Tuesday.
- Enough visitors in each arm.
- Enough payers in each arm — this is the one that matters for a revenue call. ARPU is only as precise as the number of people who actually paid, so a funnel with light traffic will read “not enough data yet — need about N more payers” instead of inventing a winner.
That last line is the honesty that makes the tool worth trusting. A low-traffic funnel gets told the truth — how many more payers it needs — rather than a confident-looking verdict built on ten sales.
[IMAGE OF an experiment detail with a “Ready: B wins” verdict, the ARPU lift, probability, and a gates checklist]
How the winner is decided
The winner is chosen on ARPU — revenue per visitor — not on a conversion-rate bump that might come with cheaper customers. A page that converts more people but at a lower price can easily lose here, and that’s correct: you’re optimizing money, not signups. The lift comes with a probability you can act on, and a couple of guards keep it honest:
- One big spender can’t decide it. The verdict is re-checked with the single largest payer removed; if the result depends on one whale, it’s downgraded, not called.
- Guardrails. An arm that wins on revenue but tanks your trial-start rate or spikes refunds is blocked from being crowned — a short-term revenue bump that comes back as chargebacks isn’t a win.
Promote — make the winner live
Concluding a test is a decision, and there are two ways to make it:
- Promote takes the winning page and makes it the page everyone sees. Its content replaces the slot page, the other variant pages are removed, and the funnel publishes — the winner is live in one step.
- Discard ends the test and keeps the control. Use it when the verdict is No difference, or when you’ve simply learned what you needed and want your variant page back.
Either way, ending the test is what releases the variant page the experiment was holding.
[IMAGE OF the Promote dialog showing the winning arm, the ARPU lift, and the publish confirmation]
History is permanent
A concluded test doesn’t disappear. Its results are frozen — the exact numbers at the moment you called it — and its winning page is kept forever, with lineage: you can trace which page came out of which test, and which test spun off from which. When you promote a winner, the lift you banked is recorded against it, so months later you can still see why the current paywall looks the way it does and what it beat to get there.
Nothing about a finished experiment is recomputed or overwritten. The verdict you promoted on is the verdict you keep.
How visitors get assigned
Each visitor is assigned one variant and keeps it across visits — a returning visitor always sees the arm they saw before, so the split stays stable rather than re-rolling every visit.
Their variant is fixed the moment the funnel loads, so entering an email partway through can’t flip them to a different arm. A single person’s journey is never split across two variants.
When a funnel serves several campaigns
If more than one active campaign routes the same funnel, a page test’s numbers are pooled across all of them — exposures and revenue are counted wherever the funnel runs. That’s the honest total, but it isn’t a clean per-campaign read. If you need to isolate one campaign’s behavior, run the funnel in only that campaign for the length of the test.