Checkout
2 tests · 2 winners
+5.6% pooled effect, +3.7% to +7.5%
Low confidence. Fewer than three tests: a lead, not yet a rule.
In checkout, verify the traffic split before the result.
Steps, reassurance, one-page checkout: checkout is short, and every visitor lost along the way counts. The examples come from the demo store.
- One-page checkoutConversionSample ratio mismatch.19,800 visitorsHidden until diagnosed
- Recently completed
- Fewer form fieldsConversionNo detectable effect.completed Sep 28+1.0%, range -1.0% to +3.0%
- Secure payment badgesConversionNo detectable effect.completed Aug 17+0.6%, range -1.3% to +2.6%
- Guest checkoutConversionConclusive. Proven gaincompleted Jun 18+7.3%, range +4.5% to +10.2%
Maison Lune, a fictional demo store · data computed by the engine
Test checkout step by step.
The visitor has decided to buy. Tests here target what makes them give up along the way.
- Number of stepsOne page or several, and how progress is shown.
- Account creationRequired, optional, or offered after the order.
- FormsFields requested, address autocomplete, typing on mobile.
- PaymentPayment methods offered, one-click payment for returning customers.
- ReassuranceSecure payment, reminders about returns and delivery times.
- ShippingCarrier choice, express delivery, pickup points.
Maison Lune, a fictional demo store
A traffic split mismatch suspends the test before anything is read.
“One-page checkout” was meant to split visitors evenly. A got 10,454, B got 9,346. The traffic split test gives p < 0.001: the gap is very unlikely to be chance.
The diagnosis, browser by browser.
The gap comes from Safari iOS: B got 1,108 fewer visitors than A there. On the other browsers, the split is normal (p = 1.00).
Probant shows where to look for the cause: in whatever runs only on that browser.
Results stay hidden until the diagnosis. A conversion difference measured on unbalanced groups would say nothing about the variant, only about who got lost along the way.
| Browser | A | B | Split | Test |
|---|---|---|---|---|
| Safari iOS | 2,960 | 1,852 | 61.5% / 38.5% | p < 0.001 |
| Chrome Android | 3,380 | 3,379 | 50.0% / 50.0% | p = 1.00 |
| Chrome desktop | 2,640 | 2,641 | 50.0% / 50.0% | p = 1.00 |
| Safari macOS | 1,474 | 1,474 | 50.0% / 50.0% | p = 1.00 |
| All browsers | 10,454 | 9,346 | 52.8% / 47.2% | p < 0.001 |
| Without Safari iOS | 7,494 | 7,494 | 50.0% / 50.0% | p = 1.00 |
Checkout traffic decides what you can test there.
Fewer visitors reach checkout than the product page, so only a clear effect can be measured there in a month. Probant puts a number on it for each idea, before launch.
Checkout trafficdemo store
1,800visitors a day
3.1%smallest effect measurable in 30 days on “Commande”
An idea with a smaller expected effect won’t reach a conclusion in a month, and Probant tells you before launch.
- Rank 5Express shipping offered at checkout
- Expected effect
- +2.5%based on 4 shipping tests
- Time to conclude
- 48 dayson orders
- Estimated gain
- +€62,900per month
- Rank 8Numbered checkout steps
- Expected effect
- +1.7%prior2 checkout tests, too few
- Time to conclude
- Undetectable102 days on ordersmeasurable in 30 days from +3.1%
- Estimated gain
- +€43,000per month
- Rank 13One-click payment for returning customers
- Expected effect
- +1.7%prior2 checkout tests, too few
- Time to conclude
- Undetectable102 days on ordersmeasurable in 30 days from +3.1%
- Estimated gain
- +€43,000per month
3 pending ideas, in Probant’s ranking order. Expected effect drawn from past tests on the same theme; duration based on the area’s traffic.
The library shows what’s already been tried in checkout.
Before relaunching an idea, the area’s completed tests help you avoid repeating a loser or retesting a winner that’s already rolled out.
Lessons by theme
Reassurance
2 tests · 1 winner
+1.6% pooled effect, +0.1% to +3.2%
Low confidence. Fewer than three tests: a lead, not yet a rule.
5 completed tests
- Fewer form fieldsAug 31 → Sep 28 · Conversion · Original keptNo detectable effect.+1.0%, range -1.0% to +3.0%
Shorter form, effect too small to detect at this traffic
- Secure payment badgesJul 20 → Aug 17 · Conversion · Original keptNo detectable effect.+0.6%, range -1.3% to +2.6%
No effect, so trust in payment wasn’t the obstacle
- Guest checkoutMay 18 → Jun 18 · Conversion · ShippedConclusive. Proven gain+7.3%, range +4.5% to +10.2%
Requiring an account was holding back first purchases
- Checkout in four numbered stepsFeb 23 → Mar 23 · Conversion · DroppedConclusive. Proven loss-4.2%, range -6.3% to -2.1%
Showing the number of steps discouraged shoppers from the first one
- Address autocompleteDec 22 → Jan 19 · Conversion · Original keptNo detectable effect.+0.9%, range -1.1% to +2.9%
Faster to fill in, with no net effect on orders
In checkout, an error costs an order.
Checkout touches payment: a variant that breaks in a single browser gets lost in an average.
- Traffic splitA redirect, a blocked script, or a cache can keep visitors out of one version. The traffic split test catches it before any reading.
- JavaScript errorsA variant that breaks payment in one browser should stop on its own.
- Revenue per visitorMore orders with smaller carts can lose money.
Demo store guardrails
Monitored during every test and every rollout step. A proven drop stops the test or sends visitors back to the original.
- JavaScript errors
- visitors who hit at least one error · increase of +25% at most
- Page speed (LCP)
- 75th percentile of page views · increase of +4% at most
The Probant features that help in this area.
Other use cases
The same engine, for other site areas and other teams. The demo opens on the screen this page describes.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway