How many days to detect the effect you’re aiming for.
Enter the traffic of the pages you’re testing, the metric’s current value, and the lift you want to be able to detect. It’s the same calculation Probant runs for every test plan.
Prefilled with the Cart area of the demo store (Maison Lune, fictional) and the test plan for “Free shipping progress bar,” planned for 18 days.
Test length
18 days
With 3,402 visitors a day in the test, it takes 18 days to be able to detect a 5.52% lift in revenue per visitor, with three checkpoints.
- Visitors per variant
- 29,456
- Total visitors
- 58,912
- Checkpoint overhead
- +1.7% visitors28,963 per variant with a single checkpoint
| Duration | Smallest detectable lift |
|---|---|
| 7 days | 8.7% |
| 14 days | 6.1% |
| 28 days | 4.3% |
Three steps, all handled by the engine.
The calculator runs the same calculation as Probant’s statistical engine, the one that prepares a real test plan.
Visitors for a test read once.
For a conversion rate, the Fleiss formula without continuity correction; for an amount per visitor, n = 2(z₁₋α/₂ + z₁₋β)²σ²/δ², where σ is the standard deviation and δ the difference to detect in euros. Risk 5%, power 80%.
The cost of checkpoints.
Being able to conclude early when a boundary is crossed costs a few visitors: +0.8% for 2 checkpoints, +1.7% for 3 checkpoints, +2.4% for 4 checkpoints, +2.8% for 5 checkpoints. These factors are tabulated by Jennison and Turnbull (2000).
Duration in whole days.
Visitors needed divided by the visitors sent to the test each day, rounded up to the next whole day, assuming constant traffic split evenly between the two versions. Checkpoints fall at regular intervals, rounded to the day: for example, boundaries of 3.47, 2.45, 2.00 for three checkpoints.
The formulas and their sources are detailed on the statistical engine page; the role of checkpoints, on the How Probant decides page.
Plan this test
A test plan calculates the same duration from your store’s measured traffic, sets its checkpoints, and seals them before the first visitor.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway