Every idea is estimated before it takes up a page.
A page can only carry a limited number of tests. For each idea, Probant estimates the expected effect from your past tests, the time it would take to see it, what it would earn, and which one to test first.
Free shipping from €50
- Expected effect
- +2.5%
- based on 4 shipping tests
- To conclude
- 14 days
- on checkout starts
- Estimated gain
- +€62,900
- per month
Read the transcript
Which one should you test first? Probant estimates each idea’s effect from your past tests, works out its duration on your traffic, and ranks it in the queue.
The expected effect comes from your past tests. For “Free shipping from €50”, the 4 completed “shipping” tests are pooled: +2.5%, range +1.3% to +3.6%.
Duration is based on the zone’s traffic. Measuring orders would take 62 days; checkout starts concludes in 14 days.
The queue sorts itself. 7 worth testing, 11 needs rework, and 3 avoid. At the top: “Free shipping from €50”, +€62,900 expected per month for 2 days of effort.
Rank your own ideas in the demo, no sign-up needed.
The backlog, ranked by priority.
The demo store’s 24 ideas: 7 worth testing, 11 needs rework, 3 avoid, and 3 already in test. The first ones to test:
- 1Free shipping from €50Cart · effort SExpected effect: +2.5%based on 4 shipping testsTo conclude: 14 dayson checkout startsGain per month: +€62,900per monthPriority: 100worth testing
- 2Free returns noted in the cartCart · effort SExpected effect: +1.7%prior · 2 reassurance tests, too fewTo conclude: 29 dayson checkout startsGain per month: +€43,000per monthPriority: 68worth testing
- 3Secure payment methods shown in the cartCart · effort SExpected effect: +1.7%prior · 2 reassurance tests, too fewTo conclude: 29 dayson checkout startsGain per month: +€43,000per monthPriority: 68worth testing
- 4Free-shipping threshold shown on the home pageHome page · effort SExpected effect: +2.5%based on 4 shipping testsTo conclude: 15 dayson product clicksGain per month: +€22,600per monthPriority: 36worth testing
- 5Express shipping offered at checkoutCheckout · effort LExpected effect: +2.5%based on 4 shipping testsTo conclude: 48 dayson ordersGain per month: +€62,900per monthPriority: 13worth testing
- 6Delivered price shown on category pagesCategory page · effort MExpected effect: +1.7%prior · 2 pricing tests, too fewTo conclude: 38 dayson product clicksGain per month: +€22,400per monthPriority: 11worth testing
- 7Saved cart shown on the home pageHome page · effort MExpected effect: +1.7%priorTo conclude: 32 dayson product clicksGain per month: +€15,500per monthPriority: 9worth testing
- 8Numbered checkout stepsCheckout · effort SExpected effect: +1.7%prior · 2 checkout tests, too fewTo conclude: 102 dayson ordersGain per month: +€43,000per monthPriority: 20needs rework
Where these figures come from.
Three calculations, updated after every completed test and every change in traffic.
The expected effect comes from your past tests.
When the library holds at least 3 tests on the same theme, the idea gets their pooled effect, weighted by each test’s precision. The 4 “shipping” tests give +2.5%, range +1.3% to +3.6%. Below 3 tests, the idea gets the prior from the whole library: +1.7%.
The duration is computed from the page’s traffic.
An expected effect is only useful if it can be measured. Probant counts the days needed at the page’s real traffic. For “Free shipping from €50”, measuring orders would take 62 days; checkout starts, which happens more often, concludes in 14 days.
- A more sensitive metric is suggested only if it makes the idea measurable within 60 days
- An idea that takes too long to measure is marked “needs rework”, with the smallest effect visible within a month
Duration at the page’s real traffic, visitors split evenly, 5% risk, 80% power.
The score tells you which one to test first.
Priority divides the expected monthly gain by the days of effort, and prorates a test that lasts more than a month, since it occupies the page that much longer. The score, out of 100, is relative to the best idea in the backlog.
= €31,450 per day of effort, the best value in the backlog: 100 / 100
- S2 days
- M5 days
- L10 days
In detail.
- Expected effect
- Pooled effect of the theme from 3 completed tests; otherwise the prior from the whole library (+1.7% across 48 tests).
- Duration
- Days needed to detect the expected effect at the page’s traffic, split evenly, with the risk and power used in test plans.
- Gain
- Expected effect applied to the monthly revenue of visits that go through the page, rounded to the nearest hundred euros.
- Score
- Monthly gain × min(1, 30 days ÷ duration) ÷ days of effort (S: 2 days, M: 5 days, L: 10 days), scaled to 100.
- Verdict
- Worth testing (measurable within 60 days), needs rework (takes longer to measure), avoid (expected effect negative or zero).
- Statuses
- to assess, ready, in preparation, in test: set by the team, alongside the engine’s verdict.
Frequently asked questions.
What if my store has no completed tests yet?
Without a library, there is no expected effect specific to your store: the score builds up as tests complete. Every test, winner or not, sharpens the next estimates.
Why is a promising idea marked “needs rework”?
Because its expected effect would take too long to show at its page’s traffic. Rather than occupying the page for months for a vague answer, Probant shows the smallest effect measurable within a month, so you can aim for a bolder change or a busier page.
Next: Metrics and margin
See this feature in the demo, on the screen this page describes, or move on to the next one.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway