Promotions
3 tests · 1 loser
-0.9% pooled effect, -2.1% to +0.3%
Medium confidence. Enough tests, but the pooled effect is still compatible with no effect.
The product page clears up doubts before the purchase.
Reviews, installment payments, recommendations, shipping: what reassures or distracts at the moment of adding to cart. The examples come from the demo store.
- Orbe product page: title, photo, and badgeAdd to cartConclusive. Proven gain80,512 visitors+26.6%, range +13.9% to +41.0%
- Customer reviews at the top of the pageRevenue per visitorNo detectable effect.64,100 visitors-0.4%, range -4.0% to +3.3%
- “You may also like” recommendationsRevenue per visitorStopped by a guardrail.stopped 3 h agoReturns +1.2 pts
- Pay-in-3 option highlightedRevenue per visitorNot conclusive yet…day 6 of 17+3.5%, range -3.4% to +11.0%
- Recently completed
- Related products under the buy buttonAdd to cartNo detectable effect.completed Oct 12-1.1%, range -3.2% to +1.1%
Maison Lune, a fictional demo store · data computed by the engine
Every block on the page can help or distract.
Product pages pile up blocks over time. Testing a block checks that it drives purchases, not just that people see it.
- Customer reviewsWhere they sit, the rating under the title, the number of reviews shown.
- Installment paymentsPay-in-installments highlighted under the price of expensive items.
- RecommendationsSuggested products, which can grow the cart or pull attention away from the buy button.
- Shipping and returnsFees, delivery times, and free returns shown right on the product page.
- Photos and videosGallery, lifestyle photos, assembly video.
- Buy buttonWhere it sits, and on mobile, a button that stays visible as you scroll.
Maison Lune, a fictional demo store
On the product page, a test can also end without a winner.
Four tests running or completed in the demo store, four different outcomes: conclusive, no detectable effect, stopped by a guardrail, and not conclusive yet. Each has its own screen.
Customer reviews at the top of the page: no detectable effect.
After 21 days and 64,100 visitors, the range runs from -4.0% to +3.3%: it contains zero. Probant reports neither a gain nor a loss.
Narrowing it to ±2% would take 71 days in total. The report puts a number on that cost instead of letting the test run on.
The library already held “Average rating under the title”, also with no detectable effect: favorable trend, but the rating was already visible further down.
No detectable effect.
Revenue per visitor: -0.4%, range -4.0% to +3.3%
64,100visitors
21 daysof testing
71 daysin total for a ±2% range
Pay-in-3 option highlighted: not conclusive yet.
Day 6 of 17: today’s lift is +3.5%, with a range of -3.4% to +11.0%. One checkpoint reached, no boundary crossed.
Next checkpoint: day 11, with a boundary of 2.45. Until then, you can look at the daily range, but it decides nothing.
If paying in 3 installments appears under the price of lights over €150, more visitors will add them to cart, because price is the top reason for leaving in the survey.
“You may also like” recommendations: stopped by a guardrail.
Return rate: 1.6% of orders on the original, 2.9% with the recommendations. The relative difference, +77.9%, has a range of +25.4% to +130.3%: the whole range exceeds the +25.0% limit set in the test plan.
The test stopped on Nov 10 at 05:12 AM without waiting for anyone.
Guardrail ranges are always valid: you can read them at every check without triggering false alarms.
Product page ideas go through the same calculation.
Expected effect drawn from past tests on the same theme; duration based on product page traffic. An idea whose theme lowered the metric on average is marked “Avoid.”
Product page trafficdemo store
9,100visitors a day
4.0%smallest effect measurable in 30 days on “Commande”
An idea with a smaller expected effect won’t reach a conclusion in a month, and Probant tells you before launch.
- Rank 9Estimated delivery date
- Expected effect
- +2.5%based on 4 shipping tests
- Time to conclude
- Undetectable79 days on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€57,200per month
- Rank 10Pickup points shown on the product page
- Expected effect
- +2.5%based on 4 shipping tests
- Time to conclude
- Undetectable79 days on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€57,200per month
- Rank 112-year warranty highlighted
- Expected effect
- +1.7%prior2 reassurance tests, too few
- Time to conclude
- Undetectable168 days on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€39,100per month
- Rank 12Lifestyle photos in the product gallery
- Expected effect
- +1.7%prior2 media tests, too few
- Time to conclude
- Undetectable168 days on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€39,100per month
- Rank 14Assembly video for pendant lights
- Expected effect
- +1.7%prior2 media tests, too few
- Time to conclude
- Undetectable168 days on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€39,100per month
- Rank 15Lampshade sizes as buttons on mobile
- Expected effect
- +0.4%based on 5 mobile choices tests
- Time to conclude
- Undetectableover a year on ordersmeasurable in 30 days from +4.0%
- Estimated gain
- +€10,000per month
The first 6 of 7 pending ideas, in Probant’s ranking order. Expected effect drawn from past tests on the same theme; duration based on the area’s traffic.
The library keeps what didn’t work, too.
Winners, losers, and tests with no detectable effect, on one scale. Lessons group the tests on each theme.
Lessons by theme
Reviews
1 test
+2.6% pooled effect, -0.4% to +5.5%
Low confidence. Fewer than three tests: a lead, not yet a rule.
Media
2 tests
-1.3% pooled effect, -2.7% to 0.0%
Low confidence. Fewer than three tests: a lead, not yet a rule.
The 6 most recent of 12 completed tests
- Related products under the buy buttonSep 21 → Oct 12 · Add to cart · Original keptNo detectable effect.-1.1%, range -3.2% to +1.1%
Draws some attention away from the button, with no net loss
- Free returns highlightedAug 20 → Sep 11 · Revenue per visitor · ShippedConclusive. Proven gain+3.4%, range +0.8% to +6.1%
Clearing up doubts about returns reassures new customers
- Strikethrough prices in redAug 3 → Aug 28 · Revenue per visitor · DroppedConclusive. Proven loss-3.4%, range -6.0% to -0.7%
More clicks, but smaller orders
- Average rating under the titleJul 6 → Jul 27 · Add to cart · Original keptNo detectable effect.+2.6%, range -0.3% to +5.6%
Favorable trend, but the rating was already visible further down
- Shipping costs shown on the product pageJun 15 → Jul 9 · Revenue per visitor · ShippedConclusive. Proven gain+2.1%, range +0.2% to +4.0%
The surprise at the cart was costing sales
- Countdown timer on flash salesMay 18 → Jun 8 · Revenue per visitor · Original keptNo detectable effect.+0.5%, range -1.6% to +2.7%
No extra purchases despite the ticking clock
A block that sells can also drive returns.
On the product page, side effects show up after the order, or somewhere other than the page.
- ReturnsA poorly targeted recommendation or an overblown promise gets paid for in returns a few weeks later.
- SpeedA heavier video or gallery slows down rendering on mobile.
- ErrorsA block injected by script can break the page in some browsers.
Demo store guardrails
Monitored during every test and every rollout step. A proven drop stops the test or sends visitors back to the original.
- Return rate
- visitors who made at least one return request · increase of +25% at most
- Page speed (LCP)
- 75th percentile of page views · increase of +4% at most
- JavaScript errors
- visitors who hit at least one error · increase of +25% at most
The Probant features that help in this area.
GuardrailsA test that breaks something stops on its own.Learn more Visual editorBuild the variant without a developer, with the assistant.Learn more Test plan and durationDuration calculated before the first visitor.Learn more Learnings libraryThe effects of your finished tests, on one scale.Learn more
Other use cases
The same engine, for other site areas and other teams. The demo opens on the screen this page describes.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway