In the cart, measure gains in margin.

Free-shipping threshold, progress bar, fees shown: what happens in the cart decides the order. The examples come from the demo store.

Create your workspace Explore the demo

Maison Lune Cartdemo

TestStatusLift and range
Free shipping progress barRevenue per visitorConclusive. Proven gain61,240 visitors+6.0%, range +2.2% to +10.0%
Recently completed
Free shipping from €60Revenue per visitorConclusive. Proven gaincompleted Oct 2+5.8%, range +2.0% to +9.8%
Delivery time shown in the cartConversionNo detectable effect.completed Aug 24+1.2%, range -0.9% to +3.4%
Savings shown in the cartRevenue per visitorNo detectable effect.completed Jul 20-0.3%, range -3.8% to +3.3%

Maison Lune, a fictional demo store · data computed by the engine

In the cart, every fee shown weighs on the order.

Visitors have picked their products; now they need a reason to pay. Tests here target what changes the order amount or removes a doubt.

Free-shipping thresholdThe order amount that unlocks free shipping. It moves average order value and shipping costs.
Progress barHow much more to add to reach the threshold, shown as the cart fills up.
Fees shown earlyShipping costs shown before the cart, so there’s no last-minute surprise.
Promo codeA prominent field can send visitors off to hunt for a code elsewhere.
ReassuranceFree returns, secure payment, delivery times: what removes doubt at the moment of paying.
Add-on productsSuggesting one more item, at the risk of distracting from checkout.

Maison Lune, a fictional demo store

Read the gain in margin too, after the cost of free shipping.

“Free shipping progress bar” raises revenue per visitor by +6.0% (from +2.2% to +10.0%) and has no established effect on margin per visitor (+3.0% measured, from -0.7% to +6.7%). The gap between the two has a measured cause: more orders clear the threshold, and every free delivery has a cost.

Revenue and margin, side by side.

The share of orders with free shipping goes from 41.0% to 57.0%. That shipping costs the store €23,800 a month, already deducted from margin: the expected margin (+€27,700 per month, from -€5,400 to +€60,900) is much smaller than the expected revenue (+€125,700 a month), and its range doesn’t establish it yet.

A tool that measures only revenue would have reported the revenue gain without its cost.

Free shipping progress bar

Revenue per visitorprimary+6.0%+2.2% to +10.0%
Margin per visitorafter product costs, discounts, and shipping+3.0%-0.7% to +6.7%
Orders with free shippingexplains the gap between revenue and margin41.0% → 57.0%+16 points

+€125,700Expected revenue / month, corrected for the winner’s curse

+€27,700Expected margin / month, from -€5,400 to +€60,900

-€23,800Free shipping per month, already deducted from margin

The verdict comes at the planned checkpoint, not before.

The plan set three checkpoints, on days 6, 12, and 18, with very strict boundaries early on. The boundary was crossed at the day 18 checkpoint, which is when the test became conclusive.

The reported gain is corrected for the winner’s curse: +4.9% instead of the +6.0% measured, based on the store’s past tests.

Free shipping progress bar

Cart · all devices · launched Oct 23 by Camille · day 18

Conclusive. Proven gain

Expected revenue, corrected for the winner’s curse+€125,700 a month

Free shipping progress bar

Cart · all devices · launched Oct 23 by Camille · day 18

Conclusive.

Result. Showing progress toward free shipping: revenue per visitor +6.0% measured, +4.9% expected once corrected for the winner’s curse. The effect on margin isn’t established yet: +3.0% measured, range from -0.7% to +6.7%.

Why. More orders reach the threshold: customers add an item rather than pay for shipping.

Next. Expected revenue: +€125,700 per month after correcting for the winner’s curse. Then test a €50 threshold, measured on margin.

Metrics

Metric	Original	Variant	Difference	95% range
Revenue per visitorprimaryOriginal €24.58 · Variant €26.06	€24.58	€26.06	+6.0%	+2.2% to +10.0%
Margin per visitorafter product costs, discounts, and shippingOriginal €10.03 · Variant €10.33	€10.03	€10.33	+3.0%	-0.7% to +6.7%
Conversion rateOriginal 20.0% · Variant 20.8%	20.0%	20.8%	+3.9%	+0.7% to +7.2%
Average order valueOriginal €122.90 · Variant €125.40	€122.90	€125.40	+2.0%	0.0% to +4.1%
Orders with free shippingexplains the gap between revenue and marginOriginal 41.0% · Variant 57.0%	41.0%	57.0%	+16 points	Expected effect of the variant

Guardrails

Guardrail	Original	Variant	Difference	95% range
Return rateprovisional, 30-day value on Dec 9Original 1.6% · Variant 1.7%	1.6%	1.7%	+2.6%	Within limit, limit +25.0%
Mobile speed (LCP)Original 2.41 s · Variant 2.42 s	2.41 s	2.42 s	+0.4%	Within limit, limit +4.0%
JavaScript errorsshare of visitors with an error · limit +25%Original 0.12% · Variant 0.12%	0.12%	0.12%	+0.2%	Watch

Amounts

Expected revenue / month+€125,700+€41,600 to +€209,800

Expected margin / month+€27,700-€5,400 to +€60,900

Free shipping-€23,800per month, already deducted from margin

Checks

Planned sample size104%

A/B split50.0 / 50.0

Sample ratio mismatch (SRM) testp = 0.81

Saw both versions0.3%

Guardrails breachednone

Boundary crossed at checkpointday 18 · z = 3.1

Orders reconciled98.6%

Known costs (margin)96.7%

Winner’s cursecorrected, +4.9%

Full report from the demo store. Lifts, ranges, checks, and amounts are computed by the Probant engine.

A winner goes live in steps.

“Simplified order summary”, the winner of an earlier test, is rolled out to all visitors.

“Free shipping progress bar” is awaiting a decision: the report suggests starting the rollout at 25% of traffic.

The next idea in the backlog already has its numbers: “Free shipping from €50”, expected effect +2.5% based on 4 shipping tests.

Free shipping progress barnot started · first step 25%

25%

50%

100%

Free shipping from €6050% step in progress

25%

50%

100%

Simplified order summaryrolled out to 100% on Mar 26

25%

50%

100%

The next ideas get their numbers before launch.

Each idea gets an expected effect, drawn from past tests on the same theme, and a duration based on its area’s traffic. Shipping ideas from other areas are listed here too.

Cart trafficdemo store

3,400visitors a day

3.5%smallest effect measurable in 30 days on “Commande”

1.7%on “Checkout start”, which happens more often, in the same time

An idea with a smaller expected effect won’t reach a conclusion in a month, and Probant tells you before launch.

Rank 1
Free shipping from €50Ready to test: Positive expected effect, measurable in under 60 days on this area’s traffic.
Expected effect
+2.5%based on 4 shipping tests
Time to conclude
14 daysmeasuring checkout startsinstead of 62 days on orders
Estimated gain
+€62,900per month
Rank 2
Free returns noted in the cartReady to test: Positive expected effect, measurable in under 60 days on this area’s traffic.
Expected effect
+1.7%prior2 reassurance tests, too few
Time to conclude
29 daysmeasuring checkout startsinstead of 132 days on orders
Estimated gain
+€43,000per month
Rank 3
Secure payment methods shown in the cartReady to test: Positive expected effect, measurable in under 60 days on this area’s traffic.
Expected effect
+1.7%prior2 reassurance tests, too few
Time to conclude
29 daysmeasuring checkout startsinstead of 132 days on orders
Estimated gain
+€43,000per month

3 pending ideas, in Probant’s ranking order. Expected effect drawn from past tests on the same theme; duration based on the area’s traffic.

Past tests show what worked in the cart.

The demo store’s completed tests, on one scale. A lesson only becomes a rule once enough tests point the same way.

Lessons by theme

Shipping

4 tests · 3 winners

+2.5% pooled effect, +1.3% to +3.6%

High confidence. Together, the tests point the same way: the pooled range excludes zero.

Reassurance

2 tests · 1 winner

+1.6% pooled effect, +0.1% to +3.2%

Low confidence. Fewer than three tests: a lead, not yet a rule.

The 6 most recent of 8 completed tests

Free shipping from €60Sep 7 → Oct 2 · Revenue per visitor · Shipped
Customers add an item to reach the threshold
Conclusive. Proven gain+5.8%, range +2.0% to +9.8%
Delivery time shown in the cartAug 3 → Aug 24 · Conversion · Original kept
Possible effect, too small to detect at this traffic
No detectable effect.+1.2%, range -0.9% to +3.4%
Savings shown in the cartJun 29 → Jul 20 · Revenue per visitor · Original kept
Customers don’t seem to count their savings
No detectable effect.-0.3%, range -3.8% to +3.3%
Gift wrapping offered in the cartMay 25 → Jun 15 · Conversion · Dropped
One more option meant more hesitation before paying
Conclusive. Proven loss-2.4%, range -4.5% to -0.3%
Complementary products in the cartApr 27 → May 18 · Revenue per visitor · Original kept
Slightly larger orders, but not enough orders to conclude
No detectable effect.+3.1%, range -0.6% to +7.0%
Promo code field collapsedMar 19 → Apr 9 · Conversion · Shipped
A visible promo field sent shoppers off to look for a code elsewhere
Conclusive. Proven gain+1.8%, range +0.2% to +3.4%

Check a cart gain against margin and returns.

A cart change can raise revenue and cost you somewhere else.

MarginFree shipping or a discount can push revenue up and margin down. Probant measures both.
ReturnsAn item added to reach a threshold sometimes comes back. Return rates are known later than orders.
Reaching checkoutIt happens more often than an order, so it’s faster to measure: useful when traffic is short.

Demo store guardrails

Monitored during every test and every rollout step. A proven drop stops the test or sends visitors back to the original.

Return rate: visitors who made at least one return request · increase of +25% at most
Page speed (LCP): 75th percentile of page views · increase of +4% at most

The Probant features that help in this area.

ReportsA one-sentence verdict, with revenue and margin in euros.Learn more GuardrailsA test that breaks something stops on its own.Learn more RolloutsStep by step, with automatic rollback.Learn more The winner’s curseWhy the reported gain is too good, on average.Learn more

Other use cases

The same engine, for other site areas and other teams. The demo opens on the screen this page describes.

Create your workspace Explore the demo

Open demo, no sign-up
Free to start, no overage charges
Database and compute in Amsterdam, with Scaleway