Skip to content

Hosted in Europe. Your test data is stored in the European Union.Hosted and stored in EuropeLearn more

Log in Create your workspace

From idea to rollout, in one tool.Product overview

Prepare

Decide

Roll out

Scale

Visual editorBuild the variant without a developer, with the assistant.Test plan and durationDuration calculated before the first visitor.IdeasEach idea estimated from your past tests.Metrics and marginNo-code goals, margin calculated from your orders.

ReportsA one-sentence verdict, with revenue and margin in euros.Session recordings and click mapsSee what visitors do with each variant.GuardrailsA test that breaks something stops on its own.

RolloutsStep by step, with automatic rollback.PersonalizationAn experience for one audience, measured against a holdout.Feature flagsTurn a feature on by rule, and off in one click.

Learnings libraryThe effects of your finished tests, on one scale.Installation and integrationsTwo lines of code, in the browser or on the server.Team and governanceRoles, launch approvals, and an audit log.

Explore the demo Pricing

The rules behind every verdict, set before launch.The full method

The method, step by step

How Probant decides The peeking trap The winner’s curse The statistics engine Glossary Verify a report

MethodThe winner’s curseWhy the reported gain is too good, on average.See the correction

Revenue per visitor · Free shipping progress bar

Measured lift+6.0%

After correction+4.9%

+€125,700per month after correction, instead of +€153,300

Corrected using the demo store’s 48 finished tests.

Rerun the simulation with your own number of tests and duration.A/A simulator

By site area and by team.All use cases

You are…

Know what pays off, without a statistician.

The features that matter to you:

ReportsA one-sentence verdict, with revenue and margin in euros.Metrics and marginNo-code goals, margin calculated from your orders.Visual editorBuild the variant without a developer, with the assistant.See the use case

Cart · demo storeFree shipping progress barConclusive. Proven gain+€125,700 per month · revenue per visitor +6.0%, range +2.2% to +10.0%

Computed by the engine on a demo test.

By site areaCart and shipping Product page Checkout Mobile navigation Category pages

Explore the demo

Blog, docs, and tools.All resources

A/A simulatorSee for yourself what peeking costs.Why checking a test every day produces false winners.Run the simulator

BlogMethod, measurement, and lessons learned.

Nov 12What is an A/B test? Definition, steps, and an example Nov 7Measure margin, not just revenue Nov 7The winner’s curse: why the announced gain falls short

Duration calculatorHow many days it takes to detect the effect you want.DocumentationInstall the script, handle consent, send events.Video tutorialsInstall, launch a test, and read a report, on video.ComparisonsProbant vs. other tools, with sources.AB TastyKameleoonVWOOptimizelyAdobe TargetDynamic Yield+11 more

ChangelogWhat changed in Probant, by date.Case study formatContext, test, result, and decision, on the demo store.Migrate to ProbantImport your AB Tasty, VWO, or Kameleoon history, recalculated by Probant.

Articles, docs, and calculators, free to use.

One fixed monthly price, no overage charges.See pricing

How many visitors do you test each month?

A tested visitor is each unique visitor who sees a test or a rollout during the month. It’s the only thing that sets the price.

Billing

Your planEssential€590/monthor €5,900 per yearFor a store that tests continuously, down to revenue per visitor.

Up to 250,000 tested visitors per month
All features, unlimited tests within your volume
No overage charges

Choose Essential

Your bill never goes over your plan. Once you reach the volume, running tests keep going.Billing questions

A one-sentence verdict, in euros.

The report opens with the conclusion and its range. The gain is corrected for the winner’s curse before being converted into euros of revenue and margin, and the checks are listed alongside for anyone who wants to verify.

Create your workspace Explore the demo

Free shipping progress bar

Conclusive. Proven gain

Result. Showing progress toward free shipping: revenue per visitor +6.0% measured, +4.9% expected once corrected for the winner’s curse. The effect on margin isn’t established yet: +3.0% measured, range from -0.7% to +6.7%.

Why. More orders reach the threshold: customers add an item rather than pay for shipping.

Next. Expected revenue: +€125,700 per month after correcting for the winner’s curse. Then test a €50 threshold, measured on margin.

Expected revenue / month+€125,700range +€41,600 to +€209,800

Boundary crossed at checkpointday 18 · z = 3.1

0:40 · no sound, looping

Read the transcript

Can you trust this result? The verdict at the top of the report, with everything you need to check it alongside.

The report starts with the conclusion. “Free shipping progress bar”: one sentence, the test’s status, then each metric with its 95% range.

The checks sit next to the verdict. 9 checks, each with its value: sample, traffic split, guardrails, matched orders. The report also lists what isn’t known yet.

The gap between segments isn’t proven. Mobile +9.5%, Desktop +2.7%: after Holm’s correction for 2 comparisons, the range of the gap includes zero. The variant rolls out to everyone.

Each test outcome has its own screen. 6 outcomes, from a clear win to a guardrail stop, each with its own title and dot color.

The “Free shipping progress bar” report is in the demo, no sign-up needed.

The full report, as it appears.

Report for the “Free shipping progress bar” test on the demo store. Differences, 95% ranges, amounts, and checks are computed by the engine from the test’s raw data.

Free shipping progress bar

Cart · all devices · launched Oct 23 by Camille · day 18

Conclusive.

Result. Showing progress toward free shipping: revenue per visitor +6.0% measured, +4.9% expected once corrected for the winner’s curse. The effect on margin isn’t established yet: +3.0% measured, range from -0.7% to +6.7%.

Why. More orders reach the threshold: customers add an item rather than pay for shipping.

Next. Expected revenue: +€125,700 per month after correcting for the winner’s curse. Then test a €50 threshold, measured on margin.

Metrics

Metric	Original	Variant	Difference	95% range
Revenue per visitorprimaryOriginal €24.58 · Variant €26.06	€24.58	€26.06	+6.0%	+2.2% to +10.0%
Margin per visitorafter product costs, discounts, and shippingOriginal €10.03 · Variant €10.33	€10.03	€10.33	+3.0%	-0.7% to +6.7%
Conversion rateOriginal 20.0% · Variant 20.8%	20.0%	20.8%	+3.9%	+0.7% to +7.2%
Average order valueOriginal €122.90 · Variant €125.40	€122.90	€125.40	+2.0%	0.0% to +4.1%
Orders with free shippingexplains the gap between revenue and marginOriginal 41.0% · Variant 57.0%	41.0%	57.0%	+16 points	Expected effect of the variant

Guardrails

Guardrail	Original	Variant	Difference	95% range
Return rateprovisional, 30-day value on Dec 9Original 1.6% · Variant 1.7%	1.6%	1.7%	+2.6%	Within limit, limit +25.0%
Mobile speed (LCP)Original 2.41 s · Variant 2.42 s	2.41 s	2.42 s	+0.4%	Within limit, limit +4.0%
JavaScript errorsshare of visitors with an error · limit +25%Original 0.12% · Variant 0.12%	0.12%	0.12%	+0.2%	Watch

Amounts

Expected revenue / month+€125,700+€41,600 to +€209,800

Expected margin / month+€27,700-€5,400 to +€60,900

Free shipping-€23,800per month, already deducted from margin

Checks

Planned sample size104%

A/B split50.0 / 50.0

Sample ratio mismatch (SRM) testp = 0.81

Saw both versions0.3%

Guardrails breachednone

Boundary crossed at checkpointday 18 · z = 3.1

Orders reconciled98.6%

Known costs (margin)96.7%

Winner’s cursecorrected, +4.9%

What the report does for you.

Three checks run before any figure is shown.

The reported gain is corrected before it is promised.

On average, a test’s winner overstates its effect: it was picked partly because chance favored it. Probant pulls the measured difference toward what your 48 past tests make plausible. Here, +6.0% measured becomes +4.9% reported, and that figure is the one converted into euros.

Expected revenue: +€125,700 per month, range +€41,600 to +€209,800
Expected margin: +€27,700 per month
Free shipping cost: -€23,800 per month, already deducted from the margin

Winner’s curseprior drawn from 48 completed tests

Past test effects: +1.7% on average Measured: +6.0% Reported: +4.9%

Amounts, after correction

Expected revenue / month+€125,700+€41,600 to +€209,800

Expected margin / month+€27,700-€5,400 to +€60,900

Free shipping-€23,800per month, already deducted from margin

Checks are listed next to the verdict.

Sample reached, traffic split, visitors seen in both versions, guardrails, decisive checkpoint, reconciled orders: 9 checks, each with its value. What the test cannot tell yet is written down too.

Checksnext to the verdict

Planned sample size104%

A/B split50.0 / 50.0

Sample ratio mismatch (SRM) testp = 0.81

Saw both versions0.3%

Guardrails breachednone

Boundary crossed at checkpointday 18 · z = 3.1

Orders reconciled98.6%

Known costs (margin)96.7%

Winner’s cursecorrected, +4.9%

What we don’t know yet

“Return rate”: 30-day value, complete on Dec 9, 2026 at the earliest if the test stopped today.

Segments don’t create false patterns.

Slicing a result by device or by new vs. returning visitors will eventually show a “better” segment. Probant tests the difference between segments and corrects for multiple comparisons with the Holm method before saying whether it is real. Here, it is not, so the variant rolls out to everyone.

Segments · Revenue per visitor95% ranges

Mobile+9.5%+4.4% to +15.0%

Desktop+2.7%-3.1% to +8.7%

New+7.4%+2.6% to +12.5%

Returning+4.4%-1.8% to +11.1%

Difference between segmentsNot shownDevice and new vs. returning compared, Holm correction.

A multivariate test is read section by section.

On “Orbe product page: title, photo, and badge” (Title, Photo, Badge: 12 combinations), each section is compared with its original across all traffic, interactions between sections are measured, and the 4 comparisons reported in the plan are corrected with the Holm method. Each combination can still be read as a variant.

Title “Benefit”: +3.0% vs. the original (-2.4% to +8.7%), not conclusive after correction
Title “Short”: -4.1% vs. the original (-9.2% to +1.3%), not conclusive after correction
Photo “In a room”: +11% vs. the original (+6.0% to +16%), conclusive after correction
Badge “Made in France”: +6.6% vs. the original (+2.0% to +11%), conclusive after correction
Recommendation: Combination D · In a room + Made in France
Interactions weren’t declared in the plan: they’re shown for reference and don’t decide anything. The recommendation follows the main effects, section by section.

Every test outcome has its own screen.

Winner, loser, no detectable effect, too early, sample ratio mismatch, stopped by a guardrail: 6 situations, 6 different readings. The examples come from the demo store.

Conclusive. Proven gain
The variant wins, with its range.
Free shipping progress barCart · Revenue per visitor
Showing progress toward free shipping: revenue per visitor +6.0% measured, +4.9% expected once corrected for the winner’s curse. The effect on margin isn’t established yet: +3.0% measured, range from -0.7% to +6.7%.
Measured difference
+6.0%
Corrected expected revenue
+€125,700 / month
Conclusive. Proven loss
The variant loses: the whole range is below zero.
Strikethrough prices in redProduct page · Revenue per visitor
More clicks, but smaller orders. Difference -3.4%, range -6.0% to -0.7%.
Measured difference
-3.4%
Ended on
Aug 28
Completed test from the library
No detectable effect.
B makes no measurable difference to revenue per visitor.
Customer reviews at the top of the pageProduct page · Revenue per visitor
After 21 days and 64,100 visitors, the difference is -0.4%, range -4.0% to +3.3%. Narrowing it to ±2% would take 50 days more.
Measured difference
-0.4%
For ±2%
71 days in total
Not conclusive yet…
Too early to conclude. Next checkpoint in 4 days.
Simplified mobile menuNavigation · Revenue per visitor
Estimated difference +1.9%, range -3.7% to +7.8%. At the day 9 checkpoint, z = 0.6, far from the boundary of 2.45.
Progress
day 9 of 13
Next checkpoint
day 13, boundary 2.00
Sample ratio mismatch.
Results hidden: B is getting fewer visitors than planned.
One-page checkoutCheckout · Conversion
B received 9,346 visitors versus 10,454 for A, which is 47.2% instead of 50% (χ² = 62.0). The gap comes from Safari iOS, where B is missing 1,108 visitors.
Share of B
47.2%
Missing visitors
1,108
Stopped by a guardrail.
B pushes the return rate from 1.6% to 2.9% of orders.
“You may also like” recommendationsProduct page · Revenue per visitor
Relative increase of +78%, always-valid range +25% to +130%, beyond the +25% tolerance set in the test plan. Switched back to the original on Nov 10 at 05:12.
Return rate
1.6% → 2.9%
Test plan tolerance
+25%

In detail.

Summary: Result, Why, Next: three sentences, in plain language, without jargon.
Ranges: 95% on every metric and every guardrail, on the relative difference between the variant and the original.
Amounts: Revenue and margin per month, rounded to the nearest hundred euros: a gain estimated to the euro would suggest a precision it doesn’t have.
Winner’s curse: Bayesian shrinkage toward the average effect of your completed tests, applied before any conversion into euros.
Checks: Planned sample size, A/B split, Sample ratio mismatch (SRM) test, Saw both versions, Guardrails breached, Boundary crossed at checkpoint, Orders reconciled, Known costs (margin), Winner’s curse.
Segments: Mobile, Desktop, New, Returning, with a heterogeneity test corrected by Holm.
Verdicts: Winner, loser, no notable effect, not settled yet, or blocked by a sample ratio mismatch or a guardrail.

Frequently asked questions.

Why is the reported gain smaller than the measured difference?

Because a winner is picked partly thanks to chance. Correcting before reporting avoids promising a figure that won’t materialize: +6.0% measured, +4.9% reported on the demo store.

What does “No detectable effect” mean?

That the range includes zero at the end of the test. It is not “no effect”: an effect too small for this traffic is still possible. The report says how many more days it would take to decide, and what that would cost.

Why are some results hidden?

When the split of visitors between A and B is off, the two groups are no longer comparable: any difference would mix the variant’s effect with that of a malfunction. Results stay hidden until the cause is diagnosed.

Learn more.

The winner’s curseWhy the reported gain is too good, on average.Learn more Measure margin, not just revenueRevenue and margin don’t always move together. The free-shipping example shows how a test can grow sales and still shrink profit.Learn more GuardrailsA test that breaks something stops on its own.Learn more

Next: Session recordings and click maps

See this feature in the demo, on the screen this page describes, or move on to the next one.

Create your workspace Explore the demo

Open demo, no sign-up
Free to start, no overage charges
Database and compute in Amsterdam, with Scaleway