Probant on video.

Does this free-shipping bar really pay off? One test, from the first change to the decision.

The variant is built on the real page. One click under the cart, and variant B says: “Only €11.00 away from free shipping.”

Each visitor always sees the same version. 18 days, three checkpoints, and a target effect of 5.5%: the plan is set before the first visitor.

Watch every day, decide at checkpoints. On day 6, z was 2.30, below the boundary of 3.47, so the test went on. The boundary is crossed on day 18.

A verdict in one sentence, in euros. +6.0% measured, +4.9% reported after correcting for the winner’s curse: +€125,700 in revenue per month.

The winner rolls out in steps. 25%, 50%, then 100% of visitors, with guardrails rechecked at each step. At the first proven harm, everyone goes back to the original.

The test joins the library. The store’s 48 completed tests sit on the same scale. Most effects are small (median +0.9%), which is why gains are corrected.

A/B tests that tell the truth. The demo is open, no sign-up needed.

Do you need a developer to build the variant? Click on the page or describe the change in one sentence. Probant applies it to your real page and checks it before any visitor sees it.

Click on the page. The product page opens in the editor. Clicking “Ajouter au panier” selects it: rewrite its text, hide it, or ask the assistant.

Or describe the change. The assistant finds the button ([data-add-to-cart]), inserts the date just below it, and asks before touching preorders.

Checked before launch. Measured on the preview: contrast of 5.3:1 against a threshold of 4.5:1, 1.1 kB added, and 1 of 2 selectors flagged as fragile.

Try the editor on the demo store, no sign-up needed.

How long until you know? Probant starts from your traffic: you know the duration before the first visitor, and the rule stays fixed after that.

You pick a target effect; Probant gives the duration. On “Add to cart” (8.2%), 8% can be detected in 30 days at the zone’s traffic; 2% would take 466 days.

Watched daily, decided at checkpoints. “Free shipping progress bar”: on day 6, z = 2.30 cleared the classic threshold of 1.96 but not the boundary of 3.47. The test continued until day 18.

At launch, the plan is sealed. Target effect, 58,900 visitors, three checkpoints, and guardrails: a fingerprint lets you verify that nothing has changed.

Work out the duration of your next test in the demo, no sign-up needed.

Which one should you test first? Probant estimates each idea’s effect from your past tests, works out its duration on your traffic, and ranks it in the queue.

The expected effect comes from your past tests. For “Free shipping from €50”, the 4 completed “shipping” tests are pooled: +2.5%, range +1.3% to +3.6%.

Duration is based on the zone’s traffic. Measuring orders would take 62 days; checkout starts concludes in 14 days.

The queue sorts itself. 7 worth testing, 11 needs rework, and 3 avoid. At the top: “Free shipping from €50”, +€62,900 expected per month for 2 days of effort.

Rank your own ideas in the demo, no sign-up needed.

What should you actually measure? A click becomes a metric without a line of code, and margin is calculated from your orders.

Define a conversion without code. A click on the page becomes a goal. Events your site already sends are suggested as metrics once they reach 5 visitors in 30 days.

Each metric shows what it can detect. The smallest effect visible in four weeks, next to the typical gain of your past tests (+3.7%). CUPED tightens the bars: up to 6% less variance.

Margin is calculated from your orders. Back-office orders, the cost of each product, and store fees: margin per visitor is calculated on the server, and your costs don’t go through the page.

A revenue winner can lose on margin. “Free returns highlighted”: +3.4% in revenue per visitor, but -8.9% in margin, range -12.5% to -5.1%.

These metrics are in the demo, no sign-up needed.

Can you trust this result? The verdict at the top of the report, with everything you need to check it alongside.

The report starts with the conclusion. “Free shipping progress bar”: one sentence, the test’s status, then each metric with its 95% range.

The checks sit next to the verdict. 9 checks, each with its value: sample, traffic split, guardrails, matched orders. The report also lists what isn’t known yet.

The gap between segments isn’t proven. Mobile +9.5%, Desktop +2.7%: after Holm’s correction for 2 comparisons, the range of the gap includes zero. The variant rolls out to everyone.

Each test outcome has its own screen. 6 outcomes, from a clear win to a guardrail stop, each with its own title and dot color.

The “Free shipping progress bar” report is in the demo, no sign-up needed.

Why does B win? Sessions and click maps show what visitors do with each variant.

Each session reads as timestamped events. What happened, page by page, to the second. Here, 1 rage click burst, 2 dead clicks, and left without converting after the tested element: 12 points.

The sessions to watch first. For “Free shipping progress bar”, 12 sessions kept out of 14: the ones with the most signals first, alternating between variants.

One click map per variant. On /panier, 12,201 visitors. In B, “Ambre candle · €14.90” goes from 4.8% to 8.4% of visitors: customers add an item rather than pay for shipping.

Probant receives only timestamped events. You turn it on yourself, and each visitor has to consent. Sessions are deleted 30 days after their last activity.

This test’s sessions are in the demo, no sign-up needed.

What if the variant breaks something? Each guardrail watches a metric that must not get worse, and stops the test as soon as the harm is proven.

Each guardrail has a limit, set in the plan. It is breached only if its whole range is past the limit; otherwise it stays under watch.

A proven increase, but still under the limit. On “You may also like” recommendations, the increase in return rate is proven from Nov 4, but not yet past +25%, so the test continues.

Proven harm stops the test. On Nov 10 at 05:12, the whole range is past +25%: return rate from 1.6% to 2.9%. Visitors go back to the original.

A sample ratio mismatch blocks the test. On “One-page checkout”, B got only 47.2% of visitors: results are hidden, and the diagnosis points to Safari iOS.

The stopped test and its guardrails are in the demo, no sign-up needed.

Ship the winner to everyone, right away? The share of visitors goes up in steps, and drops to zero at the first proven harm.

Each step has a minimum duration. 25%, 50%, then 100%. “Free shipping from €60” is at 50%.

Guardrails are rechecked at every checkpoint. The ranges stay valid whatever day you read them, so checking them at every checkpoint doesn’t multiply false alarms.

At the first proven harm, roll back. Same rule for a test: “You may also like” recommendations, return rate from 1.6% to 2.9%. All visitors went back to the original on Nov 10 at 05:12.

This rollout and this test are in the demo, no sign-up needed.

Should loyal mobile shoppers see something different? A version shown only to one audience, measured against a holdout kept on the original.

An audience is built from criteria. 17 criteria, combined with AND or OR. “Loyal mobile shoppers”: on mobile and returning visitor.

A holdout stays on the original. 10% of the audience, to measure the effect: “Express delivery for loyal mobile shoppers” shows +12.6% on “Add to cart,” checkpoint after checkpoint.

The one that does worse is paused. “Shipping costs shown in Belgium and Switzerland” shows -22.9% against its holdout: Probant suggests pausing it at the Nov 3 checkpoint.

A test can suggest one. In “Related products under the buy button”, the variant shows +8.3% among returning visitors and -6.0% among everyone else.

These personalizations and their holdouts are in the demo.

Release a feature to a few customers, and turn it off if it breaks? A feature flag serves a value to your server code, without redeploying.

Rules are read in order. For “Instant search”, each visitor goes down the stack: flag off, specific IDs, attributes, default rule. The first one that applies decides.

The share only goes up. “Photos in reviews” went from 10% to 25% of visitors. Bucketing uses the visitor ID, so an exposed visitor stays exposed.

Your servers read the value locally. The server client reloads the configuration every 60 seconds. If Probant doesn’t respond, your code gets the default value it passed.

Off everywhere within a minute. Nov 9, 06:42 PM: JavaScript errors up on Safari. The “off” value goes out immediately through the API, and reaches your servers within 60 seconds.

These feature flags and their rules are in the demo, no sign-up needed.

What’s left once a test ends? Every completed test joins the library, and the engine uses it to judge the next ones.

Every effect on the same scale. The demo’s 37 completed tests, in the order they ended. Most effects are small: median +0.9%.

Lessons by theme, with their confidence. A lesson needs at least three tests on the same theme. Its confidence is high when their pooled effect excludes zero.

What was tested, where, and when. One row per zone of the journey: Product page had 16, Category page just one, still running. Hollow bars are tests still running.

It corrects the next winners. From these 48 effects, the engine derives a prior: mean +1.7%, spread 3.4%. A winner measured at +6.0% is reported at +4.9%.

The demo library already holds 37 completed tests. Open it, no sign-up needed.

How many lines does it take to start? From the snippet pasted in the <head> to your team’s tools.

One snippet, in the <head>. The script weighs 6,953 bytes compressed, under its 7 KB budget. It loads asynchronously and doesn’t hide the page.

Nothing is sent before the visitor consents. Before consent, Probant sets no cookie, assigns no one, and sends nothing. If the visitor accepts mid-visit, the current page stays the original: the variant starts on the next page.

Server-side, the page arrives already changed. The server helper assigns the visitor before rendering: the variant is there from the first paint, and the script only measures.

Results go to your team’s tools. Alerts go to Slack or a signed webhook. Other outputs: CSV export, share link, management API, MCP server, and analytics tools.

The demo is open, no sign-up needed.

Who launched this test, and who reviewed it? Before launch, another member reviews the plan. After that, the audit log keeps every change.

Launches are requested, then reviewed. “Launch” becomes “Request approval”: the request locks the plan, then another member approves it.

If the draft changes, the approval is dropped. The request carries the draft’s fingerprint: make one more change and you have to ask again.

The audit log records who did what. Every action that changes the workspace, with its author and target, including the ones Probant takes on its own. Kept 13 months, exportable as CSV.

Everyone has a role, permission by permission. Four fixed roles, or your own, built from 25 named permissions. The owner can require two-factor authentication for everyone.

The demo is open, no sign-up needed.