Personalize, with proof that it helps.
A personalization shows a version of your pages to an audience: returning mobile visitors, a campaign, a country… Part of that audience stays on the original. Probant measures whether the personalization helps, and suggests pausing it when it does worse.
- 1Express delivery for loyal mobile shoppersOn mobile and returning visitorLive+12.6%
- 2Welcome code for Instagram visitorsComes from social media and campaign “automne-26”Live1,681 visitors in 7 days
- 3Shipping costs shown in Belgium and SwitzerlandIn BE or CHPaused-22.9%
- 4Size guide for visitors from searchComes from search engine and on /produits/*Draft—
Read the transcript
Should loyal mobile shoppers see something different? A version shown only to one audience, measured against a holdout kept on the original.
An audience is built from criteria. 17 criteria, combined with AND or OR. “Loyal mobile shoppers”: on mobile and returning visitor.
A holdout stays on the original. 10% of the audience, to measure the effect: “Express delivery for loyal mobile shoppers” shows +12.6% on “Add to cart,” checkpoint after checkpoint.
The one that does worse is paused. “Shipping costs shown in Belgium and Switzerland” shows -22.9% against its holdout: Probant suggests pausing it at the Nov 3 checkpoint.
A test can suggest one. In “Related products under the buy button”, the variant shows +8.3% among returning visitors and -6.0% among everyone else.
These personalizations and their holdouts are in the demo.
Personalize without losing measurement.
The visuals come from the demo store’s personalizations, measured by the engine when the page loads.
An audience is built from criteria.
Device, new or returning, traffic source, campaign, country, landing page, pages already viewed, number of visits: criteria combine in groups, with AND or OR, and each one can be reversed with “except”. A criterion whose value is unknown is false, even when reversed: a visitor whose country is unknown is neither in France nor outside France.
- Attributes from your site with probant.identify: short labels, up to 20, with no email address or name
- Audience size estimated over the last 7 days, when its criteria are known to the server
- An audience can be saved and reused across personalizations
- Device
- New or returning
- Traffic source
- Campaign (utm_campaign)
- Landing page
- Country
- Current page
- Page already seen
- Number of visits
- Site attribute
- Cookie
- Page variable
- Imported audience (CDP)
- Segment (behavior)
- Browser language
- Return after an absence
- Cart filled during the visit
probant.identify({ plan: "premium" })Kept in the page: neither stored nor sentA holdout stays on the original.
Part of the audience, 10% by default, keeps seeing the original. It is the only fair comparison, because an audience’s visitors are not like the others. For “Express delivery for loyal mobile shoppers”, “Add to cart”: 10.8% on the holdout, 12.2% with the personalization, so +12.6% (range +4.6% to +20.6%) after 58 days.
- A holdout of 5% to 50% of the audience, locked at launch; beyond that, you might as well run a test
- Always-valid ranges: a personalization has no planned end and can be checked every day without multiplying false winners
- No difference is shown while a group has fewer than 5 converted visitors, as with “Welcome code for Instagram visitors”, live for 5 days
- W1
- W2
- W3
- W4
- W5
- W6
- W7
- W8
- W9
One that does worse is suggested for a pause.
At the end of each week since launch, Probant rereads the range. If it is entirely below zero, the personalization does demonstrably worse than the original, and Probant suggests pausing it. For “Shipping costs shown in Belgium and Switzerland”, “Add to cart”: 10.1% on the holdout, 7.8% with the personalization; the pause was suggested at the Nov 3 checkpoint.
- A sample ratio mismatch between the two groups suspends any reading
- While paused, the whole audience sees the original again; on resume, everyone returns to their group
- W1
- W2
Nov 3 checkpoint: the whole range is below zero.
A test can suggest a personalization.
A variant can clearly win for some visitors and lose for others. On a live test’s page, Probant compares its effect by device, new or returning, and traffic source, and corrects all these comparisons together to rule out false patterns. For “Related products under the buy button”, the variant gets +8.3% among returning visitors and -6.0% among the others: rolling it out to everyone would have been a mistake.
- Suggested only if the difference holds after correction, the variant clearly wins in the segment, and it doesn’t also win elsewhere
- The segment was found after the fact, so the suggested personalization keeps a 10% holdout to confirm the effect on new visitors
“Related products” does better for returning visitors: +8.3% on “Add to cart”, versus -6.0% for the others.
In detail.
- Priority
- Personalizations are evaluated in list order, which you can change. The first one whose audience matches wins, holdout included: the holdout sees the original, not the next personalization.
- With tests
- A personalization and a test can target the same page: the personalization’s changes apply after the test’s. The workspace holdout sees no personalization, just as it sees no test.
- Assignment
- The group is drawn from the visitor’s identifier, independently of tests: it stays the same if the visitor leaves the audience and comes back. Holdout, audience, and changes are locked at launch.
- Changes
- Replaced text, hidden element, inserted block, through the form or in the visual editor; no variant code. The server checks each change before saving it.
- Schedule
- Optional start and end dates: the personalization starts and stops on its own. Once ended, its measurement remains available.
- Script
- Nothing more to install: the script loads a small module on demand, only if the workspace serves at least one personalization. Other sites don’t download it.
- Consent
- When the workspace requires consent, nothing is evaluated, read, stored, or sent before the visitor agrees. The country is read from the request’s IP address, only if an audience uses it; the address itself is not stored.
- Permissions
- A contributor prepares a personalization as a draft; launching it, pausing it, or changing the order is up to members who can launch a test.
Frequently asked questions.
How is this different from an A/B test?
A test compares versions across all of a page’s traffic, for a computed duration, then reaches a decision. A personalization targets an audience, with no planned end, and keeps a small holdout to check that it still helps. Beyond a 50% holdout, you might as well run a test.
Why not compare the audience with the rest of the site?
Because its visitors are not like the others: loyal mobile visitors don’t behave like newcomers. The holdout, randomly drawn from the same audience, is the only comparison that isolates the personalization’s effect.
Can I check the results every day?
Yes. The ranges are always valid, so checking them every day doesn’t increase the error risk. A pause is only suggested at checkpoints, every 7 days.
What can I pass with probant.identify?
Short labels: a plan, a number of orders, a subscription. No email address, name, or customer ID. Up to 20 attributes of 100 characters each; they stay in the page, neither stored in the browser nor sent to Probant.
Next: Feature flags
See this feature in the demo, on the screen this page describes, or move on to the next one.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway