Probant vs. Convert
Both tools compared point by point, based on Convert’s public documentation, accessed November 11, 2026. Every statement about Convert links to its source.
What Convert offers.
Convert Experiences presents itself as a “privacy-first” A/B testing and personalization platform, founded in 2009.1 It combines A/B testing, Shopify price testing, full stack, and an MCP server, with no separate products.2 Three methods to choose from: frequentist, Bayesian, or sequential.345
Probant is for sites that want A/B test verdicts that hold up, with a plan set before launch and guardrails monitored through rollout. It measures what each conversion is worth: the margin on an order, a qualified or signed lead, a paid subscription.
The differences, point by point.
On the left, what Convert’s public documentation says, each sentence with its source. On the right, what Probant does. “Not mentioned” means nothing was found in that documentation, which does not prove a feature is missing.
| Criterion | Convert | Probant |
|---|---|---|
| Statistical method | Three approaches: frequentist (t-test, two-sided by default, 95% by default), Bayesian (non-informative prior), and sequential. The method is set at the project level and can be overridden per experiment; the default method for a new project is not stated explicitly.345 | Frequentist, with a sequential plan set before launch: target effect, duration, and checkpoints. The verdict fits in one sentence, with a 95% range. Where to check |
| Peeking | Sequential mode uses “Asymptotic Confidence Sequences” (Waudby-Smith et al.), which allow continuous monitoring while controlling the false positive rate; the default tuning parameter is 5,000 visitors. Sequential testing is in the Pro plan (not in Growth).678 | Decisions are only made at planned checkpoints, with O’Brien-Fleming boundaries: very strict at first, close to the usual threshold at the end. During a rollout, ranges are always valid (mSPRT), so you can check them at any time. Where to check |
| Traffic split (SRM) | SRM check by chi-square goodness-of-fit test, triggered at 99% confidence, to be turned on in the project settings; it shows an SRM label in the report. Blocking the result is not mentioned.910 | The traffic split is checked with a chi-square test every time the report is read. If the split is off, the test moves to “blocked” status and no winner is declared. Where to check |
| Guardrails | Convert recommends treating secondary goals as guardrails to monitor; an automatic stop on their degradation is not mentioned. An automation can stop losing variants based on confidence (primary goal) and redistribute their traffic.1112 | Guardrail metrics are tracked with their range; a guardrail is only breached if the whole range crosses the limit. A breached guardrail blocks the test’s verdict, stops a running test on its own, and during a rollout moves visitors back to the original automatically. Where to check |
| Gradual rollouts | Full-stack feature rollouts open a feature to a percentage of visitors, and web “Deploys” push a change to an audience. No automatic rollback on metric degradation is documented.1314 | Steps of 25% → 50% → 100% by default, 10% → 25% → 50% → 100% in cautious mode. Automatic rollback when a guardrail is breached; an exposed visitor stays exposed in the following steps. Where to check |
| Multi-armed bandit | Bandit auto-allocation (Thompson Sampling by default, Epsilon-Greedy, UCB), optimizing the primary goal only; in frequentist mode it requires sequential testing. Included from the Pro plan.1516 | Optimization test: Thompson sampling, reallocated at most once a day, with a minimum share per variant. It maximizes a conversion rate, or an amount per visitor such as revenue: purchase probability and average order value drawn from their distributions, with values capped at a percentile as in an A/B test. Optionally, shares per device (mobile, tablet, desktop): one draw per device, each with its own learning and floor. A visitor always keeps their variant: a new split only applies to visitors who arrive after it. The screen says so: it maximizes conversions (or the amount per visitor) during the test, without proving that a variant is better. The difference is read period by period (and device by device), with no declared winner; the CSV export gives the same reading. Where to check |
| Revenue and margin per visitor | RPV is treated as a non-binomial metric with its own confidence method; reports show a confidence interval (or a credible interval in Bayesian mode) on the difference. By default, only each visitor’s first order counts.171819 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). | Revenue and margin per visitor, each with its range; the gain is stated in euros. Margin is calculated on the server from reconciled orders, the cost of each product (Shopify or CSV), and store fees; no margin verdict below 90% of revenue covered by known costs. Where to check |
| Winner’s curse | A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026). | The reported gain is corrected for the winner’s curse by Bayesian shrinkage toward the effects of the store’s past tests. Where to check |
| Bayesian mode | Bayesian mode with a non-informative prior by default, “Chance to Win” with an adjustable decision threshold (95% by default), credible interval, and Risk and Expected Uplift columns (hidden by default).2021 | Optional, off by default: a Bayesian reading next to the verdict, with the probability that the variant does better, the credible interval, and the expected loss of each choice. The prior is centered on zero and as wide as the effects measured by the store’s completed tests; while there are too few of them to estimate it, the Bayesian reading is not shown. The decision always follows the sealed plan; when the two readings disagree, the report explains why. Where to check |
| Multiple comparisons | In frequentist mode, a choice of multiple comparisons correction: Bonferroni, Šidák, or none (Šidák recommended for critical tests). The documentation does not say whether the correction also covers several metrics or segments.22 | Segment comparisons are corrected with the Holm method. With several variants, the error risk is split across comparisons (Bonferroni). Where to check |
| Multivariate testing | Multivariate tests with combined sections and variations (2 elements × 2 versions = 4 combinations in the example), from the Pro plan. Reporting on interaction effects is not documented.2324 | Full factorial design generated by Probant: two to four sections, each with two to four levels, up to sixteen combinations, each served as a variant. Main effects of each section across all traffic and interactions between two sections, corrected with the Holm method on the comparisons stated in the plan. Based on the workspace’s measured traffic, the plan says whether the test can reach a conclusion and suggests removing a level or a section; no best combination is declared without a check. Where to check |
| Variance reduction (CUPED) | Variance reduction (CUPED) is not mentioned in their public documentation (as of November 11, 2026). | CUPED applied to live test reports, using each visitor’s history before their first exposure; without history, the difference stays unadjusted and the report says so. Where to check |
| Visual editor | In-app visual editor (the site loads in the editor) and code editors; a Chrome extension for debugging and for migrating from VWO.2526 | Two ways to edit, producing the same changes: a page preview loaded by Probant, without its scripts, or the live page on the site, scripts included. On the site, the editor is loaded by the script already installed, through a short-lived signed link for one draft and one variant, with no browser extension. It only sends the chosen changes to Probant. Where to check |
| Widgets | Only native pop-ups built in the visual editor are documented (triggered on load, click, or scroll); a library of ready-made ecommerce widgets (countdown, social proof, survey) is not mentioned.27 | Banner, countdown, pop-up, badge, reassurance, action bar, delivery date, social proof, promo code, free shipping, and survey, set up without code in the editor and saved in the workspace library for other tests. A widget is just a set of ordinary changes: escaped text, links over https or to the site, and styles applied without a style attribute, so a strict CSP does not need loosening. The countdown follows the site’s time zone and disappears at the deadline; the pop-in opens on arrival, on exit intent, or after scrolling, closes with the keyboard, and does not reappear during the visit if consent allows. Their small modules only load for visitors in the variant. The delivery date is calculated in the visitor’s browser (cutoff time, business days, public holidays); social proof shows the exact number of purchases received over a rolling week, and only above a threshold. The survey asks up to 8 questions (closed answers, recommendation scale, open question) with conditional logic, on the page or through a link outside the site (email, QR code), based on 70 templates (recommendation, satisfaction, effort, checkout blockers, cancellation, concept test, multi-question surveys…); open answers are redacted before sending (email addresses, phone numbers, cards, IBANs), kept for 180 days, and grouped by theme; each question is read by variant, with its range and response rate, and comparisons across all questions are corrected together; a response through a link is attached to the variant the visitor saw, recalculated by Probant, or stays outside the test. Where to check |
| Script and flicker | The script goes in the head; anti-flicker by default (body hidden), and synchronous loading presented as the only guaranteed way to avoid flicker. Stated total size: 25 kB compressed without Signals, 83 kB with Signals.282930 | Script under 7 KB compressed, loaded asynchronously, that does not hide the page while it loads. Server-side assignment available, for a display with no flicker at all. Where to check |
| Granular permissions | Five fixed access levels per project (Browse, Editor, Reviewer, Publish, Administrator), also applied to API keys. Creating custom roles is not mentioned.3132 | Four fixed roles, plus custom roles the owner builds from a catalog of named permissions: prepare, launch and stop, roll out, approve a launch, write variant JavaScript, manage flags, metrics, margin, members, and keys. No privilege escalation: billing, roles, transfer, and deletion of the workspace stay with the owner; no one can grant more than their own permissions, and an API key cannot exceed its author’s. Where to check |
| Data hosting | The pricing page lists EU servers in Frankfurt on all plans; the security page lists AWS, region eu-west-1, and Hetzner for the login page; delivery through the Akamai CDN. The two official pages do not give the same location.333435 | Database and compute in Amsterdam, with Scaleway, in the European Union. Scaleway, which hosts the database and compute, is a French company. Published subprocessor list: Scaleway (Amsterdam, Netherlands (nl-ams region), in the European Union), Brevo (European Union: servers in France and Germany, encrypted backups in Belgium (Google Cloud), according to Brevo), Mistral AI (European Union by default (European API endpoint), according to Mistral), ClickHouse (option not used), Stripe (Contract with Stripe Payments Europe, Ltd. (Ireland); data transferred to Stripe, LLC (United States), according to Stripe), Sentry (European Union: Frankfurt (Germany), including backups, according to Sentry). Depending on the feature used, Mistral may temporarily transfer data outside the European Union to the subprocessors it publishes; its data processing agreement limits these transfers to countries the EU recognizes as adequate, or covers them with standard contractual clauses. Stripe transfers payment data to Stripe, LLC, in the United States, under the Data Privacy Framework and, failing that, standard contractual clauses. Whatever region is chosen, Sentry may keep the data of Probant’s Sentry account (members, settings, audit log, project descriptions) in the United States; reported errors stay in Frankfurt. Where to check |
| Consent by default | Blocking cookies, and optionally experiments, before consent has to be turned on in code (consentRequired command); it is not presented as the default behavior. Visitor cookie _conv_v lasts up to 6 months; first-party cookies only.363738 | Consent required by default: no cookie, no assignment, and nothing sent before the visitor agrees. Consent given mid-page: the page does not change in front of the visitor; their variant starts on the next page. First-party cookie, lasting 390 days at most, never extended. Where to check |
| Pricing model | Priced by monthly tested users (MTU): Growth $299/month billed annually or $399/month billed monthly for 100k MTU, Pro $420/month annually or $599/month, Enterprise on quote (annual). Overage billed at $399 per 100k (Growth) or $699 per 250k (Pro, Enterprise), can be turned off; excluding tax.394041 | Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges. Agency plan: 1,000,000 tested visitors per month, shared across client workspaces, from 3 workspaces, for €1,490 per month or €14,900 per year. Where to check |
What Convert also does.
Probant shares these points with Convert.
A sequential test
Sequential mode uses “Asymptotic Confidence Sequences” (Waudby-Smith et al.), which allow continuous monitoring while controlling the false positive rate; the default tuning parameter is 5,000 visitors. Sequential testing is in the Pro plan (not in Growth).678
A multiple comparisons correction
In frequentist mode, a choice of multiple comparisons correction: Bonferroni, Šidák, or none (Šidák recommended for critical tests). The documentation does not say whether the correction also covers several metrics or segments.22
Revenue per visitor, with its range
RPV is treated as a non-binomial metric with its own confidence method; reports show a confidence interval (or a credible interval in Bayesian mode) on the difference. By default, only each visitor’s first order counts.171819
Dynamic traffic allocation
Bandit auto-allocation (Thompson Sampling by default, Epsilon-Greedy, UCB), optimizing the primary goal only; in frequentist mode it requires sequential testing. Included from the Pro plan.1516
Multivariate testing
Multivariate tests with combined sections and variations (2 elements × 2 versions = 4 combinations in the example), from the Pro plan. Reporting on interaction effects is not documented.2324
When to choose Convert over Probant.
Based on Convert’s public documentation.
You want public pricing and EU servers
Priced by monthly tested users (MTU): Growth $299/month billed annually or $399/month billed monthly for 100k MTU, Pro $420/month annually or $599/month, Enterprise on quote (annual). Overage billed at $399 per 100k (Growth) or $699 per 250k (Pro, Enterprise), can be turned off; excluding tax.394041 The pricing page lists EU servers in Frankfurt on all plans; the security page lists AWS, region eu-west-1, and Hetzner for the login page; delivery through the Akamai CDN. The two official pages do not give the same location.333435
You want heatmaps and session recordings
Native heatmaps (clicks, taps, scroll) by page and variant, and Convert Signals session recordings filtered on friction; monthly quota shared per project, web only.4243
You want to choose your statistical method
Three approaches: frequentist (t-test, two-sided by default, 95% by default), Bayesian (non-informative prior), and sequential. The method is set at the project level and can be overridden per experiment; the default method for a new project is not stated explicitly.345
What sets Probant apart.
These points are not in Convert’s public documentation, accessed November 11, 2026. A missing mention does not prove a feature is missing; if these points matter to you, ask Convert.
Margin per visitor
Probant measures revenue and margin per visitor, each with its range.
ConvertMargin per visitor is not mentioned in their public documentation (as of November 11, 2026).
Learn moreWinner’s curse correction
The reported gain is corrected by Bayesian shrinkage toward the effects of the store’s past tests.
ConvertA winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).
Learn moreSRM that blocks the verdict
A traffic split imbalance moves the test to “blocked” status, and no winner is declared.
ConvertSRM check by chi-square goodness-of-fit test, triggered at 99% confidence, to be turned on in the project settings; it shows an SRM label in the report. Blocking the result is not mentioned.910 Automatic blocking of the test is not mentioned there (as of November 11, 2026).
Consent required by default
No cookie, no assignment, and nothing sent before the visitor agrees.
ConvertBlocking cookies, and optionally experiments, before consent has to be turned on in code (consentRequired command); it is not presented as the default behavior. Visitor cookie _conv_v lasts up to 6 months; first-party cookies only.363738 Consent required by default is not mentioned there (as of November 11, 2026).
Frequently asked questions.
How much does Convert cost?
Priced by monthly tested users (MTU): Growth $299/month billed annually or $399/month billed monthly for 100k MTU, Pro $420/month annually or $599/month, Enterprise on quote (annual). Overage billed at $399 per 100k (Growth) or $699 per 250k (Pro, Enterprise), can be turned off; excluding tax.394041 Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges.
Where does Convert host data?
The pricing page lists EU servers in Frankfurt on all plans; the security page lists AWS, region eu-west-1, and Hetzner for the login page; delivery through the Akamai CDN. The two official pages do not give the same location.333435 Probant hosts its database and application in Amsterdam, with Scaleway.
Does Convert measure margin?
Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). Probant measures revenue and margin per visitor, each with its range, from orders and the cost of each product.
Can you look at Convert results while a test is running?
Sequential mode uses “Asymptotic Confidence Sequences” (Waudby-Smith et al.), which allow continuous monitoring while controlling the false positive rate; the default tuning parameter is 5,000 visitors. Sequential testing is in the Pro plan (not in Growth).678 Probant sets its checkpoints before launch (O’Brien-Fleming) and uses always-valid ranges during rollouts.
Sources.
Official pages from Convert, accessed November 11, 2026. Quotes are reproduced in their original language.
- Source 1: www.convert.com/llm-info
Convert Experiences is a privacy-first A/B testing and personalization platform founded in 2009
- Source 2: www.convert.com/pricing
We offer A/B testing, split testing, Shopify price testing, full-stack, AI flows (with a MCP server) and customer insights - without separate products or add-ons.
- Source 3: support.convert.com/hc/en-us/articles/statistical-models-in-convertcoms-ab-testing-platform
Convert offers three powerful statistical approaches—Frequentist, Bayesian, and Sequential—each catering to different testing needs.
- Source 4: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
Important Update: As of October 13th, 2023, all new experiences utilize t-tests instead of z-tests.
- Source 5: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
The default is now set to "two-tailed" testing to align with standard statistical practices and ensure greater reliability.
- Source 6: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
Our sequential testing employs Asymptotic Confidence Sequences for analysis, a method grounded in the advancements by Waudby-Smith et al. (2023).
- Source 7: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
Confidence sequences effectively address the "peeking problem" commonly associated with interim analysis in A/B testing.
- Source 8: www.convert.com/pricing
Everything in Growth plus: • FREE VWO Migration • Multivariate Testing • Multi-Page Testing • Full Stack & Feature Flags • Multi-Arm Bandit (Auto-Allocate) • Sequential Testing
- Source 9: support.convert.com/hc/en-us/articles/4412567998221-what-is-srm
If there is a big enough variation, our test will trigger at 99% confidence!
- Source 10: support.convert.com/hc/en-us/articles/4412567998221-what-is-srm
You can enable the SRM checks on your Project Configuration > More Settings and be able to see the SRM tags on the report itself.
- Source 11: www.convert.com/blog/a-b-testing/how-to-prevent-p-hacking-convert
Secondary goals will be tracked as guardrails, not success metrics.
- Source 12: support.convert.com/hc/en-us/articles/205160325-my-experiments-stopped-what-happened
When a variation is identified as performing worse than the others with sufficient confidence, it can be stopped automatically and its traffic redistributed across the remaining active variations.
- Source 14: support.convert.com/hc/en-us/articles/115000018331-what-about-deployments
A Deploy in the app is a series of changes on the page or pages geared towards a specific audience.
- Source 15: support.convert.com/hc/en-us/articles/multi-armed-bandit-mab-auto-allocation-in-convert
Auto-allocation uses MAB strategies to shift more traffic to winning variations as the test progresses.
- Source 16: support.convert.com/hc/en-us/articles/multi-armed-bandit-mab-auto-allocation-in-convert
Requires Sequential Test Type when using Frequentist model
- Source 17: support.convert.com/hc/en-us/articles/understanding-report-metrics-in-convert
AOV is displayed as a currency column alongside RPV in the Revenue table. (RPV statistics—confidence, lift, etc.—remain calculated exactly as before.)
- Source 18: support.convert.com/hc/en-us/articles/understanding-report-metrics-in-convert
Confidence Interval – Range containing the true lift.
- Source 19: support.convert.com/hc/en-us/articles/360001108332-statistical-confidence-for-rpv
First, we calculate the standard deviation and then we apply Z-Statistics to calculate the probability.
- Source 20: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
Convert employs uninformative priors—each variant starts with a 50% win probability.
- Source 21: support.convert.com/hc/en-us/articles/understanding-risk-and-expected-uplift-in-bayesian-reports
Risk: The probability that choosing a specific variant will result in a lower-performing outcome.
- Source 22: support.convert.com/hc/en-us/articles/10157554451085-statistical-methods-used
Concerning multiple comparison correction techniques, you can choose from Bonferroni, Sidak, or None.
- Source 23: support.convert.com/hc/en-us/articles/205208075-creating-a-multivariate-experiment
Unlike A/B tests, multivariate experiments involve testing more than one element on a page in different combinations.
- Source 24: support.convert.com/hc/en-us/articles/205208075-creating-a-multivariate-experiment
This example will show how to test two different elements, meaning we will have four different variations in total, as demonstrated below:
- Source 25: support.convert.com/hc/en-us/articles/360001098232-the-visual-editor-in-convert-experiences
The Convert Experiences Visual Editor, allows you to create variations of your pages for your experiments.
- Source 26: support.convert.com/hc/en-us/articles/360001098232-the-visual-editor-in-convert-experiences
All code editors that are available via Convert Visual Editor are described here.
- Source 27: support.convert.com/hc/en-us/articles/custom-pop-ups-in-the-visual-editor
Custom Pop-ups let you design and launch simple pop-up messages directly from the Visual Editor as part of a web A/B experience.
- Source 28: support.convert.com/hc/en-us/articles/360028491292-loading-the-convert-tracking-code-async-while-preventing-flashing
The only guaranteed way to not flashing in most cases is to load the tracking code in sync mode.
- Source 29: support.convert.com/hc/en-us/articles/understanding-convert-signals-and-signals-observer-min-js-script-in-convert
The total size when Convert Signals® is active would be: 83kB (245kB uncompressed) and when inactive the total size is: 25kB (66kB uncompressed).
- Source 30: www.convert.com/pricing
Install it by adding a single line of code, the SmartInsert™ technology prevents flickering out-of-the-box
- Source 31: support.convert.com/hc/en-us/articles/205151065-collaborators-share-with-multiple-users-accounts
Each collaborator can have one of five access levels to a project:
- Source 32: support.convert.com/hc/en-us/articles/205151065-collaborators-share-with-multiple-users-accounts
API keys use the same role-based permission model as collaborators.
- Source 34: www.convert.com/security
We use Amazon Web Services (AWS) for our production servers and databases.
- Source 35: www.convert.com/security
spread over two different availability zones in the eu-west-1 AWS region.
- Source 36: support.convert.com/hc/en-us/articles/360037946851-delay-cookie-writing-and-data-collection-until-visitor-consent-is-provided
To ensure Convert blocks cookie writing but still shows experiments, add the following code in the Convert App > Your Project > Configuration > Global Project JavaScript section:
- Source 37: support.convert.com/hc/en-us/articles/360037946851-delay-cookie-writing-and-data-collection-until-visitor-consent-is-provided
In this setup, both cookie writing and experience execution are blocked until the visitor gives consent. This method provides the strictest level of privacy protection.
- Source 38: support.convert.com/hc/en-us/articles/204495429-convert-experiences-tracking-cookies-structure
This cookie is the visitor-centric cookie that has a lifetime of max 6 months from the last update time.
- Source 39: www.convert.com/pricing
Convert Experiences starts at $299 per month billed annually, or $399 per month billed monthly, on the Growth plan with 100,000 monthly tested users (MTUs).
- Source 40: www.convert.com/pricing
Pro starts at $420 per month billed annually ($599 per month billed monthly), and Enterprise is priced on request as an annual plan.
- Source 41: www.convert.com/pricing
You can purchase additional packets of 100,000 unique tested users for $399/mo (in the Growth plan) and 250,000 unique tested users for $699/mo (in the Pro & Enterprise plans). You can either allow or disable these overquota charges.
- Source 42: support.convert.com/hc/en-us/articles/heatmaps-native-user-interaction-visualizations-in-convert
Convert’s native heatmaps remove that friction by generating click, tap and scroll heatmaps automatically for every page and variation in your experiment.
- Source 43: support.convert.com/hc/en-us/articles/heatmaps-native-user-interaction-visualizations-in-convert
Convert Signals is a session recording feature embedded in the Reports and Locations section of Convert Experiences.
Migrate from Convert
Your Convert history can be imported and is reread by Probant’s engine: every verdict is recalculated from visitors and conversions.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway