Probant vs. Amplitude
Both tools compared point by point, based on Amplitude’s public documentation, accessed November 11, 2026. Every statement about Amplitude links to its source.
What Amplitude offers.
Amplitude connects analytics, feedback, experimentation, and code in one suite.1 The company is listed on Nasdaq; Web Experiment is aimed mainly at marketers.23 Sequential testing is its default method; a t-test and a Bayesian reading are available.45
Probant is for sites that want A/B test verdicts that hold up, with a plan set before launch and guardrails monitored through rollout. It measures what each conversion is worth: the margin on an order, a qualified or signed lead, a paid subscription.
The differences, point by point.
On the left, what Amplitude’s public documentation says, each sentence with its source. On the right, what Probant does. “Not mentioned” means nothing was found in that documentation, which does not prove a feature is missing.
| Criterion | Amplitude | Probant |
|---|---|---|
| Statistical method | Three methods to choose from: sequential test (default), fixed-horizon t-test, and Bayesian; Thompson Sampling for bandits.45 | Frequentist, with a sequential plan set before launch: target effect, duration, and checkpoints. The verdict fits in one sentence, with a 95% range. Where to check |
| Peeking | The default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67 | Decisions are only made at planned checkpoints, with O’Brien-Fleming boundaries: very strict at first, close to the usual threshold at the end. During a rollout, ranges are always valid (mSPRT), so you can check them at any time. Where to check |
| Traffic split (SRM) | SRM is detected by a sequential chi-square test (alpha = 0.01) and notified by email or Slack; the documentation suggests treating the results as suspect but does not describe blocking.8910 | The traffic split is checked with a chi-square test every time the report is read. If the split is off, the test moves to “blocked” status and no winner is declared. Where to check |
| Guardrails | You can declare “guardrail” metrics that must not degrade, but no automatic stop or rollback is documented.1112 | Guardrail metrics are tracked with their range; a guardrail is only breached if the whole range crosses the limit. A breached guardrail blocks the test’s verdict, stops a running test on its own, and during a rollout moves visitors back to the original automatically. Where to check |
| Gradual rollouts | Rollout by percentage and by segment, with manual rollback; no automatic steps or automatic rollback on metric degradation are documented.1314 | Steps of 25% → 50% → 100% by default, 10% → 25% → 50% → 100% in cautious mode. Automatic rollback when a guardrail is breached; an exposed visitor stays exposed in the following steps. Where to check |
| Multi-armed bandit | Multi-armed bandits with Thompson Sampling and hourly, daily, or weekly reallocation; a paid option on the Growth and Enterprise plans.151617 | Optimization test: Thompson sampling, reallocated at most once a day, with a minimum share per variant. It maximizes a conversion rate, or an amount per visitor such as revenue: purchase probability and average order value drawn from their distributions, with values capped at a percentile as in an A/B test. Optionally, shares per device (mobile, tablet, desktop): one draw per device, each with its own learning and floor. A visitor always keeps their variant: a new split only applies to visitors who arrive after it. The screen says so: it maximizes conversions (or the amount per visitor) during the test, without proving that a variant is better. The difference is read period by period (and device by device), with no declared winner; the CSV export gives the same reading. Where to check |
| Revenue and margin per visitor | Revenue metrics (average order value, revenue per user) through “sum” or “property average,” with a confidence interval on the lift.181920 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). | Revenue and margin per visitor, each with its range; the gain is stated in euros. Margin is calculated on the server from reconciled orders, the cost of each product (Shopify or CSV), and store fees; no margin verdict below 90% of revenue covered by known costs. Where to check |
| Winner’s curse | A quality check flags suspicious lifts (“suspiciously high uplift detection”).21 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026). | The reported gain is corrected for the winner’s curse by Bayesian shrinkage toward the effects of the store’s past tests. Where to check |
| Bayesian mode | Bayesian mode with a Beta(1,1) or non-informative normal prior, custom priors per metric, probability to beat control and to be best, and expected loss control.222324 | Optional, off by default: a Bayesian reading next to the verdict, with the probability that the variant does better, the credible interval, and the expected loss of each choice. The prior is centered on zero and as wide as the effects measured by the store’s completed tests; while there are too few of them to estimate it, the Bayesian reading is not shown. The decision always follows the sealed plan; when the two readings disagree, the report explains why. Where to check |
| Multiple comparisons | Bonferroni correction on by default, applied to the number of variants and of primary/secondary metrics; it is off in Bayesian mode.252627 | Segment comparisons are corrected with the Holm method. With several variants, the error risk is split across comparisons (Bonferroni). Where to check |
| Multivariate testing | The product page mentions multivariate tests; the documentation describes A/B/n tests, and a factorial design or interaction report is not mentioned.28 | Full factorial design generated by Probant: two to four sections, each with two to four levels, up to sixteen combinations, each served as a variant. Main effects of each section across all traffic and interactions between two sections, corrected with the Holm method on the comparisons stated in the plan. Based on the workspace’s measured traffic, the plan says whether the test can reach a conclusion and suggests removing a level or a section; no best combination is declared without a check. Where to check |
| Variance reduction (CUPED) | CUPED is available on all plans; it is not supported in Bayesian mode or, by default, for retention.29303132 | CUPED applied to live test reports, using each visitor’s history before their first exposure; without history, the difference stays unadjusted and the report says so. Where to check |
| Visual editor | Visual editor on all plans, which loads the site with an overlay (the script must be installed; blocked by ad blockers). The custom code editor is limited to the Growth and Enterprise plans.333435 | Two ways to edit, producing the same changes: a page preview loaded by Probant, without its scripts, or the live page on the site, scripts included. On the site, the editor is loaded by the script already installed, through a short-lived signed link for one draft and one variant, with no browser extension. It only sends the chosen changes to Probant. Where to check |
| Widgets | Library of ready-made widgets (call-to-action button, banner, banner with button, modal pop-up) that can be customized in the editor and reused. No countdown or social proof is documented.3637 | Banner, countdown, pop-up, badge, reassurance, action bar, delivery date, social proof, promo code, free shipping, and survey, set up without code in the editor and saved in the workspace library for other tests. A widget is just a set of ordinary changes: escaped text, links over https or to the site, and styles applied without a style attribute, so a strict CSP does not need loosening. The countdown follows the site’s time zone and disappears at the deadline; the pop-in opens on arrival, on exit intent, or after scrolling, closes with the keyboard, and does not reappear during the visit if consent allows. Their small modules only load for visitors in the variant. The delivery date is calculated in the visitor’s browser (cutoff time, business days, public holidays); social proof shows the exact number of purchases received over a rolling week, and only above a threshold. The survey asks up to 8 questions (closed answers, recommendation scale, open question) with conditional logic, on the page or through a link outside the site (email, QR code), based on 70 templates (recommendation, satisfaction, effort, checkout blockers, cancellation, concept test, multi-question surveys…); open answers are redacted before sending (email addresses, phone numbers, cards, IBANs), kept for 180 days, and grouped by theme; each question is read by variant, with its range and response rate, and comparisons across all questions are corrected together; a response through a link is attached to the variant the visitor saw, recalculated by Probant, or stays outside the test. Where to check |
| Script and flicker | Synchronous script placed at the top of the <head> (recommended) or asynchronous with a snippet that hides the page until the changes apply or a timeout expires. Published base size: 79 KB uncompressed, 20 KB compressed.383940 | Script under 7 KB compressed, loaded asynchronously, that does not hide the page while it loads. Server-side assignment available, for a display with no flicker at all. Where to check |
| Granular permissions | Roles and permissions per product on Growth; creating and customizing roles (extended RBAC) is limited to the Enterprise plan.41 | Four fixed roles, plus custom roles the owner builds from a catalog of named permissions: prepare, launch and stop, roll out, approve a launch, write variant JavaScript, manage flags, metrics, margin, members, and keys. No privilege escalation: billing, roles, transfer, and deletion of the workspace stay with the owner; no one can grant more than their own permissions, and an API key cannot exceed its author’s. Where to check |
| Data hosting | Choice between US and EU data centers; the isolated EU data center is hosted by AWS in Frankfurt, and the Web Experiment script has a dedicated EU URL.4243 | Database and compute in Amsterdam, with Scaleway, in the European Union. Scaleway, which hosts the database and compute, is a French company. Published subprocessor list: Scaleway (Amsterdam, Netherlands (nl-ams region), in the European Union), Brevo (European Union: servers in France and Germany, encrypted backups in Belgium (Google Cloud), according to Brevo), Mistral AI (European Union by default (European API endpoint), according to Mistral), ClickHouse (option not used), Stripe (Contract with Stripe Payments Europe, Ltd. (Ireland); data transferred to Stripe, LLC (United States), according to Stripe), Sentry (European Union: Frankfurt (Germany), including backups, according to Sentry). Depending on the feature used, Mistral may temporarily transfer data outside the European Union to the subprocessors it publishes; its data processing agreement limits these transfers to countries the EU recognizes as adequate, or covers them with standard contractual clauses. Stripe transfers payment data to Stripe, LLC, in the United States, under the Data Privacy Framework and, failing that, standard contractual clauses. Whatever region is chosen, Sentry may keep the data of Probant’s Sentry account (members, settings, audit log, project descriptions) in the United States; reported errors stay in Frankfurt. Where to check |
| Consent by default | The Web Experiment script offers a consent mode (storage and impressions kept in memory until acceptance), but it is off by default. It does not cover analytics, which is handled separately.444546 | Consent required by default: no cookie, no assignment, and nothing sent before the visitor agrees. Consent given mid-page: the page does not change in front of the visitor; their variant starts on the next page. First-party cookie, lasting 390 days at most, never extended. Where to check |
| Pricing model | Priced by event volume, unlimited seats: Free, Plus (from $0, no stated overage penalty), Growth, and Enterprise on quote; paid add-ons are a percentage of the plan. One or two active experiments depending on the plan, more with an add-on.4748495051 | Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges. Agency plan: 1,000,000 tested visitors per month, shared across client workspaces, from 3 workspaces, for €1,490 per month or €14,900 per year. Where to check |
What Amplitude also does.
Probant shares these points with Amplitude.
A sequential test
The default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67
A multiple comparisons correction
Bonferroni correction on by default, applied to the number of variants and of primary/secondary metrics; it is off in Bayesian mode.252627
Variance reduction (CUPED)
CUPED is available on all plans; it is not supported in Bayesian mode or, by default, for retention.29303132
Revenue per visitor, with its range
Revenue metrics (average order value, revenue per user) through “sum” or “property average,” with a confidence interval on the lift.181920
A visual editor
Visual editor on all plans, which loads the site with an overlay (the script must be installed; blocked by ad blockers). The custom code editor is limited to the Growth and Enterprise plans.333435
When to choose Amplitude over Probant.
Based on Amplitude’s public documentation.
You already use Amplitude Analytics
Amplitude connects analytics, feedback, experimentation, and code in one suite.1
You want native session replay and heatmaps
Native session replay (web and mobile) on all plans, with a monthly quota; native heatmaps from the Plus plan.525354
You want a free plan
Free plan with no credit card or time limit, 2 million events per month, limited to 1 active experiment; no time-limited trial. Eligible startups get one free year of Growth.555657
What sets Probant apart.
These points are not in Amplitude’s public documentation, accessed November 11, 2026. A missing mention does not prove a feature is missing; if these points matter to you, ask Amplitude.
Margin per visitor
Probant measures revenue and margin per visitor, each with its range.
AmplitudeMargin per visitor is not mentioned in their public documentation (as of November 11, 2026).
Learn moreWinner’s curse correction
The reported gain is corrected by Bayesian shrinkage toward the effects of the store’s past tests.
AmplitudeA quality check flags suspicious lifts (“suspiciously high uplift detection”).21 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).
Learn moreSRM that blocks the verdict
A traffic split imbalance moves the test to “blocked” status, and no winner is declared.
AmplitudeSRM is detected by a sequential chi-square test (alpha = 0.01) and notified by email or Slack; the documentation suggests treating the results as suspect but does not describe blocking.8910 Automatic blocking of the test is not mentioned there (as of November 11, 2026).
Consent required by default
No cookie, no assignment, and nothing sent before the visitor agrees.
AmplitudeThe Web Experiment script offers a consent mode (storage and impressions kept in memory until acceptance), but it is off by default. It does not cover analytics, which is handled separately.444546 Consent required by default is not mentioned there (as of November 11, 2026).
Frequently asked questions.
How much does Amplitude cost?
Priced by event volume, unlimited seats: Free, Plus (from $0, no stated overage penalty), Growth, and Enterprise on quote; paid add-ons are a percentage of the plan. One or two active experiments depending on the plan, more with an add-on.4748495051 Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges.
Where does Amplitude host data?
Choice between US and EU data centers; the isolated EU data center is hosted by AWS in Frankfurt, and the Web Experiment script has a dedicated EU URL.4243 Probant hosts its database and application in Amsterdam, with Scaleway.
Does Amplitude measure margin?
Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). Probant measures revenue and margin per visitor, each with its range, from orders and the cost of each product.
Can you look at Amplitude results while a test is running?
The default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67 Probant sets its checkpoints before launch (O’Brien-Fleming) and uses always-valid ranges during rollouts.
Sources.
Official pages from Amplitude, accessed November 11, 2026. Quotes are reproduced in their original language.
- Source 1: amplitude.com/about
From analytics to feedback to experimentation to code, Amplitude connects the full improvement loop so every release makes the next one smarter.
- Source 2: amplitude.com/press/third-quarter-2026-financial-results
Amplitude, Inc. (Nasdaq: AMPL), a leading AI-powered platform, today announced financial results for its third quarter ended September 30, 2026.
- Source 3: amplitude.com/docs/feature-experiment/feature-vs-web-experimentation
Web Experiment is most useful for digital marketers and growth marketing.
- Source 4: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
For teams running standard product experiments with straightforward decision criteria, Amplitude Experiment defaults to sequential testing, which offers fast results with strong false positive control.
- Source 5: amplitude.com/docs/feature-experiment/workflow/finalize-statistical-preferences
T-Testing: A traditional statistical test that compares the means of two groups, such as the control and treatment groups, to determine if differences are statistically significant.
- Source 6: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
Amplitude Experiment uses a family of sequential tests called mixture sequential probability ratio test (mSPRT).
- Source 7: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
With sequential testing, results stay valid whenever you view them. You can end an experiment early based on observations made to that point.
- Source 8: amplitude.com/docs/feature-experiment/troubleshooting/sample-ratio-mismatch
Amplitude uses a sequential version of a chi-squared test with alpha = .01 to detect an SRM.
- Source 9: amplitude.com/docs/feature-experiment/troubleshooting/sample-ratio-mismatch
Treat the results of any experiment with an SRM as suspect.
- Source 10: amplitude.com/docs/feature-experiment/notifications
Notifications for Experiments (Web and Feature) about to start or end, when Amplitude detects a sample-ratio mismatch (SRM), or when the experiment's recommendation metric reaches statsig.
- Source 11: amplitude.com/docs/feature-experiment/key-terms
A metric you want to protect from regression while you increase your success metrics.
- Source 12: amplitude.com/docs/feature-experiment/workflow/define-goals
Guardrail metrics monitor important metrics that shouldn't degrade during the experiment:
- Source 13: amplitude.com/docs/feature-experiment/workflow/feature-flag-rollouts
Set the rollout percentage for this feature. This is the percentage of users in the flag's user segments who receive the new feature.
- Source 14: amplitude.com/docs/feature-experiment
Release behind a flag, ramp to the audience you choose, and turn it off the moment something looks wrong.
- Source 15: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
Multi-armed bandit experiments use Thompson sampling.
- Source 16: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
Multi-armed Bandits is available as an add-on on the Growth and Enterprise plans.
- Source 17: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
You can configure multi-armed bandit experiments to reallocate traffic hourly, daily, or weekly.
- Source 18: amplitude.com/docs/feature-experiment/workflow/define-goals
Revenue metrics: Average order value, revenue for each user, subscription upgrades.
- Source 19: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
Average Sum of Property sums the values of an event per user on a specific property.
- Source 20: amplitude.com/docs/feature-experiment/analysis-view
The confidence interval is a range of values that includes the parameter you're trying to measure.
- Source 22: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
For binary metrics, Amplitude uses a Beta(1,1) prior (a uniform distribution). For continuous metrics, Amplitude uses an uninformative normal prior.
- Source 23: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
Chance to Beat Control: the probability that the treatment variant performs better than the control.
- Source 24: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
Instead, Bayesian analysis controls expected loss, providing a more nuanced approach to decision-making under uncertainty.
- Source 25: amplitude.com/docs/feature-experiment/advanced-techniques/bonferroni-correction
Amplitude enables the Bonferroni correction by default. You can manually toggle it off in your statistical settings.
- Source 26: amplitude.com/docs/feature-experiment/advanced-techniques/bonferroni-correction
Amplitude Experiment performs Bonferroni corrections on both the number of treatments and the number of primary and secondary metrics:
- Source 27: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
Amplitude disables Bonferroni correction when you use this method.
- Source 28: amplitude.com/docs/feature-experiment
Run A/B, multivariate, and bandit tests so the data, not the loudest opinion, makes the call.
- Source 29: amplitude.com/docs/feature-experiment/workflow/finalize-statistical-preferences
Controlled-experiment using pre-existing data, also known as CUPED, is an optional statistical technique that reduces variance in Amplitude Experiment.
- Source 30: amplitude.com/docs/feature-experiment/workflow/finalize-statistical-preferences
CUPED is available on all plans.
- Source 31: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
Amplitude Experiment's Bayesian implementation doesn't support CUPED (variance reduction).
- Source 32: amplitude.com/docs/feature-experiment/workflow/define-goals
By default, the Retention metric doesn't support CUPED, exposure attribution settings, or calendar day windows.
- Source 33: amplitude.com/docs/web-experiment/set-up-a-web-experiment
Open your site in the Visual Editor, choose the elements you want to experiment with, and change their content or properties directly.
- Source 34: amplitude.com/docs/web-experiment/set-up-a-web-experiment
The Visual Editor loads the site at the URL you specified on experiment creation and adds an overlay.
- Source 35: amplitude.com/docs/web-experiment
Custom Code Editor, Experiment-level access control, and Experiment Results chart are available on the Growth and Enterprise plans.
- Source 36: amplitude.com/docs/web-experiment/out-of-the-box-widgets
Out-of-the-box (OOTB) widgets let you test new website elements like modals and banners without designing or coding them yourself.
- Source 37: amplitude.com/docs/web-experiment/out-of-the-box-widgets
Modal pop-up: a pop-up modal that appears on your page after a set amount of time.
- Source 38: amplitude.com/docs/web-experiment/implementation
Paste the script into the <head> element of your site, as high as possible to avoid flickering.
- Source 39: amplitude.com/docs/web-experiment/implementation
The snippet masks elements on the page until Amplitude applies all changes.
- Source 41: amplitude.com/docs/admin/account-management/role-based-access-controls-rbac
RBAC role customization requires an Enterprise plan. If you aren't on an Enterprise plan, you can't customize roles or create new ones.
- Source 42: amplitude.com/docs/data/privacy-and-consent-implementation
Amplitude offers both US and EU data center options, and that choice may affect your internal review process.
- Source 43: amplitude.com/press/amplitude-launches-new-data-centre-in-germany-to-support-rapid-customer-growth-in-the-eu
Amplitude selected the AWS data centre in Frankfurt because Germany enforces some of the most stringent privacy and technology standards in the world.
- Source 44: amplitude.com/docs/web-experiment/cookies-and-consent-management
consentRequired | false | When true, the script gates device storage and third-party requests on consent.
- Source 45: amplitude.com/docs/web-experiment/cookies-and-consent-management
While consent is undecided, the script still evaluates experiments and applies variants with no flicker, but it holds every cookie and browser storage write in memory, buffers impression events, and doesn't contact any third-party origin.
- Source 46: amplitude.com/docs/web-experiment/cookies-and-consent-management
Consent gating covers the Web Experiment script only.
- Source 47: amplitude.com/pricing#faq
The Amplitude Plus plan starts at $0 and scales with your event volume as you grow.
- Source 48: amplitude.com/pricing#faq
For larger teams, the Growth and Enterprise plans offer custom pricing based on your volume and feature needs
- Source 49: amplitude.com/pricing
Amplitude Plus lets your team keep sending the data that matters as you grow, without overage penalties, surprise renewal spikes, or pressure to cut back on instrumentation.
- Source 52: amplitude.com/docs/session-replay
Session Replay records user sessions and plays them back alongside the events you track in Amplitude.
- Source 53: amplitude.com/docs/session-replay
To see aggregate clicks and scrolls on a page, use Heatmaps
- Source 55: amplitude.com/pricing#faq
Amplitude offers something better than a trial: a free plan with no time limit.
- Source 56: amplitude.com/pricing#faq
The Free plan gives you 2 million events per month, with access to core analytics, session replay, feature flags, and more.
- Source 57: amplitude.com/pricing
Startups with under $10M in funding and fewer than 20 employees receive one free year on our Growth plan.
Migrate from Amplitude
Your Amplitude history can be imported as a generic CSV (visitors and conversions per variant) and is reread by Probant’s engine: every verdict is recalculated from visitors and conversions.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway