Skip to content
Hosted in Europe. Your test data is stored in the European Union.Hosted and stored in EuropeLearn more

Probant vs. Amplitude

Both tools compared point by point, based on Amplitude’s public documentation, accessed November 11, 2026. Every statement about Amplitude links to its source.

57 sources from 27 official pages, accessed November 11, 2026.

What Amplitude offers.

Amplitude connects analytics, feedback, experimentation, and code in one suite.1 The company is listed on Nasdaq; Web Experiment is aimed mainly at marketers.23 Sequential testing is its default method; a t-test and a Bayesian reading are available.45

Probant is for sites that want A/B test verdicts that hold up, with a plan set before launch and guardrails monitored through rollout. It measures what each conversion is worth: the margin on an order, a qualified or signed lead, a paid subscription.

The differences, point by point.

On the left, what Amplitude’s public documentation says, each sentence with its source. On the right, what Probant does. “Not mentioned” means nothing was found in that documentation, which does not prove a feature is missing.

Probant and Amplitude, point by point. Note numbers link to the sources at the bottom of the page.
CriterionAmplitudeProbant
Statistical methodThree methods to choose from: sequential test (default), fixed-horizon t-test, and Bayesian; Thompson Sampling for bandits.45Frequentist, with a sequential plan set before launch: target effect, duration, and checkpoints. The verdict fits in one sentence, with a 95% range. Where to check
PeekingThe default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67Decisions are only made at planned checkpoints, with O’Brien-Fleming boundaries: very strict at first, close to the usual threshold at the end. During a rollout, ranges are always valid (mSPRT), so you can check them at any time. Where to check
Traffic split (SRM)SRM is detected by a sequential chi-square test (alpha = 0.01) and notified by email or Slack; the documentation suggests treating the results as suspect but does not describe blocking.8910The traffic split is checked with a chi-square test every time the report is read. If the split is off, the test moves to “blocked” status and no winner is declared. Where to check
GuardrailsYou can declare “guardrail” metrics that must not degrade, but no automatic stop or rollback is documented.1112Guardrail metrics are tracked with their range; a guardrail is only breached if the whole range crosses the limit. A breached guardrail blocks the test’s verdict, stops a running test on its own, and during a rollout moves visitors back to the original automatically. Where to check
Gradual rolloutsRollout by percentage and by segment, with manual rollback; no automatic steps or automatic rollback on metric degradation are documented.1314Steps of 25% → 50% → 100% by default, 10% → 25% → 50% → 100% in cautious mode. Automatic rollback when a guardrail is breached; an exposed visitor stays exposed in the following steps. Where to check
Multi-armed banditMulti-armed bandits with Thompson Sampling and hourly, daily, or weekly reallocation; a paid option on the Growth and Enterprise plans.151617Optimization test: Thompson sampling, reallocated at most once a day, with a minimum share per variant. It maximizes a conversion rate, or an amount per visitor such as revenue: purchase probability and average order value drawn from their distributions, with values capped at a percentile as in an A/B test. Optionally, shares per device (mobile, tablet, desktop): one draw per device, each with its own learning and floor. A visitor always keeps their variant: a new split only applies to visitors who arrive after it. The screen says so: it maximizes conversions (or the amount per visitor) during the test, without proving that a variant is better. The difference is read period by period (and device by device), with no declared winner; the CSV export gives the same reading. Where to check
Revenue and margin per visitorRevenue metrics (average order value, revenue per user) through “sum” or “property average,” with a confidence interval on the lift.181920 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026).Revenue and margin per visitor, each with its range; the gain is stated in euros. Margin is calculated on the server from reconciled orders, the cost of each product (Shopify or CSV), and store fees; no margin verdict below 90% of revenue covered by known costs. Where to check
Winner’s curseA quality check flags suspicious lifts (“suspiciously high uplift detection”).21 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).The reported gain is corrected for the winner’s curse by Bayesian shrinkage toward the effects of the store’s past tests. Where to check
Bayesian modeBayesian mode with a Beta(1,1) or non-informative normal prior, custom priors per metric, probability to beat control and to be best, and expected loss control.222324Optional, off by default: a Bayesian reading next to the verdict, with the probability that the variant does better, the credible interval, and the expected loss of each choice. The prior is centered on zero and as wide as the effects measured by the store’s completed tests; while there are too few of them to estimate it, the Bayesian reading is not shown. The decision always follows the sealed plan; when the two readings disagree, the report explains why. Where to check
Multiple comparisonsBonferroni correction on by default, applied to the number of variants and of primary/secondary metrics; it is off in Bayesian mode.252627Segment comparisons are corrected with the Holm method. With several variants, the error risk is split across comparisons (Bonferroni). Where to check
Multivariate testingThe product page mentions multivariate tests; the documentation describes A/B/n tests, and a factorial design or interaction report is not mentioned.28Full factorial design generated by Probant: two to four sections, each with two to four levels, up to sixteen combinations, each served as a variant. Main effects of each section across all traffic and interactions between two sections, corrected with the Holm method on the comparisons stated in the plan. Based on the workspace’s measured traffic, the plan says whether the test can reach a conclusion and suggests removing a level or a section; no best combination is declared without a check. Where to check
Variance reduction (CUPED)CUPED is available on all plans; it is not supported in Bayesian mode or, by default, for retention.29303132CUPED applied to live test reports, using each visitor’s history before their first exposure; without history, the difference stays unadjusted and the report says so. Where to check
Visual editorVisual editor on all plans, which loads the site with an overlay (the script must be installed; blocked by ad blockers). The custom code editor is limited to the Growth and Enterprise plans.333435Two ways to edit, producing the same changes: a page preview loaded by Probant, without its scripts, or the live page on the site, scripts included. On the site, the editor is loaded by the script already installed, through a short-lived signed link for one draft and one variant, with no browser extension. It only sends the chosen changes to Probant. Where to check
WidgetsLibrary of ready-made widgets (call-to-action button, banner, banner with button, modal pop-up) that can be customized in the editor and reused. No countdown or social proof is documented.3637Banner, countdown, pop-up, badge, reassurance, action bar, delivery date, social proof, promo code, free shipping, and survey, set up without code in the editor and saved in the workspace library for other tests. A widget is just a set of ordinary changes: escaped text, links over https or to the site, and styles applied without a style attribute, so a strict CSP does not need loosening. The countdown follows the site’s time zone and disappears at the deadline; the pop-in opens on arrival, on exit intent, or after scrolling, closes with the keyboard, and does not reappear during the visit if consent allows. Their small modules only load for visitors in the variant. The delivery date is calculated in the visitor’s browser (cutoff time, business days, public holidays); social proof shows the exact number of purchases received over a rolling week, and only above a threshold. The survey asks up to 8 questions (closed answers, recommendation scale, open question) with conditional logic, on the page or through a link outside the site (email, QR code), based on 70 templates (recommendation, satisfaction, effort, checkout blockers, cancellation, concept test, multi-question surveys…); open answers are redacted before sending (email addresses, phone numbers, cards, IBANs), kept for 180 days, and grouped by theme; each question is read by variant, with its range and response rate, and comparisons across all questions are corrected together; a response through a link is attached to the variant the visitor saw, recalculated by Probant, or stays outside the test. Where to check
Script and flickerSynchronous script placed at the top of the <head> (recommended) or asynchronous with a snippet that hides the page until the changes apply or a timeout expires. Published base size: 79 KB uncompressed, 20 KB compressed.383940Script under 7 KB compressed, loaded asynchronously, that does not hide the page while it loads. Server-side assignment available, for a display with no flicker at all. Where to check
Granular permissionsRoles and permissions per product on Growth; creating and customizing roles (extended RBAC) is limited to the Enterprise plan.41Four fixed roles, plus custom roles the owner builds from a catalog of named permissions: prepare, launch and stop, roll out, approve a launch, write variant JavaScript, manage flags, metrics, margin, members, and keys. No privilege escalation: billing, roles, transfer, and deletion of the workspace stay with the owner; no one can grant more than their own permissions, and an API key cannot exceed its author’s. Where to check
Data hostingChoice between US and EU data centers; the isolated EU data center is hosted by AWS in Frankfurt, and the Web Experiment script has a dedicated EU URL.4243Database and compute in Amsterdam, with Scaleway, in the European Union. Scaleway, which hosts the database and compute, is a French company. Published subprocessor list: Scaleway (Amsterdam, Netherlands (nl-ams region), in the European Union), Brevo (European Union: servers in France and Germany, encrypted backups in Belgium (Google Cloud), according to Brevo), Mistral AI (European Union by default (European API endpoint), according to Mistral), ClickHouse (option not used), Stripe (Contract with Stripe Payments Europe, Ltd. (Ireland); data transferred to Stripe, LLC (United States), according to Stripe), Sentry (European Union: Frankfurt (Germany), including backups, according to Sentry). Depending on the feature used, Mistral may temporarily transfer data outside the European Union to the subprocessors it publishes; its data processing agreement limits these transfers to countries the EU recognizes as adequate, or covers them with standard contractual clauses. Stripe transfers payment data to Stripe, LLC, in the United States, under the Data Privacy Framework and, failing that, standard contractual clauses. Whatever region is chosen, Sentry may keep the data of Probant’s Sentry account (members, settings, audit log, project descriptions) in the United States; reported errors stay in Frankfurt. Where to check
Consent by defaultThe Web Experiment script offers a consent mode (storage and impressions kept in memory until acceptance), but it is off by default. It does not cover analytics, which is handled separately.444546Consent required by default: no cookie, no assignment, and nothing sent before the visitor agrees. Consent given mid-page: the page does not change in front of the visitor; their variant starts on the next page. First-party cookie, lasting 390 days at most, never extended. Where to check
Pricing modelPriced by event volume, unlimited seats: Free, Plus (from $0, no stated overage penalty), Growth, and Enterprise on quote; paid add-ons are a percentage of the plan. One or two active experiments depending on the plan, more with an add-on.4748495051Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges. Agency plan: 1,000,000 tested visitors per month, shared across client workspaces, from 3 workspaces, for €1,490 per month or €14,900 per year. Where to check

What Amplitude also does.

Probant shares these points with Amplitude.

  • A sequential test

    The default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67

  • A multiple comparisons correction

    Bonferroni correction on by default, applied to the number of variants and of primary/secondary metrics; it is off in Bayesian mode.252627

  • Variance reduction (CUPED)

    CUPED is available on all plans; it is not supported in Bayesian mode or, by default, for retention.29303132

  • Revenue per visitor, with its range

    Revenue metrics (average order value, revenue per user) through “sum” or “property average,” with a confidence interval on the lift.181920

  • A visual editor

    Visual editor on all plans, which loads the site with an overlay (the script must be installed; blocked by ad blockers). The custom code editor is limited to the Growth and Enterprise plans.333435

When to choose Amplitude over Probant.

Based on Amplitude’s public documentation.

  • You already use Amplitude Analytics

    Amplitude connects analytics, feedback, experimentation, and code in one suite.1

  • You want native session replay and heatmaps

    Native session replay (web and mobile) on all plans, with a monthly quota; native heatmaps from the Plus plan.525354

  • You want a free plan

    Free plan with no credit card or time limit, 2 million events per month, limited to 1 active experiment; no time-limited trial. Eligible startups get one free year of Growth.555657

What sets Probant apart.

These points are not in Amplitude’s public documentation, accessed November 11, 2026. A missing mention does not prove a feature is missing; if these points matter to you, ask Amplitude.

  • Margin per visitor

    Probant measures revenue and margin per visitor, each with its range.

    AmplitudeMargin per visitor is not mentioned in their public documentation (as of November 11, 2026).

    Learn more
  • Winner’s curse correction

    The reported gain is corrected by Bayesian shrinkage toward the effects of the store’s past tests.

    AmplitudeA quality check flags suspicious lifts (“suspiciously high uplift detection”).21 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).

    Learn more
  • SRM that blocks the verdict

    A traffic split imbalance moves the test to “blocked” status, and no winner is declared.

    AmplitudeSRM is detected by a sequential chi-square test (alpha = 0.01) and notified by email or Slack; the documentation suggests treating the results as suspect but does not describe blocking.8910 Automatic blocking of the test is not mentioned there (as of November 11, 2026).

  • Consent required by default

    No cookie, no assignment, and nothing sent before the visitor agrees.

    AmplitudeThe Web Experiment script offers a consent mode (storage and impressions kept in memory until acceptance), but it is off by default. It does not cover analytics, which is handled separately.444546 Consent required by default is not mentioned there (as of November 11, 2026).

Frequently asked questions.

How much does Amplitude cost?

Priced by event volume, unlimited seats: Free, Plus (from $0, no stated overage penalty), Growth, and Enterprise on quote; paid add-ons are a percentage of the plan. One or two active experiments depending on the plan, more with an add-on.4748495051 Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges.

Where does Amplitude host data?

Choice between US and EU data centers; the isolated EU data center is hosted by AWS in Frankfurt, and the Web Experiment script has a dedicated EU URL.4243 Probant hosts its database and application in Amsterdam, with Scaleway.

Does Amplitude measure margin?

Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). Probant measures revenue and margin per visitor, each with its range, from orders and the cost of each product.

Can you look at Amplitude results while a test is running?

The default sequential test is an mSPRT (mixture sequential probability ratio test): you can check results and stop early without inflating false positives. Available on all plans.67 Probant sets its checkpoints before launch (O’Brien-Fleming) and uses always-valid ranges during rollouts.

Sources.

Official pages from Amplitude, accessed November 11, 2026. Quotes are reproduced in their original language.

  1. Source 1: amplitude.com/about
    From analytics to feedback to experimentation to code, Amplitude connects the full improvement loop so every release makes the next one smarter.
  2. Source 2: amplitude.com/press/third-quarter-2026-financial-results
    Amplitude, Inc. (Nasdaq: AMPL), a leading AI-powered platform, today announced financial results for its third quarter ended September 30, 2026.
  3. Source 3: amplitude.com/docs/feature-experiment/feature-vs-web-experimentation
    Web Experiment is most useful for digital marketers and growth marketing.
  4. Source 4: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    For teams running standard product experiments with straightforward decision criteria, Amplitude Experiment defaults to sequential testing, which offers fast results with strong false positive control.
  5. Source 5: amplitude.com/docs/feature-experiment/workflow/finalize-statistical-preferences
    T-Testing: A traditional statistical test that compares the means of two groups, such as the control and treatment groups, to determine if differences are statistically significant.
  6. Source 6: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
    Amplitude Experiment uses a family of sequential tests called mixture sequential probability ratio test (mSPRT).
  7. Source 7: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
    With sequential testing, results stay valid whenever you view them. You can end an experiment early based on observations made to that point.
  8. Source 8: amplitude.com/docs/feature-experiment/troubleshooting/sample-ratio-mismatch
    Amplitude uses a sequential version of a chi-squared test with alpha = .01 to detect an SRM.
  9. Source 9: amplitude.com/docs/feature-experiment/troubleshooting/sample-ratio-mismatch
    Treat the results of any experiment with an SRM as suspect.
  10. Source 10: amplitude.com/docs/feature-experiment/notifications
    Notifications for Experiments (Web and Feature) about to start or end, when Amplitude detects a sample-ratio mismatch (SRM), or when the experiment's recommendation metric reaches statsig.
  11. Source 11: amplitude.com/docs/feature-experiment/key-terms
    A metric you want to protect from regression while you increase your success metrics.
  12. Source 12: amplitude.com/docs/feature-experiment/workflow/define-goals
    Guardrail metrics monitor important metrics that shouldn't degrade during the experiment:
  13. Source 13: amplitude.com/docs/feature-experiment/workflow/feature-flag-rollouts
    Set the rollout percentage for this feature. This is the percentage of users in the flag's user segments who receive the new feature.
  14. Source 14: amplitude.com/docs/feature-experiment
    Release behind a flag, ramp to the audience you choose, and turn it off the moment something looks wrong.
  15. Source 15: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
    Multi-armed bandit experiments use Thompson sampling.
  16. Source 16: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
    Multi-armed Bandits is available as an add-on on the Growth and Enterprise plans.
  17. Source 17: amplitude.com/docs/feature-experiment/workflow/multi-armed-bandit-experiments
    You can configure multi-armed bandit experiments to reallocate traffic hourly, daily, or weekly.
  18. Source 18: amplitude.com/docs/feature-experiment/workflow/define-goals
    Revenue metrics: Average order value, revenue for each user, subscription upgrades.
  19. Source 19: amplitude.com/docs/feature-experiment/under-the-hood/experiment-sequential-testing
    Average Sum of Property sums the values of an event per user on a specific property.
  20. Source 20: amplitude.com/docs/feature-experiment/analysis-view
    The confidence interval is a range of values that includes the parameter you're trying to measure.
  21. Source 21: amplitude.com/docs/feature-experiment
    Suspiciously high uplift detection
  22. Source 22: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    For binary metrics, Amplitude uses a Beta(1,1) prior (a uniform distribution). For continuous metrics, Amplitude uses an uninformative normal prior.
  23. Source 23: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    Chance to Beat Control: the probability that the treatment variant performs better than the control.
  24. Source 24: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    Instead, Bayesian analysis controls expected loss, providing a more nuanced approach to decision-making under uncertainty.
  25. Source 25: amplitude.com/docs/feature-experiment/advanced-techniques/bonferroni-correction
    Amplitude enables the Bonferroni correction by default. You can manually toggle it off in your statistical settings.
  26. Source 26: amplitude.com/docs/feature-experiment/advanced-techniques/bonferroni-correction
    Amplitude Experiment performs Bonferroni corrections on both the number of treatments and the number of primary and secondary metrics:
  27. Source 27: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    Amplitude disables Bonferroni correction when you use this method.
  28. Source 28: amplitude.com/docs/feature-experiment
    Run A/B, multivariate, and bandit tests so the data, not the loudest opinion, makes the call.
  29. Source 29: amplitude.com/docs/feature-experiment/workflow/finalize-statistical-preferences
    Controlled-experiment using pre-existing data, also known as CUPED, is an optional statistical technique that reduces variance in Amplitude Experiment.
  30. Source 31: amplitude.com/docs/feature-experiment/experiment-theory/bayesian-statistics
    Amplitude Experiment's Bayesian implementation doesn't support CUPED (variance reduction).
  31. Source 32: amplitude.com/docs/feature-experiment/workflow/define-goals
    By default, the Retention metric doesn't support CUPED, exposure attribution settings, or calendar day windows.
  32. Source 33: amplitude.com/docs/web-experiment/set-up-a-web-experiment
    Open your site in the Visual Editor, choose the elements you want to experiment with, and change their content or properties directly.
  33. Source 34: amplitude.com/docs/web-experiment/set-up-a-web-experiment
    The Visual Editor loads the site at the URL you specified on experiment creation and adds an overlay.
  34. Source 35: amplitude.com/docs/web-experiment
    Custom Code Editor, Experiment-level access control, and Experiment Results chart are available on the Growth and Enterprise plans.
  35. Source 36: amplitude.com/docs/web-experiment/out-of-the-box-widgets
    Out-of-the-box (OOTB) widgets let you test new website elements like modals and banners without designing or coding them yourself.
  36. Source 37: amplitude.com/docs/web-experiment/out-of-the-box-widgets
    Modal pop-up: a pop-up modal that appears on your page after a set amount of time.
  37. Source 38: amplitude.com/docs/web-experiment/implementation
    Paste the script into the <head> element of your site, as high as possible to avoid flickering.
  38. Source 39: amplitude.com/docs/web-experiment/implementation
    The snippet masks elements on the page until Amplitude applies all changes.
  39. Source 40: amplitude.com/docs/web-experiment/performance
    Base script | 79KB | 20KB
  40. Source 41: amplitude.com/docs/admin/account-management/role-based-access-controls-rbac
    RBAC role customization requires an Enterprise plan. If you aren't on an Enterprise plan, you can't customize roles or create new ones.
  41. Source 42: amplitude.com/docs/data/privacy-and-consent-implementation
    Amplitude offers both US and EU data center options, and that choice may affect your internal review process.
  42. Source 43: amplitude.com/press/amplitude-launches-new-data-centre-in-germany-to-support-rapid-customer-growth-in-the-eu
    Amplitude selected the AWS data centre in Frankfurt because Germany enforces some of the most stringent privacy and technology standards in the world.
  43. Source 44: amplitude.com/docs/web-experiment/cookies-and-consent-management
    consentRequired | false | When true, the script gates device storage and third-party requests on consent.
  44. Source 45: amplitude.com/docs/web-experiment/cookies-and-consent-management
    While consent is undecided, the script still evaluates experiments and applies variants with no flicker, but it holds every cookie and browser storage write in memory, buffers impression events, and doesn't contact any third-party origin.
  45. Source 46: amplitude.com/docs/web-experiment/cookies-and-consent-management
    Consent gating covers the Web Experiment script only.
  46. Source 47: amplitude.com/pricing#faq
    The Amplitude Plus plan starts at $0 and scales with your event volume as you grow.
  47. Source 48: amplitude.com/pricing#faq
    For larger teams, the Growth and Enterprise plans offer custom pricing based on your volume and feature needs
  48. Source 49: amplitude.com/pricing
    Amplitude Plus lets your team keep sending the data that matters as you grow, without overage penalties, surprise renewal spikes, or pressure to cut back on instrumentation.
  49. Source 50: amplitude.com/pricing
    Expanded packages are priced as a percentage of your platform plan.
  50. Source 51: amplitude.com/pricing
    Active experiments | 1 | 1 | 2 Add-on for more | 2 Add-on for more
  51. Source 52: amplitude.com/docs/session-replay
    Session Replay records user sessions and plays them back alongside the events you track in Amplitude.
  52. Source 53: amplitude.com/docs/session-replay
    To see aggregate clicks and scrolls on a page, use Heatmaps
  53. Source 54: amplitude.com/pricing
    Heatmaps | Not included | Included | Included | Included
  54. Source 55: amplitude.com/pricing#faq
    Amplitude offers something better than a trial: a free plan with no time limit.
  55. Source 56: amplitude.com/pricing#faq
    The Free plan gives you 2 million events per month, with access to core analytics, session replay, feature flags, and more.
  56. Source 57: amplitude.com/pricing
    Startups with under $10M in funding and fewer than 20 employees receive one free year on our Growth plan.