Probant vs. PostHog
Both tools compared point by point, based on PostHog’s public documentation, accessed November 11, 2026. Every statement about PostHog links to its source.
What PostHog offers.
PostHog combines product analytics, session replay, feature flags, A/B testing, surveys, and a data warehouse in one suite.1 The company was founded in 2020 and its team is fully remote.23 Its tests are Bayesian by default, with frequentist as an option.45
Probant is for sites that want A/B test verdicts that hold up, with a plan set before launch and guardrails monitored through rollout. It measures what each conversion is worth: the margin on an order, a qualified or signed lead, a paid subscription.
The differences, point by point.
On the left, what PostHog’s public documentation says, each sentence with its source. On the right, what Probant does. “Not mentioned” means nothing was found in that documentation, which does not prove a feature is missing.
| Criterion | PostHog | Probant |
|---|---|---|
| Statistical method | Two engines: Bayesian (default) and frequentist (t-tests and confidence intervals). Sequential testing is an option of the frequentist engine.45 | Frequentist, with a sequential plan set before launch: target effect, duration, and checkpoints. The verdict fits in one sentence, with a 95% range. Where to check |
| Peeking | Sequential testing with “confidence sequences” (Waudby-Smith et al. 2023) in the frequentist engine, which can be turned on per test or by default at the project level; off by default.678 | Decisions are only made at planned checkpoints, with O’Brien-Fleming boundaries: very strict at first, close to the usual threshold at the end. During a rollout, ranges are always valid (mSPRT), so you can check them at any time. Where to check |
| Traffic split (SRM) | Automatic chi-square test from 100 exposures, with a yellow warning if p < 0.001; the result is not blocked, and the docs ask you to investigate. An AI “scout” can also flag an SRM in the Self-driving inbox.9101112 | The traffic split is checked with a chi-square test every time the report is read. If the split is off, the test moves to “blocked” status and no winner is declared. Where to check |
| Guardrails | No dedicated “guardrail” type is documented: secondary metrics are used to check for negative effects and work like primary ones. No automatic stop of a test on degradation is documented.1314 | Guardrail metrics are tracked with their range; a guardrail is only breached if the whole range crosses the limit. A breached guardrail blocks the test’s verdict, stops a running test on its own, and during a rollout moves visitors back to the original automatically. Where to check |
| Gradual rollouts | Gradual rollout by percentage and scheduled changes (recurring or cron). No automatic rollback on metric degradation is documented.1516 | Steps of 25% → 50% → 100% by default, 10% → 25% → 50% → 100% in cautious mode. Automatic rollback when a guardrail is breached; an exposed visitor stays exposed in the following steps. Where to check |
| Multi-armed bandit | Dynamic allocation is not mentioned in their public documentation (as of November 11, 2026). | Optimization test: Thompson sampling, reallocated at most once a day, with a minimum share per variant. It maximizes a conversion rate, or an amount per visitor such as revenue: purchase probability and average order value drawn from their distributions, with values capped at a percentile as in an A/B test. Optionally, shares per device (mobile, tablet, desktop): one draw per device, each with its own learning and floor. A visitor always keeps their variant: a new split only applies to visitors who arrive after it. The screen says so: it maximizes conversions (or the amount per visitor) during the test, without proving that a variant is better. The difference is read period by period (and device by device), with no declared winner; the CSV export gives the same reading. Where to check |
| Revenue and margin per visitor | Mean metrics (revenue per user) and ratios (revenue per order) shown with a 95% confidence interval (frequentist) or a credible interval (Bayesian).171819 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). | Revenue and margin per visitor, each with its range; the gain is stated in euros. Margin is calculated on the server from reconciled orders, the cost of each product (Shopify or CSV), and store fees; no margin verdict below 90% of revenue covered by known costs. Where to check |
| Winner’s curse | The Bayesian engine uses a non-informative prior: results only reflect the observed data.20 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026). | The reported gain is corrected for the winner’s curse by Bayesian shrinkage toward the effects of the store’s past tests. Where to check |
| Bayesian mode | Bayesian engine by default with probability to beat control (“chance to win”) and a 95% credible interval. Fixed non-informative prior, with no documented choice of prior; no expected loss mentioned.2021 | Optional, off by default: a Bayesian reading next to the verdict, with the probability that the variant does better, the credible interval, and the expected loss of each choice. The prior is centered on zero and as wide as the effects measured by the store’s completed tests; while there are too few of them to estimate it, the Bayesian reading is not shown. The decision always follows the sealed plan; when the two readings disagree, the report explains why. Where to check |
| Multiple comparisons | The Bayesian docs state that no multiple comparisons correction is applied; the frequentist docs state that none is applied “by default,” without describing an option. They advise checking consistency across metrics.222324 | Segment comparisons are corrected with the Holm method. With several variants, the error risk is split across comparisons (Bonferroni). Where to check |
| Multivariate testing | The docs use “multivariate” tests to mean flags with several variants (2 to 20). No factorial design or interaction report is documented.2526 | Full factorial design generated by Probant: two to four sections, each with two to four levels, up to sixteen combinations, each served as a variant. Main effects of each section across all traffic and interactions between two sections, corrected with the Holm method on the comparisons stated in the plan. Based on the workspace’s measured traffic, the plan says whether the test can reach a conclusion and suggests removing a level or a section; no best combination is declared without a check. Where to check |
| Variance reduction (CUPED) | Optional CUPED, set per project and per test, on mean metrics and ordered funnels; ratios, retention, and unordered funnels are not adjusted. 14-day pre-exposure window by default (1 to 30).272829 | CUPED applied to live test reports, using each visitor’s history before their first exposure; without history, the difference stays unadjusted and the report says so. Where to check |
| Visual editor | No-code web experiments in the PostHog toolbar or the Chrome extension: change the text, CSS, or HTML of elements. Frameworks that often re-render or replace DOM elements can overwrite these changes. Must be turned on in the snippet (disable_web_experiments: false).30313233 | Two ways to edit, producing the same changes: a page preview loaded by Probant, without its scripts, or the live page on the site, scripts included. On the site, the editor is loaded by the script already installed, through a short-lived signed link for one draft and one variant, with no browser extension. It only sends the chosen changes to Probant. Where to check |
| Widgets | Built-in surveys (NPS, CSAT, and multiple-choice pop-ups…) with no code. No ready-made marketing widget (banner, countdown, social proof) is mentioned.34 | Banner, countdown, pop-up, badge, reassurance, action bar, delivery date, social proof, promo code, free shipping, and survey, set up without code in the editor and saved in the workspace library for other tests. A widget is just a set of ordinary changes: escaped text, links over https or to the site, and styles applied without a style attribute, so a strict CSP does not need loosening. The countdown follows the site’s time zone and disappears at the deadline; the pop-in opens on arrival, on exit intent, or after scrolling, closes with the keyboard, and does not reappear during the visit if consent allows. Their small modules only load for visitors in the variant. The delivery date is calculated in the visitor’s browser (cutoff time, business days, public holidays); social proof shows the exact number of purchases received over a rolling week, and only above a threshold. The survey asks up to 8 questions (closed answers, recommendation scale, open question) with conditional logic, on the page or through a link outside the site (email, QR code), based on 70 templates (recommendation, satisfaction, effort, checkout blockers, cancellation, concept test, multi-question surveys…); open answers are redacted before sending (email addresses, phone numbers, cards, IBANs), kept for 180 days, and grouped by theme; each question is read by variant, with its range and response rate, and comparisons across all questions are corrected together; a response through a link is attached to the variant the visitor saw, recalculated by Probant, or stays outside the test. Where to check |
| Script and flicker | Asynchronous client-side evaluation: without precautions, a delay before the variant applies is visible. No anti-flicker script is documented; the docs recommend server-side evaluation and bootstrapping flags. Script size not stated.353637 | Script under 7 KB compressed, loaded asynchronously, that does not hide the page while it loads. Server-side assignment available, for a display with no flicker at all. Where to check |
| Granular permissions | Three organization levels (Member, Admin, Owner). No access control on Free and pay-as-you-go; per-member access on Boost and Scale; roles (RBAC) per project and resource on Enterprise only.383940 | Four fixed roles, plus custom roles the owner builds from a catalog of named permissions: prepare, launch and stop, roll out, approve a launch, write variant JavaScript, manage flags, metrics, margin, members, and keys. No privilege escalation: billing, roles, transfer, and deletion of the workspace stay with the owner; no one can grant more than their own permissions, and an API key cannot exceed its author’s. Where to check |
| Data hosting | Region chosen at sign-up: EU (AWS eu-central-1, Frankfurt) or United States (AWS us-east-1, Virginia). Some subprocessors remain in the US (standard contractual clauses).4142 | Database and compute in Amsterdam, with Scaleway, in the European Union. Scaleway, which hosts the database and compute, is a French company. Published subprocessor list: Scaleway (Amsterdam, Netherlands (nl-ams region), in the European Union), Brevo (European Union: servers in France and Germany, encrypted backups in Belgium (Google Cloud), according to Brevo), Mistral AI (European Union by default (European API endpoint), according to Mistral), ClickHouse (option not used), Stripe (Contract with Stripe Payments Europe, Ltd. (Ireland); data transferred to Stripe, LLC (United States), according to Stripe), Sentry (European Union: Frankfurt (Germany), including backups, according to Sentry). Depending on the feature used, Mistral may temporarily transfer data outside the European Union to the subprocessors it publishes; its data processing agreement limits these transfers to countries the EU recognizes as adequate, or covers them with standard contractual clauses. Stripe transfers payment data to Stripe, LLC, in the United States, under the Data Privacy Framework and, failing that, standard contractual clauses. Whatever region is chosen, Sentry may keep the data of Probant’s Sentry account (members, settings, audit log, project descriptions) in the United States; reported errors stay in Frankfurt. Where to check |
| Consent by default | By default, the SDK sets a 365-day first-party ph_<token>_posthog cookie and captures from page load. You need to configure opt_out_capturing_by_default or cookieless mode to capture and store nothing before consent.43444546 | Consent required by default: no cookie, no assignment, and nothing sent before the visitor agrees. Consent given mid-page: the page does not change in front of the visitor; their variant starts on the next page. First-party cookie, lasting 390 days at most, never extended. Where to check |
| Pricing model | Usage-based pricing per product, with no per-user fee: experiments are included in feature flags, billed per flag request (from $0.0001 per request above 1 million, decreasing after that). Optional platform packages from $250 to $2,000/month; discounts for annual commitment.47484950515253 | Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges. Agency plan: 1,000,000 tested visitors per month, shared across client workspaces, from 3 workspaces, for €1,490 per month or €14,900 per year. Where to check |
What PostHog also does.
Probant shares these points with PostHog.
A sequential test
Sequential testing with “confidence sequences” (Waudby-Smith et al. 2023) in the frequentist engine, which can be turned on per test or by default at the project level; off by default.678
Variance reduction (CUPED)
Optional CUPED, set per project and per test, on mean metrics and ordered funnels; ratios, retention, and unordered funnels are not adjusted. 14-day pre-exposure window by default (1 to 30).272829
Revenue per visitor, with its range
Mean metrics (revenue per user) and ratios (revenue per order) shown with a 95% confidence interval (frequentist) or a credible interval (Bayesian).171819
A Bayesian reading
Bayesian engine by default with probability to beat control (“chance to win”) and a 95% credible interval. Fixed non-informative prior, with no documented choice of prior; no expected loss mentioned.2021
An asynchronous script that does not hide the page
Asynchronous client-side evaluation: without precautions, a delay before the variant applies is visible. No anti-flicker script is documented; the docs recommend server-side evaluation and bootstrapping flags. Script size not stated.353637
When to choose PostHog over Probant.
Based on PostHog’s public documentation.
You want analytics, session replay, and tests in one suite
Native session replay (5,000 free recordings per month) linked to an experiment’s variants, and native heatmaps (3 pages on the free plan).54555657
You want a European cloud and a free usage tier
Region chosen at sign-up: EU (AWS eu-central-1, Frankfurt) or United States (AWS us-east-1, Virginia). Some subprocessors remain in the US (standard contractual clauses).4142 Free plan with no credit card: 1 project, 1-year retention, a free quota on each product, including 1 million feature flag requests (and therefore experiments) per month; the quota carries over to the paid plan.5859
You want open source
Open-source code and a free Docker Compose deployment under the MIT license; PostHog states that self-hosting is not officially supported.6061
What sets Probant apart.
These points are not in PostHog’s public documentation, accessed November 11, 2026. A missing mention does not prove a feature is missing; if these points matter to you, ask PostHog.
Margin per visitor
Probant measures revenue and margin per visitor, each with its range.
PostHogMargin per visitor is not mentioned in their public documentation (as of November 11, 2026).
Learn moreWinner’s curse correction
The reported gain is corrected by Bayesian shrinkage toward the effects of the store’s past tests.
PostHogThe Bayesian engine uses a non-informative prior: results only reflect the observed data.20 A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).
Learn moreSRM that blocks the verdict
A traffic split imbalance moves the test to “blocked” status, and no winner is declared.
PostHogAutomatic chi-square test from 100 exposures, with a yellow warning if p < 0.001; the result is not blocked, and the docs ask you to investigate. An AI “scout” can also flag an SRM in the Self-driving inbox.9101112 Automatic blocking of the test is not mentioned there (as of November 11, 2026).
Consent required by default
No cookie, no assignment, and nothing sent before the visitor agrees.
PostHogBy default, the SDK sets a 365-day first-party ph_<token>_posthog cookie and captures from page load. You need to configure opt_out_capturing_by_default or cookieless mode to capture and store nothing before consent.43444546 Consent required by default is not mentioned there (as of November 11, 2026).
Frequently asked questions.
How much does PostHog cost?
Usage-based pricing per product, with no per-user fee: experiments are included in feature flags, billed per flag request (from $0.0001 per request above 1 million, decreasing after that). Optional platform packages from $250 to $2,000/month; discounts for annual commitment.47484950515253 Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges.
Where does PostHog host data?
Region chosen at sign-up: EU (AWS eu-central-1, Frankfurt) or United States (AWS us-east-1, Virginia). Some subprocessors remain in the US (standard contractual clauses).4142 Probant hosts its database and application in Amsterdam, with Scaleway.
Does PostHog measure margin?
Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). Probant measures revenue and margin per visitor, each with its range, from orders and the cost of each product.
Can you look at PostHog results while a test is running?
Sequential testing with “confidence sequences” (Waudby-Smith et al. 2023) in the frequentist engine, which can be turned on per test or by default at the project level; off by default.678 Probant sets its checkpoints before launch (O’Brien-Fleming) and uses always-valid ranges during rollouts.
Sources.
Official pages from PostHog, accessed November 11, 2026. Quotes are reproduced in their original language.
- Source 1: posthog.com/llms.txt
It includes tools for product analytics, session replay, feature flags, A/B testing, error tracking, surveys, AI observability, web analytics, a data warehouse, and more.
- Source 3: posthog.com/handbook/company/culture
Our team is 100% remote, and distributed across more than 20 countries.
- Source 4: posthog.com/docs/experiments/start-here
Statistical approaches – Choose between Bayesian (default) or frequentist analysis depending on your needs
- Source 5: posthog.com/docs/experiments/statistics-frequentist
It uses t-tests and confidence intervals to determine whether differences between variants are statistically significant
- Source 6: posthog.com/docs/experiments/statistics-frequentist
When sequential testing is enabled, PostHog replaces the fixed-horizon t-test with a sequential test based on Waudby-Smith et al. 2023.
- Source 7: posthog.com/docs/experiments/statistics-frequentist
As a project default: Go to Experiments > Settings and check Apply sequential testing by default.
- Source 8: posthog.com/docs/experiments/best-practices
Alternatively, if you enable sequential testing, PostHog produces always-valid p-values that stay bounded by alpha no matter how often you check results.
- Source 9: posthog.com/docs/experiments/exposures
Sample ratio mismatch (SRM) is an automatic check that compares your actual user distribution across variants against your configured split percentages.
- Source 10: posthog.com/docs/experiments/exposures
PostHog automatically calculates SRM once your experiment has at least 100 total exposures.
- Source 11: posthog.com/docs/experiments/exposures
PostHog uses a significance threshold of p < 0.001 to flag mismatches.
- Source 12: posthog.com/docs/experiments/exposures
If you see a sample ratio mismatch warning, investigate before drawing conclusions from your experiment:
- Source 13: posthog.com/docs/experiments/metrics
Secondary metrics provide more context for your experiment. They can help you understand the impact of your primary metric and ensure your experiment didn't have negative side effects.
- Source 14: posthog.com/docs/experiments/metrics
This is just a way of organizing metrics - primary and secondary metrics work the same way.
- Source 15: posthog.com/docs/feature-flags/scheduled-flag-changes
Scheduling feature flag changes enable you to automatically change flag properties at a future point in time.
- Source 16: posthog.com/docs/feature-flags/phased-rollout
Phased rollouts, also known as phased releases, are a way to roll out new features safely by testing a feature works in production with a small group before incrementally moving to progressively bigger (and more important) groups.
- Source 17: posthog.com/docs/experiments/statistics-frequentist
Measure numeric values for each user, such as revenue per user or session duration.
- Source 18: posthog.com/docs/experiments/metrics
Revenue per purchase: Track average order value by dividing total revenue by number of purchases
- Source 19: posthog.com/docs/experiments/analyzing-results
Confidence/Credible intervals: The horizontal bars show the 95% confidence interval (frequentist) or credible interval (Bayesian):
- Source 20: posthog.com/docs/experiments/statistics-bayesian
PostHog uses non-informative priors with a mean of 0 (no expected effect) and very large variance (highly uncertain).
- Source 21: posthog.com/docs/experiments/statistics-bayesian
Instead of p-values and confidence intervals, it calculates the probability that one variant is better than another and directly answers the question: "Should I ship this change?"
- Source 23: posthog.com/docs/experiments/statistics-frequentist
No correction for multiple comparisons is applied by default
- Source 24: posthog.com/docs/experiments/statistics-frequentist
When reading results, look at the full pattern across metrics, not individual ones in isolation.
- Source 25: posthog.com/docs/experiments/no-code-web-experiments
No-code web experiments enable you to run A/B tests, multivariate tests, and other experiments that modify your website without writing a single line of code.
- Source 26: posthog.com/docs/experiments/traffic-allocation
Experiments support 2 to 20 variants: a baseline plus up to 19 test variants.
- Source 27: posthog.com/docs/experiments/cuped
CUPED (Controlled-experiment Using Pre-Experiment Data) is an optional variance reduction method for experiment metrics.
- Source 28: posthog.com/docs/experiments/cuped
Other metrics are analyzed without adjustment even when CUPED is enabled. This includes ratio and retention metrics, unordered funnels, and funnels with data warehouse steps.
- Source 29: posthog.com/docs/experiments/cuped
The look-back window defaults to 14 days and can be set between 1 and 30 days.
- Source 30: posthog.com/docs/experiments/no-code-web-experiments
You can build your web experiment directly in the PostHog toolbar, our no-code tool.
- Source 31: posthog.com/docs/experiments/no-code-web-experiments
Use the toolbar to modify the element's text, CSS, or HTML.
- Source 32: posthog.com/docs/experiments/no-code-web-experiments
Frameworks that frequently re-render or replace DOM elements may overwrite any no-code changes you apply
- Source 33: posthog.com/docs/experiments/no-code-web-experiments
No-code web experiments require disable_web_experiments: false in your PostHog web snippet configuration.
- Source 34: posthog.com/pricing.md
Build in-app popups with NPS, CSAT, multiple choice, free text, ratings, and emoji reactions. No code required for popup surveys, or use the API for full control.
- Source 36: posthog.com/docs/experiments/no-code-web-experiments
If you're evaluating the experiment flag on the same page where the element needs to change, you'll see a brief delay as PostHog loads and processes the experiment's feature flag.
- Source 37: posthog.com/docs/feature-flags/bootstrapping
Bootstrapping Feature Flags makes precomputed flag values available as soon as a client-side PostHog SDK initializes. This prevents flicker and enables startup logic, such as redirects, to use flags before the SDK finishes its first /flags request.
- Source 38: posthog.com/docs/settings/access-control
The three access levels are: Member, Admin, and Owner.
- Source 39: posthog.com/docs/settings/access-control
These plans do not currently offer any access control features.
- Source 40: posthog.com/docs/settings/access-control
Note: While you can create roles on any plan, they can only be used for access control on Enterprise plans.
- Source 41: posthog.com/docs/privacy/gdpr-compliance
If you require robust GDPR compliance, we recommend using PostHog Cloud EU – a managed version of PostHog that's hosted on servers based in Frankfurt.
- Source 42: posthog.com/handbook/company/security
Customers can choose whether to host data on our AWS servers in the EU (eu-central-1 in Germany) or the US (us-east-1 in Virginia).
- Source 43: posthog.com/docs/libraries/js/persistence
By default, PostHog uses localStorage+cookie persistence.
- Source 44: posthog.com/docs/libraries/js/persistence
The cookie name is ph_<project_token>_posthog, and it expires after 365 days.
- Source 45: posthog.com/docs/privacy/data-collection
To opt users out by default, set opt_out_capturing_by_default to true in the init call.
- Source 46: posthog.com/docs/privacy/data-collection
PostHog uses a privacy-preserving hash to count users who opt out, and only starts storing cookies once the user opts in.
- Source 47: posthog.com/pricing.md
Paid (pay-as-you-go) — $0/mo base price. You get a free tier on every product, then pay only for what you use above it.
- Source 49: posthog.com/pricing.md
Experiments is bundled with Feature Flags — same billing, same free tier.
- Source 50: posthog.com/docs/experiments/start-here
Above 1 million we have usage-based pricing starting at $0.0001/request with discounts as volume increases
- Source 51: posthog.com/pricing.md
Boost | $250/mo | Unlimited projects, white labeling, HIPAA BAA, SSO enforcement, and features for collaboration with team members.
- Source 52: posthog.com/pricing.md
Enterprise | $2000/mo | RBAC, dedicated support, training, and more.
- Source 53: posthog.com/pricing.md
For annual credit commitments, additional discounts are available — and they're fully transparent:
- Source 54: posthog.com/docs/session-replay
Session replay records what real users do in your product and plays it back like a DVR, with a synced DevTools panel showing console logs, network requests, and errors at the exact moment they happened.
- Source 55: posthog.com/docs/experiments/analyzing-results
Click View recordings to see session recordings of experiment participants.
- Source 56: posthog.com/docs/toolbar/heatmaps
On the free plan, PostHog collects heatmap data for up to 3 pages that you choose.
- Source 57: posthog.com/pricing.md
Unit: recording | Prices decrease with volume | Monthly volume | Price per recording | |---|---| | First 5,000 | Free |
- Source 58: posthog.com/pricing.md
Free — No credit card required. Generous monthly usage limits on every product. 1 project. 1 year data retention. Community support.
- Source 59: posthog.com/docs/experiments/start-here
First 1 million feature flag requests per month are free
- Source 60: posthog.com/docs/self-host
PostHog is open-source and freely available for anyone to host themselves. We offer a free Docker Compose deployment under an MIT license.
Migrate from PostHog
Your PostHog history can be imported as a generic CSV (visitors and conversions per variant) and is reread by Probant’s engine: every verdict is recalculated from visitors and conversions.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway