Probant vs. Shoplift
Both tools compared point by point, based on Shoplift’s public documentation, accessed November 11, 2026. Every statement about Shoplift links to its source.
What Shoplift offers.
Shoplift presents itself as the Shopify-native A/B testing and CRO platform for growing brands.1 Founded in 2021, it is headquartered in New York.2 Its results are Bayesian, with a probability to win.3
Probant is for sites that want A/B test verdicts that hold up, with a plan set before launch and guardrails monitored through rollout. It measures what each conversion is worth: the margin on an order, a qualified or signed lead, a paid subscription.
The differences, point by point.
On the left, what Shoplift’s public documentation says, each sentence with its source. On the right, what Probant does. “Not mentioned” means nothing was found in that documentation, which does not prove a feature is missing.
| Criterion | Shoplift | Probant |
|---|---|---|
| Statistical method | A Bayesian engine (“Probability to Win” and lift with a range); a frequentist mode is not mentioned.3 | Frequentist, with a sequential plan set before launch: target effect, duration, and checkpoints. The verdict fits in one sentence, with a 95% range. Where to check |
| Peeking | No sequential test is documented: a verdict is only given if the probability stays above a threshold for several days (“stability hold”). An option also lets you stop the test automatically as soon as significance is reached.45 | Decisions are only made at planned checkpoints, with O’Brien-Fleming boundaries: very strict at first, close to the usual threshold at the end. During a rollout, ranges are always valid (mSPRT), so you can check them at any time. Where to check |
| Traffic split (SRM) | No automatic SRM check is documented in the app. The API guide suggests monitoring the split imbalance yourself in a Slack alert script.6 | The traffic split is checked with a chi-square test every time the report is read. If the split is off, the test moves to “blocked” status and no winner is declared. Where to check |
| Guardrails | No native guardrail metric is documented. The API guide suggests writing a script (for example with Claude) that sends alerts and can pause a failing test.7 | Guardrail metrics are tracked with their range; a guardrail is only breached if the whole range crosses the limit. A breached guardrail blocks the test’s verdict, stops a running test on its own, and during a rollout moves visitors back to the original automatically. Where to check |
| Gradual rollouts | Applying the winning variant is possible for theme and template tests.8 A stepped rollout with automatic rollback is not mentioned in their public documentation (as of November 11, 2026). | Steps of 25% → 50% → 100% by default, 10% → 25% → 50% → 100% in cautious mode. Automatic rollback when a guardrail is breached; an exposed visitor stays exposed in the following steps. Where to check |
| Multi-armed bandit | The traffic split does not change during a test, to keep the same distribution until the end.9 | Optimization test: Thompson sampling, reallocated at most once a day, with a minimum share per variant. It maximizes a conversion rate, or an amount per visitor such as revenue: purchase probability and average order value drawn from their distributions, with values capped at a percentile as in an A/B test. Optionally, shares per device (mobile, tablet, desktop): one draw per device, each with its own learning and floor. A visitor always keeps their variant: a new split only applies to visitors who arrive after it. The screen says so: it maximizes conversions (or the amount per visitor) during the test, without proving that a variant is better. The difference is read period by period (and device by device), with no declared winner; the CSV export gives the same reading. Where to check |
| Revenue and margin per visitor | Revenue per visitor and average order value can be the test goal; the lift is shown with an uncertainty range.1011 Shoplift’s comparison page credits Intelligems with profit tracking based on product costs.12 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). | Revenue and margin per visitor, each with its range; the gain is stated in euros. Margin is calculated on the server from reconciled orders, the cost of each product (Shopify or CSV), and store fees; no margin verdict below 90% of revenue covered by known costs. Where to check |
| Winner’s curse | A winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026). | The reported gain is corrected for the winner’s curse by Bayesian shrinkage toward the effects of the store’s past tests. Where to check |
| Bayesian mode | “Probability to Win” and lift shown with a range; Leaning, High Confidence, and Clear Winner stages. Prior and expected loss not documented.13 | Optional, off by default: a Bayesian reading next to the verdict, with the probability that the variant does better, the credible interval, and the expected loss of each choice. The prior is centered on zero and as wide as the effects measured by the store’s completed tests; while there are too few of them to estimate it, the Bayesian reading is not shown. The decision always follows the sealed plan; when the two readings disagree, the report explains why. Where to check |
| Multiple comparisons | A multiple comparisons correction is not mentioned in their public documentation (as of November 11, 2026). | Segment comparisons are corrected with the Holm method. With several variants, the error risk is split across comparisons (Bonferroni). Where to check |
| Multivariate testing | Shoplift states that it does not support A/B/n for now: a test compares an original and one variant; mutual exclusion lets you run several tests in parallel.14 | Full factorial design generated by Probant: two to four sections, each with two to four levels, up to sixteen combinations, each served as a variant. Main effects of each section across all traffic and interactions between two sections, corrected with the Holm method on the comparisons stated in the plan. Based on the workspace’s measured traffic, the plan says whether the test can reach a conclusion and suggests removing a level or a section; no best combination is declared without a check. Where to check |
| Variance reduction (CUPED) | Variance reduction (CUPED) is not mentioned in their public documentation (as of November 11, 2026). | CUPED applied to live test reports, using each visitor’s history before their first exposure; without history, the difference stays unadjusted and the report says so. Where to check |
| Visual editor | The variant (a duplicated template or theme) is edited in the Shopify theme editor. “API” tests run JavaScript written by a developer.1516 | Two ways to edit, producing the same changes: a page preview loaded by Probant, without its scripts, or the live page on the site, scripts included. On the site, the editor is loaded by the script already installed, through a short-lived signed link for one draft and one variant, with no browser extension. It only sends the chosen changes to Probant. Where to check |
| Widgets | Lift Assist provides theme sections ready to test, styled with the store’s colors and fonts; included from the Advanced plan.17 | Banner, countdown, pop-up, badge, reassurance, action bar, delivery date, social proof, promo code, free shipping, and survey, set up without code in the editor and saved in the workspace library for other tests. A widget is just a set of ordinary changes: escaped text, links over https or to the site, and styles applied without a style attribute, so a strict CSP does not need loosening. The countdown follows the site’s time zone and disappears at the deadline; the pop-in opens on arrival, on exit intent, or after scrolling, closes with the keyboard, and does not reappear during the visit if consent allows. Their small modules only load for visitors in the variant. The delivery date is calculated in the visitor’s browser (cutoff time, business days, public holidays); social proof shows the exact number of purchases received over a rolling week, and only above a threshold. The survey asks up to 8 questions (closed answers, recommendation scale, open question) with conditional logic, on the page or through a link outside the site (email, QR code), based on 70 templates (recommendation, satisfaction, effort, checkout blockers, cancellation, concept test, multi-question surveys…); open answers are redacted before sending (email addresses, phone numbers, cards, IBANs), kept for 180 days, and grouped by theme; each question is read by variant, with its range and response rate, and comparisons across all questions are corrected together; a response through a link is attached to the variant the visitor saw, recalculated by Probant, or stays outside the test. Where to check |
| Script and flicker | The shoplift.liquid snippet is placed right after <head> and added automatically. During a test, anti-flicker hides the page, usually for less than 150 ms; stated impact of 1 to 2 Lighthouse points (2 to 4 points on the pricing page). Size not published; conversions tracked server-side through the Shopify Web Pixel.181920 | Script under 7 KB compressed, loaded asynchronously, that does not hide the page while it loads. Server-side assignment available, for a display with no flicker at all. Where to check |
| Granular permissions | Access follows Shopify’s app permission.21 Custom roles or granular permissions are not mentioned in their public documentation (as of November 11, 2026). | Four fixed roles, plus custom roles the owner builds from a catalog of named permissions: prepare, launch and stop, roll out, approve a launch, write variant JavaScript, manage flags, metrics, margin, members, and keys. No privilege escalation: billing, roles, transfer, and deletion of the workspace stay with the owner; no one can grant more than their own permissions, and an API key cannot exceed its author’s. Where to check |
| Data hosting | Data is transferred to and processed in the United States; the infrastructure runs on Google Cloud.2223 An EU hosting region is not mentioned in their public documentation (as of November 11, 2026). | Database and compute in Amsterdam, with Scaleway, in the European Union. Scaleway, which hosts the database and compute, is a French company. Published subprocessor list: Scaleway (Amsterdam, Netherlands (nl-ams region), in the European Union), Brevo (European Union: servers in France and Germany, encrypted backups in Belgium (Google Cloud), according to Brevo), Mistral AI (European Union by default (European API endpoint), according to Mistral), ClickHouse (option not used), Stripe (Contract with Stripe Payments Europe, Ltd. (Ireland); data transferred to Stripe, LLC (United States), according to Stripe), Sentry (European Union: Frankfurt (Germany), including backups, according to Sentry). Depending on the feature used, Mistral may temporarily transfer data outside the European Union to the subprocessors it publishes; its data processing agreement limits these transfers to countries the EU recognizes as adequate, or covers them with standard contractual clauses. Stripe transfers payment data to Stripe, LLC, in the United States, under the Data Privacy Framework and, failing that, standard contractual clauses. Whatever region is chosen, Sentry may keep the data of Probant’s Sentry account (members, settings, audit log, project descriptions) in the United States; reported errors stay in Frankfurt. Where to check |
| Consent by default | By default, Shoplift reads Shopify’s Customer Privacy API and only records analytics data after consent. Assignment to a variant (cookies and browser storage) happens without consent, and events queued before consent are sent once it is given.24252627 | Consent required by default: no cookie, no assignment, and nothing sent before the visitor agrees. Consent given mid-page: the page does not change in front of the visitor; their variant starts on the next page. First-party cookie, lasting 390 days at most, never extended. Where to check |
| Pricing model | Public pricing based on monthly unique visitors to the whole site (not only tested ones). Core from $99/month ($74 billed annually), Advanced $399/month (pricing, API, personalization), Pro $999/month. Above the quota, tests pause until the reset or an upgrade.28293031 | Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges. Agency plan: 1,000,000 tested visitors per month, shared across client workspaces, from 3 workspaces, for €1,490 per month or €14,900 per year. Where to check |
What Shoplift also does.
Probant shares these points with Shoplift.
Revenue per visitor, with its range
Revenue per visitor and average order value can be the test goal; the lift is shown with an uncertainty range.1011
Ready-made widgets
Lift Assist provides theme sections ready to test, styled with the store’s colors and fonts; included from the Advanced plan.17
A Bayesian reading
“Probability to Win” and lift shown with a range; Leaning, High Confidence, and Clear Winner stages. Prior and expected loss not documented.13
When to choose Shoplift over Probant.
Based on Shoplift’s public documentation.
You want to test entire Shopify themes without code
The variant (a duplicated template or theme) is edited in the Shopify theme editor. “API” tests run JavaScript written by a developer.1516
You want measurement that follows Shopify’s privacy API
By default, Shoplift reads Shopify’s Customer Privacy API and only records analytics data after consent. Assignment to a variant (cookies and browser storage) happens without consent, and events queued before consent are sent once it is given.24252627
You want public pricing
Public pricing based on monthly unique visitors to the whole site (not only tested ones). Core from $99/month ($74 billed annually), Advanced $399/month (pricing, API, personalization), Pro $999/month. Above the quota, tests pause until the reset or an upgrade.28293031
What sets Probant apart.
These points are not in Shoplift’s public documentation, accessed November 11, 2026. A missing mention does not prove a feature is missing; if these points matter to you, ask Shoplift.
Margin per visitor
Probant measures revenue and margin per visitor, each with its range.
ShopliftShoplift’s comparison page credits Intelligems with profit tracking based on product costs.12 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026).
Learn moreWinner’s curse correction
The reported gain is corrected by Bayesian shrinkage toward the effects of the store’s past tests.
ShopliftA winner’s curse correction is not mentioned in their public documentation (as of November 11, 2026).
Learn moreSRM that blocks the verdict
A traffic split imbalance moves the test to “blocked” status, and no winner is declared.
ShopliftNo automatic SRM check is documented in the app. The API guide suggests monitoring the split imbalance yourself in a Slack alert script.6 Automatic blocking of the test is not mentioned there (as of November 11, 2026).
Consent required by default
No cookie, no assignment, and nothing sent before the visitor agrees.
ShopliftBy default, Shoplift reads Shopify’s Customer Privacy API and only records analytics data after consent. Assignment to a variant (cookies and browser storage) happens without consent, and events queued before consent are sent once it is given.24252627 Consent required by default is not mentioned there (as of November 11, 2026).
Frequently asked questions.
How much does Shoplift cost?
Public pricing based on monthly unique visitors to the whole site (not only tested ones). Core from $99/month ($74 billed annually), Advanced $399/month (pricing, API, personalization), Pro $999/month. Above the quota, tests pause until the reset or an upgrade.28293031 Creating a Probant workspace is free; paid plans cost €590 to €1,990 per month depending on the number of tested visitors (2 months free with annual billing), with no overage charges.
Where does Shoplift host data?
Data is transferred to and processed in the United States; the infrastructure runs on Google Cloud.2223 An EU hosting region is not mentioned in their public documentation (as of November 11, 2026). Probant hosts its database and application in Amsterdam, with Scaleway.
Does Shoplift measure margin?
Shoplift’s comparison page credits Intelligems with profit tracking based on product costs.12 Margin per visitor is not mentioned in their public documentation (as of November 11, 2026). Probant measures revenue and margin per visitor, each with its range, from orders and the cost of each product.
Can you look at Shoplift results while a test is running?
No sequential test is documented: a verdict is only given if the probability stays above a threshold for several days (“stability hold”). An option also lets you stop the test automatically as soon as significance is reached.45 Probant sets its checkpoints before launch (O’Brien-Fleming) and uses always-valid ranges during rollouts.
Sources.
Official pages from Shoplift, accessed November 11, 2026. Quotes are reproduced in their original language.
- Source 1: www.shoplift.ai/press-kit
Shoplift is the Shopify-native A/B testing and conversion rate optimization (CRO) platform built for high-growth DTC brands.
- Source 2: www.shoplift.ai/press-kit
Founded in 2021 and headquartered in New York City, Shoplift is the only CRO platform to earn "Shopify Plus Certified" status — handpicked by Shopify and less than 0.8% of all Shopify apps have earned this distinction.
- Source 3: docs.shoplift.ai/analyze/reports/statistical-significance
While a test runs, Shoplift continuously evaluates test results using Bayesian statistical methods.
- Source 4: docs.shoplift.ai/analyze/reports/statistical-significance
Probability to Win has to hold at or above a level for multiple days before Shoplift assigns a stage. That stability hold is the guardrail that keeps you from acting on a lucky moment instead of a settled result.
- Source 5: docs.shoplift.ai/test/scheduling-tests
Choose to end the test automatically when statistical significance is achieved, ensuring you only gather data as long as it’s meaningful.
- Source 6: docs.shoplift.ai/api-reference/public-api-1/shoplift-api/api-guides
Split drift: controlSplit and variantSplit should stay near your configured traffic split. A persistent gap suggests a sample ratio problem worth investigating.
- Source 7: docs.shoplift.ai/api-reference/public-api-1/shoplift-api/api-guides
Auto-pause on failure. For hard failures, go beyond alerting: call POST /tests/:id/pause and note the pause in the Slack message.
- Source 8: docs.shoplift.ai/test/implementing-winning-tests
Applying your variant is available for theme and template tests only.
- Source 9: docs.shoplift.ai/get-started/frequently-asked-questions
You can't adjust the split percentage while a test is running. This ensures accuracy and validity of your results by maintaining the same sample distribution throughout the duration of the test.
- Source 10: docs.shoplift.ai/analyze/reports/goals
Goals are the primary outcome metrics merchants choose to optimize in their experiments (e.g. Conversion Rate, Average Order Value, Revenue per Visitor).
- Source 11: docs.shoplift.ai/analyze/reports/statistical-significance
Lift is shown with a range rather than a single point. The range reflects how much uncertainty is still in the estimate.
- Source 12: www.shoplift.ai/compare/intelligems
Profit optimization Load your product costs into Intelligems manually, and Intelligems's analytics will give you basic profit reporting
- Source 13: docs.shoplift.ai/analyze/reports/statistical-significance
Probability to Win is the main metric on your report. It's the chance your variant beats the original.
- Source 14: docs.shoplift.ai/get-started/frequently-asked-questions
Shoplift does not currently support A/B/n testing, but our out-of-the-box mutual exclusion allows you to run multiple tests at once.
- Source 15: docs.shoplift.ai/get-started/quickstart/edit-your-variant
After duplicating or selecting your variant, you can click "Edit in Shopify" to open the Shopify Theme Editor and make direct edits to your template or theme.
- Source 16: docs.shoplift.ai/test/javascript-api/overview
API tests require developer involvement. Most merchants will need their developer or agency to write the JavaScript and (for manual API tests) implement the trigger logic.
- Source 17: docs.shoplift.ai/test/lift-assist-tm/what-is-lift-assist-tm
Lift Assist™ is Shoplift's recommendation engine which dynamically provides theme sections to extend your store's functionality and give you the tools you need to run powerful tests otherwise not capable with your theme.
- Source 18: docs.shoplift.ai/analyze/tracking/liquid-snippet
A small theme snippet titled shoplift.liquid will be inserted into the /snippets/ directory in your live theme, and is referenced in your theme.liquid file directly after the opening <head> tag.
- Source 19: www.shoplift.ai/pricing
When tests are live, our anti-flicker script momentarily blanks out your pages to ensure shoppers don't see new experiences, but this is typically resolved within 150 milliseconds (and as low as 90 milliseconds) and is imperceivable to shoppers.
- Source 20: docs.shoplift.ai/partners-and-developers/performance/pagespeed
When measured against industry standards (such as Google Lighthouse), Shoplift's testing script introduces a negligible performance impact, typically resulting in a minimal 1-2 point reduction in the overall Lighthouse score.
- Source 21: docs.shoplift.ai/get-started/frequently-asked-questions
Shoplift uses Shopify's permission system. Any staff member with access to your Shopify store and the Shoplift app can access Shoplift by opening the app from the Shopify admin.
- Source 22: www.shoplift.ai/privacy-policy
The information that we collect through or in connection with the Services is transferred to and processed in the United States for the purposes described above.
- Source 23: www.shoplift.ai/data-processing-addendum
Google LLC / USA / Infrastructure, storage, logging, crash reporting and DNS (Google Cloud Platform)
- Source 24: docs.shoplift.ai/partners-and-developers/privacy
By default, Shoplift leverages Shopify’s Customer Privacy API, which determines whether a visitor has granted analytics consent to collect data.
- Source 25: docs.shoplift.ai/api-reference/javascript-api/javascript-api/setanalyticsconsent
Visitors are always assigned to a variant to ensure a consistent experience.
- Source 26: docs.shoplift.ai/api-reference/javascript-api/javascript-api/setanalyticsconsent
Syncs all queued events to server (including events queued before consent was granted)
- Source 27: docs.shoplift.ai/get-started/frequently-asked-questions
Shoplift uses browser storage and cookies to persist test assignments.
- Source 30: www.shoplift.ai/pricing
Visitor counts are based on total monthly website visitors, not tested visitors.
- Source 31: docs.shoplift.ai/account/plans-and-billing/subscription-plans
When you exceed the usage limit on your plan, Shoplift will temporarily pause your active tests and prevent new tests from being launched until the overage is resolved.
Migrate from Shoplift
Your Shoplift history can be imported as a generic CSV (visitors and conversions per variant) and is reread by Probant’s engine: every verdict is recalculated from visitors and conversions.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway