- 10%Aug 17 → Aug 20
- 25%Aug 21 → Aug 23
- 50%Aug 24 → Aug 30
- 100%since Aug 31
On small screens, measure every menu and every filter.
Menu, filters, and search on mobile: where visitors look for a product before they find it. The examples come from the demo store.
- Simplified mobile menuRevenue per visitorNot conclusive yet…day 9 of 13+1.9%, range -3.7% to +7.8%
- Recently completed
- Search suggestionsProduct clickNo detectable effect.completed Sep 28+1.1%, range -0.9% to +3.1%
- Simplified filters on mobileProduct clickConclusive. Proven gaincompleted Aug 14+4.0%, range +1.4% to +6.6%
- Breadcrumbs on category pagesProduct clickNo detectable effect.completed Jun 8+0.3%, range -1.5% to +2.1%
Maison Lune, a fictional demo store · data computed by the engine
On mobile, finding a product takes more taps.
Space is tight: every element shown hides another. Tests here target what shortens the path to a product.
- MenuNumber of sections, depth, photos in the menu.
- FiltersHow many to show, where to put them, keeping them visible while scrolling.
- SortThe options offered and the default sort.
- SearchAlways visible or behind an icon, suggestions as you type, typo tolerance.
- Product gridOne or two products per row, photo size.
- Buy buttonFixed at the bottom of the screen, or in its place on the page.
Maison Lune, a fictional demo store
On mobile, one test is running and an earlier winner went live in steps.
The simplified menu hasn’t reached its sample size yet. Simplified filters on mobile, the winner of an earlier test: rolled out to 100% on Aug 31.
Simplified mobile menu: not conclusive yet.
Day 9 of 13, 5,350 visitors a day. Today’s lift in Revenue per visitor: +1.9%, range -3.7% to +7.8%.
Two of three checkpoints reached, no boundary crossed. Next checkpoint: day 13, with a boundary of 2.00.
Simplified filters on mobile: rolled out to 100% on Aug 31.
Origin: winner in August. At each step, exposed visitors are compared with those still on the original, guardrail by guardrail, with always-valid ranges.
Before the variant goes to everyone, every range has to sit entirely on the right side of its limit: after that, there’s no original left to compare against.
Mobile ideas are sized on their area’s traffic.
Search ideas and mobile browsing ideas, whatever their area. In a low-traffic area, Probant suggests a more frequent metric, or tells you to aim for a bigger effect.
Search trafficdemo store
900visitors a day
15.8%smallest effect measurable in 30 days on “Commande”
3.6%on “Product click”, which happens more often, in the same time
An idea with a smaller expected effect won’t reach a conclusion in a month, and Probant tells you before launch.
- Rank 17Category suggestions while typing
- Expected effect
- +1.7%prior2 search tests, too few
- Time to conclude
- Undetectableover a year on ordersmeasurable in 30 days from +3.6%
- Estimated gain
- +€2,700per month
- Rank 18Typo-tolerant search
- Expected effect
- +1.7%prior2 search tests, too few
- Time to conclude
- Undetectableover a year on ordersmeasurable in 30 days from +3.6%
- Estimated gain
- +€2,700per month
2 pending ideas, in Probant’s ranking order. Expected effect drawn from past tests on the same theme; duration based on the area’s traffic.
The library groups mobile tests by theme.
Completed tests from the Navigation area, and lessons from the themes that cover mobile browsing and search.
Lessons by theme
Mobile browsing
5 tests · 2 winners
+0.4% pooled effect, -0.3% to +1.2%
Medium confidence. Enough tests, but the pooled effect is still compatible with no effect.
Search
2 tests
+1.3% pooled effect, -0.1% to +2.6%
Low confidence. Fewer than three tests: a lead, not yet a rule.
The 6 most recent of 7 completed tests
- Search suggestionsSep 7 → Sep 28 · Product click · Original keptNo detectable effect.+1.1%, range -0.9% to +3.1%
Possible gain, too small to detect at this traffic
- Simplified filters on mobileJul 23 → Aug 14 · Product click · ShippedConclusive. Proven gain+4.0%, range +1.4% to +6.6%
Fewer filters, more products viewed
- Breadcrumbs on category pagesMay 18 → Jun 8 · Product click · Original keptNo detectable effect.+0.3%, range -1.5% to +2.1%
Useful to a few visitors, with no overall effect
- Sorting cut to three options on mobileMar 30 → Apr 20 · Product click · Original keptNo detectable effect.-0.6%, range -1.8% to +0.6%
Fewer sort options neither helped nor hurt
- Search bar always visible on mobileFeb 16 → Mar 9 · Product click · Original keptNo detectable effect.+1.4%, range -0.5% to +3.4%
More searches, but no more product clicks
- Mega menu with photosJan 12 → Feb 2 · Product click · DroppedConclusive. Proven loss-2.9%, range -4.8% to -1.0%
The photos slowed the menu down and buried the departments
On mobile, speed is part of the result.
A richer menu or a finer search can load more code: what helps on one side can slow things down on the other.
- Rendering speedTime to render the main content, measured on each version’s visitors.
- JavaScript errorsA rewritten menu or filter can break in one specific mobile browser.
- TargetingTarget a mobile-specific change at mobile devices, so desktop visitors don’t dilute the effect.
Demo store guardrails
Monitored during every test and every rollout step. A proven drop stops the test or sends visitors back to the original.
- Page speed (LCP)
- 75th percentile of page views · increase of +4% at most
- JavaScript errors
- visitors who hit at least one error · increase of +25% at most
The Probant features that help in this area.
Other use cases
The same engine, for other site areas and other teams. The demo opens on the screen this page describes.
- Open demo, no sign-up
- Free to start, no overage charges
- Database and compute in Amsterdam, with Scaleway