Navigation · Revenue per visitor · day 9 of 13

Simplified mobile menu

Not conclusive yetRunning · day 9 of 13

Not decided yet.

Estimated difference +1.86% on “revenue per visitor”, likely between -3.71% and +7.77% (95% range). Planned sample size not reached yet, and the range (from -3.7% to +7.8%) still contains 0.

Exposed visitors: 48,150; A 24,100 · B 24,050
Visitors with a value: 5,839; at least 5 per variant to estimate
Planned sample size: 69%; 34,775 visitors per variant · planned end Nov 14, 2026

Reliability

Traffic split as plannedDeviation from the planned split is consistent with chance (p = 0.82).
Visitors seen in several variantsNone.

Test plan: detect 7%, 3 checkpoints

Check 1 · 23,184 visitorsday 4 · z 0.8 < 3.47
Check 2 · 46,367 visitorsday 9 · z 0.6 < 2.45
Check 3 · 69,550 visitorsz threshold 2.00

Until the sample size is reached, only a threshold crossed at a checkpoint allows a conclusion.

Variants2

AOriginal24,100 visitors · 355,426.80 in total · 14.75
BSimplified mobile menu24,050 visitors · 361,302.20 in total · 15.02+1.86%

Variant	Visitors	Sum	Per visitor	Difference vs. A · range
AOriginal	24,100	355,426.80	14.75	baseline
BSimplified mobile menu	24,050	361,302.20	15.02	+1.86%-3.7 → +7.8

D’où vient l’écart de B ?

Ce que compte le chiffre

« Revenue per visitor » est une valeur par visiteur exposé : la somme de ce qu’il envoie après l’exposition, 0 s’il n’a rien envoyé. Chaque visiteur compte dans la variante de sa première exposition.

Les comptes

A · Original : 355,426.80 au total sur 24,100 visiteurs, soit 14.75 par visiteur.
B · Simplified mobile menu : 361,302.20 au total sur 24,050 visiteurs, soit 15.02 par visiteur.

Le calcul

Écart relatif = valeur par visiteur de B ÷ valeur par visiteur de A − 1 = 15.02 ÷ 14.75 − 1 = +1.86%. Méthode
Test de la différence entre les deux variantes : z = 0.64, p = 0.520 (bilatéral).
Fourchette à 95.0% : from -3.71% to +7.77%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.820). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

Value per visitor: the sum of their events after exposure, 0 if they sent nothing.

Revenue per visitor, day by day

95% range each day. The shaded area covers the remaining days, and upcoming checkpoints are dotted.

Day 9 · today

Estimated difference: +1.9%
Range: -3.7 → +7.8
Visitors: 48,200

checkpoint 1 · z 0.8 < 3.47checkpoint 2 · z 0.6 < 2.45checkpoint 3 · D13 · threshold 2.00+30%+20%+10%0-10%-20%-30%

Overlapping tests2

This test runs in parallel on all the traffic to its pages. It overlaps the other running tests, and Probant checks that none of them changes another’s effect.

“Simplified mobile menu” × “Pay-in-3 option highlighted”

No conclusion yet

Revenue per visitor · 17,000 shared visitors

Rows: “Simplified mobile menu” · columns: “Pay-in-3 option highlighted” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Pay-in-3 option highlighted” (columns)
	A · original	B · Variant
A · original	14.654,262 vis.	15.194,251 vis.
B · Variant	15.164,240 vis.	15.384,247 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -4.72 and +4.09, a range wider than the effect these tests target (±0.73).

interaction gap -0.32 · range -4.72 → +4.09 · adjusted p 1.00

“Simplified mobile menu” × “Sort by popularity”

No conclusion yet

Revenue per visitor · 5,794 shared visitors · no conclusion yet

Rows: “Simplified mobile menu” · columns: “Sort by popularity” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	10.991,452 vis.	11.201,461 vis.
B · Variant	11.681,437 vis.	11.841,444 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -7.40 and +7.30, a range wider than the effect these tests target (±0.55).

interaction gap -0.05 · range -7.40 → +7.30 · adjusted p 1.00

Product click · 5,794 shared visitors · no conclusion yet

Rows: “Simplified mobile menu” · columns: “Sort by popularity” · rate of “Product click”

Product click by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	40.2%1,452 vis.	40.8%1,461 vis.
B · Variant	41.6%1,437 vis.	42.0%1,444 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -8.1 pts and +7.7 pts, a range wider than the effect these tests target (±2.0 pts).

interaction gap -0.2 pts · range -8.1 pts → +7.7 pts · adjusted p 1.00

Visitors seen by both tests, in the variant of their first exposure to each; the metric is counted after both exposures. 95% range, valid every time you check, with risk corrected across 5 comparisons.

Not decided yet.

Estimated difference +1.86% on “revenue per visitor”, likely between -3.71% and +7.77% (95% range). Planned sample size not reached yet, and the range (from -3.7% to +7.8%) still contains 0.

Exposed visitors

48,150

A 24,100 · B 24,050

Visitors with a value

5,839

at least 5 per variant to estimate

Planned sample size

69%

34,775 visitors per variant · planned end Nov 14, 2026

Variants2

AOriginal24,100 visitors · 355,426.80 in total · 14.75

BSimplified mobile menu24,050 visitors · 361,302.20 in total · 15.02+1.86%

Variant	Visitors	Sum	Per visitor	Difference vs. A · range
AOriginal	24,100	355,426.80	14.75	baseline
BSimplified mobile menu	24,050	361,302.20	15.02	+1.86%-3.7 → +7.8

D’où vient l’écart de B ?

Ce que compte le chiffre

« Revenue per visitor » est une valeur par visiteur exposé : la somme de ce qu’il envoie après l’exposition, 0 s’il n’a rien envoyé. Chaque visiteur compte dans la variante de sa première exposition.

Les comptes

A · Original : 355,426.80 au total sur 24,100 visiteurs, soit 14.75 par visiteur.
B · Simplified mobile menu : 361,302.20 au total sur 24,050 visiteurs, soit 15.02 par visiteur.

Le calcul

Écart relatif = valeur par visiteur de B ÷ valeur par visiteur de A − 1 = 15.02 ÷ 14.75 − 1 = +1.86%. Méthode
Test de la différence entre les deux variantes : z = 0.64, p = 0.520 (bilatéral).
Fourchette à 95.0% : from -3.71% to +7.77%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.820). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

Value per visitor: the sum of their events after exposure, 0 if they sent nothing.

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Pay-in-3 option highlighted” (columns)
	A · original	B · Variant
A · original	14.654,262 vis.	15.194,251 vis.
B · Variant	15.164,240 vis.	15.384,247 vis.

A · original

B · Variant

A · original

14.654,262 vis.

15.194,251 vis.

B · Variant

15.164,240 vis.

15.384,247 vis.

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	10.991,452 vis.	11.201,461 vis.
B · Variant	11.681,437 vis.	11.841,444 vis.

A · original

B · Variant

A · original

10.991,452 vis.

11.201,461 vis.

B · Variant

11.681,437 vis.

11.841,444 vis.

Product click by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	40.2%1,452 vis.	40.8%1,461 vis.
B · Variant	41.6%1,437 vis.	42.0%1,444 vis.

A · original

B · Variant

A · original

40.2%1,452 vis.

40.8%1,461 vis.

B · Variant

41.6%1,437 vis.

42.0%1,444 vis.

The peeking trap

The peeking trap

Simplified mobile menu

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Revenue per visitor, day by day

Overlapping tests2

“Simplified mobile menu” × “Pay-in-3 option highlighted”

“Simplified mobile menu” × “Sort by popularity”

The peeking trap

Simplified mobile menu

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Revenue per visitor, day by day

Overlapping tests2

“Simplified mobile menu” × “Pay-in-3 option highlighted”

“Simplified mobile menu” × “Sort by popularity”