Product page · Revenue per visitor · day 6 of 17

Pay-in-3 option highlighted

Not conclusive yetRunning · day 6 of 17

Not decided yet.

Estimated difference +3.53% on “revenue per visitor”, likely between -3.40% and +11.0% (95% range). Planned sample size not reached yet, and the range (from -3.4% to +11.0%) still contains 0.

Exposed visitors: 31,560; A 15,800 · B 15,760
Visitors with a value: 3,846; at least 5 per variant to estimate
Planned sample size: 35%; 44,710 visitors per variant · planned end Nov 21, 2026

Reliability

Traffic split as plannedDeviation from the planned split is consistent with chance (p = 0.82).
Visitors seen in several variantsNone.

Test plan: detect 6%, 3 checkpoints

Check 1 · 29,807 visitorsday 6 · z 1.0 < 3.47
Check 2 · 59,614 visitorsz threshold 2.45
Check 3 · 89,420 visitorsz threshold 2.00

Until the sample size is reached, only a threshold crossed at a checkpoint allows a conclusion.

Variants2

AOriginal15,800 visitors · 233,018.40 in total · 14.75
BPay-in-3 option highlighted15,760 visitors · 240,630.00 in total · 15.27+3.53%

Variant	Visitors	Sum	Per visitor	Difference vs. A · range
AOriginal	15,800	233,018.40	14.75	baseline
BPay-in-3 option highlighted	15,760	240,630.00	15.27	+3.53%-3.4 → +11.0

D’où vient l’écart de B ?

Ce que compte le chiffre

« Revenue per visitor » est une valeur par visiteur exposé : la somme de ce qu’il envoie après l’exposition, 0 s’il n’a rien envoyé. Chaque visiteur compte dans la variante de sa première exposition.

Les comptes

A · Original : 233,018.40 au total sur 15,800 visiteurs, soit 14.75 par visiteur.
B · Pay-in-3 option highlighted : 240,630.00 au total sur 15,760 visiteurs, soit 15.27 par visiteur.

Le calcul

Écart relatif = valeur par visiteur de B ÷ valeur par visiteur de A − 1 = 15.27 ÷ 14.75 − 1 = +3.53%. Méthode
Test de la différence entre les deux variantes : z = 0.98, p = 0.327 (bilatéral).
Fourchette à 95.0% : from -3.40% to +11.0%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.822). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

Value per visitor: the sum of their events after exposure, 0 if they sent nothing.

Revenue per visitor, day by day

95% range each day. The shaded area covers the remaining days, and upcoming checkpoints are dotted.

Day 6 · today

Estimated difference: +3.5%
Range: -3.4 → +11.0
Visitors: 31,600

checkpoint 1 · z 1.0 < 3.47checkpoint 2 · D11 · threshold 2.45checkpoint 3 · D17 · threshold 2.00+30%+20%+10%0-10%-20%-30%

Overlapping tests2

These tests run at the same time, in parallel or in another zone, so some of their visitors also see this test. Probant checks that neither changes the other’s effect.

“Simplified mobile menu” × “Pay-in-3 option highlighted”

No conclusion yet

Revenue per visitor · 17,000 shared visitors

Rows: “Simplified mobile menu” · columns: “Pay-in-3 option highlighted” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Pay-in-3 option highlighted” (columns)
	A · original	B · Variant
A · original	14.654,262 vis.	15.194,251 vis.
B · Variant	15.164,240 vis.	15.384,247 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -4.72 and +4.09, a range wider than the effect these tests target (±0.73).

interaction gap -0.32 · range -4.72 → +4.09 · adjusted p 1.00

“Pay-in-3 option highlighted” × “Sort by popularity”

No conclusion yet

Revenue per visitor · 4,607 shared visitors · no conclusion yet

Rows: “Pay-in-3 option highlighted” · columns: “Sort by popularity” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	17.331,148 vis.	17.211,139 vis.
B · Variant	17.931,163 vis.	18.311,157 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -9.20 and +10.21, a range wider than the effect these tests target (±0.87).

interaction gap +0.51 · range -9.20 → +10.21 · adjusted p 1.00

Product click · 4,607 shared visitors · no conclusion yet

Rows: “Pay-in-3 option highlighted” · columns: “Sort by popularity” · rate of “Product click”

Product click by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	35.0%1,148 vis.	36.1%1,139 vis.
B · Variant	35.2%1,163 vis.	34.9%1,157 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -9.9 pts and +7.2 pts, a range wider than the effect these tests target (±1.8 pts).

interaction gap -1.3 pts · range -9.9 pts → +7.2 pts · adjusted p 1.00

Visitors seen by both tests, in the variant of their first exposure to each; the metric is counted after both exposures. 95% range, valid every time you check, with risk corrected across 5 comparisons.

Not decided yet.

Estimated difference +3.53% on “revenue per visitor”, likely between -3.40% and +11.0% (95% range). Planned sample size not reached yet, and the range (from -3.4% to +11.0%) still contains 0.

Exposed visitors

31,560

A 15,800 · B 15,760

Visitors with a value

3,846

at least 5 per variant to estimate

Planned sample size

35%

44,710 visitors per variant · planned end Nov 21, 2026

Variants2

AOriginal15,800 visitors · 233,018.40 in total · 14.75

BPay-in-3 option highlighted15,760 visitors · 240,630.00 in total · 15.27+3.53%

Variant	Visitors	Sum	Per visitor	Difference vs. A · range
AOriginal	15,800	233,018.40	14.75	baseline
BPay-in-3 option highlighted	15,760	240,630.00	15.27	+3.53%-3.4 → +11.0

D’où vient l’écart de B ?

Ce que compte le chiffre

« Revenue per visitor » est une valeur par visiteur exposé : la somme de ce qu’il envoie après l’exposition, 0 s’il n’a rien envoyé. Chaque visiteur compte dans la variante de sa première exposition.

Les comptes

A · Original : 233,018.40 au total sur 15,800 visiteurs, soit 14.75 par visiteur.
B · Pay-in-3 option highlighted : 240,630.00 au total sur 15,760 visiteurs, soit 15.27 par visiteur.

Le calcul

Écart relatif = valeur par visiteur de B ÷ valeur par visiteur de A − 1 = 15.27 ÷ 14.75 − 1 = +3.53%. Méthode
Test de la différence entre les deux variantes : z = 0.98, p = 0.327 (bilatéral).
Fourchette à 95.0% : from -3.40% to +11.0%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.822). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

Value per visitor: the sum of their events after exposure, 0 if they sent nothing.

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Pay-in-3 option highlighted” (columns)
	A · original	B · Variant
A · original	14.654,262 vis.	15.194,251 vis.
B · Variant	15.164,240 vis.	15.384,247 vis.

A · original

B · Variant

A · original

14.654,262 vis.

15.194,251 vis.

B · Variant

15.164,240 vis.

15.384,247 vis.

Revenue per visitor by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	17.331,148 vis.	17.211,139 vis.
B · Variant	17.931,163 vis.	18.311,157 vis.

A · original

B · Variant

A · original

17.331,148 vis.

17.211,139 vis.

B · Variant

17.931,163 vis.

18.311,157 vis.

Product click by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	35.0%1,148 vis.	36.1%1,139 vis.
B · Variant	35.2%1,163 vis.	34.9%1,157 vis.

A · original

B · Variant

A · original

35.0%1,148 vis.

36.1%1,139 vis.

B · Variant

35.2%1,163 vis.

34.9%1,157 vis.

The peeking trap

The peeking trap

Pay-in-3 option highlighted

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Revenue per visitor, day by day

Overlapping tests2

“Simplified mobile menu” × “Pay-in-3 option highlighted”

“Pay-in-3 option highlighted” × “Sort by popularity”

The peeking trap

Pay-in-3 option highlighted

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Revenue per visitor, day by day

Overlapping tests2

“Simplified mobile menu” × “Pay-in-3 option highlighted”

“Pay-in-3 option highlighted” × “Sort by popularity”