Category page · Product click · day 3 of 14

Sort by popularity

Not conclusive yetRunning · day 3 of 14

Not decided yet.

Estimated difference +1.54% on the rate of “product click”, likely between -2.84% and +6.13% (95% range). Planned sample size not reached yet, and the range (from -2.8% to +6.1%) still contains 0.

Exposed visitors: 11,680; A 5,850 · B 5,830
Converted visitors: 4,708; at least 5 per variant to estimate
Planned sample size: 21%; 27,254 visitors per variant · planned end Nov 21, 2026

Reliability

Traffic split as plannedDeviation from the planned split is consistent with chance (p = 0.85).
Visitors seen in several variantsNone.

Test plan: detect 3%, 3 checkpoints

Check 1 · 18,169 visitorsz threshold 3.47
Check 2 · 36,338 visitorsz threshold 2.45
Check 3 · 54,507 visitorsz threshold 2.00

Until the sample size is reached, only a threshold crossed at a checkpoint allows a conclusion.

Variants2

AOriginal5,850 visitors · 2,340 converters · 40.0%
BSort by popularity5,830 visitors · 2,368 converters · 40.6%+1.54%

Variant	Visitors	Converters	Rate	Difference vs. A · range
AOriginal	5,850	2,340	40.0%	baseline
BSort by popularity	5,830	2,368	40.6%	+1.54%-2.8 → +6.1

D’où vient l’écart de B ?

Ce que compte le chiffre

La part des visiteurs exposés au test qui envoient « Product click » après l’avoir vu. Chaque visiteur compte une fois, dans la variante de sa première exposition.

Les comptes

A · Original : 2,340 convertis sur 5,850 visiteurs, soit 40.0%.
B · Sort by popularity : 2,368 convertis sur 5,830 visiteurs, soit 40.6%.

Le calcul

Écart relatif = taux de B ÷ taux de A − 1 = 40.6% ÷ 40.0% − 1 = +1.54%. Méthode
Test de la différence entre les deux variantes : z = 0.68, p = 0.496 (bilatéral).
Fourchette à 95.0% : from -2.84% to +6.13%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.853). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

A visitor counts as converted if they send “Product click” after seeing the test, in the variant of their first exposure.

Product click, day by day

95% range each day. The shaded area covers the remaining days, and upcoming checkpoints are dotted.

Day 3 · today

Estimated difference: +1.5%
Range: -2.8 → +6.1
Visitors: 11,700

checkpoint 1 · D5 · threshold 3.47checkpoint 2 · D9 · threshold 2.45checkpoint 3 · D14 · threshold 2.00+15%+10%+5%0-5%-10%-15%

Overlapping tests2

These tests run at the same time, in parallel or in another zone, so some of their visitors also see this test. Probant checks that neither changes the other’s effect.

“Simplified mobile menu” × “Sort by popularity”

No conclusion yet

Revenue per visitor · 5,794 shared visitors · no conclusion yet

Rows: “Simplified mobile menu” · columns: “Sort by popularity” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	10.991,452 vis.	11.201,461 vis.
B · Variant	11.681,437 vis.	11.841,444 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -7.40 and +7.30, a range wider than the effect these tests target (±0.55).

interaction gap -0.05 · range -7.40 → +7.30 · adjusted p 1.00

Product click · 5,794 shared visitors · no conclusion yet

Rows: “Simplified mobile menu” · columns: “Sort by popularity” · rate of “Product click”

Product click by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	40.2%1,452 vis.	40.8%1,461 vis.
B · Variant	41.6%1,437 vis.	42.0%1,444 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -8.1 pts and +7.7 pts, a range wider than the effect these tests target (±2.0 pts).

interaction gap -0.2 pts · range -8.1 pts → +7.7 pts · adjusted p 1.00

“Pay-in-3 option highlighted” × “Sort by popularity”

No conclusion yet

Revenue per visitor · 4,607 shared visitors · no conclusion yet

Rows: “Pay-in-3 option highlighted” · columns: “Sort by popularity” · value per visitor of “Revenue per visitor”

Revenue per visitor by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	17.331,148 vis.	17.211,139 vis.
B · Variant	17.931,163 vis.	18.311,157 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -9.20 and +10.21, a range wider than the effect these tests target (±0.87).

interaction gap +0.51 · range -9.20 → +10.21 · adjusted p 1.00

Product click · 4,607 shared visitors · no conclusion yet

Rows: “Pay-in-3 option highlighted” · columns: “Sort by popularity” · rate of “Product click”

Product click by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	35.0%1,148 vis.	36.1%1,139 vis.
B · Variant	35.2%1,163 vis.	34.9%1,157 vis.

No interaction proven, but the data cannot rule one out yet: the interaction difference is likely between -9.9 pts and +7.2 pts, a range wider than the effect these tests target (±1.8 pts).

interaction gap -1.3 pts · range -9.9 pts → +7.2 pts · adjusted p 1.00

Visitors seen by both tests, in the variant of their first exposure to each; the metric is counted after both exposures. 95% range, valid every time you check, with risk corrected across 5 comparisons.

Not decided yet.

Exposed visitors

11,680

A 5,850 · B 5,830

Converted visitors

4,708

at least 5 per variant to estimate

Planned sample size

21%

27,254 visitors per variant · planned end Nov 21, 2026

Variants2

AOriginal5,850 visitors · 2,340 converters · 40.0%

BSort by popularity5,830 visitors · 2,368 converters · 40.6%+1.54%

Variant	Visitors	Converters	Rate	Difference vs. A · range
AOriginal	5,850	2,340	40.0%	baseline
BSort by popularity	5,830	2,368	40.6%	+1.54%-2.8 → +6.1

D’où vient l’écart de B ?

Ce que compte le chiffre

La part des visiteurs exposés au test qui envoient « Product click » après l’avoir vu. Chaque visiteur compte une fois, dans la variante de sa première exposition.

Les comptes

A · Original : 2,340 convertis sur 5,850 visiteurs, soit 40.0%.
B · Sort by popularity : 2,368 convertis sur 5,830 visiteurs, soit 40.6%.

Le calcul

Écart relatif = taux de B ÷ taux de A − 1 = 40.6% ÷ 40.0% − 1 = +1.54%. Méthode
Test de la différence entre les deux variantes : z = 0.68, p = 0.496 (bilatéral).
Fourchette à 95.0% : from -2.84% to +6.13%. Ce sont tous les écarts que ce même test ne rejette pas (méthode de Fieller) : « significatif » et « fourchette qui exclut zéro » disent toujours la même chose. Méthode

Ce qui limite la lecture

Répartition contrôlée : les visiteurs se répartissent comme prévu, aux variations du hasard près (p = 0.853). Méthode
L’échantillon prévu n’est pas atteint : même une fourchette qui exclut zéro ne permet pas encore de conclure, sauf si une borne de contrôle du plan est franchie. Méthode

A visitor counts as converted if they send “Product click” after seeing the test, in the variant of their first exposure.

Revenue per visitor by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	10.991,452 vis.	11.201,461 vis.
B · Variant	11.681,437 vis.	11.841,444 vis.

A · original

B · Variant

A · original

10.991,452 vis.

11.201,461 vis.

B · Variant

11.681,437 vis.

11.841,444 vis.

Product click by variant seen in “Simplified mobile menu” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	40.2%1,452 vis.	40.8%1,461 vis.
B · Variant	41.6%1,437 vis.	42.0%1,444 vis.

A · original

B · Variant

A · original

40.2%1,452 vis.

40.8%1,461 vis.

B · Variant

41.6%1,437 vis.

42.0%1,444 vis.

Revenue per visitor by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	17.331,148 vis.	17.211,139 vis.
B · Variant	17.931,163 vis.	18.311,157 vis.

A · original

B · Variant

A · original

17.331,148 vis.

17.211,139 vis.

B · Variant

17.931,163 vis.

18.311,157 vis.

Product click by variant seen in “Pay-in-3 option highlighted” (rows) and “Sort by popularity” (columns)
	A · original	B · Variant
A · original	35.0%1,148 vis.	36.1%1,139 vis.
B · Variant	35.2%1,163 vis.	34.9%1,157 vis.

A · original

B · Variant

A · original

35.0%1,148 vis.

36.1%1,139 vis.

B · Variant

35.2%1,163 vis.

34.9%1,157 vis.

The peeking trap

The peeking trap

Sort by popularity

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Product click, day by day

Overlapping tests2

“Simplified mobile menu” × “Sort by popularity”

“Pay-in-3 option highlighted” × “Sort by popularity”

The peeking trap

Sort by popularity

Not decided yet.

Reliability

Variants2

Ce que compte le chiffre

Les comptes

Le calcul

Ce qui limite la lecture

Product click, day by day

Overlapping tests2

“Simplified mobile menu” × “Sort by popularity”

“Pay-in-3 option highlighted” × “Sort by popularity”