The lift divided by its standard error: how many times the lift exceeds the expected noise. At each checkpoint, Probant compares the z-score with the O’Brien-Fleming boundary.
Au dernier contrôle (jour 18), l’écart de Revenue per visitor vaut 6.02% et son erreur type 1.95%: score z = 6.02% ÷ 1.95% = 3.09.
L’écart dépasse donc le bruit attendu 3.09 fois. La borne d’O’Brien-Fleming de ce contrôle est 2.00: elle est franchie, le test conclut.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
The threshold the z-score must exceed at a checkpoint to conclude. Very high at the first checkpoint, it falls toward the usual threshold at the last one, so the overall risk stays at 5%. Probant computes it for 1 to 5 checkpoints.
The probability of seeing a lift at least this large if the two versions performed the same. Below 5%, the lift is called significant; in Probant, that’s the same as a 95% confidence interval that excludes zero.
A moment set in the test plan when Probant reads the test to decide. Between checkpoints, the daily range is shown, but it isn’t enough to conclude. A test plan includes three by default, evenly spaced.
Create your workspace, or explore the demo first with data from a fictional store.