The threshold the z-score must exceed at a checkpoint to conclude. Very high at the first checkpoint, it falls toward the usual threshold at the last one, so the overall risk stays at 5%. Probant computes it for 1 to 5 checkpoints.
Le plan prévoit three points de contrôle. Pour conclure, le score z doit dépasser 3.47 au jour 6, 2.45 au jour 12, and 2.00 au jour 18, contre 1.96 pour un test lu une seule fois au même risque de 5%.
Ce surcroît de sécurité au début coûte peu: 1.7% de visiteurs en plus que pour un test lu une seule fois, pour avoir le droit de s’arrêter plus tôt si l’écart est net.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
A moment set in the test plan when Probant reads the test to decide. Between checkpoints, the daily range is shown, but it isn’t enough to conclude. A test plan includes three by default, evenly spaced.
The lift divided by its standard error: how many times the lift exceeds the expected noise. At each checkpoint, Probant compares the z-score with the O’Brien-Fleming boundary.
The probability of declaring a winner when the two versions perform the same. Probant holds it at 5% over the whole test, checkpoints included. Reading a test every day at the usual threshold pushes it far higher.
Create your workspace, or explore the demo first with data from a fictional store.