A moment set in the test plan when Probant reads the test to decide. Between checkpoints, the daily range is shown, but it isn’t enough to conclude. A test plan includes three by default, evenly spaced.
Autre nom : interim analysis
Le plan en prévoit three. Jour 6, score 2.30 contre une borne de 3.47: pas franchie, le test continue; jour 12, score 2.32 contre une borne de 2.45: pas franchie, le test continue; jour 18, score 3.09 contre une borne de 2.00: franchie, le test peut conclure.
Au jour 6, un test lu une seule fois aurait déjà conclu (score au-dessus de 1.96); la borne du premier contrôle, plus stricte, a évité de s’arrêter sur un écart encore fragile. Entre deux contrôles, la fourchette du jour s’affiche, mais elle ne décide de rien.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
The threshold the z-score must exceed at a checkpoint to conclude. Very high at the first checkpoint, it falls toward the usual threshold at the last one, so the overall risk stays at 5%. Probant computes it for 1 to 5 checkpoints.
The lift divided by its standard error: how many times the lift exceeds the expected noise. At each checkpoint, Probant compares the z-score with the O’Brien-Fleming boundary.
The probability of declaring a winner when the two versions perform the same. Probant holds it at 5% over the whole test, checkpoints included. Reading a test every day at the usual threshold pushes it far higher.
Create your workspace, or explore the demo first with data from a fictional store.