The probability of seeing a lift at least this large if the two versions performed the same. Below 5%, the lift is called significant; in Probant, that’s the same as a 95% confidence interval that excludes zero.
Le test mesure +6.0% sur le Revenue per visitor. Si les deux versions se valaient, un écart au moins aussi grand, dans un sens ou dans l’autre, aurait une probabilité de 0.002: la valeur p.
Elle est sous 5%: l’écart est significatif, et la fourchette à 95% (from +2.2% to +10.0%) exclut zéro. Les deux disent la même chose; la fourchette dit en plus de combien la variante fait mieux.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
The range of true lifts compatible with the data, at the 95% level. Probant always shows the lift with its range: if the range is entirely above zero, the variant wins; if it contains zero, nothing is proven.
The probability of declaring a winner when the two versions perform the same. Probant holds it at 5% over the whole test, checkpoints included. Reading a test every day at the usual threshold pushes it far higher.
The lift divided by its standard error: how many times the lift exceeds the expected noise. At each checkpoint, Probant compares the z-score with the O’Brien-Fleming boundary.
Create your workspace, or explore the demo first with data from a fictional store.