The probability of detecting the target effect if it really exists. Probant plans at 80% power: a real effect of the target size goes unnoticed 20% of the time.
Le plan vise +5.5% sur le Revenue per visitor avec une puissance de 80%: si la variante rapporte vraiment autant, le test a 80% de chances de le voir, et 20% de le manquer.
Si l’effet réel n’était que de +2.8%, la puissance tomberait à 29%: le test conclurait le plus souvent “pas d’effet détectable”, ce qui ne veut pas dire “aucun effet”.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
The smallest lift the test has a good chance of detecting if it exists, with 80% power and a 5% risk. The smaller it is, the more visitors you need: aiming for an effect half as large takes about four times as many visitors.
The number of visitors set by the test plan before launch, based on the target effect and the area’s traffic. Until it’s reached, only a boundary crossed at a checkpoint allows a decision.
The probability of declaring a winner when the two versions perform the same. Probant holds it at 5% over the whole test, checkpoints included. Reading a test every day at the usual threshold pushes it far higher.
Create your workspace, or explore the demo first with data from a fictional store.