A range you can reread at any time and stop on at the first crossing without inflating the risk (Johari, Pekelis, and Walsh, 2017). It’s wider than a classic confidence interval. Probant uses it for rollout guardrails.
Autre nom : mSPRT, confidence sequence
50/50 test, garde-fou “Return rate”, limite +25%. La fourchette classique va from +45.8% to +109.9%; la fourchette toujours valide, from +25.4% to +130.3%, plus large.
C’est le prix de la relecture à tout moment: on peut s’arrêter au premier jour où elle dépasse la limite sans gonfler le risque. Ici, toute la fourchette dépasse la limite: le garde-fou est franchi.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
A metric that must not get worse (return rate, page speed, errors), with a limit set in the test plan. A guardrail stops the test only if its whole range exceeds the limit; a range that straddles the limit doesn’t prove harm.
Automatic withdrawal of a version being rolled out as soon as a guardrail is crossed, that is, as soon as its whole always-valid range exceeds the limit. All traffic goes back to the original.
The range of true lifts compatible with the data, at the 95% level. Probant always shows the lift with its range: if the range is entirely above zero, the variant wins; if it contains zero, nothing is proven.
Create your workspace, or explore the demo first with data from a fictional store.