A correction that pulls the measured lift toward the average of past tests, more so when the test is small. This is how Probant corrects for the winner’s curse.
Le test mesure +6.0% sur le Revenue per visitor, avec une erreur type de 1.9%. Les 48 tests passés de la boutique valent +1.7% en moyenne. Le rétrécissement rapproche l’estimation de cette moyenne: +4.9%.
Le gain mensuel annoncé suit: +€125,700 par mois après correction, au lieu de +€153,300 sur l’écart brut. Plus le test est petit, plus la correction est forte.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
What past tests say about the typical size of real effects, before you look at a new test. Probant estimates it from the store’s completed tests and uses it to correct for the winner’s curse.
The tendency of a winning test’s reported gain to overstate the real gain. Among many tests, the ones that clear the threshold are partly the ones chance favored, so their measured lift is too good on average.
The difference between the variant and the original, relative to the original: B / A − 1. A positive effect means the variant does better, in proportion to the starting value.
Create your workspace, or explore the demo first with data from a fictional store.