What past tests say about the typical size of real effects, before you look at a new test. Probant estimates it from the store’s completed tests and uses it to correct for the winner’s curse.
Probant estime l’a priori sur les 48 tests terminés de la bibliothèque: les effets réels y valent +1.7% en moyenne, avec une dispersion de 3.4% d’un test à l’autre.
Le test “Free shipping progress bar” mesure +6.0% sur le Revenue per visitor. Rapproché de cet a priori, l’effet attendu devient +4.9%: c’est ce chiffre, et non le premier, qui sert à estimer le gain mensuel.
Chiffres calculés par le moteur de Probant sur la boutique de démonstration, Maison Lune, fictive : ouvrir la démo.
A correction that pulls the measured lift toward the average of past tests, more so when the test is small. This is how Probant corrects for the winner’s curse.
The tendency of a winning test’s reported gain to overstate the real gain. Among many tests, the ones that clear the threshold are partly the ones chance favored, so their measured lift is too good on average.
The difference between the variant and the original, relative to the original: B / A − 1. A positive effect means the variant does better, in proportion to the starting value.
Create your workspace, or explore the demo first with data from a fictional store.