Measure margin, not just revenue

Revenue measures what the customer pays

An e-commerce test is often judged on conversion rate or revenue per visitor. Both metrics are easy to collect, since the store’s script knows the amount of each order at checkout. They measure what the customer pays. They say nothing about what the store keeps.

As long as a test only changes how a page looks, the difference matters little: one more order brings in roughly the usual margin. Once a test touches what an order costs, the two metrics diverge. A discount, or free shipping above a threshold, can increase sales while reducing the margin on each order. The revenue gain can then cost more than it brings in.

The free-shipping bar raises revenue by +6.0%. What about margin?

In the demo store, the “Free shipping progress bar” test shows in the cart how much is left to spend before shipping becomes free. Over 18 days and 61,240 visitors, the report gives these results:

Demo store (fictional). Margin per visitor: revenue, minus product costs, discounts, and the shipping costs the store covers, divided by exposed visitors.

The two metrics don’t say the same thing. The variant raises revenue per visitor by +6.0% (from +2.2% to +10.0%); it has no established effect on margin per visitor (+3.0% measured, from -0.7% to +6.7%). The margin interval goes below zero: -0.7%, versus +2.2% for revenue.

Free shipping absorbs the bigger cart

Revenue per visitor breaks down into two factors: the share of visitors who order, and the average order value. Margin per visitor breaks down the same way: the share of visitors who order, and the average margin per order. The report’s differences let you separate the two.

• Conversion rate: +3.9%. More visitors order, and each extra order brings in margin.
• Average order value: +2.0%. Customers add an item to reach the free-shipping threshold.
• Margin per order: -0.90%. The bigger cart doesn’t leave more margin, because shipping is now free on a larger share of orders: 57.0% versus 41.0%, or +16 points.

The margin gain therefore comes mostly from the extra orders, while the bigger cart pays for shipping. A team that judges this test on revenue alone credits the variant with a gain the store doesn’t keep.

In euros, the gap is plain to see

The report converts the differences to a month of cart traffic, after correcting for the winner’s curse (see “The winner’s curse: why the announced gain falls short”). Amounts are rounded to the nearest hundred euros.

Expected revenue per month

+€125,700range from +€41,600 to +€209,800

Expected margin per month

+€27,700range from -€5,400 to +€60,900

Extra free shipping

-€23,800per month, already deducted from margin

A team that judges the test on revenue will announce +€125,700 per month. The store will keep +€27,700 of margin, once products, discounts, and shipping on free-shipping orders are paid for.

On margin, the interval runs from -0.7% to +6.7% and includes zero: the margin gain isn’t established, even though revenue goes up.

Margin is measured with back-office data

A page script knows the amount paid. It knows neither product costs nor actual shipping costs. Margin therefore comes from the back office: revenue, minus product costs, discounts, and the shipping costs the store covers.

In Probant, it’s calculated on the server, never in the page. It uses the orders matched to exposed visitors and their line items (Shopify webhooks, or the orders API with the workspace’s secret key), the unit cost of each product in effect on the order date (synced from Shopify, imported as a CSV, or sent with the line item), and the fees set in the workspace. Neither cost nor margin goes through the script, so anyone reading the page’s requests can’t see them.

POST /api/v1/orders
Authorization: Bearer sk_…
Content-Type: application/json

{ "orderId": "1001", "visitorId": "…", "amount": 89.90,
  "tax": 14.98, "currency": "EUR", "createdAt": "2026-11-10T08:30:00Z",
  "lines": [
    { "sku": "BOUGIE-180", "quantity": 2, "unitPrice": 29,
      "discount": 5.8, "unitCost": 6.4 }
  ] }

The built-in margin-per-visitor metric can then decide a test, or explain it next to revenue. The report shows the share of revenue covered by known costs, and only concludes on margin if that share reaches 90%. Field details and limits are in the documentation.

Matching back-office orders also lets you check the measurement: in the demo report, the matched-orders check shows 98.6%.

Probant puts margin next to revenue

The demo store’s report shows both measures side by side (see Reports):

• margin per visitor next to revenue per visitor, each with its range;
• the share of free-shipping orders, which explains the gap between the two;
• monthly revenue and margin amounts, corrected for the winner’s curse (without a margin-specific history, the correction obtained on revenue is applied to margin as is);
• the monthly cost of the extra free shipping, already deducted from margin.

The report summary draws the next step: “Expected revenue: +€125,700 per month after correcting for the winner’s curse. Then test a €50 threshold, measured on margin.”