Picture the monthly meeting. Your email platform says campaigns and automations drove €48,000 in revenue, the open rate sits at a healthy 41%, and the click rate rose again. Then the finance director asks one question: if we stopped sending tomorrow, how much of that €48,000 would we lose?
Nobody in the room can answer, because none of the standard email marketing KPIs on that dashboard measures it. (The figures are an illustration, but you have probably sat in that meeting.)
After more than 30 years in digital marketing, I have watched email metrics drift from imperfect to unreliable. Privacy proxies open your emails and security scanners click your links. Your platform then credits itself with sales that would have happened anyway.
This article separates the numbers that still mean something from the ones you should retire, and gives you a framework and a 90-day plan to prove what email adds to your business.
Email Marketing KPIs Made Easy
Why your email marketing KPIs stopped telling the truth
The problem has been building since 2021, and it now affects every layer of the funnel. Each popular metric breaks in a different way, so it helps to look at them one at a time before deciding what to keep.
Opens: half the signal comes from machines
Apple Mail Privacy Protection downloads every image in an email, tracking pixel included, on Apple’s servers, whether or not a person ever reads it. Brevo’s 2026 benchmark puts the scale in plain view: across its customers, the average open rate was 20.73% with Apple’s automatic opens removed and 33.87% with them included. Omnivery, which sells bot detection, estimates that non-human opens have held near 50% of B2C opens since 2023. So when you pick a subject line because it “won” on opens, you are often picking the version that happened to land with more iPhone users.
The damage spreads beyond reporting. Send-time optimisation trained on open timestamps learns when Apple’s servers fetch mail, and engagement segments built on “opened in the last 90 days” keep people who never read anything looking active forever.
Meanwhile, AI summaries in Gmail and Apple Mail let readers absorb your message without the open or click you would normally record. Treat open rate as unknowable for performance purposes and keep it, at most, as a smoke alarm for sudden deliverability drops at one mailbox provider.
Clicks: the same problem, a few years later
Security gateways such as Microsoft Defender, Proofpoint and Mimecast test every link before your recipient sees the email, and mailbox providers have started pre-fetching links too. Omnivery measured B2C bot clicks rising from about 2% through 2023 to about 16% in 2026, with Gmail jumping from roughly 6% to 22% in late February 2026. In B2B it gets worse. Omeda found that more than 96% of clicks on event, whitepaper, research and webinar sends were bots in its Q4 2025 data. Both sources sell solutions to this problem, so take the exact figures as indicative, but the direction is clear.
Inflated clicks do more than flatter a report. In an automation, a bot click fires the same branch as a human click. Your “clicked the pricing page, alert the sales team” rule sends reps chasing scanners, and your “clicked, so skip the reminder” rule suppresses the follow-up a real person needed.
Attributed revenue: a correlation inside a time window
Your email platform’s revenue figure counts any order placed within a set window after someone opened or clicked. Klaviyo, for example, uses a 5-day window for email opens and clicks by default. Your most loyal customer, who buys every month regardless, gets counted as email revenue every time she glances at a newsletter. Add Apple’s automatic opens and your platform can credit a sale to an email nobody saw.
Google Analytics fails in the opposite direction, especially in Europe. When a cookie banner offers “Reject all” on its first screen, a USENIX Security study run with the French regulator CNIL found around 60% of people rejected tracking. Those visits and orders never reach GA4 with their email source attached. You end up with an ESP figure that over-counts and an analytics figure that under-counts, and the true number sits somewhere between them. Reporting either one as “email revenue” misleads the business.
Opens, raw clicks and attributed revenue all describe activity near an email. None of them tells you what changed because you sent it.
Three ways to count email revenue, and the one that holds up
Every company counts email revenue in one of three ways, whether it knows it or not. Two of them rely on attribution rules that someone chose, and the third relies on an experiment. The comparison below shows why the experiment wins, even for a small team.
A holdout group is a random slice of your audience that receives no marketing email for a set period.
You compare what they spend with what everyone else spends. The gap is incremental revenue: money that exists because you sent email. It captures influence that never produced a click, such as someone who read an AI summary and then typed your address into a browser, and it ignores the loyal customer who would have bought anyway.
The four-layer framework for email marketing KPIs
A holdout gives you the final answer, but you cannot wait 90 days to learn whether this week’s send went to spam. You need a short stack of metrics, arranged so that each layer protects the one above it.
Here is the stack we use with clients.
| Layer | Check weekly | Report monthly or quarterly |
|---|---|---|
| 1. Deliverability guardrails | Gmail spam rate, compliance status, bounce rate | Inbox placement trend, rejections |
| 2. Human engagement | Unique human click rate, replies, bot share of clicks | Share of list engaged in the last 90 days |
| 3. Conversion efficiency | Revenue per recipient, conversion per recipient | Margin per recipient after discounts |
| 4. Business outcome | Holdout running, group sizes healthy | Incremental revenue and margin; B2B pipeline; fully loaded ROI |
Layer 1: deliverability guardrails
Nothing else matters if your email never reaches the inbox, and the dashboards changed under everyone’s feet last year. Google retired Postmaster Tools v1 and its reputation charts on 30 September 2025, so the user-reported spam rate and the new Compliance Status page are now your main Gmail signals.
Keep the spam rate below 0.1% and never let it reach 0.3%, the point at which Gmail stops cutting you any slack. Since November 2025, Gmail rejects mail from bulk senders that fail its authentication and unsubscribe rules, which our guide to email sender requirements in 2027 covers step by step.
Read “delivered” with suspicion. It means the receiving server accepted the message, and says nothing about whether it reached the inbox. Validity’s 2026 benchmark puts global inbox placement at 87.2% and Europe at 91.1%, so a dashboard showing 98% delivered can hide one email in ten going to spam or disappearing. Seed-list tests help you spot trends, but seed accounts have no reading history, so treat them as a thermometer, not a verdict.
Layer 2: human engagement
Your best remaining engagement metric is the unique human click rate: people who clicked after you remove bots, divided by emails delivered. Click-to-open rate is finished, because half its denominator comes from machines. Most platforms now offer some bot filtering, so switch it on first, then add your own rules on top.
Omeda publishes a useful starting set. It discards a click that follows another within two seconds, and all of a recipient’s clicks when there are ten or more in 30 seconds or an unusually high number of IP addresses and browser signatures. Add a hidden “honeypot” link that no human can see, because anything that clicks it is a machine. Then check whether a click led to a real session on your website. A click with no visit behind it is almost always a scanner.
Layer 3: conversion efficiency per recipient
Revenue per recipient (RPR) divides revenue by emails delivered, which removes the effect of list size and send frequency. It also exposes the gap between automations and campaigns. Klaviyo’s 2026 benchmarks show flows producing about 41% of email revenue from 5.3% of sends, with roughly 18 times the revenue per recipient of campaigns. That is a strong argument for investing in automation.
It is also attributed revenue, with all the problems above. Flows fire when someone has already shown intent, by leaving a cart or browsing a product, so part of that 18× comes from who receives them rather than what the email does.
Flows need holdouts more than campaigns do, especially discount-led flows that may hand margin to people who were about to buy at full price. Track margin per recipient alongside RPR for exactly this reason, and measure conversion per recipient (orders divided by delivered) so bot clicks never pollute the denominator.
Layer 4: incremental business outcome
This is the layer you show the finance director. For ecommerce, it means incremental revenue and margin from your holdout, repeat-purchase rate and time to second purchase compared with the held-out group, and customer lifetime value by cohort.
For B2B, it means sales-qualified leads, opportunities and pipeline value per 1,000 recipients, taken from your CRM and compared with a held-out group of accounts. Finish with a fully loaded ROI: incremental gross margin minus every cost (platform licences, data, people, agency fees, creative and discounts), divided by those costs.
How to run an email holdout test without a data team
Most of the clients who come to us assume holdout testing belongs to companies with data scientists. It needs a random split and one spreadsheet that joins your email list to your orders. The design decisions are few, and getting them right matters more than any tool.
Randomise people, not individual sends, and keep each person in the same group for the whole test. Hold out 10% of your marketable contacts if your list is large, and 15–20% if you have fewer than about 10,000, because a tiny control group produces noise. Exclude transactional email such as receipts and password resets, since nobody should miss those. Run the test for at least one full purchase cycle; Rejoiner, for example, compares revenue per customer at around 90 days. Short tests overstate lift because the held-out group has not had time to buy through other routes.
Take the outcome from your order system or CRM, never from the email platform. The two numbers you need are average revenue per person in the mailed group and in the holdout. Incremental revenue equals the difference multiplied by the number of people mailed, and lift equals the difference divided by the holdout’s average. Divide incremental revenue by what your platform attributed and you get an incrementality factor, which you can use to discount dashboard figures until the next test.
As an illustration: you have 20,000 marketable contacts and hold out 2,000. Over 90 days the mailed group spends €14.20 per person and the holdout €12.90. Lift is 10.1%, and incremental revenue is €1.30 × 18,000 = €23,400. If your platform attributed €61,000 to email in the same period, your incrementality factor is 0.38. At a 40% gross margin that is €9,360 of incremental margin; against €4,500 of total costs, email returns about €1.08 of profit for every €1 spent.
That illustration also shows the main limitation. A 2,000-person holdout produces wide error margins, so report a range rather than a single number. To detect a 20% relative lift on a 2% conversion rate with standard statistical confidence, you need about 21,000 people in each group (my own calculation, using a conventional two-group test at 95% confidence and 80% power).
A smaller list should test the whole email programme rather than individual campaigns, use a larger holdout share, measure a frequent outcome such as any purchase, and run longer.
Your platform may handle the split for you. Braze lets you set a global control group of 1–15% of users with built-in uplift reporting. Klaviyo’s Global Holdout Groups require at least 400,000 profiles, so most SMEs build it manually: a random segment excluded from every campaign, a filter at the start of each flow, and an export joined to Shopify or CRM data. It takes an afternoon to set up.
What honest numbers look like
Expect your first holdout to deflate your dashboard. The best published experiments show email working, by margins far smaller than attribution suggests. A peer-reviewed field test with an Italian beauty retailer, published in the Journal of the Academy of Marketing Science, measured a 6.5% revenue lift from the email and direct-mail mix it tested.
For a business turning over €2 million a year, 6.5% is €130,000, which is a serious contribution.
It is also a long way from the 30–40% of revenue some dashboards claim.
Holdouts also reveal costs that campaign reports hide. In a field experiment at a large North American retailer, halving the frequency of coupon emails cut unsubscribes by 59% while reducing short-term revenue by 5–8%, and customers who unsubscribed went on to spend 36% less per month. Sending more lifts this week’s revenue and quietly erodes next year’s. Only a test that runs long enough will show you that trade-off.
The same caution applies to the ROI figures quoted across the industry. The famous “$36 for every $1” comes from Litmus surveys of marketers, and the DMA UK’s figure of almost £41 per £1 comes from 250 marketers estimating their own returns. Both are averages of self-assessments, usually built on attributed revenue and partial costs. They are useful for arguing that email costs less than paid media. They are not evidence of what email earns for your company, and your board deserves your own number.
B2B: when bots read your whitepaper
B2B teams suffer most from inflated clicks, because corporate security gateways scan nearly everything and sales teams act on the signal. If your CRM scores leads on email clicks, a large share of your “hot” leads may be Microsoft servers. Never trigger a sales alert on a click alone. Require a human signal first: a website session of meaningful length, a reply, a form submission or a booked meeting.
Measure B2B email by what reaches the pipeline. Report sales-qualified leads, opportunities and pipeline value per 1,000 recipients, and run the holdout at account level, because buying groups share inboxes, forward emails and talk to each other.
Keep sales reps’ one-to-one email outside the test and hold out nurture programmes only. Replies and meetings are among the few signals bots cannot fake, so give them more weight in your scoring than any click.
Where AI and automation help, and where they don’t
Measurement is one of the most useful places to put automation, because the work is repetitive and rule-based.
A nightly Make or n8n scenario pulls click events from your email platform, applies your bot rules, flags human clicks on each contact record, joins the holdout flag to your orders and refreshes a dashboard. A second scenario alerts you on Slack when the Gmail spam rate creeps above 0.08%, or the bot share of clicks jumps overnight.
- ✓Applying bot-click rules every night and flagging human clicks on each contact
- ✓Joining holdout flags to orders or CRM opportunities and refreshing a dashboard
- ✓Alerting you when the Gmail spam rate or bot share of clicks jumps
- ✓Writing the formulas or SQL for your analysis and drafting the monthly report
- ✓Classifying replies by sentiment and intent
- −Estimating incrementality from observational data with no experiment behind it
- −Deciding which business outcome matters for your company
- −Proving their own predictive scores work without a holdout to test them against
- −Catching delayed security scans that fall outside simple time-window rules
Large language models earn their place in the analysis layer. They write the SQL or spreadsheet formulas for your holdout join, classify reply sentiment, spot anomalies in exported logs and draft the monthly narrative.
Keep them away from estimating incrementality from observational data, because they will produce a confident number with no experiment behind it. Every figure an AI gives you should trace back to the joined table, and the same goes for the predictive scores your email platform’s AI features produce: judge them against a holdout before you trust them.
Your 90-day plan
In the first two weeks, clean your instruments. Switch on bot-click and Apple-open exclusion in your email platform, change attribution to clicks only with a short window, add a honeypot link to your templates and connect Google Postmaster Tools v2 and Microsoft SNDS. Then build the one table that matters: every email recipient, their holdout flag and their orders or CRM opportunities, matched by email address or customer ID.
In week three, start a programme-wide holdout of 10% (or 15–20% on a smaller list) covering campaigns and non-transactional flows. From week four, add flow-level holdouts on your two biggest revenue flows, usually the abandoned cart and the welcome series, starting with whichever carries a discount. Stop presenting your platform’s revenue as the email number. Show it next to the GA4 figure as an upper and lower bound, and say plainly that the real answer is coming.
On day 90, report incremental revenue, incremental margin and fully loaded ROI with a range, plus your incrementality factor. Keep a permanent 5–10% control group running afterwards so the measure never goes stale. Then go back to the finance director with a number you can defend.
Score your email programme before your next board meeting
The Email Marketing Audit Toolkit walks you through deliverability, tracking and measurement checks so you know which of your numbers to trust and where your revenue leaks.
Leave a Reply