How We Test Email Marketing Software: Our 30-Day Deliverability Benchmark

Every email marketing review on EmailCompare is based on hands-on testing, not press releases or vendor demos. We buy subscriptions anonymously, send real emails to real inboxes, and measure the results. Here is exactly how our 30-day testing protocol works—and why you can trust our recommendations.

The Testing Stack

Infrastructure

Seed lists: GlockApps synthetic seed list (800 addresses) + 200 real team addresses across Gmail, Outlook, Yahoo, and Apple Mail
Testing period: 30 days per platform
Email volume: 5,000 emails per platform (welcome sequences, promotional blasts, abandoned cart flows, re-engagement campaigns)
Tracking tools: GlockApps, Mail-Tester, Google Analytics UTM tracking, custom webhook logging
IP warming: All accounts start with 100 emails/day for 3 days before full volume

What We Measure

Metric	Weight	How We Test
Deliverability	35%	Inbox placement rate across 4 ESPs, spam folder rate, missing rate
Automation	25%	Workflow build time, trigger variety, conditional logic depth, template quality
Ease of Use	20%	New-user onboarding time, template builder speed, mobile experience
Value	15%	Price per subscriber, feature-to-price ratio, free tier usefulness
Support	5%	Live chat wait time, ticket resolution quality, knowledge base depth

Deliverability Testing in Detail

Step 1: Seed List Setup

We import 800 GlockApps synthetic addresses and 200 real addresses into each platform. The synthetic addresses are distributed across:

Gmail (40%)
Outlook (25%)
Yahoo (20%)
Apple Mail (15%)

This distribution mirrors real-world email client market share.

Step 2: Campaign Types

Over 30 days, we send 5 campaigns per platform:

Welcome sequence (1,000 emails): 5-email automated sequence to new subscribers
Promotional blast (1,000 emails): Single promotional email with CTA
Abandoned cart flow (1,000 emails): 3-email recovery sequence (e-commerce platforms only)
Re-engagement campaign (1,000 emails): 3-email sequence to inactive subscribers
Newsletter broadcast (1,000 emails): Weekly newsletter-style content

Step 3: Inbox Placement Analysis

After each campaign, we check GlockApps reports for:

Inbox placement percentage
Spam folder percentage
Promotions tab percentage (Gmail only)
Missing/bounced percentage

We also manually check 20 real inboxes to verify synthetic data accuracy.

Step 4: Reputation Monitoring

We monitor:

Sender Score (senderscore.org)
Google Postmaster Tools
Microsoft SNDS
Spam complaint rates per platform

A platform with a spam complaint rate above 0.1% gets an automatic deliverability penalty.

Automation Testing in Detail

Build Time Benchmarks

We time how long it takes to build three standard workflows:

Welcome sequence: 5 emails, 2 conditional branches (opened vs. didn't open)
Abandoned cart: 3 emails, 1 conditional branch (purchased vs. didn't purchase)
Re-engagement: 3 emails, final "remove from list" action

Our build time targets:

Under 20 minutes: Excellent (⭐⭐⭐⭐⭐)
20–30 minutes: Good (⭐⭐⭐⭐)
30–45 minutes: Average (⭐⭐⭐)
Over 45 minutes: Poor (⭐⭐)

Trigger Variety Score

We count the number of unique trigger types available:

40+ triggers: Excellent
25–39 triggers: Good
15–24 triggers: Average
Under 15 triggers: Poor

Conditional Logic Depth

We test how complex branching can get:

Simple: IF/THEN only (AWeber)
Moderate: IF/THEN/ELSE with 2 branches (ConvertKit, Moosend)
Advanced: Nested IF/THEN/ELSE with multiple conditions (GetResponse)
Enterprise: Full visual programming with variables, scoring, and API calls (ActiveCampaign)

Why Anonymous Testing Matters

We never accept free review units or enterprise trials from vendors. All accounts are purchased with personal credit cards under fake business names. This ensures:

No priority support: We get the same support experience as any paying customer
No whitelisted IPs: Our deliverability tests reflect real shared IP conditions
No feature unlocks: We test exactly what a new customer gets on day one
No vendor influence: Our reviews are 100% independent

Limitations of Our Testing

Our methodology has constraints you should know:

List size: We test with 1,000 subscribers. Results may differ for 100,000+ subscriber lists
Industry variance: A B2B SaaS newsletter performs differently than a fashion e-commerce store
Time variance: Deliverability changes as IP reputation evolves. Our 30-day snapshot is representative but not permanent
Geographic bias: Our tests are US-centric. European and Asian deliverability may differ

How to Replicate Our Tests

Want to verify our results? Here is the DIY version:

Sign up for a free trial (no vendor contact)
Import 100 real addresses + 100 GlockApps seeds
Send 5 campaigns over 14 days
Track with GlockApps ($25/month) and Mail-Tester (free)
Compare your results to our published data

Most platforms offer 14–30 day trials, so you can run a mini-version of our test for under $50.

The Bottom Line

Our testing is not perfect, but it is the most rigorous independent email marketing review process on the web. We send real emails, measure real inboxes, and publish real data. When we say ActiveCampaign has 97.1% deliverability, that number came from 5,000 actual emails landing in actual inboxes—not a vendor press release.

See our latest email marketing reviews →