How split testing can make or break your email open rates

Katie Lewis

Sometimes we have to throw everything good we’ve learned out the window.

Examples: When there’s a “Game of Thrones” marathon on HBO but the house is a wreck, or when there’s a spinach salad on the menu underneath a 10-cheese lasagna special.

Ever since Emma’s subject line split testing feature first launched, we’ve gleaned a wealth of information from our own internal tests as well as those our customers have shared.

Below are four case studies that found the following: laying out your cards isn’t always the best policy, a thoughtful subject line is worth the time spent on it and limiting your appeal can yield a yawn of a response.


Testing the level of detail revealed

Case Study 1: The appeal for help

One of our early split tests was an email announcing Emma 25, a program that invites nonprofits to apply for free email service.

The variations—three of 'em—ranged from the less detailed "Help us find this year's Emma 25 honorees" to the more informative "Emma 25 is here: Help a small nonprofit win email for life."



The winner? A third variation, which landed somewhere in the middle of succinct and detailed: "It's Emma 25 time: We're giving away free email service!" 

This version boasted a 15% lift and, in the end, gained us 2,700 more opens.


Case Study 2: The sale announcement

Our client BOCA sent a split test in mid-December letting Sanctuary Medical Center customers know about holiday discounts, and they found that a little mystery about the sale entices the recipient to find out more.

The subject line “Limited-Time Pricing on Select Services” squeaked by with nine more opens than “FREE Clarisonic Offer + 15% Off Fractional Resurfacing & Great Gift Ideas.”



That narrow margin meant more to BOCA than you might think. When you play it out, sending the winning variation meant 193 more opens, a 13% lift.

BOCA set their test to "autosend," as well, allowing them to finish the test in a hands-off way rather than having to manually send the winning subject line to the remaining recipients.

Testing the subject matter itself

Case Study 3: The monthly newsletter

There was an obvious winner in Belron US’s test: “Vehicle Maintenance Advice from Safelite AutoGlass” flounced the run-of-the-mill “Safelite AutoGlass Car Care Newsletter” subject line. 


And it paid off: The winning subject line yielded 4,883 more opens, an 18% lift.

This is just a good, old-fashioned reminder that something—nearly anything—is better than using “newsletter” in the subject line.

Here’s an example of an easy testing formula to use: In this case, “Vehicle Maintenance” could easily be replaced with “Interior Car Care” or “Roadtrip Checklist.” 

The subject lines are distinct yet similar enough that you can glean useful information from the success of one version over another. (In other words, don’t change capitalization and punctuation and branding and the offer.)


Case Study 4: The year-end email

“Brand new sets at our studio!” was the winning subject line over “25% Off Select Packages today and tomorrow only!” for Courtney Dellafiora Photography.

If your recipients don’t have the time, money or desire to purchase your product right now, you limit yourself by crafting a subject line that only reaches those in the market in the next day or so. On the other hand, it could provide the extra bit of urgency that drives someone to make a purchase. 



And that's the beauty of split testing. It helps you get to know your audience and learn what works best for them. 

The more split tests you perform, you won’t just think you know the best one—you will know the best one, plus you’ll come to better understand your audience’s habits in the process.

Just keep in mind your goal and what you want to achieve in your email when thinking up your subject lines for split testing. Do you want an increased open rate? More clicks? To grow your list?

There’s no need to rely on your gut when it comes to first impressions, and don’t gamble with your profitability: test, test, test.

Happy testing, and keep us posted with what you learn: @emmaemail.



Previous Article
72 new template designs that’ll make email marketing easier than ever
72 new template designs that’ll make email marketing easier than ever

Your time is important. There are only so many hours in the day, and far...

Next Article
Email marketing secrets from a creative freelancer
Email marketing secrets from a creative freelancer

What we learned from photographer Michelle Koechle

Want to engage your audience and grow your brand? Try Emma’s robust, easy-to-use product today.

Get a Demo