Subject lines: Everything you know may be wrong. Part 4–The Results

So we’ve prepped our test question, organized our list into randomly sampled groups, made the big send, and waited a couple of weeks for the results.

Now is the moment of truth. The results.

And they tell us…

Oooh not good. In fact, this is 10% lower than our average open rate (not that a 40% open rate is anything to scoff at). However, Group 2’s (that’s the one at the top—just to be confusing) headline was all but identical to previous email headlines, which means that there was likely another reason for the low open rates.

More likely: our audience isn’t that in to PPC (and our previous PPC newsletters show it as well). As for open rate, headline 2 (following the old “6 tips to…” format) performed minutely better, not enough to give us meaningful data.

You will also notice the abysmal CTR, nearly half of our average. Ouch. Let’s get a little more info on this:

Email 1:

Email 2:

Hard to read, but if you squint you will see that the unsubscribe rate it low, and the predominant click is on an external article (not necessarily a bad thing). This is likely caused by my call to read it in the “from the editor” section, but it also could be a result of interest in subjects like keyword research—a subject that seems useful, but that few people really understand. Guess we have an idea for the next PPC blog post.

All and all I guess it was a little anti-climactic. Looks like David might have been right back in Part 1: “By changing titles drastically you risk losing readers in the short run, as they see something with a different title, and assume its something they don’t want.” As our changes were some what minor, our emails targeted at a small and dedicated group, and our company branding always prominent, the results where somewhat less than dramatic.