William Charlwood

If you are doing any sort of testing then you need to make sure you have enough data to make rational decisions. I recently ran a split test using my autoresponder. What I did was send out two different emails to a relatively small list (around 400) and measured the response rates.

It was a classic A/B Split Test in other words although the sample size was quite small.

Here’s Version A of my email

Hello Firstname

Recently I mentioned a free ebook that makes me good money.

I didn’t write it. It is free to read. You don’t need to supply
an email address to get access to it.

But if you read it you’ll discover something pretty neat.
The book is quite short and very punchy.

To make it work for you here’s a very simple idea: put your
link to the book as a signature in your emails.

You can copy the example below if you want.

With best wishes

William Charlwood

PS Discover the ultimate online super tip.
http://www.supertips.com/ultimate/x/?id=104

And here is Version B

Hello Firstname

Recently I mentioned a free ebook that makes me good money.

I didn’t write it. It is free to read. You don’t need to supply
an email address to get access to it.

But if you read it you’ll discover something pretty neat.
The book is quite short and very punchy.

To make it work for you here’s a very simple idea: put your
link to the book as a signature in your emails.

You can copy the example below if you want.

With best wishes

William Charlwood

Discover the ultimate online super tip.
http://www.supertips.com/ultimate/x/?id=104

.

Spot the difference!

Actually the only difference was the use of “PS” in version A.

Initially I was intrigued to see that Version A was significantly more successful in terms of click through rates. At one point there was a 91% likelihood that its relative out-performance would be repeatable. In other words, statistically it looked as though a similar exercise done again would show that Version A generated a higher click through rate 91% of the time.But as data dribbled in, the relative response rates got closer and closer. At the moment Version A has generated a click through rate of 23.5% and Version B a click through rate of 22.7%. These click through rates are quite high for emails and often one problem you face is getting people to read them in the first place.

The figures themselves are too close to separate and suggest that the results of this particular split test are neutral which is a shame! I rather enjoyed having made what looked like an interesting discovery, namely that using PS as opposed to not using PS made a difference.

But I was wrong to leap to this conclusion too early.

Just for completeness, the subject line of both emails was the same:

A Quick Suggestion