More Thoughts on Split Testing - or Why I Think Internet Marketing is an Art , Not a Science

A month or so ago I wrote about a split test that I'd conducted with my newsletter and how it had left me feeling rather confused.

I was talking about this recently with a friend and he told me a similar story.  This man, like me, is into internet marketing.  He has several Twitter accounts that he uses to try to get people onto his mailing list.  Like many internet marketers, when someone follows him on Twitter, he sends a direct message offering them some e-books.  The message gives a link to his website where the visitor is invited to give his or her name and email address in exchange for the books.

Because the website link is quite long, he uses Tiny.cc to shorten it.  A couple of months ago, it occurred to him that, if he were to use a different shortened link for each Twitter account, he could test different messages and see which got the most conversions.

He arbitrarily divided his accounts into three groups and wrote three different messages.  After a few weeks, he checked how many direct messages he'd sent out with each link and how many clicks each one had received.  The results were interesting, but not in the way he had been hoping.

In the first group, account A had had a larger percentage of clicks than account B or account C and the difference was statistically significant.  Similarly, in the second group, there was a statistically significant difference between the results of two of the accounts.  And yet, comparing the top converters in each group with each other, the difference was not significant.  And this was the same for the lowest converters.

So what are we to conclude from this?  We could use the quotation frequently attributed to Mark Twain about 'lies, damn lies and statistics' and say that 'statistically significant' doesn't necessarily prove anything.  Or we could say - as is certainly the case - that every single person who was sent a direct message was an individual and, as such, could not logically be expected to act in the same way as anyone else.

All we know is that the people who clicked the links were interested in internet marketing.  And we have to assume that a fair proportion of the people to whom the messages were sent were also interested, because they had decided to follow my friend.  But here we run into the first difficulty.  My friend follows about 30 new people a day.  He tries to pick people with similar interests but it's impossible to be sure.  Some of the people who follow him may just be following back because that's what they do, rather than out of interest.  We can assume that anyone who decides to follow him (without first having been followed) must do so because of interest in his tweets.  But, even here, we cannot be sure that they would want the e-books.  They may, themselves, be established marketers or they may have downloaded a large number of e-books and not have time or energy to read any more.  Or, of course, they may not read their direct messages.

Looked at like this, it's little wonder that the results were confusing.  And it's the same - indeed, it must be the same - in any situation where individual choice is concerned.  If 100 people watch a movie, it's unlikely that all of them will enjoy it equally.  If 90 per cent love it, it will be a runaway success . . . but the other 10 per cent may hate it. 

What all this has proved to me is the vital importance of not losing sight of our target audience.  What is it that they (or, rather, most of them) want?  Can we understand their motivation?  If they're not buying our products is it necessarily because we've written a bad sales letter?  Or is it because they've already got a similar product, or can't see the need for the product - or even that they've not got the money available to spend on it?  Whether they open our emails and whether they buy our products depend on many factors and I think we need to keep this in mind.  What works one day may not work the next.  Obsessing overmuch about split testing may, in the end, be counterproductive. 



Photo:  Iconfinder.com, designer: http://ivancoyier.deviantart.com/

Have You Heard of URIBL?

Have you heard of URIBL? Well, maybe you have . . . but I hadn't until last week. It describes itself as "a real time URL blacklist served via DNS to identify unsolicited bulk and commercial email." And that's something, I imagine, that most of us would applaud. People can submit sites for inclusion on the list. This makes it harder for emails from that site or mentioning that site to get through spam filters.

However, I found out about URIBL not because I wanted to report a site . . . but because I discovered I was ON it!!! I was sending a newsletter from my autoresponder to my mailing list, telling them about my previous post on this blog: What do You Need to be a Successful Internet Marketer? When I'd finished writing it, I checked it and clicked the 'spam' button which tells you (based on a number of factors) how likely your email is to get through people's spam filters (bearing in mind, as I've moaned about before, that on Microsoft Outlook, even if you whitelist addresses, they can still get shoved into the spam folder).

Everything about my newsletter was fine . . . everything, that is, except for the fact that I was blacklisted by URIBL. This was the first I'd heard of URIBL. I had no idea what it was . . . and I was worried because, although it didn't seem to mean that my emails wouldn't get through, it reduced their chances. Now, unfortunately, the message about this on my autoresponder was in lower case, and I read it as URLBL. So when I tried Googling it, nothing came up. Not knowing quite what to do about it, I put it on one side until I could ask someone who might know the answer.So I continued with a piece of writing I was doing - and then went to www.tiny.cc to shorten a link I wanted to insert. (If you've never used Tiny, I'd recommend it - you put in your link, click a button, and your new link comes up. If you register (which is free), you can also keep a record of all your abbreviations, so that you can use them again.)

Well, as I say, I went on to Tiny to shorten a link to this blog. And a big red sign came up saying "Check terms and conditions". Which I did. And I found that Tiny won't shorten links to sites that are blacklisted on URIBL. Now it was getting serious! But at least I did now have the correct title of the site and was able to find it. I checked and, yes, I was on their list. Fortunately, it's fairly easy to ask to be removed - you put in your website url and explain why the listing is incorrect. I told them that I sent emails only to people who had given them to me through my opt-in box, that all my emails have an 'unsubscribe' link at the bottom, and that I had never sent an unsolicited email in my life.

I was taken off the list very quickly - although they didn't let me know; I had to go to the site and look. But it left me wondering how on earth they could think that a blog hosted by Blogger was capable of sending out bulk unsolicited email. So I had another look at the website and worked out what must have happened. Under 'List Information' it states: "URIBL lists domains that appear in spam, NOT where they were sent from." Which, to me, seems a very odd way of doing it.

So, why am I going on about this? Well,there is a lesson, I think, to be learned from it. I have to assume that someone on my mailing list received one of my emails in which I gave a link to my blog, and for some reason that person thought it was spam. And this could be because I've not been sending out emails very frequently . . . perhaps every ten days or so, which means it's quite easy for a recipient to forget that he or she actually signed up to the list.

When someone signs in to my opt-in box, to acquire the four books on internet marketing that I'm offering (have you got yours yet?), they are taken to a 'thank you' page where they're told that they will receive an email explaining how to download them, and they're given the address it will come from and asked to whitelist it. But, clearly, that's not enough. We obviously need to keep reminding them who we are - and, while not advocating sending an email a day, perhaps three a week might be best!

I'm left wondering how many sites get wrongly listed . . . and why a site can be blacklisted on a single complaint without, apparently, any investigation. At least they make it easy to get off again.