When a great click through rate isn’t really that great

When a great click through rate isn’t really that great

new york post cover foot long 300x225 When a great click through rate isnt really that greatEarlier this year, Matt Corby, a Perth teenager, posted a picture on Facebook showing that their famed 12″ foot long sub wasn’t actually 12″ in length but was in fact 11″ long (at least at the 17 stores he visited). The subsequent uproar that occurred struck me as an object lesson in measurement and how its important to report on metrics properly – particularly those metrics whose definition is generally commonly  ‘understood’.

I saw first hand earlier this year how this can lead to problems when I observed a client reporting the % click through rate for a campaign as:

Unique clicks / number of emails opened *100

as opposed to the IAB defined

Unique clicks / number of emails delivered *100

While this might not cause an issue if appropriately (and explicitly) defined, in this instance the stats were simply passed off as standard ‘% click through rates.

This mislabeling / miscalculating of the click through rate had a stunning effect on the email % click through results for the client:

Emails
sent
Emails
delivered
Opens Unique
clicks
% Click thru rate
IAB defined
% Click thru rate
(Client defined)
110 100 30 15 15/100*100 = 15% 15/30*100 = 50%

While this might not be an issue in and of itself (provided the results were reported on a consistent basis and in a vacuum), in this instance this was not the case. The US division was reporting the results in this rather odd way while the European division was reporting campaign results according to IAB best practice. I think you can see where this is leading. When the results from each division were placed side-by-side the US arm appeared staggeringly more effective than the European arm… I did flag this to the relevant team leaders but the last I heard each arm was continuing to report their respective stats in the same way.

My advice to clients is as follows. If you see something that appears too good to be true, it usually is. If in doubt check the methodology being used and above all make sure that you are comparing apples to apples and oranges to oranges.

No related posts.