In a previous post, I talked about using the Multivariate Testing (MVT) feature, which has been redesigned and rebranded in April’s Shanghai release. Our statistics across our 1000s of customers tell us that, when comparing response rates between campaign using MVT and a reasonably comparable sample of campaigns not using MVT, those campaigns relying on MVT have a much higher open rate (49%) and click through (37%) rate. Imagine on a campaign sent to only 10,000 customers getting hundreds MORE responses.
I continue to wonder why A/B testing isn't used more systematically, at least when targeting new audiences and with new compaigns. In follow-up conversations I’ve learned that the most common reason that emarketers do not test campaigns is that they do not have the time to conceive of the alternative versions of the content. “I’ve got huge time pressures,” so the excuse goes, “I can’t spend up to double the time conceiving of and creating multiple emails.”
This is simply a misunderstanding of the feature. The new MVT workflow has been designed to make it incredibly easy to define just the derivative content for testing a subject line, the sending time or the sender.* There is no duplication of messages needed to test out those aspects. Just define the alternative content. In seconds you could set up a test to determine, for example, which of the following subject lines would be most effective:
- 40% off [Insert product name] now through Sunday
- Mr [Insert personalized last name], time is running out for the offer on [insert product name]
- [Insert Title & personalized last name], over 800 customers like you have already received their [Insert product name]
Many readers may say, “I already have a good idea which terms work best for my message subjects.” Most likely this is true.
But what if you’re wrong? How much money are you leaving on the table by relying on your gut feel rather that hard numbers?
There really is no excuse for not testing alternatives when the new MVT user experience makes it so easy to validate your assumptions.
*Granted, it is slightly more time consuming to test images, dynamic content and entire messages, the other testing options that MVT supports. But none of these necessarily leads to double the amount of creative work. And again, a few hours investment can yield significant increases in conversion.