Testing is one of the most important aspects of running a pay per click campaign. But which type of testing is more effective?
There are two types of testing that you need to be aware of: A/B testing and multivariate testing. A/B testing allows you to compare two versions of the same element side by side. One version of the tested element is seen by a percentage your visitors while the other version is seen by an equal percentage. The test is designed to give you an idea of which version of the element you are testing is more effective in getting the desired result.
Multivariate testing on the other hand allows you to test various elements at one time. By testing multiple variants simultaneously you can see which combination of elements work best on your landing pages. The question is, is multivariate testing more effective than A/B testing?
I think in some cases it can be, but for most advertisers, and for most advertising campaigns, I would say that A/B testing is preferable. A/B testing is always best for testing ad content, but when it comes to landing pages, multivariate testing can sometimes be best. I would not, however, run any multivariate tests until I narrowed down my options with A/B testing.
A/B testing is almost always preferable even on your landing pages. Multivariate testing should be reserved for picking the best of the best of your elements AFTER you have run your initial A/B tests. When you build your landing page, start your testing by starting with the topmost element first (for instance, your headline). Choose two of your best headlines and compare them with an A/B test. Work your way down your page this way until you’ve tested all of the elements you want to test (but only do one A/B test at a time). After gathering all the data on your A/B testing then you can run your multivariate test.
When it comes time to run your multivariate test, take all of your best elements, the ones that proved to be the best versions among your elements and perform a multivariate test on those elements. This time, you are mixing and matching elements and comparing pages. When you run this test, take all of your elements and rearrange them so that they appear differently on the page and run those two pages side by side in a multivariate fashion to see which pages gets the best results. For instance, your headline might appear the same on both pages but other elements like h2 and h3 tags, photo and image placements, font colors, background color, etc. will be different so that you can compare the entire package and see which elements work well together and which elements do not. You may have to run several multivariate tests to get the best results, but always start your testing with A/B testing and narrow things down from there.
Hey Brick Marketing,
I find it interesting that you say that people should always run AB tests and then run multivariate tests after finding out what works.
I’ve found that if you have healthy and active traffic (active meaning apropriately pre-targeted) that a multivariant test really works well for figuring out what works very quickly.
Would you agree with this, or do you think it’s still best to start with just an AB test?
Thanks for posting this overview explanation. The graphic is one of the more simple explanations I’ve seen of AB testing, and I think it helps people getting started with testing to understand it very easily.
Warmest,
Jonathan Kraft
Jonathan, great question. I’m not sure that “always” is the best way to put it. This is just one way. There are as many ways to test a landing page as there are landing pages. But I think for most landing pages you can serve the campaign better by starting with A/B tests. Otherwise, in your multivariate testing you’ll be testing elements that are inferior against inequitable variables. By running the A/B tests first, you narrow down your options then you can use the multivariate tests to prove which of the best options work together best.
In your question, you ask about healthy traffic. The above explanation is assuming that you are starting with a fresh new landing page, in which case you likely have not built up your traffic yet. If your landing page is aged and getting a healthy dose of traffic, as you say, then it might be prudent to run the multivariate test. I’d say that would be fine.