Comments on: Is A/B Or Multivariate Testing More Effective? http://payperclickjournal.com/ab-multivariate-testing/01/07/2009/ Thu, 18 Mar 2010 21:50:02 +0000 hourly 1 By: Brick Marketing http://payperclickjournal.com/ab-multivariate-testing/01/07/2009/comment-page-1/#comment-1983 Brick Marketing Tue, 20 Jan 2009 12:38:39 +0000 http://payperclickjournal.com/?p=995#comment-1983 Jonathan, great question. I'm not sure that "always" is the best way to put it. This is just one way. There are as many ways to test a landing page as there are landing pages. But I think for most landing pages you can serve the campaign better by starting with A/B tests. Otherwise, in your multivariate testing you'll be testing elements that are inferior against inequitable variables. By running the A/B tests first, you narrow down your options then you can use the multivariate tests to prove which of the best options work together best. In your question, you ask about healthy traffic. The above explanation is assuming that you are starting with a fresh new landing page, in which case you likely have not built up your traffic yet. If your landing page is aged and getting a healthy dose of traffic, as you say, then it might be prudent to run the multivariate test. I'd say that would be fine. Jonathan, great question. I’m not sure that “always” is the best way to put it. This is just one way. There are as many ways to test a landing page as there are landing pages. But I think for most landing pages you can serve the campaign better by starting with A/B tests. Otherwise, in your multivariate testing you’ll be testing elements that are inferior against inequitable variables. By running the A/B tests first, you narrow down your options then you can use the multivariate tests to prove which of the best options work together best.

In your question, you ask about healthy traffic. The above explanation is assuming that you are starting with a fresh new landing page, in which case you likely have not built up your traffic yet. If your landing page is aged and getting a healthy dose of traffic, as you say, then it might be prudent to run the multivariate test. I’d say that would be fine.

]]> By: Jonathan Kraft http://payperclickjournal.com/ab-multivariate-testing/01/07/2009/comment-page-1/#comment-1982 Jonathan Kraft Tue, 20 Jan 2009 06:48:40 +0000 http://payperclickjournal.com/?p=995#comment-1982 Hey Brick Marketing, I find it interesting that you say that people should always run AB tests and then run multivariate tests after finding out what works. I've found that if you have healthy and active traffic (active meaning apropriately pre-targeted) that a multivariant test really works well for figuring out what works very quickly. Would you agree with this, or do you think it's still best to start with just an AB test? Thanks for posting this overview explanation. The graphic is one of the more simple explanations I've seen of AB testing, and I think it helps people getting started with testing to understand it very easily. Warmest, Jonathan Kraft <a href="http://www.TheMagicOfTesting.com" title="The Magic and Power of Testing" rel="nofollow"> Hey Brick Marketing,

I find it interesting that you say that people should always run AB tests and then run multivariate tests after finding out what works.

I’ve found that if you have healthy and active traffic (active meaning apropriately pre-targeted) that a multivariant test really works well for figuring out what works very quickly.

Would you agree with this, or do you think it’s still best to start with just an AB test?

Thanks for posting this overview explanation. The graphic is one of the more simple explanations I’ve seen of AB testing, and I think it helps people getting started with testing to understand it very easily.

Warmest,

Jonathan Kraft

]]>