An Introduction to Split Testing and Taguchi Testing

An important aspect of maintaining a successful Internet advertising campaign is being able to gauge your level of efficiency. A football coach makes constant adjustments to his tactics in response the dynamics of the game. With advertising, this dynamic also holds true. To be successful, to maximize your traffic and profits, you need to understand how well your advertising “game plan” is working out, and make adjustments accordingly. What is the best way to gauge your effectiveness? Through testing… and no, we're not taking about a pop quiz or a math exam. Welcome to the world of split testing and taguchi testing.

Testing in Internet marketing follows the same basic premise of the statistical testing used for scientific research and policy-making. In economics, statistical testing is used, for instance, to measure the effects certain policy-decisions (such as tax cuts) have on the overall economy. Being able to get a solid idea of how well certain elements of your marketing is working allows you to tailor a more successful future course of action.

If you wanted to gauge the effectiveness of an ad you currently have running, you need something with which to measure it against. Create a near duplicate of this ad (or its landing page), except with a single minor change. This seemingly subtle differentiation is important, because it represents what you are testing for. By changing the background color from white to, say, beige, and then tracking the resultant fluctuations in visitations/clicks/etc., you will get a good idea about which color is more effective. Similarly, you could change the color to something more drastic like green or red, and you could also alter something else about the ad, like the ordering of the text. The possibilities are endless, but it is important to remember that you should only change one element at a time – more than one change will negate the point of the test because it would become unclear what element of your ad/landing page is accounting for the increase/decrease in traffic.

Taguchi testing is like split testing, but involves subjecting more elements (or variables) of your ad/landing page to scrutiny. This is why it is also called multivariate testing. The benefits of taguchi over split testing are obvious; being able to assess the relative important of more than one variable in a given test saves time and may offer a more “holistic” picture of what you want to change. However, while split testing is relatively easy to conduct and free (Google Web Optimizer is a handy tool), taguchi testing is more involved and may involve the purchase of software.

On a last but important note: as you are tracking the results of your tests, be sure to have accumulated a decent number of visitations before you make any conclusions. In statistics, this number is called the sample size. Simply put, you might jump to a false conclusion about the relative effectiveness of a variable you are testing if that variable, and the one against which it is being tested, don’t have enough visitations. If you bought three apples and two of them happened to be rotten, that probably wouldn’t lead you to the conclusion that two-thirds of all the apples from that particular orchard are rotten. If you have three thousand apples and two thousand of them were rotten, you could make a more confident conclusion about the fraction of inedible fruits that orchard is producing. 

Mandarich Media Group uses
and recommends Affiliate Prophet
for split and Taguchi testing!