In “A/B Testing – You’re doing it Wrong” Justin Baker brings up a dynamic that we’ve been struggling with for some time now. His thesis is that most A/B testing focuses on fine-tuning things like button placement, CTAs, and banner images. While in many cases those factors won’t get to the root of why your landing page isn’t converting. According to Justin “A/B testing for many companies becomes a band-aid for poor value propositions.”
Our clients frequently request A/B testing. But in many cases, the time and expense required to do proper layout testing doesn’t represent good ROI for some clients. Why do clients often want A/B testing so badly? Often it’s less about optimization and more about having metrics to backstop decisions. Tell me you haven’t heard “Yeah I know we missed our objectives, but we tested it and that was the best performing page.”
Use the Power of A/B Testing for the Forces of Good
Justin (and we are in total agreement) isn’t against A/B testing. He is simply promoting a more “conscious” kind of A/B testing. Justin says “sustainable growth does not result from changing a button from red to blue, it comes from building a product that people want to use.”
For example, if your site is generating tons of traffic, but your conversions are poor try testing with a different audience. You could also try testing a streamlined checkout process. Or even test a new discounting or pricing approach. Try changing something that will have an actual impact on client experience. Don’t focus on trying to wrangle clicks through cosmetic changes.
As a web agency, it is sometimes hard to resist the urge to test everything. It’s profitable. Also, those metrics make all our decisions like the result of Vulcan logic. But when we look at it through the lens of trying to be a good partner and providing solid value the picture changes. Using that lens we see A/B testing as a tool. Useful like any good tool should be, but not a panacea. Because of the expense and time required to do effective testing it is a tool that should be used to unpick complicated issues, not to make up for poor UI or graphic design.
You can read Justin’s article here.
