Split Testing Across an Entire Website
The most important tests study elements of a webpage that are likely to have a major impact on conversion. Marketing messages from HackerSafe (www.hackersafe.com) promise that the presence of its HackerSafe logo on a site increase conversion an average of 14%. The HackerSafe logo is an example of a “Risk Reversal” element, designed to reduce customer sense of risk and anxiety when completing a transaction.
Specialty retailer TinyRide.com sells a variety of baby strollers, car seats, and baby travel accessories. In mid-2006 they decided to try the HackerSafe Logo on their site, which devotes a large portion of screen real estate to risk reversal. The HackerSafe logo, in combination with the VeriSign Security logo, promised to be a formidable signal of safety to users. As with all new changes to the TinyRide site, however, the HackerSafe had to prove its value before earning a permanent space in the storefront. An A/B Test was designed and run using the Vertster site testing tool.
Because HackerSafe generally works best when it appears on every page of a website, we decided to split test the entire site, rather than a single landing page. This type of split test may seem complex, but the set up is pretty easy. The first requirement is the ability to dynamically insert content into a specific area on any webpage. Most testing tools allow you to do this with JavaScript or Dynamic HTML. By selecting a region in the site template, our test content now appears on every page of the site.
We created two alternative blocks of html to fit into this region. Each user was assigned to see either (1) a control version without the HackerSafe shield, or (2) a test version with the shield. An important consideration is to make sure that every time a visitor returns to the site, he continues to see the version he was initially shown.
Over two weeks, data on sales conversion and revenue per visitor (RPV) was collected and compared, and a point of statistical confidence was reached. According to our data, it is over 90% likely that the HackerSafe shield would continue to help site performance in the future (a 90%+ “Confidence Level”). In the final analysis, inclusion of the HackerSafe shield increased both the conversion rate and revenue per visitor by 15%.
Testing across your entrie website can determine the effect of various kinds of graphics, search boxes, subscription forms, etc. To do this you must be able to (1) change the element in a site-wide template and (2) define what constitutes a successful “conversion” so that you can monitor it.
Topics: Analytics, Articles, Books, Copywriting, Events, Landing Page Design, List Building, News, Split Testing |
May 26th, 2009 at 2:30 pm
test commment goes here