What Are A/B Tests?
A/B tests or split testing – is the research method where you compare the baseline data with the test one.
For example, having the search form for the last-minute offers (option A) you suppose the form’s loud color should increase your sales. To clarify the more efficient variant, you should practice both of them and then make a comparison on the basis of statistically significant traffic. That’s A/B tests.
Two options are tested, as a rule. However, no one can stop you from testing several assumptions, for example, to check a flight ticket search form in various colors. You check two options, and then you test the winner plus one extra option.
Note that there’re no gold standards in marketing. If you ever see or hear that one variant works for somebody, it could be nothing for you and your target audience. It’s better to rely on your own test results but not on theoretical information, that generates income for others.
Why You Should Do Split Tests?
Now you make $5 from 1000 visitors, but you probably can get $10 or even more. To find out the most profitable strategy you should perform tests.
It’s necessary to determine the purpose before testing. Suppose your aim is to convert more leads into flight tickets sales. For this purpose, you try to place a calendar widget instead of the traditional search form. To find the best option you must measure the results.
Not all tests deal with an increase in income. You can test the arrangement of the affiliate tools and watch the changes in such behavioral factors as length of time spent at the site, the percentage of “reads up”. Ultimately, it can influence the number of visitors that come back to your project within a month.
Regardless of your objectives within A/B tests, you should define the goal before you start.
What Should You Test on the Page?
You check your assumptions and forecasts in order to see how it could be better than it is now. That’s why you make the following changes in your project:
- Website design
- Page setting
- The design of forms, widgets, banners and other elements.
- The images of goods, as well as pictures and photos in the articles.
- Text: titles and headings, product description, calls-to-action on the buttons, the texts of special offers, advertising actions and other marketing things. It’s important to test all the text parameters – content, the number of words, symbols or lines, typeface, and the arrangement of textual blocks.
- Functionality.
All of these are great bases for the assumptions, changes, and tests. You could check the changes in the visitors’ behavior when you remove the sidebar. You could add the bypass to the AdSense advertising blocks to see what you come up with, you can replace AdSense blocks with some alternatives, etc.
It’s possible to perform global tests, for example, to compare the target audience’s response to the domain name in the national (.kz) or international (.com) zones for the contextual advertising. This’s relevant for example, for the Kazakhstan market where we paod double commission until the end of 2018. The tests require time and money but allow the best variant to be revealed.
You can test the following tools and programs within Travelpayouts:
- Affiliate programs within the same niche. For example, at Travelpayouts you can find different event tickets’ affiliate programs. Try each of them and find out the affiliate that fits you best.
- Affiliate programs in diverse niches. Find your audience’s preferences for car or bus rental while they are traveling around Europe.
- Affiliate tools. For example, you can use various formats of widgets. Don’t create the tests on the advertising formats alone but work with its placement and surroundings. The interesting fact is that the author of the project aviabiletebi.org Macho Kevlishvili, redesigned the search form more than 10 times before he found the best variant. Learn more about his experience from our interview.
Do experiments outside Travelpayouts too. For example, you can test where the customers are more willing to book hotels – on Hotellook or Hotelscombined and what hotel affiliate program provides more revenue.
How to Perform A/B Tests: 10 Steps
The systematic approach is very important in A/B tests. Before you initiate the changes on the page, you should set the goals you are going to reach. Below you will find the guide on how to run a split test and several useful softwares.
How to perform a simple test of the page – the step-by-step plan:
- Check the pages of your website where people spend the shortest time or close them at once. If you use Google Analytics, take data from there.
- Watch the users’ behavior via Webvisor from Yandex or other similar software. The users might not see the buttons.
- Plan changes on the basis of analytics. Make conclusions and assumptions.
- Make a plan for testing the changes.
- Make changes.
- Compare only two versions. Multi-alternative tests with the comparison of 3 versions aren’t able to generate the qualitative results. Test only one parameter at a time. If you change the hotel widget color and place simultaneously you can’t understand the reasons for the increase in the conversions.
- Make the simultaneous comparison of both versions via software that we’ve described. If you place one banner on the site and then switch to another one in a week, the results won’t be completely correct because of the traffic diversity due to various fests, sellouts, and other events that influence the sales.
- Determine the optimal percentage of the sample. The baseline would lead for the first days of the test but then the test version would win. Use the sample size calculator to clarify the time of analysis of the results.
- Test in stages in the case of three or more variants in the assumptions.
- Choose the best variant and work with it.
Tools for Running Website A/B Tests
Here you’ll find out 4 services for split tests to use without programming skills.
Manual Tests
You can test your website even without special software. Try to place the diverse banners, forms, and widgets and to arrange them in different ways on the site. Watch the conversion in the Travelpayouts’ dashboard.
The shortcoming is that you cannot test two versions of the same page simultaneously in order to show the variant A to half of the target audience and to display the variant B to the other half of it, which is more preferable and rational.
However, it’s better to try manual tests than to sit idly by in the case of not having access to the special software and tools. Though this method isn’t complete because of its disability of making the objective and accurate comparison, it allows making some useful conclusions.
1. Google Analytics Content Experiments
The tool Content Experiments for A/B tests is hidden inside the well-known Google Analytics. Choose the menu item “Behavior” – “Experiments”:
The guide on how to use Google Analytics Content Experiments read in the official guide or in the detailed instruction in CreateandCode’s blog.
2. Ab Tasty
AB Tasty is the easy and functional service for A/B testing of the various levels. Pros:
- Flexible: you can create any testing scripts.
- Accurate: you can target the test performing to the definite segment of the audience.
- Informative: you get an in-depth report on events.
The special plugin is available for WordPress websites owners that simplifies the tests. Learn more from the promo-video: VIDEO
The only services issue is its cost – it’s rather expensive. The most affordable plan is 29 euro a month that’s available for annual subscription. Moreover, cheap traffic maybe wouldn’t be enough to go around because only 5 000 visitors are included.
3. Optimizely
Optimizely is a professional product for complete researches and great experiments.
The service allows processing various test scripts and stays very simple. The most complicated thing you’re expected to do is to fill the line of code in the element <head> on the webpage. Then you can change the buttons’ colors, texts, and images on the test page via the convenient control panel. The procedure is similar to the page editing in a photo editor – you just move the elements with your mouse or change their color with one click.
Optimizely can segment the site’s audience on the browsers they use, the source of the traffic, ad campaigns, type of device, etc. You get the detailed reports with the results of the tests.
Download the free Optimizely plugin for WordPress based blogs to perform A/B tests of any elements, including headings.
The cost of the add-in is individual with a view to the task. To buy the subscription you should put your bid in and await for the manager’s reply.
4. Clickmeter.com
With the aid of the Clickmeter service, it’s possible to divide affiliate traffic in any correlation and direct it to the different pages. You just fill the links in, and Clickmeter does the rest. The service specializes in traffic sharing and detailed analysis of link redirection.
Compare the number of purchases on Level. Travel and Online tours, for example.
A/B tests of various pages isn’t Clickmeter’s priority task. It’s designed to compare the diverse ad campaigns’ efficiency. It helps to compare the affiliate programs from the same niche via traffic sharing in the desired relation to each other.
You need the additional domain to use all its options. Clickmeter doesn’t provide domains for traffic sharing.
What to Do With the Split Tests’ Results?
Capture
This is dull but necessary. If you’re bothered to record the results once, you’ll definitely forget them and the test will get away with it.
While using software for A/B tests, the results are saved automatically. It’s convenient but doesn’t solve all the problems. In any case, when you capture some data, you must sync its components up to each other.
For example, one service displays data on your project’s visitors and the number of clicks but you’ve to export the financial disclosure from AdSense, Travelpayouts, and other sources to sync the traffic and revenue data.
Determine the Statistical Significance
The statistical significance displays the probability of the same A/B test result’s appearance when the page is compared with itself. If this possibility is strong, the results of the test are sporadic and framed.
The calculator for statistical significance also exists.
After you’ve revealed the statistical significance, you should choose and implement the winner or leave as it stood and think about the possible error in the test or the irrelevance of the changes. To figure out the new test is the next step. Try, be wrong, learn, and try again – only such an approach ultimately leads to the results.
Use
Choose the variant that has benefited from the statistically significant test. For example, if you have revealed the higher conversion of the search form with the Avia sales logo than the conversion of the unmarked one, use the first variant.
Sometimes the initial A is better than test variant B. This is normal practice. You have tested both variants and found out the best one. Leave the initial version in this case and think about the possible improvements. Then it’s time to perform the next A/B test.
Share with us, in the comments, your split test experience and your ideas for running A/B tests on the travel websites.