Testing Your Assumptions: Conversion Rate Optimization Strategies
The major challenge in testing the assumptions of conversion rate optimization is in determining what to test, in what order to test them and how many things to test at each instance.
The ideas we discuss in this post are gleaned from the CXL Institute course on Conversion Rate Optimization, and I would very well recommend it for serious marketing professionals.
The natural order is to identify what problems users are having challenges with, come up with the urgent ones and start testing solutions to those problems.
Starting from the obvious problems, we could get some results but most of the tines that would not be enough to raise the conversion satisfactorily so we need to get more creative with the solutions.
If you’re trying to increase the sign up rate, and have not discovered any problems with the process, we may want to start with creating alternative means of signing up.
Use social sign ups like Facebook, twitter, google, apple ID, Instagram or even reducing the number of steps in the sign up process to keep it as simple as one button but not with the sentiments of social sign ups.
Understanding the metric you want to test and how that improves your process is important to knowing what creative process to employ.
If you only depend on data you will not have enough data evidence to justify social logins or sign ups so sometimes it doesn’t need to be validated with previous data. A note of warning should however be sounded here — if you only go with your gut feelings you will most probably make big mistakes.
Sometimes toy would hit a local maximum and still not having enough uplifts to justify the work done. In this case you will need to completely re-imagine what the whole process will look like supposing you strip off all the process present now. This could pose a great risk but if carefully done has a good shot at success.
In re-imagining your strategy, you need to target specific problems instead of being solution specific. Don’t be attached to a particularly solution instead of looking at the problems to be solved.
In determining how many changes you need to make at a time, there are a few factors you would want to consider.
A . How much traffic do you have: For high traffic websites that sees over about five hundred thousand or more visitors a month, you may be able to test very few changes like 5 changes and the conversion would change noticeably from what it used to be but if your website traffic is on the lower sides of about hundred thousand a month or less, you may need to make many changes for it to accumulate into a noticeable change.
B. How scientific do you want to be: If it is important that you can pinpoint exactly what affected the change in conversion rate, you may want to make fewer changes and run the experiment for a longer time. This will help bring in more data for analysis even with fewer changes.
You are to find the balance for your project on the scale of getting more result quickly and being able to tell a coherent story with the result.
A/B test VS MVT
A/B test is ideal for testing dramatic changes of a layout while Multivariate test is ideal for testing the effect of some interactive elements.
Multi-variant requires much more traffic to be able to detect the effect of different elements at a time. If you have less than 100,000 visitors a month, you may want to just stick to A/B testing.
For instance, if you have three versions of your copy, image and buttons, there are already so many variations and more data and traffic will be needed for each segment to be able to reliably interpret the results. In cases where you have less traffic you may have too many false positives in your result.
Bandit Testing
In bandit testing, the traffic allocated to each variation is not specific, the traffic change with respect to the performance of each version of the change. If version B is performing better, the Bandit algorithm shows the version to more users than the A. the idea is that you earn while you learn. You are maximizing the amount of money you are making per minute. So instead of having to waste traffic on a non performing version of the site, the algorithm automatically optimizes to make the best of the traffic.
This is great for seasonal short-term campaigns. This is a good way to maximize the amount of money we make during the seasonal campaigns like Mother’s Day Week promo Sale. This type of testing is best for cases where there are fewer hands to manually optimize things. It adjusts by itself and makes the best of the promotional sales season. It is good for automation foe scale type of case.
With bandit testing, learning is minimized because the test is adjusted from time to time by the algorithm to get the best financial result of the campaign.
Existence testing:
There could be cases where it is hard to decide what will be on the home page maybe because of internal politics in the company where several teams want their message on the home page.
For this case we can employ the existence testing which is to simply remove women sections or elements on the page and see if or how it affects sales or conversion rate. In cases where there is no difference or hurts the conversion rate, it should be kept out of the page.
Iterative Testing
When we take a page or a set of pages and make subtle changes to test a wide range of ideas. In this system, you can attribute result to very specific changes. You take out a small part of a paragraph or a picture adjusted and measure the effect it had on the website. This could also mean changing a bunch od things at the same time but on each occasion, you are able to attribute the change to a result on the page. People use iterative testing because its allows the audience to gradually adapt to the changes, they are not thrown off balance with dramatic changes.
It can be a cheap way of testing things as it mostly don’t cost the company huge amounts to change s dingle thing or two on the website.
It allows the organization to gradually cultivate the culture of testing things. People could have team meetings on what to test and the results from previous little changes.
This could also be a good way to test the assumptions of the leadership and get their by-in on other more dramatic changes.
Innovative Testing
This is a more creative way of testing assumptions when the iterative testing doesn’t prove very profitable.
This is better for small traffic sites where larger changes has more chances of making significant impact on the conversion rate than the subtle little adjustments. Innovative testing can lead to significant change in behavior and conversion rate.
Split Path testing
This is a system of testing where we are taking the user through two different conversion paths. The case of one-button purchase VS multiple step-purchase on amazon is a typical one. This helps you measure the effectiveness of each conversion path, where customers were dropping off faster and the more profitable steps on each path.
Conclusion
These testing styles could be combines in some cases for optimum results. The parameter like length of time for testing, segmentation, traffic size and many other variables play roles in choosing which testing style is viable for your case.
The knowledge shared on this post is from the CXL Institute mini degree course on Conversion Optimization.
