Many digital marketers who use A/B and multivariate testing as the cornerstone of their optimizations strategy barely scratch the surface when it comes to the data they're capable of collecting. The strategies offered below will enhance the quality of your metrics, your learnings, and therefore the possible positive outcomes of any test.
Here are three ways to improve the quality of your test results:
1. Identify a Sensible Primary Metric
It is integral to identify a primary metric prior to starting a test. The metric will serve as the key piece for determining if the test is successful. The chosen metric should be directly impacted by the changes being implemented on the page as part of the test. To meet these criteria, ensure that the action is in close ‘proximity’ to the test page.
For example, if you are changing content on the first page of a four-page credit card application, it can be tempting to identify the primary metric as completed applications. However, it is more logical that the content on page one would have a bigger impact on whether or not a user moves on to page two, not on whether they complete the application as a whole.
There are a number of non-test related reasons why a user might drop out of the application between pages two and four. Therefore, using application completions as the primary indicator of success can be misleading, while measuring the number of users who reach page two is more directly related to the test being run on page one.
Overall, identifying a sensible primary metric will help you keep the data focused and organized.
2. Utilize Action Attributes Where Possible
Action attributes allow for an increased level of granularity when analyzing test data. Hypothetically, you might have a page with multiple products listed with “Buy Now” buttons next to each product. You have three different options in this scenario:
Option C is ideal because having one action will keep all of the data manageable, while the attribution allows you to filter campaign data by product to see the number of clicks associated with each product.
3. Segmentation & Personalization Criteria
While action attributes make granular filtering possible on the metric level, segmentation and personalization attributes make it possible to filter on the user level.
For example, if you are curious to see if users who arrive to the page via different channels behave differently, you can create custom segments for users who fall into each traffic source and then look at the data exclusively for users who fall into that segment. Segments can be created based on any identifiable user attribute whether it be age, device, geo-location, etc. These capabilities are often underutilized, leaving valuable, high-quality data untapped.
In the end, there are many steps a user can take to ensure the quality of their test results. Today we’ve covered three ways we ensure that the results of our clients' optimization tests are as good as possible. A lot goes into making a test successful, from set up to metrics, so ensuring you have best practices in place before embarking on any test is vital.