X

Welcome to the Oracle Modern Marketing Blog:
The latest in marketing strategy, technology, and innovation.

Building Segments Through A/Bn Testing

Background: Mercury Rising

A prominent and highly graded auto insurance company, Mercury Insurance Group approached Oracle Maxymiser to help with product packaging and quote funnel personalization. Its most common policy coverage, called “basic,” included liability, collision, and medical insurance, but Mercury wanted to learn if different packages would resonate better with different user profiles.

Mercury gave site visitors four package options: basic, enhanced, premium, and custom. Generally speaking, from basic to premium, a package became more expensive but provided more coverage. The custom package enabled users to build their own plan, in case they had an uncommon combination of requirements.

Challenge: First Isn’t Always Best

At first glance it seemed Mercury had a great personalization plan. Mercury gleaned basic information about all of its users through their IP address, which revealed device type, browser, and geographic location. When users requested a quote, they were taken to Mercury’s quote funnel; here, they filled out fields that revealed even more about them, including the make and model of their vehicle, their vehicle’s age, and if they’d ever been in an accident.

Its most common policy coverage, called “basic,” included liability, collision, and medical insurance, but Mercury wanted to learn if different packages would resonate better with different user profiles.

This last step, however, was where Mercury understandably wanted to get better. It didn’t use the data it obtained in the quote funnel to give users the best package for their needs and goals. Instead, no matter how users answered quote funnel questions, they were automatically shown the basic insurance package. While the three other options—enhanced, premium, and custom—were there as clickable links or tabs, the first option shown by default was ‘basic.’

This could have been impacting how people converted on the other three insurance types: As a recent study by UC Berkeley found, the sequence in which options are presented is “a driving force behind our decisions” as customers (Berkeley Haas).

Solution: Easy As A/Bn

Methodology

Oracle Maxymiser and Mercury worked together to build a test campaign that identified not only individual segments but also the insurance package that spurred the most conversions for each one. We did this by tweaking the end of the quote funnel: Instead of showing everyone the basic package by default at the purchase point, we now randomly showed one of the four package types. This was an A/Bn test.

In A/Bn testing, three or more versions of an experience are compared (theoretically an infinite amount can be compared, hence the n.) As the name implies, it’s an extension of A/B testing, in which only two versions of an experience are compared. In Mercury’s case there were four versions total: one for each package someone could be shown at the end of the funnel. Through this funnel, users still had to fill in the usual fields about themselves, providing information on their vehicles, accident history, and so on.

In A/Bn testing, three or more versions of an experience are compared (theoretically an infinite amount can be compared, hence the n.) As the name implies, it’s an extension of A/B testing.

From this, clear patterns in conversion emerged! We could connect the data that users entered in the funnel with how often users purchased a certain kind of insurance. For example, if people who had never been in an accident were less likely to pay more for better coverage, this test found it and spotlighted the trend. We presented patterns like this to Mercury with our post-campaign reporting tool, Campaign Insights.

Learnings

The A/Bn test justified Mercury’s instinct to promote its basic package; this option converted the most overall, so if a package did have to be shown by default, basic was the right choice. But more importantly, Campaign Insights found links between user traits and the likelihood users bought a certain package.

For example, two major learnings concerned the custom insurance package, in which users could build their own plan. We discovered that people who had recently been in a car accident converted most on this option. Also, users preferred this option if they lived in Fort Lauderdale, Florida, the location of a major Mercury office. (Florida is one of the 13 states in which Mercury writes auto insurance.)

Application

With tons of new data at its fingertips, Mercury could personalize its quote funnel! Now the company could provide each individual user with the package most likely to convert, delivering a non-random experience based specifically on how he or she had filled in the funnel fields.

This isn’t to say Mercury is finished testing—far from it. Optimization testing should be continuous, as customer segments will evolve as market factors change. Through A/Bn testing, however, Oracle Maxymiser was able to help Mercury find its footing, identify user segments, and start its personalization journey.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.