Effective Testing and Optimization across Digital Marketing Channels

December 21, 2021 | 4 minute read
Doug Sundahl
Senior Director of Analytic & Strategic Services, Oracle Digital Experience Agency
Text Size 100%:

Consumer behaviors, desires, and needs are changing faster than ever. Simply watching your analytics and listening to your customer service feedback isn’t enough, if it ever was. To stay aligned with your customers across all of your digital channels, having a robust testing program in place is critical.

But how do you set up an effective testing program across your digital channels? How do you avoid making mistakes that undermine your testing? How do you determine the right kind of test to use and right parameters?

To answer those questions and more, Oracle Digital Experience Agency hosted a value-packed webinar, featuring JT Capps, Director of Analytic & Strategic Services, Oracle Digital Experience Agency; Doug Sundahl, Senior Director of Analytic & Strategic Services, Oracle Digital Experience Agency; and Chris De Marinis, Senior Consultant, Oracle Maxymiser.

Here are 5 key takeaways from the "Effective Testing and Optimization Across Digital Marketing Channels" on-demand webinar:

1. Small Gains Throughout the Email Funnel Can Lead to Big Gains Overall.

A/B testing program usually results in pretty small gains in performance in a particular stage of the email funnel. For example, testing a cleaner email design might boost your click-to-open rate by a few percentage points. However, when you rack up a bunch of small increases to different parts of the funnel they have a compounding effect.

JT Capps, Director of Analytic & Strategic Services at Oracle Digital Experience Agency, shared an example where increasing a campaign’s open rate by 2 percentage points, click-to-open rate by 2 percentage points, conversion rate by 0.25 percentage points, and average order value by $5 leads to a 67% increase in revenue for the campaign.

2. Do Both Campaign Optimization and Strategic Testing.

Campaign optimization entails testing on a particular campaign to boost its performance, whereas strategic testing seeks out long-term lessons that can be applied to a brand’s contact strategy or content strategy, for example.

Doug Sundahl, Senior Director of Analytic & Strategic Services at Oracle Digital Experience Agency, says that he often sees brands that are focused just on strategic testing to build up their learnings. But that’s a missed opportunity to pick up incremental gains in campaign performance. Make sure you’re doing both, he says.

3. Time of Day Can Significantly Impact Your Testing Results.

Many brands run morning tests to determine which email content to send out in the afternoon. A good example of that is a 10-10-80 test, where in the morning 10% of your audience is sent one version of email and another 10% is sent another version, and then a winner is determined hours later and that winning version is then sent to the remaining 80% of your active list.

“You’re going to roll out the wrong winner 10% to 20% of the time,” says Sundahl, because morning shoppers are sometimes very different than afternoon and evening shoppers. That’s particularly true for financial services and insurance customers, he says.

However, he stresses that brands shouldn’t get hung up on perfection when testing. “Continuing to test,” he says, “is more important than occasionally being wrong.”

4. Multivariate Testing Is Best for Determining Ideal Combinations.

Multivariate testing allows you to test multiple elements at once. For example, on a webpage, you could test two different sets of product copy, two different product images, and two different sets of CTA button copy. Because you’re testing more options at once, the time to results is significantly longer than a simple A/B test. 

However, it gives you insights about all the different combinations of your testing elements that you can’t get any other way, says Chris De Marinis, Senior Consultant for Oracle Maxymiser.

“If there’s a combination effect, we’re going to be able to see that in our data,” he says. “We’re going to be able to identify the best possible combinations.”

5. Don’t Expect Clear Testing Wins All the Time...or Even Most of the Time.

Even the best testing programs—those with great hypotheses and testing executions—are going to have lots of tests that are failures or inconclusive. It’s reasonable to expect that 10-20% of your tests will be clear wins, that another 10-20% will be clear losses, and that the rest will not result in meaningful differences.

“The goal is to have some wins and save yourself from making some mistakes,” says Sundahl. “And if you do that consistently, you’re going to drive results.”

If you want even more insights about effective testing and optimization strategies, watch "Effective Testing and Optimization Across Digital Marketing Channels" on-demand:


We hope you enjoyed “Effective Testing and Optimization Across Digital Marketing Channels.” For even more information, check out some of our other on-demand webinars, including:


Need help with your digital marketing campaigns? Oracle Digital Experience Agency has hundreds of marketing and communication experts ready to help Oracle customers create stronger connections with their customers, partners, and employees, even if they’re not using an Oracle platform as the foundation of that experience. Handling everything from creative and strategy to content planning and project management, we consistently exceed our clients’ expectations, earning a customer satisfaction rate of 96%.

For help overcoming your challenges or seizing your opportunities, talk to your Oracle account manager, visit us online, or email us at OracleAgency_US@Oracle.com.

Doug Sundahl

Senior Director of Analytic & Strategic Services, Oracle Digital Experience Agency

Doug Sundahl is Senior Director of Analytic & Strategic Services at Oracle Digital Experience Agency. His background in the email marketing space includes 20 years of experience with email service providers (ESPs) and digital agencies. His customer-centric approach to using data and personalization enables him to improve customer experiences and marketing effectiveness and create positive business outcomes.

Previous Post

Key ingredients of a great content marketing strategy for 2022

Lee Odden | 4 min read

Next Post

6 examples of lead nurturing campaigns

Autumn Black | 4 min read