Machine learning tools like Persado and Jacquard were writing promotional email subject lines and other copy long before generative AI came along. But unlike genAI, which is primarily about saving time, machine learning-powered copywriting tools are all about increasing performance.
What words and phrases should I use to communicate most effectively? That is the question these tools have sought to answer by analyzing years-worth of campaigns and connecting higher performance with certain words and lower performance with others.
Who Are These ML Tools for?
These tools aren’t for every brand. A good candidate has a list of at least 1 million active subscribers. Having a large audience allows tests across more segments.
An ideal candidate also sends campaigns at least a few times a week. That’s because larger audiences and higher frequencies give these machine learning systems the scale they need to adequately understand which particular words and patterns of words resonate with your unique audience.
The less scale you have the slower the model is to learn and the less confident its recommendations will be.
What About the Results?
Unfortunately, even when Oracle Digital Experience Agency clients have had more than adequate scale, they’ve experienced mixed results. The two most common complaints we’ve heard are:
- The lift in email performance wasn’t sustainable beyond a few months
- The results in general didn’t justify the expense
If you’re considering these tools, we have some advice to get you on the path to success.
First, Optimize for the Correct Metric
One problem is that many brands using machine learning subject line writing tools are optimizing for opens. That can happen for a few reasons.
The most common one is that these ML-powered subject line tools often use opens as the default optimizing criteria. Marketers may not change that default or actually switch the optimizing criteria to opens because they think that more opens equal more clicks. This is a common misconception that leads brands to use vague, curiosity-peaking, and “clever” subject lines that are poorly aligned with the content of their emails. Those subject lines may boost opens, but then clicks fall off because the subject line enticed the wrong subscribers to open. (Making things even worse, these kinds of subject lines also tend to erode engagement long-term as subscribers feel tricked into wasting their time on content that isn’t relevant to them and therefore open fewer emails in the future.)
The truth is the goal of a promotional subject line is to get the subscribers who are most likely to convert to open the email. Given that, you want to optimize your subject lines to drive action as far down the funnel as possible. Ideally, that’s conversions. However, for most programs, reaching statistical significance on conversions takes too long, making it difficult to be nimble in their testing. Email clicks are an acceptable compromise that gets you to statistical significance much more quickly with a metric that correlates to conversions far better than opens do.
That brings us to the final reason that some brands optimize their subject lines for opens: It allows them to reach statistical significance much more quickly than if they were using clicks or conversions. That’s true, but being quickly certain about the influence of an unreliable indicator is not the path to consistently better performance.
Related post: 6 Ways that Subject Line Writing Has Changed
Second, Stay on Brand
Despite being trained on the historical performance of your subject lines, these ML subject line writing tools often don’t understand your brand, so they will suggest copy that’s off-brand—sometimes wildly off-brand. As the protector of your brand, it’s up to you to recognize these off-brand suggestions and reject them.
We’ve seen many instances where suggested subject lines:
- Include lots of shouty ALL CAPS
- Wildly overuse urgency as a tactic, undermining its usefulness
- Begin with “RE:” to give the false impression that the email is a reply (which is a violation of the CAN-SPAM Act)
These tools generally include the ability to establish from brand guidelines, including do-not-use lists of words. We highly recommend maximizing your use of these features.
At the same time, we recommend keeping a close eye on opportunities to use branded language that your AI isn’t likely to suggest, such as the names of your products, words from your jingle, and other words that have special brand meaning. So, it’s as much about leveraging your brand in your wording as avoiding words that go off brand.
Related post: AI-Generated Text: Generative AI Concerns & Opportunities for Marketers
And Third, Don’t Blindly Test
While all the major providers of ML subject line writing tools have accumulated enough real-world experience into their databases to be able to provide good suggestions, it won’t have any experience with your particular subscribers in the beginning. Building up that experience takes time.
Your vendor will likely encourage you to test at a high frequency—and across channels, if they offer that functionality. That’s fine. As we mentioned earlier, more tests equals faster learning.
However, even once your vendor does have significant experience with your audience, we still recommend always testing ML-created subject lines against a control subject line or two that your team creates. Human-generated subject lines, especially ones from experienced copywriters who are close to your brand, can outperform ML-generated ones a significant percentage of the time. Don’t assume the algorithm will always win.
Related post: A/B Testing Pitfalls: How Marketers Can Avoid Costly Mistakes
The Bottom Line
While machine learning tools for optimizing email send times and recommending products have been far more successful to date, we believe ML-powered copywriting tools have a bright future, especially as more brands leverage generative AI for copy. As these two AI tools merge, machine learning will bring increased performance to the time-savings generated by generative AI.
But this evolutionary convergence may take a while. In time, we expect more competition, lower infrastructure costs, and perhaps different pricing models will change the return on investment calculus that have caused some of our clients to walk away from this technology.
In the meantime, you can maximize your chances of success by:
- Optimizing your subject lines for clicks (or conversions) rather than opens
- Rejecting off-brand subject lines you wouldn’t accept from a human
- Confirming the performance of AI-suggested subject lines by testing against human-created control subject lines
Doing so will ensure that you stay in firm control of the experience you’re creating for your subscribers and that those experiences perform.
—————
Need help with your email optimization and copywriting? Oracle Digital Experience Agency has hundreds of marketing and communication experts ready to help Responsys, Eloqua, Unity, and other Oracle customers create stronger connections with their customers and employees—even if they’re not using an Oracle platform as the foundation of that experience. With an NPS of 66, our clients are thrilled with the award-winning work our creative, strategy, and other specialists do for them.
For help overcoming your challenges or seizing your opportunities, talk to your Oracle account manager, visit us online, or email us at OracleAgency_US@Oracle.com.
Now updated, this blog post was originally published on May 28, 2019 by Chad S. White, with contributions from Mark Sambor, Bradford Johnson, and Peter Briggs.