Like so many application providers, email inbox providers are integrating generative AI into their inbox apps, or allowing GenAI tools to work on top of them. One of the primary uses is to provide summaries of emails.
To better understand how these tools are summarizing email content today, Oracle Digital Experience Agency ran tests with Apple Intelligence and Android AI using emails with varying content and coding elements. Before we detail those test results, let’s talk about the devices where these tools are available.
The Availability of AI Summaries
Apple Intelligence and Android AI require processing power and RAM storage that’s currently only available on the newest smartphones. For instance, Apple Intelligence only works on iPhone 15 Pro, iPhone 15 Pro Max, and the iPhone 16 lineup. And Android AI works on the Google Pixel 8 and higher, as well as higher-end Samsung Galaxy phones, including the Galaxy S24 series, Z Fold 6, and Z Flip 6.
This means AI summaries for email aren’t widely available now and will reach a wider audience mostly through the sale of new Apple and Android phones. That will take time, since smartphone replacement cycles are around 3 years and the availability of AI isn’t as big of an upgrade driver as other standard phone features like storage and camera quality, according to CNET. Given that, AI summaries likely won’t impact the majority of your mobile email subscribers until sometime in 2026.
It’s worth noting that in addition to those on-device generative AI engines, some older Android phones have access to AI summaries via a web-based Gemini engine. While Apple is highly unlikely to follow suit due to their privacy positioning, Google’s move does make AI summaries more widely available, but only for selective use after an email is opened. Let’s talk more about why these post-open AI summaries are likely to have a much smaller effect on marketers than pre-open summaries.
Related post: Understand the other Apple iOS 18 features that will impact marketers
The Two Kinds of AI Summaries in Email
The two places where inbox providers are using AI summaries are in the inbox view pre-open and in the email itself post-open.
Pre-open, AI summaries are replacing preview text. For instance, in Apple Mail, where AI summaries are on by default, summaries appear under the subject line. They generally take up two lines (up to about 40 characters), but sometimes only occupy one line, as you can see in the example below.
This is a huge loss for email marketers, who have become great at using visible or invisible preheader text to control what displays as preview text. In particular, it makes it riskier to use jokey, clever, vague, and mysterious subject lines that rely on their preview text partner to deliver a punchline or clarify the content of the email.
Moreover, while we had good experiences with inbox AI summaries in our limited testing, some outlets have asserted that overall quality is mixed and shared examples of awkward and confusing summaries.
Post-open, AI summaries of an email are available with the click of a button. In Apple Mail, these summaries are generally several lines long and push more of the email’s content below the fold. Android AI provides two options—a Summarize this email option that provides about three lines of synopsis, and a Longer Summary option that provides a handful of bullet points. In both cases, the summaries appear over the email content in a modal. (See examples below.)
Because these summaries do not appear by default, they’re likely to be used sparingly and therefore have much less of an impact on marketers. Since most newsletter and promotional email copy is already highly condensed into teaser-style copy, subscribers likely won’t see much time-savings from these summaries. Plus, the summaries inherently lack the images that are central to how consumers scan most promotional emails today.
How Do Email Design Choices Impact AI Summaries?
Oracle Digital Experience Agency did extensive testing to see how AI summaries were impacted by different email coding and content, including semantic and non-semantic live text, and image-based emails with and without alt text.
Graphical text
Designing your emails as one big image has been unwise for well over a decade, as using ample HTML text is good for email accessibility, for when images are blocked and subscribers are in low-bandwidth environments, and, to a lesser extent, for email deliverability. AI summaries complicate this issue, as both Apple Intelligence and Android AI fully ignore any graphical text that’s included in your emails.
A lack of sufficient HTML text can result in subscribers getting a “too short to summarize” message from Apple Intelligence. When denied enough HTML text, Android AI will include bullet points in its longer summaries that often reiterate who the sender is, pull in points from the footer text, and point out that there is contact information and an unsubscribe link. These summaries are thoroughly uncompelling.
Related post: Hot digital marketing redesign trends
Alt text
Further complicating an image-heavy design approach, Apple Intelligence also completely ignores any alt text you’ve added to your images. Android AI ignores it as well, unless there’s little HTML text. In those instances, it ingests alt text.
Related report: Accessibility & Inclusive Design Ideas checklist (free, no-form download).
Preheader text
Android AI ingests preheader text just like other body text. However, Apple Intelligence appears to recognize preheader text as being different from the rest of the body text. In some instances, Apple Intelligence relies heavily on the preheader text, doing a basic rewrite of it. But other times, it completely ignores the preheader text. In our testing, this behavior was inconsistent, not following any clear pattern.
Subject lines
You’ll obviously want to have descriptive and helpful subject lines regardless, but Android AI ingests subject lines, while Apple Intelligence ignores them.
Content hierarchy
On both iPhone and Android, AI summaries weren’t impacted by semantic text like heading levels or by font size. However, HTML text higher in the message was given priority, especially in the short pre-open summaries. The longer post-open summaries were less impacted by content position. In these cases, summaries included content from throughout the message.
Related post: Understand the anatomy of an email message
AI Summary Consistency
Both Apple Intelligence and Android AI feature light-weight on-device generative AI engines, which safeguards user privacy by keeping all data on the user’s device. Given this decentralized structure, we expected considerable variation between AI summaries on different devices, but that’s not what we saw.
For Apple Intelligence, email summaries were incredibly consistent among all of the people participating in our study. It’s possible this consistency was due to Apple Intelligence being newly installed on all of our participants’ devices, and that many of the devices themselves were new.
That said, we saw minor variations in the summaries produced by Android AI. Word choices varied slightly and the order of items in longer summary lists varied. Still, the level of consistency was also high.
Our Recommendations
While the near-term impact of AI summaries for email will be negligible due to the limited number of consumers with access to the latest smartphones, they will have an increasing impact over the next 3 years as phone replacements occur. During this time, we expect on-device generative AI engines to improve, creating better summaries.
Even with improvements, we think few subscribers will choose to use post-open AI summaries on promotional emails, given how central visual content is to the value of these campaigns. On the other hand, pre-open AI summaries will have a much bigger impact, since they’re on by default and are replacing existing text content. While perhaps as many as half of subscribers will turn off these summaries, the rest will keep them on.
Along with schema-based Automatic Extraction, pre-open AI summaries are part of a growing trend of inbox providers claiming preview text as their own and removing it from the control of senders.
Given all of that, we recommend over time that brands:
All generative AI engines are evolving, so it will be important to pay attention to further developments and future releases. Testing will also be critical in measuring impact, but be mindful to measure tactics over many campaigns to understand the cumulative effects.
—————
Need help with your email marketing campaigns? Oracle Digital Experience Agency has hundreds of marketing and communication experts ready to help Responsys, Eloqua, Unity, and other Oracle customers create stronger connections with their customers and employees—even if they’re not using an Oracle platform as the foundation of that experience. With a 94% satisfaction rate, our clients are thrilled with the award-winning work our creative, strategy, and other specialists do for them, giving us an outstanding NPS of 82.
For help overcoming your challenges or seizing your opportunities, talk to your Oracle account manager, visit us online, or email us at OracleAgency_US@Oracle.com.
To stay up to date on customer experience best practices and news, subscribe to Oracle Digital Experience Agency’s award-winning, twice-monthly newsletter. View archive and subscribe →
Kaiti (Livermore) Gary is a Senior Director on the Analytic & Strategic Services team at Oracle Digital Experience Agency. Her background includes over 16 years of client and agency consulting experience in the in a variety of marketing capacities including product management, customer experience and digital marketing. Given her diverse background, she excels in the development of holistic and innovative marketing solutions that balance strategy, technology and operational needs.
Sarah Gallardo is a Lead Email Developer for Creative Services at Oracle Digital Experience Agency and an email accessibility specialist. With a career spanning over 12 years, she boasts extensive experience in a wide variety of email development solutions. Her advocacy for accessible emails spans more than six years, demonstrating her commitment to inclusivity in digital communication.
Next Post