AI is revolutionising the way companies operate, and Oracle is leading the charge by integrating AI into the very fabric of its software products. By bringing AI and data closer together, we’re unlocking new possibilities across the board. Fusion is already packed with hundreds of embedded AI features that have the potential to enhance productivity and efficiency across its various pillars. But with all the hype around AI, it’s easy to assume it’s a silver bullet. We need more than just speculation and anecdotal evidence to prove its value.
Oracle’s Customer Success Services has been analysing Fusion usage data, allowing us to move beyond assumptions and truly understand how AI is impacting our customers through a rigorous quantitative approach. In particular, we have defined several metrics such as:
- AI Usage Rate: What proportion of tasks used the corresponding AI Assist functionality?
- Task Completion Rate: What proportion of tasks were seen through to completion, with and without AI usage? For example, did the user actually send that AI-generated email to their supplier base, or did they cancel it?
- Efficiency Lift: Does AI usage lead to faster, more efficient task completion?
These metrics are rendered in an Oracle Analytics Cloud dashboard that visualises this data, enabling data-driven decisions to enhance our AI offerings.
Telemetry
Fusion leverages the OpenTelemetry framework to capture information about page visits and button clicks, including which buttons are clicked and a timestamp of each interaction, without capturing sensitive information such as user-generated content or AI inputs/outputs. These individual interactions are emitted as signals. Telemetry is the process of collecting and analyzing these signals to provide insight into how the user interacts with Fusion.
Our analysis of this usage/telemetry data is built on the assumption of a linear user workflow for each AI-assisted feature, allowing us to track interactions from start to finish. Take, for instance, the feature in SCM that allows users to create an email message to suppliers and buyers; AI assistance for this feature was added in Fusion 25A. Users start the email writing process by clicking the “Create Message” button, and are then given the option to use AI assistance. The task ends when the user clicks “Send” or “Cancel”. This linear workflow is illustrated in the demo screenshots below.
We measure task performance by tracking user interactions and aggregating them by session ID. This allows us to calculate:
- Total tasks performed
- Percentage of tasks that used AI assistance
- Completion rates (with/without AI)
- Average task duration from start to finish
These quantities are then used to calculate various AI metrics, some of which we discuss in more detail below.
Task Completion Rate
AI has the potential to help users complete more tasks by reducing friction and cognitive load. For example, AI can simplify tasks like filling out forms by clarifying instructions or auto-filling fields, making the process less overwhelming. Our Fusion telemetry data highlights this potential of AI to improve task completion rates.
Let’s consider the task of employee goal creation in HCM, with AI support added in Fusion 24A. Rather than typing out goal details manually in each text box, the AI Assist feature generates goal details based on the goal name you enter, your business title and department, thereby streamlining the process. The plot below shows the AI-Assisted vs Unassisted Completion Rate from 1 July to 31 December 2025 across all production environments reporting telemetry data. The AI-Assisted Completion Rate (%) is defined as (Total AI-Assisted Tasks Completed / Total AI-Assisted Tasks) × 100, and analogously for the Unassisted (aka. Non-AI) Completion Rate.
Tens of thousands of goals are created each day, with the unassisted task Completion Rate averaging 54%. In comparison, thousands are created each day by our customers using AI assistance, with a consistently higher average Completion Rate of 78%. This data can be promising since it suggests that, by leveraging AI to reduce friction and streamline the process of goal creation, it is more likely that users will see the task through to a successful completion.
Efficiency Lift
AI models are generating excitement for their efficiency-boosting potential. Our Efficiency Lift metric (the percentage increase in efficiency when using AI, compared to unassisted work) highlights the significant benefits businesses could reap by saving time on tasks.
Let’s consider the telemetry data for the Job and Position Profile Authoring Assistance feature, released in Fusion 24A. Without AI assistance, the average time to create a job profile stands at 2 minutes, averaged over ~12,000 unassisted tasks in production environments. Users must manually write a job/profile description, describe the job’s responsibilities, list expected qualifications, and so on. In contrast, by simply entering a job name and clicking the AI Assist button, the AI can generate content for all these fields on behalf of the user, thereby yielding significant time savings. This is reflected in the telemetry data, with the average time taken to create a job profile with AI being just 45 seconds, albeit averaged over a smaller available sample size of ~300 AI-assisted tasks in production environments.
Conclusion
In conclusion, Oracle’s Fusion Cloud Applications are harnessing the power of AI to drive meaningful improvements in how businesses operate. By leveraging large-scale telemetry data and quantitative metrics, we’re moving beyond the hype to demonstrate tangible benefits in task completion rates and efficiency. While AI adoption is still in its early stages for many businesses, our data clearly shows its potential to streamline workflows, reduce friction, and deliver time savings.
Acknowledgements
The telemetry analysis project has been a collective effort, with special thanks to my Oracle colleagues Wojciech Burzynski, Magdalena Pietrzak, Lee Sacco, Erica Orona, Tiffany Hansen, Lauren Rodenberg, Janani Ramya Pragatheeswaran, and the wider project team.