Imagine stepping up to a camera, taking a golf swing, and having an AI golf pro giving you tips before your next attempt: “Your tempo was steady, but your finish was less stable.” There might be a little friendly banter along the way, too: “Looking sharp in blue. Let’s see if your swing matches.” You could then ask, “What should I focus on first?” and discuss your swing result with an AI coach.
At Oracle AI World, our Swing Smarter demo brings this experience to life through motion analysis on Oracle Cloud Infrastructure (OCI) and live voice interaction powered by Agora. We partnered with Agora to combine personalized spoken commentary with a conversational AI coach, giving participants a chance to explore their results as if they were discussing strategy with an actual caddy or golf pro.
From camera footage to movement insights
The experience begins with a camera capturing the participant’s swing. Pose estimation identifies body keypoints, including shoulders, elbows, wrists, hips, and knees. Tracking those points across frames helps the application describe the golfer’s movement.
Golf-specific logic then calculates indicators such as tempo, arm extension, follow-through, and finish balance. These contribute to a score and give participants specific aspects of their swing to explore.
The analysis focuses on camera-observed movement. Camera positioning and visibility affect the estimates, while measurements such as ball speed and spin require additional instrumentation.
Participants can review their results, replay the attempt, and share their experience through a QR code or email.
Turning analysis into a conversation
A score becomes more useful when a participant can ask what it means.
The application uses OCI Generative AI to turn observations and findings into personalized commentary, from camera-aware remarks to post-swing analysis. Through Agora’s Conversational AI Engine, the generated text becomes spoken feedback delivered to the browser.
Participants can then select Talk to your coach to start a live voice conversation. The application supplies the selected swing’s saved metrics and analysis as context, allowing the agent to discuss that attempt.
Questions might include:
- “What does my tempo score mean?”
- “Which part of my swing should I focus on?”
- “Can you explain the finish-balance result?”
Agora coordinates speech recognition, response generation, and speech synthesis through its configured providers. Its real-time communication services carry the audio, while live captions and agent-status updates help participants follow the exchange.
The conversation uses the saved result as its reference point. This gives the AI coach a defined basis for explaining metrics and discussing a practice focus, with instructions to acknowledge missing information and avoid inventing measurements.
The result is a feedback cycle participants can engage with naturally: hear an observation, ask for clarification, and choose a focus before swinging again.
The architecture behind the experience on OCI
The demo connects three parts:
- The participant experience: A browser handles camera capture, displays the pose overlay and results, and provides access to replay, sharing, and voice conversation.
- The OCI application and analysis workflow: Application services on OCI Compute coordinate the session, run pose processing and scoring, manage results, and use OCI Generative AI to prepare commentary. Autonomous Database stores users, events, and results, while OCI Object Storage holds swing videos and replay files.
- The Agora conversational AI and real-time media layers: Agora RTC streams camera video to OCI for analysis and carries audio between the browser and AI coach. Agora’s Conversational AI Engine coordinates speech recognition, AI responses, and speech synthesis so participants can discuss their results naturally, including interrupting to ask follow-up questions. Agora Signaling delivers captions and agent-status updates. The application sends commentary text and relevant result context to Agora, connecting the voice experience to the swing analysis.
For background on pose models and deployment choices, see The Anatomy of Motion: Human Pose Estimation Architectures on OCI.
Golf is the starting point. The same approach could help people learn a new movement, practice a skill, or engage with an interactive experience. In each case, conversation makes the feedback easier to understand and act upon.
Beyond golf
Sports and fitness. An application could evaluate a squat, tennis serve, or practice drill against activity-specific criteria. Spoken feedback could highlight observations between repetitions, and a voice agent could explain them when the participant asks for more detail. Coaches would define and validate the measures appropriate to the activity.
Workplace ergonomics. Video analysis could help specialists review repeated reaching, bending, or posture changes across a task. A conversational interface could make those observations easier to explore alongside workstation details and other context, helping reviewers identify moments that deserve closer attention.
Physical skills training. Instructors could define observable checkpoints for a movement sequence. An application could highlight where a learner’s attempt differs from those checkpoints, then explain the relevant step through voice interaction. Tasks involving tools or fine hand movements may require additional detection models.
Interactive experiences. Museums, games, and event activations could respond to gestures or completed challenges with personalized narration. Visitors could ask how an activity works, participate through movement, and discuss their results afterward.
Across these examples, the capture, analysis, and conversation components provide a reusable foundation. Each domain still needs its own metrics, expert input, and validation to make the feedback useful.
Take a swing at Oracle AI World
Our golf demo brings OCI motion analysis and Agora voice interaction together in a hands-on experience. See your movement translated into results, hear personalized commentary, and ask the coach about your attempt.
Visit our booth at Oracle AI World to experience Swing Smarter with OCI and Agora. Bring your swing and your questions.
Explore Oracle AI World to plan your visit.