What’s even hotter than GenAI? That’s easy: GenAI plus retrieval-augmented generation (RAG). Why? Because RAG can address major limitations of large language models (LLMs). A RAG-based approach allows GenAI to provide responses that are both contextually relevant and accurate, even with niche and highly specific information. How? By dynamically retrieving data from enterprise applications to generate detailed, relevant content. With the right implementations and security features, RAG promises to make GenAI a defining feature of the next generation of enterprise apps. This post explains the basics of RAG and how we’re deploying it in Fusion Apps.

Understanding the limits of LLMs

GenAI uses LLMs—trained on a huge amount of data—to understand, summarize, and create new content based on instructions, or “prompts.” One downside of LLMs is that their responses are limited to the information that they’re trained on. So, they don’t perform well on tasks that require new or sensitive, private data. LLMs struggle with tasks that:

  • Depend on confidential information, like comparing revenue sources and amounts in consecutive quarters
  • Are specific to an individual, such as explaining how that person’s current number of vacation days has been accrued
  • Use data that fluctuates constantly, like assessing a supplier’s solvency based on financial statements and credit ratings

Retraining an LLM regularly to keep it updated is prohibitively expensive, resource intensive, and impractical, and it still wouldn’t solve the confidentiality issue.

Enter RAG

Simply put, RAG helps an LLM give better answers by providing it with current, relevant information. Based on the context of a user request, RAG retrieves and safely allows an LLM to use information about a company, its employees, and its suppliers—or anything else in a connected enterprise data store. And it can work by reducing security risks through tokenization and encryption (see details below).

How RAG works in Fusion Apps

RAG in Fusion Apps uses semantic search to fetch relevant data for GenAI-powered features. Semantic search provides an understanding of the intent behind a user’s prompt by considering context—things like location, search history, word variations, and phrases. It then uses machine learning techniques to find the information stored in Fusion Apps that best answers the prompt.  

The specific steps are as follows:

  1. Prompt input: A user inputs a question or prompt.
  2. Document retrieval: The system processes the query to fetch the most relevant data from a predefined data store.
  3. Tokenization: Company-specific data selected for use by the LLM request is considered confidential and is tokenized to ensure anonymity before being handed off to the LLM. By definition, tokenized data has no inherent meaning or value.
  4. Content integration: The retrieved, tokenized data is then fed into the LLM along with the original query. This step is crucial because it augments the information available to the generative model, allowing it to produce more accurate and contextually appropriate responses.
  5. Content generation: Using both the retrieved data and the user’s original query, the LLM generates a response. The model can interpolate between the information in the retrieved, tokenized data and its pretrained knowledge, thereby generating highly relevant, detailed responses.
  6. Detokenization: Data from Step 3 is reconstituted, restoring its meaning in Fusion Apps.
  7. Output: The final, generated content is then presented as the response to the user’s prompt.

Additional benefit

RAG offers benefits beyond the obvious one of using current, confidential data to provide company- and user-specific responses. Generally speaking, LLMs cannot cite a source for information provided in a response because of the vast amount of data they’re trained on. However, RAG-generated responses can since they come from a specific, known data store. This also makes it possible to identify—and correct—erroneous data at the source. And unlike retraining LLMs, which is time-consuming and expensive, it’s relatively inexpensive to regularly update the information in the data stores that RAG uses.  

Fusion Apps security

Oracle’s implementation of RAG addresses confidentiality concerns. It connects to enterprise data that’s already in Oracle-supplied data solutions; and it can run without moving the data back and forth between databases, which is faster, more cost-effective, and reduces security risks.

RAG use cases in Fusion Apps

Here are two specific use cases showing how RAG in Fusion Apps might help employees and organizations work more efficiently and effectively:

  • Oracle Cloud Procurement uses RAG to automatically extract and summarize information found in documents that accompany prospective supplier registrations. This helps with supplier assessment, risk management, and onboarding. It’s particularly beneficial for organizations focused on compliance and supplier diversification.
  • HCM Benefits Advisor uses RAG to provide personalized, accurate, and specific recommendations in response to employees’ natural-language queries—e.g., “If my son gets braces, how much of the cost will be covered by my dental insurance?.” It does this by retrieving and processing data from the controlling benefits documentation. (Coming soon)

Conclusion

The power and capabilities of LLMs and GenAI are widely known and understood. RAG builds on the benefits of LLMs by making responses more timely, more accurate, and more contextually relevant. For business applications, RAG is an important technological advancement that holds the promise of expanding LLM’s utility without compromising security.

Additional resources

To learn more about RAG, check out:

Related posts you might like

 

Get more information link

If you’re an Oracle customer and want to get new stories from The Fusion Insider by email, sign up for Oracle Cloud Customer Connect. If you’re an Oracle Partner and want to learn more, visit the Oracle Partner Community.