Generative AI (GenAI) is advancing rapidly, emerging as one of the most exciting and transformative technologies. Just imagine the power of instantly getting actionable insights from vast amounts of unstructured data without needing expertise in AI or having to move data out of the database. Consider a semiconductor manufacturer sorting through extensive and complex manufacturing logs to identify and predict part failure probabilities and generate a summary of the issues in natural language. Or envision an international bank dealing with the challenging responsibility of detecting money laundering and fraudulent transactions while managing structured data sets and intricate legal documents.

The potential of Generative AI is incredibly promising!

Introducing HeatWave GenAI, the industry’s first automated in-database Generative AI service. HeatWave GenAI is unique because it seamlessly integrates large language models (LLMs) and embedding generation within the database, eliminating the need for external services. This integration enhances performance and security, simplifies application complexity, and reduces costs as it runs on CPUs. HeatWave GenAI integrates seamlessly with other in-database HeatWave capabilities such as machine learning, analytics, and Lakehouse, opening up a new class of applications and providing higher-quality responses. With HeatWave GenAI, you can effortlessly generate new and realistic content, speed up manual or repetitive tasks like summarizing large documents, and engage in natural language interactions.

And the most compelling aspect? HeatWave GenAI is not only an easy-to-use Generative AI solution but is also available at no additional cost. This means you can enjoy the benefits of this efficient Generative AI solution without being an AI expert or straining your budget, making it a smart and cost-effective choice for your business needs.

Current Challenges in Generative AI

Enterprises currently face a range of complex challenges when implementing Generative AI. These challenges include navigating the complex AI landscape, leveraging specialized AI expertise, and managing the significant financial investments required for Generative AI deployment.

• Complexity: Implementing Generative AI solutions requires carefully selecting and integrating large language models (LLMs), meticulously integrating and managing separate vector databases, selecting embedding models and precisely generating vector embeddings, and seamlessly implementing natural language capabilities. These capabilities are often limited to specific regions, vendors  and cloud providers, resulting in inconsistent responses across different regions and clouds. All these factors significantly contribute to the complexity of implementing Generative AI solutions.
• AI expertise: Understanding Generative AI requires a robust technical grasp of LLMs and embedding models. These models must be carefully selected, integrated, trained, and fine-tuned to achieve optimal performance while controlling costs. Even experienced professionals  in  technical fields may lack the necessary expertise to fully comprehend and effectively apply Generative AI algorithms and frameworks to address their domain problems. Furthermore, the field of artificial intelligence is rapidly evolving, making the learning curve very steep.
• Security: HeatWave GenAI offers in-database LLMs and in-database embedding generation.  This in-database approach significantly improves security by eliminating the need to move data outside of the service, mitigating potential vulnerabilities associated with data transfers.
• High costs: Implementing Generative AI solutions requires a significant investment, including provisioning expensive GPUs and other computing resources, allocating ample storage for vector embeddings, and hiring specialized talent to build and manage these state-of-the-art systems.

HeatWave GenAI

HeatWave is a fully managed database service that enables enterprises to efficiently run data warehousing, analytics, machine learning, and transaction processing inside a single database. This eliminates the complexity, latency, risks, and cost of extract, transform, and load (ETL) duplication. Its highly scale-out design enables enterprises to achieve unmatched performance and price performance for processing structured and semi-structured data.

With the introduction of HeatWave GenAI, we are expanding HeatWave capabilities to process unstructured data, enabling enterprises to tap into the power of Generative AI.

HeatWave GenAI offers an integrated environment for Generative AI, where every aspect of the Generative AI pipeline is brought together within the database. It includes in-database LLMs and in-database embedding generation, and it works seamlessly with other in-database features such as machine learning, analytics, and Lakehouse. HeatWave GenAI can also provide users the choice to use LLMs from external services such as OCI Generative AI.

Let us look at some of the key features of HeatWave GenAI:

In-database Large Language Models

HeatWave GenAI supports in-database LLMs, which means that LLMs are included within HeatWave. These in-database LLMs have smaller set of parameters, and are quantized LLMs that run on the same compute resources as the database operations. For example, quantized versions of Mistral-7B-Instruct and Llama3-8B-Instruct. They provide a good trade-off between cost and response quality and are available in all regions where HeatWave is available. Also, your queries and LLM responses stay within HeatWave service tenancy providing enhanced security.

HeatWave also supports OCI Generative AI Service, which uses large foundation models, such as the Cohere command and LLaMA by Meta, to generate high-quality responses. These LLMs are comparatively more expensive because they run on GPUs.

In-database embedding generation

Embeddings play an important role in providing context for LLMs and significantly enhance semantic searches. In most scenarios, embeddings are generated within the application and then transferred to the database, introducing another layer of complexity to the application. With HeatWave GenAI, you can effortlessly generate embeddings within the database without the complexity of having the user select specific embedding models. This simplified approach means significantly reduces application complexity and also eliminates the need for detailed  ML  knowledge to generate the embeddings. Additionally, HeatWave GenAI seamlessly integrates with the automated in-database vector store, saving you from the hassle of transferring embeddings to a separate vector database.

Synergy with HeatWave AutoML

HeatWave offers GenAI, machine learning, analytics, Lakehouse, and transaction processing capabilities inside the database.

For instance, you can feed  all your manufacturing logs into a Large Language Model (LLM) and request it to detect anomalies and pinpoint the cause of the anomalies. However, this process can become costly and slow when dealing with a large volume of logs. Instead, you can make use of in-database machine learning to sift through the logs and detect  relevant anomalies  and then only input those anomalous logs into the LLMs. The LLMs can then summarize the findings and provide the results in a simple and understandable natural language.

Harnessing generative AI within machine learning can substantially boost accuracy, optimize performance and reduce costs

 

Synergy with HeatWave Lakehouse

HeatWave provides support for Lakehouse, enabling it to process semi-structured data stored in Object Storage. This support has been expanded to include unstructured data, and there is seamless integration between the Generative AI capabilities and Lakehouse. This integration empowers HeatWave GenAI to seamlessly ingest unstructured data from Object Storage using the existing APIs, leveraging it to provide context to LLMs and significantly improving semantic searches.

You can upload unstructured enterprise documents like PDFs, HTML, TXT, PPT, or DOCX files to Object Storage. HeatWave parses the document, generates embeddings inside the database, and stores them in the vector store. When you ask a question in natural language, embeddings are generated for the question, and then a semantic search is performed with the stored embeddings to identify the most relevant documents related to the question. These documents augment the prompt given to the LLM, enabling it to offer a more contextual response.

Benefits of HeatWave GenAI

The HeatWave GenAI offers a multitude of benefits. Here are just a few of the highlights:

Simplicity

• HeatWave GenAI offers in-database LLMs. You do not have to undertake the task of selecting, integrating, and managing external LLMs.
• No AI expertise is needed to use the LLMs. You just have to load the LLMs into the HeatWave cluster (in heatwave and from OCI Gen AI) , enabling you to generate and summarize content and much more.
• HeatWave GenAI manages all stages of the AI data pipeline. Generative AI applications only need to interface with HeatWave Gen AI, whichhas  simple APIs. This simplicity makes it easy for an enterprise to develop turnkey Generative AI applications and achieve immediate benefits by seamlessly integrating AI into their business without the complexity of developing their own AI solutions.

In other generative AI solutions, users have to integrate multiple services. For example, in AWS, developing a generative AI application is complex because users need to understand and integrate several services such as Bedrock, Aurora, Pinecone, Knowledge Base, and OpenSearch. This process can take several days, and any mistakes require recreating the vector, leading to significant costs.

Flexibility

•  In-database LLMs are designed to run on CPUs, allowing HeatWave GenAI to be available everywhere HeatWave is present. As a result, your applications can be accessed in any OCI region or dedicated regions.

 

• When using LLMs, you have the flexibility to choose between in-database LLMs, which are stored and processed within HeatWave, or LLMs supported by OCI Generative AI service. This flexibility enables you to select the option that best aligns with your requirements.
• In addition to natural language, HeatWave GenAI is also available through the SQL interface.

Performance

• HeatWave GenAI operates on CPUs within the database, ensuring consistent and reliable performance.
• HeatWave is not a shared service and there is no network latency, providing performance isolation.
• In-database LLMs enhance agility and performance since there is minimal data movement.
• HeatWave’s parallel processing delivers scalable and fast performance.

Security

• All data transformations during generating embeddings are performed inside HeatWave, and the resulting embeddings are stored within the HeatWave service.
• Your propriety data, which provides context to the LLMs, does not leave the database, and provides you with data isolation.

Reduced cost

• HeatWave GenAI runs on HeatWave cluster with no additional costs.
• HeatWave GenAI runs on CPUs, which are significantly more cost-effective, are general purpose making them a convenient choice for running multiple workloads, readily available, and easy to get started with.
• The vector embeddings, which provide context to the LLMs and enable enterprises to use their own proprietary data, can take up substantial space. These embeddings are stored in Object Storage within the service, providing a cost-effective solution.
• The system resources are optimized for thread count, batch size, and segment size configuration, further reducing the cost.

HeatWave GenAI Use Cases

HeatWave provides an integrated and automated platform that offers Generative AI with machine learning, analytics across data warehouses and data lakes, and transaction processing, making it easy to build applications. Let’s look at a few HeatWave GenAI use cases.

Content generation

HeatWave GenAI empowers enterprises to create content in multiple languages for diverse needs, such as social media posts, blog articles, and email campaigns. Additionally, it can improve existing content based on your insights and suggestions. By automating content generation with HeatWave GenAI, enterprises can save time and resources while expediting time to market.

Content summarization

HeatWave GenAI enables you to quickly and effortlessly generate concise and accurate summaries of documents, reports, and logs while retaining essential information. For example, this feature can be used in e-commerce website to generate concise summaries that highlight both the positive and negative aspects of a product. This saves customers time and effort, as they can quickly grasp the key points without getting lost in the details. Simply upload any document and witness instant, efficient summarization with HeatWave GenAI.

Retrieval Augmented Generation

The in-database LLMs are trained on publicly available data. However, enterprises can leverage HeatWave GenAI to generate content based on their own proprietary data. To achieve this, HeatWave can convert the proprietary data into embeddings and store them in vector store. These embeddings can be used to provide enterprise-specific context for LLMs in Retrieval Augmented Generation (RAG) use cases. With RAG support in HeatWave GenAI, you can get more accurate and contextually relevant answers, and perform semantic searches on unstructured data.

Natural language interaction

HeatWave GenAI enables you to communicate with unstructured documents using natural language and receive responses in natural language, preserving context for follow-up questions.

Simple steps to use HeatWave GenAI

HeatWave GenAI is easy to use for content generation, summarization, and RAG (Retrieval-Augmented Generation) use cases. Let us see how to use HeatWave GenAI for content summarization and RAG.

Content summarization

1. Load the model

MySQL>CALL sys.ML_MODEL_LOAD(“mistral-7b-instruct-v1”, NULL);

2. Set the document you want to summarize.

MySQL> SET @document = “heatwave_doc_to_summarize”;

3. Run the query to summarize the contents of the file.

MySQL>SELECT JSON_PRETTY(sys.ML_GENERATE(@document, JSON_OBJECT(“task”, “summarization”, “model_id”, “mistral-7b-instruct-v1”)));

4. HeatWave GenAI summarizes the document for you.

+————————————————————+

|{

 “text”: “MySQL HeatWave is a fully managed database service that allows developers to quickly develop and deploy secure, cloud-native applications using the world’s most popular open-source database. It provides a unified MySQL cloud database service for transactions, real-time analytics across data warehouses and data lakes, and machine learning—without the complexity, latency, risks, and cost of ETL duplication. HeatWave is designed to run analytics on data which is stored in MySQL databases or in object storage, without the need for ETL. It is built on an innovative, in-memory analytics engine that is architected for scalability and performance. It is available on OCI, AWS, Azure, and in customers’ data centers with OCI Dedicated Region.”

}|

+————————————————————+

Retrieval Augmented Generation

1. Load the model

MySQL>CALL sys.ML_MODEL_LOAD(“mistral-7b-instruct-v1”, NULL);

2. Load the files that contain propriety enterprise data in the HeatWave cluster.

MySQL> CALL sys.vector_store_load(

 “oci://user_bucket@user_namespace/pdf_files/”,

 ‘{

  “formats”: [“pdf”],

  “table_name”: “pdf_store”,

  “schema_name”: “user_documents”

  }’);

3. Run the query to generate more accurate and contextually relevant responses with RAG.

MySQL> CALL sys.ML_RAG(“What is Heatwave ?”, @output,  NULL);

4. HeatWave GenAI sifts the documents and provides you with the response with the correct context.

MySQL> SELECT JSON_PRETTY(@output) \G;

**************************** 1. row ***************************

JSON_PRETTY(@output): {

  “text”: ” HeatWave is a unified MySQL cloud database service that provides transactions, real-time analytics across data warehouses and data lakes, and machine learning capabilities. It is designed to simplify…”,

      “citations”: [

  {

“segment”:  MySQL HeatWave provides a unified MySQL cloud…”,

“distance”: 0.36409956216812134,

“document_name”:”https://objectstorage.uk-london- 1.oraclecloud.com/pdf_files/doc.pdf”},

 …

]}

Conclusion

HeatWave GenAI signifies a remarkable leap forward in the realm of Generative AI. It offers an automated in-database service that addresses the complexities, AI expertise, performance, security, and cost challenges associated with implementing Generative AI solutions. HeatWave GenAI provides automated in-database LLMs, in-database embedding generation, and seamless integration with machine learning, empowering businesses to leverage Generative AI for new use cases. With HeatWave GenAI, enterprises can unlock the potential of Generative AI without the traditional barriers, making it a valuable addition to any organization’s technological arsenal.

Are you ready to embark on your generative AI journey with HeatWave GenAI? Here are additional  resources to help you choose your first use case:enterprises can save time and resources while expediting time to market.

• HeatWave GenAI
• HeatWave GenAI for Healthcare Demo
• HeatWave GenAI AskME Demo
• Develop New Applications with HeatWave GenAI