Developing applications using LangChain and MySQL HeatWave GenAI – Part 1

MySQL HeatWave is a fully managed database service designed to help enterprises efficiently handle data warehousing, analytics, machine learning, and transaction processing within a single database. With the launch of HeatWave GenAI, the platform’s capabilities now extend to unstructured data, empowering enterprises to harness the potential of Generative AI. HeatWave GenAI features in-database LLMs, in-database embedding generation, and works seamlessly with other in-database functions such as machine learning, analytics, and Lakehouse. Additionally, HeatWave GenAI allows users the flexibility to leverage large parameter LLMs from OCI Generative AI Services.

This post shows how to use MySQL HeatWave GenAI capabilities with Langchain. While HeatWave GenAI provides powerful, integrated capabilities for interacting with your data using natural language directly within the database, many advanced applications require more complex logic and orchestration. Developers often need to build sophisticated workflows that chain multiple LLM calls, integrate with external APIs, or create autonomous agents that can take actions. This is where a dedicated framework becomes essential, creating a natural transition to a tool like LangChain.

LangChain is a framework designed to simplify the development of applications powered by large language models (LLMs). It provides a set of tools and abstractions to build complex workflows, allowing developers to integrate LLMs with external data sources, APIs, and systems.

HeatWave GenAI can be easily integrated with LangChain applications by specifying a custom LLM wrapper class and then using it like any other LLM within LangChain. With this integration, creating a chatbot becomes as simple as

    llm = MyLLM()
    llm.invoke(
            [
                HumanMessage(content="Hi! I'm Bob"),
                AIMessage(content="Hello Bob! How can I assist you today?"),
                HumanMessage(content="What's my name?"),
            ]
        )
    
    Hello Bob! You've asked a great question! Your name is indeed "Bob". Is there anything else I can help you with, or would you like to chat about something in particular?

This LLM object can further be used in any LangChain application, whether its chatbots or RAG agents. Here is the implementation of the MyLLM class

    class MyLLM(LLM):
        """
        Custom class for HeatWave LLMs
        """
        # HeatWave GenAI LLM to use. Can be an in-HeatWave LLM or OCI Generative AI LLM.
        # https://dev.mysql.com/doc/heatwave/en/mys-hw-genai-supported-models.html
    
        model_id: str = "llama3.2-3b-instruct-v1"
    
        # Helper function to execute SQL queries and return the results as a Pandas DataFrame
        def _execute_sql(self, sql: str) -> pd.DataFrame:
            mycursor.execute(sql)
            return pd.DataFrame(mycursor.fetchall(), columns=mycursor.column_names)
    
        def _call(self, prompt: str, stop: Optional[List[str]] = None, **kwargs: Any,) -> str:
            """Run the LLM on the given input.
            """
            if stop is not None:
                raise NotImplementedError("stop kwargs are not permitted.")
    
            output = self._execute_sql(
                f"""SELECT sys.ML_GENERATE("{prompt}", JSON_OBJECT("task", "generation", "model_id", "{self.model_id}"));"""
            )
            return json.loads(output.iat[0, 0])["text"]
    
        @property
        def _llm_type(self) -> str:
            """Get the type of language model used by this chat model"""
            return "custom"

The MyLLM class overrides the LangChain LLM class to internally call HeatWave GenAI. HeatWave exposes its Generative AI interface as SQL stored routines, therefore we need a execute_sql method to run SQL queries on the database. The results of the SQL query are converted to a Pandas Dataframe, allowing us to use the results in Python. The execute_sql function requires a mycursor object to be defined, which can be created using MySQL connector Python.

The above example uses an in-database LLM (llama3.2-3b-instruct-v1), which has been optimized to run on the HeatWave cluster. HeatWave provides support for in-database LLMs at no additional cost and accessible from the SQL interface itself, simplifying application deployment architectures.

As you can see, using HeatWave GenAI with LangChain is simple, and the above custom class should serve most usecases. You can of course extend the above class to add more supported options like temperature, max_tokens or any other supported parameter. Happy hacking!

We invite you to try HeatWave GenAI. If you’re new to Oracle Cloud Infrastructure, try Oracle Cloud Free Trial, a free 30-day trial with US$300 in credits.