Welcome to the cutting-edge world, where artificial intelligence (AI) isn’t just a buzzword but a tangible reality shaping the future of technology. Today, we’re venturing into an exciting journey that marries the versatility of LangChain, a powerful library for building language model applications, with the robustness of Oracle Cloud Infrastructure (OCI) Data Science, a fully managed and serverless platform for data science teams to build, train, and manage machine learning (ML) models. This blog post is your gateway to understanding and harnessing the combined strengths of these two formidable tools in the realm of AI and ML.

background

Imagine a scenario where the intuitive language processing capabilities of LangChain meet the scalable, secure, and collaborative environment of the OCI Data Science platform. What you get is an unparalleled synergy that can revolutionize how we approach complex AI tasks, from developing conversational agents to creating advanced analytics solutions. Whether you’re a seasoned data scientist, an AI enthusiast, or a curious explorer in the tech landscape, this post guides you through the process of integrating these two platforms, showcasing practical applications, and unveiling new possibilities.

Our journey walks you through a comprehensive example of deploying Llama 2 model in the OCI Data Science service. We also help you create a simple prompt template and set up a large language model (LLM) chain. Finally, you learn how to invoke your Llama 2 model by running your LLM chain with text input. By the end of this post, you should have a clear roadmap for applying the combined power of LangChain and OCI Data Science to elevate your AI projects to new heights.

So, prepare to embark on a thrilling exploration where language meets cloud computing, and innovation meets practicality. Let’s unlock the potential of AI together!

Deploy LLM with OCI Data Science

OCI Data Science AI Quick Actions offers a streamlined way to deploy LLM models on the OCI. With just a few clicks, you can have your model up and running, taking advantage of the robust infrastructure and advanced capabilities of OCI. This quick deployment process allows you to focus on developing your application rather than managing the underlying infrastructure.

Deploying Chat model with AI Quick Action

For more details on AI Quick Actions, you may checkout the product tour video.

You may also deploy the LLM with custom container or service managed containers. For more details, check our post and tutorials:

Once your model is deployed and running, you can write the code to interact with your model and begin building LangChain applications.

Build Application with LangChain

Python 3.9 or newer is required to use the latest feature in LangChain. Building LangChain applications also requires some dependencies, including LangChain and Oracle Accelerated Data Science (ADS). LangChain is a framework for developing applications powered by language models. The Oracle Accelerated Data Science (ADS) library provides the components to connect the LLMs on OCI with other LangChain components.

Here are a few tutorials:

You can checkout the ADS documentation for more details on authentication, completion models, chat models and tool calling.

Deploy LangChain Application

Once you have your LangChain application, you can deploy it as a REST endpoint with OCI data science service. To streamline the process, the ADS library provides utilites to prepare your LangChain application as model artifact, save it into model catalog and deploy it as model deployment. Checkout the tutourial on Oracle samples Github for step by step examples.

Since the LLM and the LangChain application are deployed separately, you can have multiple LangChain applicatins deployed on CPU shape to interact with LLM deployed on GPU shape. This gives you the flexiblity to manage and scale the applications independently.

What’s next?

In this blog post, we explored how to effectively utilize LangChain for interacting with LLMs hosted on OCI Data Science. For more information and examples for deploying different language models in OCI Data Science, check out the Oracle GitHub samples repository and download Oracle ADS from PyPI to streamline the process.

Try Oracle Cloud Free Trial! A 30-day trial with US$300 in free credits gives you access to Oracle Cloud Infrastructure Data Science service. For more information, see the following resources:

Got questions? Reach out to us at ask-oci-data-science_grp@oracle.com.