Introduction

In another post, I discussed the nitty-gritty of Oracle's Generative AI service and why it is one of enterprises' most trusted, go-to generative AI offerings.

This post takes the next step to get our hands wet and learn how can we leverage the service using the well-documented inference APIs.

We will see how to create a small Visual Builder application that passes user prompts to the GenerateText service API, gets back the response, and shows it to the user. So lets get started!

Implementation

In the recorded walk-through (video below), I am demonstrating the important aspects of how one can get the information that is required to complete the use case.

Step 1 – Access the documentation

You can find the Oracle Generative AI Service Inference API documentation here. As of this writing, the supported endpoints are:

Get yourself familiar with the endpoints, request/response shapes, and examples given in the documentation of each of the above services.

Step 2 – Get a sample "working" payload

Sometimes the sample/example payloads given in the official documentation may be old as compared to what the API is expecting in its latest form. This may happen because the service is still new and rapidly evolving each day.

I have included how you can get the "working" payload as a crucial step in this demonstration. We need this for the next step. Here is a sample:

{
  "compartmentId": "ocid1.compartment.oc1..<your-compartment-id>",
  "servingMode": {
    "modelId": "ocid1.generativeaimodel.oc1.us-chicago-1.<model-id>",
    "servingType": "ON_DEMAND"
  },
  "inferenceRequest": {
    "prompt": "As a corporate vice president, generate an email congratulating a team that has just shipped a new cloud service. Emphasize the great positive impact the new service will have on the productivity of their customers.",
    "maxTokens": 600,
    "temperature": 1,
    "frequencyPenalty": 0,
    "presencePenalty": 0,
    "topP": 0.75,
    "topK": 0,
    "returnLikelihoods": "GENERATION",
    "isStream": true,
    "stopSequences": [],
    "runtimeType": "COHERE"
  }
}

 

Step 3 – Create Service Connection in Visual Builder Application

Next, you will see how you create and test a service connection in your VB application by using the information gathered in the above steps.

You will see that for authentication, I am using OCI API Signature. This is one of the recommended approaches to connect to any OCI service, and it needs you to generate the API Keys for your user in the OCI console. For more on the connection type see the blog about using OCI API Signature Authenticaction in VB.

Step 4 – Build the UI and test

For the sake of simplicity, you will see that the UI shown in the demo has limited controls. An input text area where user can enter their desired prompt, one button to invoke the GenerateText service using the connection created in the above step, one button to clear all the variables, and one output text area to display the response from the GenerateText service call.

Though there are several attributes in the request payload (refer to step 2 for sample), for this demonstration I have hardcoded most of them in the request variable and the only attribute that is based on user input is "prompt". I am mapping the user input to this attribute in the request as you will be able to point out in the action chain.

You can enhance the UI as per your liking to include other parameters (temperature, topK, topP, etc) that can be passed to the service call. Users can modify the parameters and submit the request to see the impact they have on the response.

I am providing the git repo link below. Feel free to fork/download the app and play around with it!

Demonstration

 

Resources

Conclusion

In this post, we dived into leveraging Oracle's Generative AI service through its inference APIs by creating a Visual Builder application. The application takes user prompts and use the GenerateText API to generate and display responses. We covered accessing API documentation, identifying working payloads, setting up service connections using OCI API Signature for authentication, and building a simple UI. A step-by-step recorded demonstration and a sample payload are included for practical guidance, along with a link to a GitHub repository for further exploration.