X

The latest cloud infrastructure announcements, technical solutions, and enterprise cloud insights.

Subscribe to email updates

Get insight into API Gateway logs using OCI Logging Analytics

Paul Jenkins
Senior Principal Product Manager

In my previous blog, I gave an overview of Oracle Cloud Infrastructure (OCI) API Gateway observability. This time I’m taking a deeper look into using OCI Logging Analytics to gain insight on OCI API Gateway logs.

This post covers setting up API Gateway logging, setting up Logging Analytics, setting up Service Connector Hub to send API Gateway logs to Logging Analytics, and creating a Dashboard. Prerequisites include a working knowledge of OCI API Gateway and OCI in general.

For the sake of simplicity, I have created all resources in the same compartment. Your security might require that different log types be stored and analyzed in different compartments, which is fully supported.

Enable API Gateway logging

Gateway logs are controlled at the deployment level within a gateway and are disabled by default. To collect the data to build any real insights, we need to set the deployment logging level to Information and enable the logs.

Navigate to your deployment in the Console, select Edit, and set the logging policy to Information. This setting is the default, but it’s worth checking.

A screenshot of the API Request Policies page in the OCI Console.

Next, navigate to Logs for the deployment.

A screenshot of the Deployment Details page for Echo, showing the logs.

There are two log categories, access and execution. For completeness, enable both. However, if you want to limit log storage requirements, enable only the access log.

A screenshot of the Enable Log window in the Console.

After completion, click Log Group to see the API Gateway log records.

A screenshot of the Log Group screen showing the API Gateway log records.

API Gateway logging is now active, and you can search the logs by clicking Explore with Log Search.

Set up Logging Analytics

To use Logging Analytics, you need to set up some security policies to allow the service correct access to resources in your tenancy.

The easiest way is to follow the first two sections of the quick start guide. These instructions get the policies set up and Logging Analytics enabled in your tenancy for the region you’re working in. We don’t need to install a management agent because we have logging in the API Gateway deployment enabled. The rest of the quick start guide is useful for learning about different ways to explore logs. If you need more detailed information, see the service documentation.

Next, we need to set up a log group within Logging Analytics as a target for the API Gateway logs.

Navigate to Logging Analytics and then click Administration. Select Log Groups.

A screenshot of the Log Groups page in the Console.

Click Create Log Group. Enter a name and click Create.

Send API Gateway logs to Logging Analytics using Service Connector Hub

Service Connector Hub is a cloud message bus platform that offers a single pane of glass for describing, running, and monitoring the movement of data between services in Oracle Cloud Infrastructure.

We can easily create a connector to send API Gateway log data from the log group that we created when we enabled API Gateway logging to the log group we created when we set up Logging Analytics.

Navigate to Service Connector Hub.

A screenshot of the Service Connectors page in the Service Connector Hub section of the OCI Console.

Click Create Service Connector.

A screenshot of the Create Service Connector page.

Enter a name and description.

Select Logging as the source and Logging Analytics as the target.

To configure the source connection, select the API Gateway log group and select the run log. Click +Another Log, select the same log group, and select the access log.

To configure the target connection, select the Logging Analytics log group that we created earlier.

A screenshot of the Create Service Connector page with the Log Group menu highlighted.

Click Create. Check that logs are sent by selecting the Service Connector and clicking Metrics. Data shows up when data is being sent.

A screenshot of the Metrics section for the service connector.

This process can take a couple of minutes, depending on the amount of traffic going through the gateway.

Now that we have log data generated by the gateway and being sent to Logging Analytics with Service Connector Hub, we can explore the log data.

Using Logging Analytics to create a dashboard

The last stage is to create a saved log search and add this to a dashboard.

Navigate to Logging Analytics and select Log Explorer. A default search like the following example appears.

A screenshot of an example Log Explorer in the Console.

To get more insights into the gateway traffic, change the visualization and the search. Click the pie chart under Visualizations on the left panel and then select the Link icon.

A screenshot of the Log Explorer, showing the access traffic.

This selection changes the appearance to something not useful, so we need to change the search command.

A screenshot of the Access Traffic page, showing the log sources and display options.

Copy the following search statement into the search command line:

* | link 'OPC Request ID' | stats unique(Path) as Path, unique('OCI Resource Name') as Deployment, unique(Method) as Method, avg('Content Size Out') as 'Total Bytes', unique(Duration) as Duration, avg(Duration) as 'Ave. Duration (sec)', unique(Status) as Status, unique('Flow Code') as 'Flow Code', unique(Label) as Label | eval 'Ave. Duration' = toDuration(formatDuration('Ave. Duration (sec)' * 1000)) | classify topcount = 300 'Start Time', Deployment, 'Ave. Duration', 'Total Bytes', Status, Label as 'API Gateway Logs'

A screenshot of the search statement added into the Access Traffic page.

Click Run.

You get a visualization of status code distribution over time with the average API Call duration and a list of log record details. You can change the visualization filters to see different views.

A screenshot of the Access Traffic page in the Console, showing the visualization filters.

In the top right of the screen, click Actions. Select Save As, enter a name for the search, select Add to Dashboard, select NEW DASHBOARD, and enter a name and description.

A screenshot of the Save Search window.

Now navigate to Logging Analytics and Dashboards. Select the dashboard you created, which looks like the following screenshot.

A screenshot of the example MyNewDashboard screen, showing the save search results.

Because we created a dashboard as a shortcut when saving the search, it created a default sized widget that doesn’t display enough of the information. So, we need to resize it.

To go into edit mode, click Edit.

A screenshot of the edit mode of the widgets.

Hover over the bottom edge of the widget and resize it to fill the dashboard.

A screenshot of the expanded widget, showing more information.

Click Done, and the dashboard now displays a full screen of information.

A screenshot of the expanded dashboard.

You now have a Logging Analytics dashboard for API Gateway log analysis.

Next steps

You can create other saved searches and add them to the dashboard. You can save and add them in edit mode by dragging them from the widget list on the right side and arranging them on the canvas.

The following screenshot is an example of visualizations that you can bring together in a dashboard that can be useful for API Gateway managers.

A screenshot of an example dashboard with color-coded graphs and widgets.

For an overview of the service, see the OCI Logging Analytics launch announcement.

For more ideas for dashboards using Logging Analytics for other services, see the following blog posts:

Join the discussion

Comments ( 1 )
  • Omri Tuesday, February 9, 2021
    very helpful
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha

Recent Content