As a continuation to Part 1, where we ingested logs from Oracle Cloud Guard to the Logging Analytics service through the Streaming service. In part 2, we use Oracle Cloud Infrastructure (OCI) Functions service to ingest Oracle Cloud Guard logs to Logging Analytics.

The following graphic depicts the architecture for the solution:

A graphic depicting the architecture for the solution,moving from Cloud Guard to Events to Functions to Logging Analytics.

When Cloud Guard emits events that match with the defined rule conditions, the OCI Events service triggers the specified function. When invoked, the Functions code reads logs data from Events and sends them to the Logging Analytics upload API endpoint. We can then view the logs from Log Explorer dashboard and process them for any further visualizations and insights.

Overview steps

  1. Enable a cloud event rule in responder recipe.

  2. Create a log group under Logging Analytics to store logs.

  3. Create a log parser and source under Logging Analytics.

  4. Create an OCI function in Python or any supported language to send Cloud Guard logs to Logging Analytics.

  5. Create a dynamic group and provide appropriate policies for the function to access Logging Analytics.

  6. Create an Event rule to trigger the deployed function when the rule conditions for specified event types are met.

For the first three steps, follow the implementation in Part 1. Enable the cloud event rule in responder recipe. Create a log group under Logging Analytics. While provisioning, utilize OCI’s compartment structure to fine tune user access to Logging Analytics components. Create a log parser and source under Logging Analytics.

Step 4

To create the function, use the following steps:

  • In the Oracle Cloud Console menu, navigate to Developer services and select Functions.

  • Select an existing application or click Create Application. Create a function within your application.

  • I’m using OCI Cloud Shell to create my function. Cloud Shell is a recommended option as it does not require any setup prerequisites. If you’re new to OCI Functions, follow sections A, B, and C in the Functions Quick Start on Cloud Shell.

  • We recommend creating a sample Python function first. The following command generates a folder “cgtola” with three files :func.py, func.yaml, and requirements.txt.

    fn init --runtime python cgtola
  • Then create a function with the following code:

    func.py:

    import io
			import json
			import logging 
			import oci 
			from fdk import response 

			 #The below method will receive the list of log entries from OCI as input in the form of bytestream and is defined in func.yaml 
			 def handler(ctx, data: io.BytesIO = None): 
			     funDataStr = data.read().decode('utf-8') 
			     funData =  json.loads(funDataStr) 
			     logging.getLogger().info(funData) 
			
			    #send the log data to a temporary json file. /tmp is the supported writable directory for OCI functions 
			     with open('/tmp/test.json', 'w', encoding='utf-8') as f: 
			         json.dump(funData, f, ensure_ascii=False, indent=4) 
			     file_details=io.open("/tmp/test.json") 
			     signer = oci.auth.signers.get_resource_principals_signer() 
			     log_analytics_client = oci.log_analytics.LogAnalyticsClient({},signer = signer)

			 # Send the request to service, some parameters are not required, see API doc for more info 

			     log_group_id="ocid1.loganalyticsloggroup.oc1.iad.amaaaaaas4n35viawkqpxtefipgoqemkc2n55b7hylaucb2vck6z4xxxxxxx" 
			     upload_log_file_response=log_analytics_client.upload_log_file(namespace_name="orasenatdpltsecitom04", upload_name="CG_logs", log_source_name="Cloud Guard_functions_source", filename="test.json", opc_meta_loggrpid=log_group_id, upload_log_file_body=file_details)

    func.yaml:

    			 schema_version: 20180708 
			 name: cgtola 
			 version: 0.0.31 
			 runtime: python 
			 build_image: fnproject/python:3.9-dev 
			 run_image: fnproject/python:3.9 
			 entrypoint: /python/bin/fdk /function/func.py handler 
			 memory: 256

    requirements.txt:

    			 fdk>=0.1.48 
			 oci

  • Deploy the function with the following command:

    fn -v deploy --app CG-LA

When invoked by the Events service, the Functions code fetches log data from Events service and uploads it to the Logging Analytics API.

Step 5

A screenshot of the Dynamic Group Details page in the Oracle Cloud Console.

We now create policies to grant the created dynamic group access to Logging Analytics resources. For details, refer to Enable Access to Logging Analytics and its Resources.

  • In the menu, select Identity and Security, Identity, and Policies and click Create Policy. Give the following policy statements:

    • allow dynamic-group Lasya-DG to manage loganalytics-features-family in tenancy

    • allow dynamic-group Lasya-DG to manage loganalytics-resources-family in compartment Lasya

Give the first policy only at tenancy (root) level. The second policy can be either given at child compartment level or tenancy (root) level.

Step 6

Create an event to trigger our function when the specified Cloud Guard event types are matched.

A screenshot of the Edit Rule page, showing how to connect to the Events service.

We can verify the uploaded Cloud Guard logs on the Log Explorer dashboard. In the menu, select Observability & Management, Logging Analytics, and Log Explorer.

A screenshot of the Log Explorer in Logging Analytics.

Conclusion

In this blog, we covered how to ingest Cloud Guard events to Logging Analytics using OCI Functions. Logging Analytics makes it easy for you to unlock the value of log data for any source across Oracle Cloud Infrastructure, on-premises, and third-party clouds like Amazon Web Services (AWS) and Azure quickly without involving any data scientists.

For more use cases of Logging Analytics, see the Observability and Management blog.