X

Oracle Management Cloud Blog covers the latest releases, customer stories, how-to guides and more.

  • July 10, 2020

Security Insights for your web apps with OMC Log Analytics

Nazih Bachir DJOUMI, and Scott Elvington

1. Introduction

Oracle Cloud Infrastructure (OCI) Web Application Firewall (WAF) is an Oracle Cloud Service that protects your web applications against threats. Logs are available within the WAF Service. In this blog, we’re going to leverage those logs in order to build a comprehensive dashboard with Oracle Management Cloud (OMC) and get insights on what’s happening in the Web Application from a security perspective.

The final result of this blog will be the dashboard shown below, including tabs for Activity Overview, Top 10 OWASP Threats, and detected events from several sources.  An exported version of this dashboard will be available at the end of this blog for you to import into your own OMC environment.
 

In this article, we will see how to forward WAF Logs to an OCI Object Storage Bucket, configure an OCI Event Service to trigger an OCI Serverless Function, utilize REST APIs to generate OAuth tokens to upload the logs into OMC Log Analytics, and finally import the above mentioned dashboard..

As a prerequisite, you should be familiar with OMC, IDCS, OCI and rest APIs. You will need an Oracle cloud account already provisioned with a specific compartment. If you don't already have an account, you can sign up for Oracle Cloud's Free Tier. Let’s proceed with my component called omcwaf_compartment, a WAF policy configured in that compartment, an OMC tenant, an OCI Object Storage created in the compartment and an IDCS account. You will need to make sure that you have admin access to all the accounts.

Before starting, let’s prepare all the needed details from OCI, OMC, IDCS, WAF policy and OCI Object Storage.  You may find it useful to put the collected details along with their step number into a text editor for easy reference in later steps.

All the needed resources to complete this integration are available for download.

2. COLLECT INFORMATION

2.1 From OCI

2.1.1. Go to Administration > Tenancy details and copy the tenancy OCID 

2.1.2.  In Administration > Tenancy, pick the Region Name. Go to Administration > Region Management save the corresponding Region Identifier

2.1.3. In Administration > Identity > Users, create a new user for the integration between the WAF Log and the OCI Bucket. Add the API Public Key and copy the Fingerprint. 
2.1.4. Copy the OCID
2.1.5. Click on Customer Secret Keys, and generate a new secret key. Save the secret key in a safe place
2.1.6. Copy the Access Key
2.1.7. In Administration > Identity > Groups, create a new group and add the user previously created on this group. Copy the group OCID
2.1.8. Copy the group name.
 

2.2 From OMC

2.2.1. Go to Administration > Agents, click on the navigation button on the right then select Download Agents.  Select  Gateway as the Agent Type, then copy TENANT_NAME from the bottom of the page
2.2.2. Copy the OMC URL from the browser URL bar ending with oraclecloud.com 

2.3. From IDCS

From OCI, Go to Identity > Federation , click on OracleIdentityCloudService. Copy the IDCS Console URL.  If you are not federating IDCS identity for OCI, you can obtain your IDCS Console URL when you log out of OMC.  It should have the format of https://idcs-<guid>.identity.oraclecloud.com

2.4. From IDCS

2.4.1. Go to Security > WAF Policies the click on the policy already created. Copy the Policy OCID.
2.4.2. Copy the CNAME Target 
2.4.3. Copy the domain name of the target application 

2.5. From Object Storage

2.5.1. Go to Object Storage and click on the Bucket you already created.  Copy the OCID
2.5.2. Copy the Bucket name
2.5.3. Copy the namespace
2.5.4. Click on the compartment and copy the compartment OCID
2.5.5. Copy the compartment name
 

3. FORWARD WAF LOG TO OCI BUCKET

In order to forward WAF Logs to your OCI Bucket your created previously, you should create an SR with Oracle Support. 

3.1. Set IAM Policy

The user created in step 2.1.3 must have write permission on the bucket.  To do so, we need to grant privileges on the group that contain this user.
3.1.1. Go to Administration > Identity > Policy and create the below policy statement: 
allow group <group_name_step_2.1.8> to manage object-family in compartment <compartment_name_step_2.5.5>
3.1.2. Copy the policy OCID

3.2. Raise a SR with Oracle Support

Once the policy is set, raise an SR on your Oracle WAF Portal support and provide the following information:
•    Domain name of the application (step 2.4.3), and additional domain name if applicable. 
•    Access Key (step 2.1.6) 
•    Secret Key (step 2.1.5)
•    WAF Policy OCID (step 2.4.1)
•    Bucket Name (step 2.5.2)
•    Bucket OCID (step 2.5.1)
•    Namespace (step 2.5.3)
•    Tenancy OCID (step 2.1.1)
•    Compartment OCID (step 2.5.4)
•    Policy OCID (step: 3.1.2)
•    Region identifier (step 2.1.2)
•    Bucket Region.
•    Upload Prefix: "%{+YYYY}/%{+MM}/%{+dd}/%{[log_type]}"

The implementation should take a few days before seeing the logs on your OCI Bucket.
Once completed, you should see logs arriving from WAF to your OCI Bucket:


4. SET UP OAUTH FOR OMC

Here, we are going to create a client application, that uses a token to connect into OMC.
This saves you from providing your username and password to authenticate the function.
By granting the OMC Admin role to the client application, the application will be able to upload logs to OMC LA.

4.1. Obtain OMC Access Token

Connect to IDCS and search to OMCEXTERNAL_<your_OMC_tenant> application and click on it.
Click on generate access token, a generate Token popup should open.
Select Customized Scope and Invoke IDCS APIs

Download the token.
This token will be used in the next steps.
 

4.2. Create an application for OAuth

4.2.1. Create a JSON file named newClientApp.json with the following content.
The name, displayName field can be customized, but note that the name must end with _APPID
Replace <OMC_URL> by the one in step 2.2.2.
{
  "name": "APPOMC_SERVICEAPI_APPID",
  "displayName": "APPOMC_SERVICEAPI",
  "description": "Test client for serviceapi",
  "isAliasApp": false,
  "active": true,
  "isOAuthClient": true,
  "clientType": "confidential",
  "allowedGrants": [
    "client_credentials"
  ],
  "allowedScopes": [
    {
      "fqs": "https://<OMC_URL>/serviceapi/"
    }
  ],
  "isOAuthResource": true,
  "accessTokenExpiry": 86400,
  "audience": "https://<OMC_URL>",
  "scopes": [
    {
      "value": "/serviceapi/"
    }
  ],
  "basedOnTemplate": {
    "value": "OPCAppTemplateId"
  },
  "serviceTypeVersion": "1.0",
  "serviceTypeURN": "OMCEXTERNAL",
  "schemas": [
    "urn:ietf:params:scim:schemas:oracle:idcs:App"
  ]
}


4.2.2. Run the following command to create the application:

Curl -X POST https://<IDCS_DOMAIN>/admin/v1/Apps -H ‘Content-Type: application/json’ -H “Authorization: Bearer <OAuth_Access_Token>” -d “@newClientApp.json”

<OAuth Access Token> is the format token value below you saved in the file of the Step 4.1:
{"app_access_token":"<OAuth Access Token>”}

Replace <IDCS DOMAIN> by the domain got on the step: 2.3. Domain should be like:
https://idcs-xxxxxxxxxxxxxxxxxxx.identity.oraclecloud.com
From the response, save the <client secret>, the <id> and the <name>, we will use them later.

4.3. Grant OMC Admin Role to Client App

On IDCS, click on Application, then your OMCEXTERNAL_<your_OMC_instance> instance
Click on Application Roles and assign the Application previously created to OMC Administrator

5. SET UP FUNCTION ENVIRONMENT

5.1. Prerequisites

Group and user we are going to use can be the same as the one created in step 2.1.3
5.1.1. Create a VCN and a subnet on your compartment. The VCN must egress via either NAT Gateway, Internet Gateway of Service
5.1.2. Create a policy in the root compartment with the following statements:
Allow service FAAS to use virtual-network-family in tenancy
Allow service FaaS to read repos in tenancy

5.1.3. As the user created in step 2.1.3 is not a tenancy administrator, so add the following statement:
Allow group <group-name> to manage repos in tenancy
Allow group <group-name> to read metrics in tenancy
Allow group <group-name> to read objectstorage-namespaces in tenancy
Allow group <group-name> to use virtual-network-family in tenancy
Allow group <group-name> to manage functions-family in tenancy
Allow group <group-name> to use cloud-shell in tenancy

Replace the <group-name> by the one on step 2.1.8

Note: If necessary, you can restrict these policy statements by compartment

5.2. Create a function

5.2.1. On OCI Console, go to Developer Services and click on Functions
5.2.2. Select your compartment, then click on “Create Application”, let’s name it load-waf-logs-app. Select the VCN and the subnet previously created on the prerequisite part then click on save.
5.2.3. Once the Application is created, click on it and follow the Getting Started steps on the left side of the page using the Cloud Shell Setup:
1.    Launch Cloud Shell
2.    Set up fn CLI on Cloud Shell 
3.    Update the context with the function’s compartment
4.    Update the context with the location of the Registry you want to use. 
5.    Already generated in obtain OAuth OMC token
6.    Log to the registry. Note: use the user created previously, and the OAuth token. 
7.    Verify your setup

Stop the Getting Started steps here, and continue with the following instruction:

5.2.4. Create a loadlogs python function by entering: 
fn init --runtime python loadlogs
       A directory called loadlogs is created with 3 files: func.py, func.yaml, requirements.txt 
5.2.5. Edit the file requirements.txt to contain these three lines:
fdk
requests
oci
5.2.6. Edit the file func.py and replace it with the ODU python code available on the resources.
5.2.7. Deploy the function by running:
fn -v deploy --app load-waf-logs-app

5.3. Create function parameters

Click on the application load-waf-logs-app, click on Configuration on the left menu then add the following parameters:

•    apiuser: is the application name chosen in the step 4.2 ending in _APPID
•    apipwd: is the Application client secret saved at the end of the step 4.2 
•    idcsurl: is the IDCS domain URL used at the end of the step 4.2 
•    input_bucket: is the bucket name from the step 2.5.2
•    logSourceName: must by WAF_LOGS since it’s the logSourceName used on the exported resources of this blog
•    omcurl: is the OMC URL from the browser URL bar in step 2.2.2
•    uploadName: use it to help isolate initial test uploads within Log Analytics but leave the parameter with an empty value or remove it when going to production

Note: idcsurl and omcurl values should NOT include a trailing "/"


 

5.4. Create a Dynamic group for the Function

In order to use other OCI Services, the function must be part of a dynamic group. To do so, from OCI console, go to Identity > Dynamic Group and create a new dynamic group fn_oci for example.
On the matching rules, add the following statement by using the correct compartment OCID from step 2.5.4:
ALL {resource.type = 'fnfunc', resource.compartment.id = 'ocid1.compartment.oc1..aaaaaxxxxx'}

6. SET UP EVENT RULE

6.1.  Bucket Storage configuration

6.1.1. Go to your Object Storage bucket where WAF Logs are stored, and enable Emit Object Events options:

6.2. Create IAM Policy

The dynamic group previously created need to manage objects within your tenancy 
To do so, go to identity > Policies and add the following statement:

allow dynamic-group <name_choosen_in_step_5.4> to manage objects in tenancy

Note: If necessary, you can restrict these policy statements by compartment.

6.3. Create an event rule

From your OCI console menu, go to Application Integration then Events Service.
Created a new rule as following:
Event Matching:

  • Event types
    • Service Name: Object Storage
    • Event Type: Object – Create
  • Attributes:
    • bucketName: bucket name from the step 2.5.2

 
Actions:
-    Function application need to be called.
  


7. CONFIGURE OMC 

7.1. Import Log Parsers  

Download the Log Parsers export from the resources. Each Parser is a .zip file which contains the content.xml
From OMC menu, go to Log Analytics then click on Administration home. 
Click on the gear icon on the top right corner then click on import configuration content.
Select your parsers.

7.2. Import Log sources 

Download the Log Source export from the resources. and select the .zip file which contains the content.xml 
From OMC menu, go to Log Analytics then click on Administration home. 
Click on the gear icon on the top right corner then click on import configuration content.
Select the .zip file.


7.3. Import dashboard

Download the Dashboard json export from the resources.
Launch the following curl command to import the dashboard into your OMC instance:
curl -X PUT https://xxxx.omc.ocp.oraclecloud.com//serviceapi/dashboards.service/import -H 'cache-control: no-cache' -H 'Content-type: application/json' -u 'omc_username' --data @/path/to/exported/dashboard.json -o /tmp/import_output
Use your OMC credentials.

8. RESOURCES

9. FINAL RESULT

The configuration is now completed. New log files arriving on the Object Storage Bucket will be uploaded to OMC under WAF_LOGS log source and start populating   OCI WAF dashboard as below: 

  • A Global Overview summarizing the global access requests Map, by Country, by URL, by Application by Code Error, by Code Error by Time. it gives also insight on Security Rules triggered, by IP Address and by Time.   


  • A second tab giving more insights on the OWASP Top 10 threats: 

  • A third tab with details on all Threat Intelligence Feeds detection

  • Another tab giving all details about detections based on Access Rules

 

  • Finally, a tab with details on all threats detected and blocked by the Javascript Challenge feature

I hope you found this blog helpful. Here are a couple of next steps to help you get started: 

  • If you don't have an Oracle Cloud Infrastructure account, try our Free Tier.
  • To try the dashboard, download the json export from the resources.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.