Usage Insights for Oracle Analytics Cloud using OCI Logging

May 24, 2024 | 8 minute read
Srinivasa Pula
Consulting Member of Technical Staff
Text Size 100%:

This article describes how to enable Oracle Cloud Infrastructure (OCI) service logs so you can track events and activities in Oracle Analytics Cloud (OAC). The article includes Python scripts that enable you to extract and transform OCI logging data, for seamless integration into Oracle Autonomous Data Warehouse (ADW) for further analysis. It also provides a prebuilt data visualization project (DVA) that offers sample canvases and visualizations where you can derive actionable insights and make informed decisions based on OAC usage.

OCI Service Logs for Oracle Analytics Cloud

Oracle Cloud Infrastructure (OCI) offers service logs to track diagnostics information and audit events in Oracle Analytics Cloud. Two categories of logs are available for Oracle Analytics Cloud:

  • Diagnostic logs: Diagnostic logs capture information about data flow, dataset, sequence, and query events. For example, job status, job completion times, and information about failed jobs to help you monitor and troubleshoot failed jobs.
  • Audit logs: Audit logs track create, update, delete, and permission change operations on catalog objects. Catalog objects include classic analyses and dashboards, workbooks, pixel-perfect reports, folders, datasets, self-service connections, data flows, sequences, and more. Audit logs record user exports for workbooks, classic analyses, and dashboards. Audit logs help to maintain security and compliance by keeping track of changes made to the system and its data.

Additional features of OCI logging:

  • Log Retention: Ensures that logs are retained for a specified period. The default retention period is 1 month and the maximum allowed period is 6 months.

By leveraging OCI logging capabilities, you can gain deeper insights into the operations and security of your Oracle Analytics Cloud environment.

Required IAM Policies

You need several OCI IAM policies to set up and monitor audit and diagnostic logs:

  • Enable Service Logs and Create Log Groups

Example: Let users in the analytics_admin group enable logging for Oracle Analytics Cloud instances and set up log groups.
allow group analytics_admins to manage log-groups in compartment myOACProduction

  • Read Oracle Analytics Cloud Logs

Example: Let users in the analytics_users group read logs for Oracle Analytics Cloud instances.

allow group analytics_users to read log-content in compartment myOACProduction

For details, see Monitor Usage and Diagnostic Logs.

Enable Service Logs for Oracle Analytics Cloud

  1. Log into the OCI console and create a log group under the desired compartment.
  2. Click Enable service log to enable the logs for Oracle Analytics Cloud.

        Enable service logs   

  1. Enter “Analytics Cloud” in search services and click Manage Logs.

Manage Logs

  1. Enable both Audit and Diagnostics logs for your Oracle Analytics Cloud instance.

diag and audit logs

  1. Select the log group you created, provide a log name for both the Audit and Diagnostics logs, and click Enable.

log group

For more details, see Enabling Logging for a Resource.

Solution Architecture
Solution Architecture

Prerequisites

  1. Configure the OCI CLI.

Install OCI CLI and configure to download the diagnostic and audit logs. Follow the instructions for your operating system. See  Installing the CLI.

  1. Verify your existing Python installation (if any). If required, install Python 3.6 or higher. 
    To verify your Python installation:   
    python3 –version
  2. Install cx_Oracle module:
    sudo python3 -m pip install cx_Oracle
  3. Install Oracle Instant Client, based on your operating system.  For example:

    Linux x86-64 (64-bit)   - 
    Oracle Instant Client Downloads for Linux x86-64 (64-bit)

Steps

1: Download the Oracle ADW wallet directory and unzip the files under: instant_client_dir/network/admin

2: Update the  sqlnet.ora file under instant_client_dir/network/admin and change the wallet location.

 network admin

3: Set the TNS_ADMIN environment variable. For example, for Linux 7 or Linux 8. 

            nano  ~/.bashrc  or  vi ~/.bashrc

Add the line:

     export  TNS_ADMIN=/tmp/instantclient_21_13/network/admin/

 Run the following command after the updated bashrc profile:
           
source ~/.bashrc

4: Download the text file Create Tables_DDL.sql and create the required tables in Oracle ADW.

create sql

5: Create the following directories.

mkdir scripts

mkdir data

mkdir failed_load_data

mkdir logs

6: Download oac_python_scripts.zip and unzip the sample Python scripts under the scripts folder.

scripts

7: Locate  config.ini (in the scripts folder) and update the load_type based on the load period. For initial load, enter load_type=full_load and for incremental load, enter load_type= incremental.

Load type

8: Update the config.ini file with the directory locations you created in step 5.

9: Update the config.ini file with your OCI logging compartment OCID, log group OCID, diagnostics log OCID, and audit log OCID.

   OCi Log details

10: Enter the details in config.ini for initial load, that is, start_year, start_month,start_day, end_year,end_month, and end_day under [FULL_LOAD]. If you hit the error message "User-rate limit exceeded", increase load_interval_minutes to a higher value. For example, load_interval_minutes=1440 (one day). Note: Enter the interval in minutes. Also, try downloading the logs for a shorter time range, that is, days between start_day and end_day.

 11. After updating config.ini, run data_load_refresh.py to load the logs for the selected period.
python3 data_load_refresh.py

12: For incremental load and scheduling script  for every 15 minutes, you can use a cron job:

*/15 * * * *  cd /tmp/scripts && python3 data_load_refresh.py

13: Download the sample DVA project Insights of Usage Tracking and Audit Summary.dva and import the DVA project into your Oracle Analytics Cloud.

DVA project

14: In Oracle Analytics Cloud, update the self-service data connection (for example, Usage_Audit_Monitoring) with your ADW credentials.

DV Connection

15: Importing the DVA project creates several required datasets (DATAFLOW_SUMMARY_DATA, PHYSICAL_QUERY_LOG_DATA, AUIDT_LOG_DATA). Update these datasets with your tables by editing the datasets used in "Insights of Usage Tracking and Audit Summary".

Update tables

   Troubleshooting Tips

  • Error:  NotAuthorizedOrNotFound
    Not authenticated

Solution: Check the OCI configuration file. Ensure that you added the correct region, user OCID, private key path, and fingerprint. Also, make sure that you added the correct compartment OCID, log group OCID, and log OCID in config.ini.

  • Error: User rate limit exceeded

Rate limit error

Solution: This error occurs when you download logs in full load for a long period which sends too many OPC requests. To resolve this issue, either increase load_interval_minutes  in config.ini  to a higher value (for example load_interval_minutes=1440) or download the logs for a smaller time range (for example, the date range between end_day and start_day must be less).

  • Error: No module named ‘cx_Oracle’

    cx_module error

Solution:  The cx_Oracle module is missing. Install the module using: sudo pip3 install cx_Oracle

  • Error: DPI-1047: Cannot locate a 64-bit Oracle Client library: "libnnz21.so
    instant client lib error

Solution:  In the config.ini file, check that the instant client library path and directory path where these libraries are located are both correct.

       Call to action

Enabling OCI logs for Oracle Analytics Cloud is highly beneficial as you can closely monitor the usage of Oracle Analytics Cloud. By leveraging audit, service, and custom logs, you can enhance security, ensure compliance, and optimize the performance of your OAC environment. Regular monitoring and proactive management of logs will help you maintain a robust and efficient analytics platform
To learn more about Oracle Analytics, visit oracle.com/analytics, follow us on twitter@OracleAnalytics, and connect with us on LinkedIn.

Srinivasa Pula

Consulting Member of Technical Staff

Oracle Analytics Service Excellence, CEAL Team


Previous Post

A Pepsi Challenge for Analytics & BI? Gartner’s 2024 Analytics & BI Bake-Off

James Richardson | 4 min read

Next Post


Disaster Recovery of Oracle Analytics Server on Oracle Cloud using OCI Full Stack Disaster Recovery