If you don't try limit monitoring with Oracle Functions, you'll hate yourself later

November 4, 2021 | 11 minute read
Flavius Dinu
DevOps Lead
Text Size 100%:

Co-authored by Gabriel Feodorov.

Checking for every soft limit in your Oracle Cloud Infrastructure (OCI) tenancy can be a pain, but this information gives you better control of your resources. Imagine being subscribed on over 20 regions and you need to know where you have more than 50% available resources for virtual cloud network (VCN) creation. How would you go about that?

The process involves going to the limits, quotas, and usage, then selecting the VCN service, cycling through all your existing compartments, taking notes, and repeating this process 20 times. That’s for one limit only. What if you do this for 100 soft limits on all 20 regions? It can take you several days or even weeks to perform.

In this post, we cover an easier solution that checks for all soft limits, based on a threshold in all regions, and sends you a notification by email. This solution involves the following services:

  • Terraform

  • Python

  • Oracle Functions

  • Events

  • Notifications

An OCI Object Storage bucket is created in tenancy that contains a dummy file. Based on a lifecycle policy, the file is deleted in a number of days that you set. When a deletion occurs in that Object Storage bucket, Oracle Functions is triggered by an event, and they gather all the limits that go over the threshold. These limits are sent to a notification topic, and that topic publishes the data to all its subscribers. In our example, we use an email address, but you can have multiple email addresses, slack channels, SMS, and more. Finally, the functions create an object in the bucket, resetting the cycle.

Prerequisites

First, download the code from GitHub. You also need the following components

  • Compartment

  • Network (VCN, subnet, IG, and NAT)

  • Oracle FN

  • Docker

  • A dynamic group for the functions

  • A policy that lets Object Storage manage buckets

  • A policy that permits the dynamic group manages resources inside your tenancy

Create the dynamic group and the policies

In the OCI Console, go to Identity & Security. Under Identity, click Dynamic Groups.

Follow the instructions to create a dynamic group and give the dynamic group a name. Our example uses fn-dg. Add the following rule in fn-dg:

ALL {resource.type = 'fnfunc', resource.compartment.id = 'ocid1.compartment.oc1..aaaaaaaa23______smwa'}

The resource.compartment.id is the compartment in which you deploy the functions.

In the OCI Console, go to Identity & Security. Under Identity, click Policies and create the following policy with these rules:

Allow service objectstorage-us-ashburn-1 to manage object-family in Tenancy (where us-ashburn-1 should be your home region)

Allow dynamic-group fn-dg to manage resources in Tenancy

Install Docker

To install docker, follow these steps.

Install Fn Project

Before installing Fn project, ensure that you have Docker running. Go to your terminal and run the following command:

  • MacOS: brew update && brew install fn

  • Linux and MacOS: curl -LSs https://raw.githubusercontent.com/fnproject/cli/master/install | sh

For more information, see Install Fn.

Create an auth token

To create an auth token, go to your IAM user, select the Auth Tokens tab, and click Generate Token, as shown in the following screenshot:

A screenshot of the Auth Token page with the Generate Token button outlined in yellow.

The auth token shows up only once, so save it. This token is the password used for logging in to the Docker registry.

Install Terraform

Go to terraform.io and download the package for your operating system and architecture. Terraform is distributed as a single binary. Install Terraform by unzipping it and moving it to a directory included in your system’s path. You need the latest version available.

Running the Terraform Code

Now, we go to the downloaded code in the Terraform directory. The Terraform code is used to prepare the scheduling part of the function and it deploys the following objects:

  • One Object Storage bucket

  • One bucket lifecycle policy

  • One notification topic (The function publishes messages to this topic, if the limits are lower than the threshold)

  • One Notifications subscription

The Object Storage bucket has a lifecycle policy that deletes all objects after seven days. Based on this policy, an event triggers the function.

The first time, you need to run the function manually or delete an object inside this bucket. At the end of a run, the function creates an object inside the bucket. Based on the delete policy we created, it runs weekly.

Before diving into the code, you need to solve some dependencies. To prepare the provider.auto.tfvars file, you need the following items:

  • Tenancy OCID

  • User OCID

  • Local path to your private OCI API key

  • Fingerprint of your public OCI API key

  • Region

Get the tenancy and user OCIDs

Log in to the Console using your credentials (tenancy name, user name, and password). If you don’t know them, contact a tenancy administrator. To obtain the tenancy OCID, after logging in, from the menu, select Administration and then Tenancy Details. Find the tenancy OCID under Tenancy Information. It follows the naming convention “**ocid1..."

To get the user OCID, after logging in, from the menu, select Identity and then Users. Find and click your user. Have this page open for uploading the oci_api_public_key. From this page, you can get the user OCID, which follows the naming convention “**ocid1...”

Create the OCI API key pair and upload it to your user page

Create an oci_api_key pair to authenticate to OCI, as specified in the documentation. Create the .oci directory in the home of the current user with the following command:

$ mkdir ~/.oci

Generate the OCI API private key.

$ openssl genrsa -out ~/.oci/oci_api_key.pem 2048

Ensure that only the current user can access this key with the following command:

$ chmod go-rwx ~/.oci/oci_api_key.pem

Generate the OCI API public key from the private key.

$ openssl rsa -pubout -in ~/.oci/oci_api_key.pem -out ~/.oci/oci_api_key_public.pem

To make API calls, upload the public key to the OCI Console for your user. Go to your user page and select API Keys. Click Add Public Key and paste the contents there.

After uploading the public key, you can see its fingerprint in the Console. You need that fingerprint for your provider.auto.tfvars file. You can also get the fingerprint by running the following command on your local workstation by using your newly generated OCI API private key.

$ openssl rsa -pubout -outform DER -in ~/.oci/oci_api_key.pem | openssl md5 -c

Getting the region

Although you might know your region name, you need its identifier for the provider.auto.tfvars file. For example, US East Ashburn has us-ashburn-1 as its identifier. To obtain your region identifier, in the OCI Console, navigate to Administration and click Region Management. Select the region that you’re interested in and save the region identifier.

Prepare the provider.auto.tfvars file

You need to modify the provider.auto.tfvars file to reflect the values that you’ve captured. The provider_oci map has the following keys:

  • Tenancy: Add your tenancy_id as the value.

  • user_id: Add your user_id as the value.

  • Fingerprint: Add the fingerprint of the public oci_api_key_public.pem that you generated and uploaded to your user in the Console.

  • key_file_path: Add the local path to your private oci_api_key.pem key.

  • Region: Add the region identifier.

Your provider.auto.tfvars looks like the following output:

provider_oci = {
tenancy = "ocid1...."
user_id = "ocid1...."
fingerprint = "45:87:.."
key_file_path = "/root/.oci/oci_api_key.pem"
region = "us-ashburn-1"
}

Dependencies

Apart from the provider.auto.tfvars, you need to address some other dependencies in the terraform.tfvars file.

Compartment dependency

To run the sample code, you need to prepare a map variable, like in the terraform.tfvars file, called compartment_ids, which holds key or value pairs with the names and IDs of the compartments that you want to use. This variable is external to offer multiple ways of linking them from a Terraform perspective.

compartment_ids = {

  sandbox = "ocid1...."

}

Application dependency

subnet_ids = {

  hur1pub  = "ocid1.subnet.oc1.iad.aaa"

}

app_params = {

  thunder_app = {

    compartment_name = "sandbox"

    subnet_name      = ["hur1pub"]

    display_name     = "thunder_app"

    config = {

      "MY_FUNCTION_CONFIG" : "ConfVal"

    }

    freeform_tags = {

    }

  }

}

The application needs a subnet to reside in. Prepare the map variable subnet_ids with the name and the subnet OCID of your choosing, and specify that name inside the variable app_paramsunder subnet_name.

Subscription dependency

subscription1 = {

  comp_name  = "sandbox"

  endpoint   = "test.test@oracle.com"

  protocol   = "EMAIL"

  topic_name = "topic1"

}

For the endpoint, add the email that receives the limits notifications from the function.

Terraform parameters explained

Application parameters

  • compartment_name: The compartment name in which the event is created

  • subnet_name: Name of the subnet or subnets in which the functions run in the application

  • display_name: Name of the application

  • config: Values that are passed onto the function as environment variables

Bucket parameters

  • compartment_name: The compartment name in which the bucket is created

  • name: The name of the bucket

  • access_type: The type of public access on this bucket

  • storage_tier: The type of storage tier on this bucket

  • events_enabled: Whether events are emitted for object state changes in this bucket

  • kms_key_name: The name of the KMS master key used for encrypting the bucket. An empty string means that the KMS is not used and the bucket is encrypted using Oracle-managed keys

Object lifecycle policy parameters

  • bucket_name: The name of the bucket to which the lifecycle policy is applied

  • rule_name: The name of the policy

  • is_enabled: Whether the policy is enabled

  • action: The type of action for this policy

  • time_amount: Specifies the age of objects to apply the rule to. It is interpreted in units defined by the time_unit parameter.

  • time_unit: The unit used to interpret time_amount parameter

Topic parameters

  • comp_name: The compartment name in which the topic is created

  • topic_name: The name of the topic

  • description: The description of the topic

Subscription parameters

  • comp_name: The compartment name in which the subscription is created

  • endpoint: A locator that corresponds to the subscription protocol

  • protocol: The protocol used for the subscription

  • topic_name: The name of the topic the subscription is applied to

To run the Terraform code, go to the Terraform directory with the following commands:

$ terraform init

$ terraform plan

$ terraform apply

Save the topic_id, app_name and bucket_name exported as outputs from Terraform for the next step. After modifying the terraform.tfvars file and running the Terraform code, you receive an email regarding the notification subscription. Accept to receive emails with the reports.

Deploying the Oracle functions

Now, go to downloaded code in the serverless deployment directory. The deployment.py utility deploys one function per region (only the subscribed regions), a main function that runs all the other functions, and an event that triggers the main function.

All the functions and event are deployed in the home region, and limits are checked for all the subscribed regions.

The script that creates the limit functions for all regions has a few args that you provide.

$ python3.9 deployment.py --help

usage: deployment.py [-h] -user USER -password PASSWORD -compartment_id COMPARTMENT_ID -app_name APP_NAME -topic_id TOPIC_ID -percentage PERCENTAGE -bucket_name BUCKET_NAME -fn_prefix FN_PREFIX

You have the following optional arguments:

  • -h, --help: Show the help message and exit

  • -user USER: The user used for connecting to the docker registry. tenancy_namespace\user_email or tenancy_namespace\federation_client\user_email

  • -password: The auth token value used for logging in to the docker registry

  • -compartment_id COMPARTMENT_ID: The comp id in which the functions are created

  • -app_name APP_NAME: The name of the app in which the functions are created

  • -topic_id TOPIC_ID: The id of the topic used for publishing limit messages

  • -percentage PERCENTAGE: The threshold percentage

  • -bucket_name BUCKET_NAME: The name of the bucket used by the main function

  • -fn_prefix FN_PREFIX: The prefix you want to use for your function names

The user and password args are used for connecting to the docker registry.

Precede the user with the tenancy_namespace. If your user is federated, it also needs the federation client in the name.

Using Oracle Identity Cloud service, a tenancy with the namespace myawesometenancy and a user called myawesomeuser produces the following name:

myawesometenancy\oracleidentitycloudservice\myawesomeuser

  • Password: The created auth token

  • compartment_id: the ID of the compartment in which the functions are created

  • app_name: Name of the application in which the functions are created

  • topic_id: The ID of the topic used for publishing the limits

  • Percentage: The threshold of the limit checks

  • bucket_name: The name of the bucket used for scheduling the functions

  • fn_prefix: The prefix of the functions

The following code block shows an example deployment:

python3.9 deployment.py -user  myawesometenancy\oracleidentitycloudservice\myawesomeuser -password '' -compartment_id ocid1.compartment.oc1.. -app_name test2 -topic_id ocid1.onstopic.oc1.iad. -percentage 90 -bucket_name name_of_the_bucket -fn_prefix name_prefix_of_functions

Results

After the code runs successfully, you receive an email per subscribed region with the data for that region. The email looks similar to the following example:

A screenshot of the example email.

The threshold is for total used, so by setting a threshold to 90%, you receive details for all the resources that have less or equal to 10% available.

Conclusion

Now, you have a working solution that sends you limits data on a weekly basis. You can easily modify the lifecycle retention policy in the bucket by modifying the Terraform code as specified. You can easily trigger the functions by deleting the object from the bucket.

If you want to learn more about Oracle Functions, check out this blog post.

Flavius Dinu

DevOps Lead


Previous Post

Announcing Oracle Cloud Infrastructure AI services

Wes Prichard | 5 min read

Next Post


Free IT certifications for Oracle Cloud Infrastructure

Mike Chen | 4 min read