How to get a list of all free-form tags in your tenancy

June 5, 2024 | 6 minute read
Vinay Kalra
Principal Cloud Solution Architect
Text Size 100%:

Introduction

I came across a question from a customer that I was unsure of; so I decided to investigate. The customer asked how to get a list of free form tags being used in a tenancy?  As a refresher,  there are two types of tags; defined tags and free form tags.  The very nature of free form tags is that anyone can add a key/value pair for a resource.  For a mature tenancy there can be many tags created and the customer was interested in a clean up effort. This post describes the methods I found to obtain a list of free form tags.

 

Details

Currently there's no direct way to list all the free form tags being used via the console, however, the following three methods below will detail how you can generate a list of free form tags.  Taking a step further, I wanted to find out how many times a tag was being used.

 

The first method is to download the cost and usage report from the console. To do this you must first be in your home region.  Then from the hamburger menu select 'Billing and Cost Management'-> 'Cost and Usage Report' (Under Cost Management).  Here you will see a list of compressed (.csv.gz) files that you can uncompress/open in 'Excel'.  The file contains columns of all the freeform tags used with the name 'tags/<Key>'.  Within each column a value indicates when the tag was used in a resource (row). 

 
The data in these columns maybe sparse but any Excel expert (not me) can consolidate the data into an more readable list and provide a count.
 

The second method is using the OCI Command Line Interface (CLI). Here's a command that you can use to search your tenancy and get a print out of all non-null tags:

oci --profile <ociConfig profile name>  search resource structured-search --query-text "QUERY all resources where lifeCycleState != 'TERMINATED' && lifeCycleState != 'FAILED'" --query 'data.items[?"freeform-tags" != `{}`].{name:"display-name",resource:"resource-type",tagValue:"freeform-tags"}'

 

The example above will query (structured-search --query-text ) all resources that are not "TERMINATED" and where the current state is not "FAILED".  The '--query' statement will modify the output to return only three attributes and only if the value for the free form tag is not null.

Sample output:

[
{
    "name": "oke-1",
    "resource": "PrivateIp",
    "tagValue": {
      "OKEnodePoolName": "pool1"
    }
  },
  {
    "name": "oke-2",
    "resource": "Vnic",
    "tagValue": {
      "OKEnodePoolName": "pool1"
    }
  },
  {
    "name": "oke-3",
    "resource": "PrivateIp",
    "tagValue": {
      "OKEnodePoolName": "pool1"
    }
  }

A couple of important points here;  the OCI CLI supports pagination.  If the result set is more than 500 entries you can obtain the next set of entries by using the '--page <text>'˘ flag in subsequent calls.  The <text> needed will be in the 'opc_next_page' attribute from the previous call. 
 
NOTE:  The 'opc_next_page' attribute does not appear when using the '--query' flag. Removing this flag will give you all the attributes for every entry returned with the ;opc_next_page' attribute at the end of the output.  Another option is to use the '--all' flag to return all the data in one call.
 
 
The third method is using the REST API.  The OCI CLI is basically a wrapper around REST API calls.  You can determine the exact REST call by adding a '--debug' flag to the call above. 
 
For example:
oci --profile <ociConfig profile name> --debug search resource structured-search --query-text "QUERY all resources where lifeCycleState != 'TERMINATED' && lifeCycleState != 'FAILED'" --query 'data.items[?"freeform-tags" != `{}`].{name:"display-name",resource:"resource-type",tagValue:"freeform-tags"}'
 
Before the actual JSON output you will see a series of DEBUG statements as follows:

DEBUG:oci_cli.cli_metrics: 2024-06-06 00:18:36.635722: Metrics is not enabled
macOS-14.5-x86_64-i386-64bit
System name: Darwin
System release : 23.5.0
System version: Darwin Kernel Version 23.5.0: Wed May  1 20:09:52 PDT 2024; root:xnu-10063.121.3~5/RELEASE_X86_64

env OCI_PYTHON_SDK_NO_SERVICE_IMPORTS is set
DEBUG:oci_cli.cli_util:Config File: dict_keys(['log_requests', 'additional_user_agent', 'pass_phrase', 'user', 'fingerprint', 'key_file', 'tenancy', 'region'])
DEBUG:oci_cli.cli_util:region: Environment Variable or Parameter
DEBUG:oci.base_client.4368624832:Endpoint: https://query.us-ashburn-1.oci.oraclecloud.com/20180409
INFO:oci.base_client.4368624832: 2024-06-06 00:18:36.698118: Request: POST https://query.us-ashburn-1.oci.oraclecloud.com/20180409/resources
Not using Expect header...
send: b'POST /20180409/resources HTTP/1.1\r\nuser-agent: Oracle-PythonSDK/2.126.4 (python 3.12.3; x86_64-Darwin) Oracle-PythonCLI/3.41.0\r\naccept-encoding: gzip, deflate\r\naccept: application/json\r\nconnection: keep-alive\r\ncontent-type: application/json\r\nopc-request-id: EB43.......\r\nopc-client-retries: true\r\nopc-client-info: Oracle-PythonSDK/2.126.4\r\nContent-Length: 121\r\ndate: Thu, 06 Jun 2024 00:18:36 GMT\r\nhost: query.us-ashburn-1.oci.oraclecloud.com\r\nx-content-sha256: ....=\r\nauthorization: Signature algorithm="rsa-sha256",headers="date (request-target) host content-length content-type x-content-sha256",keyId="ocid1.tenancy.oc1..aaaaaa.../ocid1.user.oc1....../:.::::",signature="b.....==",version="1"\r\n\r\n{"query": "QUERY all resources where lifeCycleState != \'TERMINATED\' && lifeCycleState != \'FAILED\'", "type": "Structured"}'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Thu, 06 Jun 2024 00:18:37 GMT
header: opc-request-id: EB43.....
header: opc-next-page: eyJh......
header: Content-Type: application/json
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31536000; includeSubDomains;
header: Transfer-Encoding: chunked
DEBUG:oci.base_client.4368624832: 2024-06-06 00:18:39.339216: time elapsed for request EB43....: 2.640909961000034
DEBUG:oci.base_client.4368624832: 2024-06-06 00:18:39.339395: time elapsed in response: 0:00:02.319587
DEBUG:oci.base_client.4368624832: 2024-06-06 00:18:39.339501: Response status: 200
DEBUG:oci.base_client.4368624832: 2024-06-06 00:18:39.365416: python SDK time elapsed for deserializing: 0.022091594999437802
DEBUG:oci.base_client.4368624832: 2024-06-06 00:18:39.365830: Response returned
DEBUG:oci.base_client.4368624832:time elapsed for request: 2.667724425000415
DEBUG:oci_cli.cli_util: 2024-06-06 00:18:39.375956: time elapsed calling to_dict from render: 0.00996785200004524
DEBUG:oci_cli.cli_util: 2024-06-06 00:18:39.378485: time elapsed evaluating expression: 0.0021094949997859658
DEBUG:oci_cli.cli_util: 2024-06-06 00:18:39.380999: total memory usage before printing: 60.4MiB (63348736)

  • The highlighted statement above is the actual REST API call the CLI invoked to do a search for all resources with the desired POST data as shown here:
 

INFO:oci.base_client.4368624832: 2024-06-06 00:18:36.698118: Request: POST https://query.us-ashburn-1.oci.oraclecloud.com/20180409/resources

I will not be getting into details on how to invoke the REST APIs. There are plenty of examples/posts on this subject using Postman or python to make REST calls.

 

Now that I have the json file, I want to capture the number of times each key/'value pairs exists.  This will give me a good idea on how many tags I have as well as the count for each.

 

To accomplish this I wrote a simple python script to parse the JSON data and list out all tags with their count.  The script inputs the JSON file and the name of the attribute.  In my example above I specified the name of the tag attribute as 'tagValue'

 

I will leave it to the reader to write the full script but here is the main snippet to parse the JSON file and print the output:

 
# Check if the data is a list (array)
        if isinstance(data, list):
            attribute_counts = defaultdict(int)
 
            for i, item in enumerate(data):
                if isinstance(item, dict) and attribute in item:
                    attr_value = item[attribute]
                    if isinstance(attr_value, dict) and len(attr_value) > 0:
                        # Convert the dictionary to a tuple of sorted items to handle duplicates
                        sorted_attr_value = tuple(sorted(attr_value.items()))
                        attribute_counts[sorted_attr_value] += 1
                    else:
                        print(f"Item {i}: Attribute '{attribute}' is not a non-empty JSON object.")
                else:
                    print(f"Item {i}: Attribute '{attribute}' not found.")
 
            # Print out the counts of unique key-value pairs
            for attr_value, count in attribute_counts.items():
                print(f"{dict(attr_value)}: {count} times")
        else:
            print("The JSON file does not contain an array at the top level.")
 
 
The output will look something like:

{'key1': 'value1'}: 10 times
{'key1': 'value2'}: 13 times
{'key2': 'value1'}: 144 times

And there you have it, a list of tags with their count.

Thanks for reading!
 
 

Vinay Kalra

Principal Cloud Solution Architect

Vinay Kalra is a 'Principal Cloud Solution Architect' of the North America Cloud Technology and Engineering Team. Vinay's area of expertise is in Identity and Access Management (IAM) and Security for Oracle Cloud Infrastrucure (OCI).  Vinay has been in this space as a Software Engineer/Architect for over 20 years,


Previous Post

Automating KMS Key Rotation for Enhanced Volume Security

Ramesh Balajepalli | 5 min read

Next Post


Integrating a SIEM solution with Oracle Cloud Applications using audits

Mani Krishnan | 3 min read