Clean up old OCI GoldenGate trails using Python and the OCI CLI

October 27, 2023 | 2 minute read
Eloi Lopes
Cloud Engineer
Text Size 100%:

Oracle Cloud Infrastructure (OCI) GoldenGate is a fully managed service providing a real-time data mesh platform, which uses replication to keep data highly available and enable real-time analysis.

Each OCI GoldenGate deployment displays the list of its trail files in the deployment details page:
GoldenGate trails in OCI Console

It is common to accumulate unused trail files, especially in development environments where many tests are conducted. An easy way to clean up all of these trails is by using the following Python code. The code below assumes that you have the OCI CLI installed. If you haven't already installed OCI CLI, please refer to the references section for instructions.

### Import Packages
import oci
import requests
import json
from base64 import b64encode
import base64


# Auth Config
CONFIG_PROFILE = "DEFAULT"
config = oci.config.from_file('~/.oci/config', CONFIG_PROFILE)

#OCI GoldenGate API
version ="v2"
username = '<gg username>'
password = '<password>'
issueCommand=f'/services/{version}/commands/execute'

#OCI GoldenGate deployment name
sourceUrl = "https://<OCI GG URL>"

encoded_credentials = b64encode(bytes(f'{username}:{password}',
                                encoding='ascii')).decode('ascii')

auth_header = f'Basic {encoded_credentials}'
payload=""
header = f'{auth_header}'
headers = {
'Authorization':  header,
'Cookie': ''
}

#initilize service client
golden_gate_client = oci.golden_gate.GoldenGateClient(config)

#function to delete trail
def deleteTrail(inputHearders, inputBody,url):
    responsePost = requests.post(url+issueCommand, headers=inputHearders, json=inputBody)
    print (responsePost.json())

list_trail_files_response = golden_gate_client.list_trail_files(
    deployment_id="<OCI ID for OCI GG deployment>")

#data dictionary for trail files
data_dict = json.loads(str(list_trail_files_response.data.items))

#iterate through trail files
for i in data_dict:
#Only for trails that are not currently in use by any extract or replicat.
    if i['consumers'] == None and i['producer'] == None:
       #print(i['consumers'], i['producer'], i['trail_file_id'])
        trailsDict={
         "name": "purge",
         "purgeType": "trails",
         "trails": [ { "name": i['trail_file_id'] } ],
         "useCheckpoints": False,
         "keep": [ {
         "type": "min",
         "units": "files",
         "value": 0}]
        }
        #delete trail
        deleteTrail(headers,trailsDict,sourceUrl)

The code above only deletes trails that are not in use by extracts and replicats. The code supplied with this article comes with no support, so use at your own risk.

References

Eloi Lopes

Cloud Engineer

Eloi is a Cloud Solution Engineer for Data Integration and Analytics. He has experience in developing ELT / ETL projects and delivering analytics solutions to multiple customers across different industries. He is an eager learner and will keep improving his skills with main focus on real time data solutions.


Previous Post

OCI Database Migration service announces OCI GoldenGate integration (now available)

Jorge Martinez | 2 min read

Next Post


Oracle GoldenGate for Big Data Articles get a new look!

Anuradha Chepuri | 3 min read