The StatsD integration with Telegraf involves using Telegraf as an aggregator and processor of StatsD metrics before forwarding them to a time-series database such as Autonomous Database, InfluxDB or any other output supported by Telegraf.
Oracle GoldenGate 23ai now includes StatsD formatted metrics out of the box, with 100+ metrics, which can be activated during deployment installation or later via the API.
curl -svu username:password \
-X PATCH http://hostname:port/services/v2/deployments/deployment_name \
--data '{
"metrics":
{
"enabled":true,
"servers":
[
{
"type":"pmsrvr",
"protocol":"uds"
},
{
"type":"statsd",
"host":"<statsd_host>"
}
]
}
}'
StatsD is a network daemon that listens for statistics, like counters and timers, sent over UDP or TCP and sends aggregates to one or more pluggable backend services (e.g., Autonomous Database, Graphite, InfluxDB, Prometheus, and TimescaleDB). It was originally developed by Etsy and is designed to improve the performance of tracking events, metrics, or status data points in real-time.
Telegraf on the other hand is a server-based agent developed by InfluxData, designed for collecting, processing, aggregating, and writing metrics from a variety of sources into a time series database. It is an open-source tool that is part of the InfluxData platform. Telegraf supports a plugin-driven architecture which allows it to integrate with a multitude of systems, services, and API data formats.
The GoldenGate 23ai integration of StatsD and Telegraf, means you can configure Telegraf to receive the GoldenGate 23ai metrics sent using the StatsD protocol. GoldenGate 23ai sends metrics over UDP because it's lightweight and non-blocking to Telegraf which then inserts into a storage location, typically a time series database for permanent storage.
You can then consume the real-time and historical metrics from the time series database to build customized dashboards. This will give you the ability to visualize the metrics from all deployments in a single pane of glass.
StatsD Protocol Support:
Configuration:
[[inputs.statsd]]
interval = "5s"
Metrics Handling:
Output:
Benefits:
To make it simpler I built with a series of containers. I created one container for each service, but you can install Telegraf on GoldenGate host or somewhere else. In this case I created a dedicated host for Telegraf and GoldenGate pushes the metrics to that host.
Note: The metrics push for the databases are for illustration only and is not part of the demonstration.
$cd to directory copied from GitHub
export ogg_release="XXXXXX"
docker build --tag=alexlima/goldengate234:${ogg_release} --tag=alexlima/goldengate234:latest \
--build-arg INSTALLER=fbo_ggs_Linux_x64_Oracle_services_shiphome.zip . 2>&1 | tee build_${ogg_release}.log
docker pull alexlima/oracle-free:23.3-slim-ogg
docker pull influxdb:2.7-alpine
docker image tag influxdb:2.7-alpine alexlima/influxdb:2.7-alpine
docker pull telegraf
docker image tag telegraf:latest alexlima/telegraf:1.30.2
docker pull grafana/grafana
docker image tag grafana/grafana:latest alexlima/grafana:10.4.2
$cd /u99/oggdemo
$docker compose up -d
$docker compose logs -f
alexlima@alexlima-mac ~ % docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ce26134db476 alexlima/grafana:10.4.2 "/run.sh" 26 hours ago Up 26 hours 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp grafana-server
448b27e21d0c alexlima/telegraf:1.30.2 "/entrypoint.sh tele…" 26 hours ago Up 26 hours 8092/udp, 8125/udp, 8094/tcp, 0.0.0.0:8125->8125/tcp, :::8125->8125/tcp telegraf
d5f50bd685d4 alexlima/oracle-free:23.3-slim-ogg "/opt/oracle/run.sh" 26 hours ago Up 26 hours 0.0.0.0:1524->1521/tcp, :::1524->1521/tcp db233demoW
72a7fbc3e09b alexlima/oracle-free:23.3-slim-ogg "/opt/oracle/run.sh" 26 hours ago Up 26 hours 0.0.0.0:1525->1521/tcp, :::1525->1521/tcp db233demoE
d2c7fc3d4547 alexlima/goldengate234:latest "/usr/local/bin/depl…" 26 hours ago Up 26 hours (healthy) 22/tcp, 80/tcp, 0.0.0.0:281->443/tcp, :::281->443/tcp ogg234demo
0745051c0710 alexlima/influxdb:2.7-alpine "./entrypoint.sh" 26 hours ago Up 26 hours 0.0.0.0:8086->8086/tcp, :::8086->8086/tcp influxdb
$./create_replication.sh
$./enable_statsd.sh
Note: Other Important files to understand are .env which set all the environments variable to build the lab, entrypoint.sh which set up the database and telegraf.conf for telegraf configuration.
https://localhost:1055/
oggadmin / Welcome##123
You should see two extract and two replicat.
Note: Before proceeding, please stop and re-start all extracts and replicats.
Login to InfluxDB
http://localhost:8086/
admin / Welcome##123
To avoid manual steps at this stage an Admin Token and a Bucket were created during the build of the container. Look into the “DOCKER_INFLUXDB_INIT_ADMIN_TOKEN” variable in the .env file. Keep that handy as you will need it to create connection from Grafana to InfluxDB
Next let’s look if the Bucket got created. Look into the file entrypoint.sh file, that how the bucket was created.
Now that you know the Bucket was successfully created, let’s look and see if the data is populating. Click on Data Explorer.
You can now see the metrics populating the Bucket, you can use the Script Editor to start visualizing the data.
At this point, you have GoldenGate pushing metrics to the InfluxDB database, you can now connect your favorite dashboard tools as create your visualizations.
For this example, we will be attaching Grafana.
https://localhost:3000
admin / Welcome##123
From the hamburger menu, select Add New Connection
Select InfluxDB
In the connection properties window below add:
Name: influxdb
Select Flux for Query Language
URL: http://influxdb:8086
Organization: OGG_DEMO
Token: Get from the .env file
Default Bucket: goldengate-bucket
Save & Test
Note that it’s just a template for you to understand the concept, you can modify and create new dashboards as it fits your requirement.
Go to Dashboards and in the top right cornet click “New” and import.
Upload the Main Dashboard-1714512160772.json file and select the database from the dropdown.
Go back to Dashboards and select “Main Dashboard.”
You can now see the Template Dashboard in Grafana.
Additional Information:
Oracle GoldenGate YouTube Channel
Alex is Director of Product Management in the GoldenGate group focusing on GoldenGate Core Product, GoldenGate for Oracle, and Goldengate Foundation Suite. Alex has a 20-year extensive background in managing, integrating, and architecting Oracle database solutions to customers worldwide, specializing in process improvement, performance, tunning, and high availability.
Previous Post