Automatically deploy an APM Java agent in Kubernetes Environments using the OpenTelemetry Operator

As of November 2024, the latest version of the Java agent image will be available from the Oracle Container Registry 

This blog reviews how to use the OpenTelemetry operator to automatically inject and configure the OCI Application Performance Monitoring (APM) Java agent into your application pods on Kubernetes (K8s) clusters.

The OCI APM Java agent has been developed to monitor Java applications and there are multiple ways to deploy it in Kubernetes environments and other environments.  The OpenTelemetry operator is the newest deployment option and is an implementation of a K8s operator providing an easy way to automatically inject Java agents to JVMs running on K8s Pods. The latest release of the OCI APM Java agent (1.12) now supports this additional deployment option for ease of use in K8s environments.

Pre-requisites to automatically deploy APM Java agent in Kubernetes using OpenTelemetry:

First, install the OpenTelemetry Operator into the K8s cluster. 

There are three options to choose from: Operator release manifest, Operator helm chart, or Operator Hub. In most cases, a cert-manager should be installed. If the helm chart option is used, there is an option to generate a self-signed cert instead.

Option 1: Operator helm chart

This option is to use helm chart to install OpenTelemetry operator.

• Add helm chart repository.

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts

helm repo update

• Create Kubernetes namespace for OpenTelemetry operator.

kubectl create namespace opentelemetry-operator-system

• Install OpenTelemetry operator.

helm install opentelemetry-operator open-telemetry/opentelemetry-operator \

--namespace opentelemetry-operator-system \

--set "manager.collectorImage.repository=otel/opentelemetry-collector-k8s"\

--set admissionWebhooks.certManager.enabled=false\

--set admissionWebhooks.autoGenerateCert.enabled=true

Option 2: Operator release manifest

This option is to use YAML files to install OpenTelemetry operator.

To install the cert manager, run the following:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.17.1/cert-manager.yaml

To install the OpenTelemetry operator, run the following:

kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml

To manage automatic instrumentation, the operator needs to be provided with information about the agent and its configuration.  Also required is a way to “tell” the operator which of the pods should be instrumented.  The first is done via a Custom Resource Definition (CRD). The second is done by way of special annotation.

1.  Create a K8s custom resource (CR)

This Custom Resource will be used by the operator to copy the agent into the pod and add to it the required configuration.

Run the following command to create the CR on your K8s cluster.

Run the following command to create the CR on your K8s cluster.

kubectl apply -f – <<EOF
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: inst-apm-java
  namespace: opentelemetry-operator-system
spec:
  java:
    image: container-registry.oracle.com/oci_observability_management/oci-apm-java-agent:latest
    env:
      – name: OTEL_com_oracle_apm_agent_data_upload_endpoint
        value: <data-upload-endpoint>
      – name: OTEL_com_oracle_apm_agent_private_data_key
        value: <private-data-key>
EOF

* There are some placeholders in the above CR. Replace them with values in the below table.

An example:

kubectl apply -f – <<EOF
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: inst-apm-java
  namespace: opentelemetry-operator-system
spec:
  java:
    image: container-registry.oracle.com/oci_observability_management/oci-apm-java-agent:latest
    env:
      – name: OTEL_com_oracle_apm_agent_data_upload_endpoint
        value: https://aaaaa12345678aaaaaaaaaaabc.apm-agt.us-phoenix-1.oci.oraclecloud.com
      – name: OTEL_com_oracle_apm_agent_private_data_key
        value: 1234567890ABCDEFGHIJKLMNOPQRSTUV
EOF

The created CR can be queried by the following command.

kubectl get otelinst -n opentelemetry-operator-system

All endpoints and environment variables are correct is required for auto-instrumentation to work properly.

2. Add the K8s annotation

The OpenTelemetry operator uses K8s annotation to decide which pods should be auto-injected with the OCI APM Java agent.

The annotation can be added to a namespace, so that all pods within that namespace will get injected; or the annotation can be added to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources.

Annotation:

instrumentation.opentelemetry.io/inject-java: "opentelemetry-operator-system/inst-apm-java"

Run the below command to start editing your namespace:

kubectl edit namespace <your-namespace-name>

After running the command, an editor (e.g., vi) is opened to edit the namespace.

Add the annotation to the namespace. Note that indentation is important to make it a valid YAML file.

An example of a namespace with the added annotation:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    kubernetes.io/metadata.name: mynamespace
  annotations:
    instrumentation.opentelemetry.io/inject-java: "opentelemetry-operator-system/inst-apm-java"
  name: mynamespace
spec:

3.  Restart K8s pod

Run the below command to restart the pod where you want to auto-inject the OCI APM Java agent:

kubectl delete pod <your-pod-name> -n <your-namespace-name>

After the pod gets restarted, run the below command to verify that your pod has been auto-injected with the OCI APM Java agent:

kubectl get pod <your-pod-name> -n <your-namespace-name> -o yaml

The OCI APM Java agent and the OpenTelemetry Operator make it easy to get started with monitoring your Java applications running on K8s. Log in to OCI today and get started!

Resources

• Try OCI APM Always Free 
• Visit Provision and Deploy APM Java Agent on Application Servers
• Visit Application Performance Monitoring: Instrument Java on Kubernetes for Monitoring and Diagnostics
• Visit OpenTelemetry Operator for the Kubernetes page
• OCI APM documentation 
• OCI APM blog space