Java and .NET Application Observability with OpenTelemetry and Oracle Database

The three pillars of observability are metrics, logging and distributed tracing. OpenTelemetry is an industry standard observability framework that creates and manages telemetry data for these three pillars.

Tracing records the request paths taken either by an application or end user as it propagates through a multi-service architecture. OpenTelemetry traces are defined implicitly by their spans. A trace can be thought of as a directed acyclic graph of spans where the edges between spans are defined as parent/child relationships.

The Oracle database JDBC and Oracle Data Provider for .NET (ODP.NET) 23c drivers have been instrumented with a hook for tracing database round trips; this hook enables support for OpenTelemetry from the Java and .NET applications down to database calls. Let's take a look at this new Oracle OpenTelemetry tracing support first for Java, then for .NET.

Java Distributed Tracing - OpenTelemetry

The Oracle JDBC driver has been instrumented with a hook for tracing database round trips; this hook enables the support of OpenTelemetry. For OpenTelemetry, the Oracle JDBC has integrated “OpenTelemetry Instrumentation For Java” @ https://tinyurl.com/2762xkak. It comes, out of the box, with a JDBC instrumentation, and a Java agent. No user Java code change required.

How does it work?

1. The Java agent Create a parent span
2. The Oracle JDBC furnishes a wrapper interface for top level jdbc API alls
3. The implementation of the wrapper interface creates a BEFORE and AFTER child spans
4. Observe the exported instrumentations at the specified endpoints (e.g., logging, Zipkin, Jaeger, etc)
5. Optionally, export the collected data into an OpenTelemetry Collector.

Java Trace Event Listener for OpenTelemetry

The Oracle JDBC driver may generate events such as database roundtrips during query execution, IP address retries while establishing a connection, the beginning of Application Continuity (AC) recovery from a database outage, a successful AC recovery.

The JDBC driver defines an oracle.jdbc.spi.TraceEventListenerProvider Service Provider interface
that can be used to register a listener for publishing those events to OpenTelemetry or registering a custom TraceEventListener. See the oracle.jdbc.TraceEventListener javadoc for more details.

A new TraceEventListener with OpenTelemetry support is being published along the other JDBC providers @ https://github.com/oracle-samples/ojdbc-extensions. That provider is an implementation of the  TraceEventListener interface.

SpringBoot example
spring.datasource.url=jdbc:oracle:thin:@tcps://adb.us-phoenix-1.oraclecloud.com:1521/xyz.adb.oraclecloud.com?oracle.jdbc.provider.traceEventListener=open-telemetry-trace-event-listener-provider

The OpenTelemetry support works in tandem with the Java agent in “OpenTelemetry Instrumentation For Java” which furnishes an out of the box and non-intrusive JDBC instrumentation for OpenTelemetry

java -javaagent:path/to/opentelemetry-javaagent.jar …

Expect more built-in and open-source providers in future releases.

.NET Distributed Tracing - OpenTelemetry

Managed ODP.NET and ODP.NET Core application programming interfaces (APIs) have been instrumented to support industry standard OpenTelemetry tracing. .NET developers and operators can customize the telemetry spans and trace collected.

ODP.NET OpenTelemetry publishes activity tags, or attributes, in the trace spans. These ODP.NET spans become child spans if the .NET application creates a (parent) span before calling ODP.NET instrumented APIs. These tags can include the database name, ODP.NET connection string, user, statement executed, exception type and message, stack trace, and additional information.

ODP.NET OpenTelemetry is available as a free NuGet Gallery package. It requires an exporter, such Jaeger or Zipkin, to visualize and analyze the traces. These exporters are available as NuGet packages as well and also include the OpenTelemetry .NET SDK as a NuGet dependency.

Using ODP.NET OpenTelemetry

Once the ODP.NET OpenTelemetry NuGet package is installed, manual instrumentation is invoked upon application startup using the extension method, TracerProviderBuilder.AddOracleDataProviderInstrumentation. Here's a code sample showing how:

using OpenTelemetry.Trace;

Sdk.CreateTracerProviderBuilder()
.AddOracleDataProviderInstrumentation()  // ODP.NET extension method 
.AddConsoleExporter()
.Build();

ODP.NET OpenTelemetry developers can choose to instrument various activities, such as exceptions, database round trips, and connection opens and closures. This code sample shows how to enable tracing for connection level attributes, exceptions, OracleDataReader Read calls, and text-based SQL commands (i.e. CommandType.Text):

Sdk.CreateTracerProviderBuilder()
.AddOracleDataProviderInstrumentation(o =>
        {
          o.EnableConnectionLevelAttributes = true;
          o.RecordException = true;
          o.InstrumentOracleDataReaderRead = true;
          o.SetDbStatementForText = true;
        })
.AddConsoleExporter()
.Build();

After enabling ODP.NET OpenTelemetry, whenever a provider instrumented API is called, the configured exporter will receive the generated OpenTelemetry traces. On GitHub, we have published a complete ODP.NET OpenTelemetry code sample using the Console Exporter.

The instrumented ODP.NET APIs include: 

• OracleCommand

  • ExecuteNonQuery()

  • ExecuteNonQueryAsync(CancellationToken cancellationToken)

  • ExecuteNonQueryAsync()

  • All ExecuteReader overloads

  • All ExecuteReaderAsync overloads

  • ExecuteScalar()

  • ExecuteScalarAsync(CancellationToken cancellationToken)

  • ExecuteStream()

  • ExecuteToStream(Stream)

  • ExecuteXmlReader()

  • All ExecuteXmlReaderAsync overloads

• OracleDataAdapter

  • All Fill overloads

• OracleDataReader

  • Read()

  • ReadAsync()

Every ODP.NET database round trip originating from one of these APIs is instrumented as well.

In future releases, ODP.NET will support more options, instrument more APIs, and automatic instrumentation, which will allow using OpenTelemetry without having to modify your .NET code.

Start Using Oracle OpenTelemetry

Now that you've learned the basics of using Oracle JDBC and ODP.NET OpenTelemetry, you can get started generating traces with your Oracle database applications.