OCI Container Instances (CI) makes it easy to run containerized workloads without standing up or managing a cluster. Customers choose CI for the speed and portability of containers with a managed per-workload isolation and predictable costs. As those workloads move beyond dev/test into production, customers also need a reliable way to centralize logs and metrics for troubleshooting.

In this blog, we will show you how to leverage two existing open source solutions on CI to export to OCI Logging and access your container logs: the sidecar collector pattern and direct in-application logging with the OCI SDK. Each approach makes different trade-offs depending on whether you want zero code changes or the leanest runtime with native OCI integration. When choosing the option that aligns with your organization’s operational direction, also make sure they align with your security, privacy, and compliance guidelines. Either approach offers centralized logging and metrics turn scattered events into a coherent narrative for troubleshooting and for operational/compliance needs.

Solution Overview

  1. The sidecar collector pattern, demonstrated in the OCI logging and monitoring sidecars repository, runs dedicated sidecar containers that continuously forward logs and metrics from shared application log and metric files to OCI Logging and OCI Monitoring. The sidecars handle file rotation performed by the application and can also optionally rotate logs and metric files themselves when the application does not.
    • Pros: This pattern keeps the application and the log and metric forwarders loosely coupled because they interact only through the shared log and metric files. That separation lets you modify, replace, or extend the forwarders independently without changing the application itself. The repository included here is the most production-ready pattern because it handles file rotation correctly and persists spool and checkpoint state on disk, allowing recovering from sidecar restarts within the same container instance.
    • Cons: Running separate containers for log and metric forwarding can increase resource consumption compared to embedding the forwarder within the application container.
  2. Direct in-application logging through the OCI SDK (for example: Direct OCI Logging) emits logs and metrics from your application directly to OCI Logging and OCI Monitoring using the OCI SDK/REST APIs.
    • Pros: No extra containers, first-class integration with OCI Logging and OCI Monitoring, and fewer moving parts at runtime.
    • Cons: Requires code changes and language-specific instrumentation. You are also responsible for batching, retries, and backoff.

Now let’s walk through each option in more detail, including prerequisites, deployment steps, and validation guidance.

Option 1: Sidecar collector pattern

OCI logging and monitoring sidecars repository demonstrates a sidecar-style observability pattern for OCI Container Instances. You can also check out CI Compose (multi-container with docker-compose.yaml) that lets you deploy a multi-container app to CI using a docker-compose.yaml.

Prerequisites:

Before deploying this pattern, you need to create a small set of OCI resources. Together, they provide network connectivity for the container instance, an OCI Logging destination for forwarded log lines, IAM permissions so the sidecars can authenticate with resource principals, and a runtime environment where the application and sidecars can run together.

You can create these prerequisites with Terraform, the OCI Console, the OCI CLI, or an internal platform or deployment pipeline. The repository already includes Terraform under `container_instance/` to provision the VCN, subnet, route table, internet gateway, security list, log group, custom log, dynamic group, IAM policy, and container instance. If you use Terraform, you mainly need to provide your image URLs and deployment-specific variable values.

  1. Network resources. Ensure the following networking resource are defined:
    • VCN for the deployment boundary.
    • Subnet where the container instance VNIC will be attached.
    • Route table that gives the subnet outbound connectivity.
    • Internet gateway or another valid outbound path depending on your network design.
    • Security list or NSG that allows the traffic your application requires.

In the sample repository, Terraform simplifies the creation of the VCN, subnet, route table, internet gateway, and security list.

  1. Logging resources

Define a log group and a custom log inside that log group. The log group acts as the parent container for the logging resource, while the custom log is the actual destination that receives application log lines. The log forwarder sends data to that custom log by using its OCID.

Pass the custom log OCID to the log forwarder as:

    OCI_LOG_OBJECT_ID

  1. Monitoring resources

Define the monitoring inputs for custom metrics, including the namespace under which the metrics will be emitted. You can also define a resource group if you want to organize related custom metrics. The metrics forwarder sends metric batches to OCI Monitoring using the regional telemetry ingestion endpoint.

The default monitoring inputs are passed into the metrics forwarder as:

  • OCI_MONITORING_NAMESPACE
  • OCI_MONITORING_COMPARTMENT_ID

Optional monitoring inputs include:

  • OCI_MONITORING_RESOURCE_GROUP
  • OCI_MONITORING_INGESTION_ENDPOINT
  1. IAM resources

Ensure you have the following IAM resources:

  • Dynamic group that matches the target container instance, which runtime resources are treated as principals.
  • Policy that allows that dynamic group to push log content and publish custom metrics,
  • Permission for the runtime to pull images from the registry you use.

At a minimum, the policy should allow the matched dynamic group to do the following:

  • Use log content in the target compartment or log group.
  • Use metrics in the target compartment.
  • Read container repositories or otherwise pull the required images.

Walk-through Steps

  1. Ensure the prerequisites are in place. If you are using the example repository, Terraform can provision most of the required OCI resources for you.
  2. Prepare images.
    • Prepare the application container image and make sure its log file path and metric file path are configurable. If you are just testing the pattern, you can use the generator image provided in the example repository. It starts an HTTP server that writes log messages and metric records to shared files, which makes end-to-end testing easier.
    • Prepare or reuse the log forwarder container image.Prepare or reuse the metrics forwarder container image.
    • Prepare or reuse the metrics forwarder container image.
    • Push these images to container registries that are accessible from the container instance.
  3. Launch the container instance.
    • Launch a container instance with the application container, the log forwarder sidecar if logging is enabled, the metrics forwarder sidecar if metrics are enabled, and the required EmptyDir style shared volumes.
    • Configure the following container environment variables.
      • Application container environment variables (for the example generator):
        • LOG_FILE_PATH: path of the shared log file
        • METRIC_FILE_PATH: path of the shared metrics file
      • Log forwarder container environment variables:
        • LOG_FILE_PATH: path of the shared log file
        • OCI_LOG_OBJECT_ID: OCI custom log OCID
      • Metrics forwarder container environment variables:
        • METRIC_FILE_PATH: path of the shared metrics file
        • OCI_MONITORING_NAMESPACE: default custom metrics namespace
        • OCI_MONITORING_COMPARTMENT_ID: target compartment OCID for custom metrics
    • Verify OCI Logging.
      • If you are using the generator image, post custom log messages using curl and the public IP of the container instance. The exact command is provided in the repository README.
      • Verify that the custom log messages appear in OCI Logging. This can be verified through the OCI Console.
    • Verify OCI Monitoring.
      • If you are using the generator image, post custom metric messages using curl and the public IP of the container instance.
      • Verify that the custom metrics appear in OCI Monitoring under the configured namespace and compartment. You can confirm this in the OCI Console or through Monitoring queries.

Operational Notes

  • The log forwarder preserves batching, on-disk spool recovery, inode-aware file tracking, and optional logrotate integration.
  • The metrics forwarder preserves batching, on-disk spool recovery, inode-aware file tracking, parsing of newline-delimited JSON metric records, and optional logrotate integration.
  • Both sidecars are now implemented as small Go binaries, which keeps the container images lighter while preserving the current behavior.
  • With CI Compose approach, you define your app and sidecar(s) declaratively, including volumes, environment, and secrets. So that you can deploy consistently across environments.

Option 2: Direct from the application using the OCI SDK

If you prefer native OCI services and minimal runtime components, you can send logs and metrics directly from the application using the OCI SDK, without a sidecar.

Prerequisites

  • Create a Log Group and Log in OCI Logging, then note the Log OCID.
  • For metrics, decide on namespace and dimensions in OCI Monitoring.
  • IAM policies such as:
    • Allow dynamic-group <name> to use log-content in tenancy
    • Allow dynamic-group <name> to use metrics in tenancy
  • Authentication will use a resource principal or an auth method you configure.
  • Network egress to OCI service endpoints (public or private, per your design).

Walkthrough Steps

Logging Example in Java

Use OCI Logging Ingestion Java SDK in your application process to publish batched log entries.

AbstractAuthenticationDetailsProvider provider = OciAuthProviderFactory.createFromEnv();
LoggingClient logging = LoggingClient.builder().build(provider);
logging.setRegion(Region.fromRegionId(System.getenv("OCI_REGION")));

Date now = new Date();
LogEntry entry = LogEntry.builder()
        .id(UUID.randomUUID().toString())
        .time(now)
        .data("order created: id=123")
        .build();

LogEntryBatch batch = LogEntryBatch.builder()
        .source("checkout-service")
        .type("app-log")
        .subject("orders")
        .defaultlogentrytime(now)
        .entries(List.of(entry))
        .build();

PutLogsDetails details = PutLogsDetails.builder()
        .specversion("1.0")
        .logEntryBatches(List.of(batch))
        .build();

logging.putLogs(
        PutLogsRequest.builder()
                .logId(System.getenv("LOG_OCID"))
                .putLogsDetails(details)
                .build());

Metrics Example in Java

Use the OCI Monitoring Java SDK to push custom metrics directly from the application.

AbstractAuthenticationDetailsProvider provider = OciAuthProviderFactory.createFromEnv();
MonitoringClient monitoring = MonitoringClient.builder().build(provider);
monitoring.setRegion(Region.fromRegionId(System.getenv("OCI_REGION")));

Date now = new Date();
Datapoint point = Datapoint.builder()
        .timestamp(now)
        .value(123.4)
        .count(1)
        .build();

MetricDataDetails metric = MetricDataDetails.builder()
        .namespace("my_service")
        .resourceGroup("payments")
        .name("request_latency_ms")
        .dimensions(Map.of("service", "checkout", "env", "prod"))
        .datapoints(List.of(point))
        .build();

PostMetricDataDetails details = PostMetricDataDetails.builder()
        .metricData(List.of(metric))
        .build();

monitoring.postMetricData(
        PostMetricDataRequest.builder().postMetricDataDetails(details).build());

Best Practices

  • Batch and compress where possible; avoid per-line API calls.
  • Standardize fields/dimensions (`service`, `env`, `region`, `version`) for querying and alarms.
  • Use resource principals in OCI runtime; avoid embedding static credentials in images.
  • Add retry with exponential backoff and jitter for transient failures.
  • Monitor ingestion errors and respect service limits/quotas.

Notes

  • This approach provides a leaner runtime architecture with fewer moving parts because no sidecar is required.
  • It is less flexible when teams need portability across multiple logging backends, and it works best when OCI Logging and OCI Monitoring are the primary targets.

Closing Thoughts

You can use the following guidance to choose the open source option that best fits your needs:

  • Choose the OCI logging and monitoring sidecars if you want a production ready solution with no code changes and standardized collection across many services and languages, especially when using external backends.
    • Choose CI Compose if you like the sidecar pattern but want a single declarative definition for a multi-container deployment.
  • Choose the direct SDK approach if you want native OCI integration and the leanest runtime, and you’re willing to update application code.

With your choice made, your next step is to establish a security, compliance, and operations checklist for your logging pipeline.

  • Secrets: Store API tokens in OCI Vault and inject them at runtime; never bake them into images.
  • Network: Restrict egress to known endpoints, always use TLS throughout. and consider private endpoints where possible.
  • IAM: Apply least-privilege policies and scope dynamic groups narrowly.
  • Data hygiene: Keep sensitive data out of logs, mask tokens and PII, and set log retention appropriately.
  • Cost and reliability: Batch, compress, and rate-limit where appropriate, plan for backpressure, and monitor ingestion errors.

If you plan to use external SaaS backends, such as Grafana Cloud, Loki, or Prometheus, verify that they align with Oracle’s internal security and compliance guidelines.

References