Announcing the availability of OCI Connector Hub service logs

We’re excited to announce the general availability of Oracle Cloud Infrastructure Connector Hub service logs, which provide you greater visibility into connector functioning. These service logs can help you to troubleshoot connector issues easily so that you can focus on what matters most – transferring data from sources to targets within and beyond Oracle Cloud Infrastructure (OCI) in an effortless and seamless way. 

Connector Hub service logs

The key application of the logs is determining issues in data movement through Connector Hub. Using the logs from the Connector Hub, you can get details of the data flow from source to target and identify issues at any stage of the transfer. For example, if you created a connector to send data from OCI Logging to OCI Streaming, but you don’t see the data appearing in the stream, how do you identify the problem?

In this example, the issue can be that the policy to write to the stream target is missing. In this and many other cases, service logs can be a powerful tool at your disposal to figure out what and where the issues are without needing to raise support tickets. You can use logs to make quick, informed decisions about managing your connectors.

Connector Hub logs provide information about connector runs and details about the data moved (messages) if data transfers successfully or error details if failure occurs. For more information, see Sample Success and Error Logs.

Getting started

You can enable Connector Hub service logs using the following three methods. Choose the one that suits your workflow best.

• Directly enable logs when creating a connector.

  

• Enable logs on the connector details page by selecting Logs link and clicking the Enable Log slider.

  

• On the central logs page, click Enable Service Log and select Connector Hub.

  

After the logs are enabled successfully, you can see Enable Log option is enabled.

Clicking the log name opens the Log Detail page.

Log fields

Sample success and error logs

The following extracts illustrate what the logs look like in successful and error conditions.

Success log

{
  "datetime": 1690935026859,
  "logContent": {
    "data": {
      "level": "INFO",
      "message": "Run succeeded - Read 2 messages from source and wrote 2 messages to target",
      "messageType": "CONNECTOR_RUN_COMPLETED"
    },
    "id": "f83205ef-0bef-47d0-b6b2-362afc4a2e9a",
    "oracle": {
      "compartmentid": "ocid1.compartment.oc1..aaaaaaaa77y4rj3njic7xkz2qgw6xzzvhm4qacla6zz4jc3s3g4dcg2iypfa",
      "ingestedtime": "2023-08-02T00:10:28.990Z",
      "loggroupid": "ocid1.loggroup.oc1.phx.amaaaaaaqlphkzyaw5qvgypsmjxyjsbnuvypfa4pgxzlwc4s5leyhbgo4ima",
      "logid": "ocid1.log.oc1.phx.amaaaaaaqlphkzyavka6hf3gxqcxftnqiepqa5hfvqsnc4olmfgucghghjlq",
      "resourceid": "ocid1.serviceconnector.oc1.phx.amaaaaaaqlphkzya3jmx5iwl6tsuyqxlv4vq5xieeqm5mtgzollkrk5sv44a",
      "tenantid": "ocid1.tenancy.oc1..aaaaaaaahifpge3gx5meuvbvjldmlnr55syzsyadugu6hfaerndttvnr57za"
    },
    "source": "KP_SourceStream1_TargetStream1",
    "specversion": "1.0",
    "time": "2023-08-02T00:10:26.859Z",
    "type": "com.oraclecloud.sch.serviceconnector.runlog"
  },
  "regionId": "us-phoenix-1"
}

Error log

{
  "datetime": 1690936656793,
  "logContent": {
    "data": {
      "level": "ERROR",
      "message": "Run failed due to Destination error, Error Code : 404 NotAuthorizedOrNotFound. For troubleshooting tips, see https://docs.oracle.com/iaas/Content/service-connector-hub/troubleshooting.htm.",
      "messageType": "CONNECTOR_RUN_COMPLETED"
    },
    "id": "cbe82e67-4fed-4a2e-91e8-582f6c881eba",
    "oracle": {
      "compartmentid": "ocid1.compartment.oc1..aaaaaaaa77y4rj3njic7xkz2qgw6xzzvhm4qacla6zz4jc3s3g4dcg2iypfa",
      "ingestedtime": "2023-08-02T00:37:42.973Z",
      "loggroupid": "ocid1.loggroup.oc1.phx.amaaaaaaqlphkzyaw5qvgypsmjxyjsbnuvypfa4pgxzlwc4s5leyhbgo4ima",
      "logid": "ocid1.log.oc1.phx.amaaaaaaqlphkzyavka6hf3gxqcxftnqiepqa5hfvqsnc4olmfgucghghjlq",
      "resourceid": "ocid1.serviceconnector.oc1.phx.amaaaaaaqlphkzya3jmx5iwl6tsuyqxlv4vq5xieeqm5mtgzollkrk5sv44a",
      "tenantid": "ocid1.tenancy.oc1..aaaaaaaahifpge3gx5meuvbvjldmlnr55syzsyadugu6hfaerndttvnr57za"
    },
    "source": "KP_SourceStream1_TargetStream1",
    "specversion": "1.0",
    "time": "2023-08-02T00:37:36.793Z",
    "type": "com.oraclecloud.sch.serviceconnector.runlog"
  },
  "regionId": "us-phoenix-1"
}

Pricing

Using Connector Hub incurs no charges. However, service logs are stored in the OCI Logging service, and pricing is $0.05/GB log storage per month after the first 10 GB of storage per month.

Try the feature today!

Connector Hub is accessible through the Oracle Cloud Console under the Analytics and AI section, software development kit (SDK), CLI, REST API, and Terraform. To learn more about Connector Hub, see the Connector Hub documentation.

To try Connector Hub for yourself, sign up for the Oracle Cloud Free Tier or sign into your account.