article by Viraj Purang, October 2020 (re-published and updated version)
If you need to view the Insights data for a Digital Assistant and at the same time also view the events for its constituent skills, you can export the data to a CSV file and create your own reports from there.
You also can automate the export, which gives you a way to monitor how well an ODA is functioning at certain time intervals, for example. You do this making by REST calls to the Insights for Skills API from a script, as described in this topic. The section below is an essential read before you jump into the API usage documentation.
The API endpoint (/api/v1/bots/insights/dataExports) that has been used in the accompanying text below, creates and starts an asynchronous task to export a digital assistant's or skill’s insights data into a ZIP file that's stored in the instance.
The response includes a jobId, which a user can utilize for the task's ID in other Insights Data Management Export operations. This data can be filtered by a digital assistant's ID, a skill's bot name and optional version, or a bot ID. In addition to the options presented above, the data to be exported can be filtered by a date range.
Given the various permutations and combinations possible, it is important to disambiguate and clarify the following:
1. odaId parameter is a new parameter that has been specifically added to the aforesaid API for exporting Digital Assistant’s Insights data. This is a unique ID that identifies a version of a digital assistant to export. Underneath the covers, the odaId is just the botId of the given Digital Assistant (DA).
2. However, while using this API, the data being exported can be filtered by digital assistant or by skill but not both. This “odaId” and “botId/botName” parameters are mutually exclusive. They cannot be mixed in 1 call.
3. When used, the “botid” refers to a specific version of a specific Skill. So using botId will only export data pertaining to that skill version. E.g. a botId corresponding to Pizza bot will only export Pizza skill entries, the same is true for Wine Skill. Within the context of the botId parameter , the DA is a special case as it is also a Skill. Hence if the user provided botId=<botId of a DA>, only entries pertaining to the DA skill (like NoMatch, Exit, Help – executed at the DA level only) will be output. The entries from the accompanying skills (like Pizza and Wine) will not be added.
4. For the purpose of exporting Insights data corresponding to a Digital Assistant and its skills in a single file, only “odaId” parameter can and should be used. If used correctly, the exported zip file will contain data for the digital assistant and its skills in a single file. The file has a “D_” prefix, to clearly identify that it contains data pertaining to the Digital Assistant. It also contains a few additional columns as described below.
1. A CHANNEL_SESSION_ID that in conjunction with the BOT_NAME and CHANNEL_ID, uniquely identifies a new session. A user can see a different value in this field by using the RESET button (or by letting the session expire).
2. A BOT_NAME that identifies the specific bot that the exported row belongs to. This can be used to identify switch between different skills and DA. (e.g. While looking at a dump from the CbPizzaWinBot – First 3 entries may belong to DA, the next “n” entries belong to the Pizza bot and the next “m” entries belong to the Wine bot)
3. A CHANNEL_ID that identifies the specific channel on which the conversation was conducted.
5. As mentioned above, the exported zip file will contain data for the digital assistant and its skills in a single file. It is however important to clarify that the skill data included in the file excludes any data for the given skill that was created outside of a Digital Assistant context. (E.g. If Pizza bot was directly invoked as a skill and not from a Digital Assistant, any entries added will be excluded)
6. This export of DA Insights is not intended to be used for imports. Additionally, the export will not contain any “B_” or “S_” prefixed files. So no aggregate data or speech insights are added to the zip.
Note:
The cURL command examples given below use the URL structure https://<host>/<resource-path>, where <host> is the REST server.
API Usage Steps
To download insights data:
1. Get an AUTH TOKEN
2. Start an export job.
curl -X POST "https://<host>/api/v1/bots/insights/dataExports?odaId=${ODA_BOT_ID}" -H "Content-Type: application/json" -H "Authorization: Bearer ${TOKEN}" -d @example-request-payload.json
where the contents of example-request-payload.json contains any of the optional properties. For example:
{
"name":"CbPizzaWineBotJuneExport",
"taskType":"EXPORT"
}
In addition to the above payload, please also specify the following options on the cURL command line:
• ODA_BOT_ID: Set this variable (value for the query parameter – odaId) with the id of ODA for which export task is to be scheduled.
• -X POST to indicate the type of request.
• -H "Authorization: Bearer ${TOKEN}" to provide the access token.
• -H "Content-Type: application/json" to identify the content type of the request body.
• (Optional) -i to include the HTTP headers in the output.
This call will return the jobId. Attached below is an example of the response body.
{
"jobId" : "7f124ab1-5fd8-4c40-b8b1-f1344430c801"
}
You can use this ID of the task to monitor its progress. In this example, the Job ID is 7f124ab1-5fd8-4c40-b8b1-f1344430c801. You need this value to get the export status, get information about the export task, get the exported data, and delete the export task. In the section below, this jobId is referenced as ${JOB_ID}.
3. Monitor the export job's status by entering a command like this:
curl -H "Authorization: Bearer $TOKEN" -X GET "https://<host>/api/v1/bots/insights/dataExports/${JOB_ID}"
Specify the following options on the cURL command line:
• Set JOB_ID variable to refer to the jobId returned by the Start/Initiate export call.
• -X GET to indicate the type of request.
• -H "Authorization: Bearer <access token>" to provide the access token.
• (Optional) -i to include the HTTP header in the output.
Here's an example of the response body.
{
"name": " CbPizzaWineBotJuneExport",
"createdBy": "John.Doe",
"jobType": "EXPORT",
"jobId": "7f124ab1-5fd8-4c40-b8b1-f1344430c801",
"status": "EXPORT_SUCCEEDED",
"error": "NA",
"jobTimestamp": "<Date in the following format – YYYY-DD-MMThh:mm:ss>",
"filenames": [
"ocid1.odainstance.oc1.phx.amaaaaaaquabhxqauov73ii6co4mbf44ppu5mx2ymuxpidoeftuxce3c6x3q~7f124ab1-5fd8-4c40-b8b1-f1344430c801~insightsdata~0.zip"
]
}
4. If the above job succeeds, download a ZIP that contains the CSV file, as shown in this example:
curl -X GET "https://<host>/api/v1/bots/insights/dataExports /${JOB_ID}/files/${FILENAME}" -H "Authorization: Bearer $TOKEN" -O -J -L
Specify the following options on the cURL command line:
• Set JOB_ID variable to refer to the jobId returned by the Start/Initiate export call.
• Set FILENAME variable to refer to a specific file from the output of the previous command that checks the current status of the export.
• -X GET to indicate the type of request.
• -H "Authorization: Bearer <access token>" to provide the access token.
• (Optional) -i to include the HTTP header in the output.
5. Here's an example of the CSV file (Sample Download). Please note that these files start with “D_” prefix:
6. Once the zip from step 4, has been downloaded, please also download the corresponding Convert.py python script from the following link that is specifically for 20.08 to generate locally a transformed version of the CSV data that is more conducive to viewing conversations.
The output is based on a format that prior versions of ODA used for export. For more details about the script, refer to the Oracle Support/Knowledge Base article KM 2689677.1 at the following location link1 (or link2). (These links may require authentication). The script’s download link is also embedded within the same article. The usage is as given below:
Usage: python3 Convert.py <inputZipFile> <outputDirectory>
Please note that the script requires Python 3.x (Python v 3.8.6 or above for Windows users), and the inputZipFile refers to the just downloaded zip file that contains the data for the digital assistant and its skills.
Downloads
Sample Digital Assistant CSV export
Related Documentation
REST APIs for Oracle Digital Assistant
Send signed request to Oracle Digital Assistant REST APIs in OCI
