GoldenGate Stream Analytics (GGSA) 19.1.0.0.7 has introduced management operations through REST APIs. This enables continous integration and delivery (CICD) approaches to include GGSA into your devops pipelines. The supported operations for GGSA pipelines are:
Besides pipeline management, we have also introduced user management REST APIs. These APIs enable you to automate bulk creation of users as well as to create automated scripts to synchronize users with other user management solutions. Operations for user management are:
In order to show a simple deployment flow, please see the diagram below. It performs the following steps:
- Export a draft pipeline from the development environment using the Export PipelineAPI
- (Optional) Commit the pipeline to a source control system such as GIT. This is not part of GGSA APIs
- Create a user on a newly provisioned production environment using the Create User API
- Import the pipeline to the production environment using the Import Pipeline API
- Start the pipeline using the Publish Pipeline API
![Automated Deployment using GGSA API](https://blogs.oracle.com/content/published/api/v1.1/assets/CONTF93C329F460944A4B9358D94BCF3BE25/Medium?cb=_cache_aa24&format=jpg&channelToken=8b43304987cf4ca092ab7a83133d013d)
The following steps illustrate a simplified deployment flow:
- You need to obtain the ID of the pipeline to export. You can either get this through the List Pipeline API, or through the GGSA Web UI as part of the URL when opening a pipeline:
![Pipeline Editor UI](https://blogs.oracle.com/content/published/api/v1.1/assets/CONT491FE16CE75F47EFAB4BA73D1D675681/Medium?cb=_cache_aa24&format=jpg&channelToken=8b43304987cf4ca092ab7a83133d013d)
- Use the ID as well as your credentials to call the Export Pipeline API from the devops machine. In CICD use cases this call would typically be performed from a script such as Ansible, Chef, or others. Below is the CURL command for simplicity.
curl -k --compressed -X GET -u osaadmin:MYPASSWORD https://111.111.111.111/osa/services/v1/pipelines/45C38FE6-83D0-4F07-A7D5-7FDDFD11B400/export -o export_pipeline.zip
|
Please note that the export command will include all dependencies of the pipeline, such as sources, targets, references, connections, predictive models, geo fences, visualizations, and custom jars. You can provide a full filesystem path in the -o parameter to specify the directory the zip file will be stored in. We are using the -k switch of curl to bypass the check of the default self-signed certificate. For production installations it is advised to install a trusted SSL certificate.
- You can now commit the contents of the zip file to your Source Code Control system, such as GIT. This is optional and outside the scope of this tutorial, please consult documentation of GIT for details.
- As a next step a user can be created on the target environment using the following CURL command.
curl -k --compressed -X POST -u osaadmin:MYPASSWORD -H 'Content-Type: application/json' -d '{"credential":"DEVOPSPASSWORD"}' https://222.222.222.222/osa/admin/services/v1/authorizations/admin/devopsuser
|
- Now import the previously exported zip file using the following CURL command.
curl -k --compressed -X PATCH -u devopsuser:DEVOPSPASSWORD -F artifacts=@export_pipeline.zip https://222.222.222.222/osa/services/v1/artifacts/import?overwrite=true
|
The result of the API call is a JSON document with IDs of all imported objects. The IDs of the original objects are maintained, so that it overwrites old objects if imported into the same GGSA instance.
- Finally publish the pipeline using the following CURL command.
curl -k --compressed -X PATCH -u devopsuser:DEVOPSPASSWORD
https://222.222.222.222/osa/services/v1/pipelines/45C38FE6-83D0-4F07-A7D5-7FDDFD11B400/publish
|
- After import and publish, you can now see the imported objects and published pipeline.
![Imported and Published Pipeline](https://blogs.oracle.com/content/published/api/v1.1/assets/CONTD67CE516CF504CB2AEE604EE2605E1DD/Medium?cb=_cache_aa24&format=jpg&channelToken=8b43304987cf4ca092ab7a83133d013d)
This blog demonstrated the use of management APIs in GoldenGate Stream Analytics.
For more information, please visit
Alex Kotopoulis
Director of Product Management
Alex is Director of Product Management in the Database team and covers Database Migration, Streaming and Big Data products such as OCI Database Migration, Oracle Stream Analytics, and GoldenGate.