You have them? Me too! Every new install, new version and adapter, updates the list of jars that are needed for the different handlers to work correctly. We don't need this kind of stress in our Data integration projects for Oracle GoldenGate, where things are supposed to be simple & easy, kind of like Autonomous DB.
Do we have a comprehensive list of this version = that set of jars and lib files?
Managing versions for all the Open Source projects is a difficult task and any provided list would be quickly outdated, which highlights how great the Dependency Downloader in release 21c of GoldenGate for BigData will be.
The What's New section of the Release notes tells us that this is a set of utility scripts that have been added to download 3rd party dependency libraries for various integrations and that all we need to supply for each script is the desired version as an argument.
The release note provides the default location as {OGGBD install}/DependencyDownloader which is true for the Classic architecture installation. i.e. unzip & un-tar the software downloadable from eDelivery. (As of 15.Sept.2021) only classic is currently available).
Those of you leveraging the OCI Marketplace image and the Microservices Architecture (MSA) will find the file here: /u01/app/ogg/opt/DependencyDownloader
The Dependency Downloader is not a single tool but a set of shell scripts, one per supported Adapter Handler in fact. The scripts fall into two main categories
In either case, Apache Maven is used to search, download, build and package the required dependencies. Maven is one of those tools you will find in CI/CD processes & DevOps organisation, providing consistent builds and deployments of projects.
Execution is simple
*version - some handlers can be used without a specific version, like Snowflake
That's it on the OCI Marketplace image and the script created a subdirectory in the ./dependencies/ directory for the script and version that I supplied.
During a recent customer project, things were not so simple. A proxy was required which we configured in the config_proxy.sh script but during the execution it reported that the proxy_port=0 and failed to download the required package files.
To work-around this, I commented out the proxy configuration variables in config_proxy.sh script and used the normal session environment variables to set up the proxy:
This took us a step further but not ver far, as the script failed again. This time with a warning
[WARNING] Could not transfer metadata or POM
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
This issue, or similar, may occur when an attempt is made to download dependencies from a HTTPS repository that uses self-signed certificates.
This is common and most HTTP client tools provide options to ignore verification and certificate validation. Maven system properties are no different and the default behaviour can be changed.
There is a simple trick is to identify which script executes the Maven client dependency download process. In each of the handler dependency downloader scripts, the last line shows the location and script that will eventually call the Maven tool.
Just before the very last line there will be a comment like this:
#Execute the script to execute the maven project
Followed by:
. ./internal_scripts/<downloader_execution_script_name>
For my case, we simply made a backup of the supplied script and added these 3 system properties just before the proxy parameters
All on a single line, of course :)
GoldenGate 21c has made significant strides in simplifying the installation, deployment, configuration and management of GoldenGate. The Dependency Downloader is a great additional, providing a set of utility scripts that help teams implement the BigData handlers.
I am hoping we can have this included for all deployments, like JDBC drivers for 3rd party RDBMs and exposed as well via the Service Manager or Administration Server UI.
That would be great, but for know I am happy to settle with passing a version number to a script via CLI.
Hope you find this information useful and helps speed up your next project.
Regards
Sydney
One thing Golf teaches you, is attention to detail, practice to execution and trust.
These same principals apply to expertise in an given area, and even more so in Software Systems.