Better Marketplace Listings with CI/CD

Ben Lackey
Senior Principal Partner Manager, Data and AI Partnerships

As we work to build out Oracle Marketplace, we're always trying to find ways to improve both customer and partner experience.  To that end, we recently introduced Oracle Cloud Marketplace CI/CD. Since then, we've continued to work with our partner, H2O.ai, on rolling this out.  It's been great to see positive feedback and that our work is helping partners:

“Oracle Marketplace CI/CD enables us to quickly maintain and update the H2O.ai listing on Oracle Marketplace, providing a more up to date and better quality listing than is possible with a manual process.”

-Peter Sollimo, Sr. Director of Channel Sales at H2O.ai

In this blog post, we're going to dig a bit deeper into the idea of a listing as code and the value that it provides to customers, to partners, and to Oracle itself.

The Old Way

When first deploying ISV products on clouds, companies often go through a process much like the one shown in the following diagram. The obvious place to start is the Marketplace, which provides a nice GUI for deploying software. That's a great initial step. However, GUIs can be limiting. For example, it's hard to understand the details of what's happening in a Marketplace deployment using a GUI or customize that deployment.

So, it can be helpful to tinker with the underlying infrastructure as code (IaC). For that, various clouds, including Oracle Cloud Infrastructure, offer Quick Start programs. But, the Quick Start are traditionally distinct from the Marketplace. It's a different code base and a rather brutal shift from Marketplace to Quick Start.

Diagram that shows the progress from the marketplace to quickstarts, a custom module, and a custom solution.

Both Marketplace listings and Quick Start have limitations. For example, in working with a customer a while ago, we started with the Quick Start for Confluent. It helped them get deployed quickly, but they had some particular naming requirements for the network and disks. So, they forked the Quick Start and began customizing it to meet their needs, making their own custom module. One great thing about that experience is that it was continuous. There was no need to switch between templates.

Improving Customer Experience

In maintaining Marketplace listings, one thing I’ve always found frustrating is how many IaC templates there are. On other platforms, I've had the misfortune of maintaining IaC templates for Marketplace listings and Quick Starts, and then another set for people to actually use in production because neither of the first two met that requirement. At Oracle, we’re trying to improve on that model.

We want to make the transition from initial experimentation to production deployment less jarring. So, for many of our Marketplace listings, the Terraform in them are identical to what is in the Quick Start. The Marketplace GUI lets you get started easily. When you want more detail, you can download the Terraform directly from Marketplace and inspect it that way. However, you can also go to the Quick Start and get a nice versioned copy of that Terraform. Because it lives in GitHub, you can see previous commits, open issues, and even contribute yourself. We welcome pull requests!

The Quick Start and the Marketplace listings are kept up to date with a continuous integration/continuous delivery (CI/CD) pipeline. When a user makes a commit to the repo for H2O.ai, for example, several GitHub actions start. The code runs through several tests written in Terratest. Assuming those pass and it deploys cleanly, another GitHub action calls the Marketplace API and updates the listing. This process ensures that the listings are current.

The key takeaway is that the IaC we offer through both the Marketplace and Quick Start is better tested and more up to date than it would be with a more manual process, leading to a better customer experience.

Improving Partner Experience

About a year ago, we introduced the Oracle Quick Start. It’s a collection of repos hosted on GitHub that make it easy to deploy both Oracle and partner products on Oracle Cloud Infrastructure. We started with 22 examples. Since then, it’s expanded to 81, including partners like Cloudera, Confluent, and SAS. There are also Oracle examples, such as Oracle Essbase and Agile PLM. You can learn more about it in this blog post.

The initial idea with Quick Start was to have a public place to collaborate with partners as we built IaC integrations to deploy their products on Oracle Cloud Infrastructure. When we introduced the Marketplace in February 2019, it quickly became obvious that the Quick Start could also be used as the basis for Marketplace listings.

So, working with partners, we began to go through a process that was the opposite of the preceding one:

Diagram that shows the progress from the quickstart to the marketplace.

We'd create a Quick Start as the first step of working with a partner. After that was deploying cleanly, we'd begin work on a Marketplace listing—writing copy for that listing, building a dummy image so the metering would function properly, and so on.

The CI/CD makes the process even smoother. When we have a Quick Start and actions in place, it becomes straightforward to update a listing. Listings need updates for three common reasons:

  • A new version of the software has been released.
  • A new version of the OS has been released.
  • A bug has been fixed or new functionality has been added to the IaC in the listing itself

With a manual listing process, changes are often batched because the amount of effort to update a listing from end to end can take hours of hands-on time and days of total time. With a CI/CD, that changes. For example, changing the version in a listing is a simple change to the Terraform.

Screenshot of GitHub that shows a commit with one changed file, in which the version number of the H2O.ai quickstart was updated.

Take the preceding commit as an example. The H2O.ai Quick Start was on DAI version 1.7.0. By updating the Terraform, it was possible to change it to 1.8.5. With the CI/CD, a GitHub action is triggered.

Screenshot that shows an action triggered in the workflow on GitHub for the quickstart.

The action first calls Terratest to test that the deployment works properly. Assuming that works properly, it makes a call to the Marketplace API to update the listing.

Doing this work manually would take days. With a CI/CD in place, it takes only minutes. The result is an up-to-date listing for H2O.ai Driverless AI.

Screenshot that shows the H2O Driverless AI app page in the Oracle Cloud Marketplace.

Getting Started

We're working with our partners to roll out this functionality to many of the repos and listings that exist today. Ultimately, we’d like to get the process to a complete self-service state. To that end, we have a template repo to use as a starting point. In the current process, a partner works with their assigned Oracle Marketplace architect to instantiate that repo as a template instance.

With that in place, there's some setup to connect the repo to the partner's Marketplace partner portal account. With all that in place, a partner is ready to go. The end-to-end process is as follows:

  1. Commit a change to the Quick Start repo.
  2. The GitHub action runs.
  3. The deployment is tested.
  4. A Marketplace API call submits the deployment.
  5. Vulnerability and malware scans are conducted.
  6. The listing is updated.

The following screenshot shows an example of how this workflow is represented:

Screenshot that shows the process listed in the text.

H2O.ai is the first listing we’ve worked on to add this functionality. We expect to roll out this functionality to many more listings shortly. If you’re interested in being part of this program or learning more about it, reach out to me at ben.lackey@oracle.com.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.