Deploy Hadoop Easily on Oracle Cloud Infrastructure Using Resource Manager

Zachary Smith
Principal Solution Architect

Image courtesy of https://hadoop.apache.org/

Hello, my name is Zachary Smith, and I'm a Solutions Architect working on Big Data for Oracle Cloud Infrastructure. You might have seen my blog post about Big Data Performance on Oracle Cloud Infrastructure.

Today, I want to share with you two recent Terraform automation template releases that work with Oracle Cloud Infrastructure Resource Manager: one template for Cloudera Enterprise Data Hub, and one template for Hortonworks Data Platform. These templates let you deploy a Cloudera or Hortonworks cluster in minutes, with just a few clicks!

These templates differ from deployment using standalone Terraform, in that the configuration required to make standalone Terraform work has been removed. That configuration is inherent in Resource Manager, which is native to Oracle Cloud Infrastructure. This simplification lowers the complexity required to get Terraform working, and it provides a persistent location for Terraform metadata. You can maintain and share deployment metadata, so multiple users can manage the same deployments without setting up and managing a Terraform deployment environment.

How to Get Started

  1. Download the zip file for either the Cloudera or Hortonworks Terraform deployment with Resource Manager.

  2. Sign in to the Oracle Cloud Infrastructure Console. If you don't have an account, sign up for a free trial.

  3. Click the main menu in the top-left corner, select Resource Manager, and then select Stacks.

  4. Click Create Stack.

  5. On the Stack Information page, upload the zip file that you downloaded in step 1 by browsing for it or dragging it directly to the page. Then, click Next.

  6. On the Configure Variables page, enter the contents of an SSH public-private key pair.

    You can also customize your deployment to fit your requirements. Specify which shapes you want to deploy for each role in the cluster, and which versions of each stack you want to deploy. Some things to note:

    • Cloudera deployments currently support only version 6 or later for Cloudera Manager and Enterprise Data Hub.
    • Hortonworks deployments support versions 3.x and 2.x. To find out which minimum versions are required, see the Hortonworks support matrix.
    • By default, these templates use a “simple” deployment, which means that cluster security is not enabled by default. To enable Kerberos for secure cluster operation, change the value of the DEPLOYMENT_TYPE option to secure.
  7. Click Next, and then click Create.

    Now you're ready to deploy!

  8. From the Terraform Actions menu, select Plan. Wait a few moments for this job to complete, and then select Apply from the same menu.

    This job runs for a few minutes, and outputs a URL for you to use to access your deployment manager, depending on which distribution you deployed (Cloudera Manager or Ambari for Cloudera or Hortonworks). This URL becomes accessible a few minutes after the Apply job is done.

That's how easy it is to deploy Cloudera Enterprise Data Hub or Hortonworks Data Platform on Oracle Cloud Infrastructure. Try it out yourself, and experience the blazing fast performance of Hadoop on Oracle Cloud Infrastructure.

Join the discussion

Comments ( 1 )
  • Ashish Nabira Thursday, August 1, 2019
    Thank you... really useful
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.