Deploy Hadoop Easily on Oracle Cloud Infrastructure Using Resource Manager

July 19, 2019 | 3 minute read
Zachary Smith
Principal Solution Architect
Text Size 100%:

Image courtesy of https://hadoop.apache.org/

Hello, my name is Zachary Smith, and I'm a Solutions Architect working on Big Data for Oracle Cloud Infrastructure. You might have seen my blog post about Big Data Performance on Oracle Cloud Infrastructure.

Today, I want to share with you two recent Terraform automation template releases that work with Oracle Cloud Infrastructure Resource Manager: one template for Cloudera Enterprise Data Hub, and one template for Hortonworks Data Platform. These templates let you deploy a Cloudera or Hortonworks cluster in minutes, with just a few clicks!

These templates differ from deployment using standalone Terraform, in that the configuration required to make standalone Terraform work has been removed. That configuration is inherent in Resource Manager, which is native to Oracle Cloud Infrastructure. This simplification lowers the complexity required to get Terraform working, and it provides a persistent location for Terraform metadata. You can maintain and share deployment metadata, so multiple users can manage the same deployments without setting up and managing a Terraform deployment environment.

How to Get Started

  1. Download the zip file for either the Cloudera or Hortonworks Terraform deployment with Resource Manager.

  2. Sign in to the Oracle Cloud Infrastructure Console. If you don't have an account, sign up for a free trial.

  3. Click the main menu in the top-left corner, select Resource Manager, and then select Stacks.

  4. Click Create Stack.

  5. On the Stack Information page, upload the zip file that you downloaded in step 1 by browsing for it or dragging it directly to the page. Then, click Next.

  6. On the Configure Variables page, enter the contents of an SSH public-private key pair.

    You can also customize your deployment to fit your requirements. Specify which shapes you want to deploy for each role in the cluster, and which versions of each stack you want to deploy. Some things to note:

    • Cloudera deployments currently support only version 6 or later for Cloudera Manager and Enterprise Data Hub.
    • Hortonworks deployments support versions 3.x and 2.x. To find out which minimum versions are required, see the Hortonworks support matrix.
    • By default, these templates use a “simple” deployment, which means that cluster security is not enabled by default. To enable Kerberos for secure cluster operation, change the value of the DEPLOYMENT_TYPE option to secure.
  7. Click Next, and then click Create.

    Now you're ready to deploy!

  8. From the Terraform Actions menu, select Plan. Wait a few moments for this job to complete, and then select Apply from the same menu.

    This job runs for a few minutes, and outputs a URL for you to use to access your deployment manager, depending on which distribution you deployed (Cloudera Manager or Ambari for Cloudera or Hortonworks). This URL becomes accessible a few minutes after the Apply job is done.

That's how easy it is to deploy Cloudera Enterprise Data Hub or Hortonworks Data Platform on Oracle Cloud Infrastructure. Try it out yourself, and experience the blazing fast performance of Hadoop on Oracle Cloud Infrastructure.

Zachary Smith

Principal Solution Architect

I have a background in Big Data Platform Architecture, and have been working on automation and support of Big Data frameworks on OCI.


Previous Post

Mounting an Object Storage Bucket as File System on Oracle Linux

Marcin Zablocki | 2 min read

Next Post


Best Practices for Using Tags to Manage Costs, Operations, and Governance

Martin Sleeman | 9 min read
Oracle Chatbot
Disconnected