X
  • March 26, 2018

Deploying Highly Available DC/OS on Oracle Cloud Infrastructure with Terraform

Changbin Gong
Principal Product Manager and Cloud Advisor

Introduction

This post provides a Terraform template to automatically deploy DC/OS on Oracle Cloud Infrastructure

DC/OS is an open source and distributed data center operating system based on the Apache Mesos distributed system kernel. It manages multiple systems from a single interface and enables the deployment of containers, distributed services, and applications into these systems. 

DC/OS consists of a group of master and agent nodes that form a cluster. 

The Terraform template automatically deploys this DC/OS cluster on Oracle Cloud Infrastructure. The template consists of a set of Terraform modules and an example base configuration that is used to provision and configure the resources that are needed to run a highly available and configurable DC/OS cluster on Oracle Cloud Infrastructure. 

Oracle Cloud Infrastructure Environment

You can deploy the DC/OS cluster to any region within Oracle Cloud Infrastructure.  For your high availability needs, we recommend deploying DC/OS master and agent nodes across multiple availability domains of an Oracle Cloud Infrastructure region. 

The following diagram illustrates an example deployment of DC/OS cluster with master, public agent, GPU agent, and regular private agent nodes. 

The Terraform template provisions the following compute instances: 

  • Bootstrap node
  • Master node
  • Public agent node
  • Agent node
  • GPU agent node

The template accepts several input variables to choose instance shapes (including GPU),  the number of these instances, and how these instances are placed across multiple availability domains etc.  If your requirements extend beyond the base configuration, you can customize the related modules to form your own configuration. 

Deploy DC/OS Cluster

Prerequisites

  • Download and install Terraform (v0.10.3 or later).
  • Download and install the Oracle Cloud Infrastructure Terraform Provider (v2.0.0 or later).
  • Download Oracle Cloud Infrastructure Terraform DC/OS Installer template.
    • This template is in process to be uploaded to Oracle public Github. Please reach out to me to get this template. 

Quick Start

  • Customize your configuration
    • Open "env-vars" file in the project root that specifies your configuration. 
    • Set mandatory Oracle Cloud Infrastructure input variables related to your tenancy, user, and compartment:
      • Tenancy OCID
      • User OCID
      • API fingerprint
      • API private key
      • Compartment OCID
      • Public and private key pairs for SSH access to DC/OS instances
    • Check key default input variables and update them accordingly:
      • dcos_installer_url:   This is the URL to get the DC/OS code.  The default value is set to the early access release. You can change to stable release as: https://downloads.dcos.io/dcos/stable/dcos_generate_config.sh
      • dcos_master_ad1_count, dcos_master_ad2_count, dcos_master_ad3_count:  The number of master nodes in each availability domain. 
      • dcos_agent_ad1_count, dcos_agent_ad2_count, dcos_agent_ad3_count:  The number of agent nodes in each availability domain.
      • dcos_public_agent_ad1_count, dcos_public_agent_ad2_count, dcos_public_agent_ad3_count:  The number of public agent nodes in each availability domain.
      • dcos_gpu_agent_ad1_count, dcos_gpu_agent_ad2_count, dcos_gpu_agent_ad3_count:  The number of GPU agent nodes in each availability domain.
  • Source "env-vars"
    • `$  .  env-vars `
  • Initialize Terraform
    • `terraform init`
  • View what Terraform plans to do before actually doing it
    • `terraform plan`
  • Provision DC/OS resources and setup DC/OS cluster on Oracle Cloud Infrastructure
    • `terraform apply`

When resources provisioning is completed,  the Terraform template shows the following output:

Outputs:

 

master_private_ips = [

  xx.xx.xx.xx,

  xx.xx.xx.xx,

  xx.xx.xx.xx

]

master_public_ips = [

  xx.xx.xx.xx,

  xx.xx.xx.xx, 

  xx.xx.xx.xx,

]

Notes:

The deployment of the agent, public agent, and GPU agent nodes depends on the deployment of the bootstrap node. However, with the Terraform module, there is no mechanism to explicitly set the dependency as is typically done in a Terraform "resources" dependency. The keyword "depends_on" does not work in Terraform modules. To work around this, we use the implicit dependency by using variables.  

For instance, in module "dcos_agent_ad1", a variable "dcos_bootstrap_instance_id" is defined which depends on the creation of "dcos_bootstrap" instance id of module "dcos_bootstrap". 

module  "dcos_agent_ad1" {

    source              = "./instances/agent"

    count               = "${var.dcos_agent_ad1_count}"

    availability_domain = "${lookup(data.oci_identity_availability_domains.ADs.availability_domains[0],"name")}"

    compartment_ocid    = "${var.compartment_ocid}"

    tenancy_ocid        = "${var.compartment_ocid}"

    dcos_cluster_name   = "${var.dcos_cluster_name}"

    image               = "${var.InstanceImageOCID[var.region]}"

    shape               = "${var.AgentInstanceShape}"

    subnet_id           = "${module.vcn.subnet_ad1_id}"

    ssh_public_key      = "${var.ssh_public_key}"

    ssh_private_key     = "${var.ssh_private_key}"

    display_name_prefix = "ad1"

    dcos_bootstrap_instance_id = "${module.dcos_bootstrap.instance_id}"

}

Access DC/OS Cluster

Once the DC/OS deployment is completed,  you can access the DC/OS dashboard at http://<master_public_ip>/

Next Steps

You can scale up and down the number of agent, public agent, or GPU agent nodes by using input variables.  For instance,  you can scale up the number of agent nodes as:

$ terraform apply -var "dcos_agent_id1_count=xx"

 

Conclusion

Oracle Cloud Infrastructure provides a high performance and flexible environment to run DC/OS and its services.  With this terraform DC/OS  installer,  you can automate deployment of DC/OS cluster on OCI.   

 

 

 

Join the discussion

Comments ( 2 )
  • Pierluigi Thursday, January 3, 2019
    What about the dc/os template ?
  • Michel Friday, March 8, 2019
    Hi
    Do you have the Oracle Cloud Infrastructure Terraform DC/OS Installer template available somewhere?

    Thanks

    Michel
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.