X

Secure MongoDB on Oracle Bare Metal Cloud Services

By Craig Carl

Oracle Bare Metal Cloud Services offers a set of features that makes it especially compelling for NoSQL and Big Data workloads, including:

  • Low latency 10Gb network
  • No hypervisor overhead on Bare Metal instances
  • Bare Metal instances with up to 28.8TB of NVMe SSDs and 512GBs of memory
  • Multi-Availability Domain model that makes it easy to build HA datasets
  • Security and isolation tools that make it a secure place to build infrastructure

MongoDB, a leading database program, has recently become a popular target for cyber criminals looking for a high value ransomware target. In the past month alone, thousands of MongoDB servers have fallen victim to ransomware, forcing organizations to either pay the ransom, or risk losing critical data.

According to a recent study by the Ponemon Institute that included a survey of 350 organizations in 11 countries the average cost of a data breach is 3.79 million dollars, that’s strong motivation to build secure infrastructure.

The Castle approach -

I talk to Oracle customers every day about the best ways to build secure infrastructure in the cloud, defense in depth is always an important part of those conversations. Defense in depth, or the Castle approach to IT security is an information assurance concept that stipulates that no breach of any single security control will result in an exploitable vulnerability. This approach requires that you to build redundancy into every level of your IT defenses.

Start with completely private subnets –

On Oracle Bare Metal Cloud Services (BMCS), compute instances are launched into subnets in a Virtual Cloud Network (VCN). Subnets can be created that allow unlimited access to/from the Internet, to allow only certain traffic both inbound and outbound (ingress and egress) or to deny all Internet access, ingress and egress. We call subnets that allow any Internet traffic at all, ingress or egress, ‘public subnets’ and subnets that don’t support any Internet access ‘private subnets’.

public_private.png

Our first layer of defense is to create subnets that don’t have any access to or from the Internet. The BMCS platform provides three different ways to isolate instances from the Internet, we’ll use all three because good defense in depth calls for as much redundancy as possible –

  1. Use a route table without a default gateway
  2. Don’t attach an Internet Gateway to the subnet
  3. Deny all to the Internet on both the ingress and egress Security Lists (firewalls)

Any one of these methods is, by itself, enough to isolate these instances from the Internet. By using all three we ensure that accidental exposure is unlikely and significantly raises the level of effort for an attacker.

Malicious Administrators -

To protect against a malicious administrator, compromised credentials, or a single administrator making a whole bunch of mistakes we can use the BMCS’s IAM policy language to ensure that no single administrator has permissions to make all the changes required. Once the appropriate policies have been deployed two administrators would have to be involved before these instances could be exposed. BMCS user permissions are highly granular and can be scoped up and down as required.

Secure compute instances –

By default the Linux operating systems that BMCS delivers to customers have their host firewalls enabled and configured to deny all inbound connections except port 22. The host firewall should never be stopped and ports should never be opened to all inbound connections, a source CIDR range should always be specified, that CIDR range should be as small as possible, limited to a VCN subnet or on-premises network.

The SSH service is configured to not support password based authentication, customers provide the public half of a keypair when they launch an instance. Never enable username/password authentication for SSH, keypair or certificate based auth are your only viable options today.

As a best practice customers should build separation of duties in their IT organizations such that administrators who have permissions to manage the BMCS VCN configuration don’t have the ability to SSH into an instance. Once that’s done it would take three administrators to expose MongoDB on these instances to the Internet; two to change the VCN and another to change the host firewalls. This protects against any two malicious administrators or set of compromised credentials.

Using the compute isolation capabilities of BMCS, secure-by-default OSes, and some best practices this MongoDB deployment is off to a secure start.

Securing MongoDB –

While early versions of MongoDB lacked some basic security capabilities more recent versions have improved dramatically. The most recent versions of MongoDB support –

Accessing MongoDB –

A MongoDB that’s completely inaccessible isn’t much good to anybody so we’ll need to find a way for clients to securely access the database.

For enterprises that need access from on-premises networks, BMCS offers VPN and other private connectivity options. These private connection options continue the theme of isolation and privacy and are the best option where business requirements allow. In this case the ingress firewalls on the subnets that host the MongoDB would allow access only to MongoDB specific ports where the source IP is in the RFC1918 CIDR range of the on-premises network(s).

File Jan 12, 7 17 16 AM.jpeg

For use cases where Internet facing applications also hosted on BMCS need access to MongoDB we’ll add one or more public subnets to the VCN and configure the Security Lists appropriately, which is conservative. Every subnet has a stateful ingress and egress firewall, we’ll use both to continue the theme of defense in depth.

private_public.png

With these assumptions –

  • The CIDR range of your VCN is 10.0.0.0/27
  • Your application servers are installed in a public subnet
    • Public subnet = 10.0.0.24/29
  • The applications in the public subnet only listen on ports 80 and 443
  • Your MongoDB is replicated across two Availability Domains, one private subnet per AD
    • Private subnet in AD1 = 10.0.0.0/29
    • Private subnet in AD2 = 10.0.0.8/29
  • MongoDB is listening on the default port of 27017

Security Lists deny all by default so we would need to open certain (source|destination)/port pairs –

  • Public subnet in AD1
    • ingress
      • § source 0.0.0.0/0, ports 80,443
      • egress
        • § destination 10.0.0.0/29, port 27017
        • § destination 10.0.0.8/29, port 27017
  • Private subnet in AD1
    • ingress
      • § source 10.0.0.24/29, port 27017
      • § source 10.0.0.8/29, port 27017 (for MongoDB replication)
      • egress
        • § destination 10.0.0.8/29, port 27017
  • Private subnet in AD2
    • ingress
      • § source 10.0.0.24/29, port 27017
      • § source 10.0.0.0/29, port 27017 (for MongoDB replication)
      • egress
        • § destination 10.0.0.0/29, port 27017

These rules would need to be duplicated on the host firewalls, defense in depth requires redundancy. You should have noticed at this point that we don’t have any way to connect via SSH to any instance in any of these subnets. Good eye.

Administrative access –

We always need to aggressively protect SSH access to instances, we’ve already agreed to never enable username/password auth, but we can do more.

If you are using VPN you should access port 22 directly from your on-premises network using the dynamic Security List technique described below, you don’t need a bastion host.

If you need SSH access to the infrastructure over the Internet you should use a bastion host with the added layer of protection described below. It’s important that the private half of the keypair used on the MongoDB instances is never on the bastion host. Scott Lowe has written a great primer on bastion hosts and how to configure your SSH client to make using a bastion transparent.

We never want to allow port 22 access from the subnet(s) where our applications live, we want to protect against an attacker gaining access to an application and jumping from there to a MongoDB instance. This means we won’t put our bastion host in the same subnet as the application servers, it will get its own.

Because we’re hosting this infrastructure in the cloud we can easily add a layer of protection that wouldn’t be practical on-premises. This is one of my favorite, only in the cloud security tricks, on-demand SSH access. On BMCS, changes to Security Lists generally take less than 15 seconds to become effective so why not just add a port 22 rule to a Security List only when an administrator needs it and remove the rule when the work is done? Using one of our SDKs or the API directly it’s easy to write a script that will –

  1. Determine the source IP of connections from the administrator’s client
  2. Add rules to the Security Lists of both the bastion subnet and the private subnets
    1. Bastion subnet
      1. ingress
        1. source <client IP>/32, port 22
        2. egress
          1. destination 10.0.0.0/29, port 22
          2. destination 10.0.0.8/29, port 22
          3. Private subnets
            1. ingress
              1. source <CIDR range of bastion subnet>, port 22
  3. Wait a few seconds for the changes to take effect
  4. SSH into a MongoDB host through the bastion
    1. http://blog.scottlowe.org/2015/11/21/using-ssh-bastion-host/
  5. Remove the port 22 rules on disconnect
    1. The truly cautious will want to protect against the possibility that the script doesn’t remove the SL rules when the SSH session doesn’t end cleanly. The most robust way to do this is a cron job on each instance that removes the rules whenever there isn’t an active SSH session.

Secure, performant, cost effective, scalable, easy to use, and enterprise focused, Oracle Bare Metal Cloud Services is the best place to build infrastructure supporting NoSQL and Big Data workloads.

You can be up and running in the OBMC in less than 2 hours, signup at https://shop.oracle.com/product/BareMetal or contact your Oracle sales team.

Technical questions and feedback on this post are both encouraged, craig.carl@oracle.com.

Join the BMCS community at https://community.oracle.com/community/cloud_computing/bare-metal.

BMCS white papers - https://docs.us-phoenix-1.oraclecloud.com/Content/General/Reference/aqswhitepapers.htm

Craig
$ sudo ship_it

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha