OCI Networking Best Practices - Part Three - OCI Network Connectivity

March 20, 2023 | 8 minute read
Ben Woltz
Principal Cloud Architect
Text Size 100%:

Introduction

In this blog series we are going to discuss Oracle Cloud Infrastructure (OCI) networking best practices and provide you with some recommendations and tips to help you design, build, secure and manage your OCI network infrastructure.  This is the third blog in this series and will cover OCI network connectivity such as IPSec Virtual Private Network (VPN) and FastConnect.  The topics for this blog series are outlined below:

 

 

Ensure Your Network Connectivity is Fully Redundant

Rationale

As our customers continue to grow their cloud deployments, more and more of their critical applications and workloads are now being hosted in OCI and therefore accessed externally from on-premise or other Cloud Service Providers (CSP) via connectivity methods such as IPSec VPN and FastConnect.  With this growth, customers need to ensure those critical applications in OCI are available and connected in a redundant method so that it can support both planned and unplanned outages.

Recommendation

  • Check your Dynamic Routing Gateway's (DRG) Redundancy Status for a quick indication, however further clarification may be needed to validate (see Tips below).
  • For FastConnect connections, review the FastConnect Redundancy Best Practices to:
    • Understand how many FastConnect locations your OCI region provides
    • Find your FastConnect scenario, understand the level of diversity it provides, and what your options are to increase the diversity
    • Make sure you don't have any single points of failure all along the path, including the 3rd party or Oracle partner's network, and on-prem.
  • For IPSec VPN connections:
    • Deploy two Customer Premise Equipment (CPE) with a second set of IPSec tunnels 
    • The two CPEs should preferably be located in different datacenters or geography if possible, for maximum diversity
    • If the two CPEs are in the same datacenter, at a minimum ensure the CPEs are on separate power supplies, Local Area Network (LAN) switches, and connected to different Internet Service Providers (ISP) to provide the highest level of diversity
IPSec CPE
IPSec Single and Dual CPE
  • Make sure your secondary connection can handle the bandwidth in case the primary is down.  For example, using an IPSec VPN as a backup for a 1Gb FastConnect as a primary may be appropriate, but using an IPSec VPN or a 1Gb FastConnect as a backup for a 10Gb FastConnect as a primary may not work for you.
  • Use Border Gateway Protocol (BGP) for dynamically advertising of routes to provide predictable automatic network failover
  • Perform failover tests, including:
    • When you first provision your redundant connections to validate it's working correctly before you place them into production
    • On a regular basis (e.g. every 6 months, every year, etc.) during scheduled outage windows to validate failover is still working correctly as changes can be made in your environment after the initial failover test that can break the failover.  If you only test it when you first provision the redundant connectivity, you run the risk of finding out it's not working when an actual outage occurs and it's too late. 
    • Don't forget to also validate that failing back to the primary works!

Tip: Make sure to review the OCI Connectivity Redundancy Guide for detailed redundancy options, including specific routing details to accomplish proper failover.  For example, using BGP attributes such as Autonomous System (AS) path and local preference to manipulate the routing to accomplish your desired failover.

Tip: With dual/redundant direct FastConnects, you control which FastConnect location (Physical Location) and OCI FastConnect edge router (Specify Router Proximity) they terminate on in the console when you provision them.  When using an Oracle partner for dual/redundant FastConnects, you control which FastConnect location they go to in the partner's portal.

Tip: For FastConnect, you can click on BGP Information tab in the virtual circuit and see the Logical Device that it's terminated on.  If the device is different for two FastConnects, then that will validate they are terminating on redundant OCI edge routers.  The first portion of the Logical Device name is the FastConnect location and will also tell you if the two FastConnects are terminating to different FastConnect locations.  For example, iad4-j1-vpn1 is a separate router and in a separate FastConnect location as iad5-j1-vpn1.

Consider Using an Oracle FastConnect Partner 

Rationale

There are a few connectivity models for FastConnect connectivity, including a direct or colocation with Oracle, using a third-party, and using an Oracle FastConnect Partner.  Using an Oracle FastConnect partner provides many benefits to our customers including:

  • Diversity and Redundancy
    Our partners are physically onboarded per our standard requirements into our FastConnect locations with multiple high-speed links terminating on separate OCI FastConnect routers located in the same, or in some of our larger regions, diverse FastConnect locations.
  • Ease of Provisioning
    Because the physical connectivity is already established and shared, the provisioning process of an Oracle FastConnect Partner connection is done relatively quickly and easy using the consoles of both OCI and the partner.  You can be up and running with your FastConnect within minutes to hours. 
  • Cost
    Using an Oracle Partner is a low cost option as there is no need for an expensive third-party telecommunications circuit and the costs of the physical connections to the partner are shared with all customers.
  • Large Partner Ecosystem
    We have over 80 unique partners across all OCI regions providing over 600 total connections providing a high likelihood that our customers can find a partner they can use.
FastConnect Connectivity Models Comparison
FastConnect Connectivity Models Comparison

Recommendation

  • Review the available FastConnect Partners in your specific OCI regions in this link
  • Consider using a FastConnect Partner on the list that you have an existing contract or relationship with

Understand When a FastConnect is Needed Versus an IPSec VPN

Rationale

IPSec VPN connections are often used when customers are building a small environment or conducting a Proof of Concept (POC). Many customers tend to start their OCI deployments with an IPSec VPN because they are familiar with the technology, it's relatively low cost for a small deployment or POC, and it can be quickly provisioned.

Over some time, as the POC becomes successful and the customer wishes to move to production or overall confidence in the OCI public cloud increases, a tipping point is often reached whereby moving to a dedicated connection like FastConnect is preferable. This is usually because the level of integration between cloud-based and on-premises based applications is high from a frequency, volume, or latency-sensitive perspective. 

A FastConnect may also be adopted early when a migration candidate is considered a critical application, particularly when consistent wide area network performance is a vital requirement.

OCI Connectivity Options and Considerations
OCI Connectivity Options and Considerations

IPSec VPN is routed across the public internet, and, as a result, the total bandwidth available is subject to the limits provided by your Internet Service Provider (ISP). Bandwidth can often be variable and prone to network congestion and should be considered best effort with no service levels.  Our customers typically see a least 250Mbps of throughput on an individual IPSec tunnel, with it often times exceeding that, but the actual bandwidth achieved at any point in time will depend on many factors outside of Oracle's control.

Recommendation

  • Think about and document your bandwidth and latency requirements early in the design phase of your project
  • Provision a FastConnect if you require consistent and predictable network performance (bandwidth, latency, jitter, etc.)
  • Understand the costs involved with provisioning a FastConnect.  Many customers assume the costs are higher than they actually are and will default to an IPSec VPN as it's perceived as free.  The added value customers can receive from a FastConnect is often well worth the minimal increase in costs for a FastConnect, especially when using an Oracle partner.

Use Descriptive Names for IPSec Tunnels

Rationale

When you provision an IPSec VPN connection in OCI, Oracle gives you two tunnels to use for that IPSec VPN connection.  By default, the tunnel names do not provide much valuable or descriptive information for the tunnel and can lead to confusion later on when you're trying to find more information about a specific tunnel.  The default tunnel naming convention is ipsectunnel<year><month><day><time>-<tunnelnumber>, an example is 'ipsectunnel20220112214811-1'.  

You also cannot delete a tunnel if you are not using it, which also leads to confusion as users unfamiliar with the network can misinterpret the console and think they have a production tunnel in a down state but in fact its just an unused tunnel that cannot be deleted.  

Recommendation

  • Create more descriptive tunnels names when you provision the tunnels in OCI console
    • For example, Tunnel 1 and Tunnel 2 or Primary and Backup
  • If you are not using one of the tunnels, consider using a descriptive name letting others know it's not in use.
    • For example, Unused Tunnel or Not In Use

Use Custom DRG Route Tables and Import Route Distributions

Rationale

When provisioning a DRG, you may notice that it will associate autogenerated DRG route tables to the DRG attachments, and associate autogenerated import route distributions to those DRG route tables.  These autogenerated route tables and import route distributions are intended for those most basic connectivity scenarios.  Most of our customers at some point will need to do something with their DRG that is more advanced and these autogenerated route tables and import route distributions will need to be changed.  Instead of trying to modify the autogenerated route tables and import route distributions, it is recommended that customers create their own custom DRG route tables and import route distributions as it will be cleaner and easier to understand the logic compared to modifying the default autogenerated ones. 

Autogenerated DRG Route Tables
Autogenerated DRG Route Tables
Autogenerated DRG Import Route Distributions
Autogenerated DRG Import Route Distributions

Recommendation

 

Ben Woltz

Principal Cloud Architect

Ben Woltz is a Principal Cloud Architect for OCI with over 25 years of experience in the IT networking space.  During those 25 years, his career has included working for both enterprises and service providers and his roles have spanned from delivery and support to sales.  He applies the experience he's gained from this broad background to his current role with Oracle where he helps Oracle's customers ensure their solutions are designed for successful deployment in the cloud.  


Previous Post

Using OCI devops and terraform to synchronize an OCI bucket with a git repository

Christian Weeks | 10 min read

Next Post


Scaling Web Application Firewalls, Network Load Balancers, and Next Gen Firewalls in OCI

Raffi Shahabazian | 7 min read