Announcing OCI intra-VCN routing and VCN gateway ingress routing enhancements

August 3, 2022 | 15 minute read
Lilian Quan
Senior Principal Solutions Architect
Text Size 100%:

Oracle Cloud Infrastructure (OCI) provides software-defined virtual networking. The centralized network control plane has the complete information and control of what’s deployed in the virtual cloud networks (VCNs), such as subnets, instances, and network gateway, which allows OCI to simplify routing in the VCN networks.

By design, OCI takes care of routing inside a VCN using the central control plane and database. It performs direct local routing, routing traffic directly between any two endpoints within a VCN. Customers don’t need to worry about setting up routing for them. When traffic enters a VCN from the internet through an internet gateway or a NAT gateway, or from an OCI service through a service gateway, OCI also takes care of the routing. It routes the ingress traffic directly to destinations inside the VCN.

The simplified routing gives our customers an easy jump-start to build and manage their VCN networks to host their cloud resources. To further enrich our OCI routing functions, we’re now introducing the following new routing features and enhancements:

  • Intra-VCN routing: Allows you to define your own routes for intra-VCN traffic that goes between two different subnets in the same VCN.

  • Internet and NAT gateway ingress routing: Allows you to define your own routes for inbound public traffic to your VCNs through an internet or NAT gateway.

  • Service gateway ingress routing enhancement: Allows you to define your own routes for service gateway to route traffic from OCI services to destinations inside your VCN.

These new capabilities provide more flexibility and easier ways to control how traffic within a VCN or entering a VCN from the internet or OCI services is routed. They augment the existing VCN direct local routing by offering straightforward solutions to apply more control over intra-VCN or ingress traffic routing. By taking advantage of these new capabilities and the original VCN direct local routing, you can achieve a simpler and more scalable network design that meets your routing requirements.

In the following sections of this blog, we introduce these new features individually with feature overviews, what problems the feature solves, and what network designs it enables. When a firewall is presented in a design example, we use the OCI network firewall. To learn more about the OCI network firewall, its security features and the key customer use cases, refer to the blog Defense in Depth, Layering using OCI Network Firewall, written by my colleague Troy Levin

Intra-VCN routing

Intra-VCN routing supports user-defined routing between subnets in the same VCN. Before introducing of this feature, traffic between two endpoints in the same VCN is always routed directly from the source to the destination and you can’t change this behavior. Now, you can define your route rules to insert an intermediate hop in the routing of the inter-subnet traffic (the traffic between two subnets) within a VCN.

Intra-VCN route rules are defined in the subnet VCN route tables. The target of an intra-VCN route rule can be a private IP in the same VCN, a local peering gateway (LPG) or dynamic routing gateway (DRG).

The problems it solves

Intra-VCN routing makes it easier to insert virtual network appliances, such as a virtual firewall, into the forwarding path of inter-subnet traffic in a single VCN. Using a firewall for traffic between different application tiers (often referred to as east-west traffic or lateral traffic) has become a common practice for more secure application deployments. If the application tiers are deployed in the same VCN, the direct routing for intra-VCN traffic doesn’t allow you to add a firewall as an intermediate hop in the routing path of the application traffic.

As a workaround, you need to deploy the two application tiers in two separate VCNs and use inter-VCN routing to add the firewall into the routing path of the east-west traffic. The downside of this workaround is that as your applications grow, your network becomes exponentially complicated with an ever-increasing number of VCNs.

Network designs enabled by intra-VCN routing

Let’s continue the east-west traffic firewalling as an example. With the intra-VCN routing feature, you can now deploy your application tiers in separate subnets in the same VCN and easily insert a firewall between two application tiers by simply configuring custom route rules between the subnets with the firewall as the target.

One design option is to deploy a firewall in each application VCN, as shown in the following example, which uses an OCI network firewall. The route tables of the two subnets have a route rule for each other with the firewall’s private IP address 10.1.0.99 as the target. The traffic between the subnets is routed to the firewall. The firewall routes post-inspection traffic to the firewall subnet gateway (10.1.0.1 in this example). The firewall subnet uses the default direct local routing in the VCN to route traffic to the destination in each direction, either the web tier subnet or the app tier subnet.

Firewall Insertion Per-VCN

 

 

 

 

 

 

 

 

 

 

You can also deploy a firewall in a central service hub VCN and have your applications in multiple spoke VCNs to share the central firewall. This option is considered a more cost-effective and operationally simpler design because you only need to deploy a single firewall.

The following diagram shows an example of such a design. In this example, the service hub VCN and the application spoke VCNs are connected by a DRG. The web tier and app tier of the example application are deployed in two subnets of the same VCN. The two subnets have intra-VCN subnet route rules to reach each other with the DRG as the target. The DRG forwards the application traffic to the firewall in the service-hub VCN for inspection. The diagram shows only one application VCN, but in production deployments, you can have more application VCNs as spokes attached to the DRG to share the central firewall.

A graphic depicting central firewall insertion with intra-VCN routing.

The details of inter-VCN connectivity by DRG or using DRG for centralized network virtual appliance design are out of the scope of this blog. If you’re interested in further learning on these topics, the following documents are good resources:

The key benefit of using intra-VCN routing to insert a firewall for east-west traffic inspection is that it eliminates the VCN sprawling. You can deploy application tiers in subnets of the same VCN and add a firewall between different tiers. It keeps the network simpler and more scalable.

Internet and NAT gateway ingress routing

The traffic from the internet is routed into a VCN through its internet gateway or NAT gateway. Each VCN has its own internet or NAT gateway. Before the support of ingress routing, internet or NAT gateways always route the ingress public traffic directly to the destinations in its VCN. You can’t alter this direct routing.

With ingress routing on internet and NAT gateways, you can now define custom routing on the internet or NAT gateway so that, instead of routing the ingress public traffic directly to the destinations, the internet or NAT gateway can route the traffic to the targets you specified in your custom route rules. For instance, you can tell the internet or NAT gateway to send the traffic to a firewall or another network virtual appliances.

The following diagrams show the differences between the scenarios with and without ingress routing on internet and NAT gateways:

A graphic showing the differences between with and without internet gateway (IGW) and NAT gateway (NATGW) ingress routing.

To use ingress routing on internet and NAT gateways, associate a VCN route table with the internet or NAT gateway and define the route rules that you want the internet or NAT gateway to use in this route table. These route rules tell the internet or NAT gateway how to route the ingress public traffic. The route rules can have a private IP address in the VCN as the target. If you have no custom routes defined for a destination in the VCN in the internet or NAT gateway route table, the internet or NAT gateway routes the ingress traffic directly to the destination using the original direct local routing.

The problems it solves

Internet or NAT gateway ingress routing allows you to easily insert a network virtual appliance, such as a virtual firewall, between the internet or NAT gateway and the destinations in the VCN so that the gateway can route the ingress public traffic to the virtual appliance first for inspection or other process. This requirement is becoming more common. For example, many web applications hosted in the cloud offer their services to public users on the internet. The users access the application services from the internet through the public IP address of the application web tier. Because the internet is considered an untrusted environment, the application owners often require the ingress public traffic from the users to be inspected by a firewall before it reaches any of their resources in their VCNs.

Typically, the web tier of these web applications is deployed in a public subnet with a load balancer and several instances in the backend set. The public IP address for accessing the application service is often allocated on the web tier load balancer. The traffic between internet users and the web tier is routed through the internet gateway of the VCN. Inserting a firewall into the forwarding path between the internet gateway and the web tier requires the firewall to be added as an intermediate routing hop. The current direct local routing on internet gateway interferes with this intent because it always routes the traffic directly to destinations in the VCN. If you keep the public IP address for the service on the web tier load balancer, the ingress traffic bypasses the firewall with direct routing by the internet gateway.

The example in the following diagram depicts the issue. In the example, the application web tier public IP address is 150.136.118.162. It’s allocated to the application web tier load balancer that has a private IP 10.1.1.104. When internet users access the application service, they send traffic to the public IP address 150.136.118.162. The traffic is routed to the internet gateway of the application VCN. The internet gateway changes the destination IP address to the web yier load balancer’s private IP 10.1.1.104, then routes the traffic directly to the load balancer. The firewall is bypassed in the routing of the ingress traffic.

A graphic depicting the challenge of internet firewall insertion with a firewall bypass.

As a workaround to avoid the firewall bypassing issue, you need to move the public IP address for the application service onto the firewall so that the internet gateway routes the ingress public traffic to the firewall. The firewall performs destination NAT (DNAT) to change the destination IP address to the web tier load balancer’s private IP address, then routes the packet towards the load balancer.

The following diagram depicts a design with this workaround. In this design, the public IP address to access the application service (150.136.118.162) is moved onto the firewall. The firewall’s private IP address is 10.1.0.99. Internet users send traffic to the firewall public IP address 150.136.118.162. When the IGW receives the ingress traffic, it changes the destination IP address to the firewall’s private IP 10.1.0.99, then routes the traffic to the firewall. The firewall translates the destination IP address from its own private IP, 10.1.0.99, to the web tier load balancer’s private IP, 10.1.1.1.04, then routes the post-inspection packets to the firewall subnet gateway at 10.1.0.1. The firewall subnet uses the default VCN direct local routing to route the packets directly to the web tier load balancer.

A graphic depicting the internet gateway firewall insertion workaround.

This workaround has the following challenges:

  • The complexity in the network design and the firewall configuration

  • Limited scalability of how many IP addresses each firewall virtual network interface card (vNIC) can have

  • Extra burden on the firewall for NATing can impact performance.

  • Can reach the firewall NAT scalability limit

Internet and NAT gateway ingress routing enables simpler designs for virtual firewalls or any other network virtual appliance insertion into the ingress public traffic forwarding path.

The designs enabled by internet and NAT gateway ingress routing

With internet and NAT gateway ingress routing, you can create your own route rules in the gateway route table to overwrite the default direct routing behavior of the gateway. Now, to achieve the same design goal of firewall insertion, you can create a route rule for the destination subnet or the VCN CIDR (if you want the gateway to send all ingress traffic to the firewall) with the firewall’s private IP as the target. It adds the firewall to the native routing path of the ingress public traffic.

The following diagram shows the simplified firewall insertion design enabled by internet gateway ingress routing:

A graphic depicting internet gateway firewall onsertion with ingress routing.

In this example, we keep the application public IP address 150.136.118.162 on its web tier load balancer that has a private IP 10.1.1.104. To ensure that the ingress public traffic always goes through the firewall, we use ingress routing on the internet gateway. In the internet gateway ingress route table, we add a route rule for the web tier subnet CIDR 10.1.1.0/24 with the firewall’s private IP address 10.1.0.99 as the target. The traffic from the application users on the internet has the public IP address 150.136.118.162 as the destination and is routed to the internet gateway.

The internet gateway changes the destination IP address from 150.136.118.162 to the web tier load balancer’s private IP 10.1.1.104, then routes the packet to the firewall based on the route rule for 10.1.1.0/24 in its ingress route table. The firewall inspects the packet and sends the post-inspection packet to the firewall subnet gateway (10.1.0.1). The firewall subnet can use the default VCN direct local routing to route the packet to the web tier load balancer based on the destination IP 10.1.1.104. For the return traffic, the web tier subnet routes the traffic to the firewall using the 0.0.0.0/0 route rule with the firewall as the target in its subnet VCN route table. The firewall subnet routes the post-inspection traffic to the internet gateway using the 0.0.0.0/0 route in its route table.

With this new design, the firewall is added into the native routing path between the internet gateway and the application web tier. It removes the need for NATing on the firewall. Because you no longer need to move the application public IP address onto the firewall, the maximum number of IP addresses supported per firewall VNIC is no longer a concern. You get a much simpler and more scalable solution to insert a firewall (or any network virtual appliances) for the ingress public traffic through a natural routing design.

Service gateway ingress routing enhancement for traffic from OCI services

Service gateways previously supported ingress routing for the OCI transit routing design. It was used for on-premises compute instances to access Oracle Service Network through a service gateway in a hub VCN. The route table associated with the service gateway can have routes for customers’ on-premises networks with a DRG as the target. However, if the destinations were in the VCN, the service gateway always routed the traffic directly to them. You couldn’t alter the routes.

With this service gateway ingress routing enhancement, you can define your own route rules for the service gateway to route traffic towards the VCN internal destinations. These custom route rules must be created in the VCN route table associated with the service gateway. The target in these route rules can be a DRG or a private IP in the VCN.

The problems it solves

This enhancement for service gateway ingress routing enables easy insertion of network virtual appliance, such as a virtual firewall, into the routing path of the traffic from OCI services into the VCN through a service gateway.

Many customers consider the traffic from OCI services to be untrusted external traffic. They want the service gateway to send it through a firewall for security inspection before it reaches any of their internal resources in the VCN. Without this service gateway ingress routing enhancement, no simple way to insert a firewall exists because the service gateway always routes the traffic directly to the destinations in the VCN, and the firewall is bypassed. The example in the following diagram depicts the challenge:

A graphic depicting the challenges of service gateway (SGW) firewall insertion with firewall bypass.

In this example, the app tier of the application needs to access an OCI service. While the traffic from the app tier to the OCI service can be routed through the firewall without any issue, the return traffic from the OCI service to the app tier bypasses the firewall because the service gateway changes the destination IP to the app tier private IP, 10.1.2.106, and then routes it directly to the app tier destination.

To avoid the firewall bypass issue, you need a way to make the traffic appear to be between the firewall and the OCI service from the service gateway’s point of view. As a workaround, for the initial traffic from the app tier to the OCI service, you can have the firewall to NAT the source IP address from the app tier private IP to the firewall’s private IP before sending it further to the service gateway. With this change, the service gateway sees the outbound traffic to the OCI service is sourced by the firewall. So, when it receives the response traffic from the OCI service back to the application, it changes the destination IP to the firewall’s private IP and route the packet to the firewall. The firewall then needs to change the destination IP back to the actual app tier private IP and route the post-inspection packet to the app tier. The following diagram shows an example with this workaround design:

A graphic depicting the workaround for service gateway firewall insertion.

This workaround design relies on translating the network address on the firewall. Its scale and performance can be limited by the firewall NAT scale and performance. The service gateway ingress routing enhancement for traffic to VCN internal destinations offers a simpler solution.

The designs enabled by the service gateway ingress routing enhancement

With the service gateway ingress routing enhancement, you can insert the firewall into the natural forwarding path for the traffic from the OCI service. Define a custom route rule in the service gateway route table that tells the service gateway to route the traffic to the firewall. This route rule ensures that the traffic coming from the OCI service back to the application is always routed through the firewall even if the destination IP address is the app tier private IP. You don’t need the firewall to do any network address translating. The following diagram shows an example with the simplified design for firewall insertion in OCI service traffic routing.

A graphic depicting the service gateway firewall insertion with ingress intra-VCN routing.

In this example, the application subnet route table has a route for the OCI service with the firewall’s private IP address 10.1.0.88 as the target. With this route, the egress traffic sourced by the app tier to the OCI service is routed to the firewall. The firewall subnet route table has a route for the OCI service with the service gateway as the target so that the egress traffic is routed to the service gateway after firewall inspection. The service gateway receives the traffic and sees the source of the app tier private IP 10.1.2.106. So, with the response traffic from the OCI service, the service gateway changes the destination IP address to 10.1.2.106. Because the service gateway route table has a route rule for the application subnet 10.1.2.0/24 with the firewall’s private IP 10.1.0.88 as the target, the service gateway routes the traffic to the firewall. The firewall subnet routes the post-inspection traffic to the app tier using the VCN direct local routing.

This design offers a native routing solution to insert a virtual firewall (or any network virtual appliance) between the service gateway and the destinations in the VCN. This more-scalable and high-performance solution doesn’t need to translate the network address with the firewall.

Conclusion

With the new capabilities of intra-VCN routing, internet and NAT gateway ingress routing, and the service gateway ingress routing enhancement, OCI provides more flexibility for variant routing designs. You can use these new capabilities together with the original VCN direct local routing to achieve simpler yet more scalable network designs to insert virtual firewall or any other virtual network appliance into the native routing paths for your intra-VCN traffic or ingress traffic from the internet or OCI services.

At Oracle Cloud Infrastructure, we keep our customers’ needs and best interests as our topmost priority. We continuously evolve our solutions and functionalities to better serve our customers’ needs. Stay tuned and be the first to learn about the latest and greatest in our most recent developments.

Lilian Quan

Senior Principal Solutions Architect


Previous Post

Announcing IPv6 ULA and multiple prefix support

Shahreen Fredrich | 5 min read

Next Post


Gain Greater insights from Oracle Enterprise Manager data using OCI Operations Insights

Murtaza Husain | 7 min read