A routing scenario, transit traffic on OCI - Part Two

March 24, 2023 | 5 minute read
Andrei Stoian
Master Principal Cloud Architect | North America Cloud Engineering
Text Size 100%:

Last week, an interesting case was brought to my attention, for short, one of our customers asked if by using a specific routing configuration (detailed below) the OCI can transport the IP packets from one on-premises site to another. The general use case for using the OCI as a transit network was already analyzed in one of my blogs -  A routing scenario, transit traffic on OCI, but let's find if the new case will still deliver the same result.

For the new case, the following networking diagram is used to depict the concept and the traffic path.

1

As we can see, there are two on-premises sites, Site 1 and Site 2.

In OCI we are using two regions, Region 1 and Region 2 and between the regions the RPC (Remote Peering Connection) is in place.

Site 1 is connected to Region 1 by using an IPSec connection or FastConnect (BGP over IPSec for our configuration), the same applies for Site 2 in Region 2.

The main goal is to have IP connectivity between 10.114.187.0/24 and 10.0.1.16/29 subnets using the DRGs and the RPC in between. Is that possible? Let's find out.

Configuration 1 - Dinamically imported routes

In the configuration 1 we will just use the dynamic route importing from the IPSec or FastConnect attachment to the RPC and vice-versa.

We have two goals to achieve (if possible): 1. To verify if the subnet from Site 1 was advertised to Site 2 and vice-versa; 2. If the IP connectivity is established between the two sites.

The DRG route table for IPSec attachment in Region 1 has a route distribution which imports the prefixes received over the RPC:

2

That is great, the 10.0.1.16/29 is received from the RPC so, we might think that the DRG from Region 1 does not have any information that actually 10.0.1.16/29 is received by the Region 2 DRG from on-premises Site 2 and 10.0.1.16/29 can be further advertised to Site 1. If that will be the case we should be able see 10.0.1.16/29 is the BGP table on Site 1 CPE (Customer Premises Equipment):

3

It seems that 10.0.1.16/29 is not advertised to Site 1.

Let's verify what is happening at Site 2.

The DRG route table for IPSec attachment in Region 2 has a route distribution which imports the prefixes received over the RPC:

4

The obvious question now is, is 10.114.187.0/24 advertised to Site 2?

5

Analyzing the Site 2 CPE reveals that no 10.114.187.0/24 was advertised.

It is very clear now the traffic between the two sites will not work since the routing information was not propagated. Why the routing information is not propagated will be next question?

It is not propagated because the IPSec or FastConnect received routes carries the BGP ASN in the BGP Path Attributes Update Message. Please refer to OCI DRGv2 and Route Conflict blog post to verify the routes with the BGP AS_PATH information. That being said, the route provenance being IPSec or FastConnect (even if it is received over the RPC) it will not get advertised to any IPSec or FastConnect - OCI cannot be used as a transit network, unde the section "Route propagation restrictions".

Configuration 2 - Static configured routes

What if we configure static routes and "force" the DRGs to announce the respective IP prefixes to Site 1 and Site 2 CPEs? After the CPEs receives the correct routes, the traffic should work between Site 1 and Site 2 using the RPC, isn't it? Let's find out.

Configure a static route for 10.0.1.16/29 in the DRG IPSec route table attachment and verify if Site 1 CPE did receive it:

6

7

Good news, Site 1 CPE has a valid route to Site 2, 10.0.1.16/29.

Now, let's do the same configuration for Region 2 DRG and verify if Site 2 receives a valid route for 10.114.187.0/24:

8

9

Site 2 indeed received the BGP Update from the DRG in Region 2 and the route was marked as best and preferred.

We confirmed that both sites received the correct routes, the traffic should work between the two sites. Let's verify:

10

Yes, that is the expected behavior, the traffic will not work even if we "forced" the BGP updates for the two IP prefixes. OCI will drop the traffic since it is seen as being originated from on-premises and having as destination another on-premises in other words, entering one IPSec tunnel and existing another IPSec tunnel. Do we have any configuration that can be used to avoid this behavior? The answer is yes, and it has been presented here.

Andrei Stoian

Master Principal Cloud Architect | North America Cloud Engineering


Previous Post

Data Import Options and Guidelines for Fusion Applications Suite

Bala Mahalingam | 8 min read

Next Post


OCI Inter-Region Cloud Connectivity Use Case - FastConnect and Remote Peering Connections

Raffi Shahabazian | 8 min read