Advance Routing for Multi-Homed Hosts
By Neeraj Gupta on Mar 09, 2012
Earlier we discussed about a host participating in different networks or subnets, referred to as multi-homed host. Here I am going to talk about how to handle layer 3 routing. I will break down my discussion into two scenarios - Simple and Not-So-Simple.
Lets assume a host with four network interfaces connected to unique layer 3 subnets respectively. Three subnets are private LANs and the fourth one is a bigger one - WAN. The smaller networks could be for your management, development or testing lets say. And the bigger one is internet or intranet where it is not easy to define how many hosts or services will be there. This bigger network may even be sub-divided into more networks and almost always a router is present here. As you know that router's main function is to route traffic across unique broadcast subnets.
So our multi-homed host will have a default gateway defined towards this bigger network or WAN. Whenever a communication has to happen to someone outside of our known networks, we forward it to the default gateway. This default gateway is also called as router. Let me write this down in simple terms here.
Host's Network Participation Requirements
eth0 - 188.8.131.52 / 24 with gateway IP 184.108.40.206bond0 - 192.168.10.1 / 24 with no gateway requirementsbond1 - 10.214.28.101 / 24 with no gateway requirementsbond2 - 172.23.7.128 / 24 with no gateway requirements
Looks like this machine only needs to be talking to the corporate network through eth0 via 220.127.116.11. Problem solved ! We can simply put this default gateway in /etc/sysconfig/networking file or /etc/sysconfig/network-scripts/ifcfg-eth0.
Not So Simple Scenario
Now if we take the same host from above scenario and instead of one connectivity to a bigger network, we make two such connections. One could be towards the real Internet and another one could be towards corporate wide area network. And we still maintain another two for management and internal communications. If we continue to use the standard way to configure our default gateway then only one of the two bigger networks will be accessible. Simply because default gateways are interface or layer 3 subnet bound.
Host's Network Participation Requirements
As you can see here that eth0 and bond1 need to have their own respective default gateways.eth0 - 18.104.22.168 / 24 with gateway IP 22.214.171.124bond0 - 192.168.10.1 / 24 with no gateway requirementsbond1 - 10.214.28.101 / 24 with gateway IP 10.214.28.1bond2 - 172.23.7.128 / 24 with no gateway requirements
bond1 and bond2 do not have any default gateway requirements. They are simply confined to their actual layer 3 subnet.
If you simply add a default route then only one can be in effect at a time.
Let me re-phrase the above discussion in form of a problem statement.
How can a multi-homed host be made accessible over more than one networks across different routers ?
Linux has advanced routing capabilities made possible through iproute2 tools. This allows us to specify more than one default gateways or router addresses. I am presenting a sample config based on Oracle Enterprise Linux 5 but this can be easily adapted to other flavors including 'Vanilla' distributionsBasically, we create some rules and tables for routing lookups. We will need some unique table IDs. I am going to use 224 and 225.
They should not have been used before. You can check like this:
Look at the first column, the output should not have 224 or 225. Otherwise, use some other number.ip rule list
For eth0, create the following two files.
vi /etc/sysconfig/network-scripts/route-eth0from 126.96.36.199/32 table 224to 188.8.131.52 table 224
For bond1, create following two files.184.108.40.206/24 dev eth0 table 224default via 220.127.116.11 dev eth0 table 224
vi /etc/sysconfig/network-scripts/route-bond1from 10.214.28.10/32 table 225to 10.214.28.10 table 225
10.214.28.0/24 dev bond1 table 225default via 10.214.28.1 dev bond1 table 225
Now you can restart the network to make these new configs effective. But do it at some planned time because this will interrupt your host's access. You may also use 'ip' commands for a runtime execution.
Thats all. And your host should be now accessible across both routers.
Well, some of you may be wondering by now why I have not mentioned anything about static routes. I have not forgotten ! Use of static routes is for scenarios in between. If you have a well-known subnet beyond a router, then you should certainly add a static route for that. For example, if one of the machine connected to bond1 network also knows about another network and has routing capabilities, you can use static route through that.
Ok, so that is all for this post. As always, your comments are most welcome. Thanks !