• March 10, 2012

Advance Routing for Multi-Homed Hosts

Earlier we discussed about a host participating in different networks or subnets, referred to as multi-homed host. Here I am going to talk about how to handle layer 3 routing. I will break down my discussion into two scenarios - Simple and Not-So-Simple.

Simple Scenario

Lets assume a host with four network interfaces connected to unique layer 3 subnets respectively. Three subnets are private LANs and the fourth one is a bigger one - WAN. The smaller networks could be for your management, development or testing lets say. And the bigger one is internet or intranet where it is not easy to define how many hosts or services will be there. This bigger network may even be sub-divided into more networks and almost always a router is present here. As you know that router's main function is to route traffic across unique broadcast subnets.

So our multi-homed host will have a default gateway defined towards this bigger network or WAN. Whenever a communication has to happen to someone outside of our known networks, we forward it to the default gateway. This default gateway is also called as router. Let me write this down in simple terms here.

Host's Network Participation Requirements

eth0  - / 24  with gateway IP
bond0 -  / 24  with no gateway requirements
bond1 - / 24  with no gateway requirements
bond2 -  / 24  with no gateway requirements

Looks like this machine only needs to be talking to the corporate network through eth0 via Problem solved ! We can simply put this default gateway in /etc/sysconfig/networking file or /etc/sysconfig/network-scripts/ifcfg-eth0.

Not So Simple Scenario

Now if we take the same host from above scenario and instead of one connectivity to a bigger network, we make two such connections. One could be towards the real Internet and another one could be towards corporate wide area network. And we still maintain another two for management and internal communications. If we continue to use the standard way to configure our default gateway then only one of the two bigger networks will be accessible. Simply because default gateways are interface or layer 3 subnet bound.

Host's Network Participation Requirements

eth0  - / 24  with gateway IP
bond0 -  / 24  with no gateway requirements
bond1 - / 24  with gateway IP
bond2 -  / 24  with no gateway requirements
As you can see here that eth0 and bond1 need to have their own respective default gateways.
bond1 and bond2 do not have any default gateway requirements. They are simply confined to their actual layer 3 subnet.

If you simply add a default route then only one can be in effect at a time. 


Let me re-phrase the above discussion in form of a problem statement.

How can a multi-homed host be made accessible over more than one networks across different routers ?


Linux has advanced routing capabilities made possible through iproute2 tools. This allows us to specify more than one default gateways or router addresses. I am presenting a sample config based on Oracle Enterprise Linux 5 but this can be easily adapted to other flavors including 'Vanilla' distributions :)

Basically, we create some rules and tables for routing lookups. We will need some unique table IDs. I am going to use 224 and 225.

They should not have been used before. You can check like this:

ip rule list
Look at the first column, the output should not have 224 or 225. Otherwise, use some other number.

For eth0, create the following two files.

vi /etc/sysconfig/network-scripts/rule-eth0
from table 224
to table 224
vi /etc/sysconfig/network-scripts/route-eth0 dev eth0 table 224
default via dev eth0 table 224
For bond1, create following two files.

vi /etc/sysconfig/network-scripts/rule-bond1
from table 225
to table 225
vi /etc/sysconfig/network-scripts/route-bond1 dev bond1 table 225
default via dev bond1 table 225

Now you can restart the network to make these new configs effective. But do it at some planned time because this will interrupt your host's access. You may also use 'ip' commands for a runtime execution.

Thats all. And your host should be now accessible across both routers.

Static Routes

Well, some of you may be wondering by now why I have not mentioned anything about static routes. I have not forgotten ! Use of static routes is for scenarios in between. If you have a well-known subnet beyond a router, then you should certainly add a static route for that. For example, if one of the machine connected to bond1 network also knows about another network and has routing capabilities, you can use static route through that.

Ok, so that is all for this post. As always, your comments are most welcome. Thanks !

Join the discussion

Comments ( 2 )
  • guest Wednesday, January 2, 2013

    Wonderful Article.

  • Juan Thursday, September 26, 2013

    Great! worked perfectly

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.