Configuring IP Filter Support for Failover Services with Solaris Cluster 3.2
By hhnguyen on Mar 05, 2007
For information on the IP Filter feature, I suggest you look at the Solaris 10 System Administration Guide.
This write-up is not in current Solaris Cluster 3.2 current documentation, but we will get documented with our next patch.
IP filter in Solaris 10 (FCS to up to Solaris 10 11/06 (update 3)) exists as a STREAMS module that must reside right below IP in any stream. The network/pfil and network/ipfilter SMF services autopush pfil module (based on /etc/ipf/pfil.ap) and loads in filtering rules (from /etc/ipf/ipf.conf) at boot time. Filtering rules can be based on interface, IP addresses/subnets, protocols, and ports.
NOTE : Scalable services are not supported with ipfilter at this time. These steps are only to configure ipfilter for failover services. For enabling cluster wide filters ensure the configuration procedure is replicated on all nodes
Edit /etc/iu.ap so that lines corresponding to public NIC devices have "clhbsndr pfil" as the module list. The nodes should be rebooted for
the change to take effect. Nodes can be rebooted in a rolling fashion.
Note that "pfil" must be the last module in the list.
ex : # scstat -i -h `uname -n`
-- IPMP Groups --
|IPMP Group: node1||sc_ipmp1||Online||ce2||Online|
|IPMP Group: node1||sc_ipmp1||Online||bge2||Online|
|IPMP Group: node1||sc_ipmp0||Online||ce0||Online|
|IPMP Group: node1||sc_ipmp0||Online||bge0||Online|
/etc/iu.ap will have these lines modified as below ( for ce/bge )
ce -1 0 clhbsndr pfil
bge -1 0 clhbsndr pfil
Add filter rules to /etc/ipf/ipf.conf on all nodes as needed. See ipf(4) manpage for more information on IP Filter rules syntax.
ex: block in quick on bge0 from 220.127.116.11/23 to any
block in quick on ce0 from 18.104.22.168/23 to any
Enable the ipfilter SMF service.
svcadm enable /network/ipfilter:default
Cluster fails over network addresses from node to node. No special procedure or code is needed at the time of failover.
Please make sure filtering rules that reference IP addresses of LogicalHostname and SharedAddress resources are identical on all cluster nodes.
Rules on a standby node will reference a non-existent IP address. But this rule is still part of IP filter's active rule set and will become effective once the node receives the address after a failover.
In addition, rules must be set up that apply uniformly to all NICs in the same IPMP group. That is, if a rule is interface-specific, the same rule must also exist for other interfaces in the same IPMP group.
In non-cluster mode, the line in /etc/iu.ap containing clhbsndr would not take effect because clhbsndr is not registered. But subsequent autopush setup by SMF service network/pfil (which fails in \*cluster\* mode) would succeed. This ensures that pfil is also pushed in non-cluster mode.
The user must also update /etc/ipf/pfil.ap in addition to /etc/iu.ap. Updates to pfil.ap is slightly different. Refer to IP Filter documentation for more details.
IP filter does not support stateful filtering with IPMP since outgoing packets for the same session can go through multiple interfaces. Sun Cluster requires the use of IPMP groups and hence inherits the same limitation.
Thus, only stateless filtering is supported with clustering.
NAT in routing mode is not supported. That is, a cluster node must not be set up as a router/gateway to forward packets to and from another node. This is because such setup easily creates a single point of failure for the whole cluster without first making the routing mechanism HA.
Use of NAT for translation of local addresses is supported. NAT translation rewrites packets on-the-wire and hence is transparent to cluster software.
Note: NAT rules containing IP addresses that are managed by clustering (e.g. LogicalHostname resources) must be replicated on all cluster nodes.
IPFILTER on Cluster transport
If you have the same type of adapter for private and public network, the edits made to /etc/iu.ap can result in pfil getting pushed on private network streams. But the cluster transport module would remove all unwanted modules at stream creation. Hence pfil would be removed. No special user procedure is required.