A reference architecture for Oracle Cloud Infrastructure with two network appliances
Transparent routing can simplify appliance scaling
Network appliances often sit in the middle of application paths. A firewall, software-defined wide area network (SD-WAN) function, packet inspection engine, or routing service may need to see the original client and destination addresses before it can make a decision. Conventional load balancing can mask some of that information through network address translation and can make it harder to keep both directions of a stateful flow on the same appliance.
A private Oracle Cloud Infrastructure (OCI) Network Load Balancer (NLB) can operate transparently in that path. It selects an appliance for each flow and preserves the packet headers as the appliance forwards traffic to the destination. That gives teams room to scale an appliance tier without tying each workload to a particular firewall or router.
The architecture routes traffic through two appliances
Traffic begins in the client subnet and is addressed to the application, not to an NLB listener. A route rule sends that traffic to the private NLB, which selects one of the two appliances and passes the original packet to it. The appliance then forwards the packet to the target service. Return traffic follows the same pattern because the application subnet also uses the NLB as its next hop to the client subnet.
For the service model and transparent routing behavior, see the Oracle Cloud Infrastructure Network Load Balancer overview and the documentation for source and destination preservation.
Figure 1 Two appliance transparent NLB architecture
Every layer has a role in the return path
The VCN, the appliances, and the NLB have to work together. A healthy backend and a working listener are not enough if the route tables and security rules do not support a symmetric transit path.
| Layer | Configuration | Why it matters |
| VCN routing | Route client-to-application and application-to-client traffic to the NLB private IP. | Both directions pass through the same NLB decision point. |
| Appliance VNIC | Disable source and destination checking on every appliance VNIC. | OCI otherwise drops packets not addressed to the appliance itself. |
| Appliance routing | Enable packet forwarding and configure the vendor equivalent of reverse path validation. | The appliance forwards preserved traffic instead of treating it as local. |
| Security | Allow application flows, appliance transit traffic, and the NLB health-check port. | Header preservation means rules must permit the real source and destination flow. |
Each appliance needs its vendor-supported forwarding mode enabled, along with the appropriate reverse path validation settings for transparent traffic. In OCI, the VNIC source and destination check must be disabled. A health endpoint can listen on TCP port 8080 while the application uses TCP port 5201. Using separate ports makes it easier to tell a health-check failure from a packet-forwarding failure.
A TCP health check only shows that something is listening on the configured port. It does not show that the appliance can still forward packets to the target subnet. In production, the health endpoint should reflect dataplane readiness by checking forwarding, required routes, and the availability of the appliance policy engine.
The NLB settings that shape traffic behavior
A small set of NLB settings determines whether this behaves like a transparent appliance tier or a conventional NAT-based load balancer.
| Parameter | Reference setting | Behavioral effect |
| Source and destination preservation | Enabled on the private NLB | Original client and target IP addresses and ports are retained. |
| Symmetric hashing | Enabled | Both directions of a flow are sent to the same appliance. |
| Load balancing policy | Five tuple hash | New connections can spread across appliances while an existing connection remains sticky. |
| Health check | TCP 8080 | A failed readiness check removes the appliance from new-flow selection. |
| Interval timeout retries | 1 second 800 milliseconds 1 retry | Speeds up detection but can create false failovers if the endpoint is unstable. |
| Instant failover | Enabled | Existing traffic can be moved after the selected appliance becomes unhealthy. |
| Fail open | Disabled | Unhealthy appliances are not selected when every backend has failed. |
| TCP reset on failover | Enabled when an L3 IP listener is used | The failed connection is closed so the client can reconnect through a healthy appliance. |
Hashing can be easy to misread during a test. One TCP connection stays on one appliance for its lifetime. Separate connections use different source ports, so the NLB can distribute them across healthy appliances. The result is hash based, not round robin, so an exact fifty-fifty split is not expected.
Failover deserves a separate expectation. When the NLB marks an appliance unhealthy, it can redirect existing traffic. For TCP, that move can include a reset, after which the client opens a new connection to a healthy appliance. If the appliance is a stateful firewall, session synchronization is still needed to preserve the application session across appliances.
TCP reset is available only with L3 IP listeners. It is useful when ending a connection cleanly is preferable to leaving the client waiting on a flow that cannot continue. It does not preserve a stateful application session, so the client must reconnect.
The same pattern can support other network functions
This pattern can place a scalable firewall tier between application segments, steer private ingress through an inspection service, or support an active-active tier for SD-WAN and network monitoring. It also gives teams a way to validate how a vendor appliance handles original packet headers, asymmetric failures, and session state before it is introduced into a production path.
Before using this pattern in production, validate the appliance dataplane, health model, and return path in a controlled environment. Then review the OCI guidance for Network Load Balancer health checks before selecting a production appliance design.