Multi-NIC GCP VM: why replies leave the wrong interface

Attaching a second interface to a Compute Engine VM gives it another network attachment, not a symmetric return path. The guest still chooses an interface and a source address from its own routes and rules. Google Cloud then applies VPC routing and firewall policy around whichever interface the packet left by. When those two decisions disagree, requests arrive on one NIC and replies leave by the other, and the branch sees a half-open conversation.

Walk the path in five steps and the disagreement shows up before you write a policy rule.

Step 1: map cloud NICs to guest devices

Cloud names interfaces nic0, nic1 and so on, and the guest sees device names that do not have to match. Resolve the mapping from instance metadata or from the interface's own address before anything else. The ordinary DHCP-provided default route is associated with nic0, which is why the internet-facing path so often wins by accident. Writing policy against a guessed device name is how a correct rule lands on the wrong interface.

Record addresses, device names and the VPC each one belongs to. That three-column map is the input to every later step.

Step 2: name the two route decisions

The first decision belongs to the guest: which interface and which source address for this packet. The second belongs to the cloud: what the VPC's routes and firewall policy permit around that interface.

A route visible in the VPN-facing VPC does not become a guest route selecting that NIC. The cloud route describes what the network can deliver, while the guest route decides what the packet does.

Step 3: pick a return-path policy

For a small, stable set of branch prefixes, explicit guest destination routes are the simplest expression: replies to those prefixes leave by the VPN-facing interface. When several services intentionally use different local source addresses, source-based rules that select a table per interface keep each reply in the right context. The multiple-interface documentation describes the NIC naming and default-route behaviour these designs rely on.

Each table needs valid connected routes and next hops, plus the destinations it will be asked for. A rule pointing into an incomplete table is not a return-path design. Use the documented gateway for that NIC instead of inventing an Ethernet gateway from the subnet's appearance.

Step 4: check the guest's own filters

Check where the server listens and which local source the reply carries. A wildcard listener does not override routing policy, so binding to all addresses changes nothing about the interface the answer leaves by.

On Linux, review reverse-path validation against the intended design. Disabling it globally as a first move trades a routing bug for a spoofing exposure, and it hides the asymmetry you were trying to find.

Step 5: test one flow per path

Verify a permitted branch request and an unrelated internet request separately, each with its real source and destination. Capture on both interfaces when you can, because the useful evidence is which interface carried the reply. Then confirm the cloud-side policy honoured the path you designed. One successful request proves one flow, not a symmetric design.

Keep the two tests apart in your notes, since they exercise different routes and different firewall rules.

What a second NIC does not give you

A second interface adds a network attachment and another address. It does not add automatic symmetric routing and it does not bypass firewall policy. Treating it as a routing fix is why multi-NIC designs accumulate rules nobody can explain later.

If the VM must forward other machines' traffic, plan that router role explicitly, including the cloud setting that permits forwarding and the guest filters that go with it.