How IPsec NAT Traversal Wraps the ESP Inside UDP

Two clients behind the same router talk to one IPsec gateway, and one keeps dropping while the other stays up. Native ESP is IP protocol 50 with no ports, so a translator has nothing to key a mapping on when several clients share one address. NAT traversal closes that gap by wrapping ESP in UDP on 4500, and that header explains most of the symptoms that follow. It also points at the device to change.

Why the data plane moves to port 4500

During setup IKEv2 compares the addresses and ports each side believes it is using, by exchanging hashes in its NAT detection notifications. When the hashes disagree, a translator sits somewhere in the path, and both peers switch the protected traffic to UDP encapsulation. That is why a session that faced a translator looks different from one that did not.

The wrapper is an envelope, not a protocol change: ESP keeps its header, integrity check and encryption, and UDP supplies only the ports the translator needs. Port 500 carries the IKE exchange; 4500 carries the encapsulated data after detection, and often the later IKE messages too. Neither port makes the flow a TCP connection, the assumption that misleads most people reading a capture.

Check the two stages in order

Prove the control exchange first, then the data path, and keep the two apart in your notes. A capture that shows traffic on 500 and never on 4500 usually means the data path is filtered, so the fix belongs in the firewall rule rather than in the client.

Watch both ports with tcpdump -ni any 'udp port 500 or udp port 4500', and read the translation table on the router while a session is live. That table shows the source port the gateway will actually see, and a capture on the laptop disagrees by design: the laptop believes 500 while the gateway sees 4500 with a translated source port, and both are correct. The same flow, before and after translation:

client 192.0.2.10:500 -> router 203.0.113.7:62000 -> gateway 198.51.100.9:4500

Symptoms the wrapper explains, and two it does not

A router that permits the control exchange and blocks the encapsulated data produces the tunnel-up-nothing-throughs report. Mapping tables expire, so an idle session needs keepalive traffic the peers send on purpose, and those packets are not health checks. When several clients behind one gateway fail together, examine that gateway translation and any legacy passthrough feature. The wrapper does not explain an application that fails for every client after the tunnel is up, or a certificate warning on a block page; both live above this layer.

Read the mapping table for two clients at once: if both show the same translated port, the device is not separating them the way the gateway requires. A passthrough mode that hands native ESP to just one host is not equivalent to serving several translated clients, although vendors describe both as IPsec friendly. Double NAT adds a mapping layer and another expiry point, but it does not automatically break outbound connections.

Keepalive traffic deserves its own line in the runbook, because it is what holds the reply path open through idle periods. Enable it where the vendor documents it, and check it first when a session dies after minutes of silence rather than on first use.

Change the owner of the mapping, then re-test

Follow the supported NAT-traversal settings on both peers instead of guessing at a port. Editing the local source port on one client cannot change the service listening at the far end, and it usually breaks the mapping already built. If the client sits behind a device you do not control, the change belongs to its owner.

After any change, repeat both stage checks and confirm a data packet crosses, not merely that the status screen turned green. Keep the tuples, the mapping table output and the rule change in one note, so the next person does not restart from the control exchange. The strongSwan NAT traversal notes cover the encapsulation and keepalive settings behind both stages.