Why L2TP/IPsec Needs UDP 1701 as Well as IPsec

Firewalls that allow IKE and ESP but not UDP 1701 break L2TP connections in a way that looks like a credential problem. The reason is that L2TP is a second protocol travelling inside the first. IPsec carries and authenticates the exchange, and L2TP runs its own tunnel over UDP 1701, which the local stack still has to deliver. Knowing where each header appears explains which device needs which rule.

Two Protocols, Two Port Sets

An L2TP/IPsec connection uses IKE on UDP 500, NAT traversal on UDP 4500 where translation is present, and ESP as IP protocol 50 where it is not. L2TP itself is a tunnel protocol on UDP 1701, and it runs after IPsec has authenticated the peer. The two halves fail differently, and the log lines reporting them come from different components. A rule set covering only the first half produces a connection that negotiates and then dies at the tunnel stage.

Port numbers describe transport endpoints, and ESP is not a port at all. That is why a firewall interface offering only TCP and UDP rules cannot express a complete IPsec policy. Protocol 50 has to be allowed as a protocol, and a rule that mentions only UDP will never match it.

The Wrapper Explains the Port

On the wire, an L2TP/IPsec connection normally shows IKE on 500 and encapsulated ESP on 4500. The UDP 1701 header sits inside the protected payload, which is why a capture on the outer interface does not show it. The local host still has to deliver that datagram to the L2TP daemon, and delivery happens after decapsulation. The 1701 rule therefore belongs on the endpoint's own firewall even when no middlebox ever sees the port.

The order of processing explains most of the confusion. If the stack applies the IPsec policy before the host filter, the decapsulated L2TP packet is matched as UDP 1701; if the order is reversed, the outer ports are what the rule set sees. Both behaviours exist in practice, which is why the same rule set appears to work on one platform and fail on another.

Check the Path in Order

Capture the outer interface first and confirm that IKE and encapsulated ESP are moving in both directions. Then check the local firewall counters for the decapsulated traffic. The order matters, because a capture showing nothing at all means the problem sits upstream of the host.

sudo tcpdump -ni eth0 'udp port 500 or udp port 4500'
sudo nft list ruleset | grep -i -A2 1701
sudo ss -lunp | grep 1701

The capture confirms the first half of the exchange and shows whether the peer is answering. The rule listing shows whether a 1701 rule exists and where it sits relative to any drop policy. The socket listing confirms the L2TP daemon is bound and listening, ruling out a service that never started. Read all three before changing a rule, because a missing listener and a filtered port produce identical complaints from the user.

What a Port Scan Cannot Show

An external scan of UDP 1701 against a correctly configured server usually reports nothing, and that is the expected result. The port is not exposed to the internet; it is reached only after IPsec has authenticated the peer. Treating the closed result as a fault leads to opening a port that should stay closed. The scan cannot see inside the tunnel any more than it can see the application traffic.

Test with a real client instead, and read the stage it reaches. If the connection negotiates and then stalls, the L2TP daemon or the local filter is the suspect; if it never negotiates, the outer ports are. That single split removes most of the guessing about which rule to add, and the protocol layering makes it predictable rather than mysterious.