MTU Black Hole: Why Small Requests Work but Uploads Stall

A page loads, a small API call returns, and then a file upload hangs until the client gives up. Nothing in the server log explains it, because the packets were dropped in the middle of the path without a useful answer being sent back. That pattern is an MTU black hole: a link narrower than the sender assumes, plus a silent refusal to say so. The size that fails is repeatable, which is what makes it diagnosable.

What Makes the Hole Black

Every link has a maximum transmission unit, and a packet larger than the next hop's limit must either be fragmented or rejected with an ICMP message that says so. Path MTU discovery depends on that message arriving and being honoured. Filtering it, or ignoring it, leaves the sender retrying a packet that can never fit.

Encapsulation makes this common around VPNs, because a tunnel adds an outer header to a packet that was already sized for the physical link. The inner packet is legal at 1500 bytes and the encapsulated one is not, so it dies at the boundary while small packets sail through. A completed handshake and a working login page prove only that small packets work. The largest packet the application tries to send is the one that matters.

Find the Size Where It Breaks

Binary-search the payload size with a do-not-fragment test, from a host on one side of the tunnel to a host on the other. The largest payload that succeeds is your usable path MTU minus the header overhead.

ping -M do -s 1400 -c 2 192.0.2.20
ping -M do -s 1200 -c 2 192.0.2.20
ping -f -l 1372 192.0.2.20

A payload of 1472 bytes succeeds on a clean 1500-byte Ethernet path once the 28 bytes of IPv4 and ICMP headers are added. When the same test only succeeds at a smaller payload once the tunnel is up, the difference is the encapsulation overhead, and the tunnel boundary is where the rejection message should have been generated. Record both numbers, before and after connecting, because that pair is the proof. Repeat the test for IPv6 when the application uses it, since the header is larger and the failure threshold moves.

Read the Evidence Before You Blame the Tunnel

Remote application errors, congestion and ordinary loss also stall uploads, so one failed transfer does not identify a black hole. What separates them is the size relationship: a repeatable success at 1200 bytes and a repeatable stall at 1400 bytes, in the same direction, through the same endpoint.

A capture at both ends shows the large packets leaving and never arriving, with no rejection in between. That is the moment to look at the boundary device instead of the server, which is answering every request it actually receives. Congestion stalls too, but it stalls different sizes at different times, which is a different signature.

Check the Repair on Both Transports

The supported repair is to let the boundary report the real limit, or to clamp TCP's maximum segment size there so no oversized segment is attempted. Lowering every MTU on the network, or disabling the firewall, hides the symptom and leaves a configuration nobody can reason about later.

ip link show dev tun0
sudo nft list ruleset | grep -i -E 'mtu|mss'
sysctl net.ipv4.tcp_mtu_probing
ping -M do -s 1400 -c 2 192.0.2.20

Verify with the same size test, then with the application that failed, and check a second protocol, because UDP applications never receive TCP's segment-size correction. Keep the original values in the ticket, since a repair that cannot be reversed is a second problem waiting to happen. If the size test passes and the upload still stalls, the black hole was not the cause, and the investigation should move on rather than widen the change.