Git Stalls Without the VPN? OpenSSH IPQoS Is Marking Packets

A clone from the office works, the same clone over a phone hotspot works, and the same clone from a hotel network hangs until a VPN is switched on. The SSH handshake finishes and the session goes quiet on the first real transfer. One line added to the client configuration cures git and ssh together. When a single client setting repairs several tools at once, the fault is in what the client sends rather than in the link it sends over.

Read the symptom as a routing difference

A tunnel that cures a problem is rarely a repair. It swaps the route, the carrier and the middleboxes, while the laptop keeps producing exactly the same packets. If one route mishandles a header and another passes it through untouched, the difference lives in middlebox policy rather than in the client. The tunnel is a witness here, not a fix.

The point where the session dies narrows it further. Nothing that fails at name lookup or at key exchange looks like this. A session that authenticates, prints a banner and then freezes on the first bulk write has already proved that routing, naming and credentials are all working.

The client stamps a QoS byte on every SSH socket

Every OpenSSH connection from 7.8 onward carries a value in the IP header that routers may use to prioritise traffic. Terminal sessions get one value and bulk copies get another, on the theory that a human typing deserves better treatment than a file transfer. Most networks ignore the hint completely, which is why it goes unnoticed for years. A minority of Wi-Fi drivers, mobile carriers and VPN concentrators act on it, and some of them act badly.

When that value is rewritten or discarded mid-stream, the connection does not fail, it stops progressing. Retransmissions climb, the window collapses, and the application reports nothing because nothing errored. That silence is why people describe the symptom as a hang rather than as a failure.

Confirm the client build first with ssh -V. Anything at or above 7.8 carries the behaviour by default.

Three commands that separate this from MTU and DNS

Two lookalikes deserve elimination before you edit any configuration file. An MTU fault lets small exchanges through and freezes on the first large packet, while a naming fault never produces a session at all.

ssh -o IPQoS=none -v [email protected]
ssh -G [email protected] | grep -i ipqos
sudo tcpdump -ni any -vv "tcp port 22"

The first command repeats the failing operation with the marking switched off, and a successful run is the strongest single piece of evidence available. The second prints the configuration the client will actually apply to that destination, which catches a stale entry in a file you had forgotten about. The third shows the TOS byte in both directions while the transfer is stuck, and the interesting pattern is a value on the broken route with none on the working one.

Run the capture on the laptop rather than on the server, because the sender sets the value and either direction can alter it. Compare a working route and a failing route with the same capture, and the difference appears without guesswork.

Put the exception in one file, one host at a time

An exception belongs beside the host it applies to, with a comment that says why it exists.

Host gitlab.example.internal
    IPQoS none
    # direct route rewrites the marking; the tunnel route passes it

Scoping matters more than the value itself. A wildcard entry switches the behaviour off everywhere, throws away a hint that helps on well-behaved networks, and leaves nobody able to say when it appeared. A per-host entry is readable, reversible and trivial to delete once the network is repaired. Use the user file for a personal workaround and a managed file when a whole fleet needs the same exception.

Be explicit about what the exception does not cover. It applies to SSH, so an HTTPS remote, a package mirror and a browser all sit outside it, and a failure in any of those needs its own evidence. On a chain through a bastion, every hop uses the client configuration, and the hop that stalls is the one worth scoping.

Close the loop with the network owner

Hand the finding over as four facts: client version, route tested, VPN client in use, and which direction carried the marking. Those four are enough for a network owner to reproduce the treatment and decide whether it is deliberate.

If the exception does not cure the stall, remove it. A workaround that survives a failed experiment becomes folklore, and the next person spends an hour proving it does nothing. If it does cure the stall, the path is still wrong for every other tool that crosses it, and the repair belongs with whoever operates the middlebox.

The client change is a stopgap with an expiry date. Write that expiry into the comment.