OpenVPN TCP Stalls: rto_min Does Not Fix Head-of-Line

An interactive shell over the tunnel freezes for several seconds while a file copy runs, then resumes as if nothing happened. The download is the only thing with a problem, yet the shell waits too, and both recover at the same moment. That pattern is the signature of one reliable stream carrying many conversations, and it survives almost every tuning change people try first.

The symptom is worth reading carefully because it points at a mechanism rather than a setting: the pause is shared, so the cause is shared, and the shared object is the outer connection. Everything else in the log looks normal while it happens.

The symptom: everything pauses at once

Two inner connections stalling together is the clue. A slow download and a slow shell are separate TCP conversations with separate sequence numbers and separate congestion state, so a fault in either one should not touch the other. When they pause in lockstep, something above them is ordering their bytes.

That something is the tunnel's transport. In TCP mode, OpenVPN writes the encapsulated packets into one TCP connection to the peer, and TCP's contract is in-order delivery of a single byte stream. Later bytes are not handed to the application before the missing ones, no matter which inner conversation they belong to.

Note what the symptom does not tell you. It does not say the path is broken, that the server is overloaded, or that the cipher is slow. Those produce different shapes: uniformly low throughput, timeouts, or failures that affect one direction. A shared pause that ends when one lost segment is retransmitted is specifically an ordering effect.

One outer byte stream, many inner conversations

Picture the tunnel carrying a download and a shell at the same time. A segment of the outer connection is lost. The following segments may arrive at the receiving host, and the receiving TCP stack will acknowledge them, but it cannot deliver them upward until the gap is filled. OpenVPN therefore has nothing to read for the shell, even though the shell's own bytes arrived long ago.

This is head-of-line blocking inside one connection, and it does not require the two applications to share an inner connection. They share the outer one, which is enough.

The second-order effect makes it worse. An inner sender that sees no acknowledgement may retransmit data the outer layer is already trying to deliver. That retransmission becomes additional tunnel traffic on a path that just proved it can lose segments, so the recovery work competes with the recovery that matters.

None of this guarantees a collapse. On a clean path with short round trips and generous queues, the arrangement can be perfectly usable for years. The failure appears when loss, delay and queue depth line up, which is why the reports cluster around congested uplinks and mobile links rather than around well-provisioned offices.

Why a longer inner timer cannot help

Raising the minimum retransmission timeout on a route, with rto_min, postpones timer-based retransmission for locally generated TCP traffic that uses that route. It is a real knob with a real effect, and it addresses the second-order problem rather than the first.

ip route show
ip route show cache 2>/dev/null | head
ss -tin dst 10.70.0.1

What it cannot do is release outer bytes out of order, recover the missing segment faster, or make the remote application server adopt the same timer. The receiving stack still delivers in order, and the sender at the far end still uses its own round-trip measurements.

It also does not disable every other loss-recovery mechanism. Where genuine loss requires timer-based recovery, a larger minimum makes the application wait longer before it acts, which is the opposite of what a latency-sensitive session needs.

A fixed floor of twelve seconds appears in advice on this topic as though the existence of a TCP tunnel implied it. The retransmission timer described in RFC 6298 derives from measured round-trip time with exponential backoff, so a constant floor is a policy choice for a specific path rather than a protocol requirement. Applied broadly, it hides the symptom while changing recovery for every flow that route covers.

What tcp-nodelay does and does not do

The tunnel option that disables Nagle-style buffering on the transport socket has a narrow purpose: it stops small writes from waiting for more data before transmission. That helps interactive traffic avoid an extra delay on an otherwise healthy path.

It does not turn the transport into an unordered one, and it does not shorten recovery after a lost segment. A shell that pauses for a retransmission timeout will still pause with that option enabled, because the option never touched the ordering rule.

Confusing the two is easy because both are described as latency fixes. One removes an artificial wait introduced by buffering; the other would require changing what the transport guarantees, which no socket option does.

How to compare transports fairly

If the operator permits it and both services exist, compare a UDP service and a TCP service carrying the same workload over the same access path to the same server. Changing country, provider or server generation between the two tests introduces a variable that will dominate the result.

Watch for the shared pause rather than for a throughput number. Several inner applications stalling together during outer loss is the observation that identifies head-of-line blocking; a single lower speed-test figure does not.

Authorized captures on the tunnel interface and on the physical interface separate the two layers cleanly, because inner and outer retransmissions belong to different connections with different sequence spaces. Seeing inner retransmissions while the outer stream shows a gap is the direct evidence.

Switching to UDP removes the extra reliable stream and leaves inner TCP to do its own recovery. It does not promise zero loss or low latency, and a saturated uplink, oversized queues or an MTU problem will still need fixing. Traffic outside the tunnel is never held behind the tunnel's sequence gap, though it can still suffer when both paths share one congested link, and that shared bottleneck deserves its own investigation rather than being folded into this one.