VPN Server Load Explained: Users, CPU or Uplink?

A gateway sitting at 0.4 load can still crawl for every user behind it. Load average counts tasks that are runnable or stuck in uninterruptible sleep, and on a four-core host a single saturated process is enough to push it toward 1.0 while three cores sit bored. Reading that number as headroom is the most common mistake in endpoint selection. What actually constrains a tunnel is usually one of four things: one busy core, the uplink, the queue in front of it, or a per-session policy nobody documented.

Load average counts tasks, not throughput

The kernel samples the run queue and the count of tasks waiting on I/O, then smooths both over one, five and fifteen minutes. Nothing in that definition mentions bytes, packets or tunnels. A host whose storage is wedged can report a high load with almost no traffic, and a host pushing a gigabit through a kernel data path can report a comfortable one.

Correlating load with tunnel performance therefore needs a per-core view. If one core is pegged, the aggregate percentage on a dashboard will still look fine:

cat /proc/loadavg
mpstat -P ALL 1 1
top -H -b -n 1 | head -n 20

The third command is the one that matters most on VPN hosts. Userspace packet processing in OpenVPN 2.x is largely serialised per process, so a single busy thread can become the ceiling while the dashboard reports plenty of idle capacity. If one thread holds most of a core during your slow period, the host is not under-provisioned in aggregate; it is constrained in the dimension that happens to matter.

Note the timing too. Fifteen-minute averages hide the burst that ruined a call, so a screenshot taken after the complaint is weak evidence. Sample during the problem, and keep the timestamp with the output.

The uplink and the queue in front of it

Interface counters answer the bandwidth question that CPU figures cannot. They also separate capacity from loss, which are different failures with different fixes:

ip -s link show dev eth0
tc -s qdisc show dev eth0
nstat -az TcpRetransSegs TcpExtTCPLostRetransmit

A link that is close to its provisioned rate during your test is the explanation, and no amount of spare CPU changes it. The qdisc line adds the detail that matters for interactive traffic: backlog and drops tell you whether packets wait in a queue before they leave. A queue that fills under a bulk upload produces exactly the symptom people blame on the remote server, sluggish calls on a connection that still benchmarks well.

Retransmission counters catch the case where packets leave but never arrive cleanly. If retransmits climb during the slow window and the interface is nowhere near capacity, the problem lives in the path rather than on the host, and moving to a different endpoint in the same city will not help.

Session counts describe demand, not headroom

Every public directory shows some number of connected users, and almost none of them tell you what those users are doing. Twenty sessions doing interactive work and four sessions pulling large files produce very different loads on the same hardware. The second pattern is invisible in a user count, and it is the one that ruins an evening. So a session number is best read as a hint about demand, never as a statement about spare capacity.

On a host you control, the status file gives the missing dimension. OpenVPN writes a periodic snapshot that includes per-client byte counters, which is enough to see whether one session is responsible for the whole uplink:

status /run/openvpn/server.status 10
# then, after at least one interval
grep -E "^CLIENT_LIST" /run/openvpn/server.status

Two samples ten seconds apart turn those cumulative counts into a rate per client. That is the measurement a user count is standing in for, and it is also how you discover that a single misconfigured client is saturating the link every night at the same hour.

Per-session limits hide inside policy

A server can be idle and still refuse to give you bandwidth. Rate limits, per-client bandwidth caps and traffic shaping are often applied per common name or per pool, so two users on the same gateway can get wildly different results. The mechanism is not subtle, but it is invisible from outside because the tunnel comes up normally in both cases.

Look for shaping on the tunnel interface rather than the physical one. Check whether the limit follows the user, the certificate or the address pool. If your test account lands in the uncapped pool, you will conclude the server is fine and the complaining user is wrong. That conclusion is worse than no conclusion, because it closes the ticket.

Compare two endpoints with one fixed task

The client-side experiment is still worth running, but only if the variables are pinned. Use the same device, the same access network, the same protocol and the same destination file, then repeat against a second endpoint at a similar distance. Anything that changes between runs, including the time of day, contaminates the result.

Record failures as carefully as successes. A series where three endpoints work and one times out is more informative than four speed numbers, because it points at a path or policy problem rather than a capacity one. Run each case at least twice; a single slow sample proves nothing on a shared network.

Finally, keep the numbers you gathered with the claim they support. "This endpoint handled the transfer in the same window that the other one stalled" is defensible, while "this endpoint is faster" invites an argument you cannot win with a homepage score. The OpenVPN 2.6 manual documents the status file and the shaping options if you need the exact directives for the host side.