Unattended VPN Server Updates Need a Recovery Path You Tested
An update can restart the very service you use to reach the server. Compare an out-of-band console with a tunnel-only recovery plan before automating.
Unattended updates are a reasonable policy for a server you reach over its own VPN, and they are a single point of self-inflicted failure. The package operation restarts the service, the tunnel drops, and the administrative path you were using disappears with it. Two recovery models exist, and the choice between them depends on one condition: whether you have a route in that does not traverse the thing being updated.
Two recovery models, one deciding condition
The first model trusts the update and assumes you can reconnect afterwards. It works when the VPN is one of several ways to reach the host and when the failure of the tunnel does not remove your ability to diagnose it. The second model pre-positions an independent route, usually the hosting provider's console, and treats the tunnel as the thing most likely to be broken by maintenance. Both models are defensible, and only one of them requires work in advance.
The deciding condition is simple to state and easy to skip. If every administrative path to the machine runs through the service being updated, you have no recovery model at all, only an expectation. Say that out loud in the change review.
Testing that condition takes ten minutes and has to happen before the first unattended update, not during the incident. Open the console while everything works, confirm the credentials are current, and close it again. Confirm the console opens to the host you think it does. Then confirm the login you would actually use at 3 a.m. still works.
That last part matters more than it sounds. A console account whose password was rotated six months ago and never verified is not a recovery path; it is a document describing one.
What a backup does not restore
A configuration archive is a record of files, and an update changes more than files. Package state, kernel modules, enabled unit links and firewall rules applied through a separate tool all live outside a config directory, and a restore that ignores them produces a machine that looks configured and does not forward.
| What changed | Restored by a configuration backup | Needs its own action |
|---|---|---|
| Service unit or startup ordering | Partly, if the unit file was included | Package state and enabled links |
| Kernel module or interface support | No | Kernel or module rollback, or a boot-time decision |
| Peer or device authorization | Only if the backup predates the revocation | A deliberate decision about which peers return |
| Firewall rules applied outside the config | Only if they were captured | Replaying and verifying each rule |
Authorization deserves its own line. A backup taken before a compromised device was removed will happily restore that device's access. Rolling back to it during an outage is how an incident becomes two incidents, and the decision to restore should be made about peers as explicitly as it is made about files.
The mechanism of the rollback matters too. Copying an older executable over a package-managed one is not the supported reversal of a package change, and it leaves the package database describing a state the filesystem does not match. Use the package manager's own downgrade path or the snapshot the provider offers. A downgrade that the package database records is one you can repeat.
None of this argues against backups. It argues for knowing which layer each artifact covers, because the failure mode of a partial restore is more confusing than the original outage. Write the layer next to each backup in the runbook.
Checkpoint, update, then verify from outside
Record the state before the change in a form you can compare afterwards: the installed versions, the service units involved, the listening sockets and the current peer list. Four commands cover most of it on a Linux host.
systemctl list-units --type=service | grep -E 'openvpn|wg-quick'
ss -lunp | grep -E ':51820|:1194'
journalctl -u wg-quick@wg0 --since '-10 min'The first shows which units exist and whether they are running, the second confirms the listener is actually bound, and the third gives you the service's own account of the last few minutes. None of them proves a client can connect, which is why the next step happens from outside the host. Keep those readings as your comparison baseline.
Verify from a network that does not use the tunnel: authenticate one authorized client, resolve a name, and reach one intended application. A local status line saying the interface is up is a statement about the interface, not about routing, keys or the policy that admits a peer.
Then test the reconnect path deliberately. Restart the service once and confirm the tunnel comes back without manual intervention, because a service that only starts correctly at boot is a service that will surprise you at the worst moment.
When it fails, use the path you tested
The recovery sequence is unglamorous and it works: reach the host through the independent route, read the service's own logs, and restore the specific checkpoint that corresponds to the failure. Change one thing per attempt so the next log reading means something.
Resist the urge to make several unrecorded changes to get connectivity back. A machine fixed by five simultaneous edits cannot be explained afterwards, and the next update will reintroduce whichever of them was actually load-bearing.
Then decide the permanent policy with evidence. If the update exposed a dependency nobody documented, the fix is documentation and a test, not disabling updates. If it exposed a service that cannot restart cleanly, the fix is in the unit, not in the schedule.
The comparison resolves cleanly once the condition is named. An independent route makes unattended updates a manageable risk. Their absence makes the same policy a scheduled outage with an unknown recovery time, and no amount of automation changes that arithmetic.