Separate VPN Health Checks From Application Health Checks

Your tunnel process can be running while the protected connection is unusable, and a failing web page is not proof that the tunnel broke. When one probe stands in for all of them, a restart policy restarts the wrong component and calls it automation. Each check below names the layer it observes, the action its failure triggers, and the evidence that the action worked. The last step turns the whole set into a page someone can follow at three in the morning.

Step 1: name the layer each check observes

Ask what each probe can actually see before writing any of them. Liveness proves the daemon is up, a handshake or peer listing proves the relationship exists, a routed request proves delivery, and a lookup proves resolution. They fail in that order, and none of them implies the next one. A green process check sitting beside a dead tunnel is the normal case, not a contradiction.

Write the four checks down in that order, with the exact command and the expected output for each. A probe whose expected output you cannot state will be read differently by every operator who sees the alert. Put the same list in the alert text, not only in the runbook, so the person woken at night meets the layer before the symptom. Keep every command copy-pasteable on the host that runs it.

Then add the application check as a fifth, separate item. A page whose account service is down does not justify restarting the tunnel.

Step 2: tie each failure to one action

For every check, decide what restarts, what stays blocked, and what pages a human. A tunnel-layer failure can justify restarting the tunnel; a resolver failure points at the DNS integration on the client; an application-layer failure should almost never touch the tunnel.

State what a failure must not do, because the dangerous defaults are the permissive ones. An unhealthy status must never become permission for direct application traffic, since a fallback that quietly bypasses the tunnel removes the reason the tunnel exists. Narrow the actions as well: restarting a shared component because one destination is slow turns a local problem into a fleet-wide one. It also destroys the evidence you needed to find the cause.

Step 3: keep the probes harmless and cheap

Use small, read-only requests: a static resource, a health endpoint that reports its own dependency, or a TCP connect to a port you control. Avoid credentials, private browsing and anything that mutates state on every interval, because a probe that writes is a load test on a schedule.

Record the destination, the route expectation and the timeout for each probe. The route expectation is the field people omit, and it is what separates a tunnel failure from a routing mistake when you read the alert later. Derive the interval and the failure threshold from the recovery time you promise, and keep the timeout shorter than the interval. A probe still running when the next one starts reports one failure twice and looks like a flap.

Step 4: break each layer and watch the right probe go red

Stop the tunnel process and confirm the process check fails while DNS may still answer from cache. Change a route and confirm the routed request fails while the process and handshake stay healthy. Break DNS alone and confirm the application check reports resolution rather than a tunnel error. If two probes always move together, you have fewer checks than you think.

Test the recovery direction with the same discipline. After you restore the layer, the probe should clear without a manual restart, and the action that fired should leave no bypass behind. Watch one full cycle before trusting the green state, because a probe that only clears by hand hides a broken recovery path.

Step 5: put the runbook where the alert points

One page with five rows carries the whole design: layer, command, expected result, action on failure, owner. Add a line per probe on what it deliberately does not test. That line is what stops the next person from adding a sixth check that measures everything and distinguishes nothing. Link the page from the alert text, or nobody will open it while the incident is live.

In container deployments, a dependency condition shapes startup and nothing else. The Compose startup order and health condition documentation describes how readiness gates the order services start in; none of that enforces egress policy afterwards, and none of it fails closed when the tunnel drops. Continuous enforcement needs its own policy and its own probe, written on the same page.

The runbook is finished when the correct layer recovers without anyone reading an unavailable website as a broken tunnel. That tests the document, not the tunnel.