Tunnel Still Up, Route Gone: Separating Two VPN Failures

One host goes unreachable while the tunnel interface still reports itself as established, and the two observations are not in conflict. Something removed the path the traffic was using: a neighbor withdrew its prefix, a tracked static route was pulled, or a different route became preferred. The security association can stay perfectly healthy through all three. Separating the events means asking which layer changed first rather than whether the VPN failed.

Three stages that look like one event

A routing decision has three stages. The prefix is received, it is selected as best, and it is installed in the forwarding table. Each stage can fail on its own, and the failure looks identical from the application side. A prefix that was learned but lost selection is indistinguishable from one that never arrived when the only evidence you look at is the forwarding table.

That is why the first question is which stage broke, not which protocol is at fault. The answer decides who owns the incident and which tool the next check uses.

Read the installed route first

Ask the kernel where traffic for the failing destination actually goes. ip route get prints the next hop, the interface and the source address the kernel would use, and it answers for one destination rather than for the whole table. If the answer names the tunnel, the forwarding table still believes the path exists and the problem sits above it.

ip route get 10.20.5.10
ip route show 10.20.0.0/16

An answer of "unreachable" means no installed route at all, which points at a withdrawal or a missing install. An answer naming a physical interface means the traffic left the tunnel without anyone deciding that it should, and the next check is the preference that won. An answer naming the tunnel while packets still disappear points further down the path, at filtering or at the far side.

Then look at the protocol's own view

A routing daemon keeps more than one table, and the interesting comparison is between them. For BGP, the neighbor session state, the received advertisement and the installed route are three separate pieces of evidence. A prefix can sit in the received table and never reach the kernel because a route with a shorter administrative distance wins. Check all three before concluding that the other side stopped advertising.

vtysh -c "show bgp summary"
vtysh -c "show bgp ipv4 unicast 10.20.0.0/16"

The summary tells you whether the session is established, the per-prefix view tells you whether the prefix was accepted, and the kernel lookup tells you whether it was installed. Reading them in that order keeps a session flap from being mistaken for a withdrawal, and a withdrawal from being mistaken for a policy rejection.

When the clocks disagree

Put the tunnel log, the routing log and the first application failure on one timeline using their real timezones. If peer loss comes first, the tunnel took the route down with it. If the withdrawal comes first, routing changed and the tunnel reacted afterwards. Those are different incidents with different owners, and the order is the whole finding.

One minute of clock skew is enough to reverse that conclusion, so confirm the timezone and the offset before trusting the sequence. Devices that log in local time without an offset are the usual source of a confident but wrong timeline.

Recovery without a loop

Two paths can be eligible after a withdrawal: the one the protocol selects, and the one you pin by hand. Only the first is a decision the network keeps making for you, which is why a manual route is a stopgap rather than a recovery plan. An alternate announcement also does not preserve firewall or NAT state for a session that is already established, so new flows may work while the original one stays broken. Confirm which backup should become active and whether the remote site will accept a return path through it.

Do not install a permanent static override before you understand why the withdrawal happened. That shortcut can pin traffic to a failed path or quietly bypass a deliberate routing policy, and it buries the original cause under a new configuration. Retest the application once the route is restored, and confirm the chosen prefix has a working data path rather than only an entry in a table.