BGP Session Up, Customer Path Broken: MPLS over DMVPN

The peering has been up for weeks, the prefix count looks correct, and one customer's traffic still takes the long way around or never arrives. Two different things are being judged by the same indicator. A BGP session reports that two peers can exchange messages over a TCP connection; it says nothing about whether the overlay can carry a labelled packet for that particular customer.

That gap is easiest to see in an MPLS-over-DMVPN design, where a hub path and a direct spoke path can behave differently for reasons that have nothing to do with the peering.

What the session actually proves

BGP runs over TCP, so each peer needs a route to the other endpoint address before anything else can happen. Pick a loopback as that endpoint and you have created a reachability requirement, not a forwarding guarantee. The session can then stay up through an interface that the customer's traffic would never use.

Record four facts at both ends before drawing conclusions: the session's source address, its destination address, the routing context it sits in, and which interface the kernel or router would choose for that pair. Those facts describe the control plane. Customer packets add a VRF, a label stack and an overlay mapping on top of them.

Four mechanisms, not one

Cisco's documentation for MPLS over DMVPN separates the pieces cleanly. MP-BGP distributes customer overlay labels between provider edge devices. MPLS uses those labels to select the forwarding context for a packet. NHRP maps an overlay address to the transport endpoint that a dynamic tunnel needs. IPsec protects the peer path where it applies.

Any of the four can be right while another is wrong, and the combination is what carries traffic. A route exchange can complete while the label binding for one VRF is missing, while the NHRP mapping points at a stale transport address, or while the tunnel exists but no policy permits the inner packet. None of those conditions bounce the session.

Compare the hub path with the shortcut, one variable at a time

Fix the source host, the destination host and the VRF, then trace the same flow twice. On the hub path, note the next hop, the service label that identifies the remote customer context, and the overlay-to-transport mapping in use. On the intended direct path, note the same three things. The difference between the two lists is the fault, or the absence of one.

Hold the flow constant during the comparison. Changing the source host, the VRF or the destination between observations produces two unrelated stories, and it is the most common way this exercise goes wrong.

Keep loopback tests in their own compartment

A ping sourced from a router loopback is a useful test, and it exercises a different case than a host flow does. The source address belongs to the router, its routing context may be global rather than a VRF, and the reply path can differ from the one a customer prefix would take. So “the loopback answers” and “this application works over the shortcut” are separate results, and only the second one describes the customer.

Test both, then label them separately in your notes. When they disagree, you have learned something specific about where the design diverges, which is more than a single green checkmark will ever tell you.

Version claims and defect claims need their own evidence

Field reports sometimes recommend changing a host mask or a BGP source address, and those reports are worth reading only alongside the defect they were written about and the releases it affected. A workaround from one release is not a statement that every /32 prefix is unsupported by the architecture.

Use documentation that matches your DMVPN phase, platform and release, and remember that one chapter can cover several arrangements, including FlexVPN. An example from one arrangement does not transfer silently to another. Choose session endpoints for the control-plane design you actually need, then verify the customer forwarding path separately, because rewriting every loopback mask is not a substitute for finding out which job failed.