Linux bond active-backup on EVPN all-active: what breaks

Put an active-backup bond on the server, put an all-active Ethernet segment on the fabric, and half the return traffic can vanish without a single error counter moving. Neither configuration is wrong on its own. The problem lives in the handoff between them, where one side picks a single port and the other assumes every attached leaf can deliver. Follow one frame in each direction and the mismatch becomes obvious.

What active-backup does to an incoming frame

Bonding mode 1 keeps exactly one slave in the transmitting path and swaps it only when the current one fails. That part is well known. What matters here is the receive side: the kernel hands a frame to the stack only if it arrives on the active slave, and drops frames that appear on any other slave. The all_slaves_active setting exists precisely to relax that rule, and it defaults to off. ip -d link show bond0 prints the current value next to the mode.

/proc/net/bonding/bond0 names the active slave and reports link state for each slave. That file is the only honest source for which port is currently carrying traffic.

A host in this mode has one usable receive path at a time, chosen locally with no knowledge of the fabric. If a frame for that host arrives on the other cable, the switch port counts it as received, the host stack never sees it, and every counter except the host's own drop counter looks perfectly healthy.

What all-active multihoming does to a remote next hop

EVPN multihoming describes a set of leaves that share one Ethernet segment, identified by an ESI, and advertises that segment to the rest of the fabric. In the all-active model, remote peers install more than one next hop for the segment's addresses. Unicast toward the server can then arrive at either leaf, and the fabric is free to choose per flow. That is the whole point of the design, and it is also the assumption the server bond has to meet.

Flooded traffic follows a different rule. The leaves elect a designated forwarder per segment, so broadcast, unknown unicast and multicast normally enter through one of them. Unicast and flooding therefore make two separate decisions, and only one of them shows up in a host's bond state.

Where the two elections disagree

The kernel chooses its active slave from link state and its own policy. The fabric chooses next hops from EVPN routes, and a designated forwarder from an election among the leaves. Neither process consults the other. When they pick different leaves the failure is direction-specific: what the server sends leaves normally, and the replies disappear.

Two supported ways out exist, and they are not interchangeable. Give the host a bond the fabric can treat as one logical attachment, which in practice means an LACP bond negotiated with the leaves, and both links are genuinely active together. Or accept single-active delivery from the fabric as well, so the segment follows the same one-port-at-a-time model as the bond. What does not work is an all-active segment in front of a bond that only listens on one port, unless you deliberately enable reception on the inactive slaves and accept the duplicate delivery that follows.

Why the backup cable keeps looking healthy

A cable carrying no traffic still reports link up, and a standby slave in a bond is designed to look ready. Operators read that as redundancy and stop asking questions.

Ask instead which physical port the return frame used. On the host, count received packets per slave rather than per bond, since the bond counter hides the asymmetry. On the leaf, look at the bridge forwarding entries for the server's MAC and at the next-hop set the EVPN route carries. If the entry points at the leaf whose host link is the standby, the design is delivering into a port the host ignores. That is a configuration mismatch rather than a hardware fault, and no amount of link-state monitoring will surface it.

Test delivery in both directions, not link state

Build a matrix with one row per failure and one column per direction, then fill in delivered traffic rather than interface state. Row one is normal operation, which is exactly where the mismatch hides, because the symptom looks like an application problem.

Then remove the active host link, remove one leaf, remove that leaf's uplinks, and restore each in turn. After every step, record the active slave, the EVPN segment state, the remote forwarding entry and the observed throughput in both directions. A leaf that loses its fabric path while its host link stays up is the interesting case, because the attachment remains advertised and can quietly become a dead end. Test with fresh unicast flows and with the flooded traffic the application depends on, since the two use different forwarding decisions.

Accept the design only when the matrix shows bidirectional delivery through the path you intended. Two cables with link up is not that result.