ESP Arrived but Nothing Decrypted? Follow the Inbound Stages

Watch a firewall's encryption counter climb while the tunnel delivers nothing, and the instinct is to widen an access rule. That instinct skips the receiver's own sequence of checks. An inbound ESP packet passes through several stages, and each one can end the packet's life for a different reason: no matching association, a replay or integrity failure, or an inner packet that is fine but not permitted onward. Naming the stage that failed turns a vague "IPsec is broken" into a specific counter and a specific fix.

Arrival is one observation, not three

An outside capture showing ESP reach the gateway proves only that the outer packet arrived. Acceptance means the receiver found the association and validated the packet. Delivery means the recovered inner packet was routed to its destination and answered. Those are separate results, and a dashboard that merges them hides exactly the information you need.

The same distinction explains a common report: one peer's encryption counter rises while the other reports no useful decrypted traffic. Both statements can be true at once, because they describe different stages of the same attempt.

Association lookup happens before decryption

The receiver reassembles fragments if needed, then finds the security association for the packet. For ordinary unicast, the SPI and the destination select it. No valid association means no further processing, which is why a stale or mismatched SPI produces silence rather than an error visible to the application.

Associations are directional and can change during a rekey, so record the receiving peer, the SPI and the timestamp beside the inbound association you believe is current. A displayed outbound association, or a healthy IKE relationship, is not a substitute for the one that should receive this packet.

Replay and integrity checks reject before a payload exists

Once an association is found, the receiver applies the algorithms that association specifies. If anti-replay is enabled, a sequence number outside the window is rejected. An integrity check failure also ends processing, because the ICV does not match the packet as received. Neither case produces a decrypted inner packet, so nothing reaches routing, policy or the application.

Use the implementation's documented errors and counters for these conditions instead of treating every receive failure as an access-list denial. Disabling integrity or replay protection to make traffic appear does not show that the incoming packet was legitimate; it removes the check that was reporting the truth.

UDP port 4500 is a container, not a verdict

NAT traversal carries ESP inside UDP, and the same port also carries IKE and keepalive messages. Activity on 4500 therefore tells you that something is using the port, not that the application produced an ESP packet. Classify the payload before you attribute it to data traffic.

This matters when a capture is the only evidence available. A keepalive arriving on schedule can make a dead tunnel look alive, and an IKE message can make a capture look busier than the data path really is.

Only after extraction do routing and policy apply

When decryption succeeds, the recovered inner packet enters the normal path: policy checks, routing, forwarding rules and the destination host. A packet can pass every security check and still be dropped here, and the reply path has to exist as well. That failure belongs to forwarding, not to ESP.

Keep counters tied to their documented stage. Interface receipt, accepted ESP, decrypted bytes and forwarded inner packets need not be the same number, especially when an accelerated path handles the data plane and the control plane reports separately. A generic dashboard label is not proof that two counters mean the same thing.

For one fresh authorized operation, keep the last confirmed boundary and the first relevant error. That pair tells the next administrator whether to investigate transport delivery, association and integrity, or ordinary inner forwarding, and it avoids the usual fix of making every rule permissive to move a number.