ESP Decrypts Fine and the Inner Packet Still Goes Nowhere

The ESP receive counter increments, the tunnel stays up, and the application never sees the packet. Decryption ran, integrity verification passed, and then the receiver had to decide what the protected payload actually is. That last step is where a format mismatch appears, and from the outside it looks nothing like a firewall problem.

What you see when the label lies

Nothing in the client log distinguishes this from a routing black hole. Both produce an application timeout beside a healthy tunnel, which is why the counters get blamed first.

A rising receive counter is genuinely good news: packets arrive and authenticate. It is not evidence that the inner packet made sense to anything above the crypto layer.

The type lives in the trailer

In ordinary IPsec tunnel mode, the ESP-protected payload contains a complete inner IP packet. The trailer's Next Header field identifies what that payload is, and it is not a TCP port, not a protocol number in the outer header, and not part of the initial SPI and sequence-number field. RFC 4303 section 2.6 defines it as the payload-type field, which is a precise name for a precise job.

The values you will meet are small and worth remembering: IPv4 is 4 (0x04) and IPv6 is 41 (0x29).

Those numbers come from the IANA protocol number assignments rather than local convention. A sender declaring IPv4 while enclosing an IPv6 packet has produced a format inconsistency, and no firewall rule repairs bytes that contradict their own label.

Telling it apart from the usual suspects

Most missing packets are not encoding defects. Check selectors first, because a traffic selector that excludes the IPv6 prefix drops traffic before anything is encapsulated. Then check routing, then local filtering on the receiving host. Only after those are excluded does a payload-format mismatch become a plausible explanation.

An encrypted capture will not settle it, because the protected Next Header sits inside the encrypted payload. You need implementation diagnostics or a supported decrypted observation point, and both need authorisation before use.

If nobody can produce that evidence, you do not have a format bug. You have a theory, and the selectors remain the better bet.

What the repair is not

Do not disable integrity checks to make traffic pass. Do not rewrite encrypted packets at an intermediate router, which cannot work and will corrupt the flow while looking helpful in a capture. Do not open a wider application firewall rule on the theory that a port is being blocked. None of those touch the label mismatch.

If a confirmed mismatch comes from a peer implementation, keep a minimal sanitised reproduction and take it to that implementation's supported channel.

Verify the flow, not the counter

After the correction, test the actual IPv6 flow end to end: an application request through the tunnel, its reply, and the address pair each side observes. A rising ESP counter only proves packets are being received and authenticated. It says nothing about whether the inner packet was usable.

Record both results in the ticket. Counter movement plus a working flow is a fix; counter movement alone is exactly what you had before you started.