TCP Retransmissions vs IPsec Replay Drops: Read Each Layer

Three different tools report three different duplicates, and only one of them is a network event you can act on. A TCP retransmission is a sender re-sending a range, a duplicate ACK is a receiver repeating an acknowledgement, and an ESP replay drop is a security association refusing a sequence number it has already accepted. None of the three implies either of the others. Read each counter at the layer that produces it before you join them into a story.

Three counters that never measure the same event

The TCP retransmission annotation means the sender placed a sequence range on the wire again. Loss is one reason for that, and the capture you hold may simply be missing the original packet or the acknowledgement that would explain the timing. A duplicate ACK is the receiver repeating a cumulative acknowledgement under TCP recovery rules, which is neither a duplicate application write nor a second copy of the data.

ESP works in its own sequence space. Each security association numbers the packets it sends, and the receiver keeps a window of numbers it has already accepted, so an ESP packet that arrives twice is discarded on those terms alone. A retransmitted TCP segment therefore travels in a fresh ESP packet with a fresh sequence number, which is correct behaviour at both layers. The same bytes can look new in one counter and old in another, and neither layer is lying.

What the analyzer can and cannot see

Start with the analyzer vocabulary, because the filter names describe observations rather than endpoint internals. Counting with the same names the tool uses keeps you honest about what was actually measured. A count from an incomplete capture is a count of what survived, not of what happened on the wire.

tshark -r trace.pcap -Y "tcp.analysis.retransmission" | wc -l
tshark -r trace.pcap -Y "tcp.analysis.duplicate_ack" | wc -l

Read the expert information beside those numbers, since checksum and ordering warnings change how much weight they deserve. A merged trace from two sensors can show one frame twice, and counting both copies invents a recovery event that never happened.

ESP counters live in kernel state

ESP sequence numbers belong to the security association, not to the TCP stream, so the gateway is the place to ask. The state listing reports per-SA packet and error counters that no capture can reconstruct.

ip -s xfrm state
tshark -r trace.pcap -Y "esp" -T fields -e esp.seq | head

RFC 4303 defines the sequence number and the anti-replay window that consumes it, which is why the counter rises even when nothing is wrong. Monotonic outer sequence numbers with no matching TCP anomaly point at a different sender or a duplicated path rather than at a retransmission storm. An outer capture of an encrypted tunnel normally hides the inner TCP sequence numbers, so it cannot prove what the inner stack did. Capture inside the tunnel only where policy allows, and never disable anti-replay to silence an unrelated warning.

Where the vantage point changes the answer

Before escalating, write down how the capture was taken: the interface, whether the capture process reported drops, and whether the timestamps are comparable across sensors. One vantage point cannot see both ends of a path, so any statement about receiver behaviour is an inference rather than a measurement. A trace taken on the gateway and a trace taken on the host will disagree about ordering under load, and that disagreement usually belongs to the capture rather than to the tunnel. Correlating two views is what turns a count into evidence.

Use one permitted application flow and record the endpoint counters beside the capture counts at the same moment. If the numbers disagree, trust the layer that owns the event and treat the other as an observation artifact until proven otherwise. That rule settles most of these tickets without touching the tunnel configuration.

Reading the numbers in the right order

Start at the layer where the question was asked: a TCP warning is answered from a TCP capture, and an ESP drop is answered from the security association counters on the gateway. Only then ask whether the two views describe the same packet. A conclusion that needs both layers should name both, because a claim about the tunnel resting on a TCP annotation alone is a guess.

Record the layer, the counter and the vantage point for every finding, and the next reader will not merge three events into one imaginary fault. Escalate with those three fields and the fix usually follows from the first one.