Tracing OpenVPN Control Packets: tls-auth Before TLS

A tls-auth failure is usually reported as a handshake problem, and it never was one. The HMAC is checked on the control packet before any TLS processing happens, so a mismatched key or direction produces a failure that looks like a certificate problem and is decided two layers earlier. Treating the envelope as its own object makes the diagnosis short: read the fields, confirm the check fires early, then compare the three things both ends must agree on.

Read the envelope in the order it is built

A control packet carries a session identifier, the optional tls-auth HMAC, replay and acknowledgement information, and a reliable message identifier, with the message type payload after that. When the message is a handshake message, the payload happens to be TLS ciphertext. That is the whole relationship: TLS material travels inside a message that the envelope already protects. The OpenVPN network-protocol reference lists the control-channel fields in that order.

The reliability layer is easy to mistake for cryptography. Acknowledgements, retransmission and the message identifier exist so control messages arrive once and in order over an unreliable transport. They are not authentication, and a correct message identifier proves nothing about who sent the packet.

Keep this grouping conceptual. It is not a byte-order diagram, and it is not enough to compute a tag, because the fields that enter the HMAC depend on implementation, version and mode.

Show that the check fires before the handshake

A receiver holding the shared key can verify the HMAC and drop a forged packet immediately, without allocating handshake state or doing public-key work. The protection covers the initial handshake packets too, which is exactly why a wrong key shows up as a failed connection rather than as a late authentication error.

Raise the daemon's verbosity and watch the control channel rather than the tun device:

openvpn --config /etc/openvpn/server.conf --verb 6

What you are looking for is a control-channel HMAC verification failure reported at the packet level, before any TLS alert. A TLS alert in the log means the envelope was accepted and the failure moved inward, which is a different investigation with a different fix.

Direction handling sits here as well. When a static key file is used with tls-auth, the two ends take opposite key directions, and the protocol uses that split so a captured packet cannot be reflected back at its sender. Getting the direction backwards keeps the key identical and still fails every packet. Change one end at a time, then restart both daemons so the pair is loaded together.

Match the key, direction and digest on both ends

Three values have to agree and only one of them is the file. The static key content must be identical on both ends, the direction must be opposite, and the digest used for the HMAC must match. Check the configuration before touching the key material, because a single reversed direction explains more failures than a corrupted file:

grep -E 'tls-auth|tls-crypt|key-direction|auth ' /etc/openvpn/server.conf

Generate the shared secret with the tool rather than with an editor, and keep its permissions tight:

openvpn --genkey secret /etc/openvpn/ta.key
chmod 600 /etc/openvpn/ta.key

Older releases accepted a slightly different option order for key generation, and profiles written for those releases still circulate. The generated file is the same kind of object either way, so the check is whether both ends hold identical bytes, not which spelling produced them.

Do not confuse tls-auth with tls-crypt. Both protect the control channel, but tls-crypt also encrypts it and hides the handshake from observers, and it replaces the direction split with its own key handling. A profile using one will not interoperate with a peer configured for the other, and no amount of digest tuning closes that gap.

Reproduce a tag only from the real format

Byte-exact reproduction is a separate task from running a VPN, and it only works with the real inputs: the implementation and version that built the packet, the selected digest, the direction, and the actual bytes including session identifier and acknowledgement array. A textual rendering of the field list is not those bytes.

Tag length follows from the digest, so a fixed-length assumption breaks the moment someone changes it. Hash the packet as the implementation does, not as a table in a blog post presents it, and compare against a capture taken from the same peer rather than against an expectation you derived by hand.

Keep that exercise away from production profiles. Nothing about reproducing a tag tells you whether a deployment should enable the protection.

Roll back a mismatch without removing protection

The tempting test is to delete tls-auth, watch the connection succeed and conclude the key was wrong. That answer costs you the protection and teaches nothing about which of the three values disagreed. Get the approved matching configuration from the server side and compare against it instead.

Stage the change so it is reversible. Keep the previous server configuration and key file, apply the corrected pair, verify that a client completes the handshake, and only then remove the backup. If clients fail after the change, restore the previous files rather than improvising a third variant during an outage. The restore should be one command you already know before you start.

Finally, remember what a passed check means. It proves the peer holds the shared key, not that its certificate or account is authorized, and it says nothing about the negotiated data-channel cipher. Authorization and data protection are established later, so a clean envelope check is a starting point rather than a verdict.