One banaction, two transports: fixing a fail2ban VPN jail

Nothing in fail2ban drops a packet on its own: it shells out to an action, and that action is what names a protocol and a port. When the jail reports a ban while the daemon keeps accepting logins, the ban is usually real and narrower than the log implies. A client that reconnects over the other transport never meets the rule that was installed. Compare what the action installed against what the daemon listens on before you change anything, because the evidence lives in the firewall chain the action wrote, not in the jail list.

Read the chain, not the jail summary

fail2ban inserts its own chain ahead of the service chain, so a counter listing shows exactly what exists. A chain named after the jail, holding one rule whose protocol column reads udp, is a complete explanation for TCP logins that keep arriving. Only UDP packets from the banned source are being rejected. The service has a TCP listener, and nothing in that chain touches it.

Watch for the neighbouring-jail effect too. One host can look perfectly configured simply because a different jail supplies the half that is missing, and the same build does nothing where that jail is not installed.

The action names one protocol, and that is where it breaks

Each banaction is a firewall command template, and the protocol sits in one of its placeholders. The narrowness is intentional, since a web jail and a DNS jail have no business blocking identical ports. It becomes a trap when the service listens on more than one transport, which VPN daemons routinely do. The filter may watch failed logins from both transports while the action covers one.

SoftEther and OpenVPN deployments usually listen on UDP and TCP at once, and because the log line looks the same either way the filter watches both. The asymmetry never shows in the jail's own report. fail2ban counted the failures correctly and then installed a rule that applies to half the traffic those failures arrived on.

Make the ban cover both transports

Fix it in the jail definition instead; hand-written rules will be shuffled by the next reload. Spell out two actions, one for each protocol:

action = %(banaction)s[name=%(__name__)s-tcp, actionname=%(__name__)s-tcp, protocol="tcp", port="%(port)s", chain="%(chain)s"]
         %(banaction)s[name=%(__name__)s-udp, actionname=%(__name__)s-udp, protocol="udp", port="%(port)s", chain="%(chain)s"]

Do not try to shorten that with protocol = all. The default banaction is a multiport action, and it expands to -p <protocol> -m multiport --dports <port>; with no L4 protocol there is nothing for the port match to test, so iptables rejects the rule while the jail still reports a successful ban. An all-ports action such as iptables-allports or nftables-allports does accept a single entry, but it bans every port on that source, which is a different decision and needs its own justification. Leave protocol at tcp or udp and accept the two entries.

Prove the rule exists rather than trusting the setting, because a setting describes an intention until a chain holds it:

iptables -S f2b-vpn-tcp
iptables -S f2b-vpn-udp

Each chain should list one rule, with -p tcp in the first and -p udp in the second, both naming the banned address. Then confirm the same address is refused on both transports. A ban you have not seen in the chain is a hope, and the next report of the same symptom will cost the same hour again.

What a ban still misses, and the next case

What the action does is reject a single source, and only on the ports listed in it. It does nothing about an attacker rotating addresses, and it cannot separate a hostile machine from a carrier-grade NAT exit shared by hundreds of users, where one ban locks out people who did nothing. Make sure the filter reads the address the client really has, not a forwarded header or a proxy address. Keep the window short enough that a mistake expires by itself.

None of that argues for skipping the protocol fix; it argues for holding two ideas at once. The ban is narrower than the jail summary suggests, and even a complete ban is narrower than the threat. Put the firewall chain beside the daemon's listener list and compare protocol by protocol, because those two lists decide whether a ban can reach the service at all.

Keep the reload command and the chain listing in the runbook next to the jail. The next person will need both, and the jail summary will tell them neither.