OpenVPN pod in Kubernetes forwards nothing? Check ip_forward

Your laptop shows a green tunnel, the client log reports that routes were pushed, and the only address that answers is the VPN server's own tunnel IP. Pings past it die no matter what you aim at, whether that is a ClusterIP, another pod's address, or a public hostname. That combination is diagnostic rather than vague: decapsulation works, and the second hop does not.

I met this on a Debian node running a chart-installed OpenVPN image, and the tunnel was the easy part. The client took a 10.8.0.x lease, 10.8.0.1 responded, and 10.96.0.10, the cluster DNS service, timed out. Nothing in the server log looked unhealthy, which is exactly why people start rewriting iptables rules they never needed to touch.

A VPN pod is a router, and it is worth saying that plainly. Encrypted packets arrive, get unwrapped in the pod's own network namespace, and then have to leave through a different interface. Linux refuses that second step unless forwarding is enabled inside that very namespace.

What the client can and cannot reach

The address that answers belongs to the tunnel endpoint, and the VPN process owns it locally, so no forwarding is involved at all. Everything else, including another pod, a service VIP, or the internet, needs the kernel to move a packet from tun0 to eth0. When that move is refused, the symptom is uniform silence rather than a rejection, because nothing generates an ICMP error back down the tunnel. You get timeouts, not unreachable messages, and that is the clue that nobody is routing at all.

Two minutes of classification narrows the layer. If DNS names fail while raw pings to a service IP succeed, the problem is DNS plumbing, not forwarding. If every destination beyond the peer address times out, including a bare IP, start at the namespace.

Read the setting inside the namespace

Every pod gets its own network namespace, and net.ipv4.ip_forward is a per-namespace sysctl. Changing it on the node tells you nothing about the container, and it is a reliable way to spend an afternoon on the wrong machine.

kubectl exec -it deploy/openvpn -- sysctl net.ipv4.ip_forward
# net.ipv4.ip_forward = 0

A zero confirms the diagnosis. The value is not inherited from the node's runtime configuration either: it starts from the image default every time a container is created. A fix applied by hand with kubectl exec ... sysctl -w therefore vanishes on the next restart and comes back as a fault that looks intermittent.

Enabling it so the manifest keeps it

The durable option is an init container that runs in the same pod sandbox and writes the value before the VPN container starts. Containers in a pod share a network namespace, so the setting carries into the main container.

initContainers:
- name: enable-forwarding
  image: busybox:1.36
  command: ["sysctl", "-w", "net.ipv4.ip_forward=1"]
  securityContext:
    privileged: true

That privileged: true is the line reviewers push back on, and they are right to. Writing a sysctl needs either privilege or a narrowly scoped capability, and a cluster running the restricted Pod Security Standard may refuse a privileged init container outright. If admission rejects the pod, you learn at creation time, which at least is honest and easy to read.

The cleaner alternative is the pod-level securityContext.sysctls list, which lets the kubelet apply namespaced sysctls without granting privilege to any container. It only covers names on the cluster's allowlist. Whether net.ipv4.ip_forward is on that list depends on how the kubelet was configured, and a pod that asks for a sysctl outside it is refused at admission or stuck in CreateContainerConfigError while the sandbox is retried. Check the allowlist before you assume the field is broken.

Before hand-rolling either approach, see whether the chart already exposes the switch, often as something like ipForwardInitContainer: true. Using the chart value keeps your override file small and survives the next upgrade, which hand-patched Deployment edits do not.

The second half: return path and policy

Forwarding on is necessary, not sufficient. If clients use a private pool that the cluster network has no route for, replies cannot find their way back into the tunnel, and the pod needs to masquerade that traffic toward its destination.

Policy sits on top of that. NetworkPolicy objects and cloud firewall rules are evaluated against the source address a packet carries after translation, so a rule written for the client pool can quietly stop matching once masquerading is enabled.

When clients only need to reach services inside the cluster, having the VPN terminate at the service VIP is a smaller design than routing into the pod network. It also keeps the client pool out of the CNI's routing table entirely.

Isolating the failing hop from the client

Work outward and stop at the first silent step. Start with the VPN peer, then try some pod's address, then a service VIP, and only after that a public destination. Each success moves the boundary one hop, and each failure names the layer worth inspecting. After any rollout, exec in and read the sysctl again, because a replaced pod starts from the image default and a fix that was never in the manifest is already gone.

The Kubernetes documentation on using sysctls in a cluster lists which names can be set through the pod security context, and it is the first place to look when admission refuses the change.