Android VpnService: TUN read stalls vs ENOBUFS write errors
A read that never returns and a write that fails with ENOBUFS have opposite causes. Measure the advertised window and the packet length to pick the fix.
Two symptoms show up in the same VpnService loop and get blamed on the same thing: a reader that never returns, and a write that fails with ENOBUFS. They are not variations of one bug. A stalled read is a disagreement about flow control with the peer; a failed write is a packet the interface will not accept. Treat them as one problem and you will spend a day tuning buffers that were never involved. The split below is how I decide which half of the loop to instrument first.
One loop, two independent failure points
Your service reads an IP packet from the TUN file descriptor, inspects it, and writes a reply back to the same descriptor. Everything that breaks in that path surfaces at one of those two calls, which is why both get reported as the tunnel being broken. The calls have different contracts. A read waits for the kernel to hand you a packet, and a write hands the kernel a packet it is allowed to refuse.
Reads are the easy half to reason about. A read that returns nothing is either waiting for traffic that never arrives or waiting because the peer stopped sending.
A stalled read is a flow-control disagreement
Reading a TUN device blocks until a packet is available. That is designed behavior, and it is why a thread parked inside read looks normal on an idle link. It becomes a hang when your own TCP stack keeps accepting data from the local app but never advertises enough window for the remote peer to send more. The peer fills the window you advertised, waits for an ACK and a window update, and stops. Your reader then has nothing to return, and the app sees a connection that is open and silent.
The common causes are a stack that writes into the socket without tracking the advertised window, and one that forgets to send a window update after draining its buffer. From outside they look identical: bytes go out, nothing comes back, no error is raised. Blocking the reader thread on parsing, DNS resolution or disk I/O produces the same shape of symptom for an unrelated reason, so confirm the thread is actually inside read before blaming flow control.
Check the peer send queue and the window it last saw. If the peer is holding data because your window is zero, the fix is in your ACK path, not in the TUN read.
ENOBUFS is a size problem, not a buffer shortage
A write that returns ENOBUFS on a TUN interface is usually rejecting the packet length. The interface has an MTU, a packet larger than it cannot be queued, and retrying the same packet fails the same way every time. That determinism is the tell. A genuine buffer shortage clears as pressure drops, while a size rejection does not care how long you wait.
TCP has a mechanism for exactly this, and it is the step people skip: the MSS clamp. When you synthesize segments in user space you choose the payload size, and picking it from the local interface MTU instead of the tunnel MTU emits segments the tunnel cannot carry. Clamp to the smaller of the two. The clamp covers TCP only, so an oversized UDP datagram has to be fragmented or dropped by your code.
Log the length of every packet you write while you debug. The length that fails is the answer, and it will not move between retries.
Three measurements that separate the cases
You do not need a debugger for either failure. Three observations are enough to route yourself to the right section, and they take less time than a rebuild.
| What you see | Where to look |
|---|---|
| Reader parked, peer send queue empty, last advertised window zero | Window updates in your ACK path |
| Write fails on one length and succeeds on a smaller one | MSS clamp and tunnel MTU |
| Both directions look fine, app still sees no data | Protected socket and routing loop |
If the third row matches, the loop itself is fine and the problem sits where the tunnel cannot see it: an unguarded transport socket carrying your own VPN traffic back into the tunnel, which usually shows up as a connect that succeeds and then delivers nothing.
Fix the plumbing both paths depend on
Neither symptom is worth chasing until the interface is built correctly. The builder calls that matter are the address, the route, the MTU and consent, in that order:
Builder()
.addAddress(10.8.0.2, 32)
.addRoute(0.0.0.0, 0)
.setMtu(1400)
.establish()Set the MTU to the tunnel's real value rather than leaving it at whatever the sample used. Everything you later write is bounded by that number, and a mismatch between the MTU you set and the MSS you clamp to is the most common way to reintroduce the write failure after a fix. Then protect the socket that carries traffic to the VPN server itself. Without protect, those packets are routed back into your own tunnel and the connection never completes, which looks like a stalled read but is a routing loop.
Consent cannot be bypassed. A service started without prepare returning null gets no interface, and establish returns null rather than throwing, so check the descriptor before you start your loop. The VpnService reference is the place to confirm the builder contract instead of copying a sample.
Verify one direction at a time
Send a UDP echo through the tunnel and watch packet lengths before you touch TCP. Then open a handshake and confirm the segments stay under the MSS you computed. Count packets rather than connections, so a stall appears as a gap in the counter instead of a hung socket. If reads stop after a burst of writes, the window is the suspect; if writes fail immediately, the length is.
Then reboot the device and repeat once, because a VpnService that only works until the first network change is a routing problem rather than a flow-control one.