rsync over VPN: a retry that reuses the broken transfer

Your upload stalls at four gigabytes of a twelve-gigabyte archive, and the habit learned from download managers says to press resume. rsync will not resume the TCP stream, because that stream is gone with the tunnel's last retransmission. What it can do is keep the bytes it already received and use them as a basis for the next attempt, which is a different mechanism with a different set of rules. The flags you choose decide whether that reuse happens at all.

The retry rebuilds state, it does not resume the stream

When the SSH session drops, the remote end is left with whatever it flushed to disk and nothing else. There is no session token to reattach to and no sequence number to replay. The next run starts a fresh connection, reads the source again, and asks the receiver what it already has.

That last question is why a partial file is worth keeping. The receiver's copy gives rsync something to compare against, and the algorithm sends only the differences it can identify. It is still a new transfer of metadata and a new pass over the source, so the saved time is proportional to how much of the file matches, not to how far the previous connection had progressed. If the receiver retained nothing, the next attempt is a full copy no matter how many gigabytes the first one moved.

Decide where the reusable fragment lives

The default behaviour writes the partial file at the destination path, which makes a half-written archive visible under the final name. --partial-dir moves it into a directory of your choosing, and the manual describes that directory as retained for use as data by a later transfer. A relative name is resolved under the destination directory, which is what you want when the two ends may have different layouts.

rsync -e ssh --partial-dir=.rsync-partial --info=progress2 -- \
  large-archive.bin [email protected]:/srv/incoming/

Order matters at the destination too: the partial directory needs enough room for the fragment, and the volume needs enough for the completed file as well. Filling the target volume mid-transfer produces a failure that looks like a network problem but is not, and it can leave the retained fragment truncated. Check free space on the receiving host before the run starts, not after it fails. A few seconds with df -h over the same SSH path is the cheapest possible test of the plan.

Permissions are a smaller but real concern. A partial directory writable by unrelated accounts lets someone else substitute content that your next run will treat as already received. Keep it owned by the transfer account, and do not put it somewhere other services publish from.

Run the same copy, not a cleverer one

The reuse only applies when the source and destination still describe the same object. Same path, same name, same file. Renaming the source or pointing at a different destination directory starts a fresh comparison against nothing useful.

Do not add --whole-file while expecting the delta behaviour. That option tells rsync to copy the entire file without the rolling-checksum comparison, which is a reasonable choice on a fast local link and exactly the wrong one after a failed long-haul upload. --inplace also changes the story: it writes into the destination file directly, so an interruption leaves a file that mixes old and new content with no separate fragment to fall back on.

The progress line reports aggregate progress for the transfer. It is not a completion signal. A number that reaches one hundred percent while the connection is still open means the data has been pushed, not that the receiver has renamed, synced, or accepted anything.

Accept the object only after it matches

Confirm the exit status first, then compare the result rather than trusting the absence of errors. A size check catches most truncated transfers; a checksum comparison catches the rest, and both are cheap compared with running the upload again.

If the archive is published by a downstream process, make that process wait for the accepted file. A retained fragment that lands in a watched directory under the final name gets picked up by whatever scans next, and the consumer's error will be about corrupt input rather than about a transfer that never finished.

Leave the source alone until the recipient confirms. Regenerating the archive between attempts changes the comparison basis, and the next run either transfers nearly everything or produces a file that matches neither the old nor the new source.

When the link, not the flag, is the limit

Header and protocol overhead is one cost of a transfer, and it is rarely the one that decides how long a large file takes. Restart behaviour matters more: a method that survives a drop without manual steps delivers a usable file sooner than a faster method that has to begin again. Storage on both ends and the authentication path belong in the same comparison.

Compare alternative permitted paths with the same file and the same destination, and note where each one restarts from after an interruption. That single column tells you more about a twelve-gigabyte upload over a VPN than any peak rate measured on a short test.