Testing a VPN Privacy Claim: Keep Evidence That Holds
Write the claim, the observer and the setup before you look at a result. A routing test shows an exit; it never shows what the operator retains.
The test starts with a sentence someone wants to be true, and the evidence either supports a narrow version of it or nothing at all. A public address check shows which exit one request used. It does not show what the provider retains, who can read it, or what happens to a session that never completes. Walking the test as a documented procedure stops you claiming more than the measurement can carry.
Pick One Observable Claim
Write the claim as a single testable sentence before opening any tool: for this profile, on this network, requests to a public address echo leave through the provider's stated exit. That sentence names a subject, a condition and an observation, which is what makes it falsifiable. A claim such as the VPN protects privacy has no observable form, so no measurement can settle it. Keep a second sentence for what the test will not cover, because retention, third-party access and traffic correlation all live outside a browser test.
Write that second sentence in advance, not after the result arrives. It is much easier to be honest about scope while the outcome is still unknown than after a reassuring screenshot lands in the ticket. A scope note added afterwards tends to describe the result rather than the question.
Setup Facts a Reader Needs to Repeat the Measurement
The setup is what makes a result reproducible, and it is what people skip when the answer looks good. Version strings, profile names, access network and address families belong in the note, along with any split-tunnelling rule that could take traffic out of the tunnel. Name the event under test as well, since a working connection, an unexpected drop, a manual disconnect and a network handover exercise different rules. A test that mixes those events proves little about any one of them.
Use non-sensitive traffic in an environment you are authorized to test. Capturing unrelated packets makes the record look technical and adds exposure without adding evidence. Point the capture at the tunnel interface only, and set a file size limit so a long run cannot fill the disk. Stop the capture before changing anything, so the file matches the setup line you wrote.
Say What Was Measured and What It Means
Record what the tool printed and what the tool could see. A resolver answer, an address-echo result and a route lookup answer different questions, and a location label inside a web page is an inference about a database rather than a fact about the exit. Write the command that produced each line next to it, because a value without its command cannot be reproduced. Keep the timestamp in the same timezone as the change ticket, or the sequence becomes unreadable a week later.
curl -s https://icanhazip.com
curl -s https://ipinfo.io/ip; echo
resolvectl status --no-pager | head -15Two independent echo services that agree are stronger than one, and a disagreement is itself a finding worth writing down. The resolver output shows whether DNS followed the tunnel or kept using the local network, which is a separate claim from the exit address. Save the raw output with a timestamp, and keep contrary results rather than the most reassuring screenshot. When a change moves the result, apply one variable at a time, because switching server and client together leaves you unable to say which one mattered.
What Belongs in the Record, and What Must Be Stripped
Retain the method, the non-secret output and the limitations, then strip the rest. Credential-bearing profiles, private hostnames and account identifiers do not need to travel with the evidence, and a test record that leaks them has damaged the thing it was meant to demonstrate. The five fields below are the shortest shape that still survives a handover.
| Field | What to write |
|---|---|
| Claim | One testable sentence naming the profile, the network and the observation |
| Observer and scope | Who could see the traffic or the record, and under which conditions |
| Setup | Client and server versions, profile name, access network, address families, split-tunnel rules, event under test |
| Raw result | Tool output with a timestamp, exit addresses from both echo services, resolver answer |
| What it does not support | Retention, third-party access, correlation, untested address families |
Attach the policy statement you are comparing against, with its date and its scope. RFC 6973 frames privacy analysis around named observers and defined scope, and that framing suits a test record: who could observe what, under which conditions, and which parts stayed unexamined.
End with one paragraph stating what the evidence supports and what it does not. That paragraph is the deliverable, and everything above it exists to make it honest. If it cannot be written without hedging every clause, the test measured less than the ticket claims.