Self-Signed VPN Certificate: What It Means and What It Trusts

Two beliefs do most of the damage here: that a self-signed VPN certificate means the tunnel is not encrypted, and that installing whatever the VPN client asked for only affects the VPN. Neither survives contact with a real deployment. A self-signed certificate describes a signature relationship, not a security verdict, and the reach of the trust you add depends on which store it lands in. I have watched a helpdesk import a provider root into a Windows machine to silence one warning, then spend an afternoon explaining why the browser stopped complaining about an unrelated portal. The two halves of this question belong together, so treat them as one.

A certificate is not a trust store

The file the server presents carries a public key, a subject name, an issuer name, validity dates and extensions. Self-signed means the subject and the issuer are the same entity and the signature verifies with the certificate's own key. Nothing in that definition says the connection is weak, and nothing says the client will accept it.

Acceptance comes from somewhere else: a CA file you point the client at, a fingerprint you compare by hand, or a root that already sits in a store the client reads. A deployment can be perfectly sound while its certificate would fail every public test. openssl verify against the system bundle will report an unknown issuer, and that report is correct rather than a fault.

OpenVPN's ca option is the clearest case of app-scoped trust. That file names the anchors the client will accept, and it may hold one private CA or, in a small deployment, the server's own self-signed certificate. Version 2.6 also accepts --peer-fingerprint for pinning a SHA-256 fingerprint instead of distributing a CA file. WireGuard sits further out still: it has no certificates at all, only a static public key per peer, so the self-signed question does not map onto it.

Where you install trust decides who inherits it

Put the same certificate in three places and you get three different blast radii. A bundle read only by the VPN client affects that client. A root in the operating system store affects every program that consults that store. A browser with its own trust database may ignore both, which is why a warning can survive an import that appeared to work.

On Debian or Ubuntu, dropping a file in /usr/local/share/ca-certificates/ and running update-ca-certificates alters the system-wide bundle. On Windows, certutil -addstore -f Root writes to the machine store and needs an elevated prompt, while adding -user targets the current user's store instead; the two are not interchangeable. Firefox keeps its own NSS database, so an OS-level import may never show up there.

Scope matters because of what an authority can do. An issuing CA can sign for names it has no relationship with, so anything that trusts that store will accept those signatures. A single pinned leaf cannot do that; it authenticates exactly one peer identity. If the goal is to make one VPN connect, an app-scoped CA or a pinned leaf reaches it, while a machine-wide root is a far larger concession than the symptom requires.

You are still encrypting without a public CA

The TLS handshake derives session keys whether or not the peer certificate chains to something you already trust. Self-signed does not mean plaintext, and it does not mean the traffic is readable in flight. What it means is that the client has no a-priori reason to believe the key belongs to the server it asked for.

That gap is where real attacks live. A client configured to accept any certificate, whether through a disabled verification setting or a trust-this-anyway button clicked once and forgotten, will encrypt happily to whoever answers. The interception happens in the authentication step, before application data moves. Encryption without an identity check keeps a passive observer out and does little else.

Trusting the issuer does not skip the name check

Chain validation, name matching and certificate purpose are separate decisions. A chain can be valid while the name is wrong, and a name can match while the certificate was issued for a different role. RFC 5280 defines the fields that carry those constraints, and how strictly a client enforces them is a per-application matter.

OpenVPN makes the split visible. verify-x509-name pins the expected subject through name, name-prefix or subject matching. remote-cert-tls server requires the peer certificate to be usable as a server certificate, which rejects a client-only certificate even when the issuer is trusted. Neither setting is replaced by shipping a CA file.

Inspect the certificate before you decide anything:

• openssl x509 -in server.crt -noout -subject -issuer -dates -ext subjectAltName shows who issued it, for which names, and until when.
• openssl verify -CAfile ca.crt server.crt checks whether the chain you plan to ship actually validates the leaf.
• openssl s_client -connect vpn.example.net:443 -servername vpn.example.net -CAfile ca.crt -verify_return_error works for TLS endpoints such as a web portal; the OpenVPN protocol on 1194 is not a plain TLS stream, so that test does not apply there.

A certificate with an expired date, no subjectAltName for the name you connect to, or a missing serverAuth usage fails regardless of how carefully the anchor was installed.

Five checks before you accept one

Compare the fingerprint over a channel that is not the connection you are trying to repair: a console, an installer you already have, a signed configuration file. openssl x509 -in server.crt -noout -fingerprint -sha256 on the server and the same value in the client log is enough to know you are looking at one object.

Next, decide what you are installing: one leaf, or an authority that can mint more. Then decide where it goes, and name the applications that read that store. Write down the removal path while you still remember it, because trust-this-and-move-on is how a temporary import turns permanent.

Finally, plan the change. A pinned certificate or fingerprint has to be updated through the same trusted channel when the certificate rotates, and a private CA has its own issuance and revocation lifecycle. If the private key leaks, re-trusting a copy of the same certificate repairs nothing; the identity has to be replaced.

If a warning survives all five checks, the useful question is not which button silences it but which store the client actually reads and where that root came from. Answer that, and the import you are about to make becomes a decision instead of a reflex.