VPN MANAGER / Development notes
What I first wanted to build
The host's original network stays unchanged.
Designated apps use only
their designated exits.
I first wanted to build VPN Manager because most ordinary VPN clients seem
geared toward switching the whole computer together.
But that is not
what I need.
I want to specify which apps use which VPN, while
unmanaged programs keep using the host's original network.
For example, I may have a browser, a downloader and other apps open at the
same time.
The browser goes to Instance A, the
downloader goes to Instance B, and the others use the
original network.
A and B each have their own WireGuard identity,
DNS, Gateway, connection resources and lifecycle.
If A disconnects, I would rather the browser lose its network connection
outright.
It must not automatically move to B, much less fall back to
the ISP's original exit.
If A disconnects: The browser has no network.
Instance
B and unmanaged programs keep working normally, without having to switch
along with it.
I also want to manage accounts, Profiles and Gateways from different VPN
providers in the same system.
Ideally, designated apps or services
should be isolated directly at the OS network layer, without Docker or
changing the whole computer's default route.
Traffic information is another feature I want to build.
I want to see
how much traffic each Instance uses, which Gateways are congested, how
long DNS takes and which process consumes the most CPU or memory.
A
single Connected indicator does not mean much to me.
People have already built similar things, and many assemble their own
environments using netns, proxies and firewalls.
But I want to bring
multi-VPN management, app isolation, failure recovery and monitoring
together, without having to wire a pile of things together each time.
Start with the overall system architecture.
Current operating interface
01The backbone of the system
The overall control path is initially planned as follows:
React UI→ VPN Web API / VPN Core→ Instance Manager→
Instance Controller
Here I make
Instance Manager a separate process.
It handles
starting, stopping, generation, resource leases and recovery for all
Instances.
Controller only receives operation intent from the
UI/API; it does not forward apps' TCP/UDP data.
Anything that needs OS privileges goes to
Platform Manager.
It handles network namespaces,
process sandboxes, FDs, firewall policies and macOS capture/deny.
I
do not want other components changing system network settings directly
just because it is convenient.
Store persists state, Metrics collects data, and Performance Manager
handles analysis and authorized tests.
These things should not sit
in the app packet path.
For example, slower Metrics or database
writes must not queue packets already being transmitted; new state
changes still follow the original fail-closed rules.
| Execution boundary | Main responsibilities |
|---|---|
vpn-controllerd |
Serves React static assets, VPN Web API and VPN Core, and coordinates Provider/Security/Metrics/Performance control. |
vpn-instance-managerd |
Sole cross-Instance lifecycle owner; manages desired state, generation, identity/resource leases, the outer Watchdog and recovery reconciliation. |
vpn-platformd |
Exclusively handles privileged OS operations; runs Linux/macOS Adapters, sandboxes, FD/policy creation, spawn, seal and safety evidence collection. |
vpn-stored |
Sole SQLite owner; manages control.sqlite,
metrics.sqlite, secrets, revisions, the operation
journal and backups.
|
vpn-instance × N |
An independent worker for each VPN; owns its own GotaTun, DNS, Gateway, Port, Metrics and inner Watchdog. |
vpn-provider-worker |
Runs API operations, route-bound IP preflight and restricted connectors per Provider job; does not create, start or stop Instances itself. |
| macOS system extension (if enabled) | Handles flow capture, deny and bridging for managed apps; provides safety boundaries according to native platform capabilities. |
Complete component ownership
I first classify components by who manages them and who has authority
to operate them.
Drawing a pile of boxes alone is not useful.
What matters is who is affected when a process dies, and who can
restart it.
02VPN Instance architecture
Each Instance has its own GotaTun, DNS Controller, Port Controller,
Gateway Manager, Metrics, and Watchdog.
WireGuard identity, DNS
cache, sockets, buffers, and queues also need to be separate.
CPU,
NIC, and OS can be shared, but A's execution state should not be mixed
with B's.
There is one point I want to make clear first.
GotaTun is a WireGuard engine, not a complete TCP/IP socket stack in
itself.
So connecting GotaTun does not mean that all TCP/UDP connection
handling is already there. That part still needs to be handled
separately.
On Linux, I lean toward directly using the kernel TCP/IP stack inside an
isolated network namespace.
For an explicit Proxy on macOS, each
Instance uses its own smoltcp along with GotaTun.
Their
sockets, buffers, and queues also need to be independent. Controller SQL
or Provider HTTP calls must not hold up actual network traffic.
DNS also needs to follow its Instance.
If resolution fails, report
failure. Do not quietly switch to asking the host resolver.
After
receiving HTTP CONNECT or SOCKS5 TCP/UDP, the Gateway must actually
hand it to its own Instance to establish the destination connection.
Port
Forward can only forward to a verified App endpoint within the same
Instance. It must not take the shortcut of connecting directly to the
host's 127.0.0.1.
The difference between managed Apps, Gateway, and Port Forward
I divide traffic entry points into three types.
Managed app must first enter an isolated environment
and cannot choose the host network on its own.
Gateway only handles traffic explicitly handed to the
HTTP/SOCKS Proxy. Connections the App establishes separately are not
automatically protected.
Port Forward is traffic arriving from the VPN. It can
only be delivered to an App within the same Instance that has a valid
AppEndpointLease .
DNS and IPv6 cannot be exceptions either.
If the Instance's DNS
path is unavailable, do not query.
If IPv6 has no verifiable tunnel
path, it must not automatically be allowed onto the host network either.
App isolation on macOS is more troublesome
The macOS side is actually more troublesome.
An explicit Proxy is
possible, but it only handles traffic actually handed to the Proxy.
QUIC,
WebRTC, or the system DNS used by an App itself may create separate
sockets.
So having a Proxy does not mean the entire App is
isolated.
03Linux: making sure there is “no way back” at the OS layer
The Linux approach is more direct.
I do not intend to change the
host's default route or move the physical NIC to another namespace.
Unmanaged
Apps continue using the original network as usual.
Platform Manager first creates a UDP socket in the host network
namespace that allows only the designated WireGuard peer.
It then
passes the restricted FD to the VPN worker in the isolated namespace.
That
namespace has only loopback and TUN. It provides no plaintext veth/NAT
exit to the host.
IP packets sent by an App first enter TUN, are encrypted by GotaTun, and
are then sent to the WireGuard peer through the UDP FD originally
created in the host namespace.
However, creating the socket on the
host is not enough.
Which IP/port it can connect to, who holds the
FD, and whether Platform policy is actually effective all need to be
checked.
What I care more about is that
the App has no ordinary host internet exit at all.
If the worker crashes, disconnect first. There is no need to
find it a backup route.
I cannot accept an App finding its own way
back to the ISP exit after the VPN disconnects.
But netns is not a complete sandbox either.
An App may
inherit another process's FD, or find a bypass through Unix sockets,
D-Bus, or host IPC.
04Instance startup and fault-recovery flow
A successful WireGuard handshake does not mean I can let Apps start accessing the internet.
When starting an Instance, first build the isolated environment and keep
it closed by default.
After the worker starts, allow only the
WireGuard handshake, DNS, and health probes.
Open App and Gateway
traffic only after confirming that the current configuration, Platform
policy, and worker generation are all correct.
If any intermediate step fails, remain
blocked.
Rebuilding a worker must always use a new
generation. An old generation that has been
Seal cannot be reopened.
If the Manager is replaced,
the Platform epoch changes, or the user presses Start/Stop during this
period, previously issued commands must no longer take effect.
I split the Watchdog into two layers.
When the Watchdog inside an
Instance detects a problem, it closes its own ingress first and then
reports the fault.
The outer Instance Manager is responsible for
recording intent, reclaiming old resources, and rebuilding.
I do
not want both sides competing to restart the same Instance at the same
time.
Manual Stop and EmergencyStop must have the
highest priority.
Suppose the database fails just when Stop is
pressed, and it is unclear whether the latest stop intent was saved
successfully. The system must not automatically reopen using the old
desired=running .
At that point, I would rather enter
recovery_required and wait for a person to confirm.
As for whether the original TCP session can reconnect after a crash, I
am not trying to guarantee that.
Losing a connection is better than
using the wrong exit.
Connection state cannot be only Connected/Disconnected
I do not want the UI to have only Connected/Disconnected states.
For
example, I set A to desired_state=running, but it is
currently reconnecting, and
enforcement=blocked.
These three things should be
displayed separately.
Wanting it to keep running and whether it can
currently send traffic out are different things.
Startup, fail-closed behavior, and automatic recovery: START/ACTIVATE sequence diagram
The flow first writes desired=running to Store. Manager
then reserves the generation and identity lease and asks Platform to
execute PrepareClosed.
The worker initially starts
in probe_only mode.
Only after confirming that
tunnel, DNS, policy, and current authorization information all agree
does it execute CommitResume and Activate.
05How is the first Provider created? This is a bootstrap problem
There is also a problem with Providers.
When VPN Manager is first
installed, there is no VPN Profile at all. I have to first obtain an
account, device, and WireGuard configuration from Mullvad or AIRVPN.
So
where should the first management request go?
My current idea is to allow conditional direct access only for the first
time.
The Provider worker receives a time-, account-,
HTTPS-origin-, DNS-, and traffic-budget-limited
BootstrapGrant.
Once the first valid Profile is
obtained, Controller asks Instance Manager to create and start the first
VPN.
After confirming that the tunnel can handle management requests, first
write the intent to close direct access into Store, then revoke the
original grant and sockets, and only finally mark
bootstrap_consumed=true and configure the global Provider
exit.
This order must not be mixed up. In particular, switching
must not be declared complete while direct access is still alive.
After that, all Provider APIs go through the VPN Instance selected by
the user.
If that Instance fails, display blocked.
Switching back to the host or automatically switching to another
Instance is prohibited.
If it really cannot be handled, the user
can manually select a new exit or explicitly authorize a one-time
rebootstrap.
I also want an IP preflight before every actual Provider HTTP request is
sent.
Including retry, pagination, and token refresh, the same
route tuple, IP family, and connector must be confirmed, and a
ProviderRequestPermit valid only for this request must be
obtained before sending.
For now, the tentative preflight deadline
is 3 seconds and the dispatch window is 2 seconds.
This will be
slower and use more checker quota, but at least it will not treat an IP
shown in the UI several minutes ago as evidence for current routing.
The UI needs to display several things separately: which Instance is
selected, whether this request actually passed through a controlled
tunnel, and whether the external IP is that Provider's exit.
For
example, selecting a Mullvad Instance to call AIRVPN's management API
can fully comply with my configuration.
An external IP that is not
AIRVPN does not imply a traffic leak.
What should the Provider Adapter be responsible for?
I want provider integrations to follow the same flow as much as
possible.
Profile validation, Instance creation, and exit binding
are handled by shared control logic. Only Mullvad's and AIRVPN's
respective device, server, and port-forward APIs are assigned to their
Adapters.
When adding a Provider, I do not want every Adapter to
have permission to directly start or stop Instances or open host
sockets.
Provider APIs have another problem: a timeout after sending a request
does not mean the other side did not execute it.
For example, the
device may actually have been created successfully, but the response was
not received.
In that case, first record it as
unknown and use the operation journal to query the remote
state. It should not simply be sent again.
PROVIDER first-bootstrap sequence diagram
To count the entire bootstrap as complete, we must confirm that the
new Instance can send management requests and that the original direct
authorization has actually been revoked. Only then do we
CommitEgress.
If a result during switching cannot be
confirmed, I would rather first stay blocked and
establish the state clearly.
06I want to see every VPN's traffic and resource use clearly
I do not want VPN Manager to end up with just a few green lights telling
me the VPN is connected.
I want to know which Instance is consuming
traffic, how many active connections a Gateway has, whether DNS latency
has risen, and which process is occupying CPU, memory, and queues.
I will separate Metrics Service and Performance Manager.
Metrics
Service collects statistics such as Instance counters, Gateway payload
bytes, WireGuard UDP bytes, DNS successes and failures, active flows,
CPU, and RSS.
Performance Manager is responsible for analyzing
throughput, latency, and resource efficiency, or running benchmarks I
explicitly start.
It cannot change Instance state on its own just
to measure speed.
Metrics should ideally not exert backpressure on the data plane.
Each
Instance sends one bounded snapshot per second. Collector validates
sequence and deduplicates, followed by CBOR/Zstd compression and
aggregation.
If the collector is congested, just drop a few samples
and mark a gap . Do not make packets wait for the database.
Traffic figures must not be calculated carelessly either.
Gateway
payload bytes and WireGuard UDP bytes are statistics from different
layers. They cannot be added together as total traffic.
On counter
reset or a data interruption, charts must show a gap or unknown, not
simply draw 0.
Goodput also requires receiver-side confirmation. It
cannot be calculated from sender counters alone.
Traffic monitoring does not need to record everything either.
Collecting
counters, histograms, and necessary error data by default is enough.
I
do not want to put every DNS query name, full URL, or packet payload
into Metrics.