Building a Full-Stack VPN Proxy on HarmonyOS: From Custom Go Toolchain to gVisor Integration
HarmonyOS (Next) represents a substantial architectural departure from standard Linux and Android. With its strict application sandbox, NAPI dynamic loading model, and musl libc runtime, bringing existing low-level network tools to the platform requires solving non-trivial compatibility challenges across the entire stack.
This post documents the end-to-end implementation of the CocoProxy VPN client on HarmonyOS—covering custom Go toolchains, NAPI lifecycle coordination, user-space TCP/IP via Google’s gVisor stack, socket protection against routing loops, and direct TUN packet crafting to defeat DNS poisoning.
1. System Architecture
1.1 Architecture Overview
1 | ┌────────────────────────────────────────────────────────┐ |
1.2 End-to-End Data Flow
1 | User visits google.com |
1.3 Key Architectural Decisions
| Decision Area | Selected Technology | Technical Rationale |
|---|---|---|
| Network Stack | gVisor (sing-tun) |
Runs entirely in user space; requires zero kernel modules |
| Go Interop | c-shared (.so) + NAPI |
HarmonyOS sandboxes prohibit standalone daemon processes |
| VPN Subnet | 172.19.0.1/30 |
Minimal clash with common local LAN subnets |
| MTU | 9000 | Reduces fragmentation overhead across high-throughput relays |
| Gateway Flag | hasGateway = false |
Tailored to HarmonyOS network manager routing specs |
| Socket Protect | NAPI ThreadSafeFunction |
Safely crosses goroutine boundaries into the ArkTS main thread |
2. Dynamic Linking & NAPI Initialization
2.1 The NAPI Loading Difference
In Node.js, dlopen locates the exported napi_register_module_v1 symbol dynamically.
On HarmonyOS, the ArkCompiler loader executes constructor functions within .init_array immediately upon dlopen(), expecting napi_module_register() to be called synchronously.
Standard Go’s //export napi_register_module_v1 fails here. We use an explicit C constructor in napi_reg.c:
1 | __attribute__((constructor)) |
2.2 Binary Patching (patch-so.ps1)
To prevent the Go runtime from auto-bootstrapping before ArkTS finishes setting up TLS states, our build pipeline patches the binary:
- Short-circuit
_rt0_arm64_linux_lib: Overwrites the entry instruction withRET(0xd65f03c0), deferring Go runtime startup until ArkTS explicitly callsstartTun. - Relocate runtime entry address: Dynamically calculates
_rt0_arm64_linux_lib_goand stores its virtual address ing_rt0_go_offset.
3. Network Subsystem Implementation
3.1 VPN Configuration & Virtual TUN
In ArkTS:
1 | import vpnExtension from '@ohos.net.vpnExtension'; |
3.2 Preventing Routing Loops: Socket Protection via ThreadSafeFunction
All outbound system traffic is routed into tunFd by default. If the Go proxy opens a TCP socket to its upstream remote proxy, that connection would also be swept into tunFd, producing an immediate infinite routing loop.
HarmonyOS provides vpnConnection.protect(fd) to exempt specific sockets from the VPN routing table. However:
protectcan only be invoked from the ArkTS main thread.- Outbound proxy dials happen asynchronously inside Go goroutines.
We bridge this using a NAPI ThreadSafeFunction (tsfn):
1 | Go Goroutine ArkTS Main Thread |
3.3 Integrating gVisor’s User-Space TCP/IP Stack
We configure sing-tun with the raw file descriptor passed from ArkTS:
1 | tunOptions := tun.Options{ |
3.4 Bypassing the gVisor Loopback Trap for DNS
In standard setups, stack.WritePacket() is used to return simulated DNS responses. In gVisor’s user-space implementation, however, calling WritePacket on locally terminated UDP packets causes the stack to consume the reply internally rather than emitting it back out the TUN descriptor.
The Solution:
We construct complete IPv4 + UDP packets manually in memory and write them directly into the underlying Linux tunFd via syscall.Write:
1 | func writeDnsResponseDirect(rawTunFd int, srcIP, dstIP net.IP, srcPort, dstPort uint16, dnsPayload []byte) error { |
4. Key Takeaways
- User-Space Stacks are Indispensable: Without root or kernel tun devices, running gVisor in user space via
tunFdis the only viable path for high-performance proxy routing on locked platforms. - NAPI Lifecycles Differ: Desktop Node.js assumptions fall apart on mobile OS platforms. Respect
.init_arrayordering and guard against premature Go runtime startup. - Bridge Sockets Early: Crossing between goroutine thread pools and the host UI thread via NAPI
ThreadSafeFunctionmust be bounded by deterministic timeouts to avoid socket starvation.
With this architecture in place, CocoProxy delivers high throughput and rock-solid connection resilience inside the HarmonyOS sandboxed environment.