Building a Full-Stack VPN Proxy on HarmonyOS: From Custom Go Toolchain to gVisor Integration

HarmonyOS (Next) represents a substantial architectural departure from standard Linux and Android. With its strict application sandbox, NAPI dynamic loading model, and musl libc runtime, bringing existing low-level network tools to the platform requires solving non-trivial compatibility challenges across the entire stack.

This post documents the end-to-end implementation of the CocoProxy VPN client on HarmonyOS—covering custom Go toolchains, NAPI lifecycle coordination, user-space TCP/IP via Google’s gVisor stack, socket protection against routing loops, and direct TUN packet crafting to defeat DNS poisoning.


1. System Architecture

1.1 Architecture Overview

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
┌────────────────────────────────────────────────────────┐
│ HarmonyOS Device │
│ │
│ ┌──────────────┐ ┌────────────────────────────┐ │
│ │ Browser/App │ │ CocoProxy VPN Extension │ │
│ │ │ │ │ │
│ │ Issues HTTP │ │ ┌────────────────────────┐ │ │
│ │ requests │ │ │ ArkTS VPN Ability │ │ │
│ └──────┬───────┘ │ │ (VpnExtensionAbility) │ │ │
│ │ │ └──────────┬─────────────┘ │ │
│ │ Route: │ │ NAPI │ │
│ │ 0.0.0.0/0 │ ▼ │ │
│ ▼ │ ┌────────────────────────┐ │ │
│ ┌──────────────┐ │ │ libcocoproxy.so │ │ │
│ │ TUN Device │◄────┼─┤ (Go c-shared .so) │ │ │
│ │ 172.19.0.1 │ │ │ │ │ │
│ │ tunFd │ │ │ · gVisor Network Stack │ │ │
│ └──────────────┘ │ │ · SOCKS5 Engine │ │ │
│ │ │ · DNS Cache & Reverse │ │ │
│ │ │ · Socket Protect Calls │ │ │
│ │ └────────────────────────┘ │ │
│ └─────────────┬──────────────┘ │
└─────────────────────────────────────┼──────────────────┘
│ Outbound (protect)
▼
┌──────────────────┐
│ Remote Proxy │
│ (SOCKS5 Server) │──────→ Target Site
└──────────────────┘

1.2 End-to-End Data Flow

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
User visits google.com
│
▼
[1] DNS Query: google.com → Local DNS (returns polluted/bogus IP)
│
▼
[2] Outbound TCP to BogusIP:443 matches routing table 0.0.0.0/0
│
▼
[3] TUN virtual interface captures raw IP packet → Handed to gVisor stack
│
▼
[4] NewConnection event: Destination = BogusIP:443
│
│ Reverse DNS cache lookup: BogusIP → "google.com" 🔑
│
▼
[5] Outbound TCP socket created to proxy server (marked via protect socket)
│
▼
[6] Outbound payload wrapped in SOCKS5 CONNECT (Domain Mode: google.com:443)
│
▼
[7] Remote proxy resolves actual IP and establishes true destination TCP link
│
▼
[8] Full-duplex transparent relay: Browser ◄──► gVisor ◄──► Proxy ◄──► Target

1.3 Key Architectural Decisions

Decision Area Selected Technology Technical Rationale
Network Stack gVisor (sing-tun) Runs entirely in user space; requires zero kernel modules
Go Interop c-shared (.so) + NAPI HarmonyOS sandboxes prohibit standalone daemon processes
VPN Subnet 172.19.0.1/30 Minimal clash with common local LAN subnets
MTU 9000 Reduces fragmentation overhead across high-throughput relays
Gateway Flag hasGateway = false Tailored to HarmonyOS network manager routing specs
Socket Protect NAPI ThreadSafeFunction Safely crosses goroutine boundaries into the ArkTS main thread

2. Dynamic Linking & NAPI Initialization

2.1 The NAPI Loading Difference

In Node.js, dlopen locates the exported napi_register_module_v1 symbol dynamically.

On HarmonyOS, the ArkCompiler loader executes constructor functions within .init_array immediately upon dlopen(), expecting napi_module_register() to be called synchronously.

Standard Go’s //export napi_register_module_v1 fails here. We use an explicit C constructor in napi_reg.c:

1
2
3
4
5
6
7
8
9
10
11
12
13
__attribute__((constructor))
void RegisterCocoProxyModule(void) {
napi_module_register(&coco_napi_module);
}

static napi_module coco_napi_module = {
.nm_version = 1,
.nm_flags = 0,
.nm_filename = NULL,
.nm_register_func = InitializeModuleWrapper,
.nm_modname = "cocoproxy",
.nm_priv = NULL,
};

2.2 Binary Patching (patch-so.ps1)

To prevent the Go runtime from auto-bootstrapping before ArkTS finishes setting up TLS states, our build pipeline patches the binary:

  1. Short-circuit _rt0_arm64_linux_lib: Overwrites the entry instruction with RET (0xd65f03c0), deferring Go runtime startup until ArkTS explicitly calls startTun.
  2. Relocate runtime entry address: Dynamically calculates _rt0_arm64_linux_lib_go and stores its virtual address in g_rt0_go_offset.

3. Network Subsystem Implementation

3.1 VPN Configuration & Virtual TUN

In ArkTS:

1
2
3
4
5
6
7
8
9
10
11
import vpnExtension from '@ohos.net.vpnExtension';

const vpnConfig: vpnExtension.VpnConfig = {
addresses: [{ address: '172.19.0.1', prefixLength: 30 }],
routes: [{ destination: '0.0.0.0/0', hasGateway: false }],
mtu: 9000,
dnsAddresses: ['172.19.0.2']
};

const vpnConnection = vpnExtension.createVpnConnection(context);
const tunFd = await vpnConnection.create(vpnConfig);

3.2 Preventing Routing Loops: Socket Protection via ThreadSafeFunction

All outbound system traffic is routed into tunFd by default. If the Go proxy opens a TCP socket to its upstream remote proxy, that connection would also be swept into tunFd, producing an immediate infinite routing loop.

HarmonyOS provides vpnConnection.protect(fd) to exempt specific sockets from the VPN routing table. However:

  • protect can only be invoked from the ArkTS main thread.
  • Outbound proxy dials happen asynchronously inside Go goroutines.

We bridge this using a NAPI ThreadSafeFunction (tsfn):

1
2
3
4
5
6
7
8
9
Go Goroutine                          ArkTS Main Thread
│ │
│ conn.Control(func(fd) { │
│ call_protect_tsfn(fdId, fd) ───────┼──→ tsfn callback
│ Wait for confirmation (timeout) │ vpnConnection.protect(fd)
│ ... │ notify_fd_protected(fdId)
│ Confirmed! │
▼ ▼
Outbound TCP connects safely outside VPN tunnel

3.3 Integrating gVisor’s User-Space TCP/IP Stack

We configure sing-tun with the raw file descriptor passed from ArkTS:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
tunOptions := tun.Options{
Name: "vpn-tun",
Inet4Address: []netip.Prefix{netip.MustParsePrefix("172.19.0.1/30")},
MTU: 9000,
AutoRoute: false,
FileDescriptor: tunFd,
}

stackOptions := tun.StackOptions{
Context: context.Background(),
Tun: tunDevice,
Handler: proxyHandler,
UDPTimeout: 5 * time.Minute,
}

tunStack, err := tun.NewStack("gvisor", stackOptions)
if err != nil {
return err
}
tunStack.Start()

3.4 Bypassing the gVisor Loopback Trap for DNS

In standard setups, stack.WritePacket() is used to return simulated DNS responses. In gVisor’s user-space implementation, however, calling WritePacket on locally terminated UDP packets causes the stack to consume the reply internally rather than emitting it back out the TUN descriptor.

The Solution:
We construct complete IPv4 + UDP packets manually in memory and write them directly into the underlying Linux tunFd via syscall.Write:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
func writeDnsResponseDirect(rawTunFd int, srcIP, dstIP net.IP, srcPort, dstPort uint16, dnsPayload []byte) error {
totalLen := 20 + 8 + len(dnsPayload)
buf := make([]byte, totalLen)

// Build IPv4 Header
buf[0] = 0x45 // Version 4, IHL 5
binary.BigEndian.PutUint16(buf[2:4], uint16(totalLen))
buf[8] = 64 // TTL
buf[9] = 17 // Protocol UDP
copy(buf[12:16], srcIP.To4())
copy(buf[16:20], dstIP.To4())
binary.BigEndian.PutUint16(buf[10:12], computeChecksum(buf[0:20]))

// Build UDP Header
binary.BigEndian.PutUint16(buf[20:22], srcPort)
binary.BigEndian.PutUint16(buf[22:24], dstPort)
binary.BigEndian.PutUint16(buf[24:26], uint16(8+len(dnsPayload)))

copy(buf[28:], dnsPayload)

// Direct kernel syscall
_, err := syscall.Write(rawTunFd, buf)
return err
}

4. Key Takeaways

  1. User-Space Stacks are Indispensable: Without root or kernel tun devices, running gVisor in user space via tunFd is the only viable path for high-performance proxy routing on locked platforms.
  2. NAPI Lifecycles Differ: Desktop Node.js assumptions fall apart on mobile OS platforms. Respect .init_array ordering and guard against premature Go runtime startup.
  3. Bridge Sockets Early: Crossing between goroutine thread pools and the host UI thread via NAPI ThreadSafeFunction must be bounded by deterministic timeouts to avoid socket starvation.

With this architecture in place, CocoProxy delivers high throughput and rock-solid connection resilience inside the HarmonyOS sandboxed environment.