GCE userspace VPC tunnels

This guide runs an Apoxy VPC tunnel agent on a Google Compute Engine VM without a kernel TUN device. The VM reads its Apoxy credential from Secret Manager, starts a dual-stack demo service, and connects to the relay in userspace mode.

The request path is:

$terminalTXT

HTTPS → Gateway → HTTPRoute → Backend → VPCService → relay → GCE VM → service

Check prerequisites

The Apoxy CLI is authenticated to the intended project.
The managed Apoxy project is ready.
The Google Cloud CLI is authenticated to a project where you can create Compute Engine, IAM, and Secret Manager resources.
jq is installed locally.

Choose the relay nearest the VM from apoxy vpc relay list, then set the demo values:

$terminalSH

apoxy auth --check
apoxy gateway get default
apoxy vpc network get default
apoxy vpc relay list

export GCP_PROJECT=<your-project-id>
export GCP_ZONE=<your GCP zone>
export GCP_NETWORK=<your GCP network>
export GCP_SUBNET=<your GCP subnet>
export INSTANCE_NAME=apoxy-tunnel-agent
export SERVICE_ACCOUNT_NAME=apoxy-tunnel-agent
export SECRET_NAME=apoxy-vpc-token
export AGENT_NAME=gce-private-api
export RELAY_ADDRESS=<closest relay address>.relay.apoxy.net:6081

gcloud auth list --filter=status:ACTIVE
gcloud config set project "$GCP_PROJECT"
gcloud services enable \
  compute.googleapis.com \
  iam.googleapis.com \
  secretmanager.googleapis.com

Store the tunnel credential

Create a dedicated service account and store the default VPC network's token without a trailing newline:

$terminalSH

gcloud iam service-accounts create "$SERVICE_ACCOUNT_NAME" \
  --display-name="Apoxy VPC tunnel agent"
export SERVICE_ACCOUNT_EMAIL="${SERVICE_ACCOUNT_NAME}@${GCP_PROJECT}.iam.gserviceaccount.com"

APOXY_VPC_TOKEN="$(
  apoxy vpc network get default -o json | jq -jr '.status.credentials.token'
)"
test -n "$APOXY_VPC_TOKEN"
printf '%s' "$APOXY_VPC_TOKEN" | gcloud secrets create "$SECRET_NAME" \
  --replication-policy=automatic \
  --data-file=-
unset APOXY_VPC_TOKEN

gcloud secrets add-iam-policy-binding "$SECRET_NAME" \
  --member="serviceAccount:${SERVICE_ACCOUNT_EMAIL}" \
  --role=roles/secretmanager.secretAccessor

Only the dedicated service account can read this secret. The VM receives the secret's name and other non-secret settings as instance metadata; the credential itself is never stored there.

Create the VM

Download the maintained startup script:

$terminalSH

curl --fail --silent --show-error \
  --output apoxy-tunnel-agent-gce.sh \
  https://apoxy.dev/docs/templates/apoxy-tunnel-agent-gce.sh

Create a Debian 12 VM with the dedicated identity:

$terminalSH

gcloud compute instances create "$INSTANCE_NAME" \
  --zone="$GCP_ZONE" \
  --machine-type=e2-small \
  --image-family=debian-12 \
  --image-project=debian-cloud \
  --boot-disk-type=pd-balanced \
  --boot-disk-size=20GB \
  --network="$GCP_NETWORK" \
  --subnet="$GCP_SUBNET" \
  --service-account="$SERVICE_ACCOUNT_EMAIL" \
  --scopes=cloud-platform \
  --metadata="apoxy-vpc-secret=${SECRET_NAME},apoxy-agent-name=${AGENT_NAME},apoxy-vpc-name=default,apoxy-relay-address=${RELAY_ADDRESS}" \
  --metadata-from-file=startup-script=apoxy-tunnel-agent-gce.sh

The script installs Docker, copies the Secret Manager value to a root-only file, and runs a pinned multi-architecture Apoxy image with host networking. Userspace mode forwards traffic arriving on the VPC overlay to the VM's loopback network, so it does not need NET_ADMIN or /dev/net/tun.

Network egress

The VM needs outbound HTTPS, DNS, and UDP access to the selected relay port. The command above assigns an external IPv4 address. To use --no-address, provide Cloud NAT or equivalent internet egress. No inbound rule is required for the demo service itself.

Verify the agent

Inspect the startup service if the tunnel is not ready yet:

$terminalSH

gcloud compute ssh "$INSTANCE_NAME" \
  --zone="$GCP_ZONE" \
  --command='sudo systemctl status google-startup-scripts.service --no-pager'

Then verify both services, the agent health endpoint, and the private backend:

$terminalSH

gcloud compute ssh "$INSTANCE_NAME" \
  --zone="$GCP_ZONE" \
  --command='sudo systemctl is-active apoxy-demo-backend.service apoxy-tunnel.service; curl --fail http://127.0.0.1:8081/healthz; curl --fail http://127.0.0.1:8080/'
apoxy vpc tunnel list

Both services return active, the health endpoint reports one active connection, and the backend returns Hello through the Apoxy GCE userspace tunnel. The tunnel list includes an agent labeled tunnel.apoxy.dev/name=gce-private-api.

Credential visibility

The alpha client does not yet accept a token file directly. The container reads the root-only file at startup, but the expanded token is visible in the agent process arguments to an operator with host-level access. Keep the VM access-restricted and rotate the token after a credential exposure.

Publish the private service

Create a VPCService that selects the GCE agent:

$gce-vpc-service.yamlYAML

apiVersion: vpc.apoxy.dev/v1alpha1
kind: VPCService
metadata:
  name: gce-private-api
spec:
  networkRef:
    name: default
  selector:
    matchLabels:
      tunnel.apoxy.dev/name: gce-private-api

$terminalSH

apoxy apply -f gce-vpc-service.yaml
apoxy vpc service get gce-private-api -o yaml

Continue when Ready=True and at least one endpoint is present.

Attach a hostname to the default Gateway. Replace gce-api.example.com; for an Apoxy-managed zone, also set spec.zone to the zone root.

$gce-domain.yamlYAML

apiVersion: core.apoxy.dev/v1alpha3
kind: DomainRecord
spec:
  name: gce-api.example.com
  tls: {}
  target:
    ref:
      group: gateway.apoxy.dev
      kind: Gateway
      name: default

$terminalSH

apoxy apply -f gce-domain.yaml

Route port 8080 to the VPCService's internal name:

$gce-route.yamlYAML

apiVersion: core.apoxy.dev/v1alpha2
kind: Backend
metadata:
  name: gce-private-api
spec:
  endpoints:
  - fqdn: gce-private-api.default.vpc.apoxy.net
---
apiVersion: gateway.apoxy.dev/v1
kind: HTTPRoute
metadata:
  name: gce-private-api
spec:
  parentRefs:
  - name: default
    port: 443
  hostnames:
  - gce-api.example.com
  rules:
  - backendRefs:
    - group: core.apoxy.dev
      kind: Backend
      name: gce-private-api
      port: 8080

$terminalSH

apoxy apply -f gce-route.yaml
apoxy domain list \
  --field-selector spec.name=gce-api.example.com \
  -o yaml
curl https://gce-api.example.com/

A successful request returns Hello through the Apoxy GCE userspace tunnel.

Clean up

Delete the Apoxy objects before terminating their selected agent, then remove the dedicated Google Cloud resources:

$terminalSH

apoxy delete -f gce-route.yaml
apoxy delete -f gce-domain.yaml
apoxy delete -f gce-vpc-service.yaml

gcloud compute instances delete "$INSTANCE_NAME" \
  --zone="$GCP_ZONE" \
  --quiet
gcloud secrets delete "$SECRET_NAME" --quiet
gcloud iam service-accounts delete "$SERVICE_ACCOUNT_EMAIL" --quiet