Getting startedGuidesReferenceChangelog
Apoxy:// Docs / Reference / Infra Projects

Infra Projects

Internal desired-state records that assign projects to shards and report project API server health.

An infra.apoxy.dev/v1alpha1 Project is the shard-local desired-state record for one Apoxy project.

Heads up

Internal API: Cosmos and infra-apiserver own this resource. Do not create, update, or delete it manually.

Desired state

Each infra-apiserver database represents one shard. A Project in that database assigns the project to the shard:

  • Object presence assigns the project to the shard.
  • Object deletion removes the assignment.
  • spec.apiServerEndpoint changes reconnect consumers to the new project API server.
  • Health status never adds or removes membership.

The project UUID is metadata.name. spec.shardID and the infra.apoxy.dev/shard-id label protect consumers from wrong-shard wiring and stale database restores.

Write ownership

Write access is split by subresource:

  • Project spec and deletion - The Cosmos project lifecycle identity owns creates, updates, and deletes.
  • Project status - The infra-apiserver loopback identity owns health observations.
  • Reads and watches - Authenticated shard services list and watch Projects for membership.

The setup workflow writes the Project after it persists the shard assignment and before it provisions the namespace, storage, or project API server. A failed provisioning run leaves the Project present. Rerun Project Setup to repair provisioning without reconstructing membership.

Project Teardown deletes the Project before later resource cleanup. It does not delete the Cosmos project row or shard assignment.

Health status

The infra-apiserver probes https://<spec.apiServerEndpoint>/readyz. The first successful response sets Ready=True. Three consecutive failures set Ready=False; one successful probe recovers immediately.

Endpoint changes reset health to Ready=Unknown until the new generation responds. Healthy probe timestamps are persisted at most once per minute to limit database writes and watch traffic.

Consumers treat status as observation only. They keep an unhealthy Project assigned and let existing connection retry logic recover when its API server returns.

Failure recovery

  • Setup failed after Project creation - Rerun Project Setup. The workflow reuses the assignment and upserts the same Project.
  • Project is missing from infra-apiserver - Rerun Project Setup to recreate it.
  • Endpoint is wrong - Correct the lifecycle inputs and rerun Project Setup. Do not patch the Project directly.
  • Shard move requested - Use the dedicated shard migration workflow. Ordinary Project Setup rejects moves.
  • Health probe failed - Repair the project API server or endpoint. Health failure never triggers teardown.

Watch reconnects preserve the last known desired membership. A transport failure does not produce an empty membership snapshot.

Membership rollout modes

Shared shard services select the source with --shard_membership_mode:

  • terra - Poll Cosmos once per second. This remains the rollout default.
  • compare - Use Terra and report differences against infra Projects.
  • infra-preferred - Use infra Projects after the initial list. Fall back to Terra when infra access fails initially or remains unavailable for 30 seconds.
  • infra - Use infra Project list/watch only.

Backplane and tunnelproxy reuse --infra_addr and --infra_insecure. Shared workerd-manager accepts the same flags. Status-only Project updates refresh health comparison data but do not reconnect project clients.