Run your first agent
Apply a minimal example under `clrk dev`, watch a sandbox spawn, and confirm the Anthropic API key never reaches the agent.
This is the smallest CLRK setup that exercises the parts that actually matter: a sandboxed agent makes an outbound LLM call, the proxy injects the API key on the way out, and one OTLP record per call lands in your observability pipeline. End to end in about a minute.
What you'll build
A DaemonAgent named echo-bot that POSTs to api.anthropic.com on a
loop. The Anthropic key lives in a Kubernetes Secret. A
CredentialInjectionPolicy swaps it in at the egress proxy. The agent
itself never observes the real key - we'll verify that at the end by
reading the OTLP-captured request.
Prerequisites
- The
clrkCLI installed and on your$PATH. kubectlinstalled (the verification steps below use it for theEgressGatewayandSecretchecks).- Docker running on the host.
- An Anthropic API key in your shell:
export ANTHROPIC_API_KEY=sk-ant-.... - macOS or Linux. You do not need an existing Kubernetes cluster -
clrk devbrings up its own k3s in a container.
Bring it up
From a checkout of github.com/apoxy-dev/clrk:
ANTHROPIC_API_KEY=sk-ant-... clrk dev \
--apply _examples/echo-bot/manifests \
--secret anthropic-credentials=ANTHROPIC_API_KEY:api-keyclrk dev boots k3s, the controller-manager, and a worker; then it
materializes the anthropic-credentials Secret from your shell env and
server-side-applies the example manifests. The --secret flag takes
NAME=ENVVAR[:KEY] - without :KEY the data key defaults to ENVVAR
lowercased with _ replaced by -. We pass :api-key explicitly so
the CredentialInjectionPolicy can reference it by name.
Wait roughly thirty seconds. The TUI settles when k3d-clrk-dev-server-0,
controller-manager, and worker-0 are all reporting healthy.
What just happened
Five CRDs are at play. Four of them ship as YAML in
_examples/echo-bot/manifests
(aiproviderroute, credentialinjectionpolicy, daemonagent,
egressgateway); the fifth, the default WorkerPool, is created for
you by clrk dev, so its YAML is not in that directory. Open the
manifests alongside the list below - you will meet each of these in
every other guide:
DaemonAgentis a long-lived agent process. The supervisor respawns it according torestartPolicy. UseDaemonAgentfor self-driven loops and watchers. For request-driven agents (HTTP endpoints, cron triggers) useTaskAgentinstead - see Package a custom agent.WorkerPoolis the fleet of worker pods your sandboxes land on. Thedefaultpool is auto-created byclrk dev. In production you define your own with sizing and node selectors.EgressGatewayis the TLS-terminating proxy every outbound connection from the sandbox traverses. It is where credentials get injected and where OTLP records are emitted.AIProviderRoutedeclares the upstream LLM provider (here, Anthropic) and binds policy filters to it.CredentialInjectionPolicyattaches aSecretto a route and rewrites the named header on requests crossing the proxy.
The agent itself is a curl loop. The interesting part is what
surrounds it.
See it running
You can talk to the in-cluster apiserver via the host kubeconfig that
clrk dev writes to ~/.clrk/kubeconfig.host. The apiserver is
published on localhost at a free port chosen at startup; that kubeconfig
already points at it, so just export KUBECONFIG=~/.clrk/kubeconfig.host
(or pass --local to clrk).
export KUBECONFIG=~/.clrk/kubeconfig.host
# Confirm the DaemonAgent reconciled.
clrk agents get echo-bot --local
# Confirm the EgressGateway came up.
kubectl get egressgateway echo-bot
# Confirm the Secret is present (clrk dev materialized it from --secret).
kubectl get secret anthropic-credentialsWatch the TUI as the agent runs. After the first cycle (roughly five
seconds in), the otel-logs pane carries one line per call:
15:04:05.123 POST api.anthropic.com/v1/messages 200 540ms req=312B resp=1024B provider=anthropic model=claude-haiku-4-5 input_tokens=12 output_tokens=24 route=default/anthropic trace=a1b2c3d4The matching span lands in the otel-traces pane. These records come
from the EgressGateway's ext_proc, the same surface that emits to your
real OTLP collector in production - see Send telemetry to OTLP
endpoints.
Confirm the agent never saw the key
The authority is the OTLP record itself, not the process environment.
Two things to check in the otel-traces pane:
- The call returned 200. Anthropic rejects requests without a
valid
x-api-keywith401 Unauthorized. A 200 paired with non-zeroinput_tokensandoutput_tokensproves the proxy supplied the credential - the agent did not. - The captured request header is redacted. Request and response
headers are always attached to the span as event attributes named
http.request.header.<name>. Expand the span and thehttp.request.header.x-api-keyattribute reads[redacted]- CLRK replaces known credential headers (authorization,x-api-key,anthropic-api-key, ...) with[redacted]before exporting, so telemetry never carries the real value. Theotlp.captureBodyblock on the EgressGateway (maxBytes: 65536) additionally captures the request/response body bytes (base64); it does not control header capture.
If you want to double-check from the agent's side, edit
_examples/echo-bot/manifests/daemonagent.yaml to add
echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY" to the loop body and
re-apply. You will see the variable is unset (the manifest does not
declare it under spec.template.spec.env), and the call still
succeeds - because the injection happens at the proxy, after the
request has already left the sandbox.
Tear down
Ctrl-C the dev TUI to stop the local cluster. State persists under
~/.clrk/; the next clrk dev reattaches to it. For a clean slate:
rm -rf ~/.clrkWhere to next
- Run an agent on demand instead of in a loop - see Package a custom agent.
- Trigger an agent from a cron schedule - see Schedule recurring agents.
- Wire the OTLP records you just saw into Honeycomb, Tempo, or your collector - see Send telemetry to OTLP endpoints.
- Understand the full credential-injection lifecycle - see Hide credentials from agents.
- Restrict the agent so it can only reach the upstreams you allow - see Lock down agent egress.