# Run your first agent

> Apply a minimal example under `clrk dev`, watch a sandbox spawn, and confirm the Anthropic API key never reaches the agent.

This is the smallest CLRK setup that exercises the parts that actually
matter: a sandboxed agent makes an outbound LLM call, the proxy
injects the API key on the way out, and one OTLP record per call lands
in your observability pipeline. End to end in about a minute.

## What you'll build

A `DaemonAgent` named `echo-bot` that POSTs to `api.anthropic.com` on a
loop. The Anthropic key lives in a Kubernetes `Secret`. A
`CredentialInjectionPolicy` swaps it in at the egress proxy. The agent
itself never observes the real key - we'll verify that at the end by
reading the OTLP-captured request.

## Prerequisites

- The `clrk` CLI installed and on your `$PATH`.
- `kubectl` installed (the verification steps below use it for the
  `EgressGateway` and `Secret` checks).
- Docker running on the host.
- An Anthropic API key in your shell: `export ANTHROPIC_API_KEY=sk-ant-...`.
- macOS or Linux. You do not need an existing Kubernetes cluster - `clrk
  dev` brings up its own k3s in a container.

## Bring it up

From a checkout of [github.com/apoxy-dev/clrk](https://github.com/apoxy-dev/clrk):

```bash
ANTHROPIC_API_KEY=sk-ant-... clrk dev \
  --apply _examples/echo-bot/manifests \
  --secret anthropic-credentials=ANTHROPIC_API_KEY:api-key
```

`clrk dev` boots k3s, the controller-manager, and a worker; then it
materializes the `anthropic-credentials` Secret from your shell env and
server-side-applies the example manifests. The `--secret` flag takes
`NAME=ENVVAR[:KEY]` - without `:KEY` the data key defaults to `ENVVAR`
lowercased with `_` replaced by `-`. We pass `:api-key` explicitly so
the `CredentialInjectionPolicy` can reference it by name.

Wait roughly thirty seconds. The TUI settles when `k3d-clrk-dev-server-0`,
`controller-manager`, and `worker-0` are all reporting healthy.

## What just happened

```mermaid
flowchart TB
  DA[DaemonAgent] --> S[sandbox]
  WP[WorkerPool] -. runs .-> S
  S --> EG[EgressGateway]
  CIP[CredentialInjectionPolicy] -. "swap key" .-> EG
  EG --> APR[AIProviderRoute]
  APR --> A[Anthropic]
```

Five CRDs are at play. Four of them ship as YAML in
[`_examples/echo-bot/manifests`](https://github.com/apoxy-dev/clrk/tree/main/_examples/echo-bot/manifests)
(`aiproviderroute`, `credentialinjectionpolicy`, `daemonagent`,
`egressgateway`); the fifth, the `default` WorkerPool, is created for
you by `clrk dev`, so its YAML is not in that directory. Open the
manifests alongside the list below - you will meet each of these in
every other guide:

- **`DaemonAgent`** is a long-lived agent process. The supervisor
  respawns it according to `restartPolicy`. Use `DaemonAgent` for
  self-driven loops and watchers. For request-driven agents (HTTP
  endpoints, cron triggers) use `TaskAgent` instead - see [Package a
  custom agent](./package-a-custom-agent).
- **`WorkerPool`** is the fleet of worker pods your sandboxes land on.
  The `default` pool is auto-created by `clrk dev`. In production you
  define your own with sizing and node selectors.
- **`EgressGateway`** is the TLS-terminating proxy every outbound
  connection from the sandbox traverses. It is where credentials get
  injected and where OTLP records are emitted.
- **`AIProviderRoute`** declares the upstream LLM provider (here,
  Anthropic) and binds policy filters to it.
- **`CredentialInjectionPolicy`** attaches a `Secret` to a route and
  rewrites the named header on requests crossing the proxy.

The agent itself is a `curl` loop. The interesting part is what
surrounds it.

## See it running

You can talk to the in-cluster apiserver via the host kubeconfig that
`clrk dev` writes to `~/.clrk/kubeconfig.host`. The apiserver is
published on localhost at a free port chosen at startup; that kubeconfig
already points at it, so just `export KUBECONFIG=~/.clrk/kubeconfig.host`
(or pass `--local` to `clrk`).

```bash
export KUBECONFIG=~/.clrk/kubeconfig.host

# Confirm the DaemonAgent reconciled.
clrk agents get echo-bot --local

# Confirm the EgressGateway came up.
kubectl get egressgateway echo-bot

# Confirm the Secret is present (clrk dev materialized it from --secret).
kubectl get secret anthropic-credentials
```

Watch the TUI as the agent runs. After the first cycle (roughly five
seconds in), the `otel-logs` pane carries one line per call:

```
15:04:05.123 POST api.anthropic.com/v1/messages 200 540ms req=312B resp=1024B provider=anthropic model=claude-haiku-4-5 input_tokens=12 output_tokens=24 route=default/anthropic trace=a1b2c3d4
```

The matching span lands in the `otel-traces` pane. These records come
from the EgressGateway's ext_proc, the same surface that emits to your
real OTLP collector in production - see [Send telemetry to OTLP
endpoints](./send-telemetry-to-otlp).

## Confirm the agent never saw the key

The authority is the OTLP record itself, not the process environment.
Two things to check in the `otel-traces` pane:

1. **The call returned 200.** Anthropic rejects requests without a
   valid `x-api-key` with `401 Unauthorized`. A 200 paired with
   non-zero `input_tokens` and `output_tokens` proves the proxy
   supplied the credential - the agent did not.
2. **The captured request header is redacted.** Request and response
   headers are always attached to the span as event attributes named
   `http.request.header.<name>`. Expand the span and the
   `http.request.header.x-api-key` attribute reads `[redacted]` - 
   CLRK replaces known credential headers (`authorization`,
   `x-api-key`, `anthropic-api-key`, ...) with `[redacted]` before
   exporting, so telemetry never carries the real value. The
   `otlp.captureBody` block on the EgressGateway (`maxBytes: 65536`)
   additionally captures the request/response body bytes (base64); it
   does not control header capture.

If you want to double-check from the agent's side, edit
`_examples/echo-bot/manifests/daemonagent.yaml` to add
`echo "ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY"` to the loop body and
re-apply. You will see the variable is unset (the manifest does not
declare it under `spec.template.spec.env`), and the call still
succeeds - because the injection happens at the proxy, after the
request has already left the sandbox.

## Tear down

`Ctrl-C` the dev TUI to stop the local cluster. State persists under
`~/.clrk/`; the next `clrk dev` reattaches to it. For a clean slate:

```bash
rm -rf ~/.clrk
```

## Where to next

- Run an agent on demand instead of in a loop - see [Package a custom
  agent](./package-a-custom-agent).
- Trigger an agent from a cron schedule - see [Schedule recurring
  agents](./schedule-recurring-agents).
- Wire the OTLP records you just saw into Honeycomb, Tempo, or your
  collector - see [Send telemetry to OTLP
  endpoints](./send-telemetry-to-otlp).
- Understand the full credential-injection lifecycle - see [Hide
  credentials from agents](./hide-credentials-from-agents).
- Restrict the agent so it can only reach the upstreams you allow - 
  see [Lock down agent egress](./lock-down-agent-egress).