Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

docs(gateway): a page per supported gateway with setup and deployment

Claudeae80336ff6af
diff --git a/docs/docs.json b/docs/docs.json
index 9d72348f..d85abf5e 100644
--- a/docs/docs.json
+++ b/docs/docs.json
@@ -151,7 +151,15 @@
"pages": [
"runtimes/index",
"runtimes/integration-model",
- "runtimes/llm-gateway",
+ {
+ "group": "LLM Gateways",
+ "root": "runtimes/llm-gateway",
+ "pages": [
+ "runtimes/gateways/litellm",
+ "runtimes/gateways/envoy",
+ "runtimes/gateways/portkey"
+ ]
+ },
{
"group": "Local Agents",
"pages": [
diff --git a/docs/runtimes/gateways/envoy.mdx b/docs/runtimes/gateways/envoy.mdx
new file mode 100644
index 00000000..d317aa1d
--- /dev/null
+++ b/docs/runtimes/gateways/envoy.mdx
@@ -0,0 +1,216 @@
+---
+title: "Envoy, Envoy Gateway and Istio"
+description: "Record agent model calls that pass through Envoy with beacon-gateway as an External Processing server"
+---
+
+## How it attaches
+
+`beacon-gateway` is an Envoy
+[External Processing](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter)
+(`ext_proc`) server. Envoy opens one gRPC stream per HTTP exchange and sends it the request and
+reply headers and bodies. Anything built on Envoy's HTTP filter chain can use it: plain Envoy,
+[Envoy Gateway](https://gateway.envoyproxy.io/) and Istio gateways or sidecars.
+
+Envoy carries the provider's own wire format, so `beacon-gateway` decodes the bodies the client
+sent and the provider returned. Only a `POST` with a body to a model endpoint is decoded. The
+path decides the protocol:
+
+| Path ends in | Protocol |
+|--------------|----------|
+| `/messages` | Anthropic Messages |
+| `/chat/completions` | OpenAI Chat Completions |
+| `/responses` | OpenAI Responses |
+
+Everything else (model lists, token counting, embeddings, OpenAI Assistants thread messages)
+passes through untouched.
+
+## What is supported
+
+| | |
+|---|---|
+| Protocols | Anthropic Messages, OpenAI Chat Completions and OpenAI Responses, JSON and server-sent events. Gzip replies are decompressed. |
+| Streaming | Yes. With the reply body `STREAMED`, chunks reach the client as they arrive and `beacon-gateway` records the reply when it ends. |
+| Token usage | Yes, from the provider's reply (`token.usage`). |
+| Sessions | A session header the client sends (`x-claude-code-session-id`, `session_id`, `x-session-id`, `conversation_id`), else a digest of the conversation's root when an identity is known, else one session per call. Claude Code sends `x-claude-code-session-id`. |
+| Identity | From request headers you name in `identity_headers`, typically set by Envoy's `jwt_authn` (`claim_to_headers`) or the gateway in front. Without identity headers and without a session header, every call is its own session. |
+| Client address | The first address in `x-forwarded-for` (set `use_remote_address: true` on the connection manager). |
+| Call id | `x-request-id`, which Envoy sets on every request. |
+
+<Note>
+Envoy must see the request in plain text. Put the `ext_proc` filter on a listener where the
+client's TLS is terminated (an API gateway, an egress gateway that originates TLS to the provider),
+not on a TCP passthrough.
+</Note>
+
+## Configure beacon-gateway
+
+```yaml title="beacon-gateway.yaml"
+gateways:
+ - type: envoy
+ addr: 0.0.0.0:4021
+ identity_headers:
+ user: x-user-id # any request header; credential headers are refused
+ email: x-user-email
+ team: x-team-id
+ # max_body: 33554432 # bytes held per body after decompression (default 32 MiB)
+outputs:
+ - jsonl: /var/log/beacon/gateway-events.jsonl
+ kinds: [beacon.event]
+```
+
+A body larger than `max_body` passes through unrecorded. The listener speaks plain gRPC with no
+authentication, so expose it only to Envoy.
+
+## Deploy with Envoy
+
+`gateway/deploy/envoy/envoy.yaml` is a complete, observe-only configuration. The parts that
+matter:
+
+```yaml title="envoy.yaml (excerpt)"
+http_filters:
+ - name: envoy.filters.http.ext_proc
+ typed_config:
+ "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
+ grpc_service:
+ envoy_grpc: { cluster_name: beacon_gateway }
+ message_timeout: 2s # the 200ms default is too short for a long conversation
+ failure_mode_allow: true # beacon-gateway down or slow: calls pass, unrecorded
+ processing_mode:
+ request_header_mode: SEND
+ response_header_mode: SEND
+ request_body_mode: BUFFERED
+ response_body_mode: STREAMED
+ request_trailer_mode: SKIP
+ response_trailer_mode: SKIP
+ - name: envoy.filters.http.router
+```
+
+- **Request `BUFFERED`.** The request arrives whole and is sent in one message. Raise
+ `per_connection_buffer_limit_bytes` on the listener and the provider cluster to match
+ `max_body` (the example uses 32 MiB); Envoy's 1 MiB default is too small for a long agent
+ conversation.
+- **Reply `STREAMED`.** Each chunk is forwarded as soon as `beacon-gateway` acknowledges it, which
+ it does at once, so streaming keeps its token-by-token latency.
+- **The `beacon_gateway` cluster needs HTTP/2** (`http2_protocol_options`), as every gRPC
+ cluster does.
+- **Route timeout.** Long generations outlast Envoy's 15 s default; the example sets `600s`.
+
+`beacon-gateway` tells buffered from streamed bodies by what arrives, so no setting on its side
+has to match Envoy's processing mode.
+
+Validate and run the example:
+
+```bash
+cd gateway
+./beacon-gateway -config deploy/beacon-gateway.yaml # with the envoy entry enabled
+docker run --rm -v $PWD/deploy/envoy:/cfg envoyproxy/envoy:v1.37-latest --mode validate -c /cfg/envoy.yaml
+docker run --rm -p 10000:10000 -v $PWD/deploy/envoy:/cfg envoyproxy/envoy:v1.37-latest -c /cfg/envoy.yaml
+```
+
+The example's `provider` cluster points at a local test upstream. For a real provider, point it
+at `api.anthropic.com:443` or `api.openai.com:443` with an `UpstreamTlsContext` and set
+`auto_host_rewrite: true` on the route (both are in the file's comments). Then point clients at
+Envoy, for example `ANTHROPIC_BASE_URL=http://<envoy>:10000` for Claude Code.
+
+## Deploy with Envoy Gateway
+
+Run `beacon-gateway` as a `Deployment` with a `Service` on port 4021, and attach it to the
+`HTTPRoute` that carries model traffic with an `EnvoyExtensionPolicy`:
+
+```yaml title="envoy-extension-policy.yaml"
+apiVersion: gateway.envoyproxy.io/v1alpha1
+kind: EnvoyExtensionPolicy
+metadata:
+ name: beacon-gateway
+ namespace: llm
+spec:
+ targetRefs:
+ - group: gateway.networking.k8s.io
+ kind: HTTPRoute
+ name: models
+ extProc:
+ - backendRefs:
+ - name: beacon-gateway
+ port: 4021
+ failOpen: true
+ messageTimeout: 2s
+ processingMode:
+ request:
+ body: Buffered
+ response:
+ body: Streamed
+```
+
+Set the route's request timeout high enough for long generations, and raise the proxy's buffer
+limit (`ClientTrafficPolicy` `connection.bufferLimit`) to cover large requests. If the gateway
+authenticates callers with a `SecurityPolicy` JWT provider, use its `claimToHeaders` to put the
+caller's claims on headers and name those headers in `identity_headers`.
+
+## Deploy with Istio
+
+Insert the filter with an `EnvoyFilter` on the workload that terminates the model traffic, such
+as an egress gateway that originates TLS to the provider:
+
+```yaml title="envoyfilter.yaml"
+apiVersion: networking.istio.io/v1alpha3
+kind: EnvoyFilter
+metadata:
+ name: beacon-gateway
+ namespace: istio-system
+spec:
+ workloadSelector:
+ labels:
+ istio: egressgateway
+ configPatches:
+ - applyTo: HTTP_FILTER
+ match:
+ context: GATEWAY
+ listener:
+ filterChain:
+ filter:
+ name: envoy.filters.network.http_connection_manager
+ subFilter:
+ name: envoy.filters.http.router
+ patch:
+ operation: INSERT_BEFORE
+ value:
+ name: envoy.filters.http.ext_proc
+ typed_config:
+ "@type": type.googleapis.com/envoy.extensions.filters.http.ext_proc.v3.ExternalProcessor
+ grpc_service:
+ envoy_grpc:
+ cluster_name: outbound|4021||beacon-gateway.beacon.svc.cluster.local
+ message_timeout: 2s
+ failure_mode_allow: true
+ processing_mode:
+ request_header_mode: SEND
+ response_header_mode: SEND
+ request_body_mode: BUFFERED
+ response_body_mode: STREAMED
+ request_trailer_mode: SKIP
+ response_trailer_mode: SKIP
+```
+
+Name the `beacon-gateway` Service port `grpc` (or set `appProtocol: grpc`) so Istio uses HTTP/2
+to it. For a sidecar, use `context: SIDECAR_OUTBOUND` and select the workload's labels instead;
+the sidecar sees plain text only when the application sends HTTP and the mesh originates TLS.
+
+## Troubleshooting
+
+| Symptom | Check |
+|---------|-------|
+| No events | The path ends in a supported endpoint, the call is a `POST`, and Envoy reaches `beacon-gateway` (Envoy's `ext_proc` stats count stream failures). |
+| Large requests unrecorded | The body exceeded `max_body`, or Envoy's buffer limit rejected it. Raise both together. |
+| Calls fail when `beacon-gateway` restarts | `failure_mode_allow: true` (Envoy) or `failOpen: true` (Envoy Gateway) is missing. |
+| Each call is its own session | The client sends no session header and no `identity_headers` are configured. |
+
+## Related
+
+<Columns cols={2}>
+ <Card title="LLM gateways" icon="network-wired" href="/runtimes/llm-gateway">
+ What gateway collection records and the settings shared by every gateway.
+ </Card>
+ <Card title="LiteLLM" icon="layer-group" href="/runtimes/gateways/litellm">
+ Attach to a LiteLLM proxy instead.
+ </Card>
+</Columns>
diff --git a/docs/runtimes/gateways/litellm.mdx b/docs/runtimes/gateways/litellm.mdx
new file mode 100644
index 00000000..7f5eb777
--- /dev/null
+++ b/docs/runtimes/gateways/litellm.mdx
@@ -0,0 +1,129 @@
+---
+title: "LiteLLM"
+description: "Record agent model calls that pass through a LiteLLM proxy with beacon-gateway"
+---
+
+## How it attaches
+
+`beacon-gateway` attaches to the LiteLLM proxy as a
+[Generic Guardrail API](https://docs.litellm.ai/docs/proxy/guardrails/quick_start) guardrail
+(`guardrail: generic_guardrail_api`). LiteLLM calls it twice per model call and waits for each
+answer:
+
+| Guardrail mode | LiteLLM sends | Beacon records |
+|----------------|---------------|----------------|
+| `pre_call` | The whole request: system text, tools, history and tool results | `session.context`, `mcp.tool_listed`, `prompt.submitted`, tool results |
+| `post_call` | The model's reply text and its tool calls | `agent.message`, `command.executed`, `file.read`, `mcp.tool_invoked`, `tool.invoked` |
+
+Both calls are `POST <api_base>/beta/litellm_basic_guardrail_api`. `beacon-gateway` answers
+`{"action": "NONE"}`, so the call goes ahead unchanged.
+
+## What is supported
+
+| | |
+|---|---|
+| Protocols | Every protocol LiteLLM serves (Anthropic Messages, OpenAI Chat Completions, OpenAI Responses). LiteLLM normalizes them to chat messages before it calls the guardrail. |
+| Streaming | Yes. With `streaming_buffer_until_moderated: false` the stream reaches the client as it arrives and the reply is recorded when it ends. |
+| Sessions | LiteLLM's `litellm_trace_id` names the session on both phases, unless the client sends a session header (`x-claude-code-session-id`, `session_id`, `x-session-id`, `conversation_id`) that LiteLLM forwards. |
+| Identity | From the LiteLLM virtual key: user id, user email, team id, org id, and the key's alias (or its hash). |
+| Client headers | Only headers on the guardrail's `extra_headers` list. LiteLLM sends the others as `[present]`, and Beacon ignores those. Credential headers are never recorded. |
+| Token usage | Not recorded. LiteLLM's guardrail payload carries the reply's text and tool calls, not its token counts. Use LiteLLM's own spend logs, or endpoint collection, for usage. |
+
+## Deploy
+
+<Steps>
+ <Step title="Build and start beacon-gateway">
+ ```bash
+ cd gateway
+ go build -o beacon-gateway ./cmd/beacon-gateway
+ ./beacon-gateway -config deploy/beacon-gateway.yaml
+ ```
+
+ The shipped configuration (and the built-in one, used when `-config` is omitted) enables
+ the LiteLLM adapter on `127.0.0.1:4019`:
+
+ ```yaml title="beacon-gateway.yaml"
+ gateways:
+ - type: litellm
+ addr: 127.0.0.1:4019
+ outputs:
+ - jsonl: /var/log/beacon/gateway-events.jsonl
+ kinds: [beacon.event]
+ ```
+
+ Run it next to the proxy (same host or same pod) so the guardrail call stays on loopback.
+ If LiteLLM reaches it over a network, bind `addr` to that interface and keep the port off
+ the internet: the listener has no authentication of its own.
+ </Step>
+ <Step title="Add the two guardrails to LiteLLM">
+ Add both entries to the proxy's `config.yaml` (`gateway/deploy/litellm.yaml` is a complete
+ example):
+
+ ```yaml title="config.yaml"
+ guardrails:
+ - guardrail_name: beacon-gateway-request
+ litellm_params:
+ guardrail: generic_guardrail_api
+ mode: pre_call
+ api_base: http://127.0.0.1:4019
+ default_on: true
+ unreachable_fallback: fail_open
+ extra_headers: [x-claude-code-session-id, session_id]
+ - guardrail_name: beacon-gateway-response
+ litellm_params:
+ guardrail: generic_guardrail_api
+ mode: post_call
+ api_base: http://127.0.0.1:4019
+ default_on: true
+ unreachable_fallback: fail_open
+ extra_headers: [x-claude-code-session-id, session_id]
+ streaming_buffer_until_moderated: false
+ ```
+
+ - `default_on: true` runs the guardrail on every call without clients opting in.
+ - `unreachable_fallback: fail_open` keeps the model path up when `beacon-gateway` is down;
+ those calls go through unrecorded.
+ - `extra_headers` lets the session headers through. Add any other header you want on the
+ event; never add credential headers (Beacon drops them anyway).
+ </Step>
+ <Step title="Restart the proxy and send a call">
+ ```bash
+ litellm --config config.yaml
+ ```
+
+ Send a model call through the proxy, then check the output file and the counters:
+
+ ```bash
+ tail -n 5 /var/log/beacon/gateway-events.jsonl
+ curl -s http://127.0.0.1:4029/stats
+ ```
+ </Step>
+</Steps>
+
+### Kubernetes
+
+Run `beacon-gateway` as a sidecar container in the LiteLLM proxy's pod, listening on
+`127.0.0.1:4019`, and keep `api_base: http://127.0.0.1:4019`. Every proxy replica then has its
+own `beacon-gateway`. With the open build's in-memory store each replica remembers only what it
+has seen, so a session whose calls land on several replicas records its system text and tool
+definitions once per replica. Write the output to a volume your log shipper reads.
+
+## Troubleshooting
+
+| Symptom | Check |
+|---------|-------|
+| No events | The guardrails have `default_on: true`, and `api_base` reaches the `addr` `beacon-gateway` listens on. `/stats` shows `calls` rising when they do. |
+| Each call is its own session | The client sends no session header, or it is missing from `extra_headers`, and LiteLLM gave no trace id. |
+| Replies missing for streamed calls | The `post_call` guardrail is present and its `streaming_buffer_until_moderated` is `false`. |
+| `user` is empty | The call used the proxy's master key or no virtual key; identity comes from the virtual key. |
+
+## Related
+
+<Columns cols={2}>
+ <Card title="LLM gateways" icon="network-wired" href="/runtimes/llm-gateway">
+ What gateway collection records and the settings shared by every gateway.
+ </Card>
+ <Card title="Envoy" icon="route" href="/runtimes/gateways/envoy">
+ Attach through Envoy, Envoy Gateway or Istio instead.
+ </Card>
+</Columns>
diff --git a/docs/runtimes/gateways/portkey.mdx b/docs/runtimes/gateways/portkey.mdx
new file mode 100644
index 00000000..ce503344
--- /dev/null
+++ b/docs/runtimes/gateways/portkey.mdx
@@ -0,0 +1,145 @@
+---
+title: "Portkey AI Gateway"
+description: "Record agent model calls that pass through Portkey with beacon-gateway as a webhook guardrail"
+---
+
+## How it attaches
+
+`beacon-gateway` is the target of Portkey's webhook guardrail check (`default.webhook`). Portkey
+runs a guardrail's checks before the provider call (`beforeRequestHook`) and after it
+(`afterRequestHook`), POSTs the hook's context to the webhook, and reads back
+`{"verdict": true|false}`.
+
+| Hook | Portkey sends | Beacon records |
+|------|---------------|----------------|
+| Input guardrail (`beforeRequestHook`) | The body the client sent, before Portkey translates it for the provider | `session.context`, `mcp.tool_listed`, `prompt.submitted`, tool results |
+| Output guardrail (`afterRequestHook`) | Portkey's reply in the client's format | `agent.message`, `token.usage`, tool calls |
+
+`beacon-gateway` always answers `200` with `{"verdict": true}`, so the call goes ahead. Portkey
+blocks on a false verdict only when the check is configured with `deny: true`; leave it unset.
+
+## What is supported
+
+| | |
+|---|---|
+| Protocols | Chat Completions (`chatComplete`) and Anthropic Messages (`messages`) requests. Other request types (completions, embeddings, OpenAI Responses) are ignored. |
+| Streaming | Requests are recorded. Replies to streamed requests are not: Portkey passes the stream through unparsed and sends the output hook no reply body. Non-streamed replies are recorded in full. |
+| Token usage | Yes, for non-streamed replies. |
+| Sessions | The `session_id` key of the client's `x-portkey-metadata` header, else a digest of the conversation's root when an identity is known, else one session per call. |
+| Identity | Keys of `x-portkey-metadata`. By default `_user` (Portkey's end-user field) is the user; `identity_metadata` maps others. |
+| Client headers | None. Portkey removes the client's headers from the webhook payload. |
+
+## Configure beacon-gateway
+
+```yaml title="beacon-gateway.yaml"
+gateways:
+ - type: portkey
+ addr: 127.0.0.1:4022
+ path: /portkey/webhook # default
+ token: ${PORTKEY_WEBHOOK_TOKEN} # required as "Authorization: Bearer <token>" when set
+ identity_metadata: # x-portkey-metadata keys
+ user: _user
+ team: team
+ org: org
+outputs:
+ - jsonl: /var/log/beacon/gateway-events.jsonl
+ kinds: [beacon.event]
+```
+
+Set `token` whenever the webhook is reachable by anything other than Portkey; `${VAR}` is
+expanded from the environment, so the token stays out of the file.
+
+## Deploy
+
+<Steps>
+ <Step title="Start beacon-gateway where Portkey can reach it">
+ ```bash
+ cd gateway
+ go build -o beacon-gateway ./cmd/beacon-gateway
+ PORTKEY_WEBHOOK_TOKEN=<random secret> ./beacon-gateway -config beacon-gateway.yaml
+ ```
+
+ With the open-source gateway self-hosted (`npx @portkey-ai/gateway`, the Docker image, or a
+ Kubernetes deployment), run `beacon-gateway` beside it and keep the webhook on loopback or
+ the cluster network. Portkey's hosted gateway calls the webhook from Portkey's
+ infrastructure, so it needs a public HTTPS URL in front of `beacon-gateway` (a reverse
+ proxy or load balancer terminating TLS), and the token.
+ </Step>
+ <Step title="Add the webhook as an input and an output guardrail">
+ In a Portkey config, add the webhook check to both hooks:
+
+ ```json title="Portkey config"
+ {
+ "input_guardrails": [
+ {
+ "default.webhook": {
+ "webhookURL": "http://127.0.0.1:4022/portkey/webhook",
+ "headers": { "Authorization": "Bearer <token>" }
+ }
+ }
+ ],
+ "output_guardrails": [
+ {
+ "default.webhook": {
+ "webhookURL": "http://127.0.0.1:4022/portkey/webhook",
+ "headers": { "Authorization": "Bearer <token>" }
+ }
+ }
+ ]
+ }
+ ```
+
+ In the Portkey app, create a guardrail with the **Webhook** check, the same URL and the
+ `Authorization` header, and attach it to the config as both an input and an output
+ guardrail. Do not turn on deny for an observe-only setup.
+
+ A webhook that times out or answers an error is an errored check, which Portkey treats as
+ passing unless the check sets `failOnError`. Leave that off so a `beacon-gateway` outage
+ never blocks model calls.
+ </Step>
+ <Step title="Send session and identity metadata from clients">
+ Clients (or the app in front of Portkey) send them in the metadata header:
+
+ ```http
+ x-portkey-metadata: {"_user": "jdoe", "session_id": "3f2c…", "team": "platform"}
+ ```
+
+ Without `session_id`, calls are grouped by the conversation's opening only when `_user` (or
+ another mapped identity) is present; otherwise every call is its own session.
+ </Step>
+ <Step title="Check it">
+ ```bash
+ tail -n 5 /var/log/beacon/gateway-events.jsonl
+ curl -s http://127.0.0.1:4029/stats
+ ```
+ </Step>
+</Steps>
+
+## Limits
+
+- **Pairing request and reply.** Portkey does not pass its trace or span id to the webhook, so
+ `beacon-gateway` pairs a reply with its request by a digest of the request body. Two calls with
+ byte-identical bodies share a call id, and a guardrail that rewrites the request before the
+ output hook breaks the pairing.
+- **Streamed replies** are not recorded (see above). Collect them from the endpoint, or have
+ clients that need reply coverage call without `stream`.
+
+## Troubleshooting
+
+| Symptom | Check |
+|---------|-------|
+| No events | The config is attached to the request (`x-portkey-config` or the virtual key's default config) and the webhook URL is reachable from where Portkey runs. |
+| `401` in `beacon-gateway`'s log | The `Authorization` header in the check does not match `token`. |
+| Prompts but no replies | The requests are streamed, or the output guardrail is missing. |
+| `user` is empty | `x-portkey-metadata` has no `_user` (or the key mapped in `identity_metadata`). |
+
+## Related
+
+<Columns cols={2}>
+ <Card title="LLM gateways" icon="network-wired" href="/runtimes/llm-gateway">
+ What gateway collection records and the settings shared by every gateway.
+ </Card>
+ <Card title="Envoy" icon="route" href="/runtimes/gateways/envoy">
+ Attach through Envoy, Envoy Gateway or Istio instead.
+ </Card>
+</Columns>
diff --git a/docs/runtimes/llm-gateway.mdx b/docs/runtimes/llm-gateway.mdx
index 4462ba6f..0ec5ea99 100644
--- a/docs/runtimes/llm-gateway.mdx
+++ b/docs/runtimes/llm-gateway.mdx
@@ -20,11 +20,15 @@ resolved for the call.
## Supported gateways
-| Gateway | Hook |
-|---------|------|
-| LiteLLM proxy | `generic_guardrail_api` guardrail, `pre_call` and `post_call` |
-| Envoy, Envoy Gateway, Istio | External Processing (`ext_proc`) filter |
-| Portkey AI Gateway | `default.webhook` guardrail |
+| Gateway | Hook | Protocols | Streamed replies | Token usage |
+|---------|------|-----------|------------------|-------------|
+| [LiteLLM proxy](/runtimes/gateways/litellm) | `generic_guardrail_api` guardrail, `pre_call` and `post_call` | All LiteLLM serves | Yes | No |
+| [Envoy, Envoy Gateway, Istio](/runtimes/gateways/envoy) | External Processing (`ext_proc`) filter | Anthropic Messages, Chat Completions, Responses | Yes | Yes |
+| [Portkey AI Gateway](/runtimes/gateways/portkey) | `default.webhook` guardrail | Chat Completions, Anthropic Messages | No | Yes (non-streamed) |
+
+Each page covers what that gateway exposes, how sessions and identity are named, and how to
+deploy it. One `beacon-gateway` process can serve several gateways at once: list one entry per
+gateway under `gateways`.
Model protocols: Anthropic Messages, OpenAI Chat Completions and OpenAI Responses, as JSON or
server-sent events.
@@ -50,6 +54,28 @@ content:
retention: metadata
```
+## Deployment notes
+
+These apply whichever gateway you attach to.
+
+- **Fail open.** Every shipped configuration lets model calls through unrecorded when
+ `beacon-gateway` is down or slow. Keep it that way: the open build observes only and never
+ blocks a call.
+- **Health and counters.** `/healthz` and `/stats` are served on the admin listener
+ (`admin: 127.0.0.1:4029`). `/stats` counts calls, calls let through unanalyzed when admission
+ was full, analysis failures and records dropped per output.
+- **Replicas.** The open build's `store` is in memory and per process. Run one `beacon-gateway`
+ per gateway replica (a sidecar), and expect a session whose calls land on several replicas to
+ record its system text and tool definitions once per replica.
+- **Load.** `pipeline.max_in_flight` (default two per CPU), `pipeline.max_body_bytes` and
+ `pipeline.queue_wait` bound the work in progress; a call that waits longer than `queue_wait`
+ goes through unrecorded. `gc.memory_limit` defaults to 80% of the container's memory limit.
+- **Outputs.** Write `beacon.event` records to a file your log shipper reads. The file uses the
+ runtime JSONL schema, so the same SIEM mappings apply.
+- **Sessions.** Clients that send a session header get one session per conversation. Claude Code
+ sends `x-claude-code-session-id`; other clients can send `session_id`, `x-session-id` or
+ `conversation_id`, or set `pipeline.session_headers`.
+
## What gets recorded
| The call carries | Event |
@@ -77,4 +103,13 @@ readers record, so a gateway's and an endpoint's record of one model call can be
<Card title="Integration model" icon="plug" href="/runtimes/integration-model">
How Beacon collects from each kind of agent runtime.
</Card>
+ <Card title="LiteLLM" icon="layer-group" href="/runtimes/gateways/litellm">
+ Guardrail API setup for the LiteLLM proxy.
+ </Card>
+ <Card title="Envoy, Envoy Gateway, Istio" icon="route" href="/runtimes/gateways/envoy">
+ External Processing setup for Envoy-based gateways.
+ </Card>
+ <Card title="Portkey" icon="shield-check" href="/runtimes/gateways/portkey">
+ Webhook guardrail setup for Portkey.
+ </Card>
</Columns>