The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
beacon-gateway
gateway/ is a Go module that attaches Beacon to an LLM gateway. A thin adapter connects to the
gateway's hook, and one shared pipeline turns every model call it sees into Beacon events in
runtime.jsonl's schema, with harness.collection_method set to gateway.
A gateway sees the conversation as the model receives it: the system prompt, the tool catalog, the
user's turns and tool results on the way in, and the model's text, token counts and tool calls on
the way out. It does not see the agent's own lifecycle, so its events complement the endpoint's
rather than replace them. Its endpoint is the gateway host and its user the identity the
gateway resolved.
Adapters
type |
Hook | Shape |
|---|---|---|
litellm |
LiteLLM generic_guardrail_api, pre_call and post_call (deploy/litellm.yaml) |
callout |
envoy |
Envoy External Processing (Envoy, Envoy Gateway, Istio; deploy/envoy/envoy.yaml) |
filter |
portkey |
Portkey default.webhook guardrail |
callout |
Protocols: Anthropic Messages, OpenAI Chat Completions and OpenAI Responses, JSON and SSE.
Running
cd gateway
go build -o beacon-gateway ./cmd/beacon-gateway
./beacon-gateway -config deploy/beacon-gateway.yaml
Without -config it serves the LiteLLM callout on 127.0.0.1:4019 and writes
gateway-events.jsonl in the working directory. /healthz and /stats are served on the admin
listener (127.0.0.1:4029). deploy/beacon-gateway.yaml documents every setting; the shipped
configurations observe only (LiteLLM fails open, Envoy streams replies and lets calls through when
the gateway is down).
content.retention: metadata writes events as Beacon's Metadata-only forwarding does: identifiers,
names, paths, command lines, model and token usage, and no prompt, reply, system instructions, tool
arguments or results.
Sessions
session.id is, in order: a session header the client sent (pipeline.session_headers, by default
x-claude-code-session-id, session_id, x-session-id, conversation_id); the gateway's own
conversation id (LiteLLM's trace id, Portkey's session_id metadata); or, failing both, a digest
of the conversation's root, root:<hex>: the gateway, the client's harness, the caller's identity,
the system text sent ahead of the messages (as written) and the first user message, which a client
resends unchanged on every call. A call's reply joins its request's session: Envoy and Portkey hand
the reply the request it answers, and the reply's session is named from it as the request's was
(LiteLLM names both by its trace id). A root needs an identity: without one (Envoy without
identity_headers, Portkey without identity metadata) different people's conversations that open
alike would be one session, so each call is its own session instead. Set identity headers to get
conversation-level sessions without a session header. The root is a heuristic: two conversations
of one caller in one harness with the same system text that open with the same message are one
session, and a client that changes its system text part-way through a conversation splits it into
two sessions there. System text sent after the messages (an Anthropic body may send it last) is
not part of the root, as reading it would mean reading past the whole history. Send a session
header, or Portkey session_id metadata, where that matters.
Naming a session reads no store: a call's two phases are named by the same function from the same request body, so they always agree, in any order and on any replica.
A resent tail is taken for a client's retry, analyzed but not recorded again, only in a session
the client or the gateway named: a session header, the gateway's conversation id, a provider
conversation id. In a root session or a call's own session a retry is recorded twice, and two
conversations that share a root never lose either one's prompts, only share its session.id.
An OpenAI Responses request that continues provider-side state carries only its new turn, so it
has no root. With conversation set, the conversation id is the session. A request that continues
previous_response_id, with no session header naming it, is its own session, <gateway>:<call id>, which its reply shares. Every event of the call names the response it continues in
raw["gateway.previous_response_id"], and the reply's events carry its own response id in
gen_ai.response.id, so a reader can put such a chain's turns back together. Such a request's
new turn is recorded every time: the same short input twice ("yes", "go on") is two turns, never
a retry.
The cost of a turn being its own session: the Responses API does not carry instructions or
tools across responses, so the client resends them every turn, and each turn records them again
(session.context, mcp.tool_listed) and has its own session.id. This is deliberate. Naming
such a turn's session from content (the caller and its instructions) would merge every
conversation one caller has with the same instructions, and naming it from the response it
continues would need state shared across calls and replicas. A client that chains responses this
way and wants one session per conversation, with its context recorded once, sends a session
header.
Events
| What the call carries | Event |
|---|---|
| New system text | session.context (gen_ai.system_instructions) |
| New MCP tool definition | mcp.tool_listed |
| New user text | prompt.submitted |
| The model's text | agent.message (gen_ai.output.messages, gen_ai.response) |
| The reply's token counts | token.usage (gen_ai.usage, gen_ai.response) |
| A shell tool call | command.executed |
| A read tool call | file.read, with file.path |
| An MCP tool call | mcp.tool_invoked, with mcp.server and mcp.tool |
| A fetch or any other tool call | tool.invoked (tool.path is a fetch's URL) |
| A read's or an MCP tool's result | file.read / mcp.tool_invoked, with gen_ai.tool.call.result and the call's arguments |
| Any other result | tool.completed (tool.failed for an error), keeping the call's target |
A gateway sees a tool call before the harness acts on it, so a call is recorded as the action it
asks for, and a rule can judge a read or a command before it happens. A read's or an MCP tool's
result repeats its call's action, which is where Beacon's rules read content from outside the
agent. The result never carries the target the call carried (the path, or the MCP server and
tool): a rule on the target fires once, on the call, and a call and its result never share an
event.id, whichever replica saw the call. The call's arguments are on the result's
gen_ai.tool.call.arguments, so a reader can still see what it answers.
System text and tool definitions are resent on every call and recorded once per session, and so is
a request's new tail (its user text and tool results) in a session the client or the gateway
named: a client that retries a request sends the same tail, which is analyzed again but not
recorded again (see Sessions). model is the model the client asked
for on every event of a call; gen_ai.response.model is the provider's own name for it. The
gateway's call id is in raw["gateway.call_id"], and the response a Responses call continues in
raw["gateway.previous_response_id"] (on a reply, when no session header names it).
Extension points
The pipeline has generic seams (pipeline.Analyzer, pipeline.Policy, pipeline.Observer,
pipeline.Sink, store.Store). A separately distributed build can compile in files of its own that
register analyzers, a policy, stores, per-call observers and outputs in
cmd/beacon-gateway/extend.go; each reads its own node of the configuration file. The open build
registers none beyond the in-memory store and the jsonl output: every call is allowed, and with
nothing that could change a verdict the pipeline answers each call as soon as it is admitted and
records it afterwards.
An analyzer, policy or observer sees a call only for the length of its own method: the callout
adapters read bodies into pooled buffers that the pipeline returns once it has read the call
(gateway.Call.Release), so none may keep the pipeline.Turn, its Call, or bytes of the
Call.Body afterwards. Everything the pipeline decodes from a body is a copy.
An output is an entry of outputs, naming its type and the record kinds it receives:
outputs:
- jsonl: /var/log/beacon/gateway-events.jsonl # shorthand for type: jsonl
kinds: [beacon.event]
- type: <a registered output> # its own keys alongside
kinds: [beacon.event, finding]
A sink receives its kinds in batches, off the call path; when it falls behind, records are dropped
and counted in /stats.
Testing
cd gateway && go vet ./... && go test ./... && go test -race ./pipeline/... ./events/... ./litellm/... ./envoy/... ./portkey/... ./llm/...
go test ./internal/perf -run '^$' -bench . -benchmem