The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
Strip remaining content fields in Metadata-only forwarding
Claude470660a37258
diff --git a/CLAUDE.md b/CLAUDE.mdindex 83eef3a9..0bb51247 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -27,7 +27,7 @@ Do not recreate or depend on removed `asymptote` mirror trees. Keep new work foc- Beacon is visibility-first endpoint telemetry for local AI agent runtimes, not a hosted policy service or general endpoint protection product.- Preserve the local-only default. The public Beacon build must not require a hosted account, remote policy fetch, hosted dashboard, or external network dependency during normal hook execution, and hooks themselves never touch the network.- `beacon login` is the account-identity channel for interactive users and is required by the interactive `beacon endpoint install` wizard. It uses browser PKCE against beacon.sh and stores the scoped CLI token only in `~/.beacon/auth/session.json` (0600 under a 0700 directory). Signing in alone never starts forwarding. User-mode `beacon endpoint connect` may send that token once to `POST /api/cli/enroll/account` to authorize minting a separate device ingest key, and `beacon mcp token create` may send it to `POST /api/cli/mcp/tokens` (scope `mcp:token:create`, listed on the sign-in page) to mint a personal MCP token that is printed once to stdout and never written to a file, and `beacon share` may send it (scope `traces:write`, also on the sign-in page) to `PUT /v1/traces/{external_id}`, `POST /v1/traces/{external_id}/events/batch` and `GET /v1/traces/{external_id}` on the session's own base URL to upload a shared trace the person chose to publish, as a `beacon.trace.v1` summary and its events (`cli/beacon/internal/account/share.go`; a `bcn_api_` API key from `--key` or `BEACON_API_KEY` may stand in for it, precedence `--key`, then a current sign-in carrying the scope, then the environment). These are explicit, user-initiated requests with redirects refused. Hooks, plugins and background jobs never make them, with one exception: the git pre-push hook, only when the person opted in with `beacon git setup --share-sessions` (public needs confirmation), may make the three share requests during a push the person started, within the hook's budget and never failing the push. Never copy the account token into endpoint config, Vector, hook environments, logs, or command output. System/root/package/MDM/CI/piped-stdin/dry-run installs stay noninteractive and account-free.-- Beacon Cloud forwarding (`cli/beacon/internal/endpoint/asymptote`, `beacon endpoint connect`, `beacon endpoint asymptote`) is Vector-shaped: Beacon writes local JSONL, Vector ships runtime lines only from the connection point forward and reads inventory from the beginning to establish the endpoint's current baseline, plus the one bounded exception of the session backfill (below), and the `bcn_device_*` key stays only in `<BaseDir>/asymptote/vector-secrets.json` via Vector's secret backend. The onboarding wizard preselects Beacon Cloud, and confirming it connects this endpoint in the same command once the local install has succeeded; an endpoint that is already enrolled is never re-enrolled, because enrollment rotates the device key. Standard mode ships locally sanitized retained content; metadata-only uses local Vector transforms to remove prompts, responses, reasoning, tool arguments/results, command output, raw fields, diffs, inventory content, and MCP definitions before buffering/upload. Keep the ingest URL `https://`, the wire contract aligned with `beacon-ingest` (`/v1/ingest/{runtime,inventory}` and `/v1/ingest/health`), and do not add another endpoint network channel. The cloud-sandbox shuttle is the bounded exception: only with `BEACON_ORIGIN=cloud`, it can upload new runtime lines to GCS, S3, or the same Beacon ingest contract because no Vector/service manager/browser/persistent disk exists there.+- Beacon Cloud forwarding (`cli/beacon/internal/endpoint/asymptote`, `beacon endpoint connect`, `beacon endpoint asymptote`) is Vector-shaped: Beacon writes local JSONL, Vector ships runtime lines only from the connection point forward and reads inventory from the beginning to establish the endpoint's current baseline, plus the one bounded exception of the session backfill (below), and the `bcn_device_*` key stays only in `<BaseDir>/asymptote/vector-secrets.json` via Vector's secret backend. The onboarding wizard preselects Beacon Cloud, and confirming it connects this endpoint in the same command once the local install has succeeded; an endpoint that is already enrolled is never re-enrolled, because enrollment rotates the device key. Standard mode ships locally sanitized retained content; metadata-only uses local Vector transforms to remove prompts, responses, reasoning, tool arguments/results, tool descriptions/definitions, retrieval queries/documents, evaluation explanations, agent descriptions, stop sequences, approval/policy reasons, command output, raw fields, diffs, inventory content, and MCP definitions before buffering/upload. Every text-shaped event field must be classified in `metadata_fields_test.go` (`metadataOnlyRemoved`, each with its own `del()` in `pack/metadata-transforms.toml`, or `metadataOnlyKept`); a new field fails that test until it is, and `BEACON_TEST_VECTOR_BIN` runs the transform on an event with every field set. Keep the ingest URL `https://`, the wire contract aligned with `beacon-ingest` (`/v1/ingest/{runtime,inventory}` and `/v1/ingest/health`), and do not add another endpoint network channel. The cloud-sandbox shuttle is the bounded exception: only with `BEACON_ORIGIN=cloud`, it can upload new runtime lines to GCS, S3, or the same Beacon ingest contract because no Vector/service manager/browser/persistent disk exists there.- Session backfill (`cli/beacon/cmd/endpoint_backfill.go`; docs in `docs/cli/endpoint-install.mdx#session-backfill` and `docs/cli/endpoint-connect.mdx#session-backfill`): after every successful install (user or system mode), every session-store collector runs once with `CollectOptions.Since` (30 days; `sessionwindow.Recent` keeps those sessions, newest first, and never touches the cursors of the rest) and a shared `writer.Budget` (24 MiB, at most 12 MiB per runtime through `Budget.Sub`; a refused append returns `writer.ErrBudgetSpent`, the collector stops with cursors past only what was written, and a manual sync reads the rest). A new session-store collector takes both fields and joins `endpointBackfillRunners`. Every connect, first or re-connect, user or system, runs the same sweep before enrollment (skipped when install just ran it) and, before the forwarder starts, `asymptote.StageBackfill` copies the window's `harness.collection_method=poll` lines into `cloud-backfill.jsonl` beside the runtime log, which the `beacon_backfill` Vector source (fingerprinted `device_and_inode`, so each atomically restaged copy is a new file) reads from the beginning into the runtime sink through the same privacy transform. Live hook/OTLP/plugin history before the connection point stays local. Re-connects restage, so a resend of already-shipped lines is expected and rests on the deterministic `event.id`; a privacy-mode change clears the checkpoints and the restage renames the new-mode copy over the old one (kept until then so the copy is always a new inode; deleted instead when nothing restages), and `disconnect` removes it. System mode runs the collectors under `withUserHome` for the active console user (`activeConsoleUser`), with cursors under the system base directory, and reports a skip when nobody is logged in. The wizard's confirm screen names the upload (`WizardOptions.BackfillDays`). Dry-run, `--no-backfill` and `BEACON_BACKFILL=0` skip all of it; the cmd tests run with `BEACON_BACKFILL=0` set in `TestMain`.- `beacon endpoint install` has a full-screen onboarding wizard on interactive user TTYs. It signs in, preselects Beacon Cloud, requires an explicit choice to opt out to Local, and asks Managed users to choose Standard or Metadata-only privacy. Signing in alone still moves no telemetry. Confirming Beacon Cloud does: the confirm screen states that it installs Beacon and starts forwarding, and names what the selected privacy mode sends, so that screen is the consent point. The managed destination is recorded only after enrollment succeeds, so a failed enrollment leaves a working local install, prints `beacon endpoint connect` as the retry, and re-offers the question on the next install. The onboarding record is written only after `lifecycle.Install` returns, because a profile written first left a failed install looking onboarded and silently skipped the wizard on the retry. Explicit `install --connect` is unchanged. The wizard is skipped for system/root/CI/piped-stdin/dry-run or `BEACON_ONBOARDING=0`; those unattended paths must never gain an account or browser dependency. Legacy `BEACON_ONBOARDING_EMAIL`/`USAGE` submissions remain only for fleet compatibility. After the destination, the wizard shows an agent tools screen; every row is preselected (opt-out, space toggles). Beacon Cloud MCP is offered only with Beacon Cloud and runs the `beacon mcp connect` flow (`connectCloudMCP` with `yes`, OAuth, detected harnesses) only after the endpoint is connected (or already was), via `finishCloudSetup`; a failed enrollment skips it and names the retry. The Beacon Agent Skills are offered for either destination and installed user-wide into `~/.agents/skills` and, when Claude Code is set up, `~/.claude/skills` (`cli/beacon/internal/agentskills`, `beacon skills install|uninstall`; a `.beacon-managed` marker decides what Beacon may replace or remove; install and repair refresh only copies Beacon already wrote). Both run after `lifecycle.Install` returns and warn rather than fail. The answers are recorded as `onboarding.cloud_mcp`/`agent_skills`. Implementation lives in `cli/beacon/internal/onboarding`, `cli/beacon/cmd/endpoint_onboarding.go`, `cli/beacon/cmd/endpoint_install.go` and `cli/beacon/cmd/agent_tools.go`; keep `docs/cli/endpoint-onboarding.mdx` and `docs/cli/skills.mdx` in sync.- The update notice on bare interactive `beacon` (`cli/beacon/cmd/root.go`, cache in `cli/beacon/internal/updatecheck/cache.go`; docs in `docs/cli/version.mdx`) follows the Codex/Claude Code pattern: it reads `~/.beacon/endpoint/update-check.json` and never waits on the network, and only when that cache is a day old does it start a detached `beacon version refresh` (hidden) that asks GitHub for the latest release and rewrites the cache, failures included. It is the CLI's one background network call; it never runs from hooks, other commands, non-TTY output, CI or dev builds, and `BEACON_UPDATE_CHECK=0` turns it off. Installs other than Homebrew and the system package get the release download, because `beacon endpoint update` applies only to the system package.diff --git a/cli/beacon/internal/endpoint/asymptote/metadata_fields_test.go b/cli/beacon/internal/endpoint/asymptote/metadata_fields_test.gonew file mode 100644index 00000000..e7709d44--- /dev/null+++ b/cli/beacon/internal/endpoint/asymptote/metadata_fields_test.go@@ -0,0 +1,338 @@+package asymptote++import (+ "encoding/json"+ "io/fs"+ "os"+ "os/exec"+ "path/filepath"+ "reflect"+ "sort"+ "strings"+ "testing"++ "github.com/pelletier/go-toml/v2"++ "github.com/asymptote-labs/agent-beacon/cli/beacon/internal/endpoint/schema"+)++// metadataOnlyRemoved is every event field Metadata-only forwarding removes: free text, documents,+// arguments, schemas and messages that a person, a model or a server wrote. Each one must have its+// own del(event.<path>) in the beacon_runtime_metadata transform. A path removes everything under it.+var metadataOnlyRemoved = []string{+ "prompt.text",+ "message",+ "raw",+ "command.output",+ "file.diff",+ "gen_ai.input",+ "gen_ai.output",+ "gen_ai.system_instructions",+ "gen_ai.tool.description",+ "gen_ai.tool.definitions",+ "gen_ai.tool.call.arguments",+ "gen_ai.tool.call.result",+ "gen_ai.retrieval",+ "gen_ai.evaluation.explanation",+ "gen_ai.agent.description",+ "gen_ai.request.stop_sequences",+ "approval.reason",+ "policy.reason",+}++// metadataOnlyKept is every text-shaped event field (string, list, map or untyped value) that+// Metadata-only forwarding keeps: identifiers, names, enums, versions, paths, the command line and+// other structured metadata the docs promise. Numbers and booleans are metadata by nature and are not+// listed. A new text-shaped field must be added here or to metadataOnlyRemoved, so that whoever adds+// it decides whether it is content.+var metadataOnlyKept = []string{+ "timestamp", "vendor", "product", "schema_version", "severity", "origin",+ "event.id", "event.kind", "event.action", "event.category", "event.fidelity",+ "endpoint.hostname", "endpoint.os", "endpoint.agent_version",+ "user.name", "user.uid",+ "harness.name", "harness.version", "harness.executable_path", "harness.config_path", "harness.collection_method",+ "error.type",+ "run.provider", "run.run_id", "run.run_attempt", "run.workflow", "run.job", "run.event_name",+ "run.commit", "run.repository", "run.branch", "run.pr", "run.pr_number", "run.actor",+ "session.id", "session.working_directory",+ "handoff.source_harness", "handoff.source_session_id",+ "share.visibility",+ "trace.id", "trace.span_id", "trace.parent_span_id",+ "jsonrpc.request.id", "jsonrpc.protocol.version",+ "network.protocol.name", "network.protocol.version", "network.transport",+ "rpc.response.status_code",+ "server.address",+ // The command line is kept: the docs list commands among what Metadata-only keeps.+ "tool.name", "tool.command", "tool.path",+ "file.path", "file.operation", "file.language", "file.diff_hash",+ "command.command",+ "mcp.server", "mcp.tool", "mcp.method.name", "mcp.protocol.version", "mcp.resource.uri", "mcp.session.id",+ "approval.decision",+ "policy.id", "policy.name", "policy.decision", "policy.enforcement",+ "prompt.id",+ "system_context.kind", "system_context.source",+ "system_context.skills.skill_name", "system_context.skills.skill_name_hash",+ "content.retention", "content.hash",+ "destination.type", "destination.mode", "destination.status",+ // Beacon's own component health; the reason is Beacon-authored, not agent content.+ "health.component", "health.status", "health.reason",+ "user_agent.name", "user_agent.version",+ "vcs.ref.head.revision", "vcs.ref.head.name", "vcs.ref.head.type",+ "vcs.repository.url.full", "vcs.attribution.method",+ "gen_ai.agent.id", "gen_ai.agent.name", "gen_ai.agent.version",+ "gen_ai.conversation.id",+ "gen_ai.data_source.id",+ "gen_ai.evaluation.name", "gen_ai.evaluation.score.label",+ "gen_ai.operation.name",+ "gen_ai.prompt.name",+ "gen_ai.provider.name",+ "gen_ai.request.encoding_formats", "gen_ai.request.model",+ "gen_ai.response.finish_reasons", "gen_ai.response.id", "gen_ai.response.model",+ "gen_ai.token.type",+ "gen_ai.tool.call.id", "gen_ai.tool.name", "gen_ai.tool.type",+ "gen_ai.workflow.name",+ "model", "repository", "branch",+}++// eventLeaf is one JSON field of the event schema. Text is true for the shapes that can carry+// content (string, list, map, untyped value); numbers and booleans are not.+type eventLeaf struct {+ Path string+ Kind reflect.Kind+ Text bool+}++func eventLeaves(t reflect.Type, prefix string) []eventLeaf {+ var leaves []eventLeaf+ for i := 0; i < t.NumField(); i++ {+ field := t.Field(i)+ if !field.IsExported() {+ continue+ }+ name, _, _ := strings.Cut(field.Tag.Get("json"), ",")+ if name == "-" {+ continue+ }+ if name == "" {+ name = field.Name+ }+ path := name+ if prefix != "" {+ path = prefix + "." + name+ }+ ft := field.Type+ for ft.Kind() == reflect.Pointer {+ ft = ft.Elem()+ }+ switch {+ case ft.Kind() == reflect.Struct:+ leaves = append(leaves, eventLeaves(ft, path)...)+ case ft.Kind() == reflect.Slice && ft.Elem().Kind() == reflect.Struct:+ leaves = append(leaves, eventLeaves(ft.Elem(), path)...)+ default:+ switch ft.Kind() {+ case reflect.String, reflect.Interface, reflect.Slice, reflect.Map:+ leaves = append(leaves, eventLeaf{Path: path, Kind: ft.Kind(), Text: true})+ default:+ leaves = append(leaves, eventLeaf{Path: path, Kind: ft.Kind()})+ }+ }+ }+ return leaves+}++func removedByMetadataOnly(path string) bool {+ for _, removed := range metadataOnlyRemoved {+ if path == removed || strings.HasPrefix(path, removed+".") {+ return true+ }+ }+ return false+}++func runtimeMetadataTransformSource(t *testing.T) string {+ t.Helper()+ var transforms struct {+ Transforms map[string]struct {+ Source string `toml:"source"`+ } `toml:"transforms"`+ }+ data, err := fs.ReadFile(packFS, privacyTransformAsset)+ if err != nil {+ t.Fatal(err)+ }+ if err := toml.Unmarshal(data, &transforms); err != nil {+ t.Fatal(err)+ }+ source := transforms.Transforms["beacon_runtime_metadata"].Source+ if source == "" {+ t.Fatal("beacon_runtime_metadata transform has no source")+ }+ return source+}++// Every content field has its own del() in the Metadata-only transform.+func TestMetadataTransformDeletesEveryContentField(t *testing.T) {+ source := runtimeMetadataTransformSource(t)+ for _, path := range metadataOnlyRemoved {+ if !strings.Contains(source, "del(event."+path+")\n") {+ t.Errorf("beacon_runtime_metadata does not remove %s: add del(event.%s)", path, path)+ }+ }+}++// Every text-shaped field of the event schema is classified as content (removed) or metadata+// (kept). A new field fails here until someone decides which it is.+func TestMetadataOnlyClassifiesEveryEventField(t *testing.T) {+ leaves := eventLeaves(reflect.TypeOf(schema.Event{}), "")+ known := map[string]bool{}+ for _, leaf := range leaves {+ known[leaf.Path] = true+ if !leaf.Text {+ continue+ }+ removed, kept := removedByMetadataOnly(leaf.Path), contains(metadataOnlyKept, leaf.Path)+ switch {+ case removed && kept:+ t.Errorf("%s is listed as both removed and kept by Metadata-only forwarding", leaf.Path)+ case !removed && !kept:+ t.Errorf("event field %s (%s) is not classified: add it to metadataOnlyRemoved if it can carry content a person, model or server wrote, or to metadataOnlyKept if it is metadata", leaf.Path, leaf.Kind)+ }+ }+ // Both lists name only fields that exist, so a renamed field cannot leave a stale entry behind.+ exists := func(path string) bool {+ for p := range known {+ if p == path || strings.HasPrefix(p, path+".") {+ return true+ }+ }+ return false+ }+ for _, path := range append(append([]string{}, metadataOnlyRemoved...), metadataOnlyKept...) {+ if !exists(path) {+ t.Errorf("%s is classified but is not an event field", path)+ }+ }+}++func contains(list []string, value string) bool {+ for _, v := range list {+ if v == value {+ return true+ }+ }+ return false+}++// sentinelEvent builds one event with every field of the schema set, each text value naming its own+// path, so the output of the transform shows exactly which fields survived.+func sentinelEvent(t reflect.Type, prefix string) map[string]interface{} {+ out := map[string]interface{}{}+ for i := 0; i < t.NumField(); i++ {+ field := t.Field(i)+ if !field.IsExported() {+ continue+ }+ name, _, _ := strings.Cut(field.Tag.Get("json"), ",")+ if name == "-" {+ continue+ }+ path := name+ if prefix != "" {+ path = prefix + "." + name+ }+ ft := field.Type+ for ft.Kind() == reflect.Pointer {+ ft = ft.Elem()+ }+ sentinel := metadataSentinel(path)+ switch ft.Kind() {+ case reflect.Struct:+ out[name] = sentinelEvent(ft, path)+ case reflect.Slice:+ if ft.Elem().Kind() == reflect.Struct {+ out[name] = []interface{}{sentinelEvent(ft.Elem(), path)}+ } else {+ out[name] = []interface{}{sentinel}+ }+ case reflect.Map, reflect.Interface:+ out[name] = map[string]interface{}{"text": sentinel}+ case reflect.String:+ out[name] = sentinel+ case reflect.Bool:+ out[name] = true+ default:+ out[name] = 1+ }+ }+ return out+}++func metadataSentinel(path string) string { return "SENTINEL:" + path + ":END" }++// Runs the Metadata-only transform with Vector's own VRL runtime on an event with every field set,+// and checks that every content field is gone and every metadata field is still there.+func TestMetadataTransformRemovesEveryContentFieldWithVector(t *testing.T) {+ vector := os.Getenv("BEACON_TEST_VECTOR_BIN")+ if vector == "" {+ t.Skip("set BEACON_TEST_VECTOR_BIN to run the metadata transform with Vector")+ }+ dir := t.TempDir()+ program := filepath.Join(dir, "runtime.vrl")+ if err := os.WriteFile(program, []byte(runtimeMetadataTransformSource(t)), 0o600); err != nil {+ t.Fatal(err)+ }+ event, err := json.Marshal(sentinelEvent(reflect.TypeOf(schema.Event{}), ""))+ if err != nil {+ t.Fatal(err)+ }+ wrapped, _ := json.Marshal(map[string]string{"message": string(event)})+ inputPath := filepath.Join(dir, "input.jsonl")+ if err := os.WriteFile(inputPath, append(wrapped, '\n'), 0o600); err != nil {+ t.Fatal(err)+ }+ out, err := exec.Command(vector, "vrl", "--input", inputPath, "--program", program, "--print-object").CombinedOutput()+ if err != nil {+ t.Fatalf("vector vrl: %v\n%s", err, out)+ }+ var transformed string+ for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {+ var result struct {+ Message string `json:"message"`+ }+ if json.Unmarshal([]byte(line), &result) == nil && result.Message != "" {+ transformed = result.Message+ }+ }+ if transformed == "" {+ t.Fatalf("vector vrl produced no event:\n%s", out)+ }+ var parsed schema.Event+ if err := json.Unmarshal([]byte(transformed), &parsed); err != nil {+ t.Fatalf("transformed event is not an event: %v\n%s", err, transformed)+ }+ var missing, leaked []string+ for _, leaf := range eventLeaves(reflect.TypeOf(schema.Event{}), "") {+ if !leaf.Text || leaf.Path == "content.retention" {+ continue // the transform rewrites retention to "metadata"+ }+ present := strings.Contains(transformed, metadataSentinel(leaf.Path))+ if removedByMetadataOnly(leaf.Path) && present {+ leaked = append(leaked, leaf.Path)+ }+ if !removedByMetadataOnly(leaf.Path) && !present {+ missing = append(missing, leaf.Path)+ }+ }+ sort.Strings(leaked)+ sort.Strings(missing)+ if len(leaked) > 0 {+ t.Errorf("Metadata-only forwarded content fields %v:\n%s", leaked, transformed)+ }+ if len(missing) > 0 {+ t.Errorf("Metadata-only removed metadata fields %v:\n%s", missing, transformed)+ }+ if parsed.Content == nil || parsed.Content.Retention != schema.ContentRetentionMetadata || parsed.Content.Included {+ t.Errorf("content marker = %+v, want metadata retention", parsed.Content)+ }+}diff --git a/cli/beacon/internal/endpoint/asymptote/pack/README.md b/cli/beacon/internal/endpoint/asymptote/pack/README.mdindex 2e5c8756..ec9c2289 100644--- a/cli/beacon/internal/endpoint/asymptote/pack/README.md+++ b/cli/beacon/internal/endpoint/asymptote/pack/README.md@@ -24,8 +24,9 @@ history.This hand-run pack uses Standard privacy. `beacon endpoint connect--privacy-mode metadata-only` renders an endpoint-specific config that removes-retained text, raw fields, diffs, inventory content, and MCP definitions before-buffering or upload. It keeps metadata: on a `session.context` skill index, the+retained text, raw fields, diffs, tool definitions, retrieval documents, approval+and policy reasons, inventory content, and MCP definitions before buffering or+upload. It keeps metadata: on a `session.context` skill index, the`system_context` block that says what the event is and names each listed skill,while the index text in `gen_ai.system_instructions` is removed.diff --git a/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml b/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.tomlindex 6c9bf7c3..08b4f1e6 100644--- a/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml+++ b/cli/beacon/internal/endpoint/asymptote/pack/metadata-transforms.toml@@ -21,9 +21,46 @@ del(event.gen_ai.system_instructions)del(event.gen_ai.tool.description)del(event.gen_ai.tool.call.arguments)del(event.gen_ai.tool.call.result)+# Tool definitions are the full schemas and descriptions of every tool offered to the model, the+# same material as an MCP server's definition, which inventory removes.+del(event.gen_ai.tool.definitions)+# Retrieval carries the query the model ran and the documents it got back.+del(event.gen_ai.retrieval)+# An evaluator's explanation and an agent's description are free text; the evaluation's name and+# score and the agent's id, name and version stay.+del(event.gen_ai.evaluation.explanation)+del(event.gen_ai.agent.description)+# Stop sequences are strings the caller chose, which can quote the conversation.+del(event.gen_ai.request.stop_sequences)+if is_object(event.gen_ai.tool.call) && is_empty(object!(event.gen_ai.tool.call)) {+ del(event.gen_ai.tool.call)+}+if is_object(event.gen_ai.tool) && is_empty(object!(event.gen_ai.tool)) {+ del(event.gen_ai.tool)+}+if is_object(event.gen_ai.evaluation) && is_empty(object!(event.gen_ai.evaluation)) {+ del(event.gen_ai.evaluation)+}+if is_object(event.gen_ai.agent) && is_empty(object!(event.gen_ai.agent)) {+ del(event.gen_ai.agent)+}+if is_object(event.gen_ai.request) && is_empty(object!(event.gen_ai.request)) {+ del(event.gen_ai.request)+}if is_object(event.gen_ai) && is_empty(object!(event.gen_ai)) {del(event.gen_ai)}+# An approval's or a policy's reason is free text: an operator's typed feedback (Kimi Code, Pi), the+# runtime's description of the request (Hermes Agent), or a policy provider's explanation, which can+# quote the command. The decision, enforcement and policy identity stay.+del(event.approval.reason)+if is_object(event.approval) && is_empty(object!(event.approval)) {+ del(event.approval)+}+del(event.policy.reason)+if is_object(event.policy) && is_empty(object!(event.policy)) {+ del(event.policy)+}# system_context says what a session.context event recorded and names the skills a skill index# listed, the way inventory names skills on disk; its text was in gen_ai.system_instructions.if is_object(event.content) {diff --git a/docs/cli/endpoint-connect.mdx b/docs/cli/endpoint-connect.mdxindex eef958fa..813329df 100644--- a/docs/cli/endpoint-connect.mdx+++ b/docs/cli/endpoint-connect.mdx@@ -77,7 +77,7 @@ The onboarding wizard records one of two modes. `connect` uses that choice unles| Mode | Forwarded data ||------|----------------|| `standard` | The complete locally sanitized event, including retained prompts, responses, reasoning, tool arguments/results, command output, and diffs. Existing local redaction and size limits still apply. |-| `metadata-only` | Omits prompts, responses, reasoning, tool arguments/results, command output, raw event fields, diffs, inventory file contents, and MCP definitions. Stable ids, hashes, byte counts, actions, timestamps, harness/session/trace identity, the `prompt.id` that groups a turn, token usage, commands, file paths, and other structured metadata remain. |+| `metadata-only` | Omits prompts, responses, reasoning, tool arguments/results, command output, raw event fields, diffs, inventory file contents, and MCP definitions. It also removes tool descriptions and definitions (`gen_ai.tool.description`, `gen_ai.tool.definitions`), retrieval queries and documents (`gen_ai.retrieval`), evaluation explanations, agent descriptions, request stop sequences, and the free-text reason on approvals and policy decisions (`approval.reason`, `policy.reason`), which can hold an operator's typed feedback; the decision itself remains. Stable ids, hashes, byte counts, actions, timestamps, harness/session/trace identity, the `prompt.id` that groups a turn, token usage, commands, file paths, and other structured metadata remain. |The transform runs locally in Vector before batching, buffering, or HTTPS upload.Changing modes requires rerunning `beacon endpoint connect`; the rendered config anddiff --git a/docs/deployment/managed.mdx b/docs/deployment/managed.mdxindex b2353a30..992e824c 100644--- a/docs/deployment/managed.mdx+++ b/docs/deployment/managed.mdx@@ -58,8 +58,9 @@ Beacon records locally first. Only a connected device forwards, and you choose h- **Standard** sends retained content, such as prompts, responses, tool arguments and results,after local redaction and size limits.-- **Metadata-only** strips prompts, responses, reasoning, tool arguments and results, command- output, diffs and inventory content on the device, before anything is uploaded.+- **Metadata-only** strips prompts, responses, reasoning, tool arguments and results, tool+ definitions, retrieval documents, approval and policy reasons, command output, diffs and+ inventory content on the device, before anything is uploaded.See [Beacon Cloud forwarding](/log-forwarding/asymptote#what-leaves-the-machine) for the full dataflow.diff --git a/docs/log-forwarding/asymptote.mdx b/docs/log-forwarding/asymptote.mdxindex a2fbc9cb..424b1cfc 100644--- a/docs/log-forwarding/asymptote.mdx+++ b/docs/log-forwarding/asymptote.mdx@@ -22,7 +22,8 @@ Inventory is read from the beginning of `inventory_state.jsonl` so anewly connected endpoint sends its current inventory baseline, including inventoryrecords written before connection. Metadata-only mode applies a local Vectortransform first, removing retained prompts, responses, reasoning, tool-arguments/results, command output, raw fields, diffs, inventory content, and MCP+arguments/results, tool descriptions and definitions, retrieval queries and+documents, approval and policy reasons, command output, raw fields, diffs, inventory content, and MCPdefinitions while preserving stable ids, hashes, counts, actions, paths, tokenusage, and other structured metadata. Beacon's [retention andredaction](/security/retention-redaction) controls still run before either mode.diff --git a/docs/security/data-flow-threat-model.mdx b/docs/security/data-flow-threat-model.mdxindex a42cfb6b..5a8e247c 100644--- a/docs/security/data-flow-threat-model.mdx+++ b/docs/security/data-flow-threat-model.mdx@@ -36,7 +36,7 @@ flowchart LR|------|-----------------|| Unintended hosted telemetry | Account sign-in forwards nothing. Beacon Cloud is preselected, and confirming it starts forwarding within the same command, which the confirm screen states before the choice is accepted, naming what the selected privacy mode sends. Choosing Local forwards nothing, and `beacon endpoint disconnect` stops a connected endpoint. Runtime forwarding starts at the end of the existing log; inventory starts at the beginning so the service receives the endpoint's current inventory baseline. The active privacy mode is recorded, and forwarding stops within a minute of revocation || Network exposure of collectors | Default OTLP receivers bind to `127.0.0.1` rather than an externally reachable interface |-| Over-collection of prompt or diff content | Standard applies local sanitization and size controls. Metadata-only additionally strips retained prompts, responses, reasoning, tool arguments/results, command output, raw fields, diffs, inventory content, and MCP definitions in Vector before buffering or upload |+| Over-collection of prompt or diff content | Standard applies local sanitization and size controls. Metadata-only additionally strips retained prompts, responses, reasoning, tool arguments/results, tool descriptions and definitions, retrieval queries and documents, approval and policy reasons, command output, raw fields, diffs, inventory content, and MCP definitions in Vector before buffering or upload || Secret leakage in retained content | Beacon applies redaction, sanitization, truncation, and event-size limits before writing supported content fields || Destination credential exposure | Persistent endpoint file-based destinations tail local JSONL, so Beacon endpoint configuration does not store Elastic, Datadog, Sumo Logic, Rapid7, Microsoft Sentinel, AWS, or Google Cloud credentials. Optional Splunk and Falcon tokens live in collector configuration. The CLI account token authorizes user-mode enrollment once and is never copied into endpoint state or Vector. The resulting per-machine device key is stored in a `0600` secrets file read through Vector's secret backend, never in `config.json`, `vector.toml`, or the process environment; it grants only per-device ingest and is revocable. Self-serve cloud-agent uploads are a separate proof-of-concept path: scoped GCS or AWS credentials are injected into the provider-managed sandbox as runtime secrets and must be rotated and access-reviewed. || Removal uncertainty | Endpoint uninstall removes managed service and configuration state, with explicit `--keep-logs` and `--keep-config` exceptions |diff --git a/docs/security/retention-redaction.mdx b/docs/security/retention-redaction.mdxindex 0bac8b85..18f1bc80 100644--- a/docs/security/retention-redaction.mdx+++ b/docs/security/retention-redaction.mdx@@ -92,7 +92,8 @@ customer-managed GCS or S3. Splunk HEC and Falcon LogScale HEC forwardingreceive the same normalized collector output according to the configuredendpoint pipeline. Beacon Cloud Standard mode forwards new runtime andinventory lines as stored locally. Metadata-only mode adds a local Vector-transform that removes retained text, tool arguments/results, command output,+transform that removes retained text, tool arguments/results, tool definitions,+retrieval queries and documents, approval and policy reasons, command output,raw fields, diffs, inventory content, and MCP definitions before buffering orupload while preserving stable metadata, hashes, counts, and token usage.