The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
Merge origin/main; keep both Implementation Notes bullets and regenerate FIELDS.md
Claudeed1c63e2eca6
diff --cc CLAUDE.mdindex 7236107c,342d3d9c..d77df15c--- a/CLAUDE.md+++ b/CLAUDE.md@@@ -353,7 -365,7 +365,8 @@@ Use these npm trusted publisher values- For macOS-only behavior, gate tests with `runtime.GOOS == "darwin"` or assert the non-Darwin contract explicitly.- Keep endpoint event schema fields stable: `vendor`, `product`, `schema_version`, required event fields, and Wazuh-compatible JSONL output are release contracts.- Preserve optional event fields for agent-native metadata (`session`, `trace`, `tool`, `file`, `command`, `mcp`, `approval`, `content`, `model`, `repository`, and `branch`) without changing existing required field semantics.+ - Set the provenance markers on every new collection path. `harness.collection_method` (`hook`, `otlp`, `plugin`, `poll`) records the mechanism; `event.fidelity` (`observed`, `inferred`) records whether the source named the action or Beacon derived it. Both are defined in `pkg/asymptoteobserve/provenance.go` and are the only sanctioned way to express that difference — do not add per-runtime confidence fields, and do not leave a derived action unmarked. A new plugin-shaped runtime must be added to `CollectionMethodForPlatform`, whose default is `hook`. An action synthesized from an adjacent observation, such as an approval built from a pre-tool notification on a runtime with no approval hook, is `inferred`.+- Keep event identity derivable rather than asserted. `gen_ai.tool.call.id` carries the runtime's own name for one tool invocation, promoted on every capture path from the shared alias list in `pkg/asymptoteobserve.ToolCallIDKeys`; a new runtime adds its spelling there rather than in a mapper. `event.id` is a deterministic UUID derived by `EventIDForLine` from the event about to be written, so both capture paths name one action the same way. Its namespace constant and derivation are a contract: changing either renames every event ID Beacon has written, and a test pins the value.- Keep `gen_ai.usage` as the single canonical token-usage representation: normalize all runtime token telemetry (span attributes, metric datapoints, legacy `llm.usage.*` aliases) into it, mirror OTel GenAI semconv JSON names exactly, and never add parallel or per-harness token fields. `gen_ai.usage.cost_usd` carries runtime-reported cost only; do not derive cost from local pricing tables.- When adding a new signal, include stable identifiers/counts/hashes alongside any retained raw content, and route raw fields through redaction, sanitization, truncation, and event-size controls.- Keep the dashboard read-only. It should inspect local status and JSONL events but must not mutate endpoint configuration or telemetry.diff --cc docs/telemetry-schema/event-schema.mdxindex 1e4a9460,28a0903e..23cb2e62--- a/docs/telemetry-schema/event-schema.mdx+++ b/docs/telemetry-schema/event-schema.mdx@@@ -36,11 -84,8 +84,12 @@@ call was seen. Anything counting real o| `severity` | `info`, `low`, `medium`, `high`, or `critical` || `endpoint` | Host and operating system context || `harness` | Runtime that produced the signal |+ | `harness.collection_method` | Optional. `hook`, `plugin`, `otlp`, or `poll` — see [Provenance](#provenance-how-beacon-knows-what-it-wrote) |+`event.id` is written alongside these: a deterministic UUID identifying the event, described under+[event identity](/telemetry-schema/normalization#event-identity). It is not required, so events+written by earlier agent versions remain valid.+## How it extends OpenTelemetryBeacon accepts OpenTelemetry attributes, including the GenAI semantic conventions, and projects them into fields that are easier to query across agent runtimes. OpenTelemetry context is preserved under `gen_ai` where available, while commonly investigated values are also normalized into Beacon fields such as `model`, `tool.name`, `prompt.text`, `command.command`, `file.path`, and `gen_ai.usage`.diff --cc pkg/asymptoteobserve/identity_test.goindex 3ae30070,00000000..f1757041mode 100644,000000..100644--- a/pkg/asymptoteobserve/identity_test.go+++ b/pkg/asymptoteobserve/identity_test.go@@@ -1,97 -1,0 +1,114 @@@+package asymptoteobserve++import (+ "regexp"+ "testing"+)++// callLine is one endpoint event the runtime named itself, with the parts the+// call identity is derived from left as parameters and everything else varying+// freely, so a test can say exactly which difference it is testing.+func callLine(harness, timestamp, callID, command string) []byte {+ return []byte(`{"timestamp":"` + timestamp + `","event":{"action":"command.executed"},` ++ `"harness":{"name":"` + harness + `"},"session":{"id":"s1","working_directory":"/repo"},` ++ `"command":{"command":"` + command + `"},"gen_ai":{"tool":{"call":{"id":"` + callID + `"}}},` ++ `"message":"Shell command executed"}`)+}++var uuidV5Pattern = regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-5[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$`)++func TestEventIDForLineIsAVersion5UUID(t *testing.T) {+ for name, line := range map[string][]byte{+ "call identity": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"),+ "content identity": []byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"session.activity"},"session":{"id":"s1"}}`),+ "unparseable": []byte(`not json at all`),+ } {+ if id := EventIDForLine(line); !uuidV5Pattern.MatchString(id) {+ t.Errorf("%s: EventIDForLine = %q, want an RFC 4122 version 5 UUID", name, id)+ }+ }+}++// The point of the call identity: the hook writes when the tool runs and the+// collector writes when its batch flushes, so the two reports of one call agree+// on almost nothing except the call ID. They must still name one event.+func TestEventIDForLineIgnoresWhatTheTwoCapturePathsDisagreeOn(t *testing.T) {+ hook := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))+ otlp := EventIDForLine(callLine("otel", "2026-08-21T18:00:07Z", "toolu_1", "echo hi"))+ if hook != otlp {+ t.Fatalf("one call reported twice derived two IDs: %s and %s", hook, otlp)+ }+}++func TestEventIDForLineSeparatesDistinctCalls(t *testing.T) {+ first := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))+ for name, other := range map[string][]byte{+ "another call": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_2", "echo hi"),+ "another target": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "rm -rf /"),+ } {+ if id := EventIDForLine(other); id == first {+ t.Errorf("%s derived the same ID as the original call: %s", name, id)+ }+ }+}++// Without a call ID there is nothing to identify the event by but the event, so+// the ID has to move when any of it does -- and stay put when none of it does,+// which is what makes re-reading a log derive the IDs it already had.+func TestEventIDForLineFallsBackToTheEventItself(t *testing.T) {+ line := []byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"session.activity"},"session":{"id":"s1"},"message":"Hook executed"}`)+ later := []byte(`{"timestamp":"2026-08-21T18:00:02Z","event":{"action":"session.activity"},"session":{"id":"s1"},"message":"Hook executed"}`)+ if EventIDForLine(line) != EventIDForLine(line) {+ t.Fatal("the same line derived two IDs")+ }+ if EventIDForLine(line) == EventIDForLine(later) {+ t.Fatal("events a second apart derived one ID")+ }+}++// A content digest and a call identity must not be able to collide, because+// they mean different things: see EventIDForLine.+func TestEventIDForLineKeepsTheTwoIdentityModesApart(t *testing.T) {+ if eventUUID("call", "x") == eventUUID("content", "x") {+ t.Fatal("the call and content identity spaces overlap")+ }+}++// The namespace and the derivation are a written contract: an event's ID has to+// survive a release, or nothing downstream can address it across two of them.+func TestEventIDForLineIsStableAcrossReleases(t *testing.T) {+ const want = "244a6772-0cbe-5230-9f79-37cb62ce25ee"+ got := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))+ if got != want {+ t.Fatalf("EventIDForLine = %s, want %s -- if this is deliberate, every ID Beacon has "++ "ever written just changed meaning", got, want)+ }+}++// The same disagreement, at the identity layer: the hook's and the collector's+// reports of one Write must derive one event ID even though they name the+// operation differently.+func TestEventIDForLineIgnoresHowTheOperationIsNamed(t *testing.T) {+ hook := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"file.modified"},"harness":{"name":"claude_code"},"session":{"id":"s1","working_directory":"/repo"},"file":{"path":"/repo/notes.md","operation":"modify"},"gen_ai":{"tool":{"call":{"id":"toolu_w"}}}}`))+ collector := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:07Z","event":{"action":"file.modified"},"harness":{"name":"claude_code"},"session":{"id":"s1","working_directory":"/repo"},"file":{"path":"/repo/notes.md","operation":"create"},"gen_ai":{"tool":{"call":{"id":"toolu_w"}}}}`))+ if hook != collector {+ t.Fatalf("one Write derived two IDs: %s and %s", hook, collector)+ }+}++++// Provenance markers must not split the identity of one action.++//++// harness.collection_method and event.fidelity exist precisely because the two++// capture paths differ, so they differ on the two reports of one call by design:++// the hook says hook/observed, the collector says otlp and may say inferred for++// an action it classified by substring. If either reached the call identity, one++// action would derive two IDs again -- which is the failure this whole change++// exists to remove, arriving by way of an unrelated and perfectly correct++// addition. Exactly how the August dedup regression happened.++func TestEventIDForLineIgnoresProvenanceMarkers(t *testing.T) {++ hook := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"command.executed","fidelity":"observed"},"harness":{"name":"claude_code","collection_method":"hook"},"session":{"id":"s1","working_directory":"/repo"},"command":{"command":"echo hi"},"gen_ai":{"tool":{"call":{"id":"toolu_1"}}}}`))++ collector := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:07Z","event":{"action":"command.executed","fidelity":"inferred"},"harness":{"name":"claude_code","collection_method":"otlp"},"session":{"id":"s1","working_directory":"/repo"},"command":{"command":"echo hi"},"gen_ai":{"tool":{"call":{"id":"toolu_1"}}}}`))++ if hook != collector {++ t.Fatalf("provenance markers split one call into two IDs: %s and %s", hook, collector)++ }++}diff --cc spec/threat-rules/FIELDS.mdindex df7157b7,93dd4932..ceb53049--- a/spec/threat-rules/FIELDS.md+++ b/spec/threat-rules/FIELDS.md@@@ -30,7 -30,7 +30,8 @@@ Regenerate with `beacon rules fields --| `e.error.type` | string || `e.event.action` | string || `e.event.category` | string |+ | `e.event.fidelity` | string |+| `e.event.id` | string || `e.event.kind` | string || `e.field_truncated` | bool || `e.file.diff_bytes` | int |