Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

Merge origin/main; keep both Implementation Notes bullets and regenerate FIELDS.md

Claudeed1c63e2eca6
diff --cc CLAUDE.md
index 7236107c,342d3d9c..d77df15c
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@@ -353,7 -365,7 +365,8 @@@ Use these npm trusted publisher values
- For macOS-only behavior, gate tests with `runtime.GOOS == "darwin"` or assert the non-Darwin contract explicitly.
- Keep endpoint event schema fields stable: `vendor`, `product`, `schema_version`, required event fields, and Wazuh-compatible JSONL output are release contracts.
- Preserve optional event fields for agent-native metadata (`session`, `trace`, `tool`, `file`, `command`, `mcp`, `approval`, `content`, `model`, `repository`, and `branch`) without changing existing required field semantics.
+ - Set the provenance markers on every new collection path. `harness.collection_method` (`hook`, `otlp`, `plugin`, `poll`) records the mechanism; `event.fidelity` (`observed`, `inferred`) records whether the source named the action or Beacon derived it. Both are defined in `pkg/asymptoteobserve/provenance.go` and are the only sanctioned way to express that difference — do not add per-runtime confidence fields, and do not leave a derived action unmarked. A new plugin-shaped runtime must be added to `CollectionMethodForPlatform`, whose default is `hook`. An action synthesized from an adjacent observation, such as an approval built from a pre-tool notification on a runtime with no approval hook, is `inferred`.
+- Keep event identity derivable rather than asserted. `gen_ai.tool.call.id` carries the runtime's own name for one tool invocation, promoted on every capture path from the shared alias list in `pkg/asymptoteobserve.ToolCallIDKeys`; a new runtime adds its spelling there rather than in a mapper. `event.id` is a deterministic UUID derived by `EventIDForLine` from the event about to be written, so both capture paths name one action the same way. Its namespace constant and derivation are a contract: changing either renames every event ID Beacon has written, and a test pins the value.
- Keep `gen_ai.usage` as the single canonical token-usage representation: normalize all runtime token telemetry (span attributes, metric datapoints, legacy `llm.usage.*` aliases) into it, mirror OTel GenAI semconv JSON names exactly, and never add parallel or per-harness token fields. `gen_ai.usage.cost_usd` carries runtime-reported cost only; do not derive cost from local pricing tables.
- When adding a new signal, include stable identifiers/counts/hashes alongside any retained raw content, and route raw fields through redaction, sanitization, truncation, and event-size controls.
- Keep the dashboard read-only. It should inspect local status and JSONL events but must not mutate endpoint configuration or telemetry.
diff --cc docs/telemetry-schema/event-schema.mdx
index 1e4a9460,28a0903e..23cb2e62
--- a/docs/telemetry-schema/event-schema.mdx
+++ b/docs/telemetry-schema/event-schema.mdx
@@@ -36,11 -84,8 +84,12 @@@ call was seen. Anything counting real o
| `severity` | `info`, `low`, `medium`, `high`, or `critical` |
| `endpoint` | Host and operating system context |
| `harness` | Runtime that produced the signal |
+ | `harness.collection_method` | Optional. `hook`, `plugin`, `otlp`, or `poll` — see [Provenance](#provenance-how-beacon-knows-what-it-wrote) |
+`event.id` is written alongside these: a deterministic UUID identifying the event, described under
+[event identity](/telemetry-schema/normalization#event-identity). It is not required, so events
+written by earlier agent versions remain valid.
+
## How it extends OpenTelemetry
Beacon accepts OpenTelemetry attributes, including the GenAI semantic conventions, and projects them into fields that are easier to query across agent runtimes. OpenTelemetry context is preserved under `gen_ai` where available, while commonly investigated values are also normalized into Beacon fields such as `model`, `tool.name`, `prompt.text`, `command.command`, `file.path`, and `gen_ai.usage`.
diff --cc pkg/asymptoteobserve/identity_test.go
index 3ae30070,00000000..f1757041
mode 100644,000000..100644
--- a/pkg/asymptoteobserve/identity_test.go
+++ b/pkg/asymptoteobserve/identity_test.go
@@@ -1,97 -1,0 +1,114 @@@
+package asymptoteobserve
+
+import (
+ "regexp"
+ "testing"
+)
+
+// callLine is one endpoint event the runtime named itself, with the parts the
+// call identity is derived from left as parameters and everything else varying
+// freely, so a test can say exactly which difference it is testing.
+func callLine(harness, timestamp, callID, command string) []byte {
+ return []byte(`{"timestamp":"` + timestamp + `","event":{"action":"command.executed"},` +
+ `"harness":{"name":"` + harness + `"},"session":{"id":"s1","working_directory":"/repo"},` +
+ `"command":{"command":"` + command + `"},"gen_ai":{"tool":{"call":{"id":"` + callID + `"}}},` +
+ `"message":"Shell command executed"}`)
+}
+
+var uuidV5Pattern = regexp.MustCompile(`^[0-9a-f]{8}-[0-9a-f]{4}-5[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$`)
+
+func TestEventIDForLineIsAVersion5UUID(t *testing.T) {
+ for name, line := range map[string][]byte{
+ "call identity": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"),
+ "content identity": []byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"session.activity"},"session":{"id":"s1"}}`),
+ "unparseable": []byte(`not json at all`),
+ } {
+ if id := EventIDForLine(line); !uuidV5Pattern.MatchString(id) {
+ t.Errorf("%s: EventIDForLine = %q, want an RFC 4122 version 5 UUID", name, id)
+ }
+ }
+}
+
+// The point of the call identity: the hook writes when the tool runs and the
+// collector writes when its batch flushes, so the two reports of one call agree
+// on almost nothing except the call ID. They must still name one event.
+func TestEventIDForLineIgnoresWhatTheTwoCapturePathsDisagreeOn(t *testing.T) {
+ hook := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))
+ otlp := EventIDForLine(callLine("otel", "2026-08-21T18:00:07Z", "toolu_1", "echo hi"))
+ if hook != otlp {
+ t.Fatalf("one call reported twice derived two IDs: %s and %s", hook, otlp)
+ }
+}
+
+func TestEventIDForLineSeparatesDistinctCalls(t *testing.T) {
+ first := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))
+ for name, other := range map[string][]byte{
+ "another call": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_2", "echo hi"),
+ "another target": callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "rm -rf /"),
+ } {
+ if id := EventIDForLine(other); id == first {
+ t.Errorf("%s derived the same ID as the original call: %s", name, id)
+ }
+ }
+}
+
+// Without a call ID there is nothing to identify the event by but the event, so
+// the ID has to move when any of it does -- and stay put when none of it does,
+// which is what makes re-reading a log derive the IDs it already had.
+func TestEventIDForLineFallsBackToTheEventItself(t *testing.T) {
+ line := []byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"session.activity"},"session":{"id":"s1"},"message":"Hook executed"}`)
+ later := []byte(`{"timestamp":"2026-08-21T18:00:02Z","event":{"action":"session.activity"},"session":{"id":"s1"},"message":"Hook executed"}`)
+ if EventIDForLine(line) != EventIDForLine(line) {
+ t.Fatal("the same line derived two IDs")
+ }
+ if EventIDForLine(line) == EventIDForLine(later) {
+ t.Fatal("events a second apart derived one ID")
+ }
+}
+
+// A content digest and a call identity must not be able to collide, because
+// they mean different things: see EventIDForLine.
+func TestEventIDForLineKeepsTheTwoIdentityModesApart(t *testing.T) {
+ if eventUUID("call", "x") == eventUUID("content", "x") {
+ t.Fatal("the call and content identity spaces overlap")
+ }
+}
+
+// The namespace and the derivation are a written contract: an event's ID has to
+// survive a release, or nothing downstream can address it across two of them.
+func TestEventIDForLineIsStableAcrossReleases(t *testing.T) {
+ const want = "244a6772-0cbe-5230-9f79-37cb62ce25ee"
+ got := EventIDForLine(callLine("claude_code", "2026-08-21T18:00:01Z", "toolu_1", "echo hi"))
+ if got != want {
+ t.Fatalf("EventIDForLine = %s, want %s -- if this is deliberate, every ID Beacon has "+
+ "ever written just changed meaning", got, want)
+ }
+}
+
+// The same disagreement, at the identity layer: the hook's and the collector's
+// reports of one Write must derive one event ID even though they name the
+// operation differently.
+func TestEventIDForLineIgnoresHowTheOperationIsNamed(t *testing.T) {
+ hook := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"file.modified"},"harness":{"name":"claude_code"},"session":{"id":"s1","working_directory":"/repo"},"file":{"path":"/repo/notes.md","operation":"modify"},"gen_ai":{"tool":{"call":{"id":"toolu_w"}}}}`))
+ collector := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:07Z","event":{"action":"file.modified"},"harness":{"name":"claude_code"},"session":{"id":"s1","working_directory":"/repo"},"file":{"path":"/repo/notes.md","operation":"create"},"gen_ai":{"tool":{"call":{"id":"toolu_w"}}}}`))
+ if hook != collector {
+ t.Fatalf("one Write derived two IDs: %s and %s", hook, collector)
+ }
+}
++
++// Provenance markers must not split the identity of one action.
++//
++// harness.collection_method and event.fidelity exist precisely because the two
++// capture paths differ, so they differ on the two reports of one call by design:
++// the hook says hook/observed, the collector says otlp and may say inferred for
++// an action it classified by substring. If either reached the call identity, one
++// action would derive two IDs again -- which is the failure this whole change
++// exists to remove, arriving by way of an unrelated and perfectly correct
++// addition. Exactly how the August dedup regression happened.
++func TestEventIDForLineIgnoresProvenanceMarkers(t *testing.T) {
++ hook := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:01Z","event":{"action":"command.executed","fidelity":"observed"},"harness":{"name":"claude_code","collection_method":"hook"},"session":{"id":"s1","working_directory":"/repo"},"command":{"command":"echo hi"},"gen_ai":{"tool":{"call":{"id":"toolu_1"}}}}`))
++ collector := EventIDForLine([]byte(`{"timestamp":"2026-08-21T18:00:07Z","event":{"action":"command.executed","fidelity":"inferred"},"harness":{"name":"claude_code","collection_method":"otlp"},"session":{"id":"s1","working_directory":"/repo"},"command":{"command":"echo hi"},"gen_ai":{"tool":{"call":{"id":"toolu_1"}}}}`))
++ if hook != collector {
++ t.Fatalf("provenance markers split one call into two IDs: %s and %s", hook, collector)
++ }
++}
diff --cc spec/threat-rules/FIELDS.md
index df7157b7,93dd4932..ceb53049
--- a/spec/threat-rules/FIELDS.md
+++ b/spec/threat-rules/FIELDS.md
@@@ -30,7 -30,7 +30,8 @@@ Regenerate with `beacon rules fields --
| `e.error.type` | string |
| `e.event.action` | string |
| `e.event.category` | string |
+ | `e.event.fidelity` | string |
+| `e.event.id` | string |
| `e.event.kind` | string |
| `e.field_truncated` | bool |
| `e.file.diff_bytes` | int |