The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
Address review: kernel-pinned identity, exact-connection lookup
Claude404eca29e702
diff --git a/CLAUDE.md b/CLAUDE.mdindex 4ce1c661..b56b80ca 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -59,7 +59,7 @@ Supported runtime surfaces today:- DeepSeek Harness (`dsh`) is installed through two user-level files: `$DSH_HOME/beacon-endpoint-hooks.json` and a row in `$DSH_HOME/cordis.patch.yml`; there is no project scope. The `@deepseek-ai/dsh-hooks-claude-code` bridge that row mounts is not shipped with the CLI (dev dependency only) and must be installed per profile with `@deepseek-ai/dsh-hook-protocol` at exactly the dsh version; a mount dsh cannot load is skipped at boot with only a warning. So `InstallDsh` first runs `hooks.InspectDshBridge` (`internal/endpoint/hooks/dsh_bridge.go`, read-only: the installation found from the `dsh` launcher on PATH, each `$DSH_HOME/profiles/*` resolved Node-style, and rows in any patch layer that mount a bridge by absolute path at Beacon's hooks file) and returns `*DshBridgeUnavailableError` without writing anything when nothing loads a matching bridge; explicit `--harness dsh` fails, `--all` and endpoint install skip with a warning. A healthy by-path mount (how Nix installs it; it must also resolve its declared dsh peers from its own location, because dsh's host-package interception applies only to profile plugins, else `peers_missing`) counts as installed and Beacon writes no row of its own (removing one it wrote earlier) so hooks never run twice. Beacon never installs the packages. `DshStatus.Active`/`Bridge`, discovery (misconfigured when inactive), doctor `dsh_hook_bridge`, and inventory (each profile's patch layer) surface it; tests build on-disk fixtures with `internal/endpoint/hooks/dshtest`. It also installs a Beacon-owned local skill at `$DSH_HOME/skills/beacon-endpoint/SKILL.md` and supports native session backfill from `$DSH_HOME/sessions` through `beacon endpoint dsh sync`, which fills assistant text/reasoning, tool failures, and token usage that the hook bridge does not expose. It rides the Claude-shaped envelope on the hook path, but `run_code` remains `tool.invoked` because nested commands fire their own hooks.- fx (vercel-labs/fx) telemetry read from the session store fx commits under `~/.fx/sessions/`, covering session start, prompts, tool calls, commands with exit codes and output, file reads/creates/edits with diffs, MCP tool calls, agent messages, history compaction, and per-turn token usage plus reported cost. fx exposes no third-party hook, plugin, or OTLP surface -- its lifecycle hooks are compiled into the binary -- so collection is a read of fx's own records, run by `beacon endpoint fx sync`. Every event carries `harness.collection_method=poll`; approvals are not synthesized (fx persists permission feedback text, not a per-call decision, matching the Cline and Pi posture) and no session-end event is invented, since fx sessions are resumable. Implementation lives in `cli/beacon/internal/fxsession` and `cli/beacon/cmd/endpoint_fx.go`.- Harness² (google-research/harness2) rollouts, read from the executor event streams Harness² records under its runs directory (`$HARNESS2_RUNS_DIR`, `~/harness2/runs`, or a `--runs-dir` an earlier sync was given and the cursor file remembers): OpenCode `run --format json` and Codex `exec --json`, recognized by their first event rather than by file name. Harness² runs Codex under its own `CODEX_HOME` with `--ephemeral` and OpenCode from a source checkout, so Beacon's Codex/OpenCode integrations do not record these sessions. Events carry `harness.name=harness2`, `harness.collection_method=poll`, the executor's own session id, and the rollout's benchmark/task/label/run/trial/attempt/harness fingerprint under `raw.harness2`. Codex `turn.completed` usage is the thread's running total and is differenced per turn; Codex exec lines carry no times, so they take the file's mtime. No prompts (an argv argument, absent from both streams) and no approvals are recorded or synthesized. `testdata/recorded` holds streams recorded from the exact OpenCode and Codex releases Harness² pins; re-record them when it bumps either. Implementation lives in `cli/beacon/internal/harness2session` and `cli/beacon/cmd/endpoint_harness2.go`.-- Attribution of OTLP events: every account on a machine exports to the same loopback collector, and the exporter's base `user` is the collector process's own account (`asymptoteobserve.NewEvent` reads `$USER`/`$USERNAME`). On Linux the collector replaces it with the account that owns the sending socket (peer attribution): `collector.ConfigYAML` renders, only where `collector.PeerAttribution` (`runtime.GOOS == "linux"`), the `beacon_otlp_http` receiver (`collector-builder/exporter/beaconjsonexporter/otlphttpreceiver`, the upstream OTLP/HTTP protocol rebuilt on confighttp so `client.Info.Addr` keeps the client port that upstream drops; it replaces `otlp.protocols.http` on the same port) and the `beacon_peer` processor (`peerprocessor`, right after `memory_limiter` and before `batch`, which loses the connection) on Beacon's own pipelines, never the OTLP forward's, plus `peer_user: true` on beaconjson. `beacon_peer` removes any `beacon.peer.uid`/`beacon.peer.user` resource attribute a sender wrote, then asks `internal/peercred` (`/proc/net/tcp{,6}`: the line whose local address is the client's; inode 0 (TIME_WAIT/orphan, uid printed 0), no match, or two owners are unknown, never guessed) and names the uid through `internal/accountname` (shared with `beacon_ingest`); the converter's `applyPeerUser` copies them into `user` only with `Options.PeerUser`, before `normalizeClaudeCoworkContext`, whose cloud identity still wins. `CheckComponentsFor` refuses a collector without both where they are rendered. An unidentified sender keeps the collector's account; macOS and Windows have no lookup and keep it too (root, or `COMPUTERNAME$` under LocalSystem). Hook and plugin events carry the person's uid or SID from `os/user.Current()` (verified by the OS in restricted mode). The person is also on the session's hook and plugin events, joined by `session.id`; the Sentinel guide carries the KQL. Never describe OTLP events on macOS or Windows as carrying the person who ran the agent.+- Attribution of OTLP events: every account on a machine exports to the same loopback collector, and the exporter's base `user` is the collector process's own account (`asymptoteobserve.NewEvent` reads `$USER`/`$USERNAME`). On Linux the collector replaces it with the account that owns the sending socket (peer attribution): `collector.ConfigYAML` renders, only where `collector.PeerAttribution` (`runtime.GOOS == "linux"`), the `beacon_otlp_http` receiver (`collector-builder/exporter/beaconjsonexporter/otlphttpreceiver`, the upstream OTLP/HTTP protocol rebuilt on confighttp so `client.Info.Addr` keeps the client port that upstream drops; it replaces `otlp.protocols.http` on the same port) and the `beacon_peer` processor (`peerprocessor`, right after `memory_limiter` and before `batch`, which loses the connection) on Beacon's own pipelines, never the OTLP forward's, plus `peer_user: true` on beaconjson. `beacon_peer` removes `beacon.peer.uid`/`beacon.peer.user` wherever a sender put them (resource, scope, record, span, span event and link, every datapoint type), then asks `internal/peercred` for the owner of the client socket of exactly this connection: the client address from `client.Info.Addr` and the collector's end from the gRPC peer's `LocalAddr` or net/http's `LocalAddrContextKey` (no server address, no lookup). `peercred` first asks the kernel with one `NETLINK_SOCK_DIAG` exact four-tuple lookup (`diag_linux.go`, constant cost per export) and only when sock_diag itself is unavailable scans `/proc/net/tcp{,6}` for the same four-tuple; inode 0 (TIME_WAIT/orphan, uid printed 0), no match, or two owners are unknown, never guessed, and an unrelated TIME_WAIT on the same port is skipped. The name comes from `internal/accountname` (shared with `beacon_ingest`). The converter reads the peer attributes from the resource only (`mergeSignalAttrs`), and `applyPeerUser` copies them into `user` only with `Options.PeerUser`, before `normalizeClaudeCoworkContext`, whose cloud identity still wins; with `PeerUser` an unidentified sender gets an empty `user`, never the collector's root, so a recorded root always means root sent it. `CheckComponentsFor` refuses a collector without both where they are rendered. macOS and Windows have no lookup and keep the collector's account (root, or `COMPUTERNAME$` under LocalSystem). Hook and plugin events carry the person's uid or SID from `os/user.Current()` (verified by the OS in restricted mode). The person is also on the session's hook and plugin events, joined by `session.id`; the Sentinel guide carries the KQL. Never describe OTLP events on macOS or Windows as carrying the person who ran the agent.- Claude Cowork admin-configured OpenTelemetry setup guidance and local validation.- OpenClaw Gateway uses a Beacon-managed plugin directory plus OpenClaw's own diagnostics OTLP guidance (`beacon endpoint integrations openclaw`). The plugin's `before_tool_call` hook is fail-closed, so the handler dispatches synchronously and returns without awaiting. Conversation hooks, including `llm_output`, require `plugins.entries.beacon-endpoint.hooks.allowConversationAccess`; without it, token and agent-message coverage is partial. Beacon reads OpenClaw JSON/JSON5 config for advice but does not rewrite it.- GitHub Copilot CLI uses MDM/customer-managed OTLP HTTP for live spans and approval-like activity, plus `beacon endpoint copilot sync` to backfill local session records from `~/.copilot/session-state`. Poll events carry assistant text, complete tool results, command/file details, and token usage with `harness.collection_method=poll`, plus the shutdown's session-wide `totalNanoAiu` differenced into `gen_ai.usage.cost_usd` once per shutdown (10^9 nano-AIU = one AI credit = US$0.01), only for sessions started on or after the 2026-06-01 AI-credit cutover (`cli/beacon/internal/copilotsession/cost.go`); approvals still come only from the live OTLP path and are not synthesized from persisted records.diff --git a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.goindex 91cc7ade..18ff2ef0 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/converter.go@@ -179,7 +179,7 @@ func (c Converter) EventsFromMetrics(metrics pmetric.Metrics) []Event {}func ShouldDropLog(resourceAttrs map[string]interface{}, record plog.LogRecord) bool {- attrs := MergeMaps(resourceAttrs, AttrsToMap(record.Attributes()))+ attrs := mergeSignalAttrs(resourceAttrs, AttrsToMap(record.Attributes()))if name := ClaudeLogEventName(attrs, record.Body().AsString()); isClaudeAPIBody(name) {// A request body is decided once it is parsed, in eventFromLog: it is kept for a WebFetch// page or for the MCP tools it advertised. Nothing is kept from a response body.@@ -350,7 +350,7 @@ func (c Converter) EventFromLog(resourceAttrs map[string]interface{}, record plo// listings are the mcp.tool_listed events for the MCP tools a Claude Code request body advertised,// which are kept in its place.func (c Converter) eventFromLog(resourceAttrs map[string]interface{}, record plog.LogRecord) (event Event, keep bool, listings []Event) {- attrs := MergeMaps(resourceAttrs, AttrsToMap(record.Attributes()))+ attrs := mergeSignalAttrs(resourceAttrs, AttrsToMap(record.Attributes()))body := record.Body().AsString()kept, isAPIBody := takeClaudeAPIBody(attrs, ClaudeLogEventName(attrs, body), c.opts.CaptureModelContext)ts := Timestamp(record.Timestamp().AsTime())@@ -388,7 +388,7 @@ func (c Converter) eventFromLog(resourceAttrs map[string]interface{}, record plo}func (c Converter) EventFromSpan(resourceAttrs map[string]interface{}, span ptrace.Span) Event {- attrs := MergeMaps(resourceAttrs, AttrsToMap(span.Attributes()))+ attrs := mergeSignalAttrs(resourceAttrs, AttrsToMap(span.Attributes()))codexTurnUsage := IsCodexTurnUsageSpan(attrs, span.Name())action := statedAction(attrs, "beacon.event.action", "event.action", "gen_ai.agent.action", "ai.agent.action")fidelity := asymptoteobserve.FidelityObserved@@ -434,7 +434,7 @@ func (c Converter) EventFromSpan(resourceAttrs map[string]interface{}, span ptra}func (c Converter) ShouldDropSpan(resourceAttrs map[string]interface{}, span ptrace.Span) bool {- attrs := MergeMaps(resourceAttrs, AttrsToMap(span.Attributes()))+ attrs := mergeSignalAttrs(resourceAttrs, AttrsToMap(span.Attributes()))harness := HarnessName(attrs, span.Name())if isCodexHarness(harness) {return !c.opts.IncludeCodexSpans && !IsCodexTurnUsageSpan(attrs, span.Name())@@ -899,7 +899,7 @@ func JSONMapAttr(attrs map[string]interface{}, key string) map[string]interface{}func (c Converter) EventFromMetric(resourceAttrs map[string]interface{}, metric pmetric.Metric) Event {- attrs := MergeMaps(resourceAttrs, map[string]interface{}{})+ attrs := mergeSignalAttrs(resourceAttrs, map[string]interface{}{})action := statedAction(attrs, "beacon.event.action", "event.action")if action == "" {action = "metric.observed"@@ -1109,7 +1109,7 @@ func numberDataPointValue(dp pmetric.NumberDataPoint) float64 {}func (c Converter) usageEventFromDataPoint(resourceAttrs map[string]interface{}, metric pmetric.Metric, dpAttrs pcommon.Map, ts pcommon.Timestamp, value float64, extra map[string]interface{}) Event {- attrs := MergeMaps(resourceAttrs, AttrsToMap(dpAttrs))+ attrs := mergeSignalAttrs(resourceAttrs, AttrsToMap(dpAttrs))action := "token.usage"if IsCostUsageMetric(metric.Name()) {action = "cost.usage"@@ -2365,8 +2365,13 @@ func (c Converter) applyPeerUser(event *Event, attrs map[string]interface{}) {if !c.opts.PeerUser {return}+ // With the processor in the pipeline, the collector's own account is never a fallback: on a+ // system install it is root, and a sender that closed its connection before the lookup would+ // otherwise be recorded exactly like root's own, kernel-attributed exports. An event whose sender+ // could not be identified carries no user at all.uid := FirstString(attrs, PeerUIDAttribute)if uid == "" {+ event.User = UserInfo{}return}event.User.UID = uid@@ -2374,3 +2379,20 @@ func (c Converter) applyPeerUser(event *Event, attrs map[string]interface{}) {// collector's name beside someone else's uid.event.User.Name = FirstString(attrs, PeerUserAttribute)}++// mergeSignalAttrs is MergeMaps(resource, signal) for one record, span or datapoint, except that the+// peer attributes come from the resource alone. beacon_peer writes them on the resource, and in a+// plain merge a record, span or datapoint attribute of the same name would win, letting a sender+// name any account it liked. The processor also strips them below the resource; this keeps the+// converter right on its own.+func mergeSignalAttrs(resource, signal map[string]interface{}) map[string]interface{} {+ merged := MergeMaps(resource, signal)+ for _, key := range []string{PeerUIDAttribute, PeerUserAttribute} {+ if value, ok := resource[key]; ok {+ merged[key] = value+ } else {+ delete(merged, key)+ }+ }+ return merged+}diff --git a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/peer_user_test.go b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/peer_user_test.goindex 395790a0..68560707 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/peer_user_test.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/beaconevent/peer_user_test.go@@ -4,6 +4,7 @@ import ("testing""go.opentelemetry.io/collector/pdata/plog"+ "go.opentelemetry.io/collector/pdata/pmetric""go.opentelemetry.io/collector/pdata/ptrace")@@ -72,3 +73,67 @@ func TestCoworkIdentityStillWinsOverThePeer(t *testing.T) {t.Fatalf("user = %+v, want the Cowork identity", event.User)}}++// A sender cannot name an account from below the resource: beacon_peer writes the resource, and a+// record, span or datapoint attribute of the same name used to win the merge.+func TestPeerAttributesBelowTheResourceAreIgnored(t *testing.T) {+ t.Setenv("USER", "root")+ conv := NewConverter(Options{PeerUser: true})++ // The kernel named alice; the record claims root.+ resource, record := peerLog("chatgpt_web")+ record.Attributes().PutStr(PeerUIDAttribute, "0")+ record.Attributes().PutStr(PeerUserAttribute, "root")+ if got := conv.EventFromLog(resource, record).User; got.Name != "alice" || got.UID != "1001" {+ t.Fatalf("log user = %+v, want the kernel's alice over the record's claim", got)+ }++ // The kernel named nobody; the record's claim must not stand in.+ unknown := map[string]interface{}{"service.name": "chatgpt_web"}+ if got := conv.EventFromLog(unknown, record).User; got != (UserInfo{}) {+ t.Fatalf("log user = %+v, want none: the only identity was the sender's own claim", got)+ }++ span := ptrace.NewSpan()+ span.SetName("tool")+ span.Attributes().PutStr(PeerUIDAttribute, "0")+ span.Attributes().PutStr(PeerUserAttribute, "root")+ if got := conv.EventFromSpan(resource, span).User; got.Name != "alice" || got.UID != "1001" {+ t.Fatalf("span user = %+v, want alice", got)+ }+ if got := conv.EventFromSpan(unknown, span).User; got != (UserInfo{}) {+ t.Fatalf("span user = %+v, want none", got)+ }++ metrics := pmetric.NewMetrics()+ rm := metrics.ResourceMetrics().AppendEmpty()+ rm.Resource().Attributes().PutStr("service.name", "claude-code")+ sum := rm.ScopeMetrics().AppendEmpty().Metrics().AppendEmpty()+ sum.SetName("claude_code.token.usage")+ dp := sum.SetEmptySum().DataPoints().AppendEmpty()+ dp.SetIntValue(10)+ dp.Attributes().PutStr("type", "input")+ dp.Attributes().PutStr(PeerUIDAttribute, "0")+ dp.Attributes().PutStr(PeerUserAttribute, "root")+ events := conv.EventsFromMetrics(metrics)+ if len(events) == 0 {+ t.Fatal("no events from the token usage metric")+ }+ for _, e := range events {+ if e.User != (UserInfo{}) {+ t.Fatalf("metric user = %+v, want none: the datapoint's claim must not stand in", e.User)+ }+ }+}++// With the processor in the pipeline, an export it could not attribute carries no user rather than+// the collector's own account, which on a system install would read exactly like root's own.+func TestAnUnidentifiedSenderCarriesNoUser(t *testing.T) {+ t.Setenv("USER", "root")+ t.Setenv("UID", "0")+ _, record := peerLog("chatgpt_web")+ event := NewConverter(Options{PeerUser: true}).EventFromLog(map[string]interface{}{"service.name": "chatgpt_web"}, record)+ if event.User != (UserInfo{}) {+ t.Fatalf("user = %+v, want none", event.User)+ }+}diff --git a/collector-builder/exporter/beaconjsonexporter/internal/peercred/diag_linux.go b/collector-builder/exporter/beaconjsonexporter/internal/peercred/diag_linux.gonew file mode 100644index 00000000..47a9f365--- /dev/null+++ b/collector-builder/exporter/beaconjsonexporter/internal/peercred/diag_linux.go@@ -0,0 +1,161 @@+//go:build linux++package peercred++import (+ "encoding/binary"+ "errors"+ "fmt"+ "net"+ "strconv"+ "syscall"+ "time"+ "unsafe"++ "golang.org/x/sys/unix"+)++// The inet_diag request and reply, as in linux/inet_diag.h. Ports and addresses are in network byte+// order; everything else is in host order.+type inetDiagSockID struct {+ SPort [2]byte+ DPort [2]byte+ Src [16]byte+ Dst [16]byte+ If uint32+ Cookie [2]uint32+}++type inetDiagReqV2 struct {+ Family uint8+ Protocol uint8+ Ext uint8+ Pad uint8+ States uint32+ ID inetDiagSockID+}++type inetDiagMsg struct {+ Family uint8+ State uint8+ Timer uint8+ Retrans uint8+ ID inetDiagSockID+ Expires uint32+ RQueue uint32+ WQueue uint32+ UID uint32+ Inode uint32+}++const (+ sockDiagByFamily = 20 // SOCK_DIAG_BY_FAMILY+ inetDiagNoCookie = 0xFFFFFFFF // INET_DIAG_NOCOOKIE+ diagTimeout = time.Second+)++// errDiagUnavailable is any failure of the netlink channel itself, as opposed to an answer; the+// caller falls back to /proc on it.+var errDiagUnavailable = errors.New("sock_diag is not available")++// sockDiagUID asks the kernel, over NETLINK_SOCK_DIAG, for the socket whose local end is client and+// remote end is server: one hash lookup (inet_diag_find_one_icsk), not a dump. An IPv4 pair is tried+// as AF_INET and then, for a dual-stack socket, as its IPv4-mapped AF_INET6 form.+func sockDiagUID(client, server *net.TCPAddr) (string, error) {+ type attempt struct {+ family uint8+ src, dst net.IP+ }+ var attempts []attempt+ if c4, s4 := client.IP.To4(), server.IP.To4(); c4 != nil && s4 != nil {+ attempts = append(attempts, attempt{unix.AF_INET, c4, s4}, attempt{unix.AF_INET6, mapped(c4), mapped(s4)})+ } else {+ attempts = append(attempts, attempt{unix.AF_INET6, client.IP.To16(), server.IP.To16()})+ }+ for _, a := range attempts {+ uid, err := sockDiagOne(a.family, a.src, client.Port, a.dst, server.Port)+ if errors.Is(err, ErrNotFound) {+ continue+ }+ return uid, err+ }+ return "", ErrNotFound+}++func sockDiagOne(family uint8, src net.IP, sport int, dst net.IP, dport int) (string, error) {+ fd, err := unix.Socket(unix.AF_NETLINK, unix.SOCK_DGRAM|unix.SOCK_CLOEXEC, unix.NETLINK_SOCK_DIAG)+ if err != nil {+ return "", fmt.Errorf("%w: %v", errDiagUnavailable, err)+ }+ defer unix.Close(fd)+ tv := unix.NsecToTimeval(diagTimeout.Nanoseconds())+ _ = unix.SetsockoptTimeval(fd, unix.SOL_SOCKET, unix.SO_RCVTIMEO, &tv)++ req := inetDiagReqV2{Family: family, Protocol: unix.IPPROTO_TCP, States: 0xFFFFFFFF}+ binary.BigEndian.PutUint16(req.ID.SPort[:], uint16(sport))+ binary.BigEndian.PutUint16(req.ID.DPort[:], uint16(dport))+ if family == unix.AF_INET {+ copy(req.ID.Src[:4], src.To4())+ copy(req.ID.Dst[:4], dst.To4())+ } else {+ copy(req.ID.Src[:], src.To16())+ copy(req.ID.Dst[:], dst.To16())+ }+ req.ID.Cookie = [2]uint32{inetDiagNoCookie, inetDiagNoCookie}++ body := (*[unsafe.Sizeof(req)]byte)(unsafe.Pointer(&req))[:]+ hdr := unix.NlMsghdr{+ Len: uint32(unix.NLMSG_HDRLEN + len(body)),+ Type: sockDiagByFamily,+ Flags: unix.NLM_F_REQUEST,+ Seq: 1,+ }+ msg := make([]byte, 0, hdr.Len)+ msg = append(msg, (*[unix.SizeofNlMsghdr]byte)(unsafe.Pointer(&hdr))[:]...)+ msg = append(msg, body...)+ if err := unix.Sendto(fd, msg, 0, &unix.SockaddrNetlink{Family: unix.AF_NETLINK}); err != nil {+ return "", fmt.Errorf("%w: %v", errDiagUnavailable, err)+ }++ buf := make([]byte, 8192)+ n, _, err := unix.Recvfrom(fd, buf, 0)+ if err != nil {+ return "", fmt.Errorf("%w: %v", errDiagUnavailable, err)+ }+ replies, err := syscall.ParseNetlinkMessage(buf[:n])+ if err != nil {+ return "", fmt.Errorf("%w: %v", errDiagUnavailable, err)+ }+ for _, reply := range replies {+ switch reply.Header.Type {+ case unix.NLMSG_ERROR:+ if len(reply.Data) < 4 {+ return "", errDiagUnavailable+ }+ errno := -int32(binary.NativeEndian.Uint32(reply.Data[:4]))+ if errno == 0 {+ continue+ }+ if syscall.Errno(errno) == syscall.ENOENT {+ return "", ErrNotFound+ }+ return "", fmt.Errorf("%w: %v", errDiagUnavailable, syscall.Errno(errno))+ case sockDiagByFamily:+ var m inetDiagMsg+ if len(reply.Data) < int(unsafe.Sizeof(m)) {+ return "", errDiagUnavailable+ }+ m = *(*inetDiagMsg)(unsafe.Pointer(&reply.Data[0]))+ // The kernel answered for exactly this connection, or the answer is not about it.+ if m.ID.SPort != req.ID.SPort || m.ID.DPort != req.ID.DPort {+ return "", ErrNotFound+ }+ if m.Inode == 0 {+ // TIME_WAIT or orphaned: uid reads 0 whoever created it.+ return "", ErrNotFound+ }+ return strconv.FormatUint(uint64(m.UID), 10), nil+ }+ }+ return "", ErrNotFound+}diff --git a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred.go b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred.goindex 971ffb78..0c8df646 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred.go@@ -30,14 +30,22 @@ var (ErrNotLoopback = errors.New("not a loopback address"))-// UID returns the account id, as a decimal string, that owns the client socket bound to client: the-// remote address of a connection the collector accepted.-func UID(client *net.TCPAddr) (string, error) {+// UID returns the account id, as a decimal string, that owns the client end of the connection from+// client to server: the remote and local addresses of a connection the collector accepted.+//+// Both ends are matched, never the client's address alone. Linux lets connections to different+// destinations share one source port, so 127.0.0.1:P can at the same time be this client and an+// unrelated socket another account opened to some other service; matching only P would then name+// that account, or give up on a stray TIME_WAIT entry, instead of finding this connection.+func UID(client, server *net.TCPAddr) (string, error) {if client == nil || client.Port <= 0 || client.Port > 65535 {return "", ErrNotFound}+ if server == nil || server.Port <= 0 || server.Port > 65535 {+ return "", ErrNotFound+ }if !client.IP.IsLoopback() {return "", ErrNotLoopback}- return lookupUID(client)+ return lookupUID(client, server)}diff --git a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux.go b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux.goindex 6e752be0..e0e781b3 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux.go@@ -5,6 +5,7 @@ package peercredimport ("bufio""encoding/binary"+ "errors""fmt""io""net"@@ -13,35 +14,48 @@ import ("strings")-// procNet is where the socket tables are read. A var so tests can point it at fixtures.+// procNet is where the fallback reads the socket tables. A var so tests can point it at fixtures.var procNet = "/proc/net"-// lookupUID finds the client socket in the kernel's TCP tables.+// diagLookup is the kernel's own exact lookup (see diag_linux.go); a var so tests can force the+// fallback.+var diagLookup = sockDiagUID++// lookupUID asks the kernel who owns the client socket of the connection client -> server.+//+// The first answer is NETLINK_SOCK_DIAG, the interface ss uses: given the connection's four-tuple+// the kernel looks that one socket up in its connection hash, so the cost does not grow with the+// number of sockets on the machine, which matters because this runs for every export. Where the+// kernel offers no sock_diag (a module left out, a sandbox that refuses netlink), the socket tables+// in /proc/net are read instead, matching the same four-tuple.//-// /proc/net/tcp and tcp6 list every socket in this network namespace, one per line:+// Either way an answer is refused rather than guessed when it cannot be the sender: a socket in+// TIME_WAIT or otherwise orphaned reports uid 0 and inode 0 whoever created it, which would name+// root for a client that has gone; and no socket at all means the client closed before the lookup,+// which a sender could do on purpose.+func lookupUID(client, server *net.TCPAddr) (string, error) {+ uid, err := diagLookup(client, server)+ if err == nil || errors.Is(err, ErrNotFound) {+ return uid, err+ }+ return procLookup(client, server)+}++// procLookup reads /proc/net/tcp and tcp6, where each socket is one line://// sl local_address rem_address st tx_queue:rx_queue tr:tm->when retrnsmt uid timeout inode// 0: 0100007F:B5D2 0100007F:10E6 01 00000000:00000000 00:00000000 00000000 1001 0 81234//-// The client end of a loopback connection is the line whose local address is the address the-// collector saw as remote, and its uid column is the account that created that socket (sk_uid,-// fixed at creation). The table is world-readable, so this needs no privilege beyond the-// collector's own, and it reads the namespace the collector listens in, which is the one a-// loopback client must share.-//-// Three cases are refused rather than guessed. A line with inode 0 is a socket in TIME_WAIT or-// otherwise orphaned: the kernel prints uid 0 for those, which would name root for a client that-// has gone. Two matching lines with different uids cannot both be the client. And no match at all-// means the client closed before the lookup, which a sender could do deliberately; the event is-// then left unattributed, never attributed to whoever else is connected.-func lookupUID(client *net.TCPAddr) (string, error) {- want := map[string]bool{}- if ip4 := client.IP.To4(); ip4 != nil {- want[procAddr4(ip4, client.Port)] = true+// The client's end of the connection is the line whose local address is the client and whose+// remote address is the collector's listener.+func procLookup(client, server *net.TCPAddr) (string, error) {+ want := map[[2]string]bool{}+ if c4, s4 := client.IP.To4(), server.IP.To4(); c4 != nil && s4 != nil {+ want[[2]string{procAddr4(c4, client.Port), procAddr4(s4, server.Port)}] = true// A dual-stack listener sees an IPv4 client as ::ffff:a.b.c.d in tcp6.- want[procAddr6(net.IP(append(net.IP{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0xff}, ip4...)), client.Port)] = true+ want[[2]string{procAddr6(mapped(c4), client.Port), procAddr6(mapped(s4), server.Port)}] = true} else {- want[procAddr6(client.IP.To16(), client.Port)] = true+ want[[2]string{procAddr6(client.IP.To16(), client.Port), procAddr6(server.IP.To16(), server.Port)}] = true}found := ""for _, table := range []string{"tcp", "tcp6"} {@@ -52,7 +66,7 @@ func lookupUID(client *net.TCPAddr) (string, error) {uid, err := scanTable(f, want)_ = f.Close()switch {- case err == ErrNotFound:+ case errors.Is(err, ErrNotFound):continuecase err != nil:return "", err@@ -67,18 +81,19 @@ func lookupUID(client *net.TCPAddr) (string, error) {return found, nil}-func scanTable(r io.Reader, want map[string]bool) (string, error) {+func scanTable(r io.Reader, want map[[2]string]bool) (string, error) {scanner := bufio.NewScanner(r)found := ""for scanner.Scan() {fields := strings.Fields(scanner.Text())// sl, local, remote, st, queues, timer, retransmits, uid, timeout, inode- if len(fields) < 10 || !want[strings.ToUpper(fields[1])] {+ if len(fields) < 10 || !want[[2]string{strings.ToUpper(fields[1]), strings.ToUpper(fields[2])}] {continue}if fields[9] == "0" {- // TIME_WAIT and orphans: the uid column is 0 for them whoever made the socket.- return "", ErrNotFound+ // TIME_WAIT and orphans: the uid column is 0 for them whoever made the socket. Not an+ // answer, and not a reason to stop looking either.+ continue}uid := fields[7]if !isDecimal(uid) {@@ -98,6 +113,10 @@ func scanTable(r io.Reader, want map[string]bool) (string, error) {return found, nil}+func mapped(ip4 net.IP) net.IP {+ return append(net.IP{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0xff, 0xff}, ip4...)+}+// procAddr4 is how the kernel prints an IPv4 socket address: the address as one 32-bit word in host// byte order, then the port, both in upper-case hex.func procAddr4(ip net.IP, port int) string {diff --git a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux_test.go b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux_test.goindex 498da146..c7e5926d 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux_test.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_linux_test.go@@ -39,12 +39,25 @@ func TestUIDNamesTheOwnerOfARealLoopbackConnection(t *testing.T) {server := <-accepteddefer server.Close()- uid, err := UID(server.RemoteAddr().(*net.TCPAddr))- if err != nil {- t.Fatalf("UID(%s) = %v", server.RemoteAddr(), err)+ remote, local := server.RemoteAddr().(*net.TCPAddr), server.LocalAddr().(*net.TCPAddr)+ want := strconv.Itoa(os.Geteuid())+ if uid, err := sockDiagUID(remote, local); err != nil || uid != want {+ if errors.Is(err, errDiagUnavailable) {+ t.Logf("sock_diag unavailable here: %v", err)+ } else {+ t.Fatalf("sock_diag UID(%s -> %s) = %q, %v; want %s", remote, local, uid, err, want)+ }+ }+ if uid, err := procLookup(remote, local); err != nil || uid != want {+ t.Fatalf("/proc UID(%s -> %s) = %q, %v; want %s", remote, local, uid, err, want)+ }+ if uid, err := UID(remote, local); err != nil || uid != want {+ t.Fatalf("UID = %q, %v; want this process's %s", uid, err, want)}- if want := strconv.Itoa(os.Geteuid()); uid != want {- t.Fatalf("UID = %s, want this process's %s", uid, want)+ // The same client address paired with another destination is another connection.+ other := &net.TCPAddr{IP: local.IP, Port: local.Port + 1}+ if uid, err := UID(remote, other); err == nil {+ t.Fatalf("UID matched a connection to %s that does not exist: %s", other, uid)}})}@@ -65,26 +78,60 @@ func TestUIDOfAClosedClientIsUnknown(t *testing.T) {if err != nil {t.Fatal(err)}- remote := server.RemoteAddr().(*net.TCPAddr)+ remote, local := server.RemoteAddr().(*net.TCPAddr), server.LocalAddr().(*net.TCPAddr)// The server closes first, so the server's end takes TIME_WAIT; then the client's end closes// with a reset so it leaves the table entirely._ = server.Close()_ = client.(*net.TCPConn).SetLinger(0)_ = client.Close()- if uid, err := UID(remote); err == nil {+ if uid, err := UID(remote, local); err == nil {t.Fatalf("UID of a closed client = %s, want unknown", uid)}}func TestUIDRefusesWhatItCannotSee(t *testing.T) {- if _, err := UID(&net.TCPAddr{IP: net.ParseIP("10.1.2.3"), Port: 5000}); !errors.Is(err, ErrNotLoopback) {+ collector := &net.TCPAddr{IP: net.ParseIP("127.0.0.1"), Port: 4318}+ if _, err := UID(&net.TCPAddr{IP: net.ParseIP("10.1.2.3"), Port: 5000}, collector); !errors.Is(err, ErrNotLoopback) {t.Fatalf("remote address err = %v, want ErrNotLoopback", err)}for _, addr := range []*net.TCPAddr{nil, {IP: net.ParseIP("127.0.0.1")}, {IP: net.ParseIP("127.0.0.1"), Port: 70000}} {- if _, err := UID(addr); !errors.Is(err, ErrNotFound) {+ if _, err := UID(addr, collector); !errors.Is(err, ErrNotFound) {t.Fatalf("UID(%v) err = %v, want ErrNotFound", addr, err)}}+ if _, err := UID(&net.TCPAddr{IP: net.ParseIP("127.0.0.1"), Port: 5000}, nil); !errors.Is(err, ErrNotFound) {+ t.Fatalf("no server address err = %v, want ErrNotFound", err)+ }+}++// The fallback is used only when sock_diag itself is unavailable, not when it answers "no such+// socket".+func TestLookupFallsBackOnlyWhenSockDiagIsUnavailable(t *testing.T) {+ old := diagLookup+ t.Cleanup(func() { diagLookup = old })+ dir := t.TempDir()+ oldProc := procNet+ procNet = dir+ t.Cleanup(func() { procNet = oldProc })+ header := " sl local_address rem_address st tx_queue rx_queue tr tm->when retrnsmt uid timeout inode\n"+ row := " 2: 0100007F:B5D2 0100007F:10E6 01 00000000:00000000 00:00000000 00000000 1001 0 102 1 0 20 4 30 10 -1\n"+ if err := os.WriteFile(filepath.Join(dir, "tcp"), []byte(header+row), 0o644); err != nil {+ t.Fatal(err)+ }+ client := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0xB5D2}+ server := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0x10E6}+ if procAddr4(client.IP, client.Port) != "0100007F:B5D2" {+ t.Skip("fixtures are little-endian")+ }++ diagLookup = func(_, _ *net.TCPAddr) (string, error) { return "", errDiagUnavailable }+ if uid, err := lookupUID(client, server); err != nil || uid != "1001" {+ t.Fatalf("with sock_diag unavailable: %q, %v; want the /proc answer", uid, err)+ }+ diagLookup = func(_, _ *net.TCPAddr) (string, error) { return "", ErrNotFound }+ if uid, err := lookupUID(client, server); !errors.Is(err, ErrNotFound) {+ t.Fatalf("with sock_diag saying no such socket: %q, %v; want ErrNotFound without a second guess", uid, err)+ }}// Fixtures in the kernel's own format: what systemd-era kernels print, with the address words in@@ -99,6 +146,7 @@ func TestScanTableFixtures(t *testing.T) {return fmt.Sprintf("%4d: %s %s %s 00000000:00000000 00:00000000 00000000 %5s 0 %s 1 0000000000000000 20 4 30 10 -1\n", sl, local, remote, st, uid, inode)}client := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0xB5D2}+ server := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0x10E6}write := func(tcp, tcp6 string) {t.Helper()if err := os.WriteFile(filepath.Join(dir, "tcp"), []byte(header+tcp), 0o644); err != nil {@@ -134,9 +182,27 @@ func TestScanTableFixtures(t *testing.T) {err: ErrNotFound,},{- name: "two owners is ambiguous",+ // Linux lets connections to different destinations share a source port.+ name: "another account's socket on the same port to another service is not this client",tcp: line(2, "0100007F:B5D2", "0100007F:10E6", "01", "1001", "102") +line(3, "0100007F:B5D2", "0100007F:1F90", "01", "1002", "103"),+ want: "1001",+ },+ {+ name: "and when this client has gone, that socket is still not named",+ tcp: line(3, "0100007F:B5D2", "0100007F:1F90", "01", "1002", "103"),+ err: ErrNotFound,+ },+ {+ name: "an unrelated time wait on the same port does not stop the lookup",+ tcp: line(1, "0100007F:B5D2", "0100007F:1538", "06", "0", "0") ++ line(2, "0100007F:B5D2", "0100007F:10E6", "01", "1001", "102"),+ want: "1001",+ },+ {+ name: "two owners of the same connection is ambiguous",+ tcp: line(2, "0100007F:B5D2", "0100007F:10E6", "01", "1001", "102") ++ line(3, "0100007F:B5D2", "0100007F:10E6", "01", "1002", "103"),err: ErrAmbiguous,},{@@ -155,11 +221,16 @@ func TestScanTableFixtures(t *testing.T) {tcp: line(2, "0100007F:B5D3", "0100007F:10E6", "01", "1001", "102"),err: ErrNotFound,},+ {+ name: "the client end, not the collector's end of the same connection",+ tcp: line(1, "0100007F:10E6", "0100007F:B5D2", "01", "0", "101"),+ err: ErrNotFound,+ },}for _, tc := range cases {t.Run(tc.name, func(t *testing.T) {write(tc.tcp, tc.tcp6)- uid, err := UID(client)+ uid, err := procLookup(client, server)if tc.err != nil {if !errors.Is(err, tc.err) {t.Fatalf("UID = %q, %v; want %v", uid, err, tc.err)diff --git a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_other.go b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_other.goindex 4a13906c..529f70b8 100644--- a/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_other.go+++ b/collector-builder/exporter/beaconjsonexporter/internal/peercred/peercred_other.go@@ -4,6 +4,6 @@ package peercredimport "net"-func lookupUID(*net.TCPAddr) (string, error) {+func lookupUID(_, _ *net.TCPAddr) (string, error) {return "", ErrUnsupported}diff --git a/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor.go b/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor.goindex e9008d05..63a3a668 100644--- a/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor.go+++ b/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor.go@@ -19,6 +19,7 @@ package peerprocessorimport ("context""net"+ "net/http""go.opentelemetry.io/collector/client""go.opentelemetry.io/collector/component"@@ -30,6 +31,7 @@ import ("go.opentelemetry.io/collector/processor""go.opentelemetry.io/collector/processor/processorhelper""go.uber.org/zap"+ grpcpeer "google.golang.org/grpc/peer""github.com/asymptote-labs/agent-beacon/collector-builder/exporter/beaconjsonexporter/internal/accountname""github.com/asymptote-labs/agent-beacon/collector-builder/exporter/beaconjsonexporter/internal/beaconevent"@@ -50,7 +52,7 @@ func NewFactory() processor.Factory {// peer resolves a client address to an account.type peer struct {- uid func(*net.TCPAddr) (string, error)+ uid func(client, server *net.TCPAddr) (string, error)name func(uid string) stringlogger *zap.Logger}@@ -75,7 +77,13 @@ func (p *peer) identify(ctx context.Context) (uid, name string) {if !ok || addr == nil {return "", ""}- uid, err := p.uid(addr)+ // The collector's own end of the connection, so the lookup matches the whole connection rather+ // than any socket that happens to share the client's port.+ server := serverAddr(ctx)+ if server == nil {+ return "", ""+ }+ uid, err := p.uid(addr, server)if err != nil {p.logger.Debug("could not identify the account that sent an export", zap.Stringer("client", addr), zap.Error(err))return "", ""@@ -83,6 +91,21 @@ func (p *peer) identify(ctx context.Context) (uid, name string) {return uid, p.name(uid)}+// serverAddr is the local address of the connection the export arrived on: the gRPC server records+// it in the request's peer, and net/http in the request context, which the beacon_otlp_http receiver+// passes down.+func serverAddr(ctx context.Context) *net.TCPAddr {+ if p, ok := grpcpeer.FromContext(ctx); ok && p != nil {+ if addr, ok := p.LocalAddr.(*net.TCPAddr); ok {+ return addr+ }+ }+ if addr, ok := ctx.Value(http.LocalAddrContextKey).(*net.TCPAddr); ok {+ return addr+ }+ return nil+}+// stamp makes a resource's peer attributes exactly what the kernel said: none of the sender's.func stamp(resource pcommon.Resource, uid, name string) {attrs := resource.Attributes()@@ -97,6 +120,74 @@ func stamp(resource pcommon.Resource, uid, name string) {}}+// strip removes the peer attributes from one attribute map below the resource. Only the resource may+// carry them, and only as this processor wrote them: the converter reads them from the resource+// alone, and removing them here keeps a sender's claim out of raw attributes and out of the HEC+// exporters, which see the data as it leaves this processor.+func strip(attrs pcommon.Map) {+ attrs.Remove(beaconevent.PeerUIDAttribute)+ attrs.Remove(beaconevent.PeerUserAttribute)+}++func stripLogs(rl plog.ResourceLogs) {+ for i := 0; i < rl.ScopeLogs().Len(); i++ {+ sl := rl.ScopeLogs().At(i)+ strip(sl.Scope().Attributes())+ for j := 0; j < sl.LogRecords().Len(); j++ {+ strip(sl.LogRecords().At(j).Attributes())+ }+ }+}++func stripTraces(rs ptrace.ResourceSpans) {+ for i := 0; i < rs.ScopeSpans().Len(); i++ {+ ss := rs.ScopeSpans().At(i)+ strip(ss.Scope().Attributes())+ for j := 0; j < ss.Spans().Len(); j++ {+ span := ss.Spans().At(j)+ strip(span.Attributes())+ for k := 0; k < span.Events().Len(); k++ {+ strip(span.Events().At(k).Attributes())+ }+ for k := 0; k < span.Links().Len(); k++ {+ strip(span.Links().At(k).Attributes())+ }+ }+ }+}++func stripMetrics(rm pmetric.ResourceMetrics) {+ for i := 0; i < rm.ScopeMetrics().Len(); i++ {+ sm := rm.ScopeMetrics().At(i)+ strip(sm.Scope().Attributes())+ for j := 0; j < sm.Metrics().Len(); j++ {+ m := sm.Metrics().At(j)+ switch m.Type() {+ case pmetric.MetricTypeGauge:+ for k := 0; k < m.Gauge().DataPoints().Len(); k++ {+ strip(m.Gauge().DataPoints().At(k).Attributes())+ }+ case pmetric.MetricTypeSum:+ for k := 0; k < m.Sum().DataPoints().Len(); k++ {+ strip(m.Sum().DataPoints().At(k).Attributes())+ }+ case pmetric.MetricTypeHistogram:+ for k := 0; k < m.Histogram().DataPoints().Len(); k++ {+ strip(m.Histogram().DataPoints().At(k).Attributes())+ }+ case pmetric.MetricTypeExponentialHistogram:+ for k := 0; k < m.ExponentialHistogram().DataPoints().Len(); k++ {+ strip(m.ExponentialHistogram().DataPoints().At(k).Attributes())+ }+ case pmetric.MetricTypeSummary:+ for k := 0; k < m.Summary().DataPoints().Len(); k++ {+ strip(m.Summary().DataPoints().At(k).Attributes())+ }+ }+ }+ }+}+var mutates = processorhelper.WithCapabilities(consumer.Capabilities{MutatesData: true})func createLogs(ctx context.Context, set processor.Settings, cfg component.Config, next consumer.Logs) (processor.Logs, error) {@@ -105,6 +196,7 @@ func createLogs(ctx context.Context, set processor.Settings, cfg component.Confiuid, name := p.identify(ctx)for i := 0; i < logs.ResourceLogs().Len(); i++ {stamp(logs.ResourceLogs().At(i).Resource(), uid, name)+ stripLogs(logs.ResourceLogs().At(i))}return logs, nil}, mutates)@@ -116,6 +208,7 @@ func createTraces(ctx context.Context, set processor.Settings, cfg component.Conuid, name := p.identify(ctx)for i := 0; i < traces.ResourceSpans().Len(); i++ {stamp(traces.ResourceSpans().At(i).Resource(), uid, name)+ stripTraces(traces.ResourceSpans().At(i))}return traces, nil}, mutates)@@ -127,6 +220,7 @@ func createMetrics(ctx context.Context, set processor.Settings, cfg component.Couid, name := p.identify(ctx)for i := 0; i < metrics.ResourceMetrics().Len(); i++ {stamp(metrics.ResourceMetrics().At(i).Resource(), uid, name)+ stripMetrics(metrics.ResourceMetrics().At(i))}return metrics, nil}, mutates)diff --git a/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor_test.go b/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor_test.goindex 42aef073..dbb86d33 100644--- a/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor_test.go+++ b/collector-builder/exporter/beaconjsonexporter/peerprocessor/processor_test.go@@ -4,6 +4,7 @@ import ("context""errors""net"+ "net/http""testing""go.opentelemetry.io/collector/client"@@ -13,6 +14,8 @@ import ("go.opentelemetry.io/collector/pdata/ptrace""go.opentelemetry.io/collector/processor/processortest"+ grpcpeer "google.golang.org/grpc/peer"+"github.com/asymptote-labs/agent-beacon/collector-builder/exporter/beaconjsonexporter/internal/beaconevent")@@ -21,7 +24,10 @@ func fakeKernel(t *testing.T, owner *net.TCPAddr, uid, name string) {t.Helper()oldUID, oldName := lookupUID, lookupNamet.Cleanup(func() { lookupUID, lookupName = oldUID, oldName })- lookupUID = func(addr *net.TCPAddr) (string, error) {+ lookupUID = func(addr, server *net.TCPAddr) (string, error) {+ if server == nil || server.Port != collectorPort {+ return "", errors.New("not the collector's end")+ }if owner != nil && addr.IP.Equal(owner.IP) && addr.Port == owner.Port {return uid, nil}@@ -35,8 +41,13 @@ func fakeKernel(t *testing.T, owner *net.TCPAddr, uid, name string) {}}+const collectorPort = 4318++// from is the context an export arrives with: the client's address in client.Info and the+// collector's own end where net/http puts it.func from(addr net.Addr) context.Context {- return client.NewContext(context.Background(), client.Info{Addr: addr})+ ctx := context.WithValue(context.Background(), http.LocalAddrContextKey, &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: collectorPort})+ return client.NewContext(ctx, client.Info{Addr: addr})}func forgedLogs() plog.Logs {@@ -164,3 +175,96 @@ func TestDeclaresThatItMutates(t *testing.T) {t.Fatal("MutatesData = false")}}++// A claim below the resource is removed everywhere a sender can put one, so only the resource+// carries an account, and only the kernel's.+func TestClaimsBelowTheResourceAreRemoved(t *testing.T) {+ alice := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 50126}+ fakeKernel(t, alice, "1001", "alice")+ set := processortest.NewNopSettings(componentType)+ forge := func(m interface{ PutStr(string, string) }) {+ m.PutStr(beaconevent.PeerUIDAttribute, "0")+ m.PutStr(beaconevent.PeerUserAttribute, "root")+ }++ logs := plog.NewLogs()+ sl := logs.ResourceLogs().AppendEmpty().ScopeLogs().AppendEmpty()+ forge(sl.Scope().Attributes())+ forge(sl.LogRecords().AppendEmpty().Attributes())+ lsink := new(consumertest.LogsSink)+ lp, _ := NewFactory().CreateLogs(context.Background(), set, &Config{}, lsink)+ if err := lp.ConsumeLogs(from(alice), logs); err != nil {+ t.Fatal(err)+ }+ gotSL := lsink.AllLogs()[0].ResourceLogs().At(0).ScopeLogs().At(0)+ for name, attrs := range map[string]int{+ "scope": gotSL.Scope().Attributes().Len(),+ "record": gotSL.LogRecords().At(0).Attributes().Len(),+ } {+ if attrs != 0 {+ t.Errorf("log %s kept %d peer attributes", name, attrs)+ }+ }++ traces := ptrace.NewTraces()+ span := traces.ResourceSpans().AppendEmpty().ScopeSpans().AppendEmpty().Spans().AppendEmpty()+ forge(span.Attributes())+ forge(span.Events().AppendEmpty().Attributes())+ forge(span.Links().AppendEmpty().Attributes())+ tsink := new(consumertest.TracesSink)+ tp, _ := NewFactory().CreateTraces(context.Background(), set, &Config{}, tsink)+ if err := tp.ConsumeTraces(from(alice), traces); err != nil {+ t.Fatal(err)+ }+ gotSpan := tsink.AllTraces()[0].ResourceSpans().At(0).ScopeSpans().At(0).Spans().At(0)+ if gotSpan.Attributes().Len()+gotSpan.Events().At(0).Attributes().Len()+gotSpan.Links().At(0).Attributes().Len() != 0 {+ t.Error("a span, span event or link kept a peer attribute")+ }++ metrics := pmetric.NewMetrics()+ ms := metrics.ResourceMetrics().AppendEmpty().ScopeMetrics().AppendEmpty().Metrics()+ forge(ms.AppendEmpty().SetEmptySum().DataPoints().AppendEmpty().Attributes())+ forge(ms.AppendEmpty().SetEmptyGauge().DataPoints().AppendEmpty().Attributes())+ forge(ms.AppendEmpty().SetEmptyHistogram().DataPoints().AppendEmpty().Attributes())+ forge(ms.AppendEmpty().SetEmptyExponentialHistogram().DataPoints().AppendEmpty().Attributes())+ forge(ms.AppendEmpty().SetEmptySummary().DataPoints().AppendEmpty().Attributes())+ msink := new(consumertest.MetricsSink)+ mp, _ := NewFactory().CreateMetrics(context.Background(), set, &Config{}, msink)+ if err := mp.ConsumeMetrics(from(alice), metrics); err != nil {+ t.Fatal(err)+ }+ got := msink.AllMetrics()[0].ResourceMetrics().At(0).ScopeMetrics().At(0).Metrics()+ n := got.At(0).Sum().DataPoints().At(0).Attributes().Len() ++ got.At(1).Gauge().DataPoints().At(0).Attributes().Len() ++ got.At(2).Histogram().DataPoints().At(0).Attributes().Len() ++ got.At(3).ExponentialHistogram().DataPoints().At(0).Attributes().Len() ++ got.At(4).Summary().DataPoints().At(0).Attributes().Len()+ if n != 0 {+ t.Errorf("metric datapoints kept %d peer attributes", n)+ }+}++// Without the collector's end of the connection the lookup cannot be pinned to this connection, so+// it is not made.+func TestNoServerAddressMeansNoAttribution(t *testing.T) {+ alice := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 50127}+ fakeKernel(t, alice, "1001", "alice")+ sink := new(consumertest.LogsSink)+ p, _ := NewFactory().CreateLogs(context.Background(), processortest.NewNopSettings(componentType), &Config{}, sink)+ ctx := client.NewContext(context.Background(), client.Info{Addr: alice})+ if err := p.ConsumeLogs(ctx, forgedLogs()); err != nil {+ t.Fatal(err)+ }+ if got := peerAttrs(t, sink.AllLogs()[0]); got[0] != "/" {+ t.Fatalf("peer attributes = %v, want none without the server address", got)+ }+}++// The gRPC server records the collector's end in the request's peer.+func TestServerAddressFromGRPCPeer(t *testing.T) {+ want := &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 4317}+ ctx := grpcpeer.NewContext(context.Background(), &grpcpeer.Peer{LocalAddr: want})+ if got := serverAddr(ctx); got == nil || got.String() != want.String() {+ t.Fatalf("serverAddr = %v, want %v", got, want)+ }+}diff --git a/docs/changelog/cli.mdx b/docs/changelog/cli.mdxindex 4596ea02..f9ff2ced 100644--- a/docs/changelog/cli.mdx+++ b/docs/changelog/cli.mdx@@ -11,7 +11,7 @@ Stay up to date with the latest changes to the Agent Beacon CLI.## Unreleased-- **OpenTelemetry events name the person who sent them on Linux** · On a system install every account exports to the same local collector, so prompts and token usage from Claude Code, Codex and the browser extension were recorded, and forwarded to SIEMs such as Microsoft Sentinel, under the collector's account, `root`, whoever typed them. The Linux collector now records each OpenTelemetry export under the account that owns the connection it arrived on, as the kernel reports it, over gRPC and HTTP alike, and ignores any account a sender names for itself. A sender that cannot be identified (it has already disconnected) keeps the collector's account, as before. Beacon's OTLP forward to another backend is unchanged. macOS and Windows are unchanged. Run `beacon endpoint repair --system` (package upgrades do this) to render the new collector config; the collector from the same release is required.+- **OpenTelemetry events name the person who sent them on Linux** · On a system install every account exports to the same local collector, so prompts and token usage from Claude Code, Codex and the browser extension were recorded, and forwarded to SIEMs such as Microsoft Sentinel, under the collector's account, `root`, whoever typed them. The Linux collector now records each OpenTelemetry export under the account that owns the connection it arrived on, as the kernel reports it, over gRPC and HTTP alike, and ignores any account a sender names for itself, on the resource or on a record, span or datapoint. An export whose sender cannot be identified (it disconnected before the collector looked) carries no `user` at all, rather than the collector's `root`, so a recorded `root` always means root sent it. Beacon's OTLP forward to another backend is unchanged. macOS and Windows are unchanged. Run `beacon endpoint repair --system` (package upgrades do this) to render the new collector config; the collector from the same release is required.- **Microsoft Sentinel forwarding with one command, on macOS, Linux and Windows** · `beacon endpoint sentinel forwarder install --system` runs the Vector bundled with Beacon as a system service that sends the runtime log to Microsoft Sentinel through the Logs Ingestion API, with a Microsoft Entra app registration (certificate or client secret). The credential is stored where only root can read it, or SYSTEM and Administrators on Windows, and is never a command-line argument. Vector checks the config before anything changes. Repair and package upgrades keep the forwarder and restart it on the new Vector; uninstall removes the service. Throttling, server errors and `401`/`403` answers are retried without a limit, and a line counts as sent only once Azure has accepted it. `status` and `remove` complete the commands. The pack's Logs Ingestion DCR template no longer needs a Data Collection Endpoint. See [Vector and the Logs Ingestion API](/log-forwarding/microsoft-sentinel#vector-and-the-logs-ingestion-api).- **The Windows MSI carries Vector** · The MSI now installs Vector 0.56.0 as `vector.exe` beside `beacon.exe` in `%ProgramFiles%\Beacon\bin`, as the macOS and Linux packages already carry it, checked against the same pinned checksums. Beacon finds it there. The package only installs it; nothing runs it until a forwarder is set up. Allow it alongside the other three programs if you run AppLocker or WDAC.- **Scheduled session sync** · `beacon endpoint install|repair --session-sync` turns on a job that runs every 15 minutes and reads new records from the agent runtimes' local session stores into the runtime log. It runs the same bounded, confined sweep install already runs once, so session-store data keeps reaching the log, and any forwarder reading it, after install. Off by default; repair and upgrades keep the setting, and `--no-session-sync` removes the job. See [Scheduled session sync](/cli/endpoint-install#scheduled-session-sync).diff --git a/docs/cli/endpoint-install.mdx b/docs/cli/endpoint-install.mdxindex a29888d2..d9c5ac88 100644--- a/docs/cli/endpoint-install.mdx+++ b/docs/cli/endpoint-install.mdx@@ -316,7 +316,7 @@ What this does and does not change:- Local accounts can no longer alter or delete events already in the log, or read other users' sessions.- Every event that arrives over the socket records the account that sent it, as the operating system reports it. A user still controls their own agent, so they can still send events of their own choosing, including ones that name another session's id, but always under their own account.-- The collector's OTLP ports on `127.0.0.1` still accept telemetry from any local process, as they do in the shared mode. Agents that export over OTLP, such as Claude Code and Codex, and the browser extension use them. On Linux, events that arrive that way are recorded under the account that owns the sending connection, as the kernel reports it, in either mode; on macOS and Windows they carry the collector's own account.+- The collector's OTLP ports on `127.0.0.1` still accept telemetry from any local process, as they do in the shared mode. Agents that export over OTLP, such as Claude Code and Codex, and the browser extension use them. On Linux, events that arrive that way are recorded under the account that owns the sending connection, as the kernel reports it, in either mode (with no `user` when the sender disconnected before the collector could look); on macOS and Windows they carry the collector's own account.While the endpoint is restricted, commands that read the system log as the person running them (`beacon endpoint dashboard`, `beacon scan`, `beacon traces`, and the session lookups in `beacon share`, `beacon handoff` and `beacon git` commit linking) see it only when run as root or by a `beacon-log` member. [`beacon endpoint doctor`](/cli/endpoint-doctor#runtime_log_access) connects to the socket and reports a collector that is not answering on it, including one that was killed and left its socket behind, and any events waiting in the caller's spool. On Windows the mode works the same way with Windows mechanisms: hooks send to the named pipe `\\.\pipe\beacon-agent-ingest`, the log directory's ACL gives SYSTEM and Administrators full control, the `beacon-log` local group read and nobody else anything, events record the sender's SID and account name, and the commands run from an elevated shell (see [On Windows](/security/restricted-log-access#on-windows)). For how the mode fits with log shippers such as Microsoft Sentinel, and what it does and does not protect, see [Restricted log access](/security/restricted-log-access).diff --git a/docs/log-forwarding/microsoft-sentinel.mdx b/docs/log-forwarding/microsoft-sentinel.mdxindex fbd57aa8..0812ff4d 100644--- a/docs/log-forwarding/microsoft-sentinel.mdx+++ b/docs/log-forwarding/microsoft-sentinel.mdx@@ -232,7 +232,7 @@ One case is not retried. When Microsoft Entra ID refuses the credential itself,### Who ran each session-On a machine several people use, the person behind an event is in `RawData.user`: `uid` is the account's uid, or its SID on Windows, and `name` its name. Hook and plugin events carry it, because each hook and plugin runs as the person who ran the agent; with [restricted log access](/security/restricted-log-access) the operating system vouches for it. On Linux, events from an agent's own OpenTelemetry export, such as Claude Code's token usage and cost or the browser extension's chats, carry the account that sent them too, as the kernel reports it. On macOS and Windows they carry the collector's own account instead (such as `root`, or the machine account `COMPUTERNAME$` on Windows), because every account sends to the same local port and nothing there identifies the sender. Claude Cowork is the exception: its events name the cloud account it reports. Both kinds share the session's id, so take the person from the hook and plugin events, and keep an event's own user where its session has none:+On a machine several people use, the person behind an event is in `RawData.user`: `uid` is the account's uid, or its SID on Windows, and `name` its name. Hook and plugin events carry it, because each hook and plugin runs as the person who ran the agent; with [restricted log access](/security/restricted-log-access) the operating system vouches for it. On Linux, events from an agent's own OpenTelemetry export, such as Claude Code's token usage and cost or the browser extension's chats, carry the account that sent them too, as the kernel reports it; one whose sender disconnected before the collector could look has no `user`. On macOS and Windows they carry the collector's own account instead (such as `root`, or the machine account `COMPUTERNAME$` on Windows), because every account sends to the same local port and nothing there identifies the sender. Claude Cowork is the exception: its events name the cloud account it reports. Both kinds share the session's id, so take the person from the hook and plugin events, and keep an event's own user where its session has none:```kql title="Attribute every event to the person who ran its session"let owners = BeaconRuntime_CLdiff --git a/docs/security/restricted-log-access.mdx b/docs/security/restricted-log-access.mdxindex 7cca181b..981341f0 100644--- a/docs/security/restricted-log-access.mdx+++ b/docs/security/restricted-log-access.mdx@@ -177,7 +177,7 @@ Restricted mode protects the **log**: what has been recorded cannot be altered o- **A user still controls their own agent.** They can still send events of their own choosing through the socket, including ones that name another session's id, but every event is recorded under their own account, as the operating system reports it. They cannot write an event that claims to be someone else's.- **A user can still change their own agent settings.** The hooks and OpenTelemetry settings that send an agent's activity to Beacon live in each user's home (for example `~/.claude/settings.json`), and that user can edit them, which stops capture of their future sessions. Restricted mode does not change this. Run [`beacon endpoint doctor --system`](/cli/endpoint-doctor#console_user_configured) from your MDM to catch it: `console_user_configured` fails when the logged-in user's agents no longer point at the collector, and `sudo beacon endpoint user-config repair-installed --system` puts the settings back.-- **The OTLP ports are open to every account.** The collector's OTLP receivers on `127.0.0.1:4317` and `4318` still accept telemetry from any local process, as in the shared mode. Agents that export over OTLP, such as Claude Code and Codex, and the browser extension use them. On Linux the collector records each such event under the account that owns the connection it arrived on, as the kernel reports it, and ignores any account the sender names itself; an event whose sender has already disconnected, or cannot be told apart, keeps the collector's own account. On macOS and Windows these events carry the collector's own account (such as `root`, or the machine account `COMPUTERNAME$` on Windows), not the person who ran the agent: every account on the machine sends to the same port, and nothing there identifies the sender. Claude Cowork's name the cloud account it reports instead, which no one checks either. The person is on the same session's hook and plugin events, which share the OTLP events' `session.id` and which restricted mode does attribute; see [Microsoft Sentinel forwarding](/log-forwarding/microsoft-sentinel#who-ran-each-session) for the join.+- **The OTLP ports are open to every account.** The collector's OTLP receivers on `127.0.0.1:4317` and `4318` still accept telemetry from any local process, as in the shared mode. Agents that export over OTLP, such as Claude Code and Codex, and the browser extension use them. On Linux the collector records each such event under the account that owns the connection it arrived on, as the kernel reports it, and ignores any account the sender names itself; an event whose sender had already disconnected when the collector looked carries no `user` at all, so it is never mistaken for root's. On macOS and Windows these events carry the collector's own account (such as `root`, or the machine account `COMPUTERNAME$` on Windows), not the person who ran the agent: every account on the machine sends to the same port, and nothing there identifies the sender. Claude Cowork's name the cloud account it reports instead, which no one checks either. The person is on the same session's hook and plugin events, which share the OTLP events' `session.id` and which restricted mode does attribute; see [Microsoft Sentinel forwarding](/log-forwarding/microsoft-sentinel#who-ran-each-session) for the join.- **Root is trusted.** An administrator can read and change the log in either mode. On Windows that is SYSTEM and any elevated administrator.## Verifydiff --git a/packaging/linux/smoke-package.sh b/packaging/linux/smoke-package.shindex c6001541..1fb5b77a 100755--- a/packaging/linux/smoke-package.sh+++ b/packaging/linux/smoke-package.sh@@ -251,10 +251,11 @@ PY# OTLP is attributed to the account that sent it, not to the root collector. Every account exports to# the same loopback collector, so before this every OTLP event on a system install read as root.-# Each export below also claims to be root through the peer attributes, which must be ignored.+# Each export below also claims to be root through the peer attributes, on the resource and on the+# record, and both claims must be ignored.otlp_from() {# $1 = account, $2 = session id; a browser-extension-shaped log over OTLP/HTTP JSON.- body='{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"chatgpt_web"}},{"key":"beacon.peer.uid","value":{"stringValue":"0"}},{"key":"beacon.peer.user","value":{"stringValue":"root"}}]},"scopeLogs":[{"logRecords":[{"timeUnixNano":"1762430000000000000","body":{"stringValue":"smoke"},"attributes":[{"key":"event.name","value":{"stringValue":"user_prompt"}},{"key":"session.id","value":{"stringValue":"'"$2"'"}}]}]}]}]}'+ body='{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"chatgpt_web"}},{"key":"beacon.peer.uid","value":{"stringValue":"0"}},{"key":"beacon.peer.user","value":{"stringValue":"root"}}]},"scopeLogs":[{"logRecords":[{"timeUnixNano":"1762430000000000000","body":{"stringValue":"smoke"},"attributes":[{"key":"event.name","value":{"stringValue":"user_prompt"}},{"key":"session.id","value":{"stringValue":"'"$2"'"}},{"key":"beacon.peer.uid","value":{"stringValue":"0"}},{"key":"beacon.peer.user","value":{"stringValue":"root"}}]}]}]}]}'docker exec "$CONTAINER" su -s /bin/sh "$1" -c "curl -fsS -o /dev/null -H 'Content-Type: application/json' --data-binary '$body' http://127.0.0.1:4318/v1/logs" || {echo "OTLP export as $1 failed" >&2; exit 1; }}