Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

fix(localingest): deliver hook events from bubblewrap sandboxes, and never into one (#1286)

kyle-elliott-asymptotec42b4cd88d03
diff --git a/CLAUDE.md b/CLAUDE.md
index 528bf8e9..f157c62f 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -28,7 +28,8 @@ Do not recreate or depend on removed `asymptote` mirror trees. Keep new work foc
- Beacon is visibility-first endpoint telemetry for local AI agent runtimes, not a hosted policy service or general endpoint protection product.
- Preserve the local-only default. The public Beacon build must not require a hosted account, remote policy fetch, hosted dashboard, or external network dependency during normal hook execution, and hooks themselves never touch the network.
- `beacon login` is the account-identity channel for interactive users and is required by the interactive `beacon endpoint install` wizard. It uses browser PKCE against beacon.sh and stores the scoped CLI token only in `~/.beacon/auth/session.json` (0600 under a 0700 directory). Signing in alone never starts forwarding. User-mode `beacon endpoint connect` may send that token once to `POST /api/cli/enroll/account` to authorize minting a separate device ingest key, and `beacon mcp token create` may send it to `POST /api/cli/mcp/tokens` (scope `mcp:token:create`, listed on the sign-in page) to mint a personal MCP token that is printed once to stdout and never written to a file, and `beacon share` may send it (scope `traces:write`, also on the sign-in page) to `PUT /v1/traces/{external_id}`, `POST /v1/traces/{external_id}/events/batch` and `GET /v1/traces/{external_id}` on the Beacon API host (`https://api.beacon.sh` for a beacon.sh sign-in, other hosts unchanged, `BEACON_API_URL` overrides) to upload a shared trace the person chose to publish, as a `beacon.trace.v1` summary and its events (`cli/beacon/internal/account/share.go`; a `bcn_api_` API key from `--key` or `BEACON_API_KEY` may stand in for it, precedence `--key`, then a current sign-in carrying the scope, then the environment). These are explicit, user-initiated requests with redirects refused. Hooks, plugins and background jobs never make them, with one exception: the git pre-push hook, only when the person opted in with `beacon git setup --share-sessions` (public needs confirmation), may make the three share requests during a push the person started, within the hook's budget and never failing the push. Never copy the account token into endpoint config, Vector, hook environments, logs, or command output. System/root/package/MDM/CI/piped-stdin/dry-run installs stay noninteractive and account-free.
-- System log access (`beacon endpoint install|repair --system --log-access shared|restricted`; docs in `docs/cli/endpoint-install.mdx#restricted-log-access`). Shared, the CLI default, is the system log as it always was: hooks append to `/var/log/beacon-agent/runtime.jsonl` directly, so it is `0666`. Restricted is opt-in for the CLI and the default for a first package install on every platform (see below; on Linux `packaging/linux/install-endpoint.sh` passes `--log-access restricted` only when neither the system `config.json` nor the runtime log exists, `/proc/1/comm` is systemd and `BEACON_SERVICE` is unset, `auto` or `systemd`; otherwise it passes nothing and the endpoint keeps its mode; `packaging/linux/beacon-settings.sh`, sourced by that script and by the postinstall (which takes only `BEACON_ENDPOINT_HARNESSES`, through `beacon_settings_harnesses` in a subshell, because its user-config step re-runs the CLI as each logged-in user with its environment and must not hand them a Splunk HEC token from the file), reads the `BEACON_*` install settings, never sourcing the file, from a root-owned, not group/other-writable `/etc/default/beacon` or `/etc/sysconfig/beacon` in shell syntax with trailing comments, skipping a malformed value with a warning naming only the key, the last assignment of a key winning within the file, the environment and `install-endpoint.sh`'s positional arguments winning over it; the postinstall prints the `usermod -aG beacon-log` hint for a restricted log; `packaging/linux/test-endpoint-scripts.sh` checks all of it). Restricted makes the root collector the only writer: its `beacon_ingest` extension (`collector-builder/exporter/beaconjsonexporter/beaconingestextension`, appending through the exporter's own `internal/runtimelog`) serves `/var/run/beacon-agent/ingest.sock`, and the hook adapter and the CLI writer, as any account but root writing the system log, hand finished lines to it through `pkg/asymptoteobserve/localingest` (`Deliverer`). The protocol is one batch per connection (`BEACON-INGEST/1`, a header naming the log, one event per line, one JSON reply saying how far it got); the server refuses a batch for any log but its own, identifies the sender only by kernel peer credentials (`SO_PEERCRED`, `LOCAL_PEERCRED`), overwrites the event's `user` with that uid and its resolved name and always derives `event.id` itself (`StampLine` re-encodes, so a sender cannot split or forge lines or reuse another event's id), and bounds connections overall and per uid, batch size, each account's bytes per second, and its own time per connection (shorter than the client's, so it reports how far it got instead of being resent). The client sends only to a socket that root owns in a directory nobody else can write (group-writable ancestors only for gid 0/1, which macOS's `/private/var/run` needs). `Deliverer.Write` sends through the socket when it answers and otherwise appends directly as before; only a direct append refused for permission (a restricted log whose collector is down) spools the line, to the account's private spool for that log (`~/.beacon/endpoint/spool/<log digest>`, 0600 under 0700), and the next line that gets through either way drains it. Root writers and per-user logs always write directly. The mode is `config.json` `log_access`, carried through repair and package upgrades (`BEACON_LOG_ACCESS` in the install scripts). A first install from any package (`.deb`/`.rpm` on systemd hosts, the macOS `.pkg`, the MSI unless `BEACON_SERVICE` names the supervised collector; no system config and no kept log) defaults to restricted in its install script, and adds to `beacon-log` (`--log-readers`, `loggroup.AddMember`: gpasswd/addgroup, dseditgroup, net.exe; add-only, failures reported not fatal; `validAccount` also admits spaces, an apostrophe and non-ASCII letters on Windows, because OOBE names local accounts from what the person typed) exactly `BEACON_LOG_READERS` when set, even empty, else only a person who installed interactively (Linux `SUDO_USER` with a controlling terminal (`/dev/tty`, not stdin, which rpm scriptlets get as a pipe) and not `DEBIAN_FRONTEND=noninteractive`, because Ansible `become` and Packer sudo set `SUDO_USER` too; macOS without `COMMAND_LINE_INSTALL` and with an `Installer` process of the console user whose lsof `txt` executable is Apple's `/System/Library/CoreServices/Installer.app`, never a bare process name; Windows MSI `UILevel` >= 4 via the custom action's `-UiLevel`), never an unattended fleet install. In both shell settings parsers an environment variable set to an empty string counts as unset (except `BEACON_LOG_READERS`, where empty means nobody), and the macOS Fleet wrapper (`packaging/macos/fleet/scripts/install.sh`) passes only arguments it was given, so neither hides the settings file or managed preferences. Fleet tooling sets the `BEACON_*` install settings where it writes settings: `/etc/default/beacon`/`/etc/sysconfig/beacon` (`packaging/linux/beacon-settings.sh`), managed or local preferences for `com.beacon.endpoint` (`packaging/macos/beacon-settings.sh`), `HKLM\SOFTWARE\Policies\Beacon\Endpoint` (`install-endpoint.ps1`); each read only when root/admin-controlled, environment wins; docs in `docs/security/restricted-log-access.mdx#package-installs`. Install creates the `beacon-log` group (`internal/endpoint/loggroup`, getent/groupadd, dscl/dseditgroup) and `writer.RestrictSystemLog` makes the directory `root:beacon-log 02750`, the log and archives `0640` and the lock root-only `0600`, after writing the root-owned `.beacon-log-access` marker beside the log; a shared log is not narrowed in place but copied to fresh root-created files and renamed over, under its lock (bounded wait), because a descriptor opened while it was 0666 survives any chmod. A failed install restores the previous mode in its rollback (`logAccessRestorer`), or, when there was no log before, removes the log, lock and marker it created, because every package script reads a log without a system config as one kept by `--keep-logs`, which would turn the retry of a failed first install into an upgrade; and a shared install removes a dead socket. The marker is what tells every root writer (`asymptoteobserve.RuntimeLogModes`) to keep the restricted modes. The OTLP receivers on loopback are not covered by the socket: any local process can still send OTLP. On Linux, macOS and Windows those events are attributed to the account that owns the sending socket (peer attribution, below), which names the sender but does not stop an account sending what it likes under its own name, so never claim more than that. Doctor's `runtime_log_access` covers the marker, the socket and pending spool. Unix sockets are local IPC, not a network channel, so the hooks-never-touch-the-network rule holds. On Windows the same mode runs over the named pipe `\\.\pipe\beacon-agent-ingest` (`localingest` `transport_windows.go`, go-winio): the collector creates it as the first instance, and when the name is taken (any account may create one) it reports a recoverable error and retries every few seconds rather than failing, so the rest of the collector stays up; it rejects remote clients and its DACL denies NETWORK and withholds `FILE_CREATE_PIPE_INSTANCE` from ordinary accounts, so it is not a network channel either; a client connects at identification level and sends only to a pipe owned by SYSTEM or Administrators; the server counts a connection against its account from the client process's token before reading and stamps the SID it reads by `ImpersonateNamedPipeClient` after the preamble (they must agree), with `Peer.UID` a string (decimal uid on Unix, SID on Windows) and the name from `LookupAccountSid`, bounded. Privileged writers there are SYSTEM and tokens with Administrators enabled (`CheckTokenMembership`; elevation for another group is not enough). The directory gets `sysacl.RestrictedLogDirDACL` (protected: SYSTEM and Administrators full, `beacon-log` local group read, nothing else; the lock SYSTEM and Administrators only) and everything below it is reset, because every account may bypass traverse checking; `writer.RestrictSystemLog` replaces every file directly in the directory (the log family under the lock; files outside it with another hard link are skipped) with a copy created with its final owner and DACL (`sysacl.CreateRestrictedFile`) and a POSIX-semantics rename, fails naming any file held open without sharing delete and then puts a log that was shared back to shared, and writes the marker last (on Windows it decides nothing for root writers, so a failed or killed run is not mistaken for a finished one). The marker counts there when SYSTEM or Administrators own it and no other account can write it. `loggroup` creates the local group with `net.exe` and looks it up qualified with the computer name. Doctor reads the restricted ACL (`writer.CheckRestrictedSystemLogACL`). `packaging/windows/smoke-restricted-log.ps1` runs in `windows-package-smoke` against the installed MSI with a real ordinary account. Hook and writer tests point `logging.EndpointIngest`/`writer.Ingest` at sockets of their own.
+- System log access (`beacon endpoint install|repair --system --log-access shared|restricted`; docs in `docs/cli/endpoint-install.mdx#restricted-log-access`). Shared, the CLI default, is the system log as it always was: hooks append to `/var/log/beacon-agent/runtime.jsonl` directly, so it is `0666`. Restricted is opt-in for the CLI and the default for a first package install on every platform (see below; on Linux `packaging/linux/install-endpoint.sh` passes `--log-access restricted` only when neither the system `config.json` nor the runtime log exists, `/proc/1/comm` is systemd and `BEACON_SERVICE` is unset, `auto` or `systemd`; otherwise it passes nothing and the endpoint keeps its mode; `packaging/linux/beacon-settings.sh`, sourced by that script and by the postinstall (which takes only `BEACON_ENDPOINT_HARNESSES`, through `beacon_settings_harnesses` in a subshell, because its user-config step re-runs the CLI as each logged-in user with its environment and must not hand them a Splunk HEC token from the file), reads the `BEACON_*` install settings, never sourcing the file, from a root-owned, not group/other-writable `/etc/default/beacon` or `/etc/sysconfig/beacon` in shell syntax with trailing comments, skipping a malformed value with a warning naming only the key, the last assignment of a key winning within the file, the environment and `install-endpoint.sh`'s positional arguments winning over it; the postinstall prints the `usermod -aG beacon-log` hint for a restricted log; `packaging/linux/test-endpoint-scripts.sh` checks all of it). Restricted makes the root collector the only writer: its `beacon_ingest` extension (`collector-builder/exporter/beaconjsonexporter/beaconingestextension`, appending through the exporter's own `internal/runtimelog`) serves `/var/run/beacon-agent/ingest.sock`, and the hook adapter and the CLI writer, as any account but root writing the system log, hand finished lines to it through `pkg/asymptoteobserve/localingest` (`Deliverer`). The protocol is one batch per connection (`BEACON-INGEST/1`, a header naming the log, one event per line, one JSON reply saying how far it got); the server refuses a batch for any log but its own, identifies the sender only by kernel peer credentials (`SO_PEERCRED`, `LOCAL_PEERCRED`), overwrites the event's `user` with that uid and its resolved name and always derives `event.id` itself (`StampLine` re-encodes, so a sender cannot split or forge lines or reuse another event's id), and bounds connections overall and per uid, batch size, each account's bytes per second, and its own time per connection (shorter than the client's, so it reports how far it got instead of being resent). The client sends only to a socket that root owns in a directory nobody else can write, under ancestors owned by root or by the sending account (a sandbox without a user namespace, such as bubblewrap, assembles its root on a tmpfs that account owns, so a bind-mounted `/var/run/beacon-agent` works there; group-writable ancestors only for gid 0/1, which macOS's `/private/var/run` needs), and only once the connected socket's own peer credentials name root as the server (`checkServer`), so no swap of the path between the check and the dial reaches another listener. Sandboxes with their own user namespace are out of scope. `Deliverer.Write` sends through the socket when it answers and otherwise appends directly as before; only a direct append that cannot reach the log (refused for permission, its directory missing, or a read-only mount: a restricted log whose collector is down, or a sandbox that leaves the log out) spools the line, to the account's private spool for that log (`~/.beacon/endpoint/spool/<log digest>`, 0600 under 0700, only in a home that exists), and the next line that gets through either way drains it. Root writers and per-user logs always write directly. No ordinary account creates the system log's directory, and no writer creates a missing home to hold a log (`asymptoteobserve.RuntimeLogDirCreatable`) or the hook's state directories, so a sandbox whose root filesystem is its own gets a `NOT recorded` explanation on stderr instead of events written into its tmpfs. The mode is `config.json` `log_access`, carried through repair and package upgrades (`BEACON_LOG_ACCESS` in the install scripts). A first install from any package (`.deb`/`.rpm` on systemd hosts, the macOS `.pkg`, the MSI unless `BEACON_SERVICE` names the supervised collector; no system config and no kept log) defaults to restricted in its install script, and adds to `beacon-log` (`--log-readers`, `loggroup.AddMember`: gpasswd/addgroup, dseditgroup, net.exe; add-only, failures reported not fatal; `validAccount` also admits spaces, an apostrophe and non-ASCII letters on Windows, because OOBE names local accounts from what the person typed) exactly `BEACON_LOG_READERS` when set, even empty, else only a person who installed interactively (Linux `SUDO_USER` with a controlling terminal (`/dev/tty`, not stdin, which rpm scriptlets get as a pipe) and not `DEBIAN_FRONTEND=noninteractive`, because Ansible `become` and Packer sudo set `SUDO_USER` too; macOS without `COMMAND_LINE_INSTALL` and with an `Installer` process of the console user whose lsof `txt` executable is Apple's `/System/Library/CoreServices/Installer.app`, never a bare process name; Windows MSI `UILevel` >= 4 via the custom action's `-UiLevel`), never an unattended fleet install. In both shell settings parsers an environment variable set to an empty string counts as unset (except `BEACON_LOG_READERS`, where empty means nobody), and the macOS Fleet wrapper (`packaging/macos/fleet/scripts/install.sh`) passes only arguments it was given, so neither hides the settings file or managed preferences. Fleet tooling sets the `BEACON_*` install settings where it writes settings: `/etc/default/beacon`/`/etc/sysconfig/beacon` (`packaging/linux/beacon-settings.sh`), managed or local preferences for `com.beacon.endpoint` (`packaging/macos/beacon-settings.sh`), `HKLM\SOFTWARE\Policies\Beacon\Endpoint` (`install-endpoint.ps1`); each read only when root/admin-controlled, environment wins; docs in `docs/security/restricted-log-access.mdx#package-installs`. Install creates the `beacon-log` group (`internal/endpoint/loggroup`, getent/groupadd, dscl/dseditgroup) and `writer.RestrictSystemLog` makes the directory `root:beacon-log 02750`, the log and archives `0640` and the lock root-only `0600`, after writing the root-owned `.beacon-log-access` marker beside the log; a shared log is not narrowed in place but copied to fresh root-created files and renamed over, under its lock (bounded wait), because a descriptor opened while it was 0666 survives any chmod. A failed install restores the previous mode in its rollback (`logAccessRestorer`), or, when there was no log before, removes the log, lock and marker it created, because every package script reads a log without a system config as one kept by `--keep-logs`, which would turn the retry of a failed first install into an upgrade; and a shared install removes a dead socket. The marker is what tells every root writer (`asymptoteobserve.RuntimeLogModes`) to keep the restricted modes. The OTLP receivers on loopback are not covered by the socket: any local process can still send OTLP. On Linux, macOS and Windows those events are attributed to the account that owns the sending socket (peer attribution, below), which names the sender but does not stop an account sending what it likes under its own name, so never claim more than that. Doctor's `runtime_log_access` covers the marker, the socket and pending spool. Unix sockets are local IPC, not a network channel, so the hooks-never-touch-the-network rule holds. On Windows the same mode runs over the named pipe `\\.\pipe\beacon-agent-ingest` (`localingest` `transport_windows.go`, go-winio): the collector creates it as the first instance, and when the name is taken (any account may create one) it reports a recoverable error and retries every few seconds rather than failing, so the rest of the collector stays up; it rejects remote clients and its DACL denies NETWORK and withholds `FILE_CREATE_PIPE_INSTANCE` from ordinary accounts, so it is not a network channel either; a client connects at identification level and sends only to a pipe owned by SYSTEM or Administrators; the server counts a connection against its account from the client process's token before reading and stamps the SID it reads by `ImpersonateNamedPipeClient` after the preamble (they must agree), with `Peer.UID` a string (decimal uid on Unix, SID on Windows) and the name from `LookupAccountSid`, bounded. Privileged writers there are SYSTEM and tokens with Administrators enabled (`CheckTokenMembership`; elevation for another group is not enough). The directory gets `sysacl.RestrictedLogDirDACL` (protected: SYSTEM and Administrators full, `beacon-log` local group read, nothing else; the lock SYSTEM and Administrators only) and everything below it is reset, because every account may bypass traverse checking; `writer.RestrictSystemLog` replaces every file directly in the directory (the log family under the lock; files outside it with another hard link are skipped) with a copy created with its final owner and DACL (`sysacl.CreateRestrictedFile`) and a POSIX-semantics rename, fails naming any file held open without sharing delete and then puts a log that was shared back to shared, and writes the marker last (on Windows it decides nothing for root writers, so a failed or killed run is not mistaken for a finished one). The marker counts there when SYSTEM or Administrators own it and no other account can write it. `loggroup` creates the local group with `net.exe` and looks it up qualified with the computer name. Doctor reads the restricted ACL (`writer.CheckRestrictedSystemLogACL`). `packaging/windows/smoke-restricted-log.ps1` runs in `windows-package-smoke` against the installed MSI with a real ordinary account. Hook and writer tests point `logging.EndpointIngest`/`writer.Ingest` at sockets of their own.
+ - Sandbox roots (Linux): no unprivileged writer puts the log directory (`RuntimeLogDirCreatable`) or the account spool (`DefaultSpoolDir` returns "") on the throwaway root of a sandbox it assembled (`asymptoteobserve.OnSandboxRoot`: `/` a tmpfs owned by this non-root account, and the path's nearest existing directory on that filesystem). Both conditions matter: a persistent account-owned `/` (a rootfs bound at `/`, an Apptainer sandbox) keeps its writes. The home-exists checks alone are not enough, because a launcher that binds a project under the home makes bwrap create the home as an empty mount point on its tmpfs. Hook state and the active-session record may still go there: they are per-session dedupe, and refusing them re-records prompts on every tool call. What the sandbox bind-mounts is another filesystem and is written as usual: a mounted `~/.beacon/endpoint/spool` spools for real, and a workspace is where #1284's workspace spool stages.
- Beacon Cloud forwarding (`cli/beacon/internal/endpoint/asymptote`, `beacon endpoint connect`, `beacon endpoint asymptote`) is Vector-shaped: Beacon writes local JSONL, Vector ships runtime lines only from the connection point forward and reads inventory from the beginning to establish the endpoint's current baseline, plus the one bounded exception of the session backfill (below), and the `bcn_device_*` key stays only in `<BaseDir>/asymptote/vector-secrets.json` via Vector's secret backend. The onboarding wizard preselects Beacon Cloud, and confirming it connects this endpoint in the same command once the local install has succeeded; an endpoint that is already enrolled is never re-enrolled, because enrollment rotates the device key. Standard mode ships locally sanitized retained content; metadata-only uses local Vector transforms to remove prompts, responses, reasoning, tool arguments/results, tool descriptions/definitions, retrieval queries/documents, evaluation explanations, agent descriptions, stop sequences, approval/policy reasons, command output, raw fields, diffs, inventory content, and MCP definitions before buffering/upload. Every text-shaped event field must be classified in `metadata_fields_test.go` (`metadataOnlyRemoved`, each with its own `del()` in `pack/metadata-transforms.toml`, or `metadataOnlyKept`); a new field fails that test until it is, and `BEACON_TEST_VECTOR_BIN` runs the transform on an event with every field set. `vector.toml` is not rendered only at connect: `asymptote.ReconcileForwarder` re-renders it in place for a connected endpoint from the current pack with the enrollment's ingest URL and recorded privacy mode, validates it with the forwarder's own Vector exactly as connect does, atomically replaces it, and, when run from install, repair, the package upgrade or the hidden reconcile the macOS postinstall runs (`ReconcileOptions.Restart`), restarts a running forwarder even when nothing changed (after `vector validate` of the current config with the forwarder's own Vector, and never after a warning, so a refused config or a missing Vector leaves the running forwarder alone), as the Sentinel reconcile does: switching to restricted log access replaces the runtime log with a copy, and Vector, which recognizes a file by its first line, keeps tailing the replaced one until it restarts; an upgrade may also have installed a new Vector. A stopped one is never started and reads the new config when it next starts. The `beacon_runtime` source reads `runtime.jsonl.1` too, as the Sentinel pack does, so lines written to a replaced log before a restart are found again at the next rotation instead of lost. It never enrolls, reads or rewrites the secrets file, makes a network request, or touches `vector-data/` (checkpoints stay, so pack changes must keep source ids and fingerprinting stable), and a rejected config is a warning that keeps the old one; it is a no-op when not connected, after an incomplete connect, without an installed unit, or where `ForwarderManager` is unsupported (supervised mode). `lifecycle.Install` runs it last when services are started (so install, repair and the Linux package upgrade), repair's teardown keeps the forwarder (`UninstallOptions.KeepManagedIngest`) instead of disconnecting it, and the macOS postinstall, which installs with `--no-start`, runs the hidden `beacon endpoint asymptote reconcile --system`. Keep the ingest URL `https://`, the wire contract aligned with `beacon-ingest` (`/v1/ingest/{runtime,inventory}` and `/v1/ingest/health`), and do not add another endpoint network channel. The cloud-sandbox shuttle is the bounded exception: only with `BEACON_ORIGIN=cloud`, it can upload new runtime lines to GCS, S3, or the same Beacon ingest contract because no Vector/service manager/browser/persistent disk exists there.
- Session backfill (`cli/beacon/cmd/endpoint_backfill.go`; docs in `docs/cli/endpoint-install.mdx#session-backfill` and `docs/cli/endpoint-connect.mdx#session-backfill`): after every successful install (user or system mode), every session-store collector runs once with `CollectOptions.Since` (30 days; `sessionwindow.Recent` keeps those sessions, newest first, and never touches the cursors of the rest) and a shared `writer.Budget` (24 MiB, at most 12 MiB per runtime through `Budget.Sub`; a refused append returns `writer.ErrBudgetSpent`, the collector stops with cursors past only what was written, and a manual sync reads the rest). A new session-store collector takes both fields, reads its store only through `internal/sessionfs` (a test parses every collector package to enforce it), and joins `endpointBackfillRunners` and that test's list. Every connect, first or re-connect, user or system, runs the same sweep before enrollment (skipped when install just ran it) and, before the forwarder starts, `asymptote.StageBackfill` copies the window's `harness.collection_method=poll` lines into `cloud-backfill.jsonl` beside the runtime log, which the `beacon_backfill` Vector source (fingerprinted `device_and_inode`, so each atomically restaged copy is a new file) reads from the beginning into the runtime sink through the same privacy transform. Live hook/OTLP/plugin history before the connection point stays local. Re-connects restage, so a resend of already-shipped lines is expected and rests on the deterministic `event.id`; a privacy-mode change clears the checkpoints and the restage renames the new-mode copy over the old one (kept until then so the copy is always a new inode; deleted instead when nothing restages), and `disconnect` removes it. System mode runs the collectors under `withUserHome` for the active console user (`activeConsoleUser`), with cursors under the system base directory, and reports a skip when nobody is logged in; root is reading a home the console user controls, so the sweep runs under `sessionfs.Confine` (an `os.Root` on that home, files owned by the home's owner only, non-blocking opens that refuse FIFOs/devices, SQLite read from a private snapshot under `<SystemBaseDir>/state/backfill-snapshots`, retried until the database size/change counter and WAL header are unchanged across the copy, and refused past `sessionfs.MaxSnapshotBytes` (512 MiB with its WAL) with the runtime's sync command named in the report), and refused paths read as absent and are listed in the report. User-mode sync and backfill stay unconfined. The wizard's confirm screen names the upload (`WizardOptions.BackfillDays`). Dry-run, `--no-backfill` and `BEACON_BACKFILL=0` skip all of it; the cmd tests run with `BEACON_BACKFILL=0` set in `TestMain`.
- `beacon endpoint install` has a full-screen onboarding wizard on interactive user TTYs. It signs in, preselects Beacon Cloud, requires an explicit choice to opt out to Local, and asks Managed users to choose Standard or Metadata-only privacy. Signing in alone still moves no telemetry. Confirming Beacon Cloud does: the confirm screen states that it installs Beacon and starts forwarding, and names what the selected privacy mode sends, so that screen is the consent point. The managed destination is recorded only after enrollment succeeds, so a failed enrollment leaves a working local install, prints `beacon endpoint connect` as the retry, and re-offers the question on the next install. The onboarding record is written only after `lifecycle.Install` returns, because a profile written first left a failed install looking onboarded and silently skipped the wizard on the retry. Explicit `install --connect` is unchanged. The wizard is skipped for system/root/CI/piped-stdin/dry-run or `BEACON_ONBOARDING=0`; those unattended paths must never gain an account or browser dependency. Legacy `BEACON_ONBOARDING_EMAIL`/`USAGE` submissions remain only for fleet compatibility. After the destination, the wizard shows an agent tools screen; every row is preselected (opt-out, space toggles). Beacon Cloud MCP is offered only with Beacon Cloud and runs the `beacon mcp connect` flow (`connectCloudMCP` with `yes`, OAuth, detected harnesses) only after the endpoint is connected (or already was), via `finishCloudSetup`; a failed enrollment skips it and names the retry. The Beacon Agent Skills are offered for either destination and installed user-wide into `~/.agents/skills` and, when Claude Code is set up, `~/.claude/skills` (`cli/beacon/internal/agentskills`, `beacon skills install|uninstall`; a `.beacon-managed` marker decides what Beacon may replace or remove; install and repair refresh only copies Beacon already wrote). Both run after `lifecycle.Install` returns and warn rather than fail. The answers are recorded as `onboarding.cloud_mcp`/`agent_skills`. Implementation lives in `cli/beacon/internal/onboarding`, `cli/beacon/cmd/endpoint_onboarding.go`, `cli/beacon/cmd/endpoint_install.go` and `cli/beacon/cmd/agent_tools.go`; keep `docs/cli/endpoint-onboarding.mdx` and `docs/cli/skills.mdx` in sync.
diff --git a/cli/beacon-hooks/internal/config/config.go b/cli/beacon-hooks/internal/config/config.go
index f3ecfe2b..b9a50c11 100644
--- a/cli/beacon-hooks/internal/config/config.go
+++ b/cli/beacon-hooks/internal/config/config.go
@@ -142,7 +142,7 @@ func GetSessionLogFile(platform, sessionID string) string {
// EnsureSessionLogDir creates the session logs directory if it doesn't exist
func EnsureSessionLogDir(platform string) error {
- return os.MkdirAll(GetSessionLogDir(platform), 0755)
+ return mkdirInHome(GetSessionLogDir(platform))
}
// RotateLogIfNeededForPlatform rotates the platform-specific log file if it exceeds LogMaxSizeBytes.
@@ -203,5 +203,19 @@ func removeOverflowArchives(path string, archives int) error {
// EnsureStateDir ensures the state directory for the given platform exists
func EnsureStateDir(platform string) error {
- return os.MkdirAll(GetStateDir(platform), 0755)
+ return mkdirInHome(GetStateDir(platform))
+}
+
+// mkdirInHome creates dir, which lies under ~/.beacon, but never the home directory itself. A
+// sandbox that assembles its own root filesystem, which the account may write, often leaves the
+// home out; a home a hook created there would vanish with the sandbox, taking whatever was written
+// into it, and would make the account's event spool look available when it is not. State in a
+// home that does exist there -- the mount point of a project bound beneath it -- is kept: it is
+// per-session dedupe state, and refusing it would record the session's prompt again on every
+// tool call. The account spool and the log check OnSandboxRoot themselves.
+func mkdirInHome(dir string) error {
+ if _, err := os.Stat(filepath.Dir(BeaconDir)); err != nil {
+ return err
+ }
+ return os.MkdirAll(dir, 0755)
}
diff --git a/cli/beacon-hooks/internal/config/config_test.go b/cli/beacon-hooks/internal/config/config_test.go
index 159f6ffc..71dde3ee 100644
--- a/cli/beacon-hooks/internal/config/config_test.go
+++ b/cli/beacon-hooks/internal/config/config_test.go
@@ -2,6 +2,7 @@ package config
import (
"os"
+ "path/filepath"
"testing"
)
@@ -62,3 +63,31 @@ func TestRotateLogIfNeededForPlatformArchivesExistingLog(t *testing.T) {
t.Fatalf(".2 archive = %q err=%v, want older log", string(data), err)
}
}
+
+// A sandbox that assembles its own root filesystem often leaves the home out. The hook's state
+// directories must not conjure one there: it would vanish with the sandbox, and make the account's
+// event spool beneath it look available when it is not.
+func TestStateDirsNeverCreateAMissingHome(t *testing.T) {
+ home := filepath.Join(t.TempDir(), "missing-home")
+ oldBeaconDir, oldCursorDir := BeaconDir, CursorDir
+ BeaconDir = filepath.Join(home, ".beacon")
+ CursorDir = filepath.Join(BeaconDir, "cursor")
+ t.Cleanup(func() { BeaconDir, CursorDir = oldBeaconDir, oldCursorDir })
+
+ if err := EnsureStateDir("cursor"); err == nil {
+ t.Fatal("EnsureStateDir succeeded under a home that does not exist")
+ }
+ if err := EnsureSessionLogDir("cursor"); err == nil {
+ t.Fatal("EnsureSessionLogDir succeeded under a home that does not exist")
+ }
+ if _, err := os.Stat(home); !os.IsNotExist(err) {
+ t.Fatalf("the home was created: %v", err)
+ }
+
+ if err := os.Mkdir(home, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ if err := EnsureSessionLogDir("cursor"); err != nil {
+ t.Fatalf("EnsureSessionLogDir under an existing home = %v", err)
+ }
+}
diff --git a/cli/beacon-hooks/internal/logging/logging.go b/cli/beacon-hooks/internal/logging/logging.go
index 42ad3f15..e80d69f1 100644
--- a/cli/beacon-hooks/internal/logging/logging.go
+++ b/cli/beacon-hooks/internal/logging/logging.go
@@ -549,8 +549,10 @@ var writeEndpointJSON = writeEndpointJSONFile
func writeEndpointJSONFile(path string, event map[string]interface{}) error {
_, dirMode, _ := asymptoteobserve.RuntimeLogModes(path)
- if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
- return err
+ if asymptoteobserve.RuntimeLogDirCreatable(path) {
+ if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
+ return err
+ }
}
data, err := encodeEndpointLine(event)
if err != nil {
@@ -655,8 +657,10 @@ func sanitizeEndpointSlice(input []interface{}) []interface{} {
// itself is asymptoteobserve.RotateRuntimeLog for all three.
func appendEndpointJSONL(path string, line []byte, rotateBytes int64, rotateArchives int) error {
_, dirMode, _ := asymptoteobserve.RuntimeLogModes(path)
- if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
- return err
+ if asymptoteobserve.RuntimeLogDirCreatable(path) {
+ if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
+ return err
+ }
}
lock, err := openEndpointRuntimeFileAs(path+".lock", path, os.O_CREATE|os.O_RDWR)
if err != nil {
diff --git a/cli/beacon/internal/endpoint/writer/writer.go b/cli/beacon/internal/endpoint/writer/writer.go
index b11a7012..4d954725 100644
--- a/cli/beacon/internal/endpoint/writer/writer.go
+++ b/cli/beacon/internal/endpoint/writer/writer.go
@@ -257,8 +257,10 @@ func SanitizeEvent(event schema.Event, maxBytes int) schema.Event {
// itself is asymptoteobserve.RotateRuntimeLog for all three.
func appendJSONL(path string, line []byte, rotateBytes int64, rotateArchives int, guard *RetentionGuard, budget *Budget) (err error) {
_, dirMode, _ := asymptoteobserve.RuntimeLogModes(path)
- if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
- return err
+ if asymptoteobserve.RuntimeLogDirCreatable(path) {
+ if err := os.MkdirAll(filepath.Dir(path), dirMode); err != nil {
+ return err
+ }
}
lock, err := openRuntimeFileAs(path+".lock", path, os.O_CREATE|os.O_RDWR)
if err != nil {
diff --git a/docs/security/restricted-log-access.mdx b/docs/security/restricted-log-access.mdx
index 52e95b37..450a24ec 100644
--- a/docs/security/restricted-log-access.mdx
+++ b/docs/security/restricted-log-access.mdx
@@ -41,6 +41,20 @@ The event format and every field stay the same, so a parser, DCR transform or SI
If the collector is stopped or restarting, a hook writes its events to a spool in that user's home, `~/.beacon/endpoint/spool` (`0700` directory, `0600` files), and returns at once, so the agent is never held up. The next event that user's hooks send delivers the spooled ones first. Each spool is capped at 8 MiB; events past that are dropped and counted rather than filling the disk. [`beacon endpoint doctor`](/cli/endpoint-doctor#runtime_log_access) reports a collector that is not answering on the socket and any events waiting in the caller's spool.
+### Agents in a sandbox
+
+A sandbox launcher such as bubblewrap usually leaves both the socket and the log out of the sandbox. To keep live hook events from such an agent, with the sending account recorded by the operating system, mount the collector's socket directory into the sandbox at the same path:
+
+```bash title="bubblewrap: give sandboxed hooks the collector"
+bwrap ... --bind /var/run/beacon-agent /var/run/beacon-agent ...
+```
+
+This works for a sandbox without its own user namespace, which is how bubblewrap runs when installed setuid. Such a sandbox builds its root filesystem on a tmpfs owned by the account it runs as, so `/`, `/var` and `/var/run` belong to that account inside it. Hooks accept parent directories owned by root or by their own account. Before sending anything, they also check that the process serving the socket is root. Sandboxes with their own user namespace are not covered.
+
+To also hold events while the collector restarts, mount the account's spool too: `--bind ~/.beacon/endpoint/spool ~/.beacon/endpoint/spool`, with the directory created beforehand (`0700`). The next event that gets through, from inside the sandbox or outside it, delivers what was held.
+
+A hook that can reach neither the socket nor a spool does not record the event. It prints `beacon-hooks: this event was NOT recorded` and the reason to the hook's stderr, and the agent carries on. On Linux, inside a sandbox whose root filesystem is a tmpfs owned by the account (bubblewrap's), a hook never puts the log or the spool on that filesystem; only directories the sandbox mounts take them. That holds even when a project bound under the home makes the home exist inside the sandbox, so events never go to files that disappear with the sandbox.
+
## Turn it on
<Note>
diff --git a/pkg/asymptoteobserve/localingest/client.go b/pkg/asymptoteobserve/localingest/client.go
index f3652970..f8b4429e 100644
--- a/pkg/asymptoteobserve/localingest/client.go
+++ b/pkg/asymptoteobserve/localingest/client.go
@@ -22,8 +22,8 @@ type Client struct {
SocketPath string
// Timeout bounds each connection, from dial to reply. Zero means DefaultClientTimeout.
Timeout time.Duration
- // TrustedOwner is the uid the socket and its directory must belong to on Unix. Zero, the
- // default, is root; tests set their own uid.
+ // TrustedOwner is the uid that must own the socket and its directory, and serve it, on Unix.
+ // Zero, the default, is root; tests set their own uid.
TrustedOwner int
// TrustedSID is, on Windows, one more account the pipe may belong to besides SYSTEM and
// Administrators, the only creators trusted otherwise. Empty in production; tests that serve a
@@ -156,7 +156,8 @@ func (c Client) sendBatch(lines [][]byte) (Response, error) {
}
// ErrUntrustedSocket is returned when something exists at the socket path but could have been put
-// there by someone other than root, or on Windows a pipe of that name was created by someone other
-// than SYSTEM or Administrators. A writer that sees it does not send there, because a socket or
-// pipe a user planted would receive every other user's events.
+// there by someone other than root, or is served by a process that is not root, or on Windows a pipe
+// of that name was created by someone other than SYSTEM or Administrators. A writer that sees it
+// does not send there, because a socket or pipe a user planted would receive every other user's
+// events.
var ErrUntrustedSocket = errors.New("untrusted ingest socket")
diff --git a/pkg/asymptoteobserve/localingest/deliver.go b/pkg/asymptoteobserve/localingest/deliver.go
index 5d460ae8..8f6ef183 100644
--- a/pkg/asymptoteobserve/localingest/deliver.go
+++ b/pkg/asymptoteobserve/localingest/deliver.go
@@ -9,6 +9,7 @@ import (
"os"
"path/filepath"
"sync/atomic"
+ "syscall"
"time"
"github.com/asymptote-labs/agent-beacon/pkg/asymptoteobserve"
@@ -21,12 +22,25 @@ const spoolDirInHome = ".beacon/endpoint/spool"
// DefaultSpoolDir is the current account's spool directory, or "" when it has no home. The home
// comes from the account, not the environment: a process started under other credentials can
// inherit its parent's HOME or USERPROFILE, which would point the spool at a home it cannot write.
+//
+// A home that does not exist counts as none, and so does a spool that would lie on the root
+// filesystem of a sandbox the account assembled (asymptoteobserve.OnSandboxRoot): a launcher that
+// bind-mounts only a project under the home makes the home exist there as an empty mount point.
+// A spool created in either would hold events that vanish with the sandbox, reported as spooled.
+// A spool directory the sandbox bind-mounts is real and is used.
func DefaultSpoolDir() string {
home, err := accounthome.Dir()
if err != nil || home == "" {
return ""
}
- return filepath.Join(home, spoolDirInHome)
+ if info, err := os.Stat(home); err != nil || !info.IsDir() {
+ return ""
+ }
+ dir := filepath.Join(home, spoolDirInHome)
+ if asymptoteobserve.OnSandboxRoot(dir) {
+ return ""
+ }
+ return dir
}
// defaultFailureBackoff is how long an account skips the socket after a failed send.
@@ -92,9 +106,10 @@ const (
// writes logPath, and it takes the line. A rejected line is reported, never retried.
// - If there is no socket, or the send fails -- a socket a crashed collector left behind, a
// collector that is hung -- the line is appended directly. On a shared log that simply works.
-// - Only when the direct append is refused for permission, as a restricted log refuses it while
-// its collector is down, is the line spooled. A collector that answered for some other log
-// will never take it, so then the refusal is returned instead.
+// - Only when the direct append is refused or cannot reach the log -- a restricted log refusing
+// it while its collector is down, or a sandbox where the log's directory is missing or
+// read-only -- is the line spooled. A collector that answered for some other log will never
+// take it, so then the refusal is returned instead.
//
// Every line that gets through takes up to a batch of this account's spool for logPath with it, by
// the same route. After a failed send the account skips the socket for FailureBackoff, in this
@@ -145,7 +160,7 @@ func (d Deliverer) WriteOutcome(logPath string, line []byte, appendLine func([]b
})
return Appended, nil
}
- if !errors.Is(err, fs.ErrPermission) || collectorElsewhere {
+ if !logUnreachable(err) || collectorElsewhere {
return Appended, err
}
if spoolErr := d.spoolLine(logPath, line); spoolErr != nil {
@@ -154,6 +169,13 @@ func (d Deliverer) WriteOutcome(logPath string, line []byte, appendLine func([]b
return Spooled, nil
}
+// logUnreachable reports whether a failed direct append means the log is out of this writer's
+// reach rather than broken: refused for permission, its directory missing, or a read-only mount.
+// Those are the states a spooled line outlasts.
+func logUnreachable(err error) bool {
+ return errors.Is(err, fs.ErrPermission) || errors.Is(err, fs.ErrNotExist) || errors.Is(err, syscall.EROFS)
+}
+
// sendFailureMarker, in the account's spool directory, is touched when a send fails. Hooks run as
// a new process per event, so the failure has to outlive the process that saw it.
const sendFailureMarker = ".send-failure"
@@ -238,7 +260,7 @@ func (d Deliverer) spoolFor(logPath string) Spool {
func (d Deliverer) spoolLine(logPath string, line []byte) error {
s := d.spoolFor(logPath)
if s.Dir == "" {
- return errors.New("no home directory to spool in")
+ return errors.New("no account spool to write: no home, or only the sandbox's own")
}
dropped, err := s.Append([][]byte{line})
if err != nil {
diff --git a/pkg/asymptoteobserve/localingest/deliver_unix_test.go b/pkg/asymptoteobserve/localingest/deliver_unix_test.go
index 6d055be6..7a7dd716 100644
--- a/pkg/asymptoteobserve/localingest/deliver_unix_test.go
+++ b/pkg/asymptoteobserve/localingest/deliver_unix_test.go
@@ -11,8 +11,11 @@ import (
"os/user"
"path/filepath"
"sync"
+ "syscall"
"testing"
"time"
+
+ "github.com/asymptote-labs/agent-beacon/pkg/asymptoteobserve/accounthome"
)
// testLog is the log the deliverer tests write; their servers accept any.
@@ -189,6 +192,32 @@ func TestWriteSpoolsWhenARestrictedLogRefusesAndDrainsAfter(t *testing.T) {
}
}
+// In a sandbox the log can be out of reach without a permission error: its directory was never
+// mounted, or it is mounted read-only. The line is spooled in those cases too, and a log that is
+// merely broken in some other way still returns its error.
+func TestWriteSpoolsWhenASandboxCannotReachTheLog(t *testing.T) {
+ for name, tc := range map[string]struct {
+ err error
+ spooled bool
+ }{
+ "directory missing": {&fs.PathError{Op: "open", Path: testLog, Err: syscall.ENOENT}, true},
+ "read-only mount": {&fs.PathError{Op: "open", Path: testLog, Err: syscall.EROFS}, true},
+ "disk full": {&fs.PathError{Op: "write", Path: testLog, Err: syscall.ENOSPC}, false},
+ } {
+ t.Run(name, func(t *testing.T) {
+ d := deliverer(t, filepath.Join(shortTempDir(t), "none.sock"))
+ err := d.Write(testLog, lines(t, 1)[0], (&directLog{failErr: tc.err}).append)
+ n, _ := d.Pending(testLog)
+ if tc.spooled && (err != nil || n != 1) {
+ t.Fatalf("Write = %v with %d pending, want the line spooled", err, n)
+ }
+ if !tc.spooled && (!errors.Is(err, tc.err) || n != 0) {
+ t.Fatalf("Write = %v with %d pending, want the error and nothing spooled", err, n)
+ }
+ })
+ }
+}
+
// Switching a restricted endpoint back to shared must not strand what was spooled meanwhile: the
// next direct append drains it.
func TestASpoolDrainsByDirectAppendOnceTheLogIsWritable(t *testing.T) {
@@ -477,3 +506,14 @@ func TestTheSpoolIsInTheAccountsHomeNotHOME(t *testing.T) {
t.Fatalf("DefaultSpoolDir = %q, want %q under the account's home", got, want)
}
}
+
+// A sandbox that assembles its own root filesystem often leaves the home out. A spool created in
+// its place would sit in the sandbox's own filesystem and vanish with it, reported as spooled.
+func TestNoSpoolInAHomeThatDoesNotExist(t *testing.T) {
+ prev := accounthome.Dir
+ t.Cleanup(func() { accounthome.Dir = prev })
+ accounthome.Dir = func() (string, error) { return filepath.Join(t.TempDir(), "missing-home"), nil }
+ if got := DefaultSpoolDir(); got != "" {
+ t.Fatalf("DefaultSpoolDir = %q, want none for a home that does not exist", got)
+ }
+}
diff --git a/pkg/asymptoteobserve/localingest/server_unix_test.go b/pkg/asymptoteobserve/localingest/server_unix_test.go
index 4e5ab7c0..495ccebb 100644
--- a/pkg/asymptoteobserve/localingest/server_unix_test.go
+++ b/pkg/asymptoteobserve/localingest/server_unix_test.go
@@ -593,6 +593,59 @@ func TestClientRefusesUntrustedSockets(t *testing.T) {
})
}
+// A sandbox without its own user namespace builds its root on a tmpfs the launching account owns, so
+// with the collector's socket directory bind-mounted in, the socket's ancestors belong to the
+// account sending. Those are trusted; an ancestor of any other account still is not.
+func TestClientTrustsSocketAncestorsOwnedByTheSender(t *testing.T) {
+ if os.Geteuid() != 0 {
+ t.Skip("needs root to give the socket's ancestors another owner")
+ }
+ const sender = 4242
+ h := startServer(t, nil)
+ ancestor := filepath.Dir(filepath.Dir(h.path))
+ if err := os.Chown(ancestor, sender, sender); err != nil {
+ t.Fatal(err)
+ }
+ prev := senderUID
+ t.Cleanup(func() { senderUID = prev })
+
+ senderUID = func() int { return sender }
+ if _, err := h.client.Send(lines(t, 1)); err != nil {
+ t.Fatalf("Send from the account owning the ancestor = %v", err)
+ }
+ senderUID = func() int { return sender + 1 }
+ if _, err := h.client.Send(lines(t, 1)); !errors.Is(err, ErrUntrustedSocket) {
+ t.Fatalf("Send from another account = %v, want ErrUntrustedSocket", err)
+ }
+ if h.rec.count() != 1 {
+ t.Fatalf("server received %d lines, want 1", h.rec.count())
+ }
+}
+
+// The path checks name who could have put the socket there; the kernel names who serves it. A
+// socket file and directory that look right are not enough when root does not serve them, which is
+// what an account that may rename the socket's ancestors could arrange in its own sandbox.
+func TestClientRefusesASocketItsOwnerDoesNotServe(t *testing.T) {
+ if os.Geteuid() != 0 {
+ t.Skip("needs root to give the socket another owner than its server")
+ }
+ const owner = 4242
+ h := startServer(t, nil)
+ for _, path := range []string{h.path, filepath.Dir(h.path)} {
+ if err := os.Lchown(path, owner, owner); err != nil {
+ t.Fatal(err)
+ }
+ }
+ c := h.client
+ c.TrustedOwner = owner
+ if _, err := c.Send(lines(t, 1)); !errors.Is(err, ErrUntrustedSocket) {
+ t.Fatalf("Send = %v, want ErrUntrustedSocket", err)
+ }
+ if h.rec.count() != 0 {
+ t.Fatal("sent to a socket its owner does not serve")
+ }
+}
+
func TestListenSetsModesWhateverTheUmask(t *testing.T) {
old := syscall.Umask(0o077)
defer syscall.Umask(old)
diff --git a/pkg/asymptoteobserve/localingest/transport_unix.go b/pkg/asymptoteobserve/localingest/transport_unix.go
index f0ec0857..f418a70e 100644
--- a/pkg/asymptoteobserve/localingest/transport_unix.go
+++ b/pkg/asymptoteobserve/localingest/transport_unix.go
@@ -8,6 +8,7 @@ import (
"net"
"os"
"path/filepath"
+ "strconv"
"syscall"
"time"
)
@@ -29,7 +30,10 @@ const SocketDirMode os.FileMode = 0o755
const SocketMode os.FileMode = 0o666
// connect dials the socket at the client's path once checkSocket has accepted it, by its resolved
-// path.
+// path, and keeps the connection only if the kernel says the process serving it is the trusted
+// owner. That last check is what makes the path checks safe to relax for a sandbox (see
+// checkSocketDirs): whatever happened to the path between the check and the dial, the client
+// sends nothing until it knows who is listening.
func (c Client) connect(timeout time.Duration) (net.Conn, error) {
socket, err := checkSocket(c.path(), c.TrustedOwner)
if err != nil {
@@ -39,13 +43,35 @@ func (c Client) connect(timeout time.Duration) (net.Conn, error) {
if err != nil {
return nil, fmt.Errorf("connect to %s: %w", c.path(), err)
}
+ if err := checkServer(conn, c.TrustedOwner); err != nil {
+ _ = conn.Close()
+ return nil, fmt.Errorf("%w: %s: %v", ErrUntrustedSocket, c.path(), err)
+ }
return conn, nil
}
+// checkServer asks the kernel which account is serving the other end of conn. For a client,
+// SO_PEERCRED and LOCAL_PEERCRED report the credentials the server listened with, which no
+// process can change afterwards.
+func checkServer(conn net.Conn, owner int) error {
+ unixConn, ok := conn.(*net.UnixConn)
+ if !ok {
+ return fmt.Errorf("not a Unix socket connection")
+ }
+ peer, err := peerCredentials(unixConn)
+ if err != nil {
+ return err
+ }
+ if peer.UID != strconv.Itoa(owner) {
+ return fmt.Errorf("served by uid %s, not uid %d", peer.UID, owner)
+ }
+ return nil
+}
+
// checkSocket decides whether a writer may send to path. The socket must be owned by owner, and so
-// must its directory, which nobody else may write; every directory above it must belong to root or
-// owner and be writable by nobody else unless it is sticky (as /tmp is), so the directory holding
-// the socket cannot be swapped for another.
+// must its directory, which nobody else may write; every directory above it must belong to root,
+// owner or the sending account, and be writable by nobody else unless it is sticky (as /tmp is),
+// so the directory holding the socket cannot be swapped by another account.
//
// It returns the socket's path with every link in its directories resolved, and the caller dials
// that path, never the one it was given: a link anywhere in the given path could otherwise be
@@ -79,8 +105,20 @@ func checkSocket(path string, owner int) (string, error) {
return resolved, nil
}
+// senderUID is the account sending; tests stand in another.
+var senderUID = os.Geteuid
+
// checkSocketDirs checks dir, which holds the socket, and every directory above it.
+//
+// An ancestor may belong to the sending account as well as to root. A sandbox without its own user
+// namespace -- bubblewrap as a sandboxing launcher runs it -- assembles its root filesystem on a
+// tmpfs the launching account owns, so with the collector's socket directory bind-mounted in, `/`,
+// `/var` and `/var/run` belong to the account whose hooks run there. Only that account and root can
+// rename such a directory, and no other account can, which is what the walk guards against. That
+// account could still swap one in its own sandbox, but then checkServer refuses whatever it points
+// at unless root serves it.
func checkSocketDirs(dir string, owner int) error {
+ sender := senderUID()
for first := true; ; first = false {
info, err := os.Lstat(dir)
if err != nil {
@@ -98,7 +136,7 @@ func checkSocketDirs(dir string, owner int) error {
if uid != owner || writableByOthers {
return fmt.Errorf("%w: %s must be owned by uid %d and writable by nobody else", ErrUntrustedSocket, dir, owner)
}
- } else if (uid != 0 && uid != owner) || !ancestorWriteSafe(info) {
+ } else if (uid != 0 && uid != owner && uid != sender) || !ancestorWriteSafe(info) {
return fmt.Errorf("%w: %s is writable by other accounts", ErrUntrustedSocket, dir)
}
parent := filepath.Dir(dir)
diff --git a/pkg/asymptoteobserve/runtimelogmode.go b/pkg/asymptoteobserve/runtimelogmode.go
index e38dffb9..d01575b1 100644
--- a/pkg/asymptoteobserve/runtimelogmode.go
+++ b/pkg/asymptoteobserve/runtimelogmode.go
@@ -98,6 +98,42 @@ func RuntimeLogIsPrivate(path string) bool {
return processOwnsHome(home)
}
+// RuntimeLogDirCreatable reports whether a writer may create the directory the runtime log at path
+// lives in. Every writer may, except in places a sandbox gets wrong:
+//
+// - Only a privileged writer creates the Unix system log's directory: install makes it,
+// root-owned, before any hook runs.
+// - Nobody creates a missing home directory to hold a log below it: an account's home exists
+// wherever the account really runs.
+// - Nobody creates it on the root filesystem of a sandbox the account assembled (OnSandboxRoot),
+// where a launcher that bind-mounts a project under the home has made the home exist as an
+// empty mount point.
+//
+// A writer that finds any of these is somewhere install and the home never were, and a log it
+// created there would take every event and vanish with the sandbox, with nothing reported.
+// Refusing leaves the write to fail, and the writer's fallbacks to handle it.
+func RuntimeLogDirCreatable(path string) bool {
+ if runtime.GOOS == "windows" {
+ return true
+ }
+ abs, err := filepath.Abs(path)
+ if err != nil {
+ return true
+ }
+ if os.Geteuid() != 0 && pathWithin(abs, unixSystemLogDir) {
+ return false
+ }
+ if OnSandboxRoot(filepath.Dir(abs)) {
+ return false
+ }
+ home, err := os.UserHomeDir()
+ if err != nil || strings.TrimSpace(home) == "" || isFilesystemRoot(home) || !pathWithin(abs, home) {
+ return true
+ }
+ info, err := os.Stat(home)
+ return err == nil && info.IsDir()
+}
+
// PrivateACLRuntimeLogFileMode is the mode a newly created private log takes when the directory
// it was created in hands it a POSIX ACL (a default ACL, `setfacl -d`). On Linux an ACL's mask is
// the file's group bits, so creating the file 0600 would set the mask to nothing and cancel the
diff --git a/pkg/asymptoteobserve/runtimelogmode_test.go b/pkg/asymptoteobserve/runtimelogmode_test.go
index 14302c04..212301e2 100644
--- a/pkg/asymptoteobserve/runtimelogmode_test.go
+++ b/pkg/asymptoteobserve/runtimelogmode_test.go
@@ -58,6 +58,43 @@ func TestSystemLogIsNeverPrivate(t *testing.T) {
}
}
+// An ordinary account never creates the system log's directory, which install makes before any hook
+// runs: one that is missing means a sandbox with a root filesystem of its own, where a log the hook
+// created would take every event and vanish with the sandbox. Any other log's directory is created
+// as before.
+func TestOnlyRootCreatesTheSystemLogDirectory(t *testing.T) {
+ if runtime.GOOS == "windows" {
+ t.Skip("the Unix system log directory does not exist on Windows")
+ }
+ system := filepath.Join(unixSystemLogDir, "runtime.jsonl")
+ if got, want := RuntimeLogDirCreatable(system), os.Geteuid() == 0; got != want {
+ t.Fatalf("RuntimeLogDirCreatable(%s) = %v, want %v for euid %d", system, got, want, os.Geteuid())
+ }
+ if other := filepath.Join(t.TempDir(), "logs", "runtime.jsonl"); !RuntimeLogDirCreatable(other) {
+ t.Fatalf("RuntimeLogDirCreatable(%s) = false, want any other log's directory creatable", other)
+ }
+}
+
+// A per-user log lives under the home, and a sandbox often leaves the home out: no writer creates a
+// missing home to hold the log, or the events go into the sandbox's own filesystem and vanish.
+func TestNoWriterCreatesAMissingHomeForItsLog(t *testing.T) {
+ if runtime.GOOS == "windows" {
+ t.Skip("Windows profiles are not the sandbox case")
+ }
+ home := filepath.Join(t.TempDir(), "missing-home")
+ setTestHome(t, home)
+ userLog := filepath.Join(home, ".beacon", "endpoint", "logs", "runtime.jsonl")
+ if RuntimeLogDirCreatable(userLog) {
+ t.Fatal("RuntimeLogDirCreatable = true under a home that does not exist")
+ }
+ if err := os.Mkdir(home, 0o700); err != nil {
+ t.Fatal(err)
+ }
+ if !RuntimeLogDirCreatable(userLog) {
+ t.Fatal("RuntimeLogDirCreatable = false under an existing home")
+ }
+}
+
func TestSettleRuntimeLogArchives(t *testing.T) {
if runtime.GOOS == "windows" {
t.Skip("Unix permission bits do not exist on Windows")
diff --git a/pkg/asymptoteobserve/sandboxroot_linux.go b/pkg/asymptoteobserve/sandboxroot_linux.go
new file mode 100644
index 00000000..0a230fab
--- /dev/null
+++ b/pkg/asymptoteobserve/sandboxroot_linux.go
@@ -0,0 +1,57 @@
+//go:build linux
+
+package asymptoteobserve
+
+import (
+ "os"
+ "path/filepath"
+ "syscall"
+)
+
+// sandboxRootDir is the root directory OnSandboxRoot inspects; a variable so tests can stand a
+// directory they own in for it.
+var sandboxRootDir = "/"
+
+// tmpfsMagic is TMPFS_MAGIC, the f_type statfs reports for a tmpfs.
+const tmpfsMagic = 0x01021994
+
+// filesystemIsTmpfs reports whether path lies on a tmpfs; a variable so tests can stand any
+// directory in for bubblewrap's root.
+var filesystemIsTmpfs = func(path string) bool {
+ var fs syscall.Statfs_t
+ return syscall.Statfs(path, &fs) == nil && int64(fs.Type) == tmpfsMagic
+}
+
+// OnSandboxRoot reports whether path, or the nearest directory above it that exists, lies on the
+// throwaway root filesystem of a sandbox this account assembled itself: a tmpfs mounted at / that
+// the account owns. Bubblewrap builds exactly that, with or without a user namespace, where a real
+// host's / is root's and a persistent one (a rootfs directory bound at /, an Apptainer sandbox) is
+// not a tmpfs. Anything created on it vanishes with the sandbox, so a writer that would put a log
+// or a spool there reports the event as not recorded instead. What the sandbox bind-mounts (a
+// workspace, a home, Beacon's own directories) is another filesystem and stays real. Root's own
+// writes never count: root needs no sandbox root of its own to write anywhere.
+func OnSandboxRoot(path string) bool {
+ euid := os.Geteuid()
+ if euid == 0 {
+ return false
+ }
+ var root syscall.Stat_t
+ if err := syscall.Stat(sandboxRootDir, &root); err != nil || int(root.Uid) != euid || !filesystemIsTmpfs(sandboxRootDir) {
+ return false
+ }
+ abs, err := filepath.Abs(path)
+ if err != nil {
+ return false
+ }
+ for dir := abs; ; {
+ var st syscall.Stat_t
+ if err := syscall.Stat(dir, &st); err == nil {
+ return st.Dev == root.Dev
+ }
+ parent := filepath.Dir(dir)
+ if parent == dir {
+ return true
+ }
+ dir = parent
+ }
+}
diff --git a/pkg/asymptoteobserve/sandboxroot_linux_test.go b/pkg/asymptoteobserve/sandboxroot_linux_test.go
new file mode 100644
index 00000000..eb847dac
--- /dev/null
+++ b/pkg/asymptoteobserve/sandboxroot_linux_test.go
@@ -0,0 +1,75 @@
+//go:build linux
+
+package asymptoteobserve
+
+import (
+ "os"
+ "path/filepath"
+ "syscall"
+ "testing"
+)
+
+// standInSandboxRoot makes root stand in for /, a tmpfs (bubblewrap's) or not.
+func standInSandboxRoot(t *testing.T, root string, tmpfs bool) {
+ t.Helper()
+ previousRoot, previousTmpfs := sandboxRootDir, filesystemIsTmpfs
+ sandboxRootDir = root
+ filesystemIsTmpfs = func(string) bool { return tmpfs }
+ t.Cleanup(func() { sandboxRootDir, filesystemIsTmpfs = previousRoot, previousTmpfs })
+}
+
+// A launcher that bind-mounts a project under the home makes the home exist, as an empty mount
+// point on the sandbox's own root filesystem. A log created there would take every event and
+// vanish with the sandbox; a directory the sandbox mounts is another filesystem and stays real.
+func TestNoLogIsCreatedOnASandboxRoot(t *testing.T) {
+ if os.Geteuid() == 0 {
+ t.Skip("root's writes never count as a sandbox's")
+ }
+ root := t.TempDir() // owned by this account, as bubblewrap's tmpfs root is
+ standInSandboxRoot(t, root, true)
+
+ home := filepath.Join(root, "home", "u")
+ if err := os.MkdirAll(filepath.Join(home, "src", "proj"), 0o755); err != nil {
+ t.Fatal(err)
+ }
+ setTestHome(t, home)
+ userLog := filepath.Join(home, ".beacon", "endpoint", "logs", "runtime.jsonl")
+ if !OnSandboxRoot(userLog) || RuntimeLogDirCreatable(userLog) {
+ t.Fatalf("a log under a home that exists only on the sandbox's root was creatable (OnSandboxRoot %v)", OnSandboxRoot(userLog))
+ }
+
+ var rootStat, devStat syscall.Stat_t
+ if err := syscall.Stat(root, &rootStat); err != nil {
+ t.Fatal(err)
+ }
+ if err := syscall.Stat("/dev", &devStat); err == nil && devStat.Dev != rootStat.Dev {
+ if OnSandboxRoot(filepath.Join("/dev", "beacon-test-missing", "spool")) {
+ t.Fatal("a path on another filesystem (a bind mount) counted as the sandbox's root")
+ }
+ }
+
+ standInSandboxRoot(t, "/", true)
+ if OnSandboxRoot(userLog) {
+ t.Fatal("a root directory this account does not own counted as a sandbox it assembled")
+ }
+}
+
+// A sandbox whose / is a persistent directory the account owns (a rootfs bound at /, an
+// Apptainer sandbox directory), with the home bound from the same disk, keeps what is written
+// there. Its writes are real and must go ahead.
+func TestAPersistentRootIsNotASandboxRoot(t *testing.T) {
+ if os.Geteuid() == 0 {
+ t.Skip("root's writes never count as a sandbox's")
+ }
+ root := t.TempDir()
+ standInSandboxRoot(t, root, false)
+ home := filepath.Join(root, "home", "u")
+ if err := os.MkdirAll(home, 0o755); err != nil {
+ t.Fatal(err)
+ }
+ setTestHome(t, home)
+ userLog := filepath.Join(home, ".beacon", "endpoint", "logs", "runtime.jsonl")
+ if OnSandboxRoot(userLog) || !RuntimeLogDirCreatable(userLog) {
+ t.Fatal("a log on a persistent, account-owned root was refused as if the root were a sandbox's tmpfs")
+ }
+}
diff --git a/pkg/asymptoteobserve/sandboxroot_other.go b/pkg/asymptoteobserve/sandboxroot_other.go
new file mode 100644
index 00000000..7bc7461b
--- /dev/null
+++ b/pkg/asymptoteobserve/sandboxroot_other.go
@@ -0,0 +1,7 @@
+//go:build !linux
+
+package asymptoteobserve
+
+// OnSandboxRoot is never true off Linux: no sandbox there assembles a throwaway root filesystem
+// the account owns (bubblewrap is Linux-only).
+func OnSandboxRoot(string) bool { return false }