The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
Validate the forwarder config before enrollment rotates the device key
shukieshahaeefd3180940
diff --git a/CLAUDE.md b/CLAUDE.mdindex 04ef3d9e..2e0b8132 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -494,7 +494,7 @@ unpacked through Chrome's developer mode.- Keep `gen_ai.usage` as the single canonical token-usage representation: normalize all runtime token telemetry (span attributes, metric datapoints, legacy `llm.usage.*` aliases) into it, mirror OTel GenAI semconv JSON names exactly, and never add parallel or per-harness token fields. `gen_ai.usage.cost_usd` carries runtime-reported cost only; do not derive cost from local pricing tables.- When adding a new signal, include stable identifiers/counts/hashes alongside any retained raw content, and route raw fields through redaction, sanitization, truncation, and event-size controls.- Keep the dashboard read-only. It should inspect local status and JSONL events but must not mutate endpoint configuration or telemetry.-- The Asymptote device key has exactly one home, `<BaseDir>/asymptote/vector-secrets.json`, read by Vector's `secret` file backend. Never copy it into `config.json`, `vector.toml`, a service unit, an environment variable, a log line, or a `--json` result; `HasSecretDestinations` is unchanged because the key is not in `config.json`. `internal/auth` is flow-agnostic (PKCE, loopback callback server with an injected exchange function, browser opener, `PostJSON`); new browser flows reuse it rather than adding a second callback server. `connect` must fail before opening a browser when Vector or a service manager is missing, so no key is minted that cannot be used.+- The Asymptote device key has exactly one home, `<BaseDir>/asymptote/vector-secrets.json`, read by Vector's `secret` file backend. Never copy it into `config.json`, `vector.toml`, a service unit, an environment variable, a log line, or a `--json` result; `HasSecretDestinations` is unchanged because the key is not in `config.json`. `internal/auth` is flow-agnostic (PKCE, loopback callback server with an injected exchange function, browser opener, `PostJSON`); new browser flows reuse it rather than adding a second callback server. `connect` must fail before opening a browser when Vector or a service manager is missing, or when Vector rejects the rendered config, so no key is minted or rotated that cannot be used; enrollment rotates a connected device's key in place, so every local check runs ahead of it.- Keep the release readiness guidance in `README.md` up to date when install, packaging, collector, or dashboard behavior changes.## CI Expectationsdiff --git a/cli/beacon/internal/endpoint/asymptote/connect.go b/cli/beacon/internal/endpoint/asymptote/connect.goindex 1700042d..dcdc926d 100644--- a/cli/beacon/internal/endpoint/asymptote/connect.go+++ b/cli/beacon/internal/endpoint/asymptote/connect.go@@ -67,12 +67,15 @@ type ConnectResult struct {// Connect enrolls this machine and starts the forwarder.//-// Failure ordering is deliberate: Vector is located before the browser opens, so a machine-// without Vector never mints a key it cannot use; the key is written to the 0600 secrets file-// before the config that references it; the config is validated before a service unit points at-// it; and the enrollment record is saved last, so status never claims a connection that did not-// finish. A re-run on an enrolled machine reuses its install id, which rotates the key in place-// on the server rather than creating a second device.+// Failure ordering is deliberate: Vector is located and the forwarder config is validated+// before enrollment, so a machine whose Vector is missing or rejects the config never mints a+// key it cannot use; the key is written to the 0600 secrets file before the config that+// references it; the written config is validated before a service unit points at it; and the+// enrollment record is saved last, so status never claims a connection that did not finish.+// A re-run on an enrolled machine reuses its install id, which rotates the key in place on the+// server rather than creating a second device. That rotation is why validation comes first:+// once the server has rotated the key, a forwarder still running with the old one is failing+// every upload, so nothing that can be checked locally may fail after that point.func Connect(ctx context.Context, opts ConnectOptions) (*ConnectResult, error) {out := opts.Outif out == nil {@@ -116,6 +119,29 @@ func Connect(ctx context.Context, opts ConnectOptions) (*ConnectResult, error) {return nil, err}}+ logPath := opts.LogPath+ if logPath == "" {+ logPath = endpointconfig.Default(opts.UserMode, "").LogPath+ }++ // Prove Vector accepts this privacy mode's config before enrollment rotates the key.+ // The ingest URL is the previous enrollment's when there is one; validate never+ // connects, so a placeholder serves a first enrollment.+ preflightURL := preflightIngestURL+ if previous != nil {+ preflightURL = previous.IngestURL+ }+ if err := ensureDir(opts.UserMode); err != nil {+ return nil, err+ }+ if err := preflightVectorConfig(vector.Path, Dir(opts.UserMode), RenderOptions{+ LogPath: logPath,+ IngestURL: preflightURL,+ PrivacyMode: privacyMode,+ }); err != nil {+ return nil, err+ }+// Pin the id before the dashboard learns it, so a failure after approval retries as// the same device.if err := WriteInstallID(opts.UserMode, installID); err != nil {@@ -153,14 +179,12 @@ func Connect(ctx context.Context, opts ConnectOptions) (*ConnectResult, error) {}result.DeviceKey = ""dataDir := DataDir(opts.UserMode)- logPath := opts.LogPath- if logPath == "" {- logPath = endpointconfig.Default(opts.UserMode, "").LogPath- }- // Render and pre-validate before touching the service manager. A privacy-mode- // change unloads the forwarder below; validating first ensures a bad config- // (unsupported VRL, permissions, etc.) never stops a working forwarder.+ // Validate the exact bytes the service will run before touching the service manager,+ // so the privacy-mode change below never stops a working forwarder for a config Vector+ // would refuse. The preflight already proved the template; what is new here is the+ // server's ingest URL and the secrets file just written. Vector refuses a data_dir that+ // does not exist, so create it first; a mode change clears and recreates it below.rendered, err := RenderVectorConfig(RenderOptions{LogPath: logPath,IngestURL: result.IngestURL,@@ -171,6 +195,9 @@ func Connect(ctx context.Context, opts ConnectOptions) (*ConnectResult, error) {if err != nil {return nil, err}+ if err := os.MkdirAll(dataDir, 0o700); err != nil {+ return nil, err+ }configPath := VectorConfigPath(opts.UserMode)if err := preValidateVectorConfig(vector.Path, configPath, []byte(rendered)); err != nil {return nil, err@@ -185,11 +212,11 @@ func Connect(ctx context.Context, opts ConnectOptions) (*ConnectResult, error) {if err := os.RemoveAll(dataDir); err != nil && !os.IsNotExist(err) {return nil, fmt.Errorf("could not clear the buffer after a privacy mode change: %w", err)}+ if err := os.MkdirAll(dataDir, 0o700); err != nil {+ return nil, err+ }}}- if err := os.MkdirAll(dataDir, 0o700); err != nil {- return nil, err- }if err := writeFileAtomic(configPath, []byte(rendered), 0o644); err != nil {return nil, err}diff --git a/cli/beacon/internal/endpoint/asymptote/connect_test.go b/cli/beacon/internal/endpoint/asymptote/connect_test.goindex d567ee17..9f93fb3c 100644--- a/cli/beacon/internal/endpoint/asymptote/connect_test.go+++ b/cli/beacon/internal/endpoint/asymptote/connect_test.go@@ -286,9 +286,23 @@ func TestConnectFailsWhenVectorValidateRejectsTheConfig(t *testing.T) {t.Setenv("HOME", home)fd := newFakeDashboard(t)fwd := &fakeForwarder{supported: true}- if _, err := Connect(context.Background(), connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 78))); err == nil || !strings.Contains(err.Error(), "vector validate failed") {+ opened := false+ opts := connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 78))+ opts.Enroll.OpenBrowser = func(string) error { opened = true; return nil }+ if _, err := Connect(context.Background(), opts); err == nil || !strings.Contains(err.Error(), "vector validate failed") {t.Fatalf("expected validate failure, got %v", err)}+ // The config is validated before enrollment, so a Vector that rejects it never opens+ // the browser and no key is minted or stored.+ if opened {+ t.Fatal("browser must not open when Vector rejects the forwarder config")+ }+ if _, err := os.Stat(SecretsPath(true)); !os.IsNotExist(err) {+ t.Fatal("no secrets file may exist when validation fails before enrollment")+ }+ if entries, _ := os.ReadDir(Dir(true)); len(entries) != 0 {+ t.Fatalf("preflight must clean up after itself, found %v", entries)+ }if len(fwd.written) != 0 || fwd.loads != 0 {t.Fatal("no unit may be written or loaded when validation fails")}@@ -449,10 +463,12 @@ func TestConnectPinsInstallIDBeforeEnrollmentSoRetriesReuseIt(t *testing.T) {fd := newFakeDashboard(t)fwd := &fakeForwarder{supported: true}- // First attempt fails after approval (vector validate rejects the config).- if _, err := Connect(context.Background(), connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 1))); err == nil {- t.Fatal("expected validate failure")+ // First attempt fails after approval (the service manager cannot start the forwarder).+ fwd.loadErr = errors.New("bootstrap failed")+ if _, err := Connect(context.Background(), connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 0))); err == nil || !strings.Contains(err.Error(), "could not be started") {+ t.Fatalf("expected a load failure, got %v", err)}+ fwd.loadErr = nilfd.mu.Lock()firstID := fd.init["device"].(map[string]any)["install_id"].(string)fd.mu.Unlock()@@ -560,7 +576,7 @@ func TestConnectClearsBufferOnPrivacyModeChange(t *testing.T) {}}-func TestConnectValidateFailureOnPrivacyChangeDoesNotStopForwarder(t *testing.T) {+func TestConnectRejectedConfigOnPrivacyChangeRotatesNoKeyAndKeepsForwarderRunning(t *testing.T) {home := t.TempDir()t.Setenv("HOME", home)isolateVectorDiscovery(t)@@ -575,16 +591,49 @@ func TestConnectValidateFailureOnPrivacyChangeDoesNotStopForwarder(t *testing.T)if fwd.loads != 1 || fwd.unloads != 0 {t.Fatalf("after first connect: loads=%d unloads=%d", fwd.loads, fwd.unloads)}+ before, err := LoadEnrollment(true)+ if err != nil {+ t.Fatal(err)+ }+ secretsBefore, err := os.ReadFile(SecretsPath(true))+ if err != nil {+ t.Fatal(err)+ }+ configBefore, err := os.ReadFile(VectorConfigPath(true))+ if err != nil {+ t.Fatal(err)+ }+ fd.mu.Lock()+ fd.init, fd.exchange = nil, nil+ fd.mu.Unlock()- // Second connect changes privacy mode but vector validate fails (exit 78).- // The forwarder must NOT be unloaded.+ // Second connect changes privacy mode but vector validate fails (exit 78). Enrollment+ // would rotate the server-side key, so it must not be reached: the running forwarder+ // keeps its key and config and nothing on disk changes.+ opened := falseopts2 := connectOptions(t, fd, fwd, fakeVector(t, "0.56.0", 78))opts2.PrivacyMode = "metadata-only"+ opts2.Enroll.OpenBrowser = func(string) error { opened = true; return nil }if _, err := Connect(context.Background(), opts2); err == nil || !strings.Contains(err.Error(), "vector validate failed") {t.Fatalf("expected validate failure, got %v", err)}- if fwd.unloads != 0 {- t.Fatalf("a validate failure during a privacy-mode change must not stop the running forwarder, unloads=%d", fwd.unloads)+ fd.mu.Lock()+ reached := fd.init != nil || fd.exchange != nil+ fd.mu.Unlock()+ if opened || reached {+ t.Fatalf("a rejected config must fail before enrollment: opened=%t reached dashboard=%t", opened, reached)+ }+ if fwd.unloads != 0 || fwd.loads != 1 {+ t.Fatalf("a rejected config must leave the running forwarder alone: loads=%d unloads=%d", fwd.loads, fwd.unloads)+ }+ secretsAfter, _ := os.ReadFile(SecretsPath(true))+ configAfter, _ := os.ReadFile(VectorConfigPath(true))+ after, _ := LoadEnrollment(true)+ if string(secretsAfter) != string(secretsBefore) || string(configAfter) != string(configBefore) || after == nil || *after != *before {+ t.Fatal("secrets, config and enrollment must be untouched by a rejected reconnect")+ }+ if after.PrivacyMode != "standard" {+ t.Fatalf("enrollment privacy mode = %q, want the original standard", after.PrivacyMode)}}@@ -644,3 +693,31 @@ func TestDisconnectIgnoresALeftoverInstallIDFromACancelledConnect(t *testing.T)t.Fatalf("an installed unit must be unloaded, unloads=%d", fwd.unloads)}}++// TestConnectWithRealVectorOnAFreshMachine runs the whole connect sequence against a real+// Vector binary, including the mode switch, so the validate steps are exercised with Vector's+// actual rules (it refuses a data_dir that does not exist yet) rather than the fake's.+func TestConnectWithRealVectorOnAFreshMachine(t *testing.T) {+ vector := os.Getenv("BEACON_TEST_VECTOR_BIN")+ if vector == "" {+ t.Skip("set BEACON_TEST_VECTOR_BIN to run connect against a real Vector")+ }+ t.Setenv("HOME", t.TempDir())+ isolateVectorDiscovery(t)+ fd := newFakeDashboard(t)+ fwd := &fakeForwarder{supported: true}++ opts := connectOptions(t, fd, fwd, vector)+ opts.PrivacyMode = "standard"+ if _, err := Connect(context.Background(), opts); err != nil {+ t.Fatalf("first connect on a fresh machine: %v", err)+ }+ opts = connectOptions(t, fd, fwd, vector)+ opts.PrivacyMode = "metadata-only"+ if _, err := Connect(context.Background(), opts); err != nil {+ t.Fatalf("switch to metadata-only: %v", err)+ }+ if fwd.loads != 2 || fwd.unloads != 1 {+ t.Fatalf("loads=%d unloads=%d", fwd.loads, fwd.unloads)+ }+}diff --git a/cli/beacon/internal/endpoint/asymptote/vector.go b/cli/beacon/internal/endpoint/asymptote/vector.goindex 92b6bb2c..ca78d656 100644--- a/cli/beacon/internal/endpoint/asymptote/vector.go+++ b/cli/beacon/internal/endpoint/asymptote/vector.go@@ -194,6 +194,43 @@ func ValidateVectorConfig(vectorBin, configPath string) error {return nil}+// preflightIngestURL stands in for the ingest URL on a first enrollment's preflight. The+// `.invalid` TLD is reserved (RFC 2606) and validate never connects, so it can resolve to+// nothing and still prove the config.+const preflightIngestURL = "https://ingest.invalid"++// preflightVectorConfig proves Vector accepts the forwarder config for the requested+// privacy mode before anything irreversible happens. Enrollment rotates the device key on+// the server, so a config Vector rejected only afterwards would leave a running forwarder+// holding a key the server no longer accepts. Everything the render points at lives in a+// scratch directory under stateDir: a data dir, because Vector refuses one that does not+// exist, and a placeholder secrets file, so the check reads no real credential and works+// before one exists.+func preflightVectorConfig(vectorBin, stateDir string, opts RenderOptions) error {+ scratch, err := os.MkdirTemp(stateDir, ".vector-preflight-*")+ if err != nil {+ return err+ }+ defer os.RemoveAll(scratch)+ opts.DataDir = filepath.Join(scratch, DataDirName)+ if err := os.Mkdir(opts.DataDir, 0o700); err != nil {+ return err+ }+ opts.SecretsFile = filepath.Join(scratch, SecretsFileName)+ if err := os.WriteFile(opts.SecretsFile, []byte(SecretsFileContent("bcn_device_preflight")), 0o600); err != nil {+ return err+ }+ rendered, err := RenderVectorConfig(opts)+ if err != nil {+ return err+ }+ configPath := filepath.Join(scratch, VectorConfigName)+ if err := os.WriteFile(configPath, []byte(rendered), 0o600); err != nil {+ return err+ }+ return ValidateVectorConfig(vectorBin, configPath)+}+// preValidateVectorConfig writes data to a temporary file next to configPath,// validates it, and removes the temp file. This lets Connect verify a new config// before stopping a running forwarder.diff --git a/docs/cli/endpoint-connect.mdx b/docs/cli/endpoint-connect.mdxindex b3107d7c..9bbfa53e 100644--- a/docs/cli/endpoint-connect.mdx+++ b/docs/cli/endpoint-connect.mdx@@ -18,20 +18,21 @@ beacon endpoint disconnect [flags]- Beacon endpoint installed (`beacon endpoint install`) and writing local JSONL.- A current `beacon login` session with `device:enroll` scope for user mode.-- Vector 0.50 or newer on the machine (Apple Silicon gets 0.58 from Homebrew or the package; Intel Macs get 0.50.0, the last x86_64 build Vector published). The signed macOS package installs it at `/opt/beacon/bin/vector`; on macOS Homebrew installs it with Beacon, as the tap's `beacon-vector` keg (named that rather than `vector` so it coexists with a Vector from `vectordotdev/brew`, and kept in `libexec` so it stays off your PATH); on Linux install the `vector` package from [vector.dev](https://vector.dev). `connect` stops before opening a browser if no usable Vector is found.+- Vector 0.50 or newer on the machine (Apple Silicon gets 0.58 from Homebrew or the package; Intel Macs get 0.50.0, the last x86_64 build Vector published). The signed macOS package installs it at `/opt/beacon/bin/vector`; on macOS Homebrew installs it with Beacon, as the tap's `beacon-vector` keg (named that rather than `vector` so it coexists with a Vector from `vectordotdev/brew`, and kept in `libexec` so it stays off your PATH); on Linux install the `vector` package from [vector.dev](https://vector.dev). `connect` stops before opening a browser if no usable Vector is found or if Vector rejects the forwarder config.- System mode only: a browser where an organization member can approve the device.## What connect does1. Locates Vector: `--vector-bin`, then `BEACON_VECTOR_BIN`, then `/opt/beacon/bin/vector`, the `beacon-vector` keg in each Homebrew prefix, a linked `vector` in each Homebrew prefix, then `PATH`. The `beacon-vector` keg is tried ahead of any other Homebrew Vector because it is the copy the tap pins at a supported version.-2. Uses the signed-in account's `device:enroll` scope to register this machine's hostname, operating system, architecture, Beacon version, install id, and install mode. The server returns a separate `bcn_device_*` ingest key. System mode uses the existing browser PKCE device approval instead.-3. Writes the device key once to `asymptote/vector-secrets.json` with mode `0600`. The account token is not persisted by the endpoint flow.-4. Renders `asymptote/vector.toml` with the selected privacy mode, runs `vector validate`, installs a service unit, and starts it.-5. Records the non-secret enrollment, including privacy mode, in `asymptote/enrollment.json` and `config.json`'s `managed_ingest` block.+2. Renders the forwarder config for the selected privacy mode into a scratch directory, with a placeholder key, and runs `vector validate` on it. A config Vector rejects stops `connect` here, before anything is sent to the server.+3. Uses the signed-in account's `device:enroll` scope to register this machine's hostname, operating system, architecture, Beacon version, install id, and install mode. The server returns a separate `bcn_device_*` ingest key. System mode uses the existing browser PKCE device approval instead.+4. Writes the device key once to `asymptote/vector-secrets.json` with mode `0600`. The account token is not persisted by the endpoint flow.+5. Renders `asymptote/vector.toml` with the selected privacy mode, runs `vector validate` on the file the service will run, installs a service unit, and starts it.+6. Records the non-secret enrollment, including privacy mode, in `asymptote/enrollment.json` and `config.json`'s `managed_ingest` block.A connect that fails at `vector validate` or at loading the service saves no enrollment record, so the endpoint is not connected and `beacon endpoint disconnect` cleans up what was written. After `disconnect --keep-credentials`, status reports not connected and names the kept device id.-Re-running `connect` on a connected machine reuses its install id, so the server rotates the existing device's key in place instead of creating a second device. Under launchd the new forwarder is loaded only after the old one has exited (Vector drains in-flight requests first), and if the new instance has not started within the wait, `connect` fails and points at `launchctl print` for the job (v1.3.2).+Re-running `connect` on a connected machine reuses its install id, so the server rotates the existing device's key in place instead of creating a second device. Because that rotation invalidates the key the running forwarder holds, the new config is validated before enrollment: a re-run whose config Vector rejects leaves the forwarder, its key, and its buffer untouched. Under launchd the new forwarder is loaded only after the old one has exited (Vector drains in-flight requests first), and if the new instance has not started within the wait, `connect` fails and points at `launchctl print` for the job (v1.3.2).## Privacy modes@@ -45,7 +46,9 @@ The onboarding wizard records one of two modes. `connect` uses that choice unlesThe transform runs locally in Vector before batching, buffering, or HTTPS upload.Changing modes requires rerunning `beacon endpoint connect`; the rendered config and-status output record the active choice.+status output record the active choice. A mode change stops the forwarder and clears its+disk buffer before the new config is installed, so events queued under the previous mode+are not uploaded under the new one.## Files and services