The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
test(sandbox): i04 proves the bundled Vector forwards Beacon's events
shukieshahffa0e476480b
diff --git a/beacon-sandbox/AGENTS.md b/beacon-sandbox/AGENTS.mdindex 12f61c7f..bbd26066 100644--- a/beacon-sandbox/AGENTS.md+++ b/beacon-sandbox/AGENTS.md@@ -66,6 +66,7 @@ real money on the user's account, so:| Approval or permission handling | `--scenario s07-denied-tool` || `endpoint install`, config paths, service startup | `--scenario i01-install-supervised` || The systemd backend, unit files, Linux system mode | `--scenario i02-install-systemd` |+| A Vector pack, the bundled Vector version, or its secrets file | `--scenario i04-vector-forwarding` || Anything Windows | `--provider github --scenario w00-probe` || Something broad, or preparing a PR | the whole suite (no `--scenario`) |diff --git a/beacon-sandbox/check/forwarding.go b/beacon-sandbox/check/forwarding.gonew file mode 100644index 00000000..e2193e84--- /dev/null+++ b/beacon-sandbox/check/forwarding.go@@ -0,0 +1,259 @@+package check++import (+ "bufio"+ "encoding/json"+ "errors"+ "fmt"+ "io/fs"+ "os"+ "regexp"+ "sort"+ "strings"+)++// DeviceKeyPattern matches an Asymptote device key, test or real.+//+// The leak checks search for the key's shape rather than its value, so they work offline too: the+// forwarding probe generates the key inside the guest and nothing on the host ever holds it. No+// Beacon event has any reason to contain this prefix, so any match is a disclosure.+var DeviceKeyPattern = regexp.MustCompile(`bcn_device_[A-Za-z0-9_]{8,}`)++// Forward is what the runner observed while the bundled Vector forwarded Beacon's runtime log to a+// loopback stand-in for the ingest service.+//+// Ran, Err and VectorExe are strings for the same reason as Removal's fields: empty means the probe+// did not make that observation, which must never read as a pass.+type Forward struct {+ // Asked is whether the scenario requested the probe. Without it, a run whose probe never started+ // would look the same as a scenario that never asked.+ Asked bool+ Ran string+ // Err is the first thing that went wrong while setting up or collecting.+ Err string+ // VectorExe is the executable of the running Vector process, from /proc/<pid>/exe.+ VectorExe string+ WantVectorExe string+ // Canary is the run's canary, which the session's own events carry.+ Canary string++ // Received is the runtime NDJSON the stand-in accepted, and ReceivedErr why it could not be read.+ Received Log+ ReceivedErr error+ // Requests is the stand-in's request log.+ Requests []ForwardRequest+ RequestsErr error+ // Runtime is Beacon's runtime log as it stood after forwarding finished, to check the forwarded+ // lines are Beacon's own bytes.+ Runtime Log+ RuntimeErr error+ // Text is other collected output, such as Vector's own log, keyed by artifact name, searched+ // for a device key.+ Text map[string]string+}++// ForwardRequest is one request the stand-in served. It records whether the bearer key matched the+// secrets file, never the header itself.+type ForwardRequest struct {+ Method string `json:"method"`+ Path string `json:"path"`+ AuthOK bool `json:"auth_ok"`+ Lines int `json:"lines"`+ Encoding string `json:"encoding"`+}++// ReadForwardRequests parses the stand-in's request log. A malformed line is an error: the log is+// written by our own stand-in, so a bad line means the probe is broken.+func ReadForwardRequests(path string) ([]ForwardRequest, error) {+ f, err := os.Open(path)+ if err != nil {+ return nil, err+ }+ defer f.Close()+ var out []ForwardRequest+ sc := bufio.NewScanner(f)+ for n := 1; sc.Scan(); n++ {+ line := strings.TrimSpace(sc.Text())+ if line == "" {+ continue+ }+ var r ForwardRequest+ if err := json.Unmarshal([]byte(line), &r); err != nil {+ return out, fmt.Errorf("%s:%d: %w", path, n, err)+ }+ out = append(out, r)+ }+ return out, sc.Err()+}++// Asymptote's validation event, written by `beacon endpoint asymptote validate`.+const (+ asymptoteDestinationType = "asymptote"+ asymptoteDestinationMode = "asymptote_managed_http"+)++func isAsymptoteValidation(e Event) bool {+ if e.ParseErr != nil {+ return false+ }+ typ, _ := e.Field("destination.type")+ mode, _ := e.Field("destination.mode")+ return typ == asymptoteDestinationType && mode == asymptoteDestinationMode+}++// Forwarding judges whether the bundled Vector delivered Beacon's events, intact and authenticated.+//+// Four things have to hold, and each is a separate way this has broken or could:+//+// - The Vector that ran is the one the packages install. The probe starts it by that path, so a+// mismatch means the image or the probe is wrong, and the run would be testing some other+// binary.+// - Every request carried the device key from the secrets file. That is the secret backend+// working; a wrong or missing key is exactly what a revoked-looking endpoint would send.+// - The validation event and at least one event from the session arrived. The validation event+// is written right after Vector starts; the session's events are written minutes later, so they+// prove Vector kept tailing the file rather than shipping one batch and stopping.+// - Every forwarded line is a line from Beacon's runtime log, byte for byte. Standard mode ships+// retained content unmodified, so any difference is Vector or the pack altering events.+//+// No device key may appear in anything collected.+func Forwarding(v *Verdict, f Forward) {+ if !f.Asked {+ return+ }+ fail := func(check, summary, why string, evidence ...string) {+ v.Add(Finding{Check: check, Severity: SevFail, Summary: summary, Why: why, Evidence: cap5(evidence)})+ }+ if f.Ran != "true" {+ fail("forwarding.probe_ran", "the forwarding probe did not run, so nothing is known about "++ "whether the bundled Vector forwards",+ "An unrun probe is not a pass. The scenario exists to run Vector; a run that did not is a "++ "broken run.")+ return+ }+ if f.Err != "" {+ fail("forwarding.probe_ran", f.Err,+ "The probe could not finish, so delivery is unproven. Check vector.out in the run directory.")+ }++ switch {+ case f.VectorExe == "":+ fail("forwarding.vector_path", "the running Vector's executable was not observed",+ "Without it the run cannot say which Vector forwarded.")+ case f.VectorExe != f.WantVectorExe:+ fail("forwarding.vector_path",+ fmt.Sprintf("Vector ran from %s, not %s", f.VectorExe, f.WantVectorExe),+ "The probe is meant to exercise the Vector the Linux packages install. A different "++ "binary means the result says nothing about the bundled one.")+ }++ runtimePosts := 0+ var badAuth []string+ switch {+ case f.RequestsErr != nil:+ fail("forwarding.authenticated", "the stand-in's request log could not be read: "+f.RequestsErr.Error(),+ "Without it there is no record that Vector sent the device key.")+ default:+ for i, r := range f.Requests {+ if r.Method == "POST" && r.Path == "/v1/ingest/runtime" {+ runtimePosts+++ }+ if !r.AuthOK {+ badAuth = append(badAuth, fmt.Sprintf("request %d: %s %s", i+1, r.Method, r.Path))+ }+ }+ if len(badAuth) > 0 {+ fail("forwarding.authenticated",+ fmt.Sprintf("%d request(s) did not carry the device key from the secrets file", len(badAuth)),+ "Vector reads the key through its secret backend. A request without it is what a "++ "revoked device looks like to the ingest service, so every batch would be refused.",+ badAuth...)+ }+ }++ if f.ReceivedErr != nil && !errors.Is(f.ReceivedErr, fs.ErrNotExist) {+ fail("forwarding.delivered", "the forwarded events could not be read: "+f.ReceivedErr.Error(), "")+ }+ var validation, session int+ var unparseable, altered []string+ runtimeLines := map[string]bool{}+ for _, e := range f.Runtime.Events {+ runtimeLines[e.Raw] = true+ }+ for _, e := range f.Received.Events {+ loc := fmt.Sprintf("%s:%d", f.Received.Path, e.Line)+ if e.ParseErr != nil {+ unparseable = append(unparseable, loc)+ continue+ }+ if isAsymptoteValidation(e) {+ validation+++ } else if f.Canary != "" && strings.Contains(e.Raw, f.Canary) {+ session+++ }+ if f.RuntimeErr == nil && !runtimeLines[e.Raw] {+ altered = append(altered, loc)+ }+ }+ if len(unparseable) > 0 {+ fail("forwarding.intact", fmt.Sprintf("%d forwarded line(s) are not valid JSON", len(unparseable)),+ "Beacon writes one JSON object per line and Vector frames them by newline. A partial or "++ "merged line means the framing is wrong, and ingest would reject it.",+ unparseable...)+ }+ switch {+ case f.RuntimeErr != nil:+ fail("forwarding.intact", "the runtime log could not be re-read after forwarding: "+f.RuntimeErr.Error(),+ "Without it the forwarded lines cannot be compared with what Beacon wrote.")+ case len(altered) > 0:+ fail("forwarding.intact",+ fmt.Sprintf("%d forwarded line(s) do not appear verbatim in Beacon's runtime log", len(altered)),+ "Standard mode ships Beacon's lines unchanged. A line that differs was altered or "++ "invented between the file and the ingest service.",+ altered...)+ }+ if validation == 0 {+ fail("forwarding.delivered", "the validation event never reached the stand-in",+ "`beacon endpoint asymptote validate` writes it right after Vector starts. If it did not "++ "arrive, Vector is not shipping the runtime log at all.")+ }+ if session == 0 {+ fail("forwarding.delivered", "no event from the agent session reached the stand-in",+ "The session's events carry this run's canary and are written minutes after Vector "++ "starts. Without them the run only shows that Vector shipped its first batch.")+ }++ var leaks []string+ for _, l := range []Log{f.Received, f.Runtime} {+ for _, e := range l.Events {+ if DeviceKeyPattern.MatchString(e.Raw) {+ leaks = append(leaks, fmt.Sprintf("%s:%d", l.Path, e.Line))+ }+ }+ }+ names := make([]string, 0, len(f.Text))+ for name := range f.Text {+ names = append(names, name)+ }+ sort.Strings(names)+ for _, name := range names {+ if DeviceKeyPattern.MatchString(f.Text[name]) {+ leaks = append(leaks, name)+ }+ }+ if len(leaks) > 0 {+ fail("forwarding.no_device_key_leak", "a device key appears in collected output",+ "The key's one home is the 0600 secrets file. Anywhere else it is a disclosure.",+ leaks...)+ }++ if len(v.Failures()) == 0 {+ v.Add(Finding{+ Check: "forwarding.delivered",+ Severity: SevInfo,+ Summary: fmt.Sprintf("the bundled Vector (%s) forwarded %d event(s) in %d authenticated "++ "request(s): the validation event and %d from the session", f.VectorExe,+ len(f.Received.Events), runtimePosts, session),+ })+ }+}diff --git a/beacon-sandbox/check/forwarding_test.go b/beacon-sandbox/check/forwarding_test.gonew file mode 100644index 00000000..3dd22e6d--- /dev/null+++ b/beacon-sandbox/check/forwarding_test.go@@ -0,0 +1,175 @@+package check++import (+ "errors"+ "os"+ "path/filepath"+ "strings"+ "testing"+)++const (+ fwdCanary = "BEACON_SANDBOX_fwd123"+ fwdVector = "/opt/beacon/bin/vector"+ fwdValidation = `{"vendor":"beacon","event":{"action":"agent.detected"},"destination":{"type":"asymptote","mode":"asymptote_managed_http"},"message":"Beacon endpoint Asymptote validation event"}`+ fwdPrompt = `{"vendor":"beacon","event":{"action":"prompt.submitted"},"prompt":{"text":"echo BEACON_SANDBOX_fwd123"}}`+ fwdOther = `{"vendor":"beacon","event":{"action":"token.usage"}}`+)++func logOf(t *testing.T, name string, lines ...string) Log {+ t.Helper()+ p := filepath.Join(t.TempDir(), name)+ if err := os.WriteFile(p, []byte(strings.Join(lines, "\n")+"\n"), 0o600); err != nil {+ t.Fatal(err)+ }+ l, err := ReadLog(p)+ if err != nil {+ t.Fatal(err)+ }+ return l+}++// healthyForward is a run where everything arrived: the validation event, a session event, every+// request authenticated, and every forwarded line present in the runtime log.+func healthyForward(t *testing.T) Forward {+ return Forward{+ Asked: true,+ Ran: "true",+ VectorExe: fwdVector,+ WantVectorExe: fwdVector,+ Canary: fwdCanary,+ Received: logOf(t, "forwarded.ndjson", fwdValidation, fwdPrompt),+ Requests: []ForwardRequest{+ {Method: "GET", Path: "/v1/ingest/health", AuthOK: true},+ {Method: "POST", Path: "/v1/ingest/runtime", AuthOK: true, Lines: 2, Encoding: "gzip"},+ },+ Runtime: logOf(t, "runtime.jsonl", fwdOther, fwdValidation, fwdPrompt),+ Text: map[string]string{"vector.out": "INFO vector: Vector has started."},+ }+}++func judgeForward(f Forward) Verdict {+ v := Verdict{Scenario: "i04"}+ Forwarding(&v, f)+ v.Resolve()+ return v+}++func failedChecks(v Verdict) string {+ var names []string+ for _, f := range v.Failures() {+ names = append(names, f.Check+": "+f.Summary)+ }+ return strings.Join(names, "\n")+}++func TestForwardingPassesAHealthyRun(t *testing.T) {+ v := judgeForward(healthyForward(t))+ if v.Outcome != Pass {+ t.Fatalf("a healthy forwarding run failed:\n%s", failedChecks(v))+ }+ if len(v.Findings) != 1 || v.Findings[0].Severity != SevInfo {+ t.Fatalf("want one info finding summarising delivery, got %+v", v.Findings)+ }+}++// A scenario that did not ask gets no finding at all, while one that asked and never ran fails.+func TestForwardingDistinguishesNotAskedFromNotRun(t *testing.T) {+ v := judgeForward(Forward{})+ if len(v.Findings) != 0 {+ t.Fatalf("a scenario that did not ask must produce nothing, got %+v", v.Findings)+ }+ v = judgeForward(Forward{Asked: true})+ if v.Outcome != Fail || !strings.Contains(failedChecks(v), "forwarding.probe_ran") {+ t.Fatalf("an asked-for probe that never ran must fail, got %s:\n%s", v.Outcome, failedChecks(v))+ }+}++// Each requirement has to be able to fail on its own. A mutation that fails every check at once+// would not show which one is doing the work.+func TestForwardingFailsEachRequirementIndependently(t *testing.T) {+ cases := []struct {+ name string+ mutate func(*Forward)+ check string+ }{+ {"probe error", func(f *Forward) { f.Err = "vector exited before it was ready" }, "forwarding.probe_ran"},+ {"vector not observed", func(f *Forward) { f.VectorExe = "" }, "forwarding.vector_path"},+ {"another vector", func(f *Forward) { f.VectorExe = "/usr/bin/vector" }, "forwarding.vector_path"},+ {"unauthenticated batch", func(f *Forward) { f.Requests[1].AuthOK = false }, "forwarding.authenticated"},+ {"request log missing", func(f *Forward) { f.Requests, f.RequestsErr = nil, os.ErrNotExist }, "forwarding.authenticated"},+ {"no validation event", func(f *Forward) {+ f.Received = logOf(t, "forwarded.ndjson", fwdPrompt)+ }, "forwarding.delivered"},+ {"no session event", func(f *Forward) {+ f.Received = logOf(t, "forwarded.ndjson", fwdValidation)+ }, "forwarding.delivered"},+ {"nothing received", func(f *Forward) {+ f.Received, f.ReceivedErr = Log{}, os.ErrNotExist+ }, "forwarding.delivered"},+ {"altered line", func(f *Forward) {+ f.Received = logOf(t, "forwarded.ndjson", fwdValidation,+ strings.Replace(fwdPrompt, "prompt.submitted", "prompt.edited", 1))+ }, "forwarding.intact"},+ {"merged lines", func(f *Forward) {+ f.Received = logOf(t, "forwarded.ndjson", fwdValidation, fwdPrompt[:40])+ }, "forwarding.intact"},+ {"runtime log unreadable", func(f *Forward) { f.RuntimeErr = errors.New("gone") }, "forwarding.intact"},+ {"key in a forwarded event", func(f *Forward) {+ leaked := strings.Replace(fwdPrompt, `"}}`, ` bcn_device_test_0123456789abcdef"}}`, 1)+ f.Received = logOf(t, "forwarded.ndjson", fwdValidation, leaked)+ f.Runtime = logOf(t, "runtime.jsonl", fwdValidation, leaked)+ }, "forwarding.no_device_key_leak"},+ {"key in vector's output", func(f *Forward) {+ f.Text["vector.out"] = "token=bcn_device_test_0123456789abcdef"+ }, "forwarding.no_device_key_leak"},+ }+ for _, tc := range cases {+ t.Run(tc.name, func(t *testing.T) {+ f := healthyForward(t)+ tc.mutate(&f)+ v := judgeForward(f)+ if v.Outcome != Fail {+ t.Fatalf("want FAIL, got %s", v.Outcome)+ }+ if got := failedChecks(v); !strings.Contains(got, tc.check) {+ t.Fatalf("want a %s failure, got:\n%s", tc.check, got)+ }+ })+ }+}++// The session's events are the evidence that Vector kept tailing; the validation event alone must+// not count as one, even though the canary is not in it.+func TestForwardingDoesNotCountTheValidationEventAsSession(t *testing.T) {+ f := healthyForward(t)+ f.Canary = "asymptote" // present in the validation event, absent from the prompt+ v := judgeForward(f)+ if !strings.Contains(failedChecks(v), "no event from the agent session") {+ t.Fatalf("the validation event was counted as a session event:\n%s", failedChecks(v))+ }+}++func TestReadForwardRequestsRejectsAMalformedLine(t *testing.T) {+ p := filepath.Join(t.TempDir(), "requests.jsonl")+ body := `{"method":"GET","path":"/v1/ingest/health","auth_ok":true}` + "\nnot json\n"+ if err := os.WriteFile(p, []byte(body), 0o600); err != nil {+ t.Fatal(err)+ }+ if _, err := ReadForwardRequests(p); err == nil {+ t.Fatal("a malformed request log must be an error, since our own stand-in wrote it")+ }+}++func TestDeviceKeyPatternMatchesTestAndRealKeys(t *testing.T) {+ for _, s := range []string{"bcn_device_test_0123456789abcdef0123456789abcdef", "Bearer bcn_device_Ab12Cd34Ef56"} {+ if !DeviceKeyPattern.MatchString(s) {+ t.Errorf("%q should match", s)+ }+ }+ for _, s := range []string{"device_key", "bcn_device_", "the bcn_device prefix"} {+ if DeviceKeyPattern.MatchString(s) {+ t.Errorf("%q should not match", s)+ }+ }+}diff --git a/beacon-sandbox/cmd/beacon-sandbox/main.go b/beacon-sandbox/cmd/beacon-sandbox/main.goindex 05117294..52c20fa0 100644--- a/beacon-sandbox/cmd/beacon-sandbox/main.go+++ b/beacon-sandbox/cmd/beacon-sandbox/main.go@@ -22,6 +22,7 @@ import ("github.com/asymptote-labs/agent-beacon/beacon-sandbox/check""github.com/asymptote-labs/agent-beacon/beacon-sandbox/credentials""github.com/asymptote-labs/agent-beacon/beacon-sandbox/dispatch"+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/image""github.com/asymptote-labs/agent-beacon/beacon-sandbox/runner""github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox""github.com/asymptote-labs/agent-beacon/beacon-sandbox/scenario"@@ -428,10 +429,42 @@ func judge(sc scenario.Scenario, art runner.Artifacts, creds credentials.Resolvecheck.Reinstall(&v, lifecycle)check.UnprivilegedUninstall(&v, lifecycle)check.FailedReinstallRollback(&v, lifecycle)+ check.Forwarding(&v, forwardOf(sc, art))v.Resolve()return v, log}+// forwardOf gathers what the forwarding probe left behind. Everything comes from the run directory+// and meta, never from the live probe, so `verify` re-judges a saved run the same way.+func forwardOf(sc scenario.Scenario, art runner.Artifacts) check.Forward {+ f := check.Forward{+ Asked: sc.VerifiesVectorForwarding(),+ Ran: art.Meta["forward_ran"],+ Err: art.Meta["forward_error"],+ VectorExe: art.Meta["forward_vector_exe"],+ WantVectorExe: image.VectorPath,+ Canary: art.Canary,+ }+ if !f.Asked {+ return f+ }+ f.Received, f.ReceivedErr = check.ReadLog(filepath.Join(art.Dir, runner.ForwardedLogName))+ f.Requests, f.RequestsErr = check.ReadForwardRequests(filepath.Join(art.Dir, runner.ForwardRequestsName))+ f.Runtime, f.RuntimeErr = check.ReadLog(filepath.Join(art.Dir, runner.ForwardRuntimeName))+ // Everything else collected, searched for a device key. Meta holds the probe's own output.+ f.Text = map[string]string{}+ for _, name := range []string{runner.ForwardVectorOutName, runner.ForwardRequestsName,+ "runtime.jsonl", "claude-result.json"} {+ if b, err := os.ReadFile(filepath.Join(art.Dir, name)); err == nil {+ f.Text[name] = string(b)+ }+ }+ if b, err := json.Marshal(art.Meta); err == nil {+ f.Text["meta"] = string(b)+ }+ return f+}+// safetyOf folds the out-of-band safety signals into a verdict.//// Shared by the normal and the run-failed paths so neither can quietly omit them: host escape and@@ -588,7 +621,7 @@ func judgeRunDir(root, dir, mutate string) (check.Verdict, error) {}}- art := runner.Artifacts{RuntimeLog: logPath, Meta: meta}+ art := runner.Artifacts{Dir: dir, RuntimeLog: logPath, Meta: meta}art.Canary = meta["canary"]// Sentinel state is not re-derivable offline, so trust what the run recorded -- including// the detail, which is the evidence a finding quotes. Dropping it made re-judged verdictsdiff --git a/beacon-sandbox/image/image.go b/beacon-sandbox/image/image.goindex 4654407b..5ef309bc 100644--- a/beacon-sandbox/image/image.go+++ b/beacon-sandbox/image/image.go@@ -31,6 +31,20 @@ const (// build. Empty string means "latest", which makes runs non-reproducible.DefaultClaudeVersion = "2.1.220"+ // VectorVersion and VectorSHA256 pin the Vector that WithVector installs, which is the release+ // and build the Linux packages bundle. musl because it is the static build: the gnu build links+ // glibc's loader and would not run on every host the packages support.+ //+ // 0.56.0 because it is the newest release that expands ${VAR} and ${VAR:-default} in config+ // files by default, which every pack Beacon renders relies on. 0.57 stops expanding the+ // defaulted form and 0.58 stops expanding either without an opt-in flag. Keep this in step with+ // packaging/linux/fetch-vector.sh.+ VectorVersion = "0.56.0"+ VectorSHA256 = "8c114c5e9fd9646516f014d5d837690447cf0d4f43ba4a3746713bc0612b039b"+ // VectorPath is where the Linux packages install Vector, and the first fixed location Beacon's+ // Vector discovery tries after --vector-bin and BEACON_VECTOR_BIN.+ VectorPath = BeaconDir + "/vector"+// DefaultBase is the only distro in the matrix today. M0 established that Modal// offers no arch selection, so this is amd64.DefaultBase = "ubuntu:24.04"@@ -189,6 +203,10 @@ type Spec struct {// a directory-backed fleet. See scenario.Install.NSSOnlyUser for why that distinction decides// whether a system-mode install can configure anyone at all.NSSOnlyUser bool++ // WithVector installs the release Vector at VectorPath, where the Linux packages put it. See+ // scenario.Install.VerifyVectorForwarding.+ WithVector bool}// LinuxArtifacts are the binaries the sandbox needs, as host paths.@@ -285,6 +303,10 @@ func Build(spec Spec, log func(string, ...any)) (sandbox.ImageSpec, error) {AgentUser, WorkDir, WorkDir),)+ if spec.WithVector {+ layers = append(layers, vectorLayer())+ }+if spec.WithDocker {// Group membership is set at image build time; adding it later would need a re-login to// take effect, and the session runs as this user.@@ -346,6 +368,35 @@ func accountLayers(nssOnly bool) []string {}}+// vectorLayer downloads the release Vector and installs it where the Linux packages do.+//+// At image build because that is the only moment with unrestricted egress; a running sandbox can+// reach nothing but the model API. Appended after the shared layers so every other scenario keeps+// its cached image.+//+// Each check fails the build instead of a paid run. The archive must match the pinned hash, and+// the release's own SHA256SUMS must list the same hash, so a typo in the pin is caught along with+// a bad download. The binary must have no PT_INTERP program header, because the reason to ship the+// musl build is that it is static; this is read with python3 since the image has no readelf. The+// layer ends by running `vector --version`.+func vectorLayer() string {+ archive := fmt.Sprintf("vector-%s-x86_64-unknown-linux-musl.tar.gz", VectorVersion)+ base := "https://packages.timber.io/vector/" + VectorVersion+ noInterp := `python3 -c 'import struct,sys; f=open(sys.argv[1],"rb").read(); ` ++ `o=struct.unpack_from("<Q",f,0x20)[0]; e,n=struct.unpack_from("<HH",f,0x36); ` ++ `sys.exit(any(struct.unpack_from("<I",f,o+i*e)[0]==3 for i in range(n)))' ` + VectorPath+ return "RUN set -e; cd /tmp; " ++ fmt.Sprintf("curl -fsSLO %s/%s; ", base, archive) ++ fmt.Sprintf("curl -fsSL %s/vector-%s-SHA256SUMS -o vector-sums; ", base, VectorVersion) ++ fmt.Sprintf("echo '%s %s' | sha256sum -c -; ", VectorSHA256, archive) ++ fmt.Sprintf("grep -q '^%s %s$' vector-sums; ", VectorSHA256, archive) ++ fmt.Sprintf("mkdir vector-x && tar -xzf %s -C vector-x; ", archive) ++ fmt.Sprintf(`install -m 0755 "$(find vector-x -path '*/bin/vector' -type f | head -1)" %s; `, VectorPath) ++ fmt.Sprintf("rm -rf vector-x vector-sums %s; ", archive) ++ noInterp + " || { echo 'vector is dynamically linked' >&2; exit 1; }; " ++ VectorPath + " --version"+}+// PostPushLayers are the commands that must run after the artifact files land. The// provider snapshots after Put, so these run in the first launched instance instead.func PostPushLayers() string {diff --git a/beacon-sandbox/image/image_test.go b/beacon-sandbox/image/image_test.goindex e64f228a..9e406804 100644--- a/beacon-sandbox/image/image_test.go+++ b/beacon-sandbox/image/image_test.go@@ -650,3 +650,44 @@ func TestStaleProvenanceFallsBackToTheMtimeComparison(t *testing.T) {t.Errorf("a local rebuild from current sources is fresh, got: %s", why)}}++// The Vector layer is what i04 exists to test, so a layer that installed the wrong build, or+// skipped its checks, would make that scenario verify something the packages do not ship. It is+// also appended only on request, so every other scenario keeps its cached image.+func TestBuildInstallsTheBundledVectorOnlyWhenAsked(t *testing.T) {+ spec, err := Build(Spec{RepoRoot: stubRepo(t)}, nil)+ if err != nil {+ t.Fatal(err)+ }+ if joined := strings.Join(spec.Layers, "\n"); strings.Contains(joined, VectorPath) {+ t.Fatalf("Vector must not be installed unless a scenario asks for it:\n%s", joined)+ }++ spec, err = Build(Spec{RepoRoot: stubRepo(t), WithVector: true}, nil)+ if err != nil {+ t.Fatal(err)+ }+ var layer string+ for _, l := range spec.Layers {+ if strings.Contains(l, VectorPath) {+ layer = l+ }+ }+ if layer == "" {+ t.Fatal("WithVector added no layer that installs Vector")+ }+ for _, want := range []string{+ "vector-" + VectorVersion + "-x86_64-unknown-linux-musl.tar.gz",+ VectorSHA256 + " vector-" + VectorVersion,+ "sha256sum -c -",+ "vector-" + VectorVersion + "-SHA256SUMS",+ "dynamically linked",+ } {+ if !strings.Contains(layer, want) {+ t.Errorf("the Vector layer is missing %q:\n%s", want, layer)+ }+ }+ if !strings.HasSuffix(layer, VectorPath+" --version") {+ t.Errorf("the layer must end by running Vector, so a bad binary fails the build:\n%s", layer)+ }+}diff --git a/beacon-sandbox/runner/forward.go b/beacon-sandbox/runner/forward.gonew file mode 100644index 00000000..ec0971b6--- /dev/null+++ b/beacon-sandbox/runner/forward.go@@ -0,0 +1,284 @@+package runner++import (+ "context"+ "fmt"+ "path/filepath"+ "strconv"+ "strings"+ "time"++ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/image"+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"+)++// Guest paths for the forwarding probe. The secrets file is where a user-mode `beacon endpoint+// connect` puts it, so Vector reads the key the way the managed forwarder does.+const (+ forwardBaseDir = image.AgentHome + "/.beacon/endpoint/asymptote"+ forwardSecretsFile = forwardBaseDir + "/vector-secrets.json"+ forwardDataDir = forwardBaseDir + "/vector-data"+ forwardPackDir = forwardBaseDir + "/pack"+ forwardVectorOut = forwardBaseDir + "/vector.out"+ forwardIngestDir = "/tmp/beacon-ingest"+ forwardStandInPath = "/tmp/beacon-ingest-standin.py"+)++// Artifact names in the run directory, read back by the offline judge.+const (+ ForwardedLogName = "forwarded-runtime.ndjson"+ ForwardRequestsName = "forward-requests.jsonl"+ ForwardVectorOutName = "vector.out"+ // ForwardRuntimeName is a second copy of the runtime log taken after forwarding finished.+ // runtime.jsonl is collected earlier, at quiescence, and Vector may ship a line written after+ // that; comparing against the earlier copy would report such a line as invented.+ ForwardRuntimeName = "runtime-after-forwarding.jsonl"+)++// forwardStandIn serves the three ingest routes Vector's Asymptote pack calls, on loopback.+//+// It checks the bearer token against the secrets file and records only whether it matched, so the+// key never reaches its output. Runtime and inventory bodies are gunzipped and appended one line+// per event, which is the NDJSON contract of the real service.+const forwardStandIn = `import gzip, http.server, json, os, sys, threading++out_dir, secrets_path = sys.argv[1], sys.argv[2]+lock = threading.Lock()+routes = {"/v1/ingest/runtime": "runtime.ndjson", "/v1/ingest/inventory": "inventory.ndjson"}++def device_key():+ try:+ with open(secrets_path) as f:+ return json.load(f).get("device_key", "")+ except Exception:+ return ""++def append(name, text):+ with lock, open(os.path.join(out_dir, name), "a") as f:+ f.write(text)++class Handler(http.server.BaseHTTPRequestHandler):+ protocol_version = "HTTP/1.1"++ def auth_ok(self):+ key = device_key()+ return bool(key) and self.headers.get("Authorization", "") == "Bearer " + key++ def body(self):+ if self.headers.get("Transfer-Encoding", "").lower() == "chunked":+ data = b""+ while True:+ size = int(self.rfile.readline().strip() or b"0", 16)+ if size == 0:+ self.rfile.readline()+ return data+ data += self.rfile.read(size)+ self.rfile.readline()+ return self.rfile.read(int(self.headers.get("Content-Length") or 0))++ def reply(self, code):+ self.send_response(code)+ self.send_header("Content-Length", "0")+ self.end_headers()++ def do_GET(self):+ ok = self.auth_ok()+ append("requests.jsonl", json.dumps({"method": self.command, "path": self.path, "auth_ok": ok}) + "\n")+ self.reply(200 if ok else 401)++ # Vector's http sink healthcheck sends HEAD, not GET.+ do_HEAD = do_GET++ def do_POST(self):+ data = self.body()+ encoding = self.headers.get("Content-Encoding", "")+ if encoding == "gzip":+ data = gzip.decompress(data)+ lines = [l for l in data.decode("utf-8", "replace").split("\n") if l.strip()]+ ok = self.auth_ok()+ name = routes.get(self.path)+ if ok and name:+ append(name, "".join(l + "\n" for l in lines))+ append("requests.jsonl", json.dumps({"method": "POST", "path": self.path, "auth_ok": ok,+ "lines": len(lines), "encoding": encoding}) + "\n")+ self.reply(401 if not ok else (200 if name else 404))++ def log_message(self, *args):+ pass++server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)+with open(os.path.join(out_dir, "port.tmp"), "w") as f:+ f.write(str(server.server_address[1]))+os.rename(os.path.join(out_dir, "port.tmp"), os.path.join(out_dir, "port"))+server.serve_forever()+`++// forwardSetup starts the stand-in and the bundled Vector, then writes the validation event.+//+// Runs after install and before the session, because the pack reads the runtime log from the end:+// anything written before Vector starts is never shipped, which is the managed forwarder's+// contract too.+//+// The device key is generated inside the guest with shell builtins and written straight to the+// 0600 secrets file. Every script passed to Exec lands in a process's argv, so a key made on the+// host would be disclosed by the harness itself. Nothing outside the guest ever holds it; the leak+// checks search for its shape instead.+func forwardSetup(ctx context.Context, g guest, agent sandbox.ExecOpts, logPath string,+ art *Artifacts, logf func(string, ...any)) error {++ art.Meta["forward_ran"] = "true"+ fail := func(format string, a ...any) error {+ err := fmt.Errorf(format, a...)+ art.Meta["forward_error"] = oneLine(err.Error())+ logf("forwarding: %v", err)+ return err+ }++ start := fmt.Sprintf(`set -eu+umask 077+mkdir -p %[1]s %[2]s %[3]s+chmod 0700 %[1]s+key="bcn_device_test_$(od -An -tx1 -N16 /dev/urandom | tr -d ' \n')"+printf '{"device_key": "%%s"}\n' "$key" > %[4]s+unset key+chmod 0600 %[4]s+cat > %[5]s <<'PY'+%[6]sPY+rm -f %[3]s/port+setsid nohup python3 %[5]s %[3]s %[4]s > %[3]s/standin.log 2>&1 < /dev/null &+i=0+while [ ! -s %[3]s/port ] && [ "$i" -lt 50 ]; do i=$((i+1)); sleep 0.2; done+if [ ! -s %[3]s/port ]; then echo "the ingest stand-in did not start: $(tail -c 400 %[3]s/standin.log)"; exit 1; fi+echo "FORWARD_PORT=$(cat %[3]s/port)"`,+ forwardBaseDir, forwardDataDir, forwardIngestDir, forwardSecretsFile, forwardStandInPath, forwardStandIn)+ r, err := g.Exec(ctx, start, agent)+ if err != nil || r.ExitCode != 0 {+ return fail("could not start the ingest stand-in (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stdout+r.Stderr))+ }+ port := ""+ for _, line := range strings.Split(r.Stdout, "\n") {+ if v, ok := strings.CutPrefix(strings.TrimSpace(line), "FORWARD_PORT="); ok {+ port = v+ }+ }+ if _, err := strconv.Atoi(port); err != nil {+ return fail("the ingest stand-in reported no port: %s", oneLine(r.Stdout))+ }+ art.Meta["forward_port"] = port++ env := fmt.Sprintf("BEACON_ASYMPTOTE_INGEST_URL=http://127.0.0.1:%s BEACON_ASYMPTOTE_SECRETS_FILE=%s "++ "BEACON_ASYMPTOTE_DATA_DIR=%s", port, forwardSecretsFile, forwardDataDir)+ pack := fmt.Sprintf(`set -eu+beacon endpoint asymptote install-pack --user --output %[1]s --log-path %[2]s+export %[3]s+%[4]s --version+%[4]s validate --skip-healthchecks %[1]s/vector.toml`,+ forwardPackDir, shq(logPath), env, image.VectorPath)+ r, err = g.Exec(ctx, pack+" 2>&1", agent)+ art.Meta["forward_pack_validate"] = oneLine(r.Stdout)+ if err != nil || r.ExitCode != 0 {+ return fail("vector rejected the Asymptote pack (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stdout))+ }+ for _, line := range strings.Split(r.Stdout, "\n") {+ if line = strings.TrimSpace(line); strings.HasPrefix(line, "vector ") {+ art.Meta["forward_vector_version"] = oneLine(line)+ break+ }+ }++ // Started by the packaged path, then identified from /proc: that is what the check compares+ // against, rather than trusting the path this script typed.+ //+ // Ready means Vector's startup healthcheck reached the stand-in, which happens after its+ // sources are built. The extra seconds cover the file source's first scan, which records the+ // end of the log; a line written before that is not shipped.+ run := fmt.Sprintf(`set -u+export %[1]s+setsid nohup %[2]s --config %[3]s/vector.toml > %[4]s 2>&1 < /dev/null &+i=0+while [ "$i" -lt 60 ]; do+ pid="$(pgrep -n -x vector || true)"+ if [ -n "$pid" ] && grep -q '"path": "/v1/ingest/health"' %[5]s/requests.jsonl 2>/dev/null; then break; fi+ i=$((i+1)); sleep 0.5+done+pid="$(pgrep -n -x vector || true)"+if [ -z "$pid" ]; then echo "vector exited: $(tail -c 600 %[4]s)"; exit 1; fi+if ! grep -q '"path": "/v1/ingest/health"' %[5]s/requests.jsonl 2>/dev/null; then+ echo "vector never called the health check: $(tail -c 600 %[4]s)"; exit 1+fi+sleep 3+echo "FORWARD_VECTOR_EXE=$(readlink /proc/$pid/exe)"`,+ env, image.VectorPath, forwardPackDir, forwardVectorOut, forwardIngestDir)+ r, err = g.Exec(ctx, run, agent)+ for _, line := range strings.Split(r.Stdout, "\n") {+ if v, ok := strings.CutPrefix(strings.TrimSpace(line), "FORWARD_VECTOR_EXE="); ok {+ art.Meta["forward_vector_exe"] = v+ }+ }+ if err != nil || r.ExitCode != 0 {+ return fail("vector did not start (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stdout))+ }+ logf("forwarding: vector %s running from %s, stand-in on 127.0.0.1:%s",+ art.Meta["forward_vector_version"], art.Meta["forward_vector_exe"], port)++ r, err = g.Exec(ctx, fmt.Sprintf("beacon endpoint asymptote validate --user --log-path %s 2>&1", shq(logPath)), agent)+ art.Meta["forward_validate_output"] = oneLine(r.Stdout)+ if err != nil || r.ExitCode != 0 {+ return fail("beacon endpoint asymptote validate failed (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stdout))+ }+ return nil+}++// forwardCollect waits for the session's events to reach the stand-in, then brings its record+// home.+//+// The pack batches for up to 60 seconds, so the wait is a poll for the two things the check needs,+// bounded well past one batch. It is skipped when setup failed, since nothing is running to wait+// for; whatever the stand-in and Vector wrote is still collected. Vector and the stand-in are+// stopped afterwards; on a disposable sandbox that is tidiness, on a machine that outlives the run+// it is required.+func forwardCollect(ctx context.Context, g guest, agent sandbox.ExecOpts, logPath, canary, runDir string,+ setupOK bool, art *Artifacts, logf func(string, ...any)) {++ if !setupOK {+ collectForwardArtifacts(ctx, g, agent, logPath, runDir, art, logf)+ return+ }+ start := time.Now()+ wait := fmt.Sprintf(`i=0+f=%[1]s/runtime.ndjson+while [ "$i" -lt 75 ]; do+ if grep -q 'asymptote_managed_http' "$f" 2>/dev/null && grep -q -F %[2]s "$f" 2>/dev/null; then echo FORWARD_ARRIVED; break; fi+ i=$((i+1)); sleep 2+done+if [ -n "$(pgrep -n -x vector || true)" ]; then echo VECTOR_ALIVE=true; else echo VECTOR_ALIVE=false; fi`,+ forwardIngestDir, shq(canary))+ r, err := g.Exec(ctx, wait, agent)+ art.Meta["forward_wait_seconds"] = fmt.Sprintf("%.0f", time.Since(start).Seconds())+ art.Meta["forward_arrived"] = fmt.Sprintf("%v", err == nil && strings.Contains(r.Stdout, "FORWARD_ARRIVED"))+ art.Meta["forward_vector_alive"] = fmt.Sprintf("%v", strings.Contains(r.Stdout, "VECTOR_ALIVE=true"))+ logf("forwarding: arrived=%s after %ss, vector alive=%s",+ art.Meta["forward_arrived"], art.Meta["forward_wait_seconds"], art.Meta["forward_vector_alive"])++ collectForwardArtifacts(ctx, g, agent, logPath, runDir, art, logf)+}++// collectForwardArtifacts stops Vector and the stand-in and copies their records home.+func collectForwardArtifacts(ctx context.Context, g guest, agent sandbox.ExecOpts, logPath, runDir string,+ art *Artifacts, logf func(string, ...any)) {++ _, _ = g.Exec(ctx, "pkill -x vector; pkill -f "+shq(forwardStandInPath)+"; true", agent)++ for remote, name := range map[string]string{+ forwardIngestDir + "/runtime.ndjson": ForwardedLogName,+ forwardIngestDir + "/requests.jsonl": ForwardRequestsName,+ forwardVectorOut: ForwardVectorOutName,+ logPath: ForwardRuntimeName,+ } {+ if err := g.Get(ctx, remote, filepath.Join(runDir, name)); err != nil {+ art.Meta["forward_missing_"+strings.TrimSuffix(name, filepath.Ext(name))] = oneLine(err.Error())+ logf("forwarding: could not collect %s: %v", remote, err)+ }+ }+ art.ForwardedLog = filepath.Join(runDir, ForwardedLogName)+}diff --git a/beacon-sandbox/runner/runner.go b/beacon-sandbox/runner/runner.goindex bbfed751..00d09ac4 100644--- a/beacon-sandbox/runner/runner.go+++ b/beacon-sandbox/runner/runner.go@@ -52,6 +52,9 @@ type Artifacts struct {// ArgvCheckRan records whether the argv scan actually executed, so a skipped check is// never mistaken for a passing one.ArgvCheckRan bool+ // ForwardedLog is what the ingest stand-in received from the bundled Vector, for scenarios+ // that verify forwarding. The other forwarding artifacts sit beside it in Dir.+ ForwardedLog string// Meta records what produced this run.Meta map[string]string}@@ -274,6 +277,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts OptPreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: sc.Timeout() + time.Minute,}var res sandbox.Result+ forwardOK := falseif sc.Install != nil {installedLog, err := doInstall(ctx, g, sc, privileged, agent, lay, sh, &art, logf)// Uninstall afterwards when the sandbox is this machine, whether or not the install@@ -295,6 +299,12 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Optreturn art, err}logPath = installedLog+ // Before the session, because the pack tails the log from the end: only lines written+ // after Vector starts are shipped. A failed setup is recorded and the session still runs,+ // so the capture half of the verdict is not lost with it.+ if sc.VerifiesVectorForwarding() {+ forwardOK = forwardSetup(ctx, g, agent, logPath, &art, logf) == nil+ }logf("running session against the installed endpoint (budget $%.2f, timeout %s)",sc.Budget(), sc.Timeout())res, err = g.Exec(ctx, sh.PlainSession(sessionDir, claudeFlags(sc, prompt, sh)), sessionOpts)@@ -468,6 +478,10 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Optart.RuntimeLog = localLoglogf("collected %s", localLog)+ if sc.VerifiesVectorForwarding() {+ forwardCollect(ctx, g, agent, logPath, canary, runDir, forwardOK, &art, logf)+ }+// Reinstall before uninstall, for the same reason uninstall runs here at all: the log is already// on the host, so restarting the collector cannot disturb a capture assertion.if sc.VerifiesRestartOnReinstall() {@@ -614,6 +628,7 @@ func imageSpecFor(p sandbox.Platform, sc scenario.Scenario, opts Options,ClaudeVersion: opts.ClaudeVersion,WithDocker: sc.NeedsRealSystemd(),NSSOnlyUser: sc.NeedsNSSOnlyUser(),+ WithVector: sc.VerifiesVectorForwarding(),}, logf)}diff --git a/beacon-sandbox/runner/runner_test.go b/beacon-sandbox/runner/runner_test.goindex a321f1a0..86c5a23b 100644--- a/beacon-sandbox/runner/runner_test.go+++ b/beacon-sandbox/runner/runner_test.go@@ -7,6 +7,7 @@ import ("strings""testing"+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/check""github.com/asymptote-labs/agent-beacon/beacon-sandbox/image""github.com/asymptote-labs/agent-beacon/beacon-sandbox/scenario")@@ -260,3 +261,34 @@ func TestArgvSamplerEmitsWhetherItSawTheAgent(t *testing.T) {}}}++// The sampler's device-key matcher has to find a key on a command line and must not fire on the+// forwarding probe's own script, which spells the key's prefix while generating it. Run through+// the host's awk, since a pattern this tool only ever reads as a string proves nothing.+func TestArgvSamplerDeviceKeyMatcher(t *testing.T) {+ awk, err := exec.LookPath("awk")+ if err != nil {+ t.Skip("no awk on this host")+ }+ if !strings.Contains(posixShell{}.ArgvSampler(), "/"+deviceKeyAwkPattern+"/") {+ t.Fatal("the sampler does not use deviceKeyAwkPattern")+ }+ cases := map[string]bool{+ "vector --token bcn_device_test_0123456789abcdef": true,+ "curl -H Authorization: Bearer bcn_device_Ab12Cd34Ef56": true,+ `sh -c key="bcn_device_test_$(od -An -tx1 -N16 /dev/urandom)"`: false,+ `printf '{"device_key": "%s"}\n' "$key"`: false,+ "/opt/beacon/bin/vector --config /home/agent/.beacon/pack/vector.toml": false,+ }+ for line, want := range cases {+ cmd := exec.Command(awk, "/"+deviceKeyAwkPattern+"/ {found=1} END {exit !found}")+ cmd.Stdin = strings.NewReader(line + "\n")+ got := cmd.Run() == nil+ if got != want {+ t.Errorf("awk matcher on %q = %v, want %v", line, got, want)+ }+ if re := check.DeviceKeyPattern.MatchString(line); re != want {+ t.Errorf("check.DeviceKeyPattern on %q = %v, want %v; the two must agree", line, re, want)+ }+ }+}diff --git a/beacon-sandbox/runner/shell.go b/beacon-sandbox/runner/shell.goindex fdf66f21..d654e7af 100644--- a/beacon-sandbox/runner/shell.go+++ b/beacon-sandbox/runner/shell.go@@ -142,6 +142,10 @@ func (posixShell) ArgvVerdictPath() string { return "/tmp/beacon-sandbox-argv-ve// used `grep -qF "$KEY"`, which the shell expands into grep's argv, so the `ps` in the same// pipeline saw the grep process and reported a leak the tool had created. awk reads the key// from ENVIRON instead, and the sampler's own processes are skipped as a second guard.+//+// It also searches for an Asymptote device key by shape. The forwarding probe generates that key+// inside the guest, so no process outside it knows the value to search for; see+// deviceKeyAwkPattern.func (s posixShell) ArgvSampler() string {return fmt.Sprintf(`set -urm -f %[1]s@@ -158,6 +162,11 @@ scan() {index($0,k){found=1}END{exit !found}'}+scan_device() {+ awk '/awk|ARGV_SAMPLER_SELF/ {next}+ /%[3]s/ {found=1}+ END{exit !found}'+}n=0saw_agent=0# Bounded so a leaked sandbox cannot spin forever; the instance is destroyed long before this.@@ -168,7 +177,11 @@ while [ "$n" -lt 900 ]; do# process holding the key is not evidence of anything.case "$procs" in *claude*) saw_agent=1 ;; esacif printf '%%s\n' "$procs" | scan; then- echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=ps" > "$VERDICT"+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=ps what=api_key" > "$VERDICT"+ exit 0+ fi+ if printf '%%s\n' "$procs" | scan_device; then+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=ps what=device_key" > "$VERDICT"exit 0fifor f in /proc/[0-9]*/cmdline; do@@ -176,7 +189,11 @@ while [ "$n" -lt 900 ]; docmd="$(tr '\0' '\n' < "$f" 2>/dev/null || true)"case "$cmd" in *claude*) saw_agent=1 ;; esacif printf '%%s\n' "$cmd" | scan; then- echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=proc" > "$VERDICT"+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=proc what=api_key" > "$VERDICT"+ exit 0+ fi+ if printf '%%s\n' "$cmd" | scan_device; then+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=proc what=device_key" > "$VERDICT"exit 0fidone@@ -186,9 +203,16 @@ doneSAMPLERchmod +x %[2]snohup %[2]s %[1]s > /dev/null 2>&1 &-echo ARGV_SAMPLER_STARTED`, s.ArgvVerdictPath(), "/tmp/beacon-sandbox-argv-sampler.sh")+echo ARGV_SAMPLER_STARTED`, s.ArgvVerdictPath(), "/tmp/beacon-sandbox-argv-sampler.sh", deviceKeyAwkPattern)}+// deviceKeyAwkPattern is check.DeviceKeyPattern spelled for awk: "bcn_device_" and at least eight+// key characters. The eight are written out because mawk, Ubuntu's default awk, has not always+// supported interval expressions. The minimum length is what keeps the forwarding probe's own+// script from matching: it contains `bcn_device_test_$(od ...)`, five key characters and then `$`.+const deviceKeyAwkPattern = `bcn_device_[A-Za-z0-9_][A-Za-z0-9_][A-Za-z0-9_][A-Za-z0-9_]` ++ `[A-Za-z0-9_][A-Za-z0-9_][A-Za-z0-9_][A-Za-z0-9_]`+// ---------------------------------------------------------------------------// PowerShell// ---------------------------------------------------------------------------diff --git a/beacon-sandbox/scenario/scenario.go b/beacon-sandbox/scenario/scenario.goindex 818f33b4..ccd9e9ab 100644--- a/beacon-sandbox/scenario/scenario.go+++ b/beacon-sandbox/scenario/scenario.go@@ -119,6 +119,15 @@ type Install struct {// Only the account's *visibility* changes; the username, home directory and uid are otherwise// identical, so everything a scenario asserts stays comparable to the local-account lanes.NSSOnlyUser bool `yaml:"nss_only_user,omitempty"`+ // VerifyVectorForwarding runs the bundled Vector against Beacon's Asymptote pack and checks+ // what reached a loopback ingest stand-in.+ //+ // The Linux packages ship Vector at /opt/beacon/bin/vector, and it is the engine behind Beacon+ // Managed and six customer-managed destinations. Nothing else in the suite runs it, so a pack+ // Vector rejects, a secret backend that sends the wrong key, or a binary that does not start+ // on a stripped host would all ship unnoticed. The stand-in serves the ingest contract on+ // 127.0.0.1, so the run needs no account and no egress beyond the model API.+ VerifyVectorForwarding bool `yaml:"verify_vector_forwarding,omitempty"`}// Platform selects the guest OS family a scenario is written for.@@ -213,6 +222,11 @@ func (s Scenario) Validate() error {return fmt.Errorf("%s: platform %q must be linux or windows", s.ID, s.Platform)}if s.TargetPlatform() == PlatformWindows {+ // Vector is installed by the Linux image. The Windows runner is provisioned by CI and has+ // no bundled Vector to run.+ if s.VerifiesVectorForwarding() {+ return fmt.Errorf("%s: verify_vector_forwarding is Linux-only", s.ID)+ }// systemd does not exist on Windows, and the nested-container recipe that provides it is// a Linux arrangement. A scenario asking for both would be arranged as neither and would// then quietly verify whatever backend happened to be selected.@@ -257,6 +271,13 @@ func (s Scenario) Validate() error {return fmt.Errorf("%s: nss_only_user requires mode=system; a user-mode install "+"resolves $HOME and never looks the account up by name", s.ID)}+ // The probe runs Vector by hand as the agent user, against the user-mode log and secrets+ // file. System mode keeps both under root-owned paths, and the nested lane runs the+ // endpoint somewhere the stand-in is not, so either would need a different probe.+ if s.Install.VerifyVectorForwarding && (s.Install.Mode == "system" || s.Install.NeedsRealSystemd) {+ return fmt.Errorf("%s: verify_vector_forwarding needs a user-mode install outside the "++ "nested systemd lane, because the probe runs Vector as the agent user", s.ID)+ }// A user-mode uninstall needs no privileges, so there is no privilege failure to catch// and the probe would assert something that cannot happen.if s.Install.VerifyUnprivilegedUninstallFails && s.Install.Mode != "system" {@@ -458,6 +479,11 @@ func (s Scenario) NeedsNSSOnlyUser() bool {return s.Install != nil && s.Install.NSSOnlyUser}+// VerifiesVectorForwarding reports whether the bundled Vector must forward to a loopback stand-in.+func (s Scenario) VerifiesVectorForwarding() bool {+ return s.Install != nil && s.Install.VerifyVectorForwarding+}+// NeedsRealSystemd reports whether this scenario must run somewhere systemd is genuinely PID 1.func (s Scenario) NeedsRealSystemd() bool {return s.Install != nil && s.Install.NeedsRealSystemddiff --git a/beacon-sandbox/scenario/scenario_test.go b/beacon-sandbox/scenario/scenario_test.goindex 867c64fe..78fe800e 100644--- a/beacon-sandbox/scenario/scenario_test.go+++ b/beacon-sandbox/scenario/scenario_test.go@@ -436,3 +436,32 @@ func TestNeedsRealSystemdRejectsContradictions(t *testing.T) {t.Errorf("system+systemd is the intended combination: %v", err)}}++// The forwarding probe runs Vector by hand as the agent user against the user-mode log and secrets+// file, so it only means something in that arrangement.+func TestVerifyVectorForwardingRejectsArrangementsTheProbeCannotRun(t *testing.T) {+ base := func(in *Install) Scenario {+ return Scenario{ID: "i", Prompt: "do a thing", Install: in,+ Expect: []Expect{{Action: "prompt.submitted", Why: "baseline"}}}+ }+ for _, in := range []*Install{+ {VerifyVectorForwarding: true, Mode: "system", Service: "none"},+ {VerifyVectorForwarding: true, NeedsRealSystemd: true, Mode: "system", Service: "systemd"},+ } {+ if err := base(in).Validate(); err == nil {+ t.Errorf("%+v cannot run the forwarding probe and must be rejected", in)+ }+ }+ win := base(&Install{VerifyVectorForwarding: true, Mode: "user", Service: "none"})+ win.Platform = PlatformWindows+ if err := win.Validate(); err == nil {+ t.Error("a Windows scenario has no bundled Vector and must be rejected")+ }+ ok := base(&Install{VerifyVectorForwarding: true, Mode: "user", Service: "none"})+ if err := ok.Validate(); err != nil {+ t.Errorf("user mode with no service manager is the intended arrangement: %v", err)+ }+ if !ok.VerifiesVectorForwarding() {+ t.Error("VerifiesVectorForwarding should report the request")+ }+}diff --git a/beacon-sandbox/scenarios/i04-vector-forwarding.yaml b/beacon-sandbox/scenarios/i04-vector-forwarding.yamlnew file mode 100644index 00000000..4b0964ba--- /dev/null+++ b/beacon-sandbox/scenarios/i04-vector-forwarding.yaml@@ -0,0 +1,37 @@+id: i04-vector-forwarding+# Does the Vector that the Linux packages bundle actually forward Beacon's events?+#+# The .deb, .rpm and Linux archives ship Vector at /opt/beacon/bin/vector. It runs Beacon Managed+# forwarding and six customer-managed destinations, and nothing else in the suite starts it. So a+# pack that Vector rejects, a secret backend that sends the wrong key, or a Vector build that will+# not start on the target host would all go unnoticed.+#+# The image installs that Vector build at that path. After a user-mode install the probe renders+# Beacon's Asymptote pack, writes a throwaway device key to the 0600 secrets file, and runs Vector+# against a loopback stand-in for the ingest service. `beacon endpoint asymptote validate` writes+# the first event, and the session writes the rest. The check wants both at the stand-in,+# authenticated and byte-identical to the runtime log.+#+# User mode in the gVisor lane on purpose: there is no systemd here, so `beacon endpoint connect`+# and the forwarder service are out of reach. Vector runs by hand as the agent user, which is+# what the forwarder service does minus the service.+install:+ mode: user+ service: none+ expect_status_running: true+ expect_service_kind: none+ verify_vector_forwarding: true++prompt: >-+ Run exactly this shell command and report its output: echo {{canary}} | tee {{sentinel}}+sentinel: /home/agent/work/i04.sentinel+budget_usd: 1.0+timeout_seconds: 420++expect:+ - action: prompt.submitted+ contains: ["{{canary}}"]+ fields: ["prompt.text", "session.id", "harness.name"]+ why: >-+ The session's events are what the forwarding check looks for at the stand-in. If they never+ reached the runtime log, nothing downstream can be judged.diff --git a/docs/contributing/beacon-sandbox.mdx b/docs/contributing/beacon-sandbox.mdxindex 8ddf27e9..6eb2b15f 100644--- a/docs/contributing/beacon-sandbox.mdx+++ b/docs/contributing/beacon-sandbox.mdx@@ -205,6 +205,7 @@ go run ./cmd/beacon-sandbox run # all of them, ~30 min| `s07-denied-tool` | Approvals and denials || `i01-install-supervised` | `beacon endpoint install`, then a session against the installed collector || `i02-install-systemd` | A system-mode install managed by systemd, then a session against it |+| `i04-vector-forwarding` | The Vector the Linux packages ship, forwarding a session's events to a local stand-in for Beacon's ingest service |Use one test while you are iterating. Run all of them before opening a pull request.