The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu
Go agent-ai-cliagent-harnessesagent-securitybrowser-extensionciclaude-codecloudcodexcursordetection-engineeringjevknowledge-basememorymobile-device-managementobservabilitysecuritysecurity-information-and-event-managementsecurity-toolstelemetrytraces
feat(sandbox): run scenarios on a Windows guest via a disposable CI runner
shukieshaha1bf439c6081
diff --git a/.github/workflows/windows-sandbox.yml b/.github/workflows/windows-sandbox.ymlnew file mode 100644index 00000000..7cd8631a--- /dev/null+++ b/.github/workflows/windows-sandbox.yml@@ -0,0 +1,186 @@+# Runs a beacon-sandbox scenario against a real Claude Code session on Windows.+#+# Why a workflow rather than a rented instance: Modal offers Linux only, and a GitHub-hosted+# Windows runner is already a disposable machine with a real Service Control Manager,+# administrator rights and UAC disabled -- everything an endpoint install needs. So the runner+# *is* the sandbox, and `beacon-sandbox run --provider local` drives it from the inside. The+# collected artifacts are uploaded and judged on the maintainer's machine by the same pure+# `check` package that judges a Modal run, so a Windows verdict is produced by the same rules.+#+# workflow_dispatch only, and deliberately. The job needs ANTHROPIC_API_KEY to run a real agent+# session with --dangerously-skip-permissions; on a public repository that secret must never be+# reachable from untrusted code, so there is no pull_request trigger here and there must never be+# a pull_request_target one.+name: Windows sandbox++on:+ workflow_dispatch:+ inputs:+ scenario:+ description: "Scenario id to run (empty runs every windows scenario)"+ required: false+ type: string+ claude_version:+ description: "Claude Code version to pin"+ required: false+ default: "2.1.220"+ type: string++permissions:+ contents: read++concurrency:+ # One at a time: concurrent runs would race on the runner-wide agent install and on the+ # system-mode endpoint state, and a scenario that failed for that reason looks like a Beacon bug.+ group: windows-sandbox+ cancel-in-progress: false++jobs:+ sandbox:+ runs-on: windows-2025+ # The credential lives in an environment rather than at the repository level, matching how the+ # release secrets are scoped.+ environment: windows-sandbox+ timeout-minutes: 45+ defaults:+ run:+ shell: pwsh+ steps:+ - uses: actions/checkout@v4++ - uses: actions/setup-go@v5+ with:+ go-version-file: cli/beacon/go.mod++ # Runs the generated-PowerShell parse test, which is skipped on a maintainer's machine+ # without pwsh. The generated scripts are assembled in Go, so a syntax error in one is+ # invisible to the compiler and would surface as "the check did not run" -- which a verdict+ # reports as unverified rather than failing loudly. This is the gate that stops that.+ - name: Test the harness itself+ working-directory: beacon-sandbox+ run: go test ./...++ # Beacon does not compile for Windows yet: internal/endpoint/writer, inventory, ingest and+ # service/supervised all use POSIX-only syscalls. Until that is fixed the build is expected+ # to fail, and the run continues so the probe can still report everything that does not+ # depend on a Beacon binary. Once the portability layer lands, remove continue-on-error --+ # a silently missing binary must not read as a passing run.+ # The Makefile's build-windows-amd64 target is not used: GNU make is not guaranteed on the+ # Windows image, and the two steps it performs are clearer spelled out than shimmed. The+ # ordering is the part that matters -- the hooks binary is embedded into the CLI, so it has+ # to exist before the CLI is built or the CLI embeds the placeholder.+ - name: Build beacon and beacon-hooks for Windows+ id: build+ continue-on-error: true+ run: |+ Set-Location cli\beacon-hooks+ go build -ldflags "-s -w" -o beacon-hooks.exe .+ if ($LASTEXITCODE -ne 0) { throw "beacon-hooks does not build for windows/amd64 yet" }+ Copy-Item beacon-hooks.exe ..\beacon\internal\embedded\hooks.bin -Force+ Set-Location ..\beacon+ go build -o beacon-windows-amd64.exe .+ if ($LASTEXITCODE -ne 0) { throw "beacon does not build for windows/amd64 yet" }++ - name: Build the collector for Windows+ id: collector+ continue-on-error: true+ run: |+ go install go.opentelemetry.io/collector/cmd/[email protected]+ Set-Location collector-builder+ # builder mkdirs its output path without -p, so the parent has to exist. A fresh+ # checkout has no dist/ at all, which is why this only ever failed in CI.+ New-Item -ItemType Directory -Force -Path dist | Out-Null+ $env:GOOS = 'windows'+ $env:GOARCH = 'amd64'+ $env:CGO_ENABLED = '0'+ & "$(go env GOPATH)\bin\builder.exe" --config builder.yaml+ if ($LASTEXITCODE -ne 0) { throw "collector build failed" }+ New-Item -ItemType Directory -Force -Path dist\beacon-otelcol\windows_amd64 | Out-Null+ Copy-Item dist\beacon-otelcol\beacon-otelcol.exe dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe++ # The layout beacon-sandbox's WindowsLayout() describes. Created here rather than by the+ # harness because on this platform the CI job is what provisions the guest.+ - name: Stage the guest layout+ run: |+ New-Item -ItemType Directory -Force -Path C:\beacon-sandbox\bin, C:\beacon-sandbox\work | Out-Null+ if (Test-Path cli\beacon\beacon-windows-amd64.exe) {+ Copy-Item cli\beacon\beacon-windows-amd64.exe C:\beacon-sandbox\bin\beacon.exe+ }+ if (Test-Path collector-builder\dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe) {+ Copy-Item collector-builder\dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe C:\beacon-sandbox\bin\beacon-otelcol.exe+ }+ "C:\beacon-sandbox\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append+ # A seeded git repo gives scenarios somewhere realistic to work and exercises Beacon's+ # repository/branch enrichment, mirroring the Linux image's fixture.+ Set-Location C:\beacon-sandbox\work+ New-Item -ItemType Directory -Force -Path repo | Out-Null+ Set-Location repo+ git init -q+ git config user.email [email protected]+ git config user.name beacon-sandbox+ "# fixture" | Out-File -FilePath README.md -Encoding utf8+ git add -A+ git commit -qm initial++ # The native installer, which is the only Windows install path that self-updates. Pinned so+ # a verdict is attributable to a specific agent build.+ - name: Install Claude Code+ run: |+ $version = '${{ inputs.claude_version }}'+ if ([string]::IsNullOrWhiteSpace($version)) { $version = '2.1.220' }+ # Whitelist rather than escape: the value reaches a shell, and Claude Code versions are+ # dotted numerals, so anything else is a mistake worth rejecting outright.+ if ($version -notmatch '^[0-9]+(\.[0-9]+)*$') {+ throw "invalid claude_version '$version': expected a dotted numeric version"+ }+ $installer = Join-Path $env:TEMP 'install-claude.ps1'+ Invoke-WebRequest -Uri 'https://claude.ai/install.ps1' -OutFile $installer+ & $installer $version+ "$env:LOCALAPPDATA\Programs\claude" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append++ - name: Report the environment the probe ran in+ run: |+ Write-Output "pwsh: $($PSVersionTable.PSVersion)"+ Write-Output "identity: $([System.Security.Principal.WindowsIdentity]::GetCurrent().Name)"+ $principal = New-Object Security.Principal.WindowsPrincipal([System.Security.Principal.WindowsIdentity]::GetCurrent())+ Write-Output "elevated: $($principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator))"+ Write-Output "beacon build: ${{ steps.build.outcome }}"+ Write-Output "collector build: ${{ steps.collector.outcome }}"+ & claude --version+ if (Test-Path C:\beacon-sandbox\bin\beacon.exe) { & C:\beacon-sandbox\bin\beacon.exe version }++ - name: Run the scenario+ id: run+ continue-on-error: true+ working-directory: beacon-sandbox+ env:+ ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}+ run: |+ # Not $args: that name is a PowerShell automatic variable holding the script's own+ # arguments, and appending to it does not behave the way it reads.+ $sandboxArgs = @('run', '--provider', 'local')+ if (-not [string]::IsNullOrWhiteSpace('${{ inputs.scenario }}')) {+ $sandboxArgs += @('--scenario', '${{ inputs.scenario }}')+ }+ go run ./cmd/beacon-sandbox @sandboxArgs++ # Always uploaded. A failed run's artifacts are exactly what a maintainer needs, and the+ # host-state and argv metadata are recorded even when collection itself failed.+ - name: Upload collected artifacts+ if: always()+ uses: actions/upload-artifact@v4+ with:+ name: beacon-sandbox-runs+ path: beacon-sandbox/runs/+ if-no-files-found: warn+ retention-days: 14++ # The scenario's own exit status decides the job, after the artifacts are safely uploaded.+ # Without this the job would go green on a failing scenario because of continue-on-error.+ - name: Reflect the scenario outcome+ if: always()+ run: |+ Write-Output "scenario outcome: ${{ steps.run.outcome }}"+ if ('${{ steps.run.outcome }}' -ne 'success') {+ throw "the scenario did not pass; read the uploaded runs/ artifacts and the verdict.json inside them"+ }diff --git a/beacon-sandbox/check/invariants.go b/beacon-sandbox/check/invariants.goindex ea850c31..ffd06e3b 100644--- a/beacon-sandbox/check/invariants.go+++ b/beacon-sandbox/check/invariants.go@@ -266,6 +266,9 @@ type HostSafety struct {SecretInArgv bool// ArgvCheckRan is false when the scan could not run, so its result proves nothing.ArgvCheckRan bool+ // Disposability is the evidence a local-guest run offers in place of a comparison. Only+ // meaningful when HostChanged is hostguard.EphemeralDescription.+ Disposability string}// Safety adds findings for the host-isolation and credential-handling guarantees.@@ -292,6 +295,26 @@ func Safety(v *Verdict, s HostSafety) {Summary: "files unchanged, but the service probe could not run, so services are unverified",Why: "treat the service half as unproven; the file half of the guard did pass",})+ case s.HostChanged == hostguard.EphemeralDescription:+ // The guest was this machine, so there was no comparison to make: the install the+ // scenario is testing *is* the change a comparison would report. Isolation therefore+ // rests on the machine being disposable, which is a real but weaker guarantee -- so it+ // is reported rather than passed over in silence, and the evidence is named so a reader+ // can tell "GitHub hands out a fresh VM per job" from "the operator said so".+ //+ // Warn, not fail: a dispatched CI run is a legitimate and intended configuration. Warn,+ // not silence: the file-and-service comparison every other run gets did not happen here,+ // and silence in this tool means verified-clean.+ why := "the comparison every other run gets did not happen; isolation is only as good as " ++ "the machine being disposable"+ if s.Disposability != "" {+ why += " (" + s.Disposability + ")"+ }+ v.Add(Finding{+ Check: "safety.host_untouched", Severity: SevWarn,+ Summary: "host isolation was not verified by comparison because the guest was this machine",+ Why: why,+ })case s.HostChanged != hostguard.CleanDescription:v.Add(Finding{Check: "safety.host_untouched", Severity: SevFail,diff --git a/beacon-sandbox/cmd/beacon-sandbox/main.go b/beacon-sandbox/cmd/beacon-sandbox/main.goindex c1babdb8..54288b5f 100644--- a/beacon-sandbox/cmd/beacon-sandbox/main.go+++ b/beacon-sandbox/cmd/beacon-sandbox/main.go@@ -72,11 +72,16 @@ func usage() {fmt.Fprint(os.Stderr, `beacon-sandbox -- verify Beacon against real agent activity in a disposable sandboxdoctor [--fix] [--json] check prerequisites and say how to fix what is missing- run [--scenario ID] [--repeat N] [--keep-sandbox] [--suite DIR]+ run [--scenario ID] [--repeat N] [--keep-sandbox] [--suite DIR] [--provider NAME]verify <run-dir>... [--mutate MODE]diff <before-dir> <after-dir>clean [--dir runs]+Where scenarios run (--provider):+ modal a disposable Linux sandbox. The default, and Linux-only+ github dispatch to a GitHub Actions Windows runner, then collect and judge locally+ local this machine, for use inside a disposable CI runner+Anthropic credential, in the order they resolve:--modal-secret NAME a secret already stored with the provider; the value never entersthis process, so the artifact leak check reports unverified@@ -120,6 +125,8 @@ func cmdRun(args []string) error {claudeVer := fs.String("claude-version", "", "pin Claude Code version (default: image default)")modalSecret := fs.String("modal-secret", "", "name of a Modal secret holding ANTHROPIC_API_KEY; the value never enters this process")keyCommand := fs.String("api-key-command", "", "command whose stdout is the Anthropic API key (e.g. 'op read op://vault/anthropic/key')")+ provider := fs.String("provider", "modal", "where scenarios run: modal (disposable Linux), github (dispatch to a Windows runner), or local (this machine)")+ allowHostMutation := fs.Bool("allow-host-mutation", false, "with --provider local, run on a machine this tool cannot prove is disposable")fs.Parse(args)root, err := repoRoot()@@ -156,17 +163,32 @@ func cmdRun(args []string) error {}ctx := context.Background()- prov, err := sandbox.NewModal(ctx, appName)+ prov, closeProv, err := openProvider(ctx, *provider, *allowHostMutation)if err != nil {return err}- defer prov.Close()+ defer closeProv()++ // Scenarios are not portable, so run only the ones written for this guest -- and say how many+ // were set aside. `run` with no --scenario is the "everything" command, and silently returning+ // a subset of it would read as full coverage.+ target := platformOf(prov)+ sui, skipped := sui.For(target)+ if skipped > 0 {+ fmt.Printf("suite: %d %s scenario(s); %d written for another platform and not run\n",+ len(sui.Scenarios), target, skipped)+ }+ if len(sui.Scenarios) == 0 {+ return fmt.Errorf("no %s scenarios in %s, so this run would assert nothing", target, scDir)+ }var verdicts []check.Verdict+ matched := falsefor _, sc := range sui.Scenarios {if *only != "" && sc.ID != *only {continue}+ matched = truefor i := 0; i < *repeat; i++ {label := sc.IDif *repeat > 1 {@@ -200,9 +222,48 @@ func cmdRun(args []string) error {verdicts = append(verdicts, v)}}+ // A --scenario that matched nothing must not exit 0 with an empty summary: the caller asked+ // for a specific check and got none, which is a failure to run rather than a clean result.+ // A typo'd id, or one belonging to the other platform, both land here.+ if *only != "" && !matched {+ return fmt.Errorf("no %s scenario with id %q in %s; `--provider %s` runs %s scenarios",+ target, *only, scDir, *provider, target)+ }return summarize(verdicts)}+// openProvider resolves the --provider choice, returning a cleanup function so a Modal client is+// always closed.+func openProvider(ctx context.Context, name string, allowHostMutation bool) (sandbox.Provider, func(), error) {+ switch name {+ case "modal":+ p, err := sandbox.NewModal(ctx, appName)+ if err != nil {+ return nil, func() {}, err+ }+ return p, p.Close, nil+ case "local":+ p, err := sandbox.NewLocalExec(sandbox.LocalExecOptions{AllowHostMutation: allowHostMutation})+ if err != nil {+ return nil, func() {}, err+ }+ return p, func() {}, nil+ case "github":+ return nil, func() {}, fmt.Errorf("--provider github dispatches a workflow rather than " ++ "driving a machine directly; it is not wired up yet")+ default:+ return nil, func() {}, fmt.Errorf("unknown --provider %q; want modal, github, or local", name)+ }+}++// platformOf maps a provider's guest platform onto the scenario vocabulary.+func platformOf(p sandbox.Provider) scenario.Platform {+ if p.Platform().IsWindows() {+ return scenario.PlatformWindows+ }+ return scenario.PlatformLinux+}+// judge is the pure step: artifacts in, verdict out. Returns the parsed log too so callers// can build a fingerprint without re-reading it.func judge(sc scenario.Scenario, art runner.Artifacts, creds credentials.Resolved, planted string) (check.Verdict, check.Log) {@@ -257,9 +318,10 @@ func judge(sc scenario.Scenario, art runner.Artifacts, creds credentials.Resolve// credential disclosure are the two findings that must survive every other kind of failure.func safetyOf(v *check.Verdict, art runner.Artifacts) {check.Safety(v, check.HostSafety{- HostChanged: art.Meta["host_state"],- SecretInArgv: art.Meta["secret_in_argv"] == "true",- ArgvCheckRan: art.Meta["argv_check_ran"] == "true",+ HostChanged: art.Meta["host_state"],+ SecretInArgv: art.Meta["secret_in_argv"] == "true",+ ArgvCheckRan: art.Meta["argv_check_ran"] == "true",+ Disposability: art.Meta["disposability"],})}diff --git a/beacon-sandbox/hostguard/hostguard.go b/beacon-sandbox/hostguard/hostguard.goindex aee4f895..3b8767c8 100644--- a/beacon-sandbox/hostguard/hostguard.go+++ b/beacon-sandbox/hostguard/hostguard.go@@ -97,10 +97,37 @@ func guardedPaths() []guarded {guarded{filepath.Join(h, ".config", "systemd", "user"), watchBeaconEntries},guarded{"/etc/systemd/system", watchBeaconEntries},)+ case "windows":+ // For a contributor pointing the harness at a Windows VM they own. The dispatched CI+ // path does not compare snapshots at all -- see EphemeralDescription -- but a guard that+ // silently watched nothing on Windows would be indistinguishable from one that watched+ // and found nothing, which is the conflation this package exists to avoid.+ out = append(out,+ guarded{filepath.Join(programData(), "Beacon", "Endpoint", "config.json"), watchFull},+ guarded{filepath.Join(programData(), "Beacon", "Endpoint", "logs"), watchExistence},+ guarded{filepath.Join(programFiles(), "Beacon"), watchExistence},+ )}return out}+// programData and programFiles resolve the Windows system directories, falling back to the+// conventional locations. Read from the environment because a redirected or localized install+// puts them elsewhere, and a guard watching the wrong path reports clean forever.+func programData() string {+ if v := os.Getenv("ProgramData"); v != "" {+ return v+ }+ return `C:\ProgramData`+}++func programFiles() string {+ if v := os.Getenv("ProgramFiles"); v != "" {+ return v+ }+ return `C:\Program Files`+}+// GuardedPaths returns the watched paths, for reporting and tests.func GuardedPaths() []string {g := guardedPaths()@@ -204,6 +231,23 @@ func beaconServices() (services []string, available bool) {out = append(out, fields[0])}}+ case "windows":+ // `sc.exe query type= service state= all` lists every service; the display name lines are+ // what carry "beacon". Parsed rather than piped through PowerShell so the probe does not+ // depend on which shell is installed.+ b, err := exec.Command("sc.exe", "query", "type=", "service", "state=", "all").Output()+ if err != nil {+ return nil, false+ }+ for _, line := range strings.Split(string(b), "\n") {+ name, ok := strings.CutPrefix(strings.TrimSpace(line), "SERVICE_NAME:")+ if !ok {+ continue+ }+ if name = strings.TrimSpace(name); strings.Contains(strings.ToLower(name), "beacon") {+ out = append(out, name)+ }+ }}sort.Strings(out)return out, true@@ -236,6 +280,17 @@ const CleanDescription = "host state unchanged"// layer must be able to tell them apart without string-sniffing.const PartialDescription = "files unchanged; service probe unavailable, so services are unverified"+// EphemeralDescription is recorded when the guest *is* this machine, so comparing before and+// after cannot mean anything.+//+// A backend like that runs the scenario where the harness stands, which makes the install the+// scenario is testing indistinguishable from the escape this guard looks for -- a correct run+// would report the most serious finding the tool has. So isolation is claimed on a different+// basis: the machine is disposable and somebody else destroys it. That is a genuinely weaker+// claim than "nothing on the developer's machine moved", and it gets its own value rather than+// borrowing CleanDescription precisely so a reader of a verdict can tell which one they got.+const EphemeralDescription = "guest is this machine; isolation rests on it being disposable rather than on a comparison"+func (d Diff) Describe() string {if d.Clean() {if d.ServicesUnverified {diff --git a/beacon-sandbox/image/image.go b/beacon-sandbox/image/image.goindex 3218bbaa..7ff9751f 100644--- a/beacon-sandbox/image/image.go+++ b/beacon-sandbox/image/image.go@@ -9,6 +9,7 @@ package imageimport ("fmt""os"+ "path/filepath""strings""github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"@@ -31,6 +32,137 @@ const (DefaultBase = "ubuntu:24.04")+// Layout is where things live inside the guest.+//+// Split out from the package constants because those describe one environment -- the Linux+// image this package builds -- and Windows runs somewhere this package does not provision at+// all. The CI job places the binaries and creates the work directory there, so the harness+// needs to *describe* the layout without owning it.+type Layout struct {+ Platform sandbox.Platform+ // AgentUser is the account the session runs as. Empty means "whoever is already running",+ // which is the Windows case: there is no way to become another user without a password.+ AgentUser string+ AgentHome string+ WorkDir string+ BeaconDir string+ // PathPrepend is prepended to PATH so both the agent and Beacon are reachable.+ PathPrepend []string+}++// LinuxLayout is the layout of the image this package builds.+func LinuxLayout() Layout {+ return Layout{+ Platform: sandbox.PlatformPosix,+ AgentUser: AgentUser,+ AgentHome: AgentHome,+ WorkDir: WorkDir,+ BeaconDir: BeaconDir,+ PathPrepend: PathPrepend(),+ }+}++// WindowsLayout is the layout the windows-sandbox CI job provisions.+//+// No AgentUser: the runner is already an administrator with UAC disabled, and this backend+// cannot switch users anyway. Whether that is acceptable to the agent is an open question the+// w00-probe scenario answers -- Claude Code refuses --dangerously-skip-permissions as root on+// POSIX, and if it refuses for an elevated Windows user too, the CI job will have to create a+// standard account and run the whole harness as them.+func WindowsLayout() Layout {+ home := windowsHome()+ beaconDir := `C:\beacon-sandbox\bin`+ return Layout{+ Platform: sandbox.PlatformWindows,+ AgentHome: home,+ WorkDir: `C:\beacon-sandbox\work`,+ BeaconDir: beaconDir,+ // The agent installs to %LOCALAPPDATA%\Programs, which is where install.ps1 puts it.+ PathPrepend: []string{beaconDir, filepath.Join(home, "AppData", "Local", "Programs", "claude")},+ }+}++// LayoutFor selects the layout for a platform.+func LayoutFor(p sandbox.Platform) Layout {+ if p.IsWindows() {+ return WindowsLayout()+ }+ return LinuxLayout()+}++// PrivilegedUser names the account a privileged step runs as, or empty when this platform has+// no account to switch to.+//+// Windows is the empty case, and that is not a gap: the runner is already an administrator with+// UAC disabled, so a system-mode install has the rights it needs without becoming anyone. There+// is also no way to become another user without a password, so asking would fail rather than+// escalate.+func (l Layout) PrivilegedUser() string {+ if l.Platform.IsWindows() {+ return ""+ }+ return "root"+}++// DefaultLogPath is where Beacon writes the runtime log when `endpoint status --json` could not+// be parsed.+//+// A fallback only. The install scenarios read the real path out of status, because a scenario+// that hardcodes the expected path agrees with itself rather than with Beacon -- which is the+// one thing an install scenario exists to check.+func (l Layout) DefaultLogPath(system bool) string {+ if l.Platform.IsWindows() {+ if system {+ programData := os.Getenv("ProgramData")+ if programData == "" {+ programData = `C:\ProgramData`+ }+ return filepath.Join(programData, "Beacon", "Endpoint", "logs", "runtime.jsonl")+ }+ return filepath.Join(l.AgentHome, ".beacon", "endpoint", "logs", "runtime.jsonl")+ }+ if system {+ return "/var/log/beacon-agent/runtime.jsonl"+ }+ return l.AgentHome + "/.beacon/endpoint/logs/runtime.jsonl"+}++// ConsoleUser names the human whose agent runtime a system-mode install must configure.+//+// A system endpoint runs elevated, and `endpoint install` configures the *installing* user's+// settings -- which for a package install is not the person at the keyboard. Beacon resolves+// that person from SUDO_USER and then logind; a system-mode scenario has to name them or its+// capture assertions would be testing an unused settings file. On Windows there is one account,+// so it is that one.+func (l Layout) ConsoleUser() string {+ if l.AgentUser != "" {+ return l.AgentUser+ }+ if v := os.Getenv("USERNAME"); v != "" {+ return v+ }+ if v := os.Getenv("USER"); v != "" {+ return v+ }+ return filepath.Base(l.AgentHome)+}++// windowsHome resolves the guest home directory.+//+// USERPROFILE is the value Windows programs actually read. Falling back to a literal rather+// than to an error keeps a dispatched run from failing on a runner that has not exported it;+// the layout is only ever used to build guest paths, so a wrong guess surfaces immediately as+// a missing file rather than silently.+func windowsHome() string {+ if v := os.Getenv("USERPROFILE"); v != "" {+ return v+ }+ if v := os.Getenv("HOME"); v != "" {+ return v+ }+ return `C:\Users\runneradmin`+}+// Spec parameterizes the environment.type Spec struct {Base stringdiff --git a/beacon-sandbox/runner/runner.go b/beacon-sandbox/runner/runner.goindex bfe629fd..61199c24 100644--- a/beacon-sandbox/runner/runner.go+++ b/beacon-sandbox/runner/runner.go@@ -80,6 +80,11 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Optreturn Artifacts{}, fmt.Errorf("no Anthropic credential resolved; run `beacon-sandbox doctor`")}+ // The dialect and the guest layout both follow from the provider's platform, never from the+ // host's runtime.GOOS -- a Mac dispatching a Windows run is the normal case, not the exception.+ sh := shellFor(p.Platform())+ lay := image.LayoutFor(p.Platform())+canary := "BEACON_SANDBOX_" + randomToken()runDir := filepath.Join(opts.OutDir, fmt.Sprintf("%s-%s", sc.ID, canary[len(canary)-8:]))// 0700, not 0755: these artifacts retain prompt text and command output, and a@@ -94,34 +99,49 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts OptMeta: map[string]string{"scenario": sc.ID,"provider": p.Name(),+ "platform": string(p.Platform()),},}- // Fingerprint the developer's own Beacon state before anything runs, so a stray local- // install would be caught rather than assumed impossible.+ // Establish how isolation is proven for this run, and record it either way.+ //+ // The usual proof is a comparison: fingerprint the developer's own Beacon state before and+ // after, so a stray local install is caught rather than assumed impossible. That only means+ // something when the guest is a *different* machine. A backend that runs the scenario where+ // this process stands inverts it -- the install the scenario is testing is itself the change+ // the guard would report -- so for those the claim has to rest on the machine being+ // disposable instead, and say which of the two it relied on.//- // The comparison and the meta write happen together in this defer, and deliberately so.- // An earlier version wrote meta.json inline before returning, which meant host_state --- // set here, after that write -- never reached disk. The live verdict was still correct- // because judging happens after Run returns, but an offline `verify` read the field as- // empty and treated the host check as clean. An unrunnable check that reads as a pass is- // the exact failure this tool exists to prevent. Writing here also means an early error- // return still leaves the metadata behind for debugging.- hostBefore := hostguard.Take()+ // The comparison and the meta write happen together in this defer, and deliberately so. An+ // earlier version wrote meta.json inline before returning, which meant host_state -- set+ // here, after that write -- never reached disk. The live verdict was still correct because+ // judging happens after Run returns, but an offline `verify` read the field as empty and+ // treated the host check as clean. An unrunnable check that reads as a pass is the exact+ // failure this tool exists to prevent. Writing here also means an early error return still+ // leaves the metadata behind for debugging.+ local := p.MutatesHost()+ var hostBefore hostguard.Snapshot+ if !local {+ hostBefore = hostguard.Take()+ }defer func() {- art.HostDiff = hostguard.Compare(hostBefore, hostguard.Take())- art.Meta["host_state"] = art.HostDiff.Describe()+ if local {+ art.Meta["host_state"] = hostguard.EphemeralDescription+ art.Meta["disposability"] = sandbox.DisposabilityEvidence()+ } else {+ art.HostDiff = hostguard.Compare(hostBefore, hostguard.Take())+ art.Meta["host_state"] = art.HostDiff.Describe()+ }if b, err := json.MarshalIndent(art.Meta, "", " "); err == nil {_ = os.WriteFile(filepath.Join(runDir, "meta.json"), append(b, '\n'), 0o600)}}()+ if local {+ logf("guest is this machine (%s); isolation rests on it being disposable: %s",+ p.Name(), sandbox.DisposabilityEvidence())+ }- logf("building image (cached layers are free)")- imgSpec, err := image.Build(image.Spec{- RepoRoot: opts.RepoRoot,- ClaudeVersion: opts.ClaudeVersion,- WithDocker: sc.NeedsRealSystemd(),- }, logf)+ imgSpec, err := imageSpecFor(p.Platform(), sc, opts, logf)if err != nil {return art, err}@@ -136,7 +156,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts OptCPU: 2,MemLimitMiB: 4096,Timeout: sc.Timeout() + 5*time.Minute, // headroom for setup and collection- Workdir: image.WorkDir,+ Workdir: lay.WorkDir,Secrets: inlineSecrets(opts.Creds),SecretNames: providerSecrets(opts.Creds),BlockEgress: sc.BlockEgress,@@ -172,15 +192,27 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt}agent := sandbox.ExecOpts{- User: image.AgentUser, Dir: image.WorkDir, HomeDir: image.AgentHome,- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: 5 * time.Minute,- }-- // Finish the artifact layer: the snapshot captured the files, but chmod and symlinks- // have to happen in a live instance. Always the sandbox itself, never the guest -- this- // prepares the very tree the guest will mount.- if _, err := p.Exec(ctx, inst, image.PostPushLayers(), sandbox.ExecOpts{User: "root", Timeout: time.Minute}); err != nil {- return art, fmt.Errorf("prepare binaries: %w", err)+ User: lay.AgentUser, Dir: lay.WorkDir, HomeDir: lay.AgentHome,+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: 5 * time.Minute,+ }+ // privileged is the same options with whatever elevation this platform expresses. On POSIX+ // that is the root account; on Windows there is no account to switch to -- the runner is+ // already an administrator with UAC disabled, and the local backend cannot become anyone+ // else. Naming the difference once keeps every later call site single-path.+ privileged := sandbox.ExecOpts{+ User: lay.PrivilegedUser(), Dir: lay.WorkDir, HomeDir: lay.AgentHome,+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: 5 * time.Minute,+ }++ // Finish the artifact layer: the snapshot captured the files, but chmod and symlinks have to+ // happen in a live instance. Skipped when nothing was pushed, which is the local backend's+ // case -- the CI job placed the binaries before this process started, so there is no staged+ // tree to fix up.+ if len(imgSpec.Files) > 0 {+ if _, err := p.Exec(ctx, inst, image.PostPushLayers(),+ sandbox.ExecOpts{User: lay.PrivilegedUser(), Timeout: time.Minute}); err != nil {+ return art, fmt.Errorf("prepare binaries: %w", err)+ }}// Everything the scenario itself does happens in the guest. For most scenarios that is the@@ -200,7 +232,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// Not just recorded -- checked. An empty version line means beacon or claude is not reachable// in the guest, and every later failure is then a symptom of that rather than of Beacon. The// first nested run printed "versions:" with nothing after it and kept going.- v, err := g.Exec(ctx, "beacon version; claude --version; id -un; uname -m", agent)+ v, err := g.Exec(ctx, versionProbe(p.Platform()), agent)if err != nil {return art, fmt.Errorf("could not run commands in the %s guest: %w", g.Describe(), err)}@@ -211,21 +243,21 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt"test is not reachable there (rc=%d): %s", v.ExitCode, oneLine(v.Stdout+v.Stderr))}- sentinel := scenario.Expand(sc.Sentinel, canary, "", image.WorkDir)+ sentinel := scenario.Expand(sc.Sentinel, canary, "", lay.WorkDir)if sc.Setup != "" {- setup := scenario.Expand(sc.Setup, canary, sentinel, image.WorkDir)+ setup := scenario.Expand(sc.Setup, canary, sentinel, lay.WorkDir)if r, err := g.Exec(ctx, setup, agent); err != nil || r.ExitCode != 0 {return art, fmt.Errorf("scenario setup failed (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stderr))}}- logPath := filepath.ToSlash(filepath.Join(image.WorkDir, "runtime.jsonl"))- prompt := scenario.Expand(sc.Prompt, canary, sentinel, image.WorkDir)+ logPath := sh.Join(lay.WorkDir, "runtime.jsonl")+ prompt := scenario.Expand(sc.Prompt, canary, sentinel, lay.WorkDir)// Start the credential sampler BEFORE the session, so it observes the agent's own- // processes while they are alive. See argvSamplerScript.- if _, err := p.Exec(ctx, inst, argvSamplerScript(), sandbox.ExecOpts{- User: "root", Timeout: time.Minute,+ // processes while they are alive. See shell.ArgvSampler.+ if _, err := p.Exec(ctx, inst, sh.ArgvSampler(), sandbox.ExecOpts{+ User: lay.PrivilegedUser(), Timeout: time.Minute,}); err != nil {logf("argv sampler failed to start: %v", err)}@@ -236,19 +268,21 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// manager. A persistent install instead exercises `beacon endpoint install`: service// manager selection, unit installation, status reporting, and whether an installed// collector actually receives telemetry. Nothing else in the suite can tell you that.+ sessionDir := workDirFor(sc, lay, sh)+ sessionOpts := sandbox.ExecOpts{+ User: lay.AgentUser, Dir: sessionDir, HomeDir: lay.AgentHome,+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: sc.Timeout() + time.Minute,+ }var res sandbox.Resultif sc.Install != nil {- installedLog, err := doInstall(ctx, g, sc, agent, &art, logf)+ installedLog, err := doInstall(ctx, g, sc, privileged, agent, lay, sh, &art, logf)if err != nil {return art, err}logPath = installedLoglogf("running session against the installed endpoint (budget $%.2f, timeout %s)",sc.Budget(), sc.Timeout())- res, err = g.Exec(ctx, plainSessionScript(sc, prompt), sandbox.ExecOpts{- User: image.AgentUser, Dir: workDirFor(sc), HomeDir: image.AgentHome,- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: sc.Timeout() + time.Minute,- })+ res, err = g.Exec(ctx, sh.PlainSession(sessionDir, claudeFlags(sc, prompt, sh)), sessionOpts)if err != nil {logf("session exec error: %v", err)}@@ -256,10 +290,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt} else {logf("running session (budget $%.2f, timeout %s)", sc.Budget(), sc.Timeout())var err error- res, err = g.Exec(ctx, sessionScript(sc, prompt, logPath), sandbox.ExecOpts{- User: image.AgentUser, Dir: workDirFor(sc), HomeDir: image.AgentHome,- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: sc.Timeout() + time.Minute,- })+ res, err = g.Exec(ctx, sh.CIExecSession(logPath, sessionDir, claudeFlags(sc, prompt, sh)), sessionOpts)if err != nil {logf("session exec error: %v", err)}@@ -279,9 +310,9 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// the default working directory silently found nothing whenever `cwd:` was set -- s05 was// the only such scenario and it reported session_ok=false on an otherwise perfect run,// disabling exactly the signal that distinguishes a dead agent from a capture gap.- outPath := shq(sessionFile(sc, "claude-out.json"))- errPath := shq(sessionFile(sc, "claude-err.txt"))- if r, err := g.Exec(ctx, "cat "+outPath+" 2>/dev/null || true", agent); err == nil {+ outPath := sessionFile(sc, "claude-out.json", lay, sh)+ errPath := sessionFile(sc, "claude-err.txt", lay, sh)+ if r, err := g.Exec(ctx, sh.ReadFile(outPath), agent); err == nil {var result map[string]anyif json.Unmarshal([]byte(strings.TrimSpace(r.Stdout)), &result) == nil {art.SessionOK, art.SessionKnown = sessionOutcome(result)@@ -294,7 +325,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Optlogf("claude: %s", art.Meta["claude_result"])_ = os.WriteFile(filepath.Join(runDir, "claude-result.json"), []byte(r.Stdout), 0o600)} else {- if e, err2 := g.Exec(ctx, "tail -c 800 "+errPath+" 2>/dev/null || true", agent); err2 == nil {+ if e, err2 := g.Exec(ctx, sh.TailBytes(errPath, 800), agent); err2 == nil {art.Meta["claude_stderr"] = oneLine(e.Stdout)}}@@ -310,9 +341,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// The probe always emits a marker, so its output is never ambiguous. `cat f || echo// __MISSING__` produced empty stdout for both a failed exec and a legitimately empty// sentinel file, which would have misreported the second as a broken probe.- probe := fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",- shq(sentinel))- r, execErr := g.Exec(ctx, probe, agent)+ r, execErr := g.Exec(ctx, sh.ProbeFile(sentinel), agent)art.SentinelProbed = execErr == nil &&(strings.Contains(r.Stdout, "__FOUND__") || strings.Contains(r.Stdout, "__MISSING__"))if !art.SentinelProbed {@@ -338,8 +367,8 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// Read what the in-session sampler observed while the agent was actually running. See// argvSamplerScript for why this cannot be a post-session scan.- if r, err := p.Exec(ctx, inst, "cat "+shq(argvVerdictPath)+" 2>/dev/null || true",- sandbox.ExecOpts{User: "root", Timeout: time.Minute}); err == nil {+ if r, err := p.Exec(ctx, inst, sh.ReadFile(sh.ArgvVerdictPath()),+ sandbox.ExecOpts{User: lay.PrivilegedUser(), Timeout: time.Minute}); err == nil {sawAgent := strings.Contains(r.Stdout, "saw_agent=1")art.Meta["argv_saw_agent"] = fmt.Sprintf("%v", sawAgent)switch {@@ -393,16 +422,16 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt// Wait for quiescence rather than sleeping a fixed amount: the collector batches at 5s// and Claude Code's metric interval defaults to 60s, so a fixed sleep is either wasteful// or wrong. Record how long it took -- that latency is itself a finding.- stable, waited := waitForQuiescence(ctx, g, agent, logPath, logf)+ stable, waited := waitForQuiescence(ctx, g, agent, sh, logPath, logf)art.Meta["quiescence_seconds"] = fmt.Sprintf("%.0f", waited.Seconds())art.Meta["quiescent"] = fmt.Sprintf("%v", stable)- local := filepath.Join(runDir, "runtime.jsonl")- if err := g.Get(ctx, logPath, local); err != nil {+ localLog := filepath.Join(runDir, "runtime.jsonl")+ if err := g.Get(ctx, logPath, localLog); err != nil {return art, fmt.Errorf("collect runtime log: %w", err)}- art.RuntimeLog = local- logf("collected %s", local)+ art.RuntimeLog = localLog+ logf("collected %s", localLog)// meta.json is written by the deferred host-state block above, so it always includes the// host comparison.@@ -423,52 +452,47 @@ func sessionOutcome(result map[string]any) (ok, known bool) {return !isErr, true}+// versionProbe asks the guest to identify itself.+//+// Not just recorded but checked by the caller: an empty version line means beacon or claude is+// not reachable in the guest, and every later failure is then a symptom of that rather than of+// Beacon.+func versionProbe(p sandbox.Platform) string {+ if p.IsWindows() {+ return "& beacon version; & claude --version; " ++ "[System.Security.Principal.WindowsIdentity]::GetCurrent().Name; $env:PROCESSOR_ARCHITECTURE"+ }+ return "beacon version; claude --version; id -un; uname -m"+}+// sessionFile resolves a file the agent session wrote, relative to the directory that session// actually ran in.-func sessionFile(sc scenario.Scenario, name string) string {- return filepath.ToSlash(filepath.Join(workDirFor(sc), name))+func sessionFile(sc scenario.Scenario, name string, lay image.Layout, sh shell) string {+ return sh.Join(workDirFor(sc, lay, sh), name)}// claudeFlags builds the agent invocation shared by both session shapes.-func claudeFlags(sc scenario.Scenario, prompt string) []string {+func claudeFlags(sc scenario.Scenario, prompt string, sh shell) []string {flags := []string{- "-p", shq(prompt),+ "-p", sh.Quote(prompt),"--output-format", "json","--dangerously-skip-permissions",fmt.Sprintf("--max-budget-usd %.2f", sc.Budget()),}if len(sc.Tools) > 0 {- flags = append(flags, "--tools", shq(strings.Join(sc.Tools, ",")))+ flags = append(flags, "--tools", sh.Quote(strings.Join(sc.Tools, ",")))} else if sc.Tools != nil {- flags = append(flags, "--tools", shq(""))+ flags = append(flags, "--tools", sh.Quote(""))}if len(sc.DisallowedTools) > 0 {- flags = append(flags, "--disallowedTools", shq(strings.Join(sc.DisallowedTools, ",")))+ flags = append(flags, "--disallowedTools", sh.Quote(strings.Join(sc.DisallowedTools, ",")))}return flags}-// sessionScript builds the in-guest command.-//-// The child is wrapped in `sh -c` so Claude's stdout is redirected *inside* `beacon ci exec`.-// Redirecting outside it would interleave Beacon's own report with Claude's JSON in one file,-// which made the first M0 run's output unparseable.-func sessionScript(sc scenario.Scenario, prompt, logPath string) string {- inner := "claude " + strings.Join(claudeFlags(sc, prompt), " ") +- " > claude-out.json 2> claude-err.txt"- // The script must exit with `beacon ci exec`'s status, not with `tail`'s. Ending on the tail- // made every session look successful to p.Exec no matter how ci exec fared, and nothing read- // CI_EXEC_RC either -- so a collector that failed outright was indistinguishable from one- // that worked. Reported by the Copilot reviewer.- return fmt.Sprintf("beacon ci exec --harness claude --log-path %s -- sh -c %s "+- "> ci-out.txt 2> ci-err.txt; rc=$?; echo CI_EXEC_RC=$rc; "+- "tail -c 400 ci-out.txt; exit $rc",- shq(logPath), shq(inner))-}-// waitForQuiescence polls the log until its size stops changing, bounded.func waitForQuiescence(ctx context.Context, g guest,- opts sandbox.ExecOpts, logPath string, logf func(string, ...any)) (bool, time.Duration) {+ opts sandbox.ExecOpts, sh shell, logPath string, logf func(string, ...any)) (bool, time.Duration) {const (interval = 3 * time.Second@@ -480,7 +504,7 @@ func waitForQuiescence(ctx context.Context, g guest,stableSince := time.Time{}for time.Since(start) < maxWait {- r, err := g.Exec(ctx, "wc -c < "+shq(logPath)+" 2>/dev/null || echo 0", opts)+ r, err := g.Exec(ctx, sh.FileSize(logPath), opts)if err == nil {cur := strings.TrimSpace(r.Stdout)if cur == last && cur != "0" {@@ -504,11 +528,33 @@ func waitForQuiescence(ctx context.Context, g guest,return false, time.Since(start)}-func workDirFor(sc scenario.Scenario) string {+func workDirFor(sc scenario.Scenario, lay image.Layout, sh shell) string {if sc.Cwd == "" {- return image.WorkDir+ return lay.WorkDir+ }+ return sh.Join(lay.WorkDir, sc.Cwd)+}++// imageSpecFor describes the environment a scenario needs.+//+// Windows takes the short path deliberately: nothing here provisions it. The CI job built the+// binaries, installed the agent runtime and created the work directory before this process+// started, so there is no image to build and no artifact to push. What would otherwise be+// checked here -- that the binary under test is actually reachable -- is checked better a few+// lines later by asking the guest for `beacon version`, which tests reachability rather than a+// path guess.+func imageSpecFor(p sandbox.Platform, sc scenario.Scenario, opts Options,+ logf func(string, ...any)) (sandbox.ImageSpec, error) {++ if p.IsWindows() {+ return sandbox.ImageSpec{Base: "windows/amd64"}, nil}- return filepath.ToSlash(filepath.Join(image.WorkDir, sc.Cwd))+ logf("building image (cached layers are free)")+ return image.Build(image.Spec{+ RepoRoot: opts.RepoRoot,+ ClaudeVersion: opts.ClaudeVersion,+ WithDocker: sc.NeedsRealSystemd(),+ }, logf)}func randomToken() string {@@ -639,8 +685,9 @@ func sampleCountOf(verdict string) string {// The log path comes from `endpoint status --json` rather than being assumed, because the whole// point of an install scenario is to check what Beacon actually did — hardcoding the expected// path would make the scenario agree with itself instead of with Beacon.-func doInstall(ctx context.Context, g guest,- sc scenario.Scenario, agent sandbox.ExecOpts, art *Artifacts, logf func(string, ...any)) (string, error) {+func doInstall(ctx context.Context, g guest, sc scenario.Scenario,+ privileged, agent sandbox.ExecOpts, lay image.Layout, sh shell,+ art *Artifacts, logf func(string, ...any)) (string, error) {in := sc.Installsystem := in.Mode == "system"@@ -652,12 +699,14 @@ func doInstall(ctx context.Context, g guest,if in.Service != "" {svc = " --service=" + in.Service}- // System-mode install requires root; user mode must not have it, or the install would- // write system paths while claiming to be a user install.+ // System-mode install needs elevation; user mode must not have it, or the install would+ // write system paths while claiming to be a user install. On Windows the two options differ+ // only in what they cannot do -- there is one account, already elevated -- which is a real+ // weakness of the Windows user-mode scenario and is why w02 asserts the supervised backend+ // rather than trusting that it ran unprivileged.opts := agentif system {- opts = sandbox.ExecOpts{User: "root", Dir: image.WorkDir, Timeout: 5 * time.Minute,- PathPrepend: image.PathPrepend()}+ opts = privileged}art.Meta["install_mode"] = in.Modeart.Meta["install_service"] = in.Service@@ -687,13 +736,13 @@ func doInstall(ctx context.Context, g guest,// SUDO_USER is how the real path identifies them (`sudo apt install ./beacon.deb`), so it is// what gets set here rather than inventing a flag that no user would pass.if system {- cmd := fmt.Sprintf("SUDO_USER=%s beacon endpoint user-config repair-installed "+- "--system --harness claude --json 2>&1", image.AgentUser)+ cmd := sh.WithEnv("SUDO_USER", lay.ConsoleUser(),+ "beacon endpoint user-config repair-installed --system --harness claude --json 2>&1")u, err := g.Exec(ctx, cmd, opts)art.Meta["install_user_config"] = oneLine(u.Stdout + u.Stderr)if err != nil || u.ExitCode != 0 {return "", fmt.Errorf("configuring %s's agent runtime failed (rc=%d): %v %s",- image.AgentUser, u.ExitCode, err, art.Meta["install_user_config"])+ lay.ConsoleUser(), u.ExitCode, err, art.Meta["install_user_config"])}logf("user config: %s", art.Meta["install_user_config"])// A skip is not a pass. The command reports one rather than failing, because on a real@@ -701,11 +750,11 @@ func doInstall(ctx context.Context, g guest,// treating a skip as success would let a broken resolver produce a green run.if strings.Contains(art.Meta["install_user_config"], "skipped_reason") {return "", fmt.Errorf("user-config skipped instead of configuring %s: %s",- image.AgentUser, art.Meta["install_user_config"])+ lay.ConsoleUser(), art.Meta["install_user_config"])}}- logPath, err := installedLogPath(ctx, g, scope, opts, in, art, logf)+ logPath, err := installedLogPath(ctx, g, scope, opts, in, lay, art, logf)if err != nil {return "", err}@@ -723,7 +772,7 @@ func doInstall(ctx context.Context, g guest,// documented default path — but ExpectStatusRunning is enforced, since a scenario asking for a// running service and silently accepting an unparseable status would verify nothing.func installedLogPath(ctx context.Context, g guest,- scope string, opts sandbox.ExecOpts, in *scenario.Install, art *Artifacts,+ scope string, opts sandbox.ExecOpts, in *scenario.Install, lay image.Layout, art *Artifacts,logf func(string, ...any)) (string, error) {s, err := g.Exec(ctx, "beacon endpoint status "+scope+" --json 2>&1", opts)@@ -766,15 +815,5 @@ func installedLogPath(ctx context.Context, g guest,"the selected backend can be confirmed; refusing to report a pass the run did not earn")}logf("status unparseable; falling back to the documented default log path")- if in.Mode == "system" {- return "/var/log/beacon-agent/runtime.jsonl", nil- }- return image.AgentHome + "/.beacon/endpoint/logs/runtime.jsonl", nil-}--// plainSessionScript runs the agent with no collector wrapper, for install scenarios where a-// persistent collector is already running as a service.-func plainSessionScript(sc scenario.Scenario, prompt string) string {- return "claude " + strings.Join(claudeFlags(sc, prompt), " ") +- " > claude-out.json 2> claude-err.txt; echo CLAUDE_RC=$?"+ return lay.DefaultLogPath(in.Mode == "system"), nil}diff --git a/beacon-sandbox/runner/runner_test.go b/beacon-sandbox/runner/runner_test.goindex b284d179..a321f1a0 100644--- a/beacon-sandbox/runner/runner_test.go+++ b/beacon-sandbox/runner/runner_test.go@@ -1,7 +1,6 @@package runnerimport (- "fmt""os""os/exec""path/filepath"@@ -18,8 +17,9 @@ import (// otherwise perfect run. That disabled precisely the signal that distinguishes "the agent never// ran" from "Beacon missed the event", which is the distinction the whole verdict model rests on.func TestSessionFilesResolveInsideTheScenarioWorkingDirectory(t *testing.T) {+ lay, sh := image.LinuxLayout(), posixShell{}withCwd := scenario.Scenario{ID: "s05-repo-task", Cwd: "repo"}- got := sessionFile(withCwd, "claude-out.json")+ got := sessionFile(withCwd, "claude-out.json", lay, sh)if !strings.HasSuffix(got, "/repo/claude-out.json") {t.Errorf("a scenario with cwd:repo must read from that directory, got %q", got)@@ -28,14 +28,16 @@ func TestSessionFilesResolveInsideTheScenarioWorkingDirectory(t *testing.T) {t.Error("a bare relative name is what caused the original silent miss")}// It must be the same directory the session was launched in, or the two can drift apart.- if !strings.HasPrefix(got, workDirFor(withCwd)) {- t.Errorf("readback path %q is not under the session working dir %q", got, workDirFor(withCwd))+ if !strings.HasPrefix(got, workDirFor(withCwd, lay, sh)) {+ t.Errorf("readback path %q is not under the session working dir %q",+ got, workDirFor(withCwd, lay, sh))}}// The common case must keep working: no cwd means the default working directory.func TestSessionFilesDefaultToTheWorkDir(t *testing.T) {- got := sessionFile(scenario.Scenario{ID: "s01-hello"}, "claude-out.json")+ got := sessionFile(scenario.Scenario{ID: "s01-hello"}, "claude-out.json",+ image.LinuxLayout(), posixShell{})want := image.WorkDir + "/claude-out.json"if got != want {@@ -46,7 +48,8 @@ func TestSessionFilesDefaultToTheWorkDir(t *testing.T) {// Paths are consumed by a POSIX shell inside the guest, so they must use forward slashes// regardless of the host that built them.func TestSessionFilePathsAreAlwaysSlashed(t *testing.T) {- got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt")+ got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt",+ image.LinuxLayout(), posixShell{})if strings.Contains(got, `\`) {t.Errorf("guest paths must be slash-separated, got %q", got)@@ -56,12 +59,32 @@ func TestSessionFilePathsAreAlwaysSlashed(t *testing.T) {}}+// The mirror of the previous test for the other dialect. A Windows guest consumes these paths+// through Windows APIs, so a forward slash that "mostly works" would still be wrong in every+// verdict a human reads -- and a bare drive letter is a *relative* path, which is wrong in a way+// that silently writes to the wrong directory.+func TestWindowsSessionFilePathsUseBackslashesAndKeepTheDriveRoot(t *testing.T) {+ lay, sh := image.WindowsLayout(), powerShell{}+ got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt", lay, sh)++ if strings.Contains(got, "/") {+ t.Errorf("windows guest paths must be backslash-separated, got %q", got)+ }+ if !strings.HasSuffix(got, `\nested\dir\claude-err.txt`) {+ t.Errorf("nested cwd not resolved, got %q", got)+ }+ // "C:" without a separator means the current directory on that drive, not its root.+ if strings.HasPrefix(got, `C:`) && !strings.HasPrefix(got, `C:\`) {+ t.Errorf("a drive letter must keep its separator, got %q", got)+ }+}+// The argv scan is a disclosure claim, so it must rest on an observation actually made. An// earlier version scanned only after `beacon ci exec` returned, when every process that might// have held the key was already gone -- ARGV_CLEAN was close to vacuous. Cursor Bugbot flagged// it on the first commit. These pin the properties that make the sampler meaningful.func TestArgvSamplerRunsInBackgroundAndSamplesRepeatedly(t *testing.T) {- got := argvSamplerScript()+ got := posixShell{}.ArgvSampler()if !strings.Contains(got, "nohup") || !strings.HasSuffix(strings.TrimSpace(got), "ARGV_SAMPLER_STARTED") {t.Error("the sampler must detach so the session can run alongside it")@@ -79,7 +102,7 @@ func TestArgvSamplerRunsInBackgroundAndSamplesRepeatedly(t *testing.T) {// The matcher must never carry the key in its own argv, or the scan reports a leak it created.func TestArgvSamplerNeverPutsTheKeyInAnArgv(t *testing.T) {- got := argvSamplerScript()+ got := posixShell{}.ArgvSampler()if strings.Contains(got, `grep -qF "$ANTHROPIC_API_KEY"`) || strings.Contains(got, "grep -F $ANTHROPIC_API_KEY") {t.Error("expanding the key into a matcher's argv is the self-poisoning bug")@@ -92,7 +115,7 @@ func TestArgvSamplerNeverPutsTheKeyInAnArgv(t *testing.T) {// With no key there is nothing to search for, so the sampler must say so rather than write a// clean verdict. A check that cannot run must never read as one that passed.func TestArgvSamplerReportsInvalidRatherThanCleanWithoutAKey(t *testing.T) {- got := argvSamplerScript()+ got := posixShell{}.ArgvSampler()if !strings.Contains(got, "ARGV_CHECK_INVALID_NO_KEY") {t.Error("a missing key must produce an explicit invalid marker")@@ -126,7 +149,9 @@ func TestSampleCountIsExtractedForTheRecord(t *testing.T) {// either, so a collector that failed outright was indistinguishable from one that worked.// Reported by the Copilot reviewer.func TestSessionScriptPropagatesTheCollectorExitStatus(t *testing.T) {- got := sessionScript(scenario.Scenario{ID: "s01", Prompt: "hi"}, "hi", "/work/runtime.jsonl")+ sh := posixShell{}+ sc := scenario.Scenario{ID: "s01", Prompt: "hi"}+ got := sh.CIExecSession("/work/runtime.jsonl", image.WorkDir, claudeFlags(sc, "hi", sh))if !strings.Contains(got, "rc=$?") || !strings.Contains(got, "exit $rc") {t.Errorf("the script must capture and re-raise ci exec's status:\n%s", got)@@ -145,9 +170,7 @@ func TestSessionScriptPropagatesTheCollectorExitStatus(t *testing.T) {// be told apart -- and the first is an infrastructure problem while the second is a real absence of// agent work. An explicit marker on every branch removes the ambiguity.func TestSentinelProbeAlwaysEmitsAMarker(t *testing.T) {- // Rebuild the probe the runner uses, for the same sentinel path.- probe := fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",- shq("/home/agent/work/out.txt"))+ probe := posixShell{}.ProbeFile("/home/agent/work/out.txt")for _, want := range []string{"__FOUND__", "__MISSING__"} {if !strings.Contains(probe, want) {@@ -203,7 +226,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {t.Skip("no sh on PATH")}f := filepath.Join(t.TempDir(), "sampler.sh")- if err := os.WriteFile(f, []byte(argvSamplerScript()), 0o600); err != nil {+ if err := os.WriteFile(f, []byte(posixShell{}.ArgvSampler()), 0o600); err != nil {t.Fatal(err)}out, err := exec.Command(sh, "-n", f).CombinedOutput()@@ -212,7 +235,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {}// The inner sampler body is a heredoc, which `sh -n` does not parse. Extract and check it too,// since that is where the polling loop lives.- body := argvSamplerScript()+ body := posixShell{}.ArgvSampler()start := strings.Index(body, "#!/bin/sh")end := strings.Index(body, "\nSAMPLER\n")if start < 0 || end < 0 || end < start {@@ -230,7 +253,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {// A clean verdict from a sampler that never saw an agent process must read as unverified. This is// the invariant the whole tool rests on: silence means verified-clean, never "we did not look."func TestArgvSamplerEmitsWhetherItSawTheAgent(t *testing.T) {- script := argvSamplerScript()+ script := posixShell{}.ArgvSampler()for _, want := range []string{"saw_agent=0", "saw_agent=$saw_agent", "*claude*"} {if !strings.Contains(script, want) {t.Errorf("sampler no longer records whether the agent was in view (missing %q)", want)diff --git a/beacon-sandbox/runner/shell.go b/beacon-sandbox/runner/shell.gonew file mode 100644index 00000000..4fe7d742--- /dev/null+++ b/beacon-sandbox/runner/shell.go@@ -0,0 +1,349 @@+package runner++import (+ "encoding/base64"+ "fmt"+ "path"+ "strings"++ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"+)++// shell builds the guest-side scripts the runner needs, in the guest's own dialect.+//+// Every one of these was a bare POSIX string before Windows existed as a target. They are+// gathered behind an interface rather than sprinkled with `if windows` because the runner's+// value is that a Windows verdict is produced by *the same* control flow as a Linux one -- the+// canary, the sentinel gate, quiescence, collection and judging all stay single-path, and only+// the dialect differs. Branching inline is how the two would drift into one being more+// forgiving than the other.+type shell interface {+ // Quote renders a value as one literal argument.+ Quote(string) string+ // Join builds a guest path from segments.+ Join(...string) string+ // ProbeFile prints __FOUND__ then the contents, or __MISSING__. Never empty, so a failed+ // exec is distinguishable from an empty file.+ ProbeFile(path string) string+ // FileSize prints the byte count, or 0 when absent.+ FileSize(path string) string+ // ReadFile prints the whole file, or nothing when absent.+ ReadFile(path string) string+ // TailBytes prints the last n bytes.+ TailBytes(path string, n int) string+ // WithEnv prefixes a script with one environment assignment.+ WithEnv(name, value, script string) string+ // CIExecSession wraps an agent invocation in `beacon ci exec`, propagating its status.+ CIExecSession(logPath, workDir string, claudeArgs []string) string+ // PlainSession runs the agent with no collector wrapper, for install scenarios.+ PlainSession(workDir string, claudeArgs []string) string+ // ArgvSampler starts the background credential sampler.+ ArgvSampler() string+ // ArgvVerdictPath is where the sampler writes what it saw.+ ArgvVerdictPath() string+}++// shellFor returns the dialect for a platform.+func shellFor(p sandbox.Platform) shell {+ if p.IsWindows() {+ return powerShell{}+ }+ return posixShell{}+}++// ---------------------------------------------------------------------------+// POSIX+// ---------------------------------------------------------------------------++type posixShell struct{}++func (posixShell) Quote(s string) string { return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'" }++func (posixShell) Join(parts ...string) string { return path.Join(parts...) }++// ProbeFile always emits a marker, so its output is never ambiguous. `cat f || echo+// __MISSING__` produced empty stdout for both a failed exec and a legitimately empty sentinel+// file, which would have misreported the second as a broken probe.+func (s posixShell) ProbeFile(p string) string {+ return fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",+ s.Quote(p))+}++func (s posixShell) FileSize(p string) string {+ return "wc -c < " + s.Quote(p) + " 2>/dev/null || echo 0"+}++func (s posixShell) ReadFile(p string) string {+ return "cat " + s.Quote(p) + " 2>/dev/null || true"+}++func (s posixShell) TailBytes(p string, n int) string {+ return fmt.Sprintf("tail -c %d %s 2>/dev/null || true", n, s.Quote(p))+}++func (s posixShell) WithEnv(name, value, script string) string {+ return name + "=" + s.Quote(value) + " " + script+}++// CIExecSession wraps the agent in `beacon ci exec`.+//+// The child is wrapped in `sh -c` so the agent's stdout is redirected *inside* `beacon ci+// exec`. Redirecting outside it would interleave Beacon's own report with the agent's JSON in+// one file, which made the first M0 run's output unparseable.+//+// The script must exit with `beacon ci exec`'s status, not with `tail`'s. Ending on the tail+// made every session look successful no matter how ci exec fared, so a collector that failed+// outright was indistinguishable from one that worked.+func (s posixShell) CIExecSession(logPath, _ string, claudeArgs []string) string {+ inner := "claude " + strings.Join(claudeArgs, " ") + " > claude-out.json 2> claude-err.txt"+ return fmt.Sprintf("beacon ci exec --harness claude --log-path %s -- sh -c %s "++ "> ci-out.txt 2> ci-err.txt; rc=$?; echo CI_EXEC_RC=$rc; "++ "tail -c 400 ci-out.txt; exit $rc",+ s.Quote(logPath), s.Quote(inner))+}++func (posixShell) PlainSession(_ string, claudeArgs []string) string {+ return "claude " + strings.Join(claudeArgs, " ") ++ " > claude-out.json 2> claude-err.txt; echo CLAUDE_RC=$?"+}++func (posixShell) ArgvVerdictPath() string { return "/tmp/beacon-sandbox-argv-verdict" }++// ArgvSampler starts a background sampler that watches for the injected credential in the+// guest process table for as long as the agent session runs.+//+// It has to run concurrently with the session, and an earlier version did not: the scan+// executed after `beacon ci exec` had already returned, so every process that might have held+// the key in argv was already gone and ARGV_CLEAN was close to vacuous -- a disclosure claim+// resting on an observation never made.+//+// Both `ps` output and every /proc/<pid>/cmdline are checked, since a shell can hide argv from+// one but not the other. It runs as root because another user's cmdline is otherwise+// unreadable, and a check that can only see its own process proves nothing.+//+// The matcher must not carry the key in its own argv or the scan finds itself: a first version+// used `grep -qF "$KEY"`, which the shell expands into grep's argv, so the `ps` in the same+// pipeline saw the grep process and reported a leak the tool had created. awk reads the key+// from ENVIRON instead, and the sampler's own processes are skipped as a second guard.+func (s posixShell) ArgvSampler() string {+ return fmt.Sprintf(`set -u+rm -f %[1]s+if [ -z "${ANTHROPIC_API_KEY:-}" ]; then+ echo "ARGV_CHECK_INVALID_NO_KEY samples=0" > %[1]s+ exit 0+fi+cat > %[2]s <<'SAMPLER'+#!/bin/sh+VERDICT="$1"+scan() {+ awk 'BEGIN{k=ENVIRON["ANTHROPIC_API_KEY"]; if (k=="") exit 1}+ /awk|ARGV_SAMPLER_SELF/ {next}+ index($0,k){found=1}+ END{exit !found}'+}+n=0+saw_agent=0+# Bounded so a leaked sandbox cannot spin forever; the instance is destroyed long before this.+while [ "$n" -lt 900 ]; do+ n=$((n+1))+ procs="$(ps -eo args= 2>/dev/null || true)"+ # Whether the agent was ever in view at all. A clean verdict from a sampler that never saw the+ # process holding the key is not evidence of anything.+ case "$procs" in *claude*) saw_agent=1 ;; esac+ if printf '%%s\n' "$procs" | scan; then+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=ps" > "$VERDICT"+ exit 0+ fi+ for f in /proc/[0-9]*/cmdline; do+ [ -r "$f" ] || continue+ cmd="$(tr '\0' '\n' < "$f" 2>/dev/null || true)"+ case "$cmd" in *claude*) saw_agent=1 ;; esac+ if printf '%%s\n' "$cmd" | scan; then+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=proc" > "$VERDICT"+ exit 0+ fi+ done+ echo "ARGV_CLEAN samples=$n saw_agent=$saw_agent" > "$VERDICT"+ sleep 1+done+SAMPLER+chmod +x %[2]s+nohup %[2]s %[1]s > /dev/null 2>&1 &+echo ARGV_SAMPLER_STARTED`, s.ArgvVerdictPath(), "/tmp/beacon-sandbox-argv-sampler.sh")+}++// ---------------------------------------------------------------------------+// PowerShell+// ---------------------------------------------------------------------------++type powerShell struct{}++// Quote renders a PowerShell single-quoted literal, where the only escape is a doubled quote.+// Single-quoted is deliberate: PowerShell performs no expansion inside it, so a value carrying+// `$`, backtick or `%` cannot become something else.+func (powerShell) Quote(s string) string {+ return "'" + strings.ReplaceAll(s, "'", "''") + "'"+}++// Join uses backslashes, since these paths are consumed by Windows APIs and shown to a human+// reading a verdict. Forward slashes would mostly work and would look wrong in every report.+func (powerShell) Join(parts ...string) string {+ cleaned := make([]string, 0, len(parts))+ for _, p := range parts {+ if p = strings.Trim(strings.ReplaceAll(p, "/", `\`), `\`); p != "" {+ cleaned = append(cleaned, p)+ }+ }+ if len(cleaned) == 0 {+ return ""+ }+ // A leading drive letter keeps its separator. "C:" alone is a *relative* path on Windows --+ // the current directory on that drive -- so trimming the backslash off "C:\work" would+ // silently change which directory a script writes to. Everything else joins plainly, since+ // the trim above already removed interior duplicates.+ if len(cleaned) > 1 && strings.HasSuffix(cleaned[0], ":") {+ return cleaned[0] + `\` + strings.Join(cleaned[1:], `\`)+ }+ return strings.Join(cleaned, `\`)+}++// ProbeFile mirrors the POSIX contract: a marker on every branch, so empty stdout can only mean+// the exec itself failed.+//+// -LiteralPath throughout, not -Path: -Path treats [, ] and ? as wildcards, and a sentinel or+// work directory containing them would silently resolve to nothing.+func (s powerShell) ProbeFile(p string) string {+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) { '__FOUND__'; "++ "Get-Content -LiteralPath %[1]s -Raw } else { '__MISSING__' }", s.Quote(p))+}++func (s powerShell) FileSize(p string) string {+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) "++ "{ (Get-Item -LiteralPath %[1]s).Length } else { 0 }", s.Quote(p))+}++func (s powerShell) ReadFile(p string) string {+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) "++ "{ Get-Content -LiteralPath %[1]s -Raw }", s.Quote(p))+}++// TailBytes reads bytes rather than lines, matching `tail -c`, and clamps the start so a file+// shorter than n is returned whole instead of throwing.+func (s powerShell) TailBytes(p string, n int) string {+ q := s.Quote(p)+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) { "++ "$b = [System.IO.File]::ReadAllBytes(%[1]s); "++ "$start = [Math]::Max(0, $b.Length - %[2]d); "++ "[System.Text.Encoding]::UTF8.GetString($b, $start, $b.Length - $start) }", q, n)+}++// WithEnv sets a process-scoped variable. PowerShell has no `VAR=value cmd` prefix form -- the+// POSIX spelling is a syntax error here, which is why this is on the interface at all.+func (s powerShell) WithEnv(name, value, script string) string {+ return fmt.Sprintf("$env:%s = %s; %s", name, s.Quote(value), script)+}++// CIExecSession wraps the agent in `beacon ci exec`, keeping the agent's stdout redirected+// inside the wrapper so Beacon's own report cannot interleave with the agent's JSON.+//+// The inner command travels as base64 into a file rather than as a nested quoted string. The+// prompt carries a canary, arbitrary punctuation and possibly quotes, and it would otherwise+// have to survive two levels of PowerShell quoting -- the failure mode being a prompt that+// arrives subtly altered, which looks exactly like the agent ignoring instructions. This is the+// same reasoning the nested-container guest already uses for crossing two shells.+func (s powerShell) CIExecSession(logPath, workDir string, claudeArgs []string) string {+ inner := s.innerSessionScript(workDir, claudeArgs)+ return fmt.Sprintf(`%s+& beacon ci exec --harness claude --log-path %s -- pwsh -NoProfile -File $innerPath > ci-out.txt 2> ci-err.txt+$rc = $LASTEXITCODE+if ($null -eq $rc) { $rc = 0 }+"CI_EXEC_RC=$rc"+%s+exit $rc`,+ inner, s.Quote(logPath), s.TailBytes("ci-out.txt", 400))+}++// PlainSession runs the agent directly, for install scenarios where a persistent collector is+// already running as a service.+func (s powerShell) PlainSession(workDir string, claudeArgs []string) string {+ return fmt.Sprintf(`%s+& pwsh -NoProfile -File $innerPath+$rc = $LASTEXITCODE+if ($null -eq $rc) { $rc = 0 }+"CLAUDE_RC=$rc"`, s.innerSessionScript(workDir, claudeArgs))+}++// innerSessionScript materializes the agent invocation as a .ps1 and leaves its path in+// $innerPath.+//+// Set-Location inside the inner script is load-bearing: the redirects are relative, and the+// process that ends up running this file is a grandchild of the shell that wrote it, so its+// working directory cannot be assumed.+func (s powerShell) innerSessionScript(workDir string, claudeArgs []string) string {+ body := fmt.Sprintf("Set-Location -LiteralPath %s\n"++ "& claude %s > claude-out.json 2> claude-err.txt\n"++ "exit $LASTEXITCODE\n",+ s.Quote(workDir), strings.Join(claudeArgs, " "))+ enc := base64.StdEncoding.EncodeToString([]byte(body))+ return fmt.Sprintf(`$innerPath = Join-Path $env:TEMP 'beacon-sandbox-session.ps1'+[System.IO.File]::WriteAllText($innerPath, [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('%s')))`, enc)+}++func (powerShell) ArgvVerdictPath() string {+ return `C:\Windows\Temp\beacon-sandbox-argv-verdict`+}++// ArgvSampler is the Windows half of the credential-in-argv check, and it has the same job as+// the POSIX one: observe the process table *while* the agent runs, and never carry the key in+// its own command line.+//+// Win32_Process replaces both /proc and ps -- Windows has no equivalent of either, and+// CommandLine on that class is the only place a full argv is readable for another process.+// The key is compared with String.Contains against a value read from the environment, so it+// never reaches an argument list; the sampler also skips its own PID, the same second guard the+// POSIX version keeps.+//+// Start-Process rather than a background job: a PowerShell job dies with the shell that created+// it, and this shell exits as soon as Exec returns -- which is before the session even starts.+// The sampler has to outlive its parent, exactly as nohup makes it on POSIX.+func (s powerShell) ArgvSampler() string {+ verdict := s.ArgvVerdictPath()+ scriptPath := `C:\Windows\Temp\beacon-sandbox-argv-sampler.ps1`+ body := `param([string]$Verdict)+$key = $env:ANTHROPIC_API_KEY+if (-not $key) {+ Set-Content -LiteralPath $Verdict -Value 'ARGV_CHECK_INVALID_NO_KEY samples=0'+ exit 0+}+$me = $PID+$n = 0+$sawAgent = 0+# Bounded so a leaked runner cannot spin forever; the job ends long before this.+while ($n -lt 900) {+ $n+++ $procs = @(Get-CimInstance Win32_Process -ErrorAction SilentlyContinue)+ foreach ($p in $procs) {+ if ($p.ProcessId -eq $me) { continue }+ $cl = $p.CommandLine+ if (-not $cl) { continue }+ if ($cl -like '*claude*') { $sawAgent = 1 }+ if ($cl.Contains($key)) {+ Set-Content -LiteralPath $Verdict -Value "ARGV_LEAK samples=$n saw_agent=$sawAgent via=win32process"+ exit 0+ }+ }+ Set-Content -LiteralPath $Verdict -Value "ARGV_CLEAN samples=$n saw_agent=$sawAgent"+ Start-Sleep -Seconds 1+}+`+ enc := base64.StdEncoding.EncodeToString([]byte(body))+ return fmt.Sprintf(`$ErrorActionPreference = 'Continue'+Remove-Item -LiteralPath %[1]s -ErrorAction SilentlyContinue+if (-not $env:ANTHROPIC_API_KEY) {+ Set-Content -LiteralPath %[1]s -Value 'ARGV_CHECK_INVALID_NO_KEY samples=0'+ exit 0+}+[System.IO.File]::WriteAllText(%[2]s, [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('%[3]s')))+Start-Process -FilePath 'pwsh' -ArgumentList '-NoProfile','-NonInteractive','-File',%[2]s,%[1]s -WindowStyle Hidden+'ARGV_SAMPLER_STARTED'`, s.Quote(verdict), s.Quote(scriptPath), enc)+}diff --git a/beacon-sandbox/runner/shell_test.go b/beacon-sandbox/runner/shell_test.gonew file mode 100644index 00000000..95b4c8d2--- /dev/null+++ b/beacon-sandbox/runner/shell_test.go@@ -0,0 +1,309 @@+package runner++import (+ "encoding/base64"+ "os"+ "os/exec"+ "path/filepath"+ "regexp"+ "strings"+ "testing"++ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/scenario"+)++// The dialect must follow the *provider's* platform, not the host's. A Mac dispatching a Windows+// run is the normal case, so keying off runtime.GOOS would generate bash for a Windows guest and+// every script would fail as a syntax error.+func TestShellFollowsTheGuestPlatformNotTheHost(t *testing.T) {+ if _, ok := shellFor(sandbox.PlatformWindows).(powerShell); !ok {+ t.Errorf("a Windows guest must get the PowerShell dialect, got %T",+ shellFor(sandbox.PlatformWindows))+ }+ if _, ok := shellFor(sandbox.PlatformPosix).(posixShell); !ok {+ t.Errorf("a POSIX guest must get the POSIX dialect, got %T",+ shellFor(sandbox.PlatformPosix))+ }+}++// Quoting is the seam where a canary or prompt gets silently altered, and an altered prompt looks+// exactly like the agent ignoring instructions. PowerShell's only escape inside a single-quoted+// literal is a doubled quote, and nothing else is expanded -- which is why single quotes are used.+func TestPowerShellQuotingIsLiteral(t *testing.T) {+ sh := powerShell{}+ cases := map[string]string{+ `plain`: `'plain'`,+ `it's`: `'it''s'`,+ `$env:PATH`: `'$env:PATH'`,+ "back`tick": "'back`tick'",+ `100% done`: `'100% done'`,+ `C:\Program Files`: `'C:\Program Files'`,+ }+ for in, want := range cases {+ if got := sh.Quote(in); got != want {+ t.Errorf("Quote(%q) = %s, want %s", in, got, want)+ }+ }+}++// Every dialect must be able to emit both markers. Empty stdout has to mean "the exec failed" and+// nothing else -- `cat f || echo __MISSING__` produced empty output for both a failed exec and a+// legitimately empty sentinel, and those are an infrastructure problem and a real absence of agent+// work respectively.+func TestBothDialectsProbeUnambiguously(t *testing.T) {+ for _, sh := range []shell{posixShell{}, powerShell{}} {+ probe := sh.ProbeFile("/tmp/out.txt")+ for _, want := range []string{"__FOUND__", "__MISSING__"} {+ if !strings.Contains(probe, want) {+ t.Errorf("%T probe cannot emit %s: %s", sh, want, probe)+ }+ }+ }+}++// -LiteralPath, not -Path. -Path treats [, ] and ? as wildcards, so a sentinel or work directory+// containing one would resolve to nothing and the probe would report the agent idle.+func TestPowerShellFileOperationsDoNotGlob(t *testing.T) {+ sh := powerShell{}+ scripts := map[string]string{+ "ProbeFile": sh.ProbeFile(`C:\work\a[1].txt`),+ "FileSize": sh.FileSize(`C:\work\a[1].txt`),+ "ReadFile": sh.ReadFile(`C:\work\a[1].txt`),+ "TailBytes": sh.TailBytes(`C:\work\a[1].txt`, 400),+ }+ // Leading whitespace is what distinguishes the *parameter* -Path from the cmdlet name+ // Test-Path, and -LiteralPath does not contain "-Path" at all.+ bareParam := regexp.MustCompile(`\s-Path\b`)+ for name, script := range scripts {+ if bareParam.MatchString(script) {+ t.Errorf("%s uses -Path, which treats [ ] ? as wildcards: %s", name, script)+ }+ }+}++// PowerShell has no `VAR=value cmd` prefix form; the POSIX spelling is a syntax error there. This+// is the whole reason WithEnv is on the interface, and it is the same shape as the bug that makes+// every hook command Beacon writes unusable on Windows.+func TestWithEnvUsesEachDialectsOwnForm(t *testing.T) {+ if got := (posixShell{}).WithEnv("SUDO_USER", "agent", "beacon x"); got != "SUDO_USER='agent' beacon x" {+ t.Errorf("posix WithEnv = %q", got)+ }+ got := powerShell{}.WithEnv("SUDO_USER", "runneradmin", "beacon x")+ if !strings.HasPrefix(got, "$env:SUDO_USER = 'runneradmin';") {+ t.Errorf("powershell WithEnv must assign $env:, got %q", got)+ }+ if strings.HasPrefix(got, "SUDO_USER=") {+ t.Error("the POSIX inline-assignment form is a syntax error in PowerShell")+ }+}++// The Windows sampler has the same contract as the POSIX one: it must outlive the shell that+// started it, loop rather than sample once, and record whether the agent was ever in view.+func TestWindowsArgvSamplerDetachesAndLoops(t *testing.T) {+ got := powerShell{}.ArgvSampler()++ // A PowerShell background job dies with its parent, and the parent exits as soon as Exec+ // returns -- before the session even starts. Start-Process survives it, as nohup does on POSIX.+ if !strings.Contains(got, "Start-Process") {+ t.Error("the sampler must detach; a background job dies with the shell that created it")+ }+ if strings.Contains(got, "Start-Job") {+ t.Error("Start-Job does not outlive its parent shell, so the sampler would never observe the session")+ }+ if !strings.HasSuffix(strings.TrimSpace(got), "'ARGV_SAMPLER_STARTED'") {+ t.Error("the starter must announce itself, or a sampler that never ran looks like one that found nothing")+ }++ body := decodedSamplerBody(t, got)+ if !strings.Contains(body, "while ($n -lt") || !strings.Contains(body, "Start-Sleep") {+ t.Error("a single sample cannot observe a live session; it must loop")+ }+ // Win32_Process.CommandLine is the only place another process's full argv is readable on+ // Windows -- there is no /proc and no ps.+ if !strings.Contains(body, "Win32_Process") || !strings.Contains(body, ".CommandLine") {+ t.Errorf("the sampler must read command lines from Win32_Process:\n%s", body)+ }+ for _, want := range []string{"saw_agent=$sawAgent", "ARGV_CLEAN", "ARGV_LEAK"} {+ if !strings.Contains(body, want) {+ t.Errorf("sampler body is missing %q:\n%s", want, body)+ }+ }+}++// The matcher must never carry the key in its own command line, or the scan finds itself and+// reports a leak the tool created. On POSIX that meant awk reading ENVIRON; here it means+// String.Contains against a value read from $env, plus skipping its own PID.+func TestWindowsArgvSamplerNeverPutsTheKeyInACommandLine(t *testing.T) {+ got := powerShell{}.ArgvSampler()+ body := decodedSamplerBody(t, got)++ if !strings.Contains(body, "$key = $env:ANTHROPIC_API_KEY") {+ t.Error("the key must be read from the environment inside the sampler")+ }+ if !strings.Contains(body, "$cl.Contains($key)") {+ t.Errorf("the comparison must be a method call so the key never reaches an argument list:\n%s", body)+ }+ // Select-String and findstr both take the pattern as an argument, which is the self-poisoning+ // shape: the sampler's own process table entry would then contain the key.+ for _, bad := range []string{"Select-String", "findstr"} {+ if strings.Contains(body, bad) {+ t.Errorf("%s takes the pattern as an argument, which puts the key in argv", bad)+ }+ }+ if !strings.Contains(body, "$p.ProcessId -eq $me") {+ t.Error("the sampler must skip its own process, the second guard against finding itself")+ }+}++// With no key there is nothing to search for, so the sampler must say so rather than write a clean+// verdict. A check that could not run must never read as one that passed.+func TestWindowsArgvSamplerReportsInvalidRatherThanCleanWithoutAKey(t *testing.T) {+ got := powerShell{}.ArgvSampler()++ if !strings.Contains(got, "ARGV_CHECK_INVALID_NO_KEY") {+ t.Error("a missing key must produce an explicit invalid marker")+ }+ // The guard has to come first in the starter, before anything that could emit a clean verdict.+ guard := strings.Index(got, "if (-not $env:ANTHROPIC_API_KEY)")+ start := strings.Index(got, "Start-Process")+ if guard < 0 || start < 0 || guard > start {+ t.Error("the no-key guard must precede launching the sampler")+ }+ body := decodedSamplerBody(t, got)+ bodyGuard := strings.Index(body, "if (-not $key)")+ bodyClean := strings.Index(body, "ARGV_CLEAN")+ if bodyGuard < 0 || bodyClean < 0 || bodyGuard > bodyClean {+ t.Error("the sampler body must check for the key before any path that can emit ARGV_CLEAN")+ }+}++// The session's inner command travels as base64 rather than as a nested quoted string. The prompt+// carries a canary and arbitrary punctuation, and surviving two levels of PowerShell quoting is+// how a prompt arrives subtly altered -- which is indistinguishable from the agent ignoring it.+func TestWindowsSessionScriptCarriesThePromptOutOfBandAndPropagatesStatus(t *testing.T) {+ sh := powerShell{}+ prompt := `write 'BEACON_SANDBOX_abc' to $out -- 100% now`+ got := sh.CIExecSession(`C:\work\runtime.jsonl`, `C:\work`,+ claudeFlags(stubScenario(), prompt, sh))++ if strings.Contains(got, prompt) {+ t.Error("the prompt must not appear inline; it should be base64 in the staged inner script")+ }+ inner := decodeFirstBase64(t, got)+ // The prompt must survive quoting unchanged, which is the property that matters: an altered+ // prompt is indistinguishable from the agent ignoring instructions. Checked by undoing+ // PowerShell's only single-quote escape and requiring the original back.+ if !strings.Contains(inner, sh.Quote(prompt)) {+ t.Errorf("the staged inner script must carry the quoted prompt:\n%s", inner)+ }+ if roundTripped := strings.ReplaceAll(sh.Quote(prompt), "''", "'"); roundTripped != "'"+prompt+"'" {+ t.Errorf("quoting does not round-trip: %q became %q", prompt, roundTripped)+ }+ // Set-Location is load-bearing: the redirects are relative and the process that runs this file+ // is a grandchild of the shell that wrote it, so its working directory cannot be assumed.+ if !strings.Contains(inner, "Set-Location -LiteralPath 'C:\\work'") {+ t.Errorf("the inner script must set its own working directory:\n%s", inner)+ }+ if !strings.Contains(inner, "> claude-out.json 2> claude-err.txt") {+ t.Error("the agent's stdout must be redirected inside the collector wrapper, not outside it")+ }+ // Same invariant as the POSIX side: the script exits with ci exec's status, and the diagnostic+ // tail runs before that exit so its output is still captured.+ if !strings.Contains(got, "$rc = $LASTEXITCODE") || !strings.Contains(got, "exit $rc") {+ t.Errorf("the script must capture and re-raise ci exec's status:\n%s", got)+ }+ if strings.Index(got, "exit $rc") < strings.Index(got, "ReadAllBytes('ci-out.txt')") {+ t.Error("the tail should run before the exit so its output is still captured")+ }+}++// $LASTEXITCODE is $null when no native command has run yet, and `exit $null` exits 0 -- which+// would report a session that never launched as a success.+func TestWindowsSessionScriptsGuardAgainstANullExitCode(t *testing.T) {+ sh := powerShell{}+ for name, script := range map[string]string{+ "CIExecSession": sh.CIExecSession("log", `C:\work`, []string{"-p", "'hi'"}),+ "PlainSession": sh.PlainSession(`C:\work`, []string{"-p", "'hi'"}),+ } {+ if !strings.Contains(script, "if ($null -eq $rc) { $rc = 0 }") {+ t.Errorf("%s must normalize a null $LASTEXITCODE:\n%s", name, script)+ }+ }+}++// The generated PowerShell is assembled in Go, so a syntax error in it is invisible to the+// compiler and would surface as "the check did not run" -- which a verdict reports as unverified+// rather than failing loudly. Parsed with PowerShell's own parser when one is available.+func TestGeneratedPowerShellParses(t *testing.T) {+ pwsh, err := exec.LookPath("pwsh")+ if err != nil {+ t.Skip("no pwsh on PATH; the windows-sandbox CI job covers this")+ }+ sh := powerShell{}+ scripts := map[string]string{+ "ArgvSampler": sh.ArgvSampler(),+ "SamplerBody": decodedSamplerBody(t, sh.ArgvSampler()),+ "CIExecSession": sh.CIExecSession(`C:\work\runtime.jsonl`, `C:\work`, claudeFlags(stubScenario(), "hi", sh)),+ "PlainSession": sh.PlainSession(`C:\work`, claudeFlags(stubScenario(), "hi", sh)),+ "ProbeFile": sh.ProbeFile(`C:\work\out.txt`),+ "FileSize": sh.FileSize(`C:\work\out.txt`),+ "ReadFile": sh.ReadFile(`C:\work\out.txt`),+ "TailBytes": sh.TailBytes(`C:\work\out.txt`, 400),+ "VersionProbe": versionProbe(sandbox.PlatformWindows),+ }+ for name, script := range scripts {+ path := filepath.Join(t.TempDir(), name+".ps1")+ if err := os.WriteFile(path, []byte(script), 0o600); err != nil {+ t.Fatal(err)+ }+ // Parse without executing: the parser reports syntax errors and nothing runs.+ check := `$errors = $null+[void][System.Management.Automation.Language.Parser]::ParseFile('` + path + `', [ref]$null, [ref]$errors)+if ($errors) { $errors | ForEach-Object { $_.Message }; exit 1 }`+ if out, err := exec.Command(pwsh, "-NoProfile", "-NonInteractive", "-Command", check).CombinedOutput(); err != nil {+ t.Errorf("generated %s is not valid PowerShell: %v\n%s\n--- script ---\n%s",+ name, err, out, script)+ }+ }+}++// stubScenario is a minimal scenario for script-shape tests.+func stubScenario() scenario.Scenario {+ return scenario.Scenario{ID: "w00-probe", Prompt: "unused; the prompt is passed separately"}+}++// decodedSamplerBody extracts and decodes the staged sampler script.+func decodedSamplerBody(t *testing.T, starter string) string {+ t.Helper()+ return decodeLastBase64(t, starter)+}++var base64Literal = regexp.MustCompile(`FromBase64String\('([A-Za-z0-9+/=]+)'\)`)++func decodeFirstBase64(t *testing.T, script string) string {+ t.Helper()+ m := base64Literal.FindStringSubmatch(script)+ if m == nil {+ t.Fatalf("no base64 payload found in:\n%s", script)+ }+ return decode(t, m[1])+}++func decodeLastBase64(t *testing.T, script string) string {+ t.Helper()+ all := base64Literal.FindAllStringSubmatch(script, -1)+ if len(all) == 0 {+ t.Fatalf("no base64 payload found in:\n%s", script)+ }+ return decode(t, all[len(all)-1][1])+}++func decode(t *testing.T, enc string) string {+ t.Helper()+ b, err := base64.StdEncoding.DecodeString(enc)+ if err != nil {+ t.Fatalf("payload is not valid base64: %v", err)+ }+ return string(b)+}diff --git a/beacon-sandbox/sandbox/localexec.go b/beacon-sandbox/sandbox/localexec.gonew file mode 100644index 00000000..71405545--- /dev/null+++ b/beacon-sandbox/sandbox/localexec.go@@ -0,0 +1,298 @@+package sandbox++import (+ "context"+ "errors"+ "fmt"+ "io"+ "os"+ "os/exec"+ "path/filepath"+ "runtime"+ "strings"+)++// LocalExec is the Provider for "the machine this process is already on".+//+// It exists because Modal has no Windows, and the alternatives all cost something worse than+// this does. A GitHub Actions Windows runner is already a disposable machine with a real+// Service Control Manager, administrator rights and no UAC -- everything an endpoint install+// needs. So rather than rent an instance from inside it, the runner *is* the instance, and+// this backend just runs commands where it stands.+//+// The consequence is the important part. Every other Provider hands back a separate machine,+// which is what lets hostguard prove Beacon was never installed outside the sandbox. Here+// guest and host are the same machine, so that proof has to come from somewhere else -- see+// MutatesHost and requireDisposable.+type LocalExec struct {+ // shell is the interpreter Exec hands scripts to.+ shell string+ shellArgs []string+ platform Platform+ // workdir is the base directory instances report; scripts may cd elsewhere.+ workdir string+}++// LocalExecOptions configures the backend.+type LocalExecOptions struct {+ // Workdir is the base directory for launched instances. Empty uses the process's own.+ Workdir string+ // AllowHostMutation runs scenarios on a machine this tool cannot prove is disposable.+ //+ // Off by default, and the default is the safe one: a scenario installs Beacon for real,+ // registers services and rewrites agent-runtime settings files. On a workstation that is+ // not a test, it is damage. Contributors who genuinely want this -- a throwaway VM they+ // own -- have to say so.+ AllowHostMutation bool+}++type localImage struct{ ref string }++func (l localImage) Ref() string { return l.ref }++type localInstance struct{ id string }++func (l localInstance) ID() string { return l.id }++// NewLocalExec returns a backend for the current machine, refusing unless the machine is+// provably disposable or the caller has explicitly accepted the risk.+func NewLocalExec(opts LocalExecOptions) (*LocalExec, error) {+ if err := requireDisposable(opts.AllowHostMutation); err != nil {+ return nil, err+ }+ l := &LocalExec{workdir: opts.Workdir}+ if runtime.GOOS == "windows" {+ l.platform = PlatformWindows+ // PowerShell 7, and not Windows PowerShell 5.1 as a fallback.+ //+ // 5.1's `>` writes UTF-16LE, which would silently corrupt every JSON artifact this tool+ // parses -- claude-out.json above all, the file that says whether the session succeeded.+ // A quiet encoding difference producing "the agent failed" is worse than refusing here,+ // and pwsh is what the runner images ship and what Claude Code itself uses on Windows.+ if _, err := exec.LookPath("pwsh"); err != nil {+ return nil, fmt.Errorf("pwsh (PowerShell 7+) is required on Windows and is not on " ++ "PATH; Windows PowerShell 5.1 redirects output as UTF-16, which corrupts the " ++ "JSON artifacts this tool reads")+ }+ l.shell = "pwsh"+ // -NoProfile matters for reproducibility: a profile can alter PATH, aliases and the+ // error-action preference, so a run would otherwise depend on the host's dotfiles.+ l.shellArgs = []string{"-NoProfile", "-NonInteractive", "-Command"}+ return l, nil+ }+ l.platform = PlatformPosix+ l.shell = "bash"+ if _, err := exec.LookPath("bash"); err != nil {+ l.shell = "sh"+ }+ l.shellArgs = []string{"-c"}+ return l, nil+}++// requireDisposable refuses to run on a machine whose destruction is not somebody else's job.+//+// GITHUB_ACTIONS alone is not enough. A self-hosted runner is a real machine that persists+// between jobs, so an install performed there survives the run and the next job inherits it --+// which is exactly the escape hostguard exists to catch, just relocated. RUNNER_ENVIRONMENT is+// GitHub's own answer to the question, so it is what gets asked.+func requireDisposable(allow bool) error {+ if allow {+ return nil+ }+ if os.Getenv("GITHUB_ACTIONS") != "true" {+ return fmt.Errorf("the local provider runs scenarios on this machine, and a scenario " ++ "installs Beacon, registers services and rewrites agent-runtime settings for real; " ++ "it is meant for a disposable CI runner. Use `--provider github` to dispatch a run to " ++ "one, or pass --allow-host-mutation if this machine is genuinely throwaway")+ }+ if env := os.Getenv("RUNNER_ENVIRONMENT"); env != "" && env != "github-hosted" {+ return fmt.Errorf("this is a %s Actions runner, which persists between jobs, so an "++ "install performed here would outlive the run; pass --allow-host-mutation only if "++ "this runner is genuinely disposable", env)+ }+ return nil+}++// DisposabilityEvidence describes why this machine was accepted, for run metadata.+//+// Recorded rather than implied. hostguard's comparison cannot mean anything here, so the+// verdict has to say what it relied on instead -- and "the operator asserted it" is a+// materially weaker claim than "GitHub hands out a fresh VM per job", which a reader of the+// verdict is entitled to tell apart.+func DisposabilityEvidence() string {+ if os.Getenv("GITHUB_ACTIONS") == "true" {+ env := os.Getenv("RUNNER_ENVIRONMENT")+ if env == "" {+ env = "unreported"+ }+ if env == "github-hosted" {+ return "github-hosted ephemeral runner (fresh VM per job)"+ }+ return "GitHub Actions runner, environment " + env + ", accepted by explicit operator opt-in"+ }+ return "operator asserted this machine is disposable (--allow-host-mutation)"+}++func (l *LocalExec) Name() string { return "local" }++func (l *LocalExec) Platform() Platform { return l.platform }++// MutatesHost is true, and saying so is the whole point of the flag: the runner reads it and+// swaps hostguard's comparison for a disposability assertion instead of reporting a false+// escape.+func (l *LocalExec) MutatesHost() bool { return true }++// EnsureImage is a no-op. Provisioning happened before this process started -- the CI job+// installed the agent runtime and built the binaries under test -- so there is no image to+// build and nothing to cache. The spec's layers are reported in the ref so a verdict still+// records what environment was expected.+func (l *LocalExec) EnsureImage(_ context.Context, spec ImageSpec) (Image, error) {+ base := spec.Base+ if base == "" {+ base = runtime.GOOS + "/" + runtime.GOARCH+ }+ return localImage{ref: "local:" + base}, nil+}++// Snapshot is not available: there is no instance boundary to snapshot across. Returning an+// error rather than a silent no-op keeps a caller that depends on snapshotting from believing+// it got one.+func (l *LocalExec) Snapshot(_ context.Context, _ Instance) (Image, error) {+ return nil, fmt.Errorf("the local provider cannot snapshot: the guest is this machine")+}++// Launch records the workdir and returns a handle. Nothing starts, because the machine is+// already running.+func (l *LocalExec) Launch(_ context.Context, _ Image, spec LaunchSpec) (Instance, error) {+ if spec.Workdir != "" {+ l.workdir = spec.Workdir+ }+ // Secrets are deliberately not applied to this process's environment. They are already+ // present here -- the CI job injected them -- and mutating the parent's environment would+ // leak them into every later command, including ones a scenario never meant to see them.+ host, _ := os.Hostname()+ if host == "" {+ host = "localhost"+ }+ return localInstance{id: fmt.Sprintf("local:%s:%d", host, os.Getpid())}, nil+}++// Exec runs a script through the platform's shell.+func (l *LocalExec) Exec(ctx context.Context, _ Instance, script string, opts ExecOpts) (Result, error) {+ // User switching is not supported: this process cannot become another user without+ // privileges it has no business holding, and su/runas would need a password. A scenario+ // that needs an unprivileged account must have the CI job create one and run the whole+ // tool as them. Reported rather than ignored, because silently running as the wrong user+ // is how a scenario ends up verifying root's unused settings file.+ if opts.User != "" && !l.currentUserIs(opts.User) {+ return Result{}, fmt.Errorf("the local provider cannot switch to user %q; run the "++ "harness as that user instead", opts.User)+ }+ if opts.Timeout > 0 {+ var cancel context.CancelFunc+ ctx, cancel = context.WithTimeout(ctx, opts.Timeout)+ defer cancel()+ }++ cmd := exec.CommandContext(ctx, l.shell, append(l.shellArgs, script)...)+ cmd.Dir = l.resolveDir(opts.Dir)+ cmd.Env = l.execEnv(opts)++ var stdout, stderr strings.Builder+ cmd.Stdout = &stdout+ cmd.Stderr = &stderr+ err := cmd.Run()++ res := Result{Stdout: stdout.String(), Stderr: stderr.String()}+ // An ExitError carries the status and is not a failure of Exec itself; every caller reads+ // ExitCode and decides what it means. Anything else -- the shell missing, a timeout -- is a+ // harness failure and must surface as one, or "the command failed" and "we never ran it"+ // become the same result.+ if err != nil {+ var ee *exec.ExitError+ if errors.As(err, &ee) {+ res.ExitCode = ee.ExitCode()+ return res, nil+ }+ return res, fmt.Errorf("exec %s: %w", l.shell, err)+ }+ return res, nil+}++// execEnv builds the child environment, honoring the same ExecOpts fields the remote backends do.+func (l *LocalExec) execEnv(opts ExecOpts) []string {+ env := os.Environ()+ if opts.HomeDir != "" {+ env = append(env, "HOME="+opts.HomeDir)+ if l.platform.IsWindows() {+ // Windows programs read USERPROFILE, not HOME. Setting only HOME would leave a+ // scenario's home-relative paths pointing at the real profile.+ env = append(env, "USERPROFILE="+opts.HomeDir)+ }+ }+ if len(opts.PathPrepend) > 0 {+ sep := ":"+ if l.platform.IsWindows() {+ sep = ";"+ }+ env = append(env, "PATH="+strings.Join(opts.PathPrepend, sep)+sep+os.Getenv("PATH"))+ }+ return env+}++func (l *LocalExec) resolveDir(dir string) string {+ switch {+ case dir != "":+ return dir+ case l.workdir != "":+ return l.workdir+ default:+ return ""+ }+}++// currentUserIs reports whether the requested user is the one already running.+func (l *LocalExec) currentUserIs(want string) bool {+ for _, key := range []string{"USER", "USERNAME", "LOGNAME"} {+ if v := os.Getenv(key); v != "" && strings.EqualFold(v, want) {+ return true+ }+ }+ return false+}++// Put and Get are local copies. They keep the Provider shape so the runner does not care+// which backend it has, and they still cross a boundary worth crossing: run artifacts land in+// the run directory rather than being read from wherever the scenario left them.+func (l *LocalExec) Put(_ context.Context, _ Instance, localPath, remotePath string) error {+ return copyFile(localPath, remotePath)+}++func (l *LocalExec) Get(_ context.Context, _ Instance, remotePath, localPath string) error {+ return copyFile(remotePath, localPath)+}++// Terminate is a no-op: destroying this machine is the CI job's business, not ours.+func (l *LocalExec) Terminate(_ context.Context, _ Instance) error { return nil }++func copyFile(src, dst string) error {+ in, err := os.Open(src)+ if err != nil {+ return err+ }+ defer in.Close()+ if err := os.MkdirAll(filepath.Dir(dst), 0o700); err != nil {+ return err+ }+ // 0600, matching the run directory: collected artifacts retain prompt text and command+ // output, and on a shared machine a wider mode hands that to every local user.+ out, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600)+ if err != nil {+ return err+ }+ if _, err := io.Copy(out, in); err != nil {+ out.Close()+ return err+ }+ return out.Close()+}diff --git a/beacon-sandbox/sandbox/modal.go b/beacon-sandbox/sandbox/modal.goindex b1aa82f2..0aaa8b67 100644--- a/beacon-sandbox/sandbox/modal.go+++ b/beacon-sandbox/sandbox/modal.go@@ -57,6 +57,14 @@ func (m *Modal) Close() {func (m *Modal) Name() string { return "modal" }+// Platform is always POSIX: Modal offers Linux only, which is the whole reason a second+// backend exists.+func (m *Modal) Platform() Platform { return PlatformPosix }++// MutatesHost is false -- a Modal sandbox is a different machine, so hostguard's comparison+// means what it says.+func (m *Modal) MutatesHost() bool { return false }+func (m *Modal) EnsureImage(ctx context.Context, spec ImageSpec) (Image, error) {img := m.client.Images.FromRegistry(spec.Base, nil)if len(spec.Layers) > 0 {diff --git a/beacon-sandbox/sandbox/provider.go b/beacon-sandbox/sandbox/provider.goindex 6bf5073f..93548b21 100644--- a/beacon-sandbox/sandbox/provider.go+++ b/beacon-sandbox/sandbox/provider.go@@ -20,6 +20,23 @@ const (LaneVM Lane = "vm")+// Platform names the guest operating-system family.+//+// It decides two things that cannot be guessed from anything else: which dialect Exec+// interprets a script as, and which path syntax a scenario has to speak. Everything above+// the provider branches on this rather than on the *host's* runtime.GOOS, because those are+// not the same question -- a Mac dispatching a Windows run is the normal case.+type Platform string++const (+ PlatformPosix Platform = "posix"+ PlatformWindows Platform = "windows"+)++// IsWindows is a readability shorthand for the many call sites that only care which side of+// the split they are on.+func (p Platform) IsWindows() bool { return p == PlatformWindows }+// ImageSpec describes a reproducible environment. Layers are ordered and cached by the// provider, so an unchanged prefix is free on re-run.type ImageSpec struct {@@ -102,6 +119,20 @@ type Provider interface {// Name identifies the backend in run metadata.Name() string+ // Platform reports the guest OS family, so callers know which script dialect Exec+ // accepts and which path syntax to build.+ Platform() Platform++ // MutatesHost reports whether this backend runs the scenario on the same machine the+ // caller is on, rather than in a separate instance.+ //+ // This is not a detail: hostguard exists to prove Beacon was never installed outside the+ // sandbox, and it compares the *caller's* machine before and after. When guest and host+ // are the same machine that comparison inverts -- a correct install trips the most serious+ // finding this tool has. A backend that says true is telling the runner to assert+ // disposability some other way instead of silently skipping the check.+ MutatesHost() bool+// EnsureImage builds or resolves an environment. Implementations should be// idempotent and lean on content-addressed caching.EnsureImage(ctx context.Context, spec ImageSpec) (Image, error)diff --git a/beacon-sandbox/scenario/scenario.go b/beacon-sandbox/scenario/scenario.goindex 812182ff..aecb3313 100644--- a/beacon-sandbox/scenario/scenario.go+++ b/beacon-sandbox/scenario/scenario.go@@ -67,10 +67,27 @@ type Install struct {NeedsRealSystemd bool `yaml:"needs_real_systemd,omitempty"`}+// Platform selects the guest OS family a scenario is written for.+//+// A scenario is not portable by default and pretending otherwise would be worse than declaring+// it: the prompt names shell commands, the sentinel is an absolute path, and the expectations+// assert what Beacon captured from a particular runtime. So a scenario says which world it+// belongs to, and the suite loader runs only the ones that match the target.+type Platform string++const (+ // PlatformLinux is the default, so every scenario written before Windows existed keeps+ // working unchanged.+ PlatformLinux Platform = "linux"+ PlatformWindows Platform = "windows"+)+// Scenario is one declarative case.type Scenario struct {- ID string `yaml:"id"`- Prompt string `yaml:"prompt"`+ ID string `yaml:"id"`+ // Platform defaults to linux when unset.+ Platform Platform `yaml:"platform,omitempty"`+ Prompt string `yaml:"prompt"`// Install, when set, performs a persistent endpoint install and runs the session against// it, with no `beacon ci exec` wrapper anywhere.Install *Install `yaml:"install,omitempty"`@@ -101,6 +118,14 @@ type Suite struct {Scenarios []Scenario}+// TargetPlatform resolves the platform with its default.+func (s Scenario) TargetPlatform() Platform {+ if s.Platform == "" {+ return PlatformLinux+ }+ return s.Platform+}+// Timeout resolves the session bound with a sane default.func (s Scenario) Timeout() time.Duration {if s.TimeoutSeconds > 0 {@@ -128,6 +153,27 @@ func (s Scenario) Validate() error {if len(s.Expect) == 0 {return fmt.Errorf("%s: at least one expectation is required, else the run asserts nothing", s.ID)}+ switch s.TargetPlatform() {+ case PlatformLinux, PlatformWindows:+ default:+ return fmt.Errorf("%s: platform %q must be linux or windows", s.ID, s.Platform)+ }+ if s.TargetPlatform() == PlatformWindows {+ // systemd does not exist on Windows, and the nested-container recipe that provides it is+ // a Linux arrangement. A scenario asking for both would be arranged as neither and would+ // then quietly verify whatever backend happened to be selected.+ if s.NeedsRealSystemd() {+ return fmt.Errorf("%s: needs_real_systemd cannot apply to a windows scenario; "++ "the Windows service manager is the SCM, which is always present", s.ID)+ }+ // An absolute POSIX sentinel on Windows is the mistake that would be hardest to notice:+ // the agent writes somewhere, the probe looks somewhere else, and the run reports the+ // agent idle rather than the scenario being wrong.+ if strings.HasPrefix(s.Sentinel, "/") {+ return fmt.Errorf("%s: sentinel %q is a POSIX path in a windows scenario; use "++ "{{workdir}} or a drive-qualified path", s.ID, s.Sentinel)+ }+ }for i, e := range s.Expect {if strings.TrimSpace(e.Action) == "" {return fmt.Errorf("%s: expect[%d] is missing action", s.ID, i)@@ -151,16 +197,16 @@ func (s Scenario) Validate() error {"container has no logind session to attach to", s.ID)}switch s.Install.ExpectServiceKind {- case "", "systemd", "launchd", "none":+ case "", "systemd", "launchd", "windows-service", "none":default:- return fmt.Errorf("%s: install.expect_service_kind %q must be systemd, launchd or none",- s.ID, s.Install.ExpectServiceKind)+ return fmt.Errorf("%s: install.expect_service_kind %q must be systemd, launchd, "++ "windows-service or none", s.ID, s.Install.ExpectServiceKind)}switch s.Install.Service {- case "", "auto", "systemd", "launchd", "none":+ case "", "auto", "systemd", "launchd", "windows-service", "none":default:- return fmt.Errorf("%s: install.service %q must be auto, systemd, launchd, or none",- s.ID, s.Install.Service)+ return fmt.Errorf("%s: install.service %q must be auto, systemd, launchd, "++ "windows-service, or none", s.ID, s.Install.Service)}}@@ -210,6 +256,10 @@ func Load(path string) (Scenario, error) {}// LoadSuite reads every *.yaml in dir, sorted by filename so ordering is deterministic.+//+// Every scenario is validated, including ones this platform will not run: a syntax error or a+// stale field in a Windows scenario should fail a Linux contributor's `go test`, not wait for+// somebody to dispatch a Windows run.func LoadSuite(dir string) (Suite, error) {paths, err := filepath.Glob(filepath.Join(dir, "*.yaml"))if err != nil {@@ -229,6 +279,23 @@ func LoadSuite(dir string) (Suite, error) {return suite, nil}+// For returns the scenarios written for one platform.+//+// Reports how many were set aside rather than silently narrowing the suite. `run` with no+// --scenario is the "everything" command, and a caller who asked for everything and got two+// thirds of it is entitled to know -- silent truncation reads as full coverage.+func (s Suite) For(p Platform) (matching Suite, skipped int) {+ out := Suite{Name: s.Name}+ for _, sc := range s.Scenarios {+ if sc.TargetPlatform() == p {+ out.Scenarios = append(out.Scenarios, sc)+ continue+ }+ skipped+++ }+ return out, skipped+}+// Expand substitutes run-scoped placeholders into a template string.func Expand(tmpl string, canary, sentinel, workdir string) string {r := strings.NewReplacer(diff --git a/beacon-sandbox/scenario/scenario_test.go b/beacon-sandbox/scenario/scenario_test.goindex 641423b7..867c64fe 100644--- a/beacon-sandbox/scenario/scenario_test.go+++ b/beacon-sandbox/scenario/scenario_test.go@@ -348,6 +348,71 @@ func TestInstallModeAndServiceAreValidated(t *testing.T) {}}+// An unset platform must keep meaning Linux, or every scenario written before Windows existed+// would silently stop being selected by the suite loader.+func TestPlatformDefaultsToLinux(t *testing.T) {+ if got := (Scenario{ID: "s"}).TargetPlatform(); got != PlatformLinux {+ t.Errorf("an unset platform must default to linux, got %q", got)+ }+ if got := (Scenario{ID: "s", Platform: PlatformWindows}).TargetPlatform(); got != PlatformWindows {+ t.Errorf("an explicit platform must be honored, got %q", got)+ }+}++// A POSIX sentinel in a Windows scenario is the mistake hardest to notice at runtime: the agent+// writes somewhere, the probe looks somewhere else, and the run reports the agent idle rather+// than the scenario being wrong. So it is rejected at authoring time, before a runner is spent.+func TestWindowsScenariosRejectPosixSentinelsAndSystemd(t *testing.T) {+ base := func(mutate func(*Scenario)) Scenario {+ s := Scenario{ID: "w", Platform: PlatformWindows, Prompt: "do a thing",+ Expect: []Expect{{Action: "prompt.submitted", Why: "baseline"}}}+ mutate(&s)+ return s+ }+ if err := base(func(s *Scenario) { s.Sentinel = "/home/agent/work/out.txt" }).Validate(); err == nil {+ t.Error("a POSIX absolute sentinel must be rejected in a windows scenario")+ }+ if err := base(func(s *Scenario) {+ s.Install = &Install{Mode: "system", NeedsRealSystemd: true}+ }).Validate(); err == nil {+ t.Error("needs_real_systemd must be rejected in a windows scenario")+ }+ // The spellings a Windows scenario legitimately uses.+ for _, sentinel := range []string{`{{workdir}}\out.txt`, `C:\work\out.txt`} {+ if err := base(func(s *Scenario) { s.Sentinel = sentinel }).Validate(); err != nil {+ t.Errorf("sentinel %q must be accepted: %v", sentinel, err)+ }+ }+ // An unknown platform is a typo, and a typo that defaulted to linux would run a Windows+ // scenario's PowerShell prompt against bash.+ if err := (Scenario{ID: "w", Platform: "win32", Prompt: "x",+ Expect: []Expect{{Action: "a", Why: "b"}}}).Validate(); err == nil {+ t.Error("an unrecognized platform must be rejected rather than defaulting")+ }+}++// The suite loader must report what it set aside. `run` with no --scenario is the "everything"+// command, so returning a subset of it silently would read as full coverage.+func TestSuiteForReportsWhatItSkipped(t *testing.T) {+ suite := Suite{Scenarios: []Scenario{+ {ID: "s01"},+ {ID: "w00", Platform: PlatformWindows},+ {ID: "s02", Platform: PlatformLinux},+ }}++ linux, skipped := suite.For(PlatformLinux)+ if len(linux.Scenarios) != 2 || skipped != 1 {+ t.Errorf("linux: got %d scenarios and %d skipped, want 2 and 1", len(linux.Scenarios), skipped)+ }+ win, skipped := suite.For(PlatformWindows)+ if len(win.Scenarios) != 1 || skipped != 2 {+ t.Errorf("windows: got %d scenarios and %d skipped, want 1 and 2", len(win.Scenarios), skipped)+ }+ if win.Scenarios[0].ID != "w00" {+ t.Errorf("wrong scenario selected: %q", win.Scenarios[0].ID)+ }+}+// A systemd requirement combined with the supervised backend, or without system mode, would// silently exercise something other than what the scenario claims — and the run would pass,// reporting coverage it does not have.diff --git a/beacon-sandbox/scenarios/w00-probe.yaml b/beacon-sandbox/scenarios/w00-probe.yamlnew file mode 100644index 00000000..80b71157--- /dev/null+++ b/beacon-sandbox/scenarios/w00-probe.yaml@@ -0,0 +1,59 @@+id: w00-probe+platform: windows+# The first Windows scenario, and its job is to answer questions rather than to guard behavior.+#+# Beacon has no Windows support yet -- the CLI does not even compile for it at the time this was+# written -- so this scenario runs against the `beacon ci exec` collector path and asks the four+# things the rest of the Windows port needs settled before it can be designed. Every answer is+# read out of the collected artifacts and the run log:+#+# 1. Will Claude Code run `-p --dangerously-skip-permissions` as an *elevated* user? It refuses+# as root on POSIX, and a GitHub Actions Windows runner is an administrator with UAC+# disabled. If it refuses, meta.claude_stderr says so and the harness needs a standard+# account to run sessions as -- the Windows analogue of the `agent` user in the Linux image.+# 2. Does `beacon ci exec` supervise a PowerShell child correctly on Windows, and does the+# collector it starts write the runtime log? meta.ci_exec_rc and the collected runtime.jsonl+# answer that; internal/ci/session.go already carries Windows branches, so this is checking+# an existing claim rather than a new one.+# 3. Does the capture pipeline populate the same fields it does on Linux once the paths are+# Windows paths? The expectations below are the Linux ones, deliberately unchanged.+# 4. Does the argv credential sampler observe anything through Win32_Process? meta.argv_samples+# and meta.argv_saw_agent report it, and a clean-but-never-saw-the-agent result is recorded+# as unverified rather than clean.+#+# The command is deliberately PowerShell rather than bash: the agent's default shell on Windows+# is PowerShell, and asking it for a bash command would test Git-for-Windows' presence instead+# of Beacon.+prompt: >-+ Run exactly this PowerShell command and then report its output verbatim:+ Write-Output '{{canary}}' | Tee-Object -FilePath '{{sentinel}}'+sentinel: '{{workdir}}\w00.sentinel'+budget_usd: 1.0+timeout_seconds: 420++expect:+ - action: prompt.submitted+ contains: ["{{canary}}"]+ why: >-+ Baseline. If this is absent nothing else below is interpretable, because the session either+ never reached the model or the collector never received anything -- which is question 2, not+ a capture gap.++ - action: command.executed+ contains: ["{{canary}}"]+ fields: ["command.command", "tool.name"]+ optional: true+ why: >-+ Threat rules match scalar leaves, so e.command.command is the detection surface for the+ whole rules/risky-command/ category on Windows exactly as it is on Linux. Optional because+ this probe is establishing whether it works at all -- a gap here is a finding to act on, not+ a regression to block on, and marking it required would stop the probe reporting the other+ three answers.++ - action: approval.allowed+ fields: ["approval.decision"]+ optional: true+ why: >-+ Claude Code's tool_decision carries decision=accept regardless of platform. Optional for the+ same reason as above: this scenario exists to find out, and the answer belongs in the+ verdict either way.