Skip to content

The cross-harness, self-improving memory layer for AI agents. Join our community: https://discord.com/invite/zdNChS2fBu

feat(sandbox): run scenarios on a Windows guest via a disposable CI runner

shukieshaha1bf439c6081
diff --git a/.github/workflows/windows-sandbox.yml b/.github/workflows/windows-sandbox.yml
new file mode 100644
index 00000000..7cd8631a
--- /dev/null
+++ b/.github/workflows/windows-sandbox.yml
@@ -0,0 +1,186 @@
+# Runs a beacon-sandbox scenario against a real Claude Code session on Windows.
+#
+# Why a workflow rather than a rented instance: Modal offers Linux only, and a GitHub-hosted
+# Windows runner is already a disposable machine with a real Service Control Manager,
+# administrator rights and UAC disabled -- everything an endpoint install needs. So the runner
+# *is* the sandbox, and `beacon-sandbox run --provider local` drives it from the inside. The
+# collected artifacts are uploaded and judged on the maintainer's machine by the same pure
+# `check` package that judges a Modal run, so a Windows verdict is produced by the same rules.
+#
+# workflow_dispatch only, and deliberately. The job needs ANTHROPIC_API_KEY to run a real agent
+# session with --dangerously-skip-permissions; on a public repository that secret must never be
+# reachable from untrusted code, so there is no pull_request trigger here and there must never be
+# a pull_request_target one.
+name: Windows sandbox
+
+on:
+ workflow_dispatch:
+ inputs:
+ scenario:
+ description: "Scenario id to run (empty runs every windows scenario)"
+ required: false
+ type: string
+ claude_version:
+ description: "Claude Code version to pin"
+ required: false
+ default: "2.1.220"
+ type: string
+
+permissions:
+ contents: read
+
+concurrency:
+ # One at a time: concurrent runs would race on the runner-wide agent install and on the
+ # system-mode endpoint state, and a scenario that failed for that reason looks like a Beacon bug.
+ group: windows-sandbox
+ cancel-in-progress: false
+
+jobs:
+ sandbox:
+ runs-on: windows-2025
+ # The credential lives in an environment rather than at the repository level, matching how the
+ # release secrets are scoped.
+ environment: windows-sandbox
+ timeout-minutes: 45
+ defaults:
+ run:
+ shell: pwsh
+ steps:
+ - uses: actions/checkout@v4
+
+ - uses: actions/setup-go@v5
+ with:
+ go-version-file: cli/beacon/go.mod
+
+ # Runs the generated-PowerShell parse test, which is skipped on a maintainer's machine
+ # without pwsh. The generated scripts are assembled in Go, so a syntax error in one is
+ # invisible to the compiler and would surface as "the check did not run" -- which a verdict
+ # reports as unverified rather than failing loudly. This is the gate that stops that.
+ - name: Test the harness itself
+ working-directory: beacon-sandbox
+ run: go test ./...
+
+ # Beacon does not compile for Windows yet: internal/endpoint/writer, inventory, ingest and
+ # service/supervised all use POSIX-only syscalls. Until that is fixed the build is expected
+ # to fail, and the run continues so the probe can still report everything that does not
+ # depend on a Beacon binary. Once the portability layer lands, remove continue-on-error --
+ # a silently missing binary must not read as a passing run.
+ # The Makefile's build-windows-amd64 target is not used: GNU make is not guaranteed on the
+ # Windows image, and the two steps it performs are clearer spelled out than shimmed. The
+ # ordering is the part that matters -- the hooks binary is embedded into the CLI, so it has
+ # to exist before the CLI is built or the CLI embeds the placeholder.
+ - name: Build beacon and beacon-hooks for Windows
+ id: build
+ continue-on-error: true
+ run: |
+ Set-Location cli\beacon-hooks
+ go build -ldflags "-s -w" -o beacon-hooks.exe .
+ if ($LASTEXITCODE -ne 0) { throw "beacon-hooks does not build for windows/amd64 yet" }
+ Copy-Item beacon-hooks.exe ..\beacon\internal\embedded\hooks.bin -Force
+ Set-Location ..\beacon
+ go build -o beacon-windows-amd64.exe .
+ if ($LASTEXITCODE -ne 0) { throw "beacon does not build for windows/amd64 yet" }
+
+ - name: Build the collector for Windows
+ id: collector
+ continue-on-error: true
+ run: |
+ go install go.opentelemetry.io/collector/cmd/[email protected]
+ Set-Location collector-builder
+ # builder mkdirs its output path without -p, so the parent has to exist. A fresh
+ # checkout has no dist/ at all, which is why this only ever failed in CI.
+ New-Item -ItemType Directory -Force -Path dist | Out-Null
+ $env:GOOS = 'windows'
+ $env:GOARCH = 'amd64'
+ $env:CGO_ENABLED = '0'
+ & "$(go env GOPATH)\bin\builder.exe" --config builder.yaml
+ if ($LASTEXITCODE -ne 0) { throw "collector build failed" }
+ New-Item -ItemType Directory -Force -Path dist\beacon-otelcol\windows_amd64 | Out-Null
+ Copy-Item dist\beacon-otelcol\beacon-otelcol.exe dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe
+
+ # The layout beacon-sandbox's WindowsLayout() describes. Created here rather than by the
+ # harness because on this platform the CI job is what provisions the guest.
+ - name: Stage the guest layout
+ run: |
+ New-Item -ItemType Directory -Force -Path C:\beacon-sandbox\bin, C:\beacon-sandbox\work | Out-Null
+ if (Test-Path cli\beacon\beacon-windows-amd64.exe) {
+ Copy-Item cli\beacon\beacon-windows-amd64.exe C:\beacon-sandbox\bin\beacon.exe
+ }
+ if (Test-Path collector-builder\dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe) {
+ Copy-Item collector-builder\dist\beacon-otelcol\windows_amd64\beacon-otelcol.exe C:\beacon-sandbox\bin\beacon-otelcol.exe
+ }
+ "C:\beacon-sandbox\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
+ # A seeded git repo gives scenarios somewhere realistic to work and exercises Beacon's
+ # repository/branch enrichment, mirroring the Linux image's fixture.
+ Set-Location C:\beacon-sandbox\work
+ New-Item -ItemType Directory -Force -Path repo | Out-Null
+ Set-Location repo
+ git init -q
+ git config user.email [email protected]
+ git config user.name beacon-sandbox
+ "# fixture" | Out-File -FilePath README.md -Encoding utf8
+ git add -A
+ git commit -qm initial
+
+ # The native installer, which is the only Windows install path that self-updates. Pinned so
+ # a verdict is attributable to a specific agent build.
+ - name: Install Claude Code
+ run: |
+ $version = '${{ inputs.claude_version }}'
+ if ([string]::IsNullOrWhiteSpace($version)) { $version = '2.1.220' }
+ # Whitelist rather than escape: the value reaches a shell, and Claude Code versions are
+ # dotted numerals, so anything else is a mistake worth rejecting outright.
+ if ($version -notmatch '^[0-9]+(\.[0-9]+)*$') {
+ throw "invalid claude_version '$version': expected a dotted numeric version"
+ }
+ $installer = Join-Path $env:TEMP 'install-claude.ps1'
+ Invoke-WebRequest -Uri 'https://claude.ai/install.ps1' -OutFile $installer
+ & $installer $version
+ "$env:LOCALAPPDATA\Programs\claude" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
+
+ - name: Report the environment the probe ran in
+ run: |
+ Write-Output "pwsh: $($PSVersionTable.PSVersion)"
+ Write-Output "identity: $([System.Security.Principal.WindowsIdentity]::GetCurrent().Name)"
+ $principal = New-Object Security.Principal.WindowsPrincipal([System.Security.Principal.WindowsIdentity]::GetCurrent())
+ Write-Output "elevated: $($principal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator))"
+ Write-Output "beacon build: ${{ steps.build.outcome }}"
+ Write-Output "collector build: ${{ steps.collector.outcome }}"
+ & claude --version
+ if (Test-Path C:\beacon-sandbox\bin\beacon.exe) { & C:\beacon-sandbox\bin\beacon.exe version }
+
+ - name: Run the scenario
+ id: run
+ continue-on-error: true
+ working-directory: beacon-sandbox
+ env:
+ ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+ run: |
+ # Not $args: that name is a PowerShell automatic variable holding the script's own
+ # arguments, and appending to it does not behave the way it reads.
+ $sandboxArgs = @('run', '--provider', 'local')
+ if (-not [string]::IsNullOrWhiteSpace('${{ inputs.scenario }}')) {
+ $sandboxArgs += @('--scenario', '${{ inputs.scenario }}')
+ }
+ go run ./cmd/beacon-sandbox @sandboxArgs
+
+ # Always uploaded. A failed run's artifacts are exactly what a maintainer needs, and the
+ # host-state and argv metadata are recorded even when collection itself failed.
+ - name: Upload collected artifacts
+ if: always()
+ uses: actions/upload-artifact@v4
+ with:
+ name: beacon-sandbox-runs
+ path: beacon-sandbox/runs/
+ if-no-files-found: warn
+ retention-days: 14
+
+ # The scenario's own exit status decides the job, after the artifacts are safely uploaded.
+ # Without this the job would go green on a failing scenario because of continue-on-error.
+ - name: Reflect the scenario outcome
+ if: always()
+ run: |
+ Write-Output "scenario outcome: ${{ steps.run.outcome }}"
+ if ('${{ steps.run.outcome }}' -ne 'success') {
+ throw "the scenario did not pass; read the uploaded runs/ artifacts and the verdict.json inside them"
+ }
diff --git a/beacon-sandbox/check/invariants.go b/beacon-sandbox/check/invariants.go
index ea850c31..ffd06e3b 100644
--- a/beacon-sandbox/check/invariants.go
+++ b/beacon-sandbox/check/invariants.go
@@ -266,6 +266,9 @@ type HostSafety struct {
SecretInArgv bool
// ArgvCheckRan is false when the scan could not run, so its result proves nothing.
ArgvCheckRan bool
+ // Disposability is the evidence a local-guest run offers in place of a comparison. Only
+ // meaningful when HostChanged is hostguard.EphemeralDescription.
+ Disposability string
}
// Safety adds findings for the host-isolation and credential-handling guarantees.
@@ -292,6 +295,26 @@ func Safety(v *Verdict, s HostSafety) {
Summary: "files unchanged, but the service probe could not run, so services are unverified",
Why: "treat the service half as unproven; the file half of the guard did pass",
})
+ case s.HostChanged == hostguard.EphemeralDescription:
+ // The guest was this machine, so there was no comparison to make: the install the
+ // scenario is testing *is* the change a comparison would report. Isolation therefore
+ // rests on the machine being disposable, which is a real but weaker guarantee -- so it
+ // is reported rather than passed over in silence, and the evidence is named so a reader
+ // can tell "GitHub hands out a fresh VM per job" from "the operator said so".
+ //
+ // Warn, not fail: a dispatched CI run is a legitimate and intended configuration. Warn,
+ // not silence: the file-and-service comparison every other run gets did not happen here,
+ // and silence in this tool means verified-clean.
+ why := "the comparison every other run gets did not happen; isolation is only as good as " +
+ "the machine being disposable"
+ if s.Disposability != "" {
+ why += " (" + s.Disposability + ")"
+ }
+ v.Add(Finding{
+ Check: "safety.host_untouched", Severity: SevWarn,
+ Summary: "host isolation was not verified by comparison because the guest was this machine",
+ Why: why,
+ })
case s.HostChanged != hostguard.CleanDescription:
v.Add(Finding{
Check: "safety.host_untouched", Severity: SevFail,
diff --git a/beacon-sandbox/cmd/beacon-sandbox/main.go b/beacon-sandbox/cmd/beacon-sandbox/main.go
index c1babdb8..54288b5f 100644
--- a/beacon-sandbox/cmd/beacon-sandbox/main.go
+++ b/beacon-sandbox/cmd/beacon-sandbox/main.go
@@ -72,11 +72,16 @@ func usage() {
fmt.Fprint(os.Stderr, `beacon-sandbox -- verify Beacon against real agent activity in a disposable sandbox
doctor [--fix] [--json] check prerequisites and say how to fix what is missing
- run [--scenario ID] [--repeat N] [--keep-sandbox] [--suite DIR]
+ run [--scenario ID] [--repeat N] [--keep-sandbox] [--suite DIR] [--provider NAME]
verify <run-dir>... [--mutate MODE]
diff <before-dir> <after-dir>
clean [--dir runs]
+Where scenarios run (--provider):
+ modal a disposable Linux sandbox. The default, and Linux-only
+ github dispatch to a GitHub Actions Windows runner, then collect and judge locally
+ local this machine, for use inside a disposable CI runner
+
Anthropic credential, in the order they resolve:
--modal-secret NAME a secret already stored with the provider; the value never enters
this process, so the artifact leak check reports unverified
@@ -120,6 +125,8 @@ func cmdRun(args []string) error {
claudeVer := fs.String("claude-version", "", "pin Claude Code version (default: image default)")
modalSecret := fs.String("modal-secret", "", "name of a Modal secret holding ANTHROPIC_API_KEY; the value never enters this process")
keyCommand := fs.String("api-key-command", "", "command whose stdout is the Anthropic API key (e.g. 'op read op://vault/anthropic/key')")
+ provider := fs.String("provider", "modal", "where scenarios run: modal (disposable Linux), github (dispatch to a Windows runner), or local (this machine)")
+ allowHostMutation := fs.Bool("allow-host-mutation", false, "with --provider local, run on a machine this tool cannot prove is disposable")
fs.Parse(args)
root, err := repoRoot()
@@ -156,17 +163,32 @@ func cmdRun(args []string) error {
}
ctx := context.Background()
- prov, err := sandbox.NewModal(ctx, appName)
+ prov, closeProv, err := openProvider(ctx, *provider, *allowHostMutation)
if err != nil {
return err
}
- defer prov.Close()
+ defer closeProv()
+
+ // Scenarios are not portable, so run only the ones written for this guest -- and say how many
+ // were set aside. `run` with no --scenario is the "everything" command, and silently returning
+ // a subset of it would read as full coverage.
+ target := platformOf(prov)
+ sui, skipped := sui.For(target)
+ if skipped > 0 {
+ fmt.Printf("suite: %d %s scenario(s); %d written for another platform and not run\n",
+ len(sui.Scenarios), target, skipped)
+ }
+ if len(sui.Scenarios) == 0 {
+ return fmt.Errorf("no %s scenarios in %s, so this run would assert nothing", target, scDir)
+ }
var verdicts []check.Verdict
+ matched := false
for _, sc := range sui.Scenarios {
if *only != "" && sc.ID != *only {
continue
}
+ matched = true
for i := 0; i < *repeat; i++ {
label := sc.ID
if *repeat > 1 {
@@ -200,9 +222,48 @@ func cmdRun(args []string) error {
verdicts = append(verdicts, v)
}
}
+ // A --scenario that matched nothing must not exit 0 with an empty summary: the caller asked
+ // for a specific check and got none, which is a failure to run rather than a clean result.
+ // A typo'd id, or one belonging to the other platform, both land here.
+ if *only != "" && !matched {
+ return fmt.Errorf("no %s scenario with id %q in %s; `--provider %s` runs %s scenarios",
+ target, *only, scDir, *provider, target)
+ }
return summarize(verdicts)
}
+// openProvider resolves the --provider choice, returning a cleanup function so a Modal client is
+// always closed.
+func openProvider(ctx context.Context, name string, allowHostMutation bool) (sandbox.Provider, func(), error) {
+ switch name {
+ case "modal":
+ p, err := sandbox.NewModal(ctx, appName)
+ if err != nil {
+ return nil, func() {}, err
+ }
+ return p, p.Close, nil
+ case "local":
+ p, err := sandbox.NewLocalExec(sandbox.LocalExecOptions{AllowHostMutation: allowHostMutation})
+ if err != nil {
+ return nil, func() {}, err
+ }
+ return p, func() {}, nil
+ case "github":
+ return nil, func() {}, fmt.Errorf("--provider github dispatches a workflow rather than " +
+ "driving a machine directly; it is not wired up yet")
+ default:
+ return nil, func() {}, fmt.Errorf("unknown --provider %q; want modal, github, or local", name)
+ }
+}
+
+// platformOf maps a provider's guest platform onto the scenario vocabulary.
+func platformOf(p sandbox.Provider) scenario.Platform {
+ if p.Platform().IsWindows() {
+ return scenario.PlatformWindows
+ }
+ return scenario.PlatformLinux
+}
+
// judge is the pure step: artifacts in, verdict out. Returns the parsed log too so callers
// can build a fingerprint without re-reading it.
func judge(sc scenario.Scenario, art runner.Artifacts, creds credentials.Resolved, planted string) (check.Verdict, check.Log) {
@@ -257,9 +318,10 @@ func judge(sc scenario.Scenario, art runner.Artifacts, creds credentials.Resolve
// credential disclosure are the two findings that must survive every other kind of failure.
func safetyOf(v *check.Verdict, art runner.Artifacts) {
check.Safety(v, check.HostSafety{
- HostChanged: art.Meta["host_state"],
- SecretInArgv: art.Meta["secret_in_argv"] == "true",
- ArgvCheckRan: art.Meta["argv_check_ran"] == "true",
+ HostChanged: art.Meta["host_state"],
+ SecretInArgv: art.Meta["secret_in_argv"] == "true",
+ ArgvCheckRan: art.Meta["argv_check_ran"] == "true",
+ Disposability: art.Meta["disposability"],
})
}
diff --git a/beacon-sandbox/hostguard/hostguard.go b/beacon-sandbox/hostguard/hostguard.go
index aee4f895..3b8767c8 100644
--- a/beacon-sandbox/hostguard/hostguard.go
+++ b/beacon-sandbox/hostguard/hostguard.go
@@ -97,10 +97,37 @@ func guardedPaths() []guarded {
guarded{filepath.Join(h, ".config", "systemd", "user"), watchBeaconEntries},
guarded{"/etc/systemd/system", watchBeaconEntries},
)
+ case "windows":
+ // For a contributor pointing the harness at a Windows VM they own. The dispatched CI
+ // path does not compare snapshots at all -- see EphemeralDescription -- but a guard that
+ // silently watched nothing on Windows would be indistinguishable from one that watched
+ // and found nothing, which is the conflation this package exists to avoid.
+ out = append(out,
+ guarded{filepath.Join(programData(), "Beacon", "Endpoint", "config.json"), watchFull},
+ guarded{filepath.Join(programData(), "Beacon", "Endpoint", "logs"), watchExistence},
+ guarded{filepath.Join(programFiles(), "Beacon"), watchExistence},
+ )
}
return out
}
+// programData and programFiles resolve the Windows system directories, falling back to the
+// conventional locations. Read from the environment because a redirected or localized install
+// puts them elsewhere, and a guard watching the wrong path reports clean forever.
+func programData() string {
+ if v := os.Getenv("ProgramData"); v != "" {
+ return v
+ }
+ return `C:\ProgramData`
+}
+
+func programFiles() string {
+ if v := os.Getenv("ProgramFiles"); v != "" {
+ return v
+ }
+ return `C:\Program Files`
+}
+
// GuardedPaths returns the watched paths, for reporting and tests.
func GuardedPaths() []string {
g := guardedPaths()
@@ -204,6 +231,23 @@ func beaconServices() (services []string, available bool) {
out = append(out, fields[0])
}
}
+ case "windows":
+ // `sc.exe query type= service state= all` lists every service; the display name lines are
+ // what carry "beacon". Parsed rather than piped through PowerShell so the probe does not
+ // depend on which shell is installed.
+ b, err := exec.Command("sc.exe", "query", "type=", "service", "state=", "all").Output()
+ if err != nil {
+ return nil, false
+ }
+ for _, line := range strings.Split(string(b), "\n") {
+ name, ok := strings.CutPrefix(strings.TrimSpace(line), "SERVICE_NAME:")
+ if !ok {
+ continue
+ }
+ if name = strings.TrimSpace(name); strings.Contains(strings.ToLower(name), "beacon") {
+ out = append(out, name)
+ }
+ }
}
sort.Strings(out)
return out, true
@@ -236,6 +280,17 @@ const CleanDescription = "host state unchanged"
// layer must be able to tell them apart without string-sniffing.
const PartialDescription = "files unchanged; service probe unavailable, so services are unverified"
+// EphemeralDescription is recorded when the guest *is* this machine, so comparing before and
+// after cannot mean anything.
+//
+// A backend like that runs the scenario where the harness stands, which makes the install the
+// scenario is testing indistinguishable from the escape this guard looks for -- a correct run
+// would report the most serious finding the tool has. So isolation is claimed on a different
+// basis: the machine is disposable and somebody else destroys it. That is a genuinely weaker
+// claim than "nothing on the developer's machine moved", and it gets its own value rather than
+// borrowing CleanDescription precisely so a reader of a verdict can tell which one they got.
+const EphemeralDescription = "guest is this machine; isolation rests on it being disposable rather than on a comparison"
+
func (d Diff) Describe() string {
if d.Clean() {
if d.ServicesUnverified {
diff --git a/beacon-sandbox/image/image.go b/beacon-sandbox/image/image.go
index 3218bbaa..7ff9751f 100644
--- a/beacon-sandbox/image/image.go
+++ b/beacon-sandbox/image/image.go
@@ -9,6 +9,7 @@ package image
import (
"fmt"
"os"
+ "path/filepath"
"strings"
"github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"
@@ -31,6 +32,137 @@ const (
DefaultBase = "ubuntu:24.04"
)
+// Layout is where things live inside the guest.
+//
+// Split out from the package constants because those describe one environment -- the Linux
+// image this package builds -- and Windows runs somewhere this package does not provision at
+// all. The CI job places the binaries and creates the work directory there, so the harness
+// needs to *describe* the layout without owning it.
+type Layout struct {
+ Platform sandbox.Platform
+ // AgentUser is the account the session runs as. Empty means "whoever is already running",
+ // which is the Windows case: there is no way to become another user without a password.
+ AgentUser string
+ AgentHome string
+ WorkDir string
+ BeaconDir string
+ // PathPrepend is prepended to PATH so both the agent and Beacon are reachable.
+ PathPrepend []string
+}
+
+// LinuxLayout is the layout of the image this package builds.
+func LinuxLayout() Layout {
+ return Layout{
+ Platform: sandbox.PlatformPosix,
+ AgentUser: AgentUser,
+ AgentHome: AgentHome,
+ WorkDir: WorkDir,
+ BeaconDir: BeaconDir,
+ PathPrepend: PathPrepend(),
+ }
+}
+
+// WindowsLayout is the layout the windows-sandbox CI job provisions.
+//
+// No AgentUser: the runner is already an administrator with UAC disabled, and this backend
+// cannot switch users anyway. Whether that is acceptable to the agent is an open question the
+// w00-probe scenario answers -- Claude Code refuses --dangerously-skip-permissions as root on
+// POSIX, and if it refuses for an elevated Windows user too, the CI job will have to create a
+// standard account and run the whole harness as them.
+func WindowsLayout() Layout {
+ home := windowsHome()
+ beaconDir := `C:\beacon-sandbox\bin`
+ return Layout{
+ Platform: sandbox.PlatformWindows,
+ AgentHome: home,
+ WorkDir: `C:\beacon-sandbox\work`,
+ BeaconDir: beaconDir,
+ // The agent installs to %LOCALAPPDATA%\Programs, which is where install.ps1 puts it.
+ PathPrepend: []string{beaconDir, filepath.Join(home, "AppData", "Local", "Programs", "claude")},
+ }
+}
+
+// LayoutFor selects the layout for a platform.
+func LayoutFor(p sandbox.Platform) Layout {
+ if p.IsWindows() {
+ return WindowsLayout()
+ }
+ return LinuxLayout()
+}
+
+// PrivilegedUser names the account a privileged step runs as, or empty when this platform has
+// no account to switch to.
+//
+// Windows is the empty case, and that is not a gap: the runner is already an administrator with
+// UAC disabled, so a system-mode install has the rights it needs without becoming anyone. There
+// is also no way to become another user without a password, so asking would fail rather than
+// escalate.
+func (l Layout) PrivilegedUser() string {
+ if l.Platform.IsWindows() {
+ return ""
+ }
+ return "root"
+}
+
+// DefaultLogPath is where Beacon writes the runtime log when `endpoint status --json` could not
+// be parsed.
+//
+// A fallback only. The install scenarios read the real path out of status, because a scenario
+// that hardcodes the expected path agrees with itself rather than with Beacon -- which is the
+// one thing an install scenario exists to check.
+func (l Layout) DefaultLogPath(system bool) string {
+ if l.Platform.IsWindows() {
+ if system {
+ programData := os.Getenv("ProgramData")
+ if programData == "" {
+ programData = `C:\ProgramData`
+ }
+ return filepath.Join(programData, "Beacon", "Endpoint", "logs", "runtime.jsonl")
+ }
+ return filepath.Join(l.AgentHome, ".beacon", "endpoint", "logs", "runtime.jsonl")
+ }
+ if system {
+ return "/var/log/beacon-agent/runtime.jsonl"
+ }
+ return l.AgentHome + "/.beacon/endpoint/logs/runtime.jsonl"
+}
+
+// ConsoleUser names the human whose agent runtime a system-mode install must configure.
+//
+// A system endpoint runs elevated, and `endpoint install` configures the *installing* user's
+// settings -- which for a package install is not the person at the keyboard. Beacon resolves
+// that person from SUDO_USER and then logind; a system-mode scenario has to name them or its
+// capture assertions would be testing an unused settings file. On Windows there is one account,
+// so it is that one.
+func (l Layout) ConsoleUser() string {
+ if l.AgentUser != "" {
+ return l.AgentUser
+ }
+ if v := os.Getenv("USERNAME"); v != "" {
+ return v
+ }
+ if v := os.Getenv("USER"); v != "" {
+ return v
+ }
+ return filepath.Base(l.AgentHome)
+}
+
+// windowsHome resolves the guest home directory.
+//
+// USERPROFILE is the value Windows programs actually read. Falling back to a literal rather
+// than to an error keeps a dispatched run from failing on a runner that has not exported it;
+// the layout is only ever used to build guest paths, so a wrong guess surfaces immediately as
+// a missing file rather than silently.
+func windowsHome() string {
+ if v := os.Getenv("USERPROFILE"); v != "" {
+ return v
+ }
+ if v := os.Getenv("HOME"); v != "" {
+ return v
+ }
+ return `C:\Users\runneradmin`
+}
+
// Spec parameterizes the environment.
type Spec struct {
Base string
diff --git a/beacon-sandbox/runner/runner.go b/beacon-sandbox/runner/runner.go
index bfe629fd..61199c24 100644
--- a/beacon-sandbox/runner/runner.go
+++ b/beacon-sandbox/runner/runner.go
@@ -80,6 +80,11 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
return Artifacts{}, fmt.Errorf("no Anthropic credential resolved; run `beacon-sandbox doctor`")
}
+ // The dialect and the guest layout both follow from the provider's platform, never from the
+ // host's runtime.GOOS -- a Mac dispatching a Windows run is the normal case, not the exception.
+ sh := shellFor(p.Platform())
+ lay := image.LayoutFor(p.Platform())
+
canary := "BEACON_SANDBOX_" + randomToken()
runDir := filepath.Join(opts.OutDir, fmt.Sprintf("%s-%s", sc.ID, canary[len(canary)-8:]))
// 0700, not 0755: these artifacts retain prompt text and command output, and a
@@ -94,34 +99,49 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
Meta: map[string]string{
"scenario": sc.ID,
"provider": p.Name(),
+ "platform": string(p.Platform()),
},
}
- // Fingerprint the developer's own Beacon state before anything runs, so a stray local
- // install would be caught rather than assumed impossible.
+ // Establish how isolation is proven for this run, and record it either way.
+ //
+ // The usual proof is a comparison: fingerprint the developer's own Beacon state before and
+ // after, so a stray local install is caught rather than assumed impossible. That only means
+ // something when the guest is a *different* machine. A backend that runs the scenario where
+ // this process stands inverts it -- the install the scenario is testing is itself the change
+ // the guard would report -- so for those the claim has to rest on the machine being
+ // disposable instead, and say which of the two it relied on.
//
- // The comparison and the meta write happen together in this defer, and deliberately so.
- // An earlier version wrote meta.json inline before returning, which meant host_state --
- // set here, after that write -- never reached disk. The live verdict was still correct
- // because judging happens after Run returns, but an offline `verify` read the field as
- // empty and treated the host check as clean. An unrunnable check that reads as a pass is
- // the exact failure this tool exists to prevent. Writing here also means an early error
- // return still leaves the metadata behind for debugging.
- hostBefore := hostguard.Take()
+ // The comparison and the meta write happen together in this defer, and deliberately so. An
+ // earlier version wrote meta.json inline before returning, which meant host_state -- set
+ // here, after that write -- never reached disk. The live verdict was still correct because
+ // judging happens after Run returns, but an offline `verify` read the field as empty and
+ // treated the host check as clean. An unrunnable check that reads as a pass is the exact
+ // failure this tool exists to prevent. Writing here also means an early error return still
+ // leaves the metadata behind for debugging.
+ local := p.MutatesHost()
+ var hostBefore hostguard.Snapshot
+ if !local {
+ hostBefore = hostguard.Take()
+ }
defer func() {
- art.HostDiff = hostguard.Compare(hostBefore, hostguard.Take())
- art.Meta["host_state"] = art.HostDiff.Describe()
+ if local {
+ art.Meta["host_state"] = hostguard.EphemeralDescription
+ art.Meta["disposability"] = sandbox.DisposabilityEvidence()
+ } else {
+ art.HostDiff = hostguard.Compare(hostBefore, hostguard.Take())
+ art.Meta["host_state"] = art.HostDiff.Describe()
+ }
if b, err := json.MarshalIndent(art.Meta, "", " "); err == nil {
_ = os.WriteFile(filepath.Join(runDir, "meta.json"), append(b, '\n'), 0o600)
}
}()
+ if local {
+ logf("guest is this machine (%s); isolation rests on it being disposable: %s",
+ p.Name(), sandbox.DisposabilityEvidence())
+ }
- logf("building image (cached layers are free)")
- imgSpec, err := image.Build(image.Spec{
- RepoRoot: opts.RepoRoot,
- ClaudeVersion: opts.ClaudeVersion,
- WithDocker: sc.NeedsRealSystemd(),
- }, logf)
+ imgSpec, err := imageSpecFor(p.Platform(), sc, opts, logf)
if err != nil {
return art, err
}
@@ -136,7 +156,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
CPU: 2,
MemLimitMiB: 4096,
Timeout: sc.Timeout() + 5*time.Minute, // headroom for setup and collection
- Workdir: image.WorkDir,
+ Workdir: lay.WorkDir,
Secrets: inlineSecrets(opts.Creds),
SecretNames: providerSecrets(opts.Creds),
BlockEgress: sc.BlockEgress,
@@ -172,15 +192,27 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
}
agent := sandbox.ExecOpts{
- User: image.AgentUser, Dir: image.WorkDir, HomeDir: image.AgentHome,
- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: 5 * time.Minute,
- }
-
- // Finish the artifact layer: the snapshot captured the files, but chmod and symlinks
- // have to happen in a live instance. Always the sandbox itself, never the guest -- this
- // prepares the very tree the guest will mount.
- if _, err := p.Exec(ctx, inst, image.PostPushLayers(), sandbox.ExecOpts{User: "root", Timeout: time.Minute}); err != nil {
- return art, fmt.Errorf("prepare binaries: %w", err)
+ User: lay.AgentUser, Dir: lay.WorkDir, HomeDir: lay.AgentHome,
+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: 5 * time.Minute,
+ }
+ // privileged is the same options with whatever elevation this platform expresses. On POSIX
+ // that is the root account; on Windows there is no account to switch to -- the runner is
+ // already an administrator with UAC disabled, and the local backend cannot become anyone
+ // else. Naming the difference once keeps every later call site single-path.
+ privileged := sandbox.ExecOpts{
+ User: lay.PrivilegedUser(), Dir: lay.WorkDir, HomeDir: lay.AgentHome,
+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: 5 * time.Minute,
+ }
+
+ // Finish the artifact layer: the snapshot captured the files, but chmod and symlinks have to
+ // happen in a live instance. Skipped when nothing was pushed, which is the local backend's
+ // case -- the CI job placed the binaries before this process started, so there is no staged
+ // tree to fix up.
+ if len(imgSpec.Files) > 0 {
+ if _, err := p.Exec(ctx, inst, image.PostPushLayers(),
+ sandbox.ExecOpts{User: lay.PrivilegedUser(), Timeout: time.Minute}); err != nil {
+ return art, fmt.Errorf("prepare binaries: %w", err)
+ }
}
// Everything the scenario itself does happens in the guest. For most scenarios that is the
@@ -200,7 +232,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// Not just recorded -- checked. An empty version line means beacon or claude is not reachable
// in the guest, and every later failure is then a symptom of that rather than of Beacon. The
// first nested run printed "versions:" with nothing after it and kept going.
- v, err := g.Exec(ctx, "beacon version; claude --version; id -un; uname -m", agent)
+ v, err := g.Exec(ctx, versionProbe(p.Platform()), agent)
if err != nil {
return art, fmt.Errorf("could not run commands in the %s guest: %w", g.Describe(), err)
}
@@ -211,21 +243,21 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
"test is not reachable there (rc=%d): %s", v.ExitCode, oneLine(v.Stdout+v.Stderr))
}
- sentinel := scenario.Expand(sc.Sentinel, canary, "", image.WorkDir)
+ sentinel := scenario.Expand(sc.Sentinel, canary, "", lay.WorkDir)
if sc.Setup != "" {
- setup := scenario.Expand(sc.Setup, canary, sentinel, image.WorkDir)
+ setup := scenario.Expand(sc.Setup, canary, sentinel, lay.WorkDir)
if r, err := g.Exec(ctx, setup, agent); err != nil || r.ExitCode != 0 {
return art, fmt.Errorf("scenario setup failed (rc=%d): %v %s", r.ExitCode, err, oneLine(r.Stderr))
}
}
- logPath := filepath.ToSlash(filepath.Join(image.WorkDir, "runtime.jsonl"))
- prompt := scenario.Expand(sc.Prompt, canary, sentinel, image.WorkDir)
+ logPath := sh.Join(lay.WorkDir, "runtime.jsonl")
+ prompt := scenario.Expand(sc.Prompt, canary, sentinel, lay.WorkDir)
// Start the credential sampler BEFORE the session, so it observes the agent's own
- // processes while they are alive. See argvSamplerScript.
- if _, err := p.Exec(ctx, inst, argvSamplerScript(), sandbox.ExecOpts{
- User: "root", Timeout: time.Minute,
+ // processes while they are alive. See shell.ArgvSampler.
+ if _, err := p.Exec(ctx, inst, sh.ArgvSampler(), sandbox.ExecOpts{
+ User: lay.PrivilegedUser(), Timeout: time.Minute,
}); err != nil {
logf("argv sampler failed to start: %v", err)
}
@@ -236,19 +268,21 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// manager. A persistent install instead exercises `beacon endpoint install`: service
// manager selection, unit installation, status reporting, and whether an installed
// collector actually receives telemetry. Nothing else in the suite can tell you that.
+ sessionDir := workDirFor(sc, lay, sh)
+ sessionOpts := sandbox.ExecOpts{
+ User: lay.AgentUser, Dir: sessionDir, HomeDir: lay.AgentHome,
+ PreserveEnv: true, PathPrepend: lay.PathPrepend, Timeout: sc.Timeout() + time.Minute,
+ }
var res sandbox.Result
if sc.Install != nil {
- installedLog, err := doInstall(ctx, g, sc, agent, &art, logf)
+ installedLog, err := doInstall(ctx, g, sc, privileged, agent, lay, sh, &art, logf)
if err != nil {
return art, err
}
logPath = installedLog
logf("running session against the installed endpoint (budget $%.2f, timeout %s)",
sc.Budget(), sc.Timeout())
- res, err = g.Exec(ctx, plainSessionScript(sc, prompt), sandbox.ExecOpts{
- User: image.AgentUser, Dir: workDirFor(sc), HomeDir: image.AgentHome,
- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: sc.Timeout() + time.Minute,
- })
+ res, err = g.Exec(ctx, sh.PlainSession(sessionDir, claudeFlags(sc, prompt, sh)), sessionOpts)
if err != nil {
logf("session exec error: %v", err)
}
@@ -256,10 +290,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
} else {
logf("running session (budget $%.2f, timeout %s)", sc.Budget(), sc.Timeout())
var err error
- res, err = g.Exec(ctx, sessionScript(sc, prompt, logPath), sandbox.ExecOpts{
- User: image.AgentUser, Dir: workDirFor(sc), HomeDir: image.AgentHome,
- PreserveEnv: true, PathPrepend: image.PathPrepend(), Timeout: sc.Timeout() + time.Minute,
- })
+ res, err = g.Exec(ctx, sh.CIExecSession(logPath, sessionDir, claudeFlags(sc, prompt, sh)), sessionOpts)
if err != nil {
logf("session exec error: %v", err)
}
@@ -279,9 +310,9 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// the default working directory silently found nothing whenever `cwd:` was set -- s05 was
// the only such scenario and it reported session_ok=false on an otherwise perfect run,
// disabling exactly the signal that distinguishes a dead agent from a capture gap.
- outPath := shq(sessionFile(sc, "claude-out.json"))
- errPath := shq(sessionFile(sc, "claude-err.txt"))
- if r, err := g.Exec(ctx, "cat "+outPath+" 2>/dev/null || true", agent); err == nil {
+ outPath := sessionFile(sc, "claude-out.json", lay, sh)
+ errPath := sessionFile(sc, "claude-err.txt", lay, sh)
+ if r, err := g.Exec(ctx, sh.ReadFile(outPath), agent); err == nil {
var result map[string]any
if json.Unmarshal([]byte(strings.TrimSpace(r.Stdout)), &result) == nil {
art.SessionOK, art.SessionKnown = sessionOutcome(result)
@@ -294,7 +325,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
logf("claude: %s", art.Meta["claude_result"])
_ = os.WriteFile(filepath.Join(runDir, "claude-result.json"), []byte(r.Stdout), 0o600)
} else {
- if e, err2 := g.Exec(ctx, "tail -c 800 "+errPath+" 2>/dev/null || true", agent); err2 == nil {
+ if e, err2 := g.Exec(ctx, sh.TailBytes(errPath, 800), agent); err2 == nil {
art.Meta["claude_stderr"] = oneLine(e.Stdout)
}
}
@@ -310,9 +341,7 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// The probe always emits a marker, so its output is never ambiguous. `cat f || echo
// __MISSING__` produced empty stdout for both a failed exec and a legitimately empty
// sentinel file, which would have misreported the second as a broken probe.
- probe := fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",
- shq(sentinel))
- r, execErr := g.Exec(ctx, probe, agent)
+ r, execErr := g.Exec(ctx, sh.ProbeFile(sentinel), agent)
art.SentinelProbed = execErr == nil &&
(strings.Contains(r.Stdout, "__FOUND__") || strings.Contains(r.Stdout, "__MISSING__"))
if !art.SentinelProbed {
@@ -338,8 +367,8 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// Read what the in-session sampler observed while the agent was actually running. See
// argvSamplerScript for why this cannot be a post-session scan.
- if r, err := p.Exec(ctx, inst, "cat "+shq(argvVerdictPath)+" 2>/dev/null || true",
- sandbox.ExecOpts{User: "root", Timeout: time.Minute}); err == nil {
+ if r, err := p.Exec(ctx, inst, sh.ReadFile(sh.ArgvVerdictPath()),
+ sandbox.ExecOpts{User: lay.PrivilegedUser(), Timeout: time.Minute}); err == nil {
sawAgent := strings.Contains(r.Stdout, "saw_agent=1")
art.Meta["argv_saw_agent"] = fmt.Sprintf("%v", sawAgent)
switch {
@@ -393,16 +422,16 @@ func Run(ctx context.Context, p sandbox.Provider, sc scenario.Scenario, opts Opt
// Wait for quiescence rather than sleeping a fixed amount: the collector batches at 5s
// and Claude Code's metric interval defaults to 60s, so a fixed sleep is either wasteful
// or wrong. Record how long it took -- that latency is itself a finding.
- stable, waited := waitForQuiescence(ctx, g, agent, logPath, logf)
+ stable, waited := waitForQuiescence(ctx, g, agent, sh, logPath, logf)
art.Meta["quiescence_seconds"] = fmt.Sprintf("%.0f", waited.Seconds())
art.Meta["quiescent"] = fmt.Sprintf("%v", stable)
- local := filepath.Join(runDir, "runtime.jsonl")
- if err := g.Get(ctx, logPath, local); err != nil {
+ localLog := filepath.Join(runDir, "runtime.jsonl")
+ if err := g.Get(ctx, logPath, localLog); err != nil {
return art, fmt.Errorf("collect runtime log: %w", err)
}
- art.RuntimeLog = local
- logf("collected %s", local)
+ art.RuntimeLog = localLog
+ logf("collected %s", localLog)
// meta.json is written by the deferred host-state block above, so it always includes the
// host comparison.
@@ -423,52 +452,47 @@ func sessionOutcome(result map[string]any) (ok, known bool) {
return !isErr, true
}
+// versionProbe asks the guest to identify itself.
+//
+// Not just recorded but checked by the caller: an empty version line means beacon or claude is
+// not reachable in the guest, and every later failure is then a symptom of that rather than of
+// Beacon.
+func versionProbe(p sandbox.Platform) string {
+ if p.IsWindows() {
+ return "& beacon version; & claude --version; " +
+ "[System.Security.Principal.WindowsIdentity]::GetCurrent().Name; $env:PROCESSOR_ARCHITECTURE"
+ }
+ return "beacon version; claude --version; id -un; uname -m"
+}
+
// sessionFile resolves a file the agent session wrote, relative to the directory that session
// actually ran in.
-func sessionFile(sc scenario.Scenario, name string) string {
- return filepath.ToSlash(filepath.Join(workDirFor(sc), name))
+func sessionFile(sc scenario.Scenario, name string, lay image.Layout, sh shell) string {
+ return sh.Join(workDirFor(sc, lay, sh), name)
}
// claudeFlags builds the agent invocation shared by both session shapes.
-func claudeFlags(sc scenario.Scenario, prompt string) []string {
+func claudeFlags(sc scenario.Scenario, prompt string, sh shell) []string {
flags := []string{
- "-p", shq(prompt),
+ "-p", sh.Quote(prompt),
"--output-format", "json",
"--dangerously-skip-permissions",
fmt.Sprintf("--max-budget-usd %.2f", sc.Budget()),
}
if len(sc.Tools) > 0 {
- flags = append(flags, "--tools", shq(strings.Join(sc.Tools, ",")))
+ flags = append(flags, "--tools", sh.Quote(strings.Join(sc.Tools, ",")))
} else if sc.Tools != nil {
- flags = append(flags, "--tools", shq(""))
+ flags = append(flags, "--tools", sh.Quote(""))
}
if len(sc.DisallowedTools) > 0 {
- flags = append(flags, "--disallowedTools", shq(strings.Join(sc.DisallowedTools, ",")))
+ flags = append(flags, "--disallowedTools", sh.Quote(strings.Join(sc.DisallowedTools, ",")))
}
return flags
}
-// sessionScript builds the in-guest command.
-//
-// The child is wrapped in `sh -c` so Claude's stdout is redirected *inside* `beacon ci exec`.
-// Redirecting outside it would interleave Beacon's own report with Claude's JSON in one file,
-// which made the first M0 run's output unparseable.
-func sessionScript(sc scenario.Scenario, prompt, logPath string) string {
- inner := "claude " + strings.Join(claudeFlags(sc, prompt), " ") +
- " > claude-out.json 2> claude-err.txt"
- // The script must exit with `beacon ci exec`'s status, not with `tail`'s. Ending on the tail
- // made every session look successful to p.Exec no matter how ci exec fared, and nothing read
- // CI_EXEC_RC either -- so a collector that failed outright was indistinguishable from one
- // that worked. Reported by the Copilot reviewer.
- return fmt.Sprintf("beacon ci exec --harness claude --log-path %s -- sh -c %s "+
- "> ci-out.txt 2> ci-err.txt; rc=$?; echo CI_EXEC_RC=$rc; "+
- "tail -c 400 ci-out.txt; exit $rc",
- shq(logPath), shq(inner))
-}
-
// waitForQuiescence polls the log until its size stops changing, bounded.
func waitForQuiescence(ctx context.Context, g guest,
- opts sandbox.ExecOpts, logPath string, logf func(string, ...any)) (bool, time.Duration) {
+ opts sandbox.ExecOpts, sh shell, logPath string, logf func(string, ...any)) (bool, time.Duration) {
const (
interval = 3 * time.Second
@@ -480,7 +504,7 @@ func waitForQuiescence(ctx context.Context, g guest,
stableSince := time.Time{}
for time.Since(start) < maxWait {
- r, err := g.Exec(ctx, "wc -c < "+shq(logPath)+" 2>/dev/null || echo 0", opts)
+ r, err := g.Exec(ctx, sh.FileSize(logPath), opts)
if err == nil {
cur := strings.TrimSpace(r.Stdout)
if cur == last && cur != "0" {
@@ -504,11 +528,33 @@ func waitForQuiescence(ctx context.Context, g guest,
return false, time.Since(start)
}
-func workDirFor(sc scenario.Scenario) string {
+func workDirFor(sc scenario.Scenario, lay image.Layout, sh shell) string {
if sc.Cwd == "" {
- return image.WorkDir
+ return lay.WorkDir
+ }
+ return sh.Join(lay.WorkDir, sc.Cwd)
+}
+
+// imageSpecFor describes the environment a scenario needs.
+//
+// Windows takes the short path deliberately: nothing here provisions it. The CI job built the
+// binaries, installed the agent runtime and created the work directory before this process
+// started, so there is no image to build and no artifact to push. What would otherwise be
+// checked here -- that the binary under test is actually reachable -- is checked better a few
+// lines later by asking the guest for `beacon version`, which tests reachability rather than a
+// path guess.
+func imageSpecFor(p sandbox.Platform, sc scenario.Scenario, opts Options,
+ logf func(string, ...any)) (sandbox.ImageSpec, error) {
+
+ if p.IsWindows() {
+ return sandbox.ImageSpec{Base: "windows/amd64"}, nil
}
- return filepath.ToSlash(filepath.Join(image.WorkDir, sc.Cwd))
+ logf("building image (cached layers are free)")
+ return image.Build(image.Spec{
+ RepoRoot: opts.RepoRoot,
+ ClaudeVersion: opts.ClaudeVersion,
+ WithDocker: sc.NeedsRealSystemd(),
+ }, logf)
}
func randomToken() string {
@@ -639,8 +685,9 @@ func sampleCountOf(verdict string) string {
// The log path comes from `endpoint status --json` rather than being assumed, because the whole
// point of an install scenario is to check what Beacon actually did — hardcoding the expected
// path would make the scenario agree with itself instead of with Beacon.
-func doInstall(ctx context.Context, g guest,
- sc scenario.Scenario, agent sandbox.ExecOpts, art *Artifacts, logf func(string, ...any)) (string, error) {
+func doInstall(ctx context.Context, g guest, sc scenario.Scenario,
+ privileged, agent sandbox.ExecOpts, lay image.Layout, sh shell,
+ art *Artifacts, logf func(string, ...any)) (string, error) {
in := sc.Install
system := in.Mode == "system"
@@ -652,12 +699,14 @@ func doInstall(ctx context.Context, g guest,
if in.Service != "" {
svc = " --service=" + in.Service
}
- // System-mode install requires root; user mode must not have it, or the install would
- // write system paths while claiming to be a user install.
+ // System-mode install needs elevation; user mode must not have it, or the install would
+ // write system paths while claiming to be a user install. On Windows the two options differ
+ // only in what they cannot do -- there is one account, already elevated -- which is a real
+ // weakness of the Windows user-mode scenario and is why w02 asserts the supervised backend
+ // rather than trusting that it ran unprivileged.
opts := agent
if system {
- opts = sandbox.ExecOpts{User: "root", Dir: image.WorkDir, Timeout: 5 * time.Minute,
- PathPrepend: image.PathPrepend()}
+ opts = privileged
}
art.Meta["install_mode"] = in.Mode
art.Meta["install_service"] = in.Service
@@ -687,13 +736,13 @@ func doInstall(ctx context.Context, g guest,
// SUDO_USER is how the real path identifies them (`sudo apt install ./beacon.deb`), so it is
// what gets set here rather than inventing a flag that no user would pass.
if system {
- cmd := fmt.Sprintf("SUDO_USER=%s beacon endpoint user-config repair-installed "+
- "--system --harness claude --json 2>&1", image.AgentUser)
+ cmd := sh.WithEnv("SUDO_USER", lay.ConsoleUser(),
+ "beacon endpoint user-config repair-installed --system --harness claude --json 2>&1")
u, err := g.Exec(ctx, cmd, opts)
art.Meta["install_user_config"] = oneLine(u.Stdout + u.Stderr)
if err != nil || u.ExitCode != 0 {
return "", fmt.Errorf("configuring %s's agent runtime failed (rc=%d): %v %s",
- image.AgentUser, u.ExitCode, err, art.Meta["install_user_config"])
+ lay.ConsoleUser(), u.ExitCode, err, art.Meta["install_user_config"])
}
logf("user config: %s", art.Meta["install_user_config"])
// A skip is not a pass. The command reports one rather than failing, because on a real
@@ -701,11 +750,11 @@ func doInstall(ctx context.Context, g guest,
// treating a skip as success would let a broken resolver produce a green run.
if strings.Contains(art.Meta["install_user_config"], "skipped_reason") {
return "", fmt.Errorf("user-config skipped instead of configuring %s: %s",
- image.AgentUser, art.Meta["install_user_config"])
+ lay.ConsoleUser(), art.Meta["install_user_config"])
}
}
- logPath, err := installedLogPath(ctx, g, scope, opts, in, art, logf)
+ logPath, err := installedLogPath(ctx, g, scope, opts, in, lay, art, logf)
if err != nil {
return "", err
}
@@ -723,7 +772,7 @@ func doInstall(ctx context.Context, g guest,
// documented default path — but ExpectStatusRunning is enforced, since a scenario asking for a
// running service and silently accepting an unparseable status would verify nothing.
func installedLogPath(ctx context.Context, g guest,
- scope string, opts sandbox.ExecOpts, in *scenario.Install, art *Artifacts,
+ scope string, opts sandbox.ExecOpts, in *scenario.Install, lay image.Layout, art *Artifacts,
logf func(string, ...any)) (string, error) {
s, err := g.Exec(ctx, "beacon endpoint status "+scope+" --json 2>&1", opts)
@@ -766,15 +815,5 @@ func installedLogPath(ctx context.Context, g guest,
"the selected backend can be confirmed; refusing to report a pass the run did not earn")
}
logf("status unparseable; falling back to the documented default log path")
- if in.Mode == "system" {
- return "/var/log/beacon-agent/runtime.jsonl", nil
- }
- return image.AgentHome + "/.beacon/endpoint/logs/runtime.jsonl", nil
-}
-
-// plainSessionScript runs the agent with no collector wrapper, for install scenarios where a
-// persistent collector is already running as a service.
-func plainSessionScript(sc scenario.Scenario, prompt string) string {
- return "claude " + strings.Join(claudeFlags(sc, prompt), " ") +
- " > claude-out.json 2> claude-err.txt; echo CLAUDE_RC=$?"
+ return lay.DefaultLogPath(in.Mode == "system"), nil
}
diff --git a/beacon-sandbox/runner/runner_test.go b/beacon-sandbox/runner/runner_test.go
index b284d179..a321f1a0 100644
--- a/beacon-sandbox/runner/runner_test.go
+++ b/beacon-sandbox/runner/runner_test.go
@@ -1,7 +1,6 @@
package runner
import (
- "fmt"
"os"
"os/exec"
"path/filepath"
@@ -18,8 +17,9 @@ import (
// otherwise perfect run. That disabled precisely the signal that distinguishes "the agent never
// ran" from "Beacon missed the event", which is the distinction the whole verdict model rests on.
func TestSessionFilesResolveInsideTheScenarioWorkingDirectory(t *testing.T) {
+ lay, sh := image.LinuxLayout(), posixShell{}
withCwd := scenario.Scenario{ID: "s05-repo-task", Cwd: "repo"}
- got := sessionFile(withCwd, "claude-out.json")
+ got := sessionFile(withCwd, "claude-out.json", lay, sh)
if !strings.HasSuffix(got, "/repo/claude-out.json") {
t.Errorf("a scenario with cwd:repo must read from that directory, got %q", got)
@@ -28,14 +28,16 @@ func TestSessionFilesResolveInsideTheScenarioWorkingDirectory(t *testing.T) {
t.Error("a bare relative name is what caused the original silent miss")
}
// It must be the same directory the session was launched in, or the two can drift apart.
- if !strings.HasPrefix(got, workDirFor(withCwd)) {
- t.Errorf("readback path %q is not under the session working dir %q", got, workDirFor(withCwd))
+ if !strings.HasPrefix(got, workDirFor(withCwd, lay, sh)) {
+ t.Errorf("readback path %q is not under the session working dir %q",
+ got, workDirFor(withCwd, lay, sh))
}
}
// The common case must keep working: no cwd means the default working directory.
func TestSessionFilesDefaultToTheWorkDir(t *testing.T) {
- got := sessionFile(scenario.Scenario{ID: "s01-hello"}, "claude-out.json")
+ got := sessionFile(scenario.Scenario{ID: "s01-hello"}, "claude-out.json",
+ image.LinuxLayout(), posixShell{})
want := image.WorkDir + "/claude-out.json"
if got != want {
@@ -46,7 +48,8 @@ func TestSessionFilesDefaultToTheWorkDir(t *testing.T) {
// Paths are consumed by a POSIX shell inside the guest, so they must use forward slashes
// regardless of the host that built them.
func TestSessionFilePathsAreAlwaysSlashed(t *testing.T) {
- got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt")
+ got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt",
+ image.LinuxLayout(), posixShell{})
if strings.Contains(got, `\`) {
t.Errorf("guest paths must be slash-separated, got %q", got)
@@ -56,12 +59,32 @@ func TestSessionFilePathsAreAlwaysSlashed(t *testing.T) {
}
}
+// The mirror of the previous test for the other dialect. A Windows guest consumes these paths
+// through Windows APIs, so a forward slash that "mostly works" would still be wrong in every
+// verdict a human reads -- and a bare drive letter is a *relative* path, which is wrong in a way
+// that silently writes to the wrong directory.
+func TestWindowsSessionFilePathsUseBackslashesAndKeepTheDriveRoot(t *testing.T) {
+ lay, sh := image.WindowsLayout(), powerShell{}
+ got := sessionFile(scenario.Scenario{ID: "x", Cwd: "nested/dir"}, "claude-err.txt", lay, sh)
+
+ if strings.Contains(got, "/") {
+ t.Errorf("windows guest paths must be backslash-separated, got %q", got)
+ }
+ if !strings.HasSuffix(got, `\nested\dir\claude-err.txt`) {
+ t.Errorf("nested cwd not resolved, got %q", got)
+ }
+ // "C:" without a separator means the current directory on that drive, not its root.
+ if strings.HasPrefix(got, `C:`) && !strings.HasPrefix(got, `C:\`) {
+ t.Errorf("a drive letter must keep its separator, got %q", got)
+ }
+}
+
// The argv scan is a disclosure claim, so it must rest on an observation actually made. An
// earlier version scanned only after `beacon ci exec` returned, when every process that might
// have held the key was already gone -- ARGV_CLEAN was close to vacuous. Cursor Bugbot flagged
// it on the first commit. These pin the properties that make the sampler meaningful.
func TestArgvSamplerRunsInBackgroundAndSamplesRepeatedly(t *testing.T) {
- got := argvSamplerScript()
+ got := posixShell{}.ArgvSampler()
if !strings.Contains(got, "nohup") || !strings.HasSuffix(strings.TrimSpace(got), "ARGV_SAMPLER_STARTED") {
t.Error("the sampler must detach so the session can run alongside it")
@@ -79,7 +102,7 @@ func TestArgvSamplerRunsInBackgroundAndSamplesRepeatedly(t *testing.T) {
// The matcher must never carry the key in its own argv, or the scan reports a leak it created.
func TestArgvSamplerNeverPutsTheKeyInAnArgv(t *testing.T) {
- got := argvSamplerScript()
+ got := posixShell{}.ArgvSampler()
if strings.Contains(got, `grep -qF "$ANTHROPIC_API_KEY"`) || strings.Contains(got, "grep -F $ANTHROPIC_API_KEY") {
t.Error("expanding the key into a matcher's argv is the self-poisoning bug")
@@ -92,7 +115,7 @@ func TestArgvSamplerNeverPutsTheKeyInAnArgv(t *testing.T) {
// With no key there is nothing to search for, so the sampler must say so rather than write a
// clean verdict. A check that cannot run must never read as one that passed.
func TestArgvSamplerReportsInvalidRatherThanCleanWithoutAKey(t *testing.T) {
- got := argvSamplerScript()
+ got := posixShell{}.ArgvSampler()
if !strings.Contains(got, "ARGV_CHECK_INVALID_NO_KEY") {
t.Error("a missing key must produce an explicit invalid marker")
@@ -126,7 +149,9 @@ func TestSampleCountIsExtractedForTheRecord(t *testing.T) {
// either, so a collector that failed outright was indistinguishable from one that worked.
// Reported by the Copilot reviewer.
func TestSessionScriptPropagatesTheCollectorExitStatus(t *testing.T) {
- got := sessionScript(scenario.Scenario{ID: "s01", Prompt: "hi"}, "hi", "/work/runtime.jsonl")
+ sh := posixShell{}
+ sc := scenario.Scenario{ID: "s01", Prompt: "hi"}
+ got := sh.CIExecSession("/work/runtime.jsonl", image.WorkDir, claudeFlags(sc, "hi", sh))
if !strings.Contains(got, "rc=$?") || !strings.Contains(got, "exit $rc") {
t.Errorf("the script must capture and re-raise ci exec's status:\n%s", got)
@@ -145,9 +170,7 @@ func TestSessionScriptPropagatesTheCollectorExitStatus(t *testing.T) {
// be told apart -- and the first is an infrastructure problem while the second is a real absence of
// agent work. An explicit marker on every branch removes the ambiguity.
func TestSentinelProbeAlwaysEmitsAMarker(t *testing.T) {
- // Rebuild the probe the runner uses, for the same sentinel path.
- probe := fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",
- shq("/home/agent/work/out.txt"))
+ probe := posixShell{}.ProbeFile("/home/agent/work/out.txt")
for _, want := range []string{"__FOUND__", "__MISSING__"} {
if !strings.Contains(probe, want) {
@@ -203,7 +226,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {
t.Skip("no sh on PATH")
}
f := filepath.Join(t.TempDir(), "sampler.sh")
- if err := os.WriteFile(f, []byte(argvSamplerScript()), 0o600); err != nil {
+ if err := os.WriteFile(f, []byte(posixShell{}.ArgvSampler()), 0o600); err != nil {
t.Fatal(err)
}
out, err := exec.Command(sh, "-n", f).CombinedOutput()
@@ -212,7 +235,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {
}
// The inner sampler body is a heredoc, which `sh -n` does not parse. Extract and check it too,
// since that is where the polling loop lives.
- body := argvSamplerScript()
+ body := posixShell{}.ArgvSampler()
start := strings.Index(body, "#!/bin/sh")
end := strings.Index(body, "\nSAMPLER\n")
if start < 0 || end < 0 || end < start {
@@ -230,7 +253,7 @@ func TestArgvSamplerScriptIsValidShell(t *testing.T) {
// A clean verdict from a sampler that never saw an agent process must read as unverified. This is
// the invariant the whole tool rests on: silence means verified-clean, never "we did not look."
func TestArgvSamplerEmitsWhetherItSawTheAgent(t *testing.T) {
- script := argvSamplerScript()
+ script := posixShell{}.ArgvSampler()
for _, want := range []string{"saw_agent=0", "saw_agent=$saw_agent", "*claude*"} {
if !strings.Contains(script, want) {
t.Errorf("sampler no longer records whether the agent was in view (missing %q)", want)
diff --git a/beacon-sandbox/runner/shell.go b/beacon-sandbox/runner/shell.go
new file mode 100644
index 00000000..4fe7d742
--- /dev/null
+++ b/beacon-sandbox/runner/shell.go
@@ -0,0 +1,349 @@
+package runner
+
+import (
+ "encoding/base64"
+ "fmt"
+ "path"
+ "strings"
+
+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"
+)
+
+// shell builds the guest-side scripts the runner needs, in the guest's own dialect.
+//
+// Every one of these was a bare POSIX string before Windows existed as a target. They are
+// gathered behind an interface rather than sprinkled with `if windows` because the runner's
+// value is that a Windows verdict is produced by *the same* control flow as a Linux one -- the
+// canary, the sentinel gate, quiescence, collection and judging all stay single-path, and only
+// the dialect differs. Branching inline is how the two would drift into one being more
+// forgiving than the other.
+type shell interface {
+ // Quote renders a value as one literal argument.
+ Quote(string) string
+ // Join builds a guest path from segments.
+ Join(...string) string
+ // ProbeFile prints __FOUND__ then the contents, or __MISSING__. Never empty, so a failed
+ // exec is distinguishable from an empty file.
+ ProbeFile(path string) string
+ // FileSize prints the byte count, or 0 when absent.
+ FileSize(path string) string
+ // ReadFile prints the whole file, or nothing when absent.
+ ReadFile(path string) string
+ // TailBytes prints the last n bytes.
+ TailBytes(path string, n int) string
+ // WithEnv prefixes a script with one environment assignment.
+ WithEnv(name, value, script string) string
+ // CIExecSession wraps an agent invocation in `beacon ci exec`, propagating its status.
+ CIExecSession(logPath, workDir string, claudeArgs []string) string
+ // PlainSession runs the agent with no collector wrapper, for install scenarios.
+ PlainSession(workDir string, claudeArgs []string) string
+ // ArgvSampler starts the background credential sampler.
+ ArgvSampler() string
+ // ArgvVerdictPath is where the sampler writes what it saw.
+ ArgvVerdictPath() string
+}
+
+// shellFor returns the dialect for a platform.
+func shellFor(p sandbox.Platform) shell {
+ if p.IsWindows() {
+ return powerShell{}
+ }
+ return posixShell{}
+}
+
+// ---------------------------------------------------------------------------
+// POSIX
+// ---------------------------------------------------------------------------
+
+type posixShell struct{}
+
+func (posixShell) Quote(s string) string { return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'" }
+
+func (posixShell) Join(parts ...string) string { return path.Join(parts...) }
+
+// ProbeFile always emits a marker, so its output is never ambiguous. `cat f || echo
+// __MISSING__` produced empty stdout for both a failed exec and a legitimately empty sentinel
+// file, which would have misreported the second as a broken probe.
+func (s posixShell) ProbeFile(p string) string {
+ return fmt.Sprintf("if [ -f %[1]s ]; then echo __FOUND__; cat %[1]s; else echo __MISSING__; fi",
+ s.Quote(p))
+}
+
+func (s posixShell) FileSize(p string) string {
+ return "wc -c < " + s.Quote(p) + " 2>/dev/null || echo 0"
+}
+
+func (s posixShell) ReadFile(p string) string {
+ return "cat " + s.Quote(p) + " 2>/dev/null || true"
+}
+
+func (s posixShell) TailBytes(p string, n int) string {
+ return fmt.Sprintf("tail -c %d %s 2>/dev/null || true", n, s.Quote(p))
+}
+
+func (s posixShell) WithEnv(name, value, script string) string {
+ return name + "=" + s.Quote(value) + " " + script
+}
+
+// CIExecSession wraps the agent in `beacon ci exec`.
+//
+// The child is wrapped in `sh -c` so the agent's stdout is redirected *inside* `beacon ci
+// exec`. Redirecting outside it would interleave Beacon's own report with the agent's JSON in
+// one file, which made the first M0 run's output unparseable.
+//
+// The script must exit with `beacon ci exec`'s status, not with `tail`'s. Ending on the tail
+// made every session look successful no matter how ci exec fared, so a collector that failed
+// outright was indistinguishable from one that worked.
+func (s posixShell) CIExecSession(logPath, _ string, claudeArgs []string) string {
+ inner := "claude " + strings.Join(claudeArgs, " ") + " > claude-out.json 2> claude-err.txt"
+ return fmt.Sprintf("beacon ci exec --harness claude --log-path %s -- sh -c %s "+
+ "> ci-out.txt 2> ci-err.txt; rc=$?; echo CI_EXEC_RC=$rc; "+
+ "tail -c 400 ci-out.txt; exit $rc",
+ s.Quote(logPath), s.Quote(inner))
+}
+
+func (posixShell) PlainSession(_ string, claudeArgs []string) string {
+ return "claude " + strings.Join(claudeArgs, " ") +
+ " > claude-out.json 2> claude-err.txt; echo CLAUDE_RC=$?"
+}
+
+func (posixShell) ArgvVerdictPath() string { return "/tmp/beacon-sandbox-argv-verdict" }
+
+// ArgvSampler starts a background sampler that watches for the injected credential in the
+// guest process table for as long as the agent session runs.
+//
+// It has to run concurrently with the session, and an earlier version did not: the scan
+// executed after `beacon ci exec` had already returned, so every process that might have held
+// the key in argv was already gone and ARGV_CLEAN was close to vacuous -- a disclosure claim
+// resting on an observation never made.
+//
+// Both `ps` output and every /proc/<pid>/cmdline are checked, since a shell can hide argv from
+// one but not the other. It runs as root because another user's cmdline is otherwise
+// unreadable, and a check that can only see its own process proves nothing.
+//
+// The matcher must not carry the key in its own argv or the scan finds itself: a first version
+// used `grep -qF "$KEY"`, which the shell expands into grep's argv, so the `ps` in the same
+// pipeline saw the grep process and reported a leak the tool had created. awk reads the key
+// from ENVIRON instead, and the sampler's own processes are skipped as a second guard.
+func (s posixShell) ArgvSampler() string {
+ return fmt.Sprintf(`set -u
+rm -f %[1]s
+if [ -z "${ANTHROPIC_API_KEY:-}" ]; then
+ echo "ARGV_CHECK_INVALID_NO_KEY samples=0" > %[1]s
+ exit 0
+fi
+cat > %[2]s <<'SAMPLER'
+#!/bin/sh
+VERDICT="$1"
+scan() {
+ awk 'BEGIN{k=ENVIRON["ANTHROPIC_API_KEY"]; if (k=="") exit 1}
+ /awk|ARGV_SAMPLER_SELF/ {next}
+ index($0,k){found=1}
+ END{exit !found}'
+}
+n=0
+saw_agent=0
+# Bounded so a leaked sandbox cannot spin forever; the instance is destroyed long before this.
+while [ "$n" -lt 900 ]; do
+ n=$((n+1))
+ procs="$(ps -eo args= 2>/dev/null || true)"
+ # Whether the agent was ever in view at all. A clean verdict from a sampler that never saw the
+ # process holding the key is not evidence of anything.
+ case "$procs" in *claude*) saw_agent=1 ;; esac
+ if printf '%%s\n' "$procs" | scan; then
+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=ps" > "$VERDICT"
+ exit 0
+ fi
+ for f in /proc/[0-9]*/cmdline; do
+ [ -r "$f" ] || continue
+ cmd="$(tr '\0' '\n' < "$f" 2>/dev/null || true)"
+ case "$cmd" in *claude*) saw_agent=1 ;; esac
+ if printf '%%s\n' "$cmd" | scan; then
+ echo "ARGV_LEAK samples=$n saw_agent=$saw_agent via=proc" > "$VERDICT"
+ exit 0
+ fi
+ done
+ echo "ARGV_CLEAN samples=$n saw_agent=$saw_agent" > "$VERDICT"
+ sleep 1
+done
+SAMPLER
+chmod +x %[2]s
+nohup %[2]s %[1]s > /dev/null 2>&1 &
+echo ARGV_SAMPLER_STARTED`, s.ArgvVerdictPath(), "/tmp/beacon-sandbox-argv-sampler.sh")
+}
+
+// ---------------------------------------------------------------------------
+// PowerShell
+// ---------------------------------------------------------------------------
+
+type powerShell struct{}
+
+// Quote renders a PowerShell single-quoted literal, where the only escape is a doubled quote.
+// Single-quoted is deliberate: PowerShell performs no expansion inside it, so a value carrying
+// `$`, backtick or `%` cannot become something else.
+func (powerShell) Quote(s string) string {
+ return "'" + strings.ReplaceAll(s, "'", "''") + "'"
+}
+
+// Join uses backslashes, since these paths are consumed by Windows APIs and shown to a human
+// reading a verdict. Forward slashes would mostly work and would look wrong in every report.
+func (powerShell) Join(parts ...string) string {
+ cleaned := make([]string, 0, len(parts))
+ for _, p := range parts {
+ if p = strings.Trim(strings.ReplaceAll(p, "/", `\`), `\`); p != "" {
+ cleaned = append(cleaned, p)
+ }
+ }
+ if len(cleaned) == 0 {
+ return ""
+ }
+ // A leading drive letter keeps its separator. "C:" alone is a *relative* path on Windows --
+ // the current directory on that drive -- so trimming the backslash off "C:\work" would
+ // silently change which directory a script writes to. Everything else joins plainly, since
+ // the trim above already removed interior duplicates.
+ if len(cleaned) > 1 && strings.HasSuffix(cleaned[0], ":") {
+ return cleaned[0] + `\` + strings.Join(cleaned[1:], `\`)
+ }
+ return strings.Join(cleaned, `\`)
+}
+
+// ProbeFile mirrors the POSIX contract: a marker on every branch, so empty stdout can only mean
+// the exec itself failed.
+//
+// -LiteralPath throughout, not -Path: -Path treats [, ] and ? as wildcards, and a sentinel or
+// work directory containing them would silently resolve to nothing.
+func (s powerShell) ProbeFile(p string) string {
+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) { '__FOUND__'; "+
+ "Get-Content -LiteralPath %[1]s -Raw } else { '__MISSING__' }", s.Quote(p))
+}
+
+func (s powerShell) FileSize(p string) string {
+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) "+
+ "{ (Get-Item -LiteralPath %[1]s).Length } else { 0 }", s.Quote(p))
+}
+
+func (s powerShell) ReadFile(p string) string {
+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) "+
+ "{ Get-Content -LiteralPath %[1]s -Raw }", s.Quote(p))
+}
+
+// TailBytes reads bytes rather than lines, matching `tail -c`, and clamps the start so a file
+// shorter than n is returned whole instead of throwing.
+func (s powerShell) TailBytes(p string, n int) string {
+ q := s.Quote(p)
+ return fmt.Sprintf("if (Test-Path -LiteralPath %[1]s -PathType Leaf) { "+
+ "$b = [System.IO.File]::ReadAllBytes(%[1]s); "+
+ "$start = [Math]::Max(0, $b.Length - %[2]d); "+
+ "[System.Text.Encoding]::UTF8.GetString($b, $start, $b.Length - $start) }", q, n)
+}
+
+// WithEnv sets a process-scoped variable. PowerShell has no `VAR=value cmd` prefix form -- the
+// POSIX spelling is a syntax error here, which is why this is on the interface at all.
+func (s powerShell) WithEnv(name, value, script string) string {
+ return fmt.Sprintf("$env:%s = %s; %s", name, s.Quote(value), script)
+}
+
+// CIExecSession wraps the agent in `beacon ci exec`, keeping the agent's stdout redirected
+// inside the wrapper so Beacon's own report cannot interleave with the agent's JSON.
+//
+// The inner command travels as base64 into a file rather than as a nested quoted string. The
+// prompt carries a canary, arbitrary punctuation and possibly quotes, and it would otherwise
+// have to survive two levels of PowerShell quoting -- the failure mode being a prompt that
+// arrives subtly altered, which looks exactly like the agent ignoring instructions. This is the
+// same reasoning the nested-container guest already uses for crossing two shells.
+func (s powerShell) CIExecSession(logPath, workDir string, claudeArgs []string) string {
+ inner := s.innerSessionScript(workDir, claudeArgs)
+ return fmt.Sprintf(`%s
+& beacon ci exec --harness claude --log-path %s -- pwsh -NoProfile -File $innerPath > ci-out.txt 2> ci-err.txt
+$rc = $LASTEXITCODE
+if ($null -eq $rc) { $rc = 0 }
+"CI_EXEC_RC=$rc"
+%s
+exit $rc`,
+ inner, s.Quote(logPath), s.TailBytes("ci-out.txt", 400))
+}
+
+// PlainSession runs the agent directly, for install scenarios where a persistent collector is
+// already running as a service.
+func (s powerShell) PlainSession(workDir string, claudeArgs []string) string {
+ return fmt.Sprintf(`%s
+& pwsh -NoProfile -File $innerPath
+$rc = $LASTEXITCODE
+if ($null -eq $rc) { $rc = 0 }
+"CLAUDE_RC=$rc"`, s.innerSessionScript(workDir, claudeArgs))
+}
+
+// innerSessionScript materializes the agent invocation as a .ps1 and leaves its path in
+// $innerPath.
+//
+// Set-Location inside the inner script is load-bearing: the redirects are relative, and the
+// process that ends up running this file is a grandchild of the shell that wrote it, so its
+// working directory cannot be assumed.
+func (s powerShell) innerSessionScript(workDir string, claudeArgs []string) string {
+ body := fmt.Sprintf("Set-Location -LiteralPath %s\n"+
+ "& claude %s > claude-out.json 2> claude-err.txt\n"+
+ "exit $LASTEXITCODE\n",
+ s.Quote(workDir), strings.Join(claudeArgs, " "))
+ enc := base64.StdEncoding.EncodeToString([]byte(body))
+ return fmt.Sprintf(`$innerPath = Join-Path $env:TEMP 'beacon-sandbox-session.ps1'
+[System.IO.File]::WriteAllText($innerPath, [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('%s')))`, enc)
+}
+
+func (powerShell) ArgvVerdictPath() string {
+ return `C:\Windows\Temp\beacon-sandbox-argv-verdict`
+}
+
+// ArgvSampler is the Windows half of the credential-in-argv check, and it has the same job as
+// the POSIX one: observe the process table *while* the agent runs, and never carry the key in
+// its own command line.
+//
+// Win32_Process replaces both /proc and ps -- Windows has no equivalent of either, and
+// CommandLine on that class is the only place a full argv is readable for another process.
+// The key is compared with String.Contains against a value read from the environment, so it
+// never reaches an argument list; the sampler also skips its own PID, the same second guard the
+// POSIX version keeps.
+//
+// Start-Process rather than a background job: a PowerShell job dies with the shell that created
+// it, and this shell exits as soon as Exec returns -- which is before the session even starts.
+// The sampler has to outlive its parent, exactly as nohup makes it on POSIX.
+func (s powerShell) ArgvSampler() string {
+ verdict := s.ArgvVerdictPath()
+ scriptPath := `C:\Windows\Temp\beacon-sandbox-argv-sampler.ps1`
+ body := `param([string]$Verdict)
+$key = $env:ANTHROPIC_API_KEY
+if (-not $key) {
+ Set-Content -LiteralPath $Verdict -Value 'ARGV_CHECK_INVALID_NO_KEY samples=0'
+ exit 0
+}
+$me = $PID
+$n = 0
+$sawAgent = 0
+# Bounded so a leaked runner cannot spin forever; the job ends long before this.
+while ($n -lt 900) {
+ $n++
+ $procs = @(Get-CimInstance Win32_Process -ErrorAction SilentlyContinue)
+ foreach ($p in $procs) {
+ if ($p.ProcessId -eq $me) { continue }
+ $cl = $p.CommandLine
+ if (-not $cl) { continue }
+ if ($cl -like '*claude*') { $sawAgent = 1 }
+ if ($cl.Contains($key)) {
+ Set-Content -LiteralPath $Verdict -Value "ARGV_LEAK samples=$n saw_agent=$sawAgent via=win32process"
+ exit 0
+ }
+ }
+ Set-Content -LiteralPath $Verdict -Value "ARGV_CLEAN samples=$n saw_agent=$sawAgent"
+ Start-Sleep -Seconds 1
+}
+`
+ enc := base64.StdEncoding.EncodeToString([]byte(body))
+ return fmt.Sprintf(`$ErrorActionPreference = 'Continue'
+Remove-Item -LiteralPath %[1]s -ErrorAction SilentlyContinue
+if (-not $env:ANTHROPIC_API_KEY) {
+ Set-Content -LiteralPath %[1]s -Value 'ARGV_CHECK_INVALID_NO_KEY samples=0'
+ exit 0
+}
+[System.IO.File]::WriteAllText(%[2]s, [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('%[3]s')))
+Start-Process -FilePath 'pwsh' -ArgumentList '-NoProfile','-NonInteractive','-File',%[2]s,%[1]s -WindowStyle Hidden
+'ARGV_SAMPLER_STARTED'`, s.Quote(verdict), s.Quote(scriptPath), enc)
+}
diff --git a/beacon-sandbox/runner/shell_test.go b/beacon-sandbox/runner/shell_test.go
new file mode 100644
index 00000000..95b4c8d2
--- /dev/null
+++ b/beacon-sandbox/runner/shell_test.go
@@ -0,0 +1,309 @@
+package runner
+
+import (
+ "encoding/base64"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "regexp"
+ "strings"
+ "testing"
+
+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/sandbox"
+ "github.com/asymptote-labs/agent-beacon/beacon-sandbox/scenario"
+)
+
+// The dialect must follow the *provider's* platform, not the host's. A Mac dispatching a Windows
+// run is the normal case, so keying off runtime.GOOS would generate bash for a Windows guest and
+// every script would fail as a syntax error.
+func TestShellFollowsTheGuestPlatformNotTheHost(t *testing.T) {
+ if _, ok := shellFor(sandbox.PlatformWindows).(powerShell); !ok {
+ t.Errorf("a Windows guest must get the PowerShell dialect, got %T",
+ shellFor(sandbox.PlatformWindows))
+ }
+ if _, ok := shellFor(sandbox.PlatformPosix).(posixShell); !ok {
+ t.Errorf("a POSIX guest must get the POSIX dialect, got %T",
+ shellFor(sandbox.PlatformPosix))
+ }
+}
+
+// Quoting is the seam where a canary or prompt gets silently altered, and an altered prompt looks
+// exactly like the agent ignoring instructions. PowerShell's only escape inside a single-quoted
+// literal is a doubled quote, and nothing else is expanded -- which is why single quotes are used.
+func TestPowerShellQuotingIsLiteral(t *testing.T) {
+ sh := powerShell{}
+ cases := map[string]string{
+ `plain`: `'plain'`,
+ `it's`: `'it''s'`,
+ `$env:PATH`: `'$env:PATH'`,
+ "back`tick": "'back`tick'",
+ `100% done`: `'100% done'`,
+ `C:\Program Files`: `'C:\Program Files'`,
+ }
+ for in, want := range cases {
+ if got := sh.Quote(in); got != want {
+ t.Errorf("Quote(%q) = %s, want %s", in, got, want)
+ }
+ }
+}
+
+// Every dialect must be able to emit both markers. Empty stdout has to mean "the exec failed" and
+// nothing else -- `cat f || echo __MISSING__` produced empty output for both a failed exec and a
+// legitimately empty sentinel, and those are an infrastructure problem and a real absence of agent
+// work respectively.
+func TestBothDialectsProbeUnambiguously(t *testing.T) {
+ for _, sh := range []shell{posixShell{}, powerShell{}} {
+ probe := sh.ProbeFile("/tmp/out.txt")
+ for _, want := range []string{"__FOUND__", "__MISSING__"} {
+ if !strings.Contains(probe, want) {
+ t.Errorf("%T probe cannot emit %s: %s", sh, want, probe)
+ }
+ }
+ }
+}
+
+// -LiteralPath, not -Path. -Path treats [, ] and ? as wildcards, so a sentinel or work directory
+// containing one would resolve to nothing and the probe would report the agent idle.
+func TestPowerShellFileOperationsDoNotGlob(t *testing.T) {
+ sh := powerShell{}
+ scripts := map[string]string{
+ "ProbeFile": sh.ProbeFile(`C:\work\a[1].txt`),
+ "FileSize": sh.FileSize(`C:\work\a[1].txt`),
+ "ReadFile": sh.ReadFile(`C:\work\a[1].txt`),
+ "TailBytes": sh.TailBytes(`C:\work\a[1].txt`, 400),
+ }
+ // Leading whitespace is what distinguishes the *parameter* -Path from the cmdlet name
+ // Test-Path, and -LiteralPath does not contain "-Path" at all.
+ bareParam := regexp.MustCompile(`\s-Path\b`)
+ for name, script := range scripts {
+ if bareParam.MatchString(script) {
+ t.Errorf("%s uses -Path, which treats [ ] ? as wildcards: %s", name, script)
+ }
+ }
+}
+
+// PowerShell has no `VAR=value cmd` prefix form; the POSIX spelling is a syntax error there. This
+// is the whole reason WithEnv is on the interface, and it is the same shape as the bug that makes
+// every hook command Beacon writes unusable on Windows.
+func TestWithEnvUsesEachDialectsOwnForm(t *testing.T) {
+ if got := (posixShell{}).WithEnv("SUDO_USER", "agent", "beacon x"); got != "SUDO_USER='agent' beacon x" {
+ t.Errorf("posix WithEnv = %q", got)
+ }
+ got := powerShell{}.WithEnv("SUDO_USER", "runneradmin", "beacon x")
+ if !strings.HasPrefix(got, "$env:SUDO_USER = 'runneradmin';") {
+ t.Errorf("powershell WithEnv must assign $env:, got %q", got)
+ }
+ if strings.HasPrefix(got, "SUDO_USER=") {
+ t.Error("the POSIX inline-assignment form is a syntax error in PowerShell")
+ }
+}
+
+// The Windows sampler has the same contract as the POSIX one: it must outlive the shell that
+// started it, loop rather than sample once, and record whether the agent was ever in view.
+func TestWindowsArgvSamplerDetachesAndLoops(t *testing.T) {
+ got := powerShell{}.ArgvSampler()
+
+ // A PowerShell background job dies with its parent, and the parent exits as soon as Exec
+ // returns -- before the session even starts. Start-Process survives it, as nohup does on POSIX.
+ if !strings.Contains(got, "Start-Process") {
+ t.Error("the sampler must detach; a background job dies with the shell that created it")
+ }
+ if strings.Contains(got, "Start-Job") {
+ t.Error("Start-Job does not outlive its parent shell, so the sampler would never observe the session")
+ }
+ if !strings.HasSuffix(strings.TrimSpace(got), "'ARGV_SAMPLER_STARTED'") {
+ t.Error("the starter must announce itself, or a sampler that never ran looks like one that found nothing")
+ }
+
+ body := decodedSamplerBody(t, got)
+ if !strings.Contains(body, "while ($n -lt") || !strings.Contains(body, "Start-Sleep") {
+ t.Error("a single sample cannot observe a live session; it must loop")
+ }
+ // Win32_Process.CommandLine is the only place another process's full argv is readable on
+ // Windows -- there is no /proc and no ps.
+ if !strings.Contains(body, "Win32_Process") || !strings.Contains(body, ".CommandLine") {
+ t.Errorf("the sampler must read command lines from Win32_Process:\n%s", body)
+ }
+ for _, want := range []string{"saw_agent=$sawAgent", "ARGV_CLEAN", "ARGV_LEAK"} {
+ if !strings.Contains(body, want) {
+ t.Errorf("sampler body is missing %q:\n%s", want, body)
+ }
+ }
+}
+
+// The matcher must never carry the key in its own command line, or the scan finds itself and
+// reports a leak the tool created. On POSIX that meant awk reading ENVIRON; here it means
+// String.Contains against a value read from $env, plus skipping its own PID.
+func TestWindowsArgvSamplerNeverPutsTheKeyInACommandLine(t *testing.T) {
+ got := powerShell{}.ArgvSampler()
+ body := decodedSamplerBody(t, got)
+
+ if !strings.Contains(body, "$key = $env:ANTHROPIC_API_KEY") {
+ t.Error("the key must be read from the environment inside the sampler")
+ }
+ if !strings.Contains(body, "$cl.Contains($key)") {
+ t.Errorf("the comparison must be a method call so the key never reaches an argument list:\n%s", body)
+ }
+ // Select-String and findstr both take the pattern as an argument, which is the self-poisoning
+ // shape: the sampler's own process table entry would then contain the key.
+ for _, bad := range []string{"Select-String", "findstr"} {
+ if strings.Contains(body, bad) {
+ t.Errorf("%s takes the pattern as an argument, which puts the key in argv", bad)
+ }
+ }
+ if !strings.Contains(body, "$p.ProcessId -eq $me") {
+ t.Error("the sampler must skip its own process, the second guard against finding itself")
+ }
+}
+
+// With no key there is nothing to search for, so the sampler must say so rather than write a clean
+// verdict. A check that could not run must never read as one that passed.
+func TestWindowsArgvSamplerReportsInvalidRatherThanCleanWithoutAKey(t *testing.T) {
+ got := powerShell{}.ArgvSampler()
+
+ if !strings.Contains(got, "ARGV_CHECK_INVALID_NO_KEY") {
+ t.Error("a missing key must produce an explicit invalid marker")
+ }
+ // The guard has to come first in the starter, before anything that could emit a clean verdict.
+ guard := strings.Index(got, "if (-not $env:ANTHROPIC_API_KEY)")
+ start := strings.Index(got, "Start-Process")
+ if guard < 0 || start < 0 || guard > start {
+ t.Error("the no-key guard must precede launching the sampler")
+ }
+ body := decodedSamplerBody(t, got)
+ bodyGuard := strings.Index(body, "if (-not $key)")
+ bodyClean := strings.Index(body, "ARGV_CLEAN")
+ if bodyGuard < 0 || bodyClean < 0 || bodyGuard > bodyClean {
+ t.Error("the sampler body must check for the key before any path that can emit ARGV_CLEAN")
+ }
+}
+
+// The session's inner command travels as base64 rather than as a nested quoted string. The prompt
+// carries a canary and arbitrary punctuation, and surviving two levels of PowerShell quoting is
+// how a prompt arrives subtly altered -- which is indistinguishable from the agent ignoring it.
+func TestWindowsSessionScriptCarriesThePromptOutOfBandAndPropagatesStatus(t *testing.T) {
+ sh := powerShell{}
+ prompt := `write 'BEACON_SANDBOX_abc' to $out -- 100% now`
+ got := sh.CIExecSession(`C:\work\runtime.jsonl`, `C:\work`,
+ claudeFlags(stubScenario(), prompt, sh))
+
+ if strings.Contains(got, prompt) {
+ t.Error("the prompt must not appear inline; it should be base64 in the staged inner script")
+ }
+ inner := decodeFirstBase64(t, got)
+ // The prompt must survive quoting unchanged, which is the property that matters: an altered
+ // prompt is indistinguishable from the agent ignoring instructions. Checked by undoing
+ // PowerShell's only single-quote escape and requiring the original back.
+ if !strings.Contains(inner, sh.Quote(prompt)) {
+ t.Errorf("the staged inner script must carry the quoted prompt:\n%s", inner)
+ }
+ if roundTripped := strings.ReplaceAll(sh.Quote(prompt), "''", "'"); roundTripped != "'"+prompt+"'" {
+ t.Errorf("quoting does not round-trip: %q became %q", prompt, roundTripped)
+ }
+ // Set-Location is load-bearing: the redirects are relative and the process that runs this file
+ // is a grandchild of the shell that wrote it, so its working directory cannot be assumed.
+ if !strings.Contains(inner, "Set-Location -LiteralPath 'C:\\work'") {
+ t.Errorf("the inner script must set its own working directory:\n%s", inner)
+ }
+ if !strings.Contains(inner, "> claude-out.json 2> claude-err.txt") {
+ t.Error("the agent's stdout must be redirected inside the collector wrapper, not outside it")
+ }
+ // Same invariant as the POSIX side: the script exits with ci exec's status, and the diagnostic
+ // tail runs before that exit so its output is still captured.
+ if !strings.Contains(got, "$rc = $LASTEXITCODE") || !strings.Contains(got, "exit $rc") {
+ t.Errorf("the script must capture and re-raise ci exec's status:\n%s", got)
+ }
+ if strings.Index(got, "exit $rc") < strings.Index(got, "ReadAllBytes('ci-out.txt')") {
+ t.Error("the tail should run before the exit so its output is still captured")
+ }
+}
+
+// $LASTEXITCODE is $null when no native command has run yet, and `exit $null` exits 0 -- which
+// would report a session that never launched as a success.
+func TestWindowsSessionScriptsGuardAgainstANullExitCode(t *testing.T) {
+ sh := powerShell{}
+ for name, script := range map[string]string{
+ "CIExecSession": sh.CIExecSession("log", `C:\work`, []string{"-p", "'hi'"}),
+ "PlainSession": sh.PlainSession(`C:\work`, []string{"-p", "'hi'"}),
+ } {
+ if !strings.Contains(script, "if ($null -eq $rc) { $rc = 0 }") {
+ t.Errorf("%s must normalize a null $LASTEXITCODE:\n%s", name, script)
+ }
+ }
+}
+
+// The generated PowerShell is assembled in Go, so a syntax error in it is invisible to the
+// compiler and would surface as "the check did not run" -- which a verdict reports as unverified
+// rather than failing loudly. Parsed with PowerShell's own parser when one is available.
+func TestGeneratedPowerShellParses(t *testing.T) {
+ pwsh, err := exec.LookPath("pwsh")
+ if err != nil {
+ t.Skip("no pwsh on PATH; the windows-sandbox CI job covers this")
+ }
+ sh := powerShell{}
+ scripts := map[string]string{
+ "ArgvSampler": sh.ArgvSampler(),
+ "SamplerBody": decodedSamplerBody(t, sh.ArgvSampler()),
+ "CIExecSession": sh.CIExecSession(`C:\work\runtime.jsonl`, `C:\work`, claudeFlags(stubScenario(), "hi", sh)),
+ "PlainSession": sh.PlainSession(`C:\work`, claudeFlags(stubScenario(), "hi", sh)),
+ "ProbeFile": sh.ProbeFile(`C:\work\out.txt`),
+ "FileSize": sh.FileSize(`C:\work\out.txt`),
+ "ReadFile": sh.ReadFile(`C:\work\out.txt`),
+ "TailBytes": sh.TailBytes(`C:\work\out.txt`, 400),
+ "VersionProbe": versionProbe(sandbox.PlatformWindows),
+ }
+ for name, script := range scripts {
+ path := filepath.Join(t.TempDir(), name+".ps1")
+ if err := os.WriteFile(path, []byte(script), 0o600); err != nil {
+ t.Fatal(err)
+ }
+ // Parse without executing: the parser reports syntax errors and nothing runs.
+ check := `$errors = $null
+[void][System.Management.Automation.Language.Parser]::ParseFile('` + path + `', [ref]$null, [ref]$errors)
+if ($errors) { $errors | ForEach-Object { $_.Message }; exit 1 }`
+ if out, err := exec.Command(pwsh, "-NoProfile", "-NonInteractive", "-Command", check).CombinedOutput(); err != nil {
+ t.Errorf("generated %s is not valid PowerShell: %v\n%s\n--- script ---\n%s",
+ name, err, out, script)
+ }
+ }
+}
+
+// stubScenario is a minimal scenario for script-shape tests.
+func stubScenario() scenario.Scenario {
+ return scenario.Scenario{ID: "w00-probe", Prompt: "unused; the prompt is passed separately"}
+}
+
+// decodedSamplerBody extracts and decodes the staged sampler script.
+func decodedSamplerBody(t *testing.T, starter string) string {
+ t.Helper()
+ return decodeLastBase64(t, starter)
+}
+
+var base64Literal = regexp.MustCompile(`FromBase64String\('([A-Za-z0-9+/=]+)'\)`)
+
+func decodeFirstBase64(t *testing.T, script string) string {
+ t.Helper()
+ m := base64Literal.FindStringSubmatch(script)
+ if m == nil {
+ t.Fatalf("no base64 payload found in:\n%s", script)
+ }
+ return decode(t, m[1])
+}
+
+func decodeLastBase64(t *testing.T, script string) string {
+ t.Helper()
+ all := base64Literal.FindAllStringSubmatch(script, -1)
+ if len(all) == 0 {
+ t.Fatalf("no base64 payload found in:\n%s", script)
+ }
+ return decode(t, all[len(all)-1][1])
+}
+
+func decode(t *testing.T, enc string) string {
+ t.Helper()
+ b, err := base64.StdEncoding.DecodeString(enc)
+ if err != nil {
+ t.Fatalf("payload is not valid base64: %v", err)
+ }
+ return string(b)
+}
diff --git a/beacon-sandbox/sandbox/localexec.go b/beacon-sandbox/sandbox/localexec.go
new file mode 100644
index 00000000..71405545
--- /dev/null
+++ b/beacon-sandbox/sandbox/localexec.go
@@ -0,0 +1,298 @@
+package sandbox
+
+import (
+ "context"
+ "errors"
+ "fmt"
+ "io"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "runtime"
+ "strings"
+)
+
+// LocalExec is the Provider for "the machine this process is already on".
+//
+// It exists because Modal has no Windows, and the alternatives all cost something worse than
+// this does. A GitHub Actions Windows runner is already a disposable machine with a real
+// Service Control Manager, administrator rights and no UAC -- everything an endpoint install
+// needs. So rather than rent an instance from inside it, the runner *is* the instance, and
+// this backend just runs commands where it stands.
+//
+// The consequence is the important part. Every other Provider hands back a separate machine,
+// which is what lets hostguard prove Beacon was never installed outside the sandbox. Here
+// guest and host are the same machine, so that proof has to come from somewhere else -- see
+// MutatesHost and requireDisposable.
+type LocalExec struct {
+ // shell is the interpreter Exec hands scripts to.
+ shell string
+ shellArgs []string
+ platform Platform
+ // workdir is the base directory instances report; scripts may cd elsewhere.
+ workdir string
+}
+
+// LocalExecOptions configures the backend.
+type LocalExecOptions struct {
+ // Workdir is the base directory for launched instances. Empty uses the process's own.
+ Workdir string
+ // AllowHostMutation runs scenarios on a machine this tool cannot prove is disposable.
+ //
+ // Off by default, and the default is the safe one: a scenario installs Beacon for real,
+ // registers services and rewrites agent-runtime settings files. On a workstation that is
+ // not a test, it is damage. Contributors who genuinely want this -- a throwaway VM they
+ // own -- have to say so.
+ AllowHostMutation bool
+}
+
+type localImage struct{ ref string }
+
+func (l localImage) Ref() string { return l.ref }
+
+type localInstance struct{ id string }
+
+func (l localInstance) ID() string { return l.id }
+
+// NewLocalExec returns a backend for the current machine, refusing unless the machine is
+// provably disposable or the caller has explicitly accepted the risk.
+func NewLocalExec(opts LocalExecOptions) (*LocalExec, error) {
+ if err := requireDisposable(opts.AllowHostMutation); err != nil {
+ return nil, err
+ }
+ l := &LocalExec{workdir: opts.Workdir}
+ if runtime.GOOS == "windows" {
+ l.platform = PlatformWindows
+ // PowerShell 7, and not Windows PowerShell 5.1 as a fallback.
+ //
+ // 5.1's `>` writes UTF-16LE, which would silently corrupt every JSON artifact this tool
+ // parses -- claude-out.json above all, the file that says whether the session succeeded.
+ // A quiet encoding difference producing "the agent failed" is worse than refusing here,
+ // and pwsh is what the runner images ship and what Claude Code itself uses on Windows.
+ if _, err := exec.LookPath("pwsh"); err != nil {
+ return nil, fmt.Errorf("pwsh (PowerShell 7+) is required on Windows and is not on " +
+ "PATH; Windows PowerShell 5.1 redirects output as UTF-16, which corrupts the " +
+ "JSON artifacts this tool reads")
+ }
+ l.shell = "pwsh"
+ // -NoProfile matters for reproducibility: a profile can alter PATH, aliases and the
+ // error-action preference, so a run would otherwise depend on the host's dotfiles.
+ l.shellArgs = []string{"-NoProfile", "-NonInteractive", "-Command"}
+ return l, nil
+ }
+ l.platform = PlatformPosix
+ l.shell = "bash"
+ if _, err := exec.LookPath("bash"); err != nil {
+ l.shell = "sh"
+ }
+ l.shellArgs = []string{"-c"}
+ return l, nil
+}
+
+// requireDisposable refuses to run on a machine whose destruction is not somebody else's job.
+//
+// GITHUB_ACTIONS alone is not enough. A self-hosted runner is a real machine that persists
+// between jobs, so an install performed there survives the run and the next job inherits it --
+// which is exactly the escape hostguard exists to catch, just relocated. RUNNER_ENVIRONMENT is
+// GitHub's own answer to the question, so it is what gets asked.
+func requireDisposable(allow bool) error {
+ if allow {
+ return nil
+ }
+ if os.Getenv("GITHUB_ACTIONS") != "true" {
+ return fmt.Errorf("the local provider runs scenarios on this machine, and a scenario " +
+ "installs Beacon, registers services and rewrites agent-runtime settings for real; " +
+ "it is meant for a disposable CI runner. Use `--provider github` to dispatch a run to " +
+ "one, or pass --allow-host-mutation if this machine is genuinely throwaway")
+ }
+ if env := os.Getenv("RUNNER_ENVIRONMENT"); env != "" && env != "github-hosted" {
+ return fmt.Errorf("this is a %s Actions runner, which persists between jobs, so an "+
+ "install performed here would outlive the run; pass --allow-host-mutation only if "+
+ "this runner is genuinely disposable", env)
+ }
+ return nil
+}
+
+// DisposabilityEvidence describes why this machine was accepted, for run metadata.
+//
+// Recorded rather than implied. hostguard's comparison cannot mean anything here, so the
+// verdict has to say what it relied on instead -- and "the operator asserted it" is a
+// materially weaker claim than "GitHub hands out a fresh VM per job", which a reader of the
+// verdict is entitled to tell apart.
+func DisposabilityEvidence() string {
+ if os.Getenv("GITHUB_ACTIONS") == "true" {
+ env := os.Getenv("RUNNER_ENVIRONMENT")
+ if env == "" {
+ env = "unreported"
+ }
+ if env == "github-hosted" {
+ return "github-hosted ephemeral runner (fresh VM per job)"
+ }
+ return "GitHub Actions runner, environment " + env + ", accepted by explicit operator opt-in"
+ }
+ return "operator asserted this machine is disposable (--allow-host-mutation)"
+}
+
+func (l *LocalExec) Name() string { return "local" }
+
+func (l *LocalExec) Platform() Platform { return l.platform }
+
+// MutatesHost is true, and saying so is the whole point of the flag: the runner reads it and
+// swaps hostguard's comparison for a disposability assertion instead of reporting a false
+// escape.
+func (l *LocalExec) MutatesHost() bool { return true }
+
+// EnsureImage is a no-op. Provisioning happened before this process started -- the CI job
+// installed the agent runtime and built the binaries under test -- so there is no image to
+// build and nothing to cache. The spec's layers are reported in the ref so a verdict still
+// records what environment was expected.
+func (l *LocalExec) EnsureImage(_ context.Context, spec ImageSpec) (Image, error) {
+ base := spec.Base
+ if base == "" {
+ base = runtime.GOOS + "/" + runtime.GOARCH
+ }
+ return localImage{ref: "local:" + base}, nil
+}
+
+// Snapshot is not available: there is no instance boundary to snapshot across. Returning an
+// error rather than a silent no-op keeps a caller that depends on snapshotting from believing
+// it got one.
+func (l *LocalExec) Snapshot(_ context.Context, _ Instance) (Image, error) {
+ return nil, fmt.Errorf("the local provider cannot snapshot: the guest is this machine")
+}
+
+// Launch records the workdir and returns a handle. Nothing starts, because the machine is
+// already running.
+func (l *LocalExec) Launch(_ context.Context, _ Image, spec LaunchSpec) (Instance, error) {
+ if spec.Workdir != "" {
+ l.workdir = spec.Workdir
+ }
+ // Secrets are deliberately not applied to this process's environment. They are already
+ // present here -- the CI job injected them -- and mutating the parent's environment would
+ // leak them into every later command, including ones a scenario never meant to see them.
+ host, _ := os.Hostname()
+ if host == "" {
+ host = "localhost"
+ }
+ return localInstance{id: fmt.Sprintf("local:%s:%d", host, os.Getpid())}, nil
+}
+
+// Exec runs a script through the platform's shell.
+func (l *LocalExec) Exec(ctx context.Context, _ Instance, script string, opts ExecOpts) (Result, error) {
+ // User switching is not supported: this process cannot become another user without
+ // privileges it has no business holding, and su/runas would need a password. A scenario
+ // that needs an unprivileged account must have the CI job create one and run the whole
+ // tool as them. Reported rather than ignored, because silently running as the wrong user
+ // is how a scenario ends up verifying root's unused settings file.
+ if opts.User != "" && !l.currentUserIs(opts.User) {
+ return Result{}, fmt.Errorf("the local provider cannot switch to user %q; run the "+
+ "harness as that user instead", opts.User)
+ }
+ if opts.Timeout > 0 {
+ var cancel context.CancelFunc
+ ctx, cancel = context.WithTimeout(ctx, opts.Timeout)
+ defer cancel()
+ }
+
+ cmd := exec.CommandContext(ctx, l.shell, append(l.shellArgs, script)...)
+ cmd.Dir = l.resolveDir(opts.Dir)
+ cmd.Env = l.execEnv(opts)
+
+ var stdout, stderr strings.Builder
+ cmd.Stdout = &stdout
+ cmd.Stderr = &stderr
+ err := cmd.Run()
+
+ res := Result{Stdout: stdout.String(), Stderr: stderr.String()}
+ // An ExitError carries the status and is not a failure of Exec itself; every caller reads
+ // ExitCode and decides what it means. Anything else -- the shell missing, a timeout -- is a
+ // harness failure and must surface as one, or "the command failed" and "we never ran it"
+ // become the same result.
+ if err != nil {
+ var ee *exec.ExitError
+ if errors.As(err, &ee) {
+ res.ExitCode = ee.ExitCode()
+ return res, nil
+ }
+ return res, fmt.Errorf("exec %s: %w", l.shell, err)
+ }
+ return res, nil
+}
+
+// execEnv builds the child environment, honoring the same ExecOpts fields the remote backends do.
+func (l *LocalExec) execEnv(opts ExecOpts) []string {
+ env := os.Environ()
+ if opts.HomeDir != "" {
+ env = append(env, "HOME="+opts.HomeDir)
+ if l.platform.IsWindows() {
+ // Windows programs read USERPROFILE, not HOME. Setting only HOME would leave a
+ // scenario's home-relative paths pointing at the real profile.
+ env = append(env, "USERPROFILE="+opts.HomeDir)
+ }
+ }
+ if len(opts.PathPrepend) > 0 {
+ sep := ":"
+ if l.platform.IsWindows() {
+ sep = ";"
+ }
+ env = append(env, "PATH="+strings.Join(opts.PathPrepend, sep)+sep+os.Getenv("PATH"))
+ }
+ return env
+}
+
+func (l *LocalExec) resolveDir(dir string) string {
+ switch {
+ case dir != "":
+ return dir
+ case l.workdir != "":
+ return l.workdir
+ default:
+ return ""
+ }
+}
+
+// currentUserIs reports whether the requested user is the one already running.
+func (l *LocalExec) currentUserIs(want string) bool {
+ for _, key := range []string{"USER", "USERNAME", "LOGNAME"} {
+ if v := os.Getenv(key); v != "" && strings.EqualFold(v, want) {
+ return true
+ }
+ }
+ return false
+}
+
+// Put and Get are local copies. They keep the Provider shape so the runner does not care
+// which backend it has, and they still cross a boundary worth crossing: run artifacts land in
+// the run directory rather than being read from wherever the scenario left them.
+func (l *LocalExec) Put(_ context.Context, _ Instance, localPath, remotePath string) error {
+ return copyFile(localPath, remotePath)
+}
+
+func (l *LocalExec) Get(_ context.Context, _ Instance, remotePath, localPath string) error {
+ return copyFile(remotePath, localPath)
+}
+
+// Terminate is a no-op: destroying this machine is the CI job's business, not ours.
+func (l *LocalExec) Terminate(_ context.Context, _ Instance) error { return nil }
+
+func copyFile(src, dst string) error {
+ in, err := os.Open(src)
+ if err != nil {
+ return err
+ }
+ defer in.Close()
+ if err := os.MkdirAll(filepath.Dir(dst), 0o700); err != nil {
+ return err
+ }
+ // 0600, matching the run directory: collected artifacts retain prompt text and command
+ // output, and on a shared machine a wider mode hands that to every local user.
+ out, err := os.OpenFile(dst, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o600)
+ if err != nil {
+ return err
+ }
+ if _, err := io.Copy(out, in); err != nil {
+ out.Close()
+ return err
+ }
+ return out.Close()
+}
diff --git a/beacon-sandbox/sandbox/modal.go b/beacon-sandbox/sandbox/modal.go
index b1aa82f2..0aaa8b67 100644
--- a/beacon-sandbox/sandbox/modal.go
+++ b/beacon-sandbox/sandbox/modal.go
@@ -57,6 +57,14 @@ func (m *Modal) Close() {
func (m *Modal) Name() string { return "modal" }
+// Platform is always POSIX: Modal offers Linux only, which is the whole reason a second
+// backend exists.
+func (m *Modal) Platform() Platform { return PlatformPosix }
+
+// MutatesHost is false -- a Modal sandbox is a different machine, so hostguard's comparison
+// means what it says.
+func (m *Modal) MutatesHost() bool { return false }
+
func (m *Modal) EnsureImage(ctx context.Context, spec ImageSpec) (Image, error) {
img := m.client.Images.FromRegistry(spec.Base, nil)
if len(spec.Layers) > 0 {
diff --git a/beacon-sandbox/sandbox/provider.go b/beacon-sandbox/sandbox/provider.go
index 6bf5073f..93548b21 100644
--- a/beacon-sandbox/sandbox/provider.go
+++ b/beacon-sandbox/sandbox/provider.go
@@ -20,6 +20,23 @@ const (
LaneVM Lane = "vm"
)
+// Platform names the guest operating-system family.
+//
+// It decides two things that cannot be guessed from anything else: which dialect Exec
+// interprets a script as, and which path syntax a scenario has to speak. Everything above
+// the provider branches on this rather than on the *host's* runtime.GOOS, because those are
+// not the same question -- a Mac dispatching a Windows run is the normal case.
+type Platform string
+
+const (
+ PlatformPosix Platform = "posix"
+ PlatformWindows Platform = "windows"
+)
+
+// IsWindows is a readability shorthand for the many call sites that only care which side of
+// the split they are on.
+func (p Platform) IsWindows() bool { return p == PlatformWindows }
+
// ImageSpec describes a reproducible environment. Layers are ordered and cached by the
// provider, so an unchanged prefix is free on re-run.
type ImageSpec struct {
@@ -102,6 +119,20 @@ type Provider interface {
// Name identifies the backend in run metadata.
Name() string
+ // Platform reports the guest OS family, so callers know which script dialect Exec
+ // accepts and which path syntax to build.
+ Platform() Platform
+
+ // MutatesHost reports whether this backend runs the scenario on the same machine the
+ // caller is on, rather than in a separate instance.
+ //
+ // This is not a detail: hostguard exists to prove Beacon was never installed outside the
+ // sandbox, and it compares the *caller's* machine before and after. When guest and host
+ // are the same machine that comparison inverts -- a correct install trips the most serious
+ // finding this tool has. A backend that says true is telling the runner to assert
+ // disposability some other way instead of silently skipping the check.
+ MutatesHost() bool
+
// EnsureImage builds or resolves an environment. Implementations should be
// idempotent and lean on content-addressed caching.
EnsureImage(ctx context.Context, spec ImageSpec) (Image, error)
diff --git a/beacon-sandbox/scenario/scenario.go b/beacon-sandbox/scenario/scenario.go
index 812182ff..aecb3313 100644
--- a/beacon-sandbox/scenario/scenario.go
+++ b/beacon-sandbox/scenario/scenario.go
@@ -67,10 +67,27 @@ type Install struct {
NeedsRealSystemd bool `yaml:"needs_real_systemd,omitempty"`
}
+// Platform selects the guest OS family a scenario is written for.
+//
+// A scenario is not portable by default and pretending otherwise would be worse than declaring
+// it: the prompt names shell commands, the sentinel is an absolute path, and the expectations
+// assert what Beacon captured from a particular runtime. So a scenario says which world it
+// belongs to, and the suite loader runs only the ones that match the target.
+type Platform string
+
+const (
+ // PlatformLinux is the default, so every scenario written before Windows existed keeps
+ // working unchanged.
+ PlatformLinux Platform = "linux"
+ PlatformWindows Platform = "windows"
+)
+
// Scenario is one declarative case.
type Scenario struct {
- ID string `yaml:"id"`
- Prompt string `yaml:"prompt"`
+ ID string `yaml:"id"`
+ // Platform defaults to linux when unset.
+ Platform Platform `yaml:"platform,omitempty"`
+ Prompt string `yaml:"prompt"`
// Install, when set, performs a persistent endpoint install and runs the session against
// it, with no `beacon ci exec` wrapper anywhere.
Install *Install `yaml:"install,omitempty"`
@@ -101,6 +118,14 @@ type Suite struct {
Scenarios []Scenario
}
+// TargetPlatform resolves the platform with its default.
+func (s Scenario) TargetPlatform() Platform {
+ if s.Platform == "" {
+ return PlatformLinux
+ }
+ return s.Platform
+}
+
// Timeout resolves the session bound with a sane default.
func (s Scenario) Timeout() time.Duration {
if s.TimeoutSeconds > 0 {
@@ -128,6 +153,27 @@ func (s Scenario) Validate() error {
if len(s.Expect) == 0 {
return fmt.Errorf("%s: at least one expectation is required, else the run asserts nothing", s.ID)
}
+ switch s.TargetPlatform() {
+ case PlatformLinux, PlatformWindows:
+ default:
+ return fmt.Errorf("%s: platform %q must be linux or windows", s.ID, s.Platform)
+ }
+ if s.TargetPlatform() == PlatformWindows {
+ // systemd does not exist on Windows, and the nested-container recipe that provides it is
+ // a Linux arrangement. A scenario asking for both would be arranged as neither and would
+ // then quietly verify whatever backend happened to be selected.
+ if s.NeedsRealSystemd() {
+ return fmt.Errorf("%s: needs_real_systemd cannot apply to a windows scenario; "+
+ "the Windows service manager is the SCM, which is always present", s.ID)
+ }
+ // An absolute POSIX sentinel on Windows is the mistake that would be hardest to notice:
+ // the agent writes somewhere, the probe looks somewhere else, and the run reports the
+ // agent idle rather than the scenario being wrong.
+ if strings.HasPrefix(s.Sentinel, "/") {
+ return fmt.Errorf("%s: sentinel %q is a POSIX path in a windows scenario; use "+
+ "{{workdir}} or a drive-qualified path", s.ID, s.Sentinel)
+ }
+ }
for i, e := range s.Expect {
if strings.TrimSpace(e.Action) == "" {
return fmt.Errorf("%s: expect[%d] is missing action", s.ID, i)
@@ -151,16 +197,16 @@ func (s Scenario) Validate() error {
"container has no logind session to attach to", s.ID)
}
switch s.Install.ExpectServiceKind {
- case "", "systemd", "launchd", "none":
+ case "", "systemd", "launchd", "windows-service", "none":
default:
- return fmt.Errorf("%s: install.expect_service_kind %q must be systemd, launchd or none",
- s.ID, s.Install.ExpectServiceKind)
+ return fmt.Errorf("%s: install.expect_service_kind %q must be systemd, launchd, "+
+ "windows-service or none", s.ID, s.Install.ExpectServiceKind)
}
switch s.Install.Service {
- case "", "auto", "systemd", "launchd", "none":
+ case "", "auto", "systemd", "launchd", "windows-service", "none":
default:
- return fmt.Errorf("%s: install.service %q must be auto, systemd, launchd, or none",
- s.ID, s.Install.Service)
+ return fmt.Errorf("%s: install.service %q must be auto, systemd, launchd, "+
+ "windows-service, or none", s.ID, s.Install.Service)
}
}
@@ -210,6 +256,10 @@ func Load(path string) (Scenario, error) {
}
// LoadSuite reads every *.yaml in dir, sorted by filename so ordering is deterministic.
+//
+// Every scenario is validated, including ones this platform will not run: a syntax error or a
+// stale field in a Windows scenario should fail a Linux contributor's `go test`, not wait for
+// somebody to dispatch a Windows run.
func LoadSuite(dir string) (Suite, error) {
paths, err := filepath.Glob(filepath.Join(dir, "*.yaml"))
if err != nil {
@@ -229,6 +279,23 @@ func LoadSuite(dir string) (Suite, error) {
return suite, nil
}
+// For returns the scenarios written for one platform.
+//
+// Reports how many were set aside rather than silently narrowing the suite. `run` with no
+// --scenario is the "everything" command, and a caller who asked for everything and got two
+// thirds of it is entitled to know -- silent truncation reads as full coverage.
+func (s Suite) For(p Platform) (matching Suite, skipped int) {
+ out := Suite{Name: s.Name}
+ for _, sc := range s.Scenarios {
+ if sc.TargetPlatform() == p {
+ out.Scenarios = append(out.Scenarios, sc)
+ continue
+ }
+ skipped++
+ }
+ return out, skipped
+}
+
// Expand substitutes run-scoped placeholders into a template string.
func Expand(tmpl string, canary, sentinel, workdir string) string {
r := strings.NewReplacer(
diff --git a/beacon-sandbox/scenario/scenario_test.go b/beacon-sandbox/scenario/scenario_test.go
index 641423b7..867c64fe 100644
--- a/beacon-sandbox/scenario/scenario_test.go
+++ b/beacon-sandbox/scenario/scenario_test.go
@@ -348,6 +348,71 @@ func TestInstallModeAndServiceAreValidated(t *testing.T) {
}
}
+// An unset platform must keep meaning Linux, or every scenario written before Windows existed
+// would silently stop being selected by the suite loader.
+func TestPlatformDefaultsToLinux(t *testing.T) {
+ if got := (Scenario{ID: "s"}).TargetPlatform(); got != PlatformLinux {
+ t.Errorf("an unset platform must default to linux, got %q", got)
+ }
+ if got := (Scenario{ID: "s", Platform: PlatformWindows}).TargetPlatform(); got != PlatformWindows {
+ t.Errorf("an explicit platform must be honored, got %q", got)
+ }
+}
+
+// A POSIX sentinel in a Windows scenario is the mistake hardest to notice at runtime: the agent
+// writes somewhere, the probe looks somewhere else, and the run reports the agent idle rather
+// than the scenario being wrong. So it is rejected at authoring time, before a runner is spent.
+func TestWindowsScenariosRejectPosixSentinelsAndSystemd(t *testing.T) {
+ base := func(mutate func(*Scenario)) Scenario {
+ s := Scenario{ID: "w", Platform: PlatformWindows, Prompt: "do a thing",
+ Expect: []Expect{{Action: "prompt.submitted", Why: "baseline"}}}
+ mutate(&s)
+ return s
+ }
+ if err := base(func(s *Scenario) { s.Sentinel = "/home/agent/work/out.txt" }).Validate(); err == nil {
+ t.Error("a POSIX absolute sentinel must be rejected in a windows scenario")
+ }
+ if err := base(func(s *Scenario) {
+ s.Install = &Install{Mode: "system", NeedsRealSystemd: true}
+ }).Validate(); err == nil {
+ t.Error("needs_real_systemd must be rejected in a windows scenario")
+ }
+ // The spellings a Windows scenario legitimately uses.
+ for _, sentinel := range []string{`{{workdir}}\out.txt`, `C:\work\out.txt`} {
+ if err := base(func(s *Scenario) { s.Sentinel = sentinel }).Validate(); err != nil {
+ t.Errorf("sentinel %q must be accepted: %v", sentinel, err)
+ }
+ }
+ // An unknown platform is a typo, and a typo that defaulted to linux would run a Windows
+ // scenario's PowerShell prompt against bash.
+ if err := (Scenario{ID: "w", Platform: "win32", Prompt: "x",
+ Expect: []Expect{{Action: "a", Why: "b"}}}).Validate(); err == nil {
+ t.Error("an unrecognized platform must be rejected rather than defaulting")
+ }
+}
+
+// The suite loader must report what it set aside. `run` with no --scenario is the "everything"
+// command, so returning a subset of it silently would read as full coverage.
+func TestSuiteForReportsWhatItSkipped(t *testing.T) {
+ suite := Suite{Scenarios: []Scenario{
+ {ID: "s01"},
+ {ID: "w00", Platform: PlatformWindows},
+ {ID: "s02", Platform: PlatformLinux},
+ }}
+
+ linux, skipped := suite.For(PlatformLinux)
+ if len(linux.Scenarios) != 2 || skipped != 1 {
+ t.Errorf("linux: got %d scenarios and %d skipped, want 2 and 1", len(linux.Scenarios), skipped)
+ }
+ win, skipped := suite.For(PlatformWindows)
+ if len(win.Scenarios) != 1 || skipped != 2 {
+ t.Errorf("windows: got %d scenarios and %d skipped, want 1 and 2", len(win.Scenarios), skipped)
+ }
+ if win.Scenarios[0].ID != "w00" {
+ t.Errorf("wrong scenario selected: %q", win.Scenarios[0].ID)
+ }
+}
+
// A systemd requirement combined with the supervised backend, or without system mode, would
// silently exercise something other than what the scenario claims — and the run would pass,
// reporting coverage it does not have.
diff --git a/beacon-sandbox/scenarios/w00-probe.yaml b/beacon-sandbox/scenarios/w00-probe.yaml
new file mode 100644
index 00000000..80b71157
--- /dev/null
+++ b/beacon-sandbox/scenarios/w00-probe.yaml
@@ -0,0 +1,59 @@
+id: w00-probe
+platform: windows
+# The first Windows scenario, and its job is to answer questions rather than to guard behavior.
+#
+# Beacon has no Windows support yet -- the CLI does not even compile for it at the time this was
+# written -- so this scenario runs against the `beacon ci exec` collector path and asks the four
+# things the rest of the Windows port needs settled before it can be designed. Every answer is
+# read out of the collected artifacts and the run log:
+#
+# 1. Will Claude Code run `-p --dangerously-skip-permissions` as an *elevated* user? It refuses
+# as root on POSIX, and a GitHub Actions Windows runner is an administrator with UAC
+# disabled. If it refuses, meta.claude_stderr says so and the harness needs a standard
+# account to run sessions as -- the Windows analogue of the `agent` user in the Linux image.
+# 2. Does `beacon ci exec` supervise a PowerShell child correctly on Windows, and does the
+# collector it starts write the runtime log? meta.ci_exec_rc and the collected runtime.jsonl
+# answer that; internal/ci/session.go already carries Windows branches, so this is checking
+# an existing claim rather than a new one.
+# 3. Does the capture pipeline populate the same fields it does on Linux once the paths are
+# Windows paths? The expectations below are the Linux ones, deliberately unchanged.
+# 4. Does the argv credential sampler observe anything through Win32_Process? meta.argv_samples
+# and meta.argv_saw_agent report it, and a clean-but-never-saw-the-agent result is recorded
+# as unverified rather than clean.
+#
+# The command is deliberately PowerShell rather than bash: the agent's default shell on Windows
+# is PowerShell, and asking it for a bash command would test Git-for-Windows' presence instead
+# of Beacon.
+prompt: >-
+ Run exactly this PowerShell command and then report its output verbatim:
+ Write-Output '{{canary}}' | Tee-Object -FilePath '{{sentinel}}'
+sentinel: '{{workdir}}\w00.sentinel'
+budget_usd: 1.0
+timeout_seconds: 420
+
+expect:
+ - action: prompt.submitted
+ contains: ["{{canary}}"]
+ why: >-
+ Baseline. If this is absent nothing else below is interpretable, because the session either
+ never reached the model or the collector never received anything -- which is question 2, not
+ a capture gap.
+
+ - action: command.executed
+ contains: ["{{canary}}"]
+ fields: ["command.command", "tool.name"]
+ optional: true
+ why: >-
+ Threat rules match scalar leaves, so e.command.command is the detection surface for the
+ whole rules/risky-command/ category on Windows exactly as it is on Linux. Optional because
+ this probe is establishing whether it works at all -- a gap here is a finding to act on, not
+ a regression to block on, and marking it required would stop the probe reporting the other
+ three answers.
+
+ - action: approval.allowed
+ fields: ["approval.decision"]
+ optional: true
+ why: >-
+ Claude Code's tool_decision carries decision=accept regardless of platform. Optional for the
+ same reason as above: this scenario exists to find out, and the answer belongs in the
+ verdict either way.