A local-first multi-agent app development platform, built on PenguinHarness
fix(ci,plugins): CI 真跑沙盒插件的测试,跑不起来就报错
lmliheng0f70719bafa3
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 71117a1f..1d8d7f14 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -130,7 +130,17 @@ jobs:# The unit suite, split four ways. Shards are drawn so that build cost and test cost# roughly balance: core and server are the two long suites and get one each, the rest- # pair up behind them.+ # pair up behind them. `rest` is recursive with exclusions, like macOS and Windows: an+ # explicit list silently drops every package nobody remembered to add (all of plugins/+ # once was, the four sandbox backends among them), whereas a new package lands here by+ # default. Its build list is theirs too, and prepares no sandbox backend: sandbox-bwrap's+ # tests fetch the bubblewrap the plugin ships themselves (its vitest global setup), so CI+ # need not know what that takes.+ #+ # `must_run` is ADELIE_MUST_RUN: the environment-dependent suites (directory names under+ # plugins/) this platform must really run. A named suite whose host probe fails turns the+ # shard red with the probe's reason instead of skipping; an unnamed one skips where it+ # cannot open, as locally.test:name: test (${{ matrix.shard }})runs-on: ubuntu-latest@@ -150,17 +160,27 @@ jobs:pnpm --filter @lmliheng/penguin-web --filter @lmliheng/penguin-ui--filter @lmliheng/penguin-cli test- shard: rest- build: "@lmliheng/penguin-desktop..."+ build: "@lmliheng/penguin-desktop...,@lmliheng/penguin-cli..."tests: >-- pnpm --filter @lmliheng/penguin-desktop- --filter @lmliheng/penguin-docs --filter @lmliheng/penguin-ui-gallery test+ pnpm -r --filter '!@lmliheng/penguin-core' --filter '!@lmliheng/penguin-server'+ --filter '!@lmliheng/penguin-web' --filter '!@lmliheng/penguin-ui'+ --filter '!@lmliheng/penguin-cli' test+ must_run: sandbox-bwrap,sandbox-dshsteps:- uses: actions/checkout@v5- uses: ./.github/actions/setupwith:build: ${{ matrix.build }}+ # Ubuntu 23.10 and later let only AppArmor-profiled programs create an unprivileged+ # user namespace, and some tests here need one. Changing it takes root and changes the+ # whole machine, so it is a host prerequisite the job sets, not something a test can.+ - name: Allow unprivileged user namespaces+ if: matrix.shard == 'rest'+ run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0- name: Unit tests (vitest)run: ${{ matrix.tests }}+ env:+ ADELIE_MUST_RUN: ${{ matrix.must_run }}# The installer bundle and the live-LLM e2e, together because both want a full build and# neither is long. The secret is exposed only in its own step (step-level env), so earlier@@ -213,6 +233,8 @@ jobs:tests: >-pnpm -r --filter '!@lmliheng/penguin-core' --filter '!@lmliheng/penguin-server' testsmoke: true+ # ADELIE_MUST_RUN, as on ubuntu: these suites must run here.+ must_run: sandbox-seatbelt,sandbox-dshsteps:- uses: actions/checkout@v5- uses: ./.github/actions/setup@@ -220,6 +242,8 @@ jobs:build: ${{ matrix.build }}- name: Unit tests (vitest)run: ${{ matrix.tests }}+ env:+ ADELIE_MUST_RUN: ${{ matrix.must_run }}- name: Terminal smoke (node-pty under Electron's Node)if: matrix.smokerun: node packages/desktop/scripts/terminal-smoke.mjsdiff --git a/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.md b/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.mdnew file mode 100644index 00000000..26208991--- /dev/null+++ b/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.md@@ -0,0 +1,37 @@+# CI really runs the sandbox plugins' suites, and a declared one that cannot open turns red++- **Date:** 2026-10-08+- **Type:** fix+- **Scope:** `ci`, `plugins`++[中文版](2026-10-08-ci-ubuntu-runs-plugin-suites.zh.md)++Every sandbox backend has a set of live tests: they really start the sandbox program, have a command+write a file outside the workspace, and assert that the kernel refuses it. These are the only tests+that prove a sandbox works. They first probe once whether the host can open the sandbox, and skip the+whole set if it cannot — running the Linux suite on macOS, skipping is correct. The problem had two+layers: Linux's `rest` shard was a list of names and `plugins/*` was never added to it (the four+sandbox backend packages ran nowhere on Linux CI), and even where they were scheduled, GitHub's+Ubuntu machines do not let ordinary programs create a user namespace by default, so every bwrap live+test would "skip" and CI would stay green — a skip and a pass cannot be told apart. Ported from+upstream PenguinHarness (#872, commit `e3a9eb66`), which the fork did not carry.++- Linux's `rest` shard takes the macOS/Windows form: run the whole repository, minus core, server,+ web, ui and cli, which the other shards already run, with the same build list. A package added+ later lands here by default.+- Each platform's `rest` shard declares the suites it must really run with `ADELIE_MUST_RUN`+ (directory names under `plugins/`, comma-separated, spaces around a name ignored): ubuntu+ `sandbox-bwrap,sandbox-dsh`, macOS `sandbox-seatbelt,sandbox-dsh`. **A declared suite that cannot+ open turns the run red**, carrying the probe's reason; an undeclared one skips as before, so a+ developer machine is unchanged. The check is one function in `scripts/must-run.mjs`.+- bubblewrap is prepared by the plugin's own tests: `sandbox-bwrap`'s vitest `globalSetup` calls+ `scripts/vendor-bwrap.mjs`, so a test run alone (no build) still tests the bwrap users get, and CI+ need not know what that takes. The vendorer's "already in place" now means the marker names the+ current pinning **and** every architecture's `bin/bwrap` exists and is executable — a deleted or+ non-executable binary is laid down again from the cache.+- Ubuntu's user-namespace switch stays in CI as a machine prerequisite (`sysctl -w …=0`), named+ without mentioning the sandbox; on a kernel without that switch the step fails rather than being+ skipped silently.+- `sandbox-bwrap`'s refusal now names Ubuntu's switch beside Debian's: Debian gates unprivileged+ user namespaces with a sysctl, Ubuntu 23.10 and later lets only AppArmor-profiled programs create+ them (24.04's default). That sentence is what an operator sees on the Sandbox card.diff --git a/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.zh.md b/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.zh.mdnew file mode 100644index 00000000..fdc3911f--- /dev/null+++ b/changelog/unreleased/2026-10-08-ci-ubuntu-runs-plugin-suites.zh.md@@ -0,0 +1,29 @@+# CI 真正跑沙盒插件的测试,跑不起来就红++- **Date:** 2026-10-08+- **Type:** fix+- **Scope:** `ci`, `plugins`++[中文版](2026-10-08-ci-ubuntu-runs-plugin-suites.zh.md)++沙盒后端各有一套 live 用例:它们真的启动沙盒程序,让一条命令去写工作区外的文件,并断言内核拒绝它。+这是唯一能证明沙盒起作用的测试。它们先探测一次宿主能否打开沙盒,打不开就整片跳过 —— 在 macOS 上跑+Linux 那套,跳过是对的。问题有两层:Linux 的 `rest` 分片是一串写死的包名,`plugins/*` 从来没被列进去+(4 个沙盒后端包在 Linux CI 上一条都不跑);而即使排上了,GitHub 的 Ubuntu 机器默认不给普通程序建+user namespace,bwrap 的 live 用例会整片「跳过」,CI 依旧全绿 —— 跳过与通过看不出区别。本次从上游+PenguinHarness 移植(#872,提交 `e3a9eb66`),此前本 fork 没有这一条。++- Linux 的 `rest` 分片改成与 macOS / Windows 同一写法:跑全仓,减去 core、server、web、ui、cli+ 这几个已各自分片的包,build 名单也对齐。以后新增的包默认就会被跑到。+- 各平台的 `rest` 分片用 `ADELIE_MUST_RUN` 声明「必须真跑」的套件(按 `plugins/` 下的目录名,+ 逗号分隔,名字两侧空格忽略):ubuntu 上是 `sandbox-bwrap,sandbox-dsh`,macOS 上是+ `sandbox-seatbelt,sandbox-dsh`。**被声明的套件开不了就红**,错误里原样带上探测给的理由;没声明的+ 照旧跳过,开发者机器上的行为不变。判定只有 `scripts/must-run.mjs` 里一个函数。+- bubblewrap 由插件自己的测试准备:`sandbox-bwrap` 的 vitest `globalSetup` 调用+ `scripts/vendor-bwrap.mjs`,所以只跑测试(不构建)也测的是用户拿到的那份 bwrap,CI 不必知道这件事。+ 该脚本的「已就位」判据补上「每个架构的 `bin/bwrap` 存在且可执行」——被删掉或去掉了可执行位的二进制+ 会从缓存重新铺好。+- Ubuntu 的 user namespace 开关留作 CI 的机器前置步骤(`sysctl -w …=0`),与沙盒无关地命名;内核没有+ 这个开关时该步骤直接失败,而不是静默跳过。+- `sandbox-bwrap` 的拒绝理由补上 Ubuntu 的开关:Debian 用 sysctl 限制非特权 user namespace,Ubuntu+ 23.10 及以后只让带 AppArmor profile 的程序创建它们(24.04 的默认)。运维在沙盒卡片上看到的就是这句话。diff --git a/plugins/sandbox-bwrap/src/index.ts b/plugins/sandbox-bwrap/src/index.tsindex 0875e59a..14955547 100644--- a/plugins/sandbox-bwrap/src/index.ts+++ b/plugins/sandbox-bwrap/src/index.ts@@ -222,8 +222,11 @@ export async function loadPenguinBwrapProvider(? internals.probe(probeTimeoutMs, runner): await probeAsync(probeTimeoutMs, runner);if (!usable) {+ // Two distributions, two switches: Debian gates unprivileged user namespaces with a sysctl,+ // Ubuntu 23.10 and later lets only AppArmor-profiled programs create them (24.04's default).+ // Both are named because the reason is all an operator sees on the Sandbox card.throw new Error(- `'${runner}' is missing or refuses the base profile (are unprivileged user namespaces enabled on this host? \`sysctl kernel.unprivileged_userns_clone\`)`,+ `'${runner}' is missing or refuses the base profile (are unprivileged user namespaces allowed on this host? Debian: \`sysctl kernel.unprivileged_userns_clone\`; Ubuntu 23.10 and later: \`sysctl kernel.apparmor_restrict_unprivileged_userns\`)`,);}return createPenguinBwrapProvider(internals);diff --git a/plugins/sandbox-bwrap/test/global-setup.ts b/plugins/sandbox-bwrap/test/global-setup.tsnew file mode 100644index 00000000..b3f6420c--- /dev/null+++ b/plugins/sandbox-bwrap/test/global-setup.ts@@ -0,0 +1,7 @@+// The live suite confines with the bwrap this plugin ships; the build vendors it, a bare test+// run does not. vendorBwrap() returns at once when it is already in place.+import { vendorBwrap } from "../../../scripts/vendor-bwrap.mjs";++export default async function setup(): Promise<void> {+ if (process.platform === "linux") await vendorBwrap();+}diff --git a/plugins/sandbox-bwrap/test/live.test.ts b/plugins/sandbox-bwrap/test/live.test.tsindex d2a4c2dc..879ca202 100644--- a/plugins/sandbox-bwrap/test/live.test.ts+++ b/plugins/sandbox-bwrap/test/live.test.ts@@ -4,7 +4,8 @@** Host-gated — a package whose whole job is kernel confinement can only be proven on a* host that has bubblewrap; elsewhere the suite skips and profile.test.ts still pins- * the profile and the fail-closed path. The backend is driven DIRECTLY here (no+ * the profile and the fail-closed path (unless ADELIE_MUST_RUN names it; see+ * scripts/must-run.mjs). The backend is driven DIRECTLY here (no* SandboxService): what a plugin package owes is that its own confinement works, and* routing/settings are the harness's behavior, tested there with fakes.*@@ -19,7 +20,8 @@ import { fileURLToPath } from "node:url";import { CommandSessionManager } from "@lmliheng/penguin-core";import type { SpawnConfiner } from "@lmliheng/penguin-core";import type { SandboxPolicy, SandboxProviderSource } from "@lmliheng/penguin-core/plugin";-import { createPenguinBwrapProvider } from "../src/index.js";+import { createPenguinBwrapProvider, loadPenguinBwrapProvider } from "../src/index.js";+import { mustRun } from "../../../scripts/must-run.mjs";/** The slice of the server's SandboxService these cases drive. */interface ServiceUnderTest {@@ -41,14 +43,16 @@ const provider = createPenguinBwrapProvider();/** null = spawn unconfined; otherwise confine under this policy (workspaceRoot filled per spawn). */let policy: Omit<SandboxPolicy, "workspaceRoot"> | null = null;-const usable = (() => {+const cannotOpen = await (async () => {try {+ if ((await loadPenguinBwrapProvider()) === null) return "penguin-bwrap runs on Linux only";provider.confine(["true"], { mode: "read-only", workspaceRoot: ws });- return true;- } catch {- return false;+ return null;+ } catch (err) {+ return err instanceof Error ? err.message : String(err);}})();+const usable = mustRun("sandbox-bwrap", cannotOpen);const mgr = new CommandSessionManager({confineSpawn: () => (argv, opts) =>diff --git a/plugins/sandbox-bwrap/vitest.config.ts b/plugins/sandbox-bwrap/vitest.config.tsnew file mode 100644index 00000000..984064bb--- /dev/null+++ b/plugins/sandbox-bwrap/vitest.config.ts@@ -0,0 +1,8 @@+// The global setup puts the bubblewrap this plugin ships in place (see test/global-setup.ts).+import { defineConfig } from "vitest/config";++export default defineConfig({+ test: {+ globalSetup: ["./test/global-setup.ts"],+ },+});diff --git a/plugins/sandbox-dsh/test/live.test.ts b/plugins/sandbox-dsh/test/live.test.tsindex 517929e6..15c572c2 100644--- a/plugins/sandbox-dsh/test/live.test.ts+++ b/plugins/sandbox-dsh/test/live.test.ts@@ -4,7 +4,8 @@* spawns through core's command sessions, real kernel denials.** Host-gated the way DSH gates its own backend e2e: one real confine decides- * usability, and a host with no usable backend skips. The adaptor is driven DIRECTLY+ * usability, and a host with no usable backend skips (unless ADELIE_MUST_RUN names it;+ * see scripts/must-run.mjs). The adaptor is driven DIRECTLY* (no SandboxService): what this package owes is that DSH's confinement works behind* our interface; routing and settings are the harness's behavior, tested there.*/@@ -15,25 +16,32 @@ import path from "node:path";import { CommandSessionManager } from "@lmliheng/penguin-core";import type { SandboxProvider } from "@lmliheng/penguin-core/plugin";import { loadDshAdaptor } from "../src/index.js";+import { mustRun } from "../../../scripts/must-run.mjs";const ws = mkdtempSync(path.join(tmpdir(), "penguin-dsh-live-"));const outsideProbe = path.join(homedir(), `penguin-dsh-live-${process.pid}.txt`);-const provider: SandboxProvider | null = await loadDshAdaptor().catch(() => null);+let loadError = "the DSH adaptor did not load";+const provider: SandboxProvider | null = await loadDshAdaptor().catch((err: unknown) => {+ loadError = err instanceof Error ? err.message : String(err);+ return null;+});/** null = spawn unconfined; otherwise confine under this mode. */let mode: "read-only" | "workspace-write" | null = null;-const usable =- provider !== null &&- (() => {- try {- provider.confine(["true"], { mode: "workspace-write", workspaceRoot: ws });- return true;- } catch {- return false;- }- })();+const cannotOpen =+ provider === null+ ? loadError+ : (() => {+ try {+ provider.confine(["true"], { mode: "workspace-write", workspaceRoot: ws });+ return null;+ } catch (err) {+ return err instanceof Error ? err.message : String(err);+ }+ })();+const usable = mustRun("sandbox-dsh", cannotOpen);const mgr = new CommandSessionManager({confineSpawn: () => (argv, opts) =>diff --git a/plugins/sandbox-seatbelt/test/live.test.ts b/plugins/sandbox-seatbelt/test/live.test.tsindex 188e4ba6..c4020e56 100644--- a/plugins/sandbox-seatbelt/test/live.test.ts+++ b/plugins/sandbox-seatbelt/test/live.test.ts@@ -3,7 +3,8 @@* core's command sessions, real kernel denials.** Host-gated — this suite can only run where Seatbelt exists, so it skips everywhere- * else and profile.test.ts carries the deterministic coverage. Written to be the exact+ * else and profile.test.ts carries the deterministic coverage (unless ADELIE_MUST_RUN+ * names it; see scripts/must-run.mjs). Written to be the exact* counterpart of the bwrap package's live suite, so the two backends are held to the* same behavioral bar.*/@@ -14,6 +15,7 @@ import path from "node:path";import { CommandSessionManager } from "@lmliheng/penguin-core";import type { SandboxPolicy } from "@lmliheng/penguin-core/plugin";import { canonicalPath, createSeatbeltProvider } from "../src/index.js";+import { mustRun } from "../../../scripts/must-run.mjs";const ws = canonicalPath(mkdtempSync(path.join(tmpdir(), "penguin-seatbelt-live-")));const outsideProbe = path.join(homedir(), `penguin-seatbelt-live-${process.pid}.txt`);@@ -22,14 +24,15 @@ const provider = createSeatbeltProvider();/** null = spawn unconfined; otherwise confine under this policy (workspaceRoot filled per spawn). */let policy: Omit<SandboxPolicy, "workspaceRoot"> | null = null;-const usable = (() => {+const cannotOpen = (() => {try {provider.confine(["true"], { mode: "read-only", workspaceRoot: ws });- return true;- } catch {- return false;+ return null;+ } catch (err) {+ return err instanceof Error ? err.message : String(err);}})();+const usable = mustRun("sandbox-seatbelt", cannotOpen);const mgr = new CommandSessionManager({confineSpawn: () => (argv, opts) =>diff --git a/scripts/must-run.d.mts b/scripts/must-run.d.mtsnew file mode 100644index 00000000..685f925c--- /dev/null+++ b/scripts/must-run.d.mts@@ -0,0 +1,2 @@+/** Types for must-run.mjs, which the plugins' typechecked live suites import. */+export declare function mustRun(suite: string, cannotOpen: string | null): boolean;diff --git a/scripts/must-run.mjs b/scripts/must-run.mjsnew file mode 100644index 00000000..74b869af--- /dev/null+++ b/scripts/must-run.mjs@@ -0,0 +1,17 @@+/**+ * ADELIE_MUST_RUN: a comma-separated list of the host-dependent suites, named by their+ * directory under plugins/, that a run requires to really run (CI sets it per platform). A+ * named suite this host cannot open fails with the reason; an unnamed one skips as before, and+ * so does a misspelled name. Spaces around a name are ignored.+ */++/** True when the suite can run here; throws when ADELIE_MUST_RUN names it and it cannot. */+export function mustRun(suite, cannotOpen) {+ const required = (process.env.ADELIE_MUST_RUN ?? "").split(",").map((name) => name.trim());+ if (cannotOpen !== null && required.includes(suite)) {+ throw new Error(+ `ADELIE_MUST_RUN requires ${suite}, and this host cannot open it: ${cannotOpen}`,+ );+ }+ return cannotOpen === null;+}diff --git a/scripts/vendor-bwrap.d.mts b/scripts/vendor-bwrap.d.mtsnew file mode 100644index 00000000..30b2847f--- /dev/null+++ b/scripts/vendor-bwrap.d.mts@@ -0,0 +1,2 @@+/** Types for vendor-bwrap.mjs, which sandbox-bwrap's typechecked test setup imports. */+export declare function vendorBwrap(): Promise<string>;diff --git a/scripts/vendor-bwrap.mjs b/scripts/vendor-bwrap.mjsindex c6375ece..0bb6aeb4 100644--- a/scripts/vendor-bwrap.mjs+++ b/scripts/vendor-bwrap.mjs@@ -241,14 +241,33 @@ async function walk(dir, prefix = "") {/*** Vendors every target into `plugins/sandbox-bwrap/vendor/<arch>/`, skipping the work when the- * directory already holds this pinning (a `.complete` marker naming it).+ * directory already holds this pinning (a `.complete` marker naming it) and every target's+ * bwrap is still there and executable.+ *+ * The executability check is what lets the plugin's own tests prepare themselves: a bare test+ * run (no build) re-vendors a binary somebody deleted or chmod-ed, from the cache, instead of+ * letting the live suite fall through to whatever `bwrap` a developer has on PATH.*/export async function vendorBwrap() {const pinning = createHash("sha256").update(`${VENDOR_FORMAT}\0${JSON.stringify(TARGETS)}`).digest("hex");const marker = path.join(OUT, ".complete");- if (fs.existsSync(marker) && (await fsp.readFile(marker, "utf8")) === pinning) return OUT;+ const runnable = (t) => {+ try {+ fs.accessSync(path.join(OUT, t.arch, "bin", "bwrap"), fs.constants.X_OK);+ return true;+ } catch {+ return false;+ }+ };+ if (+ fs.existsSync(marker) &&+ (await fsp.readFile(marker, "utf8")) === pinning &&+ TARGETS.every(runnable)+ ) {+ return OUT;+ }await fsp.rm(OUT, { recursive: true, force: true });for (const target of TARGETS) {const stage = path.join(OUT, target.arch);