Skip to content

armgit / mso

Public Imported from GitHub
0

Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.

feat(ci): three security scanners that actually run, and lanes that fail closed (#210)

Abdurrahman Fakhrul9a9578bed833
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index ec9b749d..b16e58a8 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -19,11 +19,35 @@ work proceed without VPS credentials or host access.
Run:
```bash
+bash scripts/gates-12.sh full
bun run verify
node scripts/check-docs.mjs
bash scripts/verify-build.sh
```
+**Start with `bash scripts/gates-12.sh full`.** It is the local mirror of the GitHub
+Actions `verify` job: the same checks in the same order, so a red build shows up on your
+machine instead of in your Actions minutes. `full` adds coverage, the production build and
+every browser journey; the default `fast` mode is about two minutes and skips the slow
+three.
+
+A clean run ends with either of these, and both are green:
+
+```text
+mode=full passed=17 failed=0 skipped=0
+mode=full passed=16 failed=0 skipped=1
+```
+
+The difference is the bundle budget, which `verify-build.sh` enforces inside the same run
+rather than twice, so it reports as `SKIP` rather than `PASS`.
+
+**Run only one `gates-12.sh` at a time per worktree.** Two concurrent `full` runs corrupt
+each other in ways that look like real defects: vitest refuses to start because both
+write the same `coverage/` directory, and `verify-build.sh` copies the tree into `/tmp`
+per run, so two copies can exhaust the tmpfs. `fast` mode alongside `full` is also unsafe:
+they share `/tmp/gate-out.txt`, so one run's failure text can appear under the other's
+name.
+
`bun run verify` covers typecheck, lint, the Vitest suite, repository checks and the
high/critical dependency audit. `verify-build.sh` compiles a throwaway copy of `HEAD` so it
is safe even when the checkout is also the production WorkingDirectory.
@@ -32,6 +56,64 @@ Do **not** run a bare production build inside a live production checkout merely
that code compiles; replacing `.next` underneath `next start` can create a chunk mismatch
until the service is replaced.
+## Where the CI work actually runs
+
+The `verify` job is fully reproducible locally; the security workflows are not yet, and
+this is deliberate rather than an oversight:
+
+| Workflow | Local equivalent |
+| --- | --- |
+| `ci.yml` `verify` | `bash scripts/gates-12.sh full` |
+| `ci.yml` `native-host` (macOS/Windows) | not reproducible; needs those runners |
+| `ci.yml` `dependency-watch` | `bun run audit:strict` plus `npm audit --prefix scripts/install/node-gyp --omit=dev --audit-level=high` |
+| `security-core.yml` | partially: `docs/SELF-HOSTED-CI.md` lane is BLOCKED |
+| `codeql.yml`, `dast.yml`, `dependency-review.yml`, `scorecard.yml`, `security-alerts.yml`, `codex-security.yml` | no local equivalent |
+
+`bun run ci:plan` prints the repository-owned CI plan, and `bun run ci:verify --sha
+"$(git rev-parse HEAD)"` runs it. The `security` and `release` profiles return **BLOCKED**
+on purpose: the scanner adapters are written but not yet verified end to end, and a
+blocked lane must never be reported as a pass. See
+[`docs/SELF-HOSTED-CI.md`](./docs/SELF-HOSTED-CI.md) for the migration matrix and the
+evidence rules.
+
+You can still run the three scanners by hand with the exact image digests the workflows
+pin. All three have been verified locally:
+
+```bash
+docker run --rm -v "$PWD:/src:ro" semgrep/semgrep@sha256:f1f7b71861c7b28b6e0f661225a2c4f58a484f5d0f182465c6d6b3b22f972ade \
+ semgrep scan --metrics=off --config=p/javascript --config=p/typescript \
+ --config=p/owasp-top-ten --strict --error \
+ --exclude=node_modules --exclude=.next --exclude=coverage /src
+
+docker run --rm -v "$PWD:/repo" zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f \
+ git --gitleaks-ignore-path /repo/.gitleaksignore /repo
+
+docker run --rm -v "$PWD:/src:ro" ghcr.io/google/osv-scanner@sha256:8108ae94eadea5a02c9bec6e646909d5b790b44bd62d7f5b7f0b1d6d0ffc7734 \
+ scan source --recursive /src
+```
+
+The `--exclude` flags are not decoration. CI scans a clean checkout; a working tree also
+has `node_modules`, `.next` and `coverage`, and scanning those produces findings in
+third-party code that CI never sees. Without them Semgrep reported **7 findings** here and
+**0** with them.
+
+Run the Gitleaks command **from the main checkout, not from a git worktree**. A worktree's
+`.git` is a file pointing at the real git dir, so the scanner sees no repository and reports
+`0 commits scanned` followed by `no leaks found`. That reads like a pass and proves nothing:
+the same command in the main checkout scanned **1165 commits** and is the only form of the
+check that means anything.
+
+The OSV result depends on `bun.lock`. It reports `No issues found` on a tree that already
+carries the `minimatch` override to 11.0.3, and two medium findings (`minimatch` 3.1.5 and
+10.2.6) on one that does not. If you see those two, the fix exists and your base predates it,
+so rebase rather than adding a second override block.
+
+Two things these catch that the gates do not. Semgrep fails on a **parse warning** under
+`--strict` even with zero findings, so a generic type argument on `await
+importOriginal<...>()` in a test breaks the required check; apply the type with a cast on
+its own `const` instead. Gitleaks scans all history, so a secret in an old commit fails the
+check regardless of what your diff contains.
+
The committed source of truth for pre-push policy is `scripts/gates.sh`. The actual
`.git/hooks/pre-push` file is intentionally a tiny untracked shim. Install/reinstall it with:
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 013dd510..738d74b1 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -4,11 +4,25 @@
Newest first. `docs/PROGRESS.md` is the source of truth for *why* a change was made;
this is the *what*, and it is what Settings → Account → About shows as “What's new”.
+## 2026-11-01
+
+**Added**
+
+- `tui` the panel is the default, and the banner lives in the transcript (#209)
+
+**Fixed**
+
+- `docs` regenerate the changelog after the date rolled over
+
+**Docs**
+
+- `contributing` how to run the CI work locally before opening a PR
+- `gates` the green full run, and the parallel-run trap
+
## 2026-10-31
**Added**
-- `tui` the panel is the default, and the banner lives in the transcript
- `tui` install the print sink the last change forgot, and migrate the modules that still print (#206)
- `tui` a suspended panel really leaves the alternate screen, so an approval is readable (#205)
- `tui` mid-turn writes go into the panel, and prompts suspend it instead of fighting it (#204)
@@ -1515,30 +1529,3 @@ this is the *what*, and it is what Settings → Account → About shows as “Wh
- `deps` hold incompatible toolchain majors
- `deps` migrate to lucide-react 1.33.0
-
-## 2026-08-28
-
-**Added**
-
-- `operations` add evidence-backed comparison and delegated control
-- `mcp` add bounded async exec jobs
-
-**Fixed**
-
-- `security` remediate hosted CodeQL findings
-- `security` remediate ultimate assurance findings
-- `security` resolve full assurance findings
-- `security` raise complete component scan budget
-- `security` keep Codex scan state private
-- `security` block recursive credential relocation
-
-**Chores**
-
-- `deps` bump actions/setup-python from 6.3.0 to 7.0.0
-- `deps` bump zaproxy/action-baseline
-- `deps` bump actions/setup-node from 6.5.0 to 7.0.0
-- `deps` bump actions/upload-artifact from 4.6.2 to 7.0.1
-- `deps` bump ossf/scorecard-action in the actions-minor-patch group
-- `deps-dev` bump the development-minor-patch group with 4 updates
-- `deps` bump the production-minor-patch group with 4 updates
-- `security` update Codex Security to 0.1.21
diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md
index cd19896b..a670fea5 100644
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -1,3 +1,48 @@
+## 2026-11-01: the green full run, and the parallel-run trap
+
+`bash scripts/gates-12.sh full` on the TUI panel branch: **passed=16, failed=0, skipped=1**. The
+single skip is `bundle budget`, which is enforced by `verify-build.sh` inside the same run rather
+than twice, so a skip there means the out-of-tree build already checked it.
+
+**The red runs before it were not real.** Two `gates-12.sh full` invocations were left running in
+the same worktree at once. They do not fail cleanly, they corrupt each other:
+
+- `tests + coverage` failed because both wrote the coverage output directory and removed it from
+ under the other, so vitest reported a script error rather than a threshold miss.
+- `production build` failed with "verification failed during mandatory release E2E" because
+ `verify-build.sh` copies the tree into `/tmp` per run, and two copies exhausted the 9.8 GB tmpfs.
+ The real error was `cp: error writing ... No space left on device`, three lines above the
+ verification message.
+
+Both passed once a single run had the machine to itself. A killed run also leaves `verify-build.sh`
+orphaned with `ppid 1`, still holding a `/tmp` tree, so killing the gate is not enough to free the
+disk: check for survivors before starting another.
+
+**Rule: one `gates-12.sh` run per worktree, never in parallel, not even `fast` alongside `full`.**
+They share `/tmp/gate-out.txt` and `/tmp` itself, so the outputs describe each other's failures.
+
+**The coverage numbers, from one run alone.** `bun run coverage` with no other vitest in the worktree
+and the stale `coverage/` directory removed first: **964 test files, 6,278 tests, all passing**;
+statements 53.72%, branches 50.47%, functions 42.63%, lines 57.5% against thresholds of 42 / 40 / 32 /
+43.5. The earlier failure was never a threshold miss. It was vitest refusing to start:
+
+```
+The coverage report directory ".../mso-flip/coverage" is already in use by another Vitest process
+(pid ...). Running coverage for multiple Vitest processes in the same directory at the same time is
+not supported, because they would delete each other's reports.
+```
+
+That message names the cause precisely, so it is worth reading before assuming a regression: vitest
+detects the collision itself.
+
+**One more trap, in the tooling rather than the code.** `pgrep -f "vitest run --coverage"` always
+reports a live process, because the pattern matches the `bash -c` wrapper line that *is* the command
+you just ran. The same is true of `pgrep -f "gates-12.sh"`. Use
+`pgrep -af "node .*vitest" | grep -v "bash -c"` to see what is actually running, or the check will
+insist the worktree is busy forever.
+
+
+
## 2026-10-31: every route has a test, and the workflow conflict that made each PR slow
Route test coverage went from **59 of 117 to 117 of 117**. No route in the repository is
diff --git a/scripts/ci/contract.node.mjs b/scripts/ci/contract.node.mjs
index 7d7b0403..e2cd329e 100644
--- a/scripts/ci/contract.node.mjs
+++ b/scripts/ci/contract.node.mjs
@@ -4,7 +4,7 @@ import { mkdtemp, rm, readFile, stat, symlink, writeFile, mkdir } from 'node:fs/
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { execFileSync } from 'node:child_process';
-import { planFor, verdict, exitFor } from './plan.mjs';
+import { anchorMounts, planFor, verdict, exitFor } from './plan.mjs';
import { executeStep, childEnvironment } from './process.mjs';
import { acquireLocks, privateDirectory } from './store.mjs';
import { runPipeline, inspectRepository } from './runner.mjs';
@@ -20,10 +20,27 @@ test('fixed plans reject arbitrary shell/profile input', () => {
assert.equal(planFor('verify').length, 4);
assert.equal(planFor('verify').at(-1).args[0], 'scripts/gates.sh');
});
-test('security lanes remain visibly blocked, not waived', () => {
- assert.equal(planFor('security').length, 6);
- assert(planFor('security').every((step) => step.blocked));
- assert.equal(planFor('release').length, 10);
+test('security lanes are executable or visibly blocked, never silently waived', () => {
+ const security = planFor('security');
+ const runnable = security.filter((step) => !step.blocked);
+ const blocked = security.filter((step) => step.blocked);
+ // Three scanners run against pinned digests; everything else must say out loud that it is blocked.
+ assert.deepEqual(runnable.map((step) => step.id), ['semgrep', 'gitleaks', 'osv']);
+ assert.equal(blocked.length, 7);
+ assert(blocked.every((step) => typeof step.blocked === 'string' && step.blocked.length > 20));
+ assert.equal(planFor('release').length, 14);
+});
+test('docker lanes pin image digests instead of mutable tags', () => {
+ for (const step of planFor('security').filter((s) => !s.blocked)) {
+ const image = step.args[4];
+ assert.match(image, /@sha256:[a-f0-9]{64}$/, `${step.id} must pin a digest`);
+ }
+});
+test('docker mounts are anchored to the verified repository root', () => {
+ const steps = anchorMounts(planFor('security'), '/verified/root');
+ const mounts = steps.filter((s) => !s.blocked).map((s) => s.args[3]);
+ assert.deepEqual(mounts, ['/verified/root:/src:ro', '/verified/root:/repo', '/verified/root:/src:ro']);
+ assert(!JSON.stringify(steps).includes('"./:/'), 'no unanchored relative mount may survive');
});
test('empty, partial and not-run evidence never passes', () => {
assert.equal(verdict([]), 'BLOCKED');
@@ -86,11 +103,17 @@ test('first failure stops later steps but retains a report', () => fixture(async
assert.equal(count, 1); assert.equal(result.exitCode, 1);
assert.deepEqual(result.report.steps.map((s) => s.status), ['FAIL', 'NOT_RUN', 'NOT_RUN', 'NOT_RUN']);
}));
-test('blocked security never starts commands', () => fixture(async (root) => {
- let calls = 0;
- const result = await runPipeline({ repo: root, expectedSha: sha, profile: 'release', stateRoot: root }, { inspect, execute: async () => { calls++; return pass(); } });
- assert.equal(calls, 0); assert.equal(result.exitCode, 2);
- assert.equal(result.report.steps.filter((s) => s.status === 'BLOCKED').length, 6);
+test('a blocked lane never runs, and never turns the verdict into a pass', () => fixture(async (root) => {
+ const ran = [];
+ const result = await runPipeline({ repo: root, expectedSha: sha, profile: 'release', stateRoot: root }, { inspect, execute: async (step) => { ran.push(step.id); return pass(); } });
+ // Executable lanes do run, so a partial migration still produces evidence.
+ assert.deepEqual(ran, ['native-module', 'installer-bootstrap', 'installer-core', 'repository-gates', 'semgrep', 'gitleaks', 'osv']);
+ // But seven blocked lanes keep the whole run BLOCKED: unrun is never green.
+ assert.equal(result.report.steps.filter((s) => s.status === 'BLOCKED').length, 7);
+ assert.equal(result.report.status, 'BLOCKED');
+ assert.equal(result.exitCode, 2);
+ assert.equal(result.report.releaseEligible, false);
+ assert.equal(result.report.deployed, false);
}));
test('preflight failure is persisted', () => fixture(async (root) => {
const result = await runPipeline({ repo: root, expectedSha: sha, profile: 'verify', stateRoot: root }, { inspect: async () => { throw new Error('Checkout is dirty'); } });
diff --git a/scripts/ci/plan.mjs b/scripts/ci/plan.mjs
index ac59d3ae..a9d30551 100644
--- a/scripts/ci/plan.mjs
+++ b/scripts/ci/plan.mjs
@@ -1,24 +1,60 @@
/** Repository-owned CI policy; no webhook/PR-supplied commands. */
const step = (id, command, args, timeoutMs) => ({ id, command, args, timeoutMs });
+
+// Image digests are copied verbatim from .github/workflows/security-core.yml. Pinning by digest
+// rather than tag is the whole point: a tag can be repointed, and then the local check and CI would
+// be running different scanners while both claim to be "the security scan".
+const SEMGREP_IMAGE = 'semgrep/semgrep@sha256:f1f7b71861c7b28b6e0f661225a2c4f58a484f5d0f182465c6d6b3b22f972ade';
+const GITLEAKS_IMAGE = 'zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f';
+const OSV_IMAGE = 'ghcr.io/google/osv-scanner@sha256:8108ae94eadea5a02c9bec6e646909d5b790b44bd62d7f5b7f0b1d6d0ffc7734';
+
const verify = [
step('native-module', 'node', ['-e', "require('node-pty')"], 30_000),
step('installer-bootstrap', 'bash', ['-n', 'scripts/install.sh'], 30_000),
step('installer-core', 'bash', ['-n', 'scripts/install-core.sh'], 30_000),
step('repository-gates', 'bash', ['scripts/gates.sh'], 30 * 60_000),
];
+
+// Three lanes below have been run by hand against this repository with these exact digests, so they
+// execute instead of reporting BLOCKED. The excludes on semgrep matter: CI scans a clean checkout,
+// while a working tree also holds node_modules, .next and coverage, and scanning those reports
+// findings in third-party code that CI never sees (7 findings versus 0 when measured here).
const security = [
+ step('semgrep', 'docker', ['run', '--rm', '-v', './:/src:ro', SEMGREP_IMAGE,
+ 'semgrep', 'scan', '--metrics=off',
+ '--config=p/javascript', '--config=p/typescript', '--config=p/owasp-top-ten',
+ '--strict', '--error',
+ '--exclude=node_modules', '--exclude=.next', '--exclude=coverage', '/src'], 20 * 60_000),
+ step('gitleaks', 'docker', ['run', '--rm', '-v', './:/repo', GITLEAKS_IMAGE,
+ 'git', '--gitleaks-ignore-path', '/repo/.gitleaksignore', '/repo'], 20 * 60_000),
+ step('osv', 'docker', ['run', '--rm', '-v', './:/src:ro', OSV_IMAGE,
+ 'scan', 'source', '--recursive', '/src'], 20 * 60_000),
{ id: 'codeql', blocked: 'Local CodeQL bundle, security-extended analysis and exact-SHA SARIF adapter are not yet provisioned.' },
- { id: 'security-core', blocked: 'Pinned Trivy/OSV/Gitleaks/Semgrep/ShellCheck adapter needs standalone end-to-end verification; preserve scripts/security-ultimate.sh.' },
+ { id: 'trivy', blocked: 'Pinned Trivy filesystem scan and CycloneDX SBOM adapter are not yet verified.' },
+ { id: 'shellcheck', blocked: 'Pinned ShellCheck adapter is not yet verified.' },
{ id: 'dependency-review', blocked: 'Base/head dependency-change review adapter is not yet verified.' },
{ id: 'security-inventory', blocked: 'Exact-SHA SARIF upload/read-back through the authorized GitHub connection is not yet verified.' },
{ id: 'scorecard', blocked: 'Standalone OpenSSF Scorecard adapter is not yet verified.' },
{ id: 'dast', blocked: 'Explicit isolated staging target and ZAP evidence adapter are not yet verified.' },
];
+
export const profiles = Object.freeze(['verify', 'security', 'release']);
export function planFor(profile) {
if (!profiles.includes(profile)) throw new Error('Profile must be verify, security, or release');
return structuredClone(profile === 'verify' ? verify : profile === 'security' ? security : [...verify, ...security]);
}
+
+// Docker bind mounts need an absolute source path, and `./:/src` silently resolves against whatever
+// directory the CLI happens to inherit. Rewriting the mount against the resolved repo root makes it
+// explicit, and it is what lets the runner refuse to scan a tree other than the one it verified.
+export function anchorMounts(steps, repoRoot) {
+ return steps.map((s) => {
+ if (!Array.isArray(s.args)) return s;
+ const args = s.args.map((arg) => (arg === './:/src:ro' ? `${repoRoot}:/src:ro` : arg === './:/repo' ? `${repoRoot}:/repo` : arg));
+ return { ...s, args };
+ });
+}
+
export function verdict(steps) {
if (steps.some((s) => s.status === 'FAIL')) return 'FAIL';
if (!steps.length || steps.some((s) => s.status !== 'PASS')) return 'BLOCKED';
diff --git a/scripts/ci/runner.mjs b/scripts/ci/runner.mjs
index 0a516d6d..42bbe228 100644
--- a/scripts/ci/runner.mjs
+++ b/scripts/ci/runner.mjs
@@ -1,7 +1,7 @@
import { execFileSync } from 'node:child_process';
import { realpath } from 'node:fs/promises';
import { join } from 'node:path';
-import { planFor, verdict, exitFor } from './plan.mjs';
+import { anchorMounts, planFor, verdict, exitFor } from './plan.mjs';
import { executeStep, childEnvironment } from './process.mjs';
import { createRun, saveReport, acquireLocks } from './store.mjs';
function git(cwd, ...args) {
@@ -35,23 +35,30 @@ export async function runPipeline({ repo, expectedSha, profile, stateRoot, signa
release = await acquireLocks(stateRoot, target.identity, run.runId);
target = await inspect(repo, expectedSha);
report.commit = target.head;
+ // The docker lanes carry a `./:/src` placeholder; bind it to the verified root so the scan cannot
+ // mount a different tree than the one whose SHA was just checked.
+ steps.splice(0, steps.length, ...anchorMounts(steps, target.cwd));
for (const step of steps) {
if (step.blocked) { step.status = 'BLOCKED'; step.reason = step.blocked; }
}
- if (steps.some((step) => step.status === 'BLOCKED')) {
- report.reason = 'Security migration incomplete. No selected commands executed; release remains blocked.';
- } else {
- for (const step of steps) {
- if (signal?.aborted) { step.status = 'BLOCKED'; step.reason = 'Cancelled before execution'; break; }
- await inspect(repo, expectedSha);
- step.startedAt = new Date().toISOString();
- await saveReport(run.path, report);
- Object.assign(step, await execute(step, { cwd: target.cwd, logPath: join(run.path, `${step.id}.log`), signal }));
- await saveReport(run.path, report);
- if (step.status !== 'PASS') break;
- }
+ // Executable lanes run even when other lanes are still blocked, so a partially migrated profile
+ // produces real evidence instead of nothing. The verdict stays BLOCKED either way: a lane that
+ // never ran is not a pass, and the runner must not imply security parity it does not have.
+ const anyBlocked = steps.some((step) => step.status === 'BLOCKED');
+ if (anyBlocked) {
+ report.reason = 'Security migration incomplete; executable lanes ran, blocked lanes did not, release remains blocked.';
+ }
+ for (const step of steps) {
+ if (step.blocked) continue;
+ if (signal?.aborted) { step.status = 'BLOCKED'; step.reason = 'Cancelled before execution'; break; }
await inspect(repo, expectedSha);
+ step.startedAt = new Date().toISOString();
+ await saveReport(run.path, report);
+ Object.assign(step, await execute(step, { cwd: target.cwd, logPath: join(run.path, `${step.id}.log`), signal }));
+ await saveReport(run.path, report);
+ if (step.status !== 'PASS') break;
}
+ await inspect(repo, expectedSha);
report.status = verdict(steps);
} catch (error) {
report.status = 'BLOCKED';