Skip to content

armgit / mso

Public Imported from GitHub
0

Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.

21.2 KB
Raw Blame

Development

Setup

git clone [email protected]:rahmanef63/mso.git && cd mso
curl -fsSL https://bun.sh/install | bash   # if bun is missing (see "Package manager" below)
bun install
cp .env.example .env.local   # set OS_LOGIN_PASSWORD + OS_SESSION_SECRET (openssl rand -hex 32)
bun run dev                     # http://localhost:3000
node scripts/approve-device.js <deviceId> "my laptop" --role owner   # deviceId shows on login

Layout

Every feature is a self-contained vertical slice under frontend/slices/<slug>/ (its own components, hooks, and a lib/host.ts seam for host I/O). One manifest, frontend/slices/os-shell/shell.manifest.ts, wires slices into the shell — so adding an app = one slice + one manifest entry. Host access is bounded in lib/host (Node fs/child_process, filesystem-jailed) behind signed-cookie plus live device-role auth (lib/auth). See ARCHITECTURE.md.

Quality gates

TypeScript enforces noUnusedLocals and noUnusedParameters. Remove genuinely unused imports/declarations; keep required side effects and use an explicit _ prefix only for positional interface/callback arguments.

bun run verify              # full repository quality gate
bun run test:features        # one process per area; exit 2 means skipped/incomplete evidence
bun run audit:strict         # fail closed on missing/malformed dependency evidence
bun run security:ultimate   # release assurance: independent scanners + component security review + DAST

The supported full-test host is Linux with Docker available. Provision the local-only fixture image once with bun run forge:sandbox before running bun run verify; missing sandbox prerequisites now fail the Forge tests instead of skipping them. Secret-path tests create disposable synthetic home/app directories and never read or create production .env.local.

bun run lint accepts zero warnings. Unit/integration suites contain no conditional skips. The four HTTP smoke tests belong to the explicit production suite (E2E_BASE_URL=http://host:port bun run smoke); missing configuration fails, and every required release journey runs them against its built synthetic server before browser checks.

The committed source of truth for pre-push policy is scripts/gates.sh. The actual .git/hooks/pre-push file is an intentionally tiny untracked shim; reinstall it idempotently with bash scripts/gates.sh --install.

The gate runs the repository-owned bun run verify, cycle checks, generated-changelog freshness, documentation/skill checks, comparison evidence/freshness, the strict high/critical dependency audit, and an out-of-tree production build plus mandatory synthetic browser journeys. check-contrast.mjs blocks palette regressions. None of the build verification touches the live checkout's .next. A healthy push ends with audit: clean at high/critical. and build: HEAD compiles (out-of-tree).

The local dependency audit may explicitly skip an unavailable registry; that is not a security pass. bun run audit:strict, the pre-push gate, and the assurance runner reject incomplete evidence. test:features distinguishes PASS, PARTIAL, SKIPPED and FAIL. Per-area exact-include configurations and result-set validation prevent substring-filter double counting; a root inventory contract prevents unregistered root tests. Slices without colocated unit tests are listed as coverage gaps, not invented successes. The required browser fixture covers public guides, direct login/return, launcher clicks and desktop/phone reflow, plus real local-provider rejection and immediate role demotion. Live third-party OAuth still needs provider-specific acceptance. The gate has no sibling-repository runner or dormant Convex deployment step. Worktree hook installation uses Git's shared hook path rather than assuming .git is a directory.

Browser release fixtures

Install the pinned browser once with node_modules/.bin/playwright install chromium (on CI use --with-deps). scripts/verify-build.sh runs e2e:release against the same out-of-tree build. A missing browser or failed journey blocks the push; no production password, approved device, or integration is borrowed. Fixture stores, filesystem roots and provider endpoints are temporary and loopback-only. Normal CI runs the same journeys after its production build. A daily dependency job runs the strict audit so advisory status cannot silently age out.

Coverage floors are statements 42%, branches 40%, functions 32%, lines 43.5%, ratcheted from the measured audit baseline. These are regression floors, not a claim that every feature is covered. Two-instance tests independently load origin, widget CSP and external app catalogs. The literal guard is supporting evidence, not a substitute for runtime isolation tests.

Bundle boundaries and performance verification

bun run bundle:report reads an existing production build; bun run bundle:check enforces scripts/bundle-budget.json. scripts/verify-build.sh runs the budget and scripts/e2e/bundle-performance.mjs on its isolated committed build; CI runs the same checks after its production build. The browser journey checks all five shells, absence of graph/editor/terminal/ONNX runtime chunks on cold startup, delayed first-open/close/reopen Search, eager clipboard capture, and retained Inspector tab state. It uses synthetic stores.

Perceived-performance measurement

Bundle bytes are deterministic; the numbers a person actually feels are not, so they live in a separate, deliberately weaker gate.

Command What it does Gates a push
bun run perf:probe Cold load at Slow 4G + 4× CPU against a synthetic fixture (or --base=), writes perf-report.json no
bun run perf:check Enforces scripts/perf-budget.json against a report yes
bun run perf:hydration Mark chain: FCP → last JS byte → shell mount → first paint of icons → LCP no
bun run perf:tasks CDP-trace attribution of every long task to the script that owns it no
bun run perf:apps Per-app open latency, fetched chunks and long tasks caused no

perf:hydration is the one that answers "what actually gates LCP". Reading it off the LCP element tag does not work: an element that ends up largest can be painted arbitrarily late by JavaScript, so SPAN or IMG says nothing about the cause. On this shell the chain is decisive — FCP ~1.2 s, mount ~2.6-2.9 s, LCP ~3.2-4.0 s — which means LCP is gated by downloading and executing ~940 KB of first-load JS, not by first paint.

perf:check fails hard on a missing measurement — a silent zero is how a perf gate dies — and on a 2× TBT blowup against scripts/perf-baseline.json, because this box also serves production and a wall-clock gate that flakes gets muted. The TBT comparison is skipped, with a printed reason, when the host 1-minute load average is above enforce.maxLoadAvg1ForTiming: the same commit measures 138 ms idle and 794 ms under load, and the baseline is an idle measurement. CI runs this in the verify job. Bundle bytes are deliberately not re-gated here: check-bundle.mjs already gates gzip bytes deterministically from build output, while a browser-measured byte count includes deferred chunks and moves. Targets in the budget file (tbtMs 150, lcpMs 2500, initialGzipBytes 256000) are the post-migration goal from docs/MSO-MIGRATION.md; they are printed as advisory and never enforced, since holding today's Next build to a Svelte target would just make the gate permanently red.

Regenerate the baseline with node scripts/perf-probe.mjs --json and commit the runs. Do that deliberately after a real improvement, not to silence a failure.

The report deduplicates eager catch-all entry chunks plus the framework runtime. It lists nomodule polyfills separately: charging them to modern browsers would overstate first-load bytes. Gzip/Brotli values are compression estimates, not network timings or an LCP/INP benchmark. Actual selected-shell and feature-driven lazy downloads are checked separately in the browser. Font limits apply to preloaded files, not to fonts subsequently needed by visible monospace text.

Keep app descriptors cheap. Organization retains its public OrganizationView through a lazy facade; its graph engine and shared canvas stylesheet must not be reachable eagerly from the shell manifest. Settings navigation/access checks stay eager, but section bodies load only after selection and their existing role gate. DeferredSurface uses the window host's effect/state loader rather than relying on a Suspense retry from an external-store update. Retain mounted presentation when local tab state or exit animations require it. Auto-lock, shortcut listeners, notifications and clipboard capture remain eager; only noncritical UI is deferred.

Dock/Launcher keep Next link prefetch={false}. Their separate app-module warming waits 250 ms for hover/keyboard intent, cancels pending timers on leave/blur/close, checks online/visibility/Save-Data/slow-network state, and deduplicates in-flight loads. Expensive descriptors opt out with prefetch: "never"; explicit app opens still use the normal loader. No project-specific app-id blacklist lives in AppShell.

Shared graph fitting waits for node measurements and uses both canvas width and height so short landscape windows keep their trigger readable. Only the responsive fit controller owns initial fitting; explicit Fit still shows the full graph. Geist Mono keeps its original local font, fallback stack and root CSS variable, with preload: false; no font CDN or external runtime dependency is introduced.

Optional background-removal engines/models, model quality defaults, production artifact packaging and build-cache retention are unchanged. Do not delete WASM or enable site-wide cross-origin isolation just to make a size number smaller.

Comparison governance

docs/comparison-data.json is the only hand-edited comparison source. It records criterion definitions, per-product notes, official source URLs, and repository evidence for every MSO rating. Do not hand-edit the generated README comparison reference or docs/COMPARISON.md. After an implementation or source review:

node scripts/gen-comparison.mjs
node scripts/gen-comparison.mjs --check

The checker validates MSO evidence paths, restricts competitor links to reviewed official hosts, requires the SEO-readable status words Strong, Partial, and Not offered, and expires the comparison after 90 days. A rating may improve only after its repository evidence exists. Specialist boundaries and the prioritized execution sequence live in docs/COMPETITIVE-ROADMAP.md.

Dependency update policy

Dependabot keeps production, development, and GitHub Actions updates visible, but a green security scan is not enough to merge a dependency change: the repository Verify job must pass too. Minor and patch updates are grouped to reduce PR noise. Major toolchain updates are evaluated separately because they can require source or configuration migrations.

eslint 10 and TypeScript 7 are temporarily ignored in .github/dependabot.yml: the current Next.js ESLint plugin stack fails under ESLint 10, and the current typescript-eslint stack rejects TypeScript 7. Remove an ignore only after the isolated candidate passes typecheck, lint, repository checks, and the out-of-tree production build. This is a compatibility hold, not a permanent version pin and not an exemption from Dependabot security updates.

Security scanners run as your own uid, never as root

Every docker run in .github/workflows/security-core.yml and any local reproduction of one needs --user "$(id -u):$(id -g)". This is not tidiness.

This repository is a partial clone (blob:none in .git/config). Run gitleaks as root against a read-write mount and its git sees missing blobs, decides the repo is corrupt, promisor-fetches during the scan, and rewrites refs and logs — as root. Observed once: 67 files under .git became root-owned, git commit then failed with unable to append to '.git/logs/HEAD': Permission denied, and sudo here needs an interactive password.

Recovery, without sudo, when it does happen:

docker run --rm -v "$PWD/.git:/g" alpine chown -R "$(id -u):$(id -g)" /g
git fsck --no-progress --connectivity-only   # dangling commits are normal; errors are not

Prefer :ro on the mount for scanners that only read, and --network=none for gitleaks, which never needs the network once the clone is complete.

Deploy — and the build hazard ⚠️

mso deploys via systemd on the VPS, not git push (no webhook, no Dokploy/ Vercel). The normal release path is:

bun run ship "feat(scope): describe the verified change"

That command commits, regenerates the changelog, runs the push gates and out-of-tree build, then builds in place, replaces the service process and verifies the served CSS chunk. When launched through MSO/MCP, the gated push stays attached but finalization is handed to the owner transient mso-self-update.service unit; otherwise replacing mso.service would terminate the MCP call that launched it. Track completion with:

systemctl --user is-active mso-self-update.service
tail -f ~/.mso/self-update.log     # success ends with UPDATE OK

A direct manual build-and-replace remains available for recovery, but it bypasses the release gates and should not be the normal path.

Committing by hand instead of through ship. The changelog lists commit subjects, so it can only be regenerated after the commit exists — which means any commit touching docs/ leaves docs/CHANGELOG.md stale by default. Run node scripts/gen-changelog.mjs and commit the result before considering the change finished, not after the gate tells you: every docs/ commit here has cost an extra round trip for forgetting exactly this. ship does it for you by folding the regenerated file into the same commit with --amend; if you commit by hand, expect two commits or one extra --amend.

A docs/-only commit is not exempt. The generator lists every commit subject except the ones that only regenerate this file, so a commit that edits prose still adds a line and still leaves docs/CHANGELOG.md stale. That is precisely how this rule got written: three separate docs/-only commits in one session each needed a second commit to regenerate the file.

bun run build now refuses when a live Next/MSO runtime serves the same checkout. next start loads the build manifest at boot; overwriting .next under the live process makes the already-served HTML reference chunk hashes that no longer exist on disk → every JS/CSS chunk 404/500s → the live site is broken until you restart.

Use mso build for the normal out-of-tree compile proof and mso deploy for a production replacement. To test runtime behaviour without risking prod, use a separate checkout / a demo instance on a different port — e.g. a build with NEXT_PUBLIC_OS_DEMO=1 (no login, no host access, forced mock data), served on :4006 via its own systemd unit. For a non-destructive static check, bun run typecheck && bun run lint is the cheap gate.

Recovery if a chunk mismatch is live: wait for any active self-update finalizer to finish, then run mso update --rebuild and re-run the post-deploy smoke check.

Package manager: bun installs, Node runs

bun is the installer. The runtime is still Node 22 — .nvmrc and engines.node mean what they say, and prod's ExecStart is npm run start. next, tsc, eslint and vitest all ship a #!/usr/bin/env node shebang, and bun run <script> honours a shebang, so the tools execute under Node exactly as before. Migrated from pnpm 10.32.1 on 2026-08-03; bun.lock is committed and pnpm-lock.yaml is gone.

Three things that will bite:

  • bun run test, never bun test. bun test is bun's own builtin runner. It shadows the test script, ignores vitest.config.mts (losing the **/zz-* exclude and the named root includes for proxy.test.ts / proxy-websocket.test.ts), cannot run suites that use vi.mock, and exits 0 having run nothing — so verify goes green while testing zero files. Same trap in CI.
  • node-pty must stay in trustedDependencies. It has no Linux prebuild, so it compiles at install time (hence the C++ toolchain + python3 requirement), and bun skips lifecycle scripts unless the package is trusted. Terminal code reaches it through the narrow lib/host/terminal-api.ts facade; unrelated routes no longer import a giant host barrel. Gate any dependency change on node -e "require('node-pty')" because PTY functionality is still release-critical.
  • Never bunx/bun x in a deploy script. Unlike pnpm exec, bunx downloads a missing package and runs it. On the box that serves an authenticated remote shell, that turns a capability check into a fetch-and-execute. Call node_modules/.bin/<tool> directly (see scripts/post-deploy-smoke.sh).

unrs-resolver and protobufjs postinstalls stay untrusted/blocked — both work from their prebuilt binaries. Don't "fix" the bun pm untrusted warning by trusting them. sharp was a third entry until 0.35.0 removed its install script entirely; do NOT add it to trustedDependencies to "restore" anything.

For deeper Files/Settings and Preview acceptance, run bash scripts/verify-build.sh --extended. It reuses the isolated production build and synthetic credentials; legacy browser scripts no longer read deployment secrets. Media samples skip only when ffmpeg or a system PDF is unavailable.

Next ESLint glob dependency

Next ESLint keeps fast-glob 3.3.1 and its filesystem traversal unchanged. Its only micromatch consumer uses scan/makeRe (forwarders to picomatch) and brace expansion. The exact npm alias to picomatch 2.3.2 retains the original matcher. A versioned Bun patch replaces only brace expansion with brace-expansion 5.0.12, removing braces. Both replacements are MIT-licensed maintained npm packages. All lint rules and security scans remain enabled without advisory exceptions. A matching patch for brace-expansion's CommonJS and ESM entrypoints turns every count, encoded-length, depth and rewrite limit into an error, preventing silent partial lint discovery.

Expansion fails closed above 4096 input characters, 32 nested brace levels or 1000 expanded results. Escape handling and deduplication remain covered by tests. The output-length budget is bounded by input length times the result limit. On dependency upgrades, recheck the sole consumer and the matching/expansion API, regenerate or remove the patch with bun patch, then run scripts/next-eslint-root-dirs.test.ts, a clean frozen install, raw audit and full gates. The regression fixtures preserve literal/relative, trailing-slash, recursive, brace, extglob, negative, hidden, symlink and Windows-normalized roots and actual Next link-rule enforcement. Broader tinyglobby/glob replacements were rejected because differential tests found behavior changes.

Locked native build bootstrap

The installer provisions node-gyp from the dedicated manifest and integrity-locked package-lock.json in scripts/install/node-gyp/. This does not change MSO's Bun package manager or add the build tool to the browser/runtime dependency graph. npm ci --ignore-scripts runs only in a private staging directory; successful payloads are published into a manifest-hash cache below MSO_NODE_GYP_PREFIX (or the normal user cache). Cache hits require matching manifests and the expected runner version. Failed downloads cannot publish a partial runner, replace the installer recovery trap, or remove the application's node_modules.

node-gyp 12.4.0 replaces make-fetch-happen with undici, removing http-cache-semantics. Its engines (^20.17.0 || >=22.9.0) cover MSO's Node range. node-gyp 13 requires ^22.22.2 || ^24.15.0 || >=26.0.0, narrower than ^22.12.0, so Dependabot continues ignoring node-gyp major updates.

Update the exact tool version in that manifest and regenerate its lock with npm install --package-lock-only --ignore-scripts --no-audit --no-fund --prefix scripts/install/node-gyp. Review the dependency diff, run npm audit --prefix scripts/install/node-gyp --omit=dev --audit-level=high, and run bun run test scripts/install-node-gyp.test.ts plus the installer lifecycle checks. CI and the scheduled dependency audit check this lock independently; Dependabot tracks its npm manifest separately. A local pass is not proof that a previously published GitHub security alert has closed.

Test dependency update boundary

bun run test and bun run coverage first run scripts/check-test-toolchain.mjs. The installed vitest and @vitest/coverage-v8 versions must match exactly, including patch versions; the guard does not download packages or lower coverage thresholds. Update both direct versions and bun.lock together. Dependabot's test-toolchain group keeps the runner and @vitest/* providers out of unrelated dependency groups.

The application, installer, doctor and Vitest 5 toolchain share the supported Node range ^22.12.0 || ^24.0.0 || >=26.0.0; see the official migration guide. The application runtime baseline remains .nvmrc; @types/node tracks that baseline, so moving its major requires an explicit runtime compatibility review. A successful compile with newer typings does not prove the APIs exist on the oldest supported runtime.