Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.
TypeScript browser-shellcontrol-planefile-managerhomelabmobile-firstnextjsptyreactself-hostedserver-managementshadcn-uisysadmintailwindcssterminaltypescriptvpsweb-desktopweb-terminal
fix(gateway): serialize offline recovery state
Rahman9ae226daaa47
diff --git a/SECURITY.md b/SECURITY.mdindex 4174c5ad..d51b681e 100644--- a/SECURITY.md+++ b/SECURITY.md@@ -32,6 +32,9 @@ application roles, but they still share the same deployment and underlying Unixaccess policy for a permanent Internet-facing control plane.- Keep `OS_FS_WRITE_ROOTS` narrow and review read roots deliberately.- Do not commit `.env.local`, model credentials or data under `~/.mso`.+- Gateway/update lifecycle state is owner-only and fail-closed: public tunneling requires verified+ loopback-only exposure; offline recovery uses checkout-scoped receipts, an exclusive process-identity+ lock, and durable pre-quiesce recovery intent rather than treating Git HEAD as deployment proof.- Use `NEXT_PUBLIC_OS_DEMO=1` only in a separate mock-only public demo checkout.The core gateway client follows the same immutable-artifact rule as managed apps: MSO pins thediff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.mdindex 488b3ce5..ff9ae89a 100644--- a/docs/CHANGELOG.md+++ b/docs/CHANGELOG.md@@ -13,6 +13,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.**Fixed**+- `gateway` serialize offline recovery state- `gateway` harden update and proxy edges- `gateway` close lifecycle review gaps- `security` harden gateway scanner fixturesdiff --git a/docs/INSTALL.md b/docs/INSTALL.mdindex 9c429de4..11a29761 100644--- a/docs/INSTALL.md+++ b/docs/INSTALL.md@@ -435,9 +435,13 @@ The updater then runs outside that service cgroup so it survives the restart. Wiservice (including WSL without systemd), it performs the clean fast-forward/dependency/verify/buildpath locally. If `mso web`/gateway owns the detached Next runtime, update first quiesces only thatverified runtime, leaves an active tunnel identity intact, rebuilds, then restores the runtime. A-private deployment receipt and restart marker mean a dependency/build failure after Git already-reached `origin/main` is retried by the next ordinary `mso update` instead of being mislabeled-"already up to date". Interactive CLI commands also show a throttled Git-backed update notice;+checkout-scoped private deployment receipt and restart marker mean a dependency/build failure after+Git already reached `origin/main` is retried by the next ordinary `mso update` instead of being+mislabeled "already up to date". The state is keyed by canonical checkout path, so two clones at the+same SHA cannot borrow each other's receipt. One owner-only transaction lock covers source/receipt+reconciliation through runtime restore and receipt write. Recovery intent is written to private state+before a gateway-owned runtime is quiesced, so an interrupted post-quiesce state update stays retryable.+Interactive CLI commands also show a throttled Git-backed update notice;the notice never depends on port 4005. A successful service finalizer log ends with `UPDATE OK`.If the installed source is correct but the production build tree is inconsistent, use:diff --git a/docs/PROGRESS.md b/docs/PROGRESS.mdindex c6b319e7..90ceffa0 100644--- a/docs/PROGRESS.md+++ b/docs/PROGRESS.md@@ -28,7 +28,11 @@ fallback runtime first. Re-approving the same device+role is idempotent, while r`mso device role`. CLI version is now 1.4.0. `mso update` is API-independent and is the preferredoperator command: it can fetch/fast-forward/verify/build even when port 4005 is down, and normalinteractive CLI use emits a throttled Git-backed update notice when `origin/main` is ahead.-`mso update run` remains accepted for compatibility.+`mso update run` remains accepted for compatibility. Final PR review then hardened crash/concurrency+recovery: offline deployment state is keyed by canonical checkout, the whole offline transaction is+serialized with a PID/start-ticks lock, recovery intent is durable before an owned runtime is quiesced,+and an interrupted post-quiesce state write is reconciled on retry. The shared private-state atomic+writer now propagates write/rename failures explicitly even when called from a shell conditional.The global response policy additionally sends `X-Robots-Tag: noindex, nofollow, noarchive` because anMSO control plane should not be indexed merely because a temporary HTTPS endpoint exists. Thediff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdindex 5da47c45..7ee220f6 100644--- a/docs/TROUBLESHOOTING.md+++ b/docs/TROUBLESHOOTING.md@@ -34,10 +34,12 @@ existing device role still requires the explicit `mso device role` command.Run `mso update`. The CLI updater reads/fetches `origin/main` directly and does not need the MSO API.On WSL without an active service it verifies and builds the clean updated checkout. If the fallbackruntime is gateway-owned, it is quiesced before `.next` changes and restored afterward while an-active tunnel identity is preserved. A private deployment receipt/restart marker survives partial-failures, so rerunning the same `mso update` retries dependency/build/restart work even when Git HEAD-already equals `origin/main`. Use `mso update status` for source/deployment state and `mso update log`-for the service-updater transcript.+active tunnel identity is preserved. Checkout-scoped private deployment receipts/restart markers+survive partial failures, so rerunning the same `mso update` retries dependency/build/restart work even+when Git HEAD already equals `origin/main`. Offline transactions are serialized; do not remove the+owner-only update-state directory just to bypass a pending recovery. Recovery intent is written before+a gateway-owned runtime is quiesced, so an interrupted state update remains safely retryable. Use+`mso update status` for source/deployment state and `mso update log` for the service-updater transcript.### `mso update` says the active service belongs to another checkoutdiff --git a/lib/security/private-shell-state.test.ts b/lib/security/private-shell-state.test.tsindex 2e60f807..70112b3d 100644--- a/lib/security/private-shell-state.test.ts+++ b/lib/security/private-shell-state.test.ts@@ -82,6 +82,20 @@ describe("private shell state", () => {expect(fs.statSync(file).mode & 0o777).toBe(0o600);});+ it("propagates atomic rename failure even when the caller invokes it from a conditional", () => {+ const root = tempRoot();+ const dir = path.join(root, "private"), bin = path.join(root, "bin");+ fs.mkdirSync(dir, { mode: 0o700 }); fs.mkdirSync(bin, { mode: 0o700 });+ const file = path.join(dir, "session.json");+ fs.writeFileSync(file, "old", { mode: 0o600 });+ fs.writeFileSync(path.join(bin, "mv"), "#!/bin/sh\nexit 33\n", { mode: 0o700 });+ const run = bash(`PATH=${JSON.stringify(bin)}:$PATH; if printf new | mso_private_state_atomic_write ${JSON.stringify(file)} >/dev/null; then echo unexpected; exit 91; else echo failed; fi`);+ expect(run.status).toBe(0);+ expect(run.stdout.trim()).toBe("failed");+ expect(fs.readFileSync(file, "utf8")).toBe("old");+ expect(fs.readdirSync(dir).filter((name) => name.startsWith(".mso-private-write."))).toEqual([]);+ });+it("wires both clients away from shared /tmp defaults and validates every bearer file", () => {const cli = fs.readFileSync(CLI, "utf8");const editor = fs.readFileSync(EDITOR, "utf8");diff --git a/scripts/lib/gateway-recovery.sh b/scripts/lib/gateway-recovery.shnew file mode 100755index 00000000..3e054dc2--- /dev/null+++ b/scripts/lib/gateway-recovery.sh@@ -0,0 +1,45 @@+#!/usr/bin/env bash+# Durable update-recovery intent shared only by the internal runtime-stop path.++gateway_recovery_marker_path() {+ local requested="${MSO_GATEWAY_RECOVERY_MARKER:-}"+ [ -n "$requested" ] || gateway_fail "runtime-stop requires an update recovery marker"+ case "$requested" in /*) ;; *) gateway_fail "update recovery marker must be an absolute path" ;; esac+ mso_private_state_path "$requested" || gateway_fail "unsafe update recovery marker path"+}++gateway_recovery_pending() {+ local marker+ marker="$(gateway_recovery_marker_path)"+ if [ ! -e "$marker" ] && [ ! -L "$marker" ]; then return 1; fi+ mso_private_state_validate_file "$marker" >/dev/null || gateway_fail "unsafe update recovery marker"+ [ "$(cat "$marker")" = 1 ] || gateway_fail "invalid update recovery marker"+}++gateway_mark_recovery_pending() {+ local marker+ marker="$(gateway_recovery_marker_path)"+ printf '1\n' | mso_private_state_atomic_write "$marker" >/dev/null \+ || gateway_fail "could not persist runtime recovery intent; runtime was NOT stopped"+ gateway_recovery_pending || gateway_fail "runtime recovery intent did not persist"+}++gateway_runtime_process_matches() {+ gateway_identity_matches_retry "$1"+}++gateway_runtime_identity_matches() {+ local identity="$1" instance+ instance="$(jq -r '.instanceId // empty' <<<"$identity" 2>/dev/null || true)"+ [ -n "$instance" ] && gateway_runtime_process_matches "$identity" \+ && gateway_health_url_ok "$LOCAL_URL" "$instance"+}++gateway_wait_runtime_stopped() {+ local identity="$1" i+ for i in $(seq 1 40); do+ gateway_runtime_process_matches "$identity" || return 0+ sleep 0.05+ done+ return 1+}diff --git a/scripts/lib/gateway-runtime.sh b/scripts/lib/gateway-runtime.shindex 156f8618..99222a01 100644--- a/scripts/lib/gateway-runtime.sh+++ b/scripts/lib/gateway-runtime.sh@@ -9,7 +9,8 @@ gateway_runtime_from_state() {owned="$(jq -r '.runtimeOwned // false' <<<"$state")"identity="$(jq -c '.runtimeIdentity // null' <<<"$state")"instance="$(jq -r '.runtimeIdentity.instanceId // empty' <<<"$state")"- if [ "$owned" = true ] && [ "$identity" != null ] && [ -n "$instance" ] && gateway_identity_matches_retry "$identity" && gateway_health_url_ok "$LOCAL_URL" "$instance"; then+ if [ "$owned" = true ] && [ "$identity" != null ] && [ -n "$instance" ] \+ && gateway_runtime_identity_matches "$identity"; thenRUNTIME_IDENTITY="$identity"; RUNTIME_INSTANCE_ID="$instance"; RUNTIME_OWNED=truefi}@@ -101,28 +102,50 @@ gateway_cmd_local_start_locked() {gateway_info "runtime: healthy MSO at $LOCAL_URL"}+gateway_persist_quiesced_state() {+ local state="$1" tunnel provider mode url+ tunnel="$(jq -c '.tunnelIdentity // null' <<<"$state")"+ provider="$(jq -r '.provider // "local"' <<<"$state")"+ mode="$(jq -r '.mode // "local"' <<<"$state")"+ url="$(jq -r --arg local "$LOCAL_URL" '.url // $local' <<<"$state")"+ RUNTIME_IDENTITY=null; RUNTIME_INSTANCE_ID=''; RUNTIME_OWNED=false; RUNTIME_STARTED_NOW=false+ gateway_write_state "$provider" "$mode" "$url" "$tunnel"+}+gateway_cmd_runtime_stop_locked() {- local state tunnel runtime owned provider mode url+ local state runtime owned+ gateway_recovery_marker_path >/dev/nullstate="$(gateway_state_read)"- tunnel="$(jq -c '.tunnelIdentity // null' <<<"$state")"runtime="$(jq -c '.runtimeIdentity // null' <<<"$state")"owned="$(jq -r '.runtimeOwned // false' <<<"$state")"if [ "$owned" = true ] && [ "$runtime" != null ]; then- gateway_runtime_from_state "$state"- [ "$RUNTIME_OWNED" = true ] || gateway_fail "recorded runtime ownership no longer matches a live MSO instance; refusing offline rebuild"- gateway_stop_identity "$runtime"- provider="$(jq -r '.provider // "local"' <<<"$state")"- mode="$(jq -r '.mode // "local"' <<<"$state")"- url="$(jq -r --arg local "$LOCAL_URL" '.url // $local' <<<"$state")"- RUNTIME_IDENTITY=null; RUNTIME_INSTANCE_ID=''; RUNTIME_OWNED=false; RUNTIME_STARTED_NOW=false- gateway_write_state "$provider" "$mode" "$url" "$tunnel" || gateway_fail "could not persist quiesced runtime state"- gateway_info "runtime: stopped-owned"- return 0+ if gateway_runtime_process_matches "$runtime"; then+ # PID + start ticks + Node executable prove this is the same recorded+ # process even if its HTTP listener is already unhealthy. Persist recovery+ # intent before terminating it; a failed state write is then retryable.+ gateway_mark_recovery_pending+ gateway_stop_identity "$runtime"+ gateway_wait_runtime_stopped "$runtime" || gateway_fail "owned runtime did not stop; recovery marker preserved"+ gateway_persist_quiesced_state "$state" \+ || gateway_fail "runtime stopped but quiesced state did not persist; recovery marker preserved for retry"+ gateway_info "runtime: stopped-owned"+ return 0+ fi+ # A prior attempt may have killed the verified runtime and then lost the+ # state write. Reconcile only when durable intent exists, the recorded+ # process identity is gone, and no other MSO responder owns the local port.+ if gateway_recovery_pending && ! gateway_runtime_process_matches "$runtime" && ! gateway_health_ok; then+ gateway_persist_quiesced_state "$state" \+ || gateway_fail "could not reconcile stale owned runtime; recovery marker preserved"+ gateway_info "runtime: recovered-stale-owned"+ return 0+ fi+ gateway_fail "recorded runtime ownership no longer matches a safely recoverable MSO instance; refusing offline rebuild"fiif gateway_health_ok; thengateway_fail "a loopback MSO runtime is active but is not gateway-owned; stop it before an offline update"fi- gateway_info "runtime: already-down"+ if gateway_recovery_pending; then gateway_info "runtime: recovery-pending"; else gateway_info "runtime: already-down"; fi}gateway_cmd_stop_locked() {diff --git a/scripts/lib/private-state.sh b/scripts/lib/private-state.shindex 11a9076a..7289c85e 100755--- a/scripts/lib/private-state.sh+++ b/scripts/lib/private-state.sh@@ -120,9 +120,9 @@ mso_private_state_atomic_write() {tmp=$(mktemp "$parent/.mso-private-write.XXXXXX")umask "$old_umask"trap 'rm -f -- "${tmp:-}"' RETURN- cat >"$tmp"- chmod 600 -- "$tmp"- mv -fT -- "$tmp" "$resolved"+ if ! cat >"$tmp"; then rm -f -- "$tmp"; tmp=''; trap - RETURN; return 1; fi+ if ! chmod 600 -- "$tmp"; then rm -f -- "$tmp"; tmp=''; trap - RETURN; return 1; fi+ if ! mv -fT -- "$tmp" "$resolved"; then rm -f -- "$tmp"; tmp=''; trap - RETURN; return 1; fitmp=''trap - RETURNmso_private_state_validate_file "$resolved"diff --git a/scripts/lib/update-state.sh b/scripts/lib/update-state.shnew file mode 100755index 00000000..6aabfeef--- /dev/null+++ b/scripts/lib/update-state.sh@@ -0,0 +1,107 @@+#!/usr/bin/env bash+# Checkout-scoped private deployment state + exclusive offline-update transaction lock.++UPDATE_STATE_DIR=''+UPDATE_CANONICAL_ROOT=''+DEPLOY_RECEIPT=''+RESTART_MARKER=''+UPDATE_LOCK_DIR=''+UPDATE_LOCK_HELD=0++update_proc_start_ticks() {+ local pid="$1" line rest+ [ "$pid" -gt 1 ] 2>/dev/null && kill -0 "$pid" 2>/dev/null || return 1+ IFS= read -r line <"/proc/$pid/stat" || return 1+ rest="${line##*) }"; set -- $rest+ [ "$#" -ge 20 ] || return 1+ printf '%s' "${20}"+}++init_update_state() {+ local base key+ [ -n "$UPDATE_STATE_DIR" ] && return 0+ UPDATE_CANONICAL_ROOT="$(realpath -e -- "$ROOT" 2>/dev/null || true)"+ [ -n "$UPDATE_CANONICAL_ROOT" ] || fail "cannot canonicalize update checkout: $ROOT"+ base="$(mso_private_state_dir "$UPDATE_STATE_DIR_REQUESTED")" || fail "unsafe update state directory"+ key="$(printf '%s' "$UPDATE_CANONICAL_ROOT" | sha256sum | awk '{print $1}')"+ [[ "$key" =~ ^[0-9a-f]{64}$ ]] || fail "cannot derive checkout-scoped update state key"+ UPDATE_STATE_DIR="$(mso_private_state_dir "$base/$key")" || fail "unsafe checkout-scoped update state directory"+ DEPLOY_RECEIPT="$UPDATE_STATE_DIR/deployed.json"+ RESTART_MARKER="$UPDATE_STATE_DIR/restart-runtime"+ UPDATE_LOCK_DIR="$UPDATE_STATE_DIR/transaction.lock"+}++read_receipt() {+ local root sha+ init_update_state+ if [ ! -e "$DEPLOY_RECEIPT" ] && [ ! -L "$DEPLOY_RECEIPT" ]; then return 1; fi+ mso_private_state_validate_file "$DEPLOY_RECEIPT" >/dev/null || fail "unsafe deployment receipt"+ root="$(jq -er '.root | select(type=="string")' "$DEPLOY_RECEIPT" 2>/dev/null || true)"+ sha="$(jq -er '.sha | select(type=="string")' "$DEPLOY_RECEIPT" 2>/dev/null || true)"+ [ "$root" = "$UPDATE_CANONICAL_ROOT" ] || fail "deployment receipt belongs to another checkout"+ [[ "$sha" =~ ^[0-9a-f]{40}$ ]] || fail "invalid deployment receipt sha"+ printf '%s' "$sha"+}++write_receipt() {+ local sha+ init_update_state+ sha="$(git -C "$ROOT" rev-parse HEAD)"+ jq -nc --arg root "$UPDATE_CANONICAL_ROOT" --arg sha "$sha" '{root:$root,sha:$sha}' \+ | mso_private_state_atomic_write "$DEPLOY_RECEIPT" >/dev/null+}++restart_pending() {+ init_update_state+ if [ ! -e "$RESTART_MARKER" ] && [ ! -L "$RESTART_MARKER" ]; then return 1; fi+ mso_private_state_validate_file "$RESTART_MARKER" >/dev/null || fail "unsafe runtime restart marker"+ [ "$(cat "$RESTART_MARKER")" = 1 ] || fail "invalid runtime restart marker"+}++clear_restart_pending() {+ init_update_state+ mso_private_state_remove_file "$RESTART_MARKER" >/dev/null 2>&1 || true+}++update_lock_owner_valid() {+ local owner="$UPDATE_LOCK_DIR/owner" pid ticks live+ [ -f "$owner" ] && [ ! -L "$owner" ] || return 1+ read -r pid ticks <"$owner" || return 1+ [[ "$pid" =~ ^[0-9]+$ && "$ticks" =~ ^[0-9]+$ ]] || return 1+ live="$(update_proc_start_ticks "$pid" 2>/dev/null || true)"+ [ -n "$live" ] && [ "$live" = "$ticks" ]+}++update_lock_acquire() {+ local self_ticks candidate stale i+ init_update_state+ self_ticks="$(update_proc_start_ticks $$)" || fail "cannot identify update process"+ for i in $(seq 1 1200); do+ candidate="$UPDATE_STATE_DIR/.transaction-lock.$$.$RANDOM"+ if ! mkdir -m 700 -- "$candidate" 2>/dev/null; then sleep 0.01; continue; fi+ printf '%s %s\n' "$$" "$self_ticks" >"$candidate/owner"; chmod 600 "$candidate/owner"+ if mv -T -- "$candidate" "$UPDATE_LOCK_DIR" 2>/dev/null; then UPDATE_LOCK_HELD=1; return 0; fi+ rm -rf -- "$candidate"+ if [ -d "$UPDATE_LOCK_DIR" ] && [ ! -L "$UPDATE_LOCK_DIR" ] && ! update_lock_owner_valid; then+ stale="$UPDATE_STATE_DIR/.transaction-stale.$$.$RANDOM"+ mv -T -- "$UPDATE_LOCK_DIR" "$stale" 2>/dev/null && rm -rf -- "$stale" || true+ fi+ sleep 0.05+ done+ fail "another offline update transaction is still running"+}++update_lock_release() {+ [ "$UPDATE_LOCK_HELD" = 1 ] || return 0+ if update_lock_owner_valid; then rm -f -- "$UPDATE_LOCK_DIR/owner"; rmdir -- "$UPDATE_LOCK_DIR" 2>/dev/null || true; fi+ UPDATE_LOCK_HELD=0+}++update_with_lock() {+ update_lock_acquire+ trap update_lock_release EXIT+ trap 'exit 129' HUP; trap 'exit 130' INT; trap 'exit 143' TERM+ "$@"+ update_lock_release+ trap - EXIT HUP INT TERM+}diff --git a/scripts/mso-gateway b/scripts/mso-gatewayindex b81c4608..0aea951b 100755--- a/scripts/mso-gateway+++ b/scripts/mso-gateway@@ -47,6 +47,8 @@ LOCK_HELD=0. "$ROOT/scripts/lib/gateway-tool.sh"# shellcheck source=scripts/lib/gateway-lock.sh. "$ROOT/scripts/lib/gateway-lock.sh"+# shellcheck source=scripts/lib/gateway-recovery.sh+. "$ROOT/scripts/lib/gateway-recovery.sh"# shellcheck source=scripts/lib/gateway-runtime.sh. "$ROOT/scripts/lib/gateway-runtime.sh"# shellcheck source=scripts/lib/gateway-tunnel.shdiff --git a/scripts/mso-gateway-lifecycle.test.ts b/scripts/mso-gateway-lifecycle.test.tsindex 78bcad92..177aa64a 100644--- a/scripts/mso-gateway-lifecycle.test.ts+++ b/scripts/mso-gateway-lifecycle.test.ts@@ -150,6 +150,42 @@ esacexpect(alive(tunnel.pid!)).toBe(false); expect(alive(runtime.pid!)).toBe(false);});+ it("persists recovery intent before stopping and reconciles stale ownership after a state-write failure", async () => {+ const f = fixture();+ const pid = Number(execFileSync("bash", ["-c", `nohup ${JSON.stringify(process.execPath)} -e \"process.title='next-server (fixture)';process.on('SIGTERM',()=>process.exit(0));setInterval(()=>{},1000)\" >/dev/null 2>&1 & echo $!`], { encoding: "utf8" }).trim());+ pids.add(pid);+ await new Promise((r) => setTimeout(r, 120));+ const rid = { ...identity(pid), cmdHash: null, instanceId: "fixture" };+ writeState(f.state, { provider: "cloudflare-quick", mode: "temporary", url: "https://life-fixture.trycloudflare.com",+ localUrl: "http://127.0.0.1:4005", tunnelIdentity: null, runtimeIdentity: rid, runtimeOwned: true, startedAt: "2026-01-01T00:00:00Z" });+ const recoveryDir = path.join(f.dir, "update-recovery");+ fs.mkdirSync(recoveryDir, { mode: 0o700 });+ const markerPath = path.join(recoveryDir, "restart-runtime");+ const mv = path.join(f.dir, "bin", "mv");+ fs.writeFileSync(mv, `#!/bin/sh+case "$*" in *mso-private-write*state.json*) exit 1;; esac+exec /bin/mv "$@"+`, { mode: 0o700 });+ const env = { ...f.env, MSO_GATEWAY_RECOVERY_MARKER: markerPath };+ const first = spawnSync(GATEWAY, ["runtime-stop"], { encoding: "utf8", env });+ expect(first.status).not.toBe(0);+ expect(first.stderr).toContain("recovery marker preserved");+ for (let i = 0; i < 80 && alive(pid); i++) await new Promise((r) => setTimeout(r, 20));+ expect(alive(pid)).toBe(false);+ expect(fs.statSync(markerPath).mode & 0o777).toBe(0o600);+ expect(fs.readFileSync(markerPath, "utf8").trim()).toBe("1");+ expect(JSON.parse(fs.readFileSync(path.join(f.state, "state.json"), "utf8")).runtimeOwned).toBe(true);++ fs.unlinkSync(mv);+ fs.writeFileSync(f.curl, "#!/bin/sh\nexit 7\n", { mode: 0o700 });+ const second = run(["runtime-stop"], env);+ expect(second).toContain("runtime: recovered-stale-owned");+ const reconciled = JSON.parse(fs.readFileSync(path.join(f.state, "state.json"), "utf8"));+ expect(reconciled.runtimeOwned).toBe(false);+ expect(reconciled.runtimeIdentity).toBeNull();+ expect(fs.readFileSync(markerPath, "utf8").trim()).toBe("1");+ });+it("serializes concurrent starts so only one public tunnel is spawned", async () => {const f = fixture(), startFile = f.startFile, env = f.env;const [a, b] = await Promise.all([asyncStart(env), asyncStart(env)]);diff --git a/scripts/mso-update b/scripts/mso-updateindex 865e5811..ef14b29c 100755--- a/scripts/mso-update+++ b/scripts/mso-update@@ -8,11 +8,10 @@ NOTICE_DIR="${MSO_UPDATE_NOTICE_DIR:-$HOME/.mso/private/update-notice}"UPDATE_STATE_DIR_REQUESTED="${MSO_UPDATE_STATE_DIR:-$HOME/.mso/private/update-state}"LOCAL_URL="${MSO_UPDATE_LOCAL_URL:-${MSO_GATEWAY_LOCAL_URL:-http://127.0.0.1:${MSO_PORT:-4005}}}"GATEWAY="$ROOT/scripts/mso-gateway"-UPDATE_STATE_DIR=''-DEPLOY_RECEIPT=''-RESTART_MARKER=''# shellcheck source=scripts/lib/private-state.sh. "$ROOT/scripts/lib/private-state.sh"+# shellcheck source=scripts/lib/update-state.sh+. "$ROOT/scripts/lib/update-state.sh"fail() { printf 'mso update: %s\n' "$*" >&2; exit 1; }git_root() { [ "$(git -C "$ROOT" rev-parse --show-toplevel 2>/dev/null || true)" = "$ROOT" ]; }@@ -42,43 +41,19 @@ require_service_checkout() {|| fail "active mso.service belongs to $canonical_service, not this checkout $canonical_root; run update from the service checkout"}-init_update_state() {- [ -n "$UPDATE_STATE_DIR" ] && return 0- UPDATE_STATE_DIR="$(mso_private_state_dir "$UPDATE_STATE_DIR_REQUESTED")" || fail "unsafe update state directory"- DEPLOY_RECEIPT="$UPDATE_STATE_DIR/deployed-sha"- RESTART_MARKER="$UPDATE_STATE_DIR/restart-runtime"-}--read_receipt() {- init_update_state- if [ ! -e "$DEPLOY_RECEIPT" ] && [ ! -L "$DEPLOY_RECEIPT" ]; then return 1; fi- mso_private_state_validate_file "$DEPLOY_RECEIPT" >/dev/null || fail "unsafe deployed-sha receipt"- local sha; sha="$(cat "$DEPLOY_RECEIPT")"- [[ "$sha" =~ ^[0-9a-f]{40}$ ]] || fail "invalid deployed-sha receipt"- printf '%s' "$sha"-}--write_receipt() {- init_update_state- git -C "$ROOT" rev-parse HEAD | mso_private_state_atomic_write "$DEPLOY_RECEIPT" >/dev/null-}--restart_pending() {- init_update_state- if [ ! -e "$RESTART_MARKER" ] && [ ! -L "$RESTART_MARKER" ]; then return 1; fi- mso_private_state_validate_file "$RESTART_MARKER" >/dev/null || fail "unsafe runtime restart marker"- [ "$(cat "$RESTART_MARKER")" = 1 ] || fail "invalid runtime restart marker"-}--mark_restart_pending() { printf '1\n' | mso_private_state_atomic_write "$RESTART_MARKER" >/dev/null; }-clear_restart_pending() { mso_private_state_remove_file "$RESTART_MARKER" >/dev/null 2>&1 || true; }-quiesce_offline_runtime() {local out[ -x "$GATEWAY" ] || return 0- out="$(MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$LOCAL_URL" "$GATEWAY" runtime-stop)" \+ init_update_state+ out="$(MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$LOCAL_URL" \+ MSO_GATEWAY_RECOVERY_MARKER="$RESTART_MARKER" "$GATEWAY" runtime-stop)" \|| fail "could not safely quiesce the local MSO runtime before rebuilding"- case "$out" in *'runtime: stopped-owned'*) mark_restart_pending ;; esac+ case "$out" in+ *'runtime: stopped-owned'*|*'runtime: recovered-stale-owned'*|*'runtime: recovery-pending'*)+ restart_pending || fail "gateway stopped/recovered an owned runtime without durable recovery intent" ;;+ *'runtime: already-down'*) ;;+ *) fail "unexpected gateway runtime-stop result" ;;+ esac}restore_offline_runtime() {@@ -126,8 +101,10 @@ notice() {"${local_v:-current}" "${remote_v:-newer}" "$n" "$([ "$n" = 1 ] || printf s)" >&2}-offline_build() {+offline_build_locked() {local rebuild_only="$1" n head receipt=''+ require_checkout+ service_is_active && fail "mso.service became active while waiting for the offline update lock; rerun mso update"init_update_stateif [ "$rebuild_only" != 1 ]; thenfetch_main@@ -154,6 +131,11 @@ offline_build() {printf 'updated to %s. No system service was active; run: mso web\n' "$(git -C "$ROOT" rev-parse --short HEAD)"}+offline_build() {+ init_update_state+ update_with_lock offline_build_locked "$1"+}+run_update() {local rebuild_only="$1" arg=""require_checkoutdiff --git a/scripts/mso-update.test.ts b/scripts/mso-update.test.tsindex fcc18b00..9cb555f8 100644--- a/scripts/mso-update.test.ts+++ b/scripts/mso-update.test.ts@@ -1,4 +1,4 @@-import { execFileSync } from "node:child_process";+import { execFileSync, spawn, spawnSync } from "node:child_process";import fs from "node:fs";import os from "node:os";import path from "node:path";@@ -6,18 +6,29 @@ import { afterEach, describe, expect, it } from "vitest";const SCRIPT = path.join(process.cwd(), "scripts/mso-update");const PRIVATE = path.join(process.cwd(), "scripts/lib/private-state.sh");+const UPDATE_STATE = path.join(process.cwd(), "scripts/lib/update-state.sh");const roots: string[] = [];function git(cwd: string, ...args: string[]) { return execFileSync("git", args, { cwd, encoding: "utf8" }).trim(); }-function fixture(options: { failInstallOnce?: boolean; activeService?: "same" | "other" } = {}) {+function fixture(options: { failInstallOnce?: boolean; activeService?: "same" | "other"; installDelayMs?: number } = {}) {const root = fs.mkdtempSync(path.join(os.tmpdir(), "mso-update-")); roots.push(root);const repo = path.join(root, "repo"), remote = path.join(root, "remote.git"), bin = path.join(root, "bin"), capture = path.join(root, "capture");fs.mkdirSync(path.join(repo, "scripts/lib"), { recursive: true }); fs.mkdirSync(path.join(repo, "bin")); fs.mkdirSync(bin);fs.copyFileSync(PRIVATE, path.join(repo, "scripts/lib/private-state.sh"));+ fs.copyFileSync(UPDATE_STATE, path.join(repo, "scripts/lib/update-state.sh"));fs.writeFileSync(path.join(repo, "scripts/verify-build.sh"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });fs.writeFileSync(path.join(repo, "scripts/mso-gateway"), `#!/bin/shprintf 'gateway %s\\n' "$*" >> "${capture}"-case "$1" in runtime-stop) echo 'runtime: stopped-owned' ;; local-start) echo 'runtime: healthy MSO at fixture' ;; *) exit 2 ;; esac+case "$1" in+ runtime-stop)+ [ -n "\${MSO_GATEWAY_RECOVERY_MARKER:-}" ] || exit 9+ mkdir -p "$(dirname "\$MSO_GATEWAY_RECOVERY_MARKER")"+ chmod 700 "$(dirname "\$MSO_GATEWAY_RECOVERY_MARKER")"+ printf '1\\n' > "\$MSO_GATEWAY_RECOVERY_MARKER"; chmod 600 "\$MSO_GATEWAY_RECOVERY_MARKER"+ echo 'runtime: stopped-owned' ;;+ local-start) echo 'runtime: healthy MSO at fixture' ;;+ *) exit 2 ;;+esac`, { mode: 0o755 });fs.writeFileSync(path.join(repo, "bin/mso"), '#!/bin/sh\nVERSION="1.3.0"\n', { mode: 0o755 });fs.mkdirSync(path.join(repo, "node_modules/next/dist/bin"), { recursive: true });@@ -39,13 +50,32 @@ exit 3const failOnce = path.join(root, "fail-install-once");fs.writeFileSync(path.join(bin, "bun"), `#!/bin/shprintf 'bun %s\\n' "$*" >> "${capture}"+if [ "${options.installDelayMs ?? 0}" -gt 0 ] && [ "$1" = install ]; then sleep ${Math.max(0, (options.installDelayMs ?? 0) / 1000)}; fiif [ "${options.failInstallOnce ? "1" : "0"}" = 1 ] && [ "$1" = install ] && [ ! -f "${failOnce}" ]; then touch "${failOnce}"; exit 23; fi`, { mode: 0o755 });fs.writeFileSync(path.join(bin, "node"), `#!/bin/sh\nprintf 'node %s\\n' "$*" >> "${capture}"\n`, { mode: 0o755 });const env = { ...process.env, HOME: path.join(root, "home"), PATH: `${bin}:${process.env.PATH}`, MSO_UPDATE_ROOT: repo,MSO_UPDATE_NOTICE_DIR: path.join(root, "notice"), MSO_UPDATE_STATE_DIR: path.join(root, "update-state"),MSO_UPDATE_LOCAL_URL: "http://127.0.0.1:4555" };- return { root, repo, old, newer, capture, env };+ return { root, repo, remote, old, newer, capture, env };+}++function receipts(base: string) {+ if (!fs.existsSync(base)) return [] as Array<{ path: string; root: string; sha: string }>;+ return fs.readdirSync(base, { withFileTypes: true }).filter((e) => e.isDirectory()).flatMap((e) => {+ const file = path.join(base, e.name, "deployed.json");+ if (!fs.existsSync(file)) return [];+ return [{ path: file, ...JSON.parse(fs.readFileSync(file, "utf8")) as { root: string; sha: string } }];+ });+}++function runAsync(env: NodeJS.ProcessEnv) {+ return new Promise<{ code: number | null; stdout: string; stderr: string }>((resolve) => {+ const child = spawn(SCRIPT, [], { env, stdio: ["ignore", "pipe", "pipe"] });+ let stdout = "", stderr = "";+ child.stdout.on("data", (v) => stdout += v); child.stderr.on("data", (v) => stderr += v);+ child.on("close", (code) => resolve({ code, stdout, stderr }));+ });}afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });@@ -91,8 +121,45 @@ describe("mso update without a running web API", () => {const calls = fs.readFileSync(f.capture, "utf8");expect(calls.match(/bun install/g)?.length).toBe(2);expect(calls).toContain("gateway local-start");- const receipt = fs.readFileSync(path.join(f.root, "update-state", "deployed-sha"), "utf8").trim();- expect(receipt).toBe(f.newer);+ const rs = receipts(path.join(f.root, "update-state"));+ expect(rs).toHaveLength(1);+ expect(rs[0].root).toBe(fs.realpathSync(f.repo));+ expect(rs[0].sha).toBe(f.newer);+ });++ it("scopes deployment receipts to the canonical checkout, even at the same commit", () => {+ const f = fixture();+ execFileSync(SCRIPT, [], { env: f.env, encoding: "utf8" });+ const repoB = path.join(f.root, "repo-b");+ execFileSync("git", ["clone", "-q", f.remote, repoB]);+ const envB = { ...f.env, MSO_UPDATE_ROOT: repoB };+ const before = fs.readFileSync(f.capture, "utf8");+ const out = execFileSync(SCRIPT, [], { env: envB, encoding: "utf8" });+ expect(out).toContain("No system service was active");+ const after = fs.readFileSync(f.capture, "utf8");+ expect((after.match(/bun install/g) ?? []).length).toBe((before.match(/bun install/g) ?? []).length + 1);+ const rs = receipts(path.join(f.root, "update-state"));+ expect(rs).toHaveLength(2);+ expect(new Set(rs.map((r) => r.root))).toEqual(new Set([fs.realpathSync(f.repo), fs.realpathSync(repoB)]));+ expect(rs.every((r) => r.sha === f.newer)).toBe(true);+ });++ it("serializes the complete offline update so a concurrent caller cannot rebuild under a restored runtime", async () => {+ const f = fixture({ installDelayMs: 700 });+ const first = runAsync(f.env);+ for (let i = 0; i < 100; i++) {+ if (fs.existsSync(f.capture) && fs.readFileSync(f.capture, "utf8").includes("bun install")) break;+ await new Promise((r) => setTimeout(r, 20));+ }+ const second = runAsync(f.env);+ const [a, b] = await Promise.all([first, second]);+ expect(a.code).toBe(0); expect(b.code).toBe(0);+ const calls = fs.readFileSync(f.capture, "utf8");+ expect((calls.match(/bun install/g) ?? [])).toHaveLength(1);+ expect((calls.match(/node node_modules\/next\/dist\/bin\/next build/g) ?? [])).toHaveLength(1);+ expect((calls.match(/gateway runtime-stop/g) ?? [])).toHaveLength(1);+ expect((calls.match(/gateway local-start/g) ?? [])).toHaveLength(1);+ expect([a.stdout, b.stdout].join("\n")).toContain("offline deployment receipt matches");});it("prints an update notice from cached Git state without touching the web API", () => {