Skip to content

armgit / mso

Public Imported from GitHub
0

Browser-based graphical shell and control plane for a Linux server you own — real terminal, file manager, live metrics and BYOK AI in one mobile-first workspace. Next.js 16 · React 19 · node-pty · Tailwind v4 · shadcn/ui. Public alpha, self-hosted.

fix(update): serialize all checkout runtimes

Rahman29991a759544
diff --git a/README.md b/README.md
index 562ae5a6..a2a8e61e 100644
--- a/README.md
+++ b/README.md
@@ -175,7 +175,7 @@ If it explicitly says the current shell cannot see the user launcher yet, run th
**WSL2:** the CLI is supported even when systemd is not active. In that case the installer now
finishes the CLI instead of failing in `systemctl`, but it skips `mso.service`. `mso onboard` and
`mso web` can start the already-built production runtime detached on `127.0.0.1` and track ownership
-under private state; `mso update` can update/verify/build without a running API and safely quiesces/restores a gateway-owned fallback runtime instead of rebuilding `.next` underneath it. For the full
+under private state; `mso update` can update/verify/build without a running API and safely inventories, quiesces and restores every gateway-owned fallback runtime for that checkout instead of rebuilding `.next` underneath any of them. For the full
background service, enable systemd in `/etc/wsl.conf`, exit all WSL sessions from Windows, reopen
the distro, then re-run the installer with `--onboard`.
diff --git a/SECURITY.md b/SECURITY.md
index 4a233726..de0b17c0 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -39,7 +39,7 @@ application roles, but they still share the same deployment and underlying Unix
Cloudflared nor Next can execute before its process lifetime is tracked; interruption before release
makes the held child self-exit. Gateway state is namespaced by canonical checkout + loopback origin,
lifecycle/update serialization uses crash-released kernel `flock`; a checkout-wide runtime
- exclusion prevents `mso web`/gateway runtime starts during offline `.next` mutation, and a live
+ exclusion prevents `mso web`/gateway runtime starts during any in-place `.next` mutation; service-active and offline updates inventory/quiesce every checkout-owned fallback runtime, and a live
tunnel with a dead local runtime is reconciled before reuse. Public readiness must match the exact
local build/runtime identity rather than only the MSO version.
- Use `NEXT_PUBLIC_OS_DEMO=1` only in a separate mock-only public demo checkout.
diff --git a/bin/mso-update-offline.test.ts b/bin/mso-update-offline.test.ts
index 939dece8..0bcd30dd 100644
--- a/bin/mso-update-offline.test.ts
+++ b/bin/mso-update-offline.test.ts
@@ -33,6 +33,7 @@ function fixture() {
"scripts/lib/private-state.sh",
"scripts/lib/update-state.sh",
"scripts/lib/runtime-exclusion.sh",
+ "scripts/lib/update-gateway-runtimes.sh",
]) copy(repo, rel);
fs.writeFileSync(
path.join(fakebin, "curl"),
diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md
index 92ace847..e9e37144 100644
--- a/docs/CHANGELOG.md
+++ b/docs/CHANGELOG.md
@@ -13,6 +13,7 @@ this is the *what*, and it is what Settings → About shows as “What's new”.
**Fixed**
+- `update` serialize all checkout runtimes
- `gateway` reject active named tunnel switches
- `gateway` serialize runtime recovery and updates
- `gateway` close final lifecycle boundaries
diff --git a/docs/INSTALL.md b/docs/INSTALL.md
index 23319a8b..09b01625 100644
--- a/docs/INSTALL.md
+++ b/docs/INSTALL.md
@@ -447,13 +447,11 @@ The updater verifies the incoming checkout/build before replacing the service. W
that invoked `mso update`; a secondary clone is never allowed to restart an unrelated live service.
The updater then runs outside that service cgroup so it survives the restart. With no active
service (including WSL without systemd), it performs the clean fast-forward/dependency/verify/build
-path locally. If `mso web`/gateway owns the detached Next runtime, update first quiesces only that
-verified runtime, leaves an active tunnel identity intact, rebuilds, then restores the runtime. A
+path locally. Before either the service-active or offline in-place build, update inventories every gateway state for the canonical checkout, quiesces each verified owned fallback runtime, leaves active tunnel identities intact, rebuilds, then restores every previously-owned origin. A
checkout-scoped private deployment receipt and restart marker mean a dependency/build failure after
Git already reached `origin/main` is retried by the next ordinary `mso update` instead of being
mislabeled "already up to date". The state is keyed by canonical checkout path, so two clones at the
-same SHA cannot borrow each other's receipt. One owner-only transaction lock covers source/receipt
-reconciliation through runtime restore and receipt write. Recovery intent is written to private state
+same SHA cannot borrow each other's receipt. One owner-only transaction lock covers every service-active or offline in-place update, from source/receipt reconciliation through gateway runtime restore and receipt write. Recovery intent is written to private state
before a gateway-owned runtime is quiesced, so an interrupted post-quiesce state update stays retryable.
Interactive CLI commands also show a throttled Git-backed update notice;
the notice never depends on port 4005. A successful service finalizer log ends with `UPDATE OK`.
diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md
index 692ce2b7..671cd860 100644
--- a/docs/PROGRESS.md
+++ b/docs/PROGRESS.md
@@ -29,8 +29,7 @@ fallback runtime first. Re-approving the same device+role is idempotent, while r
operator command: it can fetch/fast-forward/verify/build even when port 4005 is down, and normal
interactive CLI use emits a throttled Git-backed update notice when `origin/main` is ahead.
`mso update run` remains accepted for compatibility. Final PR review then hardened crash/concurrency
-recovery: offline deployment state is keyed by canonical checkout, the whole offline transaction is
-serialized with a PID/start-ticks lock, recovery intent is durable before an owned runtime is quiesced,
+recovery: deployment state is keyed by canonical checkout, update transactions are serialized with kernel `flock`, recovery intent is durable before an owned runtime is quiesced,
and an interrupted post-quiesce state write is reconciled on retry. The shared private-state atomic
writer now propagates write/rename failures explicitly even when called from a shell conditional. A
full pre-push run then exposed a fork/signal race before the tunnel fingerprint existed. Tunnel launch
@@ -41,8 +40,7 @@ gateway/update serialization to kernel `flock` (no stale-lock ABA reclaim), scop
canonical checkout + selected loopback origin, and made public readiness compare the exact local
`version + buildId + runtimeInstanceId`. This closes the cross-clone control and wrong-deployment
readiness classes without widening the raw app bind. The last P1 review pass added a checkout-wide
-shared/exclusive runtime exclusion: offline update owns it exclusively through `.next` mutation and
-deployment-receipt persistence, while all runtime-start paths take the shared side. An already-live
+shared/exclusive runtime exclusion: every in-place update owns it exclusively through `.next` mutation and deployment-state persistence, while all runtime-start paths take the shared side. Service-active updates now inventory and restore all gateway-owned fallback origins for the checkout instead of assuming only one runtime exists. An already-live
tunnel is no longer treated as sufficient proof of readiness; `gateway start` recovers a dead local
runtime, preserves the tunnel PID, and re-probes the exact public health identity before success.
diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.md
index 9241b5c0..1b999fed 100644
--- a/docs/TROUBLESHOOTING.md
+++ b/docs/TROUBLESHOOTING.md
@@ -32,9 +32,7 @@ existing device role still requires the explicit `mso device role` command.
### How do I update when the web UI / port 4005 is down?
Run `mso update`. The CLI updater reads/fetches `origin/main` directly and does not need the MSO API.
-On WSL without an active service it verifies and builds the clean updated checkout. If the fallback
-runtime is gateway-owned, it is quiesced before `.next` changes and restored afterward while an
-active tunnel identity is preserved. Checkout-scoped private deployment receipts/restart markers
+On WSL without an active service it verifies and builds the clean updated checkout. Every gateway-owned fallback runtime for that canonical checkout is inventoried and quiesced before `.next` changes, then restored afterward while active tunnel identities are preserved. Checkout-scoped private deployment receipts/restart markers
survive partial failures, so rerunning the same `mso update` retries dependency/build/restart work even
when Git HEAD already equals `origin/main`. Offline transactions are serialized; do not remove the
owner-only update-state directory just to bypass a pending recovery. Recovery intent is written before
@@ -49,8 +47,7 @@ by the installer) instead of letting a secondary clone rebuild itself and restar
### `mso gateway start` says an offline update is mutating this checkout
-This is an intentional safety exclusion. An offline `mso update` holds the checkout-wide runtime lock
-while `.next` can change, so `mso web`, onboarding fallback, and gateway runtime recovery cannot start
+This is an intentional safety exclusion. Every in-place `mso update` (service-active or offline) holds the checkout-wide runtime lock while `.next` can change, so `mso web`, onboarding fallback, and gateway runtime recovery cannot start
Next from a partially-mutated build. Let the update finish, then rerun the same gateway/web command.
### `mso gateway start` cannot install or verify cloudflared
diff --git a/lib/host/self-update.test.ts b/lib/host/self-update.test.ts
index 1a170d05..e7b356f0 100644
--- a/lib/host/self-update.test.ts
+++ b/lib/host/self-update.test.ts
@@ -54,7 +54,7 @@ describe("updateUnitArgs", () => {
expect(args.slice(0, 2)).toEqual(["--user", "--collect"]);
expect(args).toContain("--unit=mso-self-update");
expect(args).toContain("--property=WorkingDirectory=/srv/mso");
- expect(args).toContain("/srv/mso/scripts/self-update.sh");
+ expect(args).toContain("/srv/mso/scripts/mso-service-update");
expect(args).not.toContain("--rebuild-only");
expect(args.some((arg) => arg.startsWith("--property=User="))).toBe(false);
expect(args.join(" ")).not.toContain("sudo");
diff --git a/lib/host/self-update.ts b/lib/host/self-update.ts
index 84e458c4..02b1ba33 100644
--- a/lib/host/self-update.ts
+++ b/lib/host/self-update.ts
@@ -78,7 +78,7 @@ export function updateUnitArgs(root: string, logPath: string, rebuildOnly = fals
"--property=TimeoutStartSec=3600",
`--setenv=MSO_UPDATE_LOG=${logPath}`,
"/bin/bash",
- path.join(root, "scripts", "self-update.sh"),
+ path.join(root, "scripts", "mso-service-update"),
...(rebuildOnly ? ["--rebuild-only"] : []),
];
}
@@ -268,7 +268,7 @@ export async function startUpdate(rebuildOnly = false): Promise<UpdateStatus> {
const blocked = blockingReason(status, rebuildOnly);
if (blocked) throw new HostError(blocked);
- const script = path.join(repoRoot(), "scripts", "self-update.sh");
+ const script = path.join(repoRoot(), "scripts", "mso-service-update");
if (!(await fs.stat(script).catch(() => null))) throw new HostError(`missing ${script}`);
const started = await run(
diff --git a/lib/host/ship-handoff.test.ts b/lib/host/ship-handoff.test.ts
index 22b4bc46..4be49811 100644
--- a/lib/host/ship-handoff.test.ts
+++ b/lib/host/ship-handoff.test.ts
@@ -15,7 +15,7 @@ let sha = "";
beforeAll(async () => {
await fs.mkdir(path.join(repo, "scripts"), { recursive: true });
- await fs.writeFile(path.join(repo, "scripts", "self-update.sh"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });
+ await fs.writeFile(path.join(repo, "scripts", "mso-service-update"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });
await run("git", ["init", "-q", "-b", "main"], { cwd: repo });
await run("git", ["config", "user.name", "MSO Test"], { cwd: repo });
await run("git", ["config", "user.email", "[email protected]"], { cwd: repo });
@@ -60,7 +60,7 @@ describe("MCP ship handoff", () => {
`--setenv=MSO_UPDATE_LOG=${log}`,
`--setenv=MSO_EXPECTED_SHA=${sha}`,
"/bin/bash",
- path.join(repo, "scripts", "self-update.sh"),
+ path.join(repo, "scripts", "mso-service-update"),
"--ship-finalize",
]);
expect(stdout).toContain(`release_sha=${sha}`);
diff --git a/scripts/lib/update-gateway-runtimes.sh b/scripts/lib/update-gateway-runtimes.sh
new file mode 100755
index 00000000..22632dd2
--- /dev/null
+++ b/scripts/lib/update-gateway-runtimes.sh
@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+# Checkout-wide inventory of gateway-owned fallback runtimes around in-place builds.
+
+update_gateway_restore_pending() {
+ init_update_state
+ if [ -e "$GATEWAY_RESTORE_FILE" ] || [ -L "$GATEWAY_RESTORE_FILE" ]; then
+ mso_private_state_validate_file "$GATEWAY_RESTORE_FILE" >/dev/null || fail "unsafe gateway runtime restore inventory"
+ jq -e 'type == "array" and all(.[]; type == "string" and length > 0)' "$GATEWAY_RESTORE_FILE" >/dev/null 2>&1 \
+ || fail "invalid gateway runtime restore inventory"
+ [ "$(jq 'length' "$GATEWAY_RESTORE_FILE")" -gt 0 ] && return 0
+ fi
+ restart_pending
+}
+
+update_gateway_restore_urls() {
+ init_update_state
+ if [ -e "$GATEWAY_RESTORE_FILE" ] || [ -L "$GATEWAY_RESTORE_FILE" ]; then
+ mso_private_state_validate_file "$GATEWAY_RESTORE_FILE" >/dev/null || fail "unsafe gateway runtime restore inventory"
+ jq -er 'if type == "array" and all(.[]; type == "string" and length > 0) then .[] else error("invalid") end' \
+ "$GATEWAY_RESTORE_FILE" 2>/dev/null || fail "invalid gateway runtime restore inventory"
+ elif restart_pending; then
+ # Backward compatibility with the single-runtime marker used by older MSO.
+ printf '%s\n' "${LOCAL_URL:-${MSO_GATEWAY_LOCAL_URL:-http://127.0.0.1:${MSO_PORT:-4005}}}"
+ fi
+}
+
+update_gateway_owned_urls() {
+ local base dir file state root owned url
+ init_update_state
+ base="$(mso_private_state_dir "${MSO_GATEWAY_STATE_BASE:-$HOME/.mso/private/gateway}")" \
+ || fail "unsafe gateway state base"
+ shopt -s nullglob
+ for dir in "$base"/*; do
+ [ -d "$dir" ] && [ ! -L "$dir" ] || continue
+ file="$dir/state.json"
+ [ -e "$file" ] || [ -L "$file" ] || continue
+ mso_private_state_validate_file "$file" >/dev/null || fail "unsafe gateway state while preparing update"
+ state="$(jq -ce . "$file" 2>/dev/null)" || fail "corrupt gateway state while preparing update"
+ root="$(jq -er '.root | select(type=="string")' <<<"$state" 2>/dev/null || true)"
+ [ "$root" = "$UPDATE_CANONICAL_ROOT" ] || continue
+ owned="$(jq -r '.runtimeOwned // false' <<<"$state")"
+ [ "$owned" = true ] || continue
+ url="$(jq -er '.localUrl | select(type=="string" and length>0)' <<<"$state" 2>/dev/null || true)"
+ [ -n "$url" ] || fail "gateway-owned runtime is missing its loopback origin"
+ printf '%s\n' "$url"
+ done
+ shopt -u nullglob
+}
+
+update_gateway_collect_urls() {
+ local restored owned
+ restored="$(update_gateway_restore_urls)" || return 1
+ owned="$(update_gateway_owned_urls)" || return 1
+ printf '%s\n%s\n' "$restored" "$owned" | awk 'NF && !seen[$0]++'
+}
+
+update_gateway_write_restore_urls() {
+ local payload
+ payload="$(cat | jq -Rsc 'split("\n") | map(select(length > 0)) | unique')" || fail "could not encode gateway restore inventory"
+ printf '%s\n' "$payload" | mso_private_state_atomic_write "$GATEWAY_RESTORE_FILE" >/dev/null \
+ || fail "could not persist gateway restore inventory"
+}
+
+update_gateway_quiesce_all() {
+ local url out collected
+ local -a urls=()
+ init_update_state
+ collected="$(update_gateway_collect_urls)" || fail "could not inventory checkout gateway runtimes"
+ [ -n "$collected" ] || return 0
+ mapfile -t urls <<<"$collected"
+ printf '%s\n' "${urls[@]}" | update_gateway_write_restore_urls
+ for url in "${urls[@]}"; do
+ out="$(MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$url" \
+ MSO_GATEWAY_RECOVERY_MARKER="$RESTART_MARKER" "$GATEWAY" runtime-stop)" \
+ || fail "could not safely quiesce gateway runtime at $url"
+ case "$out" in
+ *'runtime: stopped-owned'*|*'runtime: recovered-stale-owned'*|*'runtime: recovery-pending'*|*'runtime: already-down'*) ;;
+ *) fail "unexpected gateway runtime-stop result for $url" ;;
+ esac
+ done
+}
+
+update_gateway_restore_all() {
+ local url restored
+ local -a urls=()
+ init_update_state
+ restored="$(update_gateway_restore_urls)" || fail "could not read gateway runtime restore inventory"
+ [ -n "$restored" ] || { clear_restart_pending; return 0; }
+ mapfile -t urls <<<"$restored"
+ for url in "${urls[@]}"; do
+ MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$url" \
+ "$GATEWAY" local-start >/dev/null || fail "updated build is healthy, but fallback runtime restore failed at $url"
+ done
+ clear_restart_pending
+ mso_private_state_remove_file "$GATEWAY_RESTORE_FILE" >/dev/null 2>&1 || true
+}
diff --git a/scripts/lib/update-state.sh b/scripts/lib/update-state.sh
index d76d81ac..a38d44c0 100755
--- a/scripts/lib/update-state.sh
+++ b/scripts/lib/update-state.sh
@@ -5,6 +5,7 @@ UPDATE_STATE_DIR=''
UPDATE_CANONICAL_ROOT=''
DEPLOY_RECEIPT=''
RESTART_MARKER=''
+GATEWAY_RESTORE_FILE=''
UPDATE_LOCK_DIR=''
UPDATE_LOCK_HELD=0
UPDATE_LOCK_FD=''
@@ -29,6 +30,7 @@ init_update_state() {
UPDATE_STATE_DIR="$(mso_private_state_dir "$base/$key")" || fail "unsafe checkout-scoped update state directory"
DEPLOY_RECEIPT="$UPDATE_STATE_DIR/deployed.json"
RESTART_MARKER="$UPDATE_STATE_DIR/restart-runtime"
+ GATEWAY_RESTORE_FILE="$UPDATE_STATE_DIR/gateway-runtimes.json"
UPDATE_LOCK_DIR="$UPDATE_STATE_DIR/transaction.lock"
}
diff --git a/scripts/mso-service-update b/scripts/mso-service-update
new file mode 100755
index 00000000..7ef771b3
--- /dev/null
+++ b/scripts/mso-service-update
@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+# Serialize an in-place service update with every checkout-owned fallback runtime.
+set -euo pipefail
+umask 077
+
+ROOT="${MSO_UPDATE_ROOT:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)}"
+UPDATE_STATE_DIR_REQUESTED="${MSO_UPDATE_STATE_DIR:-$HOME/.mso/private/update-state}"
+LOCAL_URL="${MSO_UPDATE_LOCAL_URL:-${MSO_GATEWAY_LOCAL_URL:-http://127.0.0.1:${MSO_PORT:-4005}}}"
+GATEWAY="$ROOT/scripts/mso-gateway"
+SELF_UPDATE="$ROOT/scripts/self-update.sh"
+
+fail() { printf 'mso service update: %s\n' "$*" >&2; exit 1; }
+[ -x "$GATEWAY" ] || fail "missing gateway lifecycle helper: $GATEWAY"
+[ -f "$SELF_UPDATE" ] || fail "missing service updater: $SELF_UPDATE"
+
+# shellcheck source=scripts/lib/private-state.sh
+. "$ROOT/scripts/lib/private-state.sh"
+# shellcheck source=scripts/lib/update-state.sh
+. "$ROOT/scripts/lib/update-state.sh"
+# shellcheck source=scripts/lib/runtime-exclusion.sh
+. "$ROOT/scripts/lib/runtime-exclusion.sh"
+# shellcheck source=scripts/lib/update-gateway-runtimes.sh
+. "$ROOT/scripts/lib/update-gateway-runtimes.sh"
+
+cleanup_locks() {
+ runtime_exclusion_release || true
+ update_lock_release || true
+}
+trap cleanup_locks EXIT
+trap 'exit 129' HUP; trap 'exit 130' INT; trap 'exit 143' TERM
+
+update_lock_acquire
+runtime_exclusion_acquire_exclusive \
+ || fail "could not acquire checkout runtime exclusion before service update"
+update_gateway_quiesce_all
+
+if ! /bin/bash "$SELF_UPDATE" "$@"; then
+ # Do not restart fallback runtimes against an in-place tree whose build may have
+ # failed. The private restore inventory survives and the next successful update
+ # will restore every previously-owned origin.
+ fail "service update failed; fallback runtime restore intent was preserved"
+fi
+
+runtime_exclusion_release
+update_gateway_restore_all
+update_lock_release
+trap - EXIT HUP INT TERM
diff --git a/scripts/mso-service-update.test.ts b/scripts/mso-service-update.test.ts
new file mode 100644
index 00000000..459f839f
--- /dev/null
+++ b/scripts/mso-service-update.test.ts
@@ -0,0 +1,94 @@
+import { execFileSync, spawn, spawnSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import fs from "node:fs";
+import os from "node:os";
+import path from "node:path";
+import { afterEach, describe, expect, it } from "vitest";
+
+const WRAPPER = path.join(process.cwd(), "scripts/mso-service-update");
+const PRIVATE = path.join(process.cwd(), "scripts/lib/private-state.sh");
+const UPDATE_STATE = path.join(process.cwd(), "scripts/lib/update-state.sh");
+const RUNTIME_EXCLUSION = path.join(process.cwd(), "scripts/lib/runtime-exclusion.sh");
+const UPDATE_GATEWAYS = path.join(process.cwd(), "scripts/lib/update-gateway-runtimes.sh");
+const roots: string[] = [];
+
+function copy(root: string, rel: string, source: string) {
+ const dst = path.join(root, rel); fs.mkdirSync(path.dirname(dst), { recursive: true });
+ fs.copyFileSync(source, dst); fs.chmodSync(dst, 0o755);
+}
+
+function state(home: string, name: string, root: string, localUrl: string, runtimeOwned = true) {
+ const dir = path.join(home, ".mso/private/gateway", name);
+ fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
+ for (const p of [path.join(home, ".mso"), path.join(home, ".mso/private"), path.join(home, ".mso/private/gateway")]) fs.chmodSync(p, 0o700);
+ fs.writeFileSync(path.join(dir, "state.json"), `${JSON.stringify({ root, localUrl, runtimeOwned })}\n`, { mode: 0o600 });
+}
+
+function fixture(delayMs = 0) {
+ const base = fs.mkdtempSync(path.join(os.tmpdir(), "mso-service-update-")); roots.push(base);
+ const repo = path.join(base, "repo"), home = path.join(base, "home"), capture = path.join(base, "capture");
+ fs.mkdirSync(path.join(repo, "scripts/lib"), { recursive: true }); fs.mkdirSync(home);
+ copy(repo, "scripts/lib/private-state.sh", PRIVATE); copy(repo, "scripts/lib/update-state.sh", UPDATE_STATE);
+ copy(repo, "scripts/lib/runtime-exclusion.sh", RUNTIME_EXCLUSION); copy(repo, "scripts/lib/update-gateway-runtimes.sh", UPDATE_GATEWAYS);
+ fs.writeFileSync(path.join(repo, "scripts/mso-gateway"), `#!/bin/sh
+printf '%s %s\n' "$1" "\${MSO_GATEWAY_LOCAL_URL:-}" >> ${JSON.stringify(capture)}
+if [ "$1" = runtime-stop ]; then printf '1\n' > "$MSO_GATEWAY_RECOVERY_MARKER"; chmod 600 "$MSO_GATEWAY_RECOVERY_MARKER"; echo 'runtime: stopped-owned'; exit 0; fi
+[ "$1" = local-start ] && exit 0
+exit 2
+`, { mode: 0o755 });
+ fs.writeFileSync(path.join(repo, "scripts/self-update.sh"), `#!/bin/sh
+printf 'self-start\n' >> ${JSON.stringify(capture)}
+sleep ${Math.max(0, delayMs) / 1000}
+printf 'self-end\n' >> ${JSON.stringify(capture)}
+`, { mode: 0o755 });
+ const canonical = fs.realpathSync(repo);
+ state(home, "one", canonical, "http://127.0.0.1:4555");
+ state(home, "two", canonical, "http://127.0.0.1:4666");
+ state(home, "other", path.join(base, "other-checkout"), "http://127.0.0.1:4777");
+ const env = { ...process.env, HOME: home, MSO_UPDATE_ROOT: repo,
+ MSO_UPDATE_STATE_DIR: path.join(base, "update-state"), MSO_RUNTIME_EXCLUSION_DIR: path.join(base, "runtime-exclusion") };
+ return { base, repo, capture, env };
+}
+
+function runAsync(env: NodeJS.ProcessEnv) {
+ return new Promise<{ code: number | null; stdout: string; stderr: string }>((resolve) => {
+ const child = spawn(WRAPPER, [], { env, stdio: ["ignore", "pipe", "pipe"] }); let stdout = "", stderr = "";
+ child.stdout.on("data", (v) => stdout += v); child.stderr.on("data", (v) => stderr += v);
+ child.on("close", (code) => resolve({ code, stdout, stderr }));
+ });
+}
+
+afterEach(() => { for (const root of roots.splice(0)) fs.rmSync(root, { recursive: true, force: true }); });
+
+describe("service-active update lifecycle", () => {
+ it("quiesces and restores every gateway-owned fallback runtime for this checkout", () => {
+ const f = fixture(); execFileSync(WRAPPER, [], { env: f.env, encoding: "utf8" });
+ const calls = fs.readFileSync(f.capture, "utf8");
+ expect(calls).toContain("runtime-stop http://127.0.0.1:4555");
+ expect(calls).toContain("runtime-stop http://127.0.0.1:4666");
+ expect(calls).toContain("local-start http://127.0.0.1:4555");
+ expect(calls).toContain("local-start http://127.0.0.1:4666");
+ expect(calls).not.toContain("4777");
+ expect(calls.indexOf("runtime-stop")).toBeLessThan(calls.indexOf("self-start"));
+ expect(calls.lastIndexOf("local-start")).toBeGreaterThan(calls.indexOf("self-end"));
+ });
+
+ it("fails closed on a corrupt persisted gateway restore inventory", () => {
+ const f = fixture();
+ const key = createHash("sha256").update(fs.realpathSync(f.repo)).digest("hex");
+ const dir = path.join(f.base, "update-state", key); fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
+ fs.chmodSync(path.join(f.base, "update-state"), 0o700);
+ fs.writeFileSync(path.join(dir, "gateway-runtimes.json"), "{bad\n", { mode: 0o600 });
+ const out = spawnSync(WRAPPER, [], { env: f.env, encoding: "utf8" });
+ expect(out.status).not.toBe(0);
+ expect(out.stderr).toMatch(/restore inventory|inventory checkout gateway runtimes/);
+ expect(fs.existsSync(f.capture) ? fs.readFileSync(f.capture, "utf8") : "").not.toContain("self-start");
+ });
+
+ it("serializes two synchronous service updates under the same checkout transaction lock", async () => {
+ const f = fixture(450); const first = runAsync(f.env); await new Promise((r) => setTimeout(r, 60)); const second = runAsync(f.env);
+ const [a, b] = await Promise.all([first, second]); expect(a.code).toBe(0); expect(b.code).toBe(0);
+ const lifecycle = fs.readFileSync(f.capture, "utf8").split("\n").filter((v) => v.startsWith("self-"));
+ expect(lifecycle).toEqual(["self-start", "self-end", "self-start", "self-end"]);
+ });
+});
diff --git a/scripts/mso-update b/scripts/mso-update
index 47268b29..177a1363 100755
--- a/scripts/mso-update
+++ b/scripts/mso-update
@@ -14,6 +14,8 @@ GATEWAY="$ROOT/scripts/mso-gateway"
. "$ROOT/scripts/lib/update-state.sh"
# shellcheck source=scripts/lib/runtime-exclusion.sh
. "$ROOT/scripts/lib/runtime-exclusion.sh"
+# shellcheck source=scripts/lib/update-gateway-runtimes.sh
+. "$ROOT/scripts/lib/update-gateway-runtimes.sh"
fail() { printf 'mso update: %s\n' "$*" >&2; exit 1; }
git_root() { [ "$(git -C "$ROOT" rev-parse --show-toplevel 2>/dev/null || true)" = "$ROOT" ]; }
@@ -43,27 +45,9 @@ require_service_checkout() {
|| fail "active mso.service belongs to $canonical_service, not this checkout $canonical_root; run update from the service checkout"
}
-quiesce_offline_runtime() {
- local out
- [ -x "$GATEWAY" ] || return 0
- init_update_state
- out="$(MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$LOCAL_URL" \
- MSO_GATEWAY_RECOVERY_MARKER="$RESTART_MARKER" "$GATEWAY" runtime-stop)" \
- || fail "could not safely quiesce the local MSO runtime before rebuilding"
- case "$out" in
- *'runtime: stopped-owned'*|*'runtime: recovered-stale-owned'*|*'runtime: recovery-pending'*)
- restart_pending || fail "gateway stopped/recovered an owned runtime without durable recovery intent" ;;
- *'runtime: already-down'*) ;;
- *) fail "unexpected gateway runtime-stop result" ;;
- esac
-}
+quiesce_offline_runtime() { update_gateway_quiesce_all; }
-restore_offline_runtime() {
- restart_pending || return 0
- MSO_GATEWAY_ROOT="$ROOT" MSO_GATEWAY_ENV="$ROOT/.env.local" MSO_GATEWAY_LOCAL_URL="$LOCAL_URL" "$GATEWAY" local-start >/dev/null \
- || fail "build succeeded but the previously-owned local MSO runtime could not be restored"
- clear_restart_pending
-}
+restore_offline_runtime() { update_gateway_restore_all; }
print_status() {
local n local_v remote_v head receipt=''
@@ -81,7 +65,7 @@ print_status() {
fi
init_update_state
if [ -e "$DEPLOY_RECEIPT" ] || [ -L "$DEPLOY_RECEIPT" ]; then receipt="$(read_receipt)"; fi
- if [ "$receipt" != "$head" ] || restart_pending; then
+ if [ "$receipt" != "$head" ] || update_gateway_restore_pending; then
printf 'source is current at %s, but offline deployment verification/restart is pending. run: mso update\n' "${head:0:7}"
else
printf 'mso %s is up to date (%s)\n' "${local_v:-unknown}" "${head:0:7}"
@@ -115,7 +99,7 @@ offline_build_locked() {
fi
head="$(git -C "$ROOT" rev-parse HEAD)"
if [ -e "$DEPLOY_RECEIPT" ] || [ -L "$DEPLOY_RECEIPT" ]; then receipt="$(read_receipt)"; fi
- if [ "$rebuild_only" != 1 ] && [ "$receipt" = "$head" ] && ! restart_pending; then
+ if [ "$rebuild_only" != 1 ] && [ "$receipt" = "$head" ] && ! update_gateway_restore_pending; then
printf 'MSO is already up to date and the offline deployment receipt matches.\n'
return 0
fi
@@ -134,7 +118,7 @@ offline_build_locked() {
write_receipt
runtime_exclusion_release
restore_offline_runtime
- if restart_pending; then fail "runtime restart marker remained after restore"; fi
+ if update_gateway_restore_pending; then fail "gateway runtime restore inventory remained after restore"; fi
printf 'updated to %s. No system service was active; run: mso web\n' "$(git -C "$ROOT" rev-parse --short HEAD)"
}
@@ -152,10 +136,10 @@ run_update() {
if command -v systemd-run >/dev/null 2>&1 && systemctl --user show-environment >/dev/null 2>&1; then
mkdir -p "$(dirname "$LOG")"; chmod 700 "$(dirname "$LOG")" 2>/dev/null || true
systemd-run --user --collect --unit=mso-self-update --property=TimeoutStartSec=3600 \
- "--property=WorkingDirectory=$ROOT" "--setenv=MSO_UPDATE_LOG=$LOG" /bin/bash "$ROOT/scripts/self-update.sh" $arg >/dev/null
+ "--property=WorkingDirectory=$ROOT" "--setenv=MSO_UPDATE_LOG=$LOG" /bin/bash "$ROOT/scripts/mso-service-update" $arg >/dev/null
printf 'update started safely outside mso.service. Follow: mso update log\n'
else
- /bin/bash "$ROOT/scripts/self-update.sh" $arg
+ /bin/bash "$ROOT/scripts/mso-service-update" $arg
tail -n 30 "$LOG" 2>/dev/null || true
fi
else offline_build "$rebuild_only"; fi
diff --git a/scripts/mso-update.test.ts b/scripts/mso-update.test.ts
index c1094396..aea2bc69 100644
--- a/scripts/mso-update.test.ts
+++ b/scripts/mso-update.test.ts
@@ -8,6 +8,8 @@ const SCRIPT = path.join(process.cwd(), "scripts/mso-update");
const PRIVATE = path.join(process.cwd(), "scripts/lib/private-state.sh");
const UPDATE_STATE = path.join(process.cwd(), "scripts/lib/update-state.sh");
const RUNTIME_EXCLUSION = path.join(process.cwd(), "scripts/lib/runtime-exclusion.sh");
+const UPDATE_GATEWAYS = path.join(process.cwd(), "scripts/lib/update-gateway-runtimes.sh");
+const SERVICE_UPDATE = path.join(process.cwd(), "scripts/mso-service-update");
const roots: string[] = [];
function git(cwd: string, ...args: string[]) { return execFileSync("git", args, { cwd, encoding: "utf8" }).trim(); }
@@ -18,9 +20,12 @@ function fixture(options: { failInstallOnce?: boolean; activeService?: "same" |
fs.copyFileSync(PRIVATE, path.join(repo, "scripts/lib/private-state.sh"));
fs.copyFileSync(UPDATE_STATE, path.join(repo, "scripts/lib/update-state.sh"));
fs.copyFileSync(RUNTIME_EXCLUSION, path.join(repo, "scripts/lib/runtime-exclusion.sh"));
+ fs.copyFileSync(UPDATE_GATEWAYS, path.join(repo, "scripts/lib/update-gateway-runtimes.sh"));
+ fs.copyFileSync(SERVICE_UPDATE, path.join(repo, "scripts/mso-service-update")); fs.chmodSync(path.join(repo, "scripts/mso-service-update"), 0o755);
+ fs.writeFileSync(path.join(repo, "scripts/self-update.sh"), `#!/bin/sh\nprintf 'self-update %s\\n' "$*" >> "${capture}"\n`, { mode: 0o755 });
fs.writeFileSync(path.join(repo, "scripts/verify-build.sh"), "#!/bin/sh\nexit 0\n", { mode: 0o755 });
fs.writeFileSync(path.join(repo, "scripts/mso-gateway"), `#!/bin/sh
-printf 'gateway %s\\n' "$*" >> "${capture}"
+printf 'gateway %s %s\\n' "$*" "\${MSO_GATEWAY_LOCAL_URL:-}" >> "${capture}"
case "$1" in
runtime-stop)
[ -n "\${MSO_GATEWAY_RECOVERY_MARKER:-}" ] || exit 9
@@ -56,10 +61,15 @@ if [ "${options.installDelayMs ?? 0}" -gt 0 ] && [ "$1" = install ]; then sleep
if [ "${options.failInstallOnce ? "1" : "0"}" = 1 ] && [ "$1" = install ] && [ ! -f "${failOnce}" ]; then touch "${failOnce}"; exit 23; fi
`, { mode: 0o755 });
fs.writeFileSync(path.join(bin, "node"), `#!/bin/sh\nprintf 'node %s\\n' "$*" >> "${capture}"\n`, { mode: 0o755 });
- const env = { ...process.env, HOME: path.join(root, "home"), PATH: `${bin}:${process.env.PATH}`, MSO_UPDATE_ROOT: repo,
+ const home = path.join(root, "home");
+ const gatewayScope = path.join(home, ".mso/private/gateway/default-scope");
+ fs.mkdirSync(gatewayScope, { recursive: true, mode: 0o700 }); fs.chmodSync(path.join(home, ".mso"), 0o700);
+ fs.chmodSync(path.join(home, ".mso/private"), 0o700); fs.chmodSync(path.join(home, ".mso/private/gateway"), 0o700);
+ fs.writeFileSync(path.join(gatewayScope, "state.json"), JSON.stringify({ root: fs.realpathSync(repo), localUrl: "http://127.0.0.1:4555", runtimeOwned: true }) + "\n", { mode: 0o600 });
+ const env = { ...process.env, HOME: home, PATH: `${bin}:${process.env.PATH}`, MSO_UPDATE_ROOT: repo,
MSO_UPDATE_NOTICE_DIR: path.join(root, "notice"), MSO_UPDATE_STATE_DIR: path.join(root, "update-state"),
- MSO_UPDATE_LOCAL_URL: "http://127.0.0.1:4555" };
- return { root, repo, remote, old, newer, capture, env };
+ MSO_RUNTIME_EXCLUSION_DIR: path.join(root, "runtime-exclusion"), MSO_UPDATE_LOCAL_URL: "http://127.0.0.1:4555" };
+ return { root, repo, remote, old, newer, capture, env, home };
}
function receipts(base: string) {
diff --git a/scripts/self-update.sh b/scripts/self-update.sh
index e56e344d..60754668 100755
--- a/scripts/self-update.sh
+++ b/scripts/self-update.sh
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
-# self-update.sh — pull main, prove it compiles, build, restart. The body of the
-# Settings → About update button.
+# self-update.sh — inner pull/verify/build/restart stage. All public update entry points
+# go through scripts/mso-service-update first so checkout runtime inventory/locks are enforced.
#
# RUN BY systemd-run --user, NOT by mso.service (lib/host/self-update.ts explains
# why: replacing mso.service kills that service's whole cgroup, so an updater child
diff --git a/scripts/ship-handoff.sh b/scripts/ship-handoff.sh
index 6c1cca02..373d2f54 100755
--- a/scripts/ship-handoff.sh
+++ b/scripts/ship-handoff.sh
@@ -18,8 +18,8 @@ case "$EXPECTED_SHA" in
*[!0-9a-f]*) echo "expected SHA must be exactly 40 lowercase hex characters" >&2; exit 2 ;;
esac
ROOT="$(cd "$ROOT" && pwd -P)"
-SELF_UPDATE="$ROOT/scripts/self-update.sh"
-[ -f "$SELF_UPDATE" ] || { echo "missing $SELF_UPDATE" >&2; exit 2; }
+SERVICE_UPDATE="$ROOT/scripts/mso-service-update"
+[ -f "$SERVICE_UPDATE" ] || { echo "missing $SERVICE_UPDATE" >&2; exit 2; }
[ "$(git -C "$ROOT" rev-parse --show-toplevel 2>/dev/null || true)" = "$ROOT" ] \
|| { echo "$ROOT is not the repository root" >&2; exit 2; }
[ "$(git -C "$ROOT" rev-parse HEAD)" = "$EXPECTED_SHA" ] \
@@ -48,7 +48,7 @@ systemd-run \
"--property=TimeoutStartSec=3600" \
"--setenv=MSO_UPDATE_LOG=$LOG" \
"--setenv=MSO_EXPECTED_SHA=$EXPECTED_SHA" \
- /bin/bash "$SELF_UPDATE" --ship-finalize
+ /bin/bash "$SERVICE_UPDATE" --ship-finalize
printf 'release_sha=%s\n' "$EXPECTED_SHA"
printf 'release_unit=%s.service\n' "$UNIT"