Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

feat(console): share a remote Switch server with everyone on its host (CHOO-2893) (#537)

Christian McDermott155066ffb3cd
diff --git a/console/.gitignore b/console/.gitignore
index a8e608708..234ec9fc2 100644
--- a/console/.gitignore
+++ b/console/.gitignore
@@ -115,4 +115,5 @@ apps/*/tooling/node-deps/node_modules/
# Vitest / Browser testing
.vitest-attachments/
+.vitest/
**/__screenshots__/
diff --git a/console/agents/architecture/remote-execution.md b/console/agents/architecture/remote-execution.md
index 58bd11633..1fc8435a6 100644
--- a/console/agents/architecture/remote-execution.md
+++ b/console/agents/architecture/remote-execution.md
@@ -19,6 +19,7 @@ silently does less on a remote host. This page is the map of the remote half.
| Remote dependency detection and install | `dependencies/remote-dependency-manager.ts`, `dependencies/ssh-install-runner.ts` |
| The persistent SDK host | `packages/agent-providers/src/host/`, `src/main/core/sdk-host/` |
| A Switch Console-managed Switch server | `src/main/core/managed-switch-server/` |
+| A remote server's shared state: published settings, who uses it | `managed-switch-server/stack-state.ts`, `console-register.ts` |
| Renderer surfaces | `src/renderer/features/remote-hosts/` |
## Reachability is a state machine, not a boolean (CHOO-1682)
@@ -79,6 +80,216 @@ Two properties worth preserving:
you nothing about whether `dockerd` is up, and running an installer over a stopped
service would misreport the cause.
+## A remote Switch server is shared (CHOO-2893)
+
+A server Switch Console runs on a remote host is used by everyone with access
+to that host, from their own Consoles and often under their own accounts. The
+access boundary is being able to run Docker there — which already means being
+able to read every secret the stack has — so nothing here pretends otherwise.
+
+**The host is the source of truth for its stack.** A remote stack's ports and
+credentials used to live only in the encrypted store of the Console that
+started it; a second Console had neither, generated its own, rewrote the
+stack's `.env` and took the running server down. Now:
+
+- The `.env` the stack was last started with is published into a Docker volume
+ beside the stack's own (`<project>_console-state`, labelled outside compose's
+ project so `compose down -v` leaves it). Every account that can reach the
+ daemon can read it, where the working dir holding the real `.env` belongs to
+ whoever started the stack first. It is read and written through a throwaway
+ container of the stack's own Postgres image, with secrets on stdin, never in
+ a command line — see `stack-state.ts`. One shell launcher per round trip
+ checks the image is there, creates the volume when writing, and runs the
+ container. A host without the helper image uses the image the stack's own
+ Postgres container runs; only with neither is it pulled — under a pull's
+ timeout, never by `docker run` inside a quick one.
+- "Nothing of the stack" is read from empty `docker ps` / `docker volume ls`
+ output, so a command over SSH succeeds only on an exit status of 0: a channel
+ that closes without one — a dropped connection — or on a signal is a failure,
+ never an empty answer (`SshExecutionContext.exec`, `RemoteServerHost`).
+- A start reads the host first (`inspectStack`) and takes the published copy,
+ then this account's `.env`. New credentials are made **only** on a host with
+ nothing of the stack at all. Another account's unpublished stack, or a host
+ that cannot be read, is refused before anything is written — this desktop's
+ cache cannot tell whether someone has reset the stack since, and starting
+ from it, then publishing it, would lock everyone out. The cache may fill
+ gaps in a partial `.env` only when it agrees with every port and credential
+ the file does carry, for the same reason.
+- The published copy is stamped with the creation time of the Postgres volume
+ it was written for (after compose up on a first start, when the volume did
+ not exist at publish). A copy whose stamp no longer matches is ignored: a
+ Console from before settings were shared can reset the stack without
+ knowing the copy exists, leaving it naming credentials that open nothing.
+ This account's own `.env` is stamped the same way (`.env.db` beside it), and
+ one written for a database since recreated is not used either — it reads as
+ another account's unshared stack — so a reset, start and `compose down` from
+ such a Console cannot leave a stale copy to be trusted.
+- Every account's Console writes the published `.env` verbatim. Compose run
+ from a second working dir with a byte-identical `.env` recreates nothing
+ (measured; it follows from the bundled compose referencing nothing by path),
+ which is what lets several accounts run one stack.
+- **Connect** joins a running stack without writing its settings differently or
+ running compose. **Disconnect** leaves it running for everyone else; only
+ **Delete for everyone** resets it. Both take this Console off the register,
+ so it stops counting as a user; leaving does not wait on the host for that.
+- A Console re-reads the host at launch, on reachability recovery, and when a
+ running stack stops answering (rate-limited), so a stack another Console
+ stopped, restarted on new ports or reset shows as it is. A stack shown as
+ stopped is sent nothing, so opening its page has the host looked at again:
+ one someone has started or removed is taken up; one still stopped keeps
+ what the page says about it, and one this Console removed itself is not
+ reported as removed. A stack that could not be taken up says why on the page
+ rather than showing as simply stopped; one running with settings that cannot
+ be read now keeps the forward this Console already has. An update this Console held sessions back for is
+ finished once the host is at its pin, whoever made it. The status carries a
+ `notice` for what this Console did not do.
+- A start checks, before it changes anything on the host, that this Console
+ can reach the stack afterwards — its ports free here, its address not
+ another server's (`checkNetworking`, `assertManagedServerUrlFree`) — since
+ compose restarts or updates it for everyone before the forward is opened.
+ A new stack's ports are this desktop's to choose, so a kept choice with a
+ port now taken here is chosen again.
+- Joining or picking up a stack at this build's version also replaces an older
+ compose file in the account's working dir, which is then the stack's own:
+ Stop and Reset run from it.
+
+**One Console changes a shared stack at a time.** Two Consoles pressing Start
+on an empty host together would otherwise both find nothing, both make
+credentials, and both publish them and run compose — the second publish wins
+the file, and the database is created with whichever credentials compose read
+first. So whoever changes a stack holds its lock (`stack-lock.ts`), a lease in
+the state volume:
+
+- Everything that reads the host to act on it takes the lock *first*: start,
+ the update at launch, *Update and connect*, restarts from notices, Connect,
+ and the check at launch or recovery. The second of two starts therefore finds
+ the first one's stack and joins it with the same credentials. A Connect or a
+ check holds it only while reading and adopting, not while waiting for the
+ server to answer. The register and the Add Server preview take none; the
+ preview says who holds it.
+- Start, Connect and the check **wait** for a held lock, with the holder and
+ what they are doing on the status (`waitingFor`) and a *Stop waiting* that
+ changes nothing. Stop and reset **refuse** instead (`ServerBusyError`, naming
+ the holder) — stopping a server someone has just started because Stop was
+ clicked while they started it is not what was asked — and a reset takes the
+ lock before it deletes the stack's agents.
+- It is a lease: `ttlSeconds` at a time, renewed while its holder works, with a
+ hard `maxHoldSeconds` above anything a start can take (a backup may run 30
+ minutes). A Console that crashes or loses its network holds it for at most a
+ lease; a restarted Console takes back its own previous run's at once (the
+ same id and name — two desktops restored from one backup share only the id),
+ and a lease this run failed to give back is its own to take over, not to
+ wait for. Expiry is decided by the host's clock inside the script, never a
+ desktop's.
+- Taking and renewing happen under a `flock` on `/state/.mutex`, held for the
+ script's lifetime, so exactly one of two Consoles gets it — including two
+ taking over one lapsed lease. The containers share one volume on one kernel,
+ so the lock holds across them and dies with the script. Every script that
+ writes the volume takes that mutex (`state-mutex.ts`); the register's trim and
+ the published copy's temp files needed it anyway. A write that fails ends the
+ script with an error: "taken" over a lock file a full disk left empty would
+ let the next Console take it too.
+- A Console that lost its lease while away could come back and carry on, so
+ publishing, stamping and withdrawing the settings check the lease's token
+ under the mutex in the same step, and compose runs only after
+ `assertHeld`. The pipeline refuses a shared host without a lease, and a local
+ one with one.
+- A stop or reset that did not get the lock — held, or not askable — touched
+ nothing, so this Console keeps its forward and the phase it had.
+
+This only binds Consoles that have it, which is every Console that shares
+stacks at all. The lock scripts and the mutex are tested against a real volume
+through Docker (`stack-lock.docker.test.ts`), since macOS has no `flock`.
+
+**Updating a shared stack is an update for everyone.** A Console brings a
+managed stack up to its own switch-core pin when it takes the stack up
+(`managed-upgrade.ts`), and on a shared host that restarts it for everyone
+using it, from the account that happened to do it — the pre-update backup lands
+in that account's working dir. So:
+
+- **Connect** joins as-is only at this build's own version. An older stack is
+ brought up to date as a start from the host's settings (the connect step
+ says so and names who else uses it); a newer one is refused, since its
+ database has migrated past anything this build can run.
+- Connect and the version checks go by the version the core container runs
+ when the stack is up, not the one its published settings name: a start that
+ published and then failed leaves the old containers running.
+- At launch or reachability recovery, a running stack behind the pin is updated
+ on its own only when no other Console has used it in the last
+ `RECENTLY_SEEN_DAYS` (the register on the host says who). Otherwise its
+ upgrade is `held`: the stack is still forwarded, sessions on it wait
+ (`ensureReady`), and the server page offers *Update for everyone*, naming
+ who it reaches. A register that cannot be read counts as others using it. An
+ update this account already started (its journal is there) is resumed
+ regardless.
+
+**Usage data on a shared stack.** One "Share usage data" answer covers a
+Console and the server it runs, but a shared server serves everyone using it.
+A start always applies this Console's "no"; it applies a "yes" only where that
+takes nobody's "no" away — the stack already shares, or no other Console has
+used it lately (an unreadable register counts as others). The page's notice
+offers a restart to apply a "no", never to switch sharing on over others.
+
+**Identity is shared; attribution is not.** Everyone signs in as the stack's one
+seeded admin — sessions are owner-only on the server with no admin override, so
+per-person accounts would hide each person's sessions from the others. Instead:
+
+- Each Console has a random id (`console-identity.ts`, deliberately not the
+ telemetry install id) and a `user@host` name, sent as `X-Switch-Console-Id` /
+ `X-Switch-Console-Name` to managed servers only; switch-core stamps them on
+ its log lines. A server someone else runs signs each person in as themselves
+ and is told nothing about the desktop.
+- Each Console records itself in the state volume (`console-register.ts`): a
+ register of who uses the stack and an activity log of starts, connects,
+ stops, resets and disconnects. Every action refreshes the Console's entry;
+ otherwise it is refreshed at most once a day, since each write is a
+ container run on the host — when it last did is kept across launches. A
+ disconnect takes the Console off the register
+ and keeps its line of activity. A reset keeps the activity. The
+ record is for people, never a control, and a write that fails does not fail
+ the operation: the server page says what could not be recorded
+ (`recordWarning`) until a later record gets through.
+
+**Several Consoles, one agent.** Under one account, each Console holds its own
+row for the same agent and writes the agent's one watcher on the host. Model
+and instructions come from the agent's config file on the host, so they agree.
+Auto-approve lives in each row, so what they share is the last choice a person
+made, kept beside the watcher (`auto-approve.json`): every watcher write takes
+auto-approve from it and brings the row in line, except the write that follows
+the person changing it (`pushRemoteAutoApprove`). A change goes on the host
+before the row (`keepAutoApproveChoice`), so a watcher write racing it takes
+the new value; when the watcher is not starting sessions — stopped, or
+automatic sessions off — nothing rewrites the spec now, so the change goes
+into the choice and the spec together (`recordAutoApproveOnHost`). Only an explicit
+change writes the choice: a saved spec is written from whichever row wrote the
+watcher last — an older Console's too — so it is never taken for one.
+
+Two Consoles under one account acting on one session do not collide: prompts
+queue in the session's SDK host and run in turn, and either one's interrupt or
+stop ends the turn for both — it is one session, reached through the same
+sidecar.
+
+**Removing an agent.** A plain remove takes the row out of this Console and
+leaves the agent running on its host, for its automatic sessions and anyone
+else using it there. *Also remove it from the host* stops it there and deletes
+the files Console provisioned. Deleting it in Switch implies that: a deleted
+identity has nothing left to run. In a directory several agents share, only
+this agent's credentials go: the one-file settings from before each agent had
+its own are stripped only when they name this agent.
+
+**Agents another account runs are not loaded.** Their working directories,
+credentials, watchers and sessions are in that account's home, which this
+account cannot read — and a session is reached through its owner's sidecar — so
+Load existing agents does not offer them. They are on the shared server all the
+same, and can be talked to in its rooms.
+
+**Limitations.** Rootless Docker gives each account its own daemon, so there is
+nothing to share. A remote server's port numbers must also be free on each
+desktop, because the forward mirrors them and agent endpoints depend on the
+number. Two different Console versions on one account restart each other's SDK
+hosts. Agents that run on someone's laptop appear in the server view but cannot
+be loaded.
+
## Persistent execution
The same SDK host implementation serves local and SSH sessions, hosted
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.test.ts
index a0c85a369..f27dce8dd 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.test.ts
@@ -107,6 +107,12 @@ vi.mock('@main/core/sessions/session-hooks', () => ({
vi.mock('@main/core/telemetry/telemetry-service', () => ({ trackEvent: h.trackEvent }));
const { deleteAgent } = await import('./deleteAgent');
+const { getAgentLocation } = await import('./agent-location');
+const { stopRemoteWatcher } = await import('./remote-watcher');
+const { autoSessionWatcher } = await import('@main/core/switch-rooms/auto-session-watcher');
+const { stopSharedAgentSessions } = await import('./stop-shared-agent-sessions');
+const { deleteAgent: gatewayDeleteAgent } =
+ await import('@main/core/switch-servers/gateway-client');
const CREDS = JSON.stringify({
env: { SWITCH_API_ENDPOINT: 'https://s', SWITCH_API_TOKEN: 'tok-123', SWITCH_AGENT_ID: 'sw-1' },
@@ -139,6 +145,25 @@ describe('deleteAgent', () => {
expect(await fs.exists(agentSettingsRelativePath('codex-hoot'))).toBe(false);
});
+ it('strips the shared settings file only of this agent’s own credentials', async () => {
+ h.state.agent = {
+ id: 'agent-1',
+ name: 'cc-hoot',
+ providerId: 'claude-code',
+ locationId: 'loc',
+ switchAgentId: 'sw-1',
+ };
+ h.state.fs = fakeFs({});
+
+ await deleteAgent('agent-1', {
+ deleteInSwitch: false,
+ removeProvisionedFiles: true,
+ trigger: 'user',
+ });
+
+ expect(h.removeSwitchCredentials).toHaveBeenCalledWith(expect.anything(), 'sw-1');
+ });
+
it('removes both the credentials and the definition for a repo-agents provider', async () => {
const fs = fakeFs({
[agentSettingsRelativePath('cc-hoot')]: CREDS,
@@ -194,6 +219,77 @@ describe('deleteAgent', () => {
expect(await fs.exists(agentSettingsRelativePath('cc-sibling'))).toBe(true);
});
+ describe('the auto-session watcher', () => {
+ it('leaves a remote agent’s watcher running on a plain remove', async () => {
+ // It serves every Console using the agent, and the removal dialog
+ // promises the sidecar is untouched unless asked.
+ vi.mocked(getAgentLocation).mockResolvedValueOnce({ sshHost: 'vm-1', dir: '/repo' } as never);
+
+ await deleteAgent('agent-1', {
+ deleteInSwitch: false,
+ removeProvisionedFiles: false,
+ trigger: 'user',
+ });
+
+ expect(stopRemoteWatcher).not.toHaveBeenCalled();
+ expect(autoSessionWatcher.stopForAgent).not.toHaveBeenCalled();
+ });
+
+ it('stops a remote agent’s watcher when the agent is terminated', async () => {
+ vi.mocked(getAgentLocation).mockResolvedValueOnce({ sshHost: 'vm-1', dir: '/repo' } as never);
+
+ await deleteAgent('agent-1', {
+ deleteInSwitch: false,
+ removeProvisionedFiles: true,
+ trigger: 'user',
+ });
+
+ expect(stopRemoteWatcher).toHaveBeenCalledExactlyOnceWith('agent-1');
+ });
+
+ it('removes a remote agent from its host when it is deleted in Switch', async () => {
+ h.state.sshHost = 'vm-1';
+ h.state.agent = {
+ id: 'agent-1',
+ name: 'cc-hoot',
+ providerId: 'claude',
+ locationId: 'loc',
+ serverId: 'srv-1',
+ switchAgentId: 'sw-1',
+ };
+ const { getServer } = await import('@main/core/switch-servers/servers-store');
+ vi.mocked(getServer).mockResolvedValue({ id: 'srv-1' } as never);
+ const fs = fakeFs({
+ [agentSettingsRelativePath('cc-hoot')]: CREDS,
+ '.claude/agents/cc-hoot.md': '# cc-hoot',
+ });
+ h.state.fs = fs;
+
+ await deleteAgent('agent-1', {
+ deleteInSwitch: true,
+ removeProvisionedFiles: false,
+ trigger: 'user',
+ });
+
+ expect(gatewayDeleteAgent).toHaveBeenCalledOnce();
+ expect(stopRemoteWatcher).toHaveBeenCalledExactlyOnceWith('agent-1');
+ expect(stopSharedAgentSessions).toHaveBeenCalledOnce();
+ expect(await fs.exists(agentSettingsRelativePath('cc-hoot'))).toBe(false);
+ expect(await fs.exists('.claude/agents/cc-hoot.md')).toBe(false);
+ });
+
+ it('stops a local agent’s watcher, which is this Console’s own child, on any remove', async () => {
+ await deleteAgent('agent-1', {
+ deleteInSwitch: false,
+ removeProvisionedFiles: false,
+ trigger: 'user',
+ });
+
+ expect(autoSessionWatcher.stopForAgent).toHaveBeenCalledWith('agent-1');
+ expect(stopRemoteWatcher).not.toHaveBeenCalled();
+ });
+ });
+
it('discards the controller state a local agent would otherwise leave behind', async () => {
await deleteAgent('agent-1', {
deleteInSwitch: false,
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.ts b/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.ts
index a7a07625f..7bd7851a5 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/deleteAgent.ts
@@ -47,7 +47,8 @@ export type DeleteAgentOptions = {
/**
* Also tear down what was provisioned on disk for this agent (its
* `.switch/agents/<name>.json` credentials, provider definition files, launch
- * profile) and kill its sidecar on the host.
+ * profile) and kill its sidecar on the host. Implied by `deleteInSwitch` for
+ * a remote agent.
*
* Required with no default, because the right answer depends on who owns the
* on-disk state: an agent this Console created can carry its files out, but an
@@ -161,7 +162,7 @@ async function removeProvisionedFiles(agent: Agent, location: Location): Promise
error: String(error),
});
});
- await removeSwitchCredentials(ctx.fs);
+ await removeSwitchCredentials(ctx.fs, agent.switchAgentId);
// A provider that registers the Switch server itself (Codex) leaves a
// per-agent launch profile under the user's home — a different scope than
// ctx.fs, reached through its own home filesystem (local or remote).
@@ -185,9 +186,9 @@ async function removeProvisionedFiles(agent: Agent, location: Location): Promise
* and answering rooms for an agent that no longer exists. A controller on a
* remote host is stopped and discarded only for a full cleanup — see 4.
* 4. Only when `removeProvisionedFiles` or `deleteInSwitch` is set: the remote
- * controller and its state, and (for `removeProvisionedFiles`) the Switch
- * credentials + definition file provisioned on disk for THIS agent and its
- * sidecar. A plain remove leaves the working directory and the host's
+ * controller and its state, and (for `removeProvisionedFiles`, or a remote
+ * agent deleted in Switch) the Switch credentials + definition file
+ * provisioned on disk for THIS agent and its sidecar. A plain remove leaves the working directory and the host's
* processes untouched — on a shared host they may belong to another install
* (CHOO-2560). Sibling agents' files are never touched either way.
*
@@ -225,7 +226,11 @@ async function removeAgent(
location: Location | null,
options: DeleteAgentOptions
): Promise<void> {
- const terminate = options.removeProvisionedFiles || options.deleteInSwitch;
+ // A remote agent deleted in Switch has nothing left to run, so it is removed
+ // from its host too.
+ const removeFiles =
+ options.removeProvisionedFiles || (options.deleteInSwitch && !!location?.sshHost);
+ const terminate = removeFiles || options.deleteInSwitch;
const stopController = () =>
location?.sshHost ? stopRemoteWatcher(agentId) : autoSessionWatcher.stopForAgent(agentId);
const stoppedUpFront = terminate && agent !== undefined;
@@ -279,7 +284,7 @@ async function removeAgent(
await setAutoSessionAgent(agentId, false);
await setControllerStopped(agentId, false);
- if (agent && location && options.removeProvisionedFiles) {
+ if (agent && location && removeFiles) {
await removeProvisionedFiles(agent, location).catch((error) => {
log.warn('deleteAgent: failed to remove provisioned files', {
agentId,
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.test.ts
index 4bbaa3f74..feaa3d3e5 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.test.ts
@@ -1,10 +1,11 @@
import { expect, it, vi } from 'vitest';
-import { initializeRemoteDiscovery } from './remote-watcher';
+import { initializeRemoteDiscovery, pushRemoteAutoApprove } from './remote-watcher';
const mocks = vi.hoisted(() => ({
start: vi.fn(),
stop: vi.fn(),
handlers: new Map<string, (value: unknown) => void>(),
+ applyControllerState: vi.fn(async () => {}),
}));
vi.mock('./agent-events', () => ({
agentEvents: {
@@ -18,7 +19,10 @@ vi.mock('./getAgents', () => ({
getAgents: async () => [{ id: 'existing', switchAgentId: 'registered' }],
}));
vi.mock('./getAgentById', () => ({ getAgentById: vi.fn() }));
-vi.mock('@main/core/sdk-host/shared-watcher', () => ({ configureSharedWatcher: vi.fn() }));
+vi.mock('@main/core/sdk-host/shared-watcher', () => ({
+ configureSharedWatcher: vi.fn(),
+ applyControllerState: mocks.applyControllerState,
+}));
vi.mock('@main/core/switch-rooms/auto-session-store', () => ({ listAutoSessionAgentIds: vi.fn() }));
it('discovers existing and newly onboarded agents and stops removed identities', async () => {
@@ -37,3 +41,10 @@ it('discovers existing and newly onboarded agents and stops removed identities',
mocks.handlers.get('agent:deleted')!('existing');
expect(mocks.stop).toHaveBeenCalledWith('existing');
});
+
+it('rewrites the watcher with this Console’s auto-approve, not the host’s, then follows its sessions', async () => {
+ await pushRemoteAutoApprove('agent-1');
+
+ expect(mocks.applyControllerState).toHaveBeenCalledWith('agent-1', 'restore', 'this-console');
+ expect(mocks.start).toHaveBeenCalledWith('agent-1');
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.ts b/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.ts
index 94fd09b97..05420dd3d 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/remote-watcher.ts
@@ -34,7 +34,13 @@ export async function startRemoteDiscovery(agentId: string): Promise<void> {
*/
export async function ensureRemoteWatcher(agentId: string): Promise<void> {
if (!(await getAgentById(agentId))?.switchAgentId) return;
- await applyControllerState(agentId, 'restore');
+ await applyControllerState(agentId, 'restore', 'host');
+ remoteSessionReconciler.start(agentId);
+}
+/** {@link ensureRemoteWatcher}, but writing this Console's auto-approve to the
+ * host instead of adopting the host's. */
+export async function pushRemoteAutoApprove(agentId: string): Promise<void> {
+ await applyControllerState(agentId, 'restore', 'this-console');
remoteSessionReconciler.start(agentId);
}
export async function stopRemoteWatcher(agentId: string): Promise<void> {
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.test.ts
index c2e1d4584..ecb601026 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.test.ts
@@ -33,7 +33,7 @@ describe('removeSwitchCredentials', () => {
}),
});
- await removeSwitchCredentials(fs);
+ await removeSwitchCredentials(fs, 'agent-123');
expect(fs.files.has(SETTINGS_PATH)).toBe(false);
});
@@ -51,7 +51,7 @@ describe('removeSwitchCredentials', () => {
}),
});
- await removeSwitchCredentials(fs);
+ await removeSwitchCredentials(fs, 'a');
const parsed = JSON.parse(fs.files.get(SETTINGS_PATH)!) as Record<string, unknown>;
expect(parsed.env).toEqual({ EDITOR: 'vim' });
@@ -62,14 +62,25 @@ describe('removeSwitchCredentials', () => {
const original = JSON.stringify({ env: { EDITOR: 'vim' } });
const fs = fakeFs({ [SETTINGS_PATH]: original });
- await removeSwitchCredentials(fs);
+ await removeSwitchCredentials(fs, 'a');
+
+ expect(fs.files.get(SETTINGS_PATH)).toBe(original);
+ });
+
+ it('leaves the credentials of the agent the shared file belongs to', async () => {
+ const original = JSON.stringify({
+ env: { SWITCH_API_ENDPOINT: 'e', SWITCH_API_TOKEN: 't', SWITCH_AGENT_ID: 'agent-main' },
+ });
+ const fs = fakeFs({ [SETTINGS_PATH]: original });
+
+ await removeSwitchCredentials(fs, 'agent-sibling');
expect(fs.files.get(SETTINGS_PATH)).toBe(original);
});
it('is a no-op when there is no settings file', async () => {
const fs = fakeFs({});
- await expect(removeSwitchCredentials(fs)).resolves.toBeUndefined();
+ await expect(removeSwitchCredentials(fs, 'a')).resolves.toBeUndefined();
expect(fs.files.size).toBe(0);
});
});
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.ts b/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.ts
index 5d2134807..80e4c6184 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/remove-switch-settings.ts
@@ -10,9 +10,12 @@ import { removeSwitchSettings } from './write-switch-settings';
* directory and works byte-identically for a local directory and a remote SSH
* host, so one call covers both.
*/
-export async function removeSwitchCredentials(fs: PluginFs): Promise<void> {
+export async function removeSwitchCredentials(
+ fs: PluginFs,
+ switchAgentId: string | null
+): Promise<void> {
const existing = await fs.read(SWITCH_SETTINGS_RELATIVE_PATH);
- const result = removeSwitchSettings(existing);
+ const result = removeSwitchSettings(existing, switchAgentId);
if (result.kind === 'skip') return;
if (result.kind === 'delete') {
await fs.delete(SWITCH_SETTINGS_RELATIVE_PATH);
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/reset-remote-agent.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/reset-remote-agent.test.ts
index 88f646e30..3a36e47c8 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/reset-remote-agent.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/reset-remote-agent.test.ts
@@ -81,7 +81,7 @@ it('stops all server-owned sessions for this agent before removing local views',
);
expect(mocks.stop).toHaveBeenCalledExactlyOnceWith('local', 'remote-only');
expect(mocks.remove).toHaveBeenCalledTimes(1);
- expect(mocks.restart).toHaveBeenCalledWith('local', 'explicit');
+ expect(mocks.restart).toHaveBeenCalledWith('local', 'explicit', 'host');
});
it('brings back a controller somebody had stopped by hand', async () => {
// Being stopped is durable and survives a reset, so putting the controller
@@ -91,7 +91,7 @@ it('brings back a controller somebody had stopped by hand', async () => {
mocks.list.mockResolvedValue([session]);
await resetRemoteAgent('local');
expect(mocks.setStopped).toHaveBeenCalledWith('local', false);
- expect(mocks.restart).toHaveBeenCalledWith('local', 'explicit');
+ expect(mocks.restart).toHaveBeenCalledWith('local', 'explicit', 'host');
});
it('keeps local state and auto-start disabled when stop has an unknown outcome', async () => {
mocks.list.mockResolvedValue([session]);
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.test.ts
index 189c1935e..eb0447e12 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.test.ts
@@ -1,65 +1,155 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
-const updateAgent = vi.fn(
- async ({ agentId, autoApprove }): Promise<{ id: string; autoApprove?: boolean } | undefined> => ({
- id: agentId,
- autoApprove,
+const calls = vi.hoisted(() => [] as string[]);
+const agentRow = vi.hoisted(() => ({
+ current: { id: 'agent-1', switchAgentId: 'sw-1', autoApprove: false } as
+ | { id: string; switchAgentId: string | null; autoApprove: boolean }
+ | undefined,
+}));
+const updateAgent = vi.hoisted(() =>
+ vi.fn(async ({ agentId, autoApprove }: { agentId: string; autoApprove: boolean }) => {
+ calls.push(`row ${autoApprove}`);
+ return { id: agentId, autoApprove };
})
);
-const getRemoteAgentLocation = vi.fn();
-const listAutoSessionAgentIds = vi.fn();
-const ensureRemoteWatcher = vi.fn(async (_id: string) => {});
+const getRemoteAgentLocation = vi.hoisted(() => vi.fn());
+const listAutoSessionAgentIds = vi.hoisted(() => vi.fn(async (): Promise<string[]> => []));
+const listStoppedControllerAgentIds = vi.hoisted(() => vi.fn(async (): Promise<string[]> => []));
+const pushRemoteAutoApprove = vi.hoisted(() =>
+ vi.fn(async (_id: string) => void calls.push('push'))
+);
+const recordAutoApproveOnHost = vi.hoisted(() =>
+ vi.fn(async (_id: string, autoApprove: boolean) => void calls.push(`host ${autoApprove}`))
+);
-vi.mock('./updateAgent', () => ({ updateAgent: (p: unknown) => updateAgent(p) }));
-vi.mock('./agent-location', () => ({
- getRemoteAgentLocation: (a: unknown) => getRemoteAgentLocation(a),
-}));
+const keepAutoApproveChoice = vi.hoisted(() =>
+ vi.fn(async (_id: string, autoApprove: boolean) => void calls.push(`choice ${autoApprove}`))
+);
+vi.mock('./getAgentById', () => ({ getAgentById: async () => agentRow.current }));
+vi.mock('./updateAgent', () => ({ updateAgent }));
+vi.mock('./agent-location', () => ({ getRemoteAgentLocation }));
vi.mock('@main/core/switch-rooms/auto-session-store', () => ({
- listAutoSessionAgentIds: () => listAutoSessionAgentIds(),
+ listAutoSessionAgentIds,
+ listStoppedControllerAgentIds,
}));
-vi.mock('./remote-watcher', () => ({
- ensureRemoteWatcher: (id: string) => ensureRemoteWatcher(id),
+vi.mock('./remote-watcher', () => ({ pushRemoteAutoApprove }));
+vi.mock('@main/core/sdk-host/shared-watcher', () => ({
+ keepAutoApproveChoice,
+ recordAutoApproveOnHost,
}));
import { setAgentAutoApprove } from './setAgentAutoApprove';
describe('setAgentAutoApprove', () => {
beforeEach(() => {
- updateAgent.mockClear();
- getRemoteAgentLocation.mockReset();
- listAutoSessionAgentIds.mockReset();
- ensureRemoteWatcher.mockClear();
+ vi.clearAllMocks();
+ calls.length = 0;
+ agentRow.current = { id: 'agent-1', switchAgentId: 'sw-1', autoApprove: false };
+ getRemoteAgentLocation.mockResolvedValue({ id: 'loc-1' });
+ listAutoSessionAgentIds.mockResolvedValue([]);
+ listStoppedControllerAgentIds.mockResolvedValue([]);
});
- it('writes the agent row and does not touch a local agent (read fresh at spawn)', async () => {
+ it('writes the row and nothing else for a local agent (read fresh at spawn)', async () => {
getRemoteAgentLocation.mockResolvedValue(null);
await setAgentAutoApprove({ agentId: 'agent-1', enabled: true });
- expect(updateAgent).toHaveBeenCalledWith({ agentId: 'agent-1', autoApprove: true });
- expect(ensureRemoteWatcher).not.toHaveBeenCalled();
+ expect(calls).toEqual(['row true']);
});
- it('re-pushes the spec to a remote agent whose watcher is running (auto_session on)', async () => {
- getRemoteAgentLocation.mockResolvedValue({ id: 'loc-1' });
+ it('keeps the choice on the host, then the row, then rewrites a watcher that starts sessions', async () => {
+ // The choice first, so a racing watcher write takes the new value instead
+ // of putting the old one back.
listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
- await setAgentAutoApprove({ agentId: 'agent-1', enabled: false });
+ await setAgentAutoApprove({ agentId: 'agent-1', enabled: true });
- expect(ensureRemoteWatcher).toHaveBeenCalledWith('agent-1');
+ expect(calls).toEqual(['choice true', 'row true', 'push']);
});
- it('skips the re-push for a remote agent with auto_session off (nothing running to refresh)', async () => {
- getRemoteAgentLocation.mockResolvedValue({ id: 'loc-1' });
- listAutoSessionAgentIds.mockResolvedValue([]);
+ it('says the watcher lags when the push does not reach the host, keeping the saved choice', async () => {
+ // The host's choice already holds the new value; rolling the row back
+ // would let the next watcher write silently undo it.
+ listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
+ pushRemoteAutoApprove.mockRejectedValueOnce(new Error('host unreachable'));
+
+ await expect(setAgentAutoApprove({ agentId: 'agent-1', enabled: true })).rejects.toThrow(
+ /saved, but the agent's watcher on its host could not be updated yet \(host unreachable\)/
+ );
+
+ expect(calls).toEqual(['choice true', 'row true']);
+ expect(pushRemoteAutoApprove).toHaveBeenCalledOnce();
+ });
+
+ it('changes nothing when the host could not take the choice', async () => {
+ listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
+ keepAutoApproveChoice.mockRejectedValueOnce(new Error('host unreachable'));
+
+ await expect(setAgentAutoApprove({ agentId: 'agent-1', enabled: true })).rejects.toThrow(
+ /host unreachable/
+ );
+
+ expect(updateAgent).not.toHaveBeenCalled();
+ expect(pushRemoteAutoApprove).not.toHaveBeenCalled();
+ });
+
+ it('writes only the row for an agent with no Switch identity, which has no watcher yet', async () => {
+ agentRow.current = { id: 'agent-1', switchAgentId: null, autoApprove: false };
+ listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
await setAgentAutoApprove({ agentId: 'agent-1', enabled: true });
- expect(ensureRemoteWatcher).not.toHaveBeenCalled();
+ expect(calls).toEqual(['row true']);
+ });
+
+ it('keeps the choice on the host first for a watcher that starts no sessions', async () => {
+ await setAgentAutoApprove({ agentId: 'agent-1', enabled: true });
+
+ expect(recordAutoApproveOnHost).toHaveBeenCalledWith('agent-1', true);
+ expect(calls).toEqual(['host true', 'row true']);
+ });
+
+ it('keeps the choice on the host for a stopped watcher, which nothing rewrites until it starts', async () => {
+ listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
+ listStoppedControllerAgentIds.mockResolvedValue(['agent-1']);
+
+ await setAgentAutoApprove({ agentId: 'agent-1', enabled: true });
+
+ expect(calls).toEqual(['host true', 'row true']);
+ expect(pushRemoteAutoApprove).not.toHaveBeenCalled();
+ });
+
+ it('leaves the row alone when the host could not keep the choice', async () => {
+ recordAutoApproveOnHost.mockRejectedValueOnce(new Error('host unreachable'));
+
+ await expect(setAgentAutoApprove({ agentId: 'agent-1', enabled: true })).rejects.toThrow(
+ /host unreachable/
+ );
+
+ expect(updateAgent).not.toHaveBeenCalled();
+ });
+
+ it('says so when the agent is deleted while the change is being made', async () => {
+ getRemoteAgentLocation.mockResolvedValue(null);
+ updateAgent.mockResolvedValueOnce(undefined as never);
+
+ await expect(setAgentAutoApprove({ agentId: 'agent-1', enabled: true })).rejects.toThrow(
+ /No agent with id agent-1/
+ );
+ });
+
+ it('says why the watcher lags when the push failed with something other than an Error', async () => {
+ listAutoSessionAgentIds.mockResolvedValue(['agent-1']);
+ pushRemoteAutoApprove.mockRejectedValueOnce('ssh closed');
+
+ await expect(setAgentAutoApprove({ agentId: 'agent-1', enabled: true })).rejects.toThrow(
+ /could not be updated yet \(ssh closed\)/
+ );
});
it('throws when the agent does not exist', async () => {
- updateAgent.mockResolvedValueOnce(undefined);
+ agentRow.current = undefined;
await expect(setAgentAutoApprove({ agentId: 'ghost', enabled: true })).rejects.toThrow(
/No agent with id ghost/
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.ts b/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.ts
index eb5038df4..2bf5f9f91 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/setAgentAutoApprove.ts
@@ -1,6 +1,11 @@
-import { listAutoSessionAgentIds } from '@main/core/switch-rooms/auto-session-store';
+import { keepAutoApproveChoice, recordAutoApproveOnHost } from '@main/core/sdk-host/shared-watcher';
+import {
+ listAutoSessionAgentIds,
+ listStoppedControllerAgentIds,
+} from '@main/core/switch-rooms/auto-session-store';
import { getRemoteAgentLocation } from './agent-location';
-import { ensureRemoteWatcher } from './remote-watcher';
+import { getAgentById } from './getAgentById';
+import { pushRemoteAutoApprove } from './remote-watcher';
import { updateAgent } from './updateAgent';
export type AgentAutoApproveParams = { agentId: string; enabled: boolean };
@@ -8,24 +13,42 @@ export type AgentAutoApproveParams = { agentId: string; enabled: boolean };
/**
* Toggle an agent's per-agent bypass-permissions setting (CHOO-1664).
*
- * Writes the agent row, then makes the change reach auto-started sessions:
- * - Local agents need nothing extra — the in-process auto-session watcher reads
- * `agent.autoApprove` fresh each time it spawns a session.
- * - Remote agents bake the setting into the VM watcher's launch spec. When the
- * agent's on-VM watcher is running (auto_session enabled), re-ensure the
- * sidecar so the spec file is rewritten with the new value; the running sidecar
- * re-reads it live and applies it to the next auto-started session without a
- * restart. When auto_session is off there is no watcher to refresh — the next
- * `ensureRemoteWatcher` (toggle-on / boot) picks up the current value.
- *
- * The re-ensure is allowed to throw: if the VM is unreachable the setting cannot
- * take effect live, and the caller should surface that rather than pretend it did.
+ * A remote agent's watchers are shared by every Console on the account and read
+ * the choice kept on the host, so the choice goes on the host before the row
+ * and the row never claims a value the host did not take. A failure after that
+ * throws, saying only the running watcher lags.
*/
export async function setAgentAutoApprove(params: AgentAutoApproveParams): Promise<void> {
- const agent = await updateAgent({ agentId: params.agentId, autoApprove: params.enabled });
+ const agent = await getAgentById(params.agentId);
if (!agent) throw new Error(`No agent with id ${params.agentId}`);
-
- if ((await getRemoteAgentLocation(agent)) === null) return;
- if (!(await listAutoSessionAgentIds()).includes(agent.id)) return;
- await ensureRemoteWatcher(agent.id);
+ const setRow = async (autoApprove: boolean) => {
+ if (!(await updateAgent({ agentId: agent.id, autoApprove }))) {
+ throw new Error(`No agent with id ${params.agentId}`);
+ }
+ };
+ if (!agent.switchAgentId || (await getRemoteAgentLocation(agent)) === null) {
+ await setRow(params.enabled);
+ return;
+ }
+ const [spawning, stopped] = await Promise.all([
+ listAutoSessionAgentIds(),
+ listStoppedControllerAgentIds(),
+ ]);
+ if (!spawning.includes(agent.id) || stopped.includes(agent.id)) {
+ await recordAutoApproveOnHost(agent.id, params.enabled);
+ await setRow(params.enabled);
+ return;
+ }
+ await keepAutoApproveChoice(agent.id, params.enabled);
+ await setRow(params.enabled);
+ try {
+ await pushRemoteAutoApprove(agent.id);
+ } catch (error) {
+ const reason = error instanceof Error ? error.message : String(error);
+ throw new Error(
+ `Auto-approve is saved, but the agent's watcher on its host could not be updated yet ` +
+ `(${reason}). It runs its next sessions with the new setting once it is.`,
+ { cause: error }
+ );
+ }
}
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.test.ts b/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.test.ts
index ecbfeb293..9a5794bd3 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.test.ts
@@ -390,7 +390,7 @@ describe('removeSwitchSettings', () => {
agentId: 'agent-123',
});
- expect(removeSwitchSettings(provisioned)).toEqual({ kind: 'delete' });
+ expect(removeSwitchSettings(provisioned, 'agent-123')).toEqual({ kind: 'delete' });
});
it('strips only the SWITCH_* keys and Switch rules, preserving everything else', () => {
@@ -405,7 +405,7 @@ describe('removeSwitchSettings', () => {
},
});
- const result = removeSwitchSettings(existing);
+ const result = removeSwitchSettings(existing, 'agent-123');
expect(result.kind).toBe('write');
const parsed = JSON.parse((result as { content: string }).content) as Record<string, unknown>;
@@ -430,7 +430,7 @@ describe('removeSwitchSettings', () => {
env: { SWITCH_API_ENDPOINT: 'e', SWITCH_API_TOKEN: 't', SWITCH_AGENT_ID: 'a' },
});
- const result = removeSwitchSettings(existing);
+ const result = removeSwitchSettings(existing, 'a');
expect(result.kind).toBe('write');
const parsed = JSON.parse((result as { content: string }).content) as Record<string, unknown>;
@@ -443,20 +443,31 @@ describe('removeSwitchSettings', () => {
it('skips files that are not a provisioned Switch agent', () => {
// Absent file.
- expect(removeSwitchSettings(null)).toEqual({ kind: 'skip' });
+ expect(removeSwitchSettings(null, 'a')).toEqual({ kind: 'skip' });
// Unparseable.
- expect(removeSwitchSettings('{not json')).toEqual({ kind: 'skip' });
+ expect(removeSwitchSettings('{not json', 'a')).toEqual({ kind: 'skip' });
// Empty object.
- expect(removeSwitchSettings('{}')).toEqual({ kind: 'skip' });
+ expect(removeSwitchSettings('{}', 'a')).toEqual({ kind: 'skip' });
// A real config with no Switch creds -> leave it untouched.
- expect(removeSwitchSettings(JSON.stringify({ env: { OTHER: 'x' } }))).toEqual({ kind: 'skip' });
+ expect(removeSwitchSettings(JSON.stringify({ env: { OTHER: 'x' } }), 'a')).toEqual({
+ kind: 'skip',
+ });
});
it('tears down even a partially-provisioned file (only some SWITCH_* keys)', () => {
// Defensive: if a write was interrupted and only the token survived, teardown
// must still remove it rather than leaving a dangling secret.
const existing = JSON.stringify({ env: { SWITCH_API_TOKEN: 'secret-token' } });
- expect(removeSwitchSettings(existing)).toEqual({ kind: 'delete' });
+ expect(removeSwitchSettings(existing, 'agent-123')).toEqual({ kind: 'delete' });
+ });
+
+ it('leaves another agent’s credentials in a shared directory alone', () => {
+ const existing = JSON.stringify({
+ env: { SWITCH_API_ENDPOINT: 'e', SWITCH_API_TOKEN: 't', SWITCH_AGENT_ID: 'agent-main' },
+ });
+
+ expect(removeSwitchSettings(existing, 'agent-sibling')).toEqual({ kind: 'skip' });
+ expect(removeSwitchSettings(existing, null)).toEqual({ kind: 'skip' });
});
it('leaves the directory undetectable as a Switch agent after teardown', async () => {
@@ -467,7 +478,7 @@ describe('removeSwitchSettings', () => {
});
const raw = await fs.readFile(path.join(dir, SWITCH_SETTINGS_RELATIVE_PATH), 'utf8');
- const result = removeSwitchSettings(raw);
+ const result = removeSwitchSettings(raw, 'agent-123');
// The file was ours alone -> delete it, which makes the dir undetectable.
expect(result).toEqual({ kind: 'delete' });
await fs.rm(path.join(dir, SWITCH_SETTINGS_RELATIVE_PATH), { force: true });
diff --git a/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.ts b/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.ts
index 1db3963d3..943d90c17 100644
--- a/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.ts
+++ b/console/apps/switch-console-desktop/src/main/core/agents/write-switch-settings.ts
@@ -185,7 +185,10 @@ export type RemoveSwitchSettingsResult =
| { kind: 'write'; content: string }
| { kind: 'delete' };
-export function removeSwitchSettings(existingRaw: string | null): RemoveSwitchSettingsResult {
+export function removeSwitchSettings(
+ existingRaw: string | null,
+ switchAgentId: string | null
+): RemoveSwitchSettingsResult {
if (existingRaw === null) return { kind: 'skip' };
let parsed: unknown;
@@ -208,6 +211,11 @@ export function removeSwitchSettings(existingRaw: string | null): RemoveSwitchSe
env !== null &&
('SWITCH_API_ENDPOINT' in env || 'SWITCH_API_TOKEN' in env || 'SWITCH_AGENT_ID' in env);
if (!hasSwitchCreds) return { kind: 'skip' };
+ // Another agent's credentials in the legacy one-file layout are its to keep.
+ // A file naming no agent is an interrupted write, and goes.
+ if (typeof env.SWITCH_AGENT_ID === 'string' && env.SWITCH_AGENT_ID !== switchAgentId) {
+ return { kind: 'skip' };
+ }
const result: Record<string, unknown> = { ...existing };
diff --git a/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.test.ts b/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.test.ts
index 1d662128e..a4c0de41c 100644
--- a/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.test.ts
@@ -175,6 +175,36 @@ describe('SshExecutionContext.exec', () => {
expect(isTransportFailure(error)).toBe(false);
expect(error).toMatchObject({ code: 1 });
});
+
+ it('never takes a channel that closed without an exit status for a success', async () => {
+ // A dropped `docker ps` yields empty output, which must not read as "no containers".
+ const stream = makeStream();
+ const ctx = new SshExecutionContext(
+ makeProxy((_command, cb) => {
+ cb(undefined, stream);
+ queueMicrotask(() => {
+ stream.emit('data', Buffer.from(`${EXEC_STDOUT_MARKER}\n`));
+ stream.emit('close');
+ });
+ })
+ );
+
+ await expect(ctx.exec('docker', ['ps'])).rejects.toThrow(
+ 'The command ended without an exit status; the SSH connection may have dropped.'
+ );
+ });
+
+ it('reports a command killed by a signal as failed', async () => {
+ const stream = makeStream();
+ const ctx = new SshExecutionContext(
+ makeProxy((_command, cb) => {
+ cb(undefined, stream);
+ queueMicrotask(() => stream.emit('close', null, 'SIGTERM'));
+ })
+ );
+
+ await expect(ctx.exec('docker', ['ps'])).rejects.toThrow('The command was killed by SIGTERM.');
+ });
});
describe('stripExecBanner', () => {
diff --git a/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.ts b/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.ts
index 71fb02202..a74b8d534 100644
--- a/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.ts
+++ b/console/apps/switch-console-desktop/src/main/core/execution-context/ssh-execution-context.ts
@@ -79,6 +79,18 @@ export function buildSshCommand(
return buildRemoteShellCommand(profile ?? FALLBACK_REMOTE_SHELL_PROFILE, body);
}
+/** Why a remote command that did not exit 0 failed, from what ssh2 reported. */
+export function exitDescription(
+ code: number | null | undefined,
+ signal: string | undefined
+): string {
+ if (code === undefined) {
+ return 'The command ended without an exit status; the SSH connection may have dropped.';
+ }
+ if (code === null) return `The command was killed${signal ? ` by ${signal}` : ''}.`;
+ return `Process exited with code ${code}`;
+}
+
export class SshExecutionContext implements IExecutionContext {
readonly root?: string;
readonly supportsLocalSpawn = false;
@@ -165,14 +177,17 @@ export class SshExecutionContext implements IExecutionContext {
stderr += d.toString('utf-8');
});
- stream.on('close', (code: number | null) => {
+ // ssh2 passes the exit status, null plus a signal when the command was
+ // killed, or nothing when the channel closed without either (a dropped
+ // connection). Only 0 is success; partial output is not an answer.
+ stream.on('close', (code: number | null | undefined, signal?: string) => {
settle(() => {
const cleanStdout = stripExecBanner(stdout, EXEC_STDOUT_MARKER);
- if ((code ?? 0) === 0) {
+ if (code === 0) {
resolve({ stdout: cleanStdout, stderr });
} else {
reject(
- Object.assign(new Error(stderr || `Process exited with code ${code}`), {
+ Object.assign(new Error(stderr || exitDescription(code, signal)), {
stdout: cleanStdout,
stderr,
code: code ?? undefined,
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts
index 211073f8e..8346226d1 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/compose.ts
@@ -2,9 +2,11 @@ import { log } from '@main/lib/logger';
import {
BUILD_OVERRIDE_FILE_NAME,
COMPOSE_FILE_NAME,
+ CORE_SERVICE,
ENV_FILE_NAME,
LOCAL_SERVER_PROFILES,
} from './constants';
+import { commandFailure } from './error-text';
import type { ServerHost } from './host/types';
/** Pulls can take minutes on a cold machine; give compose a generous ceiling. */
@@ -44,9 +46,7 @@ export async function runDocker(
});
return stdout;
} catch (error) {
- const stderr = (error as { stderr?: string } | undefined)?.stderr;
- const message = stderr?.trim() || (error instanceof Error ? error.message : String(error));
- throw new Error(`${full.join(' ')} failed: ${message}`);
+ throw new Error(`${full.join(' ')} failed: ${commandFailure(error)}`);
}
}
@@ -182,9 +182,9 @@ export async function runningServices(host: ServerHost): Promise<string[]> {
}
}
-/** Whether the core `switch` service is up — our proxy for "the stack is up". */
+/** Whether the core service is up — our proxy for "the stack is up". */
export async function isStackRunning(host: ServerHost): Promise<boolean> {
- return (await runningServices(host)).includes('switch');
+ return (await runningServices(host)).includes(CORE_SERVICE);
}
/**
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.test.ts
new file mode 100644
index 000000000..5eeb0d7f4
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.test.ts
@@ -0,0 +1,332 @@
+import { execFileSync } from 'node:child_process';
+import { chmodSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import type { StackStateHost } from './stack-state';
+import type * as StackState from './stack-state';
+import { UNDER_STATE_MUTEX } from './state-mutex';
+import { stateScriptEnv } from './test-helpers/state-script-shell';
+
+const getConsoleIdentity = vi.hoisted(() =>
+ vi.fn(() => Promise.resolve({ id: '3f2a9c1e-5b7d-4e8f-a1b2-c3d4e5f6a7b8', name: 'alice@laptop' }))
+);
+const stateVolumeExists = vi.hoisted(() => vi.fn());
+const readStateVolume = vi.hoisted(() => vi.fn());
+const writeStateVolume = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const logWarn = vi.hoisted(() => vi.fn());
+
+vi.mock('@main/core/switch-servers/console-identity', () => ({ getConsoleIdentity }));
+vi.mock('@main/core/app/utils', () => ({ resolveAppVersion: () => Promise.resolve('0.36.0') }));
+vi.mock('@main/lib/logger', () => ({ log: { warn: logWarn } }));
+vi.mock('./stack-state', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackState>()),
+ stateVolumeExists,
+ readStateVolume,
+ writeStateVolume,
+}));
+
+const { RECORD_SCRIPT, readRegister, writeRecord } = await import('./console-register');
+
+const SELF = '3f2a9c1e-5b7d-4e8f-a1b2-c3d4e5f6a7b8';
+
+let hosts = 0;
+
+function host(account = 'alice') {
+ const exec = vi.fn(async () => ({ stdout: `${account}\n`, stderr: '' }));
+ return {
+ // A label of its own each time: the account is remembered per host label.
+ host: { label: `vm-${++hosts}`, ctx: { exec } } as unknown as StackStateHost,
+ exec,
+ };
+}
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ vi.useFakeTimers({ now: new Date('2026-09-23T15:00:00.000Z'), toFake: ['Date'] });
+});
+
+afterEach(() => {
+ vi.useRealTimers();
+});
+
+describe('recording a Console on the host', () => {
+ it('writes its register entry and a line of activity, both on stdin', async () => {
+ const { host: h, exec } = host();
+
+ await writeRecord(h, 'stopped');
+
+ expect(exec).toHaveBeenCalledWith('id', ['-un'], expect.anything());
+ const [, script, input, args] = writeStateVolume.mock.calls[0] as unknown as [
+ unknown,
+ string,
+ string,
+ string[],
+ ];
+ expect(args).toEqual([SELF, 'act']);
+ const [entry, activity, rest] = input.split('\n');
+ expect(JSON.parse(entry!)).toEqual({
+ consoleId: SELF,
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ appVersion: '0.36.0',
+ lastSeenAt: '2026-09-23T15:00:00.000Z',
+ });
+ expect(JSON.parse(activity!)).toEqual({
+ at: '2026-09-23T15:00:00.000Z',
+ action: 'stopped',
+ consoleId: SELF,
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ });
+ expect(rest).toBe('');
+ expect(script).not.toContain('alice');
+ expect(script).toContain('read -r entry');
+ });
+
+ it('refreshes only the register for a quiet sighting', async () => {
+ const { host: h } = host();
+
+ await writeRecord(h, null);
+
+ const [, , input, args] = writeStateVolume.mock.calls[0] as unknown as [
+ unknown,
+ string,
+ string,
+ string[],
+ ];
+ expect(args).toEqual([SELF, 'seen']);
+ expect(input.trim().split('\n')).toHaveLength(1);
+ });
+
+ it('takes a disconnecting Console off the register, keeping the line of activity', async () => {
+ const { host: h } = host();
+
+ await writeRecord(h, 'disconnected');
+
+ const [, , input, args] = writeStateVolume.mock.calls[0] as unknown as [
+ unknown,
+ string,
+ string,
+ string[],
+ ];
+ expect(args).toEqual([SELF, 'leave']);
+ expect(JSON.parse(input.split('\n')[1]!)).toMatchObject({ action: 'disconnected' });
+ });
+
+ it('asks a host which account it is once, not per record or per connection', async () => {
+ const { host: h, exec } = host();
+ const again = { ...h, ctx: { exec } } as unknown as StackStateHost;
+
+ await writeRecord(h, 'started');
+ await writeRecord(again, null);
+
+ expect(exec).toHaveBeenCalledOnce();
+ expect(writeStateVolume).toHaveBeenCalledTimes(2);
+ });
+
+ it('asks again after the host failed to say which account it is', async () => {
+ const { host: h, exec } = host();
+ exec.mockRejectedValueOnce(new Error('ssh dropped'));
+
+ await expect(writeRecord(h, 'started')).rejects.toThrow(/ssh dropped/);
+ await writeRecord(h, 'started');
+
+ expect(exec).toHaveBeenCalledTimes(2);
+ expect(writeStateVolume).toHaveBeenCalledOnce();
+ });
+
+ it('records the account as unknown when the host names none', async () => {
+ const { host: h } = host('');
+
+ await writeRecord(h, 'started');
+
+ const [, , input] = writeStateVolume.mock.calls[0] as unknown as [unknown, unknown, string];
+ expect(input).toContain('"hostAccount":"unknown"');
+ });
+
+ it('refuses to use an id that is not one as a file name', async () => {
+ getConsoleIdentity.mockResolvedValueOnce({ id: '../../etc/passwd', name: 'x' });
+ const { host: h } = host();
+
+ await expect(writeRecord(h, 'started')).rejects.toThrow(/not one/);
+ expect(writeStateVolume).not.toHaveBeenCalled();
+ });
+});
+
+describe('the record script, run for real', () => {
+ let state: string;
+
+ beforeEach(() => {
+ state = mkdtempSync(path.join(tmpdir(), 'console-register-'));
+ });
+
+ afterEach(() => {
+ rmSync(state, { recursive: true, force: true });
+ });
+
+ const env = stateScriptEnv();
+
+ function run(id: string, mode: string, input: string): void {
+ const script = RECORD_SCRIPT.replaceAll('/state', state);
+ execFileSync('sh', ['-c', script, 'record', id, mode], { input, env });
+ }
+
+ it('does everything under the state mutex, so two Consoles recording at once lose nothing', () => {
+ // Whether the mutex holds is tested in stack-lock.docker.test.ts.
+ expect(RECORD_SCRIPT.startsWith(`set -e\n${UNDER_STATE_MUTEX}\n`)).toBe(true);
+ });
+
+ const entry = (name: string) => JSON.stringify({ consoleId: name, lastSeenAt: 'now' });
+ const line = (action: string) => JSON.stringify({ action });
+
+ it('fails, rather than reporting a record, when the write does not happen', () => {
+ // A read-only dir stands in for a full disk or read-only volume.
+ mkdirSync(path.join(state, 'consoles'));
+ chmodSync(path.join(state, 'consoles'), 0o500);
+
+ expect(() => run('aaa', 'act', `${entry('aaa')}\n${line('started')}\n`)).toThrow();
+ chmodSync(path.join(state, 'consoles'), 0o700);
+ });
+
+ it('keeps one entry per Console and appends activity only when something was done', () => {
+ run('aaa', 'act', `${entry('aaa')}\n${line('started')}\n`);
+ run('bbb', 'seen', `${entry('bbb')}\n`);
+ run('aaa', 'seen', `${entry('aaa')}\n`);
+
+ expect(readdirSync(path.join(state, 'consoles')).sort()).toEqual(['aaa.json', 'bbb.json']);
+ expect(readFileSync(path.join(state, 'activity.jsonl'), 'utf8')).toBe(`${line('started')}\n`);
+ });
+
+ it('removes only the leaving Console’s entry, and records that it left', () => {
+ run('aaa', 'act', `${entry('aaa')}\n${line('connected')}\n`);
+ run('bbb', 'act', `${entry('bbb')}\n${line('connected')}\n`);
+
+ run('bbb', 'leave', `${entry('bbb')}\n${line('disconnected')}\n`);
+
+ expect(readdirSync(path.join(state, 'consoles'))).toEqual(['aaa.json']);
+ expect(readFileSync(path.join(state, 'activity.jsonl'), 'utf8').trim().split('\n')).toEqual([
+ line('connected'),
+ line('connected'),
+ line('disconnected'),
+ ]);
+ });
+
+ it('lets a Console leave that was never recorded', () => {
+ run('ccc', 'leave', `${entry('ccc')}\n${line('disconnected')}\n`);
+
+ expect(readdirSync(path.join(state, 'consoles'))).toEqual([]);
+ });
+});
+
+describe('reading the register', () => {
+ const alice = {
+ consoleId: SELF,
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ appVersion: '0.36.0',
+ lastSeenAt: '2026-09-23T10:00:00.000Z',
+ };
+ const bob = {
+ consoleId: 'b0b0b0b0-0000-4000-8000-000000000000',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.36.0',
+ lastSeenAt: '2026-09-23T12:00:00.000Z',
+ };
+ const started = {
+ at: '2026-09-23T09:00:00.000Z',
+ action: 'started',
+ consoleId: SELF,
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ };
+ const stopped = {
+ ...started,
+ at: '2026-09-23T11:00:00.000Z',
+ action: 'stopped',
+ consoleId: bob.consoleId,
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ };
+
+ it('lists the Consoles most recently seen first, and the activity newest first', async () => {
+ stateVolumeExists.mockResolvedValue(true);
+ readStateVolume.mockResolvedValue(
+ [
+ JSON.stringify(alice),
+ JSON.stringify(bob),
+ '---switch-console-activity---',
+ JSON.stringify(started),
+ JSON.stringify(stopped),
+ '',
+ ].join('\n')
+ );
+ const { host: h } = host();
+
+ expect(await readRegister(h)).toEqual({
+ self: SELF,
+ consoles: [bob, alice],
+ activity: [stopped, started],
+ });
+ });
+
+ it('skips a line it cannot read rather than hiding everyone else', async () => {
+ stateVolumeExists.mockResolvedValue(true);
+ readStateVolume.mockResolvedValue(
+ [
+ JSON.stringify(alice),
+ '{"consoleId":',
+ JSON.stringify({ ...bob, lastSeenAt: 7 }),
+ '---switch-console-activity---',
+ JSON.stringify({ ...started, action: 'exploded' }),
+ JSON.stringify(stopped),
+ ].join('\n')
+ );
+ const { host: h } = host();
+
+ const register = await readRegister(h);
+
+ expect(register.consoles).toEqual([alice]);
+ expect(register.activity).toEqual([stopped]);
+ expect(logWarn).toHaveBeenCalledWith(expect.stringContaining('skipped unreadable'), {
+ consoles: 2,
+ activity: 1,
+ });
+ });
+
+ it('skips lines that are not records at all', async () => {
+ stateVolumeExists.mockResolvedValue(true);
+ readStateVolume.mockResolvedValue(
+ [
+ JSON.stringify(alice),
+ '"just a string"',
+ '---switch-console-activity---',
+ '42',
+ 'null',
+ ].join('\n')
+ );
+ const { host: h } = host();
+
+ const register = await readRegister(h);
+
+ expect(register.consoles).toEqual([alice]);
+ expect(register.activity).toEqual([]);
+ });
+
+ it('reads a register that has no activity yet', async () => {
+ stateVolumeExists.mockResolvedValue(true);
+ readStateVolume.mockResolvedValue(`${JSON.stringify(bob)}\n`);
+ const { host: h } = host();
+
+ expect(await readRegister(h)).toEqual({ self: SELF, consoles: [bob], activity: [] });
+ });
+
+ it('is empty, not an error, before anything has been recorded', async () => {
+ stateVolumeExists.mockResolvedValue(false);
+ const { host: h } = host();
+
+ expect(await readRegister(h)).toEqual({ self: SELF, consoles: [], activity: [] });
+ expect(readStateVolume).not.toHaveBeenCalled();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.ts
new file mode 100644
index 000000000..d06bf8c6c
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/console-register.ts
@@ -0,0 +1,216 @@
+import { resolveAppVersion } from '@main/core/app/utils';
+import { getConsoleIdentity } from '@main/core/switch-servers/console-identity';
+import { log } from '@main/lib/logger';
+import type {
+ StackActivityAction,
+ StackActivityEntry,
+ StackConsole,
+ StackRegister,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+import {
+ readStateVolume,
+ type StackStateHost,
+ stateVolumeExists,
+ writeStateVolume,
+} from './stack-state';
+import { CONSOLE_ID_PATTERN, UNDER_STATE_MUTEX } from './state-mutex';
+
+/**
+ * Who uses a shared remote stack and what they last did to it, kept in its
+ * state volume (CHOO-2893). Everyone signs in to the server as its one admin,
+ * so the server cannot say who stopped it; each Console records itself in
+ * `consoles/<id>.json` and an appended `activity.jsonl` instead. It is a
+ * record, not a control: nothing is allowed or refused on its strength, and a
+ * Console that fails to write it still does what it was asked.
+ */
+
+/** Only the latest entries are read back; the file keeps a few hundred. */
+const ACTIVITY_SHOWN = 50;
+const ACTIVITY_TRIM_AT = 1000;
+const ACTIVITY_KEPT = 500;
+
+/** Separates the two halves of a read, which come back as one stdout. */
+const ACTIVITY_MARKER = '---switch-console-activity---';
+
+/** `seen` refreshes this Console's entry, `act` also adds an activity line,
+ * and `leave` adds the line and removes the entry, so a disconnected Console
+ * is no longer named as using the stack. */
+type RecordMode = 'seen' | 'act' | 'leave';
+
+function recordModeFor(action: StackActivityAction | null): RecordMode {
+ if (action === null) return 'seen';
+ return action === 'disconnected' ? 'leave' : 'act';
+}
+
+/** Applies a {@link RecordMode} (`$2`) for the Console `$1`, reading the entry
+ * and activity line from stdin so no value is spliced into the script. */
+export const RECORD_SCRIPT = [
+ 'set -e',
+ UNDER_STATE_MUTEX,
+ 'umask 077',
+ 'mkdir -p /state/consoles',
+ 'IFS= read -r entry || exit 1',
+ 'if [ "$2" = leave ]; then',
+ ' rm -f "/state/consoles/$1.json"',
+ 'else',
+ ' printf "%s\\n" "$entry" > "/state/consoles/.$1.tmp"',
+ ' mv "/state/consoles/.$1.tmp" "/state/consoles/$1.json"',
+ 'fi',
+ 'if [ "$2" != seen ]; then',
+ ' IFS= read -r activity || exit 1',
+ ' printf "%s\\n" "$activity" >> /state/activity.jsonl',
+ ` if [ "$(wc -l < /state/activity.jsonl)" -gt ${ACTIVITY_TRIM_AT} ]; then`,
+ ` tail -n ${ACTIVITY_KEPT} /state/activity.jsonl > /state/.activity.tmp`,
+ ' mv /state/.activity.tmp /state/activity.jsonl',
+ ' fi',
+ 'fi',
+].join('\n');
+
+const READ_SCRIPT = [
+ 'for f in /state/consoles/*.json; do [ -f "$f" ] && cat "$f"; done',
+ `echo '${ACTIVITY_MARKER}'`,
+ `tail -n ${ACTIVITY_SHOWN} /state/activity.jsonl 2>/dev/null || true`,
+].join('\n');
+
+const ACTIONS: readonly StackActivityAction[] = [
+ 'started',
+ 'connected',
+ 'stopped',
+ 'reset',
+ 'disconnected',
+];
+
+/** By host label. An SSH alias always logs in as one account, so it is asked
+ * once per run rather than per operation. */
+const accounts = new Map<string, Promise<string>>();
+
+export function hostAccount(host: StackStateHost): Promise<string> {
+ let account = accounts.get(host.label);
+ if (!account) {
+ account = host.ctx
+ .exec('id', ['-un'], { timeout: 20_000 })
+ .then(({ stdout }) => stdout.trim() || 'unknown');
+ // A failed ask is not remembered: the next record asks again.
+ account.catch(() => accounts.delete(host.label));
+ accounts.set(host.label, account);
+ }
+ return account;
+}
+
+/** `action` null is a quiet sighting that only refreshes the entry. Throws on
+ * failure: the supervisor decides what a failed record means for its operation. */
+export async function writeRecord(
+ host: StackStateHost,
+ action: StackActivityAction | null
+): Promise<void> {
+ const identity = await getConsoleIdentity();
+ if (!CONSOLE_ID_PATTERN.test(identity.id)) {
+ throw new Error(`Refusing to record a console id that is not one: ${identity.id}`);
+ }
+ const at = new Date().toISOString();
+ const account = await hostAccount(host);
+ const entry: StackConsole = {
+ consoleId: identity.id,
+ name: identity.name,
+ hostAccount: account,
+ appVersion: await resolveAppVersion(),
+ lastSeenAt: at,
+ };
+ const lines = [JSON.stringify(entry)];
+ if (action !== null) {
+ const activity: StackActivityEntry = {
+ at,
+ action,
+ consoleId: identity.id,
+ name: identity.name,
+ hostAccount: account,
+ };
+ lines.push(JSON.stringify(activity));
+ }
+ await writeStateVolume(host, RECORD_SCRIPT, `${lines.join('\n')}\n`, [
+ identity.id,
+ recordModeFor(action),
+ ]);
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+ return typeof value === 'object' && value !== null;
+}
+
+function asConsole(value: unknown): StackConsole | null {
+ if (!isRecord(value)) return null;
+ const { consoleId, name, hostAccount, appVersion, lastSeenAt } = value;
+ if (
+ typeof consoleId !== 'string' ||
+ typeof name !== 'string' ||
+ typeof hostAccount !== 'string' ||
+ typeof appVersion !== 'string' ||
+ typeof lastSeenAt !== 'string'
+ )
+ return null;
+ return { consoleId, name, hostAccount, appVersion, lastSeenAt };
+}
+
+function asActivity(value: unknown): StackActivityEntry | null {
+ if (!isRecord(value)) return null;
+ const { at, action, consoleId, name, hostAccount } = value;
+ if (
+ typeof at !== 'string' ||
+ typeof action !== 'string' ||
+ !(ACTIONS as readonly string[]).includes(action) ||
+ typeof consoleId !== 'string' ||
+ typeof name !== 'string' ||
+ typeof hostAccount !== 'string'
+ )
+ return null;
+ return { at, action: action as StackActivityAction, consoleId, name, hostAccount };
+}
+
+function parseLines<T>(
+ text: string,
+ as: (value: unknown) => T | null
+): { items: T[]; bad: number } {
+ const items: T[] = [];
+ let bad = 0;
+ for (const line of text.split('\n')) {
+ const trimmed = line.trim();
+ if (!trimmed) continue;
+ let parsed: unknown;
+ try {
+ parsed = JSON.parse(trimmed);
+ } catch {
+ bad++;
+ continue;
+ }
+ const item = as(parsed);
+ if (item === null) bad++;
+ else items.push(item);
+ }
+ return { items, bad };
+}
+
+/** Newest first; empty on a host with no state volume yet. Unparseable lines
+ * are skipped and logged: one Console's bad write must not hide everyone else. */
+export async function readRegister(host: StackStateHost): Promise<StackRegister> {
+ const self = (await getConsoleIdentity()).id;
+ if (!(await stateVolumeExists(host))) return { self, consoles: [], activity: [] };
+
+ const out = await readStateVolume(host, READ_SCRIPT);
+ const split = out.indexOf(ACTIVITY_MARKER);
+ const consolesText = split === -1 ? out : out.slice(0, split);
+ const activityText = split === -1 ? '' : out.slice(split + ACTIVITY_MARKER.length);
+
+ const consoles = parseLines(consolesText, asConsole);
+ const activity = parseLines(activityText, asActivity);
+ if (consoles.bad + activity.bad > 0) {
+ log.warn(`remote-switch-server: skipped unreadable register lines on ${host.label}`, {
+ consoles: consoles.bad,
+ activity: activity.bad,
+ });
+ }
+ return {
+ self,
+ consoles: consoles.items.sort((a, b) => b.lastSeenAt.localeCompare(a.lastSeenAt)),
+ activity: activity.items.reverse(),
+ };
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/constants.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/constants.ts
index a4fa4d984..dd082ba78 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/constants.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/constants.ts
@@ -29,11 +29,28 @@ export const LOCAL_SERVER_PROJECT_NAME =
export const REMOTE_SERVER_PROJECT_NAME =
USER_DATA_DIR_NAME === 'switchdash' ? 'switchdash-remote' : `${USER_DATA_DIR_NAME}-remote`;
+/**
+ * The volume holding what every Console sharing a remote stack must agree on:
+ * the published `.env`, the register and the activity log. Labelled with
+ * {@link STACK_STATE_LABEL} rather than compose's project label, so compose
+ * never removes it and a reset keeps the record of who reset the stack.
+ */
+export const STACK_STATE_VOLUME_SUFFIX = 'console-state';
+export const STACK_STATE_LABEL = 'com.switch-console.stack-state';
+
+/** For the throwaway containers that reach the state volume. The stack's own
+ * Postgres image, so any host that has run the stack already has it;
+ * `stack-state.test.ts` holds it to the bundled compose file. */
+export const STACK_HELPER_IMAGE = 'postgres:16-alpine';
+
/** File names of the compose file and generated `.env` inside the host working
* dir. Kept relative so `docker compose -f <name>` resolves against the working
* dir on either host, independent of its absolute path. */
export const COMPOSE_FILE_NAME = 'standalone-docker-compose.yml';
export const ENV_FILE_NAME = '.env';
+/** The database volume the `.env` beside it was written for, like the
+ * published copy's stamp. */
+export const ENV_STAMP_FILE_NAME = '.env.db';
/** File name of the dev-only build override layered on top of the compose file
* when the stack is built from a local checkout (see checkout-build.ts). */
@@ -76,3 +93,7 @@ export const LOCAL_SERVER_MATTERMOST_TEAM = 'switch';
* name the seeder actually used — the generated env, the embed's silent login,
* and the sign-in Switch Console shows the user — so it is named once here. */
export const LOCAL_SERVER_MATTERMOST_USER = 'user';
+
+/** Its container stands for "the stack is up", its image tag is the stack's
+ * version, and its environment carries the server's settings. */
+export const CORE_SERVICE = 'switch';
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/deployed-version.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/deployed-version.ts
index dee7b0831..e46afff1e 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/deployed-version.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/deployed-version.ts
@@ -5,8 +5,9 @@ import type {
SwitchVersionDriftDirection,
} from '@shared/core/managed-switch-server/managed-switch-server';
import { runningImages } from './compose';
-import { ENV_FILE_NAME } from './constants';
+import { CORE_SERVICE, ENV_FILE_NAME } from './constants';
import { readEnvValue } from './env-file';
+import { errorText } from './error-text';
import type { ServerHost } from './host/types';
/**
@@ -18,9 +19,6 @@ import type { ServerHost } from './host/types';
* other side of the comparison, read back off the host.
*/
-/** The core service whose image tag defines the stack's version. */
-const CORE_SERVICE = 'switch';
-
export type DeployedVersion =
/** Read successfully. `source` records how, since the two differ in strength:
* `container` is what is running right now; `env-file` is what the last start
@@ -135,7 +133,7 @@ export async function readVersionStatus(
/** The tag of a `[registry[:port]/]repo[:tag]` image reference, or null when it
* carries no tag (an implicit `latest`, or a `@sha256:` digest pin — neither
* names a version we can compare). */
-function imageTag(image: string): string | null {
+export function imageTag(image: string): string | null {
const lastSlash = image.lastIndexOf('/');
const name = lastSlash === -1 ? image : image.slice(lastSlash + 1);
if (name.includes('@')) return null;
@@ -144,7 +142,3 @@ function imageTag(image: string): string | null {
const tag = name.slice(colon + 1).trim();
return tag.length > 0 ? tag : null;
}
-
-function errorText(error: unknown): string {
- return error instanceof Error ? error.message : String(error);
-}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
index d527965aa..b23f90ceb 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.test.ts
@@ -2,7 +2,7 @@ import { readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { describe, expect, it } from 'vitest';
-import { buildEnvFile } from './env-file';
+import { buildEnvFile, keysDisagreeing, readStackEnv, telemetryRequested } from './env-file';
import type { LocalServerSecrets } from './secret-values';
const secrets: LocalServerSecrets = {
@@ -154,3 +154,167 @@ describe('buildEnvFile', () => {
expect(vars.FRONTEND_BASE_URL).toBe('http://localhost:51000');
});
});
+
+describe('readStackEnv', () => {
+ const ports = { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 };
+ const written = buildEnvFile({
+ version: '1.2.3',
+ registry: 'ghcr.io',
+ namespace: 'sandbox-quantum',
+ ports,
+ secrets,
+ telemetryEnabled: true,
+ });
+
+ it('reads back exactly what buildEnvFile wrote', () => {
+ expect(readStackEnv(written)).toEqual({
+ kind: 'complete',
+ env: { ports, secrets, version: '1.2.3' },
+ });
+ });
+
+ it('reads a file written before the database role split', () => {
+ const legacy = written
+ .replace('DB_USER=switch_app', 'DB_USER=postgres')
+ .replace(`DB_PASSWORD=${secrets.dbRuntimePassword}`, `DB_PASSWORD=${secrets.dbPassword}`)
+ .replace(/^DB_OWNER_USER=.*$/m, '')
+ .replace(/^DB_OWNER_PASSWORD=.*$/m, '');
+
+ expect(readStackEnv(legacy)).toEqual({
+ kind: 'complete',
+ env: {
+ ports,
+ secrets: { ...secrets, dbRuntimePassword: null },
+ version: '1.2.3',
+ },
+ });
+ });
+
+ it('names the owner password a file from before the role split is missing', () => {
+ const legacy = written
+ .replace('DB_USER=switch_app', 'DB_USER=postgres')
+ .replace(/^DB_PASSWORD=.*$/m, '')
+ .replace(/^DB_OWNER_USER=.*$/m, '')
+ .replace(/^DB_OWNER_PASSWORD=.*$/m, '');
+
+ expect(readStackEnv(legacy)).toMatchObject({
+ kind: 'incomplete',
+ missing: expect.arrayContaining(['DB_PASSWORD']),
+ });
+ });
+
+ it('reads whether a stack asks to share usage data', () => {
+ expect(telemetryRequested('TELEMETRY_ENABLED=true\n')).toBe(true);
+ expect(telemetryRequested('TELEMETRY_ENABLED=false\n')).toBe(false);
+ // Absent is off: switch-core's own default for the gate.
+ expect(telemetryRequested('GATEWAY_HOST_PORT=3300\n')).toBe(false);
+ });
+
+ it('names every key it could not find instead of inventing a value', () => {
+ const partial = written
+ .replace(/^JWT_SECRET_KEY=.*$/m, '')
+ .replace(/^API_HOST_PORT=.*$/m, 'API_HOST_PORT=')
+ .replace(/^MATTERMOST_USER_PASSWORD=.*$/m, '');
+
+ expect(readStackEnv(partial)).toEqual({
+ kind: 'incomplete',
+ missing: ['API_HOST_PORT', 'JWT_SECRET_KEY', 'MATTERMOST_USER_PASSWORD'],
+ });
+ });
+
+ it('does not read a port that is not one', () => {
+ const garbled = written
+ .replace('GATEWAY_HOST_PORT=51000', 'GATEWAY_HOST_PORT=http://localhost:51000')
+ .replace('POSTGRES_HOST_PORT=51003', 'POSTGRES_HOST_PORT=70000');
+
+ expect(readStackEnv(garbled)).toEqual({
+ kind: 'incomplete',
+ missing: ['GATEWAY_HOST_PORT', 'POSTGRES_HOST_PORT'],
+ });
+ });
+
+ it('refuses a current-layout file whose runtime password is gone', () => {
+ const noRuntime = written.replace(/^DB_PASSWORD=.*$/m, '');
+
+ expect(readStackEnv(noRuntime)).toEqual({ kind: 'incomplete', missing: ['DB_PASSWORD'] });
+ });
+
+ it('refuses a file that names no schema owner at all', () => {
+ const noOwner = written
+ .replace(/^DB_OWNER_PASSWORD=.*$/m, '')
+ .replace(/^DB_OWNER_USER=.*$/m, '');
+
+ expect(readStackEnv(noOwner)).toEqual({
+ kind: 'incomplete',
+ missing: ['DB_OWNER_PASSWORD'],
+ });
+ });
+
+ it('treats an empty file as missing everything', () => {
+ const reading = readStackEnv('');
+
+ expect(reading.kind).toBe('incomplete');
+ expect(reading.kind === 'incomplete' && reading.missing).toHaveLength(10);
+ });
+});
+
+describe('keysDisagreeing', () => {
+ const ports = { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 };
+ const written = buildEnvFile({
+ version: '1.2.3',
+ registry: 'ghcr.io',
+ namespace: 'sandbox-quantum',
+ ports,
+ secrets,
+ telemetryEnabled: true,
+ });
+
+ it('finds nothing wrong with a copy of the same settings', () => {
+ expect(keysDisagreeing(written, { ports, secrets })).toEqual([]);
+ });
+
+ it('names the ports and credentials a copy of another generation gets wrong', () => {
+ const copy = {
+ ports: { ...ports, gateway: 3300 },
+ secrets: { ...secrets, dbPassword: 'old-owner-pw', gatewayAdminPassword: 'old-admin' },
+ };
+
+ expect(keysDisagreeing(written, copy).sort()).toEqual([
+ 'DB_OWNER_PASSWORD',
+ 'GATEWAY_ADMIN_PASSWORD',
+ 'GATEWAY_HOST_PORT',
+ ]);
+ });
+
+ it('judges only what the file carries, and not what a start rewrites', () => {
+ const partial = written
+ .replace(/^JWT_SECRET_KEY=.*$/m, '')
+ .replace('SWITCH_VERSION=1.2.3', 'SWITCH_VERSION=0.0.1');
+
+ expect(keysDisagreeing(partial, { ports, secrets: { ...secrets, jwtSecretKey: 'x' } })).toEqual(
+ []
+ );
+ });
+
+ it('reads DB_PASSWORD as the owner’s in a file written before the role split', () => {
+ const legacy = written
+ .replace('DB_USER=switch_app', 'DB_USER=postgres')
+ .replace(`DB_PASSWORD=${secrets.dbRuntimePassword}`, `DB_PASSWORD=${secrets.dbPassword}`)
+ .replace(/^DB_OWNER_USER=.*$/m, '')
+ .replace(/^DB_OWNER_PASSWORD=.*$/m, '');
+
+ expect(keysDisagreeing(legacy, { ports, secrets })).toEqual([]);
+ expect(
+ keysDisagreeing(legacy, { ports, secrets: { ...secrets, dbPassword: 'other' } })
+ ).toEqual(['DB_PASSWORD']);
+ });
+
+ it('reads DB_PASSWORD as the runtime role’s when the owner’s line is what is missing', () => {
+ const partial = written.replace(/^DB_OWNER_PASSWORD=.*$/m, '');
+
+ expect(keysDisagreeing(partial, { ports, secrets })).toEqual([]);
+ expect(
+ keysDisagreeing(partial, { ports, secrets: { ...secrets, dbRuntimePassword: 'other' } })
+ ).toEqual(['DB_PASSWORD']);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
index a554c0415..330f11c69 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/env-file.ts
@@ -113,3 +113,157 @@ export function readEnvValue(env: string, key: string): string | null {
}
return null;
}
+
+/**
+ * What another Console needs to run the same stack. `dbRuntimePassword` is
+ * null for a `.env` from before switch-core split its database roles; the
+ * caller fills it in (see `withRuntimePassword` in `secret-values.ts`).
+ */
+export type StackEnv = {
+ ports: LocalServerPorts;
+ secrets: Omit<LocalServerSecrets, 'dbRuntimePassword'> & { dbRuntimePassword: string | null };
+ version: string | null;
+};
+
+export type StackEnvReading =
+ | { kind: 'complete'; env: StackEnv }
+ /** Never guessed: minting a replacement for any of these would lock the
+ * stack out of a volume the original created. */
+ | { kind: 'incomplete'; missing: string[] };
+
+const PORT_KEYS: [keyof LocalServerPorts, string][] = [
+ ['gateway', 'GATEWAY_HOST_PORT'],
+ ['api', 'API_HOST_PORT'],
+ ['mattermost', 'MATTERMOST_HOST_PORT'],
+ ['postgres', 'POSTGRES_HOST_PORT'],
+];
+
+type PlainSecret =
+ | 'agentRegistrationToken'
+ | 'jwtSecretKey'
+ | 'gatewayAdminPassword'
+ | 'mattermostAdminPassword'
+ | 'mattermostUserPassword';
+
+const SECRET_KEYS: [PlainSecret, string][] = [
+ ['agentRegistrationToken', 'AGENT_REGISTRATION_TOKEN'],
+ ['jwtSecretKey', 'JWT_SECRET_KEY'],
+ ['gatewayAdminPassword', 'GATEWAY_ADMIN_PASSWORD'],
+ ['mattermostAdminPassword', 'MATTERMOST_ADMIN_PASSWORD'],
+ ['mattermostUserPassword', 'MATTERMOST_USER_PASSWORD'],
+];
+
+function readPort(env: string, key: string): number | null {
+ const raw = readEnvValue(env, key);
+ if (raw === null || !/^\d{1,5}$/.test(raw)) return null;
+ const port = Number.parseInt(raw, 10);
+ return port > 0 && port < 65536 ? port : null;
+}
+
+/** The two database passwords, from whichever layout wrote the file. */
+function readDatabasePasswords(
+ env: string,
+ missing: string[]
+): { dbPassword: string; dbRuntimePassword: string | null } | null {
+ const ownerPassword = readEnvValue(env, 'DB_OWNER_PASSWORD');
+ if (ownerPassword !== null) {
+ const runtimePassword = readEnvValue(env, 'DB_PASSWORD');
+ if (runtimePassword === null) {
+ missing.push('DB_PASSWORD');
+ return null;
+ }
+ return { dbPassword: ownerPassword, dbRuntimePassword: runtimePassword };
+ }
+ if (readEnvValue(env, 'DB_USER') === 'postgres') {
+ const legacyOwnerPassword = readEnvValue(env, 'DB_PASSWORD');
+ if (legacyOwnerPassword === null) {
+ missing.push('DB_PASSWORD');
+ return null;
+ }
+ return { dbPassword: legacyOwnerPassword, dbRuntimePassword: null };
+ }
+ missing.push('DB_OWNER_PASSWORD');
+ return null;
+}
+
+/**
+ * Keys whose values differ from `copy`, ignoring keys the file lacks. A
+ * desktop's copy may fill an incomplete `.env` only when it agrees: a copy from
+ * before someone else's reset would lock the stack out of its new database.
+ */
+export function keysDisagreeing(
+ env: string,
+ copy: { ports: LocalServerPorts; secrets: LocalServerSecrets }
+): string[] {
+ const expected = new Map<string, string>();
+ for (const [field, key] of PORT_KEYS) expected.set(key, String(copy.ports[field]));
+ for (const [field, key] of SECRET_KEYS) expected.set(key, copy.secrets[field]);
+ // Which password `DB_PASSWORD` is depends on the layout that wrote the file.
+ if (readEnvValue(env, 'DB_OWNER_PASSWORD') !== null) {
+ expected.set('DB_OWNER_PASSWORD', copy.secrets.dbPassword);
+ expected.set('DB_PASSWORD', copy.secrets.dbRuntimePassword);
+ } else if (readEnvValue(env, 'DB_USER') === 'postgres') {
+ expected.set('DB_PASSWORD', copy.secrets.dbPassword);
+ } else {
+ expected.set('DB_PASSWORD', copy.secrets.dbRuntimePassword);
+ }
+ const disagreeing: string[] = [];
+ for (const [key, value] of expected) {
+ const found = readEnvValue(env, key);
+ if (found !== null && found !== value) disagreeing.push(key);
+ }
+ return disagreeing;
+}
+
+/**
+ * The inverse of {@link buildEnvFile}. Accepts the current layout
+ * (`DB_OWNER_PASSWORD` the owner's, `DB_PASSWORD` the runtime role's) and the
+ * one from before the role split (`DB_USER=postgres`, `DB_PASSWORD` the owner's).
+ */
+export function readStackEnv(env: string): StackEnvReading {
+ const missing: string[] = [];
+
+ const ports: Partial<LocalServerPorts> = {};
+ for (const [field, key] of PORT_KEYS) {
+ const port = readPort(env, key);
+ if (port === null) missing.push(key);
+ else ports[field] = port;
+ }
+
+ const plain: Partial<Record<PlainSecret, string>> = {};
+ for (const [field, key] of SECRET_KEYS) {
+ const value = readEnvValue(env, key);
+ if (value === null) missing.push(key);
+ else plain[field] = value;
+ }
+
+ const database = readDatabasePasswords(env, missing);
+
+ if (missing.length > 0 || database === null) return { kind: 'incomplete', missing };
+ return {
+ kind: 'complete',
+ env: {
+ ports: ports as LocalServerPorts,
+ secrets: { ...(plain as Record<PlainSecret, string>), ...database },
+ version: readEnvValue(env, 'SWITCH_VERSION'),
+ },
+ };
+}
+
+/**
+ * How pydantic-settings reads a boolean, narrowed to the forms that can reach
+ * this variable.
+ *
+ * Absent means off, and that is a reading rather than a fallback: switch-core's
+ * own default for the gate is off, so a stack started before this existed — or
+ * by an operator who never set it — is genuinely not reporting.
+ */
+export function envFlag(value: string | null | undefined): boolean {
+ if (value === null || value === undefined) return false;
+ return ['1', 'true', 'yes', 'on'].includes(value.trim().toLowerCase());
+}
+
+/** Whether a stack's `.env` asks switch-core to share usage data. */
+export function telemetryRequested(envText: string): boolean {
+ return envFlag(readEnvValue(envText, TELEMETRY_ENABLED_KEY));
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/error-text.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/error-text.ts
new file mode 100644
index 000000000..e51c95d18
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/error-text.ts
@@ -0,0 +1,9 @@
+export function errorText(error: unknown): string {
+ return error instanceof Error ? error.message : String(error);
+}
+
+/** The command's stderr when it wrote any, else what was thrown. */
+export function commandFailure(error: unknown): string {
+ const stderr = (error as { stderr?: string } | undefined)?.stderr?.trim();
+ return stderr || errorText(error);
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.test.ts
new file mode 100644
index 000000000..62da60921
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.test.ts
@@ -0,0 +1,11 @@
+import { expect, it, vi } from 'vitest';
+
+vi.mock('../paths', () => ({ localServerDir: () => '/tmp/switch-local-server' }));
+vi.mock('../docker', () => ({ detectDocker: vi.fn(), dockerExecutable: () => 'docker' }));
+vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), info: vi.fn(), error: vi.fn() } }));
+
+const { LocalServerHost } = await import('./local-host');
+
+it('shares nothing with anyone, so a start here publishes nothing and takes no lock', () => {
+ expect(new LocalServerHost().sharedState).toBeNull();
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.ts
index 6bb96a871..9ff4f4576 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/local-host.ts
@@ -60,6 +60,7 @@ export class LocalServerHost implements ServerHost {
readonly stateDir = localServerDir();
readonly secretsKey = 'local-switch-server:secrets';
readonly label = 'this computer';
+ readonly sharedState = null;
readonly ctx: IExecutionContext = new LocalExecutionContext({ root: this.workingDir });
async writeFile(relPath: string, content: string, mode?: number): Promise<void> {
@@ -140,6 +141,10 @@ export class LocalServerHost implements ServerHost {
return pickFreePorts();
}
+ async checkNetworking(): Promise<void> {
+ // Nothing is forwarded, so nothing can be in the way here.
+ }
+
async establishNetworking(): Promise<void> {
// Local ports are already published on 127.0.0.1 — nothing to forward.
}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.errors.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.errors.test.ts
new file mode 100644
index 000000000..ddf967195
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.errors.test.ts
@@ -0,0 +1,38 @@
+import { EventEmitter } from 'node:events';
+import type * as Net from 'node:net';
+import { describe, expect, it, vi } from 'vitest';
+import type { SshClientProxy } from '@main/core/ssh/lifecycle/ssh-client-proxy';
+
+/**
+ * A real socket cannot portably fail to bind for a reason other than being taken
+ * (a low port is refused on Linux, allowed on macOS), so the listener is faked.
+ */
+
+class RefusingServer extends EventEmitter {
+ listen() {
+ const error = Object.assign(new Error('listen EACCES: permission denied 127.0.0.1:80'), {
+ code: 'EACCES',
+ });
+ queueMicrotask(() => this.emit('error', error));
+ return this;
+ }
+ close() {}
+}
+
+vi.mock('node:net', async (importOriginal) => ({
+ ...(await importOriginal<typeof Net>()),
+ createServer: () => new RefusingServer(),
+}));
+vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), info: vi.fn() } }));
+
+const { PortForwarder } = await import('./port-forward');
+
+describe('PortForwarder', () => {
+ it('passes on why a port could not be bound when it is not that it is taken', async () => {
+ const forwarder = new PortForwarder({} as SshClientProxy, 'vm-1');
+
+ await expect(forwarder.start([80])).rejects.toThrow(
+ 'cannot bind local port 80 for vm-1: listen EACCES: permission denied 127.0.0.1:80'
+ );
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.test.ts
new file mode 100644
index 000000000..88784eeaa
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.test.ts
@@ -0,0 +1,147 @@
+import { connect, createServer, type Server } from 'node:net';
+import { Duplex } from 'node:stream';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+import type { SshClientProxy } from '@main/core/ssh/lifecycle/ssh-client-proxy';
+
+const logWarn = vi.hoisted(() => vi.fn());
+vi.mock('@main/lib/logger', () => ({ log: { warn: logWarn, info: vi.fn() } }));
+
+const { PortForwarder } = await import('./port-forward');
+
+function occupy(): Promise<{ server: Server; port: number }> {
+ return new Promise((resolve, reject) => {
+ const server = createServer();
+ server.once('error', reject);
+ server.listen(0, '127.0.0.1', () => {
+ const address = server.address();
+ resolve({ server, port: typeof address === 'object' && address ? address.port : 0 });
+ });
+ });
+}
+
+const opened: Server[] = [];
+
+afterEach(async () => {
+ await Promise.all(opened.splice(0).map((s) => new Promise((r) => s.close(() => r(null)))));
+});
+
+describe('PortForwarder', () => {
+ it('says which port is taken here, and why it has to be that one', async () => {
+ const { server, port } = await occupy();
+ opened.push(server);
+ const forwarder = new PortForwarder({} as SshClientProxy, 'vm-1');
+
+ await expect(forwarder.start([port])).rejects.toThrow(
+ new RegExp(
+ `Port ${port} is already in use on this computer, so the Switch server on vm-1 ` +
+ `cannot be reached from here\\..*same port number`
+ )
+ );
+ });
+
+ it('releases the ports it did bind when a later one is taken', async () => {
+ const { server, port: taken } = await occupy();
+ opened.push(server);
+ const { server: probe, port: free } = await occupy();
+ await new Promise((r) => probe.close(() => r(null)));
+ const forwarder = new PortForwarder({} as SshClientProxy, 'vm-1');
+
+ await expect(forwarder.start([free, taken])).rejects.toThrow(/already in use/);
+
+ // The first listener was torn down with the failure, so its port is free.
+ const again = createServer();
+ await new Promise<void>((resolve, reject) => {
+ again.once('error', reject);
+ again.listen(free, '127.0.0.1', () => resolve());
+ });
+ opened.push(again);
+ });
+
+ it('carries a connection through to the same port on the host', async () => {
+ const { server: probe, port } = await occupy();
+ await new Promise((r) => probe.close(() => r(null)));
+ // The host's end of the channel: echoes what it is sent, upper-cased.
+ const channel = new Duplex({
+ read() {},
+ write(chunk: Buffer, _encoding, done) {
+ this.push(chunk.toString().toUpperCase());
+ done();
+ },
+ });
+ const forwardOut = vi.fn(async () => channel);
+ const forwarder = new PortForwarder({ forwardOut } as unknown as SshClientProxy, 'vm-1');
+ await forwarder.start([port]);
+
+ const reply = await new Promise<string>((resolve, reject) => {
+ const socket = connect(port, '127.0.0.1', () => socket.write('hello'));
+ socket.once('data', (data) => {
+ resolve(data.toString());
+ socket.destroy();
+ });
+ socket.once('error', reject);
+ });
+ forwarder.stop();
+
+ expect(forwardOut).toHaveBeenCalledWith(port);
+ expect(reply).toBe('HELLO');
+ });
+
+ it('closes the connection, and says why, when the host will not open a channel', async () => {
+ const { server: probe, port } = await occupy();
+ await new Promise((r) => probe.close(() => r(null)));
+ const forwardOut = vi.fn(async () => {
+ throw new Error('administratively prohibited');
+ });
+ const forwarder = new PortForwarder({ forwardOut } as unknown as SshClientProxy, 'vm-1');
+ await forwarder.start([port]);
+
+ await new Promise<void>((resolve) => {
+ const socket = connect(port, '127.0.0.1');
+ socket.on('error', () => {});
+ socket.once('close', () => resolve());
+ });
+ forwarder.stop();
+
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(new RegExp(`could not open channel to remote :${port} \\(vm-1\\)`)),
+ expect.objectContaining({ err: expect.any(Error) })
+ );
+ });
+
+ it('checks ports can be forwarded without keeping them, saying which is taken', async () => {
+ const { server, port: taken } = await occupy();
+ opened.push(server);
+ const { server: probe, port: free } = await occupy();
+ await new Promise((r) => probe.close(() => r(null)));
+
+ await expect(PortForwarder.check([free], 'vm-1')).resolves.toBeUndefined();
+ await expect(PortForwarder.check([free, taken], 'vm-1')).rejects.toThrow(
+ new RegExp(`Port ${taken} is already in use on this computer`)
+ );
+
+ // Checked, not kept: the free port can still be listened on.
+ const again = createServer();
+ await new Promise<void>((resolve, reject) => {
+ again.once('error', reject);
+ again.listen(free, '127.0.0.1', () => resolve());
+ });
+ opened.push(again);
+ });
+
+ it('stops listening when stopped, and can be stopped twice', async () => {
+ const { server: probe, port } = await occupy();
+ await new Promise((r) => probe.close(() => r(null)));
+ const forwarder = new PortForwarder({} as SshClientProxy, 'vm-1');
+ await forwarder.start([port]);
+
+ forwarder.stop();
+ forwarder.stop();
+
+ const again = createServer();
+ await new Promise<void>((resolve, reject) => {
+ again.once('error', reject);
+ again.listen(port, '127.0.0.1', () => resolve());
+ });
+ opened.push(again);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.ts
index 28efc9520..e5a424097 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/port-forward.ts
@@ -2,6 +2,30 @@ import { createServer, type Server, type Socket } from 'node:net';
import type { SshClientProxy } from '@main/core/ssh/lifecycle/ssh-client-proxy';
import { log } from '@main/lib/logger';
+/** Listen on `127.0.0.1:<port>`, saying why not when that fails. */
+function bind(server: Server, port: number, label: string): Promise<void> {
+ return new Promise((resolve, reject) => {
+ const onError = (err: NodeJS.ErrnoException) =>
+ reject(
+ new Error(
+ err.code === 'EADDRINUSE'
+ ? // Likely on a shared host: the stack was set up from another
+ // computer, whose free ports this one's need not be.
+ `Port ${port} is already in use on this computer, so the Switch server on ` +
+ `${label} cannot be reached from here. Switch Console reaches a remote ` +
+ `server through the same port number on this computer as on the host, so ` +
+ `whatever is using port ${port} here has to stop first.`
+ : `cannot bind local port ${port} for ${label}: ${err.message}`
+ )
+ );
+ server.once('error', onError);
+ server.listen(port, '127.0.0.1', () => {
+ server.removeListener('error', onError);
+ resolve();
+ });
+ });
+}
+
/**
* A persistent local→remote loopback bridge over the SSH connection.
*
@@ -41,22 +65,27 @@ export class PortForwarder {
}
}
- private listen(port: number): Promise<void> {
+ /**
+ * Throw what {@link start} would if any of `ports` cannot be bound here, without
+ * keeping them. Asked before a shared-host start changes anything, so a taken
+ * port is found before the stack is restarted for everyone.
+ */
+ static async check(ports: number[], label: string): Promise<void> {
+ for (const port of ports) {
+ const server = createServer();
+ await bind(server, port, label);
+ await new Promise((resolve) => server.close(() => resolve(null)));
+ }
+ }
+
+ private async listen(port: number): Promise<void> {
const server = createServer((socket) => this.bridge(socket, port));
this.servers.push(server);
- return new Promise((resolve, reject) => {
- const onError = (err: Error) =>
- reject(new Error(`cannot bind local port ${port} for ${this.label}: ${err.message}`));
- server.once('error', onError);
- server.listen(port, '127.0.0.1', () => {
- server.removeListener('error', onError);
- // Later listener errors are non-fatal — log rather than crash.
- server.on('error', (err) =>
- log.warn(`port-forward: listener error on :${port} (${this.label})`, { err })
- );
- resolve();
- });
- });
+ await bind(server, port, this.label);
+ // Later listener errors are non-fatal — log rather than crash.
+ server.on('error', (err) =>
+ log.warn(`port-forward: listener error on :${port} (${this.label})`, { err })
+ );
}
private bridge(socket: Socket, port: number): void {
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.test.ts
new file mode 100644
index 000000000..2d7a85572
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.test.ts
@@ -0,0 +1,138 @@
+import { EventEmitter } from 'node:events';
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+import type * as SshExecution from '@main/core/execution-context/ssh-execution-context';
+
+vi.mock('@main/core/execution-context/ssh-execution-context', async (importOriginal) => ({
+ exitDescription: (await importOriginal<typeof SshExecution>()).exitDescription,
+ buildSshCommand: (dir: string, command: string, args: string[]) =>
+ `cd ${dir} && ${command} ${args.join(' ')}`,
+ SshExecutionContext: class {},
+}));
+vi.mock('@main/core/fs/impl/ssh-fs', () => ({ SshFileSystem: class {} }));
+vi.mock('@main/core/ssh/connect/connect-agent-ssh', () => ({ ensureSshConnected: vi.fn() }));
+vi.mock('@main/lib/logger', () => ({ log: { info: vi.fn(), warn: vi.fn() } }));
+vi.mock('../paths', () => ({ remoteServerStateDir: (slug: string) => `/state/${slug}` }));
+vi.mock('./remote-free-port', () => ({ pickRemoteFreePorts: vi.fn() }));
+vi.mock('@main/core/locations/location-transport', () => ({
+ sshConnectionIdForHost: (host: string) => `ssh:${host}`,
+}));
+
+const { RemoteServerHost } = await import('./remote-host');
+
+/** An SSH channel: stdout and stderr, a stdin that records what is written. */
+class FakeStream extends EventEmitter {
+ readonly stderr = new EventEmitter();
+ written: string | null = null;
+ destroyed = false;
+ end(input: string) {
+ this.written = input;
+ }
+ destroy() {
+ this.destroyed = true;
+ }
+}
+
+let stream: FakeStream;
+const commands: string[] = [];
+
+function host() {
+ const proxy = {
+ getRemoteShellProfile: async () => ({}),
+ exec: (command: string, cb: (error: Error | null, stream: FakeStream) => void) => {
+ commands.push(command);
+ cb(null, stream);
+ },
+ };
+ return new RemoteServerHost({
+ sshHost: 'vm-1',
+ proxy: proxy as never,
+ ctx: {} as never,
+ workingDir: '/home/bob/stack',
+ });
+}
+
+/** Let the profile lookup resolve and the exec callback run. */
+const started = () => new Promise((resolve) => setTimeout(resolve, 0));
+
+beforeEach(() => {
+ stream = new FakeStream();
+ commands.length = 0;
+});
+
+describe('streamCommand', () => {
+ it('hands on every line of stdout and stderr, and resolves on a clean exit', async () => {
+ const lines: string[] = [];
+ const done = host().streamCommand('docker', ['compose', 'up'], (line) => lines.push(line));
+ await started();
+
+ stream.emit('data', Buffer.from('pulling\r\n\nstarting\n'));
+ stream.stderr.emit('data', Buffer.from('warning: slow\n'));
+ stream.emit('close', 0);
+
+ await expect(done).resolves.toBeUndefined();
+ expect(lines).toEqual(['pulling', 'starting', 'warning: slow']);
+ expect(commands).toEqual(['cd /home/bob/stack && docker compose up']);
+ expect(stream.written).toBeNull();
+ });
+
+ it('rejects with the tail of stderr on a failing exit', async () => {
+ const done = host().streamCommand('docker', ['compose', 'up'], () => {});
+ await started();
+
+ stream.stderr.emit('data', Buffer.from('port is already allocated\n'));
+ stream.emit('close', 1);
+
+ await expect(done).rejects.toThrow('docker failed (exit 1): port is already allocated');
+ });
+
+ it('rejects a channel that closed without an exit status, as a dropped connection does', async () => {
+ const done = host().streamCommand('docker', ['ps'], () => {});
+ await started();
+
+ stream.emit('close');
+
+ await expect(done).rejects.toThrow(
+ 'docker failed: The command ended without an exit status; the SSH connection may have dropped.'
+ );
+ });
+
+ it('rejects a command killed by a signal, saying which', async () => {
+ const done = host().streamCommand('docker', ['ps'], () => {});
+ await started();
+
+ stream.emit('close', null, 'SIGKILL');
+
+ await expect(done).rejects.toThrow('docker failed: The command was killed by SIGKILL.');
+ });
+});
+
+describe('writeCommandInput', () => {
+ it('writes the input to stdin, and discards the output', async () => {
+ const done = host().writeCommandInput('docker', ['run', '-i'], 'SECRET=1\n', {
+ timeoutMs: 1000,
+ });
+ await started();
+
+ stream.emit('data', Buffer.from('motd from the login shell\n'));
+ stream.emit('close', 0);
+
+ await expect(done).resolves.toBeUndefined();
+ expect(stream.written).toBe('SECRET=1\n');
+ expect(commands[0]).not.toContain('SECRET');
+ });
+
+ it('gives up, and closes the channel, when the command outlives its timeout', async () => {
+ vi.useFakeTimers();
+ try {
+ const done = host().writeCommandInput('docker', ['run', '-i'], 'x', { timeoutMs: 50 });
+ const refused = expect(done).rejects.toThrow('docker timed out after 50ms');
+ await vi.advanceTimersByTimeAsync(0);
+ await vi.advanceTimersByTimeAsync(60);
+
+ await refused;
+ expect(stream.destroyed).toBe(true);
+ } finally {
+ vi.useRealTimers();
+ }
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.ts
index 75e0da953..71f1e3827 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/remote-host.ts
@@ -1,5 +1,6 @@
import {
buildSshCommand,
+ exitDescription,
SshExecutionContext,
} from '@main/core/execution-context/ssh-execution-context';
import type { IExecutionContext } from '@main/core/execution-context/types';
@@ -13,6 +14,7 @@ import type { DockerAvailability } from '@shared/core/managed-switch-server/mana
import { REMOTE_SERVER_PROJECT_NAME } from '../constants';
import type { LocalServerPorts } from '../free-port';
import { remoteServerStateDir } from '../paths';
+import type { StackStateHost } from '../stack-state';
import { PortForwarder } from './port-forward';
import { pickRemoteFreePorts } from './remote-free-port';
import { hostSlug, remoteSecretsKey } from './remote-identity';
@@ -48,6 +50,7 @@ export class RemoteServerHost implements ServerHost {
readonly secretsKey: string;
readonly label: string;
readonly ctx: IExecutionContext;
+ readonly sharedState: StackStateHost = this;
private readonly sshHost: string;
private readonly proxy: SshClientProxy;
@@ -96,6 +99,33 @@ export class RemoteServerHost implements ServerHost {
args: string[],
onLine: (line: string) => void,
opts: { timeoutMs?: number } = {}
+ ): Promise<void> {
+ return this.runOverSsh(command, args, { onLine, timeoutMs: opts.timeoutMs });
+ }
+
+ /**
+ * Run a command on the host with `input` as its stdin, for handing it a secret:
+ * stdin never shows in the host's process table, where arguments are readable
+ * by every account. Output is discarded. Rejects on non-zero exit or timeout.
+ */
+ writeCommandInput(
+ command: string,
+ args: string[],
+ input: string,
+ opts: { timeoutMs: number }
+ ): Promise<void> {
+ return this.runOverSsh(command, args, { input, timeoutMs: opts.timeoutMs });
+ }
+
+ /**
+ * Run `command` in the host's login shell under {@link workingDir}, rejecting
+ * with the stderr tail on a non-zero exit or on timeout. `onLine` gets each
+ * non-empty output line; `input`, if given, is written to stdin and closed.
+ */
+ private runOverSsh(
+ command: string,
+ args: string[],
+ opts: { onLine?: (line: string) => void; input?: string; timeoutMs?: number }
): Promise<void> {
return new Promise((resolve, reject) => {
this.proxy
@@ -110,9 +140,10 @@ export class RemoteServerHost implements ServerHost {
let stderrTail = '';
let settled = false;
const emit = (buf: Buffer) => {
+ if (!opts.onLine) return;
for (const raw of buf.toString('utf8').split('\n')) {
const line = raw.replace(/\r$/, '');
- if (line.length > 0) onLine(line);
+ if (line.length > 0) opts.onLine(line);
}
};
const settle = (fn: () => void) => {
@@ -135,13 +166,21 @@ export class RemoteServerHost implements ServerHost {
stderrTail = (stderrTail + buf.toString('utf8')).slice(-4000);
emit(buf);
});
- stream.on('close', (code: number | null) => {
+ // Only an exit status of 0 is a success: see SshExecutionContext.exec.
+ stream.on('close', (code: number | null | undefined, signal?: string) => {
settle(() => {
- if ((code ?? 0) === 0) resolve();
- else reject(new Error(`${command} failed (exit ${code}): ${stderrTail.trim()}`));
+ const tail = stderrTail.trim();
+ if (code === 0) resolve();
+ else if (typeof code === 'number') {
+ reject(new Error(`${command} failed (exit ${code}): ${tail}`));
+ } else {
+ const why = exitDescription(code, signal);
+ reject(new Error(`${command} failed: ${why}${tail ? ` (${tail})` : ''}`));
+ }
});
});
stream.on('error', (err: Error) => settle(() => reject(err)));
+ if (opts.input !== undefined) stream.end(opts.input);
});
})
.catch(reject);
@@ -192,6 +231,13 @@ export class RemoteServerHost implements ServerHost {
return pickRemoteFreePorts(this.ctx);
}
+ async checkNetworking(ports: LocalServerPorts): Promise<void> {
+ await PortForwarder.check(
+ FORWARDED_SERVICES.map((s) => ports[s]),
+ this.label
+ );
+ }
+
async establishNetworking(ports: LocalServerPorts): Promise<void> {
this.forwarder?.stop();
const forwarder = new PortForwarder(this.proxy, this.label);
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/types.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/types.ts
index 2930ec0e4..9e26d2d40 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/types.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/host/types.ts
@@ -1,6 +1,7 @@
import type { IExecutionContext } from '@main/core/execution-context/types';
import type { DockerAvailability } from '@shared/core/managed-switch-server/managed-switch-server';
import type { LocalServerPorts } from '../free-port';
+import type { StackStateHost } from '../stack-state';
/**
* A host that runs a Switch Console-managed Switch stack via `docker compose`.
@@ -52,6 +53,10 @@ export interface ServerHost {
* the SSH alias). */
readonly label: string;
+ /** Handle on the stack state every Console sharing this host reads (see
+ * `stack-state.ts`); null for the local stack, which is this desktop's alone. */
+ readonly sharedState: StackStateHost | null;
+
/** Write `content` to `relPath` under {@link workingDir}, creating parent
* directories. `mode` (e.g. 0o600 for the secret-bearing `.env`) is enforced
* when provided. */
@@ -96,6 +101,11 @@ export interface ServerHost {
* over the SSH connection, kept alive across reconnects. */
establishNetworking(ports: LocalServerPorts): Promise<void>;
+ /** Throw, saying why, when {@link establishNetworking} could not make `ports`
+ * reachable (e.g. a port taken on the desktop). Asked before a start changes
+ * anything on the host. */
+ checkNetworking(ports: LocalServerPorts): Promise<void>;
+
/** Tear down any networking started by {@link establishNetworking}. Called on
* stop/reset and disposal. Idempotent. */
teardownNetworking(): Promise<void>;
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
index 4143140ef..741b58cbb 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.test.ts
@@ -40,7 +40,8 @@ vi.mock('./managed-upgrade', async (importOriginal) => ({
...(await importOriginal<typeof ManagedUpgrade>()),
readUpgradeJournal: m.journal,
}));
-vi.mock('./pipeline', () => ({ startStack: m.start, stopStack: m.stop, resetStack: vi.fn() }));
+const resetStack = vi.hoisted(() => vi.fn(async () => {}));
+vi.mock('./pipeline', () => ({ startStack: m.start, stopStack: m.stop, resetStack }));
vi.mock('./host/local-host', () => ({
LocalServerHost: class {
dispose() {}
@@ -63,6 +64,7 @@ const started: StartLocalServerResult = {
kind: 'started',
serverId: 'local',
telemetryEnabled: false,
+ warning: null,
};
/** A start that runs the upgrade the way the pipeline does: announce, then work. */
@@ -255,3 +257,11 @@ it('keeps an owed upgrade pending across a stop', async () => {
upgrade: { state: 'pending', from: '0.10.0', to: '0.11.0' },
});
});
+
+it('resets the stack on this computer holding no lock, since nobody else shares it', async () => {
+ const service = new LocalServerService();
+
+ await service.reset();
+
+ expect(resetStack).toHaveBeenCalledWith(expect.anything(), null);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
index 7c683e056..202050e02 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/local-server-service.ts
@@ -257,6 +257,8 @@ export class LocalServerService {
onUpgrade: (owed) => this.setStatus({ upgrade: upgradeState(owed, true) }),
signal: this.startAbort.signal,
checkoutRoot,
+ // Nobody else shares the local stack, so there is nothing to lock.
+ lease: null,
});
if (result.kind === 'docker-unavailable') {
this.setStatus({ phase: 'error', error: result.detail });
@@ -325,7 +327,7 @@ export class LocalServerService {
const host: ServerHost = new LocalServerHost();
try {
this.setStatus({ phase: 'stopping', message: 'Stopping containers…' });
- await stopStack(host);
+ await stopStack(host, null);
const upgrade = this.status.upgrade;
this.setStatus({
phase: 'stopped',
@@ -364,7 +366,7 @@ export class LocalServerService {
const server = await getManagedServer();
if (server) await deleteAgentsForServer(server.id);
this.setStatus({ phase: 'stopping', message: 'Destroying containers and data…' });
- await resetStack(host);
+ await resetStack(host, null);
this.setStatus({
phase: 'stopped',
message: null,
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.test.ts
index ae34da0f3..31b4dbca2 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.test.ts
@@ -2,6 +2,7 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
const getLocalStatus = vi.hoisted(() => vi.fn());
const getRemoteStatus = vi.hoisted(() => vi.fn());
+const recheck = vi.hoisted(() => vi.fn());
const isHostBlockedMock = vi.hoisted(() => vi.fn(() => false));
const getReachability = vi.hoisted(() => vi.fn());
@@ -9,14 +10,18 @@ vi.mock('./local-server-service', () => ({
localServerService: { getStatus: getLocalStatus },
}));
vi.mock('./remote-server-service', () => ({
- remoteServerService: { getStatus: getRemoteStatus },
+ remoteServerService: { getStatus: getRemoteStatus, recheck },
}));
vi.mock('@main/core/remote-hosts/production-host-reachability', () => ({
hostReachabilityService: { isBlocked: isHostBlockedMock, get: getReachability },
}));
-const { isManagedServerRunning, managedServerHostBlocked, managedServerStoppedPhase } =
- await import('./managed-server-status');
+const {
+ isManagedServerRunning,
+ managedServerHostBlocked,
+ managedServerStoppedPhase,
+ noteManagedServerUnanswered,
+} = await import('./managed-server-status');
function reachability(overrides: Record<string, unknown>) {
return {
@@ -203,3 +208,27 @@ describe('managedServerStoppedPhase', () => {
expect(getRemoteStatus).not.toHaveBeenCalled();
});
});
+
+describe('noteManagedServerUnanswered', () => {
+ beforeEach(() => {
+ vi.clearAllMocks();
+ });
+
+ it('asks the host again about a remote stack that stopped answering', () => {
+ noteManagedServerUnanswered(server({ managementKind: 'remote', sshHost: 'host-a' }));
+
+ expect(recheck).toHaveBeenCalledExactlyOnceWith('host-a');
+ });
+
+ it('leaves the local stack alone: nobody else can change it', () => {
+ noteManagedServerUnanswered(server({ managementKind: 'local' }));
+
+ expect(recheck).not.toHaveBeenCalled();
+ });
+
+ it('leaves a server someone else runs alone', () => {
+ noteManagedServerUnanswered(server({ managed: false, managementKind: null }));
+
+ expect(recheck).not.toHaveBeenCalled();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.ts
index f724f08d4..ea561a19e 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/managed-server-status.ts
@@ -69,3 +69,13 @@ export function managedServerStoppedPhase(server: SwitchServer): LocalServerPhas
}
return phase === 'stopped' || phase === 'error' ? phase : null;
}
+
+/**
+ * A call to a managed server got no answer. A remote stack is shared, so the
+ * likeliest cause is another Console stopping or resetting it: re-read the host
+ * (throttled) so that shows as a status rather than a string of failed calls.
+ */
+export function noteManagedServerUnanswered(server: SwitchServer): void {
+ if (!server.managed || server.managementKind !== 'remote' || server.sshHost === null) return;
+ remoteServerService.recheck(server.sshHost);
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
index 296037cbf..49a344f8a 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-matrix-migration.test.ts
@@ -17,6 +17,9 @@ const dockerRunOneOffMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
const prepareUpgradeMock = vi.hoisted(() => vi.fn(() => Promise.resolve(null)));
const finishUpgradeMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+vi.mock('./console-register', () => ({
+ readRegister: vi.fn(async () => ({ self: 'me', consoles: [], activity: [] })),
+}));
vi.mock('@shared/app-identity', async (importOriginal) => ({
...(await importOriginal<typeof AppIdentity>()),
COMPATIBLE_SWITCH_VERSION: '0.24.0',
@@ -57,6 +60,7 @@ vi.mock('./telemetry-consent', () => ({
readDeployedTelemetry: vi.fn(),
}));
vi.mock('@main/core/switch-servers/servers-store', () => ({
+ assertManagedServerUrlFree: () => Promise.resolve(),
ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),
setActiveServerId: vi.fn(),
}));
@@ -77,8 +81,10 @@ function options(checkoutRoot: string | null = null) {
);
const host = {
label: 'this computer',
+ sharedState: null,
writeFile,
detectDocker: () => Promise.resolve({ available: true, version: '27.0.0' }),
+ checkNetworking: vi.fn(() => Promise.resolve()),
establishNetworking: vi.fn(() => Promise.resolve()),
};
return {
@@ -93,6 +99,7 @@ function options(checkoutRoot: string | null = null) {
onUpgrade: vi.fn(),
signal: new AbortController().signal,
checkoutRoot,
+ lease: null,
},
};
}
@@ -127,6 +134,7 @@ describe('startStack across the Matrix boundary', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
// The old stack comes up and is drained before the new compose file lands:
// that file is what removes the homeserver being read from.
@@ -183,6 +191,7 @@ describe('startStack across the Matrix boundary', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
expect(dockerRunOneOffMock).not.toHaveBeenCalled();
});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
new file mode 100644
index 000000000..efb8db229
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline-shared-host.test.ts
@@ -0,0 +1,1095 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+import type * as AppIdentity from '@shared/app-identity';
+import type * as DeployedVersion from './deployed-version';
+import type * as EnvFile from './env-file';
+import type { StackEnv } from './env-file';
+import type { ServerHost } from './host/types';
+import type { LocalServerSecrets } from './secret-values';
+import type { ServerLease } from './stack-lock';
+import type * as StackState from './stack-state';
+import type { StackOnHost, StackStateHost } from './stack-state';
+
+/**
+ * Start and connect on a host whose stack other Consoles share (CHOO-2893).
+ * The rule under test: the host's settings win over this desktop's copy, and
+ * new credentials are made only when the host has nothing of the stack.
+ */
+
+const inspectStackMock = vi.hoisted(() => vi.fn<() => Promise<StackOnHost>>());
+const publishEnvMock = vi.hoisted(() => vi.fn((): Promise<string | null> => Promise.resolve(null)));
+const withdrawPublishedEnvMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const stampPublishedEnvMock = vi.hoisted(() =>
+ vi.fn((): Promise<string | null> => Promise.resolve('2026-09-01T10:00:00Z'))
+);
+const composeUpMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const composeDownMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const waitForHealthMock = vi.hoisted(() => vi.fn(() => Promise.resolve(true)));
+const readDeployedVersionMock = vi.hoisted(() =>
+ vi.fn(
+ (): Promise<{ kind: string; version?: string; source?: string }> =>
+ Promise.resolve({ kind: 'deployed', version: '0.11.0', source: 'env-file' })
+ )
+);
+const buildEnvFileMock = vi.hoisted(() => vi.fn((_params: unknown) => 'BUILT_ENV\n'));
+const loadOrCreateSecretsMock = vi.hoisted(() => vi.fn());
+const readSecretsMock = vi.hoisted(() => vi.fn());
+const storeSecretsMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const clearSecretsMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const resolvePortsMock = vi.hoisted(() => vi.fn());
+const readPersistedPortsMock = vi.hoisted(() => vi.fn());
+const rememberPortsMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const ensureManagedServerMock = vi.hoisted(() => vi.fn(() => Promise.resolve({ id: 'srv-1' })));
+const passwordLoginMock = vi.hoisted(() => vi.fn(() => Promise.resolve({ success: true })));
+const logError = vi.hoisted(() => vi.fn());
+const logWarn = vi.hoisted(() => vi.fn());
+
+const readRegisterMock = vi.hoisted(() => vi.fn());
+vi.mock('./console-register', () => ({ readRegister: readRegisterMock }));
+vi.mock('@shared/app-identity', async (importOriginal) => ({
+ ...(await importOriginal<typeof AppIdentity>()),
+ COMPATIBLE_SWITCH_VERSION: '0.11.0',
+}));
+vi.mock('@main/lib/logger', () => ({ log: { error: logError, warn: logWarn, info: vi.fn() } }));
+vi.mock('./stack-state', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackState>()),
+ inspectStack: inspectStackMock,
+ publishEnv: publishEnvMock,
+ stampPublishedEnv: stampPublishedEnvMock,
+ withdrawPublishedEnv: withdrawPublishedEnvMock,
+ unsharedStackMessage: (host: string, dir: string | null) => `unshared ${host} ${dir}`,
+}));
+vi.mock('./deployed-version', async (importOriginal) => ({
+ ...(await importOriginal<typeof DeployedVersion>()),
+ readDeployedVersion: readDeployedVersionMock,
+}));
+vi.mock('./compose', () => ({ composeUp: composeUpMock, composeDown: composeDownMock }));
+vi.mock('./health', () => ({ waitForHealth: waitForHealthMock }));
+const prepareUpgradeMock = vi.hoisted(() => vi.fn(async () => null));
+vi.mock('./managed-upgrade', () => ({
+ prepareUpgrade: prepareUpgradeMock,
+ finishUpgrade: vi.fn(),
+}));
+vi.mock('./bundled-compose', () => ({ bundledComposeYaml: () => 'services: {}' }));
+vi.mock('./env-file', async (importOriginal) => ({
+ ...(await importOriginal<typeof EnvFile>()),
+ buildEnvFile: buildEnvFileMock,
+}));
+vi.mock('./secrets', () => ({
+ loadOrCreateSecrets: loadOrCreateSecretsMock,
+ readSecrets: readSecretsMock,
+ storeSecrets: storeSecretsMock,
+ clearSecrets: clearSecretsMock,
+}));
+vi.mock('./ports', () => ({
+ resolvePorts: resolvePortsMock,
+ readPersistedPorts: readPersistedPortsMock,
+ rememberPorts: rememberPortsMock,
+ clearPorts: vi.fn(() => Promise.resolve()),
+}));
+const telemetryConsentMock = vi.hoisted(() => vi.fn(() => Promise.resolve(false)));
+vi.mock('./telemetry-consent', () => ({ telemetryConsent: telemetryConsentMock }));
+const assertManagedServerUrlFreeMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+vi.mock('@main/core/switch-servers/servers-store', () => ({
+ assertManagedServerUrlFree: assertManagedServerUrlFreeMock,
+ ensureManagedServer: ensureManagedServerMock,
+ setActiveServerId: vi.fn(() => Promise.resolve()),
+}));
+vi.mock('@main/core/switch-servers/auth', () => ({ passwordLogin: passwordLoginMock }));
+vi.mock('@main/core/agents/resolve-servers', () => ({ resolveAgentServers: vi.fn() }));
+vi.mock('./matrix-migration', () => ({
+ crossesMatrixBoundary: () => false,
+ runBackfill: vi.fn(),
+}));
+
+const { adoptRunningStack, connectStack, resetStack, startStack, stopStack } =
+ await import('./pipeline');
+
+const hostSecrets: LocalServerSecrets = {
+ dbPassword: 'host-owner-pw',
+ dbRuntimePassword: 'host-runtime-pw',
+ agentRegistrationToken: 'host-agent-token',
+ jwtSecretKey: 'host-jwt',
+ gatewayAdminPassword: 'host-admin-pw',
+ mattermostAdminPassword: 'host-mm-admin',
+ mattermostUserPassword: 'host-mm-user',
+};
+const hostPorts = { gateway: 41000, api: 41001, mattermost: 41002, postgres: 41003 };
+const cachedSecrets: LocalServerSecrets = { ...hostSecrets, gatewayAdminPassword: 'cached-pw' };
+const cachedPorts = { gateway: 3300, api: 8000, mattermost: 8065, postgres: 5432 };
+
+function present(
+ overrides: Partial<Extract<StackOnHost, { kind: 'present' }>> = {},
+ env: Partial<StackEnv> = {}
+): StackOnHost {
+ return {
+ kind: 'present',
+ env: { ports: hostPorts, secrets: hostSecrets, version: '0.11.0', ...env },
+ raw: 'PUBLISHED_ENV\n',
+ source: 'published',
+ running: true,
+ published: true,
+ runningVersion: null,
+ stamp: null,
+ ...overrides,
+ };
+}
+
+function sharedHost() {
+ const writeFile = vi.fn<(relPath: string, content: string, mode?: number) => Promise<void>>(() =>
+ Promise.resolve()
+ );
+ const checkNetworking = vi.fn(() => Promise.resolve());
+ const establishNetworking = vi.fn(() => Promise.resolve());
+ const teardownNetworking = vi.fn(() => Promise.resolve());
+ const readFile = vi.fn<(relPath: string) => Promise<string | null>>(() => Promise.resolve(null));
+ const removeFile = vi.fn<(relPath: string) => Promise<void>>(() => Promise.resolve());
+ const sharedState = { label: 'vm-1' } as unknown as StackStateHost;
+ const host = {
+ label: 'vm-1',
+ sharedState,
+ writeFile,
+ readFile,
+ removeFile,
+ checkNetworking,
+ establishNetworking,
+ teardownNetworking,
+ detectDocker: () => Promise.resolve({ available: true, version: '27.0.0' }),
+ } as unknown as ServerHost;
+ return {
+ host,
+ sharedState,
+ writeFile,
+ readFile,
+ removeFile,
+ checkNetworking,
+ establishNetworking,
+ teardownNetworking,
+ };
+}
+
+function heldLease() {
+ const assertHeld = vi.fn(() => Promise.resolve());
+ const release = vi.fn(() => Promise.resolve());
+ return {
+ lease: { token: 'lease-token', lost: false, assertHeld, release } as unknown as ServerLease,
+ assertHeld,
+ release,
+ };
+}
+
+let held = heldLease();
+
+function startOptions(host: ServerHost) {
+ return {
+ host,
+ ref: { kind: 'remote' as const, sshHost: 'vm-1' },
+ serverName: 'Team server',
+ activate: true,
+ onMessage: vi.fn(),
+ onLog: vi.fn(),
+ onUpgrade: vi.fn(),
+ signal: new AbortController().signal,
+ checkoutRoot: null,
+ lease: held.lease,
+ };
+}
+
+function connectOptions(host: ServerHost) {
+ return {
+ host,
+ ref: { kind: 'remote' as const, sshHost: 'vm-1' },
+ serverName: 'Team server',
+ onMessage: vi.fn(),
+ signal: new AbortController().signal,
+ lease: held.lease,
+ };
+}
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ held = heldLease();
+ telemetryConsentMock.mockResolvedValue(false);
+ readRegisterMock.mockResolvedValue({ self: 'me', consoles: [], activity: [] });
+ publishEnvMock.mockResolvedValue(null);
+ stampPublishedEnvMock.mockResolvedValue('2026-09-01T10:00:00Z');
+ waitForHealthMock.mockResolvedValue(true);
+ loadOrCreateSecretsMock.mockResolvedValue(cachedSecrets);
+ resolvePortsMock.mockResolvedValue(cachedPorts);
+ readSecretsMock.mockResolvedValue(null);
+ readPersistedPortsMock.mockResolvedValue(null);
+});
+
+describe('starting a shared stack', () => {
+ it('gives an account that never joined the stack a compose file before the upgrade backup reads it', async () => {
+ inspectStackMock.mockResolvedValue(present({}, { version: '0.10.0' }));
+ const { host, writeFile } = sharedHost();
+
+ expect(await startStack(startOptions(host))).toMatchObject({ kind: 'started' });
+
+ const composeWrite = writeFile.mock.calls.findIndex(
+ ([path]) => path === 'standalone-docker-compose.yml'
+ );
+ expect(composeWrite).toBeGreaterThanOrEqual(0);
+ expect(writeFile.mock.invocationCallOrder[composeWrite]).toBeLessThan(
+ prepareUpgradeMock.mock.invocationCallOrder[0]!
+ );
+ });
+
+ it('leaves an older stack’s compose file for the start to rewrite after its backup', async () => {
+ inspectStackMock.mockResolvedValue(present({}, { version: '0.10.0' }));
+ const { host, writeFile, readFile } = sharedHost();
+ readFile.mockImplementation(async (path) =>
+ path === 'standalone-docker-compose.yml' ? 'services: { older: {} }' : null
+ );
+
+ await startStack(startOptions(host));
+
+ const composeWrites = writeFile.mock.calls.filter(
+ ([path]) => path === 'standalone-docker-compose.yml'
+ );
+ // Once, by the start itself, after the backup.
+ expect(composeWrites).toHaveLength(1);
+ });
+
+ it('replaces an account’s older compose file for a stack at this build’s version', async () => {
+ inspectStackMock.mockResolvedValue(present({ running: true }));
+ const { host, writeFile, readFile } = sharedHost();
+ readFile.mockImplementation(async (path) =>
+ path === 'standalone-docker-compose.yml' ? 'services: { older: {} }' : null
+ );
+
+ await adoptRunningStack(
+ host,
+ present() as Extract<StackOnHost, { kind: 'present' }>,
+ held.lease
+ );
+
+ expect(writeFile).toHaveBeenCalledWith('standalone-docker-compose.yml', 'services: {}');
+ });
+
+ it('runs the host’s stack with the host’s settings, not this desktop’s', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host } = sharedHost();
+
+ expect(await startStack(startOptions(host))).toMatchObject({ kind: 'started' });
+
+ expect(buildEnvFileMock).toHaveBeenCalledWith(
+ expect.objectContaining({ secrets: hostSecrets, ports: hostPorts })
+ );
+ expect(loadOrCreateSecretsMock).not.toHaveBeenCalled();
+ expect(resolvePortsMock).not.toHaveBeenCalled();
+ // The desktop's copy is refreshed from the host.
+ expect(storeSecretsMock).toHaveBeenCalledWith(host, hostSecrets);
+ expect(rememberPortsMock).toHaveBeenCalledWith(host, hostPorts);
+ expect(ensureManagedServerMock).toHaveBeenCalledWith(
+ {
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ },
+ { kind: 'remote', sshHost: 'vm-1' }
+ );
+ expect(passwordLoginMock).toHaveBeenCalledWith(
+ expect.anything(),
+ 'host-admin-pw'
+ );
+ });
+
+ it('brings this account’s working dir in step with the stack before checking its version', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, writeFile } = sharedHost();
+ const order: string[] = [];
+ writeFile.mockImplementation(async (name, content) => {
+ order.push(`write ${name} ${content.trim()}`);
+ });
+ readDeployedVersionMock.mockImplementation(async () => {
+ order.push('version check');
+ return { kind: 'deployed', version: '0.11.0', source: 'env-file' };
+ });
+
+ await startStack(startOptions(host));
+
+ expect(order.indexOf('write .env PUBLISHED_ENV')).toBe(0);
+ expect(order.indexOf('version check')).toBeGreaterThan(0);
+ });
+
+ it('leaves the working dir alone when that is where the settings were read from', async () => {
+ inspectStackMock.mockResolvedValue(present({ source: 'working-dir', published: false }));
+ const { host, writeFile } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(writeFile.mock.calls.filter(([, content]) => content === 'PUBLISHED_ENV\n')).toEqual([]);
+ });
+
+ it('publishes the .env it wrote before compose reads it, and stamps it after', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, sharedState } = sharedHost();
+ const order: string[] = [];
+ publishEnvMock.mockImplementation(async () => {
+ order.push('publish');
+ // A first start: no database volume to stamp the copy with yet.
+ return null;
+ });
+ composeUpMock.mockImplementation(async () => {
+ order.push('compose up');
+ });
+ stampPublishedEnvMock.mockImplementation(async () => {
+ order.push('stamp');
+ return '2026-09-01T10:00:00Z';
+ });
+
+ await startStack(startOptions(host));
+
+ expect(publishEnvMock).toHaveBeenCalledWith(sharedState, 'BUILT_ENV\n', held.lease);
+ expect(stampPublishedEnvMock).toHaveBeenCalledWith(sharedState, held.lease);
+ expect(order).toEqual(['publish', 'compose up', 'stamp']);
+ });
+
+ it('fails the start, touching no container, when the settings cannot be shared', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ publishEnvMock.mockRejectedValue(new Error('docker run failed'));
+ const { host } = sharedHost();
+
+ await expect(startStack(startOptions(host))).rejects.toThrow('docker run failed');
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('fills in the runtime password a pre-role-split stack never had', async () => {
+ inspectStackMock.mockResolvedValue(
+ present({}, { secrets: { ...hostSecrets, dbRuntimePassword: null } })
+ );
+ const { host } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ const [{ secrets }] = buildEnvFileMock.mock.calls[0] as [{ secrets: LocalServerSecrets }];
+ expect(secrets.dbPassword).toBe('host-owner-pw');
+ expect(secrets.dbRuntimePassword).toMatch(/.{16,}/);
+ });
+
+ it('makes new credentials only on a host with nothing of the stack', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(loadOrCreateSecretsMock).toHaveBeenCalledOnce();
+ expect(buildEnvFileMock).toHaveBeenCalledWith(
+ expect.objectContaining({ secrets: cachedSecrets, ports: cachedPorts })
+ );
+ expect(publishEnvMock).toHaveBeenCalledOnce();
+ });
+
+ it('refuses another account’s unshared stack without writing anything', async () => {
+ inspectStackMock.mockResolvedValue({
+ kind: 'unshared',
+ ownerDir: '/home/alice/.switchdash/switch-server',
+ running: true,
+ });
+ const { host, writeFile } = sharedHost();
+
+ expect(await startStack(startOptions(host))).toEqual({
+ kind: 'error',
+ message: 'unshared vm-1 /home/alice/.switchdash/switch-server',
+ });
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ expect(loadOrCreateSecretsMock).not.toHaveBeenCalled();
+ expect(logError).toHaveBeenCalledOnce();
+ });
+
+ it('refuses a host it cannot read when this desktop has no copy to fall back on', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'unreadable', reason: 'docker ps failed' });
+ const { host, writeFile } = sharedHost();
+
+ const result = await startStack(startOptions(host));
+
+ expect(result.kind).toBe('error');
+ expect(result.kind === 'error' && result.message).toMatch(
+ /Could not read .* on vm-1 \(docker ps failed\).*nothing was changed/
+ );
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(loadOrCreateSecretsMock).not.toHaveBeenCalled();
+ });
+
+ it('refuses a host it cannot read even when this desktop holds a copy', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'unreadable', reason: 'published copy timed out' });
+ readSecretsMock.mockResolvedValue(cachedSecrets);
+ readPersistedPortsMock.mockResolvedValue(cachedPorts);
+ const { host, writeFile } = sharedHost();
+
+ const result = await startStack(startOptions(host));
+
+ expect(result.kind === 'error' && result.message).toMatch(
+ /Could not read .* on vm-1 \(published copy timed out\).*nothing was changed/
+ );
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('fills a partial host .env from this desktop’s copy when they agree, and says so', async () => {
+ inspectStackMock.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'working-dir',
+ missing: ['JWT_SECRET_KEY'],
+ // Everything the host does hold agrees with the copy.
+ raw: 'GATEWAY_HOST_PORT=3300\nGATEWAY_ADMIN_PASSWORD=cached-pw\nDB_OWNER_PASSWORD=host-owner-pw\n',
+ running: false,
+ });
+ readSecretsMock.mockResolvedValue(cachedSecrets);
+ readPersistedPortsMock.mockResolvedValue(cachedPorts);
+ const { host } = sharedHost();
+
+ expect(await startStack(startOptions(host))).toMatchObject({ kind: 'started' });
+ expect(buildEnvFileMock).toHaveBeenCalledWith(
+ expect.objectContaining({ secrets: cachedSecrets, ports: cachedPorts })
+ );
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringContaining("this desktop's copy"),
+ expect.objectContaining({ reason: 'its settings are missing JWT_SECRET_KEY' })
+ );
+ });
+
+ it('refuses to fill a partial settings file from a copy of another generation', async () => {
+ inspectStackMock.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: 'GATEWAY_HOST_PORT=3300\nGATEWAY_ADMIN_PASSWORD=host-admin-pw\n',
+ running: false,
+ });
+ readSecretsMock.mockResolvedValue(cachedSecrets);
+ readPersistedPortsMock.mockResolvedValue(cachedPorts);
+ const { host, writeFile } = sharedHost();
+
+ const result = await startStack(startOptions(host));
+
+ expect(result.kind === 'error' && result.message).toMatch(
+ /missing JWT_SECRET_KEY.*out of date \(GATEWAY_ADMIN_PASSWORD differ.*nothing was changed/
+ );
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('does not count a copy without its ports as one to fall back on', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'unreadable', reason: 'ssh dropped' });
+ readSecretsMock.mockResolvedValue(cachedSecrets);
+ const { host } = sharedHost();
+
+ expect((await startStack(startOptions(host))).kind).toBe('error');
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+});
+
+describe('connecting to a shared stack', () => {
+ it('joins a running stack without running compose or rewriting its settings', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, writeFile, establishNetworking } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({
+ kind: 'connected',
+ serverId: 'srv-1',
+ deployedVersion: '0.11.0',
+ });
+
+ expect(composeUpMock).not.toHaveBeenCalled();
+ expect(buildEnvFileMock).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ // The stack's own `.env`, so Stop and Restart work from here.
+ expect(writeFile).toHaveBeenCalledWith('.env', 'PUBLISHED_ENV\n', 0o600);
+ expect(writeFile).toHaveBeenCalledWith('standalone-docker-compose.yml', 'services: {}');
+ expect(establishNetworking).toHaveBeenCalledWith(hostPorts);
+ expect(storeSecretsMock).toHaveBeenCalledWith(host, hostSecrets);
+ expect(passwordLoginMock).toHaveBeenCalledWith(
+ expect.anything(),
+ 'host-admin-pw'
+ );
+ });
+
+ it('shares a stack this account started before settings were shared', async () => {
+ inspectStackMock.mockResolvedValue(
+ present({ source: 'working-dir', published: false, raw: 'OWN_ENV\n' })
+ );
+ const { host, sharedState, writeFile } = sharedHost();
+
+ expect((await connectStack(connectOptions(host))).kind).toBe('connected');
+ expect(publishEnvMock).toHaveBeenCalledWith(sharedState, 'OWN_ENV\n', held.lease);
+ expect(writeFile).not.toHaveBeenCalledWith('.env', expect.anything(), expect.anything());
+ });
+
+ it('brings an account’s older compose file to the one the stack runs, when joining it', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, writeFile, readFile } = sharedHost();
+ readFile.mockResolvedValue('services: { older: {} }');
+
+ await connectStack(connectOptions(host));
+
+ expect(writeFile).toHaveBeenCalledWith('standalone-docker-compose.yml', 'services: {}');
+ expect(writeFile).toHaveBeenCalledWith('.env', 'PUBLISHED_ENV\n', 0o600);
+ });
+
+ it('sends an older stack to be updated, touching nothing: this Console cannot use it as it is', async () => {
+ inspectStackMock.mockResolvedValue(present({}, { version: '0.10.0' }));
+ const { host, writeFile, establishNetworking } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({
+ kind: 'behind',
+ deployed: '0.10.0',
+ expected: '0.11.0',
+ });
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(establishNetworking).not.toHaveBeenCalled();
+ expect(storeSecretsMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('goes by the version the stack runs, not the one its settings ask for', async () => {
+ // A start elsewhere published the new version and then failed, leaving
+ // the old containers running.
+ inspectStackMock.mockResolvedValue(
+ present({ runningVersion: '0.10.0' }, { version: '0.11.0' })
+ );
+ const { host } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({
+ kind: 'behind',
+ deployed: '0.10.0',
+ expected: '0.11.0',
+ });
+ });
+
+ it('refuses a stack newer than this Console, saying to update the Console', async () => {
+ inspectStackMock.mockResolvedValue(present({}, { version: '0.12.0' }));
+ const { host, writeFile, establishNetworking } = sharedHost();
+
+ const result = await connectStack(connectOptions(host));
+
+ expect(result).toMatchObject({ kind: 'error' });
+ expect(result.kind === 'error' && result.message).toMatch(
+ /runs switch-core 0\.12\.0, newer than the 0\.11\.0 this Console runs.*Update Switch Console/
+ );
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(establishNetworking).not.toHaveBeenCalled();
+ });
+
+ it('joins a stack whose version cannot be compared as it is', async () => {
+ inspectStackMock.mockResolvedValue(present({}, { version: 'dev-checkout' }));
+ const { host } = sharedHost();
+
+ expect((await connectStack(connectOptions(host))).kind).toBe('connected');
+ });
+
+ it('sends a stopped stack to Start, touching nothing', async () => {
+ inspectStackMock.mockResolvedValue(present({ running: false }));
+ const { host, writeFile, establishNetworking } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({ kind: 'not-running' });
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(establishNetworking).not.toHaveBeenCalled();
+ expect(storeSecretsMock).not.toHaveBeenCalled();
+ });
+
+ it('reports an empty host as having nothing to join', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({ kind: 'absent' });
+ });
+
+ it('explains an unshared stack instead of joining it', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'unshared', ownerDir: null, running: true });
+ const { host, writeFile } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({
+ kind: 'unshared',
+ ownerDir: null,
+ message: 'unshared vm-1 null',
+ });
+ expect(writeFile).not.toHaveBeenCalled();
+ });
+
+ it('names the keys a partial settings file is missing', async () => {
+ inspectStackMock.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['DB_PASSWORD'],
+ raw: 'DB_OWNER_PASSWORD=host-owner-pw\n',
+ running: true,
+ });
+ const { host } = sharedHost();
+
+ const result = await connectStack(connectOptions(host));
+
+ expect(result.kind === 'error' && result.message).toContain('missing DB_PASSWORD');
+ });
+
+ it('says where it looked when the running stack does not answer', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ waitForHealthMock.mockResolvedValue(false);
+ const { host } = sharedHost();
+
+ expect(await connectStack(connectOptions(host))).toEqual({
+ kind: 'error',
+ message:
+ 'The Switch server on vm-1 is running, but did not answer at http://localhost:41000.',
+ });
+ expect(ensureManagedServerMock).not.toHaveBeenCalled();
+ });
+
+ it('has nothing to connect to on a host whose stack nobody shares', async () => {
+ const { host } = sharedHost();
+ const local = { ...host, sharedState: null } as unknown as ServerHost;
+
+ await expect(connectStack(connectOptions(local))).rejects.toThrow(/not shared/);
+ });
+});
+
+describe('resetting a shared stack', () => {
+ it('withdraws the published settings along with the data they opened', async () => {
+ const { host, sharedState } = sharedHost();
+
+ await resetStack(host, held.lease);
+
+ expect(composeDownMock).toHaveBeenCalledWith(host, true);
+ expect(withdrawPublishedEnvMock).toHaveBeenCalledWith(sharedState, held.lease);
+ expect(clearSecretsMock).toHaveBeenCalledWith(host);
+ });
+
+ it('has nothing to withdraw for a stack nobody shares', async () => {
+ const { host } = sharedHost();
+
+ await resetStack({ ...host, sharedState: null } as unknown as ServerHost, null);
+
+ expect(withdrawPublishedEnvMock).not.toHaveBeenCalled();
+ });
+});
+
+describe('starting where nothing of the stack is left', () => {
+ it('drops what this account still holds of a stack that is gone before checking versions', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host, removeFile } = sharedHost();
+ const order: string[] = [];
+ removeFile.mockImplementation(async (path) => void order.push(`remove ${path}`));
+ readDeployedVersionMock.mockImplementation(async () => {
+ order.push('version check');
+ return { kind: 'absent' };
+ });
+
+ expect(await startStack(startOptions(host))).toMatchObject({ kind: 'started' });
+
+ expect(order.indexOf('remove .env')).toBeGreaterThanOrEqual(0);
+ expect(order.indexOf('remove .env')).toBeLessThan(order.indexOf('version check'));
+ });
+});
+
+describe('usage data on a shared stack', () => {
+ const bobRecently = {
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+ },
+ ],
+ activity: [],
+ };
+ const telemetryOf = () => {
+ const params = buildEnvFileMock.mock.calls.at(-1)?.[0] as
+ | { telemetryEnabled: boolean }
+ | undefined;
+ if (!params) throw new Error('no .env was built');
+ return params.telemetryEnabled;
+ };
+
+ it('always applies this Console’s no', async () => {
+ inspectStackMock.mockResolvedValue(present({ raw: 'TELEMETRY_ENABLED=true\n' }));
+ telemetryConsentMock.mockResolvedValue(false);
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(telemetryOf()).toBe(false);
+ });
+
+ it('does not turn sharing on over the others using the stack', async () => {
+ inspectStackMock.mockResolvedValue(present({ raw: 'TELEMETRY_ENABLED=false\n' }));
+ telemetryConsentMock.mockResolvedValue(true);
+ readRegisterMock.mockResolvedValue(bobRecently);
+
+ expect(await startStack(startOptions(sharedHost().host))).toMatchObject({
+ kind: 'started',
+ telemetryEnabled: false,
+ });
+ expect(telemetryOf()).toBe(false);
+ });
+
+ it('keeps sharing on a stack that already shares', async () => {
+ inspectStackMock.mockResolvedValue(present({ raw: 'TELEMETRY_ENABLED=true\n' }));
+ telemetryConsentMock.mockResolvedValue(true);
+ readRegisterMock.mockResolvedValue(bobRecently);
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(telemetryOf()).toBe(true);
+ expect(readRegisterMock).not.toHaveBeenCalled();
+ });
+
+ it('applies this Console’s yes when nobody else uses the stack', async () => {
+ inspectStackMock.mockResolvedValue(present({ raw: 'TELEMETRY_ENABLED=false\n' }));
+ telemetryConsentMock.mockResolvedValue(true);
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(telemetryOf()).toBe(true);
+ });
+
+ it('keeps sharing off for a stack started afresh where others were using the one before', async () => {
+ // A reset keeps the register.
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ telemetryConsentMock.mockResolvedValue(true);
+ readRegisterMock.mockResolvedValue(bobRecently);
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(telemetryOf()).toBe(false);
+ });
+
+ it('keeps sharing off when it cannot tell who uses the stack', async () => {
+ inspectStackMock.mockResolvedValue(present({ raw: 'TELEMETRY_ENABLED=false\n' }));
+ telemetryConsentMock.mockResolvedValue(true);
+ readRegisterMock.mockRejectedValue(new Error('no such volume'));
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(telemetryOf()).toBe(false);
+ });
+});
+
+describe('stamping the published settings', () => {
+ it('stamps after compose only a copy published before its database existed', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ publishEnvMock.mockResolvedValue('2026-08-31T09:00:00Z');
+ const { host, writeFile } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(stampPublishedEnvMock).not.toHaveBeenCalled();
+ // The stamp it was published with, not asked for again.
+ expect(writeFile).toHaveBeenCalledWith('.env.db', '2026-08-31T09:00:00Z\n', 0o600);
+ });
+
+ it('does not fail a start that happened because the stamp could not be written, and says what it leaves open', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ stampPublishedEnvMock.mockRejectedValueOnce(new Error('helper container timed out'));
+
+ const result = await startStack(startOptions(sharedHost().host));
+
+ expect(result).toMatchObject({ kind: 'started' });
+ expect(result.kind === 'started' && result.warning).toMatch(
+ /could not be stamped with its database \(helper container timed out\)/
+ );
+ expect(logError).toHaveBeenCalledWith(
+ expect.stringMatching(/could not stamp the published settings on vm-1/),
+ expect.anything()
+ );
+ });
+});
+
+it('says what a stamp that failed with something other than an Error said', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ stampPublishedEnvMock.mockRejectedValueOnce('volume busy');
+
+ const result = await startStack(startOptions(sharedHost().host));
+
+ expect(result.kind === 'started' && result.warning).toMatch(/with its database \(volume busy\)/);
+});
+
+describe('the paths a shared start or join refuses or degrades on', () => {
+ it('refuses a partial host .env when this desktop holds no copy to fill it from', async () => {
+ inspectStackMock.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: 'GATEWAY_HOST_PORT=3300\n',
+ running: false,
+ });
+ const { host, writeFile } = sharedHost();
+
+ const result = await startStack(startOptions(host));
+
+ expect(result.kind === 'error' && result.message).toMatch(
+ /Could not read .* on vm-1 \(its settings are missing JWT_SECRET_KEY\).*nothing was changed/
+ );
+ expect(writeFile).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('starts a stack whose automatic sign-in fails, leaving the sign-in to the page', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ passwordLoginMock.mockResolvedValueOnce({ success: false, error: 'bad password' } as never);
+
+ expect(await startStack(startOptions(sharedHost().host))).toMatchObject({ kind: 'started' });
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(/auto sign-in failed/),
+ expect.objectContaining({ error: 'bad password' })
+ );
+ });
+
+ it('has nothing to adopt on a host whose stack nobody shares', async () => {
+ const { host } = sharedHost();
+ const local = { ...host, sharedState: null } as unknown as ServerHost;
+ const stack = present();
+
+ await expect(
+ adoptRunningStack(local, stack as Extract<StackOnHost, { kind: 'present' }>, held.lease)
+ ).rejects.toThrow(/not shared/);
+ });
+
+ it('reports Docker being unavailable before looking to join', async () => {
+ const { host } = sharedHost();
+ const noDocker = {
+ ...host,
+ detectDocker: () =>
+ Promise.resolve({ available: false, reason: 'daemon-down', detail: 'no daemon' }),
+ } as unknown as ServerHost;
+
+ expect(await connectStack(connectOptions(noDocker))).toEqual({
+ kind: 'docker-unavailable',
+ reason: 'daemon-down',
+ detail: 'no daemon',
+ });
+ expect(inspectStackMock).not.toHaveBeenCalled();
+ });
+
+ it('says why it cannot join a host whose settings cannot be read', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'unreadable', reason: 'docker ps timed out' });
+
+ expect(await connectStack(connectOptions(sharedHost().host))).toEqual({
+ kind: 'error',
+ message: "Could not read the Switch server's settings on vm-1: docker ps timed out",
+ });
+ });
+});
+
+describe('the server lock through a start, a join, a stop and a reset', () => {
+ it('refuses to change a shared stack without holding its lock', async () => {
+ await expect(startStack({ ...startOptions(sharedHost().host), lease: null })).rejects.toThrow(
+ /without holding its lock/
+ );
+
+ expect(inspectStackMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('checks the lock is still its own before running compose', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const order: string[] = [];
+ held.assertHeld.mockImplementation(async () => void order.push('check'));
+ composeUpMock.mockImplementation(async () => void order.push('compose up'));
+
+ await startStack(startOptions(sharedHost().host));
+
+ expect(order).toEqual(['check', 'compose up']);
+ });
+
+ it('publishes nothing and runs nothing once the lock was taken over while it was away', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ held.assertHeld.mockRejectedValueOnce(new Error('lock taken over'));
+
+ await expect(startStack(startOptions(sharedHost().host))).rejects.toThrow('lock taken over');
+
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('gives the lock back once a join has read the stack, before waiting for it to answer', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const order: string[] = [];
+ held.release.mockImplementation(async () => void order.push('release'));
+ waitForHealthMock.mockImplementation(async () => {
+ order.push('health');
+ return true;
+ });
+
+ expect(await connectStack(connectOptions(sharedHost().host))).toMatchObject({
+ kind: 'connected',
+ });
+ expect(order).toEqual(['release', 'health']);
+ });
+
+ it('stops nothing when the lock was taken over before the stop', async () => {
+ held.assertHeld.mockRejectedValueOnce(new Error('lock taken over'));
+
+ await expect(stopStack(sharedHost().host, held.lease)).rejects.toThrow('lock taken over');
+
+ expect(composeDownMock).not.toHaveBeenCalled();
+ });
+
+ it('resets nothing when the lock was taken over before the reset', async () => {
+ held.assertHeld.mockRejectedValueOnce(new Error('lock taken over'));
+
+ await expect(resetStack(sharedHost().host, held.lease)).rejects.toThrow('lock taken over');
+
+ expect(composeDownMock).not.toHaveBeenCalled();
+ expect(withdrawPublishedEnvMock).not.toHaveBeenCalled();
+ expect(clearSecretsMock).not.toHaveBeenCalled();
+ });
+
+ it('stops a stack nobody shares with no lock at all', async () => {
+ const local = { ...sharedHost().host, sharedState: null } as unknown as ServerHost;
+
+ await stopStack(local, null);
+
+ expect(composeDownMock).toHaveBeenCalledWith(local, false);
+ });
+
+ it('refuses a lock for a stack nobody shares', async () => {
+ const local = { ...sharedHost().host, sharedState: null } as unknown as ServerHost;
+
+ await expect(stopStack(local, held.lease)).rejects.toThrow(/no lock to hold/);
+ expect(composeDownMock).not.toHaveBeenCalled();
+ });
+});
+
+describe('what a start makes sure of before it changes anything', () => {
+ it('checks this Console can reach the stack before updating it for everyone', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, checkNetworking } = sharedHost();
+ const order: string[] = [];
+ checkNetworking.mockImplementation(async () => void order.push('check ports'));
+ assertManagedServerUrlFreeMock.mockImplementation(async () => void order.push('check address'));
+ prepareUpgradeMock.mockImplementation(async () => {
+ order.push('back up');
+ return null;
+ });
+ composeUpMock.mockImplementation(async () => void order.push('compose up'));
+
+ await startStack(startOptions(host));
+
+ expect(checkNetworking).toHaveBeenCalledWith(hostPorts);
+ expect(assertManagedServerUrlFreeMock).toHaveBeenCalledWith('http://localhost:41000', {
+ kind: 'remote',
+ sshHost: 'vm-1',
+ });
+ expect(order).toEqual(['check ports', 'check address', 'back up', 'compose up']);
+ });
+
+ it('changes nothing on the host when a port it needs is taken on this computer', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, checkNetworking } = sharedHost();
+ checkNetworking.mockRejectedValueOnce(
+ new Error('Port 41000 is already in use on this computer')
+ );
+
+ await expect(startStack(startOptions(host))).rejects.toThrow(/41000 is already in use/);
+
+ expect(prepareUpgradeMock).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ expect(composeUpMock).not.toHaveBeenCalled();
+ });
+
+ it('changes nothing on the host when its address is already another server’s here', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ assertManagedServerUrlFreeMock.mockRejectedValueOnce(
+ new Error('http://localhost:41000 is already the address of “My local server”')
+ );
+
+ await expect(startStack(startOptions(sharedHost().host))).rejects.toThrow(
+ /already the address/
+ );
+
+ expect(composeUpMock).not.toHaveBeenCalled();
+ expect(publishEnvMock).not.toHaveBeenCalled();
+ });
+});
+
+describe('recording which database this account’s settings are for', () => {
+ it('stamps the .env it started the stack with, once compose has made the database', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host, writeFile } = sharedHost();
+
+ const result = await startStack(startOptions(host));
+
+ expect(result).toMatchObject({ kind: 'started', warning: null });
+ expect(writeFile).toHaveBeenCalledWith('.env.db', '2026-09-01T10:00:00Z\n', 0o600);
+ expect(composeUpMock.mock.invocationCallOrder[0]).toBeLessThan(
+ stampPublishedEnvMock.mock.invocationCallOrder[0]!
+ );
+ });
+
+ it('leaves no stamp to vouch for settings when the stack has no database to stamp', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ stampPublishedEnvMock.mockResolvedValueOnce(null);
+ const { host, removeFile } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(removeFile).toHaveBeenCalledWith('.env.db');
+ });
+
+ it('copies the published stamp with the published settings it brings into the working dir', async () => {
+ inspectStackMock.mockResolvedValue(present({ stamp: '2026-09-02T11:00:00Z' }));
+ const { host, writeFile } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(writeFile).toHaveBeenCalledWith('.env.db', '2026-09-02T11:00:00Z\n', 0o600);
+ });
+
+ it('drops the stamp with the rest of what a gone stack left behind', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host, removeFile } = sharedHost();
+
+ await startStack(startOptions(host));
+
+ expect(removeFile.mock.calls.slice(0, 2)).toEqual([['.env'], ['.env.db']]);
+ });
+
+ it('says so, without failing the start, when this account’s stamp cannot be written', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ stampPublishedEnvMock.mockRejectedValueOnce(new Error('volume inspect timed out'));
+
+ const result = await startStack(startOptions(sharedHost().host));
+
+ expect(result.kind === 'started' && result.warning).toMatch(
+ /could not be stamped with its database \(volume inspect timed out\)/
+ );
+ });
+});
+
+describe('choosing ports for a new stack', () => {
+ it('chooses new ports when one it kept from a stack since gone is taken here', async () => {
+ inspectStackMock.mockResolvedValue({ kind: 'absent' });
+ const { host, checkNetworking } = sharedHost();
+ const fresh = { gateway: 42000, api: 42001, mattermost: 42002, postgres: 42003 };
+ (host as unknown as { pickFreePorts: () => Promise<typeof fresh> }).pickFreePorts = vi.fn(
+ async () => fresh
+ );
+ checkNetworking.mockRejectedValueOnce(new Error('Port 3300 is already in use'));
+
+ expect(await startStack(startOptions(host))).toMatchObject({ kind: 'started' });
+
+ expect(rememberPortsMock).toHaveBeenCalledWith(host, fresh);
+ expect(buildEnvFileMock).toHaveBeenCalledWith(expect.objectContaining({ ports: fresh }));
+ });
+
+ it('keeps an existing stack’s ports, which others use, and refuses instead', async () => {
+ inspectStackMock.mockResolvedValue(present());
+ const { host, checkNetworking } = sharedHost();
+ const pickFreePorts = vi.fn();
+ (host as unknown as { pickFreePorts: typeof pickFreePorts }).pickFreePorts = pickFreePorts;
+ checkNetworking.mockRejectedValue(new Error('Port 41000 is already in use'));
+
+ await expect(startStack(startOptions(host))).rejects.toThrow(/41000 is already in use/);
+ expect(pickFreePorts).not.toHaveBeenCalled();
+ expect(checkNetworking).toHaveBeenCalledWith(hostPorts);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
index f5e827c8e..36fbbb6c5 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.test.ts
@@ -24,6 +24,9 @@ const prepareUpgradeMock = vi.hoisted(() =>
vi.fn<(...args: unknown[]) => Promise<unknown>>(() => Promise.resolve(null))
);
const finishUpgradeMock = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+vi.mock('./console-register', () => ({
+ readRegister: vi.fn(async () => ({ self: 'me', consoles: [], activity: [] })),
+}));
vi.mock('@shared/app-identity', async (importOriginal) => ({
...(await importOriginal<typeof AppIdentity>()),
COMPATIBLE_SWITCH_VERSION: '0.11.0',
@@ -63,6 +66,7 @@ vi.mock('./telemetry-consent', () => ({
}));
const setActiveServerIdMock = vi.hoisted(() => vi.fn());
vi.mock('@main/core/switch-servers/servers-store', () => ({
+ assertManagedServerUrlFree: () => Promise.resolve(),
ensureManagedServer: () => Promise.resolve({ id: 'srv-1' }),
setActiveServerId: setActiveServerIdMock,
}));
@@ -84,8 +88,10 @@ function options() {
);
const host = {
label: 'this computer',
+ sharedState: null,
writeFile,
detectDocker: () => Promise.resolve({ available: true, version: '27.0.0' }),
+ checkNetworking: vi.fn(() => Promise.resolve()),
establishNetworking: vi.fn(() => Promise.resolve()),
};
return {
@@ -100,6 +106,7 @@ function options() {
onUpgrade: vi.fn(),
signal: new AbortController().signal,
checkoutRoot: null as string | null,
+ lease: null,
},
};
}
@@ -145,6 +152,7 @@ describe('startStack version guard', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
expect(writeFile).toHaveBeenCalledWith(ENV_FILE_NAME, expect.any(String), 0o600);
expect(composeUpMock).toHaveBeenCalledOnce();
@@ -157,6 +165,7 @@ describe('startStack version guard', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
});
@@ -171,6 +180,7 @@ describe('startStack version guard', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
});
@@ -182,6 +192,7 @@ describe('startStack version guard', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
// Degraded, but disclosed — a transient probe failure must not make the app
// unstartable.
@@ -199,6 +210,7 @@ describe('startStack version guard', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
});
@@ -256,6 +268,7 @@ describe('startStack checkout build', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
const override = writeFile.mock.calls.find(
@@ -278,6 +291,7 @@ describe('startStack checkout build', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: false,
+ warning: null,
});
expect(logWarn).toHaveBeenCalled();
});
@@ -294,6 +308,7 @@ describe('startStack checkout build', () => {
kind: 'started',
serverId: 'srv-1',
telemetryEnabled: true,
+ warning: null,
});
expect(buildEnvFileMock).toHaveBeenCalledWith(
expect.objectContaining({ telemetryEnabled: true })
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
index 0982bfa60..ce6af3bf6 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/pipeline.ts
@@ -1,32 +1,52 @@
import { resolveAgentServers } from '@main/core/agents/resolve-servers';
import { passwordLogin } from '@main/core/switch-servers/auth';
-import { ensureManagedServer, setActiveServerId } from '@main/core/switch-servers/servers-store';
+import {
+ assertManagedServerUrlFree,
+ ensureManagedServer,
+ setActiveServerId,
+} from '@main/core/switch-servers/servers-store';
import { log } from '@main/lib/logger';
import { COMPATIBLE_SWITCH_VERSION, RELEASE_REPO_OWNER } from '@shared/app-identity';
import {
CHECKOUT_IMAGE_TAG,
+ type ConnectRemoteServerResult,
+ othersRecentlySeen,
type StartLocalServerResult,
} from '@shared/core/managed-switch-server/managed-switch-server';
import type { ManagedServerRef } from '@shared/core/switch-servers/switch-servers';
import { bundledComposeYaml } from './bundled-compose';
import { checkoutBuildOverrideYaml } from './checkout-build';
import { composeDown, composeUp } from './compose';
+import { readRegister } from './console-register';
import {
BUILD_OVERRIDE_FILE_NAME,
COMPOSE_FILE_NAME,
ENV_FILE_NAME,
+ ENV_STAMP_FILE_NAME,
GHCR_REGISTRY,
LOCAL_SERVER_ADMIN_EMAIL,
} from './constants';
import { classifyVersionDrift, readDeployedVersion } from './deployed-version';
-import { buildEnvFile } from './env-file';
-import { apiUrlFor, gatewayUrlFor } from './free-port';
+import { buildEnvFile, keysDisagreeing, telemetryRequested } from './env-file';
+import { apiUrlFor, gatewayUrlFor, type LocalServerPorts } from './free-port';
import { waitForHealth } from './health';
import type { ServerHost } from './host/types';
import { finishUpgrade, type OwedUpgrade, prepareUpgrade } from './managed-upgrade';
import { crossesMatrixBoundary, runBackfill } from './matrix-migration';
-import { clearPorts, resolvePorts } from './ports';
-import { clearSecrets, loadOrCreateSecrets } from './secrets';
+import { clearPorts, readPersistedPorts, rememberPorts, resolvePorts } from './ports';
+import { type LocalServerSecrets, withRuntimePassword } from './secret-values';
+import { clearSecrets, loadOrCreateSecrets, readSecrets, storeSecrets } from './secrets';
+import type { ServerLease } from './stack-lock';
+import {
+ driftOf,
+ inspectStack,
+ publishEnv,
+ stampPublishedEnv,
+ type StackOnHost,
+ type StackStateHost,
+ unsharedStackMessage,
+ withdrawPublishedEnv,
+} from './stack-state';
import { telemetryConsent } from './telemetry-consent';
/**
@@ -57,8 +77,39 @@ export type StartStackOptions = {
/** Dev-only: root of the Switch checkout to build the stack's images from,
* instead of pulling this build's pinned images. Null is the released path. */
checkoutRoot: string | null;
+ /** The shared stack's lock, held for the whole start. Null for the local
+ * stack, which nobody shares. */
+ lease: ServerLease | null;
};
+/** The shared state and its lock, or null for a stack nobody shares. A lock
+ * without shared state, or shared state without a lock, throws. */
+function sharedUnderLock(
+ host: ServerHost,
+ lease: ServerLease | null
+): { state: StackStateHost; lease: ServerLease } | null {
+ if (host.sharedState !== null && lease !== null) return { state: host.sharedState, lease };
+ if (host.sharedState === null && lease === null) return null;
+ throw new Error(
+ host.sharedState !== null
+ ? `Refusing to change the shared stack on ${host.label} without holding its lock.`
+ : `The stack on ${host.label} is not shared, so it has no lock to hold.`
+ );
+}
+
+/** {@link sharedUnderLock} for an operation only a shared stack has; `what`
+ * names it in the error. */
+function requireSharedUnderLock(
+ host: ServerHost,
+ lease: ServerLease,
+ what: string
+): { state: StackStateHost; lease: ServerLease } {
+ if (host.sharedState === null) {
+ throw new Error(`The stack on ${host.label} is not shared, so there is nothing to ${what}.`);
+ }
+ return { state: host.sharedState, lease };
+}
+
/**
* Refuse to point an existing stack at an OLDER switch-core than it already
* runs. switch-core migrates its database forward at startup and Alembic has no
@@ -125,24 +176,6 @@ async function refuseDowngrade(
return { kind: 'version-downgrade', deployed: drift.deployed, expected: drift.expected };
}
-/**
- * Full start pipeline: detect Docker → refuse a downgrade → back up and
- * journal an upgrade → GHCR login → materialise compose + `.env` → `compose up`
- * → establish networking → health-gate → register (+ activate) → silent admin
- * sign-in → reconcile agent servers → close the upgrade journal. Returns
- * without registering anything if Docker is unavailable, the stack is newer
- * than this build, or it never turns healthy.
- *
- * Doubles as the update path: the `.env` and compose file are re-materialised
- * from this build every time, so `compose up -d` on an already-running stack
- * re-pulls the newly pinned tags and recreates only the changed containers,
- * leaving the data volumes in place for switch-core to migrate forward. A
- * stack behind the pin has its database dumped first (see managed-upgrade.ts).
- *
- * With `checkoutRoot` set (dev only) the images are built from that working
- * tree on every start instead of pulled, and tagged {@link CHECKOUT_IMAGE_TAG}
- * so nothing downstream mistakes them for a release.
- */
/**
* Move a stack past the last version that can read a Matrix homeserver, copying
* its history first.
@@ -190,15 +223,303 @@ async function migrateOffMatrix(
};
}
+export type StackSettings = { secrets: LocalServerSecrets; ports: LocalServerPorts };
+
+/**
+ * Where a start's ports and credentials come from. New credentials are made
+ * only when nothing of the stack is on the host: made anywhere else, they lock
+ * the stack out of the Postgres volume its first credentials created.
+ */
+type StartPlan =
+ | { kind: 'adopt'; stack: Extract<StackOnHost, { kind: 'present' }> }
+ /** Nothing on the host, or the local stack: this desktop's copy, or new
+ * credentials when it has none. */
+ | { kind: 'fresh' }
+ /** The host's settings have gaps, and this desktop's copy agrees with every
+ * setting the host does hold, so the copy fills them. */
+ | { kind: 'cached'; reason: string }
+ /** Starting could replace the credentials of a stack already there. */
+ | { kind: 'refused'; message: string };
+
+async function planStart(host: ServerHost): Promise<StartPlan> {
+ if (host.sharedState === null) return { kind: 'fresh' };
+ const stack = await inspectStack(host.sharedState);
+ switch (stack.kind) {
+ case 'present':
+ return { kind: 'adopt', stack };
+ case 'absent':
+ return { kind: 'fresh' };
+ case 'unshared':
+ return { kind: 'refused', message: unsharedStackMessage(host.label, stack.ownerDir) };
+ case 'unreadable':
+ // Not answered from this desktop's copy: someone may have reset the stack
+ // since, and publishing stale credentials would lock everyone out.
+ return {
+ kind: 'refused',
+ message:
+ `Could not read the Switch server's settings on ${host.label} (${stack.reason}). ` +
+ `Starting from this desktop's copy could put back credentials someone has since ` +
+ `replaced, so nothing was changed. Try again once the host answers.`,
+ };
+ case 'incomplete': {
+ const reason = `its settings are missing ${stack.missing.join(', ')}`;
+ const secrets = await readSecrets(host);
+ const ports = await readPersistedPorts(host);
+ if (secrets === null || ports === null) {
+ return {
+ kind: 'refused',
+ message:
+ `Could not read the Switch server's settings on ${host.label} (${reason}). ` +
+ `Starting without them could replace the credentials of a server that is already ` +
+ `there, so nothing was changed.`,
+ };
+ }
+ // A copy that disagrees with the host predates a reset of the stack.
+ const disagreeing = keysDisagreeing(stack.raw, { secrets, ports });
+ if (disagreeing.length > 0) {
+ return {
+ kind: 'refused',
+ message:
+ `The Switch server's settings on ${host.label} are missing ` +
+ `${stack.missing.join(', ')}, and this desktop's copy of them is out of date ` +
+ `(${disagreeing.join(', ')} differ from the host's), so it cannot fill the gap. ` +
+ `Starting from it would lock the server out of its database, so nothing was changed.`,
+ };
+ }
+ log.warn(
+ `managed-switch-server: the stack's settings on ${host.label} are partial; ` +
+ `filling them from this desktop's copy`,
+ { reason }
+ );
+ return { kind: 'cached', reason };
+ }
+ }
+}
+
+/** The host's settings as a full bundle, kept as this desktop's copy. */
+async function adoptSettings(
+ host: ServerHost,
+ stack: Extract<StackOnHost, { kind: 'present' }>
+): Promise<StackSettings> {
+ // A `.env` from before the database role split has no runtime password; one
+ // is made here, and the next start gives the role it.
+ const { secrets } = withRuntimePassword({
+ ...stack.env.secrets,
+ dbRuntimePassword: stack.env.secrets.dbRuntimePassword ?? '',
+ });
+ await storeSecrets(host, secrets);
+ await rememberPorts(host, stack.env.ports);
+ return { secrets, ports: stack.env.ports };
+}
+
+/**
+ * Take up a stack running on a shared host without touching it: keep its
+ * settings as this desktop's copy, bring the working dir in step, and publish
+ * its `.env` if it predates shared settings so the next person can join.
+ */
+export async function adoptRunningStack(
+ host: ServerHost,
+ stack: Extract<StackOnHost, { kind: 'present' }>,
+ lease: ServerLease
+): Promise<StackSettings> {
+ const shared = requireSharedUnderLock(host, lease, 'adopt');
+ const settings = await adoptSettings(host, stack);
+ await bringWorkingDirInStep(host, stack);
+ if (!stack.published) await publishEnv(shared.state, stack.raw, shared.lease);
+ return settings;
+}
+
+/** Make this account's working dir match a stack found on the host, so compose
+ * (version check, upgrade backup, Stop, Restart) reads what the stack runs with. */
+export async function bringWorkingDirInStep(
+ host: ServerHost,
+ stack: Extract<StackOnHost, { kind: 'present' }>
+): Promise<void> {
+ if (stack.source === 'published') {
+ await host.writeFile(ENV_FILE_NAME, stack.raw, 0o600);
+ await writeEnvStamp(host, stack.stamp);
+ }
+ // At this build's version the bundled file is what the stack runs, and Stop
+ // and Reset must know every service. At another version an existing file is
+ // left for a start to rewrite after any backup has read it; a missing one
+ // gets this build's, since a published stack is past the Matrix line.
+ if (driftOf(stack) === null || (await host.readFile(COMPOSE_FILE_NAME)) === null) {
+ await host.writeFile(COMPOSE_FILE_NAME, bundledComposeYaml());
+ }
+}
+
+/** Record which database this account's `.env` was written for. Null removes
+ * the record, so a stale one cannot vouch for it. */
+async function writeEnvStamp(host: ServerHost, stamp: string | null): Promise<void> {
+ if (stamp === null) await host.removeFile(ENV_STAMP_FILE_NAME);
+ else await host.writeFile(ENV_STAMP_FILE_NAME, `${stamp}\n`, 0o600);
+}
+
+/**
+ * Whether a start shares usage data. This Console's "no" always applies; its
+ * "yes" only where it overrides nobody's "no": the stack already shares, or
+ * nobody else has used it lately. An unreadable register counts as others.
+ */
+async function shareUsageData(
+ host: ServerHost,
+ plan: StartPlan,
+ consent: boolean
+): Promise<boolean> {
+ if (!consent || host.sharedState === null) return consent;
+ // The register is read whatever the plan: it outlives a reset, and a stack
+ // with partial settings is still someone's.
+ if (plan.kind === 'adopt' && telemetryRequested(plan.stack.raw)) return true;
+ try {
+ const others = othersRecentlySeen(await readRegister(host.sharedState), new Date());
+ if (others.length === 0) return true;
+ } catch (error) {
+ log.warn(`managed-switch-server: could not read who uses the server on ${host.label}`, {
+ error,
+ });
+ }
+ log.info(
+ `managed-switch-server: the server on ${host.label} does not share usage data and others ` +
+ `use it, so this start keeps it off`
+ );
+ return false;
+}
+
+async function settingsFor(
+ host: ServerHost,
+ plan: Exclude<StartPlan, { kind: 'refused' }>
+): Promise<StackSettings> {
+ if (plan.kind === 'adopt') return adoptSettings(host, plan.stack);
+ // `cached` was only chosen because a copy exists, so neither call mints.
+ return { secrets: await loadOrCreateSecrets(host), ports: await resolvePorts(host) };
+}
+
+/** Re-pick a fresh stack's remembered ports when one is now taken here:
+ * nothing on the host depends on them yet. */
+async function portsReachableHere(
+ host: ServerHost,
+ plan: Exclude<StartPlan, { kind: 'refused' }>,
+ settings: StackSettings
+): Promise<StackSettings> {
+ if (plan.kind !== 'fresh') return settings;
+ try {
+ await host.checkNetworking(settings.ports);
+ return settings;
+ } catch (error) {
+ log.info(`managed-switch-server: choosing new ports for a new stack on ${host.label}`, {
+ reason: error instanceof Error ? error.message : String(error),
+ });
+ const ports = await host.pickFreePorts();
+ await rememberPorts(host, ports);
+ return { ...settings, ports };
+ }
+}
+
+/**
+ * Register the stack, activate it if asked, and sign in as its admin with the
+ * password Switch Console generated. A failed sign-in is only logged: the
+ * server view falls back to its sign-in panel.
+ */
+async function registerAndSignIn(
+ ref: ManagedServerRef,
+ serverName: string,
+ settings: StackSettings,
+ activate: boolean,
+ onMessage: (message: string) => void
+): Promise<string> {
+ const server = await ensureManagedServer(
+ {
+ name: serverName,
+ gatewayUrl: gatewayUrlFor(settings.ports),
+ apiUrl: apiUrlFor(settings.ports),
+ },
+ ref
+ );
+ if (activate) await setActiveServerId(server.id);
+
+ onMessage('Signing in…');
+ const login = await passwordLogin(
+ server,
+ LOCAL_SERVER_ADMIN_EMAIL,
+ settings.secrets.gatewayAdminPassword
+ );
+ if (!login.success) {
+ log.warn('managed-switch-server: auto sign-in failed; server will show a sign-in prompt', {
+ error: login.error,
+ });
+ }
+
+ await resolveAgentServers();
+ return server.id;
+}
+
+/**
+ * Full start pipeline: detect Docker → plan where the settings come from →
+ * refuse a downgrade → back up and journal an upgrade → materialise compose +
+ * `.env` → publish the `.env` → `compose up` → establish networking →
+ * health-gate → register (+ activate) → silent admin sign-in → reconcile agent
+ * servers → close the upgrade journal. Returns without registering
+ * anything if Docker is unavailable, the host's stack cannot safely be started
+ * from here, the stack is newer than this build, or it never turns healthy.
+ *
+ * Doubles as the update path: the `.env` and compose file are re-materialised
+ * from this build every time, so `compose up -d` on an already-running stack
+ * re-pulls the newly pinned tags and recreates only the changed containers,
+ * leaving the data volumes in place for switch-core to migrate forward. A
+ * stack behind the pin has its database dumped first (see managed-upgrade.ts).
+ *
+ * With `checkoutRoot` set (dev only) the images are built from that working
+ * tree on every start instead of pulled, and tagged {@link CHECKOUT_IMAGE_TAG}
+ * so nothing downstream mistakes them for a release.
+ */
export async function startStack(opts: StartStackOptions): Promise<StartLocalServerResult> {
- const { host, ref, serverName, activate, onMessage, onLog, onUpgrade, signal, checkoutRoot } =
- opts;
+ const {
+ host,
+ ref,
+ serverName,
+ activate,
+ onMessage,
+ onLog,
+ onUpgrade,
+ signal,
+ checkoutRoot,
+ lease,
+ } = opts;
+ const shared = sharedUnderLock(host, lease);
const docker = await host.detectDocker();
if (!docker.available) {
return { kind: 'docker-unavailable', reason: docker.reason, detail: docker.detail };
}
+ if (host.sharedState !== null) onMessage('Reading the server’s settings on the host…');
+ const plan = await planStart(host);
+ if (plan.kind === 'refused') {
+ log.error(`managed-switch-server: refusing to start the stack on ${host.label}`, {
+ reason: plan.message,
+ });
+ return { kind: 'error', message: plan.message };
+ }
+
+ // Before anything reads the working dir: another account may have changed
+ // the stack since, and the version check below reads this `.env`.
+ if (plan.kind === 'adopt') {
+ await bringWorkingDirInStep(host, plan.stack);
+ } else if (plan.kind === 'fresh' && host.sharedState !== null) {
+ // The working dir belongs to a stack that is gone. Left there, its version
+ // would pass for the deployed one: a false downgrade refusal, or a backup
+ // of a database that does not exist.
+ await host.removeFile(ENV_FILE_NAME);
+ await host.removeFile(ENV_STAMP_FILE_NAME);
+ await finishUpgrade(host);
+ }
+
+ // Check this Console can reach the stack before changing it: a port clash
+ // found only after compose would leave everyone else's server restarted and
+ // this Console without it.
+ const settings = await portsReachableHere(host, plan, await settingsFor(host, plan));
+ await host.checkNetworking(settings.ports);
+ await assertManagedServerUrlFree(gatewayUrlFor(settings.ports), ref);
+
onMessage('Checking the deployed version…');
const downgrade = await refuseDowngrade(host, checkoutRoot);
if (downgrade) return downgrade;
@@ -211,6 +532,9 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
// thing that can read it. Runs against the stack as currently deployed, so
// it must happen before the compose file and `.env` are re-materialised for
// the new version — those are what would take Tuwunel away.
+ // Compose cannot check the lock atomically the way publishing does, so a
+ // Console that lost it stops here.
+ await shared?.lease.assertHeld();
const migration = await migrateOffMatrix(host, checkoutRoot, onMessage, onLog);
if (migration) return migration;
@@ -219,25 +543,27 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
if (checkoutRoot !== null) {
await host.writeFile(BUILD_OVERRIDE_FILE_NAME, checkoutBuildOverrideYaml(checkoutRoot));
}
- const secrets = await loadOrCreateSecrets(host);
- const ports = await resolvePorts(host);
- const gatewayUrl = gatewayUrlFor(ports);
- const apiUrl = apiUrlFor(ports);
// Read here rather than taken from the caller: a start is the moment the
// user's answer reaches the server, and no supervisor can forget to carry it.
- const telemetryEnabled = await telemetryConsent();
- await host.writeFile(
- ENV_FILE_NAME,
- buildEnvFile({
- version: checkoutRoot !== null ? CHECKOUT_IMAGE_TAG : COMPATIBLE_SWITCH_VERSION,
- registry: GHCR_REGISTRY,
- namespace: RELEASE_REPO_OWNER,
- ports,
- secrets,
- telemetryEnabled,
- }),
- 0o600
- );
+ const telemetryEnabled = await shareUsageData(host, plan, await telemetryConsent());
+ const env = buildEnvFile({
+ version: checkoutRoot !== null ? CHECKOUT_IMAGE_TAG : COMPATIBLE_SWITCH_VERSION,
+ registry: GHCR_REGISTRY,
+ namespace: RELEASE_REPO_OWNER,
+ ports: settings.ports,
+ secrets: settings.secrets,
+ telemetryEnabled,
+ });
+ await host.writeFile(ENV_FILE_NAME, env, 0o600);
+
+ // Published before compose runs, so the shared copy names what the stack was
+ // last asked to run with even if `up` fails halfway. A failure fails the
+ // start: unpublished settings get overwritten by the next Console.
+ let publishedStamp: string | null = null;
+ if (shared !== null) {
+ onMessage('Sharing the server’s settings on the host…');
+ publishedStamp = await publishEnv(shared.state, env, shared.lease);
+ }
onMessage(
checkoutRoot !== null
@@ -245,49 +571,156 @@ export async function startStack(opts: StartStackOptions): Promise<StartLocalSer
: 'Starting containers (pulling images if needed)…'
);
await composeUp(host, onLog, checkoutRoot !== null);
+ // A copy published before its database existed is stamped now. The stack is
+ // up, so a failure here is a warning, not a failed start.
+ let warning: string | null = null;
+ if (shared !== null) {
+ try {
+ await writeEnvStamp(
+ host,
+ publishedStamp ?? (await stampPublishedEnv(shared.state, shared.lease))
+ );
+ } catch (error) {
+ const reason = error instanceof Error ? error.message : String(error);
+ log.error(`managed-switch-server: could not stamp the published settings on ${host.label}`, {
+ error,
+ });
+ warning =
+ `Started, but its settings on ${host.label} could not be stamped with its ` +
+ `database (${reason}). Until the next start stamps them, a reset from a Console older ` +
+ `than sharing could leave them looking current to the others.`;
+ }
+ }
// Make the published ports reachable from the desktop (no-op locally; a
// mirrored SSH forward remotely) BEFORE the health probe, so the probe takes
// the same path clients will.
- await host.establishNetworking(ports);
+ await host.establishNetworking(settings.ports);
onMessage('Waiting for the server to become healthy…');
- const healthy = await waitForHealth(gatewayUrl, { signal });
+ const healthy = await waitForHealth(gatewayUrlFor(settings.ports), { signal });
if (!healthy) {
return { kind: 'error', message: 'The server did not become healthy in time.' };
}
- const server = await ensureManagedServer({ name: serverName, gatewayUrl, apiUrl }, ref);
- if (activate) await setActiveServerId(server.id);
+ const serverId = await registerAndSignIn(ref, serverName, settings, activate, onMessage);
+ if (upgrade) await finishUpgrade(host);
+ return { kind: 'started', serverId, telemetryEnabled, warning };
+}
- // Switch Console generated the admin password, so sign in on the user's behalf
- // rather than showing a login wall for a secret they never saw. A failure here
- // does not fail the start — the stack is healthy; the server view falls back
- // to its sign-in panel.
- onMessage('Signing in…');
- const login = await passwordLogin(server, LOCAL_SERVER_ADMIN_EMAIL, secrets.gatewayAdminPassword);
- if (!login.success) {
- log.warn('managed-switch-server: auto sign-in failed; server will show a sign-in prompt', {
- error: login.error,
- });
+export type ConnectStackOptions = {
+ host: ServerHost;
+ ref: ManagedServerRef;
+ serverName: string;
+ onMessage: (message: string) => void;
+ /** Aborts an in-flight health wait (cancel/quit). */
+ signal: AbortSignal;
+ /** The stack's lock, so a stack half-way through someone else's start does
+ * not read as stopped. Released once its settings are adopted. */
+ lease: ServerLease;
+};
+
+/** `behind` is a stack at an older switch-core than this build pins, which the
+ * caller updates as a start. */
+export type ConnectStackResult =
+ | Exclude<ConnectRemoteServerResult, { kind: 'cancelled' }>
+ | { kind: 'behind'; deployed: string; expected: string };
+
+/**
+ * Join a stack already running on a shared host without running compose, so
+ * nobody else using it notices. A stopped, older or unreadable stack is
+ * reported rather than worked around, and nothing here makes new credentials.
+ */
+export async function connectStack(opts: ConnectStackOptions): Promise<ConnectStackResult> {
+ const { host, ref, serverName, onMessage, signal, lease } = opts;
+ const shared = requireSharedUnderLock(host, lease, 'connect to');
+
+ const docker = await host.detectDocker();
+ if (!docker.available) {
+ return { kind: 'docker-unavailable', reason: docker.reason, detail: docker.detail };
}
- await resolveAgentServers();
- if (upgrade) await finishUpgrade(host);
- return { kind: 'started', serverId: server.id, telemetryEnabled };
+ onMessage('Reading the server’s settings on the host…');
+ const stack = await inspectStack(shared.state);
+ switch (stack.kind) {
+ case 'absent':
+ return { kind: 'absent' };
+ case 'unshared':
+ return {
+ kind: 'unshared',
+ ownerDir: stack.ownerDir,
+ message: unsharedStackMessage(host.label, stack.ownerDir),
+ };
+ case 'incomplete':
+ return {
+ kind: 'error',
+ message:
+ `The Switch server on ${host.label} is set up, but its settings are missing ` +
+ `${stack.missing.join(', ')}, so this Console cannot join it.`,
+ };
+ case 'unreadable':
+ return {
+ kind: 'error',
+ message: `Could not read the Switch server's settings on ${host.label}: ${stack.reason}`,
+ };
+ case 'present':
+ break;
+ }
+ if (!stack.running) return { kind: 'not-running' };
+ const drift = driftOf(stack);
+ if (drift?.direction === 'upgrade') {
+ return { kind: 'behind', deployed: drift.deployed, expected: drift.expected };
+ }
+ if (drift?.direction === 'downgrade') {
+ return {
+ kind: 'error',
+ message:
+ `The Switch server on ${host.label} runs switch-core ${drift.deployed}, newer than the ` +
+ `${drift.expected} this Console runs, so this Console cannot use it. Update Switch ` +
+ `Console, then connect.`,
+ };
+ }
+
+ onMessage('Preparing this account’s copy of the server’s settings…');
+ const settings = await adoptRunningStack(host, stack, lease);
+ await lease.release();
+
+ await host.establishNetworking(settings.ports);
+
+ onMessage('Waiting for the server to answer…');
+ const gatewayUrl = gatewayUrlFor(settings.ports);
+ if (!(await waitForHealth(gatewayUrl, { signal }))) {
+ return {
+ kind: 'error',
+ message: `The Switch server on ${host.label} is running, but did not answer at ${gatewayUrl}.`,
+ };
+ }
+
+ const serverId = await registerAndSignIn(ref, serverName, settings, true, onMessage);
+ return {
+ kind: 'connected',
+ serverId,
+ deployedVersion: stack.runningVersion ?? stack.env.version,
+ };
}
-/** Stop the stack's containers and tear down networking (leaves data + config). */
-export async function stopStack(host: ServerHost): Promise<void> {
+/** Stop the stack's containers and tear down networking (leaves data + config).
+ * `lease` is the shared stack's lock, null for the local one. */
+export async function stopStack(host: ServerHost, lease: ServerLease | null): Promise<void> {
+ await sharedUnderLock(host, lease)?.lease.assertHeld();
await composeDown(host, false);
await host.teardownNetworking();
}
/** Destroy the stack, its data volumes, stored secrets, and port choice — the
- * irreversible clean-slate reset. */
-export async function resetStack(host: ServerHost): Promise<void> {
+ * irreversible clean-slate reset. On a shared host the published settings go
+ * too, since their credentials now open nothing. */
+export async function resetStack(host: ServerHost, lease: ServerLease | null): Promise<void> {
+ const shared = sharedUnderLock(host, lease);
+ await shared?.lease.assertHeld();
await composeDown(host, true);
await host.teardownNetworking();
+ if (shared !== null) await withdrawPublishedEnv(shared.state, shared.lease);
await clearSecrets(host);
await clearPorts(host);
// Nothing is left to resume. The backups stay on disk.
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.test.ts
new file mode 100644
index 000000000..7bb23127e
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.test.ts
@@ -0,0 +1,41 @@
+import { mkdtempSync, rmSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { afterEach, beforeEach, expect, it, vi } from 'vitest';
+import type { ServerHost } from './host/types';
+import { readPersistedPorts, rememberPorts, resolvePorts } from './ports';
+
+let stateDir: string;
+const pickFreePorts = vi.fn();
+
+function host(): ServerHost {
+ return { stateDir, pickFreePorts } as unknown as ServerHost;
+}
+
+beforeEach(() => {
+ stateDir = join(mkdtempSync(join(tmpdir(), 'ports-')), 'remote', 'vm-1');
+ pickFreePorts.mockReset();
+});
+
+afterEach(() => {
+ rmSync(join(stateDir, '..', '..'), { recursive: true, force: true });
+});
+
+const theirs = { gateway: 41000, api: 41001, mattermost: 41002, postgres: 41003 };
+
+it('keeps the ports a shared stack publishes as this desktop’s copy, so a later start reuses them', async () => {
+ // Picking new ones would move the stack off the ports everyone else reaches it on.
+ await rememberPorts(host(), theirs);
+
+ expect(await readPersistedPorts(host())).toEqual(theirs);
+ expect(await resolvePorts(host())).toEqual(theirs);
+ expect(pickFreePorts).not.toHaveBeenCalled();
+});
+
+it('replaces an earlier copy with the ports the host holds now', async () => {
+ await rememberPorts(host(), { gateway: 3300, api: 8000, mattermost: 8065, postgres: 5432 });
+
+ await rememberPorts(host(), theirs);
+
+ expect(await readPersistedPorts(host())).toEqual(theirs);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.ts
index 3536958c1..1be49a5a5 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/ports.ts
@@ -49,8 +49,14 @@ export function readPersistedPorts(host: HostStateDir): Promise<LocalServerPorts
return loadPersisted(host);
}
-/** Drop the persisted choice so the next start picks fresh ports (reset path). */
-export async function clearPorts(host: ServerHost): Promise<void> {
+/** Record the ports a host's stack publishes, as read off the host itself. */
+export async function rememberPorts(host: ServerHost, ports: LocalServerPorts): Promise<void> {
+ await persist(host, ports);
+}
+
+/** Drop the persisted choice so the next start picks fresh ports (reset path),
+ * or so a Console that disconnects keeps nothing of a stack it no longer uses. */
+export async function clearPorts(host: HostStateDir): Promise<void> {
try {
await writeFile(portsFilePath(host), JSON.stringify({}), 'utf8');
} catch {
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.test.ts
new file mode 100644
index 000000000..698361980
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.test.ts
@@ -0,0 +1,45 @@
+import { beforeEach, expect, it, vi } from 'vitest';
+
+const service = vi.hoisted(() => ({
+ getStatuses: vi.fn(() => []),
+ detectDocker: vi.fn(),
+ probe: vi.fn(),
+ register: vi.fn(),
+ start: vi.fn(),
+ connect: vi.fn(),
+ disconnect: vi.fn(),
+ stop: vi.fn(),
+ reset: vi.fn(),
+ cancelWait: vi.fn(),
+ refresh: vi.fn(),
+}));
+vi.mock('./remote-server-service', () => ({ remoteServerService: service }));
+
+const { remoteSwitchServerController: rpc } = await import('./remote-controller');
+
+beforeEach(() => vi.clearAllMocks());
+
+it('hands the renderer’s shared-server calls to the supervisor for the host they name', async () => {
+ await rpc.probe('vm-1');
+ await rpc.register('vm-1');
+ await rpc.connect({ sshHost: 'vm-1', name: 'Team server' });
+ await rpc.start({ sshHost: 'vm-1', name: 'Team server' });
+ await rpc.disconnect('vm-1');
+ await rpc.stop('vm-1');
+ await rpc.reset('vm-1');
+ await rpc.detectDocker('vm-1');
+ await rpc.cancelWait('vm-1');
+ await rpc.refresh('vm-1');
+
+ expect(service.probe).toHaveBeenCalledWith('vm-1');
+ expect(service.register).toHaveBeenCalledWith('vm-1');
+ expect(service.connect).toHaveBeenCalledWith('vm-1', 'Team server');
+ expect(service.start).toHaveBeenCalledWith('vm-1', 'Team server');
+ expect(service.disconnect).toHaveBeenCalledWith('vm-1');
+ expect(service.stop).toHaveBeenCalledWith('vm-1');
+ expect(service.reset).toHaveBeenCalledWith('vm-1');
+ expect(service.detectDocker).toHaveBeenCalledWith('vm-1');
+ expect(service.cancelWait).toHaveBeenCalledWith('vm-1');
+ expect(service.refresh).toHaveBeenCalledWith('vm-1');
+ expect(await rpc.getStatuses()).toEqual([]);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.ts
index be1e02144..3d0624c18 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-controller.ts
@@ -1,6 +1,9 @@
import type {
+ ConnectRemoteServerResult,
DockerAvailability,
- StartLocalServerResult,
+ RemoteStackProbe,
+ StackRegister,
+ StartRemoteServerResult,
} from '@shared/core/managed-switch-server/managed-switch-server';
import type { RemoteServerStatus } from '@shared/events/remoteSwitchServerEvents';
import { createRPCController } from '@shared/lib/ipc/rpc';
@@ -13,9 +16,30 @@ export const remoteSwitchServerController = createRPCController({
detectDocker: (sshHost: string): Promise<DockerAvailability> =>
remoteServerService.detectDocker(sshHost),
- start: (params: { sshHost: string; name: string }): Promise<StartLocalServerResult> =>
+ probe: (sshHost: string): Promise<RemoteStackProbe> => remoteServerService.probe(sshHost),
+
+ register: (sshHost: string): Promise<StackRegister> => remoteServerService.register(sshHost),
+
+ start: (params: { sshHost: string; name: string }): Promise<StartRemoteServerResult> =>
remoteServerService.start(params.sshHost, params.name),
+ /** Look at a stack shown as stopped again, as its page opens. */
+ refresh: (sshHost: string): Promise<void> => {
+ remoteServerService.refresh(sshHost);
+ return Promise.resolve();
+ },
+
+ /** Stop waiting for another Console's hold on the host's stack. */
+ cancelWait: (sshHost: string): Promise<void> => {
+ remoteServerService.cancelWait(sshHost);
+ return Promise.resolve();
+ },
+
+ connect: (params: { sshHost: string; name: string }): Promise<ConnectRemoteServerResult> =>
+ remoteServerService.connect(params.sshHost, params.name),
+
+ disconnect: (sshHost: string): Promise<void> => remoteServerService.disconnect(sshHost),
+
stop: (sshHost: string): Promise<void> => remoteServerService.stop(sshHost),
reset: (sshHost: string): Promise<void> => remoteServerService.reset(sshHost),
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.test.ts
index f77b13dfe..df14ddb6d 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.test.ts
@@ -1,277 +1,1779 @@
-import { beforeEach, expect, it, vi } from 'vitest';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import type * as AppIdentity from '@shared/app-identity';
-import type { StartLocalServerResult } from '@shared/core/managed-switch-server/managed-switch-server';
+import type { RemoteServerStatus } from '@shared/events/remoteSwitchServerEvents';
+import type * as DeployedVersion from './deployed-version';
import type * as ManagedUpgrade from './managed-upgrade';
-import type { StartStackOptions } from './pipeline';
+import type * as StackLock from './stack-lock';
+import type { StackOnHost } from './stack-state';
+import type * as StackState from './stack-state';
-type Change = { current: { sshHost: string; status: string } };
-const m = vi.hoisted(() => {
- const listeners: ((change: Change) => void)[] = [];
- const blocked = new Set<string>();
- return {
- listeners,
- blocked,
- reachability: {
- on(_event: string, listener: (change: Change) => void) {
- listeners.push(listener);
- },
- isBlocked: (sshHost: string) => blocked.has(sshHost),
- requireReachable(sshHost: string) {
- if (blocked.has(sshHost)) throw new Error(`${sshHost} is unreachable`);
- },
- },
- servers: vi.fn(),
- running: vi.fn(),
- version: vi.fn(),
- journal: vi.fn(),
- ports: vi.fn(),
- establish: vi.fn(),
- start: vi.fn<(opts: StartStackOptions) => Promise<StartLocalServerResult>>(),
- };
-});
+/**
+ * The remote supervisor's side of shared stacks: joining one, leaving one
+ * without touching it, and following what other Consoles do to it.
+ */
+
+const requireReachable = vi.hoisted(() => vi.fn());
+const isBlocked = vi.hoisted(() => vi.fn(() => false));
+const onReachability = vi.hoisted(() => vi.fn());
+const createRemoteServerHost = vi.hoisted(() => vi.fn());
+const inspectStack = vi.hoisted(() => vi.fn<() => Promise<StackOnHost>>());
+const connectStack = vi.hoisted(() => vi.fn());
+const startStack = vi.hoisted(() => vi.fn());
+const stopStack = vi.hoisted(() => vi.fn(async () => {}));
+const resetStack = vi.hoisted(() => vi.fn(async () => {}));
+const deleteAgentsForServer = vi.hoisted(() => vi.fn(async () => ({ failed: [] })));
+const adoptRunningStack = vi.hoisted(() => vi.fn());
+const listManagedServers = vi.hoisted(() => vi.fn());
+const getRemoteManagedServer = vi.hoisted(() => vi.fn());
+const ensureManagedServer = vi.hoisted(() => vi.fn());
+const removeServer = vi.hoisted(() => vi.fn());
+const clearSecrets = vi.hoisted(() => vi.fn());
+const clearPorts = vi.hoisted(() => vi.fn());
+const readVersionStatus = vi.hoisted(() =>
+ vi.fn(() => Promise.resolve({ deployedVersion: '0.11.0', drift: null }))
+);
+const readDeployedTelemetry = vi.hoisted(() =>
+ vi.fn(() => Promise.resolve({ known: true, enabled: false }))
+);
+const emitted = vi.hoisted(() => [] as RemoteServerStatus[]);
+const writeRecord = vi.hoisted(() =>
+ vi.fn((_host: unknown, _action: unknown) => Promise.resolve())
+);
+const readRegister = vi.hoisted(() => vi.fn());
+const stateVolumeExists = vi.hoisted(() => vi.fn(async () => true));
+const readUpgradeJournal = vi.hoisted(() => vi.fn(() => Promise.resolve(null)));
+
+vi.mock('@main/core/remote-hosts/production-host-reachability', () => ({
+ hostReachabilityService: { requireReachable, isBlocked, on: onReachability },
+}));
+vi.mock('./host/remote-host', () => ({ createRemoteServerHost }));
+vi.mock('./stack-state', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackState>()),
+ inspectStack,
+ stateVolumeExists,
+}));
vi.mock('@shared/app-identity', async (importOriginal) => ({
...(await importOriginal<typeof AppIdentity>()),
COMPATIBLE_SWITCH_VERSION: '0.11.0',
}));
-vi.mock('@main/core/remote-hosts/production-host-reachability', () => ({
- hostReachabilityService: m.reachability,
+vi.mock('./pipeline', () => ({
+ bringWorkingDirInStep: async (
+ host: { writeFile: (...args: unknown[]) => Promise<void> },
+ stack: { source: string; raw: string }
+ ) => {
+ if (stack.source === 'published') await host.writeFile('.env', stack.raw, 0o600);
+ },
+ connectStack,
+ adoptRunningStack,
+ startStack,
+ stopStack,
+ resetStack,
+}));
+vi.mock('@main/core/switch-servers/servers-store', () => ({
+ listManagedServers,
+ getRemoteManagedServer,
+ ensureManagedServer,
+ removeServer,
}));
-vi.mock('@main/lib/events', () => ({ events: { emit: vi.fn() } }));
-vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), error: vi.fn(), info: vi.fn() } }));
+vi.mock('@main/core/switch-servers/delete-server-agents', () => ({ deleteAgentsForServer }));
vi.mock('@main/core/telemetry/managed-server', () => ({
reportManagedServerOutcome: vi.fn(),
reportManagedServerStart: vi.fn(),
reportManagedServerStartThrew: vi.fn(),
}));
-vi.mock('@main/core/switch-servers/delete-server-agents', () => ({
- deleteAgentsForServer: vi.fn(),
+vi.mock('@main/lib/events', () => ({
+ events: { emit: (_channel: unknown, status: RemoteServerStatus) => emitted.push(status) },
}));
-vi.mock('@main/core/switch-servers/servers-store', () => ({
- listManagedServers: m.servers,
- getRemoteManagedServer: vi.fn(),
+vi.mock('@main/lib/logger', () => ({ log: { info: vi.fn(), warn: vi.fn(), error: vi.fn() } }));
+vi.mock('./paths', () => ({ remoteServerStateDir: (slug: string) => `/user-data/remote/${slug}` }));
+vi.mock('./secrets', () => ({ clearSecrets }));
+vi.mock('./ports', () => ({ clearPorts }));
+vi.mock('./deployed-version', async (importOriginal) => ({
+ ...(await importOriginal<typeof DeployedVersion>()),
+ readVersionStatus,
}));
-vi.mock('./compose', () => ({ isStackRunning: m.running }));
-vi.mock('./deployed-version', () => ({ readVersionStatus: m.version }));
-vi.mock('./ports', () => ({ readPersistedPorts: m.ports }));
-vi.mock('./telemetry-consent', () => ({ readDeployedTelemetry: vi.fn() }));
-vi.mock('./managed-upgrade', async (importOriginal) => ({
- ...(await importOriginal<typeof ManagedUpgrade>()),
- readUpgradeJournal: m.journal,
+vi.mock('./telemetry-consent', () => ({ readDeployedTelemetry }));
+vi.mock('./console-register', () => ({
+ writeRecord,
+ readRegister,
+ hostAccount: async () => 'me',
}));
-vi.mock('./pipeline', () => ({ startStack: m.start, stopStack: vi.fn(), resetStack: vi.fn() }));
-vi.mock('./host/remote-host', () => ({
- createRemoteServerHost: async (sshHost: string) => ({
- sshHost,
- establishNetworking: m.establish,
- dispose: vi.fn(),
+
+/** The server lock, recording takes and releases in order unless a test holds it
+ * for someone else. The real module's errors and constants are kept. */
+const lockEvents = vi.hoisted(() => [] as string[]);
+const acquireServerLock = vi.hoisted(() =>
+ vi.fn(async (_host: unknown, claim: { action: string }, _opts: unknown) => {
+ lockEvents.push(`take ${claim.action}`);
+ return {
+ token: 'lease-token',
+ lost: false,
+ assertHeld: vi.fn(async () => {}),
+ release: vi.fn(async () => {
+ lockEvents.push(`release ${claim.action}`);
+ }),
+ };
+ })
+);
+const readServerLock = vi.hoisted(() => vi.fn(async () => null));
+vi.mock('./stack-lock', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackLock>()),
+ acquireServerLock,
+ readServerLock,
+}));
+const kvStore = vi.hoisted(() => new Map<string, unknown>());
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ constructor(private readonly namespace: string) {}
+ async get(key: string) {
+ return kvStore.get(`${this.namespace}:${key}`) ?? null;
+ }
+ async set(key: string, value: unknown) {
+ kvStore.set(`${this.namespace}:${key}`, value);
+ }
+ async del(key: string) {
+ kvStore.delete(`${this.namespace}:${key}`);
+ }
+ },
+}));
+vi.mock('@main/core/switch-servers/console-identity', () => ({
+ getConsoleIdentity: async () => ({
+ id: 'aaaaaaaa-0000-4000-8000-000000000001',
+ name: 'me@laptop',
}),
}));
-const { RemoteServerService } = await import('./remote-server-service');
+vi.mock('./managed-upgrade', async (importOriginal) => ({
+ ...(await importOriginal<typeof ManagedUpgrade>()),
+ readUpgradeJournal,
+}));
-const behind = {
- deployedVersion: '0.10.0',
- drift: { deployed: '0.10.0', expected: '0.11.0', direction: 'upgrade' },
+const ports = { gateway: 41000, api: 41001, mattermost: 41002, postgres: 41003 };
+const secrets = {
+ dbPassword: 'owner-pw',
+ dbRuntimePassword: 'runtime-pw',
+ agentRegistrationToken: 'agent-token',
+ jwtSecretKey: 'jwt',
+ gatewayAdminPassword: 'admin-pw',
+ mattermostAdminPassword: 'mm-admin',
+ mattermostUserPassword: 'mm-user',
};
-const inStep = { deployedVersion: '0.11.0', drift: null };
-function startedOn(serverId: string): StartLocalServerResult {
- return { kind: 'started', serverId, telemetryEnabled: false };
+function present(running: boolean): StackOnHost {
+ return {
+ kind: 'present',
+ env: { ports, secrets, version: '0.11.0' },
+ raw: 'PUBLISHED\n',
+ source: 'published',
+ running,
+ published: true,
+ runningVersion: null,
+ stamp: null,
+ };
}
-/** A start that runs the upgrade the way the pipeline does: announce, then work. */
-function upgradingStart(result?: StartLocalServerResult) {
- return async (opts: StartStackOptions) => {
- opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
- return result ?? startedOn(opts.ref.kind === 'remote' ? `srv-${opts.ref.sshHost}` : 'local');
+function fakeHost() {
+ return {
+ label: 'vm-1',
+ detectDocker: vi.fn(() => Promise.resolve({ available: true, version: '27.0.0' })),
+ establishNetworking: vi.fn(() => Promise.resolve()),
+ writeFile: vi.fn((_name: string, _content: string, _mode?: number) => Promise.resolve()),
+ dispose: vi.fn(),
};
}
-function announce(sshHost: string, status: string): void {
- for (const listener of m.listeners) listener({ current: { sshHost, status } });
+const RECORD = {
+ id: 'srv-1',
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+};
+
+/** Launch the service and wait out the reconciles it starts in the background. */
+async function boot(service: {
+ initialize(): Promise<void>;
+ ensureReady(sshHost: string, serverName: string): Promise<void>;
+}) {
+ await service.initialize();
+ await service.ensureReady('vm-1', 'Team server');
+}
+
+/** The service is a module singleton; each case gets a fresh one. */
+async function loadService() {
+ vi.resetModules();
+ return (await import('./remote-server-service')).remoteServerService;
}
beforeEach(() => {
- vi.resetAllMocks();
- m.listeners.length = 0;
- m.blocked.clear();
- m.servers.mockResolvedValue([
- { id: 'srv-builder', name: 'Builder', managementKind: 'remote', sshHost: 'builder' },
- ]);
- m.running.mockResolvedValue(true);
- m.version.mockResolvedValue(behind);
- m.journal.mockResolvedValue(null);
- m.ports.mockResolvedValue({ gateway: 1, api: 2, mattermost: 3, postgres: 4 });
- m.start.mockImplementation(upgradingStart());
+ vi.clearAllMocks();
+ emitted.length = 0;
+ lockEvents.length = 0;
+ kvStore.clear();
+ isBlocked.mockReturnValue(false);
+ listManagedServers.mockResolvedValue([RECORD]);
+ getRemoteManagedServer.mockResolvedValue(RECORD);
+ adoptRunningStack.mockResolvedValue({ secrets, ports });
});
-it('upgrades a running remote stack that is behind at boot, instead of adopting it', async () => {
- const service = new RemoteServerService();
- await service.initialize();
- await service.ensureReady('builder', 'Builder');
-
- expect(m.start).toHaveBeenCalledOnce();
- expect(m.start).toHaveBeenCalledWith(
- expect.objectContaining({
- ref: { kind: 'remote', sshHost: 'builder' },
- serverName: 'Builder',
- activate: false,
- })
- );
- expect(m.establish).not.toHaveBeenCalled();
- expect(service.getStatus('builder')).toMatchObject({
- phase: 'running',
- serverId: 'srv-builder',
- upgrade: null,
- drift: null,
+describe('connect', () => {
+ it('keeps the host — it owns the forward — and reports the stack running', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({
+ kind: 'connected',
+ serverId: 'srv-1',
+ deployedVersion: '0.11.0',
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({ kind: 'connected' });
+
+ expect(host.dispose).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'running',
+ serverId: 'srv-1',
+ deployedVersion: '0.11.0',
+ deployedTelemetry: { known: true, enabled: false },
+ error: null,
+ });
+ expect(connectStack).toHaveBeenCalledWith(
+ expect.objectContaining({
+ ref: { kind: 'remote', sshHost: 'vm-1' },
+ serverName: 'Team server',
+ })
+ );
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, 'connected');
+ });
+
+ it('joins an older stack by updating it, since this Console cannot use it as it is', async () => {
+ const joining = fakeHost();
+ const starting = fakeHost();
+ createRemoteServerHost.mockResolvedValueOnce(joining).mockResolvedValueOnce(starting);
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue({
+ kind: 'started',
+ serverId: 'srv-1',
+ telemetryEnabled: false,
+ warning: null,
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'connected',
+ serverId: 'srv-1',
+ deployedVersion: '0.11.0',
+ });
+
+ // Made the active server, as a Connect click does.
+ expect(startStack).toHaveBeenCalledWith(
+ expect.objectContaining({ host: starting, serverName: 'Team server', activate: true })
+ );
+ expect(joining.dispose).toHaveBeenCalledOnce();
+ expect(starting.dispose).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'running', serverId: 'srv-1' });
+ // What others see is what happened: the stack was started again.
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(starting, 'started');
+ });
+
+ it('reports an update that failed while joining as the reason it could not join', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue({
+ kind: 'error',
+ message: 'The server did not become healthy in time.',
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'error',
+ message: 'The server did not become healthy in time.',
+ });
+ expect(service.getStatus('vm-1').phase).toBe('error');
+ });
+
+ it('leaves a stopped stack stopped, for Start, and lets the host go', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'not-running' });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({ kind: 'not-running' });
+ expect(service.getStatus('vm-1').phase).toBe('stopped');
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(writeRecord).not.toHaveBeenCalled();
+ });
+
+ it('shows why another account’s unshared stack cannot be joined', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'unshared', ownerDir: null, message: 'not shared' });
+ const service = await loadService();
+
+ await service.connect('vm-1', 'Team server');
+
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'error', error: 'not shared' });
+ });
+
+ it('reports a failure to reach the host as an error, not a throw', async () => {
+ createRemoteServerHost.mockRejectedValue(new Error('ssh: connection refused'));
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'error',
+ message: 'ssh: connection refused',
+ });
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'error',
+ error: 'ssh: connection refused',
+ });
});
});
-it('holds sessions for the host while its upgrade runs', async () => {
- let finish: (result: StartLocalServerResult) => void = () => {};
- m.start.mockImplementation(
- (opts) =>
- new Promise((resolve) => {
- opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
- finish = resolve;
- })
- );
- const service = new RemoteServerService();
- void service.initialize();
- let ready = false;
- const session = service.ensureReady('builder', 'Builder').then(() => {
- ready = true;
- });
-
- await vi.waitFor(() => expect(m.start).toHaveBeenCalledOnce());
- expect(service.getStatus('builder').upgrade).toEqual({
- state: 'updating',
- from: '0.10.0',
- to: '0.11.0',
- });
- expect(ready).toBe(false);
- finish(startedOn('srv-builder'));
- await session;
- expect(ready).toBe(true);
+describe('disconnect', () => {
+ it('drops this Console’s hold on the stack and touches nothing on the host', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ createRemoteServerHost.mockClear();
+
+ writeRecord.mockClear();
+
+ await service.disconnect('vm-1');
+
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, 'disconnected');
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ expect(removeServer).toHaveBeenCalledExactlyOnceWith('srv-1');
+ expect(clearSecrets).toHaveBeenCalledWith({ secretsKey: 'remote-switch-server:vm-1:secrets' });
+ expect(clearPorts).toHaveBeenCalledWith({ stateDir: '/user-data/remote/vm-1' });
+ expect(service.getStatuses()).toEqual([]);
+ // The renderer is told the stack is no longer this Console's to show.
+ expect(emitted.at(-1)).toMatchObject({ sshHost: 'vm-1', phase: 'stopped', serverId: null });
+ });
+
+ it('leaves the register of a stopped server too, through a host of its own', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ getRemoteManagedServer.mockResolvedValue(RECORD);
+ const service = await loadService();
+
+ await service.disconnect('vm-1');
+
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, 'disconnected');
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(removeServer).toHaveBeenCalledExactlyOnceWith('srv-1');
+ });
+
+ it('does not wait on a slow host to leave, and closes the host it opened late', async () => {
+ vi.useFakeTimers();
+ try {
+ const host = fakeHost();
+ let opened: (value: unknown) => void = () => {};
+ createRemoteServerHost.mockReturnValue(new Promise((resolve) => (opened = resolve)));
+ getRemoteManagedServer.mockResolvedValue(RECORD);
+ const service = await loadService();
+
+ const leaving = service.disconnect('vm-1');
+ await vi.advanceTimersByTimeAsync(20_000);
+ await expect(leaving).resolves.toBeUndefined();
+ expect(removeServer).toHaveBeenCalledOnce();
+
+ opened(host);
+ await vi.runAllTimersAsync();
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(writeRecord).not.toHaveBeenCalled();
+ } finally {
+ vi.useRealTimers();
+ }
+ });
+
+ it('creates no register just to leave one where the stack never had any', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ getRemoteManagedServer.mockResolvedValue(RECORD);
+ stateVolumeExists.mockResolvedValueOnce(false);
+ const service = await loadService();
+
+ await service.disconnect('vm-1');
+
+ expect(writeRecord).not.toHaveBeenCalled();
+ expect(removeServer).toHaveBeenCalledOnce();
+ });
+
+ it('refuses while another operation on the host is running', async () => {
+ let finishConnect: (value: unknown) => void = () => {};
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockReturnValue(new Promise((resolve) => (finishConnect = resolve)));
+ const service = await loadService();
+ const connecting = service.connect('vm-1', 'Team server');
+ await vi.waitFor(() => expect(connectStack).toHaveBeenCalled());
+
+ await expect(service.disconnect('vm-1')).rejects.toThrow(/already in progress/);
+
+ finishConnect({ kind: 'not-running' });
+ await connecting;
+ });
});
-it('adopts a running stack that is in step without restarting it', async () => {
- m.version.mockResolvedValue(inStep);
- const service = new RemoteServerService();
- await service.initialize();
- await service.ensureReady('builder', 'Builder');
+describe('a record that cannot be written', () => {
+ it('lets the operation stand, and says on the server page what others will not see', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ writeRecord.mockRejectedValueOnce(new Error('docker run on vm-1 failed: no space left'));
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({ kind: 'connected' });
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'running',
+ recordWarning:
+ 'This Console could not record on vm-1 that it connected to it, so others using the ' +
+ 'server will not see that in its activity: docker run on vm-1 failed: no space left',
+ });
+ });
+
+ it('clears the warning once a record gets through', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ writeRecord.mockRejectedValueOnce(new Error('volume busy'));
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ expect(service.getStatus('vm-1').recordWarning).toMatch(/volume busy/);
+
+ // Launch picks the stack back up and records the sighting.
+ inspectStack.mockResolvedValue(present(true));
+ await boot(service);
- expect(m.start).not.toHaveBeenCalled();
- expect(m.establish).toHaveBeenCalledOnce();
- expect(service.getStatus('builder')).toMatchObject({
- phase: 'running',
- serverId: 'srv-builder',
- upgrade: null,
+ expect(service.getStatus('vm-1').recordWarning).toBeNull();
+ });
+
+ it('does not stop a disconnect, which has nowhere left to show it', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ writeRecord.mockRejectedValueOnce(new Error('volume busy'));
+
+ await expect(service.disconnect('vm-1')).resolves.toBeUndefined();
+ expect(removeServer).toHaveBeenCalledOnce();
});
});
-it('upgrades a stopped stack that is behind at its next start, not before', async () => {
- m.running.mockResolvedValue(false);
- const service = new RemoteServerService();
- const upgraded = vi.fn();
- service.onUpgradeFinished(upgraded);
- await service.initialize();
+describe('disconnecting from a host that is out of reach', () => {
+ it('still lets go, without waiting on the host to record it', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ writeRecord.mockClear();
+ isBlocked.mockReturnValue(true);
- await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
- /Builder is stopped on switch-core 0\.10\.0/
- );
- expect(m.start).not.toHaveBeenCalled();
+ await service.disconnect('vm-1');
- expect(await service.start('builder', 'Builder')).toMatchObject({ kind: 'started' });
- expect(m.start).toHaveBeenCalledWith(expect.objectContaining({ activate: true }));
- await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
- expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+ expect(writeRecord).not.toHaveBeenCalled();
+ expect(removeServer).toHaveBeenCalledOnce();
+ });
});
-it('upgrades a host that was unreachable at boot once it comes back, holding sessions meanwhile', async () => {
- m.blocked.add('builder');
- const service = new RemoteServerService();
- await service.initialize();
- await service.ensureReady('builder', 'Builder');
- expect(m.start).not.toHaveBeenCalled();
-
- let finish: (result: StartLocalServerResult) => void = () => {};
- m.start.mockImplementation(
- (opts) =>
- new Promise((resolve) => {
- opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
- finish = resolve;
- })
- );
- m.blocked.delete('builder');
- announce('builder', 'reachable');
- // Asked straight after the recovery, as a watcher restoring its controller does.
- let ready = false;
- const session = service.ensureReady('builder', 'Builder').then(() => {
- ready = true;
- });
-
- await vi.waitFor(() => expect(m.start).toHaveBeenCalledOnce());
- expect(ready).toBe(false);
- finish(startedOn('srv-builder'));
- await session;
- expect(service.getStatus('builder').upgrade).toBeNull();
+describe('picking a shared stack back up', () => {
+ it('adopts a running stack with the settings the host holds', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(adoptRunningStack).toHaveBeenCalledWith(host, present(true), expect.anything());
+ expect(lockEvents).toEqual(['take checking', 'release checking']);
+ expect(host.establishNetworking).toHaveBeenCalledWith(ports);
+ expect(host.dispose).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'running', notice: null });
+ expect(ensureManagedServer).not.toHaveBeenCalled();
+ // Seen, which refreshes the register, but nothing was done to the stack.
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, null);
+ });
+
+ it('follows a stack another Console restarted on different ports', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ getRemoteManagedServer.mockResolvedValue({
+ ...RECORD,
+ gatewayUrl: 'http://localhost:3300',
+ apiUrl: 'http://localhost:8000',
+ });
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(ensureManagedServer).toHaveBeenCalledWith(
+ {
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ },
+ { kind: 'remote', sshHost: 'vm-1' }
+ );
+ });
+
+ it('shows a stopped stack as stopped, with nothing to explain when this Console did not see it run', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(false));
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', notice: null });
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('does not stop the launch when the host cannot be read', async () => {
+ createRemoteServerHost.mockRejectedValue(new Error('timed out'));
+ const service = await loadService();
+
+ await expect(service.initialize()).resolves.toBeUndefined();
+ expect(service.getStatus('vm-1').phase).toBe('stopped');
+ });
});
-it('surfaces a failed upgrade with its error and retries when the host is reconciled again', async () => {
- m.start.mockImplementationOnce(
- upgradingStart({ kind: 'error', message: 'The server did not become healthy in time.' })
- );
- const service = new RemoteServerService();
- const upgraded = vi.fn();
- service.onUpgradeFinished(upgraded);
- await service.initialize();
+describe('recording that this Console still uses the stack', () => {
+ it('records it when picking the stack up, but not again at every re-check', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+
+ await boot(service);
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(expect.anything(), null);
+
+ // Requests fail; the stack is looked at again, and is still there.
+ service.recheck('vm-1');
+ await vi.waitFor(() => expect(inspectStack).toHaveBeenCalledTimes(2));
+ await vi.waitFor(() => expect(service.getStatus('vm-1').phase).toBe('running'));
- await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
- /Updating Builder from switch-core 0\.10\.0 to 0\.11\.0 failed: The server did not become healthy/
- );
- expect(service.getStatus('builder')).toMatchObject({
- phase: 'error',
- upgrade: { state: 'failed', error: 'The server did not become healthy in time.' },
+ expect(writeRecord).toHaveBeenCalledOnce();
});
- announce('builder', 'reachable');
- await vi.waitFor(() => expect(m.start).toHaveBeenCalledTimes(2));
- await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
- expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+ it('records it again once a day has passed', async () => {
+ vi.useFakeTimers({ toFake: ['Date'] });
+ try {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+
+ vi.setSystemTime(Date.now() + 25 * 60 * 60 * 1000);
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(writeRecord).toHaveBeenCalledTimes(2));
+ } finally {
+ vi.useRealTimers();
+ }
+ });
});
-it('does not hold one host’s sessions behind another host’s upgrade', async () => {
- m.servers.mockResolvedValue([
- { id: 'srv-builder', name: 'Builder', managementKind: 'remote', sshHost: 'builder' },
- { id: 'srv-other', name: 'Other', managementKind: 'remote', sshHost: 'other' },
- ]);
- m.start.mockImplementation(
- (opts) =>
- new Promise(() => {
- opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
- })
- );
- m.version.mockImplementation(async (host: { sshHost: string }) =>
- host.sshHost === 'builder' ? behind : inStep
- );
- const service = new RemoteServerService();
- await service.initialize();
+describe('recheck', () => {
+ async function runningService() {
+ const first = fakeHost();
+ createRemoteServerHost.mockResolvedValue(first);
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+ return { service, first };
+ }
+
+ it('says so when another Console stopped the stack this one was using', async () => {
+ const { service, first } = await runningService();
+ const second = fakeHost();
+ createRemoteServerHost.mockResolvedValue(second);
+ inspectStack.mockResolvedValue(present(false));
- await expect(service.ensureReady('other', 'Other')).resolves.toBeUndefined();
- expect(service.getStatus('builder').upgrade?.state).toBe('updating');
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(service.getStatus('vm-1').phase).toBe('stopped'));
+ expect(service.getStatus('vm-1').notice).toMatch(/stopped outside this Console/);
+ // The old forward pointed at a stack that is gone.
+ expect(first.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('says so when another Console reset the stack away', async () => {
+ const { service } = await runningService();
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'absent' });
+
+ service.recheck('vm-1');
+
+ await vi.waitFor(() =>
+ expect(service.getStatus('vm-1').notice).toMatch(/Nothing is set up on vm-1 any more/)
+ );
+ expect(service.getStatus('vm-1').phase).toBe('stopped');
+ });
+
+ it('keeps the forward it holds when the stack is still where it was', async () => {
+ const { service, first } = await runningService();
+ const second = fakeHost();
+ createRemoteServerHost.mockResolvedValue(second);
+
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(second.dispose).toHaveBeenCalledOnce());
+ expect(first.dispose).not.toHaveBeenCalled();
+ expect(second.establishNetworking).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'running', notice: null });
+ });
+
+ it.each([
+ [
+ 'cannot be reached',
+ () => createRemoteServerHost.mockRejectedValue(new Error('ssh: timed out')),
+ ],
+ [
+ 'cannot say what it has',
+ () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'unreadable', reason: 'docker ps failed' });
+ },
+ ],
+ ])('keeps a running stack running, and its forward, when the host %s', async (_, arrange) => {
+ const { service, first } = await runningService();
+ arrange();
+
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(service.getStatus('vm-1').notice).toMatch(/Could not check/));
+ expect(service.getStatus('vm-1').phase).toBe('running');
+ expect(first.dispose).not.toHaveBeenCalled();
+ });
+
+ it('looks at most once per interval, however many calls fail', async () => {
+ const { service } = await runningService();
+ createRemoteServerHost.mockClear();
+
+ service.recheck('vm-1');
+ service.recheck('vm-1');
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(createRemoteServerHost).toHaveBeenCalledOnce());
+ });
+
+ it('leaves alone a stack this Console does not think is running', async () => {
+ const service = await loadService();
+
+ service.recheck('vm-1');
+
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ });
+
+ it('leaves a blocked host to the reachability manager', async () => {
+ const { service } = await runningService();
+ createRemoteServerHost.mockClear();
+ isBlocked.mockReturnValue(true);
+
+ service.recheck('vm-1');
+
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ });
});
-it('refuses sessions when the host cannot be reached for its upgrade, rather than hanging', async () => {
- const service = new RemoteServerService();
- m.running.mockImplementation(async () => {
- // The host drops between the probe and the start.
- m.blocked.add('builder');
- return true;
+describe('a host coming back while an operation starts', () => {
+ it('does not read the host beside the operation, nor give back its flag', async () => {
+ // Launch finds the stack stopped, so there is no forward and a recovered
+ // host is picked back up.
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(false));
+ const service = await loadService();
+ await boot(service);
+ const [, onChange] = onReachability.mock.calls[0] as [
+ string,
+ (change: { current: { status: string; sshHost: string } }) => void,
+ ];
+ inspectStack.mockClear();
+
+ // The host comes back; the pick-up is waiting on the server list...
+ let listed!: () => void;
+ listManagedServers.mockImplementationOnce(
+ () => new Promise((resolve) => (listed = () => resolve([RECORD])))
+ );
+ onChange({ current: { status: 'reachable', sshHost: 'vm-1' } });
+
+ // ...when the user clicks Connect, which holds the host until it is done.
+ let joined!: () => void;
+ connectStack.mockImplementationOnce(
+ () =>
+ new Promise((resolve) => {
+ joined = () => resolve({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ })
+ );
+ const connecting = service.connect('vm-1', 'Team server');
+ await vi.waitFor(() => expect(connectStack).toHaveBeenCalledOnce());
+
+ listed();
+ await new Promise((resolve) => setTimeout(resolve, 0));
+
+ expect(inspectStack).not.toHaveBeenCalled();
+ // The flag is still Connect's: a second operation is refused.
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({
+ kind: 'error',
+ message: expect.stringMatching(/already in progress/),
+ });
+
+ joined();
+ expect(await connecting).toMatchObject({ kind: 'connected' });
});
- await service.initialize();
+});
+
+describe('a stopped stack another account updated', () => {
+ it('reads its version from the published settings, not this account’s stale copy', async () => {
+ const host = fakeHost();
+ const order: string[] = [];
+ host.writeFile.mockImplementation(async (name: string) => void order.push(`write ${name}`));
+ readVersionStatus.mockImplementation(async () => {
+ order.push('version');
+ return { deployedVersion: '0.11.0', drift: null };
+ });
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(false));
+ const service = await loadService();
+
+ await boot(service);
- await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
- /failed: builder is unreachable/
- );
+ expect(host.writeFile).toHaveBeenCalledWith('.env', 'PUBLISHED\n', 0o600);
+ expect(order.indexOf('write .env')).toBeLessThan(order.indexOf('version'));
+ });
});
+
+describe('connecting after the stack was brought up to date elsewhere', () => {
+ it('drops the update this Console last saw owed', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(false));
+ readVersionStatus.mockResolvedValueOnce({
+ deployedVersion: '0.10.0',
+ drift: { deployed: '0.10.0', expected: '0.11.0', direction: 'upgrade' },
+ } as never);
+ const service = await loadService();
+ await service.initialize();
+ await expect(service.ensureReady('vm-1', 'Team server')).rejects.toThrow(/stopped/);
+
+ // Someone else started it at the pin; this Console joins it.
+ connectStack.mockResolvedValue({
+ kind: 'connected',
+ serverId: 'srv-1',
+ deployedVersion: '0.11.0',
+ });
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({ kind: 'connected' });
+
+ expect(service.getStatus('vm-1').upgrade).toBeNull();
+ await expect(service.ensureReady('vm-1', 'Team server')).resolves.toBeUndefined();
+ });
+});
+
+describe('a start that could not do everything', () => {
+ it('says on the server page what it left open', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ startStack.mockResolvedValue({
+ kind: 'started',
+ serverId: 'srv-1',
+ telemetryEnabled: false,
+ warning: 'Started, but its shared settings could not be stamped.',
+ });
+ const service = await loadService();
+
+ expect(await service.start('vm-1', 'Team server')).toMatchObject({ kind: 'started' });
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'running',
+ notice: 'Started, but its shared settings could not be stamped.',
+ });
+ });
+});
+
+describe('probe', () => {
+ it('tells the renderer what the host has, without a secret in it', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+
+ const probe = await service.probe('vm-1');
+
+ expect(probe).toEqual({
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.11.0',
+ shared: true,
+ drift: null,
+ busy: null,
+ });
+ expect(acquireServerLock).not.toHaveBeenCalled();
+ expect(JSON.stringify(probe)).not.toContain('admin-pw');
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('reports Docker being unavailable before looking for a stack', async () => {
+ const host = fakeHost();
+ host.detectDocker.mockResolvedValue({
+ available: false,
+ reason: 'daemon-down',
+ detail: 'no daemon',
+ } as never);
+ createRemoteServerHost.mockResolvedValue(host);
+ const service = await loadService();
+
+ expect(await service.probe('vm-1')).toEqual({
+ kind: 'docker-unavailable',
+ reason: 'daemon-down',
+ detail: 'no daemon',
+ });
+ expect(inspectStack).not.toHaveBeenCalled();
+ });
+});
+
+describe('register', () => {
+ it('reads through the live host when this Console holds one', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'connected', serverId: 'srv-1', deployedVersion: null });
+ readRegister.mockResolvedValue({ self: 'me', consoles: [], activity: [] });
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ createRemoteServerHost.mockClear();
+
+ expect(await service.register('vm-1')).toEqual({ self: 'me', consoles: [], activity: [] });
+ expect(readRegister).toHaveBeenCalledWith(host);
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ });
+
+ it('opens and closes a host of its own otherwise', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ readRegister.mockResolvedValue({ self: 'me', consoles: [], activity: [] });
+ const service = await loadService();
+
+ await service.register('vm-1');
+
+ expect(readRegister).toHaveBeenCalledWith(host);
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+});
+
+describe('stopping and resetting a shared stack', () => {
+ it('stops it, says so on the host, and lets the forward go', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ const service = await loadService();
+
+ await service.stop('vm-1');
+
+ expect(stopStack).toHaveBeenCalledWith(host, expect.objectContaining({ token: 'lease-token' }));
+ expect(lockEvents).toEqual(['take stopping', 'release stopping']);
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, 'stopped');
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'stopped',
+ error: null,
+ notice: null,
+ });
+ });
+
+ it('turns an update held for others into one owed at the next start once stopped', async () => {
+ // A stopped stack reaches nobody, so the next start updates it without asking.
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ readVersionStatus.mockResolvedValueOnce({
+ deployedVersion: '0.10.0',
+ drift: { deployed: '0.10.0', expected: '0.11.0', direction: 'upgrade' },
+ } as never);
+ readRegister.mockResolvedValue({
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date().toISOString(),
+ },
+ ],
+ activity: [],
+ });
+ const service = await loadService();
+ await service.initialize();
+ await expect(service.ensureReady('vm-1', 'Team server')).rejects.toThrow(/Others use it too/);
+
+ await service.stop('vm-1');
+
+ expect(service.getStatus('vm-1').upgrade).toEqual({
+ state: 'pending',
+ from: '0.10.0',
+ to: '0.11.0',
+ });
+ });
+
+ it('reports a stop that failed, and gives the forward back', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ stopStack.mockRejectedValueOnce(new Error('compose down failed'));
+ const service = await loadService();
+
+ await expect(service.stop('vm-1')).rejects.toThrow(/compose down failed/);
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'error',
+ error: 'compose down failed',
+ });
+ expect(writeRecord).not.toHaveBeenCalled();
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('resets it, deleting its agents first and keeping who did it on the host', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ const service = await loadService();
+
+ await service.reset('vm-1');
+
+ expect(deleteAgentsForServer).toHaveBeenCalledWith('srv-1');
+ expect(deleteAgentsForServer.mock.invocationCallOrder[0]).toBeLessThan(
+ resetStack.mock.invocationCallOrder[0]!
+ );
+ expect(writeRecord).toHaveBeenCalledExactlyOnceWith(host, 'reset');
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'stopped',
+ deployedVersion: null,
+ drift: null,
+ upgrade: null,
+ });
+ });
+
+ it('refuses either while another operation on the host is running', async () => {
+ let finish: (value: unknown) => void = () => {};
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockReturnValue(new Promise((resolve) => (finish = resolve)));
+ const service = await loadService();
+ const connecting = service.connect('vm-1', 'Team server');
+ await vi.waitFor(() => expect(connectStack).toHaveBeenCalled());
+
+ await expect(service.stop('vm-1')).rejects.toThrow(/already in progress/);
+ await expect(service.reset('vm-1')).rejects.toThrow(/already in progress/);
+
+ finish({ kind: 'not-running' });
+ await connecting;
+ });
+});
+
+describe('connecting when it cannot', () => {
+ it('reports Docker being unavailable on the host', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({
+ kind: 'docker-unavailable',
+ reason: 'daemon-down',
+ detail: 'Cannot connect to the Docker daemon',
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({
+ kind: 'docker-unavailable',
+ });
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'error',
+ error: 'Cannot connect to the Docker daemon',
+ });
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('reports an update on joining that would be a downgrade, in words', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue({
+ kind: 'version-downgrade',
+ deployed: '0.12.0',
+ expected: '0.11.0',
+ });
+ const service = await loadService();
+
+ const result = await service.connect('vm-1', 'Team server');
+
+ expect(result).toMatchObject({ kind: 'error' });
+ expect(result.kind === 'error' && result.message).toMatch(/0\.12\.0/);
+ });
+
+ it('passes on Docker being unavailable for the update on joining', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue({
+ kind: 'docker-unavailable',
+ reason: 'not-installed',
+ detail: 'docker: command not found',
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'docker-unavailable',
+ reason: 'not-installed',
+ detail: 'docker: command not found',
+ });
+ });
+
+ it('reports a history copy that failed during the update on joining, in words', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue({
+ kind: 'matrix-migration-failed',
+ deployed: '0.10.0',
+ expected: '0.11.0',
+ detail: 'backfill exited 1',
+ });
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toMatchObject({ kind: 'error' });
+ });
+});
+
+describe('connecting through a failure it did not expect', () => {
+ it('reports the failure and lets the host go', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockRejectedValue(new Error('ssh: connection reset'));
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'error',
+ message: 'ssh: connection reset',
+ });
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'error',
+ error: 'ssh: connection reset',
+ });
+ expect(host.dispose).toHaveBeenCalledOnce();
+ });
+
+ it('passes the steps of a join on to the page as they happen', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockImplementation(async (opts: { onMessage: (message: string) => void }) => {
+ opts.onMessage('Waiting for the server to answer…');
+ return { kind: 'connected', serverId: 'srv-1', deployedVersion: '0.11.0' };
+ });
+ const service = await loadService();
+
+ await service.connect('vm-1', 'Team server');
+
+ expect(emitted.some((status) => status.message === 'Waiting for the server to answer…')).toBe(
+ true
+ );
+ });
+});
+
+describe('leaving a server this Console has no record of', () => {
+ it('still lets go of the host and forgets its credentials', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ getRemoteManagedServer.mockResolvedValue(undefined);
+ const service = await loadService();
+
+ await service.disconnect('vm-1');
+
+ expect(removeServer).not.toHaveBeenCalled();
+ expect(clearSecrets).toHaveBeenCalledWith({ secretsKey: 'remote-switch-server:vm-1:secrets' });
+ });
+});
+
+describe('what a launch says about a stack it cannot take up', () => {
+ it('names another account’s unshared stack', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({
+ kind: 'unshared',
+ ownerDir: '/home/alice/.switchdash',
+ running: true,
+ });
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped' });
+ expect(service.getStatus('vm-1').notice).toMatch(
+ /another account \(from \/home\/alice\/\.switchdash\)/
+ );
+ });
+
+ it('names what a partial stack is missing', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: '',
+ running: false,
+ });
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1').notice).toMatch(/missing JWT_SECRET_KEY/);
+ });
+
+ it('still reports a stopped stack when this account’s copy of its settings cannot be refreshed', async () => {
+ const host = fakeHost();
+ host.writeFile.mockRejectedValue(new Error('read-only file system'));
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(false));
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'stopped',
+ deployedVersion: '0.11.0',
+ });
+ });
+
+ it('says on the page when the record that it still uses the server cannot be written', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ writeRecord.mockRejectedValueOnce(new Error('volume busy'));
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1').recordWarning).toBe(
+ 'Could not record on vm-1 that this Console uses the server, so others may not see it ' +
+ 'among its users: volume busy'
+ );
+ });
+});
+
+describe('re-checking while something else is running', () => {
+ it('leaves the host alone until that operation is done', async () => {
+ let finish: (value: unknown) => void = () => {};
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockReturnValueOnce(
+ Promise.resolve({ kind: 'connected', serverId: 'srv-1', deployedVersion: '0.11.0' })
+ );
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ startStack.mockReturnValue(new Promise((resolve) => (finish = resolve)));
+ const restarting = service.start('vm-1', 'Team server');
+ inspectStack.mockClear();
+
+ service.recheck('vm-1');
+ await new Promise((resolve) => setTimeout(resolve, 0));
+
+ expect(inspectStack).not.toHaveBeenCalled();
+ finish({ kind: 'started', serverId: 'srv-1', telemetryEnabled: false, warning: null });
+ await restarting;
+ });
+});
+
+describe('the server lock', () => {
+ const bob = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting' as const,
+ heldForSeconds: 40,
+ expiresInSeconds: 80,
+ };
+ const started = { kind: 'started', serverId: 'srv-1', telemetryEnabled: false, warning: null };
+ const connected = { kind: 'connected', serverId: 'srv-1', deployedVersion: '0.11.0' };
+
+ type WaitOptions = { signal: AbortSignal; onWaiting: (holder: typeof bob) => void };
+
+ /** Held by bob until the wait is cancelled. */
+ function heldUntilCancelled() {
+ acquireServerLock.mockImplementationOnce(async (_host, _claim, opts) => {
+ const { signal, onWaiting } = opts as WaitOptions;
+ onWaiting(bob);
+ await new Promise((resolve) => signal.addEventListener('abort', resolve, { once: true }));
+ // The module the service loaded, whose class it checks for.
+ const { ServerLockWaitCancelled } = await import('./stack-lock');
+ throw new ServerLockWaitCancelled(bob);
+ });
+ }
+
+ /** Held by bob until the returned function is called, then taken. */
+ function heldUntilFreed(): () => void {
+ const take = acquireServerLock.getMockImplementation()!;
+ let free = () => {};
+ acquireServerLock.mockImplementationOnce(async (host, claim, opts) => {
+ (opts as WaitOptions).onWaiting(bob);
+ await new Promise<void>((resolve) => (free = resolve));
+ return take(host, claim, opts);
+ });
+ return () => free();
+ }
+
+ /** Call after loadService: built from the module the service loaded, whose
+ * class it checks for. */
+ async function busyError() {
+ const { ServerBusyError } =
+ await import('@shared/core/managed-switch-server/managed-switch-server');
+ return new ServerBusyError(bob, 'vm-1');
+ }
+
+ it('waits for someone else’s start, saying who for, and only then looks at the host', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ startStack.mockResolvedValue(started);
+ const free = heldUntilFreed();
+ const service = await loadService();
+
+ const starting = service.start('vm-1', 'Team server');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+
+ expect(service.getStatus('vm-1').message).toBe(
+ 'Waiting for bob@desk (as bob) to finish starting the server…'
+ );
+ expect(startStack).not.toHaveBeenCalled();
+
+ free();
+ expect(await starting).toMatchObject({ kind: 'started' });
+ expect(startStack).toHaveBeenCalledWith(
+ expect.objectContaining({ lease: expect.objectContaining({ token: 'lease-token' }) })
+ );
+ expect(service.getStatus('vm-1').waitingFor).toBeNull();
+ expect(lockEvents).toEqual(['take starting', 'release starting']);
+ });
+
+ it('stops waiting when cancelled, leaving the server and its forward as they were', async () => {
+ const live = fakeHost();
+ const restarting = fakeHost();
+ createRemoteServerHost.mockResolvedValueOnce(live).mockResolvedValueOnce(restarting);
+ connectStack.mockResolvedValue(connected);
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ heldUntilCancelled();
+
+ const start = service.start('vm-1', 'Team server');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+ service.cancelWait('vm-1');
+
+ expect(await start).toEqual({ kind: 'cancelled' });
+ expect(startStack).not.toHaveBeenCalled();
+ expect(live.dispose).not.toHaveBeenCalled();
+ expect(restarting.dispose).toHaveBeenCalledOnce();
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'running',
+ waitingFor: null,
+ message: null,
+ error: null,
+ });
+ });
+
+ it('gives the lock back before letting the host go when a start fails', async () => {
+ const host = fakeHost();
+ host.dispose.mockImplementation(() => void lockEvents.push('dispose'));
+ createRemoteServerHost.mockResolvedValue(host);
+ startStack.mockResolvedValue({ kind: 'error', message: 'compose up failed' });
+ const service = await loadService();
+
+ await service.start('vm-1', 'Team server');
+
+ expect(lockEvents).toEqual(['take starting', 'release starting', 'dispose']);
+ });
+
+ it('gives the lock back when a start throws', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ startStack.mockRejectedValueOnce(new Error('ssh dropped'));
+ const service = await loadService();
+
+ expect(await service.start('vm-1', 'Team server')).toMatchObject({ kind: 'error' });
+
+ expect(lockEvents).toEqual(['take starting', 'release starting']);
+ });
+
+ it('joins under the lock, and gives it back before letting the host go', async () => {
+ const host = fakeHost();
+ host.dispose.mockImplementation(() => void lockEvents.push('dispose'));
+ createRemoteServerHost.mockResolvedValue(host);
+ connectStack.mockResolvedValue({ kind: 'not-running' });
+ const service = await loadService();
+
+ await service.connect('vm-1', 'Team server');
+
+ expect(connectStack).toHaveBeenCalledWith(
+ expect.objectContaining({ lease: expect.objectContaining({ token: 'lease-token' }) })
+ );
+ expect(lockEvents).toEqual(['take connecting', 'release connecting', 'dispose']);
+ });
+
+ it('tells the others it is updating when joining an older server updates it', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ startStack.mockResolvedValue(started);
+ const service = await loadService();
+
+ await service.connect('vm-1', 'Team server');
+
+ expect(lockEvents).toEqual([
+ 'take connecting',
+ 'release connecting',
+ 'take updating',
+ 'release updating',
+ ]);
+ });
+
+ it('ends a join that was waiting when cancelled, having changed nothing', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ heldUntilCancelled();
+ const service = await loadService();
+
+ const joining = service.connect('vm-1', 'Team server');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+ service.cancelWait('vm-1');
+
+ expect(await joining).toEqual({ kind: 'cancelled' });
+ expect(connectStack).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', waitingFor: null });
+ });
+
+ it('refuses a stop while someone else changes the server, touching nothing and keeping the forward', async () => {
+ const live = fakeHost();
+ createRemoteServerHost.mockResolvedValue(live);
+ connectStack.mockResolvedValue(connected);
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ acquireServerLock.mockRejectedValueOnce(await busyError());
+
+ await expect(service.stop('vm-1')).rejects.toThrow(
+ /^bob@desk \(as bob\) is starting the server on vm-1 right now, so nothing was changed\./
+ );
+
+ expect(stopStack).not.toHaveBeenCalled();
+ expect(live.dispose).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'running', error: null });
+ expect(writeRecord).not.toHaveBeenCalledWith(live, 'stopped');
+ });
+
+ it('lets the host go when a refused stop had to open one of its own, and stays as it was', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ const service = await loadService();
+ acquireServerLock.mockRejectedValueOnce(await busyError());
+
+ await expect(service.stop('vm-1')).rejects.toThrow(/right now/);
+
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', error: null });
+ });
+
+ it('keeps a running server and its forward when a stop cannot even ask for the lock', async () => {
+ const live = fakeHost();
+ createRemoteServerHost.mockResolvedValue(live);
+ connectStack.mockResolvedValue(connected);
+ const service = await loadService();
+ await service.connect('vm-1', 'Team server');
+ acquireServerLock.mockRejectedValueOnce(new Error('docker run on vm-1 failed: no daemon'));
+
+ await expect(service.stop('vm-1')).rejects.toThrow(/no daemon/);
+
+ expect(stopStack).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'running', error: null });
+ expect(live.dispose).not.toHaveBeenCalled();
+ });
+
+ it('lets go of a host a reset opened for itself when it cannot ask for the lock', async () => {
+ const host = fakeHost();
+ createRemoteServerHost.mockResolvedValue(host);
+ const service = await loadService();
+ acquireServerLock.mockRejectedValueOnce(new Error('id: cannot find name for user ID 1001'));
+
+ await expect(service.reset('vm-1')).rejects.toThrow(/cannot find name/);
+
+ expect(deleteAgentsForServer).not.toHaveBeenCalled();
+ expect(host.dispose).toHaveBeenCalledOnce();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', error: null });
+ });
+
+ it('comes back to where it was when the update after Update and connect stops waiting', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockResolvedValue({ kind: 'behind', deployed: '0.10.0', expected: '0.11.0' });
+ const take = acquireServerLock.getMockImplementation()!;
+ acquireServerLock.mockImplementationOnce(take).mockImplementationOnce(async (_h, _c, opts) => {
+ const { signal, onWaiting } = opts as WaitOptions;
+ onWaiting(bob);
+ await new Promise((resolve) => signal.addEventListener('abort', resolve, { once: true }));
+ const { ServerLockWaitCancelled } = await import('./stack-lock');
+ throw new ServerLockWaitCancelled(bob);
+ });
+ const service = await loadService();
+
+ const joining = service.connect('vm-1', 'Team server');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+ service.cancelWait('vm-1');
+
+ expect(await joining).toEqual({ kind: 'cancelled' });
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', waitingFor: null });
+ });
+
+ it('refuses a reset while someone else changes the server, before deleting any agent', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ const service = await loadService();
+ acquireServerLock.mockRejectedValueOnce(await busyError());
+
+ await expect(service.reset('vm-1')).rejects.toThrow(/right now, so nothing was changed/);
+
+ expect(deleteAgentsForServer).not.toHaveBeenCalled();
+ expect(resetStack).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({ phase: 'stopped', error: null });
+ });
+
+ it('takes the lock for a reset before deleting its agents', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ deleteAgentsForServer.mockImplementationOnce(async () => {
+ lockEvents.push('delete agents');
+ return { failed: [] };
+ });
+ const service = await loadService();
+
+ await service.reset('vm-1');
+
+ expect(resetStack).toHaveBeenCalledWith(
+ expect.anything(),
+ expect.objectContaining({ token: 'lease-token' })
+ );
+ expect(lockEvents).toEqual(['take resetting', 'delete agents', 'release resetting']);
+ });
+
+ it('stops waiting to check the server when cancelled, and says it has not checked it', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ heldUntilCancelled();
+ const service = await loadService();
+ await service.initialize();
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+
+ service.cancelWait('vm-1');
+ await service.ensureReady('vm-1', 'Team server');
+
+ expect(inspectStack).not.toHaveBeenCalled();
+ expect(service.getStatus('vm-1')).toMatchObject({
+ waitingFor: null,
+ notice:
+ 'Stopped waiting for bob@desk (as bob) to finish starting the server, so this Console ' +
+ 'has not checked it since.',
+ });
+ });
+
+ it('gives up every wait for the lock when the app quits', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ heldUntilCancelled();
+ const service = await loadService();
+
+ const starting = service.start('vm-1', 'Team server');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').waitingFor).toEqual(bob));
+ service.dispose();
+
+ expect(await starting).toEqual({ kind: 'cancelled' });
+ });
+
+ it('tells the setup step who is changing the server right now', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'absent' });
+ readServerLock.mockResolvedValueOnce(bob as never);
+ const service = await loadService();
+
+ expect(await service.probe('vm-1')).toEqual({ kind: 'absent', busy: bob });
+ });
+
+ it('shows nobody when who holds the lock cannot be read, since starting takes it anyway', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'absent' });
+ readServerLock.mockRejectedValueOnce(new Error('docker run failed'));
+ const service = await loadService();
+
+ expect(await service.probe('vm-1')).toEqual({ kind: 'absent', busy: null });
+ });
+});
+
+describe('the paths the rest leave', () => {
+ async function running() {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+ return service;
+ }
+
+ it('reports a reset that failed, keeping the agents it had not reached yet', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ resetStack.mockRejectedValueOnce(new Error('compose down -v failed'));
+ const service = await loadService();
+
+ await expect(service.reset('vm-1')).rejects.toThrow(/compose down -v failed/);
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'error',
+ error: 'compose down -v failed',
+ });
+ expect(writeRecord).not.toHaveBeenCalled();
+ expect(lockEvents).toEqual(['take resetting', 'release resetting']);
+ });
+
+ it('wakes what it turned away for an update once someone else has made it', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ readVersionStatus.mockResolvedValueOnce({
+ deployedVersion: '0.10.0',
+ drift: { deployed: '0.10.0', expected: '0.11.0', direction: 'upgrade' },
+ } as never);
+ readRegister.mockResolvedValue({
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date().toISOString(),
+ },
+ ],
+ activity: [],
+ });
+ const service = await loadService();
+ const woken = vi.fn();
+ service.onUpgradeFinished(woken);
+ await service.initialize();
+ await expect(service.ensureReady('vm-1', 'Team server')).rejects.toThrow(/Others use it too/);
+
+ // Bob updates it; this Console's calls fail while it restarts, and it looks again.
+ service.recheck('vm-1');
+
+ await vi.waitFor(() => expect(woken).toHaveBeenCalledWith('srv-1'));
+ expect(service.getStatus('vm-1').upgrade).toBeNull();
+ });
+
+ describe('looking again at a stopped server when its page opens', () => {
+ /** Booted onto a stack another Console stopped while this one watched. */
+ async function stoppedByOthers() {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+ inspectStack.mockResolvedValue(present(false));
+ service.recheck('vm-1');
+ await vi.waitFor(() => expect(service.getStatus('vm-1').phase).toBe('stopped'));
+ expect(service.getStatus('vm-1').notice).toMatch(/stopped outside this Console/);
+ // Past the interval the re-check above started.
+ vi.spyOn(Date, 'now').mockReturnValue(Date.now() + 60_000);
+ lockEvents.length = 0;
+ return service;
+ }
+
+ afterEach(() => vi.restoreAllMocks());
+
+ it('picks up a stack someone has started again', async () => {
+ const service = await stoppedByOthers();
+ inspectStack.mockResolvedValue(present(true));
+
+ service.refresh('vm-1');
+
+ await vi.waitFor(() => expect(service.getStatus('vm-1').phase).toBe('running'));
+ expect(service.getStatus('vm-1').notice).toBeNull();
+ });
+
+ it('leaves a stack still stopped as the page shows it, saying who stopped it', async () => {
+ const service = await stoppedByOthers();
+
+ service.refresh('vm-1');
+ await vi.waitFor(() => expect(inspectStack).toHaveBeenCalledTimes(3));
+ await new Promise((resolve) => setTimeout(resolve, 10));
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'stopped',
+ notice: expect.stringMatching(/stopped outside this Console/),
+ });
+ // A look, not a check: no lock taken.
+ expect(lockEvents).toEqual([]);
+ });
+
+ it('says so when someone has removed it', async () => {
+ const service = await stoppedByOthers();
+ inspectStack.mockResolvedValue({ kind: 'absent' });
+
+ service.refresh('vm-1');
+
+ await vi.waitFor(() =>
+ expect(service.getStatus('vm-1').notice).toMatch(/Nothing is set up on vm-1 any more/)
+ );
+ });
+
+ it('does nothing for a server shown running, which its own requests keep an eye on', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+ createRemoteServerHost.mockClear();
+
+ service.refresh('vm-1');
+
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ });
+ });
+
+ it('says why a running server could not be taken up at launch, instead of showing it stopped', async () => {
+ const host = fakeHost();
+ host.establishNetworking.mockRejectedValueOnce(new Error('Port 41000 is already in use'));
+ createRemoteServerHost.mockResolvedValue(host);
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1').notice).toBe(
+ 'The server on vm-1 is running, but this Console could not connect to it: ' +
+ 'Port 41000 is already in use'
+ );
+ });
+
+ it('says a host could not be asked at launch', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'unreadable', reason: 'ssh: connection refused' });
+ const service = await loadService();
+
+ await boot(service);
+
+ expect(service.getStatus('vm-1')).toMatchObject({
+ phase: 'stopped',
+ notice: 'Could not check the server on vm-1: ssh: connection refused',
+ });
+ });
+
+ it('keeps a running server whose settings cannot be read now, and the forward that reaches it', async () => {
+ const live = fakeHost();
+ createRemoteServerHost.mockResolvedValue(live);
+ inspectStack.mockResolvedValue(present(true));
+ const service = await loadService();
+ await boot(service);
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: '',
+ running: true,
+ });
+
+ service.recheck('vm-1');
+
+ await vi.waitFor(() =>
+ expect(service.getStatus('vm-1').notice).toBe(
+ "The server's settings on vm-1 are missing JWT_SECRET_KEY."
+ )
+ );
+ expect(service.getStatus('vm-1').phase).toBe('running');
+ expect(live.dispose).not.toHaveBeenCalled();
+ });
+
+ it('does not report a server this Console reset itself as removed by someone', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue({ kind: 'absent' });
+ const service = await loadService();
+ await service.reset('vm-1');
+ vi.spyOn(Date, 'now').mockReturnValue(Date.now() + 60_000);
+
+ service.refresh('vm-1');
+ await vi.waitFor(() => expect(inspectStack).toHaveBeenCalled());
+ await new Promise((resolve) => setTimeout(resolve, 10));
+
+ expect(service.getStatus('vm-1').notice).toBeNull();
+ vi.restoreAllMocks();
+ });
+
+ it('records that it still uses a server at most daily, across launches too', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ inspectStack.mockResolvedValue(present(true));
+ await boot(await loadService());
+ expect(writeRecord).toHaveBeenCalledOnce();
+
+ // The next launch: a new service, the same store.
+ await boot(await loadService());
+
+ expect(writeRecord).toHaveBeenCalledOnce();
+ });
+
+ it('passes a start’s steps, log lines and update on to the page', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ startStack.mockImplementationOnce(async (opts: StartStackOptionsLike) => {
+ opts.onMessage('Pulling images…');
+ opts.onLog('pulled switch-core');
+ opts.onUpgrade({ from: '0.27.0', to: '0.28.0' });
+ expect(service.getStatus('vm-1')).toMatchObject({
+ message: 'Pulling images…',
+ upgrade: { state: 'updating', from: '0.27.0', to: '0.28.0' },
+ });
+ return { kind: 'started', serverId: 'srv-1', telemetryEnabled: false, warning: null };
+ });
+ const service = await loadService();
+
+ await service.start('vm-1', 'Team server');
+
+ expect(emitted).toContainEqual({ sshHost: 'vm-1', line: 'pulled switch-core' });
+ });
+
+ it('says what a record that failed with something other than an Error said', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ writeRecord.mockRejectedValueOnce('volume is read-only');
+ const service = await loadService();
+
+ await service.stop('vm-1');
+
+ expect(service.getStatus('vm-1').recordWarning).toMatch(/: volume is read-only$/);
+ });
+
+ it('reports a join that failed with something other than an Error', async () => {
+ createRemoteServerHost.mockResolvedValue(fakeHost());
+ connectStack.mockRejectedValueOnce('channel closed');
+ const service = await loadService();
+
+ expect(await service.connect('vm-1', 'Team server')).toEqual({
+ kind: 'error',
+ message: 'channel closed',
+ });
+ });
+
+ it('keeps a running stack, saying why, when a re-check fails with something other than an Error', async () => {
+ const service = await running();
+ createRemoteServerHost.mockRejectedValueOnce('ssh gone');
+
+ service.recheck('vm-1');
+
+ await vi.waitFor(() =>
+ expect(service.getStatus('vm-1').notice).toBe('Could not check the server on vm-1: ssh gone')
+ );
+ expect(service.getStatus('vm-1').phase).toBe('running');
+ });
+
+ it('does not re-check a server this Console has since forgotten', async () => {
+ const service = await running();
+ createRemoteServerHost.mockClear();
+ listManagedServers.mockResolvedValue([]);
+
+ service.recheck('vm-1');
+ await vi.waitFor(() => expect(listManagedServers).toHaveBeenCalled());
+ await new Promise((resolve) => setTimeout(resolve, 10));
+
+ expect(createRemoteServerHost).not.toHaveBeenCalled();
+ });
+
+ it('survives a re-check that cannot even list the servers', async () => {
+ const service = await running();
+ listManagedServers.mockRejectedValueOnce(new Error('database is locked'));
+
+ service.recheck('vm-1');
+ await new Promise((resolve) => setTimeout(resolve, 10));
+
+ expect(service.getStatus('vm-1').phase).toBe('running');
+ });
+});
+
+type StartStackOptionsLike = {
+ onMessage: (message: string) => void;
+ onLog: (line: string) => void;
+ onUpgrade: (owed: { from: string; to: string }) => void;
+};
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
index 459be044c..fa440e22e 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.ts
@@ -1,43 +1,101 @@
import type { HostReachabilityChange } from '@main/core/remote-hosts/host-reachability-service';
import { hostReachabilityService } from '@main/core/remote-hosts/production-host-reachability';
+import { getConsoleIdentity } from '@main/core/switch-servers/console-identity';
import { deleteAgentsForServer } from '@main/core/switch-servers/delete-server-agents';
import {
+ ensureManagedServer,
getRemoteManagedServer,
listManagedServers,
+ removeServer,
} from '@main/core/switch-servers/servers-store';
import {
reportManagedServerOutcome,
reportManagedServerStart,
reportManagedServerStartThrew,
} from '@main/core/telemetry/managed-server';
+import { KV } from '@main/db/kv';
import { events } from '@main/lib/events';
import { log } from '@main/lib/logger';
import { COMPATIBLE_SWITCH_VERSION } from '@shared/app-identity';
import {
+ type ConnectRemoteServerResult,
type DockerAvailability,
- type StartLocalServerResult,
+ type RemoteStackProbe,
+ type ServerLockAction,
+ type ServerLockHolder,
+ type StackActivityAction,
+ type StackRegister,
+ type StartRemoteServerResult,
managedServerUpgradeBlockedReason,
matrixMigrationFailedMessage,
+ othersRecentlySeen,
+ stoppedWaitingForLockMessage,
switchVersionDowngradeMessage,
+ waitingForLockMessage,
} from '@shared/core/managed-switch-server/managed-switch-server';
import {
type RemoteServerStatus,
remoteServerLogChannel,
remoteServerStatusChannel,
} from '@shared/events/remoteSwitchServerEvents';
-import { isStackRunning } from './compose';
+import { hostAccount, readRegister, writeRecord } from './console-register';
import { readVersionStatus } from './deployed-version';
+import { apiUrlFor, gatewayUrlFor, type LocalServerPorts } from './free-port';
import { createRemoteServerHost, type RemoteServerHost } from './host/remote-host';
+import { hostSlug, remoteSecretsKey } from './host/remote-identity';
import {
owedUpgrade,
readUpgradeJournal,
type UpgradeJournal,
upgradeState,
} from './managed-upgrade';
-import { resetStack, startStack, stopStack } from './pipeline';
-import { readPersistedPorts } from './ports';
+import { remoteServerStateDir } from './paths';
+import {
+ adoptRunningStack,
+ bringWorkingDirInStep,
+ type ConnectStackResult,
+ connectStack,
+ resetStack,
+ startStack,
+ stopStack,
+} from './pipeline';
+import { clearPorts } from './ports';
+import { clearSecrets } from './secrets';
+import {
+ acquireServerLock,
+ CONSOLE_INSTANCE,
+ type LockClaim,
+ readServerLock,
+ SERVER_LOCK_TIMING,
+ type ServerLease,
+ ServerLockWaitCancelled,
+} from './stack-lock';
+import {
+ inspectStack,
+ probeFromStack,
+ stateVolumeExists,
+ type StackOnHost,
+ type StackStateHost,
+ unsharedStackMessage,
+} from './stack-state';
import { readDeployedTelemetry } from './telemetry-consent';
+/** Minimum gap between re-reads of a stack that stopped answering: each is an
+ * SSH round trip plus `docker` calls, prompted by failing requests that burst. */
+const RECHECK_INTERVAL_MS = 30_000;
+
+/** How often picking a stack back up also records this Console as a user, so an
+ * idle Console stays inside the register's two-week window without a host write
+ * at every launch. */
+const SIGHTING_INTERVAL_MS = 24 * 60 * 60 * 1000;
+
+/** When this Console last got a record onto each host, by alias, across launches. */
+const sightings = new KV<Record<string, string>>('remote-server-sightings');
+
+/** How long leaving waits to take this Console off a host's register. Leaving
+ * is local; the record is a courtesy to the others and not worth a hang. */
+const LEAVE_RECORD_TIMEOUT_MS = 20_000;
+
function initialStatus(sshHost: string): RemoteServerStatus {
return {
sshHost,
@@ -53,37 +111,105 @@ function initialStatus(sshHost: string): RemoteServerStatus {
deployedTelemetry: null,
message: null,
error: null,
+ notice: null,
+ recordWarning: null,
+ waitingFor: null,
};
}
+const RECORDED_AS: Record<StackActivityAction, string> = {
+ started: 'started it',
+ connected: 'connected to it',
+ stopped: 'stopped it',
+ reset: 'reset it',
+ disconnected: 'disconnected from it',
+};
+
+function recordWarningFor(
+ hostLabel: string,
+ action: StackActivityAction | null,
+ reason: string
+): string {
+ if (action === null) {
+ return (
+ `Could not record on ${hostLabel} that this Console uses the server, so others may not ` +
+ `see it among its users: ${reason}`
+ );
+ }
+ return (
+ `This Console could not record on ${hostLabel} that it ${RECORDED_AS[action]}, so others ` +
+ `using the server will not see that in its activity: ${reason}`
+ );
+}
+
+/** The notice for a stack found not running, or null. `wasRunning` means this
+ * Console last saw it up, so it stopped under us and needs explaining. */
+function noticeForIdleStack(
+ hostLabel: string,
+ stack: Exclude<StackOnHost, { kind: 'unreadable' }>,
+ wasRunning: boolean,
+ removedHere: boolean
+): string | null {
+ switch (stack.kind) {
+ case 'present':
+ return wasRunning
+ ? `The server on ${hostLabel} was stopped outside this Console — from another Console, or on the host.`
+ : null;
+ case 'absent':
+ if (removedHere) return null;
+ return (
+ `Nothing is set up on ${hostLabel} any more: the server was removed. Its activity says ` +
+ `by whom. Starting it sets up a new, empty one.`
+ );
+ case 'unshared':
+ return unsharedStackMessage(hostLabel, stack.ownerDir);
+ case 'incomplete':
+ return `The server's settings on ${hostLabel} are missing ${stack.missing.join(', ')}.`;
+ }
+}
+
/**
* Supervises Switch Console-managed Switch stacks on remote hosts (one per SSH
* alias), via the shared {@link startStack} pipeline on a {@link
* RemoteServerHost}. Unlike the local service, a started host is KEPT ALIVE in
* `hosts` because it owns the persistent port-forward that makes the stack
- * reachable from the desktop; it is disposed only on stop/reset/quit. The
- * containers themselves run detached, so a remote stack (and its remote-host
- * agents) stays up while Switch Console is closed — only the desktop-side forward
- * goes away.
+ * reachable from the desktop; it is disposed only on stop/reset/disconnect/
+ * quit. The containers themselves run detached, so a remote stack (and its
+ * remote-host agents) stays up while Switch Console is closed — only the
+ * desktop-side forward goes away.
*
* A host whose stack is behind this build's switch-core pin is upgraded when it
* is reconciled — at boot, and whenever the host becomes reachable again — and
* {@link ensureReady} holds sessions for its agents until that has finished.
+ *
+ * A remote stack is shared by everyone with access to its host, so its state is
+ * read from the host (at launch, on reconnect, and when it stops answering)
+ * rather than remembered.
*/
export class RemoteServerService {
private readonly statuses = new Map<string, RemoteServerStatus>();
private readonly hosts = new Map<string, RemoteServerHost>();
private readonly busy = new Set<string>();
private readonly startAborts = new Map<string, AbortController>();
+ /** The wait for another Console's lock in flight per host, which
+ * {@link cancelWait} ends. */
+ private readonly lockWaits = new Map<string, AbortController>();
private initialization: Promise<void> | null = null;
- /** The reconcile or start in flight per host, which {@link ensureReady} waits out. */
+ /** The reconcile, start or connect in flight per host, which {@link ensureReady} waits out. */
private readonly operations = new Map<string, Promise<void>>();
/** The start in flight per host, which an automatic upgrade and a Start click share. */
- private readonly starting = new Map<string, Promise<StartLocalServerResult>>();
+ private readonly starting = new Map<string, Promise<StartRemoteServerResult>>();
private readonly upgradeListeners = new Set<(serverId: string) => void>();
/** Hosts {@link ensureReady} has turned something away for since their last
* upgrade finished, so that it can be told when they are ready. */
private readonly refused = new Set<string>();
+ private readonly lastRecheck = new Map<string, number>();
+ /** When this Console last got a record onto each host, as far as this run
+ * knows; {@link sightings} carries it across launches. */
+ private readonly lastRecorded = new Map<string, number>();
+ /** Hosts whose stack this Console reset, until it starts or joins one there
+ * again: finding nothing on them is no news to report. */
+ private readonly removedHere = new Set<string>();
getStatuses(): RemoteServerStatus[] {
return [...this.statuses.values()];
@@ -99,6 +225,73 @@ export class RemoteServerService {
events.emit(remoteServerStatusChannel, next);
}
+ /** Record this Console, and what it did, on the stack's host. A failed write
+ * does not fail the operation it describes; it shows on the server page until
+ * a later record succeeds. */
+ private async record(
+ sshHost: string,
+ host: StackStateHost,
+ action: StackActivityAction | null
+ ): Promise<void> {
+ try {
+ await writeRecord(host, action);
+ this.lastRecorded.set(sshHost, Date.now());
+ await sightings.set(sshHost, new Date().toISOString());
+ this.setStatus(sshHost, { recordWarning: null });
+ } catch (error) {
+ const reason = error instanceof Error ? error.message : String(error);
+ log.warn(
+ `remote-switch-server: could not record ${action ?? 'this Console'} on ${host.label}`,
+ {
+ error,
+ }
+ );
+ this.setStatus(sshHost, { recordWarning: recordWarningFor(host.label, action, reason) });
+ }
+ }
+
+ private async claimFor(host: RemoteServerHost, action: ServerLockAction): Promise<LockClaim> {
+ const identity = await getConsoleIdentity();
+ return {
+ consoleId: identity.id,
+ instance: CONSOLE_INSTANCE,
+ name: identity.name,
+ hostAccount: await hostAccount(host),
+ action,
+ };
+ }
+
+ /** Take the stack's lock for `action`, waiting for whoever holds it. The wait
+ * shows on the status; {@link cancelWait} ends it with {@link ServerLockWaitCancelled}. */
+ private async waitForLock(
+ sshHost: string,
+ host: RemoteServerHost,
+ action: ServerLockAction
+ ): Promise<ServerLease> {
+ const abort = new AbortController();
+ this.lockWaits.set(sshHost, abort);
+ try {
+ return await acquireServerLock(host, await this.claimFor(host, action), {
+ mode: 'wait',
+ timing: SERVER_LOCK_TIMING,
+ signal: abort.signal,
+ onWaiting: (holder) =>
+ this.setStatus(sshHost, { waitingFor: holder, message: waitingForLockMessage(holder) }),
+ });
+ } finally {
+ this.lockWaits.delete(sshHost);
+ if (this.getStatus(sshHost).waitingFor !== null) {
+ this.setStatus(sshHost, { waitingFor: null, message: null });
+ }
+ }
+ }
+
+ /** Stop waiting for another Console's lock on `sshHost`. What was waiting —
+ * a start, a join, a check — ends having changed nothing. */
+ cancelWait(sshHost: string): void {
+ this.lockWaits.get(sshHost)?.abort();
+ }
+
async detectDocker(sshHost: string): Promise<DockerAvailability> {
hostReachabilityService.requireReachable(sshHost);
const host = await createRemoteServerHost(sshHost);
@@ -109,6 +302,51 @@ export class RemoteServerService {
}
}
+ /**
+ * What `sshHost` has of a stack, so the UI can offer the one action that is
+ * safe there: Connect to a running one, Start a stopped or absent one, or
+ * neither for one this account cannot read. Reads only.
+ */
+ async probe(sshHost: string): Promise<RemoteStackProbe> {
+ hostReachabilityService.requireReachable(sshHost);
+ const host = await createRemoteServerHost(sshHost);
+ try {
+ const docker = await host.detectDocker();
+ if (!docker.available) {
+ return { kind: 'docker-unavailable', reason: docker.reason, detail: docker.detail };
+ }
+ const [stack, busy] = await Promise.all([inspectStack(host), this.lockHolder(host)]);
+ return probeFromStack(host.label, stack, busy);
+ } finally {
+ host.dispose();
+ }
+ }
+
+ /** Who holds the stack's lock, for the setup step to show. Only a hint (whatever
+ * it offers takes the lock itself), so a failed read is logged and shown as nobody. */
+ private async lockHolder(host: RemoteServerHost): Promise<ServerLockHolder | null> {
+ try {
+ return await readServerLock(host);
+ } catch (error) {
+ log.warn(`remote-switch-server: could not read the server lock on ${host.label}`, { error });
+ return null;
+ }
+ }
+
+ /** Who uses the stack on `sshHost` and what they last did to it, as the
+ * Consoles sharing it have recorded on the host. Reads only. */
+ async register(sshHost: string): Promise<StackRegister> {
+ hostReachabilityService.requireReachable(sshHost);
+ const live = this.hosts.get(sshHost);
+ if (live) return readRegister(live);
+ const host = await createRemoteServerHost(sshHost);
+ try {
+ return await readRegister(host);
+ } finally {
+ host.dispose();
+ }
+ }
+
/** Re-establish forwards + status for remote stacks that survived the last
* quit, so their desktop reachability is restored on launch, and upgrade the
* ones that are behind. Hosts reconcile independently and in the background;
@@ -196,27 +434,113 @@ export class RemoteServerService {
await this.reconcileHost(sshHost, server);
}
- /** Adopt an already-running remote stack: re-open its forward and mark it
- * running. Skipped while the host is blocked — the reachability manager will
- * call back through {@link onHostReachable} when it recovers.
+ /** Re-read a running stack that stopped answering, at most once per
+ * {@link RECHECK_INTERVAL_MS}. Fire-and-forget; the outcome arrives as a status. */
+ recheck(sshHost: string): void {
+ this.lookAgain(sshHost, 'running', async (server) => {
+ log.info(`remote-switch-server: ${sshHost} stopped answering; reading the host again`);
+ await this.reconcileHost(sshHost, server);
+ });
+ }
+
+ /**
+ * Re-read a stack shown as stopped when its page opens: it gets no requests, so
+ * nothing else would notice another Console starting or removing it. Reconciled
+ * only if it changed, so a still-stopped stack keeps its notice. Rate-limited
+ * with {@link recheck}.
+ */
+ refresh(sshHost: string): void {
+ this.lookAgain(sshHost, 'stopped', async (server) => {
+ const host = await createRemoteServerHost(sshHost);
+ let stack: StackOnHost;
+ try {
+ stack = await inspectStack(host);
+ } finally {
+ host.dispose();
+ }
+ const unchanged = stack.kind === 'unreadable' || (stack.kind === 'present' && !stack.running);
+ if (unchanged || this.busy.has(sshHost)) return;
+ log.info(`remote-switch-server: ${sshHost} changed while this Console showed it stopped`);
+ await this.reconcileHost(sshHost, server);
+ });
+ }
+
+ /** Run `look` for a host this Console shows in `phase`, at most once per
+ * {@link RECHECK_INTERVAL_MS} and never beside another operation on it. */
+ private lookAgain(
+ sshHost: string,
+ phase: 'running' | 'stopped',
+ look: (server: { id: string; name: string }) => Promise<void>
+ ): void {
+ if (this.busy.has(sshHost)) return;
+ if (this.getStatus(sshHost).phase !== phase) return;
+ if (hostReachabilityService.isBlocked(sshHost)) return;
+ const now = Date.now();
+ if (now - (this.lastRecheck.get(sshHost) ?? 0) < RECHECK_INTERVAL_MS) return;
+ this.lastRecheck.set(sshHost, now);
+ void this.track(sshHost, async () => {
+ const server = (await this.remoteHosts()).get(sshHost);
+ if (!server || this.busy.has(sshHost)) return;
+ await look(server);
+ }).catch((error: unknown) => {
+ log.warn(`remote-switch-server: re-check failed for ${sshHost}`, { error });
+ });
+ }
+
+ /**
+ * Read the host and take its stack up as it is: a running stack is adopted with
+ * the host's settings (following its ports if they moved), one not running is
+ * shown as stopped. Skipped while the host is blocked — the reachability
+ * manager calls back through {@link onHostReachable} when it recovers.
*
* Also records how the host's deployed switch-core compares to this build's
* pin (CHOO-1736). A running stack that is behind, or one whose last upgrade
- * was interrupted, is upgraded instead of adopted; a stopped one that is
- * behind is marked to be upgraded at its next start. The check runs even
- * when the stack is down: its data volumes still hold the schema the last
- * version migrated to, which is what makes a downgrade unsafe. */
+ * from this account was interrupted, is upgraded instead of adopted; a
+ * stopped one that is behind is marked to be upgraded at its next start. The
+ * check runs even when the stack is down: its data volumes still hold the
+ * schema the last version migrated to, which is what makes a downgrade unsafe.
+ */
private async reconcileHost(
sshHost: string,
server: { id: string; name: string }
): Promise<void> {
if (hostReachabilityService.isBlocked(sshHost)) return;
+ // Checked here too, with no await before the add: callers check before
+ // awaiting the server list, and a Start clicked in that gap must not run beside this.
+ if (this.busy.has(sshHost)) return;
+ const wasRunning = this.getStatus(sshHost).phase === 'running';
+ this.busy.add(sshHost);
+ // The live forward stays until the host answers: a failed read says nothing
+ // about the stack, and dropping the forward would strand a server still up.
+ const live = this.hosts.get(sshHost) ?? null;
let host: RemoteServerHost | null = null;
- let adopted = false;
+ let lease: ServerLease | null = null;
+ let kept = false;
let upgradeNow = false;
+ let seenRunning = false;
try {
host = await createRemoteServerHost(sshHost);
- const running = await isStackRunning(host);
+ // Read only once nobody else is changing the stack: one half-way through
+ // someone's start or update would read as stopped, or as behind.
+ lease = await this.waitForLock(sshHost, host, 'checking');
+ const stack = await inspectStack(host);
+ if (stack.kind === 'unreadable') {
+ this.leaveUnanswered(sshHost, stack.reason, false);
+ return;
+ }
+ seenRunning = stack.kind !== 'absent' && stack.running;
+ // A stopped stack's version is read from this account's `.env`, stale once
+ // another account has updated it; refresh it from the published copy, best-effort.
+ if (stack.kind === 'present' && !stack.running) {
+ await bringWorkingDirInStep(host, stack).catch((error: unknown) => {
+ log.warn(
+ `remote-switch-server: could not refresh this account's settings on ${sshHost}`,
+ {
+ error,
+ }
+ );
+ });
+ }
const version = await readVersionStatus(host, COMPATIBLE_SWITCH_VERSION);
let journal: UpgradeJournal | null;
try {
@@ -234,31 +558,82 @@ export class RemoteServerService {
return;
}
const owed = owedUpgrade(version.drift, journal);
- upgradeNow = owed !== null && (running || journal !== null);
- if (running && !upgradeNow) {
- const ports = await readPersistedPorts(host);
- if (ports) {
- await host.establishNetworking(ports);
+ // Only a stack this account can start from the host's own settings: an
+ // upgrade is a start, and a start is refused for anything else.
+ upgradeNow = owed !== null && stack.kind === 'present' && (stack.running || journal !== null);
+ // Updating restarts the stack for everyone, so one others used lately waits
+ // for someone here to run it. An update this account already started is
+ // resumed: the stack is half-migrated.
+ const held = upgradeNow && journal === null && (await this.othersUseIt(sshHost, host));
+ if (held) upgradeNow = false;
+ if (upgradeNow) {
+ // The start replaces this Console's forward, if it holds one.
+ } else if (stack.kind === 'present' && stack.running) {
+ const settings = await adoptRunningStack(host, stack, lease);
+ const moved = await this.followPorts(sshHost, server.id, settings.ports);
+ if (!live || moved) {
+ this.releaseHost(sshHost, live);
+ await host.establishNetworking(settings.ports);
this.hosts.set(sshHost, host);
- adopted = true;
- this.setStatus(sshHost, { phase: 'running', serverId: server.id });
- // Only for a running stack: a stopped one sends nothing, so it
- // cannot be out of step with the user's answer.
- this.setStatus(sshHost, { deployedTelemetry: await readDeployedTelemetry(host) });
+ kept = true;
+ }
+ this.setStatus(sshHost, {
+ phase: 'running',
+ serverId: server.id,
+ error: null,
+ notice: null,
+ });
+ // Brought up to date by someone else since this Console turned sessions
+ // away for the update: they can run now.
+ if (owed === null) this.releaseRefused(sshHost, server.id);
+ // Only for a running stack: a stopped one sends nothing, so it
+ // cannot be out of step with the user's answer.
+ this.setStatus(sshHost, { deployedTelemetry: await readDeployedTelemetry(host) });
+ if (Date.now() - (await this.lastSighting(sshHost)) >= SIGHTING_INTERVAL_MS) {
+ await this.record(sshHost, host, null);
}
- // Running but we don't know its ports — leave it stopped; the user can
- // restart to re-derive them rather than forward to the wrong ports.
+ } else if (stack.kind !== 'present' && stack.kind !== 'absent' && stack.running && live) {
+ // Up and reached through the forward already held, but its settings are
+ // unreadable from here: keep what works and say why.
+ this.setStatus(sshHost, {
+ phase: 'running',
+ serverId: server.id,
+ notice: noticeForIdleStack(host.label, stack, wasRunning, false),
+ });
+ } else {
+ this.releaseHost(sshHost, live);
+ this.setStatus(sshHost, {
+ phase: 'stopped',
+ serverId: server.id,
+ deployedTelemetry: null,
+ notice: noticeForIdleStack(host.label, stack, wasRunning, this.removedHere.has(sshHost)),
+ });
}
- this.setStatus(sshHost, { ...version, upgrade: owed && upgradeState(owed, upgradeNow) });
+ this.setStatus(sshHost, {
+ ...version,
+ upgrade: owed && (held ? { state: 'held', ...owed } : upgradeState(owed, upgradeNow)),
+ });
} catch (error) {
- log.warn(`remote-switch-server: boot reconcile failed for ${sshHost}`, { error });
+ if (error instanceof ServerLockWaitCancelled) {
+ this.setStatus(sshHost, { notice: stoppedWaitingForLockMessage(error.holder) });
+ return;
+ }
+ log.warn(`remote-switch-server: reconcile failed for ${sshHost}`, { error });
+ this.leaveUnanswered(
+ sshHost,
+ error instanceof Error ? error.message : String(error),
+ seenRunning
+ );
} finally {
- // The adopted host owns the live port-forward; anything else is throwaway.
- if (!adopted) host?.dispose();
+ // Before the host goes: giving the lock back runs over its connection.
+ await lease?.release();
+ // A host that became the live one owns its forward; any other is throwaway.
+ if (!kept) host?.dispose();
+ this.busy.delete(sshHost);
}
if (!upgradeNow) return;
try {
- await this.beginStart(sshHost, server.name, false);
+ await this.beginStart(sshHost, server.name, false, 'updating');
} catch (error) {
// Only the reachability check throws rather than reporting a result.
log.warn(`remote-switch-server: could not upgrade ${sshHost}`, { error });
@@ -266,20 +641,81 @@ export class RemoteServerService {
}
}
+ /** Whether other Consoles have used the stack lately, per its register. An
+ * unreadable register counts as yes, so an update is asked about, not forced. */
+ private async othersUseIt(sshHost: string, host: RemoteServerHost): Promise<boolean> {
+ try {
+ return othersRecentlySeen(await readRegister(host), new Date()).length > 0;
+ } catch (error) {
+ log.warn(`remote-switch-server: could not read who uses the server on ${sshHost}`, {
+ error,
+ });
+ return true;
+ }
+ }
+
+ /** The host could not be read, or its stack not taken up. That is not news
+ * about the stack, so the phase and forward stay as they were and only the
+ * notice says why. `seenRunning`: the stack read as running before the failure. */
+ private leaveUnanswered(sshHost: string, reason: string, seenRunning: boolean): void {
+ log.warn(`remote-switch-server: could not check the stack on ${sshHost}`, { reason });
+ this.setStatus(sshHost, {
+ notice: seenRunning
+ ? `The server on ${sshHost} is running, but this Console could not connect to it: ${reason}`
+ : `Could not check the server on ${sshHost}: ${reason}`,
+ });
+ }
+
+ /** When this Console last got a record onto `sshHost`: this run's, else the
+ * one a previous launch kept. */
+ private async lastSighting(sshHost: string): Promise<number> {
+ const inRun = this.lastRecorded.get(sshHost);
+ if (inRun !== undefined) return inRun;
+ const kept = await sightings.get(sshHost);
+ const at = kept === null ? Number.NaN : Date.parse(kept);
+ return Number.isFinite(at) ? at : 0;
+ }
+
+ /** Point the server's record at the ports the stack publishes, if another
+ * Console restarted it on different ones. Returns whether they moved. */
+ private async followPorts(
+ sshHost: string,
+ serverId: string,
+ ports: LocalServerPorts
+ ): Promise<boolean> {
+ const gatewayUrl = gatewayUrlFor(ports);
+ const apiUrl = apiUrlFor(ports);
+ const record = await getRemoteManagedServer(sshHost);
+ if (!record || (record.gatewayUrl === gatewayUrl && record.apiUrl === apiUrl)) return false;
+ log.info(`remote-switch-server: the stack on ${sshHost} now publishes different ports`, {
+ serverId,
+ from: record.gatewayUrl,
+ to: gatewayUrl,
+ });
+ await ensureManagedServer(
+ { name: record.name, gatewayUrl, apiUrl },
+ { kind: 'remote', sshHost }
+ );
+ return true;
+ }
+
/** Start (or restart) the host's stack at this build's pin, upgrading it
* first if it is behind. Joins a start already in flight for the host. */
- start(sshHost: string, serverName: string): Promise<StartLocalServerResult> {
- return this.track(sshHost, () => this.beginStart(sshHost, serverName, true));
+ start(sshHost: string, serverName: string): Promise<StartRemoteServerResult> {
+ return this.track(sshHost, () => this.beginStart(sshHost, serverName, true, 'starting'));
}
+ /** `action` is what the others waiting on the lock are told this Console is
+ * doing: an update when it is run to bring the stack up to date. */
private beginStart(
sshHost: string,
serverName: string,
- activate: boolean
- ): Promise<StartLocalServerResult> {
+ activate: boolean,
+ action: 'starting' | 'updating'
+ ): Promise<StartRemoteServerResult> {
const inFlight = this.starting.get(sshHost);
if (inFlight) return inFlight;
- const run = this.runStart(sshHost, serverName, activate).finally(() => {
+ const run = this.runStart(sshHost, serverName, activate, action).finally(() => {
this.starting.delete(sshHost);
});
this.starting.set(sshHost, run);
@@ -289,8 +725,9 @@ export class RemoteServerService {
private async runStart(
sshHost: string,
serverName: string,
- activate: boolean
- ): Promise<StartLocalServerResult> {
+ activate: boolean,
+ action: 'starting' | 'updating'
+ ): Promise<StartRemoteServerResult> {
if (this.busy.has(sshHost)) {
return { kind: 'error', message: `An operation is already in progress for ${sshHost}.` };
}
@@ -298,29 +735,40 @@ export class RemoteServerService {
this.busy.add(sshHost);
const abort = new AbortController();
this.startAborts.set(sshHost, abort);
- // Replace any prior live host (and its forward) for this alias.
- this.hosts.get(sshHost)?.dispose();
- this.hosts.delete(sshHost);
+ const before = this.getStatus(sshHost);
let host: RemoteServerHost | null = null;
try {
this.setStatus(sshHost, {
phase: 'starting',
error: null,
+ notice: null,
message: `Connecting to ${sshHost}…`,
});
host = await createRemoteServerHost(sshHost);
+ const lease = await this.waitForLock(sshHost, host, action);
+ // Replace any prior live host (and its forward) for this alias — only
+ // now that the start goes ahead, so a cancelled wait leaves it as it was.
+ this.hosts.get(sshHost)?.dispose();
+ this.hosts.delete(sshHost);
this.setStatus(sshHost, { message: 'Checking Docker…' });
- const result = await startStack({
- host,
- ref: { kind: 'remote', sshHost },
- serverName,
- activate,
- onMessage: (message) => this.setStatus(sshHost, { message }),
- onLog: (line) => events.emit(remoteServerLogChannel, { sshHost, line }),
- onUpgrade: (owed) => this.setStatus(sshHost, { upgrade: upgradeState(owed, true) }),
- signal: abort.signal,
- checkoutRoot: null,
- });
+ let result: Awaited<ReturnType<typeof startStack>>;
+ try {
+ result = await startStack({
+ host,
+ ref: { kind: 'remote', sshHost },
+ serverName,
+ activate,
+ onMessage: (message) => this.setStatus(sshHost, { message }),
+ onLog: (line) => events.emit(remoteServerLogChannel, { sshHost, line }),
+ onUpgrade: (owed) => this.setStatus(sshHost, { upgrade: upgradeState(owed, true) }),
+ signal: abort.signal,
+ checkoutRoot: null,
+ lease,
+ });
+ } finally {
+ // Before anything below can dispose of the host it runs over.
+ await lease.release();
+ }
if (result.kind === 'docker-unavailable') {
this.setStatus(sshHost, { phase: 'error', error: result.detail });
this.failUpgrade(sshHost, result.detail);
@@ -362,15 +810,25 @@ export class RemoteServerService {
drift: null,
upgrade: null,
deployedTelemetry: { known: true, enabled: result.telemetryEnabled },
+ notice: result.warning,
});
- if (this.refused.delete(sshHost)) {
- for (const listener of this.upgradeListeners) listener(result.serverId);
- }
+ this.releaseRefused(sshHost, result.serverId);
+ this.removedHere.delete(sshHost);
+ await this.record(sshHost, host, 'started');
}
reportManagedServerStart('remote', result);
return result;
} catch (error) {
host?.dispose();
+ if (error instanceof ServerLockWaitCancelled) {
+ this.setStatus(sshHost, {
+ phase: before.phase,
+ message: null,
+ error: before.error,
+ notice: before.notice,
+ });
+ return { kind: 'cancelled' };
+ }
const message = error instanceof Error ? error.message : String(error);
log.error(`remote-switch-server: start failed for ${sshHost}`, { error });
this.setStatus(sshHost, { phase: 'error', error: message });
@@ -383,6 +841,13 @@ export class RemoteServerService {
}
}
+ /** Tell those {@link ensureReady} turned away for `sshHost` that its stack
+ * is at this build's pin now. */
+ private releaseRefused(sshHost: string, serverId: string): void {
+ if (!this.refused.delete(sshHost)) return;
+ for (const listener of this.upgradeListeners) listener(serverId);
+ }
+
/** Record why a host's upgrade did not finish. A start that was not an
* upgrade has nothing to record here; its error is on the status already. */
private failUpgrade(sshHost: string, error: string): void {
@@ -393,6 +858,209 @@ export class RemoteServerService {
});
}
+ /** Join the stack already running on `sshHost` without changing anything on the
+ * host (see {@link connectStack}). The host is kept on success: it owns the forward. */
+ connect(sshHost: string, serverName: string): Promise<ConnectRemoteServerResult> {
+ return this.track(sshHost, async () => {
+ const result = await this.runConnect(sshHost, serverName);
+ return result.kind === 'behind' ? this.connectByUpdating(sshHost, serverName) : result;
+ });
+ }
+
+ private async runConnect(
+ sshHost: string,
+ serverName: string
+ ): Promise<ConnectStackResult | { kind: 'cancelled' }> {
+ if (this.busy.has(sshHost)) {
+ return { kind: 'error', message: `An operation is already in progress for ${sshHost}.` };
+ }
+ hostReachabilityService.requireReachable(sshHost);
+ this.busy.add(sshHost);
+ const abort = new AbortController();
+ this.startAborts.set(sshHost, abort);
+ const before = this.getStatus(sshHost);
+ let host: RemoteServerHost | null = null;
+ try {
+ this.setStatus(sshHost, {
+ phase: 'starting',
+ error: null,
+ notice: null,
+ message: `Connecting to ${sshHost}…`,
+ });
+ host = await createRemoteServerHost(sshHost);
+ const lease = await this.waitForLock(sshHost, host, 'connecting');
+ this.releaseHost(sshHost, this.hosts.get(sshHost) ?? null);
+ let result: ConnectStackResult;
+ try {
+ result = await connectStack({
+ host,
+ ref: { kind: 'remote', sshHost },
+ serverName,
+ onMessage: (message) => this.setStatus(sshHost, { message }),
+ signal: abort.signal,
+ lease,
+ });
+ } finally {
+ // Joining gives it back as soon as it has read the stack; this is for
+ // every way out before that.
+ await lease.release();
+ }
+ if (result.kind === 'connected') {
+ this.hosts.set(sshHost, host);
+ // Joined only at this build's pin, so nothing is owed any more.
+ this.setStatus(sshHost, {
+ phase: 'running',
+ serverId: result.serverId,
+ message: null,
+ error: null,
+ upgrade: null,
+ });
+ // Sessions and watchers turned away while an update was owed can run
+ // now; the ones waiting out that update carry on by themselves.
+ this.releaseRefused(sshHost, result.serverId);
+ this.setStatus(sshHost, await readVersionStatus(host, COMPATIBLE_SWITCH_VERSION));
+ this.setStatus(sshHost, { deployedTelemetry: await readDeployedTelemetry(host) });
+ this.removedHere.delete(sshHost);
+ await this.record(sshHost, host, 'connected');
+ return result;
+ }
+ host.dispose();
+ if (result.kind === 'behind') {
+ // Back to where it was: the update that follows is a start of its own,
+ // which takes this as the state to return to if its wait is cancelled.
+ this.setStatus(sshHost, {
+ phase: before.phase,
+ message: `Updating the server from switch-core ${result.deployed} to ${result.expected}…`,
+ });
+ } else if (result.kind === 'not-running' || result.kind === 'absent') {
+ this.setStatus(sshHost, { phase: 'stopped', message: null });
+ } else if (result.kind === 'docker-unavailable') {
+ this.setStatus(sshHost, { phase: 'error', message: null, error: result.detail });
+ } else {
+ this.setStatus(sshHost, { phase: 'error', message: null, error: result.message });
+ }
+ return result;
+ } catch (error) {
+ host?.dispose();
+ if (error instanceof ServerLockWaitCancelled) {
+ this.setStatus(sshHost, {
+ phase: before.phase,
+ message: null,
+ error: before.error,
+ notice: before.notice,
+ });
+ return { kind: 'cancelled' };
+ }
+ const message = error instanceof Error ? error.message : String(error);
+ log.error(`remote-switch-server: connect failed for ${sshHost}`, { error });
+ this.setStatus(sshHost, { phase: 'error', message: null, error: message });
+ return { kind: 'error', message };
+ } finally {
+ this.busy.delete(sshHost);
+ this.startAborts.delete(sshHost);
+ }
+ }
+
+ /** Join a stack behind this build's switch-core pin by updating it: a start from
+ * the stack's own settings, and so an update for everyone using it. */
+ private async connectByUpdating(
+ sshHost: string,
+ serverName: string
+ ): Promise<ConnectRemoteServerResult> {
+ const result = await this.beginStart(sshHost, serverName, true, 'updating');
+ switch (result.kind) {
+ case 'started':
+ return {
+ kind: 'connected',
+ serverId: result.serverId,
+ deployedVersion: COMPATIBLE_SWITCH_VERSION,
+ };
+ case 'docker-unavailable':
+ case 'cancelled':
+ case 'error':
+ return result;
+ case 'version-downgrade':
+ return {
+ kind: 'error',
+ message: switchVersionDowngradeMessage(result.deployed, result.expected),
+ };
+ case 'matrix-migration-failed':
+ return {
+ kind: 'error',
+ message: matrixMigrationFailedMessage(result.deployed, result.expected),
+ };
+ }
+ }
+
+ /**
+ * Stop using the stack on `sshHost` from this Console, leaving it running for
+ * everyone else: close the forward, remove the server record (its agents are
+ * unlinked and kept) and drop the local credentials.
+ */
+ async disconnect(sshHost: string): Promise<void> {
+ if (this.busy.has(sshHost))
+ throw new Error(`An operation is already in progress for ${sshHost}.`);
+ this.busy.add(sshHost);
+ try {
+ // Recorded when the host is reachable: a Console left listed counts as a
+ // user for weeks, holding the others' updates. Leaving must not wait on or
+ // fail for the host, and a failure is only logged: the page goes with it.
+ const live = this.hosts.get(sshHost) ?? null;
+ if (!hostReachabilityService.isBlocked(sshHost)) await this.recordLeaving(sshHost, live);
+ this.releaseHost(sshHost, live);
+ const server = await getRemoteManagedServer(sshHost);
+ if (server) await removeServer(server.id);
+ await clearSecrets({ secretsKey: remoteSecretsKey(sshHost) });
+ await clearPorts({ stateDir: remoteServerStateDir(hostSlug(sshHost)) });
+ this.setStatus(sshHost, initialStatus(sshHost));
+ this.statuses.delete(sshHost);
+ this.lastRecheck.delete(sshHost);
+ this.lastRecorded.delete(sshHost);
+ this.removedHere.delete(sshHost);
+ await sightings.del(sshHost);
+ } finally {
+ this.busy.delete(sshHost);
+ }
+ }
+
+ /** Record this Console leaving on `sshHost`'s register, given up after
+ * {@link LEAVE_RECORD_TIMEOUT_MS}. Never creates a register where there is none. */
+ private async recordLeaving(sshHost: string, live: RemoteServerHost | null): Promise<void> {
+ const opened: { host: RemoteServerHost | null; abandoned: boolean } = {
+ host: null,
+ abandoned: false,
+ };
+ const record = async () => {
+ let host = live;
+ if (!host) {
+ host = await createRemoteServerHost(sshHost);
+ // Opened after leaving gave up on it: nothing else will close it.
+ if (opened.abandoned) {
+ host.dispose();
+ return;
+ }
+ opened.host = host;
+ }
+ if (await stateVolumeExists(host)) await writeRecord(host, 'disconnected');
+ };
+ let timer: ReturnType<typeof setTimeout> | undefined;
+ const timeout = new Promise<never>((_, reject) => {
+ timer = setTimeout(
+ () => reject(new Error(`no answer within ${LEAVE_RECORD_TIMEOUT_MS / 1000}s`)),
+ LEAVE_RECORD_TIMEOUT_MS
+ );
+ });
+ try {
+ await Promise.race([record(), timeout]);
+ } catch (error) {
+ log.warn(`remote-switch-server: could not record disconnected on ${sshHost}`, { error });
+ } finally {
+ clearTimeout(timer);
+ opened.abandoned = true;
+ opened.host?.dispose();
+ }
+ }
+
async stop(sshHost: string): Promise<void> {
if (this.busy.has(sshHost))
throw new Error(`An operation is already in progress for ${sshHost}.`);
@@ -402,21 +1070,32 @@ export class RemoteServerService {
// therefore sit inside the region that reports, and inside the one that
// gives the busy flag back.
let host: RemoteServerHost | null = null;
+ let lease: ServerLease | null = null;
try {
hostReachabilityService.requireReachable(sshHost);
host = this.hosts.get(sshHost) ?? (await createRemoteServerHost(sshHost));
+ // Refused rather than waited for: stopping a server someone else just
+ // started is not what the click asked for.
+ lease = await this.lockOrRefuse(sshHost, host, 'stopping');
this.setStatus(sshHost, { phase: 'stopping', message: 'Stopping containers…' });
- await stopStack(host);
+ await stopStack(host, lease);
const upgrade = this.getStatus(sshHost).upgrade;
this.setStatus(sshHost, {
phase: 'stopped',
message: null,
error: null,
+ notice: null,
deployedTelemetry: null,
upgrade: upgrade && upgradeState(upgrade, false),
});
+ await this.record(sshHost, host, 'stopped');
reportManagedServerOutcome('stop', 'remote', 'success');
} catch (error) {
+ if (lease === null && host !== null) {
+ // The lock was not had, so nothing was touched: see lockOrRefuse.
+ host = null;
+ throw error;
+ }
this.setStatus(sshHost, {
phase: 'error',
error: error instanceof Error ? error.message : String(error),
@@ -424,6 +1103,7 @@ export class RemoteServerService {
reportManagedServerOutcome('stop', 'remote', 'failure');
throw error;
} finally {
+ await lease?.release();
this.releaseHost(sshHost, host);
this.busy.delete(sshHost);
}
@@ -441,25 +1121,38 @@ export class RemoteServerService {
// Reaching the host is inside the reported region for the reason `stop`
// gives.
let host: RemoteServerHost | null = null;
+ let lease: ServerLease | null = null;
try {
hostReachabilityService.requireReachable(sshHost);
host = this.hosts.get(sshHost) ?? (await createRemoteServerHost(sshHost));
+ // Taken before the agents go, and refused rather than waited for, as for
+ // a stop: a reset turned away must not have deleted anything first.
+ lease = await this.lockOrRefuse(sshHost, host, 'resetting');
this.setStatus(sshHost, { phase: 'stopping', message: 'Removing agents…' });
const server = await getRemoteManagedServer(sshHost);
if (server) await deleteAgentsForServer(server.id);
this.setStatus(sshHost, { phase: 'stopping', message: 'Destroying containers and data…' });
- await resetStack(host);
+ await resetStack(host, lease);
this.setStatus(sshHost, {
phase: 'stopped',
message: null,
error: null,
+ notice: null,
deployedTelemetry: null,
deployedVersion: null,
drift: null,
upgrade: null,
});
+ this.removedHere.add(sshHost);
+ // Kept through the reset on purpose: who destroyed a shared server is
+ // exactly what its other users will ask.
+ await this.record(sshHost, host, 'reset');
reportManagedServerOutcome('reset', 'remote', 'success');
} catch (error) {
+ if (lease === null && host !== null) {
+ host = null;
+ throw error;
+ }
this.setStatus(sshHost, {
phase: 'error',
error: error instanceof Error ? error.message : String(error),
@@ -467,18 +1160,19 @@ export class RemoteServerService {
reportManagedServerOutcome('reset', 'remote', 'failure');
throw error;
} finally {
+ await lease?.release();
this.releaseHost(sshHost, host);
this.busy.delete(sshHost);
}
}
/**
- * Drop the host a teardown used, and the map entry it may have come from.
+ * Drop the host an operation used, and the map entry it may have come from.
*
- * Null when the teardown never got one — the alias is then left alone rather
- * than un-keyed, because an entry dropped without being disposed strands the
- * port-forward it owns and lets the reachability reconciler adopt the same
- * stack a second time.
+ * Null when the operation never got one — the alias is then left alone
+ * rather than un-keyed, because an entry dropped without being disposed
+ * strands the port-forward it owns and lets the reachability reconciler adopt
+ * the same stack a second time.
*/
private releaseHost(sshHost: string, host: RemoteServerHost | null): void {
if (!host) return;
@@ -486,11 +1180,35 @@ export class RemoteServerService {
this.hosts.delete(sshHost);
}
- /** Abort in-flight health waits and drop all forwards (app quit). The remote
- * containers keep running; only the desktop-side tunnels close. */
+ /**
+ * Take the lock for a stop or reset, refusing rather than waiting. On failure
+ * nothing was touched, so a live forward stays; a host opened just for this is
+ * disposed here, and the caller must not release it again.
+ */
+ private async lockOrRefuse(
+ sshHost: string,
+ host: RemoteServerHost,
+ action: 'stopping' | 'resetting'
+ ): Promise<ServerLease> {
+ try {
+ return await acquireServerLock(host, await this.claimFor(host, action), {
+ mode: 'refuse',
+ timing: SERVER_LOCK_TIMING,
+ });
+ } catch (error) {
+ if (host !== this.hosts.get(sshHost)) host.dispose();
+ throw error;
+ }
+ }
+
+ /** Abort in-flight health waits and lock waits and drop all forwards (app
+ * quit). The remote containers keep running; only the desktop-side tunnels
+ * close. */
dispose(): void {
for (const abort of this.startAborts.values()) abort.abort();
this.startAborts.clear();
+ for (const abort of this.lockWaits.values()) abort.abort();
+ this.lockWaits.clear();
for (const host of this.hosts.values()) host.dispose();
this.hosts.clear();
}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts
new file mode 100644
index 000000000..7c3c4e46a
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/remote-server-service.upgrade.test.ts
@@ -0,0 +1,464 @@
+import { beforeEach, expect, it, vi } from 'vitest';
+import type * as AppIdentity from '@shared/app-identity';
+import type { StartLocalServerResult } from '@shared/core/managed-switch-server/managed-switch-server';
+import type * as ManagedUpgrade from './managed-upgrade';
+import type { StartStackOptions } from './pipeline';
+import type * as StackLock from './stack-lock';
+import type { StackOnHost } from './stack-state';
+import type * as StackState from './stack-state';
+
+/**
+ * Upgrading remote stacks to this build's switch-core pin, and holding sessions
+ * until that has finished. Shared stacks are covered in remote-server-service.test.ts.
+ */
+
+type Change = { current: { sshHost: string; status: string } };
+const m = vi.hoisted(() => {
+ const listeners: ((change: Change) => void)[] = [];
+ const blocked = new Set<string>();
+ return {
+ listeners,
+ blocked,
+ reachability: {
+ on(_event: string, listener: (change: Change) => void) {
+ listeners.push(listener);
+ },
+ isBlocked: (sshHost: string) => blocked.has(sshHost),
+ requireReachable(sshHost: string) {
+ if (blocked.has(sshHost)) throw new Error(`${sshHost} is unreachable`);
+ },
+ },
+ servers: vi.fn(),
+ inspect: vi.fn<() => Promise<StackOnHost>>(),
+ adopt: vi.fn(),
+ version: vi.fn(),
+ journal: vi.fn(),
+ establish: vi.fn(),
+ register: vi.fn(),
+ connect: vi.fn(),
+ start: vi.fn<(opts: StartStackOptions) => Promise<StartLocalServerResult>>(),
+ };
+});
+vi.mock('@shared/app-identity', async (importOriginal) => ({
+ ...(await importOriginal<typeof AppIdentity>()),
+ COMPATIBLE_SWITCH_VERSION: '0.11.0',
+}));
+vi.mock('@main/core/remote-hosts/production-host-reachability', () => ({
+ hostReachabilityService: m.reachability,
+}));
+vi.mock('@main/lib/events', () => ({ events: { emit: vi.fn() } }));
+vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), error: vi.fn(), info: vi.fn() } }));
+vi.mock('@main/core/telemetry/managed-server', () => ({
+ reportManagedServerOutcome: vi.fn(),
+ reportManagedServerStart: vi.fn(),
+ reportManagedServerStartThrew: vi.fn(),
+}));
+vi.mock('@main/core/switch-servers/delete-server-agents', () => ({
+ deleteAgentsForServer: vi.fn(),
+}));
+vi.mock('@main/core/switch-servers/servers-store', () => ({
+ listManagedServers: m.servers,
+ getRemoteManagedServer: vi.fn(),
+ ensureManagedServer: vi.fn(),
+ removeServer: vi.fn(),
+}));
+vi.mock('./stack-state', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackState>()),
+ inspectStack: m.inspect,
+ stateVolumeExists: async () => true,
+}));
+vi.mock('./console-register', () => ({
+ writeRecord: vi.fn(),
+ readRegister: m.register,
+ hostAccount: async () => 'me',
+}));
+
+/** The server lock, recording takes and releases in order. The real module's
+ * errors and constants are kept. */
+const lockEvents = vi.hoisted(() => [] as string[]);
+const acquireServerLock = vi.hoisted(() =>
+ vi.fn(async (_host: unknown, claim: { action: string }, _opts: unknown) => {
+ lockEvents.push(`take ${claim.action}`);
+ return {
+ token: 'lease-token',
+ lost: false,
+ assertHeld: vi.fn(async () => {}),
+ release: vi.fn(async () => {
+ lockEvents.push(`release ${claim.action}`);
+ }),
+ };
+ })
+);
+const readServerLock = vi.hoisted(() => vi.fn(async () => null));
+vi.mock('./stack-lock', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackLock>()),
+ acquireServerLock,
+ readServerLock,
+}));
+const kvStore = vi.hoisted(() => new Map<string, unknown>());
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ constructor(private readonly namespace: string) {}
+ async get(key: string) {
+ return kvStore.get(`${this.namespace}:${key}`) ?? null;
+ }
+ async set(key: string, value: unknown) {
+ kvStore.set(`${this.namespace}:${key}`, value);
+ }
+ async del(key: string) {
+ kvStore.delete(`${this.namespace}:${key}`);
+ }
+ },
+}));
+vi.mock('@main/core/switch-servers/console-identity', () => ({
+ getConsoleIdentity: async () => ({
+ id: 'aaaaaaaa-0000-4000-8000-000000000001',
+ name: 'me@laptop',
+ }),
+}));
+
+vi.mock('./paths', () => ({ remoteServerStateDir: (slug: string) => `/user-data/remote/${slug}` }));
+vi.mock('./secrets', () => ({ clearSecrets: vi.fn() }));
+vi.mock('./ports', () => ({ clearPorts: vi.fn() }));
+vi.mock('./deployed-version', () => ({ readVersionStatus: m.version }));
+vi.mock('./telemetry-consent', () => ({ readDeployedTelemetry: vi.fn() }));
+vi.mock('./managed-upgrade', async (importOriginal) => ({
+ ...(await importOriginal<typeof ManagedUpgrade>()),
+ readUpgradeJournal: m.journal,
+}));
+vi.mock('./pipeline', () => ({
+ bringWorkingDirInStep: async (
+ host: { writeFile: (...args: unknown[]) => Promise<void> },
+ stack: { source: string; raw: string }
+ ) => {
+ if (stack.source === 'published') await host.writeFile('.env', stack.raw, 0o600);
+ },
+ startStack: m.start,
+ adoptRunningStack: m.adopt,
+ connectStack: m.connect,
+ stopStack: vi.fn(),
+ resetStack: vi.fn(),
+}));
+vi.mock('./host/remote-host', () => ({
+ createRemoteServerHost: async (sshHost: string) => ({
+ sshHost,
+ label: sshHost,
+ establishNetworking: m.establish,
+ writeFile: vi.fn(async () => {}),
+ dispose: vi.fn(),
+ }),
+}));
+
+const { RemoteServerService } = await import('./remote-server-service');
+
+const behind = {
+ deployedVersion: '0.10.0',
+ drift: { deployed: '0.10.0', expected: '0.11.0', direction: 'upgrade' },
+};
+const inStep = { deployedVersion: '0.11.0', drift: null };
+const ports = { gateway: 1, api: 2, mattermost: 3, postgres: 4 };
+
+function present(running: boolean): StackOnHost {
+ return {
+ kind: 'present',
+ env: { ports, secrets: {} as never, version: '0.10.0' },
+ raw: 'PUBLISHED\n',
+ source: 'published',
+ running,
+ published: true,
+ runningVersion: null,
+ stamp: null,
+ };
+}
+
+function startedOn(serverId: string): StartLocalServerResult {
+ return { kind: 'started', serverId, telemetryEnabled: false, warning: null };
+}
+
+/** A start that runs the upgrade the way the pipeline does: announce, then work. */
+function upgradingStart(result?: StartLocalServerResult) {
+ return async (opts: StartStackOptions) => {
+ opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
+ return result ?? startedOn(opts.ref.kind === 'remote' ? `srv-${opts.ref.sshHost}` : 'local');
+ };
+}
+
+function announce(sshHost: string, status: string): void {
+ for (const listener of m.listeners) listener({ current: { sshHost, status } });
+}
+
+beforeEach(() => {
+ kvStore.clear();
+ vi.resetAllMocks();
+ m.listeners.length = 0;
+ m.blocked.clear();
+ m.servers.mockResolvedValue([
+ { id: 'srv-builder', name: 'Builder', managementKind: 'remote', sshHost: 'builder' },
+ ]);
+ m.inspect.mockResolvedValue(present(true));
+ m.adopt.mockResolvedValue({ ports, secrets: {} });
+ m.version.mockResolvedValue(behind);
+ m.journal.mockResolvedValue(null);
+ m.register.mockResolvedValue({ self: 'me', consoles: [], activity: [] });
+ m.start.mockImplementation(upgradingStart());
+});
+
+it('upgrades a running remote stack that is behind at boot, instead of adopting it', async () => {
+ const service = new RemoteServerService();
+ await service.initialize();
+ await service.ensureReady('builder', 'Builder');
+
+ expect(m.start).toHaveBeenCalledOnce();
+ expect(m.start).toHaveBeenCalledWith(
+ expect.objectContaining({
+ ref: { kind: 'remote', sshHost: 'builder' },
+ serverName: 'Builder',
+ activate: false,
+ })
+ );
+ expect(m.establish).not.toHaveBeenCalled();
+ expect(service.getStatus('builder')).toMatchObject({
+ phase: 'running',
+ serverId: 'srv-builder',
+ upgrade: null,
+ drift: null,
+ });
+});
+
+it('holds sessions for the host while its upgrade runs', async () => {
+ let finish: (result: StartLocalServerResult) => void = () => {};
+ m.start.mockImplementation(
+ (opts) =>
+ new Promise((resolve) => {
+ opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
+ finish = resolve;
+ })
+ );
+ const service = new RemoteServerService();
+ void service.initialize();
+ let ready = false;
+ const session = service.ensureReady('builder', 'Builder').then(() => {
+ ready = true;
+ });
+
+ await vi.waitFor(() => expect(m.start).toHaveBeenCalledOnce());
+ expect(service.getStatus('builder').upgrade).toEqual({
+ state: 'updating',
+ from: '0.10.0',
+ to: '0.11.0',
+ });
+ expect(ready).toBe(false);
+ finish(startedOn('srv-builder'));
+ await session;
+ expect(ready).toBe(true);
+});
+
+it('adopts a running stack that is in step without restarting it', async () => {
+ m.version.mockResolvedValue(inStep);
+ const service = new RemoteServerService();
+ await service.initialize();
+ await service.ensureReady('builder', 'Builder');
+
+ expect(m.start).not.toHaveBeenCalled();
+ expect(m.establish).toHaveBeenCalledOnce();
+ expect(service.getStatus('builder')).toMatchObject({
+ phase: 'running',
+ serverId: 'srv-builder',
+ upgrade: null,
+ });
+});
+
+it('upgrades a stopped stack that is behind at its next start, not before', async () => {
+ m.inspect.mockResolvedValue(present(false));
+ const service = new RemoteServerService();
+ const upgraded = vi.fn();
+ service.onUpgradeFinished(upgraded);
+ await service.initialize();
+
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
+ /Builder is stopped on switch-core 0\.10\.0/
+ );
+ expect(m.start).not.toHaveBeenCalled();
+
+ expect(await service.start('builder', 'Builder')).toMatchObject({ kind: 'started' });
+ expect(m.start).toHaveBeenCalledWith(expect.objectContaining({ activate: true }));
+ await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
+ expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+});
+
+it('upgrades a host that was unreachable at boot once it comes back, holding sessions meanwhile', async () => {
+ m.blocked.add('builder');
+ const service = new RemoteServerService();
+ await service.initialize();
+ await service.ensureReady('builder', 'Builder');
+ expect(m.start).not.toHaveBeenCalled();
+
+ let finish: (result: StartLocalServerResult) => void = () => {};
+ m.start.mockImplementation(
+ (opts) =>
+ new Promise((resolve) => {
+ opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
+ finish = resolve;
+ })
+ );
+ m.blocked.delete('builder');
+ announce('builder', 'reachable');
+ // Asked straight after the recovery, as a watcher restoring its controller does.
+ let ready = false;
+ const session = service.ensureReady('builder', 'Builder').then(() => {
+ ready = true;
+ });
+
+ await vi.waitFor(() => expect(m.start).toHaveBeenCalledOnce());
+ expect(ready).toBe(false);
+ finish(startedOn('srv-builder'));
+ await session;
+ expect(service.getStatus('builder').upgrade).toBeNull();
+});
+
+it('surfaces a failed upgrade with its error and retries when the host is reconciled again', async () => {
+ m.start.mockImplementationOnce(
+ upgradingStart({ kind: 'error', message: 'The server did not become healthy in time.' })
+ );
+ const service = new RemoteServerService();
+ const upgraded = vi.fn();
+ service.onUpgradeFinished(upgraded);
+ await service.initialize();
+
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
+ /Updating Builder from switch-core 0\.10\.0 to 0\.11\.0 failed: The server did not become healthy/
+ );
+ expect(service.getStatus('builder')).toMatchObject({
+ phase: 'error',
+ upgrade: { state: 'failed', error: 'The server did not become healthy in time.' },
+ });
+
+ announce('builder', 'reachable');
+ await vi.waitFor(() => expect(m.start).toHaveBeenCalledTimes(2));
+ await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
+ expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+});
+
+it('does not hold one host’s sessions behind another host’s upgrade', async () => {
+ m.servers.mockResolvedValue([
+ { id: 'srv-builder', name: 'Builder', managementKind: 'remote', sshHost: 'builder' },
+ { id: 'srv-other', name: 'Other', managementKind: 'remote', sshHost: 'other' },
+ ]);
+ m.start.mockImplementation(
+ (opts) =>
+ new Promise(() => {
+ opts.onUpgrade({ from: '0.10.0', to: '0.11.0' });
+ })
+ );
+ m.version.mockImplementation(async (host: { sshHost: string }) =>
+ host.sshHost === 'builder' ? behind : inStep
+ );
+ const service = new RemoteServerService();
+ await service.initialize();
+
+ await expect(service.ensureReady('other', 'Other')).resolves.toBeUndefined();
+ expect(service.getStatus('builder').upgrade?.state).toBe('updating');
+});
+
+it('refuses sessions when the host cannot be reached for its upgrade, rather than hanging', async () => {
+ const service = new RemoteServerService();
+ m.inspect.mockImplementation(async () => {
+ // The host drops between the probe and the start.
+ m.blocked.add('builder');
+ return present(true);
+ });
+ await service.initialize();
+
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
+ /failed: builder is unreachable/
+ );
+});
+
+/** A register in which someone else last used the server `daysAgo` days ago. */
+function usedByBob(daysAgo: number) {
+ return {
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.36.0',
+ lastSeenAt: new Date(Date.now() - daysAgo * 24 * 60 * 60 * 1000).toISOString(),
+ },
+ ],
+ activity: [],
+ };
+}
+
+it('holds the update of a running server others used lately, until someone here runs it', async () => {
+ m.register.mockResolvedValue(usedByBob(1));
+ const service = new RemoteServerService();
+ const upgraded = vi.fn();
+ service.onUpgradeFinished(upgraded);
+ await service.initialize();
+
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(
+ /Builder runs switch-core 0\.10\.0 .* Others use it too/
+ );
+ expect(m.start).not.toHaveBeenCalled();
+ // Still reachable from here while it waits: only sessions are held.
+ expect(m.establish).toHaveBeenCalledOnce();
+ expect(service.getStatus('builder')).toMatchObject({
+ phase: 'running',
+ upgrade: { state: 'held', from: '0.10.0', to: '0.11.0' },
+ });
+
+ expect(await service.start('builder', 'Builder')).toMatchObject({ kind: 'started' });
+ await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
+ expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+});
+
+it('updates on its own when nobody else has used the server for over two weeks', async () => {
+ m.register.mockResolvedValue(usedByBob(15));
+ const service = new RemoteServerService();
+ await service.initialize();
+ await service.ensureReady('builder', 'Builder');
+
+ expect(m.start).toHaveBeenCalledOnce();
+});
+
+it('resumes an update this account already started, whoever else uses the server', async () => {
+ m.register.mockResolvedValue(usedByBob(1));
+ m.journal.mockResolvedValue({ from: '0.10.0', startedAt: '2026-01-01T00:00:00Z' });
+ const service = new RemoteServerService();
+ await service.initialize();
+ await service.ensureReady('builder', 'Builder');
+
+ expect(m.start).toHaveBeenCalledOnce();
+});
+
+it('asks rather than updates when it cannot tell who uses the server', async () => {
+ m.register.mockRejectedValue(new Error('no such volume'));
+ const service = new RemoteServerService();
+ await service.initialize();
+
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(/Others use it too/);
+ expect(m.start).not.toHaveBeenCalled();
+});
+
+it('lets turned-away sessions run once a Connect finds the stack brought up to date elsewhere', async () => {
+ m.inspect.mockResolvedValue(present(false));
+ const service = new RemoteServerService();
+ const upgraded = vi.fn();
+ service.onUpgradeFinished(upgraded);
+ await service.initialize();
+ await expect(service.ensureReady('builder', 'Builder')).rejects.toThrow(/stopped/);
+
+ // Someone else started it at the pin; this Console joins it.
+ m.connect.mockResolvedValue({
+ kind: 'connected',
+ serverId: 'srv-builder',
+ deployedVersion: '0.11.0',
+ });
+ m.version.mockResolvedValue(inStep);
+ await service.connect('builder', 'Builder');
+
+ expect(upgraded).toHaveBeenCalledExactlyOnceWith('srv-builder');
+ await expect(service.ensureReady('builder', 'Builder')).resolves.toBeUndefined();
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts
index f5a924ce5..3398831f6 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secret-values.ts
@@ -38,3 +38,19 @@ export function generateSecrets(): LocalServerSecrets {
mattermostUserPassword: token(),
};
}
+
+/**
+ * Fill in `dbRuntimePassword` for secrets written before switch-core added its
+ * restricted runtime role. Safe where regenerating the bundle is not: nothing on
+ * the existing volume used it, and `init-db` sets the role's password on every start.
+ */
+export function withRuntimePassword(secrets: LocalServerSecrets): {
+ secrets: LocalServerSecrets;
+ migrated: boolean;
+} {
+ if (secrets.dbRuntimePassword) return { secrets, migrated: false };
+ return {
+ secrets: { ...secrets, dbRuntimePassword: generateSecrets().dbRuntimePassword },
+ migrated: true,
+ };
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.test.ts
index 6d4c38a77..1d1629511 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.test.ts
@@ -18,7 +18,7 @@ vi.mock('@main/core/secrets/encrypted-app-secrets-store', () => ({
UndecryptableSecretError: FakeUndecryptableSecretError,
}));
-const { loadOrCreateSecrets, readSecrets } = await import('./secrets');
+const { loadOrCreateSecrets, readSecrets, storeSecrets } = await import('./secrets');
describe('generateSecrets', () => {
it('populates every field with a non-empty value', () => {
@@ -129,3 +129,17 @@ describe('readSecrets', () => {
expect(setSecret).not.toHaveBeenCalled();
});
});
+
+describe('storeSecrets', () => {
+ it('keeps a bundle taken from a shared host as this desktop’s copy, under the host’s key', async () => {
+ setSecret.mockReset();
+ const secrets = generateSecrets();
+
+ await storeSecrets({ secretsKey: 'remote-switch-server:vm-1:secrets' } as never, secrets);
+
+ expect(setSecret).toHaveBeenCalledWith(
+ 'remote-switch-server:vm-1:secrets',
+ JSON.stringify(secrets)
+ );
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.ts
index df47f8d74..6079094a2 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/secrets.ts
@@ -3,7 +3,7 @@ import {
UndecryptableSecretError,
} from '@main/core/secrets/encrypted-app-secrets-store';
import type { ServerHost } from './host/types';
-import { generateSecrets, type LocalServerSecrets } from './secret-values';
+import { generateSecrets, type LocalServerSecrets, withRuntimePassword } from './secret-values';
/**
* A stored bundle that will not parse is not the same as no bundle at all, and
@@ -31,29 +31,6 @@ async function readStoredSecrets(host: Pick<ServerHost, 'secretsKey'>): Promise<
}
}
-/**
- * `dbRuntimePassword` did not exist before switch-core split its database
- * connection into an owner role and a restricted runtime role, so a bundle
- * stored before that split parses as `LocalServerSecrets` with that one field
- * missing. Filling it in is safe in a way regenerating the bundle is not:
- * nothing on the existing Postgres volume was ever created with it, since the
- * runtime role did not exist yet either. `init-db` creates that role and
- * `ALTER ROLE`s its password idempotently on every start (see
- * `deploy/local/standalone-docker-compose.yml`), so whatever value lands here
- * simply becomes the role's password on the next start, the same way a
- * freshly generated bundle's would.
- */
-function withRuntimePassword(secrets: LocalServerSecrets): {
- secrets: LocalServerSecrets;
- migrated: boolean;
-} {
- if (secrets.dbRuntimePassword) return { secrets, migrated: false };
- return {
- secrets: { ...secrets, dbRuntimePassword: generateSecrets().dbRuntimePassword },
- migrated: true,
- };
-}
-
/**
* Return the host's stored secret bundle, generating and persisting one on first
* use. Reusing the same secrets across restarts is required — the Postgres
@@ -105,6 +82,15 @@ export async function readSecrets(
return stored.kind === 'present' ? withRuntimePassword(stored.secrets).secrets : null;
}
+/** Cache `secrets` as this desktop's copy of the host's bundle, replacing any
+ * earlier copy. The host, not this cache, is the source of truth. */
+export async function storeSecrets(
+ host: Pick<ServerHost, 'secretsKey'>,
+ secrets: LocalServerSecrets
+): Promise<void> {
+ await encryptedAppSecretsStore.setSecret(host.secretsKey, JSON.stringify(secrets));
+}
+
/** Drop the host's secret bundle so the next start generates fresh credentials.
* Only safe to call after the data volumes have been destroyed (reset). */
export async function clearSecrets(host: Pick<ServerHost, 'secretsKey'>): Promise<void> {
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.docker.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.docker.test.ts
new file mode 100644
index 000000000..12d5132a1
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.docker.test.ts
@@ -0,0 +1,359 @@
+import { execFile, execFileSync, spawn } from 'node:child_process';
+import { randomBytes } from 'node:crypto';
+import { promisify } from 'node:util';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { ServerBusyError } from '@shared/core/managed-switch-server/managed-switch-server';
+import { STACK_HELPER_IMAGE, STACK_STATE_VOLUME_SUFFIX } from './constants';
+import type { LockClaim, LockTiming, ServerLease } from './stack-lock';
+import type { StackStateHost } from './stack-state';
+
+/**
+ * Only a real volume and helper image show the lock holding: it rests on
+ * `flock` across containers sharing one volume, and on the host's clock.
+ * Skipped where there is no Docker daemon; CI runs it on Linux.
+ */
+
+vi.mock('@main/core/switch-servers/console-identity', () => ({
+ getConsoleIdentity: () =>
+ Promise.resolve({ id: '3f2a9c1e-5b7d-4e8f-a1b2-c3d4e5f6a7b8', name: 'alice@laptop' }),
+}));
+vi.mock('@main/core/app/utils', () => ({ resolveAppVersion: () => Promise.resolve('0.37.0') }));
+vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), error: vi.fn(), info: vi.fn() } }));
+
+const { acquireServerLock, LOCK_SCRIPT, readServerLock, ServerLockLostError } =
+ await import('./stack-lock');
+const { publishEnv, withdrawPublishedEnv, writeStateVolume, readStateVolume, runStateScript } =
+ await import('./stack-state');
+const { RECORD_SCRIPT } = await import('./console-register');
+const { LOCK_LOST_MESSAGE } = await import('./state-mutex');
+
+const run = promisify(execFile);
+
+function dockerAvailable(): boolean {
+ try {
+ execFileSync('docker', ['info', '--format', '{{.ServerVersion}}'], {
+ stdio: 'pipe',
+ timeout: 15_000,
+ });
+ return true;
+ } catch {
+ return false;
+ }
+}
+
+const DOCKER = dockerAvailable();
+
+function localHost(project: string): StackStateHost {
+ return {
+ label: 'test-host',
+ dockerBin: 'docker',
+ composeProjectName: project,
+ workingDir: '/nonexistent',
+ readFile: () => Promise.resolve(null),
+ ctx: {
+ supportsLocalSpawn: true,
+ exec: (command: string, args: string[] = [], opts: { timeout?: number } = {}) =>
+ run(command, args, { timeout: opts.timeout, maxBuffer: 8 * 1024 * 1024 }),
+ } as unknown as StackStateHost['ctx'],
+ writeCommandInput: (command, args, input, opts) =>
+ new Promise<void>((resolve, reject) => {
+ const child = spawn(command, args, { timeout: opts.timeoutMs });
+ let stderr = '';
+ child.stderr.on('data', (chunk: Buffer) => (stderr += chunk.toString()));
+ child.stdout.resume();
+ child.on('error', reject);
+ child.on('close', (code) =>
+ code === 0 ? resolve() : reject(new Error(`exit ${code}: ${stderr.trim()}`))
+ );
+ child.stdin.end(input);
+ }),
+ };
+}
+
+function claim(overrides: Partial<LockClaim> = {}): LockClaim {
+ return {
+ consoleId: 'aaaaaaaa-0000-4000-8000-000000000001',
+ instance: 'run-1',
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ action: 'starting',
+ ...overrides,
+ };
+}
+
+const bob = (overrides: Partial<LockClaim> = {}) =>
+ claim({
+ consoleId: 'bbbbbbbb-0000-4000-8000-000000000002',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ ...overrides,
+ });
+
+/** No renewals unless a test asks for them, so expiry can be watched. */
+const QUIET: LockTiming = {
+ ttlSeconds: 2,
+ renewEveryMs: 60_000,
+ maxHoldSeconds: 600,
+ pollEveryMs: 200,
+};
+
+const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
+
+describe.skipIf(!DOCKER)('the server lock, against a real state volume', () => {
+ let host: StackStateHost;
+ let project: string;
+ const leases: ServerLease[] = [];
+
+ beforeEach(() => {
+ project = `switch-lock-test-${randomBytes(4).toString('hex')}`;
+ host = localHost(project);
+ });
+
+ afterEach(async () => {
+ for (const lease of leases.splice(0)) await lease.release();
+ await run('docker', ['volume', 'rm', '-f', `${project}_${STACK_STATE_VOLUME_SUFFIX}`]).catch(
+ () => undefined
+ );
+ });
+
+ async function take(c: LockClaim, timing: LockTiming = QUIET): Promise<ServerLease> {
+ const lease = await acquireServerLock(host, c, { mode: 'refuse', timing });
+ leases.push(lease);
+ return lease;
+ }
+
+ it('is taken, refused to someone else naming its holder, and free again once released', async () => {
+ const alice = await take(claim());
+
+ const refused = await take(bob()).catch((error: unknown) => error);
+ expect(refused).toBeInstanceOf(ServerBusyError);
+ expect((refused as InstanceType<typeof ServerBusyError>).holder).toMatchObject({
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ action: 'starting',
+ });
+ expect((refused as Error).message).toMatch(
+ /alice@laptop \(as alice\) is starting the server on test-host right now/
+ );
+
+ await alice.release();
+ await expect(take(bob())).resolves.toBeDefined();
+ }, 60_000);
+
+ it('goes to exactly one of two Consoles taking it at once', async () => {
+ const results = await Promise.allSettled([take(claim()), take(bob())]);
+
+ expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
+ const rejected = results.find((r) => r.status === 'rejected') as PromiseRejectedResult;
+ expect(rejected.reason).toBeInstanceOf(ServerBusyError);
+ }, 60_000);
+
+ it('lets a waiting Console in once the holder releases it, saying who it waited for', async () => {
+ const alice = await take(claim());
+ const onWaiting = vi.fn();
+ const waiting = acquireServerLock(host, bob(), {
+ mode: 'wait',
+ timing: QUIET,
+ signal: new AbortController().signal,
+ onWaiting,
+ });
+
+ await sleep(1000);
+ await alice.release();
+ leases.push(await waiting);
+
+ expect(onWaiting).toHaveBeenCalledWith(expect.objectContaining({ name: 'alice@laptop' }));
+ }, 60_000);
+
+ it('can be taken over once its holder stops renewing it', async () => {
+ await take(claim());
+
+ await sleep(2500);
+
+ await expect(take(bob())).resolves.toBeDefined();
+ }, 60_000);
+
+ it('goes to exactly one of two Consoles taking over the same lapsed lease', async () => {
+ await take(claim());
+ await sleep(2500);
+
+ const results = await Promise.allSettled([
+ take(bob()),
+ take(claim({ consoleId: 'cccccccc-0000-4000-8000-000000000003', name: 'carol@pc' })),
+ ]);
+
+ expect(results.filter((r) => r.status === 'fulfilled')).toHaveLength(1);
+ }, 60_000);
+
+ it('stays held past its lease while its holder renews it', async () => {
+ await take(claim(), { ...QUIET, renewEveryMs: 500 });
+
+ await sleep(3000);
+
+ await expect(take(bob())).rejects.toBeInstanceOf(ServerBusyError);
+ }, 60_000);
+
+ it('cannot be renewed past the longest hold, and then goes to whoever asks', async () => {
+ const alice = await take(claim(), { ...QUIET, renewEveryMs: 500, maxHoldSeconds: 3 });
+
+ await sleep(4000);
+
+ await expect(alice.assertHeld()).rejects.toBeInstanceOf(ServerLockLostError);
+ await expect(take(bob())).resolves.toBeDefined();
+ }, 60_000);
+
+ it('is not released by a holder that lost it, and knows it was lost', async () => {
+ const alice = await take(claim());
+ await sleep(2500);
+ await take(bob());
+
+ await expect(alice.assertHeld()).rejects.toBeInstanceOf(ServerLockLostError);
+ await alice.release();
+
+ await expect(
+ take(claim({ consoleId: 'cccccccc-0000-4000-8000-000000000003' }))
+ ).rejects.toMatchObject({ holder: expect.objectContaining({ name: 'bob@desk' }) });
+ }, 60_000);
+
+ it('is not taken back by a Console that only shares the id, as a restored backup would', async () => {
+ await take(claim({ instance: 'run-before-crash' }));
+
+ await expect(
+ take(claim({ instance: 'other-desk', name: 'alice@other-desk' }))
+ ).rejects.toBeInstanceOf(ServerBusyError);
+ }, 60_000);
+
+ it('is taken over by a take that names it as a lease this run failed to give back', async () => {
+ await take(claim());
+ const args = (op: string, token: string, stale: string) => [
+ op,
+ token,
+ 'aaaaaaaa-0000-4000-8000-000000000001',
+ 'run-1',
+ 'starting',
+ '2',
+ '600',
+ 'alice@laptop',
+ 'alice',
+ stale,
+ ];
+ const held = (await readStateVolume(host, 'sed -n 2p /state/lock')).trim();
+
+ expect(
+ (await runStateScript(host, LOCK_SCRIPT, args('take', 'other', ''))).split('\n')[0]
+ ).toBe('held');
+ expect(
+ (await runStateScript(host, LOCK_SCRIPT, args('take', 'next', `x ${held} y`))).split('\n')[0]
+ ).toBe('taken');
+ }, 60_000);
+
+ it('is taken back at once by the same Console after a restart, but not by the same run', async () => {
+ await take(claim({ instance: 'run-before-crash' }));
+
+ await expect(take(claim({ instance: 'run-before-crash' }))).rejects.toBeInstanceOf(
+ ServerBusyError
+ );
+ await expect(take(claim({ instance: 'run-after-restart' }))).resolves.toBeDefined();
+ }, 60_000);
+
+ it('is only taken under the state mutex', async () => {
+ await writeStateVolume(host, 'true', '', []);
+ const holding = run('docker', [
+ 'run',
+ '--rm',
+ '--volume',
+ `${project}_${STACK_STATE_VOLUME_SUFFIX}:/state`,
+ '--entrypoint',
+ 'sh',
+ STACK_HELPER_IMAGE,
+ '-c',
+ 'exec 9>/state/.mutex; flock 9; touch /state/held; sleep 3; rm /state/held',
+ ]);
+ while ((await readStateVolume(host, '[ -f /state/held ] && echo held; true')) !== 'held\n') {
+ await sleep(100);
+ }
+
+ const started = Date.now();
+ await take(claim());
+
+ expect(Date.now() - started).toBeGreaterThanOrEqual(1500);
+ await holding;
+ }, 60_000);
+
+ it('is not taken when its file cannot be written, so nobody else takes it too', async () => {
+ // A directory in the temp file's place stands in for a full disk.
+ await writeStateVolume(host, 'mkdir /state/.lock.tmp', '', []);
+
+ await expect(take(claim())).rejects.toThrow();
+
+ expect(await readServerLock(host)).toBeNull();
+ }, 60_000);
+
+ it('is read without being changed, and reads as free once released', async () => {
+ expect(await readServerLock(host)).toBeNull();
+ const alice = await take(claim({ action: 'updating' }));
+
+ expect(await readServerLock(host)).toMatchObject({ name: 'alice@laptop', action: 'updating' });
+
+ await alice.release();
+ expect(await readServerLock(host)).toBeNull();
+ }, 60_000);
+
+ it('lets only its holder publish or withdraw the settings', async () => {
+ const alice = await take(claim());
+ await sleep(2500);
+ const bobs = await take(bob());
+
+ await expect(publishEnv(host, 'A=1\n', alice)).rejects.toThrow(LOCK_LOST_MESSAGE);
+ await expect(withdrawPublishedEnv(host, alice)).rejects.toThrow(LOCK_LOST_MESSAGE);
+ await publishEnv(host, 'B=2\n', bobs);
+
+ expect(await readStateVolume(host, 'cat /state/stack.env')).toBe('B=2\n');
+ }, 60_000);
+});
+
+describe.skipIf(!DOCKER)('the state mutex, against a real state volume', () => {
+ let host: StackStateHost;
+ let project: string;
+
+ beforeEach(() => {
+ project = `switch-lock-test-${randomBytes(4).toString('hex')}`;
+ host = localHost(project);
+ });
+
+ afterEach(async () => {
+ await run('docker', ['volume', 'rm', '-f', `${project}_${STACK_STATE_VOLUME_SUFFIX}`]).catch(
+ () => undefined
+ );
+ });
+
+ it('makes a record wait while another script holds the mutex', async () => {
+ // Container start-up jitter dwarfs a trim, so racing real records rarely
+ // shows anything; holding the mutex for a known time does.
+ await writeStateVolume(host, 'true', '', []);
+ const holding = run('docker', [
+ 'run',
+ '--rm',
+ '--volume',
+ `${project}_${STACK_STATE_VOLUME_SUFFIX}:/state`,
+ '--entrypoint',
+ 'sh',
+ STACK_HELPER_IMAGE,
+ '-c',
+ 'exec 9>/state/.mutex; flock 9; touch /state/held; sleep 3; rm /state/held',
+ ]);
+ while ((await readStateVolume(host, '[ -f /state/held ] && echo held; true')) !== 'held\n') {
+ await sleep(100);
+ }
+
+ const started = Date.now();
+ await writeStateVolume(host, RECORD_SCRIPT, '{"entry":1}\n{"line":1}\n', [
+ 'dddddddd-0000-4000-8000-000000000001',
+ 'act',
+ ]);
+
+ expect(Date.now() - started).toBeGreaterThanOrEqual(1500);
+ expect(await readStateVolume(host, '[ -f /state/held ] && echo held; true')).toBe('');
+ await holding;
+ }, 60_000);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.test.ts
new file mode 100644
index 000000000..d6608a773
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.test.ts
@@ -0,0 +1,423 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import type { LockClaim, LockTiming } from './stack-lock';
+import type * as StackState from './stack-state';
+import type { StackStateHost } from './stack-state';
+
+/** The host's answers are scripted here; stack-lock.docker.test.ts runs the
+ * script against a real volume. */
+
+const runStateScript = vi.hoisted(() => vi.fn<(...args: unknown[]) => Promise<string>>());
+const readStateVolume = vi.hoisted(() => vi.fn<(...args: unknown[]) => Promise<string>>());
+const stateVolumeExists = vi.hoisted(() => vi.fn(async () => true));
+const logWarn = vi.hoisted(() => vi.fn());
+const logError = vi.hoisted(() => vi.fn());
+
+vi.mock('@main/lib/logger', () => ({ log: { warn: logWarn, error: logError, info: vi.fn() } }));
+vi.mock('./stack-state', async (importOriginal) => ({
+ ...(await importOriginal<typeof StackState>()),
+ runStateScript,
+ readStateVolume,
+ stateVolumeExists,
+}));
+
+const {
+ acquireServerLock,
+ parseLockReply,
+ readServerLock,
+ ServerLease,
+ ServerLockLostError,
+ ServerLockWaitCancelled,
+} = await import('./stack-lock');
+const { ServerBusyError } =
+ await import('@shared/core/managed-switch-server/managed-switch-server');
+
+const host = { label: 'vm-1' } as StackStateHost;
+
+const claim: LockClaim = {
+ consoleId: 'aaaaaaaa-0000-4000-8000-000000000001',
+ instance: 'run-1',
+ name: 'alice@laptop',
+ hostAccount: 'alice',
+ action: 'starting',
+};
+
+const timing: LockTiming = {
+ ttlSeconds: 120,
+ renewEveryMs: 30_000,
+ maxHoldSeconds: 5400,
+ pollEveryMs: 5_000,
+};
+
+function lockFile(fields: { expires: number; since: number; action?: string; name?: string }) {
+ return [
+ 'switch-console-lock v1',
+ 'their-token',
+ String(fields.expires),
+ String(fields.since),
+ 'bbbbbbbb-0000-4000-8000-000000000002',
+ 'run-9',
+ fields.action ?? 'starting',
+ fields.name ?? 'bob@desk',
+ 'bob',
+ '',
+ ].join('\n');
+}
+
+const reply = (status: string, now: number, file = '') => `${status}\n${now}\n${file}`;
+
+beforeEach(() => {
+ vi.clearAllMocks();
+});
+
+afterEach(() => {
+ vi.useRealTimers();
+});
+
+describe('reading the script’s answer', () => {
+ it('names the holder, with how long it has held the lock and how long it has left', () => {
+ expect(parseLockReply(reply('held', 1000, lockFile({ expires: 1090, since: 960 })))).toEqual({
+ status: 'held',
+ holder: {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 40,
+ expiresInSeconds: 90,
+ live: true,
+ },
+ });
+ });
+
+ it('says a lapsed lock is lapsed, and no holder where there is no lock', () => {
+ expect(
+ parseLockReply(reply('peek', 1000, lockFile({ expires: 990, since: 800 }))).holder
+ ).toMatchObject({ live: false, expiresInSeconds: 0 });
+ expect(parseLockReply(reply('released', 1000))).toEqual({ status: 'released', holder: null });
+ });
+
+ it('makes do with a lock file whose fields it cannot read', () => {
+ const file = [
+ 'switch-console-lock v1',
+ 'their-token',
+ 'soon',
+ 'earlier',
+ 'bbbbbbbb-0000-4000-8000-000000000002',
+ 'run-9',
+ 'starting',
+ '',
+ '',
+ '',
+ ].join('\n');
+
+ expect(parseLockReply(reply('held', 1000, file)).holder).toEqual({
+ name: 'unknown',
+ hostAccount: 'unknown',
+ action: 'starting',
+ heldForSeconds: 0,
+ expiresInSeconds: 0,
+ live: false,
+ });
+ });
+
+ it('refuses an answer it does not understand rather than take the lock for free', () => {
+ expect(() => parseLockReply('sh: flock: not found\n')).toThrow(/Unexpected answer/);
+ expect(() => parseLockReply('taken\nnot-a-clock\n')).toThrow(/Unexpected answer/);
+ });
+
+ it('shows an action from a later Console as a check, and logs it', () => {
+ const { holder } = parseLockReply(
+ reply('held', 1000, lockFile({ expires: 1090, since: 960, action: 'migrating' }))
+ );
+
+ expect(holder?.action).toBe('checking');
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(/action this Console does not know/),
+ { action: 'migrating' }
+ );
+ });
+});
+
+describe('taking the lock', () => {
+ it('passes who is taking it, one line each, and nothing that could split the file', async () => {
+ runStateScript.mockResolvedValueOnce(reply('taken', 1000));
+
+ const lease = await acquireServerLock(
+ host,
+ { ...claim, name: 'alice\n@laptop\t', hostAccount: '' },
+ { mode: 'refuse', timing }
+ );
+ await lease.release();
+
+ const args = runStateScript.mock.calls[0]![2] as string[];
+ expect(args[0]).toBe('take');
+ expect(args.slice(2)).toEqual([
+ claim.consoleId,
+ 'run-1',
+ 'starting',
+ '120',
+ '5400',
+ 'alice @laptop',
+ 'unknown',
+ '',
+ ]);
+ });
+
+ it('refuses a console id that is not one without asking the host', async () => {
+ await expect(
+ acquireServerLock(host, { ...claim, consoleId: '../../etc' }, { mode: 'refuse', timing })
+ ).rejects.toThrow(/not one/);
+ expect(runStateScript).not.toHaveBeenCalled();
+ });
+
+ it('refuses at once, naming the holder, when told not to wait', async () => {
+ runStateScript.mockResolvedValueOnce(
+ reply('held', 1000, lockFile({ expires: 1090, since: 960 }))
+ );
+
+ const error = await acquireServerLock(host, claim, { mode: 'refuse', timing }).catch(
+ (e: unknown) => e
+ );
+
+ expect(error).toBeInstanceOf(ServerBusyError);
+ expect((error as Error).message).toMatch(/^bob@desk \(as bob\) is starting the server on vm-1/);
+ // Only what a person reads goes on to the renderer.
+ expect((error as InstanceType<typeof ServerBusyError>).holder).toEqual({
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 40,
+ expiresInSeconds: 90,
+ });
+ });
+
+ it('waits, saying who for, and takes it once it is free', async () => {
+ vi.useFakeTimers();
+ runStateScript
+ .mockResolvedValueOnce(reply('held', 1000, lockFile({ expires: 1090, since: 960 })))
+ .mockResolvedValueOnce(reply('taken', 1005));
+ const onWaiting = vi.fn();
+
+ const taking = acquireServerLock(host, claim, {
+ mode: 'wait',
+ timing,
+ signal: new AbortController().signal,
+ onWaiting,
+ });
+ await vi.advanceTimersByTimeAsync(timing.pollEveryMs);
+ const lease = await taking;
+ await lease.release();
+
+ expect(onWaiting).toHaveBeenCalledWith({
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 40,
+ expiresInSeconds: 90,
+ });
+ expect(runStateScript.mock.calls.filter((c) => (c[2] as string[])[0] === 'take')).toHaveLength(
+ 2
+ );
+ });
+
+ it('stops waiting when cancelled, saying who it was waiting for', async () => {
+ runStateScript.mockResolvedValue(reply('held', 1000, lockFile({ expires: 1090, since: 960 })));
+ const abort = new AbortController();
+
+ const taking = acquireServerLock(host, claim, {
+ mode: 'wait',
+ timing,
+ signal: abort.signal,
+ onWaiting: () => abort.abort(),
+ });
+
+ const error = await taking.catch((e: unknown) => e);
+ expect(error).toBeInstanceOf(ServerLockWaitCancelled);
+ expect((error as InstanceType<typeof ServerLockWaitCancelled>).holder).toMatchObject({
+ name: 'bob@desk',
+ });
+ });
+
+ it('stops waiting when cancelled in the middle of a wait, without asking again', async () => {
+ vi.useFakeTimers();
+ runStateScript.mockResolvedValue(reply('held', 1000, lockFile({ expires: 1090, since: 960 })));
+ const abort = new AbortController();
+
+ const taking = acquireServerLock(host, claim, {
+ mode: 'wait',
+ timing,
+ signal: abort.signal,
+ onWaiting: vi.fn(),
+ }).catch((e: unknown) => e);
+ await vi.advanceTimersByTimeAsync(timing.pollEveryMs / 2);
+ abort.abort();
+
+ expect(await taking).toBeInstanceOf(ServerLockWaitCancelled);
+ expect(runStateScript).toHaveBeenCalledOnce();
+ });
+
+ it('does not ask at all when the wait was cancelled before it began', async () => {
+ const abort = new AbortController();
+ abort.abort();
+
+ await expect(
+ acquireServerLock(host, claim, {
+ mode: 'wait',
+ timing,
+ signal: abort.signal,
+ onWaiting: vi.fn(),
+ })
+ ).rejects.toBeInstanceOf(ServerLockWaitCancelled);
+ expect(runStateScript).not.toHaveBeenCalled();
+ });
+
+ it('refuses an answer that is neither taken nor held', async () => {
+ runStateScript.mockResolvedValueOnce(reply('renewed', 1000));
+
+ await expect(acquireServerLock(host, claim, { mode: 'refuse', timing })).rejects.toThrow(
+ /Unexpected answer from the server lock on vm-1: renewed/
+ );
+ });
+});
+
+describe('holding the lock', () => {
+ function lease() {
+ return new ServerLease(host, 'my-token', claim, timing);
+ }
+
+ it('renews itself while held, and stops once released', async () => {
+ vi.useFakeTimers();
+ runStateScript.mockResolvedValue(reply('renewed', 1000));
+ const held = lease();
+
+ await vi.advanceTimersByTimeAsync(timing.renewEveryMs * 2);
+ expect(runStateScript.mock.calls.map((c) => (c[2] as string[])[0])).toEqual(['renew', 'renew']);
+
+ await held.release();
+ await vi.advanceTimersByTimeAsync(timing.renewEveryMs * 2);
+ expect(runStateScript.mock.calls.map((c) => (c[2] as string[])[0])).toEqual([
+ 'renew',
+ 'renew',
+ 'release',
+ ]);
+ });
+
+ it('knows once a renewal finds it taken over, and then refuses without asking again', async () => {
+ vi.useFakeTimers();
+ runStateScript.mockResolvedValueOnce(
+ reply('lost', 1000, lockFile({ expires: 1100, since: 990, name: 'carol@pc' }))
+ );
+ const held = lease();
+
+ await vi.advanceTimersByTimeAsync(timing.renewEveryMs);
+
+ expect(held.lost).toBe(true);
+ const error = await held.assertHeld().catch((e: unknown) => e);
+ expect(error).toBeInstanceOf(ServerLockLostError);
+ expect((error as InstanceType<typeof ServerLockLostError>).holder).toMatchObject({
+ name: 'carol@pc',
+ });
+ expect(runStateScript).toHaveBeenCalledOnce();
+ expect(logError).toHaveBeenCalled();
+
+ // Nothing of its own left on the host to give back.
+ await held.release();
+ expect(runStateScript).toHaveBeenCalledOnce();
+ });
+
+ it('keeps it through a renewal that could not reach the host, and says so', async () => {
+ vi.useFakeTimers();
+ runStateScript.mockRejectedValueOnce(new Error('ssh dropped'));
+ const held = lease();
+
+ await vi.advanceTimersByTimeAsync(timing.renewEveryMs);
+
+ expect(held.lost).toBe(false);
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(/could not renew/),
+ expect.anything()
+ );
+ await held.release();
+ });
+
+ it('asks the host when asked whether it still holds it', async () => {
+ runStateScript.mockResolvedValueOnce(reply('renewed', 1000));
+ const held = lease();
+
+ await held.assertHeld();
+
+ expect((runStateScript.mock.calls[0]![2] as string[]).slice(0, 2)).toEqual([
+ 'renew',
+ 'my-token',
+ ]);
+ await held.release();
+ });
+
+ it('counts a lock that was removed outright as lost to nobody', async () => {
+ runStateScript.mockResolvedValueOnce(reply('lost', 1000));
+ const held = lease();
+
+ const error = await held.assertHeld().catch((e: unknown) => e);
+
+ expect(error).toBeInstanceOf(ServerLockLostError);
+ expect((error as InstanceType<typeof ServerLockLostError>).holder).toBeNull();
+ await held.release();
+ });
+
+ it('takes over, rather than waits for, a lease this run failed to give back', async () => {
+ runStateScript.mockRejectedValueOnce(new Error('ssh dropped'));
+ const held = new ServerLease({ label: 'vm-2' } as StackStateHost, 'left-token', claim, timing);
+ await held.release();
+ runStateScript.mockResolvedValueOnce(reply('taken', 1000));
+
+ const next = await acquireServerLock({ label: 'vm-2' } as StackStateHost, claim, {
+ mode: 'refuse',
+ timing,
+ });
+ await next.release();
+
+ const take = runStateScript.mock.calls.find((c) => (c[2] as string[])[0] === 'take')!;
+ expect((take[2] as string[]).at(-1)).toBe('left-token');
+ });
+
+ it('gives it back once however often it is released, and logs a failure to', async () => {
+ runStateScript.mockRejectedValueOnce(new Error('ssh dropped'));
+ const held = lease();
+
+ await held.release();
+ await held.release();
+
+ expect(runStateScript).toHaveBeenCalledOnce();
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(/lapses by itself within 120s/),
+ expect.anything()
+ );
+ await expect(held.assertHeld()).rejects.toThrow(/already released/);
+ });
+});
+
+describe('looking at the lock', () => {
+ it('reads it without creating a volume where there is none', async () => {
+ stateVolumeExists.mockResolvedValueOnce(false);
+
+ expect(await readServerLock(host)).toBeNull();
+ expect(readStateVolume).not.toHaveBeenCalled();
+ });
+
+ it('names a live holder and nobody for a lapsed one', async () => {
+ readStateVolume.mockResolvedValueOnce(
+ reply('peek', 1000, lockFile({ expires: 1090, since: 960 }))
+ );
+ expect(await readServerLock(host)).toEqual({
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 40,
+ expiresInSeconds: 90,
+ });
+
+ readStateVolume.mockResolvedValueOnce(
+ reply('peek', 1000, lockFile({ expires: 999, since: 900 }))
+ );
+ expect(await readServerLock(host)).toBeNull();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.ts
new file mode 100644
index 000000000..a0117afe5
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-lock.ts
@@ -0,0 +1,401 @@
+import { randomBytes, randomUUID } from 'node:crypto';
+import { log } from '@main/lib/logger';
+import {
+ ServerBusyError,
+ type ServerLockAction,
+ type ServerLockHolder,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+import {
+ readStateVolume,
+ runStateScript,
+ type StackStateHost,
+ stateVolumeExists,
+} from './stack-state';
+import {
+ CONSOLE_ID_PATTERN,
+ LOCK_LOST_MESSAGE,
+ SERVER_LOCK_FILE,
+ UNDER_STATE_MUTEX,
+} from './state-mutex';
+
+/**
+ * The lock on a shared remote stack (CHOO-2893), held by whichever Console is
+ * changing it. Without it, two Consoles starting an empty host at once would
+ * both publish credentials, and the database would be created with ones only
+ * one of them holds. It is a lease in the state volume, renewed while its
+ * holder works and expired by the host's clock. A holder may lose it while
+ * away, so credential writes check the token in the same step
+ * (`WHILE_HOLDING_SERVER_LOCK`) and compose runs only after
+ * {@link ServerLease.assertHeld}.
+ */
+
+/** A restarted Console takes back a lease its crashed run left, but waits on
+ * one held by this same run (the same host under a second alias). */
+export const CONSOLE_INSTANCE = randomUUID();
+
+export type LockTiming = {
+ /** How long a lease lasts from its last renewal. */
+ ttlSeconds: number;
+ renewEveryMs: number;
+ /** Caps a lease however often it is renewed, so a Console hung while
+ * renewing cannot keep everyone out. Must exceed a start: a backup may run
+ * 30 minutes and compose 20. */
+ maxHoldSeconds: number;
+ pollEveryMs: number;
+};
+
+export const SERVER_LOCK_TIMING: LockTiming = {
+ ttlSeconds: 120,
+ renewEveryMs: 30_000,
+ maxHoldSeconds: 90 * 60,
+ pollEveryMs: 5_000,
+};
+
+/** Who is taking the lock and why; shown to the others while they wait. */
+export type LockClaim = {
+ consoleId: string;
+ instance: string;
+ name: string;
+ hostAccount: string;
+ action: ServerLockAction;
+};
+
+const LOCK_ACTIONS: readonly ServerLockAction[] = [
+ 'starting',
+ 'updating',
+ 'connecting',
+ 'checking',
+ 'stopping',
+ 'resetting',
+];
+
+/** First line of the lock file. A later format must keep the expiry on the
+ * third line and the token on the second, which is all an older Console reads
+ * of it. */
+const LOCK_MAGIC = 'switch-console-lock v1';
+
+/**
+ * `$10` lists, space-separated, leases this run failed to give back, which it
+ * may take over. Prints a status line, the host's clock, then the lock file,
+ * so a refusal can say who holds it. A failed write exits with an error, not
+ * an answer: "taken" over an empty lock file on a full disk would let the next
+ * Console take it too.
+ */
+export const LOCK_SCRIPT = [
+ 'set -u',
+ 'op=$1 token=$2 console=$3 instance=$4 action=$5 ttl=$6 max=$7 name=$8 account=$9 stale=${10:-}',
+ 'umask 077',
+ UNDER_STATE_MUTEX,
+ `lock=${SERVER_LOCK_FILE}`,
+ 'now=$(date +%s)',
+ 'line() { sed -n "$1p" "$lock" 2>/dev/null; }',
+ 'number() { case $1 in ""|*[!0-9]*) return 1;; esac; }',
+ 'held=$(line 2) expires=$(line 3) since=$(line 4)',
+ 'report() { printf "%s\\n%s\\n" "$1" "$now"; [ -f "$lock" ] && cat "$lock"; exit 0; }',
+ 'until_for() { u=$((now + ttl)); c=$(($1 + max)); [ "$u" -gt "$c" ] && u=$c; echo "$u"; }',
+ 'case $op in',
+ ' take)',
+ ' if [ -f "$lock" ] && number "$expires" && [ "$expires" -gt "$now" ]; then',
+ // Taken over when this Console's crashed previous run left it, or this run
+ // failed to give it back.
+ ' if [ "$(line 5)" = "$console" ] && [ "$(line 6)" != "$instance" ] && [ "$(line 8)" = "$name" ]; then :',
+ ' else case " $stale " in *" $held "*) ;; *) report held ;; esac',
+ ' fi',
+ ' fi',
+ ` printf "%s\\n" "${LOCK_MAGIC}" "$token" "$(until_for "$now")" "$now" "$console" "$instance" "$action" "$name" "$account" > /state/.lock.tmp || exit 1`,
+ ' mv /state/.lock.tmp "$lock" || exit 1',
+ ' report taken ;;',
+ ' renew)',
+ ' [ -n "$held" ] && [ "$held" = "$token" ] && number "$since" || report lost',
+ ' u=$(until_for "$since")',
+ ' [ "$u" -gt "$now" ] || report lost',
+ ' sed "3s/.*/$u/" "$lock" > /state/.lock.tmp || exit 1',
+ ' mv /state/.lock.tmp "$lock" || exit 1',
+ ' report renewed ;;',
+ ' release)',
+ ' if [ -n "$held" ] && [ "$held" = "$token" ]; then rm -f "$lock" || exit 1; report released; fi',
+ ' report other ;;',
+ ' *) echo "unknown lock operation: $op" >&2; exit 2 ;;',
+ 'esac',
+].join('\n');
+
+const PEEK_SCRIPT = [
+ 'printf "%s\\n%s\\n" peek "$(date +%s)"',
+ `[ -f ${SERVER_LOCK_FILE} ] && cat ${SERVER_LOCK_FILE}`,
+ 'true',
+].join('\n');
+
+type LockStatus = 'taken' | 'held' | 'renewed' | 'lost' | 'released' | 'other' | 'peek';
+
+export type LockReply = {
+ status: LockStatus;
+ /** Whoever the lock file names, lapsed or not. */
+ holder: (ServerLockHolder & { live: boolean }) | null;
+};
+
+const STATUSES: readonly LockStatus[] = [
+ 'taken',
+ 'held',
+ 'renewed',
+ 'lost',
+ 'released',
+ 'other',
+ 'peek',
+];
+
+/** Throws on anything unrecognized: a lock that cannot be understood must not
+ * be taken for one that is free. */
+export function parseLockReply(stdout: string): LockReply {
+ const [status = '', clock = '', ...file] = stdout.split('\n');
+ const now = Number(clock);
+ if (!(STATUSES as readonly string[]).includes(status) || !Number.isFinite(now)) {
+ throw new Error(`Unexpected answer from the server lock: ${stdout.trim().slice(0, 200)}`);
+ }
+ if (file.length < 9 || file[1] === '') return { status: status as LockStatus, holder: null };
+ const expires = Number(file[2]);
+ const since = Number(file[3]);
+ const action = file[6]!;
+ const known = (LOCK_ACTIONS as readonly string[]).includes(action);
+ if (!known)
+ log.warn('stack-lock: the lock names an action this Console does not know', { action });
+ return {
+ status: status as LockStatus,
+ holder: {
+ name: file[7] || 'unknown',
+ hostAccount: file[8] || 'unknown',
+ action: known ? (action as ServerLockAction) : 'checking',
+ heldForSeconds: Number.isFinite(since) ? Math.max(0, now - since) : 0,
+ expiresInSeconds: Number.isFinite(expires) ? Math.max(0, expires - now) : 0,
+ live: Number.isFinite(expires) && expires > now,
+ },
+ };
+}
+
+/** `live` is this module's business and is not sent on. */
+function shownHolder(holder: ServerLockHolder & { live: boolean }): ServerLockHolder {
+ return {
+ name: holder.name,
+ hostAccount: holder.hostAccount,
+ action: holder.action,
+ heldForSeconds: holder.heldForSeconds,
+ expiresInSeconds: holder.expiresInSeconds,
+ };
+}
+
+/** One line each in the lock file, which is read line by line. */
+function oneLine(text: string): string {
+ let printable = '';
+ for (let i = 0; i < text.length; i++) {
+ const code = text.charCodeAt(i);
+ printable += code < 0x20 || code === 0x7f ? ' ' : text[i];
+ }
+ return printable.trim().slice(0, 200) || 'unknown';
+}
+
+/** Leases this run failed to give back, by host, so its next take there takes
+ * them over rather than waiting on itself. */
+const unreleased = new Map<string, Set<string>>();
+
+function scriptArgs(
+ op: 'take' | 'renew' | 'release',
+ token: string,
+ claim: LockClaim,
+ timing: LockTiming,
+ stale: Iterable<string>
+): string[] {
+ return [
+ op,
+ token,
+ claim.consoleId,
+ claim.instance,
+ claim.action,
+ String(timing.ttlSeconds),
+ String(timing.maxHoldSeconds),
+ oneLine(claim.name),
+ oneLine(claim.hostAccount),
+ [...stale].join(' '),
+ ];
+}
+
+/** The lock was taken over by another Console while this one held it. */
+export class ServerLockLostError extends Error {
+ constructor(readonly holder: ServerLockHolder | null) {
+ super(LOCK_LOST_MESSAGE);
+ this.name = 'ServerLockLostError';
+ }
+}
+
+export class ServerLockWaitCancelled extends Error {
+ constructor(readonly holder: ServerLockHolder | null) {
+ super('Stopped waiting for the server, so nothing was changed.');
+ this.name = 'ServerLockWaitCancelled';
+ }
+}
+
+/** Renews itself until released; once a renewal finds it taken over,
+ * {@link assertHeld} throws without asking the host again. Release it in a
+ * `finally`; releasing twice is harmless. */
+export class ServerLease {
+ private released = false;
+ private lostTo: { holder: ServerLockHolder | null } | null = null;
+ private renewing: Promise<void> | null = null;
+ private readonly timer: ReturnType<typeof setInterval>;
+
+ constructor(
+ private readonly host: StackStateHost,
+ readonly token: string,
+ private readonly claim: LockClaim,
+ private readonly timing: LockTiming
+ ) {
+ this.timer = setInterval(() => this.renewInBackground(), timing.renewEveryMs);
+ this.timer.unref?.();
+ }
+
+ /** Whether a renewal has found the lock taken over. */
+ get lost(): boolean {
+ return this.lostTo !== null;
+ }
+
+ private async renew(): Promise<void> {
+ const reply = parseLockReply(
+ await runStateScript(
+ this.host,
+ LOCK_SCRIPT,
+ scriptArgs('renew', this.token, this.claim, this.timing, [])
+ )
+ );
+ if (reply.status === 'renewed') return;
+ const holder = reply.holder?.live ? shownHolder(reply.holder) : null;
+ this.lostTo = { holder };
+ log.error(`stack-lock: lost the lock on ${this.host.label} while holding it`, {
+ action: this.claim.action,
+ holder,
+ });
+ throw new ServerLockLostError(holder);
+ }
+
+ private renewInBackground(): void {
+ if (this.released || this.lostTo || this.renewing) return;
+ this.renewing = this.renew()
+ .catch((error: unknown) => {
+ // Loss is recorded by renew; any other failure is retried next time,
+ // and the lease outlasts several renewals.
+ if (!(error instanceof ServerLockLostError)) {
+ log.warn(`stack-lock: could not renew the lock on ${this.host.label}; retrying`, {
+ error,
+ });
+ }
+ })
+ .finally(() => {
+ this.renewing = null;
+ });
+ }
+
+ /** Renew now, and throw {@link ServerLockLostError} if the lock is no longer
+ * this lease's. Called before anything that cannot check it in the same step. */
+ async assertHeld(): Promise<void> {
+ if (this.released) throw new Error('This server lock was already released.');
+ if (this.lostTo) throw new ServerLockLostError(this.lostTo.holder);
+ await this.renew();
+ }
+
+ /** A failure to reach the host is logged, not thrown: the guarded operation
+ * is over either way, and the lease lapses within {@link LockTiming.ttlSeconds}. */
+ async release(): Promise<void> {
+ if (this.released) return;
+ this.released = true;
+ clearInterval(this.timer);
+ if (this.lostTo) return;
+ try {
+ await this.renewing;
+ await runStateScript(
+ this.host,
+ LOCK_SCRIPT,
+ scriptArgs('release', this.token, this.claim, this.timing, [])
+ );
+ } catch (error) {
+ const left = unreleased.get(this.host.label) ?? new Set<string>();
+ left.add(this.token);
+ unreleased.set(this.host.label, left);
+ log.warn(
+ `stack-lock: could not release the lock on ${this.host.label}; it lapses by itself ` +
+ `within ${this.timing.ttlSeconds}s`,
+ { error }
+ );
+ }
+ }
+}
+
+function sleep(ms: number, signal: AbortSignal): Promise<void> {
+ return new Promise((resolve, reject) => {
+ // Already aborted: the listener below would never hear it.
+ if (signal.aborted) {
+ reject(signal.reason);
+ return;
+ }
+ const timer = setTimeout(() => {
+ signal.removeEventListener('abort', onAbort);
+ resolve();
+ }, ms);
+ const onAbort = () => {
+ clearTimeout(timer);
+ reject(signal.reason);
+ };
+ signal.addEventListener('abort', onAbort, { once: true });
+ });
+}
+
+export type AcquireOptions =
+ /** Wait until free or `signal` aborts, telling `onWaiting` the holder each
+ * time the lock is found held. */
+ | {
+ mode: 'wait';
+ timing: LockTiming;
+ signal: AbortSignal;
+ onWaiting: (holder: ServerLockHolder) => void;
+ }
+ /** Throw `ServerBusyError` at once if someone else holds it. */
+ | { mode: 'refuse'; timing: LockTiming };
+
+export async function acquireServerLock(
+ host: StackStateHost,
+ claim: LockClaim,
+ opts: AcquireOptions
+): Promise<ServerLease> {
+ if (!CONSOLE_ID_PATTERN.test(claim.consoleId)) {
+ throw new Error(
+ `Refusing to take a server lock for a console id that is not one: ${claim.consoleId}`
+ );
+ }
+ const token = randomBytes(16).toString('hex');
+ let last: ServerLockHolder | null = null;
+ for (;;) {
+ if (opts.mode === 'wait' && opts.signal.aborted) throw new ServerLockWaitCancelled(last);
+ const stale = unreleased.get(host.label) ?? new Set<string>();
+ const reply = parseLockReply(
+ await runStateScript(host, LOCK_SCRIPT, scriptArgs('take', token, claim, opts.timing, stale))
+ );
+ if (reply.status === 'taken') {
+ unreleased.delete(host.label);
+ return new ServerLease(host, token, claim, opts.timing);
+ }
+ if (reply.status !== 'held' || reply.holder === null) {
+ throw new Error(`Unexpected answer from the server lock on ${host.label}: ${reply.status}`);
+ }
+ last = shownHolder(reply.holder);
+ if (opts.mode === 'refuse') throw new ServerBusyError(last, host.label);
+ opts.onWaiting(last);
+ try {
+ await sleep(opts.timing.pollEveryMs, opts.signal);
+ } catch {
+ throw new ServerLockWaitCancelled(last);
+ }
+ }
+}
+
+/** A look that changes nothing and creates no volume where there is none. */
+export async function readServerLock(host: StackStateHost): Promise<ServerLockHolder | null> {
+ if (!(await stateVolumeExists(host))) return null;
+ const { holder } = parseLockReply(await readStateVolume(host, PEEK_SCRIPT));
+ return holder?.live ? shownHolder(holder) : null;
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts
new file mode 100644
index 000000000..c4bc8c265
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.test.ts
@@ -0,0 +1,969 @@
+import { execFileSync } from 'node:child_process';
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { dirname, join } from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import type { ExecResult } from '@main/core/execution-context/types';
+import { STACK_HELPER_IMAGE, STACK_STATE_LABEL } from './constants';
+import { buildEnvFile } from './env-file';
+import type { LocalServerSecrets } from './secret-values';
+import type { ServerLease } from './stack-lock';
+import {
+ driftOf,
+ inspectStack,
+ listProjectResources,
+ probeFromStack,
+ publishEnv,
+ readPublishedCopy,
+ readStateVolume,
+ runStateScript,
+ type StackStateHost,
+ stackStateVolume,
+ stampPublishedEnv,
+ withdrawPublishedEnv,
+} from './stack-state';
+import { LOCK_LOST_MESSAGE, WHILE_HOLDING_SERVER_LOCK } from './state-mutex';
+import { stateScriptEnv } from './test-helpers/state-script-shell';
+
+const PROJECT = 'switchdash-remote';
+const lease = { token: 'lease-token' } as ServerLease;
+const WORKING_DIR = '/home/bob/.switchdash/switch-server';
+const OTHER_DIR = '/home/alice/.switchdash/switch-server';
+
+const secrets: LocalServerSecrets = {
+ dbPassword: 'db-pw',
+ dbRuntimePassword: 'db-runtime-pw',
+ agentRegistrationToken: 'agent-token',
+ jwtSecretKey: 'jwt-key',
+ gatewayAdminPassword: 'gw-admin',
+ mattermostAdminPassword: 'mm-admin',
+ mattermostUserPassword: 'mm-user',
+};
+const ports = { gateway: 51000, api: 51001, mattermost: 51002, postgres: 51003 };
+
+function envFor(overrides: Partial<LocalServerSecrets> = {}, version = '0.27.0'): string {
+ return buildEnvFile({
+ version,
+ registry: 'ghcr.io',
+ namespace: 'sandbox-quantum',
+ ports,
+ secrets: { ...secrets, ...overrides },
+ telemetryEnabled: false,
+ });
+}
+
+type HostState = {
+ /** `docker ps --format` lines: service, state, working dir, image. */
+ containers: string[];
+ dataVolumes: string[];
+ stateVolume: boolean;
+ published: string | null;
+ /** The database volume the published copy says it was written for. */
+ publishedStamp: string | null;
+ databaseCreatedAt: string;
+ own: string | null;
+ /** This account's `.env.db`. */
+ ownStamp: string | null;
+ imagePresent: boolean;
+ /** Commands (joined args) that fail. */
+ failing: RegExp | null;
+ readFileFails: boolean;
+};
+
+/** The `docker run` the state launcher execs, from the arguments it is given. */
+function dockerRun(launcherArgs: string[]): string[] {
+ const [, , , , image = '', , , , , mount = '', stdin = '', script = '', ...rest] = launcherArgs;
+ return [
+ 'run',
+ '--rm',
+ ...(stdin === 'yes' ? ['--interactive'] : []),
+ '--network',
+ 'none',
+ '--volume',
+ mount,
+ '--entrypoint',
+ 'sh',
+ image,
+ '-c',
+ script,
+ 'stack-state',
+ ...rest,
+ ];
+}
+
+function fakeHost(initial: Partial<HostState> = {}) {
+ const state: HostState = {
+ containers: [],
+ dataVolumes: [],
+ stateVolume: false,
+ published: null,
+ publishedStamp: null,
+ databaseCreatedAt: '2026-09-01T10:00:00Z',
+ own: null,
+ ownStamp: null,
+ imagePresent: true,
+ failing: null,
+ readFileFails: false,
+ ...initial,
+ };
+ const calls: string[][] = [];
+ /** Mimics the launcher: image check, volume creation, then its `docker run`. */
+ const launch = (args: string[]): string[] => {
+ const [, , , , , volume = '', label = '', create = ''] = args;
+ if (!state.imagePresent) {
+ throw Object.assign(new Error('exit 97'), {
+ stderr: 'switch-console: the helper image is not on this host',
+ });
+ }
+ if (create === 'yes') {
+ calls.push(['volume', 'create', '--label', label, volume]);
+ state.stateVolume = true;
+ }
+ return dockerRun(args);
+ };
+ const exec = vi.fn(async (command: string, args: string[] = []): Promise<ExecResult> => {
+ if (command === 'sh') args = launch(args);
+ calls.push(args);
+ const joined = args.join(' ');
+ if (state.failing?.test(joined)) {
+ throw Object.assign(new Error('exit 1'), { stderr: 'Cannot connect to the Docker daemon' });
+ }
+ if (args[0] === 'ps') return { stdout: state.containers.join('\n'), stderr: '' };
+ if (args[0] === 'volume' && args[1] === 'ls') {
+ const filter = args[args.indexOf('--filter') + 1]!;
+ if (filter.startsWith(`label=${STACK_STATE_LABEL}=`)) {
+ return { stdout: state.stateVolume ? `${PROJECT}_console-state\n` : '', stderr: '' };
+ }
+ return { stdout: state.dataVolumes.join('\n'), stderr: '' };
+ }
+ if (args[0] === 'volume' && args[1] === 'inspect') {
+ return { stdout: `${state.databaseCreatedAt}\n`, stderr: '' };
+ }
+ if (args[0] === 'volume' && args[1] === 'create') {
+ state.stateVolume = true;
+ return { stdout: `${PROJECT}_console-state\n`, stderr: '' };
+ }
+ if (args[0] === 'image') {
+ if (!state.imagePresent)
+ throw Object.assign(new Error('exit 1'), { stderr: 'No such image' });
+ return { stdout: 'sha256:abc\n', stderr: '' };
+ }
+ if (args[0] === 'pull') {
+ state.imagePresent = true;
+ return { stdout: '', stderr: '' };
+ }
+ if (args[0] === 'run') {
+ return {
+ stdout: `${state.published ?? ''}\n---switch-console-stamp---\n${state.publishedStamp ?? ''}\n`,
+ stderr: '',
+ };
+ }
+ throw new Error(`unexpected docker ${joined}`);
+ });
+ const writeCommandInput = vi.fn(
+ async (_command: string, args: string[], _input: string, _opts: { timeoutMs: number }) => {
+ calls.push(launch(args));
+ }
+ );
+ const host: StackStateHost = {
+ ctx: { exec } as unknown as StackStateHost['ctx'],
+ dockerBin: 'docker',
+ composeProjectName: PROJECT,
+ label: 'vm-1',
+ workingDir: WORKING_DIR,
+ readFile: vi.fn(async (path: string) => {
+ if (state.readFileFails) throw new Error('Permission denied');
+ return path === '.env.db' ? state.ownStamp : state.own;
+ }),
+ writeCommandInput,
+ };
+ return { host, state, calls, exec, writeCommandInput };
+}
+
+beforeEach(() => {
+ vi.clearAllMocks();
+});
+
+describe('the helper image', () => {
+ it('is the stack’s own Postgres image, so a host that ran the stack already has it', () => {
+ const compose = readFileSync(
+ join(
+ dirname(fileURLToPath(import.meta.url)),
+ 'resources/standalone-docker-compose.pinned.yml'
+ ),
+ 'utf8'
+ );
+ const postgres = /^ {2}postgres:\n(?: {4}.*\n)*? {4}image: (\S+)$/m.exec(compose);
+
+ expect(postgres?.[1]).toBe(STACK_HELPER_IMAGE);
+ });
+});
+
+describe('listProjectResources', () => {
+ it('finds the project by label, so another account’s stack is seen without its compose file', async () => {
+ const { host, calls } = fakeHost({
+ containers: [
+ `switch\trunning\t${OTHER_DIR}\tghcr.io/sandbox-quantum/switch-core:0.28.0`,
+ `postgres\texited\t${OTHER_DIR}\tpostgres:16-alpine`,
+ ],
+ dataVolumes: [`${PROJECT}_pgdata`, `${PROJECT}_mmdata`],
+ stateVolume: true,
+ });
+
+ expect(await listProjectResources(host)).toEqual({
+ containers: [
+ {
+ service: 'switch',
+ state: 'running',
+ workingDir: OTHER_DIR,
+ image: 'ghcr.io/sandbox-quantum/switch-core:0.28.0',
+ },
+ {
+ service: 'postgres',
+ state: 'exited',
+ workingDir: OTHER_DIR,
+ image: 'postgres:16-alpine',
+ },
+ ],
+ dataVolumes: [`${PROJECT}_pgdata`, `${PROJECT}_mmdata`],
+ stateVolume: true,
+ });
+ for (const args of calls) expect(args).not.toContain('-f');
+ expect(calls[0]).toEqual(
+ expect.arrayContaining(['--all', `label=com.docker.compose.project=${PROJECT}`])
+ );
+ });
+
+ it('reads a container without a working-dir label as unknown rather than empty', async () => {
+ const { host } = fakeHost({ containers: ['switch\trunning\t\tswitch-core:0.28.0'] });
+
+ const { containers } = await listProjectResources(host);
+
+ expect(containers).toEqual([
+ { service: 'switch', state: 'running', workingDir: null, image: 'switch-core:0.28.0' },
+ ]);
+ });
+});
+
+describe('the published copy', () => {
+ it('is read through a throwaway container that mounts the volume read-only', async () => {
+ const env = envFor();
+ const { host, calls } = fakeHost({ stateVolume: true, published: env });
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: null });
+ const run = calls.find((args) => args[0] === 'run')!;
+ expect(run).toEqual(
+ expect.arrayContaining([
+ '--rm',
+ '--network',
+ 'none',
+ `${stackStateVolume(host)}:/state:ro`,
+ STACK_HELPER_IMAGE,
+ ])
+ );
+ });
+
+ it('reads a copy written before stamps existed as unstamped', async () => {
+ const env = envFor();
+ const { host, exec } = fakeHost({ stateVolume: true });
+ exec.mockImplementation(async (command: string, args: string[] = []) => {
+ if (command === 'sh') return { stdout: env, stderr: '' };
+ throw new Error(`unexpected docker ${args.join(' ')}`);
+ });
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: null });
+ });
+
+ it('reads as absent when the volume holds no copy', async () => {
+ const { host } = fakeHost({ stateVolume: true, published: '' });
+
+ expect(await readPublishedCopy(host)).toBeNull();
+ });
+
+ it('pulls a missing helper image under a pull’s timeout, then runs, in one trip each time', async () => {
+ const { host, calls, exec } = fakeHost({
+ stateVolume: true,
+ published: envFor(),
+ imagePresent: false,
+ });
+
+ await readPublishedCopy(host);
+ await readPublishedCopy(host);
+
+ expect(calls.map((args) => args[0])).toEqual(['pull', 'run', 'run']);
+ expect(exec).toHaveBeenCalledWith(
+ 'docker',
+ ['pull', '--quiet', STACK_HELPER_IMAGE],
+ expect.objectContaining({ timeout: 10 * 60_000 })
+ );
+ });
+
+ it('is published on stdin and never as an argument, which every account could read', async () => {
+ const env = envFor();
+ const { host, calls, writeCommandInput } = fakeHost();
+
+ await publishEnv(host, env, lease);
+
+ expect(calls).toContainEqual([
+ 'volume',
+ 'create',
+ '--label',
+ `${STACK_STATE_LABEL}=${PROJECT}`,
+ `${PROJECT}_console-state`,
+ ]);
+ expect(writeCommandInput).toHaveBeenCalledOnce();
+ const [, launched, input] = writeCommandInput.mock.calls[0]!;
+ const args = dockerRun(launched);
+ // No database volume yet (a first start), so an empty stamp line.
+ expect(input).toBe(`\n${env}`);
+ expect(args.join(' ')).not.toContain(secrets.gatewayAdminPassword);
+ expect(args.join(' ')).not.toContain(secrets.dbPassword);
+ expect(args).toEqual(
+ expect.arrayContaining(['--interactive', `${PROJECT}_console-state:/state`])
+ );
+ // Written aside and moved into place, so a reader never sees half a file.
+ const script = args[args.indexOf('-c') + 1]!;
+ expect(script).toMatch(/umask 077\n.*\ncat > .*stack\.env\.tmp.*\nmv /s);
+ expect(script).toContain('rm -f "/state/stack.db"');
+ for (const argv of calls) expect(argv.join(' ')).not.toContain(secrets.jwtSecretKey);
+ });
+
+ it('is published only while the lock is still this start’s, checked in the same step', async () => {
+ const { host, writeCommandInput } = fakeHost();
+
+ await publishEnv(host, envFor(), lease);
+
+ const args = dockerRun(writeCommandInput.mock.calls[0]![1]);
+ expect(args.slice(args.indexOf('-c') + 2)).toEqual(['stack-state', lease.token]);
+ expect(args[args.indexOf('-c') + 1]).toContain(WHILE_HOLDING_SERVER_LOCK);
+ });
+
+ it('reads the database volume it was written for, when that was recorded', async () => {
+ const env = envFor();
+ const { host } = fakeHost({
+ stateVolume: true,
+ published: env,
+ publishedStamp: '2026-09-01T10:00:00Z',
+ });
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: '2026-09-01T10:00:00Z' });
+ });
+
+ it('is stamped with the database volume it is published for, when there is one', async () => {
+ const env = envFor();
+ const { host, writeCommandInput } = fakeHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+
+ await publishEnv(host, env, lease);
+
+ const [, , input] = writeCommandInput.mock.calls[0]!;
+ expect(input).toBe(`2026-09-01T10:00:00Z\n${env}`);
+ });
+
+ it('is left unstamped, and says so, when a start leaves no database volume to stamp it with', async () => {
+ const { host, writeCommandInput } = fakeHost({ stateVolume: true, dataVolumes: [] });
+
+ await stampPublishedEnv(host, lease);
+
+ expect(writeCommandInput).not.toHaveBeenCalled();
+ });
+
+ it('is stamped after a first start, once the database volume exists', async () => {
+ const { host, writeCommandInput } = fakeHost({
+ stateVolume: true,
+ dataVolumes: [`${PROJECT}_pgdata`, `${PROJECT}_mmdata`],
+ });
+
+ await stampPublishedEnv(host, lease);
+
+ const [, launched, input] = writeCommandInput.mock.calls[0]!;
+ const args = dockerRun(launched);
+ expect(input).toBe('2026-09-01T10:00:00Z\n');
+ expect(args[args.indexOf('-c') + 1]).toContain('/state/stack.db');
+ });
+
+ it('is withdrawn on reset, and only it: the activity record survives', async () => {
+ const { host, writeCommandInput } = fakeHost({ stateVolume: true });
+
+ await withdrawPublishedEnv(host, lease);
+
+ const args = dockerRun(writeCommandInput.mock.calls[0]![1]);
+ const script = args[args.indexOf('-c') + 1]!;
+ expect(script).toContain('rm -f "/state/stack.env"');
+ expect(script).toContain('"/state/stack.db"');
+ expect(script).not.toMatch(/rm -rf|activity/);
+ });
+
+ it('has nothing to withdraw, and creates nothing, when the volume was never made', async () => {
+ const { host, writeCommandInput, calls } = fakeHost();
+
+ await withdrawPublishedEnv(host, lease);
+
+ expect(writeCommandInput).not.toHaveBeenCalled();
+ expect(calls.some((args) => args[1] === 'create')).toBe(false);
+ });
+});
+
+describe('the published copy’s scripts, run for real', () => {
+ let dir: string;
+
+ beforeEach(() => {
+ dir = mkdtempSync(join(tmpdir(), 'stack-state-'));
+ });
+
+ afterEach(() => {
+ rmSync(dir, { recursive: true, force: true });
+ });
+
+ /** A host whose state volume is `dir`, with every script run by `sh`. */
+ function realHost(initial: Partial<HostState> = {}) {
+ const fake = fakeHost({ stateVolume: true, ...initial });
+ writeFileSync(join(dir, 'lock'), `switch-console-lock v1\n${lease.token}\n`);
+ const env = stateScriptEnv();
+ const sh = (args: string[], input: string) => {
+ const run = dockerRun(args);
+ const at = run.indexOf('-c');
+ const script = run[at + 1]!.replaceAll('/state', dir);
+ return execFileSync('sh', ['-c', script, ...run.slice(at + 2)], {
+ input,
+ encoding: 'utf8',
+ env,
+ });
+ };
+ fake.writeCommandInput.mockImplementation(async (_command, args, input) => {
+ sh(args, input);
+ });
+ const exec = fake.exec.getMockImplementation()!;
+ fake.exec.mockImplementation(async (command: string, args: string[] = []) =>
+ command === 'sh' ? { stdout: sh(args, ''), stderr: '' } : exec(command, args)
+ );
+ return fake;
+ }
+
+ it('reads back exactly the copy and stamp it wrote', async () => {
+ const env = envFor();
+ const { host } = realHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+
+ await publishEnv(host, env, lease);
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: '2026-09-01T10:00:00Z' });
+ });
+
+ it('drops an earlier stamp when publishing for a database not created yet', async () => {
+ const { host, state } = realHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+ await publishEnv(host, envFor({ dbPassword: 'first' }), lease);
+
+ // A reset: no database volume when the next start publishes.
+ state.dataVolumes = [];
+ const env = envFor({ dbPassword: 'second' });
+ await publishEnv(host, env, lease);
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: null });
+
+ // And compose creates it; the start then stamps the copy with it.
+ state.dataVolumes = [`${PROJECT}_pgdata`];
+ state.databaseCreatedAt = '2026-09-24T12:00:00Z';
+ await stampPublishedEnv(host, lease);
+
+ expect(await readPublishedCopy(host)).toEqual({ env, stamp: '2026-09-24T12:00:00Z' });
+ });
+
+ it('writes nothing once another Console has taken the lock over', async () => {
+ const { host } = realHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+ await publishEnv(host, envFor({ dbPassword: 'theirs' }), lease);
+ const stale = { token: 'token-this-console-lost' } as ServerLease;
+
+ await expect(publishEnv(host, envFor({ dbPassword: 'mine' }), stale)).rejects.toThrow(
+ LOCK_LOST_MESSAGE
+ );
+ await expect(stampPublishedEnv(host, stale)).rejects.toThrow(LOCK_LOST_MESSAGE);
+ await expect(withdrawPublishedEnv(host, stale)).rejects.toThrow(LOCK_LOST_MESSAGE);
+
+ expect(await readPublishedCopy(host)).toMatchObject({ env: envFor({ dbPassword: 'theirs' }) });
+ });
+
+ it('withdraws the copy and its stamp together', async () => {
+ const { host } = realHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+ await publishEnv(host, envFor(), lease);
+
+ await withdrawPublishedEnv(host, lease);
+
+ expect(await readPublishedCopy(host)).toBeNull();
+ expect(() => readFileSync(join(dir, 'stack.db'))).toThrow(/ENOENT/);
+ });
+});
+
+describe('inspectStack', () => {
+ it('reports a host with nothing of the stack as absent', async () => {
+ const { host } = fakeHost();
+
+ expect(await inspectStack(host)).toEqual({ kind: 'absent' });
+ });
+
+ it('prefers the published copy, and says whether the stack is up', async () => {
+ const env = envFor();
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${OTHER_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: env,
+ });
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'present',
+ source: 'published',
+ running: true,
+ published: true,
+ raw: env,
+ env: { ports, secrets, version: '0.27.0' },
+ });
+ });
+
+ it('takes the published copy over a stale one left in this account’s working dir', async () => {
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${OTHER_DIR}`],
+ stateVolume: true,
+ published: envFor({ dbPassword: 'new-owner-pw' }),
+ own: envFor({ dbPassword: 'stale-owner-pw' }),
+ });
+
+ const stack = await inspectStack(host);
+
+ expect(stack.kind === 'present' && stack.env.secrets.dbPassword).toBe('new-owner-pw');
+ });
+
+ it('trusts a published copy written for the database that is there', async () => {
+ const env = envFor();
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${OTHER_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: env,
+ publishedStamp: '2026-09-01T10:00:00Z',
+ });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'present', source: 'published' });
+ });
+
+ it('ignores a published copy written for a database that has since been recreated', async () => {
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${OTHER_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: envFor({ dbPassword: 'before-the-reset' }),
+ publishedStamp: '2026-09-01T10:00:00Z',
+ databaseCreatedAt: '2026-09-20T08:30:00Z',
+ });
+
+ expect(await inspectStack(host)).toEqual({
+ kind: 'unshared',
+ ownerDir: OTHER_DIR,
+ running: true,
+ });
+ });
+
+ it('falls back to this account’s own settings past a stale published copy', async () => {
+ const own = envFor({ dbPassword: 'after-the-reset' });
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${WORKING_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: envFor({ dbPassword: 'before-the-reset' }),
+ publishedStamp: '2026-09-01T10:00:00Z',
+ databaseCreatedAt: '2026-09-20T08:30:00Z',
+ own,
+ });
+
+ const stack = await inspectStack(host);
+
+ expect(stack).toMatchObject({ kind: 'present', source: 'working-dir', published: false });
+ expect(stack.kind === 'present' && stack.env.secrets.dbPassword).toBe('after-the-reset');
+ });
+
+ it('reads this account’s own stack, not yet published, from its working dir', async () => {
+ const env = envFor();
+ const { host } = fakeHost({
+ containers: [`switch\texited\t${WORKING_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ own: env,
+ });
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'present',
+ source: 'working-dir',
+ running: false,
+ published: false,
+ raw: env,
+ });
+ });
+
+ it('treats settings with no stack behind them as a first start', async () => {
+ const { host } = fakeHost({ own: envFor(), stateVolume: true, published: envFor() });
+
+ expect(await inspectStack(host)).toEqual({ kind: 'absent' });
+ });
+
+ it('refuses to take another account’s unpublished stack for this one', async () => {
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${OTHER_DIR}`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ own: envFor({ dbPassword: 'stale' }),
+ });
+
+ expect(await inspectStack(host)).toEqual({
+ kind: 'unshared',
+ ownerDir: OTHER_DIR,
+ running: true,
+ });
+ });
+
+ it('treats data with no containers and no readable settings as someone else’s', async () => {
+ const { host } = fakeHost({ dataVolumes: [`${PROJECT}_pgdata`] });
+
+ expect(await inspectStack(host)).toEqual({ kind: 'unshared', ownerDir: null, running: false });
+ });
+
+ it('treats data with no containers as this account’s when it has the settings', async () => {
+ // As `compose down` from an older Console leaves it: no containers, never published.
+ const { host } = fakeHost({ dataVolumes: [`${PROJECT}_pgdata`], own: envFor() });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'present', source: 'working-dir' });
+ });
+
+ it('trusts this account’s settings written for the database that is there', async () => {
+ const { host } = fakeHost({
+ dataVolumes: [`${PROJECT}_pgdata`],
+ own: envFor(),
+ ownStamp: '2026-09-01T10:00:00Z\n',
+ });
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'present',
+ source: 'working-dir',
+ stamp: '2026-09-01T10:00:00Z',
+ });
+ });
+
+ it('does not take this account’s settings for a database since recreated elsewhere', async () => {
+ const { host } = fakeHost({
+ dataVolumes: [`${PROJECT}_pgdata`],
+ own: envFor(),
+ ownStamp: '2026-08-01T09:00:00Z\n',
+ databaseCreatedAt: '2026-09-20T08:30:00Z',
+ });
+
+ expect(await inspectStack(host)).toEqual({ kind: 'unshared', ownerDir: null, running: false });
+ });
+
+ it('asks about the database once when both copies need comparing with it', async () => {
+ const { host, calls } = fakeHost({
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: envFor(),
+ publishedStamp: '2026-08-01T09:00:00Z',
+ own: envFor(),
+ ownStamp: '2026-08-01T09:00:00Z\n',
+ databaseCreatedAt: '2026-09-20T08:30:00Z',
+ });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'unshared' });
+ expect(calls.filter((args) => args[0] === 'volume' && args[1] === 'inspect')).toHaveLength(1);
+ });
+
+ it('cannot tell whose settings they are when the database cannot be asked about', async () => {
+ const { host } = fakeHost({
+ dataVolumes: [`${PROJECT}_pgdata`],
+ own: envFor(),
+ ownStamp: '2026-08-01T09:00:00Z\n',
+ failing: /^volume inspect/,
+ });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'unreadable' });
+ });
+
+ it('names what a partial published copy is missing', async () => {
+ const partial = envFor().replace(/^JWT_SECRET_KEY=.*$/m, '');
+ const { host } = fakeHost({
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: partial,
+ });
+
+ expect(await inspectStack(host)).toEqual({
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: partial,
+ running: false,
+ });
+ });
+
+ it('says what failed when the daemon gives no reason of its own', async () => {
+ const { host, exec } = fakeHost();
+ exec.mockRejectedValue(new Error('channel closed'));
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'unreadable',
+ reason: expect.stringContaining('channel closed'),
+ });
+ });
+
+ it('says what failed when the failure is not even an Error', async () => {
+ const { host, exec } = fakeHost();
+ exec.mockRejectedValue('connection reset by peer');
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'unreadable',
+ reason: expect.stringContaining('connection reset by peer'),
+ });
+ });
+
+ it('trusts a stamped copy as before when the database volume names no creation time', async () => {
+ const env = envFor();
+ const { host } = fakeHost({
+ containers: [`switch\trunning\t${WORKING_DIR}\tghcr.io/x/switch-core:0.11.0`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: env,
+ publishedStamp: '2026-09-01T10:00:00Z',
+ databaseCreatedAt: '',
+ });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'present', source: 'published' });
+ });
+
+ it('reports a daemon it cannot ask as unreadable, never as absent', async () => {
+ const { host } = fakeHost({ failing: /^ps/ });
+
+ const stack = await inspectStack(host);
+
+ expect(stack.kind).toBe('unreadable');
+ expect(stack.kind === 'unreadable' && stack.reason).toMatch(/vm-1.*Cannot connect/);
+ });
+
+ it('reports a working dir it cannot read as unreadable, never as absent', async () => {
+ const { host } = fakeHost({ readFileFails: true });
+
+ expect(await inspectStack(host)).toEqual({
+ kind: 'unreadable',
+ reason: 'Permission denied',
+ });
+ });
+});
+
+describe('the version a stack runs', () => {
+ it('is read from the image of its running core, beside the version its settings ask for', async () => {
+ const { host } = fakeHost({
+ containers: [
+ `switch\trunning\t${OTHER_DIR}\tghcr.io/sandbox-quantum/switch-core:0.26.0`,
+ `postgres\trunning\t${OTHER_DIR}\tpostgres:16-alpine`,
+ ],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: envFor({}, '0.27.0'),
+ });
+
+ expect(await inspectStack(host)).toMatchObject({
+ kind: 'present',
+ runningVersion: '0.26.0',
+ env: { version: '0.27.0' },
+ });
+ });
+
+ it('is not known from containers that are not running', async () => {
+ const { host } = fakeHost({
+ containers: [`switch\texited\t${OTHER_DIR}\tghcr.io/sandbox-quantum/switch-core:0.26.0`],
+ dataVolumes: [`${PROJECT}_pgdata`],
+ stateVolume: true,
+ published: envFor(),
+ });
+
+ expect(await inspectStack(host)).toMatchObject({ kind: 'present', runningVersion: null });
+ });
+});
+
+describe('what the setup step is told about a host', () => {
+ const present = {
+ kind: 'present' as const,
+ env: { ports, secrets, version: '0.27.0' },
+ raw: 'RAW\n',
+ source: 'published' as const,
+ running: true,
+ published: true,
+ runningVersion: null,
+ stamp: null,
+ };
+
+ it('compares the version a running stack runs, else the one its settings name', () => {
+ expect(driftOf({ ...present, runningVersion: '0.26.0' })).toMatchObject({
+ deployed: '0.26.0',
+ });
+ expect(driftOf(present)).toMatchObject({ deployed: '0.27.0' });
+ expect(driftOf({ ...present, env: { ...present.env, version: null } })).toBeNull();
+ });
+
+ it('reports the version the stack runs, and never a secret', () => {
+ const probe = probeFromStack('vm-1', { ...present, runningVersion: '0.26.0' }, null);
+
+ expect(probe).toMatchObject({ kind: 'present', deployedVersion: '0.26.0', shared: true });
+ expect(JSON.stringify(probe)).not.toContain(secrets.gatewayAdminPassword);
+ });
+
+ it('says an unshared stack is another account’s even when it cannot say whose', () => {
+ const probe = probeFromStack(
+ 'vm-1',
+ { kind: 'unshared', ownerDir: null, running: false },
+ null
+ );
+
+ expect(probe.kind === 'unshared' && probe.message).toMatch(
+ /^The Switch server on vm-1 was set up from another account and its settings have not been shared/
+ );
+ });
+
+ it('names where another account’s unshared stack was started from', () => {
+ const probe = probeFromStack(
+ 'vm-1',
+ {
+ kind: 'unshared',
+ ownerDir: OTHER_DIR,
+ running: true,
+ },
+ null
+ );
+
+ expect(probe).toMatchObject({ kind: 'unshared', running: true, ownerDir: OTHER_DIR });
+ expect(probe.kind === 'unshared' && probe.message).toMatch(
+ /set up from another account \(from \/home\/alice\/\.switchdash\/switch-server\)/
+ );
+ });
+
+ it('passes on what a partial stack is missing, a reason it could not look, and an empty host', () => {
+ expect(
+ probeFromStack(
+ 'vm-1',
+ {
+ kind: 'incomplete',
+ source: 'published',
+ missing: ['JWT_SECRET_KEY'],
+ raw: 'JWT_SECRET_KEY=\n',
+ running: false,
+ },
+ null
+ )
+ ).toEqual({ kind: 'incomplete', running: false, missing: ['JWT_SECRET_KEY'] });
+ expect(probeFromStack('vm-1', { kind: 'unreadable', reason: 'ssh dropped' }, null)).toEqual({
+ kind: 'unreadable',
+ reason: 'ssh dropped',
+ });
+ expect(probeFromStack('vm-1', { kind: 'absent' }, null)).toEqual({
+ kind: 'absent',
+ busy: null,
+ });
+ });
+
+ it('passes on who is changing the stack right now, for Start or Connect to wait for', () => {
+ const busy = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting' as const,
+ heldForSeconds: 30,
+ expiresInSeconds: 90,
+ };
+
+ expect(probeFromStack('vm-1', { kind: 'absent' }, busy)).toEqual({ kind: 'absent', busy });
+ expect(probeFromStack('vm-1', present, busy)).toMatchObject({ kind: 'present', busy });
+ });
+});
+
+describe('the state launcher, run for real', () => {
+ let dir: string;
+
+ beforeEach(() => {
+ dir = mkdtempSync(join(tmpdir(), 'state-launcher-'));
+ });
+
+ afterEach(() => {
+ rmSync(dir, { recursive: true, force: true });
+ });
+
+ /** Real `sh`, with a fake docker that logs its arguments and has the helper
+ * image only when told to. */
+ function launcherHost(imagePresent: boolean, ownImage: string | null = null) {
+ const log = join(dir, 'docker.log');
+ const docker = join(dir, 'docker');
+ writeFileSync(
+ docker,
+ [
+ '#!/bin/sh',
+ `printf '%s\\n' "$*" >> '${log}'`,
+ `if [ "$1" = image ] && [ "$3" = '${STACK_HELPER_IMAGE}' ]; then ${imagePresent ? 'exit 0' : 'exit 1'}; fi`,
+ `if [ "$1" = image ]; then [ "$3" = '${ownImage ?? ''}' ] && exit 0; exit 1; fi`,
+ `if [ "$1" = ps ]; then echo '${ownImage ?? ''}'; exit 0; fi`,
+ 'if [ "$1" = run ]; then echo ran; fi',
+ 'exit 0',
+ ].join('\n'),
+ { mode: 0o755 }
+ );
+ const { host } = fakeHost();
+ const real: StackStateHost = {
+ ...host,
+ dockerBin: docker,
+ ctx: {
+ exec: async (command: string, args: string[] = []) => {
+ try {
+ return { stdout: execFileSync(command, args, { encoding: 'utf8' }), stderr: '' };
+ } catch (error) {
+ const stderr = (error as { stderr?: Buffer | string }).stderr?.toString() ?? '';
+ throw Object.assign(new Error('exit'), { stderr });
+ }
+ },
+ } as unknown as StackStateHost['ctx'],
+ };
+ const asked = () => readFileSync(log, 'utf8').trim().split('\n');
+ return { host: real, asked };
+ }
+
+ it('checks the image, creates the volume labelled, and runs, in one command', async () => {
+ const { host, asked } = launcherHost(true);
+
+ expect((await runStateScript(host, 'echo hi', ['arg one'])).trim()).toBe('ran');
+
+ const [image, create, run] = asked();
+ expect(image).toBe(`image inspect ${STACK_HELPER_IMAGE}`);
+ expect(create).toBe(
+ `volume create --label ${STACK_STATE_LABEL}=${PROJECT} ${PROJECT}_console-state`
+ );
+ expect(run).toMatch(/^run --rm --network none --volume \S+_console-state:\/state /);
+ expect(run).toMatch(/-c echo hi stack-state arg one$/);
+ });
+
+ it('creates nothing for a read', async () => {
+ const { host, asked } = launcherHost(true);
+
+ await readStateVolume(host, 'cat /state/x');
+
+ expect(asked().some((line) => line.startsWith('volume create'))).toBe(false);
+ expect(asked().at(-1)).toMatch(/_console-state:\/state:ro /);
+ });
+
+ it('runs in the stack’s own Postgres image on a host without the helper image', async () => {
+ const { host, asked } = launcherHost(false, 'postgres:17-alpine');
+
+ await runStateScript(host, 'echo hi', []);
+
+ expect(asked().at(-1)).toMatch(/--entrypoint sh postgres:17-alpine -c echo hi/);
+ expect(asked().some((line) => line.startsWith('pull'))).toBe(false);
+ });
+
+ it('says the helper image is missing rather than letting docker run pull it', async () => {
+ const { host, asked } = launcherHost(false);
+
+ // The pull goes to the fake docker as well, which "succeeds" without
+ // fetching anything, so the retry finds the image still missing.
+ await expect(runStateScript(host, 'echo hi', [])).rejects.toThrow(
+ /helper image is not on this host/
+ );
+ expect(asked().filter((line) => line.startsWith('run'))).toEqual([]);
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.ts
new file mode 100644
index 000000000..c6bc506a0
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/stack-state.ts
@@ -0,0 +1,584 @@
+import { log } from '@main/lib/logger';
+import { COMPATIBLE_SWITCH_VERSION } from '@shared/app-identity';
+import type {
+ RemoteStackProbe,
+ ServerLockHolder,
+ SwitchVersionDrift,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+import {
+ CORE_SERVICE,
+ ENV_FILE_NAME,
+ ENV_STAMP_FILE_NAME,
+ STACK_HELPER_IMAGE,
+ STACK_STATE_LABEL,
+ STACK_STATE_VOLUME_SUFFIX,
+} from './constants';
+import { classifyVersionDrift, imageTag } from './deployed-version';
+import { readStackEnv, type StackEnv } from './env-file';
+import { commandFailure, errorText } from './error-text';
+import type { ServerHost } from './host/types';
+import type { ServerLease } from './stack-lock';
+import { WHILE_HOLDING_SERVER_LOCK } from './state-mutex';
+
+/**
+ * What a remote host has of a Switch Console-managed stack, read off the host
+ * rather than this desktop's store (CHOO-2893). Every account sharing the host
+ * needs the credentials the stack's volumes were created with; a Console that
+ * generated its own would lock the stack out of its database. So the `.env`
+ * the stack last started with is published to a Docker volume every account
+ * reaches through the daemon, and compose run from any working dir with that
+ * byte-identical `.env` leaves the containers alone.
+ */
+
+/** Only the remote host shares its stack, so only it provides this. */
+export type StackStateHost = Pick<
+ ServerHost,
+ 'ctx' | 'dockerBin' | 'composeProjectName' | 'label' | 'workingDir' | 'readFile'
+> & {
+ writeCommandInput(
+ command: string,
+ args: string[],
+ input: string,
+ opts: { timeoutMs: number }
+ ): Promise<void>;
+};
+
+const COMPOSE_PROJECT_LABEL = 'com.docker.compose.project';
+const COMPOSE_SERVICE_LABEL = 'com.docker.compose.service';
+const COMPOSE_WORKING_DIR_LABEL = 'com.docker.compose.project.working_dir';
+
+const STATE_MOUNT = '/state';
+const PUBLISHED_ENV_FILE = 'stack.env';
+/** Beside the published copy: which database volume it was written for. */
+const PUBLISHED_STAMP_FILE = 'stack.db';
+/** Separates the copy from its stamp when both come back from one read. */
+const STAMP_MARKER = '---switch-console-stamp---';
+
+/** Holds the Postgres data whose credentials the published copy must match. */
+const DATABASE_VOLUME = 'pgdata';
+
+const QUICK_TIMEOUT_MS = 60_000;
+/** A host that has never run the stack has to pull the helper image first. */
+const PULL_TIMEOUT_MS = 10 * 60_000;
+
+export function stackStateVolume(host: Pick<StackStateHost, 'composeProjectName'>): string {
+ return `${host.composeProjectName}_${STACK_STATE_VOLUME_SUFFIX}`;
+}
+
+/** Nothing passed here may be a secret: the arguments are visible in the
+ * host's process table. */
+async function docker(
+ host: StackStateHost,
+ args: string[],
+ timeout: number = QUICK_TIMEOUT_MS
+): Promise<string> {
+ try {
+ const { stdout } = await host.ctx.exec(host.dockerBin, args, {
+ timeout,
+ maxBuffer: 8 * 1024 * 1024,
+ });
+ return stdout;
+ } catch (error) {
+ throw new Error(`docker ${args[0]} on ${host.label} failed: ${commandFailure(error)}`);
+ }
+}
+
+function lines(stdout: string): string[] {
+ return stdout
+ .split('\n')
+ .map((line) => line.trim())
+ .filter(Boolean);
+}
+
+export type ProjectContainer = {
+ service: string;
+ /** Docker's own word: `running`, `exited`, `created`, … */
+ state: string;
+ /** The working dir of the account whose compose created this container. */
+ workingDir: string | null;
+ image: string;
+};
+
+export type ProjectResources = {
+ /** Every container of the stack's compose project, stopped ones included. */
+ containers: ProjectContainer[];
+ /** The stack's own named volumes — Postgres and Mattermost data. */
+ dataVolumes: string[];
+ stateVolume: boolean;
+};
+
+/** Found by label rather than through a compose file, so it sees a stack
+ * another account started from a working dir this one cannot read. */
+export async function listProjectResources(host: StackStateHost): Promise<ProjectResources> {
+ const [containers, dataVolumes, stateVolume] = await Promise.all([
+ listContainers(host),
+ listDataVolumes(host),
+ stateVolumeExists(host),
+ ]);
+ return { containers, dataVolumes, stateVolume };
+}
+
+async function listContainers(host: StackStateHost): Promise<ProjectContainer[]> {
+ return lines(
+ await docker(host, [
+ 'ps',
+ '--all',
+ '--filter',
+ `label=${COMPOSE_PROJECT_LABEL}=${host.composeProjectName}`,
+ '--format',
+ `{{.Label "${COMPOSE_SERVICE_LABEL}"}}\t{{.State}}\t{{.Label "${COMPOSE_WORKING_DIR_LABEL}"}}\t{{.Image}}`,
+ ])
+ ).map((line) => {
+ const [service = '', state = '', workingDir = '', image = ''] = line.split('\t');
+ return { service, state, workingDir: workingDir || null, image };
+ });
+}
+
+export async function stateVolumeExists(host: StackStateHost): Promise<boolean> {
+ const stateVolumes = lines(
+ await docker(host, [
+ 'volume',
+ 'ls',
+ '--filter',
+ `label=${STACK_STATE_LABEL}=${host.composeProjectName}`,
+ '--format',
+ '{{.Name}}',
+ ])
+ );
+ return stateVolumes.includes(stackStateVolume(host));
+}
+
+async function listDataVolumes(host: StackStateHost): Promise<string[]> {
+ return lines(
+ await docker(host, [
+ 'volume',
+ 'ls',
+ '--filter',
+ `label=${COMPOSE_PROJECT_LABEL}=${host.composeProjectName}`,
+ '--format',
+ '{{.Name}}',
+ ])
+ );
+}
+
+/** The database volume's creation time. It changes exactly when a reset
+ * recreates the volume, so it tells a copy written for this database from a
+ * leftover. */
+async function databaseStamp(host: StackStateHost, dataVolumes: string[]): Promise<string | null> {
+ const volume = `${host.composeProjectName}_${DATABASE_VOLUME}`;
+ if (!dataVolumes.includes(volume)) return null;
+ const stamp = await docker(host, ['volume', 'inspect', '--format', '{{.CreatedAt}}', volume]);
+ return stamp.trim() || null;
+}
+
+/** What the launcher says on stderr when the host has no helper image, so the
+ * caller can pull it under a pull's timeout and try again. */
+const HELPER_IMAGE_MISSING = 'switch-console: the helper image is not on this host';
+
+/**
+ * Runs a container against the state volume in one round trip. Without the
+ * helper image it falls back to the image the stack's Postgres container runs
+ * (same shell and `flock`); only when neither exists does it fail, so the
+ * caller pulls under a pull's timeout rather than inside a quick one.
+ *
+ * `$1…$8`: docker binary, helper image, volume, its label, `yes` to create it,
+ * compose project, mount, `yes` for stdin; then the script and its arguments.
+ */
+const LAUNCHER = [
+ 'docker=$1 image=$2 volume=$3 label=$4 create=$5 project=$6 mount=$7 stdin=$8',
+ 'shift 8',
+ 'if ! "$docker" image inspect "$image" >/dev/null 2>&1; then',
+ ` own=$("$docker" ps -a --filter "label=com.docker.compose.project=$project" --filter label=com.docker.compose.service=postgres --format '{{.Image}}' | head -n 1)`,
+ ' if [ -n "$own" ] && "$docker" image inspect "$own" >/dev/null 2>&1; then image=$own',
+ ` else echo '${HELPER_IMAGE_MISSING}' >&2; exit 97; fi`,
+ 'fi',
+ 'if [ "$create" = yes ]; then "$docker" volume create --label "$label" "$volume" >/dev/null || exit 1; fi',
+ 'interactive=',
+ 'if [ "$stdin" = yes ]; then interactive=--interactive; fi',
+ 'script=$1',
+ 'shift',
+ 'exec "$docker" run --rm $interactive --network none --volume "$mount" --entrypoint sh "$image" -c "$script" stack-state "$@"',
+].join('\n');
+
+type StateContainer = {
+ /** A read never creates the volume; the caller has checked it exists. */
+ mode: 'read' | 'write';
+ script: string;
+ /** Arrive as `$1…`. Visible in the host's process table: never a secret. */
+ scriptArgs: string[];
+};
+
+function launcherArgs(host: StackStateHost, container: StateContainer, interactive: boolean) {
+ const mount = `${stackStateVolume(host)}:${STATE_MOUNT}${container.mode === 'read' ? ':ro' : ''}`;
+ return [
+ '-c',
+ LAUNCHER,
+ 'state-launcher',
+ host.dockerBin,
+ STACK_HELPER_IMAGE,
+ stackStateVolume(host),
+ `${STACK_STATE_LABEL}=${host.composeProjectName}`,
+ container.mode === 'write' ? 'yes' : 'no',
+ host.composeProjectName,
+ mount,
+ interactive ? 'yes' : 'no',
+ container.script,
+ ...container.scriptArgs,
+ ];
+}
+
+/** Run `launch` once, and again after pulling the helper image when the host
+ * turns out not to have it. */
+async function withHelperImage<T>(host: StackStateHost, launch: () => Promise<T>): Promise<T> {
+ try {
+ return await launch();
+ } catch (error) {
+ if (!errorText(error).includes(HELPER_IMAGE_MISSING)) throw error;
+ await docker(host, ['pull', '--quiet', STACK_HELPER_IMAGE], PULL_TIMEOUT_MS);
+ return launch();
+ }
+}
+
+async function runStateContainer(host: StackStateHost, container: StateContainer): Promise<string> {
+ return withHelperImage(host, async () => {
+ try {
+ const { stdout } = await host.ctx.exec('sh', launcherArgs(host, container, false), {
+ timeout: QUICK_TIMEOUT_MS,
+ maxBuffer: 8 * 1024 * 1024,
+ });
+ return stdout;
+ } catch (error) {
+ throw new Error(`docker run on ${host.label} failed: ${commandFailure(error)}`);
+ }
+ });
+}
+
+/** Mounts the state volume read-only. The volume must already exist: `docker
+ * run -v` would otherwise create it, and a read must not. */
+export async function readStateVolume(host: StackStateHost, script: string): Promise<string> {
+ return runStateContainer(host, { mode: 'read', script, scriptArgs: [] });
+}
+
+/** `input` goes on stdin, the only way a secret may reach the script. Creates
+ * the volume on first use. */
+export async function writeStateVolume(
+ host: StackStateHost,
+ script: string,
+ input: string,
+ scriptArgs: string[]
+): Promise<void> {
+ await withHelperImage(host, () =>
+ host.writeCommandInput(
+ 'sh',
+ launcherArgs(host, { mode: 'write', script, scriptArgs }, true),
+ input,
+ { timeoutMs: QUICK_TIMEOUT_MS }
+ )
+ );
+}
+
+/** Read-write, returning stdout. `scriptArgs` are visible in the host's
+ * process table; a secret goes through {@link writeStateVolume}'s stdin.
+ * Creates the volume on first use. */
+export async function runStateScript(
+ host: StackStateHost,
+ script: string,
+ scriptArgs: string[]
+): Promise<string> {
+ return runStateContainer(host, { mode: 'write', script, scriptArgs });
+}
+
+/** The published `.env`, and the database volume it was written for (null
+ * when that was not recorded). */
+export type PublishedCopy = { env: string; stamp: string | null };
+
+export async function readPublishedCopy(host: StackStateHost): Promise<PublishedCopy | null> {
+ const out = await readStateVolume(
+ host,
+ `cat "${STATE_MOUNT}/${PUBLISHED_ENV_FILE}" 2>/dev/null; printf '\\n%s\\n' '${STAMP_MARKER}'; ` +
+ `cat "${STATE_MOUNT}/${PUBLISHED_STAMP_FILE}" 2>/dev/null; true`
+ );
+ // The marker follows a newline of its own; taking exactly that one back off
+ // leaves the copy byte for byte as published.
+ const split = out.lastIndexOf(`\n${STAMP_MARKER}`);
+ const env = split === -1 ? out : out.slice(0, split);
+ const stamp = split === -1 ? '' : out.slice(split + 1 + STAMP_MARKER.length).trim();
+ if (env.trim().length === 0) return null;
+ return { env, stamp: stamp || null };
+}
+
+/** Reads the stamp from stdin's first line and the copy from the rest; the
+ * server lock's token is `$1`. */
+const PUBLISH_SCRIPT = [
+ 'set -e',
+ WHILE_HOLDING_SERVER_LOCK,
+ 'umask 077',
+ 'IFS= read -r stamp',
+ `cat > "${STATE_MOUNT}/.${PUBLISHED_ENV_FILE}.tmp"`,
+ `mv "${STATE_MOUNT}/.${PUBLISHED_ENV_FILE}.tmp" "${STATE_MOUNT}/${PUBLISHED_ENV_FILE}"`,
+ 'if [ -n "$stamp" ]; then',
+ ` printf "%s\\n" "$stamp" > "${STATE_MOUNT}/.${PUBLISHED_STAMP_FILE}.tmp"`,
+ ` mv "${STATE_MOUNT}/.${PUBLISHED_STAMP_FILE}.tmp" "${STATE_MOUNT}/${PUBLISHED_STAMP_FILE}"`,
+ 'else',
+ ` rm -f "${STATE_MOUNT}/${PUBLISHED_STAMP_FILE}"`,
+ 'fi',
+].join('\n');
+
+/**
+ * Atomically publish `env`, the exact file a start gives compose, only while
+ * `lease` still holds the server lock (checked in the same step), so a Console
+ * that lost the lock cannot publish its credentials over its successor's. The
+ * copy is written before its stamp, so a read in between sees a mismatch,
+ * never a false match. Returns null when there is no database volume yet; the
+ * start then calls {@link stampPublishedEnv} once compose has created it.
+ */
+export async function publishEnv(
+ host: StackStateHost,
+ env: string,
+ lease: ServerLease
+): Promise<string | null> {
+ const stamp = await databaseStamp(host, await listDataVolumes(host));
+ await writeStateVolume(host, PUBLISH_SCRIPT, `${stamp ?? ''}\n${env}`, [lease.token]);
+ return stamp;
+}
+
+/** Unstamped, a first start's copy would still look current after a reset
+ * from a Console that does not publish. */
+export async function stampPublishedEnv(
+ host: StackStateHost,
+ lease: ServerLease
+): Promise<string | null> {
+ const stamp = await databaseStamp(host, await listDataVolumes(host));
+ if (stamp === null) {
+ log.warn(
+ `stack-state: the stack on ${host.label} has no database volume after starting, ` +
+ `so its published settings are not stamped with one`
+ );
+ return null;
+ }
+ await writeStateVolume(
+ host,
+ `set -e\n${WHILE_HOLDING_SERVER_LOCK}\n` +
+ `umask 077 && IFS= read -r stamp && ` +
+ `printf "%s\\n" "$stamp" > "${STATE_MOUNT}/.${PUBLISHED_STAMP_FILE}.tmp" && ` +
+ `mv "${STATE_MOUNT}/.${PUBLISHED_STAMP_FILE}.tmp" "${STATE_MOUNT}/${PUBLISHED_STAMP_FILE}"`,
+ `${stamp}\n`,
+ [lease.token]
+ );
+ return stamp;
+}
+
+/** For a reset: the copy's credentials die with the data volumes, and the next
+ * Console would otherwise adopt credentials that open nothing. The rest of the
+ * volume, including the activity record, is kept. */
+export async function withdrawPublishedEnv(
+ host: StackStateHost,
+ lease: ServerLease
+): Promise<void> {
+ if (!(await stateVolumeExists(host))) return;
+ await writeStateVolume(
+ host,
+ `set -e\n${WHILE_HOLDING_SERVER_LOCK}\n` +
+ `rm -f "${STATE_MOUNT}/${PUBLISHED_ENV_FILE}" "${STATE_MOUNT}/.${PUBLISHED_ENV_FILE}.tmp" ` +
+ `"${STATE_MOUNT}/${PUBLISHED_STAMP_FILE}"`,
+ '',
+ [lease.token]
+ );
+}
+
+export type StackEnvSource = 'published' | 'working-dir';
+
+export type StackOnHost =
+ /** No containers or data: the only case where new credentials may be made. */
+ | { kind: 'absent' }
+ /** `published` is false for a stack started before publishing existed,
+ * which the next start or connect publishes. */
+ | {
+ kind: 'present';
+ env: StackEnv;
+ /** The exact `.env` text, so what gets published is what compose read. */
+ raw: string;
+ source: StackEnvSource;
+ running: boolean;
+ published: boolean;
+ /** What the running core container is on, where `env.version` is what it
+ * was last asked to be; they differ when a start published then failed. */
+ runningVersion: string | null;
+ /** The database volume these settings were written for, if recorded. */
+ stamp: string | null;
+ }
+ /** Another account's never-published stack, whose settings this account
+ * cannot read. Starting here would recreate it with new credentials. */
+ | { kind: 'unshared'; ownerDir: string | null; running: boolean }
+ /** `raw` lets a start check another copy of the settings against what this
+ * one does carry before filling the gaps from it. */
+ | {
+ kind: 'incomplete';
+ source: StackEnvSource;
+ missing: string[];
+ raw: string;
+ running: boolean;
+ }
+ /** Distinct from `absent`: an unreachable daemon is not an empty host. */
+ | { kind: 'unreadable'; reason: string };
+
+export function unsharedStackMessage(hostLabel: string, ownerDir: string | null): string {
+ const where = ownerDir ? ` (from ${ownerDir})` : '';
+ return (
+ `The Switch server on ${hostLabel} was set up from another account${where} and its settings ` +
+ `have not been shared, so this account cannot read them. Starting it from here would ` +
+ `replace its credentials and take it down, so nothing was changed. It is shared the next ` +
+ `time an up-to-date Switch Console starts or connects to it from the account that set it up.`
+ );
+}
+
+/** Compared with this build's pin by the version it runs when running, else
+ * by the one its settings name. */
+export function driftOf(
+ stack: Extract<StackOnHost, { kind: 'present' }>
+): SwitchVersionDrift | null {
+ const version = stack.runningVersion ?? stack.env.version;
+ return version === null ? null : classifyVersionDrift(version, COMPATIBLE_SWITCH_VERSION);
+}
+
+/** Enough for the renderer to choose between Connect and Start, and nothing
+ * secret. `busy` is the Console holding the stack's lock, if any. */
+export function probeFromStack(
+ hostLabel: string,
+ stack: StackOnHost,
+ busy: ServerLockHolder | null
+): RemoteStackProbe {
+ switch (stack.kind) {
+ case 'absent':
+ return { kind: 'absent', busy };
+ case 'present':
+ return {
+ kind: 'present',
+ running: stack.running,
+ deployedVersion: stack.runningVersion ?? stack.env.version,
+ shared: stack.published,
+ drift: driftOf(stack),
+ busy,
+ };
+ case 'unshared':
+ return {
+ kind: 'unshared',
+ running: stack.running,
+ ownerDir: stack.ownerDir,
+ message: unsharedStackMessage(hostLabel, stack.ownerDir),
+ };
+ case 'incomplete':
+ return { kind: 'incomplete', running: stack.running, missing: stack.missing };
+ case 'unreadable':
+ return { kind: 'unreadable', reason: stack.reason };
+ }
+}
+
+function isRunning(resources: ProjectResources): boolean {
+ return resources.containers.some(
+ (container) => container.service === CORE_SERVICE && container.state === 'running'
+ );
+}
+
+function fromEnvText(
+ raw: string,
+ source: StackEnvSource,
+ resources: ProjectResources,
+ published: boolean,
+ stamp: string | null
+): StackOnHost {
+ const running = isRunning(resources);
+ const reading = readStackEnv(raw);
+ if (reading.kind === 'incomplete') {
+ return { kind: 'incomplete', source, missing: reading.missing, raw, running };
+ }
+ return {
+ kind: 'present',
+ env: reading.env,
+ raw,
+ source,
+ running,
+ published,
+ runningVersion: runningCoreVersion(resources),
+ stamp,
+ };
+}
+
+function runningCoreVersion(resources: ProjectResources): string | null {
+ const core = resources.containers.find(
+ (container) => container.service === CORE_SERVICE && container.state === 'running'
+ );
+ return core ? imageTag(core.image) : null;
+}
+
+/**
+ * Trusted in this order: no containers or data means a first start, whatever
+ * settings are lying about; then the published copy, unless stamped for a
+ * database volume since recreated by a Console that does not publish; then
+ * this account's own `.env`, only when nothing on the daemon says the stack
+ * belongs to another account.
+ */
+export async function inspectStack(host: StackStateHost): Promise<StackOnHost> {
+ let resources: ProjectResources;
+ let published: PublishedCopy | null = null;
+ // Asked once, and only when a stamp needs comparing with it.
+ let database: string | null | undefined;
+ let own: string | null;
+ let ownStamp: string | null;
+ try {
+ [resources, own, ownStamp] = await Promise.all([
+ listProjectResources(host),
+ host.readFile(ENV_FILE_NAME),
+ host.readFile(ENV_STAMP_FILE_NAME),
+ ]);
+ if (resources.stateVolume) published = await readPublishedCopy(host);
+ if (published?.stamp) database = await databaseStamp(host, resources.dataVolumes);
+ } catch (error) {
+ return { kind: 'unreadable', reason: errorText(error) };
+ }
+
+ const hasProject = resources.containers.length > 0 || resources.dataVolumes.length > 0;
+ if (!hasProject) return { kind: 'absent' };
+
+ const running = isRunning(resources);
+ // A copy with no stamp, or no database volume to compare it with, cannot be
+ // judged and is trusted.
+ const stale = published?.stamp != null && database != null && published.stamp !== database;
+ if (published !== null && !stale) {
+ return fromEnvText(published.env, 'published', resources, true, published.stamp);
+ }
+ if (stale) {
+ log.warn(
+ `stack-state: the published settings on ${host.label} were written for a database that ` +
+ `has since been recreated, by a Console that does not share its settings; ignoring them`
+ );
+ }
+
+ // Never published: it is ours only if this account created its containers.
+ const foreignDir = resources.containers
+ .map((container) => container.workingDir)
+ .find((dir): dir is string => dir !== null && dir !== host.workingDir);
+ if (foreignDir !== undefined) return { kind: 'unshared', ownerDir: foreignDir, running };
+ if (own === null) return { kind: 'unshared', ownerDir: null, running };
+ // A copy stamped for a database since recreated belongs to a stack that is
+ // gone: a Console that does not share its settings reset it and started it
+ // with its own credentials, leaving nothing else to say whose it is now.
+ const stamp = ownStamp?.trim() || null;
+ if (stamp !== null) {
+ let current: string | null;
+ try {
+ current =
+ database !== undefined ? database : await databaseStamp(host, resources.dataVolumes);
+ } catch (error) {
+ return { kind: 'unreadable', reason: errorText(error) };
+ }
+ if (current !== null && current !== stamp) {
+ log.warn(
+ `stack-state: this account's settings on ${host.label} were written for a database ` +
+ `that has since been recreated elsewhere; not using them`
+ );
+ return { kind: 'unshared', ownerDir: null, running };
+ }
+ }
+ return fromEnvText(own, 'working-dir', resources, false, stamp);
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/state-mutex.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/state-mutex.ts
new file mode 100644
index 000000000..a55af0784
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/state-mutex.ts
@@ -0,0 +1,36 @@
+/**
+ * Serialises the scripts that write a shared stack's state volume
+ * (CHOO-2893), whose read-then-writes are only safe with nobody else's in the
+ * middle. Each takes an exclusive `flock` on `/state/.mutex`; the containers
+ * share one kernel, so it holds across them, and it drops however the script
+ * exits. It is held for milliseconds, unlike the server lock (stack-lock.ts)
+ * that a start holds for minutes.
+ */
+
+/** A Console id becomes a file name in the register and a lock-script
+ * argument, so anything else is refused before it reaches a script. */
+export const CONSOLE_ID_PATTERN = /^[0-9A-Fa-f-]{1,64}$/;
+
+/** Held for the rest of the script. */
+export const UNDER_STATE_MUTEX = [
+ 'exec 9>/state/.mutex',
+ "flock 9 || { echo 'could not lock the state volume' >&2; exit 1; }",
+].join('\n');
+
+export const SERVER_LOCK_FILE = '/state/lock';
+export const SERVER_LOCK_TOKEN_LINE = 2;
+
+export const LOCK_LOST_MESSAGE =
+ 'This Console no longer holds the lock on this Switch server — another Console took it over ' +
+ 'after this one stopped answering — so it changed nothing more.';
+
+/** Takes the state mutex, then exits 3 unless the server lock is still held
+ * with the token in `$1`. The check and the write after it share the mutex, so
+ * nobody can take the lock between them. */
+export const WHILE_HOLDING_SERVER_LOCK = [
+ UNDER_STATE_MUTEX,
+ `if [ "$(sed -n ${SERVER_LOCK_TOKEN_LINE}p ${SERVER_LOCK_FILE} 2>/dev/null)" != "$1" ]; then`,
+ ` echo '${LOCK_LOST_MESSAGE}' >&2`,
+ ' exit 3',
+ 'fi',
+].join('\n');
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/telemetry-consent.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/telemetry-consent.ts
index bb256a445..117b9bcf5 100644
--- a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/telemetry-consent.ts
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/telemetry-consent.ts
@@ -1,8 +1,9 @@
import { appSettingsService } from '@main/core/settings/settings-service';
import type { DeployedTelemetry } from '@shared/core/managed-switch-server/managed-switch-server';
import { runningServiceEnv } from './compose';
-import { ENV_FILE_NAME } from './constants';
-import { readEnvValue, TELEMETRY_ENABLED_KEY } from './env-file';
+import { CORE_SERVICE, ENV_FILE_NAME } from './constants';
+import { envFlag, readEnvValue, TELEMETRY_ENABLED_KEY } from './env-file';
+import { errorText } from './error-text';
import type { ServerHost } from './host/types';
/**
@@ -22,22 +23,6 @@ export async function telemetryConsent(): Promise<boolean> {
return (await appSettingsService.get('telemetry')).enabled;
}
-/** The compose service whose environment carries the server's telemetry gate. */
-const CORE_SERVICE = 'switch';
-
-/**
- * How pydantic-settings reads a boolean, narrowed to the forms that can reach
- * this variable.
- *
- * Absent means off, and that is a reading rather than a fallback: switch-core's
- * own default for the gate is off, so a stack started before this existed — or
- * by an operator who never set it — is genuinely not reporting.
- */
-function envFlag(value: string | null | undefined): boolean {
- if (value === null || value === undefined) return false;
- return ['1', 'true', 'yes', 'on'].includes(value.trim().toLowerCase());
-}
-
/**
* Whether the stack on `host` is actually sharing usage data.
*
@@ -72,7 +57,3 @@ export async function readDeployedTelemetry(host: ServerHost): Promise<DeployedT
return { known: false, reason: failures.join('; ') };
}
-
-function errorText(error: unknown): string {
- return error instanceof Error ? error.message : String(error);
-}
diff --git a/console/apps/switch-console-desktop/src/main/core/managed-switch-server/test-helpers/state-script-shell.ts b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/test-helpers/state-script-shell.ts
new file mode 100644
index 000000000..d0ae6a7e3
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/managed-switch-server/test-helpers/state-script-shell.ts
@@ -0,0 +1,21 @@
+import { execFileSync } from 'node:child_process';
+import { chmodSync, mkdtempSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+
+/**
+ * For running the state scripts in local `sh`. Where there is no `flock`
+ * (macOS), a no-op stand-in goes first on the PATH: these tests run one script
+ * at a time, and stack-lock.docker.test.ts tests the mutex on a real volume.
+ */
+export function stateScriptEnv(): NodeJS.ProcessEnv {
+ try {
+ execFileSync('sh', ['-c', 'command -v flock'], { stdio: 'pipe' });
+ return process.env;
+ } catch {
+ const bin = mkdtempSync(join(tmpdir(), 'no-flock-'));
+ writeFileSync(join(bin, 'flock'), '#!/bin/sh\nexit 0\n');
+ chmodSync(join(bin, 'flock'), 0o755);
+ return { ...process.env, PATH: `${bin}:${process.env.PATH ?? ''}` };
+ }
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.test.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.test.ts
index 7d724a3dd..0f25ef093 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.test.ts
@@ -1,11 +1,30 @@
+import { createHash } from 'node:crypto';
import { readFile, stat } from 'node:fs/promises';
import { expect, it, vi } from 'vitest';
import type { IExecutionContext } from '@main/core/execution-context/types';
vi.mock('@main/core/agent-runtime/impl/resolve-sidecar-bundle', () => ({
resolveSharedHostBundlePath: vi.fn(),
}));
-vi.mock('@main/core/ssh/connect/connect-agent-ssh', () => ({ ensureSshConnected: vi.fn() }));
-const { runSharedHostCommand } = await import('./shared-host-deployment');
+const ssh = vi.hoisted(() => ({
+ ensureSshConnected: vi.fn(async () => ({ proxy: true })),
+ exec: vi.fn(async () => ({
+ stdout: '/home/alice/.local/state/switch/sdk-watchers/abc\n',
+ stderr: '',
+ })),
+ contexts: [] as unknown[][],
+}));
+vi.mock('@main/core/ssh/connect/connect-agent-ssh', () => ({
+ ensureSshConnected: ssh.ensureSshConnected,
+}));
+vi.mock('@main/core/execution-context/ssh-execution-context', () => ({
+ SshExecutionContext: class {
+ exec = ssh.exec;
+ constructor(...args: unknown[]) {
+ ssh.contexts.push(args);
+ }
+ },
+}));
+const { resolveWatcherRoot, runSharedHostCommand } = await import('./shared-host-deployment');
it.each([false, true])(
'keeps configuration out of launch arguments and cleans files on failure=%s',
async (fail) => {
@@ -35,3 +54,22 @@ it.each([false, true])(
await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' });
}
);
+
+it('finds a watcher’s state root on a host without deploying anything there', async () => {
+ const { root } = await resolveWatcherRoot(
+ { kind: 'ssh', connectionId: 'conn-1', host: 'vm-1', dir: '/work' } as never,
+ '/work',
+ 'switch-agent-1'
+ );
+
+ expect(root).toBe('/home/alice/.local/state/switch/sdk-watchers/abc');
+ expect(ssh.ensureSshConnected).toHaveBeenCalledWith('conn-1', 'vm-1');
+ expect(ssh.exec).toHaveBeenCalledOnce();
+ const [command, args] = ssh.exec.mock.calls[0] as unknown as [string, string[]];
+ expect(command).toBe('node');
+ expect(args.slice(2)).toEqual([
+ createHash('sha256').update('switch-agent-1').digest('hex'),
+ 'sdk-watchers',
+ 'switch-agent-1',
+ ]);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.ts
index eb9db6d3a..58b244486 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-host-deployment.ts
@@ -86,6 +86,26 @@ export async function deploySharedHost(
return { ctx, root, entrypoint };
}
+/** The state root of `identity`'s watcher on an SSH host, and a context to act
+ * in it, without deploying the host bundle. */
+export async function resolveWatcherRoot(
+ transport: Extract<LocationTransport, { kind: 'ssh' }>,
+ sessionPath: string,
+ identity: string
+): Promise<{ ctx: IExecutionContext; root: string }> {
+ const proxy = await ensureSshConnected(transport.connectionId, transport.host);
+ const ctx = new SshExecutionContext(proxy, { root: sessionPath });
+ const key = createHash('sha256').update(identity).digest('hex');
+ const { stdout } = await ctx.exec('node', [
+ '-e',
+ RESOLVE_STATE_ROOT,
+ key,
+ 'sdk-watchers',
+ identity,
+ ]);
+ return { ctx, root: stdout.trim() };
+}
+
export async function runSharedHostCommand(
transport: LocationTransport,
deployed: Awaited<ReturnType<typeof deploySharedHost>>,
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.auto-approve.test.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.auto-approve.test.ts
new file mode 100644
index 000000000..e48653d0b
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.auto-approve.test.ts
@@ -0,0 +1,272 @@
+import { execFile } from 'node:child_process';
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { promisify } from 'node:util';
+import { afterEach, beforeEach, expect, it, vi } from 'vitest';
+
+/**
+ * Auto-approve on a watcher that several Consoles under one account write
+ * (CHOO-2893). The scripts run for real, with `node`, against a directory
+ * standing in for the watcher's state root on the host.
+ */
+
+const mocks = vi.hoisted(() => ({
+ agent: vi.fn(),
+ location: vi.fn(),
+ deploy: vi.fn(),
+ runCommand: vi.fn(),
+ updateAgent: vi.fn(),
+ stopLegacySidecar: vi.fn(),
+}));
+
+vi.mock('@main/core/agents/getAgentById', () => ({ getAgentById: mocks.agent }));
+vi.mock('@main/core/managed-switch-server/session-readiness', () => ({
+ ensureServerSessionReady: vi.fn(async () => {}),
+}));
+vi.mock('@main/core/switch-servers/servers-store', () => ({ getServer: vi.fn(async () => null) }));
+vi.mock('@main/core/switch-rooms/auto-session-store', () => ({
+ listAutoSessionAgentIds: vi.fn(async () => []),
+ listStoppedControllerAgentIds: vi.fn(async () => []),
+}));
+vi.mock('@main/core/ssh/connect/connect-agent-ssh', () => ({ ensureSshConnected: vi.fn() }));
+vi.mock('@main/core/execution-context/ssh-execution-context', () => ({
+ SshExecutionContext: vi.fn(),
+}));
+vi.mock('@main/core/agents/updateAgent', () => ({ updateAgent: mocks.updateAgent }));
+vi.mock('@main/core/agents/agent-location', () => ({ getAgentLocation: mocks.location }));
+vi.mock('@main/core/locations/location-manager', () => ({
+ locationManager: {
+ openLocation: async () => ({ success: true, data: { fs: {}, settings: {} } }),
+ },
+}));
+vi.mock('@main/core/locations/location-runtime-factory', () => ({
+ resolveSessionEnv: async () => ({ sessionEnvVars: {} }),
+}));
+vi.mock('./shared-agent-runtime', () => ({
+ buildSharedHostConfig: async () => ({
+ session: { sessionId: 'watcher', agentId: 'switch-agent-1' },
+ start: {
+ provider: 'claude',
+ input: {
+ sessionId: 'watcher',
+ cwd: '/work',
+ runtimeMode: (await mocks.agent()).autoApprove ? 'full-access' : 'approval-required',
+ },
+ },
+ execution: { credentialsPath: '/work/.switch/agents/scout.json' },
+ }),
+}));
+vi.mock('./shared-host-deployment', () => ({
+ deploySharedHost: mocks.deploy,
+ resolveWatcherRoot: async () => ({ ctx, root }),
+ runSharedHostCommand: mocks.runCommand,
+}));
+vi.mock('./legacy-sidecar', () => ({ stopLegacySidecar: mocks.stopLegacySidecar }));
+vi.mock('./watcher-inspection', () => ({ waitForWatcherStop: 'process.exit(0)' }));
+vi.mock('./adopt-subagent', () => ({ adoptSubagent: vi.fn() }));
+vi.mock('./local-host', () => ({ startLocalWatcher: vi.fn(), stopLocalWatcher: vi.fn() }));
+vi.mock('@main/lib/logger', () => ({ log: { info: vi.fn(), warn: vi.fn() } }));
+
+const {
+ configureSharedWatcher,
+ configureSharedWatcherFor,
+ keepAutoApproveChoice,
+ recordAutoApproveOnHost,
+} = await import('./shared-watcher');
+
+const run = promisify(execFile);
+let root: string;
+
+/** A host whose commands are run here, for real. */
+const ctx = {
+ exec: async (command: string, args: string[]) => {
+ const { stdout, stderr } = await run(command, args);
+ return { stdout, stderr };
+ },
+};
+
+function agent(autoApprove: boolean) {
+ return { id: 'agent-1', name: 'scout', switchAgentId: 'switch-agent-1', autoApprove };
+}
+
+function savedSpec(runtimeMode: string) {
+ writeFileSync(
+ join(root, 'config.json'),
+ JSON.stringify({ session: { agentId: 'switch-agent-1' }, start: { input: { runtimeMode } } })
+ );
+}
+
+/** A person's choice, kept beside the watcher by a sharing-aware Console. */
+function chosen(runtimeMode: string) {
+ writeFileSync(join(root, 'auto-approve.json'), JSON.stringify({ runtimeMode, at: 'then' }));
+}
+
+function readChoice(): { runtimeMode: string } {
+ return JSON.parse(readFileSync(join(root, 'auto-approve.json'), 'utf8'));
+}
+
+function readSpec(): { start: { input: { runtimeMode: string } } } {
+ return JSON.parse(readFileSync(join(root, 'config.json'), 'utf8'));
+}
+
+function writtenMode(): string {
+ const [, , config] = mocks.runCommand.mock.calls.at(-1) as [
+ unknown,
+ unknown,
+ { start: { input: { runtimeMode: string } } },
+ ];
+ return config.start.input.runtimeMode;
+}
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ root = mkdtempSync(join(tmpdir(), 'watcher-root-'));
+ mocks.location.mockResolvedValue({
+ id: 'remote',
+ dir: '/work',
+ sshHost: 'builder',
+ connectionId: 'connection-1',
+ });
+ mocks.deploy.mockImplementation(async () => ({ ctx, root, entrypoint: 'shared-host.mjs' }));
+});
+
+afterEach(() => {
+ rmSync(root, { recursive: true, force: true });
+});
+
+it('takes auto-approve from the host when another Console changed it', async () => {
+ mocks.agent.mockResolvedValue(agent(false));
+ savedSpec('full-access');
+ chosen('full-access');
+
+ await configureSharedWatcher('agent-1', { connected: true, spawning: true }, 'explicit');
+
+ expect(writtenMode()).toBe('full-access');
+ expect(mocks.updateAgent).toHaveBeenCalledWith({ agentId: 'agent-1', autoApprove: true });
+});
+
+it('leaves the row alone when it already agrees with the host', async () => {
+ mocks.agent.mockResolvedValue(agent(true));
+ savedSpec('full-access');
+ chosen('full-access');
+
+ await configureSharedWatcher('agent-1', { connected: true, spawning: true }, 'explicit');
+
+ expect(writtenMode()).toBe('full-access');
+ expect(mocks.updateAgent).not.toHaveBeenCalled();
+});
+
+it('does not take a saved spec for a choice when nobody made one', async () => {
+ // An older Console wrote this spec from its own row and kept no choice.
+ mocks.agent.mockResolvedValue(agent(true));
+ savedSpec('approval-required');
+
+ await configureSharedWatcher('agent-1', { connected: true, spawning: true }, 'explicit');
+
+ expect(writtenMode()).toBe('full-access');
+ expect(mocks.updateAgent).not.toHaveBeenCalled();
+});
+
+it('writes the first watcher from the row, with no spec on the host yet', async () => {
+ mocks.agent.mockResolvedValue(agent(true));
+
+ await configureSharedWatcher('agent-1', { connected: true, spawning: true }, 'explicit');
+
+ expect(writtenMode()).toBe('full-access');
+ expect(mocks.updateAgent).not.toHaveBeenCalled();
+});
+
+it('writes this Console’s value when the person using it has just changed it', async () => {
+ mocks.agent.mockResolvedValue(agent(true));
+ savedSpec('approval-required');
+
+ await configureSharedWatcherFor('agent-1', { connected: true, spawning: true }, 'explicit', {
+ name: undefined,
+ autoApprove: 'this-console',
+ });
+
+ expect(writtenMode()).toBe('full-access');
+ expect(mocks.updateAgent).not.toHaveBeenCalled();
+});
+
+it('does not take a subagent watcher’s setting for its parent’s', async () => {
+ mocks.agent.mockResolvedValue(agent(false));
+ savedSpec('full-access');
+ chosen('full-access');
+ // A subagent watcher reads its Switch id from the credentials file.
+ const readId = vi.spyOn(ctx, 'exec');
+ readId.mockImplementation(async (command: string, args: string[]) => {
+ if (args[1]?.includes('SWITCH_AGENT_ID')) return { stdout: 'sub-id\n', stderr: '' };
+ const { stdout, stderr } = await run(command, args);
+ return { stdout, stderr };
+ });
+
+ await configureSharedWatcher(
+ 'agent-1',
+ { connected: true, spawning: true },
+ 'explicit',
+ 'helper'
+ );
+
+ expect(writtenMode()).toBe('approval-required');
+ expect(mocks.updateAgent).not.toHaveBeenCalled();
+ readId.mockRestore();
+});
+
+it('keeps a changed setting on the host for a watcher that starts no sessions', async () => {
+ mocks.agent.mockResolvedValue(agent(false));
+ savedSpec('approval-required');
+
+ // The value is the one just chosen, which the row does not hold yet.
+ await recordAutoApproveOnHost('agent-1', true);
+
+ expect(readSpec().start.input.runtimeMode).toBe('full-access');
+ expect(readChoice().runtimeMode).toBe('full-access');
+ expect(mocks.runCommand).not.toHaveBeenCalled();
+ expect(mocks.deploy).not.toHaveBeenCalled();
+});
+
+it('keeps auto-approve turned off on the host as asking for approval', async () => {
+ mocks.agent.mockResolvedValue(agent(true));
+ savedSpec('full-access');
+
+ await recordAutoApproveOnHost('agent-1', false);
+
+ expect(readSpec().start.input.runtimeMode).toBe('approval-required');
+ expect(readChoice().runtimeMode).toBe('approval-required');
+});
+
+it('keeps a choice on the host without touching a watcher about to be rewritten', async () => {
+ mocks.agent.mockResolvedValue(agent(false));
+ savedSpec('approval-required');
+
+ await keepAutoApproveChoice('agent-1', true);
+
+ expect(readChoice().runtimeMode).toBe('full-access');
+ expect(readSpec().start.input.runtimeMode).toBe('approval-required');
+ expect(mocks.deploy).not.toHaveBeenCalled();
+});
+
+it('keeps nothing on a host for an agent that is not there, or never linked, or local', async () => {
+ mocks.agent.mockResolvedValueOnce(undefined);
+ await expect(keepAutoApproveChoice('agent-1', true)).rejects.toThrow(/does not exist/);
+
+ mocks.agent.mockResolvedValueOnce({ ...agent(true), switchAgentId: null });
+ await keepAutoApproveChoice('agent-1', true);
+
+ mocks.agent.mockResolvedValueOnce(agent(true));
+ mocks.location.mockResolvedValueOnce({ id: 'local', dir: '/work', sshHost: null });
+ await recordAutoApproveOnHost('agent-1', true);
+
+ expect(() => readChoice()).toThrow(/ENOENT/);
+});
+
+it('has nothing to record for an agent that has never had a watcher', async () => {
+ mocks.agent.mockResolvedValue(agent(true));
+ rmSync(root, { recursive: true, force: true });
+
+ await recordAutoApproveOnHost('agent-1', true);
+
+ expect(() => readSpec()).toThrow(/ENOENT/);
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.locality.test.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.locality.test.ts
index 893a4cf36..ccf6bc0a9 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.locality.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.locality.test.ts
@@ -34,6 +34,7 @@ vi.mock('@main/core/execution-context/ssh-execution-context', () => ({
},
}));
vi.mock('@main/core/agents/agent-location', () => ({ getAgentLocation: mocks.location }));
+vi.mock('@main/core/agents/updateAgent', () => ({ updateAgent: vi.fn() }));
vi.mock('@main/core/locations/location-manager', () => ({
locationManager: {
openLocation: async () => ({ success: true, data: { fs: {}, settings: {} } }),
@@ -173,7 +174,7 @@ it.each([true, false])(
async (autoStart) => {
mocks.location.mockResolvedValue({ id: 'local', dir: '/work', sshHost: null });
mocks.spawning.mockResolvedValue(autoStart ? ['agent-1'] : []);
- await applyControllerState('agent-1', 'restore');
+ await applyControllerState('agent-1', 'restore', 'host');
expect(mocks.startLocal.mock.calls[0][1]).toEqual({ intent: 'restore', spawning: autoStart });
}
);
@@ -210,7 +211,7 @@ it('leaves a stopped controller off the air however auto-start is set', async ()
mocks.location.mockResolvedValue({ id: 'local', dir: '/work', sshHost: null });
mocks.stopped.mockResolvedValue(['agent-1']);
mocks.spawning.mockResolvedValue(['agent-1']);
- await applyControllerState('agent-1', 'restore');
+ await applyControllerState('agent-1', 'restore', 'host');
expect(mocks.stopLocal).toHaveBeenCalledWith('switch-agent-1');
expect(mocks.startLocal).not.toHaveBeenCalled();
});
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.ts
index dbc23e825..d7565577d 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/shared-watcher.ts
@@ -1,6 +1,8 @@
import { readFile } from 'node:fs/promises';
+import type { SharedHostConfig } from '@switch-console/agent-providers';
import { getAgentLocation } from '@main/core/agents/agent-location';
import { getAgentById } from '@main/core/agents/getAgentById';
+import { updateAgent } from '@main/core/agents/updateAgent';
import { SshExecutionContext } from '@main/core/execution-context/ssh-execution-context';
import { locationManager } from '@main/core/locations/location-manager';
import { resolveSessionEnv } from '@main/core/locations/location-runtime-factory';
@@ -13,6 +15,7 @@ import {
} from '@main/core/switch-rooms/auto-session-store';
import { controllerConnectionId } from '@main/core/switch-rooms/session-connection-id';
import { getServer } from '@main/core/switch-servers/servers-store';
+import { log } from '@main/lib/logger';
import { adoptSubagent } from './adopt-subagent';
import { stopLegacySidecar } from './legacy-sidecar';
import {
@@ -22,7 +25,11 @@ import {
type WatcherIntent,
} from './local-host';
import { buildSharedHostConfig } from './shared-agent-runtime';
-import { deploySharedHost, runSharedHostCommand } from './shared-host-deployment';
+import {
+ deploySharedHost,
+ resolveWatcherRoot,
+ runSharedHostCommand,
+} from './shared-host-deployment';
import { removeWatcherRoots, waitForWatcherStop } from './watcher-inspection';
const READ_SWITCH_AGENT_ID =
@@ -43,6 +50,105 @@ async function readSubagentSwitchId(
return stdout.trim();
}
+/**
+ * Where a remote watcher's auto-approve comes from when it is written. Several
+ * Consoles on one account write the same watcher from their own rows, so the
+ * shared value is the last explicit choice, kept in `auto-approve.json` rather
+ * than read from the saved spec, which any Console's row may have written.
+ *
+ * - `host`: that choice, when there is one; this Console's row is synced to it.
+ * - `this-console`: this Console's row, just changed by its user.
+ */
+export type AutoApproveSource = 'host' | 'this-console';
+
+type ConsoleRuntimeMode = 'full-access' | 'approval-required';
+
+const AUTO_APPROVE_CHOICE_FILE = 'auto-approve.json';
+
+/** Prints the auto-approve choice kept beside a watcher, or nothing. */
+const READ_AUTO_APPROVE_CHOICE = `const fs=require('node:fs');try{const c=JSON.parse(fs.readFileSync(require('node:path').join(process.argv[1],'${AUTO_APPROVE_CHOICE_FILE}'),'utf8'));console.log(c?.runtimeMode??'')}catch(e){if(e.code!=='ENOENT')throw e}`;
+
+/**
+ * Atomically keeps an auto-approve choice beside a watcher and, with a third
+ * argument `spec`, in its saved spec too. A missing root is a no-op: the first
+ * watcher is written from the row.
+ */
+const RECORD_AUTO_APPROVE_CHOICE = `const fs=require('node:fs'),path=require('node:path'),crypto=require('node:crypto');const [root,mode,spec]=process.argv.slice(1);if(!fs.existsSync(root))process.exit(0);const put=(file,data)=>{const tmp=file+'.'+crypto.randomUUID();fs.writeFileSync(tmp,JSON.stringify(data),{mode:0o600});fs.renameSync(tmp,file)};if(spec==='spec'){const f=path.join(root,'config.json');let c=null;try{c=JSON.parse(fs.readFileSync(f,'utf8'))}catch(e){if(e.code!=='ENOENT')throw e}if(c){c.start.input.runtimeMode=mode;put(f,c)}}put(path.join(root,'${AUTO_APPROVE_CHOICE_FILE}'),{runtimeMode:mode,at:new Date().toISOString()})`;
+
+function runtimeModeFor(autoApprove: boolean): ConsoleRuntimeMode {
+ return autoApprove ? 'full-access' : 'approval-required';
+}
+
+type HostContext = Awaited<ReturnType<typeof deploySharedHost>>['ctx'];
+
+async function readAutoApproveChoice(
+ ctx: HostContext,
+ root: string
+): Promise<ConsoleRuntimeMode | null> {
+ const { stdout } = await ctx.exec('node', ['-e', READ_AUTO_APPROVE_CHOICE, root]);
+ const mode = stdout.trim();
+ return mode === 'full-access' || mode === 'approval-required' ? mode : null;
+}
+
+/**
+ * Take the choice on the host for a watcher about to be written, and bring
+ * this Console's row in line so its toggle shows what the agent runs with.
+ */
+async function adoptHostAutoApprove(
+ agentId: string,
+ ctx: HostContext,
+ root: string,
+ config: SharedHostConfig
+): Promise<void> {
+ const chosen = await readAutoApproveChoice(ctx, root);
+ if (chosen === null || chosen === config.start.input.runtimeMode) return;
+ log.info('shared-watcher: taking auto-approve from the host, where another Console set it', {
+ agentId,
+ runtimeMode: chosen,
+ });
+ config.start.input.runtimeMode = chosen;
+ await updateAgent({ agentId, autoApprove: chosen === 'full-access' });
+}
+
+async function writeAutoApproveChoice(
+ agentId: string,
+ autoApprove: boolean,
+ what: 'spec' | 'choice-only'
+): Promise<void> {
+ const agent = await getAgentById(agentId);
+ if (!agent) throw new Error(`Agent ${agentId} does not exist.`);
+ if (!agent.switchAgentId) return;
+ const location = await getAgentLocation(agent);
+ const transport = locationTransport(location);
+ if (transport.kind !== 'ssh') return;
+ const { ctx, root } = await resolveWatcherRoot(transport, location.dir, agent.switchAgentId);
+ await ctx.exec('node', [
+ '-e',
+ RECORD_AUTO_APPROVE_CHOICE,
+ root,
+ runtimeModeFor(autoApprove),
+ what,
+ ]);
+}
+
+/**
+ * Keep a person's auto-approve choice on the host ahead of the push that
+ * rewrites the watcher, so a racing watcher write takes the new value. Takes
+ * the value rather than reading the row, so the caller can write it first.
+ */
+export async function keepAutoApproveChoice(agentId: string, autoApprove: boolean): Promise<void> {
+ await writeAutoApproveChoice(agentId, autoApprove, 'choice-only');
+}
+
+/** {@link keepAutoApproveChoice}, and the saved spec too, for a watcher that is
+ * not starting sessions and so has nothing about to rewrite its spec. */
+export async function recordAutoApproveOnHost(
+ agentId: string,
+ autoApprove: boolean
+): Promise<void> {
+ await writeAutoApproveChoice(agentId, autoApprove, 'spec');
+}
+
/**
* What an agent's controller should be doing.
*
@@ -60,16 +166,21 @@ export type ControllerState = { connected: boolean; spawning: boolean };
* This is the read of those two settings — callers that are not themselves
* deciding one of them should come through here rather than assemble a state.
*/
-export async function applyControllerState(agentId: string, intent: WatcherIntent): Promise<void> {
+export async function applyControllerState(
+ agentId: string,
+ intent: WatcherIntent,
+ autoApprove: AutoApproveSource
+): Promise<void> {
const [stopped, spawning] = await Promise.all([
listStoppedControllerAgentIds(),
listAutoSessionAgentIds(),
]);
const connected = !stopped.includes(agentId);
- await configureSharedWatcher(
+ await configureSharedWatcherFor(
agentId,
{ connected, spawning: connected && spawning.includes(agentId) },
- intent
+ intent,
+ { name: undefined, autoApprove }
);
}
@@ -94,11 +205,25 @@ export async function discardControllerState(agentId: string): Promise<void> {
await ctx.exec('node', ['-e', removeWatcherRoots, agent.switchAgentId]);
}
-export async function configureSharedWatcher(
+/** Which watcher — the agent's own, or its subagent `name`'s — and where its
+ * auto-approve comes from. */
+export type WatcherTarget = { name: string | undefined; autoApprove: AutoApproveSource };
+
+/** {@link configureSharedWatcherFor}, taking auto-approve from the host. */
+export function configureSharedWatcher(
agentId: string,
state: ControllerState,
intent: WatcherIntent,
name?: string
+): Promise<void> {
+ return configureSharedWatcherFor(agentId, state, intent, { name, autoApprove: 'host' });
+}
+
+export async function configureSharedWatcherFor(
+ agentId: string,
+ state: ControllerState,
+ intent: WatcherIntent,
+ { name, autoApprove }: WatcherTarget
): Promise<void> {
const agent = await getAgentById(agentId);
if (!agent) throw new Error(`Agent ${agentId} does not exist.`);
@@ -178,5 +303,11 @@ export async function configureSharedWatcher(
await ctx.exec('node', ['-e', waitForWatcherStop, root]);
return;
}
+ // A subagent's watcher runs with its parent's setting, which the parent's
+ // own watcher has already taken from the host.
+ const ownWatcher = !name || name === agent.name;
+ if (ownWatcher && autoApprove === 'host') {
+ await adoptHostAutoApprove(agentId, ctx, root, config);
+ }
await runSharedHostCommand(transport, { ctx, root, entrypoint }, config, '--ensure-watch', false);
}
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.test.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.test.ts
index fd48e02b9..d23f48ef8 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.test.ts
@@ -45,7 +45,7 @@ it.each(['update', 'restart'] as const)(
expect(mocks.apply).not.toHaveBeenCalled();
finish();
await pending;
- expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit');
+ expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit', 'host');
}
);
it('does not start a competing watcher if stopping fails', async () => {
@@ -57,7 +57,7 @@ it('restarts a controller whose automatic sessions are off', async () => {
// There is a controller to restart either way now, so this no longer refuses
// with "Automatic sessions are off" — and it does not turn them on to proceed.
await manageAgentSidecar('agent', 'restart');
- expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit');
+ expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit', 'host');
});
it('takes the connection away on stop and records that somebody did', async () => {
await manageAgentSidecar('agent', 'stop');
@@ -73,7 +73,7 @@ it('puts a stopped agent back on the air on start, and stops recording it stoppe
await manageAgentSidecar('agent', 'start');
expect(mocks.setStopped).toHaveBeenCalledWith('agent', false);
expect(mocks.configure).not.toHaveBeenCalled();
- expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit');
+ expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit', 'host');
});
it('refuses to update a local agent, which has no deployed sidecar', async () => {
mocks.remote.mockResolvedValue(null);
diff --git a/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.ts b/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.ts
index d707b1497..b559c8cfc 100644
--- a/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.ts
+++ b/console/apps/switch-console-desktop/src/main/core/sdk-host/sidecar-management.ts
@@ -29,5 +29,5 @@ export async function manageAgentSidecar(
// else took. Update and Restart stop first so the new bundle is what starts.
if (action !== 'start')
await configureSharedWatcher(agentId, { connected: false, spawning: false }, 'explicit');
- await applyControllerState(agentId, 'explicit');
+ await applyControllerState(agentId, 'explicit', 'host');
}
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.test.ts
index 135c757d0..fde1e8fa7 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.test.ts
@@ -59,7 +59,7 @@ beforeEach(() => {
it('gives every Switch-linked agent a controller at boot, and only those', async () => {
await autoSessionWatcher.initialize();
- expect(mocks.apply.mock.calls).toEqual([['linked', 'restore']]);
+ expect(mocks.apply.mock.calls).toEqual([['linked', 'restore', 'host']]);
});
it('restores controllers at boot rather than asking for one back', async () => {
@@ -67,7 +67,7 @@ it('restores controllers at boot rather than asking for one back', async () => {
// A displaced controller stood down because another client took this agent's
// connection. Booting Console is not somebody asking for it back, so the
// sweep must not clear that and start the two trading the connection.
- expect(mocks.apply).toHaveBeenCalledWith('linked', 'restore');
+ expect(mocks.apply).toHaveBeenCalledWith('linked', 'restore', 'host');
});
it('keeps sweeping when one agent’s controller cannot start', async () => {
@@ -77,7 +77,7 @@ it('keeps sweeping when one agent’s controller cannot start', async () => {
]);
mocks.apply.mockRejectedValueOnce(new Error('Host unreachable'));
await autoSessionWatcher.initialize();
- expect(mocks.apply).toHaveBeenCalledWith('linked', 'restore');
+ expect(mocks.apply).toHaveBeenCalledWith('linked', 'restore', 'host');
});
it('drops a deleted agent from the automatic-session mirror', async () => {
@@ -101,7 +101,7 @@ it('starts a controller whose host was unreachable at boot, once the host comes
await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledTimes(2));
// Still a restore: the host returning is not somebody asking for a connection
// another client has since taken, nor for a controller stopped by hand.
- expect(mocks.apply.mock.calls[1]).toEqual(['remote', 'restore']);
+ expect(mocks.apply.mock.calls[1]).toEqual(['remote', 'restore', 'host']);
});
it('leaves agents on other hosts alone when one host comes back', async () => {
@@ -118,7 +118,7 @@ it('leaves agents on other hosts alone when one host comes back', async () => {
mocks.reachability.announce({ current: { sshHost: 'host', status: 'reachable' } });
await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledTimes(1));
- expect(mocks.apply).toHaveBeenCalledWith('here', 'restore');
+ expect(mocks.apply).toHaveBeenCalledWith('here', 'restore', 'host');
});
it('sweeps again for a host that flaps while its recovery is still running', async () => {
@@ -144,7 +144,7 @@ it('sweeps again for a host that flaps while its recovery is still running', asy
arrive();
await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledTimes(3));
- expect(mocks.apply.mock.calls[2]).toEqual(['remote', 'restore']);
+ expect(mocks.apply.mock.calls[2]).toEqual(['remote', 'restore', 'host']);
});
it('does nothing for a host that has only just gone away', async () => {
@@ -161,7 +161,7 @@ it('does nothing for a host that has only just gone away', async () => {
it('applies the saved settings when asked to reconcile an agent', async () => {
await autoSessionWatcher.reconcile('agent');
- expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit');
+ expect(mocks.apply).toHaveBeenCalledWith('agent', 'explicit', 'host');
});
it('stops everything it hosts when Console closes', async () => {
@@ -183,7 +183,7 @@ it('does not hold one server’s agents behind another server’s update', async
});
const boot = autoSessionWatcher.initialize();
- await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledWith('elsewhere', 'restore'));
+ await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledWith('elsewhere', 'restore', 'host'));
finish();
await boot;
});
@@ -200,5 +200,5 @@ it('starts the controllers of a server once an update they were refused for fini
for (const listener of mocks.upgraded) listener('local');
await vi.waitFor(() => expect(mocks.apply).toHaveBeenCalledTimes(1));
- expect(mocks.apply).toHaveBeenCalledWith('upgraded', 'restore');
+ expect(mocks.apply).toHaveBeenCalledWith('upgraded', 'restore', 'host');
});
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.ts b/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.ts
index 1665e536f..71565d0be 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-rooms/auto-session-watcher.ts
@@ -85,7 +85,7 @@ class AutoSessionWatcher {
// holding, not a decision to reclaim a connection something else has
// since taken: one that stood down stays down until someone asks for it
// by name.
- await applyControllerState(agent.id, 'restore');
+ await applyControllerState(agent.id, 'restore', 'host');
} catch (error) {
log.error('Shared SDK watcher could not start', {
agentId: agent.id,
@@ -152,7 +152,7 @@ class AutoSessionWatcher {
// A host coming back is not somebody asking for a connection back, so
// a controller that stood down after a takeover stays down and one
// stopped by hand stays stopped.
- await applyControllerState(agent.id, 'restore');
+ await applyControllerState(agent.id, 'restore', 'host');
} catch (error) {
log.error('Shared SDK watcher could not start after its host returned', {
agentId: agent.id,
@@ -176,7 +176,7 @@ class AutoSessionWatcher {
* way — only Stop, or deleting the agent, takes an agent's connection away.
*/
reconcile(agentId: string): Promise<void> {
- return applyControllerState(agentId, 'explicit');
+ return applyControllerState(agentId, 'explicit', 'host');
}
reconcileSubagent(agentId: string, name: string, enabled: boolean): Promise<void> {
return configureSharedWatcher(
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.test.ts
new file mode 100644
index 000000000..175c53f03
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.test.ts
@@ -0,0 +1,140 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+const readSecrets = vi.hoisted(() => vi.fn());
+const getSessionCookie = vi.hoisted(() => vi.fn());
+const setSessionCookie = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const logWarn = vi.hoisted(() => vi.fn());
+
+vi.mock('electron', () => ({ BrowserWindow: vi.fn(), session: { fromPartition: vi.fn() } }));
+vi.mock('@main/core/managed-switch-server/secrets', () => ({ readSecrets }));
+vi.mock('@main/core/managed-switch-server/host/host-for-server', () => ({
+ managedServerSecretsKey: (server: { sshHost: string | null }) =>
+ server.sshHost
+ ? `remote-switch-server:${server.sshHost}:secrets`
+ : 'local-switch-server:secrets',
+}));
+vi.mock('./servers-store', () => ({ getSessionCookie, setSessionCookie }));
+const consoleIdentityHeaders = vi.hoisted(() =>
+ vi.fn(async (): Promise<Record<string, string>> => ({}))
+);
+vi.mock('./console-identity', () => ({ consoleIdentityHeaders }));
+vi.mock('@main/lib/logger', () => ({ log: { warn: logWarn, error: vi.fn(), info: vi.fn() } }));
+
+const { reauthenticateManagedServer, refreshSession } = await import('./auth');
+
+const REMOTE = {
+ id: 'srv-remote',
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+} as never;
+const EXTERNAL = {
+ id: 'srv-external',
+ name: 'Company server',
+ gatewayUrl: 'https://switch.example.com',
+ apiUrl: 'https://switch-api.example.com',
+ managed: false,
+ managementKind: null,
+ sshHost: null,
+} as never;
+
+const fetchMock = vi.fn();
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ vi.stubGlobal('fetch', fetchMock);
+});
+
+afterEach(() => {
+ vi.unstubAllGlobals();
+});
+
+describe('reauthenticateManagedServer', () => {
+ it('signs in with the stored admin password', async () => {
+ readSecrets.mockResolvedValue({ gatewayAdminPassword: 'stored-pw' });
+ fetchMock.mockResolvedValue({
+ status: 200,
+ ok: true,
+ headers: { getSetCookie: () => ['switch_auth=fresh-jwt; Path=/; HttpOnly'] },
+ json: async () => ({ id: 'u1' }),
+ });
+ getSessionCookie.mockResolvedValue('fresh-jwt');
+
+ expect(await reauthenticateManagedServer(REMOTE)).toBe('fresh-jwt');
+
+ expect(readSecrets).toHaveBeenCalledWith({ secretsKey: 'remote-switch-server:vm-1:secrets' });
+ const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit];
+ expect(url).toBe('http://localhost:41000/gateway/auth/login');
+ expect(JSON.parse(init.body as string)).toEqual({
+ email: '[email protected]',
+ password: 'stored-pw',
+ });
+ expect(setSessionCookie).toHaveBeenCalledWith('srv-remote', 'fresh-jwt');
+ });
+
+ it('gives up, and makes up no password, when none is stored', async () => {
+ readSecrets.mockResolvedValue(null);
+
+ expect(await reauthenticateManagedServer(REMOTE)).toBeNull();
+
+ expect(fetchMock).not.toHaveBeenCalled();
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringContaining('no stored credentials'),
+ expect.objectContaining({ server: 'srv-remote' })
+ );
+ });
+
+ it('holds no credentials for a server someone else runs', async () => {
+ expect(await reauthenticateManagedServer(EXTERNAL)).toBeNull();
+
+ expect(readSecrets).not.toHaveBeenCalled();
+ expect(fetchMock).not.toHaveBeenCalled();
+ });
+
+ it('falls back to the sign-in panel when the stored password is refused', async () => {
+ readSecrets.mockResolvedValue({ gatewayAdminPassword: 'stale-pw' });
+ fetchMock.mockResolvedValue({
+ status: 401,
+ ok: false,
+ headers: { getSetCookie: () => [] },
+ text: async () => '',
+ });
+
+ expect(await reauthenticateManagedServer(REMOTE)).toBeNull();
+ expect(setSessionCookie).not.toHaveBeenCalled();
+ });
+});
+
+describe('refreshSession', () => {
+ it('identifies the Console to a server it manages while renewing, and keeps the new session', async () => {
+ consoleIdentityHeaders.mockResolvedValueOnce({
+ 'X-Switch-Console-Id': 'console-1',
+ 'X-Switch-Console-Name': 'alice@laptop',
+ });
+ fetchMock.mockResolvedValue({
+ ok: true,
+ headers: { getSetCookie: () => ['switch_auth=fresh-jwt; Path=/; HttpOnly'] },
+ });
+
+ expect(await refreshSession(REMOTE, 'old-jwt')).toBe('fresh-jwt');
+
+ const [url, init] = fetchMock.mock.calls[0] as [string, RequestInit];
+ expect(url).toBe('http://localhost:41000/gateway/auth/refresh');
+ expect(init.headers).toMatchObject({
+ Cookie: 'switch_auth=old-jwt',
+ 'X-Switch-Console-Id': 'console-1',
+ 'X-Switch-Console-Name': 'alice@laptop',
+ });
+ expect(setSessionCookie).toHaveBeenCalledWith('srv-remote', 'fresh-jwt');
+ });
+
+ it('keeps the current session when renewal is refused', async () => {
+ fetchMock.mockResolvedValue({ ok: false, status: 401, headers: { getSetCookie: () => [] } });
+
+ expect(await refreshSession(REMOTE, 'old-jwt')).toBeNull();
+ expect(setSessionCookie).not.toHaveBeenCalled();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.ts
index 3db499561..ca8db7e51 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/auth.ts
@@ -3,9 +3,10 @@ import { err, ok, type Result } from '@switch-console/shared';
import { BrowserWindow, session as electronSession } from 'electron';
import { LOCAL_SERVER_ADMIN_EMAIL } from '@main/core/managed-switch-server/constants';
import { managedServerSecretsKey } from '@main/core/managed-switch-server/host/host-for-server';
-import { loadOrCreateSecrets } from '@main/core/managed-switch-server/secrets';
+import { readSecrets } from '@main/core/managed-switch-server/secrets';
import { log } from '@main/lib/logger';
import type { SwitchServer, SwitchUser } from '@shared/core/switch-servers/switch-servers';
+import { consoleIdentityHeaders } from './console-identity';
import { getSessionCookie, setSessionCookie } from './servers-store';
const SWITCH_AUTH_COOKIE = 'switch_auth';
@@ -44,11 +45,12 @@ export async function passwordLogin(
email: string,
password: string
): Promise<Result<SwitchUser, LoginError>> {
+ const identity = await consoleIdentityHeaders(server);
let response: Response;
try {
response = await fetch(gatewayUrl(server, '/auth/login'), {
method: 'POST',
- headers: { 'Content-Type': 'application/json', Accept: 'application/json' },
+ headers: { 'Content-Type': 'application/json', Accept: 'application/json', ...identity },
body: JSON.stringify({ email, password }),
redirect: 'manual',
signal: AbortSignal.timeout(30_000),
@@ -109,7 +111,11 @@ export async function refreshSession(
try {
response = await fetch(gatewayUrl(server, '/auth/refresh'), {
method: 'POST',
- headers: { Accept: 'application/json', Cookie: `${SWITCH_AUTH_COOKIE}=${currentJwt}` },
+ headers: {
+ Accept: 'application/json',
+ Cookie: `${SWITCH_AUTH_COOKIE}=${currentJwt}`,
+ ...(await consoleIdentityHeaders(server)),
+ },
redirect: 'manual',
signal: AbortSignal.timeout(30_000),
});
@@ -140,17 +146,26 @@ export async function refreshSession(
}
/**
- * Silent re-login for the managed local server. Switch Console generated that
- * server's admin password, so when its session is missing or expired we can
- * sign in again with no user interaction — the local server is meant to be
- * always signed in. Persists the fresh cookie (via `passwordLogin`) and returns
- * it for immediate reuse, or `null` when re-login failed (the caller then falls
- * back to the normal sign-in path). No-op for non-managed servers, whose
+ * Silent re-login for a managed server. Switch Console holds that server's
+ * admin password, so when its session is missing or expired we can sign in
+ * again with no user interaction — a managed server is meant to be always
+ * signed in. Persists the fresh cookie (via `passwordLogin`) and returns it for
+ * immediate reuse, or `null` when re-login failed (the caller then falls back
+ * to the normal sign-in path). No-op for non-managed servers, whose
* credentials Switch Console does not hold.
+ *
+ * Reads the stored credentials and never makes them: minted ones would match no
+ * running stack, and would then be kept as though they were the stack's.
*/
export async function reauthenticateManagedServer(server: SwitchServer): Promise<string | null> {
if (!server.managed) return null;
- const secrets = await loadOrCreateSecrets({ secretsKey: managedServerSecretsKey(server) });
+ const secrets = await readSecrets({ secretsKey: managedServerSecretsKey(server) });
+ if (secrets === null) {
+ log.warn('Managed Switch server has no stored credentials to sign in with', {
+ server: server.id,
+ });
+ return null;
+ }
const result = await passwordLogin(
server,
LOCAL_SERVER_ADMIN_EMAIL,
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.test.ts
new file mode 100644
index 000000000..9bb94dca8
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.test.ts
@@ -0,0 +1,156 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+const { store } = vi.hoisted(() => ({
+ store: { get: vi.fn(), setOrThrow: vi.fn() },
+}));
+const os = vi.hoisted(() => ({
+ hostname: vi.fn(() => 'alice-laptop.local'),
+ userInfo: vi.fn(() => ({ username: 'alice' })),
+}));
+const logWarn = vi.hoisted(() => vi.fn());
+
+vi.mock('@main/db/kv', () => ({
+ KV: class {
+ get = store.get;
+ setOrThrow = store.setOrThrow;
+ },
+}));
+vi.mock('node:os', () => os);
+vi.mock('@main/lib/logger', () => ({ log: { warn: logWarn } }));
+
+const CANONICAL_UUID =
+ /^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$/;
+
+/** The id is memoised, so each case needs its own instance of the module. */
+async function loadModule() {
+ vi.resetModules();
+ return import('./console-identity');
+}
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ store.get.mockResolvedValue(null);
+ store.setOrThrow.mockResolvedValue(undefined);
+ os.hostname.mockReturnValue('alice-laptop.local');
+ os.userInfo.mockReturnValue({ username: 'alice' });
+});
+
+describe('the console id', () => {
+ it('is a random UUID created and stored the first time it is needed', async () => {
+ const { getConsoleIdentity } = await loadModule();
+
+ const { id } = await getConsoleIdentity();
+
+ expect(id).toMatch(CANONICAL_UUID);
+ expect(store.setOrThrow).toHaveBeenCalledWith('consoleId', id);
+ });
+
+ it('is the stored one when there is one, and is not rewritten', async () => {
+ store.get.mockResolvedValue('stored-console-id');
+ const { getConsoleIdentity } = await loadModule();
+
+ await expect(getConsoleIdentity()).resolves.toMatchObject({ id: 'stored-console-id' });
+ expect(store.setOrThrow).not.toHaveBeenCalled();
+ });
+
+ it('is created once however many callers ask at the same time', async () => {
+ const { getConsoleIdentity } = await loadModule();
+
+ const ids = await Promise.all([getConsoleIdentity(), getConsoleIdentity()]);
+
+ expect(ids[0]!.id).toBe(ids[1]!.id);
+ expect(store.setOrThrow).toHaveBeenCalledOnce();
+ });
+
+ it('is retried after a failed write rather than failing forever', async () => {
+ store.setOrThrow.mockRejectedValueOnce(new Error('disk full'));
+ const { getConsoleIdentity } = await loadModule();
+
+ await expect(getConsoleIdentity()).rejects.toThrow('disk full');
+ await expect(getConsoleIdentity()).resolves.toMatchObject({ id: expect.any(String) });
+ });
+});
+
+describe('the console name', () => {
+ it('is user@host of the desktop', async () => {
+ const { consoleName } = await loadModule();
+
+ expect(consoleName()).toBe('[email protected]');
+ });
+
+ it('is reduced to what the server will keep, so both show the same name', async () => {
+ os.userInfo.mockReturnValue({ username: 'Ada Lovelace' });
+ os.hostname.mockReturnValue('lab machine;rm -rf');
+ const { consoleName } = await loadModule();
+
+ expect(consoleName()).toBe('Ada-Lovelace@lab-machinerm--rf');
+ });
+
+ it('says unknown for a user or host name nothing of which the server would keep', async () => {
+ os.userInfo.mockReturnValue({ username: '☃' });
+ os.hostname.mockReturnValue('☃☃');
+ const { consoleName } = await loadModule();
+
+ expect(consoleName()).toBe('unknown@unknown');
+ });
+
+ it('is capped at the length the server keeps', async () => {
+ os.hostname.mockReturnValue('h'.repeat(200));
+ const { consoleName } = await loadModule();
+
+ expect(consoleName()).toHaveLength(64);
+ });
+
+ it('warns about an unreadable desktop user once, however often the name is asked for', async () => {
+ os.userInfo.mockImplementation(() => {
+ throw new Error('no passwd entry');
+ });
+ const { consoleName } = await loadModule();
+
+ for (let i = 0; i < 5; i++) consoleName();
+
+ expect(logWarn).toHaveBeenCalledOnce();
+ });
+
+ it('says unknown, and warns, when the desktop user cannot be read', async () => {
+ os.userInfo.mockImplementation(() => {
+ throw new Error('no passwd entry');
+ });
+ const { consoleName } = await loadModule();
+
+ expect(consoleName()).toBe('[email protected]');
+ expect(logWarn).toHaveBeenCalledOnce();
+ });
+});
+
+describe('consoleIdentityHeaders', () => {
+ it('identifies the Console to a server it manages', async () => {
+ store.get.mockResolvedValue('stored-console-id');
+ const { consoleIdentityHeaders } = await loadModule();
+
+ await expect(consoleIdentityHeaders({ managed: true })).resolves.toEqual({
+ 'X-Switch-Console-Id': 'stored-console-id',
+ 'X-Switch-Console-Name': '[email protected]',
+ });
+ });
+
+ it('sends no identity, and says so, rather than failing the call when the id cannot be had', async () => {
+ store.get.mockResolvedValue(null);
+ store.setOrThrow.mockRejectedValueOnce(new Error('disk full'));
+ const { consoleIdentityHeaders } = await loadModule();
+
+ await expect(consoleIdentityHeaders({ managed: true })).resolves.toEqual({});
+ expect(logWarn).toHaveBeenCalledWith(
+ expect.stringMatching(/will not be told which Console/),
+ expect.objectContaining({ error: expect.any(Error) })
+ );
+ });
+
+ it('tells a server someone else runs nothing, and creates no id for it', async () => {
+ const { consoleIdentityHeaders } = await loadModule();
+
+ await expect(consoleIdentityHeaders({ managed: false })).resolves.toEqual({});
+ expect(store.get).not.toHaveBeenCalled();
+ expect(store.setOrThrow).not.toHaveBeenCalled();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.ts
new file mode 100644
index 000000000..297a9ba07
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/console-identity.ts
@@ -0,0 +1,105 @@
+import { randomUUID } from 'node:crypto';
+import { hostname, userInfo } from 'node:os';
+import { KV } from '@main/db/kv';
+import { log } from '@main/lib/logger';
+import type { SwitchServer } from '@shared/core/switch-servers/switch-servers';
+
+/**
+ * Who this copy of Switch Console is, to the servers it manages (CHOO-2893).
+ * Everyone on a shared VM signs in as the stack's one admin, so the Console
+ * identifies itself on every call and in the register kept beside the stack.
+ *
+ * Deliberately not the telemetry install id, which exists only with consent
+ * and is promised to go nowhere but the telemetry relay.
+ */
+
+/** Matches what `request_context.py` in core keeps of the header, so the
+ * Console shows the same name the server records. */
+const NAME_CHARACTERS = /[^A-Za-z0-9._@+-]/g;
+const MAX_NAME_LENGTH = 64;
+
+export const CONSOLE_ID_HEADER = 'X-Switch-Console-Id';
+export const CONSOLE_NAME_HEADER = 'X-Switch-Console-Name';
+
+export type ConsoleIdentity = {
+ /** Random, created on first use and stored locally; not derived from the
+ * machine or the user. */
+ id: string;
+ /** `user@host` of the desktop this Console runs on, for people to read. */
+ name: string;
+};
+
+const store = new KV<{ consoleId: string }>('console-identity');
+
+let pendingId: Promise<string> | null = null;
+
+async function loadId(): Promise<string> {
+ const existing = await store.get('consoleId');
+ if (existing) return existing;
+ const created = randomUUID();
+ await store.setOrThrow('consoleId', created);
+ return created;
+}
+
+function consoleId(): Promise<string> {
+ pendingId ??= loadId().catch((error: unknown) => {
+ // A failed write must not poison every later call with a rejected promise.
+ pendingId = null;
+ throw error;
+ });
+ return pendingId;
+}
+
+/** One half of `user@host`, reduced to what the server keeps — or `unknown`
+ * when nothing of it survives, rather than a bare `@` telling nobody apart. */
+function namePart(raw: string): string {
+ return raw.replace(/\s+/g, '-').replace(NAME_CHARACTERS, '') || 'unknown';
+}
+
+/** Warn once: the name is read on every managed-server request. */
+let userNameMissingReported = false;
+
+function desktopUser(): string {
+ try {
+ return userInfo().username;
+ } catch (error) {
+ // A user with no passwd entry (some containers) has no name to give.
+ if (!userNameMissingReported) {
+ userNameMissingReported = true;
+ log.warn('console-identity: could not read the desktop user name', { error });
+ }
+ return 'unknown';
+ }
+}
+
+/** The name this Console shows to others: `user@host`, reduced to what the
+ * server will keep. Read fresh, since a renamed machine should say so. */
+export function consoleName(): string {
+ return `${namePart(desktopUser())}@${namePart(hostname())}`.slice(0, MAX_NAME_LENGTH);
+}
+
+export async function getConsoleIdentity(): Promise<ConsoleIdentity> {
+ return { id: await consoleId(), name: consoleName() };
+}
+
+/**
+ * The headers that identify this Console to `server`, or none. Only a managed
+ * server is told: that is where people share one sign-in. An id that cannot be
+ * read is logged and sends no headers rather than failing the call.
+ */
+export async function consoleIdentityHeaders(
+ server: Pick<SwitchServer, 'managed'>
+): Promise<Record<string, string>> {
+ if (!server.managed) return {};
+ try {
+ const identity = await getConsoleIdentity();
+ return { [CONSOLE_ID_HEADER]: identity.id, [CONSOLE_NAME_HEADER]: identity.name };
+ } catch (error) {
+ log.warn(
+ "console-identity: could not read this Console's id, so the server will not be told " +
+ 'which Console this request is from',
+ { error }
+ );
+ return {};
+ }
+}
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/create-bridge.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/create-bridge.test.ts
index 9f5331552..4e9c28984 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/create-bridge.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/create-bridge.test.ts
@@ -6,14 +6,17 @@ const reauthenticateManagedServer = vi.hoisted(() => vi.fn());
const managedServerHostBlocked = vi.hoisted(() => vi.fn(() => null));
const managedServerStoppedPhase = vi.hoisted(() => vi.fn(() => null));
+const noteManagedServerUnanswered = vi.hoisted(() => vi.fn());
vi.mock('@main/core/managed-switch-server/managed-server-status', () => ({
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
}));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ refreshSession, reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({ consoleIdentityHeaders: async () => ({}) }));
const { createBridgeOnServer } = await import('./create-bridge');
const { fetchBridgeTypes } = await import('./gateway-client');
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/create-room.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/create-room.test.ts
index 305c70662..61135081e 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/create-room.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/create-room.test.ts
@@ -6,14 +6,17 @@ const reauthenticateManagedServer = vi.hoisted(() => vi.fn());
const managedServerHostBlocked = vi.hoisted(() => vi.fn(() => null));
const managedServerStoppedPhase = vi.hoisted(() => vi.fn(() => null));
+const noteManagedServerUnanswered = vi.hoisted(() => vi.fn());
vi.mock('@main/core/managed-switch-server/managed-server-status', () => ({
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
}));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ refreshSession, reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({ consoleIdentityHeaders: async () => ({}) }));
const { createRoomOnServer } = await import('./create-room');
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.test.ts
index a8e84aecb..374b18f79 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.test.ts
@@ -18,14 +18,22 @@ const managedServerHostBlocked = vi.hoisted(() => vi.fn<() => HostReachability |
const managedServerStoppedPhase = vi.hoisted(() =>
vi.fn<() => LocalServerPhase | null>(() => null)
);
+const noteManagedServerUnanswered = vi.hoisted(() => vi.fn());
vi.mock('@main/core/managed-switch-server/managed-server-status', () => ({
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
}));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ refreshSession, reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({
+ consoleIdentityHeaders: async (server: { managed: boolean }) =>
+ server.managed
+ ? { 'X-Switch-Console-Id': 'console-1', 'X-Switch-Console-Name': 'alice@laptop' }
+ : {},
+}));
const {
createRoom,
@@ -164,6 +172,60 @@ describe('gatewayFetch proactive session renewal', () => {
});
});
+describe('gatewayFetch console attribution', () => {
+ beforeEach(() => {
+ vi.clearAllMocks();
+ vi.stubGlobal('fetch', fetchMock);
+ fetchMock.mockImplementation(async () => okMeResponse());
+ getSessionCookie.mockResolvedValue(makeJwt(2 * 60 * 60));
+ });
+
+ afterEach(() => {
+ vi.unstubAllGlobals();
+ });
+
+ function headersOf(call: unknown[] | undefined): Record<string, string> {
+ return ((call?.[1] as RequestInit | undefined)?.headers ?? {}) as Record<string, string>;
+ }
+
+ it('says which Console is calling a server it manages', async () => {
+ await fetchMe(MANAGED);
+
+ expect(headersOf(fetchMock.mock.calls[0])).toMatchObject({
+ 'X-Switch-Console-Id': 'console-1',
+ 'X-Switch-Console-Name': 'alice@laptop',
+ });
+ });
+
+ it('keeps saying so on the retry after a silent re-login', async () => {
+ reauthenticateManagedServer.mockResolvedValue(makeJwt(24 * 60 * 60));
+ fetchMock.mockResolvedValueOnce(unauthorizedResponse());
+
+ await fetchMe(MANAGED);
+
+ expect(fetchMock).toHaveBeenCalledTimes(2);
+ expect(headersOf(fetchMock.mock.calls[1])).toMatchObject({
+ 'X-Switch-Console-Id': 'console-1',
+ });
+ });
+
+ it('asks for the host to be read again when a managed server does not answer', async () => {
+ fetchMock.mockRejectedValueOnce(new TypeError('fetch failed'));
+
+ await expect(fetchMe(MANAGED)).rejects.toThrow(/Could not reach/);
+
+ expect(noteManagedServerUnanswered).toHaveBeenCalledExactlyOnceWith(MANAGED);
+ });
+
+ it('tells a server someone else runs nothing about the desktop', async () => {
+ await fetchMe(SERVER);
+
+ const headers = headersOf(fetchMock.mock.calls[0]);
+ expect(headers).not.toHaveProperty('X-Switch-Console-Id');
+ expect(headers).not.toHaveProperty('X-Switch-Console-Name');
+ });
+});
+
describe('gatewayFetch managed-server silent re-auth', () => {
beforeEach(() => {
vi.clearAllMocks();
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.ts
index 31fdcd62b..9c3c761ee 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-client.ts
@@ -2,6 +2,7 @@ import type { KnownAgentType } from '@main/core/agents/known-agent-type';
import {
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
} from '@main/core/managed-switch-server/managed-server-status';
import { ManagedServerStoppedError } from '@shared/core/managed-switch-server/managed-switch-server';
import { HostUnreachableError } from '@shared/core/remote-hosts/reachability';
@@ -27,6 +28,7 @@ import type {
SwitchUser,
} from '@shared/core/switch-servers/switch-servers';
import { reauthenticateManagedServer, refreshSession } from './auth';
+import { consoleIdentityHeaders } from './console-identity';
import { getSessionCookie } from './servers-store';
/** The gateway management API is mounted under `/gateway` on the server. */
@@ -166,8 +168,9 @@ async function gatewayFetch(
const stopped = managedServerStoppedPhase(server);
if (stopped) throw new ManagedServerStoppedError(server, stopped);
+ const identity = await consoleIdentityHeaders(server);
const sendOnce = async (cookie: string | null): Promise<Response> => {
- const headers: Record<string, string> = { Accept: 'application/json' };
+ const headers: Record<string, string> = { Accept: 'application/json', ...identity };
if (options.body !== undefined) {
headers['Content-Type'] = 'application/json';
}
@@ -184,6 +187,7 @@ async function gatewayFetch(
signal: AbortSignal.timeout(30_000),
});
} catch (cause) {
+ noteManagedServerUnanswered(server);
throw new GatewayError(
'network',
`Could not reach ${server.gatewayUrl}: ${cause instanceof Error ? cause.message : String(cause)}`
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.test.ts
index 1d672f069..362754733 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.test.ts
@@ -3,7 +3,10 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
const getSessionCookie = vi.hoisted(() => vi.fn());
const reauthenticateManagedServer = vi.hoisted(() => vi.fn());
const cookiesSet = vi.hoisted(() => vi.fn(async () => {}));
-const fromPartition = vi.hoisted(() => vi.fn(() => ({ cookies: { set: cookiesSet } })));
+const onBeforeSendHeaders = vi.hoisted(() => vi.fn());
+const fromPartition = vi.hoisted(() =>
+ vi.fn(() => ({ cookies: { set: cookiesSet }, webRequest: { onBeforeSendHeaders } }))
+);
const loadURL = vi.hoisted(() => vi.fn(async () => {}));
const BrowserWindow = vi.hoisted(() =>
vi.fn(function () {
@@ -14,6 +17,12 @@ const BrowserWindow = vi.hoisted(() =>
vi.mock('electron', () => ({ BrowserWindow, session: { fromPartition } }));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({
+ consoleIdentityHeaders: async (server: { managed: boolean }) =>
+ server.managed
+ ? { 'X-Switch-Console-Id': 'console-1', 'X-Switch-Console-Name': 'alice@laptop' }
+ : {},
+}));
vi.mock('@main/lib/logger', () => ({ log: { warn: vi.fn(), error: vi.fn() } }));
const { openAuthenticatedGatewayPage } = await import('./gateway-web');
@@ -87,4 +96,37 @@ describe('openAuthenticatedGatewayPage', () => {
expect(cookiesSet).not.toHaveBeenCalled();
expect(loadURL).toHaveBeenCalledWith('http://127.0.0.1:8080/');
});
+
+ it('marks the dashboard’s requests to a managed gateway with this Console', async () => {
+ getSessionCookie.mockResolvedValue('stored-jwt');
+
+ await openAuthenticatedGatewayPage(MANAGED, 'http://127.0.0.1:8080/');
+
+ expect(onBeforeSendHeaders).toHaveBeenCalledOnce();
+ const [filter, listener] = onBeforeSendHeaders.mock.calls[0] as [
+ { urls: string[] },
+ (
+ details: { requestHeaders: Record<string, string> },
+ callback: (response: { requestHeaders: Record<string, string> }) => void
+ ) => void,
+ ];
+ expect(filter.urls).toEqual(['http://127.0.0.1:8080/*']);
+ const callback = vi.fn();
+ listener({ requestHeaders: { Accept: 'text/html' } }, callback);
+ expect(callback).toHaveBeenCalledWith({
+ requestHeaders: {
+ Accept: 'text/html',
+ 'X-Switch-Console-Id': 'console-1',
+ 'X-Switch-Console-Name': 'alice@laptop',
+ },
+ });
+ });
+
+ it('adds nothing to the requests of a server someone else runs', async () => {
+ getSessionCookie.mockResolvedValue('stored-jwt');
+
+ await openAuthenticatedGatewayPage(SERVER, 'http://127.0.0.1:8080/');
+
+ expect(onBeforeSendHeaders).not.toHaveBeenCalled();
+ });
});
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.ts
index 90cffecc9..6c6bf02cf 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/gateway-web.ts
@@ -2,6 +2,7 @@ import { BrowserWindow, session as electronSession } from 'electron';
import { log } from '@main/lib/logger';
import type { SwitchServer } from '@shared/core/switch-servers/switch-servers';
import { reauthenticateManagedServer } from './auth';
+import { consoleIdentityHeaders } from './console-identity';
import { getSessionCookie } from './servers-store';
const SWITCH_AUTH_COOKIE = 'switch_auth';
@@ -58,6 +59,15 @@ export async function openAuthenticatedGatewayPage(
}
}
+ // Tag the dashboard's requests with this Console too, confined to the
+ // gateway's origin for the same reason as the cookie.
+ const identity = await consoleIdentityHeaders(server);
+ if (Object.keys(identity).length > 0) {
+ ses.webRequest.onBeforeSendHeaders({ urls: [`${gatewayOrigin}/*`] }, (details, callback) => {
+ callback({ requestHeaders: { ...details.requestHeaders, ...identity } });
+ });
+ }
+
const win = new BrowserWindow({
width: 1280,
height: 860,
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
index 2102d693d..7a746c190 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.db.test.ts
@@ -37,7 +37,7 @@ vi.mock('@main/core/telemetry/telemetry-service', () => ({
}));
// Imported after the mocks so the module binds to the mocked db + secrets store.
-const { addServer, ensureManagedServer, removeServer, renameServer } =
+const { addServer, assertManagedServerUrlFree, ensureManagedServer, removeServer, renameServer } =
await import('./servers-store');
describe('servers-store: rename & delete', () => {
@@ -115,6 +115,172 @@ describe('servers-store: rename & delete', () => {
});
});
+ describe('ensureManagedServer URL clashes', () => {
+ async function insertServer(values: Partial<typeof switchServers.$inferInsert>) {
+ await fixture.db.insert(switchServers).values({
+ id: 'x',
+ name: 'x',
+ gatewayUrl: 'http://localhost:1',
+ apiUrl: 'http://localhost:2',
+ ...values,
+ });
+ }
+
+ it('never takes over another managed server’s row by URL', async () => {
+ await insertServer({
+ id: 'local-1',
+ name: 'My local server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ managed: true,
+ managementKind: 'local',
+ });
+
+ await expect(
+ ensureManagedServer(
+ {
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ },
+ { kind: 'remote', sshHost: 'vm-1' }
+ )
+ ).rejects.toThrow(
+ /http:\/\/localhost:41000 is already the address of “My local server” \(the server Switch Console runs on this computer\).*on vm-1/
+ );
+ const [local] = await fixture.db
+ .select()
+ .from(switchServers)
+ .where(eq(switchServers.id, 'local-1'));
+ expect(local).toMatchObject({ managementKind: 'local', sshHost: null });
+ });
+
+ it('says a start would clash before it runs, writing nothing', async () => {
+ await insertServer({
+ id: 'local-1',
+ name: 'My local server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ managed: true,
+ managementKind: 'local',
+ });
+
+ await expect(
+ assertManagedServerUrlFree('http://localhost:41000/', { kind: 'remote', sshHost: 'vm-1' })
+ ).rejects.toThrow(/already the address of “My local server”/);
+ await expect(
+ assertManagedServerUrlFree('http://localhost:42000', { kind: 'remote', sshHost: 'vm-1' })
+ ).resolves.toBeUndefined();
+ expect(await fixture.db.select().from(switchServers)).toHaveLength(1);
+ });
+
+ it('never gives the local server a remote server’s row by URL either', async () => {
+ await insertServer({
+ id: 'remote-1',
+ name: 'Team server',
+ gatewayUrl: 'http://localhost:41000',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+ });
+
+ await expect(
+ ensureManagedServer(
+ { name: 'Local', gatewayUrl: 'http://localhost:41000', apiUrl: 'http://localhost:41001' },
+ { kind: 'local' }
+ )
+ ).rejects.toThrow(/on this computer/);
+ });
+
+ it('never takes over the row of a server on another host', async () => {
+ await insertServer({
+ id: 'remote-2',
+ name: 'Other VM',
+ gatewayUrl: 'http://localhost:41000',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-2',
+ });
+
+ await expect(
+ ensureManagedServer(
+ { name: 'Team', gatewayUrl: 'http://localhost:41000', apiUrl: 'http://localhost:41001' },
+ { kind: 'remote', sshHost: 'vm-1' }
+ )
+ ).rejects.toThrow(/runs on vm-2/);
+ });
+
+ it('still adopts a server someone registered by hand at the same address', async () => {
+ await insertServer({
+ id: 'external-1',
+ name: 'Tunnel to the VM',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ });
+
+ const adopted = await ensureManagedServer(
+ { name: 'Team', gatewayUrl: 'http://localhost:41000', apiUrl: 'http://localhost:41001' },
+ { kind: 'remote', sshHost: 'vm-1' }
+ );
+
+ expect(adopted).toMatchObject({
+ id: 'external-1',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+ });
+ });
+
+ it('refuses to move a server onto an address another record holds, in words', async () => {
+ await insertServer({
+ id: 'remote-1',
+ name: 'Team',
+ gatewayUrl: 'http://localhost:3300',
+ apiUrl: 'http://localhost:8000',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+ });
+ await insertServer({
+ id: 'external-1',
+ name: 'Company server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ });
+
+ // Without the check this is a raw unique-index failure.
+ await expect(
+ ensureManagedServer(
+ { name: 'Team', gatewayUrl: 'http://localhost:41000', apiUrl: 'http://localhost:41001' },
+ { kind: 'remote', sshHost: 'vm-1' }
+ )
+ ).rejects.toThrow(/already the address of “Company server”/);
+ });
+
+ it('follows a server to new ports when nothing else holds them', async () => {
+ await insertServer({
+ id: 'remote-1',
+ name: 'Team',
+ gatewayUrl: 'http://localhost:3300',
+ apiUrl: 'http://localhost:8000',
+ managed: true,
+ managementKind: 'remote',
+ sshHost: 'vm-1',
+ });
+
+ const moved = await ensureManagedServer(
+ { name: 'ignored', gatewayUrl: 'http://localhost:41000', apiUrl: 'http://localhost:41001' },
+ { kind: 'remote', sshHost: 'vm-1' }
+ );
+
+ expect(moved).toMatchObject({
+ id: 'remote-1',
+ name: 'Team',
+ gatewayUrl: 'http://localhost:41000',
+ });
+ });
+ });
+
describe('ensureManagedServer telemetry', () => {
it('reports a new local managed server once', async () => {
await ensureManagedServer(
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
index 9c027fa26..164f2d2ee 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/servers-store.ts
@@ -107,12 +107,67 @@ export async function listManagedServers(): Promise<SwitchServer[]> {
return rows.map(mapRow);
}
+/** Where a managed server's stack runs, in the words the UI uses. */
+function managedPlace(server: SwitchServer): string {
+ return server.managementKind === 'remote' && server.sshHost
+ ? `on ${server.sshHost}`
+ : 'on this computer';
+}
+
+/** Two server records cannot share a gateway URL, and a shared remote stack's
+ * mirrored `localhost` port may be one this Console already gave another server. */
+export class ManagedServerUrlConflictError extends Error {
+ constructor(gatewayUrl: string, holder: SwitchServer, target: string) {
+ super(
+ `${gatewayUrl} is already the address of “${holder.name}”` +
+ (holder.managed ? ` (the server Switch Console runs ${managedPlace(holder)})` : '') +
+ `, and the server ${target} uses the same port. Switch Console reaches a remote ` +
+ `server through the same port number on this computer, so both cannot be registered. ` +
+ `Remove or disconnect “${holder.name}”, then try again.`
+ );
+ this.name = 'ManagedServerUrlConflictError';
+ }
+}
+
+/** The record `ref` already has, and the one at `gatewayUrl`, refusing a
+ * clash as {@link ensureManagedServer} describes. */
+async function managedServerSlot(
+ gatewayUrl: string,
+ ref: ManagedServerRef
+): Promise<{ existingForTarget: SwitchServer | null; atUrl: SwitchServer | null }> {
+ const existingForTarget =
+ ref.kind === 'remote' ? await getRemoteManagedServer(ref.sshHost) : await getManagedServer();
+ const atUrl = await getServerByGatewayUrl(gatewayUrl);
+ if (atUrl && atUrl.id !== existingForTarget?.id && (existingForTarget || atUrl.managed)) {
+ throw new ManagedServerUrlConflictError(
+ gatewayUrl,
+ atUrl,
+ ref.kind === 'remote' ? `on ${ref.sshHost}` : 'on this computer'
+ );
+ }
+ return { existingForTarget, atUrl };
+}
+
+/** Throw {@link ManagedServerUrlConflictError} when {@link ensureManagedServer}
+ * would, without writing anything, so a start can check before changing the stack. */
+export async function assertManagedServerUrlFree(
+ gatewayUrl: string,
+ ref: ManagedServerRef
+): Promise<void> {
+ await managedServerSlot(normaliseUrl(gatewayUrl), ref);
+}
+
/**
* Upsert a managed server record for the given target (the single local stack,
* or the stack on a specific remote host). Reuses the existing row for that
* target if there is one (updating its URLs, which change when ports are
- * repicked), else adopts a row already at this gateway URL, else inserts. Keeps
- * exactly one row per managed target rather than duplicating on URL changes.
+ * repicked), else adopts an external row already at this gateway URL, else
+ * inserts. Keeps exactly one row per managed target rather than duplicating on
+ * URL changes.
+ *
+ * Another managed target's row is never adopted by URL: taking it over would
+ * silently repoint that server and its agents at a different stack. Clashes
+ * throw {@link ManagedServerUrlConflictError}.
*/
export async function ensureManagedServer(
params: AddServerParams,
@@ -122,9 +177,8 @@ export async function ensureManagedServer(
const apiUrl = normaliseUrl(params.apiUrl);
const managementKind = ref.kind;
const sshHost = ref.kind === 'remote' ? ref.sshHost : null;
- const existingForTarget =
- ref.kind === 'remote' ? await getRemoteManagedServer(ref.sshHost) : await getManagedServer();
- const existing = existingForTarget ?? (await getServerByGatewayUrl(gatewayUrl));
+ const { existingForTarget, atUrl } = await managedServerSlot(gatewayUrl, ref);
+ const existing = existingForTarget ?? atUrl;
if (existing) {
// Preserve the stored name on restart: the name is set once at creation and
// then owned by the user (rename). Only the URLs/kind refresh when a managed
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/update-bridge.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/update-bridge.test.ts
index fd62ec8bf..ac43e82ca 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/update-bridge.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/update-bridge.test.ts
@@ -6,14 +6,17 @@ const reauthenticateManagedServer = vi.hoisted(() => vi.fn());
const managedServerHostBlocked = vi.hoisted(() => vi.fn(() => null));
const managedServerStoppedPhase = vi.hoisted(() => vi.fn(() => null));
+const noteManagedServerUnanswered = vi.hoisted(() => vi.fn());
vi.mock('@main/core/managed-switch-server/managed-server-status', () => ({
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
}));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ refreshSession, reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({ consoleIdentityHeaders: async () => ({}) }));
const { updateBridgeOnServer } = await import('./update-bridge');
diff --git a/console/apps/switch-console-desktop/src/main/core/switch-servers/update-template.test.ts b/console/apps/switch-console-desktop/src/main/core/switch-servers/update-template.test.ts
index e799f8eca..68bc422ab 100644
--- a/console/apps/switch-console-desktop/src/main/core/switch-servers/update-template.test.ts
+++ b/console/apps/switch-console-desktop/src/main/core/switch-servers/update-template.test.ts
@@ -5,13 +5,16 @@ const refreshSession = vi.hoisted(() => vi.fn());
const reauthenticateManagedServer = vi.hoisted(() => vi.fn());
const managedServerHostBlocked = vi.hoisted(() => vi.fn(() => null));
const managedServerStoppedPhase = vi.hoisted(() => vi.fn(() => null));
+const noteManagedServerUnanswered = vi.hoisted(() => vi.fn());
vi.mock('@main/core/managed-switch-server/managed-server-status', () => ({
managedServerHostBlocked,
managedServerStoppedPhase,
+ noteManagedServerUnanswered,
}));
vi.mock('./servers-store', () => ({ getSessionCookie }));
vi.mock('./auth', () => ({ refreshSession, reauthenticateManagedServer }));
+vi.mock('./console-identity', () => ({ consoleIdentityHeaders: async () => ({}) }));
const { updateTemplate, GatewayError } = await import('./gateway-client');
diff --git a/console/apps/switch-console-desktop/src/main/lib/rpc-logging.test.ts b/console/apps/switch-console-desktop/src/main/lib/rpc-logging.test.ts
new file mode 100644
index 000000000..7b5b240e3
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/main/lib/rpc-logging.test.ts
@@ -0,0 +1,58 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+import {
+ ManagedServerStoppedError,
+ ServerBusyError,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+
+const log = vi.hoisted(() => ({ debug: vi.fn(), error: vi.fn(), info: vi.fn(), warn: vi.fn() }));
+vi.mock('./logger', () => ({ log }));
+vi.mock('./log-context', () => ({
+ runWithLogContext: (_context: unknown, run: () => unknown) => run(),
+}));
+
+const { withRPCLogContext } = await import('./rpc-logging');
+
+beforeEach(() => vi.clearAllMocks());
+
+async function fail(error: unknown): Promise<void> {
+ await expect(
+ withRPCLogContext('remoteSwitchServer.stop', ['vm-1'], () => Promise.reject(error))
+ ).rejects.toBe(error);
+}
+
+describe('logging a failed call', () => {
+ it('logs a real failure as an error, and still fails the call', async () => {
+ await fail(new Error('compose down failed'));
+
+ expect(log.error).toHaveBeenCalledWith(
+ 'RPC handler failed',
+ expect.objectContaining({ component: 'rpc:remoteSwitchServer.stop' })
+ );
+ expect(log.debug).not.toHaveBeenCalled();
+ });
+
+ it('keeps a refusal because another Console is changing the server out of the error log', async () => {
+ await fail(
+ new ServerBusyError(
+ {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 30,
+ expiresInSeconds: 90,
+ },
+ 'vm-1'
+ )
+ );
+
+ expect(log.debug).toHaveBeenCalledWith('RPC handler failed', expect.anything());
+ expect(log.error).not.toHaveBeenCalled();
+ });
+
+ it('keeps a call to a stopped server out of the error log', async () => {
+ await fail(new ManagedServerStoppedError({ id: 'srv-1', name: 'Team server' }, 'stopped'));
+
+ expect(log.debug).toHaveBeenCalled();
+ expect(log.error).not.toHaveBeenCalled();
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/main/lib/rpc-logging.ts b/console/apps/switch-console-desktop/src/main/lib/rpc-logging.ts
index 769ccba05..1ade1e61b 100644
--- a/console/apps/switch-console-desktop/src/main/lib/rpc-logging.ts
+++ b/console/apps/switch-console-desktop/src/main/lib/rpc-logging.ts
@@ -1,4 +1,7 @@
-import { ManagedServerStoppedError } from '@shared/core/managed-switch-server/managed-switch-server';
+import {
+ ManagedServerStoppedError,
+ ServerBusyError,
+} from '@shared/core/managed-switch-server/managed-switch-server';
import { HostUnreachableError } from '@shared/core/remote-hosts/reachability';
import type { RPCInvocationWrapper } from '@shared/lib/ipc/rpc';
import { serializeLogValue, type LogContext } from '@shared/logger';
@@ -48,13 +51,14 @@ export const withRPCLogContext: RPCInvocationWrapper = (channel, args, invoke) =
});
function logRPCFailure(channel: string, error: unknown) {
- // An unreachable host — or a managed stack the user has stopped — is a
- // modeled, displayed state, not a fault of the call: every view scoped to it
- // refuses in the same way, so logging each one as an error buries real
- // failures under a repeating wall of the same fact. The state's own
- // transition is logged once, where it belongs.
+ // An unreachable host, a stopped managed stack, or one another Console is
+ // changing is a modeled, displayed state, not a fault of the call: every view
+ // scoped to it refuses the same way, so logging each as an error buries real
+ // failures. The state's own transition is logged once, where it belongs.
const modeled =
- error instanceof HostUnreachableError || error instanceof ManagedServerStoppedError;
+ error instanceof HostUnreachableError ||
+ error instanceof ManagedServerStoppedError ||
+ error instanceof ServerBusyError;
const level = modeled ? 'debug' : 'error';
log[level]('RPC handler failed', {
event: 'rpc_failed',
diff --git a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.test.ts b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.test.ts
index fdf82bb22..aaef213db 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.test.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.test.ts
@@ -4,16 +4,50 @@ import { removesProvisionedFiles } from './delete-agent-choices';
// Console provisioned these files on this machine and nothing else reads them,
// so removing the agent and leaving its credentials behind is not offered.
it('always removes the files of an agent on this machine', () => {
- expect(removesProvisionedFiles({ sshHost: null, dir: '/work', chosen: false })).toBe(true);
- expect(removesProvisionedFiles({ sshHost: null, dir: '/work', chosen: true })).toBe(true);
+ for (const chosen of [false, true]) {
+ for (const deleteInSwitch of [false, true]) {
+ expect(removesProvisionedFiles({ sshHost: null, dir: '/work', chosen, deleteInSwitch })).toBe(
+ true
+ );
+ }
+ }
});
it('leaves the choice to the operator on a shared host', () => {
- expect(removesProvisionedFiles({ sshHost: 'builder', dir: '/work', chosen: false })).toBe(false);
- expect(removesProvisionedFiles({ sshHost: 'builder', dir: '/work', chosen: true })).toBe(true);
+ expect(
+ removesProvisionedFiles({
+ sshHost: 'builder',
+ dir: '/work',
+ chosen: false,
+ deleteInSwitch: false,
+ })
+ ).toBe(false);
+ expect(
+ removesProvisionedFiles({
+ sshHost: 'builder',
+ dir: '/work',
+ chosen: true,
+ deleteInSwitch: false,
+ })
+ ).toBe(true);
+});
+
+it('removes a remote agent from its host when it is deleted in Switch', () => {
+ expect(
+ removesProvisionedFiles({
+ sshHost: 'builder',
+ dir: '/work',
+ chosen: false,
+ deleteInSwitch: true,
+ })
+ ).toBe(true);
});
it('removes nothing when no working directory is known', () => {
- expect(removesProvisionedFiles({ sshHost: null, dir: null, chosen: true })).toBe(false);
- expect(removesProvisionedFiles({ sshHost: 'builder', dir: null, chosen: true })).toBe(false);
+ expect(
+ removesProvisionedFiles({ sshHost: null, dir: null, chosen: true, deleteInSwitch: true })
+ ).toBe(false);
+ expect(
+ removesProvisionedFiles({ sshHost: 'builder', dir: null, chosen: true, deleteInSwitch: true })
+ ).toBe(false);
});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.ts b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.ts
index c662b5d92..20f0c000c 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-choices.ts
@@ -1,16 +1,18 @@
/**
- * Whether removing an agent also deletes the files Console provisioned for it.
+ * Whether removing an agent also removes it from where it runs: the files
+ * Console provisioned for it and, on a remote host, what it runs there.
*
* On a shared SSH host those files may belong to another install — an agent
- * loaded rather than created — so removing them is the operator's choice. On
- * this machine Console put them there and nothing else reads them, so leaving
- * stale credentials behind is not a choice worth offering.
+ * loaded rather than created — so removing them is the operator's choice,
+ * unless the agent is also deleted in Switch and has nothing left to run. On
+ * this machine Console put the files there and nothing else reads them.
*/
export function removesProvisionedFiles(input: {
sshHost: string | null;
dir: string | null;
chosen: boolean;
+ deleteInSwitch: boolean;
}): boolean {
if (input.dir === null) return false;
- return input.sshHost === null ? true : input.chosen;
+ return input.sshHost === null ? true : input.chosen || input.deleteInSwitch;
}
diff --git a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-modal.tsx b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-modal.tsx
index 96ea0f4b3..3b5e6cefe 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-modal.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/locations/components/delete-agent-modal.tsx
@@ -32,7 +32,9 @@ export function DeleteAgentModal({ agentLabel, sshHost, dir, onSuccess, onClose
const [chosen, setChosen] = useState(false);
const onThisMachine = sshHost === null;
const filesPlace = dir ? (sshHost ? `${sshHost}:${dir}` : dir) : null;
- const removeProvisionedFiles = removesProvisionedFiles({ sshHost, dir, chosen });
+ const removeProvisionedFiles = removesProvisionedFiles({ sshHost, dir, chosen, deleteInSwitch });
+ // Deleting in Switch forces removal from the host, so the host box follows it.
+ const removeFromHost = !onThisMachine && removeProvisionedFiles;
return (
<>
@@ -45,7 +47,7 @@ export function DeleteAgentModal({ agentLabel, sshHost, dir, onSuccess, onClose
Switch Console.{' '}
{onThisMachine
? 'Its sessions stop, and the credentials and settings files Console provisioned for it are deleted. The working directory itself, and everything else in it, are left alone.'
- : 'Its working directory, the credentials stored there, and any running sidecar are untouched unless you choose below.'}
+ : `It keeps running on ${sshHost}, with its automatic sessions and for anyone else who uses it there, unless you choose below.`}
</p>
{onThisMachine && filesPlace && (
@@ -55,20 +57,23 @@ export function DeleteAgentModal({ agentLabel, sshHost, dir, onSuccess, onClose
)}
{!onThisMachine && filesPlace && (
- <label className="group/field flex cursor-pointer items-start gap-2.5">
+ <label
+ className={`group/field flex items-start gap-2.5 ${deleteInSwitch ? 'cursor-default' : 'cursor-pointer'}`}
+ >
<Checkbox
- checked={chosen}
+ checked={removeFromHost}
+ disabled={deleteInSwitch}
onCheckedChange={(checked) => setChosen(checked === true)}
className="mt-0.5"
/>
<span className="flex flex-col gap-0.5">
- <span className="text-sm font-medium">
- Also delete its files on <code className="break-all">{filesPlace}</code>
- </span>
+ <span className="text-sm font-medium">Also remove it from {sshHost}</span>
<span className="text-xs text-foreground-muted">
- Removes the credentials and definition files provisioned in the working directory
- and stops the agent{"'"}s sidecar. On a shared host these may belong to another
- install — an agent you loaded rather than created should usually keep them.
+ {deleteInSwitch ? <>Deleting it in Switch removes it from {sshHost} too. </> : null}
+ Stops it running there and deletes the credentials and settings files Console
+ provisioned in <code className="break-all">{dir}</code>; the rest of the directory
+ is left alone. Anyone else using this agent on {sshHost} loses it too — an agent you
+ loaded rather than created is usually best left running.
</span>
</span>
</label>
@@ -83,8 +88,9 @@ export function DeleteAgentModal({ agentLabel, sshHost, dir, onSuccess, onClose
<span className="flex flex-col gap-0.5">
<span className="text-sm font-medium">Also delete this agent in Switch</span>
<span className="text-xs text-foreground-muted">
- Permanently deletes its identity on the Switch server. This can’t be undone. Only this
- agent is affected — others in the same directory are left alone.
+ Permanently deletes its identity on the Switch server
+ {onThisMachine ? '' : ` and removes it from ${sshHost}`}. This can’t be undone. Only
+ this agent is affected — others in the same directory are left alone.
</span>
</span>
</label>
@@ -97,7 +103,11 @@ export function DeleteAgentModal({ agentLabel, sshHost, dir, onSuccess, onClose
variant="destructive"
onClick={() => onSuccess({ deleteInSwitch, removeProvisionedFiles })}
>
- {deleteInSwitch ? 'Remove & delete in Switch' : 'Remove'}
+ {deleteInSwitch
+ ? 'Remove & delete in Switch'
+ : removeFromHost
+ ? `Remove from Console & ${sshHost}`
+ : 'Remove'}
</ConfirmButton>
</DialogFooter>
</>
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/AddServerModal.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/AddServerModal.tsx
index 868374054..bbd3db87a 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/AddServerModal.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/AddServerModal.tsx
@@ -1,4 +1,4 @@
-import { CircleCheck, Globe, Laptop, Server, TriangleAlert } from 'lucide-react';
+import { CircleCheck, Globe, Info, Laptop, Server, TriangleAlert } from 'lucide-react';
import { observer } from 'mobx-react-lite';
import { useCallback, useEffect, useRef, useState } from 'react';
import { HostReachabilityNotice } from '@renderer/features/remote-hosts/host-reachability-notice';
@@ -20,6 +20,10 @@ import { Field, FieldGroup, FieldLabel } from '@renderer/lib/ui/field';
import { Input } from '@renderer/lib/ui/input';
import { Spinner } from '@renderer/lib/ui/spinner';
import { WizardStepHeader } from '@renderer/lib/ui/wizard-step-header';
+import {
+ lockHolderSentence,
+ othersRecentlySeen,
+} from '@shared/core/managed-switch-server/managed-switch-server';
import type {
AddServerChoiceName,
AddServerStepName,
@@ -32,7 +36,9 @@ import { LinkAccountsStep } from './link-accounts-step';
import { localServerStore } from './local-server-store';
import { LogTail } from './log-tail';
import { remoteServerStore } from './remote-server-store';
+import { type RemoteSetupAction, remoteSetupAction } from './remote-setup-action';
import { ServerSignInFields, useServerSignIn } from './server-sign-in';
+import { affectedSentence } from './shared-consoles';
import { switchServersStore } from './switch-servers-store';
/**
@@ -289,8 +295,8 @@ function ChooseStep({
/>
<ChoiceCard
icon={<Server className="size-5" />}
- title="Run a server on a remote host"
- description="Switch Console sets it up over SSH on a host you've onboarded. Stays running when Switch Console is closed."
+ title="Run or join a server on a remote host"
+ description="Switch Console sets one up over SSH on a host you've onboarded, or joins the one already running there. Stays running when Switch Console is closed."
onClick={onRemoteHost}
/>
<ChoiceCard
@@ -511,7 +517,8 @@ function DockerStatus({
// Step 2b — remote-host managed setup (pick an onboarded SSH host → start)
// ---------------------------------------------------------------------------
-const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
+/** Exported for its test; the modal is its only other user. */
+export const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
onBack,
onDone,
onClose,
@@ -537,9 +544,14 @@ const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
});
}, [store]);
+ // What is offered depends on what the host already has, so look first, and
+ // again when a host that was out of reach comes back.
+ const hostBlocked = sshHost ? store.isHostBlocked(sshHost) : false;
useEffect(() => {
- if (sshHost) void store.checkDocker(sshHost);
- }, [store, sshHost]);
+ if (!sshHost || hostBlocked) return;
+ void store.checkDocker(sshHost);
+ void store.probe(sshHost);
+ }, [store, sshHost, hostBlocked]);
const running = sshHost ? store.isRunning(sshHost) : false;
const starting = sshHost ? store.isTransitioning(sshHost) : false;
@@ -548,19 +560,46 @@ const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
const dockerUnavailable = docker && !docker.available ? docker : null;
const status = sshHost ? store.statusFor(sshHost) : null;
const logs = sshHost ? store.logsFor(sshHost) : [];
+ const probe = sshHost ? store.probeFor(sshHost) : null;
+ const action = sshHost ? remoteSetupAction(sshHost, probe, store.isProbing(sshHost)) : null;
+ // Another Console changing the server right now; Start or Connect will wait for it.
+ const busy = probe?.kind === 'absent' || probe?.kind === 'present' ? probe.busy : null;
+ const waiting = status?.waitingFor ?? null;
+ const joining = action?.kind === 'connect';
- const hostBlocked = sshHost ? store.isHostBlocked(sshHost) : false;
- const canStart = !!sshHost && name.trim().length > 0 && dockerReady && !starting && !hostBlocked;
- const primaryLabel = running ? 'Done' : status?.phase === 'error' ? 'Retry' : 'Start';
+ const canAct =
+ !!sshHost &&
+ name.trim().length > 0 &&
+ dockerReady &&
+ !starting &&
+ !hostBlocked &&
+ (action?.kind === 'connect' || action?.kind === 'start');
+ const updating = action?.kind === 'connect' && action.updatesTo !== null;
+ // Joining an older server updates it for everyone using it, so name who that is.
+ useEffect(() => {
+ if (sshHost && updating) void store.loadRegister(sshHost);
+ }, [store, sshHost, updating]);
+ const affected =
+ sshHost && updating
+ ? affectedSentence(othersRecentlySeen(store.registerFor(sshHost), new Date()), new Date())
+ : null;
+ const primaryLabel = running
+ ? 'Done'
+ : joining
+ ? updating
+ ? 'Update and connect'
+ : 'Connect'
+ : 'Start';
const onPrimary = () => {
if (running) onDone(sshHost ? (store.statusFor(sshHost).serverId ?? null) : null);
+ else if (sshHost && joining) void store.connect(sshHost, name.trim());
else if (sshHost) void store.start(sshHost, name.trim());
};
return (
<>
<DialogHeader showCloseButton={false}>
- <DialogTitle>Set up a server on a remote host</DialogTitle>
+ <DialogTitle>Run or join a server on a remote host</DialogTitle>
</DialogHeader>
<DialogContentArea className="space-y-4 pt-0">
{hosts === null ? (
@@ -623,7 +662,23 @@ const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
</Field>
)}
- {sshHost && !running && (
+ {sshHost && action && !running && (
+ <RemoteStackNotice
+ sshHost={sshHost}
+ action={action}
+ affected={affected}
+ onCheckAgain={() => void store.probe(sshHost)}
+ />
+ )}
+
+ {sshHost && busy && !starting && !running && (
+ <Alert>
+ <Info className="size-4" />
+ <AlertTitle>{lockHolderSentence(busy)}</AlertTitle>
+ </Alert>
+ )}
+
+ {sshHost && action?.kind === 'start' && !action.existing && !running && (
<div className="bg-card space-y-2 rounded-lg border border-border p-3">
<p className="text-xs font-medium text-foreground-muted">Starting will:</p>
<ul className="space-y-1.5 text-xs text-foreground-muted">
@@ -654,7 +709,8 @@ const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
<AlertTitle>Server is running on {sshHost}</AlertTitle>
<AlertDescription>
It's in your servers list, reachable from this computer while Switch Console is
- open.
+ open. Anyone else with access to {sshHost} can connect to it from their own Switch
+ Console.
</AlertDescription>
</Alert>
)}
@@ -688,19 +744,106 @@ const RemoteHostSetupStep = observer(function RemoteHostSetupStep({
<Button variant="outline" onClick={onBack}>
Back
</Button>
+ ) : sshHost && waiting ? (
+ // Only a wait for another Console can be cancelled: nothing has
+ // been changed yet.
+ <Button variant="outline" onClick={() => void store.cancelWait(sshHost)}>
+ Stop waiting
+ </Button>
) : (
<Button variant="outline" onClick={onClose} disabled>
Cancel
</Button>
)}
- <ConfirmButton onClick={onPrimary} disabled={!running && !canStart}>
- {starting ? 'Starting…' : primaryLabel}
+ <ConfirmButton onClick={onPrimary} disabled={!running && !canAct}>
+ {starting
+ ? updating
+ ? 'Updating…'
+ : joining
+ ? 'Connecting…'
+ : 'Starting…'
+ : primaryLabel}
</ConfirmButton>
</DialogFooter>
</>
);
});
+/**
+ * What the chosen host already has, and so what the primary button will do.
+ * Renders nothing for an empty host.
+ */
+export function RemoteStackNotice({
+ sshHost,
+ action,
+ affected,
+ onCheckAgain,
+}: {
+ sshHost: string;
+ action: RemoteSetupAction;
+ /** Who else an update on joining reaches, when anyone does. */
+ affected: string | null;
+ onCheckAgain: () => void;
+}) {
+ switch (action.kind) {
+ case 'checking':
+ return (
+ <div className="flex items-center gap-2 text-sm text-foreground-muted">
+ <Spinner className="size-3.5" />
+ <span>Looking for a Switch server on {sshHost}…</span>
+ </div>
+ );
+ case 'connect':
+ return (
+ <Alert>
+ <Info className="size-4" />
+ <AlertTitle>
+ A Switch server is already running on {sshHost}
+ {action.deployedVersion ? ` (switch-core ${action.deployedVersion})` : ''}
+ </AlertTitle>
+ <AlertDescription>
+ {action.updatesTo === null
+ ? 'Connecting adds it to this Console without restarting it, so anyone already using it carries on undisturbed.'
+ : `This Console needs switch-core ${action.updatesTo}, so connecting updates it — for everyone who uses it. Its database is backed up first, and it restarts once, keeping its rooms, agents and data.`}
+ {action.updatesTo !== null && affected && ` ${affected}`}
+ {!action.shared &&
+ ' It was set up before servers could be shared; connecting shares it, so others with access to the host can connect too.'}
+ </AlertDescription>
+ </Alert>
+ );
+ case 'start':
+ if (!action.existing) return null;
+ return (
+ <Alert>
+ <Info className="size-4" />
+ <AlertTitle>A Switch server is set up on {sshHost}, but stopped</AlertTitle>
+ <AlertDescription>
+ Starting it keeps its rooms, agents and data — and starts it for everyone who uses it.
+ </AlertDescription>
+ </Alert>
+ );
+ case 'blocked':
+ return (
+ <Alert variant="destructive">
+ <TriangleAlert className="size-4" />
+ <AlertTitle>{action.title}</AlertTitle>
+ <AlertDescription>
+ <span>{action.detail}</span>{' '}
+ <button
+ type="button"
+ onClick={onCheckAgain}
+ className="underline underline-offset-2 hover:text-foreground"
+ >
+ Check again
+ </button>
+ </AlertDescription>
+ </Alert>
+ );
+ case 'docker':
+ return null;
+ }
+}
+
// ---------------------------------------------------------------------------
// Step 2c — external server form (connect by URL; also the edit form)
// ---------------------------------------------------------------------------
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/DeleteServerModal.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/DeleteServerModal.tsx
index a01bfec93..e5e1b5c18 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/DeleteServerModal.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/DeleteServerModal.tsx
@@ -13,6 +13,8 @@ import {
} from '@renderer/lib/ui/dialog';
import { Field, FieldGroup, FieldLabel } from '@renderer/lib/ui/field';
import { Input } from '@renderer/lib/ui/input';
+import { remoteServerStore } from './remote-server-store';
+import { whoElseSentence } from './shared-consoles';
import { switchServersStore } from './switch-servers-store';
type DeleteServerModalArgs = {
@@ -21,6 +23,9 @@ type DeleteServerModalArgs = {
type Props = BaseModalProps<void> & DeleteServerModalArgs;
+/** Removing a remote server: stop using it from here, or destroy it for everyone. */
+type RemoteRemoval = 'disconnect' | 'delete';
+
function countLinkedAgents(serverId: string): number {
let count = 0;
for (const agents of agentsStore.byLocation.values()) {
@@ -29,6 +34,35 @@ function countLinkedAgents(serverId: string): number {
return count;
}
+function RemovalChoice({
+ selected,
+ title,
+ children,
+ onSelect,
+}: {
+ selected: boolean;
+ title: string;
+ children: React.ReactNode;
+ onSelect: () => void;
+}) {
+ return (
+ <button
+ type="button"
+ role="radio"
+ aria-checked={selected}
+ onClick={onSelect}
+ className={`w-full space-y-0.5 rounded-md border p-3 text-left ${
+ selected
+ ? 'border-primary bg-background-tertiary-2'
+ : 'border-border hover:bg-background-tertiary-2'
+ }`}
+ >
+ <span className="block text-sm font-medium text-foreground">{title}</span>
+ <span className="block text-xs text-foreground-muted">{children}</span>
+ </button>
+ );
+}
+
export const DeleteServerModal = observer(function DeleteServerModal({
serverId,
onSuccess,
@@ -39,29 +73,49 @@ export const DeleteServerModal = observer(function DeleteServerModal({
const [confirmText, setConfirmText] = useState('');
const [isDeleting, setIsDeleting] = useState(false);
const [error, setError] = useState<string | null>(null);
+ // On a shared host destroying the stack has to be chosen, not fallen into.
+ const [removal, setRemoval] = useState<RemoteRemoval>('disconnect');
useEffect(() => {
void agentsStore.load().then(() => setLinkedAgents(countLinkedAgents(serverId)));
}, [serverId]);
const managed = server?.managed ?? false;
- // Managed delete tears down real infrastructure and data, so guard it with a
- // type-the-name confirmation. External delete is a reversible de-register.
- const needsTypeConfirm = managed;
+ const remoteHost =
+ server?.managed && server.managementKind === 'remote' && server.sshHost ? server.sshHost : null;
+
+ // The server page has usually read it already; only a cold open asks the host.
+ useEffect(() => {
+ if (remoteHost && !remoteServerStore.registerFor(remoteHost)) {
+ void remoteServerStore.loadRegister(remoteHost);
+ }
+ }, [remoteHost]);
+
+ const destroying = managed && (remoteHost === null || removal === 'delete');
+ // Destroying a stack tears down real infrastructure and data, so it is
+ // guarded with a type-the-name confirmation. Letting go of one is not.
+ const needsTypeConfirm = destroying;
const typeConfirmed = !needsTypeConfirm || confirmText.trim() === (server?.name ?? '').trim();
const handleDelete = useCallback(async () => {
if (!server || !typeConfirmed) return;
setIsDeleting(true);
setError(null);
- const ok = await switchServersStore.deleteServer(serverId);
+ const ok = !remoteHost
+ ? await switchServersStore.deleteServer(serverId)
+ : removal === 'disconnect'
+ ? await remoteServerStore.disconnect(remoteHost, serverId)
+ : await remoteServerStore.deleteForEveryone(remoteHost, serverId);
if (ok) {
onSuccess();
} else {
- setError(switchServersStore.errorText ?? 'Could not delete the server.');
+ setError(
+ (remoteHost ? remoteServerStore.errorText : switchServersStore.errorText) ??
+ 'Could not remove the server.'
+ );
setIsDeleting(false);
}
- }, [server, typeConfirmed, serverId, onSuccess]);
+ }, [server, typeConfirmed, remoteHost, removal, serverId, onSuccess]);
if (!server) {
return (
@@ -85,26 +139,60 @@ export const DeleteServerModal = observer(function DeleteServerModal({
linkedAgents === 0
? 'No agents are linked to it.'
: `${linkedAgents} linked ${linkedAgents === 1 ? 'agent' : 'agents'} will be unlinked but kept — you can re-link them to another server.`;
+ const affected = remoteHost
+ ? whoElseSentence(remoteServerStore.registerFor(remoteHost), new Date())
+ : null;
+
+ const title = remoteHost
+ ? `Remove “${server.name}”?`
+ : managed
+ ? `Delete “${server.name}”?`
+ : `Disconnect from “${server.name}”?`;
+ const actionLabel = destroying
+ ? isDeleting
+ ? 'Deleting…'
+ : remoteHost
+ ? 'Delete for everyone'
+ : 'Delete server'
+ : isDeleting
+ ? 'Disconnecting…'
+ : 'Disconnect';
return (
<>
<DialogHeader showCloseButton={false}>
<div className="flex items-center gap-2">
<TriangleAlert className="size-4 text-red-500" />
- <DialogTitle>
- {managed ? `Delete “${server.name}”?` : `Disconnect from “${server.name}”?`}
- </DialogTitle>
+ <DialogTitle>{title}</DialogTitle>
</div>
</DialogHeader>
<DialogContentArea className="space-y-3 pt-0">
- {managed ? (
+ {remoteHost ? (
+ <div role="radiogroup" className="space-y-2">
+ <RemovalChoice
+ selected={removal === 'disconnect'}
+ title="Disconnect this Console"
+ onSelect={() => setRemoval('disconnect')}
+ >
+ Switch Console stops using the server and forgets its credentials. It keeps running on{' '}
+ {remoteHost} for everyone else, with all its rooms, agents and data — you can connect
+ to it again later.
+ </RemovalChoice>
+ <RemovalChoice
+ selected={removal === 'delete'}
+ title="Delete it for everyone"
+ onSelect={() => setRemoval('delete')}
+ >
+ Tears down the stack on {remoteHost} — its containers,{' '}
+ <strong className="text-foreground">all data and secrets</strong> — for everyone who
+ uses it. This can’t be undone.
+ </RemovalChoice>
+ </div>
+ ) : managed ? (
<p className="text-sm text-foreground-muted">
- This permanently tears down the managed stack{' '}
- {server.managementKind === 'remote' && server.sshHost
- ? `on ${server.sshHost}`
- : 'on this computer'}{' '}
- — its containers, <strong className="text-foreground">all data and secrets</strong> —
- and removes it from Switch Console. This can’t be undone.
+ This permanently tears down the managed stack on this computer — its containers,{' '}
+ <strong className="text-foreground">all data and secrets</strong> — and removes it from
+ Switch Console. This can’t be undone.
</p>
) : (
<p className="text-sm text-foreground-muted">
@@ -112,6 +200,7 @@ export const DeleteServerModal = observer(function DeleteServerModal({
itself isn’t touched — you can connect to it again later.
</p>
)}
+ {destroying && affected && <p className="text-sm text-foreground-muted">{affected}</p>}
<p className="text-sm text-foreground-muted">{agentsNote}</p>
{needsTypeConfirm && (
@@ -146,13 +235,7 @@ export const DeleteServerModal = observer(function DeleteServerModal({
onClick={() => void handleDelete()}
disabled={!typeConfirmed || isDeleting}
>
- {managed
- ? isDeleting
- ? 'Deleting…'
- : 'Delete server'
- : isDeleting
- ? 'Disconnecting…'
- : 'Disconnect'}
+ {actionLabel}
</ConfirmButton>
</DialogFooter>
</>
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/RemoteServerControls.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/RemoteServerControls.tsx
index 47e7a340c..02550033d 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/RemoteServerControls.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/RemoteServerControls.tsx
@@ -1,11 +1,14 @@
-import { TriangleAlert } from 'lucide-react';
+import { Info, TriangleAlert } from 'lucide-react';
import { observer } from 'mobx-react-lite';
import { useEffect, useState } from 'react';
import { Alert, AlertDescription, AlertTitle } from '@renderer/lib/ui/alert';
import { Spinner } from '@renderer/lib/ui/spinner';
+import { othersRecentlySeen } from '@shared/core/managed-switch-server/managed-switch-server';
import { LogTail } from './log-tail';
import { remoteServerStore } from './remote-server-store';
import { phaseLabel, StackAction, StackSection, StackStatusRow } from './server-stack-section';
+import { useSharedActionConfirm } from './shared-action-confirm';
+import { sharedWithSentence } from './shared-consoles';
/**
* Lifecycle for a managed Switch stack running in Docker on an SSH host. The
@@ -21,12 +24,19 @@ export const RemoteServerControls = observer(function RemoteServerControls({
}) {
const store = remoteServerStore;
const [showActivity, setShowActivity] = useState(false);
+ const confirm = useSharedActionConfirm(sshHost);
useEffect(() => {
void store.init();
void store.checkDocker(sshHost);
+ void store.loadRegister(sshHost);
+ // Someone else may have started or removed a stack shown as stopped.
+ void store.refresh(sshHost);
}, [store, sshHost]);
+ // A remote stack is shared: stopping or restarting it here does so for everyone.
+ const others = othersRecentlySeen(store.registerFor(sshHost), new Date());
+
const status = store.statusFor(sshHost);
const hostBlocked = store.isHostBlocked(sshHost);
// Every lifecycle action rides the host's SSH connection, so none of them can
@@ -69,13 +79,13 @@ export const RemoteServerControls = observer(function RemoteServerControls({
<StackAction
label="Restart"
disabled={transitioning}
- onClick={() => void store.start(sshHost, name)}
+ onClick={() => confirm.request('restart', () => void store.start(sshHost, name))}
/>
<StackAction
label="Stop"
danger
disabled={transitioning}
- onClick={() => void store.stop(sshHost)}
+ onClick={() => confirm.request('stop', () => void store.stop(sshHost))}
/>
</>
) : (
@@ -89,10 +99,39 @@ export const RemoteServerControls = observer(function RemoteServerControls({
/>
<div className="space-y-3">
+ {sharedWithSentence(others) && (
+ <p className="text-xs text-foreground-muted">{sharedWithSentence(others)}</p>
+ )}
+
+ {status.notice && !transitioning && (
+ <Alert>
+ <Info className="size-4" />
+ <AlertTitle>{status.notice}</AlertTitle>
+ </Alert>
+ )}
+
+ {status.recordWarning && !transitioning && (
+ <Alert>
+ <TriangleAlert className="size-4 text-amber-500" />
+ <AlertTitle>{status.recordWarning}</AlertTitle>
+ </Alert>
+ )}
+
{status.message && transitioning && (
<div className="flex items-center gap-2 text-sm text-foreground-muted">
<Spinner className="size-3.5" />
<span>{status.message}</span>
+ {/* Only a wait for another Console can be cancelled: nothing has
+ been changed yet. */}
+ {status.waitingFor && (
+ <button
+ type="button"
+ onClick={() => void store.cancelWait(sshHost)}
+ className="text-foreground-muted underline underline-offset-2 transition-colors hover:text-foreground"
+ >
+ Stop waiting
+ </button>
+ )}
</div>
)}
@@ -119,6 +158,8 @@ export const RemoteServerControls = observer(function RemoteServerControls({
the log is what you go looking for, not what the section is for. */}
{showActivity && logs.length > 0 && <LogTail lines={logs} placeholder={null} />}
</div>
+
+ {confirm.dialog}
</StackSection>
);
});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/VersionDriftNotice.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/VersionDriftNotice.tsx
index d7d0a2c7b..78a253629 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/VersionDriftNotice.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/VersionDriftNotice.tsx
@@ -15,9 +15,11 @@ const RESUME_NOTE = 'Sessions on this server resume once the update finishes.';
* this build of Switch Console pins (CHOO-1736).
*
* A stack that is behind the pin is upgraded by Console on its own, so for that
- * case this is a status rather than an offer: the update in progress, the
- * error it failed with (and a retry), or — for a stopped stack — that starting
- * it will update it. Sessions on the server wait for it either way.
+ * case this is mostly a status rather than an offer: the update in progress,
+ * the error it failed with (and a retry), or — for a stopped stack — that
+ * starting it will update it. The exception is a running shared server others
+ * have used lately, which is updated only when someone here says so, naming
+ * who it reaches. Sessions on the server wait for it either way.
*
* The downgrade direction gets no action, only an explanation: the stack's
* database has already migrated forward and switch-core cannot roll back, so
@@ -28,11 +30,14 @@ export function VersionDriftNotice({
upgrade,
progress,
disabled,
+ affected,
onRestart,
}: {
drift: SwitchVersionDrift | null;
/** The upgrade the stack owes this build, if any. */
upgrade: ManagedServerUpgrade | null;
+ /** Who else an update of a shared server reaches, when anyone does. */
+ affected: string | null;
/** The current step of a start in flight ("Pulling images…"), or null. */
progress: string | null;
/** True while a lifecycle operation is in flight (or the host is unreachable). */
@@ -46,6 +51,7 @@ export function VersionDriftNotice({
upgrade={upgrade}
progress={progress}
disabled={disabled}
+ affected={affected}
onRetry={onRestart}
/>
);
@@ -115,11 +121,13 @@ export function ServerUpgradeNotice({
upgrade,
progress,
disabled,
+ affected,
onRetry,
}: {
upgrade: ManagedServerUpgrade;
progress: string | null;
disabled: boolean;
+ affected: string | null;
onRetry: () => void;
}) {
const versions = `switch-core ${upgrade.from} → ${upgrade.to}`;
@@ -153,6 +161,25 @@ export function ServerUpgradeNotice({
</Alert>
);
}
+ if (upgrade.state === 'held') {
+ return (
+ <Alert variant="warning">
+ <ArrowUpCircle className="size-4" />
+ <AlertTitle>{`switch-core ${upgrade.to} is required`}</AlertTitle>
+ <AlertDescription>
+ <p>{`This server runs switch-core ${upgrade.from}. Others use it too, so Switch Console has not updated it on its own: updating restarts it for everyone. Its rooms, agents and data are kept, and the database is backed up first.`}</p>
+ {affected && <p>{affected}</p>}
+ <p>{RESUME_NOTE}</p>
+ </AlertDescription>
+ <AlertAction>
+ <Button size="sm" disabled={disabled} onClick={onRetry}>
+ <RefreshCw className="size-4" />
+ Update for everyone
+ </Button>
+ </AlertAction>
+ </Alert>
+ );
+ }
return (
<Alert variant="warning">
<ArrowUpCircle className="size-4" />
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.test.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.test.ts
index c404a9a74..1d9136724 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.test.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.test.ts
@@ -8,12 +8,20 @@ describe('managedTelemetryNotice', () => {
running: false,
deployed: { known: true, enabled: true },
consent: false,
+ sharedWithOthers: false,
})
).toEqual({ kind: 'in-step' });
});
it('says nothing when nothing has been observed', () => {
- expect(managedTelemetryNotice({ running: true, deployed: null, consent: true })).toEqual({
+ expect(
+ managedTelemetryNotice({
+ running: true,
+ deployed: null,
+ consent: true,
+ sharedWithOthers: false,
+ })
+ ).toEqual({
kind: 'in-step',
});
});
@@ -25,6 +33,7 @@ describe('managedTelemetryNotice', () => {
running: true,
deployed: { known: true, enabled: consent },
consent,
+ sharedWithOthers: false,
})
).toEqual({ kind: 'in-step' });
}
@@ -36,6 +45,7 @@ describe('managedTelemetryNotice', () => {
running: true,
deployed: { known: true, enabled: false },
consent: true,
+ sharedWithOthers: false,
})
).toEqual({ kind: 'stale', consent: true });
});
@@ -48,6 +58,7 @@ describe('managedTelemetryNotice', () => {
running: true,
deployed: { known: true, enabled: true },
consent: false,
+ sharedWithOthers: false,
})
).toEqual({ kind: 'stale', consent: false });
});
@@ -58,7 +69,30 @@ describe('managedTelemetryNotice', () => {
running: true,
deployed: { known: false, reason: 'ssh timed out' },
consent: true,
+ sharedWithOthers: false,
})
).toEqual({ kind: 'unknown', reason: 'ssh timed out' });
});
+
+ it('does not offer to turn sharing on over others using the server', () => {
+ expect(
+ managedTelemetryNotice({
+ running: true,
+ deployed: { known: true, enabled: false },
+ consent: true,
+ sharedWithOthers: true,
+ })
+ ).toEqual({ kind: 'in-step' });
+ });
+
+ it('still offers to apply a no on a server others use', () => {
+ expect(
+ managedTelemetryNotice({
+ running: true,
+ deployed: { known: true, enabled: true },
+ consent: false,
+ sharedWithOthers: true,
+ })
+ ).toEqual({ kind: 'stale', consent: false });
+ });
});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.ts
index c6ec067c3..2bebacd04 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/managed-telemetry-notice.ts
@@ -29,9 +29,14 @@ export function managedTelemetryNotice(args: {
deployed: DeployedTelemetry | null;
/** The user's current answer. */
consent: boolean;
+ /** Whether others use this server too, or it cannot be told. A start keeps a
+ * shared server's sharing off, so a restart from here cannot apply a "yes". */
+ sharedWithOthers: boolean;
}): ManagedTelemetryNotice {
- const { running, deployed, consent } = args;
+ const { running, deployed, consent, sharedWithOthers } = args;
if (!running || deployed === null) return { kind: 'in-step' };
if (!deployed.known) return { kind: 'unknown', reason: deployed.reason };
- return deployed.enabled === consent ? { kind: 'in-step' } : { kind: 'stale', consent };
+ if (deployed.enabled === consent) return { kind: 'in-step' };
+ if (consent && sharedWithOthers) return { kind: 'in-step' };
+ return { kind: 'stale', consent };
}
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.test.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.test.ts
new file mode 100644
index 000000000..eaa0d8595
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.test.ts
@@ -0,0 +1,382 @@
+import { beforeEach, describe, expect, it, vi } from 'vitest';
+
+const rpcRemote = vi.hoisted(() => ({
+ probe: vi.fn(),
+ connect: vi.fn(),
+ disconnect: vi.fn(),
+ register: vi.fn(),
+ start: vi.fn(),
+ stop: vi.fn(),
+ reset: vi.fn(),
+ cancelWait: vi.fn(),
+ refresh: vi.fn(),
+ getStatuses: vi.fn(),
+}));
+const agentsLoad = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const serversInit = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const forgetRemovedServer = vi.hoisted(() => vi.fn(() => Promise.resolve()));
+const isBlocked = vi.hoisted(() => vi.fn(() => false));
+
+const statusListeners = vi.hoisted(() => [] as ((status: unknown) => void)[]);
+vi.mock('@renderer/lib/ipc', () => ({
+ rpc: { remoteSwitchServer: rpcRemote },
+ events: {
+ on: (_channel: unknown, listener: (status: unknown) => void) => {
+ statusListeners.push(listener);
+ return () => {};
+ },
+ },
+}));
+vi.mock('@renderer/features/locations/stores/agents-store', () => ({
+ agentsStore: { load: agentsLoad },
+}));
+vi.mock('@renderer/features/remote-hosts/host-reachability-store', () => ({
+ hostReachabilityStore: { isBlocked, hydrate: vi.fn() },
+}));
+vi.mock('./switch-servers-store', () => ({
+ switchServersStore: { init: serversInit, forgetRemovedServer },
+}));
+
+const { RemoteServerStore } = await import('./remote-server-store');
+
+const REGISTER = { self: 'me', consoles: [], activity: [] };
+
+beforeEach(() => {
+ vi.clearAllMocks();
+ isBlocked.mockReturnValue(false);
+ rpcRemote.register.mockResolvedValue(REGISTER);
+});
+
+describe('probe', () => {
+ it('keeps what the host has, for the setup step to decide on', async () => {
+ rpcRemote.probe.mockResolvedValue({
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: true,
+ });
+ const store = new RemoteServerStore();
+
+ const pending = store.probe('vm-1');
+ expect(store.isProbing('vm-1')).toBe(true);
+ await pending;
+
+ expect(store.isProbing('vm-1')).toBe(false);
+ expect(store.probeFor('vm-1')).toEqual({
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: true,
+ });
+ });
+
+ it('records Docker being unavailable where the Docker notice reads it', async () => {
+ rpcRemote.probe.mockResolvedValue({
+ kind: 'docker-unavailable',
+ reason: 'daemon-down',
+ detail: 'no daemon',
+ });
+ const store = new RemoteServerStore();
+
+ await store.probe('vm-1');
+
+ expect(store.dockerFor('vm-1')).toEqual({
+ available: false,
+ reason: 'daemon-down',
+ detail: 'no daemon',
+ });
+ });
+
+ it('stops looking, and says why, when the host cannot be asked', async () => {
+ rpcRemote.probe.mockRejectedValue(new Error('ssh: connection refused'));
+ const store = new RemoteServerStore();
+
+ await store.probe('vm-1');
+
+ expect(store.isProbing('vm-1')).toBe(false);
+ expect(store.probeFor('vm-1')).toMatchObject({ kind: 'unreadable' });
+ expect(store.error).toBeNull();
+ });
+
+ it('asks nothing of a host that is out of reach', async () => {
+ isBlocked.mockReturnValue(true);
+ const store = new RemoteServerStore();
+
+ await store.probe('vm-1');
+
+ expect(rpcRemote.probe).not.toHaveBeenCalled();
+ });
+});
+
+describe('connect', () => {
+ it('joins the running server and refreshes the server list and who uses it', async () => {
+ rpcRemote.connect.mockResolvedValue({
+ kind: 'connected',
+ serverId: 'srv-1',
+ deployedVersion: '0.27.0',
+ });
+ const store = new RemoteServerStore();
+
+ await store.connect('vm-1', 'Team server');
+
+ expect(rpcRemote.connect).toHaveBeenCalledWith({ sshHost: 'vm-1', name: 'Team server' });
+ expect(rpcRemote.start).not.toHaveBeenCalled();
+ expect(serversInit).toHaveBeenCalledOnce();
+ await vi.waitFor(() => expect(store.registerFor('vm-1')).toEqual(REGISTER));
+ expect(store.error).toBeNull();
+ });
+
+ it('looks again when the server turned out not to be running, so Start is offered', async () => {
+ rpcRemote.connect.mockResolvedValue({ kind: 'not-running' });
+ rpcRemote.probe.mockResolvedValue({
+ kind: 'present',
+ running: false,
+ deployedVersion: '0.27.0',
+ shared: true,
+ });
+ const store = new RemoteServerStore();
+
+ await store.connect('vm-1', 'Team server');
+
+ await vi.waitFor(() =>
+ expect(store.probeFor('vm-1')).toMatchObject({ kind: 'present', running: false })
+ );
+ expect(serversInit).not.toHaveBeenCalled();
+ });
+
+ it('says why another account’s unshared server cannot be joined', async () => {
+ rpcRemote.connect.mockResolvedValue({
+ kind: 'unshared',
+ ownerDir: null,
+ message: 'set up from another account',
+ });
+ rpcRemote.probe.mockResolvedValue({ kind: 'absent', busy: null });
+ const store = new RemoteServerStore();
+
+ await store.connect('vm-1', 'Team server');
+
+ expect(store.error).toBe('set up from another account');
+ });
+
+ it('is not busy once it has finished, whatever the outcome', async () => {
+ rpcRemote.connect.mockRejectedValue(new Error('ipc closed'));
+ const store = new RemoteServerStore();
+
+ expect(await store.connect('vm-1', 'Team server')).toBeNull();
+
+ expect(store.isTransitioning('vm-1')).toBe(false);
+ expect(store.error).toBeTruthy();
+ });
+});
+
+describe('deleting a server for everyone', () => {
+ it('resets its stack and then leaves it, forgetting what it knew of the host', async () => {
+ // Left on the register, this Console would count as a user and hold the
+ // others' updates; a stale register here would mislead the next server.
+ const store = new RemoteServerStore();
+ await store.loadRegister('vm-1');
+
+ expect(await store.deleteForEveryone('vm-1', 'srv-1')).toBe(true);
+
+ expect(rpcRemote.reset).toHaveBeenCalledWith('vm-1');
+ expect(rpcRemote.disconnect).toHaveBeenCalledWith('vm-1');
+ expect(rpcRemote.reset.mock.invocationCallOrder[0]).toBeLessThan(
+ rpcRemote.disconnect.mock.invocationCallOrder[0]!
+ );
+ expect(forgetRemovedServer).toHaveBeenCalledWith('srv-1');
+ expect(agentsLoad).toHaveBeenCalledOnce();
+ expect(store.registerFor('vm-1')).toBeNull();
+ });
+
+ it('keeps the server when its stack could not be reset', async () => {
+ rpcRemote.reset.mockRejectedValueOnce(new Error('host unreachable'));
+ const store = new RemoteServerStore();
+
+ expect(await store.deleteForEveryone('vm-1', 'srv-1')).toBe(false);
+
+ expect(rpcRemote.disconnect).not.toHaveBeenCalled();
+ expect(forgetRemovedServer).not.toHaveBeenCalled();
+ expect(store.error).toMatch(/not deleted/);
+ });
+});
+
+describe('disconnect', () => {
+ it('lets go of the server and forgets what it knew of the host', async () => {
+ rpcRemote.probe.mockResolvedValue({ kind: 'absent', busy: null });
+ const store = new RemoteServerStore();
+ await store.probe('vm-1');
+ await store.loadRegister('vm-1');
+
+ expect(await store.disconnect('vm-1', 'srv-1')).toBe(true);
+
+ expect(rpcRemote.disconnect).toHaveBeenCalledWith('vm-1');
+ expect(rpcRemote.stop).not.toHaveBeenCalled();
+ expect(forgetRemovedServer).toHaveBeenCalledWith('srv-1');
+ expect(agentsLoad).toHaveBeenCalledOnce();
+ expect(store.probeFor('vm-1')).toBeNull();
+ expect(store.registerFor('vm-1')).toBeNull();
+ });
+
+ it('reports a failure rather than pretending the server is gone', async () => {
+ rpcRemote.disconnect.mockRejectedValue(new Error('An operation is already in progress'));
+ const store = new RemoteServerStore();
+
+ expect(await store.disconnect('vm-1', 'srv-1')).toBe(false);
+
+ expect(forgetRemovedServer).not.toHaveBeenCalled();
+ expect(store.error).toBeTruthy();
+ });
+});
+
+describe('loadRegister', () => {
+ it('keeps a failed read apart from the page’s error', async () => {
+ rpcRemote.register.mockRejectedValue(new Error('docker run failed'));
+ const store = new RemoteServerStore();
+
+ await store.loadRegister('vm-1');
+
+ expect(store.registerErrorFor('vm-1')).toBeTruthy();
+ expect(store.error).toBeNull();
+ });
+});
+
+describe('waiting for another Console', () => {
+ const bob = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting' as const,
+ heldForSeconds: 40,
+ expiresInSeconds: 80,
+ };
+
+ it('counts as busy while the main process waits on someone else’s lock', async () => {
+ rpcRemote.getStatuses.mockResolvedValue([]);
+ const store = new RemoteServerStore();
+ const first = statusListeners.length;
+ await store.init();
+ // Statuses first, then log lines.
+ const onStatus = statusListeners[first]!;
+
+ onStatus({ ...store.statusFor('vm-1'), phase: 'stopped', waitingFor: bob });
+
+ expect(store.isTransitioning('vm-1')).toBe(true);
+ onStatus({ ...store.statusFor('vm-1'), waitingFor: null });
+ expect(store.isTransitioning('vm-1')).toBe(false);
+ });
+
+ it('asks the main process to stop waiting', async () => {
+ rpcRemote.cancelWait.mockResolvedValue(undefined);
+ const store = new RemoteServerStore();
+
+ await store.cancelWait('vm-1');
+
+ expect(rpcRemote.cancelWait).toHaveBeenCalledWith('vm-1');
+ expect(store.error).toBeNull();
+ });
+
+ it('says so when it could not ask', async () => {
+ rpcRemote.cancelWait.mockRejectedValue(new Error('ipc closed'));
+ const store = new RemoteServerStore();
+
+ await store.cancelWait('vm-1');
+
+ expect(store.error).toBe('Could not stop waiting for the server.');
+ });
+
+ it('treats a start whose wait was cancelled as nothing having happened', async () => {
+ rpcRemote.start.mockResolvedValue({ kind: 'cancelled' });
+ const store = new RemoteServerStore();
+
+ await store.start('vm-1', 'Team server');
+
+ expect(store.error).toBeNull();
+ expect(serversInit).not.toHaveBeenCalled();
+ expect(store.isTransitioning('vm-1')).toBe(false);
+ });
+
+ it('shows what the host has again after a join whose wait was cancelled', async () => {
+ rpcRemote.connect.mockResolvedValue({ kind: 'cancelled' });
+ rpcRemote.probe.mockResolvedValue({ kind: 'absent', busy: bob });
+ const store = new RemoteServerStore();
+
+ await store.connect('vm-1', 'Team server');
+
+ expect(store.error).toBeNull();
+ await vi.waitFor(() => expect(store.probeFor('vm-1')).toEqual({ kind: 'absent', busy: bob }));
+ });
+});
+
+describe('the paths the rest leave', () => {
+ it('records Docker being unavailable when joining finds it so', async () => {
+ rpcRemote.connect.mockResolvedValue({
+ kind: 'docker-unavailable',
+ reason: 'daemon-down',
+ detail: 'Docker is not running on vm-1.',
+ });
+ rpcRemote.probe.mockResolvedValue({ kind: 'absent', busy: null });
+ const store = new RemoteServerStore();
+
+ await store.connect('vm-1', 'Team server');
+
+ expect(store.dockerFor('vm-1')).toEqual({
+ available: false,
+ reason: 'daemon-down',
+ detail: 'Docker is not running on vm-1.',
+ });
+ expect(store.error).toBe('Docker is not running on vm-1.');
+ });
+
+ it('reads who uses a server again after starting or stopping it', async () => {
+ rpcRemote.start.mockResolvedValue({
+ kind: 'started',
+ serverId: 'srv-1',
+ telemetryEnabled: false,
+ warning: null,
+ });
+ rpcRemote.stop.mockResolvedValue(undefined);
+ const store = new RemoteServerStore();
+
+ await store.start('vm-1', 'Team server');
+ await store.stop('vm-1');
+
+ expect(rpcRemote.register).toHaveBeenCalledTimes(2);
+ expect(store.registerFor('vm-1')).toEqual(REGISTER);
+ });
+
+ it('asks nothing of a host that is out of reach for who uses it', async () => {
+ isBlocked.mockReturnValue(true);
+ const store = new RemoteServerStore();
+
+ await store.loadRegister('vm-1');
+
+ expect(rpcRemote.register).not.toHaveBeenCalled();
+ expect(store.registerErrorFor('vm-1')).toBeNull();
+ });
+});
+
+describe('looking again at a stopped server', () => {
+ it('asks the main process to look', async () => {
+ rpcRemote.refresh.mockResolvedValue(undefined);
+
+ await new RemoteServerStore().refresh('vm-1');
+
+ expect(rpcRemote.refresh).toHaveBeenCalledWith('vm-1');
+ });
+
+ it('asks nothing of a host that is out of reach', async () => {
+ isBlocked.mockReturnValue(true);
+
+ await new RemoteServerStore().refresh('vm-1');
+
+ expect(rpcRemote.refresh).not.toHaveBeenCalled();
+ });
+
+ it('says so when it could not ask', async () => {
+ rpcRemote.refresh.mockRejectedValue(new Error('ipc closed'));
+ const store = new RemoteServerStore();
+
+ await store.refresh('vm-1');
+
+ expect(store.error).toBe('Could not check the server again.');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.ts
index c5f2972a0..afeeef89b 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-server-store.ts
@@ -4,9 +4,12 @@ import { hostReachabilityStore } from '@renderer/features/remote-hosts/host-reac
import { describeFailure } from '@renderer/lib/errors/describe-failure';
import { events, rpc } from '@renderer/lib/ipc';
import type {
+ ConnectRemoteServerResult,
DeployedTelemetry,
DockerAvailability,
ManagedServerUpgrade,
+ RemoteStackProbe,
+ StackRegister,
SwitchVersionDrift,
} from '@shared/core/managed-switch-server/managed-switch-server';
import {
@@ -37,6 +40,9 @@ function defaultStatus(sshHost: string): RemoteServerStatus {
deployedTelemetry: null,
message: null,
error: null,
+ notice: null,
+ recordWarning: null,
+ waitingFor: null,
};
}
@@ -51,6 +57,13 @@ export class RemoteServerStore {
private readonly logsByHost = new Map<string, string[]>();
private readonly dockerByHost = new Map<string, DockerAvailability>();
private readonly busyHosts = new Set<string>();
+ private readonly probes = new Map<string, RemoteStackProbe>();
+ private readonly probingHosts = new Set<string>();
+ /** Who uses each host's stack, as recorded on the host. */
+ private readonly registers = new Map<string, StackRegister>();
+ /** Why a register could not be read, kept apart from `error`: a record that
+ * failed to load says nothing about whether the server works. */
+ private readonly registerErrors = new Map<string, string>();
/** The sentence the page leads with. Never raw exception text. */
error: string | null = null;
/** Diagnostics for the same failure, rendered under `error` rather than in it. */
@@ -122,8 +135,35 @@ export class RemoteServerStore {
}
isTransitioning(sshHost: string): boolean {
- const phase = this.phaseFor(sshHost);
- return this.busyHosts.has(sshHost) || phase === 'starting' || phase === 'stopping';
+ const status = this.statusFor(sshHost);
+ return (
+ this.busyHosts.has(sshHost) ||
+ status.phase === 'starting' ||
+ status.phase === 'stopping' ||
+ status.waitingFor !== null
+ );
+ }
+
+ /** Have the main process look at a stack shown as stopped again, since a
+ * stopped one is sent nothing that would notice another Console starting it.
+ * What it finds arrives as a status. */
+ async refresh(sshHost: string): Promise<void> {
+ if (this.isHostBlocked(sshHost)) return;
+ try {
+ await rpc.remoteSwitchServer.refresh(sshHost);
+ } catch (cause) {
+ this.setError(cause, 'Could not check the server again.');
+ }
+ }
+
+ /** Stop waiting for another Console's hold on the host's stack. Whatever was
+ * waiting ends having changed nothing, and says so itself. */
+ async cancelWait(sshHost: string): Promise<void> {
+ try {
+ await rpc.remoteSwitchServer.cancelWait(sshHost);
+ } catch (cause) {
+ this.setError(cause, 'Could not stop waiting for the server.');
+ }
}
/** Set when the host's switch-core differs from the version this build pins. */
@@ -150,6 +190,24 @@ export class RemoteServerStore {
return this.dockerByHost.get(sshHost) ?? null;
}
+ /** What the host was last found to have of a stack, or null before it has
+ * been looked at. */
+ probeFor(sshHost: string): RemoteStackProbe | null {
+ return this.probes.get(sshHost) ?? null;
+ }
+
+ isProbing(sshHost: string): boolean {
+ return this.probingHosts.has(sshHost);
+ }
+
+ registerFor(sshHost: string): StackRegister | null {
+ return this.registers.get(sshHost) ?? null;
+ }
+
+ registerErrorFor(sshHost: string): string | null {
+ return this.registerErrors.get(sshHost) ?? null;
+ }
+
async init(): Promise<void> {
void hostReachabilityStore.hydrate();
if (!this.off) {
@@ -192,6 +250,151 @@ export class RemoteServerStore {
}
}
+ /**
+ * Look at what `sshHost` has of a stack, so the setup step can offer the one
+ * action that is safe there — Connect to a running stack, Start a stopped or
+ * absent one, neither for one this account cannot read.
+ */
+ async probe(sshHost: string): Promise<void> {
+ if (this.isHostBlocked(sshHost)) return;
+ runInAction(() => this.probingHosts.add(sshHost));
+ try {
+ const probe = await rpc.remoteSwitchServer.probe(sshHost);
+ runInAction(() => {
+ this.probes.set(sshHost, probe);
+ if (probe.kind === 'docker-unavailable') {
+ this.dockerByHost.set(sshHost, {
+ available: false,
+ reason: probe.reason,
+ detail: probe.detail,
+ });
+ }
+ });
+ } catch (cause) {
+ // Kept as the host's answer rather than as a page error: the setup step
+ // must stop saying it is looking, and offer to look again.
+ const { headline } = describeFailure(cause, 'Could not check the host for a Switch server.');
+ runInAction(() => this.probes.set(sshHost, { kind: 'unreadable', reason: headline }));
+ } finally {
+ runInAction(() => this.probingHosts.delete(sshHost));
+ }
+ }
+
+ /**
+ * Join the stack already running on `sshHost`. Anything short of joining it,
+ * a cancelled wait included, re-reads the host so the step shows what is
+ * there now — a stack found stopped offers Start instead.
+ */
+ async connect(sshHost: string, name: string): Promise<ConnectRemoteServerResult | null> {
+ runInAction(() => {
+ this.busyHosts.add(sshHost);
+ this.error = null;
+ this.errorDetail = null;
+ this.logsByHost.set(sshHost, []);
+ });
+ try {
+ const result = await rpc.remoteSwitchServer.connect({ sshHost, name });
+ if (result.kind === 'connected') {
+ await switchServersStore.init();
+ void this.loadRegister(sshHost);
+ } else if (result.kind === 'docker-unavailable') {
+ runInAction(() => {
+ this.dockerByHost.set(sshHost, {
+ available: false,
+ reason: result.reason,
+ detail: result.detail,
+ });
+ this.error = result.detail;
+ });
+ } else if (result.kind === 'unshared' || result.kind === 'error') {
+ runInAction(() => {
+ this.error = result.message;
+ });
+ }
+ if (result.kind !== 'connected') void this.probe(sshHost);
+ return result;
+ } catch (cause) {
+ this.setError(cause, 'Could not connect to the server.');
+ return null;
+ } finally {
+ runInAction(() => this.busyHosts.delete(sshHost));
+ }
+ }
+
+ /** Read who uses the stack on `sshHost` and what they last did. */
+ async loadRegister(sshHost: string): Promise<void> {
+ if (this.isHostBlocked(sshHost)) return;
+ try {
+ const register = await rpc.remoteSwitchServer.register(sshHost);
+ runInAction(() => {
+ this.registers.set(sshHost, register);
+ this.registerErrors.delete(sshHost);
+ });
+ } catch (cause) {
+ const { headline } = describeFailure(cause, 'Could not read who uses this server.');
+ runInAction(() => this.registerErrors.set(sshHost, headline));
+ }
+ }
+
+ /**
+ * Stop using the server on `sshHost` from this Console, leaving it running
+ * for everyone else. Agents are unlinked and kept, as for any removal.
+ * Returns false when it failed, with `error` saying why.
+ */
+ async disconnect(sshHost: string, serverId: string): Promise<boolean> {
+ runInAction(() => {
+ this.busyHosts.add(sshHost);
+ this.error = null;
+ this.errorDetail = null;
+ });
+ try {
+ await rpc.remoteSwitchServer.disconnect(sshHost);
+ this.forget(sshHost);
+ await switchServersStore.forgetRemovedServer(serverId);
+ await agentsStore.load();
+ return true;
+ } catch (cause) {
+ this.setError(cause, 'Could not disconnect from the server.');
+ return false;
+ } finally {
+ runInAction(() => this.busyHosts.delete(sshHost));
+ }
+ }
+
+ /**
+ * Delete the server on `sshHost` for everyone: reset its stack, then
+ * disconnect so this Console stops counting as one of its users. Returns
+ * false when it failed, with `error` saying why; a server whose stack could
+ * not be reset is kept.
+ */
+ async deleteForEveryone(sshHost: string, serverId: string): Promise<boolean> {
+ runInAction(() => {
+ this.busyHosts.add(sshHost);
+ this.error = null;
+ this.errorDetail = null;
+ });
+ try {
+ await rpc.remoteSwitchServer.reset(sshHost);
+ } catch (cause) {
+ this.setError(cause, 'Could not shut down the server’s stack, so it was not deleted.');
+ return false;
+ } finally {
+ runInAction(() => this.busyHosts.delete(sshHost));
+ }
+ return this.disconnect(sshHost, serverId);
+ }
+
+ /** Forget what was read about a host this Console no longer uses. */
+ forget(sshHost: string): void {
+ runInAction(() => {
+ this.statuses.delete(sshHost);
+ this.probes.delete(sshHost);
+ this.registers.delete(sshHost);
+ this.registerErrors.delete(sshHost);
+ this.logsByHost.delete(sshHost);
+ });
+ }
+
async start(sshHost: string, name: string): Promise<void> {
runInAction(() => {
this.busyHosts.add(sshHost);
@@ -218,8 +421,11 @@ export class RemoteServerStore {
runInAction(() => {
this.error = result.message;
});
+ } else if (result.kind === 'cancelled') {
+ // The user stopped waiting for another Console; nothing was changed.
} else {
await switchServersStore.init();
+ void this.loadRegister(sshHost);
}
} catch (cause) {
this.setError(cause, 'Could not start the server.');
@@ -236,6 +442,7 @@ export class RemoteServerStore {
});
try {
await rpc.remoteSwitchServer.stop(sshHost);
+ void this.loadRegister(sshHost);
} catch (cause) {
this.setError(cause, 'Could not stop the server.');
} finally {
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.test.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.test.ts
new file mode 100644
index 000000000..a06baf7b8
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.test.ts
@@ -0,0 +1,168 @@
+import { describe, expect, it } from 'vitest';
+import { remoteSetupAction } from './remote-setup-action';
+
+describe('remoteSetupAction', () => {
+ it('offers nothing until the host has been looked at', () => {
+ expect(remoteSetupAction('vm-1', null, false)).toEqual({ kind: 'checking' });
+ expect(remoteSetupAction('vm-1', { kind: 'absent', busy: null }, true)).toEqual({
+ kind: 'checking',
+ });
+ });
+
+ it('offers to join a running stack rather than start it again', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: false,
+ drift: null,
+ busy: null,
+ },
+ false
+ )
+ ).toEqual({ kind: 'connect', deployedVersion: '0.27.0', shared: false, updatesTo: null });
+ });
+
+ it('says that joining an older stack updates it, since this Console cannot use it as it is', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: true,
+ drift: { deployed: '0.27.0', expected: '0.28.0', direction: 'upgrade' },
+ busy: null,
+ },
+ false
+ )
+ ).toEqual({ kind: 'connect', deployedVersion: '0.27.0', shared: true, updatesTo: '0.28.0' });
+ });
+
+ it('offers nothing on a stack newer than this Console, and says to update the Console', () => {
+ const action = remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.29.0',
+ shared: true,
+ drift: { deployed: '0.29.0', expected: '0.28.0', direction: 'downgrade' },
+ busy: null,
+ },
+ false
+ );
+
+ expect(action).toMatchObject({ kind: 'blocked' });
+ expect(action.kind === 'blocked' && action.detail).toMatch(/Update Switch Console/);
+ });
+
+ it('offers no Start on a stopped stack newer than this Console, which would be refused', () => {
+ const action = remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: false,
+ deployedVersion: '0.29.0',
+ shared: true,
+ drift: { deployed: '0.29.0', expected: '0.28.0', direction: 'downgrade' },
+ busy: null,
+ },
+ false
+ );
+
+ expect(action).toEqual({
+ kind: 'blocked',
+ title: 'The server on vm-1 is newer than this Console',
+ detail:
+ 'It was last run on switch-core 0.29.0, and this Console runs 0.28.0, so it cannot use ' +
+ 'it. Update Switch Console, then start it.',
+ });
+ });
+
+ it('joins a stack whose version cannot be compared as it is', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: true,
+ deployedVersion: 'dev-checkout',
+ shared: true,
+ drift: { deployed: 'dev-checkout', expected: '0.28.0', direction: 'unknown' },
+ busy: null,
+ },
+ false
+ )
+ ).toMatchObject({ kind: 'connect', updatesTo: null });
+ });
+
+ it('offers to start a stopped stack, keeping what it has', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ {
+ kind: 'present',
+ running: false,
+ deployedVersion: '0.27.0',
+ shared: true,
+ drift: null,
+ busy: null,
+ },
+ false
+ )
+ ).toEqual({ kind: 'start', existing: true });
+ });
+
+ it('offers to set one up on an empty host', () => {
+ expect(remoteSetupAction('vm-1', { kind: 'absent', busy: null }, false)).toEqual({
+ kind: 'start',
+ existing: false,
+ });
+ });
+
+ it('offers nothing on another account’s unshared stack, and says why', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ { kind: 'unshared', running: true, ownerDir: null, message: 'set up from another account' },
+ false
+ )
+ ).toEqual({
+ kind: 'blocked',
+ title: 'The server on vm-1 belongs to another account',
+ detail: 'set up from another account',
+ });
+ });
+
+ it('offers nothing when the stack’s settings are partial', () => {
+ const action = remoteSetupAction(
+ 'vm-1',
+ { kind: 'incomplete', running: false, missing: ['JWT_SECRET_KEY'] },
+ false
+ );
+
+ expect(action.kind).toBe('blocked');
+ expect(action.kind === 'blocked' && action.detail).toMatch(/missing JWT_SECRET_KEY/);
+ });
+
+ it('offers nothing on a host it could not read, rather than guessing it is empty', () => {
+ expect(remoteSetupAction('vm-1', { kind: 'unreadable', reason: 'ssh dropped' }, false)).toEqual(
+ { kind: 'blocked', title: 'Could not check vm-1 for a Switch server', detail: 'ssh dropped' }
+ );
+ });
+
+ it('leaves Docker being unavailable to the Docker notice', () => {
+ expect(
+ remoteSetupAction(
+ 'vm-1',
+ { kind: 'docker-unavailable', reason: 'daemon-down', detail: 'no daemon' },
+ false
+ )
+ ).toEqual({ kind: 'docker' });
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.ts
new file mode 100644
index 000000000..6b6a29544
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/remote-setup-action.ts
@@ -0,0 +1,78 @@
+import type { RemoteStackProbe } from '@shared/core/managed-switch-server/managed-switch-server';
+
+/**
+ * What the remote setup step offers for a host, from what the host was found
+ * to have (CHOO-2893). A host is shared, so the step never assumes it is empty:
+ * it looks first, then offers the one action that is safe there.
+ */
+export type RemoteSetupAction =
+ /** Not looked at yet, or being looked at. */
+ | { kind: 'checking' }
+ /** Join a running stack as it is or, when `updatesTo` is set, update it for
+ * everyone first, since this Console cannot use it as it is. */
+ | {
+ kind: 'connect';
+ deployedVersion: string | null;
+ shared: boolean;
+ updatesTo: string | null;
+ }
+ /** Start a stack — `existing` when one is set up but stopped, whose data and
+ * credentials the start keeps; otherwise a new one. */
+ | { kind: 'start'; existing: boolean }
+ /** Nothing is safe to do from this account; the reason is for the user. */
+ | { kind: 'blocked'; title: string; detail: string }
+ /** Docker is not usable on the host; the Docker notice says why. */
+ | { kind: 'docker' };
+
+export function remoteSetupAction(
+ hostLabel: string,
+ probe: RemoteStackProbe | null,
+ probing: boolean
+): RemoteSetupAction {
+ if (probing || probe === null) return { kind: 'checking' };
+ switch (probe.kind) {
+ case 'absent':
+ return { kind: 'start', existing: false };
+ case 'present':
+ // Stopped or running, a start is refused as a downgrade, so none is offered.
+ if (probe.drift?.direction === 'downgrade') {
+ return {
+ kind: 'blocked',
+ title: `The server on ${hostLabel} is newer than this Console`,
+ detail:
+ `It ${probe.running ? 'runs' : 'was last run on'} switch-core ${probe.drift.deployed}, ` +
+ `and this Console runs ${probe.drift.expected}, so it cannot use it. Update Switch ` +
+ `Console, then ${probe.running ? 'connect' : 'start it'}.`,
+ };
+ }
+ if (!probe.running) return { kind: 'start', existing: true };
+ return {
+ kind: 'connect',
+ deployedVersion: probe.deployedVersion,
+ shared: probe.shared,
+ updatesTo: probe.drift?.direction === 'upgrade' ? probe.drift.expected : null,
+ };
+ case 'unshared':
+ return {
+ kind: 'blocked',
+ title: `The server on ${hostLabel} belongs to another account`,
+ detail: probe.message,
+ };
+ case 'incomplete':
+ return {
+ kind: 'blocked',
+ title: `The server on ${hostLabel} cannot be read`,
+ detail:
+ `Its settings are missing ${probe.missing.join(', ')}. Starting it from here could ` +
+ `replace the credentials its data was created with, so it is not offered.`,
+ };
+ case 'unreadable':
+ return {
+ kind: 'blocked',
+ title: `Could not check ${hostLabel} for a Switch server`,
+ detail: probe.reason,
+ };
+ case 'docker-unavailable':
+ return { kind: 'docker' };
+ }
+}
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/server-reset-section.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/server-reset-section.tsx
index b8883f45a..309eaed4a 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/server-reset-section.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/server-reset-section.tsx
@@ -21,11 +21,18 @@ import {
*/
export function ServerResetSection({
dialogTitle,
+ shared,
+ affected,
disabled,
onConfirm,
}: {
/** Names the stack being destroyed — the confirmation has to say which. */
dialogTitle: string;
+ /** Whether other people can be using the stack: true for a remote one. */
+ shared: boolean;
+ /** Who else it reaches, for the confirmation to name — or that it could not
+ * be told; null when nobody. */
+ affected: string | null;
disabled: boolean;
onConfirm: () => void;
}) {
@@ -64,6 +71,12 @@ export function ServerResetSection({
<strong className="text-foreground">every agent you've configured against it</strong>.
This can't be undone. A fresh Start rebuilds an empty stack from scratch.
</DialogDescription>
+ {shared && (
+ <DialogDescription className="mt-2">
+ It is deleted for <strong className="text-foreground">everyone who uses it</strong>,
+ not only for this Console. {affected}
+ </DialogDescription>
+ )}
</DialogContentArea>
<DialogFooter>
<DialogClose render={<Button variant="outline" size="sm" />}>Cancel</DialogClose>
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-action-confirm.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-action-confirm.tsx
new file mode 100644
index 000000000..c02aacbde
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-action-confirm.tsx
@@ -0,0 +1,91 @@
+import { TriangleAlert } from 'lucide-react';
+import { type ReactNode, useState } from 'react';
+import { Button } from '@renderer/lib/ui/button';
+import {
+ Dialog,
+ DialogClose,
+ DialogContent,
+ DialogContentArea,
+ DialogDescription,
+ DialogFooter,
+ DialogHeader,
+ DialogTitle,
+} from '@renderer/lib/ui/dialog';
+import { remoteServerStore } from './remote-server-store';
+import { sharedWithOthers, whoElseSentence } from './shared-consoles';
+
+/** A lifecycle action that reaches everyone using a shared remote server. */
+export type SharedAction = 'stop' | 'restart';
+
+/**
+ * Ask before a stop or restart of a remote server reaches the other people
+ * using it. Asks when others have used it lately or when the register has not
+ * been read, the same rule the main process follows. Pass null for a server
+ * nobody else shares (the local one), which never asks.
+ *
+ * Read from an observer component, since the register lives in a MobX store.
+ */
+export function useSharedActionConfirm(sshHost: string | null): {
+ request: (action: SharedAction, run: () => void) => void;
+ dialog: ReactNode;
+ /** Whether an action here reaches anyone else, as the confirmation decides it. */
+ shared: boolean;
+ /** Who else it reaches, for a notice that names them; null when nobody. */
+ who: string | null;
+} {
+ const [pending, setPending] = useState<{ action: SharedAction; run: () => void } | null>(null);
+ // Separate from `pending`, which is kept while the dialog animates closed so
+ // that its wording does not flip from Stop to Restart on the way out.
+ const [open, setOpen] = useState(false);
+ const register = sshHost ? remoteServerStore.registerFor(sshHost) : null;
+ const now = new Date();
+ const shared = sshHost !== null && sharedWithOthers(register, now);
+ const who = sshHost !== null ? whoElseSentence(register, now) : null;
+
+ const request = (action: SharedAction, run: () => void) => {
+ if (!shared) {
+ run();
+ return;
+ }
+ setPending({ action, run });
+ setOpen(true);
+ };
+
+ const dialog = (
+ <Dialog open={open} onOpenChange={setOpen}>
+ <DialogContent>
+ <DialogHeader>
+ <TriangleAlert className="size-4 text-amber-500" />
+ <DialogTitle>
+ {pending?.action === 'stop'
+ ? `Stop the server on ${sshHost} for everyone?`
+ : `Restart the server on ${sshHost} for everyone?`}
+ </DialogTitle>
+ </DialogHeader>
+ <DialogContentArea>
+ <DialogDescription>
+ {who}{' '}
+ {pending?.action === 'stop'
+ ? 'Their agents stop answering until someone starts it again. Its rooms, agents and data are kept.'
+ : 'Their agents stop answering while it restarts, and it comes back on the version this Console runs.'}
+ </DialogDescription>
+ </DialogContentArea>
+ <DialogFooter>
+ <DialogClose render={<Button variant="outline" size="sm" />}>Cancel</DialogClose>
+ <Button
+ variant={pending?.action === 'stop' ? 'destructive' : 'default'}
+ size="sm"
+ onClick={() => {
+ setOpen(false);
+ pending?.run();
+ }}
+ >
+ {pending?.action === 'stop' ? 'Stop for everyone' : 'Restart for everyone'}
+ </Button>
+ </DialogFooter>
+ </DialogContent>
+ </Dialog>
+ );
+
+ return { request, dialog, shared, who };
+}
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles-section.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles-section.tsx
new file mode 100644
index 000000000..bf11ff4a3
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles-section.tsx
@@ -0,0 +1,75 @@
+import { observer } from 'mobx-react-lite';
+import { RelativeTime } from '@renderer/lib/ui/relative-time';
+import { remoteServerStore } from './remote-server-store';
+import { activitySentence, describeConsole } from './shared-consoles';
+
+const ACTIVITY_SHOWN = 5;
+
+/**
+ * Who uses a shared remote server, and what they last did to it. Everyone signs
+ * in as the server's one admin, so the server cannot tell them apart; each
+ * Console records itself on the host instead, and this reads that record.
+ * Shows nothing until the record has been read, or when it is empty.
+ */
+export const SharedConsolesSection = observer(function SharedConsolesSection({
+ sshHost,
+}: {
+ sshHost: string;
+}) {
+ const store = remoteServerStore;
+ const register = store.registerFor(sshHost);
+ const error = store.registerErrorFor(sshHost);
+ if (!register && !error) return null;
+ if (register && register.consoles.length === 0 && register.activity.length === 0 && !error) {
+ return null;
+ }
+
+ return (
+ <section className="space-y-3">
+ <div className="space-y-0.5">
+ <h3 className="text-sm font-medium text-foreground">Consoles using this server</h3>
+ <p className="text-xs text-foreground-muted">
+ Anyone with access to {sshHost} can connect from their own Switch Console. They all sign
+ in as the server’s admin, so each Console records itself on the host — that record is what
+ is listed here.
+ </p>
+ </div>
+
+ {error && <p className="text-xs text-destructive">{error}</p>}
+
+ {register && register.consoles.length > 0 && (
+ <ul className="divide-y divide-border rounded-lg border border-border">
+ {register.consoles.map((console) => (
+ <li
+ key={console.consoleId}
+ className="flex items-center justify-between gap-3 px-3 py-2 text-sm"
+ >
+ <span className="min-w-0 truncate text-foreground">
+ {describeConsole(console)}
+ {console.consoleId === register.self && (
+ <span className="text-foreground-muted"> — this Console</span>
+ )}
+ </span>
+ <span className="shrink-0 text-xs text-foreground-muted">
+ {console.appVersion} · seen <RelativeTime value={console.lastSeenAt} />
+ </span>
+ </li>
+ ))}
+ </ul>
+ )}
+
+ {register && register.activity.length > 0 && (
+ <div className="space-y-1">
+ <p className="text-xs font-medium text-foreground-muted">Recent activity</p>
+ <ul className="space-y-0.5 text-xs text-foreground-muted">
+ {register.activity.slice(0, ACTIVITY_SHOWN).map((entry) => (
+ <li key={`${entry.at}-${entry.consoleId}-${entry.action}`}>
+ {activitySentence(entry, register.self)} · <RelativeTime value={entry.at} />
+ </li>
+ ))}
+ </ul>
+ </div>
+ )}
+ </section>
+ );
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.test.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.test.ts
new file mode 100644
index 000000000..b29216544
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.test.ts
@@ -0,0 +1,108 @@
+import { describe, expect, it } from 'vitest';
+import {
+ othersRecentlySeen,
+ type StackConsole,
+ type StackRegister,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+import {
+ activitySentence,
+ affectedSentence,
+ describeConsole,
+ sharedWithSentence,
+} from './shared-consoles';
+
+const NOW = new Date('2026-09-23T12:00:00.000Z');
+
+function seen(name: string, hoursAgo: number, id = name): StackConsole {
+ return {
+ consoleId: id,
+ name,
+ hostAccount: name.split('@')[0]!,
+ appVersion: '0.36.0',
+ lastSeenAt: new Date(NOW.getTime() - hoursAgo * 3_600_000).toISOString(),
+ };
+}
+
+function register(consoles: StackConsole[], self = 'me'): StackRegister {
+ return { self, consoles, activity: [] };
+}
+
+describe('othersRecentlySeen', () => {
+ it('leaves out this Console and anyone not seen for a fortnight', () => {
+ const others = othersRecentlySeen(
+ register([seen('me@laptop', 0, 'me'), seen('bob@desk', 2), seen('carol@old', 24 * 15)]),
+ NOW
+ );
+
+ expect(others.map((c) => c.name)).toEqual(['bob@desk']);
+ });
+
+ it('is empty before anything has been read', () => {
+ expect(othersRecentlySeen(null, NOW)).toEqual([]);
+ });
+
+ it('does not count an entry whose time it cannot read', () => {
+ const others = othersRecentlySeen(
+ register([{ ...seen('bob@desk', 1), lastSeenAt: 'yesterday' }]),
+ NOW
+ );
+
+ expect(others).toEqual([]);
+ });
+});
+
+describe('the sentences', () => {
+ it('names the desktop and the account it reaches the host as', () => {
+ expect(describeConsole(seen('bob@desk', 1))).toBe('bob@desk (as bob)');
+ });
+
+ it('says nothing about sharing when nobody else uses the server', () => {
+ expect(sharedWithSentence([])).toBeNull();
+ expect(affectedSentence([], NOW)).toBeNull();
+ });
+
+ it('names one, two, or two and a count of the rest', () => {
+ expect(sharedWithSentence([seen('bob@desk', 1)])).toBe(
+ 'Shared with bob@desk. Stopping or restarting it affects them too.'
+ );
+ expect(sharedWithSentence([seen('bob@desk', 1), seen('carol@lab', 2)])).toMatch(
+ /^Shared with bob@desk and carol@lab\./
+ );
+ expect(
+ sharedWithSentence([seen('bob@desk', 1), seen('carol@lab', 2), seen('dan@home', 3)])
+ ).toMatch(/^Shared with bob@desk, carol@lab and 1 other\./);
+ expect(
+ sharedWithSentence([
+ seen('bob@desk', 1),
+ seen('carol@lab', 2),
+ seen('dan@home', 3),
+ seen('erin@studio', 4),
+ ])
+ ).toMatch(/^Shared with bob@desk, carol@lab and 2 others\./);
+ });
+
+ it('tells a confirmation who it reaches and when they were last there', () => {
+ expect(affectedSentence([seen('bob@desk', 2), seen('carol@lab', 30)], NOW)).toBe(
+ 'Also used recently by bob@desk (as bob), 2 hours ago; carol@lab (as carol), 1 day ago.'
+ );
+ });
+
+ it('keeps a long list short', () => {
+ const many = ['a', 'b', 'c', 'd', 'e'].map((n, i) => seen(`${n}@host`, i + 1));
+
+ expect(affectedSentence(many, NOW)).toMatch(/; and 2 more\.$/);
+ });
+
+ it('says what happened, and says it was this Console when it was', () => {
+ const entry = {
+ at: NOW.toISOString(),
+ action: 'stopped' as const,
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ };
+
+ expect(activitySentence(entry, 'me')).toBe('bob@desk stopped it');
+ expect(activitySentence({ ...entry, consoleId: 'me' }, 'me')).toBe('This Console stopped it');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.ts
new file mode 100644
index 000000000..fd1e93273
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/shared-consoles.ts
@@ -0,0 +1,81 @@
+import { formatDistanceStrict } from 'date-fns';
+import {
+ othersRecentlySeen,
+ type StackActivityAction,
+ type StackActivityEntry,
+ type StackConsole,
+ type StackRegister,
+} from '@shared/core/managed-switch-server/managed-switch-server';
+
+/**
+ * How the people sharing a remote server are described (CHOO-2893). One place,
+ * so the server page, Stop, Reset and Delete name them the same way.
+ */
+
+/** `bob@desk (as bob)` — the desktop, and the account it reaches the host as. */
+export function describeConsole(console: Pick<StackConsole, 'name' | 'hostAccount'>): string {
+ return `${console.name} (as ${console.hostAccount})`;
+}
+
+function names(first: StackConsole, rest: StackConsole[]): string {
+ const [second, ...more] = rest;
+ if (!second) return first.name;
+ if (more.length === 0) return `${first.name} and ${second.name}`;
+ return `${first.name}, ${second.name} and ${more.length} other${more.length === 1 ? '' : 's'}`;
+}
+
+/**
+ * The line under the server's controls saying it is shared, or null when
+ * nobody else has used it recently.
+ */
+export function sharedWithSentence(others: StackConsole[]): string | null {
+ const [first, ...rest] = others;
+ if (!first) return null;
+ return `Shared with ${names(first, rest)}. Stopping or restarting it affects them too.`;
+}
+
+/**
+ * Who a destructive action will reach, for its confirmation — each other
+ * Console with when it was last seen. Null when nobody else has used it.
+ */
+export function affectedSentence(others: StackConsole[], now: Date): string | null {
+ if (others.length === 0) return null;
+ const who = others
+ .slice(0, 3)
+ .map((c) => `${describeConsole(c)}, ${formatDistanceStrict(new Date(c.lastSeenAt), now)} ago`)
+ .join('; ');
+ const more = others.length > 3 ? `; and ${others.length - 3} more` : '';
+ return `Also used recently by ${who}${more}.`;
+}
+
+/**
+ * Whether an action on a shared server reaches anyone else: others have used
+ * it lately, or who uses it has not been read — which counts as others, as it
+ * does in the main process.
+ */
+export function sharedWithOthers(register: StackRegister | null, now: Date): boolean {
+ return register === null || othersRecentlySeen(register, now).length > 0;
+}
+
+/**
+ * Who else an action on a shared server reaches, or that it could not be told.
+ * Null when nobody else uses it.
+ */
+export function whoElseSentence(register: StackRegister | null, now: Date): string | null {
+ if (register === null) return 'Switch Console could not check who else uses it.';
+ return affectedSentence(othersRecentlySeen(register, now), now);
+}
+
+const ACTION_WORDS: Record<StackActivityAction, string> = {
+ started: 'started it',
+ connected: 'connected',
+ stopped: 'stopped it',
+ reset: 'reset it',
+ disconnected: 'disconnected',
+};
+
+/** `bob@desk stopped it` — one line of the server's activity. */
+export function activitySentence(entry: StackActivityEntry, self: string): string {
+ const who = entry.consoleId === self ? 'This Console' : entry.name;
+ return `${who} ${ACTION_WORDS[entry.action]}`;
+}
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/switch-servers-store.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/switch-servers-store.ts
index 053bb39aa..d10d1e625 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/switch-servers-store.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/switch-servers-store.ts
@@ -413,29 +413,37 @@ export class SwitchServersStore {
this.clearError();
try {
await rpc.switchServers.removeServer(serverId);
- const [servers, activeServerId] = await Promise.all([
- rpc.switchServers.listServers(),
- rpc.switchServers.getActiveServerId(),
- ]);
- runInAction(() => {
- this.servers = servers;
- this.activeServerId = activeServerId;
- this.statuses.delete(serverId);
- this.authConfigs.delete(serverId);
- this.authConfigWanted.delete(serverId);
- this.statusUnreachable.delete(serverId);
- this.authConfigUnreachable.delete(serverId);
- });
- // The removed id also sits in the server view's saved params, where it
- // outlives the record and would be read back — as a page for a server
- // that is gone, and as gateway calls for an id nothing can resolve.
- appState.navigation.revalidate();
- await this.ensureActiveServer();
+ await this.forgetRemovedServer(serverId);
} catch (cause) {
this.setError(cause, 'Could not remove the server.');
}
}
+ /**
+ * Re-read what a server's removal leaves behind in this store. Public for the
+ * remote store, which disconnects from a shared remote server itself.
+ */
+ async forgetRemovedServer(serverId: string): Promise<void> {
+ const [servers, activeServerId] = await Promise.all([
+ rpc.switchServers.listServers(),
+ rpc.switchServers.getActiveServerId(),
+ ]);
+ runInAction(() => {
+ this.servers = servers;
+ this.activeServerId = activeServerId;
+ this.statuses.delete(serverId);
+ this.authConfigs.delete(serverId);
+ this.authConfigWanted.delete(serverId);
+ this.statusUnreachable.delete(serverId);
+ this.authConfigUnreachable.delete(serverId);
+ });
+ // The removed id also sits in the server view's saved params, where it
+ // outlives the record and would be read back — as a page for a server
+ // that is gone, and as gateway calls for an id nothing can resolve.
+ appState.navigation.revalidate();
+ await this.ensureActiveServer();
+ }
+
async setActive(serverId: string): Promise<void> {
this.clearError();
try {
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/version-drift-notice.test.ts b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/version-drift-notice.test.ts
index ec6b5bcdb..ed1e08917 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/version-drift-notice.test.ts
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/version-drift-notice.test.ts
@@ -14,11 +14,13 @@ function render(props: {
upgrade: ManagedServerUpgrade | null;
progress?: string | null;
disabled?: boolean;
+ affected?: string | null;
}): string {
return renderToStaticMarkup(
createElement(VersionDriftNotice, {
progress: null,
disabled: false,
+ affected: null,
onRestart: vi.fn(),
...props,
})
@@ -34,6 +36,21 @@ function text(html: string): string {
}
describe('VersionDriftNotice for a server behind the pin', () => {
+ it('asks before updating a server others use, saying who it reaches', () => {
+ const html = text(
+ render({
+ drift: behind,
+ upgrade: { state: 'held', from: '0.10.0', to: '0.11.0' },
+ affected: 'Also used recently by bob@desk (as bob), 2 hours ago.',
+ })
+ );
+
+ expect(html).toMatch(/runs switch-core 0\.10\.0/);
+ expect(html).toMatch(/updating restarts it for everyone/);
+ expect(html).toMatch(/bob@desk \(as bob\), 2 hours ago/);
+ expect(html).toMatch(/Update for everyone/);
+ });
+
it('shows the update in progress, with its current step, and no action', () => {
const html = render({
drift: behind,
diff --git a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/view.tsx b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/view.tsx
index 7cb17f9c0..15bf30100 100644
--- a/console/apps/switch-console-desktop/src/renderer/features/switch-servers/view.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/features/switch-servers/view.tsx
@@ -50,6 +50,8 @@ import { ServerResetSection } from './server-reset-section';
import { ServerSectionTitlebar } from './server-section-titlebar';
import { ServerSignInFields, useServerSignIn } from './server-sign-in';
import { ServerStatTiles } from './server-stat-tiles';
+import { useSharedActionConfirm } from './shared-action-confirm';
+import { SharedConsolesSection } from './shared-consoles-section';
import { switchRoomsStore } from './switch-rooms-store';
import { switchServersStore } from './switch-servers-store';
import { TelemetryConsentNotice } from './TelemetryConsentNotice';
@@ -93,6 +95,9 @@ const ServerMainPanel = observer(function ServerMainPanel() {
const serverId = useServerId();
const store = switchServersStore;
const server = store.servers.find((s) => s.id === serverId);
+ // A remote server is shared, so restarting it from a notice reaches everyone.
+ const remoteHost = server?.managementKind === 'remote' ? server.sshHost : null;
+ const confirm = useSharedActionConfirm(remoteHost);
const showEditServerModal = useShowModal('addServerModal');
const showRenameServerModal = useShowModal('renameServerModal');
const showDeleteServerModal = useShowModal('deleteServerModal');
@@ -260,16 +265,25 @@ const ServerMainPanel = observer(function ServerMainPanel() {
)}
<DropdownMenuSeparator />
{/* Only a server Switch Console runs is one it can delete; for
- anyone else's, all we can do is let go of it. Both stay red:
- either way every agent pointed at this server loses it. */}
+ anyone else's, all we can do is let go of it. A remote one
+ offers both, since its host is shared. All stay red: either
+ way every agent pointed at this server loses it. */}
<DropdownMenuItem
variant="destructive"
onClick={() =>
showDeleteServerModal({ serverId, onSuccess: () => navigate('home') })
}
>
- {server.managed ? <Trash2 className="size-4" /> : <Unplug className="size-4" />}
- {server.managed ? 'Delete server…' : 'Disconnect from server…'}
+ {server.managed && server.managementKind !== 'remote' ? (
+ <Trash2 className="size-4" />
+ ) : (
+ <Unplug className="size-4" />
+ )}
+ {!server.managed
+ ? 'Disconnect from server…'
+ : server.managementKind === 'remote'
+ ? 'Disconnect or delete…'
+ : 'Delete server…'}
</DropdownMenuItem>
</DropdownMenuContent>
</DropdownMenu>
@@ -290,7 +304,14 @@ const ServerMainPanel = observer(function ServerMainPanel() {
upgrade={serverUpgrade(server)}
progress={serverProgress(server)}
disabled={stackTransitioning}
- onRestart={() => restartStack(server)}
+ affected={confirm.who}
+ onRestart={() =>
+ // A held update names who it reaches on its own button; a stopped
+ // server reaches nobody.
+ isManagedRunning(server) && serverUpgrade(server)?.state !== 'held'
+ ? confirm.request('restart', () => restartStack(server))
+ : restartStack(server)
+ }
/>
{/* Same placement, and for the same reason: a consent decision that has
@@ -301,10 +322,12 @@ const ServerMainPanel = observer(function ServerMainPanel() {
running: server.managed && isManagedRunning(server),
deployed: serverDeployedTelemetry(server),
consent: telemetry?.enabled ?? true,
+ sharedWithOthers: confirm.shared,
})}
disabled={stackTransitioning}
- onRestart={() => restartStack(server)}
+ onRestart={() => confirm.request('restart', () => restartStack(server))}
/>
+ {confirm.dialog}
{detailsVisible && unreachable && <ServerUnreachableCard serverId={serverId} />}
@@ -336,6 +359,10 @@ const ServerMainPanel = observer(function ServerMainPanel() {
<LocalServerControls />
))}
+ {server.managed && server.managementKind === 'remote' && server.sshHost && (
+ <SharedConsolesSection sshHost={server.sshHost} />
+ )}
+
{detailsVisible && !unreachable && (
<div className="flex items-center justify-between gap-3">
<div className="min-w-0 space-y-0.5">
@@ -368,6 +395,8 @@ const ServerMainPanel = observer(function ServerMainPanel() {
? `Reset server on ${server.sshHost}`
: 'Reset server on this computer'
}
+ shared={server.managementKind === 'remote'}
+ affected={confirm.who}
disabled={stackTransitioning}
onConfirm={() => {
if (server.managementKind === 'remote' && server.sshHost) {
diff --git a/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.test.ts b/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.test.ts
index de2ab368d..4d93ecfc2 100644
--- a/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.test.ts
+++ b/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.test.ts
@@ -114,6 +114,16 @@ describe('describeFailure', () => {
detail: null,
});
});
+
+ it('passes a refusal naming who is changing a shared server straight through', () => {
+ const message =
+ 'bob@desk (as bob) is starting the server on vm-1 right now, so nothing was changed. Try ' +
+ 'again once they are done.';
+ expect(describeFailure(rpcError('ServerBusyError', message), 'Could not stop.')).toEqual({
+ headline: message,
+ detail: null,
+ });
+ });
});
describe('failureText', () => {
diff --git a/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.ts b/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.ts
index 1cd536707..947dac288 100644
--- a/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.ts
+++ b/console/apps/switch-console-desktop/src/renderer/lib/errors/describe-failure.ts
@@ -88,8 +88,9 @@ export function describeFailure(error: unknown, fallback: string): FailureDescri
case 'HostUnreachableError':
return describeHostUnreachable(error, fallback);
case 'ManagedServerStoppedError':
- // Already a modeled, actionable sentence naming the server and the page
- // to fix it on.
+ case 'ServerBusyError':
+ // Already a modeled, actionable sentence: the server and the page to
+ // fix it on, or who is changing the server and when to try again.
return { headline: error.message, detail: null };
}
}
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-agent-modal.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-agent-modal.test.tsx
new file mode 100644
index 000000000..b7bed5587
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-agent-modal.test.tsx
@@ -0,0 +1,129 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+import {
+ DeleteAgentModal,
+ type DeleteAgentModalResult,
+} from '@renderer/features/locations/components/delete-agent-modal';
+import { Dialog, DialogContent } from '@renderer/lib/ui/dialog';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render(sshHost: string | null) {
+ const onSuccess = vi.fn<(result: DeleteAgentModalResult) => void>();
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () =>
+ root!.render(
+ <Dialog open>
+ <DialogContent>
+ <DeleteAgentModal
+ agentId="agent-1"
+ agentLabel="reviewer"
+ sshHost={sshHost}
+ dir="/srv/reviewer"
+ onSuccess={onSuccess}
+ onClose={() => {}}
+ />
+ </DialogContent>
+ </Dialog>
+ )
+ );
+ return onSuccess;
+}
+
+function checkbox(label: RegExp): HTMLElement {
+ const text = [...document.querySelectorAll('label')].find((l) => label.test(l.textContent ?? ''));
+ if (!text) throw new Error(`no option matching ${label}`);
+ const box = text.querySelector<HTMLElement>('[role="checkbox"], button, input');
+ if (!box) throw new Error(`no checkbox in ${label}`);
+ return box;
+}
+
+function isChecked(box: HTMLElement): boolean {
+ return box.getAttribute('aria-checked') === 'true' || box.hasAttribute('data-checked');
+}
+
+function isDisabled(box: HTMLElement): boolean {
+ return box.getAttribute('aria-disabled') === 'true' || box.hasAttribute('data-disabled');
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+function confirm(): HTMLButtonElement {
+ const button = [...document.querySelectorAll('button')].find((b) =>
+ /^Remove/.test(b.textContent ?? '')
+ );
+ if (!button) throw new Error('no confirm button');
+ return button;
+}
+
+describe('removing a remote agent', () => {
+ it('leaves it running on its host unless asked', async () => {
+ const onSuccess = await render('vm-1');
+
+ expect(document.body.textContent).toContain('It keeps running on vm-1');
+ await click(confirm());
+
+ expect(onSuccess).toHaveBeenCalledWith({
+ deleteInSwitch: false,
+ removeProvisionedFiles: false,
+ });
+ });
+
+ it('removes it from the host when asked', async () => {
+ const onSuccess = await render('vm-1');
+
+ await click(checkbox(/Also remove it from vm-1/));
+ expect(confirm().textContent).toContain('Remove from Console & vm-1');
+ await click(confirm());
+
+ expect(onSuccess).toHaveBeenCalledWith({ deleteInSwitch: false, removeProvisionedFiles: true });
+ });
+
+ it('removes it from the host, and holds that choice, when it is deleted in Switch', async () => {
+ const onSuccess = await render('vm-1');
+
+ await click(checkbox(/Also delete this agent in Switch/));
+
+ const fromHost = checkbox(/Also remove it from vm-1/);
+ expect(isChecked(fromHost)).toBe(true);
+ expect(isDisabled(fromHost)).toBe(true);
+ expect(document.body.textContent).toContain('Deleting it in Switch removes it from vm-1 too.');
+ await click(confirm());
+
+ expect(onSuccess).toHaveBeenCalledWith({ deleteInSwitch: true, removeProvisionedFiles: true });
+ });
+});
+
+describe('removing an agent on this machine', () => {
+ it('offers no host choice, and always removes the files Console provisioned', async () => {
+ const onSuccess = await render(null);
+
+ expect(document.body.textContent).not.toContain('Also remove it from');
+ await click(confirm());
+
+ expect(onSuccess).toHaveBeenCalledWith({ deleteInSwitch: false, removeProvisionedFiles: true });
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-server-modal.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-server-modal.test.tsx
new file mode 100644
index 000000000..4c264ebc9
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/delete-server-modal.test.tsx
@@ -0,0 +1,298 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+const servers = vi.hoisted(() => ({
+ current: [] as unknown[],
+ deleteServer: vi.fn(),
+ errorText: null as string | null,
+}));
+vi.mock('@renderer/features/switch-servers/switch-servers-store', () => ({
+ switchServersStore: {
+ get servers() {
+ return servers.current;
+ },
+ deleteServer: (...args: unknown[]) => servers.deleteServer(...args),
+ get errorText() {
+ return servers.errorText;
+ },
+ },
+}));
+
+const remote = vi.hoisted(() => ({
+ register: null as unknown,
+ disconnect: vi.fn(),
+ deleteForEveryone: vi.fn(),
+ errorText: null as string | null,
+ loadRegister: vi.fn(),
+}));
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: {
+ registerFor: () => remote.register,
+ loadRegister: (...args: unknown[]) => remote.loadRegister(...args),
+ disconnect: (...args: unknown[]) => remote.disconnect(...args),
+ deleteForEveryone: (...args: unknown[]) => remote.deleteForEveryone(...args),
+ get errorText() {
+ return remote.errorText;
+ },
+ },
+}));
+
+vi.mock('@renderer/features/locations/stores/agents-store', () => ({
+ agentsStore: {
+ load: () => Promise.resolve(),
+ byLocation: new Map(),
+ },
+}));
+
+import { DeleteServerModal } from '@renderer/features/switch-servers/DeleteServerModal';
+import { Dialog, DialogContent } from '@renderer/lib/ui/dialog';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+beforeEach(() => {
+ servers.current = [];
+ servers.deleteServer.mockReset().mockResolvedValue(true);
+ servers.errorText = null;
+ remote.register = null;
+ remote.disconnect.mockReset().mockResolvedValue(true);
+ remote.deleteForEveryone.mockReset().mockResolvedValue(true);
+ remote.errorText = null;
+ remote.loadRegister.mockReset().mockResolvedValue(undefined);
+});
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+const REMOTE_SERVER = {
+ id: 'srv-1',
+ name: 'Team Server',
+ gatewayUrl: 'https://gateway.example',
+ apiUrl: 'https://api.example',
+ managed: true,
+ managementKind: 'remote' as const,
+ sshHost: 'vm-1',
+ createdAt: '2026-01-01T00:00:00.000Z',
+ updatedAt: '2026-01-01T00:00:00.000Z',
+};
+
+const EXTERNAL_SERVER = {
+ id: 'srv-2',
+ name: 'External Server',
+ gatewayUrl: 'https://gateway.example',
+ apiUrl: 'https://api.example',
+ managed: false,
+ managementKind: null,
+ sshHost: null,
+ createdAt: '2026-01-01T00:00:00.000Z',
+ updatedAt: '2026-01-01T00:00:00.000Z',
+};
+
+async function render(serverId: string) {
+ const onSuccess = vi.fn();
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () =>
+ root!.render(
+ <Dialog open>
+ <DialogContent>
+ <DeleteServerModal serverId={serverId} onSuccess={onSuccess} onClose={() => {}} />
+ </DialogContent>
+ </Dialog>
+ )
+ );
+ return onSuccess;
+}
+
+function radio(label: RegExp): HTMLButtonElement {
+ const button = [...document.querySelectorAll('button[role="radio"]')].find((b) =>
+ label.test(b.textContent ?? '')
+ );
+ if (!button) throw new Error(`no radio option matching ${label}`);
+ return button as HTMLButtonElement;
+}
+
+function confirmButton(): HTMLButtonElement {
+ const button = [...document.querySelectorAll('button:not([role="radio"])')].find((b) =>
+ /^(Disconnect|Delete)/.test(b.textContent ?? '')
+ );
+ if (!button) throw new Error('no confirm button');
+ return button as HTMLButtonElement;
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+async function type(input: HTMLInputElement, value: string): Promise<void> {
+ await act(async () => {
+ const setter = Object.getOwnPropertyDescriptor(
+ window.HTMLInputElement.prototype,
+ 'value'
+ )!.set!;
+ setter.call(input, value);
+ input.dispatchEvent(new Event('input', { bubbles: true }));
+ });
+}
+
+describe('removing a remote managed server', () => {
+ it('defaults to disconnecting this Console rather than destroying the stack', async () => {
+ servers.current = [REMOTE_SERVER];
+ const onSuccess = await render(REMOTE_SERVER.id);
+
+ expect(radio(/Disconnect this Console/).getAttribute('aria-checked')).toBe('true');
+ expect(radio(/Delete it for everyone/).getAttribute('aria-checked')).toBe('false');
+ expect(confirmButton().textContent).toContain('Disconnect');
+
+ await click(confirmButton());
+
+ expect(remote.disconnect).toHaveBeenCalledWith('vm-1', REMOTE_SERVER.id);
+ expect(remote.deleteForEveryone).not.toHaveBeenCalled();
+ expect(onSuccess).toHaveBeenCalled();
+ });
+
+ it('requires typing the server name before it will delete it for everyone', async () => {
+ servers.current = [REMOTE_SERVER];
+ await render(REMOTE_SERVER.id);
+
+ await click(radio(/Delete it for everyone/));
+ expect(confirmButton().disabled).toBe(true);
+
+ const input = document.querySelector('input') as HTMLInputElement;
+ await type(input, 'wrong name');
+ expect(confirmButton().disabled).toBe(true);
+
+ await type(input, REMOTE_SERVER.name);
+ expect(confirmButton().disabled).toBe(false);
+
+ await click(confirmButton());
+
+ expect(remote.deleteForEveryone).toHaveBeenCalledWith('vm-1', REMOTE_SERVER.id);
+ expect(remote.disconnect).not.toHaveBeenCalled();
+ expect(servers.deleteServer).not.toHaveBeenCalled();
+ });
+
+ it('shows who else uses the server once destroying it is chosen', async () => {
+ servers.current = [REMOTE_SERVER];
+ remote.register = {
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+ },
+ ],
+ activity: [],
+ };
+ await render(REMOTE_SERVER.id);
+
+ expect(document.body.textContent).not.toContain('Also used recently by');
+
+ await click(radio(/Delete it for everyone/));
+
+ expect(document.body.textContent).toContain('Also used recently by');
+ expect(document.body.textContent).toContain('bob@desk (as bob)');
+ });
+
+ it('says who else uses it could not be checked when the register is null', async () => {
+ servers.current = [REMOTE_SERVER];
+ remote.register = null;
+ await render(REMOTE_SERVER.id);
+
+ await click(radio(/Delete it for everyone/));
+
+ expect(document.body.textContent).toContain('Switch Console could not check who else uses it.');
+ });
+
+ it('shows the store error when disconnecting fails', async () => {
+ servers.current = [REMOTE_SERVER];
+ remote.disconnect.mockResolvedValue(false);
+ remote.errorText = 'SSH connection to vm-1 timed out.';
+ await render(REMOTE_SERVER.id);
+
+ await click(confirmButton());
+
+ expect(document.body.textContent).toContain('SSH connection to vm-1 timed out.');
+ });
+});
+
+describe('removing an external server', () => {
+ it('has no destroy-for-everyone choice and calls switchServersStore.deleteServer', async () => {
+ servers.current = [EXTERNAL_SERVER];
+ const onSuccess = await render(EXTERNAL_SERVER.id);
+
+ expect(document.querySelectorAll('[role="radiogroup"]')).toHaveLength(0);
+
+ await click(confirmButton());
+
+ expect(servers.deleteServer).toHaveBeenCalledWith(EXTERNAL_SERVER.id);
+ expect(remote.disconnect).not.toHaveBeenCalled();
+ expect(remote.deleteForEveryone).not.toHaveBeenCalled();
+ expect(onSuccess).toHaveBeenCalled();
+ });
+});
+
+describe('the paths the rest leave', () => {
+ it('says a server that has gone is no longer there', async () => {
+ servers.current = [];
+ await render('srv-gone');
+
+ expect(document.body.textContent).toContain('This server is no longer available.');
+ });
+
+ it('says it could not remove the server when the store gives no reason', async () => {
+ servers.current = [REMOTE_SERVER];
+ remote.disconnect.mockResolvedValue(false);
+ remote.errorText = null;
+ await render(REMOTE_SERVER.id);
+
+ await click(confirmButton());
+
+ expect(document.body.textContent).toContain('Could not remove the server.');
+ });
+
+ it('offers to delete a server on this computer outright, which nobody else uses', async () => {
+ servers.current = [{ ...REMOTE_SERVER, managementKind: 'local', sshHost: null }];
+ await render(REMOTE_SERVER.id);
+
+ expect(document.body.textContent).toContain('Delete “Team Server”?');
+ expect(document.querySelectorAll('[role="radiogroup"]')).toHaveLength(0);
+ expect(confirmButton().textContent).toContain('Delete server');
+ });
+
+ it('shows why a server someone else runs could not be let go of', async () => {
+ servers.current = [EXTERNAL_SERVER];
+ servers.deleteServer.mockResolvedValue(false);
+ servers.errorText = 'The server record is locked by another window.';
+ await render(EXTERNAL_SERVER.id);
+
+ await click(confirmButton());
+
+ expect(document.body.textContent).toContain('The server record is locked by another window.');
+ });
+
+ it('asks whether to disconnect from a server someone else runs', async () => {
+ servers.current = [EXTERNAL_SERVER];
+ await render(EXTERNAL_SERVER.id);
+
+ expect(document.body.textContent).toContain('Disconnect from “External Server”?');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-host-setup-step.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-host-setup-step.test.tsx
new file mode 100644
index 000000000..e52e580fd
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-host-setup-step.test.tsx
@@ -0,0 +1,343 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+const state = vi.hoisted(() => ({
+ hosts: [{ sshHost: 'vm-1', name: 'Team VM' }] as { sshHost: string; name: string }[],
+ probe: null as unknown,
+ status: null as unknown,
+ transitioning: false,
+ running: false,
+ register: null as unknown,
+ connect: vi.fn(),
+ start: vi.fn(),
+ cancelWait: vi.fn(),
+ loadRegister: vi.fn(),
+ probeHost: vi.fn(async (_sshHost: unknown) => {}),
+}));
+
+vi.mock('@renderer/lib/ipc', () => ({
+ rpc: { remoteHosts: { listHosts: () => Promise.resolve(state.hosts) } },
+ events: { on: () => () => {} },
+}));
+vi.mock('@renderer/features/remote-hosts/host-reachability-notice', () => ({
+ HostReachabilityNotice: () => null,
+}));
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: {
+ init: () => Promise.resolve(),
+ checkDocker: () => Promise.resolve(),
+ probe: (sshHost: string) => state.probeHost(sshHost),
+ isHostBlocked: () => false,
+ isRunning: () => state.running,
+ isTransitioning: () => state.transitioning,
+ dockerFor: () => ({ available: true, version: '27.0.0' }),
+ statusFor: () => state.status,
+ logsFor: () => [],
+ probeFor: () => state.probe,
+ isProbing: () => false,
+ registerFor: () => state.register,
+ loadRegister: (...args: unknown[]) => state.loadRegister(...args),
+ connect: (...args: unknown[]) => state.connect(...args),
+ start: (...args: unknown[]) => state.start(...args),
+ cancelWait: (...args: unknown[]) => state.cancelWait(...args),
+ error: null,
+ errorDetail: null,
+ },
+}));
+
+import { RemoteHostSetupStep } from '@renderer/features/switch-servers/AddServerModal';
+import { Dialog, DialogContent } from '@renderer/lib/ui/dialog';
+import '@renderer/index.css';
+
+const bob = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting' as const,
+ heldForSeconds: 40,
+ expiresInSeconds: 80,
+};
+
+function status(overrides: Record<string, unknown> = {}) {
+ return {
+ sshHost: 'vm-1',
+ phase: 'stopped',
+ upgrade: null,
+ serverId: null,
+ version: '0.28.0',
+ deployedVersion: null,
+ drift: null,
+ checkoutBuild: null,
+ deployedTelemetry: null,
+ message: null,
+ error: null,
+ notice: null,
+ recordWarning: null,
+ waitingFor: null,
+ ...overrides,
+ };
+}
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+beforeEach(() => {
+ state.hosts = [{ sshHost: 'vm-1', name: 'Team VM' }];
+ state.probe = { kind: 'absent', busy: null };
+ state.status = status();
+ state.transitioning = false;
+ state.running = false;
+ state.register = { self: 'me', consoles: [], activity: [] };
+ state.connect.mockReset();
+ state.start.mockReset();
+ state.cancelWait.mockReset();
+ state.loadRegister.mockReset();
+});
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render() {
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () =>
+ root!.render(
+ <Dialog open>
+ <DialogContent>
+ <RemoteHostSetupStep onBack={() => {}} onDone={() => {}} onClose={() => {}} />
+ </DialogContent>
+ </Dialog>
+ )
+ );
+ // The host list arrives on the next tick; with one host it is chosen.
+ await act(async () => {});
+}
+
+function text(): string {
+ return document.querySelector('[role="dialog"]')?.textContent ?? '';
+}
+
+/** Matched on its text; the primary button's is followed by its shortcut, so
+ * it is matched by how it starts. */
+function button(label: RegExp): HTMLButtonElement {
+ const found = [...document.querySelectorAll('button')].find((b) =>
+ label.test(b.textContent?.trim() ?? '')
+ );
+ if (!found) throw new Error(`no button matching ${label}`);
+ return found as HTMLButtonElement;
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+describe('what the step offers for what the host has', () => {
+ it('offers Start on an empty host', async () => {
+ await render();
+
+ await click(button(/^Start/));
+
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team VM Switch server');
+ });
+
+ it('offers Connect for a running server on this Console’s version', async () => {
+ state.probe = {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.28.0',
+ shared: true,
+ drift: null,
+ busy: null,
+ };
+ await render();
+
+ await click(button(/^Connect/));
+
+ expect(state.connect).toHaveBeenCalledWith('vm-1', 'Team VM Switch server');
+ });
+
+ it('offers Update and connect for an older one, naming who else the update reaches', async () => {
+ state.probe = {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: true,
+ drift: { deployed: '0.27.0', expected: '0.28.0', direction: 'upgrade' },
+ busy: null,
+ };
+ state.register = {
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+ },
+ ],
+ activity: [],
+ };
+ await render();
+
+ expect(state.loadRegister).toHaveBeenCalledWith('vm-1');
+ expect(text()).toContain('bob@desk (as bob)');
+ await click(button(/^Update and connect/));
+
+ expect(state.connect).toHaveBeenCalledWith('vm-1', 'Team VM Switch server');
+ });
+
+ it('says Updating… while that update runs', async () => {
+ state.probe = {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.27.0',
+ shared: true,
+ drift: { deployed: '0.27.0', expected: '0.28.0', direction: 'upgrade' },
+ busy: null,
+ };
+ state.transitioning = true;
+ state.status = status({ phase: 'starting', message: 'Updating the server…' });
+ await render();
+
+ expect(button(/^Updating…/)).toBeDefined();
+ });
+});
+
+describe('another Console changing the server', () => {
+ it('says who is changing it, and that Start will wait for them', async () => {
+ state.probe = { kind: 'absent', busy: bob };
+ await render();
+
+ expect(text()).toContain(
+ 'bob@desk (as bob) is starting this server right now. Starting or connecting here waits ' +
+ 'until they are done.'
+ );
+ });
+
+ it('says it for a server that is there too', async () => {
+ state.probe = {
+ kind: 'present',
+ running: false,
+ deployedVersion: '0.28.0',
+ shared: true,
+ drift: null,
+ busy: { ...bob, action: 'stopping' },
+ };
+ await render();
+
+ expect(text()).toContain('bob@desk (as bob) is stopping this server right now.');
+ });
+
+ it('stops saying it once this Console is the one starting', async () => {
+ state.probe = { kind: 'absent', busy: bob };
+ state.transitioning = true;
+ state.status = status({ phase: 'starting', message: 'Starting containers…' });
+ await render();
+
+ expect(text()).not.toContain('is starting this server right now');
+ });
+
+ it('offers Stop waiting while it waits for them, and stops waiting when asked', async () => {
+ state.transitioning = true;
+ state.status = status({
+ phase: 'starting',
+ message: 'Waiting for bob@desk (as bob) to finish starting the server…',
+ waitingFor: bob,
+ });
+ await render();
+
+ expect(text()).toContain('Waiting for bob@desk (as bob) to finish starting the server…');
+ await click(button(/^Stop waiting$/));
+
+ expect(state.cancelWait).toHaveBeenCalledWith('vm-1');
+ });
+
+ it('offers no way out of a start that is not waiting on anyone', async () => {
+ state.transitioning = true;
+ state.status = status({ phase: 'starting', message: 'Starting containers…' });
+ await render();
+
+ expect(() => button(/^Stop waiting$/)).toThrow();
+ expect(button(/^Cancel$/).disabled).toBe(true);
+ });
+});
+
+describe('the rest of the step', () => {
+ it('says it is looking before the host has answered', async () => {
+ state.probe = null;
+ await render();
+
+ expect(text()).toContain('Looking for a Switch server on vm-1…');
+ });
+
+ it('says Connecting… while it joins', async () => {
+ state.probe = {
+ kind: 'present',
+ running: true,
+ deployedVersion: '0.28.0',
+ shared: true,
+ drift: null,
+ busy: null,
+ };
+ state.transitioning = true;
+ state.status = status({ phase: 'starting', message: 'Connecting to vm-1…' });
+ await render();
+
+ expect(button(/^Connecting…/)).toBeDefined();
+ });
+
+ it('ends on the server it started or joined once it is running', async () => {
+ state.running = true;
+ state.status = status({ phase: 'running', serverId: 'srv-1' });
+ const onDone = vi.fn();
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () =>
+ root!.render(
+ <Dialog open>
+ <DialogContent>
+ <RemoteHostSetupStep onBack={() => {}} onDone={onDone} onClose={() => {}} />
+ </DialogContent>
+ </Dialog>
+ )
+ );
+ await act(async () => {});
+
+ expect(text()).toContain('Server is running on vm-1');
+ await click(button(/^Done/));
+
+ expect(onDone).toHaveBeenCalledWith('srv-1');
+ });
+});
+
+describe('a host with nothing safe to do from this account', () => {
+ it('looks at the host again when asked to', async () => {
+ state.probe = {
+ kind: 'unshared',
+ running: true,
+ ownerDir: '/home/alice/.switchdash/switch-server',
+ message: 'The Switch server on vm-1 was set up from another account.',
+ };
+ await render();
+ state.probeHost.mockClear();
+
+ await click(button(/^Check again$/));
+
+ expect(state.probeHost).toHaveBeenCalledWith('vm-1');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-server-controls.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-server-controls.test.tsx
new file mode 100644
index 000000000..685307714
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-server-controls.test.tsx
@@ -0,0 +1,277 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+function defaultStatus() {
+ return {
+ sshHost: 'vm-1',
+ phase: 'running' as const,
+ upgrade: null,
+ serverId: 'srv-1',
+ version: '0.37.0',
+ deployedVersion: '0.37.0',
+ drift: null,
+ checkoutBuild: null,
+ deployedTelemetry: null,
+ message: null as string | null,
+ error: null as string | null,
+ notice: null as string | null,
+ recordWarning: null as string | null,
+ waitingFor: null as unknown,
+ };
+}
+
+const state = vi.hoisted(() => ({
+ register: null as unknown,
+ status: undefined as unknown,
+ hostBlocked: false,
+ transitioning: false,
+ running: true,
+ docker: null as unknown,
+ drift: null as unknown,
+ logs: [] as string[],
+ error: null as string | null,
+ errorDetail: null as string | null,
+ start: vi.fn(),
+ stop: vi.fn(),
+ cancelWait: vi.fn(),
+ refresh: vi.fn(),
+}));
+
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: {
+ init: () => Promise.resolve(),
+ checkDocker: () => Promise.resolve(),
+ loadRegister: () => Promise.resolve(),
+ registerFor: () => state.register,
+ statusFor: () => state.status,
+ isHostBlocked: () => state.hostBlocked,
+ isTransitioning: () => state.transitioning,
+ isRunning: () => state.running,
+ dockerFor: () => state.docker,
+ driftFor: () => state.drift,
+ logsFor: () => state.logs,
+ get error() {
+ return state.error;
+ },
+ get errorDetail() {
+ return state.errorDetail;
+ },
+ start: (...args: unknown[]) => state.start(...args),
+ stop: (...args: unknown[]) => state.stop(...args),
+ cancelWait: (...args: unknown[]) => state.cancelWait(...args),
+ refresh: (...args: unknown[]) => state.refresh(...args),
+ },
+}));
+
+import { RemoteServerControls } from '@renderer/features/switch-servers/RemoteServerControls';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+const OTHER_CONSOLE = {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+};
+
+beforeEach(() => {
+ state.register = { self: 'me', consoles: [], activity: [] };
+ state.status = defaultStatus();
+ state.hostBlocked = false;
+ state.transitioning = false;
+ state.running = true;
+ state.docker = null;
+ state.drift = null;
+ state.logs = [];
+ state.error = null;
+ state.errorDetail = null;
+ state.start.mockReset();
+ state.stop.mockReset();
+ state.cancelWait.mockReset();
+});
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render() {
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () => root!.render(<RemoteServerControls sshHost="vm-1" name="Team Server" />));
+}
+
+function button(label: RegExp): HTMLButtonElement {
+ const found = [...document.querySelectorAll('button')].find((b) =>
+ label.test(b.textContent ?? '')
+ );
+ if (!found) throw new Error(`no button matching ${label}`);
+ return found as HTMLButtonElement;
+}
+
+function dialogText(): string {
+ return document.querySelector('[role="dialog"]')?.textContent ?? '';
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+describe('sharing notice', () => {
+ it('says who else has used the server recently', async () => {
+ state.register = { self: 'me', consoles: [OTHER_CONSOLE], activity: [] };
+ await render();
+
+ expect(document.body.textContent).toContain(
+ 'Shared with bob@desk. Stopping or restarting it affects them too.'
+ );
+ });
+
+ it('says nothing when nobody else has used it', async () => {
+ state.register = { self: 'me', consoles: [], activity: [] };
+ await render();
+
+ expect(document.body.textContent).not.toContain('Shared with');
+ });
+});
+
+describe('host notices', () => {
+ it('shows the status notice as an alert', async () => {
+ state.status = { ...defaultStatus(), notice: 'Someone else started this stack.' };
+ await render();
+
+ expect(document.body.textContent).toContain('Someone else started this stack.');
+ });
+
+ it('shows the record warning as an alert', async () => {
+ state.status = {
+ ...defaultStatus(),
+ recordWarning: 'Could not record that this Console started it.',
+ };
+ await render();
+
+ expect(document.body.textContent).toContain('Could not record that this Console started it.');
+ });
+});
+
+describe('stopping or restarting a shared stack', () => {
+ it('asks before stopping it when others have used it lately', async () => {
+ state.register = { self: 'me', consoles: [OTHER_CONSOLE], activity: [] };
+ await render();
+
+ await click(button(/^Stop$/));
+
+ expect(state.stop).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Stop the server on vm-1 for everyone\?/);
+
+ await click(button(/Stop for everyone/));
+
+ expect(state.stop).toHaveBeenCalledWith('vm-1');
+ });
+
+ it('asks before restarting it when others have used it lately', async () => {
+ state.register = { self: 'me', consoles: [OTHER_CONSOLE], activity: [] };
+ await render();
+
+ await click(button(/^Restart$/));
+
+ expect(state.start).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Restart the server on vm-1 for everyone\?/);
+
+ await click(button(/Restart for everyone/));
+
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('stops at once when nobody else has used it', async () => {
+ state.register = { self: 'me', consoles: [], activity: [] };
+ await render();
+
+ await click(button(/^Stop$/));
+
+ expect(state.stop).toHaveBeenCalledWith('vm-1');
+ expect(dialogText()).toBe('');
+ });
+
+ it('restarts at once when nobody else has used it', async () => {
+ state.register = { self: 'me', consoles: [], activity: [] };
+ await render();
+
+ await click(button(/^Restart$/));
+
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ expect(dialogText()).toBe('');
+ });
+});
+
+describe('a stack ahead of this build', () => {
+ it('disables Start rather than let a downgrade run', async () => {
+ state.running = false;
+ state.status = { ...defaultStatus(), phase: 'stopped' };
+ state.drift = { deployed: '0.40.0', expected: '0.37.0', direction: 'downgrade' };
+ await render();
+
+ expect(button(/^Start$/).disabled).toBe(true);
+ });
+});
+
+describe('waiting for another Console', () => {
+ const bob = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting' as const,
+ heldForSeconds: 40,
+ expiresInSeconds: 80,
+ };
+
+ it('says who it is waiting for, and stops waiting when asked', async () => {
+ state.transitioning = true;
+ state.status = {
+ ...defaultStatus(),
+ phase: 'starting',
+ message: 'Waiting for bob@desk (as bob) to finish starting the server…',
+ waitingFor: bob,
+ };
+ await render();
+
+ expect(document.body.textContent).toContain(
+ 'Waiting for bob@desk (as bob) to finish starting the server…'
+ );
+ await click(button(/^Stop waiting$/));
+
+ expect(state.cancelWait).toHaveBeenCalledWith('vm-1');
+ });
+
+ it('offers no way to stop a start that is not waiting on anyone', async () => {
+ state.transitioning = true;
+ state.status = { ...defaultStatus(), phase: 'starting', message: 'Starting containers…' };
+ await render();
+
+ expect(document.body.textContent).toContain('Starting containers…');
+ expect(() => button(/^Stop waiting$/)).toThrow();
+ });
+});
+
+describe('opening the page', () => {
+ it('has the host looked at again, in case someone else started or removed the server', async () => {
+ state.refresh.mockReset();
+ await render();
+
+ expect(state.refresh).toHaveBeenCalledWith('vm-1');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-stack-notice.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-stack-notice.test.tsx
new file mode 100644
index 000000000..2d10d3353
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/remote-stack-notice.test.tsx
@@ -0,0 +1,153 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+import { RemoteStackNotice } from '@renderer/features/switch-servers/AddServerModal';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render(props: Parameters<typeof RemoteStackNotice>[0]) {
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () => root!.render(<RemoteStackNotice {...props} />));
+}
+
+describe('connecting to a stack already running on the host', () => {
+ it('says connecting will not restart it when this Console already matches its version', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'connect', deployedVersion: '0.37.0', shared: true, updatesTo: null },
+ affected: null,
+ onCheckAgain: vi.fn(),
+ });
+
+ expect(document.body.textContent).toContain(
+ 'Connecting adds it to this Console without restarting it, so anyone already using it carries on undisturbed.'
+ );
+ });
+
+ it('says connecting updates it for everyone when this Console needs a newer switch-core, and names who it reaches', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'connect', deployedVersion: '0.36.0', shared: true, updatesTo: '0.28.0' },
+ affected: 'Also used recently by bob@desk (as bob), 2 hours ago.',
+ onCheckAgain: vi.fn(),
+ });
+
+ expect(document.body.textContent).toContain('This Console needs switch-core 0.28.0');
+ expect(document.body.textContent).toContain('updates it');
+ expect(document.body.textContent).toContain('for everyone who uses it');
+ expect(document.body.textContent).toContain(
+ 'Also used recently by bob@desk (as bob), 2 hours ago.'
+ );
+ });
+});
+
+describe('a host with nothing safe to do from this account', () => {
+ it('shows the title and detail, and calls onCheckAgain when asked to check again', async () => {
+ const onCheckAgain = vi.fn();
+ await render({
+ sshHost: 'vm-1',
+ action: {
+ kind: 'blocked',
+ title: 'The server on vm-1 belongs to another account',
+ detail: 'Its settings were never shared with this account.',
+ },
+ affected: null,
+ onCheckAgain,
+ });
+
+ expect(document.body.textContent).toContain('The server on vm-1 belongs to another account');
+ expect(document.body.textContent).toContain(
+ 'Its settings were never shared with this account.'
+ );
+
+ const checkAgain = [...document.querySelectorAll('button')].find((b) =>
+ /Check again/.test(b.textContent ?? '')
+ );
+ expect(checkAgain).toBeDefined();
+ await act(async () => checkAgain!.click());
+
+ expect(onCheckAgain).toHaveBeenCalledOnce();
+ });
+});
+
+describe('starting a stack already set up but stopped', () => {
+ it('says starting keeps its data', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'start', existing: true },
+ affected: null,
+ onCheckAgain: vi.fn(),
+ });
+
+ expect(document.body.textContent).toContain('A Switch server is set up on vm-1, but stopped');
+ expect(document.body.textContent).toContain('Starting it keeps its rooms, agents and data');
+ });
+});
+
+describe('the rest of what the step can say', () => {
+ it('says it is looking while the host has not answered yet', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'checking' },
+ affected: null,
+ onCheckAgain: vi.fn(),
+ });
+
+ expect(document.body.textContent).toContain('Looking for a Switch server on vm-1…');
+ });
+
+ it('shares a server set up before sharing existed when connecting, and names no version it cannot read', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'connect', deployedVersion: null, shared: false, updatesTo: null },
+ affected: null,
+ onCheckAgain: vi.fn(),
+ });
+
+ expect(document.body.textContent).toContain('A Switch server is already running on vm-1');
+ expect(document.body.textContent).not.toContain('(switch-core');
+ expect(document.body.textContent).toContain('connecting shares it');
+ });
+
+ it('says nothing for an empty host, or one whose Docker notice already says why', async () => {
+ await render({
+ sshHost: 'vm-1',
+ action: { kind: 'start', existing: false },
+ affected: null,
+ onCheckAgain: vi.fn(),
+ });
+ expect(container!.textContent).toBe('');
+
+ await act(async () =>
+ root!.render(
+ <RemoteStackNotice
+ sshHost="vm-1"
+ action={{ kind: 'docker' }}
+ affected={null}
+ onCheckAgain={vi.fn()}
+ />
+ )
+ );
+ expect(container!.textContent).toBe('');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/server-reset-section.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/server-reset-section.test.tsx
new file mode 100644
index 000000000..32ceeaafd
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/server-reset-section.test.tsx
@@ -0,0 +1,99 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+import { ServerResetSection } from '@renderer/features/switch-servers/server-reset-section';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render(props: {
+ shared: boolean;
+ affected: string | null;
+ disabled?: boolean;
+ onConfirm?: () => void;
+}) {
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () =>
+ root!.render(
+ <ServerResetSection
+ dialogTitle="Reset “Team Server”?"
+ shared={props.shared}
+ affected={props.affected}
+ disabled={props.disabled ?? false}
+ onConfirm={props.onConfirm ?? vi.fn()}
+ />
+ )
+ );
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+function openDialog(): Promise<void> {
+ return click(
+ [...document.querySelectorAll('button')].find((b) => /^Reset…$/.test(b.textContent ?? ''))!
+ );
+}
+
+function dialogText(): string {
+ return document.querySelector('[role="dialog"]')?.textContent ?? '';
+}
+
+describe('resetting a shared remote server', () => {
+ it('says it is deleted for everyone who uses it, and names them', async () => {
+ await render({
+ shared: true,
+ affected: 'Also used recently by bob@desk (as bob), 2 hours ago.',
+ });
+
+ await openDialog();
+
+ expect(dialogText()).toContain('deleted for everyone who uses it');
+ expect(dialogText()).toContain('Also used recently by bob@desk (as bob), 2 hours ago.');
+ });
+
+ it('calls onConfirm and closes when the reset is confirmed', async () => {
+ const onConfirm = vi.fn();
+ await render({ shared: true, affected: null, onConfirm });
+
+ await openDialog();
+ await click(
+ [...document.querySelectorAll('button')].find((b) =>
+ /^Reset and delete all agents$/.test(b.textContent ?? '')
+ )!
+ );
+
+ expect(onConfirm).toHaveBeenCalledOnce();
+ expect(document.querySelector('[role="dialog"]')?.hasAttribute('data-closed')).toBe(true);
+ });
+});
+
+describe('resetting a local, unshared server', () => {
+ it('says nothing about being shared with anyone else', async () => {
+ await render({ shared: false, affected: null });
+
+ await openDialog();
+
+ expect(dialogText()).not.toContain('everyone who uses it');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/server-view-shared.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/server-view-shared.test.tsx
new file mode 100644
index 000000000..174d91452
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/server-view-shared.test.tsx
@@ -0,0 +1,358 @@
+import type * as ReactQuery from '@tanstack/react-query';
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+const state = vi.hoisted(() => ({
+ server: null as unknown,
+ running: true,
+ upgrade: null as unknown,
+ drift: null as unknown,
+ deployedTelemetry: null as unknown,
+ consent: true,
+ register: null as unknown,
+ start: vi.fn(),
+ reset: vi.fn(),
+ localStart: vi.fn(),
+ localDrift: null as unknown,
+ showModal: vi.fn(),
+}));
+
+vi.mock('@renderer/lib/layout/navigation-provider', () => ({
+ useParams: () => ({ params: { serverId: 'srv-1' } }),
+ useNavigate: () => ({ navigate: vi.fn() }),
+}));
+vi.mock('@renderer/lib/modal/modal-provider', () => ({
+ useShowModal: (id: string) => (args: unknown) => state.showModal(id, args),
+}));
+vi.mock('@tanstack/react-query', async (importOriginal) => ({
+ ...(await importOriginal<typeof ReactQuery>()),
+ useQueryClient: () => ({ invalidateQueries: () => Promise.resolve() }),
+}));
+vi.mock('@renderer/features/settings/use-app-settings-key', () => ({
+ useAppSettingsKey: () => ({ value: { enabled: state.consent } }),
+}));
+vi.mock('@renderer/features/switch-servers/switch-servers-store', () => ({
+ switchServersStore: {
+ get servers() {
+ return [state.server];
+ },
+ statusFor: () => ({ connected: false }),
+ refreshing: new Set(),
+ isUnreachable: () => false,
+ isHostBlocked: () => false,
+ refreshStatus: () => Promise.resolve(),
+ ensureAuthConfig: () => Promise.resolve(),
+ refreshServer: () => Promise.resolve(),
+ error: null,
+ errorDetail: null,
+ },
+}));
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: {
+ isRunning: () => state.running,
+ isTransitioning: () => false,
+ isHostBlocked: () => false,
+ registerFor: () => state.register,
+ driftFor: () => state.drift,
+ upgradeFor: () => state.upgrade,
+ statusFor: () => ({ message: null }),
+ deployedTelemetryFor: () => state.deployedTelemetry,
+ start: (...args: unknown[]) => state.start(...args),
+ reset: (...args: unknown[]) => state.reset(...args),
+ },
+}));
+vi.mock('@renderer/features/switch-servers/local-server-store', () => ({
+ localServerStore: {
+ isRunning: true,
+ isTransitioning: false,
+ get drift() {
+ return state.localDrift;
+ },
+ upgrade: null,
+ message: null,
+ deployedTelemetry: null,
+ start: (...args: unknown[]) => state.localStart(...args),
+ reset: vi.fn(),
+ },
+}));
+vi.mock('@renderer/features/remote-hosts/host-reachability-store', () => ({
+ hostReachabilityStore: {
+ hydrate: () => Promise.resolve(),
+ get: () => null,
+ isBlocked: () => false,
+ },
+}));
+vi.mock('@renderer/features/remote-hosts/host-unreachable-panel', () => ({
+ HostUnreachablePanel: () => null,
+}));
+vi.mock('@renderer/features/locations/stores/agents-store', () => ({
+ agentsStore: { load: () => Promise.resolve() },
+}));
+vi.mock('@renderer/features/switch-servers/switch-rooms-store', () => ({
+ switchRoomsStore: { refreshRoomState: () => Promise.resolve() },
+}));
+vi.mock('@renderer/features/switch-servers/RemoteServerControls', () => ({
+ RemoteServerControls: () => <div>remote controls</div>,
+}));
+vi.mock('@renderer/features/switch-servers/LocalServerControls', () => ({
+ LocalServerControls: () => <div>local controls</div>,
+}));
+vi.mock('@renderer/features/switch-servers/shared-consoles-section', () => ({
+ SharedConsolesSection: ({ sshHost }: { sshHost: string }) => (
+ <div>{`consoles using ${sshHost}`}</div>
+ ),
+}));
+vi.mock('@renderer/features/switch-servers/MessagingAppsCard', () => ({
+ MessagingAppsCard: () => null,
+}));
+vi.mock('@renderer/features/switch-servers/server-stat-tiles', () => ({
+ ServerStatTiles: () => null,
+}));
+vi.mock('@renderer/features/switch-servers/server-sign-in', () => ({
+ ServerSignInFields: () => null,
+ useServerSignIn: () => ({}),
+}));
+
+import { serverView } from '@renderer/features/switch-servers/view';
+import '@renderer/index.css';
+
+const REMOTE = {
+ id: 'srv-1',
+ name: 'Team Server',
+ gatewayUrl: 'http://localhost:41000',
+ apiUrl: 'http://localhost:41001',
+ managed: true,
+ managementKind: 'remote' as const,
+ sshHost: 'vm-1',
+ createdAt: '2026-01-01T00:00:00.000Z',
+ updatedAt: '2026-01-01T00:00:00.000Z',
+};
+
+const BOB = {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+};
+
+const SHARED = { self: 'me', consoles: [BOB], activity: [] };
+const ALONE = { self: 'me', consoles: [], activity: [] };
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+beforeEach(() => {
+ state.server = REMOTE;
+ state.running = true;
+ state.upgrade = null;
+ state.drift = null;
+ state.deployedTelemetry = null;
+ state.consent = true;
+ state.register = SHARED;
+ state.start.mockReset();
+ state.reset.mockReset();
+ state.localStart.mockReset();
+ state.localDrift = null;
+ state.showModal.mockReset();
+});
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render() {
+ const Page = serverView.MainPanel;
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () => root!.render(<Page />));
+}
+
+function button(label: RegExp): HTMLElement {
+ const found = [...document.querySelectorAll('button, [role="menuitem"]')].find((b) =>
+ label.test(b.textContent?.trim() ?? '')
+ );
+ if (!found) throw new Error(`no button matching ${label}`);
+ return found as HTMLElement;
+}
+
+function dialogText(): string {
+ return [...document.querySelectorAll('[role="dialog"]')].map((d) => d.textContent).join(' ');
+}
+
+function menuTrigger(): HTMLElement {
+ return document.querySelector('[aria-label="Server actions"]') as HTMLElement;
+}
+
+async function click(element: HTMLElement): Promise<void> {
+ await act(async () => element.click());
+}
+
+describe('the server menu', () => {
+ it('offers to disconnect from or delete a remote server, which others may use', async () => {
+ await render();
+ await click(menuTrigger());
+
+ await click(button(/^Disconnect or delete…$/));
+
+ expect(state.showModal).toHaveBeenCalledWith(
+ 'deleteServerModal',
+ expect.objectContaining({ serverId: 'srv-1' })
+ );
+ });
+
+ it('offers to delete a server on this computer, which nobody else uses', async () => {
+ state.server = { ...REMOTE, managementKind: 'local', sshHost: null };
+ await render();
+ await click(menuTrigger());
+
+ expect(button(/^Delete server…$/)).toBeDefined();
+ });
+
+ it('offers only to disconnect from a server someone else runs', async () => {
+ state.server = { ...REMOTE, managed: false, managementKind: null, sshHost: null };
+ await render();
+ await click(menuTrigger());
+
+ expect(button(/^Disconnect from server…$/)).toBeDefined();
+ });
+});
+
+describe('restarting from a notice', () => {
+ it('asks before a retried update restarts a server others use, naming them', async () => {
+ state.upgrade = { state: 'failed', from: '0.27.0', to: '0.28.0', error: 'pull failed' };
+ await render();
+
+ await click(button(/^Retry$/));
+
+ expect(state.start).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Restart the server on vm-1 for everyone\?/);
+ expect(dialogText()).toContain('bob@desk (as bob)');
+ await click(button(/^Restart for everyone$/));
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('updates at once from a held update, whose own button names who it reaches', async () => {
+ state.upgrade = { state: 'held', from: '0.27.0', to: '0.28.0' };
+ await render();
+
+ expect(document.body.textContent).toContain('bob@desk (as bob)');
+ await click(button(/^Update for everyone$/));
+
+ expect(dialogText()).toBe('');
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('starts a stopped server that owes an update without asking: it reaches nobody', async () => {
+ state.running = false;
+ state.upgrade = { state: 'pending', from: '0.27.0', to: '0.28.0' };
+ await render();
+
+ await click(button(/^Start and update$/));
+
+ expect(dialogText()).toBe('');
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('restarts at once when nobody else uses the server', async () => {
+ state.register = ALONE;
+ state.upgrade = { state: 'failed', from: '0.27.0', to: '0.28.0', error: 'pull failed' };
+ await render();
+
+ await click(button(/^Retry$/));
+
+ expect(dialogText()).toBe('');
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('asks before restarting to apply a usage-data “no” to a server others use', async () => {
+ state.consent = false;
+ state.deployedTelemetry = { known: true, enabled: true };
+ await render();
+
+ await click(button(/^Restart to apply$/));
+
+ expect(state.start).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Restart the server on vm-1 for everyone\?/);
+ });
+
+ it('offers no restart for a “yes” a shared server would not apply over the others', async () => {
+ state.consent = true;
+ state.deployedTelemetry = { known: true, enabled: false };
+ await render();
+
+ expect(() => button(/^Restart to apply$/)).toThrow();
+ });
+
+ it('offers that restart when nobody else uses the server', async () => {
+ state.register = ALONE;
+ state.consent = true;
+ state.deployedTelemetry = { known: true, enabled: false };
+ await render();
+
+ await click(button(/^Restart to apply$/));
+
+ expect(state.start).toHaveBeenCalledWith('vm-1', 'Team Server');
+ });
+
+ it('restarts a server on this computer without asking anyone', async () => {
+ state.server = { ...REMOTE, managementKind: 'local', sshHost: null };
+ state.register = null;
+ state.localDrift = { deployed: 'dev-checkout', expected: '0.28.0', direction: 'unknown' };
+ await render();
+
+ await click(button(/^Restart to update$/));
+
+ expect(dialogText()).toBe('');
+ expect(state.localStart).toHaveBeenCalled();
+ });
+
+ it('asks before restarting a remote server others use to fix a version mismatch', async () => {
+ state.drift = { deployed: 'dev-checkout', expected: '0.28.0', direction: 'unknown' };
+ await render();
+
+ await click(button(/^Restart to update$/));
+
+ expect(state.start).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Restart the server on vm-1 for everyone\?/);
+ });
+});
+
+describe('the rest of a shared server’s page', () => {
+ it('lists the Consoles using a remote server', async () => {
+ await render();
+
+ expect(document.body.textContent).toContain('consoles using vm-1');
+ });
+
+ it('lists none for a server on this computer', async () => {
+ state.server = { ...REMOTE, managementKind: 'local', sshHost: null };
+ await render();
+
+ expect(document.body.textContent).not.toContain('consoles using');
+ });
+
+ it('says a reset deletes it for everyone, naming them, and resets that host', async () => {
+ await render();
+
+ await click(button(/^Reset…$/));
+
+ expect(dialogText()).toContain('everyone who uses it');
+ expect(dialogText()).toContain('bob@desk (as bob)');
+ await click(button(/^Reset and delete all agents$/));
+ expect(state.reset).toHaveBeenCalledWith('vm-1');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-action-confirm.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-action-confirm.test.tsx
new file mode 100644
index 000000000..b8cf61cfa
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-action-confirm.test.tsx
@@ -0,0 +1,138 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+const register = vi.hoisted(() => ({ current: null as unknown }));
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: { registerFor: () => register.current },
+}));
+
+import { useSharedActionConfirm } from '@renderer/features/switch-servers/shared-action-confirm';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+function Harness({ sshHost, run }: { sshHost: string | null; run: () => void }) {
+ const confirm = useSharedActionConfirm(sshHost);
+ return (
+ <>
+ <button type="button" data-testid="stop" onClick={() => confirm.request('stop', run)}>
+ Stop
+ </button>
+ {confirm.dialog}
+ </>
+ );
+}
+
+async function clickStop(sshHost: string | null) {
+ const run = vi.fn();
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () => root!.render(<Harness sshHost={sshHost} run={run} />));
+ await act(async () =>
+ (container!.querySelector('[data-testid="stop"]') as HTMLButtonElement).click()
+ );
+ return run;
+}
+
+function dialogText(): string {
+ return document.querySelector('[role="dialog"]')?.textContent ?? '';
+}
+
+describe('asking before a shared stop or restart', () => {
+ it('stops at once when nobody else has used the server lately', async () => {
+ register.current = { self: 'me', consoles: [], activity: [] };
+
+ const run = await clickStop('vm-1');
+
+ expect(run).toHaveBeenCalledOnce();
+ expect(dialogText()).toBe('');
+ });
+
+ it('asks, naming them, when others have', async () => {
+ register.current = {
+ self: 'me',
+ consoles: [
+ {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+ },
+ ],
+ activity: [],
+ };
+
+ const run = await clickStop('vm-1');
+
+ expect(run).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/Stop the server on vm-1 for everyone\?/);
+ expect(dialogText()).toMatch(/bob@desk \(as bob\)/);
+ });
+
+ it('asks when who uses the server has not been read', async () => {
+ register.current = null;
+
+ const run = await clickStop('vm-1');
+
+ expect(run).not.toHaveBeenCalled();
+ expect(dialogText()).toMatch(/could not check who else uses it/);
+ });
+
+ it('never asks for a server nobody else shares', async () => {
+ register.current = null;
+
+ const run = await clickStop(null);
+
+ expect(run).toHaveBeenCalledOnce();
+ });
+});
+
+describe('changing one’s mind', () => {
+ it('does nothing when the question is cancelled', async () => {
+ register.current = null;
+
+ const run = await clickStop('vm-1');
+ const cancel = [...document.querySelectorAll('button')].find(
+ (b) => b.textContent?.trim() === 'Cancel'
+ )!;
+ await act(async () => cancel.click());
+
+ expect(run).not.toHaveBeenCalled();
+ // It closes as a stop: the words do not change on the way out.
+ expect(dialogText()).not.toMatch(/Restart/);
+ await vi.waitFor(() => expect(dialogText()).toBe(''));
+ });
+
+ it('runs the action once when it is confirmed', async () => {
+ register.current = null;
+
+ const run = await clickStop('vm-1');
+ const confirm = [...document.querySelectorAll('button')].find(
+ (b) => b.textContent?.trim() === 'Stop for everyone'
+ )!;
+ await act(async () => confirm.click());
+
+ expect(run).toHaveBeenCalledOnce();
+ expect(dialogText()).not.toMatch(/Restart/);
+ await vi.waitFor(() => expect(dialogText()).toBe(''));
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-consoles-section.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-consoles-section.test.tsx
new file mode 100644
index 000000000..f6798dea9
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/shared-consoles-section.test.tsx
@@ -0,0 +1,130 @@
+import { act } from 'react';
+import { createRoot, type Root } from 'react-dom/client';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+
+vi.hoisted(() => {
+ window.electronAPI ??= {
+ invoke: () => Promise.resolve(undefined),
+ eventOn: () => () => {},
+ eventSend: () => {},
+ } as unknown as typeof window.electronAPI;
+});
+
+const state = vi.hoisted(() => ({
+ register: null as unknown,
+ registerError: null as string | null,
+}));
+
+vi.mock('@renderer/features/switch-servers/remote-server-store', () => ({
+ remoteServerStore: {
+ registerFor: () => state.register,
+ registerErrorFor: () => state.registerError,
+ },
+}));
+
+import { SharedConsolesSection } from '@renderer/features/switch-servers/shared-consoles-section';
+import '@renderer/index.css';
+
+let container: HTMLDivElement | null = null;
+let root: Root | null = null;
+
+const SELF_CONSOLE = {
+ consoleId: 'me',
+ name: 'me@laptop',
+ hostAccount: 'deploy',
+ appVersion: '0.37.0',
+ lastSeenAt: new Date(Date.now() - 5 * 60 * 1000).toISOString(),
+};
+
+const OTHER_CONSOLE = {
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ appVersion: '0.36.0',
+ lastSeenAt: new Date(Date.now() - 60 * 60 * 1000).toISOString(),
+};
+
+beforeEach(() => {
+ state.register = null;
+ state.registerError = null;
+});
+
+afterEach(async () => {
+ await act(async () => root?.unmount());
+ container?.remove();
+ container = null;
+ root = null;
+});
+
+async function render() {
+ container = document.createElement('div');
+ document.body.appendChild(container);
+ root = createRoot(container);
+ await act(async () => root!.render(<SharedConsolesSection sshHost="vm-1" />));
+}
+
+describe('listing who uses a shared server', () => {
+ it('renders nothing before the register has been read', async () => {
+ state.register = null;
+ state.registerError = null;
+ await render();
+
+ expect(container?.textContent).toBe('');
+ });
+
+ it('renders nothing for a server nobody has recorded anything on', async () => {
+ state.register = { self: 'me', consoles: [], activity: [] };
+ await render();
+
+ expect(container?.textContent).toBe('');
+ });
+
+ it('lists each Console, marking this Console among them', async () => {
+ state.register = { self: 'me', consoles: [SELF_CONSOLE, OTHER_CONSOLE], activity: [] };
+ await render();
+
+ const items = [...document.querySelectorAll('li')];
+ const selfItem = items.find((li) => li.textContent?.includes('me@laptop'));
+ const otherItem = items.find((li) => li.textContent?.includes('bob@desk'));
+
+ expect(selfItem?.textContent).toContain('me@laptop (as deploy)');
+ expect(selfItem?.textContent).toContain('this Console');
+ expect(otherItem?.textContent).toContain('bob@desk (as bob)');
+ expect(otherItem?.textContent).not.toContain('this Console');
+ });
+
+ it('shows recent activity lines', async () => {
+ state.register = {
+ self: 'me',
+ consoles: [SELF_CONSOLE],
+ activity: [
+ {
+ at: new Date().toISOString(),
+ action: 'started',
+ consoleId: 'me',
+ name: 'me@laptop',
+ hostAccount: 'deploy',
+ },
+ {
+ at: new Date().toISOString(),
+ action: 'stopped',
+ consoleId: 'bob',
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ },
+ ],
+ };
+ await render();
+
+ expect(document.body.textContent).toContain('This Console started it');
+ expect(document.body.textContent).toContain('bob@desk stopped it');
+ });
+
+ it('shows the error reading the register, when there is one', async () => {
+ state.register = null;
+ state.registerError = 'Could not read who uses this server.';
+ await render();
+
+ expect(document.body.textContent).toContain('Could not read who uses this server.');
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/renderer/tests/browser/version-drift-notice-layout.test.tsx b/console/apps/switch-console-desktop/src/renderer/tests/browser/version-drift-notice-layout.test.tsx
index ee136a25e..97b60c3c0 100644
--- a/console/apps/switch-console-desktop/src/renderer/tests/browser/version-drift-notice-layout.test.tsx
+++ b/console/apps/switch-console-desktop/src/renderer/tests/browser/version-drift-notice-layout.test.tsx
@@ -32,6 +32,7 @@ async function renderNotice(deployed: string, expected: string): Promise<HTMLDiv
upgrade={{ state: 'pending', from: deployed, to: expected }}
progress={null}
disabled={false}
+ affected={null}
onRestart={() => {}}
/>
)
diff --git a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
index c1b864c36..aee6b4d2f 100644
--- a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
+++ b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/managed-switch-server.ts
@@ -137,11 +137,15 @@ export type DeployedTelemetry =
* `error` until a retry succeeds.
* - `pending` — the stack is stopped. It is upgraded when it is next started,
* never started at the old version.
+ * - `held` — the stack is running on a shared host and others have used it
+ * recently. The update restarts it for all of them, so it waits for someone
+ * here to run it.
*/
export type ManagedServerUpgrade =
| { state: 'updating'; from: string; to: string }
| { state: 'failed'; from: string; to: string; error: string }
- | { state: 'pending'; from: string; to: string };
+ | { state: 'pending'; from: string; to: string }
+ | { state: 'held'; from: string; to: string };
/** Why a session cannot start on a managed server whose upgrade has not
* finished. Shared so the thrown error and the UI say the same thing. */
@@ -152,6 +156,9 @@ export function managedServerUpgradeBlockedReason(
if (upgrade.state === 'pending') {
return `${serverName} is stopped on switch-core ${upgrade.from} and needs switch-core ${upgrade.to}. Start it from the server's page to update it; sessions on it start once the update finishes.`;
}
+ if (upgrade.state === 'held') {
+ return `${serverName} runs switch-core ${upgrade.from} and this Console needs switch-core ${upgrade.to}. Others use it too, so it is not updated without asking: update it from the server's page. Sessions on it start once the update finishes.`;
+ }
return `Updating ${serverName} from switch-core ${upgrade.from} to ${upgrade.to} failed: ${upgrade.error} Retry the update from the server's page; sessions on it stay paused until it succeeds.`;
}
@@ -226,6 +233,9 @@ export type StartLocalServerResult =
* record what the server is now doing without probing for what it just
* wrote. */
telemetryEnabled: boolean;
+ /** Something the start could not do that does not undo it, for the
+ * server page to say; null when there is nothing to say. */
+ warning: string | null;
}
| { kind: 'docker-unavailable'; reason: 'not-installed' | 'daemon-down'; detail: string }
| { kind: 'version-downgrade'; deployed: string; expected: string }
@@ -234,3 +244,200 @@ export type StartLocalServerResult =
// and still on `deployed`; retrying is safe and resumes where it stopped.
| { kind: 'matrix-migration-failed'; deployed: string; expected: string; detail: string }
| { kind: 'error'; message: string };
+
+/** Outcome of starting a remote stack: a start's, or `cancelled` when the user
+ * stopped waiting for another Console's hold on the stack, before anything
+ * was changed. */
+export type StartRemoteServerResult = StartLocalServerResult | { kind: 'cancelled' };
+
+/**
+ * Outcome of joining a remote stack another Console started. Everything short
+ * of `connected` leaves the host exactly as it was.
+ */
+export type ConnectRemoteServerResult =
+ | {
+ kind: 'connected';
+ serverId: string;
+ /** The switch-core version the stack's settings name, when they do. */
+ deployedVersion: string | null;
+ }
+ /** The stack is set up but stopped: Start it, which keeps its data. */
+ | { kind: 'not-running' }
+ /** Nothing is set up on the host: Start sets one up. */
+ | { kind: 'absent' }
+ /** Another account's stack whose settings were never shared. `message` says
+ * so and what fixes it. */
+ | { kind: 'unshared'; ownerDir: string | null; message: string }
+ | { kind: 'docker-unavailable'; reason: 'not-installed' | 'daemon-down'; detail: string }
+ /** The user stopped waiting for another Console's hold on the stack. */
+ | { kind: 'cancelled' }
+ | { kind: 'error'; message: string };
+
+/** Something a Console did to a shared remote stack, as recorded on its host. */
+export type StackActivityAction = 'started' | 'connected' | 'stopped' | 'reset' | 'disconnected';
+
+/**
+ * A Console that uses a shared remote stack, as it last recorded itself on the
+ * stack's host. Everyone sharing the stack signs in as the one admin account,
+ * so this, not the server, is where they are told apart.
+ */
+export type StackConsole = {
+ /** The Console's random id: the same one it sends the server. */
+ consoleId: string;
+ /** `user@host` of the desktop the Console runs on. */
+ name: string;
+ /** The account on the stack's host that Console reaches it as. */
+ hostAccount: string;
+ appVersion: string;
+ /** ISO timestamp of the last time it started, joined or picked up the stack. */
+ lastSeenAt: string;
+};
+
+export type StackActivityEntry = {
+ /** ISO timestamp. */
+ at: string;
+ action: StackActivityAction;
+ consoleId: string;
+ name: string;
+ hostAccount: string;
+};
+
+/** The Consoles recorded on a shared stack's host, and what they last did to it. */
+export type StackRegister = {
+ /** This Console's own id, so it can tell itself apart in the lists. */
+ self: string;
+ /** Most recently seen first. */
+ consoles: StackConsole[];
+ /** Most recent first, and only the latest few. */
+ activity: StackActivityEntry[];
+};
+
+/** How long a Console counts as still using a shared server after it was last
+ * seen: long enough to span a holiday, short enough that someone who tried it
+ * once in the spring is not warned about in the autumn. */
+export const RECENTLY_SEEN_DAYS = 14;
+
+/** The other Consoles seen on the server recently, most recent first — the
+ * people a stop, restart, reset or update from here will affect. */
+export function othersRecentlySeen(register: StackRegister | null, now: Date): StackConsole[] {
+ if (!register) return [];
+ const cutoff = now.getTime() - RECENTLY_SEEN_DAYS * 24 * 60 * 60 * 1000;
+ return register.consoles.filter((c) => {
+ if (c.consoleId === register.self) return false;
+ const seen = Date.parse(c.lastSeenAt);
+ return Number.isFinite(seen) && seen >= cutoff;
+ });
+}
+
+/** What a Console holding a shared server's lock is doing to it. */
+export type ServerLockAction =
+ | 'starting'
+ | 'updating'
+ | 'connecting'
+ | 'checking'
+ | 'stopping'
+ | 'resetting';
+
+/**
+ * A Console's hold on a shared server's lock: whoever changes a server others
+ * use holds it, so two Consoles cannot start, update, stop or reset it over
+ * each other. Both durations are by the host's clock.
+ */
+export type ServerLockHolder = {
+ name: string;
+ hostAccount: string;
+ action: ServerLockAction;
+ heldForSeconds: number;
+ /** When the lock lapses unless its holder renews it — which a Console that
+ * has gone away no longer does. */
+ expiresInSeconds: number;
+};
+
+const LOCK_ACTION_PHRASE: Record<ServerLockAction, string> = {
+ starting: 'starting',
+ updating: 'updating',
+ connecting: 'connecting to',
+ checking: 'checking',
+ stopping: 'stopping',
+ resetting: 'resetting',
+};
+
+function lockHolderName(holder: ServerLockHolder): string {
+ return `${holder.name} (as ${holder.hostAccount})`;
+}
+
+function minutes(seconds: number): string {
+ const whole = Math.max(1, Math.ceil(seconds / 60));
+ return whole === 1 ? 'a minute' : `${whole} minutes`;
+}
+
+/** What a Console waiting for the lock says while it waits. */
+export function waitingForLockMessage(holder: ServerLockHolder): string {
+ return `Waiting for ${lockHolderName(holder)} to finish ${LOCK_ACTION_PHRASE[holder.action]} the server…`;
+}
+
+/** What a check of the server says when the user stopped waiting to make it. */
+export function stoppedWaitingForLockMessage(holder: ServerLockHolder | null): string {
+ const what = holder
+ ? `${lockHolderName(holder)} to finish ${LOCK_ACTION_PHRASE[holder.action]} the server`
+ : 'the server';
+ return `Stopped waiting for ${what}, so this Console has not checked it since.`;
+}
+
+/** What the Add Server step says of a server someone is changing right now. */
+export function lockHolderSentence(holder: ServerLockHolder): string {
+ return (
+ `${lockHolderName(holder)} is ${LOCK_ACTION_PHRASE[holder.action]} this server right now. ` +
+ `Starting or connecting here waits until they are done.`
+ );
+}
+
+/** Why a stop or reset was refused: someone else is changing the server. */
+export function serverBusyMessage(hostLabel: string, holder: ServerLockHolder): string {
+ return (
+ `${lockHolderName(holder)} is ${LOCK_ACTION_PHRASE[holder.action]} the server on ` +
+ `${hostLabel} right now, so nothing was changed. Try again once they are done. If that ` +
+ `Console has gone away, its hold on the server clears by itself within ` +
+ `${minutes(holder.expiresInSeconds)}.`
+ );
+}
+
+/**
+ * A stop or reset of a shared server turned away because another Console is
+ * changing it right now. Nothing was touched. Lives with the model, like
+ * {@link ManagedServerStoppedError}, so the RPC boundary and the renderer can
+ * recognise a refusal that is a state to show rather than a fault.
+ */
+export class ServerBusyError extends Error {
+ constructor(
+ readonly holder: ServerLockHolder,
+ hostLabel: string
+ ) {
+ super(serverBusyMessage(hostLabel, holder));
+ this.name = 'ServerBusyError';
+ }
+}
+
+/**
+ * What a remote host has of a stack, for the renderer to decide what to offer
+ * — Connect, Start, or neither. The main process's reading of the host with
+ * every secret left out. `busy` is the Console changing the stack right now,
+ * if any, which whatever is offered will wait for.
+ */
+export type RemoteStackProbe =
+ | { kind: 'absent'; busy: ServerLockHolder | null }
+ /** A stack whose settings this account can read. `shared` is false for one
+ * this account started before settings were shared; connecting shares it.
+ * `drift` compares the version its settings name with this build's pin. */
+ | {
+ kind: 'present';
+ running: boolean;
+ deployedVersion: string | null;
+ shared: boolean;
+ drift: SwitchVersionDrift | null;
+ busy: ServerLockHolder | null;
+ }
+ | { kind: 'unshared'; running: boolean; ownerDir: string | null; message: string }
+ | { kind: 'incomplete'; running: boolean; missing: string[] }
+ | { kind: 'unreadable'; reason: string }
+ | { kind: 'docker-unavailable'; reason: 'not-installed' | 'daemon-down'; detail: string };
diff --git a/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/server-lock-messages.test.ts b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/server-lock-messages.test.ts
new file mode 100644
index 000000000..86ac94039
--- /dev/null
+++ b/console/apps/switch-console-desktop/src/shared/core/managed-switch-server/server-lock-messages.test.ts
@@ -0,0 +1,67 @@
+import { describe, expect, it } from 'vitest';
+import {
+ lockHolderSentence,
+ ServerBusyError,
+ type ServerLockHolder,
+ serverBusyMessage,
+ stoppedWaitingForLockMessage,
+ waitingForLockMessage,
+} from './managed-switch-server';
+
+const bob: ServerLockHolder = {
+ name: 'bob@desk',
+ hostAccount: 'bob',
+ action: 'starting',
+ heldForSeconds: 40,
+ expiresInSeconds: 80,
+};
+
+describe('messages about the server lock', () => {
+ it('names who a wait is for and what they are doing', () => {
+ expect(waitingForLockMessage(bob)).toBe(
+ 'Waiting for bob@desk (as bob) to finish starting the server…'
+ );
+ expect(waitingForLockMessage({ ...bob, action: 'connecting' })).toBe(
+ 'Waiting for bob@desk (as bob) to finish connecting to the server…'
+ );
+ });
+
+ it('tells the setup step that Start or Connect will wait', () => {
+ expect(lockHolderSentence({ ...bob, action: 'updating' })).toBe(
+ 'bob@desk (as bob) is updating this server right now. Starting or connecting here waits ' +
+ 'until they are done.'
+ );
+ });
+
+ it('refuses a stop or reset naming who is busy, and when a gone Console stops counting', () => {
+ expect(serverBusyMessage('vm-1', bob)).toBe(
+ 'bob@desk (as bob) is starting the server on vm-1 right now, so nothing was changed. Try ' +
+ 'again once they are done. If that Console has gone away, its hold on the server clears ' +
+ 'by itself within 2 minutes.'
+ );
+ expect(serverBusyMessage('vm-1', { ...bob, expiresInSeconds: 20 })).toMatch(
+ /within a minute\.$/
+ );
+ expect(serverBusyMessage('vm-1', { ...bob, expiresInSeconds: 0 })).toMatch(
+ /within a minute\.$/
+ );
+ });
+
+ it('carries the refusal and its holder as an error the RPC boundary recognises', () => {
+ const error = new ServerBusyError(bob, 'vm-1');
+
+ expect(error.name).toBe('ServerBusyError');
+ expect(error.holder).toBe(bob);
+ expect(error.message).toBe(serverBusyMessage('vm-1', bob));
+ });
+
+ it('says a check that stopped waiting has not happened, with or without a holder', () => {
+ expect(stoppedWaitingForLockMessage({ ...bob, action: 'resetting' })).toBe(
+ 'Stopped waiting for bob@desk (as bob) to finish resetting the server, so this Console has ' +
+ 'not checked it since.'
+ );
+ expect(stoppedWaitingForLockMessage(null)).toBe(
+ 'Stopped waiting for the server, so this Console has not checked it since.'
+ );
+ });
+});
diff --git a/console/apps/switch-console-desktop/src/shared/events/remoteSwitchServerEvents.ts b/console/apps/switch-console-desktop/src/shared/events/remoteSwitchServerEvents.ts
index cda92be77..abf317588 100644
--- a/console/apps/switch-console-desktop/src/shared/events/remoteSwitchServerEvents.ts
+++ b/console/apps/switch-console-desktop/src/shared/events/remoteSwitchServerEvents.ts
@@ -1,9 +1,24 @@
-import type { LocalServerStatus } from '@shared/core/managed-switch-server/managed-switch-server';
+import type {
+ LocalServerStatus,
+ ServerLockHolder,
+} from '@shared/core/managed-switch-server/managed-switch-server';
import { defineEvent } from '@shared/lib/ipc/events';
/** Status of a remote-managed stack, tagged with the SSH host it runs on (the
* renderer keeps one entry per host). Reuses the local status shape. */
-export type RemoteServerStatus = LocalServerStatus & { sshHost: string };
+export type RemoteServerStatus = LocalServerStatus & {
+ sshHost: string;
+ /** Something done to the stack from elsewhere (another Console, or the host
+ * itself) that the user should know, or null. */
+ notice: string | null;
+ /** Why this Console could not record what it did on the stack's host, or
+ * null. The operation still went ahead, but other Consoles will not see it.
+ * Cleared by the next record that succeeds. */
+ recordWarning: string | null;
+ /** The other Console whose lock this one is waiting on before it can change
+ * or read the stack, or null. The wait can be cancelled. */
+ waitingFor: ServerLockHolder | null;
+};
export const remoteServerStatusChannel = defineEvent<RemoteServerStatus>(
'remote-switch-server:status'
diff --git a/core/switch_core/logging_context.py b/core/switch_core/logging_context.py
index 001816a3e..7dc1b7436 100644
--- a/core/switch_core/logging_context.py
+++ b/core/switch_core/logging_context.py
@@ -52,6 +52,8 @@ CONTEXT_FIELDS: tuple[str, ...] = (
"agent_id",
"room_id",
"user_id",
+ "console_id",
+ "console_name",
)
@@ -62,6 +64,10 @@ class LogContext:
agent_id: str | None = None
room_id: str | None = None
user_id: str | None = None
+ # The Switch Console that sent the request (see `request_context.py`):
+ # people can share one sign-in, so `user_id` alone cannot tell them apart.
+ console_id: str | None = None
+ console_name: str | None = None
_EMPTY = LogContext()
@@ -165,4 +171,6 @@ class LogContextFilter(logging.Filter):
record.agent_id = context.agent_id
record.room_id = context.room_id
record.user_id = context.user_id
+ record.console_id = context.console_id
+ record.console_name = context.console_name
return True
diff --git a/core/switch_core/request_context.py b/core/switch_core/request_context.py
index 84a197698..b393b71ee 100644
--- a/core/switch_core/request_context.py
+++ b/core/switch_core/request_context.py
@@ -1,4 +1,4 @@
-"""ASGI middleware binding a request id to the log context.
+"""ASGI middleware binding a request's own identifiers to the log context.
Outermost in the stack, so every line emitted while serving a request — ours,
uvicorn's, a library's — can be tied back to the request that caused it. The
@@ -6,6 +6,12 @@ identifiers a request is *about* are bound further in, where they are resolved:
the agent by the bearer middleware, the user by the gateway's authentication
dependency.
+Two are read here: the request id (``X-Request-Id``, or generated) and the
+Switch Console that sent it (``X-Switch-Console-Id``, ``X-Switch-Console-Name``),
+which tells apart people sharing one sign-in. The console headers are
+attribution, not authentication: any caller can send them, so nothing may be
+authorised on them.
+
Plain ASGI rather than Starlette's ``BaseHTTPMiddleware`` on purpose: that one
runs the rest of the app in a separate task, so a context variable set here
would not be visible to the endpoint.
@@ -13,6 +19,7 @@ would not be visible to the endpoint.
from __future__ import annotations
+import string
from uuid import uuid4
from starlette.types import ASGIApp, Receive, Scope, Send
@@ -20,9 +27,20 @@ from starlette.types import ASGIApp, Receive, Scope, Send
from switch_core.logging_context import log_context
_REQUEST_ID_HEADER = b"x-request-id"
-# An id from a caller is untrusted input that ends up in every log line for the
-# request, so it is truncated and stripped of anything that could forge a line.
+_CONSOLE_ID_HEADER = b"x-switch-console-id"
+_CONSOLE_NAME_HEADER = b"x-switch-console-name"
+
+# Everything read here is untrusted input that ends up in every log line for
+# the request, so each value is truncated and stripped of anything that could
+# forge a line or break the `key=value` text format.
_MAX_REQUEST_ID_LENGTH = 64
+# A console id is a UUID; the canonical form is 36 characters.
+_MAX_CONSOLE_ID_LENGTH = 36
+_CONSOLE_ID_CHARACTERS = frozenset(string.hexdigits + "-")
+# A console name is `user@host`: POSIX user names and DNS host names both fit
+# in this set, and nothing in it can end a field in either log format.
+_MAX_CONSOLE_NAME_LENGTH = 64
+_CONSOLE_NAME_CHARACTERS = frozenset(string.ascii_letters + string.digits + "._@+-")
class RequestContextMiddleware:
@@ -34,12 +52,39 @@ class RequestContextMiddleware:
await self.app(scope, receive, send)
return
- with log_context(request_id=_request_id(scope)):
+ headers = dict(scope.get("headers", []))
+ with log_context(
+ request_id=_request_id(headers),
+ console_id=_restricted(
+ headers,
+ _CONSOLE_ID_HEADER,
+ _CONSOLE_ID_CHARACTERS,
+ _MAX_CONSOLE_ID_LENGTH,
+ ),
+ console_name=_restricted(
+ headers,
+ _CONSOLE_NAME_HEADER,
+ _CONSOLE_NAME_CHARACTERS,
+ _MAX_CONSOLE_NAME_LENGTH,
+ ),
+ ):
await self.app(scope, receive, send)
-def _request_id(scope: Scope) -> str:
- headers = dict(scope.get("headers", []))
- raw = headers.get(_REQUEST_ID_HEADER, b"").decode("utf-8", "replace")
+def _header(headers: dict[bytes, bytes], name: bytes) -> str:
+ return headers.get(name, b"").decode("utf-8", "replace")
+
+
+def _request_id(headers: dict[bytes, bytes]) -> str:
+ raw = _header(headers, _REQUEST_ID_HEADER)
sanitised = "".join(c for c in raw if c.isprintable() and c not in " \t")
return sanitised[:_MAX_REQUEST_ID_LENGTH] or uuid4().hex
+
+
+def _restricted(
+ headers: dict[bytes, bytes], name: bytes, allowed: frozenset[str], limit: int
+) -> str | None:
+ """The header's value reduced to `allowed` characters and `limit` long, or
+ None when nothing is left."""
+ kept = "".join(c for c in _header(headers, name) if c in allowed)
+ return kept[:limit] or None
diff --git a/core/tests/switch_core/test_logging_config.py b/core/tests/switch_core/test_logging_config.py
index fd4b85962..9f01440ec 100644
--- a/core/tests/switch_core/test_logging_config.py
+++ b/core/tests/switch_core/test_logging_config.py
@@ -252,13 +252,33 @@ def test_filter_stamps_every_field(monkeypatch: pytest.MonkeyPatch) -> None:
"x", logging.INFO, __file__, 1, "m", args=(), exc_info=None
)
- with log_context(request_id="r", agent_id="a", user_id="u"):
+ with log_context(
+ request_id="r",
+ agent_id="a",
+ user_id="u",
+ console_id="c",
+ console_name="alice@laptop",
+ ):
LogContextFilter("acme").filter(record)
assert record.tenant_id == "acme"
assert record.request_id == "r"
assert record.agent_id == "a"
assert record.user_id == "u"
+ assert record.console_id == "c"
+ assert record.console_name == "alice@laptop"
+
+
+def test_the_console_rides_alongside_the_user_in_json(json_lines) -> None:
+ logger, lines = json_lines
+
+ with log_context(user_id="admin", console_id="c-1", console_name="bob@vm"):
+ logger.info("stopped the stack")
+
+ entry = _one(lines)
+ assert entry["user_id"] == "admin"
+ assert entry["console_id"] == "c-1"
+ assert entry["console_name"] == "bob@vm"
def test_bound_tenant_beats_the_deployment_default() -> None:
diff --git a/core/tests/switch_core/test_request_context.py b/core/tests/switch_core/test_request_context.py
index 36591f993..bd03f6a5a 100644
--- a/core/tests/switch_core/test_request_context.py
+++ b/core/tests/switch_core/test_request_context.py
@@ -75,12 +75,81 @@ async def test_a_hostile_request_id_cannot_forge_log_lines() -> None:
assert len(context.request_id) == 64
+async def test_a_request_without_console_headers_is_unattributed() -> None:
+ app = _Recorder()
+
+ context = await _call(app, [])
+
+ assert context.console_id is None
+ assert context.console_name is None
+
+
+async def test_the_sending_console_is_bound() -> None:
+ app = _Recorder()
+
+ context = await _call(
+ app,
+ [
+ (b"x-switch-console-id", b"3f2a9c1e-5b7d-4e8f-a1b2-c3d4e5f6a7b8"),
+ (b"x-switch-console-name", b"[email protected]"),
+ ],
+ )
+
+ assert context.console_id == "3f2a9c1e-5b7d-4e8f-a1b2-c3d4e5f6a7b8"
+ assert context.console_name == "[email protected]"
+
+
+async def test_hostile_console_headers_cannot_forge_log_lines() -> None:
+ app = _Recorder()
+
+ context = await _call(
+ app,
+ [
+ (b"x-switch-console-id", b"abc\nERROR forged " + b"f" * 100),
+ (b"x-switch-console-name", b"eve@host user_id=admin\n" + b"x" * 200),
+ ],
+ )
+
+ assert context.console_id is not None
+ assert context.console_name is not None
+ for value in (context.console_id, context.console_name):
+ assert "\n" not in value
+ assert " " not in value
+ assert "=" not in value
+ assert len(context.console_id) == 36
+ assert len(context.console_name) == 64
+ assert context.console_name.startswith("eve@hostuser_idadmin")
+
+
+async def test_a_console_header_with_nothing_usable_is_unattributed() -> None:
+ app = _Recorder()
+
+ context = await _call(
+ app,
+ [
+ (b"x-switch-console-id", b"zzzz ###"),
+ (b"x-switch-console-name", b"\n\t ;;"),
+ ],
+ )
+
+ assert context.console_id is None
+ assert context.console_name is None
+
+
async def test_the_context_is_unbound_after_the_request() -> None:
app = _Recorder()
- await _call(app, [])
+ await _call(
+ app,
+ [
+ (b"x-switch-console-id", b"3f2a9c1e"),
+ (b"x-switch-console-name", b"alice@laptop"),
+ ],
+ )
assert current_log_context().request_id is None
+ assert current_log_context().console_id is None
+ assert current_log_context().console_name is None
async def test_the_context_is_unbound_when_the_app_raises() -> None:
diff --git a/docs/old/observability.md b/docs/old/observability.md
index 85f5373e1..980dc08d0 100644
--- a/docs/old/observability.md
+++ b/docs/old/observability.md
@@ -149,7 +149,11 @@ handler — so records from libraries carry them too. A room id is a log field
and deliberately not a metric attribute: it is unbounded and belongs to one
tenant, which rules it out of a dashboard label for the reasons under "Why
there is a catalogue" — and following one room through a failure is the single
-most common thing anyone asks these logs for. Set `LOG_FORMAT=json` to get them as
+most common thing anyone asks these logs for. A request from Switch Console also
+carries `console_id` and `console_name`: on a server a Console runs for its
+user, everyone with access to the host signs in as the one seeded account, so
+`user_id` cannot tell them apart and these can. They are attribution the caller
+supplies, not authentication. Set `LOG_FORMAT=json` to get them as
fields rather than inside the message text; the default is `text`, which is for
reading in a terminal, so anywhere the logs are actually collected wants the
JSON form. With `OTLP_LOGS_ENABLED` the same records are *also* posted to the