Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(session): keep a stopped session down across restarts

Yudhi Armyndharisa3a8884de791
diff --git a/.env.example b/.env.example
index 99a6a7f60..13bc25e4e 100644
--- a/.env.example
+++ b/.env.example
@@ -47,7 +47,8 @@ LOG_LEVEL=info # error | warn | info | debug | verbose
# see docs/13-horizontal-scaling.md). Auto-start selects sessions with a non-null `phone`. Both a
# completed logout (200, credentials wiped) and an incomplete logout (502, `code: 'SESSION_LOGOUT_INCOMPLETE'`)
# clear `phone`, so neither is auto-started on boot — an incomplete-logout session must be started
-# explicitly and the logout retried. A session that must stay down can simply be left as-is.
+# explicitly and the logout retried. A session stopped through POST /stop or /force-kill is not
+# auto-started either (nor adopted by the takeover sweep) until an explicit POST /start.
# Docker Compose: this value only reaches the container from v0.12.0 onward — the bundled
# docker-compose.yml did not forward it before then (#985), so setting it on an earlier version had
# no effect and sessions stayed DISCONNECTED on boot regardless.
diff --git a/docs/05-database-design.md b/docs/05-database-design.md
index ca2973c44..f0fba72a4 100644
--- a/docs/05-database-design.md
+++ b/docs/05-database-design.md
@@ -208,6 +208,7 @@ erDiagram
timestamp claimedAt
varchar nodeUrl
timestamp leaseExpiresAt
+ varchar desiredState
timestamp createdAt
timestamp updatedAt
}
@@ -319,6 +320,10 @@ CREATE TABLE sessions (
"claimedAt" TIMESTAMP,
"nodeUrl" VARCHAR(2048),
"leaseExpiresAt" TIMESTAMP,
+ -- 'stopped' after POST /stop or /force-kill, NULL again after an explicit POST /start. Boot
+ -- auto-start and the takeover sweep skip a stopped row. Server-owned: not in any session
+ -- response, only in the /api/infra/export-data backup.
+ "desiredState" VARCHAR(20),
"createdAt" TIMESTAMP NOT NULL DEFAULT NOW(),
"updatedAt" TIMESTAMP NOT NULL DEFAULT NOW()
);
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.md
index 5bb6ff6a2..36dcc9395 100644
--- a/docs/06-api-specification.md
+++ b/docs/06-api-specification.md
@@ -636,7 +636,9 @@ No request body.
Returned via `transformSession`. Status typically transitions to `initializing` / `qr_ready`.
-**Errors:** `400` session already started / already starting · `401` · `403` · `404` not found · `409` credential teardown for the same session name still in flight (retryable; body carries `code: 'SESSION_NAME_TEARDOWN_PENDING'`; no destructive side effect runs before the refusal — a retry after cleanup settles proceeds) · `504` the engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or an unreachable `proxyUrl`); the engine is torn down, so the start can be retried once the cause is addressed
+A session stopped with `POST /stop` or `POST /force-kill` is marked to run again only once the start gets past the `400` and `409` refusals below; a refused start leaves the stop in place.
+
+**Errors:** `400` session already started / already starting · `401` · `403` · `404` not found · `409` credential teardown for the same session name still in flight (retryable; body carries `code: 'SESSION_NAME_TEARDOWN_PENDING'`; no destructive side effect runs before the refusal — a retry after cleanup settles proceeds), or, with no `code`, a `POST /stop` or `POST /force-kill` of this session that finished while the start was waiting (the stop stands; a new `POST /start` clears it and starts the session) · `504` the engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or an unreachable `proxyUrl`); the engine is torn down, so the start can be retried once the cause is addressed
#### POST /api/sessions/:sessionId/stop
@@ -672,6 +674,8 @@ No request body.
Returned via `transformSession`; status typically becomes `disconnected`.
+A stopped session stays down: boot auto-start (`AUTO_START_SESSIONS`) and the multi-node takeover sweep skip it until an explicit `POST /start`. The stop is recorded before the teardown, so a `502` stop keeps it too; a `409` refusal records nothing.
+
**Errors:** `401` · `403` · `404` not found · `409` another node currently holds this session's live engine (multi-node deployments) · `502` `SESSION_STOP_INCOMPLETE` — session stopped locally but the engine teardown did not complete (retryable; the graceful disconnect and the force-destroy escalation both failed, status `disconnected`, no success audit)
#### POST /api/sessions/:sessionId/logout
@@ -707,8 +711,8 @@ report which.
With `AUTO_START_SESSIONS=true`, auto-start selects sessions whose `phone` is non-null. Both a `200`
and a `502` logout clear `phone`, and a WhatsApp-initiated unlink reported by a running engine clears
it the same way, so none of those are auto-started on boot — an incomplete-logout (`502`)
-session must be started explicitly and the logout retried by hand. A session that must stay down can
-simply be left as-is.
+session must be started explicitly and the logout retried by hand. A session stopped through
+`POST /stop` or `POST /force-kill` is not auto-started either, until an explicit `POST /start`.
**Auth:** API key (OPERATOR) · **Scope:** session-scoped
@@ -778,6 +782,8 @@ No request body.
Returned via `transformSession`.
+Like `stop`, a force-killed session stays down across restarts and takeover until an explicit `POST /start`; a `400` refusal records nothing.
+
**Errors:** `400` session is not started (no live engine to kill) · `401` · `403` · `404` not found
#### POST /api/sessions/:sessionId/pairing-code
diff --git a/docs/13-horizontal-scaling.md b/docs/13-horizontal-scaling.md
index 6ac996ca2..f98c66b7b 100644
--- a/docs/13-horizontal-scaling.md
+++ b/docs/13-horizontal-scaling.md
@@ -44,6 +44,8 @@
> `qr_ready` row is marked too while it has no phone, since the sweep never adopts a row without one;
> a `qr_ready` row with a phone keeps its status, so the correction never makes it adoptable.
> Nothing else revisits such a row, since the boot reset skips a foreign claim that is still live.
+> A session an operator stopped (`POST /stop` or `/force-kill`) is never adopted, even when its
+> claim did lapse, until an explicit `POST /start`.
>
> Known limitation: a holder that is alive but cannot reach the database for more than three lease
> TTLs minus one heartbeat (160s at defaults) is marked disconnected by a peer too, and does not write its status back when it reconnects. Its
diff --git a/src/database/migrations/1786600000000-AddSessionDesiredState.ts b/src/database/migrations/1786600000000-AddSessionDesiredState.ts
new file mode 100644
index 000000000..95033245d
--- /dev/null
+++ b/src/database/migrations/1786600000000-AddSessionDesiredState.ts
@@ -0,0 +1,21 @@
+import { MigrationInterface, QueryRunner } from 'typeorm';
+
+/**
+ * Adds `sessions.desiredState`: 'stopped' once an operator stops a session, cleared by an explicit
+ * start. Boot auto-start and the takeover sweep skip stopped rows, so a deliberate stop is no
+ * longer undone by the next restart. NULL for every existing row, which keeps today's behaviour.
+ * Hand-authored because `synchronize` is off on the `data` connection for Postgres.
+ */
+export class AddSessionDesiredState1786600000000 implements MigrationInterface {
+ name = 'AddSessionDesiredState1786600000000';
+
+ public async up(queryRunner: QueryRunner): Promise<void> {
+ if (await queryRunner.hasColumn('sessions', 'desiredState')) return;
+ await queryRunner.query(`ALTER TABLE "sessions" ADD COLUMN "desiredState" varchar(20)`);
+ }
+
+ public async down(queryRunner: QueryRunner): Promise<void> {
+ if (!(await queryRunner.hasColumn('sessions', 'desiredState'))) return;
+ await queryRunner.query(`ALTER TABLE "sessions" DROP COLUMN "desiredState"`);
+ }
+}
diff --git a/src/database/migrations/__tests__/1786600000000-AddSessionDesiredState.spec.ts b/src/database/migrations/__tests__/1786600000000-AddSessionDesiredState.spec.ts
new file mode 100644
index 000000000..b778331fa
--- /dev/null
+++ b/src/database/migrations/__tests__/1786600000000-AddSessionDesiredState.spec.ts
@@ -0,0 +1,42 @@
+import { DataSource } from 'typeorm';
+import { AddSessionDesiredState1786600000000 } from '../1786600000000-AddSessionDesiredState';
+
+describe('AddSessionDesiredState migration', () => {
+ let ds: DataSource;
+
+ beforeEach(async () => {
+ ds = new DataSource({ type: 'better-sqlite3', database: ':memory:' });
+ await ds.initialize();
+ await ds.query(`CREATE TABLE "sessions" ("id" varchar PRIMARY KEY NOT NULL, "name" varchar NOT NULL)`);
+ await ds.query(`INSERT INTO "sessions" ("id", "name") VALUES ('s1', 'main')`);
+ });
+
+ afterEach(async () => {
+ await ds.destroy();
+ });
+
+ it('adds a nullable column that reads NULL on existing rows, and drops it again', async () => {
+ const runner = ds.createQueryRunner();
+ const migration = new AddSessionDesiredState1786600000000();
+
+ await migration.up(runner);
+ expect(await runner.hasColumn('sessions', 'desiredState')).toBe(true);
+ const rows = (await runner.query(`SELECT "desiredState" FROM "sessions"`)) as Array<{ desiredState: unknown }>;
+ expect(rows).toEqual([{ desiredState: null }]);
+
+ await migration.down(runner);
+ expect(await runner.hasColumn('sessions', 'desiredState')).toBe(false);
+ await runner.release();
+ });
+
+ it('up() and down() are idempotent', async () => {
+ const runner = ds.createQueryRunner();
+ const migration = new AddSessionDesiredState1786600000000();
+
+ await migration.up(runner);
+ await expect(migration.up(runner)).resolves.not.toThrow();
+ await migration.down(runner);
+ await expect(migration.down(runner)).resolves.not.toThrow();
+ await runner.release();
+ });
+});
diff --git a/src/database/migrations/__tests__/__fixtures__/known-migration-drift.json b/src/database/migrations/__tests__/__fixtures__/known-migration-drift.json
index eb07675c9..0554872fb 100644
--- a/src/database/migrations/__tests__/__fixtures__/known-migration-drift.json
+++ b/src/database/migrations/__tests__/__fixtures__/known-migration-drift.json
@@ -44,8 +44,8 @@
"CREATE INDEX \"IDX_ingress_events_dispatchState\" ON \"ingress_events\" (\"dispatchState\", \"createdAt\") ",
"CREATE INDEX \"IDX_ingress_events_createdAt\" ON \"ingress_events\" (\"createdAt\") ",
"CREATE UNIQUE INDEX \"UQ_ingress_events_instance_delivery\" ON \"ingress_events\" (\"pluginId\", \"instanceId\", \"providerDeliveryId\") ",
- "CREATE TABLE \"temporary_sessions\" (\"id\" varchar PRIMARY KEY NOT NULL, \"name\" varchar(100) NOT NULL, \"status\" varchar(50) NOT NULL DEFAULT ('created'), \"phone\" varchar(20), \"pushName\" varchar(100), \"config\" text NOT NULL DEFAULT ('{}'), \"proxyUrl\" varchar(255), \"proxyType\" varchar(10), \"connectedAt\" text, \"lastActiveAt\" text, \"createdAt\" datetime NOT NULL DEFAULT (datetime('now')), \"updatedAt\" datetime NOT NULL DEFAULT (datetime('now')), \"nodeId\" varchar(190), \"claimedAt\" text, \"leaseExpiresAt\" text, \"nodeUrl\" varchar(2048), CONSTRAINT \"UQ_ac984ccbd8b01af155e1874e8cb\" UNIQUE (\"name\"))",
- "INSERT INTO \"temporary_sessions\"(\"id\", \"name\", \"status\", \"phone\", \"pushName\", \"config\", \"proxyUrl\", \"proxyType\", \"connectedAt\", \"lastActiveAt\", \"createdAt\", \"updatedAt\", \"nodeId\", \"claimedAt\", \"leaseExpiresAt\", \"nodeUrl\") SELECT \"id\", \"name\", \"status\", \"phone\", \"pushName\", \"config\", \"proxyUrl\", \"proxyType\", \"connectedAt\", \"lastActiveAt\", \"createdAt\", \"updatedAt\", \"nodeId\", \"claimedAt\", \"leaseExpiresAt\", \"nodeUrl\" FROM \"sessions\"",
+ "CREATE TABLE \"temporary_sessions\" (\"id\" varchar PRIMARY KEY NOT NULL, \"name\" varchar(100) NOT NULL, \"status\" varchar(50) NOT NULL DEFAULT ('created'), \"phone\" varchar(20), \"pushName\" varchar(100), \"config\" text NOT NULL DEFAULT ('{}'), \"proxyUrl\" varchar(255), \"proxyType\" varchar(10), \"connectedAt\" text, \"lastActiveAt\" text, \"createdAt\" datetime NOT NULL DEFAULT (datetime('now')), \"updatedAt\" datetime NOT NULL DEFAULT (datetime('now')), \"nodeId\" varchar(190), \"claimedAt\" text, \"leaseExpiresAt\" text, \"nodeUrl\" varchar(2048), \"desiredState\" varchar(20), CONSTRAINT \"UQ_ac984ccbd8b01af155e1874e8cb\" UNIQUE (\"name\"))",
+ "INSERT INTO \"temporary_sessions\"(\"id\", \"name\", \"status\", \"phone\", \"pushName\", \"config\", \"proxyUrl\", \"proxyType\", \"connectedAt\", \"lastActiveAt\", \"createdAt\", \"updatedAt\", \"nodeId\", \"claimedAt\", \"leaseExpiresAt\", \"nodeUrl\", \"desiredState\") SELECT \"id\", \"name\", \"status\", \"phone\", \"pushName\", \"config\", \"proxyUrl\", \"proxyType\", \"connectedAt\", \"lastActiveAt\", \"createdAt\", \"updatedAt\", \"nodeId\", \"claimedAt\", \"leaseExpiresAt\", \"nodeUrl\", \"desiredState\" FROM \"sessions\"",
"DROP TABLE \"sessions\"",
"ALTER TABLE \"temporary_sessions\" RENAME TO \"sessions\"",
"DROP INDEX \"IDX_webhooks_sessionId\"",
diff --git a/src/modules/infra/infra-data.controller.spec.ts b/src/modules/infra/infra-data.controller.spec.ts
index 6ef989f49..3c3929ab2 100644
--- a/src/modules/infra/infra-data.controller.spec.ts
+++ b/src/modules/infra/infra-data.controller.spec.ts
@@ -146,12 +146,25 @@ describe('InfraDataController.importData round-trips export-data (no silent mess
expect(new Date(restored.leaseExpiresAt as unknown as string).toISOString()).toBe(leaseExpiresAt.toISOString());
});
+ it('keeps a stopped session stopped across an export and import', async () => {
+ await seedSession('s1');
+ await seedSession('s2');
+ await ds.getRepository(Session).update({ id: 's1' }, { desiredState: 'stopped' });
+
+ const dump = await controller.exportData();
+ const res = await controller.importData({ tables: dump.tables });
+ expect(res.warnings).toEqual([]);
+
+ expect((await ds.getRepository(Session).findOneByOrFail({ id: 's1' })).desiredState).toBe('stopped');
+ expect((await ds.getRepository(Session).findOneByOrFail({ id: 's2' })).desiredState).toBeNull();
+ });
+
it('carries LIVE claims forward by their remaining time, including a peer node’s', async () => {
await seedSession('s1');
await seedSession('s2');
const dump = await controller.exportData();
- // Seeded AFTER the export on purpose: the sessions importer writes 12 columns and none of them is
+ // Seeded AFTER the export on purpose: the sessions importer writes 13 columns and none of them is
// ownership, so the dump cannot carry these values — and stamping them here keeps the assertion
// margin free of the export's cost. That margin is consumed by the import preamble and the
// commit, NOT by the stall below: a late timer moves the committed lease and `Date.now()` by the
diff --git a/src/modules/infra/migration-tables.types.ts b/src/modules/infra/migration-tables.types.ts
index 83360fd4a..932918838 100644
--- a/src/modules/infra/migration-tables.types.ts
+++ b/src/modules/infra/migration-tables.types.ts
@@ -12,6 +12,8 @@ export interface SessionRow {
lastActiveAt: string | null;
createdAt: string;
updatedAt: string;
+ // Absent from backups written before the column existed.
+ desiredState?: string | null;
}
export interface WebhookRow {
diff --git a/src/modules/infra/table-importers.spec.ts b/src/modules/infra/table-importers.spec.ts
index ca759fd49..56e0ab87f 100644
--- a/src/modules/infra/table-importers.spec.ts
+++ b/src/modules/infra/table-importers.spec.ts
@@ -27,3 +27,20 @@ describe('sessions table importer', () => {
expect(sessions?.skip?.(row({ name: '../alice' }) as never)).toMatch(/unsafe name/);
});
});
+
+describe('sessions table importer: desiredState', () => {
+ const sessions = TABLE_IMPORTERS.find(importer => importer.key === 'sessions');
+ const base = { id: '0a941dac-a965-45e7-b318-74ae8be134f0', name: 'my-bot', status: 'disconnected' };
+ const mapped = (extra: Record<string, unknown>): unknown[] => sessions!.map({ ...base, ...extra } as never);
+
+ it('writes desiredState so a stopped session stays down after a restore', () => {
+ expect(sessions?.sql).toContain('"desiredState"');
+ expect(sessions?.sql).toContain('$13');
+ expect(mapped({ desiredState: 'stopped' })[12]).toBe('stopped');
+ });
+
+ it('restores a row without the field, or with an unknown value, as eligible (NULL)', () => {
+ expect(mapped({})[12]).toBeNull();
+ expect(mapped({ desiredState: 'bogus' })[12]).toBeNull();
+ });
+});
diff --git a/src/modules/infra/table-importers.ts b/src/modules/infra/table-importers.ts
index 1ecc9082c..fb2e60262 100644
--- a/src/modules/infra/table-importers.ts
+++ b/src/modules/infra/table-importers.ts
@@ -62,8 +62,8 @@ export const TABLE_IMPORTERS: AnyTableImporter[] = [
defineTableImporter({
key: 'sessions',
label: 'session',
- sql: `INSERT INTO sessions (id, name, status, phone, "pushName", config, "proxyUrl", "proxyType", "connectedAt", "lastActiveAt", "createdAt", "updatedAt")
- VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12)`,
+ sql: `INSERT INTO sessions (id, name, status, phone, "pushName", config, "proxyUrl", "proxyType", "connectedAt", "lastActiveAt", "createdAt", "updatedAt", "desiredState")
+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13)`,
id: (session: SessionRow) => session.id,
// Both columns reach an auth-directory path: the id keys the directory itself, and the name is
// still weighed against it (the boot migration and the legacy purge on delete). An unvalidated
@@ -90,6 +90,9 @@ export const TABLE_IMPORTERS: AnyTableImporter[] = [
session.lastActiveAt,
session.createdAt,
session.updatedAt,
+ // 'stopped' is the only value the column holds. A backup written before the column existed,
+ // or an unknown value, restores as NULL: eligible for auto-start and takeover, as before.
+ session.desiredState === 'stopped' ? 'stopped' : null,
],
}),
diff --git a/src/modules/session/entities/session.entity.ts b/src/modules/session/entities/session.entity.ts
index f01f8a62a..ef65c3b82 100644
--- a/src/modules/session/entities/session.entity.ts
+++ b/src/modules/session/entities/session.entity.ts
@@ -80,6 +80,16 @@ export class Session {
@Column({ type: dateColumnType(), nullable: true, transformer: DateTransformer })
leaseExpiresAt!: Date | null;
+ /**
+ * Operator intent, server-owned and not part of any session response (only the export-data
+ * backup carries it): 'stopped' once POST /stop or POST /force-kill took the session down,
+ * cleared again by an explicit POST /start. Boot auto-start and the takeover sweep skip a stopped
+ * row, so a deliberate stop survives a restart. NULL (every row that predates the column) means
+ * eligible, exactly as before.
+ */
+ @Column({ type: 'varchar', length: 20, nullable: true })
+ desiredState!: 'stopped' | null;
+
@CreateDateColumn()
createdAt!: Date;
diff --git a/src/modules/session/logout-teardown-race.spec.ts b/src/modules/session/logout-teardown-race.spec.ts
index f1cb4de31..9405db0dd 100644
--- a/src/modules/session/logout-teardown-race.spec.ts
+++ b/src/modules/session/logout-teardown-race.spec.ts
@@ -39,6 +39,7 @@ function createMockSession(overrides: Partial<Session> = {}): Session {
nodeId: null,
claimedAt: null,
leaseExpiresAt: null,
+ desiredState: null,
nodeUrl: null,
createdAt: new Date(),
updatedAt: new Date(),
diff --git a/src/modules/session/session-engine-controls.ts b/src/modules/session/session-engine-controls.ts
index 55979ede2..6e258c874 100644
--- a/src/modules/session/session-engine-controls.ts
+++ b/src/modules/session/session-engine-controls.ts
@@ -1,4 +1,10 @@
-import { BadGatewayException, BadRequestException, HttpStatus, NotFoundException } from '@nestjs/common';
+import {
+ BadGatewayException,
+ BadRequestException,
+ ConflictException,
+ HttpStatus,
+ NotFoundException,
+} from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { Repository, DataSource } from 'typeorm';
import { Session, SessionStatus } from './entities/session.entity';
@@ -32,6 +38,9 @@ import {
type ReconnectState,
} from './session-engine-lifecycle.service';
+/** A start refused because an operator stopped the session. Same 409 on the wire as any conflict. */
+export class SessionStoppedException extends ConflictException {}
+
/**
* The deps + core call-ins SessionEngineControls needs from the lifecycle. Built ONCE in the
* lifecycle's constructor. The dependency VALUES (sessionRepository … broadcaster) are captured at
@@ -148,7 +157,12 @@ export class SessionEngineControls {
}
}
- async start(id: string): Promise<Session> {
+ /**
+ * `explicit` marks an operator's POST /start, the only start that may clear an operator's stop
+ * (desiredState), and only a stop that was already on the row when it began. It clears it only
+ * once past every refusal below, so a refused start leaves the stop exactly as it found it.
+ */
+ async start(id: string, { explicit = false }: { explicit?: boolean } = {}): Promise<Session> {
// Reserve the slot SYNCHRONOUSLY at entry — before even the requireSession await. Two
// near-simultaneous start() calls must not both pass the check and orphan an engine (the has()
// -> engines.set() window spans the awaited hook below), and the infra import pre-flight
@@ -164,6 +178,13 @@ export class SessionEngineControls {
try {
const session = await this.requireSession(id);
+ // An operator's stop outranks any start but POST /start, so this refuses boot auto-start and
+ // the takeover sweep, which pick their rows once and launch them seconds apart, when a stop
+ // landed after that read.
+ if (session.desiredState === 'stopped' && !explicit) {
+ throw new SessionStoppedException(`Session ${id} was stopped`);
+ }
+
if (this.engines.has(id)) {
throw new BadRequestException('Session is already started');
}
@@ -203,6 +224,20 @@ export class SessionEngineControls {
// process). Idempotent: a no-op when no reconnect state exists (the common fresh-start case).
this.host.cancelReconnect(id);
+ // The operator's stop, read again past every refusal and after the mark is cleared. A stop that
+ // finished while this start waited above had no engine to take down and just lost its mark,
+ // but its record is on the row: only a POST /start sent to an already stopped session clears
+ // it, and any other start yields to it with the mark put back. A stop from here on sets the
+ // mark again and retires this start after its init. A launch that fails after the clear still
+ // counts as the start the operator asked for.
+ if ((await this.requireSession(id)).desiredState === 'stopped') {
+ if (!explicit || session.desiredState !== 'stopped') {
+ this.stoppingSessions.add(id);
+ throw new SessionStoppedException(`Session ${id} was stopped`);
+ }
+ await this.sessionRepository.update({ id, desiredState: 'stopped' }, { desiredState: null });
+ }
+
// Execute hook before starting
await this.hookManager.execute(
'session:starting',
@@ -478,6 +513,20 @@ export class SessionEngineControls {
this.stoppingSessions.add(id);
this.host.cancelReconnect(id);
+ // A force-kill is a stop too: the session stays down across restarts and takeover until an
+ // explicit start. Recorded only once there is an engine to kill, and before the teardown, so a
+ // POST /start that lands after the eviction clears it and stands. A failed write must not keep a
+ // wedged engine alive, so it is logged and the kill goes on.
+ await this.sessionRepository
+ .update(id, { desiredState: 'stopped' })
+ .catch((error: unknown) =>
+ this.logger.error(
+ 'Failed to record the force-kill as a stop',
+ error instanceof Error ? error.message : String(error),
+ { sessionId: id, action: 'force_kill_keep_down_failed' },
+ ),
+ );
+
// Announced by the write below rather than from the engine callback, as in stop().
this.operatorTeardowns.set(id, engine);
try {
diff --git a/src/modules/session/session-engine-lifecycle.service.ts b/src/modules/session/session-engine-lifecycle.service.ts
index fe35592b7..4a08677be 100644
--- a/src/modules/session/session-engine-lifecycle.service.ts
+++ b/src/modules/session/session-engine-lifecycle.service.ts
@@ -420,8 +420,8 @@ export class SessionEngineLifecycle {
// settlement profile the inline methods had (the Task-1 delegate rule above).
/** Delegate: SessionEngineControls.start. */
- start(id: string): Promise<Session> {
- return this.controls.start(id);
+ start(id: string, options?: { explicit?: boolean }): Promise<Session> {
+ return this.controls.start(id, options);
}
/** Delegate: SessionEngineControls.stop. */
diff --git a/src/modules/session/session.controller.spec.ts b/src/modules/session/session.controller.spec.ts
index 2ef5f2068..b9cf15783 100644
--- a/src/modules/session/session.controller.spec.ts
+++ b/src/modules/session/session.controller.spec.ts
@@ -30,6 +30,7 @@ describe('SessionController — create() response contract', () => {
claimedAt: null,
nodeUrl: null,
leaseExpiresAt: null,
+ desiredState: null,
createdAt: new Date('2026-01-01T00:00:00Z'),
updatedAt: new Date('2026-01-01T00:00:00Z'),
};
@@ -120,6 +121,7 @@ describe('SessionController — logout() audit + error forwarding contract', ()
claimedAt: null,
nodeUrl: null,
leaseExpiresAt: null,
+ desiredState: null,
createdAt: new Date('2026-01-01T00:00:00Z'),
updatedAt: new Date('2026-01-01T00:00:00Z'),
};
@@ -185,6 +187,7 @@ describe('SessionController — start/stop lifecycle', () => {
claimedAt: null,
nodeUrl: null,
leaseExpiresAt: null,
+ desiredState: null,
createdAt: new Date('2026-01-01T00:00:00Z'),
updatedAt: new Date('2026-01-01T01:00:00Z'),
};
@@ -219,6 +222,14 @@ describe('SessionController — start/stop lifecycle', () => {
expect(sessionService.engineLoaded).toHaveBeenCalledWith(expect.objectContaining({ id: 'sess-uuid-1' }));
});
+ it('start is an explicit start, which clears an operator stop', async () => {
+ sessionService.start.mockResolvedValue({ ...runningEntity });
+
+ await controller.start('sess-uuid-1');
+
+ expect(sessionService.start).toHaveBeenCalledWith('sess-uuid-1', { explicit: true });
+ });
+
it('start audits SESSION_STARTED once the service has resolved', async () => {
sessionService.start.mockResolvedValue({ ...runningEntity });
diff --git a/src/modules/session/session.controller.ts b/src/modules/session/session.controller.ts
index 71ca4d975..d35aaa16a 100644
--- a/src/modules/session/session.controller.ts
+++ b/src/modules/session/session.controller.ts
@@ -292,10 +292,12 @@ export class SessionController {
'SESSION_NAME_TEARDOWN_PENDING`; wait for it to settle and retry. No destructive side ' +
'effect runs before this refusal. Also returned when another node currently holds this ' +
"session's engine: only the owner may start it, and the claim is refused before any engine " +
- 'is launched, so no second connection to the account is opened.',
+ 'is launched, so no second connection to the account is opened. Also returned, with no `code`, ' +
+ 'when a stop or force-kill of this session finished while the start was waiting: the stop ' +
+ 'stands, and a new POST /start clears it and starts the session.',
})
async start(@Param('sessionId', ParseUUIDPipe) id: string): Promise<SessionResponseDto> {
- const session = await this.sessionService.start(id);
+ const session = await this.sessionService.start(id, { explicit: true });
await this.auditService.logInfo(AuditAction.SESSION_STARTED, {
sessionId: session.id,
sessionName: session.name,
diff --git a/src/modules/session/session.service.spec.ts b/src/modules/session/session.service.spec.ts
index 42b5ac8a0..2937e3e02 100644
--- a/src/modules/session/session.service.spec.ts
+++ b/src/modules/session/session.service.spec.ts
@@ -3,7 +3,7 @@ import { SessionRestrictionStore } from './session-restriction-store.service';
import { PresenceStore } from './presence-store.service';
import { Test, TestingModule } from '@nestjs/testing';
import { getRepositoryToken, getDataSourceToken } from '@nestjs/typeorm';
-import { Repository, DataSource, In, QueryFailedError } from 'typeorm';
+import { Repository, DataSource, In, IsNull, Not, QueryFailedError } from 'typeorm';
import {
NotFoundException,
ConflictException,
@@ -18,6 +18,7 @@ import { EngineNotSupportedError } from '../../common/errors/engine-not-supporte
import { ConfigService } from '@nestjs/config';
import { SessionService, AUTOSTART_THROTTLE_MS } from './session.service';
import { SessionOwnershipService } from './session-ownership.service';
+import { SessionStoppedException } from './session-engine-controls';
import { decideReconnect, STABLE_READY_MS } from './reconnect-policy';
import { ACK_RECONCILE_DELAY_MS } from './message-projector.service';
import { SessionEngineLifecycle, type ReconnectState } from './session-engine-lifecycle.service';
@@ -93,6 +94,7 @@ function createMockSession(overrides: Partial<Session> = {}): Session {
nodeId: null,
claimedAt: null,
leaseExpiresAt: null,
+ desiredState: null,
nodeUrl: null,
createdAt: new Date(),
updatedAt: new Date(),
@@ -7897,6 +7899,387 @@ describe('SessionService', () => {
});
});
+ // ── operator stop intent ──────────────────────────────────────────
+ describe('operator stop intent (desiredState)', () => {
+ const STOPPED = { desiredState: 'stopped' };
+ const updates = (): unknown[][] => (repository.update as jest.Mock).mock.calls as unknown[][];
+ const withOwnership = (over: Record<string, jest.Mock> = {}) => {
+ const ownership = {
+ claim: jest.fn().mockResolvedValue(true),
+ release: jest.fn().mockResolvedValue(undefined),
+ isHeldByOtherNode: jest.fn().mockResolvedValue(false),
+ ...over,
+ };
+ Object.assign(service as unknown as Record<string, unknown>, { ownership });
+ return ownership;
+ };
+
+ beforeEach(() => {
+ (repository.update as jest.Mock).mockResolvedValue({ affected: 1 });
+ jest.spyOn(lifecycle, 'isEngineActive').mockReturnValue(false);
+ });
+
+ it('stop() records the stop before tearing down', async () => {
+ const stop = jest.spyOn(lifecycle, 'stop').mockResolvedValue(createMockSession());
+
+ await service.stop('sess-uuid-1');
+
+ expect(repository.update).toHaveBeenCalledWith('sess-uuid-1', STOPPED);
+ const writeOrder = (repository.update as jest.Mock).mock.invocationCallOrder[0];
+ expect(writeOrder).toBeLessThan(stop.mock.invocationCallOrder[0]);
+ });
+
+ it('stop() keeps the record on the 502 SESSION_STOP_INCOMPLETE path', async () => {
+ jest.spyOn(lifecycle, 'stop').mockRejectedValue(new BadGatewayException({ code: 'SESSION_STOP_INCOMPLETE' }));
+
+ await expect(service.stop('sess-uuid-1')).rejects.toBeInstanceOf(BadGatewayException);
+
+ expect(repository.update).toHaveBeenCalledWith('sess-uuid-1', STOPPED);
+ });
+
+ it('stop() refused because a live peer holds the session records nothing', async () => {
+ withOwnership({ isHeldByOtherNode: jest.fn().mockResolvedValue(true) });
+
+ await expect(service.stop('sess-uuid-1')).rejects.toBeInstanceOf(ConflictException);
+
+ expect(updates()).not.toContainEqual(['sess-uuid-1', STOPPED]);
+ });
+
+ it('stop() whose record write fails takes nothing down and leaves no stop mark behind', async () => {
+ (repository.update as jest.Mock).mockRejectedValueOnce(new Error('db down'));
+ const stop = jest.spyOn(lifecycle, 'stop');
+
+ await expect(service.stop('sess-uuid-1')).rejects.toThrow('db down');
+
+ expect(stop).not.toHaveBeenCalled();
+ expect((lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions.has('sess-uuid-1')).toBe(
+ false,
+ );
+ });
+
+ it('forceKill() records the stop before the teardown; a "not started" refusal does not', async () => {
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());
+ await expect(service.forceKill('sess-uuid-1')).rejects.toBeInstanceOf(BadRequestException);
+ expect(updates()).not.toContainEqual(['sess-uuid-1', STOPPED]);
+
+ await service.start('sess-uuid-1');
+ await service.forceKill('sess-uuid-1');
+
+ const at = updates().findIndex(([, patch]) => (patch as { desiredState?: unknown }).desiredState === 'stopped');
+ expect(at).toBeGreaterThanOrEqual(0);
+ expect((repository.update as jest.Mock).mock.invocationCallOrder[at]).toBeLessThan(
+ mockEngine.forceDestroy.mock.invocationCallOrder[0],
+ );
+ });
+
+ it('forceKill() still kills, resolves and hands the claim back when the record write fails', async () => {
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());
+ await service.start('sess-uuid-1');
+ const ownership = withOwnership();
+ (repository.update as jest.Mock).mockImplementation((_id: unknown, patch: { desiredState?: unknown }) =>
+ patch.desiredState === 'stopped' ? Promise.reject(new Error('db busy')) : Promise.resolve({ affected: 1 }),
+ );
+ const logError = jest.spyOn((lifecycle as unknown as { logger: { error: jest.Mock } }).logger, 'error');
+
+ await expect(service.forceKill('sess-uuid-1')).resolves.toBeDefined();
+
+ expect(mockEngine.forceDestroy).toHaveBeenCalled();
+ expect(ownership.release).toHaveBeenCalledWith('sess-uuid-1');
+ expect(logError).toHaveBeenCalledWith(
+ expect.any(String),
+ 'db busy',
+ expect.objectContaining({ action: 'force_kill_keep_down_failed' }),
+ );
+ });
+
+ it('only an explicit start clears the record, after a successful claim and before the launch', async () => {
+ const ownership = withOwnership();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ desiredState: 'stopped' }));
+ const clear = JSON.stringify([{ id: 'sess-uuid-1', desiredState: 'stopped' }, { desiredState: null }]);
+ const clearAt = () => updates().findIndex(call => JSON.stringify(call) === clear);
+
+ await expect(service.start('sess-uuid-1')).rejects.toBeInstanceOf(SessionStoppedException);
+ expect(clearAt()).toBe(-1);
+
+ await service.start('sess-uuid-1', { explicit: true });
+ const cleared = (repository.update as jest.Mock).mock.invocationCallOrder[clearAt()];
+ expect(ownership.claim.mock.invocationCallOrder[1]).toBeLessThan(cleared);
+ expect(cleared).toBeLessThan((engineFactory.create as jest.Mock).mock.invocationCallOrder[0]);
+ });
+
+ // Auto-start and takeover read their candidate rows once and launch them seconds apart, so a
+ // stop can land between that read and this row's turn.
+ it('a non-explicit start refuses a row stopped after the caller read it, and hands the claim back', async () => {
+ const ownership = withOwnership();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ desiredState: 'stopped' }));
+
+ await expect(service.start('sess-uuid-1')).rejects.toBeInstanceOf(SessionStoppedException);
+
+ expect(engineFactory.create).not.toHaveBeenCalled();
+ expect(ownership.release).toHaveBeenCalledWith('sess-uuid-1');
+ });
+
+ it('an explicit start refused by a live peer clears nothing', async () => {
+ withOwnership({ claim: jest.fn().mockResolvedValue(false) });
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());
+
+ await expect(service.start('sess-uuid-1', { explicit: true })).rejects.toBeInstanceOf(ConflictException);
+
+ expect(repository.update).not.toHaveBeenCalled();
+ });
+
+ it('boot auto-start only selects rows nobody stopped', async () => {
+ process.env.AUTO_START_SESSIONS = 'true';
+ (repository.find as jest.Mock).mockResolvedValue([]);
+ try {
+ withOwnership({ claimableWhere: jest.fn().mockReturnValue([{ nodeId: IsNull() }, { nodeId: 'node-a' }]) });
+ Object.assign((service as unknown as { ownership: object }).ownership, {
+ onLeaseLoss: jest.fn(),
+ setEngineLiveness: jest.fn(),
+ startHeartbeat: jest.fn(),
+ stopHeartbeat: jest.fn(),
+ releaseAll: jest.fn().mockResolvedValue(undefined),
+ });
+ service.onApplicationBootstrap();
+ await (service as unknown as { autoStartRun: Promise<void> }).autoStartRun;
+
+ const [[{ where }]] = (repository.find as jest.Mock).mock.calls as [[{ where: Record<string, unknown>[] }]];
+ expect(where).toHaveLength(2);
+ for (const clause of where) {
+ expect(clause).toMatchObject({
+ phone: Not(IsNull()),
+ status: SessionStatus.DISCONNECTED,
+ desiredState: IsNull(),
+ });
+ }
+ } finally {
+ delete process.env.AUTO_START_SESSIONS;
+ await service.onModuleDestroy();
+ }
+ });
+
+ describe('against a real database', () => {
+ let db: DataSource;
+ let sessions: Repository<Session>;
+
+ beforeAll(async () => {
+ db = new DataSource({ type: 'better-sqlite3', database: ':memory:', entities: [Session], synchronize: true });
+ await db.initialize();
+ sessions = db.getRepository(Session);
+ });
+
+ afterAll(async () => {
+ await db.destroy();
+ });
+
+ it('a stopped session is not auto-started on the next boot until an explicit start', async () => {
+ const linked = { status: SessionStatus.DISCONNECTED, phone: '628111', config: {} };
+ const stopped = await sessions.save(sessions.create({ name: 'stopped', ...linked }));
+ const crashed = await sessions.save(sessions.create({ name: 'crashed', ...linked }));
+ (repository.find as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['find']>[0]) =>
+ sessions.find(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ jest.spyOn(lifecycle, 'stop').mockResolvedValue(createMockSession());
+ const engineStart = jest.spyOn(lifecycle, 'start').mockResolvedValue(createMockSession());
+
+ await service.stop(stopped.id);
+
+ process.env.AUTO_START_SESSIONS = 'true';
+ const startSpy = jest.spyOn(service, 'start');
+ try {
+ service.onApplicationBootstrap();
+ await (service as unknown as { autoStartRun: Promise<void> }).autoStartRun;
+ expect(startSpy.mock.calls.map(([id]) => id)).toEqual([crashed.id]);
+
+ engineStart.mockRestore();
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ await service.start(stopped.id, { explicit: true });
+ expect((await sessions.findOneByOrFail({ id: stopped.id })).desiredState).toBeNull();
+ } finally {
+ delete process.env.AUTO_START_SESSIONS;
+ await service.onModuleDestroy();
+ }
+ });
+
+ // A refusal (another start holds the reservation, the credential-teardown fence) means the
+ // start did not happen, so the operator's stop must survive it. A launch that fails midway
+ // still counts as the start the operator asked for.
+ it('an explicit start refused before launching keeps the stop; one that failed launching clears it', async () => {
+ const stopped = await sessions.save(
+ sessions.create({
+ name: 'refused',
+ status: SessionStatus.DISCONNECTED,
+ phone: '628222',
+ config: {},
+ desiredState: 'stopped',
+ }),
+ );
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ const desired = async () => (await sessions.findOneByOrFail({ id: stopped.id })).desiredState;
+
+ const reserved = (lifecycle as unknown as { initializingSessions: Set<string> }).initializingSessions;
+ reserved.add(stopped.id);
+ await expect(service.start(stopped.id, { explicit: true })).rejects.toThrow('Session is already starting');
+ reserved.delete(stopped.id);
+ expect(await desired()).toBe('stopped');
+
+ const fences = (lifecycle as unknown as { fences: { awaitPendingTeardown(name: string): Promise<void> } })
+ .fences;
+ const fenced = new ConflictException('Session credentials are still being removed; retry shortly');
+ jest.spyOn(fences, 'awaitPendingTeardown').mockRejectedValueOnce(fenced);
+ await expect(service.start(stopped.id, { explicit: true })).rejects.toBe(fenced);
+ expect(await desired()).toBe('stopped');
+
+ mockEngine.initialize.mockRejectedValueOnce(new GatewayTimeoutException('authentication timed out'));
+ await expect(service.start(stopped.id, { explicit: true })).rejects.toBeInstanceOf(GatewayTimeoutException);
+ expect(await desired()).toBeNull();
+ });
+
+ // A stop keeps the engine registered until its teardown ends, so a start sent meanwhile is
+ // refused as "already started" by the engine that is going down, not by one it launched.
+ it('an explicit start refused while a stop tears the engine down keeps the stop', async () => {
+ const running = await sessions.save(
+ sessions.create({ name: 'stopping', status: SessionStatus.DISCONNECTED, phone: '628333', config: {} }),
+ );
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ await service.start(running.id);
+ let refused: unknown;
+ mockEngine.disconnect.mockImplementationOnce(async () => {
+ refused = await service.start(running.id, { explicit: true }).catch((error: unknown) => error);
+ });
+
+ await service.stop(running.id);
+
+ expect(refused).toBeInstanceOf(BadRequestException);
+ expect((refused as Error).message).toBe('Session is already started');
+ expect((await sessions.findOneByOrFail({ id: running.id })).desiredState).toBe('stopped');
+ });
+
+ // A stop that finishes while a start waits at the credential-teardown fence finds no engine to
+ // take down, and the start drops its mark once past the fence. The start must yield to it.
+ it('a start yields to a stop that finished while it waited at the teardown fence', async () => {
+ const idle = await sessions.save(
+ sessions.create({ name: 'fenced', status: SessionStatus.DISCONNECTED, phone: '628666', config: {} }),
+ );
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ const fences = (lifecycle as unknown as { fences: { awaitPendingTeardown(name: string): Promise<void> } })
+ .fences;
+ const stopping = (lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions;
+
+ for (const explicit of [false, true]) {
+ await sessions.update(idle.id, { desiredState: null });
+ let waiting!: () => void;
+ let settle!: () => void;
+ const atFence = new Promise<void>(resolve => (waiting = resolve));
+ jest.spyOn(fences, 'awaitPendingTeardown').mockImplementationOnce(() => {
+ waiting();
+ return new Promise<void>(resolve => (settle = resolve));
+ });
+
+ const start = service.start(idle.id, { explicit });
+ await atFence;
+ await service.stop(idle.id);
+ settle();
+
+ await expect(start).rejects.toBeInstanceOf(SessionStoppedException);
+ expect(engineFactory.create).not.toHaveBeenCalled();
+ expect(stopping.has(idle.id)).toBe(true);
+ expect((await sessions.findOneByOrFail({ id: idle.id })).desiredState).toBe('stopped');
+ }
+ });
+
+ // The start that holds the reservation already cleared the stop; the refused one must not
+ // bring it back over the session that start is launching.
+ it('an explicit start refused by another explicit start leaves the stop cleared', async () => {
+ const stopped = await sessions.save(
+ sessions.create({
+ name: 'racing',
+ status: SessionStatus.DISCONNECTED,
+ phone: '628555',
+ config: {},
+ desiredState: 'stopped',
+ }),
+ );
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ const desired = async () => (await sessions.findOneByOrFail({ id: stopped.id })).desiredState;
+ let reached!: () => void;
+ let finish!: () => void;
+ const launching = new Promise<void>(resolve => (reached = resolve));
+ mockEngine.initialize.mockImplementationOnce(() => {
+ reached();
+ return new Promise<void>(resolve => (finish = resolve));
+ });
+
+ const first = service.start(stopped.id, { explicit: true });
+ await launching;
+ expect(await desired()).toBeNull();
+
+ await expect(service.start(stopped.id, { explicit: true })).rejects.toThrow('Session is already starting');
+ expect(await desired()).toBeNull();
+
+ finish();
+ await first;
+ expect(await desired()).toBeNull();
+ });
+
+ // The kill records its stop before the teardown, so a POST /start that lands after the eviction
+ // (here, while the kill writes DISCONNECTED) clears it and the new engine stays marked to run.
+ it('a force-kill leaves an explicit start sent after the eviction marked to run', async () => {
+ const running = await sessions.save(
+ sessions.create({ name: 'killed', status: SessionStatus.DISCONNECTED, phone: '628444', config: {} }),
+ );
+ (repository.findOne as jest.Mock).mockImplementation((opts: Parameters<Repository<Session>['findOne']>[0]) =>
+ sessions.findOne(opts),
+ );
+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>
+ sessions.update(...args),
+ );
+ const desired = async () => (await sessions.findOneByOrFail({ id: running.id })).desiredState;
+ await service.start(running.id);
+ const updateStatus = jest.spyOn(
+ lifecycle as unknown as { updateStatus(id: string, status: SessionStatus): Promise<void> },
+ 'updateStatus',
+ );
+ updateStatus.mockImplementationOnce(async id => {
+ await service.start(id, { explicit: true });
+ });
+
+ await service.forceKill(running.id);
+ expect(engineFactory.create).toHaveBeenCalledTimes(2);
+ expect(await desired()).toBeNull();
+
+ // A kill with nothing launched after it records the stop.
+ await service.forceKill(running.id);
+ expect(await desired()).toBe('stopped');
+ });
+ });
+ });
+
// ── onApplicationBootstrap (auto-start) ───────────────────────────
describe('onApplicationBootstrap', () => {
// The launch loop is detached from the hook so the HTTP listener binds without waiting for
@@ -7981,6 +8364,24 @@ describe('SessionService', () => {
}
});
+ it('reports a session stopped after the scan as skipped, not as a failed start', async () => {
+ process.env.AUTO_START_SESSIONS = 'true';
+ (repository.find as jest.Mock).mockResolvedValue([{ id: 'a', name: 'A' }]);
+ jest.spyOn(service, 'start').mockRejectedValueOnce(new SessionStoppedException('Session a was stopped'));
+ const logger = (service as unknown as { logger: { log: jest.Mock; error: jest.Mock } }).logger;
+ const logError = jest.spyOn(logger, 'error');
+ const logInfo = jest.spyOn(logger, 'log');
+
+ service.onApplicationBootstrap();
+ await autoStartRun();
+
+ expect(logError).not.toHaveBeenCalled();
+ expect(logInfo).toHaveBeenCalledWith(
+ 'Auto-start skipped for session A: stopped by an operator',
+ expect.objectContaining({ action: 'auto_start_skipped' }),
+ );
+ });
+
// Nest binds the HTTP listener only after every bootstrap hook settles, and a launch is a
// Chromium start bounded by resolveEngineInitTimeoutMs (>= 60s) with a 2s throttle between
// sessions. Awaiting the loop here kept the port CLOSED for that whole time, so every liveness
diff --git a/src/modules/session/session.service.ts b/src/modules/session/session.service.ts
index c601b33cf..7d121cf24 100644
--- a/src/modules/session/session.service.ts
+++ b/src/modules/session/session.service.ts
@@ -39,6 +39,7 @@ import { resolveFeatureFlags } from '../../config/feature-flags';
import { IWhatsAppEngine, ChatSummary, ChatState } from '../../engine/interfaces/whatsapp-engine.interface';
import { createLogger } from '../../common/services/logger.service';
import { HookManager } from '../../core/hooks';
+import { SessionStoppedException } from './session-engine-controls';
// Type-only: the module binds this class to PLUGIN_SESSION_PORT with a `useExisting` alias, which
// TypeScript does not check, so `implements` is what keeps the two in step.
import type { PluginSessionPort } from '../../core/plugins/plugin-host-ports';
@@ -233,9 +234,15 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
// Restricted to sessions this node may claim. Without it every replica scans the same rows and
// races to launch the same engines, which is a WhatsApp account being opened twice, not merely
// duplicated work.
+ // A session an operator stopped (desiredState 'stopped') stays down until an explicit start.
const claimable = this.ownership?.claimableWhere() ?? [{}];
const sessions = await this.sessionRepository.find({
- where: claimable.map(clause => ({ ...clause, phone: Not(IsNull()), status: SessionStatus.DISCONNECTED })),
+ where: claimable.map(clause => ({
+ ...clause,
+ phone: Not(IsNull()),
+ status: SessionStatus.DISCONNECTED,
+ desiredState: IsNull(),
+ })),
});
if (sessions.length === 0) return;
@@ -262,11 +269,19 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
action: 'auto_start_success',
});
} catch (error: unknown) {
- const errorMessage = error instanceof Error ? error.message : 'Unknown error';
- this.logger.error(`Auto-start failed for session: ${session.name}`, errorMessage, {
- sessionId: session.id,
- action: 'auto_start_failed',
- });
+ if (error instanceof SessionStoppedException) {
+ // Stopped after the scan read it; the start refused it, as it should.
+ this.logger.log(`Auto-start skipped for session ${session.name}: stopped by an operator`, {
+ sessionId: session.id,
+ action: 'auto_start_skipped',
+ });
+ } else {
+ const errorMessage = error instanceof Error ? error.message : 'Unknown error';
+ this.logger.error(`Auto-start failed for session: ${session.name}`, errorMessage, {
+ sessionId: session.id,
+ action: 'auto_start_failed',
+ });
+ }
}
// Throttle between sequential Chromium launches; no need to wait after the last one.
if (i < sessions.length - 1) {
@@ -533,7 +548,12 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
}
}
- async start(id: string): Promise<Session> {
+ /**
+ * `explicit` marks an operator's POST /start: only that clears a stop (desiredState), and only once
+ * the engine start is past its refusals. Boot auto-start and the takeover sweep never clear it, and
+ * the engine start refuses them a row still marked stopped when it reads it.
+ */
+ async start(id: string, { explicit = false }: { explicit?: boolean } = {}): Promise<Session> {
// Claimed before the engine is launched, never after: launching first and discovering the
// session belongs elsewhere would already have opened a second connection to the account.
if (this.ownership && !(await this.ownership.claim(id))) {
@@ -544,7 +564,7 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
}
let session: Session;
try {
- session = await this.startWithTransientRetry(id);
+ session = await this.startWithTransientRetry(id, explicit);
} catch (error) {
// A failed or refused start must not leave the claim pinned here — the heartbeat would renew
// it and the session could never be started anywhere else. Released only when nothing is
@@ -572,10 +592,10 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
* unbounded loop here would hold the concurrency slot hostage. HTTP-shaped refusals (409
* not-ready, 4xx) are NOT transient - they propagate immediately.
*/
- private async startWithTransientRetry(id: string): Promise<Session> {
+ private async startWithTransientRetry(id: string, explicit: boolean): Promise<Session> {
const stopRequestsBefore = this.stopRequests.get(id);
try {
- return await this.engineLifecycle.start(id);
+ return await this.engineLifecycle.start(id, { explicit });
} catch (error) {
if (!isTransientLaunchFailure(error)) throw error;
this.logger.warn(`Transient launch failure for session ${id}; retrying once`, {
@@ -594,7 +614,7 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
await this.findOne(id);
throw new ConflictException(`Session ${id} is running on another node`);
}
- return this.engineLifecycle.start(id);
+ return this.engineLifecycle.start(id, { explicit });
}
}
@@ -604,6 +624,14 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
let session: Session;
try {
if (this.ownership) await this.assertNotHeldElsewhere(id);
+ // Recorded BEFORE the teardown, so the 502 SESSION_STOP_INCOMPLETE path keeps it too: a
+ // stopped session stays down across restarts and takeover until an explicit start. A failed
+ // write took nothing down, so the mark and count are undone as for a failed ownership read.
+ await this.keepDown(id).catch((error: unknown) => {
+ this.engineLifecycle.clearStopping(id);
+ this.uncountStopRequest(id);
+ throw error;
+ });
session = await this.engineLifecycle.stop(id);
} catch (error) {
// Only the local 502 (SESSION_STOP_INCOMPLETE) releases: it evicted the engine and wrote
@@ -642,16 +670,22 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
async forceKill(id: string): Promise<Session> {
try {
+ // The engine kill records the stop itself, once it has an engine to kill.
const session = await this.engineLifecycle.forceKill(id);
await this.releaseUnlessEngineActive(id);
return session;
} catch (error) {
- // Same 400 rule as logout().
+ // Same 400 rule as logout(): a "not started" refusal took nothing down.
if (!(error instanceof BadRequestException)) await this.releaseUnlessEngineActive(id);
throw error;
}
}
+ /** Persist the operator's stop so boot auto-start and the takeover sweep leave the session down. */
+ private async keepDown(id: string): Promise<void> {
+ await this.sessionRepository.update(id, { desiredState: 'stopped' });
+ }
+
private markStopping(id: string): void {
this.stopRequests.set(id, (this.stopRequests.get(id) ?? 0) + 1);
this.engineLifecycle.markStopping(id);
diff --git a/src/modules/takeover/session-takeover.service.spec.ts b/src/modules/takeover/session-takeover.service.spec.ts
index 14541c871..d90054373 100644
--- a/src/modules/takeover/session-takeover.service.spec.ts
+++ b/src/modules/takeover/session-takeover.service.spec.ts
@@ -1,5 +1,6 @@
import { ConflictException } from '@nestjs/common';
import { SessionTakeoverService } from './session-takeover.service';
+import { SessionStoppedException } from '../session/session-engine-controls';
import { Session, SessionStatus } from '../session/entities/session.entity';
import type { SessionService } from '../session/session.service';
import type { SessionOwnershipService } from '../session/session-ownership.service';
@@ -122,6 +123,17 @@ describe('SessionTakeoverService', () => {
expect(start).not.toHaveBeenCalled();
});
+ it('leaves a session an operator stopped down, and adopts one nobody stopped', async () => {
+ const { svc, start } = build([
+ lapsed({ name: 'stopped', status: SessionStatus.DISCONNECTED, desiredState: 'stopped' }),
+ lapsed({ name: 'dropped', status: SessionStatus.DISCONNECTED, desiredState: null }),
+ ]);
+
+ await svc.sweep();
+
+ expect(start.mock.calls).toEqual([['id-dropped']]);
+ });
+
it('a lost claim race is a non-event: no batch reconcile, and the next candidate still starts', async () => {
jest.useFakeTimers();
const start = jest.fn().mockRejectedValueOnce(new ConflictException('held elsewhere')).mockResolvedValueOnce({});
@@ -136,6 +148,19 @@ describe('SessionTakeoverService', () => {
expect(reap).toHaveBeenCalledWith('id-ours', expect.any(String));
});
+ it('a session stopped after the sweep read it is logged as skipped, not as a lost claim race', async () => {
+ const start = jest.fn().mockRejectedValueOnce(new SessionStoppedException('Session id-halted was stopped'));
+ const { svc, reap } = build([lapsed({ name: 'halted' })], { startImpl: start });
+ const debug = jest.spyOn((svc as unknown as { logger: { debug: jest.Mock } }).logger, 'debug');
+
+ await svc.sweep();
+
+ expect(reap).not.toHaveBeenCalled();
+ expect(debug.mock.calls.map(([message]) => message as string)).toEqual([
+ 'Session halted skipped: stopped by an operator',
+ ]);
+ });
+
it('a non-conflict start failure is logged and does not abort the rest of the sweep', async () => {
jest.useFakeTimers();
const start = jest.fn().mockRejectedValueOnce(new Error('chromium died')).mockResolvedValueOnce({});
diff --git a/src/modules/takeover/session-takeover.service.ts b/src/modules/takeover/session-takeover.service.ts
index f392adfbf..9f339d8ed 100644
--- a/src/modules/takeover/session-takeover.service.ts
+++ b/src/modules/takeover/session-takeover.service.ts
@@ -7,6 +7,7 @@ import { Session, SessionStatus } from '../session/entities/session.entity';
import { SessionOwnershipService } from '../session/session-ownership.service';
import { ShutdownService } from '../../common/services/shutdown.service';
import { SessionService } from '../session/session.service';
+import { SessionStoppedException } from '../session/session-engine-controls';
import { BulkMessageService } from '../message/bulk-message.service';
/**
@@ -172,7 +173,10 @@ export class SessionTakeoverService implements OnApplicationBootstrap, OnModuleD
// than leaving them stuck in PROCESSING until some node happens to reboot.
await this.bulkMessages.reapProcessingBatches(session.id, 'session adopted from a lapsed node');
} catch (error) {
- if (error instanceof ConflictException) {
+ if (error instanceof SessionStoppedException) {
+ // Stopped between the sweep's read and this start; the start refused it, as it should.
+ this.logger.debug(`Session ${session.name} skipped: stopped by an operator`, { sessionId: session.id });
+ } else if (error instanceof ConflictException) {
// A peer won the race — exactly the claim doing its job.
this.logger.debug(`Session ${session.name} was adopted by another node first`, { sessionId: session.id });
} else {
@@ -190,7 +194,8 @@ export class SessionTakeoverService implements OnApplicationBootstrap, OnModuleD
private isEligible(session: Session): boolean {
// Only authenticated sessions (phone set): an engine is worth relaunching exactly when the
- // saved credentials can restore the link without a human scanning anything.
- return Boolean(session.phone) && TAKEOVER_STATUSES.has(session.status);
+ // saved credentials can restore the link without a human scanning anything. A session an
+ // operator stopped stays down until an explicit start, wherever its claim lapsed.
+ return Boolean(session.phone) && TAKEOVER_STATUSES.has(session.status) && session.desiredState !== 'stopped';
}
}