Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(session): refuse engine starts once shutdown begins (#1795)
Yudhi Armyndharis9c236931acd4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e5ab61ad1..f3b73688f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -24,6 +24,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed+- Refuse session starts with `503` once shutdown begins, and leave a start that shutdown interrupts for the next boot instead of marking it failed.- Retire the pending webhook outbox record when a redrive delivers, so the sweep does not send the event again.- Stop dashboard multi-page loads and a pending chat mark-as-read from sending requests after logout.- Keep a new dashboard login signed in when a request sent with the previous API key fails afterwards.diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 742cbacba..982579e1a 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -666,7 +666,7 @@ Returned via `transformSession`. Status typically transitions to `initializing`A session stopped with `POST /stop` or `POST /force-kill` is marked to run again only once the start gets past the `400` and `409` refusals below; a refused start leaves the stop in place.-**Errors:** `400` session already started / already starting, or this node already holds `MAX_CONCURRENT_SESSIONS` slots, counting running and starting engines and sessions waiting to relaunch after a failed reconnect (`Maximum concurrent sessions reached (N)`, answered only after the `404` and the `409` for a session running on another node; the session is not started, so retry once a slot frees or on a node with room) · `401` · `403` · `404` not found · `409` credential teardown for the same session name still in flight (retryable; body carries `code: 'SESSION_NAME_TEARDOWN_PENDING'`; no destructive side effect runs before the refusal — a retry after cleanup settles proceeds), or another node currently holds this session's engine (refused before any engine is launched), or, with no `code`, a `POST /stop`, a `POST /force-kill` that found a running engine to kill, or a data import (`stopOrphans`) of this session that began or finished while the start was waiting, or a delete of it still running (the start yields and launches nothing; after a stop or force-kill a new `POST /start` clears it and starts the session, after a delete or an import that removed the session the retry answers `404`) · `504` the engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or, on whatsapp-web.js, an unreachable `proxyUrl`); the engine is torn down, so the start can be retried once the cause is addressed+**Errors:** `400` session already started / already starting, or this node already holds `MAX_CONCURRENT_SESSIONS` slots, counting running and starting engines and sessions waiting to relaunch after a failed reconnect (`Maximum concurrent sessions reached (N)`, answered only after the `404` and the `409` for a session running on another node; the session is not started, so retry once a slot frees or on a node with room) · `401` · `403` · `404` not found · `409` credential teardown for the same session name still in flight (retryable; body carries `code: 'SESSION_NAME_TEARDOWN_PENDING'`; no destructive side effect runs before the refusal — a retry after cleanup settles proceeds), or another node currently holds this session's engine (refused before any engine is launched), or, with no `code`, a `POST /stop`, a `POST /force-kill` that found a running engine to kill, or a data import (`stopOrphans`) of this session that began or finished while the start was waiting, or a delete of it still running (the start yields and launches nothing; after a stop or force-kill a new `POST /start` clears it and starts the session, after a delete or an import that removed the session the retry answers `404`) · `503` the gateway is shutting down, or the engine's browser or socket connection died during launch and the one automatic retry failed too; no engine is left running, so the start can be retried · `504` the engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or, on whatsapp-web.js, an unreachable `proxyUrl`); the engine is torn down, so the start can be retried once the cause is addressed#### POST /api/sessions/:sessionId/stopdiff --git a/openapi.json b/openapi.jsonindex 7de2f8a62..cee945e2e 100644--- a/openapi.json+++ b/openapi.json@@ -1278,6 +1278,16 @@}}},+ "503": {+ "description": "The gateway is shutting down, or the engine's browser or socket connection died during launch and the one automatic retry failed too; no engine is left running, so the start can be retried.",+ "content": {+ "application/json": {+ "schema": {+ "$ref": "#/components/schemas/ErrorResponse"+ }+ }+ }+ },"504": {"description": "The engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or, on whatsapp-web.js, an unreachable proxyUrl or the auth timeout); the engine is torn down and the start can be retried.","content": {diff --git a/src/modules/session/session-engine-controls.ts b/src/modules/session/session-engine-controls.tsindex 1b706b16d..7b16d6bda 100644--- a/src/modules/session/session-engine-controls.ts+++ b/src/modules/session/session-engine-controls.ts@@ -4,6 +4,7 @@ import {ConflictException,HttpStatus,NotFoundException,+ ServiceUnavailableException,} from '@nestjs/common';import { ConfigService } from '@nestjs/config';import { Repository, DataSource } from 'typeorm';@@ -42,6 +43,13 @@ import {/** A start refused because an operator stopped the session. Same 409 on the wire as any conflict. */export class SessionStoppedException extends ConflictException {}+/** A start refused because the gateway is shutting down. Retryable once it is back. */+export class ServerShuttingDownException extends ServiceUnavailableException {+ constructor() {+ super('Server is shutting down');+ }+}+/*** The stop marks. Each add() stamps the id afresh, even one already marked, so start() can tell a mark* set while it waited from one that was already there when it began.@@ -114,6 +122,8 @@ export interface SessionEngineControlsHost {updateStatus(id: string, status: SessionStatus): Promise<void>;/** Ownership gate, same contract as SessionEngineWiringHost.ownsSession. */ownsSession(id: string): boolean;+ /** True once shutdown() has begun; the lifecycle never reopens. */+ isClosed(): boolean;stoppingSessions: StopMarks;/** The engine each operator-initiated teardown is retiring; see the lifecycle field of the same name. */operatorTeardowns: Map<string, IWhatsAppEngine>;@@ -338,8 +348,10 @@ export class SessionEngineControls {// Fenced on ownership like the engine callbacks: initializeEngine can await a slow// Chromium launch for minutes, and this node's lease can lapse and be taken over inside// that window. FAILED is excluded from the boot reset AND from the takeover sweep, so- // writing it onto a row a peer now owns strands the session on every node.- if (this.host.ownsSession(id)) {+ // writing it onto a row a peer now owns strands the session on every node. Skipped during+ // shutdown too: an init that rejects then usually does so because shutdown destroyed its+ // engine, and the INITIALIZING row is reset by the next boot, where FAILED would not be.+ if (this.host.ownsSession(id) && !this.host.isClosed()) {await this.host.updateStatus(id, SessionStatus.FAILED).catch(() => undefined);}}diff --git a/src/modules/session/session-engine-lifecycle.service.ts b/src/modules/session/session-engine-lifecycle.service.tsindex e00a9c3c9..d7851813f 100644--- a/src/modules/session/session-engine-lifecycle.service.ts+++ b/src/modules/session/session-engine-lifecycle.service.ts@@ -34,7 +34,12 @@ import {SessionEngineEventWiring,SessionEngineWiringHost,} from './session-engine-event-wiring';-import { SessionEngineControls, StopMarks, type StopHooks } from './session-engine-controls';+import {+ SessionEngineControls,+ ServerShuttingDownException,+ StopMarks,+ type StopHooks,+} from './session-engine-controls';import { SessionOwnershipService, nodeOwnsSession } from './session-ownership.service';/**@@ -226,6 +231,9 @@ export class SessionEngineLifecycle {private readonly wiringHost: SessionEngineWiringHost;private readonly controls: SessionEngineControls;+ // Set by shutdown() in the same tick as its registry snapshot; initializeEngine refuses from then on.+ private closed = false;+// Reconnection state per sessionprivate reconnectStates: Map<string, ReconnectState> = new Map();@@ -408,6 +416,7 @@ export class SessionEngineLifecycle {// call-ins are NON-async passthrough closures (the Task-1 delegate rule above).this.controls = new SessionEngineControls({ownsSession: (id: string) => this.ownsSession(id),+ isClosed: () => this.closed,sessionRepository: this.sessionRepository,engineFactory: this.engineFactory,engines: this.engineRegistry,@@ -468,6 +477,7 @@ export class SessionEngineLifecycle {/** Delegate: SessionEngineControls.shutdown. */shutdown(): Promise<void> {+ this.closed = true;return this.controls.shutdown();}@@ -630,6 +640,12 @@ export class SessionEngineLifecycle {}private async initializeEngine(id: string, session: Session): Promise<void> {+ // shutdown() destroys only the engines registered when it starts. A start or a transient retry+ // still on its way here would register one nothing tears down, so it fails instead; its caller+ // releases the claim. A 503 is not a transient launch failure, so it is not retried.+ if (this.closed) {+ throw new ServerShuttingDownException();+ }// A stop that landed before this engine exists had nothing to tear down and already wrote its// DISCONNECTED; registering now would overwrite it with INITIALIZING and then retire. start()// clears its own mark and executeReconnect checks it on entry, so a mark seen here always came@@ -695,8 +711,10 @@ export class SessionEngineLifecycle {// engine.initialize() below — an intervening await would re-open the retirement window, and it is// also why ONE isLiveEngine check is enough: the two guards are separated by a synchronous Set// lookup, so nothing can swap the engine between them (a second, identical check used to sit- // after the stop mark and could never disagree with this one).- if (!this.isLiveEngine(id, engine)) {+ // after the stop mark and could never disagree with this one). `closed` covers an engine that+ // shutdown() is already destroying: it stays registered until every destroy settles. Returned,+ // not thrown: start()'s failure path would force-destroy the engine shutdown is destroying.+ if (this.closed || !this.isLiveEngine(id, engine)) {return;}if (this.stoppingSessions.has(id)) {diff --git a/src/modules/session/session.controller.ts b/src/modules/session/session.controller.tsindex eda224a0f..d24f88574 100644--- a/src/modules/session/session.controller.ts+++ b/src/modules/session/session.controller.ts@@ -327,6 +327,12 @@ export class SessionController {'clears it and starts the session; after a delete, or an import that removed the session, the ' +'retry answers 404.',})+ @ApiResponse({+ status: 503,+ description:+ "The gateway is shutting down, or the engine's browser or socket connection died during launch and " ++ 'the one automatic retry failed too; no engine is left running, so the start can be retried.',+ })@ApiResponse({status: 504,description:diff --git a/src/modules/session/session.service.spec.ts b/src/modules/session/session.service.spec.tsindex f45eedcd7..47ed67d64 100644--- a/src/modules/session/session.service.spec.ts+++ b/src/modules/session/session.service.spec.ts@@ -18,7 +18,7 @@ import { EngineNotSupportedError } from '../../common/errors/engine-not-supporteimport { ConfigService } from '@nestjs/config';import { SessionService, AUTOSTART_THROTTLE_MS } from './session.service';import { SessionOwnershipService } from './session-ownership.service';-import { SessionStoppedException } from './session-engine-controls';+import { ServerShuttingDownException, SessionStoppedException } from './session-engine-controls';import { decideReconnect, STABLE_READY_MS } from './reconnect-policy';import { ACK_RECONCILE_DELAY_MS } from './message-projector.service';import { SessionEngineLifecycle, type ReconnectState } from './session-engine-lifecycle.service';@@ -2200,6 +2200,157 @@ describe('SessionService', () => {expect(await nodeIdOf(row.id)).toBeNull();});+ // The retry would re-claim the row after shutdown released it and launch an engine nothing tears down.+ it('start() does not retry a transient failure once shutdown ran during the delay', async () => {+ const row = await seed();+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));+ mockEngine.initialize.mockRejectedValueOnce(new EngineTransportError('Protocol error: Target closed'));+ const claim = jest.spyOn(ownership, 'claim');++ const outcome = service.start(row.id).catch((error: unknown) => error);+ await until(() => mockEngine.initialize.mock.calls.length === 1 && !lifecycle.isEngineActive(row.id));+ await service.onModuleDestroy();++ expect(await outcome).toMatchObject({ status: 503 });+ expect(claim).toHaveBeenCalledTimes(1);+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ expect(lifecycle.isEngineActive(row.id)).toBe(false);+ expect(await nodeIdOf(row.id)).toBeNull();+ });++ // The first attempt's FAILED is skipped by the boot reset, auto-start and the takeover sweep,+ // and shutdown refuses the retry that would have overwritten it.+ describe('a transient failure whose retry shutdown refuses leaves the session for the next boot', () => {+ const statusOf = async (id: string): Promise<SessionStatus> => (await sessions.findOneByOrFail({ id })).status;+ const untilFailed = async (id: string): Promise<void> => {+ for (let i = 0; i < 200 && (await statusOf(id)) !== SessionStatus.FAILED; i++) {+ await new Promise(resolve => setImmediate(resolve));+ }+ expect(await statusOf(id)).toBe(SessionStatus.FAILED);+ };+ const arrange = async (): Promise<Session> => {+ const row = await seed({ status: SessionStatus.DISCONNECTED, phone: '123' });+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));+ (repository.update as jest.Mock).mockImplementation((...args: Parameters<Repository<Session>['update']>) =>+ sessions.update(...args),+ );+ mockEngine.initialize.mockRejectedValueOnce(new EngineTransportError('Protocol error: Target closed'));+ return row;+ };+ const originalFlag = process.env.AUTO_START_SESSIONS;++ afterEach(() => {+ if (originalFlag === undefined) delete process.env.AUTO_START_SESSIONS;+ else process.env.AUTO_START_SESSIONS = originalFlag;+ });++ it('for an explicit start refused during the delay', async () => {+ const row = await arrange();++ const outcome = service.start(row.id, { explicit: true }).catch((error: unknown) => error);+ await untilFailed(row.id);+ await service.onModuleDestroy();++ expect(await outcome).toBeInstanceOf(ServerShuttingDownException);+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ expect(await statusOf(row.id)).toBe(SessionStatus.DISCONNECTED);+ expect(await nodeIdOf(row.id)).toBeNull();+ });++ it('for a boot auto-start refused during the delay', async () => {+ const row = await arrange();+ process.env.AUTO_START_SESSIONS = 'true';+ (repository.find as jest.Mock).mockResolvedValue([createMockSession({ id: row.id })]);++ service.onApplicationBootstrap();+ await untilFailed(row.id);+ await service.onModuleDestroy();++ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ expect(await statusOf(row.id)).toBe(SessionStatus.DISCONNECTED);+ expect(await nodeIdOf(row.id)).toBeNull();+ });++ it('for a retry whose re-claim was pending when shutdown began', async () => {+ const row = await arrange();+ const claimNow = ownership.claim.bind(ownership);+ let releaseClaim: () => void = () => undefined;+ const reclaiming = new Promise<void>(reached => {+ jest+ .spyOn(ownership, 'claim')+ .mockImplementationOnce(claimNow)+ .mockImplementationOnce(id => {+ reached();+ return new Promise<boolean>(resolve => (releaseClaim = () => resolve(claimNow(id))));+ });+ });++ const startingHooks = (): number =>+ (hookManager.execute as jest.Mock).mock.calls.filter(([event]) => event === 'session:starting').length;++ const outcome = service.start(row.id, { explicit: true }).catch((error: unknown) => error);+ await reclaiming;+ expect(await statusOf(row.id)).toBe(SessionStatus.FAILED);+ expect(startingHooks()).toBe(1);+ await service.onModuleDestroy();+ releaseClaim();++ expect(await outcome).toBeInstanceOf(ServerShuttingDownException);+ expect(startingHooks()).toBe(1);+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ expect(await statusOf(row.id)).toBe(SessionStatus.DISCONNECTED);+ expect(await nodeIdOf(row.id)).toBeNull();+ });++ it('but not on a row a peer adopted since, whose FAILED is its own', async () => {+ const row = await arrange();++ const outcome = service.start(row.id, { explicit: true }).catch((error: unknown) => error);+ await untilFailed(row.id);+ await sessions.update(row.id, { nodeId: 'node-b', leaseExpiresAt: new Date(Date.now() + 60_000) });+ await service.onModuleDestroy();++ expect(await outcome).toBeInstanceOf(ServerShuttingDownException);+ expect(await statusOf(row.id)).toBe(SessionStatus.FAILED);+ expect(await nodeIdOf(row.id)).toBe('node-b');+ });+ });++ it('start() after shutdown released the claims does not claim the row again', async () => {+ const row = await seed({ status: SessionStatus.DISCONNECTED });+ const claim = jest.spyOn(ownership, 'claim');+ await service.onModuleDestroy();++ await expect(service.start(row.id)).rejects.toMatchObject({ status: 503 });+ expect(claim).not.toHaveBeenCalled();+ expect(await nodeIdOf(row.id)).toBeNull();+ });++ // The claim resolves after shutdown released this node's rows; the start must not clear the stop or run the hook.+ it('start() still waiting on its claim when shutdown begins neither clears a stop nor runs the hook', async () => {+ const row = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id, desiredState: 'stopped' }));+ const claimNow = ownership.claim.bind(ownership);+ let releaseClaim: () => void = () => undefined;+ const claim = jest+ .spyOn(ownership, 'claim')+ .mockImplementationOnce(id => new Promise<boolean>(resolve => (releaseClaim = () => resolve(claimNow(id)))));++ const outcome = service.start(row.id, { explicit: true }).catch((error: unknown) => error);+ await until(() => claim.mock.calls.length === 1);+ await service.onModuleDestroy();+ releaseClaim();++ expect(await outcome).toMatchObject({ status: 503 });+ expect(hookManager.execute).not.toHaveBeenCalled();+ expect((repository.update as jest.Mock).mock.calls).not.toContainEqual([+ { id: row.id, desiredState: 'stopped' },+ { desiredState: null },+ ]);+ expect(engineFactory.create).not.toHaveBeenCalled();+ expect(await nodeIdOf(row.id)).toBeNull();+ });+// The earlier stop's mark is still set when the row read fails, before start() clears it.it('start() still retries a transient failure when only an earlier stop() left its mark', async () => {const row = await seed();@@ -8548,6 +8699,132 @@ describe('SessionService', () => {expect(mockEngine.destroy).toHaveBeenCalled();expect(service.isActive('sess-uuid-1')).toBe(false);});++ it('refuses a start still on its way to the engine when shutdown tears the engines down', async () => {+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());+ let releaseHook: () => void = () => undefined;+ const hookHeld = new Promise<void>(resolve => {+ (hookManager.execute as jest.Mock).mockImplementationOnce(+ (_e: string, data: unknown) =>+ new Promise(done => {+ releaseHook = () => done({ continue: true, data });+ resolve();+ }),+ );+ });++ const start = service.start('sess-uuid-1');+ await hookHeld;+ await service.onModuleDestroy();+ releaseHook();++ await expect(start).rejects.toMatchObject({ status: 503 });+ expect(engineFactory.create).not.toHaveBeenCalled();+ expect(service.isActive('sess-uuid-1')).toBe(false);+ });++ it('refuses a start that arrives after shutdown before it clears a stop or runs the hook', async () => {+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ desiredState: 'stopped' }));+ await service.onModuleDestroy();++ await expect(service.start('sess-uuid-1', { explicit: true })).rejects.toMatchObject({ status: 503 });+ expect(hookManager.execute).not.toHaveBeenCalled();+ expect((repository.update as jest.Mock).mock.calls).not.toContainEqual([+ { id: 'sess-uuid-1', desiredState: 'stopped' },+ { desiredState: null },+ ]);+ expect(engineFactory.create).not.toHaveBeenCalled();+ });++ it('does not initialize an engine whose first status write settles while shutdown destroys it', async () => {+ const flush = async (): Promise<void> => {+ for (let i = 0; i < 20; i++) await new Promise(resolve => setImmediate(resolve));+ };+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());+ let releaseWrite: () => void = () => undefined;+ (repository.update as jest.Mock).mockImplementation((_id: unknown, patch: { status?: SessionStatus }) =>+ patch?.status === SessionStatus.INITIALIZING+ ? new Promise(resolve => (releaseWrite = () => resolve({ affected: 1 })))+ : Promise.resolve({ affected: 1 }),+ );+ let releaseDestroy: () => void = () => undefined;+ mockEngine.destroy.mockImplementationOnce(() => new Promise<void>(resolve => (releaseDestroy = resolve)));++ const start = service.start('sess-uuid-1');+ await flush();+ expect(engineFactory.create).toHaveBeenCalledTimes(1);+ const shutdown = service.onModuleDestroy();+ await flush();+ expect(mockEngine.destroy).toHaveBeenCalledTimes(1);+ releaseWrite();+ await flush();+ releaseDestroy();+ await shutdown;+ await start;++ expect(mockEngine.initialize).not.toHaveBeenCalled();+ expect(service.isActive('sess-uuid-1')).toBe(false);+ });++ it('does not initialize a reconnect engine whose first status write settles while shutdown destroys it', async () => {+ const flush = async (): Promise<void> => {+ for (let i = 0; i < 20; i++) await new Promise(resolve => setImmediate(resolve));+ };+ let releaseWrite: () => void = () => undefined;+ (repository.update as jest.Mock).mockImplementation((_id: unknown, patch: { status?: SessionStatus }) =>+ patch?.status === SessionStatus.INITIALIZING+ ? new Promise(resolve => (releaseWrite = () => resolve({ affected: 1 })))+ : Promise.resolve({ affected: 1 }),+ );+ let releaseDestroy: () => void = () => undefined;+ mockEngine.destroy.mockImplementationOnce(() => new Promise<void>(resolve => (releaseDestroy = resolve)));+ const internals = lifecycle as unknown as {+ reconnectStates: Map<string, ReconnectState>;+ executeReconnect: (id: string, session: Session, state: ReconnectState) => Promise<void>;+ };+ const state: ReconnectState = { attempts: 1, timer: null, maxAttempts: 5, baseDelay: 5000 };+ internals.reconnectStates.set('sess-uuid-1', state);++ const reconnect = internals.executeReconnect('sess-uuid-1', createMockSession(), state);+ await flush();+ expect(engineFactory.create).toHaveBeenCalledTimes(1);+ const shutdown = service.onModuleDestroy();+ await flush();+ expect(mockEngine.destroy).toHaveBeenCalledTimes(1);+ releaseWrite();+ await flush();+ releaseDestroy();+ await shutdown;+ await reconnect;++ expect(mockEngine.initialize).not.toHaveBeenCalled();+ expect(service.isActive('sess-uuid-1')).toBe(false);+ });++ // FAILED is left out of the boot reset and auto-start, so the session would stay down after a routine restart.+ it('does not mark a session FAILED when shutdown destroys its engine mid-initialize', async () => {+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());+ let rejectInit: (error: Error) => void = () => undefined;+ mockEngine.initialize.mockImplementationOnce(+ () => new Promise<void>((_resolve, reject) => (rejectInit = reject)),+ );+ mockEngine.destroy.mockImplementationOnce(() => {+ rejectInit(new Error('Target closed'));+ return Promise.resolve();+ });+ const updateStatus = jest.spyOn(lifecycle, 'updateStatus');++ const start = service.start('sess-uuid-1').catch((error: unknown) => error);+ for (let i = 0; i < 50 && mockEngine.initialize.mock.calls.length === 0; i++) {+ await new Promise(resolve => setImmediate(resolve));+ }+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ await service.onModuleDestroy();++ expect(await start).toMatchObject({ message: 'Target closed' });+ expect(updateStatus).not.toHaveBeenCalledWith('sess-uuid-1', SessionStatus.FAILED);+ expect(service.isActive('sess-uuid-1')).toBe(false);+ });});// ── operator stop intent ──────────────────────────────────────────@@ -9111,6 +9388,74 @@ describe('SessionService', () => {);});+ it('reports a launch refused by shutdown as abandoned, not as a failed start', async () => {+ process.env.AUTO_START_SESSIONS = 'true';+ (repository.find as jest.Mock).mockResolvedValue([{ id: 'a', name: 'A' }]);+ jest.spyOn(service, 'start').mockImplementationOnce(() => {+ (service as unknown as { shuttingDown: boolean }).shuttingDown = true;+ return Promise.reject(new ServerShuttingDownException());+ });+ const logger = (service as unknown as { logger: { log: jest.Mock; error: jest.Mock } }).logger;+ const logError = jest.spyOn(logger, 'error');+ const logInfo = jest.spyOn(logger, 'log');++ service.onApplicationBootstrap();+ await autoStartRun();++ expect(logError).not.toHaveBeenCalled();+ expect(logInfo).toHaveBeenCalledWith(+ 'Auto-start abandoned for session A: shutting down',+ expect.objectContaining({ action: 'auto_start_aborted' }),+ );+ });++ // onModuleDestroy awaits this loop, so the inter-launch throttle would only delay the exit.+ it('ends the run without the inter-launch throttle once a launch is refused by shutdown', async () => {+ process.env.AUTO_START_SESSIONS = 'true';+ (repository.find as jest.Mock).mockResolvedValue([+ { id: 'a', name: 'A' },+ { id: 'b', name: 'B' },+ ]);+ const startSpy = jest.spyOn(service, 'start').mockImplementationOnce(() => {+ (service as unknown as { shuttingDown: boolean }).shuttingDown = true;+ return Promise.reject(new ServerShuttingDownException());+ });++ jest.useFakeTimers();+ try {+ service.onApplicationBootstrap();+ let ended = false;+ void autoStartRun().then(() => (ended = true));+ await jest.advanceTimersByTimeAsync(AUTOSTART_THROTTLE_MS - 1);++ expect(ended).toBe(true);+ expect(startSpy).toHaveBeenCalledTimes(1);+ } finally {+ jest.useRealTimers();+ }+ });++ // EngineTransportError is a 503 too; a real launch failure stays an error even during shutdown.+ it('reports a transport failure that coincides with shutdown as a failed start', async () => {+ process.env.AUTO_START_SESSIONS = 'true';+ (repository.find as jest.Mock).mockResolvedValue([{ id: 'a', name: 'A' }]);+ jest.spyOn(service, 'start').mockImplementationOnce(() => {+ (service as unknown as { shuttingDown: boolean }).shuttingDown = true;+ return Promise.reject(new EngineTransportError('Protocol error: Target closed'));+ });+ const logger = (service as unknown as { logger: { error: jest.Mock } }).logger;+ const logError = jest.spyOn(logger, 'error');++ service.onApplicationBootstrap();+ await autoStartRun();++ expect(logError).toHaveBeenCalledWith(+ 'Auto-start failed for session: A',+ 'Protocol error: Target closed',+ expect.objectContaining({ action: 'auto_start_failed' }),+ );+ });+// Nest binds the HTTP listener only after every bootstrap hook settles, and a launch is a// Chromium start bounded by resolveEngineInitTimeoutMs (>= 60s) with a 2s throttle between// sessions. Awaiting the loop here kept the port CLOSED for that whole time, so every probe indiff --git a/src/modules/session/session.service.ts b/src/modules/session/session.service.tsindex 031f3a869..84b533bfe 100644--- a/src/modules/session/session.service.ts+++ b/src/modules/session/session.service.ts@@ -57,7 +57,7 @@ import { HookManager } from '../../core/hooks';import { LidMappingStoreService } from '../../engine/identity/lid-mapping-store.service';import { resolveJidCandidates } from '../../engine/identity/jid-candidates';import { Message } from '../message/entities/message.entity';-import { SessionStoppedException } from './session-engine-controls';+import { ServerShuttingDownException, SessionStoppedException } from './session-engine-controls';// Type-only: the module binds this class to PLUGIN_SESSION_PORT with a `useExisting` alias, which// TypeScript does not check, so `implements` is what keeps the two in step.import type { PluginSessionPort } from '../../core/plugins/plugin-host-ports';@@ -301,6 +301,14 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicatsessionId: session.id,action: 'auto_start_skipped',});+ } else if (error instanceof ServerShuttingDownException) {+ // The launch in flight when shutdown began, refused so nothing outlives the teardown.+ this.logger.log(`Auto-start abandoned for session ${session.name}: shutting down`, {+ sessionId: session.id,+ action: 'auto_start_aborted',+ });+ // Nothing later in the run would start either, and shutdown waits on this loop.+ return;} else {const errorMessage = error instanceof Error ? error.message : 'Unknown error';this.logger.error(`Auto-start failed for session: ${session.name}`, errorMessage, {@@ -602,6 +610,9 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat* the engine start refuses them a row still marked stopped when it reads it.*/async start(id: string, { explicit = false }: { explicit?: boolean } = {}): Promise<Session> {+ // Refused before the claim, the stop clear and the session:starting hook: none of them should run+ // for a start the exiting process will not keep.+ this.refuseIfShuttingDown();// At the cap, refused before the claim: the engine's own cap check runs after it, and its// refusal releases the claim to no node. A row nobody holds is never adopted by a peer, while a// lapsed lease left where it is gets taken over by one with room. The engine check stays the@@ -631,6 +642,10 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat}}+ private refuseIfShuttingDown(): void {+ if (this.shuttingDown) throw new ServerShuttingDownException();+ }+private async claimAndStart(id: string, explicit: boolean): Promise<Session> {// Read before the claim, so a stop that lands while the claim is pending counts against this start.const stopRequestsBefore = this.stopRequests.get(id);@@ -684,6 +699,8 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicatif (this.stopRequests.get(id) !== stopRequestsBefore) {throw new SessionStoppedException(`Session ${id} was stopped`);}+ // Shutdown may have begun while the claim was pending; the stop clear and the hook must not run.+ this.refuseIfShuttingDown();try {return await this.engineLifecycle.start(id, { explicit });} catch (error) {@@ -702,17 +719,38 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicatif (this.stopRequests.get(id) !== stopRequestsBefore) {throw new SessionStoppedException(`Session ${id} was stopped`);}- // The lease may have lapsed while the first attempt ran; the retry must keep holding the- // claim, never 409 on the session it already owns.- if (this.ownership && !(await this.ownership.claim(id))) {- await this.findOne(id);- throw new ConflictException(`Session ${id} is running on another node`);- }- // Checked again: a stop that landed during the re-claim leaves the claim for claimAndStart's catch to release.- if (this.stopRequests.get(id) !== stopRequestsBefore) {- throw new SessionStoppedException(`Session ${id} was stopped`);+ try {+ // Shutdown may have released this node's claims during the delay; re-claiming would pin the row.+ this.refuseIfShuttingDown();+ // The lease may have lapsed while the first attempt ran; the retry must keep holding the+ // claim, never 409 on the session it already owns.+ if (this.ownership && !(await this.ownership.claim(id))) {+ await this.findOne(id);+ throw new ConflictException(`Session ${id} is running on another node`);+ }+ // Checked again: a stop that landed during the re-claim leaves the claim for claimAndStart's catch to release.+ if (this.stopRequests.get(id) !== stopRequestsBefore) {+ throw new SessionStoppedException(`Session ${id} was stopped`);+ }+ // And shutdown may have begun during the re-claim; the stop clear and the hook must not run.+ this.refuseIfShuttingDown();+ return await this.engineLifecycle.start(id, { explicit });+ } catch (retryError) {+ // The first attempt may have left FAILED, which the boot reset, auto-start and the takeover+ // sweep all skip, and shutdown refused the retry that would have overwritten it. Handed back+ // as DISCONNECTED so the next boot restores the session. Best-effort: the refusal is the answer.+ // Fenced like the boot reset: a FAILED on a row a peer has since adopted is the peer's.+ if (retryError instanceof ServerShuttingDownException) {+ const claimable = this.ownership?.claimableWhere() ?? [{}];+ await this.sessionRepository+ .update(+ claimable.map(clause => ({ ...clause, id, status: SessionStatus.FAILED })),+ { status: SessionStatus.DISCONNECTED },+ )+ .catch(() => undefined);+ }+ throw retryError;}- return this.engineLifecycle.start(id, { explicit });}}