Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(session): close ownership and stop-mark gaps in session lifecycle

Yudhi Armyndharis23851548aa55
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3ce9b6e97..90af2ab25 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,6 +19,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- A session whose automatic reconnect fails to relaunch the engine (a network, DNS or browser launch error) keeps retrying with backoff instead of stopping in `failed` until restarted by hand; an authentication failure or a stale browser profile still ends in `failed` ([#1580](https://github.com/rmyndharis/OpenWA/issues/1580)).
- A session that runs out of reconnect attempts fires the `session:error` plugin hook when it lands in `failed`.
+- A session that runs out of reconnect attempts keeps the last attempt's failure reason in `lastError` and in the `session:error` hook, after the attempts message.
+- A stop during the retry delay after a transient start failure is no longer undone by the retry.
+- In a multi-node deployment, a start cut short by a concurrent stop releases its claim, so a peer no longer adopts and restarts the stopped session.
+- A stop or delete that fails on a database error no longer blocks the session's next automatic reconnect or makes a later start answer "already started".
+- A logout or force-kill refused as "not started" leaves the session's claim untouched, so a crashed node's session stays visible to the takeover sweep.
+- `GET /api/sessions/:sessionId/presence/:chatId` returns `null` once the session has no running engine, instead of the last presence reported before a stop, logout, force-kill or failure.
- An explicit `maxReconnectAttempts` is honoured and the reconnect delay is capped at 5 minutes; the budget used to restart once the backoff passed 5 minutes, so the limit was never reached and the documented 1-hour cap never applied.
## [0.23.5] - 2026-09-14
diff --git a/src/modules/session/session-engine-controls.ts b/src/modules/session/session-engine-controls.ts
index 26f1e70bd..a152e61f0 100644
--- a/src/modules/session/session-engine-controls.ts
+++ b/src/modules/session/session-engine-controls.ts
@@ -124,6 +124,20 @@ export class SessionEngineControls {
return this.sessionRestrictions.attachTo(this.sessionErrors.attachTo(session));
}
+ /**
+ * requireSession for stop()/delete(), whose caller sets the stop mark before this read. A read
+ * that fails has retired nothing, so the mark must not outlive it: left on a running session, its
+ * next disconnect would never reconnect and start() would keep refusing it as already started.
+ */
+ private async requireSessionOrDropStopMark(id: string): Promise<Session> {
+ try {
+ return await this.requireSession(id);
+ } catch (error) {
+ this.stoppingSessions.delete(id);
+ throw error;
+ }
+ }
+
async start(id: string): Promise<Session> {
// Reserve the slot SYNCHRONOUSLY at entry — before even the requireSession await. Two
// near-simultaneous start() calls must not both pass the check and orphan an engine (the has()
@@ -257,7 +271,7 @@ export class SessionEngineControls {
}
async stop(id: string): Promise<Session> {
- const session = await this.requireSession(id);
+ const session = await this.requireSessionOrDropStopMark(id);
// Mark as tearing down BEFORE cleanup so an in-flight reconnect can't resurrect it.
this.stoppingSessions.add(id);
@@ -440,7 +454,7 @@ export class SessionEngineControls {
}
async delete(id: string): Promise<void> {
- const session = await this.requireSession(id);
+ const session = await this.requireSessionOrDropStopMark(id);
// FENCE #1 — fail-fast on an ALREADY-PENDING credential teardown for this session NAME, BEFORE
// any lifecycle mutation. A logout teardown that lost its deadline race is still running and ends
diff --git a/src/modules/session/session-engine-event-wiring.ts b/src/modules/session/session-engine-event-wiring.ts
index 5fdbba3f4..b9b3e3111 100644
--- a/src/modules/session/session-engine-event-wiring.ts
+++ b/src/modules/session/session-engine-event-wiring.ts
@@ -27,7 +27,7 @@ import { userPart } from '../../engine/identity/wa-id';
import { SessionEngineLeafEvents } from './session-engine-leaf-events';
/** The lastError an engine-internal reconnect episode records; onQRCode clears only this one. */
-const RECONNECT_LOOP_REASON = 'Reconnecting after a dropped connection';
+export const RECONNECT_LOOP_REASON = 'Reconnecting after a dropped connection';
/**
* The call-ins SessionEngineEventWiring needs from the lifecycle core. Built ONCE in the
diff --git a/src/modules/session/session-engine-lifecycle.service.ts b/src/modules/session/session-engine-lifecycle.service.ts
index 553838fa8..9f33cc99f 100644
--- a/src/modules/session/session-engine-lifecycle.service.ts
+++ b/src/modules/session/session-engine-lifecycle.service.ts
@@ -29,7 +29,11 @@ import { HookManager } from '../../core/hooks';
import { SessionLifecycleFences } from './session-lifecycle-fences';
import { SessionStatusBroadcaster } from './session-status-broadcaster';
import { SessionEngineLeafEvents } from './session-engine-leaf-events';
-import { SessionEngineEventWiring, SessionEngineWiringHost } from './session-engine-event-wiring';
+import {
+ RECONNECT_LOOP_REASON,
+ SessionEngineEventWiring,
+ SessionEngineWiringHost,
+} from './session-engine-event-wiring';
import { SessionEngineControls } from './session-engine-controls';
import { SessionOwnershipService, nodeOwnsSession } from './session-ownership.service';
@@ -988,19 +992,22 @@ export class SessionEngineLifecycle {
action: 'reconnect_failed',
});
// Don't leave the session silently stuck DISCONNECTED — mark it terminally FAILED with a reason
- // so findOne/findAll surface it via `lastError` and the dashboard shows it needs a restart.
- this.sessionErrors.set(id, decision.reason);
+ // so findOne/findAll surface it via `lastError` and the dashboard shows it needs a restart. The
+ // last attempt's own failure is kept alongside: it is the diagnosis, and this write would
+ // otherwise erase it. The engine-internal loop banner is not a cause, so it is not repeated.
+ const lastError = this.sessionErrors.get(id);
+ const reason =
+ lastError && !lastError.startsWith(RECONNECT_LOOP_REASON)
+ ? `${decision.reason} Last error: ${lastError}`
+ : decision.reason;
+ this.sessionErrors.set(id, reason);
// Same ownership fence as the engine callbacks: a reconnect chain that exhausts itself after
// this node's lease lapsed must not park a peer's session in FAILED, which nothing resets
// automatically. The in-memory error above is per-process and harmless either way.
if (this.ownsSession(id)) void this.updateStatus(id, SessionStatus.FAILED);
// The hook signal a terminal failure carries: a failed re-init no longer fires session:error per
// attempt, so the episode's one terminal end reports it here.
- void this.hookManager.execute(
- 'session:error',
- { reason: decision.reason },
- { sessionId: id, source: 'SessionService' },
- );
+ void this.hookManager.execute('session:error', { reason }, { sessionId: id, source: 'SessionService' });
// Terminal path — evict the dead engine so it neither holds a concurrency slot nor makes a
// subsequent start() reject the session as "already started". This mirrors onError's terminal
// path (the same rationale: leaving the engine in the map wedges the session). The engine may
diff --git a/src/modules/session/session.service.spec.ts b/src/modules/session/session.service.spec.ts
index d655731cb..b89fb8628 100644
--- a/src/modules/session/session.service.spec.ts
+++ b/src/modules/session/session.service.spec.ts
@@ -16,6 +16,8 @@ import {
import { EngineTransportError } from '../../common/errors/engine-transport.error';
import { ConfigService } from '@nestjs/config';
import { SessionService, AUTOSTART_THROTTLE_MS } from './session.service';
+import { SessionOwnershipService } from './session-ownership.service';
+import { decideReconnect } from './reconnect-policy';
import { ACK_RECONCILE_DELAY_MS } from './message-projector.service';
import { SessionEngineLifecycle, type ReconnectState } from './session-engine-lifecycle.service';
import {
@@ -1369,7 +1371,7 @@ describe('SessionService', () => {
expect(stopping.has('sess-uuid-1')).toBe(false);
});
- // The counterpart, and the reason the reclamation is narrowed to 404 rather than every failure:
+ // The counterpart, and the reason the reclamation is narrowed to 404 and a failed read:
// a refusal against a session that DOES exist must still leave its mark, which is the
// documented "harmless, cleared by the next start()" behaviour the mark relies on.
it.each([
@@ -1386,6 +1388,160 @@ describe('SessionService', () => {
expect(stopping.has('sess-uuid-1')).toBe(true);
stopping.delete('sess-uuid-1');
});
+
+ // A failed read decided nothing. A mark left on a running session made its next disconnect skip
+ // the reconnect while the dead engine stayed registered, so start() answered "already started".
+ it.each([
+ ['stop', (s: SessionService) => s.stop('sess-uuid-1')],
+ ['delete', (s: SessionService) => s.delete('sess-uuid-1')],
+ ])('%s() failing on its session read leaves a running session able to reconnect', async (_verb, call) => {
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession());
+ (repository.update as jest.Mock).mockResolvedValue({ affected: 1 });
+ await service.start('sess-uuid-1');
+ const callbacks = (mockEngine.initialize.mock.calls as [EngineEventCallbacks][])[0][0];
+ const internals = lifecycle as unknown as {
+ stoppingSessions: Set<string>;
+ reconnectStates: Map<string, ReconnectState>;
+ executeReconnect: (id: string, session: Session, state: ReconnectState) => Promise<void>;
+ };
+
+ (repository.findOne as jest.Mock).mockRejectedValueOnce(new Error('Connection terminated unexpectedly'));
+ await expect(call(service)).rejects.toThrow('Connection terminated unexpectedly');
+ expect(internals.stoppingSessions.has('sess-uuid-1')).toBe(false);
+
+ callbacks.onDisconnected?.('socket closed');
+ await new Promise(resolve => setImmediate(resolve));
+ const state = internals.reconnectStates.get('sess-uuid-1')!;
+ expect(state.timer).not.toBeNull();
+ clearTimeout(state.timer!);
+ await internals.executeReconnect('sess-uuid-1', createMockSession(), state);
+
+ expect(mockEngine.initialize).toHaveBeenCalledTimes(2);
+ expect(lifecycle.isEngineActive('sess-uuid-1')).toBe(true);
+ });
+
+ it.each([
+ ['stop', (s: SessionService) => s.stop('sess-uuid-1')],
+ ['delete', (s: SessionService) => s.delete('sess-uuid-1')],
+ ])('%s() whose ownership query fails leaves no stop mark', async (_verb, call) => {
+ const ownership = withOwnership();
+ ownership.isHeldByOtherNode.mockRejectedValue(new Error('Connection terminated unexpectedly'));
+ const stopping = (lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions;
+
+ await expect(call(service)).rejects.toThrow('Connection terminated unexpectedly');
+
+ expect(stopping.has('sess-uuid-1')).toBe(false);
+ });
+
+ // The claim predicates are SQL, so these run the real ownership service on a real database.
+ describe('against a real database', () => {
+ let db: DataSource;
+ let sessions: Repository<Session>;
+ let ownership: SessionOwnershipService;
+
+ const seed = (overrides: Partial<Session> = {}): Promise<Session> =>
+ sessions.save(sessions.create({ name: 'owned', status: SessionStatus.READY, config: {}, ...overrides }));
+ const nodeIdOf = async (id: string): Promise<string | null> => (await sessions.findOneByOrFail({ id })).nodeId;
+ const until = async (condition: () => boolean): Promise<void> => {
+ for (let i = 0; i < 200 && !condition(); i++) await new Promise(resolve => setImmediate(resolve));
+ expect(condition()).toBe(true);
+ };
+
+ beforeAll(async () => {
+ db = new DataSource({ type: 'better-sqlite3', database: ':memory:', entities: [Session], synchronize: true });
+ await db.initialize();
+ sessions = db.getRepository(Session);
+ });
+
+ afterAll(async () => {
+ await db.destroy();
+ });
+
+ beforeEach(() => {
+ ownership = new SessionOwnershipService(sessions, {
+ get: (key: string) => ({ 'session.nodeId': 'node-a', 'session.leaseTtlMs': 60_000 })[key],
+ } as unknown as ConfigService);
+ Object.assign(service as unknown as Record<string, unknown>, { ownership });
+ (repository.update as jest.Mock).mockResolvedValue({ affected: 1 });
+ });
+
+ afterEach(async () => {
+ await sessions.clear();
+ });
+
+ // The stop could not release while the start still held the session; nothing else would, so
+ // the row kept naming this node until a peer adopted the session the operator stopped.
+ it('start() retired by a concurrent stop() hands the claim back', async () => {
+ const row = await seed();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));
+ let resolveInit: () => void = () => undefined;
+ mockEngine.initialize.mockImplementationOnce(() => new Promise<void>(resolve => (resolveInit = resolve)));
+
+ const starting = service.start(row.id);
+ await until(() => mockEngine.initialize.mock.calls.length === 1);
+ await service.stop(row.id);
+ expect(await nodeIdOf(row.id)).toBe('node-a');
+
+ resolveInit();
+ await starting;
+
+ expect(lifecycle.isEngineActive(row.id)).toBe(false);
+ expect(await nodeIdOf(row.id)).toBeNull();
+ });
+
+ it('start() that succeeds keeps its claim', async () => {
+ const row = await seed();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));
+
+ await service.start(row.id);
+
+ expect(await nodeIdOf(row.id)).toBe('node-a');
+ });
+
+ // The failed first attempt stays in its 2s retry delay while the stop lands.
+ it('start() does not retry a transient failure once a stop() landed during the delay', async () => {
+ const row = await seed();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));
+ mockEngine.initialize.mockRejectedValueOnce(new EngineTransportError('Protocol error: Target closed'));
+
+ const starting = service.start(row.id);
+ const outcome = starting.catch((error: unknown) => error);
+ await until(() => mockEngine.initialize.mock.calls.length === 1 && !lifecycle.isEngineActive(row.id));
+ await service.stop(row.id);
+
+ expect(await outcome).toBeInstanceOf(EngineTransportError);
+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);
+ expect(lifecycle.isEngineActive(row.id)).toBe(false);
+ expect(await nodeIdOf(row.id)).toBeNull();
+ });
+
+ // The earlier stop's mark is still set when the row read fails, before start() clears it.
+ it('start() still retries a transient failure when only an earlier stop() left its mark', async () => {
+ const row = await seed();
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));
+ await service.stop(row.id);
+ (repository.findOne as jest.Mock).mockRejectedValueOnce(new Error('Connection terminated unexpectedly'));
+
+ await service.start(row.id);
+
+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);
+ expect(await nodeIdOf(row.id)).toBe('node-a');
+ });
+
+ // A 400 changes nothing, and release() also clears a lapsed foreign claim: that took a crashed
+ // node's session out of the lapsed-claim sweep for good.
+ it.each([
+ ['logout', (s: SessionService, id: string) => s.logout(id)],
+ ['forceKill', (s: SessionService, id: string) => s.forceKill(id)],
+ ])('%s() refused as not started keeps a lapsed foreign claim', async (_verb, call) => {
+ const row = await seed({ nodeId: 'dead-node', leaseExpiresAt: new Date(Date.now() - 60_000) });
+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));
+
+ await expect(call(service, row.id)).rejects.toThrow(BadRequestException);
+
+ expect(await nodeIdOf(row.id)).toBe('dead-node');
+ });
+ });
});
describe('isEngineActive', () => {
@@ -2341,6 +2497,110 @@ describe('SessionService', () => {
expect(errorHooks()).toBe(1);
expect(i.reconnectStates.has(ID)).toBe(false);
expect(i.engines.has(ID)).toBe(false);
+ // The budget message alone says nothing about why the attempts failed; the last one's reason
+ // stays on lastError and in the hook payload.
+ const exhausted = decideReconnect({ attempts: 5, maxAttempts: 5, baseDelay: 0 }) as { reason: string };
+ expect(i.sessionErrors.get(ID)).toBe(`${exhausted.reason} Last error: net::ERR_NAME_NOT_RESOLVED`);
+ const hook = (hookManager.execute as jest.Mock).mock.calls.find(([n]) => n === 'session:error') as unknown[];
+ expect(hook[1]).toEqual({ reason: i.sessionErrors.get(ID) });
+ });
+
+ it('does not nest the engine-internal reconnect banner inside the exhausted reason', () => {
+ const i = internals();
+ const state = armState({ attempts: 5, maxAttempts: 5 });
+ i.sessionErrors.set(ID, 'Reconnecting after a dropped connection (attempt 5, down for 40s).');
+
+ (lifecycle as unknown as { scheduleReconnect: (id: string, s: Session) => void }).scheduleReconnect(
+ ID,
+ createMockSession(),
+ );
+
+ expect(i.reconnectStates.has(ID)).toBe(false);
+ expect(state.timer).toBeNull();
+ const exhausted = decideReconnect({ attempts: 5, maxAttempts: 5, baseDelay: 0 }) as { reason: string };
+ expect(i.sessionErrors.get(ID)).toBe(exhausted.reason);
+ });
+
+ // The success path has its own replacement check: a parked failure of the OLD attempt must not
+ // run against the state a delete + start put in its place while that init was still running.
+ it('drops a parked failure when a delete-then-start replaced the state before init resolved', async () => {
+ const i = internals();
+ let callbacks: EngineEventCallbacks = {};
+ let resolveInit: () => void = () => undefined;
+ mockEngine.initialize.mockImplementationOnce((cb: EngineEventCallbacks) => {
+ callbacks = cb;
+ return new Promise<void>(res => (resolveInit = res));
+ });
+ const replacement = {
+ ...mockEngine,
+ initialize: jest.fn().mockResolvedValue(undefined),
+ forceDestroy: jest.fn(),
+ };
+ const state = armState();
+
+ const run = i.executeReconnect(ID, createMockSession(), state);
+ await flush();
+ reportFailure(callbacks, 'net::ERR_CONNECTION_RESET');
+ expect(state.parkedFailure).toBeDefined();
+ await service.delete(ID);
+ (engineFactory.create as jest.Mock).mockReturnValueOnce(replacement);
+ await service.start(ID);
+ const fresh = i.reconnectStates.get(ID);
+ (hookManager.execute as jest.Mock).mockClear();
+ (repository.update as jest.Mock).mockClear();
+ resolveInit();
+ await run;
+ await flush();
+
+ expect(fresh).toBeDefined();
+ expect(i.reconnectStates.get(ID)).toBe(fresh);
+ expect(i.engines.get(ID)).toBe(replacement);
+ expect(replacement.forceDestroy).not.toHaveBeenCalled();
+ expect(failedWrites()).toBe(0);
+ expect(errorHooks()).toBe(0);
+ });
+
+ // A retryable report arriving after a terminal one in the same init must not turn the terminal
+ // failure into a retry, which would loop on credentials that can never work.
+ it('keeps an earlier terminal report parked when a retryable one follows in the same init', async () => {
+ const i = internals();
+ mockEngine.initialize.mockImplementationOnce((cb: EngineEventCallbacks) => {
+ cb.onError?.(`${AUTH_FAILURE_REASON}: bad credentials`);
+ cb.onError?.('net::ERR_CONNECTION_RESET');
+ return Promise.reject(new Error('net::ERR_CONNECTION_RESET'));
+ });
+ const state = armState();
+
+ await i.executeReconnect(ID, createMockSession(), state);
+ await flush();
+
+ expect(failedWrites()).toBe(1);
+ expect(errorHooks()).toBe(1);
+ expect(hookManager.execute).toHaveBeenCalledWith(
+ 'session:error',
+ { reason: `${AUTH_FAILURE_REASON}: bad credentials` },
+ expect.anything(),
+ );
+ expect(state.timer).toBeNull();
+ expect(i.reconnectStates.has(ID)).toBe(false);
+ });
+
+ // Only a bare FAILED was parked, so the rejection message is the one place the terminal cause
+ // shows up.
+ it('stays terminal when only a bare FAILED was parked and the rejection names a terminal cause', async () => {
+ const i = internals();
+ mockEngine.initialize.mockImplementationOnce((cb: EngineEventCallbacks) => {
+ cb.onStateChanged?.(EngineStatus.FAILED);
+ return Promise.reject(new Error(`${AUTH_FAILURE_REASON}: bad credentials`));
+ });
+ const state = armState();
+
+ await i.executeReconnect(ID, createMockSession(), state);
+ await flush();
+
+ expect(failedWrites()).toBe(1);
+ expect(state.timer).toBeNull();
+ expect(state.attempts).toBe(1);
});
it('keeps an init failure on the start() path terminal as before', async () => {
@@ -3136,6 +3396,59 @@ describe('SessionService', () => {
);
expect(statusWrites).toHaveLength(0);
});
+
+ // A replaced engine whose own backoff is still running must not report on the replacement.
+ it('ignores reports from an engine that is no longer the live one', async () => {
+ const callbacks = await startAndCapture();
+ (webhookService.dispatch as jest.Mock).mockClear();
+ const engines = (service as unknown as { engines: EngineRegistry }).engines;
+ engines.set('sess-uuid-1', { ...mockEngine } as never);
+ const attemptsBefore = getSessionReconnectAttemptsTotal();
+
+ for (let attempt = 1; attempt <= 5; attempt++) callbacks.onReconnecting?.(attempt, 60_000);
+
+ const sessionErrors = (service as unknown as { sessionErrors: SessionErrorStore }).sessionErrors;
+ expect(sessionErrors.get('sess-uuid-1')).toBeUndefined();
+ expect(loopDispatches()).toHaveLength(0);
+ expect(getSessionReconnectAttemptsTotal()).toBe(attemptsBefore);
+ });
+
+ // The counters are module-global, so the assertions are deltas.
+ it('counts every attempt and one loop alert per fifth attempt', async () => {
+ const callbacks = await startAndCapture();
+ const attemptsBefore = getSessionReconnectAttemptsTotal();
+ const alertsBefore = getSessionReconnectLoopAlertsTotal();
+
+ for (let attempt = 1; attempt <= 5; attempt++) callbacks.onReconnecting?.(attempt, 60_000);
+
+ expect(getSessionReconnectAttemptsTotal() - attemptsBefore).toBe(5);
+ expect(getSessionReconnectLoopAlertsTotal() - alertsBefore).toBe(1);
+ });
+
+ // One engine can live through several episodes; each one's downtime starts at its own attempt 1.
+ it('measures the downtime of the current episode, not since an earlier one', async () => {
+ const callbacks = await startAndCapture();
+ const sessionErrors = (service as unknown as { sessionErrors: SessionErrorStore }).sessionErrors;
+ const start = Date.now();
+ const now = jest.spyOn(Date, 'now');
+ try {
+ for (let attempt = 1; attempt <= 5; attempt++) {
+ now.mockReturnValue(start + (attempt - 1) * 60_000);
+ callbacks.onReconnecting?.(attempt, 60_000);
+ }
+ const secondEpisode = start + 60 * 60_000;
+ for (let attempt = 1; attempt <= 5; attempt++) {
+ now.mockReturnValue(secondEpisode + (attempt - 1) * 20_000);
+ callbacks.onReconnecting?.(attempt, 20_000);
+ }
+ } finally {
+ now.mockRestore();
+ }
+
+ expect(sessionErrors.get('sess-uuid-1')).toBe(
+ 'Reconnecting after a dropped connection (attempt 5, down for 80s).',
+ );
+ });
});
// A restriction is WhatsApp judging the account, not a fault on our side, so it has its own
@@ -3403,6 +3716,25 @@ describe('SessionService', () => {
await expect(service.getPresence('sess-uuid-1', '[email protected]')).resolves.toBeNull();
});
+
+ // Presence belongs to the connection: once no engine is registered, the last report is not current.
+ it('reports null once the session is stopped', async () => {
+ const callbacks = await startAndCapture();
+ callbacks.onPresenceUpdate?.(presence('composing'));
+
+ await service.stop('sess-uuid-1');
+
+ await expect(service.getPresence('sess-uuid-1', '[email protected]')).resolves.toBeNull();
+ });
+
+ it('reports null once a terminal engine failure evicted the engine', async () => {
+ const callbacks = await startAndCapture();
+ callbacks.onPresenceUpdate?.(presence('composing'));
+
+ callbacks.onError?.(`${AUTH_FAILURE_REASON}: bad credentials`);
+
+ await expect(service.getPresence('sess-uuid-1', '[email protected]')).resolves.toBeNull();
+ });
});
// Three events rather than one carrying an outcome field, so a consumer that only cares about
diff --git a/src/modules/session/session.service.ts b/src/modules/session/session.service.ts
index de24b924a..f8a698538 100644
--- a/src/modules/session/session.service.ts
+++ b/src/modules/session/session.service.ts
@@ -121,6 +121,11 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
private autoStartRun: Promise<void> = Promise.resolve();
/** Set at the top of onModuleDestroy so the detached run stops launching further sessions. */
private shuttingDown = false;
+ /**
+ * stop()/delete() requests per session, counted so the transient start retry can tell a stop issued
+ * after it began from a mark left over by an earlier stop, which start() clears by design.
+ */
+ private readonly stopRequests = new Map<string, number>();
constructor(
@InjectRepository(Session, 'data')
@@ -454,11 +459,12 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
// post-INITIALIZING check by the time that write settles; awaiting anything first — the fence's
// COUNT, or delete()'s own requireSession — would let the mark land after that window. A mark
// left behind when the fence refuses (409) is harmless and is cleared by the next start().
- this.engineLifecycle.markStopping(id);
+ this.markStopping(id);
try {
if (this.ownership) await this.assertNotHeldElsewhere(id);
await this.engineLifecycle.delete(id);
await this.ownership?.release(id);
+ this.stopRequests.delete(id);
} catch (error) {
this.discardStopMarkForMissingSession(id, error);
throw error;
@@ -490,7 +496,14 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
* holder may be gone, and taking over is exactly what the claim rule allows.
*/
private async assertNotHeldElsewhere(id: string): Promise<void> {
- if (await this.ownership?.isHeldByOtherNode(id)) {
+ // Both callers set the stop mark before this query. A refusal keeps it (see delete()), but a
+ // query that failed decided nothing, and a mark left on a session running here would stop its
+ // next disconnect from reconnecting.
+ const heldElsewhere = await this.ownership?.isHeldByOtherNode(id).catch((error: unknown) => {
+ this.engineLifecycle.clearStopping(id);
+ throw error;
+ });
+ if (heldElsewhere) {
throw new ConflictException(`Session ${id} is running on another node`);
}
}
@@ -504,8 +517,9 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
await this.findOne(id);
throw new ConflictException(`Session ${id} is running on another node`);
}
+ let session: Session;
try {
- return await this.startWithTransientRetry(id);
+ session = await this.startWithTransientRetry(id);
} catch (error) {
// A failed or refused start must not leave the claim pinned here — the heartbeat would renew
// it and the session could never be started anywhere else. Released only when nothing is
@@ -514,6 +528,11 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
await this.releaseUnlessEngineActive(id);
throw error;
}
+ // A start retired by a concurrent stop() resolves normally but leaves no engine, and that stop
+ // skipped its release while this start still held the session. Hand the claim back here, or the
+ // row keeps naming this node until the lease lapses and a peer adopts the stopped session.
+ await this.releaseUnlessEngineActive(id);
+ return session;
}
/**
@@ -529,6 +548,7 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
* not-ready, 4xx) are NOT transient - they propagate immediately.
*/
private async startWithTransientRetry(id: string): Promise<Session> {
+ const stopRequestsBefore = this.stopRequests.get(id);
try {
return await this.engineLifecycle.start(id);
} catch (error) {
@@ -539,6 +559,10 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
error: error instanceof Error ? error.message : String(error),
});
await setTimeout(SESSION_START_RETRY_DELAY_MS);
+ // Retrying would bring back a session that a stop() issued since this start began just took
+ // down. A mark alone does not prove that: an earlier stop's mark survives until start() clears
+ // it, and a first attempt failing before that point would otherwise lose its retry.
+ if (this.stopRequests.get(id) !== stopRequestsBefore) throw error;
// The lease may have lapsed while the first attempt ran; the retry must keep holding the
// claim, never 409 on the session it already owns.
if (this.ownership && !(await this.ownership.claim(id))) {
@@ -551,7 +575,7 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
async stop(id: string): Promise<Session> {
// Synchronous stop-mark before the awaited fence — see delete() for why.
- this.engineLifecycle.markStopping(id);
+ this.markStopping(id);
let session: Session;
try {
if (this.ownership) await this.assertNotHeldElsewhere(id);
@@ -582,9 +606,10 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
await this.releaseUnlessEngineActive(id);
return session;
} catch (error) {
- // The 502-incomplete path tears the engine down too, and a "not started" refusal never had
- // one — either way a claim that no longer covers an engine must not survive the call.
- await this.releaseUnlessEngineActive(id);
+ // The 502-incomplete path tears the engine down, so its claim must not survive the call. A 400
+ // "not started" refusal changed nothing and keeps whatever claim there is: release() also
+ // clears a LAPSED foreign claim, which would take a crashed node's session out of takeover.
+ if (!(error instanceof BadRequestException)) await this.releaseUnlessEngineActive(id);
throw error;
}
}
@@ -595,11 +620,17 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
await this.releaseUnlessEngineActive(id);
return session;
} catch (error) {
- await this.releaseUnlessEngineActive(id);
+ // Same 400 rule as logout().
+ if (!(error instanceof BadRequestException)) await this.releaseUnlessEngineActive(id);
throw error;
}
}
+ private markStopping(id: string): void {
+ this.stopRequests.set(id, (this.stopRequests.get(id) ?? 0) + 1);
+ this.engineLifecycle.markStopping(id);
+ }
+
/** Hand the claim back unless something still runs here (engine, in-flight start, pending reconnect). */
private async releaseUnlessEngineActive(id: string): Promise<void> {
if (!this.ownership || this.engineLifecycle.isEngineActive(id)) {
@@ -721,6 +752,9 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat
*/
async getPresence(id: string, chatId: string): Promise<ChatPresence | null> {
await this.findOne(id);
+ // Presence belongs to a connection: with no engine registered (stopped, logged out, killed or
+ // failed) whatever was last reported is no longer current.
+ if (!this.engines.has(id)) return null;
return this.presence.get(id, chatId);
}