Free, Open Source, Self-Hosted WhatsApp API Gateway
TypeScript apibotgatewayself-hostedwhatsappwhatsapp-apiwhatsapp-automationwhatsapp-botwhatsapp-gatewaywhatsapp-gateway-apiwhatsapp-webwhatsapp-web-api
fix(session): preserve stops across ownership races
Yudhi Armyndharisa6ae77e9c3a9
diff --git a/docs/06-api-specification.md b/docs/06-api-specification.mdindex 6c872e3e6..3abd730ff 100644--- a/docs/06-api-specification.md+++ b/docs/06-api-specification.md@@ -673,6 +673,8 @@ Returned via `transformSession`. Status typically transitions to `initializing`A session stopped with `POST /stop` or `POST /force-kill` is marked to run again only once the start gets past the `400` and `409` refusals below; a refused start leaves the stop in place.+On multiple nodes, an explicit start may clear a peer's stop recorded before it successfully claimed ownership. It checks ownership and the stop record again after the start hook, before creating an engine; a lost claim or failed renewal answers `409` with no engine launched. A stop or claim loss during engine initialization retires that engine when initialization finishes.+**Errors:** `400` session already started / already starting, or this node already holds `MAX_CONCURRENT_SESSIONS` slots, counting running and starting engines and sessions waiting to relaunch after a failed reconnect (`Maximum concurrent sessions reached (N)`, answered only after the `404` and the `409` for a session running on another node; the session is not started, so retry once a slot frees or on a node with room) · `401` · `403` · `404` not found · `409` credential teardown for the same session name still in flight (retryable; body carries `code: 'SESSION_NAME_TEARDOWN_PENDING'`; no destructive side effect runs before the refusal — a retry after cleanup settles proceeds), or another node currently holds this session's engine (refused before any engine is launched), or, with no `code`, a `POST /stop`, a `POST /force-kill` that found a running engine to kill, or a data import (`stopOrphans`) of this session that began or finished while the start was waiting, or a delete of it still running or sent while the start was waiting and failed once it began removing (the start yields and launches nothing; after a stop, a force-kill or a delete that failed part-way a new `POST /start` clears it and starts the session, after a delete or an import that removed the session the retry answers `404`) · `503` the gateway is shutting down, or the engine's browser or socket connection died during launch and the one automatic retry failed too; no engine is left running, so the start can be retried · `504` the engine did not finish starting within its timeout (WhatsApp Web or the network unreachable, a stalled browser or resource limit, or, on whatsapp-web.js, an unreachable `proxyUrl`); the engine is torn down, so the start can be retried once the cause is addressed#### POST /api/sessions/:sessionId/stop@@ -710,7 +712,9 @@ No request body.Returned via `transformSession`; status typically becomes `disconnected`.-A stopped session stays down: boot auto-start (`AUTO_START_SESSIONS`) and the multi-node takeover sweep skip it until an explicit `POST /start`. The stop is recorded before the teardown, so a `502` stop keeps it too; a `409` refusal records nothing.+A stopped session stays down: boot auto-start (`AUTO_START_SESSIONS`) and the multi-node takeover sweep skip it until an explicit `POST /start`. The stop is recorded before the teardown, so a `502` stop keeps it too; a `409` refusal records nothing. Ownership is checked again in the stop write: if a live peer claimed the session after the initial check, the stop answers `409` without touching its engine or record. A stop that already passed the initial check can still replace a pending delete's record; a concurrent start then yields rather than clearing it.++An accepted stop on an expired or missing foreign lease clears that claim in the same write. If the former owner still runs, its next successful heartbeat detects lease loss and retires its engine. The stop response does not wait for that remote teardown.**Errors:** `401` · `403` · `404` not found · `409` another node currently holds this session's live engine (multi-node deployments) · `502` `SESSION_STOP_INCOMPLETE` — session stopped locally but the engine teardown did not complete (retryable; the graceful disconnect and the force-destroy escalation both failed, status `disconnected`, no success audit)diff --git a/docs/13-horizontal-scaling.md b/docs/13-horizontal-scaling.mdindex 1746bbe2a..400d14aab 100644--- a/docs/13-horizontal-scaling.md+++ b/docs/13-horizontal-scaling.md@@ -52,6 +52,8 @@> died with its node, is never adopted, even when its claim did lapse, until an explicit `POST /start`.> While versions are mixed during an upgrade, a node still on 0.25.0 does not read a `DELETE`'s record:> it can adopt such a session, and a `POST /start` it serves leaves the record in place.+> Older nodes also lack the final ownership check on a stop write, so they can still report a stop+> as successful while a peer starts the session. Upgrade every node before relying on that fence.>> Known limitation: a holder that is alive but cannot reach the database for more than three lease> TTLs minus one heartbeat (160s at defaults) is marked disconnected by a peer too, and does not write its status back when it reconnects. Itsdiff --git a/src/modules/session/session-delete-claim.spec.ts b/src/modules/session/session-delete-claim.spec.tsindex 7cfb54935..a63dca889 100644--- a/src/modules/session/session-delete-claim.spec.ts+++ b/src/modules/session/session-delete-claim.spec.ts@@ -7,6 +7,7 @@ import { mkdtempSync, rmSync } from 'fs';import { tmpdir } from 'os';import { join } from 'path';import { DataSource, DataSourceOptions } from 'typeorm';+import { postgresUtcExtra } from '../../database/postgres-utc';import { SessionService } from './session.service';import { SessionEngineLifecycle } from './session-engine-lifecycle.service';import { SessionErrorStore } from './session-error-store.service';@@ -66,6 +67,7 @@ const makeEngine = () => ({destroy: jest.fn().mockResolvedValue(undefined),forceDestroy: jest.fn().mockResolvedValue(undefined),disconnect: jest.fn().mockResolvedValue(undefined),+ logout: jest.fn().mockResolvedValue(undefined),getQRCode: jest.fn().mockReturnValue(null),});type FakeEngine = ReturnType<typeof makeEngine>;@@ -171,6 +173,9 @@ describe('A session deleted on one node while a peer shares the database', () =>const ownership = module.get(SessionOwnershipService);// What SessionService wires at bootstrap, without the interval: renew() asks whether anything still runs.ownership.setEngineLiveness(id => lifecycle.isEngineActive(id));+ ownership.onLeaseLoss(async ids => {+ await lifecycle.stopOrphanEngines(ids);+ });return { db, service: module.get(SessionService), lifecycle, ownership, engines, hooks, purge };}@@ -185,6 +190,7 @@ describe('A session deleted on one node while a peer shares the database', () =>database: process.env.DATABASE_NAME || 'openwa',schema: `session_delete_claim_${randomUUID().replace(/-/g, '')}`,entities,+ extra: postgresUtcExtra(),};const setup = await new DataSource(options).initialize();await setup.query(`CREATE SCHEMA "${options.schema}"`);@@ -270,6 +276,687 @@ describe('A session deleted on one node while a peer shares the database', () =>};}+ it.each([+ ['before the start clears its stop', 'clear-before', 'stopped'],+ ['after the start clears its stop', 'clear-after', 'stopped'],+ ['before launch', 'hook', null],+ ['during launch', 'initialize', null],+ ['after the start returns', 'returned', null],+ ['before a retried launch', 'retry', null],+ ] as const)('refuses a stale peer stop %s once a live owner has claimed', async (_case, phase, desiredState) => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState });+ const claiming = deferred();+ const claimGo = deferred();+ const checked = deferred();+ const stopGo = deferred();+ const reached = deferred();+ const go = deferred();+ const claim = a.ownership.claim.bind(a.ownership);+ jest.spyOn(a.ownership, 'claim').mockImplementationOnce(async next => {+ claiming.resolve();+ await claimGo.promise;+ return claim(next);+ });+ const held = b.ownership.isHeldByOtherNode.bind(b.ownership);+ jest.spyOn(b.ownership, 'isHeldByOtherNode').mockImplementationOnce(async next => {+ const answer = await held(next);+ checked.resolve();+ await stopGo.promise;+ return answer;+ });+ const sessions = a.db.getRepository(Session);+ if (phase.startsWith('clear')) {+ const update = sessions.update.bind(sessions);+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState !== null) return update(where, patch);+ if (phase === 'clear-before') {+ reached.resolve();+ await go.promise;+ }+ const result = await update(where, patch);+ if (phase === 'clear-after') {+ reached.resolve();+ await go.promise;+ }+ return result;+ });+ } else if (phase === 'initialize') {+ const engines = (a.lifecycle as unknown as { engines: EngineRegistry }).engines;+ const register = engines.set.bind(engines);+ jest.spyOn(engines, 'set').mockImplementation((next, engine) => {+ (engine as unknown as FakeEngine).initialize.mockImplementation(async () => {+ reached.resolve();+ await go.promise;+ });+ return register(next, engine);+ });+ } else if (phase !== 'returned') {+ a.hooks.execute.mockImplementation(async (event: string, data: unknown) => {+ if (event === 'session:starting') {+ reached.resolve();+ await go.promise;+ }+ return { continue: true, data };+ });+ if (phase === 'retry') {+ jest.spyOn(sessions, 'findOne').mockRejectedValueOnce(new Error('Connection terminated unexpectedly'));+ }+ }+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ let stopping: Promise<unknown> | undefined;+ try {+ await claiming.promise;+ stopping = b.service.stop(id).catch((error: unknown) => error);+ await checked.promise;+ claimGo.resolve();+ if (phase === 'returned') await starting;+ else await reached.promise;+ stopGo.resolve();++ expect(await stopping).toBeInstanceOf(ConflictException);+ go.resolve();+ expect(await starting).not.toBeInstanceOf(Error);+ expect(a.service.getEngine(id)).toBe(a.engines[0]);+ expect(await sessions.findOneByOrFail({ id })).toMatchObject({ nodeId: 'node-a', desiredState: null });+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ expect((b.lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions.has(id)).toBe(false);+ expect(b.ownership.ownedIds()).toEqual([]);+ } finally {+ claimGo.resolve();+ stopGo.resolve();+ go.resolve();+ await Promise.allSettled([starting, stopping]);+ }+ });++ it.each(['final read', 'conditional clear'])(+ 'yields before launch when a peer replaces a deleting record before the %s',+ async when => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'deleting' });+ const claiming = deferred();+ const claimGo = deferred();+ const checked = deferred();+ const stopGo = deferred();+ const clearing = deferred();+ const clearGo = deferred();+ const claim = a.ownership.claim.bind(a.ownership);+ jest.spyOn(a.ownership, 'claim').mockImplementationOnce(async next => {+ claiming.resolve();+ await claimGo.promise;+ return claim(next);+ });+ const held = b.ownership.isHeldByOtherNode.bind(b.ownership);+ jest.spyOn(b.ownership, 'isHeldByOtherNode').mockImplementationOnce(async next => {+ const answer = await held(next);+ checked.resolve();+ await stopGo.promise;+ return answer;+ });+ const sessions = a.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ if (when === 'final read') {+ jest.spyOn(SessionLifecycleFences.prototype, 'awaitPendingTeardown').mockImplementationOnce(async () => {+ clearing.resolve();+ await clearGo.promise;+ });+ } else {+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState === null) {+ clearing.resolve();+ await clearGo.promise;+ }+ return update(where, patch);+ });+ }+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ let stopping: Promise<unknown> | undefined;+ try {+ await claiming.promise;+ stopping = b.service.stop(id);+ await checked.promise;+ claimGo.resolve();+ await clearing.promise;+ stopGo.resolve();+ await stopping;+ expect(await sessions.findOneByOrFail({ id })).toMatchObject({ nodeId: 'node-a', desiredState: 'stopped' });+ clearGo.resolve();++ expect(await starting).toBeInstanceOf(SessionStoppedException);+ expect(a.engines).toHaveLength(0);+ expect(a.hooks.execute.mock.calls.some(([event]: [string]) => event === 'session:starting')).toBe(false);+ expect((await sessions.findOneByOrFail({ id })).desiredState).toBe('stopped');+ expect(a.ownership.ownedIds()).toEqual([]);+ expect(leftovers(id).stopCount).toBe(false);+ } finally {+ claimGo.resolve();+ stopGo.resolve();+ clearGo.resolve();+ await Promise.allSettled([starting, stopping]);+ }+ },+ );++ it.each(['before', 'after'])('keeps a stop on the same owner sent %s the start clears its record', async when => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ const reached = deferred();+ const go = deferred();+ const sessions = a.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState !== null) return update(where, patch);+ if (when === 'before') {+ reached.resolve();+ await go.promise;+ }+ const result = await update(where, patch);+ if (when === 'after') {+ reached.resolve();+ await go.promise;+ }+ return result;+ });+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ try {+ await reached.promise;+ await a.service.stop(id);+ go.resolve();+ await starting;+ expect(a.service.getEngine(id)).toBeUndefined();+ expect(await sessions.findOneByOrFail({ id })).toMatchObject({ desiredState: 'stopped', nodeId: null });+ expect(leftovers(id).stopCount).toBe(false);+ } finally {+ go.resolve();+ await starting;+ }+ });++ it('answers not found before launch if the session is removed during the conditional clear', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ const clearing = deferred();+ const go = deferred();+ const sessions = a.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState === null) {+ clearing.resolve();+ await go.promise;+ }+ return update(where, patch);+ });+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ try {+ await clearing.promise;+ await a.service.delete(id);+ go.resolve();+ expect(await starting).toBeInstanceOf(NotFoundException);+ expect(a.engines).toHaveLength(0);+ expect(a.hooks.execute.mock.calls.some(([event]: [string]) => event === 'session:starting')).toBe(false);+ expect(a.ownership.ownedIds()).toEqual([]);+ expect(leftovers(id).stopCount).toBe(false);+ } finally {+ go.resolve();+ await starting;+ }+ });++ it.each([+ ['unclaimed with a retained lease', { nodeId: null, leaseExpiresAt: new Date(Date.now() + 60_000) }, null],+ ['expired', { nodeId: 'departed', leaseExpiresAt: lapsed() }, null],+ ['without a lease', { nodeId: 'departed', leaseExpiresAt: null }, null],+ ] as const)('records a stop when the previous owner is %s', async (_case, fields, nodeId) => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, ...fields });+ await b.service.stop(id);+ expect(await a.db.getRepository(Session).findOneByOrFail({ id })).toMatchObject({+ desiredState: 'stopped',+ nodeId,+ });+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ });++ it('records a stop after the former owner logged out and released its claim', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ await a.service.start(id, { explicit: true });+ await a.service.logout(id);+ await b.service.stop(id);+ expect(await a.db.getRepository(Session).findOneByOrFail({ id })).toMatchObject({+ desiredState: 'stopped',+ nodeId: null,+ });+ expect(a.service.getEngine(id)).toBeUndefined();+ });++ it('answers not found when a session disappears while its stop reads it', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ const sessions = b.db.getRepository(Session);+ const findOne = sessions.findOne.bind(sessions);+ jest.spyOn(sessions, 'findOne').mockImplementationOnce(async options => {+ const row = await findOne(options);+ await a.service.delete(id);+ return row;+ });+ await expect(b.service.stop(id)).rejects.toBeInstanceOf(NotFoundException);+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ expect((b.lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions.has(id)).toBe(false);+ });++ it('kills a stale local engine without recording a stop over its new live owner', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ await b.service.start(id, { explicit: true });+ await a.db.getRepository(Session).update(id, { leaseExpiresAt: lapsed() });+ await a.service.start(id, { explicit: true });++ await b.service.forceKill(id);++ expect(b.service.getEngine(id)).toBeUndefined();+ expect(a.service.getEngine(id)).toBe(a.engines[0]);+ expect(await a.db.getRepository(Session).findOneByOrFail({ id })).toMatchObject({+ desiredState: null,+ nodeId: 'node-a',+ });+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ });++ it.each([+ ['expired', 'before'],+ ['expired', 'after'],+ ['no lease', 'before'],+ ['no lease', 'after'],+ ['expired', 'initialize'],+ ['no lease', 'initialize'],+ ])('honors an accepted peer stop when the start lease is %s %s its clear', async (lease, when) => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ const claimReached = deferred(),+ claimGo = deferred();+ const peerReached = deferred(),+ peerGo = deferred();+ const clearReached = deferred(),+ clearGo = deferred();+ const claim = a.ownership.claim.bind(a.ownership);+ jest.spyOn(a.ownership, 'claim').mockImplementationOnce(async next => {+ claimReached.resolve();+ await claimGo.promise;+ return claim(next);+ });+ const peerCheck = b.ownership.isHeldByOtherNode.bind(b.ownership);+ jest.spyOn(b.ownership, 'isHeldByOtherNode').mockImplementationOnce(async next => {+ const held = await peerCheck(next);+ peerReached.resolve();+ await peerGo.promise;+ return held;+ });+ const repo = a.db.getRepository(Session);+ const update = repo.update.bind(repo);+ if (when === 'initialize') {+ const engines = (a.lifecycle as unknown as { engines: EngineRegistry }).engines;+ const register = engines.set.bind(engines);+ jest.spyOn(engines, 'set').mockImplementation((next, engine) => {+ (engine as unknown as FakeEngine).initialize.mockImplementation(async () => {+ clearReached.resolve();+ await clearGo.promise;+ });+ return register(next, engine);+ });+ } else {+ jest.spyOn(repo, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState !== null) return update(where, patch);+ if (when === 'before') {+ clearReached.resolve();+ await clearGo.promise;+ }+ const result = await update(where, patch);+ if (when === 'after') {+ clearReached.resolve();+ await clearGo.promise;+ }+ return result;+ });+ }+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ await claimReached.promise;+ const stopping = b.service.stop(id);+ try {+ await peerReached.promise;+ claimGo.resolve();+ await clearReached.promise;+ await repo.update(id, { leaseExpiresAt: lease === 'expired' ? lapsed() : null });+ peerGo.resolve();+ await stopping;+ clearGo.resolve();+ await starting;+ const row = await repo.findOneByOrFail({ id });+ expect(a.service.getEngine(id)).toBeUndefined();+ expect(row.desiredState).toBe('stopped');+ if (when !== 'initialize') expect(a.engines).toHaveLength(0);+ } finally {+ claimGo.resolve();+ peerGo.resolve();+ clearGo.resolve();+ await Promise.allSettled([starting, stopping]);+ }+ });++ it('does not create an engine for a session deleted by a peer during its start hook without ownership', async () => {+ Object.assign(a.service, { ownership: undefined });+ Object.assign(a.lifecycle, { ownership: undefined });+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ a.hooks.execute.mockImplementation(async (event: string, data: unknown) => {+ if (event === 'session:starting') await b.service.delete(id);+ return { continue: true, data };+ });++ await expect(a.service.start(id, { explicit: true })).rejects.toBeInstanceOf(NotFoundException);+ expect(a.engines).toHaveLength(0);+ expect(leftovers(id).stopCount).toBe(false);+ });++ it.each(['before', 'after'])(+ 'returns the inactive session when a local stop arrives %s the final admission write',+ async when => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ const reached = deferred();+ const go = deferred();+ const sessions = a.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ let admissions = 0;+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState !== null || ++admissions !== 2) return update(where, patch);+ if (when === 'before') {+ reached.resolve();+ await go.promise;+ }+ const result = await update(where, patch);+ if (when === 'after') {+ reached.resolve();+ await go.promise;+ }+ return result;+ });+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ try {+ await reached.promise;+ await a.service.stop(id);+ go.resolve();+ expect(await starting).toMatchObject({ id, status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ expect(a.engines).toHaveLength(0);+ expect(leftovers(id).stopCount).toBe(false);+ } finally {+ go.resolve();+ await starting;+ }+ },+ );++ it.each(['first write', 'final write', 'final return'])(+ 'preserves an accepted stop when the lease expires during the %s',+ async phase => {+ Object.assign(a.ownership, {+ configService: { get: (key: string) => ({ 'session.nodeId': 'node-a', 'session.leaseTtlMs': 200 })[key] },+ });+ const id = await seed({ status: SessionStatus.DISCONNECTED, desiredState: 'stopped' });+ const reached = deferred();+ const go = deferred();+ const recorded = deferred();+ const releaseGo = deferred();+ const release = b.ownership.release.bind(b.ownership);+ jest.spyOn(b.ownership, 'release').mockImplementationOnce(async next => {+ recorded.resolve();+ await releaseGo.promise;+ return release(next);+ });+ const sessions = a.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ let admissions = 0;+ let expires = new Date();+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState === null && ++admissions === (phase === 'first write' ? 1 : 2)) {+ const result = phase === 'final return' ? await update(where, patch) : undefined;+ expires = (await sessions.findOneByOrFail({ id })).leaseExpiresAt!;+ reached.resolve();+ await go.promise;+ if (result) return result;+ }+ return update(where, patch);+ });+ const starting = a.service.start(id, { explicit: true }).catch((error: unknown) => error);+ let stopping: Promise<unknown> | undefined;+ try {+ await reached.promise;+ await new Promise(resolve => setTimeout(resolve, Math.max(0, expires.getTime() - Date.now()) + 25));+ stopping = b.service.stop(id);+ await recorded.promise;+ go.resolve();+ await starting;+ expect(a.engines).toHaveLength(0);+ expect((await sessions.findOneByOrFail({ id })).desiredState).toBe('stopped');+ } finally {+ go.resolve();+ releaseGo.resolve();+ await Promise.allSettled([starting, stopping]);+ }+ },+ );++ it.each([+ ['live owner', 'node-a', 'live', 'stopped', true],+ ['live owner without a stop', 'node-a', 'live', null, true],+ ['changed stop record', 'node-a', 'live', 'deleting', false],+ ['foreign owner', 'node-b', 'live', 'stopped', false],+ ['released owner', null, 'live', 'stopped', false],+ ['expired owner', 'node-a', 'expired', 'stopped', false],+ ['owner without a lease', 'node-a', 'missing', 'stopped', false],+ ] as const)('admits only the recorded state of the %s', async (_case, nodeId, lease, desiredState, admitted) => {+ const leaseExpiresAt = lease === 'missing' ? null : new Date(Date.now() + (lease === 'live' ? 5_000 : -1_000));+ const id = await seed({ status: SessionStatus.DISCONNECTED, nodeId, leaseExpiresAt, desiredState });++ expect(await a.ownership.prepareStart(id, desiredState === null ? null : 'stopped')).toBe(admitted);++ const row = await a.db.getRepository(Session).findOneByOrFail({ id });+ expect(row.desiredState).toBe(admitted ? null : desiredState);+ expect(row.nodeId).toBe(nodeId);+ if (admitted) expect(row.leaseExpiresAt!.getTime()).toBeGreaterThan(leaseExpiresAt!.getTime());+ else expect(row.leaseExpiresAt).toEqual(leaseExpiresAt);+ });++ it.each(['foreign', 'expired', 'missing'] as const)(+ 'retires an initializing engine whose ownership becomes %s without a stop',+ async loss => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ const reached = deferred();+ const go = deferred();+ const engines = (a.lifecycle as unknown as { engines: EngineRegistry }).engines;+ const register = engines.set.bind(engines);+ jest.spyOn(engines, 'set').mockImplementation((next, engine) => {+ (engine as unknown as FakeEngine).initialize.mockImplementation(async () => {+ reached.resolve();+ await go.promise;+ });+ return register(next, engine);+ });+ const starting = a.service.start(id, { explicit: true });+ try {+ await reached.promise;+ await b.db.getRepository(Session).update(id, {+ nodeId: loss === 'foreign' ? 'node-b' : 'node-a',+ leaseExpiresAt: loss === 'foreign' ? new Date(Date.now() + 60_000) : loss === 'expired' ? lapsed() : null,+ });+ go.resolve();+ await starting;+ expect(a.service.getEngine(id)).toBeUndefined();+ expect(a.engines[0].destroy).toHaveBeenCalled();+ expect((await a.db.getRepository(Session).findOneByOrFail({ id })).desiredState).toBeNull();+ } finally {+ go.resolve();+ await starting;+ }+ },+ );++ it.each([+ ['initializing', 'expired'],+ ['started', 'expired'],+ ['initializing', 'missing'],+ ['started', 'missing'],+ ])('retires a %s engine whose lease is %s after a peer stop before renewal', async (phase, lease) => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ const reached = deferred();+ const go = deferred();+ const recorded = deferred();+ const releaseGo = deferred();+ const engines = (a.lifecycle as unknown as { engines: EngineRegistry }).engines;+ const register = engines.set.bind(engines);+ jest.spyOn(engines, 'set').mockImplementation((next, engine) => {+ (engine as unknown as FakeEngine).initialize.mockImplementation(async () => {+ reached.resolve();+ if (phase === 'initializing') await go.promise;+ });+ return register(next, engine);+ });+ const release = b.ownership.release.bind(b.ownership);+ jest.spyOn(b.ownership, 'release').mockImplementationOnce(async next => {+ recorded.resolve();+ await releaseGo.promise;+ return release(next);+ });+ const starting = a.service.start(id, { explicit: true });+ let stopping: Promise<unknown> | undefined;+ try {+ if (phase === 'started') await starting;+ else await reached.promise;+ await a.db.getRepository(Session).update(id, { leaseExpiresAt: lease === 'expired' ? lapsed() : null });+ stopping = b.service.stop(id);+ await recorded.promise;+ await a.ownership.renew();+ releaseGo.resolve();+ await stopping;+ go.resolve();+ await starting;+ expect(a.service.getEngine(id)).toBeUndefined();+ expect((await a.db.getRepository(Session).findOneByOrFail({ id })).desiredState).toBe('stopped');+ } finally {+ go.resolve();+ releaseGo.resolve();+ await Promise.allSettled([starting, stopping]);+ }+ });++ it('refuses an expired-owner stop when a heartbeat renews before the conditional release', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED });+ await a.service.start(id, { explicit: true });+ await a.db.getRepository(Session).update(id, { leaseExpiresAt: lapsed() });+ const reached = deferred();+ const go = deferred();+ const sessions = b.db.getRepository(Session);+ const update = sessions.update.bind(sessions);+ jest.spyOn(sessions, 'update').mockImplementation(async (where, patch) => {+ if (patch.desiredState === 'stopped' && patch.nodeId === null) {+ reached.resolve();+ await go.promise;+ }+ return update(where, patch);+ });+ const stopping = b.service.stop(id).catch((error: unknown) => error);+ try {+ await reached.promise;+ await a.ownership.renew();+ go.resolve();+ expect(await stopping).toBeInstanceOf(ConflictException);+ expect(a.service.getEngine(id)).toBe(a.engines[0]);+ expect(await sessions.findOneByOrFail({ id })).toMatchObject({ nodeId: 'node-a', desiredState: null });+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ expect((b.lifecycle as unknown as { stoppingSessions: Set<string> }).stoppingSessions.has(id)).toBe(false);+ } finally {+ go.resolve();+ await stopping;+ }+ });++ (pg ? it : it.skip).each(['stop', 'claim'])(+ 'yields admission after an accepted peer %s commits while it waits for a row lock',+ async action => {+ const expires = new Date(Date.now() + 500);+ const id = await seed({+ status: SessionStatus.DISCONNECTED,+ desiredState: 'stopped',+ nodeId: 'node-a',+ leaseExpiresAt: expires,+ });+ const [{ pid }] = await a.db.query<Array<{ pid: number }>>('SELECT pg_backend_pid() AS pid');+ const lock = b.db.createQueryRunner();+ const sessions = b.db.getRepository(Session);+ const manager = sessions.manager;+ await lock.connect();+ await lock.startTransaction();+ await lock.manager.getRepository(Session).findOne({ where: { id }, lock: { mode: 'pessimistic_write' } });+ const preparing = a.ownership.prepareStart(id, 'stopped');+ try {+ let waiting = false;+ for (let i = 0; i < 100; i++) {+ const [activity] = await b.db.query<Array<{ wait_event_type: string | null }>>(+ 'SELECT wait_event_type FROM pg_stat_activity WHERE pid=$1',+ [pid],+ );+ if (activity?.wait_event_type === 'Lock') {+ waiting = true;+ break;+ }+ await new Promise(resolve => setTimeout(resolve, 10));+ }+ expect(waiting).toBe(true);+ await new Promise(resolve => setTimeout(resolve, Math.max(0, expires.getTime() - Date.now()) + 25));+ // Run the real peer operation on the connection holding the row. Its commit changes the+ // tuple the waiting UPDATE must recheck, rather than merely releasing an unchanged lock.+ Object.assign(sessions, { manager: lock.manager });+ if (action === 'stop') await b.service.stop(id);+ else expect(await b.ownership.claim(id)).toBe(true);+ await lock.commitTransaction();++ expect(await preparing).toBe(false);+ expect(await a.db.getRepository(Session).findOneByOrFail({ id })).toMatchObject({+ nodeId: action === 'stop' ? null : 'node-b',+ desiredState: 'stopped',+ });+ expect((b.service as unknown as { stopRequests: Map<string, unknown> }).stopRequests.has(id)).toBe(false);+ } finally {+ Object.assign(sessions, { manager });+ if (lock.isTransactionActive) await lock.rollbackTransaction();+ await lock.release();+ await preparing;+ }+ },+ );++ it('settles abandoned work when an accepted stop clears an expired foreign claim', async () => {+ const id = await seed({ status: SessionStatus.DISCONNECTED, nodeId: 'departed', leaseExpiresAt: lapsed() });+ const settled = jest.fn().mockResolvedValue(undefined);+ b.ownership.onAdoption(settled);++ await b.service.stop(id);+ await new Promise(resolve => setImmediate(resolve));++ expect(settled).toHaveBeenCalledWith(id);+ expect(settled).toHaveBeenCalledTimes(1);+ expect(await b.db.getRepository(Session).findOneByOrFail({ id })).toMatchObject({+ desiredState: 'stopped',+ nodeId: null,+ });+ });++ it.each(['released', 'same owner'])('does not settle a %s claim as abandoned foreign work', async owner => {+ const id = await seed({+ status: SessionStatus.DISCONNECTED,+ nodeId: owner === 'released' ? null : 'node-b',+ leaseExpiresAt: lapsed(),+ });+ const settled = jest.fn().mockResolvedValue(undefined);+ b.ownership.onAdoption(settled);++ expect(await b.ownership.stopExpiredForeignClaim(id)).toBe(false);+ await new Promise(resolve => setImmediate(resolve));++ expect(settled).not.toHaveBeenCalled();+ expect((await b.db.getRepository(Session).findOneByOrFail({ id })).desiredState).toBeNull();+ });+const gone = { sessions: 0, messages: 0, outbox: 0, deliveryFailures: 0 };it.each([diff --git a/src/modules/session/session-engine-controls.ts b/src/modules/session/session-engine-controls.tsindex a284ca683..309daccbd 100644--- a/src/modules/session/session-engine-controls.ts+++ b/src/modules/session/session-engine-controls.ts@@ -144,11 +144,13 @@ export interface StopHooks {afterRead?: () => Promise<void>;/*** Runs when the first session read fails (for stop() and delete(), after the stop mark is dropped):- * undoes the caller's request count.+ * withdraws the caller's stop request.*/onReadFailed?: () => void;/** forceKill() only: runs synchronously once there is an engine to kill; records the stop request. */onEngineFound?: () => void;+ /** forceKill() only: persist the stop with the caller's current ownership predicate. */+ recordStop?: () => Promise<void>;/** delete() only: runs once the delete starts removing the session's data, the session:deleted hook first. */onRemoving?: () => void;}@@ -184,6 +186,7 @@ export interface SessionEngineControlsHost {cancelReconnect(id: string): void;initializeEngine(id: string, session: Session): Promise<void>;isSessionRetired(id: string): Promise<boolean>;+ prepareStart(id: string, desiredState: Session['desiredState']): Promise<boolean>;purgeAuthDirsIfDeleted(id: string): Promise<void>;updateStatus(id: string, status: SessionStatus): Promise<void>;/** Ownership gate, same contract as SessionEngineWiringHost.ownsSession. */@@ -394,7 +397,7 @@ export class SessionEngineControls {// counts as the start the operator asked for. A delete issued since this start began that failed once it// began removing has dropped its mark and left a record this start may have read too: it still stands.const { desiredState } = await this.requireSession(id);- if ((desiredState && (!explicit || !session.desiredState)) || stoppedSince?.()) {+ if ((desiredState && (!explicit || desiredState !== session.desiredState)) || stoppedSince?.()) {this.stoppingSessions.add(id);throw new SessionStoppedException(`Session ${id} was stopped`);}@@ -410,13 +413,14 @@ export class SessionEngineControls {if (this.host.isClosed()) {throw closedRefusal();}- if (desiredState) {- await this.sessionRepository.update({ id, desiredState }, { desiredState: null });- // The clear stands (this start was the operator's), but the hook must not fire for a start- // shutdown overtook during the write.- if (this.host.isClosed()) {- throw closedRefusal();- }+ if (!(await this.host.prepareStart(id, desiredState))) {+ await this.requireSession(id);+ this.stoppingSessions.add(id);+ throw new SessionStoppedException(`Session ${id} was stopped`);+ }+ // The clear stands, but a shutdown during the write must not let the hook run.+ if (this.host.isClosed()) {+ throw closedRefusal();}// Execute hook before starting@@ -429,6 +433,22 @@ export class SessionEngineControls {},);+ // A local teardown already retired this launch; preserve its normal inactive-session response.+ if (this.stoppingSessions.has(id)) return this.requireSession(id);++ // The hook may outlive the lease. Renew this claim only if no stop or owner has replaced it,+ // then register synchronously: a peer stop either changes the row first, or sees the live renewal.+ const prepared = await this.host.prepareStart(id, null);+ // A local stop can arrive during admission too. It owns the inactive response even when its+ // write made admission fail; a delete may already have dropped its mark but retain its request.+ if (this.stoppingSessions.has(id) || stoppedSince?.()) return this.requireSession(id);+ if (!prepared) {+ await this.requireSession(id);+ this.stoppingSessions.add(id);+ throw new SessionStoppedException(`Session ${id} was stopped`);+ }+ if (this.host.isClosed()) throw closedRefusal();+// Initialize reconnect state from the (untrusted) opaque session.config — coerced + clamped// so a poisoned value can't drive a NaN/immediate-relaunch storm or an unbounded loop.const { maxAttempts, baseDelay } = resolveReconnectConfig(session.config);@@ -753,15 +773,15 @@ export class SessionEngineControls {// explicit start. Recorded only once there is an engine to kill, and before the teardown, so a// POST /start that lands after the eviction clears it and stands. A failed write must not keep a// wedged engine alive, so it is logged and the kill goes on.- await this.sessionRepository- .update(id, { desiredState: 'stopped' })- .catch((error: unknown) =>- this.logger.error(- 'Failed to record the force-kill as a stop',- error instanceof Error ? error.message : String(error),- { sessionId: id, action: 'force_kill_keep_down_failed' },- ),- );+ const write = (): Promise<unknown> =>+ hooks.recordStop ? hooks.recordStop() : this.sessionRepository.update(id, { desiredState: 'stopped' });+ await write().catch((error: unknown) =>+ this.logger.error(+ 'Failed to record the force-kill as a stop',+ error instanceof Error ? error.message : String(error),+ { sessionId: id, action: 'force_kill_keep_down_failed' },+ ),+ );// Announced by the write below rather than from the engine callback, as in stop().this.operatorTeardowns.set(id, engine);diff --git a/src/modules/session/session-engine-lifecycle.service.ts b/src/modules/session/session-engine-lifecycle.service.tsindex ebdc080b3..093ea3e3d 100644--- a/src/modules/session/session-engine-lifecycle.service.ts+++ b/src/modules/session/session-engine-lifecycle.service.ts@@ -1,7 +1,7 @@import { Injectable, HttpException, HttpStatus, Optional } from '@nestjs/common';import { ConfigService } from '@nestjs/config';import { InjectRepository, InjectDataSource } from '@nestjs/typeorm';-import { Repository, DataSource } from 'typeorm';+import { Repository, DataSource, IsNull } from 'typeorm';import { Session, SessionStatus } from './entities/session.entity';import { EngineFactory } from '../../engine/engine.factory';import { EngineRegistry } from '../../engine/engine-registry.service';@@ -438,6 +438,7 @@ export class SessionEngineLifecycle {cancelReconnect: id => this.cancelReconnect(id),initializeEngine: (id, session) => this.initializeEngine(id, session),isSessionRetired: id => this.isSessionRetired(id),+ prepareStart: (id, desiredState) => this.prepareStart(id, desiredState),purgeAuthDirsIfDeleted: id => this.purgeAuthDirsIfDeleted(id),updateStatus: (id, status) => this.updateStatus(id, status),stoppingSessions: this.stoppingSessions,@@ -1292,18 +1293,33 @@ export class SessionEngineLifecycle {}, delay);}+ private prepareStart(id: string, desiredState: Session['desiredState']): Promise<boolean> {+ if (this.ownership) return this.ownership.prepareStart(id, desiredState);+ return this.sessionRepository+ .update({ id, desiredState: desiredState ?? IsNull() }, { desiredState: null })+ .then(({ affected }) => affected !== 0);+ }+/**- * True once a session must stay down: it is explicitly marked tearing-down, or it was deleted- * outright while a slow engine.initialize() was in flight. delete() clears its `stoppingSessions`- * mark in its finally (ms) and removes the session row well before a Chromium launch resolves, so- * the mark alone can't catch a delete that raced a (re)connect — the session row is the source of- * truth a post-init guard must re-check before keeping the engine it just created.+ * A start or reconnect cannot keep an engine after a stop, deletion or loss of its live claim.+ * A peer writes no local stop mark, and delete() drops its own mark before a slow init resolves,+ * so the post-init decision also reads the durable record and the current claim.*/private async isSessionRetired(id: string): Promise<boolean> {if (this.stoppingSessions.has(id)) {return true;}- return (await this.sessionRepository.findOne({ where: { id } })) == null;+ const session = await this.sessionRepository.findOne({ where: { id } });+ return (+ session == null ||+ !!session.desiredState ||+ !!(+ this.ownership &&+ (session.nodeId !== this.ownership.nodeId ||+ session.leaseExpiresAt == null ||+ session.leaseExpiresAt <= new Date())+ )+ );}/**diff --git a/src/modules/session/session-ownership.service.ts b/src/modules/session/session-ownership.service.tsindex d1bb20dcb..ebbb1b026 100644--- a/src/modules/session/session-ownership.service.ts+++ b/src/modules/session/session-ownership.service.ts@@ -2,7 +2,7 @@ import { Injectable, Optional } from '@nestjs/common';import { InjectRepository } from '@nestjs/typeorm';import { ConfigService } from '@nestjs/config';import { hostname } from 'node:os';-import { In, IsNull, LessThan, Repository } from 'typeorm';+import { In, IsNull, LessThan, LessThanOrEqual, MoreThan, Not, Repository } from 'typeorm';import { Session } from './entities/session.entity';import { createLogger } from '../../common/services/logger.service';import { DateTransformer } from '../../common/transformers/date.transformer';@@ -177,6 +177,40 @@ export class SessionOwnershipService {return claimed;}+ /** Renew this node's claim and clear only the record this start read; never reclaim a released row. */+ async prepareStart(sessionId: string, desiredState: Session['desiredState']): Promise<boolean> {+ const now = new Date();+ const leaseExpiresAt = new Date(now.getTime() + this.leaseTtlMs);+ const { affected } = await this.sessions.update(+ {+ id: sessionId,+ nodeId: this.nodeId,+ leaseExpiresAt: MoreThan(now),+ desiredState: desiredState ?? IsNull(),+ },+ { desiredState: null, leaseExpiresAt },+ );+ // Even a successful write can return too late to admit an engine on the renewal it wrote.+ if (affected !== 1 || leaseExpiresAt <= new Date()) return false;+ this.lastSeenLease.set(sessionId, leaseExpiresAt.getTime());+ return true;+ }++ /** Record a stop and release an abandoned foreign claim without letting its heartbeat revive it. */+ async stopExpiredForeignClaim(sessionId: string): Promise<boolean> {+ const foreign = { id: sessionId, nodeId: Not(this.nodeId) };+ const { affected } = await this.sessions.update(+ [+ { ...foreign, leaseExpiresAt: IsNull() },+ { ...foreign, leaseExpiresAt: LessThanOrEqual(new Date()) },+ ],+ { desiredState: 'stopped', nodeId: null, claimedAt: null, leaseExpiresAt: null, nodeUrl: null },+ );+ if (affected !== 1) return false;+ this.followAdoption(sessionId);+ return true;+ }+/*** Give a session up, so a peer can take it without waiting for the lease to lapse.*diff --git a/src/modules/session/session.service.spec.ts b/src/modules/session/session.service.spec.tsindex 9435ca200..fa7345432 100644--- a/src/modules/session/session.service.spec.ts+++ b/src/modules/session/session.service.spec.ts@@ -1827,16 +1827,24 @@ describe('SessionService', () => {// ── ownership claim lifecycle (multi-node) ────────────────────────describe('ownership release', () => {- type OwnershipStub = { claim: jest.Mock; release: jest.Mock; isHeldByOtherNode: jest.Mock };+ type OwnershipStub = {+ nodeId: string;+ claim: jest.Mock;+ release: jest.Mock;+ isHeldByOtherNode: jest.Mock;+ stopExpiredForeignClaim: jest.Mock;+ };// The trailing @Optional constructor dep the DI module deliberately omits; poked onto the// instance like the other white-box seams in this file.const withOwnership = (claimResult = true): OwnershipStub => {const ownership: OwnershipStub = {+ nodeId: 'node-a',claim: jest.fn().mockResolvedValue(claimResult),release: jest.fn().mockResolvedValue(undefined),// No live peer holds it unless a test says so — the single-node answer.isHeldByOtherNode: jest.fn().mockResolvedValue(false),+ stopExpiredForeignClaim: jest.fn().mockResolvedValue(false),};Object.assign(service as unknown as Record<string, unknown>, { ownership });return ownership;@@ -2964,6 +2972,11 @@ describe('SessionService', () => {() => (repository.update as jest.Mock).mockRejectedValueOnce(new Error('db down')),'db down',],+ [+ 'refused when its record is written',+ () => (repository.update as jest.Mock).mockResolvedValueOnce({ affected: 0 }),+ ConflictException,+ ],])('start() still retries a transient failure when a stop %s landed during the delay',async (_case, arrange, refusal) => {@@ -2987,22 +3000,26 @@ describe('SessionService', () => {// A refused stop withdraws only its own request: one that DID take the engine down still stands// when a later request is refused, so the retry it must cancel stays cancelled. Clearing every// request instead would relaunch the session the first stop took down and re-claim its row.- it('start() does not retry when an effective stop during the delay is followed by a refused one', async () => {- const row = await seed();- (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));- mockEngine.initialize.mockRejectedValueOnce(new EngineTransportError('Protocol error: Target closed'));+ it.each(['ownership read', 'record write'])(+ 'start() does not retry when an effective stop is followed by one refused at the %s',+ async refusal => {+ const row = await seed();+ (repository.findOne as jest.Mock).mockResolvedValue(createMockSession({ id: row.id }));+ mockEngine.initialize.mockRejectedValueOnce(new EngineTransportError('Protocol error: Target closed'));- const starting = service.start(row.id);- const outcome = starting.catch((error: unknown) => error);- await until(() => mockEngine.initialize.mock.calls.length === 1 && !lifecycle.isEngineActive(row.id));- await service.stop(row.id);- jest.spyOn(ownership, 'isHeldByOtherNode').mockResolvedValueOnce(true);- await expect(service.stop(row.id)).rejects.toThrow(ConflictException);+ const starting = service.start(row.id);+ const outcome = starting.catch((error: unknown) => error);+ await until(() => mockEngine.initialize.mock.calls.length === 1 && !lifecycle.isEngineActive(row.id));+ await service.stop(row.id);+ if (refusal === 'ownership read') jest.spyOn(ownership, 'isHeldByOtherNode').mockResolvedValueOnce(true);+ else (repository.update as jest.Mock).mockResolvedValueOnce({ affected: 0 });+ await expect(service.stop(row.id)).rejects.toThrow(ConflictException);- expect(await outcome).toBeInstanceOf(SessionStoppedException);- expect(mockEngine.initialize).toHaveBeenCalledTimes(1);- expect(lifecycle.isEngineActive(row.id)).toBe(false);- });+ expect(await outcome).toBeInstanceOf(SessionStoppedException);+ expect(mockEngine.initialize).toHaveBeenCalledTimes(1);+ expect(lifecycle.isEngineActive(row.id)).toBe(false);+ },+ );// A 400 changes nothing, and release() also clears a lapsed foreign claim: that took a crashed// node's session out of the lapsed-claim sweep for good.@@ -10056,9 +10073,11 @@ describe('SessionService', () => {const updates = (): unknown[][] => (repository.update as jest.Mock).mock.calls as unknown[][];const withOwnership = (over: Record<string, jest.Mock> = {}) => {const ownership = {+ nodeId: 'node-a',claim: jest.fn().mockResolvedValue(true),release: jest.fn().mockResolvedValue(undefined),isHeldByOtherNode: jest.fn().mockResolvedValue(false),+ stopExpiredForeignClaim: jest.fn().mockResolvedValue(false),...over,};Object.assign(service as unknown as Record<string, unknown>, { ownership });@@ -10785,6 +10804,7 @@ describe('SessionService', () => {claim: jest.fn().mockResolvedValue(true),release: jest.fn().mockResolvedValue(undefined),isHeldByOtherNode: jest.fn().mockResolvedValue(false),+ stopExpiredForeignClaim: jest.fn().mockResolvedValue(false),onLeaseLoss: jest.fn(),setEngineLiveness: jest.fn(),startHeartbeat: jest.fn(),diff --git a/src/modules/session/session.service.ts b/src/modules/session/session.service.tsindex 779100a85..9ad8b4c61 100644--- a/src/modules/session/session.service.ts+++ b/src/modules/session/session.service.ts@@ -1057,6 +1057,7 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicatconst session = await this.engineLifecycle.forceKill(id, {onReadFailed: withdraw,onEngineFound: () => this.takeStop(issue()),+ recordStop: () => this.keepDown(id),});await this.releaseAfterTeardown(id);return session;@@ -1072,11 +1073,26 @@ export class SessionService implements OnModuleDestroy, OnModuleInit, OnApplicat}/**- * Persist the operator's stop so boot auto-start and the takeover sweep leave the session down. A delete's- * take-back clears only its own 'deleting', so this needs no ordering against one.+ * Persist a stop only while no live peer owns the engine. The initial ownership read may predate a peer's+ * claim; deciding in this write also fences a stop delayed until after that peer's start returned. A delete's+ * record still accepts a stop: replacing 'deleting' cannot be mistaken for that record by a start or take-back.*/private async keepDown(id: string): Promise<void> {- await this.sessionRepository.update(id, { desiredState: 'stopped' });+ const where: string | FindOptionsWhere<Session>[] = this.ownership+ ? [+ { id, nodeId: IsNull() },+ { id, nodeId: this.ownership.nodeId },+ { id, desiredState: 'deleting' },+ ]+ : id;+ const { affected } = await this.sessionRepository.update(where, { desiredState: 'stopped' });+ if (affected === 0) {+ // Clear an abandoned foreign claim with the record, before its heartbeat can revive it. A+ // pending delete took the branch above and keeps its claim throughout removal.+ if (this.ownership && (await this.ownership.stopExpiredForeignClaim(id))) return;+ await this.findOne(id);+ throw new ConflictException(`Session ${id} is running on another node`);+ }}/**