Multiplayer agent harness for work.
Make swarm message, notification, spawn and lifetime budgets opt-in (#2406)
Joshua Franceaac880b7ecf3
diff --git a/docs/swarms.md b/docs/swarms.mdindex 649c98ca6..c28619c4e 100644--- a/docs/swarms.md+++ b/docs/swarms.md@@ -66,9 +66,11 @@ unavailable model fails closed at dispatch.Optional `settings` overrides the defaults below at initial spawn. The backendcan configure defaults with `SWARM_DEFAULTS`, a JSON object such as`{"agents":16,"turnMs":900000}`. Values must be positive safe integers within the-safety bounds; unknown settings are rejected. Resolved settings and the chosen-`backend` are stored once. Identical initial requests can be retried; subsequent-spawns inherit the settings and cannot reconfigure the swarm.+safety bounds; unknown settings are rejected. Resolved settings and the chosen `backend` are stored once. Identical+initial requests can be retried. Later spawns by the root session may pass+`settings` again to change any setting except `turnMs` (for example, to raise+`agents`); workers inherit the settings and cannot reconfigure the swarm. The+backend cannot change after creation.Omit `count` to spawn one; `contexts` also determines the count when provided. Asingle `context` supplies the same initial JSON to every worker. Context may be any@@ -108,7 +110,7 @@ never retires the existing forum.### Identity, peers, and character-`GET /v1/swarm` returns `id`, `self`, `peers`, `backend`, `settings`, and `expiresAt`. Peer+`GET /v1/swarm` returns `id`, `self`, `peers`, `backend`, `settings`, and `expiresAt` when a work window is set. Peerrecords keep trusted IDs, ancestry, session links, storage selection, and lifecyclestate separate from arbitrary `context` data. Only this swarm is listed, not everysession with the same owner, and never all private sessions in the organization.@@ -154,8 +156,8 @@ delivery target, and the delivery layer also refuses swarm result delivery.Set `notify:false` for a durable message without a wakeup. Sending to oneself doesnot wake oneself. There are no automatic reply-to-reply notifications: agents-explicitly decide whether a response warrants another turn. Every generated turn-consumes the same finite notification budget.+explicitly decide whether a response warrants another turn. When `notifications`+is set, every generated turn consumes that budget.### Ask, read, and tail@@ -234,25 +236,30 @@ refreshes every few seconds only while workers are active and the tab is visible| ------------------------------------------------ | -------------------- || `agents` (root and failed reservations included) | 32 || `depth` below root | 4 |-| `spawnRequests` | 32 |-| `messages` including initial work | 128 |-| `notifications` including initial work | 256 || `contextBytes` / `textBytes` | 8,192 each |-| `lifetimeMs` from first spawn | 3,600,000 |+| `spawnRequests` | unset (no limit) |+| `messages` including initial work | unset (no limit) |+| `notifications` including initial work | unset (no limit) |+| `lifetimeMs` from first spawn | unset (no limit) || `turnMs` per notification | 600,000 (10 minutes) || `waitMs` per read | 10,000 |The root and at least one worker require `agents >= 2`. All other settings acceptany positive integer; `turnMs` is limited only by the 2,147,483,647 ms timer ceiling. Read requests can still use `waitMs:0`.-Per-swarm budgets are reserved atomically with the messages and members consuming-them. Both the orchestrator and independent worker cancellation honor the stored-turn deadline; existing organization-wide limits can shorten it. Internal reconciliation limits, the 32-message read page, and-bounded run retry counts are not swarm configuration.--After expiration, history remains readable, but new work, notifications, and-character changes are rejected. Human-initiated ordinary session turns remain-ordinary QM actions. Workers and their durable results remain visible; computers-can be retired through the existing sandbox inventory. There is no automatic+`spawnRequests`, `messages`, `notifications`, and `lifetimeMs` are off unless a+spawn or `SWARM_DEFAULTS` sets them. When set, they are enforced as budgets that are+reserved atomically with the members and messages that consume them. Both the+orchestrator and independent worker cancellation honor the stored turn deadline;+existing organization-wide limits can shorten it. Internal reconciliation limits,+the 32-message read page, and bounded run retry counts are not swarm+configuration.++When `lifetimeMs` is set and the window passes, history remains readable, but new+work, notifications, and character changes are rejected until the root extends+`lifetimeMs` on a later spawn. Swarms without `lifetimeMs` never expire. Human-initiated+ordinary session turns remain ordinary QM actions.+Workers and their durable results remain visible; computers can be retired+through the existing sandbox inventory. There is no automaticfilesystem export on retirement: publish useful outputs through existing Filesor your version-control workflow first.diff --git a/plugins/web-ui/src/swarm-strip.ts b/plugins/web-ui/src/swarm-strip.tsindex 31d7c8a68..152ab3fd7 100644--- a/plugins/web-ui/src/swarm-strip.ts+++ b/plugins/web-ui/src/swarm-strip.ts@@ -18,7 +18,7 @@ interface SwarmView {self: SwarmPeer;peers: SwarmPeer[];board?: { sandboxId: string };- expiresAt: number;+ expiresAt?: number;}type Control = "active" | "paused" | "stopped";diff --git a/src/swarms/swarm-service.ts b/src/swarms/swarm-service.tsindex 91b108d47..ff32685e3 100644--- a/src/swarms/swarm-service.ts+++ b/src/swarms/swarm-service.ts@@ -18,6 +18,7 @@ import { errMessage, swallow } from "../util/errors.ts";import {assertSwarmOpen,SWARM_LIMITS,+ swarmExpired,type Swarm,type SwarmMember,type SwarmMessage,@@ -71,7 +72,7 @@ export interface SwarmService {board?: Swarm["board"];backend: Swarm["backend"];settings: SwarmSettings;- expiresAt: number;+ expiresAt?: number;}>;context(caller: SwarmCaller, context: unknown): Promise<SwarmMember>;spawn(caller: SwarmCaller, input: SpawnInput): Promise<SwarmMember[]>;@@ -331,7 +332,7 @@ export function createSwarmService(deps: {settings,backend: provider.name,createdAt,- expiresAt: createdAt + settings.lifetimeMs,+ ...(settings.lifetimeMs === undefined ? {} : { expiresAt: createdAt + settings.lifetimeMs }),...(forumSandboxId ? {} : { board: { sandboxId: randomUUID() } }),members: [{@@ -372,7 +373,7 @@ export function createSwarmService(deps: {if (!recipient || recipient.state === "reserved") continue;const control = controlState(swarm, recipient.id);if (control === "paused") continue;- if (recipient.state === "failed" || control === "stopped" || Date.now() >= swarm.expiresAt) {+ if (recipient.state === "failed" || control === "stopped" || swarmExpired(swarm)) {await step(() =>store.update(swarm.id, (current) => {current.messages.find((item) => item.id === message.id)!.notifications[recipientId] = { state: "failed" };@@ -483,12 +484,7 @@ export function createSwarmService(deps: {const failed = current.members.find((peer) => peer.id === member.id)!;if (failed.state !== "reserved") return;failed.error = errMessage(error).slice(0, 500);- if (- provisioningTimedOut ||- Date.now() >= deadline ||- failed.attempts >= 3 ||- Date.now() >= current.expiresAt- ) {+ if (provisioningTimedOut || Date.now() >= deadline || failed.attempts >= 3 || swarmExpired(current)) {failed.state = "failed";failed.cleanupPending = true;}@@ -579,7 +575,7 @@ export function createSwarmService(deps: {...(swarm.board ? { board: swarm.board } : {}),backend: swarm.backend,settings: swarm.settings,- expiresAt: swarm.expiresAt,+ ...(swarm.expiresAt === undefined ? {} : { expiresAt: swarm.expiresAt }),};},async context(caller, context) {@@ -601,9 +597,10 @@ export function createSwarmService(deps: {if (previous.signature !== fingerprint) throw new Error("requestId reused with different content");return existing!.members.filter((member) => previous.memberIds.includes(member.id)).map(view);}- if (existing && (input.settings !== undefined || input.backend !== undefined))- throw new Error("settings are only allowed on initial swarm creation");- const settings = existing?.settings ?? resolveSwarmSettings(input.settings, defaults);+ if (existing && input.backend !== undefined) throw new Error("backend is only allowed on initial swarm creation");+ if (existing && input.settings !== undefined && auth.memberId !== existing.id)+ throw new Error("only the root session can change swarm settings");+ const settings = resolveSwarmSettings(input.settings, existing?.settings ?? defaults);boundedText(input.text, settings.textBytes, "text");const requestedRuntime = spawnRuntime(input);const count = input.count ?? input.contexts?.length ?? 1;@@ -613,7 +610,8 @@ export function createSwarmService(deps: {const contexts = Array.from({ length: count }, (_value, index) =>jsonContext(input.contexts ? input.contexts[index] : defaultContext, settings.contextBytes),);- if (!existing && count > settings.notifications) throw new Error("swarm notification budget exhausted");+ if (!existing && count > (settings.notifications ?? Infinity))+ throw new Error("swarm notification budget exhausted");if (input.forumSandboxId) {const forum = await deps.sandboxes.access(auth.actorId, input.forumSandboxId);const session = await sessions.get(auth.sessionId);@@ -625,20 +623,27 @@ export function createSwarmService(deps: {if (previous.signature !== fingerprint) throw new Error("requestId reused with different content");return;}- if (- canonicalJson(settings) !== canonicalJson(swarm.settings) ||- (input.backend !== undefined && input.backend !== swarm.backend)- )- throw new Error("conflicting initial swarm settings");+ if (!existing) {+ if (+ canonicalJson(settings) !== canonicalJson(swarm.settings) ||+ (input.backend !== undefined && input.backend !== swarm.backend)+ )+ throw new Error("conflicting initial swarm settings");+ } else if (input.settings !== undefined) {+ const next = resolveSwarmSettings(input.settings, swarm.settings);+ if (next.turnMs !== swarm.settings.turnMs) throw new Error("turnMs cannot change after swarm creation");+ swarm.settings = next;+ if (next.lifetimeMs !== undefined) swarm.expiresAt = swarm.createdAt + next.lifetimeMs;+ }assertSwarmOpen(swarm);const parent = swarm.members.find((member) => member.id === auth.memberId)!;if (parent.depth >= swarm.settings.depth) throw new Error("swarm depth budget exhausted");if (swarm.members.length + count > swarm.settings.agents) throw new Error("swarm agent budget exhausted");- if (Object.keys(swarm.spawnRequests).length >= swarm.settings.spawnRequests)+ if (Object.keys(swarm.spawnRequests).length >= (swarm.settings.spawnRequests ?? Infinity))throw new Error("swarm spawn budget exhausted");if (- swarm.messages.length >= swarm.settings.messages ||- swarm.notificationCount + count > swarm.settings.notifications+ swarm.messages.length >= (swarm.settings.messages ?? Infinity) ||+ swarm.notificationCount + count > (swarm.settings.notifications ?? Infinity))throw new Error("swarm work budget exhausted");const runtime = inheritRuntime(parent.runtime, requestedRuntime);@@ -720,11 +725,12 @@ export function createSwarmService(deps: {return;}assertSwarmOpen(swarm);- if (swarm.messages.length >= swarm.settings.messages) throw new Error("swarm message budget exhausted");+ if (swarm.messages.length >= (swarm.settings.messages ?? Infinity))+ throw new Error("swarm message budget exhausted");if (input.replyTo && !swarm.messages.some((message) => message.id === input.replyTo))throw new Error("reply target is not in this swarm");const recipients = input.notify === false ? [] : audience.filter((peer) => peer !== auth.memberId);- if (swarm.notificationCount + recipients.length > swarm.settings.notifications)+ if (swarm.notificationCount + recipients.length > (swarm.settings.notifications ?? Infinity))throw new Error("swarm notification budget exhausted");swarm.notificationCount += recipients.length;swarm.messages.push({diff --git a/src/swarms/swarm-settings.ts b/src/swarms/swarm-settings.tsindex 5d387924b..807bd2722 100644--- a/src/swarms/swarm-settings.ts+++ b/src/swarms/swarm-settings.ts@@ -1,29 +1,29 @@export const SWARM_DEFAULTS = {agents: 32,depth: 4,- messages: 128,- notifications: 256,- spawnRequests: 32,contextBytes: 8_192,textBytes: 8_192,waitMs: 10_000,turnMs: 600_000,- lifetimeMs: 3_600_000,};-export type SwarmSettings = typeof SWARM_DEFAULTS;+const SWARM_CAPS = ["messages", "notifications", "spawnRequests", "lifetimeMs"] as const;++export type SwarmSettings = typeof SWARM_DEFAULTS & Partial<Record<(typeof SWARM_CAPS)[number], number>>;const MAX_TIMER_MS = 2_147_483_647;export function resolveSwarmSettings(input: unknown = {}, defaults: SwarmSettings = SWARM_DEFAULTS): SwarmSettings {if (!input || typeof input !== "object" || Array.isArray(input)) throw new Error("invalid swarm settings");for (const key of Object.keys(input)) {- if (!Object.hasOwn(SWARM_DEFAULTS, key)) throw new Error(`unknown swarm setting: ${key}`);+ if (!Object.hasOwn(SWARM_DEFAULTS, key) && !SWARM_CAPS.includes(key as never))+ throw new Error(`unknown swarm setting: ${key}`);}- const settings = { ...defaults, ...input };- for (const key of Object.keys(SWARM_DEFAULTS) as Array<keyof SwarmSettings>) {+ const settings: SwarmSettings = { ...defaults, ...input };+ for (const key of [...Object.keys(SWARM_DEFAULTS), ...SWARM_CAPS] as Array<keyof SwarmSettings>) {const value = settings[key];- if (!Number.isSafeInteger(value) || value < 1 || (key === "turnMs" && value > MAX_TIMER_MS))+ if (value === undefined && SWARM_CAPS.includes(key as never)) continue;+ if (!Number.isSafeInteger(value) || value! < 1 || (key === "turnMs" && value! > MAX_TIMER_MS))throw new Error(`invalid swarm setting: ${key}`);}if (settings.agents < 2) throw new Error("swarm settings must allow the root and a worker");diff --git a/src/swarms/swarm-store.ts b/src/swarms/swarm-store.tsindex 3585e7e97..323bdfd94 100644--- a/src/swarms/swarm-store.ts+++ b/src/swarms/swarm-store.ts@@ -54,7 +54,7 @@ export interface Swarm {ownerId: string;participants: string[];createdAt: number;- expiresAt: number;+ expiresAt?: number;template: SwarmTemplate;settings: SwarmSettings;backend: SandboxBackendName;@@ -67,8 +67,12 @@ export interface Swarm {pending: boolean;}+export function swarmExpired(swarm: Swarm): boolean {+ return swarm.expiresAt !== undefined && Date.now() >= swarm.expiresAt;+}+export function assertSwarmOpen(swarm: Swarm): void {- if (Date.now() >= swarm.expiresAt) throw new NonRetryableTurnError("swarm work window expired");+ if (swarmExpired(swarm)) throw new NonRetryableTurnError("swarm work window expired");}export interface SwarmStore {diff --git a/test/swarm-settings.test.ts b/test/swarm-settings.test.tsindex 8fddbaf8a..478f8827b 100644--- a/test/swarm-settings.test.ts+++ b/test/swarm-settings.test.ts@@ -25,6 +25,20 @@ test("backend defaults validate every setting and preserve unspecified defaults"assert.deepEqual(resolveSwarmSettings(large), { ...SWARM_DEFAULTS, ...large });});+test("budget settings are optional, off by default, and honored from SWARM_DEFAULTS", () => {+ const caps = { messages: 128, notifications: 256, spawnRequests: 32, lifetimeMs: 28_800_000 };+ assert.deepEqual(resolveSwarmSettings({}), SWARM_DEFAULTS);+ for (const key of Object.keys(caps)) {+ assert.equal(key in SWARM_DEFAULTS, false);+ for (const value of [0, -1, 1.5, "1", null])+ assert.throws(() => resolveSwarmSettings({ [key]: value }), new RegExp(key));+ }+ assert.deepEqual(resolveSwarmSettings(caps), { ...SWARM_DEFAULTS, ...caps });+ const swarmDefaults = loadConfig({ SWARM_DEFAULTS: JSON.stringify(caps) }).swarmDefaults;+ assert.deepEqual(swarmDefaults, { ...SWARM_DEFAULTS, ...caps });+ assert.deepEqual(resolveSwarmSettings({ messages: 500 }, swarmDefaults), { ...swarmDefaults, messages: 500 });+});+test("resolved defaults, initial overrides, children, and restarted services share one immutable settings contract", async () => {const f = await swarmFixture();const defaults = resolveSwarmSettings({@@ -45,8 +59,8 @@ test("resolved defaults, initial overrides, children, and restarted services shaconst restarted = createSwarmService({ ...f.serviceOptions, defaults: resolveSwarmSettings({ turnMs: 999 }) });assert.deepEqual((await restarted.inspect(caller)).settings, { ...defaults, turnMs: 777 });await assert.rejects(- restarted.spawn(caller, { requestId: "change", text: "work", settings: { turnMs: 999 } }),- /initial/,+ restarted.spawn(caller, { requestId: "change", text: "work", settings: { depth: 5 } }),+ /only the root/,);await assert.rejects(restarted.spawn(caller, { requestId: "child", text: "work" }), /depth/);await assert.rejects(restarted.context(caller, { long: "x".repeat(20) }), /context/);@@ -57,8 +71,24 @@ test("resolved defaults, initial overrides, children, and restarted services shaconst run = (await f.runs.list()).find((r) => r.request.swarm?.recipientId === sibling!.id)!;assert.equal(run.request.turnWallClockMs, 777);await assert.rejects(restarted.spawn(f.caller, { requestId: "excess", text: "work" }), /agent budget/);+ await assert.rejects(+ restarted.spawn(f.caller, { requestId: "turn", text: "work", settings: { agents: 4, turnMs: 999 } }),+ /turnMs cannot change/,+ );+ const [extra] = await restarted.spawn(f.caller, {+ requestId: "raise",+ text: "work",+ settings: { agents: 4, spawnRequests: 3 },+ });+ assert.ok(extra);+ assert.deepEqual((await restarted.inspect(f.caller)).settings, {+ ...defaults,+ turnMs: 777,+ agents: 4,+ spawnRequests: 3,+ });const swarm = (await f.store.get(f.root.id))!;- assert.equal(swarm.expiresAt - swarm.createdAt, 60_000);+ assert.equal(swarm.expiresAt! - swarm.createdAt, 60_000);});test("canonical recipient sets deduplicate retries but reject changed sets", async () => {@@ -160,3 +190,24 @@ test("operators can disable swarms without changing other background work", () =assert.equal(disabled.backgroundWorkEnabled, true);assert.throws(() => loadConfig({ SWARMS_ENABLED: "invalid" }), /SWARMS_ENABLED/);});++test("concurrent root settings changes both apply", async () => {+ const f = await swarmFixture();+ await f.service.spawn(f.caller, { requestId: "pool", text: "work" });+ await f.service.sweep();+ const update = f.store.update.bind(f.store);+ const ready = Promise.withResolvers<void>();+ let arrivals = 0;+ f.store.update = async (...args) => {+ if (++arrivals === 2) ready.resolve();+ await ready.promise;+ return update(...args);+ };+ await Promise.all([+ f.service.spawn(f.caller, { requestId: "agents", text: "work", settings: { agents: 64 } }),+ f.service.spawn(f.caller, { requestId: "depth", text: "work", settings: { depth: 8 } }),+ ]);+ const { settings } = (await f.store.get(f.root.id))!;+ assert.equal(settings.agents, 64);+ assert.equal(settings.depth, 8);+});diff --git a/test/swarm-simplification.test.ts b/test/swarm-simplification.test.tsindex e6221d03f..490dbf3bd 100644--- a/test/swarm-simplification.test.ts+++ b/test/swarm-simplification.test.ts@@ -145,11 +145,27 @@ test("initial pool cannot commit after its work window expires", async (context)assert.equal(await f.store.get(f.root.id), null);});+test("a swarm without lifetimeMs keeps accepting work long after it was created", async (context) => {+ const f = await swarmFixture();+ context.mock.timers.enable({ apis: ["Date"], now: Date.now() });+ const [worker] = await f.service.spawn(f.caller, { requestId: "initial", text: "work" });+ await f.service.sweep();+ assert.equal((await f.store.get(f.root.id))!.expiresAt, undefined);+ context.mock.timers.tick(30 * 86_400_000);+ const human = { kind: "human" as const, actorId: "alice", sessionId: f.root.id };+ const message = await f.service.send(human, { requestId: "later", audience: "all", text: "more work" });+ await f.service.sweep();+ assert.ok((await f.runs.list()).some((run) => run.request.swarm?.messageId === message.id));+ assert.equal((await f.service.inspect(human)).peers.find((p) => p.id === worker!.id)?.state, "ready");+ const [second] = await f.service.spawn(human, { requestId: "second", text: "more" });+ assert.ok(second);+});+test("initial work window uses one timestamp even when the clock advances between reads", async (context) => {const f = await swarmFixture();let now = Date.now();context.mock.method(Date, "now", () => now++);await f.service.spawn(f.caller, { requestId: "initial", text: "work", settings: { lifetimeMs: 60_000 } });const swarm = (await f.store.get(f.root.id))!;- assert.equal(swarm.expiresAt - swarm.createdAt, 60_000);+ assert.equal(swarm.expiresAt! - swarm.createdAt, 60_000);});diff --git a/test/swarms.test.ts b/test/swarms.test.tsindex 2d12adcc3..217946775 100644--- a/test/swarms.test.ts+++ b/test/swarms.test.ts@@ -606,7 +606,7 @@ test("swarm settings are fixed at creation and reject conflicting attempts", asyassert.equal(swarm.settings.turnMs, 600_000);await assert.rejects(service.spawn(caller, { requestId: "second", text: "Work", settings: { turnMs: 300_000 } }),- /settings are only allowed on initial swarm creation/,+ /turnMs cannot change after swarm creation/,);});@@ -642,17 +642,18 @@ test("recursive spawn depth is enforced independently of arbitrary character", atest("notification and message budgets include initial work and prevent recursion loops", async () => {const { service, caller, store, root } = await swarmFixture();- await service.spawn(caller, { requestId: "one", text: "Work" });+ const settings = { messages: 40, notifications: 20 };+ await service.spawn(caller, { requestId: "one", text: "Work", settings });await service.sweep();await store.update(root.id, (swarm) => {- swarm.notificationCount = SWARM_DEFAULTS.notifications;+ swarm.notificationCount = settings.notifications;});await assert.rejects(service.send(caller, { requestId: "notify", audience: "all", text: "Work" }),/notification budget/,);await service.send(caller, { requestId: "quiet", audience: "all", text: "No wake", notify: false });- for (let index = 2; index < SWARM_DEFAULTS.messages; index++) {+ for (let index = 2; index < settings.messages; index++) {await service.send(caller, { requestId: `quiet-${index}`, audience: "all", text: "Archive", notify: false });}await assert.rejects(@@ -662,6 +663,25 @@ test("notification and message budgets include initial work and prevent recursioassert.equal((await service.read(caller, {})).length, 32);});+test("spawn request budget applies only when set", async () => {+ const { service, caller } = await swarmFixture();+ await service.spawn(caller, { requestId: "one", text: "Work", settings: { spawnRequests: 2 } });+ await service.spawn(caller, { requestId: "two", text: "Work" });+ await assert.rejects(service.spawn(caller, { requestId: "three", text: "Work" }), /spawn budget/);+});++test("without caps, message, notification and spawn counts are unlimited", async () => {+ const { service, caller, runs } = await swarmFixture();+ await service.spawn(caller, { requestId: "one", count: 2, text: "Work", settings: { agents: 64 } });+ await service.sweep();+ for (let index = 0; index < 130; index++)+ await service.send(caller, { requestId: `notify-${index}`, audience: "all", text: "Work" });+ await service.sweep();+ assert.equal((await runs.list({ limit: 1_000 })).filter((run) => run.request.swarm).length, 262);+ for (let index = 0; index < 33; index++) await service.spawn(caller, { requestId: `spawn-${index}`, text: "Work" });+ assert.equal((await service.inspect(caller)).peers.length, 36);+});+test("restart between enqueue and outbox acknowledgement does not redeliver", async () => {const { service, serviceOptions, caller, store, root, runs } = await swarmFixture();await service.spawn(caller, { requestId: "pool", count: 2, text: "Work" });@@ -847,6 +867,20 @@ test("expired swarms retain readable history but cannot reserve or wake more worassert.equal((await runs.list()).length, 1);});+test("the root can extend an expired work window", async () => {+ const { service, caller, store, root } = await swarmFixture();+ await service.spawn(caller, { requestId: "one", text: "Work", settings: { lifetimeMs: 60_000 } });+ await store.update(root.id, (swarm) => {+ swarm.createdAt -= 120_000;+ swarm.expiresAt = swarm.createdAt + 60_000;+ });+ await assert.rejects(service.spawn(caller, { requestId: "two", text: "Work" }), /expired/);+ await service.spawn(caller, { requestId: "three", text: "Work", settings: { lifetimeMs: 3_600_000 } });+ const swarm = (await store.get(root.id))!;+ assert.equal(swarm.expiresAt, swarm.createdAt + 3_600_000);+ await service.send(caller, { requestId: "message", text: "Work", audience: "all" });+});+test("memory store has transaction rollback and detached read parity", async () => {const { service, caller, root, store } = await swarmFixture();await service.spawn(caller, { requestId: "one", text: "Work" });