Skip to content

armgit / qm

Public Imported from GitHub
0

Multiplayer agent harness for work.

Copy legacy tape steps automatically before the tape drop (#2349)

Regan Bell9b4540944680
diff --git a/docs/SPEC.md b/docs/SPEC.md
index 33363ab9d..df23042cb 100644
--- a/docs/SPEC.md
+++ b/docs/SPEC.md
@@ -175,7 +175,7 @@ Growing the codebase to keep every path it ever took alive. The canonical case:
- **Sidebar surface** (2026-10-14): The web sidebar's `surfaceOf` (`plugins/web-ui/src/sessions.ts`) is a conditional that string-matched `threadRef` prefixes (`web:`, `dm:`, `ch:`) because older sessions lacked a typed `surface`, and trusted the stored field only for subagents. When automation sessions landed in the wrong group, the proposed fix added more prefixes. Every session now stores its surface; read that field alone. _Status:_ still present on main.
- **Credential capture redesign** (2026-10-08): The login-as-a-skill handoff planned that "a stale skill calling /v1/keychain/use gets a clear refusal" and that "logins saved under the old capture system still load." Both keep the old system on life support. Remove the endpoint and the old capture system outright; stale skills fail and get fixed. _Status:_ [qm#2146](https://github.com/yc-software/qm/pull/2146) deletes `/v1/keychain/use` (open).
- **Sandbox routing retirement** (2026-10-02): Retiring `sandbox_routing` came with a locked, marker-guarded one-time upgrade import that preserved legacy IDs, explicit nulls, and machine IDs, kept the old table as rollback input, and told operators to keep credentials for legacy-mapped providers. A plain migration would have done. _Status:_ merged in [qm#2019](https://github.com/yc-software/qm/pull/2019).
-- **Session tape beside session entries** (2026-09-16): The tape was meant to replace `session_entries`, but caution kept both. `SESSION_TAPE_MODE` shadow/serve chose between them, exact-transcript annotations were added so the tape could reproduce every historical field, and a verifying, lease-aware operator backfill with dry runs and partial-scan statuses guarded the cutover. We got stuck here for weeks; one migration and a hard cutover would have left one system instead of two. _Status:_ code reads and writes only `session_entries` since [qm#2273](https://github.com/yc-software/qm/pull/2273); `session_tape` is dropped by a gated migration once its step metadata has been copied out ([qm#2280](https://github.com/yc-software/qm/pull/2280)).
+- **Session tape beside session entries** (2026-09-16): The tape was meant to replace `session_entries`, but caution kept both. `SESSION_TAPE_MODE` shadow/serve chose between them, exact-transcript annotations were added so the tape could reproduce every historical field, and a verifying, lease-aware operator backfill with dry runs and partial-scan statuses guarded the cutover. We got stuck here for weeks; one migration and a hard cutover would have left one system instead of two. _Status:_ code reads and writes only `session_entries` since [qm#2273](https://github.com/yc-software/qm/pull/2273); startup migrations copy `session_tape`'s step metadata out ([qm#2280](https://github.com/yc-software/qm/pull/2280)) and then drop it behind a gate.
- **Legacy Inbox loop:** `src/loops/inbox-migration.ts` migrates each owner's legacy Inbox loop at runtime, with its own phase record in UI-state preferences, a pause-and-disable sequence, and a repair path for half-finished runs. _Status:_ still on main; deletion approved for the loops rebuild.
### Over-testing
diff --git a/docs/upgrading-runtime.md b/docs/upgrading-runtime.md
index e381dad14..4f1052f5c 100644
--- a/docs/upgrading-runtime.md
+++ b/docs/upgrading-runtime.md
@@ -43,6 +43,17 @@ retention also removes expired entries from the legacy table, preventing a later
restart from importing them again. The legacy table remains available for older
binaries.
+Upgrading from a release that still writes `session_tape` copies its step metadata
+into `legacy_tape_steps` during startup migrations, then drops the tape. Stop every
+older core and worker before starting the new release: a tape write after the copy
+starts makes the drop refuse, and that startup fails with the reset to run. The copy
+runs on the direct migration connection without a statement timeout and resumes
+after an interrupted startup. It walks every session at roughly 30 sessions per
+second, so a few thousand sessions take a couple of minutes; above that, run
+`node src/migrate-main.ts` with the release's configuration before starting the
+servers so readiness checks do not time out. Instances that start meanwhile wait for the copy on
+the migration lock, so a database or role `statement_timeout` must allow that wait.
+
Rolling back the runtime restores code and configuration, not database contents.
Older binaries do not display cron history written only to the new journal; those
records remain stored in `cron_fires`. Validate both old and new readers against the
diff --git a/src/persistence/pg-pool.ts b/src/persistence/pg-pool.ts
index 467feaaa7..c199ce0d1 100644
--- a/src/persistence/pg-pool.ts
+++ b/src/persistence/pg-pool.ts
@@ -102,6 +102,7 @@ export interface PgMigrationDefinition {
statements: readonly string[];
expectedChecksum?: string;
legacyId?: string;
+ prepare?: readonly string[];
}
export interface PgMigration extends PgMigrationDefinition {
@@ -189,6 +190,7 @@ export function definePgMigration(
statements: readonly string[],
expectedChecksum?: string,
legacyId?: string,
+ prepare: readonly string[] = [],
): PgMigration {
if (!/^[a-z0-9][a-z0-9._/-]*$/i.test(id) || id.includes("..")) {
throw new Error(`pg-pool: invalid migration id ${JSON.stringify(id)}`);
@@ -201,7 +203,15 @@ export function definePgMigration(
`pg-pool: migration ${id} source checksum mismatch (expected=${expectedChecksum}, actual=${checksum})`,
);
}
- return { id, statements: normalized, checksum, ...(legacyId ? { legacyId } : {}) };
+ const preparation = prepare.map((statement) => statement.trim()).filter((statement) => statement.length > 0);
+ for (const statement of preparation) assertOneStatement(statement);
+ return {
+ id,
+ statements: normalized,
+ checksum,
+ ...(legacyId ? { legacyId } : {}),
+ ...(preparation.length ? { prepare: preparation } : {}),
+ };
}
export async function applyPgMigrations(pool: Pool, migrations: readonly PgMigration[]): Promise<void> {
@@ -229,6 +239,18 @@ export async function applyPgMigrations(pool: Pool, migrations: readonly PgMigra
}
continue;
}
+ if (migration.prepare) {
+ await client.query("SET statement_timeout = 0");
+ try {
+ for (const statement of migration.prepare) await client.query(statement);
+ } catch (error) {
+ await client
+ .query("RESET statement_timeout")
+ .catch(swallowAs(`pg-pool: reset statement timeout after preparing ${migration.id}`, undefined));
+ throw error;
+ }
+ await client.query("RESET statement_timeout");
+ }
if (migration.statements.some((statement) => concurrentIndexName(statement))) {
if (migration.legacyId || !migration.statements.every((statement) => concurrentIndexName(statement))) {
throw new Error(`pg-pool: concurrent index migration ${migration.id} must contain only concurrent indexes`);
@@ -419,7 +441,13 @@ export function createPgPool(
? [{ id: idOrDefinitions, statements: statementsOrMaintenance as readonly string[] }]
: idOrDefinitions;
const migrations = definitions.map((definition) =>
- definePgMigration(definition.id, definition.statements, definition.expectedChecksum, definition.legacyId),
+ definePgMigration(
+ definition.id,
+ definition.statements,
+ definition.expectedChecksum,
+ definition.legacyId,
+ definition.prepare,
+ ),
);
for (const migration of migrations) registerPgMigration(connectionString, migration);
const maintenanceSource =
@@ -558,6 +586,7 @@ export function createPgPool(
definition.statements,
definition.expectedChecksum,
definition.legacyId,
+ definition.prepare,
);
registerPgMigration(connectionString, migration);
if (closed) throw new Error("Postgres store is closed");
@@ -567,7 +596,13 @@ export function createPgPool(
function registerMigration(definition: PgMigrationDefinition): void {
registerPgMigration(
connectionString,
- definePgMigration(definition.id, definition.statements, definition.expectedChecksum, definition.legacyId),
+ definePgMigration(
+ definition.id,
+ definition.statements,
+ definition.expectedChecksum,
+ definition.legacyId,
+ definition.prepare,
+ ),
);
}
return { pool, sessionPool, q, query, registerMigration, migrate, close };
diff --git a/src/sessions/legacy-tape-steps.ts b/src/sessions/legacy-tape-steps.ts
index f3b945d63..9f3b4c79d 100644
--- a/src/sessions/legacy-tape-steps.ts
+++ b/src/sessions/legacy-tape-steps.ts
@@ -273,6 +273,21 @@ END $batch$`,
skippingProcedure(),
];
+export const PREPARE_TAPE_DROP_SQL = [
+ `DO $prepare_tape_drop$
+BEGIN
+ IF to_regclass('session_tape') IS NULL OR (SELECT done FROM legacy_tape_steps_progress) THEN
+ RETURN;
+ END IF;
+ IF (SELECT started_at FROM legacy_tape_steps_progress) IS NULL THEN
+ DELETE FROM session_tape t WHERE NOT EXISTS (SELECT 1 FROM sessions s WHERE s.id = t.session_id);
+ END IF;
+ IF EXISTS (SELECT 1 FROM session_tape WHERE kind = 'message') THEN
+ CALL prepare_legacy_tape_steps(50);
+ END IF;
+END $prepare_tape_drop$`,
+];
+
export const DROP_SESSION_TAPE_SQL = [
`SET LOCAL lock_timeout = '5s'`,
`DO $tape_drop_gate$
diff --git a/src/sessions/postgres-session-store.ts b/src/sessions/postgres-session-store.ts
index f0bab5c23..c05297897 100644
--- a/src/sessions/postgres-session-store.ts
+++ b/src/sessions/postgres-session-store.ts
@@ -39,7 +39,12 @@ import type {
StoreOptions,
} from "./session-store.ts";
import { tsPrefixQuery } from "./entry-search.ts";
-import { DROP_SESSION_TAPE_SQL, LEGACY_TAPE_STEPS_SIZE_GUARD_SQL, LEGACY_TAPE_STEPS_SQL } from "./legacy-tape-steps.ts";
+import {
+ DROP_SESSION_TAPE_SQL,
+ LEGACY_TAPE_STEPS_SIZE_GUARD_SQL,
+ LEGACY_TAPE_STEPS_SQL,
+ PREPARE_TAPE_DROP_SQL,
+} from "./legacy-tape-steps.ts";
import {
cronIdOf,
legacyOriginPattern,
@@ -814,6 +819,7 @@ export function createPostgresSessionStore(connectionString: string, opts: Store
},
{
id: "sessions/store/0031-drop-session-tape",
+ prepare: PREPARE_TAPE_DROP_SQL,
statements: DROP_SESSION_TAPE_SQL,
},
],
diff --git a/test/postgres-legacy-tape-steps.test.ts b/test/postgres-legacy-tape-steps.test.ts
index d5a628c42..a609fc466 100644
--- a/test/postgres-legacy-tape-steps.test.ts
+++ b/test/postgres-legacy-tape-steps.test.ts
@@ -78,6 +78,85 @@ const progress = async (raw: import("pg").Pool) =>
)
).rows[0];
+const toolCallSession = (raw: import("pg").Pool, id: string) =>
+ seedSession(
+ raw,
+ id,
+ [message([{ type: "toolCall", id: `call-${id}`, name: "execute", arguments: { z: id, a: 1 } }])],
+ [["tool_call", { tool: "execute", callId: `call-${id}`, a: 1, z: id }]],
+ );
+
+const tapeState = async (raw: import("pg").Pool) => ({
+ tape: (await raw.query("SELECT to_regclass('session_tape') AS t")).rows[0].t,
+ steps: (await raw.query("SELECT session_id, wire FROM legacy_tape_steps ORDER BY session_id")).rows,
+ done: (await raw.query("SELECT done FROM legacy_tape_steps_progress")).rows[0].done,
+});
+
+test(
+ "pg migrations copy legacy tape steps on their own before dropping the tape, past deleted sessions",
+ { skip },
+ async () => {
+ await withSchema(async (raw, url) => {
+ await toolCallSession(raw, "legacy");
+ await toolCallSession(raw, "zdeleted");
+ await raw.query("DELETE FROM sessions WHERE id = 'zdeleted'");
+ await seedSession(raw, "tapeless", [], [["user", { text: "hi" }]]);
+ const pg = (await import("pg")).default;
+ const timed = new globalThis.URL(url);
+ timed.searchParams.set("options", `${timed.searchParams.get("options")} -c statement_timeout=4321`);
+ const startup = new pg.Pool({ connectionString: timed.toString(), max: 1 });
+ try {
+ await applyPgMigrations(startup, registeredPgMigrations(url));
+ assert.equal((await startup.query("SHOW statement_timeout")).rows[0].statement_timeout, "4321ms");
+ } finally {
+ await startup.end();
+ }
+ assert.deepEqual(await tapeState(raw), {
+ tape: null,
+ steps: [{ session_id: "legacy", wire: { argumentsJson: '{"z":"legacy","a":1}' } }],
+ done: true,
+ });
+ });
+ },
+);
+
+test("pg migrations drop a tape whose sessions were all deleted", { skip }, async () => {
+ await withSchema(async (raw, url) => {
+ await toolCallSession(raw, "gone");
+ await raw.query("DELETE FROM sessions");
+ await applyPgMigrations(raw, registeredPgMigrations(url));
+ assert.deepEqual(await tapeState(raw), { tape: null, steps: [], done: false });
+ });
+});
+
+test("pg migrations resume an interrupted tape copy once when two instances start together", { skip }, async () => {
+ await withSchema(async (raw, url) => {
+ for (const id of ["s1", "s2", "s3"]) await toolCallSession(raw, id);
+ await raw.query("CALL prepare_legacy_tape_steps(1)");
+ await raw.query("UPDATE legacy_tape_steps_progress SET last_session = 's1', done = false");
+ await raw.query("DELETE FROM legacy_tape_steps WHERE session_id <> 's1'");
+
+ const pg = (await import("pg")).default;
+ const other = new pg.Pool({ connectionString: url });
+ try {
+ await Promise.all([
+ applyPgMigrations(raw, registeredPgMigrations(url)),
+ applyPgMigrations(other, registeredPgMigrations(url)),
+ ]);
+ } finally {
+ await other.end();
+ }
+ assert.deepEqual(await tapeState(raw), {
+ tape: null,
+ steps: ["s1", "s2", "s3"].map((id) => ({
+ session_id: id,
+ wire: { argumentsJson: `{"z":"${id}","a":1}` },
+ })),
+ done: true,
+ });
+ });
+});
+
test("pg legacy tape steps keep each step's model and exact block wire, and map turns to runs", { skip }, async () => {
await withSchema(async (raw) => {
await seedSession(
@@ -221,8 +300,6 @@ test(
[["tool_call", { tool: "execute", callId: "c1", a: 2, b: 1 }]],
);
const migrations = registeredPgMigrations(url);
- await assert.rejects(applyPgMigrations(raw, migrations), /run CALL prepare_legacy_tape_steps\(\) to completion/);
-
await raw.query("CALL prepare_legacy_tape_steps()");
await raw.query("UPDATE legacy_tape_steps_progress SET skipped_sessions = 101");
await assert.rejects(