Multiplayer agent harness for work.
docs: add Token performance to the wall of shame (#2160)
Regan Bellf92b8e808d7d
diff --git a/docs/SPEC.md b/docs/SPEC.mdindex fbf878c8a..970fe32ef 100644--- a/docs/SPEC.md+++ b/docs/SPEC.md@@ -180,14 +180,22 @@ Keeping old endpoints, data shapes, and code paths alive next to their replacemeDescription and examples pending human input.-### Performance (4 examples)+### Performance (3 examples)-Doing unnecessary work, repeating expensive work, or making independent work wait. Watch for hot polling loops, unbounded database reads, unstable prompt prefixes that defeat caching, and locks that serialize unrelated operations.+Doing unnecessary work, repeating expensive work, or making independent work wait. Watch for hot polling loops, unbounded database reads, and locks that serialize unrelated operations. Token costs belong under Token performance.- **Polling that writes even when nothing is due:** The cron scheduler sent a synthetic tick job every second and repeatedly submitted cron jobs. Idle scheduling still generated database writes and queue churn. _Status:_ [qm#2007](https://github.com/yc-software/qm/pull/2007) removes the per-second tick job; mutation and completion hooks keep jobs current, with periodic repair on the scheduler lease holder.- **Unbounded admin list queries:** The session list combined per-row JSON subqueries with full-history sorting and OFFSET pagination. A routine admin page could saturate Postgres and starve the shared connection pool used by actual turns. _Status:_ [qm#2076](https://github.com/yc-software/qm/pull/2076) makes the list query bounded and removes repeated work from each row.- **Serializing independent sandbox operations:** A single exclusive machine lock covered commands, file operations, and status checks. One long-running command blocked unrelated sessions using the same computer. _Status:_ [qm#1748](https://github.com/yc-software/qm/pull/1748) lets ordinary operations share the lock while keeping destructive lifecycle operations exclusive.++### Token performance (4 examples)++Tokens are the hot path. Anything sent to the model on every turn is paid for on every turn, in money, latency, and attention. Keep prompt prefixes stable so the KV cache holds, and make every token in the system prompt and per-turn context earn its place: no duplicates, no inventories the request doesn't need, no boilerplate repeated per item. Default to small and let the agent fetch the rest on demand.+- **Defeating prompt caching with incidental changes:** Changing countdowns, fresh sandbox/job snapshots, and inventories rendered in discovery order kept changing otherwise reusable prompt prefixes. _Status:_ [qm#1618](https://github.com/yc-software/qm/pull/1618) stabilizes snapshots and timestamps; [qm#1690](https://github.com/yc-software/qm/pull/1690) fixes the remaining inventory-order churn.+- **Per-turn `<environment>` note:** Memory, scheduled-work snapshots, and time blocks ride along with every turn whether or not the request needs them. The whole memory notebook is sent at session start and after compaction, then as deltas; on a long-lived DM that is hundreds of bullets. The spec describes vector recall, but the whole notebook is sent. _Status:_ still present on main.+- **Redundant system prompt:** The same instructions appear more than once: scope guidance that copies the org block and adds to it is re-sent in full (only exact matches are skipped), and every credential is listed twice (as execute handles and again as keychains), on top of a full skills index and a shared-file listing. Caching softens the price, not the attention cost. _Status:_ still present on main.+- **Per-fact provenance in memory:** Each remembered fact carries its own date and scope tag, and the same fact recalled from two scopes is printed twice, so the boilerplate grows linearly with the notebook. Dedupe across scopes when rendering, save each fact once, and state provenance once per group. _Status:_ still present on main.### Honorable mentions