Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge remote-tracking branch 'origin/pr/1826' into release/3.2.10
AkitaOnRails014bb8832bab
diff --cc CHANGELOG.mdindex 19dd3475,6e632414..be19850b--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -11,124 -11,11 +11,127 @@@ version in [`docs/changelog/`](docs/cha## [Unreleased]+### Changed++- Documented the settings that `config.default.toml` had left out, and added+ `docs/config-properties.md`, a reference table of every key `config.toml`+ accepts with its default, environment variable and purpose. The template now+ describes the `llm_*` keys (including `llm_compat_strict` and+ `[[llm_fallbacks]]`), the embedding endpoint and prefix keys,+ `tcp_keepalive_secs`, `base_path`, `release_base_url`, `cors_allow_origins`,+ `consolidate_on_session_end`, `backfill_on_start`, `run_autowire`,+ `claude_true_yolo`, the `contradiction_band_*` keys, and the `[run]`,+ `[hook_spool]`, `[handoff]`, `[maintenance]`, `[dream]`, `[retrieval]`,+ `[search.fts]`, `[profile]` and `[[admission_webhooks]]` sections. It also+ lists the remaining keys of `[decay]` (including `[decay.half_life_days]`),+ `[consolidation]`, `[auto_improve]`, `[auto_scope]` and `[auth]`, and the+ `[auto_improve.scheduler.experience_entropy_filter]` table. The new entries+ are all commented examples, and keys with a built-in default show that+ default. `AGENTS.md` now requires a new config key to land in both files.+### Fixed+- Sorted recent page lists by timestamp rather than RFC3339 text, so updates+ with different fractional-second precision appear in chronological order+ in the legacy recent API, project activity, folder previews and namespace+ listings. Kept title-sort and equal-timestamp path tie-breaks (#1247).++- Fixed `restore` replacing live data when a backup without `config.toml`+ contains API keys that the configured pepper cannot verify. It now refuses+ before the swap, preserving the live wiki, database and config, and names+ the original configuration as the remedy (#1495).++- Preserved explicit DOCX/PPTX line breaks and DOCX tabs in document imports+ when XML uses paired tags, including formatted PowerPoint breaks in slides+ and speaker notes. DOCX carriage returns now also keep adjacent text runs+ on separate lines instead of joining their words. Paragraph tab-stop+ definitions no longer insert spurious tabs into the extracted text (#1786).++- Kept a wiki-sync server page when its repository file was recreated after+ the sync planned a deletion. `sync --propagate-deletes --apply` now rechecks+ local absence before sending the delete, preserves the page's baseline and+ reports the changed file for the next sync, including with `--prefer repo`+ (#1787).+- Transcript imports (`backfill`, the `run` ledger, `recover`) now drop Codex+ memory-writing agent runs by the same signals the live hook uses: the+ checkout cwd, an event's recorded cwd, or a Codex user prompt that opens+ with `## Memory Writing Agent`. A custom `CODEX_HOME` whose last path+ component is not `.codex` previously reimported those sessions (#1790).+- Admission webhook HTTP requests now follow redirects only within the+ configured `scheme://host:port` origin (up to 10 hops) and fail on+ cross-origin redirects without forwarding page payloads or applying foreign+ response mutations, surfacing the redirect refusal cause in logs and+ `reject`-policy errors (#1794).+- `ai-memory project access` and `ai-memory project grants` now refuse empty+ or whitespace-only `--workspace`, `--project`, and (for access) `--mode`+ values before sending any request to the server. These commands take+ required names and never call `resolve_scope`, so the #1774 choke point+ did not cover them (#1791).+- `ai-memory user grant` and `ai-memory user revoke` now refuse empty or+ whitespace-only `--user`, `--workspace`, `--project`, and (for grant)+ `--level` values before sending any request to the server. These commands+ take required names and never call `resolve_scope`, so the #1774 choke+ point did not cover them (#1792).+- Stopped the lifecycle relay from treating an oversized HTTP response as a+ transport failure and immediately resending its batch. A response beyond the+ 64 KiB acknowledgement cap now retains its HTTP status, stops the flush, and+ keeps unacknowledged events pending, including on HTTP 429 backpressure (#1799).+- Decoded WebVTT cue character references before document-import sanitization,+ preserving literal escaped tags and applying the existing text-size limit+ to the decoded text. SRT extraction is unchanged (#1800).+- Preserved native OIDC hook-spool entries when the POSIX or PowerShell script+ bundle drained the shared queue. Scripts now leave those entries for native+ credential resolution instead of posting them anonymously and retiring them+ on a 2xx or 401 response; static, anonymous and legacy entries still drain+ normally (#1801).+- Refused backup restores with a missing or invalid gzip trailer before+ replacing live data, even when all tar entries were readable (#1805).+- Kept separate pre-migration archives when installations share a backup+ destination and start in the same second, instead of overwriting an+ existing installation's snapshot (#1806).+- Restored the previous wiki file when `restore-page` could not commit its+ SQLite update, including a rejected case-colliding historical path. Restores+ now share the page writer's lock through conformance, file replacement and+ indexing so a concurrent write to that same path cannot leave disk and+ SQLite disagreeing (#1807).+- Kept concurrent page edits current during opt-in cold-cluster dedup and dream+ merges. Each write batch now skips if a selected source version changed,+ preserving the newer body, tier and pin along with unchanged peers (#1808).+- Dropping the final store writer handle now finishes when its command queue+ is full. Accepted writes still drain before the writer thread exits, including+ when callers have cancelled their waits (#1809).++- Scheduled empty-project cleanup now preserves projects that have sent or+ received agent messages, including claimed/cancelled history, so it cannot+ silently delete their inbox/outbox through a project cascade (#1810).++- Fixed `server add` rejecting valid inline tables in `servers.toml`, including+ token rotations and URL changes. Updates now accept the same table forms as+ reads and removal, preserving registered roots and other profiles (#1812).+- Kept OpenCode V1 failed tool results in the managed workstream ledger,+ including their visible error text and error status, instead of importing+ only the call and silently dropping the failure reason (#1814).+- Initialized the TypeScript SDK's MCP session before the first `callTool`,+ fixing HTTP 422 responses with `--http-stateful`. Raw calls still skip+ schema checks, and later wrapped calls reuse the session (#1816).+- Preserved literal reasoning tags and their contents inside structured JSON+ strings when parsing OpenAI-compatible tolerant replies and Anthropic text+ replies. Reasoning blocks outside the JSON are still removed, including+ when a strict OpenAI-compatible request falls back to tolerant parsing (#1817).+- Disabled HTTP redirect following on the native hook client (`build_client`)+ so 3xx responses from SSO gateways, captive portals, or proxies fail closed+ instead of following to a 200 OK login page, which dropped spooled hook events+ and injected login HTML into agent handoff context (#1818).+- Supported singular units (`day`, `week`, `month`) and weeks in+ `parse_since_to_epoch` for git commit filtering during consolidation+ bootstrapping, trimmed surrounding whitespace, rejected negative intervals,+ and used checked arithmetic to prevent integer overflow panics (#1820).+- Kept OpenCode 2 failed tool results in the managed workstream ledger,+ including their visible error text and error status, instead of importing+ only the call and recording a status-error loss (#1823).+ - Imported OpenCode 2 compaction session messages into the managed workstream+ ledger (`summary`, or `error.message` on a failed compaction) instead of+ dropping them with no event and no loss (#1824).- Refused empty and whitespace-only `--workspace` and `--project` flags at theshared CLI scope-resolution choke point (`resolve_scope`,`resolve_scope_noting_marker_project`, `resolve_scope_with_marker`, anddiff --cc crates/ai-memory-workstream/src/transcript.rsindex 0d5f356e,d15dd322..7edaab6d--- a/crates/ai-memory-workstream/src/transcript.rs+++ b/crates/ai-memory-workstream/src/transcript.rs@@@ -8903,177 -8792,139 +8942,329 @@@ mod testsassert!(second.losses.is_empty());}+ #[test]+ fn opencode2_failed_tool_results_keep_the_visible_error() {+ let home = tempfile::tempdir().unwrap();+ let db = opencode_db(home.path(), None);+ fs::create_dir_all(db.parent().unwrap()).unwrap();+ let connection = Connection::open(&db).unwrap();+ connection+ .execute_batch(+ "CREATE TABLE session_v2(id TEXT PRIMARY KEY, directory TEXT NOT NULL, \+ time_updated INTEGER NOT NULL);\+ CREATE TABLE session_message(id TEXT PRIMARY KEY, session_id TEXT, type TEXT, \+ time_updated INTEGER, data TEXT);",+ )+ .unwrap();+ connection+ .execute(+ "INSERT INTO session_v2 VALUES ('s2', ?1, 10)",+ [home.path().join("repo").to_string_lossy().to_string()],+ )+ .unwrap();+ // Completed tools use `content`; terminal failures use `error` as a+ // string (older OpenCode) or `{message}` (current schema). Optional+ // error `content` is the visible result when present.+ let rows = [+ (+ "completed",+ 1,+ json!({+ "status": "completed",+ "input": {"cmd": "ok"},+ "content": [{"type": "text", "text": "success"}],+ "error": {"type": "ignored", "message": "not a completed result"}+ }),+ ),+ (+ "failed-object",+ 2,+ json!({+ "status": "error",+ "input": {"cmd": "fail"},+ "error": {"type": "Tool.Error", "message": "Permission denied"}+ }),+ ),+ (+ "failed-string",+ 3,+ json!({+ "status": "error",+ "input": {"cmd": "fail"},+ "error": "disk full"+ }),+ ),+ (+ "failed-content",+ 4,+ json!({+ "status": "error",+ "input": {"cmd": "fail"},+ "error": {"type": "Tool.Error", "message": "hidden"},+ "content": [{"type": "text", "text": "visible content error"}]+ }),+ ),+ (+ "pending",+ 5,+ json!({+ "status": "pending",+ "input": {"cmd": "wait"},+ "error": {"type": "Tool.Error", "message": "not a terminal result"}+ }),+ ),+ (+ "running",+ 6,+ json!({+ "status": "running",+ "input": {"cmd": "wait"},+ "error": {"type": "Tool.Error", "message": "not a terminal result"}+ }),+ ),+ (+ "streaming",+ 7,+ json!({+ "status": "streaming",+ "input": {"cmd": "wait"},+ "error": "not a terminal result"+ }),+ ),+ (+ "malformed",+ 8,+ json!({+ "status": "error",+ "input": {"cmd": "fail"},+ "error": {"type": "Tool.Error"}+ }),+ ),+ ];+ for (id, updated, state) in rows {+ connection+ .execute(+ "INSERT INTO session_message VALUES (?1, 's2', 'assistant', ?2, ?3)",+ params![+ id,+ updated,+ json!({"content":[{"type":"tool","name":"bash","state":state}]})+ .to_string()+ ],+ )+ .unwrap();+ }+ let before = fs::read(&db).unwrap();+ let transcript = export_opencode2(home.path(), None, "s2", None).unwrap();+ let results: Vec<_> = transcript+ .events+ .iter()+ .filter(|event| event.kind == WorkstreamEventKind::ToolResult)+ .collect();+ assert_eq!(+ results+ .iter()+ .map(|event| event.content.as_str())+ .collect::<Vec<_>>(),+ [+ "success",+ "Permission denied",+ "disk full",+ "visible content error"+ ]+ );+ assert_eq!(results[0].metadata, json!({"status": "completed"}));+ assert_eq!(+ results[1].metadata,+ json!({"status": "error", "is_error": true})+ );+ assert_eq!(+ results[1].source_record_id.as_deref(),+ Some("failed-object")+ );+ assert_eq!(+ results[2].metadata,+ json!({"status": "error", "is_error": true})+ );+ assert_eq!(+ results[3].metadata,+ json!({"status": "error", "is_error": true})+ );+ for status in ["pending", "running", "streaming"] {+ assert!(+ transcript+ .losses+ .iter()+ .any(|loss| loss.contains(&format!("ended with status {status}"))),+ "missing loss for {status}: {:?}",+ transcript.losses+ );+ }+ assert_eq!(+ transcript+ .losses+ .iter()+ .filter(|loss| loss.contains("ended with status error"))+ .count(),+ 1,+ "malformed error should stay a loss: {:?}",+ transcript.losses+ );+ let replay = export_opencode2(home.path(), None, "s2", None).unwrap();+ assert_eq!(+ transcript++ .events++ .iter()++ .map(|event| &event.event_id)++ .collect::<Vec<_>>(),++ replay++ .events++ .iter()++ .map(|event| &event.event_id)++ .collect::<Vec<_>>()++ );++ assert!(++ export_opencode2(home.path(), None, "s2", transcript.source_cursor.as_deref())++ .unwrap()++ .events++ .is_empty()++ );++ assert_eq!(fs::read(&db).unwrap(), before);++ }+++ #[test]+ fn opencode2_compaction_summaries_are_imported() {+ let home = tempfile::tempdir().unwrap();+ let db = opencode_db(home.path(), None);+ fs::create_dir_all(db.parent().unwrap()).unwrap();+ let connection = Connection::open(&db).unwrap();+ connection+ .execute_batch(+ "CREATE TABLE session_v2(id TEXT PRIMARY KEY, directory TEXT NOT NULL, \+ time_updated INTEGER NOT NULL);\+ CREATE TABLE session_message(id TEXT PRIMARY KEY, session_id TEXT, type TEXT, \+ time_updated INTEGER, data TEXT);",+ )+ .unwrap();+ connection+ .execute(+ "INSERT INTO session_v2 VALUES ('s2', ?1, 10)",+ [home.path().join("repo").to_string_lossy().to_string()],+ )+ .unwrap();+ let rows = [+ (+ "done",+ 1,+ json!({+ "status": "completed",+ "reason": "auto",+ "summary": "compacted four turns"+ }),+ ),+ (+ "running",+ 2,+ json!({+ "status": "running",+ "reason": "manual",+ "summary": "compacting now"+ }),+ ),+ (+ "failed-object",+ 3,+ json!({+ "status": "failed",+ "reason": "auto",+ "error": {"type": "ProviderError", "message": "rate limited"}+ }),+ ),+ (+ "failed-string",+ 4,+ json!({+ "status": "failed",+ "reason": "auto",+ "error": "provider timeout"+ }),+ ),+ (+ "failed-empty",+ 5,+ json!({+ "status": "failed",+ "reason": "auto",+ "error": {"type": "ProviderError"}+ }),+ ),+ (+ "empty-completed",+ 6,+ json!({+ "status": "completed",+ "reason": "auto",+ "summary": " "+ }),+ ),+ ];+ for (id, updated, data) in rows {+ connection+ .execute(+ "INSERT INTO session_message VALUES (?1, 's2', 'compaction', ?2, ?3)",+ params![id, updated, data.to_string()],+ )+ .unwrap();+ }+ let before = fs::read(&db).unwrap();+ let transcript = export_opencode2(home.path(), None, "s2", None).unwrap();+ let events: Vec<_> = transcript+ .events+ .iter()+ .filter(|event| event.kind == WorkstreamEventKind::Compaction)+ .collect();+ assert_eq!(+ events+ .iter()+ .map(|event| event.content.as_str())+ .collect::<Vec<_>>(),+ ["compacted four turns", "rate limited", "provider timeout"]+ );+ assert_eq!(events[0].metadata, json!({"status": "completed"}));+ assert_eq!(events[1].metadata, json!({"status": "failed"}));+ assert_eq!(events[1].source_record_id.as_deref(), Some("failed-object"));+ assert_eq!(events[2].metadata, json!({"status": "failed"}));+ for status in ["running", "failed"] {+ assert!(+ transcript+ .losses+ .iter()+ .any(|loss| loss.contains(&format!("compaction ended with status {status}"))),+ "missing loss for {status}: {:?}",+ transcript.losses+ );+ }+ assert_eq!(+ transcript+ .losses+ .iter()+ .filter(|loss| loss.contains("compaction ended with status failed"))+ .count(),+ 1,+ "empty failed compaction should stay a loss: {:?}",+ transcript.losses+ );+ assert!(+ !transcript+ .losses+ .iter()+ .any(|loss| loss.contains("status completed")),+ "empty completed compaction is omitted, not a loss: {:?}",+ transcript.losses+ );+ let replay = export_opencode2(home.path(), None, "s2", None).unwrap();+ assert_eq!(+ transcript.events.iter().map(|event| &event.event_id)diff --cc docs/managed-workstreams.mdindex 0ee32c32,e60821a4..80a6f13a--- a/docs/managed-workstreams.md+++ b/docs/managed-workstreams.md@@@ -562,14 -562,10 +562,18 @@@ another shell, pass `--workstream-id <uthe source harness, role, event sequence, and content. Historical tool activityis labelled completed evidence and must never be replayed as a pending call.+OpenCode V1 tool failures contribute their visible `state.error` text as a+result, with `status: "error"` and `is_error: true` in the ledger metadata.+Successful calls keep their ordinary `state.output` result.++OpenCode 2 tool failures contribute their visible error text as a result, with+`status: "error"` and `is_error: true` in the ledger metadata. Successful calls+keep their ordinary `state.content` text parts.++ OpenCode 2 compaction messages import their `summary` as a compaction event+ (and, when a failed compaction has no summary, `error.message`). The V1 adapter+ already imported compaction parts the same way.+Search and tail also return optional `source_record_id` and `metadata` fields.The text CLI prints the source record and correlation metadata as escaped JSON;`--json` preserves these fields for consumers. Older server responses without