Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
Merge remote-tracking branch 'refs/remotes/audit-open/1552'
AkitaOnRails44566c592006
diff --cc CHANGELOG.mdindex 62b8336e,10d7cb02..0c684040--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -285,139 -223,13 +285,146 @@@ version in [`docs/changelog/`](docs/charelay and wiki sync companions take the same `rustls-webpki` bump.### Fixed+- An empty structured answer from a Responses-API model is now reported as+ empty content rather than a JSON parse error. The `opencode`, `openai-oauth`+ and `codex` paths defaulted a missing `output_text` to an empty string and+ handed it to the parser, so the operator saw `serde: EOF while parsing a+ value at line 1 column 0`, naming neither the model nor the cause.+ ([#1529](https://github.com/akitaonrails/ai-memory/pull/1529))+- Detected `core.hooksPath` when a git config set it on the same physical+ line as the `[core]` section header, so `ai-memory-wikisync install-hook`+ no longer writes a `.git/hooks/post-merge` that git would never run.+ ([#1530](https://github.com/akitaonrails/ai-memory/pull/1530))+- Counted a file's final line in `ai-memory-wikisync` diff summaries when it+ does not end in a newline, so the `bytes/lines` figures no longer contradict+ the `first difference at line N` printed beside them.+ ([#1531](https://github.com/akitaonrails/ai-memory/pull/1531))+- `ai-memory run` now refuses an `--env`/`--env-file` entry whose key carries+ whitespace. `--env-file` lines are trimmed as a whole, not per half, so the+ common dotenv spelling `KEY = value` produced a key with a trailing space —+ an entry no shell or harness can read, so the override silently did nothing.+ ([#1532](https://github.com/akitaonrails/ai-memory/pull/1532))+- Discovered a repository page whose `--include` family names a bare top-level+ page (`README.md`), which `ai-memory-wikisync sync` previously left out of the+ repository side entirely, so a newly created one was never imported.+ ([#1533](https://github.com/akitaonrails/ai-memory/pull/1533))+- A Copilot `/responses` answer stopped by `max_output_tokens` is now reported+ as a truncated response naming the model and its output-token count, instead+ of an unexpected response shape reading "no error detail" — `error` is null+ on an incomplete answer, so nothing explained the stop and the class implied+ an unparseable reply rather than a `max_tokens` to raise.+ ([#1534](https://github.com/akitaonrails/ai-memory/pull/1534))+- Omitted TTL-expired pages from the page count in `memory_briefing` and+ `memory_explore`. Every page list in a briefing already hid them, and the+ project cards have since 3.1.x, so a project with an expired page reported+ more latest pages than any list in the same response could show. The+ all-versions total is unchanged, as is `ai-memory status`.+ ([#1535](https://github.com/akitaonrails/ai-memory/pull/1535))+++- `memory_read_page` now reads a blank `path` or `query` as omitted, as its+ sibling arguments already do. An empty or whitespace-only `query` searched+ for an empty FTS expression and reported the inevitable miss as an internal+ error (-32603), which a looping client reads as "retry"; an empty `path`+ failed as "invalid path: empty path" and never tried the `query` passed+ beside it. Both now reach the instructive "exactly one of `path` or `query`"+ refusal, and a blank `path` beside a real `query` runs the search.+ ([#1536](https://github.com/akitaonrails/ai-memory/pull/1536))+- Reported a wiki page renamed away with an external tool, which arrives as a+ rename event rather than a removal, so the next auto-commit drops it from the+ tree instead of leaving a checkpoint naming a page that no longer exists.+ ([#1537](https://github.com/akitaonrails/ai-memory/pull/1537))+- Omitted TTL-expired pages from the 7- and 30-day activity counters in+ `memory_briefing`. A page written with a short TTL made a project look+ active while the brief's own recent-page list was empty, which is the+ signal an agent uses to decide how much context to pull. Session and+ observation counters have no TTL and are unchanged.+ ([#1539](https://github.com/akitaonrails/ai-memory/pull/1539))++- Kept a literal `<mark>` tag from a page's own text out of a search snippet's+ markup, so it no longer renders as an unbalanced opening tag that highlights+ the rest of the excerpt as if it were a query match.+ ([#1540](https://github.com/akitaonrails/ai-memory/pull/1540))+- `install.ps1` now refuses a plain-http GitHub release base. The scheme check+ sat inside a condition whose arms already implied https, so it could never+ fire, and an `http://github.com/...` base matched neither arm and skipped it+ entirely — while the archive and its checksum are fetched from that same+ origin, so the checksum proved nothing against an on-path rewrite.+ ([#1541](https://github.com/akitaonrails/ai-memory/pull/1541))+- An unspaced-CJK search carrying `AND`, `NOT` or `NEAR` no longer returns the+ pages the operator excluded. The auxiliary CJK stream discarded every boolean+ operator and `OR`-joined the CJK runs it found, and because that stream is+ unioned into the primary one, `東京 NOT 大阪` fused the 大阪-only pages back in+ and ranked them as if they had been asked for. The stream now declines such a+ query, so unspaced-CJK recall narrows to the pre-feature answer for it instead+ of inverting; `OR` queries are unaffected.+ ([#1542](https://github.com/akitaonrails/ai-memory/pull/1542))+- An `openai-oauth` answer the Codex backend stopped at its output-token+ budget is now a terminal truncation instead of a retryable `502`. The+ streaming parser read `response.incomplete` as a provider fault, so the+ fallback chain advanced to the next candidate and opened a circuit, and the+ SessionEnd consolidation job kept re-sending the same prompt on its backoff+ schedule, each attempt truncating again at the same budget.+ ([#1543](https://github.com/akitaonrails/ai-memory/pull/1543))++- `memory_read_page` now answers a missing read target with `invalid params`+ (-32602) instead of an internal error (-32603). A page absent under the+ resolved scope, and a query that matched nothing, are caller-fixable: the+ store answered, there is simply nothing there, and a client reads -32603 as+ "retry". `memory_status` and `memory_read_session_observations` already+ answer -32602 for an unknown project or session. The messages and the+ auto-resolved-scope hint are unchanged, and a genuine disk or store failure+ on the same path still answers -32603.+ ([#1544](https://github.com/akitaonrails/ai-memory/pull/1544))+- An OpenCode subagent session's end no longer hands the next session a+ sub-task baton. The generated plugin forwarded the session's `parentID` as+ the `agent_id` subagent marker on `session.created` but not on+ `session.deleted`, so the `session-end` request carried no marker, the+ router did not recognize a child session, and it published a summary page+ and an open handoff that the next root session in that checkout consumed.+ ([#1546](https://github.com/akitaonrails/ai-memory/pull/1546))+- `install-mcp --client opencode --flavor <dialect>` now writes the flavored+ URL into the OpenCode entry. Both OpenCode entry builders read the server+ URL directly instead of the flavored one, so the flag was silently dropped+ for a client its own documentation names: a Vertex-routed install got a bare+ URL and 400'd every `tools/list`. OpenCode installs that do not pass+ `--flavor` are unchanged.+ ([#1547](https://github.com/akitaonrails/ai-memory/pull/1547))++- A `title:` or `body:` filter on an unspaced-CJK search now narrows the+ auxiliary CJK stream too. The stream read `:` as a plain run separator and+ dropped the qualifier, so its MATCH searched every indexed column including+ the path, and because that stream is unioned into the primary one,+ `title:支付服务` returned pages holding the term only in their body or path.+ ([#1548](https://github.com/akitaonrails/ai-memory/pull/1548))+- An Anthropic structured answer the model stopped at `max_tokens` is now a+ terminal truncation instead of a successful result. The provider did not read+ `stop_reason`, so a `tool_use` block holding only the JSON produced before the+ budget ran out was returned as a complete answer: a consolidation or bootstrap+ chunk could write a fraction of its pages and report success. On models that+ reject a forced `tool_choice` the same stop reported "no tool_use block"+ instead. Non-structured completions are unchanged — prose cut at the budget is+ still returned.+ ([#1549](https://github.com/akitaonrails/ai-memory/pull/1549))+- Counted a page needing only an L0 abstract embedding in `ai-memory embed+ --dry-run` and `/admin/embed`. A page whose body vector was already current+ appeared in neither the would-embed nor the up-to-date total, so the dry run+ reported no work for a scope where a live run writes abstract embeddings.+ ([#1550](https://github.com/akitaonrails/ai-memory/pull/1550))++- The PowerShell `session-start` hooks for Claude Code, Command Code and Devin+ now wrap the handoff in the same `hookSpecificOutput` envelope their POSIX+ peers emit. On Windows the baton was written as bare text, so the harness+ logged "Hook output does not start with {, treating as plain text" on every+ start and the handoff arrived unstructured instead of as `additionalContext`.+ ([#1551](https://github.com/akitaonrails/ai-memory/pull/1551))+ - The Japanese iteration mark 々 (and the other letters of CJK Symbols and+ Punctuation) now stay inside a CJK run. They terminated one, so a word like+ 日々 was indexed as two single characters and searching it emitted a bare+ `"日"` that matched every page containing 日 while 日々 itself was unfindable+ as a unit. Indexing changes with this, so existing rows keep their old terms+ until they are rewritten or the index is rebuilt by a compaction.+ ([#1552](https://github.com/akitaonrails/ai-memory/pull/1552))- Memory health no longer counts a link from a superseded or TTL-expired pageversion as a reference. A page whose only inbound link was removed by a lateredit was reported as linked, because the old version's `links` row survivesdiff --cc crates/ai-memory-store/src/fts_query.rsindex 0a743cd1,a2be1d42..7a5d4dca--- a/crates/ai-memory-store/src/fts_query.rs+++ b/crates/ai-memory-store/src/fts_query.rs@@@ -533,43 -523,19 +552,56 @@@ mod testsassert_eq!(prepare_cjk_fts5_query("latin only"), None);}+ #[test]+ fn cjk_stream_declines_a_query_the_expander_would_invert() {+ assert_eq!(prepare_cjk_fts5_query("東京 NOT 大阪"), None);+ assert_eq!(prepare_cjk_fts5_query("東京 AND 大阪"), None);+ assert_eq!(prepare_cjk_fts5_query("東京 NEAR 大阪"), None);+ // Control: `OR` is the one operator the run expander reproduces, so+ // the auxiliary stream still runs for it — and a lowercase `not` is+ // not FTS5 syntax, so it must not suppress the stream either.+ let both = Some("\"東京\" OR \"大阪\"".to_string());+ assert_eq!(prepare_cjk_fts5_query("東京 OR 大阪"), both);+ assert_eq!(prepare_cjk_fts5_query("東京 not 大阪"), both);+ assert_parses(&prepare_cjk_fts5_query("東京 OR 大阪").unwrap());+ }++ #[test]+ fn cjk_queries_keep_a_known_column_filter() {+ let title = prepare_cjk_fts5_query("title:支付服务").unwrap();+ assert_eq!(title, "title:\"支付 付服 服务\"");+ assert_parses(&title);+ let body = prepare_cjk_fts5_query("body:支付服务").unwrap();+ assert_eq!(body, "body:\"支付 付服 服务\"");+ assert_parses(&body);+ let single = prepare_cjk_fts5_query("title:支").unwrap();+ assert_eq!(single, "title:\"支\"");+ assert_parses(&single);+ // Control: the bare query is untouched, and a column FTS5 does not+ // know stays unqualified here exactly as the primary stream+ // neutralises it, rather than being emitted as invalid syntax.+ assert_eq!(+ prepare_cjk_fts5_query("支付服务"),+ Some("\"支付 付服 服务\"".to_string())+ );+ let unknown = prepare_cjk_fts5_query("author:支付服务").unwrap();+ assert_eq!(unknown, "\"支付 付服 服务\"");+ assert_parses(&unknown);+ }++ #[test]+ fn cjk_queries_keep_the_japanese_iteration_mark() {+ let query = prepare_cjk_fts5_query("日々").unwrap();+ assert_eq!(query, "\"日々\"");+ assert_parses(&query);+ // Control: the same block's punctuation still separates two runs,+ // so this is a widened letter range and not a widened block.+ assert_eq!(+ prepare_cjk_fts5_query("日、本"),+ Some("\"日\" OR \"本\"".to_string())+ );+ }+#[test]fn cjk_query_input_is_bounded_before_expansion() {let query = prepare_cjk_fts5_query(&"日".repeat(10_000)).unwrap();diff --cc crates/ai-memory-store/tests/suite/cjk_search.rsindex 060ed1d9,be483e2a..452539f4--- a/crates/ai-memory-store/tests/suite/cjk_search.rs+++ b/crates/ai-memory-store/tests/suite/cjk_search.rs@@@ -237,120 -237,60 +237,178 @@@ async fn superseded_cjk_text_is_hidden_assert_eq!(after_rebuild[0].path.as_str(), "repair.md");}+#[tokio::test]+async fn cjk_stream_does_not_fuse_back_a_not_excluded_term() {+ let tmp = tempfile::tempdir().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let (ws, project) = scope(&store, "default", "app").await;++ store+ .writer+ .upsert_page(page(ws, project, "tokyo.md", "東京", "office notes"))+ .await+ .unwrap();+ // 大阪 only ever appears inside an unspaced run, so unicode61 gives the+ // primary stream no token to match: this page is reachable through the+ // auxiliary CJK stream alone.+ store+ .writer+ .upsert_page(page(+ ws,+ project,+ "osaka.md",+ "Kansai",+ "関西の大阪支社について",+ ))+ .await+ .unwrap();++ // Control: the excluded page IS indexed and IS findable on its own, so an+ // absence below means the operator was honoured, not that nothing matched.+ let osaka = store+ .reader+ .search_pages_for_project(ws, project, "大阪".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = osaka.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"osaka.md"),+ "control search failed: {paths:?}"+ );++ let excluded = store+ .reader+ .search_pages_for_project(ws, project, "東京 NOT 大阪".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = excluded.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"tokyo.md"),+ "the requested term lost its page: {paths:?}"+ );+ assert!(+ !paths.contains(&"osaka.md"),+ "the CJK stream fused back the NOT-excluded term: {paths:?}"+ );+}++#[tokio::test]+async fn cjk_stream_honours_a_title_column_filter() {+ let tmp = tempfile::tempdir().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let (ws, project) = scope(&store, "default", "app").await;++ store+ .writer+ .upsert_page(page(+ ws,+ project,+ "titled.md",+ "支付服务概要",+ "English body",+ ))+ .await+ .unwrap();+ store+ .writer+ .upsert_page(page(+ ws,+ project,+ "bodied.md",+ "Callback notes",+ "启用微信支付服务后检查回调。",+ ))+ .await+ .unwrap();++ // Control: both pages carry 支付服务 in an unspaced run and both are+ // reachable through the CJK stream, so the narrowed search below failing+ // to return one of them means the column filter held.+ let unqualified = store+ .reader+ .search_pages_for_project(ws, project, "支付服务".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = unqualified.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"titled.md"),+ "control search failed: {paths:?}"+ );+ assert!(+ paths.contains(&"bodied.md"),+ "control search failed: {paths:?}"+ );++ let titled = store+ .reader+ .search_pages_for_project(ws, project, "title:支付服务".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = titled.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"titled.md"),+ "the title match lost its page: {paths:?}"+ );+ assert!(+ !paths.contains(&"bodied.md"),+ "the CJK stream ignored the title filter: {paths:?}"+ );+}+++ #[tokio::test]+ async fn cjk_search_finds_a_word_written_with_the_iteration_mark() {+ let tmp = tempfile::tempdir().unwrap();+ let store = Store::open(tmp.path()).unwrap();+ let (ws, project) = scope(&store, "default", "app").await;++ store+ .writer+ .upsert_page(page(+ ws,+ project,+ "daily.md",+ "運用",+ "日々の作業ログを残す。",+ ))+ .await+ .unwrap();+ store+ .writer+ .upsert_page(page(+ ws,+ project,+ "country.md",+ "概要",+ "日本の明日の予定。",+ ))+ .await+ .unwrap();++ // Control: the decoy IS indexed and IS findable on its own terms, so its+ // absence below means 日々 matched as a unit rather than nothing matching.+ let decoy = store+ .reader+ .search_pages_for_project(ws, project, "日本".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = decoy.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"country.md"),+ "control search failed: {paths:?}"+ );++ let iterated = store+ .reader+ .search_pages_for_project(ws, project, "日々".into(), 10, None)+ .await+ .unwrap();+ let paths: Vec<_> = iterated.iter().map(|hit| hit.path.as_str()).collect();+ assert!(+ paths.contains(&"daily.md"),+ "日々 was not findable as a unit: {paths:?}"+ );+ assert!(+ !paths.contains(&"country.md"),+ "日々 degraded to a bare 日 character match: {paths:?}"+ );+ }