Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts.
Rust ai-agentsai-gatewayclaude-codecodexcoding-agentcost-optimizationdeepseekllm-gatewayllm-routerlocal-firstmodel-routingrusttask-routingtauri
fix: keep retained conversation analysis responsive during capture (#149)
澄潭9d9939ac9e18
diff --git a/assets/skills/hiroute-management/SKILL.md b/assets/skills/hiroute-management/SKILL.mdindex d41b6c2e..b4b8ca2e 100644--- a/assets/skills/hiroute-management/SKILL.md+++ b/assets/skills/hiroute-management/SKILL.md@@ -10,8 +10,8 @@ metadata:# HiRoute management-Use installed hiroute. This Skill is shared by all supported Agents; caller/target and-model/management/collaboration facets are independent.+Use installed hiroute. All Agents share this Skill; caller/target and model/management/collaboration+facets are independent.## Goal and target@@ -19,16 +19,15 @@ Journey: target → source → Plan → connect → native activation → matchinames may repeat. Saved, loaded, selected and served differ.Start with hiroute quickstart/help; select returned --context CONTEXT_ID and optional --plan PLAN_ID.-Hermes needs actual --hermes-root /ABS/ROOT, never an IM profile inferred from daemon env.-Resolve ambiguity; retain missing/unreadable targets for repair.+Hermes needs actual --hermes-root /ABS/ROOT, not daemon env. Resolve ambiguous/unreadable targets.Missing CLI: use Desktop Settings → CLI or standalone installer. Agent → HiRoute management enablesthis Skill. Recover the existing service using status/doctor/bounded logs; never start a second daemon.## Contracts and result capture-Use family/leaf --help and schema show --command-id COMMAND --include-schema for contracts/examples.-Help works offline. Planned handlers are unavailable. Get IDs/revisions from options/Preview.+Use family/leaf --help and schema show --command-id COMMAND --include-schema. Planned handlers are+unavailable; get IDs/revisions from options/Preview.Save full stdout JSON, separate stderr and actual exit privately. Exit 2 means invalid input;--error-details gives hints. Read the full response: accepted is pending; query success can contain a failed Operation.@@ -141,7 +140,9 @@ Use `agents connect preview --context CONTEXT_ID --apply-file PATH` with requestfor tasks. Omissions remain. Skill-only needs no model Plan/plugin. Delegation still requirescollaboration policy/user intent. Native help supplies choices:-Select native options from installed help. Hermes uses exact root/protocol; preserve Responses+Binary/Skill upgrades do not refresh model plugins. Preview/Apply the saved model facet with+access_token=keep, then reload/start a native process; preserve other facets.+Select native options from help. Hermes uses exact root/protocol; preserve Responsesreasoning. Recommend >=0.21.7 for replay; compatible older versions remain allowed, while requiredplugin/SDK interfaces may block. No unsolicited upgrade/patch/thinking disable. Codex needs exactroot/profile and native/default model choice; Claude uses the intended preset, preserving others.@@ -169,9 +170,8 @@ conflicts. Restore may require native selection/reload and caller continuity.## Analyze history and savings-Use same-OS-user CLI access; explicit content/search needs no GUI read ticket. Do not use global-`--agent` for management: that selects the separate sealed collaboration channel. A supplied grant-is still checked; model Gateway keys cannot authorize management.+Same-OS-user content/search needs no GUI ticket. Global `--agent` selects sealed collaboration,+not management. Supplied grants are checked; model keys grant no management authority.Choose the user's goal: switch explanation, conversation, expense ranking or savings. Start one scope:@@ -182,12 +182,13 @@ hiroute sessions list --view switches --period 7dhiroute sessions list --view conversations --period 7d --query TEXT```-Use sessions/value help for schemas/choices. Report requested/effective bounds, offsets and retention-clipping. Civil days are service-local; never mix period/day/absolute modes. New views reject --model-switch.+Use sessions/value help. Report requested/effective bounds, offsets and clipping. Civil days are+service-local; do not mix time modes. New views reject --model-switch.-Retain returned analysis_scope. Use it for the other paths and narrower session/request drill-downs,-without a new time selector. Follow next_cursor with the same view/filters/limit; collect every page-before calling a ranking complete. A page length/member preview is not a total. Exact commands:+Retain analysis_scope for related views and narrower session/request reads; add no time selector.+Search must start with --query TEXT: a fixed scope rejects adding/changing its keyword. Start a new+search scope, keeping the original for comparisons. Page with unchanged view/filters/limit to END;+page length/member preview is not a total. Exact commands:```shhiroute sessions show SESSION_ID --view routing --analysis-scope TOKEN@@ -196,10 +197,9 @@ hiroute value show --group-by request --order-by cost-desc --session SESSION_IDhiroute sessions show SESSION_ID --view conversation --request REQUEST_ID --analysis-scope TOKEN```-Carry scope through your own new requests/compaction, pagination and follow-up questions. Fresh-requests, settlement or prices do not change its pinned relations/numbers/prices. Only narrow it;-never silently replace the scope or compare totals across refreshed scopes. Expiry, withdrawn proof,-affected deletion or authority change requires an explicit fresh initial query; give its filled command.+Keep scope across requests/compaction and follow-ups. New traffic/settlement/prices do not change+pinned results. Never silently refresh or compare totals across scopes. Expiry, withdrawn proof,+affected deletion or authority change needs a fresh initial query; give its filled command.Explain recorded decisions, served model/reasoning/publication. Stages are not switches. between_turnrequires proven adjacent completed user turns. continuation_boundary is compression, possibly mid-turn;@@ -220,8 +220,10 @@ highest-known-only rankings, signed paired savings and missing/excluded coveragehas no percentage. Decision overhead is unmeasured. Coding Plan estimates are API-equivalent, not acash bill or proof of equal quality. Ordinary value show preserves historical valuation.-For pending/index_unavailable report missing coverage, progress/last-progress/stall and retry guidance.-timeout/reader_busy are retryable, not empty. Missing/deleted/expired capture proves no routing error.+For pending/index_unavailable report coverage/progress/stall. For timeout/reader_busy follow+error.observation_read and retry the valid scope after retry_after_ms; bound retries, then report the+exact failure. Do not call these an absent subsystem. Narrow content to a request/smaller page.+Missing/deleted/expired capture proves no routing error.Compare task/answer/tools with decision source/group/probability/threshold/competence/reasoning beforeproposing changes. No auto-tuning, quality inference from cost or manual-swap classification proof.Explain uncertainty; authorized edits need a fresh complete baseline and filled recovery commands.@@ -240,7 +242,7 @@ effects; submit submission.arguments and retain file/operation_lookup. Use the Ocapabilities. After uncertainty use operations find/get or replay the **identical file**; never replacea possibly accepted key. Confirmed unaccepted stale input needs fresh Preview. Reuse completed IDs.-Keep notes outside this managed Skill. Honor ownership/user scope for effects, models, unrelated+Keep notes outside this Skill. Honor ownership/user scope for effects, models, unrelatedcontent and cancellation. Authorization persists. Explain quota. Quickstart/status never call models.## Workerdiff --git a/contracts/cli/contract-set.v1.json b/contracts/cli/contract-set.v1.jsonindex 688a2537..8bf491b0 100644--- a/contracts/cli/contract-set.v1.json+++ b/contracts/cli/contract-set.v1.json@@ -1,5 +1,5 @@{- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","files": {"agent-check-request.v1.schema.json": "sha256:fbd9e9b19282a5ed8c3e30b5ae16538bc01318a70e4bb7c7a55e8e0472553feb","agent-launch-descriptor-request.v1.schema.json": "sha256:28f2f9d2d7f9a1618bea0f13383002429bc9e0ff56166cbee86958a245d0fd48",@@ -7,7 +7,7 @@"agent-scan-request.v1.schema.json": "sha256:2112519120417189cfea67dbc23bcbbeabd573073df6a167fa774a5a766c6653","compute-connection-options-request.v1.schema.json": "sha256:4bc609ffeac30f22dbccb34b907b778722fb45c44d7d8d501afb0b74bf8b58ba","compute-scan-request.union.schema.json": "sha256:0b3fa6c790441aab8dc5a2f8002490ffbd19530fc6e7fd251dead24565d31159",- "coverage-manifest.v1.json": "sha256:44c9178f91eba56cd62f8a944cac9979aa065279503612610315f61123ba90a3",+ "coverage-manifest.v1.json": "sha256:d0d6c484d6b5bac06b17e0ea12819a505cd57b130ac90f636dd8514d434126dd","delegation-cancel-request.v1.schema.json": "sha256:0822c53fc16acfcd618fbef66410939f33f634313389a6df4705f414a468b878","delegation-continue-request.v1.schema.json": "sha256:da2f81607f37d154a0fa698a107b37d4b0e9d5a03693dcbcc41e0b7e65db5b93","delegation-get-request.v1.schema.json": "sha256:85b29839c1c508018ebe40e4b2bb5c1dce7c3a87c27efa30b196b2d1b6658471",@@ -19,14 +19,14 @@"local-control-hello.v2.schema.json": "sha256:32fbce882152a886b79bd139d3b4e459b48d94e0634e56ccb189b2c9971ad772","local-control.v2.schema.json": "sha256:4b654de90cd6e333b3953379e5b2e1c5125f1dcbe307b7c7c73f12bb2032d18c","machine-envelope.v2.schema.json": "sha256:23e019a3d024ae302a44105caff4ec95d3fb409a29f936dea1360e543ae6abd6",- "manifest.v1.json": "sha256:9e66240f88a5d5bda52993ad059adbb40b61276458afb9ac923806707ddeef3d",+ "manifest.v1.json": "sha256:3695d96499e34d7cc484357caccd219d3e895d90c34c019652f9680a2fe5e8eb","operation-cancel-request.v1.schema.json": "sha256:0071cbcf27de3bfb7f6644478fe2b73bb5387ebe33db93629ff5672959e8495f","operation-lookup.v1.schema.json": "sha256:0fbaaa15b0e75b4186979bdd856bed0485d55a4bd33a33f3caf3bd92e01a7850","plan-content-change.v2.schema.json": "sha256:3c5ba659a8705e5355653d7fdd9584080f551396f65bc355349c53e38d1352bd","plan-draft-change.v1.schema.json": "sha256:b59d054c4e4d9f23803e269b5e97ce09cc44f46deeccaf6189fa182a8903b478","plan-editor.v2.schema.json": "sha256:bb1cd2184286958753b1d115c03396e5cdadaa0d31253a5a8eb0d6996bfeb654","plan-lifecycle-change.v1.schema.json": "sha256:4b82ce14414cd1d8f8765cfabd535bf5de24ef559b4ae8728ad365fe37a186a9",- "planned-manifest.v1.json": "sha256:19a4509cfe4b00c8b3ad3c5b597dec70aca1854ba1080ba33578af40177ae508",+ "planned-manifest.v1.json": "sha256:27166c3ec3c7a8434b01b704ccb297831da5d027f001029e8d8ce922789ecd2c","prepare-discovered-model-connection-request.v1.schema.json": "sha256:4451d8d7d06160f399d683bc3987ac2bd458af09994e4e0d989687667f6af1bd","publication-checkpoint-change.v1.schema.json": "sha256:6b4c58af4a4610abe76a79b23b2febe06bea2a8fa2aeb008653d22e1cff485d9","session-list-query.v1.schema.json": "sha256:1f71a89fc019b59c3499708c0760b694b8cb8532028b56d98702cd3b2721cd44",diff --git a/contracts/cli/coverage-manifest.v1.json b/contracts/cli/coverage-manifest.v1.jsonindex 1b10733b..28f360a8 100644--- a/contracts/cli/coverage-manifest.v1.json+++ b/contracts/cli/coverage-manifest.v1.json@@ -1233,7 +1233,7 @@}}],- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","schema_version": {"major": 1,"minor": 0diff --git a/contracts/cli/manifest.v1.json b/contracts/cli/manifest.v1.jsonindex e9eff05e..0a7ab78e 100644--- a/contracts/cli/manifest.v1.json+++ b/contracts/cli/manifest.v1.json@@ -5,7 +5,7 @@},"product_contract_revision": "487a807c8dc25cfb24e8b097882cd3bb9716a009","proposal_map_revision": "ab4da93cc49c6ec56a01e046230411ff1ca82300",- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","visibility": "release","commands": [{@@ -2414,7 +2414,7 @@"arguments_and_options": "JSON input: choose exactly one of --request-stdin or --request-file <PATH>. Both read one bounded UTF-8 JSON request with the same strict typed parser; unknown fields, duplicate members, trailing JSON, repeated/mixed sources and human-option mixtures are rejected before Local Control. A file must be regular; relative paths resolve from the current directory (prefix option-shaped filenames with ./). Existing command byte limits and separate protected-input/capability channels remain unchanged. Human options retain the v1 query. Fact-only listing is available to the same-UID Local Control peer; --query searches retained text and therefore still requires a separately delivered protected capability. --request-stdin accepts only a strict v2 sessions intent.","effects": "Read-only; no product side effect.","network_and_model_use": "No model use; any external dependency is declared by the typed Application operation.",- "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.",+ "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Set it when creating a search scope; an existing scope rejects a new or different keyword. Start a separate search scope and retain the original comparison scope. Repeating its original keyword is allowed. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.","output": "One hiroute.machine-envelope/v2 JSON object on stdout; diagnostics belong on stderr.","exit_status": "0 success/accepted; 2 usage; 3 conflict/blocker; 4 capability denied; 5 not found; 6 unavailable; 7 action required; 8 needs attention.","examples": [@@ -2466,7 +2466,7 @@"arguments_and_options": "JSON input: choose exactly one of --request-stdin or --request-file <PATH>. Both read one bounded UTF-8 JSON request with the same strict typed parser; unknown fields, duplicate members, trailing JSON, repeated/mixed sources and human-option mixtures are rejected before Local Control. A file must be regular; relative paths resolve from the current directory (prefix option-shaped filenames with ./). Existing command byte limits and separate protected-input/capability channels remain unchanged. Human options retain the v1 lookup and content defaults to none. Same-UID facts/timeline reads need no extra token; messages, tool content, catalog, ancestry, content pages, and search still require an exact protected capability.","effects": "Read-only; no product side effect.","network_and_model_use": "No model use; any external dependency is declared by the typed Application operation.",- "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.",+ "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Set it when creating a search scope; an existing scope rejects a new or different keyword. Start a separate search scope and retain the original comparison scope. Repeating its original keyword is allowed. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.","output": "One hiroute.machine-envelope/v2 JSON object on stdout; diagnostics belong on stderr.","exit_status": "0 success/accepted; 2 usage; 3 conflict/blocker; 4 capability denied; 5 not found; 6 unavailable; 7 action required; 8 needs attention.","examples": [diff --git a/contracts/cli/planned-manifest.v1.json b/contracts/cli/planned-manifest.v1.jsonindex 0210ca61..89cdddd5 100644--- a/contracts/cli/planned-manifest.v1.json+++ b/contracts/cli/planned-manifest.v1.json@@ -5,7 +5,7 @@},"product_contract_revision": "487a807c8dc25cfb24e8b097882cd3bb9716a009","proposal_map_revision": "ab4da93cc49c6ec56a01e046230411ff1ca82300",- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","visibility": "planned","commands": [{@@ -3864,7 +3864,7 @@"arguments_and_options": "JSON input: choose exactly one of --request-stdin or --request-file <PATH>. Both read one bounded UTF-8 JSON request with the same strict typed parser; unknown fields, duplicate members, trailing JSON, repeated/mixed sources and human-option mixtures are rejected before Local Control. A file must be regular; relative paths resolve from the current directory (prefix option-shaped filenames with ./). Existing command byte limits and separate protected-input/capability channels remain unchanged. Human options retain the v1 query. Fact-only listing is available to the same-UID Local Control peer; --query searches retained text and therefore still requires a separately delivered protected capability. --request-stdin accepts only a strict v2 sessions intent.","effects": "Read-only; no product side effect.","network_and_model_use": "No model use; any external dependency is declared by the typed Application operation.",- "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.",+ "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Set it when creating a search scope; an existing scope rejects a new or different keyword. Start a separate search scope and retain the original comparison scope. Repeating its original keyword is allowed. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.","output": "One hiroute.machine-envelope/v2 JSON object on stdout; diagnostics belong on stderr.","exit_status": "0 success/accepted; 2 usage; 3 conflict/blocker; 4 capability denied; 5 not found; 6 unavailable; 7 action required; 8 needs attention.","examples": [@@ -3916,7 +3916,7 @@"arguments_and_options": "JSON input: choose exactly one of --request-stdin or --request-file <PATH>. Both read one bounded UTF-8 JSON request with the same strict typed parser; unknown fields, duplicate members, trailing JSON, repeated/mixed sources and human-option mixtures are rejected before Local Control. A file must be regular; relative paths resolve from the current directory (prefix option-shaped filenames with ./). Existing command byte limits and separate protected-input/capability channels remain unchanged. Human options retain the v1 lookup and content defaults to none. Same-UID facts/timeline reads need no extra token; messages, tool content, catalog, ancestry, content pages, and search still require an exact protected capability.","effects": "Read-only; no product side effect.","network_and_model_use": "No model use; any external dependency is declared by the typed Application operation.",- "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.",+ "automation": "Same-OS-user Local Control reads retained message/tool content and searches without a separate GUI ticket. Supplied capability FDs remain validated; Gateway keys grant no management authority.\n\nAnalysis journey (opt in; legacy views/cursors keep their meanings):\n hiroute sessions list --view conversations --period 7d --limit 50\n hiroute sessions list --view switches --day 2026-10-16\n hiroute sessions show SESSION --view routing --analysis-scope TOKEN\n hiroute sessions show SESSION --view conversation --request REQUEST --analysis-scope TOKEN\n\nUse --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N; choose one mode. Civil days follow this service host's local day (actual offsets are returned); absolute bounds are UTC milliseconds, half-open. Default is 7d. Analysis scopes return requested/effective bounds, expiry and relation/numeric/price watermarks. Retention may clip their start so used evidence stays retained for the token lifetime. No stable interval and missing native evidence are explicit gaps, not empty-success claims.\n\nRetain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.\n\n--query searches retained content in the conversations list. Set it when creating a search scope; an existing scope rejects a new or different keyword. Start a separate search scope and retain the original comparison scope. Repeating its original keyword is allowed. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.\n\nLegacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing.","output": "One hiroute.machine-envelope/v2 JSON object on stdout; diagnostics belong on stderr.","exit_status": "0 success/accepted; 2 usage; 3 conflict/blocker; 4 capability denied; 5 not found; 6 unavailable; 7 action required; 8 needs attention.","examples": [diff --git a/crates/application-api/src/commands/help.rs b/crates/application-api/src/commands/help.rsindex 91abdf67..3fc3e6f2 100644--- a/crates/application-api/src/commands/help.rs+++ b/crates/application-api/src/commands/help.rs@@ -288,7 +288,7 @@ Use --period today|yesterday|7d|30d, --day YYYY-MM-DD, or --from-ms N --to-ms N;Retain analysis_scope for ranking/switch/content drill-down; follow next_cursor with the same view/filters/limit and no new time selection. Follow-ups can narrow --session/--request, never widen the original Plan/Agent/query scope. New requests or unrelated lineage do not change a pinned range. Expired scope/cursor, withdrawal/full deletion or changed authority require a fresh initial query; do not silently compare totals across refreshed scopes. A page can be shorter than limit; its length is not the total.---query searches retained content in the conversations list. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.+--query searches retained content in the conversations list. Set it when creating a search scope; an existing scope rejects a new or different keyword. Start a separate search scope and retain the original comparison scope. Repeating its original keyword is allowed. Pending/unavailable index is marked partial. --model-switch is invalid for new views; its legacy any|only|exclude behavior is unchanged. Recorded selection/competence stages explain routing; stage boundaries and credential relay are not user-turn model changes. Compare actual accepted endpoints and explicit reasoning settings. continuation_boundary marks a verified native compression boundary, including mid-turn compression; turn_context_reset/unknown/pending never establish adjacent user turns. Missing proof leaves physical sessions separate. Member previews include exact counts and partial markers. Request content is bounded; reference_only/partial entries supply content_id/request_id for typed V2 content pagination, never expose provider private state. Use sessions receipt RECEIPT for immutable routing facts and observation plan-quality samples for stage details. Missing/deleted/expired capture does not prove wrong routing.Legacy sessions list --include-unlinked --query TEXT and sessions show SESSION --content messages-and-tools remain available. Typed V2 accepts sessions/search/session_search, timeline/catalog/ancestry/content, and family-specific analysis requests. Read schema show --command-id sessions.show --include-schema for complete contracts. Observation errors opt in to timeout, reader_busy, index_unavailable or storage_unavailable diagnostics; retry the same scope where still valid. No read performs a classifier call or changes routing."#;diff --git a/crates/observation/Cargo.toml b/crates/observation/Cargo.tomlindex e0a4247f..57cbfee5 100644--- a/crates/observation/Cargo.toml+++ b/crates/observation/Cargo.toml@@ -17,4 +17,5 @@ thiserror.workspace = truezeroize.workspace = true[dev-dependencies]+rusqlite = { workspace = true, features = ["trace"] }tempfile.workspace = truediff --git a/crates/observation/src/analysis/mod.rs b/crates/observation/src/analysis/mod.rsindex 996da861..27f215e7 100644--- a/crates/observation/src/analysis/mod.rs+++ b/crates/observation/src/analysis/mod.rs@@ -11,6 +11,8 @@ mod native_origin;mod prices;mod projection;mod read;+#[cfg(test)]+mod read_admission_tests;mod routing;mod schema;mod scope;diff --git a/crates/observation/src/analysis/read.rs b/crates/observation/src/analysis/read.rsindex 67f785cb..360cf9a8 100644--- a/crates/observation/src/analysis/read.rs+++ b/crates/observation/src/analysis/read.rs@@ -15,6 +15,9 @@ impl LocalObservationStore {capture: Option<&AnalysisPricesV1>,now: i64,) -> Result<ObservationAnalysisPageV1, Error> {+ let deadline = reader+ .query_deadline()+ .unwrap_or_else(|| std::time::Instant::now() + std::time::Duration::from_millis(500));reader.check(now, q.view.content(), q.query.is_some())?;if reader.allowed_runs().is_some() {return Err(Error::Unauthorized);@@ -59,6 +62,20 @@ impl LocalObservationStore {} else {rusqlite::TransactionBehavior::Deferred};+ // Captured traffic and maintenance already serialize through this+ // writer. Join that admission only when persisting a new scope/capture,+ // rather than repeatedly losing SQLite's shorter busy slice to them.+ // Waiting consumes the caller's original deadline. Existing scopes do+ // not take the writer, and external writers still use SQLite's bound.+ let writer = if matches!(behavior, rusqlite::TransactionBehavior::Immediate) {+ Some(+ self.connection+ .try_lock_until(deadline)+ .ok_or(Error::Timeout)?,+ )+ } else {+ None+ };let tx = db.transaction_with_behavior(behavior)?;let (mut s, offset) = if q.analysis_scope.is_some() || q.cursor.is_some() {scope::load(self, &tx, reader, q, now)?@@ -241,6 +258,7 @@ impl LocalObservationStore {scope::save(&tx, &s)?;}tx.commit()?;+ drop(writer);if q.view.content() {views::content(self, &db, reader, &mut page.requests, now)?;}diff --git a/crates/observation/src/analysis/read_admission_tests.rs b/crates/observation/src/analysis/read_admission_tests.rsnew file mode 100644index 00000000..52014e46--- /dev/null+++ b/crates/observation/src/analysis/read_admission_tests.rs@@ -0,0 +1,140 @@+use crate::{DigestAuthority, LocalObservationStore, query_v2::search_tests};+use hiroute_domain::*;+use std::{sync::mpsc, time::Duration};++fn range() -> AnalysisTimeRangeV1 {+ AnalysisTimeRangeV1 {+ requested_from_ms: 0,+ requested_to_ms: 1000,+ effective_from_ms: 0,+ effective_to_ms: 1000,+ from_utc_offset_seconds: 0,+ to_utc_offset_seconds: 0,+ clipped_for_retention: false,+ }+}++fn indexed(root: &std::path::Path) -> LocalObservationStore {+ let store = LocalObservationStore::open(root, DigestAuthority::new([2; 32])).unwrap();+ search_tests::source(&store, b"original needle question");+ let mut builder = crate::text_index::TextIndexBuilder::new(&store).unwrap();+ builder.cycle(&store).unwrap();+ drop(builder);+ store+}++fn query() -> ObservationAnalysisQueryV1 {+ let mut q = ObservationAnalysisQueryV1::new(AnalysisViewV1::Conversations);+ q.query = Some("needle".into());+ q+}++#[test]+fn fresh_keyword_scope_shares_writer_admission_with_capture_inside_the_existing_deadline() {+ let root = tempfile::tempdir().unwrap();+ let store = indexed(root.path());+ std::thread::scope(|threads| {+ let (writer_ready, ready) = mpsc::channel();+ let (release, released) = mpsc::channel();+ let writer_store = &store;+ let writer = threads.spawn(move || {+ let mut db = writer_store.connection.lock();+ let tx = db+ .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate)+ .unwrap();+ writer_ready.send(()).unwrap();+ released.recv().unwrap();+ tx.commit().unwrap();+ });+ ready.recv().unwrap();+ let (started, start) = mpsc::channel();+ let reading_store = &store;+ let read = threads.spawn(move || {+ let reader = search_tests::reader(true)+ .with_query_deadline(std::time::Instant::now() + Duration::from_millis(500));+ started.send(()).unwrap();+ reading_store.observed_analysis(&reader, &query(), Some(&range()), None, 500)+ });+ start.recv().unwrap();+ // A real capture/maintenance transaction can exceed SQLite's 100 ms+ // busy slice without consuming the existing 500 ms query allowance.+ std::thread::sleep(Duration::from_millis(200));+ release.send(()).unwrap();+ writer.join().unwrap();+ let result = read.join().unwrap().unwrap();+ assert_eq!(result.conversations.len(), 1);+ assert!(!result.gaps.iter().any(|gap| gap == "index_unavailable"));+ });+}++#[test]+fn writer_admission_respects_the_callers_remaining_deadline_without_creating_a_scope() {+ let root = tempfile::tempdir().unwrap();+ let store = indexed(root.path());+ std::thread::scope(|threads| {+ let mut db = store.connection.lock();+ let tx = db+ .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate)+ .unwrap();+ let (finished, result) = mpsc::channel();+ let reading_store = &store;+ threads.spawn(move || {+ let reader = search_tests::reader(true)+ .with_query_deadline(std::time::Instant::now() + Duration::from_millis(40));+ finished+ .send(reading_store.observed_analysis(&reader, &query(), Some(&range()), None, 500))+ .unwrap();+ });+ let read = result.recv_timeout(Duration::from_millis(500));+ tx.commit().unwrap();+ drop(db);+ assert!(matches!(+ read.unwrap(),+ Err(crate::query_v2::ObservationV2Error::Busy+ | crate::query_v2::ObservationV2Error::Timeout)+ ));+ assert_eq!(+ store+ .connection+ .lock()+ .query_row(+ "SELECT COUNT(*) FROM observation_analysis_scopes_v1",+ [],+ |r| r.get::<_, u64>(0)+ )+ .unwrap(),+ 0+ );+ });+}++#[test]+fn an_existing_keyword_scope_stays_read_only_while_capture_owns_the_writer() {+ let root = tempfile::tempdir().unwrap();+ let store = indexed(root.path());+ let mut q = query();+ q.analysis_scope = store+ .observed_analysis(&search_tests::reader(true), &q, Some(&range()), None, 500)+ .unwrap()+ .analysis_scope;+ std::thread::scope(|threads| {+ let mut db = store.connection.lock();+ let tx = db+ .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate)+ .unwrap();+ let (finished, result) = mpsc::channel();+ let reading_store = &store;+ let selected = &q;+ threads.spawn(move || {+ let reader = search_tests::reader(true)+ .with_query_deadline(std::time::Instant::now() + Duration::from_millis(200));+ finished+ .send(reading_store.observed_analysis(&reader, selected, None, None, 500))+ .unwrap();+ });+ let read = result.recv_timeout(Duration::from_millis(500));+ tx.commit().unwrap();+ drop(db);+ assert_eq!(read.unwrap().unwrap().conversations.len(), 1);+ });+}diff --git a/crates/observation/src/analysis/schema.rs b/crates/observation/src/analysis/schema.rsindex b4b7e594..a4e00d0c 100644--- a/crates/observation/src/analysis/schema.rs+++ b/crates/observation/src/analysis/schema.rs@@ -163,7 +163,7 @@ pub(super) fn get(db: &Connection, key: &str) -> rusqlite::Result<u64> {.unwrap_or(0))}pub(super) fn set(db: &Connection, key: &str, value: &str) -> rusqlite::Result<()> {- db.execute("INSERT INTO observation_analysis_meta_v1(key,value) VALUES(?1,?2) ON CONFLICT(key) DO UPDATE SET value=excluded.value",params![key,value])?;+ db.execute("INSERT INTO observation_analysis_meta_v1(key,value) VALUES(?1,?2) ON CONFLICT(key) DO UPDATE SET value=excluded.value WHERE observation_analysis_meta_v1.value!=excluded.value",params![key,value])?;Ok(())}pub(super) fn advance(db: &Connection, key: &str) -> rusqlite::Result<u64> {diff --git a/crates/observation/src/content/projection.rs b/crates/observation/src/content/projection.rsindex 50cc68a6..12418051 100644--- a/crates/observation/src/content/projection.rs+++ b/crates/observation/src/content/projection.rs@@ -345,15 +345,28 @@ pub(super) fn validate_stream_coordinates(Ok(())}+// Finish only visits its request/direction/fork before sorting the small result.+// The workspace/content-ID ordering index otherwise makes some planner estimates+// scan all retained content while holding the ingestion writer.+const ACKNOWLEDGED_BLOBS_SQL: &str =+ "SELECT content_id, content_blob_digest FROM content_instances_v2 INDEXED BY content_instances_v2_request+ WHERE workspace_id=?1 AND request_id=?2 AND direction=?3 AND fork_id=?4 AND state='complete'+ ORDER BY content_id";+const MISSING_BLOBS_SQL: &str = "SELECT content_id, content_blob_digest FROM content_instances_v2 INDEXED BY content_instances_v2_request+ WHERE workspace_id=?1 AND request_id=?2 AND direction=?3 AND fork_id=?4 AND state!='complete'+ ORDER BY content_id";++#[cfg(test)]+#[path = "projection_tests.rs"]+mod tests;+pub(super) fn acknowledged_blobs(transaction: &Transaction<'_>,envelope: &ConversationContentEnvelopeV1,) -> Result<Vec<ObservationBlobAcknowledgementV1>, ContentProjectionError> {- let mut statement = transaction.prepare(- "SELECT content_id, content_blob_digest FROM content_instances_v2- WHERE workspace_id=?1 AND request_id=?2 AND direction=?3 AND fork_id=?4 AND state='complete'- ORDER BY content_id",- ).map_err(|_| ContentProjectionError::ActivityStorage)?;+ let mut statement = transaction+ .prepare(ACKNOWLEDGED_BLOBS_SQL)+ .map_err(|_| ContentProjectionError::ActivityStorage)?;statement.query_map(params![@@ -378,11 +391,9 @@ pub(super) fn missing_blobs(transaction: &Transaction<'_>,envelope: &ConversationContentEnvelopeV1,) -> Result<Vec<ObservationBlobAcknowledgementV1>, ContentProjectionError> {- let mut statement = transaction.prepare(- "SELECT content_id, content_blob_digest FROM content_instances_v2- WHERE workspace_id=?1 AND request_id=?2 AND direction=?3 AND fork_id=?4 AND state!='complete'- ORDER BY content_id",- ).map_err(|_| ContentProjectionError::ActivityStorage)?;+ let mut statement = transaction+ .prepare(MISSING_BLOBS_SQL)+ .map_err(|_| ContentProjectionError::ActivityStorage)?;statement.query_map(params![diff --git a/crates/observation/src/content/projection_tests.rs b/crates/observation/src/content/projection_tests.rsnew file mode 100644index 00000000..0a2ec460--- /dev/null+++ b/crates/observation/src/content/projection_tests.rs@@ -0,0 +1,151 @@+use super::{ACKNOWLEDGED_BLOBS_SQL, MISSING_BLOBS_SQL};+use crate::{DigestAuthority, LocalObservationStore};+use hiroute_domain::WorkspaceId;+use rusqlite::{Connection, StatementStatus, params};++fn query(db: &Connection, sql: &str) -> (Vec<(String, String)>, i32) {+ let mut statement = db.prepare(sql).unwrap();+ let rows = statement+ .query_map(+ params![WorkspaceId::DEFAULT, "selected", "request_input", "fork"],+ |row| Ok((row.get(0)?, row.get(1)?)),+ )+ .unwrap()+ .collect::<Result<Vec<_>, _>>()+ .unwrap();+ (rows, statement.get_status(StatementStatus::VmStep))+}++fn terminal_stream_query(sql: &str, complete: bool) {+ let root = tempfile::tempdir().unwrap();+ let store = LocalObservationStore::open(root.path(), DigestAuthority::new([2; 32])).unwrap();+ let mut db = store.connection.lock();+ let tx = db.transaction().unwrap();+ // Metadata-only query fixture: a large older request and a tiny current+ // stream reproduce skewed retained history. Keep the earlier planner stats+ // when a new stream arrives, as in a live store. Native capture is separate.+ let insert = "INSERT INTO content_instances_v2(workspace_id,content_id,message_instance_id,+ request_id,direction,fork_id,part_ordinal,content_kind,canonical_media_type,+ content_blob_digest,has_content_ref,accumulated_byte_count,state,created_at_unix_nanos)+ VALUES(?1,?2,?2,?3,?4,?5,0,'text','text/plain',?6,0,0,?7,100000000)";+ for n in 0..10_000 {+ tx.execute(+ insert,+ params![+ WorkspaceId::DEFAULT,+ format!("retained-{n:05}"),+ "older",+ "request_input",+ "fork",+ format!("digest-{n}"),+ "complete"+ ],+ )+ .unwrap();+ }+ tx.execute_batch("ANALYZE content_instances_v2;").unwrap();+ for (workspace, id, request, direction, fork, state) in [+ (+ WorkspaceId::DEFAULT,+ "z",+ "selected",+ "request_input",+ "fork",+ "complete",+ ),+ (+ WorkspaceId::DEFAULT,+ "a",+ "selected",+ "request_input",+ "fork",+ "complete",+ ),+ (+ WorkspaceId::DEFAULT,+ "pending",+ "selected",+ "request_input",+ "fork",+ "installing",+ ),+ (+ WorkspaceId::DEFAULT,+ "other-direction",+ "selected",+ "response_delivered",+ "fork",+ "complete",+ ),+ (+ WorkspaceId::DEFAULT,+ "other-fork",+ "selected",+ "request_input",+ "other",+ "installing",+ ),+ (+ "workspace-other",+ "other-workspace",+ "selected",+ "request_input",+ "fork",+ "complete",+ ),+ ] {+ tx.execute(+ insert,+ params![+ workspace,+ id,+ request,+ direction,+ fork,+ format!("digest-{id}"),+ state+ ],+ )+ .unwrap();+ }+ tx.commit().unwrap();+ let expected = if complete {+ vec![+ ("a".into(), "digest-a".into()),+ ("z".into(), "digest-z".into()),+ ]+ } else {+ vec![("pending".into(), "digest-pending".into())]+ };+ let (actual, work) = query(&db, sql);+ assert_eq!(actual, expected);+ assert!(+ work < 1_000,+ "finishing a small stream scanned retained history: {work} steps"+ );++ // The observed production failure chose the workspace/content-ID ordering+ // index. Keep that negative control even if planner heuristics later change.+ let broad = sql+ .replace(" INDEXED BY content_instances_v2_request", "")+ .replace(+ "FROM content_instances_v2",+ "FROM content_instances_v2 NOT INDEXED",+ );+ let (same, broad_work) = query(&db, &broad);+ assert_eq!(same, actual);+ assert!(+ broad_work > work * 20,+ "unrelated content must not be visited: {work}/{broad_work}"+ );+}++#[test]+fn terminal_acknowledged_blobs_only_visit_the_selected_stream() {+ terminal_stream_query(ACKNOWLEDGED_BLOBS_SQL, true);+}++#[test]+fn terminal_missing_blobs_only_visit_the_selected_stream() {+ terminal_stream_query(MISSING_BLOBS_SQL, false);+}diff --git a/crates/observation/src/query_v2/README.md b/crates/observation/src/query_v2/README.mdindex 0af2a8a3..aa0234f4 100644--- a/crates/observation/src/query_v2/README.md+++ b/crates/observation/src/query_v2/README.md@@ -51,11 +51,30 @@ keywords avoid loading every content occurrence or folded body. Returned anchorsalways verify the exact folded substring and load original offset maps. Currentreads use returned content identities. A partial covering index on ready-sourceidentities keeps publication checks in metadata; it still requires the complete-lookup publication marker. A workspace-local ready-source set skips complete-digests before inspecting missing sources under the reader's-filters; a missing source index or incomplete publication still reports a gap.+lookup publication marker. Gap detection merges the ordered covering live-source+and ready-source indexes with `EXCEPT`, materializing only unready digests before+opening occurrence metadata under the reader's filters. It avoids both large+temporary complete-source sets and large metadata pages for repeated history;+missing indexes or incomplete publication still report authorized gaps.Scale regressions include distinct source digests and common short keywords;repeating many occurrences of a single blob does not cover these query costs.+The maintenance indexer limits metadata-only scanning separately from body input.+After a stable complete pass it sleeps until the existing source revision or its+writer connection changes; analysis-scope writes do not restart that scan. A+change behind an in-progress cursor requires another pass before declaring idle.+Unchanged analysis checkpoints perform no writes during ordinary idle maintenance.+Fresh analysis scopes and price captures share the existing capture writer's+admission within the caller's original deadline, so sustained native capture does+not repeatedly beat them to SQLite's shorter busy slice. Existing scope reads+retain their concurrent read transactions; content reads release the writer first.+Idle physical blob collection also merges covering identities on a separate read+connection. Only selected unreferenced candidates take the writer, and current+references and paths are rechecked before deleting a file. The batch bound stays+in the candidate merge, including a large orphan backlog.+Terminal content acknowledgement and missing-blob checks use the existing request+index to select their request/direction/fork before sorting content IDs. An ordering+index must not make these capture writes scan unrelated retained history while+analysis waits for the writer.The released `search` intent retains occurrence pagination and its V2 responseshape. It is a distinct supported interface, not the Desktop session-list query.diff --git a/crates/observation/src/query_v2/mod.rs b/crates/observation/src/query_v2/mod.rsindex 648721b6..c342e0b8 100644--- a/crates/observation/src/query_v2/mod.rs+++ b/crates/observation/src/query_v2/mod.rs@@ -13,7 +13,7 @@ mod requests;mod schema;mod search;#[cfg(test)]-mod search_tests;+pub(crate) mod search_tests;#[cfg(test)]mod search_window_tests;mod session_search;diff --git a/crates/observation/src/query_v2/search_tests.rs b/crates/observation/src/query_v2/search_tests.rsindex 47ed5046..5e798568 100644--- a/crates/observation/src/query_v2/search_tests.rs+++ b/crates/observation/src/query_v2/search_tests.rs@@ -3,7 +3,7 @@ use crate::{DigestAuthority, LocalObservationStore, text_index::TextIndexBuilderuse hiroute_domain::*;use rusqlite::params;-pub(super) fn source(store: &LocalObservationStore, text: &[u8]) -> String {+pub(crate) fn source(store: &LocalObservationStore, text: &[u8]) -> String {let digest = store.authority.content_blob_digest("text/plain", text)@@ -18,7 +18,7 @@ pub(super) fn source(store: &LocalObservationStore, text: &[u8]) -> String {VALUES(?1,'content','occurrence','request','request','fork',0,'text','text/plain',?2,0,?3,'complete',100000000)",params![WorkspaceId::DEFAULT,digest,text.len()]).unwrap();digest}-pub(super) fn reader(content: bool) -> ObservationReaderContext {+pub(crate) fn reader(content: bool) -> ObservationReaderContext {ObservationReaderContext::local_user(WorkspaceId::default(),"user".into(),@@ -29,7 +29,7 @@ pub(super) fn reader(content: bool) -> ObservationReaderContext {).unwrap()}-pub(super) fn query(keyword: &str, limit: u16) -> ObservationSearchQueryV2 {+pub(crate) fn query(keyword: &str, limit: u16) -> ObservationSearchQueryV2 {ObservationSearchQueryV2 {agent_id: None,plan_id: None,diff --git a/crates/observation/src/query_v2/session_search.rs b/crates/observation/src/query_v2/session_search.rsindex 2932e72b..479d5e51 100644--- a/crates/observation/src/query_v2/session_search.rs+++ b/crates/observation/src/query_v2/session_search.rs@@ -81,11 +81,11 @@ pub(super) fn candidate_sql(only_switch: bool, exact_gram: bool) -> String {format!("{with}SELECT c.workspace_id,c.request_id,MIN(c.rowid) AS content_rowid- FROM observation_text_blocks_v2 b+ FROM observation_text_lookup_v1+ CROSS JOIN observation_text_blocks_v2 b ON b.rowid=observation_text_lookup_v1.rowidCROSS JOIN content_instances_v2 c INDEXED BY content_instances_v2_search_blobON c.workspace_id=b.workspace AND c.content_blob_digest=b.digest- WHERE b.workspace=?1- AND b.rowid IN(SELECT rowid FROM observation_text_lookup_v1 WHERE observation_text_lookup_v1 MATCH ?11)+ WHERE observation_text_lookup_v1 MATCH ?11 AND b.workspace=?1AND EXISTS(SELECT 1 FROM ({READY_SOURCES}) ready WHERE ready.digest=b.digest){exact} AND {CONTENT_FILTERS}GROUP BY c.workspace_id,c.request_id@@ -159,17 +159,26 @@ fn candidates_for(}pub(super) fn index_partial_sql(only_switch: bool) -> String {- // The uncorrelated, workspace-local ready set is built once. Check the- // digest index before loading content metadata or authorizing a gap; ready- // sources cannot contribute a gap regardless of their visibility. Pin this- // order so the request-search index cannot scatter the complete gap scan.+ // Merge the ordered covering source indexes before opening any occurrence+ // metadata. Materializing the complete live and ready sets first creates+ // large temporary B-trees even when every source is ready. Only the usually+ // small remainder needs materialization and current visibility checks.let filters = all_filters();+ let with = if only_switch {+ format!("{}, ", prefix(true))+ } else {+ "WITH ".into()+ };format!(- "{} SELECT EXISTS(- SELECT 1 FROM content_instances_v2 c INDEXED BY content_instances_v2_live_blob {REQUEST_JOINS}- WHERE (c.content_blob_digest IS NULL OR c.content_blob_digest NOT IN({READY_SOURCES}))- AND {filters}{})",- prefix(only_switch),+ "{with} unready_sources AS MATERIALIZED (+ SELECT DISTINCT content_blob_digest FROM content_instances_v2+ INDEXED BY content_instances_v2_live_blob WHERE workspace_id=?1 AND state='complete'+ EXCEPT {READY_SOURCES} ORDER BY 1+ ) SELECT EXISTS(+ SELECT 1 FROM unready_sources u+ CROSS JOIN content_instances_v2 c INDEXED BY content_instances_v2_search_blob+ ON c.workspace_id=?1 AND c.content_blob_digest IS u.content_blob_digest+ {REQUEST_JOINS} WHERE {filters}{})",switch_filter(only_switch))}diff --git a/crates/observation/src/query_v2/session_search_tests.rs b/crates/observation/src/query_v2/session_search_tests.rsindex 3d5e837a..90db4e18 100644--- a/crates/observation/src/query_v2/session_search_tests.rs+++ b/crates/observation/src/query_v2/session_search_tests.rs@@ -17,6 +17,12 @@ fn live_search_250k_occurrences_and_60k_sources_under_real_deadline() {search_scale(60_000, 250_000);}+#[test]+#[ignore = "targeted retained-native-history scale gate; execute outside ordinary parallel unit tests"]+fn live_search_450k_occurrences_and_100k_sources_under_real_deadline() {+ search_scale(100_000, 450_000);+}+fn search_scale(distinct: usize, occurrences: usize) {let root = tempfile::tempdir().unwrap();let store = LocalObservationStore::open(root.path(), DigestAuthority::new([2; 32])).unwrap();@@ -165,9 +171,23 @@ fn search_scale(distinct: usize, occurrences: usize) {} else if !returned_identities {assert!(plan.iter().any(|(_, _, step)| step- .starts_with("SEARCH c USING INDEX content_instances_v2_live_blob ")+ .starts_with("SEARCH content_instances_v2 USING COVERING INDEX content_instances_v2_live_blob ")&& step.contains("(workspace_id=?)")),- "gap checks must traverse digest order before loading request metadata: {plan:?}"+ "deduplication must read the covering source index, not large metadata pages: {plan:?}"+ );+ assert!(+ plan.iter().any(|(_, _, step)| step == "MERGE (EXCEPT)")+ && plan+ .iter()+ .any(|(_, _, step)| step == "MATERIALIZE unready_sources"),+ "ready history must be subtracted by ordered merge, without materializing complete source sets: {plan:?}"+ );+ assert!(+ plan.iter().any(|(_, _, step)| step.starts_with(+ "SEARCH c USING COVERING INDEX content_instances_v2_search_blob "+ ) && step+ .contains("(workspace_id=? AND content_blob_digest=?)")),+ "only unready source identities may open candidate occurrences: {plan:?}");}assert!(diff --git a/crates/observation/src/store/retention.rs b/crates/observation/src/store/retention.rsindex 053678f3..58dd64f6 100644--- a/crates/observation/src/store/retention.rs+++ b/crates/observation/src/store/retention.rs@@ -6,7 +6,7 @@ use hiroute_domain::{ObservationQueryError, SessionDeletionOutcomeV1, SessionDeletionPreviewV1,SessionDeletionSpecV1, SessionId, TombstoneReason,};-use rusqlite::{Transaction, params};+use rusqlite::{OptionalExtension, Transaction, params};use super::value_rollup::{archive_session_value, delete_session_tombstones, delete_session_value_rollups,@@ -15,12 +15,13 @@ use super::value_rollup::{use super::{LocalObservationStore, increment_store_revision, store_revision};pub(super) const GARBAGE_CANDIDATES_SQL: &str =- "SELECT b.workspace_id, b.blob_digest, b.object_path FROM content_blobs_v2 b- WHERE NOT EXISTS (- SELECT 1 FROM content_instances_v2 c- WHERE c.workspace_id=b.workspace_id- AND c.content_blob_digest=b.blob_digest AND c.state='complete'- ) ORDER BY b.rowid LIMIT ?1";+ "SELECT workspace_id,blob_digest FROM content_blobs_v2+ EXCEPT SELECT workspace_id,content_blob_digest FROM content_instances_v2 WHERE state='complete'+ ORDER BY 1,2 LIMIT ?1";++#[cfg(test)]+#[path = "retention_gc_tests.rs"]+mod gc_tests;impl LocalObservationStore {pub(super) fn preview_session_deletion_base(@@ -319,27 +320,55 @@ impl LocalObservationStore {if limit == 0 || limit > 200 {return Err(ObservationQueryError::InvalidQuery);}+ let candidates = self.garbage_candidates(limit)?;+ self.collect_unreferenced_candidates(&candidates)+ }++ fn garbage_candidates(+ &self,+ limit: usize,+ ) -> Result<Vec<(String, String)>, ObservationQueryError> {+ // The common idle pass is metadata-only. Merge covering identities on+ // its own read connection, without occupying capture/scope admission.+ let connection = rusqlite::Connection::open_with_flags(+ &self.activity_path,+ rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY,+ )+ .map_err(|_| ObservationQueryError::Unavailable)?;+ let mut statement = connection+ .prepare(GARBAGE_CANDIDATES_SQL)+ .map_err(|_| ObservationQueryError::Unavailable)?;+ statement+ .query_map([limit], |row| Ok((row.get(0)?, row.get(1)?)))+ .map_err(|_| ObservationQueryError::Unavailable)?+ .collect::<Result<Vec<_>, _>>()+ .map_err(|_| ObservationQueryError::Corrupt)+ }++ fn collect_unreferenced_candidates(+ &self,+ candidates: &[(String, String)],+ ) -> Result<u64, ObservationQueryError> {+ if candidates.is_empty() {+ return Ok(0);+ }let mut connection = self.connection.lock();- let unreferenced = {- let mut statement = connection- .prepare(GARBAGE_CANDIDATES_SQL)- .map_err(|_| ObservationQueryError::Unavailable)?;- statement- .query_map([limit], |row| {- Ok((- row.get::<_, String>(0)?,- row.get::<_, String>(1)?,- row.get::<_, String>(2)?,- ))- })- .map_err(|_| ObservationQueryError::Unavailable)?- .collect::<Result<Vec<_>, _>>()- .map_err(|_| ObservationQueryError::Corrupt)?- };let transaction = connection- .transaction()+ .transaction_with_behavior(rusqlite::TransactionBehavior::Immediate).map_err(|_| ObservationQueryError::Unavailable)?;- for (workspace, digest, path) in &unreferenced {+ let mut removed = 0;+ for (workspace, digest) in candidates {+ // Candidate discovery is not deletion authority: capture can make+ // a blob live in between. Recheck its current path and references+ // under the writer reservation before touching any physical file.+ let path: Option<String> = transaction.query_row(+ "SELECT b.object_path FROM content_blobs_v2 b WHERE b.workspace_id=?1 AND b.blob_digest=?2+ AND NOT EXISTS(SELECT 1 FROM content_instances_v2 c WHERE c.workspace_id=b.workspace_id AND c.content_blob_digest=b.blob_digest AND c.state='complete')",+ params![workspace,digest], |row|row.get(0),+ ).optional().map_err(|_|ObservationQueryError::Unavailable)?;+ let Some(path) = path else {+ continue;+ };match fs::remove_file(path) {Ok(()) => {}Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}@@ -351,15 +380,16 @@ impl LocalObservationStore {params![workspace, digest],).map_err(|_| ObservationQueryError::Unavailable)?;+ removed += 1;}- if !unreferenced.is_empty() {+ if removed != 0 {increment_store_revision(&transaction).map_err(|_| ObservationQueryError::Unavailable)?;}transaction.commit().map_err(|_| ObservationQueryError::Unavailable)?;- Ok(unreferenced.len() as u64)+ Ok(removed)}fn remove_orphan_blob_files(&self) -> Result<(), ObservationQueryError> {diff --git a/crates/observation/src/store/retention_gc_tests.rs b/crates/observation/src/store/retention_gc_tests.rsnew file mode 100644index 00000000..ef579d8d--- /dev/null+++ b/crates/observation/src/store/retention_gc_tests.rs@@ -0,0 +1,41 @@+use super::*;++#[test]+fn blob_made_live_after_gc_candidate_discovery_keeps_its_file_and_metadata() {+ let root = tempfile::tempdir().unwrap();+ let store =+ LocalObservationStore::open(root.path(), crate::DigestAuthority::new([2; 32])).unwrap();+ let digest = crate::query_v2::search_tests::source(&store, b"captured content");+ store+ .connection+ .lock()+ .execute("UPDATE content_instances_v2 SET state='installing'", [])+ .unwrap();+ let candidates = store.garbage_candidates(32).unwrap();+ assert_eq!(candidates.len(), 1);+ // Reproduce a capture publication between the read snapshot and removal.+ store+ .connection+ .lock()+ .execute("UPDATE content_instances_v2 SET state='complete'", [])+ .unwrap();+ assert_eq!(+ store.collect_unreferenced_candidates(&candidates).unwrap(),+ 0+ );+ assert_eq!(+ std::fs::read(root.path().join("indexed-source")).unwrap(),+ b"captured content"+ );+ assert!(+ store+ .connection+ .lock()+ .query_row(+ "SELECT EXISTS(SELECT 1 FROM content_blobs_v2 WHERE blob_digest=?1)",+ [digest],+ |row| row.get::<_, bool>(0)+ )+ .unwrap()+ );+}diff --git a/crates/observation/src/store/tests/blob_gc.rs b/crates/observation/src/store/tests/blob_gc.rsindex dad839f6..d1ba26c3 100644--- a/crates/observation/src/store/tests/blob_gc.rs+++ b/crates/observation/src/store/tests/blob_gc.rs@@ -38,6 +38,52 @@ fn idle_query_steps(store: &LocalObservationStore) -> i32 {statement.get_status(StatementStatus::VmStep)}+#[test]+fn idle_blob_gc_does_not_take_the_capture_writer_for_candidate_metadata() {+ let root = tempdir().unwrap();+ let store = std::sync::Arc::new(open_store(root.path()));+ seed(&store, 32, 1);+ let writer = store.connection.lock();+ std::thread::scope(|threads| {+ let (finished, result) = std::sync::mpsc::channel();+ let worker_store = std::sync::Arc::clone(&store);+ threads.spawn(move || {+ finished+ .send(worker_store.collect_garbage_batch(32))+ .unwrap();+ });+ let idle = result.recv_timeout(std::time::Duration::from_millis(500));+ // Release even on failure, so the negative control cannot leave a+ // blocked maintenance thread behind when the assertion unwinds.+ drop(writer);+ assert_eq!(idle.unwrap().unwrap(), 0);+ });+}++#[test]+fn blob_gc_candidate_metadata_merges_covering_identities_before_opening_blob_rows() {+ let root = tempdir().unwrap();+ let store = open_store(root.path());+ let db = store.connection.lock();+ let mut statement = db+ .prepare(&format!("EXPLAIN QUERY PLAN {GARBAGE_CANDIDATES_SQL}"))+ .unwrap();+ let plan = statement+ .query_map([32], |row| row.get::<_, String>(3))+ .unwrap()+ .collect::<Result<Vec<_>, _>>()+ .unwrap();+ assert!(+ plan.iter().any(|step| step == "MERGE (EXCEPT)"),+ "idle candidate work must merge ordered covering sources: {plan:?}"+ );+ assert!(+ plan.iter().any(|step| step+ .contains("content_blobs_v2 USING COVERING INDEX sqlite_autoindex_content_blobs_v2_1")),+ "live blob metadata must not be opened: {plan:?}"+ );+}+#[test]fn blob_gc_live_lookup_stays_linear_and_existing_databases_gain_index() {let root = tempdir().unwrap();@@ -56,7 +102,17 @@ fn blob_gc_live_lookup_stays_linear_and_existing_databases_gain_index() {DROP INDEX content_instances_v2_search_blob;",).unwrap();- let without_index = idle_query_steps(&store);+ // Negative control for the original correlated lookup. The current merge+ // algorithm also avoids its quadratic work when an index is unavailable.+ let without_index = {+ let db = store.connection.lock();+ let mut statement = db.prepare(+ "SELECT b.workspace_id,b.blob_digest FROM content_blobs_v2 b WHERE NOT EXISTS(+ SELECT 1 FROM content_instances_v2 c WHERE c.workspace_id=b.workspace_id AND c.content_blob_digest=b.blob_digest AND c.state='complete') ORDER BY b.rowid LIMIT ?1",+ ).unwrap();+ assert!(!statement.exists([32]).unwrap());+ statement.get_status(StatementStatus::VmStep)+ };assert!(without_index > indexed * 20,"regression fixture must expose the expensive old lookup"diff --git a/crates/observation/src/text_index/worker.rs b/crates/observation/src/text_index/worker.rsindex cd976e6a..820d5b9a 100644--- a/crates/observation/src/text_index/worker.rs+++ b/crates/observation/src/text_index/worker.rs@@ -10,6 +10,7 @@ use std::{const BLOCK: usize = 64 * 1024;const OVERLAP: usize = 1024;const SCAN_ROWS: usize = 256;+const SOURCE_SCAN_BUDGET: Duration = Duration::from_millis(50);const NEXT_SOURCE_SQL: &str = "SELECT b.rowid,b.workspace_id,b.blob_digest,b.object_path,b.byte_count,b.media_type FROM content_blobs_v2 bWHERE b.rowid>?1 AND b.rowid<=?2 AND b.state='complete' AND (b.media_type LIKE 'text/%' OR b.media_type='application/json' OR b.media_type LIKE 'application/vnd.hiroute.%')AND NOT EXISTS(SELECT 1 FROM observation_text_index_v2 i WHERE i.workspace=b.workspace_id AND i.digest=b.blob_digest)@@ -32,6 +33,8 @@ pub struct TextIndexBuilder {_owner: crate::maintenance::LocalWorkerGuard,building: Option<Building>,scan_after: i64,+ scan_checkpoint: Option<(u64, u64)>,+ idle_checkpoint: Option<(u64, u64)>,lookup_backfill: super::lookup::Backfill,}impl TextIndexBuilder {@@ -52,6 +55,8 @@ impl TextIndexBuilder {Ok(Self {building: None,scan_after: 0,+ scan_checkpoint: None,+ idle_checkpoint: None,lookup_backfill: Default::default(),_owner: owner,})@@ -62,13 +67,31 @@ impl TextIndexBuilder {pub fn cycle(&mut self, store: &LocalObservationStore) -> Result<usize, Error> {let started = Instant::now();let mut bytes = self.lookup_backfill.cycle(store, started)?;+ let checkpoint = source_checkpoint(store)?;+ if self.building.is_none() && self.idle_checkpoint == Some(checkpoint) {+ return Ok(0);+ }+ self.idle_checkpoint = None;+ self.scan_checkpoint.get_or_insert(checkpoint);let mut completed = 0;while bytes < 8 * 1024 * 1024 && started.elapsed() < Duration::from_millis(500) {if self.building.is_none() {+ // Empty metadata windows consume no byte budget. Do not spend+ // the whole input-processing allowance rescanning ready history.+ if started.elapsed() >= SOURCE_SCAN_BUDGET {+ break;+ }let (building, exhausted) = next(store, &mut self.scan_after)?;self.building = building;if self.building.is_none() {if exhausted {+ // A liveness change behind this pass's cursor requires+ // another pass, even if the tail has already been read.+ let current = source_checkpoint(store)?;+ if self.scan_checkpoint == Some(current) {+ self.idle_checkpoint = Some(current);+ }+ self.scan_checkpoint = None;break;}continue;@@ -138,6 +161,18 @@ impl TextIndexBuilder {}}+fn source_checkpoint(store: &LocalObservationStore) -> Result<(u64, u64), Error> {+ let connection = store.connection.lock();+ // Accepted fact/content mutations and deletion advance the persisted core+ // revision, including compatible writers on other connections. Local changes+ // also catch cache repair and liveness transitions during maintenance. Read+ // scopes written by separate query connections do not restart an idle scan.+ Ok((+ crate::store::store_revision(&connection).map_err(|_| Error::Unavailable)?,+ connection.total_changes(),+ ))+}+fn next(store: &LocalObservationStore, after: &mut i64) -> Result<(Option<Building>, bool), Error> {let source = {let connection = store.connection.lock();diff --git a/crates/observation/src/text_index/worker_tests.rs b/crates/observation/src/text_index/worker_tests.rsindex 8b93a856..efdac3da 100644--- a/crates/observation/src/text_index/worker_tests.rs+++ b/crates/observation/src/text_index/worker_tests.rs@@ -1,6 +1,131 @@use super::*;use rusqlite::StatementStatus;+thread_local! {+ static STATEMENTS: std::cell::Cell<usize> = const { std::cell::Cell::new(0) };+}++fn count_statement(_: &str) {+ STATEMENTS.with(|count| count.set(count.get() + 1));+}++#[test]+fn idle_indexer_does_not_rescan_ready_history_after_settlement_or_analysis_reads() {+ let root = tempfile::tempdir().unwrap();+ let store =+ LocalObservationStore::open(root.path(), crate::DigestAuthority::new([2; 32])).unwrap();+ store.connection.lock().execute_batch(+ "WITH RECURSIVE n(i) AS (VALUES(1) UNION ALL SELECT i+1 FROM n WHERE i<8192)+ INSERT INTO content_blobs_v2 SELECT 'workspace','blob-'||i,'unused',0,'text/plain','complete',1 FROM n;+ INSERT INTO observation_text_index_v2 SELECT workspace_id,blob_digest,'ready',rowid FROM content_blobs_v2;+ INSERT INTO observation_text_lookup_sources_v1 SELECT workspace,digest FROM observation_text_index_v2;",+ ).unwrap();+ store.settle_pending_valuations(16).unwrap();+ store.reconcile_analysis_batch(500).unwrap();+ let mut builder = TextIndexBuilder::new(&store).unwrap();+ store.connection.lock().trace(Some(count_statement));+ for _ in 0..32 {+ builder.cycle(&store).unwrap();+ if builder.idle_checkpoint.is_some() {+ break;+ }+ }+ assert!(builder.idle_checkpoint.is_some());+ for _ in 0..3 {+ // These are the ordinary idle maintenance and query paths that share+ // the production store; neither changes content needing an index.+ let before = store.connection.lock().total_changes();+ assert_eq!(store.settle_pending_valuations(16).unwrap(), 0);+ assert_eq!(store.reconcile_analysis_batch(500).unwrap(), 0);+ assert_eq!(store.connection.lock().total_changes(), before);+ let reader = crate::query_v2::search_tests::reader(true);+ let q = hiroute_domain::ObservationAnalysisQueryV1::new(+ hiroute_domain::AnalysisViewV1::Conversations,+ );+ let range = hiroute_domain::AnalysisTimeRangeV1 {+ requested_from_ms: 0,+ requested_to_ms: 500,+ effective_from_ms: 0,+ effective_to_ms: 500,+ from_utc_offset_seconds: 0,+ to_utc_offset_seconds: 0,+ clipped_for_retention: false,+ };+ store+ .observed_analysis(&reader, &q, Some(&range), None, 500)+ .unwrap();+ STATEMENTS.with(|count| count.set(0));+ assert_eq!(builder.cycle(&store).unwrap(), 0);+ assert!(STATEMENTS.with(|count| count.get()) < 8);+ }+ // A real source revision wakes the same bounded scanner. This control+ // demonstrates the work an unconditional idle rescan would repeat.+ crate::store::increment_store_revision(&store.connection.lock()).unwrap();+ STATEMENTS.with(|count| count.set(0));+ for _ in 0..32 {+ builder.cycle(&store).unwrap();+ if builder.idle_checkpoint.is_some() {+ break;+ }+ }+ assert!(STATEMENTS.with(|count| count.get()) > 32);+}++#[test]+fn indexer_rechecks_a_late_live_source_changed_by_a_peer_writer() {+ let root = tempfile::tempdir().unwrap();+ let store =+ LocalObservationStore::open(root.path(), crate::DigestAuthority::new([2; 32])).unwrap();+ crate::query_v2::search_tests::source(&store, b"late needle");+ store+ .connection+ .lock()+ .execute("UPDATE content_instances_v2 SET state='installing'", [])+ .unwrap();+ let mut builder = TextIndexBuilder::new(&store).unwrap();+ assert_eq!(builder.cycle(&store).unwrap(), 0);+ assert!(builder.idle_checkpoint.is_some());+ let peer = rusqlite::Connection::open(&store.activity_path).unwrap();+ peer.execute("UPDATE content_instances_v2 SET state='complete'", [])+ .unwrap();+ crate::store::increment_store_revision(&peer).unwrap();+ assert_eq!(builder.cycle(&store).unwrap(), 1);+ let hits = store+ .search_observed_text(+ &crate::query_v2::search_tests::reader(true),+ &crate::query_v2::search_tests::query("needle", 10),+ 500,+ )+ .unwrap();+ assert_eq!(hits.hits.len(), 1);+}++#[test]+fn indexer_does_not_declare_idle_when_a_source_changes_behind_the_cursor() {+ let root = tempfile::tempdir().unwrap();+ let store =+ LocalObservationStore::open(root.path(), crate::DigestAuthority::new([2; 32])).unwrap();+ crate::query_v2::search_tests::source(&store, b"late needle");+ store+ .connection+ .lock()+ .execute("UPDATE content_instances_v2 SET state='installing'", [])+ .unwrap();+ let mut builder = TextIndexBuilder::new(&store).unwrap();+ // Resume a bounded pass after it visited the old source. Completion raced+ // that pass, so its final checkpoint must not cover the unvisited change.+ builder.scan_checkpoint = Some(source_checkpoint(&store).unwrap());+ builder.scan_after = 1;+ store+ .connection+ .lock()+ .execute("UPDATE content_instances_v2 SET state='complete'", [])+ .unwrap();+ assert_eq!(builder.cycle(&store).unwrap(), 0);+ assert!(builder.idle_checkpoint.is_none());+ assert_eq!(builder.cycle(&store).unwrap(), 1);+}+#[test]fn index_selection_bounds_database_work_even_after_a_large_indexed_history() {let root = tempfile::tempdir().unwrap();diff --git a/e2e/product/schema/contract-set.v1.json b/e2e/product/schema/contract-set.v1.jsonindex 44d8932f..248fdfbc 100644--- a/e2e/product/schema/contract-set.v1.json+++ b/e2e/product/schema/contract-set.v1.json@@ -1,10 +1,10 @@{- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","files": {"dependency-gate.v1.json": "sha256:b8ac81155266dca921f159d1cfabec6605f2e4bab0320ad6b23a3dbee51e6fa1","product-action.v1.schema.json": "sha256:faf815b27fd292e564a7d51de20c5a2825abbe710315a2364a9932ae3b5e4b7e","product-oracle-evidence.v1.schema.json": "sha256:faab6582bf3fba307b0f9f2949c51726409a209b0fa62e847114e0c6301038c6",- "product-oracle-manifest.v1.json": "sha256:9ae83b0f9e7425452963dc7a11f2f904b98b78b83a62090392c0255c5884ccfc",+ "product-oracle-manifest.v1.json": "sha256:ba3b1fd3c817246edaee08d7682db3042b9b60adcda20da9c606d9b63e1fe376","product-result.v1.schema.json": "sha256:72fc76be77572e40eed7516754911b45fdc9d55a8c089ba74c21427a33809def"},"schema_version": {diff --git a/e2e/product/schema/product-oracle-manifest.v1.json b/e2e/product/schema/product-oracle-manifest.v1.jsonindex 0102b5f0..5fce153e 100644--- a/e2e/product/schema/product-oracle-manifest.v1.json+++ b/e2e/product/schema/product-oracle-manifest.v1.json@@ -1,5 +1,5 @@{- "descriptor_digest": "sha256:d3ee28185884a046b0f7b936de524dc678437fa63c263ac53257ac5f9c28953a",+ "descriptor_digest": "sha256:40921e7e63889bbe1d604e35e186d5c23400f9711ffa247388b05276e4e29268","expected_red_reasons": [],"planned_commands": ["schema.list",diff --git a/issue-spec/specs/session-history/spec.md b/issue-spec/specs/session-history/spec.mdindex eded420f..d32e5c55 100644--- a/issue-spec/specs/session-history/spec.md+++ b/issue-spec/specs/session-history/spec.md@@ -150,6 +150,14 @@ No ready matches is complete empty only when no authorized index gap remains.Four reader permits and the real 500 ms SQLite deadline remain; timeout is retryablefailure, never empty success. Failure clears continuation and retry starts fresh.+#### Scenario: Search repeated native conversation history++- **WHEN** retained native traffic contains many repeated content occurrences and+ large request metadata, while authorized indexes are ready+- **THEN** live and scoped conversation search check source readiness within the+ existing query budget without reading every occurrence's metadata; genuinely+ missing authorized indexes remain partial rather than complete empty results.+### Requirement: Completeness is independent of model success and preview textFact and body failures cannot block the model response. Retain available portions