Solution for long term memory for agent coding CLIs and to facilitate handoff between different agent vendors
Rust
fix(test): make workstream acceptance deterministic
AkitaOnRails960dbff7a6df
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 6c716345..2232b1ce 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Fixed+- The opt-in managed-workstream real-harness acceptance runner now verifies+ context delivery from the managed-run cursor/acknowledgement state and a new+ persisted assistant event instead of requiring the model to quote a prior+ sentinel. Large Claude Code hook packets can be file-backed, so acceptance no+ longer passes or fails based on whether the model chooses to use `Read`.+ The deterministic fake Grok leg covers the same assertion path. (#242)+### Added- `install-mcp --client claude-code --session-aware` now registers anai-memory-owned stdio bridge that forwards Claude Code'sdiff --git a/docs/managed-harness-contributions.md b/docs/managed-harness-contributions.mdindex 3c5ec783..2338d843 100644--- a/docs/managed-harness-contributions.md+++ b/docs/managed-harness-contributions.md@@ -146,9 +146,11 @@ A managed-harness PR should include focused coverage for:- the `AgentKind::ALL` schema invariant when a kind is added;- deterministic fake-process acceptance in`scripts/managed-workstream-acceptance.sh`; and-- a manual real-harness pass that switches into the new harness, observes the- prior sentinel, persists its reply, and resumes its original native session- when revisited.+- a manual real-harness pass that switches into the new harness, records+ successful delivery of its assigned context delta, persists a new assistant+ event, and resumes its original native session when revisited. Do not make+ pass/fail depend on the model quoting packet text: some harnesses externalize+ large hook results to a file, making recall depend on a model tool-use choice.The deterministic phase remains credential-free and suitable for frequentlocal runs. Real model calls stay opt-in and outside CI. Record the tested CLIdiff --git a/docs/managed-workstreams.md b/docs/managed-workstreams.mdindex 160c83f6..48ab700e 100644--- a/docs/managed-workstreams.md+++ b/docs/managed-workstreams.md@@ -339,7 +339,21 @@ native session and verifies automatic fresh-session recovery and repointing.Native session creation, read-only extraction, cross-harness injection, andreturning resume paths are all exercised. Docker wrapper host execution andremote URL preservation are covered separately by the `ai-memory-cli`-packaging tests. Set+packaging tests.++The real-harness phase treats the model as the system under transport, not as+the test oracle. For each leg it records the prior ledger sequence, then+requires a newly imported assistant event from that harness. When a context+delta is expected, it first verifies that the prior ledger endpoint is newer+than that harness's delivery cursor, then requires the latest managed run to+report that exact endpoint as `sync_through` with `context_delivered = 1`. It+does not require the model to quote a prior sentinel: Claude Code may+externalize a large hook result to a file, and whether a model chooses to read+that file is not a deterministic continuity signal. The deterministic fake+Grok cross-harness fixture exercises the same assertion helper without+credentials or model calls.++Set`AI_MEMORY_ACCEPTANCE_HARNESSES="kimi-cli codex"` to select aKimi-to-Codex-to-Kimi round trip (Kimi aliases normalize to the installed`kimi` executable), `AI_MEMORY_ACCEPTANCE_DETERMINISTIC_ONLY=1` to skip modeldiff --git a/scripts/managed-workstream-acceptance.sh b/scripts/managed-workstream-acceptance.shindex 348f6707..a1fc81fa 100755--- a/scripts/managed-workstream-acceptance.sh+++ b/scripts/managed-workstream-acceptance.sh@@ -85,6 +85,83 @@ doneexit 1}+workstream_hex() {+ local name=$1+ sqlite3 "$DATA/db/memory.sqlite" \+ "SELECT lower(hex(id)) FROM workstreams WHERE name = '$name' ORDER BY selected_at DESC LIMIT 1;"+}++latest_workstream_sequence() {+ local hex=$1+ sqlite3 "$DATA/db/memory.sqlite" \+ "SELECT COALESCE(MAX(sequence), 0) FROM workstream_events WHERE workstream_id = x'$hex';"+}++current_delivery_cursor() {+ local hex=$1+ local agent=$2+ sqlite3 "$DATA/db/memory.sqlite" \+ "SELECT COALESCE((+ SELECT delivery_cursor FROM workstream_native_sessions+ WHERE workstream_id = x'$hex'+ AND agent_kind = '$agent'+ AND is_current = 1+ ), 0);"+}++# Assert the deterministic boundaries of one real or fake managed leg:+# the harness produced a new assistant event, and any assigned context delta+# was acknowledged by its hook/launcher delivery path. Model recall is not a+# protocol oracle: large hook results may be file-backed, and whether a model+# chooses to read that file must not decide acceptance.+assert_managed_leg() {+ local hex=$1+ local expected_agent=$2+ local before_sequence=$3+ local before_delivery=$4+ local expect_context=$5+ local log=$6+ local native_id delivery sync_after sync_through context_delivered++ native_id=$(sqlite3 "$DATA/db/memory.sqlite" \+ "SELECT native_session_id FROM workstream_events+ WHERE workstream_id = x'$hex'+ AND sequence > $before_sequence+ AND agent_kind = '$expected_agent'+ AND role = 'assistant'+ ORDER BY sequence DESC LIMIT 1;")+ [ -n "$native_id" ] || {+ printf '%s did not persist a new assistant event after ledger sequence %s\n' \+ "$expected_agent" "$before_sequence" >&2+ tail -120 "$log" >&2+ return 1+ }++ if [ "$expect_context" = 1 ]; then+ if [ "$before_sequence" -le "$before_delivery" ]; then+ printf '%s had no managed context delta to assign (ledger: %s, cursor: %s)\n' \+ "$expected_agent" "$before_sequence" "$before_delivery" >&2+ tail -120 "$log" >&2+ return 1+ fi+ delivery=$(sqlite3 -separator '|' "$DATA/db/memory.sqlite" \+ "SELECT sync_after, sync_through, context_delivered+ FROM managed_runs+ WHERE workstream_id = x'$hex' AND agent_kind = '$expected_agent'+ ORDER BY started_at DESC LIMIT 1;")+ IFS='|' read -r sync_after sync_through context_delivered <<<"$delivery"+ if [ -z "$delivery" ] || [ "${sync_through:-0}" -ne "$before_sequence" ] ||+ [ "${context_delivered:-0}" -ne 1 ]; then+ printf '%s did not acknowledge its assigned managed context delta (state: %s)\n' \+ "$expected_agent" "${delivery:-missing}" >&2+ tail -120 "$log" >&2+ return 1+ fi+ fi++ printf '%s\n' "$native_id"+}+FAKE="$TMP/fake-harness.sh"cat >"$FAKE" <<'EOF'#!/usr/bin/env bash@@ -555,6 +632,8 @@ jq -e \# A fresh grok session joining the established edge-kimi workstream must# receive that workstream's history through `--rules`, with historical tool# activity labelled as completed evidence.+grok_cross_before=$(latest_workstream_sequence "$kimi_ws_hex")+grok_cross_delivery=$(current_delivery_cursor "$kimi_ws_hex" grok)(cd "$REPO"GROK_HOME="$GROK_FAKE_HOME" \@@ -578,6 +657,9 @@ grep -q 'AMWS-FAKE-KIMI' "$TMP/grok-cross-argv.log" || {printf 'the grok --rules packet did not carry the prior harness history\n' >&2exit 1}+assert_managed_leg "$kimi_ws_hex" grok "$grok_cross_before" \+ "$grok_cross_delivery" 1 \+ "$LOGS/edge-grok-cross.log" >/dev/null# A blank first launch remains eligible for one-time native-session adoption.# Use a pseudo-terminal because redirected/scripted launches deliberately skip@@ -771,21 +853,6 @@ install_hook pi "$PI_EXTENSION"install_hook omp "$OMP_EXTENSION"install_hook kimi-code "$KIMI_ACCEPTANCE_HOME/config.toml"-uuid_from_hex() {- local hex=$1- printf '%s-%s-%s-%s-%s\n' \- "${hex:0:8}" "${hex:8:4}" "${hex:12:4}" "${hex:16:4}" "${hex:20:12}"-}--workstream_id() {- local name=$1- local hex- hex=$(sqlite3 "$DATA/db/memory.sqlite" \- "SELECT lower(hex(id)) FROM workstreams WHERE name = '$name' ORDER BY selected_at DESC LIMIT 1;")- [ "${#hex}" -eq 32 ] || return 1- uuid_from_hex "$hex"-}-agent_wire_name() {case "$1" inclaude) printf 'claude-code\n' ;;@@ -802,18 +869,22 @@ uppercase() {run_harness() {local harness=$1local current=$2- local previous=$3- local first_run=$4+ local first_run=$3+ local expect_context=$4local log="$LOGS/real-$harness-$current.log"local promptlocal expected_agent+ local before_sequence=0+ local before_delivery=0+ local existing_hexlocal -a wrapper_args native_argsexpected_agent=$(agent_wire_name "$harness")- if [ -z "$previous" ]; then- prompt="Do not use tools. Reply with exactly: $current"- else- prompt="Do not use tools. In the newest assistant event from the injected ai-memory managed-workstream context, find the final whitespace-delimited token beginning with AMWS-. Reply with exactly that token, one space, then $current."+ existing_hex=$(workstream_hex "$WORKSTREAM_NAME")+ if [ -n "$existing_hex" ]; then+ before_sequence=$(latest_workstream_sequence "$existing_hex")+ before_delivery=$(current_delivery_cursor "$existing_hex" "$expected_agent")fi+ prompt="Reply with exactly: $current"if [ "$first_run" = 1 ]; thenwrapper_args=(--new "$WORKSTREAM_NAME")else@@ -891,48 +962,55 @@ run_harness() {"${wrapper_args[@]}" "$harness" "${native_args[@]}") >"$log" 2>&1fi- local id results event native_id- id=$(workstream_id "$WORKSTREAM_NAME")- results=$("$BIN" --data-dir "$DATA" workstream-search \- --workstream-id "$id" --limit 100 --json "$current")- event=$(jq -c --arg agent "$expected_agent" --arg current "$current" \- '[.[] | select(.agent == $agent and .role == "assistant" and (.content | contains($current)))] | last // empty' \- <<<"$results")- [ -n "$event" ] || {- printf '%s did not persist an assistant event containing %s\n' "$harness" "$current" >&2+ local hex native_id+ hex=$(workstream_hex "$WORKSTREAM_NAME")+ [ "${#hex}" -eq 32 ] || {+ printf 'could not resolve workstream %s after the %s leg\n' \+ "$WORKSTREAM_NAME" "$harness" >&2tail -120 "$log" >&2return 1}- if [ -n "$previous" ] && ! jq -e --arg previous "$previous" \- '.content | contains($previous)' <<<"$event" >/dev/null; then- printf '%s did not demonstrate receipt of prior sentinel %s\n' "$harness" "$previous" >&2- jq -r '.content' <<<"$event" >&2+ if ! native_id=$(assert_managed_leg "$hex" "$expected_agent" \+ "$before_sequence" "$before_delivery" "$expect_context" "$log"); thenreturn 1fi- native_id=$(jq -r '.native_session_id' <<<"$event")printf '%s\n' "$native_id"}WORKSTREAM_NAME="native-acceptance-$(date +%s)-$$"RUN_TAG="$(date +%s)-$$"-previous=""first_harness=${harnesses[0]}+first_agent=$(agent_wire_name "$first_harness")first_native=""+previous_agent=""index=0for harness in "${harnesses[@]}"; docurrent="AMWS-$RUN_TAG-$(uppercase "$harness")"+ current_agent=$(agent_wire_name "$harness")first_run=0+ expect_context=0[ "$index" -ne 0 ] || first_run=1- native_id=$(run_harness "$harness" "$current" "$previous" "$first_run")+ if [ -n "$previous_agent" ] && [ "$current_agent" != "$previous_agent" ]; then+ expect_context=1+ fi+ if ! native_id=$(run_harness "$harness" "$current" "$first_run" \+ "$expect_context"); then+ exit 1+ fiif [ "$index" -eq 0 ]; thenfirst_native=$native_idfi- previous=$current+ previous_agent=$current_agentindex=$((index + 1))donereturn_sentinel="AMWS-$RUN_TAG-$(uppercase "$first_harness")-RETURN"-returned_native=$(run_harness "$first_harness" "$return_sentinel" "$previous" 0)+return_expects_context=0+[ "$first_agent" = "$previous_agent" ] || return_expects_context=1+if ! returned_native=$(run_harness "$first_harness" "$return_sentinel" 0 \+ "$return_expects_context"); then+ exit 1+fi[ "$returned_native" = "$first_native" ] || {printf '%s resumed native session %s, expected %s\n' \"$first_harness" "$returned_native" "$first_native" >&2