Local-first 🧠memory for Hermes Agent that lives in your Obsidian vault and remembers project context. Nightly 😴 dream passes turn repeat corrections into confirmed preferences with measurable confidence. Adapters ship for Claude Code, Codex, and OpenClaw, with an MCP server for anything else.
Session Lifecycle Capture Durability — not losing or distorting memory at session boundaries
Status: implementation-ready
Author: product-tech-lead (via Phase 0 brainstorm)
Audience: implementation
Branch: feat/session-lifecycle-capture-durability
Problem statement
Open Second Brain captures memory at session boundaries — when the agent's
host runtime (Hermes CLI/TUI/gateway) fires on_session_end and
on_pre_compress. Three real durability gaps exist at exactly those
boundaries:
Interrupted sessions were silently dropped. Before Hermes PRs #50004/#50003/#50312 (now merged), a SIGHUP/SIGTERM/force-quit/restart-drain never flushed the in-flight transcript and never fired
on_session_end. The conversation that was interrupted mid-way left no memory trace. Now that Hermes flushes the transcript and fireson_session_end(interrupted=True), OSB must (a) honour the flag, (b) consume the now-available pre-restart transcript, and (c) avoid double-counting turns when the same session is resumed later. Today OSB swallowsinterruptedinto_kwargsand ignores it.End-of-session capture silently did not fire on
/exit. Hermes #49315 (merged) restored the memory-provideron_session_endhook on CLI/exit(a Phase 4 god-file refactor had rebound the atexit_active_agent_refto the wrong module, so the hook never ran). OSB may carry a local workaround/poll that compensated for the missing hook; that workaround must be verified-unnecessary and removed, otherwise it is dead/misleading code.No post-compaction survival check for pinned anchors. OSB owns the pinned-context store (
pinned.ts) and a PRE-compaction capture pass (pre-compact-extract.tsthat pulls decisions/rules out before compaction). It has no symmetric POST-compaction survival audit: nothing verifies a pinned fact still lives in the ACTIVE (non-summary) region after Hermes summarizes the conversation, and nothing re-asserts the ones that drifted into the summary. This is the exact long-conversation failure mode pinned context exists to fight.
These three share one substrate (provider.py, session-lifecycle.ts,
pinned.ts, pre-compact-extract.ts, the continuity store) and fold into one
release: "do not lose and do not distort memory at session boundaries."
Grounding: what already exists (do NOT rebuild)
- The hook contract already hands
messagestoon_session_end;interruptedarrives as a kwarg. The Python provider is a thin orchestrator — deterministic logic belongs in the TS core, not Python. sync_turnalready persists every turn durably tohermes_home/open-second-brain/session-transcript.jsonl(_persist_turn), so an interrupted session's turns are recoverable from disk independent of the in-memorymessageslist.- The capture boundary in
session-lifecycle.tsalready dedupes sessions and suppresses repeated message text; double-counting protection has a substrate. resolveSessionLineage+recordLineageObservation+isCompressionEvidenceEventalready exist — a resume after an interrupt is already detectable as a lineage child.- The continuity store's generic
recordId()is content-derived overkind + createdAt + sourceRefs + payload;extractPreCompactRecordsalso carries an explicit payload-leveldedupe_keybuilt fromsessionId + turnStart + turnEnd + type + contentHash. Reuse those existing idempotency seams instead of inventing a new ledger. extractPreCompactRecords(kindpre_compact_extract) is the symmetric pre-compaction pass that this release's survival audit complements.on_pre_compressalready exists as a seam;pre-compress-pack.tsis a read-only bundle that the host runtime injects before compression.- All bridge calls go through
_safe_call(fail-soft: hooks never raise into Hermes).
Scope
- t_c181f92b — interrupted-session capture. Surface
interruptedfrom the Pythonon_session_endkwarg as an explicit, absent-by-default field on the host hook payload; thread it intoNormalizedPayload. On an interruptedSessionEnd, consume the pre-restart transcript (themessageslist and/or the persistedsession-transcript.jsonl) so the in-flight turns reach the same extraction as a clean close. Record the interrupted status honestly in the audit record. Suppress double-counting on resume by reusing the existing capture-boundary dedupe, signaldedup_hash, and continuity payload-leveldedupe_keyconventions scoped to the samesessionId/turn range. Fail-soft throughout. - t_3b8fe3a1 — verify
/exithook, drop workaround. Confirm the provider'son_session_endcapture now fires on CLI/exitagainst the merged #49315, and remove any local workaround/poll that compensated for the missing hook (none was found by static grep acrossplugins/hermes/andsrc/, so this is primarily a verification + a guard against regressions). The work product is a regression test asserting end-of-session capture runs on the/exitpath, plus deletion of any dead compensating code discovered during verification. - t_12c8b256 — post-compaction pinned-anchor survival audit. A
deterministic, no-LLM audit that detects a Hermes compaction event (scans the
conversation for Hermes compressor handoff prefixes (
SUMMARY_PREFIX, ASCII fallback, and legacy[CONTEXT SUMMARY]:), hashes the summary body to detect a NEW compaction vs. one already audited), then for each pinned anchor runs a keyword probe scoped to the ACTIVE (non-summary) region. An anchor whose probes hit only inside the summary block counts as drifted and is re-asserted as a reminder block via the existing pinned store; survivors cost zero tokens. Optional windowed-embedding probe is off by default. Fail-open; bounded drift log. Sits as a new module symmetric topre-compact-extract.ts.
Out of scope
- No replacement of existing lifecycle/continuity idempotency seams
with a new ledger. Reuse capture-boundary suppression, signal
dedup_hash, continuityrecordId(), and existing payload-leveldedupe_keypatterns; a new append-only ledger would risk regressions in already-shipped capture paths and violates "additive where possible." (The honest slice of Variant 3, rejected.) - No new
SessionContinuityEventabstraction / unified pipeline. Three parallel additive code paths are lower-risk than a core-store refactor for a single release. Unification is available as a later refactor if the parallel paths prove worth merging. - No preemptive / every-turn blanket re-injection. Re-injection is a repair action (on-drift), not a per-turn injection — keeps prompt-cache stable.
- No embedding index built eagerly. The optional embedding probe, if ever added, stays behind a flag and off by default; the audit defaults to keyword probes only.
- No changes to the host runtime's own compaction or transcript-flush behavior. OSB consumes what Hermes emits; it does not alter Hermes.
- No locale-specific keyword tables. Anchor keywords/probes are derived from the pinned content itself, locale-agnostic.
Chosen approach
Variant 2 — Minimal seam-reuse, three independent additive diffs. Each task is a small diff against an existing seam, gated by an absent-by-default field or flag, so unchanged installs are byte-identical.
- t_c181f92b: an optional
interruptedboolean on the hook payload →NormalizedPayload, consumed by the existingSessionEndbranch. Resume detection reusesresolveSessionLineage; de-dup reuses capture-boundary suppression, signaldedup_hash, and existing continuity payload-leveldedupe_keyconventions rather than a new ledger. - t_3b8fe3a1: verification + deletion of any local workaround in
provider.py; a regression test for the/exitpath. - t_12c8b256: a new post-compaction audit module reusing
extractPreCompactRecords-shaped continuity records andreadPinnedContext, with a bounded drift log, behind an absent-by-default config flag.
This is the lowest-risk path because the release scope itself catalogues the exact substrates each task needs, and the repo conventions explicitly require additive / byte-identical-when-unused behaviour. It avoids both Variant 1's premature TS consolidation (which pushes transcript bytes across the bridge even when the flag is absent) and Variant 3's risky core-store replacement.
Design decisions
- Python stays dumb.
provider.pyonly surfaces theinterruptedkwarg onto the payload and forwards the transcript path; no capture decisions in Python. Deterministic logic, dedupe, and audit live in the TS core. This honours the "core is LLM-free and unit-testable" invariant and keeps the shim fail-soft by construction. interruptedis absent-by-default. The payload field is optional; when Hermes does not set it, behaviour is byte-identical to today. No field is invented when the host does not emit it.- Double-counting is prevented by existing idempotency seams, not by a
new ledger. A resumed session shares the original
sessionId; turns captured by the pre-interrupt flush are suppressed on re-capture via capture-boundary message suppression, signaldedup_hash, and the same payload-leveldedupe_keypattern already used bypre_compact_extract(sessionId + turnStart + turnEnd + type + contentHash). No new idempotency store. - Honest surfacing of an un-honourable interrupted flag. If the
interrupted flag is present but the transcript cannot be consumed (empty
messages, unreadable transcript path), the audit record recordsinterrupted: truewith atranscript_consumed: falsedetail rather than silently coercing to a cleanSessionEnd. No misleading fallback. - Survival audit is the symmetric complement to
extractPreCompactRecords. It reuses the continuity store and the samepre_compact_extract-shaped record (a newpost_compact_auditkind where the lifecycle differs), a Hermes compaction-prefix scan (SUMMARY_PREFIX, ASCII fallback, and legacy prefix) + summary-body hash to detect a NEW compaction, and a keyword probe over the non-summary region. Re-injection is on-drift only. - Deterministic, fail-open audit. The survival audit never calls an LLM; the optional embedding probe is off by default. A probe or re-injection failure must never break a turn (fail-open), and the drift log is bounded.
- English-only, locale-agnostic. Probe keywords are derived from the pinned anchor text itself; no hardcoded locale word lists.
File changes (planned surface)
t_c181f92b
plugins/hermes/provider.py— surfaceinterruptedfrom theon_session_endkwarg onto the hook payload; forward the transcript path. (Thin adapter only.)src/core/brain/session-lifecycle.ts— add optionalinterruptedtoNormalizedPayload; in theSessionEndbranch, wheninterruptedis true, consume the pre-restart transcript and record it honestly; reuse existing lineage + dedupe for resume de-dup.src/cli/brain/verbs/session-hook.ts— pass the field through (payload is already read-from-stdin verbatim; only typed access changes).- Focused tests under
tests/.
t_3b8fe3a1
plugins/hermes/provider.py— delete any workaround/poll discovered during verification (static grep found none; verification will confirm at runtime).- A regression test asserting end-of-session capture fires on the
/exitpath. plugins/hermes/_base.py— only if the fallback ABC needs the field threaded (likely no change;_kwargsalready absorbs it).
t_12c8b256
src/core/brain/post-compact-audit.ts(new) — compaction detection (Hermes compaction-prefix scan + summary-body hash), keyword-probe survival audit scoped to the non-summary region, selective re-assertion viareadPinnedContext/appendPinnedContext, bounded drift log.src/core/brain/pinned.ts— expose any read helper the audit needs (reuse existingreadPinnedContext); no budget change.- A CLI verb / host hook entry that runs the audit post-compaction, behind an
absent-by-default config flag (
post_compact_survival_audit). - Focused tests under
tests/.
Risks
- Double-counting on a real interrupt→resume cycle. Mitigated by reusing
capture-boundary suppression, signal
dedup_hash, and payload-level continuitydedupe_keypatterns scoped tosessionId/turn range, plus an explicit test that resumes an interrupted session and asserts zero duplicate records and zero re-captured turns. Risk: low. - Transcript-consumption correctness under partial flush. The pre-restart
transcript may be partially flushed. Mitigated by treating the persisted
session-transcript.jsonlas the source of truth (already durable) and recordingtranscript_consumedhonestly when only a partial read is possible. Risk: low. - Compaction-detection false positives. A Hermes compaction-prefix literal appearing in user text could masquerade as a compaction boundary. Mitigated by hashing the summary body and comparing against a per-session audited-compaction set, so only a genuinely NEW compaction triggers an audit. Risk: low-medium.
- Survival-audit keyword-probe drift (recall/precision). Keyword probes may miss a drifted anchor that was rephrased. Mitigated by fail-open behaviour (a missed re-assertion is a soft loss, not a turn-breaker) and the optional off-by-default embedding probe for users who want higher recall. Risk: low.
/exitverification flakiness. The verification is runtime-dependent on the merged #49315; CI cannot run a live Hermes. Mitigated by a unit-level regression test that drives the provider hook directly, plus a static confirmation that no workaround code remains. Risk: low.