Self-improving AI Employees for everyone
Memory Lifecycle
ARCHIVED 2026-09-04 — Superseded. Documents the retired
input-memorymarkdown model (pre-RFC-0002). Theservices/memory/helper API, vector store and claude session bridge sections were still broadly accurate, but the test citations were not:tests/services/memory/test_markdown.pydoes not exist, the CloudEvents lock istests/test_cloudevents_node_parameters.py, andparse_memory_markdown/trim_markdown_windowhave no test coverage.For current state see:
- agent_context_flow.md and RFC-0002 — conversation store
- memory_compaction.md — token tracking and compaction
- cli_agent_framework.md → Memory bridge — claude session continuity
⚠ Partly superseded by RFC-0002
This document describes the pre-RFC-0002 model, in which
simpleMemoryhung off an agent'sinput-memoryhandle and held the conversation transcript as Markdown. That model no longer exists. Conversation state is now the plain conversation store (agent_conversations, keyed by(workflow_id, generation, agent_node_id)— see agent_context_flow.md; the intermediate "Context journal" described in RFC-0002 was itself replaced), andsimpleMemory(displayed as "Memory") is aToolNodeoninput-toolsholding durable facts the agent explicitly remembers — it has no transcript and no markdown surface.Still accurate here: §2 (the
services/memory/markdown + JSONL helper API), §3 (services.memorycanonicality,append_memory_turns_atomic), §4 (vector store), §6 (theclaude_code_agentstable-cwdsession bridge), §9, and §10.Stale here: §1's framing, §5 (state-clear orchestration —
memory_contentis gone andclear_memorymoved toservices/skills/handlers.py), §7 (the seven-step lifecycle), §8 (per-agent memory shapes), and §11's warning about writing tosimpleMemory.memory_content.For the current model see "Context and Memory (RFC-0002)" in
CLAUDE.mdand RFC-0002.
Canonical home for how conversation memory is loaded, appended, trimmed, archived, cleared, and resumed across every OpenCompany agent. Replaces the partial explanations previously scattered across agent_architecture.md, cli_agent_framework.md, rlm_service.md, and memory_compaction.md.
Related docs:
- memory_compaction.md —
CompactionService, token thresholds, pricing- agent_architecture.md — agent loop that calls these helpers
- cli_agent_framework.md —
claude_code_agentnative session bridge- rlm_service.md —
rlm_agentengine-specific adapter
1. Two storage formats
| Format | File | Used by | Wire shape |
|---|---|---|---|
| Markdown transcript | services/memory/markdown.py |
aiAgent, chatAgent, rlm_agent, claude_code_agent (mirror) | ### **Human/Assistant** (timestamp) blocks under a top-level # Conversation History heading |
| Anthropic-Messages JSONL | services/memory/jsonl.py |
Standalone primitive — not used by any agent today | One JSON object per line: `{"role": "user" |
The markdown format is the visible UI surface — simpleMemory.memory_content
is rendered as a markdown editor in the parameter panel. The normalized JSONL
helpers remain a standalone primitive; the live claude_code_agent bridge
uses claude's own native session JSONL on disk (see §6) and mirrors the
transcript into memory_content via the markdown helpers for visibility.
2. Normalized helper APIs
Markdown transcript
Three pure functions, no I/O, locked by tests/services/memory/:
from services.memory import (
parse_memory_markdown, # str -> List[services.llm.protocol.Message]
append_to_memory_markdown, # (content, role: "human"|"ai", message) -> str
trim_markdown_window, # (content, window_size_pairs) -> (trimmed, removed_texts)
)
parse_memory_markdown— regex-extracts### **(Human|Assistant)**[^\n]*\n(body)blocks; returns normalized nativeMessage(role="user"|"assistant", content=...)instances ready forChatUnifier.append_to_memory_markdown— drops the empty-state placeholder (*No messages yet.*), appends a### **{label}** (YYYY-MM-DD HH:MM:SS)\n{message}\nentry. Timestamp comes fromdatetime.now().trim_markdown_window— keeps the lastwindow_size * 2blocks (N user/assistant pairs); returns(trimmed_content, removed_texts). Removed bodies are returned so the caller can hand them to the long-term vector store (§4).
Empty-state placeholder
# Conversation History
*No messages yet.*
This is also the value clear_agent_session_state resets memory_content to (see §5). Defined as DEFAULT_MEMORY_CONTENT in services/memory/state.py.
Standalone JSONL
The JSONL module is a tested, re-exported primitive, but no production agent currently calls it:
from services.memory import (
parse_jsonl, # str -> List[services.llm.protocol.Message]
append_message, # (text, role, content, **metadata) -> str
trim_window, # (text, window_size_pairs) -> (trimmed, removed_lines)
)
parse_jsonlaccepts onlyuserandassistantrows. String content passes through; list content is reduced to its text blocks, joined with spaces. Tool blocks, unknown roles, invalid JSON, non-string content, and extra metadata are omitted from the returned nativeMessagevalues.append_messagewritesrole,content, and supplied metadata to one JSON object and always ends the result with a newline. Metadata remains on that serialized line, butparse_jsonldoes not copy it intoMessage; this is wire preservation, not a metadata round trip through the parser.trim_windowcounts non-empty rows and keeps the lastwindow_size * 2. When trimming is required, it rebuilds the result without blank lines and returns older serialized rows verbatim; under capacity it returns the original input unchanged.
Claude Code's CLI owns and reads its native session JSONL independently. The helpers above do not parse or rewrite those live CLI files.
Direct process-local message log
services/memory_store.py is a separate
process-local log. add_message normalizes legacy roles (human → user,
ai → assistant); get_messages returns timestamped dictionaries for the
simpleMemory.read operation; and get_native_messages returns the stored
native Message values directly, filtered to user/assistant. Its optional
window is a count of individual messages, not user/assistant pairs.
This log is currently written directly by rlm_agent. It is not the
DB-backed memory_content transcript used passively by aiAgent/chatAgent and
the Claude UI mirror, and it does not survive a process restart.
3. Canonical implementation and call sites
services.memory is the canonical implementation for DB-backed markdown,
the standalone JSONL helpers, vector memory, and reset orchestration. The
direct process-local services.memory_store log described above remains a
separate surface. services.ai keeps only a thin
_parse_memory_markdown compatibility wrapper and aliases the canonical
vector-store cache so older internal imports continue to resolve without
creating duplicate vector state. Runtime appends use
services.memory.runtime.append_memory_turns_atomic, which performs the
append/trim mutation transactionally and idempotently.
| File | Canonical surface |
|---|---|
services/ai.py — native in-process agents |
parse_memory_markdown, append_memory_turns_atomic, canonical vector store |
services/temporal/agent_activities.py — agent.persist_turn.v1 |
append_memory_turns_atomic |
services/cli_agent/service.py — claude_code_agent mirror |
markdown/runtime helpers and canonical vector store |
services/memory/state.py — reset orchestration |
canonical vector-store cache + atomic parameter mutation |
services/memory_store.py — direct process-local log |
add_message, get_messages, get_native_messages, clear_session |
4. Long-term vector store
services/memory/vector_store.py
provides a shared async embedder factory for direct openai.AsyncOpenAI,
ollama.AsyncClient, and optional sentence-transformers adapters, then
ranks entries locally by cosine similarity. Hugging Face model load and
encoding run in a worker thread. create_embedder is the strict constructor:
it raises EmbedderUnavailableError for a missing SDK/optional extra or a
missing OpenAI credential. The higher-level get_memory_vector_store factory
catches availability failures, logs a warning, and returns None, so
long-term archival remains best-effort.
from services.memory import get_memory_vector_store
store = await get_memory_vector_store(
"user-123",
provider="openai",
model="text-embedding-3-small",
auth_service=auth_service,
)
if store:
await store.add_texts(removed_texts) # archive trimmed messages
results = await store.similarity_search(q) # retrieve next-turn context
There is one cache. services.ai._memory_vector_stores is a compatibility
alias to services.memory.vector_store._memory_vector_stores, so state clear,
native agents, and the CLI bridge observe the same stores. Cache identity is
session + provider + model + endpoint + credential fingerprint; plaintext
credentials are never stored in a key. Clearing a session removes and closes
every matching configuration.
5. State clear orchestration
services/memory/state.py exposes one function:
await clear_agent_session_state(
session_id: str,
workflow_id: str = None,
clear_long_term: bool = False, # True → drop vector store
memory_node_id: str | None = None, # When set → reset memory_content + wipe last_session_id
) -> {cleared_vector_store, cleared_todo_keys, cleared_memory_node}
Three stores are cleared in one call:
- Vector store (when
clear_long_term=True) — delete and close every provider/model cache entry for the session (theservices.aialias observes the same deletion). - TodoService — every candidate key (
workflow_id,session_id,"default") is cleared becauseserver/nodes/tool/write_todos/__init__.pyusesctx.workflow_id or ctx.node_id or "default"as the storage key and we want to clear whichever fallback the write path actually used. - simpleMemory node fields (when
memory_node_idprovided) —memory_content→DEFAULT_MEMORY_CONTENT; diagnostic/back-compatlast_session_id→None; orphanmemory_jsonlfield popped if present. Normal Claude continuity uses stablecwdplus--continue, so clearing this field alone neither deletes Claude's native JSONL nor guarantees a fresh Claude conversation.
Frontend clear_memory WS handler (routers/websocket.py:2167) calls this; UI presents a Reset Memory button on the simpleMemory parameter panel.
Workflow Reset archives then clears Memory
Temporal workflow Reset and the Simple Memory panel's Clear Memory are separate lifecycle operations:
| Operation | memory_content / memory_jsonl |
session metadata | vector cache | compaction/token state |
|---|---|---|---|---|
| Workflow Reset | archived, then reset | cleared | cleared | reset for connected sessions |
| Clear Memory | reset | cleared | optional (clear_long_term) |
cleared for the selected session |
workflow_runtime_reset removes the editor's disposable execution projection
and invalidates memory/compaction projections. Before clearing, Reset copies
the authoritative Simple Memory parameters into the archived generation's
runtime_data.simple_memory map. It then resets the live memory-node fields,
connected agent sessions, vector caches, direct memory-store sessions,
conversation rows, and token counters. The next Temporal generation therefore
starts with the empty DB-backed conversation placeholder. Claude's own
on-disk session history is a separate lifecycle; --continue can still find
it until that native session data is cleared.
The archived copy is immutable history; the live node_parameters row remains
the source of truth and is reset to the empty-state value for the next Start.
6. claude_code_agent native session resume bridge
claude_code_agent does not inject memory_content as a system prompt.
When memory is connected, normal spawns use Claude's --continue flag from a
stable working directory. Claude then selects its latest native JSONL
transcript under
<CLAUDE_CONFIG_DIR>/projects/<project_key>/<session_id>.jsonl. Full plumbing
is in cli_agent_framework.md → Memory bridge.
| Mechanism | Where | Why |
|---|---|---|
Stable cwd=repo_root |
AICliSession.cwd() when memory_bound=True |
Keeps Claude's cwd-derived project key constant so --continue searches the same project history |
--continue for normal continuity |
claude_code_agent.execute() |
Every memory-bound task opts into continue_session; with no prior session Claude starts fresh, otherwise it loads the latest session for that cwd |
| Returned session ID | _persist_memory in services/cli_agent/service.py |
Saves the latest returned UUID to simpleMemory.last_session_id for diagnostics and backward compatibility; normal node execution does not read it to select the next session |
memory_content is the UI mirror, not the continuation channel.
_persist_memory appends every successful run's user prompt + assistant
response to it via the same markdown helpers aiAgent uses, and records the
most recent r.session_id in the same DB write. It broadcasts
node_parameters_updated (CloudEvents v1.0 envelope, source_hint="cli") so
the simpleMemory parameter panel auto-refetches mid-run. Explicit
--resume <UUID> remains an internal compatibility/crash-recovery mechanism,
but it is not driven by last_session_id during the normal node path.
Parallel-batch guard. When memory is wired AND len(tasks) > 1,
claude_code_agent raises NodeUserError because concurrent memory-bound CLI
turns under one cwd could race over the same session state.
7. The lifecycle in motion (aiAgent example)
Turn N starts
│
▼
1. The agent plugin/prepare step reads simpleMemory.memory_content from DB
│
▼
2. _parse_memory_markdown(content) → List[native Message]
│
▼
3. (Optional) await get_memory_vector_store(config);
await store.similarity_search(prompt)
→ prepend retrieved long-term context as a system message
│
▼
4. Agent loop runs (run_native_agent_loop: ChatUnifier + native tool calls)
│
▼
5. _track_token_usage(...) — see memory_compaction.md
→ if threshold tripped: CompactionService.compact_context(...) → reset messages
│
▼
6. append_memory_turns_atomic(prompt, response, window_size)
→ append markdown pair + trim in one idempotent DB transaction
│
▼
7. (Optional) await store.add_texts(removed_texts) — long-term archival
F4.B AgentWorkflow runs the work represented by steps 4-7 as separate
Temporal activities.
prepare_payload.v1 records llm_engine and the message wire version;
new executions use native Message Wire V2, preserving Gemini thought
signatures, Anthropic signed/redacted thinking, and OpenAI continuation
metadata. Histories recorded before cutover lack that engine field; their
messages are in a retired wire format and the LLM step refuses them.
persist_turn.v1 appends per turn, so a mid-loop failure does not lose
progress.
8. Engine-specific adapters
| Engine | Memory shape on input | Memory shape on output | Notes |
|---|---|---|---|
aiAgent / chatAgent |
Native Message list parsed from markdown |
Atomic appended pair + trim | Native SDK path; see §7 |
rlm_agent |
DB-backed Markdown injected as REPL pre-context | Prompt/response appended to the direct process-local services.memory_store log; DB Markdown is not updated |
RLM internal recursion uses its own state machine; only entry/exit cross the memory boundary |
claude_code_agent |
Stable cwd + --continue lets Claude select its native JSONL; memory_content ignored on input |
Bridge writes markdown mirror + saves diagnostic/back-compat last_session_id |
See §6 |
9. Compaction trigger
Compaction is a separate concern documented in memory_compaction.md. The trigger point is step 5 in §7. Threshold = model_context_length * compaction_ratio (default 0.8 = 80%, env COMPACTION_RATIO via Settings; per-user override in UserSettings.compaction_ratio; JSON llm_defaults.json:agent.compaction.ratio is the last-resort fallback).
The compaction service is the SSOT for thresholds, native-API integration (Anthropic compact-2026-01-12 beta, OpenAI context_management.compact_threshold), and client-side summarization fallback. Memory lifecycle (this doc) is the SSOT for storage shape and helper signatures.
10. Pytest invariants
server/tests/services/memory/
├── test_markdown.py # parse → append → trim round-trips
├── test_jsonl.py # parse_jsonl + append_message + trim_window
├── test_state.py # cross-store reset behavior
├── test_memory_store.py # legacy/canonical role normalization
└── test_vector_store.py # native cosine ranking and empty operations
The CloudEvents node_parameters_updated envelope emitted on every memory write is locked by tests/test_status_broadcasts.py:test_node_parameters_updated_envelope_shape — see status_broadcaster.md for the contract.
11. What NOT to do
- Don't write to
simpleMemory.memory_contentwithout broadcastingnode_parameters_updated— the parameter panel won't refetch and users will see stale content. - Don't add new local copies of the markdown helpers. Import from
services.memory(or fromservices.memory.markdown/.jsonldirectly). - Don't bypass
clear_agent_session_stateto "just delete the markdown" — TodoService and the vector store will leak across sessions. - Don't inject
memory_contentas a system prompt forclaude_code_agent— stable-cwd--continueis the normal continuation channel;memory_contentis for the human reader. - Don't run claude_code_agent with
memory_bound=Truein parallel batches — the continuity guard raisesNodeUserErrorand you'll burn a credit.