Self-improving AI Employees for everyone
Fix: agent transcripts outgrow compaction and brick the next firing
ARCHIVED 2026-09-24 — Shipped (branch
fix/agent-compaction-pressure, merged to main). The approved plan plus an as-built record, including where the implementation departed from the plan. Not maintained.For current state see:
- agent_context_flow.md → Transcript size: the rules and the replay contract.
- errors.md entry 28: the incident, the fix, and the one-time recovery for rows saved before it.
Context
errors.log (2026-09-05) records one deployed chat workflow, AI_Assistant_1: an aiAgent on gemini-3.8-flash with writeTodos, simpleMemory and tikhubAction. Each tikhubAction call appended its whole upstream response (roughly 400,000 characters) to the transcript. The agent.execute_llm_step input reached 528 KB at iteration 5 and 1.4 MB from iteration 6 on, with a Temporal PayloadSizeWarning every turn. Compaction never fired, because its only trigger is 80% of a 1,048,576-token window, compared with a running sum. The 1.4 MB transcript was saved to agent_conversations, and the next chat message failed in agent.prepare_payload with non-retryable ConversationTooLarge (1,429,978 bytes against a 1,000,000 cap). Every later firing failed the same way until the user cleared the Context panel.
Decisions
Confirmed by the user 2026-09-09:
ConversationTooLargestays a hard, non-retryable failure; the fix prevents new transcripts from reaching it.- The tool-result cap is per-user through a Settings slider, with an env fallback.
- The wire dedupe (every tool message stores its text twice) is deferred to its own change.
- Compaction's token gate measures the live context, not a cumulative sum.
Approved with the plan on 2026-09-24:
5. Byte pressure clears the oldest tool results rather than summarizing the turn the model has not read. Clearing is free, cannot fail, and is the pattern of Anthropic's context editing (clear_tool_uses) and Claude Code's micro-compaction. The summarizer keeps the latest turn verbatim.
6. New behavior is gated on values recorded in the agent.prepare_payload result, not on a workflow.patched marker. Histories recorded before the change carry none of the new keys and replay through the original code (the technique capture_temporal_routing_input uses to freeze routing flags).
7. Default cap 100,000 characters, external tools only, matching the shell and browser output caps. Exempt: delegated agents (AI_AGENT_TYPES), _builtin_skill, taskManager, _builtin_check_delegated_tasks.
Root causes
- Nothing bounded a tool result before it entered the transcript: the workflow appended
_serialise_tool_result(tool_result)whole, the in-process loopjson.dumps(result)whole. - Every tool message is stored twice (
contentplus a defaulttool_resultblock). Deferred (decision 3). - The token gate compared a running sum of every step's usage (
context_usage_total) with the threshold, and never looked at bytes. - The seed guard's comment said the run degrades with a warning; the code raises.
save_conversationhad no size check. _serialise_tool_resultis shared with the delegation brief and the child's answer text, so the cap could not live inside it.
As built
Phase 1, cap external tool results (both paths).
server/services/tool_output.py:bound_tool_output(text, limit)keeps the firstlimitcharacters and appends[Tool result truncated: showing the first N of M characters. Call the tool again with narrower parameters to see the rest.]. Re-bounding keeps the original M.tool_output_is_capped(node_type)exempts decision 7's list.resolve_tool_output_limit(user_settings, settings)applies per-usertool_result_max_chars>Settings.tool_result_max_chars, with theagent_recursion_limitguard (a positive int).- Settings:
Settings.tool_result_max_chars(envTOOL_RESULT_MAX_CHARS, default 100,000,0disables),.env.template,UserSettings.tool_result_max_chars, a migration whose column default is read from the model field, and the hand-writtenDatabase.get_user_settingsdict. - Temporal:
prepare_agent_payloadrecordstool_result_max_chars;AgentWorkflow.runboundstool_contentright after_serialise_tool_resultfor capped, non-delegation calls. - In-process:
run_native_agent_loop(..., tool_output_limit=None)bounds the serialized result; image blocks are read from the untouched result, sollm_mediarefs survive.AIService.execute_agentandexecute_chat_agentresolve and pass the limit. - Client:
toolResultMaxCharsinclient/src/components/ui/settingsPanel/schema.ts(range in the sharedTOOL_RESULT_MAX_CHARS_RANGE), and a "Tool Result Limit" slider (10K to 200K, step 10K) in the Memory & Compaction section ofSettingsPanel.tsx.
Phase 2, transcript pressure in AgentWorkflow (runs recorded with context_pressure_version 1).
prepare_agent_payloadalso recordstranscript_budget_bytes(three quarters ofTEMPORAL_PAYLOAD_WARN_BYTES, derived intranscript_budget_bytes()) andcontext_pressure_version.server/services/temporal/agent_context_pressure.py(pure, workflow-safe). After each tool turn:clear_old_tool_results: over the budget, results from earlier turns become a placeholder, oldest first, external before agent-internal, down to half the budget.tool_call_idandnamesurvive; the wire is rebuilt so the duplicated block goes too.- Summarize when
projected_request_tokens(the step'stotal_tokens, plus the turn's added characters net of what was cleared, at four per token) reaches the threshold, or whenearlier_turns_over_budget(still over budget with the earlier turns holding more than half of it). Only the turns before the latest are summarized (summarizer_view), and the swap is[system, user(summary + current request), latest turn verbatim]. With nothing older than the latest turn (has_summarizable_history), it skips. clip_latest_turn: if the latest turn alone still overflows, its external results are cut to one shared length found by bisection on the exact serialized size, never below 2,000 characters.
- Without the key, the original cumulative gate and whole-transcript swap run unchanged.
Phase 3, seed guard. Comment rewritten to describe the raise. The message now reads "Clear the conversation from the Context panel, then run again. Settings > Tool Result Limit caps how much one tool call can add to it." _save_conversation logs a WARNING over half of _SEED_TRANSCRIPT_MAX_BYTES. prepare_agent_payload logs a WARNING instead of silently dropping the threshold when it cannot be computed.
Phase 4, docs. errors.md entry 28; agent_context_flow.md (new "Transcript size" section, compaction rewrite, invariant 10); memory_compaction.md; TEMPORAL_ARCHITECTURE.md (the loop pseudo-code, the agent.prepare_payload and agent.compact_context rows: compaction failure is terminal, not best-effort); data_node.md; CLAUDE.md; .env.template.
Differences from the approved plan, and why
- Module location.
server/services/tool_output.py, not underservices/llm/as planned: importing anything there runs that package's__init__, which readsllm_defaults.jsonfrom disk. The new module needs onlyconstantsand the standard library, so workflow code imports it at module level likeget_node_class. - Truncation note. The approved text said "The full result is in this tool's node output panel". That is false for tools that hide their Output section (
simpleMemory,dataSource,visionAnalyze,processManager) and only true for a tool's latest call, so the note only tells the model how to get the rest. - Order: clear, then summarize, then clip. The plan clipped the latest turn inside the clearing step, before any summary; that could cut unread results to fit a history the summary was about to replace. Clipping now runs last, against the room that remains.
- Byte-forced summary. The plan summarized for token pressure only. A transcript whose bulk is text (long user or assistant messages) cannot be brought under budget by clearing tool results, so it also summarizes when the earlier turns alone keep it over budget, but not when the latest turn is the excess (clipping handles that without a paid call).
- Token arithmetic. Checked per provider: every provider's
total_tokensis the whole prompt, cache reads and writes included, plus output (Anthropic adds the cache counters itself; OpenAI and Gemini already count them in input). The gate uses the step'stotal_tokens; no provider change was needed. - Settings getter.
Database.get_user_settingsbuilds its dict by hand, so it needed the new key (the plan said no edit);server/tests/test_user_settings_contract.pyenforces it.
Tests
server/tests/services/test_tool_output.py: bounding and re-bounding, the note, exemptions, limit precedence, and one test that keeps the four copies of the default (.env.template,Settings,UserSettings, client schema) equal.server/tests/temporal/test_agent_context_pressure.py: clearing order and target, pairing, clipping to one length, the minimum length, token arithmetic, summarizer view.server/tests/temporal/test_agent_workflow_pressure.py:AgentWorkflow.runend to end with scripted activities: cap and exemptions, legacy histories untouched, clearing across turns, clipping a single overflowing turn, the next-request gate against the legacy running sum, the kept latest turn, byte-forced summary, compaction off.server/tests/temporal/test_prepare_payload_pressure.py: the recorded keys and precedence, the seed guard at and over the cap, the threshold warning, the save warning.server/tests/temporal/test_agent_workflow.py:TestTranscriptPressureReplaySafetysource guards.server/tests/services/test_native_agent_runtime.py: in-process cap,llm_mediasurvival, skill exemption, no limit.client/src/components/ui/settingsPanel/schema.test.ts: round trip, mapping, default for old rows, range.
Verification
cd server && uv run pytest tests/ -q
bun run typecheck
bun --cwd=client run test
End to end, for the user to run (it needs the real app and data directory): clear the oversized conversation once from the Context panel, run the tikhub prompt that caused the incident, and expect no PayloadSizeWarning, tool messages ending with the truncation note, an INFO line when old results are cleared, and a second chat message that runs.
Deferred and found along the way
- Wire dedupe (decision 3).
- Pressure relief on the in-process path, which gets only the cap.
CompactionService.track()also sums every request'stotal_tokensacross a session, the same running-sum problem the workflow had. - Chunked summarization for models whose window is smaller than the budget.
- From reading
services/pricing.pyandconfig/pricing.json(not exercised by a test): pricing assumes Anthropic's usage semantics for every provider: OpenAI and OpenAI-compatible cached tokens are charged at the input rate and again at the cache rate, o4-mini reasoning tokens are charged twice, and Gemini thinking tokens are never charged. Separate billing change. - Not verified:
BaseNode.execute_as_toolmay hand the model{"result": None}for Task Manager calls whose payload hassuccessbut noresultkey.