Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
fix(temporal): compaction retries like an LLM step; an empty summary fails (1b.27)
trohitg427c4ac43e17
diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdindex 49ae8be3..d0b2092f 100644--- a/docs-internal/TEMPORAL_ARCHITECTURE.md+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -653,7 +653,7 @@ addition. Seven are the core loop activities:| `agent.execute_llm_step` | One LLM turn. The native branch decodes Message Wire V2, rebuilds `ToolDef` values, calls `run_native_llm_step(ChatUnifier, ...)` with SDK retries disabled, heartbeats every 20 s while awaiting the provider (1-minute heartbeat timeout), and returns the exact assistant message + tool calls + normalized usage. Each attempt has a 10-minute `start_to_close_timeout`, or the provider's own when `agent.prepare_payload` recorded `llm_attempt_timeout_seconds` (30 minutes for local and OpenAI-compatible servers, [Native LLM SDK](./native_llm_sdk.md)). Guards against un-invokable payloads: post-filter system-only message lists raise `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)`. || `agent.refresh_tools` | Translates `workflow_ops` add_node ops (`component_kind="tool"` OR `usable_as_tool=True`; never a chat model or the Skills node) into fresh `AgentToolSpec`-derived `tool_payload` entries via `_build_tool_from_node`. Workflow extends `tools` + `tool_index` from the result. || `agent.persist_turn` | Appends the latest human/assistant exchange to memory markdown, trims the window, broadcasts `node.parameters.updated`. |-| `agent.compact_context` | The shared client-side summarizer, run under transcript pressure (step 6 above). Not best-effort: after the activity's retries a failure ends the run with `CompactionError`, because past that point the transcript could only grow until the provider rejected it. |+| `agent.compact_context` | The shared client-side summarizer, run under transcript pressure (step 6 above), with the LLM step's retry policy, attempt timeout and heartbeat. Not best-effort: a transient provider error waits out the outage, and any other failure (an empty summary among them) ends the run with `CompactionError` at the third attempt, because past that point the transcript could only grow until the provider rejected it. || `agent.store_output` | Writes `output_main` / `output_top` / `output_0` so downstream nodes resolve `{{aiAgent.response}}` via `ParameterResolver`. || `agent.broadcast_progress` | Emits `WorkflowEvent.agent_progress` (CloudEvents v1.0) + optional raw-dict `update_node_status` for canvas-glow color. Single helper drives every phase emit. |diff --git a/docs-internal/agent_context_flow.md b/docs-internal/agent_context_flow.mdindex 1a01b45a..ac6fd46a 100644--- a/docs-internal/agent_context_flow.md+++ b/docs-internal/agent_context_flow.md@@ -244,9 +244,10 @@ The trigger (projected tokens against the threshold, transcript bytes againstthe budget), the applied swap (before → after counts, summary size, keptmessages), each clear and each cut log at INFO, as does the activity(rendered chars in, summary chars + summarizer usage out). A summarizer-failure after the activity's retries is terminal for the run-(`CompactionError`), not best-effort: past that point the transcript could-only grow until the provider rejected it.+failure is terminal for the run (`CompactionError`), not best-effort: a+transient provider error waits out the outage like an LLM step, and any other+failure (an empty summary among them) ends the run at the third attempt. Past+that point the transcript could only grow until the provider rejected it.- **Load failures raise.** `agent.prepare_payload` raises`ApplicationError("ConversationLoadFailed")` (retryable) when the rowdiff --git a/docs-internal/llm_gateway.md b/docs-internal/llm_gateway.mdindex a3ceed19..218f1ffb 100644--- a/docs-internal/llm_gateway.md+++ b/docs-internal/llm_gateway.md@@ -43,7 +43,7 @@ It applies only while the gateway serves Gemini, and the owner's own choice alwa**Attribution** (`services/gateway/attribution.py`). Ids come from the structlog log context: `BaseNode.execute` binds the workflow, execution, root execution, chat run, employee job and user ids; the Temporal activity interceptor binds `temporal_workflow_id`, `temporal_run_id`, `temporal_activity_id` and `temporal_attempt`; the LLM-step and compaction activities bind the `attribution` dict their payload carries (`_attribution_ids` in `agent_workflow.py`). `log_context` restores outer values on exit, so a tool node run inside an agent keeps the agent's ids. Headers: session = chat run, else root execution, else execution, else Temporal workflow; tags `source:`, `workflow:`, `user:`; every allowlisted id as metadata. Inside a Temporal activity, calls send a replay-cache namespace from the key's fingerprint and the Temporal workflow, run and activity ids (not the attempt), so a retried activity replays at $0 and a re-run never replays another run's answer. Gemini's native route is the exception: LiteLLM 1.103.4 records its session id and tags but drops the metadata header, and ignores the cache body, so OpenCompany sends no cache body there and a retried Gemini step is billed again.-**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow raises such failures instead of retrying them (`stop_on_user_action`).+**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow handles them like an LLM step (`raise_llm_errors`), so they fail once instead of retrying.## Layoutdiff --git a/docs-internal/memory_compaction.md b/docs-internal/memory_compaction.mdindex ab4b1e20..21f50283 100644--- a/docs-internal/memory_compaction.md+++ b/docs-internal/memory_compaction.md@@ -16,9 +16,12 @@ All providers use the same client-side summarization path:`run_native_llm_step(ChatUnifier, ...)`, then replaces the active memoryhistory with a structured five-section summary. The current agent runtime doesnot enable Anthropic or OpenAI provider-managed compaction. Compaction reduces-active context pressure and never stops an agent early, but on the Temporal-path a summarizer that still fails after the activity's retries ends the run-with `CompactionError` (past that point the transcript could only grow).+active context pressure and never stops an agent early. An empty summary is a+failure that records and replaces nothing. On the Temporal path the summarizer+runs like an LLM step (the step's retry policy, attempt timeout and+heartbeat): a transient provider error waits out the outage, and any other+failure ends the run with `CompactionError` at the third attempt (past that+point the transcript could only grow).**Inspired by:** Claude Code's structured compaction patterndiff --git a/server/services/compaction.py b/server/services/compaction.pyindex 7a92b07b..9ffed7f7 100644--- a/server/services/compaction.py+++ b/server/services/compaction.py@@ -305,16 +305,17 @@ class CompactionService:model: str,*,explicit_max_retries: int = 2,- stop_on_user_action: bool = False,+ raise_llm_errors: bool = False,) -> Dict[str, Any]:"""Perform compaction by summarizing conversation history.Uses the AI service to generate a structured summary following Claude Code pattern.- A failed summarizer call returns ``success: False``. With- ``stop_on_user_action``, a failure only the user can fix (a spent- allowance or exhausted credits, a rejected key, missing access) raises- the :class:`LLMError` instead, since retrying compaction cannot help.+ A failed summarizer call, or an empty summary, returns ``success:+ False`` and records nothing, so an empty reply never replaces the+ conversation. With ``raise_llm_errors`` a summarizer+ :class:`LLMError` raises instead, for a caller that retries or stops+ on it the way it does for an LLM step (the Temporal activity)."""if not self._ai_service:logger.warning("[Compaction] AI service not wired")@@ -388,6 +389,11 @@ Provide a concise but complete summary."""iteration=0,)summary = _extract_text_from_response(response.content)+ if not summary.strip():+ # Saved behind the header below, an empty reply would replace+ # the whole conversation.+ logger.error("[Compaction] Failed: empty summary")+ return {"success": False, "error": "empty summary", "usage": compaction_usage}new_memory = f"# Conversation Summary (Compacted)\n*Generated: {datetime.now(timezone.utc).isoformat()}*\n\n{summary}"@@ -403,7 +409,7 @@ Provide a concise but complete summary."""}except LLMError as e:- if stop_on_user_action and e.requires_user_action:+ if raise_llm_errors:raise# user_message: the raw SDK text can carry payload fragments.logger.error(f"[Compaction] Failed: {e.user_message}")diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.pyindex 8b720a7f..fc5daf8d 100644--- a/server/services/temporal/agent_activities.py+++ b/server/services/temporal/agent_activities.py@@ -2703,7 +2703,9 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:"model": str,}- Returns ``{"success": bool, "summary": str, "usage": dict}``.+ Returns ``{"success": bool, "summary": str, "usage": dict}``. The+ summarizer runs like an LLM step: it heartbeats while it waits, and the+ workflow gives it the step's retry policy and attempt timeout."""from services.compaction import get_compaction_service@@ -2756,22 +2758,26 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:try:async with log_context(**(payload.get("attribution") or {})):- result = await svc.compact_context(- session_id=str(payload.get("session_id") or "default"),- node_id=payload["node_id"],- memory_content=rendered,- provider=payload["provider"],- api_key=await _resolve_activity_api_key(payload),- model=payload["model"],- # Temporal owns activity retries; never repeat an ambiguous provider- # request inside the activity or SDK.- explicit_max_retries=0,- # A spent allowance or a rejected key stops the run with its- # reason, like an LLM step, instead of retrying compaction.- stop_on_user_action=True,+ result = await _await_with_llm_heartbeats(+ svc.compact_context(+ session_id=str(payload.get("session_id") or "default"),+ node_id=payload["node_id"],+ memory_content=rendered,+ provider=payload["provider"],+ api_key=await _resolve_activity_api_key(payload),+ model=payload["model"],+ # Temporal owns activity retries; never repeat an ambiguous provider+ # request inside the activity or SDK.+ explicit_max_retries=0,+ # Every LLM error comes back here, as for an LLM step: a+ # retryable one waits out the outage, and one only the owner+ # can fix stops the run with its reason.+ raise_llm_errors=True,+ ),+ detail="Compacting agent context",)except LLMError as error:- raise _as_temporal_llm_error(error) from None+ raise _as_temporal_llm_error(error, attempt=_llm_activity_attempt()) from Noneif result.get("success") and result.get("summary"):usage = result.get("usage") or {}activity.logger.info(@@ -2781,14 +2787,14 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:f"out={usage.get('output_tokens', 0)})")if not result.get("success") or not result.get("summary"):- # Compaction is the run's pressure-relief valve. Raising (retryable)- # gives transient summarizer failures the activity policy's retry- # budget; if it still cannot compact, the workflow fails the run+ # Compaction is the run's pressure-relief valve. A failure that is no+ # LLM error (an empty summary, a summarizer bug) gets two more tries+ # under the step's unlimited policy, then the workflow fails the run# loudly rather than letting the transcript grow unbounded.raise ApplicationError(f"Compaction failed: {result.get('error') or 'empty summary'}",type="CompactionFailed",- non_retryable=False,+ non_retryable=_llm_activity_attempt() >= 3,)return resultdiff --git a/server/services/temporal/agent_workflow.py b/server/services/temporal/agent_workflow.pyindex a43c58b8..dbe4fd70 100644--- a/server/services/temporal/agent_workflow.py+++ b/server/services/temporal/agent_workflow.py@@ -92,15 +92,15 @@ from .execution_control import ExecutionControl# attempt gets it through ``llm_attempt_timeout``.LLM_STEP_TIMEOUT = timedelta(minutes=10)PERSIST_TURN_TIMEOUT = timedelta(seconds=30)-COMPACT_MEMORY_TIMEOUT = timedelta(minutes=5)LLM_STEP_HEARTBEAT_TIMEOUT = timedelta(minutes=1)def llm_attempt_timeout(payload: Dict[str, Any]) -> timedelta:- """One LLM-step attempt's ``start_to_close_timeout``: the provider's,- as ``agent.prepare_payload`` recorded it (local and OpenAI-compatible- servers), else ``LLM_STEP_TIMEOUT``, which a history recorded without- the key also gets. Options only (RFC-0006 §5 rule 6)."""+ """One LLM attempt's ``start_to_close_timeout`` (the step's and+ compaction's): the provider's, as ``agent.prepare_payload`` recorded it+ (local and OpenAI-compatible servers), else ``LLM_STEP_TIMEOUT``, which+ a history recorded without the key also gets. Options only (RFC-0006+ §5 rule 6)."""seconds = payload.get("llm_attempt_timeout_seconds")return timedelta(seconds=seconds) if seconds else LLM_STEP_TIMEOUT@@ -187,11 +187,12 @@ def _default_max_iterations() -> int:except Exception: # noqa: BLE001return 200-# Retry policy for the agent's own activities (LLM step, persist,-# compact). Tool activities use their plugin's policy. Wave 12 D1:+# Retry policy for the agent's own short activities (persist, progress,+# cleanup). The LLM step and compaction use ``LLM_STEP_RETRY``, and tool+# activities their plugin's policy. Wave 12 D1:# delegates to the shared constant so the policy's# non_retryable_error_types include ``NodeUserError`` — user-correctable-# failures inside the LLM step fail fast instead of burning 3 retries.+# failures fail fast instead of burning 3 retries.AGENT_ACTIVITY_RETRY: RetryPolicy = DEFAULT_ACTIVITY_RETRY# Agent/delegated children start with NO execution/run timeout, and the@@ -2970,12 +2971,15 @@ class AgentWorkflow:await self._wait_until_resumed()async with self._execution_control.action():try:+ # The summarizer is an LLM call: it waits out an outage+ # and gets the provider's attempt timeout, as the step does.compact_result = await workflow.execute_activity("agent.compact_context",args=[compact_payload],activity_id=f"compact-context-{iteration + 1}",- start_to_close_timeout=COMPACT_MEMORY_TIMEOUT,- retry_policy=AGENT_ACTIVITY_RETRY,+ start_to_close_timeout=llm_attempt_timeout(payload),+ heartbeat_timeout=LLM_STEP_HEARTBEAT_TIMEOUT,+ retry_policy=LLM_STEP_RETRY,)except Exception as compact_error:# Compaction is the run's pressure-relief valve. If itdiff --git a/server/tests/services/test_compaction_native.py b/server/tests/services/test_compaction_native.pyindex 11894210..de17ee3f 100644--- a/server/tests/services/test_compaction_native.py+++ b/server/tests/services/test_compaction_native.py@@ -4,7 +4,7 @@ from unittest.mock import AsyncMockimport pytestfrom services.compaction import CompactionService-from services.llm.protocol import LLMResponse, Usage+from services.llm.protocol import LLMError, LLMErrorCategory, LLMResponse, Usageclass _Database:@@ -79,3 +79,47 @@ async def test_compaction_uses_chat_unifier_with_native_messages(monkeypatch):assert "long history" in request["messages"][0].contentassert request["sdk_max_retries"] == 0assert request["translate_errors"] is False+++_REQUEST = dict(session_id="session-1", node_id="agent-1", memory_content="long history " * 20,+ provider="openai", api_key="secret", model="gpt-test")+++def _service(monkeypatch, chat):+ for mock in (_Database.get_or_create_session_token_state, _Database.save_token_metric,+ _Database.save_compaction_event, _Database.update_session_token_state):+ mock.reset_mock()+ registry = SimpleNamespace(get_max_output_tokens=lambda *_args: 2048)+ monkeypatch.setattr("services.model_registry.get_model_registry", lambda: registry)+ service = CompactionService(_Database(), SimpleNamespace(compaction_enabled=True))+ service.set_ai_service(SimpleNamespace(chat_unifier=SimpleNamespace(chat=chat)))+ return service++[email protected]("content", ["", " \n"])+async def test_an_empty_summary_fails_and_replaces_nothing(monkeypatch, content):+ # 1b.27: saved behind its header, an empty reply replaced the conversation.+ reply = LLMResponse(content=content, usage=Usage(input_tokens=40, output_tokens=0, total_tokens=40))+ service = _service(monkeypatch, AsyncMock(return_value=reply))++ result = await service.compact_context(**_REQUEST)++ assert result["success"] is False and result["error"] == "empty summary"+ assert result["usage"]["total_tokens"] == 40 # the call was billed+ _Database.save_token_metric.assert_awaited_once()+ _Database.save_compaction_event.assert_not_awaited()+ assert all(call.args[1].get("cumulative_total") != 0 for call in _Database.update_session_token_state.await_args_list)+++async def test_an_llm_error_is_raised_only_when_asked(monkeypatch):+ error = LLMError("private body", "openai", category=LLMErrorCategory.RATE_LIMIT, retryable=True)+ service = _service(monkeypatch, AsyncMock(side_effect=error))++ result = await service.compact_context(**_REQUEST, explicit_max_retries=0)+ assert result["success"] is False and result["error"] == error.user_message++ with pytest.raises(LLMError) as raised:+ await service.compact_context(**_REQUEST, explicit_max_retries=0, raise_llm_errors=True)+ assert raised.value.category == LLMErrorCategory.RATE_LIMITdiff --git a/server/tests/temporal/test_agent_llm_contract.py b/server/tests/temporal/test_agent_llm_contract.pyindex eb8ae86e..3db0bb05 100644--- a/server/tests/temporal/test_agent_llm_contract.py+++ b/server/tests/temporal/test_agent_llm_contract.py@@ -610,6 +610,45 @@ class TestLLMAttemptTimeout:assert result["success"] is Trueassert options[0]["start_to_close_timeout"] == expected+ @pytest.mark.asyncio+ @pytest.mark.parametrize(("recorded", "expected"), _ATTEMPT_TIMEOUTS)+ async def test_compaction_runs_with_the_llm_steps_options(+ self, monkeypatch, patched_workflow, recorded, expected+ ):+ # 1b.27: the summarizer waits out an outage like the step it relieves.+ from services.temporal._retry_policies import LLM_STEP_RETRY+ from services.temporal.agent_workflow import AgentWorkflow++ steps = []+ options = []++ async def fake_execute_activity(name, *, args, **kwargs):+ if name == "agent.prepare_payload":+ return {**_payload(), "compaction_threshold": 5, **recorded}+ if name == "agent.execute_llm_step":+ steps.append(name)+ if len(steps) == 1:+ return {"kind": "tool_calls", "calls": [{"id": "missing-1", "name": "not_connected", "args": {}}],+ "usage": {"input_tokens": 7, "output_tokens": 3, "total_tokens": 10}}+ return {"kind": "final", "content": "done", "thinking": None,+ "usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6}}+ if name == "agent.compact_context":+ options.append(kwargs)+ return {"success": True, "summary": "compacted history", "usage": {}}+ if name in {"agent.broadcast_progress", "agent.store_output", "agent.skill.clear"}:+ return {}+ raise AssertionError(f"Unexpected activity {name}")++ monkeypatch.setattr(patched_workflow, "execute_activity", fake_execute_activity)++ result = await AgentWorkflow().run({"node_id": "agent-1", "execution_id": "root-run-1"})++ assert result["success"] is True+ assert options[0]["activity_id"] == "compact-context-1"+ assert options[0]["start_to_close_timeout"] == expected+ assert options[0]["heartbeat_timeout"] == timedelta(minutes=1)+ assert options[0]["retry_policy"] is LLM_STEP_RETRY+@pytest.mark.asyncio@pytest.mark.parametrize(("recorded", "expected"), _ATTEMPT_TIMEOUTS)async def test_pi_workflow_takes_the_recorded_timeout(self, monkeypatch, recorded, expected):diff --git a/server/tests/temporal/test_compaction_activity.py b/server/tests/temporal/test_compaction_activity.pynew file mode 100644index 00000000..b45075b0--- /dev/null+++ b/server/tests/temporal/test_compaction_activity.py@@ -0,0 +1,65 @@+"""agent.compact_context fails and waits like an LLM step (1b.27)."""++from dataclasses import replace+from datetime import timedelta+from types import SimpleNamespace+from unittest.mock import AsyncMock++import pytest+from temporalio.exceptions import ApplicationError, ApplicationErrorCategory+from temporalio.testing import ActivityEnvironment++from services.llm.protocol import LLMError, LLMErrorCategory, Message, message_to_wire+from services.temporal import agent_activities+from services.temporal.agent_activities import compact_context++_PAYLOAD = {+ "node_id": "agent-1",+ "provider": "openai",+ "model": "test-model",+ "messages": [message_to_wire(Message(role="user", content="go"))],+}+++def summarizer(monkeypatch):+ service = SimpleNamespace(compact_context=AsyncMock())+ monkeypatch.setattr("services.compaction.get_compaction_service", lambda: service)+ monkeypatch.setattr(agent_activities, "_resolve_activity_api_key", AsyncMock(return_value="owner-key"))+ return service.compact_context+++async def _attempt(number):+ env = ActivityEnvironment()+ env.info = replace(env.info, attempt=number)+ return await env.run(compact_context, _PAYLOAD)++[email protected](("attempt", "final"), [(1, False), (2, False), (3, True), (9, True)])+async def test_an_empty_summary_stops_the_run_at_the_third_attempt(summarizer, attempt, final):+ summarizer.return_value = {"success": False, "error": "empty summary", "usage": {}}+ with pytest.raises(ApplicationError) as raised:+ await _attempt(attempt)+ assert raised.value.type == "CompactionFailed"+ assert raised.value.non_retryable is final+++async def test_a_rate_limit_waits_like_an_llm_step(summarizer):+ summarizer.side_effect = LLMError("private body", "openai", category=LLMErrorCategory.RATE_LIMIT, retryable=True)+ with pytest.raises(ApplicationError) as raised:+ await _attempt(4)+ failure = raised.value+ assert failure.type == "LLMError.rate_limit" and not failure.non_retryable+ assert failure.category == ApplicationErrorCategory.BENIGN+ assert failure.next_retry_delay == timedelta(seconds=40) # the step's backoff at attempt 4+ assert summarizer.await_args.kwargs["raise_llm_errors"] is True+++async def test_a_missing_service_stays_retryable(monkeypatch):+ monkeypatch.setattr("services.compaction.get_compaction_service", lambda: None)+ with pytest.raises(ApplicationError) as raised:+ await _attempt(5)+ assert raised.value.type == "CompactionFailed" and not raised.value.non_retryable