Skip to content

Self-improving AI Employees for everyone

fix(temporal): compaction retries like an LLM step; an empty summary fails (1b.27)

trohitg427c4ac43e17
diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.md
index 49ae8be3..d0b2092f 100644
--- a/docs-internal/TEMPORAL_ARCHITECTURE.md
+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md
@@ -653,7 +653,7 @@ addition. Seven are the core loop activities:
| `agent.execute_llm_step` | One LLM turn. The native branch decodes Message Wire V2, rebuilds `ToolDef` values, calls `run_native_llm_step(ChatUnifier, ...)` with SDK retries disabled, heartbeats every 20 s while awaiting the provider (1-minute heartbeat timeout), and returns the exact assistant message + tool calls + normalized usage. Each attempt has a 10-minute `start_to_close_timeout`, or the provider's own when `agent.prepare_payload` recorded `llm_attempt_timeout_seconds` (30 minutes for local and OpenAI-compatible servers, [Native LLM SDK](./native_llm_sdk.md)). Guards against un-invokable payloads: post-filter system-only message lists raise `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)`. |
| `agent.refresh_tools` | Translates `workflow_ops` add_node ops (`component_kind="tool"` OR `usable_as_tool=True`; never a chat model or the Skills node) into fresh `AgentToolSpec`-derived `tool_payload` entries via `_build_tool_from_node`. Workflow extends `tools` + `tool_index` from the result. |
| `agent.persist_turn` | Appends the latest human/assistant exchange to memory markdown, trims the window, broadcasts `node.parameters.updated`. |
-| `agent.compact_context` | The shared client-side summarizer, run under transcript pressure (step 6 above). Not best-effort: after the activity's retries a failure ends the run with `CompactionError`, because past that point the transcript could only grow until the provider rejected it. |
+| `agent.compact_context` | The shared client-side summarizer, run under transcript pressure (step 6 above), with the LLM step's retry policy, attempt timeout and heartbeat. Not best-effort: a transient provider error waits out the outage, and any other failure (an empty summary among them) ends the run with `CompactionError` at the third attempt, because past that point the transcript could only grow until the provider rejected it. |
| `agent.store_output` | Writes `output_main` / `output_top` / `output_0` so downstream nodes resolve `{{aiAgent.response}}` via `ParameterResolver`. |
| `agent.broadcast_progress` | Emits `WorkflowEvent.agent_progress` (CloudEvents v1.0) + optional raw-dict `update_node_status` for canvas-glow color. Single helper drives every phase emit. |
diff --git a/docs-internal/agent_context_flow.md b/docs-internal/agent_context_flow.md
index 1a01b45a..ac6fd46a 100644
--- a/docs-internal/agent_context_flow.md
+++ b/docs-internal/agent_context_flow.md
@@ -244,9 +244,10 @@ The trigger (projected tokens against the threshold, transcript bytes against
the budget), the applied swap (before → after counts, summary size, kept
messages), each clear and each cut log at INFO, as does the activity
(rendered chars in, summary chars + summarizer usage out). A summarizer
-failure after the activity's retries is terminal for the run
-(`CompactionError`), not best-effort: past that point the transcript could
-only grow until the provider rejected it.
+failure is terminal for the run (`CompactionError`), not best-effort: a
+transient provider error waits out the outage like an LLM step, and any other
+failure (an empty summary among them) ends the run at the third attempt. Past
+that point the transcript could only grow until the provider rejected it.
- **Load failures raise.** `agent.prepare_payload` raises
`ApplicationError("ConversationLoadFailed")` (retryable) when the row
diff --git a/docs-internal/llm_gateway.md b/docs-internal/llm_gateway.md
index a3ceed19..218f1ffb 100644
--- a/docs-internal/llm_gateway.md
+++ b/docs-internal/llm_gateway.md
@@ -43,7 +43,7 @@ It applies only while the gateway serves Gemini, and the owner's own choice alwa
**Attribution** (`services/gateway/attribution.py`). Ids come from the structlog log context: `BaseNode.execute` binds the workflow, execution, root execution, chat run, employee job and user ids; the Temporal activity interceptor binds `temporal_workflow_id`, `temporal_run_id`, `temporal_activity_id` and `temporal_attempt`; the LLM-step and compaction activities bind the `attribution` dict their payload carries (`_attribution_ids` in `agent_workflow.py`). `log_context` restores outer values on exit, so a tool node run inside an agent keeps the agent's ids. Headers: session = chat run, else root execution, else execution, else Temporal workflow; tags `source:`, `workflow:`, `user:`; every allowlisted id as metadata. Inside a Temporal activity, calls send a replay-cache namespace from the key's fingerprint and the Temporal workflow, run and activity ids (not the attempt), so a retried activity replays at $0 and a re-run never replays another run's answer. Gemini's native route is the exception: LiteLLM 1.103.4 records its session id and tags but drops the metadata header, and ignores the cache body, so OpenCompany sends no cache body there and a retried Gemini step is billed again.
-**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow raises such failures instead of retrying them (`stop_on_user_action`).
+**Errors** (`services/gateway/errors.py`, `services/llm/protocol.py`). `_classify_error` maps the `budget_exceeded` type to BILLING before any status check, so the run fails once with no retry loop, and a deployment-spawned run pauses its deployment. Failures of gateway calls get gateway wording through `public_message` and the new `public_hint` (allowance used up; model not on the gateway, from `model_cost_map_missing`; reconnect the gateway key). Compaction inside the agent workflow handles them like an LLM step (`raise_llm_errors`), so they fail once instead of retrying.
## Layout
diff --git a/docs-internal/memory_compaction.md b/docs-internal/memory_compaction.md
index ab4b1e20..21f50283 100644
--- a/docs-internal/memory_compaction.md
+++ b/docs-internal/memory_compaction.md
@@ -16,9 +16,12 @@ All providers use the same client-side summarization path:
`run_native_llm_step(ChatUnifier, ...)`, then replaces the active memory
history with a structured five-section summary. The current agent runtime does
not enable Anthropic or OpenAI provider-managed compaction. Compaction reduces
-active context pressure and never stops an agent early, but on the Temporal
-path a summarizer that still fails after the activity's retries ends the run
-with `CompactionError` (past that point the transcript could only grow).
+active context pressure and never stops an agent early. An empty summary is a
+failure that records and replaces nothing. On the Temporal path the summarizer
+runs like an LLM step (the step's retry policy, attempt timeout and
+heartbeat): a transient provider error waits out the outage, and any other
+failure ends the run with `CompactionError` at the third attempt (past that
+point the transcript could only grow).
**Inspired by:** Claude Code's structured compaction pattern
diff --git a/server/services/compaction.py b/server/services/compaction.py
index 7a92b07b..9ffed7f7 100644
--- a/server/services/compaction.py
+++ b/server/services/compaction.py
@@ -305,16 +305,17 @@ class CompactionService:
model: str,
*,
explicit_max_retries: int = 2,
- stop_on_user_action: bool = False,
+ raise_llm_errors: bool = False,
) -> Dict[str, Any]:
"""Perform compaction by summarizing conversation history.
Uses the AI service to generate a structured summary following Claude Code pattern.
- A failed summarizer call returns ``success: False``. With
- ``stop_on_user_action``, a failure only the user can fix (a spent
- allowance or exhausted credits, a rejected key, missing access) raises
- the :class:`LLMError` instead, since retrying compaction cannot help.
+ A failed summarizer call, or an empty summary, returns ``success:
+ False`` and records nothing, so an empty reply never replaces the
+ conversation. With ``raise_llm_errors`` a summarizer
+ :class:`LLMError` raises instead, for a caller that retries or stops
+ on it the way it does for an LLM step (the Temporal activity).
"""
if not self._ai_service:
logger.warning("[Compaction] AI service not wired")
@@ -388,6 +389,11 @@ Provide a concise but complete summary."""
iteration=0,
)
summary = _extract_text_from_response(response.content)
+ if not summary.strip():
+ # Saved behind the header below, an empty reply would replace
+ # the whole conversation.
+ logger.error("[Compaction] Failed: empty summary")
+ return {"success": False, "error": "empty summary", "usage": compaction_usage}
new_memory = f"# Conversation Summary (Compacted)\n*Generated: {datetime.now(timezone.utc).isoformat()}*\n\n{summary}"
@@ -403,7 +409,7 @@ Provide a concise but complete summary."""
}
except LLMError as e:
- if stop_on_user_action and e.requires_user_action:
+ if raise_llm_errors:
raise
# user_message: the raw SDK text can carry payload fragments.
logger.error(f"[Compaction] Failed: {e.user_message}")
diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.py
index 8b720a7f..fc5daf8d 100644
--- a/server/services/temporal/agent_activities.py
+++ b/server/services/temporal/agent_activities.py
@@ -2703,7 +2703,9 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:
"model": str,
}
- Returns ``{"success": bool, "summary": str, "usage": dict}``.
+ Returns ``{"success": bool, "summary": str, "usage": dict}``. The
+ summarizer runs like an LLM step: it heartbeats while it waits, and the
+ workflow gives it the step's retry policy and attempt timeout.
"""
from services.compaction import get_compaction_service
@@ -2756,22 +2758,26 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:
try:
async with log_context(**(payload.get("attribution") or {})):
- result = await svc.compact_context(
- session_id=str(payload.get("session_id") or "default"),
- node_id=payload["node_id"],
- memory_content=rendered,
- provider=payload["provider"],
- api_key=await _resolve_activity_api_key(payload),
- model=payload["model"],
- # Temporal owns activity retries; never repeat an ambiguous provider
- # request inside the activity or SDK.
- explicit_max_retries=0,
- # A spent allowance or a rejected key stops the run with its
- # reason, like an LLM step, instead of retrying compaction.
- stop_on_user_action=True,
+ result = await _await_with_llm_heartbeats(
+ svc.compact_context(
+ session_id=str(payload.get("session_id") or "default"),
+ node_id=payload["node_id"],
+ memory_content=rendered,
+ provider=payload["provider"],
+ api_key=await _resolve_activity_api_key(payload),
+ model=payload["model"],
+ # Temporal owns activity retries; never repeat an ambiguous provider
+ # request inside the activity or SDK.
+ explicit_max_retries=0,
+ # Every LLM error comes back here, as for an LLM step: a
+ # retryable one waits out the outage, and one only the owner
+ # can fix stops the run with its reason.
+ raise_llm_errors=True,
+ ),
+ detail="Compacting agent context",
)
except LLMError as error:
- raise _as_temporal_llm_error(error) from None
+ raise _as_temporal_llm_error(error, attempt=_llm_activity_attempt()) from None
if result.get("success") and result.get("summary"):
usage = result.get("usage") or {}
activity.logger.info(
@@ -2781,14 +2787,14 @@ async def compact_context(payload: Dict[str, Any]) -> Dict[str, Any]:
f"out={usage.get('output_tokens', 0)})"
)
if not result.get("success") or not result.get("summary"):
- # Compaction is the run's pressure-relief valve. Raising (retryable)
- # gives transient summarizer failures the activity policy's retry
- # budget; if it still cannot compact, the workflow fails the run
+ # Compaction is the run's pressure-relief valve. A failure that is no
+ # LLM error (an empty summary, a summarizer bug) gets two more tries
+ # under the step's unlimited policy, then the workflow fails the run
# loudly rather than letting the transcript grow unbounded.
raise ApplicationError(
f"Compaction failed: {result.get('error') or 'empty summary'}",
type="CompactionFailed",
- non_retryable=False,
+ non_retryable=_llm_activity_attempt() >= 3,
)
return result
diff --git a/server/services/temporal/agent_workflow.py b/server/services/temporal/agent_workflow.py
index a43c58b8..dbe4fd70 100644
--- a/server/services/temporal/agent_workflow.py
+++ b/server/services/temporal/agent_workflow.py
@@ -92,15 +92,15 @@ from .execution_control import ExecutionControl
# attempt gets it through ``llm_attempt_timeout``.
LLM_STEP_TIMEOUT = timedelta(minutes=10)
PERSIST_TURN_TIMEOUT = timedelta(seconds=30)
-COMPACT_MEMORY_TIMEOUT = timedelta(minutes=5)
LLM_STEP_HEARTBEAT_TIMEOUT = timedelta(minutes=1)
def llm_attempt_timeout(payload: Dict[str, Any]) -> timedelta:
- """One LLM-step attempt's ``start_to_close_timeout``: the provider's,
- as ``agent.prepare_payload`` recorded it (local and OpenAI-compatible
- servers), else ``LLM_STEP_TIMEOUT``, which a history recorded without
- the key also gets. Options only (RFC-0006 §5 rule 6)."""
+ """One LLM attempt's ``start_to_close_timeout`` (the step's and
+ compaction's): the provider's, as ``agent.prepare_payload`` recorded it
+ (local and OpenAI-compatible servers), else ``LLM_STEP_TIMEOUT``, which
+ a history recorded without the key also gets. Options only (RFC-0006
+ §5 rule 6)."""
seconds = payload.get("llm_attempt_timeout_seconds")
return timedelta(seconds=seconds) if seconds else LLM_STEP_TIMEOUT
@@ -187,11 +187,12 @@ def _default_max_iterations() -> int:
except Exception: # noqa: BLE001
return 200
-# Retry policy for the agent's own activities (LLM step, persist,
-# compact). Tool activities use their plugin's policy. Wave 12 D1:
+# Retry policy for the agent's own short activities (persist, progress,
+# cleanup). The LLM step and compaction use ``LLM_STEP_RETRY``, and tool
+# activities their plugin's policy. Wave 12 D1:
# delegates to the shared constant so the policy's
# non_retryable_error_types include ``NodeUserError`` — user-correctable
-# failures inside the LLM step fail fast instead of burning 3 retries.
+# failures fail fast instead of burning 3 retries.
AGENT_ACTIVITY_RETRY: RetryPolicy = DEFAULT_ACTIVITY_RETRY
# Agent/delegated children start with NO execution/run timeout, and the
@@ -2970,12 +2971,15 @@ class AgentWorkflow:
await self._wait_until_resumed()
async with self._execution_control.action():
try:
+ # The summarizer is an LLM call: it waits out an outage
+ # and gets the provider's attempt timeout, as the step does.
compact_result = await workflow.execute_activity(
"agent.compact_context",
args=[compact_payload],
activity_id=f"compact-context-{iteration + 1}",
- start_to_close_timeout=COMPACT_MEMORY_TIMEOUT,
- retry_policy=AGENT_ACTIVITY_RETRY,
+ start_to_close_timeout=llm_attempt_timeout(payload),
+ heartbeat_timeout=LLM_STEP_HEARTBEAT_TIMEOUT,
+ retry_policy=LLM_STEP_RETRY,
)
except Exception as compact_error:
# Compaction is the run's pressure-relief valve. If it
diff --git a/server/tests/services/test_compaction_native.py b/server/tests/services/test_compaction_native.py
index 11894210..de17ee3f 100644
--- a/server/tests/services/test_compaction_native.py
+++ b/server/tests/services/test_compaction_native.py
@@ -4,7 +4,7 @@ from unittest.mock import AsyncMock
import pytest
from services.compaction import CompactionService
-from services.llm.protocol import LLMResponse, Usage
+from services.llm.protocol import LLMError, LLMErrorCategory, LLMResponse, Usage
class _Database:
@@ -79,3 +79,47 @@ async def test_compaction_uses_chat_unifier_with_native_messages(monkeypatch):
assert "long history" in request["messages"][0].content
assert request["sdk_max_retries"] == 0
assert request["translate_errors"] is False
+
+
+_REQUEST = dict(session_id="session-1", node_id="agent-1", memory_content="long history " * 20,
+ provider="openai", api_key="secret", model="gpt-test")
+
+
+def _service(monkeypatch, chat):
+ for mock in (_Database.get_or_create_session_token_state, _Database.save_token_metric,
+ _Database.save_compaction_event, _Database.update_session_token_state):
+ mock.reset_mock()
+ registry = SimpleNamespace(get_max_output_tokens=lambda *_args: 2048)
+ monkeypatch.setattr("services.model_registry.get_model_registry", lambda: registry)
+ service = CompactionService(_Database(), SimpleNamespace(compaction_enabled=True))
+ service.set_ai_service(SimpleNamespace(chat_unifier=SimpleNamespace(chat=chat)))
+ return service
+
+
[email protected]("content", ["", " \n"])
+async def test_an_empty_summary_fails_and_replaces_nothing(monkeypatch, content):
+ # 1b.27: saved behind its header, an empty reply replaced the conversation.
+ reply = LLMResponse(content=content, usage=Usage(input_tokens=40, output_tokens=0, total_tokens=40))
+ service = _service(monkeypatch, AsyncMock(return_value=reply))
+
+ result = await service.compact_context(**_REQUEST)
+
+ assert result["success"] is False and result["error"] == "empty summary"
+ assert result["usage"]["total_tokens"] == 40 # the call was billed
+ _Database.save_token_metric.assert_awaited_once()
+ _Database.save_compaction_event.assert_not_awaited()
+ assert all(call.args[1].get("cumulative_total") != 0 for call in _Database.update_session_token_state.await_args_list)
+
+
+async def test_an_llm_error_is_raised_only_when_asked(monkeypatch):
+ error = LLMError("private body", "openai", category=LLMErrorCategory.RATE_LIMIT, retryable=True)
+ service = _service(monkeypatch, AsyncMock(side_effect=error))
+
+ result = await service.compact_context(**_REQUEST, explicit_max_retries=0)
+ assert result["success"] is False and result["error"] == error.user_message
+
+ with pytest.raises(LLMError) as raised:
+ await service.compact_context(**_REQUEST, explicit_max_retries=0, raise_llm_errors=True)
+ assert raised.value.category == LLMErrorCategory.RATE_LIMIT
diff --git a/server/tests/temporal/test_agent_llm_contract.py b/server/tests/temporal/test_agent_llm_contract.py
index eb8ae86e..3db0bb05 100644
--- a/server/tests/temporal/test_agent_llm_contract.py
+++ b/server/tests/temporal/test_agent_llm_contract.py
@@ -610,6 +610,45 @@ class TestLLMAttemptTimeout:
assert result["success"] is True
assert options[0]["start_to_close_timeout"] == expected
+ @pytest.mark.asyncio
+ @pytest.mark.parametrize(("recorded", "expected"), _ATTEMPT_TIMEOUTS)
+ async def test_compaction_runs_with_the_llm_steps_options(
+ self, monkeypatch, patched_workflow, recorded, expected
+ ):
+ # 1b.27: the summarizer waits out an outage like the step it relieves.
+ from services.temporal._retry_policies import LLM_STEP_RETRY
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ steps = []
+ options = []
+
+ async def fake_execute_activity(name, *, args, **kwargs):
+ if name == "agent.prepare_payload":
+ return {**_payload(), "compaction_threshold": 5, **recorded}
+ if name == "agent.execute_llm_step":
+ steps.append(name)
+ if len(steps) == 1:
+ return {"kind": "tool_calls", "calls": [{"id": "missing-1", "name": "not_connected", "args": {}}],
+ "usage": {"input_tokens": 7, "output_tokens": 3, "total_tokens": 10}}
+ return {"kind": "final", "content": "done", "thinking": None,
+ "usage": {"input_tokens": 5, "output_tokens": 1, "total_tokens": 6}}
+ if name == "agent.compact_context":
+ options.append(kwargs)
+ return {"success": True, "summary": "compacted history", "usage": {}}
+ if name in {"agent.broadcast_progress", "agent.store_output", "agent.skill.clear"}:
+ return {}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(patched_workflow, "execute_activity", fake_execute_activity)
+
+ result = await AgentWorkflow().run({"node_id": "agent-1", "execution_id": "root-run-1"})
+
+ assert result["success"] is True
+ assert options[0]["activity_id"] == "compact-context-1"
+ assert options[0]["start_to_close_timeout"] == expected
+ assert options[0]["heartbeat_timeout"] == timedelta(minutes=1)
+ assert options[0]["retry_policy"] is LLM_STEP_RETRY
+
@pytest.mark.asyncio
@pytest.mark.parametrize(("recorded", "expected"), _ATTEMPT_TIMEOUTS)
async def test_pi_workflow_takes_the_recorded_timeout(self, monkeypatch, recorded, expected):
diff --git a/server/tests/temporal/test_compaction_activity.py b/server/tests/temporal/test_compaction_activity.py
new file mode 100644
index 00000000..b45075b0
--- /dev/null
+++ b/server/tests/temporal/test_compaction_activity.py
@@ -0,0 +1,65 @@
+"""agent.compact_context fails and waits like an LLM step (1b.27)."""
+
+from dataclasses import replace
+from datetime import timedelta
+from types import SimpleNamespace
+from unittest.mock import AsyncMock
+
+import pytest
+from temporalio.exceptions import ApplicationError, ApplicationErrorCategory
+from temporalio.testing import ActivityEnvironment
+
+from services.llm.protocol import LLMError, LLMErrorCategory, Message, message_to_wire
+from services.temporal import agent_activities
+from services.temporal.agent_activities import compact_context
+
+_PAYLOAD = {
+ "node_id": "agent-1",
+ "provider": "openai",
+ "model": "test-model",
+ "messages": [message_to_wire(Message(role="user", content="go"))],
+}
+
+
+def summarizer(monkeypatch):
+ service = SimpleNamespace(compact_context=AsyncMock())
+ monkeypatch.setattr("services.compaction.get_compaction_service", lambda: service)
+ monkeypatch.setattr(agent_activities, "_resolve_activity_api_key", AsyncMock(return_value="owner-key"))
+ return service.compact_context
+
+
+async def _attempt(number):
+ env = ActivityEnvironment()
+ env.info = replace(env.info, attempt=number)
+ return await env.run(compact_context, _PAYLOAD)
+
+
[email protected](("attempt", "final"), [(1, False), (2, False), (3, True), (9, True)])
+async def test_an_empty_summary_stops_the_run_at_the_third_attempt(summarizer, attempt, final):
+ summarizer.return_value = {"success": False, "error": "empty summary", "usage": {}}
+ with pytest.raises(ApplicationError) as raised:
+ await _attempt(attempt)
+ assert raised.value.type == "CompactionFailed"
+ assert raised.value.non_retryable is final
+
+
+async def test_a_rate_limit_waits_like_an_llm_step(summarizer):
+ summarizer.side_effect = LLMError("private body", "openai", category=LLMErrorCategory.RATE_LIMIT, retryable=True)
+ with pytest.raises(ApplicationError) as raised:
+ await _attempt(4)
+ failure = raised.value
+ assert failure.type == "LLMError.rate_limit" and not failure.non_retryable
+ assert failure.category == ApplicationErrorCategory.BENIGN
+ assert failure.next_retry_delay == timedelta(seconds=40) # the step's backoff at attempt 4
+ assert summarizer.await_args.kwargs["raise_llm_errors"] is True
+
+
+async def test_a_missing_service_stays_retryable(monkeypatch):
+ monkeypatch.setattr("services.compaction.get_compaction_service", lambda: None)
+ with pytest.raises(ApplicationError) as raised:
+ await _attempt(5)
+ assert raised.value.type == "CompactionFailed" and not raised.value.non_retryable