Skip to content

Self-improving AI Employees for everyone

docs: sync TEMPORAL_ARCHITECTURE / status_broadcaster / memory_lifecycle / CLAUDE.md with F4.B runtime fixes

trohitg19eeb4071e9e
diff --git a/CLAUDE.md b/CLAUDE.md
index 1d3e81e0..71b56d52 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1641,7 +1641,7 @@ python -m services.temporal.worker
- `services/temporal/activities.py` - Legacy `execute_node_activity` (WebSocket round-trip path)
- `services/temporal/plugin_activities.py` - `collect_plugin_activities` → per-type activities for F4.A
- `services/temporal/agent_workflow.py` - `AgentWorkflow` child workflow for F4.B agent loops; its first step calls `agent.prepare_payload.v1` to resolve the DB-backed payload, then loops LLM steps + tool dispatch + per-turn persist + compaction
-- `services/temporal/agent_activities.py` - F4.B activities (5 total): `agent.execute_llm_step.v1`, `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.prepare_payload.v1`, `agent.broadcast_progress.v1` (the last emits CloudEvents-shaped `agent_progress` per LLM turn)
+- `services/temporal/agent_activities.py` - F4.B activities (6 total): `agent.prepare_payload.v1` (resolves DB params + runs `_param_resolver.resolve` so `{{templates}}` work inside the agent's own prompt), `agent.execute_llm_step.v1` (rebuilds tools via `ai_service._build_tool_from_node`; uses LangChain `messages_to_dict`/`messages_from_dict` so Gemini `thought_signature` survives), `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.store_output.v1` (wraps `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`), `agent.broadcast_progress.v1` (emits CloudEvents-shaped `agent_progress` for every phase: `starting` / `llm_step` / `executing_tool` / `tool_completed` / `completed`, optionally driving raw-dict `update_node_status` for canvas-glow color)
- `services/temporal/worker.py` - TemporalWorkerManager (registers MachinaWorkflow + AgentWorkflow + activities) + `run_standalone_worker()` + `TemporalWorkerPool` (multi-queue, future enhancement)
- `services/temporal/executor.py` - TemporalExecutor interface matching WorkflowExecutor
- `services/temporal/client.py` - Client wrapper with runtime heartbeat disabled
diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.md
index 7a93b512..54ceb5a8 100644
--- a/docs-internal/TEMPORAL_ARCHITECTURE.md
+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md
@@ -280,28 +280,49 @@ When `TEMPORAL_AGENT_WORKFLOW_ENABLED=true` and the node type is in the migratin
```
AgentWorkflow.run(context):
0. execute_activity("agent.prepare_payload.v1")
- resolves the DB-backed payload (provider / model / api_key /
- tool schemas / memory / system_message / compaction threshold)
- from the canvas context. Temporal workflows must be deterministic,
- so DB lookups + edge walking + tool schema build live in this
- activity rather than the workflow body.
+ resolves the DB-backed payload from the canvas context — runs
+ workflow_service._param_resolver.resolve so {{node.field}}
+ templates in prompt / system_message become real values BEFORE
+ the LLM sees them. Temporal workflows must be deterministic,
+ so DB lookups + edge walking + tool schema build live here.
+ Tool entries carry the raw tool_info dict — execute_llm_step
+ rebuilds the StructuredTool via ai_service._build_tool_from_node
+ inside the activity (no JSON-schema round-trip).
+ emit_phase("starting", status="executing")
loop until "final" or max_iterations:
- 1. execute_activity("agent.broadcast_progress.v1")
- emits CloudEvents-shaped `agent_progress` for the canvas badge
- (one per turn, `type="com.machinaos.agent.progress"`).
- 2. execute_activity("agent.execute_llm_step.v1") -> {kind, content|calls}
+ 1. emit_phase("llm_step", iteration=N)
+ 2. execute_activity("agent.execute_llm_step.v1")
+ returns {kind, assistant_message, calls?, content?, usage}.
+ assistant_message is the full LangChain-serialized AIMessage
+ (messages_to_dict round-trip preserves provider-specific
+ fields: Gemini thought_signature, Anthropic cache markers,
+ OpenAI reasoning_content) — appended verbatim to messages.
3. if kind == "tool_calls":
for each call:
+ emit_phase("executing_tool", tool_name=...)
execute_activity(f"node.{tool_node_type}.v{version}")
- append tool_results to messages
- 4. execute_activity("agent.persist_turn.v1") # append memory per turn
- emits CloudEvents `node_parameters_updated` via the broadcaster
- wrapper (`source_hint="agent"`).
+ emit_phase("tool_completed", tool_name=...)
+ _serialise_tool_result unwraps F4.A's {success, result, ...}
+ envelope so the LLM sees only the handler's return value
+ (matches services/ai.py:create_tool_node behavior).
+ 4. execute_activity("agent.persist_turn.v1")
+ append_to_memory_markdown(content, "human", prompt) +
+ (content, "ai", response); trim window; broadcast
+ node_parameters_updated CloudEvents (source_hint="agent").
5. if token_total >= compaction_threshold:
execute_activity("agent.compact_memory.v1")
- replace messages with summary
+ null-guarded against worker-bootstrap race; replaces messages
+ with summary only when result.success is True.
+ execute_activity("agent.store_output.v1")
+ wraps workflow_service.store_node_output for output_main /
+ output_top / output_0 — same writes NodeExecutor.execute does
+ at services/node_executor.py:197-199, so downstream nodes
+ can resolve {{aiAgent.response}} via ParameterResolver.
+ emit_phase("completed", status="success")
```
+`emit_phase(phase, status?)` is a thin helper that schedules `agent.broadcast_progress.v1`. The activity emits `WorkflowEvent.agent_progress` (CloudEvents v1.0, `type="com.machinaos.agent.progress"`) for FE consumers; when `status` is supplied it also drives a raw-dict `update_node_status` for the canvas-glow color (executing / success / error). Same dual-channel pattern F4.A's `_node_activity` uses.
+
Each LLM step is one activity, each tool call is one per-type activity (the same activities F4.A registered). Failures of tool activities surface as error messages back to the LLM (matching today's LangGraph behaviour); the agent loop continues.
**Broadcasts inside the loop** wrap `WorkflowEvent` (CloudEvents v1.0) per RFC §6.4: `agent_progress` events (`com.machinaos.agent.progress`) and `node_parameters_updated` events (`com.machinaos.node.parameters.updated`) flow through the `StatusBroadcaster.broadcast_agent_progress` and `StatusBroadcaster.broadcast_node_parameters_updated` wrappers respectively. The latter is reused by the legacy `routers/websocket.py:handle_save_node_parameters` (user-source) and `services/cli_agent/service.py:_persist_memory` (cli-source) — all three emission sites share the same envelope, distinguished by `source_hint` (`"user"` / `"cli"` / `"agent"`).
diff --git a/docs-internal/memory_lifecycle.md b/docs-internal/memory_lifecycle.md
index b3640301..ba3f0813 100644
--- a/docs-internal/memory_lifecycle.md
+++ b/docs-internal/memory_lifecycle.md
@@ -147,7 +147,7 @@ Turn N starts
broadcast_node_parameters_updated(...) # CloudEvents source_hint="agent"
```
-F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities (`execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1`); per-turn persistence means a mid-loop failure doesn't lose progress. F4.A and legacy paths run steps 4-9 inline inside one activity.
+F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities (`prepare_payload.v1` resolves DB params + `{{templates}}` via `workflow_service._param_resolver.resolve`; `execute_llm_step.v1` invokes the LLM with LangChain `messages_to_dict` / `messages_from_dict` round-trip preserving Gemini `thought_signature` + Anthropic cache markers + OpenAI reasoning content; `persist_turn.v1` appends per turn; `compact_memory.v1` summarises when threshold trips; `store_output.v1` writes to the OutputStore via `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`; `broadcast_progress.v1` emits CloudEvents `agent_progress` per phase). Per-turn persistence means a mid-loop failure doesn't lose progress. F4.A and legacy paths run steps 4-9 inline inside one activity.
## 8. Engine-specific adapters
diff --git a/docs-internal/status_broadcaster.md b/docs-internal/status_broadcaster.md
index 02bab37f..71aad469 100644
--- a/docs-internal/status_broadcaster.md
+++ b/docs-internal/status_broadcaster.md
@@ -139,7 +139,7 @@ Broadcasts are sent to all connected clients without a request-response correlat
| `api_key_status` | Credential validation result; also fired by `delete_api_key` to clear `apiKeyStatuses[provider]` | `{provider, data: {valid, hasKey, models, message, timestamp}}` |
| `credential_catalogue_updated` | Credential mutation (save / delete / oauth-disconnect) — wraps `WorkflowEvent` (CloudEvents v1.0) from `services/events/envelope.py`. Outer `type` stays this string for FE back-compat; body's `type` follows `credential.<area>.<action>` (e.g. `credential.api_key.saved`). Frontend `WebSocketContext` invalidates `useCatalogueQuery` via the 300 ms `invalidateCatalogue` debounce. | `{specversion: "1.0", id, source: "machinaos://services/credentials", type, subject: provider, time, data: {provider, customer_id?}}` |
| `node_parameters_updated` | Parameters saved by parameter-panel user, Claude CLI memory bridge, or AgentWorkflow per-turn memory append — wraps `WorkflowEvent` (CloudEvents v1.0) from `services/events/envelope.py`. Body's `type` is `com.machinaos.node.parameters.updated`. `data.source` (`"user"` / `"cli"` / `"agent"`) distinguishes the three emission sites. Replaces the legacy raw-dict broadcast at commit `7c9e873`. Locked by `tests/test_cloudevents_node_parameters.py`. | `{specversion: "1.0", id, source: "machinaos://services/parameters", type: "com.machinaos.node.parameters.updated", subject: node_id, time, data: {node_id, parameters, version, source}}` |
-| `agent_progress` | Per-turn LangGraph / AgentWorkflow progress (canvas badge "N / max") — wraps `WorkflowEvent.agent_progress` (CloudEvents v1.0). Emitted from inside `services/ai.py:execute_agent` (legacy path) and via the `agent.broadcast_progress.v1` Temporal activity (F4.B AgentWorkflow path). | `{specversion: "1.0", id, source: "machinaos://services/agent", type: "com.machinaos.agent.progress", subject: node_id, time, data: {node_id, iteration, max_iterations, phase?}}` |
+| `agent_progress` | LangGraph / AgentWorkflow lifecycle phase (canvas badge "N / max" + phase indicator) — wraps `WorkflowEvent.agent_progress` (CloudEvents v1.0). Emitted from `services/ai.py:execute_agent` (legacy) and via the `agent.broadcast_progress.v1` Temporal activity (F4.B). F4.B fires the full phase set: `starting` (entry, drives `node_status="executing"`), `llm_step` (per iteration), `executing_tool` + `tool_completed` (around each per-type tool activity), `completed` (final, drives `node_status="success"`). The activity optionally drives a raw-dict `update_node_status` alongside the CloudEvents envelope so the canvas-glow color follows the lifecycle without a separate handler — same dual-channel pattern F4.A's `_node_activity` uses. | `{specversion: "1.0", id, source: "machinaos://services/agent", type: "com.machinaos.agent.progress", subject: node_id, time, data: {node_id, iteration, max_iterations, phase?, tool_name?}}` |
| `token_usage_update` | AI execution updates token counters | `{session_id, data: {total, threshold, needs_compaction}}` |
| `compaction_starting` | Memory compaction begins | `{session_id, node_id}` |
| `compaction_completed` | Memory compaction ends | `{session_id, success, tokens_before, tokens_after}` |