Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: sync TEMPORAL_ARCHITECTURE / status_broadcaster / memory_lifecycle / CLAUDE.md with F4.B runtime fixes
trohitg19eeb4071e9e
diff --git a/CLAUDE.md b/CLAUDE.mdindex 1d3e81e0..71b56d52 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -1641,7 +1641,7 @@ python -m services.temporal.worker- `services/temporal/activities.py` - Legacy `execute_node_activity` (WebSocket round-trip path)- `services/temporal/plugin_activities.py` - `collect_plugin_activities` → per-type activities for F4.A- `services/temporal/agent_workflow.py` - `AgentWorkflow` child workflow for F4.B agent loops; its first step calls `agent.prepare_payload.v1` to resolve the DB-backed payload, then loops LLM steps + tool dispatch + per-turn persist + compaction-- `services/temporal/agent_activities.py` - F4.B activities (5 total): `agent.execute_llm_step.v1`, `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.prepare_payload.v1`, `agent.broadcast_progress.v1` (the last emits CloudEvents-shaped `agent_progress` per LLM turn)+- `services/temporal/agent_activities.py` - F4.B activities (6 total): `agent.prepare_payload.v1` (resolves DB params + runs `_param_resolver.resolve` so `{{templates}}` work inside the agent's own prompt), `agent.execute_llm_step.v1` (rebuilds tools via `ai_service._build_tool_from_node`; uses LangChain `messages_to_dict`/`messages_from_dict` so Gemini `thought_signature` survives), `agent.persist_turn.v1`, `agent.compact_memory.v1`, `agent.store_output.v1` (wraps `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`), `agent.broadcast_progress.v1` (emits CloudEvents-shaped `agent_progress` for every phase: `starting` / `llm_step` / `executing_tool` / `tool_completed` / `completed`, optionally driving raw-dict `update_node_status` for canvas-glow color)- `services/temporal/worker.py` - TemporalWorkerManager (registers MachinaWorkflow + AgentWorkflow + activities) + `run_standalone_worker()` + `TemporalWorkerPool` (multi-queue, future enhancement)- `services/temporal/executor.py` - TemporalExecutor interface matching WorkflowExecutor- `services/temporal/client.py` - Client wrapper with runtime heartbeat disableddiff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdindex 7a93b512..54ceb5a8 100644--- a/docs-internal/TEMPORAL_ARCHITECTURE.md+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -280,28 +280,49 @@ When `TEMPORAL_AGENT_WORKFLOW_ENABLED=true` and the node type is in the migratin```AgentWorkflow.run(context):0. execute_activity("agent.prepare_payload.v1")- resolves the DB-backed payload (provider / model / api_key /- tool schemas / memory / system_message / compaction threshold)- from the canvas context. Temporal workflows must be deterministic,- so DB lookups + edge walking + tool schema build live in this- activity rather than the workflow body.+ resolves the DB-backed payload from the canvas context — runs+ workflow_service._param_resolver.resolve so {{node.field}}+ templates in prompt / system_message become real values BEFORE+ the LLM sees them. Temporal workflows must be deterministic,+ so DB lookups + edge walking + tool schema build live here.+ Tool entries carry the raw tool_info dict — execute_llm_step+ rebuilds the StructuredTool via ai_service._build_tool_from_node+ inside the activity (no JSON-schema round-trip).+ emit_phase("starting", status="executing")loop until "final" or max_iterations:- 1. execute_activity("agent.broadcast_progress.v1")- emits CloudEvents-shaped `agent_progress` for the canvas badge- (one per turn, `type="com.machinaos.agent.progress"`).- 2. execute_activity("agent.execute_llm_step.v1") -> {kind, content|calls}+ 1. emit_phase("llm_step", iteration=N)+ 2. execute_activity("agent.execute_llm_step.v1")+ returns {kind, assistant_message, calls?, content?, usage}.+ assistant_message is the full LangChain-serialized AIMessage+ (messages_to_dict round-trip preserves provider-specific+ fields: Gemini thought_signature, Anthropic cache markers,+ OpenAI reasoning_content) — appended verbatim to messages.3. if kind == "tool_calls":for each call:+ emit_phase("executing_tool", tool_name=...)execute_activity(f"node.{tool_node_type}.v{version}")- append tool_results to messages- 4. execute_activity("agent.persist_turn.v1") # append memory per turn- emits CloudEvents `node_parameters_updated` via the broadcaster- wrapper (`source_hint="agent"`).+ emit_phase("tool_completed", tool_name=...)+ _serialise_tool_result unwraps F4.A's {success, result, ...}+ envelope so the LLM sees only the handler's return value+ (matches services/ai.py:create_tool_node behavior).+ 4. execute_activity("agent.persist_turn.v1")+ append_to_memory_markdown(content, "human", prompt) ++ (content, "ai", response); trim window; broadcast+ node_parameters_updated CloudEvents (source_hint="agent").5. if token_total >= compaction_threshold:execute_activity("agent.compact_memory.v1")- replace messages with summary+ null-guarded against worker-bootstrap race; replaces messages+ with summary only when result.success is True.+ execute_activity("agent.store_output.v1")+ wraps workflow_service.store_node_output for output_main /+ output_top / output_0 — same writes NodeExecutor.execute does+ at services/node_executor.py:197-199, so downstream nodes+ can resolve {{aiAgent.response}} via ParameterResolver.+ emit_phase("completed", status="success")```+`emit_phase(phase, status?)` is a thin helper that schedules `agent.broadcast_progress.v1`. The activity emits `WorkflowEvent.agent_progress` (CloudEvents v1.0, `type="com.machinaos.agent.progress"`) for FE consumers; when `status` is supplied it also drives a raw-dict `update_node_status` for the canvas-glow color (executing / success / error). Same dual-channel pattern F4.A's `_node_activity` uses.+Each LLM step is one activity, each tool call is one per-type activity (the same activities F4.A registered). Failures of tool activities surface as error messages back to the LLM (matching today's LangGraph behaviour); the agent loop continues.**Broadcasts inside the loop** wrap `WorkflowEvent` (CloudEvents v1.0) per RFC §6.4: `agent_progress` events (`com.machinaos.agent.progress`) and `node_parameters_updated` events (`com.machinaos.node.parameters.updated`) flow through the `StatusBroadcaster.broadcast_agent_progress` and `StatusBroadcaster.broadcast_node_parameters_updated` wrappers respectively. The latter is reused by the legacy `routers/websocket.py:handle_save_node_parameters` (user-source) and `services/cli_agent/service.py:_persist_memory` (cli-source) — all three emission sites share the same envelope, distinguished by `source_hint` (`"user"` / `"cli"` / `"agent"`).diff --git a/docs-internal/memory_lifecycle.md b/docs-internal/memory_lifecycle.mdindex b3640301..ba3f0813 100644--- a/docs-internal/memory_lifecycle.md+++ b/docs-internal/memory_lifecycle.md@@ -147,7 +147,7 @@ Turn N startsbroadcast_node_parameters_updated(...) # CloudEvents source_hint="agent"```-F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities (`execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1`); per-turn persistence means a mid-loop failure doesn't lose progress. F4.A and legacy paths run steps 4-9 inline inside one activity.+F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities (`prepare_payload.v1` resolves DB params + `{{templates}}` via `workflow_service._param_resolver.resolve`; `execute_llm_step.v1` invokes the LLM with LangChain `messages_to_dict` / `messages_from_dict` round-trip preserving Gemini `thought_signature` + Anthropic cache markers + OpenAI reasoning content; `persist_turn.v1` appends per turn; `compact_memory.v1` summarises when threshold trips; `store_output.v1` writes to the OutputStore via `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`; `broadcast_progress.v1` emits CloudEvents `agent_progress` per phase). Per-turn persistence means a mid-loop failure doesn't lose progress. F4.A and legacy paths run steps 4-9 inline inside one activity.## 8. Engine-specific adaptersdiff --git a/docs-internal/status_broadcaster.md b/docs-internal/status_broadcaster.mdindex 02bab37f..71aad469 100644--- a/docs-internal/status_broadcaster.md+++ b/docs-internal/status_broadcaster.md@@ -139,7 +139,7 @@ Broadcasts are sent to all connected clients without a request-response correlat| `api_key_status` | Credential validation result; also fired by `delete_api_key` to clear `apiKeyStatuses[provider]` | `{provider, data: {valid, hasKey, models, message, timestamp}}` || `credential_catalogue_updated` | Credential mutation (save / delete / oauth-disconnect) — wraps `WorkflowEvent` (CloudEvents v1.0) from `services/events/envelope.py`. Outer `type` stays this string for FE back-compat; body's `type` follows `credential.<area>.<action>` (e.g. `credential.api_key.saved`). Frontend `WebSocketContext` invalidates `useCatalogueQuery` via the 300 ms `invalidateCatalogue` debounce. | `{specversion: "1.0", id, source: "machinaos://services/credentials", type, subject: provider, time, data: {provider, customer_id?}}` || `node_parameters_updated` | Parameters saved by parameter-panel user, Claude CLI memory bridge, or AgentWorkflow per-turn memory append — wraps `WorkflowEvent` (CloudEvents v1.0) from `services/events/envelope.py`. Body's `type` is `com.machinaos.node.parameters.updated`. `data.source` (`"user"` / `"cli"` / `"agent"`) distinguishes the three emission sites. Replaces the legacy raw-dict broadcast at commit `7c9e873`. Locked by `tests/test_cloudevents_node_parameters.py`. | `{specversion: "1.0", id, source: "machinaos://services/parameters", type: "com.machinaos.node.parameters.updated", subject: node_id, time, data: {node_id, parameters, version, source}}` |-| `agent_progress` | Per-turn LangGraph / AgentWorkflow progress (canvas badge "N / max") — wraps `WorkflowEvent.agent_progress` (CloudEvents v1.0). Emitted from inside `services/ai.py:execute_agent` (legacy path) and via the `agent.broadcast_progress.v1` Temporal activity (F4.B AgentWorkflow path). | `{specversion: "1.0", id, source: "machinaos://services/agent", type: "com.machinaos.agent.progress", subject: node_id, time, data: {node_id, iteration, max_iterations, phase?}}` |+| `agent_progress` | LangGraph / AgentWorkflow lifecycle phase (canvas badge "N / max" + phase indicator) — wraps `WorkflowEvent.agent_progress` (CloudEvents v1.0). Emitted from `services/ai.py:execute_agent` (legacy) and via the `agent.broadcast_progress.v1` Temporal activity (F4.B). F4.B fires the full phase set: `starting` (entry, drives `node_status="executing"`), `llm_step` (per iteration), `executing_tool` + `tool_completed` (around each per-type tool activity), `completed` (final, drives `node_status="success"`). The activity optionally drives a raw-dict `update_node_status` alongside the CloudEvents envelope so the canvas-glow color follows the lifecycle without a separate handler — same dual-channel pattern F4.A's `_node_activity` uses. | `{specversion: "1.0", id, source: "machinaos://services/agent", type: "com.machinaos.agent.progress", subject: node_id, time, data: {node_id, iteration, max_iterations, phase?, tool_name?}}` || `token_usage_update` | AI execution updates token counters | `{session_id, data: {total, threshold, needs_compaction}}` || `compaction_starting` | Memory compaction begins | `{session_id, node_id}` || `compaction_completed` | Memory compaction ends | `{session_id, success, tokens_before, tokens_after}` |