Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs: document delegation input contract + Output contract enforcement
trohitg0db7014aad1d
diff --git a/CLAUDE.md b/CLAUDE.mdindex cff4c761..d3ff4f2b 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -454,6 +454,24 @@ Reach for `NodeUserError` for any user-correctable failure(missing required field, unknown enum value, bad regex). Reserve`RuntimeError` / `Exception` for genuinely unexpected server bugs.+**Output contract enforcement** (`BaseNode._serialize_result` in+`services/plugin/base.py`): the declared `Output` Pydantic model is+enforced at the serialization boundary, FastAPI-`response_model` style.+Dict results validate via `Output.model_validate(...).model_dump(mode="json",+exclude_unset=True)`; `BaseModel` results dump `mode="json"`; violations+produce an `error_type="OutputValidationError"` envelope at the producer.+Rules: prefer returning the `Output` instance; never put raw third-party+objects (SDK results, dataclasses) or pre-stringified JSON into result+dicts — return plain lists/dicts; Params fields that may receive+LLM-stringified JSON args coerce with `field_validator(mode="before")`+(canonical: `AndroidServiceParams._coerce_parameters`,+`WriteTodosParams._coerce_todos`). Below the plugin layer, the SQLAlchemy+engine sets `json_serializer` backed by `pydantic_core.to_jsonable_python`+(`core/database.py`) so every JSON column tolerates dataclasses /+datetimes / enums / sets. Full spec:+[docs-internal/plugin_system.md → Output contract enforcement](./docs-internal/plugin_system.md);+locked by `server/tests/test_output_contract.py`.+### 6. Cleanup & Lifecycle- **Use existing teardown methods** - e.g., `_teardown_all_cron_triggers()` for cron cleanup- **Cleanup in finally blocks** - Ensure resources are released even on error@@ -1632,7 +1650,7 @@ server/nodes/document/Workflows execute via Temporal for durability and horizontal scaling. Three dispatch paths (legacy `execute_node_activity` / per-type `node.{type}.v{version}` (F4.A) / Agent-as-child-workflow (F4.B)) gated by `TEMPORAL_PER_TYPE_DISPATCH` and `TEMPORAL_AGENT_WORKFLOW_ENABLED` settings flags.-Full architecture, dispatch matrix, per-node lifecycle, heartbeat semantics, the 14 migrating agent types + 2 bypass agents (rlm_agent / claude_code_agent), the 6 F4.B agent activities (`prepare_payload.v1` / `execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1` / `store_output.v1` / `broadcast_progress.v1`), the **`delegate_to_<x>` Temporal-native dispatch** (child agent type in `AGENT_WORKFLOW_TYPES` → `workflow.execute_child_workflow` with `parent_node_id` propagation + parent-mirror in `_emit_phase` so the parent's canvas badge advances live; falls through to `execute_activity` otherwise), and the future `TemporalWorkerPool` per-queue routing live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md). Tool-call dispatch under F4.A documented at [docs-internal/tool_building_pipeline.md §9](./docs-internal/tool_building_pipeline.md).+Full architecture, dispatch matrix, per-node lifecycle, heartbeat semantics, the 14 migrating agent types + 2 bypass agents (rlm_agent / claude_code_agent), the 6 F4.B agent activities (`prepare_payload.v1` / `execute_llm_step.v1` / `persist_turn.v1` / `compact_memory.v1` / `store_output.v1` / `broadcast_progress.v1`), the **`delegate_to_<x>` Temporal-native dispatch** (child agent type in `AGENT_WORKFLOW_TYPES` → `workflow.execute_child_workflow` with `parent_node_id` propagation + parent-mirror in `_emit_phase` so the parent's canvas badge advances live; falls through to `execute_activity` otherwise), and the future `TemporalWorkerPool` per-queue routing live in [docs-internal/TEMPORAL_ARCHITECTURE.md](./docs-internal/TEMPORAL_ARCHITECTURE.md). Tool-call dispatch under F4.A documented at [docs-internal/tool_building_pipeline.md §9](./docs-internal/tool_building_pipeline.md). **Delegation input contract**: the LLM's `{task, context}` args travel as the child workflow input's `invocation` field (per-invocation input vs node config); `prepare_agent_payload` applies it AFTER config resolution so stored node parameters never override the delegated task — empty-task calls are rejected at the parent's call boundary, and `execute_llm_step` raises `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)` on system-only message lists (providers require ≥1 non-system message).**Configuration** (`.env.template` — canonical defaults; `.env` layers user overrides):```env@@ -2501,7 +2519,7 @@ AI Agents can delegate tasks to other agents connected to their `input-tools` ha**Two execution paths** depending on whether the parent runs under Temporal F4.B:-- **F4.B (default)**: parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge (parent badge advances live alongside child iteration counter instead of freezing). Deterministic child workflow id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. Parent waits for child result through the child-workflow handle — synchronous within the parent's loop.+- **F4.B (default)**: parent `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow` when the child type is in `AGENT_WORKFLOW_TYPES`. Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent's canvas badge (parent badge advances live alongside child iteration counter instead of freezing). The LLM's `{task, context}` args travel as `child_context["invocation"]` — per-invocation input applied AFTER config resolution in `prepare_agent_payload` (`task` → system_message, `context`-or-`task` → prompt, same semantics as the legacy path), so stored node parameters never override the delegated task; empty-task calls are rejected at the call boundary without spawning. Deterministic child workflow id `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"`. Parent waits for child result through the child-workflow handle — synchronous within the parent's loop.- **Legacy / F4.A-only**: the fire-and-forget pattern below. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with `TEMPORAL_AGENT_WORKFLOW_ENABLED=false`.**Architecture: Fire-and-Forget Pattern (legacy / F4.A)**diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdindex 2b3089fc..e95b78a7 100644--- a/docs-internal/TEMPORAL_ARCHITECTURE.md+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -298,12 +298,23 @@ AgentWorkflow.run(context):(messages_to_dict round-trip preserves provider-specificfields: Gemini thought_signature, Anthropic cache markers,OpenAI reasoning_content) — appended verbatim to messages.+ After filter_empty_messages, a list with no human/ai/tool+ message raises ApplicationError(type="EmptyAgentPrompt",+ non_retryable=True) — providers require >=1 non-system+ message (Gemini pulls SystemMessages into system_instruction+ and rejects empty contents), so the failure surfaces to the+ parent LLM on attempt 1 instead of consuming the retry budget.3. if kind == "tool_calls":for each call:emit_phase("executing_tool", tool_name=...)if delegate_to_* AND child type in AGENT_WORKFLOW_TYPES:+ # {task, context} are per-invocation INPUT, not node+ # config — both empty -> tool-error back to the LLM,+ # no child spawn.execute_child_workflow("AgentWorkflow", child_context)- child_context = {**tool_payload, "parent_node_id": <self>}+ child_context = {**tool_payload,+ "parent_node_id": <self>,+ "invocation": {"task": …, "context": …}}id = f"{parent_workflow_id}-delegate-{child_node_id}-{iter}"else:execute_activity(f"node.{tool_node_type}.v{version}")@@ -343,14 +354,16 @@ AgentWorkflow.run(context):Each LLM step is one activity, each tool call is one per-type activity (the same activities F4.A registered). **Delegation tool calls** (`delegate_to_<x>`) where the child type is in `AGENT_WORKFLOW_TYPES` are the exception: they spawn a child `AgentWorkflow` via `workflow.execute_child_workflow` instead of a per-type activity, with `parent_node_id` injected into `child_context` to enable the parent-mirror broadcast above. Deterministic child workflow id is `f"{parent_workflow_id}-delegate-{child_node_id}-{iteration}"` so Temporal retries don't spawn duplicates. Non-agent tools and excluded types (`rlm_agent`, `claude_code_agent`) still go through `execute_activity` as before. Failures of tool activities surface as error messages back to the LLM (matching the in-process agent loop behaviour); the agent loop continues.+**Delegation input contract (input-vs-config separation).** The LLM's `{task, context}` args are per-invocation *input*, not node configuration, and travel as the child workflow input's `invocation` field. `prepare_agent_payload` applies it AFTER its config resolution (`{**node_data, **db_params}` — DB wins for config liveness): `task` → system_message, `context`-or-`task` → prompt — the same semantics as the legacy `handlers.tools._execute_delegated_agent`. Stored node parameters (including the empty default `prompt` the frontend persists on drop) therefore never override the delegated task. A call with both fields empty is rejected at the parent's call boundary (tool-error message to the LLM, no child spawn). Bypass agents dispatched as plain activities (`rlm_agent` / `claude_code_agent`) instead receive the remap directly in `node_data` — their per-type activity consumes `node_data` verbatim with no DB re-merge.+**Canvas-aware tools** opt into receiving the parent workflow's `nodes`/`edges` by declaring `needs_canvas: ClassVar[bool] = True` on their `BaseNode` subclass. The F4.B tool dispatch reads this via `services.node_registry.get_node_class(node_type).needs_canvas` and forwards `context.get("nodes")` / `context.get("edges")` into `tool_payload`; default plugins keep the empty-canvas optimisation. Today only `agentBuilder` opts in (it walks edges to resolve its calling agent and mutates the canvas). Operations inside agentBuilder reload via `database.get_workflow(workflow_id)` so in-run duplicate detection sees mutations from earlier calls in the same workflow run — see [agent_builder section in CLAUDE.md](../CLAUDE.md).**Seven agent activities** are registered by `collect_agent_activities()` for `AgentWorkflow` to schedule by name:| Activity | Purpose ||---|---|-| `agent.prepare_payload.v1` | Resolves the DB-backed payload (provider / model / api_key / system_message / user_prompt / tools / memory_node_id / memory_content / memory_window_size / max_iterations / thinking_config / compaction_threshold / auto_rebind_tools). Reads `UserSettings.agent_recursion_limit` + `UserSettings.auto_rebind_tools_after_canvas_change`. |-| `agent.execute_llm_step.v1` | One LLM turn — rebuilds StructuredTools from the workflow's current `tools` list and returns the assistant message + tool_calls. |+| `agent.prepare_payload.v1` | Resolves the DB-backed payload (provider / model / api_key / system_message / user_prompt / tools / memory_node_id / memory_content / memory_window_size / max_iterations / thinking_config / compaction_threshold / auto_rebind_tools). Reads `UserSettings.agent_recursion_limit` + `UserSettings.auto_rebind_tools_after_canvas_change`. Applies the optional `invocation` field (delegation children) after config resolution — per-invocation input always beats stored parameters. |+| `agent.execute_llm_step.v1` | One LLM turn — rebuilds StructuredTools from the workflow's current `tools` list and returns the assistant message + tool_calls. Guards against un-invokable payloads: post-filter system-only message lists raise `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)`. || `agent.refresh_tools.v1` | Translates `workflow_ops` add_node ops (component_kind="tool" OR usable_as_tool=True) into fresh `tool_payload` entries via `_build_tool_from_node`. Workflow extends `tools` + `tool_index` from the result. || `agent.persist_turn.v1` | Appends the latest human/assistant exchange to memory markdown, trims the window, broadcasts `node.parameters.updated`. || `agent.compact_memory.v1` | Token-budget compaction when cumulative tokens hit the threshold. Best-effort: continues with un-compacted history on failure. |diff --git a/docs-internal/agent_delegation.md b/docs-internal/agent_delegation.mdindex 43c086c8..d59fd142 100644--- a/docs-internal/agent_delegation.md+++ b/docs-internal/agent_delegation.md@@ -10,10 +10,10 @@ AI Agents can delegate tasks to other agents connected to their `input-tools` ha**Two execution paths** depending on whether the parent is running under Temporal F4.B:-- **F4.B (default, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`):** the delegation tool dispatch inside `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow`. Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent canvas badge. Parent awaits the child's result synchronously through the Temporal child-workflow handle — see [TEMPORAL_ARCHITECTURE.md §F4.B](./TEMPORAL_ARCHITECTURE.md#agent-as-child-workflow-f4b).-- **Legacy / F4.A-only:** the rest of this doc. `_execute_delegated_agent` spawns the child as an independent `asyncio.Task` and the parent continues working immediately — the **fire-and-forget** pattern. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with F4.B disabled.+- **F4.B (default, `TEMPORAL_AGENT_WORKFLOW_ENABLED=true`):** the delegation tool dispatch inside `AgentWorkflow` spawns the child as a **child `AgentWorkflow`** via `workflow.execute_child_workflow`. The LLM's `{task, context}` args travel as `child_context["invocation"]` — the per-invocation input contract: `prepare_agent_payload` applies it AFTER its config resolution (`task` → system_message, `context`-or-`task` → prompt), so the child's stored parameters (including the empty default `prompt` saved on drop) never override the delegated task. Calls with both fields empty are rejected at the parent's call boundary (tool-error to the LLM, no spawn). Parent's `node_id` is passed in `child_context["parent_node_id"]` so the child's `_emit_phase` mirrors progress onto the parent canvas badge. Parent awaits the child's result synchronously through the Temporal child-workflow handle — see [TEMPORAL_ARCHITECTURE.md §F4.B](./TEMPORAL_ARCHITECTURE.md#agent-as-child-workflow-f4b).+- **Legacy / F4.A-only:** the rest of this doc. `_execute_delegated_agent` spawns the child as an independent `asyncio.Task` and the parent continues working immediately — the **fire-and-forget** pattern. Applies the same task/context semantics by overwriting `child_params` AFTER loading them from the DB. Still the path for `rlm_agent` / `claude_code_agent` (excluded from `AGENT_WORKFLOW_TYPES`) and any deployment with F4.B disabled.-In both paths the parent passes only a `task` string and optional `context` string. Everything else -- the child's provider, model, system message, memory, skills, and tools -- comes from the child's own configuration and workflow connections.+In both paths the parent passes only a `task` string and optional `context` string — per-invocation input that always wins over stored configuration. Everything else -- the child's provider, model, memory, skills, and tools -- comes from the child's own configuration and workflow connections.## Architecturediff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.mdindex 37aeec21..8e294755 100644--- a/docs-internal/plugin_system.md+++ b/docs-internal/plugin_system.md@@ -145,6 +145,25 @@ class SpecializedAgentBase(ActionNode, abstract=True):This is the contract that makes `writeTodos` / `calculatorTool` etc. broadcast successfully through F4.A — without the polymorphic override, the wrapper's `if result.get("success")` check would treat every ToolNode result as failure and never call `update_node_output`.+### Output contract enforcement (`_serialize_result`)++The declared `Output` model is enforced at the serialization boundary — the same semantics FastAPI applies to `response_model` (validate → coerce → serialize). `BaseNode._serialize_result` (called by both `_wrap_success` implementations):++- **`BaseModel` return** → `model_dump(mode="json")` (datetimes → ISO strings, enums → values).+- **dict return + declared `Output`** → `Output.model_validate(result).model_dump(mode="json", exclude_unset=True)`. `exclude_unset` preserves the producer's exact key set — declared-but-absent `Optional` fields never materialise as `None` keys.+- **dict return, no declared `Output`** (`_EmptyOutput` default) → pass-through.+- **Violation** (wrong type in a declared field, or a non-serializable object anywhere) → standard error envelope with `error_type="OutputValidationError"` + full traceback in the operator log. A contract violation is a plugin bug that fails loudly at the producer instead of corrupting `node_outputs` persistence or the WS broadcast downstream.++Practical rules for plugin authors:++1. Prefer returning the `Output` model instance (`return ExampleOutput(...)`). Dicts work but get validated.+2. `Output` models declare `extra="allow"` + all-`Optional` fields by convention — extra context keys pass through; only type mismatches fail.+3. Never put raw third-party objects (dataclasses like deepagents' `ReadResult`, SDK response objects) into the result — unwrap to plain fields. Everything downstream (JSON column, orjson WS broadcast, `_serialise_tool_result`) expects JSON-compatible data.+4. LLM-facing string formatting is the dispatcher's job (`_serialise_tool_result` JSON-dumps the whole payload) — return real lists/dicts, not pre-stringified JSON.+5. Params fields that may receive LLM-stringified JSON (Gemini sometimes stringifies array/object args in tool calls) coerce at the boundary with `field_validator(..., mode="before")` — see `AndroidServiceParams._coerce_parameters` and `WriteTodosParams._coerce_todos` for the canonical shape.++Defense in depth below the plugin layer: the SQLAlchemy engine in `core/database.py` sets `json_serializer=lambda obj: json.dumps(to_jsonable_python(obj, fallback=str))` (`pydantic_core.to_jsonable_python` — official arbitrary-object → JSON coercion), so every JSON column on the engine tolerates dataclasses / datetimes / enums / sets even if a payload bypasses Output validation. Locked by `tests/test_output_contract.py`.+### Auto-derived uiHints`BaseNode._metadata_dict` (`server/services/plugin/base.py`) callsdiff --git a/server/nodes/README.md b/server/nodes/README.mdindex b1337671..784bb78e 100644--- a/server/nodes/README.md+++ b/server/nodes/README.md@@ -211,6 +211,13 @@ every plugin. Common ones you'll trip:`messaging` / `machina-default`).- Tool schemas (`usable_as_tool=True` or `ToolNode`) — no `$defs`,no `$ref` (LLM-compat).+- **`Output` is enforced at runtime.** Dict results from operations are+ validated against the declared `Output` model and dumped with+ `model_dump(mode="json", exclude_unset=True)` (FastAPI+ `response_model` semantics — see `BaseNode._serialize_result`). A+ type mismatch in a declared field produces an `OutputValidationError`+ envelope. Prefer returning the `Output` instance directly; keep+ `extra="allow"` + `Optional` fields so context keys pass through.Run: `pytest server/tests/test_plugin_contract.py -q`.@@ -323,6 +330,22 @@ Full reference: [docs-internal/plugin_system.md → "Self-contained plugin foldePydantic models or `Union`, the LLM-schema emission will add `$defs`and fail the invariant. Keep tool-facing Params flat; move nestedtypes to `Output` instead.+- **Return plain data, never raw third-party objects.** Backend / SDK+ result objects (deepagents `ReadResult` / `WriteResult`, httpx+ responses, …) must be unwrapped to plain fields before returning —+ the Output validation rejects them, and everything downstream+ (`node_outputs` JSON column, orjson WS broadcast,+ `_serialise_tool_result`) expects JSON-compatible data. Likewise+ return real lists/dicts, not pre-stringified JSON — LLM-facing+ serialization is the dispatcher's job.+- **Coerce LLM-stringified JSON args at the Params boundary.** Gemini+ (and others) sometimes pass array/object tool args as JSON-encoded+ strings. Fields that can receive them declare a+ `field_validator(..., mode="before")` that `json.loads`es string+ input — canonical examples: `AndroidServiceParams._coerce_parameters`,+ `WriteTodosParams._coerce_todos`. Malformed JSON should pass through+ unchanged so Pydantic raises a proper `ValidationError` the LLM can+ correct.- **Collapse advanced options with `group="..."`.** Tag rarely-tunedfields with `json_schema_extra={"group": "options"}` and they getlifted into a collapsible "Options" collection in the parameter