Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs(agents): correct skill, compaction, context and team-task docs
trohitgadbfcf14176a
diff --git a/docs-internal/agent_context_flow.md b/docs-internal/agent_context_flow.mdindex 7fc02348..3bf3946b 100644--- a/docs-internal/agent_context_flow.md+++ b/docs-internal/agent_context_flow.md@@ -102,8 +102,17 @@ build** and is intentional:The carried transcript caps at ~1 MB serialized(`_CAN_TRANSCRIPT_MAX_BYTES`) because Temporal's payload error limit is-2 MiB for the whole continue-as-new argument; over-cap degrades to the-opening prompt with a warning — never a failure.+2 MiB for the whole continue-as-new argument. Over the cap, the transcript is+dropped with a warning and the resumed run seeds like a fresh firing: from+the stored conversation when a Context node is connected (with the firing's+prompt appended again), else from the opening prompt.++A continue-as-new starts `run()` from the top, so `agent.prepare_payload`+runs again. The resumed run picks up the node's current configuration (tools,+system message, model) and reloads the stored conversation with the load+checks below, even when the carried transcript is what seeds it. If that+stored row is over the 1 MB seed cap, the resumed run fails with+`ConversationTooLarge`.## Transcript size: capped results, cleared old results@@ -184,7 +193,10 @@ only grow until the provider rejected it.Running on silently instead would burn tokens on an amnesiac prompt — theexact failure mode this design replaced. With the cap and the byte budgetabove, new transcripts stay far below the limit; a row saved before them- needs one clear.+ needs one clear. Both error types and the 1 MB cap exist only on the+ Temporal path. The in-process `_prepare_context` (`services/ai.py`) and+ `SpecializedAgentContextBridge.resolve` also refuse to run after a failed+ load, but raise a plain `ValueError`, and they load a row of any size.- **Save failures warn and continue.** The save happens *after* theprovider was called and billed, so raising there would fail a completedturn over a bookkeeping write. `_save_conversation`@@ -216,9 +228,9 @@ time, and a same-generation panel Clear terminates them explicitly`nodes/context/` owns the opt-in descriptor (`_descriptor.py`), two WShandlers (`get_agent_context` returns the **live generation only** —`{conversations, generation, agent_node_id, updated_at, message_count,-messages}`, with an `agent_node_id` selector for nodes shared by several-agents; `clear_agent_context` deletes rows and fences warm claude-processes), and one CloudEvents broadcast (`context.updated`, fired from the+messages}`, where `messages` is the requested `agent_node_id`'s transcript,+else the newest row's; `clear_agent_context` deletes rows and fences warm+claude processes), and one CloudEvents broadcast (`context.updated`, fired from theregistered save listener; payload is identity + count only — the panelrefetches through the authorized handler). The node declares **noparameters**; the connection is the whole configuration.@@ -228,7 +240,18 @@ creates, reconnects or deletes one; rewriting a legacy`simpleMemory -> input-memory` edge is the one exception. Save never restoresa deleted Context, and `workflow_validator` accepts an agent without one: itchecks only the Context edges that exist (`INVALID_CONTEXT_EDGE`,-`MULTIPLE_CONTEXTS`, `SHARED_CONTEXT`). Deleting the node is the opt-out.+`MULTIPLE_CONTEXTS`, `SHARED_CONTEXT`), and `handle_save_workflow` refuses a+graph with any of them (`invalid_context_topology`). A Context node therefore+serves exactly one agent. A leftover `simpleMemory -> input-memory` edge only+produces a `LEGACY_MEMORY_EDGE` warning. Deleting the node is the opt-out.++**Known gap: the panel is not narrowed to its own agent.** `get_agent_context`+lists every stored conversation in the workflow's live generation, not only+the one for the agent wired to this Context node. In a workflow with two+agents, each on its own Context node, each panel lists both agents in its+selector and opens on the newest row, which can be the other agent's; the+badge shows whose conversation it is. Clearing from the panel clears the+agent currently shown.The panel ([`ContextPanel.tsx`](../client/src/components/parameterPanel/ContextPanel.tsx))renders role-tinted message cards with per-message `ts` timestamps, routes@@ -260,10 +283,10 @@ workflow, not only the removed Context's agent.| 1 | The store observes; it never steers. Requests are always built from `messages`; `conversation_key` only says where to save. | The original `context_ref` regression made attaching a Context node change what the agent sent. || 2 | One key per agent per generation: `(workflow_id, generation, agent_node_id)`. Every firing — chat or task review — continues that one conversation. | Per-firing/per-session keys are how the lead came back amnesiac (`messages=2`). || 3 | Seeding precedence: carried transcript > stored conversation > bare build. | Rollover mid-run truth beats the store; the store beats cold start. |-| 4 | Load failures are LOUD (`ConversationLoadFailed` / `ConversationTooLarge`); save failures are best-effort. | Never burn tokens on an amnesiac prompt; never fail a billed turn over bookkeeping. |+| 4 | Load failures are LOUD (`ConversationLoadFailed` / `ConversationTooLarge` on Temporal; a `ValueError` on the in-process and specialized paths, which have no size cap); save failures are best-effort. | Never burn tokens on an amnesiac prompt; never fail a billed turn over bookkeeping. || 5 | Save the exact sent list, after the provider call. Nothing writes to the store before a request exists. | The journal's `prepare_context` wrote fabricated requests assembled from configuration. || 6 | Reset = new generation = new key, AND the Context node's reset hook clears the workflow's stored rows. | The panel shows the newest stored generation, so surviving rows make Reset look like a no-op. |-| 7 | `input-memory` is retired; the conversation store is the continuity carrier. Do not resurrect markdown seeding. | `normalize_workflow_graph` migrates it away and the validator rejects it; two carriers would drift. |+| 7 | `input-memory` is retired; the conversation store is the continuity carrier. Do not resurrect markdown seeding. | `normalize_workflow_graph` migrates it away and the validator warns on any left (`LEGACY_MEMORY_EDGE`); two carriers would drift. || 8 | Specialized bridges record the ORIGINAL prompt, never the augmented one. | Recording the rendered transcript nests the conversation inside itself and grows without bound. || 9 | The store never imports `nodes/`; the plugin registers its broadcaster via `register_conversation_listener`, and a listener failure can never fail a save. | Same layering rule as every plugin registry; a UI notification must not break execution. || 10 | External tool results are capped before they enter the transcript; the latest turn is never cleared or summarized; the pressure rules are chosen by the recorded `context_pressure_version`. | Uncapped results bricked every later firing (`errors.md` #28); an unread turn summarized away loses what the model asked for; a replay must schedule the commands it recorded. |diff --git a/docs-internal/agent_teams.md b/docs-internal/agent_teams.mdindex 29f414f6..636c4307 100644--- a/docs-internal/agent_teams.md+++ b/docs-internal/agent_teams.md@@ -33,6 +33,8 @@ leads cannot receive team tasks.selected teammate after persistence but are hidden from the lead's LLM.5. Temporal and legacy execution share the durable task lifecycle and use thesame precreated task ID; neither path creates a second tracking record.+ The legacy path does not currently start the teammate; see+ "Known limitations" below.## Topology@@ -53,20 +55,33 @@ delegation depth beyond two child layers.```textblocked -> queued -> running -> submitted -> accepted- | |- +-> failed +-> retry/reassign -> queued- +-> cancelled+ |+ +-> failed```+Lead-driven transitions (`AgentTeamService.mutate_durable_task`):++| Operation | Allowed from | Result |+|---|---|---|+| `modify_task` | `blocked`, `queued` | edits title, mission, context or acceptance criteria |+| `cancel_task` | `blocked`, `queued`, `running` | `cancelled` |+| `accept_task` | `submitted` | `accepted` |+| `retry_task` / `reassign_task` | `failed`, `submitted`, `cancelled` | `queued`, as a new attempt |+- `submitted` means the worker finished and the lead must review the result.- `accepted` is the completed/Done state shown by Team Monitor.-- `failed` is unresolved until retried, reassigned, or intentionally cancelled.+- `failed` is unresolved until retried or reassigned; a failed task cannot be+ cancelled.+- `cancelled` is not final: a cancelled task can be retried or reassigned.- `finish_team` succeeds only when every task is accepted or cancelled.-Mutations are optimistic and execution-scoped. Callers pass `task_id` and-`expected_revision`; the service verifies team ownership and current state.-When exactly one task is submitted, `accept_task` may safely infer its ID and-current revision. It never guesses among multiple submissions.+Mutations are optimistic and execution-scoped. Callers pass `task_id` and,+optionally, `expected_revision`; the service verifies team ownership and+current state. When `expected_revision` is omitted, Task Manager reads the+task's current revision first, so the check then only covers the moment+between that read and the write. When exactly one task is submitted,+`accept_task` may safely infer its ID and current revision. It never guesses+among multiple submissions.## Parallel scheduling@@ -117,14 +132,26 @@ execution tasks. Its Done count includes accepted tasks, not submitted work.| `assign_task` | Persist and dispatch work to a connected teammate || `list_tasks` / `get_task` | Inspect state, result, attempts, and revision || `modify_task` | Edit blocked or queued work |-| `cancel_task` | Stop queued or running work |-| `retry_task` | Queue a new attempt |+| `cancel_task` | Stop blocked, queued or running work |+| `retry_task` | Queue a new attempt of failed, submitted or cancelled work on the same teammate || `reassign_task` | Queue a new attempt on another connected teammate || `accept_task` | Approve submitted work |+| `inspect_task_trace` | Read or search the sanitized Temporal events of a task attempt (`detail`: `summary`, `failures`, `timeline` or `search`) || `finish_team` | Complete a fully resolved team execution |`mark_done` is a deprecated alias for `accept_task` and never deletes history.+## Known limitations++- **Off Temporal, Task Manager does not start teammates.** `assign_task`,+ `retry_task` and `reassign_task` persist the task as `queued` and return a+ `delegation_request`, which only `AgentWorkflow` consumes. The in-process+ branch in `_execute_task_manager` needs `ai_service` and `database` in its+ config, and the plugin tool path in `services/handlers/tools.py` does not+ pass them. So when the lead runs in-process (`AIService.execute_agent` /+ `execute_chat_agent`) rather than as an `AgentWorkflow`, the task stays+ queued and no teammate runs it.+## Key files- `server/services/plugin/edge_walker.py` — canonical teammate discovery anddiff --git a/docs-internal/memory_compaction.md b/docs-internal/memory_compaction.mdindex 0fee58a0..e8803467 100644--- a/docs-internal/memory_compaction.md+++ b/docs-internal/memory_compaction.md@@ -71,8 +71,9 @@ delivery and a repeated summarizer request can be billed twice.`anthropic_config()`, `anthropic_api_config()`, and `openai_config()` remainconfiguration helpers on `CompactionService`. The current chat and agentexecution paths do not pass those provider-managed compaction controls to the-SDK. `AgentWorkflow` currently calls `anthropic_config()` only as a-provider-agnostic way to calculate its numeric threshold.+SDK. On the Temporal path, `prepare_agent_payload` (the `agent.prepare_payload`+activity) calls `anthropic_config()` only as a provider-agnostic way to+calculate the numeric threshold `AgentWorkflow` compacts at.## Database Schema@@ -291,22 +292,28 @@ paths are unified; do not promise that they disable or override every run.### get_compaction_stats-Get token usage statistics for a session:+Get token usage statistics for a session. `model` and `provider` are optional;+with both, `threshold` is the model-aware value and `context_length` is the+model's context window (otherwise `context_length` is 0). A per-session+`custom_threshold` always wins.```javascript// Client requestws.send(JSON.stringify({type: "get_compaction_stats",- session_id: "user-session-123"+ session_id: "user-session-123",+ model: "claude-haiku-4-5", // optional; 200K context window+ provider: "anthropic" // optional}));-// Server response+// Server response (CompactionService.stats; the ws_handler decorator adds "success"){"type": "get_compaction_stats","success": true,"session_id": "user-session-123","total": 45000,- "threshold": 100000,+ "threshold": 160000, // 0.8 (default ratio) x 200000+ "context_length": 200000,"count": 0}```@@ -442,7 +449,7 @@ if session_id and ai_response:| `server/core/database.py` | CRUD methods for metrics and events || `server/core/config.py` | Environment variable configuration || `server/core/container.py` | Dependency injection setup |-| `server/services/settings/handlers.py` | WebSocket handlers `get_compaction_stats` (`handle_get_compaction_stats`) and `configure_compaction` (`handle_configure_compaction`); registered into `MESSAGE_HANDLERS` by `routers/websocket.py` |+| `server/services/settings/handlers.py` | WebSocket handlers `get_compaction_stats` (`handle_get_compaction_stats`) and `configure_compaction` (`handle_configure_compaction`); listed in its `WS_HANDLERS` dict, which `server/services/settings/__init__.py` registers through `services.ws_handler_registry.register_ws_handlers` on import (not the core `MESSAGE_HANDLERS` dict) || `server/main.py` | Service initialization on startup |## Design Decisionsdiff --git a/server/nodes/context/_handlers.py b/server/nodes/context/_handlers.pyindex f4f82a19..223142ba 100644--- a/server/nodes/context/_handlers.py+++ b/server/nodes/context/_handlers.py@@ -116,10 +116,11 @@ async def handle_get_agent_context(The panel shows the agent's CURRENT context only: rows from the neweststored generation. Prior generations stay in the store as inert history(cleared with the workflow) but are deliberately not browsable here.- ``conversations`` lists the live generation's agents so the panel can- offer a selector when several agents share the Context node;- ``messages`` is the transcript of the requested ``agent_node_id``, else- the newest row.+ ``conversations`` lists every stored conversation in the live+ generation, not only the one for the agent wired to this Context node+ (save rejects a Context shared by several agents), and the panel offers+ them in a selector; ``messages`` is the transcript of the requested+ ``agent_node_id``, else the newest row."""workflow_id = str(data.get("workflow_id") or "")diff --git a/server/routers/websocket.py b/server/routers/websocket.pyindex a299b53f..2b5b7360 100644--- a/server/routers/websocket.py+++ b/server/routers/websocket.py@@ -1368,8 +1368,8 @@ async def handle_get_node_allowlist(data: Dict[str, Any], websocket: WebSocket)# ============================================================================# Compaction Handlers — extracted to services/settings/handlers.py (Wave 13.3)# ============================================================================-# get_compaction_stats + configure_compaction moved. get_provider_usage_summary-# stays here for now (Wave 13.8 pricing extraction will move it).+# get_compaction_stats + configure_compaction moved, and so did+# get_provider_usage_summary; all three self-register from services/settings.# ============================================================================diff --git a/server/services/compaction.py b/server/services/compaction.pyindex b4efc979..d27882d4 100644--- a/server/services/compaction.py+++ b/server/services/compaction.py@@ -1,11 +1,17 @@-"""Memory compaction service using native provider APIs.--Anthropic: tool_runner with compaction_control parameter-OpenAI: context_management with compact_threshold-Others: Client-side summarization fallback--Threshold strategy: per-session custom_threshold > model-aware threshold > global default.-Model-aware threshold = 50% of model's context window (e.g., 100K for a 200K model).+"""Memory compaction service: token tracking plus client-side summarization.++Every provider compacts the same way: :meth:`CompactionService.compact_context`+sends the text to summarize through ``run_native_llm_step`` (ChatUnifier).+``anthropic_config`` / ``anthropic_api_config`` / ``openai_config`` build+provider-managed compaction settings, but no execution path passes them to+an SDK; ``anthropic_config`` is used only to compute a numeric threshold.++Threshold strategy: per-session custom_threshold > model-aware threshold.+Model-aware threshold = compaction ratio x the model's context window, never+below 10,000 tokens. The ratio is the per-user ``compaction_ratio`` when it is+set and between 0.1 and 0.9, else ``COMPACTION_RATIO`` (default 0.8);+``agent.compaction.ratio`` in ``llm_defaults.json`` is read only if Settings+cannot load. See :meth:`CompactionService._get_compaction_ratio`."""from dataclasses import asdict@@ -22,9 +28,11 @@ if TYPE_CHECKING:logger = get_logger(__name__)-# Compaction ratio is sourced from server/config/llm_defaults.json-# (``agent.compaction.ratio``). Per-user override in ``user_settings`` still-# wins via :meth:`CompactionService._get_compaction_ratio`.+# The global compaction ratio comes from ``ModelRegistryService.get_agent_defaults``:+# env ``COMPACTION_RATIO`` (core.config.Settings, default 0.8) wins, and+# ``agent.compaction.ratio`` in server/config/llm_defaults.json is used only+# when Settings cannot load. A per-user override in ``user_settings`` wins+# over both via :meth:`CompactionService._get_compaction_ratio`.def _extract_text_from_response(content) -> str:diff --git a/server/services/settings/__init__.py b/server/services/settings/__init__.pyindex 32e4df9f..fef6a984 100644--- a/server/services/settings/__init__.py+++ b/server/services/settings/__init__.py@@ -1,8 +1,9 @@"""Settings domain — Wave 13.3 extraction from ``routers/websocket.py``.-Side-effect import registers the 8 settings handlers (user_settings +-provider_defaults + validated_ai_providers + global_model + compaction-stats/config) into ``ws_handler_registry``.+Side-effect import registers the settings handlers listed in+``handlers.WS_HANDLERS`` (user_settings + provider_defaults ++validated_ai_providers + global_model + compaction stats/config ++provider usage summary) into ``ws_handler_registry``."""from __future__ import annotationsdiff --git a/server/services/settings/handlers.py b/server/services/settings/handlers.pyindex c8cf65f1..5a7887c7 100644--- a/server/services/settings/handlers.py+++ b/server/services/settings/handlers.py@@ -236,7 +236,8 @@ async def handle_save_global_model(data: Dict[str, Any], websocket: WebSocket) -async def handle_get_compaction_stats(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:"""Get compaction statistics for a session.- Optional model/provider params enable model-aware threshold (50% of context window).+ Optional model/provider params enable the model-aware threshold (the+ compaction ratio times the model's context window)."""from services.compaction import get_compaction_servicediff --git a/server/services/skill_loader.py b/server/services/skill_loader.pyindex 2c08a138..29913a77 100644--- a/server/services/skill_loader.py+++ b/server/services/skill_loader.py@@ -144,12 +144,14 @@ class SkillLoader:def _parse_skill_metadata(self, skill_md_path: Path) -> Optional[SkillMetadata]:"""Parse SKILL.md frontmatter to extract metadata.- Icon and color are NOT read from the SKILL.md frontmatter —- they are resolved from the target node's visuals.json entry- via the central :mod:`nodes._visuals` handler. The first- entry in ``allowed-tools`` that resolves to a known node- becomes the visual source. This guarantees skills mirror- their teaching node's icon / color without per-file drift.+ Icon and color are resolved from the target node through+ :mod:`nodes._visuals` (see the chain in the body below); the+ first entry in ``allowed-tools`` that resolves to a known node+ becomes the visual source, and its values override any+ frontmatter ``metadata.icon`` / ``metadata.color``. Only skills+ with no node target keep their inline metadata. Unlike+ ``BaseNode._metadata_dict``, this chain does not consult+ ``meta.json`` icon references (``get_plugin_icon_ref``)."""content = skill_md_path.read_text(encoding="utf-8")diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.pyindex dea40529..e3545eee 100644--- a/server/services/temporal/agent_activities.py+++ b/server/services/temporal/agent_activities.py@@ -1175,8 +1175,9 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:)# ---- Compaction threshold ------------------------------------------- # Model-aware threshold (50% of context window per agent.compaction.ratio- # in llm_defaults.json). Reuse the existing CompactionService helper.+ # Model-aware threshold (the compaction ratio times the model's context+ # window; see services/compaction.py for where the ratio comes from).+ # Reuse the existing CompactionService helper.# ``anthropic_config`` is async (awaits _get_compaction_ratio) — must# be awaited. ``get_compaction_service`` returns Optional[...] so the# service may not be initialized yet (e.g. worker bootstrap order).diff --git a/server/skills/GUIDE.md b/server/skills/GUIDE.mdindex 7919af39..d53df536 100644--- a/server/skills/GUIDE.md+++ b/server/skills/GUIDE.md@@ -100,7 +100,7 @@ Provide usage guidelines, examples, and constraints.| `name` | Yes | Lowercase, hyphens only. Must match pattern `^[a-z0-9]+(-[a-z0-9]+)*$` || `description` | Yes | One-line summary shown in skill lists and to the LLM || `allowed-tools` | No | Space-delimited list of LLM-facing tool names (snake_case, e.g. `stripe_action`, `apify_actor`, `whatsapp_send`). The first token whose snake→camel conversion (`stripe_action` → `stripeAction`) matches a node `type` in `server/nodes/visuals.json` becomes the skill's visual source — its icon/color come from that entry via the `_visuals` handler. **Convention:** the LLM tool name MUST be the snake_case form of the node type (e.g. `stripeAction` → `stripe_action`). The same string is the plugin's `tool_name` ClassVar (locked by `server/tests/fixtures/tool_names_snapshot.json`) so the LLM and the skill agree on the name. Mismatching the snake_case (e.g. `stripe_cli` for a `stripeAction` node) silently breaks icon resolution. |-| `metadata` | No | Arbitrary key-value pairs (author, version, category). **Don't set `icon`/`color` for skills that target a node** — those are resolved from the target node's `visuals.json` entry so the skill always mirrors the canvas. Skills with no node target (personality skills, memory operators, autonomous patterns) **may** set inline `icon` and `color` — they're the only visual source for those. |+| `metadata` | No | Arbitrary key-value pairs (author, version, category). **Don't set `icon`/`color` for skills that target a node** — those are resolved from the first `allowed-tools` entry that names a node (`SkillLoader._parse_skill_metadata`): the icon from that plugin's `icon_<type>.svg` / `icon.svg`, else its `visuals.json` entry; the colour from its `meta.json`, else `visuals.json`. `meta.json` icon references (`lucide:...`) are not consulted, so a skill targeting a node that uses only such a reference gets no icon from it. Skills with no node target (personality skills, memory operators, autonomous patterns) **may** set inline `icon` and `color` — they're the only visual source for those. |### Name Format Rules@@ -194,7 +194,7 @@ Discover.2. **No per-skill nodes**: Master Skill is the only skill node (`SKILL_NODE_TYPES` in `server/constants.py`); it connects to an agent's `input-skill` handle. There are no dedicated palette nodes per built-in skill.-3. **At Execution Time**: Only personality skills (names ending in `-personality`) are injected eagerly: their full instructions are appended to the system prompt (`server/services/skill_prompt.py`). Every other enabled skill uses progressive disclosure through the bound `Skill` tool (`server/services/skill_runtime.py`): the tool description carries a bounded name/description catalogue, `load` returns a skill's instructions and resource manifest, and `read_resource` / `search_resource` read its declared files. The system prompt is not changed for these skills.+3. **At Execution Time**: Only personality skills (names ending in `-personality`) are injected eagerly (`server/services/skill_prompt.py`). On aiAgent and chatAgent, in-process and on the Temporal agent path, their full instructions **replace** the agent's system message, so the node's own `system_message` parameter is dropped. The RLM agent is the exception: it puts them in front of the task prompt under a `## Skill Instructions` heading and keeps its system message. Every other enabled skill uses progressive disclosure through the bound `Skill` tool (`server/services/skill_runtime.py`): the tool description carries a bounded name/description catalogue, `load` returns a skill's instructions and resource manifest, and `read_resource` / `search_resource` read its declared files. The system prompt is not changed for these skills.## Skill Content Lifecycle