Skip to content

Self-improving AI Employees for everyone

docs(agents): correct skill, compaction, context and team-task docs

trohitgadbfcf14176a
diff --git a/docs-internal/agent_context_flow.md b/docs-internal/agent_context_flow.md
index 7fc02348..3bf3946b 100644
--- a/docs-internal/agent_context_flow.md
+++ b/docs-internal/agent_context_flow.md
@@ -102,8 +102,17 @@ build** and is intentional:
The carried transcript caps at ~1 MB serialized
(`_CAN_TRANSCRIPT_MAX_BYTES`) because Temporal's payload error limit is
-2 MiB for the whole continue-as-new argument; over-cap degrades to the
-opening prompt with a warning — never a failure.
+2 MiB for the whole continue-as-new argument. Over the cap, the transcript is
+dropped with a warning and the resumed run seeds like a fresh firing: from
+the stored conversation when a Context node is connected (with the firing's
+prompt appended again), else from the opening prompt.
+
+A continue-as-new starts `run()` from the top, so `agent.prepare_payload`
+runs again. The resumed run picks up the node's current configuration (tools,
+system message, model) and reloads the stored conversation with the load
+checks below, even when the carried transcript is what seeds it. If that
+stored row is over the 1 MB seed cap, the resumed run fails with
+`ConversationTooLarge`.
## Transcript size: capped results, cleared old results
@@ -184,7 +193,10 @@ only grow until the provider rejected it.
Running on silently instead would burn tokens on an amnesiac prompt — the
exact failure mode this design replaced. With the cap and the byte budget
above, new transcripts stay far below the limit; a row saved before them
- needs one clear.
+ needs one clear. Both error types and the 1 MB cap exist only on the
+ Temporal path. The in-process `_prepare_context` (`services/ai.py`) and
+ `SpecializedAgentContextBridge.resolve` also refuse to run after a failed
+ load, but raise a plain `ValueError`, and they load a row of any size.
- **Save failures warn and continue.** The save happens *after* the
provider was called and billed, so raising there would fail a completed
turn over a bookkeeping write. `_save_conversation`
@@ -216,9 +228,9 @@ time, and a same-generation panel Clear terminates them explicitly
`nodes/context/` owns the opt-in descriptor (`_descriptor.py`), two WS
handlers (`get_agent_context` returns the **live generation only** —
`{conversations, generation, agent_node_id, updated_at, message_count,
-messages}`, with an `agent_node_id` selector for nodes shared by several
-agents; `clear_agent_context` deletes rows and fences warm claude
-processes), and one CloudEvents broadcast (`context.updated`, fired from the
+messages}`, where `messages` is the requested `agent_node_id`'s transcript,
+else the newest row's; `clear_agent_context` deletes rows and fences warm
+claude processes), and one CloudEvents broadcast (`context.updated`, fired from the
registered save listener; payload is identity + count only — the panel
refetches through the authorized handler). The node declares **no
parameters**; the connection is the whole configuration.
@@ -228,7 +240,18 @@ creates, reconnects or deletes one; rewriting a legacy
`simpleMemory -> input-memory` edge is the one exception. Save never restores
a deleted Context, and `workflow_validator` accepts an agent without one: it
checks only the Context edges that exist (`INVALID_CONTEXT_EDGE`,
-`MULTIPLE_CONTEXTS`, `SHARED_CONTEXT`). Deleting the node is the opt-out.
+`MULTIPLE_CONTEXTS`, `SHARED_CONTEXT`), and `handle_save_workflow` refuses a
+graph with any of them (`invalid_context_topology`). A Context node therefore
+serves exactly one agent. A leftover `simpleMemory -> input-memory` edge only
+produces a `LEGACY_MEMORY_EDGE` warning. Deleting the node is the opt-out.
+
+**Known gap: the panel is not narrowed to its own agent.** `get_agent_context`
+lists every stored conversation in the workflow's live generation, not only
+the one for the agent wired to this Context node. In a workflow with two
+agents, each on its own Context node, each panel lists both agents in its
+selector and opens on the newest row, which can be the other agent's; the
+badge shows whose conversation it is. Clearing from the panel clears the
+agent currently shown.
The panel ([`ContextPanel.tsx`](../client/src/components/parameterPanel/ContextPanel.tsx))
renders role-tinted message cards with per-message `ts` timestamps, routes
@@ -260,10 +283,10 @@ workflow, not only the removed Context's agent.
| 1 | The store observes; it never steers. Requests are always built from `messages`; `conversation_key` only says where to save. | The original `context_ref` regression made attaching a Context node change what the agent sent. |
| 2 | One key per agent per generation: `(workflow_id, generation, agent_node_id)`. Every firing — chat or task review — continues that one conversation. | Per-firing/per-session keys are how the lead came back amnesiac (`messages=2`). |
| 3 | Seeding precedence: carried transcript > stored conversation > bare build. | Rollover mid-run truth beats the store; the store beats cold start. |
-| 4 | Load failures are LOUD (`ConversationLoadFailed` / `ConversationTooLarge`); save failures are best-effort. | Never burn tokens on an amnesiac prompt; never fail a billed turn over bookkeeping. |
+| 4 | Load failures are LOUD (`ConversationLoadFailed` / `ConversationTooLarge` on Temporal; a `ValueError` on the in-process and specialized paths, which have no size cap); save failures are best-effort. | Never burn tokens on an amnesiac prompt; never fail a billed turn over bookkeeping. |
| 5 | Save the exact sent list, after the provider call. Nothing writes to the store before a request exists. | The journal's `prepare_context` wrote fabricated requests assembled from configuration. |
| 6 | Reset = new generation = new key, AND the Context node's reset hook clears the workflow's stored rows. | The panel shows the newest stored generation, so surviving rows make Reset look like a no-op. |
-| 7 | `input-memory` is retired; the conversation store is the continuity carrier. Do not resurrect markdown seeding. | `normalize_workflow_graph` migrates it away and the validator rejects it; two carriers would drift. |
+| 7 | `input-memory` is retired; the conversation store is the continuity carrier. Do not resurrect markdown seeding. | `normalize_workflow_graph` migrates it away and the validator warns on any left (`LEGACY_MEMORY_EDGE`); two carriers would drift. |
| 8 | Specialized bridges record the ORIGINAL prompt, never the augmented one. | Recording the rendered transcript nests the conversation inside itself and grows without bound. |
| 9 | The store never imports `nodes/`; the plugin registers its broadcaster via `register_conversation_listener`, and a listener failure can never fail a save. | Same layering rule as every plugin registry; a UI notification must not break execution. |
| 10 | External tool results are capped before they enter the transcript; the latest turn is never cleared or summarized; the pressure rules are chosen by the recorded `context_pressure_version`. | Uncapped results bricked every later firing (`errors.md` #28); an unread turn summarized away loses what the model asked for; a replay must schedule the commands it recorded. |
diff --git a/docs-internal/agent_teams.md b/docs-internal/agent_teams.md
index 29f414f6..636c4307 100644
--- a/docs-internal/agent_teams.md
+++ b/docs-internal/agent_teams.md
@@ -33,6 +33,8 @@ leads cannot receive team tasks.
selected teammate after persistence but are hidden from the lead's LLM.
5. Temporal and legacy execution share the durable task lifecycle and use the
same precreated task ID; neither path creates a second tracking record.
+ The legacy path does not currently start the teammate; see
+ "Known limitations" below.
## Topology
@@ -53,20 +55,33 @@ delegation depth beyond two child layers.
```text
blocked -> queued -> running -> submitted -> accepted
- | |
- +-> failed +-> retry/reassign -> queued
- +-> cancelled
+ |
+ +-> failed
```
+Lead-driven transitions (`AgentTeamService.mutate_durable_task`):
+
+| Operation | Allowed from | Result |
+|---|---|---|
+| `modify_task` | `blocked`, `queued` | edits title, mission, context or acceptance criteria |
+| `cancel_task` | `blocked`, `queued`, `running` | `cancelled` |
+| `accept_task` | `submitted` | `accepted` |
+| `retry_task` / `reassign_task` | `failed`, `submitted`, `cancelled` | `queued`, as a new attempt |
+
- `submitted` means the worker finished and the lead must review the result.
- `accepted` is the completed/Done state shown by Team Monitor.
-- `failed` is unresolved until retried, reassigned, or intentionally cancelled.
+- `failed` is unresolved until retried or reassigned; a failed task cannot be
+ cancelled.
+- `cancelled` is not final: a cancelled task can be retried or reassigned.
- `finish_team` succeeds only when every task is accepted or cancelled.
-Mutations are optimistic and execution-scoped. Callers pass `task_id` and
-`expected_revision`; the service verifies team ownership and current state.
-When exactly one task is submitted, `accept_task` may safely infer its ID and
-current revision. It never guesses among multiple submissions.
+Mutations are optimistic and execution-scoped. Callers pass `task_id` and,
+optionally, `expected_revision`; the service verifies team ownership and
+current state. When `expected_revision` is omitted, Task Manager reads the
+task's current revision first, so the check then only covers the moment
+between that read and the write. When exactly one task is submitted,
+`accept_task` may safely infer its ID and current revision. It never guesses
+among multiple submissions.
## Parallel scheduling
@@ -117,14 +132,26 @@ execution tasks. Its Done count includes accepted tasks, not submitted work.
| `assign_task` | Persist and dispatch work to a connected teammate |
| `list_tasks` / `get_task` | Inspect state, result, attempts, and revision |
| `modify_task` | Edit blocked or queued work |
-| `cancel_task` | Stop queued or running work |
-| `retry_task` | Queue a new attempt |
+| `cancel_task` | Stop blocked, queued or running work |
+| `retry_task` | Queue a new attempt of failed, submitted or cancelled work on the same teammate |
| `reassign_task` | Queue a new attempt on another connected teammate |
| `accept_task` | Approve submitted work |
+| `inspect_task_trace` | Read or search the sanitized Temporal events of a task attempt (`detail`: `summary`, `failures`, `timeline` or `search`) |
| `finish_team` | Complete a fully resolved team execution |
`mark_done` is a deprecated alias for `accept_task` and never deletes history.
+## Known limitations
+
+- **Off Temporal, Task Manager does not start teammates.** `assign_task`,
+ `retry_task` and `reassign_task` persist the task as `queued` and return a
+ `delegation_request`, which only `AgentWorkflow` consumes. The in-process
+ branch in `_execute_task_manager` needs `ai_service` and `database` in its
+ config, and the plugin tool path in `services/handlers/tools.py` does not
+ pass them. So when the lead runs in-process (`AIService.execute_agent` /
+ `execute_chat_agent`) rather than as an `AgentWorkflow`, the task stays
+ queued and no teammate runs it.
+
## Key files
- `server/services/plugin/edge_walker.py` — canonical teammate discovery and
diff --git a/docs-internal/memory_compaction.md b/docs-internal/memory_compaction.md
index 0fee58a0..e8803467 100644
--- a/docs-internal/memory_compaction.md
+++ b/docs-internal/memory_compaction.md
@@ -71,8 +71,9 @@ delivery and a repeated summarizer request can be billed twice.
`anthropic_config()`, `anthropic_api_config()`, and `openai_config()` remain
configuration helpers on `CompactionService`. The current chat and agent
execution paths do not pass those provider-managed compaction controls to the
-SDK. `AgentWorkflow` currently calls `anthropic_config()` only as a
-provider-agnostic way to calculate its numeric threshold.
+SDK. On the Temporal path, `prepare_agent_payload` (the `agent.prepare_payload`
+activity) calls `anthropic_config()` only as a provider-agnostic way to
+calculate the numeric threshold `AgentWorkflow` compacts at.
## Database Schema
@@ -291,22 +292,28 @@ paths are unified; do not promise that they disable or override every run.
### get_compaction_stats
-Get token usage statistics for a session:
+Get token usage statistics for a session. `model` and `provider` are optional;
+with both, `threshold` is the model-aware value and `context_length` is the
+model's context window (otherwise `context_length` is 0). A per-session
+`custom_threshold` always wins.
```javascript
// Client request
ws.send(JSON.stringify({
type: "get_compaction_stats",
- session_id: "user-session-123"
+ session_id: "user-session-123",
+ model: "claude-haiku-4-5", // optional; 200K context window
+ provider: "anthropic" // optional
}));
-// Server response
+// Server response (CompactionService.stats; the ws_handler decorator adds "success")
{
"type": "get_compaction_stats",
"success": true,
"session_id": "user-session-123",
"total": 45000,
- "threshold": 100000,
+ "threshold": 160000, // 0.8 (default ratio) x 200000
+ "context_length": 200000,
"count": 0
}
```
@@ -442,7 +449,7 @@ if session_id and ai_response:
| `server/core/database.py` | CRUD methods for metrics and events |
| `server/core/config.py` | Environment variable configuration |
| `server/core/container.py` | Dependency injection setup |
-| `server/services/settings/handlers.py` | WebSocket handlers `get_compaction_stats` (`handle_get_compaction_stats`) and `configure_compaction` (`handle_configure_compaction`); registered into `MESSAGE_HANDLERS` by `routers/websocket.py` |
+| `server/services/settings/handlers.py` | WebSocket handlers `get_compaction_stats` (`handle_get_compaction_stats`) and `configure_compaction` (`handle_configure_compaction`); listed in its `WS_HANDLERS` dict, which `server/services/settings/__init__.py` registers through `services.ws_handler_registry.register_ws_handlers` on import (not the core `MESSAGE_HANDLERS` dict) |
| `server/main.py` | Service initialization on startup |
## Design Decisions
diff --git a/server/nodes/context/_handlers.py b/server/nodes/context/_handlers.py
index f4f82a19..223142ba 100644
--- a/server/nodes/context/_handlers.py
+++ b/server/nodes/context/_handlers.py
@@ -116,10 +116,11 @@ async def handle_get_agent_context(
The panel shows the agent's CURRENT context only: rows from the newest
stored generation. Prior generations stay in the store as inert history
(cleared with the workflow) but are deliberately not browsable here.
- ``conversations`` lists the live generation's agents so the panel can
- offer a selector when several agents share the Context node;
- ``messages`` is the transcript of the requested ``agent_node_id``, else
- the newest row.
+ ``conversations`` lists every stored conversation in the live
+ generation, not only the one for the agent wired to this Context node
+ (save rejects a Context shared by several agents), and the panel offers
+ them in a selector; ``messages`` is the transcript of the requested
+ ``agent_node_id``, else the newest row.
"""
workflow_id = str(data.get("workflow_id") or "")
diff --git a/server/routers/websocket.py b/server/routers/websocket.py
index a299b53f..2b5b7360 100644
--- a/server/routers/websocket.py
+++ b/server/routers/websocket.py
@@ -1368,8 +1368,8 @@ async def handle_get_node_allowlist(data: Dict[str, Any], websocket: WebSocket)
# ============================================================================
# Compaction Handlers — extracted to services/settings/handlers.py (Wave 13.3)
# ============================================================================
-# get_compaction_stats + configure_compaction moved. get_provider_usage_summary
-# stays here for now (Wave 13.8 pricing extraction will move it).
+# get_compaction_stats + configure_compaction moved, and so did
+# get_provider_usage_summary; all three self-register from services/settings.
# ============================================================================
diff --git a/server/services/compaction.py b/server/services/compaction.py
index b4efc979..d27882d4 100644
--- a/server/services/compaction.py
+++ b/server/services/compaction.py
@@ -1,11 +1,17 @@
-"""Memory compaction service using native provider APIs.
-
-Anthropic: tool_runner with compaction_control parameter
-OpenAI: context_management with compact_threshold
-Others: Client-side summarization fallback
-
-Threshold strategy: per-session custom_threshold > model-aware threshold > global default.
-Model-aware threshold = 50% of model's context window (e.g., 100K for a 200K model).
+"""Memory compaction service: token tracking plus client-side summarization.
+
+Every provider compacts the same way: :meth:`CompactionService.compact_context`
+sends the text to summarize through ``run_native_llm_step`` (ChatUnifier).
+``anthropic_config`` / ``anthropic_api_config`` / ``openai_config`` build
+provider-managed compaction settings, but no execution path passes them to
+an SDK; ``anthropic_config`` is used only to compute a numeric threshold.
+
+Threshold strategy: per-session custom_threshold > model-aware threshold.
+Model-aware threshold = compaction ratio x the model's context window, never
+below 10,000 tokens. The ratio is the per-user ``compaction_ratio`` when it is
+set and between 0.1 and 0.9, else ``COMPACTION_RATIO`` (default 0.8);
+``agent.compaction.ratio`` in ``llm_defaults.json`` is read only if Settings
+cannot load. See :meth:`CompactionService._get_compaction_ratio`.
"""
from dataclasses import asdict
@@ -22,9 +28,11 @@ if TYPE_CHECKING:
logger = get_logger(__name__)
-# Compaction ratio is sourced from server/config/llm_defaults.json
-# (``agent.compaction.ratio``). Per-user override in ``user_settings`` still
-# wins via :meth:`CompactionService._get_compaction_ratio`.
+# The global compaction ratio comes from ``ModelRegistryService.get_agent_defaults``:
+# env ``COMPACTION_RATIO`` (core.config.Settings, default 0.8) wins, and
+# ``agent.compaction.ratio`` in server/config/llm_defaults.json is used only
+# when Settings cannot load. A per-user override in ``user_settings`` wins
+# over both via :meth:`CompactionService._get_compaction_ratio`.
def _extract_text_from_response(content) -> str:
diff --git a/server/services/settings/__init__.py b/server/services/settings/__init__.py
index 32e4df9f..fef6a984 100644
--- a/server/services/settings/__init__.py
+++ b/server/services/settings/__init__.py
@@ -1,8 +1,9 @@
"""Settings domain — Wave 13.3 extraction from ``routers/websocket.py``.
-Side-effect import registers the 8 settings handlers (user_settings +
-provider_defaults + validated_ai_providers + global_model + compaction
-stats/config) into ``ws_handler_registry``.
+Side-effect import registers the settings handlers listed in
+``handlers.WS_HANDLERS`` (user_settings + provider_defaults +
+validated_ai_providers + global_model + compaction stats/config +
+provider usage summary) into ``ws_handler_registry``.
"""
from __future__ import annotations
diff --git a/server/services/settings/handlers.py b/server/services/settings/handlers.py
index c8cf65f1..5a7887c7 100644
--- a/server/services/settings/handlers.py
+++ b/server/services/settings/handlers.py
@@ -236,7 +236,8 @@ async def handle_save_global_model(data: Dict[str, Any], websocket: WebSocket) -
async def handle_get_compaction_stats(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:
"""Get compaction statistics for a session.
- Optional model/provider params enable model-aware threshold (50% of context window).
+ Optional model/provider params enable the model-aware threshold (the
+ compaction ratio times the model's context window).
"""
from services.compaction import get_compaction_service
diff --git a/server/services/skill_loader.py b/server/services/skill_loader.py
index 2c08a138..29913a77 100644
--- a/server/services/skill_loader.py
+++ b/server/services/skill_loader.py
@@ -144,12 +144,14 @@ class SkillLoader:
def _parse_skill_metadata(self, skill_md_path: Path) -> Optional[SkillMetadata]:
"""Parse SKILL.md frontmatter to extract metadata.
- Icon and color are NOT read from the SKILL.md frontmatter —
- they are resolved from the target node's visuals.json entry
- via the central :mod:`nodes._visuals` handler. The first
- entry in ``allowed-tools`` that resolves to a known node
- becomes the visual source. This guarantees skills mirror
- their teaching node's icon / color without per-file drift.
+ Icon and color are resolved from the target node through
+ :mod:`nodes._visuals` (see the chain in the body below); the
+ first entry in ``allowed-tools`` that resolves to a known node
+ becomes the visual source, and its values override any
+ frontmatter ``metadata.icon`` / ``metadata.color``. Only skills
+ with no node target keep their inline metadata. Unlike
+ ``BaseNode._metadata_dict``, this chain does not consult
+ ``meta.json`` icon references (``get_plugin_icon_ref``).
"""
content = skill_md_path.read_text(encoding="utf-8")
diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.py
index dea40529..e3545eee 100644
--- a/server/services/temporal/agent_activities.py
+++ b/server/services/temporal/agent_activities.py
@@ -1175,8 +1175,9 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
)
# ---- Compaction threshold ------------------------------------------
- # Model-aware threshold (50% of context window per agent.compaction.ratio
- # in llm_defaults.json). Reuse the existing CompactionService helper.
+ # Model-aware threshold (the compaction ratio times the model's context
+ # window; see services/compaction.py for where the ratio comes from).
+ # Reuse the existing CompactionService helper.
# ``anthropic_config`` is async (awaits _get_compaction_ratio) — must
# be awaited. ``get_compaction_service`` returns Optional[...] so the
# service may not be initialized yet (e.g. worker bootstrap order).
diff --git a/server/skills/GUIDE.md b/server/skills/GUIDE.md
index 7919af39..d53df536 100644
--- a/server/skills/GUIDE.md
+++ b/server/skills/GUIDE.md
@@ -100,7 +100,7 @@ Provide usage guidelines, examples, and constraints.
| `name` | Yes | Lowercase, hyphens only. Must match pattern `^[a-z0-9]+(-[a-z0-9]+)*$` |
| `description` | Yes | One-line summary shown in skill lists and to the LLM |
| `allowed-tools` | No | Space-delimited list of LLM-facing tool names (snake_case, e.g. `stripe_action`, `apify_actor`, `whatsapp_send`). The first token whose snake→camel conversion (`stripe_action` → `stripeAction`) matches a node `type` in `server/nodes/visuals.json` becomes the skill's visual source — its icon/color come from that entry via the `_visuals` handler. **Convention:** the LLM tool name MUST be the snake_case form of the node type (e.g. `stripeAction` → `stripe_action`). The same string is the plugin's `tool_name` ClassVar (locked by `server/tests/fixtures/tool_names_snapshot.json`) so the LLM and the skill agree on the name. Mismatching the snake_case (e.g. `stripe_cli` for a `stripeAction` node) silently breaks icon resolution. |
-| `metadata` | No | Arbitrary key-value pairs (author, version, category). **Don't set `icon`/`color` for skills that target a node** — those are resolved from the target node's `visuals.json` entry so the skill always mirrors the canvas. Skills with no node target (personality skills, memory operators, autonomous patterns) **may** set inline `icon` and `color` — they're the only visual source for those. |
+| `metadata` | No | Arbitrary key-value pairs (author, version, category). **Don't set `icon`/`color` for skills that target a node** — those are resolved from the first `allowed-tools` entry that names a node (`SkillLoader._parse_skill_metadata`): the icon from that plugin's `icon_<type>.svg` / `icon.svg`, else its `visuals.json` entry; the colour from its `meta.json`, else `visuals.json`. `meta.json` icon references (`lucide:...`) are not consulted, so a skill targeting a node that uses only such a reference gets no icon from it. Skills with no node target (personality skills, memory operators, autonomous patterns) **may** set inline `icon` and `color` — they're the only visual source for those. |
### Name Format Rules
@@ -194,7 +194,7 @@ Discover.
2. **No per-skill nodes**: Master Skill is the only skill node (`SKILL_NODE_TYPES` in `server/constants.py`); it connects to an agent's `input-skill` handle. There are no dedicated palette nodes per built-in skill.
-3. **At Execution Time**: Only personality skills (names ending in `-personality`) are injected eagerly: their full instructions are appended to the system prompt (`server/services/skill_prompt.py`). Every other enabled skill uses progressive disclosure through the bound `Skill` tool (`server/services/skill_runtime.py`): the tool description carries a bounded name/description catalogue, `load` returns a skill's instructions and resource manifest, and `read_resource` / `search_resource` read its declared files. The system prompt is not changed for these skills.
+3. **At Execution Time**: Only personality skills (names ending in `-personality`) are injected eagerly (`server/services/skill_prompt.py`). On aiAgent and chatAgent, in-process and on the Temporal agent path, their full instructions **replace** the agent's system message, so the node's own `system_message` parameter is dropped. The RLM agent is the exception: it puts them in front of the task prompt under a `## Skill Instructions` heading and keeps its system message. Every other enabled skill uses progressive disclosure through the bound `Skill` tool (`server/services/skill_runtime.py`): the tool description carries a bounded name/description catalogue, `load` returns a skill's instructions and resource manifest, and `read_resource` / `search_resource` read its declared files. The system prompt is not changed for these skills.
## Skill Content Lifecycle