Skip to content

Self-improving AI Employees for everyone

refactor(llm): migrate agents to native SDKs

trohitg3ca44ae3e93b
diff --git a/.env.template b/.env.template
index 73c541fc..f3d5c90a 100644
--- a/.env.template
+++ b/.env.template
@@ -152,6 +152,11 @@ COMPACTION_RATIO=0.8
# Per-user overrides via the UserSettings row (Settings tab).
AGENT_RECURSION_LIMIT=200
+# Temporary native-SDK rollout switch. This is read only by
+# agent.prepare_payload.v1, so changing it affects newly prepared Temporal
+# executions and never changes the engine of an existing history.
+AGENT_LLM_ENGINE=native
+
# Multi-agent delegation. The concurrency limit counts all active children
# and grandchildren in one root execution; the root agent is not counted.
MAX_CONCURRENT_SUBAGENTS=3
diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.md
index 348f8340..9553dcbd 100644
--- a/docs-internal/TEMPORAL_ARCHITECTURE.md
+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md
@@ -341,22 +341,23 @@ AgentWorkflow.run(context):
templates in prompt / system_message become real values BEFORE
the LLM sees them. Temporal workflows must be deterministic,
so DB lookups + edge walking + tool schema build live here.
- Tool entries carry the raw tool_info dict — execute_llm_step
- rebuilds the StructuredTool via ai_service._build_tool_from_node
- inside the activity (no JSON-schema round-trip).
+ _build_tool_from_node produces AgentToolSpec values. Tool entries
+ carry the serialized ToolDef declaration plus routing tool_info;
+ native LLM steps consume the definition directly.
+ Records llm_engine="native" + message_wire_version=2.
emit_phase("starting", status="executing")
loop until "final" or max_iterations:
1. emit_phase("llm_step", iteration=N)
2. execute_activity("agent.execute_llm_step.v1")
returns {kind, assistant_message, calls?, content?, usage}.
- assistant_message is the full LangChain-serialized AIMessage
- (messages_to_dict round-trip preserves provider-specific
- fields: Gemini thought_signature, Anthropic cache markers,
- OpenAI reasoning_content) — appended verbatim to messages.
- After filter_empty_messages, a list with no human/ai/tool
+ assistant_message is MessageWireV2: ordered text/reasoning/tool
+ blocks plus JSON-safe provider continuation state (Gemini thought
+ signatures, Anthropic signed/redacted thinking, OpenAI response
+ metadata). It is appended verbatim to messages.
+ After filter_empty_messages, a list with no user/assistant/tool
message raises ApplicationError(type="EmptyAgentPrompt",
non_retryable=True) — providers require >=1 non-system
- message (Gemini pulls SystemMessages into system_instruction
+ message (Gemini pulls system messages into system_instruction
and rejects empty contents), so the failure surfaces to the
parent LLM on attempt 1 instead of consuming the retry budget.
3. if kind == "tool_calls":
@@ -377,15 +378,16 @@ AgentWorkflow.run(context):
_serialise_tool_result unwraps F4.A's {success, result, ...}
envelope so the LLM sees only the handler's return value
(matches the in-process tool-call serialisation in
- services/ai.py:_run_agent_loop).
+ services/agent_runtime.py:run_native_agent_loop).
4. for each tool result with an ``operations`` field
(canvas-mutating tools — today only ``agentBuilder``):
if payload["auto_rebind_tools"] is True:
execute_activity("agent.refresh_tools.v1", {operations: …})
- translates add_node ops with component_kind=="tool" OR
- usable_as_tool=True (minus chat-model plugins) into the
- same tool_payload shape prepare_agent_payload emits.
- Reuses ai_service._build_tool_from_node + get_node_class.
+ translates add_node ops with component_kind=="tool" OR
+ usable_as_tool=True (minus chat-model plugins) into the
+ same tool_payload shape prepare_agent_payload emits.
+ Reuses ai_service._build_tool_from_node + get_node_class,
+ yielding fresh AgentToolSpec / ToolDef declarations.
tools.extend(refresh_result["tools"])
tool_index.update(...)
# next execute_llm_step.v1 sees the new tools.
@@ -405,6 +407,16 @@ AgentWorkflow.run(context):
emit_phase("completed", status="success")
```
+The `agent.prepare_payload.v1` result is recorded in history and therefore
+acts as the deterministic engine selector. New executions default to
+`llm_engine="native"` with Message Wire V2 and use `ChatUnifier` plus the
+native provider SDKs for every turn. Histories whose recorded prepare result
+has no engine marker are pre-cutover histories and follow the frozen LangChain
+path with their historical payload, activity commands, and message shape. The
+temporary `AGENT_LLM_ENGINE=langchain` emergency switch can also pin a newly
+prepared execution to that compatibility path without changing an execution
+after it starts. A native run never falls back after a provider request starts.
+
`emit_phase(phase, status?)` is a thin helper that schedules `agent.broadcast_progress.v1`. The activity emits `WorkflowEvent.agent_progress` (CloudEvents v1.0, `type="com.opencompany.agent.progress"`) for FE consumers; when `status` is supplied it also drives a raw-dict `update_node_status` for the canvas-glow color (executing / success / error). Same dual-channel pattern F4.A's `_node_activity` uses. When this workflow is itself a delegated child (`context["parent_node_id"]` set), every `emit_phase` call ALSO schedules a second broadcast against the parent's `node_id` with `phase="delegating"` — the parent's canvas badge then advances in real time while the child loops, instead of freezing at "executing" glow until the child completes.
Each LLM step is one activity and each ordinary tool call is one per-type activity. Team-lead Task Manager assignments are different: persistence happens first, then the lead starts a deterministic detached `DelegatedTaskWorkflow` with `ParentClosePolicy.ABANDON` and receives `queued` immediately. The runner owns the root-wide permit, claim, child `AgentWorkflow`, terminal result/usage persistence, `taskTrigger`, and permit release, so the assigning lead can return without polling. Direct non-team delegation retains the child-workflow path. Non-agent tools and excluded types (`rlm_agent`, `claude_code_agent`) still go through `execute_activity`. Failures surface as durable task failures and trigger review rather than being lost when the lead invocation closes.
@@ -419,9 +431,9 @@ Durable event listeners retain their deployment graph as a fallback, but each fi
| Activity | Purpose |
|---|---|
-| `agent.prepare_payload.v1` | Resolves the DB-backed payload (provider / model / api_key / system_message / user_prompt / tools / memory_node_id / memory_content / memory_window_size / max_iterations / thinking_config / compaction_threshold / auto_rebind_tools). Reads `UserSettings.agent_recursion_limit` + `UserSettings.auto_rebind_tools_after_canvas_change`. Applies the optional `invocation` field (delegation children) after config resolution — per-invocation input always beats stored parameters. |
-| `agent.execute_llm_step.v1` | One LLM turn — rebuilds StructuredTools from the workflow's current `tools` list and returns the assistant message + tool_calls. Guards against un-invokable payloads: post-filter system-only message lists raise `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)`. |
-| `agent.refresh_tools.v1` | Translates `workflow_ops` add_node ops (component_kind="tool" OR usable_as_tool=True) into fresh `tool_payload` entries via `_build_tool_from_node`. Workflow extends `tools` + `tool_index` from the result. |
+| `agent.prepare_payload.v1` | Resolves the DB-backed payload (provider / model / system_message / user_prompt / `AgentToolSpec`-derived tool definitions / memory_node_id / memory_content / memory_window_size / max_iterations / thinking_config / compaction_threshold / auto_rebind_tools), records `llm_engine` + `message_wire_version`, and leaves credential resolution at the LLM activity boundary. Reads `UserSettings.agent_recursion_limit` + `UserSettings.auto_rebind_tools_after_canvas_change`. Applies the optional `invocation` field (delegation children) after config resolution — per-invocation input always beats stored parameters. |
+| `agent.execute_llm_step.v1` | One LLM turn. The native branch decodes Message Wire V2, rebuilds `ToolDef` values, calls `run_native_llm_step(ChatUnifier, ...)` with SDK retries disabled, heartbeats while awaiting the provider, and returns the exact assistant message + tool calls + normalized usage. Guards against un-invokable payloads: post-filter system-only message lists raise `ApplicationError(type="EmptyAgentPrompt", non_retryable=True)`. |
+| `agent.refresh_tools.v1` | Translates `workflow_ops` add_node ops (`component_kind="tool"` OR `usable_as_tool=True`) into fresh `AgentToolSpec`-derived `tool_payload` entries via `_build_tool_from_node`. Workflow extends `tools` + `tool_index` from the result. |
| `agent.persist_turn.v1` | Appends the latest human/assistant exchange to memory markdown, trims the window, broadcasts `node.parameters.updated`. |
| `agent.compact_memory.v1` | Token-budget compaction when cumulative tokens hit the threshold. Best-effort: continues with un-compacted history on failure. |
| `agent.store_output.v1` | Writes `output_main` / `output_top` / `output_0` so downstream nodes resolve `{{aiAgent.response}}` via `ParameterResolver`. |
diff --git a/docs-internal/agent_architecture.md b/docs-internal/agent_architecture.md
index 561dbe2c..80e93428 100644
--- a/docs-internal/agent_architecture.md
+++ b/docs-internal/agent_architecture.md
@@ -60,13 +60,13 @@ collect_agent_connections() server/services/plugin/edge_walker.py
v
AIService.execute_agent() / execute_chat_agent() server/services/ai.py
1. Inject skills into system message
- 2. Build LangChain StructuredTools from tool_data
- 3. Call _run_agent_loop() (plain async while loop)
+ 2. Build provider-neutral AgentToolSpecs from tool_data
+ 3. Call run_native_agent_loop(ChatUnifier, ...)
4. Save memory, return result
|
v
-_run_agent_loop() execution
- invoke LLM -> if tool_calls: dispatch each via tool_executor -> loop
+run_native_agent_loop() execution
+ ChatUnifier -> native SDK -> if tool_calls: dispatch via tool_executor -> loop
Return on final response (no tool_calls) or max_iterations cap.
|
v
@@ -84,7 +84,9 @@ Result broadcast via WebSocket
| `server/services/plugin/edge_walker.py` | `collect_agent_connections()` (the renamed `_collect_agent_connections`), `format_task_context()` |
| `server/nodes/agent/_inline.py` | `prepare_agent_call()` — task-context injection + auto-prompt fallback + teammate collection; calls `collect_agent_connections` then `ai_service.execute_[chat_]agent` |
| `server/nodes/agent/<plugin>/__init__.py` | Per-plugin `execute_op()` (Wave 11 replaced `handle_ai_agent` / `handle_chat_agent`) |
-| `server/services/ai.py` | `AIService` -- `_run_agent_loop`, skill injection, tool building |
+| `server/services/ai.py` | `AIService` facade -- skill injection, tool building, memory and execution-boundary handling |
+| `server/services/agent_runtime.py` | `AgentToolSpec`, `run_native_llm_step`, and `run_native_agent_loop` |
+| `server/services/llm/` | `ChatUnifier`, provider-neutral messages/tool definitions, and native provider SDK adapters |
| `server/services/handlers/tools.py` | `execute_tool()` -- dispatch router for all tool types |
| `server/services/skill_loader.py` | `SkillLoader` -- filesystem/DB skill discovery and loading |
@@ -92,34 +94,64 @@ Result broadcast via WebSocket
## Agent Loop
-The agent loop is a plain `for iteration in range(max_iterations):` async function in `server/services/ai.py:_run_agent_loop`. No state machine, no graph DSL — each iteration:
+The agent loop is the plain async
+`services.agent_runtime.run_native_agent_loop`. No state machine or graph DSL
+is involved — each iteration:
1. Optional `progress_callback(iteration)` so consumers (UI iteration badge) get a per-turn tick.
-2. `filter_empty_messages(messages)` strips empty `HumanMessage` content (Gemini / Claude reject them).
-3. `response = await chat_model.ainvoke(filtered)` — single LLM call. The full assistant message is appended to `messages` verbatim so provider-specific metadata survives (Gemini `thought_signature`, Anthropic cache markers, OpenAI `reasoning_content`).
-4. `extract_thinking_from_response(response)` — accumulates thinking across iterations with the `--- Iteration N ---` separator (multi-step reasoning).
-5. If `response.tool_calls` is empty → return `{messages, iteration, thinking_content, truncated: False}`.
-6. Otherwise dispatch each `tool_call` via the supplied `tool_executor`, wrap each result as a `ToolMessage`, append. After all calls in this iteration return, inspect each result for a `operations` field (workflow_ops batch from canvas-mutating tools); call `rebind_from_operations(ops)` if wired, extend the local `current_tools` list, and `chat_model.bind_tools(current_tools)` again so the LLM sees the new tools in the NEXT iteration. Loop.
-
-On hitting `max_iterations`, append a terminal `AIMessage` with a truncation note and return `truncated: True`.
+2. `filter_empty_messages(messages)` strips empty native message content
+ rejected by providers.
+3. `run_native_llm_step` calls `ChatUnifier.chat`, which invokes the selected
+ native provider SDK and returns `LLMResponse`.
+4. The canonical `LLMResponse.assistant_message` is appended verbatim before
+ any tool runs, preserving Gemini thought signatures, Anthropic
+ signed/redacted thinking, and OpenAI continuation metadata. Normalized
+ `Usage` is accumulated across iterations.
+5. Thinking text is accumulated across iterations with the
+ `--- Iteration N ---` separator.
+6. If `response.tool_calls` is empty → return
+ `{messages, iteration, thinking_content, truncated: False, usage,
+ response}`.
+7. Otherwise validate each call against its `AgentToolSpec.args_schema`,
+ dispatch it through `tool_executor`, and append a native
+ `Message(role="tool", tool_call_id=...)`. Malformed arguments become a
+ deterministic tool error containing `raw_arguments` rather than crashing.
+ Results containing workflow `operations` are passed to
+ `rebind_from_operations`; returned `AgentToolSpec` values extend
+ `current_tools` for the next request.
+
+On hitting `max_iterations`, append a terminal native assistant `Message`
+with a truncation note and return `truncated: True`.
### Signature
```python
-async def _run_agent_loop(
- chat_model,
- initial_messages: List[BaseMessage],
+async def run_native_agent_loop(
+ chat_unifier,
*,
- tools: Optional[List[Any]] = None,
+ provider: str,
+ api_key: str,
+ model: str,
+ temperature: float,
+ max_tokens: int,
+ initial_messages: Sequence[Message],
+ thinking: Optional[ThinkingConfig] = None,
+ tools: Optional[Sequence[AgentToolSpec]] = None,
tool_executor: Optional[Callable] = None,
max_iterations: int = 500,
progress_callback: Optional[Callable[[int], Any]] = None,
- rebind_from_operations: Optional[Callable[[List[Dict[str, Any]]], Awaitable[List[Any]]]] = None,
+ rebind_from_operations: Optional[
+ Callable[[List[Dict[str, Any]]], Awaitable[List[AgentToolSpec]]]
+ ] = None,
) -> Dict[str, Any]:
- """Returns {messages, iteration, thinking_content, truncated}."""
+ """Returns messages, iteration, thinking, truncation, usage, and response."""
```
-Tools bind at loop start via `chat_model.bind_tools(tools)` and **rebind** mid-loop whenever a tool returns workflow_ops (today only `agentBuilder`). The rebound model is the SAME LangChain `chat_model.bind_tools` method every provider honours — no per-provider plumbing.
+There is no mutable model binding step. Each native LLM request derives its
+provider-facing declarations from the current `AgentToolSpec.definition`
+values. When a canvas-mutating tool returns workflow operations (today only
+`agentBuilder`), newly built specs extend that list and are visible on the
+next iteration.
### Termination
@@ -127,7 +159,7 @@ Tools bind at loop start via `chat_model.bind_tools(tools)` and **rebind** mid-l
|---|---|
| LLM emits no `tool_calls` | Return with `truncated: False`. This is the normal exit. |
| `tool_executor` is `None` but LLM emits tool calls | WARN + return (treat as final). |
-| `iteration` reaches `max_iterations` | Append synthetic terminal `AIMessage` + return with `truncated: True`. |
+| `iteration` reaches `max_iterations` | Append a synthetic native assistant `Message` + return with `truncated: True`. |
### `max_iterations` precedence
@@ -146,7 +178,13 @@ When `agentBuilder` (the only canvas-mutating tool today) spawns a new node mid-
Closure responsibilities:
-- **`_rebind_from_operations(ops) -> List[StructuredTool]`** (in `execute_agent` / `execute_chat_agent`): filter ops for `add_node` with the plugin class's `component_kind == "tool"` OR `usable_as_tool=True` (excluding `component_kind == "model"`), synthesize a `tool_info` dict, call `self._build_tool_from_node(tool_info)`, return the new StructuredTools. Tool configs get folded into the captured `tool_configs` dict so the `tool_executor` can dispatch the new call.
+- **`_rebind_from_operations(ops) -> List[AgentToolSpec]`** (in
+ `execute_agent` / `execute_chat_agent`): filter ops for `add_node` with the
+ plugin class's `component_kind == "tool"` OR `usable_as_tool=True`
+ (excluding `component_kind == "model"`), synthesize a `tool_info` dict,
+ call `self._build_tool_from_node(tool_info)`, and return the new specs. Tool
+ configs get folded into the captured `tool_configs` dict so
+ `tool_executor` can dispatch the new call.
- The closure is gated on the user toggle: `UserSettings.auto_rebind_tools_after_canvas_change` (default `True`). When off, the LLM is told "Available on your next turn" in the operation summary and the closure isn't wired.
For the F4.B Temporal path, the in-process closure is replaced by the `agent.refresh_tools.v1` activity; see [TEMPORAL_ARCHITECTURE.md](TEMPORAL_ARCHITECTURE.md).
@@ -158,7 +196,10 @@ Two callsites in `server/services/ai.py`:
- `execute_agent` — for `aiAgent` plugins.
- `execute_chat_agent` — for `chatAgent` + all specialized agents + team leads.
-Both build `initial_messages` (system + memory history + current prompt + skill injection), build tools via `_build_tool_from_node`, then call `_run_agent_loop` and extract the final assistant message + accumulated thinking + iteration count from the returned dict.
+Both build native `initial_messages` (system, memory history, current prompt,
+and skill injection), build tools via `_build_tool_from_node`, then call
+`run_native_agent_loop` and extract the final assistant message, accumulated
+thinking, usage, and iteration count from the returned dict.
---
@@ -356,11 +397,20 @@ This text is appended to the system message. The full SKILL.md body (instruction
## Tool Building Pipeline
-Tool building + dispatch lives in [tool_building_pipeline.md](./tool_building_pipeline.md). The five stages (DISCOVER edges → BUILD StructuredTool → BIND via `chat_model.bind_tools` → INVOKE via LLM tool_calls → DISPATCH through execute_tool) and the dispatch matrix (delegated agents / Android services / dual-purpose plugins / direct tools / search APIs / generic fallback) are documented there. The single-source-of-truth rule — execute_tool owns the tool node lifecycle, parent-agent closures only emit phase broadcasts — is the contract test enforces.
+Tool building + dispatch lives in
+[tool_building_pipeline.md](./tool_building_pipeline.md). The five stages
+(DISCOVER edges → BUILD `AgentToolSpec` / `ToolDef` → EXPOSE through
+`ChatUnifier` → INVOKE via native tool calls → DISPATCH through
+`execute_tool`) and the dispatch matrix (delegated agents / Android services /
+dual-purpose plugins / direct tools / search APIs / generic fallback) are
+documented there. The single-source-of-truth rule — `execute_tool` owns the
+tool node lifecycle, parent-agent closures only emit phase broadcasts — is the
+contract tests enforce.
Patterns covered in the canonical doc:
-- DEFAULT_TOOL_NAMES map (node_type → LLM-visible tool name)
+- Plugin `tool_name` / `tool_description` class variables and database
+ overrides (node type → LLM-visible identity)
- Database tool-schema override via ToolSchema model + Tool Schema Editor UI
- Direct tool pattern (each Android service connects to the agent independently)
- Direct Android service tools (16 entries, skip the toolkit)
@@ -374,7 +424,18 @@ Patterns covered in the canonical doc:
## Memory Integration
-Memory load + save lives in [memory_lifecycle.md](./memory_lifecycle.md). The agent loop reads `memory_data` from `collect_agent_connections()` (`server/services/plugin/edge_walker.py`, via the `input-memory` edge), prepends parsed history to the system message + current prompt, runs `_run_agent_loop`, then appends the turn + trims the window + archives trimmed text to the vector store. The markdown helpers (`parse_memory_markdown`, `append_to_memory_markdown`, `trim_markdown_window`) and the vector store (`InMemoryVectorStore` with HuggingFace embeddings) are the load-bearing surface — see the canonical doc for signatures, the markdown format, and the engine-specific adapter table (aiAgent / rlm_agent / claude_code_agent native session resume bridge).
+Memory load + save lives in
+[memory_lifecycle.md](./memory_lifecycle.md). The agent loop reads
+`memory_data` from `collect_agent_connections()`
+(`server/services/plugin/edge_walker.py`, via the `input-memory` edge),
+prepends history parsed into native `Message` values to the system message and
+current prompt, runs `run_native_agent_loop`, then atomically appends the turn,
+trims the window, and archives trimmed text to the optional vector store. The
+markdown helpers (`parse_memory_markdown`, `append_to_memory_markdown`,
+`trim_markdown_window`) and `NativeMemoryVectorStore` with the optional direct
+sentence-transformers embedder are the load-bearing surface — see the
+canonical doc for signatures, the markdown format, and the engine-specific
+table.
---
@@ -384,9 +445,9 @@ Both methods live in `server/services/ai.py` and follow the same general pattern
| Aspect | `execute_agent()` | `execute_chat_agent()` |
|--------|-------------------|----------------------|
-| **Loop call** | Always calls `_run_agent_loop` (even with zero tools — single-turn happy path returns immediately) | Conditional: calls `_run_agent_loop` if tools, simple `await chat_model.ainvoke(messages)` if no tools |
-| **Tool failure** | Re-raises exceptions (the loop wraps them as `{"error": ...}` ToolMessages on retry) | Returns `{"error": str(e)}` (softer handling) |
-| **No-tool path** | N/A -- always uses the loop | `response = await chat_model.ainvoke(messages)` (skips loop overhead) |
+| **Loop call** | Calls `run_native_agent_loop` with the resolved recursion limit | Calls `run_native_agent_loop` with the resolved recursion limit when tools exist and with `max_iterations=1` otherwise |
+| **Tool failure** | Callback re-raises; the shared loop serializes the exception into a native tool-result message | Callback returns `{"error": str(e)}`; the shared loop serializes that result into the same native tool-result shape |
+| **No-tool path** | The first native turn normally returns immediately | One native loop iteration through `ChatUnifier` |
| **Result metadata** | `agent_type: "agent"` | `agent_type: "chat" / "chat_with_skills" / "chat_with_tools" / "chat_with_skills_and_tools"` |
### Specialized Agent Routing
@@ -441,10 +502,16 @@ Two settings flags route agent execution through different Temporal paths (see [
| Flag | Off | On (default) |
|---|---|---|
| `TEMPORAL_PER_TYPE_DISPATCH` | Every node routes through the legacy `execute_node_activity` single dispatcher (WS round-trip to the FastAPI handler). | Each node routes through its per-type activity `node.{type}.v{version}` registered via `BaseNode.as_activity()`. Per-plugin retry / timeout / heartbeat configs apply. |
-| `TEMPORAL_AGENT_WORKFLOW_ENABLED` | All specialized + team leads + base agents (`aiAgent` / `chatAgent`) run inside `execute_node_activity` (`_run_agent_loop` in-activity). | The migrating agent types become Temporal **child workflows** (`AgentWorkflow`). LLM steps + tool calls become activities; `agent.prepare_payload.v1` resolves the DB-backed payload as the workflow's first step; `agent.refresh_tools.v1` rebinds the tool surface after canvas mutations. `rlm_agent` / `claude_code_agent` stay on the F4.A per-type activity path (externalised session state). |
+| `TEMPORAL_AGENT_WORKFLOW_ENABLED` | All specialized + team leads + base agents (`aiAgent` / `chatAgent`) run inside `execute_node_activity` using the in-process native loop. | The migrating agent types become Temporal **child workflows** (`AgentWorkflow`). LLM steps + tool calls become activities; `agent.prepare_payload.v1` resolves the DB-backed payload as the workflow's first step; `agent.refresh_tools.v1` refreshes the native tool surface after canvas mutations. `rlm_agent` / `claude_code_agent` stay on the F4.A per-type activity path (externalised session state). |
Both flags default to `true` in `.env.template`.
+New `AgentWorkflow` executions record `llm_engine="native"` and
+`message_wire_version=2` in the `agent.prepare_payload.v1` result. Only
+recorded histories whose prepare result predates those markers replay the
+frozen LangChain / `StructuredTool` branch; marker-bearing native executions
+never fall back after a provider request starts.
+
**Team leads** (`orchestrator_agent`, `ai_employee`) run the same
`execute_chat_agent` path as the other specialized agents but add an
`input-teammates` handle. Connected agents become `delegate_to_<type>` tools
@@ -473,7 +540,7 @@ Exposed helpers inside the REPL:
Routing: `rlm_agent` is a plugin folder like every other agent. Its
`execute_op` (`server/nodes/agent/rlm_agent/__init__.py`) delegates to
-`ai_service.rlm_service.execute(...)` rather than `_run_agent_loop`. See
+`ai_service.rlm_service.execute(...)` rather than `run_native_agent_loop`. See
[rlm_service.md](rlm_service.md) for full details.
### Chat Agent Conditional Loop
@@ -481,26 +548,43 @@ Routing: `rlm_agent` is a plugin folder like every other agent. Its
```python
# In execute_chat_agent():
if all_tools:
- # Run the agent loop with bound tools
- final_state = await _run_agent_loop(
- chat_model, messages,
+ final_state = await run_native_agent_loop(
+ self.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ initial_messages=messages,
tools=all_tools,
tool_executor=chat_tool_executor,
max_iterations=recursion_limit,
- progress_callback=_emit_progress if broadcaster else None,
)
else:
- # Simple invoke -- no loop overhead
- response = await chat_model.ainvoke(messages)
+ final_state = await run_native_agent_loop(
+ self.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ initial_messages=messages,
+ max_iterations=1,
+ )
```
-This optimisation means tool-less Chat Agent conversations skip `_run_agent_loop` entirely for faster response times.
+The no-tool path uses the same native step contract but is explicitly bounded
+to one iteration.
---
## AI Agent vs Chat Agent (Zeenie)
-`aiAgent` and `chatAgent` (display name "Zeenie") are near-identical in capability — both run the same `_run_agent_loop`, support memory / skills / tools / task input, and can delegate asynchronously. The differences are which backend method they call and the softness of error handling.
+`aiAgent` and `chatAgent` (display name "Zeenie") are near-identical in
+capability — both run `run_native_agent_loop` through `ChatUnifier`, support
+memory / skills / tools / task input, and can delegate asynchronously. The
+differences are which backend method they call and the softness of callback
+error handling.
| Feature | AI Agent (`aiAgent`) | Zeenie (`chatAgent`) |
|---------|----------------------|----------------------|
@@ -511,8 +595,8 @@ This optimisation means tool-less Chat Agent conversations skip `_run_agent_loop
| Bottom Handles | Skill, Tools | Skill, Tools |
| Left Handles | Input, Memory, Task | Input, Memory, Task |
| Backend Method | `execute_agent()` | `execute_chat_agent()` |
-| No-tool path | Always uses `_run_agent_loop` | Single `await chat_model.ainvoke(messages)` (skips loop overhead) |
-| Tool-failure handling | Re-raises (loop wraps as `{"error": ...}` ToolMessages) | Returns `{"error": str(e)}` (softer) |
+| No-tool path | Native loop returns after the first final response | Native loop is bounded to one iteration |
+| Tool-failure handling | Callback re-raises; loop creates an error tool-result message | Callback returns `{"error": str(e)}`; loop creates the tool-result message |
| Async Delegation | Yes (fire-and-forget) | Yes (fire-and-forget) |
See the full method comparison in [execute_agent vs execute_chat_agent](#execute_agent-vs-execute_chat_agent) above.
@@ -538,7 +622,9 @@ Both agents resolve their prompt the same way (logic in `server/nodes/agent/_inl
Chat Trigger → Zeenie ← HTTP Skill (SKILL.md context)
← HTTP Request (tool node)
```
-Zeenie loads SKILL.md instructions from connected skill nodes, builds tools from connected tool nodes (httpRequest, calculatorTool, …), and uses `_run_agent_loop` for tool execution when tools are connected.
+Zeenie loads SKILL.md instructions from connected skill nodes, builds
+`AgentToolSpec` values from connected tool nodes (httpRequest,
+calculatorTool, …), and uses `run_native_agent_loop` for LLM execution.
---
diff --git a/docs-internal/agent_delegation.md b/docs-internal/agent_delegation.md
index 1672c1b0..3d35abf4 100644
--- a/docs-internal/agent_delegation.md
+++ b/docs-internal/agent_delegation.md
@@ -8,7 +8,12 @@
How memory, parameters, and execution context flow when one AI agent delegates work to another agent connected as a tool.
-> **Related docs:** [tool_building_pipeline.md](./tool_building_pipeline.md) for the underlying `_build_tool_from_node` + `execute_tool` pipeline that backs `delegate_to_*` (delegation tools share the same StructuredTool construction path as other AI tools — what makes them special is the fire-and-forget background task and the `{"status": "delegated"}` lifecycle opt-out documented there).
+> **Related docs:** [tool_building_pipeline.md](./tool_building_pipeline.md)
+> for the underlying `_build_tool_from_node` + `execute_tool` pipeline that
+> backs `delegate_to_*` (delegation tools use the same provider-neutral
+> `AgentToolSpec` / `ToolDef` construction path as other AI tools — what makes
+> them special is the fire-and-forget background task and the
+> `{"status": "delegated"}` lifecycle opt-out documented there).
## Overview
@@ -520,7 +525,7 @@ Every delegated agent broadcasts status at key phases via WebSocket:
| `delegated_task` | Child starts executing | Child's node ID |
| `initializing` | Creating LLM model | Child's node ID |
| `loading_memory` | Parsing markdown history | Child's node ID |
-| `building_tools` | Converting tool nodes to StructuredTool | Child's node ID |
+| `building_tools` | Converting tool nodes to `AgentToolSpec` values | Child's node ID |
| `invoking_llm` | Calling model | Child's node ID |
| `executing_tool` | Running a tool called by child's LLM | Tool node ID |
| `delegated_complete` | Child finishes successfully | Child's node ID |
diff --git a/docs-internal/litellm_router_migration_rfc.md b/docs-internal/litellm_router_migration_rfc.md
index 185a3025..25f964f1 100644
--- a/docs-internal/litellm_router_migration_rfc.md
+++ b/docs-internal/litellm_router_migration_rfc.md
@@ -1,6 +1,10 @@
# LiteLLM Router Migration RFC
-**Status:** Draft · **Owner:** trohitg · **Created:** 2026-07-02 · **Target:** `server/services/llm/` + the chat/agent LLM paths in `server/services/ai.py`
+**Status:** Superseded by [Native LLM SDK](native_llm_sdk.md) · **Owner:** trohitg · **Created:** 2026-07-02 · **Target:** `server/services/llm/` + the chat/agent LLM paths in `server/services/ai.py`
+
+> Historical RFC only. The adopted implementation uses provider-native SDKs,
+> not LiteLLM; dependency and migration claims below are retained solely as
+> design-history context.
---
diff --git a/docs-internal/memory_compaction.md b/docs-internal/memory_compaction.md
index 3836951f..60ccb992 100644
--- a/docs-internal/memory_compaction.md
+++ b/docs-internal/memory_compaction.md
@@ -128,7 +128,7 @@ class TokenUsageMetric(SQLModel, table=True):
provider: str # openai, anthropic, gemini, groq
model: str # Model identifier
- # Core token counts (LangChain UsageMetadata compatible)
+ # Core token counts (native Usage contract)
input_tokens: int = 0
output_tokens: int = 0
total_tokens: int = 0
@@ -377,31 +377,25 @@ ws.send(JSON.stringify({
## Integration with AI Service
-### Token Extraction from LangChain
+### Token Extraction from Native Agent Responses
-LangChain normalizes token usage across all providers via `UsageMetadata`:
+Every provider normalizes token accounting into
+`services.llm.protocol.Usage`. The shared native agent loop adds usage from
+every iteration, including cache and reasoning tokens, and returns the
+aggregate in `final_state["usage"]`:
```python
-from langchain_core.messages.ai import UsageMetadata, add_usage
-
-# After AI execution
-response = await model.ainvoke(messages)
-usage = response.usage_metadata
-
-# Example usage_metadata:
-# {
-# 'input_tokens': 8,
-# 'output_tokens': 304,
-# 'total_tokens': 312,
-# 'input_token_details': {'cache_read': 0, 'cache_creation': 0},
-# 'output_token_details': {'reasoning': 256}
-# }
-
-# Aggregate across agent-loop iterations
-total_usage = None
-for msg in final_state["messages"]:
- if hasattr(msg, 'usage_metadata') and msg.usage_metadata:
- total_usage = add_usage(total_usage, msg.usage_metadata)
+final_state = await run_native_agent_loop(...)
+usage = final_state["usage"]
+
+# services.llm.protocol.Usage(
+# input_tokens=8,
+# output_tokens=304,
+# total_tokens=312,
+# cache_creation_tokens=0,
+# cache_read_tokens=0,
+# reasoning_tokens=256,
+# )
```
### Integration Point
@@ -409,31 +403,19 @@ for msg in final_state["messages"]:
In `server/services/ai.py`, after agent execution:
```python
-# After agent-loop execution, before memory save
+# After native agent-loop execution, before memory save
if memory_data and memory_data.get('session_id'):
- from services.compaction import get_compaction_service
- svc = get_compaction_service()
-
- if svc and ai_response and ai_response.usage_metadata:
- usage = ai_response.usage_metadata
- tracking = await svc.track(
- session_id=memory_data['session_id'],
- node_id=node_id,
- provider=provider,
- model=model,
- usage={
- "input_tokens": usage.get('input_tokens', 0),
- "output_tokens": usage.get('output_tokens', 0),
- "total_tokens": usage.get('total_tokens', 0),
- "cache_creation_tokens": usage.get('input_token_details', {}).get('cache_creation', 0),
- "cache_read_tokens": usage.get('input_token_details', {}).get('cache_read', 0),
- "reasoning_tokens": usage.get('output_token_details', {}).get('reasoning', 0),
- }
- )
-
- if tracking.get('needs_compaction'):
- # Trigger native compaction or client-side summarization
- ...
+ compaction_result = await self._track_token_usage(
+ session_id=memory_data['session_id'],
+ node_id=node_id,
+ provider=provider,
+ model=model,
+ ai_response=final_state["usage"],
+ all_messages=final_state["messages"],
+ memory_content=memory_data.get("memory_content", ""),
+ api_key=api_key,
+ memory_node_id=memory_data.get("node_id"),
+ )
```
## File Reference
diff --git a/docs-internal/memory_lifecycle.md b/docs-internal/memory_lifecycle.md
index 294dceba..d84aa0da 100644
--- a/docs-internal/memory_lifecycle.md
+++ b/docs-internal/memory_lifecycle.md
@@ -15,7 +15,11 @@ Canonical home for how conversation memory is loaded, appended, trimmed, archive
| **Markdown transcript** | [`services/memory/markdown.py`](../server/services/memory/markdown.py) | aiAgent, chatAgent, rlm_agent, claude_code_agent (mirror) | `### **Human/Assistant** (timestamp)` blocks under a top-level `# Conversation History` heading |
| **Anthropic-Messages JSONL** | [`services/memory/jsonl.py`](../server/services/memory/jsonl.py) | Standalone primitive — not used by any agent today | One JSON object per line: `{"role": "user"|"assistant", "content": str \| [ContentBlock], ...metadata}` |
-The markdown format is the visible UI surface — `simpleMemory.memory_content` is rendered as a markdown editor in the parameter panel. The JSONL helpers were prepared for a future SDK migration; the live `claude_code_agent` bridge uses claude's own native session JSONL on disk (see §6) and mirrors the transcript into `memory_content` via the **markdown** helpers for user visibility.
+The markdown format is the visible UI surface — `simpleMemory.memory_content`
+is rendered as a markdown editor in the parameter panel. The normalized JSONL
+helpers remain a standalone primitive; the live `claude_code_agent` bridge
+uses claude's own native session JSONL on disk (see §6) and mirrors the
+transcript into `memory_content` via the **markdown** helpers for visibility.
## 2. Markdown helper API
@@ -23,13 +27,13 @@ Three pure functions, no I/O, locked by `tests/services/memory/`:
```python
from services.memory import (
- parse_memory_markdown, # str -> List[BaseMessage]
+ parse_memory_markdown, # str -> List[services.llm.protocol.Message]
append_to_memory_markdown, # (content, role: "human"|"ai", message) -> str
trim_markdown_window, # (content, window_size_pairs) -> (trimmed, removed_texts)
)
```
-- **`parse_memory_markdown`** — regex-extracts `### **(Human|Assistant)**[^\n]*\n(body)` blocks; returns LangChain `HumanMessage` / `AIMessage` instances ready to feed an agent loop.
+- **`parse_memory_markdown`** — regex-extracts `### **(Human|Assistant)**[^\n]*\n(body)` blocks; returns normalized native `Message(role="user"|"assistant", content=...)` instances ready for `ChatUnifier`.
- **`append_to_memory_markdown`** — drops the empty-state placeholder (`*No messages yet.*`), appends a `### **{label}** (YYYY-MM-DD HH:MM:SS)\n{message}\n` entry. Timestamp comes from `datetime.now()`.
- **`trim_markdown_window`** — keeps the last `window_size * 2` blocks (N user/assistant pairs); returns `(trimmed_content, removed_texts)`. Removed bodies are returned so the caller can hand them to the long-term vector store (§4).
@@ -43,37 +47,51 @@ from services.memory import (
This is also the value `clear_agent_session_state` resets `memory_content` to (see §5). Defined as `DEFAULT_MEMORY_CONTENT` in [`services/memory/state.py`](../server/services/memory/state.py).
-## 3. Known duplication: ai.py inlines its own copies
+## 3. Canonical implementation and call sites
-[`services/ai.py:158-234`](../server/services/ai.py) carries `_parse_memory_markdown`, `_append_to_memory_markdown`, `_trim_markdown_window`, and `_get_memory_vector_store` as private helpers — byte-equivalent reimplementations of the public ones in `services/memory/`. The `services.memory` package was carved out later; existing call sites inside `ai.py` were never switched to import from it. Future cleanup will consolidate; for now both paths exist and produce identical output. Pytest doesn't lock cross-equivalence — treat the `services.memory` package as the canonical surface for **new** code.
+`services.memory` is the single implementation. `services.ai` keeps only a
+thin `_parse_memory_markdown` compatibility wrapper and aliases the canonical
+vector-store cache so older internal imports continue to resolve without
+creating duplicate state. Runtime appends use
+`services.memory.runtime.append_memory_turns_atomic`, which performs the
+append/trim mutation transactionally and idempotently.
-Newer call sites (already on the public helpers):
-
-| File | Imports |
+| File | Canonical surface |
|---|---|
-| `services/temporal/agent_activities.py:251` — `agent.persist_turn.v1` activity (F4.B) | `append_to_memory_markdown`, `trim_markdown_window` |
-| `services/cli_agent/service.py:490` — `_persist_memory` for claude_code_agent | `append_to_memory_markdown`, `trim_markdown_window` |
-| `routers/websocket.py:2167` — `clear_memory` handler | `clear_agent_session_state` |
-
-Older call sites (still on the underscore-prefixed copies inside ai.py):
-
-| File | Symbols |
-|---|---|
-| `services/ai.py:execute_agent` / `execute_chat_agent` | `_parse_memory_markdown`, `_append_to_memory_markdown`, `_trim_markdown_window`, `_memory_vector_stores` |
+| `services/ai.py` — native in-process agents | `parse_memory_markdown`, `append_memory_turns_atomic`, canonical vector store |
+| `services/temporal/agent_activities.py` — `agent.persist_turn.v1` | `append_memory_turns_atomic` |
+| `services/cli_agent/service.py` — claude_code_agent mirror | markdown/runtime helpers and canonical vector store |
+| `services/memory/state.py` — reset orchestration | canonical vector-store cache + atomic parameter mutation |
## 4. Long-term vector store
-[`services/memory/vector_store.py`](../server/services/memory/vector_store.py) caches one `InMemoryVectorStore` per `session_id`. Embeddings come from `langchain_huggingface.HuggingFaceEmbeddings(model_name="BAAI/bge-small-en-v1.5")` — lazy-imported; missing dep returns `None` with a warning (long-term archival is best-effort).
+[`services/memory/vector_store.py`](../server/services/memory/vector_store.py)
+provides a shared async embedder factory for direct `openai.AsyncOpenAI`,
+`ollama.AsyncClient`, and optional `sentence-transformers` adapters, then
+ranks entries locally by cosine similarity. Hugging Face model load and
+encoding run in a worker thread. When its `local-embeddings` extra is absent,
+the factory returns `None` with a warning, so long-term archival remains
+best-effort.
```python
from services.memory import get_memory_vector_store
-store = get_memory_vector_store("user-123")
+store = await get_memory_vector_store(
+ "user-123",
+ provider="openai",
+ model="text-embedding-3-small",
+ auth_service=auth_service,
+)
if store:
- store.add_texts(removed_texts) # archive trimmed messages
- results = store.similarity_search(q) # retrieve for the next turn
+ await store.add_texts(removed_texts) # archive trimmed messages
+ results = await store.similarity_search(q) # retrieve next-turn context
```
-**Two caches exist.** The live one used by `services/ai.py:execute_agent` is `services.ai._memory_vector_stores`. The one in `services.memory.vector_store` is a package-private second copy referenced by `clear_agent_session_state` for forward-cleanup. Same caveat as §3: consolidation is future work. The state-clear orchestration deletes from the live `services.ai` cache.
+There is one cache. `services.ai._memory_vector_stores` is a compatibility
+alias to `services.memory.vector_store._memory_vector_stores`, so state clear,
+native agents, and the CLI bridge observe the same stores. Cache identity is
+session + provider + model + endpoint + credential fingerprint; plaintext
+credentials are never stored in a key. Clearing a session removes and closes
+every matching configuration.
## 5. State clear orchestration
@@ -90,7 +108,7 @@ await clear_agent_session_state(
Three stores are cleared in one call:
-1. **Vector store** (when `clear_long_term=True`) — `del services.ai._memory_vector_stores[session_id]`.
+1. **Vector store** (when `clear_long_term=True`) — delete and close every provider/model cache entry for the session (the `services.ai` alias observes the same deletion).
2. **TodoService** — every candidate key (`workflow_id`, `session_id`, `"default"`) is cleared because [`server/nodes/tool/write_todos/__init__.py`](../server/nodes/tool/write_todos/__init__.py) uses `ctx.workflow_id or ctx.node_id or "default"` as the storage key and we want to clear whichever fallback the write path actually used.
3. **simpleMemory node fields** (when `memory_node_id` provided) — `memory_content` → `DEFAULT_MEMORY_CONTENT`; `last_session_id` → `None` (so claude_code_agent starts fresh next run instead of `--resume`-ing into a wiped transcript); orphan `memory_jsonl` field popped if present.
@@ -140,41 +158,42 @@ Turn N starts
1. handle_ai_agent / handle_chat_agent reads simpleMemory.memory_content from DB
│
▼
-2. _parse_memory_markdown(content) → List[BaseMessage]
+2. _parse_memory_markdown(content) → List[native Message]
│
▼
-3. (Optional) get_memory_vector_store(session_id).similarity_search(prompt)
+3. (Optional) await get_memory_vector_store(config);
+ await store.similarity_search(prompt)
→ prepend retrieved long-term context as a system message
│
▼
-4. Agent loop runs (_run_agent_loop: ainvoke + tool calls)
+4. Agent loop runs (run_native_agent_loop: ChatUnifier + native tool calls)
│
▼
5. _track_token_usage(...) — see memory_compaction.md
→ if threshold tripped: CompactionService.compact_context(...) → reset messages
│
▼
-6. _append_to_memory_markdown(content, "human", prompt)
- _append_to_memory_markdown(content, "ai", response)
- │
- ▼
-7. trimmed_content, removed_texts = _trim_markdown_window(content, window_size)
- │
- ▼
-8. (Optional) store.add_texts(removed_texts) — long-term archival
+6. append_memory_turns_atomic(prompt, response, window_size)
+ → append markdown pair + trim in one idempotent DB transaction
│
▼
-9. save_node_parameters(memory_node_id, {"memory_content": trimmed_content, ...})
- broadcast_node_parameters_updated(...) # CloudEvents source_hint="agent"
+7. (Optional) await store.add_texts(removed_texts) — long-term archival
```
-F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities (`prepare_payload.v1` resolves DB params + `{{templates}}` via `workflow_service._param_resolver.resolve`; `execute_llm_step.v1` invokes the LLM with LangChain `messages_to_dict` / `messages_from_dict` round-trip preserving Gemini `thought_signature` + Anthropic cache markers + OpenAI reasoning content; `persist_turn.v1` appends per turn; `compact_memory.v1` summarises when threshold trips; `store_output.v1` writes to the OutputStore via `workflow_service.store_node_output` so downstream nodes can resolve `{{aiAgent.response}}`; `broadcast_progress.v1` emits CloudEvents `agent_progress` per phase). Per-turn persistence means a mid-loop failure doesn't lose progress. F4.A and legacy paths run steps 4-9 inline inside one activity.
+F4.B `AgentWorkflow` runs steps 4-9 as separate Temporal activities.
+`prepare_payload.v1` records `llm_engine` and the message wire version;
+new executions use native Message Wire V2, preserving Gemini thought
+signatures, Anthropic signed/redacted thinking, and OpenAI continuation
+metadata. Histories recorded before cutover lack that engine field and
+deterministically use the temporary LangChain compatibility branch.
+`persist_turn.v1` appends per turn, so a mid-loop failure does not lose
+progress.
## 8. Engine-specific adapters
| Engine | Memory shape on input | Memory shape on output | Notes |
|---|---|---|---|
-| `aiAgent` / `chatAgent` | LangChain `BaseMessage` list parsed from markdown | Appended pair + trim | Standard path; see §7 |
+| `aiAgent` / `chatAgent` | Native `Message` list parsed from markdown | Atomic appended pair + trim | Native SDK path; see §7 |
| `rlm_agent` | Markdown injected as REPL pre-context | Last `FINAL(...)` call's payload | RLM internal recursion uses its own state machine; only entry/exit cross the memory boundary |
| `claude_code_agent` | `--resume <UUID>` reads native JSONL; `memory_content` ignored on input | Bridge writes markdown mirror + saves `last_session_id` | See §6 |
@@ -190,8 +209,9 @@ The compaction service is the SSOT for thresholds, native-API integration (Anthr
server/tests/services/memory/
├── test_markdown.py # parse → append → trim round-trips
├── test_jsonl.py # parse_jsonl + append_message + trim_window
-└── (state.py / vector_store.py have no dedicated test files;
- coverage rides on test_ai_agents.py + test_status_broadcasts.py)
+├── test_state.py # cross-store reset behavior
+├── test_memory_store.py # legacy/canonical role normalization
+└── test_vector_store.py # native cosine ranking and empty operations
```
The CloudEvents `node_parameters_updated` envelope emitted on every memory write is locked by `tests/test_status_broadcasts.py:test_node_parameters_updated_envelope_shape` — see [status_broadcaster.md](./status_broadcaster.md) for the contract.
diff --git a/docs-internal/native_llm_sdk.md b/docs-internal/native_llm_sdk.md
index 318ba626..54523a73 100644
--- a/docs-internal/native_llm_sdk.md
+++ b/docs-internal/native_llm_sdk.md
@@ -2,7 +2,10 @@
> **Authoring model (post-Wave-11):** each chat-model node is a self-contained Python plugin folder under `server/nodes/model/<provider>_chat_model/` that emits a `NodeSpec`. The frontend reads specs via [client/src/lib/nodeSpec.ts](../client/src/lib/nodeSpec.ts) + [adapters/nodeSpecToDescription.ts](../client/src/adapters/nodeSpecToDescription.ts) and renders through `SquareNode` with zero TS edits. See [plugin_system.md](./plugin_system.md) and [server/nodes/README.md](../server/nodes/README.md) for the plugin model, and "Adding a New Provider" below for the chat-model recipe.
-OpenCompany uses a hybrid LLM architecture: a native SDK layer in `server/services/llm/` for direct chat completions and model fetching, and a LangChain-backed agent loop in `server/services/ai.py` (the plain `_run_agent_loop` async function) for agent tool-calling. This document describes the native layer, its design, and how both paths coexist.
+OpenCompany routes chat-model nodes and all new agent executions through the
+native SDK layer in `server/services/llm/`. The shared buffered tool loop lives
+in `server/services/agent_runtime.py`. A narrowly isolated LangChain branch is
+retained only while Temporal histories recorded before the cutover drain.
## Why a Native Layer
@@ -25,6 +28,7 @@ server/services/llm/
| sdk_exception_refs are lazy "module:Class" strings resolved at except time
|-- unifier.py ChatUnifier — the dispatch facade execute_chat delegates every provider to;
| translates typed SDK errors -> NodeUserError
+|-- schema.py Provider-aware tool JSON-schema compilation
|-- vertex.py Vertex / Agent-Platform key handling
|-- messages.py filter_empty_messages, is_valid_message_content
`-- providers/
@@ -65,8 +69,10 @@ Source of truth for this list: `server/config/llm_defaults.json` (the `providers
### Native chat path vs agent dropdown — two different counts
- **Native chat path (`execute_chat` / `fetch_models`) supports 12 providers, all native** — Anthropic and Gemini via their own SDKs; OpenAI and OpenRouter via the openai SDK; and 8 more through the shared OpenAI-compatible client with a per-provider `base_url` (xai, deepseek, kimi, mistral, groq, cerebras, ollama, lmstudio — registered in `providers/_compat.py`). `execute_chat` delegates every provider to `ChatUnifier.chat`; there is no per-provider branch and no chat-path LangChain fallback. `xai` lives here.
-- **The agent dropdown exposes 11 providers** for `aiAgent`, `chatAgent` (Zeenie), and all specialized agents — the `provider` Literal in [`nodes/agent/ai_agent/__init__.py`](../server/nodes/agent/ai_agent/__init__.py), [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py), and [`_specialized.py`](../server/nodes/agent/_specialized.py): `openai`, `anthropic`, `gemini`, `openrouter`, `groq`, `cerebras`, `deepseek`, `kimi`, `mistral`, `ollama`, `lmstudio`. **`xai` is native-chat-only and is NOT in the agent Literal** (no agent dropdown entry).
-- Groq and Cerebras are available as standalone chat-model nodes AND in the agent dropdown. Chat runs on the native OpenAI-compatible client; only the agent path uses LangChain (`ChatGroq` / `ChatCerebras`) — the same chat-native / agent-LangChain split every provider has.
+- **The agent dropdown exposes the same 12 providers** for `aiAgent`,
+ `chatAgent` (Zeenie), and all specialized agents, including `xai`.
+- Groq, Cerebras, xAI, and the other compatible endpoints use the same native
+ OpenAI SDK adapter for both standalone chat and agent tool calls.
### Provider / model reference table
@@ -110,7 +116,11 @@ Ollama and LM Studio expose an OpenAI-compatible `/v1` HTTP API, so they ride th
**Both servers must have a model loaded** for the probe to return entries. The validator's "no models loaded" message is symmetric across providers.
-**Runtime path** — `execute_chat()` and `execute_agent()` read `{provider}_proxy` via `auth.get_api_key()` and pass it as `proxy_url` into the provider build (`ChatUnifier` -> `spec.factory(...)`). `OpenAIProvider.__init__` overrides `kwargs["base_url"]` with the user's URL and forces `api_key="ollama"` (the documented placeholder for unauthenticated local servers). The openai SDK then sends `POST {user_url}/chat/completions` — **traffic stays on the user's machine, never reaches api.openai.com**.
+**Runtime path** — `ChatUnifier` resolves `{provider}_proxy` and passes it to
+the registered provider factory. `OpenAIProvider` overrides `base_url` with the
+user's URL and uses the documented placeholder key for unauthenticated local
+servers. The OpenAI SDK then sends to the configured local endpoint — traffic
+never reaches api.openai.com.
**Provider detection** ([`server/constants.py:detect_ai_provider`](../server/constants.py)) MUST list `ollama` / `lmstudio` substrings, and the agent dropdown's `provider` Literal in [`ai_agent.py`](../server/nodes/agent/ai_agent/__init__.py) / [`chat_agent.py`](../server/nodes/agent/chat_agent/__init__.py) / [`_specialized.py`](../server/nodes/agent/_specialized.py) MUST include `"ollama"` / `"lmstudio"` — otherwise the chat-model node silently falls through to `'openai'` and the runtime calls the OpenAI cloud with the local-server placeholder key.
@@ -151,9 +161,12 @@ class LLMResponse:
model: str = ""
finish_reason: str = "stop"
raw: Any = None
+ assistant_message: Optional[Message] = None # canonical replay envelope
```
-Downstream code never special-cases a provider SDK shape.
+`assistant_message` is the durable source of truth. The flat `content`,
+`thinking`, and `tool_calls` fields remain convenience views; `raw` is never
+written to memory or Temporal history.
## Registry, Unifier, and Lazy Imports
@@ -215,32 +228,25 @@ Adding a new OpenAI-compatible provider is a pure config change:
No new provider class, no new import. `providers/_compat.py` registers a spec whose factory reuses `OpenAIProvider` with the config's `base_url`.
-## Native Path vs LangChain Path
-
-`AIService` in `server/services/ai.py` has two top-level execution methods. They take different routes:
-
-```
-execute_chat(parameters, node_id, node_type) -> native SDK where possible
-execute_agent(parameters, node_id) -> LangChain chat model + _run_agent_loop
-execute_chat_agent(parameters, node_id) -> LangChain chat model + _run_agent_loop
-```
-
-**`execute_chat()` routing (no per-provider branch, no LangChain fallback):**
+## Unified Native Execution Path
```
-node_type / model selects provider
+execute_chat / execute_agent / execute_chat_agent
|
v
-ChatUnifier.chat(provider=...)
+ChatUnifier.chat(provider=..., Message[], ToolDef[])
|
- +--> registry.get_provider(name) -> spec.factory(...) -> provider.chat(...) -> LLMResponse
+ +--> registry.get_provider(name)
+ +--> official SDK or OpenAI-compatible base URL
+ +--> LLMResponse(assistant_message=lossless MessageWireV2)
```
-**`execute_agent()` and `execute_chat_agent()` routing:**
-
-All providers go through `create_model()` which returns a LangChain `ChatOpenAI` / `ChatAnthropic` / `ChatGoogleGenerativeAI` / `ChatGroq` / `ChatCerebras` instance. The agent then calls `chat_model.bind_tools(tools)` and runs `_run_agent_loop` (plain async while loop, ~140 LOC). LangChain's chat-model classes are used because they unify `bind_tools` + `usage_metadata` + `content_blocks` extraction across all providers; the native-SDK layer doesn't yet have provider-unified tool binding.
-
-OpenAI-compatible providers like DeepSeek/Kimi/Mistral still use `ChatOpenAI` with `base_url` from `llm_defaults.json` in the agent path, and pass `max_tokens` via `extra_body` to bypass LangChain's `max_completion_tokens` conversion.
+`run_native_agent_loop` appends the provider's exact assistant envelope before
+executing tools. This preserves Gemini thought signatures, Anthropic signed and
+redacted thinking blocks, and OpenAI reasoning continuation state across
+in-process and Temporal turns. Temporal executions created before the migration
+remain pinned to the legacy engine by their recorded prepare-activity result;
+new executions record `llm_engine=native` and wire version 2.
## Thinking and Reasoning
@@ -252,7 +258,7 @@ class ThinkingConfig:
enabled: bool = False
budget: int = 2048 # Anthropic budget_tokens, Gemini 2.5 thinking_budget
effort: str = "medium" # OpenAI reasoning_effort (low/medium/high)
- level: str = "medium" # Gemini 3+ thinking_level
+ level: Optional[str] = None # Gemini 3+ thinking_level, only when explicit
format: str = "parsed" # Groq Qwen3 reasoning_format (parsed/hidden)
```
@@ -271,9 +277,11 @@ Each provider's `chat()` method reads only the fields it supports. The extracted
The thinking/reasoning fields (`thinkingEnabled`, `thinkingBudget`, `reasoningEffort`, `reasoningFormat`) live in the backend NodeSpec for each chat model (`server/nodes/model/<provider>_chat_model/`) and are surfaced through `AIChatModelParams` in `server/models/nodes.py`. The frontend renders them automatically via the universal parameter panel.
-### Thinking extraction (LangChain path)
+### Thinking extraction (legacy replay path)
-On the LangChain path, `extract_thinking_from_response(response, provider)` in `server/services/ai.py` pulls reasoning text out of provider-specific response shapes:
+Native providers normalize reasoning directly into `LLMResponse.thinking` and
+ordered `Message.blocks`. The helper below remains only for pre-cutover
+Temporal histories executing the legacy branch:
```python
def extract_thinking_from_response(response, provider: str) -> Optional[str]:
@@ -349,18 +357,9 @@ AI providers support optional proxy-based authentication — requests route thro
**Configuration:** proxy URLs are stored in the credentials DB under the `{provider}_proxy` pattern (e.g., `anthropic_proxy`, `openai_proxy`). Falls back to direct API key if no proxy configured. This is the SAME mechanism the native Ollama / LM Studio path uses (see "Local LLM Providers" above) — the validator persists the user's server URL under `{provider}_proxy`, and at runtime it carries into `OpenAIProvider`'s `base_url`.
-**Native chat path:** `proxy_url` flows through `ChatUnifier` into `spec.factory(...)`. **LangChain agent path:** `create_model()` sets `base_url`:
-
-```python
-def create_model(self, provider: str, api_key: str, model: str,
- temperature: float, max_tokens: int,
- thinking: Optional[ThinkingConfig] = None,
- proxy_url: Optional[str] = None):
- # ...
- if proxy_url:
- kwargs['base_url'] = proxy_url
- kwargs[config.api_key_param] = "ollama" # Ollama-style placeholder token
-```
+For every new chat and agent run, `proxy_url` flows through `ChatUnifier` into
+the cached native provider factory. The old `create_model()` proxy path exists
+only for histories already pinned to `llm_engine=langchain`.
**Use cases:** Claude Code CLI proxy for Anthropic models; native Ollama / LM Studio support; custom auth proxies; dev/testing with mock servers.
@@ -422,7 +421,8 @@ the plugin folder or add a `visuals.json` entry (`"openrouterChatModel":
1. **OpenAI-compatible provider** (DeepSeek, Kimi, Mistral pattern):
- Add an entry to `llm_defaults.json` with `base_url`, `default_model`, `detection_patterns`, `max_output_tokens._default`, `context_length._default`, `temperature_range`.
- Add the provider name to the compat list in `services/llm/providers/_compat.py` — its import loop calls `register_provider(ProviderSpec(...))` for every listed name.
- - The LangChain agent path needs no Python branching — `create_model()` reuses `ChatOpenAI` with `base_url` from `llm_defaults.json`.
+ - Chat and agent calls both use the native OpenAI SDK with that configured
+ `base_url`; do not add a branch in `AIService`.
2. **Custom-SDK provider** (Anthropic, Gemini pattern):
- Create `services/llm/providers/<name>.py` implementing the `LLMProvider` protocol.
@@ -444,7 +444,8 @@ the plugin folder or add a `visuals.json` entry (`"openrouterChatModel":
| `server/services/llm/providers/<provider>.py` | Native SDK provider (Protocol-based) + `register_provider(ProviderSpec)` at module bottom |
| `server/services/llm/registry.py` | `ProviderSpec` / `register_provider` — the provider registration contract |
| `server/services/llm/unifier.py` | `ChatUnifier` — chat dispatch facade + typed-error translation |
-| `server/services/ai.py` | Routing: native chat path (via ChatUnifier) + LangChain agent path |
+| `server/services/agent_runtime.py` | Shared native agent step/loop and `AgentToolSpec` |
+| `server/services/ai.py` | Node orchestration; legacy factory retained only for recorded Temporal histories |
| `client/src/Dashboard.tsx` | Generic `COMPONENT_BY_KIND` dispatch — no per-provider entry needed |
## Related Docs
@@ -452,4 +453,21 @@ the plugin folder or add a `visuals.json` entry (`"openrouterChatModel":
- [DESIGN.md](DESIGN.md) - overall backend architecture
- [memory_compaction.md](memory_compaction.md) - token tracking and compaction using this layer
- [pricing_service.md](pricing_service.md) - cost calculation from `LLMResponse.usage`
-- [agent_architecture.md](agent_architecture.md) - how agents use LangChain path on top of this layer
+- [agent_architecture.md](agent_architecture.md) - agent execution architecture
+
+## Temporal migration and dependency lifecycle
+
+`agent.prepare_payload.v1` records both `llm_engine` and
+`message_wire_version`. New histories default to `native`/`2`; histories whose
+recorded result lacks those fields deterministically keep the legacy activity
+payload and LangChain replay path. `AGENT_LLM_ENGINE=langchain` is an emergency
+switch for newly prepared runs only.
+
+The remaining LangChain adapter pins in `server/pyproject.toml` are therefore a
+temporary deployment compatibility requirement, not a runtime dependency of
+new executions. Remove the legacy activity branch, compatibility factory and
+pins together only after Temporal visibility confirms that no retained or
+retryable history can still select the legacy engine. The source-boundary test
+keeps `langchain*` confined to the three explicitly allowlisted legacy bridge
+modules, rejects `langgraph*`/`langsmith` everywhere else, and rejects
+`deepagents` throughout production during that drain window.
diff --git a/docs-internal/node-logic-flows/ai_agents/aiAgent.md b/docs-internal/node-logic-flows/ai_agents/aiAgent.md
index 0d123764..ca8207be 100644
--- a/docs-internal/node-logic-flows/ai_agents/aiAgent.md
+++ b/docs-internal/node-logic-flows/ai_agents/aiAgent.md
@@ -13,10 +13,11 @@
`aiAgent` is the general-purpose tool-calling agent node. It reads a prompt
and system message, optionally merges conversation history from a connected
`simpleMemory`, loads instructions from connected skill nodes, binds tool nodes
-via `chat_model.bind_tools`, and runs the tool-calling loop until the LLM produces a
-final answer. All of the heavy lifting (LLM invocation, tool execution, memory
-persistence) lives in `AIService.execute_agent`; the handler's job is purely
-to gather the connected-node payloads via
+as provider-neutral `AgentToolSpec` values, and runs
+`run_native_agent_loop` until the LLM produces a final answer. All of the
+heavy lifting (native SDK invocation, tool execution, memory persistence)
+lives behind `AIService.execute_agent`; the handler's job is purely to gather
+the connected-node payloads via
`edge_walker.collect_agent_connections` (a 5-tuple: memory, skill, tool,
input, task) and forward them.
@@ -26,7 +27,7 @@ input, task) and forward them.
|--------|-----------------|----------|---------|
| `input-main` | main | no | Upstream data. Used as auto-prompt when `prompt` is empty. |
| `input-skill` | main | no | Skill nodes (`masterSkill` is expanded into individual skills). |
-| `input-memory` | main | no | A `simpleMemory` node whose `memoryContent` is parsed into LangChain messages. |
+| `input-memory` | main | no | A `simpleMemory` node whose `memory_content` markdown is parsed into native `Message` values. |
| `input-tools` | main | no | Tool nodes (search, calculator, HTTP, Android toolkit, child agents, etc.). |
| `input-task` | main | no | `taskTrigger` output - completed delegated-task payload. |
@@ -37,7 +38,7 @@ Source: `AIAgentParams` in [`ai_agent/__init__.py`](../../../server/nodes/agent/
| Name | Type | Default | Required | Group | Description |
|------|------|---------|----------|-------|-------------|
| `prompt` | string (textarea, rows 4) | `""` | no | - | User prompt; may reference upstream node outputs via templates. Empty -> auto-prompt fallback from `input-main`. |
-| `provider` | Literal | `openai` | no | - | One of openai, anthropic, gemini, openrouter, groq, cerebras, deepseek, kimi, mistral, ollama, lmstudio. |
+| `provider` | Literal | `openai` | no | - | One of openai, anthropic, gemini, openrouter, xai, groq, cerebras, deepseek, kimi, mistral, ollama, lmstudio. |
| `model` | string | `""` | no | - | Model id; loaded dynamically from the provider. |
| `system_message` | string (rows 3) | `You are a helpful assistant` | no | - | System prompt prepended to the conversation. |
| `temperature` | number (optional) | `None` | no | options | 0-2, step 0.1. Unset falls through to `agent.default_temperature` in `llm_defaults.json`. |
@@ -133,10 +134,12 @@ flowchart TD
and potentially `compaction_starting` / `compaction_completed` events.
`BaseNode.execute()` additionally wraps the body in a `node.aiAgent.execute`
OpenTelemetry span + `log_context(node_id, node_type, workflow_id)`.
-- **External API calls**: delegated to the native LLM provider SDKs (Anthropic,
- OpenAI, Gemini, OpenRouter, xAI, DeepSeek, Kimi, Mistral) or the LangChain
- fallback (Groq, Cerebras). Tool nodes may spawn their own HTTP or subprocess
- calls via `execute_tool`.
+- **External API calls**: all 12 providers run through `ChatUnifier` and the
+ native provider layer: Anthropic uses `anthropic`, Gemini uses
+ `google-genai`, and OpenAI plus the OpenAI-compatible providers (OpenRouter,
+ Groq, Cerebras, xAI, DeepSeek, Kimi, Mistral, Ollama, and LM Studio) use
+ `openai` with the configured endpoint. Tool nodes may spawn their own HTTP
+ or subprocess calls via `execute_tool`.
- **File I/O**: none in `execute_op`. Filesystem tool nodes may read/write the
per-workflow workspace (`context.workspace_dir`).
- **Subprocess**: none directly. Tool executors (shell, process manager,
@@ -147,10 +150,11 @@ flowchart TD
- **Credentials**: provider API keys via `auth_service.get_api_key(<provider>)`,
resolved inside `AIService`. Can be overridden by a provider proxy URL
(`<provider>_proxy` API key) for Ollama-style local routing.
-- **Services**: `AIService`, `Database`, `StatusBroadcaster`, `CompactionService`,
- `PricingService`, `SkillLoader` (for `masterSkill` expansion).
-- **Python packages**: `langchain-core`, `langchain-openai`,
- `langchain-anthropic`, provider SDKs (`anthropic`, `openai`, `google-genai`).
+- **Services**: `AIService`, `ChatUnifier`, `run_native_agent_loop`,
+ `Database`, `StatusBroadcaster`, `CompactionService`, `PricingService`,
+ `SkillLoader` (for `masterSkill` expansion).
+- **Python packages**: native provider SDKs (`anthropic`, `openai`,
+ `google-genai`).
- **Environment variables**: none read directly by `execute_op`.
## Edge cases & known limits
diff --git a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md b/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
index 34b17e05..553ec567 100644
--- a/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
+++ b/docs-internal/node-logic-flows/ai_agents/simpleMemory.md
@@ -17,9 +17,10 @@ its own — `ui_hints.hideRunButton` is set. When an `aiAgent` / `chatAgent`
reads this node's saved parameters (`memory_content`, `window_size`,
`long_term_enabled`, etc.) via `_build_memory_entry` and forwards them to
`AIService.execute_agent` / `execute_chat_agent`. The service parses the
-markdown, feeds the messages to the LLM, appends the new exchange, trims to
-window, and saves the updated markdown back to the memory node via
-`database.save_node_parameters`.
+markdown into native provider-neutral `Message` values, feeds them to the
+shared native agent loop, appends the new exchange, trims to window, and saves
+the updated markdown back to the memory node through the atomic memory
+persistence helper.
The plugin's own `@Operation("read")` method is only invoked when a user
explicitly hits Run (or via direct dispatch); it returns an inspector-style
@@ -66,15 +67,18 @@ button).
| `session_id` | string | `""` | no | - | Session override. Empty -> agent node_id is used by the agent-side collection; the `read` op resolves empty to `"default"`. |
| `window_size` | int | `100` | no | - | Message *pairs* to keep in short-term memory; 1-100. |
| `memory_content` | string (code editor, markdown, rows 15) | `# Conversation History\n\n*No messages yet.*\n` | no | - | Markdown body (`### **Human** (ts)` / `### **Assistant** (ts)` blocks). Edited in-place in the UI. |
-| `long_term_enabled` | boolean | `false` | no | - | When true, trimmed messages are archived to an `InMemoryVectorStore`. |
+| `long_term_enabled` | boolean | `false` | no | - | When true, trimmed messages are archived to a per-session `NativeMemoryVectorStore`. |
| `retrieval_count` | int | `3` | no | `long_term_enabled=[true]` | Relevant messages to retrieve at query time (1-10). |
+| `embedding_provider` | options | `huggingface` | no | `long_term_enabled=[true]` | `huggingface`, `openai`, or `ollama`. |
+| `embedding_model` | string | `""` | no | `long_term_enabled=[true]` | Empty selects the provider default (`BAAI/bge-small-en-v1.5`, `text-embedding-3-small`, or `nomic-embed-text`). |
+| `embedding_endpoint` | string | `""` | no | `long_term_enabled=[true]` | Optional OpenAI base URL or Ollama host. Never contains credentials. |
| `last_session_id` | string (optional, **hidden**) | `None` | no | - | Internal/display-only: last Claude session UUID. `claude_code_agent` no longer reads it; cleared when memory is wiped. |
Parameters consumed when an agent pulls the memory:
| Name | Where it comes from | Where it is read |
|------|---------------------|------------------|
-| `memory_content`, `window_size`, `long_term_enabled`, `retrieval_count`, `last_session_id` | `database.get_node_parameters(source_node_id)` inside `edge_walker._build_memory_entry` | Packed into the `memory_data` dict and forwarded to `AIService`. |
+| `memory_content`, `window_size`, `long_term_enabled`, `retrieval_count`, `embedding_provider`, `embedding_model`, `embedding_endpoint`, `last_session_id` | `database.get_node_parameters(source_node_id)` inside `edge_walker._build_memory_entry` | Packed into the `memory_data` dict and forwarded to the agent runtime. |
Parameters consumed when the node is run directly (`@Operation("read")`):
@@ -137,12 +141,12 @@ flowchart TD
- **Window semantics**: `window_size` counts *pairs* (Human + Assistant), so
`trim_markdown_window` keeps `window_size * 2` blocks. Trimmed blocks are
returned as raw text for optional vector-store archival.
-- **Long-term retrieval**: `services.memory.get_memory_vector_store(session_id)`
- lazily creates an `InMemoryVectorStore` with
- `HuggingFaceEmbeddings(BAAI/bge-small-en-v1.5)` the first time it is requested
- for that session. On `ImportError` (e.g. `langchain_huggingface` missing) the
- function returns `None` and the agent silently skips archival - **no error is
- surfaced**.
+- **Long-term retrieval**: async
+ `services.memory.get_memory_vector_store(...)` creates a
+ `NativeMemoryVectorStore` backed by the selected direct SDK. OpenAI keys are
+ resolved from `AuthService`; cache keys retain only a credential
+ fingerprint. Missing optional `sentence-transformers` support returns
+ `None`, so the agent skips archival without failing.
- **Direct-run memory store is different**: the `read` op pulls from
`services.memory_store` (via `get_messages`), **not** from the markdown
`memory_content`. A user who clicks Run on a `simpleMemory` node actively
@@ -155,20 +159,22 @@ flowchart TD
updated markdown back to this node's `node_parameters` row via
`database.save_node_parameters` after each turn.
- **Broadcasts**: none. The node is passive.
-- **External API calls**: none in the `read` op. When `long_term_enabled` is set
- and the agent archives, `HuggingFaceEmbeddings` loads the
- `BAAI/bge-small-en-v1.5` model locally (no network once cached).
-- **File I/O**: none. The vector store is `InMemoryVectorStore` - not
- persisted to disk; it is lost on process restart.
+- **External API calls**: none in the `read` op. Agent archival may call
+ OpenAI embeddings, Ollama, or load a local SentenceTransformer according to
+ the selected provider.
+- **File I/O**: none after the embedding model has loaded. The native vector
+ index is not persisted to disk and is lost on process restart.
- **Subprocess**: none.
## External Dependencies
-- **Credentials**: none.
+- **Credentials**: OpenAI long-term memory resolves the stored OpenAI key
+ through `AuthService`. Plaintext keys are not persisted in memory-node
+ parameters or vector-store cache keys.
- **Services**: `services.memory_store` (direct-run path), `AIService` (via
agents), `Database` (for parameter persistence).
-- **Python packages**: `langchain-core` (messages), `langchain_huggingface`
- + `sentence-transformers` (long-term vector store only, optional).
+- **Python packages**: `sentence-transformers` via the optional
+ `local-embeddings` extra (long-term vector store only).
- **Environment variables**: none.
## Edge cases & known limits
@@ -178,11 +184,11 @@ flowchart TD
`read` op) are **independent**. Clicking Run on the node does not
reflect what the agents see, and vice-versa. This is a known architectural
seam noted in the CLAUDE.md overview.
-- **Vector store is in-memory only**. The per-session cache in
- `services.memory.vector_store` is a module-global dict keyed by session id.
- A server restart empties it; long-term memory is not actually durable
- without external persistence.
-- **`HuggingFaceEmbeddings` import is silent-catch**. If the dependency is
+- **Vector store is in-memory only**. Its cache is keyed by session, provider,
+ model, endpoint, and a non-secret credential fingerprint. A server restart
+ empties it; long-term memory is not actually durable without external
+ persistence.
+- **Optional embedder import is silent-catch**. If `sentence-transformers` is
missing at runtime, `get_memory_vector_store` logs a warning and returns
`None`; the agent continues without archival and the user never learns.
- **No token budgeting**. `window_size` is a message-pair count, not a token
diff --git a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
index fde458bd..fe88667e 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/cerebrasChatModel.md
@@ -11,7 +11,12 @@
## Purpose
-Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama 3.1, GPT-OSS-120b, Qwen-3-235b. The `ChatModelBase.chat` operation calls `AIService.execute_chat`, which routes through `ChatUnifier`. Like Groq, Cerebras is one of the eight OpenAI-compatible providers registered in `providers/_compat.py` (the chat-path LangChain fallback was retired; LangChain `ChatCerebras` remains only on the agent path).
+Ultra-fast inference on Cerebras' custom AI hardware. Models include Llama
+3.1, GPT-OSS-120b, Qwen-3-235b. The `ChatModelBase.chat` operation calls
+`AIService.execute_chat`, which routes through `ChatUnifier`. Like Groq,
+Cerebras is one of the eight OpenAI-compatible providers registered in
+`providers/_compat.py`; bare chat and current agent executions share this
+native provider path.
## Inputs (handles)
@@ -78,7 +83,9 @@ flowchart TD
- **Validation**: missing api_key / empty prompt -> error envelope.
- **Provider routing**: `detect_ai_provider` matches `'cerebras' in node_type.lower()` **before** the groq branch, so routing is unambiguous.
-- **Native OpenAI-compatible path**: `ChatUnifier` resolves the `cerebras` spec registered in `providers/_compat.py` (reuses `OpenAIProvider` with the Cerebras base_url). No LangChain on the chat path.
+- **Native OpenAI-compatible path**: `ChatUnifier` resolves the `cerebras`
+ spec registered in `providers/_compat.py` (reuses `OpenAIProvider` with the
+ Cerebras `base_url`) for both chat and agent requests.
- **Reasoning**: same parsed/hidden mechanism as Groq Qwen - only the Qwen-3-235b variant honors it.
- **Temperature range**: narrower (0-1.5 clamp) than OpenAI/Groq. `_resolve_temperature` applies the clamp.
@@ -93,8 +100,8 @@ flowchart TD
## External Dependencies
- **Credentials**: `auth_service.get_api_key('cerebras', 'default')` plus optional `cerebras_proxy`.
-- **Services**: `ChatUnifier` + `OpenAIProvider` (base_url-ed) on the chat path; `langchain_cerebras.ChatCerebras` on the agent path.
-- **Python packages**: `openai` (chat path), `langchain-cerebras` (agent path, optional on Python >= 3.13).
+- **Services**: `ChatUnifier` + `OpenAIProvider` with the Cerebras `base_url`.
+- **Python packages**: `openai`.
- **Environment variables**: none.
## Edge cases & known limits
diff --git a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
index 5112fae4..677d5946 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/geminiChatModel.md
@@ -78,7 +78,8 @@ flowchart TD
- **Validation**: missing api_key / empty prompt -> error envelope.
- **Provider routing**: matches `'gemini' in node_type.lower()`.
-- **Native SDK**: uses `google.genai.Client` (NOT LangChain's `langchain_google_genai`) to dodge the Windows/Python 3.13 gRPC import hang.
+- **Native SDK**: `ChatUnifier` uses `google.genai.Client` for both chat and
+ current agent requests.
- **Thinking budget**: `thinkingBudget` -> `NativeThinkingConfig.budget` -> Gemini `thinking_budget` API parameter.
- **Model string scrubbing**: `[FREE] ` prefix stripped; `owner/model` prefix stripped (non-OpenRouter).
@@ -99,7 +100,9 @@ flowchart TD
## Edge cases & known limits
-- **Windows/Python 3.13 quirk**: importing `langchain_google_genai` hangs due to a gRPC deadlock. The native path bypasses this entirely. The LangChain fallback is NEVER used for Gemini on `execute_chat` - it is only lazy-imported for agent execution.
+- **Windows/Python 3.13 quirk**: the native `google-genai` provider avoids the
+ former adapter's gRPC import deadlock. Both current chat and agent
+ executions use the native path.
- **Thinking budget units**: expressed as token count, not low/medium/high effort levels. Defaults to 2048.
- **Safety settings**: forwarded via the SDK; malformed values surface as `success=false` in the envelope.
- **`maxTokens` clamp**: capped at the model's ceiling (65K for 2.5/3.x).
diff --git a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
index bec6068d..83b6d920 100644
--- a/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
+++ b/docs-internal/node-logic-flows/ai_chat_models/groqChatModel.md
@@ -11,7 +11,11 @@
## Purpose
-Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4, Qwen3-32b, GPT-OSS. The `ChatModelBase.chat` operation calls `AIService.execute_chat`, which routes every provider through `ChatUnifier`; Groq is one of the eight OpenAI-compatible providers registered in `providers/_compat.py` (the chat-path LangChain fallback was retired — LangChain `ChatGroq` remains only on the agent path).
+Ultra-fast inference via Groq's LPU hardware. Models include Llama 3.x / 4,
+Qwen3-32b, GPT-OSS. The `ChatModelBase.chat` operation calls
+`AIService.execute_chat`, which routes through `ChatUnifier`; Groq is one of
+the eight OpenAI-compatible providers registered in `providers/_compat.py`.
+Bare chat and current agent executions share this native provider path.
## Inputs (handles)
@@ -77,7 +81,9 @@ flowchart TD
- **Validation**: missing api_key / empty prompt -> error envelope.
- **Provider routing**: matches `'groq' in node_type.lower()`. Note the ordering in `detect_ai_provider`: deepseek/kimi/mistral/cerebras checked first, so none of those accidentally resolve to groq.
-- **Native OpenAI-compatible path**: `ChatUnifier` resolves the `groq` spec registered in `providers/_compat.py` (reuses `OpenAIProvider` with the `base_url` from `llm_defaults.json`). No LangChain on the chat path.
+- **Native OpenAI-compatible path**: `ChatUnifier` resolves the `groq` spec
+ registered in `providers/_compat.py` (reuses `OpenAIProvider` with the
+ `base_url` from `llm_defaults.json`) for both chat and agent requests.
- **Reasoning**: only Qwen3-32b actually honors `reasoningFormat`. Non-Qwen models ignore the flag.
- **Model string scrubbing**: `[FREE] ` prefix stripped; `owner/` prefix stripped (non-OpenRouter).
@@ -92,8 +98,8 @@ flowchart TD
## External Dependencies
- **Credentials**: `auth_service.get_api_key('groq', 'default')` plus optional `groq_proxy`.
-- **Services**: `ChatUnifier` + `OpenAIProvider` (base_url-ed) on the chat path; `langchain_groq.ChatGroq` on the agent path.
-- **Python packages**: `openai` (chat path), `langchain-groq` (agent path).
+- **Services**: `ChatUnifier` + `OpenAIProvider` with the Groq `base_url`.
+- **Python packages**: `openai`.
- **Environment variables**: none.
## Edge cases & known limits
diff --git a/docs-internal/node-logic-flows/code_fs_process/fileModify.md b/docs-internal/node-logic-flows/code_fs_process/fileModify.md
index c70dc0e4..4d45f707 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fileModify.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fileModify.md
@@ -4,7 +4,7 @@
|------|-------|
| **Category** | code_fs_process / filesystem |
| **Backend handler** | [`server/nodes/filesystem/file_modify/__init__.py::FileModifyNode.modify`](../../../server/nodes/filesystem/file_modify/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("modify")`) |
-| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
+| **Backend** | Native `WorkspaceBackend` plus atomic write helpers in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/file-modify-skill/SKILL.md`](../../../server/skills/coding_agent/file-modify-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `file_modify` |
@@ -12,13 +12,11 @@
## Purpose
Writes a new file or edits an existing one inside the per-workflow workspace.
-Two operations: `write` (create/overwrite) and `edit` (string find-and-replace).
-Delegates to `backend.write()` / `backend.edit()` on `NushellBackend`
-(`virtual_mode=True`, `inherit_env=True`) via `get_backend()`, invoked through
-`asyncio.to_thread()`. Paths are normalised with `normalize_virtual_path()`
-first. Because deepagents' `backend.write()` refuses to overwrite, the `write`
-op first unlinks any pre-existing file at the resolved path (wholesale
-create-or-replace), guarding against `IsADirectoryError`.
+Two operations: `write` (create/overwrite) and `edit` (exact string
+find-and-replace). Paths are normalized and resolved through
+`WorkspaceBackend`, then guarded by a per-path lock. Both operations use a
+same-directory temporary file plus `os.replace` so concurrent writes cannot
+expose partial content; existing file modes and newline bytes are preserved.
## Inputs (handles)
@@ -71,13 +69,13 @@ flowchart TD
B -- yes --> E[raise NodeUserError:<br/>file_path is required]
B -- no --> C[get_backend + normalize_virtual_path]
C --> D{operation?}
- D -- write --> W[to_thread _do_write:<br/>unlink existing if file,<br/>raise IsADir if dir, backend.write]
+ D -- write --> W[path lock + worker thread:<br/>raise IsADir if dir,<br/>atomic_write_text]
W -- OSError/ValueError --> Eenv[raise NodeUserError str e]
W -- result.error --> Eenv
W -- ok --> Wok[Return dict<br/>operation=write, file_path=result.path]
D -- edit --> Ei{old_string empty?}
Ei -- yes --> Ereq[raise NodeUserError:<br/>old_string is required for edit]
- Ei -- no --> Ed[to_thread backend.edit file_path,<br/>old_string, new_string, replace_all]
+ Ei -- no --> Ed[path lock + worker thread:<br/>exact replacement + atomic_write_text]
Ed -- result.error --> Eenv
Ed -- ok --> Edok[Return dict<br/>occurrences=result.occurrences]
D -- other --> Unk[raise NodeUserError:<br/>Unknown operation: <op>]
@@ -91,10 +89,9 @@ flowchart TD
is required for edit")`.
- Anything other than `write` / `edit` -> `raise NodeUserError("Unknown
operation: <op>")` (Pydantic Literal already constrains it).
-- **`write` overwrite handling**: `_do_write` resolves the path, raises
- `IsADirectoryError` if it's a directory, `unlink()`s an existing file, then
- calls `backend.write()` (which itself refuses to overwrite). `OSError` /
- `ValueError` are caught and re-raised as `NodeUserError`.
+- **`write` overwrite handling**: `_do_write` resolves the contained path,
+ rejects directories, and atomically replaces or creates the file. `OSError`
+ / `ValueError` are caught and re-raised as `NodeUserError`.
- **Backend-level errors**: a non-empty `result.error` from `write`/`edit`
short-circuits with `raise NodeUserError(result.error)`. Common: non-unique
`old_string` when `replace_all=False`, path escape with `virtual_mode=True`.
@@ -112,30 +109,36 @@ flowchart TD
- **External API calls**: none.
- **File I/O**:
- `get_backend` ensures the workspace root exists.
- - For `write`: `unlink()` an existing file then writes `<root>/<file_path>`.
- - For `edit`: rewrites `<root>/<file_path>` in place.
+ - `write` and `edit` write a same-directory temporary file, flush it, retain
+ an existing file's mode, and atomically replace the destination.
- **Subprocess**: none.
## External Dependencies
-- **Python packages**: `deepagents` (via `NushellBackend`).
+- **Python packages**: none beyond the standard library.
- **Environment variables**: `WORKSPACE_BASE_DIR`.
## Edge cases & known limits
- **Silent overwrite on `write`**: there is no "fail if exists" flag - every
- `write` call unlinks then replaces the file.
+ `write` call atomically replaces the file.
- **`edit` without a match**: raises `NodeUserError` (backend surfaces the
message). The handler never distinguishes "not found" from "multiple
matches" - both are `NodeUserError` strings.
-- **No encoding option**: the backend opens files in text mode with the
- platform default encoding.
+- **No encoding option**: writes are always UTF-8 and preserve caller newline
+ bytes (`newline=""`).
- **`working_directory` not exposed**: `extra="ignore"` means the sandbox
cannot be widened via a node param on this node.
- **`replace_all=true` with empty `new_string`**: effectively deletes every
occurrence of `old_string`. No safeguard.
-- **Atomicity**: the backend does not do atomic writes - a crash mid-write
- can leave a truncated file.
+- **Symlink-race containment**: POSIX uses root-anchored directory descriptors,
+ `O_NOFOLLOW`, and descriptor-relative atomic rename. Windows rejects
+ observed reparse points and revalidates directory, temporary-file, and
+ target identities before and after `os.replace`.
+- **Windows residual**: Python's standard library has no handle-relative
+ Windows rename. A hostile process with permission to swap a checked parent
+ in the final check-to-`os.replace` interval can redirect the replacement;
+ post-validation detects this but cannot safely roll it back.
## Related
diff --git a/docs-internal/node-logic-flows/code_fs_process/fileRead.md b/docs-internal/node-logic-flows/code_fs_process/fileRead.md
index 1812d8cb..9492d847 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fileRead.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fileRead.md
@@ -4,18 +4,18 @@
|------|-------|
| **Category** | code_fs_process / filesystem |
| **Backend handler** | [`server/nodes/filesystem/file_read/__init__.py::FileReadNode.read`](../../../server/nodes/filesystem/file_read/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("read")`) |
-| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
+| **Backend** | Native `WorkspaceBackend` in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/file-read-skill/SKILL.md`](../../../server/skills/coding_agent/file-read-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `file_read` |
## Purpose
-Reads a file within the per-workflow workspace. Delegates to `backend.read()`
-on `NushellBackend` (a `LocalShellBackend` subclass) via `get_backend()` in
+Reads a file within the per-workflow workspace. Delegates to
+`WorkspaceBackend.read()` via `get_backend()` in
[`_backend.py`](../../../server/nodes/filesystem/_backend.py). The backend is
-instantiated with `virtual_mode=True`, `inherit_env=True`, and a `root_dir`
-pinned to `<DATA_DIR>/workspaces/<workflow_slug>/` (resolution:
+instantiated with a `root_dir` pinned to
+`<DATA_DIR>/workspaces/<workflow_slug>/` (resolution:
`working_directory` param > `ctx.workspace_dir` >
`Settings().workspace_base_resolved/default`), so path traversal outside the
workspace is rejected by the backend. The plugin first runs the requested
@@ -71,7 +71,7 @@ flowchart TD
A[read] --> B{file_path empty?}
B -- yes --> E[raise NodeUserError:<br/>file_path is required]
B -- no --> C[get_backend params, ctx.raw:<br/>root = ctx.workspace_dir OR<br/>Settings().workspace_base_resolved/default]
- C --> D[NushellBackend root_dir=root,<br/>virtual_mode=True, inherit_env=True]
+ C --> D[WorkspaceBackend root_dir=root,<br/>inherit_env=True]
D --> N[file_path = normalize_virtual_path]
N --> G[to_thread backend.read file_path, offset, limit]
G -- FileNotFound/IsADir/ValueError --> H[raise NodeUserError str e]
@@ -88,8 +88,8 @@ flowchart TD
- **Directory creation**: `get_backend` ensures the resolved root exists before
building the backend, so a read against a fresh workflow works.
- **Path normalization**: `normalize_virtual_path()` strips drive/root/UNC
- anchors; `virtual_mode=True` then rejects `..`/`~` traversal that escapes the
- root.
+ anchors; the backend rejects `..`/`~`, resolves symlinks, and verifies the
+ final path remains under the workspace root.
- **Error paths**: `FileNotFoundError` / `IsADirectoryError` / `ValueError`
(bad offset, path escape) are caught and re-raised as `NodeUserError`; a
non-empty `result.error` from the backend also raises `NodeUserError`. Both
@@ -109,8 +109,7 @@ flowchart TD
- **Credentials**: none.
- **Services**: none.
-- **Python packages**: `deepagents` (via `NushellBackend` in `_backend.py`),
- `asyncio`.
+- **Python packages**: none beyond the standard library.
- **Environment variables**: `WORKSPACE_BASE_DIR` (read by `core.config.Settings`).
## Edge cases & known limits
@@ -118,9 +117,7 @@ flowchart TD
- **Errors are `NodeUserError`**: "file not found", "is a directory", "path
escapes workspace", "bad offset" all become `error_type="NodeUserError"`
envelopes with a string message (no distinct error code per case).
-- **Binary files**: `backend.read()` decodes as text; a binary file raises a
- `UnicodeDecodeError` (uncaught here, so it surfaces with a full traceback as a
- generic error envelope, not `NodeUserError`).
+- **Binary files**: invalid UTF-8 is returned as base64 content.
- **`offset`/`limit` Pydantic-validated**: `offset>=0`, `limit` clamped 1-10000.
Negative inputs are rejected at validation, not forwarded.
- **`working_directory` not exposed**: `FileReadParams` uses `extra="ignore"`,
diff --git a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md b/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
index a477007e..f56c136c 100644
--- a/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
+++ b/docs-internal/node-logic-flows/code_fs_process/fsSearch.md
@@ -4,7 +4,7 @@
|------|-------|
| **Category** | code_fs_process / filesystem |
| **Backend handler** | [`server/nodes/filesystem/fs_search/__init__.py::FsSearchNode.search`](../../../server/nodes/filesystem/fs_search/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("search")`) |
-| **Backend** | `NushellBackend` (subclasses `deepagents.backends.LocalShellBackend`; `ls_info`, `glob_info`, `grep_raw`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
+| **Backend** | Native `WorkspaceBackend` (`ls_info`, `glob_info`, `grep_raw`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/coding_agent/fs-search-skill/SKILL.md`](../../../server/skills/coding_agent/fs-search-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `fs_search` |
@@ -12,9 +12,10 @@
## Purpose
Three-mode filesystem query node confined to the per-workflow workspace.
-Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on `NushellBackend`
-(`virtual_mode=True`, `inherit_env=True`) via `get_backend()` depending on
-`mode`. The `path` is run through `normalize_virtual_path()` first.
+Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on the native
+`WorkspaceBackend` via `get_backend()` depending on `mode`. The `path` is run
+through `normalize_virtual_path()` first, and each filesystem operation
+resolves the target beneath the workspace root with symlink containment.
## Inputs (handles)
@@ -28,7 +29,7 @@ Dispatches to `ls_info()`, `glob_info()`, or `grep_raw()` on `NushellBackend`
|------|------|---------|----------|---------------------|-------------|
| `mode` | `ls` \| `glob` \| `grep` (Literal) | `ls` | no | - | `ls`, `glob`, or `grep` |
| `path` | string | `.` | no | - | Directory path to search in (workspace-relative) |
-| `pattern` | string | `""` | yes (when `mode != ls`) | - | Glob pattern or grep regex |
+| `pattern` | string | `""` | yes (when `mode != ls`) | - | Glob pattern or literal grep text |
`FsSearchParams` uses `extra="ignore"` — there are NO `file_filter` or
`working_directory` params; the model drops unknown keys. The grep mode passes
@@ -52,8 +53,8 @@ declared on these fields in the current Params model.)
{ path: string; pattern: string; matches: Array<GrepMatch>; count: number }
```
-`FileInfo` and `GrepMatch` are the dataclass shapes returned by the
-`deepagents` backend, converted via `dict(entry)`.
+`FileInfo` and `GrepMatch` are JSON-safe dictionaries returned by the native
+backend. The node defensively copies each with `dict(entry)`.
`node_output_schemas.FsSearchOutput` declares `path` / `entries` / `matches` /
`pattern` / `count` (extra fields allowed by `_OutputBase`).
@@ -100,12 +101,12 @@ flowchart TD
- **File I/O**:
- `get_backend` ensures the workspace root exists.
- Reads directory listings and file contents under `<root>/<path>`.
-- **Subprocess**: none (`grep_raw` is a Python regex walk in recent
- `deepagents`; not a `grep(1)` exec).
+- **Subprocess**: none (`grep_raw` is a local Python file walk and literal
+ substring search; it does not invoke `grep(1)`).
## External Dependencies
-- **Python packages**: `deepagents` (via `NushellBackend`).
+- **Python packages**: standard library only.
- **Environment variables**: `WORKSPACE_BASE_DIR`.
## Edge cases & known limits
@@ -121,9 +122,8 @@ flowchart TD
- **`path='.'` resolves against the workspace root, not the server CWD**.
- **`working_directory` not exposed**: `extra="ignore"` means the sandbox
cannot be widened via a node param on this node.
-- **Dataclass `dict()` conversion**: relies on `FileInfo` / `GrepMatch`
- being dataclasses (they implement `__iter__` via dataclass). A future
- backend upgrade that returns `TypedDict`s could break this.
+- **Result copying**: `dict()` makes each native result mapping independent
+ before output validation and serialization.
## Related
diff --git a/docs-internal/node-logic-flows/code_fs_process/shell.md b/docs-internal/node-logic-flows/code_fs_process/shell.md
index 5e92026d..4cbae3df 100644
--- a/docs-internal/node-logic-flows/code_fs_process/shell.md
+++ b/docs-internal/node-logic-flows/code_fs_process/shell.md
@@ -4,22 +4,22 @@
|------|-------|
| **Category** | code_fs_process / filesystem |
| **Backend handler** | [`server/nodes/filesystem/shell/__init__.py::ShellNode.execute_op`](../../../server/nodes/filesystem/shell/__init__.py) (dispatched via `BaseNode.execute()` + `@Operation("execute")`) |
-| **Backend** | `NushellBackend.execute` (subclasses `deepagents.backends.LocalShellBackend`) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
+| **Backend** | `WorkspaceBackend.execute` (with `NushellBackend` retained as a compatibility alias) in [`server/nodes/filesystem/_backend.py`](../../../server/nodes/filesystem/_backend.py) |
| **Tests** | [`server/tests/nodes/test_code_fs_process.py`](../../../server/tests/nodes/test_code_fs_process.py) |
| **Skill (if any)** | [`server/skills/terminal/shell-skill/SKILL.md`](../../../server/skills/terminal/shell-skill/SKILL.md) |
| **Dual-purpose tool** | yes - tool name `shell_execute` |
## Purpose
-Runs a short-lived shell command inside the per-workflow workspace. The backend
-is **Nushell** (`NushellBackend`, a `LocalShellBackend` subclass) constructed
-with `inherit_env=True`, so external tools like `npm`, `node`, `python`, and
-`git` ARE reachable on PATH (it falls back to upstream `LocalShellBackend.execute()`
-/ POSIX `sh` / cmd.exe only when `nu` isn't installed). The command grammar is
-Nushell, not bash — `&&` / `||` / `$VAR` / backticks / `>` do not work; see
-`shell-skill/SKILL.md` for the Nu equivalents. For long-running processes
-(dev servers, watchers) users are directed to [`processManager`](./processManager.md);
-this node kills the command at `timeout`.
+Runs a short-lived shell command inside the per-workflow workspace. The native
+`WorkspaceBackend` is constructed with `inherit_env=True`, so external tools
+like `npm`, `node`, `python`, and `git` ARE reachable on PATH. It uses Nushell
+when `nu` is installed and otherwise uses the host shell (`sh` or `cmd.exe`).
+When Nushell is selected, `&&` / `||` / `$VAR` / backticks / `>` do not use
+bash semantics; see `shell-skill/SKILL.md` for the Nu equivalents. For
+long-running processes (dev servers, watchers), users are directed to
+[`processManager`](./processManager.md); this node kills the command at
+`timeout`.
NOTE: the node's `description` metadata still reads "sandboxed; no system PATH",
which no longer matches the `inherit_env=True` backend — a code-side copy
@@ -75,7 +75,7 @@ backend root is `ctx.workspace_dir`.
flowchart TD
A[execute_op] --> P{_BASH_CHAIN_RE matches<br/> && / || ?}
P -- yes --> Pe[raise NodeUserError:<br/>use ; or try{}catch{}]
- P -- no --> C[get_backend workspace root<br/>NushellBackend virtual_mode, inherit_env]
+ P -- no --> C[get_backend workspace root<br/>WorkspaceBackend, inherited environment]
C --> D[to_thread backend.execute command, timeout]
D --> S[strip_ansi result.output]
S --> I{exit_code?}
@@ -90,7 +90,7 @@ flowchart TD
- **Validation**: empty `command` is rejected by Pydantic `min_length=1` (no
manual check). A ` && ` / ` || ` in the command -> `raise NodeUserError` with
a Nushell-equivalent hint (`;` or `try { … } catch { … }`).
-- **Inherited PATH**: `NushellBackend(inherit_env=True)` — external tools
+- **Inherited PATH**: `WorkspaceBackend(inherit_env=True)` — external tools
(`npm`, `node`, `python`, `git`, …) ARE reachable. The previous "scrubbed
PATH" framing no longer holds. AI agents are still steered toward
`process_manager` for long-running daemons (the shell kills at `timeout`).
@@ -100,9 +100,9 @@ flowchart TD
- **Non-zero exit still returns `success: true`**: the envelope-level success
reflects whether the handler itself finished, not whether the command
succeeded. Users must inspect `exit_code` in the payload.
-- **Truncation**: the backend caps output size internally; when capped
- `truncated=true` surfaces to the caller. The exact cap lives in
- `deepagents`.
+- **Truncation**: the native backend caps output at 100,000 characters by
+ default; when capped, `truncated=true` and a trailing marker surface to the
+ caller.
- **ANSI stripping**: `result.output` is run through `core.ansi.strip_ansi`
(a wrapper over **`click.unstyle`**) before being returned as `stdout` (and
before the operator-log line), so colour/cursor codes from tools like
@@ -126,7 +126,7 @@ flowchart TD
## External Dependencies
-- **Python packages**: `deepagents` (via `NushellBackend`), `core.ansi`.
+- **Python packages**: standard library plus `core.ansi`.
- **Environment variables**: `WORKSPACE_BASE_DIR`.
- **OS utilities**: `nu` (Nushell) on PATH for the primary path; whatever the
command references is reachable because PATH is inherited.
@@ -147,9 +147,9 @@ flowchart TD
command - `input_data` from upstream nodes does not reach it.
- **No environment variable override**: users cannot inject env vars via
parameters; the backend inherits the server's env (`inherit_env=True`).
-- **Windows path traversal check is string-based**: `virtual_mode=True`
- normalises paths but trust-boundary violations depend on the backend
- implementation.
+- **Shell commands are open-world**: filesystem helper methods enforce
+ workspace containment, but a shell command can use absolute paths or invoke
+ external programs. This preserves the node's historical shell behavior.
## Related
diff --git a/docs-internal/node-logic-flows/document/embeddingGenerator.md b/docs-internal/node-logic-flows/document/embeddingGenerator.md
index 4458d63b..1a4d7a2c 100644
--- a/docs-internal/node-logic-flows/document/embeddingGenerator.md
+++ b/docs-internal/node-logic-flows/document/embeddingGenerator.md
@@ -12,8 +12,8 @@
Generate vector embeddings from a list of text chunks using one of three
backends: HuggingFace (local sentence-transformers, default), OpenAI
-(API-hosted), or Ollama (local server). The heavy embedding call runs in a
-thread pool via `asyncio.to_thread` so the event loop is not blocked.
+(API-hosted), or Ollama (local server). Local HuggingFace encoding runs in a
+worker thread; OpenAI and Ollama use their native asynchronous SDK clients.
## Inputs (handles)
@@ -28,8 +28,9 @@ thread pool via `asyncio.to_thread` so the event loop is not blocked.
| `chunks` | array | `[]` | yes | - | List of `{content, source, chunk_index}` dicts |
| `provider` | options | `huggingface` | no | - | `huggingface` / `openai` / `ollama` |
| `model` | string | `BAAI/bge-small-en-v1.5` | no | - | Model name for the chosen provider |
-| `batch_size` | number | `32` | no | - | Texts per embedding batch (1-256). Declared but not currently used by the embed op. |
+| `batch_size` | number | `32` | no | - | Texts per embedding batch (1-256), used by every provider |
| `api_key` | string (password) | `""` | no (required for openai) | `provider=openai` | OpenAI API key |
+| `endpoint` | string | `""` | no | `provider=openai/ollama` | Optional OpenAI `base_url` or Ollama `host` |
Params are snake_case (`Params = EmbeddingGeneratorParams`).
@@ -67,15 +68,14 @@ flowchart TD
A[EmbeddingGeneratorNode.embed] --> B{chunks empty?}
B -- yes --> Ret0[Return success=true<br/>zero-valued payload]
B -- no --> C[Extract text list: dict.content or str(chunk)]
- C --> D{provider}
- D -- huggingface --> Dh[Import langchain_huggingface<br/>raise NodeUserError with install hint if missing]
- D -- openai --> Do[Import langchain_openai.OpenAIEmbeddings]
- D -- ollama --> Dl[Import langchain_ollama.OllamaEmbeddings]
- D -- other --> Dx[raise NodeUserError Unknown provider]
- Dh --> E[HuggingFaceEmbeddings(model_name=model)]
- Do --> F[OpenAIEmbeddings(model=model, api_key=api_key)]
- Dl --> G[OllamaEmbeddings(model=model)]
- E --> H[asyncio.to_thread embedder.embed_documents(texts)]
+ C --> D[services.memory.vector_store.create_embedder]
+ D --> Dh[HuggingFace: lazy SentenceTransformer]
+ D --> Do[OpenAI: openai.AsyncOpenAI]
+ D --> Dl[Ollama: ollama.AsyncClient]
+ Dh --> E[SentenceTransformer model<br/>encode batches in asyncio.to_thread]
+ Do --> F[Await embeddings.create per batch]
+ Dl --> G[Await client.embed per batch]
+ E --> H[Normalized number vectors]
F --> H
G --> H
H --> I[Compute dimensions from len(embeddings[0])]
@@ -85,10 +85,15 @@ flowchart TD
## Decision Logic
- **Empty chunks**: short-circuits to the empty-payload success envelope.
-- **Provider dispatch**: string match on `provider`; `provider` is a `Literal` so Pydantic rejects unknown values before dispatch (the `else: NodeUserError` is a dead path).
-- **Embedding call**: always runs in a worker thread via `asyncio.to_thread`.
+- **Provider dispatch**: the node and long-term memory use the same
+ `create_embedder` factory and `AsyncEmbedder` protocol. `provider` is a
+ `Literal`, so Pydantic rejects unknown values before dispatch.
+- **Embedding call**: Hugging Face encoding runs in a worker thread; OpenAI
+ and Ollama use their native async SDKs directly.
- **Dimensions**: inferred from first embedding; `0` if the call returned `[]`.
-- **Missing huggingface package**: raises `NodeUserError` with an install hint (`pip install langchain-huggingface sentence-transformers`); `BaseNode.execute()` logs one WARN line (no traceback) and returns `{success: false, error_type: "NodeUserError"}`.
+- **Missing huggingface package**: raises `NodeUserError` with an install hint
+ for the `local-embeddings` extra; `BaseNode.execute()` logs one WARN line
+ (no traceback) and returns `{success: false, error_type: "NodeUserError"}`.
## Side Effects
@@ -103,14 +108,17 @@ flowchart TD
## External Dependencies
-- **Credentials**: `api_key` param (OpenAI only). Not fetched from `auth_service` - must be passed as a node parameter.
-- **Python packages**: `langchain-huggingface` + `sentence-transformers` (HF), `langchain-openai` (OpenAI), `langchain-ollama` (Ollama).
-- **Environment variables**: `OLLAMA_HOST` respected by `langchain_ollama`.
+- **Credentials**: `api_key` parameter (OpenAI only). This document node is
+ outside `AI_MODEL_TYPES`, so callers must provide it explicitly.
+- **Python packages**: optional `sentence-transformers` (HF), direct `openai`
+ SDK (OpenAI), and direct `ollama` SDK (Ollama).
+- **Environment variables**: `OLLAMA_HOST` is respected by `ollama.AsyncClient`.
## Edge cases & known limits
-- OpenAI `api_key` is read from the node parameter, NOT from `auth_service.get_api_key('openai')`. This is inconsistent with other OpenAI-using nodes (documented gotcha).
-- `batch_size` is a declared param but the embed op passes everything in one `embed_documents` call; provider-side batch limits apply.
+- OpenAI `api_key` is passed to the shared factory in memory only and is never
+ included in output.
+- `batch_size` controls requests/encoding batches for every provider.
- HF first-run model download can take minutes; the node will appear to hang.
- Model name default (`BAAI/bge-small-en-v1.5`) is HF-specific; you must override `model` when switching provider.
- No cost tracking (unlike LLM chat nodes).
diff --git a/docs-internal/node-logic-flows/document/textChunker.md b/docs-internal/node-logic-flows/document/textChunker.md
index 3deb6a56..fac92ef3 100644
--- a/docs-internal/node-logic-flows/document/textChunker.md
+++ b/docs-internal/node-logic-flows/document/textChunker.md
@@ -10,11 +10,10 @@
## Purpose
-Split documents into overlapping text chunks suitable for embedding. Uses
-LangChain's `RecursiveCharacterTextSplitter` (default) or
-`MarkdownTextSplitter` for structure-aware splitting of Markdown content.
-Typically sits between `documentParser` and `embeddingGenerator` in a RAG
-pipeline.
+Split documents into overlapping text chunks suitable for embedding. The
+dependency-free local splitter preserves the former recursive-character and
+Markdown separator/overlap semantics. Typically sits between
+`documentParser` and `embeddingGenerator` in a RAG pipeline.
## Inputs (handles)
@@ -62,8 +61,8 @@ flowchart TD
A[TextChunkerNode.chunk] --> B{documents empty?}
B -- yes --> Ret0[Return success=true<br/>chunks=[], chunk_count=0]
B -- no --> C{strategy}
- C -- markdown --> Cm[MarkdownTextSplitter]
- C -- other --> Cr[RecursiveCharacterTextSplitter]
+ C -- markdown --> Cm[Local Markdown separator set]
+ C -- other --> Cr[Local recursive separator set]
Cm --> D[For each doc: extract content + source]
Cr --> D
D --> E{content truthy?}
@@ -93,13 +92,15 @@ flowchart TD
## External Dependencies
- **Credentials**: none.
-- **Python packages**: `langchain-text-splitters>=0.3.0`.
+- **Python packages**: none beyond the standard library; splitting lives in
+ `server/nodes/document/text_chunker/_splitter.py`.
- **Environment variables**: none.
## Edge cases & known limits
- `token` strategy is mentioned in the frontend options/description but there is no branch for it; it currently falls through to recursive. Documented gotcha.
-- `chunkSize`/`chunkOverlap` are cast via `int(...)` without clamping - out-of-range values pass through to LangChain and may raise if `overlap >= chunkSize`.
+- `chunkSize`/`chunkOverlap` are cast via `int(...)`; the local splitter raises
+ if `overlap > chunkSize`.
- Pure function, no randomness - deterministic given the same inputs.
- Chunks preserve order per-document but there is no global index across documents.
diff --git a/docs-internal/node-logic-flows/specialized_agents/_pattern.md b/docs-internal/node-logic-flows/specialized_agents/_pattern.md
index 656acb02..961fe547 100644
--- a/docs-internal/node-logic-flows/specialized_agents/_pattern.md
+++ b/docs-internal/node-logic-flows/specialized_agents/_pattern.md
@@ -53,7 +53,7 @@ All 13 agents share these input handles.
| `input-main` | input/main | no | Upstream data; auto-prompt fallback when `prompt` param is empty |
| `input-skill` | input/skill | no | Skill node(s) providing SKILL.md context |
| `input-memory` | input/memory | no | `simpleMemory` node for conversation history |
-| `input-tools` | input/tools | no | Tool nodes exposed to the LLM via `chat_model.bind_tools` |
+| `input-tools` | input/tools | no | Tool nodes compiled into provider-neutral `AgentToolSpec` definitions |
| `input-task` | input/task | no | `taskTrigger` events from delegated child agents |
| `input-teammates` | input/teammates | no | **Only on `orchestrator_agent` and `ai_employee`** (via `team_lead_agent_handles()`) -- connected agents become Task Manager-authorized assignees |
@@ -72,7 +72,7 @@ There are **no** per-agent parameter extras: `orchestrator_agent` and
| Name | Type | Default | Required | displayOptions.show | Description |
|------|------|---------|----------|---------------------|-------------|
| `prompt` | string | `""` | no | - | User prompt; falls back to upstream output when empty (4 rows) |
-| `provider` | enum | `openai` | no | - | `openai` / `anthropic` / `gemini` / `openrouter` / `groq` / `cerebras` / `deepseek` / `kimi` / `mistral` / `ollama` / `lmstudio` |
+| `provider` | enum | `openai` | no | - | `openai` / `anthropic` / `gemini` / `openrouter` / `xai` / `groq` / `cerebras` / `deepseek` / `kimi` / `mistral` / `ollama` / `lmstudio` |
| `model` | string | `""` | no | - | Model ID (resolved against the provider; empty falls through to default) |
| `system_message` | string\|null | `"You are a helpful assistant"` | no | - | System instructions (3 rows) |
| `temperature` | float\|null | `None` | no | group `options` | 0.0-2.0; `None` -> `agent.default_temperature` in `llm_defaults.json` |
@@ -162,8 +162,9 @@ flowchart TD
agent's execution phase (`executing`, `executing_tool`, `success`,
`error`), plus `executing_tool` fired on connected tool nodes when the LLM
invokes them.
-- **External API calls**: one or more calls to the configured LLM provider
- (see `services/ai.py::execute_chat_agent` -> LangChain chat model + `_run_agent_loop`).
+- **External API calls**: one or more calls through `ChatUnifier` to the
+ configured native provider SDK. Tool-enabled turns use the shared
+ `run_native_agent_loop`.
- **Subprocess / file I/O**: none directly in the plugin; tools invoked by
the LLM may trigger those themselves.
@@ -171,9 +172,11 @@ flowchart TD
- **Credentials**: `auth_service.get_api_key(<provider>)` for the
configured LLM provider.
-- **Services**: `AIService.execute_chat_agent`, `CompactionService`,
- `PricingService`, `StatusBroadcaster`.
-- **Python packages**: `langchain-core`, plus provider SDKs.
+- **Services**: `AIService.execute_chat_agent`, `ChatUnifier`,
+ `run_native_agent_loop`, `CompactionService`, `PricingService`,
+ `StatusBroadcaster`.
+- **Python packages**: native provider SDKs (`anthropic`, `openai`,
+ `google-genai`).
## Edge cases & known limits
diff --git a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
index be5dc4ac..e86fa77e 100644
--- a/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
+++ b/docs-internal/node-logic-flows/specialized_agents/rlmAgent.md
@@ -95,8 +95,8 @@ flowchart TD
- **Credentials**: `auth_service.get_api_key(<provider>)` for big LM, and
optionally the small-LM provider wired as a connected chat-model node.
- **Services**: `RLMService`, `StatusBroadcaster`, `PricingService`.
-- **Python packages**: standard library + LangChain core; no special
- deepagents dependency.
+- **Python packages**: the `rlm` package plus the native OpenCompany LLM
+ configuration and tool-bridge helpers.
## Edge cases & known limits
diff --git a/docs-internal/performance.md b/docs-internal/performance.md
index 666e071f..7a842c78 100644
--- a/docs-internal/performance.md
+++ b/docs-internal/performance.md
@@ -60,7 +60,7 @@ T+0.00 — port-free begin
├── FastAPI base imports (0.27 s)
├── DI container (0.42 s)
├── Core service imports (0.36 s)
- ├── AIService import (0.70 s) ← lazy LangChain payoff
+ ├── AIService import (0.70 s) ← native boundary + lazy history shims
├── Routers + plugin walker (0.84 s, 137 plugins)
└── All imports complete (1.98 s)
T+2.25 — Lifespan startup
@@ -146,7 +146,7 @@ optimisations above; sum is what hurts.
|---|---|---|---|---|
| 1 | Plugin walker at import time (152 modules under `server/nodes/`; ~2 s warm / ~4.7 s cold, dominated by `nodes/google`'s eager `googleapiclient` import) | ~2 s | Backend | `_HANDLER_REGISTRY` populates via `BaseNode.__init_subclass__` at import. Lazy `googleapiclient` is the cheap win (see follow-ups); full lazy-loading of the walker is the big-blast-radius option. |
| 2 | TanStack Query auth bootstrap retry window | ~3-5 s | Frontend | See "remaining +5 s gap" above. |
-| 3 | LangChain ecosystem imports inside `services/ai.py` | ~0.7 s | Backend | Already lazied; further wins require refactoring `AgentState`'s `Annotated[Sequence[BaseMessage], ...]` to defer `BaseMessage` resolution. |
+| 3 | AIService/native orchestration import graph | ~0.7 s | Backend | Historical May baseline; new agent execution uses native `Message` / `AgentToolSpec` values and provider registration remains lazy. Re-measure before attributing this cost further. |
| 4 | Status-broadcaster refresh (`refresh_all_services`, 2.6 s) | ~2.6 s | Backend | Runs after `Application startup complete`, doesn't block server-ready. WhatsApp + Telegram are the long tails. |
| 5 | Process spawn + Python interpreter init | ~0.4 s | Platform | Unavoidable without compiling Python to a binary (Nuitka / PyOxidizer — explicitly out of scope). |
@@ -247,7 +247,7 @@ Tracked but explicitly **not** in any active plan.
| Defer first-launch example import off the workflows REST request | ~8-10 s of first-launch first paint | `routers/database.py:get_all_workflows` awaits `import_examples_for_user` inline, holding the HTTP response. Move to a lifespan background task (pattern: `_refresh_registry` / `_refresh_all_services` / Temporal init in `main.py`) + emit `workflow_lifecycle("imported")` per example so the sidebar refreshes. Secondary: negative cache in `AuthService.has_valid_key` (validation does one credentials-DB read per declared credential per node). |
| Cold-boot lifespan I/O (9.6 s measured 2026-07-14 vs 1.5 s pre-fix cold boot) | unclear — needs a re-run | Fresh-DB creation 4.4 s + salt/PBKDF2 + encryption ~3 s under Defender/disk contention now that the whole boot compresses into ~20 s. May be noise; measure before optimising. |
| Plugin walker lazy-loading | ~0.5-0.8 s on server-ready | Would need to defer registration until first NodeSpec request rather than at module-import time. Touches every `BaseNode` subclass — biggest blast radius of the candidates. |
-| `BaseMessage`-free agent loop (drop the only eager `langchain_core.messages` import in `services/ai.py`) | ~0.3-0.5 s | Native-SDK provider classes already use their own message types; `_run_agent_loop` could be retyped against them and the eager import dropped. |
+| Retire legacy Temporal history adapters after the rollback / retention window | negligible startup impact | New executions already use native messages and the compatibility imports are lazy. This is dependency and maintenance cleanup, not a measured cold-start win. |
| `+5 s` HTTP-ready → first-WS-connect gap | up to 5 s | Diagnostics needed (see "remaining +5 s gap"). May reveal nothing actionable. |
| Supervisor backend `ready_timeout` (default 30 s, shortest of the three services) | cosmetic | The probe is inert (one-shot, no restart/gating) but a >30 s boot prints an alarming "timed out waiting for port 3010" and skips the ready line. Post-fix boots fit the window; revisit only if cold boots regress past 30 s. |
| Standalone Nuitka / PyOxidizer release binary | full Python interpreter init (~0.4 s) + `.pyc` regeneration on cold disk | User explicitly declined when scoping the build pipeline; revisit if "ship a single binary" becomes a product requirement. |
diff --git a/docs-internal/plugin_system.md b/docs-internal/plugin_system.md
index 7dd37ac8..eb71f2a7 100644
--- a/docs-internal/plugin_system.md
+++ b/docs-internal/plugin_system.md
@@ -105,7 +105,7 @@ Practical rules for plugin authors:
1. Prefer returning the `Output` model instance (`return ExampleOutput(...)`). Dicts work but get validated.
2. `Output` models declare `extra="allow"` + all-`Optional` fields by convention — extra context keys pass through; only type mismatches fail.
-3. Never put raw third-party objects (dataclasses like deepagents' `ReadResult`, SDK response objects) into the result — unwrap to plain fields. Everything downstream (JSON column, orjson WS broadcast, `_serialise_tool_result`) expects JSON-compatible data.
+3. Never put raw backend or SDK objects (for example a native filesystem `ReadResult` or provider response object) into the result — unwrap to plain fields. Everything downstream (JSON column, orjson WS broadcast, `_serialise_tool_result`) expects JSON-compatible data.
4. LLM-facing string formatting is the dispatcher's job (`_serialise_tool_result` JSON-dumps the whole payload) — return real lists/dicts, not pre-stringified JSON.
5. Params fields that may receive LLM-stringified JSON (Gemini sometimes stringifies array/object args in tool calls) coerce at the boundary with `field_validator(..., mode="before")` — see `AndroidServiceParams._coerce_parameters` and `WriteTodosParams._coerce_todos` for the canonical shape.
@@ -647,7 +647,7 @@ server/
│ │ ├── _inline.py # prepare_agent_call()
│ │ ├── _specialized.py # SpecializedAgentBase
│ │ └── <agent>/__init__.py # one folder per agent
-│ ├── model/ # AI chat models (11 providers; xai native-chat-only)
+│ ├── model/ # AI chat models (12 providers; all agent-capable)
│ │ ├── _base.py # ChatModelBase + ChatModelParams/Output
│ │ └── <provider>_chat_model/__init__.py
│ ├── android/ # 16 Android service nodes
@@ -658,7 +658,7 @@ server/
│ │ ├── _nodejs.py # Shared NodeJSClient singleton
│ │ └── <lang>_executor/__init__.py
│ ├── filesystem/ # file_read / file_modify / shell / fs_search
-│ │ ├── _backend.py # Shared LocalShellBackend helper
+│ │ ├── _backend.py # Native workspace-contained filesystem helper
│ │ └── <op>/__init__.py
│ ├── document/ # http_scraper / parser / chunker / embedding / vector
│ │ ├── _helpers.py # delegate() wrapper
diff --git a/docs-internal/tool_building_pipeline.md b/docs-internal/tool_building_pipeline.md
index e1158b0d..9d8a24d0 100644
--- a/docs-internal/tool_building_pipeline.md
+++ b/docs-internal/tool_building_pipeline.md
@@ -1,9 +1,13 @@
# Tool Building Pipeline
-Canonical home for how AI Agents discover connected tool nodes, build LangChain `StructuredTool` instances, bind them to the LLM, and dispatch tool calls back to the right handler. Replaces the partial explanations previously scattered across `agent_architecture.md` and `agent_delegation.md`.
+Canonical home for how AI Agents discover connected tool nodes, build
+provider-neutral `AgentToolSpec` / `ToolDef` values, expose them through the
+native `ChatUnifier`, and dispatch tool calls back to the right handler.
+Replaces the partial explanations previously scattered across
+`agent_architecture.md` and `agent_delegation.md`.
> **Related docs:**
-> - [agent_architecture.md](./agent_architecture.md) — overall agent loop that holds the bound tools
+> - [agent_architecture.md](./agent_architecture.md) — overall agent loop that owns the current tool surface
> - [agent_delegation.md](./agent_delegation.md) — `delegate_to_*` tools + taskTrigger / input-task patterns
> - [TEMPORAL_ARCHITECTURE.md](./TEMPORAL_ARCHITECTURE.md) — per-type Temporal activity dispatch (F4.A) that backs tool calls when the flag is on
> - [node_creation.md](./node_creation.md) — recipe for adding a new tool node
@@ -16,11 +20,11 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain
│
▼
2. BUILD services/ai.py:AIService._build_tool_from_node
- → tool_data entry → (StructuredTool, config_dict)
+ → tool_data entry → (AgentToolSpec, config_dict)
│
▼
-3. BIND services/ai.py:_run_agent_loop
- → chat_model.bind_tools(tools)
+3. EXPOSE services/agent_runtime.py:run_native_agent_loop
+ → ToolDef list → ChatUnifier → provider schema compiler
│
▼
4. INVOKE LLM emits tool_calls in its response
@@ -34,7 +38,7 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain
## 2. Stage 1 — Discovery (plugin/edge_walker.py)
-[`collect_agent_connections(node_id, context, database)`](../server/services/plugin/edge_walker.py) walks the workflow edges and returns a 4-tuple per agent:
+[`collect_agent_connections(node_id, context, database)`](../server/services/plugin/edge_walker.py) walks the workflow edges and returns a 5-tuple per agent:
```
(memory_data, skill_data, tool_data, input_data, task_data)
@@ -49,25 +53,28 @@ Canonical home for how AI Agents discover connected tool nodes, build LangChain
## 3. Stage 2 — Build (services/ai.py:_build_tool_from_node)
-[`AIService._build_tool_from_node(tool_info)`](../server/services/ai.py#L2609) at line 2609 returns `(StructuredTool, config_dict)` or `(None, None)` on failure.
+[`AIService._build_tool_from_node(tool_info)`](../server/services/ai.py)
+returns `(AgentToolSpec, config_dict)` or `(None, None)` on failure.
### 3a. Tool name resolution
-The hardcoded `DEFAULT_TOOL_NAMES` map at [`services/ai.py:2622-2691`](../server/services/ai.py#L2622) maps `node_type` → LLM-visible tool name:
+The LLM-visible name and description come from the plugin rather than a
+central map:
-| `node_type` | Tool name | Category |
-|---|---|---|
-| `calculatorTool` | `calculator` | Dedicated tool |
-| `currentTimeTool` | `get_current_time` | Dedicated tool |
-| `duckduckgoSearch` | `web_search` | Dedicated tool |
-| `pythonExecutor` | `python_code` | Dual-purpose |
-| `whatsappSend` | `whatsapp_send` | Dual-purpose |
-| `whatsappDb` | `whatsapp_db` | Dual-purpose |
-| `aiAgent` → `delegate_to_ai_agent` | (18 agent types total) | Agent delegation |
-| `batteryMonitor` → `android_battery` | (16 service types) | Direct Android service |
-| ... | | |
-
-Adding a new tool type requires adding a row here. The frontend never reads this map — it's a backend-internal mapping.
+1. A per-node `delegate_tool_name` wins for expanded teammate/delegation
+ identities.
+2. A database `ToolSchema` override wins when present.
+3. Otherwise declared node parameters (`tool_name` / `tool_description`) may
+ override the defaults.
+4. Plugin class variables `tool_name` / `tool_description` provide the normal
+ default; `tool_description` falls back to the plugin's human-facing
+ `description`.
+5. Built-in pseudo-tools such as `Skill` and `check_delegated_tasks` use the
+ small `_PSEUDO_TOOL_FALLBACK` table.
+6. The last resort is `tool_<label>` / `Execute <label>`.
+
+The resolved name is normalized to alphanumeric characters and underscores.
+Duplicate visible names are rejected before the first billed model call.
### 3b. Schema resolution
@@ -75,57 +82,77 @@ Adding a new tool type requires adding a row here. The frontend never reads this
schema = self._get_tool_schema(node_type, schema_params)
```
-[`_get_tool_schema`](../server/services/ai.py#L2888) at line 2888 returns a Pydantic `BaseModel` subclass.
-
-Two paths:
+[`_get_tool_schema`](../server/services/ai.py) returns a Pydantic `BaseModel`
+subclass.
| Path | Source | When |
|---|---|---|
| **Database override** (preferred) | `ToolSchema` table (`server/models/database.py`) via `db.get_tool_schema(node_id)` | User customized the schema in the Tool Schema Editor UI |
-| **Dynamic generation** (fallback) | Hardcoded per-type schemas inside `_get_tool_schema` | No DB override; default behavior |
+| **Plugin schema** (normal) | Registered plugin class's Pydantic `Params` model | No DB override |
+| **Special built-in schema** | Delegation, `Skill`, and `check_delegated_tasks` definitions in `_get_tool_schema` | The LLM invocation contract intentionally differs from a node's configuration model, or no plugin exists |
+| **Generic fallback** | One required string `input` field | Unknown non-plugin node type |
`ToolSchema.schema_config` is JSON of `{description, fields: {name: {type, description, required}}}`. CRUD handlers in [`routers/websocket.py`](../server/routers/websocket.py): `get_tool_schema`, `save_tool_schema`, `delete_tool_schema`, `get_all_tool_schemas`.
-### 3c. StructuredTool construction
+### 3c. `AgentToolSpec` construction
```python
-tool = StructuredTool.from_function(
- name=tool_name,
- description=description,
- args_schema=schema,
- func=_make_callback(node_id, node_type, ...), # closure → execute_tool
- coroutine=_make_callback_async(...),
-)
config = {
"node_id": node_id,
"node_type": node_type,
- "workflow_id": workflow_id,
- "parameters": parameters,
- "ai_service": self,
- "database": database,
- "nodes": nodes,
- "edges": edges,
- # ... per-tool extras
+ "parameters": node_params,
+ "label": node_label,
+ "connected_services": connected_services,
}
+parameters = inline_schema_refs(schema.model_json_schema())
+tool = AgentToolSpec(
+ definition=ToolDef(
+ name=tool_name,
+ description=tool_description,
+ parameters=parameters,
+ ),
+ args_schema=schema,
+ execution=config,
+)
```
-The returned `config` is threaded down to `execute_tool` so handlers can read injected services + graph context without going through globals.
+`ToolDef` is the provider-facing declaration. `AgentToolSpec` retains the
+Pydantic validation model and local execution metadata. The returned `config`
+is augmented by the agent's callback with services and graph context before it
+reaches `execute_tool`.
-## 4. Stage 3 — Bind (`_run_agent_loop`)
+## 4. Stage 3 — Expose (`run_native_agent_loop`)
-[`services/ai.py:execute_agent`](../server/services/ai.py) calls `_run_agent_loop(chat_model, ...)`, which does `chat_model.bind_tools(tools)` once at the top before the iteration loop. `bind_tools` reads each tool's `args_schema` to render a JSON Schema in the LLM's tool-use prompt.
+[`services/ai.py:execute_agent`](../server/services/ai.py) calls
+`run_native_agent_loop(ChatUnifier, ..., tools=tools)`. On each LLM step,
+`run_native_llm_step` passes only the `ToolDef` declarations to
+`ChatUnifier.chat`. The selected native provider compiles
+`ToolDef.parameters` into its SDK's tool schema; there is no mutable
+chat-model binding object.
-Per-provider quirks live in the chat model class, not here — see [native_llm_sdk.md](./native_llm_sdk.md) for the OpenAI / Anthropic / Gemini differences.
+Provider-specific handling for nullable values, unions, enums, nested
+objects, and unsupported keywords remains inside the native provider layer.
+See [native_llm_sdk.md](./native_llm_sdk.md) for the OpenAI / Anthropic /
+Gemini differences.
## 5. Stage 4 — Invoke (LLM response handling)
-When the LLM emits `tool_calls` in its response, `_run_agent_loop` iterates them and dispatches each via the supplied `tool_executor` callback — the `_make_callback_async` closure created in stage 2 — and inside that closure:
+When the native `LLMResponse` contains tool calls,
+`run_native_agent_loop` first appends the exact replayable
+`assistant_message`, then validates and dispatches each call through the
+supplied `tool_executor` callback. Inside the callback:
1. Broadcast `executing_tool` to the parent agent (phase broadcast, not the tool node's lifecycle).
2. `from services.handlers.tools import execute_tool`
3. `result = await execute_tool(tool_name, tool_args, config)` — this owns the tool node's lifecycle.
4. Broadcast `tool_completed` to the parent agent.
-5. Return `result` to the loop → wrapped as a `ToolMessage`, appended to `messages`, fed back to the LLM on the next turn.
+5. Return `result` to the loop → serialized into a native
+ `Message(role="tool", tool_call_id=...)`, appended to `messages`, and fed
+ back to the LLM on the next turn.
+
+Malformed JSON arguments retain `raw_arguments` plus a parse error. The loop
+returns a deterministic tool-error result to the model instead of invoking the
+tool or crashing.
**Single-source-of-truth rule.** `execute_tool` owns `executing` / `success` / `error` broadcasts for the **tool node**. The parent-agent closure in `services/ai.py` only emits `executing_tool` / `tool_completed` phase events for the **parent agent**. Don't duplicate either.
@@ -157,7 +184,16 @@ When the LLM emits `tool_calls` in its response, `_run_agent_loop` iterates them
## 7. Gateway-tool pattern (retired)
-`androidTool` was the only **gateway tool** — a single `StructuredTool` the LLM saw as `android_device` that fanned out to multiple connected Android service nodes via a `connected_services` list. The node no longer exists: Android service tools are the 16 individual entries in `DEFAULT_TOOL_NAMES`, connected directly to `input-tools`. Legacy `service -> androidTool -> agent` graphs are rewritten on load by `services/workflow_migrations.normalize_legacy_android_toolkit`, and sub-node exclusion keys solely on the AI-agent config handles (`input-memory` / `input-tools` / `input-skill` / `input-teammates`) — the `TOOLKIT_NODE_TYPES` constant is gone.
+`androidTool` was the only **gateway tool** — a single LLM-visible
+`android_device` declaration that fanned out to multiple connected Android
+service nodes via a `connected_services` list. The node no longer exists:
+Android service plugins expose their own `tool_name` class variables and
+connect directly to `input-tools`. Legacy
+`service -> androidTool -> agent` graphs are rewritten on load by
+`services/workflow_migrations.normalize_legacy_android_toolkit`, and sub-node
+exclusion keys solely on the AI-agent config handles (`input-memory` /
+`input-tools` / `input-skill` / `input-teammates`) — the
+`TOOLKIT_NODE_TYPES` constant is gone.
## 8. Internal delegation identities (`delegate_to_*`)
@@ -188,13 +224,28 @@ Parent agents see results via three paths — see [agent_delegation.md](./agent_
When `TEMPORAL_PER_TYPE_DISPATCH=true` and the run is happening inside a Temporal workflow, tool calls do NOT go through `execute_tool` directly. Instead:
-1. `AgentWorkflow` (the F4.B child workflow) gets the LLM's tool_calls list back from `agent.execute_llm_step.v1`.
-2. For each tool call, the workflow schedules `f"node.{node_type}.v{version}"` as a Temporal activity on the plugin's declared `task_queue`.
-3. The per-type activity body in [`server/services/plugin/base.py:as_activity`](../server/services/plugin/base.py) runs the same pipeline `execute_tool` would have — broadcasts, plugin `execute()` call, terminal broadcast — but inside the activity boundary.
-4. Result returns to the workflow, gets appended to the messages history, next LLM step sees it.
+1. `agent.prepare_payload.v1` records `llm_engine="native"` and
+ `message_wire_version=2` for new executions.
+2. `AgentWorkflow` (the F4.B child workflow) gets the LLM's tool-calls list
+ back from `agent.execute_llm_step.v1`, whose native branch sends `ToolDef`
+ declarations through `ChatUnifier`.
+3. For each tool call, the workflow schedules
+ `f"node.{node_type}.v{version}"` as a Temporal activity on the plugin's
+ declared `task_queue`.
+4. The per-type activity body in
+ [`server/services/plugin/base.py:as_activity`](../server/services/plugin/base.py)
+ runs the same pipeline `execute_tool` would have — broadcasts, plugin
+ `execute()` call, terminal broadcast — but inside the activity boundary.
+5. The result returns to the workflow, is appended to the native message
+ history, and the next LLM step sees it.
This means tool calls inside an agent loop get Temporal's retry / timeout / heartbeat semantics independently of the parent agent's. A `code-exec` task burns its own retries; a `browser` task survives past the parent's `start_to_close_timeout` via its own heartbeat. See [TEMPORAL_ARCHITECTURE.md](./TEMPORAL_ARCHITECTURE.md).
+Histories recorded before the engine marker existed deterministically replay
+the frozen LangChain branch, which rebuilds the historical `StructuredTool`
+surface. That compatibility branch is selected only by the missing
+`llm_engine` marker; it is not the path for new executions.
+
## 10. Auto-skill edges
Some tools bundle a default skill — `writeTodos` ships with `write-todos-skill`, the WhatsApp tools ship with their respective skills. The frontend `useAutoSkillEdges.ts` hook detects these connections at canvas-edit time and auto-creates a phantom skill connection so the LLM gets both the tool schema (for invocation) and the skill instructions (for usage guidance) without the user having to wire two edges.
@@ -219,18 +270,26 @@ The shortest path:
1. Create the plugin folder: `server/nodes/tool/<plugin>/__init__.py` extending `ToolNode` (or `ActionNode` with `usable_as_tool = True` for dual-purpose).
2. Declare `type`, `display_name`, `Params` (the schema the LLM sees), `Output`, `@Operation`-decorated `execute()`.
-3. Add a row to `DEFAULT_TOOL_NAMES` in `services/ai.py:2622-2691` — `node_type` → LLM-visible tool name.
-4. (Optional) Add a clause to `_get_tool_schema` if the dynamic-generation default isn't what you want.
-5. (Optional) Add a dispatch clause to `_dispatch_tool` if your plugin needs custom handler logic; otherwise the generic per-plugin path runs `cls.execute()` directly.
+3. Declare the plugin's `tool_name` and, when the LLM-facing wording differs,
+ `tool_description` class variables.
+4. (Optional) Add a special schema clause to `_get_tool_schema` only when the
+ plugin's `Params` model is not the correct invocation contract.
+5. (Optional) Add a dispatch clause to `_dispatch_tool` if your plugin needs
+ custom handler logic; otherwise the generic per-plugin path runs
+ `cls.execute()` directly.
-Steps 3-5 are the only edits OUTSIDE the plugin folder. Steps 4-5 are usually unnecessary — most new tools use the dynamic default.
+Steps 4-5 are usually unnecessary — most new tools are fully described by
+their plugin class.
See [node_creation.md](./node_creation.md) for the broader recipe and [server/nodes/README.md](../server/nodes/README.md) for the 5-minute walkthrough.
## 13. What NOT to do
- Don't broadcast `executing` / `success` / `error` from inside `_dispatch_tool` or a handler — `execute_tool` owns the tool node's lifecycle. Returning `{"status": "delegated"}` is the documented opt-out.
-- Don't bypass `_build_tool_from_node` to hand-craft a `StructuredTool`. The closure shape and `config` dict contents are load-bearing — agents inject services through them.
-- Don't add a new entry to `DEFAULT_TOOL_NAMES` without also handling the schema generation. The tool will register but the LLM will see `{}` for the schema and hallucinate arguments.
+- Don't bypass `_build_tool_from_node` to hand-craft an `AgentToolSpec` or
+ `ToolDef`. Schema dereferencing, validation metadata, execution routing, and
+ duplicate-name checks are load-bearing.
+- Don't expose a tool without a real Pydantic invocation schema. An empty
+ schema makes argument generation unreliable.
- Don't catch exceptions inside the tool callback and return them as `{"error": ...}` — re-raise so `execute_tool` can broadcast `error`. The LLM-sees-tool-errors-and-continues pattern depends on this.
- Don't store per-execution state on `AIService` (it's a singleton). Use `config` or per-call closures.
diff --git a/server/main.py b/server/main.py
index 08efde7f..1267976b 100644
--- a/server/main.py
+++ b/server/main.py
@@ -532,6 +532,8 @@ async def lifespan(app: FastAPI):
except Exception as exc:
logger.debug("[CLI MCP] lifespan shutdown: %s", exc)
+ # Drain cached native SDK clients before credentials/database teardown.
+ await container.chat_unifier().aclose()
await container.cache().shutdown()
await container.database().shutdown()
logger.info("Services shutdown complete")
diff --git a/server/nodes/README.md b/server/nodes/README.md
index 79cd6789..697e2a8e 100644
--- a/server/nodes/README.md
+++ b/server/nodes/README.md
@@ -368,8 +368,8 @@ Full reference: [docs-internal/plugin_system.md → "Self-contained plugin folde
Pydantic models or `Union`, the LLM-schema emission will add `$defs`
and fail the invariant. Keep tool-facing Params flat; move nested
types to `Output` instead.
-- **Return plain data, never raw third-party objects.** Backend / SDK
- result objects (deepagents `ReadResult` / `WriteResult`, httpx
+- **Return plain data, never raw backend or third-party objects.** Backend /
+ SDK result objects (native filesystem `ReadResult`, httpx
responses, …) must be unwrapped to plain fields before returning —
the Output validation rejects them, and everything downstream
(`node_outputs` JSON column, orjson WS broadcast,
diff --git a/server/nodes/agent/_specialized.py b/server/nodes/agent/_specialized.py
index 5f6f6364..b31f89e3 100644
--- a/server/nodes/agent/_specialized.py
+++ b/server/nodes/agent/_specialized.py
@@ -40,6 +40,7 @@ class SpecializedAgentParams(BaseModel):
"anthropic",
"gemini",
"openrouter",
+ "xai",
"groq",
"cerebras",
"deepseek",
diff --git a/server/nodes/agent/ai_agent/__init__.py b/server/nodes/agent/ai_agent/__init__.py
index f09113ca..faa64ec3 100644
--- a/server/nodes/agent/ai_agent/__init__.py
+++ b/server/nodes/agent/ai_agent/__init__.py
@@ -43,13 +43,14 @@ class AIAgentParams(BaseModel):
"anthropic",
"gemini",
"openrouter",
+ "xai",
"groq",
"cerebras",
"deepseek",
"kimi",
"mistral",
# Local-server providers — agent execution reads
- # ``{provider}_proxy`` to point the LangChain client at the
+ # ``{provider}_proxy`` to point the native OpenAI client at the
# user's localhost server. Without these entries the dropdown
# silently falls back to ``"openai"`` and execute_agent ends
# up calling api.openai.com instead.
diff --git a/server/nodes/agent/chat_agent/__init__.py b/server/nodes/agent/chat_agent/__init__.py
index b91baccc..9b802442 100644
--- a/server/nodes/agent/chat_agent/__init__.py
+++ b/server/nodes/agent/chat_agent/__init__.py
@@ -30,6 +30,7 @@ class ChatAgentParams(BaseModel):
"anthropic",
"gemini",
"openrouter",
+ "xai",
"groq",
"cerebras",
"deepseek",
diff --git a/server/nodes/agent/vertex_managed_agent/__init__.py b/server/nodes/agent/vertex_managed_agent/__init__.py
index ae0d03ad..96c40707 100644
--- a/server/nodes/agent/vertex_managed_agent/__init__.py
+++ b/server/nodes/agent/vertex_managed_agent/__init__.py
@@ -569,7 +569,13 @@ class VertexManagedAgentNode(ActionNode):
configs: Dict[str, Dict[str, Any]] = {}
def _declare(structured: Any, config: Optional[Dict[str, Any]]) -> None:
- if structured.args_schema is not None:
+ # Native ``AgentToolSpec`` already carries the canonical, locally
+ # resolved ToolDef schema. Keep the args_schema fallback only for
+ # older test doubles and pre-cutover compatibility callers.
+ schema = getattr(structured, "parameters", None)
+ if schema is not None:
+ schema = inline_schema_refs(schema)
+ elif getattr(structured, "args_schema", None) is not None:
# Function-calling APIs reject schema indirection —
# inline $defs/$ref (nested BaseModel / Enum Params
# fields) instead of stripping and leaving dangling refs.
diff --git a/server/nodes/document/embedding_generator/__init__.py b/server/nodes/document/embedding_generator/__init__.py
index 783c33f3..72af71f9 100644
--- a/server/nodes/document/embedding_generator/__init__.py
+++ b/server/nodes/document/embedding_generator/__init__.py
@@ -2,11 +2,14 @@
from __future__ import annotations
-import asyncio
from typing import List, Literal, Optional
from pydantic import BaseModel, ConfigDict, Field
+from services.memory.vector_store import (
+ EmbeddingError,
+ create_embedder,
+)
from services.plugin import ActionNode, NodeContext, NodeUserError, Operation, TaskQueue
@@ -34,6 +37,18 @@ class EmbeddingGeneratorParams(BaseModel):
"displayOptions": {"show": {"provider": ["openai"]}},
},
)
+ endpoint: str = Field(
+ default="",
+ description=(
+ "Optional provider endpoint. OpenAI uses it as base_url; "
+ "Ollama uses it as host."
+ ),
+ json_schema_extra={
+ "displayOptions": {
+ "show": {"provider": ["openai", "ollama"]}
+ }
+ },
+ )
model_config = ConfigDict(extra="ignore")
@@ -84,27 +99,24 @@ class EmbeddingGeneratorNode(ActionNode):
)
texts = [c.get("content", "") if isinstance(c, dict) else str(c) for c in chunks]
+ embedder = None
+ try:
+ embedder = create_embedder(
+ provider,
+ model=model,
+ api_key=api_key,
+ endpoint=params.endpoint,
+ )
+ embeddings = await embedder.embed_documents(
+ texts,
+ batch_size=params.batch_size,
+ )
+ except (EmbeddingError, ValueError) as exc:
+ raise NodeUserError(str(exc)) from exc
+ finally:
+ if embedder is not None:
+ await embedder.aclose()
- if provider == "huggingface":
- try:
- from langchain_huggingface import HuggingFaceEmbeddings
- except ImportError:
- raise NodeUserError(
- "HuggingFace embeddings unavailable. " "pip install langchain-huggingface sentence-transformers",
- )
- embedder = HuggingFaceEmbeddings(model_name=model)
- elif provider == "openai":
- from langchain_openai import OpenAIEmbeddings
-
- embedder = OpenAIEmbeddings(model=model, api_key=api_key)
- elif provider == "ollama":
- from langchain_ollama import OllamaEmbeddings
-
- embedder = OllamaEmbeddings(model=model)
- else:
- raise NodeUserError(f"Unknown provider: {provider}")
-
- embeddings = await asyncio.to_thread(embedder.embed_documents, texts)
dimensions = len(embeddings[0]) if embeddings else 0
return EmbeddingGeneratorOutput(
diff --git a/server/nodes/document/text_chunker/__init__.py b/server/nodes/document/text_chunker/__init__.py
index 8ff98bd1..1269340c 100644
--- a/server/nodes/document/text_chunker/__init__.py
+++ b/server/nodes/document/text_chunker/__init__.py
@@ -44,10 +44,7 @@ class TextChunkerNode(ActionNode):
@Operation("chunk")
async def chunk(self, ctx: NodeContext, params: TextChunkerParams) -> TextChunkerOutput:
- from langchain_text_splitters import (
- MarkdownTextSplitter,
- RecursiveCharacterTextSplitter,
- )
+ from ._splitter import split_text
p = params.model_dump()
documents = p.get("documents") or ([{"content": p.get("text", "")}] if p.get("text") else [])
@@ -58,18 +55,20 @@ class TextChunkerNode(ActionNode):
if not documents:
return TextChunkerOutput(chunks=[], chunk_count=0)
- if strategy == "markdown":
- splitter = MarkdownTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)
- else:
- splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)
-
chunks = []
for doc in documents:
content = doc.get("content", "") if isinstance(doc, dict) else str(doc)
source = doc.get("source", "input") if isinstance(doc, dict) else "input"
if not content:
continue
- for i, chunk_text in enumerate(splitter.split_text(content)):
+ for i, chunk_text in enumerate(
+ split_text(
+ content,
+ chunk_size=chunk_size,
+ overlap=chunk_overlap,
+ markdown=strategy == "markdown",
+ )
+ ):
chunks.append(
{
"source": source,
diff --git a/server/nodes/document/text_chunker/_splitter.py b/server/nodes/document/text_chunker/_splitter.py
new file mode 100644
index 00000000..d2cf93bb
--- /dev/null
+++ b/server/nodes/document/text_chunker/_splitter.py
@@ -0,0 +1,158 @@
+"""Dependency-free character splitters used by the text chunker node."""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Iterable, Sequence
+
+
+RECURSIVE_SEPARATORS = ("\n\n", "\n", " ", "")
+MARKDOWN_SEPARATORS = (
+ "\n#{1,6} ",
+ "```\n",
+ "\n\\*\\*\\*+\n",
+ "\n---+\n",
+ "\n___+\n",
+ "\n\n",
+ "\n",
+ " ",
+ "",
+)
+
+
+def _split_with_separator(text: str, separator: str) -> list[str]:
+ """Split while retaining separators at the start of the next fragment."""
+ if not separator:
+ return list(text)
+ pieces = re.split(f"({re.escape(separator)})", text)
+ splits = [pieces[i] + pieces[i + 1] for i in range(1, len(pieces), 2)]
+ if len(pieces) % 2 == 0:
+ splits += pieces[-1:]
+ splits = [pieces[0], *splits]
+ return [item for item in splits if item]
+
+
+def _join(parts: Sequence[str], separator: str) -> str | None:
+ text = separator.join(parts).strip()
+ return text or None
+
+
+def _merge(
+ splits: Iterable[str],
+ *,
+ chunk_size: int,
+ overlap: int,
+ separator: str,
+) -> list[str]:
+ separator_len = len(separator)
+ chunks: list[str] = []
+ current: list[str] = []
+ total = 0
+ for split in splits:
+ split_len = len(split)
+ projected = total + split_len + (separator_len if current else 0)
+ if projected > chunk_size and current:
+ chunk = _join(current, separator)
+ if chunk is not None:
+ chunks.append(chunk)
+ while total > overlap or (
+ total
+ + split_len
+ + (separator_len if current else 0)
+ > chunk_size
+ and total > 0
+ ):
+ total -= len(current[0]) + (
+ separator_len if len(current) > 1 else 0
+ )
+ current = current[1:]
+ current.append(split)
+ total += split_len + (separator_len if len(current) > 1 else 0)
+ chunk = _join(current, separator)
+ if chunk is not None:
+ chunks.append(chunk)
+ return chunks
+
+
+def _recursive_split(
+ text: str,
+ separators: Sequence[str],
+ *,
+ chunk_size: int,
+ overlap: int,
+) -> list[str]:
+ separator = separators[-1]
+ remaining: Sequence[str] = ()
+ for index, candidate in enumerate(separators):
+ if not candidate:
+ separator = candidate
+ break
+ if re.search(re.escape(candidate), text):
+ separator = candidate
+ remaining = separators[index + 1 :]
+ break
+
+ splits = _split_with_separator(text, separator)
+ final: list[str] = []
+ small: list[str] = []
+ for split in splits:
+ if len(split) < chunk_size:
+ small.append(split)
+ continue
+ if small:
+ final.extend(
+ _merge(
+ small,
+ chunk_size=chunk_size,
+ overlap=overlap,
+ separator="",
+ )
+ )
+ small = []
+ if remaining:
+ final.extend(
+ _recursive_split(
+ split,
+ remaining,
+ chunk_size=chunk_size,
+ overlap=overlap,
+ )
+ )
+ else:
+ final.append(split)
+ if small:
+ final.extend(
+ _merge(
+ small,
+ chunk_size=chunk_size,
+ overlap=overlap,
+ separator="",
+ )
+ )
+ return final
+
+
+def split_text(
+ text: str,
+ *,
+ chunk_size: int,
+ overlap: int,
+ markdown: bool = False,
+) -> list[str]:
+ """Split text with the historical recursive-character semantics."""
+ if chunk_size <= 0:
+ raise ValueError(f"chunk_size must be > 0, got {chunk_size}")
+ if overlap < 0:
+ raise ValueError(f"chunk_overlap must be >= 0, got {overlap}")
+ if overlap > chunk_size:
+ raise ValueError(
+ f"Got a larger chunk overlap ({overlap}) than chunk size "
+ f"({chunk_size}), should be smaller."
+ )
+ separators = MARKDOWN_SEPARATORS if markdown else RECURSIVE_SEPARATORS
+ return _recursive_split(
+ text,
+ separators,
+ chunk_size=chunk_size,
+ overlap=overlap,
+ )
diff --git a/server/nodes/filesystem/_backend.py b/server/nodes/filesystem/_backend.py
index 31cf4b6c..290423b1 100644
--- a/server/nodes/filesystem/_backend.py
+++ b/server/nodes/filesystem/_backend.py
@@ -1,32 +1,45 @@
-"""Shared deepagents-backend helper for filesystem plugins.
-
-All four filesystem plugins (file_read / file_modify / shell /
-fs_search) open a :class:`deepagents.backends.LocalShellBackend`
-rooted at the per-workflow workspace. This helper centralises that
-setup so each plugin file stays small.
-
-Shell execution is dispatched through Nushell when available so the
-agent sees identical command semantics on Windows, macOS, and Linux
-(no cmd.exe vs sh divergence). Falls back to the upstream
-``LocalShellBackend`` (which uses ``shell=True``) when ``nu`` isn't
-installed.
+"""Native workspace filesystem backend shared by filesystem plugins.
+
+Filesystem operations use virtual POSIX paths rooted at the workflow
+workspace. Resolution follows existing symlinks, then verifies that the
+resolved target remains below that root. Shell execution retains the
+historical host-shell behavior and uses Nushell when it is available.
"""
from __future__ import annotations
import asyncio
+import base64
+import fnmatch
import os
+import posixpath
+import re
import shutil
import stat
import subprocess
import tempfile
-from pathlib import PureWindowsPath
+from dataclasses import dataclass
+from datetime import datetime
+from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import Any, Callable, Dict, Optional
+from uuid import uuid4
from weakref import WeakValueDictionary
-from deepagents.backends import LocalShellBackend
-from deepagents.backends.protocol import ExecuteResponse
-from deepagents.backends.utils import validate_path
+
+EMPTY_CONTENT_WARNING = "System reminder: File exists but has empty contents"
+
+
+@dataclass
+class ExecuteResponse:
+ output: str
+ exit_code: Optional[int] = None
+ truncated: bool = False
+
+
+@dataclass
+class ReadResult:
+ error: Optional[str] = None
+ file_data: Optional[Dict[str, Any]] = None
_path_locks: WeakValueDictionary[str, asyncio.Lock] = WeakValueDictionary()
@@ -42,13 +55,13 @@ def get_path_lock(path: os.PathLike[str] | str) -> asyncio.Lock:
return lock
-async def run_sync_until_complete(func: Callable[..., Any], /, *args: Any, **kwargs: Any) -> Any:
- """Run sync filesystem work without abandoning it on cancellation.
-
- Worker threads cannot be cancelled. Keep the caller suspended until the
- mutation finishes so its path lock remains held for the whole operation,
- then propagate the original cancellation.
- """
+async def run_sync_until_complete(
+ func: Callable[..., Any],
+ /,
+ *args: Any,
+ **kwargs: Any,
+) -> Any:
+ """Run sync filesystem work without abandoning it on cancellation."""
worker = asyncio.create_task(asyncio.to_thread(func, *args, **kwargs))
cancellation: Optional[asyncio.CancelledError] = None
@@ -67,211 +80,904 @@ async def run_sync_until_complete(func: Callable[..., Any], /, *args: Any, **kwa
except BaseException:
pass
raise cancellation
-
return worker.result()
-def atomic_write_text(path: os.PathLike[str] | str, content: str) -> None:
- """Replace ``path`` atomically with UTF-8 text from the same directory."""
- resolved = os.path.abspath(os.fspath(path))
- parent = os.path.dirname(resolved)
- os.makedirs(parent, exist_ok=True)
+def _file_identity(status: os.stat_result) -> tuple[int, int]:
+ return int(status.st_dev), int(status.st_ino)
+
+
+def _is_reparse_point(status: os.stat_result) -> bool:
+ attributes = int(getattr(status, "st_file_attributes", 0))
+ reparse_flag = int(getattr(stat, "FILE_ATTRIBUTE_REPARSE_POINT", 0))
+ return stat.S_ISLNK(status.st_mode) or bool(attributes & reparse_flag)
+
+
+def _anchored_write_supported() -> bool:
+ return (
+ os.name != "nt"
+ and hasattr(os, "O_DIRECTORY")
+ and os.open in os.supports_dir_fd
+ and os.mkdir in os.supports_dir_fd
+ and os.rename in os.supports_dir_fd
+ and os.stat in os.supports_dir_fd
+ and os.unlink in os.supports_dir_fd
+ )
+
+
+def _open_posix_directory(
+ root: Path,
+ components: tuple[str, ...],
+ *,
+ create: bool,
+) -> int:
+ flags = (
+ os.O_RDONLY
+ | os.O_DIRECTORY
+ | getattr(os, "O_NOFOLLOW", 0)
+ | getattr(os, "O_CLOEXEC", 0)
+ )
+ directory_fd = os.open(root, flags)
+ try:
+ for component in components:
+ try:
+ next_fd = os.open(
+ component,
+ flags,
+ dir_fd=directory_fd,
+ )
+ except FileNotFoundError:
+ if not create:
+ raise
+ os.mkdir(component, mode=0o777, dir_fd=directory_fd)
+ next_fd = os.open(
+ component,
+ flags,
+ dir_fd=directory_fd,
+ )
+ os.close(directory_fd)
+ directory_fd = next_fd
+ return directory_fd
+ except BaseException:
+ os.close(directory_fd)
+ raise
+
+
+def _verify_posix_directory(
+ root: Path,
+ components: tuple[str, ...],
+ expected_fd: int,
+) -> None:
+ current_fd = _open_posix_directory(root, components, create=False)
+ try:
+ if _file_identity(os.fstat(current_fd)) != _file_identity(
+ os.fstat(expected_fd)
+ ):
+ raise ValueError(
+ "Workspace parent directory changed during atomic write"
+ )
+ finally:
+ os.close(current_fd)
+
+
+def _write_utf8_to_fd(
+ file_fd: int,
+ content: str,
+ existing_mode: Optional[int],
+) -> None:
+ with os.fdopen(file_fd, "w", encoding="utf-8", newline="") as handle:
+ handle.write(content)
+ handle.flush()
+ os.fsync(handle.fileno())
+ if existing_mode is not None and hasattr(os, "fchmod"):
+ os.fchmod(handle.fileno(), existing_mode)
+
+
+def _replace_posix_entry(
+ temporary_name: str,
+ target_name: str,
+ parent_fd: int,
+) -> None:
+ os.rename(
+ temporary_name,
+ target_name,
+ src_dir_fd=parent_fd,
+ dst_dir_fd=parent_fd,
+ )
+
+
+def _atomic_write_text_posix(
+ root: Path,
+ relative: Path,
+ content: str,
+) -> None:
+ parent_components = tuple(relative.parts[:-1])
+ target_name = relative.parts[-1]
+ parent_fd = _open_posix_directory(
+ root,
+ parent_components,
+ create=True,
+ )
+ temporary_name: Optional[str] = None
+ try:
+ existing_mode: Optional[int] = None
+ try:
+ target_status = os.stat(
+ target_name,
+ dir_fd=parent_fd,
+ follow_symlinks=False,
+ )
+ except FileNotFoundError:
+ pass
+ else:
+ if _is_reparse_point(target_status):
+ raise ValueError(
+ f"Refusing to replace symlink target '{relative.as_posix()}'"
+ )
+ if stat.S_ISDIR(target_status.st_mode):
+ raise IsADirectoryError(
+ f"Cannot write to {relative.as_posix()}: path is a directory"
+ )
+ if not stat.S_ISREG(target_status.st_mode):
+ raise ValueError(
+ f"Refusing to replace non-regular file "
+ f"'{relative.as_posix()}'"
+ )
+ existing_mode = stat.S_IMODE(target_status.st_mode)
+
+ temporary_fd: Optional[int] = None
+ for _ in range(16):
+ candidate = f".opencompany-write-{uuid4().hex}"
+ try:
+ temporary_fd = os.open(
+ candidate,
+ (
+ os.O_WRONLY
+ | os.O_CREAT
+ | os.O_EXCL
+ | getattr(os, "O_NOFOLLOW", 0)
+ | getattr(os, "O_CLOEXEC", 0)
+ ),
+ 0o600,
+ dir_fd=parent_fd,
+ )
+ except FileExistsError:
+ continue
+ temporary_name = candidate
+ break
+ if temporary_fd is None or temporary_name is None:
+ raise FileExistsError(
+ "Could not allocate a unique atomic-write temporary file"
+ )
+
+ _write_utf8_to_fd(temporary_fd, content, existing_mode)
+ _verify_posix_directory(root, parent_components, parent_fd)
+ _replace_posix_entry(
+ temporary_name,
+ target_name,
+ parent_fd,
+ )
+ temporary_name = None
+ try:
+ os.fsync(parent_fd)
+ except OSError:
+ pass
+ _verify_posix_directory(root, parent_components, parent_fd)
+ finally:
+ if temporary_name is not None:
+ try:
+ os.unlink(temporary_name, dir_fd=parent_fd)
+ except OSError:
+ pass
+ os.close(parent_fd)
+
+
+def _windows_parent_snapshot(
+ root: Path,
+ components: tuple[str, ...],
+ *,
+ create: bool,
+) -> tuple[Path, tuple[tuple[int, int], ...]]:
+ current = root
+ identities: list[tuple[int, int]] = []
+ for index, component in enumerate(("", *components)):
+ if index:
+ current = current / component
+ try:
+ status = os.stat(current, follow_symlinks=False)
+ except FileNotFoundError:
+ if not create:
+ raise
+ os.mkdir(current)
+ status = os.stat(current, follow_symlinks=False)
+ else:
+ status = os.stat(current, follow_symlinks=False)
+
+ if _is_reparse_point(status):
+ raise ValueError(
+ f"Workspace path contains a symlink or junction: {current}"
+ )
+ if not stat.S_ISDIR(status.st_mode):
+ raise NotADirectoryError(
+ f"Workspace parent component is not a directory: {current}"
+ )
+ identities.append(_file_identity(status))
+
+ try:
+ current.resolve(strict=True).relative_to(root)
+ except (OSError, RuntimeError, ValueError) as exc:
+ raise ValueError(
+ f"Workspace parent resolves outside root: {current}"
+ ) from exc
+ return current, tuple(identities)
+
+
+def _verify_windows_temp(
+ temporary: Path,
+ expected_identity: tuple[int, int],
+) -> None:
+ status = os.stat(temporary, follow_symlinks=False)
+ if _is_reparse_point(status) or _file_identity(status) != expected_identity:
+ raise ValueError(
+ "Atomic-write temporary file changed before replacement"
+ )
+
+
+def _atomic_write_text_windows(
+ root: Path,
+ relative: Path,
+ content: str,
+) -> None:
+ """Best-effort contained atomic replacement for Windows.
+
+ Python exposes no Windows equivalent of POSIX ``dir_fd``-relative rename.
+ This path rejects every observed reparse component and compares parent and
+ temporary-file identities immediately before and after ``os.replace``.
+ A hostile process that can swap the parent between a check and a path-based
+ ``mkdir``/temporary-file operation can cause an empty directory or
+ temporary file to be created outside the workspace; payload bytes are not
+ written until the temporary identity is revalidated. A swap in the final
+ identity-check-to-``os.replace`` interval can redirect the completed
+ replacement; the post-check detects that event but cannot safely roll it
+ back. Python's standard library exposes no handle-relative Windows rename
+ primitive that closes these intervals.
+ """
+
+ components = tuple(relative.parts[:-1])
+ parent, parent_snapshot = _windows_parent_snapshot(
+ root,
+ components,
+ create=True,
+ )
+ target = parent / relative.parts[-1]
existing_mode: Optional[int] = None
try:
- existing_mode = stat.S_IMODE(os.stat(resolved, follow_symlinks=False).st_mode)
- except OSError:
+ target_status = os.stat(target, follow_symlinks=False)
+ except FileNotFoundError:
pass
- fd, temporary = tempfile.mkstemp(prefix=".opencompany-write-", dir=parent)
+ else:
+ if _is_reparse_point(target_status):
+ raise ValueError(
+ f"Refusing to replace symlink or junction target "
+ f"'{relative.as_posix()}'"
+ )
+ if stat.S_ISDIR(target_status.st_mode):
+ raise IsADirectoryError(
+ f"Cannot write to {relative.as_posix()}: path is a directory"
+ )
+ if not stat.S_ISREG(target_status.st_mode):
+ raise ValueError(
+ f"Refusing to replace non-regular file "
+ f"'{relative.as_posix()}'"
+ )
+ existing_mode = stat.S_IMODE(target_status.st_mode)
+
+ temporary_fd, temporary_text = tempfile.mkstemp(
+ prefix=".opencompany-write-",
+ dir=parent,
+ )
+ temporary = Path(temporary_text)
+ temporary_identity = _file_identity(os.fstat(temporary_fd))
+ replaced = False
try:
- # newline="" preserves the caller's bytes instead of translating LF
- # to CRLF on Windows. Preserve an existing file's permission bits
- # before the same-directory atomic replacement where chmod is
- # supported.
- with os.fdopen(fd, "w", encoding="utf-8", newline="") as handle:
- handle.write(content)
- handle.flush()
- os.fsync(handle.fileno())
+ current_parent, current_snapshot = _windows_parent_snapshot(
+ root,
+ components,
+ create=False,
+ )
+ if current_parent != parent or current_snapshot != parent_snapshot:
+ raise ValueError(
+ "Workspace parent directory changed during atomic write"
+ )
+ _verify_windows_temp(temporary, temporary_identity)
+
+ write_fd = temporary_fd
+ temporary_fd = -1
+ _write_utf8_to_fd(write_fd, content, existing_mode)
if existing_mode is not None:
+ # Windows has no ``os.fchmod``. Apply its supported mode bits
+ # (notably the read-only flag) to the still-private temporary
+ # entry, then revalidate that entry below before replacement.
os.chmod(temporary, existing_mode)
- os.replace(temporary, resolved)
- except Exception:
+
+ current_parent, current_snapshot = _windows_parent_snapshot(
+ root,
+ components,
+ create=False,
+ )
+ if current_parent != parent or current_snapshot != parent_snapshot:
+ raise ValueError(
+ "Workspace parent directory changed during atomic write"
+ )
+ _verify_windows_temp(temporary, temporary_identity)
try:
- os.unlink(temporary)
- except OSError:
- pass
- raise
+ current_target = os.stat(target, follow_symlinks=False)
+ except FileNotFoundError:
+ current_target = None
+ if current_target is not None and _is_reparse_point(current_target):
+ raise ValueError(
+ "Atomic-write target became a symlink or junction"
+ )
+
+ os.replace(temporary, target)
+ replaced = True
+
+ current_parent, current_snapshot = _windows_parent_snapshot(
+ root,
+ components,
+ create=False,
+ )
+ if current_parent != parent or current_snapshot != parent_snapshot:
+ raise ValueError(
+ "Workspace parent directory changed during atomic replacement"
+ )
+ replaced_status = os.stat(target, follow_symlinks=False)
+ if (
+ _is_reparse_point(replaced_status)
+ or _file_identity(replaced_status) != temporary_identity
+ ):
+ raise ValueError(
+ "Atomic-write target identity changed during replacement"
+ )
+ finally:
+ if temporary_fd >= 0:
+ os.close(temporary_fd)
+ if not replaced:
+ try:
+ current_parent, current_snapshot = _windows_parent_snapshot(
+ root,
+ components,
+ create=False,
+ )
+ if (
+ current_parent == parent
+ and current_snapshot == parent_snapshot
+ ):
+ _verify_windows_temp(temporary, temporary_identity)
+ os.unlink(temporary)
+ except (OSError, RuntimeError, ValueError):
+ pass
-def normalize_virtual_path(path: str) -> str:
- """Coerce any caller-supplied path to deepagents' canonical virtual form.
-
- LLMs (and humans) emit paths in every flavour: ``/foo`` (POSIX),
- ``C:\\foo`` (Windows drive), ``\\\\server\\share\\foo`` (UNC),
- ``foo\\bar`` / ``foo/bar`` (relative, mixed separators). deepagents'
- ``virtual_mode`` only resolves POSIX virtual paths, and its public
- :func:`validate_path` helper rejects Windows-anchored inputs.
-
- ``PureWindowsPath`` is a pure (host-OS independent) parser and is a
- superset of the POSIX grammar — Windows itself accepts ``/`` as a
- separator, so ``PureWindowsPath('/tmp/foo')`` correctly identifies
- ``/`` as the root anchor. That means a single parser covers Windows
- drives, UNC, and POSIX absolutes uniformly on any host OS. Strip
- the anchor here, then delegate to :func:`validate_path` for traversal
- rejection (``..``, ``~``) and canonical normalisation.
-
- A rejected path is an LLM/user-correctable input error, not a server
- bug — it surfaces as :class:`NodeUserError` (single WARN line, no
- traceback) with guidance the LLM can act on, instead of leaking
- deepagents' bare ``ValueError`` through the generic-exception path.
+def atomic_write_text(
+ path: os.PathLike[str] | str,
+ content: str,
+ *,
+ root_dir: os.PathLike[str] | str | None = None,
+) -> None:
+ """Atomically replace UTF-8 text without traversing mutable symlinks.
+
+ Production callers pass the workspace root. The optional default keeps
+ direct helper callers compatible by treating the existing parent as the
+ root, but it does not establish a broader containment boundary.
"""
+
+ target = Path(os.path.abspath(os.fspath(path)))
+ root = Path(
+ os.path.abspath(
+ os.fspath(root_dir) if root_dir is not None else target.parent
+ )
+ )
+ try:
+ relative = target.relative_to(root)
+ except ValueError as exc:
+ raise ValueError(
+ f"Write target '{target}' is outside workspace root '{root}'"
+ ) from exc
+ if not relative.parts:
+ raise IsADirectoryError(f"Cannot replace workspace root: {target}")
+
+ if _anchored_write_supported():
+ _atomic_write_text_posix(root, relative, content)
+ else:
+ _atomic_write_text_windows(root, relative, content)
+
+
+def _validate_virtual_path(path: str) -> str:
+ normalized_input = str(path).replace("\\", "/")
+ parts = PurePosixPath(normalized_input).parts
+ if ".." in parts or normalized_input.startswith("~"):
+ raise ValueError(f"Path traversal not allowed: {path}")
+ if re.match(r"^[a-zA-Z]:", normalized_input):
+ raise ValueError(f"Windows absolute paths are not supported: {path}")
+ normalized = posixpath.normpath(normalized_input)
+ if not normalized.startswith("/"):
+ normalized = f"/{normalized}"
+ if ".." in normalized.split("/"):
+ raise ValueError(f"Path traversal detected after normalization: {path}")
+ return normalized
+
+
+def normalize_virtual_path(path: str) -> str:
+ """Normalize user paths to canonical workspace-relative virtual paths."""
from services.plugin import NodeUserError
if not path:
return path
- pw = PureWindowsPath(path)
- if pw.drive or pw.root:
- rel = "/" + "/".join(pw.parts[1:]) if len(pw.parts) > 1 else "/"
+ windows_path = PureWindowsPath(path)
+ if windows_path.drive or windows_path.root:
+ relative = (
+ "/" + "/".join(windows_path.parts[1:])
+ if len(windows_path.parts) > 1
+ else "/"
+ )
else:
- rel = path.replace("\\", "/")
+ relative = path.replace("\\", "/")
try:
- return validate_path(rel)
- except ValueError as e:
+ return _validate_virtual_path(relative)
+ except ValueError as exc:
raise NodeUserError(
- f"{e}. The filesystem is sandboxed to the per-workflow workspace "
+ f"{exc}. The filesystem is sandboxed to the per-workflow workspace "
"root — use a path relative to the workspace (e.g. "
"'reports/data.csv'); '..' and '~' segments are rejected. To see "
"what exists, use fs_search with mode='ls'."
- ) from e
+ ) from exc
+
+
+def perform_string_replacement(
+ content: str,
+ old_string: str,
+ new_string: str,
+ replace_all: bool = False,
+) -> tuple[str, int] | str:
+ """Perform exact replacement with ambiguity and EOF-newline diagnostics."""
+ occurrences = content.count(old_string)
+ if occurrences == 0:
+ if (
+ old_string.endswith("\n")
+ and len(old_string) > 1
+ and content.endswith(old_string.removesuffix("\n"))
+ ):
+ stripped = old_string.removesuffix("\n")
+ stripped_count = content.count(stripped)
+ if stripped_count == 1:
+ return (
+ "Error: old_string ends with a newline, but the file does "
+ "not end with a newline. Retry with the trailing newline "
+ "removed from old_string (and from new_string if it also "
+ "ends with a newline)."
+ )
+ return (
+ "Error: old_string ends with a newline, but the file does "
+ "not end with a newline. With the trailing newline removed, "
+ f"old_string would appear {stripped_count} times in the file. "
+ "Retry with the trailing newline removed and add surrounding "
+ "context so the match is unique."
+ )
+ return f"Error: String not found in file: '{old_string}'"
+ if occurrences > 1 and not replace_all:
+ return (
+ f"Error: String '{old_string}' appears {occurrences} times in file. "
+ "Use replace_all=True to replace all instances, or provide a more "
+ "specific string with surrounding context."
+ )
+ return content.replace(old_string, new_string), occurrences
def _find_nu() -> Optional[str]:
- """Resolve the nushell binary via the parent process's PATH.
-
- Cached on the module so we only pay the lookup cost once. Returns
- ``None`` if nu isn't installed; callers should fall back to the
- upstream shell.
-
- Resolution uses the live host PATH (not the sandbox's stripped
- ``self._env``) deliberately: nu must be reachable from somewhere,
- and pinning to an absolute path means the eventual subprocess can
- still launch even when env={} is passed for sandboxing.
- """
if not hasattr(_find_nu, "_cached"):
_find_nu._cached = shutil.which("nu")
return _find_nu._cached
-class NushellBackend(LocalShellBackend):
- """LocalShellBackend that routes ``execute()`` through Nushell.
-
- Why nu: cross-platform command parity. cmd.exe (Windows) and POSIX
- sh have wildly different builtin sets, quoting rules, and pipeline
- semantics, which forces LLM agents to special-case the host OS.
- Nushell parses the command itself with identical grammar on every
- platform, and many common file ops (``ls``, ``cp``, ``mv``,
- ``mkdir``, ``rm``, ``open``) are nu builtins, so they don't depend
- on external binaries being on PATH.
-
- Falls back to the parent's ``execute()`` when nu isn't installed.
- """
+class WorkspaceBackend:
+ """Workspace-rooted filesystem operations plus bounded shell execution."""
+
+ def __init__(
+ self,
+ root_dir: os.PathLike[str] | str,
+ *,
+ timeout: int = 120,
+ max_output_bytes: int = 100_000,
+ env: Optional[Dict[str, str]] = None,
+ inherit_env: bool = False,
+ max_file_size_mb: int = 10,
+ **_: Any,
+ ) -> None:
+ if timeout <= 0:
+ raise ValueError(f"timeout must be positive, got {timeout}")
+ self.cwd = Path(root_dir).resolve()
+ self.cwd.mkdir(parents=True, exist_ok=True)
+ self._default_timeout = timeout
+ self._max_output_bytes = max_output_bytes
+ self._max_file_size_bytes = max_file_size_mb * 1024 * 1024
+ self._env = os.environ.copy() if inherit_env else {}
+ if env:
+ self._env.update(env)
+ self._sandbox_id = f"workspace-{uuid4().hex[:8]}"
+
+ @property
+ def id(self) -> str:
+ return self._sandbox_id
+
+ def _resolve_path(self, key: str) -> Path:
+ virtual = _validate_virtual_path(key)
+ candidate = self.cwd / virtual.lstrip("/")
+ try:
+ resolved = candidate.resolve(strict=False)
+ resolved.relative_to(self.cwd)
+ except (OSError, RuntimeError, ValueError) as exc:
+ raise ValueError(
+ f"Path '{key}' resolves outside workspace root"
+ ) from exc
+ return resolved
+
+ def _to_virtual_path(self, path: Path) -> str:
+ resolved = path.resolve(strict=False)
+ relative = resolved.relative_to(self.cwd)
+ return "/" + relative.as_posix()
+
+ def _safe_child(self, path: Path) -> Optional[Path]:
+ try:
+ resolved = path.resolve(strict=False)
+ resolved.relative_to(self.cwd)
+ return resolved
+ except (OSError, RuntimeError, ValueError):
+ return None
+
+ def _open_read_fd(self, path: Path) -> int:
+ """Open a contained regular file without following mutable symlinks.
+
+ POSIX walks from an already-open workspace directory descriptor and
+ applies ``O_NOFOLLOW`` to every component. Windows lacks ``dir_fd``
+ traversal, so its fallback verifies the opened file identity against
+ a no-follow stat and re-checks containment after opening.
+ """
+ try:
+ relative = path.relative_to(self.cwd)
+ except ValueError as exc:
+ raise ValueError(
+ f"Path '{path}' is outside workspace root"
+ ) from exc
+ if not relative.parts:
+ raise IsADirectoryError(f"Cannot read workspace directory: {path}")
+
+ supports_anchored_open = (
+ os.name != "nt"
+ and hasattr(os, "O_DIRECTORY")
+ and os.open in os.supports_dir_fd
+ )
+ no_follow = getattr(os, "O_NOFOLLOW", 0)
+
+ if supports_anchored_open:
+ directory_flags = os.O_RDONLY | os.O_DIRECTORY | no_follow
+ directory_fd = os.open(self.cwd, directory_flags)
+ try:
+ for component in relative.parts[:-1]:
+ next_fd = os.open(
+ component,
+ directory_flags,
+ dir_fd=directory_fd,
+ )
+ os.close(directory_fd)
+ directory_fd = next_fd
+ file_fd = os.open(
+ relative.parts[-1],
+ os.O_RDONLY | no_follow,
+ dir_fd=directory_fd,
+ )
+ finally:
+ os.close(directory_fd)
+ else:
+ file_fd = os.open(path, os.O_RDONLY | no_follow)
+ try:
+ opened = os.fstat(file_fd)
+ current = os.stat(path, follow_symlinks=False)
+ if stat.S_ISLNK(current.st_mode) or (
+ opened.st_dev,
+ opened.st_ino,
+ ) != (current.st_dev, current.st_ino):
+ raise ValueError(
+ f"Path '{path}' changed while it was being opened"
+ )
+ path.resolve(strict=True).relative_to(self.cwd)
+ except BaseException:
+ os.close(file_fd)
+ raise
- def execute(self, command, *, timeout=None) -> ExecuteResponse: # type: ignore[override]
- nu = _find_nu()
- if nu is None:
- return super().execute(command, timeout=timeout)
+ try:
+ if not stat.S_ISREG(os.fstat(file_fd).st_mode):
+ raise IsADirectoryError(f"Path is not a regular file: {path}")
+ except BaseException:
+ os.close(file_fd)
+ raise
+ return file_fd
+
+ def read_text_secure(self, path: os.PathLike[str] | str) -> str:
+ """Read UTF-8 text from a pre-resolved contained path safely."""
+ fd = self._open_read_fd(Path(path))
+ with os.fdopen(fd, "r", encoding="utf-8") as handle:
+ return handle.read()
+
+ def atomic_write_text(
+ self,
+ path: os.PathLike[str] | str,
+ content: str,
+ ) -> None:
+ """Atomically write a pre-resolved path within this workspace."""
+ atomic_write_text(path, content, root_dir=self.cwd)
+
+ @staticmethod
+ def _file_info(path: Path, virtual_path: str, *, is_dir: bool) -> Dict[str, Any]:
+ info: Dict[str, Any] = {
+ "path": virtual_path + ("/" if is_dir else ""),
+ "is_dir": is_dir,
+ }
+ try:
+ status = path.stat()
+ info["size"] = 0 if is_dir else int(status.st_size)
+ info["modified_at"] = datetime.fromtimestamp(
+ status.st_mtime
+ ).isoformat()
+ except OSError:
+ pass
+ return info
+ def read(self, file_path: str, offset: int = 0, limit: int = 2000) -> ReadResult:
+ try:
+ resolved = self._resolve_path(file_path)
+ if not resolved.exists() or not resolved.is_file():
+ return ReadResult(error=f"File '{file_path}' not found")
+ fd = self._open_read_fd(resolved)
+ with os.fdopen(fd, "rb") as handle:
+ raw = handle.read()
+ try:
+ content = raw.decode("utf-8")
+ encoding = "utf-8"
+ except UnicodeDecodeError:
+ content = base64.standard_b64encode(raw).decode("ascii")
+ encoding = "base64"
+ if encoding == "utf-8":
+ if not content or not content.strip():
+ content = EMPTY_CONTENT_WARNING
+ else:
+ content = content.replace("\r\n", "\n").replace("\r", "\n")
+ lines = content.splitlines(keepends=True)
+ if offset >= len(lines):
+ return ReadResult(
+ error=(
+ f"Line offset {offset} exceeds file length "
+ f"({len(lines)} lines)"
+ )
+ )
+ content = "".join(lines[offset : offset + limit])
+ return ReadResult(
+ file_data={"content": content, "encoding": encoding}
+ )
+ except (OSError, RuntimeError, ValueError) as exc:
+ return ReadResult(error=f"Error reading file '{file_path}': {exc}")
+
+ def ls_info(self, path: str) -> list[Dict[str, Any]]:
+ directory = self._resolve_path(path)
+ if not directory.exists() or not directory.is_dir():
+ return []
+ entries: list[Dict[str, Any]] = []
+ for child in directory.iterdir():
+ safe = self._safe_child(child)
+ if safe is None:
+ continue
+ try:
+ is_dir = safe.is_dir()
+ is_file = safe.is_file()
+ except OSError:
+ continue
+ if not is_dir and not is_file:
+ continue
+ entries.append(
+ self._file_info(
+ safe,
+ self._to_virtual_path(safe),
+ is_dir=is_dir,
+ )
+ )
+ entries.sort(key=lambda item: item["path"])
+ return entries
+
+ def glob_info(
+ self,
+ pattern: str,
+ path: str = "/",
+ ) -> list[Dict[str, Any]]:
+ pattern_parts = PurePosixPath(pattern.replace("\\", "/")).parts
+ if ".." in pattern_parts or pattern.startswith("~"):
+ raise ValueError("Path traversal not allowed in glob pattern")
+ directory = self._resolve_path(path)
+ if not directory.exists() or not directory.is_dir():
+ return []
+ matches: list[Dict[str, Any]] = []
+ for candidate in directory.rglob(pattern.lstrip("/")):
+ safe = self._safe_child(candidate)
+ if safe is None:
+ continue
+ try:
+ if not safe.is_file():
+ continue
+ except OSError:
+ continue
+ matches.append(
+ self._file_info(
+ safe,
+ self._to_virtual_path(safe),
+ is_dir=False,
+ )
+ )
+ matches.sort(key=lambda item: item["path"])
+ return matches
+
+ def grep_raw(
+ self,
+ pattern: str,
+ path: Optional[str] = None,
+ glob: Optional[str] = None,
+ ) -> list[Dict[str, Any]] | str:
+ try:
+ base = self._resolve_path(path or ".")
+ except (OSError, RuntimeError, ValueError) as exc:
+ return f"Error searching path '{path or '.'}': {exc}"
+ if not base.exists():
+ return []
+ candidates = [base] if base.is_file() else base.rglob("*")
+ matches: list[Dict[str, Any]] = []
+ for candidate in candidates:
+ safe = self._safe_child(candidate)
+ if safe is None:
+ continue
+ try:
+ if not safe.is_file() or safe.stat().st_size > self._max_file_size_bytes:
+ continue
+ relative = safe.relative_to(base if base.is_dir() else base.parent)
+ if glob and not (
+ fnmatch.fnmatch(relative.as_posix(), glob)
+ or fnmatch.fnmatch(safe.name, glob)
+ ):
+ continue
+ content = self.read_text_secure(safe)
+ except (OSError, RuntimeError, UnicodeDecodeError, ValueError):
+ continue
+ virtual = self._to_virtual_path(safe)
+ for line_number, line in enumerate(content.splitlines(), 1):
+ if pattern in line:
+ matches.append(
+ {
+ "path": virtual,
+ "line": line_number,
+ "text": line,
+ }
+ )
+ return matches
+
+ def _shape_process_result(
+ self,
+ *,
+ stdout: str,
+ stderr: str,
+ returncode: int,
+ ) -> ExecuteResponse:
+ parts = [stdout] if stdout else []
+ if stderr:
+ parts.extend(
+ f"[stderr] {line}" for line in stderr.strip().split("\n")
+ )
+ output = "\n".join(parts) if parts else "<no output>"
+ truncated = False
+ if len(output) > self._max_output_bytes:
+ output = output[: self._max_output_bytes]
+ output += (
+ f"\n\n... Output truncated at {self._max_output_bytes} bytes."
+ )
+ truncated = True
+ if returncode != 0:
+ output = f"{output.rstrip()}\n\nExit code: {returncode}"
+ return ExecuteResponse(
+ output=output,
+ exit_code=returncode,
+ truncated=truncated,
+ )
+
+ def execute(
+ self,
+ command: str,
+ *,
+ timeout: Optional[int] = None,
+ ) -> ExecuteResponse:
if not command or not isinstance(command, str):
return ExecuteResponse(
output="Error: Command must be a non-empty string.",
exit_code=1,
- truncated=False,
)
-
- effective_timeout = timeout if timeout is not None else self._default_timeout
+ effective_timeout = (
+ timeout if timeout is not None else self._default_timeout
+ )
if effective_timeout <= 0:
- msg = f"timeout must be positive, got {effective_timeout}"
- raise ValueError(msg)
-
- # Flags per nushell/src/command.rs (long forms):
- # -n / --no-config-file Skip user/global config (deterministic).
- # --no-history Don't read or write shell history.
- # -c / --commands Run the next argument as a single command.
- # The std library is intentionally left loaded — agents need
- # ``path``, ``str``, ``http``, etc. without per-command imports.
- argv = [nu, "-n", "--no-history", "-c", command]
-
+ raise ValueError(
+ f"timeout must be positive, got {effective_timeout}"
+ )
+ nu = _find_nu()
+ argv: str | list[str]
+ shell = nu is None
+ if nu is None:
+ argv = command
+ else:
+ argv = [nu, "-n", "--no-history", "-c", command]
try:
result = subprocess.run(
argv,
check=False,
- shell=False, # nu parses argv[-1] itself; no host shell needed.
+ shell=shell,
capture_output=True,
+ stdin=subprocess.DEVNULL,
text=True,
timeout=effective_timeout,
env=self._env,
cwd=str(self.cwd),
)
-
- # Same output-shaping convention as upstream LocalShellBackend.
- output_parts = []
- if result.stdout:
- output_parts.append(result.stdout)
- if result.stderr:
- stderr_lines = result.stderr.strip().split("\n")
- output_parts.extend(f"[stderr] {line}" for line in stderr_lines)
- output = "\n".join(output_parts) if output_parts else "<no output>"
-
- truncated = False
- if len(output) > self._max_output_bytes:
- output = output[: self._max_output_bytes]
- output += f"\n\n... Output truncated at {self._max_output_bytes} bytes."
- truncated = True
-
- if result.returncode != 0:
- output = f"{output.rstrip()}\n\nExit code: {result.returncode}"
-
- return ExecuteResponse(
- output=output,
- exit_code=result.returncode,
- truncated=truncated,
+ return self._shape_process_result(
+ stdout=result.stdout or "",
+ stderr=result.stderr or "",
+ returncode=result.returncode,
)
-
except subprocess.TimeoutExpired:
- if timeout is not None:
- msg = f"Error: Command timed out after {effective_timeout} seconds (custom timeout). The command may be stuck or require more time."
- else:
- msg = f"Error: Command timed out after {effective_timeout} seconds. For long-running commands, re-run using the timeout parameter."
- return ExecuteResponse(output=msg, exit_code=124, truncated=False)
- except Exception as e: # noqa: BLE001
+ suffix = (
+ " (custom timeout)" if timeout is not None else ""
+ )
+ return ExecuteResponse(
+ output=(
+ f"Error: Command timed out after {effective_timeout} "
+ f"seconds{suffix}. The command may be stuck or require "
+ "more time."
+ ),
+ exit_code=124,
+ )
+ except Exception as exc: # noqa: BLE001
+ runner = "nu" if nu else "the host shell"
return ExecuteResponse(
- output=f"Error executing command via nu ({type(e).__name__}): {e}",
+ output=(
+ f"Error executing command via {runner} "
+ f"({type(exc).__name__}): {exc}"
+ ),
exit_code=1,
- truncated=False,
)
+# Compatibility name retained for integrations that imported the old class.
+NushellBackend = WorkspaceBackend
+
+
def get_backend(
parameters: Dict[str, Any],
context: Optional[Dict[str, Any]] = None,
-):
- """Return a sandboxed shell backend rooted at the per-workflow workspace.
-
- Resolution order: explicit ``working_directory`` param >
- ``context.workspace_dir`` > ``Settings().workspace_base_resolved/default``.
- The directory is created if it doesn't exist. ``virtual_mode=True``
- sandboxes paths inside the root.
- """
+) -> WorkspaceBackend:
+ """Return a native backend rooted at the per-workflow workspace."""
from core.config import Settings
from core.logging import get_logger
- param_dir = parameters.get("working_directory")
- ctx_dir = context.get("workspace_dir") if context else None
- root = param_dir or ctx_dir or os.path.join(Settings().workspace_base_resolved, "default")
+ parameter_directory = parameters.get("working_directory")
+ context_directory = context.get("workspace_dir") if context else None
+ root = (
+ parameter_directory
+ or context_directory
+ or os.path.join(Settings().workspace_base_resolved, "default")
+ )
os.makedirs(root, exist_ok=True)
get_logger(__name__).info("[Filesystem] root=%s", root)
- # ``inherit_env=True`` makes the host PATH available so the agent can
- # invoke npm, node, python, git, pwd (POSIX), where (Win), etc. as
- # external commands. ``virtual_mode`` only sandboxes filesystem ops
- # per deepagents docs — shell ``execute()`` was never path-restricted —
- # so inheriting env doesn't loosen any existing security boundary.
- return NushellBackend(root_dir=root, virtual_mode=True, inherit_env=True)
+ return WorkspaceBackend(root_dir=root, inherit_env=True)
diff --git a/server/nodes/filesystem/file_modify/__init__.py b/server/nodes/filesystem/file_modify/__init__.py
index 8634131a..1e94f566 100644
--- a/server/nodes/filesystem/file_modify/__init__.py
+++ b/server/nodes/filesystem/file_modify/__init__.py
@@ -59,13 +59,11 @@ class FileModifyNode(ActionNode):
@Operation("modify")
async def modify(self, ctx: NodeContext, params: FileModifyParams) -> Any:
"""Inlined from handlers/filesystem.py (Wave 11.D.1)."""
- from deepagents.backends.utils import perform_string_replacement
-
from .._backend import (
- atomic_write_text,
get_backend,
get_path_lock,
normalize_virtual_path,
+ perform_string_replacement,
run_sync_until_complete,
)
@@ -80,7 +78,7 @@ class FileModifyNode(ActionNode):
def _do_write():
if resolved.exists() and resolved.is_dir():
raise IsADirectoryError(f"Cannot write to {file_path}: path is a directory")
- atomic_write_text(resolved, params.content)
+ backend.atomic_write_text(resolved, params.content)
try:
async with path_lock:
@@ -95,14 +93,14 @@ class FileModifyNode(ActionNode):
def _do_edit():
if not resolved.exists() or not resolved.is_file():
return None, None, f"Error: File '{file_path}' not found"
- content = resolved.read_text(encoding="utf-8")
+ content = backend.read_text_secure(resolved)
old_string = params.old_string.replace("\r\n", "\n").replace("\r", "\n")
new_string = params.new_string.replace("\r\n", "\n").replace("\r", "\n")
replacement = perform_string_replacement(content, old_string, new_string, params.replace_all)
if isinstance(replacement, str):
return None, None, replacement
new_content, occurrences = replacement
- atomic_write_text(resolved, new_content)
+ backend.atomic_write_text(resolved, new_content)
return file_path, int(occurrences), None
try:
diff --git a/server/nodes/filesystem/file_read/__init__.py b/server/nodes/filesystem/file_read/__init__.py
index 1f116fe7..580f9b32 100644
--- a/server/nodes/filesystem/file_read/__init__.py
+++ b/server/nodes/filesystem/file_read/__init__.py
@@ -70,8 +70,8 @@ class FileReadNode(ActionNode):
# File doesn't exist / is a directory / bad offset — the
# LLM should retry with corrected input, not fail the run.
raise NodeUserError(str(e)) from e
- # backend.read returns deepagents' ReadResult dataclass — unwrap
- # it into the declared Output contract (same shape fileModify /
+ # Unwrap the backend result into the declared Output contract (same
+ # shape fileModify /
# fsSearch already return for their backend results). Returning
# the raw dataclass breaks JSON persistence of node_outputs.
if result.error:
diff --git a/server/nodes/filesystem/shell/__init__.py b/server/nodes/filesystem/shell/__init__.py
index 5f76ef83..6e8da119 100644
--- a/server/nodes/filesystem/shell/__init__.py
+++ b/server/nodes/filesystem/shell/__init__.py
@@ -41,7 +41,7 @@ class ShellNode(ActionNode):
display_name = "Shell"
subtitle = "Run Command"
group = ("filesystem", "tool")
- description = "Execute shell commands (sandboxed; no system PATH)"
+ description = "Execute short-lived shell commands from the workflow workspace"
tool_name = "shell_execute"
tool_description = "Execute a shell command. Returns stdout, stderr, and exit code."
component_kind = "square"
diff --git a/server/nodes/skill/simple_memory/__init__.py b/server/nodes/skill/simple_memory/__init__.py
index e7d23509..c029ade8 100644
--- a/server/nodes/skill/simple_memory/__init__.py
+++ b/server/nodes/skill/simple_memory/__init__.py
@@ -15,7 +15,7 @@ manual "Clear Memory" button instead.
from __future__ import annotations
-from typing import Any, Optional
+from typing import Any, Literal, Optional
from pydantic import BaseModel, ConfigDict, Field
@@ -62,6 +62,41 @@ class SimpleMemoryParams(BaseModel):
description="Number of relevant memories to retrieve from long-term storage",
json_schema_extra={"displayOptions": {"show": {"long_term_enabled": [True]}}},
)
+ embedding_provider: Literal["huggingface", "openai", "ollama"] = Field(
+ default="huggingface",
+ title="Embedding Provider",
+ description=(
+ "Provider for long-term semantic memory. HuggingFace uses the "
+ "optional local-embeddings extra; OpenAI uses the stored OpenAI "
+ "credential."
+ ),
+ json_schema_extra={
+ "displayOptions": {"show": {"long_term_enabled": [True]}}
+ },
+ )
+ embedding_model: str = Field(
+ default="",
+ title="Embedding Model",
+ description=(
+ "Provider-specific embedding model. Empty uses BAAI/"
+ "bge-small-en-v1.5 for HuggingFace, text-embedding-3-small "
+ "for OpenAI, or nomic-embed-text for Ollama."
+ ),
+ json_schema_extra={
+ "displayOptions": {"show": {"long_term_enabled": [True]}}
+ },
+ )
+ embedding_endpoint: str = Field(
+ default="",
+ title="Embedding Endpoint",
+ description=(
+ "Optional OpenAI-compatible base URL or Ollama host. "
+ "Credentials are never stored here."
+ ),
+ json_schema_extra={
+ "displayOptions": {"show": {"long_term_enabled": [True]}}
+ },
+ )
last_session_id: Optional[str] = Field(
default=None,
title="Last Claude Session ID",
diff --git a/server/pyproject.toml b/server/pyproject.toml
index 2adc21a1..a885eb10 100644
--- a/server/pyproject.toml
+++ b/server/pyproject.toml
@@ -50,30 +50,21 @@ dependencies = [
# Credential Encryption
"cryptography>=44.0.0",
- # AI - All providers (directly imported in ai.py)
- # Keep this stack in lockstep. These floors cover unsafe deserialization
- # in old langchain-core, the OpenAI/text-splitter SSRF advisories, and
- # GHSA-gr75-jv2w-4656 (file-search path traversal/sandbox escape).
- "langchain==1.3.9",
+ # Native LLM SDKs. OpenAI also drives the eight OpenAI-compatible
+ # providers through configured base URLs.
+ "openai==2.45.0",
+ "anthropic==0.96.0",
+ "google-genai==2.11.0",
+
+ # Temporary pre-cutover Temporal-history compatibility. New chat and agent
+ # executions do not import these adapters. Remove this block together with
+ # the legacy activity branch after all recorded histories have drained.
"langchain-core==1.4.7",
"langchain-openai==1.1.14",
"langchain-anthropic==1.4.6",
- "anthropic==0.96.0",
- # langchain-google-genai 4.2.4+ widens its google-genai pin to <3.0.0,
- # unblocking google-genai 2.x — required for the Interactions API
- # (client.interactions with environment= + client.agents) used by the
- # vertex_managed_agent / vertex_agent_admin plugins. 4.2.5 remains
- # compatible with the langchain-core==1.4.7 security floor.
"langchain-google-genai==4.2.5",
- "google-genai==2.11.0",
"langchain-groq==1.1.2",
"langchain-cerebras==0.8.2",
- # textChunker plugin (recursive / markdown / token strategies). Bumped
- # to 1.1.2+ alongside the langchain-core security floor to clear the
- # HTMLHeaderTextSplitter SSRF advisory; the 1.1 line requires
- # langchain-core>=1.3, which we now satisfy.
- "langchain-text-splitters==1.1.2",
- "deepagents==0.6.9",
# Pydantic 2.13 changed validator ordering; pin to main's 2.12 line
# to keep AIMessage construction/tool-schema generation identical.
@@ -175,8 +166,11 @@ dependencies = [
docs = [
"beautifulsoup4>=4.12.0",
"pypdf>=5.0.0",
- # `langchain-text-splitters` moved to top-level dependencies (above) —
- # the textChunker plugin is part of the runtime, not optional.
+]
+local-embeddings = [
+ # Optional because the model + torch stack is large. The embedding node and
+ # long-term memory surface a clear install hint when it is absent.
+ "sentence-transformers",
]
keyring = [
"keyring>=25.0.0", # OS-native credential storage (Windows/macOS/Linux)
diff --git a/server/requirements.txt b/server/requirements.txt
index 7f846008..ac1d7cfd 100644
--- a/server/requirements.txt
+++ b/server/requirements.txt
@@ -12,12 +12,12 @@ annotated-doc==0.0.4
# via fastapi
annotated-types==0.7.0
# via pydantic
-anthropic==0.94.0
+anthropic==0.96.0
# via
# langchain-anthropic
# opencompany-server
# rlms
-anyio==4.14.1
+anyio==4.14.2
# via
# anthropic
# google-genai
@@ -45,8 +45,6 @@ bcrypt==5.0.0
# via opencompany-server
beautifulsoup4==4.15.0
# via crawlee
-bracex==3.0
- # via wcmatch
brotli==1.2.0 ; platform_python_implementation == 'CPython'
# via httpx
brotlicffi==1.2.0.1 ; platform_python_implementation != 'CPython'
@@ -87,8 +85,6 @@ cryptography==49.0.0
# pyjwt
ddgs==9.14.4
# via opencompany-server
-deepagents==0.5.2
- # via opencompany-server
dependency-injector==4.49.1
# via opencompany-server
distro==1.9.0
@@ -182,13 +178,12 @@ httpx==0.28.1
# google-genai
# groq
# httpx-ws
- # langgraph-sdk
# langsmith
# lmstudio
- # opencompany-server
# mcp
# ollama
# openai
+ # opencompany-server
# portkey-ai
# python-telegram-bot
httpx-sse==0.4.3
@@ -224,32 +219,20 @@ jsonschema==4.26.0
# via mcp
jsonschema-specifications==2025.9.1
# via jsonschema
-langchain==1.2.18
- # via deepagents
-langchain-anthropic==1.4.0
- # via
- # deepagents
- # opencompany-server
+langchain-anthropic==1.4.6
+ # via opencompany-server
langchain-cerebras==0.8.2
# via opencompany-server
-langchain-core==1.3.3
+langchain-core==1.4.7
# via
- # deepagents
- # langchain
# langchain-anthropic
# langchain-cerebras
# langchain-google-genai
# langchain-groq
# langchain-openai
- # langchain-text-splitters
- # langgraph
- # langgraph-checkpoint
- # langgraph-prebuilt
# opencompany-server
langchain-google-genai==4.2.5
- # via
- # deepagents
- # opencompany-server
+ # via opencompany-server
langchain-groq==1.1.2
# via opencompany-server
langchain-openai==1.1.14
@@ -258,22 +241,8 @@ langchain-openai==1.1.14
# opencompany-server
langchain-protocol==0.0.18
# via langchain-core
-langchain-text-splitters==1.1.2
- # via opencompany-server
-langgraph==1.1.10
- # via langchain
-langgraph-checkpoint==4.1.1
- # via
- # langgraph
- # langgraph-prebuilt
-langgraph-prebuilt==1.0.13
- # via langgraph
-langgraph-sdk==0.3.15
- # via langgraph
langsmith==0.10.2
- # via
- # deepagents
- # langchain-core
+ # via langchain-core
lmstudio==1.5.0
# via opencompany-server
lxml==6.1.1
@@ -307,6 +276,7 @@ ollama==0.6.2
openai==2.45.0
# via
# langchain-openai
+ # opencompany-server
# rlms
opentelemetry-api==1.43.0
# via
@@ -319,11 +289,8 @@ opentelemetry-semantic-conventions==0.64b0
# via opentelemetry-sdk
orjson==3.11.9
# via
- # langgraph-sdk
# langsmith
# opencompany-server
-ormsgpack==1.12.2
- # via langgraph-checkpoint
packaging==26.2
# via
# langchain-core
@@ -376,16 +343,14 @@ pydantic==2.12.5
# fastapi
# google-genai
# groq
- # langchain
# langchain-anthropic
# langchain-core
# langchain-google-genai
- # langgraph
# langsmith
- # opencompany-server
# mcp
# ollama
# openai
+ # opencompany-server
# portkey-ai
# pydantic-settings
# sqlmodel
@@ -397,8 +362,8 @@ pydantic-monty==0.0.18
pydantic-settings==2.14.2
# via
# crawlee
- # opencompany-server
# mcp
+ # opencompany-server
pyee==13.0.1
# via crawlee
pygments==2.20.0
@@ -407,8 +372,8 @@ pygments==2.20.0
# rich
pyjwt==2.13.0
# via
- # opencompany-server
# mcp
+ # opencompany-server
pyparsing==3.3.2
# via httplib2
pytest==9.1.1
@@ -520,7 +485,7 @@ tqdm==4.68.4
# portkey-ai
types-protobuf==7.34.1.20260518
# via temporalio
-types-requests==2.33.0.20260518
+types-requests==2.33.0.20260712
# via portkey-ai
typing-extensions==4.16.0
# via
@@ -575,17 +540,15 @@ uuid-utils==0.17.0
# langsmith
uvicorn==0.51.0
# via
- # opencompany-server
# mcp
+ # opencompany-server
uvloop==0.22.1 ; platform_python_implementation != 'PyPy' and sys_platform != 'cygwin' and sys_platform != 'win32'
# via uvicorn
watchfiles==1.2.0
# via uvicorn
-wcmatch==11.0
- # via deepagents
webencodings==0.5.1
# via html5lib
-websockets==16.1
+websockets==15.0.1
# via
# google-genai
# langsmith
@@ -596,9 +559,7 @@ wsproto==1.3.2
xdk==0.9.0
# via opencompany-server
xxhash==3.8.1
- # via
- # langgraph
- # langsmith
+ # via langsmith
yarl==1.24.2
# via
# aiohttp
diff --git a/server/services/agent_runtime.py b/server/services/agent_runtime.py
new file mode 100644
index 00000000..ef1cf177
--- /dev/null
+++ b/server/services/agent_runtime.py
@@ -0,0 +1,369 @@
+"""Provider-neutral agent execution primitives.
+
+This module is the shared boundary between orchestration (the in-process
+``AIService`` loop and Temporal activities) and the native provider SDK layer.
+It deliberately has no eager LangChain imports. ``AgentToolSpec.to_langchain``
+exists only so Temporal histories recorded before the native cutover can drain;
+it is lazy and is removed with the legacy activity branch.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+from dataclasses import dataclass, field
+from typing import Any, Awaitable, Callable, Dict, List, Optional, Sequence, Type
+
+from pydantic import BaseModel, ValidationError
+
+from core.logging import get_logger
+from services.llm.messages import filter_empty_messages
+from services.llm.protocol import (
+ LLMError,
+ LLMResponse,
+ Message,
+ ThinkingConfig,
+ ToolCall,
+ ToolDef,
+ Usage,
+)
+from services.tool_identity import DuplicateToolNameError
+
+logger = get_logger(__name__)
+
+
+@dataclass
+class AgentToolSpec:
+ """LLM-visible definition plus local execution/validation metadata."""
+
+ definition: ToolDef
+ args_schema: Optional[Type[BaseModel]] = None
+ execution: Dict[str, Any] = field(default_factory=dict)
+
+ @property
+ def name(self) -> str:
+ return self.definition.name
+
+ @property
+ def description(self) -> str:
+ return self.definition.description
+
+ @property
+ def parameters(self) -> Dict[str, Any]:
+ return self.definition.parameters
+
+ def to_langchain(self):
+ """Build the legacy adapter used only by pre-cutover Temporal runs."""
+
+ from langchain_core.tools import StructuredTool
+
+ def _placeholder(**kwargs):
+ return kwargs
+
+ return StructuredTool.from_function(
+ name=self.name,
+ description=self.description,
+ func=_placeholder,
+ args_schema=self.args_schema,
+ )
+
+
+def _tool_definition(tool: ToolDef | AgentToolSpec) -> ToolDef:
+ return tool.definition if isinstance(tool, AgentToolSpec) else tool
+
+
+def _tool_specs_by_name(
+ tools: Sequence[ToolDef | AgentToolSpec],
+) -> Dict[str, AgentToolSpec]:
+ return {
+ tool.name: tool
+ for tool in tools
+ if isinstance(tool, AgentToolSpec)
+ }
+
+
+def add_usage(total: Usage, current: Optional[Usage]) -> Usage:
+ """Accumulate normalized provider usage without mutating either input."""
+
+ if current is None:
+ return total
+ return Usage(
+ input_tokens=total.input_tokens + current.input_tokens,
+ output_tokens=total.output_tokens + current.output_tokens,
+ total_tokens=total.total_tokens + current.total_tokens,
+ cache_creation_tokens=(
+ total.cache_creation_tokens + current.cache_creation_tokens
+ ),
+ cache_read_tokens=total.cache_read_tokens + current.cache_read_tokens,
+ reasoning_tokens=total.reasoning_tokens + current.reasoning_tokens,
+ )
+
+
+async def run_native_llm_step(
+ chat_unifier,
+ *,
+ provider: str,
+ api_key: str,
+ messages: Sequence[Message],
+ model: str,
+ temperature: float,
+ max_tokens: int,
+ thinking: Optional[ThinkingConfig] = None,
+ tools: Optional[Sequence[ToolDef | AgentToolSpec]] = None,
+ sdk_max_retries: int = 0,
+ explicit_max_retries: int = 2,
+ translate_errors: bool = True,
+) -> LLMResponse:
+ """Execute one native SDK turn and return its lossless response envelope.
+
+ Agent retries are deliberately outside the provider SDK so only failures
+ normalized as retryable :class:`LLMError` values are repeated. Temporal
+ passes ``explicit_max_retries=0`` and owns activity-level retry itself.
+ """
+
+ if chat_unifier is None:
+ raise RuntimeError("ChatUnifier is required for native agent execution")
+
+ definitions = [_tool_definition(tool) for tool in (tools or ())]
+ attempts = max(0, int(explicit_max_retries)) + 1
+ for attempt in range(attempts):
+ try:
+ return await chat_unifier.chat(
+ provider=provider,
+ api_key=api_key,
+ messages=filter_empty_messages(messages),
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ thinking=thinking,
+ tools=definitions or None,
+ sdk_max_retries=max(0, int(sdk_max_retries)),
+ # Preserve structured metadata until this agent-step boundary.
+ translate_errors=False,
+ )
+ except LLMError as error:
+ if not error.retryable or attempt + 1 >= attempts:
+ if not translate_errors:
+ raise
+ from services.plugin import NodeUserError
+
+ raise NodeUserError(error.user_message) from error
+ delay = (
+ error.retry_after
+ if error.retry_after is not None
+ else 0.25 * (2**attempt)
+ )
+ await asyncio.sleep(max(0.0, min(float(delay), 5.0)))
+
+ raise AssertionError("native LLM retry loop exhausted unexpectedly")
+
+
+def _validated_tool_args(
+ call: ToolCall,
+ specs: Dict[str, AgentToolSpec],
+) -> tuple[Optional[Dict[str, Any]], Optional[str]]:
+ if call.parse_error:
+ return None, call.parse_error
+
+ spec = specs.get(call.name)
+ if spec is None or spec.args_schema is None:
+ return dict(call.args or {}), None
+
+ try:
+ value = spec.args_schema.model_validate(call.args or {})
+ return value.model_dump(mode="json"), None
+ except ValidationError as exc:
+ return None, str(exc)
+
+
+async def run_native_agent_loop(
+ chat_unifier,
+ *,
+ provider: str,
+ api_key: str,
+ model: str,
+ temperature: float,
+ max_tokens: int,
+ initial_messages: Sequence[Message],
+ thinking: Optional[ThinkingConfig] = None,
+ tools: Optional[Sequence[AgentToolSpec]] = None,
+ tool_executor: Optional[Callable[[str, Dict[str, Any]], Awaitable[Any]]] = None,
+ max_iterations: int = 500,
+ progress_callback: Optional[Callable[[int], Awaitable[Any]]] = None,
+ rebind_from_operations: Optional[
+ Callable[[List[Dict[str, Any]]], Awaitable[List[AgentToolSpec]]]
+ ] = None,
+) -> Dict[str, Any]:
+ """Run the shared buffered native tool-agent loop.
+
+ The complete assistant message returned by a provider is appended verbatim
+ before tool execution. This is essential for Gemini thought signatures,
+ Anthropic signed thinking blocks, and OpenAI reasoning continuation state.
+ """
+
+ current_tools: List[AgentToolSpec] = list(tools or ())
+ messages: List[Message] = list(initial_messages)
+ usage = Usage()
+ thinking_parts: List[str] = []
+ last_response: Optional[LLMResponse] = None
+ iteration = 0
+
+ for iteration in range(1, max_iterations + 1):
+ if progress_callback is not None:
+ try:
+ await progress_callback(iteration)
+ except Exception as exc: # progress is observational
+ logger.debug("[Agent loop] progress callback failed: %s", exc)
+
+ response = await run_native_llm_step(
+ chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ messages=messages,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ thinking=thinking,
+ tools=current_tools,
+ )
+ last_response = response
+ usage = add_usage(usage, response.usage)
+
+ assistant = response.assistant_message
+ if assistant is None:
+ assistant = Message(
+ role="assistant",
+ content=response.content,
+ tool_calls=list(response.tool_calls),
+ )
+ messages.append(assistant)
+
+ if response.thinking:
+ thinking_parts.append(
+ (
+ response.thinking
+ if not thinking_parts
+ else f"--- Iteration {iteration} ---\n{response.thinking}"
+ )
+ )
+
+ if str(response.finish_reason).upper() == "SAFETY":
+ logger.warning("[Agent loop] provider response blocked by safety filters")
+
+ calls = list(response.tool_calls or assistant.tool_calls or ())
+ if not calls:
+ return {
+ "messages": messages,
+ "iteration": iteration,
+ "thinking_content": "\n\n".join(thinking_parts) or None,
+ "truncated": False,
+ "usage": usage,
+ "response": response,
+ }
+
+ if tool_executor is None:
+ logger.warning(
+ "[Agent loop] model emitted %d tool call(s) without an executor",
+ len(calls),
+ )
+ return {
+ "messages": messages,
+ "iteration": iteration,
+ "thinking_content": "\n\n".join(thinking_parts) or None,
+ "truncated": False,
+ "usage": usage,
+ "response": response,
+ }
+
+ specs = _tool_specs_by_name(current_tools)
+ iteration_new_tools: List[AgentToolSpec] = []
+ for call in calls:
+ if call.name not in specs:
+ result: Any = {
+ "error": "Unknown tool",
+ "details": (
+ f"Tool {call.name!r} is not connected to this agent."
+ ),
+ }
+ else:
+ args, validation_error = _validated_tool_args(call, specs)
+ if validation_error:
+ result = {
+ "error": "Invalid tool arguments",
+ "details": validation_error,
+ }
+ if call.raw_arguments is not None:
+ result["raw_arguments"] = call.raw_arguments
+ else:
+ try:
+ result = await tool_executor(call.name, args or {})
+ except Exception as exc:
+ # Tool failures are fed back to the model.
+ logger.error(
+ "[Agent loop] tool %r failed: %s",
+ call.name,
+ exc,
+ )
+ result = {"error": str(exc)}
+
+ if (
+ rebind_from_operations is not None
+ and isinstance(result, dict)
+ and result.get("operations")
+ ):
+ try:
+ added = await rebind_from_operations(result["operations"])
+ if added:
+ iteration_new_tools.extend(added)
+ except DuplicateToolNameError as exc:
+ logger.warning(
+ "[Agent loop] rejected ambiguous tool rebind: %s", exc
+ )
+ result = dict(result)
+ result["rebind_error"] = exc.as_dict()
+ except Exception as exc:
+ logger.warning(
+ "[Agent loop] tool rebind failed: %s", exc, exc_info=True
+ )
+
+ messages.append(
+ Message(
+ role="tool",
+ content=json.dumps(result, default=str),
+ tool_call_id=call.id,
+ name=call.name,
+ )
+ )
+
+ if iteration_new_tools:
+ current_tools.extend(iteration_new_tools)
+ logger.info(
+ "[Agent loop] rebound %d tool(s) (total=%d)",
+ len(iteration_new_tools),
+ len(current_tools),
+ )
+
+ terminal = Message(
+ role="assistant",
+ content=(
+ f"[Recursion limit reached: {max_iterations} iterations. "
+ "Adjust agent.recursion_limit in llm_defaults.json or simplify the task.]"
+ ),
+ )
+ messages.append(terminal)
+ return {
+ "messages": messages,
+ "iteration": iteration,
+ "thinking_content": "\n\n".join(thinking_parts) or None,
+ "truncated": True,
+ "usage": usage,
+ "response": last_response,
+ }
+
+
+__all__ = [
+ "AgentToolSpec",
+ "add_usage",
+ "run_native_llm_step",
+ "run_native_agent_loop",
+]
diff --git a/server/services/ai.py b/server/services/ai.py
index 274ce549..8e0d5f2e 100644
--- a/server/services/ai.py
+++ b/server/services/ai.py
@@ -1,51 +1,42 @@
-"""AI service: LangChain-based chat-model construction + plain async agent loop."""
+"""AI service backed by the native provider SDK layer.
+
+The LangChain model factory and loop below are retained temporarily and are
+used only by pre-cutover Temporal histories. Every new in-process or durable
+agent execution uses :mod:`services.agent_runtime`.
+"""
from __future__ import annotations
import functools
-import re
import time
-import httpx
from dataclasses import dataclass
from datetime import datetime
from typing import Awaitable, Dict, Any, List, Optional, Callable, Type, TYPE_CHECKING
# ---------------------------------------------------------------------------
-# LangChain imports — fully lazy except for ``BaseMessage``.
+# LangChain imports — fully lazy and retained only for old Temporal histories.
#
# Cold-start measurement (Windows, fresh disk):
# ``from langchain_openai import ChatOpenAI`` ~20.8s (pulls openai SDK + tiktoken)
# Deferring the heavy imports until the first agent run trims server-ready time.
#
-# ``services/llm/`` (the native LLM SDK layer) handles ``execute_chat()`` and
-# ``fetch_models()`` without LangChain. LangChain is only needed on the agent
-# execution path (``execute_agent`` / ``execute_chat_agent``) for the per-
-# provider ``ChatOpenAI`` / ``ChatAnthropic`` / ``ChatGoogleGenerativeAI``
-# instances and ``chat_model.bind_tools()`` provider-unified tool calling.
-#
-# ``BaseMessage`` stays eager because ``_run_agent_loop`` type-hints it in
-# its signature; the import is cheap (no tiktoken / openai SDK pull) and
-# keeps type checkers happy. Everything else is lazy via
-# ``@functools.cache``'d helpers or local imports inside the methods that
-# need them.
+# ``services/llm/`` and ``services/agent_runtime.py`` handle every new chat
+# and agent execution. The imports below are type-checking-only or function-
+# local so a native process never loads LangChain unless it must execute an
+# already-recorded legacy Temporal activity.
# ---------------------------------------------------------------------------
-from langchain_core.messages import BaseMessage # eager: type hint for _run_agent_loop
import json
# ``NodeUserError`` is the framework's "user-correctable, no-traceback"
# sentinel used to surface typed SDK errors cleanly through
-# ``BaseNode.execute()``. The ``openai`` SDK itself is imported
-# function-locally in ``execute_agent`` / ``execute_chat_agent`` (its
-# only consumers here, via ``except openai.OpenAIError``) — a top-level
-# import cost seconds of boot time and re-created the eager-LLM-SDK
-# anti-pattern (docs-internal/performance.md).
+# ``BaseNode.execute()``.
from services.plugin import NodeUserError
from services.tool_identity import DuplicateToolNameError, ensure_unique_tool_names
if TYPE_CHECKING:
from langchain_openai import ChatOpenAI # noqa: F401
- from langchain_core.messages import HumanMessage, SystemMessage, AIMessage, ToolMessage # noqa: F401
+ from langchain_core.messages import AIMessage, BaseMessage, HumanMessage, SystemMessage, ToolMessage # noqa: F401
from langchain_core.tools import StructuredTool # noqa: F401
from pydantic import BaseModel, Field, create_model # noqa: F401
from langchain_anthropic import ChatAnthropic # noqa: F401
@@ -121,7 +112,7 @@ def __getattr__(name: str):
except ImportError as e:
return str(e)
if name == "PROVIDER_CONFIGS":
- return get_provider_configs()
+ return NATIVE_PROVIDER_CONFIGS
raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
@@ -132,7 +123,7 @@ from services.auth import AuthService
logger = get_logger(__name__)
# ---------------------------------------------------------------------------
-# Native LLM provider imports (dual-path: native for chat, LangChain for agents)
+# Native LLM provider/runtime imports for every new chat and agent execution
# ---------------------------------------------------------------------------
# ``ChatUnifier`` (injected via DI) owns provider dispatch. The legacy
# ``services.llm.factory`` module (``create_provider`` /
@@ -141,12 +132,21 @@ from services.llm.protocol import (
Message as NativeMessage,
ThinkingConfig as NativeThinkingConfig,
LLMResponse,
+ ToolDef,
+ Usage,
)
from services.llm.config import (
+ PROVIDER_CONFIGS as NATIVE_PROVIDER_CONFIGS,
+ detect_provider_from_model as native_detect_provider_from_model,
+ get_default_model as native_get_default_model,
+ get_default_model_async as native_get_default_model_async,
+ is_model_valid_for_provider as native_is_model_valid_for_provider,
resolve_max_tokens as native_resolve_max_tokens,
resolve_temperature as native_resolve_temperature,
)
from services.llm.vertex import is_vertex_express_key
+from services.llm.messages import filter_empty_messages as _filter_native_messages
+from services.agent_runtime import AgentToolSpec, run_native_agent_loop
# =============================================================================
@@ -154,84 +154,72 @@ from services.llm.vertex import is_vertex_express_key
# =============================================================================
-def _parse_memory_markdown(content: str) -> List[BaseMessage]:
- """Parse markdown memory content into LangChain messages.
-
- Markdown format:
- ### **Human** (timestamp)
- message content
-
- ### **Assistant** (timestamp)
- response content
- """
- from langchain_core.messages import HumanMessage, AIMessage
-
- messages = []
- pattern = r"### \*\*(Human|Assistant)\*\*[^\n]*\n(.*?)(?=\n### \*\*|$)"
- for role, text in re.findall(pattern, content, re.DOTALL):
- text = text.strip()
- if text:
- msg_class = HumanMessage if role == "Human" else AIMessage
- messages.append(msg_class(content=text))
- return messages
-
+_USAGE_FIELDS = (
+ "input_tokens",
+ "output_tokens",
+ "total_tokens",
+ "cache_creation_tokens",
+ "cache_read_tokens",
+ "reasoning_tokens",
+)
-def _append_to_memory_markdown(content: str, role: str, message: str) -> str:
- """Append a message to markdown memory content."""
- ts = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
- label = "Human" if role == "human" else "Assistant"
- entry = f"\n### **{label}** ({ts})\n{message}\n"
- # Remove empty state message if present
- return content.replace("*No messages yet.*\n", "") + entry
+def _coerce_native_usage(value: Any) -> Usage:
+ if isinstance(value, Usage):
+ return value
+ if isinstance(value, dict):
+ return Usage(
+ **{
+ key: value.get(key, 0)
+ for key in _USAGE_FIELDS
+ }
+ )
+ return Usage()
-def _trim_markdown_window(content: str, window_size: int) -> tuple:
- """Keep last N message pairs, return (trimmed_content, removed_texts).
- Args:
- content: Full markdown content
- window_size: Number of message PAIRS to keep (human+assistant)
+def _accumulate_compaction_usage(
+ final_state: Dict[str, Any],
+ compaction_result: Optional[Dict[str, Any]],
+) -> None:
+ """Add summarizer billing to execution usage, not context thresholds."""
- Returns:
- Tuple of (trimmed markdown, list of removed message texts for archival)
- """
- pattern = r"(### \*\*(Human|Assistant)\*\*[^\n]*\n.*?)(?=\n### \*\*|$)"
- blocks = [m[0] for m in re.findall(pattern, content, re.DOTALL)]
+ if not compaction_result or not compaction_result.get("usage"):
+ return
+ final_state["usage"] = _coerce_native_usage(
+ final_state.get("usage")
+ ) + _coerce_native_usage(compaction_result["usage"])
- if len(blocks) <= window_size * 2:
- return content, []
- keep = blocks[-(window_size * 2) :]
- removed = blocks[: -(window_size * 2)]
+def _parse_memory_markdown(content: str) -> List[NativeMessage]:
+ """Compatibility wrapper for the native Markdown memory parser."""
- # Extract text from removed blocks for vector storage
- removed_texts = []
- for block in removed:
- match = re.search(r"\n(.*)$", block, re.DOTALL)
- if match:
- removed_texts.append(match.group(1).strip())
+ from services.memory.markdown import parse_memory_markdown
- return "# Conversation History\n" + "\n".join(keep), removed_texts
+ return parse_memory_markdown(content)
-# Global cache for vector stores per session (InMemoryVectorStore)
-_memory_vector_stores: Dict[str, Any] = {}
+# Compatibility alias for callers that clear the historical cache directly.
+# The implementation and ownership now live in ``services.memory.vector_store``.
+from services.memory.vector_store import (
+ _memory_vector_stores as _memory_vector_stores,
+ get_memory_vector_store as _get_memory_vector_store,
+)
-def _get_memory_vector_store(session_id: str):
- """Get or create InMemoryVectorStore for a session."""
- if session_id not in _memory_vector_stores:
- try:
- from langchain_core.vectorstores import InMemoryVectorStore
- from langchain_huggingface import HuggingFaceEmbeddings
+async def _get_configured_memory_vector_store(
+ session_id: str,
+ memory_data: Dict[str, Any],
+ auth_service: Any,
+):
+ """Resolve one memory node's async native embedding store."""
- embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-small-en-v1.5")
- _memory_vector_stores[session_id] = InMemoryVectorStore(embeddings)
- logger.debug(f"[Memory] Created vector store for session '{session_id}'")
- except ImportError as e:
- logger.warning(f"[Memory] Vector store not available: {e}")
- return None
- return _memory_vector_stores[session_id]
+ return await _get_memory_vector_store(
+ session_id,
+ provider=memory_data.get("embedding_provider", "huggingface"),
+ model=memory_data.get("embedding_model"),
+ endpoint=memory_data.get("embedding_endpoint"),
+ auth_service=auth_service,
+ )
# =============================================================================
@@ -256,23 +244,8 @@ class ProviderConfig:
supported_params: Optional[List[str]] = None # API-supported params (from llm_defaults.json)
-@dataclass
-class ThinkingConfig:
- """Unified thinking/reasoning configuration across AI providers.
-
- LangChain parameters per provider (Jan 2026):
- - Claude: thinking={"type": "enabled", "budget_tokens": budget}, temp must be 1
- - OpenAI o-series: reasoning_effort ('minimal', 'low', 'medium', 'high')
- - Gemini 3+: thinking_level ('low', 'medium', 'high')
- - Gemini 2.5: thinking_budget (int tokens)
- - Groq: reasoning_format ('parsed', 'hidden')
- """
-
- enabled: bool = False
- budget: int = 2048 # Token budget (Claude, Gemini 2.5)
- effort: str = "medium" # Effort level: 'minimal', 'low', 'medium', 'high' (OpenAI o-series)
- level: str = "medium" # Thinking level: 'low', 'medium', 'high' (Gemini 3+)
- format: str = "parsed" # Output format: 'parsed', 'hidden' (Groq)
+# Compatibility export. Native configuration is the single source of truth.
+ThinkingConfig = NativeThinkingConfig
def _openai_headers(api_key: str) -> dict:
@@ -367,37 +340,13 @@ def _resolve_max_tokens(flattened: dict, model: str, provider: str) -> int:
def _resolve_temperature(flattened: dict, model: str, provider: str, thinking_enabled: bool) -> float:
- """Resolve temperature with model-specific constraints.
-
- Handles:
- - O-series models: always temp=1
- - Claude with thinking: always temp=1
- - Range clamping per provider
- """
- from services.model_registry import get_model_registry
-
- registry = get_model_registry()
-
- user_val = flattened.get("temperature")
- if user_val is None:
- user_val = registry.get_agent_defaults()["default_temperature"]
- user_temp = float(user_val)
-
- # O-series reasoning models always require temperature=1
- if registry.is_reasoning_model(model, provider):
- if user_temp != 1:
- logger.info(f"[AI] Reasoning model '{model}': forcing temperature to 1 (was {user_temp})")
- return 1.0
-
- # Claude thinking mode requires temperature=1
- if thinking_enabled and provider == "anthropic":
- if user_temp != 1:
- logger.info(f"[AI] Claude thinking mode: forcing temperature to 1 (was {user_temp})")
- return 1.0
-
- # Clamp to valid range for provider/model
- temp_range = registry.get_temperature_range(model, provider)
- return max(temp_range[0], min(temp_range[1], user_temp))
+ """Resolve temperature through the native provider configuration."""
+ return native_resolve_temperature(
+ flattened,
+ model,
+ provider,
+ thinking_enabled,
+ )
# Provider configurations - built from config/llm_defaults.json on first use.
@@ -451,11 +400,7 @@ def get_provider_configs() -> Dict[str, ProviderConfig]:
def detect_provider_from_model(model: str) -> str:
"""Detect AI provider from model name using registry patterns."""
- model_lower = model.lower()
- for provider_name, config in get_provider_configs().items():
- if any(pattern in model_lower for pattern in config.detection_patterns):
- return provider_name
- return "openai" # default
+ return native_detect_provider_from_model(model)
def is_model_valid_for_provider(model: str, provider: str) -> bool:
@@ -476,19 +421,12 @@ def is_model_valid_for_provider(model: str, provider: str) -> bool:
as always-valid; the local-server SDK will reject genuinely missing
models at request time with a clear 404.
"""
- if provider in ("openrouter", "ollama", "lmstudio"):
- return True
- config = get_provider_configs().get(provider)
- if not config:
- return True
- model_lower = model.lower()
- return any(pattern in model_lower for pattern in config.detection_patterns)
+ return native_is_model_valid_for_provider(model, provider)
def get_default_model(provider: str) -> str:
"""Get default model for a provider from JSON config."""
- config = get_provider_configs().get(provider)
- return config.default_model if config else "gpt-4o-mini"
+ return native_get_default_model(provider)
async def get_default_model_async(provider: str, database) -> str:
@@ -496,17 +434,7 @@ async def get_default_model_async(provider: str, database) -> str:
Priority: database user setting > JSON config file > fallback
"""
- # Check database for user-configured default
- if database:
- try:
- db_defaults = await database.get_provider_defaults(provider)
- if db_defaults and db_defaults.get("default_model"):
- return db_defaults["default_model"]
- except Exception as e:
- logger.warning(f"Failed to get DB defaults for {provider}: {e}")
-
- # Fall back to JSON config
- return get_default_model(provider)
+ return await native_get_default_model_async(provider, database)
# =============================================================================
@@ -545,76 +473,10 @@ def is_valid_message_content(content: Any) -> bool:
return bool(content)
-def filter_empty_messages(messages: List[BaseMessage]) -> List[BaseMessage]:
- """Filter out messages with empty content to prevent API errors.
-
- This is a standardized utility that handles empty message filtering for all
- AI providers (OpenAI, Anthropic/Claude, Google Gemini, and future providers).
+def filter_empty_messages(messages: List[Any]) -> List[Any]:
+ """Compatibility wrapper around the provider-neutral message filter."""
- Different providers have different sensitivities:
- - Gemini: Emits "HumanMessage with empty content was removed" warning
- - Claude/Anthropic: Throws errors for empty HumanMessage content
- - OpenAI: Generally tolerant but empty messages waste tokens
-
- This filter preserves:
- - ToolMessage: Always kept (contains tool execution results)
- - AIMessage with tool_calls: Kept even if content empty (tool calls are content)
- - SystemMessage: Kept only if has non-empty content
- - HumanMessage/others: Filtered if content is empty
-
- Args:
- messages: Sequence of LangChain BaseMessage objects
-
- Returns:
- Filtered list of messages with empty content removed
- """
- from langchain_core.messages import AIMessage, SystemMessage, ToolMessage
-
- filtered = []
-
- for m in messages:
- # ToolMessage - always keep (contains tool execution results)
- if isinstance(m, ToolMessage):
- filtered.append(m)
- continue
-
- # AIMessage with tool_calls - keep even if content is empty
- # (the tool calls themselves are the meaningful content)
- if isinstance(m, AIMessage) and hasattr(m, "tool_calls") and m.tool_calls:
- filtered.append(m)
- continue
-
- # SystemMessage - keep only if has non-empty content
- if isinstance(m, SystemMessage):
- if hasattr(m, "content") and m.content and str(m.content).strip():
- filtered.append(m)
- continue
-
- # HumanMessage and other message types - filter out empty content
- if hasattr(m, "content"):
- content = m.content
-
- # Handle list content format (Gemini returns [{"type": "text", "text": "..."}])
- if isinstance(content, list):
- has_content = any(
- (isinstance(block, dict) and block.get("text", "").strip()) or (isinstance(block, str) and block.strip())
- for block in content
- )
- if has_content:
- filtered.append(m)
-
- # Handle string content (most common)
- elif isinstance(content, str) and content.strip():
- filtered.append(m)
-
- # Handle other non-empty content types (keep if truthy)
- elif content:
- filtered.append(m)
- else:
- # Message without content attr - keep it (might be special message type)
- filtered.append(m)
-
- return filtered
+ return _filter_native_messages(messages)
def extract_thinking_from_response(response) -> tuple:
@@ -629,6 +491,9 @@ def extract_thinking_from_response(response) -> tuple:
Returns:
Tuple of (text_content: str, thinking_content: Optional[str])
"""
+ if isinstance(response, LLMResponse):
+ return response.content, response.thinking
+
text_parts = []
thinking_parts = []
@@ -1097,9 +962,14 @@ class AIService:
if not svc:
return
- # Extract usage_metadata from AIMessage (LangChain standardizes this)
+ # Native responses expose a normalized ``Usage`` object. The
+ # LangChain branches below are retained only for pre-cutover histories.
usage = None
- if hasattr(ai_response, "usage_metadata") and ai_response.usage_metadata:
+ if isinstance(ai_response, Usage):
+ usage = ai_response
+ elif isinstance(ai_response, LLMResponse):
+ usage = ai_response.usage
+ elif hasattr(ai_response, "usage_metadata") and ai_response.usage_metadata:
usage = ai_response.usage_metadata
elif hasattr(ai_response, "response_metadata"):
# Fallback: check response_metadata for usage info
@@ -1109,7 +979,7 @@ class AIService:
elif "token_usage" in meta:
usage = meta["token_usage"]
- if not usage:
+ if not usage and all_messages:
# Aggregate usage from all AI messages if single message has no usage
total_input = 0
total_output = 0
@@ -1129,6 +999,9 @@ class AIService:
"input_tokens": usage.get("input_tokens", 0) if isinstance(usage, dict) else getattr(usage, "input_tokens", 0),
"output_tokens": usage.get("output_tokens", 0) if isinstance(usage, dict) else getattr(usage, "output_tokens", 0),
"total_tokens": usage.get("total_tokens", 0) if isinstance(usage, dict) else getattr(usage, "total_tokens", 0),
+ "cache_creation_tokens": usage.get("cache_creation_tokens", 0) if isinstance(usage, dict) else getattr(usage, "cache_creation_tokens", 0),
+ "cache_read_tokens": usage.get("cache_read_tokens", 0) if isinstance(usage, dict) else getattr(usage, "cache_read_tokens", 0),
+ "reasoning_tokens": usage.get("reasoning_tokens", 0) if isinstance(usage, dict) else getattr(usage, "reasoning_tokens", 0),
}
try:
@@ -1200,8 +1073,8 @@ class AIService:
config = get_provider_configs().get(provider)
if not config:
# Provide helpful error for Cerebras if import failed
- if provider == "cerebras" and not CEREBRAS_AVAILABLE:
- error_msg = f"Cerebras provider not available: {CEREBRAS_IMPORT_ERROR or 'langchain-cerebras package not installed'}"
+ if provider == "cerebras" and not _is_cerebras_available():
+ error_msg = "Cerebras provider not available: langchain-cerebras package not installed"
logger.error(error_msg)
raise ValueError(error_msg)
raise ValueError(f"Unsupported provider: {provider}")
@@ -1403,11 +1276,6 @@ class AIService:
provider = detect_ai_provider(node_type, flattened)
- # Some APIs return owner-prefixed model IDs (e.g. "openai/gpt-oss-120b")
- # but chat API expects flat IDs — strip prefix for non-OpenRouter providers
- if provider != "openrouter" and "/" in model:
- model = model.split("/", 1)[-1]
-
# Build thinking config from parameters
thinking_config = None
if flattened.get("thinking_enabled"):
@@ -1533,15 +1401,15 @@ class AIService:
context: Optional[Dict[str, Any]] = None,
database=None,
) -> Dict[str, Any]:
- """Execute AI Agent via the plain-async agent loop.
+ """Execute AI Agent via the shared native SDK loop.
- Drives :func:`_run_agent_loop` with the LangChain chat model returned by
- :meth:`create_model`. Each iteration: invoke the model, dispatch any
- ``tool_calls`` through the supplied closure, append ``ToolMessage`` results,
- loop until a final response (or the configured recursion limit hits).
+ Each iteration invokes ``ChatUnifier``, appends the exact replayable
+ assistant message, dispatches tool calls, appends native tool-result
+ messages, and continues until a final response or the configured
+ recursion limit.
Features:
- - Tool calling via ``chat_model.bind_tools`` + the shared tool dispatcher
+ - Provider-neutral tool calling through ``AgentToolSpec``
- Real-time status broadcasts for UI animations
Args:
@@ -1555,10 +1423,6 @@ class AIService:
workflow_id: Optional workflow ID for scoped status broadcasts
context: Optional execution context with nodes, edges for nested agent delegation
"""
- import openai # local: bound before the try whose except clause needs it
-
- from langchain_core.messages import HumanMessage, SystemMessage
-
start_time = time.time()
provider = "unknown"
model = "unknown"
@@ -1629,7 +1493,7 @@ class AIService:
enabled=True,
budget=int(flattened.get("thinking_budget", 2048)),
effort=flattened.get("reasoning_effort", "medium"),
- level=flattened.get("thinking_level", "medium"),
+ level=flattened.get("thinking_level"),
format=flattened.get("reasoning_format", "parsed"),
)
logger.debug(f"[Agent] Thinking enabled: budget={thinking_config.budget}, effort={thinking_config.effort}")
@@ -1639,13 +1503,6 @@ class AIService:
# Broadcast: Initializing model
await broadcast_status("initializing", {"message": f"Initializing {provider} model...", "provider": provider, "model": model, "active_skills": [], "last_skills": [], "last_tool_name": None, "last_capability": None})
- # Check for proxy URL (stored as {provider}_proxy credential)
- proxy_url = await self.auth.get_api_key(f"{provider}_proxy")
-
- # Create LLM using the provider from node configuration
- logger.debug(f"[Agent] Creating {provider} model: {model}")
- chat_model = self.create_model(provider, api_key, model, temperature, max_tokens, thinking_config, proxy_url)
-
# Build initial messages for state. The SystemMessage is
# PREPENDED after tool building (below), because the
# delegation-guidance block at the end of the tool-building
@@ -1656,7 +1513,7 @@ class AIService:
# and the LLM would never see the delegation contract,
# which is exactly what previously broke ``aiAgent``-driven
# delegation through the ``input-tools`` handle.
- initial_messages: List[BaseMessage] = []
+ initial_messages: List[NativeMessage] = []
# Add memory history from connected simpleMemory node (markdown-based)
session_id = None
@@ -1676,17 +1533,31 @@ class AIService:
# If long-term memory enabled, retrieve relevant context
if memory_data.get("long_term_enabled"):
- store = _get_memory_vector_store(session_id)
- if store:
- try:
+ try:
+ store = await _get_configured_memory_vector_store(
+ session_id,
+ memory_data,
+ self.auth,
+ )
+ if store:
k = memory_data.get("retrieval_count", 3)
- docs = store.similarity_search(prompt, k=k)
+ docs = await store.similarity_search(prompt, k=k)
if docs:
- context = "\n---\n".join(d.page_content for d in docs)
- initial_messages.append(SystemMessage(content=f"Relevant past context:\n{context}"))
+ retrieved_context = "\n---\n".join(
+ d.page_content for d in docs
+ )
+ initial_messages.append(
+ NativeMessage(
+ role="system",
+ content=(
+ "Relevant past context:\n"
+ f"{retrieved_context}"
+ ),
+ )
+ )
logger.info(f"[Agent Memory] Retrieved {len(docs)} relevant memories from long-term store")
- except Exception as e:
- logger.debug(f"[Agent Memory] Long-term retrieval skipped: {e}")
+ except Exception as e:
+ logger.debug(f"[Agent Memory] Long-term retrieval skipped: {e}")
# Add parsed history messages
initial_messages.extend(history_messages)
@@ -1700,7 +1571,7 @@ class AIService:
)
# Add current user prompt
- initial_messages.append(HumanMessage(content=prompt))
+ initial_messages.append(NativeMessage(role="user", content=prompt))
# Build tools if provided
tools = []
@@ -1777,7 +1648,10 @@ class AIService:
# See the comment at the original ``initial_messages`` declaration
# for why this can't happen earlier.
if system_message:
- initial_messages.insert(0, SystemMessage(content=system_message))
+ initial_messages.insert(
+ 0,
+ NativeMessage(role="system", content=system_message),
+ )
# Create tool executor callback. Tool-node status lifecycle
# (executing/success/error) is owned by ``handlers.tools.execute_tool``
@@ -1809,7 +1683,7 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
)
# Inject services + graph context so execute_tool can scope its
@@ -1854,7 +1728,7 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
)
return result
@@ -1887,11 +1761,11 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
error_code=type(e).__name__,
)
- # Re-raise to let _run_agent_loop surface the error to the LLM
+ # Re-raise so the native loop can surface the error to the LLM.
raise
# Auto-rebind toggle: when the user enables "Auto-Rebind Tools
@@ -1915,10 +1789,10 @@ class AIService:
async def _rebind_from_operations(operations: List[Dict[str, Any]]) -> List[Any]:
"""Translate workflow_ops add_node ops (component_kind='tool')
- into fresh ``StructuredTool``s via the canonical
+ into fresh :class:`AgentToolSpec` values via the canonical
:meth:`_build_tool_from_node` helper. The returned list is
- appended to ``_run_agent_loop``'s ``current_tools`` and
- rebound onto the LLM so the LLM can invoke the new tool in
+ appended to the native loop's current tool surface so the LLM
+ can invoke the new tool in
its NEXT iteration without a Run-stop-Run cycle.
Tool configs are folded into the existing ``tool_configs``
@@ -2028,9 +1902,15 @@ class AIService:
max_iterations=recursion_limit,
)
- final_state = await _run_agent_loop(
- chat_model,
- initial_messages,
+ final_state = await run_native_agent_loop(
+ self.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ thinking=thinking_config,
+ initial_messages=initial_messages,
tools=tools if tools else None,
tool_executor=tool_executor if tools else None,
max_iterations=recursion_limit,
@@ -2046,8 +1926,7 @@ class AIService:
raise ValueError("No response generated from agent")
# Handle different content formats (Gemini can return list of content blocks)
- raw_content = ai_response.content
- response_content = self._extract_text_content(raw_content, ai_response)
+ response_content = self._extract_text_content(ai_response.content, ai_response)
iterations = final_state.get("iteration", 1)
# Get accumulated thinking content from state
@@ -2064,7 +1943,7 @@ class AIService:
node_id=node_id,
provider=provider,
model=model,
- ai_response=ai_response,
+ ai_response=final_state.get("usage") or ai_response,
all_messages=all_messages,
broadcaster=broadcaster,
workflow_id=workflow_id,
@@ -2072,6 +1951,9 @@ class AIService:
api_key=api_key,
memory_node_id=memory_data.get("node_id") if memory_data else None,
)
+ _accumulate_compaction_usage(
+ final_state, compaction_result
+ )
# Save to memory if connected (markdown-based with optional vector DB)
# Only save non-empty messages using standardized validation
@@ -2122,13 +2004,17 @@ class AIService:
# Store removed messages in long-term vector DB
if removed_texts and memory_data.get("long_term_enabled"):
- store = _get_memory_vector_store(session_id)
- if store:
- try:
- store.add_texts(removed_texts)
+ try:
+ store = await _get_configured_memory_vector_store(
+ session_id,
+ memory_data,
+ self.auth,
+ )
+ if store:
+ await store.add_texts(removed_texts)
logger.info(f"[Agent Memory] Archived {len(removed_texts)} messages to long-term store")
- except Exception as e:
- logger.warning(f"[Agent Memory] Failed to archive to vector store: {e}")
+ except Exception as e:
+ logger.warning(f"[Agent Memory] Failed to archive to vector store: {e}")
logger.info(f"[Agent Memory] Saved markdown to memory node '{memory_node_id}'")
@@ -2174,12 +2060,9 @@ class AIService:
"timestamp": datetime.now().isoformat(),
}
- except openai.OpenAIError as e:
- # See execute_chat for the rationale — typed SDK errors are
- # user-correctable and re-raised as NodeUserError so BaseNode
- # logs at WARN without a traceback.
+ except NodeUserError as e:
log_api_call(logger, provider, model, "agent", False, error=str(e))
- raise NodeUserError(str(e)) from e
+ raise
except Exception as e:
logger.error("[Agent] AI agent execution failed", node_id=node_id, error=str(e))
@@ -2228,10 +2111,6 @@ class AIService:
workflow_id: Optional workflow ID for scoped status broadcasts
context: Optional execution context with nodes, edges for nested agent delegation
"""
- import openai # local: bound before the try whose except clause needs it
-
- from langchain_core.messages import HumanMessage, SystemMessage
-
start_time = time.time()
provider = "unknown"
model = "unknown"
@@ -2353,7 +2232,7 @@ class AIService:
logger.debug(f"[ChatAgent] Total tools available: {len(all_tools)}")
# Debug: log all tool schemas to verify they're correct
for t in all_tools:
- schema = t.get_input_schema().model_json_schema()
+ schema = t.parameters
logger.debug(f"[ChatAgent] Tool '{t.name}' schema: {schema}")
# Flatten options collection from frontend
@@ -2391,7 +2270,7 @@ class AIService:
enabled=True,
budget=int(flattened.get("thinking_budget", 2048)),
effort=flattened.get("reasoning_effort", "medium"),
- level=flattened.get("thinking_level", "medium"),
+ level=flattened.get("thinking_level"),
format=flattened.get("reasoning_format", "parsed"),
)
@@ -2400,16 +2279,10 @@ class AIService:
# Broadcast: Initializing
await broadcast_status("initializing", {"message": f"Initializing {provider} model...", "provider": provider, "model": model, "active_skills": [], "last_skills": [], "last_tool_name": None, "last_capability": None})
- # Check for proxy URL (stored as {provider}_proxy credential)
- proxy_url = await self.auth.get_api_key(f"{provider}_proxy")
-
- # Create chat model
- chat_model = self.create_model(provider, api_key, model, temperature, max_tokens, thinking_config, proxy_url)
-
# Build messages
- messages: List[BaseMessage] = []
+ messages: List[NativeMessage] = []
if system_message:
- messages.append(SystemMessage(content=system_message))
+ messages.append(NativeMessage(role="system", content=system_message))
# Load memory history if connected (markdown-based like AI Agent)
session_id = None
@@ -2429,17 +2302,31 @@ class AIService:
# If long-term memory enabled, retrieve relevant context
if memory_data.get("long_term_enabled"):
- store = _get_memory_vector_store(session_id)
- if store:
- try:
+ try:
+ store = await _get_configured_memory_vector_store(
+ session_id,
+ memory_data,
+ self.auth,
+ )
+ if store:
k = memory_data.get("retrieval_count", 3)
- docs = store.similarity_search(prompt, k=k)
+ docs = await store.similarity_search(prompt, k=k)
if docs:
- context = "\n---\n".join(d.page_content for d in docs)
- messages.append(SystemMessage(content=f"Relevant past context:\n{context}"))
+ retrieved_context = "\n---\n".join(
+ d.page_content for d in docs
+ )
+ messages.append(
+ NativeMessage(
+ role="system",
+ content=(
+ "Relevant past context:\n"
+ f"{retrieved_context}"
+ ),
+ )
+ )
logger.info(f"[ChatAgent Memory] Retrieved {len(docs)} relevant memories from long-term store")
- except Exception as e:
- logger.debug(f"[ChatAgent Memory] Long-term retrieval skipped: {e}")
+ except Exception as e:
+ logger.debug(f"[ChatAgent Memory] Long-term retrieval skipped: {e}")
# Add parsed history messages
messages.extend(history_messages)
@@ -2452,7 +2339,7 @@ class AIService:
)
# Add current prompt
- messages.append(HumanMessage(content=prompt))
+ messages.append(NativeMessage(role="user", content=prompt))
# Broadcast: Invoking LLM
await broadcast_status(
@@ -2503,7 +2390,7 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
)
# Inject services + graph context so execute_tool can scope its
@@ -2544,7 +2431,7 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
)
logger.debug(f"[ChatAgent] Tool executed successfully: {tool_name}")
return result
@@ -2565,7 +2452,7 @@ class AIService:
root_execution_id=str((context or {}).get("root_execution_id") or "") or None,
target_node_id=str(tool_node_id or "") or None,
provider=provider,
- invocation_source="legacy",
+ invocation_source="native",
error_code=type(e).__name__,
)
return {"error": str(e)}
@@ -2663,9 +2550,15 @@ class AIService:
max_iterations=recursion_limit,
)
- final_state = await _run_agent_loop(
- chat_model,
- messages,
+ final_state = await run_native_agent_loop(
+ self.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ thinking=thinking_config,
+ initial_messages=messages,
tools=all_tools,
tool_executor=chat_tool_executor,
max_iterations=recursion_limit,
@@ -2680,27 +2573,36 @@ class AIService:
if not ai_response or not hasattr(ai_response, "content"):
raise ValueError("No response generated from agent")
- raw_content = ai_response.content
- response_content = self._extract_text_content(raw_content, ai_response)
+ response_content = self._extract_text_content(ai_response.content, ai_response)
iterations = final_state.get("iteration", 1)
thinking_content = final_state.get("thinking_content")
else:
- # Simple invoke without tools
- response = await chat_model.ainvoke(messages)
-
- # Extract response content
- raw_content = response.content
- response_content = self._extract_text_content(raw_content, response)
-
- # Extract thinking content if available
- _, thinking_content = extract_thinking_from_response(response)
+ final_state = await run_native_agent_loop(
+ self.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ model=model,
+ temperature=temperature,
+ max_tokens=max_tokens,
+ thinking=thinking_config,
+ initial_messages=messages,
+ max_iterations=1,
+ )
+ all_messages = final_state["messages"]
+ ai_response = all_messages[-1] if all_messages else None
+ if ai_response is None:
+ raise ValueError("No response generated from agent")
+ response_content = self._extract_text_content(
+ ai_response.content,
+ ai_response,
+ )
+ thinking_content = final_state.get("thinking_content")
logger.info(f"[ChatAgent] Response generated, thinking={'yes' if thinking_content else 'no'}, iterations={iterations}")
# Track token usage if memory connected (for compaction service)
- # Use ai_response for tools path, response for simple invoke
- # Also triggers compaction if threshold exceeded
- response_msg = ai_response if all_tools else response
+ # Also triggers compaction if threshold exceeded.
+ response_msg = final_state.get("usage") or ai_response
compaction_result = None
if session_id and response_msg:
compaction_result = await self._track_token_usage(
@@ -2709,13 +2611,16 @@ class AIService:
provider=provider,
model=model,
ai_response=response_msg,
- all_messages=all_messages if all_tools else messages + [response_msg],
+ all_messages=all_messages,
broadcaster=broadcaster,
workflow_id=workflow_id,
memory_content=memory_content,
api_key=api_key,
memory_node_id=memory_data.get("node_id") if memory_data else None,
)
+ _accumulate_compaction_usage(
+ final_state, compaction_result
+ )
# Save to memory if connected (markdown-based like AI Agent)
if (
@@ -2754,13 +2659,17 @@ class AIService:
# Store removed messages in long-term vector DB
if removed_texts and memory_data.get("long_term_enabled"):
- store = _get_memory_vector_store(session_id)
- if store:
- try:
- store.add_texts(removed_texts)
+ try:
+ store = await _get_configured_memory_vector_store(
+ session_id,
+ memory_data,
+ self.auth,
+ )
+ if store:
+ await store.add_texts(removed_texts)
logger.info(f"[ChatAgent Memory] Archived {len(removed_texts)} messages to long-term store")
- except Exception as e:
- logger.warning(f"[ChatAgent Memory] Failed to archive to vector store: {e}")
+ except Exception as e:
+ logger.warning(f"[ChatAgent Memory] Failed to archive to vector store: {e}")
logger.info(f"[ChatAgent Memory] Saved markdown to memory node '{memory_node_id}'")
@@ -2823,10 +2732,9 @@ class AIService:
"timestamp": datetime.now().isoformat(),
}
- except openai.OpenAIError as e:
- # Typed SDK error — see execute_chat for rationale.
+ except NodeUserError as e:
log_api_call(logger, provider, model, "chat_agent", False, error=str(e))
- raise NodeUserError(str(e)) from e
+ raise
except Exception as e:
logger.error("[ChatAgent] Execution failed", node_id=node_id, error=str(e))
@@ -2847,7 +2755,7 @@ class AIService:
await clear_skill_turn(workflow_id or "", str((context or {}).get("execution_id") or ""), node_id)
async def _build_tool_from_node(self, tool_info: Dict[str, Any]) -> tuple:
- """Convert a node configuration into a LangChain StructuredTool.
+ """Convert a node configuration into a provider-neutral tool spec.
Uses database-stored schema as source of truth if available, otherwise
falls back to dynamic schema generation.
@@ -2868,10 +2776,8 @@ class AIService:
tool_info: Dict containing node_id, node_type, parameters, and label
Returns:
- Tuple of (StructuredTool, config_dict) or (None, None) on failure
+ Tuple of (AgentToolSpec, config_dict) or (None, None) on failure
"""
- from langchain_core.tools import StructuredTool
-
# Built-in / aggregator pseudo-types — no plugin class, no ClassVar.
# These must stay as an explicit fallback dict (Wave 12 D5).
_PSEUDO_TOOL_FALLBACK = {
@@ -2985,12 +2891,6 @@ class AIService:
schema_params["db_schema_config"] = db_schema["schema_config"]
schema = self._get_tool_schema(node_type, schema_params)
- # Create StructuredTool - the func is a placeholder, actual execution via tool_executor
- def placeholder_func(**kwargs):
- return kwargs
-
- tool = StructuredTool.from_function(name=tool_name, description=tool_description, func=placeholder_func, args_schema=schema)
-
# Build config dict - include connected_services for toolkit nodes
config = {
"node_type": node_type,
@@ -3000,6 +2900,19 @@ class AIService:
"connected_services": connected_services, # Pass through for execution
}
+ from services.plugin.tool import inline_schema_refs
+
+ parameters = inline_schema_refs(schema.model_json_schema())
+ tool = AgentToolSpec(
+ definition=ToolDef(
+ name=tool_name,
+ description=tool_description,
+ parameters=parameters,
+ ),
+ args_schema=schema,
+ execution=config,
+ )
+
logger.debug(f"[Agent] Built tool '{tool_name}' with node_id={node_id}")
return tool, config
diff --git a/server/services/cli_agent/service.py b/server/services/cli_agent/service.py
index 3a0d63c5..84265497 100644
--- a/server/services/cli_agent/service.py
+++ b/server/services/cli_agent/service.py
@@ -648,12 +648,20 @@ class AICliService:
if connected_memory.get("long_term_enabled") and removed_texts:
from services.memory.vector_store import get_memory_vector_store
+ from services.plugin.deps import get_auth_service
- store = get_memory_vector_store(
+ store = await get_memory_vector_store(
connected_memory.get("session_id") or "default",
+ provider=connected_memory.get(
+ "embedding_provider",
+ "huggingface",
+ ),
+ model=connected_memory.get("embedding_model"),
+ endpoint=connected_memory.get("embedding_endpoint"),
+ auth_service=get_auth_service(),
)
if store is not None:
- await asyncio.to_thread(store.add_texts, removed_texts)
+ await store.add_texts(removed_texts)
@staticmethod
async def _resolve_repo_root(
diff --git a/server/services/compaction.py b/server/services/compaction.py
index 5af131af..b4efc979 100644
--- a/server/services/compaction.py
+++ b/server/services/compaction.py
@@ -8,6 +8,7 @@ Threshold strategy: per-session custom_threshold > model-aware threshold > globa
Model-aware threshold = 50% of model's context window (e.g., 100K for a 200K model).
"""
+from dataclasses import asdict
from pydantic import BaseModel
from datetime import datetime, timezone
from typing import Dict, Any, Optional, TYPE_CHECKING
@@ -132,31 +133,12 @@ class CompactionService:
async def track(self, session_id: str, node_id: str, provider: str, model: str, usage: Dict[str, int]) -> Dict[str, Any]:
"""Track token usage and cost, return compaction status."""
- # Calculate cost using PricingService
- pricing_service = get_pricing_service()
- cost = pricing_service.calculate_cost(
+ cost = await self._record_usage_metric(
+ session_id=session_id,
+ node_id=node_id,
provider=provider,
model=model,
- input_tokens=usage.get("input_tokens", 0),
- output_tokens=usage.get("output_tokens", 0),
- cache_read_tokens=usage.get("cache_read_tokens", 0),
- cache_creation_tokens=usage.get("cache_creation_tokens", 0),
- reasoning_tokens=usage.get("reasoning_tokens", 0),
- )
-
- # Save token metric with cost fields
- await self._db.save_token_metric(
- {
- "session_id": session_id,
- "node_id": node_id,
- "provider": provider,
- "model": model,
- **usage,
- "input_cost": cost["input_cost"],
- "output_cost": cost["output_cost"],
- "cache_cost": cost["cache_cost"],
- "total_cost": cost["total_cost"],
- }
+ usage=usage,
)
state = await self._db.get_or_create_session_token_state(session_id)
@@ -199,6 +181,46 @@ class CompactionService:
"needs_compaction": self._config.enabled and new_total >= threshold,
}
+ async def _record_usage_metric(
+ self,
+ *,
+ session_id: str,
+ node_id: str,
+ provider: str,
+ model: str,
+ usage: Dict[str, int],
+ iteration: int = 1,
+ ) -> Dict[str, float]:
+ """Persist billed usage without changing active-context counters."""
+
+ pricing_service = get_pricing_service()
+ cost = pricing_service.calculate_cost(
+ provider=provider,
+ model=model,
+ input_tokens=usage.get("input_tokens", 0),
+ output_tokens=usage.get("output_tokens", 0),
+ cache_read_tokens=usage.get("cache_read_tokens", 0),
+ cache_creation_tokens=usage.get("cache_creation_tokens", 0),
+ reasoning_tokens=usage.get("reasoning_tokens", 0),
+ )
+
+ # Save token metric with cost fields
+ await self._db.save_token_metric(
+ {
+ "session_id": session_id,
+ "node_id": node_id,
+ "provider": provider,
+ "model": model,
+ "iteration": iteration,
+ **usage,
+ "input_cost": cost["input_cost"],
+ "output_cost": cost["output_cost"],
+ "cache_cost": cost["cache_cost"],
+ "total_cost": cost["total_cost"],
+ }
+ )
+ return cost
+
async def record(
self, session_id: str, node_id: str, provider: str, model: str, tokens_before: int, tokens_after: int, summary: Optional[str] = None
) -> None:
@@ -272,6 +294,8 @@ class CompactionService:
provider: str,
api_key: str,
model: str,
+ *,
+ explicit_max_retries: int = 2,
) -> Dict[str, Any]:
"""Perform compaction by summarizing conversation history.
@@ -286,6 +310,7 @@ class CompactionService:
state = await self._db.get_or_create_session_token_state(session_id)
tokens_before = state["cumulative_total"]
+ compaction_usage: Dict[str, int] = {}
try:
prompt = f"""Summarize this conversation into a structured format:
@@ -318,24 +343,56 @@ Provide a concise but complete summary."""
model_max = get_model_registry().get_max_output_tokens(model, provider)
summary_tokens = min(4096, model_max)
- llm = self._ai_service.create_model(provider=provider, api_key=api_key, model=model, temperature=0.3, max_tokens=summary_tokens)
-
- from langchain_core.messages import HumanMessage
-
- response = await llm.ainvoke([HumanMessage(content=prompt)])
- raw_content = response.content if hasattr(response, "content") else str(response)
- summary = _extract_text_from_response(raw_content)
+ if self._ai_service.chat_unifier is None:
+ raise RuntimeError("ChatUnifier is required for compaction")
+
+ from services.agent_runtime import run_native_llm_step
+ from services.llm.protocol import Message
+
+ response = await run_native_llm_step(
+ self._ai_service.chat_unifier,
+ provider=provider,
+ api_key=api_key,
+ messages=[Message(role="user", content=prompt)],
+ model=model,
+ temperature=0.3,
+ max_tokens=summary_tokens,
+ sdk_max_retries=0,
+ explicit_max_retries=explicit_max_retries,
+ )
+ compaction_usage = asdict(response.usage)
+ await self._record_usage_metric(
+ session_id=session_id,
+ node_id=node_id,
+ provider=provider,
+ model=model,
+ usage=compaction_usage,
+ # Iteration zero distinguishes the summarizer call in the
+ # existing metric schema without adding a public field.
+ iteration=0,
+ )
+ summary = _extract_text_from_response(response.content)
new_memory = f"# Conversation Summary (Compacted)\n*Generated: {datetime.now(timezone.utc).isoformat()}*\n\n{summary}"
await self.record(session_id, node_id, provider, model, tokens_before, 0, new_memory)
logger.info(f"[Compaction] Session {session_id}: {tokens_before} -> 0 tokens")
- return {"success": True, "summary": new_memory, "tokens_before": tokens_before, "tokens_after": 0}
+ return {
+ "success": True,
+ "summary": new_memory,
+ "tokens_before": tokens_before,
+ "tokens_after": 0,
+ "usage": compaction_usage,
+ }
except Exception as e:
logger.error(f"[Compaction] Failed: {e}")
- return {"success": False, "error": str(e)}
+ return {
+ "success": False,
+ "error": str(e),
+ "usage": compaction_usage,
+ }
_service: Optional[CompactionService] = None
diff --git a/server/services/llm/__init__.py b/server/services/llm/__init__.py
index 6bcb6a59..694a8041 100644
--- a/server/services/llm/__init__.py
+++ b/server/services/llm/__init__.py
@@ -12,15 +12,27 @@ from services.llm.config import (
build_headers,
)
from services.llm.protocol import (
+ MESSAGE_WIRE_VERSION,
+ NATIVE_MESSAGE_WIRE_VERSIONS,
+ SUPPORTED_MESSAGE_WIRE_VERSIONS,
+ MessageWireV2,
ThinkingConfig,
ToolDef,
ToolCall,
+ ContentBlock,
Message,
Usage,
LLMResponse,
+ LLMError,
+ LLMErrorCategory,
LLMProvider,
+ message_to_wire,
+ message_from_wire,
+ messages_to_wire,
+ messages_from_wire,
)
from services.llm.messages import is_valid_message_content, filter_empty_messages
+from services.llm.schema import compile_tool_schema
from services.llm.registry import (
ProviderSpec,
register_provider,
@@ -48,16 +60,28 @@ __all__ = [
"resolve_temperature",
"build_headers",
# Protocol types
+ "MESSAGE_WIRE_VERSION",
+ "NATIVE_MESSAGE_WIRE_VERSIONS",
+ "SUPPORTED_MESSAGE_WIRE_VERSIONS",
+ "MessageWireV2",
"ThinkingConfig",
"ToolDef",
"ToolCall",
+ "ContentBlock",
"Message",
"Usage",
"LLMResponse",
+ "LLMError",
+ "LLMErrorCategory",
"LLMProvider",
+ "message_to_wire",
+ "message_from_wire",
+ "messages_to_wire",
+ "messages_from_wire",
# Messages
"is_valid_message_content",
"filter_empty_messages",
+ "compile_tool_schema",
# Plugin registry + unifier (Phase A — single SERVICE facade)
"ProviderSpec",
"register_provider",
diff --git a/server/services/llm/config.py b/server/services/llm/config.py
index a2ef1f2e..dc032a15 100644
--- a/server/services/llm/config.py
+++ b/server/services/llm/config.py
@@ -114,12 +114,12 @@ def detect_provider_from_model(model: str) -> str:
def is_model_valid_for_provider(model: str, provider: str) -> bool:
- # Open-world providers — OpenRouter is a multi-vendor proxy, ollama
- # and lmstudio serve user-installed local models whose names don't
- # match any "lmstudio"/"ollama" substring. Treat as always-valid;
- # the upstream API will 404 a genuinely missing model. See the
- # mirror in services/ai.py for the full rationale.
- if provider in ("openrouter", "ollama", "lmstudio"):
+ # Open-world providers — OpenRouter is a multi-vendor proxy, Ollama and
+ # LM Studio serve user-installed models, and Groq's owner-qualified model
+ # IDs (for example ``openai/gpt-oss-120b``) do not contain "groq".
+ # Treat them as provider-selected; the upstream API will return a clear
+ # 404 for a genuinely missing model.
+ if provider in ("openrouter", "ollama", "lmstudio", "groq"):
return True
cfg = PROVIDER_CONFIGS.get(provider)
if not cfg:
@@ -186,7 +186,12 @@ def resolve_temperature(params: dict, model: str, provider: str, thinking_enable
user_val = params.get("temperature")
if user_val is None:
- user_val = registry.get_agent_defaults()["default_temperature"]
+ user_val = registry.get_agent_defaults().get(
+ "default_temperature",
+ LLM_DEFAULTS.get("agent", {}).get(
+ "default_temperature", 0.7
+ ),
+ )
user_temp = float(user_val)
if registry.is_reasoning_model(model, provider):
diff --git a/server/services/llm/protocol.py b/server/services/llm/protocol.py
index a9e24ac4..f0fcfa5a 100644
--- a/server/services/llm/protocol.py
+++ b/server/services/llm/protocol.py
@@ -1,12 +1,89 @@
-"""Shared types and provider protocol for native LLM providers.
+"""Shared, durable types for native LLM providers.
-All providers implement LLMProvider (structural typing via Protocol).
+The dataclasses in this module deliberately contain only JSON-safe data. SDK
+objects are useful while normalising a response, but must never leak into the
+agent/Temporal message history. ``MessageWireV2`` is represented as a plain
+dictionary so it can be recorded by Temporal without a custom payload codec.
+
+All providers implement :class:`LLMProvider` (structural typing via Protocol).
"""
from __future__ import annotations
+import base64
+import json
+import math
from dataclasses import dataclass, field
-from typing import Any, Dict, List, Optional, Protocol, runtime_checkable
+from enum import Enum
+from typing import (
+ Any,
+ Dict,
+ Iterable,
+ List,
+ Mapping,
+ Optional,
+ Protocol,
+ TypedDict,
+ runtime_checkable,
+)
+
+
+MESSAGE_WIRE_VERSION = 2
+"""Current durable message representation version."""
+
+SUPPORTED_MESSAGE_WIRE_VERSIONS = frozenset({1, 2})
+"""Every durable version this worker can decode; never derive from current."""
+
+NATIVE_MESSAGE_WIRE_VERSIONS = frozenset({2})
+"""Versioned native-agent formats accepted by the Temporal activity."""
+
+MAX_PROVIDER_STATE_DEPTH = 20
+BINARY_STATE_MARKER = "__opencompany_bytes_base64__"
+"""Marker used to persist SDK byte strings in ordinary JSON state."""
+
+
+def encode_binary_state(value: Any) -> Any:
+ """Encode bytes for durable provider state; preserve other values exactly."""
+
+ if isinstance(value, memoryview):
+ value = value.tobytes()
+ elif isinstance(value, bytearray):
+ value = bytes(value)
+ if isinstance(value, bytes):
+ return {
+ BINARY_STATE_MARKER: base64.b64encode(value).decode("ascii")
+ }
+ return value
+
+
+def decode_binary_state(value: Any) -> Any:
+ """Decode an exact binary marker while leaving ordinary strings untouched."""
+
+ if not (
+ isinstance(value, Mapping)
+ and set(value) == {BINARY_STATE_MARKER}
+ ):
+ return value
+ encoded = value.get(BINARY_STATE_MARKER)
+ if not isinstance(encoded, str):
+ raise ValueError("Invalid durable binary-state marker")
+ try:
+ return base64.b64decode(encoded, validate=True)
+ except (ValueError, TypeError) as exc:
+ raise ValueError("Invalid durable binary-state base64") from exc
+
+
+class MessageWireV2(TypedDict):
+ """Versioned JSON object recorded in memory and Temporal histories."""
+
+ version: int
+ role: str
+ content: str
+ blocks: List[Dict[str, Any]]
+ tool_calls: List[Dict[str, Any]]
+ tool_call_id: Optional[str]
+ name: Optional[str]
+ provider_state: Dict[str, Any]
# ---------------------------------------------------------------------------
@@ -52,17 +129,91 @@ class ToolCall:
id: str
name: str
args: Dict[str, Any]
+ # Providers occasionally emit invalid or non-object JSON. Keep the raw
+ # value so a caller can return a deterministic tool error and, critically,
+ # replay the exact assistant turn back to the same provider.
+ raw_arguments: Optional[str] = None
+ parse_error: Optional[str] = None
+
+ @classmethod
+ def from_raw(cls, *, id: str, name: str, arguments: Any) -> "ToolCall":
+ """Build a call without allowing malformed model output to crash.
+
+ Valid JSON objects populate ``args``. Any other value is preserved in
+ ``raw_arguments`` and described by ``parse_error``.
+ """
+
+ if isinstance(arguments, Mapping):
+ return cls(id=id, name=name, args=dict(arguments))
+
+ raw = arguments if isinstance(arguments, str) else _json_dump(arguments)
+ try:
+ parsed = json.loads(raw)
+ except (TypeError, ValueError) as exc:
+ return cls(
+ id=id,
+ name=name,
+ args={},
+ raw_arguments=raw,
+ parse_error=f"Invalid JSON tool arguments: {exc}",
+ )
+ if not isinstance(parsed, dict):
+ return cls(
+ id=id,
+ name=name,
+ args={},
+ raw_arguments=raw,
+ parse_error=(
+ "Tool arguments must decode to a JSON object, "
+ f"not {type(parsed).__name__}"
+ ),
+ )
+ return cls(id=id, name=name, args=parsed, raw_arguments=raw)
+
+
+@dataclass
+class ContentBlock:
+ """Provider-neutral, ordered content within a message.
+
+ ``type`` is intentionally an open string rather than an enum: providers
+ can add a new block without making old workers unable to decode history.
+ Known values are ``text``, ``reasoning``, ``tool_call`` and
+ ``tool_result``.
+ """
+
+ type: str
+ text: str = ""
+ tool_call: Optional[ToolCall] = None
+ tool_call_id: Optional[str] = None
+ name: Optional[str] = None
+ metadata: Dict[str, Any] = field(default_factory=dict)
@dataclass
class Message:
- """Normalized chat message."""
+ """Normalized chat message.
+
+ The original flat fields remain the compatibility surface. ``blocks``
+ preserves provider response ordering and ``provider_state`` contains the
+ minimal same-provider continuation metadata (for example an Anthropic
+ thinking signature or Gemini thought signature).
+ """
role: str # system | user | assistant | tool
content: str = ""
tool_calls: List[ToolCall] = field(default_factory=list)
tool_call_id: Optional[str] = None
name: Optional[str] = None
+ blocks: List[ContentBlock] = field(default_factory=list)
+ provider_state: Dict[str, Any] = field(default_factory=dict)
+
+ def __post_init__(self) -> None:
+ if not self.blocks:
+ self.blocks = _default_blocks(self)
+ # Fail close at construction time for provider-produced state. This
+ # prevents an SDK response object or unbounded blob entering a durable
+ # workflow history.
+ self.provider_state = _validated_provider_state(self.provider_state)
@dataclass
@@ -76,6 +227,33 @@ class Usage:
cache_read_tokens: int = 0
reasoning_tokens: int = 0
+ def __post_init__(self) -> None:
+ for field_name in (
+ "input_tokens",
+ "output_tokens",
+ "total_tokens",
+ "cache_creation_tokens",
+ "cache_read_tokens",
+ "reasoning_tokens",
+ ):
+ setattr(self, field_name, _safe_token_count(getattr(self, field_name)))
+ if not self.total_tokens:
+ self.total_tokens = self.input_tokens + self.output_tokens
+
+ def __add__(self, other: "Usage") -> "Usage":
+ if not isinstance(other, Usage):
+ return NotImplemented
+ return Usage(
+ input_tokens=self.input_tokens + other.input_tokens,
+ output_tokens=self.output_tokens + other.output_tokens,
+ total_tokens=self.total_tokens + other.total_tokens,
+ cache_creation_tokens=(
+ self.cache_creation_tokens + other.cache_creation_tokens
+ ),
+ cache_read_tokens=self.cache_read_tokens + other.cache_read_tokens,
+ reasoning_tokens=self.reasoning_tokens + other.reasoning_tokens,
+ )
+
@dataclass
class LLMResponse:
@@ -88,6 +266,437 @@ class LLMResponse:
model: str = ""
finish_reason: str = "stop"
raw: Any = None
+ assistant_message: Optional[Message] = None
+
+ def __post_init__(self) -> None:
+ """Keep the flat response API while exposing a replayable message."""
+
+ if self.assistant_message is None:
+ blocks: List[ContentBlock] = []
+ if self.thinking:
+ blocks.append(ContentBlock(type="reasoning", text=self.thinking))
+ if self.content:
+ blocks.append(ContentBlock(type="text", text=self.content))
+ blocks.extend(
+ ContentBlock(type="tool_call", tool_call=call)
+ for call in self.tool_calls
+ )
+ self.assistant_message = Message(
+ role="assistant",
+ content=self.content,
+ tool_calls=list(self.tool_calls),
+ blocks=blocks,
+ )
+ return
+
+ # A provider may populate only the canonical message. Preserve the
+ # convenience fields expected by existing chat callers.
+ if not self.content:
+ self.content = self.assistant_message.content
+ if not self.tool_calls:
+ self.tool_calls = list(self.assistant_message.tool_calls)
+ if self.thinking is None:
+ reasoning = [
+ block.text
+ for block in self.assistant_message.blocks
+ if block.type == "reasoning" and block.text
+ ]
+ self.thinking = "\n\n".join(reasoning) or None
+
+
+class LLMErrorCategory(str, Enum):
+ """Stable categories used by retry and user-error policies."""
+
+ AUTHENTICATION = "authentication"
+ PERMISSION = "permission"
+ RATE_LIMIT = "rate_limit"
+ INVALID_REQUEST = "invalid_request"
+ NOT_FOUND = "not_found"
+ CONTEXT_LENGTH = "context_length"
+ TIMEOUT = "timeout"
+ CONNECTION = "connection"
+ SERVER = "server"
+ UNKNOWN = "unknown"
+
+
+@dataclass
+class LLMError(Exception):
+ """Provider-independent structured SDK failure."""
+
+ message: str
+ provider: str
+ category: LLMErrorCategory = LLMErrorCategory.UNKNOWN
+ retryable: bool = False
+ status_code: Optional[int] = None
+ provider_code: Optional[str] = None
+ request_id: Optional[str] = None
+ retry_after: Optional[float] = None
+ retry_after_raw: Optional[str] = None
+
+ def __post_init__(self) -> None:
+ Exception.__init__(self, self.message)
+
+ @property
+ def user_message(self) -> str:
+ """Return a category-based message that is safe for public surfaces.
+
+ ``message`` intentionally retains the original SDK text for exception
+ chaining and operator diagnostics. Provider messages can include
+ request payload fragments, internal endpoint URLs, or credential
+ details, so execution boundaries must expose this property instead.
+ """
+
+ provider_names = {
+ "anthropic": "Anthropic",
+ "gemini": "Gemini",
+ "openai": "OpenAI",
+ "openrouter": "OpenRouter",
+ "groq": "Groq",
+ "cerebras": "Cerebras",
+ "xai": "xAI",
+ "deepseek": "DeepSeek",
+ "kimi": "Kimi",
+ "mistral": "Mistral",
+ "ollama": "Ollama",
+ "lmstudio": "LM Studio",
+ }
+ provider_key = str(self.provider or "").strip().lower()
+ provider = provider_names.get(
+ provider_key, "The language model provider"
+ )
+ provider_object = (
+ provider
+ if provider_key in provider_names
+ else "the language model provider"
+ )
+ category = (
+ self.category.value
+ if isinstance(self.category, LLMErrorCategory)
+ else str(self.category or LLMErrorCategory.UNKNOWN.value)
+ )
+ messages = {
+ LLMErrorCategory.AUTHENTICATION.value: (
+ f"{provider} authentication failed. "
+ "Check the configured API key."
+ ),
+ LLMErrorCategory.PERMISSION.value: (
+ f"{provider} denied this request. "
+ "Check account and model access."
+ ),
+ LLMErrorCategory.RATE_LIMIT.value: (
+ f"{provider} is rate-limiting requests. "
+ "Retry after a short delay."
+ ),
+ LLMErrorCategory.INVALID_REQUEST.value: (
+ f"{provider} rejected the model request configuration."
+ ),
+ LLMErrorCategory.NOT_FOUND.value: (
+ f"The configured {provider_object} model or endpoint "
+ "was not found."
+ ),
+ LLMErrorCategory.CONTEXT_LENGTH.value: (
+ f"The request exceeds the {provider_object} model "
+ "context window."
+ ),
+ LLMErrorCategory.TIMEOUT.value: (
+ f"The request to {provider_object} timed out."
+ ),
+ LLMErrorCategory.CONNECTION.value: (
+ f"Could not connect to {provider_object}."
+ ),
+ LLMErrorCategory.SERVER.value: (
+ f"{provider} is temporarily unavailable."
+ ),
+ LLMErrorCategory.UNKNOWN.value: (
+ f"{provider} request failed."
+ ),
+ }
+ return messages.get(category, messages[LLMErrorCategory.UNKNOWN.value])
+
+ @classmethod
+ def from_exception(cls, provider: str, exc: BaseException) -> "LLMError":
+ status = _optional_int(
+ getattr(exc, "status_code", None)
+ or getattr(getattr(exc, "response", None), "status_code", None)
+ or getattr(exc, "code", None)
+ )
+ code = getattr(exc, "code", None)
+ if isinstance(code, int):
+ code = str(code)
+ body = getattr(exc, "body", None)
+ if not code and isinstance(body, Mapping):
+ error_body = body.get("error", body)
+ if isinstance(error_body, Mapping):
+ code = error_body.get("code") or error_body.get("type")
+
+ request_id = (
+ getattr(exc, "request_id", None)
+ or _header(getattr(exc, "response", None), "x-request-id")
+ or _header(getattr(exc, "response", None), "request-id")
+ )
+ retry_after_value = _header(
+ getattr(exc, "response", None), "retry-after"
+ )
+ retry_after = _optional_float(retry_after_value)
+ category = _classify_error(exc, status)
+ return cls(
+ message=str(exc),
+ provider=provider,
+ category=category,
+ retryable=category
+ in {
+ LLMErrorCategory.RATE_LIMIT,
+ LLMErrorCategory.TIMEOUT,
+ LLMErrorCategory.CONNECTION,
+ LLMErrorCategory.SERVER,
+ },
+ status_code=status,
+ provider_code=str(code) if code is not None else None,
+ request_id=str(request_id) if request_id is not None else None,
+ retry_after=retry_after,
+ retry_after_raw=(
+ str(retry_after_value)
+ if retry_after_value is not None
+ else None
+ ),
+ )
+
+
+# ---------------------------------------------------------------------------
+# Durable MessageWireV2 codec
+# ---------------------------------------------------------------------------
+
+
+def message_to_wire(message: Message) -> MessageWireV2:
+ """Serialize a message to the versioned, JSON-safe wire contract."""
+
+ return {
+ "version": MESSAGE_WIRE_VERSION,
+ "role": message.role,
+ "content": message.content,
+ "blocks": [_block_to_wire(block) for block in message.blocks],
+ "tool_calls": [_tool_call_to_wire(call) for call in message.tool_calls],
+ "tool_call_id": message.tool_call_id,
+ "name": message.name,
+ "provider_state": _validated_provider_state(message.provider_state),
+ }
+
+
+def message_from_wire(value: Mapping[str, Any]) -> Message:
+ """Decode MessageWireV2 (and the pre-version flat native shape)."""
+
+ version = value.get("version", value.get("wire_version", 1))
+ if version not in SUPPORTED_MESSAGE_WIRE_VERSIONS:
+ raise ValueError(f"Unsupported message wire version: {version!r}")
+
+ calls = [
+ _tool_call_from_wire(call)
+ for call in value.get("tool_calls", ())
+ if isinstance(call, Mapping)
+ ]
+ blocks = (
+ [
+ _block_from_wire(block)
+ for block in value.get("blocks", ())
+ if isinstance(block, Mapping)
+ ]
+ if version in NATIVE_MESSAGE_WIRE_VERSIONS
+ else []
+ )
+ return Message(
+ role=str(value.get("role") or value.get("type") or "user"),
+ content=str(value.get("content") or ""),
+ tool_calls=calls,
+ tool_call_id=_optional_str(value.get("tool_call_id")),
+ name=_optional_str(value.get("name")),
+ blocks=blocks,
+ provider_state=dict(value.get("provider_state") or {}),
+ )
+
+
+def messages_to_wire(messages: Iterable[Message]) -> List[MessageWireV2]:
+ return [message_to_wire(message) for message in messages]
+
+
+def messages_from_wire(values: Iterable[Mapping[str, Any]]) -> List[Message]:
+ return [message_from_wire(value) for value in values]
+
+
+def _default_blocks(message: Message) -> List[ContentBlock]:
+ blocks: List[ContentBlock] = []
+ if message.content:
+ block_type = "tool_result" if message.role == "tool" else "text"
+ blocks.append(
+ ContentBlock(
+ type=block_type,
+ text=message.content,
+ tool_call_id=message.tool_call_id,
+ name=message.name,
+ )
+ )
+ blocks.extend(
+ ContentBlock(type="tool_call", tool_call=call)
+ for call in message.tool_calls
+ )
+ return blocks
+
+
+def _tool_call_to_wire(call: ToolCall) -> Dict[str, Any]:
+ return {
+ "id": call.id,
+ "name": call.name,
+ "args": _json_safe(call.args),
+ "raw_arguments": call.raw_arguments,
+ "parse_error": call.parse_error,
+ }
+
+
+def _tool_call_from_wire(value: Mapping[str, Any]) -> ToolCall:
+ args = value.get("args")
+ return ToolCall(
+ id=str(value.get("id") or ""),
+ name=str(value.get("name") or ""),
+ args=dict(args) if isinstance(args, Mapping) else {},
+ raw_arguments=_optional_str(value.get("raw_arguments")),
+ parse_error=_optional_str(value.get("parse_error")),
+ )
+
+
+def _block_to_wire(block: ContentBlock) -> Dict[str, Any]:
+ return {
+ "type": block.type,
+ "text": block.text,
+ "tool_call": (
+ _tool_call_to_wire(block.tool_call) if block.tool_call else None
+ ),
+ "tool_call_id": block.tool_call_id,
+ "name": block.name,
+ "metadata": _json_safe(block.metadata),
+ }
+
+
+def _block_from_wire(value: Mapping[str, Any]) -> ContentBlock:
+ tool_call = value.get("tool_call")
+ metadata = value.get("metadata")
+ return ContentBlock(
+ type=str(value.get("type") or "text"),
+ text=str(value.get("text") or ""),
+ tool_call=(
+ _tool_call_from_wire(tool_call)
+ if isinstance(tool_call, Mapping)
+ else None
+ ),
+ tool_call_id=_optional_str(value.get("tool_call_id")),
+ name=_optional_str(value.get("name")),
+ metadata=dict(metadata) if isinstance(metadata, Mapping) else {},
+ )
+
+
+def _validated_provider_state(value: Any) -> Dict[str, Any]:
+ if value in (None, {}):
+ return {}
+ if not isinstance(value, Mapping):
+ raise TypeError("provider_state must be a JSON object")
+ # Continuation blocks can legitimately exceed 256 KiB. Anthropic signed
+ # thinking, Gemini thought-signature turns, and OpenAI encrypted Responses
+ # output must be replayed byte-for-byte; rejecting by size turned a valid
+ # provider response into a local failure. Keep the JSON-only/depth guards,
+ # but do not truncate or reject otherwise valid durable state.
+ return _json_safe(dict(value))
+
+
+def _json_safe(value: Any, *, _depth: int = 0) -> Any:
+ if _depth > MAX_PROVIDER_STATE_DEPTH:
+ raise ValueError(
+ f"JSON value exceeds maximum depth {MAX_PROVIDER_STATE_DEPTH}"
+ )
+ if value is None or isinstance(value, (str, bool, int)):
+ return value
+ if isinstance(value, float):
+ if not math.isfinite(value):
+ raise ValueError("Non-finite floats are not valid durable JSON")
+ return value
+ if isinstance(value, Mapping):
+ return {
+ str(key): _json_safe(item, _depth=_depth + 1)
+ for key, item in value.items()
+ }
+ if isinstance(value, (list, tuple)):
+ return [_json_safe(item, _depth=_depth + 1) for item in value]
+ raise TypeError(
+ "Durable LLM state must contain only JSON values; "
+ f"got {type(value).__name__}"
+ )
+
+
+def _safe_token_count(value: Any) -> int:
+ parsed = _optional_int(value)
+ return max(0, parsed or 0)
+
+
+def _optional_int(value: Any) -> Optional[int]:
+ try:
+ return int(value) if value is not None else None
+ except (TypeError, ValueError, OverflowError):
+ return None
+
+
+def _optional_float(value: Any) -> Optional[float]:
+ try:
+ return float(value) if value is not None else None
+ except (TypeError, ValueError, OverflowError):
+ return None
+
+
+def _optional_str(value: Any) -> Optional[str]:
+ return str(value) if value is not None else None
+
+
+def _json_dump(value: Any) -> str:
+ try:
+ return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
+ except (TypeError, ValueError):
+ return str(value)
+
+
+def _header(response: Any, name: str) -> Optional[str]:
+ headers = getattr(response, "headers", None)
+ if headers is None:
+ return None
+ try:
+ return headers.get(name)
+ except (AttributeError, TypeError):
+ return None
+
+
+def _classify_error(
+ exc: BaseException, status: Optional[int]
+) -> LLMErrorCategory:
+ name = type(exc).__name__.lower()
+ message = str(exc).lower()
+ if status == 401 or "authentication" in name or "api key" in message:
+ return LLMErrorCategory.AUTHENTICATION
+ if status == 403 or "permission" in name:
+ return LLMErrorCategory.PERMISSION
+ if status == 429 or "ratelimit" in name or "rate limit" in message:
+ return LLMErrorCategory.RATE_LIMIT
+ if status == 404 or "notfound" in name:
+ return LLMErrorCategory.NOT_FOUND
+ if (
+ "context_length" in message
+ or "context length" in message
+ or "too many tokens" in message
+ ):
+ return LLMErrorCategory.CONTEXT_LENGTH
+ if status in {400, 409, 422} or "badrequest" in name:
+ return LLMErrorCategory.INVALID_REQUEST
+ if status == 408 or "timeout" in name:
+ return LLMErrorCategory.TIMEOUT
+ if "connection" in name:
+ return LLMErrorCategory.CONNECTION
+ if status is not None and status >= 500:
+ return LLMErrorCategory.SERVER
+ return LLMErrorCategory.UNKNOWN
# ---------------------------------------------------------------------------
diff --git a/server/services/llm/providers/__init__.py b/server/services/llm/providers/__init__.py
index c0e87387..94c1cd63 100644
--- a/server/services/llm/providers/__init__.py
+++ b/server/services/llm/providers/__init__.py
@@ -23,4 +23,4 @@ from services.llm.providers import anthropic # noqa: F401
from services.llm.providers import openai # noqa: F401
from services.llm.providers import gemini # noqa: F401
from services.llm.providers import openrouter # noqa: F401
-from services.llm.providers import _compat # noqa: F401 (registers 6 OpenAI-compat providers)
+from services.llm.providers import _compat # noqa: F401 (registers 8 OpenAI-compat providers)
diff --git a/server/services/llm/providers/_compat.py b/server/services/llm/providers/_compat.py
index 2834ecdd..a573fe5b 100644
--- a/server/services/llm/providers/_compat.py
+++ b/server/services/llm/providers/_compat.py
@@ -31,9 +31,8 @@ logger = get_logger(__name__)
# Groq + Cerebras moved here in Phase D (May 2026). Both expose
# OpenAI-compatible `/v1` endpoints (``api.groq.com/openai/v1`` /
# ``api.cerebras.ai/v1``) so they share the ``OpenAIProvider`` factory
-# with the other compat providers. Registering them here makes the
-# LangChain fallback path in ``services/ai.py`` dead code, which Phase
-# D then deletes.
+# with the other compat providers. The only remaining LangChain path is the
+# temporary Temporal-history compatibility branch.
_COMPAT_PROVIDERS: Tuple[str, ...] = (
"xai",
"deepseek",
@@ -49,10 +48,8 @@ _COMPAT_PROVIDERS: Tuple[str, ...] = (
def _load_compat_base_urls() -> dict[str, str]:
"""Read ``providers.<name>.base_url`` from llm_defaults.json.
- Lazy so we don't add a hard import dependency on the JSON parser at
- module import. The JSON is already cached by ``services/ai.py`` 's
- ``_LLM_DEFAULTS``; we re-read it here so this module can register
- without depending on the ``ai`` service.
+ Kept independent from ``services.ai`` so provider registration does not
+ import the orchestration or legacy Temporal-compatibility layer.
"""
import json
from pathlib import Path
@@ -98,7 +95,10 @@ def _register_compat_providers() -> None:
name=name,
factory=OpenAIProvider,
sdk_exception_refs=("openai:OpenAIError",),
- client_kwargs={"base_url": base_url},
+ client_kwargs={
+ "base_url": base_url,
+ "provider_name": name,
+ },
)
)
diff --git a/server/services/llm/providers/anthropic.py b/server/services/llm/providers/anthropic.py
index b24b0727..bb33f696 100644
--- a/server/services/llm/providers/anthropic.py
+++ b/server/services/llm/providers/anthropic.py
@@ -2,11 +2,13 @@
from __future__ import annotations
-import json
from typing import Any, Dict, List, Optional
from core.logging import get_logger
from services.llm.protocol import (
+ ContentBlock,
+ decode_binary_state,
+ encode_binary_state,
LLMResponse,
Message,
ThinkingConfig,
@@ -21,10 +23,19 @@ logger = get_logger(__name__)
class AnthropicProvider:
provider_name = "anthropic"
- def __init__(self, api_key: str, *, proxy_url: Optional[str] = None):
+ def __init__(
+ self,
+ api_key: str,
+ *,
+ proxy_url: Optional[str] = None,
+ max_retries: int = 2,
+ ):
import anthropic
- kwargs: Dict[str, Any] = {"api_key": api_key}
+ kwargs: Dict[str, Any] = {
+ "api_key": api_key,
+ "max_retries": max(0, int(max_retries)),
+ }
if proxy_url:
kwargs["base_url"] = proxy_url
kwargs["api_key"] = "ollama"
@@ -56,7 +67,7 @@ class AnthropicProvider:
# Thinking / extended thinking
if thinking and thinking.enabled:
- budget = thinking.budget or 2048
+ budget = max(1024, int(thinking.budget or 2048))
if max_tokens <= budget:
params["max_tokens"] = budget + 1024
params["thinking"] = {"type": "enabled", "budget_tokens": budget}
@@ -96,12 +107,37 @@ class AnthropicProvider:
"""Extract system message (Anthropic takes it as a top-level param)."""
system_parts = []
api_msgs = []
- for m in messages:
+ index = 0
+ while index < len(messages):
+ m = messages[index]
if m.role == "system":
if m.content:
system_parts.append(m.content)
- else:
- api_msgs.append(self._to_api_message(m))
+ index += 1
+ continue
+
+ if m.role == "tool":
+ # A parallel Anthropic tool-use turn must be answered by one
+ # user message containing every corresponding tool_result
+ # block. Keep normalized messages provider-neutral and
+ # coalesce their consecutive run only while compiling the
+ # request.
+ result_blocks: List[Dict[str, Any]] = []
+ while index < len(messages) and messages[index].role == "tool":
+ tool_message = messages[index]
+ result_blocks.append(
+ {
+ "type": "tool_result",
+ "tool_use_id": tool_message.tool_call_id or "",
+ "content": tool_message.content,
+ }
+ )
+ index += 1
+ api_msgs.append({"role": "user", "content": result_blocks})
+ continue
+
+ api_msgs.append(self._to_api_message(m))
+ index += 1
return "\n\n".join(system_parts) if system_parts else None, api_msgs
def _to_api_message(self, m: Message) -> Dict[str, Any]:
@@ -119,6 +155,22 @@ class AnthropicProvider:
],
}
+ if m.role == "assistant":
+ state = m.provider_state
+ if state.get("provider") == self.provider_name:
+ payload = state.get("payload")
+ if isinstance(payload, dict) and isinstance(
+ payload.get("content"), list
+ ):
+ return {
+ "role": "assistant",
+ "content": [
+ self._state_block_to_api(block)
+ for block in payload["content"]
+ if isinstance(block, dict)
+ ],
+ }
+
if m.role == "assistant" and m.tool_calls:
content: List[Dict[str, Any]] = []
if m.content:
@@ -138,49 +190,154 @@ class AnthropicProvider:
@staticmethod
def _to_api_tool(tool: ToolDef) -> Dict[str, Any]:
+ from services.llm.schema import compile_tool_schema
+
return {
"name": tool.name,
"description": tool.description,
- "input_schema": tool.parameters,
+ "input_schema": compile_tool_schema(
+ tool.parameters, provider="anthropic"
+ ),
}
def _normalize(self, resp: Any, model: str) -> LLMResponse:
text_parts = []
thinking_parts = []
tool_calls = []
+ blocks: List[ContentBlock] = []
+ provider_blocks: List[Dict[str, Any]] = []
for block in resp.content:
if block.type == "text":
text_parts.append(block.text)
+ blocks.append(ContentBlock(type="text", text=block.text))
+ provider_blocks.append({"type": "text", "text": block.text})
elif block.type == "thinking":
thinking_parts.append(block.thinking)
- elif block.type == "tool_use":
- tool_calls.append(
- ToolCall(
- id=block.id,
- name=block.name,
- args=block.input if isinstance(block.input, dict) else json.loads(block.input),
+ metadata: Dict[str, Any] = {}
+ provider_block: Dict[str, Any] = {
+ "type": "thinking",
+ "thinking": block.thinking,
+ }
+ signature = getattr(block, "signature", None)
+ if isinstance(
+ signature, (str, bytes, bytearray, memoryview)
+ ):
+ durable_signature = encode_binary_state(signature)
+ metadata["signature"] = durable_signature
+ provider_block["signature"] = durable_signature
+ blocks.append(
+ ContentBlock(
+ type="reasoning",
+ text=block.thinking,
+ metadata=metadata,
)
)
+ provider_blocks.append(provider_block)
+ elif block.type == "redacted_thinking":
+ data = getattr(block, "data", "")
+ if isinstance(data, (bytes, bytearray, memoryview)):
+ data = encode_binary_state(data)
+ elif not isinstance(data, str):
+ data = str(data)
+ blocks.append(
+ ContentBlock(
+ type="reasoning",
+ metadata={"redacted": True},
+ )
+ )
+ provider_blocks.append(
+ {"type": "redacted_thinking", "data": data}
+ )
+ elif block.type == "tool_use":
+ call = ToolCall.from_raw(
+ id=block.id,
+ name=block.name,
+ arguments=block.input,
+ )
+ tool_calls.append(call)
+ blocks.append(ContentBlock(type="tool_call", tool_call=call))
+ provider_blocks.append(
+ {
+ "type": "tool_use",
+ "id": block.id,
+ "name": block.name,
+ "input": (
+ block.input
+ if isinstance(
+ block.input,
+ (dict, list, str, int, float, bool, type(None)),
+ )
+ else str(block.input)
+ ),
+ }
+ )
+ input_tokens = getattr(resp.usage, "input_tokens", 0) or 0
+ output_tokens = getattr(resp.usage, "output_tokens", 0) or 0
+ cache_creation_tokens = (
+ getattr(resp.usage, "cache_creation_input_tokens", 0) or 0
+ )
+ cache_read_tokens = (
+ getattr(resp.usage, "cache_read_input_tokens", 0) or 0
+ )
usage = Usage(
- input_tokens=getattr(resp.usage, "input_tokens", 0),
- output_tokens=getattr(resp.usage, "output_tokens", 0),
- total_tokens=getattr(resp.usage, "input_tokens", 0) + getattr(resp.usage, "output_tokens", 0),
- cache_creation_tokens=getattr(resp.usage, "cache_creation_input_tokens", 0),
- cache_read_tokens=getattr(resp.usage, "cache_read_input_tokens", 0),
+ input_tokens=input_tokens,
+ output_tokens=output_tokens,
+ total_tokens=(
+ input_tokens
+ + cache_creation_tokens
+ + cache_read_tokens
+ + output_tokens
+ ),
+ cache_creation_tokens=cache_creation_tokens,
+ cache_read_tokens=cache_read_tokens,
)
+ content = "\n".join(text_parts)
+ thinking = "\n\n".join(thinking_parts) if thinking_parts else None
+ assistant_message = Message(
+ role="assistant",
+ content=content,
+ tool_calls=tool_calls,
+ blocks=blocks,
+ provider_state={
+ "provider": self.provider_name,
+ "payload": {"content": provider_blocks},
+ },
+ )
return LLMResponse(
- content="\n".join(text_parts),
- thinking="\n\n".join(thinking_parts) if thinking_parts else None,
+ content=content,
+ thinking=thinking,
tool_calls=tool_calls,
usage=usage,
model=model,
finish_reason=resp.stop_reason or "stop",
raw=resp,
+ assistant_message=assistant_message,
)
+ @staticmethod
+ def _state_block_to_api(block: Dict[str, Any]) -> Dict[str, Any]:
+ """Rehydrate only the binary fields Anthropic returned originally."""
+
+ result = dict(block)
+ if result.get("type") == "thinking" and "signature" in result:
+ result["signature"] = decode_binary_state(result["signature"])
+ elif (
+ result.get("type") == "redacted_thinking"
+ and "data" in result
+ ):
+ result["data"] = decode_binary_state(result["data"])
+ return result
+
+ async def aclose(self) -> None:
+ close = getattr(self._client, "close", None)
+ if close is not None:
+ result = close()
+ if hasattr(result, "__await__"):
+ await result
+
# ---------------------------------------------------------------------------
# Plugin self-registration
diff --git a/server/services/llm/providers/gemini.py b/server/services/llm/providers/gemini.py
index 9323c6be..c5b53844 100644
--- a/server/services/llm/providers/gemini.py
+++ b/server/services/llm/providers/gemini.py
@@ -2,10 +2,14 @@
from __future__ import annotations
+import base64
+import hashlib
+import json
from typing import Any, Dict, List, Optional
from core.logging import get_logger
from services.llm.protocol import (
+ ContentBlock,
LLMResponse,
Message,
ThinkingConfig,
@@ -20,11 +24,23 @@ logger = get_logger(__name__)
class GeminiProvider:
provider_name = "gemini"
- def __init__(self, api_key: str, *, proxy_url: Optional[str] = None):
+ def __init__(
+ self,
+ api_key: str,
+ *,
+ proxy_url: Optional[str] = None,
+ max_retries: Optional[int] = None,
+ ):
from google import genai
from services.llm.vertex import is_vertex_express_key
+ http_options: Dict[str, Any] = {}
+ if max_retries is not None:
+ # Gemini counts the original request as an attempt.
+ http_options["retry_options"] = {
+ "attempts": max(1, int(max_retries) + 1)
+ }
self._vertex = is_vertex_express_key(api_key)
if self._vertex:
# Agent Platform / Vertex Express key — the SDK routes to
@@ -33,12 +49,20 @@ class GeminiProvider:
# which is meaningless against the Vertex endpoint.
if proxy_url:
logger.warning("gemini: proxy_url ignored in Vertex AI mode")
- self._client = genai.Client(vertexai=True, api_key=api_key)
+ kwargs: Dict[str, Any] = {
+ "vertexai": True,
+ "api_key": api_key,
+ }
+ if http_options:
+ kwargs["http_options"] = http_options
+ self._client = genai.Client(**kwargs)
return
kwargs: Dict[str, Any] = {"api_key": api_key}
if proxy_url:
- kwargs["http_options"] = {"base_url": proxy_url}
+ http_options["base_url"] = proxy_url
+ if http_options:
+ kwargs["http_options"] = http_options
self._client = genai.Client(**kwargs)
# ------------------------------------------------------------------
@@ -72,11 +96,11 @@ class GeminiProvider:
# user explicitly set thinking_level (Vertex rejects an unsolicited
# thinking_level on 2.5-era models with 400 INVALID_ARGUMENT).
if thinking and thinking.enabled:
- thinking_kwargs: Dict[str, Any] = {}
- if thinking.budget:
- thinking_kwargs["thinking_budget"] = thinking.budget
+ thinking_kwargs: Dict[str, Any] = {"include_thoughts": True}
if thinking.level:
thinking_kwargs["thinking_level"] = thinking.level
+ elif thinking.budget:
+ thinking_kwargs["thinking_budget"] = thinking.budget
config["thinking_config"] = types.ThinkingConfig(**thinking_kwargs)
# Tools
@@ -157,28 +181,61 @@ class GeminiProvider:
def _split_system_and_contents(self, messages: List[Message]):
system_parts = []
contents = []
- for m in messages:
+ index = 0
+ while index < len(messages):
+ m = messages[index]
if m.role == "system":
if m.content:
system_parts.append(m.content)
+ index += 1
continue
if m.role == "tool":
+ # Gemini expects every response to one parallel function-call
+ # batch in a single user turn. The shared agent runtime keeps
+ # one normalized Message per result, so coalesce the
+ # consecutive run here at the provider boundary.
+ response_parts: List[Dict[str, Any]] = []
+ while index < len(messages) and messages[index].role == "tool":
+ tool_message = messages[index]
+ function_response: Dict[str, Any] = {
+ "name": tool_message.name or "",
+ "response": {"result": tool_message.content},
+ }
+ if tool_message.tool_call_id:
+ function_response["id"] = tool_message.tool_call_id
+ response_parts.append(
+ {"function_response": function_response}
+ )
+ index += 1
contents.append(
{
- "role": "function",
- "parts": [
- {
- "function_response": {
- "name": m.name or "",
- "response": {"result": m.content},
- }
- }
- ],
+ "role": "user",
+ "parts": response_parts,
}
)
continue
+ if m.role == "assistant":
+ state = m.provider_state
+ if state.get("provider") == self.provider_name:
+ payload = state.get("payload")
+ if isinstance(payload, dict) and isinstance(
+ payload.get("parts"), list
+ ):
+ contents.append(
+ {
+ "role": "model",
+ "parts": [
+ self._state_to_api_part(part)
+ for part in payload["parts"]
+ if isinstance(part, dict)
+ ],
+ }
+ )
+ index += 1
+ continue
+
if m.role == "assistant" and m.tool_calls:
parts = []
if m.content:
@@ -193,22 +250,28 @@ class GeminiProvider:
}
)
contents.append({"role": "model", "parts": parts})
+ index += 1
continue
role = "model" if m.role == "assistant" else "user"
contents.append({"role": role, "parts": [{"text": m.content}]})
+ index += 1
system = "\n\n".join(system_parts) if system_parts else None
return system, contents
@staticmethod
def _to_api_tool(tool: ToolDef) -> Dict[str, Any]:
+ from services.llm.schema import compile_tool_schema
+
return {
"function_declarations": [
{
"name": tool.name,
"description": tool.description,
- "parameters": tool.parameters,
+ "parameters": compile_tool_schema(
+ tool.parameters, provider="gemini"
+ ),
}
]
}
@@ -217,47 +280,216 @@ class GeminiProvider:
text_parts = []
thinking_parts = []
tool_calls = []
+ blocks: List[ContentBlock] = []
+ provider_parts: List[Dict[str, Any]] = []
if resp.candidates:
- for part in resp.candidates[0].content.parts:
+ response_identity = getattr(resp, "response_id", None)
+ if not isinstance(response_identity, str):
+ response_identity = ""
+ for index, part in enumerate(resp.candidates[0].content.parts):
if hasattr(part, "thought") and part.thought:
- thinking_parts.append(part.text)
+ state_part = self._api_part_to_state(part)
+ if state_part:
+ provider_parts.append(state_part)
+ text = getattr(part, "text", "") or ""
+ thinking_parts.append(text)
+ metadata = {}
+ if state_part.get("thought_signature_b64"):
+ metadata["thought_signature_b64"] = state_part[
+ "thought_signature_b64"
+ ]
+ blocks.append(
+ ContentBlock(
+ type="reasoning",
+ text=text,
+ metadata=metadata,
+ )
+ )
elif hasattr(part, "function_call") and part.function_call:
fc = part.function_call
- args = dict(fc.args) if fc.args else {}
- tool_calls.append(
- ToolCall(
- id=fc.name, # Gemini doesn't have separate IDs
+ raw_args = getattr(fc, "args", {})
+ provider_call_id = getattr(fc, "id", None)
+ call_id = (
+ str(provider_call_id)
+ if provider_call_id
+ else self._stable_tool_call_id(
+ response_identity=response_identity,
+ model=model,
+ index=index,
name=fc.name,
- args=args,
+ args=raw_args,
)
)
+ call = ToolCall.from_raw(
+ id=call_id,
+ name=fc.name,
+ arguments=raw_args,
+ )
+ state_part = self._api_part_to_state(
+ part,
+ function_call_id=call_id,
+ tool_call=call,
+ )
+ if state_part:
+ provider_parts.append(state_part)
+ tool_calls.append(call)
+ blocks.append(ContentBlock(type="tool_call", tool_call=call))
elif hasattr(part, "text") and part.text:
+ state_part = self._api_part_to_state(part)
+ if state_part:
+ provider_parts.append(state_part)
text_parts.append(part.text)
+ blocks.append(ContentBlock(type="text", text=part.text))
+ else:
+ state_part = self._api_part_to_state(part)
+ if state_part:
+ provider_parts.append(state_part)
um = resp.usage_metadata if hasattr(resp, "usage_metadata") and resp.usage_metadata else None
usage = Usage(
input_tokens=getattr(um, "prompt_token_count", 0) if um else 0,
output_tokens=getattr(um, "candidates_token_count", 0) if um else 0,
total_tokens=getattr(um, "total_token_count", 0) if um else 0,
+ cache_read_tokens=getattr(um, "cached_content_token_count", 0) if um else 0,
+ reasoning_tokens=getattr(um, "thoughts_token_count", 0) if um else 0,
)
finish = "stop"
if resp.candidates and hasattr(resp.candidates[0], "finish_reason"):
fr = resp.candidates[0].finish_reason
if fr:
- finish = str(fr).lower()
+ finish = str(getattr(fr, "value", fr)).lower()
+ content = "\n".join(text_parts)
+ thinking = "\n\n".join(thinking_parts) if thinking_parts else None
+ assistant_message = Message(
+ role="assistant",
+ content=content,
+ tool_calls=tool_calls,
+ blocks=blocks,
+ provider_state={
+ "provider": self.provider_name,
+ "payload": {"parts": provider_parts},
+ },
+ )
return LLMResponse(
- content="\n".join(text_parts),
- thinking="\n\n".join(thinking_parts) if thinking_parts else None,
+ content=content,
+ thinking=thinking,
tool_calls=tool_calls,
usage=usage,
model=model,
finish_reason=finish,
raw=resp,
+ assistant_message=assistant_message,
)
+ @staticmethod
+ def _api_part_to_state(
+ part: Any,
+ *,
+ function_call_id: Optional[str] = None,
+ tool_call: Optional[ToolCall] = None,
+ ) -> Dict[str, Any]:
+ state: Dict[str, Any] = {}
+ function_call = getattr(part, "function_call", None)
+ if function_call:
+ normalized_call = tool_call or ToolCall.from_raw(
+ id=str(function_call_id or getattr(function_call, "id", "")),
+ name=str(getattr(function_call, "name", "")),
+ arguments=getattr(function_call, "args", {}),
+ )
+ call_state: Dict[str, Any] = {
+ "name": normalized_call.name,
+ "args": normalized_call.args,
+ }
+ call_id = normalized_call.id
+ if call_id:
+ call_state["id"] = str(call_id)
+ if normalized_call.raw_arguments is not None:
+ call_state["raw_arguments"] = (
+ normalized_call.raw_arguments
+ )
+ if normalized_call.parse_error is not None:
+ call_state["parse_error"] = normalized_call.parse_error
+ state["function_call"] = call_state
+ else:
+ text = getattr(part, "text", None)
+ if isinstance(text, str):
+ state["text"] = text
+ thought = getattr(part, "thought", None)
+ if isinstance(thought, bool):
+ state["thought"] = thought
+
+ signature = getattr(part, "thought_signature", None)
+ if isinstance(signature, bytes):
+ state["thought_signature_b64"] = base64.b64encode(signature).decode(
+ "ascii"
+ )
+ elif isinstance(signature, str) and signature:
+ # Some SDK versions expose the already-base64 value as str.
+ state["thought_signature_b64"] = signature
+ return state
+
+ @staticmethod
+ def _state_to_api_part(state: Dict[str, Any]) -> Dict[str, Any]:
+ part: Dict[str, Any] = {}
+ if isinstance(state.get("function_call"), dict):
+ call = state["function_call"]
+ part["function_call"] = {
+ key: call[key]
+ for key in ("name", "args", "id")
+ if key in call
+ }
+ elif isinstance(state.get("text"), str):
+ part["text"] = state["text"]
+ if isinstance(state.get("thought"), bool):
+ part["thought"] = state["thought"]
+ encoded = state.get("thought_signature_b64")
+ if isinstance(encoded, str) and encoded:
+ try:
+ part["thought_signature"] = base64.b64decode(
+ encoded, validate=True
+ )
+ except (ValueError, TypeError):
+ # The state codec is durable; corrupt continuation metadata
+ # should fail loudly instead of silently changing the turn.
+ raise ValueError("Invalid Gemini thought_signature_b64")
+ return part
+
+ @staticmethod
+ def _stable_tool_call_id(
+ *,
+ response_identity: str,
+ model: str,
+ index: int,
+ name: str,
+ args: Any,
+ ) -> str:
+ material = json.dumps(
+ [response_identity, model, index, name, args],
+ sort_keys=True,
+ ensure_ascii=False,
+ separators=(",", ":"),
+ default=str,
+ ).encode("utf-8")
+ digest = hashlib.sha256(material).hexdigest()[:16]
+ return f"gemini-{index}-{digest}"
+
+ async def aclose(self) -> None:
+ aio = getattr(self._client, "aio", None)
+ close = getattr(aio, "aclose", None)
+ if close is not None:
+ result = close()
+ if hasattr(result, "__await__"):
+ await result
+ return
+ close = getattr(self._client, "close", None)
+ if close is not None:
+ result = close()
+ if hasattr(result, "__await__"):
+ await result
+
# ---------------------------------------------------------------------------
# Plugin self-registration
diff --git a/server/services/llm/providers/openai.py b/server/services/llm/providers/openai.py
index 58994a94..46065a22 100644
--- a/server/services/llm/providers/openai.py
+++ b/server/services/llm/providers/openai.py
@@ -11,6 +11,7 @@ from typing import Any, Dict, List, Optional
from core.logging import get_logger
from services.llm.protocol import (
+ ContentBlock,
LLMResponse,
Message,
ThinkingConfig,
@@ -25,10 +26,22 @@ logger = get_logger(__name__)
class OpenAIProvider:
provider_name = "openai"
- def __init__(self, api_key: str, *, proxy_url: Optional[str] = None, base_url: Optional[str] = None):
+ def __init__(
+ self,
+ api_key: str,
+ *,
+ proxy_url: Optional[str] = None,
+ base_url: Optional[str] = None,
+ provider_name: Optional[str] = None,
+ max_retries: int = 2,
+ ):
import openai
- kwargs: Dict[str, Any] = {"api_key": api_key}
+ self.provider_name = provider_name or type(self).provider_name
+ kwargs: Dict[str, Any] = {
+ "api_key": api_key,
+ "max_retries": max(0, int(max_retries)),
+ }
url = proxy_url or base_url
if url:
kwargs["base_url"] = url
@@ -50,27 +63,55 @@ class OpenAIProvider:
thinking: Optional[ThinkingConfig] = None,
tools: Optional[List[ToolDef]] = None,
) -> LLMResponse:
+ model = self._clean_model(model)
+ policy = self._model_policy(model, thinking)
+ if policy["use_responses"] and (
+ tools or self._has_responses_state(messages)
+ ):
+ return await self._chat_responses(
+ messages=messages,
+ model=model,
+ max_tokens=max_tokens,
+ thinking=thinking,
+ tools=tools,
+ )
+
params: Dict[str, Any] = {
"model": model,
"messages": [self._to_api_message(m) for m in messages],
}
- is_reasoning = model.startswith(("o1", "o3", "o4"))
-
- # Reasoning models use max_completion_tokens, not max_tokens
- if is_reasoning:
+ # OpenAI reasoning models use max_completion_tokens. Compatible
+ # endpoints continue to receive their documented max_tokens field.
+ if policy["max_completion_tokens"]:
params["max_completion_tokens"] = max_tokens
else:
params["max_tokens"] = max_tokens
- params["temperature"] = temperature
+ if policy["temperature_allowed"]:
+ params["temperature"] = (
+ policy["fixed_temperature"]
+ if policy["fixed_temperature"] is not None
+ else temperature
+ )
- # Thinking / reasoning effort
if thinking and thinking.enabled:
- effort = thinking.effort or "medium"
- if is_reasoning:
- params["reasoning_effort"] = effort
- elif model.startswith("gpt-5"):
- params["reasoning_effort"] = effort
+ thinking_type = policy["thinking_type"]
+ if thinking_type == "effort":
+ params["reasoning_effort"] = thinking.effort or "medium"
+ elif thinking_type == "format":
+ params.setdefault("extra_body", {})["reasoning_format"] = (
+ thinking.format
+ if thinking.format in {"parsed", "hidden"}
+ else "parsed"
+ )
+ elif thinking_type == "budget":
+ params.setdefault("extra_body", {})["thinking_budget"] = max(
+ 0, int(thinking.budget)
+ )
+ elif policy["thinking_default_on"]:
+ params.setdefault("extra_body", {})["thinking"] = {
+ "type": "disabled"
+ }
# Tools
if tools:
@@ -79,6 +120,37 @@ class OpenAIProvider:
resp = await self._client.chat.completions.create(**params)
return self._normalize(resp, model)
+ async def _chat_responses(
+ self,
+ *,
+ messages: List[Message],
+ model: str,
+ max_tokens: int,
+ thinking: Optional[ThinkingConfig],
+ tools: Optional[List[ToolDef]],
+ ) -> LLMResponse:
+ """Run a self-contained Responses API turn for reasoning tool use."""
+
+ params: Dict[str, Any] = {
+ "model": model,
+ "input": self._to_responses_input(messages),
+ "max_output_tokens": max_tokens,
+ # Durable encrypted reasoning state is recorded in our message;
+ # do not depend on OpenAI retaining a prior response.
+ "store": False,
+ "include": ["reasoning.encrypted_content"],
+ }
+ if thinking and thinking.enabled:
+ params["reasoning"] = {
+ "effort": thinking.effort or "medium",
+ }
+ if tools:
+ params["tools"] = [
+ self._to_responses_tool(tool) for tool in tools
+ ]
+ response = await self._client.responses.create(**params)
+ return self._normalize_responses(response, model)
+
# ------------------------------------------------------------------
# fetch_models
# ------------------------------------------------------------------
@@ -92,8 +164,7 @@ class OpenAIProvider:
# internals
# ------------------------------------------------------------------
- @staticmethod
- def _to_api_message(m: Message) -> Dict[str, Any]:
+ def _to_api_message(self, m: Message) -> Dict[str, Any]:
if m.role == "tool":
return {
"role": "tool",
@@ -101,36 +172,131 @@ class OpenAIProvider:
"content": m.content,
}
- if m.role == "assistant" and m.tool_calls:
- return {
+ if m.role == "assistant":
+ result: Dict[str, Any] = {
"role": "assistant",
"content": m.content or None,
- "tool_calls": [
+ }
+ if m.tool_calls:
+ result["tool_calls"] = [
{
"id": tc.id,
"type": "function",
"function": {
"name": tc.name,
- "arguments": json.dumps(tc.args),
+ "arguments": (
+ tc.raw_arguments
+ if tc.raw_arguments is not None
+ else json.dumps(tc.args)
+ ),
},
}
for tc in m.tool_calls
- ],
- }
+ ]
+ state = m.provider_state
+ if state.get("provider") == self.provider_name:
+ payload = state.get("payload")
+ if isinstance(payload, dict):
+ # These are the only continuation fields currently
+ # accepted by OpenAI-compatible chat endpoints. Never
+ # forward arbitrary provider state into an API request.
+ for key in (
+ "reasoning",
+ "reasoning_content",
+ "reasoning_details",
+ "refusal",
+ ):
+ if payload.get(key) is not None:
+ result[key] = payload[key]
+ return result
return {"role": m.role, "content": m.content}
- @staticmethod
- def _to_api_tool(tool: ToolDef) -> Dict[str, Any]:
+ def _to_api_tool(self, tool: ToolDef) -> Dict[str, Any]:
+ from services.llm.schema import compile_tool_schema
+
return {
"type": "function",
"function": {
"name": tool.name,
"description": tool.description,
- "parameters": tool.parameters,
+ "parameters": compile_tool_schema(
+ tool.parameters, provider=self.provider_name
+ ),
},
}
+ def _to_responses_tool(self, tool: ToolDef) -> Dict[str, Any]:
+ from services.llm.schema import compile_tool_schema
+
+ return {
+ "type": "function",
+ "name": tool.name,
+ "description": tool.description,
+ "parameters": compile_tool_schema(
+ tool.parameters, provider=self.provider_name
+ ),
+ "strict": False,
+ }
+
+ def _to_responses_input(
+ self, messages: List[Message]
+ ) -> List[Dict[str, Any]]:
+ items: List[Dict[str, Any]] = []
+ for message in messages:
+ if message.role == "tool":
+ items.append(
+ {
+ "type": "function_call_output",
+ "call_id": message.tool_call_id or "",
+ "output": message.content,
+ }
+ )
+ continue
+
+ if message.role == "assistant":
+ state = message.provider_state
+ payload = state.get("payload")
+ if (
+ state.get("provider") == self.provider_name
+ and isinstance(payload, dict)
+ and payload.get("api") == "responses"
+ and isinstance(payload.get("output"), list)
+ ):
+ items.extend(
+ dict(item)
+ for item in payload["output"]
+ if isinstance(item, dict)
+ )
+ continue
+
+ if message.content:
+ items.append(
+ {
+ "role": "assistant",
+ "content": message.content,
+ }
+ )
+ items.extend(
+ {
+ "type": "function_call",
+ "call_id": call.id,
+ "name": call.name,
+ "arguments": (
+ call.raw_arguments
+ if call.raw_arguments is not None
+ else json.dumps(call.args)
+ ),
+ }
+ for call in message.tool_calls
+ )
+ continue
+
+ items.append(
+ {"role": message.role, "content": message.content}
+ )
+ return items
+
def _normalize(self, resp: Any, model: str) -> LLMResponse:
choice = resp.choices[0] if resp.choices else None
if not choice:
@@ -140,31 +306,86 @@ class OpenAIProvider:
content = msg.content or ""
thinking = None
tool_calls = []
-
- # Extract reasoning/thinking if present
- if hasattr(msg, "reasoning_content") and msg.reasoning_content:
- thinking = msg.reasoning_content
+ blocks: List[ContentBlock] = []
+
+ # OpenAI-compatible providers expose reasoning under several fields:
+ # DeepSeek/xAI commonly use reasoning_content, Groq uses reasoning,
+ # and OpenRouter needs the lossless reasoning_details array replayed
+ # on the next tool turn.
+ reasoning_content = getattr(msg, "reasoning_content", None)
+ reasoning = getattr(msg, "reasoning", None)
+ if isinstance(reasoning_content, str) and reasoning_content:
+ thinking = reasoning_content
+ elif isinstance(reasoning, str) and reasoning:
+ thinking = reasoning
+ if thinking:
+ blocks.append(ContentBlock(type="reasoning", text=thinking))
+
+ refusal = getattr(msg, "refusal", None)
+ if not content and isinstance(refusal, str) and refusal:
+ content = refusal
+
+ if content:
+ blocks.append(
+ ContentBlock(
+ type="text",
+ text=content,
+ metadata=(
+ {"refusal": True}
+ if isinstance(refusal, str) and refusal
+ else {}
+ ),
+ )
+ )
# Tool calls
if msg.tool_calls:
for tc in msg.tool_calls:
args = tc.function.arguments
- tool_calls.append(
- ToolCall(
- id=tc.id,
- name=tc.function.name,
- args=json.loads(args) if isinstance(args, str) else args,
- )
+ call = ToolCall.from_raw(
+ id=tc.id,
+ name=tc.function.name,
+ arguments=args,
)
+ tool_calls.append(call)
+ blocks.append(ContentBlock(type="tool_call", tool_call=call))
u = resp.usage
usage = Usage(
input_tokens=getattr(u, "prompt_tokens", 0) if u else 0,
output_tokens=getattr(u, "completion_tokens", 0) if u else 0,
total_tokens=getattr(u, "total_tokens", 0) if u else 0,
+ cache_read_tokens=self._cache_read_tokens(u),
reasoning_tokens=getattr(getattr(u, "completion_tokens_details", None), "reasoning_tokens", 0) if u else 0,
)
+ payload: Dict[str, Any] = {}
+ if isinstance(reasoning, str) and reasoning:
+ payload["reasoning"] = reasoning
+ if isinstance(reasoning_content, str) and reasoning_content:
+ payload["reasoning_content"] = reasoning_content
+ reasoning_details = getattr(msg, "reasoning_details", None)
+ if isinstance(reasoning_details, (list, tuple)):
+ encoded_details = [
+ self._dump_response_item(detail)
+ for detail in reasoning_details
+ ]
+ encoded_details = [detail for detail in encoded_details if detail]
+ if encoded_details:
+ payload["reasoning_details"] = encoded_details
+ if isinstance(refusal, str) and refusal:
+ payload["refusal"] = refusal
+ assistant_message = Message(
+ role="assistant",
+ content=content,
+ tool_calls=tool_calls,
+ blocks=blocks,
+ provider_state=(
+ {"provider": self.provider_name, "payload": payload}
+ if payload
+ else {}
+ ),
+ )
return LLMResponse(
content=content,
thinking=thinking,
@@ -173,8 +394,274 @@ class OpenAIProvider:
model=model,
finish_reason=choice.finish_reason or "stop",
raw=resp,
+ assistant_message=assistant_message,
)
+ def _normalize_responses(self, resp: Any, model: str) -> LLMResponse:
+ text_parts: List[str] = []
+ thinking_parts: List[str] = []
+ tool_calls: List[ToolCall] = []
+ blocks: List[ContentBlock] = []
+ output_state: List[Dict[str, Any]] = []
+
+ for item in getattr(resp, "output", ()) or ():
+ dumped = self._dump_response_item(item)
+ if dumped:
+ output_state.append(dumped)
+ item_type = getattr(item, "type", None)
+ if item_type == "reasoning":
+ for summary in getattr(item, "summary", ()) or ():
+ text = getattr(summary, "text", None)
+ if isinstance(text, str) and text:
+ thinking_parts.append(text)
+ blocks.append(
+ ContentBlock(type="reasoning", text=text)
+ )
+ elif item_type == "message":
+ for part in getattr(item, "content", ()) or ():
+ part_type = getattr(part, "type", None)
+ text = getattr(part, "text", None)
+ if part_type == "output_text" and isinstance(text, str):
+ text_parts.append(text)
+ blocks.append(ContentBlock(type="text", text=text))
+ elif part_type == "refusal":
+ refusal = getattr(part, "refusal", None)
+ if not isinstance(refusal, str):
+ continue
+ text_parts.append(refusal)
+ blocks.append(
+ ContentBlock(
+ type="text",
+ text=refusal,
+ metadata={"refusal": True},
+ )
+ )
+ elif item_type == "function_call":
+ call = ToolCall.from_raw(
+ id=str(
+ getattr(item, "call_id", None)
+ or getattr(item, "id", "")
+ ),
+ name=str(getattr(item, "name", "")),
+ arguments=getattr(item, "arguments", "{}"),
+ )
+ tool_calls.append(call)
+ blocks.append(ContentBlock(type="tool_call", tool_call=call))
+
+ usage_obj = getattr(resp, "usage", None)
+ input_details = getattr(usage_obj, "input_tokens_details", None)
+ output_details = getattr(usage_obj, "output_tokens_details", None)
+ usage = Usage(
+ input_tokens=(
+ getattr(usage_obj, "input_tokens", 0) if usage_obj else 0
+ ),
+ output_tokens=(
+ getattr(usage_obj, "output_tokens", 0) if usage_obj else 0
+ ),
+ total_tokens=(
+ getattr(usage_obj, "total_tokens", 0) if usage_obj else 0
+ ),
+ cache_read_tokens=(
+ getattr(input_details, "cached_tokens", 0)
+ if input_details
+ else 0
+ ),
+ reasoning_tokens=(
+ getattr(output_details, "reasoning_tokens", 0)
+ if output_details
+ else 0
+ ),
+ )
+ content = "\n".join(text_parts)
+ thinking_text = (
+ "\n\n".join(thinking_parts) if thinking_parts else None
+ )
+ assistant_message = Message(
+ role="assistant",
+ content=content,
+ tool_calls=tool_calls,
+ blocks=blocks,
+ provider_state={
+ "provider": self.provider_name,
+ "payload": {
+ "api": "responses",
+ "response_id": str(getattr(resp, "id", "") or ""),
+ "output": output_state,
+ },
+ },
+ )
+ return LLMResponse(
+ content=content,
+ thinking=thinking_text,
+ tool_calls=tool_calls,
+ usage=usage,
+ model=str(getattr(resp, "model", "") or model),
+ finish_reason=str(
+ getattr(resp, "status", "") or "stop"
+ ).lower(),
+ raw=resp,
+ assistant_message=assistant_message,
+ )
+
+ @staticmethod
+ def _dump_response_item(item: Any) -> Dict[str, Any]:
+ dump = getattr(item, "model_dump", None)
+ if dump is not None:
+ value = dump(mode="json", exclude_none=True)
+ return dict(value) if isinstance(value, dict) else {}
+ if isinstance(item, dict):
+ return dict(item)
+
+ # Lightweight fallback for SDK-compatible proxy objects and tests.
+ item_type = getattr(item, "type", None)
+ if not isinstance(item_type, str):
+ return {}
+ result: Dict[str, Any] = {"type": item_type}
+ for key in (
+ "id",
+ "status",
+ "role",
+ "call_id",
+ "name",
+ "arguments",
+ "encrypted_content",
+ ):
+ value = getattr(item, key, None)
+ if isinstance(value, (str, int, float, bool)):
+ result[key] = value
+ for key in ("content", "summary"):
+ values = getattr(item, key, None)
+ if not values:
+ continue
+ encoded: List[Dict[str, Any]] = []
+ for value in values:
+ value_dump = getattr(value, "model_dump", None)
+ if value_dump is not None:
+ dumped = value_dump(mode="json", exclude_none=True)
+ if isinstance(dumped, dict):
+ encoded.append(dumped)
+ else:
+ entry = {
+ "type": getattr(value, "type", None),
+ "text": getattr(value, "text", None),
+ }
+ encoded.append(
+ {
+ item_key: item_value
+ for item_key, item_value in entry.items()
+ if item_value is not None
+ }
+ )
+ result[key] = encoded
+ return result
+
+ def _clean_model(self, model: str) -> str:
+ # Provider model IDs are opaque. Groq requires owner-qualified IDs
+ # such as ``openai/gpt-oss-120b`` and ``qwen/qwen3-32b``; local
+ # OpenAI-compatible endpoints may use namespaces too. Only remove the
+ # UI-only OpenRouter free-tier decoration.
+ return model.removeprefix("[FREE] ")
+
+ def _model_policy(
+ self, model: str, thinking: Optional[ThinkingConfig]
+ ) -> Dict[str, Any]:
+ """Resolve JSON-driven provider quirks in one native layer."""
+
+ from services.llm.config import LLM_DEFAULTS
+
+ config = LLM_DEFAULTS.get("providers", {}).get(self.provider_name, {})
+ reasoning_models = tuple(config.get("reasoning_models", ()))
+ is_reasoning = any(model.startswith(prefix) for prefix in reasoning_models)
+ if self.provider_name == "openai":
+ is_reasoning = is_reasoning or model.startswith(("o1", "o3", "o4"))
+ thinking_models = tuple(config.get("thinking_models", ()))
+ is_configured_thinking_model = any(
+ model.startswith(prefix) or prefix in model
+ for prefix in thinking_models
+ )
+ openai_reasoning_capable = (
+ self.provider_name == "openai"
+ and (is_reasoning or is_configured_thinking_model)
+ )
+
+ fixed_temperature = None
+ for prefix, value in config.get("fixed_temperature", {}).items():
+ if model.startswith(prefix):
+ fixed_temperature = float(value)
+ break
+
+ supported = set(config.get("supported_params", ()))
+ temperature_allowed = not is_reasoning and not openai_reasoning_capable
+ if supported and "temperature" not in supported:
+ temperature_allowed = False
+
+ default_on = any(
+ model.startswith(prefix)
+ for prefix in config.get("thinking_default_on", ())
+ )
+ thinking_type = config.get("thinking_type", "none")
+ supports_configured_thinking = (
+ not thinking_models
+ or is_configured_thinking_model
+ or is_reasoning
+ )
+ if not supports_configured_thinking:
+ thinking_type = "none"
+
+ # GPT-5 and o-series models use reasoning effort. Groq's GPT-OSS
+ # models also use reasoning_effort; reasoning_format is explicitly
+ # unsupported for those models and is reserved for Qwen.
+ if self.provider_name == "openai" and (
+ model.startswith("gpt-5") or is_reasoning
+ ):
+ thinking_type = "effort"
+ elif self.provider_name == "groq" and (
+ model.startswith("openai/gpt-oss-")
+ or model.startswith("gpt-oss-")
+ ):
+ thinking_type = "effort"
+
+ return {
+ "max_completion_tokens": (
+ openai_reasoning_capable
+ ),
+ "use_responses": (
+ openai_reasoning_capable
+ ),
+ "temperature_allowed": temperature_allowed
+ and not (
+ self.provider_name == "openai"
+ and thinking is not None
+ and thinking.enabled
+ and thinking_type == "effort"
+ ),
+ "fixed_temperature": fixed_temperature,
+ "thinking_default_on": default_on,
+ "thinking_type": thinking_type,
+ }
+
+ def _has_responses_state(self, messages: List[Message]) -> bool:
+ return any(
+ message.provider_state.get("provider") == self.provider_name
+ and isinstance(message.provider_state.get("payload"), dict)
+ and message.provider_state["payload"].get("api") == "responses"
+ for message in messages
+ )
+
+ @staticmethod
+ def _cache_read_tokens(usage: Any) -> int:
+ if not usage:
+ return 0
+ details = getattr(usage, "prompt_tokens_details", None)
+ return getattr(details, "cached_tokens", 0) if details else 0
+
+ async def aclose(self) -> None:
+ close = getattr(self._client, "close", None)
+ if close is not None:
+ result = close()
+ if hasattr(result, "__await__"):
+ await result
+
# ---------------------------------------------------------------------------
# Plugin self-registration
diff --git a/server/services/llm/providers/openrouter.py b/server/services/llm/providers/openrouter.py
index 032fd800..2e03b39d 100644
--- a/server/services/llm/providers/openrouter.py
+++ b/server/services/llm/providers/openrouter.py
@@ -19,12 +19,20 @@ logger = get_logger(__name__)
class OpenRouterProvider(OpenAIProvider):
provider_name = "openrouter"
- def __init__(self, api_key: str, *, proxy_url: Optional[str] = None):
+ def __init__(
+ self,
+ api_key: str,
+ *,
+ proxy_url: Optional[str] = None,
+ max_retries: int = 2,
+ ):
import openai
+ self.provider_name = "openrouter"
kwargs: Dict[str, Any] = {
"api_key": api_key,
"base_url": proxy_url or "https://openrouter.ai/api/v1",
+ "max_retries": max(0, int(max_retries)),
"default_headers": {
"HTTP-Referer": "http://localhost:3000",
"X-Title": "OpenCompany",
diff --git a/server/services/llm/schema.py b/server/services/llm/schema.py
new file mode 100644
index 00000000..1768fafa
--- /dev/null
+++ b/server/services/llm/schema.py
@@ -0,0 +1,134 @@
+"""JSON-schema preparation for native provider tool APIs.
+
+Pydantic emits local ``$ref``/``$defs`` constructs that are valid JSON
+Schema but inconsistently supported by model providers. Tool definitions are
+small, so eagerly inlining local references gives all adapters the same stable
+input and lets the final provider pass remove only keywords its API rejects.
+"""
+
+from __future__ import annotations
+
+from copy import deepcopy
+from typing import Any, Dict, Mapping
+
+
+_META_KEYWORDS = {"$schema", "$id", "$defs", "definitions"}
+_GEMINI_UNSUPPORTED = {
+ "$comment",
+ "const",
+ "default",
+ "dependentSchemas",
+ "else",
+ "examples",
+ "if",
+ "patternProperties",
+ "then",
+ "unevaluatedProperties",
+}
+
+
+def compile_tool_schema(
+ schema: Mapping[str, Any], *, provider: str
+) -> Dict[str, Any]:
+ """Return an isolated schema accepted by the named provider.
+
+ Only local JSON pointers are expanded. External references are rejected:
+ provider requests cannot dereference project-local or network documents.
+ """
+
+ root = deepcopy(dict(schema))
+ resolved = _resolve_node(root, root=root, stack=())
+ if not isinstance(resolved, dict):
+ raise ValueError("Tool parameter schema must resolve to an object")
+ for keyword in _META_KEYWORDS:
+ resolved.pop(keyword, None)
+ return _clean_provider_schema(resolved, provider=provider)
+
+
+def _resolve_node(value: Any, *, root: Dict[str, Any], stack: tuple[str, ...]) -> Any:
+ if isinstance(value, list):
+ return [_resolve_node(item, root=root, stack=stack) for item in value]
+ if not isinstance(value, dict):
+ return value
+
+ ref = value.get("$ref")
+ if ref is not None:
+ if not isinstance(ref, str) or not ref.startswith("#/"):
+ raise ValueError(f"Only local JSON-schema references are supported: {ref!r}")
+ if ref in stack:
+ chain = " -> ".join((*stack, ref))
+ raise ValueError(f"Recursive tool schema is not supported: {chain}")
+ target: Any = root
+ for raw_part in ref[2:].split("/"):
+ part = raw_part.replace("~1", "/").replace("~0", "~")
+ if not isinstance(target, dict) or part not in target:
+ raise ValueError(f"Unresolved JSON-schema reference: {ref}")
+ target = target[part]
+ merged = _resolve_node(
+ deepcopy(target),
+ root=root,
+ stack=(*stack, ref),
+ )
+ siblings = {
+ key: item for key, item in value.items() if key != "$ref"
+ }
+ if siblings:
+ if not isinstance(merged, dict):
+ raise ValueError(f"Cannot merge siblings into non-object reference: {ref}")
+ merged.update(
+ _resolve_node(siblings, root=root, stack=stack)
+ )
+ return merged
+
+ return {
+ key: _resolve_node(item, root=root, stack=stack)
+ for key, item in value.items()
+ if key not in {"$defs", "definitions"}
+ }
+
+
+def _clean_provider_schema(value: Any, *, provider: str) -> Any:
+ if isinstance(value, list):
+ return [
+ _clean_provider_schema(item, provider=provider)
+ for item in value
+ ]
+ if not isinstance(value, dict):
+ return value
+
+ missing = object()
+ const_value = value.get("const", missing)
+ cleaned = {
+ key: _clean_provider_schema(item, provider=provider)
+ for key, item in value.items()
+ if key not in _META_KEYWORDS
+ and not (provider == "gemini" and key in _GEMINI_UNSUPPORTED)
+ }
+
+ if provider == "gemini":
+ # Gemini rejects JSON Schema ``const`` while accepting ``enum``.
+ # Dropping const silently widened Pydantic Literal fields, so retain
+ # the exact constraint through the equivalent single-value enum.
+ if const_value is not missing and "enum" not in cleaned:
+ cleaned["enum"] = [
+ _clean_provider_schema(const_value, provider=provider)
+ ]
+
+ schema_type = cleaned.get("type")
+ if isinstance(schema_type, list) and "null" in schema_type:
+ non_null = [item for item in schema_type if item != "null"]
+ if len(non_null) == 1:
+ cleaned["type"] = non_null[0]
+ cleaned["nullable"] = True
+ variants = cleaned.get("anyOf")
+ if isinstance(variants, list):
+ non_null_variants = [
+ item
+ for item in variants
+ if not (isinstance(item, dict) and item.get("type") == "null")
+ ]
+ if len(non_null_variants) == len(variants) - 1:
+ cleaned["anyOf"] = non_null_variants
+ cleaned["nullable"] = True
+
+ return cleaned
diff --git a/server/services/llm/unifier.py b/server/services/llm/unifier.py
index 34abe237..6190b4cb 100644
--- a/server/services/llm/unifier.py
+++ b/server/services/llm/unifier.py
@@ -13,10 +13,23 @@ credential lookups).
from __future__ import annotations
+import asyncio
+import hashlib
+import json
+from collections import OrderedDict
+from dataclasses import dataclass
from typing import Any, Dict, List, Optional, TYPE_CHECKING
from core.logging import get_logger
-from services.llm.protocol import LLMProvider, LLMResponse, Message, ThinkingConfig, ToolDef
+from services.llm.protocol import (
+ LLMError,
+ LLMErrorCategory,
+ LLMProvider,
+ LLMResponse,
+ Message,
+ ThinkingConfig,
+ ToolDef,
+)
from services.llm.registry import ProviderSpec, get_provider, has_provider
from services.plugin import NodeUserError
@@ -26,6 +39,15 @@ if TYPE_CHECKING:
logger = get_logger(__name__)
+@dataclass(eq=False)
+class _ClientEntry:
+ """One cached SDK client plus its in-flight lease count."""
+
+ client: LLMProvider
+ leases: int = 0
+ retired: bool = False
+
+
class ChatUnifier:
"""Single facade for chat-model execution.
@@ -36,9 +58,19 @@ class ChatUnifier:
Everything else is private.
"""
- def __init__(self, defaults: Dict[str, Any], auth_service: "AuthService"):
+ def __init__(
+ self,
+ defaults: Dict[str, Any],
+ auth_service: "AuthService",
+ *,
+ client_cache_size: int = 32,
+ ):
self._defaults = defaults
self._auth = auth_service
+ self._client_cache_size = max(0, int(client_cache_size))
+ self._client_cache: OrderedDict[str, _ClientEntry] = OrderedDict()
+ self._retired_entries: List[_ClientEntry] = []
+ self._cache_lock = asyncio.Lock()
# ------------------------------------------------------------------
# public API
@@ -55,6 +87,8 @@ class ChatUnifier:
max_tokens: int = 4096,
thinking: Optional[ThinkingConfig] = None,
tools: Optional[List[ToolDef]] = None,
+ sdk_max_retries: int = 2,
+ translate_errors: bool = True,
) -> LLMResponse:
"""Execute a chat completion against the named provider.
@@ -65,9 +99,12 @@ class ChatUnifier:
``except Exception``.
"""
spec = get_provider(provider)
- client = await self._build_client(spec, api_key)
+ entry: Optional[_ClientEntry] = None
try:
- return await client.chat(
+ entry = await self._acquire_client(
+ spec, api_key, sdk_max_retries=sdk_max_retries
+ )
+ return await entry.client.chat(
messages,
model=model,
temperature=temperature,
@@ -76,7 +113,49 @@ class ChatUnifier:
tools=tools,
)
except spec.sdk_exception_types as e:
- raise NodeUserError(str(e)) from e
+ error = LLMError.from_exception(provider, e)
+ if not translate_errors:
+ raise error from e
+ logger.warning(
+ "LLM provider request failed",
+ provider=error.provider,
+ category=error.category.value,
+ retryable=error.retryable,
+ status_code=error.status_code,
+ provider_code=error.provider_code,
+ request_id=error.request_id,
+ )
+ raise NodeUserError(error.user_message) from error
+ except (ValueError, TypeError, OSError) as e:
+ # Only normalize generic configuration/transport failures raised
+ # while constructing a provider client. Once a client exists,
+ # unexpected exceptions from provider logic remain programming
+ # errors and keep their traceback.
+ if entry is not None:
+ raise
+ category = (
+ LLMErrorCategory.CONNECTION
+ if isinstance(e, OSError)
+ else LLMErrorCategory.INVALID_REQUEST
+ )
+ error = LLMError(
+ message=str(e),
+ provider=provider,
+ category=category,
+ retryable=category == LLMErrorCategory.CONNECTION,
+ )
+ if not translate_errors:
+ raise error from e
+ logger.warning(
+ "LLM client construction failed",
+ provider=error.provider,
+ category=error.category.value,
+ retryable=error.retryable,
+ )
+ raise NodeUserError(error.user_message) from error
+ finally:
+ if entry is not None:
+ await self._release_client(entry)
async def fetch_models(self, *, provider: str, api_key: str) -> List[str]:
"""List available models from the named provider.
@@ -87,11 +166,48 @@ class ChatUnifier:
filter for free without per-provider Python.
"""
spec = get_provider(provider)
- client = await self._build_client(spec, api_key)
+ entry: Optional[_ClientEntry] = None
try:
- models = await client.fetch_models(api_key)
+ entry = await self._acquire_client(
+ spec, api_key, sdk_max_retries=2
+ )
+ models = await entry.client.fetch_models(api_key)
except spec.sdk_exception_types as e:
- raise NodeUserError(str(e)) from e
+ error = LLMError.from_exception(provider, e)
+ logger.warning(
+ "LLM model-list request failed",
+ provider=error.provider,
+ category=error.category.value,
+ retryable=error.retryable,
+ status_code=error.status_code,
+ provider_code=error.provider_code,
+ request_id=error.request_id,
+ )
+ raise NodeUserError(error.user_message) from error
+ except (ValueError, TypeError, OSError) as e:
+ if entry is not None:
+ raise
+ category = (
+ LLMErrorCategory.CONNECTION
+ if isinstance(e, OSError)
+ else LLMErrorCategory.INVALID_REQUEST
+ )
+ error = LLMError(
+ message=str(e),
+ provider=provider,
+ category=category,
+ retryable=category == LLMErrorCategory.CONNECTION,
+ )
+ logger.warning(
+ "LLM client construction failed",
+ provider=error.provider,
+ category=error.category.value,
+ retryable=error.retryable,
+ )
+ raise NodeUserError(error.user_message) from error
+ finally:
+ if entry is not None:
+ await self._release_client(entry)
blocklist = self._incompatible_models(provider)
if not blocklist:
return models
@@ -111,7 +227,30 @@ class ChatUnifier:
# internals
# ------------------------------------------------------------------
- async def _build_client(self, spec: ProviderSpec, api_key: str) -> LLMProvider:
+ async def aclose(self) -> None:
+ """Close and evict all cached SDK clients during worker shutdown."""
+
+ to_close: List[LLMProvider] = []
+ async with self._cache_lock:
+ entries = list(self._client_cache.values())
+ self._client_cache.clear()
+ for entry in entries:
+ entry.retired = True
+ if entry.leases:
+ if entry not in self._retired_entries:
+ self._retired_entries.append(entry)
+ else:
+ to_close.append(entry.client)
+ for client in to_close:
+ await self._close_client(client)
+
+ async def _build_client(
+ self,
+ spec: ProviderSpec,
+ api_key: str,
+ *,
+ sdk_max_retries: int,
+ ) -> LLMProvider:
"""Instantiate the provider implementation.
Pulls the user-configured ``{provider}_proxy`` URL from the
@@ -119,8 +258,129 @@ class ChatUnifier:
and merges it with the provider's static ``client_kwargs`` (used
by OpenAI-compatible providers to pin their ``base_url``).
"""
+ entry = await self._get_or_create_entry(
+ spec,
+ api_key,
+ sdk_max_retries=sdk_max_retries,
+ acquire=False,
+ )
+ return entry.client
+
+ async def _acquire_client(
+ self,
+ spec: ProviderSpec,
+ api_key: str,
+ *,
+ sdk_max_retries: int,
+ ) -> _ClientEntry:
+ """Return an entry leased until ``_release_client`` is called."""
+
+ return await self._get_or_create_entry(
+ spec,
+ api_key,
+ sdk_max_retries=sdk_max_retries,
+ acquire=True,
+ )
+
+ async def _get_or_create_entry(
+ self,
+ spec: ProviderSpec,
+ api_key: str,
+ *,
+ sdk_max_retries: int,
+ acquire: bool,
+ ) -> _ClientEntry:
proxy_url = await self._auth.get_api_key(f"{spec.name}_proxy")
- return spec.factory(api_key=api_key, proxy_url=proxy_url, **spec.client_kwargs)
+ factory_kwargs = {
+ "api_key": api_key,
+ "proxy_url": proxy_url,
+ "max_retries": max(0, int(sdk_max_retries)),
+ **spec.client_kwargs,
+ }
+ if not self._client_cache_size:
+ return _ClientEntry(
+ client=spec.factory(**factory_kwargs),
+ leases=1 if acquire else 0,
+ retired=True,
+ )
+
+ key = self._client_cache_key(
+ spec=spec,
+ api_key=api_key,
+ proxy_url=proxy_url,
+ sdk_max_retries=sdk_max_retries,
+ )
+ evicted: Optional[_ClientEntry] = None
+ async with self._cache_lock:
+ cached = self._client_cache.get(key)
+ if cached is not None:
+ self._client_cache.move_to_end(key)
+ if acquire:
+ cached.leases += 1
+ return cached
+ entry = _ClientEntry(
+ client=spec.factory(**factory_kwargs),
+ leases=1 if acquire else 0,
+ )
+ self._client_cache[key] = entry
+ if len(self._client_cache) > self._client_cache_size:
+ _, evicted = self._client_cache.popitem(last=False)
+ evicted.retired = True
+ if evicted.leases:
+ self._retired_entries.append(evicted)
+ if evicted is not None:
+ if not evicted.leases:
+ await self._close_client(evicted.client)
+ return entry
+
+ async def _release_client(self, entry: _ClientEntry) -> None:
+ should_close = False
+ async with self._cache_lock:
+ if entry.leases:
+ entry.leases -= 1
+ if entry.retired and not entry.leases:
+ if entry in self._retired_entries:
+ self._retired_entries.remove(entry)
+ should_close = True
+ if should_close:
+ await self._close_client(entry.client)
+
+ @staticmethod
+ def _client_cache_key(
+ *,
+ spec: ProviderSpec,
+ api_key: str,
+ proxy_url: Optional[str],
+ sdk_max_retries: int,
+ ) -> str:
+ credential_fingerprint = hashlib.sha256(
+ api_key.encode("utf-8")
+ ).hexdigest()
+ material = json.dumps(
+ {
+ "provider": spec.name,
+ "proxy_url": proxy_url or "",
+ "client_kwargs": spec.client_kwargs,
+ "credential": credential_fingerprint,
+ "max_retries": max(0, int(sdk_max_retries)),
+ },
+ sort_keys=True,
+ separators=(",", ":"),
+ default=str,
+ )
+ return hashlib.sha256(material.encode("utf-8")).hexdigest()
+
+ @staticmethod
+ async def _close_client(client: LLMProvider) -> None:
+ close = getattr(client, "aclose", None)
+ if close is None:
+ return
+ try:
+ result = close()
+ if hasattr(result, "__await__"):
+ await result
+ except Exception as exc:
+ logger.warning("Failed to close cached LLM client", error=str(exc))
def _incompatible_models(self, provider: str) -> set[str]:
"""Read ``providers.<name>.incompatible_models`` from llm_defaults.json."""
diff --git a/server/services/memory/__init__.py b/server/services/memory/__init__.py
index 69759ad9..c0724938 100644
--- a/server/services/memory/__init__.py
+++ b/server/services/memory/__init__.py
@@ -7,8 +7,8 @@ storage format / concern owns its own module:
(used by aiAgent / chatAgent / rlm_agent)
- :mod:`services.memory.jsonl` — Anthropic Messages JSONL
parse/append/trim (used by claude_code_agent's session bridge)
- - :mod:`services.memory.vector_store` — per-session
- ``InMemoryVectorStore`` for long-term archival
+ - :mod:`services.memory.vector_store` — per-session native semantic
+ store for long-term archival
- :mod:`services.memory.state` — orchestration: clear every store
keyed by an agent's conversational scope
@@ -23,7 +23,11 @@ from services.memory.markdown import (
trim_markdown_window,
)
from services.memory.vector_store import (
+ AsyncEmbedder,
+ NativeMemoryVectorStore,
_memory_vector_stores,
+ clear_memory_vector_stores,
+ create_embedder,
get_memory_vector_store,
)
from services.memory.state import clear_agent_session_state
@@ -44,8 +48,12 @@ __all__ = [
"append_to_memory_markdown",
"trim_markdown_window",
# Vector-store helpers.
+ "AsyncEmbedder",
+ "NativeMemoryVectorStore",
"_memory_vector_stores",
+ "create_embedder",
"get_memory_vector_store",
+ "clear_memory_vector_stores",
# Orchestration.
"clear_agent_session_state",
# JSONL helpers — standalone primitive, not currently used by any
diff --git a/server/services/memory/jsonl.py b/server/services/memory/jsonl.py
index fc57663f..3cd004a6 100644
--- a/server/services/memory/jsonl.py
+++ b/server/services/memory/jsonl.py
@@ -13,18 +13,18 @@ Used by :mod:`nodes.agent.claude_code_agent` (session memory) via
import json
from typing import Any, List, Tuple
-from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
+from services.llm.protocol import Message
-def parse_jsonl(text: str) -> List[BaseMessage]:
- """Standard JSONL -> LangChain ``BaseMessage`` list.
+def parse_jsonl(text: str) -> List[Message]:
+ """Standard JSONL -> native :class:`Message` list.
Tool-call content blocks collapse to text; rows with unknown roles
or unparseable JSON are skipped (forward compatibility).
"""
if not text:
return []
- out: List[BaseMessage] = []
+ out: List[Message] = []
for line in text.splitlines():
line = line.strip()
if not line:
@@ -44,9 +44,9 @@ def parse_jsonl(text: str) -> List[BaseMessage]:
if not isinstance(content, str):
continue
if role == "user":
- out.append(HumanMessage(content=content))
+ out.append(Message(role="user", content=content))
elif role == "assistant":
- out.append(AIMessage(content=content))
+ out.append(Message(role="assistant", content=content))
return out
diff --git a/server/services/memory/markdown.py b/server/services/memory/markdown.py
index 2d0b6d46..97342739 100644
--- a/server/services/memory/markdown.py
+++ b/server/services/memory/markdown.py
@@ -11,11 +11,11 @@ import re
from datetime import datetime
from typing import List
-from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
+from services.llm.protocol import Message
-def parse_memory_markdown(content: str) -> List[BaseMessage]:
- """Parse markdown memory content into LangChain messages.
+def parse_memory_markdown(content: str) -> List[Message]:
+ """Parse markdown memory content into native LLM messages.
Markdown format:
### **Human** (timestamp)
@@ -24,13 +24,17 @@ def parse_memory_markdown(content: str) -> List[BaseMessage]:
### **Assistant** (timestamp)
response content
"""
- messages: List[BaseMessage] = []
+ messages: List[Message] = []
pattern = r"### \*\*(Human|Assistant)\*\*[^\n]*\n(.*?)(?=\n### \*\*|$)"
for role, text in re.findall(pattern, content, re.DOTALL):
text = text.strip()
if text:
- msg_class = HumanMessage if role == "Human" else AIMessage
- messages.append(msg_class(content=text))
+ messages.append(
+ Message(
+ role="user" if role == "Human" else "assistant",
+ content=text,
+ )
+ )
return messages
diff --git a/server/services/memory/state.py b/server/services/memory/state.py
index d9ffeb82..770e4887 100644
--- a/server/services/memory/state.py
+++ b/server/services/memory/state.py
@@ -54,17 +54,13 @@ async def clear_agent_session_state(
(list[str]), and ``cleared_memory_node`` (bool) for
caller-visible diagnostics.
"""
- # The live vector-store cache lives in ``services.ai`` (the dict in
- # ``services.memory.vector_store`` is dormant — future cleanup will
- # consolidate). Lazy import keeps ``services.ai``'s heavy LangChain
- # deps off the hot path.
- from services.ai import _memory_vector_stores as _live_vector_stores
+ from services.memory.vector_store import clear_memory_vector_stores
from services.todo_service import UNSAVED_WORKFLOW_ID, get_todo_service
cleared_vector_store = False
- if clear_long_term and session_id in _live_vector_stores:
- del _live_vector_stores[session_id]
- cleared_vector_store = True
+ if clear_long_term:
+ cleared_vector_store = await clear_memory_vector_stores(session_id)
+ if cleared_vector_store:
logger.info(f"[Memory] Cleared vector store for session '{session_id}'")
todo_service = get_todo_service()
diff --git a/server/services/memory/vector_store.py b/server/services/memory/vector_store.py
index 74b36677..bfe908fb 100644
--- a/server/services/memory/vector_store.py
+++ b/server/services/memory/vector_store.py
@@ -1,40 +1,592 @@
-"""Per-session ``InMemoryVectorStore`` for long-term memory archival.
+"""Native async embedders and a small in-memory cosine index.
-Note: a duplicate dict ``_memory_vector_stores`` lives in
-:mod:`services.ai`. ``services.memory.state.clear_agent_session_state``
-clears the live (``services.ai``) one — this module's dict is an
-intentional second copy that future cleanup will consolidate. Keep both
-for now; the documented duplication is preserved verbatim.
+The default Hugging Face backend is optional. Merely enabling long-term
+memory never imports PyTorch or downloads a model on the event-loop thread:
+model construction and encoding run through :func:`asyncio.to_thread`.
"""
-from typing import Any, Dict
+from __future__ import annotations
+
+import asyncio
+import hashlib
+import importlib.util
+import inspect
+import math
+import sys
+import threading
+from dataclasses import dataclass, field
+from typing import Any, Dict, Iterable, List, Literal, Optional, Protocol
+from uuid import uuid4
from core.logging import get_logger
logger = get_logger(__name__)
+EmbeddingProvider = Literal["huggingface", "openai", "ollama"]
+DEFAULT_EMBEDDING_MODELS: Dict[str, str] = {
+ "huggingface": "BAAI/bge-small-en-v1.5",
+ "openai": "text-embedding-3-small",
+ "ollama": "nomic-embed-text",
+}
-# Per-``session_id`` cache. The live cache used by aiAgent etc. lives
-# in ``services.ai``; this dict is the package-private counterpart for
-# JSONL-bridge consumers and is referenced by state-clear orchestration.
-_memory_vector_stores: Dict[str, Any] = {}
+@dataclass(frozen=True)
+class MemoryVectorStoreKey:
+ """Non-secret identity for one session's compatible vector space."""
-def get_memory_vector_store(session_id: str):
- """Get or create an ``InMemoryVectorStore`` for a session.
+ session_id: str
+ provider: str
+ model: str
+ endpoint: str
+ credential_fingerprint: str
- Returns ``None`` (with a warning) when the optional LangChain
- HuggingFace dep is missing — long-term archival is best-effort.
- """
- if session_id not in _memory_vector_stores:
+
+# Legacy string keys remain accepted so state-clear can clean stores created by
+# a worker running the pre-migration code during a rolling deployment.
+_memory_vector_stores: Dict[MemoryVectorStoreKey | str, Any] = {}
+_memory_vector_stores_lock = threading.RLock()
+
+
+@dataclass(frozen=True)
+class MemoryDocument:
+ """Search result compatible with the historical ``page_content`` API."""
+
+ page_content: str
+ metadata: Dict[str, Any] = field(default_factory=dict)
+
+
+class EmbeddingError(RuntimeError):
+ """Base class for user-safe embedding configuration/request failures."""
+
+
+class EmbedderUnavailableError(EmbeddingError):
+ """The requested direct SDK or optional embedding extra is unavailable."""
+
+
+class EmbeddingProviderError(EmbeddingError):
+ """A provider SDK failed while generating embeddings."""
+
+
+class AsyncEmbedder(Protocol):
+ """Provider-neutral async embedding contract."""
+
+ async def embed_documents(
+ self,
+ texts: List[str],
+ *,
+ batch_size: int = 32,
+ ) -> List[List[float]]: ...
+
+ async def embed_query(self, text: str) -> List[float]: ...
+
+ async def aclose(self) -> None: ...
+
+
+# Compatibility name for code that imported the old protocol.
+Embedder = AsyncEmbedder
+
+
+def default_embedding_model(provider: str) -> str:
+ normalized = str(provider or "huggingface").strip().lower()
+ try:
+ return DEFAULT_EMBEDDING_MODELS[normalized]
+ except KeyError as exc:
+ raise ValueError(f"Unsupported embedding provider: {provider}") from exc
+
+
+def _normalize_provider(provider: str) -> EmbeddingProvider:
+ normalized = str(provider or "huggingface").strip().lower()
+ if normalized not in DEFAULT_EMBEDDING_MODELS:
+ raise ValueError(f"Unsupported embedding provider: {provider}")
+ return normalized # type: ignore[return-value]
+
+
+def _normalize_endpoint(endpoint: Optional[str]) -> str:
+ return str(endpoint or "").strip().rstrip("/")
+
+
+def _module_available(module_name: str) -> bool:
+ if module_name in sys.modules:
+ return sys.modules[module_name] is not None
+ try:
+ return importlib.util.find_spec(module_name) is not None
+ except (ImportError, ValueError):
+ return False
+
+
+def _vector_lists(vectors: Any) -> List[List[float]]:
+ if hasattr(vectors, "tolist"):
+ vectors = vectors.tolist()
+ return [
+ [float(value) for value in vector]
+ for vector in (vectors or [])
+ ]
+
+
+async def _close_sdk_client(client: Any) -> None:
+ close = getattr(client, "close", None)
+ if close is None:
+ return
+ result = close()
+ if inspect.isawaitable(result):
+ await result
+
+
+class SentenceTransformerEmbedder:
+ """Lazy, non-blocking adapter for the optional sentence-transformers SDK."""
+
+ def __init__(self, model_name: str) -> None:
+ self.model_name = model_name
+ self._model: Any = None
+ self._model_lock = threading.Lock()
+
+ def _encode(
+ self,
+ texts: List[str],
+ batch_size: int,
+ ) -> List[List[float]]:
+ # Loading a model may touch disk or download model assets. It executes
+ # only in the worker thread that also performs CPU-heavy encoding.
+ with self._model_lock:
+ if self._model is None:
+ try:
+ from sentence_transformers import SentenceTransformer
+ except ImportError as exc:
+ raise EmbedderUnavailableError(
+ "HuggingFace embeddings unavailable. Install the "
+ "optional 'local-embeddings' extra."
+ ) from exc
+
+ self._model = SentenceTransformer(self.model_name)
+ vectors = self._model.encode(
+ texts,
+ batch_size=batch_size,
+ convert_to_numpy=True,
+ show_progress_bar=False,
+ )
+ return _vector_lists(vectors)
+
+ async def embed_documents(
+ self,
+ texts: List[str],
+ *,
+ batch_size: int = 32,
+ ) -> List[List[float]]:
+ if not texts:
+ return []
+ try:
+ return await asyncio.to_thread(
+ self._encode,
+ list(texts),
+ batch_size,
+ )
+ except EmbedderUnavailableError:
+ raise
+ except Exception as exc:
+ raise EmbeddingProviderError(
+ "HuggingFace embedding generation failed "
+ f"({type(exc).__name__})"
+ ) from exc
+
+ async def embed_query(self, text: str) -> List[float]:
+ vectors = await self.embed_documents([text], batch_size=1)
+ return vectors[0] if vectors else []
+
+ async def aclose(self) -> None:
+ return None
+
+
+class OpenAIEmbedder:
+ """Direct ``AsyncOpenAI.embeddings`` adapter."""
+
+ def __init__(
+ self,
+ model_name: str,
+ *,
+ api_key: Optional[str],
+ endpoint: Optional[str] = None,
+ ) -> None:
+ try:
+ from openai import AsyncOpenAI
+ except ImportError as exc:
+ raise EmbedderUnavailableError(
+ "OpenAI embeddings unavailable. Install the 'openai' package."
+ ) from exc
+
+ kwargs: Dict[str, Any] = {"api_key": api_key or None}
+ if endpoint:
+ kwargs["base_url"] = endpoint
+ self.model_name = model_name
try:
- from langchain_core.vectorstores import InMemoryVectorStore
- from langchain_huggingface import HuggingFaceEmbeddings
+ self._client = AsyncOpenAI(**kwargs)
+ except Exception as exc:
+ raise EmbedderUnavailableError(
+ "OpenAI embedding client initialization failed "
+ f"({type(exc).__name__})"
+ ) from exc
+
+ async def embed_documents(
+ self,
+ texts: List[str],
+ *,
+ batch_size: int = 32,
+ ) -> List[List[float]]:
+ embeddings: List[List[float]] = []
+ for start in range(0, len(texts), batch_size):
+ batch = texts[start : start + batch_size]
+ try:
+ response = await self._client.embeddings.create(
+ model=self.model_name,
+ input=batch,
+ )
+ except Exception as exc:
+ raise EmbeddingProviderError(
+ "OpenAI embedding request failed "
+ f"({type(exc).__name__})"
+ ) from exc
+ data = sorted(response.data, key=lambda item: int(item.index))
+ vectors = [
+ [float(value) for value in item.embedding]
+ for item in data
+ ]
+ if len(vectors) != len(batch):
+ raise ValueError(
+ "OpenAI returned a different number of embeddings "
+ "than requested"
+ )
+ embeddings.extend(vectors)
+ return embeddings
- embeddings = HuggingFaceEmbeddings(model_name="BAAI/bge-small-en-v1.5")
- _memory_vector_stores[session_id] = InMemoryVectorStore(embeddings)
- logger.debug(f"[Memory] Created vector store for session '{session_id}'")
+ async def embed_query(self, text: str) -> List[float]:
+ vectors = await self.embed_documents([text], batch_size=1)
+ return vectors[0] if vectors else []
+
+ async def aclose(self) -> None:
+ await _close_sdk_client(self._client)
+
+
+class OllamaEmbedder:
+ """Direct ``ollama.AsyncClient.embed`` adapter."""
+
+ def __init__(
+ self,
+ model_name: str,
+ *,
+ endpoint: Optional[str] = None,
+ ) -> None:
+ try:
+ from ollama import AsyncClient
except ImportError as exc:
- logger.warning(f"[Memory] Vector store not available: {exc}")
+ raise EmbedderUnavailableError(
+ "Ollama embeddings unavailable. Install the 'ollama' package."
+ ) from exc
+
+ kwargs: Dict[str, Any] = {}
+ if endpoint:
+ kwargs["host"] = endpoint
+ self.model_name = model_name
+ try:
+ self._client = AsyncClient(**kwargs)
+ except Exception as exc:
+ raise EmbedderUnavailableError(
+ "Ollama embedding client initialization failed "
+ f"({type(exc).__name__})"
+ ) from exc
+
+ async def embed_documents(
+ self,
+ texts: List[str],
+ *,
+ batch_size: int = 32,
+ ) -> List[List[float]]:
+ embeddings: List[List[float]] = []
+ for start in range(0, len(texts), batch_size):
+ batch = texts[start : start + batch_size]
+ try:
+ response = await self._client.embed(
+ model=self.model_name,
+ input=batch,
+ )
+ except Exception as exc:
+ raise EmbeddingProviderError(
+ "Ollama embedding request failed "
+ f"({type(exc).__name__})"
+ ) from exc
+ vectors = (
+ response.get("embeddings")
+ if isinstance(response, dict)
+ else getattr(response, "embeddings", None)
+ )
+ if vectors is None:
+ raise ValueError("Ollama returned no embeddings")
+ batch_vectors = _vector_lists(vectors)
+ if len(batch_vectors) != len(batch):
+ raise ValueError(
+ "Ollama returned a different number of embeddings "
+ "than requested"
+ )
+ embeddings.extend(batch_vectors)
+ return embeddings
+
+ async def embed_query(self, text: str) -> List[float]:
+ vectors = await self.embed_documents([text], batch_size=1)
+ return vectors[0] if vectors else []
+
+ async def aclose(self) -> None:
+ await _close_sdk_client(self._client)
+
+
+def create_embedder(
+ provider: str,
+ *,
+ model: Optional[str] = None,
+ api_key: Optional[str] = None,
+ endpoint: Optional[str] = None,
+) -> AsyncEmbedder:
+ """Build a direct-SDK embedder without making a network request."""
+
+ normalized = _normalize_provider(provider)
+ model_name = str(model or "").strip() or default_embedding_model(normalized)
+ normalized_endpoint = _normalize_endpoint(endpoint)
+ if normalized == "huggingface":
+ if not _module_available("sentence_transformers"):
+ raise EmbedderUnavailableError(
+ "HuggingFace embeddings unavailable. Install the optional "
+ "'local-embeddings' extra."
+ )
+ return SentenceTransformerEmbedder(model_name)
+ if normalized == "openai":
+ if not str(api_key or "").strip():
+ raise EmbedderUnavailableError(
+ "OpenAI API key is required for embeddings."
+ )
+ return OpenAIEmbedder(
+ model_name,
+ api_key=api_key,
+ endpoint=normalized_endpoint or None,
+ )
+ return OllamaEmbedder(
+ model_name,
+ endpoint=normalized_endpoint or None,
+ )
+
+
+class NativeMemoryVectorStore:
+ """Concurrency-safe async cosine index with the historical tiny API."""
+
+ def __init__(
+ self,
+ embedder: AsyncEmbedder,
+ *,
+ batch_size: int = 32,
+ ) -> None:
+ self._embedder = embedder
+ self._batch_size = batch_size
+ self._entries: List[
+ tuple[str, str, List[float], Dict[str, Any]]
+ ] = []
+ self._dimension: Optional[int] = None
+ self._lock = threading.RLock()
+
+ async def add_texts(
+ self,
+ texts: Iterable[str],
+ metadatas: Optional[Iterable[Dict[str, Any]]] = None,
+ **_: Any,
+ ) -> List[str]:
+ values = [str(text) for text in texts]
+ if not values:
+ return []
+ metadata_values = list(metadatas or ({} for _ in values))
+ if len(metadata_values) != len(values):
+ raise ValueError("metadatas must have the same length as texts")
+ vectors = await self._embedder.embed_documents(
+ values,
+ batch_size=self._batch_size,
+ )
+ if len(vectors) != len(values):
+ raise ValueError("embedder returned a different number of vectors")
+ dimensions = {len(vector) for vector in vectors}
+ if len(dimensions) != 1 or not dimensions or 0 in dimensions:
+ raise ValueError("embedder returned empty or inconsistent vectors")
+ dimension = dimensions.pop()
+ ids = [uuid4().hex for _ in values]
+ with self._lock:
+ if self._dimension is not None and self._dimension != dimension:
+ raise ValueError(
+ "embedding dimension changed within one vector store"
+ )
+ self._dimension = dimension
+ self._entries.extend(
+ (
+ item_id,
+ text,
+ [float(value) for value in vector],
+ dict(metadata),
+ )
+ for item_id, text, vector, metadata in zip(
+ ids,
+ values,
+ vectors,
+ metadata_values,
+ )
+ )
+ return ids
+
+ @staticmethod
+ def _cosine(left: List[float], right: List[float]) -> float:
+ if len(left) != len(right) or not left:
+ return float("-inf")
+ denominator = math.sqrt(sum(v * v for v in left)) * math.sqrt(
+ sum(v * v for v in right)
+ )
+ if denominator == 0:
+ return 0.0
+ return sum(a * b for a, b in zip(left, right)) / denominator
+
+ async def similarity_search(
+ self,
+ query: str,
+ k: int = 4,
+ **_: Any,
+ ) -> List[MemoryDocument]:
+ if k <= 0:
+ return []
+ with self._lock:
+ entries = list(self._entries)
+ dimension = self._dimension
+ if not entries:
+ return []
+ query_vector = [
+ float(value)
+ for value in await self._embedder.embed_query(str(query))
+ ]
+ if dimension is not None and len(query_vector) != dimension:
+ raise ValueError(
+ "query embedding dimension does not match stored vectors"
+ )
+ ranked = sorted(
+ entries,
+ key=lambda entry: self._cosine(query_vector, entry[2]),
+ reverse=True,
+ )
+ return [
+ MemoryDocument(page_content=text, metadata=dict(metadata))
+ for _, text, _, metadata in ranked[:k]
+ ]
+
+ async def aclose(self) -> None:
+ await self._embedder.aclose()
+
+
+def _credential_fingerprint(api_key: Optional[str]) -> str:
+ if not api_key:
+ return "none"
+ return hashlib.sha256(api_key.encode("utf-8")).hexdigest()[:16]
+
+
+async def get_memory_vector_store(
+ session_id: str,
+ *,
+ provider: str = "huggingface",
+ model: Optional[str] = None,
+ endpoint: Optional[str] = None,
+ api_key: Optional[str] = None,
+ auth_service: Any = None,
+) -> Optional[NativeMemoryVectorStore]:
+ """Get one compatible native semantic store for a memory session.
+
+ OpenAI credentials are accepted only in-memory and represented in the
+ cache key by a short SHA-256 fingerprint. The plaintext credential is
+ never persisted in the key or store metadata.
+
+ Missing optional Hugging Face support and missing OpenAI credentials are
+ deliberate no-op outcomes so long-term memory cannot break agent runs.
+ """
+
+ normalized = _normalize_provider(provider)
+ model_name = str(model or "").strip() or default_embedding_model(normalized)
+ normalized_endpoint = _normalize_endpoint(endpoint)
+ resolved_key = str(api_key or "").strip() or None
+ if normalized == "openai" and resolved_key is None:
+ if auth_service is not None:
+ try:
+ auth_key = await auth_service.get_api_key(
+ "openai",
+ "default",
+ )
+ resolved_key = str(auth_key or "").strip() or None
+ except Exception as exc: # noqa: BLE001 - best-effort memory
+ logger.warning(
+ "[Memory] Could not resolve OpenAI embedding credential: %s",
+ exc,
+ )
+ return None
+ if not resolved_key:
+ logger.warning(
+ "[Memory] OpenAI long-term memory is disabled because no "
+ "OpenAI credential is configured"
+ )
+ return None
+
+ cache_key = MemoryVectorStoreKey(
+ session_id=str(session_id),
+ provider=normalized,
+ model=model_name,
+ endpoint=normalized_endpoint,
+ credential_fingerprint=_credential_fingerprint(resolved_key),
+ )
+ with _memory_vector_stores_lock:
+ cached = _memory_vector_stores.get(cache_key)
+ if cached is not None:
+ return cached
+ try:
+ embedder = create_embedder(
+ normalized,
+ model=model_name,
+ api_key=resolved_key,
+ endpoint=normalized_endpoint,
+ )
+ except EmbedderUnavailableError as exc:
+ logger.warning("[Memory] Vector store not available: %s", exc)
return None
- return _memory_vector_stores[session_id]
+ store = NativeMemoryVectorStore(embedder)
+ _memory_vector_stores[cache_key] = store
+ logger.debug(
+ "[Memory] Created vector store for session=%s provider=%s model=%s",
+ session_id,
+ normalized,
+ model_name,
+ )
+ return store
+
+
+async def clear_memory_vector_stores(session_id: str) -> bool:
+ """Remove and close every provider/model store for ``session_id``."""
+
+ with _memory_vector_stores_lock:
+ matching = [
+ key
+ for key in _memory_vector_stores
+ if key == session_id
+ or (
+ isinstance(key, MemoryVectorStoreKey)
+ and key.session_id == session_id
+ )
+ ]
+ stores = [_memory_vector_stores.pop(key) for key in matching]
+ seen: set[int] = set()
+ for store in stores:
+ if id(store) in seen:
+ continue
+ seen.add(id(store))
+ close = getattr(store, "aclose", None)
+ if close is None:
+ continue
+ try:
+ result = close()
+ if inspect.isawaitable(result):
+ await result
+ except Exception as exc: # noqa: BLE001 - clearing remains best effort
+ logger.warning("[Memory] Failed to close vector store: %s", exc)
+ return bool(matching)
diff --git a/server/services/memory_store.py b/server/services/memory_store.py
index 7ef66cf8..c9e5c42f 100644
--- a/server/services/memory_store.py
+++ b/server/services/memory_store.py
@@ -1,32 +1,22 @@
-"""Simple in-memory conversation storage for AI agents.
+"""Simple in-memory conversation storage for AI agents."""
-Uses standard Python data structures with LangChain message compatibility.
-No deprecated APIs - follows LangChain 0.3+ recommendations.
-"""
-
-from typing import Dict, List, Optional
-from dataclasses import dataclass, field
from datetime import datetime
import logging
+from dataclasses import dataclass, field
+from typing import Dict, List, Optional
-logger = logging.getLogger(__name__)
-
-
-@dataclass
-class Message:
- """Single conversation message."""
+from services.llm.protocol import Message
- role: str # 'human' or 'ai'
- content: str
- timestamp: str = field(default_factory=lambda: datetime.now().isoformat())
+logger = logging.getLogger(__name__)
@dataclass
class ConversationSession:
- """Conversation session with message history."""
+ """Conversation session storing canonical native LLM messages."""
session_id: str
messages: List[Message] = field(default_factory=list)
+ message_timestamps: List[str] = field(default_factory=list)
created_at: str = field(default_factory=lambda: datetime.now().isoformat())
@@ -45,7 +35,14 @@ def get_session(session_id: str) -> ConversationSession:
def add_message(session_id: str, role: str, content: str) -> None:
"""Add a message to a session."""
session = get_session(session_id)
- session.messages.append(Message(role=role, content=content))
+ canonical_role = {
+ "human": "user",
+ "ai": "assistant",
+ }.get(role, role)
+ session.messages.append(
+ Message(role=canonical_role, content=str(content))
+ )
+ session.message_timestamps.append(datetime.now().isoformat())
logger.info(f"[Memory] Added {role} message to session '{session_id}' (total: {len(session.messages)})")
@@ -53,9 +50,18 @@ def get_messages(session_id: str, window_size: Optional[int] = None) -> List[Dic
"""Get messages from a session, optionally limited to last N."""
session = get_session(session_id)
messages = session.messages
+ timestamps = session.message_timestamps
if window_size and window_size > 0:
messages = messages[-window_size:]
- return [{"role": m.role, "content": m.content, "timestamp": m.timestamp} for m in messages]
+ timestamps = timestamps[-window_size:]
+ return [
+ {
+ "role": message.role,
+ "content": message.content,
+ "timestamp": timestamp,
+ }
+ for message, timestamp in zip(messages, timestamps)
+ ]
def clear_session(session_id: str) -> int:
@@ -63,6 +69,7 @@ def clear_session(session_id: str) -> int:
if session_id in _sessions:
count = len(_sessions[session_id].messages)
_sessions[session_id].messages = []
+ _sessions[session_id].message_timestamps = []
logger.info(f"[Memory] Cleared {count} messages from session '{session_id}'")
return count
return 0
@@ -82,18 +89,17 @@ def get_all_sessions() -> List[Dict]:
return [{"session_id": s.session_id, "message_count": len(s.messages), "created_at": s.created_at} for s in _sessions.values()]
-def get_langchain_messages(session_id: str, window_size: Optional[int] = None):
- """Convert session messages to LangChain message format.
+def get_native_messages(
+ session_id: str,
+ window_size: Optional[int] = None,
+) -> List[Message]:
+ """Return the canonical messages already stored by the session."""
- Uses langchain_core.messages (modern API, not deprecated).
- """
- from langchain_core.messages import HumanMessage, AIMessage
-
- messages = get_messages(session_id, window_size)
- lc_messages = []
- for m in messages:
- if m["role"] == "human":
- lc_messages.append(HumanMessage(content=m["content"]))
- elif m["role"] == "ai":
- lc_messages.append(AIMessage(content=m["content"]))
- return lc_messages
+ messages = list(get_session(session_id).messages)
+ if window_size and window_size > 0:
+ messages = messages[-window_size:]
+ return [
+ message
+ for message in messages
+ if message.role in {"user", "assistant"}
+ ]
diff --git a/server/services/plugin/edge_walker.py b/server/services/plugin/edge_walker.py
index 48eb6c82..ddc98aa2 100644
--- a/server/services/plugin/edge_walker.py
+++ b/server/services/plugin/edge_walker.py
@@ -294,6 +294,16 @@ async def _build_memory_entry(
session_id = configured_session
else:
session_id = agent_node_id
+ embedding_provider = str(
+ memory_params.get("embedding_provider") or "huggingface"
+ ).strip().lower()
+ embedding_model = str(
+ memory_params.get("embedding_model") or {
+ "huggingface": "BAAI/bge-small-en-v1.5",
+ "openai": "text-embedding-3-small",
+ "ollama": "nomic-embed-text",
+ }.get(embedding_provider, "")
+ ).strip()
entry = {
"node_id": memory_node_id,
"session_id": session_id,
@@ -304,6 +314,11 @@ async def _build_memory_entry(
),
"long_term_enabled": memory_params.get("long_term_enabled", False),
"retrieval_count": int(memory_params.get("retrieval_count", 3)),
+ "embedding_provider": embedding_provider,
+ "embedding_model": embedding_model,
+ "embedding_endpoint": str(
+ memory_params.get("embedding_endpoint") or ""
+ ).strip(),
# Claude Code CLI session continuity: the UUID claude returned on
# the previous successful run. claude_code_agent passes this as
# `--resume <UUID>` so claude finds its own JSONL transcript at
diff --git a/server/services/plugin/tool.py b/server/services/plugin/tool.py
index fd848387..fde78a30 100644
--- a/server/services/plugin/tool.py
+++ b/server/services/plugin/tool.py
@@ -7,6 +7,7 @@ JSON Schema the LLM sees — derived from :class:`Params` automatically.
from __future__ import annotations
+from copy import deepcopy
from typing import Any, ClassVar, Dict, Optional
from core.logging import get_logger
@@ -22,17 +23,76 @@ def inline_schema_refs(schema: Dict[str, Any]) -> Dict[str, Any]:
Function-calling APIs reject schema indirection, but a Params model
with nested BaseModel or Enum fields emits ``$defs`` + ``$ref`` under
Pydantic v2 — stripping ``$defs`` alone would leave dangling refs.
- Circular refs degrade to a permissive ``{}`` at the cycle point
- (``dereference_refs`` contract); a dereference failure (out-of-document
- ref, malformed schema) falls back to a permissive object schema.
+ Circular refs degrade to a permissive ``{}`` at the cycle point.
+ A dereference failure (out-of-document ref, malformed schema) falls
+ back to a permissive object schema.
+
+ Only local JSON Pointers are accepted. Tool schemas are generated from
+ trusted Pydantic models, but keeping the resolver local and document-only
+ also prevents an accidental network/file fetch if an externally supplied
+ schema reaches this helper.
"""
- # Lazy import: langchain_core is heavyweight and this module loads at
- # plugin-registration time (cold-start rule).
- from langchain_core.utils.json_schema import dereference_refs
+ root = deepcopy(schema)
+
+ def _unescape_pointer_token(token: str) -> str:
+ return token.replace("~1", "/").replace("~0", "~")
+
+ def _lookup(ref: str) -> Any:
+ if ref == "#":
+ return root
+ if not ref.startswith("#/"):
+ raise ValueError(f"Only local JSON Pointer refs are supported: {ref!r}")
+ current: Any = root
+ for raw_token in ref[2:].split("/"):
+ token = _unescape_pointer_token(raw_token)
+ if isinstance(current, dict):
+ current = current[token]
+ elif isinstance(current, list):
+ current = current[int(token)]
+ else:
+ raise ValueError(f"Cannot traverse JSON Pointer {ref!r}")
+ return current
+
+ def _resolve(value: Any, active_refs: frozenset[str]) -> Any:
+ if isinstance(value, list):
+ return [_resolve(item, active_refs) for item in value]
+ if not isinstance(value, dict):
+ return value
+
+ if "$ref" in value:
+ ref = value["$ref"]
+ if not isinstance(ref, str):
+ raise ValueError("$ref must be a string")
+ if ref in active_refs:
+ return {}
+ target = _lookup(ref)
+ if not isinstance(target, (dict, list)):
+ raise ValueError(f"JSON Pointer {ref!r} does not target a schema")
+ resolved = _resolve(deepcopy(target), active_refs | {ref})
+ # JSON Schema permits siblings alongside $ref. Preserve them,
+ # with the referencing site taking precedence over the target.
+ siblings = {
+ key: _resolve(item, active_refs)
+ for key, item in value.items()
+ if key != "$ref"
+ }
+ if siblings:
+ if not isinstance(resolved, dict):
+ raise ValueError(f"Cannot merge siblings into ref {ref!r}")
+ resolved.update(siblings)
+ return resolved
+
+ return {
+ key: _resolve(item, active_refs)
+ for key, item in value.items()
+ if key not in {"$defs", "definitions"}
+ }
try:
- inlined = dereference_refs(schema)
- except Exception as e: # noqa: BLE001 — KeyError/ValueError from _retrieve_ref
+ inlined = _resolve(root, frozenset())
+ if not isinstance(inlined, dict):
+ raise ValueError("Root JSON Schema must be an object")
+ except (KeyError, IndexError, TypeError, ValueError) as e:
logger.warning(
"inline_schema_refs: dereference failed (%s) — falling back to permissive schema",
e,
diff --git a/server/services/rlm/service.py b/server/services/rlm/service.py
index bde4e435..0428cb52 100644
--- a/server/services/rlm/service.py
+++ b/server/services/rlm/service.py
@@ -69,7 +69,10 @@ class RLMService:
flattened = {**parameters, **options}
# Provider/model/API key resolution (same as lines 1998-2014)
- from services.ai import is_model_valid_for_provider, get_default_model_async
+ from services.llm.config import (
+ get_default_model_async,
+ is_model_valid_for_provider,
+ )
api_key = flattened.get("api_key")
provider = parameters.get("provider", "openai")
diff --git a/server/services/settings/handlers.py b/server/services/settings/handlers.py
index 95b31cca..dd2f7059 100644
--- a/server/services/settings/handlers.py
+++ b/server/services/settings/handlers.py
@@ -73,7 +73,7 @@ async def handle_save_user_settings(data: Dict[str, Any], websocket: WebSocket)
async def handle_get_provider_defaults(data: Dict[str, Any], websocket: WebSocket) -> Dict[str, Any]:
"""Get default parameters for a provider."""
from core.container import container
- from services.ai import get_default_model
+ from services.llm.config import get_default_model
from services.model_registry import get_model_registry
database = container.database()
@@ -134,7 +134,7 @@ async def handle_get_validated_ai_providers(
auth_service = container.auth_service()
database = container.database()
- from services.ai import PROVIDER_CONFIGS
+ from services.llm.config import PROVIDER_CONFIGS
AI_PROVIDERS = list(PROVIDER_CONFIGS.keys())
diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.py
index 9dbf97fc..3bda5366 100644
--- a/server/services/temporal/agent_activities.py
+++ b/server/services/temporal/agent_activities.py
@@ -35,14 +35,20 @@ References:
from __future__ import annotations
import asyncio
+from dataclasses import asdict
import hashlib
import logging
+import os
from typing import Any, Dict, List, Optional
from temporalio import activity
logger = logging.getLogger(__name__)
+_NATIVE_LLM_ENGINE = "native"
+_LEGACY_LLM_ENGINE = "langchain"
+_LLM_ENGINES = frozenset({_NATIVE_LLM_ENGINE, _LEGACY_LLM_ENGINE})
+
# Activity result shapes — keep these in sync with AgentWorkflow's
# expectations. Pydantic was considered but plain dicts keep the
@@ -65,7 +71,11 @@ def _ensure_llm_contents(messages: List[Any]) -> None:
docs.temporal.io/encyclopedia/retry-policies) — instead of burning
the retry budget on a deterministic failure.
"""
- if any(getattr(m, "type", "") in ("human", "ai", "tool") for m in messages):
+ if any(
+ (getattr(m, "role", None) or getattr(m, "type", ""))
+ in ("user", "assistant", "human", "ai", "tool")
+ for m in messages
+ ):
return
from temporalio.exceptions import ApplicationError
@@ -79,6 +89,269 @@ def _ensure_llm_contents(messages: List[Any]) -> None:
)
+def _configured_llm_engine() -> str:
+ """Return the engine that new prepare activity results will record."""
+
+ engine = os.getenv("AGENT_LLM_ENGINE", _NATIVE_LLM_ENGINE).strip().lower()
+ if engine not in _LLM_ENGINES:
+ raise ValueError(
+ "AGENT_LLM_ENGINE must be either 'native' or 'langchain'; "
+ f"received {engine!r}"
+ )
+ return engine
+
+
+async def _resolve_activity_api_key(payload: Dict[str, Any]) -> str:
+ """Resolve a provider credential inside an activity, never workflow state.
+
+ Pre-cutover histories already contain ``api_key`` and continue to use it.
+ New histories carry only provider/node identifiers so credentials are not
+ written into Temporal history. The database fallback preserves nodes that
+ keep a key in their own configuration instead of the credential service.
+ """
+
+ recorded = payload.get("api_key")
+ if isinstance(recorded, str) and recorded:
+ return recorded
+
+ from core.container import container
+
+ provider = str(payload.get("provider") or "")
+ auth = container.auth_service()
+ api_key = await auth.get_api_key(provider)
+ if not api_key:
+ api_key = await auth.get_api_key(f"{provider}_api_key")
+
+ if not api_key and payload.get("node_id"):
+ database = container.database()
+ parameters = (
+ await database.get_node_parameters(str(payload["node_id"])) or {}
+ )
+ options = parameters.get("options") or {}
+ candidate = parameters.get("api_key") or options.get("api_key")
+ if isinstance(candidate, str) and candidate:
+ api_key = candidate
+
+ if not api_key:
+ from temporalio.exceptions import ApplicationError
+
+ raise ApplicationError(
+ f"API key for provider {provider!r} is not configured",
+ type="MissingAgentProviderCredential",
+ non_retryable=True,
+ )
+ return str(api_key)
+
+
+def _as_temporal_llm_error(error: Any):
+ """Translate a structured SDK failure at the Temporal boundary.
+
+ Raw provider messages can contain payload fragments or internal endpoint
+ details. The user-facing message is category-based, while the complete
+ structured diagnostics remain JSON-safe in ``ApplicationError.details``
+ and therefore in Temporal history for operators.
+ """
+
+ from temporalio.exceptions import ApplicationError
+
+ category_value = getattr(getattr(error, "category", None), "value", None)
+ category = str(category_value or "unknown")
+ provider = str(getattr(error, "provider", None) or "LLM provider")
+ safe_message = str(
+ getattr(error, "user_message", None)
+ or "The language model provider request failed."
+ )
+ details = {
+ "provider": provider,
+ "category": category,
+ "retryable": bool(getattr(error, "retryable", False)),
+ "status_code": getattr(error, "status_code", None),
+ "provider_code": getattr(error, "provider_code", None),
+ "request_id": getattr(error, "request_id", None),
+ "retry_after": getattr(error, "retry_after", None),
+ "retry_after_raw": getattr(error, "retry_after_raw", None),
+ }
+ return ApplicationError(
+ safe_message,
+ details,
+ type=f"LLMError.{category}",
+ non_retryable=not details["retryable"],
+ )
+
+
+def _native_tool_definition(tool: Any) -> Dict[str, Any]:
+ """Serialize an AgentToolSpec (or legacy StructuredTool) for history."""
+
+ definition = getattr(tool, "definition", None)
+ if definition is not None:
+ return {
+ "name": str(definition.name),
+ "description": str(definition.description or ""),
+ "parameters": dict(definition.parameters or {}),
+ }
+
+ schema = getattr(tool, "args_schema", None)
+ if isinstance(schema, dict):
+ parameters = dict(schema)
+ elif schema is not None and hasattr(schema, "model_json_schema"):
+ parameters = schema.model_json_schema()
+ elif schema is not None and hasattr(schema, "schema"):
+ parameters = schema.schema()
+ else:
+ parameters = {"type": "object", "properties": {}}
+
+ from services.plugin.tool import inline_schema_refs
+
+ return {
+ "name": str(getattr(tool, "name", "")),
+ "description": str(getattr(tool, "description", "") or ""),
+ "parameters": inline_schema_refs(parameters),
+ }
+
+
+async def _await_with_llm_heartbeats(
+ awaitable: Any,
+ *,
+ detail: str,
+ interval_seconds: float = 20.0,
+) -> Any:
+ """Await one buffered SDK request while heartbeating the activity."""
+
+ task = asyncio.ensure_future(awaitable)
+ try:
+ while True:
+ try:
+ return await asyncio.wait_for(
+ asyncio.shield(task),
+ timeout=interval_seconds,
+ )
+ except TimeoutError:
+ activity.heartbeat(detail)
+ finally:
+ if not task.done():
+ task.cancel()
+ await asyncio.gather(task, return_exceptions=True)
+
+
+async def _execute_native_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
+ """Run one native-SDK model turn and retain the v1 activity envelope."""
+
+ from core.container import container
+ from services.agent_runtime import run_native_llm_step
+ from services.llm.messages import filter_empty_messages
+ from services.llm.protocol import (
+ LLMError,
+ MESSAGE_WIRE_VERSION,
+ NATIVE_MESSAGE_WIRE_VERSIONS,
+ Message,
+ ThinkingConfig,
+ ToolDef,
+ message_to_wire,
+ messages_from_wire,
+ )
+
+ wire_version = int(
+ payload.get("message_wire_version") or MESSAGE_WIRE_VERSION
+ )
+ if wire_version not in NATIVE_MESSAGE_WIRE_VERSIONS:
+ from temporalio.exceptions import ApplicationError
+
+ raise ApplicationError(
+ f"Unsupported native message wire version {wire_version}",
+ type="InvalidAgentMessageWireVersion",
+ non_retryable=True,
+ )
+
+ messages = filter_empty_messages(
+ messages_from_wire(payload.get("messages") or [])
+ )
+ _ensure_llm_contents(messages)
+
+ tool_defs: List[ToolDef] = []
+ for value in payload.get("tools") or []:
+ definition = value.get("definition", value)
+ tool_defs.append(
+ ToolDef(
+ name=str(definition.get("name") or ""),
+ description=str(definition.get("description") or ""),
+ parameters=dict(
+ definition.get("parameters")
+ or {"type": "object", "properties": {}}
+ ),
+ )
+ )
+
+ thinking_data = payload.get("thinking_config")
+ thinking = (
+ ThinkingConfig(**thinking_data)
+ if isinstance(thinking_data, dict)
+ else None
+ )
+ api_key = await _resolve_activity_api_key(payload)
+ try:
+ response = await _await_with_llm_heartbeats(
+ run_native_llm_step(
+ container.chat_unifier(),
+ provider=payload["provider"],
+ api_key=api_key,
+ messages=messages,
+ model=payload["model"],
+ temperature=payload.get("temperature", 0.7),
+ max_tokens=payload.get("max_tokens", 4096),
+ thinking=thinking,
+ tools=tool_defs,
+ # Temporal owns the one-shot retry contract. SDK retries could
+ # otherwise re-bill a request after an ambiguous transport loss.
+ sdk_max_retries=0,
+ explicit_max_retries=0,
+ # Keep structured metadata until this activity boundary,
+ # where it is converted to a safe Temporal failure.
+ translate_errors=False,
+ ),
+ detail=f"LLM step waiting: {payload.get('model')}",
+ )
+ except LLMError as error:
+ raise _as_temporal_llm_error(error) from error
+
+ assistant = response.assistant_message or Message(
+ role="assistant",
+ content=response.content or "",
+ tool_calls=list(response.tool_calls or []),
+ )
+ assistant_wire = message_to_wire(assistant)
+ usage = asdict(response.usage)
+
+ if response.tool_calls:
+ calls = []
+ for tool_call in response.tool_calls:
+ call = {
+ "id": tool_call.id,
+ "name": tool_call.name,
+ "args": tool_call.args,
+ }
+ raw_arguments = getattr(tool_call, "raw_arguments", None)
+ parse_error = getattr(tool_call, "parse_error", None)
+ if raw_arguments is not None:
+ call["raw_arguments"] = raw_arguments
+ if parse_error:
+ call["parse_error"] = parse_error
+ calls.append(call)
+ return {
+ "kind": "tool_calls",
+ "assistant_message": assistant_wire,
+ "calls": calls,
+ "usage": usage,
+ }
+
+ return {
+ "kind": "final",
+ "assistant_message": assistant_wire,
+ "content": response.content or "",
+ "thinking": response.thinking,
+ "usage": usage,
+ }
+
+
@activity.defn(name="agent.execute_llm_step.v1")
async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
"""Run one LLM turn with bound tools and return the structured response.
@@ -88,7 +361,7 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
{
"provider": "anthropic" | "openai" | ...,
"model": "claude-sonnet-4-6",
- "api_key": "...", # already resolved
+ "node_id": "...", # credential lookup only
"messages": [{"role": "user", "content": "..."}, ...],
"tools": [ # tool schemas (already built)
{"name": "calculator", "description": "...", "args_schema": {...}}
@@ -115,6 +388,28 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
)
activity.heartbeat(f"LLM step starting: {payload.get('model')}")
+ # The engine marker is recorded in ``agent.prepare_payload.v1`` output.
+ # Its absence is the migration sentinel for histories started before the
+ # native cutover and MUST continue through the untouched LangChain path.
+ engine_value = payload.get("llm_engine")
+ engine = (
+ _LEGACY_LLM_ENGINE
+ if engine_value is None
+ else str(engine_value).strip().lower()
+ )
+ if engine not in _LLM_ENGINES:
+ from temporalio.exceptions import ApplicationError
+
+ raise ApplicationError(
+ f"Unsupported agent LLM engine {engine!r}",
+ type="InvalidAgentLLMEngine",
+ non_retryable=True,
+ )
+ if engine == _NATIVE_LLM_ENGINE:
+ result = await _execute_native_llm_step(payload)
+ activity.heartbeat("LLM step: model returned")
+ return result
+
# Lazy import — keep top-level light so the worker can register this
# activity without pulling LangChain in for every plugin.
from core.container import container
@@ -122,7 +417,7 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
provider = payload["provider"]
model = payload["model"]
- api_key = payload["api_key"]
+ api_key = await _resolve_activity_api_key(payload)
messages = payload.get("messages", [])
tool_data = payload.get("tool_data", [])
temperature = payload.get("temperature", 0.7)
@@ -149,7 +444,13 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
for tool_info in tool_data:
tool, _config = await ai_service._build_tool_from_node(tool_info)
if tool is not None:
- bound_tools.append(tool)
+ # Stage-1 compatibility: the canonical builder now returns the
+ # provider-neutral AgentToolSpec. Old histories still need the
+ # exact LangChain binding surface until they drain.
+ to_langchain = getattr(tool, "to_langchain", None)
+ bound_tools.append(
+ to_langchain() if callable(to_langchain) else tool
+ )
if bound_tools:
chat_model = chat_model.bind_tools(bound_tools)
@@ -162,7 +463,7 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
# blows up Gemini's ``Function call is missing a thought_signature``
# on the next turn.
from langchain_core.messages import messages_from_dict
- from services.ai import filter_empty_messages
+ from services.llm.messages import filter_empty_messages
rehydrated = messages_from_dict(messages)
# Same filter ``_run_agent_loop`` runs in services/ai.py — empty-content
@@ -171,7 +472,10 @@ async def execute_llm_step(payload: Dict[str, Any]) -> Dict[str, Any]:
_ensure_llm_contents(rehydrated)
activity.heartbeat("LLM step: invoking model")
- response = await chat_model.ainvoke(rehydrated)
+ response = await _await_with_llm_heartbeats(
+ chat_model.ainvoke(rehydrated),
+ detail=f"LLM step waiting: {model}",
+ )
activity.heartbeat("LLM step: model returned")
# Extract usage if present (anthropic-style token counts).
@@ -311,13 +615,14 @@ async def compact_agent_memory(payload: Dict[str, Any]) -> Dict[str, Any]:
"node_id": str,
"memory_content": str,
"provider": str,
- "api_key": str,
+ "node_id": str,
"model": str,
}
- Returns ``{"success": bool, "summary": str, "tokens_before": int, "tokens_after": int}``.
- Caller (workflow) replaces its ``messages`` with a single system
- message wrapping the summary, resets token counters, and continues.
+ Returns ``{"success": bool, "summary": str, "tokens_before": int,
+ "tokens_after": int, "usage": dict}``. Caller replaces its ``messages``
+ with a single summary, resets only the context-threshold counter, and
+ retains both pre-compaction and summarizer usage for final billing.
"""
from services.compaction import get_compaction_service
@@ -335,14 +640,18 @@ async def compact_agent_memory(payload: Dict[str, Any]) -> Dict[str, Any]:
"summary": "",
"tokens_before": 0,
"tokens_after": 0,
+ "usage": {},
}
return await svc.compact_context(
session_id=payload["session_id"],
node_id=payload["node_id"],
memory_content=payload.get("memory_content", ""),
provider=payload["provider"],
- api_key=payload["api_key"],
+ api_key=await _resolve_activity_api_key(payload),
model=payload["model"],
+ # Temporal owns activity retries; never repeat an ambiguous provider
+ # request inside the activity or SDK.
+ explicit_max_retries=0,
)
@@ -557,15 +866,19 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
# worker can register this activity without dragging the whole
# AI service in for every plugin.
from core.container import container
- from services.ai import _resolve_max_tokens, _resolve_temperature
- from services.ai import ThinkingConfig, get_default_model_async, is_model_valid_for_provider
+ from services.llm.config import (
+ get_default_model_async,
+ is_model_valid_for_provider,
+ resolve_max_tokens,
+ resolve_temperature,
+ )
+ from services.llm.protocol import ThinkingConfig
from services.plugin.edge_walker import (
collect_agent_connections,
collect_teammate_connections,
extract_task_event_payload,
format_task_context,
)
-
node_id = context["node_id"]
node_type = context["node_type"]
workflow_id = context.get("workflow_id")
@@ -639,7 +952,7 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
f"API key for provider {provider!r} required for AgentWorkflow " f"node {node_id!r}; configure it in the Credentials Modal."
)
- max_tokens = _resolve_max_tokens(flattened, model, provider)
+ max_tokens = resolve_max_tokens(flattened, model, provider)
thinking_config_obj: Any = None
thinking_config_dict: Any = None
@@ -648,7 +961,9 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
enabled=True,
budget=int(flattened.get("thinking_budget", 2048)),
effort=flattened.get("reasoning_effort", "medium"),
- level=flattened.get("thinking_level", "medium"),
+ # Do not invent a level for Gemini 2.5/Vertex; those models reject
+ # receiving thinking_level alongside a token budget.
+ level=flattened.get("thinking_level"),
format=flattened.get("reasoning_format", "parsed"),
)
thinking_config_dict = {
@@ -659,7 +974,7 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
"format": thinking_config_obj.format,
}
- temperature = _resolve_temperature(
+ temperature = resolve_temperature(
flattened,
model,
provider,
@@ -825,6 +1140,11 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
tools_payload.append(
{
"name": tool.name,
+ # Provider-neutral function declaration for the native
+ # activity branch. It is JSON-only so it can live safely in
+ # Temporal history and never requires rebuilding a
+ # StructuredTool inside the native LLM activity.
+ "definition": _native_tool_definition(tool),
"node_type": tool_info.get("node_type", ""),
"version": version,
"task_queue": task_queue,
@@ -950,6 +1270,13 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
except Exception: # noqa: BLE001 — last-resort fallback
effective_recursion_limit = 200
+ llm_engine = _configured_llm_engine()
+ message_wire_version = 1
+ if llm_engine == _NATIVE_LLM_ENGINE:
+ from services.llm.protocol import MESSAGE_WIRE_VERSION
+
+ message_wire_version = MESSAGE_WIRE_VERSION
+
return {
"node_id": node_id,
"node_type": node_type,
@@ -957,7 +1284,6 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
"session_id": session_id,
"provider": provider,
"model": model,
- "api_key": api_key,
"max_tokens": max_tokens,
"temperature": temperature,
"system_message": system_message,
@@ -980,6 +1306,11 @@ async def prepare_agent_payload(context: Dict[str, Any]) -> Dict[str, Any]:
or context.get("execution_id")
or ""
),
+ # Sticky execution-engine selection: this activity result is recorded
+ # in Temporal history, so environment changes cannot flip a run
+ # halfway through its agent loop.
+ "llm_engine": llm_engine,
+ "message_wire_version": message_wire_version,
}
@@ -1060,6 +1391,7 @@ async def refresh_agent_tools(payload: Dict[str, Any]) -> Dict[str, Any]:
new_tools_payload.append(
{
"name": tool.name,
+ "definition": _native_tool_definition(tool),
"node_type": node_type,
"version": version,
"task_queue": task_queue,
diff --git a/server/services/temporal/agent_workflow.py b/server/services/temporal/agent_workflow.py
index f9835c9a..7a73bb20 100644
--- a/server/services/temporal/agent_workflow.py
+++ b/server/services/temporal/agent_workflow.py
@@ -73,6 +73,10 @@ COMPACT_MEMORY_TIMEOUT = timedelta(minutes=5)
# ``cls.start_to_close_timeout`` (F4.A); these are floor values.
TOOL_STEP_TIMEOUT = timedelta(minutes=10)
TOOL_HEARTBEAT_TIMEOUT = timedelta(minutes=2)
+LLM_STEP_HEARTBEAT_TIMEOUT = timedelta(minutes=1)
+
+NATIVE_LLM_ENGINE = "native"
+LEGACY_LLM_ENGINE = "langchain"
# Bounded loop count to defend against a runaway LLM. Plugin classes
# override via ``payload["max_iterations"]`` (set by
@@ -110,6 +114,83 @@ TASK_MANAGER_DELEGATION_PATCH = "agent-task-manager-delegation-v1"
DUPLICATE_TOOL_NAME_ERROR_TYPE = "DuplicateToolNameError"
+def _native_message(
+ *,
+ role: str,
+ content: str,
+ tool_call_id: Optional[str] = None,
+ name: Optional[str] = None,
+) -> Dict[str, Any]:
+ """Build the canonical v2 wire representation without SDK objects."""
+
+ from services.llm.protocol import Message, message_to_wire
+
+ return message_to_wire(
+ Message(
+ role=role,
+ content=content,
+ tool_call_id=tool_call_id,
+ name=name,
+ )
+ )
+
+
+def _append_tool_result_message(
+ messages: List[Dict[str, Any]],
+ *,
+ llm_engine: str,
+ content: str,
+ tool_call_id: str,
+ name: str,
+) -> None:
+ """Append a tool result in the history-pinned engine's wire format."""
+
+ if llm_engine == NATIVE_LLM_ENGINE:
+ messages.append(
+ _native_message(
+ role="tool",
+ content=content,
+ tool_call_id=tool_call_id,
+ name=name,
+ )
+ )
+ return
+
+ # Do not change this legacy shape: histories created before the cutover
+ # pass it through LangChain's messages_from_dict on replay.
+ messages.append(
+ {
+ "type": "tool",
+ "data": {
+ "content": content,
+ "tool_call_id": tool_call_id,
+ "name": name,
+ },
+ }
+ )
+
+
+def _native_assistant_thinking(message: Any) -> Optional[str]:
+ """Read reasoning text from a MessageWireV2 assistant turn.
+
+ ``agent.execute_llm_step.v1`` keeps its historical tool-call result keys,
+ so native tool turns carry their reasoning only inside the canonical
+ assistant message. Extracting it in workflow state preserves the public
+ accumulated-thinking behavior without changing the activity envelope.
+ """
+
+ if not isinstance(message, dict):
+ return None
+ parts = [
+ str(block.get("text") or "")
+ for block in message.get("blocks") or []
+ if isinstance(block, dict)
+ and block.get("type") == "reasoning"
+ and block.get("text")
+ ]
+ return "\n\n".join(parts) or None
+
+
def _tool_activity_id_v2(tool_node_id: str, iteration: int, call_index: int) -> str:
"""Return a stable, unique id for one tool call in one agent turn."""
return f"tool-{tool_node_id}-{iteration + 1}-{call_index + 1}"
@@ -351,7 +432,9 @@ class AgentWorkflow:
The workflow's FIRST step is to schedule
``agent.prepare_payload.v1`` which returns the fully-resolved
- payload (provider, model, api_key, tools, memory, ...). Doing
+ payload (provider, model, tools, memory, ...). Provider credentials
+ are resolved inside leaf activities and never enter workflow history.
+ Doing
prep INSIDE the workflow (as an activity) keeps the orchestrator
ignorant of agent-specific concerns and means the workflow
owns its setup — Temporal's recommended structure.
@@ -361,7 +444,6 @@ class AgentWorkflow:
{
"provider": str,
"model": str,
- "api_key": str,
"max_tokens": int,
"temperature": float,
"system_message": str,
@@ -406,6 +488,18 @@ class AgentWorkflow:
start_to_close_timeout=PERSIST_TURN_TIMEOUT * 2, # 60s default
retry_policy=AGENT_ACTIVITY_RETRY,
)
+ # The prepare result is recorded in history. Absence of the marker is
+ # the deterministic migration sentinel for pre-cutover executions.
+ has_recorded_engine = "llm_engine" in payload
+ llm_engine = str(
+ payload.get("llm_engine") or LEGACY_LLM_ENGINE
+ ).strip().lower()
+ if llm_engine not in {NATIVE_LLM_ENGINE, LEGACY_LLM_ENGINE}:
+ raise ValueError(f"Unsupported agent LLM engine {llm_engine!r}")
+ message_wire_version = int(
+ payload.get("message_wire_version")
+ or (2 if llm_engine == NATIVE_LLM_ENGINE else 1)
+ )
# Stable per-run execution id, forwarded into every tool-call
# activity so session-keyed nodes (browser) reuse one instance
# across iterations instead of minting a fresh uuid per call
@@ -473,43 +567,56 @@ class AgentWorkflow:
}
# ---- Build initial message list ---------------------------------
- # Workflow state is JSON dicts in LangChain's canonical shape
- # ({"type": "<role>", "data": {"content": "...", ...}}) so the
- # activity's ``messages_from_dict`` round-trips preserve every
- # provider-specific field (Gemini ``thought_signature``,
- # Anthropic cache markers, OpenAI reasoning content).
+ # Native executions use the SDK-neutral v2 wire codec. Legacy
+ # histories keep LangChain's canonical {type, data} shape exactly.
messages: List[Dict[str, Any]] = []
system = payload.get("system_message") or ""
if system:
- messages.append({"type": "system", "data": {"content": system}})
+ if llm_engine == NATIVE_LLM_ENGINE:
+ messages.append(_native_message(role="system", content=system))
+ else:
+ messages.append(
+ {"type": "system", "data": {"content": system}}
+ )
# Pre-loaded memory becomes an additional system note. The actual
# parse / append happens in the persist_turn activity, but the
# current markdown content seeds the conversation here.
memory_markdown = payload.get("memory_content") or ""
if memory_markdown:
- messages.append(
- {
- "type": "system",
- "data": {"content": f"## Prior conversation:\n{memory_markdown}"},
- }
- )
+ memory_content = f"## Prior conversation:\n{memory_markdown}"
+ if llm_engine == NATIVE_LLM_ENGINE:
+ messages.append(
+ _native_message(role="system", content=memory_content)
+ )
+ else:
+ messages.append(
+ {"type": "system", "data": {"content": memory_content}}
+ )
user_prompt = payload.get("user_prompt") or ""
if user_prompt:
- messages.append({"type": "human", "data": {"content": user_prompt}})
+ if llm_engine == NATIVE_LLM_ENGINE:
+ messages.append(_native_message(role="user", content=user_prompt))
+ else:
+ messages.append(
+ {"type": "human", "data": {"content": user_prompt}}
+ )
# Map LLM tool name -> {node_type, version, task_queue, node_id,
# parameters} so the workflow can schedule the right activity
# when the LLM emits a tool_call.
tool_index: Dict[str, Dict[str, Any]] = {t["name"]: t for t in tools}
- token_total = 0
compaction_threshold = payload.get("compaction_threshold")
thinking_accumulated = ""
final_content: Optional[str] = None
+ # Billing/observability is cumulative for the entire execution.
+ # Context usage is reset only after a summary replaces the active
+ # message history and exists solely to decide when to compact again.
usage_total: Dict[str, int] = {}
+ context_usage_total: Dict[str, int] = {}
# Emit "executing" + phase="starting" via the existing
# broadcast_agent_progress activity (CloudEvents
@@ -542,24 +649,67 @@ class AgentWorkflow:
phase="llm_step",
)
- # Strip per-turn fields the activity doesn't need.
- # Pass raw ``tool_info`` dicts through; the activity rebuilds
- # the real StructuredTool via ``ai_service._build_tool_from_node``
- # — the same helper the legacy agent path uses. We never
- # serialise/deserialise the args_schema (that round-trip was
- # the source of the Gemini ``properties.<field> Input should be
- # a valid dictionary`` validation crash).
- llm_payload = {
- "provider": payload["provider"],
- "model": payload["model"],
- "api_key": payload["api_key"],
- "messages": messages,
- "tool_data": [t["tool_info"] for t in tools if not t.get("llm_hidden")],
- "system_message": system,
- "temperature": payload.get("temperature", 0.7),
- "max_tokens": payload.get("max_tokens", 4096),
- "thinking_config": payload.get("thinking_config"),
- }
+ # Strip per-turn fields the activity doesn't need. Native turns
+ # receive only provider-neutral JSON tool definitions. Legacy
+ # histories retain raw tool_info so their activity rebuilds the
+ # exact StructuredTool surface it used before the cutover.
+ visible_tools = [t for t in tools if not t.get("llm_hidden")]
+ if llm_engine == NATIVE_LLM_ENGINE:
+ llm_payload = {
+ "node_id": payload["node_id"],
+ "provider": payload["provider"],
+ "model": payload["model"],
+ "messages": messages,
+ "tools": [
+ t["definition"]
+ for t in visible_tools
+ if t.get("definition")
+ ],
+ "system_message": system,
+ "temperature": payload.get("temperature", 0.7),
+ "max_tokens": payload.get("max_tokens", 4096),
+ "thinking_config": payload.get("thinking_config"),
+ "llm_engine": NATIVE_LLM_ENGINE,
+ "message_wire_version": message_wire_version,
+ }
+ else:
+ if not has_recorded_engine:
+ # Preserve both values and insertion order of the
+ # pre-cutover JSON payload so replay emits the same
+ # encoded activity command bytes for histories that
+ # already scheduled a turn.
+ llm_payload = {
+ "provider": payload["provider"],
+ "model": payload["model"],
+ "api_key": payload["api_key"],
+ "messages": messages,
+ "tool_data": [
+ t["tool_info"] for t in visible_tools
+ ],
+ "system_message": system,
+ "temperature": payload.get("temperature", 0.7),
+ "max_tokens": payload.get("max_tokens", 4096),
+ "thinking_config": payload.get("thinking_config"),
+ }
+ else:
+ # Newly prepared emergency-switch executions still use
+ # the legacy adapter but resolve credentials inside the
+ # activity, just like native runs.
+ llm_payload = {
+ "node_id": payload["node_id"],
+ "provider": payload["provider"],
+ "model": payload["model"],
+ "messages": messages,
+ "tool_data": [
+ t["tool_info"] for t in visible_tools
+ ],
+ "system_message": system,
+ "temperature": payload.get("temperature", 0.7),
+ "max_tokens": payload.get("max_tokens", 4096),
+ "thinking_config": payload.get("thinking_config"),
+ "llm_engine": LEGACY_LLM_ENGINE,
+ "message_wire_version": 1,
+ }
# Wave 17.2: one-shot retry. The LLM call is not idempotent —
# a worker crash mid-call must not silently re-bill the full
@@ -568,17 +718,60 @@ class AgentWorkflow:
# future enhancement can re-ask with context; today we
# surface the error to the canvas and stop the loop.
try:
- step_result = await workflow.execute_activity(
- "agent.execute_llm_step.v1",
- args=[llm_payload],
- activity_id=f"llm-step-{iteration + 1}",
- start_to_close_timeout=LLM_STEP_TIMEOUT,
- retry_policy=LLM_STEP_RETRY,
- )
+ if has_recorded_engine:
+ step_result = await workflow.execute_activity(
+ "agent.execute_llm_step.v1",
+ args=[llm_payload],
+ activity_id=f"llm-step-{iteration + 1}",
+ start_to_close_timeout=LLM_STEP_TIMEOUT,
+ heartbeat_timeout=LLM_STEP_HEARTBEAT_TIMEOUT,
+ retry_policy=LLM_STEP_RETRY,
+ )
+ else:
+ step_result = await workflow.execute_activity(
+ "agent.execute_llm_step.v1",
+ args=[llm_payload],
+ activity_id=f"llm-step-{iteration + 1}",
+ start_to_close_timeout=LLM_STEP_TIMEOUT,
+ retry_policy=LLM_STEP_RETRY,
+ )
except Exception as e:
cause = getattr(e, "cause", None)
- detail = str(cause) if cause is not None else str(e)
- workflow.logger.error(f"AgentWorkflow LLM step failed (iteration {iteration + 1}, " f"no auto-retry): {detail}")
+ raw_detail = str(cause) if cause is not None else str(e)
+ if not has_recorded_engine:
+ # Exact pre-cutover result behavior: replayed histories
+ # exposed the activity cause text in their terminal
+ # workflow result. Changing it would alter completion.
+ detail = raw_detail
+ else:
+ cause_type = str(getattr(cause, "type", "") or "")
+ cause_message = str(
+ getattr(cause, "message", "") or ""
+ ).strip()
+ safe_activity_types = {
+ "MissingAgentProviderCredential",
+ "InvalidAgentMessageWireVersion",
+ "EmptyAgentPrompt",
+ }
+ if cause_type.startswith("LLMError."):
+ detail = (
+ cause_message
+ or "The language model request failed."
+ )
+ elif cause_type in safe_activity_types:
+ detail = (
+ cause_message
+ or "The language model request failed."
+ )
+ else:
+ detail = (
+ "The language model step failed unexpectedly. "
+ "Retry the run or check server logs."
+ )
+ workflow.logger.error(
+ f"AgentWorkflow LLM step failed (iteration {iteration + 1}, "
+ f"no auto-retry): {raw_detail}"
+ )
# A failed model step is terminal for this run. Clear any
# turn-scoped skill badges and publish an error status before
# returning; otherwise the normal agent can remain visually
@@ -617,11 +810,23 @@ class AgentWorkflow:
for k, v in (step_result.get("usage") or {}).items():
if isinstance(v, int):
usage_total[k] = usage_total.get(k, 0) + v
- if step_result.get("thinking"):
+ if has_recorded_engine:
+ context_usage_total[k] = (
+ context_usage_total.get(k, 0) + v
+ )
+ step_thinking = step_result.get("thinking")
+ if not step_thinking and llm_engine == NATIVE_LLM_ENGINE:
+ step_thinking = _native_assistant_thinking(
+ step_result.get("assistant_message")
+ )
+ if step_thinking:
if thinking_accumulated:
- thinking_accumulated += f"\n\n--- Iteration {iteration + 1} ---\n" + step_result["thinking"]
+ thinking_accumulated += (
+ f"\n\n--- Iteration {iteration + 1} ---\n"
+ + step_thinking
+ )
else:
- thinking_accumulated = step_result["thinking"]
+ thinking_accumulated = step_thinking
kind = step_result.get("kind")
@@ -1087,18 +1292,39 @@ class AgentWorkflow:
next_delegation_to_start += 1
for call_index, call in enumerate(calls):
+ if llm_engine == NATIVE_LLM_ENGINE and call.get("parse_error"):
+ # Native adapters retain malformed arguments rather than
+ # throwing during normalization. Return a deterministic
+ # tool result and let the model repair its invocation on
+ # the next turn; never execute partially parsed args.
+ _append_tool_result_message(
+ messages,
+ llm_engine=llm_engine,
+ content=_serialise_tool_result(
+ {
+ "error": "Tool arguments were invalid JSON",
+ "detail": str(call.get("parse_error")),
+ "raw_arguments": str(
+ call.get("raw_arguments") or ""
+ ),
+ }
+ ),
+ tool_call_id=call.get("id", ""),
+ name=call.get("name", ""),
+ )
+ continue
tool_info = tool_index.get(call.get("name", ""))
if tool_info is None:
workflow.logger.warning(f"AgentWorkflow: LLM called unknown tool {call.get('name')!r}; " "returning error to model")
- messages.append(
- {
- "type": "tool",
- "data": {
- "content": (f"Error: tool {call.get('name')!r} is not " "connected to this agent."),
- "tool_call_id": call.get("id", ""),
- "name": call.get("name", ""),
- },
- }
+ _append_tool_result_message(
+ messages,
+ llm_engine=llm_engine,
+ content=(
+ f"Error: tool {call.get('name')!r} is not "
+ "connected to this agent."
+ ),
+ tool_call_id=call.get("id", ""),
+ name=call.get("name", ""),
)
continue
@@ -1137,34 +1363,28 @@ class AgentWorkflow:
if not task_description and not task_context:
# Invalid invocation — reject at the call boundary
# instead of spawning a child that cannot run.
- messages.append(
- {
- "type": "tool",
- "data": {
- "content": (
- '{"error": "delegate_to_* requires a '
- "non-empty 'task' argument describing "
- 'what the agent should do."}'
- ),
- "tool_call_id": call.get("id", ""),
- "name": tool_name,
- },
- }
+ _append_tool_result_message(
+ messages,
+ llm_engine=llm_engine,
+ content=(
+ '{"error": "delegate_to_* requires a '
+ "non-empty 'task' argument describing "
+ 'what the agent should do."}'
+ ),
+ tool_call_id=call.get("id", ""),
+ name=tool_name,
)
continue
if delegation_depth >= max_delegation_depth:
- messages.append(
- {
- "type": "tool",
- "data": {
- "content": (
- '{"error": "Maximum delegation depth '
- f'{max_delegation_depth} exceeded."}}'
- ),
- "tool_call_id": call.get("id", ""),
- "name": tool_name,
- },
- }
+ _append_tool_result_message(
+ messages,
+ llm_engine=llm_engine,
+ content=(
+ '{"error": "Maximum delegation depth '
+ f'{max_delegation_depth} exceeded."}}'
+ ),
+ tool_call_id=call.get("id", ""),
+ name=tool_name,
)
continue
tool_node_data = {
@@ -1530,15 +1750,12 @@ class AgentWorkflow:
},
)
- messages.append(
- {
- "type": "tool",
- "data": {
- "content": tool_content,
- "tool_call_id": call.get("id", ""),
- "name": call.get("name", ""),
- },
- }
+ _append_tool_result_message(
+ messages,
+ llm_engine=llm_engine,
+ content=tool_content,
+ tool_call_id=call.get("id", ""),
+ name=call.get("name", ""),
)
if yielded_own_permit:
@@ -1565,25 +1782,70 @@ class AgentWorkflow:
)
# ---- Compaction check --------------------------------------
- token_total = sum(v for k, v in usage_total.items() if k in ("input_tokens", "output_tokens"))
+ if has_recorded_engine:
+ token_total = int(
+ context_usage_total.get("total_tokens") or 0
+ )
+ if not token_total:
+ token_total = sum(
+ int(context_usage_total.get(key) or 0)
+ for key in (
+ "input_tokens",
+ "cache_creation_tokens",
+ "cache_read_tokens",
+ "output_tokens",
+ )
+ )
+ else:
+ # Exact pre-cutover schedule predicate. Captured histories
+ # ignored total/cache/reasoning fields here.
+ token_total = sum(
+ int(usage_total.get(key) or 0)
+ for key in ("input_tokens", "output_tokens")
+ )
if compaction_threshold and token_total >= compaction_threshold and memory_markdown:
workflow.logger.info(f"AgentWorkflow compaction triggered: {token_total} tokens")
+ if has_recorded_engine:
+ compact_payload = {
+ "session_id": payload.get(
+ "session_id", "default"
+ ),
+ "node_id": payload["node_id"],
+ "memory_content": memory_markdown,
+ "provider": payload["provider"],
+ "model": payload["model"],
+ }
+ else:
+ # Preserve values and insertion order of the historical
+ # activity command, including its recorded credential.
+ compact_payload = {
+ "session_id": payload.get(
+ "session_id", "default"
+ ),
+ "node_id": payload["node_id"],
+ "memory_content": memory_markdown,
+ "provider": payload["provider"],
+ "api_key": payload["api_key"],
+ "model": payload["model"],
+ }
compact_result = await workflow.execute_activity(
"agent.compact_memory.v1",
- args=[
- {
- "session_id": payload.get("session_id", "default"),
- "node_id": payload["node_id"],
- "memory_content": memory_markdown,
- "provider": payload["provider"],
- "api_key": payload["api_key"],
- "model": payload["model"],
- }
- ],
+ args=[compact_payload],
activity_id=f"compact-memory-{iteration + 1}",
start_to_close_timeout=COMPACT_MEMORY_TIMEOUT,
retry_policy=AGENT_ACTIVITY_RETRY,
)
+ # The summarizer is another billed model call. Include it in
+ # execution-wide usage whether persistence after the call
+ # succeeds or fails, but never in the active-context counter.
+ if has_recorded_engine:
+ for key, value in (
+ compact_result.get("usage") or {}
+ ).items():
+ if isinstance(value, int):
+ usage_total[key] = (
+ usage_total.get(key, 0) + value
+ )
# Compaction is best-effort. When the service errors or
# was not initialized (worker bootstrap race), keep the
# existing messages and let the loop continue — masking
@@ -1600,13 +1862,43 @@ class AgentWorkflow:
# Replace the running messages with the summary
# plus the last user prompt — same pattern
# ``CompactionService`` uses today in services/ai.py.
- messages = [
- {"type": "system", "data": {"content": system}},
- {"type": "system", "data": {"content": f"## Compacted summary:\n{summary}"}},
- {"type": "human", "data": {"content": user_prompt}},
- ]
+ compacted_content = f"## Compacted summary:\n{summary}"
+ if llm_engine == NATIVE_LLM_ENGINE:
+ messages = [
+ _native_message(
+ role="system",
+ content=system,
+ ),
+ _native_message(
+ role="system",
+ content=compacted_content,
+ ),
+ _native_message(
+ role="user",
+ content=user_prompt,
+ ),
+ ]
+ else:
+ messages = [
+ {
+ "type": "system",
+ "data": {"content": system},
+ },
+ {
+ "type": "system",
+ "data": {"content": compacted_content},
+ },
+ {
+ "type": "human",
+ "data": {"content": user_prompt},
+ },
+ ]
memory_markdown = summary
- usage_total = {}
+ if has_recorded_engine:
+ context_usage_total = {}
+ else:
+ # Exact pre-cutover completion shape.
+ usage_total = {}
else:
# Loop exited without break -- hit max_iterations.
diff --git a/server/services/temporal/worker.py b/server/services/temporal/worker.py
index 02819e3f..a6f88746 100644
--- a/server/services/temporal/worker.py
+++ b/server/services/temporal/worker.py
@@ -644,6 +644,11 @@ async def run_standalone_worker(
# Cleanup session on shutdown
if not session.closed:
await session.close()
+ # Activities resolve the singleton unifier through the global
+ # container. Close its pooled SDK clients in standalone workers too.
+ from core.container import container
+
+ await container.chat_unifier().aclose()
async def create_worker(
diff --git a/server/skills/coding_agent/file-modify-skill/SKILL.md b/server/skills/coding_agent/file-modify-skill/SKILL.md
index cffc07ab..baef80ce 100644
--- a/server/skills/coding_agent/file-modify-skill/SKILL.md
+++ b/server/skills/coding_agent/file-modify-skill/SKILL.md
@@ -11,7 +11,7 @@ metadata:
# File Modify Tool
-Write new files or edit existing files with exact string replacement. Uses deepagents filesystem backend.
+Write new files or edit existing files with exact string replacement. Uses the workspace-contained native filesystem backend.
**Path sandbox:** all paths resolve inside the per-workflow workspace root. Use workspace-relative paths (e.g. `reports/summary.md`); `..` and `~` segments are rejected, and absolute paths are remapped into the workspace.
diff --git a/server/skills/coding_agent/file-read-skill/SKILL.md b/server/skills/coding_agent/file-read-skill/SKILL.md
index f5fc5c09..8baed503 100644
--- a/server/skills/coding_agent/file-read-skill/SKILL.md
+++ b/server/skills/coding_agent/file-read-skill/SKILL.md
@@ -11,7 +11,7 @@ metadata:
# File Read Tool
-Read file contents with line-numbered output and pagination. Uses deepagents filesystem backend.
+Read file contents with line-numbered output and pagination. Uses the workspace-contained native filesystem backend.
**Path sandbox:** all paths resolve inside the per-workflow workspace root. Use workspace-relative paths (e.g. `reports/data.csv`); `..` and `~` segments are rejected, and absolute paths are remapped into the workspace. Use `fs_search` with `mode: "ls"` to discover what exists.
diff --git a/server/skills/coding_agent/fs-search-skill/SKILL.md b/server/skills/coding_agent/fs-search-skill/SKILL.md
index ede55b4f..f5fb3e1f 100644
--- a/server/skills/coding_agent/fs-search-skill/SKILL.md
+++ b/server/skills/coding_agent/fs-search-skill/SKILL.md
@@ -11,7 +11,7 @@ metadata:
# FS Search Tool
-Search the filesystem: list directories, glob pattern match files, or grep file contents. Uses deepagents filesystem backend.
+Search the filesystem: list directories, glob pattern match files, or grep file contents. Uses the workspace-contained native filesystem backend.
**Path sandbox:** all paths resolve inside the per-workflow workspace root. Use workspace-relative paths; `..` and `~` segments are rejected, and absolute paths are remapped into the workspace.
diff --git a/server/skills/terminal/shell-skill/SKILL.md b/server/skills/terminal/shell-skill/SKILL.md
index f5f4a12d..c3b1961f 100644
--- a/server/skills/terminal/shell-skill/SKILL.md
+++ b/server/skills/terminal/shell-skill/SKILL.md
@@ -124,7 +124,7 @@ Use **A** for "run these in order regardless of outcome", **B/C** when you must
## Workspace and paths
- The cwd is the per-workflow workspace; relative paths resolve there.
-- Filesystem operations elsewhere on this tool (read/write/edit via `file_*`) honour `virtual_mode=True` and reject `..`/`~` traversal. Shell `execute()` itself is **not** path-restricted (deepagents documents this), so prefer `file_read` / `file_modify` / `fs_search` for actual filesystem work.
+- Filesystem operations elsewhere on this tool (read/write/edit via `file_*`) are workspace-contained and reject `..`/`~` traversal. Shell `execute()` itself retains historical host-shell behavior and is **not** path-restricted, so prefer `file_read` / `file_modify` / `fs_search` for actual filesystem work.
## Use the right tool
diff --git a/server/tests/llm/test_legacy_langchain_boundary.py b/server/tests/llm/test_legacy_langchain_boundary.py
new file mode 100644
index 00000000..33e1d277
--- /dev/null
+++ b/server/tests/llm/test_legacy_langchain_boundary.py
@@ -0,0 +1,215 @@
+"""Keep LangChain isolated to the pre-cutover Temporal compatibility path."""
+
+from __future__ import annotations
+
+import ast
+import os
+from pathlib import Path
+import subprocess
+import sys
+from typing import Iterable, NamedTuple
+
+
+SERVER_ROOT = Path(__file__).resolve().parents[2]
+LEGACY_IMPORT_SCOPES = {
+ Path("services/ai.py"): {
+ "TYPE_CHECKING",
+ "_get_chat_anthropic",
+ "_get_chat_groq",
+ "_get_chat_cerebras",
+ "_get_google_genai_class",
+ "__getattr__",
+ "_build_provider_class_map",
+ "_build_provider_configs",
+ "_run_agent_loop",
+ "AIService.create_model",
+ },
+ Path("services/agent_runtime.py"): {
+ "AgentToolSpec.to_langchain",
+ },
+ Path("services/temporal/agent_activities.py"): {
+ "execute_llm_step",
+ },
+}
+
+
+class _ImportUse(NamedTuple):
+ module: str
+ scope: str
+ line: int
+
+
+class _ImportVisitor(ast.NodeVisitor):
+ def __init__(self) -> None:
+ self.uses: list[_ImportUse] = []
+ self._scope: list[str] = []
+
+ @property
+ def scope(self) -> str:
+ return ".".join(self._scope) if self._scope else "<module>"
+
+ def visit_ClassDef(self, node: ast.ClassDef) -> None:
+ self._scope.append(node.name)
+ self.generic_visit(node)
+ self._scope.pop()
+
+ def visit_FunctionDef(self, node: ast.FunctionDef) -> None:
+ self._scope.append(node.name)
+ self.generic_visit(node)
+ self._scope.pop()
+
+ visit_AsyncFunctionDef = visit_FunctionDef
+
+ def visit_If(self, node: ast.If) -> None:
+ is_type_checking = (
+ isinstance(node.test, ast.Name)
+ and node.test.id == "TYPE_CHECKING"
+ )
+ if is_type_checking:
+ self._scope.append("TYPE_CHECKING")
+ for statement in node.body:
+ self.visit(statement)
+ if is_type_checking:
+ self._scope.pop()
+ for statement in node.orelse:
+ self.visit(statement)
+
+ def visit_Import(self, node: ast.Import) -> None:
+ self.uses.extend(
+ _ImportUse(alias.name, self.scope, node.lineno)
+ for alias in node.names
+ )
+
+ def visit_ImportFrom(self, node: ast.ImportFrom) -> None:
+ if node.module:
+ self.uses.append(
+ _ImportUse(node.module, self.scope, node.lineno)
+ )
+
+ def visit_Call(self, node: ast.Call) -> None:
+ module: str | None = None
+ if (
+ isinstance(node.func, ast.Name)
+ and node.func.id == "__import__"
+ ):
+ module = _literal_first_argument(node)
+ elif (
+ isinstance(node.func, ast.Attribute)
+ and node.func.attr in {"import_module", "resolve_name"}
+ ):
+ module = _literal_first_argument(node)
+ if module:
+ # pkgutil.resolve_name accepts ``package:object`` references.
+ self.uses.append(
+ _ImportUse(module.split(":", 1)[0], self.scope, node.lineno)
+ )
+ self.generic_visit(node)
+
+
+def _literal_first_argument(node: ast.Call) -> str | None:
+ if not node.args:
+ return None
+ value = node.args[0]
+ return value.value if isinstance(value, ast.Constant) and isinstance(
+ value.value, str
+ ) else None
+
+
+def _third_party_imports(path: Path) -> Iterable[_ImportUse]:
+ tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
+ visitor = _ImportVisitor()
+ visitor.visit(tree)
+ return visitor.uses
+
+
+def _is_forbidden(module: str) -> bool:
+ return (
+ module == "deepagents"
+ or module.startswith("deepagents.")
+ or module.startswith(("langgraph", "langsmith"))
+ )
+
+
+def test_langchain_is_confined_to_legacy_temporal_compatibility():
+ violations: list[str] = []
+ for path in SERVER_ROOT.rglob("*.py"):
+ if "tests" in path.parts or ".venv" in path.parts:
+ continue
+ relative = path.relative_to(SERVER_ROOT)
+ allowed_scopes = LEGACY_IMPORT_SCOPES.get(relative, set())
+ for use in _third_party_imports(path):
+ if _is_forbidden(use.module):
+ violations.append(
+ f"{relative}:{use.line} [{use.scope}]: {use.module}"
+ )
+ elif (
+ use.module.startswith("langchain")
+ and use.scope not in allowed_scopes
+ ):
+ violations.append(
+ f"{relative}:{use.line} [{use.scope}]: {use.module}"
+ )
+
+ assert not violations, (
+ "LangChain may remain only in the explicitly temporary pre-cutover "
+ "Temporal compatibility surface:\n" + "\n".join(sorted(violations))
+ )
+
+
+def test_native_agent_entrypoints_do_not_use_langchain_shaped_calls():
+ tree = ast.parse(
+ (SERVER_ROOT / "services" / "ai.py").read_text(encoding="utf-8")
+ )
+ violations: list[str] = []
+ for node in ast.walk(tree):
+ if not isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)):
+ continue
+ if node.name not in {"execute_agent", "execute_chat_agent"}:
+ continue
+ for descendant in ast.walk(node):
+ if (
+ isinstance(descendant, ast.Attribute)
+ and descendant.attr
+ in {"ainvoke", "bind_tools", "get_input_schema"}
+ ):
+ violations.append(
+ f"{node.name}:{descendant.lineno}: {descendant.attr}"
+ )
+ assert not violations, (
+ "Native agent entrypoints used a LangChain-shaped API:\n"
+ + "\n".join(violations)
+ )
+
+
+def test_container_import_does_not_load_legacy_dependency_tree():
+ code = """
+import builtins
+import sys
+
+blocked = ("langchain", "langgraph", "langsmith", "deepagents")
+real_import = builtins.__import__
+
+def guarded_import(name, *args, **kwargs):
+ if name.startswith(blocked):
+ raise AssertionError(f"legacy dependency imported at boot: {name}")
+ return real_import(name, *args, **kwargs)
+
+builtins.__import__ = guarded_import
+import core.container # noqa: F401
+loaded = sorted(
+ name for name in sys.modules if name.startswith(blocked)
+)
+assert not loaded, loaded
+"""
+ env = dict(os.environ)
+ env["DEBUG"] = "false"
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ cwd=SERVER_ROOT,
+ env=env,
+ capture_output=True,
+ text=True,
+ timeout=60,
+ check=False,
+ )
+ assert result.returncode == 0, result.stdout + result.stderr
diff --git a/server/tests/llm/test_live_providers.py b/server/tests/llm/test_live_providers.py
index 72d2dc29..b2ec3a62 100644
--- a/server/tests/llm/test_live_providers.py
+++ b/server/tests/llm/test_live_providers.py
@@ -21,14 +21,18 @@ provider's tests to skip — they never fail. Recognised env vars:
MISTRAL_API_KEY → mistral
OLLAMA_BASE_URL → ollama (override default localhost:11434)
LMSTUDIO_BASE_URL → lmstudio (override default localhost:1234)
+ GROQ_API_KEY → groq
+ CEREBRAS_API_KEY → cerebras
-**Scope**: each enabled provider gets two assertions —
+**Scope**: each enabled provider gets three assertions —
1. ``ChatUnifier.fetch_models`` returns a non-empty list.
2. ``ChatUnifier.chat`` returns non-empty content for a 1-token "hi" call.
+ 3. A two-turn tool call replays the exact assistant message and accepts a
+ native tool-result message.
The tests use the smallest available model + ``max_tokens=16`` to keep
-spend negligible. Total cost per full run with all 4 cloud keys
-present: well under $0.01 at 2026 prices.
+spend negligible. Total cost per full run with every cloud credential
+present remains negligible at 2026 prices.
**Why this file is opt-in, not always-on**: the unit/mock tests in
``test_unifier_typed_errors.py`` + ``test_unifier_incompatible_models_filter.py``
@@ -40,6 +44,7 @@ them after dependency bumps or before a release, not on every commit.
from __future__ import annotations
+import json
import os
from pathlib import Path
from typing import Optional
@@ -49,8 +54,8 @@ from unittest.mock import AsyncMock, MagicMock
import services.llm # noqa: F401 — populate the provider registry
from services.llm.config import LLM_DEFAULTS
-from services.llm.protocol import Message
-from services.llm.registry import get_provider, has_provider
+from services.llm.protocol import Message, ToolDef
+from services.llm.registry import has_provider
from services.llm.unifier import ChatUnifier
@@ -114,6 +119,8 @@ _PROVIDER_KEYS = {
"deepseek": ("DEEPSEEK_API_KEY",),
"kimi": ("KIMI_API_KEY", "MOONSHOT_API_KEY"),
"mistral": ("MISTRAL_API_KEY",),
+ "groq": ("GROQ_API_KEY",),
+ "cerebras": ("CEREBRAS_API_KEY",),
}
# Models chosen for low cost + broad availability. Falls back to the
@@ -128,6 +135,8 @@ _LIVE_MODELS = {
"deepseek": "deepseek-chat",
"kimi": "kimi-k2.6",
"mistral": "mistral-small-latest",
+ "groq": "openai/gpt-oss-120b",
+ "cerebras": "gpt-oss-120b",
"ollama": None, # discovered from server
"lmstudio": None,
}
@@ -222,6 +231,88 @@ async def test_live_chat_round_trip_returns_content(live_unifier, provider):
assert response.content, f"{provider} ({model}) returned empty content: {response!r}"
[email protected]("provider", sorted(_PROVIDER_KEYS.keys()))
+async def test_live_multi_turn_tool_round_trip(live_unifier, provider):
+ """Every credentialed cloud provider preserves a native tool turn."""
+
+ api_key = _api_key(*_PROVIDER_KEYS[provider])
+ if not api_key:
+ pytest.skip(
+ f"no API key set for {provider} "
+ f"({' / '.join(_PROVIDER_KEYS[provider])})"
+ )
+
+ models = await live_unifier.fetch_models(
+ provider=provider,
+ api_key=api_key,
+ )
+ model = _resolve_live_model(provider, models)
+ if model is None:
+ pytest.skip(f"{provider}: live model list empty — nothing to call")
+
+ tool = ToolDef(
+ name="echo_value",
+ description="Return the supplied integer unchanged.",
+ parameters={
+ "type": "object",
+ "properties": {"value": {"type": "integer"}},
+ "required": ["value"],
+ "additionalProperties": False,
+ },
+ )
+ messages = [
+ Message(
+ role="user",
+ content=(
+ "Call echo_value exactly once with value 7. "
+ "Do not answer before using the tool."
+ ),
+ )
+ ]
+ first = await live_unifier.chat(
+ provider=provider,
+ api_key=api_key,
+ messages=messages,
+ model=model,
+ temperature=0.0,
+ max_tokens=96,
+ tools=[tool],
+ )
+ assert first.assistant_message is not None
+ assert first.tool_calls, (
+ f"{provider} ({model}) did not request the required tool: {first!r}"
+ )
+ call = first.tool_calls[0]
+ assert call.name == "echo_value"
+
+ messages.extend(
+ [
+ first.assistant_message,
+ Message(
+ role="tool",
+ content=json.dumps({"value": 7}),
+ tool_call_id=call.id,
+ name=call.name,
+ ),
+ ]
+ )
+ second = await live_unifier.chat(
+ provider=provider,
+ api_key=api_key,
+ messages=messages,
+ model=model,
+ temperature=0.0,
+ max_tokens=96,
+ tools=[tool],
+ )
+ assert second.content, (
+ f"{provider} ({model}) returned no final text after the tool: "
+ f"{second!r}"
+ )
+
+
# ---------------------------------------------------------------------------
# Local-server live tests (ollama / lmstudio)
# ---------------------------------------------------------------------------
@@ -233,8 +324,10 @@ async def test_live_chat_round_trip_returns_content(live_unifier, provider):
("lmstudio", "LMSTUDIO_BASE_URL", "http://localhost:1234/v1"),
])
@pytest.mark.asyncio
-async def test_live_local_server_fetch_models(live_unifier, provider, env_var, default_url):
- """Local servers (ollama / lmstudio) — fetch the user's loaded models."""
+async def test_live_local_server_native_text_and_tool_round_trip(
+ live_unifier, provider, env_var, default_url
+):
+ """Reachable local servers run fetch, text, and native tool replay."""
base_url = _local_base_url(env_var, default_url)
# Probe whether the server is actually reachable; skip cleanly if not.
try:
@@ -249,9 +342,80 @@ async def test_live_local_server_fetch_models(live_unifier, provider, env_var, d
# Local providers don't need a real API key — pass a placeholder.
models = await live_unifier.fetch_models(provider=provider, api_key="placeholder")
assert isinstance(models, list)
- # We do NOT assert non-empty — a running server with zero loaded
- # models is a legitimate state (e.g. user just started Ollama and
- # hasn't pulled anything). The unifier path is exercised either way.
+ if not models:
+ pytest.skip(
+ f"{provider} is reachable but has no loaded model to invoke"
+ )
+
+ model = _resolve_live_model(provider, models)
+ assert model is not None
+ text = await live_unifier.chat(
+ provider=provider,
+ api_key="placeholder",
+ messages=[Message(role="user", content="Say hi in 3 words.")],
+ model=model,
+ temperature=0.0,
+ max_tokens=32,
+ )
+ assert text.content, f"{provider} ({model}) returned empty content"
+
+ tool = ToolDef(
+ name="echo_value",
+ description="Return the supplied integer unchanged.",
+ parameters={
+ "type": "object",
+ "properties": {"value": {"type": "integer"}},
+ "required": ["value"],
+ "additionalProperties": False,
+ },
+ )
+ messages = [
+ Message(
+ role="user",
+ content=(
+ "Call echo_value exactly once with value 7. "
+ "Do not answer before using the tool."
+ ),
+ )
+ ]
+ first = await live_unifier.chat(
+ provider=provider,
+ api_key="placeholder",
+ messages=messages,
+ model=model,
+ temperature=0.0,
+ max_tokens=96,
+ tools=[tool],
+ )
+ if not first.tool_calls:
+ pytest.fail(
+ f"{provider} ({model}) did not request the required tool: "
+ f"{first!r}"
+ )
+ call = first.tool_calls[0]
+ messages.extend(
+ [
+ first.assistant_message,
+ Message(
+ role="tool",
+ content=json.dumps({"value": 7}),
+ tool_call_id=call.id,
+ name=call.name,
+ ),
+ ]
+ )
+ second = await live_unifier.chat(
+ provider=provider,
+ api_key="placeholder",
+ messages=messages,
+ model=model,
+ temperature=0.0,
+ max_tokens=96,
+ tools=[tool],
+ )
+ assert second.content, (
+ f"{provider} ({model}) returned no final text after native tool replay"
+ )
# ---------------------------------------------------------------------------
diff --git a/server/tests/llm/test_max_tokens_resolution.py b/server/tests/llm/test_max_tokens_resolution.py
index 5b368e18..aa4dd835 100644
--- a/server/tests/llm/test_max_tokens_resolution.py
+++ b/server/tests/llm/test_max_tokens_resolution.py
@@ -67,12 +67,15 @@ class TestAgentResolverDelegates:
src = inspect.getsource(ai._resolve_max_tokens)
assert "native_resolve_max_tokens" in src
- def test_temporal_prepare_payload_uses_agent_resolver(self):
- # The F4.B path must resolve max_tokens through the same helper.
+ def test_temporal_prepare_payload_uses_native_resolver(self):
+ # The F4.B path imports the provider-neutral resolver directly so
+ # newly prepared executions do not enter services.ai compatibility
+ # code.
from services.temporal.agent_activities import prepare_agent_payload
src = inspect.getsource(prepare_agent_payload)
- assert "_resolve_max_tokens" in src
+ assert "from services.llm.config import" in src
+ assert "resolve_max_tokens(flattened, model, provider)" in src
class TestRegistryAliasNormalization:
diff --git a/server/tests/llm/test_plugin_shape.py b/server/tests/llm/test_plugin_shape.py
index 26bac851..6cda29d8 100644
--- a/server/tests/llm/test_plugin_shape.py
+++ b/server/tests/llm/test_plugin_shape.py
@@ -35,8 +35,8 @@ def test_every_provider_has_a_module_or_is_registered_by_compat():
Dedicated providers (anthropic / openai / gemini / openrouter) live
in their own file. OpenAI-compat providers (xai / deepseek / kimi /
- mistral / ollama / lmstudio) all register through ``_compat.py``
- using the same ``OpenAIProvider`` factory.
+ mistral / ollama / lmstudio / groq / cerebras) all register through
+ ``_compat.py`` using the same ``OpenAIProvider`` factory.
"""
dedicated_files = {p.stem for p in PROVIDERS_DIR.glob("*.py") if not p.name.startswith("_")}
for provider in all_providers():
@@ -58,12 +58,9 @@ def test_every_provider_has_a_module_or_is_registered_by_compat():
# 2. No per-provider Python inside execute_chat / fetch_models bodies
# ---------------------------------------------------------------------------
#
-# Scoped contract: Phase A only refactored the chat-model node path
-# (``execute_chat`` + ``fetch_models``). The agent paths (``execute_agent``,
-# ``execute_chat_agent``) keep their ``except openai.OpenAIError`` blocks
-# until Phase G migrates the agent loop off LangChain. AST-walking each
-# method body avoids false positives from those still-LangChain code
-# paths. Phase G should extend this test to cover the agent methods.
+# Chat and agent entry points all delegate through the native unifier.
+# AST-walking each method body avoids false positives from the isolated
+# pre-cutover Temporal compatibility factory that remains elsewhere.
def _method_source(class_name: str, method_name: str, source: str) -> str:
@@ -104,10 +101,12 @@ _FORBIDDEN_PATTERNS_IN_UNIFIED_METHODS = (
)
[email protected]("method_name", ["execute_chat", "fetch_models"])
+ "method_name",
+ ["execute_chat", "fetch_models", "execute_agent", "execute_chat_agent"],
+)
def test_unified_methods_have_no_per_provider_python(method_name):
- """``execute_chat`` and ``fetch_models`` must delegate to the unifier
- instead of carrying per-provider branches or typed-exception blocks."""
+ """Every new-run LLM entry point must delegate to the native unifier."""
source = AI_PY.read_text(encoding="utf-8")
method_src = _method_source("AIService", method_name, source)
for pattern in _FORBIDDEN_PATTERNS_IN_UNIFIED_METHODS:
@@ -136,8 +135,8 @@ def test_every_provider_declares_sdk_exception_types():
assert spec.sdk_exception_types, (
f"provider {provider!r} has empty sdk_exception_types — typed "
f"SDK errors would surface as bare RuntimeErrors with tracebacks "
- f"instead of clean NodeUserError envelopes."
- )
+ f"instead of clean NodeUserError envelopes."
+ )
# And every tuple member must be an exception class
for exc in spec.sdk_exception_types:
assert isinstance(exc, type) and issubclass(exc, BaseException), (
@@ -145,6 +144,21 @@ def test_every_provider_declares_sdk_exception_types():
)
+def test_all_registered_providers_are_agent_selectable():
+ """Agent surfaces must not silently omit a native provider."""
+
+ from typing import get_args
+
+ from nodes.agent._specialized import SpecializedAgentParams
+ from nodes.agent.ai_agent import AIAgentParams
+ from nodes.agent.chat_agent import ChatAgentParams
+
+ expected = set(all_providers())
+ for params_type in (AIAgentParams, ChatAgentParams, SpecializedAgentParams):
+ annotation = params_type.model_fields["provider"].annotation
+ assert set(get_args(annotation)) == expected
+
+
# ---------------------------------------------------------------------------
# 4. ai.py imports ChatUnifier somewhere (delegation contract)
# ---------------------------------------------------------------------------
diff --git a/server/tests/llm/test_protocol_codec.py b/server/tests/llm/test_protocol_codec.py
new file mode 100644
index 00000000..8f763f27
--- /dev/null
+++ b/server/tests/llm/test_protocol_codec.py
@@ -0,0 +1,200 @@
+"""Golden tests for the durable native LLM message contract."""
+
+import json
+
+import pytest
+
+from services.llm.protocol import (
+ MESSAGE_WIRE_VERSION,
+ ContentBlock,
+ LLMError,
+ LLMErrorCategory,
+ LLMResponse,
+ Message,
+ ToolCall,
+ Usage,
+ message_from_wire,
+ message_to_wire,
+ messages_from_wire,
+ messages_to_wire,
+)
+
+
+def test_message_wire_v2_round_trips_ordered_blocks_and_provider_state():
+ good_call = ToolCall(id="call-1", name="lookup", args={"q": "weather"})
+ malformed_call = ToolCall.from_raw(
+ id="call-2",
+ name="calculate",
+ arguments='{"expression":',
+ )
+ original = Message(
+ role="assistant",
+ content="I will check.",
+ tool_calls=[good_call, malformed_call],
+ blocks=[
+ ContentBlock(
+ type="reasoning",
+ text="Need fresh data",
+ metadata={"signature": "signed-value"},
+ ),
+ ContentBlock(type="text", text="I will check."),
+ ContentBlock(type="tool_call", tool_call=good_call),
+ ContentBlock(type="tool_call", tool_call=malformed_call),
+ ],
+ provider_state={
+ "provider": "anthropic",
+ "payload": {
+ "content": [
+ {
+ "type": "thinking",
+ "thinking": "Need fresh data",
+ "signature": "signed-value",
+ }
+ ]
+ },
+ },
+ )
+
+ wire = message_to_wire(original)
+ assert wire["version"] == MESSAGE_WIRE_VERSION
+ # The payload must be safe for Temporal's ordinary JSON payload codec.
+ json.dumps(wire)
+
+ decoded = message_from_wire(json.loads(json.dumps(wire)))
+ assert [block.type for block in decoded.blocks] == [
+ "reasoning",
+ "text",
+ "tool_call",
+ "tool_call",
+ ]
+ assert decoded.provider_state == original.provider_state
+ assert decoded.tool_calls[1].args == {}
+ assert decoded.tool_calls[1].raw_arguments == '{"expression":'
+ assert decoded.tool_calls[1].parse_error.startswith(
+ "Invalid JSON tool arguments"
+ )
+
+
+def test_messages_helpers_accept_generators_and_pre_version_flat_shape():
+ encoded = messages_to_wire(
+ Message(role="user", content=str(index)) for index in range(2)
+ )
+ assert [message.content for message in messages_from_wire(encoded)] == [
+ "0",
+ "1",
+ ]
+ legacy = message_from_wire(
+ {
+ "role": "tool",
+ "content": "42",
+ "tool_call_id": "call-1",
+ "name": "calculate",
+ }
+ )
+ assert legacy.role == "tool"
+ assert legacy.blocks[0].type == "tool_result"
+
+
+def test_provider_state_rejects_sdk_objects_and_round_trips_large_payloads():
+ with pytest.raises(TypeError, match="JSON values"):
+ Message(
+ role="assistant",
+ provider_state={"provider": "test", "payload": {"raw": object()}},
+ )
+
+ blob = "x" * (300 * 1024)
+ original = Message(
+ role="assistant",
+ provider_state={
+ "provider": "test",
+ "payload": {"blob": blob},
+ },
+ )
+ wire = message_to_wire(original)
+ decoded = message_from_wire(json.loads(json.dumps(wire)))
+ assert decoded.provider_state["payload"]["blob"] == blob
+
+
+def test_response_always_has_replayable_assistant_message():
+ call = ToolCall(id="call-1", name="lookup", args={"q": "x"})
+ response = LLMResponse(
+ content="Checking",
+ thinking="Use the lookup",
+ tool_calls=[call],
+ )
+ assert response.assistant_message is not None
+ assert response.assistant_message.content == "Checking"
+ assert response.assistant_message.tool_calls == [call]
+ assert [block.type for block in response.assistant_message.blocks] == [
+ "reasoning",
+ "text",
+ "tool_call",
+ ]
+
+
+def test_usage_is_safe_and_additive():
+ first = Usage(input_tokens="10", output_tokens=5, total_tokens=None)
+ second = Usage(
+ input_tokens=-1,
+ output_tokens=2,
+ cache_read_tokens=3,
+ reasoning_tokens=4,
+ )
+ total = first + second
+ assert first.total_tokens == 15
+ assert second.input_tokens == 0
+ assert total.total_tokens == 17
+ assert total.cache_read_tokens == 3
+ assert total.reasoning_tokens == 4
+
+
+def test_structured_error_classifies_retryable_sdk_failures():
+ class RateLimitError(Exception):
+ status_code = 429
+ request_id = "request-123"
+
+ error = LLMError.from_exception("openai", RateLimitError("slow down"))
+ assert error.category == LLMErrorCategory.RATE_LIMIT
+ assert error.retryable is True
+ assert error.status_code == 429
+ assert error.request_id == "request-123"
+
+
+def test_structured_error_exposes_only_category_based_user_message():
+ raw_message = (
+ "POST https://private-gateway.internal/v1 "
+ 'payload={"authorization":"Bearer secret"}'
+ )
+ error = LLMError(
+ message=raw_message,
+ provider="openai",
+ category=LLMErrorCategory.AUTHENTICATION,
+ )
+
+ assert error.message == raw_message
+ assert error.user_message == (
+ "OpenAI authentication failed. Check the configured API key."
+ )
+ assert "private-gateway" not in error.user_message
+ assert "Bearer secret" not in error.user_message
+
+
+def test_structured_error_preserves_http_date_retry_after():
+ class Response:
+ status_code = 429
+ headers = {
+ "retry-after": "Wed, 21 Oct 2037 07:28:00 GMT",
+ "x-request-id": "request-http-date",
+ }
+
+ class RateLimitError(Exception):
+ status_code = 429
+ response = Response()
+
+ error = LLMError.from_exception(
+ "openai", RateLimitError("rate limited")
+ )
+
+ assert error.retry_after is None
+ assert error.retry_after_raw == "Wed, 21 Oct 2037 07:28:00 GMT"
+ assert error.request_id == "request-http-date"
diff --git a/server/tests/llm/test_provider_continuation.py b/server/tests/llm/test_provider_continuation.py
new file mode 100644
index 00000000..b27f7c53
--- /dev/null
+++ b/server/tests/llm/test_provider_continuation.py
@@ -0,0 +1,702 @@
+import json
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, patch
+
+import pytest
+
+from services.llm.protocol import (
+ BINARY_STATE_MARKER,
+ Message,
+ ThinkingConfig,
+ ToolDef,
+ message_from_wire,
+ message_to_wire,
+)
+
+
+def _usage(**overrides):
+ values = {
+ "prompt_tokens": 4,
+ "completion_tokens": 2,
+ "total_tokens": 6,
+ "completion_tokens_details": None,
+ "prompt_tokens_details": None,
+ }
+ values.update(overrides)
+ return SimpleNamespace(**values)
+
+
+def test_openai_preserves_malformed_arguments_and_reasoning_continuation():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ provider = OpenAIProvider("key")
+
+ tool_call = SimpleNamespace(
+ id="call-1",
+ function=SimpleNamespace(
+ name="calculate",
+ arguments='{"expression":',
+ ),
+ )
+ response = SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content="",
+ reasoning_content="Need a calculation",
+ refusal=None,
+ tool_calls=[tool_call],
+ ),
+ finish_reason="tool_calls",
+ )
+ ],
+ usage=_usage(),
+ )
+
+ normalized = provider._normalize(response, "gpt-5")
+ call = normalized.tool_calls[0]
+ assert call.args == {}
+ assert call.raw_arguments == '{"expression":'
+ assert call.parse_error
+ assert normalized.assistant_message.provider_state == {
+ "provider": "openai",
+ "payload": {"reasoning_content": "Need a calculation"},
+ }
+ api_message = provider._to_api_message(normalized.assistant_message)
+ assert api_message["reasoning_content"] == "Need a calculation"
+ assert (
+ api_message["tool_calls"][0]["function"]["arguments"]
+ == '{"expression":'
+ )
+
+
+def test_openai_compatible_reasoning_variants_round_trip():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ provider = OpenAIProvider("key", provider_name="openrouter")
+
+ response = SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content="",
+ reasoning="Visible reasoning",
+ reasoning_content=None,
+ reasoning_details=[
+ {
+ "type": "reasoning.encrypted",
+ "data": "opaque-signed-state",
+ }
+ ],
+ refusal=None,
+ tool_calls=None,
+ ),
+ finish_reason="stop",
+ )
+ ],
+ usage=_usage(),
+ )
+
+ normalized = provider._normalize(response, "anthropic/claude-sonnet-5")
+ assert normalized.thinking == "Visible reasoning"
+ assert normalized.assistant_message.provider_state["payload"] == {
+ "reasoning": "Visible reasoning",
+ "reasoning_details": [
+ {
+ "type": "reasoning.encrypted",
+ "data": "opaque-signed-state",
+ }
+ ],
+ }
+ replayed = provider._to_api_message(normalized.assistant_message)
+ assert replayed["reasoning"] == "Visible reasoning"
+ assert replayed["reasoning_details"][0]["data"] == "opaque-signed-state"
+
+
+def test_openai_chat_and_responses_refusals_are_visible_text():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ provider = OpenAIProvider("key")
+
+ chat_response = SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content=None,
+ reasoning=None,
+ reasoning_content=None,
+ reasoning_details=None,
+ refusal="I cannot help with that.",
+ tool_calls=None,
+ ),
+ finish_reason="stop",
+ )
+ ],
+ usage=_usage(),
+ )
+ chat_normalized = provider._normalize(chat_response, "gpt-5")
+ assert chat_normalized.content == "I cannot help with that."
+ assert chat_normalized.assistant_message.blocks[0].metadata == {
+ "refusal": True
+ }
+
+ responses_response = SimpleNamespace(
+ id="response-refusal",
+ model="gpt-5",
+ status="completed",
+ output=[
+ SimpleNamespace(
+ type="message",
+ content=[
+ SimpleNamespace(
+ type="refusal",
+ refusal="This request is not allowed.",
+ )
+ ],
+ )
+ ],
+ usage=None,
+ )
+ responses_normalized = provider._normalize_responses(
+ responses_response, "gpt-5"
+ )
+ assert responses_normalized.content == "This request is not allowed."
+ assert responses_normalized.assistant_message.blocks[0].metadata == {
+ "refusal": True
+ }
+
+
+async def test_openai_reasoning_tool_turn_uses_stateless_responses_api():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ provider = OpenAIProvider("key")
+
+ reasoning = SimpleNamespace(
+ type="reasoning",
+ id="reasoning-1",
+ encrypted_content="encrypted-state",
+ summary=[
+ SimpleNamespace(type="summary_text", text="Need the tool")
+ ],
+ )
+ function_call = SimpleNamespace(
+ type="function_call",
+ id="function-1",
+ call_id="call-1",
+ name="lookup",
+ arguments='{"q":"weather"}',
+ status="completed",
+ )
+ response = SimpleNamespace(
+ id="response-1",
+ model="gpt-5",
+ status="completed",
+ output=[reasoning, function_call],
+ usage=SimpleNamespace(
+ input_tokens=8,
+ output_tokens=5,
+ total_tokens=13,
+ input_tokens_details=SimpleNamespace(cached_tokens=2),
+ output_tokens_details=SimpleNamespace(reasoning_tokens=3),
+ ),
+ )
+ provider._client.responses.create = AsyncMock(return_value=response)
+
+ normalized = await provider.chat(
+ [Message(role="user", content="What is the weather?")],
+ model="gpt-5",
+ thinking=ThinkingConfig(enabled=True, effort="high"),
+ tools=[
+ ToolDef(
+ name="lookup",
+ description="Look up weather",
+ parameters={
+ "type": "object",
+ "properties": {"q": {"type": "string"}},
+ },
+ )
+ ],
+ )
+
+ params = provider._client.responses.create.call_args.kwargs
+ assert params["store"] is False
+ assert params["include"] == ["reasoning.encrypted_content"]
+ assert params["reasoning"] == {"effort": "high"}
+ assert params["tools"][0]["name"] == "lookup"
+ state = normalized.assistant_message.provider_state
+ assert state["payload"]["api"] == "responses"
+ assert state["payload"]["output"][0]["encrypted_content"] == (
+ "encrypted-state"
+ )
+
+ second_input = provider._to_responses_input(
+ [
+ Message(role="user", content="What is the weather?"),
+ normalized.assistant_message,
+ Message(
+ role="tool",
+ content='{"temperature": 20}',
+ tool_call_id="call-1",
+ name="lookup",
+ ),
+ ]
+ )
+ assert second_input[-1] == {
+ "type": "function_call_output",
+ "call_id": "call-1",
+ "output": '{"temperature": 20}',
+ }
+ assert second_input[1]["type"] == "reasoning"
+ assert second_input[1]["encrypted_content"] == "encrypted-state"
+
+
+def test_anthropic_signed_thinking_round_trips_in_original_block_order():
+ with patch("anthropic.AsyncAnthropic"):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ provider = AnthropicProvider("key")
+
+ response = SimpleNamespace(
+ content=[
+ SimpleNamespace(
+ type="thinking",
+ thinking="Check the source",
+ signature="signed-thinking",
+ ),
+ SimpleNamespace(
+ type="tool_use",
+ id="tool-1",
+ name="lookup",
+ input={"q": "source"},
+ ),
+ SimpleNamespace(type="text", text="Looking it up"),
+ ],
+ usage=SimpleNamespace(
+ input_tokens=12,
+ output_tokens=7,
+ cache_creation_input_tokens=2,
+ cache_read_input_tokens=3,
+ ),
+ stop_reason="tool_use",
+ )
+
+ normalized = provider._normalize(response, "claude-sonnet-4-6")
+ assert [block.type for block in normalized.assistant_message.blocks] == [
+ "reasoning",
+ "tool_call",
+ "text",
+ ]
+ json.dumps(normalized.assistant_message.provider_state)
+ api_message = provider._to_api_message(normalized.assistant_message)
+ assert api_message["content"] == [
+ {
+ "type": "thinking",
+ "thinking": "Check the source",
+ "signature": "signed-thinking",
+ },
+ {
+ "type": "tool_use",
+ "id": "tool-1",
+ "name": "lookup",
+ "input": {"q": "source"},
+ },
+ {"type": "text", "text": "Looking it up"},
+ ]
+ assert normalized.usage.total_tokens == 24
+
+
+def test_anthropic_parallel_tool_results_are_one_user_turn():
+ with patch("anthropic.AsyncAnthropic"):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ provider = AnthropicProvider("key")
+
+ _, api_messages = provider._split_system(
+ [
+ Message(
+ role="tool",
+ content='{"result": 1}',
+ tool_call_id="tool-1",
+ name="lookup",
+ ),
+ Message(
+ role="tool",
+ content='{"result": 2}',
+ tool_call_id="tool-2",
+ name="lookup",
+ ),
+ ]
+ )
+ assert api_messages == [
+ {
+ "role": "user",
+ "content": [
+ {
+ "type": "tool_result",
+ "tool_use_id": "tool-1",
+ "content": '{"result": 1}',
+ },
+ {
+ "type": "tool_result",
+ "tool_use_id": "tool-2",
+ "content": '{"result": 2}',
+ },
+ ],
+ }
+ ]
+
+
+def test_anthropic_redacted_thinking_round_trips():
+ with patch("anthropic.AsyncAnthropic"):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ provider = AnthropicProvider("key")
+
+ response = SimpleNamespace(
+ content=[
+ SimpleNamespace(
+ type="redacted_thinking",
+ data="base64-redacted-state",
+ ),
+ SimpleNamespace(type="text", text="Safe answer"),
+ ],
+ usage=SimpleNamespace(
+ input_tokens=4,
+ output_tokens=3,
+ cache_creation_input_tokens=0,
+ cache_read_input_tokens=0,
+ ),
+ stop_reason="end_turn",
+ )
+
+ normalized = provider._normalize(response, "claude-sonnet-4-6")
+ json.dumps(normalized.assistant_message.provider_state)
+ assert normalized.assistant_message.blocks[0].metadata == {
+ "redacted": True
+ }
+ assert provider._to_api_message(normalized.assistant_message)["content"] == [
+ {
+ "type": "redacted_thinking",
+ "data": "base64-redacted-state",
+ },
+ {"type": "text", "text": "Safe answer"},
+ ]
+
+
+def test_anthropic_binary_thinking_signature_is_base64_durable():
+ with patch("anthropic.AsyncAnthropic"):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ provider = AnthropicProvider("key")
+
+ response = SimpleNamespace(
+ content=[
+ SimpleNamespace(
+ type="thinking",
+ thinking="Binary signature",
+ signature=b"\x00signed\xff",
+ )
+ ],
+ usage=SimpleNamespace(
+ input_tokens=1,
+ output_tokens=1,
+ cache_creation_input_tokens=0,
+ cache_read_input_tokens=0,
+ ),
+ stop_reason="end_turn",
+ )
+
+ normalized = provider._normalize(response, "claude-sonnet-4-6")
+ wire = json.loads(
+ json.dumps(message_to_wire(normalized.assistant_message))
+ )
+ signature = wire["provider_state"]["payload"]["content"][0]["signature"]
+ assert signature == {BINARY_STATE_MARKER: "AHNpZ25lZP8="}
+
+ replayed = message_from_wire(wire)
+ api_message = provider._to_api_message(replayed)
+ assert api_message["content"][0]["signature"] == b"\x00signed\xff"
+
+
+def test_anthropic_binary_redacted_thinking_is_base64_durable():
+ with patch("anthropic.AsyncAnthropic"):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ provider = AnthropicProvider("key")
+
+ response = SimpleNamespace(
+ content=[
+ SimpleNamespace(
+ type="redacted_thinking",
+ data=b"\x00redacted\xff",
+ )
+ ],
+ usage=SimpleNamespace(
+ input_tokens=1,
+ output_tokens=1,
+ cache_creation_input_tokens=0,
+ cache_read_input_tokens=0,
+ ),
+ stop_reason="end_turn",
+ )
+
+ normalized = provider._normalize(response, "claude-sonnet-4-6")
+ wire = json.loads(
+ json.dumps(message_to_wire(normalized.assistant_message))
+ )
+ data = wire["provider_state"]["payload"]["content"][0]["data"]
+ assert data == {BINARY_STATE_MARKER: "AHJlZGFjdGVk/w=="}
+
+ replayed = message_from_wire(wire)
+ api_message = provider._to_api_message(replayed)
+ assert api_message["content"][0]["data"] == b"\x00redacted\xff"
+
+
+def test_gemini_signature_is_base64_durable_and_duplicate_names_get_unique_ids():
+ with patch("google.genai.Client"):
+ from services.llm.providers.gemini import GeminiProvider
+
+ provider = GeminiProvider("key")
+
+ parts = [
+ SimpleNamespace(
+ thought=True,
+ text="Need weather",
+ function_call=None,
+ thought_signature=b"\x00signed\xff",
+ ),
+ SimpleNamespace(
+ thought=False,
+ text=None,
+ function_call=SimpleNamespace(name="lookup", args={"city": "A"}),
+ thought_signature=b"first-call-signature",
+ ),
+ SimpleNamespace(
+ thought=False,
+ text=None,
+ function_call=SimpleNamespace(name="lookup", args={"city": "B"}),
+ thought_signature=b"second-call-signature",
+ ),
+ ]
+ response = SimpleNamespace(
+ response_id="response-1",
+ candidates=[
+ SimpleNamespace(
+ content=SimpleNamespace(parts=parts),
+ finish_reason="STOP",
+ )
+ ],
+ usage_metadata=SimpleNamespace(
+ prompt_token_count=10,
+ candidates_token_count=5,
+ total_token_count=17,
+ cached_content_token_count=4,
+ thoughts_token_count=2,
+ ),
+ )
+
+ normalized = provider._normalize(response, "gemini-2.5-flash")
+ assert normalized.tool_calls[0].id != normalized.tool_calls[1].id
+ assert normalized.usage.cache_read_tokens == 4
+ assert normalized.usage.reasoning_tokens == 2
+ state = normalized.assistant_message.provider_state
+ json.dumps(state)
+ assert state["payload"]["parts"][0]["thought_signature_b64"] == (
+ "AHNpZ25lZP8="
+ )
+
+ _, contents = provider._split_system_and_contents(
+ [normalized.assistant_message]
+ )
+ assert contents[0]["parts"][0]["thought_signature"] == b"\x00signed\xff"
+ assert contents[0]["parts"][1]["function_call"]["name"] == "lookup"
+ assert contents[0]["parts"][1]["function_call"]["id"] == (
+ normalized.tool_calls[0].id
+ )
+
+ _, tool_contents = provider._split_system_and_contents(
+ [
+ normalized.assistant_message,
+ Message(
+ role="tool",
+ content='{"weather":"sunny"}',
+ tool_call_id=normalized.tool_calls[0].id,
+ name="lookup",
+ ),
+ Message(
+ role="tool",
+ content='{"weather":"rain"}',
+ tool_call_id=normalized.tool_calls[1].id,
+ name="lookup",
+ ),
+ ]
+ )
+ assert tool_contents[1]["role"] == "user"
+ assert [
+ part["function_response"]["id"]
+ for part in tool_contents[1]["parts"]
+ ] == [call.id for call in normalized.tool_calls]
+
+
+def test_gemini_finish_reason_enum_normalizes_to_value():
+ from google.genai import types
+
+ with patch("google.genai.Client"):
+ from services.llm.providers.gemini import GeminiProvider
+
+ provider = GeminiProvider("key")
+
+ response = SimpleNamespace(
+ response_id="response-safety",
+ candidates=[
+ SimpleNamespace(
+ content=SimpleNamespace(parts=[]),
+ finish_reason=types.FinishReason.SAFETY,
+ )
+ ],
+ usage_metadata=None,
+ )
+ normalized = provider._normalize(response, "gemini-2.5-flash")
+ assert normalized.finish_reason == "safety"
+
+
+def test_gemini_malformed_tool_arguments_are_preserved_without_crash():
+ with patch("google.genai.Client"):
+ from services.llm.providers.gemini import GeminiProvider
+
+ provider = GeminiProvider("key")
+
+ response = SimpleNamespace(
+ response_id="response-malformed",
+ candidates=[
+ SimpleNamespace(
+ content=SimpleNamespace(
+ parts=[
+ SimpleNamespace(
+ thought=False,
+ text=None,
+ thought_signature=None,
+ function_call=SimpleNamespace(
+ id="call-malformed",
+ name="lookup",
+ args="{not-json",
+ ),
+ )
+ ]
+ ),
+ finish_reason="STOP",
+ )
+ ],
+ usage_metadata=None,
+ )
+
+ normalized = provider._normalize(response, "gemini-2.5-flash")
+ call = normalized.tool_calls[0]
+ assert call.id == "call-malformed"
+ assert call.args == {}
+ assert call.raw_arguments == "{not-json"
+ assert call.parse_error
+ function_state = normalized.assistant_message.provider_state[
+ "payload"
+ ]["parts"][0]["function_call"]
+ assert function_state["raw_arguments"] == "{not-json"
+ assert function_state["parse_error"] == call.parse_error
+ json.dumps(normalized.assistant_message.provider_state)
+
+
+async def test_compat_provider_policy_applies_kimi_and_groq_quirks():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ kimi = OpenAIProvider("key", provider_name="kimi")
+ kimi._client.chat.completions.create = AsyncMock(
+ return_value=SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content="ok",
+ reasoning_content=None,
+ refusal=None,
+ tool_calls=None,
+ ),
+ finish_reason="stop",
+ )
+ ],
+ usage=_usage(),
+ )
+ )
+ await kimi.chat(
+ [Message(role="user", content="hi")],
+ model="kimi-k2.5",
+ temperature=0.1,
+ )
+ kimi_params = kimi._client.chat.completions.create.call_args.kwargs
+ assert kimi_params["temperature"] == 0.6
+ assert kimi_params["extra_body"]["thinking"] == {"type": "disabled"}
+ assert "max_tokens" in kimi_params
+
+ groq = OpenAIProvider("key", provider_name="groq")
+ groq._client.chat.completions.create = kimi._client.chat.completions.create
+ await groq.chat(
+ [Message(role="user", content="hi")],
+ model="qwen/qwen3-32b",
+ thinking=ThinkingConfig(enabled=True, format="hidden"),
+ )
+ groq_params = groq._client.chat.completions.create.call_args.kwargs
+ assert groq_params["model"] == "qwen/qwen3-32b"
+ assert groq_params["extra_body"]["reasoning_format"] == "hidden"
+
+ await groq.chat(
+ [Message(role="user", content="hi")],
+ model="openai/gpt-oss-120b",
+ thinking=ThinkingConfig(enabled=True, effort="high"),
+ )
+ gpt_oss_params = (
+ groq._client.chat.completions.create.call_args.kwargs
+ )
+ assert gpt_oss_params["model"] == "openai/gpt-oss-120b"
+ assert gpt_oss_params["reasoning_effort"] == "high"
+ assert "extra_body" not in gpt_oss_params
+
+
+async def test_openai_reasoning_capable_model_omits_temperature_without_toggle():
+ with patch("openai.AsyncOpenAI"):
+ from services.llm.providers.openai import OpenAIProvider
+
+ provider = OpenAIProvider("key")
+
+ provider._client.chat.completions.create = AsyncMock(
+ return_value=SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content="ok",
+ reasoning=None,
+ reasoning_content=None,
+ reasoning_details=None,
+ refusal=None,
+ tool_calls=None,
+ ),
+ finish_reason="stop",
+ )
+ ],
+ usage=_usage(),
+ )
+ )
+
+ await provider.chat(
+ [Message(role="user", content="hi")],
+ model="gpt-5.6-sol",
+ temperature=0.2,
+ )
+ params = provider._client.chat.completions.create.call_args.kwargs
+ assert "temperature" not in params
+ assert params["max_completion_tokens"] == 4096
+ assert "max_tokens" not in params
diff --git a/server/tests/llm/test_provider_self_registration.py b/server/tests/llm/test_provider_self_registration.py
index 110c9f90..112c432b 100644
--- a/server/tests/llm/test_provider_self_registration.py
+++ b/server/tests/llm/test_provider_self_registration.py
@@ -6,9 +6,13 @@ full set of native + OpenAI-compat providers. If any provider drops its
``register_provider(...)`` call, this test catches it.
"""
+from types import SimpleNamespace
+from unittest.mock import patch
+
import pytest
import services.llm # noqa: F401 — side-effect import populates the registry
+from services.llm.protocol import ToolDef
from services.llm.registry import all_providers, get_provider, has_provider
@@ -30,6 +34,16 @@ EXPECTED_REGISTERED_PROVIDERS = {
"groq",
"cerebras",
}
+COMPAT_PROVIDERS = (
+ "xai",
+ "deepseek",
+ "kimi",
+ "mistral",
+ "ollama",
+ "lmstudio",
+ "groq",
+ "cerebras",
+)
def test_all_expected_providers_registered():
@@ -37,6 +51,22 @@ def test_all_expected_providers_registered():
assert set(all_providers()) >= EXPECTED_REGISTERED_PROVIDERS
+def test_all_twelve_providers_route_to_the_expected_native_class():
+ from services.llm.providers.anthropic import AnthropicProvider
+ from services.llm.providers.gemini import GeminiProvider
+ from services.llm.providers.openai import OpenAIProvider
+ from services.llm.providers.openrouter import OpenRouterProvider
+
+ assert get_provider("anthropic").factory is AnthropicProvider
+ assert get_provider("gemini").factory is GeminiProvider
+ assert get_provider("openai").factory is OpenAIProvider
+ assert get_provider("openrouter").factory is OpenRouterProvider
+ for name in COMPAT_PROVIDERS:
+ spec = get_provider(name)
+ assert spec.factory is OpenAIProvider
+ assert spec.client_kwargs["provider_name"] == name
+
+
def test_dedicated_providers_have_typed_sdk_exceptions():
"""The four dedicated SDKs declare their typed exception classes."""
import anthropic
@@ -68,6 +98,89 @@ def test_compat_providers_carry_base_url_in_client_kwargs():
)
[email protected]("provider_name", COMPAT_PROVIDERS)
+def test_every_compat_provider_inherits_shared_response_and_tool_contract(
+ provider_name,
+):
+ """One matrix gate covers normalization/schema behavior for all 8."""
+
+ from services.llm.providers.openai import OpenAIProvider
+
+ spec = get_provider(provider_name)
+ with patch("openai.AsyncOpenAI"):
+ provider = OpenAIProvider(
+ "key",
+ provider_name=provider_name,
+ base_url=spec.client_kwargs["base_url"],
+ )
+
+ response = SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(
+ content="answer",
+ reasoning="reasoning",
+ reasoning_content=None,
+ reasoning_details=None,
+ refusal=None,
+ tool_calls=[
+ SimpleNamespace(
+ id="call-1",
+ function=SimpleNamespace(
+ name="lookup",
+ arguments="{not-json",
+ ),
+ )
+ ],
+ ),
+ finish_reason="tool_calls",
+ )
+ ],
+ usage=SimpleNamespace(
+ prompt_tokens=10,
+ completion_tokens=4,
+ total_tokens=14,
+ prompt_tokens_details=SimpleNamespace(cached_tokens=2),
+ completion_tokens_details=SimpleNamespace(reasoning_tokens=3),
+ ),
+ )
+ normalized = provider._normalize(response, "provider-model")
+
+ assert normalized.content == "answer"
+ assert normalized.thinking == "reasoning"
+ assert normalized.finish_reason == "tool_calls"
+ assert normalized.usage.total_tokens == 14
+ assert normalized.usage.cache_read_tokens == 2
+ assert normalized.usage.reasoning_tokens == 3
+ assert normalized.tool_calls[0].raw_arguments == "{not-json"
+ assert normalized.tool_calls[0].parse_error
+
+ tool = provider._to_api_tool(
+ ToolDef(
+ name="lookup",
+ description="Lookup",
+ parameters={
+ "type": "object",
+ "properties": {
+ "mode": {"enum": ["fast", "deep"]},
+ "query": {
+ "anyOf": [
+ {"type": "string"},
+ {"type": "null"},
+ ]
+ },
+ },
+ "required": ["mode"],
+ },
+ )
+ )
+ assert tool["function"]["parameters"]["properties"]["mode"]["enum"] == [
+ "fast",
+ "deep",
+ ]
+ assert spec.sdk_exception_refs == ("openai:OpenAIError",)
+
+
def test_unknown_provider_raises_node_user_error():
"""Unknown provider lookups surface as NodeUserError (user-correctable)."""
from services.plugin import NodeUserError
diff --git a/server/tests/llm/test_providers.py b/server/tests/llm/test_providers.py
index f4d12d29..7dfb29b5 100644
--- a/server/tests/llm/test_providers.py
+++ b/server/tests/llm/test_providers.py
@@ -90,6 +90,32 @@ class TestAnthropicProvider:
assert call_kwargs["thinking"]["budget_tokens"] == 4096
assert call_kwargs["temperature"] == 1
+ @pytest.mark.asyncio
+ async def test_chat_clamps_thinking_budget_and_expands_max_tokens(
+ self, provider
+ ):
+ mock_resp = MagicMock()
+ mock_resp.content = [MagicMock(type="text", text="ok")]
+ mock_resp.usage = MagicMock(
+ input_tokens=5,
+ output_tokens=2,
+ cache_creation_input_tokens=0,
+ cache_read_input_tokens=0,
+ )
+ mock_resp.stop_reason = "end_turn"
+ provider._client.messages.create = AsyncMock(return_value=mock_resp)
+
+ await provider.chat(
+ [Message(role="user", content="test")],
+ model="claude-sonnet-4-6",
+ max_tokens=500,
+ thinking=ThinkingConfig(enabled=True, budget=100),
+ )
+
+ call_kwargs = provider._client.messages.create.call_args.kwargs
+ assert call_kwargs["thinking"]["budget_tokens"] == 1024
+ assert call_kwargs["max_tokens"] == 2048
+
# ---------------------------------------------------------------------------
# OpenAI: message formatting + response normalization
@@ -201,11 +227,26 @@ class TestGeminiProvider:
def test_tool_response_format(self, provider):
msgs = [
- Message(role="tool", content="42", name="calc"),
+ Message(
+ role="tool",
+ content="42",
+ name="calc",
+ tool_call_id="call-1",
+ ),
+ Message(
+ role="tool",
+ content="7",
+ name="calc",
+ tool_call_id="call-2",
+ ),
]
_, contents = provider._split_system_and_contents(msgs)
- assert contents[0]["role"] == "function"
+ assert len(contents) == 1
+ assert contents[0]["role"] == "user"
+ assert len(contents[0]["parts"]) == 2
assert "function_response" in contents[0]["parts"][0]
+ assert contents[0]["parts"][0]["function_response"]["id"] == "call-1"
+ assert contents[0]["parts"][1]["function_response"]["id"] == "call-2"
def test_normalize_text(self, provider):
part = MagicMock(thought=False, function_call=None, text="Hello")
@@ -238,8 +279,8 @@ class TestGeminiProvider:
assert result.content == "Answer"
@pytest.mark.asyncio
- async def test_chat_thinking_forwards_to_sdk(self, provider):
- """Thinking config fields are forwarded to the SDK's ThinkingConfig."""
+ async def test_chat_thinking_level_takes_precedence_over_budget(self, provider):
+ """Gemini 3 level and Gemini 2.5 budget are mutually exclusive."""
mock_types = MagicMock()
mock_resp = MagicMock()
mock_resp.candidates = []
@@ -255,8 +296,9 @@ class TestGeminiProvider:
)
call_kwargs = mock_types.ThinkingConfig.call_args[1]
- assert call_kwargs["thinking_budget"] == 4096
+ assert call_kwargs["include_thoughts"] is True
assert call_kwargs["thinking_level"] == "high"
+ assert "thinking_budget" not in call_kwargs
@pytest.mark.asyncio
async def test_chat_no_thinking_when_disabled(self, provider):
diff --git a/server/tests/llm/test_tool_schema.py b/server/tests/llm/test_tool_schema.py
new file mode 100644
index 00000000..a460a24b
--- /dev/null
+++ b/server/tests/llm/test_tool_schema.py
@@ -0,0 +1,106 @@
+import pytest
+
+from services.llm.schema import compile_tool_schema
+
+
+def test_schema_compiler_inlines_pydantic_refs_without_mutating_input():
+ schema = {
+ "$defs": {
+ "Location": {
+ "type": "object",
+ "properties": {"city": {"type": "string"}},
+ "required": ["city"],
+ }
+ },
+ "type": "object",
+ "properties": {
+ "location": {
+ "$ref": "#/$defs/Location",
+ "description": "Where to search",
+ }
+ },
+ }
+
+ compiled = compile_tool_schema(schema, provider="anthropic")
+ assert "$defs" in schema
+ assert "$defs" not in compiled
+ assert compiled["properties"]["location"] == {
+ "type": "object",
+ "properties": {"city": {"type": "string"}},
+ "required": ["city"],
+ "description": "Where to search",
+ }
+
+
+def test_gemini_schema_normalizes_nullable_and_drops_unsupported_metadata():
+ compiled = compile_tool_schema(
+ {
+ "type": "object",
+ "properties": {
+ "unit": {
+ "type": ["string", "null"],
+ "default": None,
+ "examples": ["celsius"],
+ }
+ },
+ },
+ provider="gemini",
+ )
+ unit = compiled["properties"]["unit"]
+ assert unit == {"type": "string", "nullable": True}
+
+
+def test_gemini_schema_preserves_literals_enums_and_nested_nullable_unions():
+ compiled = compile_tool_schema(
+ {
+ "type": "object",
+ "properties": {
+ "operation": {
+ "type": "string",
+ "const": "lookup",
+ },
+ "options": {
+ "type": "object",
+ "properties": {
+ "mode": {
+ "anyOf": [
+ {"type": "string", "const": "fast"},
+ {"type": "null"},
+ ]
+ },
+ "unit": {
+ "type": "string",
+ "enum": ["celsius", "fahrenheit"],
+ },
+ },
+ },
+ },
+ },
+ provider="gemini",
+ )
+
+ assert compiled["properties"]["operation"] == {
+ "type": "string",
+ "enum": ["lookup"],
+ }
+ options = compiled["properties"]["options"]["properties"]
+ assert options["mode"] == {
+ "anyOf": [{"type": "string", "enum": ["fast"]}],
+ "nullable": True,
+ }
+ assert options["unit"]["enum"] == ["celsius", "fahrenheit"]
+
+
+def test_schema_compiler_rejects_external_and_recursive_refs():
+ with pytest.raises(ValueError, match="Only local"):
+ compile_tool_schema(
+ {"$ref": "https://example.com/schema.json"},
+ provider="openai",
+ )
+
+ recursive = {
+ "$defs": {"Node": {"$ref": "#/$defs/Node"}},
+ "$ref": "#/$defs/Node",
+ }
+ with pytest.raises(ValueError, match="Recursive"):
+ compile_tool_schema(recursive, provider="openai")
diff --git a/server/tests/llm/test_unifier_client_policy.py b/server/tests/llm/test_unifier_client_policy.py
new file mode 100644
index 00000000..3735e18d
--- /dev/null
+++ b/server/tests/llm/test_unifier_client_policy.py
@@ -0,0 +1,306 @@
+import asyncio
+from unittest.mock import AsyncMock, MagicMock
+
+import openai
+import pytest
+
+from services.llm.protocol import LLMError, LLMErrorCategory, LLMResponse, Message
+from services.llm.registry import ProviderSpec
+from services.llm.unifier import ChatUnifier
+from services.plugin import NodeUserError
+
+
+async def test_unifier_caches_by_credential_and_retry_policy_and_closes_clients():
+ created = []
+
+ class Client:
+ def __init__(self, **kwargs):
+ self.kwargs = kwargs
+ self.chat = AsyncMock(return_value=LLMResponse(content="ok"))
+ self.closed = False
+ created.append(self)
+
+ async def aclose(self):
+ self.closed = True
+
+ spec = ProviderSpec(
+ name="cache-test",
+ factory=Client,
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=2,
+ )
+
+ first = await unifier._build_client(
+ spec, "secret-a", sdk_max_retries=0
+ )
+ again = await unifier._build_client(
+ spec, "secret-a", sdk_max_retries=0
+ )
+ different_policy = await unifier._build_client(
+ spec, "secret-a", sdk_max_retries=2
+ )
+
+ assert first is again
+ assert different_policy is not first
+ assert len(created) == 2
+ assert created[0].kwargs["max_retries"] == 0
+ await unifier.aclose()
+ assert all(client.closed for client in created)
+
+
+async def test_unifier_can_surface_structured_error_for_agent_retry_policy():
+ response = MagicMock()
+ response.status_code = 429
+ response.headers = {"retry-after": "1.5", "x-request-id": "req-1"}
+ error = openai.RateLimitError(
+ message="rate limited",
+ response=response,
+ body={"error": {"code": "capacity"}},
+ )
+ client = MagicMock()
+ client.chat = AsyncMock(side_effect=error)
+
+ from services.llm import registry
+
+ name = "structured-error-test"
+ spec = ProviderSpec(
+ name=name,
+ factory=lambda **kwargs: client,
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ registry._REGISTRY[name] = spec
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=0,
+ )
+ try:
+ with pytest.raises(LLMError) as caught:
+ await unifier.chat(
+ provider=name,
+ api_key="secret",
+ messages=[Message(role="user", content="hello")],
+ model="model",
+ sdk_max_retries=0,
+ translate_errors=False,
+ )
+ finally:
+ registry._REGISTRY.pop(name, None)
+
+ assert caught.value.category == LLMErrorCategory.RATE_LIMIT
+ assert caught.value.retryable
+ assert caught.value.request_id == "req-1"
+ assert caught.value.retry_after == 1.5
+ assert caught.value.retry_after_raw == "1.5"
+ assert caught.value.provider_code == "capacity"
+
+
+async def test_unifier_public_error_does_not_expose_raw_sdk_message():
+ response = MagicMock()
+ response.status_code = 400
+ response.headers = {"x-request-id": "req-private"}
+ error = openai.BadRequestError(
+ message=(
+ "POST https://private-gateway.internal/v1 "
+ 'payload={"authorization":"Bearer secret"}'
+ ),
+ response=response,
+ body={"error": {"code": "invalid_payload"}},
+ )
+ client = MagicMock()
+ client.chat = AsyncMock(side_effect=error)
+
+ from services.llm import registry
+
+ name = "public-error-test"
+ registry._REGISTRY[name] = ProviderSpec(
+ name=name,
+ factory=lambda **_kwargs: client,
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=0,
+ )
+ try:
+ with pytest.raises(NodeUserError) as caught:
+ await unifier.chat(
+ provider=name,
+ api_key="secret",
+ messages=[Message(role="user", content="hello")],
+ model="model",
+ sdk_max_retries=0,
+ )
+ finally:
+ registry._REGISTRY.pop(name, None)
+
+ assert str(caught.value) == (
+ "The language model provider rejected the model request configuration."
+ )
+ assert "private-gateway" not in str(caught.value)
+ assert "Bearer secret" not in str(caught.value)
+
+
+async def test_client_constructor_failure_is_structured_and_safe():
+ from services.llm import registry
+
+ raw_message = (
+ "Invalid proxy URL https://user:[email protected]"
+ )
+ name = "constructor-error-test"
+ registry._REGISTRY[name] = ProviderSpec(
+ name=name,
+ factory=lambda **_kwargs: (_ for _ in ()).throw(
+ ValueError(raw_message)
+ ),
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=1,
+ )
+ try:
+ with pytest.raises(NodeUserError) as caught:
+ await unifier.chat(
+ provider=name,
+ api_key="secret",
+ messages=[Message(role="user", content="hello")],
+ model="model",
+ )
+ with pytest.raises(LLMError) as structured:
+ await unifier.chat(
+ provider=name,
+ api_key="secret",
+ messages=[Message(role="user", content="hello")],
+ model="model",
+ translate_errors=False,
+ )
+ finally:
+ registry._REGISTRY.pop(name, None)
+
+ assert str(caught.value) == (
+ "The language model provider rejected the model request configuration."
+ )
+ assert "private-gateway" not in str(caught.value)
+ assert structured.value.category == LLMErrorCategory.INVALID_REQUEST
+ assert structured.value.message == raw_message
+
+
+async def test_unifier_closes_ephemeral_client_when_cache_is_disabled():
+ client = MagicMock()
+ client.chat = AsyncMock(return_value=LLMResponse(content="ok"))
+ client.aclose = AsyncMock()
+
+ from services.llm import registry
+
+ name = "ephemeral-client-test"
+ registry._REGISTRY[name] = ProviderSpec(
+ name=name,
+ factory=lambda **_kwargs: client,
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=0,
+ )
+ try:
+ await unifier.chat(
+ provider=name,
+ api_key="secret",
+ messages=[Message(role="user", content="hello")],
+ model="model",
+ )
+ finally:
+ registry._REGISTRY.pop(name, None)
+
+ client.aclose.assert_awaited_once()
+
+
+async def test_lru_eviction_defers_close_until_in_flight_chat_releases_lease():
+ started = asyncio.Event()
+ release = asyncio.Event()
+ created = {}
+
+ class Client:
+ def __init__(self, **kwargs):
+ self.api_key = kwargs["api_key"]
+ self.closed = False
+ created[self.api_key] = self
+
+ async def chat(self, *_args, **_kwargs):
+ if self.api_key == "secret-a":
+ started.set()
+ await release.wait()
+ assert not self.closed
+ return LLMResponse(content="ok")
+
+ async def aclose(self):
+ self.closed = True
+
+ from services.llm import registry
+
+ name = "leased-cache-test"
+ registry._REGISTRY[name] = ProviderSpec(
+ name=name,
+ factory=Client,
+ sdk_exception_refs=("openai:OpenAIError",),
+ )
+ auth = MagicMock()
+ auth.get_api_key = AsyncMock(return_value=None)
+ unifier = ChatUnifier(
+ defaults={"providers": {}},
+ auth_service=auth,
+ client_cache_size=1,
+ )
+
+ first_call = asyncio.create_task(
+ unifier.chat(
+ provider=name,
+ api_key="secret-a",
+ messages=[Message(role="user", content="first")],
+ model="model",
+ )
+ )
+ try:
+ await asyncio.wait_for(started.wait(), timeout=2)
+ await unifier.chat(
+ provider=name,
+ api_key="secret-b",
+ messages=[Message(role="user", content="second")],
+ model="model",
+ )
+
+ # secret-a was evicted from the size-1 cache, but its active request
+ # still owns a lease and must not see a closed SDK transport.
+ assert created["secret-a"].closed is False
+ release.set()
+ await asyncio.wait_for(first_call, timeout=2)
+ assert created["secret-a"].closed is True
+ finally:
+ release.set()
+ await asyncio.gather(first_call, return_exceptions=True)
+ await unifier.aclose()
+ registry._REGISTRY.pop(name, None)
diff --git a/server/tests/llm/test_unifier_typed_errors.py b/server/tests/llm/test_unifier_typed_errors.py
index 96d6a752..960e4526 100644
--- a/server/tests/llm/test_unifier_typed_errors.py
+++ b/server/tests/llm/test_unifier_typed_errors.py
@@ -13,7 +13,7 @@ from google.genai import errors as google_genai_errors
from unittest.mock import AsyncMock, MagicMock
import services.llm # noqa: F401 — populate registry
-from services.llm.registry import ProviderSpec, register_provider, _reset_for_tests, get_provider
+from services.llm.registry import ProviderSpec, get_provider
from services.llm.unifier import ChatUnifier
from services.llm.protocol import Message
from services.plugin import NodeUserError
@@ -64,13 +64,15 @@ def _replace_factory(provider_name: str, exc: Exception):
async def test_anthropic_typed_error_becomes_node_user_error(unifier):
original = _replace_factory("anthropic", anthropic.APIError(message="bad key", request=MagicMock(), body=None))
try:
- with pytest.raises(NodeUserError, match="bad key"):
+ with pytest.raises(NodeUserError) as caught:
await unifier.chat(
provider="anthropic",
api_key="sk-test",
messages=[Message(role="user", content="hi")],
model="claude-sonnet-4-6",
)
+ assert str(caught.value) == "Anthropic request failed."
+ assert "bad key" not in str(caught.value)
finally:
from services.llm import registry as _registry
@@ -81,13 +83,17 @@ async def test_anthropic_typed_error_becomes_node_user_error(unifier):
async def test_openai_typed_error_becomes_node_user_error(unifier):
original = _replace_factory("openai", openai.AuthenticationError(message="bad key", response=MagicMock(), body=None))
try:
- with pytest.raises(NodeUserError, match="bad key"):
+ with pytest.raises(NodeUserError) as caught:
await unifier.chat(
provider="openai",
api_key="sk-test",
messages=[Message(role="user", content="hi")],
model="gpt-5",
)
+ assert str(caught.value) == (
+ "OpenAI authentication failed. Check the configured API key."
+ )
+ assert "bad key" not in str(caught.value)
finally:
from services.llm import registry as _registry
@@ -104,13 +110,17 @@ async def test_gemini_typed_error_becomes_node_user_error(unifier):
google_genai_errors.APIError(code=400, response_json={"message": "This model only supports Interactions API."}),
)
try:
- with pytest.raises(NodeUserError, match="Interactions API"):
+ with pytest.raises(NodeUserError) as caught:
await unifier.chat(
provider="gemini",
api_key="key",
messages=[Message(role="user", content="hi")],
model="antigravity-preview-05-2026",
)
+ assert str(caught.value) == (
+ "Gemini rejected the model request configuration."
+ )
+ assert "Interactions API" not in str(caught.value)
finally:
from services.llm import registry as _registry
@@ -121,13 +131,18 @@ async def test_gemini_typed_error_becomes_node_user_error(unifier):
async def test_openrouter_typed_error_becomes_node_user_error(unifier):
original = _replace_factory("openrouter", openai.RateLimitError(message="429", response=MagicMock(), body=None))
try:
- with pytest.raises(NodeUserError, match="429"):
+ with pytest.raises(NodeUserError) as caught:
await unifier.chat(
provider="openrouter",
api_key="or-key",
messages=[Message(role="user", content="hi")],
model="anthropic/claude-sonnet-4.6",
)
+ assert str(caught.value) == (
+ "OpenRouter is rate-limiting requests. "
+ "Retry after a short delay."
+ )
+ assert "429" not in str(caught.value)
finally:
from services.llm import registry as _registry
diff --git a/server/tests/llm/test_vertex_key.py b/server/tests/llm/test_vertex_key.py
index 8363e85b..9dd56733 100644
--- a/server/tests/llm/test_vertex_key.py
+++ b/server/tests/llm/test_vertex_key.py
@@ -255,7 +255,10 @@ class TestGeminiProviderVertexMode:
)
kwargs = mock_types.ThinkingConfig.call_args[1]
- assert kwargs == {"thinking_budget": 1024}
+ assert kwargs == {
+ "include_thoughts": True,
+ "thinking_budget": 1024,
+ }
# ---------------------------------------------------------------------------
diff --git a/server/tests/nodes/test_ai_agents.py b/server/tests/nodes/test_ai_agents.py
index 115bbf00..98008340 100644
--- a/server/tests/nodes/test_ai_agents.py
+++ b/server/tests/nodes/test_ai_agents.py
@@ -87,6 +87,8 @@ class TestAIAgent:
"memory_content": "# Conversation History\n\n### **Human** (t)\nhi\n",
"long_term_enabled": True,
"retrieval_count": 5,
+ "embedding_provider": "ollama",
+ "embedding_endpoint": "http://localhost:11434",
}
)
@@ -110,6 +112,9 @@ class TestAIAgent:
assert mem["window_size"] == 7
assert mem["long_term_enabled"] is True
assert mem["retrieval_count"] == 5
+ assert mem["embedding_provider"] == "ollama"
+ assert mem["embedding_model"] == "nomic-embed-text"
+ assert mem["embedding_endpoint"] == "http://localhost:11434"
assert "### **Human**" in mem["memory_content"]
async def test_memory_explicit_session_overrides_auto(self, harness):
diff --git a/server/tests/nodes/test_code_fs_process.py b/server/tests/nodes/test_code_fs_process.py
index c657a296..a5e7f900 100644
--- a/server/tests/nodes/test_code_fs_process.py
+++ b/server/tests/nodes/test_code_fs_process.py
@@ -7,7 +7,7 @@ Frozen behavioural contract for the 8 nodes in
`docs-internal/node-logic-flows/code_fs_process/`. Each test asserts the
standard handler envelope shape and at least one payload detail from the
matching doc; external side-effects (subprocess spawn, Node.js HTTP call,
-deepagents backend, Terminal broadcast) are mocked.
+workspace backend, Terminal broadcast) are mocked.
Mocking strategy:
- pythonExecutor runs in-process (no external service) -- tests exercise
@@ -15,8 +15,8 @@ Mocking strategy:
- javascriptExecutor / typescriptExecutor: patch NodeJSClient.execute()
via the module-level _nodejs_client singleton.
- filesystem nodes (fileRead, fileModify, shell, fsSearch): patch
- services.handlers.filesystem._get_backend to return a fake backend with
- the methods each mode invokes.
+ nodes.filesystem._backend.get_backend to return a fake WorkspaceBackend
+ with the methods each mode invokes.
- processManager: patch services.process_service.get_process_service to
return a fake ProcessService with the dispatched method set to an
AsyncMock / MagicMock.
@@ -26,6 +26,7 @@ from __future__ import annotations
import asyncio
import os
+import shutil
import stat
import threading
import time
@@ -67,14 +68,14 @@ def _reset_nodejs_singleton():
class _FakeWriteResult(SimpleNamespace):
- """Mimics deepagents WriteResult / EditResult dataclasses."""
+ """Small write/edit result stand-in retained for fixture compatibility."""
def __init__(self, error: str = "", path: str = "", occurrences: int = 0):
super().__init__(error=error, path=path, occurrences=occurrences)
class _FakeReadResult(SimpleNamespace):
- """Mimics deepagents ReadResult dataclass (error, file_data)."""
+ """Mimics the native ReadResult contract (error, file_data)."""
def __init__(self, error: str | None = None, file_data: dict | None = None):
super().__init__(error=error, file_data=file_data)
@@ -304,12 +305,12 @@ class TestTypescriptExecutor:
class TestFileRead:
async def test_happy_path_returns_content(self, harness):
- # deepagents 0.5.x ``backend.read`` returns a ``ReadResult``
- # dataclass — the plugin must unwrap ``file_data["content"]``
- # instead of leaking the raw object into the output dict
+ # ``WorkspaceBackend.read`` returns a ``ReadResult`` dataclass. The
+ # plugin must unwrap ``file_data["content"]`` instead of leaking the
+ # raw object into the output dict
# (regression: "Object of type ReadResult is not JSON
# serializable" at node_outputs persistence).
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.read = MagicMock(
return_value=_FakeReadResult(file_data={"content": "line1\nline2", "encoding": "utf-8"})
)
@@ -326,7 +327,7 @@ class TestFileRead:
assert payload["content"] == "line1\nline2"
assert payload["line_count"] == 2
# ``normalize_virtual_path`` prepends ``/`` to relative inputs so the
- # path reaches deepagents in its canonical virtual-mode form.
+ # path reaches WorkspaceBackend in canonical virtual-path form.
assert payload["file_path"] == "/notes.txt"
backend.read.assert_called_once_with("/notes.txt", offset=0, limit=100)
# The whole envelope must be JSON-serializable — this is exactly
@@ -339,7 +340,7 @@ class TestFileRead:
"""A ``ReadResult`` carrying ``error`` is a failed read — it must
surface as a NodeUserError envelope, not pass as success with a
raw dataclass payload."""
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.read = MagicMock(return_value=_FakeReadResult(error="File '/missing.txt' not found"))
with _patch_fs_backend(backend):
@@ -357,7 +358,7 @@ class TestFileRead:
assert "file_path is required" in result["error"].lower()
async def test_backend_exception_becomes_error_envelope(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.read = MagicMock(side_effect=FileNotFoundError("No such file: ../../etc/passwd"))
with _patch_fs_backend(backend):
@@ -365,12 +366,12 @@ class TestFileRead:
harness.assert_envelope(result, success=False)
# ``normalize_virtual_path`` rejects ``..`` segments before the
- # backend is even called, so the error comes from deepagents'
- # ``validate_path`` helper, not the (unreached) FileNotFoundError.
+ # backend is called, so the error comes from the shared path validator,
+ # not the (unreached) FileNotFoundError.
assert "path traversal not allowed" in result["error"].lower()
# LLM-correctable input error: NodeUserError path (single WARN,
- # no traceback) carrying actionable sandbox guidance — not the
- # generic-exception path that leaks deepagents' bare ValueError.
+ # no traceback) carrying actionable workspace guidance — not the
+ # generic-exception path that leaks the validator's bare ValueError.
assert result.get("error_type") == "NodeUserError"
assert "workspace" in result["error"].lower()
@@ -402,9 +403,16 @@ class TestFileRead:
class TestFileModify:
async def test_write_happy_path(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
destination = _writable_test_path("hello.txt")
backend._resolve_path.return_value = destination
+ backend.atomic_write_text.side_effect = (
+ lambda path, content: path.write_text(
+ content,
+ encoding="utf-8",
+ newline="",
+ )
+ )
try:
with _patch_fs_backend(backend):
@@ -426,10 +434,20 @@ class TestFileModify:
destination.unlink(missing_ok=True)
async def test_edit_happy_path_returns_occurrences(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
destination = _writable_test_path("README.md")
destination.write_text("foo foo foo", encoding="utf-8")
backend._resolve_path.return_value = destination
+ backend.read_text_secure.side_effect = (
+ lambda path: path.read_text(encoding="utf-8")
+ )
+ backend.atomic_write_text.side_effect = (
+ lambda path, content: path.write_text(
+ content,
+ encoding="utf-8",
+ newline="",
+ )
+ )
try:
with _patch_fs_backend(backend):
@@ -452,7 +470,7 @@ class TestFileModify:
destination.unlink(missing_ok=True)
async def test_edit_missing_old_string_short_circuits(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
with _patch_fs_backend(backend):
result = await harness.execute(
@@ -469,10 +487,20 @@ class TestFileModify:
backend.edit.assert_not_called() if hasattr(backend, "edit") else None
async def test_backend_reports_error_in_result(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
destination = _writable_test_path("x.md")
destination.write_text("foo foo", encoding="utf-8")
backend._resolve_path.return_value = destination
+ backend.read_text_secure.side_effect = (
+ lambda path: path.read_text(encoding="utf-8")
+ )
+ backend.atomic_write_text.side_effect = (
+ lambda path, content: path.write_text(
+ content,
+ encoding="utf-8",
+ newline="",
+ )
+ )
try:
with _patch_fs_backend(backend):
@@ -493,7 +521,7 @@ class TestFileModify:
destination.unlink(missing_ok=True)
async def test_same_path_mutations_are_serialized(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
destination = _writable_test_path("shared.txt")
backend._resolve_path.return_value = destination
state = {"active": 0, "maximum": 0}
@@ -509,10 +537,8 @@ class TestFileModify:
state["active"] -= 1
try:
- with _patch_fs_backend(backend), patch(
- "nodes.filesystem._backend.atomic_write_text",
- side_effect=slow_atomic_write,
- ):
+ backend.atomic_write_text.side_effect = slow_atomic_write
+ with _patch_fs_backend(backend):
first, second = await asyncio.gather(
harness.execute(
"fileModify",
@@ -534,7 +560,7 @@ class TestFileModify:
async def test_cancellation_keeps_path_lock_until_write_finishes(self, harness):
from nodes.filesystem._backend import get_path_lock
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
destination = _writable_test_path("cancelled-write.txt")
backend._resolve_path.return_value = destination
entered = threading.Event()
@@ -547,10 +573,8 @@ class TestFileModify:
path.write_text(content, encoding="utf-8", newline="")
try:
- with _patch_fs_backend(backend), patch(
- "nodes.filesystem._backend.atomic_write_text",
- side_effect=blocking_atomic_write,
- ):
+ backend.atomic_write_text.side_effect = blocking_atomic_write
+ with _patch_fs_backend(backend):
operation = asyncio.create_task(
harness.execute(
"fileModify",
@@ -577,6 +601,77 @@ class TestFileModify:
release.set()
destination.unlink(missing_ok=True)
+ @pytest.mark.skipif(
+ os.name == "nt",
+ reason="POSIX descriptor-relative replacement race",
+ )
+ @pytest.mark.parametrize("operation", ["write", "edit"])
+ async def test_parent_symlink_swap_cannot_redirect_atomic_replace(
+ self,
+ harness,
+ operation,
+ ):
+ from nodes.filesystem import _backend as backend_module
+
+ test_root = _writable_test_path("atomic-parent-race")
+ workspace = test_root / "workspace"
+ parent = workspace / "docs"
+ detached_parent = workspace / "detached-docs"
+ outside = test_root / "outside"
+ parent.mkdir(parents=True)
+ outside.mkdir()
+ target = parent / "victim.txt"
+ outside_target = outside / "victim.txt"
+ target.write_text("before", encoding="utf-8")
+ outside_target.write_text("outside", encoding="utf-8")
+ backend = backend_module.WorkspaceBackend(workspace)
+ real_replace = backend_module._replace_posix_entry
+ swapped = False
+
+ def swap_parent_then_replace(
+ temporary_name,
+ target_name,
+ parent_fd,
+ ):
+ nonlocal swapped
+ if not swapped:
+ parent.rename(detached_parent)
+ os.symlink(outside, parent, target_is_directory=True)
+ swapped = True
+ real_replace(temporary_name, target_name, parent_fd)
+
+ if operation == "write":
+ parameters = {
+ "operation": "write",
+ "file_path": "docs/victim.txt",
+ "content": "written",
+ }
+ expected = "written"
+ else:
+ parameters = {
+ "operation": "edit",
+ "file_path": "docs/victim.txt",
+ "old_string": "before",
+ "new_string": "edited",
+ }
+ expected = "edited"
+
+ try:
+ with _patch_fs_backend(backend), patch(
+ "nodes.filesystem._backend._replace_posix_entry",
+ side_effect=swap_parent_then_replace,
+ ):
+ result = await harness.execute("fileModify", parameters)
+
+ assert swapped is True
+ harness.assert_envelope(result, success=False)
+ assert outside_target.read_text(encoding="utf-8") == "outside"
+ assert (
+ detached_parent / "victim.txt"
+ ).read_text(encoding="utf-8") == expected
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
def test_atomic_write_preserves_lf_bytes(self):
from nodes.filesystem._backend import atomic_write_text
@@ -603,7 +698,7 @@ class TestFileModify:
destination.unlink(missing_ok=True)
async def test_unknown_operation_returns_error(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
with _patch_fs_backend(backend):
result = await harness.execute(
@@ -622,7 +717,7 @@ class TestFileModify:
class TestShell:
async def test_happy_path_returns_stdout_and_exit_code(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.execute = MagicMock(return_value=_FakeExecuteResult(output="hello world\n", exit_code=0, truncated=False))
with _patch_fs_backend(backend):
@@ -647,7 +742,7 @@ class TestShell:
assert "invalid parameters" in result["error"].lower()
async def test_timeout_surfaces_exit_124(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.execute = MagicMock(return_value=_FakeExecuteResult(output="partial...", exit_code=124, truncated=True))
with _patch_fs_backend(backend):
@@ -663,7 +758,7 @@ class TestShell:
assert result["result"]["truncated"] is True
async def test_backend_exception_becomes_error_envelope(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.execute = MagicMock(side_effect=RuntimeError("backend blew up"))
with _patch_fs_backend(backend):
@@ -684,7 +779,7 @@ class TestFsSearch:
_FakeFileInfo({"name": "a.py", "is_dir": False, "size": 10}),
_FakeFileInfo({"name": "sub", "is_dir": True, "size": 0}),
]
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.ls_info = MagicMock(return_value=entries)
with _patch_fs_backend(backend):
@@ -696,7 +791,7 @@ class TestFsSearch:
assert result["result"]["entries"][0]["name"] == "a.py"
async def test_glob_requires_pattern(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
with _patch_fs_backend(backend):
result = await harness.execute("fsSearch", {"mode": "glob", "path": ".", "pattern": ""})
@@ -706,7 +801,7 @@ class TestFsSearch:
async def test_glob_happy_path(self, harness):
matches = [_FakeFileInfo({"name": "x.py", "path": "src/x.py"})]
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.glob_info = MagicMock(return_value=matches)
with _patch_fs_backend(backend):
@@ -722,7 +817,7 @@ class TestFsSearch:
async def test_grep_returns_string_error_as_error_envelope(self, harness):
# Per doc: grep_raw returns a str on error, list on success.
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
backend.grep_raw = MagicMock(return_value="invalid regex: unbalanced parenthesis")
with _patch_fs_backend(backend):
@@ -740,7 +835,7 @@ class TestFsSearch:
assert "invalid regex" in result["error"].lower()
async def test_unknown_mode_returns_error(self, harness):
- backend = MagicMock(name="LocalShellBackend")
+ backend = MagicMock(name="WorkspaceBackend")
with _patch_fs_backend(backend):
result = await harness.execute("fsSearch", {"mode": "teleport", "path": "."})
diff --git a/server/tests/nodes/test_document.py b/server/tests/nodes/test_document.py
index d868bfdd..efa31c0c 100644
--- a/server/tests/nodes/test_document.py
+++ b/server/tests/nodes/test_document.py
@@ -11,8 +11,8 @@ External side-effects are mocked:
- documentParser: each parser library is patched at the module level
(`services.handlers.document.<lib>`), or real HTML is fed through
BeautifulSoup for the happy path.
- - textChunker: pure function, exercised with real LangChain splitters.
- - embeddingGenerator: the embedder class is patched to return canned vectors.
+ - textChunker: pure local function.
+ - embeddingGenerator: native SDK clients are patched to return canned vectors.
- vectorStore: the ChromaDB client is patched to a fake collection.
"""
@@ -20,7 +20,7 @@ from __future__ import annotations
import sys
from types import SimpleNamespace
-from unittest.mock import MagicMock, patch
+from unittest.mock import AsyncMock, MagicMock, patch
import httpx
import pytest
@@ -340,14 +340,15 @@ class TestTextChunker:
class TestEmbeddingGenerator:
async def test_huggingface_happy_path_with_patched_embedder(self, harness):
- # Patch langchain_huggingface at its lazy import site.
fake_embedder = MagicMock()
- fake_embedder.embed_documents = MagicMock(return_value=[[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]])
+ fake_embedder.encode = MagicMock(
+ return_value=[[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]]
+ )
fake_hf_mod = MagicMock()
- fake_hf_mod.HuggingFaceEmbeddings = MagicMock(return_value=fake_embedder)
+ fake_hf_mod.SentenceTransformer = MagicMock(return_value=fake_embedder)
- with patch.dict(sys.modules, {"langchain_huggingface": fake_hf_mod}):
+ with patch.dict(sys.modules, {"sentence_transformers": fake_hf_mod}):
result = await harness.execute(
"embeddingGenerator",
{
@@ -370,9 +371,11 @@ class TestEmbeddingGenerator:
assert payload["dimensions"] == 3
assert payload["provider"] == "huggingface"
# Embedder was constructed with the requested model
- fake_hf_mod.HuggingFaceEmbeddings.assert_called_once()
- kwargs = fake_hf_mod.HuggingFaceEmbeddings.call_args.kwargs
- assert kwargs.get("model_name") == "BAAI/bge-small-en-v1.5"
+ fake_hf_mod.SentenceTransformer.assert_called_once_with(
+ "BAAI/bge-small-en-v1.5"
+ )
+ fake_embedder.encode.assert_called_once()
+ assert fake_embedder.encode.call_args.kwargs["batch_size"] == 32
async def test_empty_chunks_short_circuits(self, harness):
result = await harness.execute("embeddingGenerator", {"chunks": [], "provider": "huggingface"})
@@ -391,13 +394,64 @@ class TestEmbeddingGenerator:
assert "invalid parameters" in result["error"].lower()
async def test_openai_provider_uses_apikey_param(self, harness):
- fake_embedder = MagicMock()
- fake_embedder.embed_documents = MagicMock(return_value=[[0.9, 0.8]])
+ fake_client = MagicMock()
+ fake_client.embeddings.create = AsyncMock(
+ return_value=SimpleNamespace(
+ data=[SimpleNamespace(index=0, embedding=[0.9, 0.8])]
+ )
+ )
+ fake_client.close = AsyncMock()
+
+ fake_openai_mod = MagicMock()
+ fake_openai_mod.AsyncOpenAI = MagicMock(return_value=fake_client)
+
+ with patch.dict(sys.modules, {"openai": fake_openai_mod}):
+ result = await harness.execute(
+ "embeddingGenerator",
+ {
+ "chunks": [{"content": "only"}],
+ "provider": "openai",
+ "model": "text-embedding-3-small",
+ "api_key": "sk-test",
+ },
+ )
+
+ harness.assert_envelope(result, success=True)
+ fake_openai_mod.AsyncOpenAI.assert_called_once_with(api_key="sk-test")
+ fake_client.embeddings.create.assert_awaited_once_with(
+ model="text-embedding-3-small",
+ input=["only"],
+ )
+ fake_client.close.assert_awaited_once()
+
+ async def test_openai_provider_requires_api_key(self, harness):
+ result = await harness.execute(
+ "embeddingGenerator",
+ {
+ "chunks": [{"content": "only"}],
+ "provider": "openai",
+ "model": "text-embedding-3-small",
+ "api_key": "",
+ },
+ )
+ harness.assert_envelope(result, success=False)
+ assert result.get("error_type") == "NodeUserError"
+ assert "api key is required" in result["error"].lower()
+
+ async def test_openai_sdk_failure_is_a_user_safe_node_error(
+ self,
+ harness,
+ ):
+ fake_client = MagicMock()
+ fake_client.embeddings.create = AsyncMock(
+ side_effect=RuntimeError("transport exploded"),
+ )
+ fake_client.close = AsyncMock()
fake_openai_mod = MagicMock()
- fake_openai_mod.OpenAIEmbeddings = MagicMock(return_value=fake_embedder)
+ fake_openai_mod.AsyncOpenAI = MagicMock(return_value=fake_client)
- with patch.dict(sys.modules, {"langchain_openai": fake_openai_mod}):
+ with patch.dict(sys.modules, {"openai": fake_openai_mod}):
result = await harness.execute(
"embeddingGenerator",
{
@@ -408,11 +462,39 @@ class TestEmbeddingGenerator:
},
)
+ harness.assert_envelope(result, success=False)
+ assert result.get("error_type") == "NodeUserError"
+ assert "openai embedding request failed" in result["error"].lower()
+ assert "transport exploded" not in result["error"]
+ fake_client.close.assert_awaited_once()
+
+ async def test_ollama_provider_uses_native_batch_api(self, harness):
+ fake_client = MagicMock()
+ fake_client.embed = AsyncMock(
+ return_value={"embeddings": [[0.2, 0.4], [0.6, 0.8]]}
+ )
+ fake_client.close = AsyncMock()
+ fake_ollama_mod = MagicMock()
+ fake_ollama_mod.AsyncClient = MagicMock(return_value=fake_client)
+
+ with patch.dict(sys.modules, {"ollama": fake_ollama_mod}):
+ result = await harness.execute(
+ "embeddingGenerator",
+ {
+ "chunks": [{"content": "one"}, {"content": "two"}],
+ "provider": "ollama",
+ "model": "nomic-embed-text",
+ "batch_size": 2,
+ },
+ )
+
harness.assert_envelope(result, success=True)
- fake_openai_mod.OpenAIEmbeddings.assert_called_once()
- kwargs = fake_openai_mod.OpenAIEmbeddings.call_args.kwargs
- assert kwargs.get("model") == "text-embedding-3-small"
- assert kwargs.get("api_key") == "sk-test"
+ fake_client.embed.assert_awaited_once_with(
+ model="nomic-embed-text",
+ input=["one", "two"],
+ )
+ fake_client.close.assert_awaited_once()
+ assert result["result"]["dimensions"] == 2
# ============================================================================
diff --git a/server/tests/nodes/test_filesystem_backend.py b/server/tests/nodes/test_filesystem_backend.py
new file mode 100644
index 00000000..a5e550ae
--- /dev/null
+++ b/server/tests/nodes/test_filesystem_backend.py
@@ -0,0 +1,257 @@
+"""Security and behavior tests for the native workspace backend."""
+
+from __future__ import annotations
+
+import os
+import shutil
+from pathlib import Path
+from uuid import uuid4
+
+import pytest
+
+from nodes.filesystem._backend import (
+ EMPTY_CONTENT_WARNING,
+ WorkspaceBackend,
+ perform_string_replacement,
+)
+
+
+def _test_directory() -> Path:
+ path = Path.cwd() / f".test-filesystem-backend-{uuid4().hex}"
+ path.mkdir()
+ return path
+
+
+def test_backend_rejects_traversal_and_symlink_escape():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ outside = test_root / "outside"
+ root.mkdir()
+ outside.mkdir()
+ (outside / "secret.txt").write_text("secret", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ with pytest.raises(ValueError, match="traversal"):
+ backend._resolve_path("../outside/secret.txt")
+
+ link = root / "escape"
+ try:
+ os.symlink(outside, link, target_is_directory=True)
+ except (OSError, NotImplementedError):
+ pytest.skip("creating symlinks is not permitted on this platform")
+ with pytest.raises(ValueError, match="outside workspace"):
+ backend._resolve_path("/escape/secret.txt")
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_backend_read_list_glob_and_literal_grep():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ docs = root / "docs"
+ docs.mkdir(parents=True)
+ (docs / "one.md").write_text(
+ "alpha\nneedle here\n",
+ encoding="utf-8",
+ )
+ (docs / "two.txt").write_text("needle again\n", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ read = backend.read("/docs/one.md", offset=1, limit=1)
+ assert read.error is None
+ assert read.file_data == {
+ "content": "needle here\n",
+ "encoding": "utf-8",
+ }
+
+ entries = backend.ls_info("/docs")
+ assert [entry["path"] for entry in entries] == [
+ "/docs/one.md",
+ "/docs/two.txt",
+ ]
+ assert [
+ match["path"] for match in backend.glob_info("*.md", "/docs")
+ ] == ["/docs/one.md"]
+ assert backend.grep_raw("needle", "/docs", glob="*.md") == [
+ {"path": "/docs/one.md", "line": 2, "text": "needle here"}
+ ]
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_backend_empty_text_preserves_agent_reminder():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ root.mkdir()
+ (root / "empty.txt").write_bytes(b"")
+ (root / "whitespace.txt").write_text(" \n\t", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ for path in ("/empty.txt", "/whitespace.txt"):
+ result = backend.read(path)
+ assert result.error is None
+ assert result.file_data == {
+ "content": EMPTY_CONTENT_WARNING,
+ "encoding": "utf-8",
+ }
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_secure_read_rejects_symlink_substituted_after_resolution():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ outside = test_root / "outside"
+ root.mkdir()
+ outside.mkdir()
+ victim = root / "victim.txt"
+ victim.write_text("safe", encoding="utf-8")
+ secret = outside / "secret.txt"
+ secret.write_text("outside", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ # Resolve while the target is safe, then deterministically substitute
+ # a symlink before the protected open. No timing race is involved.
+ resolved = backend._resolve_path("/victim.txt")
+ victim.unlink()
+ try:
+ os.symlink(secret, victim)
+ except (OSError, NotImplementedError):
+ pytest.skip("creating symlinks is not permitted on this platform")
+
+ with pytest.raises((OSError, ValueError)):
+ backend.read_text_secure(resolved)
+ result = backend.read("/victim.txt")
+ assert result.error is not None
+ assert "outside workspace" in result.error
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_atomic_write_rejects_target_symlink_substituted_after_resolution():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ outside = test_root / "outside"
+ root.mkdir()
+ outside.mkdir()
+ victim = root / "victim.txt"
+ victim.write_text("safe", encoding="utf-8")
+ secret = outside / "secret.txt"
+ secret.write_text("outside", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ resolved = backend._resolve_path("/victim.txt")
+ victim.unlink()
+ try:
+ os.symlink(secret, victim)
+ except (OSError, NotImplementedError):
+ pytest.skip("creating symlinks is not permitted on this platform")
+
+ with pytest.raises((OSError, ValueError)):
+ backend.atomic_write_text(resolved, "replacement")
+ assert secret.read_text(encoding="utf-8") == "outside"
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_atomic_write_rejects_parent_symlink_substituted_after_resolution():
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ parent = root / "docs"
+ detached_parent = root / "detached-docs"
+ outside = test_root / "outside"
+ parent.mkdir(parents=True)
+ outside.mkdir()
+ victim = parent / "victim.txt"
+ victim.write_text("safe", encoding="utf-8")
+ outside_victim = outside / "victim.txt"
+ outside_victim.write_text("outside", encoding="utf-8")
+ backend = WorkspaceBackend(root)
+
+ resolved = backend._resolve_path("/docs/victim.txt")
+ parent.rename(detached_parent)
+ try:
+ os.symlink(outside, parent, target_is_directory=True)
+ except (OSError, NotImplementedError):
+ pytest.skip("creating symlinks is not permitted on this platform")
+
+ with pytest.raises((OSError, ValueError)):
+ backend.atomic_write_text(resolved, "replacement")
+ assert outside_victim.read_text(encoding="utf-8") == "outside"
+ assert (
+ detached_parent / "victim.txt"
+ ).read_text(encoding="utf-8") == "safe"
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_windows_fallback_revalidates_parent_before_writing_payload(
+ monkeypatch,
+):
+ from nodes.filesystem import _backend as backend_module
+
+ test_root = _test_directory()
+ try:
+ root = test_root / "workspace"
+ parent = root / "docs"
+ parent.mkdir(parents=True)
+ target = parent / "victim.txt"
+ target.write_text("safe", encoding="utf-8")
+ real_snapshot = backend_module._windows_parent_snapshot
+ snapshot_calls = 0
+
+ def changed_second_snapshot(*args, **kwargs):
+ nonlocal snapshot_calls
+ snapshot_calls += 1
+ current_parent, identities = real_snapshot(*args, **kwargs)
+ if snapshot_calls == 2:
+ changed = (*identities[:-1], (identities[-1][0], -1))
+ return current_parent, changed
+ return current_parent, identities
+
+ def fail_if_payload_is_written(*args, **kwargs):
+ pytest.fail("payload was written before parent revalidation")
+
+ monkeypatch.setattr(
+ backend_module,
+ "_windows_parent_snapshot",
+ changed_second_snapshot,
+ )
+ monkeypatch.setattr(
+ backend_module,
+ "_write_utf8_to_fd",
+ fail_if_payload_is_written,
+ )
+
+ with pytest.raises(
+ ValueError,
+ match="parent directory changed",
+ ):
+ backend_module._atomic_write_text_windows(
+ root,
+ Path("docs/victim.txt"),
+ "replacement",
+ )
+
+ assert target.read_text(encoding="utf-8") == "safe"
+ assert not list(parent.glob(".opencompany-write-*"))
+ finally:
+ shutil.rmtree(test_root, ignore_errors=True)
+
+
+def test_string_replacement_preserves_ambiguity_contract():
+ assert perform_string_replacement("foo foo", "foo", "bar") == (
+ "Error: String 'foo' appears 2 times in file. Use replace_all=True "
+ "to replace all instances, or provide a more specific string with "
+ "surrounding context."
+ )
+ assert perform_string_replacement("foo foo", "foo", "bar", True) == (
+ "bar bar",
+ 2,
+ )
diff --git a/server/tests/nodes/test_text_splitter.py b/server/tests/nodes/test_text_splitter.py
new file mode 100644
index 00000000..f6b27b51
--- /dev/null
+++ b/server/tests/nodes/test_text_splitter.py
@@ -0,0 +1,32 @@
+"""Golden compatibility tests for the dependency-free text splitters."""
+
+from nodes.document.text_chunker._splitter import split_text
+
+
+def test_recursive_splitter_golden_paragraphs():
+ text = "alpha beta gamma\n\ndelta epsilon zeta\neta theta iota"
+ assert split_text(text, chunk_size=20, overlap=5) == [
+ "alpha beta gamma",
+ "delta epsilon zeta",
+ "eta theta iota",
+ ]
+
+
+def test_recursive_splitter_golden_character_overlap():
+ text = "abcdefghij" * 8
+ assert split_text(text, chunk_size=25, overlap=5) == [
+ "abcdefghijabcdefghijabcde",
+ "abcdefghijabcdefghijabcde",
+ "abcdefghijabcdefghijabcde",
+ "abcdefghijabcdefghij",
+ ]
+
+
+def test_markdown_splitter_golden_sections():
+ text = "# One\n\nparagraph alpha beta\n\n## Two\n\nparagraph gamma delta\n"
+ assert split_text(text, chunk_size=24, overlap=5, markdown=True) == [
+ "# One",
+ "paragraph alpha beta",
+ "## Two",
+ "paragraph gamma delta",
+ ]
diff --git a/server/tests/services/memory/test_jsonl.py b/server/tests/services/memory/test_jsonl.py
index 0dd255b1..b269532a 100644
--- a/server/tests/services/memory/test_jsonl.py
+++ b/server/tests/services/memory/test_jsonl.py
@@ -11,8 +11,7 @@ from __future__ import annotations
import json
-from langchain_core.messages import AIMessage, HumanMessage
-
+from services.llm.protocol import Message
from services.memory.jsonl import append_message, parse_jsonl, trim_window
from services.memory import ( # public re-exports
parse_jsonl as parse_jsonl_reexport,
@@ -35,8 +34,10 @@ def test_parse_jsonl_basic_user_assistant_pair():
text = '{"role": "user", "content": "hi"}\n' '{"role": "assistant", "content": "hello"}\n'
msgs = parse_jsonl(text)
assert len(msgs) == 2
- assert isinstance(msgs[0], HumanMessage) and msgs[0].content == "hi"
- assert isinstance(msgs[1], AIMessage) and msgs[1].content == "hello"
+ assert isinstance(msgs[0], Message)
+ assert (msgs[0].role, msgs[0].content) == ("user", "hi")
+ assert isinstance(msgs[1], Message)
+ assert (msgs[1].role, msgs[1].content) == ("assistant", "hello")
def test_parse_jsonl_skips_unparseable_lines_forward_compat():
@@ -132,7 +133,7 @@ def test_append_message_metadata_round_trips_via_parse_jsonl():
assert obj["session_id"] == "abc-123"
assert obj["model"] == "claude"
# parse_jsonl still returns the message; metadata is preserved on the
- # wire even if it's not surfaced on BaseMessage.
+ # wire even if it isn't surfaced on the normalized Message.
msgs = parse_jsonl(text)
assert msgs[0].content == "blue."
diff --git a/server/tests/services/memory/test_memory_store.py b/server/tests/services/memory/test_memory_store.py
new file mode 100644
index 00000000..754b7693
--- /dev/null
+++ b/server/tests/services/memory/test_memory_store.py
@@ -0,0 +1,31 @@
+from services import memory_store
+from services.llm.protocol import Message
+
+
+def test_native_messages_accept_legacy_and_canonical_roles():
+ memory_store._sessions.clear()
+ try:
+ memory_store.add_message("roles", "human", "legacy user")
+ memory_store.add_message("roles", "ai", "legacy assistant")
+ memory_store.add_message("roles", "user", "native user")
+ memory_store.add_message("roles", "assistant", "native assistant")
+
+ messages = memory_store.get_native_messages("roles")
+
+ assert all(isinstance(message, Message) for message in messages)
+ assert [(message.role, message.content) for message in messages] == [
+ ("user", "legacy user"),
+ ("assistant", "legacy assistant"),
+ ("user", "native user"),
+ ("assistant", "native assistant"),
+ ]
+ wire_messages = memory_store.get_messages("roles")
+ assert [message["role"] for message in wire_messages] == [
+ "user",
+ "assistant",
+ "user",
+ "assistant",
+ ]
+ assert all(message["timestamp"] for message in wire_messages)
+ finally:
+ memory_store._sessions.clear()
diff --git a/server/tests/services/memory/test_state.py b/server/tests/services/memory/test_state.py
index 76b47af6..dde247ff 100644
--- a/server/tests/services/memory/test_state.py
+++ b/server/tests/services/memory/test_state.py
@@ -83,6 +83,38 @@ async def test_unsaved_memory_clear_removes_every_unsaved_todo_node(monkeypatch)
assert service.get(saved)
+async def test_memory_clear_removes_native_vector_store(monkeypatch):
+ from services import todo_service as todo_service_module
+ from services.memory.vector_store import (
+ MemoryVectorStoreKey,
+ _memory_vector_stores,
+ )
+
+ monkeypatch.setattr(todo_service_module, "_service", TodoService())
+ marker = object()
+ composite_key = MemoryVectorStoreKey(
+ session_id="native-session",
+ provider="openai",
+ model="text-embedding-3-small",
+ endpoint="",
+ credential_fingerprint="fingerprint",
+ )
+ _memory_vector_stores["native-session"] = marker
+ _memory_vector_stores[composite_key] = marker
+ try:
+ result = await clear_agent_session_state(
+ session_id="native-session",
+ clear_long_term=True,
+ )
+ assert result["cleared_vector_store"] is True
+ assert "native-session" not in _memory_vector_stores
+ assert composite_key not in _memory_vector_stores
+ finally:
+ _memory_vector_stores.pop("native-session", None)
+ _memory_vector_stores.pop(composite_key, None)
+
+
@pytest.mark.asyncio
async def test_memory_node_clear_uses_atomic_mutation_and_preserves_unrelated_fields(
monkeypatch,
diff --git a/server/tests/services/memory/test_vector_store.py b/server/tests/services/memory/test_vector_store.py
new file mode 100644
index 00000000..2f5924d3
--- /dev/null
+++ b/server/tests/services/memory/test_vector_store.py
@@ -0,0 +1,123 @@
+"""Tests for the dependency-free in-memory semantic index."""
+
+import pytest
+
+from services.memory.vector_store import (
+ NativeMemoryVectorStore,
+ _memory_vector_stores,
+ clear_memory_vector_stores,
+ get_memory_vector_store,
+)
+
+
+class _Embedder:
+ _vectors = {
+ "apple": [1.0, 0.0],
+ "banana": [0.0, 1.0],
+ "fruit": [0.9, 0.1],
+ }
+
+ async def embed_documents(self, texts, *, batch_size=32):
+ return [self._vectors[text] for text in texts]
+
+ async def embed_query(self, text):
+ return self._vectors[text]
+
+ async def aclose(self):
+ return None
+
+
+async def test_similarity_search_returns_cosine_ranked_documents():
+ store = NativeMemoryVectorStore(_Embedder())
+ ids = await store.add_texts(
+ ["apple", "banana"],
+ [{"source": "a"}, {"source": "b"}],
+ )
+
+ assert len(ids) == 2 and ids[0] != ids[1]
+ documents = await store.similarity_search("fruit", k=2)
+ assert [document.page_content for document in documents] == [
+ "apple",
+ "banana",
+ ]
+ assert documents[0].metadata == {"source": "a"}
+
+
+async def test_empty_and_zero_k_operations_are_noops():
+ store = NativeMemoryVectorStore(_Embedder())
+ assert await store.add_texts([]) == []
+ assert await store.similarity_search("fruit", k=0) == []
+
+
+async def test_store_cache_separates_credentials_without_exposing_them(
+ monkeypatch,
+):
+ from services.memory import vector_store as vector_store_module
+
+ class _Auth:
+ def __init__(self, key):
+ self.key = key
+
+ async def get_api_key(self, provider, session_id):
+ assert (provider, session_id) == ("openai", "default")
+ return self.key
+
+ monkeypatch.setattr(
+ vector_store_module,
+ "create_embedder",
+ lambda *args, **kwargs: _Embedder(),
+ )
+ session_id = "credential-cache-test"
+ first = await get_memory_vector_store(
+ session_id,
+ provider="openai",
+ model="text-embedding-3-small",
+ endpoint="https://example.invalid/v1/",
+ auth_service=_Auth("first-secret"),
+ )
+ again = await get_memory_vector_store(
+ session_id,
+ provider="openai",
+ model="text-embedding-3-small",
+ endpoint="https://example.invalid/v1",
+ auth_service=_Auth("first-secret"),
+ )
+ rotated = await get_memory_vector_store(
+ session_id,
+ provider="openai",
+ model="text-embedding-3-small",
+ endpoint="https://example.invalid/v1",
+ auth_service=_Auth("second-secret"),
+ )
+ try:
+ assert first is again
+ assert rotated is not first
+ matching_keys = [
+ key
+ for key in _memory_vector_stores
+ if getattr(key, "session_id", None) == session_id
+ ]
+ assert len(matching_keys) == 2
+ assert "first-secret" not in repr(matching_keys)
+ assert "second-secret" not in repr(matching_keys)
+ finally:
+ assert await clear_memory_vector_stores(session_id) is True
+
+
+async def test_missing_optional_huggingface_extra_is_a_noop(monkeypatch):
+ from services.memory import vector_store as vector_store_module
+
+ monkeypatch.setattr(
+ vector_store_module,
+ "_module_available",
+ lambda _name: False,
+ )
+ store = await get_memory_vector_store(
+ "missing-local-extra",
+ provider="huggingface",
+ )
+ assert store is None
diff --git a/server/tests/services/test_compaction_native.py b/server/tests/services/test_compaction_native.py
new file mode 100644
index 00000000..11894210
--- /dev/null
+++ b/server/tests/services/test_compaction_native.py
@@ -0,0 +1,81 @@
+from types import SimpleNamespace
+from unittest.mock import AsyncMock
+
+import pytest
+
+from services.compaction import CompactionService
+from services.llm.protocol import LLMResponse, Usage
+
+
+class _Database:
+ get_or_create_session_token_state = AsyncMock(
+ return_value={"cumulative_total": 123, "compaction_count": 0}
+ )
+ save_token_metric = AsyncMock()
+ save_compaction_event = AsyncMock()
+ update_session_token_state = AsyncMock()
+
+
+async def test_compaction_uses_chat_unifier_with_native_messages(monkeypatch):
+ _Database.save_token_metric.reset_mock()
+ _Database.update_session_token_state.reset_mock()
+ registry = SimpleNamespace(get_max_output_tokens=lambda *_args: 2048)
+ monkeypatch.setattr(
+ "services.model_registry.get_model_registry",
+ lambda: registry,
+ )
+ unifier = SimpleNamespace(
+ chat=AsyncMock(
+ return_value=LLMResponse(
+ content="compact summary",
+ usage=Usage(
+ input_tokens=40,
+ output_tokens=5,
+ total_tokens=45,
+ cache_read_tokens=2,
+ ),
+ )
+ )
+ )
+ service = CompactionService(
+ _Database(),
+ SimpleNamespace(compaction_enabled=True),
+ )
+ service.set_ai_service(SimpleNamespace(chat_unifier=unifier))
+
+ result = await service.compact_context(
+ session_id="session-1",
+ node_id="agent-1",
+ memory_content="long history " * 20,
+ provider="openai",
+ api_key="secret",
+ model="gpt-test",
+ )
+
+ assert result["success"] is True
+ assert result["usage"] == {
+ "input_tokens": 40,
+ "output_tokens": 5,
+ "total_tokens": 45,
+ "cache_creation_tokens": 0,
+ "cache_read_tokens": 2,
+ "reasoning_tokens": 0,
+ }
+ metric = _Database.save_token_metric.await_args.args[0]
+ assert metric["iteration"] == 0
+ assert metric["total_tokens"] == 45
+ # Metric-only recording must not add the summarizer tokens back into the
+ # active context counter after compaction resets it.
+ reset_updates = [
+ call.args[1]
+ for call in _Database.update_session_token_state.await_args_list
+ ]
+ assert any(update.get("cumulative_total") == 0 for update in reset_updates)
+ assert all(update.get("cumulative_total") != 45 for update in reset_updates)
+ request = unifier.chat.await_args.kwargs
+ assert request["provider"] == "openai"
+ assert request["messages"][0].role == "user"
+ assert "long history" in request["messages"][0].content
+ assert request["sdk_max_retries"] == 0
+ assert request["translate_errors"] is False
diff --git a/server/tests/services/test_native_agent_runtime.py b/server/tests/services/test_native_agent_runtime.py
new file mode 100644
index 00000000..921f0457
--- /dev/null
+++ b/server/tests/services/test_native_agent_runtime.py
@@ -0,0 +1,562 @@
+from __future__ import annotations
+
+import asyncio
+from types import SimpleNamespace
+from typing import Any
+from unittest.mock import AsyncMock
+
+import pytest
+from pydantic import BaseModel
+
+from services.agent_runtime import (
+ AgentToolSpec,
+ run_native_agent_loop,
+ run_native_llm_step,
+)
+from services.llm.protocol import (
+ LLMError,
+ LLMErrorCategory,
+ LLMResponse,
+ Message,
+ ToolCall,
+ ToolDef,
+ Usage,
+)
+from services.plugin import NodeUserError
+
+
+class _Args(BaseModel):
+ value: int
+
+
+class _FakeUnifier:
+ def __init__(self, responses: list[LLMResponse]):
+ self.responses = list(responses)
+ self.calls: list[dict[str, Any]] = []
+
+ async def chat(self, **kwargs):
+ self.calls.append(kwargs)
+ return self.responses.pop(0)
+
+
+class _Auth:
+ async def get_api_key(self, _name):
+ return None
+
+
+class _Database:
+ async def get_user_settings(self, *_args, **_kwargs):
+ return None
+
+
+def test_in_process_compaction_usage_joins_execution_wide_total():
+ from services.ai import _accumulate_compaction_usage
+
+ # Base usage already represents multiple native loop iterations.
+ final_state = {
+ "usage": Usage(
+ input_tokens=30,
+ output_tokens=6,
+ total_tokens=36,
+ cache_read_tokens=4,
+ )
+ }
+ _accumulate_compaction_usage(
+ final_state,
+ {
+ "success": True,
+ "usage": {
+ "input_tokens": 8,
+ "output_tokens": 2,
+ "total_tokens": 10,
+ "reasoning_tokens": 1,
+ },
+ },
+ )
+
+ assert final_state["usage"] == Usage(
+ input_tokens=38,
+ output_tokens=8,
+ total_tokens=46,
+ cache_read_tokens=4,
+ reasoning_tokens=1,
+ )
+
+
+def _tool(name: str) -> AgentToolSpec:
+ return AgentToolSpec(
+ definition=ToolDef(
+ name=name,
+ description=f"Run {name}",
+ parameters=_Args.model_json_schema(),
+ ),
+ args_schema=_Args,
+ execution={"node_id": name},
+ )
+
+
+async def test_native_step_retries_only_structured_retryable_errors():
+ class _RetryUnifier:
+ def __init__(self):
+ self.calls: list[dict[str, Any]] = []
+
+ async def chat(self, **kwargs):
+ self.calls.append(kwargs)
+ if len(self.calls) == 1:
+ raise LLMError(
+ message="rate limited",
+ provider="openai",
+ category=LLMErrorCategory.RATE_LIMIT,
+ retryable=True,
+ retry_after=0,
+ )
+ return LLMResponse(content="recovered")
+
+ unifier = _RetryUnifier()
+ response = await run_native_llm_step(
+ unifier,
+ provider="openai",
+ api_key="test",
+ messages=[Message(role="user", content="go")],
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ )
+
+ assert response.content == "recovered"
+ assert len(unifier.calls) == 2
+ assert all(call["sdk_max_retries"] == 0 for call in unifier.calls)
+ assert all(call["translate_errors"] is False for call in unifier.calls)
+
+
+async def test_native_step_never_surfaces_raw_provider_error_text():
+ raw_message = (
+ "POST https://private-gateway.internal/v1 payload="
+ '{"authorization":"Bearer secret"}'
+ )
+
+ class _FailingUnifier:
+ async def chat(self, **_kwargs):
+ raise LLMError(
+ message=raw_message,
+ provider="openai",
+ category=LLMErrorCategory.INVALID_REQUEST,
+ )
+
+ with pytest.raises(NodeUserError) as caught:
+ await run_native_llm_step(
+ _FailingUnifier(),
+ provider="openai",
+ api_key="test",
+ messages=[Message(role="user", content="go")],
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ explicit_max_retries=0,
+ )
+
+ assert str(caught.value) == (
+ "OpenAI rejected the model request configuration."
+ )
+ assert "private-gateway" not in str(caught.value)
+ assert "Bearer secret" not in str(caught.value)
+
+
+async def test_native_loop_replays_assistant_and_accumulates_usage():
+ first_message = Message(
+ role="assistant",
+ tool_calls=[ToolCall(id="call-1", name="one", args={"value": 3})],
+ provider_state={"provider": "gemini", "payload": {"signature": "abc"}},
+ )
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(
+ tool_calls=first_message.tool_calls,
+ assistant_message=first_message,
+ thinking="plan",
+ usage=Usage(input_tokens=10, output_tokens=2),
+ ),
+ LLMResponse(
+ content="done",
+ thinking="answer",
+ usage=Usage(input_tokens=20, output_tokens=4),
+ ),
+ ]
+ )
+ executed = []
+
+ async def execute(name, args):
+ executed.append((name, args))
+ return {"ok": args["value"]}
+
+ result = await run_native_agent_loop(
+ unifier,
+ provider="gemini",
+ api_key="test",
+ model="gemini-test",
+ temperature=0.2,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ tools=[_tool("one")],
+ tool_executor=execute,
+ )
+
+ assert executed == [("one", {"value": 3})]
+ assert result["messages"][1] is first_message
+ assert result["messages"][2].role == "tool"
+ assert unifier.calls[1]["messages"][1].provider_state["provider"] == "gemini"
+ assert result["usage"] == Usage(
+ input_tokens=30,
+ output_tokens=6,
+ total_tokens=36,
+ )
+ assert result["thinking_content"] == "plan\n\n--- Iteration 2 ---\nanswer"
+
+
+async def test_native_loop_returns_invalid_arguments_to_model_without_execution():
+ invalid = ToolCall(
+ id="bad",
+ name="one",
+ args={},
+ raw_arguments="{",
+ parse_error="Invalid JSON",
+ )
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(tool_calls=[invalid]),
+ LLMResponse(content="recovered"),
+ ]
+ )
+ executed = False
+
+ async def execute(_name, _args):
+ nonlocal executed
+ executed = True
+
+ result = await run_native_agent_loop(
+ unifier,
+ provider="openai",
+ api_key="test",
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ tools=[_tool("one")],
+ tool_executor=execute,
+ )
+
+ assert not executed
+ assert '"error": "Invalid tool arguments"' in result["messages"][2].content
+ assert result["messages"][-1].content == "recovered"
+
+
+async def test_native_loop_returns_unknown_tool_error_without_execution():
+ unknown = ToolCall(
+ id="unknown-1",
+ name="hallucinated_tool",
+ args={"value": 3},
+ )
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(tool_calls=[unknown]),
+ LLMResponse(content="recovered"),
+ ]
+ )
+ executed = False
+
+ async def execute(_name, _args):
+ nonlocal executed
+ executed = True
+ return {"incorrect": "must not run"}
+
+ result = await run_native_agent_loop(
+ unifier,
+ provider="openai",
+ api_key="test",
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ tools=[_tool("one")],
+ tool_executor=execute,
+ )
+
+ assert executed is False
+ assert '"error": "Unknown tool"' in result["messages"][2].content
+ assert "hallucinated_tool" in result["messages"][2].content
+ assert result["messages"][-1].content == "recovered"
+
+
+async def test_native_loop_rebinds_tools_after_canvas_operation():
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(
+ tool_calls=[
+ ToolCall(id="create-1", name="one", args={"value": 1})
+ ]
+ ),
+ LLMResponse(
+ tool_calls=[
+ ToolCall(id="use-2", name="two", args={"value": 2})
+ ]
+ ),
+ LLMResponse(content="done"),
+ ]
+ )
+ executed: list[str] = []
+
+ async def execute(name, _args):
+ executed.append(name)
+ if name == "one":
+ return {
+ "operations": [
+ {"type": "add_node", "node_type": "calculatorTool"}
+ ]
+ }
+ return {"ok": True}
+
+ async def rebind(_operations):
+ return [_tool("two")]
+
+ result = await run_native_agent_loop(
+ unifier,
+ provider="openai",
+ api_key="test",
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ tools=[_tool("one")],
+ tool_executor=execute,
+ rebind_from_operations=rebind,
+ )
+
+ assert executed == ["one", "two"]
+ assert [tool.name for tool in unifier.calls[1]["tools"]] == ["one", "two"]
+ assert result["messages"][-1].content == "done"
+
+
+async def test_native_loop_returns_tool_failure_and_iteration_limit():
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(
+ tool_calls=[
+ ToolCall(id="call-1", name="one", args={"value": 1})
+ ]
+ )
+ ]
+ )
+
+ async def execute(_name, _args):
+ raise RuntimeError("tool exploded")
+
+ result = await run_native_agent_loop(
+ unifier,
+ provider="openai",
+ api_key="test",
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ tools=[_tool("one")],
+ tool_executor=execute,
+ max_iterations=1,
+ )
+
+ assert '"error": "tool exploded"' in result["messages"][-2].content
+ assert result["truncated"] is True
+ assert "Recursion limit reached" in result["messages"][-1].content
+
+
+async def test_native_loop_propagates_cancellation():
+ started = asyncio.Event()
+
+ class _BlockingUnifier:
+ async def chat(self, **_kwargs):
+ started.set()
+ await asyncio.Event().wait()
+
+ task = asyncio.create_task(
+ run_native_agent_loop(
+ _BlockingUnifier(),
+ provider="openai",
+ api_key="test",
+ model="gpt-test",
+ temperature=0,
+ max_tokens=100,
+ initial_messages=[Message(role="user", content="go")],
+ )
+ )
+ await started.wait()
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+
+
[email protected]("method_name", ["execute_agent", "execute_chat_agent"])
+async def test_ai_service_agent_entrypoints_use_native_unifier(method_name):
+ from services.ai import AIService
+
+ unifier = _FakeUnifier(
+ [
+ LLMResponse(
+ content="native answer",
+ usage=Usage(input_tokens=3, output_tokens=2),
+ )
+ ]
+ )
+ database = _Database()
+ service = AIService(
+ auth_service=_Auth(),
+ database=database,
+ cache=None,
+ settings=object(),
+ chat_unifier=unifier,
+ )
+
+ result = await getattr(service, method_name)(
+ node_id="agent-1",
+ parameters={
+ "provider": "openai",
+ "model": "gpt-4o-mini",
+ "api_key": "secret",
+ "prompt": "hello",
+ "system_message": "be useful",
+ "temperature": 0.2,
+ },
+ database=database,
+ )
+
+ assert result["success"] is True, result
+ assert result["result"]["response"] == "native answer"
+ assert unifier.calls[0]["provider"] == "openai"
+ assert unifier.calls[0]["messages"][-1].role == "user"
+
+
+async def test_chat_agent_connected_tool_uses_agent_tool_spec_schema():
+ from services.ai import AIService
+
+ unifier = _FakeUnifier([LLMResponse(content="native answer")])
+ database = _Database()
+ service = AIService(
+ auth_service=_Auth(),
+ database=database,
+ cache=None,
+ settings=object(),
+ chat_unifier=unifier,
+ )
+ service._build_tool_from_node = AsyncMock(
+ return_value=(
+ _tool("one"),
+ {
+ "node_id": "tool-1",
+ "node_type": "testTool",
+ "label": "Test tool",
+ },
+ )
+ )
+
+ result = await service.execute_chat_agent(
+ node_id="chat-1",
+ parameters={
+ "provider": "openai",
+ "model": "gpt-4o-mini",
+ "api_key": "secret",
+ "prompt": "hello",
+ },
+ tool_data=[
+ {
+ "node_id": "tool-1",
+ "node_type": "testTool",
+ "label": "Test tool",
+ "parameters": {},
+ }
+ ],
+ context={"execution_id": "run-1"},
+ database=database,
+ )
+
+ assert result["success"] is True, result
+ assert unifier.calls[0]["tools"][0].parameters == _Args.model_json_schema()
+
+
[email protected]("method_name", ["execute_agent", "execute_chat_agent"])
+async def test_long_term_retrieval_preserves_execution_context_on_save(
+ method_name,
+ monkeypatch,
+):
+ import services.ai as ai_module
+ import services.memory.runtime as memory_runtime
+ from services.ai import AIService
+
+ store = SimpleNamespace(
+ similarity_search=AsyncMock(return_value=[
+ SimpleNamespace(page_content="remembered detail")
+ ])
+ )
+ monkeypatch.setattr(
+ ai_module,
+ "_get_memory_vector_store",
+ AsyncMock(return_value=store),
+ )
+ append_turns = AsyncMock(return_value=({}, [], True))
+ monkeypatch.setattr(
+ memory_runtime,
+ "append_memory_turns_atomic",
+ append_turns,
+ )
+
+ unifier = _FakeUnifier([LLMResponse(content="native answer")])
+ database = _Database()
+ service = AIService(
+ auth_service=_Auth(),
+ database=database,
+ cache=None,
+ settings=object(),
+ chat_unifier=unifier,
+ )
+ service._track_token_usage = AsyncMock(return_value=None)
+ memory_data = {
+ "node_id": "memory-1",
+ "session_id": "session-1",
+ "memory_content": "# Conversation History\n\n*No messages yet.*\n",
+ "window_size": 10,
+ "long_term_enabled": True,
+ "retrieval_count": 1,
+ }
+
+ result = await getattr(service, method_name)(
+ node_id="agent-1",
+ parameters={
+ "provider": "openai",
+ "model": "gpt-4o-mini",
+ "api_key": "secret",
+ "prompt": "hello",
+ },
+ memory_data=memory_data,
+ context={
+ "execution_id": "run-1",
+ "root_execution_id": "root-1",
+ },
+ database=database,
+ )
+
+ assert result["success"] is True, result
+ assert any(
+ "remembered detail" in message.content
+ for message in unifier.calls[0]["messages"]
+ )
+ assert append_turns.await_args.kwargs["mutation_id"].startswith(
+ "ai-memory:" if method_name == "execute_agent" else "chat-memory:"
+ )
diff --git a/server/tests/temporal/fixtures/agent_legacy_pre_native_history.json.zlib.b64 b/server/tests/temporal/fixtures/agent_legacy_pre_native_history.json.zlib.b64
new file mode 100644
index 00000000..791c7bf7
--- /dev/null
+++ b/server/tests/temporal/fixtures/agent_legacy_pre_native_history.json.zlib.b64
@@ -0,0 +1 @@
+eNrtXW1zosoS/n5+xZafjymGF5WtOlU3GjW60WzUCHLrlsXLGIgDUoCvW/vfTw+YCGqiJnA3uzFVu4kw0wxN091PzzPjj7++fMnhGXYCP/f1y3/h05cvP8L/n443DDiRQ7m/E0d7lo3pcZZhC3mmmGe5Hip8ZdiviL8oCoKy3Xzphs2r/Wq7N+wNvleH0m3nW+3mVhpW5Wrlvte4bQ+7vctOr3q16TufeOMRmcyrC6xPA2vidAPVC7BRpTIvg8CztGmA6cifxhzrtL7m5gycc9Ro2JcPIEBaN8w9t/j590ZMoPrjuymevihDpTLyjhpYM5z3sEvUZf5BDfB+cZbjToNtUa66JBPV2Kj+6edH4hO0tHGgGmqgbkl40rGjTwzLeQiH5bRnmiXOtXrN1R7++Se31Tw2qHXvtdwcXjanGtshMkvGjceJNZAMojl3gW73qTShYc0to95fGTWRaLbhava9dWs1fUUyzAE3tkZ321f7Gfv0v71q2XnC1LAmoaZyHBK4AgM/fm5Pj870+LY9eJaxxijZauJZD5ajkucxgOjI6g2xwLF6CeUZTmXyPMfyeVEQC/nRiNN5XURFLOC4JMsAk7CCJe3Ll4os859WtxFvMLI8P3j/ddQgwLZL7wVtC5dit1xW9fFkNKJXSN6x61lw0+E4fzw9o+j3+tnscwFsdi6gd9n9NuxWrqtX9zf73n96M13dxMaUHHj5U35rfepvepMKmfj4ZQOKPY5jlcllrcyXXGmoyiO8qJ9Q97YFHGPrpuUHE2/ZtVa4vIzkw2tayB2rIv4UFRVOV1HltvX9pvqikioT2yU4ezVp4Hq8ZcXE+tif2rRZEUFDUeUNVmdHpSLDC5pWEHlmVBCK2DC0eG/fGLf2h4acPvHwvY+NGlEfdiMMmwwDXPIjijvu+Cl6wfb6VaJGP/FUkneXgTlxctvt+tjzwcuFL8wFW7xg979jENqwB8Erfh9HOyUhdSNpXXa+VTvDTrVy27mKG4etemPsdTDo1ThgFVHTJz3RBzF01UA340/OgNu1iL+bEkC7/N5Q/3K68GVfUvBqynBi0rAnbUgmDq5SaRQads1RJIG5kTszjZ0HA7bmaxXkKvX+1KgTBncR27IafsPuEP26SeA8o0h31q2tmNp1m+wmEMkUIp5ExM/8fCHkJ1/hJ8d7rGkVUjet++/daqe3cUPd6mWncj287PU6jfJ9r9rdiJi6Pvaeg3kXq55ubqztuPz3RQ3E/ceW5G1ztBwDL8CHWJgY/q5F9tYuoGKqzgPevPAn5q/BWlddti8YrLhUqgtX5+7+yf2dVa4rLZumwoJZVhCjsaJ/I7VNrT4PVKkDOW/HNa7HgfHYsORXc9qYCR5rVcWzw/qQDsvUWT/QpNqUoh7dQmNFWhAFHJQqidMbebCg6Acc2RzQ0ONA7hDaX7Frvs72xY/jtEpnp/U5nNZAbns3EjI1asJyIwDTDIE4xFmQDnG2S+PsMfG4aaTq4MTULfCy0mv0G73BizBR1QHNQY59PEx86hEN2QUcqHo4v3ZXCWj3JPqVYlIIKS/WQoZrIRczlHZdKZFfj8HIaetQJ3f31fvq8FujfTVs33Zalzf7L21i1cBemF+fK1OHKlNPyG4X/e+vNW3ah/D6cPv9pYVCshU8Mi/QsBqX93KN66Dn9HDgLb9PiKUvd52mFVgqaTiAxmYqCVGbn7Q5LaonVSZ4NLJ0Cy4BrdgLJtHIVheWPbXjcjhmS9C6zWVUN6FGlsCfOQdqY3SkqkZw1fMmHn39dm0x154YFOJ6YZstv5trOHB5K1JF7uDjflNJDDFZO7vtMk7C1b2xjCOeVJ94Q3ELnTRbUCierJY9pZu4Yo4r3XjYn5KP7/hMvU5GKuQY1PFp9b4zkNrCjUzRM+Qc9f4YojuBvCMI24AjhOOBXq89hpkAERkVkLbOmjOtTpG2gADmQGaATFwTXXCaS0hmXY0VViB/cVNpBgPZHBl10YPMITzWurqk7YkmiUvcFx+hLUOzCjoeej1N6gca1xRkVpjBeEYqRfRwhspSJDTTHTIyWDJV6iInc20XU+f8WJ3T89SpazQRd6J7gGtA4j2nfQ86fM2Gtn2RwXKZRG1IOcqOIPNxykuNU2imswz1xpUhMNRc2g8S9pXOhrqJQIDThmu3QQd8Qr7OkRXoJ5BZRM9D5hUGlYrSLSOQYRvROEEXaK7ITdO47i+VbqPQqtBgZEIQCuCfMYLnMVVs4jQe3akhLXzah0JOfdkoSFBJbjj9FfQfQUVkpsnlUK8wDsaQm9NGHTI30Bf04UDHnmIvZgZaj9s24NmVJ9QW9PrCxN2m+NYg96J78GM+Zsfnveg+jnYUbOqO4hPPKSAua21mMamA3j+rgEpC8XibO2leoch8nHkF9AsnFlIr2CMhdfVnjg/ZtyFCzYNEQlf9YOh6kwdIN/wzKEw3NyK43n80ovxlnXc0x7e9waplt/ioAkKYMDaHVZEoNkORj4e8YnPcadF8BNrG85fnSstSkcv+QB7TGDxX67UVjfGQM0AFpgnFQcEJcwWI3YZNxlCJoX1nOlRYBlLYZwmxnpHZPq9IbbTOydbx+05oP7aWg9XDIso3OjSHoDmXpcgmGXD9Z/m0vqSyiq9xxtP4HN2uzdXY+H4jkMulDXLRGeVmiHILWbvsLFBuLNBkBXNPmsspojPMfc2VaxJhAG4B3HMZ3emT97izl+1gD54ppIhnSqlbxGfGM2LW2swEz5TejWdYgTsez7An1SCL3AfCM6U/AM+wKHX1Z45nuDOeOeOZJJ6xF77WF2m9c54WnjljlgOYRTxjlswwC8tm7ZazwCyxYJIRZmFPqhEXhTNm+RWYJWkHu5iFZdPDLCyfukV8YszCCllrM5OFHXwKKzvE45d2sKdVlIofaG0H/ydglmLq6s8csxBi5/0Au3n0NvCCw0V3eAhyhlTOGbr8YppKCFu4zgw+TykVotW7DCkhitxehRxsSo/gfOBc15gBQBS4RoJ6cmffPxhymyhOHygUlEbREfR6SNFYYbkN7Rhr1F0UYbEJwKKqdev4lBoDxNeQIgJTOE1CKRuaVR6DXLIlY2JQcq3VFOUuHWtES4HjALPaDBBvKbTyD0Cl9TjQCCgyQDkBMi6FWuMmacA0jyIpSAupKkCBAQoLQCNK+aBUFqoHRq2TqUppMyik3NiqpG/oOw4l9C5GT7qREBJ/G35j2jiKFX49jkLMYSCFfk8gVco6VmQCpIqZA6nTitOlM5B6NXgA/47Of8PSq4SzhHNj4CeaALDAkcMVqLN0orrYHqdtqsCBA+eZCBTAO5wqYe2M8vHCvuBMBXPtfCfwtxfOs2/4eSh2fVdzygiI5cDZC0hUW6OyRGRcbx+viakBwOIBAFhKDwByTOqW/JkX9qOstZkFAOSYdwNAHiH2eJs7qTxX+kAkvBM19TEBIMelrv7MASC1OJyH95SioTdBwFDCMJJwhn//5+VZ8PcKoisTRfLd2SMNYJxWeTPUghml9pyuYohms55Z88fPVtX7JsyAJcayYfHHoVc/hKfRfZCpft1nEjCWQltg7u0clxHNTFYU3tIlxU+6el6NYJ1nxbbi53lWLG0wx/FZ+/0swBzHZQ3muJOq9qUzk++1UAAkAqj40Ql/p7M0pH4qi5KSNrALiDg+RUBUSN0aPjMgKmatzUwAUeH9gKjE88fb3EllttIHYvGdqKkPCojE1NWfOSDSCSRb+fAYzvtjixD/jbiI9r0IxZ1hUcqFzZB0VxMPwqPNgp8NNFCuxmxr1Xy8vapmvgD4t4cNxTNsyAw28EzW3jET2JD5Phf8SbXeEn+GDa+4ykcgDZjgElMn03EHNjTgU9zQgGdTt4hPDB14LmttZgEd+PdvaCDw6HjowJ9W1Cl8HOjA/wkbGvBC6urPHDoUzwuA/pQFQN20FgC16eZDvhLtbPr6Bkq2OKM77R0ca7hJQgfR6Q/4Gw1YutFS64xlDmAZ/ryZQYZYppC1u84Cy/CZb2bAn1amPfPZfsnCIP7AZgZ8ipsZ8KXULeIzYxkxa21mgmXev5lBgRGP375cOKnSJH4gXhj/J2xmIKDs1L/5mrdXnsHzV2P9UeGAkpOQdn0XrkHZrfQ/045f2w4VvndAZOm3U+gWzZbFOdCbge5Ms2zImLkyZP1tkqA7O7CdKNcZHTGzAFTpWrA1ls36k2WMECWvt0GF+wDK9Byy//hWsUvoN4Nj28cXI5nStDcI4K3MgIOZs7jlaf6ikn7+CxNCujk=
diff --git a/server/tests/temporal/test_agent_workflow.py b/server/tests/temporal/test_agent_workflow.py
index 8cf77ae0..9dedaf9e 100644
--- a/server/tests/temporal/test_agent_workflow.py
+++ b/server/tests/temporal/test_agent_workflow.py
@@ -423,7 +423,6 @@ class TestPayloadShape:
"node_type",
"provider",
"model",
- "api_key",
"system_message",
"user_prompt",
"tools",
diff --git a/server/tests/temporal/test_agent_workflow_replay.py b/server/tests/temporal/test_agent_workflow_replay.py
new file mode 100644
index 00000000..77677a2c
--- /dev/null
+++ b/server/tests/temporal/test_agent_workflow_replay.py
@@ -0,0 +1,339 @@
+"""SDK-level replay gates for the native agent-engine cutover.
+
+Unlike the fast unit tests in ``test_native_llm_cutover.py``, these tests run
+the real workflow worker against Temporal's time-skipping test server and feed
+the recorded event history through ``Replayer``. Provider calls remain fully
+stubbed activities, so the gate needs no credentials or network API access.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import base64
+import subprocess
+import sys
+import zlib
+from datetime import timedelta
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+from temporalio import activity
+from temporalio.api.enums.v1 import EventType
+from temporalio.client import WorkflowHistory
+from temporalio.testing import WorkflowEnvironment
+from temporalio.worker import Replayer, Worker
+
+from services.llm.protocol import Message, message_to_wire
+from services.temporal.agent_workflow import AgentWorkflow
+
+
+TASK_QUEUE = "agent-native-replay-gate"
+FIXTURES_DIR = Path(__file__).with_name("fixtures")
+
+
+def _prepared_payload(*, engine: str) -> dict[str, Any]:
+ payload: dict[str, Any] = {
+ "node_id": "agent-replay",
+ "node_type": "aiAgent",
+ "workflow_id": "graph-replay",
+ "session_id": "session-replay",
+ "provider": "openai",
+ "model": "test-model",
+ "max_tokens": 100,
+ "temperature": 0,
+ "system_message": "Be useful",
+ "user_prompt": "return done",
+ "tools": [],
+ "memory_node_id": "",
+ "memory_content": "",
+ "memory_window_size": 10,
+ "max_iterations": 1,
+ "thinking_config": None,
+ "compaction_threshold": None,
+ }
+ if engine == "native":
+ payload.update(
+ {
+ "llm_engine": "native",
+ "message_wire_version": 2,
+ }
+ )
+ elif engine == "langchain":
+ payload.update(
+ {
+ "llm_engine": "langchain",
+ "message_wire_version": 1,
+ }
+ )
+ elif engine == "legacy":
+ # Pre-cutover histories included the resolved secret in prepare
+ # output. The compatibility branch must continue accepting that
+ # recorded shape, while native histories must never persist it.
+ payload["api_key"] = "legacy-recorded-test-key"
+ else:
+ raise AssertionError(f"Unexpected test engine {engine!r}")
+ return payload
+
+
[email protected](name="agent.prepare_payload.v1")
+async def _prepare_payload(context: dict[str, Any]) -> dict[str, Any]:
+ return _prepared_payload(engine=str(context["test_engine"]))
+
+
[email protected](name="agent.broadcast_progress.v1")
+async def _broadcast_progress(_payload: dict[str, Any]) -> dict[str, Any]:
+ return {"emitted": True}
+
+
[email protected](name="agent.execute_llm_step.v1")
+async def _execute_llm_step(payload: dict[str, Any]) -> dict[str, Any]:
+ if payload.get("llm_engine") == "native":
+ assert payload["message_wire_version"] == 2
+ assert "api_key" not in payload
+ assert "tool_data" not in payload
+ assert all(message["version"] == 2 for message in payload["messages"])
+ assert activity.info().heartbeat_timeout == timedelta(minutes=1)
+ assistant = message_to_wire(
+ Message(role="assistant", content="done")
+ )
+ elif payload.get("llm_engine") == "langchain":
+ # New runs under the emergency switch use the legacy adapter without
+ # persisting a credential in either activity result or input.
+ assert payload["message_wire_version"] == 1
+ assert payload["node_id"] == "agent-replay"
+ assert "api_key" not in payload
+ assert "tools" not in payload
+ assert "tool_data" in payload
+ assert all("type" in message for message in payload["messages"])
+ assert activity.info().heartbeat_timeout == timedelta(minutes=1)
+ assistant = {"type": "ai", "data": {"content": "done"}}
+ else:
+ # This is the pre-cutover contract: no engine marker, LangChain's
+ # canonical message dictionaries, and no heartbeat timeout.
+ assert "llm_engine" not in payload
+ assert "message_wire_version" not in payload
+ assert payload["api_key"] == "legacy-recorded-test-key"
+ assert "tools" not in payload
+ assert "tool_data" in payload
+ assert all("type" in message for message in payload["messages"])
+ assert activity.info().heartbeat_timeout is None
+ assistant = {"type": "ai", "data": {"content": "done"}}
+
+ return {
+ "kind": "final",
+ "assistant_message": assistant,
+ "content": "done",
+ "thinking": None,
+ "usage": {"input_tokens": 2, "output_tokens": 1},
+ }
+
+
[email protected](name="agent.store_output.v1")
+async def _store_output(_payload: dict[str, Any]) -> dict[str, Any]:
+ return {"stored": True}
+
+
[email protected](name="agent.skill.clear.v1")
+async def _clear_skills(_payload: dict[str, Any]) -> dict[str, Any]:
+ return {"cleared": True}
+
+
+_TEST_ACTIVITIES = [
+ _prepare_payload,
+ _broadcast_progress,
+ _execute_llm_step,
+ _store_output,
+ _clear_skills,
+]
+
+
+async def _run_and_capture(
+ environment: WorkflowEnvironment,
+ *,
+ engine: str,
+) -> tuple[dict[str, Any], WorkflowHistory]:
+ workflow_id = f"agent-{engine}-replay-{uuid4()}"
+ handle = await environment.client.start_workflow(
+ "AgentWorkflow",
+ {"node_id": "agent-replay", "test_engine": engine},
+ id=workflow_id,
+ task_queue=TASK_QUEUE,
+ )
+ result = await handle.result()
+ return result, await handle.fetch_history()
+
+
+def _scheduled_activities(history: WorkflowHistory) -> list[Any]:
+ return [
+ event.activity_task_scheduled_event_attributes
+ for event in history.events
+ if event.event_type == EventType.EVENT_TYPE_ACTIVITY_TASK_SCHEDULED
+ ]
+
+
+def _load_captured_history(name: str, workflow_id: str) -> WorkflowHistory:
+ """Load a frozen Temporal JSON history stored as compressed base64."""
+
+ encoded = (FIXTURES_DIR / name).read_bytes()
+ history_json = zlib.decompress(base64.b64decode(encoded)).decode("utf-8")
+ return WorkflowHistory.from_json(workflow_id, history_json)
+
+
+async def _run_replay_gate() -> None:
+ """Execute all engine branches and replay their serialized histories."""
+
+ async with await WorkflowEnvironment.start_time_skipping() as environment:
+ async with Worker(
+ environment.client,
+ task_queue=TASK_QUEUE,
+ workflows=[AgentWorkflow],
+ activities=_TEST_ACTIVITIES,
+ ):
+ native_result, native_history = await _run_and_capture(
+ environment,
+ engine="native",
+ )
+ emergency_result, emergency_history = await _run_and_capture(
+ environment,
+ engine="langchain",
+ )
+ legacy_result, legacy_history = await _run_and_capture(
+ environment,
+ engine="legacy",
+ )
+
+ assert native_result["success"] is True
+ assert native_result["result"]["response"] == "done"
+ assert emergency_result["success"] is True
+ assert emergency_result["result"]["response"] == "done"
+ assert legacy_result["success"] is True
+ assert legacy_result["result"]["response"] == "done"
+
+ expected_order = [
+ "agent.prepare_payload.v1",
+ "agent.broadcast_progress.v1",
+ "agent.broadcast_progress.v1",
+ "agent.execute_llm_step.v1",
+ "agent.store_output.v1",
+ "agent.skill.clear.v1",
+ "agent.broadcast_progress.v1",
+ ]
+ native_scheduled = _scheduled_activities(native_history)
+ emergency_scheduled = _scheduled_activities(emergency_history)
+ legacy_scheduled = _scheduled_activities(legacy_history)
+ assert [
+ item.activity_type.name for item in native_scheduled
+ ] == expected_order
+ assert [
+ item.activity_type.name for item in emergency_scheduled
+ ] == expected_order
+ assert [
+ item.activity_type.name for item in legacy_scheduled
+ ] == expected_order
+
+ native_llm = native_scheduled[3]
+ emergency_llm = emergency_scheduled[3]
+ legacy_llm = legacy_scheduled[3]
+ assert (
+ native_llm.heartbeat_timeout.ToTimedelta()
+ == timedelta(minutes=1)
+ )
+ # Temporal records the omitted optional duration as an explicit zero
+ # value in history; activity.info() still correctly exposes None.
+ assert emergency_llm.heartbeat_timeout.ToTimedelta() == timedelta(
+ minutes=1
+ )
+ assert legacy_llm.heartbeat_timeout.ToTimedelta() == timedelta(0)
+
+ [native_input] = await environment.client.data_converter.decode(
+ native_llm.input.payloads
+ )
+ [emergency_input] = await environment.client.data_converter.decode(
+ emergency_llm.input.payloads
+ )
+ [legacy_input] = await environment.client.data_converter.decode(
+ legacy_llm.input.payloads
+ )
+ assert native_input["llm_engine"] == "native"
+ assert native_input["message_wire_version"] == 2
+ assert "api_key" not in native_input
+ assert "tool_data" not in native_input
+ assert emergency_input["llm_engine"] == "langchain"
+ assert emergency_input["message_wire_version"] == 1
+ assert "api_key" not in emergency_input
+ assert "tool_data" in emergency_input
+ assert "llm_engine" not in legacy_input
+ assert "message_wire_version" not in legacy_input
+ assert "tool_data" in legacy_input
+ assert legacy_input["api_key"] == "legacy-recorded-test-key"
+
+ prepare_results = []
+ for history in (native_history, emergency_history, legacy_history):
+ completed = next(
+ event.activity_task_completed_event_attributes
+ for event in history.events
+ if event.event_type
+ == EventType.EVENT_TYPE_ACTIVITY_TASK_COMPLETED
+ )
+ [prepared] = await environment.client.data_converter.decode(
+ completed.result.payloads
+ )
+ prepare_results.append(prepared)
+
+ native_prepared, emergency_prepared, legacy_prepared = prepare_results
+ assert native_prepared["llm_engine"] == "native"
+ assert "api_key" not in native_prepared
+ assert emergency_prepared["llm_engine"] == "langchain"
+ assert "api_key" not in emergency_prepared
+ assert "llm_engine" not in legacy_prepared
+ assert legacy_prepared["api_key"] == "legacy-recorded-test-key"
+
+ replayer = Replayer(workflows=[AgentWorkflow])
+ live_histories = (
+ native_history,
+ emergency_history,
+ legacy_history,
+ )
+ for history in live_histories:
+ # JSON round-trip makes this a captured-history gate rather
+ # than replaying the live protobuf object in memory.
+ captured = WorkflowHistory.from_json(
+ history.workflow_id,
+ history.to_json(),
+ )
+ replay = await replayer.replay_workflow(captured)
+ assert replay.replay_failure is None
+
+ # This fixture is frozen rather than generated from the workflow
+ # implementation under test, protecting already-recorded histories
+ # whose prepare result had no engine/version marker.
+ frozen_legacy = _load_captured_history(
+ "agent_legacy_pre_native_history.json.zlib.b64",
+ "agent-legacy-pre-native-captured-v1",
+ )
+ replay = await replayer.replay_workflow(frozen_legacy)
+ assert replay.replay_failure is None
+
+
+def test_all_cutover_history_shapes_execute_and_replay() -> None:
+ """Run the SDK gate in a clean process with valid Windows I/O handles."""
+
+ completed = subprocess.run(
+ [
+ sys.executable,
+ "-m",
+ "tests.temporal.test_agent_workflow_replay",
+ ],
+ cwd=Path(__file__).parents[2],
+ stdin=subprocess.DEVNULL,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ timeout=90,
+ check=False,
+ )
+ assert completed.returncode == 0, "Temporal replay subprocess failed"
+
+
+if __name__ == "__main__":
+ asyncio.run(_run_replay_gate())
diff --git a/server/tests/temporal/test_native_llm_cutover.py b/server/tests/temporal/test_native_llm_cutover.py
new file mode 100644
index 00000000..da64ca2c
--- /dev/null
+++ b/server/tests/temporal/test_native_llm_cutover.py
@@ -0,0 +1,760 @@
+"""Temporal compatibility tests for the native agent LLM cutover."""
+
+from __future__ import annotations
+
+import asyncio
+from datetime import timedelta
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, MagicMock
+
+import pytest
+
+
+def _tool() -> dict:
+ definition = {
+ "name": "write_todos",
+ "description": "Write a todo list",
+ "parameters": {
+ "type": "object",
+ "properties": {"todos": {"type": "array"}},
+ },
+ }
+ return {
+ "name": "write_todos",
+ "definition": definition,
+ "node_type": "writeTodos",
+ "version": 1,
+ "task_queue": "write-todos",
+ "tool_node_id": "todo-1",
+ "parameters": {},
+ "tool_info": {
+ "node_id": "todo-1",
+ "node_type": "writeTodos",
+ "label": "Todos",
+ "parameters": {},
+ },
+ }
+
+
+def _payload(*, native: bool) -> dict:
+ value = {
+ "node_id": "agent-1",
+ "node_type": "aiAgent",
+ "workflow_id": "graph-1",
+ "session_id": "session-1",
+ "provider": "openai",
+ "model": "test-model",
+ "max_tokens": 100,
+ "temperature": 0,
+ "system_message": "Be useful",
+ "user_prompt": "do the work",
+ "tools": [_tool()],
+ "memory_node_id": "",
+ "memory_content": "",
+ "memory_window_size": 10,
+ "max_iterations": 2,
+ "thinking_config": None,
+ "compaction_threshold": None,
+ }
+ if native:
+ value.update({"llm_engine": "native", "message_wire_version": 2})
+ else:
+ value["api_key"] = "test-key"
+ return value
+
+
+def _emergency_langchain_payload() -> dict:
+ value = _payload(native=False)
+ value.pop("api_key")
+ value.update({"llm_engine": "langchain", "message_wire_version": 1})
+ return value
+
+
+def patched_workflow(monkeypatch):
+ import services.temporal.agent_workflow as workflow_module
+
+ temporal_workflow = workflow_module.workflow
+ monkeypatch.setattr(temporal_workflow, "logger", MagicMock())
+ monkeypatch.setattr(temporal_workflow, "patched", lambda _patch_id: True)
+ monkeypatch.setattr(
+ temporal_workflow,
+ "info",
+ lambda: SimpleNamespace(
+ workflow_id="agent-run-1",
+ run_id="run-id-12345678",
+ ),
+ )
+ monkeypatch.setattr(
+ workflow_module,
+ "get_node_class",
+ lambda _node_type: SimpleNamespace(needs_canvas=False),
+ )
+ return temporal_workflow
+
+
+class TestEngineSelection:
+ def test_new_runs_default_to_native(self, monkeypatch):
+ from services.temporal.agent_activities import _configured_llm_engine
+
+ monkeypatch.delenv("AGENT_LLM_ENGINE", raising=False)
+ assert _configured_llm_engine() == "native"
+
+ def test_emergency_switch_is_explicit_and_validated(self, monkeypatch):
+ from services.temporal.agent_activities import _configured_llm_engine
+
+ monkeypatch.setenv("AGENT_LLM_ENGINE", " LangChain ")
+ assert _configured_llm_engine() == "langchain"
+
+ monkeypatch.setenv("AGENT_LLM_ENGINE", "surprise")
+ with pytest.raises(ValueError, match="AGENT_LLM_ENGINE"):
+ _configured_llm_engine()
+
+
+class TestWorkflowBranchContract:
+ @pytest.mark.asyncio
+ async def test_native_history_uses_v2_messages_and_heartbeat(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ llm_calls: list[tuple[dict, dict]] = []
+
+ async def fake_execute_activity(name, *, args, **kwargs):
+ if name == "agent.prepare_payload.v1":
+ return _payload(native=True)
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ llm_calls.append((args[0], kwargs))
+ return {
+ "kind": "final",
+ "content": "done",
+ "thinking": None,
+ "usage": {"input_tokens": 3, "output_tokens": 1},
+ }
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert result["success"] is True
+ assert len(llm_calls) == 1
+ llm_payload, options = llm_calls[0]
+ assert llm_payload["llm_engine"] == "native"
+ assert llm_payload["message_wire_version"] == 2
+ assert "api_key" not in llm_payload
+ assert "tool_data" not in llm_payload
+ assert llm_payload["tools"] == [_tool()["definition"]]
+ assert [(m["version"], m["role"]) for m in llm_payload["messages"]] == [
+ (2, "system"),
+ (2, "user"),
+ ]
+ assert options["heartbeat_timeout"] == timedelta(minutes=1)
+ assert options["retry_policy"].maximum_attempts == 1
+
+ @pytest.mark.asyncio
+ async def test_compaction_preserves_cumulative_and_summarizer_usage(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ prepared = _payload(native=True)
+ prepared.update(
+ {
+ "memory_content": "prior history",
+ "compaction_threshold": 5,
+ }
+ )
+ llm_turn = 0
+ compaction_calls = 0
+
+ async def fake_execute_activity(name, *, args, **_kwargs):
+ nonlocal llm_turn, compaction_calls
+ if name == "agent.prepare_payload.v1":
+ return prepared
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ llm_turn += 1
+ if llm_turn == 1:
+ return {
+ "kind": "tool_calls",
+ "calls": [
+ {
+ "id": "missing-1",
+ "name": "not_connected",
+ "args": {},
+ }
+ ],
+ "usage": {
+ "input_tokens": 7,
+ "output_tokens": 3,
+ "total_tokens": 10,
+ },
+ }
+ return {
+ "kind": "final",
+ "content": "done",
+ "thinking": None,
+ "usage": {
+ "input_tokens": 5,
+ "output_tokens": 1,
+ "total_tokens": 6,
+ },
+ }
+ if name == "agent.compact_memory.v1":
+ compaction_calls += 1
+ return {
+ "success": True,
+ "summary": "compacted history",
+ "tokens_before": 10,
+ "tokens_after": 0,
+ "usage": {
+ "input_tokens": 4,
+ "output_tokens": 2,
+ "total_tokens": 6,
+ },
+ }
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert result["success"] is True
+ assert compaction_calls == 1
+ assert result["result"]["usage"] == {
+ "input_tokens": 16,
+ "output_tokens": 6,
+ "total_tokens": 22,
+ }
+
+ @pytest.mark.asyncio
+ async def test_markerless_compaction_preserves_legacy_payload_and_usage(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ prepared = _payload(native=False)
+ prepared.update(
+ {
+ "memory_content": "prior history",
+ "compaction_threshold": 5,
+ }
+ )
+ llm_turn = 0
+ compact_payloads = []
+
+ async def fake_execute_activity(name, *, args, **_kwargs):
+ nonlocal llm_turn
+ if name == "agent.prepare_payload.v1":
+ return prepared
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ llm_turn += 1
+ if llm_turn == 1:
+ return {
+ "kind": "tool_calls",
+ "calls": [
+ {
+ "id": "missing-1",
+ "name": "not_connected",
+ "args": {},
+ }
+ ],
+ "usage": {
+ "input_tokens": 3,
+ "output_tokens": 3,
+ "total_tokens": 100,
+ "cache_read_tokens": 50,
+ },
+ }
+ return {
+ "kind": "final",
+ "content": "done",
+ "thinking": None,
+ "usage": {
+ "input_tokens": 2,
+ "output_tokens": 1,
+ "total_tokens": 3,
+ },
+ }
+ if name == "agent.compact_memory.v1":
+ compact_payloads.append(args[0])
+ return {
+ "success": True,
+ "summary": "legacy compacted history",
+ "tokens_before": 6,
+ "tokens_after": 0,
+ "usage": {
+ "input_tokens": 400,
+ "output_tokens": 100,
+ "total_tokens": 500,
+ },
+ }
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert list(compact_payloads[0]) == [
+ "session_id",
+ "node_id",
+ "memory_content",
+ "provider",
+ "api_key",
+ "model",
+ ]
+ assert compact_payloads[0]["api_key"] == "test-key"
+ # Historical semantics reset pre-compaction usage and did not include
+ # the summarizer call in the final workflow result.
+ assert result["result"]["usage"] == {
+ "input_tokens": 2,
+ "output_tokens": 1,
+ "total_tokens": 3,
+ }
+
+ @pytest.mark.asyncio
+ async def test_markerless_compaction_threshold_ignores_total_and_cache(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ prepared = _payload(native=False)
+ prepared.update(
+ {
+ "memory_content": "prior history",
+ "compaction_threshold": 7,
+ "max_iterations": 1,
+ }
+ )
+
+ async def fake_execute_activity(name, *, args, **_kwargs):
+ if name == "agent.prepare_payload.v1":
+ return prepared
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ return {
+ "kind": "tool_calls",
+ "calls": [
+ {
+ "id": "missing-1",
+ "name": "not_connected",
+ "args": {},
+ }
+ ],
+ "usage": {
+ "input_tokens": 3,
+ "output_tokens": 3,
+ "total_tokens": 100,
+ "cache_read_tokens": 50,
+ },
+ }
+ if name == "agent.compact_memory.v1":
+ raise AssertionError(
+ "markerless threshold must use input + output only"
+ )
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+ assert result["success"] is True
+ assert result["result"]["usage"]["total_tokens"] == 100
+
+ @pytest.mark.asyncio
+ async def test_markerless_error_result_keeps_historical_raw_detail(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ async def fake_execute_activity(name, *, args, **_kwargs):
+ if name == "agent.prepare_payload.v1":
+ return _payload(native=False)
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ failure = RuntimeError("outer activity failure")
+ failure.cause = RuntimeError("legacy raw provider detail")
+ raise failure
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert result["success"] is False
+ assert result["error"] == (
+ "LLM step failed: legacy raw provider detail"
+ )
+
+ @pytest.mark.asyncio
+ async def test_missing_engine_preserves_legacy_activity_input(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ llm_calls: list[tuple[dict, dict]] = []
+
+ async def fake_execute_activity(name, *, args, **kwargs):
+ if name == "agent.prepare_payload.v1":
+ return _payload(native=False)
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ llm_calls.append((args[0], kwargs))
+ return {
+ "kind": "final",
+ "content": "done",
+ "thinking": None,
+ "usage": {},
+ }
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert result["success"] is True
+ llm_payload, options = llm_calls[0]
+ assert "llm_engine" not in llm_payload
+ assert "message_wire_version" not in llm_payload
+ assert "tools" not in llm_payload
+ assert llm_payload["tool_data"] == [_tool()["tool_info"]]
+ assert list(llm_payload) == [
+ "provider",
+ "model",
+ "api_key",
+ "messages",
+ "tool_data",
+ "system_message",
+ "temperature",
+ "max_tokens",
+ "thinking_config",
+ ]
+ assert llm_payload["messages"] == [
+ {"type": "system", "data": {"content": "Be useful"}},
+ {"type": "human", "data": {"content": "do the work"}},
+ ]
+ assert "heartbeat_timeout" not in options
+
+ @pytest.mark.asyncio
+ async def test_explicit_langchain_marker_keeps_credentials_out_of_history(
+ self,
+ monkeypatch,
+ patched_workflow,
+ ):
+ from services.temporal.agent_workflow import AgentWorkflow
+
+ prepared_payload = _emergency_langchain_payload()
+ llm_calls: list[tuple[dict, dict]] = []
+
+ async def fake_execute_activity(name, *, args, **kwargs):
+ if name == "agent.prepare_payload.v1":
+ assert "api_key" not in prepared_payload
+ return prepared_payload
+ if name == "agent.broadcast_progress.v1":
+ return {"emitted": True}
+ if name == "agent.execute_llm_step.v1":
+ llm_calls.append((args[0], kwargs))
+ return {
+ "kind": "final",
+ "content": "done",
+ "thinking": None,
+ "usage": {},
+ }
+ if name == "agent.store_output.v1":
+ return {"stored": True}
+ if name == "agent.skill.clear.v1":
+ return {"cleared": True}
+ raise AssertionError(f"Unexpected activity {name}")
+
+ monkeypatch.setattr(
+ patched_workflow,
+ "execute_activity",
+ fake_execute_activity,
+ )
+
+ result = await AgentWorkflow().run(
+ {"node_id": "agent-1", "execution_id": "root-run-1"}
+ )
+
+ assert result["success"] is True
+ llm_payload, options = llm_calls[0]
+ assert list(llm_payload) == [
+ "node_id",
+ "provider",
+ "model",
+ "messages",
+ "tool_data",
+ "system_message",
+ "temperature",
+ "max_tokens",
+ "thinking_config",
+ "llm_engine",
+ "message_wire_version",
+ ]
+ assert llm_payload["llm_engine"] == "langchain"
+ assert llm_payload["message_wire_version"] == 1
+ assert "api_key" not in llm_payload
+ assert llm_payload["tool_data"] == [_tool()["tool_info"]]
+ assert options["heartbeat_timeout"] == timedelta(minutes=1)
+
+ def test_tool_results_follow_the_history_pinned_wire_format(self):
+ from services.temporal.agent_workflow import (
+ _append_tool_result_message,
+ )
+
+ native_messages: list[dict] = []
+ _append_tool_result_message(
+ native_messages,
+ llm_engine="native",
+ content='{"ok": true}',
+ tool_call_id="call-1",
+ name="write_todos",
+ )
+ assert native_messages[0]["version"] == 2
+ assert native_messages[0]["role"] == "tool"
+ assert native_messages[0]["tool_call_id"] == "call-1"
+ assert native_messages[0]["blocks"][0]["type"] == "tool_result"
+
+ legacy_messages: list[dict] = []
+ _append_tool_result_message(
+ legacy_messages,
+ llm_engine="langchain",
+ content='{"ok": true}',
+ tool_call_id="call-1",
+ name="write_todos",
+ )
+ assert legacy_messages == [
+ {
+ "type": "tool",
+ "data": {
+ "content": '{"ok": true}',
+ "tool_call_id": "call-1",
+ "name": "write_todos",
+ },
+ }
+ ]
+
+ def test_native_tool_turn_reasoning_is_read_from_canonical_message(self):
+ from services.llm.protocol import (
+ ContentBlock,
+ Message,
+ message_to_wire,
+ )
+ from services.temporal.agent_workflow import (
+ _native_assistant_thinking,
+ )
+
+ wire = message_to_wire(
+ Message(
+ role="assistant",
+ blocks=[
+ ContentBlock(type="reasoning", text="first"),
+ ContentBlock(type="reasoning", text="second"),
+ ],
+ )
+ )
+ assert _native_assistant_thinking(wire) == "first\n\nsecond"
+
+
+class TestNativeActivityBranch:
+ def test_structured_llm_error_becomes_safe_temporal_failure(self):
+ from services.llm.protocol import LLMError, LLMErrorCategory
+ from services.temporal.agent_activities import (
+ _as_temporal_llm_error,
+ )
+
+ provider_error = LLMError(
+ message="raw body included secret request details",
+ provider="openai",
+ category=LLMErrorCategory.RATE_LIMIT,
+ retryable=True,
+ status_code=429,
+ provider_code="capacity",
+ request_id="req-123",
+ retry_after=1.5,
+ )
+ temporal_error = _as_temporal_llm_error(provider_error)
+
+ assert "raw body" not in str(temporal_error)
+ assert temporal_error.type == "LLMError.rate_limit"
+ assert temporal_error.non_retryable is False
+ assert temporal_error.details == (
+ {
+ "provider": "openai",
+ "category": "rate_limit",
+ "retryable": True,
+ "status_code": 429,
+ "provider_code": "capacity",
+ "request_id": "req-123",
+ "retry_after": 1.5,
+ "retry_after_raw": None,
+ },
+ )
+
+ @pytest.mark.asyncio
+ async def test_buffered_native_call_heartbeats_while_waiting(
+ self,
+ monkeypatch,
+ ):
+ import services.temporal.agent_activities as activity_module
+
+ heartbeat = MagicMock()
+ monkeypatch.setattr(activity_module.activity, "heartbeat", heartbeat)
+
+ async def delayed_result():
+ await asyncio.sleep(0.02)
+ return "done"
+
+ result = await activity_module._await_with_llm_heartbeats(
+ delayed_result(),
+ detail="waiting",
+ interval_seconds=0.001,
+ )
+
+ assert result == "done"
+ heartbeat.assert_called_with("waiting")
+
+ @pytest.mark.asyncio
+ async def test_native_step_disables_sdk_retries_and_keeps_v1_result_keys(
+ self,
+ monkeypatch,
+ ):
+ import core.container as container_module
+ import services.agent_runtime as runtime_module
+ from services.llm.protocol import (
+ LLMResponse,
+ Message,
+ ToolCall,
+ Usage,
+ message_to_wire,
+ )
+ from services.temporal.agent_activities import _execute_native_llm_step
+
+ unifier = object()
+ monkeypatch.setattr(
+ container_module.container,
+ "chat_unifier",
+ lambda: unifier,
+ )
+ call = ToolCall.from_raw(
+ id="call-1",
+ name="write_todos",
+ arguments="{not-json",
+ )
+ run_step = AsyncMock(
+ return_value=LLMResponse(
+ tool_calls=[call],
+ usage=Usage(input_tokens=5, output_tokens=2),
+ )
+ )
+ monkeypatch.setattr(runtime_module, "run_native_llm_step", run_step)
+
+ result = await _execute_native_llm_step(
+ {
+ "provider": "openai",
+ "model": "test-model",
+ "api_key": "secret",
+ "messages": [
+ message_to_wire(Message(role="user", content="go"))
+ ],
+ "tools": [_tool()["definition"]],
+ "temperature": 0,
+ "max_tokens": 100,
+ }
+ )
+
+ assert set(result) == {
+ "kind",
+ "assistant_message",
+ "calls",
+ "usage",
+ }
+ assert result["kind"] == "tool_calls"
+ assert result["assistant_message"]["version"] == 2
+ assert result["calls"][0]["raw_arguments"] == "{not-json"
+ assert result["calls"][0]["parse_error"]
+ assert result["usage"]["total_tokens"] == 7
+
+ assert run_step.await_args.args == (unifier,)
+ kwargs = run_step.await_args.kwargs
+ assert kwargs["sdk_max_retries"] == 0
+ assert kwargs["translate_errors"] is False
+ assert kwargs["tools"][0].name == "write_todos"