Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
docs(claude-code): align CLI agent docs and comments with the session pool
trohitg20bea1d6efc7
diff --git a/docs-internal/claude_code_interactive_mode.md b/docs-internal/claude_code_interactive_mode.mdindex 0453f799..f98289e2 100644--- a/docs-internal/claude_code_interactive_mode.md+++ b/docs-internal/claude_code_interactive_mode.md@@ -3,7 +3,7 @@> **TL;DR.** `ClaudeSessionPool` keeps a warm `claude` subprocess per> session key (the RFC-0002 conversation key `context_bridge.pool_key` when> a Context node is wired; the legacy `simpleMemory.node_id` on `input-memory`-> graphs — `services/cli_agent/service.py:359-363`) and drives it with the same flags Anthropic's+> graphs — `bound_key` in `AICliService.run_batch`) and drives it with the same flags Anthropic's> own VSCode extension uses — stdio pipes, `--output-format stream-json> --input-format stream-json --verbose --ide`. Multi-turn happens by> writing newline-delimited JSON to `proc.stdin` of the long-lived@@ -43,9 +43,9 @@ A background `stdout_reader_task` parses each line and dispatches via| **Transport** | `asyncio.subprocess.Process` with stdio pipes — same on POSIX + Windows || **Send turn** | `proc.stdin.write(json.dumps({"type":"user","message":{"role":"user","content":...}}) + "\n")` || **Read events** | `proc.stdout.readline()` loop → `json.loads(line)` → `_handle_stream_event` |-| **Completion detect** | stream-json event where `type == "result"` (set via `provider.is_final_event`) |-| **Context reset (`/clear` equivalent)** | kill subprocess + drop captured UUID — stream-json input has no slash-command path (confirmed in VSCode extension source) |-| **Crash recovery** | `acquire()` checks `process.returncode`; respawns with `--resume <captured_uuid>` so the same on-disk JSONL keeps growing |+| **Completion detect** | stream-json event where `type == "result"` (set via `provider.is_final_event`); stdout EOF also wakes the turn, which then fails with the child's exit code |+| **Context reset (`/clear` equivalent)** | kill subprocess + drop captured UUID — stream-json input has no slash-command path (confirmed in VSCode extension source). Primitive: `pool.clear` (no production caller today); Context-bound sessions are reset by `terminate_conversations` and the generation fence in `acquire` |+| **Crash recovery** | `acquire()` checks `process.returncode`; respawns with `--resume <captured_uuid>` so the same on-disk JSONL keeps growing. Only while the dead entry is still in the pool — an idle reap or LRU eviction removes the entry and its UUID |### Code layout@@ -69,7 +69,8 @@ server/services/cli_agent/ (generic framework — shared by all CLI plugins)├── service.py # AICliService.run_batch — dispatcher; every claude task goes to the pool (registry lookup)├── session.py # AICliSession — generic PTY + JSONL path; NOT used for claude (GitHub #133 / #134)├── mcp_server.py # BatchContext + FastMCP bridge (shared by every CLI provider)-├── workflow_tools.py / lockfile.py / jsonl_watcher.py / config.py / protocol.py / types.py / _cli_auth.py+├── context_bridge.py # SpecializedAgentContextBridge — Context pool_key, transcript-in-prompt, record_turn+├── workflow_tools.py / worktree.py / lockfile.py / jsonl_watcher.py / config.py / protocol.py / types.py / _cli_auth.py└── _handlers.py # codex WS handlers only (will move once codex_agent migrates)```diff --git a/docs-internal/cli_agent_framework.md b/docs-internal/cli_agent_framework.mdindex 6cc831b5..e9e5d910 100644--- a/docs-internal/cli_agent_framework.md+++ b/docs-internal/cli_agent_framework.md@@ -8,7 +8,7 @@ Multi-instance, multi-provider runtime for AI CLI agents (Claude Code, Codex, Ge| OpenAI Codex (`@openai/codex`) | shipping (no login flow yet) | User runs `codex login` manually; UI returns a graceful "not yet wired" error || Google Gemini (`@google/gemini-cli`) | v2 stub | factory raises `NotImplementedError` |-Browser automation has its own managed Chrome and browser-use runtime; it is not installed into this bun package tree. See [browser.md](./browser.md) and [browser_workspace.md](./browser_workspace.md) for its lifecycle and live control surface.+Browser automation has its own runtime (the installed Chrome/Edge/Chromium by default, plus a uv-installed browser-use CLI); it is not installed into this bun package tree. See [browser.md](./browser.md) and [browser_workspace.md](./browser_workspace.md) for its lifecycle and live control surface.## Architecture@@ -19,17 +19,20 @@ ClaudeCodeAgentNode (claude_code_agent) CodexAgentNode (codex_agent)AICliService.run_batch(provider, tasks, *, node_id, workflow_id, workspace_dir)│ provider = create_cli_provider(name) ("claude" | "codex" | "gemini"→NotImpl)│ allocate per-batch bearer token; register BatchContext in MCP token registry- │ asyncio.gather under Semaphore(5)+ │ asyncio.gather under Semaphore(max_parallel, default 5)│- ├──► AICliSession_0 (BaseProcessSupervisor + ClaudeProvider + ClaudeTaskSpec)- │ _pre_spawn(): git worktree add + write <DATA_DIR>/claude/ide/<pid>.lock- │ _do_start(): anyio.open_process + NDJSON stdout/stderr consumers- │ env: CLAUDE_IDE_LOCK + OPENCOMPANY_PARENT_RUN_ID- │ wait_for_completion(timeout)- │ cleanup(): stop()=terminate_then_kill(5s) + rm lockfile + worktree remove+ ├──► claude (registered session pool) → ClaudeSessionPool via _run_pooled_turn+ │ run_pooled(): git worktree add <workspace>/<node>/wt_session (bound)+ │ or wt_<run>_<i> per task (ephemeral, removed after the batch)+ │ pool.acquire(): cold spawn over stdio pipes, or warm reuse+ │ argv: --mcp-config <json> (MCP bearer) ... --ide (no lockfile written)+ │ env: CLAUDE_CONFIG_DIR + OPENCOMPANY_PARENT_RUN_ID+ │ pool.send_turn(): stream-json line on stdin, wait for `result` on stdout│- └──► AICliSession_N (CodexProvider + CodexTaskSpec)- (no lockfile yet — Codex CLI doesn't honor it; written when upstream supports it)+ └──► codex (no session pool) → AICliSession (BaseProcessSupervisor + CodexProvider)+ _pre_spawn(): git worktree add (lockfile only if the provider declares+ `ide_lockfile`; Codex does not)+ wait_for_completion(timeout); cleanup(): terminate_then_kill(5s) + worktree remove││ ◄──── CLI calls back via MCP/HTTP at /mcp/ide/mcp│ Authorization: Bearer <batch-token> (per-batch isolation)@@ -37,7 +40,8 @@ AICliService.run_batch(provider, tasks, *, node_id, workflow_id, workspace_dir)│ searchSkillResource, getCredential, broadcastLog▼BatchResult { tasks: [SessionResult, ...], n_succeeded, n_failed, total_cost_usd?, wall_clock_ms }- │ token deregistered in finally+ │ token deregistered in finally (non-pool path; a pooled session keeps its+ │ spawn-time token until _terminate_locked)▼existing Temporal heartbeat fires per WS broadcast (services/temporal/activities.py:259)existing tool-result envelope truncates response at 4000 chars (services/handlers/tools.py:993)@@ -105,7 +109,7 @@ the interactive billing bucket (entrypoint `claude-vscode`, NOT--output-format stream-json # events on stdout--input-format stream-json # user turns to stdin as JSON--verbose # required with stream-json for full event detail- --ide # VSCode auto-connect via lockfile+ --ide # matches the VSCode extension; no lockfile is written, MCP rides --mcp-config--model <model>[--session-id <UUID> | --resume <UUID>] # mutually exclusive--allowedTools <csv>@@ -226,7 +230,7 @@ CLI auth is delegated to the CLI's own login flow + a synthetic marker token in| `claude_code` | `claude_code_login` | `claude_code_logout` || `codex_cli` | `codex_cli_login` (returns "not yet wired") | `codex_cli_logout` |-**Claude flow** (`server/nodes/agent/claude_code_agent/_handlers.py:103`, `handle_claude_code_login`; logout at `:140`) uses the documented CLI subcommands from [code.claude.com/docs/en/cli-reference](https://code.claude.com/docs/en/cli-reference):+**Claude flow** (`handle_claude_code_login` and `handle_claude_code_logout` in `server/nodes/agent/claude_code_agent/_handlers.py`) uses the documented CLI subcommands from [code.claude.com/docs/en/cli-reference](https://code.claude.com/docs/en/cli-reference):| Subcommand | Purpose ||---|---|diff --git a/server/nodes/agent/claude_code_agent/__init__.py b/server/nodes/agent/claude_code_agent/__init__.pyindex a6e69f54..4d4460a7 100644--- a/server/nodes/agent/claude_code_agent/__init__.py+++ b/server/nodes/agent/claude_code_agent/__init__.py@@ -298,13 +298,13 @@ class ClaudeCodeAgentNode(ActionNode):# Legacy input-memory bridge: claude maintains its own session JSONL# under `<CLAUDE_CONFIG_DIR>/projects/<cwd-encoded>/<UUID>.jsonl`.- # Context resolution, explicit UUID resume, raw-event journalling,- # and binding persistence live in AICliService.+ # Context resolution (SpecializedAgentContextBridge), the pooled+ # spawn, and memory / conversation persistence live in AICliService.# The project_key is derived from cwd (`[^a-zA-Z0-9-] -> -`),# so memory continuity needs only a STABLE cwd across runs.- # That's handled by AICliService passing memory_bound=True so- # AICliSession spawns under repo_root instead of an ephemeral- # worktree — see `services/cli_agent/session.py:cwd()`.+ # `AICliService.run_batch` gives every bound run the same git+ # worktree, `<workspace>/<node>/wt_session`, branched off the+ # resolved repo root; the repo root itself is never the cwd.## Continuity: when memory is wired, resume the UUID the previous# successful run persisted on the memory node (``last_session_id``,diff --git a/server/nodes/agent/claude_code_agent/_pool.py b/server/nodes/agent/claude_code_agent/_pool.pyindex 0956dc56..02dce6be 100644--- a/server/nodes/agent/claude_code_agent/_pool.py+++ b/server/nodes/agent/claude_code_agent/_pool.py@@ -1,7 +1,8 @@"""Long-lived ``claude`` subprocess pool — VSCode-extension pattern.Keeps one warm ``claude --output-format stream-json --input-format-stream-json --verbose --ide`` subprocess per Context thread+epoch (or per+stream-json --verbose --ide`` subprocess per Context conversation key+``(workflow_id, agent_node_id, generation)`` (or per``simpleMemory.node_id`` for legacy ``input-memory`` graphs) so successive turnscan reuse the same process — same session UUID across turns, no respawn cost.Mirrors what Anthropic's official@@ -32,8 +33,8 @@ Per-turn mechanics:event lands.4. ``send_turn`` returns a :class:`SessionResult` built from theturn's events. The session UUID is captured from the result event- so subsequent batches can spawn ``--resume <UUID>`` if the- pooled process has been reaped.+ so a respawn after the pooled process crashes can emit+ ``--resume <UUID>``.Lifecycle policy:@@ -58,9 +59,16 @@ Continuity across process restarts:on the memory node by ``AICliService._persist_memory``). Never``--continue``: the CLI resolves it only against interactivesessions, so it never found the sessions this pool creates.- - If the subprocess is later reaped and a new ``acquire`` happens,- the next spawn emits ``--resume <session.current_session_uuid>``- so the SAME JSONL keeps growing across process restarts.+ - Crash: if the pooled subprocess has exited but its entry is still+ in the pool, the next ``acquire`` respawns it with+ ``--resume <session.current_session_uuid>`` so the SAME JSONL keeps+ growing across process restarts.+ - Idle reap / LRU eviction: ``_terminate_locked`` pops the entry, so+ no UUID survives in the pool and the next ``acquire`` is a plain+ cold spawn. Memory-bound runs still resume through the memory+ node's stored ``last_session_id``; Context-bound runs start a fresh+ claude session and rely on the transcript the context bridge+ renders into the prompt.- :meth:`clear` is an explicit context-reset primitive: kill thesubprocess, drop the captured UUID, let the next ``acquire``spawn fresh with no continuity flag (claude assigns a new UUID).@@ -111,8 +119,8 @@ class PooledClaudeSession:The on-disk JSONL at``<CLAUDE_CONFIG_DIR>/projects/<project_key>/<session_uuid>.jsonl``is still written by claude — it's the persistence layer that- ``--continue`` / ``--resume`` rehydrate from across process- restarts — but the runtime contract here is stdout-only because+ ``--resume`` rehydrates from across process restarts — but the+ runtime contract here is stdout-only becausethe ``result`` event (final turn outcome) goes ONLY to stdout instream-json mode, NOT to the JSONL file."""@@ -122,10 +130,12 @@ class PooledClaudeSession:memory_node_id: ClaudePoolKeyprocess: asyncio.subprocess.Processcwd: Path- # ``current_session_uuid`` is "" until the first turn's ``system/init``- # or ``result`` event reveals it. Subsequent spawns for the same- # memory_node_id (after a crash / reap) emit ``--resume <UUID>`` so- # the same on-disk JSONL keeps growing across process restarts.+ # ``current_session_uuid`` is the host-minted ``--session-id`` (or the+ # ``--resume`` UUID) from spawn, else "" until the first turn's+ # ``system/init`` or ``result`` event reveals it. A respawn after a+ # crash emits ``--resume <UUID>`` so the same on-disk JSONL keeps+ # growing; an idle reap or eviction drops the entry and this UUID+ # with it.current_session_uuid: str = ""# Bearer token baked into claude's argv (via ``--mcp-config``) at# spawn time. The token persists for the subprocess's lifetime —diff --git a/server/nodes/agent/claude_code_agent/_provider.py b/server/nodes/agent/claude_code_agent/_provider.pyindex 17bda1b7..47531d8e 100644--- a/server/nodes/agent/claude_code_agent/_provider.py+++ b/server/nodes/agent/claude_code_agent/_provider.py@@ -18,20 +18,21 @@ stream-json --verbose --ide [--mcp-config ...] [--strict-mcp-config]**Tools + skills are preserved**:- ``--mcp-config`` registers OpenCompany's FastMCP server; the spawned`claude` discovers `mcp__opencompany__*` tools via `tools/list`.- - ``--allowedTools`` carries the explicit allowlist (built-ins +- every wired MCP tool) — same shape as the prior headless path.- - Skills are materialised under ``<cwd>/.claude/skills/`` by the- shared :func:`nodes.agent.claude_code_agent._skills.materialise_skills`- helper (called from both ``AICliSession._pre_spawn`` AND- ``ClaudeSessionPool._spawn``); ``Skill`` enters+ - ``--allowedTools`` carries the explicit allowlist (every wired MCP+ tool + OpenCompany's MCP infrastructure tools; Claude built-ins only+ when a task opts them in, except ``Skill`` when a skill is wired).+ - Skills are materialised under ``<workspace_dir>/.claude/skills/``+ (discovered through ``--add-dir <workspace_dir>``) by the shared+ :func:`nodes.agent.claude_code_agent._skills.materialise_skills`+ helper, called from ``ClaudeSessionPool._spawn`` and+ ``ClaudeSessionPool._prepare_warm_reuse``; ``Skill`` enters``--allowedTools`` iff at least one skill is wired.-**Permission mode**: ``bypassPermissions`` lets every allowlist entry-fire without a TUI prompt — non-interactive automation has no human at-the keyboard to click "Allow." Behaviorally equivalent to-``--dangerously-skip-permissions`` but uses the documented permission-mode (``code.claude.com/docs/en/permission-modes``) and is the same-flag a Composio AO "permissionless" launch sets.+**Permission mode**: ``dontAsk`` (the config default) runs without a+TUI prompt and denies anything outside ``--allowedTools``, so the+strict allowlist is actually enforced — non-interactive automation has+no human at the keyboard to click "Allow." See+``code.claude.com/docs/en/permission-modes``.Session identity: a cold spawn with no continuity flag gets ahost-minted ``--session-id <UUID>`` (``ClaudeSessionPool._spawn``) so the@@ -87,18 +88,17 @@ class AnthropicClaudeProvider:self.package_name = cfg.package_nameself.binary_name = cfg.binary_nameself.ide_lock_env_var = cfg.ide_lock_env_var- # IDE lockfile lives at ``<CLAUDE_CONFIG_DIR>/ide/<pid>.lock``- # per claude's own resolution rules — there is no separate- # "lockfile path" knob. We set- # ``CLAUDE_CONFIG_DIR=OPENCOMPANY_CLAUDE_DIR`` on every spawn (see- # ``service.py``, ``session.py``, ``claude_oauth.py``), so the- # canonical lockfile dir is unambiguously- # ``OPENCOMPANY_CLAUDE_DIR/ide``. Deriving it directly from the- # single source of truth keeps the lockfile-write side (here)- # in sync with the lockfile-read side (spawned claude) without- # a duplicated JSON path that can drift. ``OPENCOMPANY_CLAUDE_DIR``- # is imported at module top from ``_oauth`` (the canonical- # source of truth for this plugin's filesystem layout).+ # IDE lockfile dir: ``<CLAUDE_CONFIG_DIR>/ide/`` per claude's own+ # resolution rules — there is no separate "lockfile path" knob.+ # We set ``CLAUDE_CONFIG_DIR=OPENCOMPANY_CLAUDE_DIR`` on every+ # spawn (``AICliService._run_pooled_turn``, ``_oauth.py``), so+ # the dir is ``OPENCOMPANY_CLAUDE_DIR/ide``. The pool path writes+ # NO lockfile here (MCP connects through ``--mcp-config``); only+ # the generic ``AICliSession._pre_spawn``, which claude no longer+ # uses, writes one. The attribute stays for the Protocol and for+ # the startup stale-lockfile sweep in ``main.py``.+ # ``OPENCOMPANY_CLAUDE_DIR`` is imported at module top from+ # ``_oauth`` (the source of truth for this plugin's layout).self.ide_lockfile_dir = OPENCOMPANY_CLAUDE_DIR / "ide"self._defaults = cfg.defaultsself._supports = cfg.supports@@ -112,8 +112,9 @@ class AnthropicClaudeProvider:Delegates to ``._oauth.claude_binary_path`` — samepath used by the credentials Login button. Lazy-installs into- ``<DATA_DIR>/packages/`` on first miss. Raises- ``FileNotFoundError`` if ``npm`` isn't on PATH.+ ``<DATA_DIR>/packages/`` with ``bun add`` on first miss. Raises+ ``RuntimeError`` if bun is missing or the install fails, and+ ``FileNotFoundError`` if the binary is still absent afterwards."""return Path(claude_binary_path())@@ -142,9 +143,9 @@ class AnthropicClaudeProvider:User prompts arrive over ``proc.stdin`` as newline-delimited JSONof shape ``{"type":"user","message":{"role":"user","content":...}}``,not as an argv positional. Events stream back on ``proc.stdout``- in the same shape claude writes to its on-disk JSONL — the FE-- facing event handlers downstream are file-source-agnostic so- we keep tailing the JSONL via ``JsonlWatcher`` either way.+ and are parsed there by ``ClaudeSessionPool``; the on-disk+ session JSONL is not read at runtime (the ``result`` event is+ stdout-only).Critically NOT emitted:@@ -160,8 +161,10 @@ class AnthropicClaudeProvider:emitted as ``--mcp-config <json>`` so the spawned clauderegisters OpenCompany's FastMCP server. Tools allowlist(``--allowedTools``) lists every wired ``mcp__opencompany__*``- plus the 5 built-ins. Skill materialisation under- ``<cwd>/.claude/skills/`` is unchanged.+ plus OpenCompany's seven MCP infrastructure tools, the built-in+ ``Skill`` only when a skill is wired, and any built-ins the task's+ ``allowed_tools`` opts in. Skills are materialised under+ ``<workspace_dir>/.claude/skills/`` by the pool before spawn."""if not isinstance(task, ClaudeTaskSpec):raise TypeError("AnthropicClaudeProvider.interactive_argv requires ClaudeTaskSpec, " f"got {type(task).__name__}")@@ -175,9 +178,10 @@ class AnthropicClaudeProvider:# which spawns claude with exactly these four flags. ``--verbose``# is required so the stream-json output includes the full event# detail (without it, only the final result line is emitted).- # ``--ide`` makes claude discover the IDE lockfile we write at- # ``<CLAUDE_CONFIG_DIR>/ide/<pid>.lock`` for auto-connection to- # OpenCompany's MCP server.+ # ``--ide`` is kept to match the extension's invocation, but the+ # pool path writes no IDE lockfile and sets no ``CLAUDE_IDE_LOCK``,+ # so there is nothing for it to discover: OpenCompany's MCP+ # server is registered only through ``--mcp-config`` below.argv += ["--output-format","stream-json",@@ -216,18 +220,16 @@ class AnthropicClaudeProvider:model = task.model or defaults.get("default_model") or self._defaults.get("default_model", "claude-sonnet-4-6")argv += ["--model", model]- # Session continuity. Three valid states, mutually exclusive:- # - ``resume_session_id`` set → ``--resume <UUID>`` (explicit;- # used by ``--fork-session`` UI + any caller that already- # knows the exact UUID).- # - ``continue_session=True`` → ``--continue`` (claude- # auto-loads the latest conversation under the current- # cwd, per code.claude.com/docs/en/cli-reference). The- # cleaner default for memory-bound runs because it avoids- # ferrying a UUID through the memory node's params — claude- # tracks its own sessions on disk under- # ``<CLAUDE_CONFIG_DIR>/projects/<project_key>/`` and- # ``--continue`` picks the newest.+ # Session continuity. Three flags, mutually exclusive, in this+ # precedence order:+ # - ``resume_session_id`` set → ``--resume <UUID>``. Memory-bound+ # runs set it from the memory node's ``last_session_id``+ # (``ClaudeCodeAgentNode.execute_op``); the pool splices it in+ # when respawning a crashed session (``ClaudeSessionPool.acquire``).+ # - ``continue_session=True`` → ``--continue``. Kept only for+ # explicit callers; the node never sets it, because the CLI+ # resolves ``--continue`` only against interactively created+ # sessions and so never finds the ones this pool creates.# - ``session_id`` set → ``--session-id <UUID>`` (host-minted# on cold spawn by ``ClaudeSessionPool._spawn`` so the UUID# is known before the first event; the CLI requires a valid@@ -249,11 +251,10 @@ class AnthropicClaudeProvider:# capabilities the workflow didn't explicitly grant — equivalent# filesystem / shell / search functionality is wired via the# ``fileRead`` / ``fileModify`` / ``fsSearch`` / ``shell`` /- # ``browser`` / ``perplexitySearch`` workflow tools, and- # connected skills are surfaced via OpenCompany's own- # ``listSkills`` / ``getSkill`` MCP tools (no materialisation- # under ``.claude/skills/`` on the pool path, so the built-in- # ``Skill`` tool has nothing to load anyway).+ # ``browser`` / ``perplexitySearch`` workflow tools. The one+ # exception is ``Skill``, added below only when a skill is wired+ # (connected skills are also reachable through OpenCompany's+ # ``listSkills`` / ``getSkill`` MCP tools).## Callers can opt specific claude built-ins back in per-task# via ``ClaudeTaskSpec.allowed_tools`` if they really need them@@ -268,9 +269,10 @@ class AnthropicClaudeProvider:allowed_list += [f"mcp__opencompany__{name}" for name in connected_tool_names]# Conditionally enable claude's built-in ``Skill`` tool ONLY# when at least one skill is wired through ``input-skill``.- # The spawn path (pool + non-pool) materialises connected- # SKILL.md files under ``<cwd>/.claude/skills/`` via the- # shared :func:`nodes.agent.claude_code_agent._skills.materialise_skills`+ # ``ClaudeSessionPool`` materialises connected SKILL.md files+ # under ``<workspace_dir>/.claude/skills/`` (reached through+ # ``--add-dir``) via the shared+ # :func:`nodes.agent.claude_code_agent._skills.materialise_skills`# helper, so the built-in skill loader has something to# discover. Skills accessed by name (`Skill <name>`) load# those materialised files. If no skill is wired, ``Skill``