Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
feat(cli_agent): multi-instance Claude Code + Codex CLI framework with MCP server
trohitg72923bc1e85e
diff --git a/docs-internal/cli_agent_framework.md b/docs-internal/cli_agent_framework.mdnew file mode 100644index 00000000..9b69c1e0--- /dev/null+++ b/docs-internal/cli_agent_framework.md@@ -0,0 +1,752 @@+# AI CLI Agent Framework — Claude Code + Codex (v1), Gemini (v2)++## Context++`claude_code_agent` ([server/services/claude_code_service.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/claude_code_service.py)) today is single-instance, single-shot, Claude-only: wraps `asyncio.create_subprocess_exec` with `proc.communicate()`, hardcoded 300s `wait_for`, bypasses the existing `_supervisor` framework, and returns plain text.++**Scope priority — heavy bias toward Claude + Codex.** v1 ships these two only; Gemini is designed-for but deferred to v2. Rationale:++- **Claude Code** = the marquee provider. Full feature surface (sessions, resume, budget, turns, `allowedTools`, `permission-mode`), best stream-json (cost+turns+session_id in `result` event), mature MCP/IDE lockfile integration. Reference implementation for the VSCode pattern.+- **Codex** = the safety-first companion. Sandbox enforcement (`read-only|workspace-write|danger-full-access`) and `--ask-for-approval` make it the right pick for risk-bounded tasks even though it lacks sessions/resume/cost.+- **Gemini** = parity is straightforward but not load-bearing — the abstraction is built to absorb it without refactor in v2.++We need:++1. **Multi-instance** — one workflow node summons N parallel CLI sessions over a list of tasks, each isolated, returning aggregated structured output, cancellable.+2. **Multi-provider abstraction now** — even though only two providers ship in v1, the Protocol/factory/config layer is built upfront so adding Gemini in v2 is a drop-in.+3. **Tool/context injection without custom IPC** — follow VSCode/Cursor exactly: host an MCP server, write a discovery lock file, let CLIs auto-discover MachinaOs tools via the standard MCP protocol. v1 wires this for Claude (where it works today) and writes the lock file for Codex (cheap; Codex CLI doesn't honor it yet but will).+4. **Native CLI auth — no token wrapping** — the CLIs already have working `claude login` / `codex login` flows that store credentials in their own canonical locations (`~/.claude/`, `~/.codex/`, `~/.gemini/`). MachinaOs **does not** copy, encrypt, or relocate those credentials. We only **trigger** the native login flow and **detect** logged-in state. No isolated `~/.claude-machina/` auth dir for the new framework. No `codex_oauth.py` / `gemini_oauth.py` token-management modules.++The fix reuses three existing assets:++- **Supervisor framework** ([_supervisor/base.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/base.py), [process.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/process.py), [util.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/util.py)) — locked idempotent start/stop, `terminate_then_kill(grace=5s)`, recursive `kill_tree`, drain-task cancel, Windows `CTRL_BREAK_EVENT`. Canonical consumer: [whatsapp/_runtime.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/nodes/whatsapp/_runtime.py).+- **Native LLM provider layer** ([llm/protocol.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/protocol.py), [factory.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/factory.py), [providers/](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/providers)) — `@runtime_checkable` Protocol + lazy-import factory + JSON config. Mirror exactly for CLI providers.+- **Credentials registry** ([credential_registry.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/credential_registry.py), [credential_providers.json](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/config/credential_providers.json)) — add `_cli_base` abstract entry + three concrete entries via deep-merge.++## Confirmed design decisions++| Decision | Choice |+|---|---|+| Source branch | `credentials-scaling-v2` (current worktree) |+| Implementation worktree | **NEW** worktree at `D:\startup\projects\machinaos-worktrees\cli-agent-framework` on branch `feat/cli-agent-framework` (off `credentials-scaling-v2`) |+| Scope (v1) | **Claude Code + Codex** (heavy bias). Gemini provider/plugin/oauth/tests deferred to v2. |+| Per-task isolation | git worktree per task (fail batch early if `working_directory` is not a git repo) |+| Concurrency cap | 5 parallel sessions per pool (`asyncio.Semaphore`) |+| Failure mode | Continue-on-error; per-task success/error returned, batch always settles |+| Resumption | Claude: `--session-id`/`--resume`. Codex: N/A. (Gemini v2: `--session-id`/`--resume`) |+| Provider feature gaps | Discriminated-union Pydantic TaskSpec — `ClaudeTaskSpec` / `CodexTaskSpec` (v2 adds `GeminiTaskSpec` to the union) |+| Tool/context injection | **VSCode pattern**: local MCP server + lock file, no custom IPC. v1 active for Claude; v1 writes lock for Codex (cheap, future-proofs); Gemini v2. |+| Node shape | Two plugins in v1: `claude_code_agent` (refactored), `codex_agent` (new). v2 adds `gemini_cli_agent`. |++## Step 0 — bootstrap (first actions after ExitPlanMode)++**0a. Copy plan into the repo as a versioned design doc** so it's reviewable, diffable, and lives alongside the existing `docs-internal/` design docs:++```bash+cp C:/Users/Tgroh/.claude/plans/analyze-the-claude-code-purrfect-sunbeam.md \+ D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/docs-internal/cli_agent_framework.md+```++Drop the path-specific Windows-style absolute paths from the in-repo copy if they look out of place; the repo copy is the source of truth from this point forward.++**0b. Create the implementation worktree** off `credentials-scaling-v2`:++```bash+cd D:/startup/projects/machinaos-worktrees/credentials-scaling-v2+git worktree add -b feat/cli-agent-framework \+ D:/startup/projects/machinaos-worktrees/cli-agent-framework \+ credentials-scaling-v2+cd D:/startup/projects/machinaos-worktrees/cli-agent-framework+```++The new design doc (`docs-internal/cli_agent_framework.md`) is automatically present in the new worktree because `git worktree add` shares the working tree of the parent branch. All subsequent edits below land in the new worktree. The `credentials-scaling-v2` worktree stays clean except for the new design doc.++**0c. (Optional, recommended)** Commit the design doc on `credentials-scaling-v2` before branching, so it's traceable:++```bash+cd D:/startup/projects/machinaos-worktrees/credentials-scaling-v2+git add docs-internal/cli_agent_framework.md+git commit -m "docs: cli agent framework design (claude+codex v1, gemini v2)"+```++## CLI feature surface (raw GitHub source confirmed)++| Feature | Claude | Codex | Gemini |+|---|---|---|---|+| Headless flag | `--print -p` | `codex exec` | `--prompt -p` |+| JSON output | `--output-format json/stream-json` | `--json` (JSONL) | `--output-format json/stream-json` |+| Session ID | `--session-id <UUID>` | n/a | **`--session-id <UUID>`** (`config.ts`) |+| Resume | `--resume <UUID>` | n/a | `--resume latest|<idx>|<UUID>` |+| Model | `--model` | `--model` | `--model` |+| Max turns | `--max-turns <n>` | n/a | n/a |+| Budget | `--max-budget-usd <usd>` | n/a | n/a |+| Allowed tools | `--allowedTools` | n/a | n/a |+| MCP config | `--mcp-config <file>` + IDE lockfile auto-discovery | pre-config only (lockfile not yet honored) | `--mcp-config` + IDE lockfile auto-discovery |+| Sandbox | `--worktree` | `--sandbox read-only|workspace-write|danger-full-access` | `--sandbox` (Docker) |+| Auth env | `CLAUDE_CONFIG_DIR` | `CODEX_API_KEY_ENV_VAR` / `OPENAI_API_KEY_ENV_VAR` / `CODEX_AGENT_IDENTITY_ENV_VAR` | `GEMINI_CLI_HOME` |+| IDE lock env | `CLAUDE_IDE_LOCK` | (n/a v1) | `GEMINI_IDE_LOCK` |+| Cost in JSON | `total_cost_usd` | absent | absent (`stats.duration_ms` only) |+| Exit codes | 0=ok, 1=error | 0=ok, non-0=error | 0=ok, 1=`FatalAuthenticationError`, EPIPE→0 |++## Confirmed JSON event schemas (vendored from source)++**Claude `--output-format stream-json`:**+```+{"type":"system","subtype":"init","session_id":"<UUID>"}+{"type":"assistant","message":{...},"session_id":"<UUID>"}+{"type":"tool_use","tool_name":"...","tool_input":{...}}+{"type":"result","subtype":"success","result":"...","total_cost_usd":0.42,+ "duration_ms":18234,"num_turns":7,"session_id":"<UUID>"}+```+Final: `type=="result"`. Carries `total_cost_usd, duration_ms, num_turns, session_id`.++**Gemini** (`packages/cli/src/nonInteractiveCli.ts`):+```typescript+enum JsonStreamEventType {+ INIT='init', MESSAGE='message', TOOL_USE='tool_use',+ TOOL_RESULT='tool_result', ERROR='error', RESULT='result'+}+// RESULT: { type:'result', timestamp:<ISO8601>, status:'success',+// session_id:<UUID>, stats:{ duration_ms } }+```+Final: `type=="result"`. Carries `session_id` + `stats.duration_ms`. No USD.++**Codex** (`codex-rs/cli/src/main.rs`, Rust + `clap` v4): JS wrapper at `codex-cli/bin/codex.js` is pass-through (`spawn(binaryPath, process.argv.slice(2), { stdio: "inherit" })`). No public stream-json schema; best-effort `type=="complete"` or stream-end fallback. `cost_usd=null` always.++## Architecture++```+ClaudeCodeAgentNode (v1) CodexAgentNode (v1) [GeminiCliAgentNode v2]+ │ Params.tasks: list[ClaudeTaskSpec] list[CodexTaskSpec]+ ▼+AICliService.run_batch(provider, params, *, node_id, workflow_id, workspace_dir)+ │ provider = create_cli_provider(provider_name)+ │ "claude" → AnthropicClaudeProvider "codex" → OpenAICodexProvider+ │ "gemini" → NotImplementedError("deferred to v2")+ │ allocate per-batch bearer token; register in MCP token registry+ ▼+AICliBatchPool(provider, sessions[N], semaphore=5, batch_token)+ │ asyncio.gather(return_exceptions=False)+ │+ ├──► AICliSession_0(BaseProcessSupervisor, ClaudeProvider, ClaudeTaskSpec)+ │ _pre_spawn(): git worktree add + write ~/.claude/ide/<pid>.lock+ │ _do_start(): anyio.open_process + NDJSON consumers+ │ env: inherits parent (claude CLI reads its native ~/.claude/)+ │ + CLAUDE_IDE_LOCK=<path>+ │ + MACHINA_PARENT_RUN_ID=<wf>:<node>:<token8>+ │ (Claude CLI auto-discovers MCP, gains mcp__machina__* tools)+ │ wait_for_completion(timeout)+ │ cleanup(): stop()=terminate_then_kill(5s) + remove lockfile + worktree remove+ └──► AICliSession_1(CodexProvider, CodexTaskSpec)+ _pre_spawn(): git worktree add (no lockfile env — Codex CLI doesn't honor it yet)+ _do_start(): argv has --sandbox + --ask-for-approval+ env: inherits parent (codex CLI reads its native ~/.codex/auth.json)+ (sandbox enforced by CLI itself)+ wait_for_completion(timeout)+ cleanup(): stop() + worktree remove+ │+ │ ◄────── CLI calls back via MCP/HTTP+ │ POST http://127.0.0.1:<port>/mcp/ide+ │ Authorization: Bearer <batch-token>+ │ tools: getWorkspaceFiles | getSkills | getCredential | broadcastLog+ ▼+BatchResult{ tasks: [...], summary: {n_tasks, n_succeeded, n_failed,+ total_cost_usd?, wall_clock_ms} }+ │ batch_token deregistered in finally+ ▼+existing Temporal heartbeat hook wakes on each broadcast (activities.py:228)+existing tool-result envelope truncates response at 4000 chars (handlers/tools.py:678)+```++## Provider abstraction (mirrors `services/llm/`)++```python+# server/services/cli_agent/protocol.py+@runtime_checkable+class AICliProvider(Protocol):+ name: str # "claude" | "codex" | "gemini"+ package_name: str # npm package+ binary_name: str # "claude" | "codex" | "gemini"+ ide_lock_env_var: Optional[str] # CLAUDE_IDE_LOCK | GEMINI_IDE_LOCK | None+ ide_lockfile_dir: Optional[Path] # ~/.claude/ide | <tmpdir>/gemini/ide+ # NOTE: no auth_dir_env_var / isolated_auth_dir. The CLI uses its own+ # native auth storage (~/.claude/, ~/.codex/, ~/.gemini/). We do not wrap.++ def binary_path(self) -> Path: ...+ def headless_argv(self, task, *, defaults) -> list[str]: ...+ def login_argv(self) -> list[str]: ...+ # Returns the CLI's own native login command, e.g. ["claude", "login"]+ # or ["codex", "login"]. Spawned interactively from the Credentials+ # Modal. CLI stores its own credentials; we don't intercept.+ def auth_status_argv(self) -> Optional[list[str]]: ...+ # Optional: a no-op command to verify auth (e.g. ["claude", "--version"]+ # combined with a no-op `--print -p ""` invocation). If None, we infer+ # from the next session's stderr matcher.+ def detect_auth_error(self, stderr: str, exit_code: int) -> bool: ...+ # Matcher for "not logged in" stderr patterns:+ # Claude: "Please run 'claude login'"+ # Codex: HTTP 401 / "OPENAI_API_KEY not set"+ # Gemini: exit_code == 1 (FatalAuthenticationError)+ def parse_event(self, line: str) -> Optional[dict]: ...+ def is_final_event(self, event: dict) -> bool: ...+ def event_to_session_result(self, events, stderr, exit_code) -> dict: ...+ # Returns: { ...shared fields..., "provider_data": {<vendor-specific>} }+ # provider_data carries vendor-only metadata (Anthropic reasoning_details,+ # Codex call_id, Gemini extra_content) without bloating the shared schema.+ # Pattern from Hermes agent/transports/types.py.+ def canonical_usage(self, events) -> dict: ...+ # Normalize vendor token-counting into {input, output, cache_read,+ # cache_write, reasoning, request_count}. Pattern from Hermes+ # agent/usage_pricing.py CanonicalUsage. Feeds existing services/pricing.py.+ def supports(self, feature: str) -> bool: ...+ # features: max_budget, max_turns, session_id, resume, mcp_runtime, json_cost, ide_lockfile+```++```python+# server/services/cli_agent/factory.py+def create_cli_provider(name: str) -> AICliProvider:+ if name == "claude":+ from .providers.anthropic_claude import AnthropicClaudeProvider+ return AnthropicClaudeProvider()+ if name == "codex":+ from .providers.openai_codex import OpenAICodexProvider+ return OpenAICodexProvider()+ if name == "gemini":+ from .providers.google_gemini import GoogleGeminiProvider+ return GoogleGeminiProvider()+ raise ValueError(f"Unknown CLI provider: {name}")+```++```jsonc+// server/config/ai_cli_providers.json+{+ "claude": { "package_name":"@anthropic-ai/claude-code", "binary_name":"claude",+ "login_argv": ["claude", "login"],+ "auth_status_argv": ["claude", "--print", "-p", "ok"],+ "ide_lock_env_var":"CLAUDE_IDE_LOCK",+ "ide_lockfile_dir":"~/.claude/ide",+ "default_model":"claude-sonnet-4-6", "default_max_turns":10,+ "default_max_budget_usd":5.0, "default_allowed_tools":"Read,Edit,Bash,Glob,Grep,Write",+ "supports":["max_budget","max_turns","session_id","resume","mcp_runtime","json_cost","ide_lockfile"] },+ "codex": { "package_name":"@openai/codex", "binary_name":"codex",+ "login_argv": ["codex", "login"],+ "auth_status_argv": ["codex", "--version"],+ "ide_lock_env_var": null, "ide_lockfile_dir": null,+ "default_model":"gpt-5.2-codex", "default_sandbox":"workspace-write",+ "default_ask_for_approval":"never", "supports":["sandbox"] },+ "gemini": { "package_name":"@google/gemini-cli", "binary_name":"gemini",+ "login_argv": ["gemini", "auth", "login"],+ "auth_status_argv": ["gemini", "--version"],+ "ide_lock_env_var":"GEMINI_IDE_LOCK",+ "ide_lockfile_dir":"<tmpdir>/gemini/ide",+ "default_model":"gemini-2.5-pro",+ "supports":["session_id","resume","sandbox","mcp_runtime","ide_lockfile"] }+}+```++## Discriminated-union TaskSpec++```python+# server/services/cli_agent/types.py+class BaseAICliTaskSpec(BaseModel):+ task_id: Optional[str] = None+ prompt: str+ branch: Optional[str] = None+ model: Optional[str] = None+ timeout_seconds: int = Field(600, ge=10, le=3600)+ system_prompt: Optional[str] = None++class ClaudeTaskSpec(BaseAICliTaskSpec):+ provider: Literal["claude"] = "claude"+ session_id: Optional[str] = None # start a named session+ resume_session_id: Optional[str] = None+ max_turns: Optional[int] = Field(None, ge=1)+ max_budget_usd: Optional[float] = Field(None, ge=0)+ allowed_tools: Optional[str] = None+ permission_mode: Literal["default","acceptEdits","plan","auto"] = "acceptEdits"++class CodexTaskSpec(BaseAICliTaskSpec):+ provider: Literal["codex"] = "codex"+ sandbox: Literal["read-only","workspace-write","danger-full-access"] = "workspace-write"+ ask_for_approval: Literal["untrusted","on-request","never"] = "never"++# GeminiTaskSpec defined in v1 types.py but plugin/provider stubbed as NotImplementedError+class GeminiTaskSpec(BaseAICliTaskSpec):+ provider: Literal["gemini"] = "gemini"+ session_id: Optional[str] = None+ resume: Optional[str] = None # "latest" | "<idx>" | "<UUID>"+ yolo: bool = False+ sandbox: bool = False++AICliTaskSpec = Annotated[+ Union[ClaudeTaskSpec, CodexTaskSpec, GeminiTaskSpec],+ Field(discriminator="provider"),+]+```++Each plugin's `Params.tasks` is hard-typed to one variant — the LLM tool schema fast-path at [ai.py:2898](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/ai.py#L2898) gets a clean per-provider schema with no `$defs`.++**v1 ships `ClaudeTaskSpec` + `CodexTaskSpec` plugins fully wired.** `GeminiTaskSpec` is in the union (and exported) so the type system, factory dispatch keys, and config JSON are ready, but `create_cli_provider("gemini")` raises `NotImplementedError("gemini provider deferred to v2")`.++## Auth model — native CLI handles its own tokens++**Principle**: each CLI ships with a working auth flow. Don't reinvent it. Don't wrap it. Don't relocate the credentials. We only **trigger** the native flow and **detect** the resulting auth state.++| Concern | What we do | What we don't do |+|---|---|---|+| Storage | CLI writes to its native location (`~/.claude/`, `~/.codex/`, `~/.gemini/`) | We **don't** copy/encrypt/relocate to `~/.claude-machina/`, our credentials.db, or anywhere else |+| Login trigger | Credentials Modal "Login with Claude/Codex/Gemini" button → WS handler spawns `provider.login_argv()` interactively (PTY or `stdio="inherit"`); user completes browser/device flow; we wait on exit | We **don't** orchestrate OAuth, exchange codes, refresh tokens, or store secrets |+| Status check | `provider.auth_status_argv()` no-op invocation; capture exit code + stderr through `provider.detect_auth_error()` | We **don't** poll `~/.claude/.credentials.json` or parse provider-internal state files |+| Refresh | The CLI handles refresh transparently | We **don't** schedule refresh jobs or manage token TTLs |+| Multi-tenancy | Single user per host (matches every dev who runs `claude`/`codex` locally) | We **don't** support per-MachinaOs-user isolated CLI auth in v1 — out of scope, v2 only if needed |+| Failure recovery | First detected `detect_auth_error()` short-circuits the batch with `error="not_authenticated"` and a hint pointing to the Credentials Modal | We **don't** retry, refresh, or attempt to re-auth mid-batch |++**Implication**: no `claude_oauth.py` / `codex_oauth.py` / `gemini_oauth.py` modules to write or maintain. The existing `claude_oauth.py` (which manages the legacy `~/.claude-machina/` install for `claude_code_service.py`) stays unchanged for back-compat with the legacy single-task path, but the new `cli_agent/` framework does not use it.++**Trade-offs accepted**:+- Anyone running MachinaOs on a host shares the host's CLI auth — same as anyone running `code .` shares VSCode's auth. Acceptable.+- We can't migrate auth between hosts via our credentials system. If users want that, they re-run `claude login` on the new host.+- Multiple parallel sessions hitting the CLI's token at once may race on refresh in rare cases. The CLIs handle this internally; if it shows up, we retry once on `detect_auth_error()` before failing the task.++## VSCode-style IDE MCP server (v1 core, no custom IPC)++VSCode Claude Code extension does, on activation:+1. Start MCP server on `127.0.0.1:<random-port>`+2. Write `~/.claude/ide/<vscode-pid>.lock` = `{"port":..., "authToken":"<32-hex>"}`+3. Spawn `claude` with `--mcp-config <tmp>` + IDE env+4. CLI reads lock, connects via Bearer `<authToken>`, gains `mcp__ide__*` tools++Gemini's VSCode companion: same shape, lock at `<tmpdir>/gemini/ide/gemini-ide-server-${PID}-${PORT}.json`.++We mirror exactly. **No custom protocol invented.**++### MachinaOs MCP tools (mirror existing capabilities)++| MCP tool | Maps to | Returns |+|---|---|---|+| `mcp__machina__getWorkspaceFiles` | `LocalShellBackend.glob/read_file` over `workspace_dir` (reuse [filesystem.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/handlers/filesystem.py)) | `{path, content?, mtime, size}` per file. `read=False` mode returns metadata only. |+| `mcp__machina__listSkills` | `SkillLoader.scan_skills()` — returns metadata only for skills connected to the parent agent node (via `BatchContext.connected_skill_names`) | List of `{name, description, allowed_tools, category}` per skill. ~100 tokens each (progressive disclosure pattern). |+| `mcp__machina__getSkill` | `SkillLoader.load_skill(name)` — returns full skill content | `{name, description, instructions, allowed_tools, metadata, scripts: {fname: content}, references: {fname: content}}`. `assets` (binary) excluded by default; available via separate `getSkillAsset` if needed. |+| `mcp__machina__getCredential` | `auth_service.get_api_key()` — gated by per-batch allowlist | `{key, value}` if name is in `BatchContext.allowed_credentials`, else 403. |+| `mcp__machina__broadcastLog` | `broadcaster.broadcast_terminal_log()` — CLI sub-tasks log to Terminal tab | `{success: true}`. Side effect: log appears in MachinaOs Terminal tab tagged with the source CLI session. |++`getDiagnostics` and `executeCode` deferred to v2.++### Skill exposure — concrete SKILL.md schema++Skills live at `server/skills/<folder>/<skill-name>/SKILL.md` with YAML frontmatter + markdown body. Reference: [`server/skills/social_agent/twitter-search-skill/SKILL.md`](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/skills/social_agent/twitter-search-skill/SKILL.md). The `SkillLoader` ([skill_loader.py:19-35](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/skill_loader.py#L19-L35)) parses to:++```python+@dataclass+class SkillMetadata:+ name: str # "twitter-search-skill"+ description: str # one-line LLM-visible hook+ allowed_tools: list[str] # ["twitter_search"] — informational v1+ metadata: dict[str, Any] # {author, version, category, icon, color}+ path: Optional[Path]++@dataclass+class Skill:+ metadata: SkillMetadata+ instructions: str # full markdown body+ scripts: dict[str, str] # exec code samples (filename→content)+ references: dict[str, str] # extra docs (filename→content)+ assets: dict[str, bytes] # binary attachments+```++**Data source priority** (already implemented in `SkillLoader`):++1. Built-in: `server/skills/<folder>/<skill>/SKILL.md`+2. Project: `.machina/skills/` in CWD+3. User-created: database (UI-edited content takes precedence after first activation)++**`allowed_tools` field is informational in v1** — the CLI agent gets its native tool set (Claude: `Read,Edit,Bash,Glob,Grep,Write` + connected MCP tools; Codex: sandbox-restricted shell). The `allowed_tools` list is included in `listSkills` response so the LLM understands intent ("this skill expects a `twitter_search` tool"), but we don't dynamically register MachinaOs tools as MCP tools per-skill in v1. v2 candidate.++**Progressive disclosure** matches Claude Code's Skill API design: `listSkills` is cheap (metadata only), `getSkill` is on-demand. CLI agent prompt should be guided to call `listSkills` first, then `getSkill(name)` only for relevant ones — keeps token budget tight.++**`BatchContext.connected_skill_names`** is populated at `run_batch()` entry by reusing [`_collect_agent_connections()`](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/nodes/agent/_inline.py) — same logic the existing `aiAgent` uses to find skills connected to its `input-skill` handle. So the CLI agent sees exactly the skills the user wired into the parent `claude_code_agent` / `codex_agent` node.++### Per-batch token isolation++- One token per `AICliBatchPool` (`secrets.token_hex(32)`)+- Lives in memory only; registered in `_active_tokens: dict[str, BatchContext]`+- Lock files (mode 0600 on POSIX) deleted in `cleanup()`+- MCP middleware validates `Authorization: Bearer <token>` → 401 if unknown/expired+- Each tool call resolves its `BatchContext` from the bearer token to scope `workspace_dir`, `node_id`, `allowed_credentials`++## Files to change / create++All marked **v1** unless explicitly noted **v2 (deferred)**.++| File | Change | LoC |+|---|---|---|+| `server/services/cli_agent/__init__.py` | **Create (v1)**. Re-exports: `create_cli_provider`, `AICliService`, `AICliBatchPool`, `AICliSession`, task-spec classes (incl. `GeminiTaskSpec` for type completeness). | ~25 |+| `server/services/cli_agent/protocol.py` | **Create (v1)**. `AICliProvider` Protocol. Mirrors [llm/protocol.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/protocol.py). | ~90 |+| `server/services/cli_agent/types.py` | **Create (v1)**. `BaseAICliTaskSpec`, `ClaudeTaskSpec`, `CodexTaskSpec`, `GeminiTaskSpec` (Pydantic schema only — no runtime impl), `AICliTaskSpec` discriminated union, `SessionResult`, `BatchResult`. | ~140 |+| `server/services/cli_agent/config.py` | **Create (v1)**. Loads `ai_cli_providers.json` (all three entries present; gemini surfaces a `available=False` flag). Mirrors [llm/config.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/config.py). | ~60 |+| `server/services/cli_agent/factory.py` | **Create (v1)**. `create_cli_provider(name)` lazy factory; `name=="gemini"` raises `NotImplementedError("gemini provider deferred to v2")`. Mirrors [llm/factory.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/factory.py). | ~55 |+| `server/services/cli_agent/providers/anthropic_claude.py` | **Create (v1, primary)**. argv: `claude --print -p ... --output-format stream-json --include-partial-messages [--session-id|--resume] --model --max-turns --max-budget-usd --allowedTools --permission-mode --append-system-prompt`. **Auth: native** — no `CLAUDE_CONFIG_DIR` injection; CLI reads `~/.claude/`. Final = `result` event. **All Protocol features wired** — sessions, resume, budget, turns, allowed_tools, permission_mode, MCP lockfile, cost. **Binary resolution chain** (Composio pattern): (1) `shutil.which("claude")`, (2) `npx --yes @anthropic-ai/claude-code` shim. `login_argv = ["claude", "login"]`. `detect_auth_error` matches `"Please run 'claude login'"`. `canonical_usage()` extracts `{input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens}` from `result.usage`. `provider_data` Dict carries `reasoning_details` if present. Reference implementation. | ~200 |+| `server/services/cli_agent/providers/openai_codex.py` | **Create (v1)**. argv: `codex exec --json --model --sandbox --ask-for-approval <prompt>`. **Auth: native** — Codex reads its own `~/.codex/auth.json` after the user runs `codex login`. We don't set `OPENAI_API_KEY`. No session/resume/budget/turns. Final = `type=="complete"` or stream-end. `cost_usd=None`. Lockfile written but Codex CLI doesn't read it yet — no-op until upstream support lands. **Sandbox is the marquee feature**, exposed prominently in `CodexTaskSpec`. **Binary resolution chain**: (1) `shutil.which("codex")`, (2) `npx --yes @openai/codex` shim. `login_argv = ["codex", "login"]`. `detect_auth_error` matches HTTP 401 patterns + `"OPENAI_API_KEY not set"`. `canonical_usage()` returns zeros (Codex doesn't expose token counts in `--json`); `provider_data` Dict carries `call_id` and `response_item_id` per Codex schema. | ~170 |+| `server/services/cli_agent/providers/google_gemini.py` | **v2 (deferred)**. argv: `gemini --prompt <p> --output-format stream-json --model [--session-id] [--resume] [--yolo] [--sandbox]`. Auth `GEMINI_CLI_HOME=~/.gemini-machina`. Final = `type=="result"`. Stub left as `# TODO v2` placeholder so the factory raises a clean `NotImplementedError("gemini provider deferred to v2")`. | 0 v1 (~160 v2) |+| `server/services/cli_agent/session.py` | **Create (v1)**. `AICliSession(BaseProcessSupervisor)`. `pipe_streams=True`, `terminate_grace_seconds=5.0`, `graceful_shutdown=(sys.platform=="win32")`. `_pre_spawn`: git worktree add + write IDE lockfile. Override `_do_start` to wire NDJSON consumers + IDE env var + inject `MACHINA_PARENT_RUN_ID=<workflow_id>:<node_id>:<token_short>` (Composio pattern). **Heartbeat stale-detection**: track `(events_count, last_tool_use)` tuple per second; if frozen >60s, log warning + dump `{argv, env_keys, last_20_events, stderr_tail}` to `~/.claude-machina/logs/session-stale-<task_id>.log` (Hermes pattern). On `wait_for_completion` `TimeoutError`, dump same diagnostic to `session-timeout-<task_id>.log` before `stop()`. `cleanup`: stop + remove lockfile + worktree remove. NOT registered in global registry. | ~360 |+| `server/services/cli_agent/lockfile.py` | **Create**. `write_ide_lockfile(pid, port, token, workspace_dir, ide_name) -> Path`, `remove_ide_lockfile(path)`. Path varies per provider (Claude: `~/.claude-machina/ide/<pid>.lock`; Gemini: `<tmpdir>/gemini/ide/gemini-ide-server-<pid>-<port>.json`). 0600 on POSIX. Stale-PID sweep on startup. | ~100 |+| `server/services/cli_agent/mcp_server.py` | **Create (v1)**. FastAPI sub-app at `/mcp/ide` using official `mcp` Python SDK's `StreamableHTTPSessionManager`. Bearer-token middleware → `_active_tokens` registry. Exposes **5 tools**: `getWorkspaceFiles`, `listSkills`, `getSkill`, `getCredential`, `broadcastLog`. Each tool dereferences its `BatchContext` (set on `run_batch` entry via `_collect_agent_connections()`) to scope `workspace_dir`, `node_id`, `connected_skill_names`, and `allowed_credentials`. Skills exposed via the existing `SkillLoader` (progressive disclosure: `listSkills` returns metadata only, `getSkill` returns full markdown + scripts + references). | ~240 |+| `server/services/cli_agent/pool.py` | **Create**. `AICliBatchPool` — owns N sessions, gathers under `Semaphore(5)`. `cancel()` calls `cleanup()` on every session. Aggregates `BatchResult`. | ~120 |+| `server/services/cli_agent/service.py` | **Create**. `AICliService.run_batch(provider, params, *, node_id, workflow_id, workspace_dir, broadcaster)`. Allocates batch token, registers `BatchContext`, instantiates pool, runs, deregisters in finally. `cancel_workflow(wf)`, `cancel_node(node_id)`. | ~160 |+| `server/services/claude_code_service.py` | **Modify** to thin shim. Builds one `ClaudeTaskSpec`, calls `AICliService.run_batch("claude", ...)`. Drops 300s `wait_for`. Keeps legacy return shape. | ~50 (was ~140) |+| `server/services/cli_agent/login.py` | **Create (v1)**. Generic `run_native_login(provider) -> dict` helper. Spawns `provider.login_argv()` with stdio inherited (the CLI handles its own browser/device flow). Returns `{success, exit_code, stderr_tail}`. Used by `cli_login_<provider>` WS handlers. ~50 lines. **Replaces** the would-be `codex_oauth.py` / `gemini_oauth.py` / new `claude_oauth.py`. | ~50 |+| `server/services/cli_agent/auth_status.py` | **Create (v1)**. Generic `check_auth(provider) -> {logged_in: bool, hint: str}` helper. Runs `provider.auth_status_argv()` and feeds output to `provider.detect_auth_error()`. Used by Credentials Modal status row. ~40 lines. | ~40 |+| ~~`server/services/codex_oauth.py`~~ | **Removed from scope** — Codex CLI handles its own auth via `~/.codex/auth.json`; we don't wrap it. | 0 |+| ~~`server/services/gemini_oauth.py`~~ | **Removed from scope** (v2 also). Gemini CLI handles its own auth via `~/.gemini/`; v2 will use the same generic `run_native_login` helper, no per-provider OAuth module. | 0 |+| `server/services/claude_oauth.py` | **No change**. Legacy module used by the legacy `claude_code_service.py` shim. The new framework does not call it. Eventually deletable when the legacy shim goes. | 0 |+| `server/main.py` | **Modify**. Mount MCP sub-app: `app.mount("/mcp/ide", get_mcp_app())` in `lifespan`. Add startup stale-lockfile sweep call. | ~8 |+| `server/nodes/agent/claude_code_agent.py` | **Modify**. `Params.tasks: list[ClaudeTaskSpec]`. `execute_op`: verify git repo, build tasks (with legacy-`prompt` fallback to single-task), call `run_batch("claude", ...)`, emit `phase=batch_started`/`batch_complete`. | ~140 |+| `server/nodes/agent/codex_agent.py` | **Create (v1)**. `CodexAgentNode` with `Params.tasks: list[CodexTaskSpec]`. Visual: orange/black icon, group `("agent","ai")`. Sandbox dropdown surfaced as primary param. | ~130 |+| `server/nodes/agent/gemini_cli_agent.py` | **v2 (deferred)**. `GeminiCliAgentNode` with `Params.tasks: list[GeminiTaskSpec]`. Visual: blue/yellow icon. | 0 v1 (~120 v2) |+| `server/nodes/visuals.json` | **Modify (v1)**. Add `codexAgent` icon+color. (v2: add `geminiCliAgent`.) | ~5 |+| `server/config/credential_providers.json` | **Modify (v1)**. Add `_cli_base` abstract entry (`kind: "native_cli"`, no token storage — just a status hook + login button) + two concretes (`claude_code`, `codex_cli`) extending it. (v2 adds `gemini_cli`.) | ~50 |+| `server/routers/websocket.py` | **Modify (v1)**. Add **generic** `cli_login` and `cli_auth_status` handlers (one pair total, dispatched by `provider` field) that delegate to `cli_agent/login.py` and `cli_agent/auth_status.py`. **Replaces** what would have been per-provider `codex_oauth_login` / `claude_oauth_login` etc. | ~40 |+| `server/services/workflow.py` | **Modify**. `stop_execution` → `get_ai_cli_service().cancel_workflow(workflow_id)` alongside existing process cancel. | ~10 |+| `server/services/_supervisor/*` | **No change**. Used as-is. | 0 |+| `server/services/temporal/activities.py` | **No change**. Heartbeats already fire per WS broadcast. | 0 |+| `server/pyproject.toml` | **Modify**. Add `mcp>=1.0` to deps (pinned). | ~2 |+| `server/tests/services/test_cli_agent_providers.py` | **Create (v1)**. Claude + Codex: `headless_argv` shape, `parse_event` schema fidelity (vendored NDJSON), `event_to_session_result` reconstruction. Heaviest coverage on Claude (full feature set). | ~250 |+| `server/tests/services/test_cli_agent_session.py` | **Create (v1)**. Generic session: 3-task parallel happy path with mocked `anyio.open_process`; cancel mid-batch; worktree cleanup on exception; lockfile written/removed. | ~280 |+| `server/tests/services/test_cli_agent_mcp.py` | **Create (v1)**. MCP server: bearer-token auth (valid/invalid/expired), per-tool scoping, lock file format matches VSCode convention, cleanup of stale PIDs. Uses `mcp` SDK test client. | ~220 |+| `server/tests/nodes/test_claude_code_agent.py` | **Create/modify (v1)**. Single-task back-compat + 3-task parallel via Claude. Heavy coverage: sessions, resume, budget, turns, allowed_tools, permission_mode, MCP tool callback. | ~220 |+| `server/tests/nodes/test_codex_agent.py` | **Create (v1)**. 3-task parallel happy path; assert no `--max-budget-usd` / `--max-turns` / `--session-id` in argv; sandbox `read-only`/`workspace-write`/`danger-full-access` flags propagate; `--ask-for-approval` honored. | ~160 |+| `server/tests/nodes/test_gemini_cli_agent.py` | **v2 (deferred)**. 3-task parallel; `--session-id` roundtrip; `--resume latest`; yolo flag. | 0 v1 (~150 v2) |+| `client/src/components/AIAgentNode.tsx` | **No change**. Renders the three new agent kinds via `useNodeSpec(type)`; the array-of-rows `tasks` editor is auto-generated from JSON Schema. | 0 |++**Net code delta (v1)**: ~2200 lines added, ~90 removed (-240 from dropping `codex_oauth.py` + per-provider OAuth WS handlers; +90 for generic `login.py` / `auth_status.py` helpers). Two plugins (Claude refactor + Codex new), two providers, MCP server (VSCode-pattern), session/pool/service, generic native-auth shim. Supervisor framework reused as-is. New runtime dep: `mcp>=1.0`.++**v1 → v2 follow-up (Gemini, ~430 LoC)**:+- `providers/google_gemini.py` (replace stub) — ~160+- `nodes/agent/gemini_cli_agent.py` — ~120+- `tests/nodes/test_gemini_cli_agent.py` — ~150+- Update `factory.py` (drop `NotImplementedError`)+- Add `geminiCliAgent` to `visuals.json` and `gemini_cli` entry in `credential_providers.json`+- (No `gemini_oauth.py`. No new WS handlers — generic `cli_login`/`cli_auth_status` already cover it.)++## AICliSession sketch++```python+# server/services/cli_agent/session.py (abridged)+class AICliSession(BaseProcessSupervisor):+ pipe_streams = True+ terminate_grace_seconds = 5.0+ graceful_shutdown = (sys.platform == "win32")++ def __init__(self, *, provider, task, repo_root, workspace_dir,+ node_id, workflow_id, broadcaster, defaults,+ mcp_port, batch_token):+ super().__init__()+ self._provider = provider+ self._task = task+ self._task_id = task.task_id or f"t_{uuid.uuid4().hex[:8]}"+ self._worktree_dir = workspace_dir / node_id / f"wt_{self._task_id}"+ self._branch = task.branch or f"machina/{self._task_id}"+ self._broadcaster = broadcaster+ self._defaults = defaults+ self._mcp_port = mcp_port+ self._batch_token = batch_token+ self._lockfile_path: Optional[Path] = None+ self._events, self._stderr = [], []+ self._exit_code: Optional[int] = None+ self._repo_root, self._node_id, self._workflow_id = repo_root, node_id, workflow_id++ @property+ def label(self): return f"AICliSession_{self._provider.name}_{self._task_id}"++ def binary_path(self): return self._provider.binary_path()+ def argv(self): return self._provider.headless_argv(self._task, defaults=self._defaults)+ def cwd(self): return self._worktree_dir++ def env(self):+ # Inherit parent env so the CLI finds its own auth in ~/.claude/, ~/.codex/, ~/.gemini/+ # We do NOT inject CLAUDE_CONFIG_DIR / GEMINI_CLI_HOME — the CLI uses its native location.+ e = {**os.environ, "PYTHONUNBUFFERED": "1"}+ if self._lockfile_path and self._provider.ide_lock_env_var:+ e[self._provider.ide_lock_env_var] = str(self._lockfile_path)+ # Composio pattern: parent-run-ID propagation for MCP correlation+ e["MACHINA_PARENT_RUN_ID"] = (+ f"{self._workflow_id}:{self._node_id}:{self._batch_token[:8]}"+ )+ return e++ async def _pre_spawn(self):+ # 1. git worktree+ self._worktree_dir.parent.mkdir(parents=True, exist_ok=True)+ result = await anyio.run_process(+ ["git", "-C", str(self._repo_root), "worktree", "add",+ str(self._worktree_dir), "-b", self._branch], check=False)+ if result.returncode != 0:+ raise RuntimeError(f"git worktree add failed: {result.stderr.decode()}")++ # 2. VSCode-style IDE lock file (CLI auto-discovers MachinaOs MCP)+ if self._provider.supports("ide_lockfile"):+ from .lockfile import write_ide_lockfile+ self._lockfile_path = write_ide_lockfile(+ pid=os.getpid(), port=self._mcp_port,+ token=self._batch_token,+ workspace_dir=self._worktree_dir,+ ide_name=self._provider.name,+ )++ async def _do_start(self):+ binary = self.binary_path()+ if not binary.exists():+ raise FileNotFoundError(f"{self.label} binary not found at {binary}")+ await self._pre_spawn()+ kwargs = {"cwd": str(self.cwd()), "env": self.env(),+ "stdout": subprocess.PIPE, "stderr": subprocess.PIPE}+ if sys.platform == "win32" and self.graceful_shutdown:+ kwargs["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP+ self._proc = await anyio.open_process(self.argv(), **kwargs)+ self._drain_tasks = [+ asyncio.create_task(self._consume_stdout(self._proc.stdout)),+ asyncio.create_task(self._consume_stderr(self._proc.stderr)),+ ]++ async def _consume_stdout(self, stream):+ async for raw in stream:+ text = raw.decode("utf-8", errors="replace").strip()+ if not text: continue+ event = self._provider.parse_event(text)+ if event is None: continue+ self._events.append(event)+ await self._on_event(event)++ async def _on_event(self, event):+ if self._provider.is_final_event(event):+ await self._broadcaster.update_node_status(self._node_id, "executing", {+ "phase":"ai_cli_subtask", "task_id":self._task_id,+ "provider":self._provider.name, "status":"succeeded",+ **{k: event.get(k) for k in+ ("total_cost_usd","duration_ms","num_turns","session_id")+ if event.get(k) is not None},+ })+ else:+ msg = event.get("message") or event.get("text") or json.dumps(event)+ await self._broadcaster.broadcast_terminal_log(+ {"source":f"{self._provider.name}:{self._task_id}",+ "level":"info", "message":msg[:500]})++ async def _consume_stderr(self, stream):+ async for raw in stream:+ text = raw.decode("utf-8", errors="replace").rstrip()+ if not text: continue+ self._stderr.append(text)+ await self._broadcaster.broadcast_terminal_log(+ {"source":f"{self._provider.name}:{self._task_id}",+ "level":"error", "message":text})++ async def wait_for_completion(self, timeout: int) -> dict:+ try:+ await asyncio.wait_for(self._proc.wait(), timeout=timeout)+ self._exit_code = self._proc.returncode+ except asyncio.TimeoutError:+ await self.stop()+ return self._build_result(success=False, error=f"timeout after {timeout}s")+ return self._build_result(success=(self._exit_code == 0))++ async def cleanup(self):+ await self.stop()+ if self._lockfile_path:+ from .lockfile import remove_ide_lockfile+ remove_ide_lockfile(self._lockfile_path)+ await anyio.run_process(+ ["git", "-C", str(self._repo_root), "worktree", "remove", "--force",+ str(self._worktree_dir)], check=False)++ def _build_result(self, *, success, error=None):+ base = self._provider.event_to_session_result(+ self._events, "\n".join(self._stderr), self._exit_code or -1)+ base.update({"task_id":self._task_id, "branch":self._branch,+ "worktree_path":str(self._worktree_dir),+ "success": success and base.get("success", True),+ "error": error or base.get("error")})+ return base+```++## Output contract++```json+{+ "tasks": [{+ "task_id": "t_<8hex>", "session_id": "<UUID|null>",+ "provider": "claude|codex|gemini",+ "prompt": "...", "branch": "machina/t_<8hex>",+ "worktree_path": "<abs path, removed after batch>",+ "response": "<truncated to 4000 chars>",+ "cost_usd": 0.42,+ "duration_ms": 18234,+ "num_turns": 7,+ "tool_calls": 12,+ "canonical_usage": {+ "input_tokens": 5000, "output_tokens": 1000,+ "cache_read": 500, "cache_write": 0,+ "reasoning_tokens": 0, "request_count": 7+ },+ "provider_data": {+ "reasoning_details": "...", // Anthropic only+ "call_id": "...", // Codex only+ "extra_content": [...] // Gemini only (v2)+ },+ "success": true, "error": null+ }],+ "summary": {+ "n_tasks": 3, "n_succeeded": 2, "n_failed": 1,+ "total_cost_usd": 1.23,+ "wall_clock_ms": 19002,+ "budget_remaining_usd": 18.77+ },+ "provider": "claude|codex|gemini",+ "timestamp": "2026-05-02T12:00:00Z"+}+```+`canonical_usage` normalizes vendor token-counting (Hermes pattern). `provider_data` carries vendor-only metadata that doesn't fit the shared schema.+`cost_usd`/`num_turns` = `null` for codex/gemini. `summary.total_cost_usd`/`budget_remaining_usd` = `null` if any task is non-Claude.++## Status broadcast events++| Phase | When | Payload |+|---|---|---|+| `batch_started` | run_batch entry | `{provider, n_tasks, max_parallel, isolation:"worktree"}` |+| `ai_cli_subtask` | per-task partial | `{task_id, provider, status:"running", message, cost_usd?, num_turns?}` |+| `ai_cli_subtask` | per-task final | `{task_id, provider, status:"succeeded"|"failed", cost_usd?, duration_ms, num_turns?, error?}` |+| `batch_complete` | aggregator finish | `{provider, n_succeeded, n_failed, total_cost_usd?, wall_clock_ms}` |++Plus `broadcast_terminal_log(source=f"{provider}:{task_id}", level)` on every NDJSON line.++## Error handling++- **Per-task failure** → `_run_one` catches → `SessionResult(success=False, error=...)`. Batch continues.+- **Per-task timeout** → `asyncio.TimeoutError` → `session.stop()` (terminate_then_kill 5s) → `error="timeout after Ns"`.+- **Auth missing** → `provider.detect_auth_error()` on first session's stderr/exit_code. First detection short-circuits remaining tasks: `error="not_authenticated"` with hint `"Open Credentials Modal and click 'Login with <Provider>'"`. The CLI handles re-auth itself via its own `<provider> login` flow; we don't.+- **CLI not installed** → `binary_path()` `FileNotFoundError` from [process.py:74-75](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/process.py#L74-L75) → pool catches → `error="cli_not_installed"` with hint pointing at credentials modal.+- **Not a git repo** → early `git rev-parse --git-dir` check in plugin's `execute_op` → `BatchResult(error="working_directory_not_git_repo")` before pool construction.+- **Worktree creation fails** → `_pre_spawn` raises → that task only fails; siblings continue.+- **Budget exhaustion (Claude)** → track `running_total_cost`; refuse to acquire semaphore for pending tasks once exceeded; in-flight tasks have CLI's own `--max-budget-usd` cap.+- **Cancellation** → `WorkflowService.stop_execution` → `service.cancel_workflow(wf)` → `pool.cancel()` → per-session `cleanup()` → `BaseSupervisor.stop()` → `terminate_then_kill` → `git worktree remove --force` + lockfile delete.+- **MCP 401** → CLI calls our MCP with bad/expired token → 401 → CLI sees tool error → continues with built-ins.+- **Lockfile leak on hard crash** → startup sweep deletes `~/.claude-machina/ide/*.lock` whose PID is no longer alive (mirrors VSCode behavior).++## Reused (DO NOT duplicate)++- [_supervisor/base.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/base.py) — `BaseSupervisor` lock + lifecycle+- [_supervisor/process.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/process.py) — `BaseProcessSupervisor` (override `_do_start`)+- [_supervisor/util.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/util.py) — `kill_tree`, `terminate_then_kill`+- [whatsapp/_runtime.py:26-116](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/nodes/whatsapp/_runtime.py#L26-L116) — reference subclass shape+- ~~[claude_oauth.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/claude_oauth.py)~~ — **NOT used** by the new framework. Legacy module for the legacy `claude_code_service.py` shim only. New framework uses native CLI auth (no wrapping).+- [llm/protocol.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/protocol.py), [factory.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/factory.py), [config.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/llm/config.py) — Protocol+factory+config blueprint+- [credential_registry.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/credential_registry.py) — `extends` deep-merge+- [handlers/tools.py:678](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/handlers/tools.py#L678) — 4000-char truncation+- [status_broadcaster.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/status_broadcaster.py) — `update_node_status`, `broadcast_terminal_log`+- [workflow.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/workflow.py) — `_get_workspace_dir`+- [agent/_inline.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/nodes/agent/_inline.py) — `collect_agent_connections()` for skill prompt+- [ai.py:2898](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/ai.py#L2898) — plugin Params doubles as LLM tool schema+- [skill_loader.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/skill_loader.py) — `SkillLoader.scan_skills()` (metadata, ~100 tokens/skill) reused by `mcp__machina__listSkills`; `SkillLoader.load_skill(name)` (full content with scripts/references) reused by `mcp__machina__getSkill`. Reference SKILL.md: [`server/skills/social_agent/twitter-search-skill/SKILL.md`](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/skills/social_agent/twitter-search-skill/SKILL.md)+- [auth.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/auth.py) — `AuthService.get_api_key()` reused by `mcp__machina__getCredential` (allowlisted)+- [handlers/filesystem.py](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/handlers/filesystem.py) — `LocalShellBackend` reused by `mcp__machina__getWorkspaceFiles`+- VSCode lock file format reference: `~/.claude/ide/<pid>.lock` = `{port, authToken, workspaceFolders, ideName, transport}` — mirrored exactly++## Verification++In order:++1. **Worktree**: `git worktree list` shows `cli-agent-framework` on `feat/cli-agent-framework`.+2. **Unit tests (v1)**:+ - `uv run pytest tests/services/test_cli_agent_providers.py -v`+ - `uv run pytest tests/services/test_cli_agent_session.py -v`+ - `uv run pytest tests/services/test_cli_agent_mcp.py -v`+ - `uv run pytest tests/nodes/test_claude_code_agent.py tests/nodes/test_codex_agent.py -v`+ - Add: `test_factory_gemini_raises_not_implemented` — calling `create_cli_provider("gemini")` raises `NotImplementedError`.+3. **Plugin contract**: `uv run pytest tests/test_plugin_contract.py tests/test_node_spec.py -k "claude or codex"` — clean per-provider schemas, no `$defs`.+4. **Live single-task back-compat (Claude)**: workflow with one `claude_code_agent`, legacy `prompt="say hello"`. Confirm `summary.n_tasks==1`, `summary.total_cost_usd != null`.+5. **Live 3-task parallel (Claude)**: `tasks=[{prompt:"echo A"},{prompt:"echo B"},{prompt:"echo C"}]`. Three distinct `claude:<task_id>` Terminal streams interleaved. Three distinct session_ids. Three worktrees created and removed. `summary.wall_clock_ms < sum(duration_ms)`.+6. **Live 3-task parallel (Codex)**: `codex_agent` node. Confirm `summary.total_cost_usd == null`, no `--max-budget-usd` / `--max-turns` / `--session-id` in argv. Verify `--sandbox workspace-write` and `--ask-for-approval never` defaults applied.+7. **Live cancel** (Claude or Codex): start long batch, click Stop. Confirm child processes die within 5s; worktrees removed; lockfiles removed.+8. **Live tool-mode (Claude)**: aiAgent → input-tools → claude_code_agent. Prompt: "use the claude_code tool to refactor utils.py and tests/test_utils.py in parallel." Confirm structured envelope; response truncated at 4000.+9. **Live resumption (Claude)**: capture `tasks[0].session_id`, run again with `tasks=[{resume_session_id:<captured>, prompt:"add error handling"}]`. Confirm context preservation.+10. **Live MCP discovery (Claude)**: with a Claude task running, `cat ~/.claude-machina/ide/<pid>.lock` and confirm format `{port, url, authToken, workspaceFolders, ideName:"machinaos", transport}`. Stream-json shows an `mcp__machina__*` tool invocation. MCP server logs the call with the per-batch token.+10a. **Skill exposure**: connect `twitter-search-skill` to a `claude_code_agent` node, run a task. Confirm CLI invokes `mcp__machina__listSkills` (returns 1 skill with `name="twitter-search-skill"`, `description`, `allowed_tools=["twitter_search"]`). Then CLI invokes `mcp__machina__getSkill("twitter-search-skill")` and receives full instructions markdown + any scripts/references. Connect a second skill (e.g., `python-skill`) and confirm only the connected skills appear in `listSkills` (scoping via `BatchContext.connected_skill_names`).+10b. **Credential allowlist**: configure batch with `allowed_credentials=["openai"]`. CLI calls `mcp__machina__getCredential("openai")` → returns key. CLI calls `mcp__machina__getCredential("anthropic")` → 403.+11. **Live MCP auth rejection**: `curl -H "Authorization: Bearer <wrong>" http://127.0.0.1:3010/mcp/ide/...` → 401.+12. **Live MCP cleanup**: lockfiles removed from `~/.claude-machina/ide/` after batch (success and cancel paths).+13. **Stale lockfile sweep**: kill server mid-batch, restart. Confirm leftover lockfile (PID dead) is deleted on startup.+14. **Codex sandbox enforcement**: Codex task with `sandbox:"read-only"` attempting to write a file → CLI rejects → captured in stderr, surfaces as `success=false` with sandbox error.+15. **Heartbeat stale-detection**: mock a session that emits no events for >60s; confirm `~/.claude-machina/logs/session-stale-<task_id>.log` is written with last-events + stderr tail.+16. **Timeout diagnostic dump**: set `timeout_seconds=5` on a long task; on timeout, confirm `session-timeout-<task_id>.log` contains argv, env keys (no secret values), last 20 events, stderr tail.+17. **Binary resolution chain**: temporarily move/rename the system `claude` binary; run a Claude task; confirm fallback to `npx --yes @anthropic-ai/claude-code` succeeds. Restore.+18. **Parent run-ID env**: with a Claude task running, MCP server logs show `MACHINA_PARENT_RUN_ID=<workflow>:<node>:<token8>` correlating each tool call to the batch.+19. **Native auth — login flow**: log out (`rm ~/.claude/.credentials.json` or equivalent). Open Credentials Modal → click "Login with Claude" → confirm `claude login` opens browser/device flow → complete it → confirm Credentials Modal status flips to "Connected" via `cli_auth_status` polling.+20. **Native auth — status check**: with `claude` logged in, `cli_auth_status` returns `{logged_in: true}`. Manually corrupt `~/.claude/.credentials.json` → `cli_auth_status` returns `{logged_in: false, hint: "Run claude login"}`.+21. **Native auth — error short-circuit**: log out, then run a 3-task batch. First task fails fast with `error="not_authenticated"`; remaining 2 tasks short-circuit with the same error (no spawn). Hint message is surfaced in the BatchResult.+22. **Auth env not injected**: `printenv | grep -i 'CLAUDE_CONFIG_DIR\|GEMINI_CLI_HOME'` inside a running task's env (via `mcp__machina__broadcastLog` reflection or `/proc/<pid>/environ` peek) confirms we're **not** setting these vars; CLI uses its native location.+23. **Smoke**: `uv run pytest -x -q` over the full server suite — no regressions from `claude_code_service.py` slimming.++## External orchestrator patterns (Hermes + Composio source review)++Researched against fresh clones at `D:/tmp/hermes-agent` (NousResearch) and `D:/tmp/composio` (ComposioHQ). Findings shaped the v1 design.++### What we copy++| Pattern | Source | Apply to |+|---|---|---|+| **`ProviderTransport` ABC + `provider_data` Dict** for vendor-specific metadata (Anthropic `reasoning_details`, Codex `call_id`/`response_item_id`, Gemini `extra_content`) without polluting the shared schema | Hermes [`agent/transports/base.py`](D:/tmp/hermes-agent/agent/transports/base.py), [`types.py`](D:/tmp/hermes-agent/agent/transports/types.py) | `AICliProvider.event_to_session_result()` returns `{...shared, provider_data: {<vendor-specific>}}`. Our Protocol stays thin; weird per-CLI metadata lives under `provider_data`. |+| **`CanonicalUsage` normalization** across vendor token-counting shapes (Anthropic vs Codex vs OpenAI handle cache tokens differently) | Hermes [`agent/usage_pricing.py`](D:/tmp/hermes-agent/agent/usage_pricing.py) | When `event_to_session_result` builds the result, normalize usage into `{input, output, cache_read, cache_write, reasoning, request_count}` — feed to existing [`services/pricing.py`](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/pricing.py) for cost. |+| **Heartbeat stale-detection** — track `(iteration, current_tool)` tuple; if unchanged for N cycles, log warning + dump diagnostic | Hermes [`tools/delegate_tool.py:1290-1357`](D:/tmp/hermes-agent/tools/delegate_tool.py) | Add to `AICliSession`: track `(events_seen, last_tool_use)` tuple per second; if frozen >60s, dump stderr buffer + last 5 events to `~/.claude-machina/logs/session-stale-<task_id>.log`. |+| **Timeout diagnostic dump** — on hard timeout, write stack/config/last-events to log file (issue ref: hermes #14726) | Hermes [`tools/delegate_tool.py:1437-1442`](D:/tmp/hermes-agent/tools/delegate_tool.py) | `wait_for_completion` `TimeoutError` branch writes `{argv, env_keys, last_20_events, stderr_tail}` to `~/.claude-machina/logs/session-timeout-<task_id>.log` before `stop()`. |+| **Binary resolution chain**: shipped assets → bundled npm pkg → PATH → npx fallback | Composio [`ts/packages/cli/src/services/run-subagent-acp.ts:66-124`](D:/tmp/composio/ts/packages/cli/src/services/run-subagent-acp.ts) | `provider.binary_path()` tries: (1) `<isolated_auth_dir>/npm/node_modules/.bin/<binary>`, (2) `shutil.which(<binary>)`, (3) `npx --yes <package_name>` shim. Mirrors what `claude_oauth.py` already partially does for Claude. |+| **Parent run-ID propagation via env var** for request tracing (lets MCP tool calls correlate to the parent batch) | Composio [`run.cmd.ts:494`](D:/tmp/composio/ts/packages/cli/src/commands/run.cmd.ts) (`COMPOSIO_CLI_PARENT_RUN_ID`) | Inject `MACHINA_PARENT_RUN_ID=<workflow_id>:<node_id>:<batch_token_short>` into every spawned CLI's env. MCP server logs this with each tool call. |+| **JSON-RPC framing pattern** for subprocess RPC (when MCP doesn't fit) | Hermes [`agent/copilot_acp_client.py:418-478`](D:/tmp/hermes-agent/agent/copilot_acp_client.py) | Reference for v2 if we ever wrap a CLI that doesn't speak stream-json. v1 uses stream-json directly (per-CLI native format). |++### What we deliberately don't copy++| Pattern | Source | Why we skip |+|---|---|---|+| **ACP adapter binaries** (`claude-code-acp`, `codex-acp`) — Composio spawns dedicated wrapper binaries that translate the CLI's native protocol into Zed's Agent Client Protocol | Composio [`run-subagent-acp.ts:533`](D:/tmp/composio/ts/packages/cli/src/services/run-subagent-acp.ts) | Adds a translation layer + extra dependency for each CLI. Stream-json is the official Anthropic format and Gemini's documented JSON output — direct invocation is cleaner. ACP is right for editors that need to speak one protocol to N agents; we're the opposite (one app, many CLIs). |+| **Naive `subprocess.Popen` lifecycle** without locks/kill-tree | Hermes [`copilot_acp_client.py:348-354`](D:/tmp/hermes-agent/agent/copilot_acp_client.py) — `terminate; sleep 2; kill` (parent only) | We already have `BaseProcessSupervisor` with locked idempotent start/stop, recursive `kill_tree`, Windows `CTRL_BREAK_EVENT`. Strictly better. |+| **MCP-over-stdio for structured output collection** — Composio runs an MCP server *inside* the child (not the parent) so children write structured output via tool call | Composio [`run-subagent-output-mcp.ts`](D:/tmp/composio/ts/packages/cli/src/services/run-subagent-output-mcp.ts) | Inverts the call direction. Useful for tools that don't have native structured output, but Claude/Codex/Gemini already emit stream-json. Skip. |+| **Task-scoped filesystem tracking** (no real isolation, just bookkeeping) | Hermes [`tools/delegate_tool.py:1403`](D:/tmp/hermes-agent/tools/delegate_tool.py) | Our git-worktree-per-task is process-level isolation enforced by the kernel (different cwd, different file system view). Strictly stronger. |+| **Tool-tag-based capability filtering** (`destructiveHint`, `readOnlyHint`) for risk gating | Composio [`ts/packages/core/src/types/mcp.types.ts`](D:/tmp/composio/ts/packages/core/src/types/mcp.types.ts) | Codex's native `--sandbox read-only|workspace-write|danger-full-access` already does this at the kernel level; tag-based hints are a soft signal that doesn't enforce. We use Codex's sandbox flag directly. |+| **No retry loop** — failed children return error, not retried | Hermes (same pattern) | Aligns with our continue-on-error settle policy. Match. |+| **`max_concurrent_children=3`** | Hermes [`hermes_cli/config.py:934`](D:/tmp/hermes-agent/hermes_cli/config.py) | We use 5; their 3 is also fine. Difference is cosmetic. |++### Explicit non-finding++Neither Hermes nor Composio writes a VSCode-style IDE lockfile (`~/.claude/ide/<pid>.lock` shape). Our v1 lockfile-discovery design is original among open-source orchestrators we surveyed and matches what the **VSCode Claude Code extension itself** does — i.e., we're cloning the upstream IDE pattern, not Hermes/Composio's pattern.++## v2 follow-up — Gemini activation++After v1 ships and stabilizes, Gemini activation is a small, mechanical PR:++1. Replace `providers/google_gemini.py` stub with full implementation (argv builder for `--prompt`/`--session-id`/`--resume`/`--yolo`/`--sandbox`; `parse_event` keyed on the `JsonStreamEventType` enum from `nonInteractiveCli.ts`; `is_final_event` = `type=="result"`; `event_to_session_result` extracts `session_id` + `stats.duration_ms`, sets `cost_usd=None`; `login_argv = ["gemini", "auth", "login"]`; `detect_auth_error` matches exit code 1 / `FatalAuthenticationError`).+2. Drop the `NotImplementedError` branch in `factory.py`.+3. Create `server/nodes/agent/gemini_cli_agent.py` with `Params.tasks: list[GeminiTaskSpec]`.+4. Add `geminiCliAgent` to `server/nodes/visuals.json`.+5. Add `gemini_cli` entry to `server/config/credential_providers.json` (extending `_cli_base`).+6. (No new WS handlers — generic `cli_login`/`cli_auth_status` already dispatch by provider field.)+7. Create `server/tests/nodes/test_gemini_cli_agent.py` (mirrors `test_codex_agent.py` shape; adds session/resume tests).+8. Vendored NDJSON fixture from the JS source for parser tests.++No changes to `protocol.py`, `types.py`, `session.py`, `pool.py`, `service.py`, `mcp_server.py`, `lockfile.py`, `login.py`, or `auth_status.py` — the v1 abstraction absorbs Gemini cleanly. **No `gemini_oauth.py` ever** — Gemini CLI handles its own auth via `~/.gemini/`.++## Risks / open considerations++- **`drain_stream` not used** — we override `_do_start`; drain task list replaced wholesale; `stop()` still cancels them via `self._drain_tasks` ([process.py:119-120](D:/startup/projects/machinaos-worktrees/credentials-scaling-v2/server/services/_supervisor/process.py#L119-L120)).+- **`--include-partial-messages`** assumes recent Claude CLI; detect via `claude --help | grep partial` once at module init, downgrade to plain `stream-json` if absent.+- **Codex auth** — Codex reads `~/.codex/auth.json` after `codex login`. We **don't** set `OPENAI_API_KEY` ourselves; the CLI handles its own auth. If the user prefers env-based auth (e.g., `CODEX_API_KEY_ENV_VAR`), they can set it in their shell — we inherit env via `os.environ`.+- **Gemini cost** — token stats only, no USD. v1 reports `cost_usd=null`; follow-up could compute via `services/pricing.py`.+- **Concurrent OAuth refresh** at sem=5 — the CLIs handle their own refresh internally. If a race causes a transient auth error, our session retries once on `detect_auth_error()` before failing. Acceptable for v1.+- **Native CLI auth = single user per host** — anyone with shell access on the host has access to the host's CLI auth, same as VSCode/`code .`. v1 doesn't isolate per-MachinaOs-user. If multi-tenant deployment becomes a need, v2 can introduce optional per-user `HOME` redirects (the old `~/.claude-machina/` pattern) gated by a config flag.+- **Discriminator + LLM tool schema** — Pydantic generates `oneOf`+`discriminator` JSON Schema, handled by most LLMs. Per-plugin `Params.tasks` is hard-typed to one variant so the fast-path schema stays clean.+- **Worktree leak on hard crash** — covered by startup sweep (mirrors VSCode lock-file behavior).+- **Three OAuth modules duplicate ~60% of `claude_oauth.py`** — defer factoring; wait for the third concrete to validate the abstraction.+- **Codex/Gemini final-event detection** is best-effort by event-type name; tests pin to vendored NDJSON streams to catch CLI schema drift.+- **MCP SDK is pre-1.0-stable** — pin a known-good version in `pyproject.toml`. Isolate the surface in `mcp_server.py` so an SDK breaking change touches one file.+- **Codex MCP discovery incomplete** — Codex CLI doesn't yet auto-read VSCode-style lockfiles. v1 still writes them (cheap); when Codex lands support, no code change needed.+- **Heavy bias toward Claude** — Claude provider is the reference implementation for the Protocol; if a Protocol method ever needs a feature only Claude has, prefer adding it to the Protocol with sensible defaults for other providers rather than designing for the lowest common denominator.+- **Gemini deferred but designed-for** — `GeminiTaskSpec` ships in `types.py` and the discriminator union in v1 so the JSON Schema fast-path doesn't need re-deriving when v2 lands. The factory raises `NotImplementedError` for gemini; tests assert this contract.+- **MCP multi-tenancy** — multiple parallel batches must not see each other's tokens or workspace. Per-batch token registry keyed by bearer token; missing/expired → 401. Tested.diff --git a/server/config/ai_cli_providers.json b/server/config/ai_cli_providers.jsonnew file mode 100644index 00000000..c38eab14--- /dev/null+++ b/server/config/ai_cli_providers.json@@ -0,0 +1,54 @@+{+ "claude": {+ "package_name": "@anthropic-ai/claude-code",+ "binary_name": "claude",+ "login_argv": ["claude", "login"],+ "auth_status_argv": ["claude", "--print", "-p", "ok"],+ "ide_lock_env_var": "CLAUDE_IDE_LOCK",+ "ide_lockfile_dir": "~/.claude/ide",+ "default_model": "claude-sonnet-4-6",+ "default_max_turns": 10,+ "default_max_budget_usd": 5.0,+ "default_allowed_tools": "Read,Edit,Bash,Glob,Grep,Write",+ "default_permission_mode": "acceptEdits",+ "supports": [+ "max_budget",+ "max_turns",+ "session_id",+ "resume",+ "mcp_runtime",+ "json_cost",+ "ide_lockfile"+ ]+ },+ "codex": {+ "package_name": "@openai/codex",+ "binary_name": "codex",+ "login_argv": ["codex", "login"],+ "auth_status_argv": ["codex", "--version"],+ "ide_lock_env_var": null,+ "ide_lockfile_dir": null,+ "default_model": "gpt-5.2-codex",+ "default_sandbox": "workspace-write",+ "default_ask_for_approval": "never",+ "supports": [+ "sandbox"+ ]+ },+ "gemini": {+ "package_name": "@google/gemini-cli",+ "binary_name": "gemini",+ "login_argv": ["gemini", "auth", "login"],+ "auth_status_argv": ["gemini", "--version"],+ "ide_lock_env_var": "GEMINI_IDE_LOCK",+ "ide_lockfile_dir": "<tmpdir>/gemini/ide",+ "default_model": "gemini-2.5-pro",+ "supports": [+ "session_id",+ "resume",+ "sandbox",+ "mcp_runtime",+ "ide_lockfile"+ ]+ }+}diff --git a/server/config/credential_providers.json b/server/config/credential_providers.jsonindex cd4b8d7a..4e787cca 100644--- a/server/config/credential_providers.json+++ b/server/config/credential_providers.json@@ -157,6 +157,51 @@"instructions": "Start the LM Studio local server from the Developer tab. API key is not required."},+ "_cli_base": {+ "_abstract": true,+ "category": "ai",+ "category_label": "AI Providers",+ "kind": "native_cli",+ "_kind_doc": "native_cli: MachinaOs does not store the credential. We trigger the CLI's own login flow (e.g. `claude login`) and detect logged-in state. The CLI handles its own ~/.claude/, ~/.codex/, ~/.gemini/ token storage.",+ "fields": [],+ "ws": {+ "login": "cli_login",+ "status": "cli_auth_status"+ }+ },++ "claude_code": {+ "extends": "_cli_base",+ "name": "Claude Code CLI",+ "color": "dracula.cyan",+ "icon_ref": "lobehub:Claude",+ "ws_provider_arg": "claude",+ "actions": [+ { "key": "login", "label": "Login with Claude", "theme_color": "cyan" },+ { "key": "status", "label": "Check Status", "theme_color": "purple" }+ ],+ "status_rows": [+ { "key": "logged_in", "label": "Logged in", "ok_field": "logged_in", "true_text": "Yes", "false_text": "No" },+ { "key": "available", "label": "CLI installed", "ok_field": "available", "true_text": "Yes", "false_text": "Run `npm install -g @anthropic-ai/claude-code`" }+ ]+ },++ "codex_cli": {+ "extends": "_cli_base",+ "name": "Codex CLI",+ "color": "dracula.orange",+ "icon_ref": "lobehub:OpenAI",+ "ws_provider_arg": "codex",+ "actions": [+ { "key": "login", "label": "Login with OpenAI", "theme_color": "orange" },+ { "key": "status", "label": "Check Status", "theme_color": "purple" }+ ],+ "status_rows": [+ { "key": "logged_in", "label": "Logged in", "ok_field": "logged_in", "true_text": "Yes", "false_text": "No" },+ { "key": "available", "label": "CLI installed", "ok_field": "available", "true_text": "Yes", "false_text": "Run `npm install -g @openai/codex`" }+ ]+ },+"whatsapp_personal": {"name": "WhatsApp Personal","category": "social",diff --git a/server/main.py b/server/main.pyindex 980eb8d4..be51e91f 100644--- a/server/main.py+++ b/server/main.py@@ -75,6 +75,10 @@ async def lifespan(app: FastAPI):# Side-effect import; the package __init__ walks its submodules.import nodes # noqa: F401+ # Side-effect import: services/cli_agent/__init__.py self-registers+ # `cli_login` / `cli_auth_status` into `services.ws_handler_registry`.+ import services.cli_agent # noqa: F401+# Wire dependency injectioncontainer.wire(modules=["routers.workflow",@@ -293,6 +297,21 @@ async def lifespan(app: FastAPI):else:_startup_log("[Temporal] Disabled")+ # Enter the CLI-agent MCP server's lifespan so its+ # StreamableHTTPSessionManager task group is initialised (Starlette+ # does NOT auto-propagate lifespans across `app.mount`). Stored on+ # `app.state` so shutdown can exit the context cleanly.+ cli_mcp_lifespan_ctx = None+ try:+ from services.cli_agent.mcp_server import get_mcp_app as _get_cli_mcp_app+ _cli_mcp_app = _get_cli_mcp_app()+ cli_mcp_lifespan_ctx = _cli_mcp_app.router.lifespan_context(_cli_mcp_app)+ await cli_mcp_lifespan_ctx.__aenter__()+ app.state.cli_mcp_lifespan_ctx = cli_mcp_lifespan_ctx+ _startup_log("[CLI MCP] StreamableHTTP session manager initialised")+ except Exception as exc:+ logger.warning("[CLI MCP] lifespan init failed: %s", exc)+_startup_log("All services initialized")print("Application startup complete", flush=True)yield@@ -361,6 +380,15 @@ async def lifespan(app: FastAPI):logger.info("Execution recovery sweeper stopped")shutdown_scheduler() # Stop APScheduler++ # Exit the CLI-agent MCP server's lifespan+ cli_mcp_lifespan_ctx = getattr(app.state, "cli_mcp_lifespan_ctx", None)+ if cli_mcp_lifespan_ctx is not None:+ try:+ await cli_mcp_lifespan_ctx.__aexit__(None, None, None)+ except Exception as exc:+ logger.debug("[CLI MCP] lifespan shutdown: %s", exc)+await container.cache().shutdown()await container.database().shutdown()logger.info("Services shutdown complete")@@ -430,6 +458,38 @@ app.include_router(credentials.router) # Credentials panel - lazy per-tile iconapp.include_router(schemas.router) # Per-node output schema endpoint (GET /api/schemas/nodes/{type}.json)+# ---------------------------------------------------------------------------+# CLI agent IDE MCP server (VSCode pattern)+# ---------------------------------------------------------------------------+#+# Spawned Claude Code / Codex CLI sessions auto-discover this server via+# a per-batch lockfile (~/.claude/ide/<pid>.lock) and call tools like+# mcp__machina__getSkill / getCredential / broadcastLog over MCP-over-HTTP.+# Bearer-token auth scoped per batch (see services/cli_agent/mcp_server.py).+try:+ from services.cli_agent.mcp_server import get_mcp_app as _get_cli_mcp_app+ _cli_mcp_app = _get_cli_mcp_app()+ app.mount("/mcp/ide", _cli_mcp_app)+ logger.info("[main] mounted CLI agent MCP server at /mcp/ide")+except Exception as exc: # pragma: no cover — defensive; MCP must not block startup+ logger.warning("[main] failed to mount CLI agent MCP server: %s", exc)+++# Stale lockfile sweep on startup — mirrors VSCode's behavior. PIDs in+# leftover lockfiles that are no longer alive get cleaned up.[email protected]_event("startup")+async def _sweep_cli_lockfiles_on_startup() -> None:+ try:+ from services.cli_agent.config import list_provider_names, get_provider_config+ from services.cli_agent.lockfile import sweep_stale_lockfiles+ for name in list_provider_names():+ cfg = get_provider_config(name)+ if cfg and cfg.ide_lockfile_dir:+ sweep_stale_lockfiles(cfg.ide_lockfile_dir)+ except Exception as exc:+ logger.debug("[main] CLI lockfile sweep failed: %s", exc)++@app.get("/health")async def health_check():"""Detailed health check with resource monitoring."""diff --git a/server/nodes/agent/claude_code_agent.py b/server/nodes/agent/claude_code_agent.pyindex a5bdfb89..51c3d00f 100644--- a/server/nodes/agent/claude_code_agent.py+++ b/server/nodes/agent/claude_code_agent.py@@ -1,48 +1,110 @@-"""Claude Code Agent — Wave 11.E.3 inlined.+"""Claude Code Agent — multi-instance via `AICliService`.-Shells out to the local ``claude`` CLI binary (isolated auth via-``services/claude_oauth.py``). Status updates broadcast on entry +-exit; skill instructions concatenated into the system prompt.+Refactored from the single-task shim. ``Params.tasks`` is a list of+``ClaudeTaskSpec``; each task gets its own git worktree, its own session+id, and runs in parallel under a 5-way semaphore.++Back-compat: an empty `tasks` array with a non-empty `prompt` falls back+to a single-task batch — preserves the legacy single-shot UX.++Skill instructions connected via the parent node's ``input-skill`` handle+are exposed to the CLI through the MCP server's ``listSkills`` /+``getSkill`` tools (NOT concatenated into the system prompt — the+agent fetches them on-demand). Connected skill names ARE collected here+and passed to ``run_batch()`` so the MCP server can scope its responses."""from __future__ import annotationsimport timefrom datetime import datetime-from typing import Any, Optional+from pathlib import Path+from typing import Any, List, Optional-from pydantic import ConfigDict, Field+from pydantic import BaseModel, ConfigDict, Fieldfrom core.logging import get_loggerfrom services.plugin import ActionNode, NodeContext, Operation, TaskQueuefrom services.plugin.edge_walker import collect_agent_connectionsfrom ._handles import STD_AGENT_HINTS, std_agent_handles-from ._specialized import SpecializedAgentOutput, SpecializedAgentParamslogger = get_logger(__name__)-class ClaudeCodeAgentParams(SpecializedAgentParams):- """Adds CLI-specific flags the handler body reads from payload."""+# Late import the cli_agent types so node_registry import doesn't pull+# the whole MCP / pool stack at module import time.+from services.cli_agent import ClaudeTaskSpec # noqa: E402- max_turns: int = Field(default=10, ge=1)- max_budget_usd: float = Field(default=5.0, ge=0.0)- system_prompt: Optional[str] = Field(default=None)- working_directory: Optional[str] = Field(default=None)- allowed_tools: str = Field(- default="Read,Edit,Bash,Glob,Grep,Write",- )++class ClaudeCodeAgentParams(BaseModel):+ """Multi-task batch parameters for Claude Code.++ Two paths:+ 1. ``tasks=[...]`` — explicit list, runs N in parallel+ 2. ``tasks=[]`` + legacy ``prompt`` — synthesises a single-task batch+ """++ tasks: List[ClaudeTaskSpec] = Field(+ default_factory=list,+ description="List of Claude tasks to run in parallel (max 5 concurrent).",+ json_schema_extra={"rows": 1},+ )+ # Legacy single-task fallback ----------------------------------------+ prompt: str = Field(+ default="",+ description="Legacy: single-prompt fallback used only when "+ "`tasks` is empty.",+ json_schema_extra={"rows": 4, "placeholder": "Or use the tasks array..."},+ )+ model: str = Field(+ default="claude-sonnet-4-6",+ description="Default model for tasks that don't override it.",+ )+ system_prompt: Optional[str] = Field(default=None, json_schema_extra={"rows": 3})+ working_directory: Optional[str] = Field(+ default=None,+ description="Git repo root. Defaults to the workflow's workspace dir.",+ )+ max_parallel: int = Field(+ default=5, ge=1, le=20,+ description="Concurrency cap.",+ )+ allowed_credentials: List[str] = Field(+ default_factory=list,+ description="Credential names the CLI is permitted to fetch via MCP.",+ )model_config = ConfigDict(extra="ignore")+class ClaudeCodeAgentOutput(BaseModel):+ """Aggregated batch output."""+ success: bool = True+ n_tasks: int = 0+ n_succeeded: int = 0+ n_failed: int = 0+ total_cost_usd: Optional[float] = None+ wall_clock_ms: int = 0+ tasks: List[Any] = Field(default_factory=list)+ provider: str = "claude"+ timestamp: Optional[str] = None+ # Legacy single-task fields, populated when n_tasks==1 for back-compat:+ response: Optional[str] = None+ session_id: Optional[str] = None+ cost_usd: Optional[float] = None+ model_config = ConfigDict(extra="allow")++class ClaudeCodeAgentNode(ActionNode):type = "claude_code_agent"display_name = "Claude Code"subtitle = "Agentic Coding"group = ("agent",)- description = "Claude Code CLI as a specialized agent"+ description = (+ "Run N parallel Claude Code CLI sessions over a list of tasks. "+ "Each task is isolated in its own git worktree."+ )component_kind = "agent"handles = std_agent_handles()ui_hints = STD_AGENT_HINTS@@ -50,84 +112,160 @@ class ClaudeCodeAgentNode(ActionNode):task_queue = TaskQueue.AI_HEAVYParams = ClaudeCodeAgentParams- Output = SpecializedAgentOutput+ Output = ClaudeCodeAgentOutput- @Operation("execute", cost={"service": "claude_code_agent", "action": "run", "count": 1})- async def execute_op(self, ctx: NodeContext, params: ClaudeCodeAgentParams) -> Any:+ @Operation(+ "execute",+ cost={"service": "claude_code_agent", "action": "run", "count": 1},+ )+ async def execute_op(+ self, ctx: NodeContext, params: ClaudeCodeAgentParams,+ ) -> Any:from core.container import container- from services.claude_code_service import get_claude_code_service+ from services.cli_agent.service import get_ai_cli_service+ from services.cli_agent.types import session_result_to_modelfrom services.status_broadcaster import get_status_broadcasterstart_time = time.time()broadcaster = get_status_broadcaster()workflow_id = ctx.workflow_idnode_id = ctx.node_id- payload = params.model_dump()await broadcaster.update_node_status(node_id, "executing",- {"message": "Starting Claude Code..."},+ {"message": "Starting Claude Code batch..."},workflow_id=workflow_id,)- # Resolve prompt: explicit param wins; otherwise pull text from- # the upstream node connected to input-main.- prompt = payload.get("prompt", "")- if not prompt:- for edge in ctx.raw.get("edges", []):- if edge.get("target") == node_id and edge.get("targetHandle") in ("input-main", None):- src = ctx.raw.get("outputs", {}).get(edge.get("source"), {})- if isinstance(src, dict):- prompt = src.get("message") or src.get("text") or src.get("content") or str(src)- elif src:- prompt = str(src)- if prompt:- break- if not prompt:- raise RuntimeError("No prompt provided")+ tasks = list(params.tasks)+ if not tasks:+ # Legacy fallback — synthesise a single task from the+ # top-level `prompt`/`model`/`system_prompt`.+ prompt = params.prompt or self._infer_prompt_from_inputs(ctx, node_id)+ if not prompt:+ raise RuntimeError(+ "claude_code_agent: provide either `tasks` or `prompt`"+ )+ tasks = [+ ClaudeTaskSpec(+ prompt=prompt,+ model=params.model,+ system_prompt=params.system_prompt,+ ),+ ]+ else:+ # Apply node-level defaults to tasks that don't override+ for i, t in enumerate(tasks):+ changed: dict = {}+ if not t.model and params.model:+ changed["model"] = params.model+ if not t.system_prompt and params.system_prompt:+ changed["system_prompt"] = params.system_prompt+ if changed:+ tasks[i] = t.model_copy(update=changed)+ # Collect connected skills (the MCP server's listSkills/getSkill+ # tools scope to these names).database = container.database()- _, skill_data, _, _, _ = await collect_agent_connections(node_id, ctx.raw, database)+ _, skill_data, _, _, _ = await collect_agent_connections(+ node_id, ctx.raw, database,+ )+ connected_skills = [+ s.get("skill_name") or s.get("label")+ for s in skill_data+ if s.get("skill_name") or s.get("label")+ ]- system_parts = []- if payload.get("system_prompt"):- system_parts.append(payload["system_prompt"])- for skill in skill_data:- instr = skill.get("parameters", {}).get("instructions", "")- if instr:- system_parts.append(instr)+ # Workspace dir — workflow.py injects this into context+ workspace_dir = ctx.raw.get("workspace_dir") or params.working_directory+ if workspace_dir is None:+ from core.config import Settings+ workspace_dir = Path(Settings().workspace_base_resolved) / (+ workflow_id or "default"+ )+ workspace_dir = Path(workspace_dir)- model = payload.get("model", "claude-sonnet-4-6")- await broadcaster.update_node_status(- node_id, "executing",- {"message": f"Running Claude Code ({model})..."},- workflow_id=workflow_id,+ repo_root = (+ Path(params.working_directory) if params.working_directory else None)- service = get_claude_code_service()- data = await service.execute(- prompt=prompt,+ svc = get_ai_cli_service()+ result = await svc.run_batch(+ "claude",+ tasks=tasks,node_id=node_id,- model=model,- cwd=payload.get("working_directory") or None,- allowed_tools=payload.get("allowed_tools", "Read,Edit,Bash,Glob,Grep,Write"),- max_turns=int(payload.get("max_turns", 10)),- max_budget_usd=float(payload.get("max_budget_usd", 5.0)),- system_prompt="\n\n".join(system_parts) if system_parts else None,+ workflow_id=workflow_id or "",+ workspace_dir=workspace_dir,+ broadcaster=broadcaster,+ repo_root=repo_root,+ connected_skill_names=connected_skills,+ allowed_credentials=params.allowed_credentials,+ max_parallel=params.max_parallel,+ )++ elapsed = time.time() - start_time+ logger.debug(+ "[claude_code_agent] node=%s tasks=%d ok=%d fail=%d elapsed=%.2fs",+ node_id, result.n_tasks, result.n_succeeded, result.n_failed, elapsed,)await broadcaster.update_node_status(- node_id, "success",- {"message": "Claude Code completed"},+ node_id, "success" if result.n_failed == 0 else "warning",+ {+ "message": (+ f"Batch complete: {result.n_succeeded}/{result.n_tasks} "+ f"succeeded"+ ),+ "n_tasks": result.n_tasks,+ "n_succeeded": result.n_succeeded,+ "n_failed": result.n_failed,+ },workflow_id=workflow_id,)- logger.debug("[ClaudeCode] node=%s elapsed=%.2fs", node_id, time.time() - start_time)++ task_models = [session_result_to_model(t).model_dump() for t in result.tasks]++ # Legacy single-task convenience fields+ legacy_response = None+ legacy_session_id = None+ legacy_cost = None+ if len(result.tasks) == 1:+ legacy_response = result.tasks[0].response+ legacy_session_id = result.tasks[0].session_id+ legacy_cost = result.tasks[0].cost_usdreturn {- "response": data.get("result", ""),- "model": model,- "provider": "anthropic",- "session_id": data.get("session_id", ""),- "usage": data.get("usage", {}),- "timestamp": datetime.now().isoformat(),+ "success": result.n_failed == 0,+ "n_tasks": result.n_tasks,+ "n_succeeded": result.n_succeeded,+ "n_failed": result.n_failed,+ "total_cost_usd": result.total_cost_usd,+ "wall_clock_ms": result.wall_clock_ms,+ "tasks": task_models,+ "provider": result.provider,+ "timestamp": result.timestamp or datetime.now().isoformat(),+ "response": legacy_response,+ "session_id": legacy_session_id,+ "cost_usd": legacy_cost,}++ @staticmethod+ def _infer_prompt_from_inputs(ctx: NodeContext, node_id: str) -> str:+ """Fall back to whatever the upstream node connected to+ ``input-main`` produced."""+ for edge in ctx.raw.get("edges", []):+ if edge.get("target") != node_id:+ continue+ handle = edge.get("targetHandle")+ if handle not in ("input-main", None):+ continue+ src = ctx.raw.get("outputs", {}).get(edge.get("source"), {})+ if isinstance(src, dict):+ for k in ("message", "text", "content", "prompt"):+ val = src.get(k)+ if val:+ return str(val)+ return str(src)+ elif src:+ return str(src)+ return ""diff --git a/server/nodes/agent/codex_agent.py b/server/nodes/agent/codex_agent.pynew file mode 100644index 00000000..7d17c888--- /dev/null+++ b/server/nodes/agent/codex_agent.py@@ -0,0 +1,239 @@+"""OpenAI Codex CLI agent — multi-instance via `AICliService`.++Sandbox-first companion to `claude_code_agent`. Each task gets its own+git worktree; sandbox is enforced by Codex itself, not by us.++Codex has no session/resume/budget/turns surface — `CodexTaskSpec`+exposes only `sandbox` + `ask_for_approval` as task-level overrides.+"""++from __future__ import annotations++import time+from datetime import datetime+from pathlib import Path+from typing import Any, List, Optional++from pydantic import BaseModel, ConfigDict, Field++from core.logging import get_logger+from services.plugin import ActionNode, NodeContext, Operation, TaskQueue+from services.plugin.edge_walker import collect_agent_connections++from ._handles import STD_AGENT_HINTS, std_agent_handles++logger = get_logger(__name__)+++from services.cli_agent import CodexTaskSpec # noqa: E402+++class CodexAgentParams(BaseModel):+ """Multi-task batch parameters for Codex."""++ tasks: List[CodexTaskSpec] = Field(+ default_factory=list,+ description="List of Codex tasks to run in parallel (max 5 concurrent).",+ json_schema_extra={"rows": 1},+ )+ prompt: str = Field(+ default="",+ description="Legacy: single-prompt fallback used only when "+ "`tasks` is empty.",+ json_schema_extra={"rows": 4, "placeholder": "Or use the tasks array..."},+ )+ model: str = Field(+ default="gpt-5.2-codex",+ description="Default model for tasks that don't override it.",+ )+ sandbox: str = Field(+ default="workspace-write",+ description="Default sandbox for tasks that don't override it. "+ "One of: read-only | workspace-write | danger-full-access.",+ )+ ask_for_approval: str = Field(+ default="never",+ description="Default approval mode: untrusted | on-request | never.",+ )+ system_prompt: Optional[str] = Field(default=None, json_schema_extra={"rows": 3})+ working_directory: Optional[str] = None+ max_parallel: int = Field(default=5, ge=1, le=20)+ allowed_credentials: List[str] = Field(default_factory=list)++ model_config = ConfigDict(extra="ignore")+++class CodexAgentOutput(BaseModel):+ success: bool = True+ n_tasks: int = 0+ n_succeeded: int = 0+ n_failed: int = 0+ wall_clock_ms: int = 0+ tasks: List[Any] = Field(default_factory=list)+ provider: str = "codex"+ timestamp: Optional[str] = None+ # Legacy single-task convenience+ response: Optional[str] = None+ model_config = ConfigDict(extra="allow")+++class CodexAgentNode(ActionNode):+ type = "codex_agent"+ display_name = "Codex"+ subtitle = "Sandboxed Coding"+ group = ("agent",)+ description = (+ "Run N parallel OpenAI Codex CLI sessions. Sandbox enforced "+ "by Codex itself; per-task git worktree isolation."+ )+ component_kind = "agent"+ handles = std_agent_handles()+ ui_hints = STD_AGENT_HINTS+ annotations = {"destructive": True, "readonly": False, "open_world": True}+ task_queue = TaskQueue.AI_HEAVY++ Params = CodexAgentParams+ Output = CodexAgentOutput++ @Operation(+ "execute",+ cost={"service": "codex_agent", "action": "run", "count": 1},+ )+ async def execute_op(+ self, ctx: NodeContext, params: CodexAgentParams,+ ) -> Any:+ from core.container import container+ from services.cli_agent.service import get_ai_cli_service+ from services.cli_agent.types import session_result_to_model+ from services.status_broadcaster import get_status_broadcaster++ start_time = time.time()+ broadcaster = get_status_broadcaster()+ workflow_id = ctx.workflow_id+ node_id = ctx.node_id++ await broadcaster.update_node_status(+ node_id, "executing",+ {"message": "Starting Codex batch..."},+ workflow_id=workflow_id,+ )++ tasks = list(params.tasks)+ if not tasks:+ prompt = params.prompt or self._infer_prompt_from_inputs(ctx, node_id)+ if not prompt:+ raise RuntimeError(+ "codex_agent: provide either `tasks` or `prompt`"+ )+ tasks = [+ CodexTaskSpec(+ prompt=prompt,+ model=params.model,+ sandbox=params.sandbox, # type: ignore[arg-type]+ ask_for_approval=params.ask_for_approval, # type: ignore[arg-type]+ system_prompt=params.system_prompt,+ ),+ ]+ else:+ for i, t in enumerate(tasks):+ changed: dict = {}+ if not t.model and params.model:+ changed["model"] = params.model+ if not t.system_prompt and params.system_prompt:+ changed["system_prompt"] = params.system_prompt+ if changed:+ tasks[i] = t.model_copy(update=changed)++ database = container.database()+ _, skill_data, _, _, _ = await collect_agent_connections(+ node_id, ctx.raw, database,+ )+ connected_skills = [+ s.get("skill_name") or s.get("label")+ for s in skill_data+ if s.get("skill_name") or s.get("label")+ ]++ workspace_dir = ctx.raw.get("workspace_dir") or params.working_directory+ if workspace_dir is None:+ from core.config import Settings+ workspace_dir = Path(Settings().workspace_base_resolved) / (+ workflow_id or "default"+ )+ workspace_dir = Path(workspace_dir)++ repo_root = (+ Path(params.working_directory) if params.working_directory else None+ )++ svc = get_ai_cli_service()+ result = await svc.run_batch(+ "codex",+ tasks=tasks,+ node_id=node_id,+ workflow_id=workflow_id or "",+ workspace_dir=workspace_dir,+ broadcaster=broadcaster,+ repo_root=repo_root,+ connected_skill_names=connected_skills,+ allowed_credentials=params.allowed_credentials,+ max_parallel=params.max_parallel,+ )++ elapsed = time.time() - start_time+ logger.debug(+ "[codex_agent] node=%s tasks=%d ok=%d fail=%d elapsed=%.2fs",+ node_id, result.n_tasks, result.n_succeeded, result.n_failed, elapsed,+ )++ await broadcaster.update_node_status(+ node_id, "success" if result.n_failed == 0 else "warning",+ {+ "message": (+ f"Batch complete: {result.n_succeeded}/{result.n_tasks} "+ f"succeeded"+ ),+ "n_tasks": result.n_tasks,+ "n_succeeded": result.n_succeeded,+ "n_failed": result.n_failed,+ },+ workflow_id=workflow_id,+ )++ task_models = [session_result_to_model(t).model_dump() for t in result.tasks]++ legacy_response = (+ result.tasks[0].response if len(result.tasks) == 1 else None+ )++ return {+ "success": result.n_failed == 0,+ "n_tasks": result.n_tasks,+ "n_succeeded": result.n_succeeded,+ "n_failed": result.n_failed,+ "total_cost_usd": result.total_cost_usd, # always None for Codex+ "wall_clock_ms": result.wall_clock_ms,+ "tasks": task_models,+ "provider": result.provider,+ "timestamp": result.timestamp or datetime.now().isoformat(),+ "response": legacy_response,+ }++ @staticmethod+ def _infer_prompt_from_inputs(ctx: NodeContext, node_id: str) -> str:+ for edge in ctx.raw.get("edges", []):+ if edge.get("target") != node_id:+ continue+ handle = edge.get("targetHandle")+ if handle not in ("input-main", None):+ continue+ src = ctx.raw.get("outputs", {}).get(edge.get("source"), {})+ if isinstance(src, dict):+ for k in ("message", "text", "content", "prompt"):+ val = src.get(k)+ if val:+ return str(val)+ return str(src)+ elif src:+ return str(src)+ return ""diff --git a/server/nodes/visuals.json b/server/nodes/visuals.jsonindex 8712c8fa..984d5a97 100644--- a/server/nodes/visuals.json+++ b/server/nodes/visuals.json@@ -89,6 +89,10 @@"icon": "lobehub:Claude","color": "#8be9fd"},+ "codex_agent": {+ "icon": "lobehub:OpenAI",+ "color": "#ffb86c"+ },"coding_agent": {"icon": "💻","color": "#8be9fd"diff --git a/server/pyproject.toml b/server/pyproject.tomlindex d20e6fe6..cd5bac6e 100644--- a/server/pyproject.toml+++ b/server/pyproject.toml@@ -111,6 +111,11 @@ dependencies = [# RLM (Recursive Language Models) agent"rlms>=0.1.1",++ # MCP (Model Context Protocol) — host server for spawned CLI sessions+ # to discover and call MachinaOs tools (mcp__machina__*) the same way+ # VSCode's Claude Code extension does.+ "mcp>=1.0.0",][project.optional-dependencies]diff --git a/server/routers/websocket.py b/server/routers/websocket.pyindex b812499a..17f11d67 100644--- a/server/routers/websocket.py+++ b/server/routers/websocket.py@@ -1391,6 +1391,12 @@ async def handle_claude_oauth_status(data: Dict[str, Any], websocket: WebSocket)return get_claude_credentials()+# NOTE: `cli_login` / `cli_auth_status` handlers are owned by+# `services/cli_agent/_handlers.py` and self-registered into+# `services.ws_handler_registry` on package import — no entries needed+# here. See `services/cli_agent/__init__.py`.++# ============================================================================# Twitter OAuth Handlers# ============================================================================diff --git a/server/services/claude_code_service.py b/server/services/claude_code_service.pyindex 5d17bc24..041b5aa5 100644--- a/server/services/claude_code_service.py+++ b/server/services/claude_code_service.py@@ -1,30 +1,40 @@-"""Claude Code service - manages CLI execution with session persistence."""+"""Legacy single-task Claude Code shim.++Kept for back-compat with any caller that still imports+`get_claude_code_service()`. New code should call+`services.cli_agent.AICliService.run_batch("claude", ...)` directly via+the `claude_code_agent` plugin.++This module:+ - Builds a single ``ClaudeTaskSpec`` from kwargs+ - Calls ``AICliService.run_batch("claude", ...)``+ - Adapts the ``BatchResult`` back into the dict shape the legacy+ callers expected++Eventually deletable once all imports point at ``cli_agent.service``.+The hardcoded 300s `wait_for` is gone — ``timeout_seconds`` is now per+task and configurable.+"""++from __future__ import annotations-import asyncio-import jsonimport os-import shutil-from typing import Any, Dict, List, Optional+from pathlib import Path+from typing import Any, Dict, Optional+from core.config import Settingsfrom core.logging import get_logger-logger = get_logger(__name__)-+from services.cli_agent import ClaudeTaskSpec+from services.cli_agent.service import get_ai_cli_service-def _find_claude_cmd() -> List[str]:- """Find claude CLI cross-platform."""- if shutil.which("claude"):- return ["claude"]- npx = shutil.which("npx")- if npx:- return [npx, "-y", "@anthropic-ai/claude-code"]- raise FileNotFoundError("Neither 'claude' nor 'npx' found in PATH")+logger = get_logger(__name__)class ClaudeCodeService:- """Executes tasks via Claude Code CLI with session tracking."""+ """Thin shim that adapts to the new AICliService."""- def __init__(self):+ def __init__(self) -> None:self._session_map: Dict[str, str] = {} # node_id -> session_idasync def execute(@@ -37,57 +47,66 @@ class ClaudeCodeService:max_turns: int = 10,max_budget_usd: float = 5.0,system_prompt: Optional[str] = None,+ timeout_seconds: int = 600,) -> Dict[str, Any]:- """Run claude CLI and return parsed JSON result."""- cmd = _find_claude_cmd() + [- "-p", prompt,- "--output-format", "json",- "--model", model,- "--max-turns", str(max_turns),- "--allowedTools", allowed_tools,- ]-- if max_budget_usd > 0:- cmd += ["--max-budget-usd", str(max_budget_usd)]-- # Resume existing session for this node- session_id = self._session_map.get(node_id)- if session_id:- cmd += ["--resume", session_id]-- if system_prompt:- cmd += ["--append-system-prompt", system_prompt]-+ """Run a single Claude Code task. Returns legacy dict shape."""if not cwd:- from core.config import Settings- cwd = os.path.join(Settings().workspace_base_resolved, 'default')+ cwd = os.path.join(Settings().workspace_base_resolved, "default")os.makedirs(cwd, exist_ok=True)- work_dir = cwd- proc = await asyncio.create_subprocess_exec(- *cmd,- stdout=asyncio.subprocess.PIPE,- stderr=asyncio.subprocess.PIPE,- cwd=work_dir,+ # Resume the prior session for this node if we have one+ resume_session_id = self._session_map.get(node_id) if node_id else None++ task = ClaudeTaskSpec(+ task_id=f"legacy_{node_id}" if node_id else "legacy",+ prompt=prompt,+ model=model,+ max_turns=max_turns,+ max_budget_usd=max_budget_usd,+ allowed_tools=allowed_tools,+ system_prompt=system_prompt,+ timeout_seconds=timeout_seconds,+ resume_session_id=resume_session_id,)- stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=300)-- if proc.returncode != 0:- err = stderr.decode("utf-8", errors="replace").strip()- logger.error("Claude Code failed (exit %d): %s", proc.returncode, err)- raise RuntimeError(err or f"Exit code {proc.returncode}")- raw = stdout.decode("utf-8", errors="replace").strip()- try:- data = json.loads(raw)- except json.JSONDecodeError:- data = {"result": raw}-- # Track session for future resume- if data.get("session_id") and node_id:- self._session_map[node_id] = data["session_id"]+ svc = get_ai_cli_service()+ workspace_dir = Path(cwd)+ repo_root = self._find_git_repo(workspace_dir)++ result = await svc.run_batch(+ "claude",+ tasks=[task],+ node_id=node_id or "legacy_node",+ workflow_id="legacy_workflow",+ workspace_dir=workspace_dir,+ broadcaster=None,+ repo_root=repo_root,+ )- return data+ if not result.tasks:+ raise RuntimeError("AICliService returned empty batch")++ sr = result.tasks[0]+ if not sr.success:+ raise RuntimeError(sr.error or "claude_code_service: task failed")++ # Persist session_id for future resume+ if sr.session_id and node_id:+ self._session_map[node_id] = sr.session_id++ return {+ "result": sr.response,+ "session_id": sr.session_id or "",+ "total_cost_usd": sr.cost_usd,+ "duration_ms": sr.duration_ms,+ "num_turns": sr.num_turns,+ "usage": {+ "input_tokens": sr.canonical_usage.input_tokens,+ "output_tokens": sr.canonical_usage.output_tokens,+ "cache_creation_input_tokens": sr.canonical_usage.cache_write,+ "cache_read_input_tokens": sr.canonical_usage.cache_read,+ },+ }def get_session_id(self, node_id: str) -> Optional[str]:return self._session_map.get(node_id)@@ -95,6 +114,18 @@ class ClaudeCodeService:def clear_session(self, node_id: str) -> None:self._session_map.pop(node_id, None)+ @staticmethod+ def _find_git_repo(start: Path) -> Optional[Path]:+ """Walk up from `start` looking for a `.git` directory."""+ cur = start.resolve()+ for _ in range(8):+ if (cur / ".git").exists():+ return cur+ if cur.parent == cur:+ return None+ cur = cur.parent+ return None+_instance: Optional[ClaudeCodeService] = Nonediff --git a/server/services/cli_agent/__init__.py b/server/services/cli_agent/__init__.pynew file mode 100644index 00000000..6970b53d--- /dev/null+++ b/server/services/cli_agent/__init__.py@@ -0,0 +1,73 @@+"""AI CLI agent framework — multi-provider, multi-instance, VSCode-pattern.++Spawns N parallel Claude Code / Codex CLI sessions per workflow node,+each isolated in its own git worktree, with a shared MCP server hosting+MachinaOs tools (``mcp__machina__*``) discovered via VSCode-style+lockfile.++Self-registers two WebSocket handlers (``cli_login`` /+``cli_auth_status``) into ``services.ws_handler_registry`` on import,+mirroring the telegram / whatsapp plugin folder pattern. The router+does not need to know about us by name.+"""++from __future__ import annotations++from services.cli_agent.cli_auth import check_auth, run_native_login+from services.cli_agent.factory import (+ ALL_PROVIDERS,+ SUPPORTED_PROVIDERS,+ create_cli_provider,+ is_supported,+)+from services.cli_agent.protocol import (+ AICliProvider,+ BatchResult,+ CanonicalUsage,+ SessionResult,+)+from services.cli_agent.types import (+ AICliTaskSpec,+ BaseAICliTaskSpec,+ BatchResultModel,+ BatchSummary,+ ClaudeTaskSpec,+ CodexTaskSpec,+ GeminiTaskSpec,+ SessionResultModel,+ session_result_to_model,+)++# --- self-registration on import -------------------------------------------+from services.ws_handler_registry import register_ws_handlers+from services.cli_agent._handlers import WS_HANDLERS++register_ws_handlers(WS_HANDLERS)+++__all__ = [+ # Factory+ "create_cli_provider",+ "is_supported",+ "SUPPORTED_PROVIDERS",+ "ALL_PROVIDERS",+ # Protocol + dataclasses+ "AICliProvider",+ "CanonicalUsage",+ "SessionResult",+ "BatchResult",+ # Pydantic specs+ "BaseAICliTaskSpec",+ "ClaudeTaskSpec",+ "CodexTaskSpec",+ "GeminiTaskSpec",+ "AICliTaskSpec",+ # Pydantic result models+ "SessionResultModel",+ "BatchSummary",+ "BatchResultModel",+ "session_result_to_model",+ # Native auth helpers (also used by the router for Credentials Modal)+ "run_native_login",+ "check_auth",+]diff --git a/server/services/cli_agent/_handlers.py b/server/services/cli_agent/_handlers.pynew file mode 100644index 00000000..26151cbb--- /dev/null+++ b/server/services/cli_agent/_handlers.py@@ -0,0 +1,67 @@+"""WebSocket handlers for the CLI agent framework.++Self-registered into ``services.ws_handler_registry`` from+``services/cli_agent/__init__.py``. The router does not import this+module directly — it discovers handlers via the registry at dispatch+time, mirroring the telegram / whatsapp plugin pattern.++Two generic handlers (one pair total — dispatched by the ``provider``+field):++- ``cli_login`` → spawns ``<provider> login``+- ``cli_auth_status`` → probes whether the CLI is logged in++Both replace what would have been per-provider OAuth handlers+(``codex_oauth_login`` etc.).+"""++from __future__ import annotations++from typing import Any, Awaitable, Callable, Dict++from fastapi import WebSocket++from services.cli_agent.cli_auth import check_auth, run_native_login+++async def handle_cli_login(+ data: Dict[str, Any],+ websocket: WebSocket, # noqa: ARG001 — registry signature+) -> Dict[str, Any]:+ """Trigger the CLI's native login flow.++ Payload: ``{provider: "claude" | "codex" | "gemini"}``.+ The CLI handles its own credential storage; we only spawn the flow.+ """+ provider = (data.get("provider") or "").strip().lower()+ if not provider:+ return {"success": False, "error": "missing 'provider' field"}+ return await run_native_login(provider)+++async def handle_cli_auth_status(+ data: Dict[str, Any],+ websocket: WebSocket, # noqa: ARG001 — registry signature+) -> Dict[str, Any]:+ """Check whether the CLI is logged in.++ Payload: ``{provider: "claude" | "codex" | "gemini"}``.+ Returns ``{logged_in, available, hint, exit_code, error}``.+ """+ provider = (data.get("provider") or "").strip().lower()+ if not provider:+ return {+ "logged_in": False, "available": False,+ "error": "missing 'provider' field",+ }+ return await check_auth(provider)+++# Registry payload — `services/cli_agent/__init__.py` registers these+# into `services.ws_handler_registry` on package import.+WSHandler = Callable[[Dict[str, Any], WebSocket], Awaitable[Dict[str, Any]]]++WS_HANDLERS: Dict[str, WSHandler] = {+ "cli_login": handle_cli_login,+ "cli_auth_status": handle_cli_auth_status,+}diff --git a/server/services/cli_agent/cli_auth.py b/server/services/cli_agent/cli_auth.pynew file mode 100644index 00000000..bca365e3--- /dev/null+++ b/server/services/cli_agent/cli_auth.py@@ -0,0 +1,252 @@+"""Native CLI auth — login + status, no token wrapping.++Replaces what would have been per-provider OAuth modules+(``codex_oauth.py``, ``gemini_oauth.py``). Each CLI ships a working+``<provider> login`` flow that handles the browser/device flow and+stores credentials in its native location (``~/.claude/``,+``~/.codex/``, ``~/.gemini/``). MachinaOs only triggers the flow and+detects the resulting state.++Two public coroutines:++- :func:`run_native_login` — spawns ``<provider> login`` (interactive+ by default; the user completes the browser/device flow directly).+- :func:`check_auth` — runs ``<provider> --version`` (or whatever+ ``auth_status_argv()`` returns), feeds output into+ ``provider.detect_auth_error()``, returns a ``{logged_in, available,+ hint, ...}`` payload for the Credentials Modal.++Both share a single ``_resolve_argv`` helper that does the npx-shim+fallback the providers' own ``binary_path()`` does.+"""++from __future__ import annotations++import asyncio+import shutil+from typing import Any, Dict, List, Optional, Tuple++from core.logging import get_logger++from services.cli_agent.factory import create_cli_provider+from services.cli_agent.protocol import AICliProvider++logger = get_logger(__name__)+++# ---------------------------------------------------------------------------+# Shared argv resolution (binary or npx fallback)+# ---------------------------------------------------------------------------++def _resolve_argv(+ provider: AICliProvider,+ argv: List[str],+) -> Tuple[Optional[List[str]], Optional[str]]:+ """Resolve ``argv[0]`` to an absolute path or an npx shim.++ Returns ``(resolved_argv, error)``. Either field is None.+ """+ if not argv:+ return None, "empty argv"+ resolved = shutil.which(argv[0])+ if resolved:+ return [resolved] + argv[1:], None+ npx = shutil.which("npx")+ if npx and provider.package_name:+ return [npx, "--yes", provider.package_name] + argv[1:], None+ return None, (+ f"Command not found: {argv[0]!r}. Install with "+ f"`npm install -g {provider.package_name}` or ensure 'npx' is on PATH."+ )+++def _build_provider(provider_name: str) -> Tuple[Optional[AICliProvider], Optional[Dict[str, Any]]]:+ """Construct a provider, mapping factory errors to result dicts."""+ try:+ return create_cli_provider(provider_name), None+ except NotImplementedError as exc:+ return None, {+ "success": False, "logged_in": False, "available": False,+ "exit_code": None, "stderr_tail": "",+ "hint": "Provider deferred to v2",+ "error": str(exc),+ }+ except ValueError as exc:+ return None, {+ "success": False, "logged_in": False, "available": False,+ "exit_code": None, "stderr_tail": "",+ "hint": "Unknown provider",+ "error": str(exc),+ }+++# ---------------------------------------------------------------------------+# Login+# ---------------------------------------------------------------------------++async def run_native_login(+ provider_name: str,+ *,+ timeout_seconds: float = 300.0,+ inherit_stdio: bool = True,+) -> Dict[str, Any]:+ """Spawn the CLI's own ``<provider> login`` command.++ With ``inherit_stdio=True`` (default) the user completes the OAuth /+ device-code flow directly in their terminal — we capture nothing.+ """+ provider, err = _build_provider(provider_name)+ if err is not None:+ return err+ assert provider is not None++ argv, resolve_err = _resolve_argv(provider, list(provider.login_argv()))+ if argv is None:+ return {+ "success": False, "exit_code": None, "stderr_tail": "",+ "error": resolve_err,+ }++ logger.info("[cli_login:%s] spawning %s", provider_name, argv)++ if inherit_stdio:+ proc = await asyncio.create_subprocess_exec(*argv)+ try:+ exit_code = await asyncio.wait_for(proc.wait(), timeout=timeout_seconds)+ except asyncio.TimeoutError:+ proc.kill()+ return {+ "success": False, "exit_code": None, "stderr_tail": "",+ "error": f"login timed out after {timeout_seconds:.0f}s",+ }+ return {+ "success": exit_code == 0,+ "exit_code": exit_code,+ "stderr_tail": "",+ "error": None if exit_code == 0 else f"login exited with code {exit_code}",+ }++ proc = await asyncio.create_subprocess_exec(+ *argv,+ stdout=asyncio.subprocess.PIPE,+ stderr=asyncio.subprocess.PIPE,+ )+ try:+ _, stderr = await asyncio.wait_for(+ proc.communicate(), timeout=timeout_seconds,+ )+ except asyncio.TimeoutError:+ proc.kill()+ return {+ "success": False, "exit_code": None, "stderr_tail": "",+ "error": f"login timed out after {timeout_seconds:.0f}s",+ }+ stderr_text = stderr.decode("utf-8", errors="replace")+ return {+ "success": proc.returncode == 0,+ "exit_code": proc.returncode,+ "stderr_tail": stderr_text[-2000:],+ "error": (+ None if proc.returncode == 0+ else f"login exited with code {proc.returncode}"+ ),+ }+++# ---------------------------------------------------------------------------+# Auth status+# ---------------------------------------------------------------------------++async def check_auth(+ provider_name: str,+ *,+ timeout_seconds: float = 15.0,+) -> Dict[str, Any]:+ """Probe whether ``provider_name`` is logged in.++ ``available`` is False when the binary isn't installed.+ """+ provider, err = _build_provider(provider_name)+ if err is not None:+ return err+ assert provider is not None++ status_argv = provider.auth_status_argv()+ if not status_argv:+ # No cheap probe configured — defer to the first session's stderr.+ return {+ "logged_in": True, # optimistic; real run will surface errors+ "available": True,+ "hint": "",+ "exit_code": None,+ "error": None,+ }++ argv, resolve_err = _resolve_argv(provider, list(status_argv))+ if argv is None:+ return {+ "logged_in": False, "available": False,+ "hint": (+ f"Install the CLI: `npm install -g {provider.package_name}`"+ if provider.package_name+ else f"Install the {provider_name} CLI"+ ),+ "exit_code": None,+ "error": resolve_err,+ }++ logger.debug("[cli_auth_status:%s] probing %s", provider_name, argv)++ try:+ proc = await asyncio.create_subprocess_exec(+ *argv,+ stdout=asyncio.subprocess.PIPE,+ stderr=asyncio.subprocess.PIPE,+ )+ stdout, stderr = await asyncio.wait_for(+ proc.communicate(), timeout=timeout_seconds,+ )+ except asyncio.TimeoutError:+ return {+ "logged_in": False, "available": True,+ "hint": "auth status probe timed out",+ "exit_code": None,+ "error": f"timeout after {timeout_seconds:.0f}s",+ }+ except FileNotFoundError as exc:+ return {+ "logged_in": False, "available": False,+ "hint": f"Install the {provider_name} CLI",+ "exit_code": None,+ "error": str(exc),+ }++ stderr_text = stderr.decode("utf-8", errors="replace")+ stdout_text = stdout.decode("utf-8", errors="replace")+ combined = f"{stderr_text}\n{stdout_text}"+ exit_code = proc.returncode if proc.returncode is not None else -1++ if provider.detect_auth_error(combined, exit_code):+ login_hint = " ".join(provider.login_argv())+ return {+ "logged_in": False, "available": True,+ "hint": f"Run `{login_hint}` (or click Login in the Credentials Modal)",+ "exit_code": exit_code,+ "error": stderr_text[-500:].strip() or "not authenticated",+ }++ if exit_code != 0:+ # Non-zero exit but not an auth error.+ return {+ "logged_in": False, "available": True,+ "hint": f"Status probe failed (exit {exit_code})",+ "exit_code": exit_code,+ "error": stderr_text[-500:].strip() or "non-zero exit",+ }++ return {+ "logged_in": True, "available": True,+ "hint": "",+ "exit_code": 0,+ "error": None,+ }diff --git a/server/services/cli_agent/config.py b/server/services/cli_agent/config.pynew file mode 100644index 00000000..4e212005--- /dev/null+++ b/server/services/cli_agent/config.py@@ -0,0 +1,112 @@+"""Provider configuration loader for the AI CLI framework.++Loads `server/config/ai_cli_providers.json` and exposes+`get_provider_config(name)` returning a typed `CliProviderConfig`. The+JSON file is the single source of truth for binary names, npm packages,+default flags, IDE lockfile paths, and per-provider feature flags.+"""++from __future__ import annotations++import json+import os+import tempfile+from dataclasses import dataclass, field+from pathlib import Path+from typing import Any, Dict, List, Optional, Tuple++from core.logging import get_logger++logger = get_logger(__name__)+++# ---------------------------------------------------------------------------+# Provider config dataclass+# ---------------------------------------------------------------------------++@dataclass+class CliProviderConfig:+ """Per-provider config from `ai_cli_providers.json`."""+ name: str+ package_name: str+ binary_name: str+ login_argv: Tuple[str, ...]+ auth_status_argv: Optional[Tuple[str, ...]]+ ide_lock_env_var: Optional[str]+ ide_lockfile_dir: Optional[Path]+ defaults: Dict[str, Any] = field(default_factory=dict)+ supports: frozenset = field(default_factory=frozenset)+++# ---------------------------------------------------------------------------+# Load config once at import time+# ---------------------------------------------------------------------------++def _config_path() -> Path:+ return Path(__file__).parent.parent.parent / "config" / "ai_cli_providers.json"+++def _load_raw() -> Dict[str, Any]:+ path = _config_path()+ try:+ with open(path, encoding="utf-8") as f:+ return json.load(f)+ except FileNotFoundError:+ logger.warning(f"ai_cli_providers.json not found at {path}")+ return {}+ except Exception as e:+ logger.warning(f"Failed to load ai_cli_providers.json: {e}")+ return {}+++def _expand_lockfile_dir(raw: Optional[str]) -> Optional[Path]:+ """Expand ~ and the literal `<tmpdir>` placeholder in lockfile paths."""+ if not raw:+ return None+ if raw.startswith("<tmpdir>"):+ return Path(tempfile.gettempdir()) / raw[len("<tmpdir>") :].lstrip("/").lstrip("\\")+ return Path(os.path.expanduser(raw))+++def _build_configs() -> Dict[str, CliProviderConfig]:+ raw = _load_raw()+ configs: Dict[str, CliProviderConfig] = {}+ for name, prov in raw.items():+ # Pull known fields out; everything else lives in `defaults` so+ # provider impls can read e.g. `default_model`, `default_max_turns`.+ defaults = {+ k: v for k, v in prov.items()+ if k.startswith("default_")+ }+ configs[name] = CliProviderConfig(+ name=name,+ package_name=prov.get("package_name", ""),+ binary_name=prov.get("binary_name", name),+ login_argv=tuple(prov.get("login_argv", [])),+ auth_status_argv=(+ tuple(prov["auth_status_argv"])+ if prov.get("auth_status_argv") else None+ ),+ ide_lock_env_var=prov.get("ide_lock_env_var"),+ ide_lockfile_dir=_expand_lockfile_dir(prov.get("ide_lockfile_dir")),+ defaults=defaults,+ supports=frozenset(prov.get("supports", [])),+ )+ return configs+++PROVIDER_CONFIGS: Dict[str, CliProviderConfig] = _build_configs()+++def get_provider_config(name: str) -> Optional[CliProviderConfig]:+ return PROVIDER_CONFIGS.get(name)+++def list_provider_names() -> List[str]:+ return list(PROVIDER_CONFIGS.keys())+++def reload_configs() -> None:+ """Reload `ai_cli_providers.json` (useful for tests)."""+ global PROVIDER_CONFIGS+ PROVIDER_CONFIGS = _build_configs()diff --git a/server/services/cli_agent/factory.py b/server/services/cli_agent/factory.pynew file mode 100644index 00000000..ab5a3b1a--- /dev/null+++ b/server/services/cli_agent/factory.py@@ -0,0 +1,48 @@+"""Factory for `AICliProvider` instances.++Mirrors `services/llm/factory.py`: lazy imports keep optional providers+from being loaded until first use. v1 ships Claude + Codex; calling+`create_cli_provider("gemini")` raises `NotImplementedError` (v2).+"""++from __future__ import annotations++from core.logging import get_logger++from services.cli_agent.protocol import AICliProvider++logger = get_logger(__name__)+++SUPPORTED_PROVIDERS = frozenset({"claude", "codex"}) # v2 adds "gemini"+ALL_PROVIDERS = frozenset({"claude", "codex", "gemini"}) # union of v1+v2 names+++def create_cli_provider(name: str) -> AICliProvider:+ """Build a CLI provider by name.++ Raises:+ NotImplementedError: for `gemini` until the v2 implementation lands.+ ValueError: for unknown names.+ """+ if name == "claude":+ from services.cli_agent.providers.anthropic_claude import AnthropicClaudeProvider+ return AnthropicClaudeProvider()++ if name == "codex":+ from services.cli_agent.providers.openai_codex import OpenAICodexProvider+ return OpenAICodexProvider()++ if name == "gemini":+ # v2 stub — surface a clean error so factory consumers can detect+ # the deferred state and offer the user an actionable hint.+ raise NotImplementedError(+ "gemini provider deferred to v2. Use 'claude' or 'codex' in v1."+ )++ raise ValueError(f"Unknown CLI provider: {name!r}. Known: {sorted(ALL_PROVIDERS)}")+++def is_supported(name: str) -> bool:+ """True if the provider is fully implemented in this version."""+ return name in SUPPORTED_PROVIDERSdiff --git a/server/services/cli_agent/lockfile.py b/server/services/cli_agent/lockfile.pynew file mode 100644index 00000000..a097aa65--- /dev/null+++ b/server/services/cli_agent/lockfile.py@@ -0,0 +1,134 @@+"""VSCode-style IDE lockfile writer + remover + stale-PID sweep.++Each spawned CLI session writes a discovery lockfile that the CLI reads+to auto-connect to MachinaOs's MCP server. We mirror the format the+official VSCode Claude Code extension writes+(``~/.claude/ide/<pid>.lock`` containing+``{port, url, authToken, workspaceFolders, ideName, transport, pid}``)+and the path convention Gemini's VSCode companion uses+(``<tmpdir>/gemini/ide/gemini-ide-server-<pid>-<port>.json``).++The startup sweep (mirroring VSCode's own behaviour) removes lockfiles+whose PIDs are no longer alive — see ``main.py`` lifespan.+"""++from __future__ import annotations++import json+import logging+import os+import sys+from pathlib import Path+from typing import List, Optional++logger = logging.getLogger(__name__)+++def _lockfile_path(+ *, ide_lockfile_dir: Path, pid: int, port: int, ide_name: str,+) -> Path:+ if ide_name == "gemini":+ return ide_lockfile_dir / f"gemini-ide-server-{pid}-{port}.json"+ return ide_lockfile_dir / f"{pid}.lock"+++# Keep `lockfile_path` as a public alias for tests that exercise the+# path-shape contract.+lockfile_path = _lockfile_path+++def write_ide_lockfile(+ *,+ ide_lockfile_dir: Path,+ pid: int,+ port: int,+ token: str,+ workspace_dir: Path,+ ide_name: str,+ url: Optional[str] = None,+ transport: str = "http",+) -> Path:+ """Write a VSCode-style IDE lockfile (mode 0600 on POSIX)."""+ ide_lockfile_dir.mkdir(parents=True, exist_ok=True)+ path = _lockfile_path(+ ide_lockfile_dir=ide_lockfile_dir, pid=pid, port=port, ide_name=ide_name,+ )++ payload = {+ "port": port,+ "url": url or f"http://127.0.0.1:{port}/mcp/ide",+ "authToken": token,+ "workspaceFolders": [str(workspace_dir)],+ "ideName": ide_name,+ "transport": transport,+ "pid": pid,+ }++ tmp = path.with_suffix(path.suffix + ".tmp")+ tmp.write_text(json.dumps(payload), encoding="utf-8")+ if sys.platform != "win32":+ try:+ os.chmod(tmp, 0o600)+ except OSError:+ pass+ tmp.replace(path)+ return path+++def remove_ide_lockfile(path: Optional[Path]) -> None:+ """Best-effort lockfile removal. Never raises."""+ if not path:+ return+ try:+ if path.exists():+ path.unlink()+ except OSError as exc:+ logger.debug("Failed to remove lockfile %s: %s", path, exc)+++def sweep_stale_lockfiles(ide_lockfile_dir: Optional[Path]) -> int:+ """Remove lockfiles whose PIDs are no longer alive.++ Mirrors VSCode's startup behaviour. Safe on missing dirs. Never raises.+ """+ if not ide_lockfile_dir or not ide_lockfile_dir.exists():+ return 0++ try:+ import psutil+ except ImportError:+ return 0++ removed = 0+ for entry in ide_lockfile_dir.iterdir():+ if not entry.is_file():+ continue+ try:+ data = json.loads(entry.read_text(encoding="utf-8"))+ except (OSError, json.JSONDecodeError):+ continue+ pid = data.get("pid")+ if not isinstance(pid, int):+ continue+ try:+ alive = psutil.pid_exists(pid)+ except Exception:+ alive = True+ if alive:+ continue+ try:+ entry.unlink()+ removed += 1+ except OSError as exc:+ logger.debug("Failed to remove stale lockfile %s: %s", entry, exc)++ if removed:+ logger.info("Swept %d stale CLI lockfile(s) from %s", removed, ide_lockfile_dir)+ return removed+++def list_active_lockfiles(ide_lockfile_dir: Optional[Path]) -> List[Path]:+ """Diagnostic helper: list lockfiles in the discovery dir."""+ if not ide_lockfile_dir or not ide_lockfile_dir.exists():+ return []+ return [p for p in ide_lockfile_dir.iterdir() if p.is_file()]diff --git a/server/services/cli_agent/mcp_server.py b/server/services/cli_agent/mcp_server.pynew file mode 100644index 00000000..61cd3f52--- /dev/null+++ b/server/services/cli_agent/mcp_server.py@@ -0,0 +1,426 @@+"""MachinaOs MCP server (VSCode pattern, no custom IPC).++Hosts a `FastMCP` ASGI sub-app at ``/mcp/ide`` that spawned CLI sessions+auto-discover via the lockfile written by ``lockfile.py``. Each session+gets a per-batch bearer token; the middleware validates it and binds the+matching ``BatchContext`` into a contextvar so tool implementations can+scope to the calling session's workspace_dir / connected_skill_names /+allowed_credentials without explicit plumbing.++Tools (5 in v1, mirroring Claude Code's progressive-disclosure pattern):+ - ``getWorkspaceFiles`` — glob/read inside the session's worktree+ - ``listSkills`` — metadata for skills connected to the parent agent+ - ``getSkill`` — full skill markdown + scripts + references+ - ``getCredential`` — gated by per-batch allowlist+ - ``broadcastLog`` — write to MachinaOs Terminal tab++The server module exposes:+ - :func:`get_mcp_app` — Starlette/ASGI sub-app for ``app.mount(...)``+ - :func:`register_batch` / :func:`unregister_batch` — `AICliService`+ calls these around `run_batch()` to register/expire tokens+ - :class:`BatchContext` — scoping data attached to each token++Tools deferred to v2: ``getDiagnostics``, ``executeCode``.+"""++from __future__ import annotations++import contextvars+import logging+import secrets+from dataclasses import dataclass, field+from pathlib import Path+from typing import Any, Awaitable, Callable, Dict, List, Optional, Set++from starlette.middleware.base import BaseHTTPMiddleware+from starlette.requests import Request+from starlette.responses import JSONResponse, Response+from starlette.routing import Mount++from core.logging import get_logger++logger = get_logger(__name__)+++# ---------------------------------------------------------------------------+# BatchContext + token registry+# ---------------------------------------------------------------------------++@dataclass+class BatchContext:+ """Scoping data attached to one batch's bearer token.++ Populated at ``AICliService.run_batch()`` entry; deregistered in the+ ``finally`` block. Tools dereference the calling batch via the+ bearer token in `Authorization` header.+ """+ workflow_id: str+ node_id: str+ workspace_dir: Path+ connected_skill_names: Set[str] = field(default_factory=set)+ allowed_credentials: Set[str] = field(default_factory=set)+ # Optional broadcaster for `broadcastLog`. Lazily resolved from the+ # global container if None.+ broadcaster: Optional[Any] = None+++# Token -> BatchContext registry. Lives in-memory only; tokens never+# touch disk or the credentials.db.+_active_tokens: Dict[str, BatchContext] = {}+++def issue_token() -> str:+ """Mint a new bearer token (32 bytes hex)."""+ return secrets.token_hex(32)+++def register_batch(token: str, ctx: BatchContext) -> None:+ """Register a batch's auth token. Idempotent on identical context."""+ if token in _active_tokens:+ # Same token registered twice — refuse to overwrite a different ctx.+ existing = _active_tokens[token]+ if existing is not ctx:+ raise ValueError("Token collision in MCP server batch registry")+ return+ _active_tokens[token] = ctx+ logger.debug(+ "[MCP] registered batch token=%s... node=%s wf=%s",+ token[:8], ctx.node_id, ctx.workflow_id,+ )+++def unregister_batch(token: str) -> None:+ """Drop a batch's token. Safe to call twice."""+ ctx = _active_tokens.pop(token, None)+ if ctx is not None:+ logger.debug("[MCP] unregistered batch token=%s...", token[:8])+++def lookup_batch(token: str) -> Optional[BatchContext]:+ return _active_tokens.get(token)+++def active_batch_count() -> int:+ return len(_active_tokens)+++# ---------------------------------------------------------------------------+# ContextVar — thread/task-local handle to the current batch+# ---------------------------------------------------------------------------++_current_batch: contextvars.ContextVar[Optional[BatchContext]] = (+ contextvars.ContextVar("machina_current_batch", default=None)+)+++def _require_batch() -> BatchContext:+ ctx = _current_batch.get()+ if ctx is None:+ raise RuntimeError(+ "MCP tool called without an active batch context. "+ "This indicates the auth middleware was bypassed."+ )+ return ctx+++# ---------------------------------------------------------------------------+# Auth middleware+# ---------------------------------------------------------------------------++class _BearerAuthMiddleware(BaseHTTPMiddleware):+ """Validate `Authorization: Bearer <token>` against the registry.++ On success: bind the matching `BatchContext` into the contextvar.+ On failure: 401.++ The MCP spec uses MCP-Protocol-Version + Authorization headers; we+ care only about the Bearer here. Health checks under `/healthz`+ bypass auth (they're for dev sanity).+ """++ async def dispatch( # type: ignore[override]+ self,+ request: Request,+ call_next: Callable[[Request], Awaitable[Response]],+ ) -> Response:+ if request.url.path.endswith("/healthz"):+ return await call_next(request)++ auth = request.headers.get("authorization") or request.headers.get(+ "Authorization"+ )+ token: Optional[str] = None+ if auth and auth.lower().startswith("bearer "):+ token = auth[7:].strip() or None++ if not token:+ return JSONResponse(+ {"error": "missing or malformed Authorization header"},+ status_code=401,+ )++ ctx = lookup_batch(token)+ if ctx is None:+ return JSONResponse(+ {"error": "invalid or expired token"},+ status_code=401,+ )++ reset_token = _current_batch.set(ctx)+ try:+ return await call_next(request)+ finally:+ _current_batch.reset(reset_token)+++# ---------------------------------------------------------------------------+# Tool registration helper — defers FastMCP import so module import is cheap+# ---------------------------------------------------------------------------++def _build_tools(mcp: Any) -> None: # FastMCP type+ """Register the 5 v1 tools on a `FastMCP` instance."""++ @mcp.tool(+ name="getWorkspaceFiles",+ description=(+ "List or read files inside the calling session's per-task git "+ "worktree. Use `read=False` for metadata-only listings; "+ "`read=True` to fetch file contents (capped at 1MB per file)."+ ),+ )+ def get_workspace_files(+ path: str = ".",+ pattern: str = "*",+ read: bool = False,+ max_bytes: int = 1_000_000,+ ) -> Dict[str, Any]:+ ctx = _require_batch()+ try:+ base = ctx.workspace_dir.resolve()+ target = (base / path).resolve()+ # Path-traversal guard+ try:+ target.relative_to(base)+ except ValueError:+ return {+ "error": "path escapes workspace_dir",+ "path": str(path),+ }++ if not target.exists():+ return {"files": [], "path": str(path)}++ entries: List[Dict[str, Any]] = []+ if target.is_file():+ files_iter = [target]+ else:+ files_iter = sorted(target.rglob(pattern))++ for p in files_iter:+ if not p.is_file():+ continue+ try:+ rel = str(p.relative_to(base))+ info: Dict[str, Any] = {+ "path": rel,+ "size": p.stat().st_size,+ "mtime": p.stat().st_mtime,+ }+ if read and p.stat().st_size <= max_bytes:+ try:+ info["content"] = p.read_text(+ encoding="utf-8", errors="replace"+ )+ except OSError as exc:+ info["read_error"] = str(exc)+ entries.append(info)+ except OSError:+ continue++ if len(entries) >= 1000:+ break++ return {"files": entries, "path": str(path)}+ except Exception as exc: # pragma: no cover — defensive+ logger.exception("[MCP] getWorkspaceFiles failed")+ return {"error": str(exc), "path": str(path)}++ @mcp.tool(+ name="listSkills",+ description=(+ "List skills connected to the parent agent node. Returns "+ "metadata only (~100 tokens per skill). Call `getSkill(name)` "+ "to fetch the full instructions."+ ),+ )+ def list_skills() -> Dict[str, Any]:+ ctx = _require_batch()+ try:+ from services.skill_loader import get_skill_loader+ loader = get_skill_loader()+ registry = loader.scan_skills()+ results = []+ for name in sorted(ctx.connected_skill_names):+ meta = registry.get(name)+ if meta is None:+ continue+ results.append({+ "name": meta.name,+ "description": meta.description,+ "allowed_tools": list(meta.allowed_tools),+ "category": (+ meta.metadata.get("category")+ if isinstance(meta.metadata, dict) else None+ ),+ })+ return {"skills": results}+ except Exception as exc: # pragma: no cover+ logger.exception("[MCP] listSkills failed")+ return {"error": str(exc), "skills": []}++ @mcp.tool(+ name="getSkill",+ description=(+ "Fetch full content for one skill: instructions (markdown), "+ "scripts (executable code samples), and references (extra "+ "docs). The skill must be connected to the parent agent node."+ ),+ )+ def get_skill(name: str) -> Dict[str, Any]:+ ctx = _require_batch()+ if name not in ctx.connected_skill_names:+ return {+ "error": f"skill {name!r} is not connected to this agent node",+ "name": name,+ }+ try:+ from services.skill_loader import get_skill_loader+ loader = get_skill_loader()+ skill = loader.load_skill(name)+ if skill is None:+ return {"error": f"skill {name!r} not found", "name": name}+ return {+ "name": skill.metadata.name,+ "description": skill.metadata.description,+ "instructions": skill.instructions,+ "allowed_tools": list(skill.metadata.allowed_tools),+ "metadata": dict(skill.metadata.metadata) if skill.metadata.metadata else {},+ "scripts": dict(skill.scripts),+ "references": dict(skill.references),+ # `assets` (binary) excluded by default — too big for MCP responses.+ }+ except Exception as exc: # pragma: no cover+ logger.exception("[MCP] getSkill failed for %r", name)+ return {"error": str(exc), "name": name}++ @mcp.tool(+ name="getCredential",+ description=(+ "Fetch a credential by provider name. Only credentials in the "+ "batch's allowlist are returned; everything else returns 403. "+ "Use sparingly — prefer the CLI's own auth where possible."+ ),+ )+ async def get_credential(name: str) -> Dict[str, Any]:+ ctx = _require_batch()+ if name not in ctx.allowed_credentials:+ return {+ "error": f"credential {name!r} not in allowlist for this batch",+ "name": name,+ "status": 403,+ }+ try:+ from core.container import container+ auth = container.auth_service()+ value = await auth.get_api_key(name)+ if not value:+ return {+ "error": f"credential {name!r} not configured",+ "name": name,+ "status": 404,+ }+ return {"name": name, "value": value}+ except Exception as exc: # pragma: no cover+ logger.exception("[MCP] getCredential failed for %r", name)+ return {"error": str(exc), "name": name, "status": 500}++ @mcp.tool(+ name="broadcastLog",+ description=(+ "Write a log line to the MachinaOs Terminal tab. Use to "+ "surface intermediate progress that would otherwise be lost "+ "between CLI sessions."+ ),+ )+ async def broadcast_log(+ message: str,+ level: str = "info",+ source: Optional[str] = None,+ ) -> Dict[str, Any]:+ ctx = _require_batch()+ if level not in ("debug", "info", "warning", "error"):+ level = "info"+ try:+ broadcaster = ctx.broadcaster+ if broadcaster is None:+ from services.status_broadcaster import get_status_broadcaster+ broadcaster = get_status_broadcaster()+ payload = {+ "source": source or f"mcp:{ctx.node_id}",+ "level": level,+ "message": message[:5000],+ }+ await broadcaster.broadcast_terminal_log(payload)+ return {"success": True}+ except Exception as exc: # pragma: no cover+ logger.exception("[MCP] broadcastLog failed")+ return {"error": str(exc)}+++# ---------------------------------------------------------------------------+# ASGI sub-app factory (mounted in main.py lifespan)+# ---------------------------------------------------------------------------++_app_singleton: Optional[Any] = None # Starlette app+++def get_mcp_app() -> Any:+ """Return the Starlette/ASGI app to mount under `/mcp/ide`.++ Idempotent — multiple calls return the same instance, so the FastAPI+ lifespan can wire it without worrying about duplicate registration.+ """+ global _app_singleton+ if _app_singleton is not None:+ return _app_singleton++ from mcp.server.fastmcp import FastMCP++ mcp = FastMCP(+ name="machinaos-cli-agent",+ instructions=(+ "MachinaOs IDE MCP server. Exposes workspace files, connected "+ "skills, scoped credentials, and a Terminal-tab log channel "+ "to the calling CLI session."+ ),+ # FastMCP's HTTP path defaults are fine; we mount the whole app+ # under /mcp/ide externally.+ )+ _build_tools(mcp)++ asgi_app = mcp.streamable_http_app()+ asgi_app.add_middleware(_BearerAuthMiddleware)++ _app_singleton = asgi_app+ return _app_singleton+++# ---------------------------------------------------------------------------+# Test/diagnostic helpers (used by tests + verification step #11/#12)+# ---------------------------------------------------------------------------++def _reset_for_tests() -> None: # pragma: no cover+ """Wipe the token registry. ONLY use in tests."""+ global _app_singleton+ _active_tokens.clear()+ _app_singleton = Nonediff --git a/server/services/cli_agent/protocol.py b/server/services/cli_agent/protocol.pynew file mode 100644index 00000000..ec12c608--- /dev/null+++ b/server/services/cli_agent/protocol.py@@ -0,0 +1,173 @@+"""Protocol for AI CLI providers (Claude Code, Codex, Gemini).++Mirrors `services/llm/protocol.py` shape: a structurally-typed Protocol+with a small, explicit surface. Each concrete provider lives under+`providers/<vendor>.py`.++The framework spawns N parallel CLI sessions (one `AICliSession` each,+backed by `BaseProcessSupervisor`) over a list of tasks. Per-CLI+differences (argv, JSON event schema, auth handling, feature support)+are isolated to the provider; the session/pool/service layer is generic.++Auth: native CLI handles its own tokens. We only trigger the login flow+(`login_argv()`) and detect logged-in state (`auth_status_argv()` ++`detect_auth_error()`). No credential wrapping, no `~/.claude-machina/`.+"""++from __future__ import annotations++from dataclasses import dataclass, field+from pathlib import Path+from typing import Any, Dict, List, Optional, Protocol, runtime_checkable+++# ---------------------------------------------------------------------------+# Shared data types+# ---------------------------------------------------------------------------++@dataclass+class CanonicalUsage:+ """Vendor-normalised token counts.++ Pattern from Hermes `agent/usage_pricing.py:CanonicalUsage` — every+ provider's usage shape (Anthropic vs Codex vs OpenAI handle cache+ differently) maps into this so the existing `services/pricing.py`+ can compute USD without per-vendor branches.+ """+ input_tokens: int = 0+ output_tokens: int = 0+ cache_read: int = 0+ cache_write: int = 0+ reasoning_tokens: int = 0+ request_count: int = 0+++@dataclass+class SessionResult:+ """Per-task result returned by `AICliSession`.++ Shared keys are the schema; vendor extras live in `provider_data`.+ """+ task_id: str+ session_id: Optional[str] = None+ provider: str = ""+ prompt: str = ""+ branch: Optional[str] = None+ worktree_path: Optional[str] = None+ response: str = ""+ cost_usd: Optional[float] = None+ duration_ms: Optional[int] = None+ num_turns: Optional[int] = None+ tool_calls: int = 0+ canonical_usage: CanonicalUsage = field(default_factory=CanonicalUsage)+ provider_data: Dict[str, Any] = field(default_factory=dict)+ success: bool = False+ error: Optional[str] = None+++@dataclass+class BatchResult:+ """Aggregated result returned by `AICliService.run_batch()`."""+ tasks: List[SessionResult] = field(default_factory=list)+ n_tasks: int = 0+ n_succeeded: int = 0+ n_failed: int = 0+ total_cost_usd: Optional[float] = None+ wall_clock_ms: int = 0+ budget_remaining_usd: Optional[float] = None+ provider: str = ""+ timestamp: str = ""+++# ---------------------------------------------------------------------------+# Provider protocol (structural typing)+# ---------------------------------------------------------------------------++@runtime_checkable+class AICliProvider(Protocol):+ """Structural Protocol for an AI CLI provider.++ Concrete classes:+ - `providers.anthropic_claude.AnthropicClaudeProvider`+ - `providers.openai_codex.OpenAICodexProvider`+ - `providers.google_gemini.GoogleGeminiProvider` (v2 stub)+ """++ name: str # "claude" | "codex" | "gemini"+ package_name: str # npm package+ binary_name: str # "claude" | "codex" | "gemini"+ ide_lock_env_var: Optional[str] # CLAUDE_IDE_LOCK | GEMINI_IDE_LOCK | None+ ide_lockfile_dir: Optional[Path] # ~/.claude/ide | <tmpdir>/gemini/ide++ # ---- spawn surface ---------------------------------------------------++ def binary_path(self) -> Path: ...+ # Resolve the CLI binary. Resolution chain (Composio pattern):+ # 1) shutil.which(<binary_name>)+ # 2) `npx --yes <package_name>` shim path+ # Raises FileNotFoundError if neither is available.++ def headless_argv(self, task: Any, *, defaults: Dict[str, Any]) -> List[str]: ...+ # Build the full argv (binary + flags) for a headless run of+ # `task` (a `<Provider>TaskSpec` Pydantic model). `defaults`+ # comes from `ai_cli_providers.json` for this provider.++ # ---- native auth (no token wrapping) --------------------------------++ def login_argv(self) -> List[str]: ...+ # CLI's own login command, e.g. ["claude", "login"]. Spawned+ # interactively from the Credentials Modal. CLI stores its own+ # credentials in `~/.claude/`, `~/.codex/`, `~/.gemini/`.++ def auth_status_argv(self) -> Optional[List[str]]: ...+ # No-op invocation to verify auth, e.g. ["claude", "--print", "-p", "ok"].+ # Returns None if no cheap probe exists; in that case the framework+ # infers from the first session's stderr.++ def detect_auth_error(self, stderr: str, exit_code: int) -> bool: ...+ # Match "not logged in" patterns:+ # Claude: "Please run 'claude login'"+ # Codex: HTTP 401 / "OPENAI_API_KEY not set"+ # Gemini: exit_code == 1 (FatalAuthenticationError)++ # ---- streaming output parsing ---------------------------------------++ def parse_event(self, line: str) -> Optional[Dict[str, Any]]: ...+ # Parse a single NDJSON line from stdout. Return None for+ # un-parseable garbage.++ def is_final_event(self, event: Dict[str, Any]) -> bool: ...+ # True if this event marks end-of-task. For Claude: `type=="result"`.+ # For Gemini: `type=="result"`. For Codex: `type=="complete"` or+ # heuristic fallback.++ def event_to_session_result(+ self,+ events: List[Dict[str, Any]],+ stderr: str,+ exit_code: int,+ ) -> Dict[str, Any]: ...+ # Reconstruct a partial dict of `SessionResult` fields from the+ # event stream. Returns:+ # {+ # "session_id": ..., "response": ..., "cost_usd": ...,+ # "duration_ms": ..., "num_turns": ..., "tool_calls": ...,+ # "success": bool, "error": Optional[str],+ # "canonical_usage": CanonicalUsage,+ # "provider_data": {<vendor-specific>},+ # }+ # provider_data carries vendor-only metadata (Anthropic+ # reasoning_details, Codex call_id, Gemini extra_content) without+ # bloating the shared schema. Pattern from+ # Hermes agent/transports/types.py NormalizedResponse.++ def canonical_usage(self, events: List[Dict[str, Any]]) -> CanonicalUsage: ...+ # Normalise vendor token-counting into the shared `CanonicalUsage`+ # shape. Pattern from Hermes agent/usage_pricing.py.++ # ---- feature gating --------------------------------------------------++ def supports(self, feature: str) -> bool: ...+ # Feature flags consulted by the session/service layer.+ # Recognised: "max_budget", "max_turns", "session_id", "resume",+ # "mcp_runtime", "json_cost", "ide_lockfile", "sandbox".diff --git a/server/services/cli_agent/providers/__init__.py b/server/services/cli_agent/providers/__init__.pynew file mode 100644index 00000000..8f2d3eec--- /dev/null+++ b/server/services/cli_agent/providers/__init__.py@@ -0,0 +1,9 @@+"""Concrete `AICliProvider` implementations.++- `anthropic_claude.AnthropicClaudeProvider` — full v1 surface+- `openai_codex.OpenAICodexProvider` — sandbox-first, no session+- `google_gemini.GoogleGeminiProvider` — v2 stub raising NotImplementedError++Imports are intentionally not eager — `factory.create_cli_provider()`+lazy-imports each one.+"""diff --git a/server/services/cli_agent/providers/anthropic_claude.py b/server/services/cli_agent/providers/anthropic_claude.pynew file mode 100644index 00000000..1afc3fc4--- /dev/null+++ b/server/services/cli_agent/providers/anthropic_claude.py@@ -0,0 +1,330 @@+"""Anthropic Claude Code CLI provider.++Reference implementation for the `AICliProvider` Protocol. Full feature+set: sessions, resume, budget, turns, allowed_tools, permission_mode,+MCP lockfile, cost-in-JSON.++Subprocess: ``claude --print -p <prompt> --output-format stream-json+--include-partial-messages [--session-id|--resume <UUID>] --model ...+--max-turns ... --max-budget-usd ... --allowedTools ...+--permission-mode ... --append-system-prompt ...``++Auth: native — CLI reads `~/.claude/`. We do NOT inject CLAUDE_CONFIG_DIR.++Final event: ``type == "result"`` carries ``total_cost_usd``,+``duration_ms``, ``num_turns``, ``session_id``, and the assistant's+``result`` string.+"""++from __future__ import annotations++import json+import shutil+from pathlib import Path+from typing import Any, Dict, List, Optional++from core.logging import get_logger++from services.cli_agent.config import get_provider_config+from services.cli_agent.protocol import CanonicalUsage+from services.cli_agent.types import ClaudeTaskSpec++logger = get_logger(__name__)++NAME = "claude"+++class AnthropicClaudeProvider:+ """`AICliProvider` for Anthropic's Claude Code CLI."""++ def __init__(self) -> None:+ cfg = get_provider_config(NAME)+ if cfg is None:+ raise RuntimeError(+ f"Provider config missing for {NAME!r}. Check ai_cli_providers.json."+ )+ self.name = NAME+ self.package_name = cfg.package_name+ self.binary_name = cfg.binary_name+ self.ide_lock_env_var = cfg.ide_lock_env_var+ self.ide_lockfile_dir = cfg.ide_lockfile_dir+ self._defaults = cfg.defaults+ self._supports = cfg.supports+ self._login_argv = cfg.login_argv+ self._auth_status_argv = cfg.auth_status_argv++ # ---- spawn surface ---------------------------------------------------++ def binary_path(self) -> Path:+ """Resolve the `claude` binary.++ Resolution chain:+ 1) `shutil.which("claude")` — system install / npm-global+ 2) Fall back to `npx --yes @anthropic-ai/claude-code` shim path++ Raises FileNotFoundError if neither is available.+ """+ which_result = shutil.which(self.binary_name)+ if which_result:+ return Path(which_result)++ npx = shutil.which("npx")+ if npx:+ # Return the npx path — `headless_argv` rebuilds argv with+ # `npx --yes <package>` when the binary isn't on PATH.+ return Path(npx)++ raise FileNotFoundError(+ f"Neither {self.binary_name!r} nor 'npx' found in PATH. "+ f"Install with: npm install -g {self.package_name}"+ )++ def headless_argv(+ self,+ task: Any, # ClaudeTaskSpec+ *,+ defaults: Dict[str, Any],+ ) -> List[str]:+ """Build the full argv (binary + flags) for one task."""+ if not isinstance(task, ClaudeTaskSpec):+ raise TypeError(+ "AnthropicClaudeProvider.headless_argv requires ClaudeTaskSpec, "+ f"got {type(task).__name__}"+ )++ # Resolve binary — if `claude` isn't on PATH, fall back to npx shim.+ which_claude = shutil.which(self.binary_name)+ if which_claude:+ argv: List[str] = [which_claude]+ else:+ npx = shutil.which("npx")+ if not npx:+ raise FileNotFoundError(+ f"Neither {self.binary_name!r} nor 'npx' found in PATH"+ )+ argv = [npx, "--yes", self.package_name]++ argv += [+ "-p", task.prompt,+ "--output-format", "stream-json",+ "--include-partial-messages",+ "--verbose", # required by Claude CLI when using stream-json with --print+ ]++ # Model+ model = (+ task.model+ or defaults.get("default_model")+ or self._defaults.get("default_model", "claude-sonnet-4-6")+ )+ argv += ["--model", model]++ # Session / resume — `resume_session_id` wins if both are set,+ # since "resume" implies the user has a prior session.+ if task.resume_session_id:+ argv += ["--resume", task.resume_session_id]+ elif task.session_id:+ argv += ["--session-id", task.session_id]++ # Max turns+ max_turns = (+ task.max_turns+ if task.max_turns is not None+ else int(defaults.get(+ "default_max_turns",+ self._defaults.get("default_max_turns", 10),+ ))+ )+ argv += ["--max-turns", str(max_turns)]++ # Budget (USD)+ max_budget = (+ task.max_budget_usd+ if task.max_budget_usd is not None+ else float(defaults.get(+ "default_max_budget_usd",+ self._defaults.get("default_max_budget_usd", 5.0),+ ))+ )+ if max_budget > 0:+ argv += ["--max-budget-usd", str(max_budget)]++ # Allowed tools+ allowed = task.allowed_tools or defaults.get(+ "default_allowed_tools",+ self._defaults.get(+ "default_allowed_tools", "Read,Edit,Bash,Glob,Grep,Write"+ ),+ )+ if allowed:+ argv += ["--allowedTools", allowed]++ # Permission mode (default acceptEdits)+ perm = task.permission_mode or defaults.get(+ "default_permission_mode",+ self._defaults.get("default_permission_mode", "acceptEdits"),+ )+ if perm:+ argv += ["--permission-mode", perm]++ # System prompt — appended to Claude Code's built-in system prompt+ if task.system_prompt:+ argv += ["--append-system-prompt", task.system_prompt]++ return argv++ # ---- native auth -----------------------------------------------------++ def login_argv(self) -> List[str]:+ return list(self._login_argv) or ["claude", "login"]++ def auth_status_argv(self) -> Optional[List[str]]:+ return list(self._auth_status_argv) if self._auth_status_argv else None++ def detect_auth_error(self, stderr: str, exit_code: int) -> bool:+ """True if stderr/exit_code indicate the user isn't logged in."""+ if not stderr and exit_code == 0:+ return False+ markers = (+ "Please run 'claude login'",+ "Please run `claude login`",+ "Not authenticated",+ "Authentication required",+ "401 Unauthorized",+ "Invalid API key",+ )+ return any(m in stderr for m in markers)++ # ---- streaming output parsing ---------------------------------------++ def parse_event(self, line: str) -> Optional[Dict[str, Any]]:+ line = line.strip()+ if not line:+ return None+ try:+ return json.loads(line)+ except json.JSONDecodeError:+ return None++ def is_final_event(self, event: Dict[str, Any]) -> bool:+ return event.get("type") == "result"++ def event_to_session_result(+ self,+ events: List[Dict[str, Any]],+ stderr: str,+ exit_code: int,+ ) -> Dict[str, Any]:+ """Reconstruct shared result fields from the event stream."""+ final = next(+ (e for e in reversed(events) if e.get("type") == "result"),+ None,+ )++ # Session ID can come from `system.init` or `result`+ session_id: Optional[str] = None+ for evt in events:+ sid = evt.get("session_id")+ if sid:+ session_id = sid+ break++ tool_calls = sum(+ 1 for evt in events+ if evt.get("type") == "tool_use"+ or (evt.get("type") == "assistant" and self._has_tool_use(evt))+ )++ provider_data: Dict[str, Any] = {}+ for evt in events:+ if evt.get("type") == "assistant":+ msg = evt.get("message") or {}+ rd = msg.get("reasoning_details") or msg.get("thinking")+ if rd is not None:+ provider_data.setdefault("reasoning_details", rd)+ break++ success = exit_code == 0 and final is not None+ error: Optional[str] = None+ if exit_code != 0:+ error = stderr.strip()[-2000:] or f"claude exited with code {exit_code}"+ elif final is None:+ error = "no result event received"++ response = ""+ cost: Optional[float] = None+ duration_ms: Optional[int] = None+ num_turns: Optional[int] = None+ if final:+ response = str(final.get("result") or "")+ cost = final.get("total_cost_usd")+ duration_ms = final.get("duration_ms")+ num_turns = final.get("num_turns")+ if final.get("subtype") == "error":+ success = False+ error = error or response or "result event reports error"++ cu = self.canonical_usage(events)++ return {+ "session_id": session_id,+ "response": response,+ "cost_usd": cost,+ "duration_ms": duration_ms,+ "num_turns": num_turns,+ "tool_calls": tool_calls,+ "canonical_usage": cu,+ "provider_data": provider_data,+ "success": success,+ "error": error,+ }++ def canonical_usage(self, events: List[Dict[str, Any]]) -> CanonicalUsage:+ """Pull token counts from the `result` event's `usage` block.++ Anthropic shape:+ {+ "input_tokens": int,+ "output_tokens": int,+ "cache_creation_input_tokens": int,+ "cache_read_input_tokens": int,+ }+ """+ final = next(+ (e for e in reversed(events) if e.get("type") == "result"),+ None,+ )+ if not final:+ return CanonicalUsage()++ usage = final.get("usage") or {}+ request_count = (+ int(final.get("num_turns") or 0)+ or sum(1 for e in events if e.get("type") == "assistant")+ )+ return CanonicalUsage(+ input_tokens=int(usage.get("input_tokens", 0)),+ output_tokens=int(usage.get("output_tokens", 0)),+ cache_read=int(usage.get("cache_read_input_tokens", 0)),+ cache_write=int(usage.get("cache_creation_input_tokens", 0)),+ reasoning_tokens=0, # Claude doesn't expose this separately+ request_count=request_count,+ )++ # ---- feature gating --------------------------------------------------++ def supports(self, feature: str) -> bool:+ return feature in self._supports++ # ---- internals -------------------------------------------------------++ @staticmethod+ def _has_tool_use(event: Dict[str, Any]) -> bool:+ msg = event.get("message") or {}+ content = msg.get("content")+ if isinstance(content, list):+ return any(+ isinstance(blk, dict) and blk.get("type") == "tool_use"+ for blk in content+ )+ return Falsediff --git a/server/services/cli_agent/providers/google_gemini.py b/server/services/cli_agent/providers/google_gemini.pynew file mode 100644index 00000000..3ec66fa5--- /dev/null+++ b/server/services/cli_agent/providers/google_gemini.py@@ -0,0 +1,80 @@+"""Google Gemini CLI provider — v2 stub.++The factory raises ``NotImplementedError`` for ``"gemini"`` in v1, but+the type+config layer is wired so v2 activation is mechanical (replace+this stub, drop the factory branch). See `cli_agent_framework.md` →+"v2 follow-up — Gemini activation" for the full v2 PR shape.++Reference for v2 implementation:+- argv: ``gemini --prompt <p> --output-format stream-json --model ...+ [--session-id <UUID>] [--resume latest|<idx>|<UUID>] [--yolo] [--sandbox]``+- Final event: ``type == "result"`` (from+ ``packages/cli/src/nonInteractiveCli.ts:JsonStreamEventType.RESULT``)+- ``result`` event carries ``session_id`` + ``stats.duration_ms``+- Exit code 1 = ``FatalAuthenticationError``+- Auth: ``gemini auth login`` writes ``~/.gemini/``+"""++from __future__ import annotations++from pathlib import Path+from typing import Any, Dict, List, Optional++from services.cli_agent.protocol import CanonicalUsage+++class GoogleGeminiProvider:+ """v2 stub. Constructed only via direct instantiation in tests; the+ factory bypasses this class entirely and raises NotImplementedError."""++ name = "gemini"++ def __init__(self) -> None:+ raise NotImplementedError(+ "GoogleGeminiProvider is a v2 stub. The factory raises "+ "NotImplementedError for 'gemini' in v1."+ )++ # The Protocol surface below is declared so static type-checkers see+ # the class as a complete `AICliProvider`. None of these are reachable+ # because `__init__` raises.++ package_name: str = "@google/gemini-cli"+ binary_name: str = "gemini"+ ide_lock_env_var: Optional[str] = "GEMINI_IDE_LOCK"+ ide_lockfile_dir: Optional[Path] = None # config.py expands to <tmpdir>/gemini/ide++ def binary_path(self) -> Path: # pragma: no cover+ raise NotImplementedError++ def headless_argv(self, task: Any, *, defaults: Dict[str, Any]) -> List[str]: # pragma: no cover+ raise NotImplementedError++ def login_argv(self) -> List[str]: # pragma: no cover+ return ["gemini", "auth", "login"]++ def auth_status_argv(self) -> Optional[List[str]]: # pragma: no cover+ return ["gemini", "--version"]++ def detect_auth_error(self, stderr: str, exit_code: int) -> bool: # pragma: no cover+ return exit_code == 1 and "FatalAuthenticationError" in stderr++ def parse_event(self, line: str) -> Optional[Dict[str, Any]]: # pragma: no cover+ raise NotImplementedError++ def is_final_event(self, event: Dict[str, Any]) -> bool: # pragma: no cover+ return event.get("type") == "result"++ def event_to_session_result(+ self,+ events: List[Dict[str, Any]],+ stderr: str,+ exit_code: int,+ ) -> Dict[str, Any]: # pragma: no cover+ raise NotImplementedError++ def canonical_usage(self, events: List[Dict[str, Any]]) -> CanonicalUsage: # pragma: no cover+ return CanonicalUsage()++ def supports(self, feature: str) -> bool: # pragma: no cover+ return feature in {"session_id", "resume", "sandbox", "mcp_runtime", "ide_lockfile"}diff --git a/server/services/cli_agent/providers/openai_codex.py b/server/services/cli_agent/providers/openai_codex.pynew file mode 100644index 00000000..3070f349--- /dev/null+++ b/server/services/cli_agent/providers/openai_codex.py@@ -0,0 +1,310 @@+"""OpenAI Codex CLI provider.++Sandbox-first companion to Claude. Codex is a Rust binary+(``codex-rs/cli``); the npm package wraps it for distribution.++Subprocess: ``codex exec --json --model <m> --sandbox <mode>+--ask-for-approval <when> <prompt>``++Auth: native — ``codex login`` writes ``~/.codex/auth.json``. We do NOT+inject ``OPENAI_API_KEY`` ourselves (the CLI reads its own auth file).+If the user prefers env-based auth they can set the env in their shell;+we inherit it via ``os.environ`` in the session.++Feature surface (v1):+ - sandbox: read-only / workspace-write / danger-full-access+ - ask_for_approval: untrusted / on-request / never+ - NO sessions, NO resume, NO budget cap, NO max-turns+ - NO IDE lockfile (Codex CLI doesn't honor it yet); session writes+ no lockfile env var++Final event detection: best-effort by event-type matching ("complete",+"task_complete", "done") with stream-end fallback. The Codex CLI's JSON+schema isn't publicly documented, so this is fragile by design — pin+tests against vendored fixtures.++Cost: not exposed in JSON output. ``cost_usd=None`` always; aggregate+``summary.total_cost_usd`` becomes ``None`` if any task is Codex.+"""++from __future__ import annotations++import json+import shutil+from pathlib import Path+from typing import Any, Dict, List, Optional++from core.logging import get_logger++from services.cli_agent.config import get_provider_config+from services.cli_agent.protocol import CanonicalUsage+from services.cli_agent.types import CodexTaskSpec++logger = get_logger(__name__)++NAME = "codex"++# Best-effort markers for Codex's "stream-end" / "task complete" event.+# Update if/when OpenAI publishes a stable schema.+_FINAL_EVENT_TYPES = frozenset({+ "complete", "task_complete", "done", "finished", "result",+})++_AUTH_ERROR_MARKERS = (+ "OPENAI_API_KEY not set",+ "401 Unauthorized",+ "Authentication required",+ "Not authenticated",+ "Please run 'codex login'",+ "Please run `codex login`",+ "Invalid API key",+ "auth/missing",+)+++class OpenAICodexProvider:+ """`AICliProvider` for OpenAI's Codex CLI."""++ def __init__(self) -> None:+ cfg = get_provider_config(NAME)+ if cfg is None:+ raise RuntimeError(+ f"Provider config missing for {NAME!r}. Check ai_cli_providers.json."+ )+ self.name = NAME+ self.package_name = cfg.package_name+ self.binary_name = cfg.binary_name+ self.ide_lock_env_var = cfg.ide_lock_env_var+ self.ide_lockfile_dir = cfg.ide_lockfile_dir+ self._defaults = cfg.defaults+ self._supports = cfg.supports+ self._login_argv = cfg.login_argv+ self._auth_status_argv = cfg.auth_status_argv++ # ---- spawn surface ---------------------------------------------------++ def binary_path(self) -> Path:+ which_codex = shutil.which(self.binary_name)+ if which_codex:+ return Path(which_codex)++ npx = shutil.which("npx")+ if npx:+ return Path(npx)++ raise FileNotFoundError(+ f"Neither {self.binary_name!r} nor 'npx' found in PATH. "+ f"Install with: npm install -g {self.package_name}"+ )++ def headless_argv(+ self,+ task: Any, # CodexTaskSpec+ *,+ defaults: Dict[str, Any],+ ) -> List[str]:+ if not isinstance(task, CodexTaskSpec):+ raise TypeError(+ "OpenAICodexProvider.headless_argv requires CodexTaskSpec, "+ f"got {type(task).__name__}"+ )++ which_codex = shutil.which(self.binary_name)+ if which_codex:+ argv: List[str] = [which_codex]+ else:+ npx = shutil.which("npx")+ if not npx:+ raise FileNotFoundError(+ f"Neither {self.binary_name!r} nor 'npx' found in PATH"+ )+ argv = [npx, "--yes", self.package_name]++ argv += ["exec", "--json"]++ model = (+ task.model+ or defaults.get("default_model")+ or self._defaults.get("default_model", "gpt-5.2-codex")+ )+ argv += ["--model", model]++ sandbox = task.sandbox or defaults.get(+ "default_sandbox",+ self._defaults.get("default_sandbox", "workspace-write"),+ )+ argv += ["--sandbox", sandbox]++ ask = task.ask_for_approval or defaults.get(+ "default_ask_for_approval",+ self._defaults.get("default_ask_for_approval", "never"),+ )+ argv += ["--ask-for-approval", ask]++ # System prompt — Codex doesn't have a dedicated --system-prompt+ # flag in `exec`. Prepend it to the user prompt with a clear+ # divider when present.+ prompt = task.prompt+ if task.system_prompt:+ prompt = f"<system>\n{task.system_prompt}\n</system>\n\n{task.prompt}"++ argv += [prompt]+ return argv++ # ---- native auth -----------------------------------------------------++ def login_argv(self) -> List[str]:+ return list(self._login_argv) or ["codex", "login"]++ def auth_status_argv(self) -> Optional[List[str]]:+ return list(self._auth_status_argv) if self._auth_status_argv else None++ def detect_auth_error(self, stderr: str, exit_code: int) -> bool:+ if not stderr and exit_code == 0:+ return False+ return any(m in stderr for m in _AUTH_ERROR_MARKERS)++ # ---- streaming output parsing ---------------------------------------++ def parse_event(self, line: str) -> Optional[Dict[str, Any]]:+ line = line.strip()+ if not line:+ return None+ try:+ return json.loads(line)+ except json.JSONDecodeError:+ return None++ def is_final_event(self, event: Dict[str, Any]) -> bool:+ return event.get("type") in _FINAL_EVENT_TYPES++ def event_to_session_result(+ self,+ events: List[Dict[str, Any]],+ stderr: str,+ exit_code: int,+ ) -> Dict[str, Any]:+ # Codex doesn't have a single canonical "result" event; treat the+ # last assistant message as the response, with the final event+ # (if any) for metadata.+ final = next(+ (e for e in reversed(events) if e.get("type") in _FINAL_EVENT_TYPES),+ None,+ )++ response = self._extract_response(events)+ provider_data = self._extract_provider_data(events)++ tool_calls = sum(+ 1 for e in events+ if e.get("type") in ("tool_call", "tool_use", "function_call")+ )++ success = exit_code == 0+ error: Optional[str] = None+ if exit_code != 0:+ error = stderr.strip()[-2000:] or f"codex exited with code {exit_code}"+ elif not response:+ # Codex sometimes emits no terminal event; only fail if there's+ # nothing useful in the stream at all.+ if not events:+ success = False+ error = "no events received from codex"++ # session_id: Codex doesn't expose one; leave None.+ session_id: Optional[str] = None+ if final and isinstance(final.get("session_id"), str):+ session_id = final["session_id"]++ # duration_ms / num_turns: best-effort lookups+ duration_ms = None+ num_turns = None+ if final:+ stats = final.get("stats") or {}+ duration_ms = stats.get("duration_ms") or final.get("duration_ms")+ num_turns = stats.get("turns") or final.get("turns")++ return {+ "session_id": session_id,+ "response": response,+ "cost_usd": None, # Codex doesn't expose cost in --json+ "duration_ms": duration_ms,+ "num_turns": num_turns,+ "tool_calls": tool_calls,+ "canonical_usage": self.canonical_usage(events),+ "provider_data": provider_data,+ "success": success,+ "error": error,+ }++ def canonical_usage(self, events: List[Dict[str, Any]]) -> CanonicalUsage:+ # Codex doesn't expose token counts in `--json` output. Return zeros.+ # If a future Codex schema adds usage, extract here.+ request_count = sum(+ 1 for e in events+ if e.get("type") in ("assistant", "message", "response")+ )+ return CanonicalUsage(request_count=request_count)++ # ---- feature gating --------------------------------------------------++ def supports(self, feature: str) -> bool:+ return feature in self._supports++ # ---- internals -------------------------------------------------------++ @staticmethod+ def _extract_response(events: List[Dict[str, Any]]) -> str:+ """Best-effort: walk the event stream looking for the final+ assistant message body."""+ # Try in priority order: explicit `result` field on a final event,+ # then last assistant `message`/`text`/`content`, then concatenated+ # delta text.+ for evt in reversed(events):+ if evt.get("type") in _FINAL_EVENT_TYPES:+ for key in ("result", "response", "text", "content"):+ val = evt.get(key)+ if isinstance(val, str) and val:+ return val+ msg = evt.get("message")+ if isinstance(msg, dict):+ val = msg.get("content") or msg.get("text")+ if isinstance(val, str):+ return val+ break # one final event is enough++ # Last assistant message+ for evt in reversed(events):+ if evt.get("type") not in ("assistant", "message"):+ continue+ for key in ("text", "content", "response"):+ val = evt.get(key)+ if isinstance(val, str) and val:+ return val+ msg = evt.get("message")+ if isinstance(msg, dict):+ val = msg.get("content") or msg.get("text")+ if isinstance(val, str) and val:+ return val++ # Concatenate delta-style text events+ parts: List[str] = []+ for evt in events:+ if evt.get("type") in ("delta", "text", "content"):+ val = evt.get("text") or evt.get("content")+ if isinstance(val, str):+ parts.append(val)+ return "".join(parts)++ @staticmethod+ def _extract_provider_data(events: List[Dict[str, Any]]) -> Dict[str, Any]:+ """Pull Codex-specific metadata into provider_data."""+ data: Dict[str, Any] = {}+ # Last assistant message — capture call_id / response_item_id+ for evt in reversed(events):+ if evt.get("type") in ("assistant", "message", "response"):+ for key in ("call_id", "response_item_id", "id", "model"):+ if key in evt and evt[key] is not None:+ data.setdefault(key, evt[key])+ break+ return datadiff --git a/server/services/cli_agent/service.py b/server/services/cli_agent/service.pynew file mode 100644index 00000000..8b17aa34--- /dev/null+++ b/server/services/cli_agent/service.py@@ -0,0 +1,415 @@+"""`AICliService.run_batch()` — top-level entry for `claude_code_agent` /+`codex_agent` plugins.++Runs N parallel `AICliSession`s under an `asyncio.Semaphore`, mirroring+the semaphore-+-gather pattern already used in+`nodes/document/file_downloader.py`. No separate pool class — the+machinery is small enough to live inline.++Per-batch lifecycle:++ 1. Verify `working_directory` is a git repo (uses `git rev-parse --show-toplevel`).+ 2. Allocate a bearer token, register a `BatchContext` in the MCP server.+ 3. `asyncio.gather` N sessions, each wrapped in `_run_session` with+ try/finally cleanup.+ 4. Aggregate per-task `SessionResult`s into a `BatchResult`.+ 5. Deregister the bearer token in the `finally` so 401s flip on the+ next MCP request after the batch settles.++Active sessions are tracked in `_active_sessions[(workflow_id, node_id)]`+so workflow cancel can target them.+"""++from __future__ import annotations++import asyncio+import os+import time+from datetime import datetime, timezone+from pathlib import Path+from typing import Any, Dict, Iterable, List, Optional, Tuple++import anyio++from core.logging import get_logger++from services.cli_agent.config import get_provider_config+from services.cli_agent.factory import create_cli_provider+from services.cli_agent.mcp_server import (+ BatchContext,+ issue_token,+ register_batch,+ unregister_batch,+)+from services.cli_agent.protocol import BatchResult, SessionResult+from services.cli_agent.session import AICliSession+from services.cli_agent.types import BaseAICliTaskSpec++logger = get_logger(__name__)+++DEFAULT_MAX_PARALLEL = 5++BatchKey = Tuple[str, str] # (workflow_id, node_id)+++class AICliService:+ """Singleton service. Use `get_ai_cli_service()` to access."""++ def __init__(self) -> None:+ # workflow_id+node_id -> live session list (for cancel targeting).+ self._active_sessions: Dict[BatchKey, List[AICliSession]] = {}+ self._lock = asyncio.Lock()++ # ------------------------------------------------------------------+ # Public API+ # ------------------------------------------------------------------++ async def run_batch(+ self,+ provider_name: str,+ *,+ tasks: Iterable[BaseAICliTaskSpec],+ node_id: str,+ workflow_id: str,+ workspace_dir: Path,+ broadcaster: Any,+ repo_root: Optional[Path] = None,+ connected_skill_names: Optional[List[str]] = None,+ allowed_credentials: Optional[List[str]] = None,+ max_parallel: int = DEFAULT_MAX_PARALLEL,+ mcp_port: Optional[int] = None,+ ) -> BatchResult:+ """Run a list of CLI tasks under one batch.++ Returns:+ `BatchResult` aggregating per-task `SessionResult`s.++ Raises:+ ValueError / NotImplementedError: provider unknown / v2-deferred.+ """+ provider = create_cli_provider(provider_name)+ task_list: List[BaseAICliTaskSpec] = list(tasks)++ # Verify the working directory is under a git repo.+ resolved_repo_root = await self._resolve_repo_root(+ workspace_dir=workspace_dir, override=repo_root,+ )+ if resolved_repo_root is None:+ return self._abort_not_git_repo(+ provider_name=provider_name,+ tasks=task_list,+ )++ # Per-batch bearer token + MCP context+ token = issue_token()+ port = mcp_port or int(os.environ.get("MACHINA_BACKEND_PORT", "3010"))+ ctx = BatchContext(+ workflow_id=workflow_id,+ node_id=node_id,+ workspace_dir=Path(workspace_dir).resolve(),+ connected_skill_names=set(connected_skill_names or []),+ allowed_credentials=set(allowed_credentials or []),+ broadcaster=broadcaster,+ )+ register_batch(token, ctx)++ cfg = get_provider_config(provider_name)+ defaults = dict(cfg.defaults) if cfg else {}++ key: BatchKey = (workflow_id, node_id)+ async with self._lock:+ if key in self._active_sessions:+ logger.warning(+ "[ai_cli_service] replacing stale session list for %s", key,+ )+ # Cancel anything previously left dangling.+ for sess in self._active_sessions[key]:+ try:+ await sess.cleanup()+ except Exception:+ pass+ self._active_sessions[key] = []++ start = time.monotonic()+ await self._broadcast_phase(broadcaster, node_id, workflow_id, "batch_started", {+ "provider": provider_name,+ "n_tasks": len(task_list),+ "max_parallel": max_parallel,+ "isolation": "worktree",+ })++ sem = asyncio.Semaphore(max(1, int(max_parallel)))++ async def run_one(task: BaseAICliTaskSpec) -> SessionResult:+ async with sem:+ session = AICliSession(+ provider=provider, task=task,+ repo_root=resolved_repo_root, workspace_dir=workspace_dir,+ node_id=node_id, workflow_id=workflow_id,+ broadcaster=broadcaster, defaults=defaults,+ mcp_port=port, batch_token=token,+ )+ async with self._lock:+ self._active_sessions[key].append(session)+ try:+ try:+ await session.start()+ except FileNotFoundError as exc:+ return self._fail_result(provider_name, task, session.task_id,+ f"cli_not_installed: {exc}")+ except RuntimeError as exc:+ # `_pre_spawn` raises on git-worktree failure.+ return self._fail_result(provider_name, task, session.task_id,+ f"worktree_setup_failed: {exc}")+ except Exception as exc:+ logger.exception("[ai_cli_service] start failed")+ return self._fail_result(provider_name, task, session.task_id,+ f"start_failed: {exc}")+ return await session.wait_for_completion(task.timeout_seconds)+ finally:+ try:+ await session.cleanup()+ except Exception as exc:+ logger.debug("[ai_cli_service] cleanup: %s", exc)+ async with self._lock:+ try:+ self._active_sessions[key].remove(session)+ except (KeyError, ValueError):+ pass++ try:+ results: List[SessionResult] = await asyncio.gather(+ *(run_one(t) for t in task_list),+ return_exceptions=False,+ )+ finally:+ async with self._lock:+ self._active_sessions.pop(key, None)+ unregister_batch(token)++ elapsed_ms = int((time.monotonic() - start) * 1000)+ n_succeeded = sum(1 for r in results if r.success)+ n_failed = len(results) - n_succeeded++ # Cost roll-up: prefer the provider's reported cost (Claude exposes+ # `total_cost_usd` natively); for providers that don't (Codex,+ # Gemini v2), derive USD from `canonical_usage` via the existing+ # PricingService — a single source of truth for all LLM cost in+ # MachinaOs.+ for r in results:+ if r.cost_usd is None:+ derived = self._derive_cost(r, task_list)+ if derived is not None:+ r.cost_usd = derived++ costs = [r.cost_usd for r in results]+ total_cost = (+ None if any(c is None for c in costs) else round(sum(c or 0 for c in costs), 6)+ )++ result = BatchResult(+ tasks=results,+ n_tasks=len(results),+ n_succeeded=n_succeeded,+ n_failed=n_failed,+ total_cost_usd=total_cost,+ wall_clock_ms=elapsed_ms,+ budget_remaining_usd=None,+ provider=provider_name,+ timestamp=datetime.now(timezone.utc).isoformat(),+ )++ await self._broadcast_phase(broadcaster, node_id, workflow_id, "batch_complete", {+ "provider": provider_name,+ "n_succeeded": n_succeeded,+ "n_failed": n_failed,+ "total_cost_usd": total_cost,+ "wall_clock_ms": elapsed_ms,+ })+ return result++ async def cancel_workflow(self, workflow_id: str) -> int:+ """Cancel every active session for a workflow. Returns count cancelled."""+ cancelled = 0+ async with self._lock:+ keys = [k for k in self._active_sessions if k[0] == workflow_id]+ sessions: List[AICliSession] = []+ for k in keys:+ sessions.extend(self._active_sessions[k])+ for sess in sessions:+ try:+ await sess.cleanup()+ cancelled += 1+ except Exception as exc:+ logger.debug("[ai_cli_service] cancel: %s", exc)+ return cancelled++ async def cancel_node(self, node_id: str) -> int:+ """Cancel every active session for a node. Returns count cancelled."""+ cancelled = 0+ async with self._lock:+ keys = [k for k in self._active_sessions if k[1] == node_id]+ sessions: List[AICliSession] = []+ for k in keys:+ sessions.extend(self._active_sessions[k])+ for sess in sessions:+ try:+ await sess.cleanup()+ cancelled += 1+ except Exception as exc:+ logger.debug("[ai_cli_service] cancel: %s", exc)+ return cancelled++ # ------------------------------------------------------------------+ # Internals+ # ------------------------------------------------------------------++ @staticmethod+ async def _resolve_repo_root(+ *,+ workspace_dir: Path,+ override: Optional[Path],+ ) -> Optional[Path]:+ """Find the git repo root via `git rev-parse --show-toplevel`.++ Contract:+ - When `override` is given, only consider that subtree.+ - When not given, try `workspace_dir` first, then `cwd`.+ """+ starts: List[Path]+ if override is not None:+ starts = [Path(override).resolve()]+ else:+ starts = [Path(workspace_dir).resolve(), Path.cwd().resolve()]++ for start in starts:+ try:+ result = await anyio.run_process(+ ["git", "-C", str(start), "rev-parse", "--show-toplevel"],+ check=False,+ )+ except FileNotFoundError:+ # `git` not on PATH at all — fail-fast, nothing to fall back to.+ return None+ if result.returncode == 0:+ root_text = (result.stdout or b"").decode("utf-8", errors="replace").strip()+ if root_text:+ return Path(root_text)+ return None++ @staticmethod+ def _derive_cost(+ result: SessionResult,+ tasks: List[BaseAICliTaskSpec],+ ) -> Optional[float]:+ """Compute USD cost from `canonical_usage` via the central+ `PricingService`. Returns None when token counts are zero (the+ provider didn't surface them) — keeps the contract that+ ``cost_usd is None`` means "we genuinely don't know the cost"."""+ cu = result.canonical_usage+ total_tokens = cu.input_tokens + cu.output_tokens + cu.cache_read + cu.cache_write+ if total_tokens == 0:+ return None++ # Find the model the task requested (or the provider's default).+ model = ""+ for t in tasks:+ if (t.task_id or "") == result.task_id:+ model = t.model or ""+ break++ try:+ from services.pricing import get_pricing_service+ pricing = get_pricing_service()+ breakdown = pricing.calculate_cost(+ provider=result.provider,+ model=model,+ input_tokens=cu.input_tokens,+ output_tokens=cu.output_tokens,+ cache_read_tokens=cu.cache_read,+ cache_creation_tokens=cu.cache_write,+ reasoning_tokens=cu.reasoning_tokens,+ )+ total = breakdown.get("total_cost")+ return float(total) if total else None+ except Exception as exc: # pragma: no cover — pricing is non-critical+ logger.debug("[ai_cli_service] pricing lookup failed: %s", exc)+ return None++ @staticmethod+ def _fail_result(+ provider_name: str,+ task: BaseAICliTaskSpec,+ task_id: str,+ error: str,+ ) -> SessionResult:+ return SessionResult(+ task_id=task_id,+ provider=provider_name,+ prompt=task.prompt,+ success=False,+ error=error,+ )++ @staticmethod+ async def _broadcast_phase(+ broadcaster: Any,+ node_id: str,+ workflow_id: str,+ phase: str,+ data: dict,+ ) -> None:+ if not broadcaster:+ return+ try:+ await broadcaster.update_node_status(+ node_id,+ "executing",+ {"phase": phase, **data},+ workflow_id=workflow_id,+ )+ except Exception:+ pass++ def _abort_not_git_repo(+ self,+ *,+ provider_name: str,+ tasks: List[BaseAICliTaskSpec],+ ) -> BatchResult:+ results: List[SessionResult] = [+ SessionResult(+ task_id=t.task_id or "t_unstarted",+ provider=provider_name,+ prompt=t.prompt,+ success=False,+ error="working_directory_not_git_repo",+ )+ for t in tasks+ ]+ return BatchResult(+ tasks=results,+ n_tasks=len(results),+ n_succeeded=0,+ n_failed=len(results),+ total_cost_usd=None,+ wall_clock_ms=0,+ budget_remaining_usd=None,+ provider=provider_name,+ timestamp=datetime.now(timezone.utc).isoformat(),+ )+++# ---------------------------------------------------------------------------+# Singleton accessor+# ---------------------------------------------------------------------------++_instance: Optional[AICliService] = None+++def get_ai_cli_service() -> AICliService:+ global _instance+ if _instance is None:+ _instance = AICliService()+ return _instancediff --git a/server/services/cli_agent/session.py b/server/services/cli_agent/session.pynew file mode 100644index 00000000..1a488af5--- /dev/null+++ b/server/services/cli_agent/session.py@@ -0,0 +1,396 @@+"""One CLI session per task — `BaseProcessSupervisor` subclass.++Each session is bound to:+ - one provider (Claude or Codex)+ - one task spec (`ClaudeTaskSpec` / `CodexTaskSpec`)+ - one git worktree (created in `_pre_spawn`, removed in `cleanup`)+ - one IDE lockfile (written in `_pre_spawn` when the provider supports it)++Inherits `BaseProcessSupervisor`'s locked idempotent start/stop, recursive+`kill_tree`, `terminate_then_kill(5s)` grace, and Windows+`CTRL_BREAK_EVENT` path. We override `_do_start` to wire NDJSON consumers+instead of the parent's generic `drain_stream(logger.info)`.++Sessions are NOT registered in the global supervisor registry — they're+owned by `AICliService.run_batch()` for the lifetime of one batch.++Liveness: relies on `wait_for_completion(timeout_seconds)` as the watchdog+and the per-broadcast Temporal heartbeat fired by every+`update_node_status()` call. We do not run our own per-second heartbeat+loop or write diagnostic dump files.+"""++from __future__ import annotations++import asyncio+import json+import os+import subprocess+import sys+import uuid+from pathlib import Path+from typing import Any, Dict, List, Optional++import anyio++from core.logging import get_logger+from services._supervisor.process import BaseProcessSupervisor+from services.cli_agent.lockfile import remove_ide_lockfile, write_ide_lockfile+from services.cli_agent.protocol import AICliProvider, CanonicalUsage, SessionResult+from services.cli_agent.types import BaseAICliTaskSpec++logger = get_logger(__name__)+++class AICliSession(BaseProcessSupervisor):+ """One CLI subprocess for one task."""++ pipe_streams = True+ terminate_grace_seconds = 5.0+ graceful_shutdown = sys.platform == "win32"++ def __init__(+ self,+ *,+ provider: AICliProvider,+ task: BaseAICliTaskSpec,+ repo_root: Path,+ workspace_dir: Path,+ node_id: str,+ workflow_id: str,+ broadcaster: Any,+ defaults: Dict[str, Any],+ mcp_port: int,+ batch_token: str,+ ) -> None:+ super().__init__()+ self._provider = provider+ self._task = task+ self._task_id = task.task_id or f"t_{uuid.uuid4().hex[:8]}"+ self._repo_root = Path(repo_root).resolve()+ self._worktree_dir = (+ Path(workspace_dir).resolve() / node_id / f"wt_{self._task_id}"+ )+ self._branch = task.branch or f"machina/{self._task_id}"+ self._broadcaster = broadcaster+ self._defaults = defaults+ self._mcp_port = mcp_port+ self._batch_token = batch_token+ self._node_id = node_id+ self._workflow_id = workflow_id++ # Streaming state+ self._events: List[Dict[str, Any]] = []+ self._stderr_lines: List[str] = []+ self._exit_code: Optional[int] = None+ self._lockfile_path: Optional[Path] = None++ # ------------------------------------------------------------------+ # Identity+ # ------------------------------------------------------------------++ @property+ def label(self) -> str:+ return f"AICliSession_{self._provider.name}_{self._task_id}"++ @property+ def task_id(self) -> str:+ return self._task_id++ @property+ def branch(self) -> str:+ return self._branch++ @property+ def worktree_dir(self) -> Path:+ return self._worktree_dir++ # ------------------------------------------------------------------+ # BaseProcessSupervisor surface+ # ------------------------------------------------------------------++ def binary_path(self) -> Path:+ return self._provider.binary_path()++ def argv(self) -> List[str]:+ return self._provider.headless_argv(self._task, defaults=self._defaults)++ def cwd(self) -> Optional[Path]:+ return self._worktree_dir++ def env(self) -> Dict[str, str]:+ # Inherit parent env so the CLI finds its own auth in+ # `~/.claude/`, `~/.codex/`, `~/.gemini/`. Native CLI auth model:+ # we never inject CLAUDE_CONFIG_DIR / GEMINI_CLI_HOME.+ e: Dict[str, str] = {**os.environ, "PYTHONUNBUFFERED": "1"}+ if self._lockfile_path and self._provider.ide_lock_env_var:+ e[self._provider.ide_lock_env_var] = str(self._lockfile_path)+ # Composio-style parent-run-ID for MCP correlation+ e["MACHINA_PARENT_RUN_ID"] = (+ f"{self._workflow_id}:{self._node_id}:{self._batch_token[:8]}"+ )+ return e++ async def _pre_spawn(self) -> None:+ """Create the per-task git worktree and (if supported) write the+ IDE lockfile. Failures abort `_do_start` cleanly via RuntimeError."""+ # 1. git worktree+ self._worktree_dir.parent.mkdir(parents=True, exist_ok=True)+ wt_proc = await anyio.run_process(+ [+ "git", "-C", str(self._repo_root),+ "worktree", "add",+ str(self._worktree_dir),+ "-b", self._branch,+ ],+ check=False,+ )+ if wt_proc.returncode != 0:+ err = (wt_proc.stderr or b"").decode("utf-8", errors="replace").strip()+ raise RuntimeError(f"git worktree add failed: {err}")++ # 2. IDE lockfile (VSCode pattern) — providers that support it+ if self._provider.supports("ide_lockfile") and self._provider.ide_lockfile_dir:+ try:+ self._lockfile_path = write_ide_lockfile(+ ide_lockfile_dir=self._provider.ide_lockfile_dir,+ pid=os.getpid(),+ port=self._mcp_port,+ token=self._batch_token,+ workspace_dir=self._worktree_dir,+ ide_name=self._provider.name,+ )+ except OSError as exc:+ logger.warning(+ "[%s] IDE lockfile write failed (%s) — continuing without MCP tools",+ self.label, exc,+ )++ # ------------------------------------------------------------------+ # _do_start: replace parent's drain tasks with NDJSON consumers+ # ------------------------------------------------------------------++ async def _do_start(self) -> None:+ binary = self.binary_path()+ if not binary.exists():+ raise FileNotFoundError(f"{self.label} binary not found at {binary}")++ await self._pre_spawn()++ kwargs: Dict[str, Any] = {+ "cwd": str(self.cwd()),+ "env": self.env(),+ "stdout": subprocess.PIPE,+ "stderr": subprocess.PIPE,+ }+ if sys.platform == "win32" and self.graceful_shutdown:+ kwargs["creationflags"] = subprocess.CREATE_NEW_PROCESS_GROUP # type: ignore[attr-defined]++ self._proc = await anyio.open_process(self.argv(), **kwargs)+ self._logger.info(+ "[%s] spawned pid=%s task=%s branch=%s",+ self.label, self._proc.pid, self._task_id, self._branch,+ )++ # NDJSON consumers replace the parent's `drain_stream(logger.info)`.+ # We populate `self._drain_tasks` so the parent's `_do_stop`+ # cancels them on stop.+ self._drain_tasks = [+ asyncio.create_task(self._consume_stdout(self._proc.stdout)),+ asyncio.create_task(self._consume_stderr(self._proc.stderr)),+ ]++ # ------------------------------------------------------------------+ # Stream consumers+ # ------------------------------------------------------------------++ async def _consume_stdout(self, stream: Optional[anyio.abc.ByteReceiveStream]) -> None:+ if stream is None:+ return+ buf = b""+ try:+ async for chunk in stream:+ buf += chunk+ while b"\n" in buf:+ raw, buf = buf.split(b"\n", 1)+ text = raw.decode("utf-8", errors="replace").strip()+ if not text:+ continue+ event = self._provider.parse_event(text)+ if event is None:+ continue+ self._events.append(event)+ await self._on_event(event)+ if buf:+ text = buf.decode("utf-8", errors="replace").strip()+ if text:+ event = self._provider.parse_event(text)+ if event is not None:+ self._events.append(event)+ await self._on_event(event)+ except (anyio.ClosedResourceError, anyio.EndOfStream, asyncio.CancelledError):+ pass+ except Exception as exc: # pragma: no cover — defensive+ self._logger.debug("[%s] stdout consumer ended: %s", self.label, exc)++ async def _consume_stderr(self, stream: Optional[anyio.abc.ByteReceiveStream]) -> None:+ if stream is None:+ return+ buf = b""+ try:+ async for chunk in stream:+ buf += chunk+ while b"\n" in buf:+ raw, buf = buf.split(b"\n", 1)+ text = raw.decode("utf-8", errors="replace").rstrip()+ if not text:+ continue+ self._stderr_lines.append(text)+ await self._safe_terminal_log(text, level="error")+ if buf:+ text = buf.decode("utf-8", errors="replace").rstrip()+ if text:+ self._stderr_lines.append(text)+ await self._safe_terminal_log(text, level="error")+ except (anyio.ClosedResourceError, anyio.EndOfStream, asyncio.CancelledError):+ pass+ except Exception as exc: # pragma: no cover+ self._logger.debug("[%s] stderr consumer ended: %s", self.label, exc)++ # ------------------------------------------------------------------+ # Event dispatch (UI broadcasts)+ # ------------------------------------------------------------------++ async def _on_event(self, event: Dict[str, Any]) -> None:+ if self._provider.is_final_event(event):+ payload = {+ "phase": "ai_cli_subtask",+ "task_id": self._task_id,+ "provider": self._provider.name,+ "status": "finalising",+ }+ for k in ("total_cost_usd", "duration_ms", "num_turns", "session_id"):+ v = event.get(k)+ if v is not None:+ payload[k] = v+ await self._safe_node_status("executing", payload)+ else:+ msg = (+ event.get("message")+ or event.get("text")+ or event.get("delta")+ or json.dumps(event)+ )+ text = msg if isinstance(msg, str) else json.dumps(msg)+ await self._safe_terminal_log(text[:500], level="info")++ async def _safe_terminal_log(self, message: str, *, level: str) -> None:+ if not self._broadcaster:+ return+ try:+ await self._broadcaster.broadcast_terminal_log({+ "source": f"{self._provider.name}:{self._task_id}",+ "level": level,+ "message": message,+ })+ except Exception:+ pass++ async def _safe_node_status(self, status: str, data: Dict[str, Any]) -> None:+ if not self._broadcaster:+ return+ try:+ await self._broadcaster.update_node_status(+ self._node_id, status, data,+ workflow_id=self._workflow_id,+ )+ except Exception:+ pass++ # ------------------------------------------------------------------+ # Public lifecycle+ # ------------------------------------------------------------------++ async def wait_for_completion(self, timeout_seconds: int) -> SessionResult:+ """Wait for the CLI to exit, with hard timeout watchdog."""+ if self._proc is None:+ return self._build_result(+ success=False, error="session never started",+ )++ try:+ await asyncio.wait_for(self._proc.wait(), timeout=timeout_seconds)+ self._exit_code = self._proc.returncode+ return self._build_result(success=(self._exit_code == 0))+ except asyncio.TimeoutError:+ await self.stop()+ return self._build_result(+ success=False,+ error=f"timeout after {timeout_seconds}s",+ )++ async def cleanup(self) -> None:+ """Stop the process and remove the worktree + lockfile."""+ try:+ await self.stop()+ except Exception as exc:+ self._logger.debug("[%s] stop during cleanup: %s", self.label, exc)++ if self._lockfile_path:+ remove_ide_lockfile(self._lockfile_path)+ self._lockfile_path = None++ # Remove the worktree (force, since branch lifecycle is the+ # batch's responsibility — best-effort).+ try:+ await anyio.run_process(+ [+ "git", "-C", str(self._repo_root),+ "worktree", "remove", "--force",+ str(self._worktree_dir),+ ],+ check=False,+ )+ except Exception as exc:+ self._logger.debug("[%s] worktree remove: %s", self.label, exc)++ # ------------------------------------------------------------------+ # Result construction+ # ------------------------------------------------------------------++ def _build_result(+ self,+ *,+ success: bool,+ error: Optional[str] = None,+ ) -> SessionResult:+ provider_result = self._provider.event_to_session_result(+ self._events,+ "\n".join(self._stderr_lines),+ self._exit_code if self._exit_code is not None else -1,+ )++ canonical = provider_result.get("canonical_usage")+ if not isinstance(canonical, CanonicalUsage):+ canonical = self._provider.canonical_usage(self._events)++ final_success = success and provider_result.get("success", True)+ final_error = error or provider_result.get("error")++ return SessionResult(+ task_id=self._task_id,+ session_id=provider_result.get("session_id"),+ provider=self._provider.name,+ prompt=self._task.prompt,+ branch=self._branch,+ worktree_path=str(self._worktree_dir),+ response=str(provider_result.get("response") or "")[:4000],+ cost_usd=provider_result.get("cost_usd"),+ duration_ms=provider_result.get("duration_ms"),+ num_turns=provider_result.get("num_turns"),+ tool_calls=int(provider_result.get("tool_calls", 0)),+ canonical_usage=canonical,+ provider_data=dict(provider_result.get("provider_data") or {}),+ success=final_success,+ error=final_error if not final_success else None,+ )diff --git a/server/services/cli_agent/types.py b/server/services/cli_agent/types.pynew file mode 100644index 00000000..7f58b3d7--- /dev/null+++ b/server/services/cli_agent/types.py@@ -0,0 +1,202 @@+"""Pydantic task specs (discriminated union) + result models.++The discriminated union lets each plugin's `Params.tasks` be hard-typed+to one variant (e.g. `list[ClaudeTaskSpec]`), so the LLM tool schema+fast-path at `services/ai.py:2898` produces a clean per-provider schema+with no `$defs`/`$ref`.++v1 ships Claude + Codex plugins fully wired. `GeminiTaskSpec` is part of+the union (and exported) so the type system, factory dispatch keys, and+config JSON are ready, but `create_cli_provider("gemini")` raises+`NotImplementedError`. v2 swaps the stub for the real impl.+"""++from __future__ import annotations++from typing import Annotated, Any, Dict, List, Literal, Optional, Union++from pydantic import BaseModel, ConfigDict, Field++from services.cli_agent.protocol import CanonicalUsage+++# ---------------------------------------------------------------------------+# Task specs (discriminated union)+# ---------------------------------------------------------------------------++class BaseAICliTaskSpec(BaseModel):+ """Shared fields for every CLI task."""+ task_id: Optional[str] = Field(+ default=None,+ description="Auto-assigned `t_<8hex>` if omitted.",+ )+ prompt: str = Field(+ ...,+ description="The task prompt sent to the CLI.",+ json_schema_extra={"rows": 4},+ )+ branch: Optional[str] = Field(+ default=None,+ description="Branch name for the per-task git worktree. "+ "Auto-named `machina/<task_id>` if omitted.",+ )+ model: Optional[str] = None+ timeout_seconds: int = Field(+ default=600, ge=10, le=3600,+ description="Hard timeout per task. On expiry the session is "+ "terminate_then_kill'd and a diagnostic dump is "+ "written to ~/.claude-machina/logs/.",+ )+ system_prompt: Optional[str] = Field(+ default=None,+ json_schema_extra={"rows": 3},+ )++ model_config = ConfigDict(extra="ignore")+++class ClaudeTaskSpec(BaseAICliTaskSpec):+ """Claude Code CLI task. Full feature set: sessions, resume, budget,+ turns, allowed_tools, permission_mode."""+ provider: Literal["claude"] = "claude"+ session_id: Optional[str] = Field(+ default=None,+ description="Start a named session. Pair with `resume_session_id` "+ "to chain conversations.",+ )+ resume_session_id: Optional[str] = Field(+ default=None,+ description="Resume from a prior session_id (returned in a previous "+ "task's `result` event).",+ )+ max_turns: Optional[int] = Field(+ default=None, ge=1,+ description="Per-task turn cap. Defaults to provider config.",+ )+ max_budget_usd: Optional[float] = Field(+ default=None, ge=0,+ description="Per-task USD budget. Defaults to provider config.",+ )+ allowed_tools: Optional[str] = Field(+ default=None,+ description="Comma-separated tool list. Defaults to "+ "Read,Edit,Bash,Glob,Grep,Write.",+ )+ permission_mode: Literal["default", "acceptEdits", "plan", "auto"] = "acceptEdits"+++class CodexTaskSpec(BaseAICliTaskSpec):+ """OpenAI Codex CLI task. Sandbox-first; no session/resume/budget/turns."""+ provider: Literal["codex"] = "codex"+ sandbox: Literal["read-only", "workspace-write", "danger-full-access"] = (+ "workspace-write"+ )+ ask_for_approval: Literal["untrusted", "on-request", "never"] = "never"+++class GeminiTaskSpec(BaseAICliTaskSpec):+ """Google Gemini CLI task (v2 — stub provider in v1).++ Schema lives in v1 so the discriminated union JSON Schema for the+ LLM tool fast-path doesn't change when v2 lands.+ """+ provider: Literal["gemini"] = "gemini"+ session_id: Optional[str] = None+ resume: Optional[str] = Field(+ default=None,+ description='"latest" | "<index>" | "<UUID>"',+ )+ yolo: bool = False+ sandbox: bool = False+++AICliTaskSpec = Annotated[+ Union[ClaudeTaskSpec, CodexTaskSpec, GeminiTaskSpec],+ Field(discriminator="provider"),+]+++# ---------------------------------------------------------------------------+# Result models (Pydantic for serialisation; mirror dataclasses in protocol.py)+# ---------------------------------------------------------------------------++class CanonicalUsagePydantic(BaseModel):+ """Pydantic mirror of `protocol.CanonicalUsage` for output serialisation."""+ input_tokens: int = 0+ output_tokens: int = 0+ cache_read: int = 0+ cache_write: int = 0+ reasoning_tokens: int = 0+ request_count: int = 0+++class SessionResultModel(BaseModel):+ """Per-task result, JSON-serialisable."""+ task_id: str+ session_id: Optional[str] = None+ provider: str = ""+ prompt: str = ""+ branch: Optional[str] = None+ worktree_path: Optional[str] = None+ response: str = ""+ cost_usd: Optional[float] = None+ duration_ms: Optional[int] = None+ num_turns: Optional[int] = None+ tool_calls: int = 0+ canonical_usage: CanonicalUsagePydantic = Field(+ default_factory=CanonicalUsagePydantic,+ )+ provider_data: Dict[str, Any] = Field(default_factory=dict)+ success: bool = False+ error: Optional[str] = None+++class BatchSummary(BaseModel):+ """Aggregated batch summary."""+ n_tasks: int = 0+ n_succeeded: int = 0+ n_failed: int = 0+ total_cost_usd: Optional[float] = None+ wall_clock_ms: int = 0+ budget_remaining_usd: Optional[float] = None+++class BatchResultModel(BaseModel):+ """Top-level batch result returned by `run_batch()`."""+ tasks: List[SessionResultModel] = Field(default_factory=list)+ summary: BatchSummary = Field(default_factory=BatchSummary)+ provider: str = ""+ timestamp: str = ""+++# ---------------------------------------------------------------------------+# Helpers+# ---------------------------------------------------------------------------++def session_result_to_model(sr: Any) -> SessionResultModel:+ """Convert a `protocol.SessionResult` dataclass to its Pydantic mirror."""+ cu = CanonicalUsagePydantic(+ input_tokens=sr.canonical_usage.input_tokens,+ output_tokens=sr.canonical_usage.output_tokens,+ cache_read=sr.canonical_usage.cache_read,+ cache_write=sr.canonical_usage.cache_write,+ reasoning_tokens=sr.canonical_usage.reasoning_tokens,+ request_count=sr.canonical_usage.request_count,+ )+ return SessionResultModel(+ task_id=sr.task_id,+ session_id=sr.session_id,+ provider=sr.provider,+ prompt=sr.prompt,+ branch=sr.branch,+ worktree_path=sr.worktree_path,+ response=sr.response,+ cost_usd=sr.cost_usd,+ duration_ms=sr.duration_ms,+ num_turns=sr.num_turns,+ tool_calls=sr.tool_calls,+ canonical_usage=cu,+ provider_data=sr.provider_data,+ success=sr.success,+ error=sr.error,+ )diff --git a/server/services/workflow.py b/server/services/workflow.pyindex fcd6cd06..fecd6a22 100644--- a/server/services/workflow.py+++ b/server/services/workflow.py@@ -461,7 +461,25 @@ class WorkflowService:workflow_id: Specific workflow to cancel. If None, cancels first running deployment."""manager = self._get_deployment_manager()- return await manager.cancel(workflow_id)+ result = await manager.cancel(workflow_id)++ # Also cancel any in-flight CLI agent batches (claude_code_agent /+ # codex_agent) for this workflow. Best-effort — failure to cancel+ # CLI sessions must not block the deployment cancel.+ if workflow_id:+ try:+ from services.cli_agent.service import get_ai_cli_service+ cli_svc = get_ai_cli_service()+ cancelled = await cli_svc.cancel_workflow(workflow_id)+ if cancelled:+ logger.info(+ "[workflow] cancelled %d CLI agent session(s) for workflow %s",+ cancelled, workflow_id,+ )+ except Exception as exc:+ logger.debug("[workflow] CLI agent cancel: %s", exc)++ return resultdef get_deployment_status(self, workflow_id: Optional[str] = None) -> Dict[str, Any]:"""Get deployment status.diff --git a/server/tests/nodes/test_specialized_agents.py b/server/tests/nodes/test_specialized_agents.pyindex a1d5d7c1..a277e381 100644--- a/server/tests/nodes/test_specialized_agents.py+++ b/server/tests/nodes/test_specialized_agents.py@@ -463,73 +463,120 @@ class TestRLMAgent:class TestClaudeCodeAgent:- def _wire_claude_service(self, result_payload=None):- service = MagicMock(name="ClaudeCodeService")- service.execute = AsyncMock(- return_value=result_payload- or {- "result": "cli response",- "session_id": "sess-abc",- "usage": {"input_tokens": 10, "output_tokens": 5},- }+ """The claude_code_agent plugin now goes through `AICliService.run_batch`+ (multi-task batch). Single-prompt input gets adapted to a one-task batch+ for back-compat. These tests mock the new service path."""++ def _wire_cli_service(+ self,+ *,+ response: str = "cli response",+ session_id: str = "sess-abc",+ cost_usd: float = 0.012,+ success: bool = True,+ error: str | None = None,+ ):+ """Mock `AICliService` with a `run_batch` that returns a one-task BatchResult."""+ from services.cli_agent.protocol import (+ BatchResult,+ CanonicalUsage,+ SessionResult,)- return service- async def test_happy_path_spawns_cli_and_returns_response(self, harness):- service = self._wire_claude_service()+ async def fake_run_batch(provider, *, tasks, **kwargs):+ tasks_list = list(tasks)+ results = []+ for t in tasks_list:+ results.append(SessionResult(+ task_id=t.task_id or "t_test",+ session_id=session_id,+ provider=provider,+ prompt=t.prompt,+ response=response,+ cost_usd=cost_usd,+ duration_ms=1234,+ num_turns=2,+ canonical_usage=CanonicalUsage(input_tokens=10, output_tokens=5),+ success=success,+ error=error,+ ))+ n_succeeded = sum(1 for r in results if r.success)+ return BatchResult(+ tasks=results,+ n_tasks=len(results),+ n_succeeded=n_succeeded,+ n_failed=len(results) - n_succeeded,+ total_cost_usd=cost_usd if all(r.success for r in results) else None,+ wall_clock_ms=1500,+ provider=provider,+ timestamp="2026-05-04T00:00:00Z",+ )++ svc = MagicMock(name="AICliService")+ svc.run_batch = AsyncMock(side_effect=fake_run_batch)+ svc.cancel_workflow = AsyncMock(return_value=0)+ svc.cancel_node = AsyncMock(return_value=0)+ return svc++ async def test_happy_path_routes_through_run_batch(self, harness):+ svc = self._wire_cli_service()with patched_container(auth_api_keys={}), patched_broadcaster(), patch(- "services.claude_code_service.get_claude_code_service",- return_value=service,+ "services.cli_agent.service.get_ai_cli_service",+ return_value=svc,):result = await harness.execute("claude_code_agent",{"prompt": "write a hello world script","model": "claude-sonnet-4-6",- "max_turns": 5,- "max_budget_usd": 2.0,},)harness.assert_envelope(result, success=True)payload = result["result"]assert payload["response"] == "cli response"- assert payload["provider"] == "anthropic"assert payload["session_id"] == "sess-abc"- assert payload["model"] == "claude-sonnet-4-6"-- # The service was called with the documented kwargs.- call = service.execute.await_args- assert call.kwargs["model"] == "claude-sonnet-4-6"- assert call.kwargs["max_turns"] == 5- assert call.kwargs["max_budget_usd"] == 2.0- assert call.kwargs["prompt"] == "write a hello world script"-- async def test_max_budget_usd_flag_is_passed_through(self, harness):- """Guards against the historical '--max-cost' bug: the handler must- forward maxBudgetUsd to the service as max_budget_usd."""- service = self._wire_claude_service()+ assert payload["provider"] == "claude"+ assert payload["n_tasks"] == 1+ assert payload["n_succeeded"] == 1++ # AICliService.run_batch was called with provider="claude" + 1 task+ call = svc.run_batch.await_args+ assert call.args[0] == "claude"+ tasks = list(call.kwargs["tasks"])+ assert len(tasks) == 1+ assert tasks[0].prompt == "write a hello world script"+ assert tasks[0].model == "claude-sonnet-4-6"++ async def test_max_budget_usd_propagates_via_task_spec(self, harness):+ """Per-task max_budget_usd must reach the ClaudeTaskSpec."""+ svc = self._wire_cli_service()with patched_container(auth_api_keys={}), patched_broadcaster(), patch(- "services.claude_code_service.get_claude_code_service",- return_value=service,+ "services.cli_agent.service.get_ai_cli_service",+ return_value=svc,):await harness.execute("claude_code_agent",- {"prompt": "x", "max_budget_usd": 7.5},+ {+ "tasks": [+ {"prompt": "x", "provider": "claude", "max_budget_usd": 7.5},+ ],+ },)- call = service.execute.await_args- assert call.kwargs["max_budget_usd"] == 7.5+ call = svc.run_batch.await_args+ tasks = list(call.kwargs["tasks"])+ assert tasks[0].max_budget_usd == 7.5async def test_no_prompt_returns_failure(self, harness):- """Unique to claude_code_agent: explicit no-prompt short-circuit."""- service = self._wire_claude_service()+ """No prompt + no tasks must short-circuit before constructing a batch."""+ svc = self._wire_cli_service()with patched_container(auth_api_keys={}), patched_broadcaster(), patch(- "services.claude_code_service.get_claude_code_service",- return_value=service,+ "services.cli_agent.service.get_ai_cli_service",+ return_value=svc,):result = await harness.execute("claude_code_agent",@@ -538,18 +585,17 @@ class TestClaudeCodeAgent:harness.assert_envelope(result, success=False)assert "prompt" in result["error"].lower()- # CLI must not have been spawned.- assert service.execute.await_count == 0+ # AICliService must not have been engaged.+ assert svc.run_batch.await_count == 0- async def test_subprocess_failure_becomes_envelope(self, harness):- """When ClaudeCodeService.execute raises, the handler returns- success=false with the error message."""- service = MagicMock(name="ClaudeCodeService")- service.execute = AsyncMock(side_effect=RuntimeError("cli exit 1: boom"))+ async def test_run_batch_failure_becomes_envelope(self, harness):+ """When AICliService.run_batch raises, the handler surfaces the error."""+ svc = MagicMock(name="AICliService")+ svc.run_batch = AsyncMock(side_effect=RuntimeError("cli exit 1: boom"))with patched_container(auth_api_keys={}), patched_broadcaster(), patch(- "services.claude_code_service.get_claude_code_service",- return_value=service,+ "services.cli_agent.service.get_ai_cli_service",+ return_value=svc,):result = await harness.execute("claude_code_agent",@@ -560,7 +606,7 @@ class TestClaudeCodeAgent:assert "boom" in result["error"]async def test_auto_prompt_fallback_from_input_main(self, harness):- service = self._wire_claude_service()+ svc = self._wire_cli_service()nodes = [{"id": "src-1", "type": "chatTrigger"},@@ -576,8 +622,8 @@ class TestClaudeCodeAgent:)with patched_container(auth_api_keys={}), patched_broadcaster(), patch(- "services.claude_code_service.get_claude_code_service",- return_value=service,+ "services.cli_agent.service.get_ai_cli_service",+ return_value=svc,):await harness.execute("claude_code_agent",@@ -586,5 +632,6 @@ class TestClaudeCodeAgent:context=ctx,)- call = service.execute.await_args- assert call.kwargs["prompt"] == "upstream text"+ call = svc.run_batch.await_args+ tasks = list(call.kwargs["tasks"])+ assert tasks[0].prompt == "upstream text"diff --git a/server/tests/services/cli_agent/__init__.py b/server/tests/services/cli_agent/__init__.pynew file mode 100644index 00000000..e69de29bdiff --git a/server/tests/services/cli_agent/test_mcp_server.py b/server/tests/services/cli_agent/test_mcp_server.pynew file mode 100644index 00000000..80222889--- /dev/null+++ b/server/tests/services/cli_agent/test_mcp_server.py@@ -0,0 +1,231 @@+"""MCP server integration tests.++Covers:+ - Bearer-token registry: register/lookup/unregister+ - 401 on missing / malformed / wrong / expired tokens+ - Per-batch scoping: tools see the right `BatchContext`+ - Lockfile format (VSCode-shape) + stale-PID sweep+"""++from __future__ import annotations++import json+import os+import tempfile+from pathlib import Path++import pytest+from httpx import ASGITransport, AsyncClient++from services.cli_agent.lockfile import (+ list_active_lockfiles,+ lockfile_path,+ remove_ide_lockfile,+ sweep_stale_lockfiles,+ write_ide_lockfile,+)+from services.cli_agent.mcp_server import (+ BatchContext,+ _reset_for_tests,+ active_batch_count,+ get_mcp_app,+ issue_token,+ lookup_batch,+ register_batch,+ unregister_batch,+)+++# ---------------------------------------------------------------------------+# Token registry+# ---------------------------------------------------------------------------++class TestTokenRegistry:+ def setup_method(self):+ _reset_for_tests()++ def test_issue_token_is_random_64_hex(self):+ a = issue_token()+ b = issue_token()+ assert a != b+ assert len(a) == 64+ int(a, 16) # parses as hex++ def test_register_lookup_unregister(self):+ token = issue_token()+ ctx = BatchContext(+ workflow_id="wf", node_id="n",+ workspace_dir=Path("."),+ )+ assert lookup_batch(token) is None+ register_batch(token, ctx)+ assert lookup_batch(token) is ctx+ assert active_batch_count() == 1+ unregister_batch(token)+ assert lookup_batch(token) is None+ assert active_batch_count() == 0++ def test_unregister_idempotent(self):+ unregister_batch("nonexistent") # should not raise++ def test_token_collision_with_different_ctx_raises(self):+ token = issue_token()+ ctx1 = BatchContext(workflow_id="a", node_id="n", workspace_dir=Path("."))+ ctx2 = BatchContext(workflow_id="b", node_id="n", workspace_dir=Path("."))+ register_batch(token, ctx1)+ with pytest.raises(ValueError, match="collision"):+ register_batch(token, ctx2)+++# ---------------------------------------------------------------------------+# ASGI auth middleware+# ---------------------------------------------------------------------------++class TestAuthMiddleware:+ def setup_method(self):+ _reset_for_tests()++ @pytest.mark.asyncio+ async def test_no_authorization_header_returns_401(self):+ app = get_mcp_app()+ async with AsyncClient(transport=ASGITransport(app=app), base_url="http://t") as c:+ r = await c.post(+ "/mcp/",+ json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"},+ )+ assert r.status_code == 401++ @pytest.mark.asyncio+ async def test_malformed_authorization_returns_401(self):+ app = get_mcp_app()+ async with AsyncClient(transport=ASGITransport(app=app), base_url="http://t") as c:+ r = await c.post(+ "/mcp/",+ json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"},+ headers={"Authorization": "Token abc"},+ )+ assert r.status_code == 401++ @pytest.mark.asyncio+ async def test_unknown_token_returns_401(self):+ app = get_mcp_app()+ async with AsyncClient(transport=ASGITransport(app=app), base_url="http://t") as c:+ r = await c.post(+ "/mcp/",+ json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"},+ headers={"Authorization": "Bearer unknown_token"},+ )+ assert r.status_code == 401++ @pytest.mark.asyncio+ async def test_expired_token_returns_401(self):+ """Token registered then unregistered behaves as unknown."""+ token = issue_token()+ register_batch(token, BatchContext(+ workflow_id="wf", node_id="n", workspace_dir=Path("."),+ ))+ unregister_batch(token)+ app = get_mcp_app()+ async with AsyncClient(transport=ASGITransport(app=app), base_url="http://t") as c:+ r = await c.post(+ "/mcp/",+ json={"jsonrpc": "2.0", "id": 1, "method": "tools/list"},+ headers={"Authorization": f"Bearer {token}"},+ )+ assert r.status_code == 401+++# ---------------------------------------------------------------------------+# Lockfile format+# ---------------------------------------------------------------------------++class TestLockfile:+ def test_claude_lockfile_path_pid_lock(self, tmp_path):+ path = lockfile_path(+ ide_lockfile_dir=tmp_path,+ pid=12345,+ port=3010,+ ide_name="claude",+ )+ assert path.name == "12345.lock"++ def test_gemini_lockfile_path_includes_port(self, tmp_path):+ path = lockfile_path(+ ide_lockfile_dir=tmp_path,+ pid=12345,+ port=3010,+ ide_name="gemini",+ )+ assert "12345" in path.name+ assert "3010" in path.name+ assert path.name.endswith(".json")++ def test_lockfile_payload_matches_vscode_shape(self, tmp_path):+ path = write_ide_lockfile(+ ide_lockfile_dir=tmp_path,+ pid=99999,+ port=3010,+ token="abc123",+ workspace_dir=tmp_path / "ws",+ ide_name="claude",+ )+ payload = json.loads(path.read_text(encoding="utf-8"))+ # VSCode-style fields+ assert payload["port"] == 3010+ assert payload["authToken"] == "abc123"+ assert payload["ideName"] == "claude"+ assert "url" in payload+ assert "workspaceFolders" in payload+ assert payload["transport"] == "http"+ # Our extra: pid for the stale-sweep+ assert payload["pid"] == 99999++ def test_default_url_constructed(self, tmp_path):+ path = write_ide_lockfile(+ ide_lockfile_dir=tmp_path,+ pid=1, port=3010, token="t",+ workspace_dir=tmp_path, ide_name="claude",+ )+ payload = json.loads(path.read_text(encoding="utf-8"))+ assert payload["url"] == "http://127.0.0.1:3010/mcp/ide"++ def test_remove_lockfile_safe_when_missing(self, tmp_path):+ # Should never raise+ remove_ide_lockfile(None)+ remove_ide_lockfile(tmp_path / "does-not-exist.lock")++ def test_sweep_removes_dead_pid_lockfile(self, tmp_path):+ # Find a guaranteed-dead PID by walking up high. PID 0 is the+ # system idle process on Windows and the swapper on POSIX, so+ # `psutil.pid_exists(0)` returns True everywhere — DON'T use 0.+ import psutil+ dead_pid = 99_999_999+ while psutil.pid_exists(dead_pid):+ dead_pid += 1+ if dead_pid > 999_999_999: # paranoid bound+ pytest.skip("could not find a dead PID")++ write_ide_lockfile(+ ide_lockfile_dir=tmp_path, pid=dead_pid, port=3010, token="t",+ workspace_dir=tmp_path, ide_name="claude",+ )+ # Write one with the live PID+ live_path = write_ide_lockfile(+ ide_lockfile_dir=tmp_path, pid=os.getpid(), port=3010, token="t",+ workspace_dir=tmp_path, ide_name="claude",+ )++ before = list_active_lockfiles(tmp_path)+ assert len(before) >= 2++ n = sweep_stale_lockfiles(tmp_path)+ # Should have removed at least the dead-PID one+ assert n >= 1++ after = list_active_lockfiles(tmp_path)+ # Live one survives+ assert live_path in after++ def test_sweep_safe_on_nonexistent_dir(self, tmp_path):+ assert sweep_stale_lockfiles(tmp_path / "no") == 0+ assert sweep_stale_lockfiles(None) == 0diff --git a/server/tests/services/cli_agent/test_providers.py b/server/tests/services/cli_agent/test_providers.pynew file mode 100644index 00000000..513fc44b--- /dev/null+++ b/server/tests/services/cli_agent/test_providers.py@@ -0,0 +1,358 @@+"""Unit tests for `AnthropicClaudeProvider` and `OpenAICodexProvider`.++Covers:+ - `headless_argv` shape (every flag in the right place, defaults+ applied, optional fields omitted when unset)+ - `parse_event` round-trips JSON correctly, returns None for garbage+ - `is_final_event` matches the right event types+ - `event_to_session_result` reconstructs cost / session_id /+ canonical_usage / response from vendored NDJSON+ - `detect_auth_error` matches "not logged in" stderr patterns+ - `supports()` flags align with `ai_cli_providers.json`+ - Factory raises NotImplementedError for gemini+"""++from __future__ import annotations++import json+from pathlib import Path++import pytest++from services.cli_agent import (+ ClaudeTaskSpec,+ CodexTaskSpec,+ create_cli_provider,+)+from services.cli_agent.factory import is_supported+from services.cli_agent.protocol import AICliProvider, CanonicalUsage+++# ---------------------------------------------------------------------------+# Factory contract+# ---------------------------------------------------------------------------++class TestFactory:+ def test_claude_creates_provider(self):+ p = create_cli_provider("claude")+ assert p.name == "claude"+ assert isinstance(p, AICliProvider)++ def test_codex_creates_provider(self):+ p = create_cli_provider("codex")+ assert p.name == "codex"+ assert isinstance(p, AICliProvider)++ def test_gemini_raises_not_implemented(self):+ with pytest.raises(NotImplementedError, match="deferred to v2"):+ create_cli_provider("gemini")++ def test_unknown_raises_value_error(self):+ with pytest.raises(ValueError, match="Unknown CLI provider"):+ create_cli_provider("openai")++ def test_is_supported(self):+ assert is_supported("claude") is True+ assert is_supported("codex") is True+ assert is_supported("gemini") is False+ assert is_supported("nope") is False+++# ---------------------------------------------------------------------------+# Claude provider+# ---------------------------------------------------------------------------++def claude_provider():+ return create_cli_provider("claude")+++class TestClaudeArgv:+ def test_minimum_required_flags(self, claude_provider):+ task = ClaudeTaskSpec(prompt="hello world")+ argv = claude_provider.headless_argv(task, defaults={})+ # `-p hello world` must be present+ assert "-p" in argv+ assert "hello world" in argv+ # stream-json output format+ assert "--output-format" in argv+ assert "stream-json" in argv+ # Defaults applied+ assert "--model" in argv+ assert "claude-sonnet-4-6" in argv # default_model from JSON+ assert "--max-turns" in argv+ assert "10" in argv # default_max_turns+ assert "--max-budget-usd" in argv+ assert "5.0" in argv # default_max_budget_usd+ assert "--allowedTools" in argv+ assert "--permission-mode" in argv++ def test_session_id_flag(self, claude_provider):+ task = ClaudeTaskSpec(prompt="x", session_id="sess-abc-123")+ argv = claude_provider.headless_argv(task, defaults={})+ assert "--session-id" in argv+ assert "sess-abc-123" in argv+ assert "--resume" not in argv++ def test_resume_wins_over_session_id(self, claude_provider):+ task = ClaudeTaskSpec(+ prompt="x",+ session_id="new-sess",+ resume_session_id="prior-sess",+ )+ argv = claude_provider.headless_argv(task, defaults={})+ assert "--resume" in argv+ assert "prior-sess" in argv+ assert "--session-id" not in argv++ def test_zero_budget_omits_flag(self, claude_provider):+ task = ClaudeTaskSpec(prompt="x", max_budget_usd=0.0)+ argv = claude_provider.headless_argv(task, defaults={})+ assert "--max-budget-usd" not in argv++ def test_system_prompt_propagates(self, claude_provider):+ task = ClaudeTaskSpec(prompt="x", system_prompt="be concise")+ argv = claude_provider.headless_argv(task, defaults={})+ assert "--append-system-prompt" in argv+ assert "be concise" in argv++ def test_wrong_task_type_raises(self, claude_provider):+ codex_task = CodexTaskSpec(prompt="x")+ with pytest.raises(TypeError, match="ClaudeTaskSpec"):+ claude_provider.headless_argv(codex_task, defaults={})+++class TestClaudeParseEvent:+ def test_valid_json_round_trips(self, claude_provider):+ line = json.dumps({"type": "result", "result": "42", "session_id": "abc"})+ event = claude_provider.parse_event(line)+ assert event is not None+ assert event["type"] == "result"+ assert event["result"] == "42"++ def test_garbage_returns_none(self, claude_provider):+ assert claude_provider.parse_event("not json") is None+ assert claude_provider.parse_event("") is None+ assert claude_provider.parse_event(" ") is None++ def test_is_final_event(self, claude_provider):+ assert claude_provider.is_final_event({"type": "result"}) is True+ assert claude_provider.is_final_event({"type": "assistant"}) is False+ assert claude_provider.is_final_event({"type": "system"}) is False+++class TestClaudeEventToSessionResult:+ """Vendored Claude stream-json fixture verified end-to-end."""++ @pytest.fixture+ def fixture_events(self):+ return [+ {"type": "system", "subtype": "init", "session_id": "sess-1"},+ {+ "type": "assistant",+ "message": {+ "content": [+ {"type": "text", "text": "Sure, here's..."},+ {"type": "tool_use", "id": "t1", "name": "Read", "input": {}},+ ],+ },+ "session_id": "sess-1",+ },+ {+ "type": "tool_use",+ "tool_name": "Edit",+ "tool_input": {"file_path": "x.py"},+ },+ {+ "type": "result",+ "subtype": "success",+ "result": "Done — refactored to async.",+ "total_cost_usd": 0.4231,+ "duration_ms": 18234,+ "num_turns": 7,+ "session_id": "sess-1",+ "usage": {+ "input_tokens": 12000,+ "output_tokens": 3500,+ "cache_creation_input_tokens": 500,+ "cache_read_input_tokens": 8000,+ },+ },+ ]++ def test_reconstructs_response(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["response"] == "Done — refactored to async."++ def test_reconstructs_cost(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["cost_usd"] == pytest.approx(0.4231)++ def test_reconstructs_session_id(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["session_id"] == "sess-1"++ def test_reconstructs_duration_and_turns(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["duration_ms"] == 18234+ assert result["num_turns"] == 7++ def test_canonical_usage_normalises(self, claude_provider, fixture_events):+ cu: CanonicalUsage = claude_provider.canonical_usage(fixture_events)+ assert cu.input_tokens == 12000+ assert cu.output_tokens == 3500+ assert cu.cache_read == 8000 # remapped from cache_read_input_tokens+ assert cu.cache_write == 500 # remapped from cache_creation_input_tokens+ assert cu.request_count == 7 # from num_turns++ def test_counts_tool_calls(self, claude_provider, fixture_events):+ # Two tool_use events: one inside an assistant message, one standalone+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["tool_calls"] >= 2++ def test_success_on_zero_exit_with_result_event(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(fixture_events, "", 0)+ assert result["success"] is True+ assert result["error"] is None++ def test_failure_on_non_zero_exit(self, claude_provider, fixture_events):+ result = claude_provider.event_to_session_result(+ fixture_events, "exploded", 1,+ )+ assert result["success"] is False+ assert "exploded" in (result["error"] or "")++ def test_failure_on_missing_result_event(self, claude_provider):+ events = [+ {"type": "system", "subtype": "init", "session_id": "s"},+ {"type": "assistant", "message": {}, "session_id": "s"},+ ]+ result = claude_provider.event_to_session_result(events, "", 0)+ assert result["success"] is False+ assert "no result event" in (result["error"] or "")+++class TestClaudeAuthDetection:+ def test_logged_out_marker(self, claude_provider):+ assert claude_provider.detect_auth_error(+ "Please run 'claude login' first.", 1,+ ) is True++ def test_clean_run_not_auth_error(self, claude_provider):+ assert claude_provider.detect_auth_error("", 0) is False++ def test_unrelated_stderr_not_auth_error(self, claude_provider):+ assert claude_provider.detect_auth_error(+ "git: pathspec 'x' did not match any files\n", 1,+ ) is False+++class TestClaudeSupports:+ def test_supports_full_feature_set(self, claude_provider):+ for feature in (+ "max_budget", "max_turns", "session_id", "resume",+ "mcp_runtime", "json_cost", "ide_lockfile",+ ):+ assert claude_provider.supports(feature), feature++ def test_does_not_support_sandbox(self, claude_provider):+ assert claude_provider.supports("sandbox") is False+++# ---------------------------------------------------------------------------+# Codex provider+# ---------------------------------------------------------------------------++def codex_provider():+ return create_cli_provider("codex")+++class TestCodexArgv:+ def test_no_session_no_budget_no_turns(self, codex_provider):+ task = CodexTaskSpec(prompt="hello")+ argv = codex_provider.headless_argv(task, defaults={})+ assert "--max-turns" not in argv+ assert "--max-budget-usd" not in argv+ assert "--session-id" not in argv+ assert "--resume" not in argv+ assert "--allowedTools" not in argv++ def test_sandbox_flag(self, codex_provider):+ task = CodexTaskSpec(prompt="x", sandbox="read-only")+ argv = codex_provider.headless_argv(task, defaults={})+ assert "--sandbox" in argv+ assert "read-only" in argv++ def test_ask_for_approval_flag(self, codex_provider):+ task = CodexTaskSpec(prompt="x", ask_for_approval="on-request")+ argv = codex_provider.headless_argv(task, defaults={})+ assert "--ask-for-approval" in argv+ assert "on-request" in argv++ def test_default_sandbox_workspace_write(self, codex_provider):+ task = CodexTaskSpec(prompt="x")+ argv = codex_provider.headless_argv(task, defaults={})+ idx = argv.index("--sandbox")+ assert argv[idx + 1] == "workspace-write"++ def test_system_prompt_prepended(self, codex_provider):+ task = CodexTaskSpec(prompt="user thing", system_prompt="be careful")+ argv = codex_provider.headless_argv(task, defaults={})+ # Codex has no --system-prompt flag; we prepend with <system> tags+ prompt_arg = argv[-1] # last arg is the prompt+ assert "<system>" in prompt_arg+ assert "be careful" in prompt_arg+ assert "user thing" in prompt_arg++ def test_wrong_task_type_raises(self, codex_provider):+ claude_task = ClaudeTaskSpec(prompt="x")+ with pytest.raises(TypeError, match="CodexTaskSpec"):+ codex_provider.headless_argv(claude_task, defaults={})+++class TestCodexEventReconstruction:+ @pytest.fixture+ def fixture_events(self):+ # Codex has no public stream-json schema; this is a plausible+ # synthetic stream that exercises our best-effort matchers.+ return [+ {"type": "message", "text": "Working on it..."},+ {"type": "assistant", "text": "Final answer is X."},+ {"type": "complete", "duration_ms": 5000, "stats": {"turns": 3}},+ ]++ def test_extracts_response_from_final_event(self, codex_provider, fixture_events):+ result = codex_provider.event_to_session_result(fixture_events, "", 0)+ # Response comes from final event's `text`/`response`/`result`/`content`+ # if present; otherwise the last assistant message.+ assert "Final answer is X." in result["response"]++ def test_cost_always_none_for_codex(self, codex_provider, fixture_events):+ result = codex_provider.event_to_session_result(fixture_events, "", 0)+ assert result["cost_usd"] is None++ def test_canonical_usage_zeros(self, codex_provider, fixture_events):+ cu = codex_provider.canonical_usage(fixture_events)+ assert cu.input_tokens == 0+ assert cu.output_tokens == 0+++class TestCodexAuthDetection:+ def test_openai_api_key_missing(self, codex_provider):+ assert codex_provider.detect_auth_error(+ "Error: OPENAI_API_KEY not set.\n", 1,+ ) is True++ def test_401_marker(self, codex_provider):+ assert codex_provider.detect_auth_error("HTTP 401 Unauthorized\n", 1) is True+++class TestCodexSupports:+ def test_supports_only_sandbox(self, codex_provider):+ assert codex_provider.supports("sandbox") is True+ for feature in (+ "max_budget", "max_turns", "session_id", "resume",+ "json_cost", "ide_lockfile",+ ):+ assert codex_provider.supports(feature) is False, featurediff --git a/server/tests/services/cli_agent/test_service.py b/server/tests/services/cli_agent/test_service.pynew file mode 100644index 00000000..fa8a1f68--- /dev/null+++ b/server/tests/services/cli_agent/test_service.py@@ -0,0 +1,143 @@+"""`AICliService` tests — fail-fast paths that don't spawn the CLI.++The full subprocess-driven path is covered by live verification (see+`docs-internal/cli_agent_framework.md` → Verification §5–7). These unit+tests cover:+ - `working_directory_not_git_repo` abort (no pool constructed)+ - resolver contract: explicit `repo_root` doesn't fall back to cwd+ - factory NotImplementedError surfaces cleanly+ - cancel_workflow / cancel_node return zero when nothing's running+"""++from __future__ import annotations++import asyncio+import tempfile+from pathlib import Path++import pytest++from services.cli_agent import ClaudeTaskSpec, CodexTaskSpec+from services.cli_agent.service import AICliService, get_ai_cli_service+++async def test_not_git_repo_returns_structured_failure():+ """Caller-supplied repo_root that isn't a git repo → fail-fast,+ every task surfaces `working_directory_not_git_repo`."""+ svc = AICliService()+ with tempfile.TemporaryDirectory() as tmp:+ result = await svc.run_batch(+ "claude",+ tasks=[+ ClaudeTaskSpec(prompt="task A"),+ ClaudeTaskSpec(prompt="task B"),+ ],+ node_id="n",+ workflow_id="wf",+ workspace_dir=Path(tmp),+ broadcaster=None,+ repo_root=Path(tmp), # explicit non-git+ )++ assert result.n_tasks == 2+ assert result.n_succeeded == 0+ assert result.n_failed == 2+ assert all(t.error == "working_directory_not_git_repo" for t in result.tasks)+ assert result.total_cost_usd is None+++async def test_explicit_repo_root_does_not_fallback_to_cwd():+ """When the caller passes an explicit `repo_root`, the resolver+ must NOT walk to cwd. This is the bug we fixed during Phase 5+ smoke-testing."""+ # cwd is the framework worktree (a git repo). If the resolver fell+ # back, it would silently succeed — bad.+ svc = AICliService()+ with tempfile.TemporaryDirectory() as tmp:+ result = await svc.run_batch(+ "claude",+ tasks=[ClaudeTaskSpec(prompt="x")],+ node_id="n",+ workflow_id="wf",+ workspace_dir=Path(tmp),+ broadcaster=None,+ repo_root=Path(tmp),+ )+ assert result.n_failed == 1+ assert result.tasks[0].error == "working_directory_not_git_repo"+++async def test_codex_provider_works():+ """Codex factory must build a provider; happy-path argv was already+ covered in test_providers.py."""+ svc = AICliService()+ with tempfile.TemporaryDirectory() as tmp:+ # Same not-git-repo path with Codex — confirms the provider+ # discrim works through the service.+ result = await svc.run_batch(+ "codex",+ tasks=[CodexTaskSpec(prompt="x", sandbox="read-only")],+ node_id="n",+ workflow_id="wf",+ workspace_dir=Path(tmp),+ broadcaster=None,+ repo_root=Path(tmp),+ )+ assert result.provider == "codex"+ assert result.tasks[0].provider == "codex"+++async def test_gemini_factory_raises_not_implemented():+ svc = AICliService()+ with tempfile.TemporaryDirectory() as tmp:+ with pytest.raises(NotImplementedError, match="deferred to v2"):+ await svc.run_batch(+ "gemini",+ tasks=[], # spec irrelevant — factory raises before construction+ node_id="n",+ workflow_id="wf",+ workspace_dir=Path(tmp),+ broadcaster=None,+ repo_root=Path(tmp),+ )+++async def test_cancel_when_no_active_pools():+ svc = AICliService()+ assert await svc.cancel_workflow("nothing") == 0+ assert await svc.cancel_node("nothing") == 0+++def test_singleton_accessor_returns_same_instance():+ a = get_ai_cli_service()+ b = get_ai_cli_service()+ assert a is b+++async def test_resolver_walks_upward_to_find_git():+ """Without `override`, the resolver tries workspace_dir then cwd.++ The cli-agent-framework worktree is a git repo. A deep child path+ should resolve via `git rev-parse --show-toplevel` to the worktree root.+ """+ deep = Path(__file__).resolve().parent / "deep" / "deeper"+ root = await AICliService._resolve_repo_root(workspace_dir=deep, override=None)+ assert root is not None+ assert (root / ".git").exists()+++async def test_resolver_returns_none_when_override_not_git():+ import tempfile+ with tempfile.TemporaryDirectory() as tmp:+ root = await AICliService._resolve_repo_root(+ workspace_dir=Path(tmp), # ignored when override is set+ override=Path(tmp),+ )+ assert root is None