Skip to content

Self-improving AI Employees for everyone

refactor(paths): centralise on-disk layout under ~/.machina/

trohitg915d41284f2b
diff --git a/.env.template b/.env.template
index 4f73f26d..a22bf3f9 100644
--- a/.env.template
+++ b/.env.template
@@ -49,8 +49,21 @@ TEMPORAL_PER_TYPE_DISPATCH=true
TEMPORAL_AGENT_WORKFLOW_ENABLED=true
TEMPORAL_BACKEND=postgres
-# Data directory (base for all persistent storage: DBs, workspaces, logs)
-DATA_DIR=data
+# Data directory (base for all persistent storage: DBs, workspaces,
+# workflows, claude state). User-home rooted by default — same
+# convention claude code itself uses (`~/.claude/`), Stripe
+# (`~/.config/stripe/`), ngrok (`~/.ngrok2/`). Survives `rm -rf` of
+# the repo and lets multiple checkouts share state.
+#
+# Resolution rules (`server/core/paths.py::machina_root`):
+# - `~/...` → user home (cross-platform via Path.expanduser)
+# - absolute → used verbatim
+# - relative → resolved under repo root (back-compat)
+#
+# Set to `.machina` if you want pre-cutover repo-local behaviour.
+# MachinaOs auto-migrates the legacy `data/` + `workflows/` layout
+# on first boot (`migrate_legacy_layout`).
+DATA_DIR=~/.machina
# Per-workflow workspace (nodes save files here, resolved under DATA_DIR)
WORKSPACE_BASE_DIR=workspaces
diff --git a/.gitignore b/.gitignore
index 1d2f9320..183e67fb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -129,14 +129,29 @@ pnpm-store/
# =============================================================================
# Claude Code's local config / session state / scheduled tasks.
.claude/
-# Per-workflow runtime workspace (CLAUDE.md "Per-Workflow Workspace
-# Directory" — `data/workspaces/<workflow_id>/`). User-uploaded files,
-# downloaded artifacts, agent-shell state. Always machine-local.
+# MachinaOs runtime root (`server/core/paths.py::machina_root`). Holds
+# claude/, workspaces/, workflows/, whatsapp/, credentials.db,
+# machina.db — every machine-local artefact. Override path via
+# `DATA_DIR` env var in `.env`. Auto-migrated from the legacy `data/`
+# + `workflows/` layout on first boot.
+/.machina/
+# Legacy data dir (pre-cutover layout). Auto-migrated into `.machina/`
+# on first boot but the empty husk may linger if rmdir failed; keep
+# ignored so partial state can't accidentally land in commits.
/data/
+# Legacy workflows dir (pre-cutover layout). Auto-migrated into
+# `.machina/workflows/` on first boot.
+/workflows/
# Patch archive left behind by `git format-patch` during one-off
# merges (see ../.merge-archive cleanup commits). Never committed.
.merge-archive/
+# Test side-effect dir from conftest's core.paths stub
+# (server/core/paths.py). conftest points the stub's machina_root
+# at this dir so plugin module imports don't touch the real user
+# home during tests. Re-created on each test run.
+server/tests/_test_machina_root/
+
# =============================================================================
# Documentation (docs-MachinaOs is a separate GitHub project)
# =============================================================================
diff --git a/CLAUDE.md b/CLAUDE.md
index e47a80b6..32c63fce 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -1470,7 +1470,7 @@ Residential proxy provider management with geo-targeting, session control, and a
- **typescriptExecutor**: **Dual-purpose node** - Execute TypeScript code via persistent Node.js server with type safety, syntax-highlighted editor and console output. Works as workflow node OR AI Agent tool (`typescript_code`).
### Filesystem & Shell Nodes (4 nodes)
-Dual-purpose tool nodes wrapping `deepagents.backends.LocalShellBackend`. Per-workflow workspace from execution context (`context["workspace_dir"]` = `data/workspaces/<workflow_id>/`). Fallback uses `Settings().workspace_base_dir` (never `os.getcwd()`).
+Dual-purpose tool nodes wrapping `deepagents.backends.LocalShellBackend`. Per-workflow workspace from execution context (`context["workspace_dir"]` = `~/.machina/workspaces/<workflow_id>/`). Fallback uses `Settings().workspace_base_dir` (never `os.getcwd()`).
**Path safety**: `nodes/filesystem/_backend.py` exposes `normalize_virtual_path()` which uses `pathlib.PureWindowsPath` (host-OS independent) to strip Windows drives, POSIX root, and UNC anchors uniformly, then delegates to `deepagents.backends.utils.validate_path` for `..`/`~` rejection. Wired into `file_read`, `file_modify`, and `fs_search` so LLM-emitted paths in any flavour (`C:\foo`, `/tmp/foo`, `\\server\share\x`, `foo\bar`) all map to a virtual path under the workspace. `virtual_mode=True` only sandboxes filesystem ops — `execute()` itself is never path-restricted (deepagents documents this).
@@ -2023,7 +2023,7 @@ Console logs are persisted to SQLite database and loaded on page refresh.
### Per-Workflow Workspace Directory
Each workflow execution gets a persistent workspace directory where nodes save output files and AI agents (especially Deep Agent) access them via filesystem tools.
-**Directory**: `data/workspaces/<workflow_id>/`
+**Directory**: `~/.machina/workspaces/<workflow_id>/`
**Configuration** (`server/core/config.py`):
```python
@@ -3486,7 +3486,7 @@ When `simpleMemory` is connected to `claude_code_agent`, memory continuity is ha
**Live UI refresh — CloudEvents v1.0 envelope.** `_persist_memory` calls `broadcaster.broadcast_node_parameters_updated(...)` after `save_node_parameters`. The broadcast carries a `WorkflowEvent` envelope (`type: "com.machinaos.node.parameters.updated"`, `subject: <memory_node_id>`, `data: {node_id, parameters, version, source: "cli"}`) wrapped in the wire-key `{type: "node_parameters_updated", data: <envelope>}`. The FE handler in `client/src/contexts/WebSocketContext.tsx` casts to `WorkflowEvent<{node_id, parameters, version, source}>`, reads `inner.node_id || envelope.subject`, and routes the inner payload to `setNodeParameters` + `queryClient.setQueryData` so the simpleMemory parameter panel auto-refreshes the moment the turn completes. Pre-cutover the FE read the old flat top-level keys (`message.parameters`, `message.node_id`) and silently dropped every broadcast — symptom was "memory only updates after a page reload."
-**Workspace dir routed via `--add-dir`.** The per-workflow workspace (`data/workspaces/<workflow_id>/`, injected into `ctx.raw` by `workflow.py:_get_workspace_dir`) is spliced into each task's `add_dir` list in `AICliService.run_batch` so the spawned claude can read files dropped by upstream nodes (`fileDownloader`, `documentParser`, code executors). Without this the workspace is invisible: memory-bound runs spawn with `cwd=repo_root` (stable for `--continue`'s `project_key`) and non-memory runs with `cwd=worktree`, neither of which sees the workspace files. Mirrors the ai_agent pattern (`services/ai.py:1899` — `config['workspace_dir'] = context.get('workspace_dir', '')`) but uses claude's native `--add-dir` instead of MCP-tool-config injection because claude has its own filesystem tools.
+**Workspace dir routed via `--add-dir`.** The per-workflow workspace (`~/.machina/workspaces/<workflow_id>/`, injected into `ctx.raw` by `workflow.py:_get_workspace_dir`) is spliced into each task's `add_dir` list in `AICliService.run_batch` so the spawned claude can read files dropped by upstream nodes (`fileDownloader`, `documentParser`, code executors). Without this the workspace is invisible: memory-bound runs spawn with `cwd=repo_root` (stable for `--continue`'s `project_key`) and non-memory runs with `cwd=worktree`, neither of which sees the workspace files. Mirrors the ai_agent pattern (`services/ai.py:1899` — `config['workspace_dir'] = context.get('workspace_dir', '')`) but uses claude's native `--add-dir` instead of MCP-tool-config injection because claude has its own filesystem tools.
**Stale-model coercion.** `ClaudeCodeAgentParams.model` is a `Literal` of supported model IDs + aliases — strict validation would reject legacy saved values like `"claude-sonnet-4.6"` (dot-spelled), `"claude-3-5-sonnet-20241022"` (date-suffixed), `""`. A `field_validator("model", "fallback_model", mode="before")` coerces unknown values to the default (`"claude-sonnet-4-6"` for `model`, `None` for `fallback_model`) so old workflows keep loading; the UI dropdown still constrains new edits.
diff --git a/docs-internal/claude_code_interactive_mode.md b/docs-internal/claude_code_interactive_mode.md
index 2c1ee62b..b61a53c0 100644
--- a/docs-internal/claude_code_interactive_mode.md
+++ b/docs-internal/claude_code_interactive_mode.md
@@ -311,7 +311,7 @@ was the wrong default).
[`nodes/agent/claude_code_agent/_skills.py::materialise_skills`](../server/nodes/agent/claude_code_agent/_skills.py)
writes one `SKILL.md` tree per connected-and-enabled skill under
`<workspace_dir>/.claude/skills/<name>/`. The workspace dir
-(`data/workspaces/<workflow_id>/`) is already passed via
+(`~/.machina/workspaces/<workflow_id>/`) is already passed via
`--add-dir`, and per the skills spec's
[Automatic discovery from parent and nested directories](https://code.claude.com/docs/en/skills#automatic-discovery-from-parent-and-nested-directories)
rule, claude scans `.claude/skills/` inside every `--add-dir` path.
@@ -361,7 +361,7 @@ skill-injection channel — workspace-dir + live-watch is canonical.
**Workspace dir — routed via `--add-dir`.**
[`AICliService.run_batch`](../server/services/cli_agent/service.py)
splices the per-workflow workspace
-(`data/workspaces/<workflow_id>/`, injected into `ctx.raw` by
+(`~/.machina/workspaces/<workflow_id>/`, injected into `ctx.raw` by
`workflow.py:_get_workspace_dir`) into each task's `add_dir` list
right after `task_list` is built — `interactive_argv` already emits
`--add-dir <path>` per entry. Runs BEFORE the pool/non-pool branch
diff --git a/docs-internal/cli_agent_framework.md b/docs-internal/cli_agent_framework.md
index 09b2bd3b..3dbfa56b 100644
--- a/docs-internal/cli_agent_framework.md
+++ b/docs-internal/cli_agent_framework.md
@@ -49,7 +49,7 @@ Reuses (do not duplicate):
- `services/skill_loader.py` — `scan_skills` / `load_skill` consumed by MCP `listSkills` / `getSkill`
- `services/auth.py` — `AuthService.get_api_key` consumed by MCP `getCredential`
- `services/credential_registry.py` — deep-merge `extends` for `_cli_base` entry
-- `nodes/agent/claude_code_agent/_oauth.py` — Claude `auth login` / `auth status` / `auth logout` wrappers, project-local npm install at `<repo>/data/claude-machina/npm/`, inherited stdio so the CLI opens the browser itself
+- `nodes/agent/claude_code_agent/_oauth.py` — Claude `auth login` / `auth status` / `auth logout` wrappers, project-local npm install at `~/.machina/claude/npm/`, inherited stdio so the CLI opens the browser itself
- `nodes/stripe/_handlers.py` — pattern reference for marker-token + catalogue broadcast
## Provider abstraction (mirrors `services/llm/`)
@@ -98,7 +98,7 @@ the interactive billing bucket (entrypoint `claude-vscode`, NOT
`sdk-cli`) since `-p` / `--print` is never emitted.
```
-<repo>/data/claude-machina/npm/node_modules/.bin/claude[.cmd]
+~/.machina/claude/npm/node_modules/.bin/claude[.cmd]
--output-format stream-json # events on stdout
--input-format stream-json # user turns to stdin as JSON
--verbose # required with stream-json for full event detail
@@ -219,7 +219,7 @@ Steps:
1. Run `claude auth status`. If it exits 0, write the marker + broadcast and return immediately (idempotent re-click).
2. Otherwise call `services.claude_oauth.initiate_claude_oauth()`:
- - Project-local install of `@anthropic-ai/claude-code` into `<repo>/data/claude-machina/npm/` via `npm install --prefix` (mirrors WhatsApp's `<repo>/node_modules/edgymeow/` layout; skipped if already installed).
+ - Project-local install of `@anthropic-ai/claude-code` into `~/.machina/claude/npm/` via `npm install --prefix` (mirrors WhatsApp's `<repo>/node_modules/edgymeow/` layout; skipped if already installed).
- `asyncio.create_subprocess_exec(claude, "auth", "login", env={..., CLAUDE_CONFIG_DIR=<repo>/data/claude-machina})` with **inherited stdio** — same way the VSCode Claude Code extension delegates to the binary. Anthropic doesn't expose `--print-url` or a programmatic OAuth helper (issue [anthropics/claude-code#7100](https://github.com/anthropics/claude-code/issues/7100), closed "not planned"), so we let the CLI open the user's browser via its own OS-level call. Returns `{success: True, pid}` immediately.
3. Schedule a background task that polls `claude auth status` every 2s up to 600s. On exit-0, write the synthetic `"cli-managed"` marker via `auth_service.store_oauth_tokens("claude_code", ...)` and broadcast `credential_catalogue_updated`. The catalogue's `stored` flag flips and the existing `OAuthConnect.tsx` primitive renders the modal as Connected.
@@ -370,7 +370,7 @@ key doesn't drift.
Claude derives `project_key` from cwd via `re.sub(r"[^a-zA-Z0-9.-]", "-", str(cwd))`
— every `:`, `\`, `/`, `_` becomes `-`. Verified against the on-disk
-`data/claude-machina/projects/` listing — Python reproduces three
+`~/.machina/claude/projects/` listing — Python reproduces three
encoded directory names byte-for-byte. The pre-bridge per-task
worktree (`<workspace>/<node_id>/wt_t_<random_8hex>`) changed cwd on
every spawn → fresh project_key every run → `--resume <UUID>` looked
@@ -548,7 +548,7 @@ Plugin contract: `tests/test_plugin_contract.py` + `tests/test_node_spec.py` —
Live verification (needs a real Claude install + auth):
-1. Empty `<repo>/data/claude-machina/`. Open Credentials Modal → click "Login with Claude Code CLI". Confirm the npm install runs (visible in backend logs), `<repo>/data/claude-machina/npm/node_modules/.bin/claude[.cmd]` appears, browser opens for Anthropic OAuth. Modal flips Connected within ~2s of CLI exit (background `claude auth status` poll detects success).
+1. Empty `~/.machina/claude/`. Open Credentials Modal → click "Login with Claude Code CLI". Confirm the npm install runs (visible in backend logs), `~/.machina/claude/npm/node_modules/.bin/claude[.cmd]` appears, browser opens for Anthropic OAuth. Modal flips Connected within ~2s of CLI exit (background `claude auth status` poll detects success).
2. Refresh the page. Modal stays Connected (`auth_service.get_oauth_tokens("claude_code")` still returns the marker; idempotent re-click also stays Connected).
3. Click Disconnect. Modal flips Disconnected (`claude auth logout` clears CLI creds + marker dropped).
4. Add a `claude_code_agent` node, set `tasks=[{prompt:"echo A"},{prompt:"echo B"},{prompt:"echo C"}]`, run. Three distinct `claude:<task_id>` Terminal streams interleaved. Three distinct session_ids. Three worktrees created and removed. `summary.wall_clock_ms < sum(duration_ms)` (proves parallelism).
diff --git a/server/core/config.py b/server/core/config.py
index 44324d40..90e17192 100644
--- a/server/core/config.py
+++ b/server/core/config.py
@@ -174,8 +174,16 @@ class Settings(BaseSettings):
api_key_encryption_key: str = Field(env="API_KEY_ENCRYPTION_KEY", min_length=32)
api_key_cache_ttl: int = Field(default=2592000, env="API_KEY_CACHE_TTL", ge=3600)
- # Data directory (base for all persistent storage: DBs, workspaces, logs)
- data_dir: str = Field(default="data", env="DATA_DIR")
+ # Data directory (base for all persistent storage: DBs, workspaces,
+ # workflows, claude state). Default is the user's home dir
+ # (``~/.machina``) — same convention claude code itself uses
+ # (``~/.claude/``), Stripe (``~/.config/stripe/``), ngrok
+ # (``~/.ngrok2/``). Survives ``rm -rf`` of the repo. Resolution
+ # rules (``~``, absolute, repo-relative) live in
+ # :func:`core.paths.machina_root`. One-time auto-migration of
+ # the pre-cutover ``data/`` + ``workflows/`` layout runs on
+ # startup via :func:`core.paths.migrate_legacy_layout`.
+ data_dir: str = Field(default="~/.machina", env="DATA_DIR")
# Credentials Database (separate encrypted database for API keys and OAuth tokens)
# Resolved relative to data_dir unless absolute
diff --git a/server/core/paths.py b/server/core/paths.py
new file mode 100644
index 00000000..0b37d43e
--- /dev/null
+++ b/server/core/paths.py
@@ -0,0 +1,274 @@
+"""Central path resolution for MachinaOs's on-disk state.
+
+Single source of truth for every directory the app reads or writes.
+The default root lives at ``~/.machina/`` — the user's home directory,
+cross-platform via :meth:`pathlib.Path.home` (``$HOME`` on POSIX,
+``%USERPROFILE%`` on Windows). Override with ``DATA_DIR`` env var.
+
+Why home-rooted (not project-local): claude's auth, the WhatsApp
+session DB, the credentials store, and per-workflow workspaces all
+survive ``rm -rf`` of the repo. Multiple MachinaOs checkouts on the
+same machine share state instead of each carrying its own copy. The
+``~/.machina/`` convention matches Stripe's ``~/.config/stripe/``,
+ngrok's ``~/.ngrok2/``, and claude code's own ``~/.claude/``.
+
+Layout (flat, no redundant nesting):
+
+ - ``~/.machina/claude/`` (was ``<repo>/data/claude-machina/``)
+ - ``~/.machina/workspaces/`` (was ``<repo>/data/workspaces/``)
+ - ``~/.machina/workflows/`` (was ``<repo>/workflows/``)
+ - ``~/.machina/whatsapp/`` (was ``<repo>/data/whatsapp/``)
+ - ``~/.machina/credentials.db``
+ - ``~/.machina/machina.db``
+
+Importable as ``from core.paths import claude_config_dir, workspace_dir, …``
+so consumers never have to recompute the root themselves (the old
+``Path(__file__).resolve().parents[N] / "data" / ...`` idiom was
+duplicated across 4+ files and brittle to file moves).
+
+DATA_DIR resolution rules (see :func:`machina_root`):
+
+ - Starts with ``~`` → ``Path.expanduser()`` (user home).
+ - Absolute path → used verbatim.
+ - Relative path → resolved under the repo root (back-compat for
+ callers who set ``DATA_DIR=data`` to keep the pre-cutover layout).
+
+Auto-migration of existing ``<repo>/data/`` and ``<repo>/workflows/``
+trees happens in :func:`migrate_legacy_layout` — see
+``server/main.py``'s startup event.
+"""
+
+from __future__ import annotations
+
+import shutil
+from pathlib import Path
+
+from core.logging import get_logger
+
+logger = get_logger(__name__)
+
+
+# Repo root: server/core/paths.py -> parents[2] is the project root.
+_REPO_ROOT = Path(__file__).resolve().parents[2]
+
+
+def project_root() -> Path:
+ """Absolute path of the MachinaOs git repo root."""
+ return _REPO_ROOT
+
+
+def machina_root() -> Path:
+ """Absolute path of ``~/.machina/`` (or the configured DATA_DIR).
+
+ Resolves :class:`core.config.Settings.data_dir` per the rules in
+ the module docstring. Default expands to ``~/.machina/``, i.e.
+ the user's home dir — same shape on Windows
+ (``%USERPROFILE%/.machina``), macOS, and Linux.
+
+ Settings is instantiated lazily so this module stays a leaf
+ dependency. The Settings class itself reads ``../.env`` at
+ construction; subsequent calls are cheap (Pydantic caches).
+ """
+ from core.config import Settings
+ settings = Settings()
+ raw = settings.data_dir
+ p = Path(raw)
+ # ``~/...`` shape: expand to the user's home dir cross-platform.
+ if raw.startswith("~"):
+ return p.expanduser().resolve()
+ # Absolute (POSIX or drive-rooted on Windows): use verbatim.
+ if p.is_absolute():
+ return p.resolve()
+ # Relative: keep the pre-cutover repo-local back-compat path.
+ return (project_root() / p).resolve()
+
+
+def claude_config_dir() -> Path:
+ """``CLAUDE_CONFIG_DIR`` for spawned claude subprocesses.
+
+ Resolves to ``<machina_root>/claude/``. Single source of truth for
+ the plugin's ``MACHINA_CLAUDE_DIR`` constant (re-exported from
+ ``nodes/agent/claude_code_agent/_oauth.py`` for back-compat).
+ """
+ return machina_root() / "claude"
+
+
+def claude_npm_dir() -> Path:
+ """Where ``npm install @anthropic-ai/claude-code`` lands.
+
+ ``<claude_config_dir>/npm/`` — keeps the project-local node_modules
+ tree co-located with the rest of claude state.
+ """
+ return claude_config_dir() / "npm"
+
+
+def workspaces_dir() -> Path:
+ """Root for per-workflow workspaces."""
+ return machina_root() / "workspaces"
+
+
+def workspace_dir(workflow_id: str) -> Path:
+ """Per-workflow workspace at ``<machina_root>/workspaces/<workflow_id>/``.
+
+ The workflow executor injects this into the execution context as
+ ``ctx.raw["workspace_dir"]`` and the cli_agent service splices it
+ into each claude task's ``--add-dir`` so claude can read upstream
+ node outputs (``fileDownloader``, ``documentParser``, code
+ executors) + materialise its connected skills under
+ ``<workspace_dir>/.claude/skills/``.
+ """
+ return workspaces_dir() / workflow_id
+
+
+def workflows_dir() -> Path:
+ """Where workflow JSON exports live (auto-loaded by example_loader).
+
+ Moved from ``<repo>/workflows/`` to ``<machina_root>/workflows/``
+ so the entire MachinaOs working set is contained in one dotted
+ directory the user can ignore from their VCS / file browser.
+ """
+ return machina_root() / "workflows"
+
+
+def whatsapp_dir() -> Path:
+ """Persistent WhatsApp session DB / state dir."""
+ return machina_root() / "whatsapp"
+
+
+def credentials_db_path() -> Path:
+ """Encrypted credentials store (Fernet + PBKDF2)."""
+ return machina_root() / "credentials.db"
+
+
+# ---------------------------------------------------------------------------
+# One-time auto-migration on app startup
+# ---------------------------------------------------------------------------
+
+# Map of legacy on-disk locations → new locations under ``.machina/``.
+# Each entry is renamed exactly once: only when the source exists and
+# the destination does not. Subsequent boots are no-ops.
+_LEGACY_RENAMES: list[tuple[Path, Path]] = []
+
+
+def _legacy_renames() -> list[tuple[Path, Path]]:
+ """Build the rename plan from current settings (lazy — needs Settings).
+
+ Sources are always under the repo root (where the pre-cutover code
+ wrote them); destinations are under whatever ``machina_root()``
+ currently resolves to (``~/.machina/`` by default). Renames may
+ cross filesystems on multi-disk setups — :func:`migrate_legacy_layout`
+ falls back to ``shutil.move`` (copy + delete) when ``Path.rename``
+ fails with ``OSError``.
+ """
+ root = machina_root()
+ src_root = project_root()
+ return [
+ # 1. workflows/ → ~/.machina/workflows/
+ (src_root / "workflows", root / "workflows"),
+ # 2. data/ → ~/.machina/ (handled below by per-subdir moves
+ # to keep the flat layout; the bare-rename of ``data/`` itself
+ # would re-introduce the nested structure)
+ # 2a. data/claude-machina/ → ~/.machina/claude/
+ (src_root / "data" / "claude-machina", root / "claude"),
+ # 2b. data/workspaces/ → ~/.machina/workspaces/
+ (src_root / "data" / "workspaces", root / "workspaces"),
+ # 2c. data/whatsapp/ → ~/.machina/whatsapp/
+ (src_root / "data" / "whatsapp", root / "whatsapp"),
+ # 2d. data/credentials.db → ~/.machina/credentials.db
+ (src_root / "data" / "credentials.db", root / "credentials.db"),
+ # 2e. data/machina.db → ~/.machina/machina.db
+ (src_root / "data" / "machina.db", root / "machina.db"),
+ ]
+
+
+def migrate_legacy_layout() -> int:
+ """Rename pre-cutover locations into the new ``.machina/`` layout.
+
+ Called once on app startup from ``main.py``. Idempotent: only
+ moves a path when the source exists AND the destination doesn't.
+ Returns the count of successful renames.
+
+ Safety contract:
+
+ - **Never overwrites existing destinations.** If
+ ``.machina/claude/`` already exists, ``data/claude-machina/``
+ is left in place even if it still exists — the user can
+ delete it manually.
+ - **Best-effort per item.** A failed rename on one entry doesn't
+ block the others; failures log at WARNING.
+ - **No data loss.** Renames are atomic (``Path.rename``) when
+ source and destination are on the same filesystem; falls back
+ to ``shutil.move`` (copy + delete) when they aren't.
+ """
+ root = machina_root()
+ try:
+ root.mkdir(parents=True, exist_ok=True)
+ except OSError as exc:
+ logger.warning(
+ "[paths] cannot create machina root %s: %s — skipping migration",
+ root, exc,
+ )
+ return 0
+
+ moved = 0
+ for src, dst in _legacy_renames():
+ if not src.exists():
+ continue
+ if dst.exists():
+ # Both source and destination exist — migration won't merge
+ # automatically (risk of overwriting newer state). Surface
+ # at WARNING so the user can choose to consolidate manually.
+ logger.warning(
+ "[paths] migration skip: %s -> %s — destination already "
+ "exists; keep the newer one and delete the other, or "
+ "merge their contents manually.",
+ src, dst,
+ )
+ continue
+ try:
+ dst.parent.mkdir(parents=True, exist_ok=True)
+ # Path.rename is atomic same-fs; falls back via shutil.move
+ # which handles cross-fs copy+delete.
+ try:
+ src.rename(dst)
+ except OSError:
+ shutil.move(str(src), str(dst))
+ moved += 1
+ logger.info("[paths] migrated %s -> %s", src, dst)
+ except Exception as exc:
+ logger.warning(
+ "[paths] migration failed %s -> %s: %s", src, dst, exc,
+ )
+
+ # If ``data/`` is now empty after the per-subdir moves, remove it
+ # so the user's tree is clean. Don't force-remove if anything's
+ # still in there (could be user-authored content).
+ legacy_data = project_root() / "data"
+ if legacy_data.is_dir():
+ try:
+ legacy_data.rmdir() # only succeeds if empty
+ logger.info("[paths] removed empty legacy data/ dir")
+ except OSError:
+ # Not empty — leave it alone.
+ pass
+
+ if moved:
+ logger.info(
+ "[paths] migration complete: %d items moved into %s",
+ moved, root,
+ )
+ return moved
+
+
+__all__ = [
+ "project_root",
+ "machina_root",
+ "claude_config_dir",
+ "claude_npm_dir",
+ "workspaces_dir",
+ "workspace_dir",
+ "workflows_dir",
+ "whatsapp_dir",
+ "credentials_db_path",
+ "migrate_legacy_layout",
+]
diff --git a/server/main.py b/server/main.py
index f8c67974..9e96e828 100644
--- a/server/main.py
+++ b/server/main.py
@@ -506,6 +506,20 @@ except Exception as exc: # pragma: no cover — defensive; MCP must not block s
logger.warning("[main] failed to mount CLI agent MCP server: %s", exc)
+# One-time on-disk layout migration: rename pre-cutover ``data/`` +
+# ``workflows/`` trees into the flat ``.machina/`` root. Idempotent;
+# no-op on subsequent boots. Must run BEFORE any lockfile sweep or
+# CLI service init so those consumers see the new paths. See
+# ``server/core/paths.py::migrate_legacy_layout``.
[email protected]_event("startup")
+async def _migrate_legacy_layout_on_startup() -> None:
+ try:
+ from core.paths import migrate_legacy_layout
+ migrate_legacy_layout()
+ except Exception as exc: # pragma: no cover — defensive
+ logger.warning("[main] legacy-layout migration failed: %s", exc)
+
+
# Stale lockfile sweep on startup — mirrors VSCode's behavior. PIDs in
# leftover lockfiles that are no longer alive get cleaned up.
@app.on_event("startup")
diff --git a/server/nodes/agent/claude_code_agent/_oauth.py b/server/nodes/agent/claude_code_agent/_oauth.py
index 57713a7c..cc41dec0 100644
--- a/server/nodes/agent/claude_code_agent/_oauth.py
+++ b/server/nodes/agent/claude_code_agent/_oauth.py
@@ -1,10 +1,13 @@
"""Claude OAuth — project-local install + the documented `claude auth` subcommands.
-The Claude Code CLI lives at ``<repo>/data/claude-machina/npm/`` (on-demand
+The Claude Code CLI lives at ``<machina_root>/claude/npm/`` (on-demand
``npm install`` on first use, mirroring the WhatsApp project-local layout).
-``CLAUDE_CONFIG_DIR`` points at ``<repo>/data/claude-machina/`` so the CLI
+``CLAUDE_CONFIG_DIR`` points at ``<machina_root>/claude/`` so the CLI
manages its own credentials inside the project tree, isolated from the
-user's own ``~/.claude/`` session.
+user's own ``~/.claude/`` session. Path resolution is centralised in
+:mod:`core.paths` — this module just re-exports the constants for
+back-compat with consumers that still ``import MACHINA_CLAUDE_DIR``
+directly.
Every subprocess call goes through ``services.events.cli.run_cli_command``
— the canonical one-shot CLI helper used by Stripe / future plugins. Auth
@@ -21,10 +24,9 @@ Stripe uses for ``stripe login --complete``.
This module lives alongside the ``claude_code_agent`` plugin so all
claude-specific code stays in one folder per the canonical plugin
-pattern. The single source of truth for ``MACHINA_CLAUDE_DIR`` lives
-here; consumers in ``services/cli_agent/`` (the generic framework)
+pattern. Consumers in ``services/cli_agent/`` (the generic framework)
and the plugin's other modules (``_provider``, ``_pool``, ``_skills``)
-import it from this module.
+import ``MACHINA_CLAUDE_DIR`` from here.
"""
from __future__ import annotations
@@ -34,20 +36,23 @@ import os
import shutil
import subprocess
import sys
-from pathlib import Path
from typing import Any, Dict
from core.logging import get_logger
+from core.paths import claude_config_dir, claude_npm_dir
from services.events.cli import run_cli_command
logger = get_logger(__name__)
-# This file: server/nodes/agent/claude_code_agent/_oauth.py
-# parents[4] is the repo root.
-_PROJECT_ROOT = Path(__file__).resolve().parents[4]
-
-MACHINA_CLAUDE_DIR = (_PROJECT_ROOT / "data" / "claude-machina").resolve()
-MACHINA_NPM_DIR = MACHINA_CLAUDE_DIR / "npm"
+# Single source of truth for path resolution lives in ``core.paths``;
+# we re-export as module constants so existing imports
+# (``from ._oauth import MACHINA_CLAUDE_DIR``) keep working without a
+# function-call indirection at every call site. Call ``machina_root``
+# / ``claude_config_dir`` at module load — Settings is already
+# initialised by the time this module is first imported (node registry
+# discovery runs after the FastAPI app is built).
+MACHINA_CLAUDE_DIR = claude_config_dir()
+MACHINA_NPM_DIR = claude_npm_dir()
# Generous timeout for browser-flow login; Stripe uses the same window.
LOGIN_TIMEOUT_SECONDS = 600.0
diff --git a/server/services/example_loader.py b/server/services/example_loader.py
index bceb0477..1cff1e09 100644
--- a/server/services/example_loader.py
+++ b/server/services/example_loader.py
@@ -7,23 +7,22 @@ handler. This module is the first-launch-only convenience wrapper.
"""
import json
import logging
-from pathlib import Path
from typing import List, Dict, Any
-logger = logging.getLogger(__name__)
+from core.paths import workflows_dir
-# Workflows folder at project root (parent of server/)
-EXAMPLES_DIR = Path(__file__).parent.parent.parent / "workflows"
+logger = logging.getLogger(__name__)
def get_example_workflows() -> List[Dict[str, Any]]:
"""Load all example workflow JSON files from disk."""
examples = []
- if not EXAMPLES_DIR.exists():
- logger.warning(f"Examples directory not found: {EXAMPLES_DIR}")
+ examples_dir = workflows_dir()
+ if not examples_dir.exists():
+ logger.warning(f"Examples directory not found: {examples_dir}")
return examples
- for file in sorted(EXAMPLES_DIR.glob("*.json")):
+ for file in sorted(examples_dir.glob("*.json")):
try:
with open(file, encoding="utf-8") as f:
workflow = json.load(f)
diff --git a/server/tests/conftest.py b/server/tests/conftest.py
index 6ad46a55..447ed34f 100644
--- a/server/tests/conftest.py
+++ b/server/tests/conftest.py
@@ -93,6 +93,26 @@ _make_submodule("core", "container", {"container": MagicMock()})
_make_submodule("core", "database", {"Database": MagicMock})
_make_submodule("core", "config", {"Settings": MagicMock})
+# core.paths — central path resolution. Stub the public surface with
+# tmpdir-rooted Paths so plugin module imports don't trip over the
+# real ``Path.home()`` lookup during test collection. Tests that
+# actually exercise on-disk behaviour use ``tempfile.TemporaryDirectory``
+# locally; the stub just keeps import-time ``MACHINA_CLAUDE_DIR =
+# claude_config_dir()`` calls from blowing up.
+_TEST_MACHINA_ROOT = Path(__file__).parent / "_test_machina_root"
+_make_submodule("core", "paths", {
+ "project_root": lambda: _TEST_MACHINA_ROOT.parent,
+ "machina_root": lambda: _TEST_MACHINA_ROOT,
+ "claude_config_dir": lambda: _TEST_MACHINA_ROOT / "claude",
+ "claude_npm_dir": lambda: _TEST_MACHINA_ROOT / "claude" / "npm",
+ "workspaces_dir": lambda: _TEST_MACHINA_ROOT / "workspaces",
+ "workspace_dir": lambda wf: _TEST_MACHINA_ROOT / "workspaces" / wf,
+ "workflows_dir": lambda: _TEST_MACHINA_ROOT / "workflows",
+ "whatsapp_dir": lambda: _TEST_MACHINA_ROOT / "whatsapp",
+ "credentials_db_path": lambda: _TEST_MACHINA_ROOT / "credentials.db",
+ "migrate_legacy_layout": lambda: 0,
+})
+
# services.pricing -- pre-stub the singleton so handler modules that do
# `from services.pricing import get_pricing_service` at module load time