Skip to content

Self-improving AI Employees for everyone

fix(paths): move stripe daemon out of workspaces, claude binary under DATA_DIR

trohitgdcfd7313a720
diff --git a/CLAUDE.md b/CLAUDE.md
index beed6528..d526fb48 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -5300,7 +5300,7 @@ This function:
- **Telegram owner detection — three layers** (f746ddf): the bot owner is captured by THREE independent paths so the realistic setup flows all converge to a working state. (1) **Explicit field**: secondary `FieldDef` in the credentials modal lets the user paste their Telegram user ID directly (recommended; works without DM'ing the bot). (2) **Pre-poll peek** in `connect()` via `_capture_owner_from_pending_updates()` — calls `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue, scans for any historical private DM, captures + persists atomically. (3) **Atomic write-through in `_on_message_received`** — persists FIRST, sets in-memory ONLY on success. Invariant: "in-memory has owner ⇒ DB has owner" so a process restart can re-capture cleanly. Failure logged at `ERROR` with `exc_info=True` (was `WARNING` previously, masking persist failures). The lazy fallback in `telegram_send.py` (read DB → `service.set_owner`) sits on top of all three.
- **Supervisor Job Object loud-failure (cli/tree.py + cli/supervisor.py)** (8a64eb9): the Windows process-tree mechanism depends entirely on a Job Object with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` (no fallback — `start_new_session` is POSIX-only). `pywin32>=308` is a hard supervisor dependency on Windows; the declaration in the root `pyproject.toml` (where `machinaos-cli` lives — `cli/pyproject.toml` was removed as a stale duplicate) carries an inline comment so future maintainers don't drop it. `_JobObject.__init__` splits failures into explicit `ImportError` (with reinstall command) and `Exception` (with `repr`) branches — both write to `stderr` instead of swallowing. `_JobObject.add()` verifies enrollment via `win32job.IsProcessInJob(handle, self._handle)` after assignment; on mismatch it queries `IsProcessInJob(handle, None)` so the warning explicitly names the wrapper-job hypothesis (npm / pnpm / conhost wrappers occasionally place us in their own non-nesting Job). `supervisor._spawn_once` checks `add_to_job` return value and emits a yellow `WARN: pid=N not enrolled in Job Object` per child if False. Without these guards, a stale pywin32 install or a wrapping Job Object silently leaks orphan Python processes on every Ctrl-C and they accumulate to the point of holding SQLite locks that block subsequent backend startup.
- **Credentials envelope-shape invariant** (dc94cde): the `useCredentialPanel` query stores `{values: CredentialFormValues, hadStored: boolean}` (an envelope, NOT a flat dict). The earlier `writeValues` called `setQueryData<CredentialFormValues>` with a spread `(prev) => ({...prev, [key]: value})`, which at runtime merged the typed character at the envelope level next to `values` and `hadStored` instead of inside `.values` — the input selector `panel.values[field.key]` re-rendered with the original (server) value on every keystroke and the input felt frozen. Fix: `writeValues` now preserves the envelope shape and updates only the inner `values` dict; `hadStored` is preserved verbatim (it reflects real server state, not local edits). When introducing other queries shaped as `{data, meta}` envelopes, follow the same pattern — never spread `prev` as if it were the inner payload.
-- **`core.paths` is the SSOT for on-disk locations**: state vs. install split — `machina_root()` (= `~/.machina/`) holds *state* (workspaces, whatsapp session DB, `credentials.db`, `workflow.db`, claude OAuth at `~/.machina/claude/`); `packages_dir()` (= `platformdirs.user_cache_path("MachinaOs")` → `~/.cache/MachinaOs/` on Linux, `~/Library/Caches/MachinaOs/` on macOS, `%LOCALAPPDATA%\MachinaOs\Cache\` on Windows) holds *downloaded binaries* via `package_dir(name)` per service (`stripe`, `claude`, `browser`). Wiping `packages_dir` forces a fresh redownload without dropping auth state. Out of scope: globally-installed binaries (`temporal`, Himalaya) and npm `package.json` deps (`edgymeow`). Shipped seed workflows are the lone exception — `example_workflows_dir()` is hardcoded to `<repo>/.machina/workflows/` (git-tracked seeds, NOT under `DATA_DIR`) so they survive `machina clean` and stay at the same path across `DATA_DIR=~/.machina` vs `DATA_DIR=.machina` configs.
+- **`core.paths` is the SSOT for on-disk locations**: state vs. install split — `machina_root()` (= `~/.machina/`) holds *state* (workspaces, whatsapp session DB, `credentials.db`, `workflow.db`, claude OAuth+binary at `~/.machina/claude/`, supervised daemon cwds at `~/.machina/daemons/<namespace>/`); `packages_dir()` (= `platformdirs.user_cache_path("MachinaOs")` → `~/.cache/MachinaOs/` on Linux, `~/Library/Caches/MachinaOs/` on macOS, `%LOCALAPPDATA%\MachinaOs\Cache\` on Windows) holds *downloaded binaries* via `package_dir(name)` per service (`stripe`, `browser`). Wiping `packages_dir` forces a fresh redownload without dropping auth state. **Claude is the deliberate exception** — its npm tree lives at `<DATA_DIR>/claude/npm/` (via `claude_npm_dir()`) alongside `CLAUDE_CONFIG_DIR` state, so the entire claude footprint sits under one operator-visible subtree. **`daemons_dir()`** (sibling of `workspaces_dir()`) is where `DaemonEventSource.workdir()` places cwds — pre-fix it placed them under `workspaces_dir() / namespace` and the `stripe listen` daemon polluted per-workflow scratch with framework state. Out of scope: globally-installed binaries (`temporal`, Himalaya) and npm `package.json` deps (`edgymeow`). Shipped seed workflows are the lone exception — `example_workflows_dir()` is hardcoded to `<repo>/.machina/workflows/` (git-tracked seeds, NOT under `DATA_DIR`) so they survive `machina clean` and stay at the same path across `DATA_DIR=~/.machina` vs `DATA_DIR=.machina` configs.
- **`machina clean` preserves `.machina/workflows/`**: `cli/commands/clean.py` iterates `<repo>/.machina/` children and skips anything in `_MACHINA_KEEP = frozenset({"workflows"})`. Wipes `claude/`, `workspaces/`, `*.db` as before; preserves the shipped seed JSONs so a fresh clone + clean doesn't lose user-importable examples. Test `cli/tests/test_clean.py::test_machina_keep_preserves_workflows` locks the keep-list.
- **No raw `print()` outside three sanctioned helpers**: `main._startup_log` (pre-logger boot markers), `core.container._clog` (DI-bootstrap markers), and `nodes.code.python_executor.captured_print` (the sandbox builtin handed to user code). Everything else goes through `logger = get_logger(__name__)`. The supervisor prefixes every aggregated line with `[HH:MM:SS.fff]` so no inner `TimeStamper` is needed; structlog console mode is deliberately timestamp-less. Test `server/tests/test_no_raw_prints.py` AST-walks the tree and flags any unsanctioned `print(...)` call.
- **`configure_logging(settings)` must run BEFORE plugin self-registration imports**: in `main.py`, `Settings()` + `configure_logging(settings)` + `init_tracing()` + `get_logger(__name__)` happen ahead of `from core.container import container` and `from routers import …`. Otherwise plugin folders that register on import call `logger.debug(...)` while structlog is still on its default processor chain (which includes `TimeStamper` + no `filter_by_level`) — symptom is double timestamps + debug records leaking despite `LOG_LEVEL=INFO`.
diff --git a/server/core/paths.py b/server/core/paths.py
index c417f9ff..21ba9c0d 100644
--- a/server/core/paths.py
+++ b/server/core/paths.py
@@ -21,17 +21,29 @@ State layout under ``~/.machina/`` (flat, no redundant nesting):
- ``~/.machina/credentials.db`` Fernet-encrypted secrets
- ``~/.machina/workflow.db`` SQLite app DB
-Binaries that MachinaOs downloads on first use (Stripe CLI, the
-``@anthropic-ai/claude-code`` npm tree, ``agent-browser``'s npm tree)
-live separately, under the OS-conventional cache dir (see
-:func:`packages_dir`). Resolved via :func:`platformdirs.user_cache_path`
-— the standard cross-platform resolver every XDG-aware tool already
-uses:
+Binaries that MachinaOs downloads on first use (Stripe CLI,
+``agent-browser``'s npm tree) live separately, under the
+OS-conventional cache dir (see :func:`packages_dir`). Resolved via
+:func:`platformdirs.user_cache_path` — the standard cross-platform
+resolver every XDG-aware tool already uses:
- Linux: ``~/.cache/MachinaOs/<service>/``
- macOS: ``~/Library/Caches/MachinaOs/<service>/``
- Windows: ``%LOCALAPPDATA%\\MachinaOs\\Cache\\<service>\\``
+Claude Code is the deliberate exception: its npm tree lives at
+``<DATA_DIR>/claude/npm/`` (see :func:`claude_npm_dir`) alongside the
+``CLAUDE_CONFIG_DIR`` auth state at ``<DATA_DIR>/claude/``. Keeping
+the entire claude footprint (binary + tokens + IDE lockfiles + session
+JSONL) under one operator-visible subtree means a single
+``mv ~/.machina/claude /backup`` carries everything claude needs;
+every operation requires both anyway, so the binary-vs-state split
+that's useful for Stripe / browser hurts here. Supervised
+event-source daemons (``stripe listen``, future plugins) keep their
+cwd under :func:`daemons_dir` — a sibling of :func:`workspaces_dir`
+so per-workflow scratch (workspaces) is never mixed with framework
+state (daemons).
+
Out of scope here: the downloaded Temporal CLI binary (pooch cache
under :func:`packages_dir`, see ``services/temporal/_install.py``),
Himalaya (``cargo``/``brew`` system install), and npm packages
@@ -199,12 +211,31 @@ def claude_config_dir() -> Path:
def claude_npm_dir() -> Path:
"""Where ``npm install @anthropic-ai/claude-code`` lands.
- Resolves to ``<packages_dir>/claude/npm/`` — under the unified
- install root (see :func:`packages_dir`). Separate from
- :func:`claude_config_dir` (auth state) so ``machina clean`` can
- wipe binaries without dropping auth.
+ Resolves to ``<DATA_DIR>/claude/npm/`` — sibling of
+ :func:`claude_config_dir` (the OAuth state dir, exposed to the
+ CLI via ``CLAUDE_CONFIG_DIR``). Keeps the entire claude footprint
+ (binary + auth state + IDE lockfiles + session JSONL) under one
+ operator-visible tree, so a single ``mv ~/.machina/claude /backup``
+ carries everything claude needs. Deliberate exception to the
+ binary-vs-state split documented on :func:`packages_dir` — see
+ the module docstring.
+ """
+ return data_path("claude") / "npm"
+
+
+def daemons_dir() -> Path:
+ """Root for supervised event-source daemons (``stripe listen``, etc.).
+
+ Resolves to ``<DATA_DIR>/daemons/`` — sibling of
+ :func:`workspaces_dir`. Daemons are framework-owned, long-lived
+ processes whose cwd is just a place to drop log / state files;
+ workspaces are per-workflow scratch owned by workflow nodes
+ (one subdir per ``Workflow.slug``). Conflating the two clutters
+ workspaces with framework state, which is what
+ ``DaemonEventSource.workdir()`` used to do before this helper
+ existed.
"""
- return package_dir("claude") / "npm"
+ return data_path("daemons")
def workspaces_dir() -> Path:
@@ -277,6 +308,7 @@ __all__ = [
"package_dir",
"claude_config_dir",
"claude_npm_dir",
+ "daemons_dir",
"workspaces_dir",
"workspace_dir",
"example_workflows_dir",
diff --git a/server/nodes/stripe/_install.py b/server/nodes/stripe/_install.py
index bdd9422e..fb02ba07 100644
--- a/server/nodes/stripe/_install.py
+++ b/server/nodes/stripe/_install.py
@@ -7,6 +7,14 @@ bin/stripe[.exe]``). A system install on PATH (brew / scoop / apt /
direct binary) is preferred — the download path only fires when no
system binary is found.
+Note: the cache directory above is the *binary* install location and
+is separate from the ``stripe listen`` daemon's working directory,
+which lives at ``<DATA_DIR>/daemons/_stripe/`` (see
+:func:`core.paths.daemons_dir`). The two were not always separate —
+prior to the daemon-cwd cleanup, the daemon ran from
+``<DATA_DIR>/workspaces/_stripe/`` and polluted per-workflow scratch
+space with framework state.
+
Pin a version here when bumping; pre-built archives are signed by
Stripe and served over GitHub's CDN.
"""
@@ -69,9 +77,10 @@ async def ensure_stripe_cli() -> Path:
Resolution order:
1. Cached path from a prior call (in-process).
2. ``shutil.which("stripe")`` — system install on PATH.
- 3. Workspace-local copy at ``_stripe/bin/stripe[.exe]``.
- 4. Fresh download from GitHub releases under
- :data:`_VERSION`.
+ 3. Previously-downloaded copy at ``package_dir("stripe")/bin/
+ stripe[.exe]`` (OS cache, see :func:`core.paths.package_dir`).
+ 4. Fresh download from GitHub releases under :data:`_VERSION`
+ into the same OS-cache directory.
"""
global _cached_path
if _cached_path and _cached_path.exists():
diff --git a/server/services/events/daemon.py b/server/services/events/daemon.py
index dd37bf50..88b63dbc 100644
--- a/server/services/events/daemon.py
+++ b/server/services/events/daemon.py
@@ -70,9 +70,9 @@ class DaemonEventSource(EventSource):
return None
def workdir(self) -> Path:
- from core.config import Settings
+ from core.paths import daemons_dir
- cwd = Path(Settings().workspace_base_resolved).resolve() / self.workflow_namespace
+ cwd = daemons_dir() / self.workflow_namespace
cwd.mkdir(parents=True, exist_ok=True)
return cwd
diff --git a/server/tests/conftest.py b/server/tests/conftest.py
index 160c2351..656bc575 100644
--- a/server/tests/conftest.py
+++ b/server/tests/conftest.py
@@ -114,6 +114,7 @@ _make_submodule(
"package_dir": lambda name: _TEST_MACHINA_ROOT / "packages" / name,
"claude_config_dir": lambda: _TEST_MACHINA_ROOT / "claude",
"claude_npm_dir": lambda: _TEST_MACHINA_ROOT / "claude" / "npm",
+ "daemons_dir": lambda: _TEST_MACHINA_ROOT / "daemons",
"workspaces_dir": lambda: _TEST_MACHINA_ROOT / "workspaces",
"workspace_dir": lambda wf: _TEST_MACHINA_ROOT / "workspaces" / wf,
"example_workflows_dir": lambda: _TEST_MACHINA_ROOT / "workflows",
diff --git a/server/tests/nodes/test_stripe_plugin.py b/server/tests/nodes/test_stripe_plugin.py
index 3615de6b..dc7e8e63 100644
--- a/server/tests/nodes/test_stripe_plugin.py
+++ b/server/tests/nodes/test_stripe_plugin.py
@@ -308,6 +308,7 @@ class TestStripePluginRegistration:
assert src.workflow_namespace == "_stripe"
# Empty binary_name disables the framework's PATH check; the
# plugin resolves the binary itself via ensure_stripe_cli (which
- # falls back to a workspace-local download on systems without
- # a system install of the Stripe CLI).
+ # falls back to an OS-cache download via
+ # ``core.paths.package_dir('stripe')`` on systems without a
+ # system install of the Stripe CLI).
assert src.binary_name == ""