Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
fix(paths): move stripe daemon out of workspaces, claude binary under DATA_DIR
trohitgdcfd7313a720
diff --git a/CLAUDE.md b/CLAUDE.mdindex beed6528..d526fb48 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -5300,7 +5300,7 @@ This function:- **Telegram owner detection — three layers** (f746ddf): the bot owner is captured by THREE independent paths so the realistic setup flows all converge to a working state. (1) **Explicit field**: secondary `FieldDef` in the credentials modal lets the user paste their Telegram user ID directly (recommended; works without DM'ing the bot). (2) **Pre-poll peek** in `connect()` via `_capture_owner_from_pending_updates()` — calls `bot.get_updates(timeout=0)` BEFORE `start_polling(drop_pending_updates=True)` discards the queue, scans for any historical private DM, captures + persists atomically. (3) **Atomic write-through in `_on_message_received`** — persists FIRST, sets in-memory ONLY on success. Invariant: "in-memory has owner ⇒ DB has owner" so a process restart can re-capture cleanly. Failure logged at `ERROR` with `exc_info=True` (was `WARNING` previously, masking persist failures). The lazy fallback in `telegram_send.py` (read DB → `service.set_owner`) sits on top of all three.- **Supervisor Job Object loud-failure (cli/tree.py + cli/supervisor.py)** (8a64eb9): the Windows process-tree mechanism depends entirely on a Job Object with `JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE` (no fallback — `start_new_session` is POSIX-only). `pywin32>=308` is a hard supervisor dependency on Windows; the declaration in the root `pyproject.toml` (where `machinaos-cli` lives — `cli/pyproject.toml` was removed as a stale duplicate) carries an inline comment so future maintainers don't drop it. `_JobObject.__init__` splits failures into explicit `ImportError` (with reinstall command) and `Exception` (with `repr`) branches — both write to `stderr` instead of swallowing. `_JobObject.add()` verifies enrollment via `win32job.IsProcessInJob(handle, self._handle)` after assignment; on mismatch it queries `IsProcessInJob(handle, None)` so the warning explicitly names the wrapper-job hypothesis (npm / pnpm / conhost wrappers occasionally place us in their own non-nesting Job). `supervisor._spawn_once` checks `add_to_job` return value and emits a yellow `WARN: pid=N not enrolled in Job Object` per child if False. Without these guards, a stale pywin32 install or a wrapping Job Object silently leaks orphan Python processes on every Ctrl-C and they accumulate to the point of holding SQLite locks that block subsequent backend startup.- **Credentials envelope-shape invariant** (dc94cde): the `useCredentialPanel` query stores `{values: CredentialFormValues, hadStored: boolean}` (an envelope, NOT a flat dict). The earlier `writeValues` called `setQueryData<CredentialFormValues>` with a spread `(prev) => ({...prev, [key]: value})`, which at runtime merged the typed character at the envelope level next to `values` and `hadStored` instead of inside `.values` — the input selector `panel.values[field.key]` re-rendered with the original (server) value on every keystroke and the input felt frozen. Fix: `writeValues` now preserves the envelope shape and updates only the inner `values` dict; `hadStored` is preserved verbatim (it reflects real server state, not local edits). When introducing other queries shaped as `{data, meta}` envelopes, follow the same pattern — never spread `prev` as if it were the inner payload.-- **`core.paths` is the SSOT for on-disk locations**: state vs. install split — `machina_root()` (= `~/.machina/`) holds *state* (workspaces, whatsapp session DB, `credentials.db`, `workflow.db`, claude OAuth at `~/.machina/claude/`); `packages_dir()` (= `platformdirs.user_cache_path("MachinaOs")` → `~/.cache/MachinaOs/` on Linux, `~/Library/Caches/MachinaOs/` on macOS, `%LOCALAPPDATA%\MachinaOs\Cache\` on Windows) holds *downloaded binaries* via `package_dir(name)` per service (`stripe`, `claude`, `browser`). Wiping `packages_dir` forces a fresh redownload without dropping auth state. Out of scope: globally-installed binaries (`temporal`, Himalaya) and npm `package.json` deps (`edgymeow`). Shipped seed workflows are the lone exception — `example_workflows_dir()` is hardcoded to `<repo>/.machina/workflows/` (git-tracked seeds, NOT under `DATA_DIR`) so they survive `machina clean` and stay at the same path across `DATA_DIR=~/.machina` vs `DATA_DIR=.machina` configs.+- **`core.paths` is the SSOT for on-disk locations**: state vs. install split — `machina_root()` (= `~/.machina/`) holds *state* (workspaces, whatsapp session DB, `credentials.db`, `workflow.db`, claude OAuth+binary at `~/.machina/claude/`, supervised daemon cwds at `~/.machina/daemons/<namespace>/`); `packages_dir()` (= `platformdirs.user_cache_path("MachinaOs")` → `~/.cache/MachinaOs/` on Linux, `~/Library/Caches/MachinaOs/` on macOS, `%LOCALAPPDATA%\MachinaOs\Cache\` on Windows) holds *downloaded binaries* via `package_dir(name)` per service (`stripe`, `browser`). Wiping `packages_dir` forces a fresh redownload without dropping auth state. **Claude is the deliberate exception** — its npm tree lives at `<DATA_DIR>/claude/npm/` (via `claude_npm_dir()`) alongside `CLAUDE_CONFIG_DIR` state, so the entire claude footprint sits under one operator-visible subtree. **`daemons_dir()`** (sibling of `workspaces_dir()`) is where `DaemonEventSource.workdir()` places cwds — pre-fix it placed them under `workspaces_dir() / namespace` and the `stripe listen` daemon polluted per-workflow scratch with framework state. Out of scope: globally-installed binaries (`temporal`, Himalaya) and npm `package.json` deps (`edgymeow`). Shipped seed workflows are the lone exception — `example_workflows_dir()` is hardcoded to `<repo>/.machina/workflows/` (git-tracked seeds, NOT under `DATA_DIR`) so they survive `machina clean` and stay at the same path across `DATA_DIR=~/.machina` vs `DATA_DIR=.machina` configs.- **`machina clean` preserves `.machina/workflows/`**: `cli/commands/clean.py` iterates `<repo>/.machina/` children and skips anything in `_MACHINA_KEEP = frozenset({"workflows"})`. Wipes `claude/`, `workspaces/`, `*.db` as before; preserves the shipped seed JSONs so a fresh clone + clean doesn't lose user-importable examples. Test `cli/tests/test_clean.py::test_machina_keep_preserves_workflows` locks the keep-list.- **No raw `print()` outside three sanctioned helpers**: `main._startup_log` (pre-logger boot markers), `core.container._clog` (DI-bootstrap markers), and `nodes.code.python_executor.captured_print` (the sandbox builtin handed to user code). Everything else goes through `logger = get_logger(__name__)`. The supervisor prefixes every aggregated line with `[HH:MM:SS.fff]` so no inner `TimeStamper` is needed; structlog console mode is deliberately timestamp-less. Test `server/tests/test_no_raw_prints.py` AST-walks the tree and flags any unsanctioned `print(...)` call.- **`configure_logging(settings)` must run BEFORE plugin self-registration imports**: in `main.py`, `Settings()` + `configure_logging(settings)` + `init_tracing()` + `get_logger(__name__)` happen ahead of `from core.container import container` and `from routers import …`. Otherwise plugin folders that register on import call `logger.debug(...)` while structlog is still on its default processor chain (which includes `TimeStamper` + no `filter_by_level`) — symptom is double timestamps + debug records leaking despite `LOG_LEVEL=INFO`.diff --git a/server/core/paths.py b/server/core/paths.pyindex c417f9ff..21ba9c0d 100644--- a/server/core/paths.py+++ b/server/core/paths.py@@ -21,17 +21,29 @@ State layout under ``~/.machina/`` (flat, no redundant nesting):- ``~/.machina/credentials.db`` Fernet-encrypted secrets- ``~/.machina/workflow.db`` SQLite app DB-Binaries that MachinaOs downloads on first use (Stripe CLI, the-``@anthropic-ai/claude-code`` npm tree, ``agent-browser``'s npm tree)-live separately, under the OS-conventional cache dir (see-:func:`packages_dir`). Resolved via :func:`platformdirs.user_cache_path`-— the standard cross-platform resolver every XDG-aware tool already-uses:+Binaries that MachinaOs downloads on first use (Stripe CLI,+``agent-browser``'s npm tree) live separately, under the+OS-conventional cache dir (see :func:`packages_dir`). Resolved via+:func:`platformdirs.user_cache_path` — the standard cross-platform+resolver every XDG-aware tool already uses:- Linux: ``~/.cache/MachinaOs/<service>/``- macOS: ``~/Library/Caches/MachinaOs/<service>/``- Windows: ``%LOCALAPPDATA%\\MachinaOs\\Cache\\<service>\\``+Claude Code is the deliberate exception: its npm tree lives at+``<DATA_DIR>/claude/npm/`` (see :func:`claude_npm_dir`) alongside the+``CLAUDE_CONFIG_DIR`` auth state at ``<DATA_DIR>/claude/``. Keeping+the entire claude footprint (binary + tokens + IDE lockfiles + session+JSONL) under one operator-visible subtree means a single+``mv ~/.machina/claude /backup`` carries everything claude needs;+every operation requires both anyway, so the binary-vs-state split+that's useful for Stripe / browser hurts here. Supervised+event-source daemons (``stripe listen``, future plugins) keep their+cwd under :func:`daemons_dir` — a sibling of :func:`workspaces_dir`+so per-workflow scratch (workspaces) is never mixed with framework+state (daemons).+Out of scope here: the downloaded Temporal CLI binary (pooch cacheunder :func:`packages_dir`, see ``services/temporal/_install.py``),Himalaya (``cargo``/``brew`` system install), and npm packages@@ -199,12 +211,31 @@ def claude_config_dir() -> Path:def claude_npm_dir() -> Path:"""Where ``npm install @anthropic-ai/claude-code`` lands.- Resolves to ``<packages_dir>/claude/npm/`` — under the unified- install root (see :func:`packages_dir`). Separate from- :func:`claude_config_dir` (auth state) so ``machina clean`` can- wipe binaries without dropping auth.+ Resolves to ``<DATA_DIR>/claude/npm/`` — sibling of+ :func:`claude_config_dir` (the OAuth state dir, exposed to the+ CLI via ``CLAUDE_CONFIG_DIR``). Keeps the entire claude footprint+ (binary + auth state + IDE lockfiles + session JSONL) under one+ operator-visible tree, so a single ``mv ~/.machina/claude /backup``+ carries everything claude needs. Deliberate exception to the+ binary-vs-state split documented on :func:`packages_dir` — see+ the module docstring.+ """+ return data_path("claude") / "npm"+++def daemons_dir() -> Path:+ """Root for supervised event-source daemons (``stripe listen``, etc.).++ Resolves to ``<DATA_DIR>/daemons/`` — sibling of+ :func:`workspaces_dir`. Daemons are framework-owned, long-lived+ processes whose cwd is just a place to drop log / state files;+ workspaces are per-workflow scratch owned by workflow nodes+ (one subdir per ``Workflow.slug``). Conflating the two clutters+ workspaces with framework state, which is what+ ``DaemonEventSource.workdir()`` used to do before this helper+ existed."""- return package_dir("claude") / "npm"+ return data_path("daemons")def workspaces_dir() -> Path:@@ -277,6 +308,7 @@ __all__ = ["package_dir","claude_config_dir","claude_npm_dir",+ "daemons_dir","workspaces_dir","workspace_dir","example_workflows_dir",diff --git a/server/nodes/stripe/_install.py b/server/nodes/stripe/_install.pyindex bdd9422e..fb02ba07 100644--- a/server/nodes/stripe/_install.py+++ b/server/nodes/stripe/_install.py@@ -7,6 +7,14 @@ bin/stripe[.exe]``). A system install on PATH (brew / scoop / apt /direct binary) is preferred — the download path only fires when nosystem binary is found.+Note: the cache directory above is the *binary* install location and+is separate from the ``stripe listen`` daemon's working directory,+which lives at ``<DATA_DIR>/daemons/_stripe/`` (see+:func:`core.paths.daemons_dir`). The two were not always separate —+prior to the daemon-cwd cleanup, the daemon ran from+``<DATA_DIR>/workspaces/_stripe/`` and polluted per-workflow scratch+space with framework state.+Pin a version here when bumping; pre-built archives are signed byStripe and served over GitHub's CDN."""@@ -69,9 +77,10 @@ async def ensure_stripe_cli() -> Path:Resolution order:1. Cached path from a prior call (in-process).2. ``shutil.which("stripe")`` — system install on PATH.- 3. Workspace-local copy at ``_stripe/bin/stripe[.exe]``.- 4. Fresh download from GitHub releases under- :data:`_VERSION`.+ 3. Previously-downloaded copy at ``package_dir("stripe")/bin/+ stripe[.exe]`` (OS cache, see :func:`core.paths.package_dir`).+ 4. Fresh download from GitHub releases under :data:`_VERSION`+ into the same OS-cache directory."""global _cached_pathif _cached_path and _cached_path.exists():diff --git a/server/services/events/daemon.py b/server/services/events/daemon.pyindex dd37bf50..88b63dbc 100644--- a/server/services/events/daemon.py+++ b/server/services/events/daemon.py@@ -70,9 +70,9 @@ class DaemonEventSource(EventSource):return Nonedef workdir(self) -> Path:- from core.config import Settings+ from core.paths import daemons_dir- cwd = Path(Settings().workspace_base_resolved).resolve() / self.workflow_namespace+ cwd = daemons_dir() / self.workflow_namespacecwd.mkdir(parents=True, exist_ok=True)return cwddiff --git a/server/tests/conftest.py b/server/tests/conftest.pyindex 160c2351..656bc575 100644--- a/server/tests/conftest.py+++ b/server/tests/conftest.py@@ -114,6 +114,7 @@ _make_submodule("package_dir": lambda name: _TEST_MACHINA_ROOT / "packages" / name,"claude_config_dir": lambda: _TEST_MACHINA_ROOT / "claude","claude_npm_dir": lambda: _TEST_MACHINA_ROOT / "claude" / "npm",+ "daemons_dir": lambda: _TEST_MACHINA_ROOT / "daemons","workspaces_dir": lambda: _TEST_MACHINA_ROOT / "workspaces","workspace_dir": lambda wf: _TEST_MACHINA_ROOT / "workspaces" / wf,"example_workflows_dir": lambda: _TEST_MACHINA_ROOT / "workflows",diff --git a/server/tests/nodes/test_stripe_plugin.py b/server/tests/nodes/test_stripe_plugin.pyindex 3615de6b..dc7e8e63 100644--- a/server/tests/nodes/test_stripe_plugin.py+++ b/server/tests/nodes/test_stripe_plugin.py@@ -308,6 +308,7 @@ class TestStripePluginRegistration:assert src.workflow_namespace == "_stripe"# Empty binary_name disables the framework's PATH check; the# plugin resolves the binary itself via ensure_stripe_cli (which- # falls back to a workspace-local download on systems without- # a system install of the Stripe CLI).+ # falls back to an OS-cache download via+ # ``core.paths.package_dir('stripe')`` on systems without a+ # system install of the Stripe CLI).assert src.binary_name == ""