Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
refactor(temporal): strip postgres backend; use official start-dev; fix Windows shutdown
trohitgfdd17f297012
diff --git a/.env.template b/.env.templateindex fd9acc26..81acde84 100644--- a/.env.template+++ b/.env.template@@ -44,50 +44,34 @@ REDIS_ENABLED=falseCACHE_TTL=300# Temporal Configuration-# Enable for durable workflow execution with automatic retries+# Enable for durable workflow execution with automatic retries.+# Backed by ``temporal server start-dev`` against a SQLite db at+# ``TEMPORAL_SQLITE_PATH``. The official ``temporal`` CLI is downloaded+# by pooch from https://temporal.download/cli/archive/latest on+# ``machina build`` (or first ``machina start`` if build was skipped).TEMPORAL_ENABLED=trueTEMPORAL_SERVER_ADDRESS=localhost:7233TEMPORAL_NAMESPACE=defaultTEMPORAL_TASK_QUEUE=machina-tasksTEMPORAL_PER_TYPE_DISPATCH=trueTEMPORAL_AGENT_WORKFLOW_ENABLED=true-# Persistence backend for the supervised Temporal cluster.-# postgres (default) -- pgserver + temporal-server binary. Data lands-# under ``DATA_DIR/postgres/`` (cross-platform-# binary bundle, no Docker required).-# sqlite -- single ``temporal api`` process. In-memory-# (lost on restart); requires the ``temporal``-# CLI on PATH (brew / scoop / cargo / ``npm-# install -g temporal-server``).-TEMPORAL_BACKEND=postgres-# Bind Temporal services to localhost only (true) or all interfaces-# (false). Keep ``true`` for dev / single-host installs.-TEMPORAL_BIND_LOCAL_ONLY=true-# Pinned Temporal server binary version. Pooch downloads + SHA-256-# verifies from temporalio/temporal GitHub releases on first boot.-TEMPORAL_BINARY_VERSION=1.31.0-# Graceful shutdown window for the Temporal child processes.-TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS=30-# Temporal persistence layout (all relative to DATA_DIR unless-# absolute). Override to point at an alternate disk / network mount.-# - sqlite_path: SQLite db file ``temporal server start-dev`` opens-# - postgres_subdir: pgserver data directory (postgres backend)-# - server_subdir: cwd for the temporal-server process (config YAML)+# gRPC frontend port the CLI dev server binds (``--port``). Default per+# https://docs.temporal.io/references/configuration.+TEMPORAL_FRONTEND_GRPC_PORT=7233+# Web UI port (``--ui-port``). ``temporal server start-dev`` defaults+# this to ``--port + 1000``; we override to 8080 to match the+# historical MachinaOS port (kept for muscle memory + bookmarks).+# Browse to http://<host>:<TEMPORAL_UI_PORT>/.+TEMPORAL_UI_PORT=8080+# SQLite db file ``temporal server start-dev --db-filename`` opens.+# Relative to DATA_DIR unless absolute.TEMPORAL_SQLITE_PATH=temporal/temporal.db-TEMPORAL_POSTGRES_SUBDIR=postgres-TEMPORAL_SERVER_SUBDIR=_temporal-# Advanced tuning (uncomment to override defaults):-# TEMPORAL_FRONTEND_GRPC_PORT=7233-# TEMPORAL_MATCHING_GRPC_PORT=7235-# TEMPORAL_HISTORY_GRPC_PORT=7234-# TEMPORAL_WORKER_GRPC_PORT=7239-# TEMPORAL_NUM_HISTORY_SHARDS=4-# TEMPORAL_DEFAULT_MAX_CONNS=20-# TEMPORAL_VISIBILITY_MAX_CONNS=4-# TEMPORAL_MAX_CONN_LIFETIME=5m-# TEMPORAL_POSTGRES_DSN= # override pgserver-managed DSN-# TEMPORAL_PG_READY_TIMEOUT_SECONDS=60-# TEMPORAL_SERVER_READY_TIMEOUT_SECONDS=120+# Graceful shutdown window for the Temporal child process.+TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS=30+# Supervisor readiness-probe window. The CLI binary may need to+# download (~114 MB zip) on first ``machina start`` if ``machina build``+# didn't pre-cache it.+TEMPORAL_SERVER_READY_TIMEOUT_SECONDS=120# Wave 12 event framework. ``true`` (default) routes trigger events# through TriggerListenerWorkflow + Temporal Signal fan-out for thediff --git a/cli/commands/_temporal_specs.py b/cli/commands/_temporal_specs.pyindex 37bb6f54..0c96bc23 100644--- a/cli/commands/_temporal_specs.py+++ b/cli/commands/_temporal_specs.py@@ -1,141 +1,52 @@-"""ServiceSpecs for the Temporal stack — shared by ``start`` and ``dev``.--Returns the right Temporal ``ServiceSpec`` set based on-``cfg.temporal_backend`` (sourced from ``TEMPORAL_BACKEND`` in-``.env.template`` -> ``.env`` -> ``os.environ`` by-:func:`cli.config.load_config`):-- - ``postgres`` (default) — two specs running the supervised runtimes:- ``temporal-postgres`` (pgserver) then ``temporal-server`` (Temporal- binary with YAML config). Both wrap ``BaseSupervisor`` singletons- via ``services.temporal._supervised_runtime`` (a thin shim that- lives next to the runtime factories it imports). Data lands under- ``DATA_DIR/postgres/``; binaries come from a pooch-managed cache- (no system install required).-- - ``sqlite`` — single spec running ``temporal api`` against an- in-memory SQLite store. Requires the standalone ``temporal`` CLI on- PATH. Lighter dev path; state is lost on restart.--The supervisor's TCP readiness probes (``ready_port``) order the-postgres → temporal startup automatically — Temporal won't start until-Postgres is accepting connections.--No backend default lives in this module: ``cfg.temporal_backend``-always carries a concrete value loaded from the env files (see-:mod:`cli.config`). Mismatch with ``core.config.Settings.-temporal_backend`` would mean the supervisor disagrees with the runtime-about which spec is active -- the v0.0.79 regression that crashed-``machina start`` after ``npm install -g machinaos``.+"""ServiceSpec for the Temporal stack — shared by ``start`` and ``dev``.++One ServiceSpec running the supervised :class:`TemporalServerRuntime`+singleton via :mod:`services.temporal._supervised_runtime` (a thin+shim that lives next to the runtime factory it imports). The runtime+spawns the official ``temporal`` CLI (downloaded by pooch from+https://temporal.download/cli/archive/latest) with the ``server+start-dev`` subcommand against a SQLite db at+``settings.temporal_sqlite_path``.++Matches the local-dev install method documented at+https://docs.temporal.io/develop/python/set-up-your-local-python."""from __future__ import annotationsimport osfrom pathlib import Path+from cli.config import Configfrom cli.platform_ import server_dirfrom cli.run import uv_runfrom cli.supervisor import RestartPolicy, ServiceSpec-# Backend names — must match ``core.config.Settings.temporal_backend``-# enum (``"sqlite" | "postgres"``). The active backend comes from-# ``cfg.temporal_backend`` (env-file driven via ``cli.config``);-# no default lives here.-_BACKEND_SQLITE = "sqlite"-_BACKEND_POSTGRES = "postgres"--# Readiness-probe windows. ``temporal-server`` may need to download-# its binary on first run (~90 MB tarball); the Postgres init step-# runs schema migrations once. These are advanced tuning knobs left as-# pure-process-env overrides (commented hints in ``.env.template``):-# overriding them is rare and per-environment, not worth a typed-# Config field.-_PG_READY_TIMEOUT_SECONDS = float(- os.environ.get("TEMPORAL_PG_READY_TIMEOUT_SECONDS", "60"),-)-_SERVER_READY_TIMEOUT_SECONDS = float(- os.environ.get("TEMPORAL_SERVER_READY_TIMEOUT_SECONDS", "120"),-)-# Graceful-shutdown grace. ``TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS`` is-# declared in ``.env.template`` and pushed into ``os.environ`` by-# ``cli.config.load_config``, so this read sees the file value.-_GRACE_SECONDS = float(- os.environ.get("TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS", "30"),-)+def temporal_specs(root: Path, cfg: Config) -> list[ServiceSpec]:+ """Return the Temporal ServiceSpec list.+ ``cfg`` is :class:`cli.config.Config`; ``cfg.temporal_port`` drives+ the gRPC readiness probe.-def _runtime_argv(factory_path: str) -> list[str]:- """Build the argv for one BaseSupervisor singleton, invoked via the- generic supervised-runtime shim. ``factory_path`` is the dotted- ``module:attr`` accessor.-- Composes the uv invocation through :func:`cli.run.uv_run` so every- ``uv run --no-sync`` callsite in the CLI shares the same flag set.- The shim itself lives at ``services.temporal._supervised_runtime``- (next to the runtime factories), so the spawned python only needs- to resolve modules out of the workspace ``.venv``.+ Readiness-probe + graceful-shutdown windows are read at call time+ from ``os.environ`` (pushed there by :func:`cli.config.load_config`+ from ``.env.template``); ``KeyError`` if missing — broken install.+ Doing the reads here, not at module import, keeps the module safe+ to import before ``load_config()`` runs (tests, REPL, reverse+ import order)."""- return uv_run(- "python", "-m", "services.temporal._supervised_runtime",- factory_path,- )---def temporal_specs(root: Path, cfg) -> list[ServiceSpec]:- """Return the Temporal ServiceSpec list for the current backend.-- ``cfg`` is :class:`cli.config.Config`; ``cfg.temporal_backend``- picks the spec set and ``cfg.temporal_port`` drives the gRPC- readiness probe (same port for both ``sqlite`` and ``postgres``).- """- backend = cfg.temporal_backend.strip().lower()- server_cwd = server_dir(root)-- if backend == _BACKEND_POSTGRES:- return [- ServiceSpec(- name="postgres",- argv=_runtime_argv("services.temporal._runtime:get_postgres_runtime"),- cwd=server_cwd,- # pgserver picks a dynamic port; skip the TCP probe here- # and rely on the temporal spec to gate readiness via- # gRPC (it can't connect until Postgres accepts).- ready_port=None,- ready_timeout=_PG_READY_TIMEOUT_SECONDS,- restart=RestartPolicy.ON_CRASH,- terminate_grace_seconds=_GRACE_SECONDS,- ),- ServiceSpec(- name="temporal",- argv=_runtime_argv(- "services.temporal._runtime:get_temporal_server_runtime",- ),- cwd=server_cwd,- ready_port=cfg.temporal_port,- ready_timeout=_SERVER_READY_TIMEOUT_SECONDS,- restart=RestartPolicy.ON_CRASH,- terminate_grace_seconds=_GRACE_SECONDS,- ),- ]-- # sqlite — single spec wrapping the pooch-installed ``temporal``- # CLI's ``server start-dev`` subcommand against a SQLite db at- # ``settings.temporal_sqlite_path``. Same supervised-runtime shim- # as the postgres backend; ``TemporalServerRuntime`` switches on- # ``settings.temporal_backend`` to pick the right binary + argv- # (see ``server/services/temporal/_runtime.py``).return [ServiceSpec(name="temporal",- argv=_runtime_argv(+ argv=uv_run(+ "python", "-m", "services.temporal._supervised_runtime","services.temporal._runtime:get_temporal_server_runtime",),- cwd=server_cwd,+ cwd=server_dir(root),ready_port=cfg.temporal_port,- ready_timeout=_SERVER_READY_TIMEOUT_SECONDS,+ ready_timeout=float(os.environ["TEMPORAL_SERVER_READY_TIMEOUT_SECONDS"]),restart=RestartPolicy.ON_CRASH,- terminate_grace_seconds=_GRACE_SECONDS,+ terminate_grace_seconds=float(os.environ["TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS"]),),]diff --git a/cli/config.py b/cli/config.pyindex 799ce970..cccc26b8 100644--- a/cli/config.py+++ b/cli/config.py@@ -3,18 +3,10 @@Stdlib-only env-var loader. ``.env.template`` ships with every installand is the canonical source of defaults; ``.env`` (created by``scripts/postinstall.js`` on ``npm install -g machinaos``) layers user-overrides on top; ``os.environ`` wins over both at the call site (so-``TEMPORAL_BACKEND=foo machina start`` still works).+overrides on top; ``os.environ`` wins over both at the call site.Merged values are pushed into ``os.environ`` so downstream consumers-that read ``os.environ`` directly (e.g.-:mod:`cli.commands._temporal_specs` deciding which Temporal ServiceSpec-set to build) see the same view as this module. Without that push the-supervisor's environment lookups would return ``None`` even after-``postinstall.js`` copied ``.env.template`` to ``.env`` -- the bug that-made v0.0.79's ``npm install -g machinaos && machina start`` path-silently fall back to the sqlite Temporal spec instead of the-``postgres`` default declared in ``.env.template``.+that read ``os.environ`` directly see the same view as this module.No Python-side hardcoded defaults: ``.env.template`` is the singlesource of truth. If the template is missing (broken install), we error@@ -79,23 +71,26 @@ class Config:nodejs_port: inttemporal_address: strtemporal_enabled: bool- temporal_backend: str+ temporal_ui_port: int@propertydef temporal_port(self) -> int:- try:- return int(self.temporal_address.rsplit(":", 1)[-1])- except (ValueError, IndexError):- return 7233+ return int(self.temporal_address.rsplit(":", 1)[-1])@propertydef all_ports(self) -> list[int]:+ # Ports the supervisor frees pre-spawn and verifies are released+ # on ``machina stop``. ``temporal_port`` (gRPC) and+ # ``temporal_ui_port`` are bound by the same ``temporal server+ # start-dev`` process, so killing one kills both — listing both+ # only matters when cleaning up stale orphans on either port.return [self.client_port,self.backend_port,self.whatsapp_port,self.nodejs_port,self.temporal_port,+ self.temporal_ui_port,]@@ -141,5 +136,5 @@ def load_config(root: Path | None = None) -> Config:nodejs_port=_int(env, "NODEJS_EXECUTOR_PORT"),temporal_address=_require(env, "TEMPORAL_SERVER_ADDRESS"),temporal_enabled=_bool(env, "TEMPORAL_ENABLED"),- temporal_backend=_require(env, "TEMPORAL_BACKEND"),+ temporal_ui_port=_int(env, "TEMPORAL_UI_PORT"),)diff --git a/cli/supervisor.py b/cli/supervisor.pyindex 339e4c49..6d761648 100644--- a/cli/supervisor.py+++ b/cli/supervisor.py@@ -184,7 +184,19 @@ class Manager:await anyio.sleep_forever()async def _stop_proc(self, proc: anyio.abc.Process, spec: ServiceSpec) -> None:- """SIGTERM (POSIX) / TerminateProcess (Win) -> wait grace -> tree-kill.+ """Graceful stop -> wait grace -> tree-kill.++ POSIX: ``proc.terminate()`` sends ``SIGTERM`` to the child.++ Windows: ``os.kill(pid, CTRL_BREAK_EVENT)`` — the only Win32+ signal-equivalent that lets a child run cleanup handlers. We+ spawn every child with ``CREATE_NEW_PROCESS_GROUP``+ (``cli/tree.py:new_session_kwargs``) so ``CTRL_BREAK_EVENT`` is+ deliverable. The alternative — ``proc.terminate()`` — maps to+ ``TerminateProcess()``, which is an instant hard kill (exit+ code 1, no flush, no atexit, no buffer drain). That's what was+ making temporal log ``stopped (exit 1)`` on every Ctrl+C.+ See https://docs.python.org/3/library/subprocess.html#subprocess.CREATE_NEW_PROCESS_GROUP.Wrapped in ``CancelScope(shield=True)`` -- this is the anyio-documented idiom for cleanup that must run to completion during@@ -200,8 +212,11 @@ class Manager:emit(spec.name, color, "stopping", stream="stderr")with anyio.CancelScope(shield=True):try:- proc.terminate()- except ProcessLookupError:+ if sys.platform == "win32":+ os.kill(proc.pid, signal.CTRL_BREAK_EVENT)+ else:+ proc.terminate()+ except (ProcessLookupError, OSError):returnwith anyio.move_on_after(spec.terminate_grace_seconds):await proc.wait()diff --git a/cli/tree.py b/cli/tree.pyindex 6583bfce..55ffa621 100644--- a/cli/tree.py+++ b/cli/tree.py@@ -1,12 +1,29 @@"""Cross-platform process-tree control.-The single most important pattern from VS Code's process model:-guarantee that supervised children die when the supervisor dies. On-POSIX we use ``setsid`` + ``killpg``. On Windows we use a Job Object-with ``JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`` so the OS atomically reaps-every descendant when the supervisor handle closes.+Two cooperating mechanisms guarantee supervised children die when we+want them to (gracefully) AND die when the supervisor dies (forcibly):++POSIX:+ - ``setsid`` makes each child a process-group leader so ``killpg``+ reaches the whole descendant tree.++Windows:+ - ``CREATE_NEW_PROCESS_GROUP`` (per-child) so the supervisor can+ target each child with ``CTRL_BREAK_EVENT`` for graceful shutdown+ without the signal also reaching the supervisor itself+ (https://docs.python.org/3/library/subprocess.html#subprocess.CREATE_NEW_PROCESS_GROUP).+ Required because ``proc.terminate()`` on Windows is+ ``TerminateProcess()`` — an instant hard kill that leaves no time+ for cleanup and forces exit code 1.+ - A single Job Object with ``JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE`` for+ automatic atomic tree-kill if the supervisor dies abnormally.+ Children created by ``CreateProcess`` auto-inherit the job, so+ grandchildren are reaped without manual tree walks+ (https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects).References:+- https://docs.python.org/3/library/subprocess.html#subprocess.CREATE_NEW_PROCESS_GROUP+- https://learn.microsoft.com/en-us/windows/win32/procthread/job-objects- https://nikhilism.com/post/2017/windows-job-objects-process-tree-management/- microsoft/node-pty (uses Job Objects in production)"""@@ -15,23 +32,30 @@ from __future__ import annotationsimport osimport signal+import subprocessimport sysfrom typing import Optionalimport psutil-# ---------------------------------------------------------------- POSIX+# ---------------------------------------------------------------- POSIX / Windows spawn kwargsdef new_session_kwargs() -> dict:- """``Popen``/``open_process`` kwargs to start the child in a new session.+ """``Popen``/``open_process`` kwargs to spawn the child in its own group.- Empty dict on Windows; uses ``start_new_session`` on POSIX so the- child is the leader of its own process group and ``killpg`` reaches- the entire descendant tree.+ POSIX: ``start_new_session=True`` (``setsid``) makes the child a+ process-group leader so ``killpg`` reaches the whole tree.++ Windows: ``creationflags=CREATE_NEW_PROCESS_GROUP`` so the child can+ receive ``CTRL_BREAK_EVENT`` from the supervisor without the console+ Ctrl+C also reaching it. Without this flag, ``proc.terminate()`` on+ Windows falls back to ``TerminateProcess()`` (instant hard kill,+ exit 1, no cleanup chance) — what we're trying to avoid for daemons+ like temporal that buffer state."""if sys.platform == "win32":- return {}+ return {"creationflags": subprocess.CREATE_NEW_PROCESS_GROUP}return {"start_new_session": True}diff --git a/server/core/config.py b/server/core/config.pyindex 2b602f42..0614a543 100644--- a/server/core/config.py+++ b/server/core/config.py@@ -47,17 +47,18 @@ class Settings(BaseSettings):# Execution Enginedlq_enabled: bool = Field(default=False, env="DLQ_ENABLED")- # Temporal Configuration- temporal_enabled: bool = Field(default=False, env="TEMPORAL_ENABLED")- temporal_server_address: str = Field(default="localhost:7233", env="TEMPORAL_SERVER_ADDRESS")- temporal_namespace: str = Field(default="default", env="TEMPORAL_NAMESPACE")- temporal_task_queue: str = Field(default="machina-tasks", env="TEMPORAL_TASK_QUEUE")+ # Temporal Configuration. All sourced from ``.env.template``+ # (canonical defaults live there); no Python-side fallbacks.+ temporal_enabled: bool = Field(env="TEMPORAL_ENABLED")+ temporal_server_address: str = Field(env="TEMPORAL_SERVER_ADDRESS")+ temporal_namespace: str = Field(env="TEMPORAL_NAMESPACE")+ temporal_task_queue: str = Field(env="TEMPORAL_TASK_QUEUE")# F4.A: per-type activity dispatch. When True, MachinaWorkflow.run() schedules# `node.{type}.v{version}` per plugin (with task_queue=cls.task_queue) instead# of the single `execute_node_activity` legacy name. Workers register per-type# activities alongside the legacy one. Default off so existing deployments# behave identically; flip via TEMPORAL_PER_TYPE_DISPATCH=true.- temporal_per_type_dispatch: bool = Field(default=False, env="TEMPORAL_PER_TYPE_DISPATCH")+ temporal_per_type_dispatch: bool = Field(env="TEMPORAL_PER_TYPE_DISPATCH")# F4.B: agent-as-child-workflow. When True, MachinaWorkflow.run() schedules# AgentWorkflow (child workflow) for the 14 migrating agent types# (aiAgent / chatAgent / 11 specialized agents / 2 team leads) instead of@@ -67,7 +68,7 @@ class Settings(BaseSettings):# break across activity boundaries). Default off. Implies# temporal_per_type_dispatch=True.temporal_agent_workflow_enabled: bool = Field(- default=False, env="TEMPORAL_AGENT_WORKFLOW_ENABLED",+ env="TEMPORAL_AGENT_WORKFLOW_ENABLED",)# Wave 12 A3: SIGTERM grace window for Temporal workers. Activities# mid-flight finish (or hand back to the server for retry) instead of@@ -75,7 +76,7 @@ class Settings(BaseSettings):# heartbeat interval — a worker drain completes within one polling# cycle. Tune via env when SIGTERM-to-restart latency matters.temporal_graceful_shutdown_seconds: int = Field(- default=30, env="TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS", ge=1,+ env="TEMPORAL_GRACEFUL_SHUTDOWN_SECONDS", ge=1,)# Wave 12 canary-flag default flipped to True (2026-05-15).# The Temporal-native event dispatch path@@ -91,79 +92,25 @@ class Settings(BaseSettings):default=True, env="EVENT_FRAMEWORK_ENABLED",)- # Persistence backend for the supervised Temporal cluster.- # 'postgres' (default) — pgserver + temporal-server binary with- # YAML config managed by services/temporal/.- # Cross-platform via pip-installed binaries.- # 'sqlite' — single ServiceSpec running `temporal api`.- # Lighter dev path; in-process SQLite, lost on- # restart. Set when you need fast iteration- # and don't care about workflow durability.- temporal_backend: Literal["sqlite", "postgres"] = Field(- default="postgres", env="TEMPORAL_BACKEND",- )- # Internal Temporal service gRPC ports — exposed for the YAML config- # renderer and the runtime's readiness probe. Defaults match the- # ports Temporal documents at https://docs.temporal.io/references/configuration+ # gRPC frontend port — the port ``temporal server start-dev``+ # exposes for clients + supervisor readiness probe. Sourced from+ # ``.env.template`` (canonical default lives there). Reference:+ # https://docs.temporal.io/references/configurationtemporal_frontend_grpc_port: int = Field(- default=7233, env="TEMPORAL_FRONTEND_GRPC_PORT", ge=1024, le=65535,- )- temporal_matching_grpc_port: int = Field(- default=7235, env="TEMPORAL_MATCHING_GRPC_PORT", ge=1024, le=65535,- )- temporal_history_grpc_port: int = Field(- default=7234, env="TEMPORAL_HISTORY_GRPC_PORT", ge=1024, le=65535,- )- temporal_worker_grpc_port: int = Field(- default=7239, env="TEMPORAL_WORKER_GRPC_PORT", ge=1024, le=65535,- )- temporal_bind_local_only: bool = Field(- default=True, env="TEMPORAL_BIND_LOCAL_ONLY",- )- temporal_num_history_shards: int = Field(- default=4, env="TEMPORAL_NUM_HISTORY_SHARDS", ge=1, le=4096,- )- temporal_default_max_conns: int = Field(- default=20, env="TEMPORAL_DEFAULT_MAX_CONNS", ge=1, le=500,- )- temporal_visibility_max_conns: int = Field(- default=4, env="TEMPORAL_VISIBILITY_MAX_CONNS", ge=1, le=500,- )- # Postgres connection rotation interval. Temporal community- # recommendation is ~5 minutes (much shorter than the 1h default)- # because Postgres' default `tcp_keepalives_idle` and pooled-driver- # state can lead to "context canceled" errors on idle connections- # being reused — periodic refresh sidesteps the issue. Accepts- # Go-duration strings (`30s`, `5m`, `1h`).- temporal_max_conn_lifetime: str = Field(- default="5m", env="TEMPORAL_MAX_CONN_LIFETIME",- )- temporal_binary_version: str = Field(- default="1.31.0", env="TEMPORAL_BINARY_VERSION",- )- temporal_postgres_dsn: Optional[str] = Field(- default=None, env="TEMPORAL_POSTGRES_DSN",- )- # On-disk locations for Temporal persistence. Both resolve relative- # to ``DATA_DIR`` (cross-platform via ``Settings._resolve_under_data``);- # absolute paths are used verbatim. ``temporal_sqlite_path`` is the- # SQLite database file the dev server opens via ``temporal server- # start-dev --db-filename ...``. ``temporal_postgres_subdir`` is the- # pgserver data directory (where PostgreSQL writes its own files- # before Temporal touches them). ``temporal_server_subdir`` is the- # cwd for the postgres-backed ``temporal-server`` process (config- # YAML lands there). All three are env-driven so test harnesses and- # alternative installs can point them wherever needed without code- # changes.- temporal_sqlite_path: str = Field(- default="temporal/temporal.db", env="TEMPORAL_SQLITE_PATH",- )- temporal_postgres_subdir: str = Field(- default="postgres", env="TEMPORAL_POSTGRES_SUBDIR",+ env="TEMPORAL_FRONTEND_GRPC_PORT", ge=1024, le=65535,)- temporal_server_subdir: str = Field(- default="_temporal", env="TEMPORAL_SERVER_SUBDIR",+ # Web UI port. ``temporal server start-dev`` defaults this to+ # ``--port + 1000`` (i.e. 8233 alongside the default 7233 gRPC+ # port); declared explicitly via ``--ui-port`` so the binding is+ # intentional and surfaces in status snapshots.+ temporal_ui_port: int = Field(+ env="TEMPORAL_UI_PORT", ge=1024, le=65535,)+ # SQLite db path passed to ``temporal server start-dev+ # --db-filename ...``. Resolved relative to ``DATA_DIR``+ # (``Settings._resolve_under_data``) unless absolute. Sourced from+ # ``.env.template`` (canonical default lives there).+ temporal_sqlite_path: str = Field(env="TEMPORAL_SQLITE_PATH")# API Keys (all optional, injected at runtime)google_maps_api_key: Optional[str] = Field(default=None, env="GOOGLE_MAPS_API_KEY")@@ -287,7 +234,7 @@ class Settings(BaseSettings):# Filesystem-path fields that may use ``~`` in ``.env``. Expanded# once at load time so every downstream consumer# (``core.paths.machina_root``, ``Settings._resolve_under_data``,- # the Temporal pgserver data dir, WhatsApp runtime dir, ...) sees+ # the Temporal SQLite path, WhatsApp runtime dir, ...) sees# a real path. ``Path.expanduser()`` is a no-op when ``~`` isn't# present, so we can run it unconditionally.@field_validator("data_dir", "log_file", mode="after")@@ -346,7 +293,12 @@ class Settings(BaseSettings):"env_file": "../.env","env_file_encoding": "utf-8","case_sensitive": False,- "extra": "forbid",+ # ``ignore`` lets stale ``.env`` files survive obsolete vars+ # (e.g. the postgres-backend Temporal knobs stripped from the+ # supervised cluster path). Without this, existing installs+ # carrying ``TEMPORAL_BACKEND`` / ``TEMPORAL_BIND_LOCAL_ONLY`` /+ # etc. would raise ValidationError on startup.+ "extra": "ignore","env_parse_none_str": "none","env_nested_delimiter": "__",}\ No newline at end of filediff --git a/server/pyproject.toml b/server/pyproject.tomlindex 5f285fb8..13cca381 100644--- a/server/pyproject.toml+++ b/server/pyproject.toml@@ -34,11 +34,11 @@ dependencies = [# YAML parsing (for SKILL.md files + Temporal config rendering)"pyyaml>=6.0",- # Temporal Postgres backend (services/temporal/)- # - pooch: cross-platform GitHub-release binary download + checksum + cache- # - pgserver: bundles PostgreSQL 16.2 binaries per platform (no Docker)+ # Temporal CLI installer (services/temporal/_install.py).+ # pooch: cross-platform binary download + cache (extracts the+ # ``temporal`` CLI from the official URL at+ # https://temporal.download/cli/archive/latest)."pooch>=1.8.2",- "pgserver>=0.1.4",# Auth"bcrypt>=4.2.0",diff --git a/server/services/temporal/__init__.py b/server/services/temporal/__init__.pyindex e3b3fa5f..8d2eea96 100644--- a/server/services/temporal/__init__.py+++ b/server/services/temporal/__init__.py@@ -20,25 +20,21 @@ When TEMPORAL_ENABLED=false:__all__ = ["TemporalExecutor","TemporalClientWrapper",- "PostgresRuntime","TemporalServerRuntime",- "get_postgres_runtime","get_temporal_server_runtime",]from .executor import TemporalExecutorfrom .client import TemporalClientWrapperfrom ._runtime import (- PostgresRuntime,TemporalServerRuntime,- get_postgres_runtime,get_temporal_server_runtime,)# ---- self-registration (Wave 11 plugin-folder pattern) -------------------# WS handlers (temporal_status / _start / _stop) + WS-connect refresh-# (broadcasts current Postgres+Temporal snapshot). Registries are-# idempotent on re-import; same callable for same key is a no-op.+# (broadcasts current Temporal snapshot). Registries are idempotent on+# re-import; same callable for same key is a no-op.from ._handlers import WS_HANDLERS as _WS_HANDLERSfrom ._refresh import refresh_temporal_status as _refresh_temporal_statusfrom services.ws_handler_registry import register_ws_handlersdiff --git a/server/services/temporal/_config.py b/server/services/temporal/_config.pydeleted file mode 100644index 3a067b1b..00000000--- a/server/services/temporal/_config.py+++ /dev/null@@ -1,258 +0,0 @@-"""Temporal YAML config rendering + Postgres schema bootstrap.--Renders a minimal ``services``/``persistence``/``global`` YAML config-pointing at the pgserver-managed Postgres. Runs ``temporal-sql-tool``-to create + version-init the ``temporal`` + ``temporal_visibility``-databases — idempotent, safe to call on every supervisor start.--Reference for the YAML shape:-https://github.com/temporalio/temporal/tree/main/config-"""-from __future__ import annotations--import asyncio-from pathlib import Path-from typing import Any-from urllib.parse import urlparse--import yaml--from core.config import Settings-from core.logging import get_logger--logger = get_logger(__name__)---def parse_postgres_uri(uri: str) -> dict[str, Any]:- """Parse a pgserver-style ``postgresql://user@host:port/db`` URI.-- Single source of truth for URI decomposition — reused by- :mod:`services.temporal._runtime` (port property, status snapshot)- and the SQL-tool subprocess invocations below.- """- parsed = urlparse(uri)- return {- "host": parsed.hostname or "127.0.0.1",- "port": parsed.port or 5432,- "user": parsed.username or "postgres",- "password": parsed.password or "",- }---# Membership ports = gRPC port + this offset. Mirrors the convention-# Temporal's reference config uses (frontend 7233/6933, history-# 7234/6934, matching 7235/6935, worker 7239/6939) so subtraction-# rather than another tunable settings group keeps the surface small.-# https://docs.temporal.io/references/configuration#services-_MEMBERSHIP_PORT_OFFSET = -300--# DB names. Required by temporal-sql-tool's `--database` and the-# YAML `datastores` block; not tunable because the schema files-# bundled with the binary hard-reference them.-_HISTORY_DB_NAME = "temporal"-_VISIBILITY_DB_NAME = "temporal_visibility"---def _sql_datastore(- pg: dict[str, Any], db_name: str, *,- max_conns: int, max_conn_lifetime: str,-) -> dict[str, Any]:- # ``maxIdleConns`` matches ``maxConns`` so the pool stays fully- # warm — Temporal's matching-engine periodic UpdateTaskQueue- # writes are bursty, and a too-small idle pool causes "context- # canceled" errors when the burst arrives faster than the pool- # can create new connections. Mirrors the Temporal community- # recommendation; see docs-internal/TEMPORAL_ARCHITECTURE.md.- return {- "sql": {- "pluginName": "postgres12",- "databaseName": db_name,- "connectAddr": f"{pg['host']}:{pg['port']}",- "connectProtocol": "tcp",- "user": pg["user"],- "password": pg["password"],- "maxConns": max_conns,- "maxIdleConns": max_conns,- "maxConnLifetime": max_conn_lifetime,- },- }---def _service_rpc(*, grpc_port: int, bind_local: bool) -> dict[str, Any]:- return {- "rpc": {- "grpcPort": grpc_port,- "membershipPort": grpc_port + _MEMBERSHIP_PORT_OFFSET,- "bindOnLocalHost": bind_local,- },- }---def render_temporal_config(*, settings: Settings, postgres_uri: str) -> Path:- """Render Temporal YAML config to ``data/_temporal/config.yaml``.-- Every tunable — ports, pool sizes, history shards, bind scope —- lives in :class:`core.config.Settings`. Single-node cluster- (frontend / matching / history / worker on separate gRPC ports- of one process); multi-node scaling is a separate sprint.- """- pg = parse_postgres_uri(postgres_uri)- bind_local = bool(settings.temporal_bind_local_only)- rpc_host = "127.0.0.1" if bind_local else "0.0.0.0"- frontend_port = settings.temporal_frontend_grpc_port- conn_lifetime = settings.temporal_max_conn_lifetime-- config: dict[str, Any] = {- "log": {"stdout": True, "level": "info"},- "persistence": {- "defaultStore": "default",- "visibilityStore": "visibility",- "numHistoryShards": settings.temporal_num_history_shards,- "datastores": {- "default": _sql_datastore(- pg, _HISTORY_DB_NAME,- max_conns=settings.temporal_default_max_conns,- max_conn_lifetime=conn_lifetime,- ),- "visibility": _sql_datastore(- pg, _VISIBILITY_DB_NAME,- max_conns=settings.temporal_visibility_max_conns,- max_conn_lifetime=conn_lifetime,- ),- },- },- "global": {- "membership": {- "name": "temporal",- "broadcastAddress": rpc_host,- "maxJoinDuration": "30s",- },- "pprof": {"port": 0}, # disable pprof; eliminates rand-port noise- },- "services": {- "frontend": _service_rpc(grpc_port=frontend_port, bind_local=bind_local),- "matching": _service_rpc(- grpc_port=settings.temporal_matching_grpc_port, bind_local=bind_local,- ),- "history": _service_rpc(- grpc_port=settings.temporal_history_grpc_port, bind_local=bind_local,- ),- "worker": _service_rpc(- grpc_port=settings.temporal_worker_grpc_port, bind_local=bind_local,- ),- },- "clusterMetadata": {- "enableGlobalNamespace": False,- "failoverVersionIncrement": 10,- "masterClusterName": "active",- "currentClusterName": "active",- "clusterInformation": {- "active": {- "enabled": True,- "initialFailoverVersion": 1,- "rpcName": "frontend",- "rpcAddress": f"{rpc_host}:{frontend_port}",- },- },- },- "dcRedirectionPolicy": {"policy": "noop"},- "archival": {- "history": {"state": "disabled"},- "visibility": {"state": "disabled"},- },- "namespaceDefaults": {- "archival": {- "history": {"state": "disabled"},- "visibility": {"state": "disabled"},- },- },- }-- out_dir = Path(settings.data_dir) / "_temporal"- out_dir.mkdir(parents=True, exist_ok=True)- config_path = out_dir / "config.yaml"- config_path.write_text(yaml.safe_dump(config, sort_keys=False), encoding="utf-8")- logger.info("[Temporal config] rendered %s", config_path)- return config_path---async def bootstrap_temporal_schemas(- *,- sql_tool: Path,- postgres_uri: str,- binary_path: Path,-) -> None:- """Create + version-init both Temporal databases.-- Idempotent. ``create-database`` is a no-op when the DB exists;- ``setup-schema`` / ``update-schema`` apply schema-version- migrations bundled with the Temporal binary.-- Schema versioned-DDL directory lives next to the binary in the- extracted archive at ``schema/postgresql/v12/{temporal,visibility}/versioned``.- """- pg = parse_postgres_uri(postgres_uri)- schema_root = binary_path.parent / "schema" / "postgresql" / "v12"-- common = [- str(sql_tool),- "--plugin", "postgres12",- "--ep", pg["host"],- "--port", str(pg["port"]),- "--user", pg["user"],- "--password", pg["password"],- ]-- for db_name, schema_dir in (- ("temporal", "temporal/versioned"),- ("temporal_visibility", "visibility/versioned"),- ):- # create-database — idempotent (treats "already exists" as success)- await _run_sql_tool(- [*common, "--database", db_name, "create-database", "--database", db_name],- ok_if_substring="already exists",- )- # setup-schema — v0.0 baseline; idempotent (no-op when schema_version table exists)- await _run_sql_tool(- [*common, "--database", db_name, "setup-schema", "-v", "0.0"],- ok_if_substring="already exists",- )- # update-schema — apply versioned DDL files; idempotent- schema_path = schema_root / schema_dir- if not schema_path.exists():- raise FileNotFoundError(- f"[Temporal config] schema dir missing: {schema_path}. "- f"Expected to ship with temporal-server binary at "- f"{binary_path.parent}/schema/postgresql/v12/"- )- await _run_sql_tool(- [*common, "--database", db_name, "update-schema", "-d", str(schema_path)],- )- logger.info("[Temporal config] schemas bootstrapped: temporal + temporal_visibility")---async def _run_sql_tool(argv: list[str], *, ok_if_substring: str | None = None) -> None:- """Run a temporal-sql-tool subcommand. Tolerates idempotency- failures (e.g. "database already exists") when ``ok_if_substring``- matches stderr."""- proc = await asyncio.create_subprocess_exec(- *argv,- stdout=asyncio.subprocess.PIPE,- stderr=asyncio.subprocess.PIPE,- )- stdout, stderr = await proc.communicate()- if proc.returncode == 0:- return- err = (stderr or b"").decode(errors="replace")- if ok_if_substring and ok_if_substring in err:- return- raise RuntimeError(- f"[Temporal config] {' '.join(argv[:4])}... failed (rc={proc.returncode}): {err}"- )---__all__ = [- "render_temporal_config",- "bootstrap_temporal_schemas",- "parse_postgres_uri",-]diff --git a/server/services/temporal/_handlers.py b/server/services/temporal/_handlers.pyindex eb5ac80c..93b580c3 100644--- a/server/services/temporal/_handlers.py+++ b/server/services/temporal/_handlers.py@@ -1,15 +1,15 @@-"""WebSocket handlers for Temporal + Postgres lifecycle commands.+"""WebSocket handlers for the Temporal lifecycle commands.Wire keys registered in :mod:`services.ws_handler_registry`:- - ``temporal_status`` → status snapshot of both runtimes- - ``temporal_start`` → idempotent start (postgres → temporal)- - ``temporal_stop`` → idempotent stop (temporal → postgres)+ - ``temporal_status`` → status snapshot of the runtime+ - ``temporal_start`` → idempotent start+ - ``temporal_stop`` → idempotent stopAll three share the snapshot shape from:func:`services.temporal._refresh.temporal_status_snapshot` — the-single source of truth for the ``{postgres, temporal}`` payload that-the WS-connect refresh callback also emits.+single source of truth for the ``{temporal}`` payload that the+WS-connect refresh callback also emits."""from __future__ import annotations@@ -26,21 +26,17 @@ logger = get_logger(__name__)async def handle_temporal_status(data: dict[str, Any], websocket: WebSocket,) -> dict[str, Any]:- """Return the status snapshot for both Postgres + Temporal runtimes."""+ """Return the status snapshot for the Temporal runtime."""return temporal_status_snapshot()async def handle_temporal_start(data: dict[str, Any], websocket: WebSocket,) -> dict[str, Any]:- """Start Postgres then Temporal. Idempotent — each ``.start()``- returns immediately if the runtime is already running."""- from services.temporal._runtime import (- get_postgres_runtime,- get_temporal_server_runtime,- )-- await get_postgres_runtime().start()+ """Start Temporal. Idempotent — ``.start()`` returns immediately if+ the runtime is already running."""+ from services.temporal._runtime import get_temporal_server_runtime+await get_temporal_server_runtime().start()return {"ok": True, **temporal_status_snapshot()}@@ -48,14 +44,10 @@ async def handle_temporal_start(async def handle_temporal_stop(data: dict[str, Any], websocket: WebSocket,) -> dict[str, Any]:- """Stop Temporal then Postgres. Idempotent."""- from services.temporal._runtime import (- get_postgres_runtime,- get_temporal_server_runtime,- )+ """Stop Temporal. Idempotent."""+ from services.temporal._runtime import get_temporal_server_runtimeawait get_temporal_server_runtime().stop()- await get_postgres_runtime().stop()return {"ok": True, **temporal_status_snapshot()}diff --git a/server/services/temporal/_install.py b/server/services/temporal/_install.pyindex 07a436cc..7a74aa18 100644--- a/server/services/temporal/_install.py+++ b/server/services/temporal/_install.py@@ -1,16 +1,18 @@-"""Cross-platform Temporal server binary downloader using pooch.--Downloads ``temporal-server`` + ``temporal-sql-tool`` from the-``temporalio/temporal`` GitHub releases. Pooch (https://www.fatiando.org/pooch/)-handles platform-asset resolution, SHA-256 verification, XDG-aware-cross-platform caching, archive extraction (tar.gz / zip), and-idempotent retrieval — replacing what would otherwise be ~150 LOC of-custom GitHub-release fetching with ~30 lines.--Checksums come from the release's ``checksums.txt``, queried once via-``gh release view --repo temporalio/temporal --json assets`` and pinned-below. Bump ``_VERSION`` → re-fetch checksums; pooch refuses any asset-whose SHA-256 doesn't match.+"""Cross-platform Temporal CLI binary downloader using pooch.++Downloads the official ``temporal`` CLI from the URL documented at+https://docs.temporal.io/develop/python/set-up-your-local-python+(``temporal.download/cli/archive/latest?platform=<os>&arch=<arch>``).++Pooch handles XDG-aware cross-platform caching and archive extraction+(zip on Windows, tar.gz elsewhere). The ``latest`` URL rotates as new+CLI versions ship, so the download is unverified (TLS gives transport+integrity); pooch caches by local filename, not URL contents, so+re-runs after the first fetch are instant cache hits.++The downloaded ``temporal`` CLI powers ``temporal server start-dev``+(the SQLite/in-memory dev server) and ad-hoc workflow / operator+commands."""from __future__ import annotations@@ -26,68 +28,11 @@ from core.logging import get_loggerlogger = get_logger(__name__)-# Default version. Override via ``TEMPORAL_BINARY_VERSION`` env var-# (read at the start of ``_fetch_sync``). Bump together with the-# matching entry in ``_CHECKSUMS`` — pooch refuses any asset whose-# SHA-256 doesn't match the registry. Schema files bundled with the-# binary feed ``temporal-sql-tool update-schema`` so binary + schema-# versions stay in lockstep automatically.-_DEFAULT_VERSION = "1.31.0"-_BASE_URL = "https://github.com/temporalio/temporal/releases/download"---# Asset-name template — Temporal's release naming convention. The-# version slot is substituted from settings (or _DEFAULT_VERSION).-# Verified against https://github.com/temporalio/temporal/releases-_ASSET_TEMPLATES: dict[tuple[str, str], str] = {- ("Linux", "x86_64"): "temporal_{v}_linux_amd64.tar.gz",- ("Linux", "aarch64"): "temporal_{v}_linux_arm64.tar.gz",- ("Darwin", "x86_64"): "temporal_{v}_darwin_amd64.tar.gz",- ("Darwin", "arm64"): "temporal_{v}_darwin_arm64.tar.gz",- ("Windows", "AMD64"): "temporal_{v}_windows_amd64.zip",- ("Windows", "ARM64"): "temporal_{v}_windows_arm64.zip",-}---# SHA-256 digests per release. Keyed by version → asset filename →-# digest. Bumping ``temporal_binary_version`` in Settings to a version-# not in this map raises a clear error at install time. Regenerate-# via ``gh release view --repo temporalio/temporal --json assets``.-_CHECKSUMS_BY_VERSION: dict[str, dict[str, str]] = {- "1.31.0": {- "temporal_1.31.0_linux_amd64.tar.gz":- "sha256:2e1fe709f7794929691dbf7044fea8935a5af5b4686758ae6477bebdc208012b",- "temporal_1.31.0_linux_arm64.tar.gz":- "sha256:d02e574e2e1ee5b888b4a57d02d8438bb92d6d5eec47fcf18bf190ac594c8ed3",- "temporal_1.31.0_darwin_amd64.tar.gz":- "sha256:c1174e1f3ad82489bb84ab6f4fd00b82ab13cb31ca81ddf8bf449e7f5ecafdb8",- "temporal_1.31.0_darwin_arm64.tar.gz":- "sha256:9023d04591eb3285f31735d9298cbd6214a0627c9e71f5c7b47991c2ffd91d05",- "temporal_1.31.0_windows_amd64.zip":- "sha256:f4b41b3d195803a2a16bc137c2fa81af3f5bbe8044c3b033411bb19af4fdc8b8",- "temporal_1.31.0_windows_arm64.zip":- "sha256:a7b39728ee2d1db51877745ffda450ad0d92034f5cd429223c42438156d6af9a",- },-}--# Names of the binaries we extract from each release tarball. The-# ``temporalio/temporal`` server release contains the cluster binaries-# only — the user-facing ``temporal`` CLI is a separate distribution-# fetched from the official URL documented at-# https://docs.temporal.io/develop/python/set-up-your-local-python:-# - ``temporal-server`` — full cluster binary used by the postgres-# backend with a YAML config-# (``temporal-server start --config <yaml>``).-# - ``temporal-sql-tool`` — schema migrator invoked once at boot to-# create + update the Postgres databases.-_BINARY_NAMES: tuple[str, ...] = ("temporal-server", "temporal-sql-tool")-# Official Temporal CLI download URL. Per the docs at# https://docs.temporal.io/develop/python/set-up-your-local-python the# CLI archive lives at ``temporal.download/cli/archive/latest`` with# ``platform`` and ``arch`` query parameters. Contains a single-# ``temporal`` binary; powers ``temporal server start-dev`` (the-# sqlite/in-memory dev server) and ad-hoc workflow / operator commands.+# ``temporal`` binary._CLI_BASE_URL = "https://temporal.download/cli/archive/latest"_CLI_PLATFORM_MAP: dict[tuple[str, str], tuple[str, str]] = {# platform.system, platform.machine -> (URL platform, URL arch)@@ -99,8 +44,8 @@ _CLI_PLATFORM_MAP: dict[tuple[str, str], tuple[str, str]] = {("Windows", "ARM64"): ("windows", "arm64"),}-# pooch cache namespace. Re-used across versions; pooch's own cache-# layout includes the asset URL so versions don't collide.+# pooch cache namespace. Pooch's own cache layout includes the asset+# URL so versions don't collide._CACHE_NAMESPACE = "machinaos-temporal"@@ -108,68 +53,31 @@ _cached: dict[str, Path] | None = None_lock = asyncio.Lock()-def _platform_key() -> tuple[str, str]:- key = (platform.system(), platform.machine())- if key not in _ASSET_TEMPLATES:- raise RuntimeError(- f"[Temporal install] Unsupported platform: {key}. "- f"Supported: {sorted(_ASSET_TEMPLATES.keys())}"- )- return key---def _asset_name(version: str) -> str:- return _ASSET_TEMPLATES[_platform_key()].format(v=version)---def _resolver(version: str) -> pooch.Pooch:- checksums = _CHECKSUMS_BY_VERSION.get(version)- if checksums is None:- raise RuntimeError(- f"[Temporal install] No SHA-256 digests pinned for version "- f"{version!r}. Known versions: {sorted(_CHECKSUMS_BY_VERSION.keys())}. "- f"Add to _CHECKSUMS_BY_VERSION (see module docstring)."- )- return pooch.create(- path=pooch.os_cache(_CACHE_NAMESPACE),- base_url=f"{_BASE_URL}/v{version}/",- registry=checksums,- )--async def ensure_temporal_binaries(settings: Settings | None = None,) -> dict[str, Path]:- """Return ``{"temporal": Path, "temporal-server": Path, "temporal-sql-tool": Path}``.-- Two fetches, merged into one dict:- - ``temporal-server`` + ``temporal-sql-tool`` from the pinned- ``temporalio/temporal`` GitHub release (SHA-verified).- - ``temporal`` CLI from the official URL at- https://docs.temporal.io/develop/python/set-up-your-local-python- (``temporal.download/cli/archive/latest``). No SHA pin — the- ``latest`` URL rotates and is served over TLS.+ """Return ``{"temporal": Path}`` — the official ``temporal`` CLI binary.Idempotent — first call downloads, subsequent calls hit the poochcache (XDG / OS-conventional dir). Async-locked so concurrent- callers don't double-download. Version comes from- ``settings.temporal_binary_version`` (env: ``TEMPORAL_BINARY_VERSION``);- falls back to ``_DEFAULT_VERSION`` when no settings passed.+ callers don't double-download.++ The ``settings`` argument is accepted for API symmetry but currently+ unused — the official ``latest`` URL has no version slot to override."""global _cached- if settings is None:- settings = Settings()- version = (settings.temporal_binary_version or _DEFAULT_VERSION).lstrip("v")+ # settings accepted for API symmetry; no fields consulted (the+ # ``latest`` URL has no version slot).+ _ = settingsasync with _lock:if _cached is not None:return _cached- cluster_paths = await asyncio.to_thread(_fetch_sync, version)cli_path = await asyncio.to_thread(_fetch_cli_sync)- _cached = {**cluster_paths, "temporal": cli_path}+ _cached = {"temporal": cli_path}logger.info(- "[Temporal install] binaries ready (cluster v=%s + official CLI): %s",- version, {k: str(v) for k, v in _cached.items()},+ "[Temporal install] binary ready: %s",+ {k: str(v) for k, v in _cached.items()},)return _cached@@ -217,38 +125,6 @@ def _fetch_cli_sync() -> Path:return match-def _fetch_sync(version: str) -> dict[str, Path]:- """Synchronous pooch fetch + post-extraction binary location.-- The release tarball contains both ``temporal-server`` and- ``temporal-sql-tool`` (plus the ``temporal`` CLI and ``tctl``) at- the root of the archive. We extract everything via pooch's- processor and return the names listed in ``_BINARY_NAMES``.- """- pup = _resolver(version)- asset = _asset_name(version)- processor = pooch.Unzip() if asset.endswith(".zip") else pooch.Untar()- extracted = pup.fetch(asset, processor=processor, progressbar=True)- suffix = ".exe" if platform.system() == "Windows" else ""-- paths: dict[str, Path] = {}- for name in _BINARY_NAMES:- target = f"{name}{suffix}"- match = next((Path(p) for p in extracted if Path(p).name == target), None)- if match is None:- raise RuntimeError(- f"[Temporal install] Binary {target!r} not found in {asset!r}. "- f"Extracted files: {[Path(p).name for p in extracted]}"- )- paths[name] = match- # Pooch preserves file mode on POSIX, but if the archive was- # repacked without exec bits, ensure the binary is runnable.- if platform.system() != "Windows":- mode = match.stat().st_mode- match.chmod(mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)- return paths--__all__ = ["ensure_temporal_binaries"]@@ -256,12 +132,9 @@ def _main() -> int:"""Standalone entry: ``python -m services.temporal._install``.Used by ``machina build`` step [6/6] to materialise the Temporal- binaries at build time instead of paying the ~90 MB download cost- on first ``machina start``. Fetches, verifies every extracted path- exists on disk, prints the resolved locations. Non-zero exit on- any failure (pooch raises on SHA mismatch / network errors / asset- missing from tarball; we add an explicit post-fetch existence pass- so partial extractions also fail loudly).+ CLI at build time instead of paying the download cost on first+ ``machina start``. Fetches, verifies the binary exists on disk,+ prints the resolved location. Non-zero exit on any failure."""import sys as _sysdiff --git a/server/services/temporal/_refresh.py b/server/services/temporal/_refresh.pyindex 85180aa8..6d1dd8d0 100644--- a/server/services/temporal/_refresh.py+++ b/server/services/temporal/_refresh.py@@ -1,4 +1,4 @@-"""WS-connect status broadcast for the Temporal+Postgres stack.+"""WS-connect status broadcast for the Temporal stack.Registered via ``status_broadcaster.register_service_refresh`` so thefrontend health indicator stays current — every WebSocket client@@ -6,8 +6,8 @@ connect triggers ``_refresh_all_services()`` which fans out to everyregistered callback. Same idiom :mod:`nodes.telegram._refresh` uses.Also exposes :func:`temporal_status_snapshot` — the single source of-truth for the ``{postgres, temporal}`` status shape consumed by both-this refresh callback and ``_handlers.py``'s WS commands.+truth for the ``{temporal}`` status shape consumed by both this refresh+callback and ``_handlers.py``'s WS commands."""from __future__ import annotations@@ -22,27 +22,21 @@ logger = get_logger(__name__)def temporal_status_snapshot() -> dict[str, Any]:- """Return ``{"postgres": {...}, "temporal": {...}}`` snapshots.+ """Return ``{"temporal": {...}}`` snapshot.- Both runtimes' ``status_snapshot()`` exposes the uniform- ``{name, running, started_at, last_error, ...extras}`` shape every- ``BaseSupervisor`` subclass returns. Shared by the WS-refresh- callback below and the ``temporal_status`` / ``_start`` / ``_stop``- handlers in :mod:`services.temporal._handlers`.+ Exposes the uniform ``{name, running, started_at, last_error,+ ...extras}`` shape every ``BaseSupervisor`` subclass returns.+ Shared by the WS-refresh callback below and the+ ``temporal_status`` / ``_start`` / ``_stop`` handlers in+ :mod:`services.temporal._handlers`."""- from services.temporal._runtime import (- get_postgres_runtime,- get_temporal_server_runtime,- )+ from services.temporal._runtime import get_temporal_server_runtime- return {- "postgres": get_postgres_runtime().status_snapshot(),- "temporal": get_temporal_server_runtime().status_snapshot(),- }+ return {"temporal": get_temporal_server_runtime().status_snapshot()}async def refresh_temporal_status(broadcaster: "StatusBroadcaster") -> None:- """Broadcast the Temporal + Postgres status snapshots on WS connect.+ """Broadcast the Temporal status snapshot on WS connect.Signature matches the ``register_service_refresh`` callback contract— :meth:`StatusBroadcaster._refresh_all_services` invokes everydiff --git a/server/services/temporal/_runtime.py b/server/services/temporal/_runtime.pyindex 49724cb8..1866659e 100644--- a/server/services/temporal/_runtime.py+++ b/server/services/temporal/_runtime.py@@ -1,19 +1,15 @@-"""Postgres + Temporal server lifecycle.--Two supervisor subclasses:-- - :class:`PostgresRuntime` subclasses :class:`BaseSupervisor` directly.- pgserver manages its own subprocess (start / stop / port binding),- so we don't drive ``anyio.open_process`` — we just wrap its- lifecycle in the uniform start/stop/status surface.-- - :class:`TemporalServerRuntime` subclasses :class:`BaseProcessSupervisor`.- Fires the binary downloaded by :mod:`services.temporal._install`- against the YAML config rendered by :mod:`services.temporal._config`.--Both use the singleton accessor pattern (``Class.get_instance()``)-from ``BaseSupervisor`` — same idiom :mod:`nodes.whatsapp._runtime`-uses for ``WhatsAppRuntime``.+"""Temporal server lifecycle.++Single supervisor subclass: :class:`TemporalServerRuntime`. Spawns the+official ``temporal`` CLI (downloaded by :mod:`services.temporal._install`+from https://temporal.download/cli/archive/latest) with the+``server start-dev`` subcommand against a SQLite db. Matches the local+dev install method documented at+https://docs.temporal.io/develop/python/set-up-your-local-python.++Uses the singleton accessor pattern (``Class.get_instance()``) from+``BaseSupervisor`` — same idiom :mod:`nodes.whatsapp._runtime` uses for+``WhatsAppRuntime``."""from __future__ import annotations@@ -25,27 +21,20 @@ from typing import Any, Optionalfrom core.config import Settingsfrom core.logging import get_logger-from services._supervisor import BaseProcessSupervisor, BaseSupervisor-from services.temporal._config import parse_postgres_uri+from services._supervisor import BaseProcessSupervisorlogger = get_logger(__name__)-# Default Postgres port; only used as a fallback in ``PostgresRuntime.port``-# before pgserver has actually started and reported its dynamic port.-_PG_DEFAULT_PORT = 5432--# How long each TCP-readiness probe waits per attempt. Mirrors-# ``cli.tcp.probe_tcp_port``'s semantics inside the server-side-# supervisor — sub-second so a stalled subprocess fails health fast.+# How long each TCP-readiness probe waits per attempt. Sub-second so a+# stalled subprocess fails health fast._PROBE_TIMEOUT_SECONDS = 1.0async def _probe_tcp_port(port: int, host: str = "127.0.0.1") -> bool:"""Return ``True`` iff a TCP connection to ``host:port`` succeedswithin :data:`_PROBE_TIMEOUT_SECONDS`. Loopback-friendly readiness- check used by both runtimes' ``health_check`` overrides. Mirrors- :func:`cli.tcp.probe_tcp_port` but keeps server-side modules- independent of the ``cli`` CLI package."""+ check used by ``health_check``. Mirrors :func:`cli.tcp.probe_tcp_port`+ but keeps server-side modules independent of the ``cli`` CLI package."""try:_, writer = await asyncio.wait_for(asyncio.open_connection(host, port),@@ -61,138 +50,24 @@ async def _probe_tcp_port(port: int, host: str = "127.0.0.1") -> bool:return False-class PostgresRuntime(BaseSupervisor):- """pgserver-managed PostgreSQL 16.2.-- pgserver (https://pypi.org/project/pgserver/) bundles full Postgres- binaries cross-platform via pip and exposes a Python API for the- process lifecycle. We wrap that API in BaseSupervisor's start/stop- surface so the rest of the system (status broadcasts, supervisor,- FastAPI lifespan) treats it uniformly with every other supervised- binary.- """-- name = "postgres"-- def __init__(self, settings: Optional[Settings] = None) -> None:- super().__init__()- if settings is None:- settings = Settings()- self.settings = settings- self._pg: Any = None # pgserver.Server instance-- # ---- subclass surface (BaseSupervisor) --------------------------------- def is_running(self) -> bool:- return self._pg is not None-- @property- def _data_dir(self) -> Path:- """Env-driven pgserver data dir (``TEMPORAL_POSTGRES_SUBDIR``),- resolved relative to ``DATA_DIR`` unless absolute."""- return Path(self.settings._resolve_under_data(self.settings.temporal_postgres_subdir))-- async def _do_start(self) -> None:- # Lazy import: pgserver pulls a heavy native dep tree; only- # load it when the Postgres backend is actually selected.- import pgserver-- data_dir = self._data_dir- data_dir.mkdir(parents=True, exist_ok=True)-- def _start_sync() -> Any:- return pgserver.get_server(str(data_dir), cleanup_mode=None)-- self._pg = await asyncio.to_thread(_start_sync)- logger.info("[%s] pgserver ready at %s", self.label, self._pg.get_uri())-- async def _do_stop(self) -> None:- pg = self._pg- self._pg = None- if pg is None:- return-- def _stop_sync() -> None:- try:- pg.cleanup()- except Exception as exc: # noqa: BLE001 — best-effort shutdown- logger.warning("[postgres] cleanup raised %r (continuing)", exc)-- await asyncio.to_thread(_stop_sync)- logger.info("[%s] stopped", self.label)-- async def health_check(self) -> bool:- if not self.is_running():- return False- # pgserver may report running before the socket is accepting;- # use the shared TCP probe — same idiom every supervised- # service uses for `ready_port` readiness checks.- return await _probe_tcp_port(self.port)-- def _extra_status(self) -> dict[str, Any]:- return {- "uri": self.uri,- "port": self.port,- "data_dir": str(self._data_dir),- }-- # ---- public read-only properties --------------------------------------- @property- def uri(self) -> Optional[str]:- return self._pg.get_uri() if self._pg else None-- @property- def port(self) -> int:- # pgserver picks a free port at start; fall back to the default- # when called before _do_start (e.g. early status snapshots).- if self._pg is None:- return _PG_DEFAULT_PORT- return parse_postgres_uri(self._pg.get_uri())["port"]--class TemporalServerRuntime(BaseProcessSupervisor):- """Temporal server, supervised via BaseProcessSupervisor.-- Single runtime class for both backends. The only difference is- which binary + argv we spawn and whether we run the Postgres- schema-bootstrap step; everything else (signal / restart /- tree-kill / health probe) is shared. Backend selected by- ``settings.temporal_backend``:-- - ``postgres`` — spawns ``temporal-server start --config <yaml>``- against a running :class:`PostgresRuntime` instance. YAML config- is rendered by- :func:`services.temporal._config.render_temporal_config`;- schemas are migrated via- :func:`services.temporal._config.bootstrap_temporal_schemas`.+ """Temporal dev server, supervised via BaseProcessSupervisor.- - ``sqlite`` — spawns ``temporal server start-dev --db-filename- <path>`` against a SQLite file. Path comes from- ``settings.temporal_sqlite_path`` (env-driven).+ Spawns ``temporal server start-dev`` against a SQLite db at+ ``settings.temporal_sqlite_path``. The ``temporal`` CLI binary is+ downloaded by :func:`services.temporal._install.ensure_temporal_binaries`+ from the official URL at+ https://temporal.download/cli/archive/latest.- Both binaries (``temporal``, ``temporal-server``,- ``temporal-sql-tool``) ship in the same pooch-cached tarball- extracted by- :func:`services.temporal._install.ensure_temporal_binaries`, so- swapping backends costs no extra download.-- All on-disk locations come from env-driven settings (no hardcoded- subdirs in this module): ``settings.temporal_sqlite_path``,- ``settings.temporal_postgres_subdir``,- ``settings.temporal_server_subdir`` — all resolved relative to- ``settings.data_dir`` unless already absolute.+ Path is env-driven via ``settings.temporal_sqlite_path``, resolved+ relative to ``settings.data_dir`` unless already absolute."""name = "temporal"pipe_streams = Truegraceful_shutdown = sys.platform == "win32"- def __init__(- self,- settings: Optional[Settings] = None,- postgres: Optional[PostgresRuntime] = None,- ) -> None:+ def __init__(self, settings: Optional[Settings] = None) -> None:super().__init__()if settings is None:settings = Settings()@@ -202,16 +77,7 @@ class TemporalServerRuntime(BaseProcessSupervisor):self.terminate_grace_seconds = float(settings.temporal_graceful_shutdown_seconds,)- # Postgres runtime is only consulted when ``backend == "postgres"``;- # lazy-bound there so the sqlite path doesn't spin up an unused- # PostgresRuntime singleton or pull pgserver imports.- self._postgres = postgresself._binaries: Optional[dict[str, Path]] = None- self._config_path: Optional[Path] = None-- @property- def backend(self) -> str:- return self.settings.temporal_backend.strip().lower()@propertydef _sqlite_path(self) -> Path:@@ -219,101 +85,55 @@ class TemporalServerRuntime(BaseProcessSupervisor):resolved relative to ``DATA_DIR`` unless absolute."""return Path(self.settings._resolve_under_data(self.settings.temporal_sqlite_path))- @property- def _server_cwd(self) -> Path:- """Env-driven cwd subdir for the temporal-server process- (``TEMPORAL_SERVER_SUBDIR``)."""- return Path(self.settings._resolve_under_data(self.settings.temporal_server_subdir))-# ---- BaseProcessSupervisor overrides ---------------------------------async def _pre_spawn(self) -> None:from services.temporal._install import ensure_temporal_binaries- # 1. Download / cache Temporal binaries (both backends share- # the same tarball; pooch deduplicates across calls).self._binaries = await ensure_temporal_binaries(self.settings)-- if self.backend == "postgres":- from services.temporal._config import (- bootstrap_temporal_schemas,- render_temporal_config,- )-- if self._postgres is None:- self._postgres = PostgresRuntime.get_instance(self.settings)- if self._postgres.uri is None:- raise RuntimeError(- f"[{self.label}] Postgres runtime not started; "- "schedule the postgres ServiceSpec before this one"- )-- # Idempotent schema bootstrap.- await bootstrap_temporal_schemas(- sql_tool=self._binaries["temporal-sql-tool"],- postgres_uri=self._postgres.uri,- binary_path=self._binaries["temporal-server"],- )-- # Render YAML config pointing at the Postgres URI.- self._config_path = render_temporal_config(- settings=self.settings,- postgres_uri=self._postgres.uri,- )- else:- # sqlite backend — ensure the parent dir for the SQLite- # file exists before ``temporal server start-dev`` opens it.- self._sqlite_path.parent.mkdir(parents=True, exist_ok=True)+ # Ensure the parent dir for the SQLite file exists before+ # ``temporal server start-dev`` opens it.+ self._sqlite_path.parent.mkdir(parents=True, exist_ok=True)def binary_path(self) -> Path:# ``_pre_spawn`` (called by ``BaseProcessSupervisor._do_start``# before this method) populates ``self._binaries`` via the# pooch downloader. Loud failure if that contract regresses.- binary = "temporal-server" if self.backend == "postgres" else "temporal"assert self._binaries is not None, (f"[{self.label}] binary_path() called before _pre_spawn() ""populated self._binaries")- return self._binaries[binary]+ return self._binaries["temporal"]def argv(self) -> list[str]:- if self.backend == "postgres":- return [- str(self.binary_path()),- "start", "--config", str(self._config_path),- ]-# ``temporal server start-dev`` is the official subcommand for# the SQLite-backed dev server. Flags documented at# https://docs.temporal.io/cli/server (subset we use):# --port frontend gRPC port (gates ready-probe)+ # --ui-port Web UI port (default ``--port + 1000``)# --db-filename SQLite file (omit for in-memory)# --metrics-port 0 disables the Prometheus endpoint# --log-level warn keeps the supervisor log readable- # --ip 127.0.0.1 when TEMPORAL_BIND_LOCAL_ONLY=true# --namespace default namespace bootstrapped at start- ip = "127.0.0.1" if self.settings.temporal_bind_local_only else "0.0.0.0"return [str(self.binary_path()), "server", "start-dev","--port", str(self.settings.temporal_frontend_grpc_port),+ "--ui-port", str(self.settings.temporal_ui_port),"--db-filename", str(self._sqlite_path),"--metrics-port", "0","--log-level", "warn",- "--ip", ip,"--namespace", self.settings.temporal_namespace,]def cwd(self) -> Path:- if self.backend == "postgres":- return self._server_cwd- # sqlite — cwd is the parent of the SQLite file so any default+ # cwd is the parent of the SQLite file so any default# output / log files land alongside the db rather than in the# supervisor's working directory.return self._sqlite_path.parentdef env(self) -> dict[str, str]:- # Temporal reads everything from YAML (postgres) or argv flags- # (sqlite); inherit parent env only.+ # ``temporal server start-dev`` reads everything from argv flags;+ # inherit parent env only.return {**os.environ}async def health_check(self) -> bool:@@ -326,27 +146,15 @@ class TemporalServerRuntime(BaseProcessSupervisor):def _extra_status(self) -> dict[str, Any]:base = super()._extra_status()- extra: dict[str, Any] = {+ return {**base,- "backend": self.backend,"grpc_port": self.settings.temporal_frontend_grpc_port,- "binary_version": self.settings.temporal_binary_version,+ "ui_port": self.settings.temporal_ui_port,+ "sqlite_path": str(self._sqlite_path),}- if self.backend == "postgres":- extra["config_path"] = (- str(self._config_path) if self._config_path else None- )- else:- extra["sqlite_path"] = str(self._sqlite_path)- return extra---# ---- module-level singleton accessors ------------------------------------def get_postgres_runtime(settings: Optional[Settings] = None) -> PostgresRuntime:- """Return the Postgres runtime singleton."""- return PostgresRuntime.get_instance(settings)+# ---- module-level singleton accessor ------------------------------------def get_temporal_server_runtime(settings: Optional[Settings] = None,@@ -356,8 +164,6 @@ def get_temporal_server_runtime(__all__ = [- "PostgresRuntime","TemporalServerRuntime",- "get_postgres_runtime","get_temporal_server_runtime",]diff --git a/server/services/temporal/_supervised_runtime.py b/server/services/temporal/_supervised_runtime.pyindex b660da41..2b6911d7 100644--- a/server/services/temporal/_supervised_runtime.py+++ b/server/services/temporal/_supervised_runtime.py@@ -2,8 +2,8 @@The CLI's ``Manager`` supervisor (``cli/supervisor.py``) schedules``ServiceSpec`` entries by spawning a subprocess and supervising it.-``BaseSupervisor``-managed runtimes (pgserver, Temporal binary) don't-ship a CLI of their own -- they're Python singletons started via+``BaseSupervisor``-managed runtimes (e.g. the Temporal CLI dev server)+don't ship a CLI of their own -- they're Python singletons started via``await runtime.start()``.This shim bridges the two: import a runtime factory by dotted path,@@ -12,23 +12,24 @@ call it, ``await singleton.start()``, then block on a shutdown signal.``await singleton.stop()``, then exit cleanly.Module location: this lives inside the ``server`` package because the-factory targets it resolves (``services.temporal._runtime:get_postgres-_runtime`` etc.) are server-side singletons. Hosting the shim here-keeps the CLI free of server-side imports and lets a single ``uv run-python -m services.temporal._supervised_runtime <factory>`` invocation-work against the workspace ``.venv`` -- no PYTHONPATH composition, no+factory targets it resolves+(``services.temporal._runtime:get_temporal_server_runtime``) are+server-side singletons. Hosting the shim here keeps the CLI free of+server-side imports and lets a single ``uv run python -m+services.temporal._supervised_runtime <factory>`` invocation work+against the workspace ``.venv`` -- no PYTHONPATH composition, nocross-venv plumbing.Invocation (inside ``ServiceSpec.argv``, built via ``cli.run.uv_run``):uv_run("python", "-m","services.temporal._supervised_runtime",- "services.temporal._runtime:get_postgres_runtime")+ "services.temporal._runtime:get_temporal_server_runtime")Equivalent to (Python idiom):- from services.temporal._runtime import get_postgres_runtime- runtime = get_postgres_runtime()+ from services.temporal._runtime import get_temporal_server_runtime+ runtime = get_temporal_server_runtime()await runtime.start()await asyncio.Event().wait() # block on signal"""