Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
feat(temporal): pin the Temporal CLI and start a new server minor empty
trohitgb357b22edfe9
diff --git a/.env.template b/.env.templateindex 40606686..70b11d75 100644--- a/.env.template+++ b/.env.template@@ -76,9 +76,10 @@ CACHE_TTL=300# Temporal Configuration# Enable for durable workflow execution with automatic retries.# Backed by ``temporal server start-dev`` against a SQLite db at-# ``TEMPORAL_SQLITE_PATH``. The official ``temporal`` CLI is downloaded-# by pooch from https://temporal.download/cli/archive/latest on+# ``TEMPORAL_SQLITE_PATH``. The official ``temporal`` CLI release pinned+# in server/config/temporal_cli.json is downloaded by pooch on# ``company build`` (or first ``company start`` if build was skipped).+# A database another server minor created is deleted at start.TEMPORAL_ENABLED=trueEMPLOYEE_TEAMS_ENABLED=trueEMPLOYEE_CAPABILITY_UPDATES_ENABLED=truediff --git a/RFC-0004-TEMPORAL-SDK-FEATURE-ADOPTION.md b/RFC-0004-TEMPORAL-SDK-FEATURE-ADOPTION.mdindex 0375fe97..7b2a73b5 100644--- a/RFC-0004-TEMPORAL-SDK-FEATURE-ADOPTION.md+++ b/RFC-0004-TEMPORAL-SDK-FEATURE-ADOPTION.md@@ -39,8 +39,8 @@ The RFC makes 18 decisions in five phases. Phase 0 fixes defects without changin| # | Decision | Status ||---|---|---|-| D1 | Pin the Temporal CLI per OpenCompany release (exact version and per-platform SHA-256, versioned path); record the server version beside each `temporal.db`; refuse a database created by a different server minor | Proposed |-| D2 | How an install moves an existing `temporal.db` to a newer server | Owner |+| D1 | Pin the Temporal CLI per OpenCompany release (exact version and per-platform SHA-256, versioned path); record the server version beside each `temporal.db`; delete a database created by a different server minor and start empty (owner, 2026-10-23; RFC-0005 E16) | Done (RFC-0005 Phase 1) |+| D2 | How an install moves an existing `temporal.db` to a newer server | Owner: it does not; the database is deleted (Q3) || D3 | Which server durable installs run on: pinned dev server with SQLite, PostgreSQL, or Temporal Cloud | Owner || D4 | Set namespace retention and the needed dynamic config explicitly instead of relying on dev-server defaults | Proposed || D5 | Upgrade temporalio 1.30.0 to 1.34.0, behind the replay gate of D10 | Proposed |@@ -277,7 +277,7 @@ Today no install changes server version by itself, because the cached binary is- Pin `version = 1.9.1` (server 1.32.0) and the per-platform SHA-256 from that release's `checksums.txt` in a committed file. The versioned URL is `https://temporal.download/assets/temporalio/cli/releases/download/v1.9.1/temporal_cli_1.9.1_<os>_<arch>.<zip|tar.gz>`; it returns the same bytes and ETag as the GitHub asset and `checksums.txt` sits beside it. This URL shape is observed behavior, not documented; the documented fallback is the GitHub release asset.- Download with `pooch.retrieve(known_hash="sha256:...")` into a versioned directory and file name, so an upgrade keeps the old binary for rollback.-- Write `temporal.db.server.json` (server version, CLI version, creation time) when the database is created, and refuse to start the dev server on a different server **minor**; patch releases of the same minor are allowed, so security fixes can ship.+- Write `temporal.db.server.json` (server version, CLI version, creation time) when the database is created, and refuse to start the dev server on a different server **minor**; patch releases of the same minor are allowed, so security fixes can ship. *Owner, 2026-10-23:* a different minor, or a database without the marker, deletes the database instead of refusing, and the install starts empty (RFC-0005 E16).- Apply the same pin to the Docker image (replace the copy-once volume step with a versioned path) and the desktop bundle.- Record the pin in `cli/` predeploy checks so CI downloads the same binary.@@ -653,7 +653,7 @@ The owner answered on 2026-10-16. Further answers came on 2026-10-23 while desig|---|---|---|---|---|| Q1 | Upgrade contract (D10): must runs that are open or paused survive an upgrade to the next release? | Always (N-1 replay gate) / Within a release line; major releases may require Reset / Any release may require Reset | Always, for N-1; a release that cannot honour it must say so and use D2 option B | Always, for N-1 (2026-10-16) || Q2 | Server for durable installs (D3) | Pinned dev server with SQLite everywhere / PostgreSQL for Docker and VM / also support Temporal Cloud | Pinned dev server on desktop; PostgreSQL for Docker and VM; Cloud later | 2026-10-16: pinned dev server with SQLite. Superseded on 2026-10-23 (RFC-0005 E1-E3): one instance per owner with free Temporal only, no Temporal Cloud. The dev server runs up to about 100 employees; above that, a hosted instance runs the open-source server on its own PostgreSQL. Hosted instances use PostgreSQL for the app database |-| Q3 | Moving an existing database (D2) | Keep its binary / recreate and re-arm / PostgreSQL | Recreate and re-arm on desktop, with notice | Keep the install's current server; moving is opt-in (2026-10-16). RFC-0005 §20 defines the dev-server-to-full-server move as a re-arm in the maintenance window |+| Q3 | Moving an existing database (D2) | Keep its binary / recreate and re-arm / PostgreSQL | Recreate and re-arm on desktop, with notice | Superseded on 2026-10-23 (RFC-0005 E16). Every install moves to the pinned server, and nothing is migrated: a database without a marker or from another server minor is deleted together with the old CLI, the install starts empty, and owners Reset and Start their employees. Before that, on 2026-10-16: keep the install's current server; moving is opt-in. RFC-0005 §20 defines the dev-server-to-full-server move as a re-arm in the maintenance window || Q4 | Failed runs (D12) | Temporal Failed / Completed plus `RunOutcome` | Temporal Failed | Temporal Failed (2026-10-16) || Q5 | Bugs in workflow code (D12) | Stuck and visible until a fix / fail fast | Stuck and visible | Open || Q6 | LLM step budget (D11) | A total budget per step (for example 30 min) / unlimited with alerts / unlimited for outages, capped for timeouts | A total budget, with longer attempts for local models | A total budget per step (2026-10-16) |@@ -673,7 +673,7 @@ The owner answered on 2026-10-16. Further answers came on 2026-10-23 while desig| Gate | Check ||---|---|| AG1 | The CLI download URL names an exact version and `known_hash` is set; a test fails on "latest" |-| AG2 | Starting with a database created by another server minor refuses with a clear message |+| AG2 | Starting with a database created by another server minor, or by no recorded server, deletes it and starts empty (owner, 2026-10-23) || AG3 | Retention is set to the configured value after namespace creation (integration test) || AG4 | A metrics endpoint or buffer exists; the alert table's signals are emitted in a test run || AG5 | `TemporalReportedProblems` turns a failing workflow task into "needs attention" in the status view |diff --git a/RFC-0005-EVENT-OS-ON-TEMPORAL.md b/RFC-0005-EVENT-OS-ON-TEMPORAL.mdindex b7f416a8..7879e713 100644--- a/RFC-0005-EVENT-OS-ON-TEMPORAL.md+++ b/RFC-0005-EVENT-OS-ON-TEMPORAL.md@@ -98,6 +98,7 @@ OpenCompany's AI employees should run like the processes of an operating system| E13 | **Fleet upgrades.** Releases reach all hosted instances together in a fixed weekly maintenance window. (§21) || E14 | **Telemetry.** Hosted instances report metrics and errors centrally, never content. (§18.5) || E15 | **The Linux model** draws on all three diagrams: the kernel map, the network stack and the storage stack. |+| E16 | **A new Temporal server starts from new.** The Temporal CLI is pinned (RFC-0004 D1). A database the pinned server minor did not create is deleted with the old CLI, and the install starts on an empty Temporal database. No migration code: generations whose controller is gone fail (`controller_execution_missing`), and owners Reset and Start their employees. The first pinned release does this on every install. (Decided 2026-10-23; §24.4) |### 3.2 RFC-0004 decisions this RFC implements@@ -843,7 +844,7 @@ A per-employee `employee_timeline` view joins ingress, deliveries, turns, outboxThe peaks are scaled from a 20-employee model: 2,600 turns/day, about 35k Temporal actions/day and 8-10 actions/s at peak. At tier L the owner's LLM provider limits bind before Temporal does, and the class floors decide who waits.**Tier S hardening.** The CLI documentation warns that the dev server "is not intended for production use" and skips some HTTP security checks. Tier S therefore:-- pins the CLI version and checksum (RFC-0004 D1);+- pins the CLI version and checksum (RFC-0004 D1), and deletes a database from another server minor rather than migrating it (E16);- persists to `--db-filename` with the WAL pragmas already set (`_runtime.py:153-158`);- binds to 127.0.0.1 only;- sets the namespace retention to 7 days;@@ -1073,6 +1074,18 @@ A marker becomes `deprecate_patch` one release after no open history can lack it| 0.8 Twitter receive | edf29a76 | Refused at Start and on a canvas Run rather than ported. A port needs paid X API access to test. || 0.9 Broadcast audience | 7745ca14 | `emit`'s `broadcast` has no default. No client read any channel or provider frame, so those are no longer broadcast, and no audience-scoped broadcast was needed. |+**Phase 1** (started 2026-10-24). The owner's decisions of 2026-10-23 set its order and scope:+- **Order:** pinned CLI, CI, fixture recorder, replay gate, golden identity test, then the temporalio upgrade from 1.30.0 to 1.34.0 (RFC-0004 D5).+- **Fresh start:** E16, with no migration code.+- **CI:** the whole server suite runs on Linux, Windows and macOS with the pinned CLI.+- **Fixtures:** a release-pipeline check fails a release that has no `fixtures/<version>/`.++Commit hashes are filled in when the phase ends.++| Step | Commit | Notes |+|---|---|---|+| 1.1 Pinned CLI | — | CLI v1.9.1 (server 1.32.0) is pinned in `server/config/temporal_cli.json` and verified with pooch's `known_hash`. It is installed under `packages/temporal/<version>/`, and every other CLI file there is deleted. `_install.prepare_database` deletes a `temporal.db` without the pinned server minor's marker. **Release note:** the first release with the pin starts every install on an empty Temporal database, so every employee needs Reset and Start. |+## 25. Defaults chosen and open items### 25.1 Defaults (change at review)diff --git a/docs-internal/SETUP.md b/docs-internal/SETUP.mdindex ff4e4b23..deb66f24 100644--- a/docs-internal/SETUP.md+++ b/docs-internal/SETUP.md@@ -155,7 +155,7 @@ the arrangement is [desktop_host_contract.md](./desktop_host_contract.md).### Temporal Server (Distributed Execution)- Provides durable workflow execution with per-node retry and horizontal scaling-- Official `temporal` CLI downloaded by `pooch` from `https://temporal.download/cli/archive/latest` on `company build` (or first backend boot with Temporal enabled)+- Official `temporal` CLI, the release pinned in `server/config/temporal_cli.json` and checked against its SHA-256, downloaded by `pooch` on `company build` (or first backend boot with Temporal enabled)- Backend-owned: the FastAPI lifespan starts `temporal server start-dev` via `TemporalServerRuntime.ensure_started()` when `TEMPORAL_ENABLED` and the configured address is loopback (external clusters are detected via TCP probe and left alone) — SQLite at `~/.opencompany/temporal.db`- Ports: gRPC `TEMPORAL_FRONTEND_GRPC_PORT`, Web UI `TEMPORAL_UI_PORT` (values in `.env.template`)- Embedded worker runs inside the Python backend (`TemporalWorkerManager` + `TemporalWorkerPool`, built by `services/temporal/lifecycle.py`; `main.py` only schedules `run_temporal_lifecycle`)diff --git a/docs-internal/TEMPORAL_ARCHITECTURE.md b/docs-internal/TEMPORAL_ARCHITECTURE.mdindex 8cea5bc5..a3d4e771 100644--- a/docs-internal/TEMPORAL_ARCHITECTURE.md+++ b/docs-internal/TEMPORAL_ARCHITECTURE.md@@ -953,8 +953,13 @@ The Temporal binary + persistence are managed in-process by the plugin-folder pa**What runs**: one process — `temporal server start-dev --port $TEMPORAL_FRONTEND_GRPC_PORT --ui-port $TEMPORAL_UI_PORT --db-filename ~/.opencompany/temporal.db --namespace default`. Both gRPC + Web UI bind to the same process (per docs.temporal.io/cli/server — "all running in a single process"). SQLite-backed durability; history persisted across restarts.+**A new server minor starts from an empty database** (RFC-0005 E16). `temporal.db.server.json` beside the database names the server that created it, and `_install.prepare_database` runs before every spawn. A database with no marker, or with one naming another server minor than the pin, is deleted together with its `-wal` and `-shm` files, so the pinned server starts empty; a patch release of the same minor keeps the database. Nothing is migrated:+- Every generation whose controller lived in the old database fails at boot with `controller_execution_missing`, and its owner resets and starts it, which clears its chat thread and Context.+- In-flight runs, Schedules, queued controller events and Temporal history are gone.+- The first release with the pin does this on every install, because no database before it has a marker. Its release notes must say so: it breaks the N-1 upgrade contract (RFC-0004 Q1) once, by the owner's choice of 2026-10-23.+**Modern libs doing the heavy lifting** (zero custom infrastructure code):-- **[`pooch`](https://pypi.org/project/pooch/)** — `services/temporal/_install.py` downloads the official `temporal` CLI archive from `https://temporal.download/cli/archive/latest?platform=<os>&arch=<arch>`. Cross-platform (Windows zip / macOS+Linux tar.gz), cached at `<DATA_DIR>/packages/temporal/` via `_cache_dir() = core.paths.package_dir("temporal")` (pooch's `path=` argument). The retrieve call passes an explicit `downloader=pooch.HTTPDownloader(timeout=300, progressbar=True)`: the timeout is per-socket-read (not total transfer), so arbitrarily slow links can finish the ~114 MB fetch — pooch's 30 s default aborted them — and `progressbar` must live on the downloader because `retrieve()` ignores its own kwarg when `downloader=` is explicit. Failed downloads never poison the cache (pooch writes to a temp file and renames atomically on success). Standalone entry: `python -m services.temporal._install` — invoked **fatally** by `company build` step [6/6] (so first `company start` doesn't pay the download; contract locked by `test_temporal_install_is_fatal_on_failure`) and **non-fatally** by npm postinstall (`scripts/install.js` try/catch — `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `company start`, so a failed eager fetch never fails `npm install -g`). The binary cache survives `company clean` (`packages` is in `_OPENCOMPANY_KEEP`, `cli/commands/clean.py`). Pre-fix this used `pooch.os_cache("opencompany-temporal")` (`~/.cache/OpenCompany/opencompany-temporal/` etc.) — a separate OS-cache namespace operators reported as "not local"; it now sits under DATA_DIR alongside the Stripe binary and the shared npm tree.+- **[`pooch`](https://pypi.org/project/pooch/)** — `services/temporal/_install.py` downloads the official `temporal` CLI release pinned in [`server/config/temporal_cli.json`](../server/config/temporal_cli.json): one exact version, its server version, and each platform's archive name and SHA-256 copied from the release's `checksums.txt` (RFC-0004 D1). pooch checks the archive against that hash before extracting it (`known_hash`), so a tampered or truncated download fails and a cached archive that stops matching is fetched again. Cross-platform (Windows zip / macOS+Linux tar.gz), cached at `<DATA_DIR>/packages/temporal/<version>/` via `_cache_dir() = core.paths.package_dir("temporal")` (pooch's `path=` argument); once the pinned binary is in place, every other file under `packages/temporal/` is deleted, including the unpinned `temporal_cli_latest_*` archives earlier releases fetched. The retrieve call passes an explicit `downloader=pooch.HTTPDownloader(timeout=300, progressbar=True)`: the timeout is per-socket-read (not total transfer), so arbitrarily slow links can finish the fetch — pooch's 30 s default aborted them — and `progressbar` must live on the downloader because `retrieve()` ignores its own kwarg when `downloader=` is explicit. Failed downloads never poison the cache (pooch writes to a temp file and renames atomically on success). Standalone entry: `python -m services.temporal._install` — invoked **fatally** by `company build` step [6/6] (so first `company start` doesn't pay the download; contract locked by `test_temporal_install_is_fatal_on_failure`) and **non-fatally** by npm postinstall (`scripts/install.js` try/catch — `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `company start`, so a failed eager fetch never fails `npm install -g`). The binary cache survives `company clean` (`packages` is in `_OPENCOMPANY_KEEP`, `cli/commands/clean.py`). Pre-fix this used `pooch.os_cache("opencompany-temporal")` (`~/.cache/OpenCompany/opencompany-temporal/` etc.) — a separate OS-cache namespace operators reported as "not local"; it now sits under DATA_DIR alongside the Stripe binary and the shared npm tree.- **`BaseProcessSupervisor` + `BaseSupervisor`** (`server/services/_supervisor/`) — the in-house supervisor base classes that `server/nodes/whatsapp/_runtime.py` also uses. Provides cross-platform signal handling (POSIX `setsid` + Windows Job Objects + `CREATE_NEW_PROCESS_GROUP` for graceful `CTRL_BREAK_EVENT` shutdown), restart policy via tenacity, log draining, status snapshots. We subclass both — zero custom supervisor logic.**Lifecycle wiring (July 2026)**: backend-owned, and modular — the whole Temporal runtime story lives in [`services/temporal/lifecycle.py`](../server/services/temporal/lifecycle.py) (`run_temporal_lifecycle`); `main.py` only schedules it as one background task. The module owns: (1) dev-server supervision via `TemporalServerRuntime.ensure_started()` before each connect attempt — a TCP probe on the gRPC port skips the spawn when a server (previously spawned or externally managed) is already listening, and non-loopback `TEMPORAL_SERVER_ADDRESS` values are never spawned at; (2) the forever-retrying connect loop; (3) the config-gated startup sweep; (4) executor + worker manager + worker pool wiring; (5) the boot-time workflow-control reconcile (`services.deployment.handlers.reconcile_active_controls_on_boot`); and (6) a **resident dev-server watchdog** — after startup the task stays alive, probing every `TEMPORAL_HEALTH_MONITOR_INTERVAL_SECONDS` and respawning a dev-server child that died (or restarting a supervisor-owned child that wedged: port bound but gRPC not SERVING for 4 consecutive probes). The runtime registers with `services._supervisor.register_supervisor` so `shutdown_all_supervisors()` actually stops it at teardown. The CLI no longer supervises Temporal: the `_temporal_specs.py` command helpers and the `_supervised_runtime.py` shim were deleted (the shim was a full second server-venv Python process, ~81 MB resident, plus a ~28 MB resident `uv run` parent — pure supervision overhead).@@ -986,7 +991,7 @@ guarantee applies to version 1 generations with their controller chain intact.**Port management**: Temporal owns `TEMPORAL_FRONTEND_GRPC_PORT` (gRPC) + `TEMPORAL_UI_PORT` (Web UI). Both bound by the same `temporal.exe` process; both listed in `cli.config.Config.all_ports` so `company stop`'s port-freeing pre-flight covers them.-**Direct CLI access**: the pooch-installed `temporal` binary lives under `<DATA_DIR>/packages/temporal/` (= `~/.opencompany/packages/temporal/` by default, on every OS). Run `temporal --version`, `temporal workflow list`, etc. directly from there.+**Direct CLI access**: the pooch-installed `temporal` binary lives under `<DATA_DIR>/packages/temporal/<version>/` (= `~/.opencompany/packages/temporal/<version>/` by default, on every OS), where `<version>` is the pin in `server/config/temporal_cli.json`. Run `temporal --version`, `temporal workflow list`, etc. directly from there.**Cluster tunables** — all sourced from `.env.template` (canonical defaults; no Python-side fallbacks). Settings fields with `Field(env="...")` require the env var to be present, surfaced via `cli.config.load_config()`'s `.env.template` → `.env` → `os.environ` merge.@@ -1072,8 +1077,8 @@ duplicate execution.# Temporal Web UIopen http://localhost:$TEMPORAL_UI_PORT-# List workflows via the local CLI binary (under <DATA_DIR>/packages/temporal/)-~/.opencompany/packages/temporal/.../temporal.exe workflow list --address localhost:$TEMPORAL_FRONTEND_GRPC_PORT+# List workflows via the local CLI binary (under <DATA_DIR>/packages/temporal/<version>/)+~/.opencompany/packages/temporal/<version>/.../temporal.exe workflow list --address localhost:$TEMPORAL_FRONTEND_GRPC_PORT# Re-fetch the binary at any timeuv run python -m services.temporal._installdiff --git a/docs-internal/cli_services_integration.md b/docs-internal/cli_services_integration.mdindex 8d0416db..a86a8cea 100644--- a/docs-internal/cli_services_integration.md+++ b/docs-internal/cli_services_integration.md@@ -15,7 +15,7 @@ OpenCompany integrates external services that manage their own lifecycle via CLIWraps the official `temporal` CLI's `server start-dev` mode (per [docs.temporal.io/develop/python/set-up-your-local-python](https://docs.temporal.io/develop/python/set-up-your-local-python)) — SQLite-backed dev server, single process, gRPC + Web UI both embedded.-**Install:** Automated. `company build` step [6/6] runs `python -m services.temporal._install`, which uses `pooch` to download the official CLI archive from `https://temporal.download/cli/archive/latest?platform=<os>&arch=<arch>` into `<DATA_DIR>/packages/temporal/` (= `~/.opencompany/packages/temporal/` by default, on every OS — via `core.paths.package_dir("temporal")`). No npm package, no system install required.+**Install:** Automated. `company build` step [6/6] runs `python -m services.temporal._install`, which uses `pooch` to download the CLI release pinned in `server/config/temporal_cli.json`, checked against that file's SHA-256, into `<DATA_DIR>/packages/temporal/<version>/` (= `~/.opencompany/packages/temporal/<version>/` by default, on every OS — via `core.paths.package_dir("temporal")`), and deletes every other CLI there. No npm package, no system install required. A `temporal.db` the pinned server minor did not create is deleted before the server starts (see [TEMPORAL_ARCHITECTURE.md](./TEMPORAL_ARCHITECTURE.md#server-management)).**Lifecycle:** Backend-owned (July 2026): the FastAPI lifespan starts the dev server via `TemporalServerRuntime.ensure_started()` ([server/services/temporal/_runtime.py](../server/services/temporal/_runtime.py)) when `TEMPORAL_ENABLED` and the configured address is loopback; `shutdown_all_supervisors()` stops it at shutdown. The CLI no longer supervises Temporal (the `_supervised_runtime.py` shim and the `_temporal_specs.py` command helpers were deleted); `company stop` still frees the Temporal ports, then kills any other Temporal process whose command line names this installation's data directory, where the binary and its SQLite database live (`cli/ports.py::kill_by_pattern`). Another checkout's or worktree's Temporal server, and a test run that merely mentions `temporal`, are left running. Its orphan cleanup works the same way: it takes the repository's worktrees from `git worktree list`, and a process working in another worktree is left alone even when it runs this checkout's Python.diff --git a/docs-internal/docker.md b/docs-internal/docker.mdindex d1f55532..239d62ec 100644--- a/docs-internal/docker.md+++ b/docs-internal/docker.md@@ -57,6 +57,8 @@ Everything the app keeps is in the `opencompany-selfhost_data` volume, mounted a`opencompany.env` holds `API_KEY_ENCRYPTION_KEY`, the key that decrypts `credentials.db`. Back the two up together and keep the file private. If the key changes, stored credentials can no longer be decrypted.+An image that pins a new Temporal server minor (`server/config/temporal_cli.json`) deletes `temporal.db` and the old CLI on its first start. Every deployment then shows Failed, and its owner resets and starts it (see [TEMPORAL_ARCHITECTURE.md](./TEMPORAL_ARCHITECTURE.md#server-management)).+Back up `/data` while the container is stopped. `temporal.db` runs in SQLite's WAL mode, so recent commits live in a `-wal` file next to it until SQLite folds them back in: a copy taken while the app runs needs the `-wal` and `-shm` files too, and a `-wal` file must never be deleted. SQLite needs working file locks, so keep `/data` on the named volume or a local disk, never NFS or SMB.## Configurationdiff --git a/docs-internal/errors.md b/docs-internal/errors.mdindex 87e64672..2ca1de7d 100644--- a/docs-internal/errors.md+++ b/docs-internal/errors.md@@ -268,7 +268,7 @@ FileNotFoundError: temporal binary not found at ...```or pooch fails to download during `company start`.-**Root cause**: The official `temporal` CLI archive is downloaded by `pooch` to `<DATA_DIR>/packages/temporal/` (= `~/.opencompany/packages/temporal/` by default, on every OS — via `core.paths.package_dir("temporal")`) during `company build` step [6/6]. If the build was skipped or interrupted, or if network access to `temporal.download` was blocked, the binary is missing.+**Root cause**: The official `temporal` CLI release pinned in `server/config/temporal_cli.json` is downloaded by `pooch` to `<DATA_DIR>/packages/temporal/<version>/` (= `~/.opencompany/packages/temporal/<version>/` by default, on every OS — via `core.paths.package_dir("temporal")`) during `company build` step [6/6]. If the build was skipped or interrupted, or if network access to `github.com` was blocked, the binary is missing. A download whose SHA-256 differs from the pin fails rather than extracting.**Fix**:```bash@@ -280,7 +280,9 @@ uv run python -m services.temporal._installcompany build```-If pooch keeps failing, check connectivity to `https://temporal.download/cli/archive/latest?platform=<os>&arch=<arch>` (the URL the docs document at https://docs.temporal.io/develop/python/set-up-your-local-python). Behind a corporate proxy, set `HTTPS_PROXY` in your environment before running.+If pooch keeps failing, check connectivity to the release URL in `server/config/temporal_cli.json` (`https://github.com/temporalio/cli/releases/download/v<version>/...`). Behind a corporate proxy, set `HTTPS_PROXY` in your environment before running.++**Every employee shows Failed after an upgrade** (`controller_execution_missing`). This is expected once, after an upgrade to a release that pins a new Temporal server minor. Before the dev server starts, the backend deletes a `temporal.db` the pinned server did not create (`services/temporal/_install.py::prepare_database`; RFC-0005 E16), so no controller survives. Reset and Start each employee. Reset clears its chat thread and Context, and nothing from the old Temporal database can be recovered.---diff --git a/docs-internal/release_build_pipeline.md b/docs-internal/release_build_pipeline.mdindex 82f6e2ea..c0132792 100644--- a/docs-internal/release_build_pipeline.md+++ b/docs-internal/release_build_pipeline.md@@ -163,7 +163,7 @@ The tarball still excludes `__pycache__/` per `package.json` `files` (cross-Pyth### 4b. Temporal binary fetch + DATA_DIR parity-- Step `[6/6]` runs `uv run python -m services.temporal._install`, which pooch-downloads the official `temporal` CLI into `<DATA_DIR>/packages/temporal/` (= `~/.opencompany/packages/temporal/` by default). Pre-fetching at build time turns the ~114 MB download into a sub-second cache hit on first `company start`. The download uses an explicit `HTTPDownloader(timeout=300)` (per-socket-read timeout — slow links can finish; pooch's 30 s default aborted them). Fatality differs by entry point: `company build` keeps the step **fatal** (locked by `test_temporal_install_is_fatal_on_failure`), while the provisioning script (`scripts/install.js`, run by `company provision`) wraps it in a **non-fatal** try/catch because `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `company start`. The cache survives `company clean` (`packages` ∈ `_OPENCOMPANY_KEEP`), so clean+build cycles don't re-download.+- Step `[6/6]` runs `uv run python -m services.temporal._install`, which pooch-downloads the `temporal` CLI release pinned in `server/config/temporal_cli.json` (checked against its SHA-256) into `<DATA_DIR>/packages/temporal/<version>/` (= `~/.opencompany/packages/temporal/<version>/` by default) and deletes any other CLI there. Pre-fetching at build time turns the download into a sub-second cache hit on first `company start`. The download uses an explicit `HTTPDownloader(timeout=300)` (per-socket-read timeout — slow links can finish; pooch's 30 s default aborted them). Fatality differs by entry point: `company build` keeps the step **fatal** (locked by `test_temporal_install_is_fatal_on_failure`), while the provisioning script (`scripts/install.js`, run by `company provision`) wraps it in a **non-fatal** try/catch because `TemporalServerRuntime._pre_spawn()` re-downloads lazily on first `company start`. The cache survives `company clean` (`packages` ∈ `_OPENCOMPANY_KEEP`), so clean+build cycles don't re-download.- **`company build` layers `.env.dev` first.** `build_command()` calls `cli.config.load_dev_overrides(root)` before the install steps, so the build's `DATA_DIR` matches what the runtime sees. Without it, a repo checkout's `company build` read `DATA_DIR=~/.opencompany` from `.env.template` and installed Temporal under user home, but `company dev` then read `DATA_DIR=.opencompany` from `.env.dev` and re-downloaded into `<repo>/.opencompany/` — a redundant ~114 MB fetch on every fresh clone.- **Safe for global installs.** `.env.dev` is git-committed for contributors but is NOT in the `files` list, so an installed (`bun add -g`) copy has no `.env.dev` — `load_dev_overrides` is a no-op and everything falls through to the `.env.template` default (`DATA_DIR=~/.opencompany`), matching `company start` / `company daemon`.diff --git a/scripts/install.js b/scripts/install.jsindex 63e5ecf9..2b3fcee6 100755--- a/scripts/install.js+++ b/scripts/install.js@@ -161,7 +161,7 @@ if (uvVersion) {// Temporal binary: downloaded eagerly during install (step [6/6])// by ``python -m services.temporal._install``, the same call that// ``company build`` already makes. The pooch cache-// (~/.cache/OpenCompany/...)+// (<DATA_DIR>/packages/temporal/<pinned version>/)// makes re-runs sub-second. Done eagerly because// ``TemporalServerRuntime._pre_spawn`` unconditionally calls// ``ensure_temporal_binaries`` -- the runtime always uses the@@ -268,8 +268,8 @@ try {}run(`uv pip install --python "${cliVenvPython}" --quiet -e .`, ROOT);- // Eagerly fetch the official Temporal CLI binary (~90 MB tarball- // from temporal.download/cli/archive/latest). Same call ``company+ // Eagerly fetch the official Temporal CLI binary (the release pinned+ // in server/config/temporal_cli.json). Same call ``company// build`` step [6/6] makes. The runtime supervisor unconditionally// uses this pooch-cached copy via// ``TemporalServerRuntime._pre_spawn`` -- it ignores any systemdiff --git a/server/config/temporal_cli.json b/server/config/temporal_cli.jsonnew file mode 100644index 00000000..d85d8be5--- /dev/null+++ b/server/config/temporal_cli.json@@ -0,0 +1,14 @@+{+ "_comment": "The Temporal CLI whose `server start-dev` runs every install's Temporal dev server (services/temporal/_install.py). Asset keys are <os>_<arch> in the CLI release's own spellings. Bump the version, server_version and every asset's file and sha256 together, copying them from the release's checksums.txt. A database created by another server minor is deleted at the next start, and the install begins on an empty Temporal database (RFC-0005 E16).",+ "version": "1.9.1",+ "server_version": "1.32.0",+ "base": "https://github.com/temporalio/cli/releases/download/v{version}/",+ "assets": {+ "linux_amd64": { "file": "temporal_cli_1.9.1_linux_amd64.tar.gz", "sha256": "09a0326a51db84d02735e53542b9ebd8c4758daf47482a9ab0abce15844e60d5" },+ "linux_arm64": { "file": "temporal_cli_1.9.1_linux_arm64.tar.gz", "sha256": "6c57c352d52fc3df34412376fd9ba6f74b7e3ace8e426e6cba8600156d36a145" },+ "darwin_amd64": { "file": "temporal_cli_1.9.1_darwin_amd64.tar.gz", "sha256": "48cdd6c84c56e27ae8e4d9a47052b89289a9c552dee029b49dd312c6fb72873f" },+ "darwin_arm64": { "file": "temporal_cli_1.9.1_darwin_arm64.tar.gz", "sha256": "41e0425378fcb4fb5766340b97435e20fe47bbff2d7bf644ec2d51f7662b7c56" },+ "windows_amd64": { "file": "temporal_cli_1.9.1_windows_amd64.zip", "sha256": "babb65844835045c91fb98930fe10b81de28db80c8c609180473d2c0b18c7589" },+ "windows_arm64": { "file": "temporal_cli_1.9.1_windows_arm64.zip", "sha256": "709dd23d856020b318676064ea0141adc35eae3716360265481b13e54988e12f" }+ }+}diff --git a/server/pyproject.toml b/server/pyproject.tomlindex 5892e68b..8cd0bbc0 100644--- a/server/pyproject.toml+++ b/server/pyproject.toml@@ -58,9 +58,9 @@ dependencies = ["pyyaml>=6.0",# Temporal CLI installer (services/temporal/_install.py).- # pooch: cross-platform binary download + cache (extracts the- # ``temporal`` CLI from the official URL at- # https://temporal.download/cli/archive/latest).+ # pooch: cross-platform binary download + cache (fetches the+ # ``temporal`` CLI release pinned in config/temporal_cli.json and+ # checks it against the pinned SHA-256)."pooch>=1.8.2",# Media metadata (services/media/inspect.py). Pure Python, zero nativediff --git a/server/services/temporal/_install.py b/server/services/temporal/_install.pyindex 03112fed..f3a73585 100644--- a/server/services/temporal/_install.py+++ b/server/services/temporal/_install.py@@ -1,35 +1,35 @@-"""Cross-platform Temporal CLI binary downloader using pooch.--Downloads the official ``temporal`` CLI from the URL documented at-https://docs.temporal.io/develop/python/set-up-your-local-python-(``temporal.download/cli/archive/latest?platform=<os>&arch=<arch>``).--The binary lands at ``<DATA_DIR>/packages/temporal/`` — the same root-that holds the Stripe and agent-browser binaries (see-:func:`core.paths.package_dir`). Legacy MachinaOS builds used-``pooch.os_cache("machinaos-temporal")`` which landed the binary-under a separate ``%LOCALAPPDATA%\\machinaos-temporal\\Cache\\``-(Windows) / ``~/.cache/machinaos-temporal/`` (Linux) tree outside-the operator-visible state root.--Pooch still drives archive extraction (zip on Windows, tar.gz-elsewhere); we just point its cache at our DATA_DIR-rooted package-dir. The ``latest`` URL rotates as new CLI versions ship, so the-download is unverified (TLS gives transport integrity); pooch caches-by local filename, not URL contents, so re-runs after the first-fetch are instant cache hits.+"""The pinned Temporal CLI, and the database its dev server may open.++Downloads the official ``temporal`` CLI release named in+``server/config/temporal_cli.json`` (version, server version and each+asset's SHA-256) into ``<DATA_DIR>/packages/temporal/<version>/`` — the+same root that holds the Stripe and agent-browser binaries (see+:func:`core.paths.package_dir`). pooch verifies the archive against the+pinned hash before extracting it (zip on Windows, tar.gz elsewhere), so+a re-run is a cache hit and a tampered or truncated download fails.+Once the pinned binary is in place, every other file under+``packages/temporal/`` is deleted: older pinned versions and the+unpinned ``temporal_cli_latest_*`` archives earlier releases fetched.++:func:`prepare_database` keeps the dev server off a database another+server created. A marker beside ``temporal.db`` names the server that+created it; a database without one, or created by another server minor,+is deleted and the install starts on an empty Temporal database+(RFC-0005 E16 chose a fresh start over migrating Temporal state).The downloaded ``temporal`` CLI powers ``temporal server start-dev``-(the SQLite/in-memory dev server) and ad-hoc workflow / operator-commands.+(the SQLite dev server) and ad-hoc workflow / operator commands."""from __future__ import annotationsimport asyncio+import jsonimport platform+import shutilimport statfrom pathlib import Path+from typing import Anyimport pooch@@ -38,14 +38,12 @@ from core.logging import get_loggerlogger = get_logger(__name__)-# Official Temporal CLI download URL. Per the docs at-# https://docs.temporal.io/develop/python/set-up-your-local-python the-# CLI archive lives at ``temporal.download/cli/archive/latest`` with-# ``platform`` and ``arch`` query parameters. Contains a single-# ``temporal`` binary.-_CLI_BASE_URL = "https://temporal.download/cli/archive/latest"+# The one place the CLI version, its server version and the per-platform+# archive hashes live. Bumping it is a JSON edit plus checksums.txt.+PIN_PATH = Path(__file__).resolve().parents[2] / "config" / "temporal_cli.json"+_CLI_PLATFORM_MAP: dict[tuple[str, str], tuple[str, str]] = {- # platform.system, platform.machine -> (URL platform, URL arch)+ # platform.system, platform.machine -> (release os, release arch)("Linux", "x86_64"): ("linux", "amd64"),("Linux", "aarch64"): ("linux", "arm64"),("Darwin", "x86_64"): ("darwin", "amd64"),@@ -54,14 +52,18 @@ _CLI_PLATFORM_MAP: dict[tuple[str, str], tuple[str, str]] = {("Windows", "ARM64"): ("windows", "arm64"),}++def cli_pin() -> dict[str, Any]:+ """The pinned release: ``version``, ``server_version``, ``base``, ``assets``."""+ return json.loads(PIN_PATH.read_text(encoding="utf-8"))++def _cache_dir() -> Path:- """Per-service install dir under :func:`core.paths.package_dir`.+ """Per-service install root under :func:`core.paths.package_dir`.- Lands at ``<DATA_DIR>/packages/temporal/`` — same root that- holds ``stripe`` and ``browser`` binaries. Legacy MachinaOS builds- used ``pooch.os_cache("machinaos-temporal")`` which landed the binary- at ``%LOCALAPPDATA%\\machinaos-temporal\\Cache\\`` (Windows) /- ``~/.cache/machinaos-temporal/`` (Linux), outside the data root.+ Lands at ``<DATA_DIR>/packages/temporal/`` — same root that holds+ ``stripe`` and ``browser`` binaries. Each pinned version gets its own+ subfolder."""from core.paths import package_dir@@ -70,6 +72,16 @@ def _cache_dir() -> Path:return p+def _binary_name() -> str:+ return "temporal.exe" if platform.system() == "Windows" else "temporal"+++def binary_installed() -> bool:+ """True when the pinned CLI is already on disk, so starting needs no download."""+ root = _cache_dir() / cli_pin()["version"]+ return root.is_dir() and any(p.is_file() for p in root.rglob(_binary_name()))++_cached: dict[str, Path] | None = None_lock = asyncio.Lock()@@ -77,18 +89,16 @@ _lock = asyncio.Lock()async def ensure_temporal_binaries(settings: Settings | None = None,) -> dict[str, Path]:- """Return ``{"temporal": Path}`` — the official ``temporal`` CLI binary.+ """Return ``{"temporal": Path}`` — the pinned ``temporal`` CLI binary.Idempotent — first call downloads, subsequent calls hit the pooch- cache (XDG / OS-conventional dir). Async-locked so concurrent- callers don't double-download.+ cache under ``<DATA_DIR>/packages/temporal/<version>/``.+ Async-locked so concurrent callers don't double-download.- The ``settings`` argument is accepted for API symmetry but currently- unused — the official ``latest`` URL has no version slot to override.+ The ``settings`` argument is accepted for API symmetry but unused —+ the version comes from ``server/config/temporal_cli.json``."""global _cached- # settings accepted for API symmetry; no fields consulted (the- # ``latest`` URL has no version slot)._ = settingsasync with _lock:@@ -104,29 +114,26 @@ async def ensure_temporal_binaries(def _fetch_cli_sync() -> Path:- """Download the official ``temporal`` CLI archive and return the binary path.+ """Download the pinned CLI archive and return the binary path.- Uses ``pooch.retrieve`` with ``known_hash=None`` because the- ``temporal.download/cli/archive/latest`` URL rotates as new CLI- versions ship — pinning a SHA would defeat the "latest" semantics- the official docs document. TLS gives us transport integrity.+ ``known_hash`` makes pooch check the archive against the pinned+ SHA-256: a mismatch raises instead of extracting, and a cached archive+ that no longer matches is downloaded again."""key = (platform.system(), platform.machine())if key not in _CLI_PLATFORM_MAP:raise RuntimeError(f"[Temporal install] Unsupported platform for CLI: {key}. " f"Supported: {sorted(_CLI_PLATFORM_MAP.keys())}")- url_platform, url_arch = _CLI_PLATFORM_MAP[key]- url = f"{_CLI_BASE_URL}?platform={url_platform}&arch={url_arch}"-- is_windows = platform.system() == "Windows"- fname = f"temporal_cli_latest_{url_platform}_{url_arch}.{'zip' if is_windows else 'tar.gz'}"- processor = pooch.Unzip() if is_windows else pooch.Untar()+ release_os, release_arch = _CLI_PLATFORM_MAP[key]+ pin = cli_pin()+ asset = pin["assets"][f"{release_os}_{release_arch}"]+ install_dir = _cache_dir() / pin["version"]extracted = pooch.retrieve(- url=url,- known_hash=None,- path=_cache_dir(),- fname=fname,- processor=processor,+ url=pin["base"].format(version=pin["version"]) + asset["file"],+ known_hash=f"sha256:{asset['sha256']}",+ path=install_dir,+ fname=asset["file"],+ processor=pooch.Unzip() if asset["file"].endswith(".zip") else pooch.Untar(),# requests' timeout is per-socket-read, not total download time —# only a 300s zero-byte stall aborts, so slow links can finish.# pooch's default is 30s, which killed npm installs on slow/WSL@@ -135,19 +142,86 @@ def _fetch_cli_sync() -> Path:downloader=pooch.HTTPDownloader(timeout=300, progressbar=True),)- target = "temporal.exe" if is_windows else "temporal"+ target = _binary_name()match = next((Path(p) for p in extracted if Path(p).name == target), None)if match is None:raise RuntimeError(f"[Temporal install] CLI binary {target!r} not found in archive. " f"Extracted files: {[Path(p).name for p in extracted]}")- if not is_windows:+ if platform.system() != "Windows":mode = match.stat().st_modematch.chmod(mode | stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH)+ _remove_other_clis(install_dir)return match-__all__ = ["ensure_temporal_binaries"]+def _remove_other_clis(install_dir: Path) -> None:+ """Delete every file under ``packages/temporal/`` but the pinned version's.++ A file still in use (a dev server of another version left running) is+ logged and kept; the next start tries again.+ """+ for entry in install_dir.parent.iterdir():+ if entry == install_dir:+ continue+ try:+ if entry.is_dir():+ shutil.rmtree(entry)+ else:+ entry.unlink()+ except OSError as exc:+ logger.warning("[Temporal install] could not delete an old CLI file", path=str(entry), error=str(exc))+ continue+ logger.info("[Temporal install] deleted an old CLI file", path=str(entry))+++def marker_path(db_path: Path) -> Path:+ """The file beside the database that names the server which created it."""+ return db_path.with_name(db_path.name + ".server.json")+++def prepare_database(db_path: Path) -> None:+ """Delete the dev server's database unless the pinned server minor created it.++ A database without a marker (every one from before the pin), or with+ one naming another server minor, is deleted together with its WAL+ files, and the marker is rewritten for the pinned server, which then+ creates an empty database. A patch release of the same minor keeps+ the database. Runs before the dev server starts, never while it holds+ the file.+ """+ pin = cli_pin()+ marker = marker_path(db_path)+ if _server_minor(_recorded_server(marker)) == _server_minor(pin["server_version"]):+ return+ for path in (db_path, Path(f"{db_path}-wal"), Path(f"{db_path}-shm")):+ if path.exists():+ path.unlink()+ logger.warning(+ "[Temporal install] deleted a database the pinned server did not create",+ path=str(path),+ server_version=pin["server_version"],+ )+ marker.write_text(+ json.dumps({"cli_version": pin["version"], "server_version": pin["server_version"]}),+ encoding="utf-8",+ )+++def _recorded_server(marker: Path) -> object:+ try:+ return json.loads(marker.read_text(encoding="utf-8"))["server_version"]+ except (OSError, ValueError, KeyError, TypeError):+ return None+++def _server_minor(version: object) -> str | None:+ if not isinstance(version, str):+ return None+ return ".".join(version.split(".")[:2])+++__all__ = ["binary_installed", "cli_pin", "ensure_temporal_binaries", "marker_path", "prepare_database"]def _main() -> int:diff --git a/server/services/temporal/_runtime.py b/server/services/temporal/_runtime.pyindex 8f1d6306..c5b8dd68 100644--- a/server/services/temporal/_runtime.py+++ b/server/services/temporal/_runtime.py@@ -1,10 +1,11 @@"""Temporal server lifecycle.Single supervisor subclass: :class:`TemporalServerRuntime`. Spawns the-official ``temporal`` CLI (downloaded by :mod:`services.temporal._install`-from https://temporal.download/cli/archive/latest) with the-``server start-dev`` subcommand against a SQLite db. Matches the local-dev install method documented at+official ``temporal`` CLI (the release pinned in+``server/config/temporal_cli.json``, downloaded by+:mod:`services.temporal._install`) with the ``server start-dev``+subcommand against a SQLite db. Matches the local dev install method+documented athttps://docs.temporal.io/develop/python/set-up-your-local-python.Uses the singleton accessor pattern (``Class.get_instance()``) from@@ -55,10 +56,11 @@ class TemporalServerRuntime(BaseProcessSupervisor):"""Temporal dev server, supervised via BaseProcessSupervisor.Spawns ``temporal server start-dev`` against a SQLite db at- ``settings.temporal_sqlite_path``. The ``temporal`` CLI binary is- downloaded by :func:`services.temporal._install.ensure_temporal_binaries`- from the official URL at- https://temporal.download/cli/archive/latest.+ ``settings.temporal_sqlite_path``. The pinned ``temporal`` CLI binary+ is downloaded by+ :func:`services.temporal._install.ensure_temporal_binaries`, and+ :func:`services.temporal._install.prepare_database` deletes a+ database the pinned server minor did not create before each spawn.Path is env-driven via ``settings.temporal_sqlite_path``, resolvedrelative to ``settings.data_dir`` unless already absolute.@@ -89,12 +91,16 @@ class TemporalServerRuntime(BaseProcessSupervisor):# ---- BaseProcessSupervisor overrides ---------------------------------async def _pre_spawn(self) -> None:- from services.temporal._install import ensure_temporal_binaries+ from services.temporal._install import ensure_temporal_binaries, prepare_databaseself._binaries = await ensure_temporal_binaries(self.settings)# Ensure the parent dir for the SQLite file exists before# ``temporal server start-dev`` opens it.self._sqlite_path.parent.mkdir(parents=True, exist_ok=True)+ # The server must never open a database another server minor+ # created: such a database is deleted here and the install starts+ # on an empty one (RFC-0005 E16).+ await asyncio.to_thread(prepare_database, self._sqlite_path)def binary_path(self) -> Path:# ``_pre_spawn`` (called by ``BaseProcessSupervisor._do_start``@@ -114,7 +120,9 @@ class TemporalServerRuntime(BaseProcessSupervisor):# paused deployments resume when the server# and workers come back; the debug-only# ``TEMPORAL_TERMINATE_RUNNING_ON_STARTUP``- # sweep (default false) is the one exception.+ # sweep (default false) is the one exception,+ # and a database from another server minor is+ # deleted in ``_pre_spawn`` instead of opened.# --metrics-port 0 disables the Prometheus endpoint# --log-level warn keeps the supervisor log readable# --namespace default namespace bootstrapped at startdiff --git a/server/services/temporal/lifecycle.py b/server/services/temporal/lifecycle.pyindex c53fbc59..772290db 100644--- a/server/services/temporal/lifecycle.py+++ b/server/services/temporal/lifecycle.py@@ -136,18 +136,16 @@ def _set_phase(app_state: Any, phase: str) -> None:def _temporal_binary_present() -> bool:- """True when the pooch-managed ``temporal`` CLI is already on disk.+ """True when the pinned ``temporal`` CLI is already on disk.- Distinguishes a first-run download (~114 MB, tens of seconds) from a- plain spawn so the readiness phase can say which one is happening.+ Distinguishes a first-run download (tens of seconds) from a plain+ spawn so the readiness phase can say which one is happening. A CLI of+ another version does not count: the pinned one is downloaded first."""try:- from core.paths import package_dir+ from services.temporal._install import binary_installed- root = package_dir("temporal")- if not root.is_dir():- return False- return any(p.is_file() and p.name.startswith("temporal") for p in root.rglob("temporal*"))+ return binary_installed()except Exception: # noqa: BLE001 — never let a probe break the connect loopreturn Truediff --git a/server/tests/temporal/test_cli_pin.py b/server/tests/temporal/test_cli_pin.pynew file mode 100644index 00000000..4061e5c6--- /dev/null+++ b/server/tests/temporal/test_cli_pin.py@@ -0,0 +1,179 @@+"""The pinned Temporal CLI and the database its dev server may open.++RFC-0004 D1 / AG1: the CLI comes from one exact release, checked against+a pinned SHA-256, never a "latest" URL. RFC-0005 E16: a database the+pinned server minor did not create is deleted with its WAL files, the+old CLI files are deleted, and the install starts on an empty Temporal+database.+"""++from __future__ import annotations++import asyncio+import json+import re+from pathlib import Path+from types import SimpleNamespace++from services.temporal import _install, _runtime+++def _db_files(db: Path) -> list[Path]:+ return [db, Path(f"{db}-wal"), Path(f"{db}-shm")]+++def _write_db(db: Path) -> None:+ for path in _db_files(db):+ path.write_bytes(b"temporal state")+++def _write_marker(db: Path, server_version: str) -> None:+ _install.marker_path(db).write_text(json.dumps({"server_version": server_version}), encoding="utf-8")+++def _same_minor_other_patch() -> str:+ major, minor, _patch = _install.cli_pin()["server_version"].split(".")+ return f"{major}.{minor}.99"+++def _other_minor() -> str:+ major, minor, _patch = _install.cli_pin()["server_version"].split(".")+ return f"{major}.{int(minor) - 1}.0"+++def test_pin_names_one_exact_release_with_a_hash_for_every_platform():+ pin = _install.cli_pin()++ assert "latest" not in _install.PIN_PATH.read_text(encoding="utf-8")+ assert re.fullmatch(r"\d+\.\d+\.\d+", pin["version"])+ assert re.fullmatch(r"\d+\.\d+\.\d+", pin["server_version"])+ assert "{version}" in pin["base"]+ assert set(pin["assets"]) == {f"{os_}_{arch}" for os_, arch in _install._CLI_PLATFORM_MAP.values()}+ for key, asset in pin["assets"].items():+ extension = "zip" if key.startswith("windows_") else "tar.gz"+ assert asset["file"] == f"temporal_cli_{pin['version']}_{key}.{extension}"+ assert re.fullmatch(r"[0-9a-f]{64}", asset["sha256"])+++def test_fetch_verifies_the_pinned_archive_and_deletes_every_other_cli(tmp_path, monkeypatch):+ monkeypatch.setattr(_install, "_cache_dir", lambda: tmp_path)+ monkeypatch.setattr(_install.platform, "system", lambda: "Linux")+ monkeypatch.setattr(_install.platform, "machine", lambda: "x86_64")+ (tmp_path / "temporal_cli_latest_linux_amd64.tar.gz").write_bytes(b"unpinned archive")+ (tmp_path / "temporal_cli_latest_linux_amd64.tar.gz.untar").mkdir()+ (tmp_path / "temporal_cli_latest_linux_amd64.tar.gz.untar" / "temporal").write_bytes(b"unpinned")+ (tmp_path / "1.0.0").mkdir()+ calls = []++ def fake_retrieve(**kwargs):+ calls.append(kwargs)+ extracted = Path(kwargs["path"]) / f"{kwargs['fname']}.untar"+ extracted.mkdir(parents=True)+ binary = extracted / "temporal"+ binary.write_bytes(b"pinned")+ return [str(extracted / "LICENSE"), str(binary)]++ monkeypatch.setattr(_install.pooch, "retrieve", fake_retrieve)++ binary = _install._fetch_cli_sync()++ pin = _install.cli_pin()+ asset = pin["assets"]["linux_amd64"]+ assert len(calls) == 1+ assert calls[0]["url"].endswith(f"/v{pin['version']}/{asset['file']}")+ assert "latest" not in calls[0]["url"]+ assert calls[0]["known_hash"] == f"sha256:{asset['sha256']}"+ assert Path(calls[0]["path"]) == tmp_path / pin["version"]+ assert calls[0]["fname"] == asset["file"]+ assert binary == tmp_path / pin["version"] / f"{asset['file']}.untar" / "temporal"+ assert [entry.name for entry in tmp_path.iterdir()] == [pin["version"]]+++def test_only_the_pinned_cli_counts_as_installed(tmp_path, monkeypatch):+ monkeypatch.setattr(_install, "_cache_dir", lambda: tmp_path)+ unpinned = tmp_path / "temporal_cli_latest_windows_amd64.zip.unzip"+ unpinned.mkdir()+ (unpinned / _install._binary_name()).write_bytes(b"unpinned")++ assert _install.binary_installed() is False++ pinned = tmp_path / _install.cli_pin()["version"] / "archive.unzip"+ pinned.mkdir(parents=True)+ (pinned / _install._binary_name()).write_bytes(b"pinned")++ assert _install.binary_installed() is True+++def test_a_new_install_records_the_pinned_server_beside_its_database(tmp_path):+ db = tmp_path / "temporal.db"++ _install.prepare_database(db)++ pin = _install.cli_pin()+ assert not db.exists()+ assert json.loads(_install.marker_path(db).read_text(encoding="utf-8")) == {+ "cli_version": pin["version"],+ "server_version": pin["server_version"],+ }+++def test_a_database_from_before_the_pin_is_deleted(tmp_path):+ db = tmp_path / "temporal.db"+ _write_db(db)++ _install.prepare_database(db)++ assert not any(path.exists() for path in _db_files(db))+ assert json.loads(_install.marker_path(db).read_text(encoding="utf-8"))["server_version"] == _install.cli_pin()["server_version"]+++def test_a_database_from_another_server_minor_is_deleted(tmp_path):+ db = tmp_path / "temporal.db"+ _write_db(db)+ _write_marker(db, _other_minor())++ _install.prepare_database(db)++ assert not any(path.exists() for path in _db_files(db))+ assert json.loads(_install.marker_path(db).read_text(encoding="utf-8"))["server_version"] == _install.cli_pin()["server_version"]+++def test_a_patch_release_of_the_same_minor_keeps_the_database(tmp_path):+ db = tmp_path / "temporal.db"+ _write_db(db)+ _write_marker(db, _same_minor_other_patch())++ _install.prepare_database(db)++ assert all(path.read_bytes() == b"temporal state" for path in _db_files(db))+++def test_an_unreadable_marker_cannot_keep_a_database(tmp_path):+ db = tmp_path / "temporal.db"+ _write_db(db)+ _install.marker_path(db).write_text("not json", encoding="utf-8")++ _install.prepare_database(db)++ assert not db.exists()+++def test_the_dev_server_prepares_its_database_before_it_spawns(tmp_path, monkeypatch):+ settings = SimpleNamespace(+ temporal_graceful_shutdown_seconds=5,+ temporal_sqlite_path="data/temporal.db",+ _resolve_under_data=lambda path: str(tmp_path / path),+ )+ runtime = _runtime.TemporalServerRuntime(settings=settings)+ prepared = []++ async def fake_binaries(_settings):+ return {"temporal": tmp_path / "temporal"}++ monkeypatch.setattr(_install, "ensure_temporal_binaries", fake_binaries)+ monkeypatch.setattr(_install, "prepare_database", prepared.append)++ asyncio.run(runtime._pre_spawn())++ assert prepared == [tmp_path / "data" / "temporal.db"]+ assert (tmp_path / "data").is_dir()