Self-improving AI Employees for everyone
Python ai-agentsai-employeeai-employeesassistantautonomous-agentsco-employeescompanygrok-botinstinctloop-agentsmuseown-your-datapersonalself-hosted
ci(desktop-release): draft the release from the tag message, guard the version, write down the procedure
trohitg03bdd860de7d
diff --git a/.github/workflows/desktop-release.yml b/.github/workflows/desktop-release.ymlindex 1c06ff82..9798568c 100644--- a/.github/workflows/desktop-release.yml+++ b/.github/workflows/desktop-release.yml@@ -8,10 +8,12 @@# APPLE_ID / APPLE_APP_SPECIFIC_PASSWORD / APPLE_TEAM_ID + a Developer ID# cert for macOS) plus flipping the mac flags in desktop/electron-builder.yml.#-# Flow: build the client + sidecar with the repo toolchain -> stage the-# app-root and fetch the pinned runtimes for the job's target -> invariant-# test -> electron-builder publishes into one DRAFT release -> the finalize-# job publishes the draft once every matrix leg is green.+# Flow: prepare drafts the GitHub Release from the annotated tag's message+# (first line = title, rest = notes) -> each matrix leg builds the client ++# sidecar with the repo toolchain, checks the synced version against the tag,+# stages the app-root with the pinned runtimes for its target, runs the+# invariant test and lets electron-builder upload into that one draft -> the+# finalize job publishes the draft once every leg is green.name: Desktop release@@ -34,8 +36,42 @@ concurrency:cancel-in-progress: falsejobs:+ prepare:+ name: Draft release+ runs-on: ubuntu-latest+ steps:+ # Drafted once, here, so the three legs upload into one release:+ # electron-builder finds an existing draft by tag and only adds assets,+ # whereas three legs racing to create it is how duplicate drafts+ # happen, and a draft electron-builder creates carries no notes. The+ # notes are the annotated tag's message, the text `git tag -a`+ # recorded; a lightweight tag falls back to GitHub's generated notes.+ # Skipped on manual dispatch (no tag): electron-builder drafts as before.+ - name: Draft the release from the tag message+ if: github.event_name == 'push'+ env:+ GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}+ TAG: ${{ github.ref_name }}+ REPO: ${{ github.repository }}+ run: |+ if gh release view "$TAG" --repo "$REPO" >/dev/null 2>&1; then+ echo "release $TAG already exists; keeping it"+ exit 0+ fi+ if [ "$(gh api "repos/$REPO/git/ref/tags/$TAG" --jq .object.type)" = tag ]; then+ sha=$(gh api "repos/$REPO/git/ref/tags/$TAG" --jq .object.sha)+ gh api "repos/$REPO/git/tags/$sha" --jq .message > tag-message.txt+ title=$(head -n 1 tag-message.txt)+ tail -n +2 tag-message.txt | sed '1{/^$/d;}' > notes.md+ gh release create "$TAG" --repo "$REPO" --draft --verify-tag --title "$title" --notes-file notes.md+ else+ echo "$TAG is a lightweight tag; drafting with generated notes"+ gh release create "$TAG" --repo "$REPO" --draft --verify-tag --title "OpenCompany $TAG" --generate-notes+ fi+build:name: ${{ matrix.name }}+ needs: prepareruns-on: ${{ matrix.os }}strategy:fail-fast: false@@ -80,10 +116,30 @@ jobs:path: desktop/vendorkey: desktop-runtimes-${{ runner.os }}-${{ hashFiles('desktop/runtimes.json') }}- - name: Sync version, stage bundle, render icon+ - name: Sync version+ working-directory: desktop+ run: bun run sync-version++ # electron-builder uploads into the draft for `v<desktop version>`. A+ # release commit that forgot to bump the root package.json would ship+ # installers and an updater feed under the previous version and open a+ # second draft for that tag; fail here instead.+ - name: Version must match the tag+ if: github.event_name == 'push'+ working-directory: desktop+ shell: bash+ env:+ TAG: ${{ github.ref_name }}+ run: |+ version=$(bun -e "console.log(require('./package.json').version)")+ if [ "v$version" != "$TAG" ]; then+ echo "desktop/package.json is $version but the tag is $TAG: the release commit must run 'company version sync $TAG'" >&2+ exit 1+ fi++ - name: Stage bundle, render iconworking-directory: desktoprun: |- bun run sync-versionbun run stage -- --target ${{ matrix.targets }}bun run gen-iconsdiff --git a/CLAUDE.md b/CLAUDE.mdindex 027309ba..17b4beed 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -44,7 +44,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire| **[Translate / Transliterate / Detect](./docs-internal/speech_provider_rfc.md#8-what-this-pattern-should-absorb-next)** | The second application of the multi-vendor pattern, and the one that retired `nodes/sarvam/`. Three nodes (`translateText`, `transliterateText`, `detectLanguage`) in [`server/nodes/translate/`](./server/nodes/translate/), backed by **three registries — one per capability**, because the asymmetry is sharper than speech's: DeepL translates and does nothing else, while Sarvam and LLM-backed providers do all three. Membership makes an unsupported selection unrepresentable rather than a runtime failure inside a paid call. Providers: **deepl** (`Authorization: DeepL-Auth-Key`, `text` is an array, free keys ending `:fx` route to a different host, and it returns `billed_characters` so cost is the provider's own figure), **sarvam** (all three over dedicated REST), **openai** (all three via `ChatUnifier` — proves the Protocol does not care what a provider does internally; billing deliberately NOT recorded because that path bills tokens the LLM layer already costs). Capabilities in `server/config/translate_defaults.json`. || **[Desktop App](./docs-internal/desktop_app.md)** | Electron shell in [`desktop/`](./desktop/) — a standalone bun package (NOT a root workspace member; `cli/tests/test_release_pipeline_config.py` locks `workspaces`). Bundles pinned `uv`, python-build-standalone CPython 3.12 and bun 1.4.0 — the only JavaScript runtime shipped; no Node, no npm (`desktop/runtimes.json`, verified against upstream checksum manifests by `scripts/fetch-runtimes.ts`, which also fetches bun's `LICENSE.md` from the bun repo — the release zip carries none — and prunes stale runtime dirs such as a leftover `node/`), stages the backend sibling layout from `bun pm pack --dry-run` (`scripts/stage.ts`, which runs the sidecar package's own `bun run build` — `bun build --target=bun`, express inlined — and copies `dist/index.js`), sets `OPENCOMPANY_BUN_BIN` / `BUN_INSTALL=<userData>/bun` / `BUN_INSTALL_CACHE_DIR` and prepends the bundled bun dir (then uv) to PATH for the backend (`env.ts`; `OPENCOMPANY_NODE_BIN` no longer exists), provisions the venv into Electron `userData` on first launch with `uv sync --frozen --python <bundled>` (stamp = sha256 of uv.lock + pyproject + Python + uv versions; falls back to `uv python install 3.12`), pre-warms `PYTHONPYCACHEPREFIX` (with a prefix set Python ignores in-tree .pyc — 32 s vs 3 s first boot), spawns uvicorn on `127.0.0.1:5678` (persisted port; attaches to a running `company serve` instead of double-spawning), waits on `/health/ready`, loads the backend-served SPA (zero frontend changes), routes `window.open` / `_blank` to the OS browser. Quit = `POST /api/desktop/shutdown` + stdin EOF, 30 s, then tree-kill. electron-builder: NSIS x64, DMG+zip arm64/x64, AppImage+deb x64, unsigned first (`mac.identity: null`; macOS auto-update notify-only until signed — Squirrel.Mac refuses unsigned). CI: `desktop-release.yml` (tag-triggered draft release + finalize) and `desktop-ci.yml` (typecheck/unit/invariants/Playwright smoke). **Gotcha: editor-hosted terminals export `ELECTRON_RUN_AS_NODE=1`, which makes Electron boot as plain Node (`electron.app` undefined) — strip it before launching; the e2e test does.** || **[Desktop Host Contract](./docs-internal/desktop_host_contract.md)** | The backend's side of being owned by a GUI shell. `core/approot.py` is the ONE module that locates the app tree (`OPENCOMPANY_APP_ROOT` / `OPENCOMPANY_CLIENT_DIST` / `OPENCOMPANY_ENV_FILE` / `OPENCOMPANY_ENV_TEMPLATE`); `tests/test_no_root_climb.py` forbids any other `Path(__file__).parents[N]` climb above `server/` and any `"client"/"dist"` / `.env.template` / `package.json` path join. `core/env_defaults.apply_file_defaults_to_environ()` does the CLI's `.env.template < .env < process env` layering at the top of `main.py`, and `Settings.env_file` is the absolute `(template, .env)` tuple — so a shell spawns `python -m uvicorn` from any cwd without the CLI package. Desktop mode (`OPENCOMPANY_DESKTOP=1`, `core/desktop.py`): parent-PID watchdog with create-time PID-reuse guard, stdin-EOF watchdog (polling via `PeekNamedPipe`/`select`, never a blocking read — a daemon thread parked in `ReadFile` at finalisation crashed exit with 0xC0000005), ctypes Job Object so children die with the backend (no pywin32 in the server venv), token-gated `POST /api/desktop/shutdown` (`routers/desktop.py`, public path; raises the signal uvicorn handles so the lifespan reaps Temporal/the bun sidecar/edgymeow), 45 s hard deadline. `GET /health/ready` (public) = DB up AND (Temporal disabled OR worker manager started), with `phase` written by `services/temporal/lifecycle.py` (`installing_temporal` / `starting_temporal` / `connecting` / `starting_workers` / `ready`). Plugin shutdown hooks now run under a 10 s per-hook timeout (`HOOK_TIMEOUT_SECONDS`), and the browser hook reads the module singleton instead of `get_browser_service()`, which lazily ran the `agent-browser` package install (`npm install` at the time; `bun add` via `core/js_runtime.add_package` today) AT SHUTDOWN and wedged every graceful exit (masked under the CLI by its 5 s grace + tree-kill). `server/uv.lock` is committed; `predeploy.yml` runs `uv lock --check`. |-| **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and the composite setup action (bun via SHA-pinned `oven-sh/setup-bun` reading the root `packageManager` pin + Node 22 as a CI-only convenience so bun runs vite / vitest / eslint on it via their node shebangs + Python 3.12 + uv; `bun install --frozen-lockfile` everywhere; publish is `bun publish --access public` after a `bun pm whoami` credential preflight with `~/.npmrc` written from `NPM_TOKEN` — no npm CLI, no `setup-node` in the publish jobs, and npm provenance attestation went away with npm) |+| **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and the composite setup action (bun via SHA-pinned `oven-sh/setup-bun` reading the root `packageManager` pin + Node 22 as a CI-only convenience so bun runs vite / vitest / eslint on it via their node shebangs + Python 3.12 + uv; `bun install --frozen-lockfile` everywhere; publish is `bun publish --access public` after a `bun pm whoami` credential preflight with `~/.npmrc` written from `NPM_TOKEN` — no npm CLI, no `setup-node` in the publish jobs, and npm provenance attestation went away with npm). Also **Cutting a release**: the annotated tag is the version and its message is the release notes; `company version sync vX.Y.Z` bumps every version file for the `chore(release)` commit the tag sits on, and `desktop-release.yml` drafts the GitHub Release from the tag message and refuses a version that differs from the tag || **[Performance](./docs-internal/performance.md)** | Cold-start measurements (Application startup complete 2.90 s warm, 21.5 s cold post-`company clean` — was 71 s before the 2026-07-14 boot-delay fixes; first WS connect 8.29 s) + warm and cold per-phase timelines + optimisation history (lazy LangChain imports, lazy LLM-SDK exception refs, pre-bundled sidecar (esbuild at the time, `bun build --target=bun` today), bytecode-at-build, PartySocket reconnect, TanStack Query auth bootstrap, `manualChunks`, Vite dep-cache preservation, off-loop Temporal build-id hash) + bottleneck inventory + reproduction commands + anti-patterns to never reintroduce (incl. eager SDK import at LLM provider registration, unconditional `.vite` wipe, synchronous `Worker()` on the event loop). Pairs with the build-time pipeline doc below. || **[Release Build Pipeline](./docs-internal/release_build_pipeline.md)** | Build pipeline for the registry tarball (installed with `bun add -g`; bun is the only JavaScript runtime and package manager anything shipped needs): `typescript@7` (native Go compiler, exact-pinned in **root** devDependencies — client keeps `typescript@^5.x` because typescript-eslint's peer range excludes 6/7, and one manifest cannot declare `typescript` twice) for type-check, measured 5.9× faster than tsc 5.9.3 (820 ms vs 4830 ms); `bun run --filter react-flow-client typecheck` delegates up to the root gate, Vite `manualChunks` + `target: 'es2022'`, `bun build --target=bun` sidecar bundle (self-contained `dist/index.js` with express inlined, run as `bun dist/index.js`; drops the `tsx` interpreter cost and needs no `node_modules` at runtime), bytecode pre-compile in two halves — `[tool.uv] compile-bytecode = true` (uv sync compiles `.venv/` site-packages) + `python -m compileall` for project source (plain `.pyc`, **no `-O`** — per PEP 488 the non-`-O` runtimes never load `.opt-1.pyc`; excludes `.venv/`, `tests/`). Single source of truth for the bytecode-compile path list: `cli.commands.build.COMPILEALL_SOURCE_DIRS`. Dev PM is **[email protected]** (tracked text `bun.lock`, `bunfig.toml` pins the isolated linker, ranged security pins in top-level `overrides`); `company build` step [1/6] is `bun install`. || **[Workflow Schema](./docs-internal/workflow-schema.md)** | JSON schema for workflows, edge handle conventions, config node architecture |diff --git a/cli/tests/test_release_pipeline_config.py b/cli/tests/test_release_pipeline_config.pyindex e63dfa47..802f9ffa 100644--- a/cli/tests/test_release_pipeline_config.py+++ b/cli/tests/test_release_pipeline_config.py@@ -699,3 +699,57 @@ def test_install_js_compileall_is_non_fatal(install_js_src: str):"compileall call in install.js must be wrapped in a try/catch ""with a Warning log so it stays non-fatal")+++# ---------------------------------------------------------------------------+# Desktop release workflow — one draft per tag, drafted from the tag message+# ---------------------------------------------------------------------------++[email protected](scope="module")+def desktop_release_yml(root: Path) -> dict:+ path = root / ".github" / "workflows" / "desktop-release.yml"+ return yaml.safe_load(path.read_text(encoding="utf-8"))+++def test_desktop_release_drafts_from_the_tag_before_the_matrix_builds(+ desktop_release_yml: dict,+):+ """``prepare`` creates the one draft the three legs upload into, titled+ and annotated from the tag message; ``finalize`` only undrafts it."""+ jobs = desktop_release_yml["jobs"]+ assert jobs["build"]["needs"] == "prepare"+ assert jobs["finalize"]["needs"] == "build"++ draft = next(+ step for step in jobs["prepare"]["steps"] if "gh release create" in step.get("run", "")+ )+ assert draft["if"] == "github.event_name == 'push'"+ for flag in ("--draft", "--verify-tag", "--notes-file"):+ assert flag in draft["run"]+ assert "git/tags/" in draft["run"] # the notes are the tag object's message+ assert "gh release view" in draft["run"] # idempotent on a workflow re-run++ undraft = next(+ step for step in jobs["finalize"]["steps"] if "gh release edit" in step.get("run", "")+ )+ assert "--draft=false" in undraft["run"]+++def test_desktop_release_refuses_a_version_that_does_not_match_the_tag(+ desktop_release_yml: dict,+):+ """The tag-triggered desktop build ships the committed desktop version+ (``bun run sync-version`` copies the root package.json; nothing there+ reads the tag), so a forgotten release bump must fail before any upload."""+ steps = desktop_release_yml["jobs"]["build"]["steps"]+ names = [step.get("name", "") for step in steps]+ sync_index = names.index("Sync version")+ guard_index = names.index("Version must match the tag")+ dist_index = next(i for i, step in enumerate(steps) if "bun run dist" in step.get("run", ""))+ assert sync_index < guard_index < dist_index++ guard = steps[guard_index]+ assert guard["if"] == "github.event_name == 'push'"+ assert guard["shell"] == "bash"+ assert "exit 1" in guard["run"]diff --git a/docs-internal/ci_cd.md b/docs-internal/ci_cd.mdindex 4320832e..d8df29cd 100644--- a/docs-internal/ci_cd.md+++ b/docs-internal/ci_cd.md@@ -40,7 +40,7 @@ The repo currently ships **three workflows** plus one composite action. A number| Predeploy | `.github/workflows/predeploy.yml` | `workflow_call` | build/lint + backend tests + CLI tests + cross-OS build/start smoke || Release | `.github/workflows/release.yml` | `v*.*.*` tag, `workflow_dispatch` | predeploy gate → publish (npm + GitHub Packages) || Desktop CI | `.github/workflows/desktop-ci.yml` | PR touching `desktop/**` or the backend host-contract files; push to main touching `desktop/**` | typecheck + unit + staged-tree invariants + `electron-vite build` + Playwright Electron smoke on ubuntu-22.04 (xvfb) and windows-latest |-| Desktop release | `.github/workflows/desktop-release.yml` | `v*.*.*` tag, `workflow_dispatch` | 3-runner matrix (windows-latest x64, macos-14 arm64+x64, ubuntu-22.04 x64) builds installers with electron-builder into one **draft** GitHub Release; `finalize` undrafts once every leg is green. Artifacts-only on dispatch unless `publish` is ticked |+| Desktop release | `.github/workflows/desktop-release.yml` | `v*.*.*` tag, `workflow_dispatch` | `prepare` drafts the GitHub Release from the annotated tag's message (first line = title, rest = notes); a 3-runner matrix (windows-latest x64, macos-14 arm64+x64, ubuntu-22.04 x64) checks the synced version against the tag, builds installers with electron-builder and uploads into that one **draft**; `finalize` undrafts once every leg is green. Artifacts-only on dispatch unless `publish` is ticked || Setup | `.github/actions/setup/action.yml` | (composite) | bun 1.4 + Node 22 (CI-only: bun runs vite / vitest / eslint on it) + Python 3.12 + uv v8 + editable CLI install |### Toolchain pin@@ -132,7 +132,49 @@ publish-npm publish-github-packagesBoth registries use `@zeenie-ai/opencompany`, matching the npm and GitHuborganization owned by the project.-There is currently no audit gate, no PyPI publish, no SLSA `attest-build-provenance` step, and no `create-github-release` / test-install stage — see [Planned](#planned-not-yet-implemented).+There is currently no audit gate, no PyPI publish, no SLSA `attest-build-provenance` step, no GitHub Release step in `release.yml` itself (the desktop workflow drafts the release from the tag message), and no test-install stage — see [Planned](#planned-not-yet-implemented).++---++## Cutting a release++The tag is the version. Everything else is derived from it, and there is no+CHANGELOG file: the annotated tag's message is the release notes (first line =+release title, the rest = body), which `desktop-release.yml` copies onto the+GitHub Release. Nothing publishes until a `v*.*.*` tag is pushed. Tag push is+also the only trigger that works: a `workflow_dispatch` run has a shallow+checkout with no tags for `company version sync` to read.++1. **Pre-flight.** `main` green on CI. `NPM_TOKEN` unexpired: the+ `bun pm whoami` preflight fails only the npmjs job, after the predeploy+ gate, while GitHub Packages and the installers still publish, so a lapsed+ token means a partial release (rotate the secret, then+ `gh run rerun <id> --failed`). `bun publish --dry-run --access public`+ packs cleanly from the checkout.+2. **Bump.** `python -m cli version sync vX.Y.Z` writes the version into the+ root, client and desktop `package.json`, `pyproject.toml` and+ `cli/__init__.py` (never `server/pyproject.toml`, which `server/uv.lock`+ records). Commit as `chore(release): vX.Y.Z`. This commit is load-bearing:+ the desktop workflow ships the committed desktop version and refuses to+ build when it differs from the tag, while the registry jobs re-run+ `version sync` from the tag themselves. `cli/tests/test_version.py`+ fails when the checked-in files disagree.+3. **Tag.** `git tag -a vX.Y.Z -F notes.md` on that commit, the notes in the+ shape of the previous tags (`git tag -l --format='%(contents)' v0.1.0`).+4. **Push** the branch, then the tag. The tag starts `release.yml` (predeploy+ gate, then npmjs + GitHub Packages) and `desktop-release.yml` (draft with+ the notes, three installer legs, undraft) in parallel.+5. **Verify.** `gh run watch`; the registry's `latest` dist-tag; the release+ page shows the notes with the installers and the three `latest*.yml`+ updater feeds; on a clean machine the install script finishes through+ `company provision`.++Semver in 0.x: a change that breaks the install or upgrade path bumps the+minor (0.1 -> 0.2), everything else the patch. There is no un-publish:+`npm deprecate` has no bun equivalent, so a bad release is followed by a+patch release. If one desktop leg fails, the release stays a draft holding+the notes and the successful legs' assets; re-run the failed job and+`finalize` undrafts it.---@@ -209,7 +251,7 @@ conditions`.The following existed in earlier drafts of this doc but are **not present in the current repo**. Listed here so the intent is preserved without misrepresenting the shipped pipeline:- **`predeploy.yml` change-detection + aggregator** — a `plan` job (`dorny/paths-filter`) gating downstream jobs, a `pre-commit` job, pytest sharding by domain, and a `ci-passed` aggregator (`re-actors/alls-green`) as the single branch-protection target. Today every predeploy job runs unconditionally and there is no aggregator job.-- **`release.yml` hardening** — `workflow_dispatch` dry-run default, a once-per-release `build-for-publish` artifact, `actions/attest-build-provenance` SLSA attestations, and a `create-github-release` job. (An earlier draft also planned a blocking `pnpm audit`; bun has no audit-equivalent gate, so the top-level `overrides` block + Dependabot alerts are the vulnerability-remediation channel instead.)+- **`release.yml` hardening** — `workflow_dispatch` dry-run default, a once-per-release `build-for-publish` artifact, `actions/attest-build-provenance` SLSA attestations, and a `create-github-release` job for the registry publish (the desktop workflow's `prepare` job now drafts the GitHub Release from the tag message; `release.yml` itself still creates none). (An earlier draft also planned a blocking `pnpm audit`; bun has no audit-equivalent gate, so the top-level `overrides` block + Dependabot alerts are the vulnerability-remediation channel instead.)- **`publish-pypi.yml`** — reusable PyPI publish (OIDC trusted publishing, `uv build --no-sources`, `pypa/gh-action-pypi-publish`). No PyPI distribution is published today.- **`test-install.yml`** — cross-platform end-user install smoke (`bun add -g`, git clone, install script) across 3 OS.- **`rollback.yml`** — manual registry deprecate (`npm deprecate` has no bun equivalent, so this would be the one place the npm CLI reappears) + optional revert PR.diff --git a/docs-internal/desktop_app.md b/docs-internal/desktop_app.mdindex 82ec401c..d72c4cf6 100644--- a/docs-internal/desktop_app.md+++ b/docs-internal/desktop_app.md@@ -170,12 +170,17 @@ bundled Python. Then remove the macOS notify-only branch in## CI-- `desktop-release.yml` on `v*.*.*` tags (and manual dispatch): matrix- windows-latest / macos-14 / ubuntu-22.04; builds the client and sidecar- with the repo toolchain, stages, runs typecheck + unit + invariant tests,- `electron-builder --publish always` into one draft release; a `finalize`- job undrafts it once all legs pass. Separate from `release.yml` so the+- `desktop-release.yml` on `v*.*.*` tags (and manual dispatch): a `prepare`+ job drafts the GitHub Release from the annotated tag's message (title ++ notes) so the three legs upload into one draft; the matrix+ windows-latest / macos-14 / ubuntu-22.04 builds the client and sidecar+ with the repo toolchain, syncs the version and refuses to continue when it+ differs from the tag (the build ships the committed desktop version, not+ the tag's), stages, runs typecheck + unit + invariant tests, then+ `electron-builder --publish always` into that draft; a `finalize` job+ undrafts it once all legs pass. Separate from `release.yml` so theregistry publish (`bun publish`) is never blocked and its locked strings are untouched.+ The step-by-step is in [ci_cd.md -> Cutting a release](./ci_cd.md#cutting-a-release).- `desktop-ci.yml` on PRs touching `desktop/**` or the backend contractfiles: typecheck, unit, invariants, build, and the Playwright Electronsmoke (xvfb on Linux) against a `uv sync`ed checkout venv.