Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
docs: document production deployment, releases, monitoring, and contributing
Jose Gabriel Ramirez Encinas80d751a91374
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex 0bb795b1f..e346b4a3a 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -3,6 +3,21 @@Thanks for your interest! evenfire is **open source** under the[Mozilla Public License 2.0](LICENSE) (MPL-2.0) and requires a signed CLA.+## Set up++- **Node.js 24.** `.nvmrc` pins `24`, every service declares+ `"node": ">=24"`, `desktop-app` declares `24.x`, and CI uses Node 24.+- Install dependencies across the repo:++ ```bash+ make install-all+ ```++ This runs `npm install` at the root, then in every service in parallel, then+ in `tests/e2e`. The root install's `postinstall` sets+ `git config core.hooksPath .githooks`, so the git hooks below are active from+ then on. `make install-git-hooks` does the same on its own.+## Dev loop (per service)Each service is a standalone Node/TypeScript package:@@ -15,18 +30,48 @@ npm testnpx tsc --noEmit```-Many services run without Kubernetes via `CLERUM_DEV_MODE=true` (see each-service README and the root README quickstart).+`control-api`, `workflow-recipes` and `packages/workflow-sdk` link+`packages/workflow-runtime-core`; run `npm ci && npm run build` there first or+their build fails with "Cannot find module '@clerum/workflow-runtime-core'".++From the repo root:++| Command | What it does |+| ---------------------- | ------------------------------------------------------------------------------------------------------------- |+| `make test-unit-all` | Checks the local service list against the CI matrix, then runs `npm test` in each service in `TEST_SERVICES`. |+| `npm run format` | Runs Prettier with `--write` over the projects and root files listed in `scripts/prettier/paths.mjs`. |+| `npm run format:check` | Same, with `--check`. Use it to see what the pre-commit hook would change. |+| `npm run style-rules` | Runs the repo's frontend style rules (see [scripts/style-rules](scripts/style-rules/README.md)). |++To run `mcp-host` on its own without Kubernetes, see+[Single-service dev mode](docs/get-started/dev-mode.md). Dev mode is for+iterating on one service; it is not full platform security.## Tests are requiredPRs must include tests for new behavior and keep `npm test` + `npx tsc --noEmit`green for every changed service. PRs without tests may be closed.-## The pre-commit hook may edit your commit+## Git hooks++`core.hooksPath` is `.githooks`:++- **pre-commit** runs `npm run version:staged`, then `npm run format:staged`,+ then `npm run style-rules:staged`.+ - `version:staged` bumps the patch version in the `package.json` of each+ package (from `packageRoots` in `scripts/prettier/paths.mjs`) whose files+ you staged.+ - `format:staged` runs Prettier on your staged files that fall inside the+ Prettier targets in `scripts/prettier/paths.mjs`.+ - `style-rules:staged` checks staged frontend files against the style rules.+- **commit-msg** runs commitlint with `@commitlint/config-conventional`+ (`commitlint.config.cjs`). Commit messages must follow+ [Conventional Commits](https://www.conventionalcommits.org/), for example+ `fix(mcp-host): ...` or `docs: ...`. Body line length is not limited.-`core.hooksPath` is `.githooks`, so committing runs `npm run version:staged`-then `npm run format:staged`. Two behaviours surprise people:+### The pre-commit hook may edit your commit++Two behaviours surprise people:- **Your commit can gain files you did not stage.** Changing source under`external-rest-api/` or `rpc-proxy/` bumps that package's `package.json`@@ -43,6 +88,50 @@ Every refusal names the file and the fix. If you need to bypass the hook toget unstuck, say so in the PR rather than leaving it silent. The two releasevalidators run in CI and will catch what the hook would have.+## Branches and pull requests++- Name branches with a conventional prefix: `feat/`, `fix/`, `hotfix/`,+ `chore/`, `docs/`, `test/`, `refactor/`, `ci/`, `build/`, `perf/` or+ `revert/`. Do not use agent or vendor prefixes such as `codex/`, `claude/`,+ `openai/` or `anthropic/`.+- Open pull requests against **`dev`**, the integration branch. `main` is the+ release branch; changes reach it when `dev` is promoted.+- CI (`.github/workflows/ci-public.yml`) builds, tests and typechecks every+ service in its matrix on each pull request.++## Desktop App (Electron)++If Electron's `postinstall` is skipped, `npm ci` still exits successfully but+the Electron runtime is never downloaded. The failure then shows up later as a+test error that looks like a product bug. For any `desktop-app` work:++1. Use Node 24.+2. Run `npm ci` **without** `--ignore-scripts`.+3. Run `npm run verify:electron` and confirm it passes before you trust any+ test or build result.++If the check reports `Electron failed to install correctly`, switch to Node 24,+repair `node_modules`, and reinstall. Do not set+`ELECTRON_SKIP_BINARY_DOWNLOAD` or point Electron at an unverified binary.++## Validating against a cluster++Unit tests run without Kubernetes. For changes that need the full platform,+use the local Minikube lanes:++- [Minikube T2 runbook](docs/testing/minikube-t2-runbook.md) — the canonical+ `make minikube-t2-preflight`, `make minikube-t2` and+ `make minikube-t2-runtime` entry points.+- [E2E guide](docs/testing/e2e-guide.md) — integration, E2E and Playwright+ suites.++## Documentation++- Public docs describe what `deploy/base` and the Minikube overlays ship. If a+ feature is off by default, say so and name the flag.+- Follow the [claims guardrails](docs/meta/claims-guardrails.md): every claim+ must be backed by code in this repo, with honest caveats.+## What we accept- Bug fixes and improvements to the platform services and core CRDs.@@ -54,10 +143,15 @@ validators run in CI and will catch what the hook would have.(see docs) — they are not merged into this repo. This keeps the core lean andmaintainable (the same model LangChain and n8n use).+## Security issues++Do not open a public issue for a vulnerability. See [SECURITY.md](SECURITY.md)+for private reporting.+## CLA-This project requires a signed Contributor License Agreement. The CLA-assistant-bot will prompt you on your first PR. Contributions may be used in the project's-commercial / managed editions.+This project requires a signed Contributor License Agreement. Contributions+may be used in the project's commercial / managed editions.-See [CLA.md](CLA.md) for the full agreement (draft, pending legal review).+See [CLA.md](CLA.md) for the agreement and how to accept it (it is a draft,+pending legal review).diff --git a/README.md b/README.mdindex d6c7d379d..22f44557f 100644--- a/README.md+++ b/README.md@@ -48,10 +48,14 @@ records the governed `initial_setup` GFS operator link atomically:git clone https://github.com/evenfire-ai/evenfire.git && cd evenfirecp .env.example .env# edit .env: set ADMIN_PASSWORD (required, no default ships) and ONE LLM key-MINIKUBE_IMAGE_TAG=latest make minikube-setup # see the note below on this override-make minikube-status # wait for every deployment READY+make minikube-setup # pulls the pinned release images; first run ~15 min+make minikube-status # wait for every deployment READY```+If setup reports that a pinned image tag does not exist (between a release bump+and its promotion), re-run with `MINIKUBE_IMAGE_TAG=latest`; see the+[Quickstart](docs/get-started/quickstart.md).+The seeded `[email protected]` Desktop account is therefore the initial GFSoperator. Re-running setup is idempotent: it may log in to verify the existingactive link, but it never recreates a revoked link or silently creates a seconddiff --git a/desktop-app/README.md b/desktop-app/README.mdindex 698cf2910..58b07dfd9 100644--- a/desktop-app/README.md+++ b/desktop-app/README.md@@ -121,6 +121,34 @@ npm run buildnpm run start```+## Package and Sign++Desktop builds need Node 24, an `npm ci` with lifecycle scripts enabled (never+`--ignore-scripts`, never `ELECTRON_SKIP_BINARY_DOWNLOAD`), and a passing+`npm run verify:electron` before any build or test result counts.++```bash+npm ci+npm run verify:electron+npm run make:mac:arm64 # or make:mac:x64, make:windows:x64, make:windows:arm64, make:linux:x64, make:linux:arm64+```++Each `make:*` script runs `electron-forge make` for one platform/arch and builds+first via the forge `prePackage` hook (`EVENFIRE_SKIP_PACKAGE_BUILD=1` skips that).+Output goes to `out/`. The makers are zip (all platforms), dmg (macOS), squirrel+(Windows), and deb/rpm (Linux). This directory's `Makefile` has no packaging+targets.++macOS signing and notarization are opt-in and run only on a macOS build host:+set `ENABLE_APPLE_CODESIGN=1`, optionally `APPLE_CODESIGN_IDENTITY` and+`APPLE_KEYCHAIN`, plus either `APPLE_API_KEY`/`APPLE_API_KEY_ID`/`APPLE_API_ISSUER`+or `APPLE_KEYCHAIN_PROFILE` (the config throws if neither is set).+`APPLE_BUNDLE_ID` overrides the default `ai.evenfire.desktop`. Windows and Linux+packages are unsigned. No workflow in this repo builds or publishes installers.++Signing details, release tags, and how the app detects required updates are in+[Ship it to your users](../docs/surfaces/desktop-app.md#ship-it-to-your-users).+## Port-forward Remote Cluster ServicesThe root launcher already starts the required port-forwards for normal local development.diff --git a/docs/README.md b/docs/README.mdindex 8a3df7709..83231a516 100644--- a/docs/README.md+++ b/docs/README.md@@ -15,12 +15,13 @@ minikube overlays ship; features that are off by default say so.## Get started-| Doc | Description |-| --------------------------------------------------------- | ---------------------------------------------- |-| [Quickstart](get-started/quickstart.md) | Full platform on minikube with one command |-| [Registry quickstart](get-started/registry-quickstart.md) | Connect, install, and publish as a self-hoster |-| [Learning path](get-started/learning-path.md) | Choose a path by role and goal |-| [FAQ](faq.md) | Common questions and troubleshooting |+| Doc | Description |+| --------------------------------------------------------- | --------------------------------------------------------------------------- |+| [Quickstart](get-started/quickstart.md) | Full platform on minikube with one command |+| [Registry quickstart](get-started/registry-quickstart.md) | Connect, install, and publish as a self-hoster |+| [Dev mode](get-started/dev-mode.md) | Run mcp-host alone without Kubernetes (contributors; not platform security) |+| [Learning path](get-started/learning-path.md) | Choose a path by role and goal |+| [FAQ](faq.md) | Common questions and troubleshooting |## Concepts@@ -80,6 +81,8 @@ minikube overlays ship; features that are off by default say so.| [Minikube full stack](deploy/minikube.md) | In-depth reference for the local platform || [Production notes](deploy/production.md) | Checklist and in-repo deploy assets || [WorkflowRecipes operations](deploy/workflow-recipes-guide.md) | Recipe ops, RBAC, debugging |+| [Releases and upgrades](deploy/releases-and-upgrades.md) | Versioning, release images, pinning a release, upgrade order |+| [Monitoring and observability](operations/monitoring.md) | Logs starter, metrics endpoints, health checks, what to alert on || **Upgrade notes** | || [Workflow output PVC upgrade](deploy/workflow-output-pvc-upgrade.md) | Per-recipe output PVC ownership change || [Plugin Workload SDK upgrade](deploy/plugin-workload-sdk-upgrade.md) | promptBridge / clientNotifications rollout order and migration |@@ -176,7 +179,7 @@ as user docs: [design/](design/README.md).| Pick or configure an LLM provider | [LLM providers overview](llm-providers/README.md) || Run the full stack locally | [Minikube](deploy/minikube.md) || Ship to a real cluster | [Production notes](deploy/production.md) |-| Upgrade an existing install | [Deploy & operate › Upgrade notes](#deploy--operate) |+| Upgrade an existing install | [Releases and upgrades](deploy/releases-and-upgrades.md) || Debug E2E | [E2E guide](testing/e2e-guide.md) || Feed docs to an LLM | [llms.txt](llms.txt) || Use the admin console | [Control UI](surfaces/control-ui.md) |diff --git a/docs/deploy/minikube.md b/docs/deploy/minikube.mdindex dab93edb2..74c7c889a 100644--- a/docs/deploy/minikube.md+++ b/docs/deploy/minikube.md@@ -60,11 +60,54 @@ Ready before E2E results are interpreted.## Full Setup (First Time — from scratch)```bash-MINIKUBE_IMAGE_TAG=latest make minikube-setup # cluster → namespaces → CRDs → keys → secrets → images → deploy → verify-make minikube-status # Confirm all services after setup+make minikube-setup # cluster → namespaces → CRDs → keys → secrets → images → deploy → verify+make minikube-status # Confirm all services after setup```The script is idempotent — safe to run again if interrupted or if something fails.+It needs `ADMIN_PASSWORD` in `.env` for the default seed profile.++### Setup options++`make minikube-setup` passes Make variables to `scripts/minikube/full-setup.sh`+and `ARGS="..."` through as flags. (`scripts/minikube/setup.sh` is an older+compatibility path that takes only `--build`; none of the options below apply+to it.)++| Option | Default | Effect |+| --- | --- | --- |+| `SEED_PROFILE=minimal\|e2e` or `ARGS="--seed-profile=e2e"` | `minimal` | `minimal` is a clean install: the `[email protected]` admin and one agent, no test fixtures. `e2e` adds the E2E fixtures: a test user, `e2e-*` recipes, and demo MCP servers (placeholder credentials unless `.env` or `mcp-servers/<svc>/secret.yaml` supplies real ones). Any other value is an error. |+| `make minikube-setup-e2e` | — | `minikube-setup SEED_PROFILE=e2e`, then, on a ghcr cluster, builds the two unpublished E2E coordinator fixtures (`make minikube-build-e2e-fixtures`). Use this rather than `--seed-profile=e2e` alone on a ghcr cluster. |+| `IMAGE_SOURCE=ghcr\|local` or `make minikube-setup-local` | `ghcr` | `ghcr` pulls published images and renders the `minikube-ghcr` overlay. `local` builds every image from source and renders `minikube`. |+| `MINIKUBE_IMAGE_TAG=<tag>` | committed pin | Render-time override of the ghcr tag in `deploy/components/ghcr-images/kustomization.yaml`; never committed. |+| `SKIP_UIS=true` or `ARGS="--skip-uis"` | `false` | Skips the Control UI, Profile UI, and Desktop App image builds and the Control UI / Profile UI Deployments, by rendering `minikube-no-uis` (or `minikube-no-uis-ghcr`). Run the frontends locally with `npm run ui`. |+| `ARGS="--skip-build"` | off | Re-deploys without acquiring images (~1 min). The overlay follows the mode recorded in `deploy/minikube/.image-manifest.json`, not `IMAGE_SOURCE`. It does not change the DB default: on an existing cluster pair it with `REUSE_DB=true` unless you want the DB rebuilt too. |+| `REUSE_DB=true` or `ARGS="--keep-db"` | off | Keeps the existing `control-postgres-data` volume instead of rebuilding the DB. |+| `ARGS="--reset-db"` | on | Rebuilds the DB (already the default). On a cluster that already has the volume it also needs `CONTROL_DB_RESET_PVC_UID` (see below). |+| `ARGS="--defer-bootstrap-seed"` or `MINIKUBE_DEFER_BOOTSTRAP_SEED=true` | off | Skips the admin/user seed so the first admin is created through the Control UI setup screen (`/api/v1/admin/auth/setup`). Requires `IMAGE_SOURCE=local` and the `minimal` seed profile; `ADMIN_PASSWORD` is then not needed. |+| `ARGS="--force-keys"` | off | Regenerates all JWT signing keys; invalidates existing tokens. |+| `MINIKUBE_MEMORY=<MB>`, `MINIKUBE_CPUS=<n>` | `10240`, `6` | Size of the minikube VM when setup has to start it (`scripts/minikube/start.sh`). |++Combine them as needed, for example+`make minikube-setup SKIP_UIS=true REUSE_DB=true ARGS="--skip-build"`.++**Database rebuild.** By default setup deletes the `control-postgres-data`+volume and replays every migration from an empty schema, so a leftover DB from+an older build cannot drift from the current schema/grant contract — that drift+otherwise surfaces at the migration gate as+`control_api_runtime relation privileges differ from the explicit access contract`.+On a fresh cluster (no volume, no reset state) there is nothing to delete and+setup continues. On a cluster that already has the volume, the reset refuses to+run unless you name the exact volume it may delete:++```bash+CONTROL_DB_RESET_PVC_UID="$(kubectl --context clerum-test -n control-plane \+ get pvc control-postgres-data -o jsonpath='{.metadata.uid}')" make minikube-setup+# or keep the data instead:+make minikube-setup REUSE_DB=true+```++`CONTROL_DB_RESET_PVC_UID=none` asserts that no such volume exists.### Images are pulled, not built@@ -75,19 +118,21 @@ run is roughly 15 minutes, dominated by network transfer; a warm cache is farquicker.`make minikube-setup-local` (equivalently `IMAGE_SOURCE=local`) restores the-previous build-everything behaviour.+previous build-everything behaviour. The overlay follows the mode: `minikube-ghcr`+for pulled images, `minikube` for local builds, and the `minikube-no-uis` /+`minikube-no-uis-ghcr` pair when `SKIP_UIS=true`.MCP servers are no longer loaded into the cluster at setup. They aredistributed through the evenfire registry and installed on demand.-### Two overrides a first run needs+### Overrides a first run may need-**`MINIKUBE_IMAGE_TAG=latest`** — required until the next release is tagged.-The ghcr component pins that release tag, and the images for it are created by-promotion _on_ the tag, so the pin names something that does not exist yet.-Without the override the pull fails with an explicit message naming the image,-the tag, and this variable — not a raw `MANIFEST_UNKNOWN`. After the release is-cut and promoted, drop the override.+**`MINIKUBE_IMAGE_TAG=latest`** — only when the committed pin names a release+that has not been promoted yet (for example on a release-prep branch). The+images for a release are created by promotion _on_ its tag, so until then the+pin names something that does not exist. The pull then fails with an explicit+message naming the image, the tag, and this variable — not a raw+`MANIFEST_UNKNOWN`. Drop the override once the release is promoted.**`MINIKUBE_MEMORY=9216`** — if minikube fails to start. The default requests10240 MB, which is above a stock Docker Desktop ceiling.@@ -296,25 +341,6 @@ An unreadable list is a refusal, never an empty list.When a context guard refuses, the context it names may belong to anothersession or cluster: stop and ask before renaming or removing it.-### Options--```bash-make minikube-setup # Full setup; rebuilds the DB from scratch every run-make minikube-setup-local # Same, building every image instead of pulling-make minikube-setup ARGS="--skip-build" # Re-deploy without re-acquiring images (~1 min)-make minikube-setup REUSE_DB=true # Preserve the existing postgres DB volume-make minikube-setup ARGS="--reset-db" # Force a DB rebuild (already the default; also fixes WAL corruption)-make minikube-setup ARGS="--force-keys" # Regenerate all JWT signing keys-```--`make minikube-setup` deletes the `control-postgres-data` volume and replays every-migration from an empty schema by default, so a leftover DB from an older build-cannot drift from the current schema/grant contract — that drift otherwise-surfaces at the migration gate as-`control_api_runtime relation privileges differ from the explicit access contract`.-The rebuild is a no-op on a fresh cluster (nothing to delete). Pass-`REUSE_DB=true` to keep an existing volume (the old fast-iterate / keep-data path).-### Manual Steps (Advanced)If you need to run individual steps:@@ -652,10 +678,15 @@ make minikube-pf-desktop # Re-establish port forwards## Restart With Database Wipe```bash-make minikube-db-reset # Deletes postgres PVC, recreates it, restarts control-api+# Deletes the postgres PVC, recreates it, restarts control-api.+# Name the exact PVC it may delete (or `none` if there is none):+CONTROL_DB_RESET_PVC_UID="$(kubectl --context clerum-test -n control-plane \+ get pvc control-postgres-data -o jsonpath='{.metadata.uid}')" make minikube-db-reset# → Allows re-running the initial admin setup in Control UI```+Without `CONTROL_DB_RESET_PVC_UID` the target stops with an error.+---## WorkflowRecipe / WRC — Configuration@@ -698,10 +729,9 @@ there) cannot read the public key and all workflow JWTs fail with `401 Invalid t| `CLERUM_MCP_HOST_IMAGE` | `clerum/mcp-host:0.9.5` | mcp_host image injected into workflows || `CLERUM_WRC_SERVICE_NAME` | not set (code fallback `workflow-recipes`) | Service name for coordinator DNS callbacks |-The first two are set in `deploy/base/control-plane/workflow-recipes.yaml` (lines-153-156). `CLERUM_WRC_SERVICE_NAME` is **not** in any manifest — it is an optional+The first two are set in `deploy/base/control-plane/workflow-recipes.yaml`. `CLERUM_WRC_SERVICE_NAME` is **not** in any manifest — it is an optionaloverride read from the process environment, with the fallback `workflow-recipes`-hardcoded in `workflow-recipes/src/reconciler/workflowRecipeReconciler.ts:622`.+hardcoded in `workflow-recipes/src/reconciler/workflowRecipeReconciler.ts`.These are env-var values, not container images, so the plain `images:` blockwould not see them. `deploy/overlays/minikube/kustomizeconfig/imagetags.yaml`@@ -1066,7 +1096,7 @@ and rpc-proxy verifies them with the public half.Service tokens may not be properly applied. Run:```bash-make minikube-setup ARGS="--skip-build" # Re-applies kustomize overlay with correct tokens+make minikube-setup REUSE_DB=true ARGS="--skip-build" # Re-applies kustomize overlay with correct tokens```### ImagePullBackOff on pods@@ -1107,10 +1137,12 @@ re-applies the manifests.### Postgres CrashLoopBackOff (WAL corruption)-Common after cold starts. The setup script detects and auto-fixes this, but you can also run manually:+Common after cold starts. The setup script detects and auto-fixes this, but you can also force a DB rebuild:```bash-make minikube-setup ARGS="--reset-db --skip-build"+CONTROL_DB_RESET_PVC_UID="$(kubectl --context clerum-test -n control-plane \+ get pvc control-postgres-data -o jsonpath='{.metadata.uid}')" \+ make minikube-setup ARGS="--reset-db --skip-build"```### JWT 401 errors after key regeneration@@ -1144,7 +1176,7 @@ make minikube-restart-all # Restart all pods to pick up new keys| `403: "Forbidden: user cannot access this host"` | hostRef does not match what is authorized in control-api, or the Host CRD does not exist | Verify `kubectl get host chatllm -n mcp-host`, verify host assignment to team || Pod in `ImagePullBackOff` | The cluster does not hold an image it was told to run | `make minikube-verify-images` to see the mode and the missing refs, then `make minikube-pull-images` (ghcr) or `make minikube-build-images` (local). See the section above. || Port forward dropped after pod restart | Kubernetes closes the tunnel when the pod restarts | `make minikube-pf-desktop` |-| `409 Conflict` on first setup | Previous data in postgres | `make minikube-db-reset` |+| `409 Conflict` on first setup | Previous data in postgres | `make minikube-db-reset` with `CONTROL_DB_RESET_PVC_UID` set (see Restart With Database Wipe) || SSE stream closes with `"auth-expired"` | mcp-host rejected 3 consecutive `/v1/runtime/status` polls with 401/403. Those polls carry only `x-clerum-edge-*` headers (no bearer token), so this is an edge-guard/host-access rejection, not RPC-token expiry. | The Desktop App clears its token cache and reconnects automatically. If it repeats, check chatllm logs for `Missing runtime edge caller context` / `Runtime edge host mismatch`. || **Workflow** — `WRC returned 404 on GET status` | Stale rollout, old WRC, or a legacy recipe CRD outside `sandbox-recipes` | Run the clean pre-gate sync, verify `kubectl get workflowrecipes -n sandbox-recipes`, and delete any legacy `workflowrecipes` in `mcp-server`. || **Workflow** — `Failed to connect to MCP servers` | NP-01: incorrect egress label (`{workloadId}` instead of `{recipeName}-{workloadId}`) or NP-02: missing ingress in `mcp-server` | List the recipe's NPs with `kubectl get netpol -n sandbox-recipes` and `-n mcp-server`. Reset workflow. |diff --git a/docs/deploy/production.md b/docs/deploy/production.mdindex 097dc2d74..8c97e548b 100644--- a/docs/deploy/production.md+++ b/docs/deploy/production.md@@ -1,64 +1,310 @@-# Production deployment (notes)--This repository ships **Kubernetes manifests**, operators, and a Helm chart for-CRDs. A single turnkey “one cloud vendor” guide is not published in this OSS-tree yet; use this page as the production checklist and link into in-repo-assets.--## What “production” means here--- All platform services deployed to a real cluster (not a single dev-mode `mcp-host`)-- JWT auth chain on the external edges (desktop → rpc-proxy, service → control-api);- `mcp-host` runtime routes still use edge trust headers, restricted by NetworkPolicy-- Default-deny NetworkPolicies enforced (Calico or equivalent CNI with policy)-- Secrets and LLM keys in cluster Secrets / your secret manager-- Non-root, capability-dropped workloads; coordinator images digest-pinned and- the connector-image allowlist switched from audit to enforce--## Building blocks in this repo--| Asset | Path |-| -------------------------- | ------------------------------------------------------------------ |-| CRDs (Helm) | [charts/clerum-crds](../../charts/clerum-crds/README.md) |-| Base manifests | [deploy/base](../../deploy/base) |-| Overlays | [deploy/overlays](../../deploy/overlays) |-| Bootstrap scripts | [scripts/](../../scripts/), [deploy/scripts](../../deploy/scripts) |-| Local full-stack reference | [minikube.md](minikube.md) |-| Plugin Workload SDK upgrade | [plugin-workload-sdk-upgrade.md](plugin-workload-sdk-upgrade.md) |-| Observability starter | [monitoring/](../../monitoring/README.md) |--Install CRDs:--```bash-helm install clerum-crds ./charts/clerum-crds-# After chart upgrades, re-apply CRD YAML (Helm 3 does not upgrade crds/ on upgrade):-kubectl apply -f ./charts/clerum-crds/crds/-```--## Recommended rollout order--1. **Cluster prerequisites** — CNI with NetworkPolicy, storage classes as needed,- registry pull credentials.-2. **CRDs** — install/apply `clerum-crds`.-3. **Namespaces + deny-all baseline** — from `deploy/base` overlays.-4. **Signing / JWT keys** — bootstrap scripts under `scripts/` and deploy docs.-5. **Control plane services** — `host-context-controller`, `control-api`, proxies.-6. **Agent runtime** — `mcp-host`, `channel-reader`, bridges.-7. **UIs** — `control-ui`, `profile-ui`; the desktop client is distributed to- users rather than deployed —- see [surfaces/desktop-app.md](../surfaces/desktop-app.md#ship-it-to-your-users).-8. **First Host + Context + channel** — smallest viable declarative config.-9. **Connectors** — McpServers on an allowlisted Context.-10. **Member invitations** — decide how invitation email is sent before you invite- anyone; see [member invitations on self-hosted](../how-to/member-invitations-self-hosted.md).- Production is where the zero-config path applies: set- `CONTROL_API_MEMBER_REGISTRATION_MODE=hosted` and control-api enrolls itself- with evenfire's registration hub, which sends the mail. It requires your- Profile UI and Control UI base URLs to be **real, publicly resolvable- domains** (the host guard refuses `localhost`, IP literals, and dotless- hostnames) plus egress to `registration.evenfire.ai`. Run your own- `member-registration-service` instead and leave the mode `remote`.-11. **Observability** — metrics + log stack; alert on approval backlog and 5xx.+# Running evenfire on your own cluster++This page is for operators who want to run evenfire on a real Kubernetes+cluster, not on minikube. It lists what the cluster needs, what you must write+yourself, the install order, and what this repository does not provide.++This repository ships Kubernetes manifests ([`deploy/base`](../../deploy/base)),+a CRD Helm chart ([`charts/clerum-crds`](../../charts/clerum-crds/README.md)),+release images on `ghcr.io/evenfire-ai`, and bootstrap scripts. The only+overlays in the repository are the minikube ones. **There is no production+overlay.** You write your own overlay on top of `deploy/base`, and+[`deploy/overlays/minikube`](../../deploy/overlays/minikube) is the worked+example to copy from.++> Code and API group still use the historical name **clerum** (`clerum.io`,+> `CLERUM_*`). Same project.++## What "production" means here++- All platform services run on a real cluster (not a single dev-mode `mcp-host`).+- The JWT auth chain is on the external edges (desktop → rpc-proxy,+ service → control-api). `mcp-host` runtime routes still use edge trust+ headers, restricted by NetworkPolicy.+- Default-deny NetworkPolicies are enforced by the CNI.+- Secrets and LLM keys live in cluster Secrets.+- Workloads run non-root with capabilities dropped. The connector-image+ allowlist is switched from audit to enforce.++## What your cluster needs++| Requirement | Why |+| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| **Kubernetes 1.30 or newer** | [`deploy/base/cluster-wide/workflowrecipe-admission.yaml`](../../deploy/base/cluster-wide/workflowrecipe-admission.yaml) uses `ValidatingAdmissionPolicy` at `admissionregistration.k8s.io/v1`, which is GA from 1.30. Older API servers reject the apply. |+| **A CNI that enforces NetworkPolicy** | Every namespace starts from deny-all. The isolation model depends on the CNI enforcing it, including `ipBlock` egress rules to the API server endpoint. The minikube setup uses Calico. A CNI that ignores NetworkPolicy runs the platform with no network isolation. |+| **Cluster-scoped install rights** | You install CRDs, ClusterRoles, ClusterRoleBindings, PriorityClasses (`clerum-batch`, `clerum-service`, `clerum-interactive-host`) and ValidatingAdmissionPolicies. See [`deploy/base/cluster-wide`](../../deploy/base/cluster-wide). |+| **CoreDNS labeled `k8s-app: kube-dns`** | Base DNS egress rules select pods in `kube-system` with that label. If your cluster DNS uses another label, patch those rules. |+| **Block storage (RWO)** | Postgres (`control-postgres-data`, 5Gi), one workspace PVC per agent (10Gi default), SharedFileSystem PVCs (RWO by default), per-recipe workflow output PVCs. |+| **RWX storage (only for legacy output)** | Base still creates the legacy shared `clerum-workflow-output` PVC (RWX, 1Gi). New WorkflowRecipes do not mount it. See [workflow output PVC upgrade](workflow-output-pvc-upgrade.md). |+| **Postgres** | Base runs `postgres:16-alpine` as one replica on one RWO PVC. See [Database](#database). |+| **Your own ingress, DNS and TLS** | Base ships no ingress controller and no Ingress objects. See [How users reach the platform](#how-users-reach-the-platform). |+| **Outbound internet** | LLM providers, image pulls, channel APIs. See [Outbound destinations](#outbound-destinations). |++## Sizing++Be careful with any single number:++- **6 CPUs / 10 GB RAM** is the Docker Desktop allocation for the minikube+ evaluation setup ([minikube.md](minikube.md#prerequisites)). It is a local+ evaluation footprint, not a production recommendation.+- The static Deployments in `deploy/base` (18 of them, with no agents running)+ **request** about 0.9 CPU and 2.7 GiB of memory in total. Requests are not+ real usage. `codex-llm-proxy` and `grok-llm-proxy` each request 768Mi even+ though both subscriptions are off by default.+- Most load comes from pods that the controllers create at runtime. These are+ not in `deploy/base`.++What the host-context-controller (HCC) creates for **each agent (Host)**:++| Resource | Namespace | Default |+| -------------------------------------------- | ---------- | --------------------------------------------------------------------------------------- |+| `mcp-host` Deployment | `mcp-host` | 100m CPU / 128Mi requested, 500m / 512Mi limit (`CONTEXT_MAPPER_HOST_RESOURCES_*`) |+| Service | `mcp-host` | |+| Workspace PVC | `mcp-host` | RWO, 10Gi (`CONTEXT_MAPPER_HOST_WORKSPACE_SIZE`) |+| `channel-reader-<host>` Deployment + Service | `channels` | Always created. `replicas: 0` until a CommunicationChannel references the Host, then 1. |++A Host with `spec.desktop` runs the desktop image instead, with a 250m / 256Mi+request and a 1 CPU / 4Gi limit by default (`CONTEXT_MAPPER_DESKTOP_RESOURCES_*`).+McpServers (connectors), workflow coordinator pods, GFS controllers (`gfsc`) and+workspace-files-controllers add their own pods. Size the cluster for the number+of agents and connectors you plan to run, then measure.++## Database++Base runs one Postgres Deployment (`control-postgres` in `control-plane`):+`postgres:16-alpine`, one replica, `strategy: Recreate`, one 5Gi RWO PVC with no+`storageClassName` set. It has no replication, no failover and no backup job.+Back it up yourself.++Moving to an external Postgres is **not** one connection string. Several+services read their own Secret, and the provisioning scripts assume the+in-cluster Deployment:++| Consumer | Where the DSN comes from |+| ------------------------ | ------------------------------------------------------------------------------------ |+| control-api | `CONTROL_API_PG_CONNECTION_STRING` from Secret `control-api-postgres-runtime` |+| trace-maintenance-worker | `TRACING_MAINTENANCE_PG_CONNECTION_STRING` from `trace-maintenance-postgres-runtime` |+| workflow-recipes | `POSTGRES_CONNECTION_STRING` from `workflow-recipes-postgres-runtime` |+| GFS controllers | Secrets `gfs-controller-db` (writer) and `gfs-controller-reader-db` (reader) |++- [`provision-control-api-runtime-roles.sh`](../../deploy/scripts/provision-control-api-runtime-roles.sh)+ and [`provision-gfs-db.sh`](../../deploy/scripts/provision-gfs-db.sh) run+ `psql` through `kubectl exec` into `deployment/control-postgres`. With an+ external database you run the equivalent role SQL yourself.+- The GFS NetworkPolicies HCC generates select the Postgres pod by label+ (`app: control-postgres`, `CONTEXT_MAPPER_GFS_POSTGRES_APP_LABEL`).+- `control-api-config` in base also carries a default DSN with the+ `postgres:postgres` credentials. The Secret-sourced env var above takes+ precedence, but do not leave the default credentials in place.++## How users reach the platform++Base ships no ingress controller, no Ingress objects and no TLS. Minikube uses+`kubectl port-forward`. On a real cluster you bring your own ingress and+certificates for these Services:++| Who | Service (namespace, port) | Notes |+| ------------------------------ | ------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |+| Administrators | `control-ui` (`control-plane`, 3000) | [Control UI](../surfaces/control-ui.md) |+| Members | `profile-ui` and `external-rest-api` (`profiles`) | Sign-in, invitations, Desktop App API |+| Desktop App | `rpc-proxy` (`rpc-proxy`, 8094) | JWT-authenticated chat and streaming |+| Slack, Teams, webhook triggers | `webhook-proxy` (`webhook-ingress`, 8095) | Inbound only if you use them. See [Slack](../how-to/connect-slack.md) and [Teams](../how-to/connect-teams.md). |++**The NetworkPolicies decide which ingress works.** Base admits public traffic+to `control-ui`, `profile-ui`, `rpc-proxy` and `webhook-proxy` only from pods+labeled `app: cloudflared` in a namespace named `ingress`. That matches the+optional Cloudflare Tunnel stack in [`deploy/base/ingress`](../../deploy/base/ingress),+which `deploy/base/kustomization.yaml` deliberately does not include. To use it,+add `../../base/ingress` (or your path to it) to your overlay and supply the+tunnel credentials Secret yourself. To use any other ingress controller, either+run it so it matches those selectors or patch those NetworkPolicies in your+overlay.++Then point the platform at your public URLs. In base these default to+`127.0.0.1`/`localhost` values that only work with port-forwards:++| Variable | Where |+| ------------------------------------------------------------------------------------------------------------------------------------- | -------------------- |+| `CONTROL_API_CONTROL_UI_BASE_URL` | `control-api-config` |+| `CONTROL_API_DESKTOP_PROFILE_UI_BASE_URL`, `CONTROL_API_DESKTOP_EXTERNAL_REST_API_BASE_URL`, `CONTROL_API_DESKTOP_RPC_PROXY_BASE_URL` | `control-api-config` |+| `CONTROL_API_OAUTH_CALLBACK_BASE_URL` | `control-api-config` |+| `CONTROL_API_GOOGLE_CLIENT_ID` (placeholder in base) | `control-api-config` |+| `RPC_PROXY_CORS_ORIGIN`, `RPC_PROXY_OAUTH_CALLBACK_BASE_URL` | `rpc-proxy-config` |++## Outbound destinations++The NetworkPolicies are port and CIDR rules, not hostname rules. Agent pods+(`mcp-host`) may reach any **public** address on TCP 80 and 443; private,+link-local, metadata, carrier-grade NAT and reserved ranges are excluded+([`deploy/base/public-egress-exceptions.yaml`](../../deploy/base/public-egress-exceptions.yaml)).+Connector (McpServer) egress is generated per server from its+`egressBindings`; `exact-host` is the default and a `cidr` binding cannot name+a private range ([McpServer reference](../crds/mcpserver.md#egress-bindings)).++If you run a perimeter firewall, these are the destinations the platform uses:++| Destination | Used by | When |+| ----------------------------------------------- | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |+| Your LLM provider's API | `mcp-host`, workflow pods | Always. Each provider has a fixed endpoint; see [LLM providers](../llm-providers/README.md). |+| `ghcr.io` | Nodes (image pulls) | When you run release images |+| Telegram Bot API | `channel-reader-<host>` | Telegram channels. The reader polls outbound; no inbound webhook is needed. |+| `login.microsoftonline.com`, Bot Framework APIs | `channel-reader-<host>` | Teams channels (plus inbound to `webhook-proxy`) |+| Slack API | `channel-reader-<host>` | Slack channels (plus inbound to `webhook-proxy`) |+| OAuth providers | control-api | [OAuth connectors](../how-to/connect-oauth-connector.md) |+| `registry.evenfire.ai` | control-api, workflow-recipes | Optional. Only if you set `CLERUM_REGISTRY_URL`; see [connect to the registry](../how-to/connect-to-registry.md). |+| `registration.evenfire.ai` | control-api | Only with `CONTROL_API_MEMBER_REGISTRATION_MODE=hosted`. See [member invitations](../how-to/member-invitations-self-hosted.md). |++## Write your overlay++Your overlay lists `deploy/base` as a resource and adds what base leaves out.+Use [`deploy/overlays/minikube/kustomization.yaml`](../../deploy/overlays/minikube/kustomization.yaml)+as the reference.++**Images.** Base refers to images by local `clerum/<name>` names. Add the+[`deploy/components/ghcr-images`](../../deploy/components/ghcr-images/kustomization.yaml)+component, which rewrites them to `ghcr.io/evenfire-ai/<name>` at the release+tag of the commit you check out, including the image references HCC and+workflow-recipes inject into the pods they create. Images are built for+`linux/amd64` and `linux/arm64`. See+[releases and upgrades](releases-and-upgrades.md#your-own-cluster) for pinning+and digests. Two HCC image settings are **not** covered by the component and+default to local names:++| HCC env var | Default | Set it when |+| ------------------------------ | -------------------------------- | ---------------------------- |+| `CONTEXT_MAPPER_GFSC_IMAGE` | `clerum/gfs-controller:test` | You use a GlobalFileSystem |+| `CONTEXT_MAPPER_DESKTOP_IMAGE` | `clerum/mcp-host-desktop:latest` | Any Host sets `spec.desktop` |++**ConfigMaps base does not define.** HCC mounts `mcp-host-config` into every+agent pod with `envFrom`, and `rpc-proxy` reads `rpc-proxy-config`. Neither is+in base; without them those pods fail to start. Copy the minikube versions+([`mcp-host-config.yaml`](../../deploy/overlays/minikube/configmaps/mcp-host-config.yaml),+[`rpc-proxy-config.yaml`](../../deploy/overlays/minikube/configmaps/rpc-proxy-config.yaml))+and replace their values: the minikube `rpc-proxy-config` contains fixed+development secrets (`RPC_PROXY_DESKTOP_COOKIE_SECRET`,+`RPC_PROXY_DESKTOP_API_TOKEN`, `RPC_PROXY_SANDBOX_UI_COOKIE_SECRET`) and+localhost URLs. Many features are off in base and on in minikube; decide each+one with [the configuration reference](../reference/configuration.md) instead+of copying minikube's choices.++**Environment-specific values base hard-codes or leaves unset.**++| Setting | Base value | What to do |+| ----------------------------------------------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| API server `cidr` in `allow-k8s-api-egress-control-plane`, `-mcp-host`, `-channels` | `10.109.0.1/32` | Patch to your API server endpoint IP (`kubectl get endpoints kubernetes -n default`). The minikube template [`k8s-api-ip.yaml.template`](../../deploy/overlays/minikube/patches/k8s-api-ip.yaml.template) explains why this is an `ipBlock`. |+| `CONTEXT_MAPPER_HOST_WORKSPACE_STORAGE_CLASS` (HCC) | `do-block-storage-retain` | Set to a block StorageClass in your cluster. |+| `storageClassName` on `control-postgres-data` and `clerum-workflow-output` | unset | Patch, or rely on your default StorageClass. |+| `CONTEXT_MAPPER_HOST_IMAGE_PULL_SECRET` (HCC) | `clerum` | Name your pull Secret in `mcp-host`, or set `""` if none is needed (minikube sets `""`). |+| `CONTEXT_MAPPER_ENFORCE_IMAGE_ALLOWLIST` (HCC) | `"false"` (audit) | Set `"true"`, and narrow `CONTEXT_MAPPER_ALLOWED_IMAGE_PREFIXES` (base includes `clerum/`). |++For a read-only check of API-server and DNS egress before you rely on+enforcement, see [`deploy/scripts/np-enforce-preflight.sh`](../../deploy/scripts/np-enforce-preflight.sh).+It was written for GKE; read it before you run it elsewhere.++## Install order++[minikube.md](minikube.md) and `scripts/minikube/full-setup.sh` show the full+dependency graph. The minikube scripts are bound to a minikube profile; the+scripts under [`deploy/scripts`](../../deploy/scripts) take a `CONTEXT` and are+the ones to use elsewhere. Several of them also require `ALLOWED_CONTEXTS`, an+exact allowlist of kube-contexts they may touch.++1. **CRDs.**++ ```bash+ helm install clerum-crds ./charts/clerum-crds+ # After chart upgrades, re-apply the CRD YAML (Helm 3 does not upgrade crds/):+ kubectl apply -f ./charts/clerum-crds/crds/+ ```++2. **Namespaces.** The key scripts write into existing namespaces, so create+ them first:++ ```bash+ kubectl apply -f deploy/base/namespaces.yaml+ ```++ Your overlay apply (step 4) adds the deny-all baselines, Roles and+ cluster-wide objects. If your CD identity may not patch Roles, run+ [`deploy/scripts/bootstrap-rbac.sh`](../../deploy/scripts/bootstrap-rbac.sh)+ once with an admin kubeconfig.++3. **Keys and secrets.** Run+ [`deploy/scripts/gen-jwt-keys.sh`](../../deploy/scripts/gen-jwt-keys.sh)+ with `CONTEXT=<your-context>`. It creates the four RSA-4096 key pairs+ (session, rpc, admin, wrc), the OAuth state HMAC and encryption key, the+ Control UI CSRF secret, the prompt-history encryption key, and the+ Secrets and public-key ConfigMaps that carry them. Then run+ [`deploy/scripts/apply-inter-service-tokens.sh`](../../deploy/scripts/apply-inter-service-tokens.sh)+ for the service-to-service tokens. Know these before you run them:+ - `gen-jwt-keys.sh` generates **new** keys on every run. Running it again+ rotates every key, including the OAuth encryption key. Run it once and+ keep the output.+ - Set `ADMIN_BOOTSTRAP_PASSWORD_HASH` (a bcrypt hash) before you run it.+ Otherwise it writes a placeholder hash.+ - It writes the `control-postgres` Secret with user `postgres` and password+ `postgres`. Replace it before Postgres first starts.+ - Base ships placeholder values for the `control-api-public-key` and+ `clerum-wrc-public-key` ConfigMaps, and `mcp-host-config` must carry the+ RPC public key as `CLERUM_AUTH_JWT_PUBLIC_KEY`. A later+ `kubectl apply -k` resets anything your overlay still holds as a+ placeholder, so put the real public keys in your overlay. (Minikube+ re-runs its key script after every apply for this reason.)+ - `apply-inter-service-tokens.sh` creates any missing Secret and then+ writes with `kubectl patch`, so later `kubectl apply -k` runs do not wipe+ the tokens. It keeps existing values. Minikube runs it again after every+ overlay apply; do the same. Outside minikube it+ refuses to run without `CONTROL_API_MEMBER_REGISTRATION_HMAC_SECRET` (or+ an existing value in the Secret).+ - Secrets for LLM keys and channel credentials are separate; see+ [LLM providers (operator guide)](llm-providers.md).++4. **Apply your overlay** (`kubectl apply -k <your-overlay>`), with the+ ghcr component pinned to one release tag.++5. **Database migrations and roles.** Run+ [`run-control-api-db-migration.sh`](../../deploy/scripts/run-control-api-db-migration.sh)+ `--overlay <your-overlay>`, then+ [`provision-control-api-runtime-roles.sh`](../../deploy/scripts/provision-control-api-runtime-roles.sh).+ If you use the Global File System, follow+ [GFS permission-store credential lifecycle](gfs-permission-store.md)+ (`provision-gfs-db.sh`, `provision-gfs-runtime.sh`; production contexts+ need `--allow-prod`).++6. **First admin.** control-api creates the bootstrap admin from+ `CONTROL_API_ADMIN_BOOTSTRAP_USERNAME`, `CONTROL_API_ADMIN_BOOTSTRAP_EMAIL`+ and `CONTROL_API_ADMIN_BOOTSTRAP_PASSWORD_HASH` when it initializes the+ schema, and only while no other active admin exists. The control-api image+ runs with `NODE_ENV=production`, so it will not start without the hash.+ It does start with the placeholder hash `gen-jwt-keys.sh` writes when+ `ADMIN_BOOTSTRAP_PASSWORD_HASH` is unset; that placeholder is not a usable+ password.+ The minikube seed scripts are test tooling; do not use them here.++7. **UIs and the Desktop App.** `control-ui` and `profile-ui` are in base. The+ Desktop App is distributed to users, not deployed; see+ [Ship it to your users](../surfaces/desktop-app.md#ship-it-to-your-users).++8. **First Host, Context and channel.** Start with the smallest declarative+ config. See the [CRD reference](../crds/README.md).++9. **Connectors.** Add McpServers on an allowlisted Context; see+ [Add an MCP server](../how-to/add-mcp-server.md).++10. **Member invitations.** Decide how invitation email is sent before you+ invite anyone; see [member invitations on self-hosted](../how-to/member-invitations-self-hosted.md).+ The zero-config path is `CONTROL_API_MEMBER_REGISTRATION_MODE=hosted`:+ control-api enrolls itself with evenfire's registration hub, which sends+ the mail. It requires your Profile UI and Control UI base URLs to be+ **real, publicly resolvable domains** (the host guard refuses `localhost`,+ IP literals and dotless hostnames) plus egress to+ `registration.evenfire.ai`. The alternative is to run your own+ `member-registration-service` (not in this repository) and keep the+ default mode, `remote`. Minikube's `127.0.0.1` URLs cannot use hosted mode.++11. **Observability.** [`monitoring/`](../../monitoring/README.md) is an+ optional Loki and Grafana starter (Helm values and a dashboard). It is+ not part of `deploy/base`.For the generation-aware Desktop GFS rollout, read the [operator paritycompatibility contract](../architecture/gfs-desktop-operator-parity.md#rollout-and-compatibility-contract)@@ -66,20 +312,52 @@ before upgrading. The first rollout intentionally requires one sign-in perexisting external session, and direct callers of `DELETE /api/v1/members/:userId`must send both a non-empty `reason` and an `Idempotency-Key`.-Use [minikube.md](minikube.md) as the **order-of-operations reference** even when-the target is not minikube: the dependency graph is the same. One exception:-minikube's `127.0.0.1` UI defaults cannot use hosted member-registration mode,-so that step is production-only.+## Upgrades++Back up Postgres, apply the CRDs from the new checkout, then roll out every+image from the same release tag. The full order and the feature-specific+runbooks are in [releases and upgrades](releases-and-upgrades.md#upgrade-notes):++- [Workflow output PVC upgrade](workflow-output-pvc-upgrade.md)+- [Plugin Workload SDK upgrade and policy migration](plugin-workload-sdk-upgrade.md)+- [GFS permission-store credential lifecycle](gfs-permission-store.md)## Security non-negotiables- Do not run with `CLERUM_DEV_MODE=true`, and never expose `mcp-host`'s runtime- port beyond the NetworkPolicy-allowed edge services — the edge trust headers+ port beyond the NetworkPolicy-allowed edge services. The edge trust headersit accepts are forgeable by anyone who can reach the port.- Keep default-deny NetworkPolicies; only open (context, server) pairs you intend.+- Switch the connector-image allowlist to enforce+ (`CONTEXT_MAPPER_ENFORCE_IMAGE_ALLOWLIST=true`).+- Custom workflow coordinator images are off by default+ (`WRC_ENABLE_CUSTOM_COORDINATOR_IMAGE=false`). If you turn them on, keep+ `WRC_REQUIRE_COORDINATOR_IMAGE_DIGEST=true` (the default), which requires a+ sha256 digest.+- Replace every development default: the `postgres:postgres` database+ credentials, the minikube `rpc-proxy-config` secrets, and any placeholder+ key or hash.- Treat approval policy as a control, not a UX preference.- Report vulnerabilities per [SECURITY.md](../../SECURITY.md).+## What this repository does not provide++- **A production overlay.** Only minikube overlays exist. The base+ kustomization mentions GKE overlays; those are not in this repository.+- **An ingress controller, Ingress objects or certificates.** The optional+ Cloudflare Tunnel stack in `deploy/base/ingress` is the only edge+ component, and it is not included by default.+- **External secret-manager integration.** There is no Vault or External+ Secrets Operator wiring. Secrets are plain Kubernetes Secrets created by the+ scripts above.+- **Self-hosted model endpoints.** Each LLM provider has a fixed API endpoint+ (Azure OpenAI takes `AZURE_OPENAI_ENDPOINT`), and agent egress excludes+ private address ranges. A model server on your private network is not+ reachable as shipped.+- **Highly available Postgres or backups.** Base runs one Postgres replica.+- **The member-registration service.** Use hosted mode or run your own.+- **Turnkey cloud-vendor guides.**+## Licenseevenfire is open source under **MPL-2.0**: self-hosting, modification, and@@ -88,8 +366,10 @@ modified MPL-licensed files. See [LICENSE](../../LICENSE).## Related+- [Releases and upgrades](releases-and-upgrades.md)+- [Configuration reference](../reference/configuration.md)+- [Minikube setup](minikube.md)- [WorkflowRecipes operations](workflow-recipes-guide.md)-- [Plugin Workload SDK upgrade and policy migration](plugin-workload-sdk-upgrade.md)- [Member invitations on self-hosted](../how-to/member-invitations-self-hosted.md)- [Architecture / topology](../architecture/platform-topology.md)- [FAQ](../faq.md)diff --git a/docs/deploy/releases-and-upgrades.md b/docs/deploy/releases-and-upgrades.mdnew file mode 100644index 000000000..fec52f429--- /dev/null+++ b/docs/deploy/releases-and-upgrades.md@@ -0,0 +1,230 @@+# Releases and upgrades++How evenfire is versioned and released, how to run a specific release, and what+to read before you upgrade.++## Version numbers++evenfire uses two kinds of version number. They mean different things.++| Version | Where it lives | What it means |+| --------------------------- | ----------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |+| **Release** `vX.Y.Z` | Git tag, image tag `ghcr.io/evenfire-ai/<image>:vX.Y.Z` | The repository release. Released images, the Desktop App, and the ghcr image pin all carry this one number. |+| **Desktop App** `X.Y.Z` | `desktop-app/package.json` | Always equal to the release version. Only the release script writes it. |+| **Service counter** `0.1.N` | `version` in each service's `package.json` (`external-rest-api`, `rpc-proxy`, `control-api`, …) | A per-service change counter. The pre-commit hook bumps the patch number on every commit that touches that service. It is not a release number. |++So `v0.10.0` is a release, and `external-rest-api` `0.1.100` only means that+service has changed about a hundred times. Do not compare service counters+across services, and do not use them to pick a release.++### How service counters are bumped++The tracked pre-commit hook (`.githooks/pre-commit`) runs+`npm run version:staged`. For every package root that has staged changes and+whose `package.json` version still equals `HEAD`, it raises the patch number by+one and stages `package.json` and `package-lock.json`. Run+`make install-git-hooks` (or `npm run install-git-hooks`) once to enable the+hook.++Two counters are also published to clients. `external-rest-api/src/releaseManifest.ts`+declares `externalRestApiVersion` and `rpcProxyVersion`, so the hook regenerates+and stages that file when either package is bumped. CI checks both rules on+every pull request:++- `update-desktop-release-manifest.mjs --validate-only` fails if the manifest+ disagrees with the package versions.+- `validate-release-version-bumps.mjs` fails if code under `external-rest-api/`+ or `rpc-proxy/` changed but that package's version did not.++`desktop-app` is not a hook package root, so the hook never bumps it. See+[CONTRIBUTING.md](../../CONTRIBUTING.md#the-pre-commit-hook-may-edit-your-commit)+for what the hook does to your commits.++## How a release is made++Images are built once, on the `dev` branch. A release never rebuilds them. It+copies the exact images that `dev` already built to the release tag.++```text+push to dev ──► build-publish.yml ──► ghcr.io/evenfire-ai/<image>:sha-<7> and :latest+ (multi-arch: linux/amd64 + linux/arm64)++prepare-release.mjs ──► release-prep commit ──► tag vX.Y.Z+ │+ ▼+ release-images.yml ──► :vX.Y.Z (and :stable)+```++1. **Continuous builds.** On every push to `dev`,+ [`build-publish.yml`](../../.github/workflows/build-publish.yml) rebuilds only+ the images whose source paths changed. Each image is built natively for+ `linux/amd64` and `linux/arm64` and merged into one manifest list tagged+ `sha-<7 chars>` and `latest`. The manifest list records the commit it was+ built from in the `org.opencontainers.image.revision` annotation. Pushes to+ `main` do not build images.+2. **Cut the release.** A maintainer runs:++ ```bash+ node scripts/release/prepare-release.mjs --version X.Y.Z --release-id vX.Y.Z \+ [--minimum-desktop-version A.B.C]+ ```++ It refuses a version that is not greater than the last committed Desktop App+ version. It writes every release coordinate (listed in+ [`release-coordinates.mjs`](../../scripts/release/release-coordinates.mjs)):++ | Coordinate | Rule |+ | ------------------------------------------------------------ | ---------------------------------------------------------------------------------- |+ | `desktop-app/package.json` and `package-lock.json` | Equals `X.Y.Z` |+ | `releaseManifest.desktopVersion` | Equals `X.Y.Z` |+ | `releaseManifest.releaseId` | Set explicitly (never `local`) |+ | `releaseManifest.minimumDesktopVersion` | A floor, at most `X.Y.Z`. Changes only when `--minimum-desktop-version` is passed. |+ | `releaseManifest.externalRestApiVersion` / `rpcProxyVersion` | Equal their own `package.json` counters |+ | `newTag` of every row in `deploy/components/ghcr-images` | Equals `vX.Y.Z` |++ `node scripts/release/validate-release-tag.mjs --version X.Y.Z` checks that+ all coordinates agree. It does not check that the images exist, because they+ are only created when the tag is pushed.++3. **Tag.** Pushing a `v*` tag to `evenfire-ai/evenfire` starts+ [`release-images.yml`](../../.github/workflows/release-images.yml). For each+ published image in [`deploy/images.json`](../../deploy/images.json),+ [`resolve-release-images.mjs`](../../scripts/release/resolve-release-images.mjs)+ resolves `:latest` to a digest once, then refuses the image if:+ - it has no revision annotation;+ - the revision it was built from is not an ancestor of the tagged commit;+ - its source paths changed between that revision and the tagged commit.++ [`promote-release-images.sh`](../../scripts/release/promote-release-images.sh)+ then checks that the digest is a manifest list with both `amd64` and `arm64`,+ and copies it (with `crane copy`) to `:vX.Y.Z`. For a strict `vX.Y.Z` tag it+ also copies it to `:stable`. If any image is refused, the job fails, but the+ other images are still copied, so a failed run can leave a partial release.+ A manual+ `workflow_dispatch` run is a dry run: it prints what it would promote and+ copies nothing.++The hosted evenfire service deploys from these same images through a separate,+private pipeline. That pipeline is not part of this repository.++### What is fixed and what moves++| Tag | Moves? |+| --------------------- | -------------------------------------------------------------------------------------- |+| `:latest`, `:sha-<7>` | `:latest` moves on every `dev` build of that image. `:sha-<7>` names one build. |+| `:vX.Y.Z` | Written by the release workflow for that tag. Nothing in normal operation rewrites it. |+| `:stable` | Moves to the newest strict `vX.Y.Z` release. |+| `@sha256:…` digest | Never moves. Use a digest when you need a byte-for-byte guarantee. |++The release workflow has no guard that stops a second run from writing+`:vX.Y.Z` again. If you need an image that cannot change, pin the digest.++## Run a specific release++### Minikube++`make minikube-setup` pulls images from `ghcr.io/evenfire-ai` by default+(`IMAGE_SOURCE=ghcr`). The `minikube-ghcr` and `minikube-no-uis-ghcr` overlays+add the [`ghcr-images`](../../deploy/components/ghcr-images/kustomization.yaml)+component. It rewrites the 25 images that are both published and deployed to+minikube from their local `clerum/<name>` refs to+`ghcr.io/evenfire-ai/<name>:vX.Y.Z`. It also rewrites image refs inside+Deployment env values, such as the images HCC uses to start child pods.++To run a release, check out its tag and run setup:++```bash+git checkout vX.Y.Z+make minikube-setup+```++`MINIKUBE_IMAGE_TAG` overrides the pinned tag for one run. It edits a temporary+copy of the deploy tree, never the committed file:++```bash+MINIKUBE_IMAGE_TAG=latest make minikube-setup # newest dev builds+MINIKUBE_IMAGE_TAG=v0.9.0 make minikube-setup # an older release+```++On a branch where the release-prep commit has landed but the tag has not been+promoted yet, the pin names images that do not exist yet. That is why+[the quickstart](../get-started/quickstart.md) uses `MINIKUBE_IMAGE_TAG=latest`.+To build every image from source instead, use `make minikube-setup-local`.+See [minikube.md](minikube.md) for the full setup.++### Your own cluster++[`deploy/base`](../../deploy/base) refers to images by local `clerum/<name>`+names. To run published images, rewrite them in your own kustomize overlay:++- include `deploy/components/ghcr-images` as a component, which pins the 25+ minikube-deployed images to the release tag of the commit you check out; or+- write your own `images:` block with `newName: ghcr.io/evenfire-ai/<name>` and+ a `digest:` per image.++Do not deploy `:latest` or `:stable` to a cluster you care about. Every image+in a cluster should come from the same release.+[`deploy/images.json`](../../deploy/images.json) lists every image, whether it+is published (`published`), and whether minikube deploys it+(`deployed_to_minikube`). Images such as `mcp-host-desktop`,+`clerum-workflow-base`, and `playwright-server` are published but are not in+the component.++The CRD Helm chart ([`charts/clerum-crds`](../../charts/clerum-crds)) has its+own chart version. It does not follow the release number. Always apply the CRDs+from the same checkout as your images.++## Desktop App versions++The Desktop App version is the release version, and its GitHub release tag is+the same `vX.Y.Z` tag. There is no separate `desktop-app-<version>` tag.++The External REST API serves `GET /desktop/release` (authenticated). It returns+the values in `releaseManifest.ts` plus `releaseTag` (`v<desktopVersion>`) and+`releaseUrl`. `releaseUrl` is built from `EXTERNAL_REST_API_DESKTOP_RELEASE_BASE_URL`,+which defaults to `https://github.com/evenfire-ai/evenfire/releases`. Point it+at your own releases page if you ship your own builds.++When the installed app is older than `minimumDesktopVersion`, the app shows+**Desktop update required** and links to `releaseUrl`. See+[Use the Desktop App](../how-to/use-the-desktop-app.md).++No workflow in this repository builds or signs Desktop App installers. To build+one yourself, use the Electron Forge scripts in `desktop-app/package.json`+(`make:mac:arm64`, `make:mac:x64`, `make:windows:x64`, `make:windows:arm64`,+`make:linux:x64`, `make:linux:arm64`) with Node 24, after `npm ci` and+`npm run verify:electron`.++## Upgrade notes++Read the notes for every release you skip over, not only the target release.++### General order++1. **Back up the database** (`control-postgres`) before any upgrade.+2. **Apply the CRDs first.**++ ```bash+ kubectl apply -f ./charts/clerum-crds/crds/+ ```++ `helm upgrade` does not update files in a chart's `crds/` directory, so you+ must apply them yourself. The order matters: the API server drops fields+ that the installed CRD schema does not know, without an error. If new+ `control-api` or `mcp-host` pods write a new field (for example+ `spec.guardrails` or `spec.llmPolicy` on a Host) while the old CRD is still+ installed, that field is silently lost. The `ROLLOUT ORDER — CRD FIRST`+ comments in+ [`charts/clerum-crds/crds/host.yaml`](../../charts/clerum-crds/crds/host.yaml)+ mark these fields. See also the [CRD reference](../crds/README.md).++3. **Then roll out the images** for the new release, all from the same tag.+4. **Follow any feature-specific runbook** below.++### Feature-specific notes++| Note | Read it when |+| ---------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |+| [Workflow output PVC upgrade](workflow-output-pvc-upgrade.md) | You use WorkflowRecipes with `spec.output.destination: pvc`. The default PVC is now one per parent recipe. |+| [Plugin Workload SDK upgrade and policy migration](plugin-workload-sdk-upgrade.md) | You use the Plugin Workload SDK. It has a required order with the SDK turned off during the upgrade. |+| [GFS permission-store credential lifecycle](gfs-permission-store.md) | You run the Global File System. It covers provisioning, rotating, and recovering the `gfs_controller` credential. |diff --git a/docs/get-started/dev-mode.md b/docs/get-started/dev-mode.mdnew file mode 100644index 000000000..a45cc1afc--- /dev/null+++ b/docs/get-started/dev-mode.md@@ -0,0 +1,142 @@+# Single-service dev mode++Dev mode runs `mcp-host` (the agent runtime) as a plain Node process, with no+Kubernetes cluster. Use it to iterate on the agent loop, a model provider, a+native tool, guardrails or an MCP server.++> **Dev mode is not full platform security.** It turns off JWT verification+> and runs without NetworkPolicies, CRDs, host-context-controller, control-api+> or the UIs. Do not expose a dev-mode process to a network you do not trust.+> For the full platform, use the [Quickstart](quickstart.md).++## Run mcp-host locally++You need Node.js 24 and an API key for one+[supported provider](../llm-providers/README.md).++```bash+cd mcp-host+npm ci+npm run build++CLERUM_DEV_MODE=true \+CLERUM_ENABLE_AUTH=false \+OPENAI_API_KEY=sk-... \+npm start+```++`npm start` runs `node dist/main.js`. The server listens on port `8080`+(`CLERUM_SERVER_PORT` changes it). Check it is up:++```bash+curl -s http://localhost:8080/v1/runtime/health+# {"status":"ok"}+```++`CLERUM_ENABLE_AUTH=false` is required. Auth defaults to on, and with auth on+mcp-host refuses to start unless `CLERUM_AUTH_JWT_PUBLIC_KEY` is set:+`FATAL: enableAuth=true but CLERUM_AUTH_JWT_PUBLIC_KEY is not set`.++## Choose the model++In dev mode there is no Host CRD and no Kubernetes Secret. The Host spec and+the API keys come from environment variables:++| Variable | Effect |+| ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |+| Provider key (`OPENAI_API_KEY`, `CLAUDE_API_KEY`, `ZAI_API_KEY`, `BAILIAN_API_KEY`, ...) | At least one is required, or startup fails. The full list is in [mcp-host Supported LLM Providers](../../mcp-host/README.md#supported-llm-providers). |+| `CLERUM_MODEL_PROVIDER` | Provider id to use. An unknown id stops startup. Unset: the first provider whose key is present is used, with its default model. |+| `CLERUM_MODEL_NAME` | Model name. Used only together with `CLERUM_MODEL_PROVIDER`. |+| `CLERUM_HOST_CONFIG` | A full Host spec as JSON. Takes precedence over the two variables above. |+| `CLERUM_HOST_NAME` | Host name. Defaults to `dev-host`. |++## Send a message++`GET /v1/runtime`, `GET /v1/runtime/health`, `GET /v1/runtime/live` and+`GET /metrics` are open. The other runtime routes (messages, approvals, status,+task results and the rest) check the edge headers that rpc-proxy and channel-reader set in a real+deployment. Dev mode keeps that check, so a bare request gets+`401 {"error":"Missing runtime edge caller context"}`.++To call mcp-host directly, send the headers channel-reader would send. The+channel headers must match the body, and `x-clerum-edge-host-ref` must match+the Host name (`dev-host` by default):++```bash+curl -s -X POST http://localhost:8080/v1/runtime/messages \+ -H 'content-type: application/json' \+ -H 'x-clerum-edge-caller: channel-reader' \+ -H 'x-clerum-edge-host-ref: dev-host' \+ -H 'x-clerum-edge-channel-type: http' \+ -H 'x-clerum-edge-channel-id: local' \+ -H 'x-clerum-edge-sender: me' \+ -d '{"content":"hi","channelType":"http","channelId":"local","sender":"me","hostRef":"dev-host"}'+```++The call waits for the turn to finish and returns the reply. Add `?async=true`+to get a `taskId` back at once and poll `GET /v1/runtime/tasks/:taskId/result`+instead. See the+[runtime REST API](../../mcp-host/README.md#runtime-rest-api-and-rpc-callback-mapping)+for the other routes.++## What works++- **The agent loop and native tools.** Tool calls run on your machine, as+ your user. The memory workspace defaults to `./workspace` (relative to where+ you start mcp-host) instead of `/workspace`; `CLERUM_MEMORY_WORKSPACE_PATH`+ changes it.+- **Approvals.** The approval gate is on by default. Set the policy with+ `CLERUM_APPROVAL_CONFIG` (the JSON form of `Host.spec.approval`). Without+ it, the approval resolver allows no approver, and `CLERUM_APPROVAL_TIMEOUT`+ defaults to `0` (never time out), so a tool that needs approval waits+ forever. For local iteration, either set `CLERUM_APPROVAL_CONFIG` or turn+ the gate off with `CLERUM_ENABLE_APPROVAL=false`. Decisions go to+ `POST /v1/runtime/approvals/approve` and `POST /v1/runtime/approvals/deny`,+ which need the same edge headers as messages. See+ [Configure approvals](../how-to/configure-approvals.md).+- **Guardrails.** Set `CLERUM_GUARDRAILS_CONFIG` to the JSON form of+ `Host.spec.guardrails`. See+ [Configure guardrails](../how-to/configure-guardrails.md).+- **MCP servers.** There is no host-context-controller to discover them, so+ list them in `CLERUM_MCP_SERVERS`. See+ [Add an MCP server, Option A](../how-to/add-mcp-server.md#option-a--mcp-host-dev-mode-contributors-no-cluster).+- **A chat channel.** channel-reader also has a dev mode that reads its+ channel config from `CLERUM_CHANNEL` and needs a real bot token. Point it at+ mcp-host with `CLERUM_MCP_HOST_URL`, and set `CLERUM_HOST_REF` to the+ mcp-host Host name (`dev-host` by default). channel-reader's own dev+ default is `dev`, which mcp-host rejects with `403`. See+ [channel-reader](../../channel-reader/README.md#dev-mode).++Each of these variables is read at startup, so restart mcp-host after you+change one. The full list is in the+[mcp-host environment reference](../../mcp-host/README.md#environment-variables).++## What does not work++| Missing | Why |+| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |+| JWT verification | `CLERUM_ENABLE_AUTH=false`, and there is no control-api or rpc-proxy in front of mcp-host to issue or check tokens. |+| NetworkPolicies | There is no cluster. Native tools and MCP servers reach whatever your machine can reach. |+| CRDs and controllers | No Host, Context, McpServer or WorkflowRecipe objects, and no host-context-controller. Changing them needs the platform. |+| Secrets | Provider keys are plain environment variables. |+| control-api, Control UI, the Desktop App | They need the full platform. Users, usage, OAuth connectors and LLM subscriptions are managed through control-api. |++## Run it in Docker++`mcp-host/Dockerfile` builds from the repository root, because mcp-host links+shared packages from `packages/`:++```bash+# from the repository root+docker build -f mcp-host/Dockerfile -t mcp-host-dev .+docker run --rm -p 8080:8080 \+ -e CLERUM_DEV_MODE=true \+ -e CLERUM_ENABLE_AUTH=false \+ -e OPENAI_API_KEY=sk-... \+ mcp-host-dev+```++The root `docker-compose.yml` declares the same dev-mode settings, plus an+optional channel-reader under the `telegram` profile. Its mcp-host build does+not work as written: it uses `context: ./mcp-host`, so the Dockerfile cannot+find `packages/`. Use the commands above instead.diff --git a/docs/get-started/quickstart.md b/docs/get-started/quickstart.mdindex 9a99b1e32..0f807ed87 100644--- a/docs/get-started/quickstart.md+++ b/docs/get-started/quickstart.md@@ -68,7 +68,7 @@ CLERUM_MODEL_PROVIDER=openai # optional with one key: openai | claude | zai## 2. Set up the cluster (one command)```bash-MINIKUBE_IMAGE_TAG=latest make minikube-setup # see the override note below+make minikube-setup```This runs an idempotent 12-step setup: minikube cluster (Calico) → namespaces →@@ -81,11 +81,11 @@ verify.instead of compiling anything. MCP servers are not loaded into the cluster atall — they are installed on demand from the evenfire registry.-`MINIKUBE_IMAGE_TAG=latest` is required today. The cluster manifests pin the-next release tag, whose images are created by promotion _on_ that tag, so the-pin currently names something that does not exist yet. Without the override the-pull fails with a message naming the image, the tag, and this variable. Drop the-override once that release is tagged and promoted.+The manifests pin a release tag. Right after a release bump, that tag's images+may not be promoted yet, and the pull fails with a message naming the image, the+tag, and `MINIKUBE_IMAGE_TAG`. In that case run+`MINIKUBE_IMAGE_TAG=latest make minikube-setup` to use the newest `dev` images+instead. See [Releases and upgrades](../deploy/releases-and-upgrades.md).To build every image from source instead, use `make minikube-setup-local`(equivalently `IMAGE_SOURCE=local`). That path needs no tag override.@@ -191,7 +191,7 @@ Details: [Connect Telegram](../how-to/connect-telegram.md).| Symptom | Fix || ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- || Agent never replies | No real LLM key in `.env`; with several keys, set `CLERUM_MODEL_PROVIDER` explicitly (see step 1) |-| `minikube start` fails on memory | Raise Docker Desktop to ≥10 GB RAM / 6 CPUs — or, if you can't spare it, `MINIKUBE_MEMORY=9216 MINIKUBE_IMAGE_TAG=latest make minikube-setup` (stock Docker Desktop's ~9.9 GB is just under the 10 GB default) |+| `minikube start` fails on memory | Raise Docker Desktop to ≥10 GB RAM / 6 CPUs — or, if you can't spare it, `MINIKUBE_MEMORY=9216 make minikube-setup` (stock Docker Desktop's ~9.9 GB is just under the 10 GB default) || Pods `Pending` early on | Calico is still coming up — wait, then `make minikube-status` || postgres CrashLoopBackOff after cold start | `make minikube-setup ARGS="--reset-db --skip-build"` || Port-forwards die | Re-run `make minikube-pf-all` (it holds them open; Ctrl-C stops) |diff --git a/docs/how-to/add-mcp-server.md b/docs/how-to/add-mcp-server.mdindex c331a868b..e142c6f26 100644--- a/docs/how-to/add-mcp-server.md+++ b/docs/how-to/add-mcp-server.md@@ -6,7 +6,8 @@ platform enforces that with NetworkPolicies on Kubernetes.## Option A — mcp-host dev mode (contributors, no cluster)-When running `mcp-host` standalone (`CLERUM_DEV_MODE=true npm run dev`), wire+When running `mcp-host` standalone (`CLERUM_DEV_MODE=true CLERUM_ENABLE_AUTH=false npm run dev`; see+[Dev mode](../get-started/dev-mode.md)), wireservers with `CLERUM_MCP_SERVERS` (JSON array). Each entry needs `name`,`contextRef`, `transport`, `enabled`, and`status: { deployed: true, ready: true }` or it is skipped as not ready:diff --git a/docs/how-to/desktop-setup-and-updates.md b/docs/how-to/desktop-setup-and-updates.mdindex 694aa8d92..3a1e84fb2 100644--- a/docs/how-to/desktop-setup-and-updates.md+++ b/docs/how-to/desktop-setup-and-updates.md@@ -91,12 +91,13 @@ update policy, so different tenants can be on different versions at the sametime. It asks that tenant's External REST API for its release info, including a`minimumDesktopVersion`. If the installed version is **below** that minimum, theapp shows a **Desktop update required** dialog; **Download update** opens the-release page for the new build (tag `desktop-app-<version>`) in your browser. (A packaged distribution is+release page for the new build (release tag `v<desktopVersion>`) in your browser. (A packaged distribution iswhere this matters most — a source checkout can just pull the latest.)This is update **detection**, not silent auto-install: the app tells the user an-update is required and links the release. Code signing, notarization, and-automatic download/install are not part of this repo — see+update is required and links the release; it never downloads or installs on+its own. macOS signing and notarization are available as an opt-in build step,+and Windows and Linux builds are unsigned — see[Ship it to your users](../surfaces/desktop-app.md#ship-it-to-your-users).## Troubleshootingdiff --git a/docs/llms.txt b/docs/llms.txtindex 22983a256..a898d7efc 100644--- a/docs/llms.txt+++ b/docs/llms.txt@@ -9,6 +9,7 @@- README: ../README.md — product pitch, security model, providers, license- Quickstart: get-started/quickstart.md — full platform on minikube, one command- Registry quickstart: get-started/registry-quickstart.md — self-hosted connect → install → publish round trip+- Dev mode (mcp-host without Kubernetes): get-started/dev-mode.md- Learning path: get-started/learning-path.md- FAQ: faq.md@@ -61,6 +62,8 @@- Minikube full stack (in-depth reference): deploy/minikube.md- Production checklist: deploy/production.md- WorkflowRecipes ops: deploy/workflow-recipes-guide.md+- Releases and upgrades (versions, release images, pinning, upgrade order): deploy/releases-and-upgrades.md+- Monitoring and observability: operations/monitoring.md- Upgrade: workflow output PVC ownership: deploy/workflow-output-pvc-upgrade.md- Upgrade: Plugin Workload SDK (promptBridge / clientNotifications): deploy/plugin-workload-sdk-upgrade.md- GFS permission store credential (provision, rotate, recover): deploy/gfs-permission-store.mddiff --git a/docs/operations/monitoring.md b/docs/operations/monitoring.mdnew file mode 100644index 000000000..3c342e7f7--- /dev/null+++ b/docs/operations/monitoring.md@@ -0,0 +1,211 @@+# Monitoring++This page is for operators. It covers what evenfire ships for logs, metrics,+health checks and tracing, and what it does not ship.++Short version:++- `deploy/base` ships no monitoring stack. There is no Prometheus, no+ `ServiceMonitor`/`PodMonitor`, and no `prometheus.io/scrape` annotations.+- The optional `monitoring/` directory is a **logs-only** starter: Loki,+ Promtail and Grafana, with one log dashboard.+- Most services expose a Prometheus `/metrics` endpoint. You bring your own+ Prometheus to scrape them.+- Usage and cost live in control-api and the Control UI (**Cost & Usage**), not+ in Grafana.++## The `monitoring/` starter++[`monitoring/`](../../monitoring/README.md) is separate from `deploy/base`. Nothing in+the kustomize tree installs it. It holds Helm values for the official Grafana+charts:++| Path | What it does |+| ---------------------------------------- | ---------------------------------------------------------------------------------------------- |+| `namespace.yaml` | Creates the `monitoring` namespace |+| `loki/values.yaml` | `grafana/loki-stack` chart `2.9.10`: Loki (100 Gi PVC) plus Promtail |+| `grafana/values.yaml` | `grafana/grafana` chart: Loki datasource at `http://loki.monitoring:3100`, 10 Gi PVC, non-root |+| `dashboards/loki-grafana-dashboard.json` | One Grafana dashboard for Kubernetes logs |+| `Makefile` | `setup`, `install_*`, `upgrade_*`, `uninstall_*`, `port_forward_grafana`, `status` |++Loki is the only datasource. The starter does not install Prometheus or any+metrics dashboard.++### Dashboard++There is one dashboard, **Loki Kubernetes Logs** (Grafana community dashboard+15141). It has two panels:++- a log-volume time series: `count_over_time` over the selected streams;+- a log panel with the matching lines.++You filter by `namespace`, `stream` and `container`, plus a free-text search.+The panels reference a fixed datasource UID that the provisioned Loki+datasource does not set. If the panels show "datasource not found", open each+panel and select the **Loki** datasource.++### Install on minikube++The values files default to the `do-block-storage` StorageClass. Minikube uses+`standard`. Loki also asks for a 100 Gi volume, which is more than most local+clusters want. Override both at install time.++The dashboard ConfigMap must exist before Grafana starts, because+`grafana/values.yaml` mounts the `grafana-dashboards` ConfigMap.++```bash+# Helm repo + monitoring namespace+make -C monitoring setup++# Dashboard ConfigMap (before Grafana)+kubectl create configmap grafana-dashboards \+ --from-file=monitoring/dashboards/loki-grafana-dashboard.json \+ -n monitoring++# Loki + Promtail, with a minikube storage class and a smaller volume+helm install loki grafana/loki-stack --version 2.9.10 -n monitoring \+ -f monitoring/loki/values.yaml \+ --set loki.persistence.storageClassName=standard \+ --set loki.persistence.size=10Gi++# Grafana+helm install grafana grafana/grafana -n monitoring \+ -f monitoring/grafana/values.yaml \+ --set persistence.storageClassName=standard++make -C monitoring status+make -C monitoring port_forward_grafana # http://localhost:3000+```++The admin user is `admin`. The values file does not set a password. Set+`adminPassword` with `--set` or an existing Secret, or read the one the chart+generates. On a cluster whose StorageClass matches the values files, the plain+`make -C monitoring install_loki` and `install_grafana` targets work as-is.++## Logs++All first-party services write logs to stdout, so Promtail (or any collector)+picks them up. The format differs by service:++| Service | Format | Level variable |+| ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | -------------- |+| control-api | Pino JSON, `svc: control-api`, ISO timestamps, token paths removed | `LOG_LEVEL` |+| workspace-files-controller | Pino and Pino HTTP JSON | |+| codex-llm-proxy, grok-llm-proxy | Pino JSON | |+| mcp-host | JSON lines (`timestamp`, `level`, `component`, `msg`), sensitive keys redacted | `LOG_LEVEL` |+| host-context-controller | JSON lines from its own logger, sensitive keys redacted | `LOG_LEVEL` |+| workflow-recipes (controller and coordinator), workflow-approval-request-reader | JSON lines from their own loggers | |++Other services may still print plain text. Don't assume every line is JSON.+The repository logging standard is in [CLAUDE.md](../../CLAUDE.md) under+"Logging standard".++## Metrics++Each endpoint below serves Prometheus text format without authentication.+None of them is wired to a scraper by default.++| Service | Where it runs | Port | Notes |+| ----------------------------- | ------------------------------------------------ | ----------------------------------- | ----------------------------------------------------------------------------------------------------------------- |+| control-api | `deploy/base` Deployment | `8090` | Node process metrics plus approvals, notifications, OAuth, rate limits, GFS upload, governed trace, LLM allowlist |+| host-context-controller (HCC) | `deploy/base` Deployment | `8081` | See [HCC metrics](../../host-context-controller/README.md#metrics) |+| mcp-host | Stamped per Host by HCC | `8080` (`CONTEXT_MAPPER_HOST_PORT`) | Guardrails, LLM allowlist and fallback, runtime auth, conversation store, plugin SDK, process metrics |+| workflow-recipes (WRC) | `deploy/base` Deployment | `8082` | Same port as its MCP endpoint |+| workflow coordinator | Per-run pod in `sandbox-recipes`, started by WRC | `8090` | Health server also serves `/metrics` |+| webhook-gateway | Per-recipe Deployment stamped by WRC | `9090` | Only when a recipe sets `spec.webhooks[]` |+| codex-llm-proxy | `deploy/base` Deployment | `9090` | See [codex-llm-proxy README](../../codex-llm-proxy/README.md) |+| grok-llm-proxy | `deploy/base` Deployment | `9090` | See [grok-llm-proxy README](../../grok-llm-proxy/README.md) |+| mcp-proxy | `deploy/base` Deployment | `8083` | |+| gfs-controller | Stamped by HCC in the `gfs` namespace | `8087` (`GFS_PORT`) | SLI counters |++webhook-proxy has a port named `metrics` on `9090`, but it only answers+`/healthz` and `/readyz`. It serves no `/metrics`.++control-api's `/metrics` is not forwarded by the nginx gateway in front of it.+Scrape the control-api pod or Service directly.++### NetworkPolicy and scraping++The cluster runs default-deny NetworkPolicies. `deploy/base` and WRC allow+ingress from a namespace named `monitoring` only to:++- codex-llm-proxy and grok-llm-proxy, port `9090`;+- webhook-gateway pods, port `9090` (WRC sets this from+ `WRC_MONITORING_NAMESPACE`, default `monitoring`);+- webhook-proxy, port `9090` (health only).++To scrape control-api, HCC, mcp-host, WRC, mcp-proxy or gfs-controller, add+your own NetworkPolicy that allows your Prometheus to reach those ports.++### Useful series++| Metric | Service | Meaning |+| ----------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------ |+| `clerum_mcp_host_guardrail_decisions_total` | mcp-host | Guardrail outcomes by `lane`, `decision`, `source`, `execution_mode`, `phase`, `outcome`, `reason_code` |+| `clerum_mcp_host_guardrail_boundary_duration_ms` | mcp-host | Guardrail evaluation latency, excluding tool or model execution |+| `clerum_llm_allowlist_missing_total` | mcp-host | Times mcp-host saw the LLM allowlist ConfigMap absent and fell back to the Host-configured model only |+| `clerum_llm_model_not_allowed_total` | mcp-host | The Host-configured model is outside the operator allowlist. Signal only; mcp-host does not reject the model |+| `clerum_llm_fallback_total` | mcp-host | mcp-host switched to a configured fallback provider/model, labelled `from`, `to`, `reason` |+| `mcp_host_runtime_auth_degraded` | mcp-host | Gauge. Runtime auth is degraded; readiness returns `503` while it is set |+| `clerum_llm_allowlist_configmap_write_failures_total` | control-api | Failed writes of the `clerum-llm-allowed-models` ConfigMap, by `phase` (`mutation`, `boot`, `sync`) |++For guardrail behaviour, see [Configure guardrails](../how-to/configure-guardrails.md).++## Health and readiness++| Service | Port | Liveness | Readiness | Readiness can fail with |+| -------------------------------- | ------ | --------------------------------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------- |+| control-api | `8090` | `/health` | `/health` | |+| host-context-controller | `8081` | `/health` (also startup) | `/ready` | `503` with `reasons[]`, for example `controller_stopped`, `mcp_watch_unsynced`, `context_watch_unsynced`, `host_watch_unsynced` |+| mcp-host | `8080` | `/v1/runtime/live` (also startup) | `/v1/runtime/health` | `503 {"status":"degraded"}` when runtime auth is degraded; liveness stays `200` |+| gfs-controller | `8087` | `/healthz` | `/readyz` | `503` with `code` `not_mounted` (drive volume missing) or `not_ready` (permission store unreachable) |+| workspace-files-controller | `8086` | `/healthz` | `/readyz` | `503` when the mount is missing, not a directory, or not writable |+| workflow-recipes | `8082` | `/health` | `/health` | |+| mcp-proxy | `8083` | `/health` (also startup) | `/ready` | |+| codex-llm-proxy, grok-llm-proxy | `9090` | `/healthz` | `/readyz` | |+| webhook-proxy | `9090` | `/healthz` | `/healthz` | |+| rpc-proxy | `8094` | `/health` | `/health` | |+| external-rest-api | `8091` | `/health` | `/health` | |+| workflow-approval-request-reader | `8098` | `/health` | `/health` | |+| Control UI | `3000` | `/` | `/` | |++For how HCC readiness works, see [HCC readiness flows](../architecture/hcc-readiness-flows.md).++## Governed tracing++Governed tracing is control-plane evidence, not an OpenTelemetry pipeline.+control-api is the only writer and reader. It records agent runs,+administrative actions and infrastructure telemetry in Postgres. evenfire does+not export traces to an external backend. See+[Governed tracing architecture](../architecture/governed-tracing.md).++The Control UI has trace pages under `/traces` (Administrative, Dashboard,+Infrastructure, Run replay). The **Traces** section is hidden from the+sidebar. You reach the pages by URL, for example `/traces/operations`.++## Usage and cost++Token usage, LLM prices, token budgets and infrastructure cost live in+control-api and the Control UI under **Cost & Usage**+(`/cost-and-usage/usage`). They are not in Grafana or the `monitoring/`+starter. Token budgets are cost control, not a security boundary. See+[claims guardrails](../meta/claims-guardrails.md).++## What to alert on++These signals exist in the code today. Thresholds are up to you.++| Signal | Why |+| --------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |+| HCC `/ready` returns `503` (pod not Ready) | Watches are unsynced or the controller stopped. Hosts and MCP servers stop reconciling. |+| gfs-controller `/readyz` returns `503` | GFS cannot serve files: volume not mounted or permission store unreachable. |+| workspace-files-controller `/readyz` returns `503` | The workspace mount is missing or read-only. |+| mcp-host `/v1/runtime/health` returns `503`, or `mcp_host_runtime_auth_degraded` is `1` | The Host's runtime auth is degraded. |+| `rate(clerum_llm_allowlist_configmap_write_failures_total[15m]) > 0` | Postgres and the delivered LLM allowlist ConfigMap have drifted. |+| `increase(clerum_llm_allowlist_missing_total[15m]) > 0` | mcp-host is running without the allowlist ConfigMap. |+| `clerum_mcp_host_guardrail_decisions_total{reason_code="hook_unavailable"}` increasing | A guardrail hook is unreachable. With `failMode: closed`, users get refusals. |+| `increase(clerum_llm_fallback_total[15m]) > 0` | Primary LLM calls are failing over. |++Because no Prometheus ships, you need your own Prometheus, the NetworkPolicies+described above, and your own alert rules to use the metric signals. Readiness+failures also show up as pods that are not Ready in `kubectl get pods`.diff --git a/docs/surfaces/desktop-app.md b/docs/surfaces/desktop-app.mdindex 5d248a773..ac4f199f3 100644--- a/docs/surfaces/desktop-app.md+++ b/docs/surfaces/desktop-app.md@@ -171,28 +171,110 @@ the frontend process, and tear every port-forward down together on exit.## Ship it to your users-This is documentation for build tooling that has existed in this repo all-along (`desktop-app/forge.config.js`) without ever being written up.--`electron-forge` is already configured with these makers:+`electron-forge` (`desktop-app/forge.config.js`) is configured with these+makers:- **zip** — macOS, Windows, and Linux- **dmg** — macOS-- **squirrel** — Windows+- **squirrel** — Windows (`EvenfireSetup.exe`)- **deb** and **rpm** — Linux-Six `npm` scripts in `desktop-app/package.json` drive them, each running-`electron-forge make --platform=<platform> --arch=<arch>` (packaging always-runs `npm run build` first, via the forge `prePackage` hook):+### Prepare the toolchain-| Script | Platform | Arch |-| ------------------- | -------- | ----- |-| `make:mac:x64` | darwin | x64 |-| `make:mac:arm64` | darwin | arm64 |-| `make:windows:x64` | win32 | x64 |-| `make:windows:arm64`| win32 | arm64 |-| `make:linux:x64` | linux | x64 |-| `make:linux:arm64` | linux | arm64 |+Packaging uses the same guard as every other Desktop build. Use Node 24+(`engines.node` is `24.x`), install with `npm ci` **without**+`--ignore-scripts` so Electron's `postinstall` downloads the runtime, then run+the check:++```bash+cd desktop-app+npm ci+npm run verify:electron # fails unless Node is 24.x and `electron` resolves to a runtime+```++A skipped `postinstall` still lets `npm ci` exit 0; `verify:electron` is what+catches it. If it reports that Electron is not installed, switch to Node 24 and+rerun `npm ci`. Do not set `ELECTRON_SKIP_BINARY_DOWNLOAD`.++### Build installers++Six `npm` scripts in `desktop-app/package.json` drive the makers, each running+`electron-forge make --platform=<platform> --arch=<arch>`. Packaging runs+`npm run build` first through the forge `prePackage` hook (set+`EVENFIRE_SKIP_PACKAGE_BUILD=1` to reuse an existing build). `desktop-app/Makefile`+has no packaging targets — only install, build, dev, start, clean, and+port-forward helpers — so use the scripts directly:++| Script | Platform | Arch | Makers |+| -------------------- | -------- | ----- | --------------------- |+| `make:mac:x64` | darwin | x64 | zip, dmg |+| `make:mac:arm64` | darwin | arm64 | zip, dmg |+| `make:windows:x64` | win32 | x64 | zip, squirrel |+| `make:windows:arm64` | win32 | arm64 | zip, squirrel |+| `make:linux:x64` | linux | x64 | zip, deb, rpm |+| `make:linux:arm64` | linux | arm64 | zip, deb, rpm |++Output lands in `desktop-app/out/` (git-ignored). Run the macOS scripts on a+macOS host: signing, notarization, and the adaptive-icon step (`xcrun actool`)+are all gated on the **build host** being macOS. The adaptive-icon+step is best-effort and falls back to the static icon unless+`EVENFIRE_REQUIRE_ADAPTIVE_ICONS=1`.++The package is built with ASAR integrity validation, `OnlyLoadAppFromAsar`, and+the `RunAsNode`, `NODE_OPTIONS`, and inspector-argument fuses turned off.++### Signing and notarization++| Platform | What the config supports |+| -------- | --------------------------------------------------------------------------------------------------------------------------- |+| macOS | Opt-in code signing and notarization, enabled only when `ENABLE_APPLE_CODESIGN=1` on a macOS build host. Off by default. |+| Windows | No signing configuration. Squirrel and zip output is unsigned. |+| Linux | No signing configuration for deb, rpm, or zip. |++macOS variables (names only — keep the values in your keychain or CI secrets):++| Variable | Effect |+| --------------------------------------------------------- | -------------------------------------------------------------------------------------------- |+| `ENABLE_APPLE_CODESIGN` | `1` turns on `osxSign` and `osxNotarize`. Any other value leaves the build unsigned. |+| `APPLE_CODESIGN_IDENTITY` | Optional signing identity passed to `osxSign`. |+| `APPLE_KEYCHAIN` | Optional keychain for signing and for keychain-profile notarization. |+| `APPLE_API_KEY`, `APPLE_API_KEY_ID`, `APPLE_API_ISSUER` | Notarize with an App Store Connect API key. All three are needed together. |+| `APPLE_KEYCHAIN_PROFILE` | Notarize with a stored `notarytool` keychain profile instead of the API key. |+| `APPLE_BUNDLE_ID` | Bundle identifier. Default `ai.evenfire.desktop`. |++With `ENABLE_APPLE_CODESIGN=1`, the build fails at config load unless one of the+two notarization credential sets is present.++**There is no published build pipeline in this repo.** No workflow in+`.github/workflows` runs `electron-forge make` or attaches installers to a+release: CI only runs `verify:electron`, the build, and the tests. (The+`mcp-host-desktop` image built there is a container image, not the Electron+app.) Signed installers are produced by whoever runs the scripts above with+their own Apple credentials.++### Versions and updates++The Desktop version is the `version` in `desktop-app/package.json`.+`scripts/release/prepare-release.mjs --version <x.y.z> --release-id <id>+[--minimum-desktop-version <x.y.z>]` bumps it together with the release+manifest in `external-rest-api/src/releaseManifest.ts`. There is no separate+desktop tag: the update link targets `v<desktopVersion>`, the platform release+tag of the release that last bumped the Desktop version.++Updates are **detected, not installed**. After sign-in the app calls the+selected environment's External REST API `GET /api/v1/desktop/release`, which+returns the manifest's `desktopVersion` and `minimumDesktopVersion`, a+`releaseTag` of `v<desktopVersion>`, and a `releaseUrl` of+`<EXTERNAL_REST_API_DESKTOP_RELEASE_BASE_URL>/tag/<releaseTag>` (default base+`https://github.com/evenfire-ai/evenfire/releases`). If the installed version is+below `minimumDesktopVersion`, the app shows **Desktop update required**;+**Download update** opens `releaseUrl` in the browser. The app has no+`autoUpdater` and never downloads or installs a build itself. Point+`EXTERNAL_REST_API_DESKTOP_RELEASE_BASE_URL` at wherever you publish your own+installers. The user-facing flow is in+[Desktop setup & updates](../how-to/desktop-setup-and-updates.md).++### Point the app at an instanceYou ship **one** packaged app, not a build per customer. A packaged build does**not** read `EXTERNAL_REST_API_BASE_URL` or `RPC_PROXY_BASE_URL` from the@@ -204,19 +286,14 @@ service, a teammate's, or local `make`-forwarded services — from the app's ownproxy address follows from that environment; there is no second URL to enter.Environments can also be pre-seeded for a fleet through Profile UI's`evenfire://desktop-environment` deep link or a `CLERUM_DESKTOP_CONFIG_PATH`-config file. Adding and switching environments, the invitation setup path, and-how a packaged app surfaces a **required** update are covered in-[Desktop setup & updates](../how-to/desktop-setup-and-updates.md).+config file. Adding and switching environments and the invitation setup path+are covered in [Desktop setup & updates](../how-to/desktop-setup-and-updates.md).Two variables **do** apply to a packaged build, read from the environment orthe bundled `.env.prod`: `MEMBER_REGISTRATION_SERVICE_BASE_URL` — which has**no working default**, so left unset it silently falls back to the placeholder`https://example.com` — and `REQUEST_TIMEOUT_MS`.-Code signing, notarization, and auto-update are **not covered here** — that-work is deferred, and this page does not say whether the produced artifacts-are signed.-## The `member-registration-service` gap`member-registration-service` is **not in this repository**. It is an@@ -245,8 +322,11 @@ above for why leaving it unset is silently wrong rather than loudly broken.- The activity stream never carries internal reasoning / chain-of-thoughtcontent, and it is read-only — message submission always goes through the`/messages` endpoint, never the stream.-- Code signing, notarization, and auto-update are out of scope for this page- — see **Ship it to your users** above.+- macOS code signing and notarization are opt-in (`ENABLE_APPLE_CODESIGN=1`+ on a macOS build host); Windows and Linux packages are unsigned. This repo+ publishes no installers, and the app only detects required updates — it+ never downloads or installs them. See+ [Ship it to your users](#ship-it-to-your-users).- For the complete environment variable reference, the E2E test suites(Vitest IPC-harness and Playwright/Electron phases), and the fullendpoint/scope table, seediff --git a/mcp-host/README.md b/mcp-host/README.mdindex 85f669311..d8aa3b581 100644--- a/mcp-host/README.md+++ b/mcp-host/README.md@@ -373,7 +373,7 @@ Available models: `qwen3-coder-plus`, `qwen3.5-plus`, `qwen3-coder-next`,**Dev mode:**```bash-CLERUM_DEV_MODE=true BAILIAN_API_KEY=sk-... CLERUM_MODEL_PROVIDER=bailian npm run dev+CLERUM_DEV_MODE=true CLERUM_ENABLE_AUTH=false BAILIAN_API_KEY=sk-... CLERUM_MODEL_PROVIDER=bailian npm run dev```Get your Coding Plan API key at: https://modelstudio.console.alibabacloud.com/@@ -434,19 +434,19 @@ The Secret name is what you reference from `Host.spec.secretRef`.```bash# OpenAI (auto-detected from API key)- OPENAI_API_KEY=sk-xxx make dev+ CLERUM_ENABLE_AUTH=false OPENAI_API_KEY=sk-xxx make dev# Claude- CLAUDE_API_KEY=sk-ant-xxx make dev-claude+ CLERUM_ENABLE_AUTH=false CLAUDE_API_KEY=sk-ant-xxx make dev-claude# Specific model- OPENAI_API_KEY=sk-xxx make dev-model MODEL=gpt-5.4-mini+ CLERUM_ENABLE_AUTH=false OPENAI_API_KEY=sk-xxx make dev-model MODEL=gpt-5.4-mini```You can also export the API key first:```bash- export OPENAI_API_KEY=sk-xxx+ export OPENAI_API_KEY=sk-xxx CLERUM_ENABLE_AUTH=falsemake dev```diff --git a/mcp-host/package-lock.json b/mcp-host/package-lock.jsonindex 1928bc87d..0e6daaf99 100644--- a/mcp-host/package-lock.json+++ b/mcp-host/package-lock.json@@ -1,12 +1,12 @@{"name": "clerum-mcp-host",- "version": "0.1.557",+ "version": "0.1.558","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "clerum-mcp-host",- "version": "0.1.557",+ "version": "0.1.558","license": "SEE LICENSE IN LICENSE","dependencies": {"@anthropic-ai/sdk": "^0.32.0",diff --git a/mcp-host/package.json b/mcp-host/package.jsonindex 2e9f5111a..17bcdd016 100644--- a/mcp-host/package.json+++ b/mcp-host/package.json@@ -1,6 +1,6 @@{"name": "clerum-mcp-host",- "version": "0.1.557",+ "version": "0.1.558","description": "MCP Host - reads Host CRD and provides LLM access via OpenAI/Claude","main": "dist/main.js","engines": {diff --git a/monitoring/README.md b/monitoring/README.mdindex 6f01df536..674cf04e1 100644--- a/monitoring/README.md+++ b/monitoring/README.md@@ -26,10 +26,20 @@ make setup# Install Loki stack (Loki + Promtail)make install_loki+# Create the dashboards ConfigMap first: Grafana's values mount it+kubectl create configmap grafana-dashboards \+ --from-file=dashboards/loki-grafana-dashboard.json \+ -n monitoring+# Install Grafanamake install_grafana```+On minikube, set the storage class to `standard` and reduce the 100 Gi Loki+volume in `loki/values.yaml` before installing. See+[Monitoring and observability](../docs/operations/monitoring.md) for what this+starter covers (logs only; no metrics scraping is shipped).+## Makefile Targets| Target | Description |@@ -58,7 +68,7 @@ The `grafana/values.yaml` configures:|------|-------------|| `dashboards/loki-grafana-dashboard.json` | Log exploration dashboard sourced from Grafana community (gnetId 15141) |-To load the dashboard, create a ConfigMap and reference it in the Grafana values:+The setup steps above create the `grafana-dashboards` ConfigMap that Grafana's values reference. To recreate it after editing the dashboard:```bashkubectl create configmap grafana-dashboards \