Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
docs(storage): flag-off recovery, F3 listing refusal, verify coverage
Alfredo Lopezf2c4c4db946e
diff --git a/docs/architecture/host-storage-layout.md b/docs/architecture/host-storage-layout.mdindex 454135ebc..657285399 100644--- a/docs/architecture/host-storage-layout.md+++ b/docs/architecture/host-storage-layout.md@@ -138,12 +138,12 @@ keep them; the database and its journals are already at the root.stderr and exits 1 before the first move, leaving the tree unchanged. The Podstays in `Init:Error`.-| Case | Line contains |-| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| F1: the PVC root is a symlink, not a directory, or resolves to `/`; `state/` or `workspace/` escapes the root or cannot be listed and searched; a database entry (`state.db`, `-wal`, `-shm`, `.pre-*.bak`) is a symlink or not a regular file; a directory that decides a collision cannot be listed | `directory is a symlink`, `is not a directory`, `escapes the workspace PVC root`, `state directory is not readable and searchable`, `workspace directory is not readable and searchable`, `SQLite file is a symlink`, `SQLite file is not a regular file`, `cannot list directory` |-| F2: more than one live database family across the root, `state/` and `workspace/` | `more than one live SQLite database` |-| F3: a name to move already exists at the root, or under both `state/` and `workspace/`; `workspace/lost+found` exists | `destination exists`, `exists under both state/ and workspace/`, `lost+found would remain` |-| F4: the PVC root, `state/`, `workspace/`, or a directory to be moved out of them is not writable by the container uid | `not writable by uid` |+| Case | Line contains |+| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| F1: the PVC root is a symlink, not a directory, or resolves to `/`; `state/` or `workspace/` escapes the root or cannot be listed and searched; a database entry (`state.db`, `-wal`, `-shm`, `.pre-*.bak`) is a symlink or not a regular file | `directory is a symlink`, `is not a directory`, `escapes the workspace PVC root`, `state directory is not readable and searchable`, `workspace directory is not readable and searchable`, `SQLite file is a symlink`, `SQLite file is not a regular file` |+| F2: more than one live database family across the root, `state/` and `workspace/` | `more than one live SQLite database` |+| F3: a name to move already exists at the root, or under both `state/` and `workspace/`; a directory that decides such a collision cannot be listed; `workspace/lost+found` exists | `destination exists`, `exists under both state/ and workspace/`, `lost+found would remain`, `cannot list directory` |+| F4: the PVC root, `state/`, `workspace/`, or a directory to be moved out of them is not writable by the container uid | `not writable by uid` |The PVC root checks of F1 run on every start, before the signature check.diff --git a/docs/deploy/host-pvc-layout-upgrade.md b/docs/deploy/host-pvc-layout-upgrade.mdindex cea246b3e..16cd07794 100644--- a/docs/deploy/host-pvc-layout-upgrade.md+++ b/docs/deploy/host-pvc-layout-upgrade.md@@ -57,15 +57,16 @@ deploy/scripts/host-layout-merge.sh inventory --context <ctx>`workspace-layout-init` and `subPath` mounts. Each `PVC` line ends with aresult:-| Result | Meaning | Action |-| ------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------ |-| `NO_SPLIT_SIGNATURE` | The tree does not carry the split-layout signature; the merge exits 0 and changes nothing | None |-| `WILL_MERGE` | The merge moves the tree to the PVC root on the Host's next start | None |-| `WILL_REFUSE_F<n> <line>` | The merge will refuse with that line and the Host will stay in `Init:Error` after the upgrade | See [Repairs](#repairs); counted in `UPGRADE_BLOCKERS` |-| `UNREADABLE <reason>` | The check Pod could not read the PVC (pull error, timeout, unexpected output) | Fix the reason and run the command again |+| Result | Meaning | Action |+| ------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------- |+| `NO_SPLIT_SIGNATURE` | The tree does not carry the split-layout signature; the merge exits 0 and changes nothing | None |+| `WILL_MERGE` | The merge moves the tree to the PVC root on the Host's next start | None |+| `WILL_REFUSE_F<n> <line>` | The merge will refuse with that line and the Host will stay in `Init:Error` after the upgrade | See [Repairs](#repairs); counted in `UPGRADE_BLOCKERS` |+| `UNREADABLE <reason>` | The check Pod could not read the PVC (pull error, timeout, unexpected output) | Fix the reason and run the command again |+| `UNKNOWN <reason>` | A Host's claim is missing from the PVC list, or a PVC labelled for a Host has another name | Find the claim the Host mounts and run the command again |The command exits 1 while `UPGRADE_BLOCKERS` is above 0, while a PVC is-`UNREADABLE`, or when the flag is set to an invalid value or to `false` with a+`UNREADABLE` or `UNKNOWN`, when Hosts exist but no PVC was inspected, or when the flag is set to an invalid value or to `false` with asplit tree present. `DESKTOP_HOSTS` repeats the export warning above.A `NO_SPLIT_SIGNATURE` result says only that the merge will not touch the@@ -77,8 +78,13 @@ signature in [Host Storage Layout](../architecture/host-storage-layout.md).Deploy the release with the flag unset or `"true"`. **Never set it to`"false"` in the upgrade deploy**: without the init, a split Host starts on anempty database at the PVC root and reports no error. `inventory` and `verify`-print `FLAG_FALSE_WITH_SPLIT_TREES=<n>` and exit 1 in that state; set the-flag back to `"true"` and the next start merges the tree.+print `FLAG_FALSE_WITH_SPLIT_TREES=<n>` and exit 1 in that state. Set the+flag back to `"true"` before the split Host starts: the next start then+merges the tree. If the Host already started with the flag off, it created a+root `state.db` next to `state/state.db`; with the flag back on, that start+refuses with F2 (`more than one live SQLite database`). Repair it as a Host+switched from stateless to stateful (see [Repairs](#repairs)): the root+family holds only the chats since the flag-off start.Running Hosts restart with the new template and merge on that start.Suspended Hosts merge on their next wake; the upgrade does not wake them.@@ -89,15 +95,15 @@ Suspended Hosts merge on their next wake; the upgrade does not wake them.deploy/scripts/host-layout-merge.sh verify --context <ctx>```-| Result | Meaning | Action |-| ------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------------------------------------------- |-| `POD_OK` | The running Host Pod's merge init exited 0 | None |-| `NO_SPLIT_SIGNATURE` | A Host without a Pod whose tree the merge will not touch | None |-| `SPLIT` | A Host without a Pod whose tree is still split | Wake the Host (open one of its chats, or `POST /api/v1/rpc/hosts/<host>/wake` on rpc-proxy) |-| `WILL_REFUSE_F<n> <line>` | A Host without a Pod whose next start will refuse | See [Repairs](#repairs) |-| `MERGE_REFUSED <line>` | The running Host Pod is in `Init:Error` with that line | See [Repairs](#repairs) |-| `MERGE_RUNNING` | The merge init has not finished | Run `verify` again |-| `UNKNOWN`, `UNREADABLE`, `UNVERIFIED` | The script could not decide | Fix the reason and run `verify` again |+| Result | Meaning | Action |+| ------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |+| `POD_OK` | The running Host Pod's merge init exited 0 and its tree carries no signature | None |+| `NO_SPLIT_SIGNATURE` | A Host without a Pod whose tree the merge will not touch | None |+| `SPLIT` | A Host whose tree is still split (with a running Pod, the tree was split again after its start) | Wake the Host (open one of its chats, or `POST /api/v1/rpc/hosts/<host>/wake` on rpc-proxy) |+| `WILL_REFUSE_F<n> <line>` | A Host whose next start will refuse | See [Repairs](#repairs) |+| `MERGE_REFUSED <line>` | The running Host Pod is in `Init:Error` with that line | See [Repairs](#repairs) |+| `MERGE_RUNNING` | The merge init has not finished | Run `verify` again |+| `UNKNOWN`, `UNREADABLE`, `UNVERIFIED` | The script could not decide | Fix the reason and run `verify` again |`HOST_LAYOUT_VERIFY=PRE_825` (exit 1) means a Host Deployment still renders`workspace-layout-init` or a `subPath` mount: the upgrade has not reached it.@@ -110,9 +116,13 @@ merge init and no Host PVC is split, refused, running, or undecided.`PVC_CONTENTS=NOT_INSPECTED` is a reminder that the script checks the layout,not the database contents.+`verify` reads the PVC of every Host, running or not; for a running Host the+init exit code and the tree must both agree.+Orphan PVCs, whose `clerum.io/host` label names no Host, are listed with-`kind=ORPHAN` and counted in `ORPHAN_SPLIT` when split. They do not block the-verdict, because no Host will start on them. A Host created later with the+`kind=ORPHAN` and counted in `ORPHAN_SPLIT` when split. A split orphan does not+block the verdict, because no Host will start on it; an orphan the script+could not read is counted in `ORPHAN_UNDECIDED` and blocks it. A Host created later with thesame name would start on that PVC: with the flag off, on an empty database.Delete the orphan PVC, or recreate its Host before turning the flag off.@@ -216,8 +226,9 @@ spec:- `kubectl set env` works only until the next apply of the overlay.- On Minikube the overlays in this repository do not pin the flag, so a newsetup returns it to `true`.-- After the turn-off, `verify` reports `HOST_LAYOUT_VERIFY=ALREADY_OFF`, or- `FLAG_FALSE_WITH_SPLIT_TREES` if a split tree appeared since.+- After the turn-off, `verify` reports `HOST_LAYOUT_VERIFY=ALREADY_OFF`,+ `FLAG_FALSE_WITH_SPLIT_TREES` if a split tree appeared since, or `FAIL` if+ a PVC could not be read.## Rollbackdiff --git a/host-context-controller/package-lock.json b/host-context-controller/package-lock.jsonindex 83acb0740..07fa8d82b 100644--- a/host-context-controller/package-lock.json+++ b/host-context-controller/package-lock.json@@ -1,12 +1,12 @@{"name": "clerum-host-context-controller",- "version": "0.1.398",+ "version": "0.1.399","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "clerum-host-context-controller",- "version": "0.1.398",+ "version": "0.1.399","license": "SEE LICENSE IN LICENSE","dependencies": {"@clerum/codex-catalog-projection": "file:../packages/codex-catalog-projection",diff --git a/host-context-controller/package.json b/host-context-controller/package.jsonindex 8d8feed1c..608899981 100644--- a/host-context-controller/package.json+++ b/host-context-controller/package.json@@ -1,6 +1,6 @@{"name": "clerum-host-context-controller",- "version": "0.1.398",+ "version": "0.1.399","description": "Host Context Controller - provides McpServer CRDs to mcp-host via REST API","main": "dist/main.js","engines": {diff --git a/host-context-controller/src/statelessDeployment.ts b/host-context-controller/src/statelessDeployment.tsindex 3641a75ec..30d57b64b 100644--- a/host-context-controller/src/statelessDeployment.ts+++ b/host-context-controller/src/statelessDeployment.ts@@ -39,15 +39,15 @@ function shellSingleQuote(value: string): string {* so they must not stop it).** Every refusal prints one line and exits 1 before the first move:- * F1 `state/` or `workspace/` cannot be listed and searched, a database- * entry is a symlink or not a regular file, or a directory that- * decides a collision cannot be listed;+ * F1 `state/` or `workspace/` cannot be listed and searched, or a database+ * entry is a symlink or not a regular file;* F2 more than one live database (a `state.db` main file or its journals)* across the root, `state/` and `workspace/`, except the root journals* a stopped merge leaves next to the remaining `state/state.db`;* F3 a name to move already exists at the root, or exists under both* `state/` and `workspace/` (an empty source directory is removed- * instead), or `workspace/lost+found` would keep `workspace/` alive;+ * instead), a directory that decides such a collision cannot be listed,+ * or `workspace/lost+found` would keep `workspace/` alive;* F4 the root, `state/`, `workspace/`, or a directory to be moved out of* them is not writable (renaming a directory rewrites its `..`).* Moves: `workspace/*`, then `state/*` except `state.db`, then