Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

docs(storage): document the superseded split database

Alfredo Lopez0b3b2a75cbb5
diff --git a/docs/architecture/host-storage-layout.md b/docs/architecture/host-storage-layout.md
index f9c357ba9..a80dd768a 100644
--- a/docs/architecture/host-storage-layout.md
+++ b/docs/architecture/host-storage-layout.md
@@ -87,21 +87,27 @@ stateful or desktop tree changes only if it carries the signature, which only
the earlier init or agent-created `state/` and `workspace/` folders with one
of those exact names produce.
-Two agent-made folder pairs on a stateful Host do stop it from starting:
-
-- a `state/` holding `state.db` or one of its journals beside the root
- database, next to a `workspace/` folder. The signature holds and F2
- refuses, because the merge cannot tell the agent's file from a second
- database. The earlier release produces the same tree without any agent: a
- Host created stateless and switched to stateful on v0.10 or v0.11 keeps its
- old history in `state/state.db` and writes new chats to a root `state.db`.
- `host-layout-merge.sh inventory` reports it as `WILL_REFUSE_F2` before the
- upgrade, and the upgrade note describes the repair;
-- a `state/` the container uid cannot list and search, next to a
- `workspace/` folder. F1 refuses before the database check, because the
- merge cannot tell whether it holds a database.
-
-In both cases the operator renames the folder from a helper Pod.
+A `state/` holding `state.db` or one of its journals beside the root
+database, next to a `workspace/` folder without a database, is superseded
+(T5). The earlier release produces this tree without any agent: a Host
+created stateless and switched to stateful on v0.10 or v0.11 keeps its old
+history in `state/state.db` and writes new chats to a root `state.db`. Every
+stateless start moves the root database into `state/` (the earlier init
+relied on the same rule), so the root database is the newer one. The merge
+keeps it, renames `state/` in one step to `state.db.superseded-<UTC ts>`,
+leaves `workspace/` in place, and exits 0. No `state/` remains, so later
+starts carry no signature. Nothing is deleted, and the new name starts with
+`state.db`, so agent file tools cannot read it or anything under it. An
+agent-made `state/` folder in that position gets the same rename.
+`host-layout-merge.sh inventory` reports the tree as `WILL_SUPERSEDE`, which
+is not an upgrade blocker. A destination that already exists, or a PVC root
+the container uid cannot write, refuses before the rename.
+
+One agent-made folder pair on a stateful Host does stop it from starting: a
+`state/` the container uid cannot list and search, next to a `workspace/`
+folder. F1 refuses before the database check, because the merge cannot tell
+whether it holds a database. The operator renames the folder from a helper
+Pod.
A split tree whose `workspace` is missing, a symlink or a regular file also
carries no signature, because the earlier init always leaves a real
@@ -115,7 +121,8 @@ Host). Recovery, with the Host stopped and the PVC mounted in a helper Pod:
file aside and `mkdir workspace`);
2. if mcp-host already created a root `state.db`, rename the root
`state.db`, `state.db-wal` and `state.db-shm` to `state.db.<label>.bak`
- names, or the next start refuses with F2 (two live databases). Chats
+ names; otherwise the next start keeps that root database and renames
+ `state/`, which holds the history, to `state.db.superseded-<ts>`. Chats
written to that database stay in the backup;
3. start the Host: the signature holds and the merge moves the history to
the root.
@@ -141,7 +148,7 @@ stays in `Init:Error`.
| Case | Line contains |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| F1: the PVC root is a symlink, not a directory, or resolves to `/`; `state/` or `workspace/` escapes the root or cannot be listed and searched; a database entry (`state.db`, `-wal`, `-shm`, `.pre-*.bak`) is a symlink or not a regular file | `directory is a symlink`, `is not a directory`, `escapes the workspace PVC root`, `state directory is not readable and searchable`, `workspace directory is not readable and searchable`, `SQLite file is a symlink`, `SQLite file is not a regular file` |
-| F2: more than one live database family across the root, `state/` and `workspace/` | `more than one live SQLite database` |
+| F2: more than one live database family across the root, `state/` and `workspace/`, except the superseded tree (root main, `state/` family, no `workspace/` family) | `more than one live SQLite database` |
| F3: a name to move already exists at the root, or under both `state/` and `workspace/`; a directory that decides such a collision cannot be listed; `workspace/lost+found` exists | `destination exists`, `exists under both state/ and workspace/`, `lost+found would remain`, `cannot list directory` |
| F4: the PVC root, `state/`, `workspace/`, or a directory to be moved out of them is not writable by the container uid | `not writable by uid` |
@@ -168,9 +175,10 @@ stateful Host the PVC root is the agent workspace, so an agent can create a
`state/state.db*` entry and trigger the merge; the merge is needed once per
Host, and the flag closes that window once every Host is merged. While the
flag is on, the init runs on every Host at every start, so such an entry can
-also make it refuse (F2 next to a root `state.db`, F1 for a symlinked
-`state.db*` entry) and leave a stateful Host in `Init:Error` until an operator
-renames the entry.
+also make it act: next to a root `state.db` and a `workspace/` folder it
+renames `state/` to `state.db.superseded-<ts>`, and a symlinked `state.db*`
+entry refuses with F1 and leaves a stateful Host in `Init:Error` until an
+operator renames the entry.
- Unset or empty means `true`. Only the exact strings `true` and `false` are
accepted; any other value fails HCC config load with
@@ -286,6 +294,11 @@ forced by a deploy.
Host, where the previous mcp-host (which protects only those three exact
names) lets agent file tools read them. Move them into `state/` from a helper
Pod after the rollback, or roll forward.
+- A `state.db.superseded-<ts>` directory is a root entry like any other: the
+ previous init moves it into `workspace/` on a stateless Host, and the merge
+ moves it back to the root on roll-forward. While rolled back, the previous
+ mcp-host protects only the three exact names, so agent file tools can read
+ the directory's other files.
- A merged root that holds a symlink (for example one an agent created under
`workspace/`, which the merge moved to the root) blocks the previous init:
it creates empty `state/` and `workspace/`, then exits 1 with
diff --git a/docs/deploy/host-pvc-layout-upgrade.md b/docs/deploy/host-pvc-layout-upgrade.md
index 854881011..5f1303dfe 100644
--- a/docs/deploy/host-pvc-layout-upgrade.md
+++ b/docs/deploy/host-pvc-layout-upgrade.md
@@ -61,6 +61,7 @@ result:
| ------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------- |
| `NO_SPLIT_SIGNATURE` | The tree does not carry the split-layout signature; the merge exits 0 and changes nothing | None |
| `WILL_MERGE` | The merge moves the tree to the PVC root on the Host's next start | None |
+| `WILL_SUPERSEDE` | A Host switched back to stateful: the merge renames `state/` to `state.db.superseded-<ts>` | None; see [Repairs](#repairs) for where the history goes |
| `WILL_REFUSE_F<n> <line>` | The merge will refuse with that line and the Host will stay in `Init:Error` after the upgrade | See [Repairs](#repairs); counted in `UPGRADE_BLOCKERS` |
| `UNREADABLE <reason>` | The check Pod could not read the PVC (pull error, timeout, unexpected output) | Fix the reason and run the command again |
| `UNKNOWN <reason>` | A Host's claim is missing from the PVC list, or a PVC labelled for a Host has another name | Find the claim the Host mounts and run the command again |
@@ -69,6 +70,9 @@ The command exits 1 while `UPGRADE_BLOCKERS` is above 0, while a PVC is
`UNREADABLE` or `UNKNOWN`, when Hosts exist but no PVC was inspected, or when the flag is set to an invalid value or to `false` with a
split tree present. `DESKTOP_HOSTS` repeats the export warning above.
+`WILL_SUPERSEDE` is not a blocker: the merge handles that tree on the Host's
+next start with no operator step.
+
Repair every `WILL_REFUSE_*` before the upgrade and run `inventory` again
until `UPGRADE_BLOCKERS=0`. A refused Host stays in `Init:Error` after the
upgrade, and a deploy that waits for Host rollouts fails on it (the minikube
@@ -87,10 +91,12 @@ empty database at the PVC root and reports no error. `inventory` and `verify`
print `FLAG_FALSE_WITH_SPLIT_TREES=<n>` and exit 1 in that state. Set the
flag back to `"true"` before the split Host starts: the next start then
merges the tree. If the Host already started with the flag off, it created a
-root `state.db` next to `state/state.db`; with the flag back on, that start
-refuses with F2 (`more than one live SQLite database`). Repair it as a Host
-switched from stateless to stateful (see [Repairs](#repairs)): the root
-family holds only the chats since the flag-off start.
+root `state.db` next to `state/state.db`. With the flag back on, the next
+start treats that tree like a Host switched from stateless to stateful: it
+keeps the root database, which holds only the chats since the flag-off
+start, and renames `state/` with the older history to
+`state.db.superseded-<ts>`. To serve the older history instead, follow
+[Repairs](#repairs).
Running Hosts restart with the new template and merge on that start.
Suspended Hosts merge on their next wake; the upgrade does not wake them.
@@ -172,28 +178,33 @@ are part of the split-layout signature. Delete the helper Pod when done; the
Host Pod retries the init on its own, or delete the Host Pod to retry at once.
**A Host switched from stateless to stateful on v0.10 or v0.11
-(`WILL_REFUSE_F2`).** A Host created stateless and later switched to stateful
-(through the admin API, or by HCC when a CommunicationChannel, a desktop or a
-SharedFileSystem co-location rejected stateless) keeps its old history in
-`state/state.db` and writes new chats to a root `state.db`. The merge cannot
-choose between two live databases. Decide which history the Host serves and
-rename the other family:
-
-- keep the old history: rename the root family. On the old release the Host
- runs stateful on that family, so do this after the upgrade, on the
- `Init:Error` Pod. The signature still holds and
- the merge moves `state/` and `workspace/` to the root on the next start. A
- file written at the root after the switch with the same name as one under
- `workspace/` (for example `IDENTITY.md`) then refuses with F3; rename one of
- the two;
-- keep the chats since the switch: rename the family under `state/`. The
- running Host does not open it, so do this before the upgrade. Unless
- `state/` also holds a `state.db*.pre-<ts>.bak` file, the tree no longer
- carries the signature: the merge exits 0 and `state/` and `workspace/`
- stay as folders in the agent's workspace, with the old files under
- `workspace/`. Move them to the root yourself if the Host needs them.
-
-The renamed family stays on the PVC; agent file tools cannot read any
+(`WILL_SUPERSEDE`, no operator step).** A Host created stateless and later
+switched to stateful (through the admin API, or by HCC when a
+CommunicationChannel, a desktop or a SharedFileSystem co-location rejected
+stateless) keeps its old history in `state/state.db` and writes new chats to
+a root `state.db`. Every stateless start moves the root database into
+`state/`, so the root database is the newer one, and it is the one the Host
+serves today. On the first start after the upgrade the merge keeps it,
+renames `state/` in one step to `state.db.superseded-<UTC ts>`, leaves
+`workspace/` where it is, and exits 0:
+
+- the old history is in `state.db.superseded-<ts>/state.db` (with its
+ journals and any `.pre-<ts>.bak` backups); nothing is deleted;
+- the old files stay under `workspace/`, which the Host sees as
+ `<workspace>/workspace/`. Move them to the root from a helper Pod if the
+ Host needs them at their old paths;
+- agent file tools cannot read `state.db.superseded-*` or anything under it,
+ because the name starts with `state.db`.
+
+To serve the old history instead, swap the two families from a helper Pod
+while no Host Pod has the PVC mounted: rename the root family to
+`state.db.<label>.bak` names and move the files of
+`state.db.superseded-<ts>/` that start with `state.db` to the root.
+
+The same tree is refused (`WILL_REFUSE_F2`) when `workspace/` also holds a
+database, or when `state/` and `workspace/` both hold one and the root has
+none. Decide which history the Host serves and rename the other families.
+The renamed families stay on the PVC; agent file tools cannot read any
`state.db*` name.
**A name in `workspace/` that already exists at the root (`WILL_REFUSE_F3`).**