Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

docs(storage): store env precedence, database in the agent tree, rollback resume case

Alfredo Lopez223d8974e796
diff --git a/docs/architecture/host-storage-layout.md b/docs/architecture/host-storage-layout.md
index c5aa4c3c6..1ad820b8f 100644
--- a/docs/architecture/host-storage-layout.md
+++ b/docs/architecture/host-storage-layout.md
@@ -23,6 +23,17 @@ HCC renders the same storage for every Host Deployment
| Init container | `workspace-layout-merge`, while `CONTEXT_MAPPER_WORKSPACE_LAYOUT_MERGE` is enabled (the default) |
| Deployment strategy | `Recreate` |
+HCC puts the store env in the container's `env`, which Kubernetes applies over
+the `mcp-host-config` ConfigMap loaded through `envFrom`, and mcp-host prefers
+`CLERUM_SESSION_DB_DIR` to `CLERUM_SESSION_DB_PATH` and to the memory
+workspace path (`resolveSessionDbPath`, `mcp-host/src/main.ts`), so a
+ConfigMap cannot move the database. The gcp-dev and gcp-prod overlays
+(`keyper-labs/evenfire-infra`, `mcp-host-config.yaml`) set
+`CLERUM_MEMORY_ENABLED` and `CLERUM_SESSION_STORE=sqlite`, and none of
+`CLERUM_SESSION_DB_DIR`, `CLERUM_SESSION_DB_PATH` or
+`CLERUM_MEMORY_WORKSPACE_PATH`. A stateful Host therefore already resolved its
+database to `/workspace/state.db`, the PVC root, before #825.
+
`Recreate` stops the old Pod before the new one starts. A rolling update can
run both Pods on the same node at once, and a `ReadWriteOnce` PVC does not
prevent that, so two processes could write the same SQLite file.
@@ -33,6 +44,15 @@ requested and on the resolved path
(`mcp-host/src/workspace/protectedPaths.ts`). The shell gate applies the same
prefix lexically and is not a security boundary.
+The database is part of the agent tree. The shell tool runs with the
+workspace root as its working directory (`mcp-host/src/core/tools/shell.ts`),
+so a command such as `rm -f *`, `find . -type f -delete` or `git clean -fdx`
+run there deletes `state.db` and its journals, and a hard link
+(`ln state.db notes`) exposes its contents under a name the prefix rule
+allows. Stateful Hosts had this exposure before #825; stateless Hosts gain it,
+because their database moves from the separate `state/` mount to the root.
+Isolating the database from the agent tree is a follow-up.
+
## The previous split layout
Before #825, stateless Hosts mounted `PVC/workspace` at `/workspace` and
@@ -163,10 +183,11 @@ forced by a deploy.
## Rollback
-- Rolling HCC back is safe for stateful and stateless Hosts, with the one
- exception below. The previous `workspace-layout-init` re-splits stateless
- trees, and `workspace-layout-merge` merges them again on roll-forward.
-- The exception is a tree cut inside the previous init. That init moves the
+- Rolling HCC back is safe for stateful and stateless Hosts, with the two
+ exceptions below (risk R6 of #825). The previous `workspace-layout-init`
+ re-splits stateless trees, and `workspace-layout-merge` merges them again
+ on roll-forward.
+- The first exception is a tree cut inside the previous init. That init moves the
root entries into `workspace/` in one loop and then `workspace/state.db*`
into `state/` in a second loop. A crash between the loops leaves
`workspace/state.db*` and a `state/` without a database. The previous init
@@ -178,6 +199,17 @@ forced by a deploy.
a roll-forward before the Host starts again. To recover, scale the Host to
zero, rename the new root `state.db*` files from a helper Pod (keep them),
and move `workspace/state.db*` to the root.
+- The second exception is the same cut when `state/` already holds a
+ `state.db*.pre-<ts>.bak` from an earlier start. The tree then carries the
+ signature, and the merge moves `workspace/` entries in name order:
+ `state.db` before `state.db-shm` and `state.db-wal`. Resume covers only
+ journals at the root next to `state/state.db`, so a merge cut after
+ `workspace/state.db` moved leaves a root database next to journals in
+ `workspace/`, and every later start refuses with F2
+ (`more than one live SQLite database`). Nothing is lost: the Host stays in
+ `Init:Error` with the database and its journals on the PVC. To recover,
+ move the remaining `workspace/state.db-*` journals to the root from a
+ helper Pod.
- A merged root that holds a symlink (for example one an agent created under
`workspace/`, which the merge moved to the root) blocks the previous init:
it creates empty `state/` and `workspace/`, then exits 1 with
@@ -192,8 +224,12 @@ forced by a deploy.
- For an operator-initiated rollback: do not roll back while any Host is in
`Init:Error` in `workspace-layout-merge`, and do not change a Host's
lifecycle while rolled back.
-- Never roll back the Host image alone; HCC's Deployment carries both
- (`CONTEXT_MAPPER_HOST_IMAGE`, `CONTEXT_MAPPER_DESKTOP_IMAGE`).
+- Never roll back the Host image alone; HCC's Deployment carries it. The
+ gcp-dev and gcp-prod overlays set both `CONTEXT_MAPPER_HOST_IMAGE` and
+ `CONTEXT_MAPPER_DESKTOP_IMAGE` (`patches/dynamic-images.yaml`);
+ `deploy/base` sets only the first, so without an overlay HCC renders desktop
+ Hosts with `clerum/mcp-host-desktop:latest`
+ (`host-context-controller/src/config.ts`).
- A desktop Host's history from before #825 lived in the container, outside
the PVC, and is lost at the first rollout unless it is copied out of the
live Pod first (`kubectl --context <ctx> cp` of `/workspace/state.db*`).
diff --git a/docs/operations/gfs-large-file-host-downloads.md b/docs/operations/gfs-large-file-host-downloads.md
index d5d7004f7..6b6e84230 100644
--- a/docs/operations/gfs-large-file-host-downloads.md
+++ b/docs/operations/gfs-large-file-host-downloads.md
@@ -402,9 +402,9 @@ computed before anything is deleted: when no plan fits, nothing is deleted and
the admission is refused. Eviction candidates are not hashed.
Shell writes do not trigger eviction. Files a command writes elsewhere in a
-workspace share the volume with the cache but are not charged to the budget. On
-stateless Hosts `state.db` shares it too: the Host's `state/` directory is a
-subPath of the same PVC. The store therefore keeps a free-space floor of
+workspace share the volume with the cache but are not charged to the budget.
+`state.db` and its journals share it too: every Host keeps them at the root of
+the same workspace PVC (`docs/architecture/host-storage-layout.md`). The store therefore keeps a free-space floor of
`max(15% of the volume, 16 MiB)`. When the store initializes, after the startup
sweep, and on every 5-minute cycle, a volume below the floor is restored by
evicting unpinned copies (pinned ones already expired included) in the
diff --git a/mcp-host/package-lock.json b/mcp-host/package-lock.json
index d0f7b6125..b2c1dba45 100644
--- a/mcp-host/package-lock.json
+++ b/mcp-host/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "clerum-mcp-host",
- "version": "0.1.572",
+ "version": "0.1.573",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "clerum-mcp-host",
- "version": "0.1.572",
+ "version": "0.1.573",
"license": "SEE LICENSE IN LICENSE",
"dependencies": {
"@anthropic-ai/sdk": "^0.32.0",
diff --git a/mcp-host/package.json b/mcp-host/package.json
index a0e5ddbe0..0ae48f2a6 100644
--- a/mcp-host/package.json
+++ b/mcp-host/package.json
@@ -1,6 +1,6 @@
{
"name": "clerum-mcp-host",
- "version": "0.1.572",
+ "version": "0.1.573",
"description": "MCP Host - reads Host CRD and provides LLM access via OpenAI/Claude",
"main": "dist/main.js",
"engines": {
diff --git a/mcp-host/src/config.ts b/mcp-host/src/config.ts
index 0cffedd47..07ed07891 100644
--- a/mcp-host/src/config.ts
+++ b/mcp-host/src/config.ts
@@ -1022,7 +1022,9 @@ export const config: Config = {
// T2.1 — Conversation store backend selector.
//
// - `memory`: legacy InMemoryConversationStore (no durability). Default
- // so the canary rollout can opt in per Host.
+ // for a process started outside HCC (local dev, tests). HCC sets
+ // `CLERUM_SESSION_STORE=sqlite` in every Host container's env, which
+ // wins over the mcp-host ConfigMap (docs/architecture/host-storage-layout.md).
// - `sqlite`: SqliteConversationStore (RAM + worker thread + state.db).
// - `dual`: both stores live, reads from RAM, writes apply to both —
// for one-week parity validation (`clerum_conversation_store_parity_total`).
diff --git a/scripts/e2e/e2e-stateless-durability.sh b/scripts/e2e/e2e-stateless-durability.sh
index faba4862e..37cf751fa 100755
--- a/scripts/e2e/e2e-stateless-durability.sh
+++ b/scripts/e2e/e2e-stateless-durability.sh
@@ -18,7 +18,7 @@
# 2. `node` + the app's own production `better-sqlite3` at
# /app/node_modules/better-sqlite3 (read-only open, busy_timeout).
# The DB path is resolved from the pod's CLERUM_SESSION_DB_DIR env
-# (HCC sets it to the PVC state mount) + `state.db`. If neither reader
+# (HCC sets it to the workspace PVC root on every Host) + `state.db`. If neither reader
# works, or the env/file is missing, the suite HARD FAILS — there is no
# silent downgrade to weaker text-only assertions.
#
@@ -412,13 +412,13 @@ else
fi
# Resolve the state.db path from the pod env (HCC injects
-# CLERUM_SESSION_DB_DIR on stateless pods) and pick the row-count reader.
+# CLERUM_SESSION_DB_DIR on every Host pod) and pick the row-count reader.
db_dir=$(kctl exec "$pod" -n "$MCP_HOST_NS" -- printenv CLERUM_SESSION_DB_DIR 2>/dev/null || echo "")
if [ -n "$db_dir" ]; then
DB_PATH="${db_dir%/}/state.db"
ok "CLERUM_SESSION_DB_DIR=${db_dir} → DB_PATH=${DB_PATH}"
else
- fail "pod ${pod} has no CLERUM_SESSION_DB_DIR env — is this pod actually running the stateless lifecycle?"
+ fail "pod ${pod} has no CLERUM_SESSION_DB_DIR env — is this pod rendered by the current HCC?"
print_results
exit 1
fi