The agent that grows with you
Python aiai-agentai-agentsanthropicchatgptclaudeclaude-codecodexhermeshermes-agentllmnous-researchopenai
fix(config-remote): review round 3 — intent-preserving CAS retry, private in-memory migration, poller roster snapshots, dotenv+managed bootstrap before the first read, deployment carried into routed children, remote-safe profile create/rename, backend-versioned roster cache, keyed TUI edits, RPC answers on backend refusal, stage2 uses the runtime's selection, refuse values a pending migration would rewrite
Ben Barclay6dd799b30a72
diff --git a/docker/stage2-hook.sh b/docker/stage2-hook.shindex 78a1cad77b0..7b80140a8d7 100755--- a/docker/stage2-hook.sh+++ b/docker/stage2-hook.sh@@ -640,16 +640,22 @@ fi# config-schema migrations that `hermes update` runs for non-Docker installs,# after first-boot seeding and before supervised gateway services start.# Set HERMES_SKIP_CONFIG_MIGRATION=1 for controlled/manual migrations.-# Remote config mode (HERMES_CONFIG_BACKEND=remote, in the container env or $HERMES_HOME/.env) has-# no local config to migrate: the agent migrates the fetched document in memory and never writes-# it back, so a leftover config.yaml must not be migrated or treated as live.-_remote_config_mode=0-if [ "${HERMES_CONFIG_BACKEND:-}" = "remote" ] \- || grep -Eqs '^[[:space:]]*(export[[:space:]]+)?HERMES_CONFIG_BACKEND=["'"'"']?remote["'"'"']?[[:space:]]*$' "$HERMES_HOME/.env"; then- _remote_config_mode=1-fi-if [ "$_remote_config_mode" = "1" ]; then+# Remote config mode (HERMES_CONFIG_BACKEND=remote) has no local config to migrate: the agent+# migrates the fetched document in memory and never writes it back, so a leftover config.yaml must+# not be migrated or treated as live. The backend is decided by the agent's own selection (same+# dotenv parser and precedence as at boot: container env, $HERMES_HOME/.env, managed .env), never+# by a second grammar here. If it cannot be decided, nothing is migrated: the agent still decides+# at boot, and a file-mode migration simply waits for the next boot.+_config_backend=$(s6-setuidgid hermes "$INSTALL_DIR/.venv/bin/python" -c '+import sys+from pathlib import Path+from hermes_cli.env_loader import selected_config_backend_name+print(selected_config_backend_name(Path(sys.argv[1])))+' "$HERMES_HOME" 2>/dev/null) || _config_backend=""+if [ "$_config_backend" = "remote" ]; thenecho "[stage2] HERMES_CONFIG_BACKEND=remote: skipping docker_config_migrate.py"+elif [ -z "$_config_backend" ]; then+ echo "[stage2] Warning: could not determine the config backend; skipping docker_config_migrate.py"elif [ -f "$HERMES_HOME/config.yaml" ]; thens6-setuidgid hermes "$INSTALL_DIR/.venv/bin/python" "$INSTALL_DIR/scripts/docker_config_migrate.py" \|| echo "[stage2] Warning: docker_config_migrate.py failed; continuing"diff --git a/hermes_cli/config_backend.py b/hermes_cli/config_backend.pyindex 7a2a80e9eb5..8bd660065d1 100644--- a/hermes_cli/config_backend.py+++ b/hermes_cli/config_backend.py@@ -19,6 +19,7 @@ whatever user layer the backend returns.from __future__ import annotationsimport os+import threadingfrom dataclasses import dataclass, fieldfrom pathlib import Pathfrom typing import Any, Mapping, Optional, Protocol, Union, runtime_checkable@@ -92,6 +93,10 @@ class ConfigBackend(Protocol):"""Env names no external secret source may supply (the plane credential, D32)."""...+ def deployment_env_names(self) -> frozenset:+ """Non-secret env names that select and address this backend (see :func:`deployment_env_names`)."""+ ...+def apply_in_memory(self, home: Path, doc: dict) -> None:"""Replace the user layer in memory only (in-memory migration, D12). Only called for abackend without file tooling; nothing is sent or written."""@@ -211,6 +216,9 @@ class FileBackend:def protected_env_names(self) -> frozenset:return frozenset()+ def deployment_env_names(self) -> frozenset:+ return frozenset()+def apply_in_memory(self, home: Path, doc: dict) -> None:raise NotImplementedError("the file backend persists migrations to config.yaml")@@ -219,6 +227,33 @@ class FileBackend:_FILE_BACKEND = FileBackend()+_BOOTSTRAP_LOCK = threading.Lock()+_BOOTSTRAPPED = False+++def _bootstrap_names() -> frozenset:+ from plugins.config_backends.remote.backend import POLL_ENV+ from plugins.config_backends.remote.credentials import PLANE_CREDENTIAL_ENV_NAMES+ return PLANE_CREDENTIAL_ENV_NAMES | {POLL_ENV}+++def _ensure_bootstrap_env() -> None:+ """Once per process, before the first selection: the deployment that selects, addresses and+ authenticates the backend comes from the process env, the launch home's ``.env`` and the+ managed ``.env`` (design §4.4 steps 1-2, D26/D32) — also when the first config read comes before+ ``load_hermes_dotenv`` (``hermes_cli.config`` reads config at import time)."""+ global _BOOTSTRAPPED+ if _BOOTSTRAPPED:+ return+ with _BOOTSTRAP_LOCK:+ if _BOOTSTRAPPED:+ return+ _BOOTSTRAPPED = True+ try:+ from hermes_cli.env_loader import apply_config_bootstrap_env+ apply_config_bootstrap_env(_bootstrap_names) # names resolved only for a remote deployment+ except ImportError: # no usable python-dotenv (a bare installer interpreter): no .env loads either+ returndef get_config_backend() -> ConfigBackend:@@ -227,6 +262,7 @@ def get_config_backend() -> ConfigBackend:Read from the environment on every call, before any config read, so no config value canselect it (D11). Raises :class:`ConfigBackendUnavailable` for a backend this build lacks."""+ _ensure_bootstrap_env()kind = os.environ.get(BACKEND_ENV, "").strip().lower() or "file"if kind == "file":return _FILE_BACKEND@@ -336,3 +372,28 @@ def write_config_key(config_path: PathLike, key_path: str, value: Any) -> None:"""Set one dotted key (``None`` removes it) — ``atomic_roundtrip_yaml_update`` semantics."""backend, home = _route(config_path)backend.write_changes(home, Changes(unset=(key_path,)) if value is None else Changes(set={key_path: value}))++++def deployment_env_names() -> frozenset:+ """The selected backend's deployment env names (which backend, which plane, which instance).++ The deployment is a property of the instance, not of a profile, but it usually arrives in the+ launch profile's ``.env``, which a child acting for ANOTHER profile drops as launch residue+ (``strip_launch_profile_env``). Such a child re-reads only its own profile's ``.env`` and would+ silently select the file backend, so these names are never residue. Credentials are not part+ of it: the child resolves its own, and fails closed without them."""+ return get_config_backend().deployment_env_names()+++def supports_file_tooling() -> bool:+ """Gate for tools that copy, edit or back up the config FILE itself (§4.3, §4.7)."""+ return get_config_backend().supports_file_tooling()+++def require_file_tooling(what: str) -> None:+ """Refuse *what* (a file-copy tool) with a clear message when the backend has no config file."""+ if not supports_file_tooling():+ raise ConfigBackendUnavailable(+ f"{what} copies the local config.yaml, and the {get_config_backend().name!r} config "+ "backend has none. It is not available with this backend.")diff --git a/hermes_cli/env_loader.py b/hermes_cli/env_loader.pyindex 339cfba5f84..984f56970cf 100644--- a/hermes_cli/env_loader.py+++ b/hermes_cli/env_loader.py@@ -439,6 +439,68 @@ def _load_dotenv_with_fallback(_sanitize_loaded_credentials() # httpx encodes headers as ASCII+def _bootstrap_env(home: Path | None = None) -> dict:+ """The process env as ``load_hermes_dotenv`` leaves it for the names the home's ``.env`` and the+ managed ``.env`` define: both override, managed last, ``${VAR}`` resolved, the loader's parser.+ Read-only: nothing is published and no file is rewritten (sanitizing imports hermes_cli.config,+ whose import-time config read must not run before the backend is decided)."""+ from dotenv.variables import parse_variables++ home = Path(home) if home is not None else _process_hermes_home()+ env = dict(os.environ)+ layers = [home / ".env"]+ try:+ from hermes_cli import managed_scope++ managed_dir = managed_scope.get_managed_dir()+ except Exception: # noqa: BLE001 — same fail-open as _apply_managed_env+ managed_dir = None+ if managed_dir is not None:+ layers.append(managed_dir / ".env")+ for path in layers:+ try:+ assignments = _parse_dotenv_file(path)+ except OSError:+ continue+ resolved: dict[str, str | None] = {}+ for name, value in assignments:+ if value is not None:+ value = "".join(atom.resolve({**env, **resolved}) for atom in parse_variables(value))+ resolved[name] = value+ env.update({k: v for k, v in resolved.items() if v is not None})+ return env+++def selected_config_backend_name(home: Path | None = None) -> str:+ """The config backend ``load_hermes_dotenv`` selects for *home*, decided without loading or+ booting anything. For a caller that must decide before Hermes starts (Docker stage2 skips the+ config-file migration in remote mode, D12)."""+ from hermes_cli.config_backend import BACKEND_ENV++ return (_bootstrap_env(home).get(BACKEND_ENV) or "").strip().lower() or "file"+++def apply_config_bootstrap_env(remote_names, home: Path | None = None) -> None:+ """Publish ``remote_names()`` (the remote backend's deployment and plane credential, D26/D32) from the+ launch home's ``.env`` and the managed ``.env``, with the values ``load_hermes_dotenv`` gives+ them. Called before the first config read of a process, which can come before+ ``load_hermes_dotenv`` (``hermes_cli.config`` reads config at import time): the backend is+ selected and authenticated from the same deployment either way. A file deployment gets only its+ selector (the file backend needs nothing else before boot); values the process env holds and+ no file defines are left alone; ``load_hermes_dotenv`` later publishes the same values."""+ from hermes_cli.config_backend import BACKEND_ENV++ env = _bootstrap_env(home)+ if (env.get(BACKEND_ENV) or "").strip().lower() == "remote":+ names = remote_names()+ else:+ names = {BACKEND_ENV} # the selector itself, so a file deployment is not read as remote meanwhile+ for name in names:+ value = env.get(name)+ if value is not None and os.environ.get(name) != value:+ os.environ[name] = value++def _sanitize_env_file_if_needed(path: Path) -> None:"""Pre-sanitize a .env file before python-dotenv reads it. Sniffs a leading BOM *before* any textdecode: UTF-16 (Notepad "Unicode") is rewritten as clean UTF-8; UTF-32 is refused (left untouched) so@@ -597,6 +659,10 @@ def load_hermes_dotenv(# remote backend fetches this home's config here, failing closed; the file backend does nothing.from hermes_cli.config_backend import get_config_backend+ # The administrator-managed .env is part of the deployment the backend boots from (it may be+ # where the selector, plane URL or instance id live), so it applies before the boot as well as+ # last below, where it keeps its precedence over secret-source results.+ _apply_managed_env(load_pass=load_pass)get_config_backend().boot(home_path)# External sources are skipped for the updater (dotenv + managed env still load): ``update`` must notdiff --git a/hermes_cli/profiles.py b/hermes_cli/profiles.pyindex f0bd12c6c91..8c51fa228b7 100644--- a/hermes_cli/profiles.py+++ b/hermes_cli/profiles.py@@ -587,6 +587,9 @@ def remove_wrapper_script(name: str) -> bool:def _migrate_profile_config_if_outdated(profile_dir: Path) -> None:"""Migrate a copied config.yaml to the current schema (non-interactive, scoped to the newprofile); otherwise the first desktop/doctor view shows a scary ``v0 -> latest`` warning."""+ from hermes_cli.config_backend import config_exists, supports_file_tooling+ if not supports_file_tooling():+ return # no local config.yaml to migrate: a remote backend migrates in memory on read (D12)if not config_exists(profile_dir / "config.yaml"):return# Creation must not fail over an unmigratable old config; `hermes doctor --fix` surfaces@@ -2279,6 +2282,18 @@ def _record_profile_rename(new_dir: Path, old_canon: str) -> None:logger.debug("profile rename: could not record previous name %r in %s: %s", old_canon, new_dir, exc)+def _refuse_rename_without_file_tooling(old_canon: str) -> None:+ """A remote config backend keys each profile's settings by its canonical name, and the plane+ has no rename: renaming the local home would leave the settings under the old name and read+ another (possibly populated) level under the new one. Refused before any side effect."""+ from hermes_cli.config_backend import get_config_backend+ backend = get_config_backend()+ if not backend.supports_file_tooling():+ raise ValueError(+ f"Cannot rename profile '{old_canon}': its config lives in the {backend.name!r} config "+ "backend under that name, and renaming it there is not supported. Nothing was changed.")++def rename_profile(old_name: str, new_name: str) -> Path:"""Rename a profile: directory, wrapper script, service, active_profile. The defaultprofile's home IS the installation root, so "renaming" it sets a presentation-only@@ -2293,6 +2308,7 @@ def rename_profile(old_name: str, new_name: str) -> Path:new_canon = _canon_valid(new_name)if new_canon == "default":raise ValueError("Cannot rename to 'default' — it is reserved.")+ _refuse_rename_without_file_tooling(old_canon)old_dir = get_profile_dir(old_canon)new_dir = get_profile_dir(new_canon)if not old_dir.is_dir():diff --git a/plugins/config_backends/remote/backend.py b/plugins/config_backends/remote/backend.pyindex 63b1817d9ae..52bb214d769 100644--- a/plugins/config_backends/remote/backend.py+++ b/plugins/config_backends/remote/backend.py@@ -7,7 +7,9 @@ level with CAS (D10). Nothing is read from or written to a local ``config.yaml``"""from __future__ import annotations+import contextvarsimport copy+import itertoolsimport loggingimport osimport random@@ -23,7 +25,7 @@ from hermes_cli.config_backend import (from . import clientfrom .credentials import PLANE_CREDENTIAL_ENV_NAMES, PlaneCredentialError-from .diff import diff, encode_changes, strip_locked, write_check+from .diff import _get_path, diff, encode_changes, json_equal, strip_locked, write_checkfrom .paths import Path as KeyPathfrom .paths import PathError, decode, encode, from_dottedfrom .values import secret_literal_path, to_wire@@ -70,13 +72,13 @@ class _ProfileState:profile_version: int = 0# The profile level's stored writerConfigVersion: None = stored null, _UNKNOWN = not reported.profile_writer: Any = None+ other_writers: List[int] = field(default_factory=list) # the other levels' int stamps (R6)locks: List[Tuple[KeyPath, str]] = field(default_factory=list)provenance: Dict[str, str] = field(default_factory=dict)changed_ns: int = 0gen: int = 0installed: int = 0 # bumped by every _install; a GET started before a newer install is stalebase: Optional[Dict[str, Any]] = None # the migrated doc writes diff against (None = server_config)- migrating: Optional[int] = None # the `installed` a running in-memory migration is forpostprocessed: bool = Falsein_postprocess: bool = Falsefetched_at: float = 0.0@@ -87,6 +89,30 @@ class _ProfileState:_UNKNOWN = object()++@dataclass+class _Private:+ """An in-memory migration's working copy of one profile's document (D12).++ The migration steps read and persist through ``hermes_cli.config`` like any reader and writer;+ while one runs, THAT thread sees this copy instead of the published one (``_PRIVATE``), and+ everyone else keeps seeing the published doc with its matching diff base. The migrated doc and+ its base are published together, in one step, when the migration ends: a concurrent writer+ never diffs a half-migrated document against the unmigrated base, or the reverse."""+ st: "_ProfileState"+ doc: Dict[str, Any]+ serial: int+ gen: int = 0+++_PRIVATE: "contextvars.ContextVar[Optional[_Private]]" = contextvars.ContextVar("remote_config_private", default=None)+_PRIVATE_SERIAL = itertools.count(1)+++def _private_for(st: "_ProfileState") -> Optional[_Private]:+ m = _PRIVATE.get()+ return m if m is not None and m.st is st else None+# Plane refusals caused by the submitted value (contract §2.5, §11.1): surfaced to the caller as# ConfigValueError with the plane's code. config_request_invalid is deliberately absent: a request# the plane cannot parse is this client's bug, not the user's value, so it stays opaque.@@ -144,7 +170,10 @@ class RemoteBackend:key = self._key(home)st = self._states.get(key)if st is not None:- if self._poller_pid != os.getpid(): # a forked child has no poller thread+ poller = self._poller+ # A forked child has no poller thread, and a poller that died must not leave every+ # cached profile without updates until the next new profile happens to re-arm it.+ if poller is None or self._poller_pid != os.getpid() or not poller.is_alive():self._ensure_poller()return stwith self._lock:@@ -153,10 +182,15 @@ class RemoteBackend:st = self._states.get(key)if st is None:st = self._initial_fetch(Path(home))- self._states[key] = st+ with self._lock: # the poller snapshots the roster under the same lock+ self._states[key] = stself._ensure_poller()return st+ def _roster(self) -> List[_ProfileState]:+ with self._lock:+ return list(self._states.values())+def _initial_fetch(self, home: Path) -> _ProfileState:"""Boot fetch with bounded retry (contract §11.4); fails closed (D2)."""profile = self._profile_for(home)@@ -232,6 +266,9 @@ class RemoteBackend:st.etag = etag or str(body.get("etag") or "")st.profile_version = int(body["profileVersion"])st.profile_writer = _profile_writer(body)+ st.other_writers = [+ lv["writerConfigVersion"] for lv in body.get("levels") or []+ if isinstance(lv, dict) and lv.get("kind") != "profile" and isinstance(lv.get("writerConfigVersion"), int)]st.locks = locksst.provenance = dict(body.get("provenance") or {})st.changed_ns = time.time_ns()@@ -246,89 +283,117 @@ class RemoteBackend:"""First read after a change: warn on unknown keys and migrate in memory (D12). Runs on aread, not at fetch, because both need ``hermes_cli.config``, which may still be importingwhen the boot fetch runs."""- if st.postprocessed or st.in_postprocess:- return+ if _private_for(st) is not None:+ return # a migration's own reads: never start another one# The flag, not st.lock, guards the work: a migration reads through hermes_cli.config (its# _CONFIG_LOCK), and holding st.lock across that could deadlock against a writer that holds- # _CONFIG_LOCK and waits for st.lock. A concurrent reader meanwhile gets the unmigrated doc.+ # _CONFIG_LOCK and waits for st.lock. A concurrent reader meanwhile gets the published+ # (unmigrated) doc, and a concurrent writer diffs against its matching base.with st.lock:if st.postprocessed or st.in_postprocess:returnst.in_postprocess = True- # The doc's schema version and its generation are captured together: a poll may install- # a newer doc at any point after this, and migrating that doc from THIS version would- # run steps its data never needed (and, D12, drop settings it holds on purpose).+ # The doc, its schema version and its generation are captured together: a poll may+ # install a newer doc at any point after this, and migrating that doc from THIS version+ # would run steps its data never needed (and, D12, drop settings it holds on purpose).seen = st.installedcurrent = int(st.doc.get("_config_version") or 0)- keys = set(st.doc)+ doc = copy.deepcopy(st.doc)try:try:from hermes_cli.config import _known_top_level_keys- from hermes_cli.config_migrations import SUPPORT_FLOOR_VERSION, run_migrations+ from hermes_cli.config_migrations import SUPPORT_FLOOR_VERSIONexcept ImportError:return # still importing; the next read retries- for key in sorted(keys - _known_top_level_keys() - {"_config_version"}):+ for key in sorted(set(doc) - _known_top_level_keys() - {"_config_version"}):if key not in self._unknown_warned:self._unknown_warned.add(key)logger.warning("Remote Config: unknown config key %r is ignored by this Hermes version", key)latest = _latest_config_version()if SUPPORT_FLOOR_VERSION <= current < latest:- self._migrate_in_memory(st, seen, current, run_migrations)+ self._migrate_in_memory(st, seen, current, doc)elif current < SUPPORT_FLOOR_VERSION:logger.warning("Remote Config: profile %r was written by config version %d, below the ""migration floor %d; not migrated", st.profile, current, SUPPORT_FLOOR_VERSION)- st.postprocessed = st.installed == seen # a doc installed meanwhile needs its own pass+ with st.lock:+ st.postprocessed = st.installed == seen # a doc installed meanwhile needs its own passfinally:st.in_postprocess = False- def _migrate_in_memory(self, st: _ProfileState, started: int, current: int, run_migrations) -> None:- """Migrate doc generation *started* (whose schema version is *current*) in memory (D12).- The migrated doc also becomes the base later writes diff against: a reader was handed the- migrated doc, so a write that leaves the migration's changes in place must not send them- (D12: never write the migration back).-- Generation-checked at start, at each step (:meth:`apply_in_memory`) and at commit: once a- poll or write has installed another doc, nothing of this migration lands on it."""+ def _run_private(self, st: _ProfileState, doc: Dict[str, Any], current: int) -> Dict[str, Any]:+ """Run the migrations from schema *current* over a private copy of *doc* and return it.+ Nothing is published: only this thread sees the copy while the steps run (:class:`_Private`)."""+ from hermes_cli.config_migrations import run_migrationsfrom hermes_constants import reset_hermes_home_override, set_hermes_home_override- with st.lock:- if st.installed != started:- return # replaced before we began; the new doc gets its own pass- st.migrating = started- token = set_hermes_home_override(st.home)+ m = _Private(st=st, doc=copy.deepcopy(doc), serial=next(_PRIVATE_SERIAL))+ m.doc["_config_version"] = current+ private_token = _PRIVATE.set(m)+ home_token = set_hermes_home_override(st.home)try:run_migrations(current, {"env_added": [], "config_added": [], "warnings": []}, True)finally:- reset_hermes_home_override(token)+ reset_hermes_home_override(home_token)+ _PRIVATE.reset(private_token)+ return m.doc++ def _migrate_in_memory(self, st: _ProfileState, started: int, current: int, doc: Dict[str, Any]) -> None:+ """Migrate doc generation *started* (*doc*, schema version *current*) in memory (D12).++ The steps work on a private copy; the result is published with its diff base in one step,+ and only while generation *started* is still the installed one: once a poll or write has+ installed another doc, nothing of this migration lands on it (that doc gets its own pass).+ The migrated doc becomes the base later writes diff against: a reader was handed the+ migrated doc, so a write that leaves the migration's changes in place must not send them+ (D12: never write the migration back)."""with st.lock:- st.migrating = Noneif st.installed != started:- return # a poll or write installed a newer doc mid-migration; its own pass migrates it- doc = copy.deepcopy(st.doc)- doc["_config_version"] = _latest_config_version()- self._apply_in_memory_locked(st, doc)- base = copy.deepcopy(doc)- base.pop("_config_version", None)- st.base = base+ return # replaced before we began+ migrated = self._run_private(st, doc, current)+ migrated["_config_version"] = _latest_config_version()+ base = copy.deepcopy(migrated)+ base.pop("_config_version", None)+ with st.lock:+ if st.installed != started:+ return # a poll or write installed a newer doc mid-migration+ self._publish_locked(st, migrated, base)logger.info("Remote Config: migrated profile %r in memory from config version %d", st.profile, current)+ @staticmethod+ def _publish_locked(st: _ProfileState, doc: Dict[str, Any], base: Dict[str, Any]) -> None:+ st.doc = doc+ st.base = base+ st.changed_ns = time.time_ns()+ st.gen += 1+# --- ConfigBackend: reads ---------------------------------------------------------------def read_user_layer(self, home: Path) -> UserLayer:st = self._state(home)self._postprocess(st)- return UserLayer(doc=copy.deepcopy(st.doc), version=self._version_of(st),- locks={encode(p): level for p, level in st.locks},+ with st.lock:+ m = _private_for(st)+ doc = copy.deepcopy(m.doc if m is not None else st.doc)+ version = self._version_of(st)+ locks = {encode(p): level for p, level in st.locks}+ return UserLayer(doc=doc, version=version, locks=locks,provenance=f"remote:{client.base_url()} profile={st.profile}")def read_user_doc_readonly(self, home: Path) -> Any:st = self._state(home)self._postprocess(st)- return st.doc+ m = _private_for(st)+ return m.doc if m is not None else st.doc@staticmethoddef _version_of(st: _ProfileState) -> Tuple[Any, ...]:# Leads with a nanosecond timestamp like the file backend's st_mtime_ns (the TUI's config- # watcher reads element 0 as a time); the etag and local generation make it exact.+ # watcher reads element 0 as a time); the etag and local generation make it exact. A+ # migration's own reads see its private copy under a version of their own, so no config+ # cache ever pairs the published version with a private doc, or the reverse. It must not+ # START with the published version either: config caches match a signature as a prefix.+ m = _private_for(st)+ if m is not None:+ return (-m.serial, m.gen, st.changed_ns, st.etag, st.gen)return (st.changed_ns, st.etag, st.gen)def version(self, home: Path) -> Tuple[Any, ...]:@@ -346,37 +411,65 @@ class RemoteBackend:# --- ConfigBackend: writes --------------------------------------------------------------+ _MAX_PREPARE = 3 # re-preparations when a poll lands between preparing and sending+def write_changes(self, home: Path, changes: Changes) -> None:+ """Send ``changes`` as one PATCH with CAS (contract §15.2), re-read and retry once on a+ conflict (§15.3).++ The edit is fixed once, against the doc the caller read (its *intent*: the key-level sets+ and unsets that turn that doc into ``changes``). A retry re-diffs that intent against the+ re-read doc, with the re-read locks; it never replays the caller's whole document, which+ would turn every key another writer changed meanwhile into an edit of ours."""st = self._state(home)+ intent: Optional[Tuple[Dict[KeyPath, Any], List[KeyPath]]] = Nonefor attempt in (1, 2):# Diff against the doc readers get, i.e. migrated in memory (D12) — also after the CAS# re-read below. Outside st.lock: a migration reads through hermes_cli.config.self._postprocess(st)- with st.lock:- built = self._build_patch(st, changes)- if built is None:- return- body, unsets = built- try:- resp = client.request("PATCH", st.home, st.profile, body=body)- except (PlaneCredentialError, client.TransportError) as exc:- raise ConfigWriteError(f"Remote Config write failed: {exc}", code="config_plane_unreachable") from exc- if resp.status == 200 and _effective_ok(resp.body):- self._install(st, resp.body, resp.etag)- self._warn_inherited(st, unsets)- return- if resp.status == 409 and attempt == 1:- try: # contract §15.3: re-read, re-diff, retry once- self._fetch_into(st, conditional=False)- except _FetchFailed as exc:- raise ConfigWriteError(f"Remote Config write conflict and re-read failed: {exc}",- code="config_version_conflict") from exc- continue- raise self._write_error(resp)+ for _ in range(self._MAX_PREPARE):+ with st.lock:+ seen = st.installed+ if intent is None:+ intent = self._intent(st, changes)+ built = self._patch_body(st, *intent)+ if built is None:+ return+ body, sets, unsets = built+ check = self._migration_check_input(st, body, sets, unsets)+ if check is not None: # outside st.lock, like any migration run+ self._check_migration_keeps(st, sets, *check)+ with st.lock:+ if st.installed != seen:+ continue # a poll landed meanwhile: prepare again against it+ try:+ resp = client.request("PATCH", st.home, st.profile, body=body)+ except (PlaneCredentialError, client.TransportError) as exc:+ raise ConfigWriteError(f"Remote Config write failed: {exc}", code="config_plane_unreachable") from exc+ if resp.status == 200 and _effective_ok(resp.body):+ self._install(st, resp.body, resp.etag)+ self._warn_inherited(st, unsets)+ return+ if resp.status == 409 and attempt == 1:+ try: # contract §15.3: re-read, re-diff, retry once+ self._fetch_into(st, conditional=False)+ except _FetchFailed as exc:+ raise ConfigWriteError(f"Remote Config write conflict and re-read failed: {exc}",+ code="config_version_conflict") from exc+ break+ raise self._write_error(resp)+ else:+ raise ConfigWriteError("Remote Config write not sent: the profile's config kept changing "+ "while the write was prepared", code="config_version_conflict")- def _build_patch(self, st: _ProfileState, changes: Changes) -> Optional[Tuple[Dict[str, Any], List[KeyPath]]]:- """The PATCH body for ``changes`` against the last read (§15.2), or None for no change."""+ def _diff_base(self, st: _ProfileState) -> Dict[str, Any]:base = copy.deepcopy(st.base if st.base is not None else st.server_config)+ base.pop("_config_version", None)+ return base++ def _intent(self, st: _ProfileState, changes: Changes) -> Tuple[Dict[KeyPath, Any], List[KeyPath]]:+ """The key-level edit ``changes`` makes to the last read doc (§15.2 steps 1-5)."""+ base = self._diff_base(st)if changes.document is not None:new = to_wire(copy.deepcopy(changes.document))if not isinstance(new, dict):@@ -388,7 +481,6 @@ class RemoteBackend:_set_path(new, path, to_wire(value, path))for key in changes.unset:_pop_path(new, from_dotted(key))- base.pop("_config_version", None)new.pop("_config_version", None)if changes.document is not None:@@ -396,6 +488,17 @@ class RemoteBackend:if dropped:print(f"Note: {len(dropped)} setting(s) locked by Remote Config were not saved: "f"{', '.join(sorted(encode(p) for p in dropped))}", file=sys.stderr)+ return diff(base, new)++ def _patch_body(self, st: _ProfileState, sets: Dict[KeyPath, Any], unsets: List[KeyPath]+ ) -> Optional[Tuple[Dict[str, Any], Dict[KeyPath, Any], List[KeyPath]]]:+ """The PATCH body applying the intent to the current doc, or None when that is no change."""+ base = self._diff_base(st)+ new = copy.deepcopy(base)+ for p, v in sets.items():+ _set_path(new, p, copy.deepcopy(v))+ for p in unsets:+ _pop_path(new, p)sets, unsets = diff(base, new)refused = write_check(sets, unsets, st.locks)if refused is not None:@@ -424,7 +527,54 @@ class RemoteBackend:body["set"] = set_bodyif unset_body:body["unset"] = unset_body- return body, unsets+ return body, sets, unsets++ def _migration_check_input(self, st: _ProfileState, body: Dict[str, Any], sets: Dict[KeyPath, Any],+ unsets: List[KeyPath]) -> Optional[Tuple[Dict[str, Any], int]]:+ """``(stored doc after this write, its schema version)`` when readers will migrate it in+ memory after the write (R6: the oldest level's stamp), else None. Call under st.lock."""+ if not sets:+ return None+ try:+ from hermes_cli.config_migrations import SUPPORT_FLOOR_VERSION+ except ImportError:+ return None+ latest = _latest_config_version()+ profile = latest if "writerConfigVersion" in body else st.profile_writer+ stamps = [*st.other_writers, *([profile] if isinstance(profile, int) else [])]+ after = min(stamps) if stamps else latest+ if not SUPPORT_FLOOR_VERSION <= after < latest:+ return None+ stored = copy.deepcopy(st.server_config)+ stored.pop("_config_version", None)+ for p, v in sets.items():+ _set_path(stored, p, copy.deepcopy(v))+ for p in unsets:+ _pop_path(stored, p)+ return stored, after++ def _check_migration_keeps(self, st: _ProfileState, sets: Dict[KeyPath, Any],+ stored: Dict[str, Any], after: int) -> None:+ """Refuse a write whose values every later read would migrate away.++ The plane stores the write as sent, but under a level stamp older than this agent's schema+ (the stamp keeps describing the level's untouched legacy data, contract §5.3), so every+ reader — this process and any later one — re-runs the migrations from that stamp over the+ stored doc. A value one of those steps rewrites (an old default the user set on purpose)+ would be acknowledged and then never read back. Refused instead, before anything is sent."""+ migrated = self._run_private(st, stored, after)+ latest = _latest_config_version()+ for p, v in sets.items():+ found, got = _get_path(migrated, p)+ changed = _first_change(p, v, found, got)+ if changed is not None:+ raise ConfigValueError(+ f"Remote Config: {encode(changed)} cannot be saved with that value. This profile's remote "+ f"settings are stored at config schema v{after}, and every read migrates them to "+ f"v{latest} in memory, which rewrites that value (migrations are never written back, "+ "D12), so it would never be read back. Nothing was sent; choose another value, or "+ "have the profile level re-saved at the current schema on the config plane.",+ code="config_migration_conflict")@staticmethoddef _write_error(resp: client.Response) -> ConfigWriteError:@@ -461,16 +611,16 @@ class RemoteBackend:def apply_in_memory(self, home: Path, doc: dict) -> None:st = self._state(home)- with st.lock:- if st.migrating is not None and st.migrating != st.installed:- return # a migration step of a doc a poll/write has since replaced: drop it- self._apply_in_memory_locked(st, doc)-- @staticmethod- def _apply_in_memory_locked(st: _ProfileState, doc: dict) -> None:- st.doc = copy.deepcopy(doc)- st.changed_ns = time.time_ns()- st.gen += 1+ m = _private_for(st)+ if m is not None: # a migration step: lands in that migration's private copy only+ m.doc = copy.deepcopy(doc)+ m.gen += 1+ return+ with st.lock: # outside a migration: the doc and the base writes diff against move together+ doc = copy.deepcopy(doc)+ base = copy.deepcopy(doc)+ base.pop("_config_version", None)+ self._publish_locked(st, doc, base)# --- ConfigBackend: capabilities and boot -----------------------------------------------@@ -483,6 +633,10 @@ class RemoteBackend:def protected_env_names(self) -> frozenset:return PLANE_CREDENTIAL_ENV_NAMES+ def deployment_env_names(self) -> frozenset:+ from hermes_cli.config_backend import BACKEND_ENV+ return frozenset({BACKEND_ENV, client.URL_ENV, client.INSTANCE_ENV, POLL_ENV})+def boot(self, home: Path) -> None:"""Design §4.4 steps 2-4: fetch this home's layer (fail closed) and flag a managed file."""self._state(home)@@ -517,12 +671,16 @@ class RemoteBackend:def _poll_loop(self) -> None:while not self._stop.is_set():- now = time.monotonic()- states = list(self._states.values())- due = [st for st in states if st.next_poll <= now]- for st in due:- self.poll_one(st)- wake = min((st.next_poll for st in self._states.values()), default=now + poll_interval())+ try:+ now = time.monotonic()+ roster = self._roster() # a snapshot: a lazily added profile must not break iteration+ for st in roster:+ if st.next_poll <= now:+ self.poll_one(st)+ wake = min((st.next_poll for st in self._roster()), default=now + poll_interval())+ except Exception: # noqa: BLE001 — the only poller thread must survive anything+ logger.warning("Remote Config: poll loop error; retrying", exc_info=True)+ wake = time.monotonic() + MIN_POLL_SECONDSself._stop.wait(max(1.0, wake - time.monotonic()))def poll_one(self, st: _ProfileState) -> bool:@@ -547,7 +705,7 @@ class RemoteBackend:return changeddef poll_all(self) -> None:- for st in list(self._states.values()):+ for st in self._roster():self.poll_one(st)@@ -560,6 +718,22 @@ def managed_config_file() -> Optional[Path]:return path if path.exists() else None # config-reader: ok — existence only, to warn it is ignored (D18)+def _first_change(path: KeyPath, sent: Any, found: bool, got: Any) -> Optional[KeyPath]:+ """The first path at or under *path* where *got* (as read back) differs from *sent*."""+ if not found:+ return path+ if isinstance(sent, dict) and isinstance(got, dict):+ for key, value in sent.items():+ changed = _first_change(path + (key,), value, key in got, got.get(key))+ if changed is not None:+ return changed+ return None+ try:+ return None if json_equal(to_wire(copy.deepcopy(got), path), sent) else path+ except ConfigValueError:+ return path++def _set_path(doc: Dict[str, Any], path: KeyPath, value: Any) -> None:node = docfor seg in path[:-1]:diff --git a/tests/plugins/config_backends/test_remote_backend.py b/tests/plugins/config_backends/test_remote_backend.pyindex 422bf3d2790..55c3a429a3d 100644--- a/tests/plugins/config_backends/test_remote_backend.py+++ b/tests/plugins/config_backends/test_remote_backend.py@@ -922,3 +922,367 @@ def test_stub_plane_deep_merge_follows_contract_6_2():assert deep_merge(upper, {"display": None}) == upperassert deep_merge(upper, {"model": None})["model"] is Noneassert deep_merge(upper, {"tags": [3]})["tags"] == [3]+++# --- review round 3 regressions -----------------------------------------------------------++def _drop_process_deployment(plane, monkeypatch, tmp_path):+ """Move the deployment out of the process env (as on a host where it lives in .env files);+ returns the managed dir. Every name stays recorded by monkeypatch, so whatever a dotenv load+ publishes is undone at teardown."""+ from hermes_cli import config_backend+ for name in remote_env(plane):+ monkeypatch.delenv(name, raising=False)+ managed = tmp_path / "managed"+ managed.mkdir()+ monkeypatch.setenv("HERMES_MANAGED_DIR", str(managed))+ monkeypatch.setattr(config_backend, "_BOOTSTRAPPED", True) # this process is past its first read+ return managed+++def test_cas_retry_of_a_document_write_replays_only_its_own_edit(plane, monkeypatch):+ """F1: another writer commits between our read and our PATCH. The 409 retry re-applies OUR+ edit to the re-read doc; it must not turn the other writer's changes into edits of ours."""+ from hermes_cli.config import set_config_value+ from plugins.config_backends.remote import client+ plane.profile("default").update(values={"display": {"personality": "concise", "compact": False}}, version=1)+ real_request = client.request+ raced = []++ def other_writer_lands_first(method, *args, **kwargs):+ if method == "PATCH" and not raced:+ raced.append(True)+ prof = plane.profile("default")+ prof["values"]["display"]["compact"] = True+ prof["values"]["terminal"] = {"timeout": 321}+ prof["version"] += 1+ return real_request(method, *args, **kwargs)++ monkeypatch.setattr(client, "request", other_writer_lands_first)++ set_config_value("display.personality", "pirate")++ assert plane.profile("default")["values"] == {+ "display": {"personality": "pirate", "compact": True}, "terminal": {"timeout": 321}}+ first, retry = plane.patches()+ assert first["body"]["expectedVersion"] == 1 and retry["body"]["expectedVersion"] == 2+ assert "unset" not in retry["body"] and "compact" not in json.dumps(retry["body"])+++def test_edit_during_an_in_memory_migration_never_sends_the_migration(plane, monkeypatch):+ """F2: another thread's in-memory migration has run its steps but not finished when a public+ edit reads and saves. The edit must diff against the doc it read, so the migration's removal+ of compression.threshold_tokens is not sent (D12)."""+ import threading++ from hermes_cli import config_migrations+ from hermes_cli.config import read_raw_config, set_config_value+ from hermes_cli.config_backend import read_config_doc+ assert backend_mod._latest_config_version() > 46+ plane.profile("default").update(values={"compression": {"threshold_tokens": 256000}}, version=1, writer=46)+ get_config_backend()._state(plane.home)+ steps_ran, resume = threading.Event(), threading.Event()+ real_run = config_migrations.run_migrations++ def paused(*args, **kwargs):+ real_run(*args, **kwargs)+ if threading.current_thread().name == "migrator": # the edit's own checks run unpaused+ steps_ran.set()+ assert resume.wait(10), "the edit waited for the migration"++ monkeypatch.setattr(config_migrations, "run_migrations", paused)+ failures = []++ def migrate(): # a lock-free reader (read_config_doc) starts the migration+ try:+ read_config_doc(plane.home / "config.yaml")+ except BaseException as exc: # noqa: BLE001 — surfaced below+ failures.append(exc)++ migrator = threading.Thread(target=migrate, name="migrator")+ migrator.start()+ try:+ assert steps_ran.wait(10)+ set_config_value("display.personality", "pirate")+ finally:+ resume.set()+ migrator.join(10)+ assert not migrator.is_alive() and not failures, failures++ (patch,) = plane.patches()+ assert "unset" not in patch["body"] and "compression" not in json.dumps(patch["body"])+ assert plane.profile("default")["values"]["compression"] == {"threshold_tokens": 256000}+ later = read_raw_config()+ assert later["display"]["personality"] == "pirate"+ assert "threshold_tokens" not in (later.get("compression") or {}) # still migrated, in memory+++def test_profile_added_while_the_poller_iterates_does_not_kill_it(plane):+ """F3: a first read of another profile inserts its state while the poll loop walks the+ roster; the only poller thread must survive it."""+ import dataclasses+ import threading++ backend = get_config_backend()+ st = backend._state(plane.home)+ work = plane.home / "profiles" / "work"+ work.mkdir(parents=True)+ inserted = threading.Event()++ class AddsAProfileMidPass(backend_mod._ProfileState):+ reads = 0++ @property+ def next_poll(self):+ AddsAProfileMidPass.reads += 1+ if AddsAProfileMidPass.reads == 2 and not inserted.is_set(): # mid-pass: the deadline scan+ backend._state(work)+ inserted.set()+ return time.monotonic() + 1000++ @next_poll.setter+ def next_poll(self, value):+ pass++ import time+ key = backend._key(plane.home)+ backend._states[key] = AddsAProfileMidPass(**{+ f.name: getattr(st, f.name) for f in dataclasses.fields(st) if f.name != "next_poll"})+ loop = threading.Thread(target=backend._poll_loop, daemon=True)+ loop.start()+ assert inserted.wait(10)+ loop.join(2)+ assert loop.is_alive()+ assert backend._key(work) in backend._states+ backend._stop.set()+++def test_a_dead_poller_is_rearmed_by_a_cached_read(plane):+ """F3: a poller that died must not leave cached profiles without updates."""+ import threading++ from hermes_cli.config import load_config+ load_config()+ backend = remote_pkg.get_remote_backend()+ dead = threading.Thread(target=lambda: None)+ dead.start()+ dead.join()+ backend._poller = dead+ load_config()+ assert backend._poller is not dead and backend._poller.is_alive()+++def test_instance_id_from_the_managed_dotenv_boots(plane, monkeypatch, tmp_path):+ """F4: the deployment in the home's .env, the instance id only in the managed .env."""+ import os++ from hermes_cli.env_loader import load_hermes_dotenv+ managed = _drop_process_deployment(plane, monkeypatch, tmp_path)+ env = remote_env(plane)+ (managed / ".env").write_text(f"HERMES_CONFIG_INSTANCE_ID={env.pop('HERMES_CONFIG_INSTANCE_ID')}\n")+ (plane.home / ".env").write_text("".join(f"{k}={v}\n" for k, v in env.items()))+ plane.upper = {"display": {"personality": "managed"}}++ load_hermes_dotenv(hermes_home=plane.home)++ gets = _gets(plane)+ assert gets and gets[0]["instance"] == INSTANCE+ assert os.environ["HERMES_CONFIG_INSTANCE_ID"] == INSTANCE+++def test_child_for_another_profile_keeps_the_remote_deployment(plane, monkeypatch, tmp_path):+ """F5: the deployment came from the launch profile's .env. A Hermes child built for another+ profile must still read that profile's config remotely, never fall back to its local file."""+ import os+ import subprocess+ import sys++ from hermes_cli.env_loader import load_hermes_dotenv+ from tools.environments.local import served_profile_child_env+ _drop_process_deployment(plane, monkeypatch, tmp_path)+ env = remote_env(plane)+ (plane.home / ".env").write_text("".join(f"{k}={v}\n" for k, v in env.items()))+ load_hermes_dotenv(hermes_home=plane.home)+ beta = plane.home / "profiles" / "beta"+ beta.mkdir(parents=True)+ (beta / "config.yaml").write_text("display:\n personality: local-child\n")+ # beta's own plane credential (credentials are never carried into another profile's child)+ (beta / ".env").write_text("".join(f"{k}={v}\n" for k, v in env.items() if k.startswith("GATEWAY_RELAY_IDP_")))+ plane.profile("beta")["values"] = {"display": {"personality": "remote-child"}}++ child_env = served_profile_child_env(target_home=beta, inherit_credentials=True)+ child_env["PYTHONPATH"] = str(Path(__file__).resolve().parents[3])+ code = ("import json\n"+ "from hermes_cli.env_loader import load_hermes_dotenv\n"+ "load_hermes_dotenv()\n"+ "from hermes_cli.config_backend import get_config_backend\n"+ "from hermes_cli.config import load_config\n"+ "print('RESULT=' + json.dumps([get_config_backend().name, load_config()['display']['personality']]))\n")+ proc = subprocess.run([sys.executable, "-c", code], env=child_env, capture_output=True, text=True,+ timeout=120, stdin=subprocess.DEVNULL, cwd=str(tmp_path))+ assert proc.returncode == 0, proc.stderr[-3000:]+ line = [ln for ln in (proc.stdout + proc.stderr).splitlines() if ln.startswith("RESULT=")][-1]+ assert json.loads(line[len("RESULT="):]) == ["remote", "remote-child"]+ assert "beta" in {r["profile"] for r in _gets(plane)}+ assert not any(k.startswith("GATEWAY_RELAY_IDP_") and v != env[k] for k, v in child_env.items())+ assert os.environ["HERMES_CONFIG_BACKEND"] == "remote"+++def test_setting_a_value_that_old_schema_migration_rewrites_is_refused(plane, capsys):+ """F8: on a level stamped 46 (option 1 keeps that stamp), 256000 is exactly what the 46->47+ step removes on every read: acknowledging it would hide it. Refused, nothing sent. A value+ the migration keeps is saved as usual."""+ from hermes_cli.config import read_raw_config+ assert backend_mod._latest_config_version() > 46+ plane.profile("default").update(values={"display": {"personality": "old"}}, version=1, writer=46)+ read_raw_config()++ code, err = _config_cmd(capsys, "set", "compression.threshold_tokens", "256000")++ assert code == 1 and "compression.threshold_tokens" in err and "schema v46" in err+ assert plane.patches() == []+ with pytest.raises(ConfigValueError) as exc:+ write_config_key(plane.home / "config.yaml", "compression.threshold_tokens", 256000)+ assert exc.value.code == "config_migration_conflict"++ write_config_key(plane.home / "config.yaml", "compression.threshold_tokens", 300000)+ assert plane.profile("default")["values"]["compression"] == {"threshold_tokens": 300000}+ remote_pkg._reset_for_tests() # a fresh process reads it back+ assert read_raw_config()["compression"]["threshold_tokens"] == 300000+++# --- profiles, TUI and RPC in remote mode -------------------------------------------------++def profile_plane(tmp_path, monkeypatch):+ """``plane`` with a real profiles root (``Path.home()/.hermes``)."""+ home = tmp_path / ".hermes"+ home.mkdir()+ monkeypatch.setattr(Path, "home", lambda: tmp_path)+ monkeypatch.setenv("HERMES_HOME", str(home))+ with StubPlane() as p:+ for k, v in remote_env(p).items():+ monkeypatch.setenv(k, v)+ monkeypatch.setattr(backend_mod, "BOOT_RETRY_DELAYS", (0.0, 0.0))+ remote_pkg._reset_for_tests()+ cred_mod._IDP_CACHE.clear()+ p.home = home+ get_config_backend()._state(home)+ yield p+ remote_pkg._reset_for_tests()+ cred_mod._IDP_CACHE.clear()+++def test_fresh_profile_creation_succeeds_against_a_healthy_plane(profile_plane):+ """F6: the staging home is never read through the remote backend."""+ from hermes_cli.profiles import create_profile+ before = len(profile_plane.requests)++ path = create_profile("fresh", no_alias=True, no_skills=True)++ assert path.is_dir() and (path / ".env").exists()+ assert not list(path.parent.glob(".fresh.staging-*"))+ assert not any(r["profile"].startswith(".") for r in profile_plane.requests[before:])+++def test_rename_is_refused_before_anything_moves(profile_plane, monkeypatch):+ """F7: the plane keys settings by profile name and has no rename."""+ from hermes_cli import profiles+ for name in ("_check_gateway_running", "_cleanup_gateway_service", "_maybe_unregister_gateway_service",+ "_stop_bot_desktop", "_live_default_multiplexer", "_maybe_register_gateway_service"):+ monkeypatch.setattr(profiles, name, lambda *a, **k: False)+ old = profile_plane.home / "profiles" / "old"+ old.mkdir(parents=True)+ profile_plane.profile("old").update(values={"model": {"default": "profile-model"}}, version=1)+ profile_plane.profile("new").update(values={"model": {"default": "target-model"}}, version=1)++ with pytest.raises(ValueError, match="not supported"):+ profiles.rename_profile("old", "new")++ assert old.is_dir() and not (profile_plane.home / "profiles" / "new").exists()+ assert profile_plane.profile("old")["values"] == {"model": {"default": "profile-model"}}+ assert profile_plane.patches() == []+++def test_roster_model_follows_the_remote_layer_not_the_local_file(profile_plane):+ """F9: a poll changes the profile's model while an ignored local config.yaml stays put."""+ from hermes_cli.profiles import list_profiles+ named = profile_plane.home / "profiles" / "roster"+ named.mkdir(parents=True)+ (named / "config.yaml").write_text("model:\n default: ignored-local\n")+ profile_plane.profile("roster").update(values={"model": {"default": "model-A", "provider": "nous"}}, version=1)++ def roster():+ return {p.name: (p.model, p.provider) for p in list_profiles()}["roster"]++ assert roster() == ("model-A", "nous")+ profile_plane.profile("roster").update(values={"model": {"default": "model-B", "provider": "openrouter"}}, version=2)+ backend = get_config_backend()+ assert backend.poll_one(backend._state(named))+ assert roster() == ("model-B", "openrouter")+++def _tui_server(monkeypatch, home):+ import hermes_cli.banner as banner+ monkeypatch.setattr(banner, "prefetch_update_check", lambda: None)+ from tui_gateway import server+ monkeypatch.setattr(server, "_hermes_home", home)+ server._cfg_cache = server._cfg_sig = server._cfg_path = None+ return server+++def test_tui_config_set_of_a_locked_key_is_refused_not_reported_saved(plane, monkeypatch):+ """F10: a keyed TUI edit under a lock answers with the lock, sends nothing, and the raw cache+ keeps the accepted value."""+ plane.upper = {"display": {"tui_theme": "dark"}}+ plane.upper_locks = [{"path": "display.tui_theme", "level": "tenant"}]+ server = _tui_server(monkeypatch, plane.home)++ answer = server._methods["config.set"](1, {"key": "theme", "value": "light"})++ assert answer["error"]["code"] == 4002 and "locked" in answer["error"]["message"]+ assert plane.patches() == []+ assert server._load_cfg_raw()["display"]["tui_theme"] == "dark"++ plane.upper_locks = []+ backend = get_config_backend()+ assert backend.poll_one(backend._state(plane.home))+ assert server._methods["config.set"](2, {"key": "theme", "value": "light"})["result"]["value"] == "light"+ assert server._load_cfg_raw()["display"]["tui_theme"] == "light"+ assert plane.profile("default")["values"] == {"display": {"tui_theme": "light"}}+++def test_rpc_refused_by_the_config_backend_still_gets_its_one_answer(plane, monkeypatch):+ """F11: ConfigBackendUnavailable is a SystemExit; an admitted request (inline or pooled) still+ answers once, with its own id."""+ import copy+ import threading++ server = _tui_server(monkeypatch, plane.home)++ def refused(rid, params):+ raise ConfigBackendUnavailable("Remote Config: cannot load the config for profile 'lazy'")++ monkeypatch.setitem(server._methods, "cc.refused", refused)+ inline = server.handle_request({"jsonrpc": "2.0", "id": "inline-1", "method": "cc.refused", "params": {}})+ assert inline["id"] == "inline-1" and "cannot load the config" in inline["error"]["message"]++ monkeypatch.setattr(server, "_LONG_HANDLERS", server._LONG_HANDLERS | {"cc.refused"})+ written = threading.Event()++ class Recorder:+ frames = []++ def write(self, obj):+ self.frames.append(copy.deepcopy(obj))+ written.set()+ return True++ def close(self):+ pass++ transport = Recorder()+ assert server.dispatch({"jsonrpc": "2.0", "id": "pooled-1", "method": "cc.refused", "params": {}}, transport) is None+ assert written.wait(10)+ (frame,) = transport.frames+ assert frame["id"] == "pooled-1" and "cannot load the config" in frame["error"]["message"]diff --git a/tests/plugins/config_backends/test_remote_multiprocess.py b/tests/plugins/config_backends/test_remote_multiprocess.pyindex 7ae0b49904c..02cd6f803c9 100644--- a/tests/plugins/config_backends/test_remote_multiprocess.py+++ b/tests/plugins/config_backends/test_remote_multiprocess.py@@ -75,3 +75,28 @@ def test_process_exits_non_zero_when_plane_is_down(plane_env, monkeypatch):assert proc.returncode != 0assert "Hermes does not start without its remote config" in proc.stderrassert "concise" not in proc.stdout # no default or cached value was served+++def test_first_config_read_before_dotenv_uses_the_dotenv_deployment(plane_env, tmp_path):+ """F4: ``hermes_cli.config`` reads config at import time, before ``load_hermes_dotenv`` runs+ (main.py imports it first). The selector is in the process env; the plane URL and credential+ are in the home's .env and the instance id only in the managed .env. That first read must+ already select, address and authenticate from that whole deployment."""+ plane, env, home = plane_env+ deployment = remote_env(plane)+ managed = tmp_path / "managed"+ managed.mkdir()+ (managed / ".env").write_text(f"HERMES_CONFIG_INSTANCE_ID={deployment.pop('HERMES_CONFIG_INSTANCE_ID')}\n")+ (home / ".env").write_text("".join(f"{k}={v}\n" for k, v in deployment.items() if k != "HERMES_CONFIG_BACKEND"))+ for name in remote_env(plane):+ env.pop(name, None)+ env.update(HERMES_CONFIG_BACKEND="remote", HERMES_MANAGED_DIR=str(managed))+ code = ("import json\n"+ "import hermes_cli.config\n"+ "from hermes_cli.env_loader import load_hermes_dotenv\n"+ "load_hermes_dotenv()\n"+ "from hermes_cli.config import load_config\n"+ "print('RESULT=' + json.dumps(load_config()['display']['personality']))\n")+ assert _result(_run([sys.executable, "-c", code], env)) == "concise"+ gets = [r for r in plane.requests if r["method"] == "GET"]+ assert gets and {r["instance"] for r in gets} == {INSTANCE}diff --git a/tests/tools/test_stage2_hook_remote_config_skip.py b/tests/tools/test_stage2_hook_remote_config_skip.pyindex adcf681fde9..b8c92b12a55 100644--- a/tests/tools/test_stage2_hook_remote_config_skip.py+++ b/tests/tools/test_stage2_hook_remote_config_skip.py@@ -1,49 +1,95 @@"""stage2 skips ``docker_config_migrate.py`` in remote config mode (config-config P4, D12): theagent migrates the fetched document in memory, so a leftover local config.yaml must not be-migrated. The mode comes from the container env or from ``$HERMES_HOME/.env``."""+migrated. stage2 decides with the agent's own backend selection, so every dotenv form the runtime+accepts (comments, spaces around ``=``, quotes, ``export``) and the runtime's precedence (the home's+``.env`` over the container env) decide the same way at both places."""from __future__ import annotations+import osimport shutilimport subprocess+import sysfrom pathlib import Pathimport pytest-STAGE2_HOOK = Path(__file__).resolve().parents[2] / "docker" / "stage2-hook.sh"+REPO = Path(__file__).resolve().parents[2]+STAGE2_HOOK = REPO / "docker" / "stage2-hook.sh"def _block() -> str:text = STAGE2_HOOK.read_text()- start = text.index("_remote_config_mode=0")+ start = text.index("# --- Migrate persisted config schema ---")end = text.index("\nfi\n", text.index("docker_config_migrate.py failed", start)) + len("\nfi\n")return text[start:end]+def _env(root: Path, backend_env: str | None) -> dict:+ env = {k: v for k, v in os.environ.items() if k != "HERMES_CONFIG_BACKEND"}+ env.update(PYTHONPATH=str(REPO), HERMES_MANAGED_DIR=str(root / "managed"))+ if backend_env is not None:+ env["HERMES_CONFIG_BACKEND"] = backend_env+ return env++def _run(home: Path, backend_env: str | None) -> str:+ """The checked-in stage2 block, with only the migration replaced by a marker."""if shutil.which("sh") is None:pytest.skip("sh not available")- env_line = "unset HERMES_CONFIG_BACKEND\n" if backend_env is None else f"HERMES_CONFIG_BACKEND='{backend_env}'\n"- bin_dir = home.parent / "bin" # s6-setuidgid is not a valid sh function name: stub it on PATH+ root = home.parent+ bin_dir = root / "bin" # s6-setuidgid is not a valid sh function name: stub it on PATHbin_dir.mkdir(exist_ok=True)stub = bin_dir / "s6-setuidgid"- stub.write_text("#!/bin/sh\necho MIGRATE_RAN\n")+ stub.write_text('#!/bin/sh\nshift\ncase "$*" in *docker_config_migrate.py*) echo MIGRATE_RAN ;; *) exec "$@" ;; esac\n')stub.chmod(0o755)+ venv_bin = root / "install" / ".venv" / "bin"+ venv_bin.mkdir(parents=True, exist_ok=True)+ python = venv_bin / "python" # a wrapper, not a symlink: a moved venv symlink loses its site-packages+ python.write_text(f'#!/bin/sh\nexec "{sys.executable}" "$@"\n')+ python.chmod(0o755)script = (- "set -eu\n" + env_line- + f'PATH="{bin_dir}:$PATH"\nHERMES_HOME="{home}"\nINSTALL_DIR=/nonexistent\n'+ "set -eu\n"+ + f'PATH="{bin_dir}:$PATH"\nHERMES_HOME="{home}"\nINSTALL_DIR="{root / "install"}"\n'+ _block())- proc = subprocess.run(["sh", "-c", script], capture_output=True, text=True, timeout=30,- stdin=subprocess.DEVNULL)+ proc = subprocess.run(["sh", "-c", script], capture_output=True, text=True, timeout=60,+ stdin=subprocess.DEVNULL, env=_env(root, backend_env))assert proc.returncode == 0, proc.stderr+ assert "could not determine the config backend" not in proc.stdout, proc.stdoutreturn proc.stdout+def _runtime_selects(home: Path, backend_env: str | None) -> str:+ """The backend a real Hermes start selects: the first config read (``hermes_cli.config`` reads+ at import time) or ``load_hermes_dotenv``'s boot, whichever selects first, reports the backend+ and ends the process right there (there is no plane in this test)."""+ code = ("import os, sys\n"+ "from hermes_cli.config_backend import FileBackend\n"+ "from plugins.config_backends.remote.backend import RemoteBackend\n"+ "def report(name):\n"+ " def stop(self, *args, **kwargs):\n"+ " print(name, flush=True)\n"+ " os._exit(0)\n"+ " return stop\n"+ "RemoteBackend._state = report('remote')\n"+ "FileBackend.boot = report('file')\n"+ "from hermes_cli import env_loader\n"+ "env_loader.load_hermes_dotenv(hermes_home=sys.argv[1], load_external_secrets=False)\n"+ "sys.exit('no backend was selected')\n")+ env = _env(home.parent, backend_env)+ env["HERMES_HOME"] = str(home)+ proc = subprocess.run([sys.executable, "-c", code, str(home)], capture_output=True, text=True,+ timeout=60, env=env, cwd=str(home.parent))+ assert proc.returncode == 0, proc.stderr+ return proc.stdout.strip().splitlines()[-1]++@pytest.fixturedef home(tmp_path):home = tmp_path / "home"home.mkdir()(home / "config.yaml").write_text("model: {}\n")+ (tmp_path / "managed").mkdir()return home@@ -56,13 +102,24 @@ def test_remote_from_container_env_skips(home):assert "MIGRATE_RAN" not in out and "skipping docker_config_migrate.py" in out[email protected]("line", ["HERMES_CONFIG_BACKEND=remote", 'HERMES_CONFIG_BACKEND="remote"',- "export HERMES_CONFIG_BACKEND='remote'"])-def test_remote_from_dotenv_skips(home, line):- (home / ".env").write_text(f"OTHER=1\n{line}\n")[email protected]("dotenv, container", [+ ("HERMES_CONFIG_BACKEND=remote", None),+ ('HERMES_CONFIG_BACKEND="remote"', None),+ ("export HERMES_CONFIG_BACKEND='remote'", None),+ ("HERMES_CONFIG_BACKEND=remote # fleet-managed", None),+ ("HERMES_CONFIG_BACKEND = remote", None),+ ("HERMES_CONFIG_BACKEND=file\n# HERMES_CONFIG_BACKEND=remote", None),+ ("HERMES_CONFIG_BACKEND=file", "remote"),+ ("HERMES_CONFIG_BACKEND=remote", "file"),+])+def test_stage2_decides_like_the_runtime(home, dotenv, container):+ (home / ".env").write_text(f"OTHER=1\n{dotenv}\n")+ runtime = _runtime_selects(home, container)+ migrated = "MIGRATE_RAN" in _run(home, container)+ assert migrated == (runtime == "file"), (dotenv, container, runtime)+++def test_remote_from_managed_dotenv_skips(home):+ (home.parent / "managed" / ".env").write_text("HERMES_CONFIG_BACKEND=remote\n")+ assert _runtime_selects(home, None) == "remote"assert "MIGRATE_RAN" not in _run(home, None)---def test_other_dotenv_values_still_migrate(home):- (home / ".env").write_text("HERMES_CONFIG_BACKEND=file\n# HERMES_CONFIG_BACKEND=remote\n")- assert "MIGRATE_RAN" in _run(home, None)diff --git a/tools/environments/local.py b/tools/environments/local.pyindex 7b8edef590c..8e2e93df5ed 100644--- a/tools/environments/local.py+++ b/tools/environments/local.py@@ -511,9 +511,12 @@ def strip_launch_profile_env(env: dict, target_home: str | Path | None = None) -# LAUNCH profile's source supplied is not the target profile's to see; the caller's scope# overlay puts back exactly the ones the target's own sources supply. The administrator-managed# .env is NOT residue: its values are policy for every profile (``_apply_managed_env`` applies- # it last, with override, so it beats the user's own .env) — leave them in place.+ # it last, with override, so it beats the user's own .env) — leave them in place. Neither is+ # the config backend's deployment (which backend, which plane, which instance): it is the+ # instance's, and a child without it reads the target profile's config from another backend.+ from hermes_cli.config_backend import deployment_env_namesfrom hermes_cli.env_loader import launch_dotenv_keys, managed_dotenv_keys, source_supplied_names- managed_names = {key.upper() for key in managed_dotenv_keys()}+ managed_names = {key.upper() for key in managed_dotenv_keys()} | set(deployment_env_names())residue_names = {key.upper() for key inset(load_env_file(launch_home / ".env")) | set(launch_dotenv_keys())diff --git a/tui_gateway/methods_config_set.py b/tui_gateway/methods_config_set.pyindex 5ca0a5175a1..ad8979c983a 100644--- a/tui_gateway/methods_config_set.py+++ b/tui_gateway/methods_config_set.py@@ -519,7 +519,11 @@ def _(rid, params: dict) -> dict:handler = _set_display_toggleif handler is None:return _err(rid, 4002, f"unknown config key: {key}")- return handler(rid, params, key, value, session)+ from hermes_cli.config_backend import ConfigWriteError+ try:+ return handler(rid, params, key, value, session)+ except ConfigWriteError as exc: # a locked key or a value the config backend refused: nothing saved+ return _err(rid, 4002, str(exc))def register(server) -> None:diff --git a/tui_gateway/rpc_dispatch.py b/tui_gateway/rpc_dispatch.pyindex b2a2587c42d..a31b553b07e 100644--- a/tui_gateway/rpc_dispatch.py+++ b/tui_gateway/rpc_dispatch.py@@ -29,11 +29,17 @@ def _handle_admitted_request(req: dict) -> dict | None:params, problem = _contracts.validate_params(contract, params)if problem is not None:return _err(rid, 4000, problem)+ from hermes_cli.config_backend import ConfigBackendUnavailable+token = _current_rpc_method.set(method)try:response = fn(rid, params)except ProfileUnavailableError as exc:return _err(rid, 4064, str(exc))+ except ConfigBackendUnavailable as exc:+ # A SystemExit on purpose (readers must not fall back to defaults), so no ``except+ # Exception`` below catches it: this request still owes its caller one answer.+ return _err(rid, 4064, str(exc.code))finally:_current_rpc_method.reset(token)if contract is not None and isinstance(response, dict) and isinstance(response.get("result"), dict):diff --git a/tui_gateway/server.py b/tui_gateway/server.pyindex 00849ea49e5..b86e15dfd2f 100644--- a/tui_gateway/server.py+++ b/tui_gateway/server.py@@ -1032,9 +1032,14 @@ def _load_cfg() -> dict:def _save_cfg(cfg: dict):global _cfg_cache, _cfg_sig, _cfg_pathfrom hermes_cli.config import atomic_config_replace- from hermes_cli.config_backend import config_cache_version+ from hermes_cli.config_backend import config_cache_version, supports_file_toolingpath = _active_config_path()atomic_config_replace(path, cfg)+ if not supports_file_tooling():+ # The backend saves what it accepts (a bulk save omits locked keys), not what was proposed:+ # drop the cache so the next raw read returns the accepted document.+ _drop_cfg_cache()+ returnwith _cfg_lock:_cfg_cache, _cfg_path = copy.deepcopy(cfg), pathtry:@@ -1043,6 +1048,12 @@ def _save_cfg(cfg: dict):_cfg_sig = None+def _drop_cfg_cache() -> None:+ global _cfg_cache, _cfg_sig, _cfg_path+ with _cfg_lock:+ _cfg_cache, _cfg_sig, _cfg_path = None, None, None++def _session_for_key(session_key: str) -> dict | None:"""First live record with this session_key (snapshot under the lock: pool handlers mutate ``_sessions``)."""with _sessions_lock:@@ -1337,6 +1348,15 @@ def _append_model_switch_marker(session: dict | None, *, model: str, provider: sdef _write_config_key(key_path: str, value):+ from hermes_cli.config_backend import Changes, get_config_backend, supports_file_tooling+ if not supports_file_tooling():+ # One explicit key: a keyed write, which refuses a locked key (ConfigLockedError) instead+ # of the bulk save's silent omission, so the caller never reports a refused value as saved.+ try:+ get_config_backend().write_changes(_active_config_path().parent, Changes(set={key_path: value}))+ finally:+ _drop_cfg_cache()+ return# Write-back round-trip: raw read is mandatory — saving the overlaid/expanded view would persist it.cfg = current = _load_cfg_raw()*parents, leaf = key_path.split(".")diff --git a/website/docs/user-guide/remote-config.md b/website/docs/user-guide/remote-config.mdindex 28b59cbd467..20e416bd272 100644--- a/website/docs/user-guide/remote-config.md+++ b/website/docs/user-guide/remote-config.md@@ -17,9 +17,9 @@ machine with one user should keep `config.yaml`.## Turning it on-Set the following in the environment or in `~/.hermes/.env`. A value in-`config.yaml` cannot select the backend, because the backend is what serves-config.+Set the following in the environment, in `~/.hermes/.env`, or in the managed+`/etc/hermes/.env` (which wins over the other two). A value in `config.yaml`+cannot select the backend, because the backend is what serves config.| Variable | Meaning ||---|---|@@ -41,6 +41,14 @@ These values, and the `HERMES_CONFIG_*` variables above, must come from(1Password, Bitwarden, a command) supplies any of them, Hermes refuses tostart.+A Hermes process started for another profile (a `hermes -p <name>` worker, a+profile's cron or bot child) keeps the `HERMES_CONFIG_*` deployment of the+process that started it, so it reads that profile's config from the plane too.+It does not inherit the plane credential: on-prem, put the+`GATEWAY_RELAY_IDP_*` values where every profile reads them (the managed+`/etc/hermes/.env`) or in each profile's `.env`. Without one, the process+exits with an error rather than read a local file.+## What changes- **Startup fetches config and fails closed.** Every Hermes process (gateway,@@ -57,12 +65,15 @@ start.running agent within one interval.- **Writes go to the profile's own level.** `hermes config set`, `/model`,the dashboard, and every other config writer send only the keys that changed,- guarded against concurrent edits. Removing a key removes it from the profile- level only: if an upper level also sets it, that value still applies, and- Hermes warns you.-- **Locked keys are refused.** `hermes config set` on a locked key prints which- level locks it and changes nothing. When a whole document is saved, locked- keys are left out, and Hermes prints a note listing them.+ guarded against concurrent edits. If another process changed the profile+ meanwhile, Hermes re-reads it and applies only its own change on top, never+ undoing the other one. Removing a key removes it from the profile level only:+ if an upper level also sets it, that value still applies, and Hermes warns+ you.+- **Locked keys are refused.** `hermes config set` (and a single setting+ changed in the TUI or desktop app) on a locked key reports which level locks+ it and changes nothing. When a whole document is saved, locked keys are left+ out, and Hermes prints a note listing them.- **Secrets stay out of the plane.** A secret-shaped key such as `api_key`accepts only a `${VAR}` reference, never the secret itself. Put the value in`.env` or a secret source and set the reference:@@ -76,7 +87,11 @@ start.- **Schema migrations run in memory.** A document written by an older Hermes ismigrated when it is read, and the result is not written back. A later writefrom a newer Hermes keeps the profile's stored schema version unless that- data is already current, so every later reader still migrates it.+ data is already current, so every later reader still migrates it. For the+ same reason, setting a value that one of those migrations rewrites (for+ example `compression.threshold_tokens: 256000`, an old default, on a profile+ stored at an older schema) is refused with a message naming the key: it would+ be saved and then never read back.- **Unknown keys** in the fetched document are ignored with a warning, forexample a key added by a newer Hermes.@@ -92,7 +107,8 @@ warns at startup, and `hermes doctor` flags the file. The managedThese commands copy or edit `config.yaml` and are refused in remote mode:- `hermes config edit` (use `hermes config set` instead)-- profile clone and profile distribution install+- profile clone, profile rename (the plane keeps a profile's settings under its+ name), and profile distribution install- `hermes backup`, `hermes import`, and snapshot restore- `hermes gateway --config <file>`