Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
fix(providers): harden the passive health record after review
rubenluetke10-beepf4e3f1757424
diff --git a/jarvis/brain/manager.py b/jarvis/brain/manager.pyindex 45832e2a34..6713ca065c 100644--- a/jarvis/brain/manager.py+++ b/jarvis/brain/manager.py@@ -11947,10 +11947,13 @@ class BrainManager:msg = str(exc)kind = _classify_provider_error(msg, default="init_fail")# The status dots read what real turns did; the record keeps- # only the classification, never this message (AP-34).- _health_ledger.record_failure(- prov_name, _health_ledger.MODALITY_BRAIN, exc, model=model- )+ # only the classification, never this message (AP-34). An+ # overridden turn (an agent chat) runs on its own credential+ # scope and model, so it is not the Brain tab's evidence.+ if turn_override is None:+ _health_ledger.record_failure(+ prov_name, _health_ledger.MODALITY_BRAIN, exc, model=model+ )# On missing_key: remove provider from the chain for the rest# of the session. Prevents each voice turn from running 8x# sequentially against the same missing keys. Never for an@@ -12323,9 +12326,10 @@ class BrainManager:# tool was actually called this turn._turn_executed = set(executed)used_provider, used_model = prov_name, model- _health_ledger.record_success(- prov_name, _health_ledger.MODALITY_BRAIN, model=model- )+ if turn_override is None:+ _health_ledger.record_success(+ prov_name, _health_ledger.MODALITY_BRAIN, model=model+ )# Bug C Fix (2026-04-29) — BrainTurnStarted/Completed publishen# NUR wenn der Brain-Call erfolgreich war (Stream lieferte@@ -12430,9 +12434,10 @@ class BrainManager:# router bricked the whole turn even though OpenRouter was funded# — AP-22). Only a genuinely transient 429 takes the cooldown path.kind = _classify_provider_error(msg, default="call_fail")- _health_ledger.record_failure(- prov_name, _health_ledger.MODALITY_BRAIN, exc, model=model- )+ if turn_override is None:+ _health_ledger.record_failure(+ prov_name, _health_ledger.MODALITY_BRAIN, exc, model=model+ )if kind == "rate_limit":self._rate_tracker.mark_rate_limited(prov_name, model)log.warning("Rate-Limited %s(%s) — 30s Cooldown aktiviert", prov_name, model)diff --git a/jarvis/brain/provider_health_ledger.py b/jarvis/brain/provider_health_ledger.pyindex 428f575a32..7e25909e2e 100644--- a/jarvis/brain/provider_health_ledger.py+++ b/jarvis/brain/provider_health_ledger.py@@ -19,9 +19,11 @@ What is kept, per ``(provider id, modality)``:* the provider-test status of the outcome (``ok``, ``bad_key``, ``no_credits``,``rate_limited``, ``model_unavailable``, ``unreachable`` — the vocabulary of- :mod:`jarvis.brain.provider_test`, so the rollup maps it exactly as it mapped a- probe), the time it happened, the model when the caller knew it, and whether- it came from normal use or an explicit test.+ :mod:`jarvis.brain.provider_test`), the time it happened, the model when the+ caller knew it, and whether it came from normal use or an explicit test;+* for a credential failure, a short one-way fingerprint of the credential it+ was judged against, so a key replaced in another process (the terminal setup+ wizard, an external keyring edit) voids the stale verdict.What is never kept: a provider's error body, a key, a prompt, a transcript(AP-34). A failure is classified the moment it is reported and only the@@ -29,34 +31,44 @@ classification survives.Rules the readers rely on:-* An unrecognised failure (``error``) from normal use is NOT evidence about the- provider — it may be a tool, a cancelled turn or our own bug — so it is not- recorded. The explicit Test button's verdict is recorded whatever it says.+* A failure of normal use counts only when it is recognisably the provider's+ answer — an HTTP status in the message, a transport failure, or a provider+ SDK error type. Marker words alone ("credit", "network") also appear in tool+ and app errors, and a false ``no_credits`` would never expire.+* An unrecognised failure (``error``) from normal use is not recorded; the+ explicit Test button's verdict is recorded whatever it says.* ``bad_key`` / ``no_credits`` are failures of the CREDENTIAL, which every- modality of the same provider id shares; a newer one on a sibling modality- wins over an older own outcome (see :meth:`ProviderHealthLedger.effective`).+ modality of the same provider id shares (see :meth:`ProviderHealthLedger.effective`).* Transient outcomes (``rate_limited``, ``unreachable``) age out after- :data:`TRANSIENT_TTL_S`, so a blip never paints a dot red for the rest of the- day while nobody uses that provider.-* Saving a new key forgets what the old key did (:func:`forget_providers`), and- changing a model forgets the model-dependent outcomes of that modality.--The record lives in memory and in ``DATA_DIR/state/provider_health.json`` so the-dots survive a restart. The file is written only when an outcome CHANGES (or an-unchanged one is older than :data:`_REFRESH_WRITE_S` on disk), never per call,-so the hot voice path pays a dict update and nothing else (AP-9). Nothing-initialises at import time (AP-26): the file is read on first use.+ :data:`TRANSIENT_TTL_S`.+* Any credential write or delete in this process forgets what the providers+ reading that slot did (wired by the web layer through+ ``jarvis.core.config.add_secret_change_listener``); changing a model forgets+ the model-dependent outcomes of that modality.++Every status change is announced to registered listeners (the web server turns+it into a ``ProviderHealthChanged`` bus event, which reaches every open window+over the existing WebSocket). The record lives in memory and in+``DATA_DIR/state/provider_health.json`` so the dots survive a restart. The+process-wide instance reads and writes that file on a background thread and+writes only when an outcome CHANGES (or an unchanged one is older than+:data:`_REFRESH_WRITE_S` on disk), so the voice path pays a dict update and+nothing else (AP-9). Nothing initialises at import time (AP-26)."""from __future__ import annotations+import asyncio+import hashlibimport jsonimport loggingimport os+import reimport threadingimport timeimport uuidfrom collections.abc import Callable, Iterable-from dataclasses import dataclass+from concurrent.futures import ThreadPoolExecutor+from dataclasses import dataclass, replacefrom pathlib import Pathfrom typing import Any@@ -121,12 +133,50 @@ _ALIASES: dict[tuple[str, str], str] = {(MODALITY_TTS, "open-router-tts"): "openrouter-tts",}+# ── What makes a failure the provider's own answer ───────────────────────────+_HTTP_STATUS_RE = re.compile(r"\b[45]\d\d\b")+#: Name fragments of the exception types provider SDKs and HTTP/WebSocket+#: transports raise (OpenAI-compatible, Anthropic, Google GenAI, httpx,+#: websockets). Matched by name so no SDK is imported here (AP-28).+_PROVIDER_ERROR_TYPE_TOKENS: tuple[str, ...] = (+ "APIStatus",+ "APIConnection",+ "APITimeout",+ "Authentication",+ "PermissionDenied",+ "RateLimit",+ "InternalServer",+ "ServiceUnavailable",+ "ClientError",+ "ServerError",+ "ResourceExhausted",+ "HTTPStatus",+ "ConnectError",+ "ConnectTimeout",+ "ReadTimeout",+ "RemoteProtocol",+ "ConnectionClosed",+ "InvalidStatus",+)+def _norm_provider(provider: str | None, modality: str) -> str:key = (provider or "").strip().casefold()return _ALIASES.get((modality, key), key)+def credential_fingerprint(credential: str | None) -> str:+ """A short one-way fingerprint of a credential ("" when there is none).++ Domain-separated and truncated: enough to notice that the key changed,+ useless for recovering it. Never the credential itself.+ """+ if not credential:+ return ""+ digest = hashlib.sha256(b"jarvis-provider-health\x00" + credential.encode("utf-8"))+ return digest.hexdigest()[:16]++@dataclass(frozen=True, slots=True)class Outcome:"""One remembered outcome. Never carries an error body or a secret."""@@ -137,6 +187,9 @@ class Outcome:at: floatsource: str = SOURCE_USEmodel: str = ""+ #: :func:`credential_fingerprint` of the key a credential failure was+ #: judged against; stamped on first read, "" until then.+ credential: str = ""def to_json(self) -> dict[str, Any]:return {@@ -146,6 +199,7 @@ class Outcome:"at": self.at,"source": self.source,"model": self.model,+ "credential": self.credential,}@classmethod@@ -159,7 +213,7 @@ class Outcome:source = str(raw.get("source") or SOURCE_USE)try:at = float(raw.get("at"))- except (TypeError, ValueError):+ except (TypeError, ValueError): # a malformed entry is dropped, never fatalreturn Noneif not provider or modality not in MODALITIES or source not in SOURCES:return None@@ -173,14 +227,20 @@ class Outcome:at=at,source=source,model=str(raw.get("model") or ""),+ credential=str(raw.get("credential") or ""),)-def failure_status(error: BaseException | str | None) -> str:+def failure_status(error: BaseException | str | None, *, strict: bool = False) -> str:"""Classify a failure into the provider-test vocabulary.The one classifier the Test button uses, so a real call and a test cannever disagree about what "401" or "insufficient_quota" means.++ ``strict`` (normal use) returns ``error`` — "not evidence" — unless the+ failure is recognisably the provider's own answer: an HTTP status in the+ message, a timeout/connection failure, or a provider SDK/transport error+ type. Marker words alone also occur in tool and app errors."""if error is None:return _pt.ERROR@@ -188,26 +248,76 @@ def failure_status(error: BaseException | str | None) -> str:text = f"{type(error).__name__}: {error}"else:text = str(error)- return _pt.classify_provider_error(text)+ status = _pt.classify_provider_error(text)+ if not strict or _HTTP_STATUS_RE.search(text):+ return status+ if isinstance(error, BaseException):+ if isinstance(error, TimeoutError | ConnectionError):+ return _pt.UNREACHABLE+ names = [cls.__name__ for cls in type(error).__mro__]+ if any(token in name for name in names for token in _PROVIDER_ERROR_TYPE_TOKENS):+ return status+ return _pt.ERROR+++ChangeListener = Callable[[str, str, str], None]class ProviderHealthLedger:- """Thread-safe record of the last real outcome per provider and modality."""+ """Thread-safe record of the last real outcome per provider and modality.++ ``background_io=False`` (tests, tools) reads the file on first access and+ writes synchronously. ``background_io=True`` (the process-wide instance)+ never touches the disk on the caller's thread: :meth:`start_loading` reads+ it on a worker thread and merges it in (a newer in-memory outcome wins),+ and writes go to a single background writer in order.+ """def __init__(self,path: Path | None = None,*,clock: Callable[[], float] = time.time,+ background_io: bool = False,) -> None:self._path = pathself._clock = clock+ self._background = background_io and path is not Noneself._lock = threading.Lock()self._write_lock = threading.Lock()self._entries: dict[tuple[str, str], Outcome] = {}self._written_at: dict[tuple[str, str], float] = {}self._loaded = path is None+ self._load_started = False+ self._load_done = threading.Event()+ if self._loaded:+ self._load_done.set()self._version = 0+ self._write_seq = 0+ self._written_seq = 0+ self._executor: ThreadPoolExecutor | None = None+ self._listeners: list[ChangeListener] = []++ # ── listeners ────────────────────────────────────────────────────────+ def add_listener(self, listener: ChangeListener) -> None:+ """``listener(provider, modality, status)`` after every status change.++ All three are "" for a bulk change. Called outside the ledger lock, on+ whichever thread recorded the outcome; it must be cheap and must not+ raise (a failing one is logged and skipped).+ """+ with self._lock:+ if listener not in self._listeners:+ self._listeners.append(listener)++ def _notify(self, provider: str, modality: str, status: str) -> None:+ with self._lock:+ listeners = list(self._listeners)+ for listener in listeners:+ try:+ listener(provider, modality, status)+ except Exception: # noqa: BLE001 — a follower must never cost the call+ log.warning("Provider health: change listener failed", exc_info=True)# ── reading ──────────────────────────────────────────────────────────@property@@ -222,12 +332,23 @@ class ProviderHealthLedger:self._ensure_loaded_locked()return self._entries.get((_norm_provider(provider, modality), modality))+ def has_credential_failure(self, provider: str) -> bool:+ """Whether any modality of ``provider`` holds a key-level failure."""+ with self._lock:+ self._ensure_loaded_locked()+ return any(+ outcome.status in CREDENTIAL_STATUSES+ for (p, m), outcome in self._entries.items()+ if p == _norm_provider(provider, m)+ )+def effective(self,provider: str,modality: str,*,now: float | None = None,+ credential: str | None = None,) -> Outcome | None:"""The outcome that should colour ``provider``'s ``modality`` right now.@@ -236,18 +357,40 @@ class ProviderHealthLedger:credential failure recorded on ANOTHER modality wins when it is newerthan this modality's own outcome — and this modality's own credentialfailure is void once a sibling answered after it;+ * ``credential`` (the current key's :func:`credential_fingerprint`,+ when the caller read it) stamps a credential failure that has none+ yet and voids one judged against a different key;* otherwise this modality's own outcome, or ``None`` (never used yet)."""current = self._clock() if now is None else nowkey_provider = _norm_provider(provider, modality)+ voided = False+ stamped = Falsewith self._lock:self._ensure_loaded_locked()+ if credential:+ for key, outcome in list(self._entries.items()):+ if key[0] != key_provider or outcome.status not in CREDENTIAL_STATUSES:+ continue+ if not outcome.credential:+ self._entries[key] = replace(outcome, credential=credential)+ stamped = True+ elif outcome.credential != credential:+ self._entries.pop(key, None)+ self._written_at.pop(key, None)+ voided = True+ if voided:+ self._version += 1own = self._entries.get((key_provider, modality))siblings = [outcomefor (p, m), outcome in self._entries.items()if p == key_provider and m != modality]+ if voided or stamped:+ self._request_write()+ if voided:+ self._notify(key_provider, "", "")if own is not None and self._expired(own, current):own = Nonenewest_credential: Outcome | None = None@@ -311,6 +454,14 @@ class ProviderHealthLedger:with self._lock:self._ensure_loaded_locked()previous = self._entries.get(key)+ if (+ previous is not None+ and previous.status == status+ and previous.credential+ and status in CREDENTIAL_STATUSES+ ):+ # The same verdict on the same (already fingerprinted) key.+ outcome = replace(outcome, credential=previous.credential)self._entries[key] = outcomechanged = previous is None or previous.status != statusif changed:@@ -318,7 +469,6 @@ class ProviderHealthLedger:due = changed or now - self._written_at.get(key, 0.0) >= _REFRESH_WRITE_Sif due:self._written_at[key] = now- payload = self._payload_locked() if due else Noneif changed:log.info("Provider health: %s/%s %s -> %s (%s)",@@ -328,8 +478,10 @@ class ProviderHealthLedger:status,source,)- if payload is not None:- self._persist(payload)+ if due:+ self._request_write()+ if changed:+ self._notify(key_provider, modality, status)return Truedef record_failure(@@ -342,9 +494,8 @@ class ProviderHealthLedger:model: str | None = None,) -> bool:"""Classify ``error`` and remember it when it is evidence (see module doc)."""- return self.record(- provider, modality, failure_status(error), source=source, model=model- )+ status = failure_status(error, strict=source == SOURCE_USE)+ return self.record(provider, modality, status, source=source, model=model)def forget(self,@@ -380,73 +531,150 @@ class ProviderHealthLedger:self._written_at.pop(key, None)if doomed:self._version += 1- payload = self._payload_locked() if doomed else None- if payload is not None:- self._persist(payload)+ if doomed:+ self._request_write()+ self._notify("", "", "")return len(doomed)- # ── internals ────────────────────────────────────────────────────────- @staticmethod- def _expired(outcome: Outcome, now: float) -> bool:- return outcome.status in TRANSIENT_STATUSES and now - outcome.at > TRANSIENT_TTL_S-- def _payload_locked(self) -> dict[str, Any]:- return {- "version": _FILE_VERSION,- "outcomes": [outcome.to_json() for outcome in self._entries.values()],- }+ # ── disk ─────────────────────────────────────────────────────────────+ def start_loading(self) -> None:+ """Read the file on a worker thread (background mode; idempotent)."""+ if not self._background:+ return+ with self._lock:+ if self._loaded or self._load_started:+ return+ self._load_started = True+ threading.Thread(+ target=self._load_in_background, name="provider-health-load", daemon=True+ ).start()++ def flush(self, timeout: float = 30.0) -> None:+ """Wait until the background loader and every queued write are done."""+ self._load_done.wait(timeout)+ executor = self._executor+ if executor is not None:+ executor.submit(lambda: None).result(timeout)++ def _load_in_background(self) -> None:+ added = False+ pending_write = False+ try:+ outcomes = self._read_file()+ with self._lock:+ added = self._merge_locked(outcomes)+ self._loaded = True+ pending_write = self._write_seq > self._written_seq+ finally:+ self._load_done.set()+ if added:+ self._notify("", "", "")+ if added and pending_write:+ # A change recorded while loading was written without the file's+ # history only if its writer gave up waiting; write the merge.+ self._request_write()def _ensure_loaded_locked(self) -> None:- if self._loaded:+ if self._loaded or self._background:+ # Background mode never reads on the caller's thread; the loader+ # merges the file in when it is done.returnself._loaded = True+ self._merge_locked(self._read_file())+ self._load_done.set()++ def _merge_locked(self, outcomes: list[Outcome]) -> bool:+ added = False+ for outcome in outcomes:+ key = (outcome.provider, outcome.modality)+ if key in self._entries:+ continue # a newer in-memory outcome wins over the file+ self._entries[key] = outcome+ self._written_at[key] = outcome.at+ added = True+ if added:+ self._version += 1+ return added++ def _read_file(self) -> list[Outcome]:path = self._pathif path is None:- return+ return []try:raw = json.loads(path.read_text(encoding="utf-8"))- except FileNotFoundError:- return+ except FileNotFoundError: # no record yet: a fresh install starts empty+ return []except (OSError, ValueError):# A damaged record only costs the history; the next real call# writes a fresh one.log.info("Provider health: %s unreadable, starting empty", path, exc_info=True)- return+ return []items = raw.get("outcomes") if isinstance(raw, dict) else Noneif not isinstance(items, list):- return- for item in items:- outcome = Outcome.from_json(item)- if outcome is None:- continue- key = (outcome.provider, outcome.modality)- self._entries[key] = outcome- self._written_at[key] = outcome.at- if self._entries:- self._version += 1+ return []+ return [outcome for item in items if (outcome := Outcome.from_json(item)) is not None]- def _persist(self, payload: dict[str, Any]) -> None:+ def _request_write(self) -> None:+ if self._path is None:+ return+ with self._lock:+ self._write_seq += 1+ if not self._background:+ self._write_latest()+ return+ with self._lock:+ if self._executor is None:+ self._executor = ThreadPoolExecutor(+ max_workers=1, thread_name_prefix="provider-health-write"+ )+ executor = self._executor+ executor.submit(self._write_latest)++ def _write_latest(self) -> None:+ """Write the CURRENT state, and never an older one over a newer one."""+ if self._background and not self._load_done.wait(30.0):+ # Writing before the file was merged in would drop its history;+ # the next change writes once the loader has finished.+ log.info("Provider health: file not loaded yet, write deferred")+ return+ with self._write_lock:+ with self._lock:+ seq = self._write_seq+ payload = {+ "version": _FILE_VERSION,+ "outcomes": [outcome.to_json() for outcome in self._entries.values()],+ }+ if seq <= self._written_seq:+ return+ self._write_file(payload)+ self._written_seq = seq++ def _write_file(self, payload: dict[str, Any]) -> None:path = self._pathif path is None:return- with self._write_lock:- temporary = path.with_name(f".{path.name}.{os.getpid()}.{uuid.uuid4().hex}.tmp")+ temporary = path.with_name(f".{path.name}.{os.getpid()}.{uuid.uuid4().hex}.tmp")+ try:+ path.parent.mkdir(parents=True, exist_ok=True)+ temporary.write_text(json.dumps(payload, indent=2), encoding="utf-8")+ os.replace(temporary, path)+ except OSError:+ log.warning("Provider health: %s not written", path, exc_info=True)+ finally:try:- path.parent.mkdir(parents=True, exist_ok=True)- temporary.write_text(json.dumps(payload, indent=2), encoding="utf-8")- os.replace(temporary, path)+ temporary.unlink(missing_ok=True)except OSError:- log.warning("Provider health: %s not written", path, exc_info=True)- finally:- try:- temporary.unlink(missing_ok=True)- except OSError:- log.debug("Provider health: temporary file cleanup failed", exc_info=True)+ log.debug("Provider health: temporary file cleanup failed", exc_info=True)++ @staticmethod+ def _expired(outcome: Outcome, now: float) -> bool:+ return outcome.status in TRANSIENT_STATUSES and now - outcome.at > TRANSIENT_TTL_S# ── process-wide instance ────────────────────────────────────────────────────_ledger: ProviderHealthLedger | None = None_ledger_guard = threading.Lock()+_global_listeners: list[ChangeListener] = []def ledger_path() -> Path:@@ -460,9 +688,15 @@ def get_ledger() -> ProviderHealthLedger:"""The process-wide ledger, created on first use (never at import)."""global _ledgerwith _ledger_guard:+ created = _ledger is Noneif _ledger is None:- _ledger = ProviderHealthLedger(ledger_path())- return _ledger+ _ledger = ProviderHealthLedger(ledger_path(), background_io=True)+ for listener in _global_listeners:+ _ledger.add_listener(listener)+ ledger = _ledger+ if created:+ ledger.start_loading()+ return ledgerdef set_ledger(ledger: ProviderHealthLedger | None) -> None:@@ -470,13 +704,60 @@ def set_ledger(ledger: ProviderHealthLedger | None) -> None:global _ledgerwith _ledger_guard:_ledger = ledger+ if ledger is not None:+ for listener in _global_listeners:+ ledger.add_listener(listener)+++def add_change_listener(listener: ChangeListener) -> None:+ """Register ``listener`` on the process-wide ledger, now and after a swap."""+ with _ledger_guard:+ if listener not in _global_listeners:+ _global_listeners.append(listener)+ ledger = _ledger+ if ledger is not None:+ ledger.add_listener(listener)+++def publish_changes_to(bus: Any, loop: asyncio.AbstractEventLoop) -> ChangeListener:+ """Announce every status change as a ``ProviderHealthChanged`` bus event.++ The bus forwards every event to every open window over the ONE existing+ WebSocket, so the dots re-read without a poll and without a new socket+ (AP-33). Thread-safe: an outcome recorded on a worker thread is handed to+ ``loop``. Returns the registered listener.+ """+ from jarvis.core.events import ProviderHealthChanged++ pending: set[asyncio.Future[Any]] = set()++ def _publish(event: Any) -> None:+ task = asyncio.ensure_future(bus.publish(event))+ pending.add(task)+ task.add_done_callback(pending.discard)++ def _listener(provider: str, modality: str, status: str) -> None:+ event = ProviderHealthChanged(+ source_layer="brain.provider_health",+ provider=provider,+ modality=modality,+ status=status,+ )+ try:+ loop.call_soon_threadsafe(_publish, event)+ except RuntimeError:+ # The loop is closed (shutdown): nobody is left to tell.+ log.debug("Provider health: event loop closed, change not announced")++ add_change_listener(_listener)+ return _listenerdef _safely(action: Callable[[], Any], what: str) -> Any:"""Run a ledger action for a caller on a live call path.A health record must never cost the call it describes, so a failure here- is logged (once per occurrence, at warning) and swallowed on purpose.+ is logged and swallowed on purpose."""try:return action()@@ -555,11 +836,22 @@ def forget_providers()-def effective_outcome(provider: str | None, modality: str) -> Outcome | None:+def effective_outcome(+ provider: str | None, modality: str, *, credential: str | None = None+) -> Outcome | None:"""See :meth:`ProviderHealthLedger.effective`; ``None`` on any failure."""if not provider:return None- return _safely(lambda: get_ledger().effective(provider, modality), "effective_outcome")+ return _safely(+ lambda: get_ledger().effective(provider, modality, credential=credential),+ "effective_outcome",+ )+++def has_credential_failure(provider: str | None) -> bool:+ if not provider:+ return False+ return bool(_safely(lambda: get_ledger().has_credential_failure(provider), "lookup"))def ledger_version() -> int:@@ -599,13 +891,17 @@ __all__ = ["TRANSIENT_STATUSES","TRANSIENT_TTL_S","USE_STATUSES",+ "add_change_listener",+ "credential_fingerprint","effective_outcome","failure_status","forget_providers","get_ledger",+ "has_credential_failure","ledger_path","ledger_version","modality_for_tier",+ "publish_changes_to","record_failure","record_status","record_success",diff --git a/jarvis/brain/section_health.py b/jarvis/brain/section_health.pyindex 6e78d22a81..fcc1370808 100644--- a/jarvis/brain/section_health.py+++ b/jarvis/brain/section_health.py@@ -75,6 +75,27 @@ def section_status_for_test(test_status: str | None, *, configured: bool) -> strreturn ERROR+#: Outcomes that clear on their own (``provider_test.RATE_LIMITED`` /+#: ``UNREACHABLE``). Spelled here to keep this module dependency-free.+_TRANSIENT_OUTCOMES = frozenset({"rate_limited", "unreachable"})+++def section_status_for_outcome(outcome_status: str) -> str:+ """Map the last REAL call's outcome (the passive health record) to a bucket.++ Unlike a fresh probe, a remembered outcome can be minutes old, and a+ throttle or a network blip has usually passed — or a fallback answered in+ its place. So transient outcomes are amber (``needs_setup``, the degraded+ state), never red; red is kept for what does not fix itself: a rejected+ key, an empty account, a missing model, a failed explicit test.+ """+ if outcome_status == _TEST_OK:+ return OK+ if outcome_status == _TEST_NOT_CONFIGURED or outcome_status in _TRANSIENT_OUTCOMES:+ return NEEDS_SETUP+ return ERROR++def aggregate(statuses: Iterable[str]) -> str:"""Roll several section statuses into one — the most urgent signal wins.@@ -96,5 +117,6 @@ __all__ = ["ERROR","UNKNOWN","section_status_for_test",+ "section_status_for_outcome","aggregate",]diff --git a/jarvis/core/config.py b/jarvis/core/config.pyindex dee0797da2..4cb061b741 100644--- a/jarvis/core/config.py+++ b/jarvis/core/config.py@@ -5048,6 +5048,15 @@ _PLATFORM_KEYRING_BACKEND: Any | None = None_LAST_KEYRING_FAILED_SLOT: str | None = None_SECRET_REVISION_LOCK = threading.Lock()_SECRET_REVISIONS: dict[str, int] = {}+# Bumped by every in-process credential write or delete, whatever the slot:+# caches that depend on "which keys exist" key on it instead of each write+# path having to remember to drop them.+_SECRET_GENERATION: int = 0+# Called with the slot name after every successful write or delete. The ONE+# place a credential change is observable in-process — every save path (the+# API-Keys routes, the Control API, CLI connect flows, the setup wizard) goes+# through ``set_secret`` / ``delete_secret``.+_SECRET_CHANGE_LISTENERS: list[Any] = []def secret_revision(key: str) -> int:@@ -5061,9 +5070,39 @@ def secret_revision(key: str) -> int:return _SECRET_REVISIONS.get(key, 0)+def secret_generation() -> int:+ """How many credential writes/deletes this process has seen (any slot)."""+ with _SECRET_REVISION_LOCK:+ return _SECRET_GENERATION+++def add_secret_change_listener(listener: Any) -> None:+ """Call ``listener(slot)`` after every successful secret write or delete.++ Idempotent per callable. A listener must be cheap and must not raise; a+ failing one is logged and skipped so a credential save never fails on it.+ """+ with _SECRET_REVISION_LOCK:+ if listener not in _SECRET_CHANGE_LISTENERS:+ _SECRET_CHANGE_LISTENERS.append(listener)++def _mark_secret_changed(key: str) -> None:+ global _SECRET_GENERATIONwith _SECRET_REVISION_LOCK:_SECRET_REVISIONS[key] = _SECRET_REVISIONS.get(key, 0) + 1+ _SECRET_GENERATION += 1+ listeners = list(_SECRET_CHANGE_LISTENERS)+ for listener in listeners:+ try:+ listener(key)+ except Exception: # noqa: BLE001 — the credential IS saved; a follower failing must not undo that+ logging.getLogger(__name__).warning(+ "secret-change listener %r failed for slot %s",+ getattr(listener, "__qualname__", listener),+ key,+ exc_info=True,+ )# Serializes _FileCredStore's load-mutate-save cycle so two in-processdiff --git a/jarvis/core/events.py b/jarvis/core/events.pyindex 3c31bcabad..b2929bcdfe 100644--- a/jarvis/core/events.py+++ b/jarvis/core/events.py@@ -451,6 +451,21 @@ class SecretConfigured(Event):action: str = "set" # "set" | "delete"+@dataclass(frozen=True, slots=True)+class ProviderHealthChanged(Event):+ """The passive provider-health record changed (a real call's outcome).++ Trigger only: open windows re-read their status dots with a plain GET, so+ a key that just failed for real shows without anyone reloading. Carries+ the classified status, never an error body or a secret (AP-34). All three+ fields are empty for a bulk change (the record loaded from disk, or a+ credential change forgot several providers at once).+ """+ provider: str = ""+ modality: str = ""+ status: str = ""++@dataclass(frozen=True, slots=True)class UiLanguageChanged(Event):"""Fired when the interface (display) language changes.diff --git a/jarvis/speech/pipeline.py b/jarvis/speech/pipeline.pyindex ffa0de5f3b..8993401f14 100644--- a/jarvis/speech/pipeline.py+++ b/jarvis/speech/pipeline.py@@ -2964,9 +2964,9 @@ class SpeechPipeline:# wrapped: ``self._stt`` is the local wake/VAD Whisper, free and# called many times a second, and metering it would be noise on the# hot path for a number that is always zero.- # No bus means nothing can be published, so nothing is wrapped either:- # ``meter_*`` hands the provider straight back for a null sink, which- # keeps a bus-less pipeline byte-for-byte what it was.+ # No bus means no spend can be published (no sink), but the providers+ # are still wrapped: the same wrapper feeds the provider-health record+ # behind the status dots, which must not depend on the cost sink.self._speech_spend = SpeechSpendRecorder(bus) if bus is not None else Noneself._tts = meter_tts(self._tts, self._speech_spend, trace_id=self._speech_trace)self._utterance_stt = meter_stt(diff --git a/jarvis/speech/usage_meter.py b/jarvis/speech/usage_meter.pyindex f91dd0c483..3c78a373cf 100644--- a/jarvis/speech/usage_meter.py+++ b/jarvis/speech/usage_meter.py@@ -31,9 +31,9 @@ transcription abandoned mid-stream still consumed the audio the provider read.Every measured call reports from a ``finally``, so an abandoned generator, acancellation and an exception all still reach the ledger.-Wrapping is free when nothing is listening: ``meter_tts(provider, None)``-returns the very same object, so an install with no cost sink pays nothing —-not an attribute lookup, not a stack frame.+The wrapper also feeds the passive provider-health record behind the status+dots (the outcome of every real speech call), so it wraps even when there is+no cost sink: ``meter_tts(provider, None)`` reports health and skips usage.Pricing lives next door in :mod:`jarvis.costs.speech_rates`; this module onlycounts. Nothing here initialises at import time (AP-26).@@ -171,7 +171,7 @@ class _MeteredProvider:def __init__(self,provider: Any,- sink: UsageSink,+ sink: UsageSink | None,*,trace_id: TraceIdSource = None,) -> None:@@ -199,8 +199,11 @@ class _MeteredProvider:"""Hand one record to the sink. Never raises, never blocks.Called from a ``finally``, which may be running during- ``GeneratorExit`` — so this is synchronous throughout.+ ``GeneratorExit`` — so this is synchronous throughout. Without a sink+ (no bus to publish spend on) there is nothing to report."""+ if self._sink is None:+ returntry:trace = self._trace_idtrace_id = str(trace() or "") if callable(trace) else str(trace or "")@@ -240,9 +243,10 @@ class _MeteredProvider:The dots never probe a provider (a probe bills the key); they read whatreal calls did, and this wrapper sees every real speech call. A failureis recorded against the provider that was asked, a success against the- vendor that actually spoke; when a fallback spoke, the failure the- asked provider absorbed (``last_failure``, when it exposes one) is- recorded too. Only the classification is kept (AP-34).+ vendor that actually spoke. A provider that swallows its own error+ (``last_failure``, when it exposes one) is charged with it whether a+ fallback then spoke or nothing came out at all. Only the+ classification is kept (AP-34).Never raises: like the usage report, health bookkeeping must not costan utterance, so a failure here is logged and dropped on purpose.@@ -255,14 +259,14 @@ class _MeteredProvider:if failure is not None:ledger.record_failure(asked, modality, failure)return+ absorbed = getattr(self._inner, "last_failure", None)+ if absorbed:+ ledger.record_failure(asked, modality, str(absorbed))if not answered:returnspoke = self._provider_label() or asked- if asked and spoke.casefold() != asked.casefold():- absorbed = getattr(self._inner, "last_failure", None)- if absorbed:- ledger.record_failure(asked, modality, str(absorbed))- ledger.record_success(spoke, modality)+ if not absorbed or spoke.casefold() != asked.casefold():+ ledger.record_success(spoke, modality)except Exception as exc: # noqa: BLE001 — see docstring: speech must not paylog.debug("Speech health record failed (%s).", exc)@@ -461,15 +465,17 @@ class MeteredSTT(_MeteredProvider):def meter_tts(provider: Any, sink: UsageSink | None, *, trace_id: TraceIdSource = None) -> Any:"""Wrap a TTS provider so every ``synthesize`` lands in the ledger.- Returns ``provider`` itself when there is nothing to report to, or when it- is already wrapped — an install without a cost sink pays nothing at all.+ Wrapped even without a cost ``sink``: the provider-health record behind+ the status dots listens to every real call either way (without a sink+ only the usage report is skipped). An already wrapped provider is+ returned as is.Typed ``Any`` in and ``Any`` out, like ``wrap_stt_with_dictionary`` nextdoor and for the same reason: ``TTSProvider.synthesize`` is declared with``async def`` while every implementation (including this wrapper) is anasync generator, so the nominal type does not describe the real contract."""- if provider is None or sink is None or isinstance(provider, MeteredTTS):+ if provider is None or isinstance(provider, MeteredTTS):return providerreturn MeteredTTS(provider, sink, trace_id=trace_id)@@ -483,10 +489,10 @@ def meter_stt(provider: Any, sink: UsageSink | None, *, trace_id: TraceIdSourcedecoding the container, and decoding to price it is exactly the kind ofwork that must never appear here.- Returns ``provider`` itself when ``sink`` is ``None`` or it is already- wrapped.+ Wrapped even without a cost ``sink`` (see :func:`meter_tts`); an already+ wrapped provider is returned as is."""- if provider is None or sink is None or isinstance(provider, MeteredSTT):+ if provider is None or isinstance(provider, MeteredSTT):return providerreturn MeteredSTT(provider, sink, trace_id=trace_id)diff --git a/jarvis/ui/web/agent_chat_routes.py b/jarvis/ui/web/agent_chat_routes.pyindex e85745d1de..f649d86e25 100644--- a/jarvis/ui/web/agent_chat_routes.py+++ b/jarvis/ui/web/agent_chat_routes.py@@ -442,9 +442,11 @@ _HEALTH_TTL_S: Final[float] = 300.0#: this runs out is reported ``unknown`` and draws no dot._HEALTH_SWEEP_S: Final[float] = 20.0-#: surface -> (checked_at, health-record version, rows). Process-local, like-#: every other short cache in the routes; a restart simply re-sweeps.-_health_cache: dict[str, tuple[float, int, list[ProviderHealthRow]]] = {}+#: surface -> (checked_at, (health-record version, credential generation), rows).+#: Process-local, like every other short cache in the routes; a restart simply+#: re-sweeps. Keyed on both counters so a new real-call outcome AND any key+#: saved or deleted — through whichever path — drop the sweep at once.+_health_cache: dict[str, tuple[float, tuple[int, int], list[ProviderHealthRow]]] = {}#: Vendor CLI runners. Their credential is a subscription login, not a key,#: so the API-Keys one-token probe is the wrong check: the dual Claude row@@ -608,9 +610,10 @@ async def get_provider_health("""_service(request) # 503 like every other route when the chat is offfrom jarvis.brain.provider_health_ledger import ledger_version+ from jarvis.core.config import secret_generationnow = time.monotonic()- version = ledger_version()+ version = (ledger_version(), secret_generation())cached = _health_cache.get(surface)if (cacheddiff --git a/jarvis/ui/web/provider_routes.py b/jarvis/ui/web/provider_routes.pyindex dc7f474449..d64e1df4f6 100644--- a/jarvis/ui/web/provider_routes.py+++ b/jarvis/ui/web/provider_routes.py@@ -1504,6 +1504,56 @@ def _outcome_detail(label: str, outcome: _health_ledger.Outcome) -> str:return f"{label}: {what} {where} ({age})"+def _credential_slots(provider_id: str) -> list[tuple[str, str | None]]:+ """The keyring slots (with ENV fallback) a provider card reads, in order."""+ chain = list(cfg_mod.PROVIDER_SECRET_CANDIDATES.get(provider_id, ()))+ spec = get_spec(provider_id)+ known = {slot for slot, _env in chain}+ if spec is not None:+ chain.extend((slot, None) for slot in spec.secret_keys if slot not in known)+ return chain+++def _current_credential_fingerprint(spec: ProviderSpec) -> str | None:+ """Fingerprint of the key ``spec`` would use now; ``None`` without one.++ Blocking (keyring reads) — call off the event loop.+ """+ for slot, env in _credential_slots(spec.id):+ try:+ value = cfg_mod.get_secret(slot, env)+ except Exception as exc: # noqa: BLE001 — an unreadable keyring voids nothing+ log.info("health: credential read for %s failed: %s", spec.id, exc)+ return None+ if value:+ return _health_ledger.credential_fingerprint(value)+ return None+++def _providers_reading_slots(slots: tuple[str, ...] | list[str]) -> list[str]:+ """Every provider card that reads any of ``slots`` — its own key slots and+ its resolution chain (a realtime card falls back to the family key)."""+ wanted = {slot for slot in slots if slot}+ return [+ spec.id+ for spec in PROVIDERS+ if wanted & {slot for slot, _env in _credential_slots(spec.id)}+ ]+++def _forget_outcomes_of_changed_slot(slot: str) -> None:+ """Secret-change listener: what the old key did says nothing about the new.++ Registered once with ``jarvis.core.config`` so EVERY in-process write path+ (API-Keys routes, Control API, CLI connect flows, the setup wizard) forgets+ in one place instead of each route remembering to.+ """+ _health_ledger.forget_providers(_providers_reading_slots((slot,)))+++cfg_mod.add_secret_change_listener(_forget_outcomes_of_changed_slot)++def _record_test_verdict(spec: ProviderSpec, status: str, *, model: str | None = None) -> None:"""Feed an explicit test's verdict into the passive health record.@@ -1622,7 +1672,13 @@ async def _tier_section_health(detail=f"{spec.label}: connected or configured",subject_id=spec.id,)- outcome = _health_ledger.effective_outcome(spec.id, modality)+ credential: str | None = None+ if _health_ledger.has_credential_failure(spec.id):+ # A key-level failure is only true of the key it was judged against:+ # one replaced outside this process (the terminal setup wizard, an+ # edited keyring) must void it. Read off the loop, and only then.+ credential = await asyncio.to_thread(_current_credential_fingerprint, spec)+ outcome = _health_ledger.effective_outcome(spec.id, modality, credential=credential)if outcome is None:if getattr(spec, "auth_mode", None) != "api_key":return SectionHealth(@@ -1638,7 +1694,7 @@ async def _tier_section_health(subject_id=spec.id,)return SectionHealth(- status=_section_health.section_status_for_test(outcome.status, configured=True),+ status=_section_health.section_status_for_outcome(outcome.status),reason=outcome.status,detail=_outcome_detail(spec.label, outcome),subject_id=spec.id,@@ -1973,8 +2029,11 @@ def _section_health_fingerprint(("dictation-polish", "1" if _polish_enabled(cfg) else "0"),("dictation-provider", str(getattr(dictation, "polish_provider", None) or "")),("advanced-reachable", repr(reachable)),- # A newly recorded real-call outcome supersedes the cached rollup.+ # A newly recorded real-call outcome supersedes the cached rollup, and+ # so does any credential write/delete — whichever path made it+ # (API-Keys, Control API, CLI connect, setup wizard).("health-ledger", str(_health_ledger.ledger_version())),+ ("secrets", str(cfg_mod.secret_generation())),)return tuple((key, subjects.get(key) or "") for key in _SECTION_HEALTH_KEYS) + configuration@@ -4156,19 +4215,6 @@ def _secret_slot_labels(slots: tuple[str, ...]) -> list[str]:return list(labels)-def _providers_reading_slots(slots: tuple[str, ...] | list[str]) -> list[str]:- """Every provider card that reads any of ``slots`` — its own key slots and- its resolution chain (a realtime card falls back to the family key)."""- wanted = {slot for slot in slots if slot}- found: list[str] = []- for spec in PROVIDERS:- reads = set(spec.secret_keys)- reads.update(slot for slot, _env in cfg_mod.PROVIDER_SECRET_CANDIDATES.get(spec.id, ()))- if reads & wanted:- found.append(spec.id)- return found--@router.post("/secrets/{key}", openapi_extra={"x-jarvis-dangerous": True})async def set_secret_value(key: str, body: SecretBody, request: Request) -> dict[str, Any]:"""Save a key for its whole provider family, or ask which scope was meant.@@ -4207,8 +4253,8 @@ async def set_secret_value(key: str, body: SecretBody, request: Request) -> dictslot,)touched = (*plan.writes, *plan.deletes)- # What the old key did says nothing about the new one.- _health_ledger.forget_providers(_providers_reading_slots(touched))+ # (The health record forgets what the old key did through the+ # secret-change listener set_secret/delete_secret call.)for slot in plan.writes:await _emit(request, SecretConfigured(key=slot, action="set"))for slot in plan.deletes:@@ -4236,7 +4282,6 @@ async def delete_secret_value(key: str, request: Request) -> dict[str, Any]:"credential store may be locked or unavailable."),)- _health_ledger.forget_providers(_providers_reading_slots((key,)))await _emit(request, SecretConfigured(key=key, action="delete"))_invalidate_section_health_state(request)return {"ok": True, "key": key}diff --git a/jarvis/ui/web/server.py b/jarvis/ui/web/server.pyindex b032d8adc9..722e83f6ec 100644--- a/jarvis/ui/web/server.py+++ b/jarvis/ui/web/server.py@@ -2574,6 +2574,13 @@ class WebServer:from jarvis.core import runtime_refs as _runtime_refs_runtime_refs.set_api_base_url(f"http://127.0.0.1:{resolved_port}")+ # Announce each change of the passive provider-health record on the bus;+ # the existing /ws fan-out carries it to every open window, whose status+ # dots then re-read (no poll, no extra socket — AP-33). Registering is+ # cheap: nothing is read or started until the first outcome arrives.+ from jarvis.brain.provider_health_ledger import publish_changes_to++ publish_changes_to(self.bus, asyncio.get_running_loop())if start_serving:assert_bind_safe(host, _control_key.get_control_key())config = uvicorn.Config(diff --git a/jarvis/ui/web/tool_model_routes.py b/jarvis/ui/web/tool_model_routes.pyindex 3906781008..17aba3c4ae 100644--- a/jarvis/ui/web/tool_model_routes.py+++ b/jarvis/ui/web/tool_model_routes.py@@ -246,6 +246,19 @@ async def set_tool_model(manager_cfg.brain.providers[provider] = pcpc.tool_model = model or Nonepc.cu_model = model or None+ # The Tool Model tab reads the passive health record: what the previous+ # model did says nothing about the one just pinned (whether the KEY works+ # still holds), and the cached rollup must show the new selection now.+ from jarvis.brain import provider_health_ledger as health_ledger+ from jarvis.ui.web.provider_routes import _invalidate_section_health_state++ if provider != "auto" and model is not None:+ health_ledger.forget_providers(+ [provider],+ modality=health_ledger.MODALITY_TOOL,+ keep_credential_failures=True,+ )+ _invalidate_section_health_state(request)await _publish_configured(request)status = _status(request)status["persisted"] = body.persistdiff --git a/tests/integration/test_provider_routes.py b/tests/integration/test_provider_routes.pyindex cd2de289d8..46c755d5b6 100644--- a/tests/integration/test_provider_routes.py+++ b/tests/integration/test_provider_routes.py@@ -24,7 +24,11 @@ from jarvis.ui.web.server import WebServerclass _InMemorySecretStore:- """Simulates the keyring via a dict — wired into cfg via monkeypatch."""+ """Simulates the keyring via a dict — wired into cfg via monkeypatch.++ Like the real ``set_secret``/``delete_secret``, a successful write or+ delete announces the changed slot (revision, generation, listeners).+ """def __init__(self) -> None:self.data: dict[str, str] = {}@@ -33,11 +37,17 @@ class _InMemorySecretStore:return self.data.get(key)def set(self, key: str, value: str) -> bool:+ from jarvis.core import config as cfg_mod+self.data[key] = value+ cfg_mod._mark_secret_changed(key)return Truedef delete(self, key: str) -> bool:+ from jarvis.core import config as cfg_mod+self.data.pop(key, None)+ cfg_mod._mark_secret_changed(key)return True@@ -506,6 +516,68 @@ def test_saving_a_key_forgets_what_the_old_key_did(assert record.get("elevenlabs", ledger.MODALITY_TTS) is not None+def test_a_key_saved_through_any_path_forgets_and_refreshes_in_one_place(+ server_with_brain: WebServer,+ secret_store: _InMemorySecretStore,+) -> None:+ """The Control API, CLI connect flows and the setup wizard all save through+ ``set_secret``; none of them calls a health route. The ONE listener on the+ credential store forgets the old verdict and the rollup cache follows."""+ from jarvis.brain import provider_health_ledger as ledger+ from jarvis.core import config as cfg_mod++ secret_store.data["openai_api_key"] = "sk-old"+ ledger.get_ledger().record("openai", ledger.MODALITY_BRAIN, "bad_key")+ with TestClient(server_with_brain.app) as client:+ before = client.get("/api/providers/section-health").json()+ # e.g. PUT /api/control/secrets/openai_api_key → cfg_mod.set_secret+ assert cfg_mod.set_secret("openai_api_key", "sk-new") is True+ after = client.get("/api/providers/section-health").json()++ assert before["sections"]["brain"]["status"] == "error"+ assert after["cached"] is False+ assert after["sections"]["brain"]["status"] == "unknown"+ assert after["sections"]["brain"]["reason"] == "unverified"+++def test_a_key_replaced_in_another_process_voids_the_stale_verdict(+ server_with_brain: WebServer,+ secret_store: _InMemorySecretStore,+) -> None:+ """The terminal wizard writes the keyring from its own process: no listener+ fires here. The credential failure carries the fingerprint of the key it+ was judged against, and a different key voids it on the next read."""+ from jarvis.brain import provider_health_ledger as ledger++ secret_store.data["openai_api_key"] = "sk-old"+ ledger.get_ledger().record("openai", ledger.MODALITY_BRAIN, "bad_key")+ with TestClient(server_with_brain.app) as client:+ before = client.get("/api/providers/section-health").json()+ secret_store.data["openai_api_key"] = "sk-new" # no in-process write+ after = client.get("/api/providers/section-health?refresh=true").json()++ assert before["sections"]["brain"]["status"] == "error"+ assert after["sections"]["brain"]["status"] == "unknown"+ assert ledger.get_ledger().get("openai", ledger.MODALITY_BRAIN) is None+++def test_a_transient_outcome_is_amber_not_red(+ server_with_brain: WebServer,+ secret_store: _InMemorySecretStore,+) -> None:+ """A throttle or a blip has usually passed (or a fallback answered):+ degraded, never the red of a rejected key."""+ from jarvis.brain import provider_health_ledger as ledger++ secret_store.data["openai_api_key"] = "sk-test"+ ledger.get_ledger().record("openai", ledger.MODALITY_BRAIN, "rate_limited")+ with TestClient(server_with_brain.app) as client:+ body = client.get("/api/providers/section-health").json()++ assert body["sections"]["brain"]["status"] == "needs_setup"+ assert body["sections"]["brain"]["reason"] == "rate_limited"++def test_list_providers_exposes_credential_help_and_billing(server_with_brain: WebServer, secret_store: _InMemorySecretStore) -> None:diff --git a/tests/unit/agent_chat/test_agent_chat_service.py b/tests/unit/agent_chat/test_agent_chat_service.pyindex baa250f2d6..89d4d85030 100644--- a/tests/unit/agent_chat/test_agent_chat_service.py+++ b/tests/unit/agent_chat/test_agent_chat_service.py@@ -321,6 +321,37 @@ def test_provider_health_sweep_sends_nothing_to_any_provider(tmp_path: Path, monassert recorder.calls == []+def test_provider_health_sweep_is_dropped_when_any_key_changes(tmp_path: Path, monkeypatch):+ """A key saved or deleted — through the API-Keys page, the Control API, a+ CLI connect flow — must show on the next open, not after five minutes,+ even when the health record had nothing for that provider."""+ import jarvis.ui.web.agent_chat_routes as routes+ from jarvis.core import config as cfg_mod+ from jarvis.ui.web import provider_routes++ present = {"grok": False}+ monkeypatch.setattr(+ provider_routes, "_is_credential_present", lambda spec, *_: present.get(spec.id, True)+ )+ monkeypatch.setattr(+ routes, "_cli_login_snapshot", lambda runner: ("ok", "ok", f"{runner}: signed in")+ )+ monkeypatch.setattr("jarvis.agent_chat.service._claude_cli_installed", lambda: False)+ monkeypatch.setattr(routes, "_health_cache", {})++ with TestClient(_app(tmp_path)) as client:+ first = client.get("/api/agent-chat/provider-health?surface=jarvis").json()+ present["grok"] = True+ cfg_mod._mark_secret_changed("grok_api_key") # what set_secret does after a write+ second = client.get("/api/agent-chat/provider-health?surface=jarvis").json()++ before = {r["provider"]: r for r in first["providers"]}+ after = {r["provider"]: r for r in second["providers"]}+ assert before["grok"]["status"] == "needs_setup"+ assert second["cached"] is False+ assert after["grok"]["status"] == "unknown"++class _CliStatus:def __init__(self, connected: bool, mode: str, message: str = "") -> None:self.connected = connecteddiff --git a/tests/unit/brain/test_provider_health_ledger.py b/tests/unit/brain/test_provider_health_ledger.pyindex 66353e1c9d..d4a253a731 100644--- a/tests/unit/brain/test_provider_health_ledger.py+++ b/tests/unit/brain/test_provider_health_ledger.py@@ -7,7 +7,9 @@ an outcome ages."""from __future__ import annotations+import asyncioimport json+import threadingfrom pathlib import Pathimport pytest@@ -27,13 +29,13 @@ class _Clock:class _CountingLedger(ledger.ProviderHealthLedger):"""Counts disk writes so the 'no write per call' rule is checkable."""- def __init__(self, path: Path, clock: _Clock) -> None:- super().__init__(path, clock=clock)+ def __init__(self, path: Path, clock: _Clock, **kwargs) -> None: # noqa: ANN003+ super().__init__(path, clock=clock, **kwargs)self.writes = 0- def _persist(self, payload): # noqa: ANN001, ANN202+ def _write_file(self, payload): # noqa: ANN001, ANN202self.writes += 1- super()._persist(payload)+ super()._write_file(payload)@pytest.fixture@@ -93,6 +95,37 @@ def test_a_failure_is_classified_like_the_test_button(record, error, expected) -assert record.get("grok", ledger.MODALITY_BRAIN).status == expected+ "error",+ [+ RuntimeError("tool gmail failed: the credit card field is empty"),+ ValueError("budget spreadsheet not found on the network drive"),+ "payment reminder could not be parsed",+ ],+)+def test_marker_words_from_a_tool_or_app_error_are_not_evidence(record, error) -> None:+ """Only the provider's own answer counts: a false no_credits would never+ expire, and these words turn up in tool and app errors all the time."""+ assert record.record_failure("openai", ledger.MODALITY_BRAIN, error) is False+ assert record.get("openai", ledger.MODALITY_BRAIN) is None+++def test_a_provider_sdk_error_without_a_status_code_still_counts(record) -> None:+ class APIConnectionError(Exception): # the name every OpenAI-compatible SDK uses+ pass++ record.record_failure("openai", ledger.MODALITY_BRAIN, APIConnectionError("Connection error."))+ assert record.get("openai", ledger.MODALITY_BRAIN).status == pt.UNREACHABLE+++def test_the_test_button_keeps_the_lenient_classifier(record) -> None:+ """An explicit test is a provider call by construction."""+ record.record_failure(+ "openai", ledger.MODALITY_BRAIN, "credit balance too low", source=ledger.SOURCE_TEST+ )+ assert record.get("openai", ledger.MODALITY_BRAIN).status == pt.NO_CREDITS++def test_no_error_body_or_secret_reaches_the_file(record, tmp_path: Path) -> None:"""AP-34: only the classification survives, never the provider's words."""body = 'Error code: 401 - {"error": "Incorrect API key provided: sk-proj-SECRET123"}'@@ -229,3 +262,141 @@ def test_module_helpers_never_raise_into_the_call_path(monkeypatch) -> None:ledger.set_ledger(_Broken(None))ledger.record_success("openai", ledger.MODALITY_BRAIN)ledger.record_failure("openai", ledger.MODALITY_BRAIN, RuntimeError("x"))+++# ── a key replaced elsewhere ─────────────────────────────────────────────────+++def test_a_credential_failure_is_voided_by_a_key_changed_elsewhere(record) -> None:+ """The terminal setup wizard runs in another process: no listener fires+ here, but the next read with the new key's fingerprint voids the verdict."""+ record.record("openai", ledger.MODALITY_BRAIN, pt.BAD_KEY)+ old = ledger.credential_fingerprint("sk-old")+ new = ledger.credential_fingerprint("sk-new")++ # First read stamps the failure with the key it was judged against.+ assert record.effective("openai", ledger.MODALITY_BRAIN, credential=old).status == pt.BAD_KEY+ assert record.get("openai", ledger.MODALITY_BRAIN).credential == old+ # Same key again: still red.+ assert record.effective("openai", ledger.MODALITY_BRAIN, credential=old).status == pt.BAD_KEY+ # A different key: the stale verdict is gone for good.+ assert record.effective("openai", ledger.MODALITY_BRAIN, credential=new) is None+ assert record.get("openai", ledger.MODALITY_BRAIN) is None+++def test_the_fingerprint_is_not_the_key() -> None:+ fingerprint = ledger.credential_fingerprint("sk-proj-SECRET123")+ assert "SECRET123" not in fingerprint+ assert len(fingerprint) == 16+ assert ledger.credential_fingerprint("") == ""+++# ── listeners ────────────────────────────────────────────────────────────────+++def test_listeners_hear_status_changes_only(record) -> None:+ heard: list[tuple[str, str, str]] = []+ record.add_listener(lambda p, m, s: heard.append((p, m, s)))++ record.record("grok", ledger.MODALITY_BRAIN, pt.OK)+ record.record("grok", ledger.MODALITY_BRAIN, pt.OK) # unchanged: silent+ record.record("grok", ledger.MODALITY_BRAIN, pt.NO_CREDITS)+ record.forget(["grok"])++ assert heard == [+ ("grok", "brain", pt.OK),+ ("grok", "brain", pt.NO_CREDITS),+ ("", "", ""), # bulk change+ ]+++def test_a_failing_listener_never_costs_the_record(record) -> None:+ def _boom(*_args: str) -> None:+ raise RuntimeError("listener down")++ record.add_listener(_boom)+ assert record.record("grok", ledger.MODALITY_BRAIN, pt.OK) is True+++def test_changes_reach_the_bus_from_any_thread(tmp_path: Path) -> None:+ """The web server bridges the record onto the bus; the /ws fan-out then+ carries it to every window. A record made on a voice worker thread must+ arrive too."""+ from jarvis.core.bus import EventBus+ from jarvis.core.events import ProviderHealthChanged++ async def scenario() -> list[ProviderHealthChanged]:+ bus = EventBus()+ seen: list[ProviderHealthChanged] = []++ async def _on(event: ProviderHealthChanged) -> None:+ seen.append(event)++ bus.subscribe(ProviderHealthChanged, _on)+ ledger.set_ledger(ledger.ProviderHealthLedger(tmp_path / "h.json"))+ ledger.publish_changes_to(bus, asyncio.get_running_loop())+ worker = threading.Thread(+ target=ledger.record_failure,+ args=("elevenlabs", ledger.MODALITY_TTS, "HTTP 402 Payment Required"),+ )+ worker.start()+ worker.join()+ for _ in range(100):+ if seen:+ break+ await asyncio.sleep(0.01)+ return seen++ seen = asyncio.run(scenario())++ assert [(e.provider, e.modality, e.status) for e in seen] == [+ ("elevenlabs", "tts", pt.NO_CREDITS)+ ]+++# ── background disk I/O ──────────────────────────────────────────────────────+++def test_background_mode_never_writes_on_the_callers_thread(tmp_path: Path, clock) -> None:+ """AP-9: the voice path records every sentence; the file is written on+ the record's own writer thread, in order, never on the caller's."""+ path = tmp_path / "provider_health.json"+ record = _CountingLedger(path, clock, background_io=True)+ record.start_loading()+ caller = threading.current_thread()+ writers: list[threading.Thread] = []+ original = record._write_file++ def _spy(payload): # noqa: ANN001, ANN202+ writers.append(threading.current_thread())+ original(payload)++ record._write_file = _spy # type: ignore[method-assign]+ record.record("grok", ledger.MODALITY_BRAIN, pt.BAD_KEY)+ record.record("openai", ledger.MODALITY_BRAIN, pt.OK)+ record.flush()++ saved = json.loads(path.read_text(encoding="utf-8"))+ assert {o["provider"] for o in saved["outcomes"]} == {"grok", "openai"}+ assert writers and all(thread is not caller for thread in writers)+++def test_background_load_merges_the_file_without_losing_newer_outcomes(+ tmp_path: Path, clock+) -> None:+ path = tmp_path / "provider_health.json"+ first = ledger.ProviderHealthLedger(path, clock=clock)+ first.record("grok", ledger.MODALITY_BRAIN, pt.NO_CREDITS)+ first.record("elevenlabs", ledger.MODALITY_TTS, pt.OK)++ reborn = ledger.ProviderHealthLedger(path, clock=clock, background_io=True)+ # A real call lands before the file has been read: it must win.+ reborn.record("grok", ledger.MODALITY_BRAIN, pt.OK)+ reborn.start_loading()+ reborn.flush()++ assert reborn.get("grok", ledger.MODALITY_BRAIN).status == pt.OK+ assert reborn.get("elevenlabs", ledger.MODALITY_TTS).status == pt.OK+ saved = json.loads(path.read_text(encoding="utf-8"))+ by_provider = {o["provider"]: o["status"] for o in saved["outcomes"]}+ assert by_provider == {"grok": pt.OK, "elevenlabs": pt.OK}diff --git a/tests/unit/brain/test_section_health.py b/tests/unit/brain/test_section_health.pyindex c707a8bbcf..7ec04211d5 100644--- a/tests/unit/brain/test_section_health.py+++ b/tests/unit/brain/test_section_health.py@@ -70,6 +70,22 @@ class TestAggregate:assert sh.aggregate([sh.UNKNOWN, sh.OK]) == sh.OK+class TestSectionStatusForOutcome:+ """A remembered real-call outcome, unlike a fresh probe, may be minutes+ old: what clears on its own is amber, only what does not is red."""++ @pytest.mark.parametrize("transient", ["rate_limited", "unreachable"])+ def test_transient_outcomes_are_amber(self, transient: str) -> None:+ assert sh.section_status_for_outcome(transient) == sh.NEEDS_SETUP++ @pytest.mark.parametrize("hard", ["bad_key", "no_credits", "model_unavailable", "error"])+ def test_lasting_failures_are_red(self, hard: str) -> None:+ assert sh.section_status_for_outcome(hard) == sh.ERROR++ def test_ok_is_ok(self) -> None:+ assert sh.section_status_for_outcome("ok") == sh.OK++def test_vocabulary_is_exactly_four() -> None:assert set(sh.SECTION_HEALTH_STATUSES) == {sh.OK, sh.NEEDS_SETUP, sh.ERROR, sh.UNKNOWN}assert len(sh.SECTION_HEALTH_STATUSES) == 4diff --git a/tests/unit/brain/test_turn_override.py b/tests/unit/brain/test_turn_override.pyindex ffb050bf3f..6247f1b178 100644--- a/tests/unit/brain/test_turn_override.py+++ b/tests/unit/brain/test_turn_override.py@@ -162,6 +162,53 @@ async def test_a_failing_override_turn_does_not_dead_list_the_provider_for_the_vassert (PICK, PICK_MODEL) not in mgr._dead_provider_models+class _RejectedKeyBrain(FakeBrain):+ """The chat seat's own key is rejected — with a real provider-shaped 401."""++ async def complete(self, req: Any): # type: ignore[no-untyped-def]+ raise RuntimeError("Error code: 401 - invalid x-api-key")+ yield # pragma: no cover — makes this an async generator++[email protected]("seat", ["answers", "rejects"])+async def test_an_override_turn_never_writes_the_brain_tabs_health(seat: str) -> None:+ """A chat runs on its own credential scope and model: its bad key must not+ paint the Brain tab red, and its good key must not erase a real bad_key."""+ from jarvis.brain import provider_health_ledger as ledger++ mgr = _manager()+ ledger.get_ledger().record(PICK, ledger.MODALITY_BRAIN, "bad_key") # the voice's truth+ scoped = FakeBrain(text_response="CHAT") if seat == "answers" else _RejectedKeyBrain()+ mgr._brain_cache[(f"{PICK}@agent", PICK_MODEL)] = scoped+ mgr._brain_cache[(PICK, PICK_MODEL)] = FakeBrain(text_response="unused")++ await mgr.generate(+ "Erzähl mir bitte etwas über die Geschichte von Rom", # i18n-allow: a substantive turn+ use_history=False,+ turn_override=TurnOverride(provider=PICK, model=PICK_MODEL),+ )++ outcome = ledger.get_ledger().get(PICK, ledger.MODALITY_BRAIN)+ assert outcome is not None and outcome.status == "bad_key"+++async def test_a_voice_turn_feeds_the_brain_tabs_health() -> None:+ from jarvis.brain import provider_health_ledger as ledger++ mgr = _manager()+ chain = mgr._build_fallback_chain("fast")+ for provider, model in chain:+ mgr._brain_cache[(provider, model)] = FakeBrain(text_response="VOICE")++ await mgr.generate("hallo", use_history=False)++ lead = chain[0][0]+ outcome = ledger.get_ledger().get(lead, ledger.MODALITY_BRAIN)+ assert outcome is not None and outcome.status == "ok"++@pytest.mark.asyncioasync def test_a_fall_through_to_the_pick_never_announces_a_brain_switch() -> None:"""Review 2026-08-25: the router lead falls through to the pick at chain index 1,diff --git a/tests/unit/core/test_secret_change_listener.py b/tests/unit/core/test_secret_change_listener.pynew file mode 100644index 0000000000..b98134f9b9--- /dev/null+++ b/tests/unit/core/test_secret_change_listener.py@@ -0,0 +1,55 @@+"""Every credential write or delete is observable in ONE place.++The API-Keys routes, the Control API, CLI connect flows and the setup wizard+all save through ``set_secret`` / ``delete_secret``. Whatever must follow a+key change (the provider-health record forgetting what the old key did, caches+keyed on "which keys exist") hangs off this one hook instead of each route.+"""+from __future__ import annotations++from jarvis.core import config as cfg_mod+++def test_a_changed_slot_reaches_every_listener_and_bumps_the_generation() -> None:+ heard: list[str] = []++ def _listener(slot: str) -> None:+ heard.append(slot)++ cfg_mod.add_secret_change_listener(_listener)+ cfg_mod.add_secret_change_listener(_listener) # idempotent+ before = cfg_mod.secret_generation()+ try:+ cfg_mod._mark_secret_changed("groq_api_key")+ finally:+ cfg_mod._SECRET_CHANGE_LISTENERS.remove(_listener)++ assert heard == ["groq_api_key"]+ assert cfg_mod.secret_generation() == before + 1+ assert cfg_mod.secret_revision("groq_api_key") >= 1+++def test_a_failing_listener_never_fails_the_save() -> None:+ def _boom(slot: str) -> None:+ raise RuntimeError("follower down")++ cfg_mod.add_secret_change_listener(_boom)+ try:+ cfg_mod._mark_secret_changed("groq_api_key") # must not raise+ finally:+ cfg_mod._SECRET_CHANGE_LISTENERS.remove(_boom)+++def test_the_health_record_forgets_what_the_old_key_did() -> None:+ """The web layer's listener maps the slot to every card that reads it."""+ from jarvis.brain import provider_health_ledger as ledger+ from jarvis.ui.web import provider_routes # noqa: F401 — registers the listener++ record = ledger.get_ledger()+ record.record("groq-api", ledger.MODALITY_STT, "bad_key")+ record.record("elevenlabs", ledger.MODALITY_TTS, "no_credits")++ cfg_mod._mark_secret_changed("groq_api_key")++ assert record.get("groq-api", ledger.MODALITY_STT) is None+ assert record.get("elevenlabs", ledger.MODALITY_TTS) is not Nonediff --git a/tests/unit/speech/test_pipeline_lightweight_wake.py b/tests/unit/speech/test_pipeline_lightweight_wake.pyindex be336e0726..571c217753 100644--- a/tests/unit/speech/test_pipeline_lightweight_wake.py+++ b/tests/unit/speech/test_pipeline_lightweight_wake.py@@ -59,8 +59,11 @@ def test_lightweight_mode_instantiates_no_faster_whisper() -> None:# it, by the runtime fallback chain; the resolved provider must still be the# configured Groq cloud STT, and the label must still NAME it rather than# naming a wrapper — the log line that reports which provider transcribed is- # the first thing anyone reads when a provider starts failing.- assert type(pipe._utterance_stt).__name__ == "DictionaryCorrectingSTT"+ # the first thing anyone reads when a provider starts failing. Outermost is+ # the speech meter, present with or without a bus: it feeds the+ # provider-health record behind the status dots.+ assert type(pipe._utterance_stt).__name__ == "MeteredSTT"+ assert type(pipe._utterance_stt._inner).__name__ == "DictionaryCorrectingSTT"assert pipe._utterance_stt.provider_label == "groq-api"diff --git a/tests/unit/speech/test_stt_provider_live_switch.py b/tests/unit/speech/test_stt_provider_live_switch.pyindex 05685eabeb..fd8b76ae44 100644--- a/tests/unit/speech/test_stt_provider_live_switch.py+++ b/tests/unit/speech/test_stt_provider_live_switch.py@@ -61,7 +61,11 @@ def test_pipeline_switch_replaces_voice_and_next_dictation_provider(assert pipeline._utterance_stt.name == "openrouter-stt"assert pipeline._utterance_stt is not old_provider- assert pipeline._probe_stt is pipeline._utterance_stt+ # The utterance lane is wrapped by the speech meter (usage + provider+ # health, with or without a bus); the preview follows the same provider.+ assert pipeline._probe_stt is getattr(+ pipeline._utterance_stt, "_inner", pipeline._utterance_stt+ )assert pipeline._config.stt.provider == "openrouter-stt"assert pipeline._dictation_stt_instance is Noneassert pipeline._voice_stt_fallback_chain is Nonediff --git a/tests/unit/speech/test_usage_meter.py b/tests/unit/speech/test_usage_meter.pyindex b5b868f019..6ea8592afb 100644--- a/tests/unit/speech/test_usage_meter.py+++ b/tests/unit/speech/test_usage_meter.py@@ -178,12 +178,18 @@ async def test_a_failing_sink_never_breaks_synthesis() -> None:@pytest.mark.asyncio-async def test_no_sink_returns_the_provider_untouched() -> None:- """Zero cost when nobody is listening — not even a wrapper object."""+async def test_no_sink_still_wraps_for_health_but_reports_no_usage() -> None:+ """Without a cost sink (no bus) there is no spend to publish, but the+ provider-health record behind the status dots still needs every real+ call — so the provider is wrapped and speech is unchanged."""provider = _FakeTTS()- assert meter_tts(provider, None) is provider- stt = _FakeSTT()- assert meter_stt(stt, None) is stt+ tts = meter_tts(provider, None)+ assert tts is not provider+ assert meter_tts(tts, None) is tts # never double-wrapped+ chunks = [c async for c in tts.synthesize("hallo")]+ assert len(chunks) == 3+ stt = meter_stt(_FakeSTT(), None)+ assert await stt.transcribe(_audio(2)) == "hello"# ---------------------------------------------------------------------------diff --git a/tests/unit/speech/test_usage_meter_health.py b/tests/unit/speech/test_usage_meter_health.pyindex b6eb001a79..4d82b0a112 100644--- a/tests/unit/speech/test_usage_meter_health.py+++ b/tests/unit/speech/test_usage_meter_health.py@@ -113,6 +113,38 @@ def test_an_absorbed_failure_is_charged_to_the_voice_that_was_asked() -> None:assert _status("gemini-flash-tts", ledger.MODALITY_TTS) == pt.OK+class _SilentTTS(_TTS):+ """A keyed plugin whose own error was swallowed and no fallback existed:+ it sets ``last_failure`` and yields nothing at all."""++ async def synthesize(+ self, text: str, voice: str | None = None, language_code: str | None = None+ ) -> AsyncIterator[AudioChunk]:+ self.last_voice_provider = self.name+ self.last_failure = "_InworldFatalError: HTTP 401"+ return+ yield # pragma: no cover — makes this an async generator+++def test_a_swallowed_failure_with_no_audio_at_all_is_still_recorded() -> None:+ """No fallback, no audio, no exception: the user heard silence, and the+ dot must say why."""+ produced = asyncio.run(_speak(meter_tts(_SilentTTS("inworld"), _Sink())))++ assert produced == 0+ assert _status("inworld", ledger.MODALITY_TTS) == pt.BAD_KEY+++def test_a_pipeline_without_a_cost_sink_still_feeds_the_dots() -> None:+ """No bus → no spend sink; the health record must not depend on it."""+ asyncio.run(_speak(meter_tts(_TTS("cartesia"), None)))+ stt = meter_stt(_STT("openai-api"), None)+ asyncio.run(stt.transcribe_pcm(_PCM, 16_000))++ assert _status("cartesia", ledger.MODALITY_TTS) == pt.OK+ assert _status("openai-api", ledger.MODALITY_STT) == pt.OK++class _SlowTTS(_TTS):async def synthesize(self, text: str, voice: str | None = None, language_code: str | None = Nonediff --git a/tests/unit/web/test_tool_model_routes.py b/tests/unit/web/test_tool_model_routes.pyindex 9e178d0f9b..9a79f1cadc 100644--- a/tests/unit/web/test_tool_model_routes.py+++ b/tests/unit/web/test_tool_model_routes.py@@ -95,6 +95,28 @@ def test_put_persists_canonical_selection_and_updates_live(assert response.json()["persisted"] is True+def test_a_new_tool_model_starts_with_a_clean_health_record(monkeypatch) -> None:+ """The Tool Model dot reads the passive record: the old model's 404 says+ nothing about the new pin, but a rejected KEY still does."""+ from jarvis.brain import provider_health_ledger as ledger++ app, _brain = _app()+ monkeypatch.setattr(tool_model_routes, "is_credential_present", lambda _s: True)+ record = ledger.get_ledger()+ record.record("gemini", ledger.MODALITY_TOOL, "model_unavailable")+ record.record("openai", ledger.MODALITY_TOOL, "bad_key")++ for provider in ("gemini", "openai"):+ response = TestClient(app).put(+ "/api/tool-model",+ json={"provider": provider, "model": f"{provider}-tool", "persist": False},+ )+ assert response.status_code == 200++ assert record.get("gemini", ledger.MODALITY_TOOL) is None+ assert record.get("openai", ledger.MODALITY_TOOL).status == "bad_key"++def test_put_reactivates_a_provider_before_its_runtime_probe(monkeypatch) -> None:app, brain = _app()monkeypatch.setattr(tool_model_routes, "is_credential_present", lambda _s: True)