Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

fix(local-models): a provider switch is what starts AND stops the local server

rubenluetke10-beep9be19299bbcf
diff --git a/docs/BUGS.md b/docs/BUGS.md
index c133713e29..98299dff2f 100644
--- a/docs/BUGS.md
+++ b/docs/BUGS.md
@@ -14351,3 +14351,79 @@ it: three surfaces that can each flip a value must never each own it. See
`docs/os-parity.md` P-40.
**Related.** BUG-201 (the v6 defect: the example became the job).
+
+## BUG-204: switching the brain to a hosted provider left the local models running — 12 GB of graphics memory held by a voice model nothing was using (HIGH, FIXED 2026-08-28)
+
+**Symptom (maintainer, 2026-08-28).** "I switched the local models on, then off
+again to get the provider back, and I don't know why they still work in the
+background." The app ran fine for most of an afternoon and then froze — the
+window, the Jarvis bar, everything — for stretches of 15 to 41 seconds.
+
+**What the box showed while it was happening.** 14.9 of 16.3 GB of graphics
+memory in use, held by three model servers, with a hosted realtime provider
+(`gemini-live`) doing the actual talking:
+
+| holder | memory |
+| --- | --- |
+| the local realtime speech stack (Qwen3-TTS) | 5.4 GB |
+| `ornith:9b-voice-32k` on Jarvis' own Ollama | 3.0 GB (`UNTIL: 2 hours from now`) |
+| a vision model on a SECOND Ollama | 3.3 GB |
+
+Free RAM was 5.9 of 31.7 GB with 81 GB committed, so the machine was paging.
+
+**Three causes, none of them the local models "misbehaving".**
+
+1. **`in_use()` asked whether a role was CONFIGURED, not whether anything was
+ SELECTED.** `jarvis/local_models/autostart.py` counted any writable role
+ holding a model tag as "local models are in use". The Ollama card keeps its
+ picks when the user switches away — that is what a card is for — so
+ `[brain.providers.ollama] model = "deepseek-r1:14b"` made the answer "in
+ use" permanently. The boot task started the server every launch and warmed
+ the chat pick with a 30-minute keep-alive, whatever the active brain was.
+2. **A switch away from the local server did nothing.** `_ready_local_server`
+ in `jarvis/brain/app_control.py` fired `autostart.kick` on a switch TO the
+ local server and returned early on every other provider. `unload_model`
+ existed but only the Unload button and an assistant tool ever called it, so
+ picking a hosted provider changed who answers and freed nothing.
+3. **`[voice] local_idle_release_minutes` was `0`** — documented as "keep
+ everything loaded while Jarvis runs". The supervisor's idle release, the one
+ mechanism that would have caught the other two, was switched off.
+
+**Two amplifiers found in the same pass.** A second `ollama serve` (from the
+Ollama tray app) was bound to `0.0.0.0:11434` and `[::]:11434` while Jarvis'
+own server held `127.0.0.1:11434`. Windows accepts both binds, so there was no
+error and no clue: a client resolving `localhost` to `::1` reached the tray
+server, one using `127.0.0.1` reached Jarvis'. `ollama ps` listed one model
+while the card held two, and the older "ollama-inventory: localhost did not
+answer" lines belong to the same split. Separately, the app inherits
+`BelowNormal` priority from the autostart task, which is what turned a paging
+machine into 29-second freezes (three of the seven stalls that afternoon sat in
+`GetQueuedCompletionStatus` — the loop was idle and simply not scheduled).
+
+**Fix.**
+
+1. `in_use()` counts an ACTIVE choice only: the active brain is the local
+ server, or the active realtime provider is a local card (spec-derived
+ through `LOCAL_PROVIDERS`, `auth_mode == "none"` — AP-21; a fallback entry
+ does not count). A stored pick is configuration, not consent. A role that
+ needs the server while nothing selects it starts it on demand instead of
+ holding it warm all day.
+2. `autostart.release()` / `release_once()` — the mirror of `kick()`: unload
+ every model `/api/ps` reports, then stop the server via
+ `ollama_runtime.stop_server()`, which only ever touches the pid Jarvis
+ recorded. It refuses while `in_use` still holds (a local voice on a hosted
+ brain still needs the server) and never stops a remote one.
+3. `_ready_local_server` became `_sync_local_server`: `kick` on a switch to the
+ local server, `release` on a switch away. Every switch path — REST, voice
+ gate, CLI, brain tool — already lands there.
+
+`[voice] local_idle_release_minutes` was set back to 15 on the maintainer's
+machine; the supervisor reads it fresh on every monitor tick, so that half took
+effect without a restart.
+
+**Not changed.** `[brain.realtime] fallback_provider = "local-realtime"` stays:
+it is a deliberate setting, and the idle release now returns what it borrows.
+An install that wants no local voice at all clears the fallback.
+
+**Related.** BUG-174 (the other local-models defect whose symptom was "voice on
+local models is unusably slow after a restart" — same stack, different half).
diff --git a/jarvis/brain/app_control.py b/jarvis/brain/app_control.py
index 15907f1de7..01120d6e00 100644
--- a/jarvis/brain/app_control.py
+++ b/jarvis/brain/app_control.py
@@ -619,7 +619,7 @@ async def _switch_brain(
persisted = _persist_brain_primary(provider)
_set_in_memory(cfg, ["brain", "primary"], provider)
- _ready_local_server(provider, cfg)
+ _sync_local_server(provider, cfg)
return {
"ok": True,
"tier": "brain",
@@ -631,22 +631,37 @@ async def _switch_brain(
}
-def _ready_local_server(provider: str, cfg: Any) -> None:
- """After a switch to the local model server: start it and warm the chat pick.
+def _sync_local_server(provider: str, cfg: Any) -> None:
+ """Bring the local model server in line with the brain that was just picked.
- Fire-and-forget through :func:`jarvis.local_models.autostart.kick`, so
- "choose local models" is followed by local models answering — not by a
- two-second connect failure and the fallback on the first turn. Every
- switch path (REST, voice gate, CLI, brain tool) lands here.
+ Both directions, fire-and-forget, because a provider switch is the whole
+ gesture a user gets:
+
+ * TO the local server — :func:`jarvis.local_models.autostart.kick`, so
+ "choose local models" is followed by local models answering, not by a
+ two-second connect failure and the fallback on the first turn.
+ * AWAY from it — :func:`jarvis.local_models.autostart.release`, so the
+ graphics memory comes back. Without this half the switch only changed
+ who answers: the model stayed resident behind its keep-alive and a
+ 6 GB card kept holding it for as long as Jarvis ran (BUG-204). The
+ release refuses while a local voice still runs on the server, and it
+ only ever stops the server this install started.
+
+ Every switch path (REST, voice gate, CLI, brain tool) lands here.
"""
try:
- from jarvis.local_models.autostart import SERVER_PROVIDER_ID, kick # lazy (AP-26)
+ from jarvis.local_models.autostart import ( # lazy (AP-26)
+ SERVER_PROVIDER_ID,
+ kick,
+ release,
+ )
- if provider != SERVER_PROVIDER_ID:
- return
- kick(cfg)
- except Exception: # noqa: BLE001 — the switch already landed; only the first answer is slower
- log.debug("local server ready-up after the switch was not scheduled", exc_info=True)
+ if provider == SERVER_PROVIDER_ID:
+ kick(cfg)
+ else:
+ release(cfg)
+ except Exception: # noqa: BLE001 — the switch already landed; only the server state lags
+ log.debug("local server sync after the switch was not scheduled", exc_info=True)
def _switch_tts(provider: str, *, cfg: Any, persist: bool, old: str | None) -> dict[str, Any]:
diff --git a/jarvis/local_models/autostart.py b/jarvis/local_models/autostart.py
index 6d78279080..f5defef96f 100644
--- a/jarvis/local_models/autostart.py
+++ b/jarvis/local_models/autostart.py
@@ -7,10 +7,9 @@ opened the Server tab. This module is the ONE place that decides whether
the local server comes up with Jarvis, and does it:
* :func:`should_autostart` — ``[brain.providers.ollama].autostart`` is on
- (the default) AND local models are actually in use: the active brain is
- the local server, or a role (chat, voice, tools & screen, deep, embedding)
- has a configured pick. An install that never chose local models starts
- nothing and spawns nothing.
+ (the default) AND local models are actually SELECTED: the active brain is
+ the local server, or the active realtime provider is a local one. An
+ install that never chose local models starts nothing and spawns nothing.
* :func:`run_once` — start an installed-but-stopped local server (never
install it: that is a click behind the dangerous-flagged route), then load
the chat pick with its keep-alive so the first answer does not pay the
@@ -20,6 +19,9 @@ the local server comes up with Jarvis, and does it:
* :func:`kick` — the same run, fire-and-forget, right after the brain was
switched to the local server, so "choose local models" means "local
models answer" without a second click.
+* :func:`release` — its mirror, right after the brain was switched AWAY:
+ unload what is resident and stop the server this install started, so
+ "choose a hosted provider" also means "give the accelerator back".
"""
from __future__ import annotations
@@ -37,6 +39,8 @@ __all__ = [
"SERVER_PROVIDER_ID",
"in_use",
"kick",
+ "release",
+ "release_once",
"run_once",
"schedule",
"should_autostart",
@@ -61,6 +65,9 @@ StatusFn = Callable[[], dict[str, object]]
StartFn = Callable[[], tuple[bool, str]]
WarmFn = Callable[[str, str, str | int], Awaitable[bool]]
RunFn = Callable[[Any], Awaitable[dict[str, Any]]]
+StopFn = Callable[[], tuple[bool, str]]
+UnloadFn = Callable[[str], Awaitable[list[str]]]
+ReleaseFn = Callable[[Any], Awaitable[dict[str, Any]]]
#: Live task references — a fire-and-forget task must be held somewhere or
#: the loop may drop it mid-run.
@@ -72,32 +79,50 @@ def _provider(cfg: Any) -> Any:
return providers.get(SERVER_PROVIDER_ID) if isinstance(providers, dict) else None
-def in_use(cfg: Any) -> tuple[bool, str]:
- """``(used, why)`` — whether anything in this install runs on the local server.
+def _active_local_realtime(cfg: Any) -> str:
+ """The active realtime provider id when it is a LOCAL card, else ``""``.
- True when the active brain is the local server, or when any writable
- role has a configured pick (the voice server's brain, the wiki's
- embedder and the deep model are local-model interfaces even while a
- cloud brain answers the chat).
+ A local realtime card runs its speech stack on this machine and takes
+ its answers from the local model server, so that server belongs up
+ while it is the SELECTED voice — a fallback entry does not count, only
+ what is live. Spec-derived through ``LOCAL_PROVIDERS``
+ (``auth_mode == "none"``), never a hand-written name list (AP-21).
"""
- from jarvis.brain.ollama_roles import WRITABLE_ROLE_IDS, current_pick # lazy (AP-26)
+ realtime = getattr(getattr(cfg, "brain", None), "realtime", None)
+ provider = str(getattr(realtime, "provider", "") or "").strip()
+ if not provider:
+ return ""
+ try:
+ from jarvis.brain.app_control import LOCAL_PROVIDERS # lazy (AP-26)
+ except Exception: # noqa: BLE001 — no spec table: fall back to "not a local voice"
+ log.debug("local-models: provider spec unreadable for the realtime check", exc_info=True)
+ return ""
+ return provider if provider in LOCAL_PROVIDERS else ""
+
+def in_use(cfg: Any) -> tuple[bool, str]:
+ """``(used, why)`` — whether an ACTIVE choice runs on the local server.
+
+ Selected, not merely configured. Two things count: the active brain is
+ the local server, or the active realtime provider is a local card whose
+ speech stack answers from this machine's model server.
+
+ A writable role that merely HOLDS a model name does NOT count, and that
+ is the whole point (BUG-204). The Ollama card keeps its picks while a
+ hosted provider answers — switching the brain to a cloud card left
+ ``chat`` pointing at a local tag, this returned "in use" forever, and a
+ multi-GB model stayed resident on the accelerator behind its keep-alive
+ for as long as Jarvis ran. Picking a provider is the switch; a stored
+ name is not. A role that needs the server while nothing selects it
+ starts it on demand instead of holding it warm all day.
+ """
primary = str(getattr(getattr(cfg, "brain", None), "primary", "") or "").strip()
if primary == SERVER_PROVIDER_ID:
return True, "the active brain"
- roles: list[str] = []
- for role in WRITABLE_ROLE_IDS:
- try:
- model, _note = current_pick(cfg, role)
- except Exception: # noqa: BLE001 — an unreadable pick is "not in use" for this decision
- log.debug("local-models autostart: pick of %s unreadable", role, exc_info=True)
- continue
- if model:
- roles.append(role)
- if roles:
- noun = "role" if len(roles) == 1 else "roles"
- return True, f"the {', '.join(roles)} {noun}"
- return False, "no role uses local models and the active brain is elsewhere"
+ realtime = _active_local_realtime(cfg)
+ if realtime:
+ return True, f"the active realtime voice ({realtime})"
+ return False, "neither the active brain nor the active voice runs on local models"
def should_autostart(cfg: Any) -> tuple[bool, str]:
@@ -135,6 +160,26 @@ def _keep_alive(cfg: Any, model: str) -> str | int:
return keep_alive if keep_alive is not None else DEFAULT_KEEP_ALIVE
+async def _default_unload(root: str) -> list[str]:
+ """Unload every model resident on ``root``; returns the names freed.
+
+ Aliases occupy their own memory, so ``/api/ps`` is the list that matters
+ rather than the configured picks. One failed unload does not stop the
+ others — the stop that follows frees the rest anyway.
+ """
+ from jarvis.brain import ollama_inventory # lazy (AP-26)
+
+ freed: list[str] = []
+ for running in await ollama_inventory.running_models(root):
+ try:
+ await ollama_inventory.unload_model(root, running.name)
+ except Exception as exc: # noqa: BLE001 — one stubborn model must not keep the rest resident
+ log.debug("local-models: unloading %s failed (%s)", running.name, exc, exc_info=True)
+ continue
+ freed.append(running.name)
+ return freed
+
+
async def _default_warm(root: str, model: str, keep_alive: str | int) -> bool:
from jarvis.brain.ollama_profiles import warm # lazy (AP-26)
@@ -247,3 +292,71 @@ def kick(cfg: Any, *, run: RunFn | None = None) -> asyncio.Task[None] | None:
log.warning("local-models: readying the server after the switch failed", exc_info=True)
return _remember(loop.create_task(_now(), name="local-models-kick"))
+
+
+async def release_once(
+ cfg: Any,
+ *,
+ unload: UnloadFn | None = None,
+ stop: StopFn | None = None,
+) -> dict[str, Any]:
+ """Hand the accelerator back: unload what is resident, stop our server.
+
+ Returns ``{"stopped", "unloaded", "detail"}`` — ``unloaded`` the models
+ freed, ``stopped`` whether the server this install spawned was stopped,
+ ``detail`` one sentence. Refuses while :func:`in_use` still holds (a
+ local voice may keep answering on a hosted brain), and the runtime only
+ ever stops the pid Jarvis recorded, so a server the user started from
+ the tray or a terminal is left alone. Never raises.
+ """
+ from jarvis.brain import ollama_runtime # lazy (AP-26)
+ from jarvis.local_models.health_monitor import server_root # lazy (AP-26)
+
+ record: dict[str, Any] = {"stopped": False, "unloaded": [], "detail": ""}
+ used, why = in_use(cfg)
+ if used:
+ record["detail"] = f"kept running — {why} still uses it"
+ return record
+ root = server_root(cfg)
+ if ollama_runtime.host_kind(root) == "remote":
+ record["detail"] = f"The server at {root} is remote; this install does not stop it."
+ return record
+ try:
+ record["unloaded"] = await (unload or _default_unload)(root)
+ except Exception as exc: # noqa: BLE001 — a server already down needs no unload
+ log.debug("local-models: unload before stop skipped (%s)", exc, exc_info=True)
+ ok, detail = await asyncio.to_thread(stop or ollama_runtime.stop_server)
+ record["stopped"] = ok
+ record["detail"] = detail
+ return record
+
+
+def release(cfg: Any, *, run: ReleaseFn | None = None) -> asyncio.Task[None] | None:
+ """After a switch AWAY from the local server: free it, fire-and-forget.
+
+ The mirror of :func:`kick`, and the reason picking a hosted provider is
+ enough to get the graphics memory back — before this, a switch away
+ started nothing and stopped nothing, so the model stayed resident
+ behind its keep-alive until Jarvis quit (BUG-204). ``None`` outside a
+ running loop, exactly like :func:`kick`.
+ """
+ try:
+ loop = asyncio.get_running_loop()
+ except RuntimeError:
+ return None
+
+ async def _now() -> None:
+ try:
+ record = await (run or release_once)(cfg)
+ log.info(
+ "local-models: server released after the brain switch — stopped=%s unloaded=%r %s",
+ record.get("stopped"),
+ record.get("unloaded"),
+ record.get("detail") or "",
+ )
+ except asyncio.CancelledError:
+ raise
+ except Exception: # noqa: BLE001 — the switch already landed; only the memory stays held
+ log.warning("local-models: releasing the server after the switch failed", exc_info=True)
+
+ return _remember(loop.create_task(_now(), name="local-models-release"))
diff --git a/tests/unit/local_models/test_autostart.py b/tests/unit/local_models/test_autostart.py
index affb2158cb..88e1ce7939 100644
--- a/tests/unit/local_models/test_autostart.py
+++ b/tests/unit/local_models/test_autostart.py
@@ -13,7 +13,12 @@ from typing import Any
import pytest
-from jarvis.core.config import BrainProviderConfig, JarvisConfig, OllamaModelOptions
+from jarvis.core.config import (
+ BrainProviderConfig,
+ BrainTierConfig,
+ JarvisConfig,
+ OllamaModelOptions,
+)
from jarvis.local_models import autostart
@@ -22,9 +27,11 @@ def _cfg(
primary: str = "openrouter",
chat: str = "",
flag: bool | None = None,
+ realtime: str = "gemini-live",
) -> JarvisConfig:
cfg = JarvisConfig()
cfg.brain.primary = primary
+ cfg.brain.realtime = BrainTierConfig(provider=realtime)
provider = BrainProviderConfig(model=chat)
if flag is not None:
provider.autostart = flag
@@ -35,15 +42,31 @@ def _cfg(
def test_nothing_is_in_use_on_a_fresh_install() -> None:
used, why = autostart.in_use(_cfg())
assert used is False
- assert "no role" in why
+ assert "neither the active brain nor the active voice" in why
start, reason = autostart.should_autostart(_cfg())
assert start is False and reason == why
-def test_the_active_brain_or_a_configured_role_counts_as_in_use() -> None:
+def test_only_an_active_choice_counts_as_in_use() -> None:
+ """A stored pick is configuration; the accelerator answers to selection.
+
+ What would lie if it broke: the Ollama card keeping a multi-GB model
+ resident after the user switched the brain to a hosted provider, because
+ the card still holds the tag they last chose (BUG-204).
+ """
assert autostart.in_use(_cfg(primary="ollama")) == (True, "the active brain")
used, why = autostart.in_use(_cfg(chat="qwen3.5:4b"))
- assert used and why == "the chat role"
+ assert used is False, why
+
+
+def test_an_active_local_voice_keeps_the_server_in_use() -> None:
+ """The local realtime card answers from this machine's model server."""
+ used, why = autostart.in_use(_cfg(realtime="local-realtime"))
+ assert used and "local-realtime" in why
+ # A hosted voice does not, even with the local card configured as a fallback.
+ cfg = _cfg(realtime="gemini-live")
+ cfg.brain.realtime.fallback_provider = "local-realtime"
+ assert autostart.in_use(cfg)[0] is False
def test_the_switch_off_wins_over_use() -> None:
@@ -183,12 +206,68 @@ def test_kick_is_a_no_op_outside_a_loop() -> None:
@pytest.mark.asyncio
-async def test_the_brain_switch_readies_the_local_server(monkeypatch: pytest.MonkeyPatch) -> None:
+async def test_the_brain_switch_readies_and_releases_the_local_server(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Both halves of the gesture: pick it and it starts, leave and it stops."""
from jarvis.brain import app_control
kicked: list[str] = []
+ released: list[str] = []
monkeypatch.setattr(autostart, "kick", lambda cfg, **_kw: kicked.append(cfg.brain.primary))
- app_control._ready_local_server("ollama", _cfg(primary="ollama"))
- app_control._ready_local_server("openrouter", _cfg())
+ monkeypatch.setattr(autostart, "release", lambda cfg, **_kw: released.append(cfg.brain.primary))
+ app_control._sync_local_server("ollama", _cfg(primary="ollama"))
+ app_control._sync_local_server("openrouter", _cfg())
assert kicked == ["ollama"]
+ assert released == ["openrouter"]
await asyncio.sleep(0)
+
+
+async def test_release_once_unloads_and_stops_when_nothing_uses_the_server() -> None:
+ stopped: list[str] = []
+
+ async def _unload(root: str) -> list[str]:
+ assert root
+ return ["ornith:9b-voice-32k", "qwen3.5:4b-voice-8k"]
+
+ record = await autostart.release_once(
+ _cfg(chat="qwen3.5:4b"),
+ unload=_unload,
+ stop=lambda: (stopped.append("go") or True, "Ollama stopped."),
+ )
+ assert stopped == ["go"]
+ assert record["stopped"] is True
+ assert record["unloaded"] == ["ornith:9b-voice-32k", "qwen3.5:4b-voice-8k"]
+
+
+async def test_release_once_keeps_a_server_something_still_uses() -> None:
+ """A local voice on a hosted brain still needs the model server."""
+
+ async def _unload(_root: str) -> list[str]:
+ raise AssertionError("must not unload")
+
+ record = await autostart.release_once(
+ _cfg(realtime="local-realtime"),
+ unload=_unload,
+ stop=lambda: (_ for _ in ()).throw(AssertionError("must not stop")),
+ )
+ assert record["stopped"] is False and "still uses it" in record["detail"]
+
+
+async def test_release_once_never_stops_a_remote_server() -> None:
+ cfg = _cfg()
+ cfg.brain.providers["ollama"].base_url = "http://box.lan:11434"
+ record = await autostart.release_once(
+ cfg,
+ unload=lambda _root: (_ for _ in ()).throw(AssertionError("must not unload")),
+ stop=lambda: (_ for _ in ()).throw(AssertionError("must not stop")),
+ )
+ assert record["stopped"] is False and "remote" in record["detail"]
+
+
+def test_release_is_a_no_op_outside_a_loop() -> None:
+ assert autostart.release(_cfg()) is None
+ assert not any(not t.done() for t in autostart._tasks)
diff --git a/tests/unit/ui/web/test_local_models_routes.py b/tests/unit/ui/web/test_local_models_routes.py
index 6383959701..916742e3d6 100644
--- a/tests/unit/ui/web/test_local_models_routes.py
+++ b/tests/unit/ui/web/test_local_models_routes.py
@@ -611,9 +611,13 @@ def test_autostart_round_trip(server: WebServer, fake, monkeypatch) -> None:
)
with TestClient(server.app) as client:
before = client.get(f"{BASE}/runtime/autostart").json()
- # The default is on, and the fixture's chat role puts the server in use.
- assert before == {"enabled": True, "in_use": True, "reason": before["reason"]}
- assert "chat" in before["reason"]
+ # The default is on, but the fixture's stored chat pick is NOT use:
+ # a card keeps its picks while a hosted provider answers (BUG-204).
+ assert before["enabled"] is True and before["in_use"] is False
+ # Selecting the local server as the brain is what puts it in use.
+ server.app.state.config.brain.primary = "ollama"
+ picked = client.get(f"{BASE}/runtime/autostart").json()
+ assert picked["in_use"] is True and "active brain" in picked["reason"]
off = client.put(f"{BASE}/runtime/autostart", json={"enabled": False}).json()
assert off["enabled"] is False and "switched off" in off["reason"]
assert client.get(f"{BASE}/runtime/autostart").json()["enabled"] is False