Make your computer a home for your AI partner. Avibe is the local-first Agent OS: Vibey, its built-in AI partner, works out of the box, and Claude Code, Codex & OpenCode run unchanged beside it — reach them from your browser, phone, or any chat app.
Python agentagent-osaiai-agentsai-partnerchatopsclaudeclaude-codecodexdevtoolsdiscord-botlark-botllmlocal-firstlong-term-memoryopencodeslack-botvibe-codingvibeywechat
fix(model-hub): hot-reload source saves instead of restarting the engine (#2138)
Call Whiteac979138f0fe
diff --git a/core/handlers/model_hub/service.py b/core/handlers/model_hub/service.pyindex 6fb5947dd..c131964f7 100644--- a/core/handlers/model_hub/service.py+++ b/core/handlers/model_hub/service.py@@ -1340,7 +1340,7 @@ class ModelHubService:updated_bindings = self._bindings(updated)# SourceBinding order is the engine-config serialization order. Agent# Route target order is not part of this projection, so a pure chain reorder- # compares equal and does not restart a healthy engine.+ # compares equal and does not touch the engine.if previous_bindings == updated_bindings:self._save_config(updated)returndiff --git a/docs/plans/model-hub-routing-modes.md b/docs/plans/model-hub-routing-modes.mdindex f268c4354..02dd3e15a 100644--- a/docs/plans/model-hub-routing-modes.md+++ b/docs/plans/model-hub-routing-modes.md@@ -369,23 +369,43 @@ transport is authorized by this amendment.### Registration Synchronization Decision-Pinned-engine experiments proved file/atomic/management configuration hot reload-with unchanged PID and uninterrupted existing streams for all three protocols.-However, neither the management acknowledgment nor `/v1/models` provides an-atomic scheduler/config-generation completion barrier. The feature therefore-retains the existing supervised restart transaction with explicit in-flight-protection instead of introducing an unproven readiness heuristic.--The adapter owns invocation leases and drains existing transport operations-before a configuration-triggered engine restart. New invocations wait behind the-same configuration barrier. Lease release belongs to transport completion,-close, or cancellation, independently of service settlement and its mutation-lock. A synchronization cancellation or exception cannot leak a barrier or-lease. Do not cancel unrelated active requests just to save configuration.-Pure reordering with unchanged registration remains restart-free, and ordinary-invocations never trigger registration changes. Exercise buffered and streaming-completion, early close, cancellation, sync failure/rollback, and concurrent Save-to prove there is no deadlock or partial-output interruption.+Saving configuration never restarts the engine and never waits for in-flight+work. Pinned-engine experiments proved that CPA hot-reloads its config with an+unchanged PID and uninterrupted existing streams for all three protocols. They+also proved that CPA does not observe an atomic rename of its watched config+file. A running engine therefore receives the rendered projection through the+management `PUT /v0/management/config.yaml`, which validates it, writes the+file in place and reloads.++Neither the acknowledgment nor `/v1/models` is an atomic config-generation+barrier, and a save may leave every routed ID unchanged (for example a+reasoning-effort edit). Each rendered projection therefore carries a+per-process generation in every API-key model's display name. Synchronization+completes when CPA's Grok-client model listing, the one format that keeps both+the routed ID and the display name, shows every expected route at this+generation and no removed one, within a bounded wait. A rejection or timeout+fails the save and restores the previous projection through the same reload+path. A lost acknowledgment or a stalled management path never stops a live+process, because neither proves its streams dead; only an engine that has+already exited is left to its next start, which renders the projection.++The adapter's routing lock orders a projection change against new admissions.+A request admitted before a save keeps its projection until the engine has+read the request body, so a removal cannot strand it; that wait is bounded by a+local write, never by inference. CPA resolves the route on reading the body and+applies a config write only after its reload debounce, so the written body+orders the request first; its only later acknowledgment, response headers,+waits for upstream output and would make a save wait on inference. Established transport leases are neither+awaited nor cancelled, and a Save settles while a turn is streaming. Leases+still gate the one remaining restart: an engine binary upgrade. Credential+revocation follows an unbinding save, so a live engine drops the secret through+the same verified hot reload, which rewrites its config; other instances'+configs are cleared. An unverifiable reload fails the revocation instead of+restarting, and the journaled pending revocation is retried. Pure reordering with+unchanged registration is a no-op, and ordinary invocations never trigger+registration changes. Exercise buffered and streaming calls across a Save,+cancellation, reload failure/rollback, and concurrent Save, to prove there is+no waiting, restart, or partial-output interruption.Acceptance states properties, with fixtures covering the supported shapes:diff --git a/tests/e2e/test_model_hub_routing.py b/tests/e2e/test_model_hub_routing.pyindex 1599e6d83..d09ce5f8d 100644--- a/tests/e2e/test_model_hub_routing.py+++ b/tests/e2e/test_model_hub_routing.py@@ -201,8 +201,8 @@ def test_mh_routing_012_persisted_dormant_opencode_override_reaches_exact_source@pytest.mark.parametrize("protocol", ["openai_responses", "openai_chat", "anthropic"])@pytest.mark.parametrize("disconnect", [False, True], ids=["complete", "client-close"])-def test_mh_routing_009_save_drains_active_stream_without_truncation_or_deadlock(tmp_path, monkeypatch, protocol, disconnect):- """MH-ROUTING-009: Save waits for real stream completion or client close, then serves the new target without deadlock."""+def test_mh_routing_009_save_during_active_stream_hot_reloads_without_restart_or_truncation(tmp_path, monkeypatch, protocol, disconnect):+ """MH-ROUTING-009: Save hot-reloads the live engine during active output; the stream is neither awaited nor truncated."""async def exercise(adapter, mock):service, menu, target, sources = _real_route_service(tmp_path, adapter, [mock], protocol, [])await service.runtime_start()@@ -234,16 +234,15 @@ def test_mh_routing_009_save_drains_active_stream_without_truncation_or_deadlocksave = asyncio.create_task(service.set_agent_chain("opencode", menu, {"hops": [{"source_id": sources[0].id, "model_id": next_target}],}))- await asyncio.sleep(0.05)- assert not save.done(), "Save must wait for the active transport to finish"+ saved = await asyncio.wait_for(save, timeout=10)assert adapter.supervisor.client_if_running().connection == old_connection+ assert adapter._active_transports == 1if disconnect:response.close()- saved = await asyncio.wait_for(save, timeout=15)mock.configure(stream="healthy")else:mock.configure(stream="healthy")- saved, tail = await asyncio.wait_for(asyncio.gather(save, response.read()), timeout=15)+ tail = await asyncio.wait_for(response.read(), timeout=15)assert terminal in prefix + tailassert saved["chain"]["manual_override"] == {"hops": [{"source_id": sources[0].id, "model_id": next_target}]}async with client.post(f"{base_url}/v1/{path}", json={**payload, "stream": False},@@ -253,6 +252,7 @@ def test_mh_routing_009_save_drains_active_stream_without_truncation_or_deadlockcaptured = [row["body"]["model"] for row in mock.requests() if row["path"] == f"/v1/{path}"]assert captured == [target, next_target]assert service.store.load().sources[0].models == []+ assert adapter.supervisor.client_if_running().connection == old_connectionfinally:mock.configure(stream="healthy")if save is not None and not save.done():diff --git a/tests/scenarios/model_hub/catalog.yaml b/tests/scenarios/model_hub/catalog.yamlindex 6b60bc599..4f50a832b 100644--- a/tests/scenarios/model_hub/catalog.yaml+++ b/tests/scenarios/model_hub/catalog.yaml@@ -356,11 +356,11 @@ scenarios:layer: e2etest: tests/e2e/test_model_hub_routing.py::test_mh_routing_008_real_gateway_cpa_exact_model_and_source- id: MH-ROUTING-009- name: Registration-changing Save waits for active output completion or client close without deadlock or truncation+ name: Registration-changing Save hot-reloads the live engine during active output without waiting, restart, or truncationstatus: assertkind: concurrencylayer: e2e- test: tests/e2e/test_model_hub_routing.py::test_mh_routing_009_save_drains_active_stream_without_truncation_or_deadlock+ test: tests/e2e/test_model_hub_routing.py::test_mh_routing_009_save_during_active_stream_hot_reloads_without_restart_or_truncation- id: MH-ROUTING-010name: Real gateway model-not-found preserves source health and records exact attributed errors superseded by successstatus: assertdiff --git a/tests/test_model_hub_migration_bearer.py b/tests/test_model_hub_migration_bearer.pyindex a222fcf1f..91abc2ce5 100644--- a/tests/test_model_hub_migration_bearer.py+++ b/tests/test_model_hub_migration_bearer.py@@ -30,6 +30,7 @@ def _adapter(tmp_path):store = EngineStateStore(tmp_path / "engine")supervisor = SimpleNamespace(state_store=store, client_if_running=lambda: None, restart_if_running=lambda: None,+ reload_config_if_running=lambda _previous=None: None,invalidate_configs=store.clear_runtime_configs,)return CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)diff --git a/tests/test_model_hub_routing_modes.py b/tests/test_model_hub_routing_modes.pyindex 53f7be4c8..14175f3d1 100644--- a/tests/test_model_hub_routing_modes.py+++ b/tests/test_model_hub_routing_modes.py@@ -300,8 +300,8 @@ def test_engine_compiler_registers_inventory_union_without_invented_reasoning(tmpayload = {}_append_source(payload, record, store)assert payload[section][0]["models"] == [- {"name": "listed", "alias": "listed", "thinking": {"levels": ["high", "low"]}},- {"name": "unlisted", "alias": "unlisted"},+ {"name": "listed", "alias": "listed", "display-name": "listed #0", "thinking": {"levels": ["high", "low"]}},+ {"name": "unlisted", "alias": "unlisted", "display-name": "unlisted #0"},]store.sync_sources([replace(binding, model_ids=(), model_reasoning_efforts=())])assert store.list_sources()[0].model_ids == ()@@ -882,7 +882,7 @@ def test_retained_opencode_route_registration_matches_resolver_without_inventing_append_source(compiled, engine_store.get_source(source.id), engine_store)compiled_models = compiled["codex-api-key"][0]["models"]assert {item["name"] for item in compiled_models} == set(binding.model_ids) | set(expected_targets)- assert all(set(item) == {"name", "alias"} for item in compiled_models)+ assert all(set(item) == {"name", "alias", "display-name"} for item in compiled_models)if state == "unknown":adapter.outcomes.append(RawCallOutcome(diff --git a/tests/test_model_hub_runtime.py b/tests/test_model_hub_runtime.pyindex c36da64e4..c8dbaab7d 100644--- a/tests/test_model_hub_runtime.py+++ b/tests/test_model_hub_runtime.py@@ -6,6 +6,7 @@ import ioimport jsonimport loggingimport subprocess+import signalimport statimport sysimport tarfile@@ -48,6 +49,7 @@ from core.handlers.model_hub.stream_wire import (from vibe.model_hub_runtime import adapter as runtime_adapter_modulefrom vibe.model_hub_runtime import client as client_modulefrom vibe.model_hub_runtime import installer as runtime_installer_module+from vibe.model_hub_runtime import supervisor as supervisor_modulefrom vibe.model_hub_runtime.adapter import CLIProxyEngineAdapterfrom vibe.model_hub_runtime.api_key_vendors import api_key_vendor_catalogfrom vibe.model_hub_runtime.client import EngineClient, EngineClientError, EngineConnection@@ -1626,6 +1628,7 @@ def test_config_generation_is_private_and_never_logs_secrets({"name": "model-a","alias": "model-a",+ "display-name": "model-a #0","thinking": {"levels": ["high", "low"]},}]@@ -1712,7 +1715,7 @@ def test_mixed_anthropic_credentials_disable_cloak_only_for_api_key_entry("cloak": {"mode": "never"},"rebuild-mid-system-message": False,"models": [- {"name": "claude-api-model", "alias": "claude-api-model"}+ {"name": "claude-api-model", "alias": "claude-api-model", "display-name": "claude-api-model #0"}],}]@@ -2319,7 +2322,8 @@ def test_long_inventory_identity_reaches_runtime_config_and_http_consumer()config = yaml.safe_load(config_path.read_text(encoding="utf-8"))assert config[config_key][0]["models"] == [- {"name": identity, "alias": identity} for identity in (*identities, route_only)+ {"name": identity, "alias": identity, "display-name": f"{identity} #0"}+ for identity in (*identities, route_only)]client = EngineClient(EngineConnection(base_url.removesuffix("/v1"), "management", "gateway"))for identity in (*identities, route_only):@@ -2612,10 +2616,13 @@ def _write_mock_engine(startup_output_repeat: int = 1,echo_runtime_secrets: bool = False,exit_before_ready: int | None = None,+ reload_delay: float = 0.0,+ drop_reload_ack: bool = False,) -> None:script = f"""#!{sys.executable}import jsonimport sys+import threadingimport timefrom http.server import BaseHTTPRequestHandler, HTTPServer@@ -2646,6 +2653,12 @@ if exit_before_ready is not None:time.sleep({startup_delay!r})health_surfaces = set()+def configured_models():+ for section in ('claude-api-key', 'codex-api-key', 'openai-compatibility'):+ for entry in config.get(section) or ():+ for model in entry.get('models') or ():+ yield f"{{entry['prefix']}}/{{model['alias']}}", model.get('display-name')+def mark_health_surface(surface):health_surfaces.add(surface)if len(health_surfaces) == 2:@@ -2667,7 +2680,7 @@ class Handler(BaseHTTPRequestHandler):def do_GET(self):if self.path == '/v1/models' and self.headers.get('Authorization') == f'Bearer {{gateway}}':mark_health_surface('gateway')- self._json(200, {{'object': 'list', 'data': []}})+ self._json(200, {{'object': 'list', 'data': [{{'id': model, 'name': name}} for model, name in configured_models()]}})returnif self.path == '/v0/management/config' and self.headers.get('X-Management-Key') == management:mark_health_surface('management')@@ -2678,6 +2691,29 @@ class Handler(BaseHTTPRequestHandler):returnself._json(401, {{'error': {{'type': 'unauthorized'}}}})+ def do_PUT(self):+ global config+ length = int(self.headers.get('Content-Length', '0'))+ body = self.rfile.read(length)+ if self.path != '/v0/management/config.yaml' or self.headers.get('X-Management-Key') != management:+ self._json(401, {{'error': {{'type': 'unauthorized'}}}})+ return+ with open(config_path, 'wb') as handle:+ handle.write(body)++ def apply():+ global config+ config = yaml.safe_load(body)+ with open('config-reloads', 'a', encoding='utf-8') as handle:+ handle.write('reload\\n')++ # CPA acknowledges the write and applies it from a debounced watcher.+ threading.Timer({reload_delay!r}, apply).start()+ if {drop_reload_ack!r}:+ self.close_connection = True+ return+ self._json(200, {{'ok': True, 'changed': ['config']}})+def do_POST(self):length = int(self.headers.get('Content-Length', '0'))payload = json.loads(self.rfile.read(length))@@ -2803,6 +2839,8 @@ def _fixture_supervisor(startup_output_repeat: int = 1,echo_runtime_secrets: bool = False,exit_before_ready: int | None = None,+ reload_delay: float = 0.0,+ drop_reload_ack: bool = False,) -> tuple[EngineSupervisor, EngineStateStore]:tmp_path.mkdir(parents=True, exist_ok=True)binary = tmp_path / "mock-engine"@@ -2813,6 +2851,8 @@ def _fixture_supervisor(startup_output_repeat=startup_output_repeat,echo_runtime_secrets=echo_runtime_secrets,exit_before_ready=exit_before_ready,+ reload_delay=reload_delay,+ drop_reload_ack=drop_reload_ack,)installer = _FixtureInstaller(binary, tmp_path / "versions" / "install-1")store = EngineStateStore(tmp_path / "state")@@ -2827,6 +2867,199 @@ def _fixture_supervisor()+def test_source_sync_hot_reloads_a_running_engine_without_restart(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ connection = supervisor.ensure_running()+ process = supervisor._process+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ asyncio.run(adapter.sync_sources([_binding(credential_ref, route_model_ids=("model-b",))]))+ client = EngineClient(connection)+ prefix = store.get_source("src_fixture123").prefix+ assert supervisor._process is process+ assert supervisor._connection == connection+ assert client.list_model_ids() == {f"{prefix}/model-a", f"{prefix}/model-b"}+ asyncio.run(adapter.sync_sources([_binding(credential_ref)]))+ assert supervisor._process is process+ assert client.list_model_ids() == {f"{prefix}/model-a"}+ instance_dir = store.root / "instances" / "install-1"+ assert (instance_dir / "config-reloads").read_text().count("reload") == 2+ assert "model-b" not in (instance_dir / "config.yaml").read_text()+ finally:+ supervisor.stop()+++def test_source_sync_waits_for_an_id_preserving_reload_to_apply(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path, reload_delay=0.3)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ supervisor.ensure_running()+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ instance_dir = store.root / "instances" / "install-1"+ try:+ # A reasoning-only save renders new config yet leaves every routed ID+ # listed; the save returns only once the engine has applied it.+ asyncio.run(+ adapter.sync_sources(+ [_binding(credential_ref, model_reasoning_efforts=(("model-a", ("low", "high")),))]+ )+ )+ assert (instance_dir / "config-reloads").read_text().count("reload") == 1+ finally:+ supervisor.stop()+++def test_source_sync_waits_for_a_reload_that_removes_every_model(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path, reload_delay=0.3)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ connection = supervisor.ensure_running()+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ # No model is expected afterwards, so readiness is the removed+ # model leaving the listing.+ asyncio.run(adapter.sync_sources([_binding(credential_ref, model_ids=())]))+ assert EngineClient(connection).list_model_ids() == frozenset()+ finally:+ supervisor.stop()+++def test_source_sync_never_restarts_a_live_engine_that_lost_the_reload_ack(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path, drop_reload_ack=True)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ connection = supervisor.ensure_running()+ process = supervisor._process+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ asyncio.run(adapter.sync_sources([_binding(credential_ref, route_model_ids=("model-b",))]))+ prefix = store.get_source("src_fixture123").prefix+ assert supervisor._process is process+ assert EngineClient(connection).list_model_ids() == {f"{prefix}/model-a", f"{prefix}/model-b"}+ finally:+ supervisor.stop()+++def test_source_sync_fails_without_restarting_a_stalled_engine(+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(supervisor_module, "MODEL_HUB_CONFIG_RELOAD_TIMEOUT_SECONDS", 0.5)+ supervisor, store = _fixture_supervisor(tmp_path)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ supervisor.ensure_running()+ process = supervisor._process+ original = store.list_sources()+ process.send_signal(signal.SIGSTOP)+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ # A stalled management path does not prove its streams dead.+ with pytest.raises(EngineStateError):+ asyncio.run(adapter.sync_sources([_binding(credential_ref, route_model_ids=("model-b",))]))+ assert supervisor._process is process+ assert process.poll() is None+ finally:+ process.send_signal(signal.SIGCONT)+ supervisor.stop()+ assert store.list_sources() == original+++def test_source_sync_of_an_exited_engine_is_applied_at_next_start(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(credential_ref)])+ supervisor.ensure_running()+ exited = supervisor._process+ exited.kill()+ exited.wait(timeout=3)+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ asyncio.run(adapter.sync_sources([_binding(credential_ref, route_model_ids=("model-b",))]))+ prefix = store.get_source("src_fixture123").prefix+ connection = supervisor.ensure_running()+ assert supervisor._process is not exited+ assert EngineClient(connection).list_model_ids() == {f"{prefix}/model-a", f"{prefix}/model-b"}+ finally:+ supervisor.stop()+++def test_credential_revocation_hot_reloads_a_live_engine_without_restart(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path)+ old_ref = store.store_api_key("old-upstream-secret", base_url="https://api.example.test/v1")+ new_ref = store.store_api_key("new-upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(old_ref)])+ connection = supervisor.ensure_running()+ process = supervisor._process+ stale_instance = store.root / "instances" / "install-0"+ stale_instance.mkdir(mode=0o700)+ stale_config = stale_instance / "config.yaml"+ stale_config.write_text("api-key: old-upstream-secret\n", encoding="utf-8")+ stale_config.chmod(0o600)+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ # Replacing a key rebinds first, then revokes the old credential.+ asyncio.run(adapter.sync_sources([_binding(new_ref)]))+ asyncio.run(adapter.revoke_api_key_credential(old_ref))+ assert supervisor._process is process+ live_config = (store.root / "instances" / "install-1" / "config.yaml").read_text()+ assert "old-upstream-secret" not in live_config+ assert "new-upstream-secret" in live_config+ assert not stale_config.exists()+ prefix = store.get_source("src_fixture123").prefix+ assert EngineClient(connection).list_model_ids() == {f"{prefix}/model-a"}+ finally:+ supervisor.stop()+++def test_credential_revocation_stays_pending_without_restarting_a_stalled_engine(+ tmp_path: Path,+ monkeypatch: pytest.MonkeyPatch,+) -> None:+ monkeypatch.setattr(supervisor_module, "MODEL_HUB_CONFIG_RELOAD_TIMEOUT_SECONDS", 0.5)+ supervisor, store = _fixture_supervisor(tmp_path)+ old_ref = store.store_api_key("old-upstream-secret", base_url="https://api.example.test/v1")+ new_ref = store.store_api_key("new-upstream-secret", base_url="https://api.example.test/v1")+ store.sync_sources([_binding(old_ref)])+ supervisor.ensure_running()+ process = supervisor._process+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ try:+ asyncio.run(adapter.sync_sources([_binding(new_ref)]))+ process.send_signal(signal.SIGSTOP)+ try:+ # An unverifiable reload must not truncate live streams; the+ # caller keeps the revocation pending and retries it.+ with pytest.raises(EngineUnavailableError):+ asyncio.run(adapter.revoke_api_key_credential(old_ref))+ assert supervisor._process is process+ assert process.poll() is None+ finally:+ process.send_signal(signal.SIGCONT)+ assert store.credential_metadata_if_present(old_ref) is not None+ asyncio.run(adapter.revoke_api_key_credential(old_ref))+ assert supervisor._process is process+ assert store.credential_metadata_if_present(old_ref) is None+ finally:+ supervisor.stop()+++def test_source_sync_of_a_stopped_engine_is_applied_at_next_start(tmp_path: Path) -> None:+ supervisor, store = _fixture_supervisor(tmp_path)+ credential_ref = store.store_api_key("upstream-secret", base_url="https://api.example.test/v1")+ adapter = CLIProxyEngineAdapter(supervisor=supervisor, state_store=store)+ asyncio.run(adapter.sync_sources([_binding(credential_ref, route_model_ids=("model-b",))]))+ assert supervisor._process is None+ try:+ connection = supervisor.ensure_running()+ prefix = store.get_source("src_fixture123").prefix+ assert EngineClient(connection).list_model_ids() == {f"{prefix}/model-a", f"{prefix}/model-b"}+ finally:+ supervisor.stop()++@pytest.mark.parametrize(("installed", "start_attempted", "running", "healthy", "expected"),[@@ -3249,14 +3482,14 @@ def test_supervisor_reaps_an_engine_whose_pid_was_never_recorded(tmp_path: Path)def test_supervisor_refresh_and_reap_stop_an_engine_a_previous_service_left_running(tmp_path: Path) -> None:record = tmp_path / "state" / "engine-process.json"- for operation in ("restart_if_running", "with_engine_excluded"):+ for operation in ("restart_if_running", "reload_config_if_running", "with_engine_excluded"):orphan = _orphan_engine(tmp_path)second, _store = _fixture_supervisor(tmp_path)if operation == "with_engine_excluded":assert second.with_engine_excluded(lambda client: client) is Noneelse:- second.restart_if_running()+ getattr(second, operation)()_wait_for(lambda: orphan.poll() is not None)assert second._process is None@@ -3295,7 +3528,11 @@ def test_supervisor_refresh_refuses_while_a_previous_engine_is_unconfirmed(monkeypatch.setattr(supervisor_module, "reap_marked_processes", lambda *_a, **_k: "unconfirmed")second, _store = _fixture_supervisor(tmp_path)- for call in (second.restart_if_running, lambda: second.with_engine_excluded(pytest.fail)):+ for call in (+ second.restart_if_running,+ second.reload_config_if_running,+ lambda: second.with_engine_excluded(pytest.fail),+ ):with pytest.raises(EngineUnavailableError) as raised:call()assert raised.value.reason == "previous_engine_alive"@@ -3865,6 +4102,7 @@ def test_adapter_uses_origin_protocol_for_engine_translation(request_protocol=None,request_headers=None,on_transport_done=None,+ on_request_sent=None,):self.request_protocol = request_protocolself.request_headers = request_headers@@ -3917,20 +4155,16 @@ def test_adapter_uses_origin_protocol_for_engine_translation(asyncio.run(run())-def test_adapter_restores_source_projection_when_restart_fails(tmp_path: Path) -> None:+def test_adapter_restores_source_projection_when_reload_fails(tmp_path: Path) -> None:class Supervisor:def __init__(self) -> None:self.restore_calls = 0- def client_if_running(self):- return object()-- def restart_if_running(self) -> None:- raise EngineUnavailableError("models.engine.health_failed")-- def ensure_running(self):+ def reload_config_if_running(self, _previous=None) -> None:+ if self.restore_calls == 0:+ self.restore_calls += 1+ raise EngineUnavailableError("models.engine.health_failed")self.restore_calls += 1- return object()async def run() -> None:store = EngineStateStore(tmp_path / "state")@@ -3956,7 +4190,7 @@ def test_adapter_restores_source_projection_when_restart_fails(tmp_path: Path) -restored = store.get_source("src_fixture123")assert restored is not Noneassert restored.credential_ref == old_ref- assert supervisor.restore_calls == 1+ assert supervisor.restore_calls == 2asyncio.run(run())@@ -3977,6 +4211,7 @@ def test_adapter_serializes_source_sync_with_new_invocations(tmp_path: Path) ->request_protocol=None,request_headers=None,on_transport_done=None,+ on_request_sent=None,):invoked_refs.append(source.credential_ref)on_transport_done()@@ -3992,7 +4227,7 @@ def test_adapter_serializes_source_sync_with_new_invocations(tmp_path: Path) ->def with_engine_excluded(self, operation):return operation(self._client)- def restart_if_running(self) -> None:+ def reload_config_if_running(self, _previous=None) -> None:restart_started.set()assert allow_restart.wait(timeout=2)diff --git a/tests/test_model_hub_takeover_integration.py b/tests/test_model_hub_takeover_integration.pyindex 43d8d133a..8d5964bb2 100644--- a/tests/test_model_hub_takeover_integration.py+++ b/tests/test_model_hub_takeover_integration.py@@ -45,6 +45,9 @@ class FixtureSupervisor:def restart_if_running(self):pass+ def reload_config_if_running(self, _previous=None):+ pass+def status(self):return {"status": {"health": "ok", "installed_version": "fixture", "verified": True,diff --git a/tests/test_model_hub_transport_leases.py b/tests/test_model_hub_transport_leases.pyindex 60a49880f..c52d68a4c 100644--- a/tests/test_model_hub_transport_leases.py+++ b/tests/test_model_hub_transport_leases.py@@ -22,12 +22,11 @@ LAST = b'data: {"choices":[{"delta":{},"finish_reason":"stop"}]}\n\ndata: [DONE]class LeaseSupervisor:def __init__(self, origin):self.connection = EngineConnection(origin, "fixture-management", "fixture-token")- self.restart_started = threading.Event()- self.allow_restart = threading.Event()- self.allow_restart.set()- self.restart_count = 0- self.restore_count = 0- self.fail_restart = False+ self.reload_started = threading.Event()+ self.allow_reload = threading.Event()+ self.allow_reload.set()+ self.reload_count = 0+ self.fail_reload = Falsedef client(self):return EngineClient(self.connection)@@ -36,16 +35,16 @@ class LeaseSupervisor:return self.client()def restart_if_running(self):- self.restart_count += 1- self.restart_started.set()- assert self.allow_restart.wait(3)- if self.fail_restart:+ raise AssertionError("a source save must hot-reload, never restart")++ def reload_config_if_running(self, _previous=None):+ self.reload_count += 1+ self.reload_started.set()+ assert self.allow_reload.wait(3)+ if self.fail_reload:+ self.fail_reload = Falseraise EngineUnavailableError("models.engine.health_failed")- def ensure_running(self):- self.restore_count += 1- return self.connection-@asynccontextmanagerasync def _transport(tmp_path):@@ -97,20 +96,14 @@ async def _transport(tmp_path):yield adapter, supervisor, binding, received, finish, requestsfinally:finish.set()- supervisor.allow_restart.set()+ supervisor.allow_reload.set()await runner.cleanup()-async def _barrier_waiting(adapter, sync):- async with asyncio.timeout(2):- while not adapter._routing_lock.locked():- assert not sync.done()- await asyncio.sleep(0)- assert not sync.done()--@pytest.mark.parametrize("stream", [False, True])-def test_sync_waits_for_buffered_and_streaming_transport_not_service_settlement(tmp_path, stream):+def test_sync_applies_during_buffered_and_streaming_transport_without_waiting(tmp_path, stream):+ """A save must not wait for, restart under, or truncate an in-flight call."""+async def run():async with _transport(tmp_path) as (adapter, supervisor, binding, received, finish, requests):invoke = asyncio.create_task(adapter.invoke(binding.source_id, "unknown", {}, stream, "opencode"))@@ -119,24 +112,23 @@ def test_sync_waits_for_buffered_and_streaming_transport_not_service_settlement(if handle is not None:assert await anext(handle.stream) == FIRSTchanged = replace(binding, route_model_ids=("unknown", "new"))- sync = asyncio.create_task(adapter.sync_sources([changed]))- await _barrier_waiting(adapter, sync)+ await asyncio.wait_for(adapter.sync_sources([changed]), 2)+ assert supervisor.reload_count == 1+ assert adapter._active_transports == 1+ # The fixture holds every upstream response until ``finish``; the+ # new route is admitted while the first call is still open.later = asyncio.create_task(adapter.invoke(binding.source_id, "new", {}, False, "opencode"))- await asyncio.sleep(0)- assert not later.done()- assert supervisor.restart_count == 0- assert len(requests) == 1- # A service mutation lock can remain held during this drain. No- # outcome()/settlement call is needed to release the transport.+ async with asyncio.timeout(2):+ while len(requests) < 2:+ await asyncio.sleep(0.01)finish.set()if handle is not None:assert b"".join([part async for part in handle.stream]) == LASTelse:handle = await asyncio.wait_for(invoke, 2)- await asyncio.wait_for(sync, 2)- assert supervisor.restart_count == 1- assert (await handle.outcome()).kind is RawOutcomeKind.SUCCESSlater_handle = await asyncio.wait_for(later, 2)+ assert (await handle.outcome()).kind is RawOutcomeKind.SUCCESS+ assert (await later_handle.outcome()).kind is RawOutcomeKind.SUCCESSawait handle.close_stream()await later_handle.close_stream()assert adapter._active_transports == 0@@ -145,76 +137,49 @@ def test_sync_waits_for_buffered_and_streaming_transport_not_service_settlement(asyncio.run(run())[email protected]("finish_kind", ["unstarted_close", "started_close", "stream_cancel", "invoke_cancel"])-def test_close_and_cancellation_release_transport_without_waiting_for_outcome(tmp_path, finish_kind):+def test_sync_keeps_an_admitted_request_routable_until_the_engine_has_read_it(tmp_path, monkeypatch):+ """A removal saved after admission must not strand a request not yet sent."""+async def run():- async with _transport(tmp_path) as (adapter, supervisor, binding, received, _finish, _requests):- streaming = finish_kind != "invoke_cancel"- invoke = asyncio.create_task(adapter.invoke(binding.source_id, "unknown", {}, streaming, "opencode"))+ async with _transport(tmp_path) as (adapter, supervisor, binding, received, finish, requests):+ connect = asyncio.Event()+ original = EngineClient.invoke++ async def delayed_invoke(self, *args, **kwargs):+ await connect.wait()+ return await original(self, *args, **kwargs)++ monkeypatch.setattr(EngineClient, "invoke", delayed_invoke)+ invoke = asyncio.create_task(adapter.invoke(binding.source_id, "unknown", {}, False, "opencode"))+ async with asyncio.timeout(2):+ while adapter._requests_sent.is_set():+ await asyncio.sleep(0.01)+ sync = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=())]))+ await asyncio.sleep(0.05)+ assert supervisor.reload_count == 0+ connect.set()await asyncio.wait_for(received.wait(), 2)- handle = await invoke if streaming else None- pending_read = None- if finish_kind in {"started_close", "stream_cancel"}:- assert await anext(handle.stream) == FIRST- if finish_kind == "stream_cancel":- pending_read = asyncio.create_task(anext(handle.stream))- await asyncio.sleep(0)- sync = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=("new",))]))- await _barrier_waiting(adapter, sync)- assert supervisor.restart_count == 0- if handle is None:- invoke.cancel()- with pytest.raises(asyncio.CancelledError):- await invoke- elif pending_read is not None:- pending_read.cancel()- with pytest.raises(asyncio.CancelledError):- await pending_read- await handle.close_stream()- else:- await handle.close_stream()- await handle.close_stream()await asyncio.wait_for(sync, 2)- assert supervisor.restart_count == 1- assert adapter._active_transports == 0- assert adapter._transports_idle.is_set()-- asyncio.run(run())---def test_cancelled_drain_does_not_cancel_active_request_or_leak_barrier(tmp_path):- async def run():- async with _transport(tmp_path) as (adapter, supervisor, binding, _received, finish, _requests):- handle = await adapter.invoke(binding.source_id, "unknown", {}, True, "opencode")- assert await anext(handle.stream) == FIRST- before = adapter.state_store.list_sources()- sync = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=("new",))]))- await _barrier_waiting(adapter, sync)- sync.cancel()- with pytest.raises(asyncio.CancelledError):- await sync- assert not adapter._routing_lock.locked()- assert adapter.state_store.list_sources() == before- assert adapter._active_transports == 1- assert supervisor.restart_count == 0+ assert supervisor.reload_count == 1+ assert requests[0]["model"].endswith("/unknown")finish.set()- assert b"".join([part async for part in handle.stream]) == LAST+ handle = await asyncio.wait_for(invoke, 2)assert (await handle.outcome()).kind is RawOutcomeKind.SUCCESS- await adapter.sync_sources([binding])- assert adapter._active_transports == 0+ await handle.close_stream()+ assert adapter._unsent_requests == 0 and adapter._active_transports == 0asyncio.run(run())[email protected]("fail_restart", [False, True])-def test_cancelled_restart_retains_barrier_until_commit_or_rollback(tmp_path, fail_restart):[email protected]("fail_reload", [False, True])+def test_cancelled_reload_holds_admission_until_commit_or_rollback(tmp_path, fail_reload):async def run():async with _transport(tmp_path) as (adapter, supervisor, binding, _received, finish, requests):original = adapter.state_store.list_sources()- supervisor.allow_restart.clear()- supervisor.fail_restart = fail_restart+ supervisor.allow_reload.clear()+ supervisor.fail_reload = fail_reloadsync = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=("changed",))]))- assert await asyncio.to_thread(supervisor.restart_started.wait, 2)+ assert await asyncio.to_thread(supervisor.reload_started.wait, 2)sync.cancel()await asyncio.sleep(0)assert not sync.done()@@ -222,16 +187,16 @@ def test_cancelled_restart_retains_barrier_until_commit_or_rollback(tmp_path, falater = asyncio.create_task(adapter.invoke(binding.source_id, "listed", {}, False, "opencode"))await asyncio.sleep(0)assert requests == []- supervisor.allow_restart.set()+ supervisor.allow_reload.set()with pytest.raises(asyncio.CancelledError):await asyncio.wait_for(sync, 2)finish.set()handle = await asyncio.wait_for(later, 2)assert (await handle.outcome()).kind is RawOutcomeKind.SUCCESSawait handle.close_stream()- if fail_restart:+ if fail_reload:assert adapter.state_store.list_sources() == original- assert supervisor.restore_count == 1+ assert supervisor.reload_count == 2else:assert adapter.state_store.list_sources()[0].route_model_ids == ("changed",)assert adapter._active_transports == 0@@ -240,33 +205,33 @@ def test_cancelled_restart_retains_barrier_until_commit_or_rollback(tmp_path, faasyncio.run(run())-def test_two_concurrent_saves_are_serial_and_restart_failure_restores_projection(tmp_path):+def test_two_concurrent_saves_are_serial_and_reload_failure_restores_projection(tmp_path):async def run():async with _transport(tmp_path) as (adapter, supervisor, binding, _received, _finish, _requests):original = adapter.state_store.list_sources()- supervisor.fail_restart = True+ supervisor.fail_reload = Truewith pytest.raises(EngineUnavailableError):await adapter.sync_sources([replace(binding, route_model_ids=("bad",))])assert adapter.state_store.list_sources() == original- assert supervisor.restore_count == 1- supervisor.fail_restart = False- supervisor.restart_started.clear()- supervisor.allow_restart.clear()+ assert supervisor.reload_count == 2+ supervisor.reload_started.clear()+ supervisor.allow_reload.clear()first = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=("first",))]))- assert await asyncio.to_thread(supervisor.restart_started.wait, 2)+ assert await asyncio.to_thread(supervisor.reload_started.wait, 2)second = asyncio.create_task(adapter.sync_sources([replace(binding, route_model_ids=("second",))]))await asyncio.sleep(0)assert adapter.state_store.list_sources()[0].route_model_ids == ("first",)- supervisor.allow_restart.set()+ supervisor.allow_reload.set()await asyncio.wait_for(asyncio.gather(first, second), 2)assert adapter.state_store.list_sources()[0].route_model_ids == ("second",)- assert supervisor.restart_count == 3+ assert supervisor.reload_count == 4assert not adapter._routing_lock.locked()asyncio.run(run())-def test_service_save_drain_releases_before_settlement_can_acquire_mutation_lock(tmp_path):+def test_service_save_completes_while_a_stream_is_active(tmp_path):+ """The reported hang: a model-list save during an active turn must settle."""from tests.test_model_hub_resolution import _service, _sourcefrom tests.test_model_hub_routing_modes import MODEL, _sparse_config@@ -280,27 +245,52 @@ def test_service_save_drain_releases_before_settlement_can_acquire_mutation_lockadapter.state_store.sync_sources(service._bindings(config))handle = await adapter.invoke(source.id, MODEL, {}, True, "opencode")assert await anext(handle.stream) == FIRST- save = asyncio.create_task(- service.set_agent_chain("claude", MODEL, {"hops": [{"source_id": source.id, "model_id": "new"}]})+ result = await asyncio.wait_for(+ service.set_agent_chain("claude", MODEL, {"hops": [{"source_id": source.id, "model_id": "new"}]}),+ 2,)- await _barrier_waiting(adapter, save)- assert service._mutation_lock.locked()-- async def settle():- async with service._mutation_lock:- return await handle.outcome()-- settlement = asyncio.create_task(settle())- await asyncio.sleep(0)- assert not settlement.done()- assert supervisor.restart_count == 0- finish.set()- assert b"".join([part async for part in handle.stream]) == LAST- result, outcome = await asyncio.wait_for(asyncio.gather(save, settlement), 2)assert result["chain"]["current"] == {"source_id": source.id, "model_id": "new"}- assert outcome.kind is RawOutcomeKind.SUCCESSassert store.config.agents["claude"].routes[MODEL].hops[0].model_id == "new"- assert supervisor.restart_count == 1+ assert supervisor.reload_count == 1+ assert not service._mutation_lock.locked()+ finish.set()+ assert b"".join([part async for part in handle.stream]) == LAST+ async with service._mutation_lock:+ assert (await handle.outcome()).kind is RawOutcomeKind.SUCCESS+ assert adapter._active_transports == 0++ asyncio.run(run())++[email protected]("finish_kind", ["unstarted_close", "started_close", "stream_cancel", "invoke_cancel"])+def test_close_and_cancellation_release_transport_without_waiting_for_outcome(tmp_path, finish_kind):+ # Leases still gate an engine binary upgrade restart.+ async def run():+ async with _transport(tmp_path) as (adapter, _supervisor, binding, received, _finish, _requests):+ streaming = finish_kind != "invoke_cancel"+ invoke = asyncio.create_task(adapter.invoke(binding.source_id, "unknown", {}, streaming, "opencode"))+ await asyncio.wait_for(received.wait(), 2)+ handle = await invoke if streaming else None+ pending_read = None+ if finish_kind in {"started_close", "stream_cancel"}:+ assert await anext(handle.stream) == FIRST+ if finish_kind == "stream_cancel":+ pending_read = asyncio.create_task(anext(handle.stream))+ await asyncio.sleep(0)+ assert not adapter._transports_idle.is_set()+ if handle is None:+ invoke.cancel()+ with pytest.raises(asyncio.CancelledError):+ await invoke+ elif pending_read is not None:+ pending_read.cancel()+ with pytest.raises(asyncio.CancelledError):+ await pending_read+ await handle.close_stream()+ else:+ await handle.close_stream()+ await handle.close_stream()+ await asyncio.wait_for(adapter._transports_idle.wait(), 2)assert adapter._active_transports == 0asyncio.run(run())diff --git a/vibe/model_hub_runtime/adapter.py b/vibe/model_hub_runtime/adapter.pyindex eb04827ef..7ccda9e07 100644--- a/vibe/model_hub_runtime/adapter.py+++ b/vibe/model_hub_runtime/adapter.py@@ -1247,6 +1247,9 @@ class CLIProxyEngineAdapter:self._active_transports = 0self._transports_idle = asyncio.Event()self._transports_idle.set()+ self._unsent_requests = 0+ self._requests_sent = asyncio.Event()+ self._requests_sent.set()self._installation_lock = asyncio.Lock()self._install_task: asyncio.Task[None] | None = Noneself._install_admission: asyncio.Future[EngineStatus] | None = None@@ -1682,9 +1685,14 @@ class CLIProxyEngineAdapter:async def sync_sources(self, bindings: Sequence[SourceBinding]) -> None:async with self._routing_lock:- await self._transports_idle.wait()- # Cancellation must retain the barrier until the finite transaction- # has either restarted or restored the prior projection.+ # The routing lock orders the projection against new admissions.+ # An admitted request keeps the projection it was admitted under+ # until the engine has read it; established streams keep their+ # connection, since CPA hot-reloads the config, so a save neither+ # waits for them nor restarts. Cancellation must retain the lock+ # until the finite transaction has applied or restored the prior+ # projection.+ await self._requests_sent.wait()await run_owned_in_thread(self._sync_sources_transaction, tuple(bindings))def _sync_sources_transaction(self, bindings: Sequence[SourceBinding]) -> None:@@ -1692,21 +1700,36 @@ class CLIProxyEngineAdapter:previous = self.state_store.list_sources()except EngineStateError:previous = []- was_running = self.supervisor.client_if_running() is not Noneself.state_store.sync_sources(bindings)try:- self.supervisor.restart_if_running()+ self.supervisor.reload_config_if_running(previous)except Exception:+ applied = self.state_store.list_sources()self.state_store.replace_sources(previous)- if was_running:- try:- self.supervisor.ensure_running()- except Exception as restore_error:- raise EngineStateError(- "source sync failed and the previous engine state could not be restored"- ) from restore_error+ try:+ self.supervisor.reload_config_if_running(applied)+ except Exception as restore_error:+ raise EngineStateError(+ "source sync failed and the previous engine state could not be restored"+ ) from restore_errorraise+ def _hold_unsent_request(self) -> Callable[[], None]:+ self._unsent_requests += 1+ self._requests_sent.clear()+ released = False++ def release() -> None:+ nonlocal released+ if released:+ return+ released = True+ self._unsent_requests -= 1+ if self._unsent_requests == 0:+ self._requests_sent.set()++ return release+def _acquire_transport(self) -> Callable[[], None]:self._active_transports += 1self._transports_idle.clear()@@ -2627,10 +2650,12 @@ class CLIProxyEngineAdapter:))release = self._acquire_transport()+ request_sent = self._hold_unsent_request()try:if on_admitted is not None:on_admitted()except BaseException:+ request_sent()release()raisetry:@@ -2646,10 +2671,13 @@ class CLIProxyEngineAdapter:request_protocol=request_protocol,request_headers=getattr(request, "headers", None),on_transport_done=release,+ on_request_sent=request_sent,)except BaseException:release()raise+ finally:+ request_sent()async def _complete_oauth(self, flow: _OAuthFlow, client: EngineClient) -> None:flow.grant_write_possible = Truediff --git a/vibe/model_hub_runtime/client.py b/vibe/model_hub_runtime/client.pyindex d7e5bcbc9..494d1c9fa 100644--- a/vibe/model_hub_runtime/client.py+++ b/vibe/model_hub_runtime/client.py@@ -355,6 +355,56 @@ class EngineClient:timeout=timeout,)+ def put_config_yaml(self, text: str, *, timeout: float | None = None) -> dict[str, Any]:+ """Replace the running engine config through its hot-reload endpoint."""+ return self._request_json_projection(+ "PUT",+ "/v0/management/config.yaml",+ _load_json_object,+ data=text.encode(),+ content_type="application/yaml",+ headers={"X-Management-Key": self.connection.management_key},+ timeout=timeout,+ )++ def list_model_ids(self, *, timeout: float | None = None) -> frozenset[str]:+ payload = self._request_json(+ "GET",+ "/v1/models",+ headers={"Authorization": f"Bearer {self.connection.gateway_token}"},+ timeout=timeout,+ )+ entries = payload.get("data")+ if not isinstance(entries, list):+ raise EngineClientError("engine API returned an invalid payload", error_type="invalid_json")+ return frozenset(+ str(entry["id"]) for entry in entries if isinstance(entry, dict) and isinstance(entry.get("id"), str)+ )++ def list_model_names(self, *, timeout: float | None = None) -> dict[str, str]:+ """Routed model ID to registered display name.++ CPA's default OpenAI listing drops display names; its Grok-client+ listing keeps the routed ID and names every provider's registration.+ """+ payload = self._request_json(+ "GET",+ "/v1/models",+ headers={+ "Authorization": f"Bearer {self.connection.gateway_token}",+ "User-Agent": "grok-shell",+ },+ timeout=timeout,+ )+ entries = payload.get("data")+ if not isinstance(entries, list):+ raise EngineClientError("engine API returned an invalid payload", error_type="invalid_json")+ return {+ str(entry["id"]): str(entry.get("name") or "")+ for entry in entries+ if isinstance(entry, dict) and isinstance(entry.get("id"), str)+ }+async def invoke(self,source: SourceRecord,@@ -365,6 +415,7 @@ class EngineClient:request_protocol: str | None = None,request_headers: Mapping[str, str] | None = None,on_transport_done: Callable[[], None] | None = None,+ on_request_sent: Callable[[], None] | None = None,) -> EngineInvokeHandle:request_protocol = request_protocol or source.protocolendpoint = _endpoint_for_protocol(request_protocol)@@ -393,7 +444,21 @@ class EngineClient:# Connecting to the local engine is bounded. Once connected, headers# and response bytes can wait on upstream inference for any duration;# completion, transport failure, or owner cancellation ends that wait.- session = aiohttp.ClientSession(timeout=timeout, trust_env=False)+ trace_configs: list[aiohttp.TraceConfig] = []+ if on_request_sent is not None:+ # CPA resolves the route as soon as it reads the body, while a+ # config write only applies after its 150ms reload debounce, so a+ # body written to the loopback socket precedes any route removal.+ # CPA sends no earlier acknowledgment: response headers wait for+ # the upstream's first chunk, which a save must never wait on.+ trace = aiohttp.TraceConfig()++ async def request_sent(*_args: Any) -> None:+ on_request_sent()++ trace.on_request_chunk_sent.append(request_sent)+ trace_configs.append(trace)+ session = aiohttp.ClientSession(timeout=timeout, trust_env=False, trace_configs=trace_configs)response: aiohttp.ClientResponse | None = Noneretry_after: str | None = Noneresponse_received_at: datetime | None = None@@ -686,16 +751,19 @@ class EngineClient:*,query: Mapping[str, str] | None = None,payload: Mapping[str, Any] | None = None,+ data: bytes | None = None,+ content_type: str = "application/json",headers: Mapping[str, str] | None = None,timeout: float | None = None,) -> _ProjectedJSON:url = self._url(path, query=query)request_timeout = timeout or self.timeoutdeadline = time.monotonic() + request_timeout- data = None if payload is None else json.dumps(payload, separators=(",", ":")).encode()+ if payload is not None:+ data = json.dumps(payload, separators=(",", ":")).encode()request_headers = dict(headers or {})if data is not None:- request_headers["Content-Type"] = "application/json"+ request_headers["Content-Type"] = content_typerequest = urllib.request.Request(url, data=data, headers=request_headers, method=method)try:opener = urllib.request.build_opener(diff --git a/vibe/model_hub_runtime/config.py b/vibe/model_hub_runtime/config.pyindex aaf1582ff..1ba2c0468 100644--- a/vibe/model_hub_runtime/config.py+++ b/vibe/model_hub_runtime/config.py@@ -21,7 +21,39 @@ def write_engine_config(runtime_secrets: RuntimeSecrets,sources: Iterable[SourceRecord],state_store: EngineStateStore,+ generation: str = "0",) -> None:+ _secure_write_text(+ path,+ render_engine_config(+ path,+ host=host,+ port=port,+ auth_dir=auth_dir,+ runtime_secrets=runtime_secrets,+ sources=sources,+ state_store=state_store,+ generation=generation,+ ),+ )+++def render_engine_config(+ path: Path,+ *,+ host: str,+ port: int,+ auth_dir: Path,+ runtime_secrets: RuntimeSecrets,+ sources: Iterable[SourceRecord],+ state_store: EngineStateStore,+ generation: str = "0",+) -> str:+ """Render the engine YAML that ``path`` holds, for a start or a hot reload.++ ``generation`` marks every configured model's display name, so a reload is+ observable as applied even when it leaves the routed model IDs unchanged.+ """if host != "127.0.0.1":raise EngineStateError("model hub engine must bind to 127.0.0.1")payload: dict[str, Any] = {@@ -63,11 +95,16 @@ def write_engine_config("ws-auth": True,}for source in sources:- _append_source(payload, source, state_store)- _secure_write_yaml(path, payload)+ _append_source(payload, source, state_store, generation)+ return yaml.safe_dump(payload, sort_keys=False, default_flow_style=False)-def _append_source(payload: dict[str, Any], source: SourceRecord, store: EngineStateStore) -> None:+def _append_source(+ payload: dict[str, Any],+ source: SourceRecord,+ store: EngineStateStore,+ generation: str = "0",+) -> None:credential = store.credential_metadata(source.credential_ref)if credential["kind"] == "oauth":# OAuth credentials are engine auth files, not YAML credential values.@@ -82,7 +119,11 @@ def _append_source(payload: dict[str, Any], source: SourceRecord, store: EngineSreasoning_by_model = dict(source.model_reasoning_efforts)models = []for model in dict.fromkeys((*source.model_ids, *source.route_model_ids)):- entry: dict[str, Any] = {"name": model, "alias": model}+ entry: dict[str, Any] = {+ "name": model,+ "alias": model,+ "display-name": reload_display_name(model, generation),+ }reasoning_efforts = reasoning_by_model.get(model, ())if reasoning_efforts:# CLIProxyAPI's measured model-registration shape is strongest-first.@@ -145,9 +186,29 @@ def _append_source(payload: dict[str, Any], source: SourceRecord, store: EngineSraise EngineStateError("unsupported source protocol")-def _secure_write_yaml(path: Path, payload: dict[str, Any]) -> None:+def expected_model_names(+ sources: Iterable[SourceRecord],+ store: EngineStateStore,+ generation: str,+) -> dict[str, str]:+ """Routed ID to display name the rendered YAML registers; OAuth models come from auth files."""+ expected: dict[str, str] = {}+ for source in sources:+ metadata = store.credential_metadata_if_present(source.credential_ref)+ if metadata is None or metadata.get("kind") != "api_key":+ continue+ for model in (*source.model_ids, *source.route_model_ids):+ expected[f"{source.prefix}/{model}"] = reload_display_name(model, generation)+ return expected+++def reload_display_name(model: str, generation: str) -> str:+ return f"{model} #{generation}"+++def _secure_write_text(path: Path, text: str) -> None:# The file is 0600 by ``write_atomic``; the directory is this function's own# concern, because the config it holds names an upstream API key.path.parent.mkdir(parents=True, exist_ok=True)path.parent.chmod(0o700)- write_atomic(path, yaml.safe_dump(payload, sort_keys=False, default_flow_style=False))+ write_atomic(path, text)diff --git a/vibe/model_hub_runtime/state.py b/vibe/model_hub_runtime/state.pyindex 148705294..dde02df3e 100644--- a/vibe/model_hub_runtime/state.py+++ b/vibe/model_hub_runtime/state.py@@ -919,8 +919,12 @@ class EngineStateStore:if path.parent in present_directories:self._remove_private_file_if_present(path)- def clear_runtime_configs(self) -> None:- """Remove persisted engine configs after any credential is revoked."""+ def clear_runtime_configs(self, *, keep: Path | None = None) -> None:+ """Remove persisted engine configs after any credential is revoked.++ ``keep`` names the live instance directory whose config was just+ re-rendered without the revoked credential.+ """with self._lock:self._ensure_private_dir(self.root)instances_dir = self.root / "instances"@@ -931,6 +935,8 @@ class EngineStateStore:mode = instance_dir.lstat().st_modeif not stat.S_ISDIR(mode):raise EngineStateError("engine instance directory is unsafe")+ if keep is not None and instance_dir == keep:+ continueconfig_path = instance_dir / "config.yaml"try:config_mode = config_path.lstat().st_modediff --git a/vibe/model_hub_runtime/supervisor.py b/vibe/model_hub_runtime/supervisor.pyindex 85bdcb865..54b456d89 100644--- a/vibe/model_hub_runtime/supervisor.py+++ b/vibe/model_hub_runtime/supervisor.py@@ -30,11 +30,11 @@ from core.process_isolation import (serialize_process_identity,signal_process_tree,)-from vibe.model_hub_runtime.client import EngineClient, EngineConnection-from vibe.model_hub_runtime.config import write_engine_config+from vibe.model_hub_runtime.client import EngineClient, EngineClientError, EngineConnection+from vibe.model_hub_runtime.config import expected_model_names, render_engine_config, write_engine_configfrom vibe.model_hub_runtime.environment import engine_subprocess_environmentfrom vibe.model_hub_runtime.installer import EngineRuntimeManager-from vibe.model_hub_runtime.state import EngineStateStore+from vibe.model_hub_runtime.state import EngineStateStore, SourceRecordlogger = logging.getLogger(__name__)@@ -42,6 +42,8 @@ _T = TypeVar("_T")MODEL_HUB_STARTUP_TIMEOUT_SECONDS = 30.0+MODEL_HUB_CONFIG_RELOAD_TIMEOUT_SECONDS = 5.0+_STARTUP_POLL_INTERVAL_SECONDS = 0.05# Durable identity of the engine this state root last spawned. A service that# dies without running ``atexit`` (SIGKILL, crash, forced restart) leaves its# isolated engine group running; the next start reaps it by identity. The marker@@ -53,7 +55,6 @@ _ENGINE_PROCESS_RECORD = "engine-process.json"class _EngineRecord:worker_fingerprint: stridentity: PersistedProcessIdentity | None = None-_STARTUP_POLL_INTERVAL_SECONDS = 0.05class EngineUnavailableError(RuntimeError):@@ -89,6 +90,7 @@ class EngineSupervisor:self._last_check: str | None = Noneself._start_attempted = Falseself._health_failure_signature: tuple[str, str, int | None] | None = None+ self._config_generation = 0def ensure_running(self) -> EngineConnection:with self._lock:@@ -139,6 +141,105 @@ class EngineSupervisor:if not self._reap_recorded_engines_locked():raise EngineUnavailableError("models.engine.stop_unconfirmed", reason="previous_engine_alive")+ def reload_config_if_running(self, previous_sources: list[SourceRecord] | None = None) -> None:+ """Hot-apply the current source projection to a live engine.++ CPA reloads its config file on change, so a Source or model-list save+ never restarts the process and never waits for in-flight streams. The+ management PUT is used because CPA does not observe an atomic rename of+ the watched file. A stopped engine renders the projection at next start.+ """+ with self._lock:+ if not self._is_running_locked() or self._connection is None:+ # A previous service may have left an engine serving the old+ # projection: a save must not succeed beside it.+ self._require_no_untracked_engine_locked()+ return+ self._reload_locked(previous_sources)++ def _reload_locked(self, previous_sources: list[SourceRecord] | None) -> None:+ assert self._connection is not None+ connection = self._connection+ sources = self.state_store.list_sources()+ generation = self._next_generation_locked()+ text = self._render_config_locked(int(connection.base_url.rsplit(":", 1)[1]), sources, generation)+ client = EngineClient(connection, timeout=MODEL_HUB_CONFIG_RELOAD_TIMEOUT_SECONDS)+ try:+ client.put_config_yaml(text)+ except EngineClientError as exc:+ if exc.status_code is not None:+ raise EngineUnavailableError("models.engine.health_failed") from exc+ if not self._is_running_locked():+ # An exited engine holds no stream; a start renders the+ # current projection.+ self._stop_locked()+ self._start_locked()+ return+ # A lost acknowledgment may still have applied, and a stalled+ # management path does not prove live streams dead. The+ # generation check decides; the process is never killed here.+ expected = expected_model_names(sources, self.state_store, generation)+ stale = frozenset(expected_model_names(previous_sources or (), self.state_store, generation)) - set(expected)+ self._await_models_locked(client, expected, stale)+ self._last_check = _utc_now()++ def _await_models_locked(+ self,+ client: EngineClient,+ expected: dict[str, str],+ stale: frozenset[str],+ ) -> None:+ # CPA applies the written file asynchronously (debounced watcher). Its+ # model listing carrying this generation's display names is the first+ # observable point at which the new projection admits, including a+ # save that leaves every routed ID unchanged. A save that removes+ # models waits for them to leave. With no API-key model before or+ # after, the projection routes nothing either way, so there is nothing+ # to confirm; CPA validated these exact bytes before acknowledging the+ # PUT, and its watcher reloads the same file.+ deadline = time.monotonic() + MODEL_HUB_CONFIG_RELOAD_TIMEOUT_SECONDS+ while True:+ try:+ listed = client.list_model_names(timeout=1.0)+ except EngineClientError:+ listed = None+ if (+ listed is not None+ and all(listed.get(model) == name for model, name in expected.items())+ and not stale & listed.keys()+ ):+ return+ if time.monotonic() >= deadline:+ raise EngineUnavailableError("models.engine.health_failed")+ time.sleep(_STARTUP_POLL_INTERVAL_SECONDS)++ def _next_generation_locked(self) -> str:+ self._config_generation += 1+ return str(self._config_generation)++ def _render_config_locked(self, port: int, sources: list[SourceRecord], generation: str) -> str:+ _managed, _binary, instance_dir, runtime_secrets = self._prepare_instance_locked()+ return render_engine_config(+ instance_dir / "config.yaml",+ host="127.0.0.1",+ port=port,+ auth_dir=self.state_store.auth_dir,+ runtime_secrets=runtime_secrets,+ sources=sources,+ state_store=self.state_store,+ generation=generation,+ )++ def _prepare_instance_locked(self):+ managed = self.installer.status()+ binary = self.installer.resolve_engine_path()+ if binary is None:+ reason = str(managed.get("reason") or "engine_not_installed")+ raise EngineUnavailableError("models.engine.install_failed", reason=reason)+ install_id = Path(str(managed.get("install_dir") or binary.parent)).name+ instance_dir, runtime_secrets = self.state_store.prepare_instance(install_id, rotate=False)+ return managed, binary, instance_dir, runtime_secrets+def note_installation_settled(self) -> None:"""Expose a newly verified binary as lazy-started, not previously down."""with self._lock:@@ -146,19 +247,25 @@ class EngineSupervisor:self._start_attempted = Falsedef invalidate_configs(self) -> None:- """Remove secret-bearing configs and recreate one only for a live engine."""+ """Remove secret-bearing configs without restarting a live engine.++ A revoked credential is already unbound, so a live engine drops it by+ hot-reloading the current projection, which also rewrites its config+ without the secret; streams keep running. An unverifiable reload+ raises instead of restarting, so the caller keeps the revocation+ pending and retries it rather than truncating active streams.+ """with self._lock:- should_restart = self._is_running_locked() and self._healthy_locked()- # Always stop: with no local handle this still reaps a recorded engine a- # previous service left running with the credential being revoked, and+ if self._is_running_locked() and self._connection is not None:+ _managed, _binary, instance_dir, _secrets = self._prepare_instance_locked()+ self.state_store.clear_runtime_configs(keep=instance_dir)+ self._reload_locked(None)+ return+ # With no local handle this still reaps a recorded engine a previous+ # service left running with the credential being revoked, and# revocation must not proceed while that engine may still hold it.self._require_stopped_locked()self.state_store.clear_runtime_configs()- if should_restart:- try:- self._start_locked()- except EngineUnavailableError:- logger.warning("Model Hub engine remains stopped after credential revocation")def status(self) -> dict[str, Any]:with self._lock:@@ -229,16 +336,7 @@ class EngineSupervisor:# start retries the reap, so this clears once the survivor is confirmed gone.if not self._reap_recorded_engines_locked():raise EngineUnavailableError("models.engine.start_failed", reason="previous_engine_alive")- managed = self.installer.status()- binary = self.installer.resolve_engine_path()- if binary is None:- reason = str(managed.get("reason") or "engine_not_installed")- raise EngineUnavailableError("models.engine.install_failed", reason=reason)- install_id = Path(str(managed.get("install_dir") or binary.parent)).name- instance_dir, runtime_secrets = self.state_store.prepare_instance(- install_id,- rotate=False,- )+ managed, binary, instance_dir, runtime_secrets = self._prepare_instance_locked()port = self._port_allocator()config_path = instance_dir / "config.yaml"write_engine_config(@@ -249,6 +347,7 @@ class EngineSupervisor:runtime_secrets=runtime_secrets,sources=self.state_store.list_sources(),state_store=self.state_store,+ generation=self._next_generation_locked(),)connection = EngineConnection(base_url=f"http://127.0.0.1:{port}",