Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.
Python agent-orchestrationaiai-agentschatopsdiscordlangchainmcpmicrosoft-teamsmulti-agentopen-sourceopenaiself-hostedslack
Optimize garbage collection, so full passes stop stalling the main loop (#712)
Abel Dantas7dfa3e7f251f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 2199b6e26..97052e097 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -136,6 +136,13 @@ version of their own to them without also giving them a release of their own.Existing duplicates are left as they are and can be reached by id.#### Fixed+- **Full garbage collections no longer stall the server for hundreds of+ milliseconds.** Once boot has built what lasts for the life of the process+ (modules, the ORM's mappers, the app's routes), the server freezes it out of+ garbage collection, so a full pass walks only what was made since. The+ clients and bridges started after that stay collectable, since replicas hand+ them to one another. The dashboard's pause chart now shows the average pause+ next to p99, as Datadog keeps the average for longer.- **Agents no longer read offline while their replica shuts down.** With severalreplicas, a stopping replica released its lease before its 5 s drain, so forup to about 6 s the other replicas counted its connected agents as gone: `@role`diff --git a/core/switch_core/main.py b/core/switch_core/main.pyindex 6eff94617..9d17aa22b 100644--- a/core/switch_core/main.py+++ b/core/switch_core/main.py@@ -1,6 +1,7 @@from __future__ import annotationsimport asyncio+import gcimport hashlibimport loggingimport os@@ -1457,6 +1458,11 @@ async def run(config: SwitchConfig, process_id: str) -> None:agent_bridge_app.router.lifespan_context = lifespan # type: ignore[assignment]+ # Before the runtime starts: what follows is the clients, bridges and work+ # this process owns, which ownership hands to another replica as replicas+ # come and go, so it must stay collectable.+ _freeze_boot_objects()+# ── Start runtime ────────────────────────────────────────────────────────await client_lifecycle.start_all()# Before the bridges start, so none on a shared app runs without the@@ -1594,6 +1600,32 @@ async def run(config: SwitchConfig, process_id: str) -> None:await _begin_shutdown.finish()+def _freeze_boot_objects() -> None:+ """Take everything boot created out of the garbage collector's way.++ A full collection walks every tracked object while holding the interpreter+ lock, and most of the heap is what boot built and keeps for the life of the+ process: modules, the ORM's mappers, the app's routes. Walking that once a+ minute is what stalls the event loop for hundreds of milliseconds.+ `gc.freeze` moves it to a permanent generation the collector skips, so+ full collections only walk what was allocated since.++ Called before the runtime starts, not once it is serving. A frozen object+ is still freed when its last reference goes, but one caught in a reference+ cycle never is, and the clients and bridges a process runs are full of+ cycles (tasks, callbacks, sessions). With several replicas, ownership moves+ that work between processes whenever one joins or leaves, so freezing it+ would keep every client and bridge this process started at boot alive+ after it hands them over.+ """+ gc.collect()+ gc.freeze()+ logger.info(+ "Froze %d objects created during boot out of garbage collection.",+ gc.get_freeze_count(),+ )++async def _seed_admin_and_bootstrap_key(session_factory: async_sessionmaker[AsyncSession],user_store: UserStore,diff --git a/deploy/observability/dashboard.json b/deploy/observability/dashboard.jsonindex 41eeb3dfb..def5e4e20 100644--- a/deploy/observability/dashboard.json+++ b/deploy/observability/dashboard.json@@ -562,7 +562,7 @@{"definition": {"type": "timeseries",- "title": "Garbage-collection pause p99 / max (ms) by generation. Needs distribution mapping, see README",+ "title": "Garbage-collection pause p99 / avg (ms) by generation. Needs distribution mapping, see README","show_legend": true,"requests": [{@@ -570,7 +570,7 @@"display_type": "line"},{- "q": "max:switch.runtime.gc_pause{$env,$service} by {generation}",+ "q": "avg:switch.runtime.gc_pause{$env,$service} by {generation}","display_type": "bars"}]