Skip to content

#1 Persistent memory for AI coding agents based on real-world benchmarks

fix(health): measure heap pressure against the V8 heap limit (#1409)

Rohit Ghumare8deb131f6e76
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 1e5b3bf3..5d5c5985 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -23,6 +23,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
### Fixed
+- **The "Memory elevated" alert no longer fires on a healthy process (#1406).** The health check divided the heap in use by the heap Node had allocated so far, which Node keeps just above what is in use, so a normal process read 90% or more and raised the alert once its memory passed 512 MB. The check now divides by the V8 heap limit, which is about 4 GB on 64-bit Node by default, and the viewer's Heap gauge shows used against that limit. The reported 589 of 629 MB is 14% of the limit and no longer alerts.
- **The viewer updates from the live stream instead of re-fetching, and stops blinking.** The dashboard used to wipe itself to "Loading dashboard..." and re-fetch everything on a 30 s timer and on every stream event; on an idle store that was 73 rebuilds and 20 requests a minute, and 21 live events cost 200 requests. The worker now pushes `session.updated`, `session.deleted`, `memory.updated` and `memory.deleted` on the `mem-live` viewer group next to the existing observation and `session.activity` events (which the viewer used to drop because it read the wrong field), and the viewer applies them to what is on screen: sessions, observation counts, session status and the memory total update with zero requests, redraws are batched to at most one every 250 ms and skipped when nothing changed. The 30 s timer is gone, no refresh path blanks a loaded view, and polling remains only as the fallback when the stream cannot connect (#921, #1340). The viewer now connects with the stream mode engine 0.22 serves first instead of failing twice on the other one. Live buffers on the Timeline and Activity tabs are capped at a few hundred entries so a large stream backlog cannot freeze the tab (#609). When the stream reconnects, the loaded views re-fetch once so changes made while it was down show up, and a list that was loading when live events arrived replays them, so a deleted session does not come back. Session counts no longer reload or blank the open session detail, bursts of memory events coalesce into one list reload, and with an isolated agent scope (`AGENT_ID` plus `AGENTMEMORY_AGENT_SCOPE=isolated`) the worker only pushes that agent's sessions and memories.
- **A failed dashboard refresh keeps the last data.** Panels that fail to refresh keep their previous values behind a "could not refresh" note instead of flipping to the new-install screen; a first load that cannot reach the API shows a retry button (#1340).
- **Dashboard numbers match the API.** The Memories card uses the server total instead of counting a capped list (#1279, #951), token savings only counts sessions that produced observations (#944), and the Graph Nodes card shows when its snapshot was taken.
diff --git a/src/health/monitor.ts b/src/health/monitor.ts
index 633c28b2..35a9de9e 100644
--- a/src/health/monitor.ts
+++ b/src/health/monitor.ts
@@ -1,3 +1,4 @@
+import { getHeapStatistics } from "node:v8";
import type { IIIClient } from "iii-sdk";
import type { HealthSnapshot } from "../types.js";
import type { StateKV } from "../state/kv.js";
@@ -69,6 +70,7 @@ export function registerHealthMonitor(
memory: {
heapUsed: mem.heapUsed,
heapTotal: mem.heapTotal,
+ heapLimit: getHeapStatistics().heap_size_limit,
rss: mem.rss,
external: mem.external,
},
diff --git a/src/health/thresholds.ts b/src/health/thresholds.ts
index 7279afad..9abf35ec 100644
--- a/src/health/thresholds.ts
+++ b/src/health/thresholds.ts
@@ -59,10 +59,9 @@ export function evaluateHealth(
degraded = true;
}
+ const heapCeiling = snapshot.memory.heapLimit || snapshot.memory.heapTotal;
const memPercent =
- snapshot.memory.heapTotal > 0
- ? (snapshot.memory.heapUsed / snapshot.memory.heapTotal) * 100
- : 0;
+ heapCeiling > 0 ? (snapshot.memory.heapUsed / heapCeiling) * 100 : 0;
const rss = snapshot.memory.rss ?? 0;
const rssAboveFloor = rss >= cfg.memoryRssFloorBytes;
const memMb = Math.round(rss / (1024 * 1024));
diff --git a/src/types.ts b/src/types.ts
index d2c63efa..38b385f0 100644
--- a/src/types.ts
+++ b/src/types.ts
@@ -225,6 +225,7 @@ export interface HealthSnapshot {
memory: {
heapUsed: number;
heapTotal: number;
+ heapLimit?: number;
rss: number;
external: number;
};
diff --git a/src/viewer/index.html b/src/viewer/index.html
index 38e25649..ea63a973 100644
--- a/src/viewer/index.html
+++ b/src/viewer/index.html
@@ -1212,9 +1212,9 @@
function humanizeHealthFlag(f) {
var m;
if ((m = /^memory_heap_tight_(\d+)%_rss(\d+)mb$/.exec(f)))
- return 'Heap is running tight: ' + m[1] + '% of allocated heap in use (process memory ' + m[2] + ' MB). Informational — Node grows the heap on demand.';
+ return 'Heap is running tight: ' + m[1] + '% of the heap limit in use (process memory ' + m[2] + ' MB).';
if ((m = /^memory_(warn|critical)_(\d+)%_rss(\d+)mb$/.exec(f)))
- return 'Memory ' + (m[1] === 'critical' ? 'critically high' : 'elevated') + ': ' + m[2] + '% of heap in use, process memory ' + m[3] + ' MB.';
+ return 'Memory ' + (m[1] === 'critical' ? 'critically high' : 'elevated') + ': ' + m[2] + '% of the heap limit in use, process memory ' + m[3] + ' MB.';
if ((m = /^cpu_(warn|critical)_(\d+)%$/.exec(f)))
return 'CPU ' + (m[1] === 'critical' ? 'critically high' : 'elevated') + ': ' + m[2] + '%.';
if ((m = /^event_loop_lag_(warn|critical)_(\d+)ms$/.exec(f)))
@@ -1720,12 +1720,12 @@
html += '<div class="card" style="margin-bottom:16px"><div class="card-title">System Resources</div>';
if (snap.memory) {
var heapUsed = Math.round((snap.memory.heapUsed || 0) / 1024 / 1024);
- var heapTotal = Math.round((snap.memory.heapTotal || 0) / 1024 / 1024);
+ var heapLimit = Math.round((snap.memory.heapLimit || snap.memory.heapTotal || 0) / 1024 / 1024);
var rss = Math.round((snap.memory.rss || 0) / 1024 / 1024);
- var heapPct = heapTotal > 0 ? Math.round((heapUsed / heapTotal) * 100) : 0;
+ var heapPct = heapLimit > 0 ? Math.round((heapUsed / heapLimit) * 100) : 0;
var rssAboveFloor = rss >= 512;
var heapColor = (heapPct > 80 && rssAboveFloor) ? 'var(--red)' : (heapPct > 60 && rssAboveFloor) ? 'var(--yellow)' : 'var(--green)';
- html += '<div class="gauge"><span class="gauge-label">Heap</span><div class="gauge-bar"><div class="gauge-fill" style="width:' + heapPct + '%;background:' + heapColor + '"></div></div><span class="gauge-value">' + heapUsed + ' / ' + heapTotal + ' MB</span></div>';
+ html += '<div class="gauge"><span class="gauge-label">Heap</span><div class="gauge-bar"><div class="gauge-fill" style="width:' + heapPct + '%;background:' + heapColor + '"></div></div><span class="gauge-value">' + heapUsed + ' / ' + heapLimit + ' MB</span></div>';
html += '<div class="gauge"><span class="gauge-label">RSS</span><div class="gauge-bar"><div class="gauge-fill" style="width:' + Math.min(100, Math.round(rss / 512 * 100)) + '%;background:var(--blue)"></div></div><span class="gauge-value">' + rss + ' MB</span></div>';
if (snap.memory.external) {
var ext = Math.round(snap.memory.external / 1024 / 1024);
diff --git a/test/health-thresholds.test.ts b/test/health-thresholds.test.ts
index 6f918a91..bdbe5d53 100644
--- a/test/health-thresholds.test.ts
+++ b/test/health-thresholds.test.ts
@@ -35,6 +35,22 @@ describe("evaluateHealth memory severity", () => {
expect(notes.find((n) => n.startsWith("memory_heap_tight_"))).toBeDefined();
});
+ it("measures heap against the V8 limit, not the allocated heap (#1406)", () => {
+ const mb = 1024 * 1024;
+ const reported = snap({
+ memory: { heapUsed: 589 * mb, heapTotal: 629 * mb, heapLimit: 4144 * mb, rss: 923 * mb, external: 0 },
+ });
+ const { status, alerts, notes } = evaluateHealth(reported);
+ expect(status).toBe("healthy");
+ expect(alerts.some((a) => a.startsWith("memory_"))).toBe(false);
+ expect(notes.some((n) => n.startsWith("memory_"))).toBe(false);
+
+ const nearLimit = snap({
+ memory: { heapUsed: 4000 * mb, heapTotal: 4100 * mb, heapLimit: 4144 * mb, rss: 4300 * mb, external: 0 },
+ });
+ expect(evaluateHealth(nearLimit).alerts.some((a) => a.startsWith("memory_critical_97%"))).toBe(true);
+ });
+
it("goes critical when heap ratio is high AND RSS is above the floor", () => {
const s = snap({
memory: {