Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

perf: bound useSseProgress frame history and fold job-slot progress in render

[._.]/ Adam Eivy4ec53b663f45
diff --git a/client/src/components/pipeline/AutopilotPanel.jsx b/client/src/components/pipeline/AutopilotPanel.jsx
index 1473b7c737..7bbf03c639 100644
--- a/client/src/components/pipeline/AutopilotPanel.jsx
+++ b/client/src/components/pipeline/AutopilotPanel.jsx
@@ -706,24 +706,14 @@ export default function AutopilotPanel({ series, onSeriesUpdate, onIssuesUpdate
const { latest, frames } = usePipelineProgress(pipelineAutopilotSseUrl, [seriesId], { enabled: active });
- // One backward walk per frame batch yields everything the live section needs:
- // the newest `progress` frame (the milestone map's cursor) and the last few
- // labelled frames (the activity log, whose last line is also the status line).
- // Walking backward and stopping early matters — `frames` is uncapped, so
- // labelling the whole array on every arriving frame is quadratic over a run.
- const { snapshot, activityLines } = useMemo(() => {
- const lines = [];
- let found = null;
- for (let i = frames.length - 1; i >= 0; i -= 1) {
- const f = frames[i];
- if (!found && f?.type === 'progress') found = f;
- if (lines.length < ACTIVITY_LINES) {
- const label = frameLabel(f);
- if (label) lines.unshift(label);
- } else if (found) break;
- }
- return { snapshot: found, activityLines: lines };
- }, [frames]);
+ // `frames` is the hook's bounded history (useSseProgress `maxFrames`), so a
+ // full pass per frame batch stays cheap: the newest `progress` frame is the
+ // milestone map's cursor; the last few labelled frames are the activity log,
+ // whose last line is also the status line.
+ const { snapshot, activityLines } = useMemo(() => ({
+ snapshot: frames.findLast((f) => f?.type === 'progress') ?? null,
+ activityLines: frames.map(frameLabel).filter(Boolean).slice(-ACTIVITY_LINES),
+ }), [frames]);
// Fall back to the status route's snapshot until this stream emits one of its
// own — a panel attaching mid-run has no frames yet.
const progress = snapshot || seedProgress;
diff --git a/client/src/hooks/README.md b/client/src/hooks/README.md
index 1441058f67..95c213da53 100644
--- a/client/src/hooks/README.md
+++ b/client/src/hooks/README.md
@@ -42,7 +42,7 @@ grep -i "what you want to do" client/src/hooks/README.md
|---|---|---|
| `useGoogleAutoConfigProgress` | Request-correlated Google OAuth setup progress with HTTP settlement and listener cleanup. | Calendar and Messages automated setup. |
| `useQuotaUpdates` | Reconcile quota cards on scrape completion, reconnect and tab show; the caller owns the entry read. | Usage, Subscriptions and Quota Burn quota cards. |
-| `useSseProgress` | Generic JSON-frame SSE subscriber: EventSource for GET, one abortable fetch stream for `method: 'POST'` without retry. | New SSE progress stream — start here, build on top. |
+| `useSseProgress` | Generic JSON-frame SSE subscriber: EventSource for GET, one abortable fetch stream for `method: 'POST'` without retry. `frames` is a bounded history (`maxFrames`, default 200; `0` for latest-only consumers). | New SSE progress stream — start here, build on top. |
| `useSseJobSlot` | Generic single-slot SSE job: captured target from preparation through completion, terminal-frame handling (`complete`/`error`/`canceled`), and `sse.closed`-without-terminal-frame recovery in one place. Returns `{ active, pending, jobId, percent, stage, context, start, attach, cancel }` (`attach(jobId, context)` adopts a job another request already started into the idle slot); reserves synchronously, exposes context during kickoff and ignores unmounted responses. Optional `readPercent` adapts other progress scales; `connected` distinguishes a live stream from reconnecting, and `onSettled(reason, context)` refreshes persisted state after completion, failure, cancellation or a terminal disconnect. Pass `startRequest`/`eventsUrl`/`cancelRequest` + per-feature toast copy. | Any "kick off a job → stream SSE → settle on terminal frame" hook. The reference-audio/YouTube/video-download/MIDI/music-video-render hooks all wrap this — don't re-roll the terminal-frame + lost-connection dance. |
| `useInstallStream` | BYO-runtime install-log stream lifecycle: `stage`/`log`/`complete`/`error` frame dispatch, capped + optionally-debounced log accumulation, "connection lost" handling, ref-stashed `onComplete`, auto-scroll. `method: 'GET'` uses EventSource; `method: 'POST'` reads one fetch stream for a non-idempotent install. | A streamed install/setup modal that shows live SSE log lines (FLUX.2, video runtimes, OpenCode). Don't re-roll the teardown/onComplete-ref dance. |
| `useModelDownloadStatus` | Image/video model cache-status + SSE pre-download. `useDownloadableAssets(descriptors, options)` shares readiness and independent integrity dismissals across asset roles. `start(id)` commands a pull; `startWhenIdle(id)` requests one — it waits for the cache verdict, skips an already-resident repo, and queues behind whatever owns the single download lane (`queuedModelId` while it waits, falsy id to retract). | Surfacing "Available" vs "Download" badge inline in the gen form; `startWhenIdle` for a picker where choosing an option IS the request for its weights. |
diff --git a/client/src/hooks/useSseJobSlot.js b/client/src/hooks/useSseJobSlot.js
index f0827f1028..c6b290e911 100644
--- a/client/src/hooks/useSseJobSlot.js
+++ b/client/src/hooks/useSseJobSlot.js
@@ -1,9 +1,10 @@
-import { useEffect, useRef, useState } from 'react';
+import { useEffect, useMemo, useRef, useState } from 'react';
import toast from '../components/ui/Toast';
import { useSseProgress, isTerminalSseFrame } from './useSseProgress.js';
import useMounted from './useMounted.js';
const defaultReadPercent = (frame) => frame.percent;
+const IDLE_PROGRESS = Object.freeze({ percent: 0, stage: null });
/**
* One single-slot SSE job: kickoff, captured target, progress and terminal
@@ -51,12 +52,12 @@ export default function useSseJobSlot({
} = {}) {
// jobId is null during preparation; context already owns the target then.
const [job, setJob] = useState(null);
- const [progress, setProgress] = useState({ percent: 0, stage: null });
const slotRef = useRef(null);
const mounted = useMounted();
useEffect(() => () => { slotRef.current = null; }, []);
const jobUrl = job?.jobId ? eventsUrl(job.jobId) : null;
- const sse = useSseProgress(jobUrl);
+ // Latest-only consumer: no history, so each frame costs one render.
+ const sse = useSseProgress(jobUrl, { maxFrames: 0 });
const latest = sse.latest;
const percent = latest ? readPercent(latest) : undefined;
@@ -65,16 +66,23 @@ export default function useSseJobSlot({
setJob(null);
};
+ // Progress is folded from `latest` during render (carried in a ref) rather
+ // than through a state-setting effect, so a frame costs one render, not two.
// Sparse metadata/warning frames preserve progress. A known mismatched URL
// belongs to an older stream; absent URL identity remains tolerated for the
- // existing SSE test seams.
- useEffect(() => {
- if (!job?.jobId || !latest) return;
- if (sse.latestUrl && sse.latestUrl !== jobUrl) return;
- setProgress((prev) => ({
+ // existing SSE test seams. The fold is idempotent per frame, so StrictMode's
+ // double memo invocation lands on the same value.
+ const progressRef = useRef(IDLE_PROGRESS);
+ const progress = useMemo(() => {
+ if (!job?.jobId || !latest) return progressRef.current;
+ if (sse.latestUrl && sse.latestUrl !== jobUrl) return progressRef.current;
+ const prev = progressRef.current;
+ const next = {
percent: Number.isFinite(percent) ? Math.round(percent) : prev.percent,
stage: typeof latest.stage === 'string' && latest.stage ? latest.stage : prev.stage,
- }));
+ };
+ progressRef.current = next;
+ return next;
}, [latest, job, jobUrl, sse.latestUrl, percent]);
useEffect(() => {
@@ -123,8 +131,8 @@ export default function useSseJobSlot({
if (!mounted.current || slotRef.current) return;
const slot = { jobId: null, context: context === undefined ? arg : context };
slotRef.current = slot;
+ progressRef.current = IDLE_PROGRESS;
setJob(slot);
- setProgress({ percent: 0, stage: null });
const isCurrent = () => mounted.current && slotRef.current === slot;
startRequest(arg)
.then(({ jobId }) => {
@@ -144,8 +152,8 @@ export default function useSseJobSlot({
if (!jobId || !mounted.current || slotRef.current) return false;
const slot = { jobId, context: context ?? null };
slotRef.current = slot;
+ progressRef.current = IDLE_PROGRESS;
setJob(slot);
- setProgress({ percent: 0, stage: null });
return true;
};
diff --git a/client/src/hooks/useSseJobSlot.test.jsx b/client/src/hooks/useSseJobSlot.test.jsx
index 7761d4b0fb..09be1bc0dd 100644
--- a/client/src/hooks/useSseJobSlot.test.jsx
+++ b/client/src/hooks/useSseJobSlot.test.jsx
@@ -53,6 +53,25 @@ it('reserves one slot synchronously and keeps default progress/terminal behavior
expect(onComplete).toHaveBeenCalledTimes(1);
});
+it('re-renders exactly once per progress frame and keeps no frame history', async () => {
+ let renders = 0;
+ const { result } = renderHook(() => {
+ renders += 1;
+ return useSseJobSlot({ eventsUrl: (id) => `/events/${id}` });
+ });
+ act(() => { result.current.attach('job-1', 'target'); });
+ const before = renders;
+ act(() => { lastEventSource().emit({ type: 'progress', percent: 10, stage: 'a' }); });
+ expect(renders - before).toBe(1);
+ const mid = renders;
+ // One act per frame: frames arrive as separate browser events, never batched.
+ for (let i = 0; i < 100; i += 1) {
+ act(() => { lastEventSource().emit({ type: 'progress', percent: i, stage: 'b' }); });
+ }
+ expect(renders - mid).toBe(100);
+ expect(result.current).toMatchObject({ percent: 99, stage: 'b' });
+});
+
describe('unmounted kickoff ownership', () => {
it.each(['resolve', 'reject'])('ignores a late %s and refuses starts through an unmounted callback', async (outcome) => {
let resolveKickoff;
diff --git a/client/src/hooks/useSseProgress.js b/client/src/hooks/useSseProgress.js
index d2ad25b80f..cbb54e5235 100644
--- a/client/src/hooks/useSseProgress.js
+++ b/client/src/hooks/useSseProgress.js
@@ -13,6 +13,17 @@ const TERMINAL_TYPES = new Set(['complete', 'canceled', 'cancelled', 'error', 'p
/** True when a frame is one of the stream-ending types this hook closes on. */
export const isTerminalSseFrame = (frame) => TERMINAL_TYPES.has(frame?.type);
+export const DEFAULT_MAX_SSE_FRAMES = 200;
+
+// One shared empty history so a `maxFrames: 0` consumer's `frames` keeps a
+// stable identity (every setFrames(EMPTY_FRAMES) bails out as a no-op).
+const EMPTY_FRAMES = Object.freeze([]);
+
+/** Append a frame, dropping the oldest ones so at most `max` are retained. */
+export const appendBoundedFrame = (frames, frame, max) => (
+ frames.length < max ? [...frames, frame] : [...frames.slice(frames.length - max + 1), frame]
+);
+
/**
* GET uses EventSource reconnects; POST reads one fetch stream without retry.
* Subscribe to a server-side SSE stream of JSON-payload progress
@@ -23,9 +34,15 @@ export const isTerminalSseFrame = (frame) => TERMINAL_TYPES.has(frame?.type);
* The stream is torn down on unmount, when `url` changes, when `enabled`
* flips false, or when a terminal frame (type === 'complete' / 'canceled'
* / 'error') arrives.
+ *
+ * `frames` is a bounded history: at most `maxFrames` (default 200) are
+ * retained, the oldest dropped first, so a multi-hour run cannot grow memory
+ * or make each append O(run length). `latest` is its own cell — a consumer
+ * that only needs the newest frame passes `maxFrames: 0` and the history is
+ * never written, so it pays one state update (one render) per frame.
*/
-export function useSseProgress(url, { enabled = true, method = 'GET' } = {}) {
- const [frames, setFrames] = useState([]);
+export function useSseProgress(url, { enabled = true, method = 'GET', maxFrames = DEFAULT_MAX_SSE_FRAMES } = {}) {
+ const [frames, setFrames] = useState(EMPTY_FRAMES);
const [latest, setLatest] = useState(null);
const [latestUrl, setLatestUrl] = useState(null);
const [isOpen, setIsOpen] = useState(false);
@@ -36,6 +53,16 @@ export function useSseProgress(url, { enabled = true, method = 'GET' } = {}) {
// that will never arrive. Reset to false on every (re)subscribe.
const [closed, setClosed] = useState(false);
const esRef = useRef(null);
+ // Read through a ref inside the stream handlers so a changed cap neither
+ // resubscribes nor is ignored.
+ const maxFramesRef = useRef(maxFrames);
+ useEffect(() => { maxFramesRef.current = maxFrames; }, [maxFrames]);
+ const record = (data, streamUrl) => {
+ const max = maxFramesRef.current;
+ if (max > 0) setFrames(prev => appendBoundedFrame(prev, data, max));
+ setLatest(data);
+ setLatestUrl(streamUrl);
+ };
useEffect(() => {
if (!url || !enabled) {
@@ -48,7 +75,7 @@ export function useSseProgress(url, { enabled = true, method = 'GET' } = {}) {
setLatestUrl(null);
return undefined;
}
- setFrames([]);
+ setFrames(EMPTY_FRAMES);
setLatest(null);
setLatestUrl(null);
setIsOpen(false);
@@ -63,9 +90,7 @@ export function useSseProgress(url, { enabled = true, method = 'GET' } = {}) {
const isCurrent = () => esRef.current === connection && !controller.signal.aborted;
const receive = (data) => {
if (!isCurrent()) return;
- setFrames(previous => [...previous, data]);
- setLatest(data);
- setLatestUrl(url);
+ record(data, url);
if (isTerminalSseFrame(data)) {
setIsOpen(false);
setClosed(true);
@@ -133,9 +158,7 @@ export function useSseProgress(url, { enabled = true, method = 'GET' } = {}) {
} catch {
return;
}
- setFrames((prev) => [...prev, data]);
- setLatest(data);
- setLatestUrl(url);
+ record(data, url);
if (TERMINAL_TYPES.has(data?.type)) {
es.close();
setIsOpen(false); // the connection is now closed — keep isOpen honest
diff --git a/client/src/hooks/useSseProgress.test.jsx b/client/src/hooks/useSseProgress.test.jsx
index 70ab1d9a85..9033fe0b0a 100644
--- a/client/src/hooks/useSseProgress.test.jsx
+++ b/client/src/hooks/useSseProgress.test.jsx
@@ -80,6 +80,36 @@ describe('useSseProgress', () => {
expect(result.current.latestUrl).toBeNull();
});
+ it('retains at most maxFrames of history, dropping the oldest, while latest is the last frame', () => {
+ const { result } = renderHook(() => useSseProgress('/x', { maxFrames: 50 }));
+ act(() => {
+ for (let i = 1; i <= 1000; i += 1) last().emit({ type: 'progress', i });
+ });
+ expect(result.current.frames).toHaveLength(50);
+ expect(result.current.frames[0]).toEqual({ type: 'progress', i: 951 });
+ expect(result.current.frames.at(-1)).toEqual({ type: 'progress', i: 1000 });
+ expect(result.current.latest).toEqual({ type: 'progress', i: 1000 });
+ });
+
+ it('defaults the cap to 200 and never writes history for a maxFrames: 0 consumer', () => {
+ const capped = renderHook(() => useSseProgress('/x'));
+ act(() => {
+ for (let i = 0; i < 250; i += 1) last().emit({ type: 'progress', i });
+ });
+ expect(capped.result.current.frames).toHaveLength(200);
+
+ const latestOnly = renderHook(() => useSseProgress('/y', { maxFrames: 0 }));
+ const initialFrames = latestOnly.result.current.frames;
+ act(() => {
+ last().emit({ type: 'progress', i: 1 });
+ last().emit({ type: 'complete' });
+ });
+ expect(latestOnly.result.current.frames).toBe(initialFrames);
+ expect(latestOnly.result.current.frames).toHaveLength(0);
+ expect(latestOnly.result.current.latest).toEqual({ type: 'complete' });
+ expect(latestOnly.result.current.closed).toBe(true);
+ });
+
it('isTerminalSseFrame matches the terminal set and rejects others', () => {
expect(isTerminalSseFrame({ type: 'complete' })).toBe(true);
expect(isTerminalSseFrame({ type: 'canceled' })).toBe(true);