Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix([issue-11659]): preserve voice connections for supported media sizes (#11692)
[._.]/68462350bc6f
diff --git a/client/src/lib/README.md b/client/src/lib/README.mdindex d55c568cbc..2be60f2837 100644--- a/client/src/lib/README.md+++ b/client/src/lib/README.md@@ -153,7 +153,7 @@ grep -i "what you want to do" client/src/lib/README.md| `avatarStyles.js` | Re-export of `server/lib/avatarStyles.js` — the CoS avatar-style vocabulary (#6253): `AVATAR_STYLES`, `AVATAR_STYLE_IDS`, `AVATAR_STYLE_LABELS`, `WEBGL_AVATAR_STYLE_IDS`. The picker labels, the lazy-load map / WebGL-stage set, and the server's `avatarStyle` zod enum all read the one list. || `assessmentTuningNotice.js` | `tuningNoticeChip(entry)` — the short form of how a measured local-model assessment says its launch configuration did NOT take effect. `tuningApplied === false` covers two opposite cases and `tuningKey` separates them: a TUNED run whose knobs never reached the daemon (`tuning not applied`), versus an UNTUNED run PortOS could not put back on backend defaults (`not at defaults`) — #4759. Shared by `ModelThroughputReport.jsx`'s table cell and `LocalModelAssessments.jsx`'s result toast; the long form is the server's exclusion reason in `getAssessmentReport`. || `callAudioBridge.js` | Pure device/format helpers for the call-host page (`pages/VoiceCallHost.jsx`, #5308 + #5311) — shared by its Call mode (FaceTime Audio) and its Capture system audio mode (meeting transcription). `missingCallHostApis(scope)` lists every required browser API this browser lacks (MediaStreamTrackProcessor/Generator, AudioWorklet, `enumerateDevices`, `setSinkId`, Web Locks) so the page names all the gaps at once instead of one per reload; `findAudioDevice(devices, label, kind)` matches a device label EXACTLY after trim+case-fold (a substring match would resolve "BlackHole 1" to the 16ch device, indistinguishable from a dead call); `describeDeviceProblem(devices, {inputLabel, outputLabel})` returns the one blocking message or `null`, and deliberately blames MICROPHONE PERMISSION rather than a missing driver when every label comes back blank — an ungranted tab enumerates unlabeled devices, and "reinstall BlackHole" would send the user to fix the wrong thing; `outputLabel` is OPTIONAL — capture mode only listens, so omitting it skips the output-device check entirely rather than reporting a device named "undefined" as missing. Format side: `downmixToMono(channels)` (BlackHole 16ch carries the call on all channels; truncates to the shortest rather than reading past one), `resampleTo16k(samples, fromRate)` (linear, 48k→16k for whisper), `floatToInt16` (CLAMPED — wrapping would flip a hot sample's sign into an audible click), `rmsLevel` for the input meter, and `buildTestTone` (faded in/out so it cannot click through the call). Plus `CALL_HOST_LOCK` (the Web Lock name that stops a second tab double-answering — shared by both modes, since they read the same physical device) and `CALL_FRAME_SAMPLES`. |-| `audioRecorder.js` | `startMemoRecording()` → `{ stream, stop, cancel }` handle whose `stop()` resolves to `{ audioBase64, mimeType, peak, durationMs }` (16 kHz mono WAV, base64); the live `stream` lets a tuner tap the same mic. `createStreamAnalyser(stream, { fftSize })` → `{ analyser, context, close }` taps an `AnalyserNode` off an existing stream (no second getUserMedia). Plus `blobToWav16k` (decode a recorded blob → `{ wav, peak }`), `float32ToWav16k(samples, sourceRate)` (resample a raw Float32 PCM buffer, e.g. an AudioWorklet capture, → `{ wav, peak }`; `{ wav: null, peak: 0 }` for empty input), `encodePcmToWav`, `TARGET_SAMPLE_RATE` (16000), `pickRecordingMimeType`, `arrayBufferToBase64`. Shared WAV-encode/resample path for both this standalone memo-capture flow AND the live voice-agent recorder in `services/voiceClient.js`. **`openAnalysisMic({ getUserMedia })`** → `{ stream, processing }` is the ONE way a pitch-analysis surface opens a mic: it requests `ANALYSIS_AUDIO_CONSTRAINTS` (echo cancellation / noise suppression / AGC off — plain booleans, never `{ exact }`, so a browser that can't honor one still opens the mic) and reads `track.getSettings()` back so the UI can say what actually stuck. `readAppliedProcessing(stream)` reports each stage as `true` / `false` / `null` (browser didn't say — never collapse unknown into off) and `hasUnwantedProcessing(p)` gates the warning. Memo capture deliberately keeps the speech-tuned defaults; Whisper wants them. |+| `audioRecorder.js` | `startMemoRecording()` → `{ stream, stop, cancel }` handle whose `stop()` resolves to `{ audioBase64, mimeType, peak, durationMs }` (16 kHz mono WAV, base64); the live `stream` lets a tuner tap the same mic. `createStreamAnalyser(stream, { fftSize })` → `{ analyser, context, close }` taps an `AnalyserNode` off an existing stream (no second getUserMedia). Plus `blobToWav16k` (decode a recorded blob → `{ wav, peak }`), `float32ToWav16k(samples, sourceRate)` (resample a raw Float32 PCM buffer, e.g. an AudioWorklet capture, → `{ wav, peak }`; `{ wav: null, peak: 0 }` for empty input), `encodePcmToWav`, `MAX_VOICE_AUDIO_BYTES` (8 MiB), `RECORDING_TOO_LONG_MESSAGE`, `TARGET_SAMPLE_RATE` (16000), `pickRecordingMimeType`, `arrayBufferToBase64`. Shared WAV-encode/resample path for both this standalone memo-capture flow AND the live voice-agent recorder in `services/voiceClient.js`. **`openAnalysisMic({ getUserMedia })`** → `{ stream, processing }` is the ONE way a pitch-analysis surface opens a mic: it requests `ANALYSIS_AUDIO_CONSTRAINTS` (echo cancellation / noise suppression / AGC off — plain booleans, never `{ exact }`, so a browser that can't honor one still opens the mic) and reads `track.getSettings()` back so the UI can say what actually stuck. `readAppliedProcessing(stream)` reports each stage as `true` / `false` / `null` (browser didn't say — never collapse unknown into off) and `hasUnwantedProcessing(p)` gates the warning. Memo capture deliberately keeps the speech-tuned defaults; Whisper wants them. || `bareUrl.js` | `parseBareUrl(text)` — the normalized URL when a capture's entire text is a URL, else null. Re-exported from `server/lib/bareUrl.js`, the rule the brain-capture route applies. || `youtubeUrl.js` | `isYoutubeVideoUrl(text)` / `youtubeVideoId(url)` re-exported from `server/lib/youtubeUrl.js`, so Quick Capture can only offer the ingest path for a URL the server accepts. Also the client-only `INGEST_OPTIONS` — the single table of the three ingest artifacts (`key` / `settingKey` / `fallback` / `label` / `hint`) — plus `defaultIngestOptions()` and `ingestOptionsFromSettings(settings)`. Everything that enumerates the switches derives from that table; add an artifact there, not at each call site. || `boundedMap.js` | `evictOldest(map, max, onEvict?)` — evict oldest-first (`map.keys().next().value`) until a Map is within `max`, with an optional per-evicted-key teardown callback for a companion timer map; `ORPHAN_BUFFER_MAX` (64), the shared cap for the orphan-event race-buffers. The one home for the "stash an unmatched event in a `jobId → …` Map, evict the oldest on overflow" idiom shared by `useSceneRenderLifecycle`, `useImageGenQueue`, and `clientErrorReporter`. |diff --git a/client/src/lib/audioRecorder.js b/client/src/lib/audioRecorder.jsindex d0b03c174a..76cdacd0eb 100644--- a/client/src/lib/audioRecorder.js+++ b/client/src/lib/audioRecorder.js@@ -9,8 +9,12 @@// `blobToWav16k`, `float32ToWav16k`) ARE shared with that module — it imports// them rather than keeping its own copy.+import { SOCKET_MEDIA_MAX_BYTES } from '../../../server/lib/socketPayloadLimits.js';import { resumeAudioContext, acquireAudioSession } from './audioContext.js';+export const MAX_VOICE_AUDIO_BYTES = SOCKET_MEDIA_MAX_BYTES;+export const RECORDING_TOO_LONG_MESSAGE = 'Recording too long. Please record a shorter message.';+export const TARGET_SAMPLE_RATE = 16000;// Pick a MediaRecorder mime the browser supports; Safari lands on mp4, othersdiff --git a/client/src/pages/FableLoomHostedJoin.jsx b/client/src/pages/FableLoomHostedJoin.jsxindex 6af6458dfc..ee48390a31 100644--- a/client/src/pages/FableLoomHostedJoin.jsx+++ b/client/src/pages/FableLoomHostedJoin.jsx@@ -8,6 +8,7 @@import { useEffect, useRef, useState } from 'react';import { io } from 'socket.io-client';+import { MAX_VOICE_AUDIO_BYTES, RECORDING_TOO_LONG_MESSAGE } from '../lib/audioRecorder.js';import useMounted from '../hooks/useMounted.js';import {AlertCircle,@@ -28,10 +29,13 @@ export default function FableLoomHostedJoin() {const [transcript, setTranscript] = useState([]);const [textInput, setTextInput] = useState('');const [isRecording, setIsRecording] = useState(false);+ const [recordingError, setRecordingError] = useState(null);const mediaRecorderRef = useRef(null);const pressedRef = useRef(false);const startGenRef = useRef(0);+ // Release preserves final recorder callbacks; a newer press retires them.+ const recordingGenRef = useRef(0);const mountedRef = useMounted();const transcriptScrollRef = useRef(null);const audioPlayerRef = useRef(null);@@ -128,8 +132,10 @@ export default function FableLoomHostedJoin() {// Microphone recording controlsconst startRecording = async () => {if (pressedRef.current || mediaRecorderRef.current || turnPhase === 'thinking' || turnPhase === 'speaking' || turnPhase === 'ended') return;+ setRecordingError(null);pressedRef.current = true;const generation = ++startGenRef.current;+ const recordingGeneration = ++recordingGenRef.current;let stream;try {stream = await navigator.mediaDevices.getUserMedia({ audio: true });@@ -155,13 +161,21 @@ export default function FableLoomHostedJoin() {};recorder.onstop = () => {+ stopTracks();+ if (!mountedRef.current || recordingGeneration !== recordingGenRef.current) return;const audioBlob = new Blob(chunks, { type: recorder.mimeType || 'audio/webm' });+ if (audioBlob.size > MAX_VOICE_AUDIO_BYTES) {+ if (mountedRef.current) {+ setRecordingError(RECORDING_TOO_LONG_MESSAGE);+ socket?.emit('hosted:turn:abort');+ }+ return;+ }audioBlob.arrayBuffer().then((buf) => {- if (mountedRef.current && socket) {+ if (mountedRef.current && recordingGeneration === recordingGenRef.current && socket) {socket.emit('hosted:mic:stop', new Uint8Array(buf));}});- stopTracks();};recorder.start(100);@@ -190,6 +204,7 @@ export default function FableLoomHostedJoin() {};useEffect(() => () => {+ ++recordingGenRef.current;pressedRef.current = false;++startGenRef.current;const active = mediaRecorderRef.current;@@ -222,6 +237,7 @@ export default function FableLoomHostedJoin() {return (<main className="h-dvh-screen bg-slate-950 text-white flex flex-col justify-between max-w-md mx-auto relative overflow-y-auto shadow-2xl">+ {recordingError && <p role="alert" className="px-4 text-sm text-red-400">{recordingError}</p>}<audio ref={audioPlayerRef} className="hidden" />{/* Header */}diff --git a/client/src/pages/FableLoomHostedJoin.test.jsx b/client/src/pages/FableLoomHostedJoin.test.jsxindex 3a30943132..55709f7d04 100644--- a/client/src/pages/FableLoomHostedJoin.test.jsx+++ b/client/src/pages/FableLoomHostedJoin.test.jsx@@ -158,6 +158,73 @@ describe('FableLoomHostedJoin', () => {vi.unstubAllGlobals();});+ it('shows an over-cap recording refusal, aborts the turn, and keeps the guest connected', async () => {+ const { MAX_VOICE_AUDIO_BYTES, RECORDING_TOO_LONG_MESSAGE } = await import('../lib/audioRecorder.js');+ window.location.hash = '#session=sess-123&token=tok-abc';+ const track = { stop: vi.fn() };+ Object.defineProperty(navigator, 'mediaDevices', {+ configurable: true,+ value: { getUserMedia: vi.fn().mockResolvedValue({ getTracks: () => [track] }) },+ });+ let recorder;+ vi.stubGlobal('MediaRecorder', class {+ constructor() { this.state = 'inactive'; this.mimeType = 'audio/webm'; recorder = this; }+ start() { this.state = 'recording'; }+ stop() { this.state = 'inactive'; this.onstop?.(); }+ });+ try {+ render(<FableLoomHostedJoin />);+ const button = screen.getByRole('button', { name: /hold talk/i });+ fireEvent.pointerDown(button);+ await waitFor(() => expect(recorder?.state).toBe('recording'));+ recorder.ondataavailable({ data: new Blob([new Uint8Array(MAX_VOICE_AUDIO_BYTES + 1)]) });+ fireEvent.pointerUp(button);+ expect(screen.getByRole('alert')).toHaveTextContent(RECORDING_TOO_LONG_MESSAGE);+ expect(mockSocket.emit).not.toHaveBeenCalledWith('hosted:mic:stop', expect.anything());+ expect(mockSocket.emit).toHaveBeenCalledWith('hosted:turn:abort');+ expect(mockSocket.disconnect).not.toHaveBeenCalled();+ expect(track.stop).toHaveBeenCalledOnce();+ } finally {+ vi.unstubAllGlobals();+ }+ });++ it('ignores an old oversized stop callback after a newer recording starts', async () => {+ const { MAX_VOICE_AUDIO_BYTES } = await import('../lib/audioRecorder.js');+ window.location.hash = '#session=sess-123&token=tok-abc';+ Object.defineProperty(navigator, 'mediaDevices', {+ configurable: true,+ value: { getUserMedia: vi.fn(async () => ({ getTracks: () => [{ stop: vi.fn() }] })) },+ });+ const recorders = [];+ vi.stubGlobal('MediaRecorder', class {+ constructor() { this.state = 'inactive'; this.mimeType = 'audio/webm'; recorders.push(this); }+ start() { this.state = 'recording'; }+ stop() { this.state = 'inactive'; }+ });+ try {+ render(<FableLoomHostedJoin />);+ const button = screen.getByRole('button', { name: /hold talk/i });+ fireEvent.pointerDown(button);+ await waitFor(() => expect(recorders).toHaveLength(1));+ recorders[0].ondataavailable({ data: new Blob([new Uint8Array(MAX_VOICE_AUDIO_BYTES + 1)]) });+ fireEvent.pointerUp(button);+ fireEvent.pointerDown(button);+ await waitFor(() => expect(recorders).toHaveLength(2));+ act(() => recorders[0].onstop());+ expect(mockSocket.emit).not.toHaveBeenCalledWith('hosted:turn:abort');+ expect(screen.queryByRole('alert')).not.toBeInTheDocument();+ recorders[1].ondataavailable({ data: new Blob(['new recording']) });+ fireEvent.pointerUp(button);+ act(() => recorders[1].onstop());+ await waitFor(() => expect(mockSocket.emit).toHaveBeenCalledWith('hosted:mic:stop', expect.any(Uint8Array)));+ const bytes = mockSocket.emit.mock.calls.find(([event]) => event === 'hosted:mic:stop')[1];+ expect(new TextDecoder().decode(bytes)).toBe('new recording');+ } finally {+ vi.unstubAllGlobals();+ }+ });+it('keeps audio chunks separate across two press and release cycles', async () => {window.location.hash = '#session=sess-123&token=tok-abc';const tracks = [{ stop: vi.fn() }, { stop: vi.fn() }];diff --git a/client/src/services/voiceClient.js b/client/src/services/voiceClient.jsindex d93fa4159a..f394068708 100644--- a/client/src/services/voiceClient.js+++ b/client/src/services/voiceClient.js@@ -9,7 +9,7 @@ import socket from './socket';import { subscribeVisibility } from '../hooks/useVisibilityEvent.js';import { sleep } from '../utils/sleep.js';import { resumeAudioContext, acquireAudioSession } from '../lib/audioContext.js';-import { blobToWav16k, float32ToWav16k } from '../lib/audioRecorder.js';+import { blobToWav16k, float32ToWav16k, MAX_VOICE_AUDIO_BYTES, RECORDING_TOO_LONG_MESSAGE } from '../lib/audioRecorder.js';import { ECHO_WINDOW_MS, MIN_TOKENS_FOR_ECHO_CHECK, MIN_SHARED_TRIGRAMS, tokenize, trigramsOf } from '../../../server/lib/voiceEcho.js';// iOS audio-session claims held while a mic stream is open — one per capture@@ -530,6 +530,10 @@ export const stopCapture = async ({ submit = true } = {}) => {const { wav, peak } = await blobToWav16k(blob);if (!isOwnerCurrent(ownerGeneration) || recorderGeneration !== stoppedGeneration) return null;+ if (wav.byteLength > MAX_VOICE_AUDIO_BYTES) {+ refuseLongRecording();+ return null;+ }socket.emit('voice:turn', { audio: wav, mimeType: 'audio/wav' });return { mimeType: 'audio/wav', size: wav.byteLength, sourceSize: blob.size, peak };};@@ -587,13 +591,22 @@ export const resetConversation = () => {export const isCapturing = () => recorder !== null;+const localVoiceErrorHandlers = new Set();+const refuseLongRecording = () => {+ for (const handler of localVoiceErrorHandlers) handler({ stage: 'turn', message: RECORDING_TOO_LONG_MESSAGE });+};+/*** Subscribe to voice events. Returns an unsubscribe function.* Events: voice:transcript, voice:llm:delta, voice:llm:done, voice:error, voice:idle*/export const onVoiceEvent = (event, handler) => {socket.on(event, handler);- return () => socket.off(event, handler);+ if (event === 'voice:error') localVoiceErrorHandlers.add(handler);+ return () => {+ socket.off(event, handler);+ if (event === 'voice:error') localVoiceErrorHandlers.delete(handler);+ };};// Persistent, user-authorized screen-capture stream for the voice agent's@@ -684,7 +697,7 @@ export const captureScreenForVision = async () => {video.pause();if (!cctx) return null;cctx.drawImage(video, 0, 0, w, h);- // JPEG @ 0.8 keeps the payload well under the server's 16 MB cap even at 4K.+ // JPEG @ 0.8 keeps normal tab captures below the encoded 8 MiB cap.return canvas.toDataURL('image/jpeg', 0.8);};@@ -693,7 +706,7 @@ export const captureScreenForVision = async () => {// requestId so the server resolves the matching waiter and a late reply can't// satisfy a newer capture.export const sendScreenshotResult = (requestId, dataUrl) => {- socket.emit('voice:screenshot:result', { requestId, dataUrl: dataUrl || null });+ socket.emit('voice:screenshot:result', { requestId, dataUrl: typeof dataUrl === 'string' && dataUrl.length <= MAX_VOICE_AUDIO_BYTES ? dataUrl : null });};const playPreviewAudio = (arrayBuffer) => enqueuePlay(arrayBuffer);@@ -869,6 +882,11 @@ const submitUtterance = async (capture) => {capture.callbacks?.onSubmit?.({ submitted: false, peak });return;}+ if (wav.byteLength > MAX_VOICE_AUDIO_BYTES) {+ refuseLongRecording();+ capture.callbacks?.onSubmit?.({ submitted: false, peak });+ return;+ }socket.emit('voice:turn', { audio: wav, mimeType: 'audio/wav' });capture.callbacks?.onSubmit?.({ submitted: true, peak, size: wav.byteLength });};diff --git a/client/src/services/voiceClient.test.js b/client/src/services/voiceClient.test.jsindex 63e875f90b..015afd4a12 100644--- a/client/src/services/voiceClient.test.js+++ b/client/src/services/voiceClient.test.js@@ -66,12 +66,15 @@ const { socket, listeners, audio, FakeAudioContext } = vi.hoisted(() => {return { socket, listeners, audio, FakeAudioContext };});-const audioRecorderStub = vi.hoisted(() => ({ blobToWav16k: null }));+const audioRecorderStub = vi.hoisted(() => ({ blobToWav16k: null, float32ToWav16k: null }));vi.mock('../lib/audioRecorder.js', async (importOriginal) => {const actual = await importOriginal();return {...actual,+ float32ToWav16k: (...args) => audioRecorderStub.float32ToWav16k+ ? audioRecorderStub.float32ToWav16k(...args)+ : actual.float32ToWav16k(...args),blobToWav16k: (...args) => audioRecorderStub.blobToWav16k? audioRecorderStub.blobToWav16k(...args): actual.blobToWav16k(...args),@@ -164,6 +167,7 @@ describe('voice playback cancellation', () => {audio.sources.length = 0;socket.emit.mockClear();audioRecorderStub.blobToWav16k = null;+ audioRecorderStub.float32ToWav16k = null;voiceClient.disposeCaptureOwner();});@@ -403,6 +407,56 @@ describe('voice playback cancellation', () => {});});+ it('sends capped screenshots and resolves oversized captures as unavailable', async () => {+ const { MAX_VOICE_AUDIO_BYTES } = await import('../lib/audioRecorder.js');+ const prefix = 'data:image/jpeg;base64,';+ const capped = prefix + 'A'.repeat(MAX_VOICE_AUDIO_BYTES - prefix.length);+ voiceClient.sendScreenshotResult('example-capture', capped);+ expect(socket.emit).toHaveBeenLastCalledWith('voice:screenshot:result', { requestId: 'example-capture', dataUrl: capped });+ voiceClient.sendScreenshotResult('example-capture', capped + 'A');+ expect(socket.emit).toHaveBeenLastCalledWith('voice:screenshot:result', { requestId: 'example-capture', dataUrl: null });+ });++ it('refuses an over-cap WAV locally through the voice error subscription and accepts the cap', async () => {+ const previousMediaDevices = Object.getOwnPropertyDescriptor(navigator, 'mediaDevices');+ const track = { stop: vi.fn() };+ Object.defineProperty(navigator, 'mediaDevices', {+ configurable: true,+ value: { getUserMedia: vi.fn().mockResolvedValue({ getTracks: () => [track] }) },+ });+ const previousGlobalRecorder = globalThis.MediaRecorder;+ globalThis.MediaRecorder = FakeMediaRecorder;+ const previousRecorder = window.MediaRecorder;+ window.MediaRecorder = FakeMediaRecorder;+ const onError = vi.fn();+ const unsubscribe = voiceClient.onVoiceEvent('voice:error', onError);+ try {+ const { MAX_VOICE_AUDIO_BYTES, RECORDING_TOO_LONG_MESSAGE } = await import('../lib/audioRecorder.js');+ for (const size of [MAX_VOICE_AUDIO_BYTES + 1, MAX_VOICE_AUDIO_BYTES]) {+ audioRecorderStub.blobToWav16k = vi.fn(async () => ({ wav: new ArrayBuffer(size), peak: 0.5 }));+ await voiceClient.startCapture();+ FakeMediaRecorder.last.dispatch('dataavailable', { data: new Blob(['x'.repeat(1000)]) });+ const result = await voiceClient.stopCapture();+ if (size > MAX_VOICE_AUDIO_BYTES) {+ expect(result).toBeNull();+ expect(socket.emit).not.toHaveBeenCalledWith('voice:turn', expect.anything());+ expect(onError).toHaveBeenCalledWith({ stage: 'turn', message: RECORDING_TOO_LONG_MESSAGE });+ } else {+ expect(result.size).toBe(size);+ expect(socket.emit).toHaveBeenCalledWith('voice:turn', expect.objectContaining({ audio: expect.any(ArrayBuffer) }));+ }+ }+ expect(track.stop).toHaveBeenCalledTimes(2);+ } finally {+ unsubscribe();+ window.MediaRecorder = previousRecorder;+ if (previousGlobalRecorder === undefined) delete globalThis.MediaRecorder;+ else globalThis.MediaRecorder = previousGlobalRecorder;+ if (previousMediaDevices) Object.defineProperty(navigator, 'mediaDevices', previousMediaDevices);+ else delete navigator.mediaDevices;+ }+ });+it('does not emit captured audio when the owner is disposed during WAV conversion', async () => {const previousMediaDevices = Object.getOwnPropertyDescriptor(navigator, 'mediaDevices');const track = { stop: vi.fn() };@@ -489,6 +543,37 @@ describe('voice playback cancellation', () => {}});+ it('settles a continuous turn locally when its converted WAV exceeds the cap', async () => {+ const { MAX_VOICE_AUDIO_BYTES, RECORDING_TOO_LONG_MESSAGE } = await import('../lib/audioRecorder.js');+ const previousMediaDevices = Object.getOwnPropertyDescriptor(navigator, 'mediaDevices');+ Object.defineProperty(navigator, 'mediaDevices', {+ configurable: true,+ value: { getUserMedia: vi.fn().mockResolvedValue({ getTracks: () => [{ stop: vi.fn() }] }) },+ });+ let now = 1000;+ vi.spyOn(performance, 'now').mockImplementation(() => now);+ audioRecorderStub.float32ToWav16k = vi.fn(async () => ({ wav: new ArrayBuffer(MAX_VOICE_AUDIO_BYTES + 1), peak: 0.5 }));+ const onError = vi.fn();+ const onSubmit = vi.fn();+ const unsubscribe = voiceClient.onVoiceEvent('voice:error', onError);+ try {+ await voiceClient.startContinuous({ onSubmit });+ const frame = FakeAudioWorkletNode.last.port.onmessage;+ now = 1700;+ frame({ data: new Float32Array(128) });+ for (let i = 0; i < 40; i++) frame({ data: new Float32Array(128).fill(0.5) });+ now = 18000;+ frame({ data: new Float32Array(128).fill(0.5) });+ await vi.waitFor(() => expect(onSubmit).toHaveBeenCalledWith({ submitted: false, peak: 0.5 }));+ expect(onError).toHaveBeenCalledWith({ stage: 'turn', message: RECORDING_TOO_LONG_MESSAGE });+ expect(socket.emit).not.toHaveBeenCalledWith('voice:turn', expect.anything());+ } finally {+ unsubscribe();+ if (previousMediaDevices) Object.defineProperty(navigator, 'mediaDevices', previousMediaDevices);+ else delete navigator.mediaDevices;+ }+ });+it('ignores VAD worklet frames and async submissions after owner disposal', async () => {const previousMediaDevices = navigator.mediaDevices;const track = { stop: vi.fn() };diff --git a/server/index.js b/server/index.jsindex aee03b4154..77bcbdcbfa 100644--- a/server/index.js+++ b/server/index.js@@ -3,6 +3,7 @@ import providerFallbackRoutes from './routes/providerFallbacks.js';import './services/databaseBootFence.js';import express from 'express';import { Server } from 'socket.io';+import { SOCKET_MAX_PAYLOAD_BYTES } from './lib/socketPayloadLimits.js';import { fileURLToPath } from 'url';import { dirname, join } from 'path';import { PATHS } from './lib/fileUtils.js';@@ -214,6 +215,7 @@ setHttpsEnabledAtBoot(httpsEnabled);// Socket.IO with relative path support for Tailscaleconst io = new Server(httpServer, {+ maxHttpBufferSize: SOCKET_MAX_PAYLOAD_BYTES,// CORS headers are reflected only for handshakes allowRequest admits: the// same browser-relay guard as the HTTP gate refuses a foreign Origin or a// rebindable Host on both the polling and websocket transports, in bothdiff --git a/server/lib/README.md b/server/lib/README.mdindex 6765f9a282..72ff3796d1 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -150,6 +150,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `postValidation.js` | MeatSpace POST (Power On Self Test) schemas — drill config (incl. adaptive toggle), drill generation/scoring, sessions, memory builder, training log. || `privacyValidation.js` | Privacy Center schemas — PII Vault (issue #2140): vault record create/update (partial PUT), list query, UUID params; the vault type/status vocabularies + the sensitive-type (`ssn`/`passport`/`drivers_license`/`financial_account`) `useForScans` hard-false rule and per-type scan defaults. Trusted Organizations registry (issue #2141): org create/update (partial PUT), list query, UUID params, and the replace-set holdings schema. Data-broker database + case ledger (issue #2144): broker list/case-list query filters, the scan-start + refresh action bodies, and the `PRIVACY_BROKER_CASE_STATES` vocabulary. || `roundsValidation.js` | Rounds workbench shape bounds + pure record sanitizers (sections, layers, recordings, pitch analysis, references, progress) — `sanitizeRound(raw, builtinIds)` and the length/count limits shared with routes/rounds.js. |+| `socketPayloadLimits.js` | Shared 8 MiB media cap and transport ceiling accounting for polling base64 encoding plus 64 KiB framing headroom. || `socketValidation.js` | Socket event payload schemas. || `spriteValidation.js` | Sprite Manager Zod schemas — record import/create/update, reference generate/lock/unlock + fork, walk-cycle generate/approve/reopen/trim, per-track generate/approve schemas built from the animation-track registry, publish binding + runtime contract, atlas publish/compile, and user-defined animation-track create/update. Split out of `validation.js` (#3873), which re-exports it. || `styleSourcePrompt.js` | `universeStyleLines(universe)` / `moodBoardSection(board)` — a resolved universe style guide or mood board (`services/creativeStyleSources.js`) as prompt text; shared by Code Animation and Creative Commissions. |diff --git a/server/lib/index.js b/server/lib/index.jsindex 2835d2e0ba..5776f2aa1b 100644--- a/server/lib/index.js+++ b/server/lib/index.js@@ -75,6 +75,7 @@ export * as postRhetoric from './postRhetoric.js';export * as postValidation from './postValidation.js';export * as privacyValidation from './privacyValidation.js';export * as roundsValidation from './roundsValidation.js';+export * from './socketPayloadLimits.js';export * as socketValidation from './socketValidation.js';export * from './socketEventContracts.js';export * from './socketEventInventory.js';diff --git a/server/lib/socketPayloadLimits.js b/server/lib/socketPayloadLimits.jsnew file mode 100644index 0000000000..d36eeaa910--- /dev/null+++ b/server/lib/socketPayloadLimits.js@@ -0,0 +1,5 @@+// Keep the transport above the handler caps even on Engine.IO polling, which+// base64-encodes binary attachments. Reserve 64 KiB for packet framing and+// metadata, including a small over-cap payload reaching its handler refusal.+export const SOCKET_MEDIA_MAX_BYTES = 8 * 1024 * 1024;+export const SOCKET_MAX_PAYLOAD_BYTES = 4 * Math.ceil(SOCKET_MEDIA_MAX_BYTES / 3) + 64 * 1024;diff --git a/server/sockets/fableLoomHosted.js b/server/sockets/fableLoomHosted.jsindex 744074e438..946147b5a2 100644--- a/server/sockets/fableLoomHosted.js+++ b/server/sockets/fableLoomHosted.js@@ -6,6 +6,7 @@* acquires general PortOS auth privileges or access to other socket events.*/+import { SOCKET_MEDIA_MAX_BYTES } from '../lib/socketPayloadLimits.js';import {_getInternalSession,abortHostedTurn,@@ -27,7 +28,7 @@ import {// two seconds of headroom over the ~20 ms bursts the audience page ships, and// 8 MB is ~4 minutes of 16 kHz mono audio for a single turn.const MAX_MIC_FRAME_BYTES = 64 * 1024;-const MAX_UTTERANCE_BYTES = 8 * 1024 * 1024;+const MAX_UTTERANCE_BYTES = SOCKET_MEDIA_MAX_BYTES;const AUDIO_TOO_LARGE = Symbol('audio-too-large');// Socket payloads are untrusted: never let Buffer.from coerce arbitrary objectsdiff --git a/server/sockets/socketPayloadLimits.test.js b/server/sockets/socketPayloadLimits.test.jsnew file mode 100644index 0000000000..ebe3652282--- /dev/null+++ b/server/sockets/socketPayloadLimits.test.js@@ -0,0 +1,119 @@+import { createServer } from 'node:http';+import { once } from 'node:events';+import { readFileSync } from 'node:fs';+import { Server } from 'socket.io';+import { io as connect } from 'socket.io-client';+import { describe, it, expect, vi } from 'vitest';+import { SOCKET_MAX_PAYLOAD_BYTES } from '../lib/socketPayloadLimits.js';+import { registerVoiceHandlers, MAX_AUDIO_BYTES, MAX_SCREENSHOT_BYTES } from './voice.js';+import { registerFableLoomHostedNamespace, MAX_UTTERANCE_BYTES } from './fableLoomHosted.js';++vi.mock('../services/voice/config.js', () => ({ getVoiceConfig: vi.fn(async () => ({ enabled: true })) }));+vi.mock('../services/voice/pipeline.js', () => ({ runTurn: vi.fn(async ({ emit, state, text }) => {+ if (text === 'capture') {+ const screenshot = new Promise((resolve) => state.screenshotWaiters.set('example-capture', resolve));+ emit('voice:screenshot:request', { requestId: 'example-capture' });+ const dataUrl = await screenshot;+ emit('test:screenshot', { length: dataUrl?.length ?? null });+ }+ emit('voice:idle', {});+ return {};+}) }));+vi.mock('../services/voice/stt.js', () => ({ transcribe: vi.fn() }));+vi.mock('../services/voice/tts.js', () => ({ synthesize: vi.fn() }));+vi.mock('../services/fableLoom/hostedSession.js', () => {+ const session = { id: 'example-session', status: 'active', turnPhase: 'listening' };+ return {+ _getInternalSession: () => session,+ abortHostedTurn: vi.fn(),+ endHostedSession: vi.fn(),+ getHostedSession: () => session,+ processHostedUtterance: vi.fn(async (_id, { io }) => {+ io.of('/fableloom-hosted').emit('hosted:accepted');+ }),+ sanitizeHostedSession: () => session,+ startHostedListening: vi.fn(),+ switchHostedEpisode: vi.fn(),+ updateHostedSession: vi.fn(),+ verifyHostedHostToken: () => false,+ verifyHostedToken: (_id, token) => token === 'example-token',+ };+});++const nextEvent = (socket, event) => once(socket, event, { signal: AbortSignal.timeout(5000) });++describe('Socket.IO media transport contract', () => {+ it('boots with the shared ceiling above encoded media caps and bounded framing', () => {+ const bootstrap = readFileSync(new URL('../index.js', import.meta.url), 'utf8');+ expect(bootstrap).toContain('maxHttpBufferSize: SOCKET_MAX_PAYLOAD_BYTES');+ for (const cap of [MAX_AUDIO_BYTES, MAX_SCREENSHOT_BYTES, MAX_UTTERANCE_BYTES]) {+ expect(SOCKET_MAX_PAYLOAD_BYTES - 4 * Math.ceil(cap / 3)).toBeGreaterThanOrEqual(64 * 1024);+ }+ });++ // Polling expands binary to base64; websocket sends raw bytes. The maximum+ // permitted recording and handler refusal must work on both shipped paths.+ it.each(['websocket', 'polling'])('accepts capped media and refuses over-cap media without losing %s subscriptions', async (transport) => {+ const http = createServer();+ const server = new Server(http, { maxHttpBufferSize: SOCKET_MAX_PAYLOAD_BYTES });+ server.on('connection', (socket) => {+ registerVoiceHandlers(socket);+ socket.on('test:ping', () => socket.emit('test:pong'));+ });+ registerFableLoomHostedNamespace(server);+ const clients = [];+ try {+ http.listen(0, '127.0.0.1');+ await once(http, 'listening');+ const origin = `http://127.0.0.1:${http.address().port}`;+ const voice = connect(origin, { transports: [transport], reconnection: false });+ clients.push(voice);+ await nextEvent(voice, 'connect');+ expect(voice.io.engine.transport.name).toBe(transport);+ for (const size of [2 * 1024 * 1024, MAX_AUDIO_BYTES]) {+ const accepted = nextEvent(voice, 'voice:idle');+ voice.emit('voice:turn', { audio: Buffer.alloc(size), mimeType: 'audio/wav' });+ await accepted;+ }+ const refused = nextEvent(voice, 'voice:error');+ voice.emit('voice:turn', { audio: Buffer.alloc(MAX_AUDIO_BYTES + 1) });+ expect((await refused)[0]).toMatchObject({ stage: 'turn', message: expect.stringContaining('audio too large') });+ const pong = nextEvent(voice, 'test:pong');+ voice.emit('test:ping');+ await pong;+ expect(voice.connected).toBe(true);++ for (const size of [MAX_SCREENSHOT_BYTES, MAX_SCREENSHOT_BYTES + 1]) {+ const request = nextEvent(voice, 'voice:screenshot:request');+ voice.emit('voice:text', { text: 'capture' });+ const [{ requestId }] = await request;+ const result = nextEvent(voice, 'test:screenshot');+ const prefix = 'data:image/jpeg;base64,';+ voice.emit('voice:screenshot:result', { requestId, dataUrl: prefix + 'A'.repeat(size - prefix.length) });+ expect((await result)[0]).toEqual({ length: size <= MAX_SCREENSHOT_BYTES ? size : null });+ }+ expect(voice.connected).toBe(true);++ const guest = connect(`${origin}/fableloom-hosted`, {+ transports: [transport], reconnection: false,+ auth: { sessionId: 'example-session', token: 'example-token', role: 'audience' },+ forceNew: true,+ });+ clients.push(guest);+ await nextEvent(guest, 'connect');+ const accepted = nextEvent(guest, 'hosted:accepted');+ guest.emit('hosted:mic:stop', Buffer.alloc(MAX_UTTERANCE_BYTES));+ await accepted;+ const guestRefused = nextEvent(guest, 'hosted:error');+ guest.emit('hosted:mic:stop', Buffer.alloc(MAX_UTTERANCE_BYTES + 1));+ expect((await guestRefused)[0]).toMatchObject({ code: 'UTTERANCE_TOO_LARGE' });+ const recovered = nextEvent(guest, 'hosted:accepted');+ guest.emit('hosted:mic:stop', Buffer.alloc(1024));+ await recovered;+ expect(guest.connected).toBe(true);+ } finally {+ clients.forEach((client) => client.disconnect());+ await new Promise((resolve) => server.close(resolve));+ }+ });+});diff --git a/server/sockets/voice.js b/server/sockets/voice.jsindex 8045465084..922c53c0ea 100644--- a/server/sockets/voice.js+++ b/server/sockets/voice.js@@ -16,6 +16,7 @@// Inbound: voice:capture:start | voice:call:audio | voice:capture:stop// Outbound: voice:capture:state+import { SOCKET_MEDIA_MAX_BYTES } from '../lib/socketPayloadLimits.js';import { socketHasCurrentHostControl } from '../services/authGate.js';import { runTurn } from '../services/voice/pipeline.js';import { getVoiceConfig } from '../services/voice/config.js';@@ -68,7 +69,9 @@ const HISTORY_MESSAGES = 24;// Payload size caps. Voice audio is typically 16 kHz mono PCM/WebM (~32 KB/s),// so 8 MB leaves headroom for ~4 min of audio even in WAV. Text utterances are// short; 4 KB covers any realistic spoken turn and rejects prompt-stuffing.-const MAX_AUDIO_BYTES = 8 * 1024 * 1024;+export const MAX_AUDIO_BYTES = SOCKET_MEDIA_MAX_BYTES;+// Encoded data-URL string length, including the prefix; not decoded image bytes.+export const MAX_SCREENSHOT_BYTES = SOCKET_MEDIA_MAX_BYTES;const MAX_TEXT_LEN = 4000;// Cap on the visible-text snapshot the client ships alongside each UI index.// Matches `MAX_TEXT_CHARS` in `client/src/services/domIndex.js` so the ~8 KB@@ -363,8 +366,8 @@ export const registerVoiceHandlers = (socket) => {// captured tab (or null if capture failed / the user denied permission). The// requestId echoes the request so a late result can't resolve a newer waiter.// Cap the payload to bound memory from a runaway/malicious client — a- // full-screen PNG data URL is typically a few MB, so 16 MB is generous.- const MAX_SCREENSHOT_BYTES = 16 * 1024 * 1024;+ // full-screen JPEG data URL is typically a few MB; cap the encoded string+ // at 8 MiB so it fits the transport alongside packet metadata.socket.on('voice:screenshot:result', (payload) => {if (!payload || typeof payload !== 'object') return;const { requestId, dataUrl: raw } = payload;