Node-based terminal manager for AI coding agents — tmux-backed terminals and parallel agent sessions as draggable nodes on an infinite pan/zoom canvas. macOS, Linux, and a browser Server Edition.
TypeScript adhdagent-orchestrationai-agentscanvasclaude-codecodexcoding-agentsdeveloper-toolselectrongeminilinuxmacosmultiplexeropencodeparallel-agentsterminalterminal-emulatorterminal-multiplexertmuxworkspace-manager
Merge origin/main into fix/smart-whisper-context-params
review7d15e243397a
diff --cc CLAUDE.mdindex 4f7ad1e5e,4b2c9ba85..028b1f1ba--- a/CLAUDE.md+++ b/CLAUDE.md@@@ -9893,17 -10079,15 +10079,26 @@@ Voice-to-text input captured via micropSIMD flags in ggml), a GPU backend (this smart-whisper has no Vulkan build option), cancelling arunning transcription, and the Server Edition (installed with `--ignore-scripts`, so it keeps thebaseline; the startup log line `[speech] whisper binary: …` says which binary loaded).- - **smart-whisper's model params are patched to start from whisper.cpp's defaults** (second patch in+ - **macOS: smart-whisper's Metal shader path is patched to the unpacked copy** (second patch in+ `scripts/patch-smart-whisper.mjs`, marker `NODETERM-PATCH(smart-whisper-metal-asar)`). smart-whisper+ sets `GGML_METAL_PATH_RESOURCES` from its own `__dirname`, which in the packaged app is inside+ `app.asar`; native code cannot read inside an asar ("Not a directory"), so Metal init failed and+ whisper.cpp ran on the CPU with no error shown. The shader file is already unpacked (asarUnpack), so+ the patch rewrites `app.asar` to `app.asar.unpacked` in that path. MEASURED with the app's own binary+ on an M5 Pro, large-v3-turbo, 2.7 s clip: 5.97 s on the CPU, 0.64 s on Metal; in the app, dictation+ went from several seconds to under one. A dev install and an explicit `GGML_METAL_PATH_RESOURCES`+ are unchanged.++- **smart-whisper's model params are patched to start from whisper.cpp's defaults** (third patch in+ `scripts/patch-smart-whisper.mjs`, marker `NODETERM-PATCH(smart-whisper-ctx-params)`, every+ platform). Its `LoadModel` declared `whisper_context_params params;` and set only `use_gpu`, so DTW,+ `gpu_device`, `flash_attn` and the rest were uninitialized stack memory and the result depended on+ what was there. MEASURED on an M5 Pro, Electron 42, large-v3-turbo: the shipped binary crashed in 4+ of 4 runs outside the app (`aheads_masks_init` then a crash in `whisper_free_state`, because garbage+ switched DTW on with large-v3's alignment heads); with the defaults it ran without a crash, 0.56 s+ for a 2.7 s clip and 0.93 s for 21 s on Metal. The slow dictation inside the packaged app is a+ separate bug: Metal cannot read its shaders inside app.asar. A newer whisper.cpp is+ faster again (0.22 s / 0.43 s, measured the same way) but needs a build change on every platform;+ that is a separate follow-up.## Packaging & auto-updatediff --cc scripts/patch-smart-whisper.d.mtsindex c6c0e1c51,6b33e2f9c..1f81e6c7c--- a/scripts/patch-smart-whisper.d.mts+++ b/scripts/patch-smart-whisper.d.mts@@@ -3,5 -3,5 +3,6 @@@export declare const PATCH_MARKER: stringexport declare const VARIANT_TARGET: stringexport declare const SIMD_FLAGS: readonly string[]+export declare const CTX_PARAMS_MARKER: stringexport declare function patchSmartWhisper(dir: string): 'patched' | 'already-patched'+ export declare const METAL_MARKER: stringdiff --cc scripts/patch-smart-whisper.mjsindex a3f841e98,4984a5bb0..737d5c303--- a/scripts/patch-smart-whisper.mjs+++ b/scripts/patch-smart-whisper.mjs@@@ -28,19 -28,16 +28,28 @@@* no-op when nothing sets the global.* - Runs before electron-rebuild in `postinstall` / `rebuild`, so the native build sees it.*- * SECOND PATCH: initialized context params (every platform)+ * SECOND PATCH: Metal shaders in a packaged app (macOS)+ * smart-whisper tells whisper.cpp where its Metal shader source is through+ * GGML_METAL_PATH_RESOURCES, built from its own __dirname. In the packaged app dist/ is inside+ * app.asar, and native code cannot read inside an asar ("Not a directory"), so Metal init fails+ * and whisper.cpp silently runs on the CPU. The shader file is already unpacked beside it+ * (asarUnpack), so the patch points at app.asar.unpacked. Measured on an M5 Pro, Electron 42,+ * large-v3-turbo, 2.7 s clip, the app's own binary: 5.97 s on the CPU vs 0.64 s on Metal. An+ * explicit GGML_METAL_PATH_RESOURCES and a dev install are left unchanged.+ *- * GUARD: src/core/speech/smart-whisper-patch.test.ts asserts the marker is present in the++ * THIRD PATCH: initialized context params (every platform)+ * smart-whisper's `LoadModel` (src/binding/model.cc) declares `whisper_context_params params;`+ * and sets only `use_gpu`, so every other field (dtw_token_timestamps, dtw_aheads_preset,+ * gpu_device, flash_attn, ...) is uninitialized stack memory. What it holds decides the outcome:+ * with DTW switched on by garbage, whisper_init_state() fails for large-v3-turbo ("tried to set+ * alignment head on text layer 14, but model only has 4 text layers") and the app aborts in+ * whisper_free_state. Measured on an M5 Pro, Electron 42, large-v3-turbo, 2.7 s and 21 s clips:+ * the shipped binary crashed in 4 of 4 runs outside the app; with the params initialized it ran+ * without a crash (0.56 s and 0.93 s on Metal). The slow dictation seen inside the packaged app is- * a separate bug (Metal cannot read its shaders inside app.asar). The fix starts from++ * a separate bug, the second patch above. The fix starts from+ * whisper_context_default_params(), exactly what whisper.cpp's own examples do.+ *- * GUARD: src/core/speech/smart-whisper-patch.test.ts asserts both markers are present in the++ * GUARD: src/core/speech/smart-whisper-patch.test.ts asserts every marker is present in the* installed sources. If it is red, your node_modules is unpatched — run `npm run rebuild`.*/@@@ -77,37 -74,32 +86,50 @@@ var module2 = (() =>const GYP_FIND = ' }\n ],\n}'+ export const METAL_MARKER = 'NODETERM-PATCH(smart-whisper-metal-asar)'++ const METAL_FIND =+ 'process.env.GGML_METAL_PATH_RESOURCES = process.env.GGML_METAL_PATH_RESOURCES || import_node_path.default.join(__dirname, "../whisper.cpp/ggml/src");'++ const METAL_REPLACE = `// ${METAL_MARKER}: native code cannot read inside app.asar; use the unpacked copy.+ process.env.GGML_METAL_PATH_RESOURCES = process.env.GGML_METAL_PATH_RESOURCES || import_node_path.default.join(__dirname, "../whisper.cpp/ggml/src").replace(/([\\\\/])app\\.asar(?=[\\\\/])/, "$1app.asar.unpacked");`++export const CTX_PARAMS_MARKER = 'NODETERM-PATCH(smart-whisper-ctx-params)'++const PARAMS_FIND = ' whisper_context_params params;\n params.use_gpu'++const PARAMS_REPLACE =+ ` // ${CTX_PARAMS_MARKER}: start from the defaults; the bare declaration left every\n` ++ ` // other field (DTW, gpu_device, flash_attn, ...) as uninitialized stack memory.\n` ++ ' whisper_context_params params = whisper_context_default_params();\n' ++ ' params.use_gpu'+/** Patches the smart-whisper package in `dir`. Returns 'patched' | 'already-patched'. */export function patchSmartWhisper(dir) {const gypPath = path.join(dir, 'binding.gyp')const jsPath = path.join(dir, 'dist', 'index.js')const gyp = fs.readFileSync(gypPath, 'utf8')const js = fs.readFileSync(jsPath, 'utf8')+ const modelPath = path.join(dir, 'src', 'binding', 'model.cc')+ const model = readOrFail(modelPath, 'src/binding/model.cc')const gypDone = gyp.includes(PATCH_MARKER)const jsDone = js.includes(PATCH_MARKER)+ const metalDone = js.includes(METAL_MARKER)- if (gypDone && jsDone && metalDone) return 'already-patched'+ const modelDone = model.includes(CTX_PARAMS_MARKER)- if (gypDone && jsDone && modelDone) return 'already-patched'++ if (gypDone && jsDone && metalDone && modelDone) return 'already-patched'++ if (!modelDone) {+ expectOnce(model, PARAMS_FIND, 'model.cc context params')+ fs.writeFileSync(modelPath, model.replace(PARAMS_FIND, PARAMS_REPLACE))+ }- if (!jsDone) {- expectOnce(js, LOADER_FIND, 'dist/index.js binary require')- fs.writeFileSync(jsPath, js.replace(LOADER_FIND, LOADER_REPLACE))+ if (!jsDone || !metalDone) {+ if (!jsDone) expectOnce(js, LOADER_FIND, 'dist/index.js binary require')+ if (!metalDone) expectOnce(js, METAL_FIND, 'dist/index.js Metal path')+ let out = js+ if (!jsDone) out = out.replace(LOADER_FIND, LOADER_REPLACE)+ if (!metalDone) out = out.replace(METAL_FIND, METAL_REPLACE)+ fs.writeFileSync(jsPath, out)}if (!gypDone) {diff --cc src/core/speech/smart-whisper-patch.test.tsindex c72e01624,5b0af9045..57c614f2e--- a/src/core/speech/smart-whisper-patch.test.ts+++ b/src/core/speech/smart-whisper-patch.test.ts@@@ -2,13 -2,7 +2,14 @@@ import { describe, expect, it } from 'vimport fs from 'node:fs'import path from 'node:path'import { createRequire } from 'node:module'-import { METAL_MARKER, PATCH_MARKER, SIMD_FLAGS, VARIANT_TARGET, patchSmartWhisper } from '../../../scripts/patch-smart-whisper.mjs'+import {+ CTX_PARAMS_MARKER,++ METAL_MARKER,+ PATCH_MARKER,+ SIMD_FLAGS,+ VARIANT_TARGET,+ patchSmartWhisper+} from '../../../scripts/patch-smart-whisper.mjs'import { WHISPER_SIMD_FLAGS } from './speech-service'import { testTmpDir } from '../test-tmp'@@@ -20,11 -14,10 +21,10 @@@*/const PKG = path.resolve(__dirname, '../../../node_modules/smart-whisper')const installed = fs.existsSync(path.join(PKG, 'binding.gyp'))-/** Pristine smart-whisper 0.8.1 binding.gyp + the loader lines of its dist/index.js (dist-index.js). */+/** Pristine smart-whisper 0.8.1 binding.gyp, src/binding/model.cc, and its dist/index.js (dist-index.js). */const FIXTURE = path.resolve(__dirname, '__fixtures__/smart-whisper-0.8.1')- function copyPackage(): string {- const dir = testTmpDir('sw-patch-')+ function copyPackage(dir = testTmpDir('sw-patch-')): string {fs.mkdirSync(path.join(dir, 'dist'), { recursive: true })fs.copyFileSync(path.join(FIXTURE, 'binding.gyp'), path.join(dir, 'binding.gyp'))// Stored flat: the repo's .gitignore drops every dist/ directory.@@@ -39,27 -30,7 +39,28 @@@ describe('patch-smart-whisper', () =>const howto = 'node_modules is unpatched — run `npm run rebuild`.'expect(fs.readFileSync(path.join(PKG, 'binding.gyp'), 'utf8'), howto).toContain(PATCH_MARKER)expect(fs.readFileSync(path.join(PKG, 'dist', 'index.js'), 'utf8'), howto).toContain(PATCH_MARKER)+ expect(fs.readFileSync(path.join(PKG, 'dist', 'index.js'), 'utf8'), howto).toContain(METAL_MARKER)+ expect(fs.readFileSync(path.join(PKG, 'src', 'binding', 'model.cc'), 'utf8'), howto).toContain(CTX_PARAMS_MARKER)+ })++ it('starts the model context params from whisper.cpp defaults and is idempotent', () => {+ // The pristine declaration leaves every field but use_gpu as stack garbage (DTW among them).+ const dir = copyPackage()+ const modelPath = path.join(dir, 'src', 'binding', 'model.cc')+ expect(fs.readFileSync(modelPath, 'utf8')).toMatch(/^\s*whisper_context_params params;$/m)+ expect(patchSmartWhisper(dir)).toBe('patched')+ const model = fs.readFileSync(modelPath, 'utf8')+ expect(model).toContain('whisper_context_params params = whisper_context_default_params();')+ expect(model).not.toMatch(/^\s*whisper_context_params params;$/m)+ expect(model).toContain('params.use_gpu = info.Length() == 2')+ expect(patchSmartWhisper(dir)).toBe('already-patched')+ expect(fs.readFileSync(modelPath, 'utf8')).toBe(model)+ })++ it('refuses a model.cc whose params declaration moved', () => {+ const dir = copyPackage()+ fs.writeFileSync(path.join(dir, 'src', 'binding', 'model.cc'), '// rewritten upstream\n')+ expect(() => patchSmartWhisper(dir)).toThrow(/anchor "model.cc context params"/)})it('builds for exactly the CPU flags the runtime checks', () => {