Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
// Dictation/IME bridge for the xterm.js terminal.
//
// ── The bug this fixes ─────────────────────────────────────────────────────────
// Apple dictation (iOS "mic" key, macOS Fn Fn) does not emit one final phrase —
// it streams progressively refined guesses and REPLACES what it previously wrote
// in the focused field. In a normal <textarea> that is invisible: the field's
// value goes "dde" → "deter" → "determin" → "determines", each edit replacing the
// last. xterm.js, though, is a character STREAM: its hidden textarea listener
// (`CoreBrowserTerminal#_inputEvent`) forwards `inputType: 'insertText'` events
// verbatim and ignores every deletion event, because a terminal has no notion of
// "replace what I just sent". So each refinement is appended to the PTY and the
// user gets the accumulated garble:
//
// ddedeterdetermindeterminedetermines ifdetermines if any code…
//
// ── The fix ────────────────────────────────────────────────────────────────────
// Intercept textarea input events in the capture phase (on `terminal.element`, so
// we run BEFORE xterm's own listener on the textarea inside it) and forward a
// DIFF instead of the raw insertion: DEL (0x7f) for each character the field
// dropped, then whatever it gained. The textarea keeps accumulating exactly as the
// OS intends, and the PTY sees the same edits a real text field would apply.
//
// Resync points (a keystroke xterm handles itself, blur, paste, composition end)
// reset the mirror without emitting anything, because those paths send to the PTY
// through xterm and may clear the textarea out from under us.
//
// ── The keypress seam ──────────────────────────────────────────────────────────
// Mirrored from @xterm/xterm 6.0.0 — `CoreBrowserTerminal#_keyPress` and
// `#_inputEvent`. Re-read those two before trusting this section against a newer
// xterm; the real-Terminal tests fail loudly if either mechanism moves.
// Not every physical keystroke is settled on `keydown`. xterm cancels that event
// for most printable keys (so the character never reaches the textarea and no
// input event fires), but it deliberately defers A-Z and keys its keyboard map
// doesn't claim — space among them — to the `keypress` handler, which forwards the
// character and then does NOT preventDefault. The browser therefore inserts it
// into the textarea and fires `input` for a character the PTY already has, and
// xterm's own `_inputEvent` drops that event via its `_keyPressHandled` guard.
// Stopping the event before xterm took that guard's say away, and diffing the
// insertion sent the character a second time — the reported "every capital letter
// and space is doubled". So an insertion with a live keypress behind it is now
// xterm's: it counts as a resync point, not ours to forward, and the event goes
// through untouched.
//
// Screen-reader mode widens the seam to keydown. There xterm sends a resolved key
// from `_keyDown` and deliberately does NOT cancel it (so the textarea keeps the
// text for the screen reader to announce) — lowercase letters, digits, Backspace
// all land in the field and fire `input` for bytes the PTY already has. So in that
// mode a live keydown latches the same way a keypress does. Outside it, a handled
// keydown is cancelled and never reaches the field.
//
// We mirror `_keyPressHandled` and only that flag. xterm's `_inputEvent` also gates
// on `_keyDownSeen` and clears an `_unprocessedDeadKey`; the omissions are
// deliberate, because our own resync points already cover both — the keydown resync
// for the first, the `compositionend` resync for the second.
//
// ── Boundaries ─────────────────────────────────────────────────────────────────
// Corrections are streamed live, which assumes the far end treats 0x7f as "erase
// the previous character" — true at a shell prompt and in the line editors of the
// TUIs this page drives. In a full-screen app that binds DEL to something else, a
// mid-phrase correction is as approximate as a human backspacing would be; live
// echo is worth more here than a settle-then-send buffer that hides the words the
// user is speaking. Because we stop the event before xterm sees it, xterm never
// clears the textarea mid-phrase either, so the mirror grows for as long as the
// dictation does (bounded by the next resync — Enter, Ctrl-C, or blur).
// 0x7f — what a terminal expects for "erase the previous character".
export const TERMINAL_DEL = '\x7f';
// Input events whose effect we translate into terminal input ourselves: text the
// field gained, and every flavour of text it dropped. Anything else (paste, undo,
// composition text, line breaks) is already handled by xterm or the keydown path
// and only resyncs our mirror — an unknown future `insert*` type must fall through
// to xterm rather than being double-sent by both of us.
const isOwnedInput = (inputType) => (
!inputType
|| inputType === 'insertText'
|| inputType === 'insertReplacementText'
|| inputType.startsWith('delete')
);
const isHighSurrogate = (code) => code >= 0xd800 && code <= 0xdbff;
const commonPrefixLength = (a, b) => {
const max = Math.min(a.length, b.length);
let i = 0;
while (i < max && a.charCodeAt(i) === b.charCodeAt(i)) i++;
// Never split a surrogate pair: cutting between the halves of an astral
// character (swapping 😀 for 😂 shares the high surrogate) would send a lone
// surrogate, which serializes to U+FFFD instead of the emoji.
return i > 0 && isHighSurrogate(a.charCodeAt(i - 1)) ? i - 1 : i;
};
/**
* Plan the terminal input for a text-field edit.
*
* @param {string} mirror what the PTY holds from this field
* @param {string} next what the field holds now
* @param {number} floor how much of `mirror` reached the PTY some other way
* (typed keystrokes, a paste). We retype from the floor rather than rewinding
* through it, so a correction can never erase text we didn't write.
* @returns {{data: string, committed: string}} `data` is the bytes to send ('' when
* nothing changed); `committed` is what the PTY holds once they're applied. The
* two differ from `next` only when `floor` blocked a full rewind — the caller
* must track `committed`, not `next`, or every later diff is computed against a
* baseline the PTY never had.
*/
export const planFieldEdit = (mirror, next, floor = 0) => {
const rewindTo = Math.min(mirror.length, Math.max(commonPrefixLength(mirror, next), floor));
const tail = next.slice(rewindTo);
// One DEL erases one CODE POINT at the far end, but JS string length counts
// UTF-16 code units — so an emoji is one erase, not two. Counting units would
// send a second DEL that eats the character before it.
const erased = [...mirror.slice(rewindTo)].length;
return {
// Pure append is the common case — skip building an empty DEL run for it.
data: rewindTo === mirror.length ? tail : TERMINAL_DEL.repeat(erased) + tail,
committed: mirror.slice(0, rewindTo) + tail,
};
};
/**
* Wire the dictation bridge onto a live xterm instance.
*
* @param {object} terminal — an xterm `Terminal`. Only its public surface is used:
* `element` (the container it mounted into, and an ancestor of `textarea`, so
* capture-phase listeners there run before xterm's own), `textarea`, `options`.
* @param {(data: string) => boolean|void} sendData — forwards to the PTY. Return
* `false` to report the data was dropped (e.g. mid session-switch); the mirror
* then treats it as text we didn't write instead of claiming the PTY has it.
* @returns {() => void} dispose
*/
export const attachDictationBridge = (terminal, sendData) => {
const container = terminal?.element;
const textarea = terminal?.textarea;
if (!container || !textarea || typeof sendData !== 'function') return () => {};
// What the PTY holds from this field. Usually equal to the textarea's value —
// it diverges only when `floor` blocked a full rewind (see planFieldEdit).
let mirror = textarea.value;
// Length of `mirror` we did not put into the PTY ourselves — see planFieldEdit.
let floor = mirror.length;
let resyncTimer = null;
// Set on `keypress` (and, in screen-reader mode, on keydown), consumed by the next
// event — see "The keypress seam" above.
let keyStrokeSeen = false;
const cancelResync = () => {
clearTimeout(resyncTimer);
resyncTimer = null;
};
const resyncNow = () => {
cancelResync();
mirror = textarea.value;
floor = mirror.length;
};
// Deferred: xterm's own handler runs after ours and may clear the textarea
// (Enter and Ctrl-C do), so read the field only once it has had its turn.
const scheduleResync = () => {
// Physical typing is the hot path and leaves nothing to reconcile — xterm
// cancels those keydowns, so the textarea stays as empty as the mirror.
if (resyncTimer != null || (mirror === '' && textarea.value === '')) return;
resyncTimer = setTimeout(resyncNow, 0);
};
const handleKeyDown = (ev) => {
// Whatever this keystroke inserts is xterm's only if a keypress follows it —
// or, in screen-reader mode, already, because xterm sent it from keydown.
keyStrokeSeen = false;
// 229 is the "composition character" every soft keyboard/IME reports — those
// keystrokes land in the textarea and are ours to diff, not a resync point.
// They fire no keypress either, which is what keeps them ours below.
if (ev.keyCode === 229 || ev.isComposing) return;
if (terminal.options?.screenReaderMode) keyStrokeSeen = true;
scheduleResync();
};
const handleKeyPress = () => { keyStrokeSeen = true; };
// The insertion a keypress produces can fail to arrive (the browser consumes the
// combination itself, focus leaves mid-keystroke). Disarm on keyup — as xterm's
// own `_keyUp` disarms `_keyPressHandled` — so the latch can't outlive the
// keystroke and disown the next insertion. Dictation supplies no key events at
// all, so waiting for another keydown to clear it would swallow a dictated phrase.
// `input` fires during the keypress default action, long before keyup, so this
// never races the consume.
const disarmKeyStroke = () => { keyStrokeSeen = false; };
const handleBlur = () => {
disarmKeyStroke();
scheduleResync();
};
const handleInput = (ev) => {
if (ev.target !== textarea) return;
// The character reached the PTY through xterm's keypress path already (see "The
// keypress seam"). Take the insertion into the floor so a later correction can't
// rewind through it, and leave the event alone. Consumed ahead of the guards
// below so the latch is strictly one-shot.
if (keyStrokeSeen) {
disarmKeyStroke();
resyncNow();
return;
}
// A real composition (IME candidate window) is xterm's CompositionHelper's job;
// it forwards the committed text on compositionend.
if (ev.isComposing) return;
if (!isOwnedInput(ev.inputType)) {
resyncNow();
return;
}
// Ours: stop the event before xterm's textarea listener can append the raw
// insertion on top of what we're about to reconcile.
ev.stopPropagation();
// We are reconciling the field right now, so a resync armed by the keystroke
// that produced this event would only re-read the same value — and pin `floor`
// to the whole phrase, silently blocking every later correction from rewinding.
cancelResync();
const { data, committed } = planFieldEdit(mirror, textarea.value, floor);
// A refused send leaves the PTY exactly as it was, so leave the mirror there
// too — the next refinement then re-diffs against what the PTY really has
// instead of erasing characters that never arrived.
if (data && sendData(data) === false) return;
// `committed`, not the field's value: when `floor` blocked a full rewind the
// PTY holds the un-rewound prefix, and claiming otherwise desyncs every
// later diff.
mirror = committed;
};
// blur/compositionend don't bubble, but capture-phase listeners still see them.
const listeners = [
['keydown', handleKeyDown],
['keypress', handleKeyPress],
['keyup', disarmKeyStroke],
['input', handleInput],
['blur', handleBlur],
['compositionend', scheduleResync],
];
for (const [type, handler] of listeners) container.addEventListener(type, handler, true);
return () => {
cancelResync();
for (const [type, handler] of listeners) container.removeEventListener(type, handler, true);
};
};