Skip to content

Open-source Grok Bot alternative for named AI teammates. Local-first desktop, web, and mobile clients with per-bot tools, instructions, and memory. Independent fork of T3 Code.

terminal plan v1

Julius Marminge877ab769345a
diff --git a/.plans/terminal-runtime-state-plan.md b/.plans/terminal-runtime-state-plan.md
new file mode 100644
index 000000000..0b90488b0
--- /dev/null
+++ b/.plans/terminal-runtime-state-plan.md
@@ -0,0 +1,154 @@
+# Terminal Runtime And State Plan
+
+## Summary
+
+Terminals appear to "hang" because PTY processes are exiting immediately at startup in the current runtime.
+
+Repro in this branch:
+
+- `node` + `node-pty`: interactive shell stays alive and accepts writes.
+- `bun` + `node-pty`: shell exits immediately (`exitCode: 0`, `signal: 1`), so the UI opens a terminal pane but no live process remains.
+
+This means the current breakage is primarily a runtime compatibility issue, plus missing reconciliation between desired terminal state and observed process state.
+
+## Root Cause (Current)
+
+1. Server dev runtime uses Bun (`apps/server/package.json` `dev` script).
+2. Terminal PTY implementation depends on `node-pty`.
+3. Under Bun, spawned PTY shells exit immediately.
+4. Frontend still mounts terminals, so UX looks like startup hang/blank terminal.
+
+## Design Goals
+
+1. Do not store full terminal output in SQLite.
+2. Preserve terminal topology and intent across reconnects/restarts:
+ - open/split/new/close
+ - active terminal/group
+ - cwd/env profile
+3. Keep runtime truth accurate even when processes change outside the app.
+4. Support deterministic recovery and idempotent command handling.
+
+## Proposed Architecture
+
+### 1) Runtime Boundary
+
+- Run terminal PTY host on Node runtime only.
+- If main server remains Bun, isolate PTY behind a Node sidecar process (JSON-RPC/IPC).
+- Treat PTY host as volatile runtime, not source of durable truth.
+
+### 2) Event-Sourced In SQLite (Control Plane Only)
+
+Persist only control/state events, not output payloads.
+
+Recommended event families:
+
+- `terminal.layout.updated`
+ - terminal ids, split groups, active terminal id/group id
+- `terminal.open.requested`
+ - threadId, terminalId, cwd, envHash/envProfileId, cols, rows
+- `terminal.close.requested`
+ - terminalId, deleteHistory
+- `terminal.resize.requested`
+ - cols, rows
+- `terminal.clear.requested`
+- `terminal.input.sent`
+ - input metadata only (`byteLength`, optional hash, optional commandBoundary flag), no raw payload
+- `terminal.runtime.started`
+ - pid, startedAt, runtime instance id
+- `terminal.runtime.exited`
+ - pid, exitCode, exitSignal, observedAt, reason (`normal|killed|external|crash|unknown`)
+- `terminal.runtime.error`
+ - normalized error code/message
+- `terminal.output.segment.recorded`
+ - segment file id + byte offsets only
+
+Use idempotency keys/command ids for each client command so reconnect retries are safe.
+
+### 3) Output Storage In Log Files (Data Plane)
+
+- Store raw output only in append-only files (per thread+terminal).
+- Rotate into segments (size-based and/or time-based).
+- Keep lightweight SQLite index rows:
+ - `segment_id`, `thread_id`, `terminal_id`, `start_offset`, `end_offset`, `created_at`, checksum.
+- Optional retention policy (max bytes / days / segment count per terminal).
+
+This keeps SQLite small while preserving replay capability.
+
+### 4) Reconciliation Loop (Runtime Truth)
+
+Do not treat client dispatch as sole truth.
+
+Maintain dual state:
+
+- Desired state: from event stream/projection.
+- Observed runtime state: from PTY host watchers (process exit, health checks, poll).
+
+On mismatch, emit runtime observation events:
+
+- process missing but desired running -> `terminal.runtime.exited` with reason `external|unknown`
+- process exists but projection says closed -> terminate process and emit correction event
+
+This makes out-of-band kills/restarts explicit and recoverable.
+
+### 5) Startup/Recovery Flow
+
+1. Rebuild terminal projection from SQLite control events.
+2. Load terminal layout/topology immediately for UI.
+3. Reconcile each desired running terminal with runtime:
+ - if process alive: attach stream + tail recent output from logs
+ - if process dead: mark exited; auto-restart only if policy says so
+4. Resume incremental output from last persisted offset.
+
+## Data Model Additions
+
+Add dedicated tables (or equivalent repos):
+
+- `terminal_control_events` (or reuse orchestration_events with terminal aggregate kind)
+- `terminal_output_segments` (index only, no raw output)
+- `terminal_runtime_snapshot` (latest observed runtime state cache)
+
+Avoid embedding output blobs in projection snapshots.
+
+## Implementation Plan
+
+### Phase 0: Stop the bleeding
+
+1. Run PTY on Node runtime in all dev/prod paths (or ship Node sidecar now).
+2. Add startup diagnostics: runtime, PTY spawn result, immediate-exit telemetry.
+3. Surface clear UI error when PTY host unavailable.
+
+### Phase 1: Durable control events
+
+1. Introduce terminal aggregate + commands/events.
+2. Persist layout and terminal lifecycle intent events.
+3. Build projection for terminal UI state from events.
+
+### Phase 2: File-backed output + index
+
+1. Implement segmented append-only output logger.
+2. Persist only segment metadata/offsets in SQLite.
+3. Update terminal history restore to tail from segments (bounded bytes/lines).
+
+### Phase 3: Runtime reconciliation
+
+1. Add process observer + periodic verifier.
+2. Emit runtime observation events on drift.
+3. Add recovery tests for out-of-band process termination.
+
+### Phase 4: Hardening
+
+1. Backpressure and truncation policies for high-volume output.
+2. Crash-safe fsync/flush strategy for segment writes.
+3. Retention and compaction jobs.
+
+## Testing Strategy
+
+Backend tests should cover:
+
+1. Rebuild projection from control events (open/split/close/resize/active terminal).
+2. Runtime reconciliation when process is killed externally.
+3. Recovery after restart with existing segment files.
+4. Idempotent command handling on duplicate dispatches.
+5. Segment index consistency and retention behavior.
+
+Use real runtime-layer tests for terminal adapter behavior where possible; only mock external process/service boundaries.