Open source CapCut alternative.
TypeScript
docs: describe judged CLI turns
David Le1c0e095ebef5
diff --git a/docs/guides/cut/ai-assistant.md b/docs/guides/cut/ai-assistant.mdindex f8470d58..e541eba5 100644--- a/docs/guides/cut/ai-assistant.md+++ b/docs/guides/cut/ai-assistant.md@@ -31,15 +31,15 @@ user message + @attachments + fresh <editor_state> snapshotThe Mac engine serves native Bun Request/Response streams and drains each turn into a disk journal independently of its readers. Reopening an unfinished turn replays that journal under the same assistant message id; saved completed replies stay settled. Stop explicitly cancels the provider. The provider sees a single MCP server named `cut`: a small stdio proxy the provider harness spawns, which forwards every tool listing and call to the engine over HTTP. The engine writes each call into the open chat stream; the tab executes it on the editor store and posts the result back; the engine hands it to the waiting provider. A call the tab never answers times out after two minutes.-A Claude or Codex turn can outlive its tab while the Mac engine remains running. The bridge retains the session's project after its last reader disconnects and executes subsequent document tools against that project on disk. Running threads keep their editor connection when the chat panel is hidden or another thread is selected. Tool claims let one attached tab execute each call; a disconnected call with an uncertain result returns that uncertainty to the model. Headless calls serialize access to the engine's editor store, and every call records its result in the journal. The Next.js development server invokes the same executor through Bun, which can load the shared editor modules. Tools that need the page's media primitives or hosted sign-in still return a typed error when detached.+A Claude or Codex turn can outlive its tab while the Mac engine remains running. The bridge retains the session's project after its last reader disconnects and executes subsequent document tools against that project on disk. Running threads keep their editor connection when the chat panel is hidden or another thread is selected. Tool claims let one attached tab execute each call; a disconnected call with an uncertain result returns that uncertainty to the model. Headless calls serialize access to the engine's editor store, and every call records its result in the journal. Tools that need the page's media primitives or hosted sign-in still return a typed error when detached.-Gemini's interactive turn runs in the browser. Its active turn survives panel and project navigation within that page, saves its final transcript after unmount, and waits for its original project before executing another editor tool. Opening another project waits for tools already using the store. Closing the browser tab ends this loop; the cloud worker's queued-turn API is a separate execution path. Project residency stays fixed throughout these transitions.+Gemini's interactive turn runs in the browser. Its active turn survives panel and project navigation within that page, saves its final transcript after unmount, and waits for its original project before executing another editor tool. Opening another project waits for tools already using the store. Closing the browser tab ends this loop; the cloud worker's queued-turn API is a separate execution path.A turn the page lost resumes itself when the thread next opens, replaying the engine journal or sending the model a continuation that finishes the ask with the failure gone from the transcript.The Gemini path skips the engine entirely — the same carve-out as AI media generation. The loop runs on the pi agent harness (the `@earendil-works` packages) in the browser: a custom transport speaks Donkey's hosted responses route with the user's session, tools execute directly against the store, and each thread's conversation persists as the harness's own structured history — past tool calls replay as real tool records, so no bookkeeping rides as prose the model could imitate. Gemini's quirks live in that transport: thought signatures replay exactly with each call, tool media returns as its own following turn, and an empty round retries before it surfaces as an error.-The turn's route is one typed judgment: a System One judge scores the ask's size, the skill to open first, and the tool areas it needs, and code applies the thresholds in the `cutJudge` setting. The first round waits a bounded moment for it, then runs the light chat model with only the routed areas' tools; a simple verdict keeps the run, chat or complex restarts it routed (a composed job gets its skill attached), and a `request_tools` call widens the catalog for the next round.+The turn's route is one typed judgment: a System One judge scores the ask's size, the skill to open first, and the tool areas it needs, and code applies the thresholds in the `cutJudge` setting. The first round waits a bounded moment for it, then runs the light chat model with only the routed areas' tools; a simple verdict keeps the run, chat or complex restarts it routed, and a `request_tools` call widens the catalog for the next round. With credits, Claude and Codex turns take the same judgment, which narrows the engine's MCP listing; Codex reads tools once per run, so a widening resumes it. Without credits the CLI decides.The same loop code runs with no browser at all. A headless session opens a project doc into the editor store, binds both transports at an explicit origin with auth headers, and runs the production chat loop against it — which is how a chat turn becomes a durable job. The page or an API caller posts a thread's messages to the cloud project's turns route; the row is the project's agent write lease (one turn at a time per project); the worker container claims it, runs the turn under a runner grant that spends that user's credits and writes that user's project, pushes the doc back through the versioned PUT, and appends the finished exchange to the stored thread.@@ -60,8 +60,8 @@ Every knowledge surface is defined once — the catalog file ships in the engine| Surface | Size today | When it enters context || --- | --- | --- || System prompt | ~13K chars (~3K tokens) | Claude: replaces the Agent SDK default. Codex: prepended to the first turn (a resumed session already has it). Gemini: the instructions field, every round. |-| Tool catalog (127 tools in 19 areas) | ~140K chars (~35K tokens) whole | Claude/Codex: the MCP tool listing. Gemini: the core plus the routed areas per request. |-| Skills library (16 docs) | ~97K chars (~24K tokens) total | Gemini: the judge's pick rides the turn. Claude/Codex: list-skills and read-skill. |+| Tool catalog (127 tools in 19 areas) | ~140K chars (~35K tokens) whole | The core plus the routed areas per turn. |+| Skills library (16 docs) | ~97K chars (~24K tokens) total | The judge's pick rides the turn; list-skills and read-skill cover the rest. || Editor snapshot | Grows with the project; media list and subtitle cues capped at 60 each | Attached to the newest user message as `<editor_state>`, rebuilt fresh every turn. || Attachments | Metadata JSON per asset | `<attached_assets>` on the message that carried them; on the Gemini path the newest message also carries the actual payloads (frames, images, text contents). || Full state | Uncapped | The get-state tool — the model calls it when the snapshot is stale or truncated. |@@ -74,11 +74,11 @@ Folder and note mentions keep their identity in saved chats and resolve on everyDecisions that gate, route, or pick (the turn route, queue triage, stock ranking, filler words, a voice, a provider's failure) are typed judgments from the judge route, thresholded in code. Reasoning and generation stay on the chat model.-The judgment that routes a turn also asks whether the whole turn is one known editor action, and fills its arguments in the same request. When it settles — mute this clip, make it 16:9, slow that one down — the editor runs the tool and writes the line with no model round, landing in about a third of a second. It settles only when the action and every argument clear their floors and four guards hold: the ask names no exact figure, touches no second item, leaves no second edit undone, and wants no shot made again. Anything short runs the ordinary loop, so a miss costs nothing. Claude and Codex run the user's own CLI, so their turns always take the model round.+The judgment that routes a turn also asks whether the whole turn is one known editor action, and fills its arguments in the same request. When it settles — mute this clip, make it 16:9, slow that one down — the editor runs the tool and writes the line with no model round, landing in about a third of a second. It settles only when the action and every argument clear their floors and four guards hold: the ask names no exact figure, touches no second item, leaves no second edit undone, and wants no shot made again. Anything short runs the ordinary loop, so a miss costs nothing.Finding the moments in a long source worth clipping is the same shape at scale: a cheap sweep scores every stretch of the transcript, the best earn three closer questions — does it stand alone, does the opening hold, does it finish — and code composes the weights and picks the winners apart from one another. One clip can be two stretches that were never next to each other.-A turn built on footage is judged once more before it closes: the judge reads what it watched, wrote down and ran against the ask, and a turn holding a slice of a source while about to describe the whole of it is handed its next pass — which source, from where, at what sampling floor — instead of its reply. The hold has to point at something the record measures: seconds nobody looked at, or calls that did not land. A doubt with no gap behind it lets the turn close, because a finished turn sent back does its work a second time.+A turn built on footage is judged once more before it closes: the judge reads what it watched, wrote down and ran against the ask, and a turn holding a slice of a source while about to describe the whole of it is handed its next pass instead of its reply. The hold has to point at something the record measures: seconds nobody looked at, or calls that did not land. A doubt with no gap behind it lets the turn close, because a finished turn sent back does its work a second time.Deciding what the user wants is prompt text, executed by the model. The prompt orders the calls it must make each turn: