The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.
Rust ffmpegfree-video-editormodel-context-protocolnon-linear-editoroffline-firstopen-source-video-editorself-hostedvideo-editingvideo-editing-automationvideo-editing-softwarevideo-editing-toolsvideo-editorvoice-cloningyoutube-automationyoutube-shorts
Delete ARCHITECTURE.md
Jareer99f2293fc48b
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.mddeleted file mode 100644index 66ca2558..00000000--- a/ARCHITECTURE.md+++ /dev/null@@ -1,904 +0,0 @@-# Concat: architecture--Concat is a video editor. It is one native Rust binary: a headless engine-that owns the edit, the pixels and the sound, and a Slint window that draws-that engine and hands it commands. FFmpeg is linked, whisper.cpp and-sherpa-onnx are compiled in, and nothing is spawned at run time.--This document is the map. It says what each part is for, which rules hold-across parts, and where the work goes next. Every crate's `lib.rs` opens-with a `//!` block that says the same thing at closer range; read this-first, then those.-------## 1. The one rule--**The engine owns the edit. The window is a view of it.**--The project, its operations, its undo history, every pixel of every frame-and every sample of the mix are decided in the engine crates, which have no-window and no idea one exists. The window reads the engine's project to-draw the bin and the lanes, and writes every edit as a command. It keeps-what the document does not need: selection, playhead, zoom, tool, which-lanes are locked, the arrangement of panes. None of that reaches the file.--Two consequences of the rule shape everything below:--- **One definition per meaning.** The speed range, the arithmetic that maps- a timeline instant to a source instant, the placement of a picture in the- frame, the mix's gain curve: each exists once, and the preview, the- export and the command layer all call it. Where two paths could disagree,- the code names the one they share and says why.-- **Degrade, never abort.** A source that runs out contributes nothing; an- unknown transition renders as a cut; a filter chain the catalogue refuses- leaves the picture untouched; a GPU that dies mid-export hands the rest to- the CPU; a document with vanished media opens without those clips. An- export or a session never ends in a panic because a file was strange.-------## 2. The map--Everything lives in `engine/`, one Cargo workspace of fourteen crates.--| Crate | Owns | Lines |-|---|---|---|-| `concat-core` | Exact time, arenas and handles, the RGBA frame, the timeline model the renderer reads. No dependencies. | 2.2k |-| `concat-project` | The document: model, every operation as a command, undo, the tolerant reader and compatible writer. | 4.7k |-| `concat-effects` | Effect packages: manifests, the FFmpeg template language, WGSL shaders, the catalogue. 45 built-in packages. | 2.3k + packages |-| `concat-media` | FFmpeg, linked. Decoding, seeking, the audio mix planner, encoding, waveforms, the reader pool. | 3.7k |-| `concat-render` | A timeline plus an instant into one frame: the plan, the CPU compositor, the wgpu compositor. | 1.9k |-| `concat-export` | Timeline to file. Transitions resolve here; the frame loop and the one-pass mix; the paused monitor's true frame. | 2.2k |-| `concat-text` | Titles as pixels: font lookup, shaping, rasterising onto a frame-sized transparent canvas. | 0.6k |-| `concat-vision` | What the engine sees in a picture: the person mask behind a cutout, brush strokes over it, the frame cut by it; the model itself behind `infer`. | 0.9k |-| `concat-host` | What a window needs that is not the edit: sessions, project folders, caches, the monitor, audio playback, the export driver, templates, job slots, app directories. | 3.8k |-| `concat-speech` | Transcription with whisper.cpp and text to speech with Kokoro, both in-process, models downloaded on demand. | 1.2k |-| `concat-api` | The Concat API: one dispatcher of JSON requests over the host - projects, edits, media, the catalogue, templates, exports, frames - that every transport calls. | 0.9k |-| `concat-cli` | A driver for the engine without a window: `probe`, `render`, and `api`, the API over stdin and stdout. | 0.3k |-| `concat` | The editor window in Slint, and the Rust that binds it to the host. | 7.2k Rust, 15.9k Slint |-| `concat-android` | The Android activity: `android_main` over the window. | 20 |--The dependency arrows point one way:--```-concat ──► concat-speech ──► concat-host ──► concat-export ──► concat-render ──► concat-core- │ │ │ │ ▲- │ │ │ ├──► concat-effects ──► concat-project ──► concat-core- │ │ │ └──► concat-vision ──► concat-project, concat-core- │ ├──► concat-media ◄──┘- │ ├──► concat-text- │ └──► concat-vision (with `infer`: the model runs here)- ├──► concat-effects (the catalogue, for the inspector's shelves)- └──► concat-vision (a press on the stage, mapped into the source)--concat-android ──► concat concat-api ──► concat-host, concat-export, concat-effects, concat-project- concat-cli ──► concat-api, concat-core, concat-media, concat-render-```--`concat-core` depends on nothing; `concat-project` is not folded into it-because the document needs serde and core's zero-dependency rule is worth-more than the adjacency. Five crates carry no native library and build for-the web as well (`concat-core`, `concat-project`, `concat-effects`,-`concat-render`, `concat-text`); CI keeps that true. `concat-vision`-links ONNX Runtime behind its `infer` feature, prebuilt for every target-the app ships on, with the platform's accelerator registered ahead of the-CPU; without the feature it is pure Rust, and that is how the renderer-takes it.-------## 3. Vocabulary--The types every crate agrees on live in `concat-core`.--**Time is exact.** `time::Rational` is a reduced `i64/i64` in seconds.-Arithmetic widens to `i128` and reduces back; a value that comes from a-file goes through `checked_new`, which errors rather than panics, so a-hostile container cannot crash the engine. `approximate` is the one seam-through which an `f64` enters, at microsecond precision, and `as_f64` is-one-way out. `FrameRate` is exact too, and `frame_at` floors, so frames are-half-open intervals: 107,892 frames of 30000/1001 accumulate to exactly-where they should. Nothing in the engine keeps a timestamp as a float.--**Graphs use handles.** `arena::Arena<T>` hands out generational-`Id<T>` handles that are `Copy`, `Send` and `Sync` whatever `T` is. Removing-a slot bumps its generation, so a stale handle reads as empty rather than-as whatever now occupies the slot. The timeline's tracks and clips are two-arenas plus an explicit bottom-first `order`.--**One pixel format.** `frame::Frame` is RGBA8, straight alpha, sRGB,-tightly packed, with bounds-checked access that returns `None` and a-`Debug` that never dumps pixels. A higher-precision frame arrives as a-second type beside it, not as a `format` field.--**The timeline the renderer reads.** `timeline::Clip` carries start,-duration, source in-point, speed or a `SpeedCurve`, reverse, transform,-opacity, blend, fades and an `Animation`. `Clip::source_time_at` is the-one piece of arithmetic every trim, slip and retime has to agree with:-affine when the speed is constant, the curve's area when it is not,-mirrored when reversed. `is_paced` says whether a decoder can be pulled-one frame per output frame or must seek. `Blend` documents its six-premultiplied formulas so the CPU and GPU compositors agree by-construction. `retime::SpeedCurve` clamps to 0.0625..16 and yields the-constant-rate pieces a sound path that can only change tempo in steps-needs. `animate::Animation` is relative: scale multiplies, offsets and-rotation add, opacity multiplies.--**Two models, one boundary.** The document model in `concat-project` is-not the core timeline. It speaks the file's terms: `f64` seconds, `String`-ids, camelCase serde, the audio side that core's clip lacks (volume, fades,-pitch preservation, filter chains), text and layer clips that have no-media. The conversion from the document to `concat_core::Timeline` happens-in exactly one place, `concat-export`'s timeline builder, and the-`f64`-to-`Rational` step happens there. Two small converters inside-`concat-project` (`speed::curve_of`, `animation::animation_of`) are the-only other crossings.-------## 4. The edit: `concat-project`--Two rules are stated at the top of the crate and hold throughout:--1. **The document format is frozen by the documents that exist.** Saved- projects load forever, tolerance rules included. `DOCUMENT_VERSION` is- bumped only when a change cannot be absorbed by defaulting.-2. **`f64` seconds and `String` ids are the document's terms**, until a- version 2 decides otherwise on purpose.--**Commands.** `commands::Command` is a serialisable enum of about thirty-operations, tagged `op` in camelCase: add and remove media, place, move,-trim, split, merge and slip clips, set speed and speed curves, transforms,-animations, text, effects, track state, timelines, and `Batch`. The-constants the window's gestures mirror are declared once here: the-minimum clip duration (1/60 s), the speed range (the engine's, verbatim),-the maximum offset, the join tolerance. `ClipPatch` distinguishes absent-from null. Ids are minted by one counter across the `m`/`c`/`t`/`tl`-prefixes, and a loaded document is walked so restored ids are never-reissued. `Outcome::applied` is honest: a command that changes nothing-reports so, and the undo history records no phantom edits. `Batch` applies-to a staged clone and commits only when every step succeeds. `SplitClips`-drops a speed curve because the map to the source is no longer affine;-`MergeClips` defers to `why_not_merge`, which returns the exact sentence-the user sees.--**Undo.** `editor::Editor` keeps whole-project snapshots in a `VecDeque`-capped at 200; `apply` pushes only when the outcome was applied and clears-redo.--**Reading and writing.** `doc.rs` defaults every field, drops clips whose-track or media vanished (text and layer clips survive), clamps hand-edited-values into range, and loads a flat single-timeline document as one-timeline. The writer emits both `timelines` and a flat mirror of the-active one, so an older build still opens a new file.--**Named shapes.** Speed presets and animation presets are stored by name-and re-materialised per clip length; an animation slot longer than the-clip is squeezed, an overlapping in/out pair is split in proportion.-------## 5. Effects: `concat-effects`--An effect is a folder. `effect.toml` declares an id of the form-`author.name`, its kind (audio, effect, filter, transition, generator), its-parameters with ranges and defaults, and one or two backends;-`fixtures.toml` pins what it renders at default, minimum and maximum. The-45 built-in packages under `packages/` are compiled into the binary by-`build.rs`, which scans the directory: adding an effect is adding a-folder, and nothing in Rust names one. User packages load from a directory-at run time.--**Two backends.** `[ffmpeg]` is a filter-chain template with-`{expression}` slots, run inside the decoder's filtergraph. The expression-language is a deliberately tiny recursive-descent parser over numbers and-eleven fixed functions; it is the security boundary as much as a-convenience, since nothing an expression evaluates to can be anything but-a number. `[wgsl]` is a shader that writes one `fn effect(uv) -> vec4<f32>`;-`shader.rs` wraps it in a prelude (the layer texture, a `Frame` uniform of-size, time and intensity, the package's `Params`) and a postlude (the-full-screen triangle and an `fs_main` that mixes the effect by intensity),-validates it through naga, and reads the `Params` struct's layout back from-the compiled module so the parameter bytes are packed exactly as the GPU-expects. A package may carry both: the shader renders wherever there is a-GPU, and the chain is what a machine without one gets.--**The catalogue.** `Catalogue::builtin()` loads once, panics on a bad-built-in, and rejects duplicate ids or aliases. Loading a package renders-its chain at default, minimum and maximum, so a type error in an expression-is a load failure and never a silent gap in an export. `video_chain`,-`video_chain_gpu` (which skips shader-backed packages), `shader_passes`-and `audio_chain` all funnel through one `compose`, which numbers labels-by emitted index so a bypassed entry consumes none. A *filter* (a colour-look) answers to an undeclared `intensity` key: `mixed` rewrites the chain-as a split-and-blend so intensity is the look's opacity.--The document is untouched by any of this. A clip stores-`{ id, params, enabled }`, and an id the catalogue does not know is skipped-at render time.-------## 6. Media: `concat-media`--The only crate that knows FFmpeg exists. It links libavformat, libavcodec,-libavfilter, libswscale and libswresample through `ffmpeg-the-third`;-FFmpeg 7 is the floor because the display-rotation side data is read from-where 7.0 moved it. The crate's one `unsafe` block walks that side data.-`build.rs` names what the static archives on a phone need from the-platform (MediaCodec and the JNI on Android, VideoToolbox and its-frameworks on iOS); on the desktop the shared libraries bring their own.--**Two traits.** `FrameSource` (width, height, `next_frame`, `position`)-and `FrameSink` (`write_frame`, `finish`) fence everything codec-shaped. A-GPU-decode backend would implement those two and nothing else in the-workspace changes.--**Decoding.** `decode::Decoder` owns one input, one video decoder and one-filtergraph, no sharing and no interior mutability. The graph's order is-load-bearing: rotation, then the crop (in source pixels, since a crop-changes what the fit is *of*), then a bilinear scale to the requested-size, then the effect chain, then a guard scale back to that size, then-`format=rgba`. Every chain is validated before anything opens: a chain-containing `;`, `[`, `]` or a newline is no longer a filter applied to this-clip. Seeking is two-phase and frame-accurate: a container seek to the-target, then frames are discarded until the one whose timestamp reaches it.-`next_paced` keeps a frame around a target instant, duplicating and-dropping to hold a rate; `looping` repeats a still forever.--**The audio planner.** `audio.rs` plans a whole mix as one FFmpeg-filtergraph. Per clip: trim, speed (pitch-preserving `atempo` stages, or-the tape path of `asetrate` between resamples), the user's chain, volume,-fades, then `adelay` to its place on the timeline; then `amix` without-normalisation, or `anull` for one clip; then a resample and pad that fix-the timestamps `atempo` leaves undefined. A twenty-line comment explains-why that last stage exists and ends with the instruction not to simplify-it away. `mix_to_file` opens every input, feeds them round-robin and-drains after every push so no input runs ahead of the others by more than-a frame; `mux` stream-copies picture and sound together. The speed range-is defined here, once, and preview and export both read it.--**Encoding.** `encode::Encoder` defaults to `libx264`, medium, CRF 18,-`yuv420p` (4:4:4 will not play in browsers), writes with `+faststart`, and-keeps both the encoder's and the muxer's time bases for the rescale. A-frame of the wrong size is an error, not a resize. `jpeg` is the poster-encoder.--**Waveforms.** `peaks.rs` streams a mono 48 kHz decode into buckets-without accumulating samples, so an hour costs what a jingle costs, and-encodes a frozen little-endian cache format.--**The reader pool.** `pool::ReaderPool` is playback's infrastructure and-explicitly not export's: export decodes every frame once in order, and a-cache there is pure overhead. The pool is a byte-budgeted LRU of frames-keyed by file, instant, size and chain (the same source frame through two-chains is two pictures) over a bounded set of warm readers. Its ownership-rule: a reader is *found* under the pool's lock and *used* under its own,-so decodes on different files never queue behind each other. Asked for an-instant, it rolls forward up to 60 frames before it seeks, clamps a request-past the end to the last frame, and on failure retries seeks backwards by-widening steps, so a file with any decodable picture cannot fail here.--`treat.rs` is the decoder's filtergraph without the decoder: RGBA in,-chain, RGBA out, for a layer clip that has no pixels of its own.-------## 7. Rendering: `concat-render`--Rendering splits in two, and the split is the crate.--1. **`plan`** answers what is on screen at this instant, from where, at- what opacity, under what transform. It touches no pixels and does no IO,- so it is fast, exactly testable, and identical for both backends. The- fade ramp multiplies into the opacity here, so a compositor only ever- sees a per-frame opacity and has no idea fades exist.-2. **`compositor`** takes the plan plus decoded pixels and blends them.--**The CPU compositor is the reference.** Placement is in pixels, converted-exactly once by whoever builds the layers. Aligned layers blend by region;-transformed layers are inverse-mapped with bilinear sampling over the-rotated bounding box, which is what makes the result hole-free at any-scale or angle. The blend table is the fixed-function set the GPU offers,-over premultiplied colour, so the two paths agree. Output is forced opaque.--**The GPU compositor exists to be fast and must match it.** `gpu.rs`-renders on wgpu into `Rgba8Unorm`, not sRGB, so blending happens on-gamma-encoded values exactly as on the CPU; when the day comes to blend in-linear light, both backends change together. Six pipelines, one per blend-mode, since blend state is baked into a pipeline. Layer textures are pooled-by size and retired after 300 idle composites. Effect shaders arrive as-`ShaderPass` data (a key, WGSL source, packed params, intensity); each is-compiled once per key and run as its own submission over ping-ponged-textures. Two outputs: `composite` reads back through a staging buffer,-and `composite_texture` leaves the result in a presentable ring for a-window that shares the device, which is how the monitor shows a frame-without a copy anywhere. Construction returns `None` without an adapter,-and a compositor that fails marks itself dead and hands every later frame-to the CPU.--**Parity is tested.** `assert_matches_cpu` composites the same layers on-both and asserts per-channel tolerance, at interior pixels for rotation-and scale where the two rasterisers legitimately differ on the half-pixel-border. The shader test compiles a real package through the catalogue's-stitching, so what the test runs is what a package runs. Every GPU test-passes silently on a machine with no adapter.-------## 8. Export: `concat-export`--The seam where the document's flattened clip list becomes the engine's-timeline, and the engine decides everything after.--**Flatten.** `flatten_timeline` walks a document timeline into-`ExportClip`s: track index and media resolved, track state folded into-per-clip flags, filter chains built through the catalogue. Text clips are-left out here and rejoin as still images once painted (section 9). Layer-clips flatten to a pixel-less entry carrying their effects. Missing media-is a skip, not an error.--**Transitions resolve first.** `resolve_transitions` runs before anything-reads the clip list, so the picture and sound paths never know transitions-exist. Every track index is doubled so a cross-fade's incoming clip gets an-odd lane directly above the pair; the incoming clip is extended backwards-by the transition's duration, clamped to the outgoing duration, the-incoming duration and the available source handle, and given a video fade-in with the sound riding the picture. Fade-to-black and fade-to-white-become frame-based fade filters appended after the clip's own effects.-Adjacency tolerance is half a frame at the output rate. An unknown kind is-a cut.--**The frame loop.** One `Decoder` per paced clip, opened at its in-point at-`fps / speed`, so pulling exactly one frame per output frame keeps each in-step with the plan's source times without seeking. Unpaced clips (a speed-curve, reverse) go through a `ReaderPool`; stills repeat. Decoders retire-the frame their clip leaves the plan. `place_layer` is the one definition-the exporter and the preview share of where a picture goes in the frame.-Layer clips composite the stack beneath them, run their shader passes or-their FFmpeg treatment, and mix the result back by strength, ramped by-their fades. Progress reports every fifteen frames and at each stage, and a-cancel flag is checked at every frame. On the GPU, effects with a shader-are omitted from the FFmpeg chain and carried as passes instead.--**Sound is one pass.** Speed curves are cut into between 8 and 400-constant-rate pieces of about a tenth of a second, each at the curve's-local mean, with the clip's fades projected onto each piece; the pieces,-the paced clips and the unmuted video clips' own audio go to-`audio::mix_to_file` as a single filtergraph. Inputs with no audio stream-are filtered out by a probe first, because FFmpeg refuses a graph that-names a stream that is not there. Picture and sound are written to scratch-siblings beside the output and muxed at the end; the scratch is removed-whatever the outcome.--**The true frame.** `preview_frame` and `preview_sources` build the same-timeline the export builds, quantise the requested instant onto the frame-grid, and pull from the pool. `quantise` is the seam where a value stops-being approximate. Two invariants are pinned: fade-to-colour transitions-are not baked for the preview, because their frame numbers assume a decode-from the clip's start that pooled seeks break, and the window draws a-matching veil instead; and a plan with layers but nothing decoded is an-error rather than a black frame, because a caller would draw that "truth"-over its own perfectly good approximation. `PreviewSources` exposes bare-layers so a GPU caller can composite on the window's device.-`preview_prefetch` warms the pool one frame apart, which is exactly what-keeps its readers rolling forward instead of seeking.-------**Cutouts happen on the decoded frame.** A clip with a cutout names the-directory its media's masks live in (the flattener fills it in when it-knows the project folder), and the frame loop asks `concat-vision` for the-mask nearest the layer's source instant, with the clip's strokes painted-on and its feather applied, and multiplies the decoded frame's alpha by-it through the clip's crop and flips. That happens before the frame-becomes a layer, so both compositors draw the same cut pixels, and an-instant with no mask yet draws the picture as shot.--## 9. Titles: `concat-text` and `host::titles`--A text clip is a style and some words; the compositor wants a picture.-`concat-text` finds the face (the system's fonts plus the project's own-files, falling back rather than failing), shapes each line with rustybuzz,-turns outlines into paths, and paints plate, shadow, outline and fill with-tiny-skia onto a canvas the size of the output frame, transparent-everywhere the words are not. Frame-sized on purpose: the compositor fits a-picture into the frame and applies the clip's transform about its centre,-so a canvas that *is* the frame fits at exactly one, decodes without-resampling, and puts the block's centre where the clip's centre is. A-title's offset and rotation then mean the same as footage's. Every size in-the style is a fraction of the frame's height, converted to pixels once at-the top, so a title looks the same at 720p and 4K.--`concat-host`'s `titles` module is the cache and the rejoin: each text-clip becomes an image `ExportClip` with the text clip's timing, transform,-keyframes and opacity, pointing at a PNG under the app's data directory-plus a JSON sidecar holding the block size the PNG cannot express. The file-is keyed by the style, the frame size, the project's fonts and a version-constant, so moving a title is free and only a change to the words or the-look repaints.-------## 10. Cutouts: `concat-vision` and `host::cutout`--A cutout takes a picture's background away without a key colour. Three-models, all run by ONNX Runtime, all one interface in `segment`: a frame-in, a probability per pixel out. Robust Video Matting is the person-model, at the picture's own aspect on a 512 edge, carrying a recurrent-state between frames so an outline holds still; IS-Net is the object-model, 1024 square, for a car or a cartoon; MediaPipe's selfie-segmentation is compiled in and answers when nothing has been-downloaded. A clip's `Subject` picks between the first two, and on-automatic the person model looks at the first frame and hands over to the-object model when it finds no one; the choice is written into the mask-store, so every run agrees. Every position in a mask is a fraction of the-source picture whatever the mask's shape, which is also how strokes are-stored; a crop, a flip or a change of output size changes nothing about-either, because `apply::Mapping` is the one place a decoded pixel is-walked back to a source fraction. A mask the model left blank shows the-picture as shot rather than nothing.--**The smart brushes name a thing, and follow it.** A stroke records the-source instant it was painted at; for a smart brush or smart eraser,-`host::brush` decodes that frame, runs SlimSAM's encoder on it and its-decoder on sixteen points along the stroke, and then walks the clip's-analysed instants forward and backward from there, each frame prompted-with points from inside the region the frame beside it had and keeping-the candidate that overlaps it most, so the region moves with its thing.-The regions go beside the media's masks, one per instant in a directory-named by a hash of the stroke; where the thing is lost the rest of the-run gets an empty region, so the grid is whole and nothing is asked-twice. `strokes::paint` keeps or drops the instant's region whole; until-one has been read, or when the model is not to hand, the smart tools-fall back to the mask's own confidence under a disc. The plain brush and-eraser are discs and nothing more.--**The store is the project's.** Each media file's masks are PNGs in-`cache/masks/<hash>-<model>/`, one per analysed source instant, named by-that instant in milliseconds, ten a second of footage and one for a-still. The file names are the index. The renderer reads them; the host's-`cutout` module writes them, decoding the media at the model's size along-each missing stretch and running the model on a batch of frames at once,-one media at a time through a `SingleFlight`. The window notices which-media want masks they lack after every change and starts the next-analysis when the last reports back; the picture is shown as shot until-its masks arrive, and the inspector says how far along they are.--**Custom is automatic plus strokes.** A brush stroke is a tool, a-diameter and a path, painted as discs onto the model's mask: the brush-and eraser do exactly what was painted; the smart brush keeps only what-the model gave some chance of being the subject, the smart eraser removes-only what it was unsure of. Each stroke is one command and one undo step.-The feather is three box blurs over the resolved mask, and the resolved-mask is cached per instant and settings so a paused monitor never-recomputes it.-------## 11. The host layer: `concat-host`--What the window needs that is not the edit. Nothing here knows about a-window; long work reports through callbacks and cancels through flags, and-the caller decides which thread it runs on. If a function here starts-deciding what an edit means, it belongs in `concat-project`.--**A project is a folder.** It holds `concat.json` (written at creation, so-a project is a real thing from the start), `cache/` (waveform peaks, the-poster, decoded audio for playback), `audio/` (narration the timeline-points at) and `assets/` (media a template brought). Saving writes a-sibling and renames it, so a truncated write loses nothing. Recents are a-machine-scoped list of twelve, filtered on read so an unplugged drive's-projects come back when it does.--**Session.** `Session` is a path, settings and an `Editor`. `apply`,-`undo` and `redo` return an `EditorView` (the project, undo and redo-availability, settings, the id a command created), so the window keeps no-model of its own. A document that fails to parse is refused unless the-manifest is settings-only, because silently replacing an edit with-emptiness is how projects get lost. The session flattens itself through-`concat-export`.--**Media.** Import probes a file into a `MediaItem` that stores enough-metadata to open meaningfully when the file is missing. Waveforms are 200-buckets a second, cached beside the project under an FNV-1a key that is-pinned by a test, because a changed hash would orphan every project's-caches. Filmstrips are up to 60 slices blitted into one wide frame, one-texture upload instead of many. The poster is regenerated only when the-manifest is newer than it.--**The monitor.** `Monitor` holds the app's one `ReaderPool` and, when the-window created one, the shared wgpu compositor. With a GPU, the paused-frame's layers go straight to `composite_texture` and no pixel comes back-down, unless a layer clip is live at that instant, in which case the CPU-composites and one frame is uploaded. Prefetch is capped at eight frames so-a confused caller cannot park the pool on a long decode.--**Playback.** Four rules, stated in the module and kept: decode each-audible span once to disk PCM under a key of path, in-point, duration,-speed, pitch mode and chain (deliberately not volume, fades or position);-one cpal stream mixes the memory-mapped clips with gain applied at mix-time from the one `gain_at`; the device's sample counter is the only clock;-the stream is supervised, rebuilt on failure or a device change with-backoff. Threads: the output supervisor, a 33 ms transport publisher, two-decode workers popping the newest job first and skipping jobs the timeline-no longer wants, and a cache sweep guarded so sweeps never pile up. The-audio callback touches exactly one lock, and only with `try_lock`, because-it must never block. Position is stored *and* messaged on play and seek,-since a paused callback returns before storing.--Every lock in playback and titles recovers from poisoning rather than-propagating it: each guards a cache or a list that is still valid after a-panic elsewhere, and refusing it would end audio or titles for the session.--**Cutouts.** `Cutouts` holds the model, loaded on first use, and the-one-analysis slot; see section 10. `outstanding` says how many instants a-request still needs without doing anything, which is how the window-decides whether to start it.--**Export, templates, jobs, directories.** `Exporter` adapts a progress-closure into the engine's `Reporter` and runs under a `SingleFlight`,-which refuses a second concurrent job and gives each run its own cancel-flag, so a cancel can only stop the job that is running. Templates are-bundles under the config directory (a document with placeholder media, the-assets it needs, a poster); instantiating one refuses missing fills before-creating anything and fills every slot in one `Batch`, so a half-fillable-set of media leaves no half-made project. Deleting a template-canonicalises and checks the path is inside the library. `AppDirs` is the-platform's config and data directories under `app.concat.editor`.-------## 12. Speech: `concat-speech`--Transcription runs whisper.cpp in-process on a clip's audio decoded to-16 kHz mono, greedy sampling, threads capped at eight, with the job's-cancel flag wired to whisper's abort callback. Six models are known,-nothing above `small`. Text to speech runs Kokoro through sherpa-onnx's C-API, 36 voices, English and Chinese (what the lexicons cover), and writes a-WAV into the project's `audio/` folder because a clip will point at it.-Both download their models on demand into the app's data directory, via a-blocking HTTP client whose read timeout is what makes the cancel flag-reachable, unpacking through a staging folder that refuses paths escaping-it, and renaming into place only when complete. Both are one-at-a-time-through `SingleFlight`. The crate is separate from `concat-host` so its-native libraries stay out of everything that does not speak.-------## 13. The API: `concat-api`--The editor driven without its window. Every way of doing that - the-command line today, a daemon on a socket, an MCP server, a plugin - is a-transport, and a transport is a loop: read a `Request`, hand it to-`Api::dispatch`, write the `Response` and whatever `Event`s fired on the-way. What a request means lives in this one crate, so two transports-cannot disagree and a method added here reaches all of them.--**The shape.** `Request` is a serde enum tagged by `method`, named-`area.verb`: `project.create`, `project.open`, `project.close`,-`project.list`, `project.get`, `project.document`, `project.save`,-`project.setVideo`, `edit.apply`, `edit.undo`, `edit.redo`,-`media.probe`, `media.import`, `catalogue.list`, `template.list`,-`template.instantiate`, `template.save`, `export.run`, `export.cancel`,-`preview.frame`, `version`. A response is `{"result": ...}` or-`{"error": "..."}`, the error being the sentence the window would show.-Events are tagged by `event`: export progress, and cutout analysis ahead-of an export. Fields are camelCase, the document's and the command-layer's spelling, so a caller learns one. `API_VERSION` names the-contract and moves only when a caller written against the previous one-would misread a reply.--**The edit is not redefined.** `edit.apply` carries a `concat-project`-`Command` exactly as the window sends one, clamps and refusals included,-so every operation the window can perform is one a script can perform-and a new command needs nothing added here. A `Batch` is one request and-one undo step. A text style in a command needs only the fields it sets.--**What the crate owns is the choreography** the window performs by hand.-`media.import` is the probe and the add. `template.instantiate` probes-every fill before anything is made, so a bad path refuses the request-rather than leaving a folder behind. `export.run` finds every mask the-timeline's cutouts still need (`Cutouts::requests`, the same list the-window asks for after each change), paints the titles, then renders, so-the file the API writes is the file the window would have written.-`preview.frame` is the paused monitor's true frame, titles included, as-a PNG.--**One `Api`, one session per project folder**, keyed by the folder's-canonical path, so `.` and its absolute spelling are one project.-A method on a folder that is not open is an error, never a silent open.-The `Api` is not thread-safe on purpose: a transport serving several-callers owns the one `Api` and serialises through it, as the window's-event loop does, and a long method blocks its caller. `Api::exporter` is-the one handle that crosses threads, so a cancel can reach a running-export.--**The first transport** is `concat-cli api`: one request per line on-stdin, one JSON object per line on stdout, events interleaved and-flushed as they happen, so a pipe from any language edits and exports a-project. A line that is not a request gets an error and the loop goes-on.-------## 14. The window: `concat`--**One library, three entry points.** `concat::run()` is the program.-`main.rs` calls it on the desktop and on iOS; `concat-android`'s-`android_main` installs Slint's Android backend and calls it.-`platform.rs` holds the three things a phone does differently: how the-backend is chosen, how a file or folder is picked, and whether there is a-title strip to drag. Everything else is one tree, one state, one set of-callbacks.--**Startup, in order.** The GPU device (`gpu.rs`) is acquired first, on-Metal, DX12 or Vulkan, because Slint takes it at backend selection; with a-device, the renderer and the engine's compositor draw on one device and a-monitor frame is a texture Slint shows as it is. Then `Host::start` brings-up playback, the monitor, the exporter, the transcriber and speech. Then-the `App`, the `Studio`, and the `Shell`: the weak window, the studio in a-`RefCell`, and the models, installed in a thread-local. Every Slint model is-handed over exactly once, since a fresh model is a reset that rebuilds-every row hanging off it. Then about 120 callbacks are bound through three-macros, each "mutate, then republish": the whole window, the lanes alone-(for the handlers a pointer drives as a stream), or the dock alone.--**`studio.rs` is the state.** Its fields group by the line in section 1:-the edit (`session`, the echo, dirty, autosave), the bin (rows with stable-ids, thumbnails, peaks, filmstrips), the view (lane locks and sizes,-selection, playhead, scroll, seconds per pixel, tool, snap, quality,-transport), the sheets (export, settings, menus, toasts), the launch screen-(recents, posters) and the workspace (the dock tree, gestures, stage-guides, drop plans). `project()` returns the echo when there is one, else-the session's project, else an empty one. `apply` sends a command to the-session, turns a refusal into a toast, and on success marks dirty, prunes-the selection, arms a 1.5 s autosave, resyncs audio (flattened clips into-playback's `ClipSpec`s through the same `audio_pieces` the export uses),-requests art and requests a preview.--**The echo.** A press clones the project; the pointer mutates the clone;-`publish` draws whichever exists; release diffs the two and emits exactly-one `MoveClips` or `TrimClip`. The inspector's knobs write the echo and-`clip_commit` diffs it into a batch; a commit with the same key within-900 ms undoes the previous step first, so a dragged knob is one undo. Stage-gestures composite the echo too, snapping against frame lines and other-pictures. Undo therefore undoes the gesture, never a pixel of it.--**`host.rs` is the bridge.** `Shell::with` reaches the state through the-thread-local, which is why it needs no lock: nothing but the event-loop-thread touches it. `spawn(work, then)` runs work on a thread and hops back-through `slint::invoke_from_event_loop`, then publishes, so a completion-never has to remember to redraw. Frames cross threads as `Frame`s, never as-Slint images. The preview is single-flight: one composite at a time, the-latest request wins, the echo is what gets composited when there is one,-and quality scales the frame by 1, ½ or ¼.--**The Slint tree.** `App` is a focus scope (shortcuts, and a click on-nothing blurs a field), the title bar, the start screen or the workspace,-then the export and settings dialogs, tooltips, the app menu and toasts;-declaration order is paint order. `Workspace` reports its own box up and-draws one `Seat` per leaf of the dock tree and one splitter per divider.-A seat is one of four panes (media, preview, inspector, timeline) with a-drop area that previews a split. The timeline pane is tabs, the tool tray,-then headers beside `Lanes`, which draws the bands, the tick grid, one-`Clip` per row (filmstrip tiles, a waveform path, fades, transition-wedges), the drop ghost, the razor guide, the ruler and, last, the-playhead. The preview pane holds the fitted stage with the selection-outlines, corner grips and rotation handle, the frame field, transport-and quality. Positions in the models are seconds, never pixels; drag-deltas are taken in window coordinates because a local delta measures-against its own moving origin; and the ghost is Rust's answer to the-hover, not the panel's guess.--Globals carry what threading would tangle: `Editor` holds every property-and callback the four views use, because two seats showing the same view-must agree; `Theme` is one `dark` switch over two palettes plus the type-roles, the size scale, radii and the content swatches no palette may-decide. `Paint` maps a clip kind to its mark and well, and the bin card-and the timeline clip wear the same two colours, so they read as one-object moved. `dock.rs` is the workspace tree, addressed by path rather-than index, with a default of library beside preview-over-inspector,-above the timeline. `chips.rs` draws drag chips as SVG so Slint rasterises-them with the window's fonts at the window's scale. `prefs.rs` is a small-JSON file of remembered choices. `presets.rs` is the Text page's looks: the-built-in ones, a folder of TOML files that adds more, and the font-installer that copies a preset's face into the app's `fonts/` once and-registers it on the project the first time the preset is placed. The-sheets - export, settings, the project's own facts, captions and speech --are structs on the studio published whole, the way the export sheet always-was, each with its worker reporting into it through `on_ui`. `sysinfo.rs` gathers one list of facts-that feeds both the About rows and the copyable block, so a fact cannot be-on the page and missing from the report. Fonts, the logo and the effect-previews are embedded by `build.rs`.--**The words.** Every string a person reads is looked up by its English:-`I18n.t("...")` in the tree, `t` and `tf` in Rust, and the names in effect-manifests and text presets on their way to the shelves. `i18n.rs` holds-the active locale - one JSON file of English key to translation, compiled-in from `locales/` or dropped by the user into the config directory's-`locales/` folder, where a file with a shipped code lays its lines over-the shipped ones - and answers each lookup from a map behind a read lock,-falling back to the key. The tree's `I18n` global reads one `lang`-property on every lookup, so Settings › Language is a single property-write and the window re-evaluates itself. `scripts/locales.py` regenerates-`en.json`, the inventory, from the source and checks every locale against-it; CI runs the check.-------## 15. Build and delivery--- **Toolchain.** Pinned in `engine/rust-toolchain.toml`. `cargo build`- needs the FFmpeg 7+ development libraries, cmake and a C++ toolchain- (whisper.cpp), and libclang (bindgen); sherpa-onnx's prebuilt libraries- are fetched by its crate. Skia needs fontconfig and freetype on Linux.-- **Profiles.** `dev` compiles dependencies at `opt-level = 2` because- decoding and compositing are unusable unoptimised. `quick` is release- without LTO, incremental, for trying a change. `app` is the shipping- binary: fat LTO, `panic = "abort"`, stripped. The workspace `Cargo.toml`- says what each knob costs and what is deliberately not set.-- **Renderer.** Skia by default; `--no-default-features --features wgpu`- swaps in FemtoVG over wgpu. The two are meant to be compared. Type- weights differ by one step between them because FemtoVG composites glyph- coverage without gamma correction.-- **Workflows.** `ci.yml` runs `fmt`, `clippy -D warnings` (which makes a- public item without documentation a failure) and the tests, plus the- web check of the five portable crates. `build-app.yml` builds the window- on six native runners (macOS arm64 and x86_64, Linux x86_64 and aarch64,- Windows x86_64 and arm64), bundles, signs and notarises on macOS, and is- called by `nightly.yml` on every green push and by `release.yml` on- every `v*` tag. `mobile.yml` cross-builds for Android and iOS.- `nix.yml` proves the flake, which builds the window with every native- dependency pinned on Linux.-- **Phones.** FFmpeg is cross-built from source by- `engine/scripts/ffmpeg-mobile.sh` as static archives with the platform's- hardware codecs (MediaCodec, VideoToolbox) and nothing else linked;- sherpa-onnx is fetched as k2-fsa's shared library by- `scripts/sherpa-mobile.sh`; `scripts/mobile-env.sh` turns what those- leave under `engine/vendor/` into the build environment and papers over- three build-script quirks it documents in place. `concat-android` is- packaged by cargo-apk; `scripts/ios-app.sh` lays out the iOS bundle.-- **Licensing.** The engine is AGPL-3.0-or-later with a plugin exception- (`LICENSE-EXCEPTIONS.md`); Slint is taken under its GPL-3.0-only option- (`THIRD_PARTY_NOTICES.md` says why the two combine).-------## 16. Testing--240 unit tests, all in-file `#[cfg(test)]` modules; there are no `tests/`-directories. The pure layers are where the coverage is dense: rational-arithmetic and the NTSC drift proof, arenas and stale handles, the retime-maths, every command's serde round trip and the tolerant reader's cases,-the expression language, manifest validation and every FFmpeg package's-fixtures (a package without fixtures fails the build), the audio graph's-shape, seek policy and LRU eviction, filter strings, all seven-transition-resolution behaviours, CPU/GPU parity, the speech model-bookkeeping, and the host's cache keys, WAV parsing, sweep ordering and-template packing, and the API's request shapes and project life cycle-in a scratch folder. Tests that touch FFmpeg encode a small file into the-temporary directory first.--The window has unit tests for its formatting helpers and stage geometry.-The seams that a window drives end to end (publish, the dock tree, the-echo and commit path, playback against a device) are exercised by using-the app, which is the work in 17.1.-------## 17. Next work--Ordered by how much each matters. Each is either an invariant a-contributor could break without knowing it, or a piece of work with a-known shape.--### 17.1 Put a real project through every flow in the window--Import, place, trim, split, transitions, a title, an effect, a speed-curve, transcription, narration, a template, an export, on each renderer-and each platform. The engine's seams are tested; the window's are used.-What that pass turns up goes above everything below.--### 17.2 The API's other transports--The dispatcher is in place and the command line speaks it. The daemon is-next: `concat serve`, JSON-RPC over stdio or a local socket, the same-`Request` and `Response` with an id on each, and `export.cancel` finally-meaningful because a second message can arrive while a first blocks.-MCP is that daemon with tool schemas derived from the request enum. Then-the window itself listening on the same socket, so a script drives the-live edit and the lanes redraw, which is `Shell::with` through-`invoke_from_event_loop` and nothing new. Two decisions come with them: a-lock file in the project folder, or a window that reloads on an external-change, since the window autosaves and last writer wins today; and-whether speech joins the dispatcher behind a feature, so a headless-export never links whisper.--### 17.3 A phone layout--The window builds and packages for Android and iOS, and what runs there is-the desktop's tree on a phone's screen. A phone wants one seat at a time,-touch-sized targets and the timeline under the monitor; the `.slint` tree-can carry a second arrangement chosen by the window's size. With it: the-system document picker, since `platform.rs` returns nothing for a pick on-a phone and `concat-media` opens by path; MediaCodec and VideoToolbox-asked for by name (they are linked, and the software decoder is what is-opened); and the monitor on Android, which composites on the CPU because-Slint's Android backend creates its own device.--### 17.4 Two paths that build different chains--On the GPU, effects with a shader leave the FFmpeg chain and travel as-passes; on the CPU they stay in the chain. A machine with an adapter and-one without therefore build different filtergraphs for the same clip, and-the CPU compositor ignores passes by design. Correct today because every-shader-backed package also carries a chain; a WGSL-only package would-render on one machine and not the other. Either require the chain for-`effect` kinds in validation, or teach the CPU path to refuse the package-loudly.--### 17.5 Small things the code already knows about--- `MoveTimeline`'s documented index is counted with the timeline removed;- the clamp is against the unremoved length.-- `doc.rs` normalises `muted: false` to absent, so an explicit false does- not round-trip byte for byte.-- `Editor::apply` clones the project before every command, including ones- that fail or change nothing.-- `Transcriber::download_model` returns early for a present file without- a final progress report, so a caller waiting on `done` waits.-- FFmpeg-touching tests write fixed file names into the temporary- directory; two concurrent runs can collide.-- The cross-fade adjacency scan is quadratic and takes the first match.-- `Kind::Transition` and `Kind::Generator` are validated and unused; the- motion transitions in the inspector are present and disabled.-- The programme level meter has no feed; playback's mix does not report- levels.-- `ConfigPane` carries a demo-era property group and `ui/demo/` is still- imported for one type.-- No clip culling beyond the visible flag; a windowing scheme is worth- writing the day a timeline needs one.-- The cutout model is one, sized for footage. A finer, salient-object- model for stills would slot in behind the same `Segmenter`, keyed by- its own `MODEL_ID` so the two never share a cache.-- A custom cutout's strokes are painted at the mask's resolution, so a- very fine brush on a large picture is as fine as 256 pixels allow.-------## 18. How to change things--- **A new operation on the edit.** Add a `Command` variant in- `concat-project/src/commands.rs`, make its `apply` report `applied`- honestly, add it to the serde round-trip test's list, then bind a- callback in `concat/src/lib.rs` that builds it from the window's state.-- **A new API method.** A `Request` variant in- `concat-api/src/message.rs` with its `method` name, a `Reply` for what- it returns if none fits, and its arm in `Api::run`. Every transport- carries it from then on; none of them changes.-- **A new effect.** Add a folder under `concat-effects/packages/` with- `effect.toml` and `fixtures.toml` (and `effect.wgsl` if it has a- shader). The build fails until the fixtures cover default, minimum and- maximum. Nothing in Rust changes.-- **A new pane.** A `PaneKind`, its entry in the `Panes` global, a branch- in `Seat`, and the `Editor` properties and callbacks it needs.-- **A new text preset.** A TOML file under the config directory's- `text-presets/`, or a folder there with `preset.toml` and the font- beside it; the format is at the top of `concat/src/presets.rs`. Nothing- in Rust changes.-- **A new keyboard chord.** A line in the key table in `app.slint` that- names a verb, and the verb's arm in `Studio::shortcut` - or, for a chord- that is also a menu row, the row's id, so the two stay one thing.-- **A new decode or encode backend.** Implement `FrameSource` or- `FrameSink` in `concat-media`. Nothing else in the workspace changes.-- **A new platform.** `platform.rs` for the window's three seams,- `concat-media/build.rs` for what FFmpeg needs linked, and a job in the- workflow that builds it.--Reading order for a cold start: this file; then `concat-core/src/lib.rs`,-`concat-project/src/lib.rs` and `concat-export/src/lib.rs`, which between-them hold the vocabulary, the document and the render loop; then-`concat/src/studio.rs` for the window's side of the line; then-`cargo doc --open` for the rest.