The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.
Rust ffmpegfree-video-editormodel-context-protocolnon-linear-editoroffline-firstopen-source-video-editorself-hostedvideo-editingvideo-editing-automationvideo-editing-softwarevideo-editing-toolsvideo-editorvoice-cloningyoutube-automationyoutube-shorts
Add the Concat API as one dispatcher crate
Jareer315ecdd52abf
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.mdindex d920884c..66ca2558 100644--- a/ARCHITECTURE.md+++ b/ARCHITECTURE.md@@ -40,7 +40,7 @@ Two consequences of the rule shape everything below:## 2. The map-Everything lives in `engine/`, one Cargo workspace of thirteen crates.+Everything lives in `engine/`, one Cargo workspace of fourteen crates.| Crate | Owns | Lines ||---|---|---|@@ -54,7 +54,8 @@ Everything lives in `engine/`, one Cargo workspace of thirteen crates.| `concat-vision` | What the engine sees in a picture: the person mask behind a cutout, brush strokes over it, the frame cut by it; the model itself behind `infer`. | 0.9k || `concat-host` | What a window needs that is not the edit: sessions, project folders, caches, the monitor, audio playback, the export driver, templates, job slots, app directories. | 3.8k || `concat-speech` | Transcription with whisper.cpp and text to speech with Kokoro, both in-process, models downloaded on demand. | 1.2k |-| `concat-cli` | A driver for the engine without a window: `probe` and `render`. | 0.2k |+| `concat-api` | The Concat API: one dispatcher of JSON requests over the host - projects, edits, media, the catalogue, templates, exports, frames - that every transport calls. | 0.9k |+| `concat-cli` | A driver for the engine without a window: `probe`, `render`, and `api`, the API over stdin and stdout. | 0.3k || `concat` | The editor window in Slint, and the Rust that binds it to the host. | 7.2k Rust, 15.9k Slint || `concat-android` | The Android activity: `android_main` over the window. | 20 |@@ -71,7 +72,8 @@ concat ──► concat-speech ──► concat-host ──► concat-export ─├──► concat-effects (the catalogue, for the inspector's shelves)└──► concat-vision (a press on the stage, mapped into the source)-concat-android ──► concat concat-cli ──► concat-core, concat-media, concat-render+concat-android ──► concat concat-api ──► concat-host, concat-export, concat-effects, concat-project+ concat-cli ──► concat-api, concat-core, concat-media, concat-render````concat-core` depends on nothing; `concat-project` is not folded into it@@ -565,7 +567,63 @@ native libraries stay out of everything that does not speak.----## 13. The window: `concat`+## 13. The API: `concat-api`++The editor driven without its window. Every way of doing that - the+command line today, a daemon on a socket, an MCP server, a plugin - is a+transport, and a transport is a loop: read a `Request`, hand it to+`Api::dispatch`, write the `Response` and whatever `Event`s fired on the+way. What a request means lives in this one crate, so two transports+cannot disagree and a method added here reaches all of them.++**The shape.** `Request` is a serde enum tagged by `method`, named+`area.verb`: `project.create`, `project.open`, `project.close`,+`project.list`, `project.get`, `project.document`, `project.save`,+`project.setVideo`, `edit.apply`, `edit.undo`, `edit.redo`,+`media.probe`, `media.import`, `catalogue.list`, `template.list`,+`template.instantiate`, `template.save`, `export.run`, `export.cancel`,+`preview.frame`, `version`. A response is `{"result": ...}` or+`{"error": "..."}`, the error being the sentence the window would show.+Events are tagged by `event`: export progress, and cutout analysis ahead+of an export. Fields are camelCase, the document's and the command+layer's spelling, so a caller learns one. `API_VERSION` names the+contract and moves only when a caller written against the previous one+would misread a reply.++**The edit is not redefined.** `edit.apply` carries a `concat-project`+`Command` exactly as the window sends one, clamps and refusals included,+so every operation the window can perform is one a script can perform+and a new command needs nothing added here. A `Batch` is one request and+one undo step. A text style in a command needs only the fields it sets.++**What the crate owns is the choreography** the window performs by hand.+`media.import` is the probe and the add. `template.instantiate` probes+every fill before anything is made, so a bad path refuses the request+rather than leaving a folder behind. `export.run` finds every mask the+timeline's cutouts still need (`Cutouts::requests`, the same list the+window asks for after each change), paints the titles, then renders, so+the file the API writes is the file the window would have written.+`preview.frame` is the paused monitor's true frame, titles included, as+a PNG.++**One `Api`, one session per project folder**, keyed by the folder's+canonical path, so `.` and its absolute spelling are one project.+A method on a folder that is not open is an error, never a silent open.+The `Api` is not thread-safe on purpose: a transport serving several+callers owns the one `Api` and serialises through it, as the window's+event loop does, and a long method blocks its caller. `Api::exporter` is+the one handle that crosses threads, so a cancel can reach a running+export.++**The first transport** is `concat-cli api`: one request per line on+stdin, one JSON object per line on stdout, events interleaved and+flushed as they happen, so a pipe from any language edits and exports a+project. A line that is not a request gets an error and the loop goes+on.++---++## 14. The window: `concat`**One library, three entry points.** `concat::run()` is the program.`main.rs` calls it on the desktop and on iOS; `concat-android`'s@@ -670,7 +728,7 @@ it; CI runs the check.----## 14. Build and delivery+## 15. Build and delivery- **Toolchain.** Pinned in `engine/rust-toolchain.toml`. `cargo build`needs the FFmpeg 7+ development libraries, cmake and a C++ toolchain@@ -708,7 +766,7 @@ it; CI runs the check.----## 15. Testing+## 16. Testing240 unit tests, all in-file `#[cfg(test)]` modules; there are no `tests/`directories. The pure layers are where the coverage is dense: rational@@ -719,30 +777,46 @@ fixtures (a package without fixtures fails the build), the audio graph'sshape, seek policy and LRU eviction, filter strings, all seventransition-resolution behaviours, CPU/GPU parity, the speech modelbookkeeping, and the host's cache keys, WAV parsing, sweep ordering and-template packing. Tests that touch FFmpeg encode a small file into the+template packing, and the API's request shapes and project life cycle+in a scratch folder. Tests that touch FFmpeg encode a small file into thetemporary directory first.The window has unit tests for its formatting helpers and stage geometry.The seams that a window drives end to end (publish, the dock tree, theecho and commit path, playback against a device) are exercised by using-the app, which is the work in 15.1.+the app, which is the work in 17.1.----## 16. Next work+## 17. Next workOrdered by how much each matters. Each is either an invariant acontributor could break without knowing it, or a piece of work with aknown shape.-### 16.1 Put a real project through every flow in the window+### 17.1 Put a real project through every flow in the windowImport, place, trim, split, transitions, a title, an effect, a speedcurve, transcription, narration, a template, an export, on each rendererand each platform. The engine's seams are tested; the window's are used.What that pass turns up goes above everything below.-### 16.2 A phone layout+### 17.2 The API's other transports++The dispatcher is in place and the command line speaks it. The daemon is+next: `concat serve`, JSON-RPC over stdio or a local socket, the same+`Request` and `Response` with an id on each, and `export.cancel` finally+meaningful because a second message can arrive while a first blocks.+MCP is that daemon with tool schemas derived from the request enum. Then+the window itself listening on the same socket, so a script drives the+live edit and the lanes redraw, which is `Shell::with` through+`invoke_from_event_loop` and nothing new. Two decisions come with them: a+lock file in the project folder, or a window that reloads on an external+change, since the window autosaves and last writer wins today; and+whether speech joins the dispatcher behind a feature, so a headless+export never links whisper.++### 17.3 A phone layoutThe window builds and packages for Android and iOS, and what runs there isthe desktop's tree on a phone's screen. A phone wants one seat at a time,@@ -754,7 +828,7 @@ asked for by name (they are linked, and the software decoder is what isopened); and the monitor on Android, which composites on the CPU becauseSlint's Android backend creates its own device.-### 16.3 Two paths that build different chains+### 17.4 Two paths that build different chainsOn the GPU, effects with a shader leave the FFmpeg chain and travel aspasses; on the CPU they stay in the chain. A machine with an adapter and@@ -765,7 +839,7 @@ render on one machine and not the other. Either require the chain for`effect` kinds in validation, or teach the CPU path to refuse the packageloudly.-### 16.4 Small things the code already knows about+### 17.5 Small things the code already knows about- `MoveTimeline`'s documented index is counted with the timeline removed;the clamp is against the unremoved length.@@ -794,12 +868,16 @@ loudly.----## 17. How to change things+## 18. How to change things- **A new operation on the edit.** Add a `Command` variant in`concat-project/src/commands.rs`, make its `apply` report `applied`honestly, add it to the serde round-trip test's list, then bind acallback in `concat/src/lib.rs` that builds it from the window's state.+- **A new API method.** A `Request` variant in+ `concat-api/src/message.rs` with its `method` name, a `Reply` for what+ it returns if none fits, and its arm in `Api::run`. Every transport+ carries it from then on; none of them changes.- **A new effect.** Add a folder under `concat-effects/packages/` with`effect.toml` and `fixtures.toml` (and `effect.wgsl` if it has ashader). The build fails until the fixtures cover default, minimum anddiff --git a/engine/Cargo.lock b/engine/Cargo.lockindex d02f8f67..95af1722 100644--- a/engine/Cargo.lock+++ b/engine/Cargo.lock@@ -1176,14 +1176,30 @@ dependencies = ["slint",]+[[package]]+name = "concat-api"+version = "0.2.1"+dependencies = [+ "concat-effects",+ "concat-export",+ "concat-host",+ "concat-project",+ "png 0.17.16",+ "serde",+ "serde_json",+ "tempfile",+]+[[package]]name = "concat-cli"version = "0.2.1"dependencies = ["clap",+ "concat-api","concat-core","concat-media","concat-render",+ "serde_json",][[package]]diff --git a/engine/Cargo.toml b/engine/Cargo.tomlindex d1f2da92..c5ebb9aa 100644--- a/engine/Cargo.toml+++ b/engine/Cargo.toml@@ -15,6 +15,7 @@ concat-project = { path = "crates/concat-project" }concat-render = { path = "crates/concat-render" }concat-export = { path = "crates/concat-export" }concat-host = { path = "crates/concat-host" }+concat-api = { path = "crates/concat-api" }concat-speech = { path = "crates/concat-speech" }concat-effects = { path = "crates/concat-effects" }concat-text = { path = "crates/concat-text" }diff --git a/engine/README.md b/engine/README.mdindex d99008e5..6e67ef9b 100644--- a/engine/README.md+++ b/engine/README.md@@ -12,7 +12,8 @@ The video engine behind Concat. Rust, no GC, no hidden control flow.| `concat-render` | Turning a timeline plus a timestamp into one finished frame. | `concat-core`, wgpu (optional) || `concat-effects` | Effect packages: manifests, chain templates, the catalogue. Each built-in effect is a folder under `packages/`. | `concat-project`, serde, toml || `concat-export` | Timeline to file: flatten, the frame-by-frame render loop, the paused monitor's true frame. | `concat-core`, `concat-media`, `concat-render`, `concat-project`, `concat-effects` |-| `concat-cli` | A binary to drive the above. The vertical slice. | the engine crates |+| `concat-api` | The Concat API: JSON requests in, responses and events out. Projects, edits, media, the catalogue, templates, exports, frames. | `concat-host`, `concat-export`, `concat-effects`, `concat-project` |+| `concat-cli` | A binary to drive the above: `probe`, `render`, and `api`, the API over stdin and stdout. | `concat-api`, the engine crates || `concat-host` | What the window needs that is not the edit: sessions, project folders, previews, playback, templates, job slots. | `concat-media`, `concat-project`, `concat-export`, cpal || `concat-speech` | Transcription (whisper.cpp, in-process) and text to speech (Kokoro via sherpa-onnx). | `concat-host`, `concat-media`, whisper-rs, sherpa-onnx || `concat` | The editor window: every pane, dialog and primitive, in Slint. The app a user launches. | slint, `concat-host`, `concat-speech` |@@ -24,6 +25,7 @@ The dependency arrows point one way:concat -> concat-speech -> concat-host -> {export, project, media} -> core-> render -> coreexport -> effects -> project+concat-cli -> concat-api -> concat-host```If you ever find yourself wanting `core` to depend on `media`, something has@@ -40,6 +42,8 @@ cargo buildcargo testcargo run -p concat-cli -- probe some-video.mp4cargo run -p concat-cli -- render some-video.mp4 out.mp4 --frames 120+cargo run -p concat-cli -- api '{"method":"catalogue.list","kind":"filter"}'+cargo run -p concat-cli -- api < edits.jsonl # one request per line; see concat-apicargo run -p concat # the editor window, debugcargo run --profile quick -p concat # optimised, rebuilds in seconds: for trying changescargo build --profile app -p concat # the shipping binary: fat LTO, panic=abort, strippeddiff --git a/engine/crates/concat-api/Cargo.toml b/engine/crates/concat-api/Cargo.tomlnew file mode 100644index 00000000..5a5b70b1--- /dev/null+++ b/engine/crates/concat-api/Cargo.toml@@ -0,0 +1,24 @@+[package]+name = "concat-api"+version.workspace = true+edition.workspace = true+rust-version.workspace = true+license.workspace = true+description = "The Concat API: one dispatcher for projects, edits, media, effects, templates, exports and frames, which every transport - the command line, a daemon, a plugin - calls."++[dependencies]+concat-project.workspace = true+concat-effects.workspace = true+concat-export.workspace = true+concat-host.workspace = true+serde = { version = "1", features = ["derive"] }+serde_json.workspace = true+# The frame method hands back a PNG: the one picture format every+# caller reads and that keeps the compositor's pixels exact.+png = "0.17"++[dev-dependencies]+tempfile = "3"++[lints]+workspace = truediff --git a/engine/crates/concat-api/src/lib.rs b/engine/crates/concat-api/src/lib.rsnew file mode 100644index 00000000..9f447205--- /dev/null+++ b/engine/crates/concat-api/src/lib.rs@@ -0,0 +1,809 @@+// SPDX-License-Identifier: AGPL-3.0-or-later+// SPDX-FileCopyrightText: 2026 Jareer and Concat contributors++//! The Concat API: the one dispatcher every way of driving the editor+//! without its window goes through.+//!+//! The command line, a daemon on a socket, an MCP server, a plugin: each is+//! a transport, and a transport is a loop that reads a [`Request`], hands+//! it to [`Api::dispatch`], and writes the [`Response`] and any [`Event`]s+//! back. Nothing about what a request *means* lives in a transport, so two+//! of them cannot disagree, and a method added here reaches all of them.+//!+//! The crate decides nothing about the edit either. Edits are+//! `concat_project` [`Command`]s carried as they are; projects, media,+//! templates, titles, cutouts and exports are `concat_host`'s. What this+//! crate owns is the choreography the window performs by hand - probe then+//! add, paint titles and find masks before rendering, save through the+//! session - stated once so a file exported here is the file the window+//! would have written.+//!+//! One [`Api`] holds one session per open project folder. It is not+//! thread-safe by design: a transport that serves several callers owns the+//! one `Api` and serialises through it, the way the window's event loop+//! does, and a long method blocks its caller. [`Api::exporter`] is the one+//! handle that crosses threads, so a cancel can reach a running export.++pub mod message;++use std::collections::BTreeMap;+use std::path::{Path, PathBuf};+use std::sync::Arc;++use concat_effects::Catalogue;+use concat_effects::manifest::Kind;+use concat_export::ExportClip;+use concat_host::cutout::{self, Cutouts};+use concat_host::export::{self, Exporter};+use concat_host::preview::{FrameSpec, Monitor};+use concat_host::session::EditorView;+use concat_host::templates::{self, SlotFill};+use concat_host::{AppDirs, ProjectInfo, Session, Titles, media, projects};+use concat_project::Command;+use concat_project::model::VideoSettings;++pub use message::{+ API_VERSION, Dirs, Done, Event, ExportSpec, Fill, PackageInfo, ParamInfo, Reply, Request,+ Response, VersionInfo, Written,+};++/// The export sheet's middle quality, and what an export gets when the+/// caller says nothing.+const DEFAULT_CRF: u8 = 20;+/// The x264 preset every export uses unless told otherwise.+const DEFAULT_PRESET: &str = "medium";++/// The dispatcher: the open sessions and the services behind them.+pub struct Api {+ dirs: AppDirs,+ /// Open projects by canonical folder path.+ sessions: BTreeMap<String, Session>,+ titles: Titles,+ cutouts: Cutouts,+ monitor: Monitor,+ exporter: Exporter,+}++impl Api {+ /// An API over this machine's app directories.+ pub fn new() -> Result<Api, String> {+ Ok(Api::with_dirs(AppDirs::locate()?))+ }++ /// An API over the given directories: a test's scratch, or an embedder+ /// with a home of its own.+ pub fn with_dirs(dirs: AppDirs) -> Api {+ Api {+ titles: Titles::new(&dirs),+ cutouts: Cutouts::new(&dirs.data),+ monitor: Monitor::new(),+ exporter: Exporter::new(),+ sessions: BTreeMap::new(),+ dirs,+ }+ }++ /// The directories this API works under.+ pub fn dirs(&self) -> &AppDirs {+ &self.dirs+ }++ /// The export slot, for a transport that needs to cancel from another+ /// thread while [`Api::dispatch`] blocks on [`Request::ExportRun`].+ pub fn exporter(&self) -> Exporter {+ self.exporter.clone()+ }++ /// Runs one request. Events fire through `notify` as the work goes;+ /// the response is what the request is worth.+ pub fn dispatch(&mut self, request: Request, notify: &mut dyn FnMut(Event)) -> Response {+ self.run(request, notify).into()+ }++ fn run(&mut self, request: Request, notify: &mut dyn FnMut(Event)) -> Result<Reply, String> {+ let view = |view: EditorView| Ok(Reply::View(Box::new(view)));+ match request {+ Request::Version => Ok(Reply::Version(self.version())),+ Request::ProjectCreate {+ location,+ name,+ video,+ } => view(self.create(&location, &name, video.unwrap_or_default())?),+ Request::ProjectOpen { path } => view(self.open(&path)?),+ Request::ProjectClose { path, save } => {+ self.close(&path, save)?;+ Ok(Reply::Done(Done {}))+ }+ Request::ProjectList => Ok(Reply::Projects(self.recents())),+ Request::ProjectGet { path } => view(self.session(&path)?.view()),+ Request::ProjectDocument { path } => {+ Ok(Reply::Document(self.session(&path)?.document()))+ }+ Request::ProjectSave { path, name } => {+ self.save(&path, name.as_deref())?;+ Ok(Reply::Done(Done {}))+ }+ Request::ProjectSetVideo { path, video } => {+ view(self.session_mut(&path)?.set_video(video)?)+ }+ Request::EditApply { path, command } => view(self.apply(&path, *command)?),+ Request::EditUndo { path } => view(self.session_mut(&path)?.undo()),+ Request::EditRedo { path } => view(self.session_mut(&path)?.redo()),+ Request::MediaProbe { path } => Ok(Reply::Media(media::probe(&path)?)),+ Request::MediaImport { path, file } => view(self.import(&path, &file)?),+ Request::CatalogueList { kind } => Ok(Reply::Packages(catalogue(kind.as_deref())?)),+ Request::TemplateList => Ok(Reply::Templates(templates::list(&self.dirs.config))),+ Request::TemplateInstantiate {+ template,+ location,+ name,+ fills,+ } => view(self.instantiate(&template, &location, &name, fills)?),+ Request::TemplateSave { path, name } => {+ Ok(Reply::Template(self.save_template(&path, &name)?))+ }+ Request::ExportRun { path, spec } => {+ Ok(Reply::Written(self.export(&path, &spec, notify)?))+ }+ Request::ExportCancel => {+ self.exporter.cancel();+ Ok(Reply::Done(Done {}))+ }+ Request::PreviewFrame {+ path,+ time,+ output,+ width,+ height,+ } => Ok(Reply::Written(self.frame(+ &path,+ time,+ &output,+ width.zip(height),+ )?)),+ }+ }++ /// [`Request::Version`].+ pub fn version(&self) -> VersionInfo {+ VersionInfo {+ api_version: API_VERSION.to_owned(),+ concat: env!("CARGO_PKG_VERSION").to_owned(),+ dirs: Dirs::from(&self.dirs),+ }+ }++ /// [`Request::ProjectCreate`]: the folder, its manifest, and a session+ /// on it, remembered in recents like a project the window made.+ pub fn create(+ &mut self,+ location: &str,+ name: &str,+ video: VideoSettings,+ ) -> Result<EditorView, String> {+ let info = projects::create(+ location,+ name,+ video.width,+ video.height,+ video.rate_num,+ video.rate_den,+ )?;+ self.adopt(info)+ }++ /// [`Request::ProjectOpen`].+ pub fn open(&mut self, path: &str) -> Result<EditorView, String> {+ let key = key_of(path);+ if let Some(session) = self.sessions.get(&key) {+ return Ok(session.view());+ }+ let info = projects::open(path)?;+ self.adopt(info)+ }++ /// Opens a session on a project the host just described and puts it at+ /// the front of the recents list.+ fn adopt(&mut self, info: ProjectInfo) -> Result<EditorView, String> {+ let session = Session::open_info(&info)?;+ // Recents are a convenience for the launch screen; a machine whose+ // config folder cannot be written still edits.+ let _ = projects::remember(&self.dirs.config, &info);+ let view = session.view();+ self.sessions.insert(key_of(&info.path), session);+ Ok(view)+ }++ /// [`Request::ProjectClose`].+ pub fn close(&mut self, path: &str, save: bool) -> Result<(), String> {+ if save {+ self.save(path, None)?;+ }+ self.sessions+ .remove(&key_of(path))+ .map(drop)+ .ok_or_else(|| not_open(path))+ }++ /// [`Request::ProjectList`].+ pub fn recents(&self) -> Vec<ProjectInfo> {+ projects::list(&self.dirs.config)+ }++ /// [`Request::ProjectSave`].+ pub fn save(&mut self, path: &str, name: Option<&str>) -> Result<(), String> {+ self.session_mut(path)?.save(name)+ }++ /// [`Request::EditApply`].+ pub fn apply(&mut self, path: &str, command: Command) -> Result<EditorView, String> {+ self.session_mut(path)?.apply(command)+ }++ /// [`Request::MediaImport`]: the probe and the add, as one.+ pub fn import(&mut self, path: &str, file: &str) -> Result<EditorView, String> {+ let item = media::probe(file)?.to_new_media();+ self.apply(path, Command::AddMedia { item })+ }++ /// [`Request::TemplateInstantiate`].+ pub fn instantiate(+ &mut self,+ template: &str,+ location: &str,+ name: &str,+ fills: Vec<Fill>,+ ) -> Result<EditorView, String> {+ // Every file is probed before anything is made, so a bad path+ // refuses the whole request rather than leaving a folder behind.+ let fills = fills+ .into_iter()+ .map(|fill| {+ Ok(SlotFill {+ media_id: fill.media_id,+ item: media::probe(&fill.file)?.to_new_media(),+ })+ })+ .collect::<Result<Vec<SlotFill>, String>>()?;+ let info = templates::instantiate(template, location, name, fills)?;+ self.adopt(info)+ }++ /// [`Request::TemplateSave`].+ pub fn save_template(+ &mut self,+ path: &str,+ name: &str,+ ) -> Result<concat_host::templates::TemplateInfo, String> {+ let session = self.session(path)?;+ templates::save(+ &self.dirs.config,+ &session.document(),+ &session.settings(),+ session.path(),+ name,+ )+ }++ /// [`Request::ExportRun`]: what the window's Export sheet does, in its+ /// order. Masks first, because the frame loop reads whatever is in the+ /// project's cache and draws the picture as shot where there is none;+ /// then titles, which rejoin the clip list as stills; then the render.+ pub fn export(+ &mut self,+ path: &str,+ spec: &ExportSpec,+ notify: &mut dyn FnMut(Event),+ ) -> Result<Written, String> {+ let session = self.session(path)?;+ if session.project().active().clips.is_empty() {+ return Err("There is nothing on the timeline to export".to_owned());+ }+ let settings = session.settings();+ let width = spec.width.unwrap_or(settings.width);+ let height = spec.height.unwrap_or(settings.height);++ let project_dir = PathBuf::from(session.path());+ let project = session.project().clone();+ self.find_masks(&project, &project_dir, notify)?;++ let session = self.session(path)?;+ let titles = self.title_clips(session, width, height);+ let host_spec = export::ExportSpec {+ output: spec.output.clone(),+ crf: spec.crf.unwrap_or(DEFAULT_CRF),+ preset: spec+ .preset+ .clone()+ .unwrap_or_else(|| DEFAULT_PRESET.to_owned()),+ };+ let mut request = export::request(session, &host_spec, titles);+ request.width = width;+ request.height = height;+ request.rate_num = spec.rate_num.unwrap_or(settings.rate_num);+ request.rate_den = spec.rate_den.unwrap_or(settings.rate_den);++ let job = self.exporter.begin()?;+ let written = export::run(&request, job.cancel_flag(), |progress| {+ notify(Event::from(progress));+ })?;+ Ok(Written {+ path: written,+ width,+ height,+ })+ }++ /// Runs every cutout analysis the timeline still needs, one after the+ /// other, so the export that follows cuts every clip it should.+ fn find_masks(+ &self,+ project: &concat_project::Project,+ project_dir: &Path,+ notify: &mut dyn FnMut(Event),+ ) -> Result<(), String> {+ for (media_id, request) in Cutouts::requests(project, project_dir) {+ if Cutouts::outstanding(&request) == 0 {+ continue;+ }+ self.cutouts.analyse(&request, &mut |progress| {+ let (fetching, fraction) = match progress {+ cutout::Progress::Fetching { received, total } => {+ (true, received as f32 / total.max(1) as f32)+ }+ cutout::Progress::Analysing(fraction) => (false, fraction),+ };+ notify(Event::CutoutProgress {+ media_id: media_id.clone(),+ fetching,+ fraction,+ });+ })?;+ }+ Ok(())+ }++ /// The timeline's titles painted for a `width` × `height` frame, as+ /// the stills that stand in for them.+ fn title_clips(&self, session: &Session, width: u32, height: u32) -> Vec<ExportClip> {+ self.titles+ .clips(session.project(), width, height)+ .into_iter()+ .map(|title| title.clip)+ .collect()+ }++ /// [`Request::PreviewFrame`]: the paused monitor's true frame, to a+ /// file.+ pub fn frame(+ &mut self,+ path: &str,+ time: f64,+ output: &str,+ size: Option<(u32, u32)>,+ ) -> Result<Written, String> {+ let session = self.session(path)?;+ let settings = session.settings();+ let (width, height) = size.unwrap_or((settings.width, settings.height));+ if width == 0 || height == 0 {+ return Err("A frame needs a width and a height".to_owned());+ }+ let mut clips = session.flattened_clips();+ clips.extend(self.title_clips(session, width, height));+ let pixels = self.monitor.frame(+ Arc::new(clips),+ &settings,+ FrameSpec {+ time,+ width,+ height,+ },+ )?;+ write_png(Path::new(output), width, height, &pixels)?;+ Ok(Written {+ path: output.to_owned(),+ width,+ height,+ })+ }++ fn session(&self, path: &str) -> Result<&Session, String> {+ self.sessions+ .get(&key_of(path))+ .ok_or_else(|| not_open(path))+ }++ fn session_mut(&mut self, path: &str) -> Result<&mut Session, String> {+ self.sessions+ .get_mut(&key_of(path))+ .ok_or_else(|| not_open(path))+ }+}++/// [`Request::CatalogueList`]: the built-in packages plus whatever+/// [`Catalogue::install`] added, in id order.+fn catalogue(kind: Option<&str>) -> Result<Vec<PackageInfo>, String> {+ let kind = match kind {+ None => None,+ Some("effect") => Some(Kind::Effect),+ Some("filter") => Some(Kind::Filter),+ Some("audio") => Some(Kind::Audio),+ Some("transition") => Some(Kind::Transition),+ Some("generator") => Some(Kind::Generator),+ Some(other) => {+ return Err(format!(+ "{other:?} is not a kind: effect, filter, audio, transition or generator"+ ));+ }+ };+ let mut packages: Vec<PackageInfo> = Catalogue::builtin()+ .packages()+ .filter(|package| kind.is_none_or(|kind| package.kind() == kind))+ .map(|package| {+ let meta = &package.manifest.effect;+ PackageInfo {+ id: meta.id.clone(),+ name: meta.name.clone(),+ kind: format!("{:?}", meta.kind).to_ascii_lowercase(),+ category: meta.category.clone(),+ description: meta.description.clone(),+ intensity: meta.intensity.clone(),+ params: package+ .manifest+ .params+ .iter()+ .map(|param| ParamInfo {+ key: param.key.clone(),+ label: param.label.clone(),+ kind: format!("{:?}", param.kind).to_ascii_lowercase(),+ min: param.min,+ max: param.max,+ default: param.default,+ step: param.step,+ unit: param.unit.clone(),+ animate: param.animate,+ values: param.values.clone(),+ labels: param.labels.clone(),+ })+ .collect(),+ }+ })+ .collect();+ packages.sort_by(|left, right| left.id.cmp(&right.id));+ Ok(packages)+}++/// The key a project folder is held under: its canonical path where the+/// folder exists, so `.` and an absolute spelling of it are one session,+/// and the path as given where it does not yet.+fn key_of(path: &str) -> String {+ std::fs::canonicalize(path)+ .map(|canonical| canonical.to_string_lossy().into_owned())+ .unwrap_or_else(|_| path.to_owned())+}++fn not_open(path: &str) -> String {+ format!("{path} is not open - open it first")+}++/// Writes RGBA pixels as a PNG, creating the folder above the file.+fn write_png(output: &Path, width: u32, height: u32, pixels: &[u8]) -> Result<(), String> {+ if let Some(parent) = output.parent()+ && !parent.as_os_str().is_empty()+ {+ std::fs::create_dir_all(parent)+ .map_err(|error| format!("could not create {}: {error}", parent.display()))?;+ }+ let file = std::fs::File::create(output)+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;+ let mut encoder = png::Encoder::new(std::io::BufWriter::new(file), width, height);+ encoder.set_color(png::ColorType::Rgba);+ encoder.set_depth(png::BitDepth::Eight);+ let mut writer = encoder+ .write_header()+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;+ writer+ .write_image_data(pixels)+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;+ writer+ .finish()+ .map_err(|error| format!("could not write {}: {error}", output.display()))+}++#[cfg(test)]+mod tests {+ use super::*;+ use concat_project::commands::NewMedia;+ use concat_project::model::MediaKind;+ use serde_json::json;++ /// An API whose config and data live in a scratch folder that goes+ /// away with the test.+ fn api() -> (Api, tempfile::TempDir) {+ let scratch = tempfile::tempdir().expect("scratch");+ let dirs = AppDirs {+ config: scratch.path().join("config"),+ data: scratch.path().join("data"),+ };+ (Api::with_dirs(dirs), scratch)+ }++ fn ok(response: Response) -> Reply {+ match response {+ Response::Result(reply) => reply,+ Response::Error(error) => panic!("refused: {error}"),+ }+ }++ fn err(response: Response) -> String {+ match response {+ Response::Result(_) => panic!("worked, unexpectedly"),+ Response::Error(error) => error,+ }+ }++ fn view(reply: Reply) -> EditorView {+ match reply {+ Reply::View(view) => *view,+ _ => panic!("not a view"),+ }+ }++ fn quiet() -> impl FnMut(Event) {+ |_| {}+ }++ fn still(path: &str) -> NewMedia {+ NewMedia {+ path: path.to_owned(),+ name: "still.png".to_owned(),+ duration: None,+ kind: MediaKind::Image,+ width: Some(640),+ height: Some(360),+ frame_rate: None,+ frame_rate_fraction: None,+ video_codec: None,+ audio_codec: None,+ has_audio: false,+ }+ }++ #[test]+ fn a_request_parses_by_method_and_carries_a_command_verbatim() {+ let request: Request = serde_json::from_value(json!({+ "method": "edit.apply",+ "path": "/p",+ "command": { "op": "addTextClip", "start": 1.5 }+ }))+ .expect("parses");+ assert_eq!(+ request,+ Request::EditApply {+ path: "/p".to_owned(),+ command: Box::new(Command::AddTextClip {+ track_id: None,+ start: 1.5,+ style: None,+ duration: None,+ offset_y: None,+ }),+ }+ );+ }++ #[test]+ fn a_response_is_a_result_or_an_error_object() {+ let worked = serde_json::to_value(Response::Result(Reply::Done(Done {}))).expect("json");+ assert_eq!(worked, json!({ "result": {} }));+ let refused = serde_json::to_value(Response::Error("no".to_owned())).expect("json");+ assert_eq!(refused, json!({ "error": "no" }));+ }++ #[test]+ fn create_edit_save_and_reopen_round_trip() {+ let (mut api, scratch) = api();+ let location = scratch.path().join("projects");+ std::fs::create_dir_all(&location).expect("location");+ let location = location.to_string_lossy().into_owned();++ let created = view(ok(api.dispatch(+ Request::ProjectCreate {+ location: location.clone(),+ name: "Round trip".to_owned(),+ video: Some(VideoSettings {+ width: 1080,+ height: 1920,+ rate_num: 60,+ rate_den: 1,+ }),+ },+ &mut quiet(),+ )));+ assert_eq!(created.settings.width, 1080);+ assert_eq!(created.settings.height, 1920);+ assert_eq!(created.settings.rate_num, 60);+ let path = format!("{location}/Round trip");+ assert!(projects::is_project(Path::new(&path)));++ let added = view(ok(api.dispatch(+ Request::EditApply {+ path: path.clone(),+ command: Box::new(Command::AddMedia {+ item: still("/nowhere/still.png"),+ }),+ },+ &mut quiet(),+ )));+ let media_id = added.created_id.expect("minted");+ let placed = view(ok(api.dispatch(+ Request::EditApply {+ path: path.clone(),+ command: Box::new(Command::AddClipAtFirstFree {+ media_id,+ start: 2.0,+ }),+ },+ &mut quiet(),+ )));+ assert!(placed.can_undo);+ assert_eq!(placed.project.active().clips.len(), 1);++ ok(api.dispatch(+ Request::ProjectClose {+ path: path.clone(),+ save: true,+ },+ &mut quiet(),+ ));+ assert_eq!(+ err(api.dispatch(Request::ProjectGet { path: path.clone() }, &mut quiet())),+ not_open(&path)+ );++ let reopened = view(ok(+ api.dispatch(Request::ProjectOpen { path: path.clone() }, &mut quiet())+ ));+ assert_eq!(reopened.project.active().clips.len(), 1);+ assert_eq!(reopened.project.active().clips[0].start, 2.0);+ assert!(!reopened.can_undo, "history does not survive a save");++ let recents = match ok(api.dispatch(Request::ProjectList, &mut quiet())) {+ Reply::Projects(list) => list,+ _ => panic!("not a list"),+ };+ assert_eq!(recents.len(), 1);+ assert_eq!(recents[0].name, "Round trip");+ }++ #[test]+ fn a_refusal_is_the_command_layers_sentence() {+ let (mut api, scratch) = api();+ let location = scratch.path().to_string_lossy().into_owned();+ let created = view(ok(api.dispatch(+ Request::ProjectCreate {+ location: location.clone(),+ name: "Refused".to_owned(),+ video: None,+ },+ &mut quiet(),+ )));+ let track_id = created.project.active().tracks[0].id.clone();+ let refused = err(api.dispatch(+ Request::EditApply {+ path: format!("{location}/Refused"),+ command: Box::new(Command::AddClip {+ media_id: "m999".to_owned(),+ track_id,+ start: 0.0,+ }),+ },+ &mut quiet(),+ ));+ assert_eq!(refused, "That media is no longer in the bin.");+ }++ #[test]+ fn creating_over_a_project_is_refused() {+ let (mut api, scratch) = api();+ let location = scratch.path().to_string_lossy().into_owned();+ let create = || Request::ProjectCreate {+ location: location.clone(),+ name: "Twice".to_owned(),+ video: None,+ };+ ok(api.dispatch(create(), &mut quiet()));+ assert!(err(api.dispatch(create(), &mut quiet())).contains("already exists"));+ }++ #[test]+ fn the_catalogue_lists_packages_with_their_knobs() {+ let packages = catalogue(Some("filter")).expect("kind");+ assert!(!packages.is_empty());+ assert!(packages.iter().all(|package| package.kind == "filter"));+ assert!(packages.windows(2).all(|pair| pair[0].id < pair[1].id));+ assert!(catalogue(Some("look")).is_err());+ let all = catalogue(None).expect("all");+ assert!(all.len() > packages.len());+ assert!(all.iter().any(|package| !package.params.is_empty()));+ }++ #[test]+ fn a_method_on_a_closed_project_says_so() {+ let (mut api, _scratch) = api();+ assert_eq!(+ err(api.dispatch(+ Request::EditUndo {+ path: "/never".to_owned()+ },+ &mut quiet()+ )),+ not_open("/never")+ );+ }++ #[test]+ fn undo_and_redo_step_the_history() {+ let (mut api, scratch) = api();+ let location = scratch.path().to_string_lossy().into_owned();+ ok(api.dispatch(+ Request::ProjectCreate {+ location: location.clone(),+ name: "History".to_owned(),+ video: None,+ },+ &mut quiet(),+ ));+ let path = format!("{location}/History");+ ok(api.dispatch(+ Request::EditApply {+ path: path.clone(),+ command: Box::new(Command::AddTextClip {+ track_id: None,+ start: 0.0,+ style: None,+ duration: Some(3.0),+ offset_y: None,+ }),+ },+ &mut quiet(),+ ));+ let undone = view(ok(+ api.dispatch(Request::EditUndo { path: path.clone() }, &mut quiet())+ ));+ assert!(undone.project.active().clips.is_empty());+ assert!(undone.can_redo);+ let redone = view(ok(api.dispatch(Request::EditRedo { path }, &mut quiet())));+ assert_eq!(redone.project.active().clips.len(), 1);+ }++ #[test]+ fn the_document_is_what_a_save_writes() {+ let (mut api, scratch) = api();+ let location = scratch.path().to_string_lossy().into_owned();+ ok(api.dispatch(+ Request::ProjectCreate {+ location: location.clone(),+ name: "Doc".to_owned(),+ video: None,+ },+ &mut quiet(),+ ));+ let path = format!("{location}/Doc");+ let document = match ok(api.dispatch(+ Request::ProjectDocument { path: path.clone() },+ &mut quiet(),+ )) {+ Reply::Document(document) => document,+ _ => panic!("not a document"),+ };+ ok(api.dispatch(+ Request::ProjectSave {+ path: path.clone(),+ name: None,+ },+ &mut quiet(),+ ));+ assert_eq!(projects::read_document(&path).expect("saved"), document);+ }+}diff --git a/engine/crates/concat-api/src/message.rs b/engine/crates/concat-api/src/message.rsnew file mode 100644index 00000000..b0617635--- /dev/null+++ b/engine/crates/concat-api/src/message.rs@@ -0,0 +1,436 @@+// SPDX-License-Identifier: AGPL-3.0-or-later+// SPDX-FileCopyrightText: 2026 Jareer and Concat contributors++//! What crosses the API boundary: a [`Request`] in, a [`Response`] out,+//! and [`Event`]s along the way.+//!+//! All three are serde types with a fixed JSON shape, and that shape is the+//! contract: a transport is free to carry it over stdin, a socket or a+//! function call, but never to reinterpret it. Requests are tagged by+//! `method`, events by `event`, and a response is `{"result": ...}` or+//! `{"error": "..."}`. Fields are camelCase, matching the document and the+//! command layer, so a caller learns one spelling.+//!+//! The edit vocabulary is not redefined here. [`Request::EditApply`] carries+//! a `concat_project` [`Command`] as it is, so every operation the window+//! can perform is one an API caller can perform, with the same clamps and+//! the same refusals, and a new command needs nothing added here.++use concat_host::export::Progress;+use concat_host::media::MediaSummary;+use concat_host::session::EditorView;+use concat_host::templates::TemplateInfo;+use concat_host::{AppDirs, ProjectInfo};+use concat_project::Command;+use concat_project::model::VideoSettings;+use serde::{Deserialize, Serialize};+use serde_json::Value;++/// The version of this contract. Bumped when a method's shape changes in a+/// way a caller written against the previous one would misread; adding a+/// method or an optional field does not bump it.+pub const API_VERSION: &str = "0.1";++/// What a caller asks for. One variant per method, named `area.verb`.+///+/// Every method that touches a project names it by its folder, the same+/// path [`Request::ProjectOpen`] took; the API keeps one session per folder+/// and a method on a folder that is not open is an error, not a silent+/// open, so a caller always knows what it is editing.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(tag = "method", rename_all_fields = "camelCase")]+pub enum Request {+ /// The API's version and the build behind it.+ #[serde(rename = "version")]+ Version,++ /// Creates a project folder under `location`, named `name`, and opens+ /// it. Refuses a folder that already holds a project.+ #[serde(rename = "project.create")]+ ProjectCreate {+ /// The directory the project folder is made in.+ location: String,+ /// The project's name; the folder is named after it.+ name: String,+ /// Frame and rate; 1080p at 30 when absent.+ #[serde(default)]+ video: Option<VideoSettings>,+ },+ /// Opens a project folder. Opening one already open returns its state+ /// as it stands, edits and all.+ #[serde(rename = "project.open")]+ ProjectOpen {+ /// The project folder.+ path: String,+ },+ /// Closes an open project, saving first when asked. Unsaved edits are+ /// dropped otherwise.+ #[serde(rename = "project.close")]+ ProjectClose {+ /// The project folder.+ path: String,+ /// Whether to write the document before closing.+ #[serde(default)]+ save: bool,+ },+ /// The projects this machine opened most recently, newest first.+ #[serde(rename = "project.list")]+ ProjectList,+ /// The state of an open project.+ #[serde(rename = "project.get")]+ ProjectGet {+ /// The project folder.+ path: String,+ },+ /// The document exactly as a save would write it.+ #[serde(rename = "project.document")]+ ProjectDocument {+ /// The project folder.+ path: String,+ },+ /// Writes the document to the project folder.+ #[serde(rename = "project.save")]+ ProjectSave {+ /// The project folder.+ path: String,+ /// A new name for the project, when renaming.+ #[serde(default)]+ name: Option<String>,+ },+ /// Sets the active timeline's frame and rate, as an undoable edit.+ #[serde(rename = "project.setVideo")]+ ProjectSetVideo {+ /// The project folder.+ path: String,+ /// The frame and rate.+ video: VideoSettings,+ },++ /// Applies one edit command. The command's own notes say what each+ /// does; a refusal comes back as the sentence the window would show.+ #[serde(rename = "edit.apply")]+ EditApply {+ /// The project folder.+ path: String,+ /// The edit. Boxed because a command is the largest thing a+ /// request carries; the JSON is the command as it is.+ command: Box<Command>,+ },+ /// Steps the history back one edit.+ #[serde(rename = "edit.undo")]+ EditUndo {+ /// The project folder.+ path: String,+ },+ /// Steps the history forward one edit.+ #[serde(rename = "edit.redo")]+ EditRedo {+ /// The project folder.+ path: String,+ },++ /// Reports what is inside a media file without touching any project.+ #[serde(rename = "media.probe")]+ MediaProbe {+ /// The file.+ path: String,+ },+ /// Probes a file and adds it to a project's bin: what dropping a file+ /// on the window does. A path already in the bin is a no-op.+ #[serde(rename = "media.import")]+ MediaImport {+ /// The project folder.+ path: String,+ /// The file to import.+ file: String,+ },++ /// Every effect package the build knows, with its parameters, so a+ /// caller can build a valid chain without reading a manifest.+ #[serde(rename = "catalogue.list")]+ CatalogueList {+ /// Restrict to one kind: "effect", "filter", "audio", "transition"+ /// or "generator".+ #[serde(default)]+ kind: Option<String>,+ },++ /// The template library.+ #[serde(rename = "template.list")]+ TemplateList,+ /// Makes a project from a template with every slot filled, and opens+ /// it. Every slot must be filled; a set that leaves one empty makes+ /// nothing.+ #[serde(rename = "template.instantiate")]+ TemplateInstantiate {+ /// The template bundle folder.+ template: String,+ /// The directory the project folder is made in.+ location: String,+ /// The project's name.+ name: String,+ /// The file for each slot.+ fills: Vec<Fill>,+ },+ /// Packs an open project into a new template bundle.+ #[serde(rename = "template.save")]+ TemplateSave {+ /// The project folder.+ path: String,+ /// The template's name.+ name: String,+ },++ /// Renders an open project's active timeline to a file, exactly as+ /// the window's Export does: titles painted, cutouts analysed, then the+ /// frame loop and the mix. Blocks until the file is written, reporting+ /// through [`Event`]s.+ #[serde(rename = "export.run")]+ ExportRun {+ /// The project folder.+ path: String,+ /// How to render it.+ #[serde(flatten)]+ spec: ExportSpec,+ },+ /// Stops the export that is running, if one is. Meaningful only from a+ /// transport that can speak while a dispatch blocks.+ #[serde(rename = "export.cancel")]+ ExportCancel,++ /// Composites the true frame at one instant and writes it as a PNG.+ #[serde(rename = "preview.frame")]+ PreviewFrame {+ /// The project folder.+ path: String,+ /// The timeline instant, in seconds.+ time: f64,+ /// The file to write.+ output: String,+ /// Frame width; the timeline's when absent.+ #[serde(default)]+ width: Option<u32>,+ /// Frame height; the timeline's when absent.+ #[serde(default)]+ height: Option<u32>,+ },+}++/// One template slot and the file that takes it.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct Fill {+ /// The placeholder's media id in the template.+ pub media_id: String,+ /// The file to put there; probed on the way in.+ pub file: String,+}++/// How an export is rendered. Everything but the output is optional and+/// defaults to what the window's sheet defaults to.+#[derive(Clone, PartialEq, Debug, Default, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct ExportSpec {+ /// The file to write.+ pub output: String,+ /// Constant rate factor; lower is better and bigger. 20 when absent.+ #[serde(default)]+ pub crf: Option<u8>,+ /// The x264 preset name. "medium" when absent.+ #[serde(default)]+ pub preset: Option<String>,+ /// Output width; the timeline's when absent.+ #[serde(default)]+ pub width: Option<u32>,+ /// Output height; the timeline's when absent.+ #[serde(default)]+ pub height: Option<u32>,+ /// Frame rate numerator; the timeline's when absent.+ #[serde(default)]+ pub rate_num: Option<i64>,+ /// Frame rate denominator; the timeline's when absent.+ #[serde(default)]+ pub rate_den: Option<i64>,+}++/// What a request hands back. Serialised as the payload alone: the variant+/// is implied by the method, so a caller that sent `project.get` reads an+/// editor view and nothing wraps it.+#[derive(Clone, Serialize)]+#[serde(untagged)]+pub enum Reply {+ /// [`Request::Version`].+ Version(VersionInfo),+ /// The project after a change, or as it stands.+ View(Box<EditorView>),+ /// [`Request::ProjectList`].+ Projects(Vec<ProjectInfo>),+ /// [`Request::ProjectDocument`].+ Document(Value),+ /// [`Request::MediaProbe`].+ Media(MediaSummary),+ /// [`Request::CatalogueList`].+ Packages(Vec<PackageInfo>),+ /// [`Request::TemplateList`].+ Templates(Vec<TemplateInfo>),+ /// [`Request::TemplateSave`].+ Template(TemplateInfo),+ /// [`Request::ExportRun`] and [`Request::PreviewFrame`]: the file+ /// written.+ Written(Written),+ /// Methods with nothing to say beyond having worked.+ Done(Done),+}++/// The build and the contract it speaks.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct VersionInfo {+ /// [`API_VERSION`].+ pub api_version: String,+ /// The Concat build.+ pub concat: String,+ /// Where this machine keeps recents, templates and models.+ pub dirs: Dirs,+}++/// The app's directories, as paths.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct Dirs {+ /// Small state: recents, settings, the template library.+ pub config: String,+ /// Large state: downloaded models, painted titles.+ pub data: String,+}++impl From<&AppDirs> for Dirs {+ fn from(dirs: &AppDirs) -> Self {+ Dirs {+ config: dirs.config.to_string_lossy().into_owned(),+ data: dirs.data.to_string_lossy().into_owned(),+ }+ }+}++/// A file the API wrote.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct Written {+ /// The file's path.+ pub path: String,+ /// Its picture size.+ pub width: u32,+ /// Its picture size.+ pub height: u32,+}++/// The empty reply, an object so every reply is one.+#[derive(Clone, Copy, PartialEq, Debug, Default, Serialize, Deserialize)]+pub struct Done {}++/// One effect package, as a caller building a chain needs it.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct PackageInfo {+ /// The id a clip's chain stores, `author.name`.+ pub id: String,+ /// What the catalogue card says.+ pub name: String,+ /// "effect", "filter", "audio", "transition" or "generator".+ pub kind: String,+ /// The shelf the card sits on.+ pub category: String,+ /// A sentence about it.+ pub description: String,+ /// The parameter the simple view shows as the one slider, if any.+ pub intensity: Option<String>,+ /// The knobs, in the order the inspector shows them.+ pub params: Vec<ParamInfo>,+}++/// One parameter of a package.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct ParamInfo {+ /// The key a chain entry's `params` stores it under.+ pub key: String,+ /// What the control is labelled.+ pub label: String,+ /// "float", "int", "bool", "enum", "color" or "point".+ #[serde(rename = "type")]+ pub kind: String,+ /// Lowest value.+ pub min: f64,+ /// Highest value.+ pub max: f64,+ /// The value an untouched control means.+ pub default: f64,+ /// Slider increment; 0 means continuous.+ pub step: f64,+ /// Displayed after the number.+ pub unit: String,+ /// Whether the control can carry keyframes.+ pub animate: bool,+ /// For "enum": the values the document may hold.+ pub values: Vec<f64>,+ /// For "enum": what each value is called, in `values` order.+ pub labels: Vec<String>,+}++/// A request's outcome. `{"result": ...}` when it worked, `{"error":+/// "..."}` with the sentence a person would be shown when it did not.+#[derive(Clone, Serialize)]+#[serde(rename_all = "camelCase")]+pub enum Response {+ /// The reply.+ Result(Reply),+ /// Why there is none.+ Error(String),+}++impl From<Result<Reply, String>> for Response {+ fn from(result: Result<Reply, String>) -> Self {+ match result {+ Ok(reply) => Response::Result(reply),+ Err(error) => Response::Error(error),+ }+ }+}++/// Something that happened while a request ran. A transport forwards+/// these as they come; a caller that does not care ignores them.+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]+#[serde(tag = "event", rename_all_fields = "camelCase")]+pub enum Event {+ /// The export moved on.+ #[serde(rename = "export.progress")]+ ExportProgress {+ /// Frames done.+ frame: i64,+ /// Frames in total.+ total: i64,+ /// "rendering", "mixing audio" or "muxing".+ stage: String,+ },+ /// A cutout's masks are being found ahead of an export.+ #[serde(rename = "cutout.progress")]+ CutoutProgress {+ /// The media being analysed.+ media_id: String,+ /// True while a model downloads, false while it runs.+ fetching: bool,+ /// How far along, `0..=1`.+ fraction: f32,+ },+}++impl From<Progress> for Event {+ fn from(progress: Progress) -> Self {+ Event::ExportProgress {+ frame: progress.frame,+ total: progress.total,+ stage: progress.stage.to_owned(),+ }+ }+}diff --git a/engine/crates/concat-cli/Cargo.toml b/engine/crates/concat-cli/Cargo.tomlindex a5c92fa3..20adf6ac 100644--- a/engine/crates/concat-cli/Cargo.toml+++ b/engine/crates/concat-cli/Cargo.toml@@ -15,6 +15,9 @@ path = "src/main.rs"concat-core.workspace = trueconcat-media.workspace = trueconcat-render.workspace = true+# The `api` command: the Concat API over stdin and stdout.+concat-api.workspace = true+serde_json.workspace = trueclap.workspace = true[lints]diff --git a/engine/crates/concat-cli/src/main.rs b/engine/crates/concat-cli/src/main.rsindex cbbfeaaa..7f773676 100644--- a/engine/crates/concat-cli/src/main.rs+++ b/engine/crates/concat-cli/src/main.rs@@ -5,9 +5,12 @@//!//! This exists so the engine can be exercised end to end without a UI. The//! `render` command is the vertical slice: probe, build a timeline, plan every-//! frame, decode, composite, encode.+//! frame, decode, composite, encode. The `api` command is the Concat API's+//! first transport: JSON requests in, JSON responses and events out, one per+//! line, so a script in any language edits and exports a project.use std::error::Error;+use std::io::{BufRead, Write};use std::path::PathBuf;use clap::{Parser, Subcommand};@@ -44,6 +47,13 @@ enum Command {#[arg(long, default_value_t = 15)]fade: u64,},++ /// Speak the Concat API: a JSON request per line on stdin, a response per+ /// line on stdout, with events in between as they happen.+ Api {+ /// One request to run instead of reading stdin.+ request: Option<String>,+ },}fn main() -> Result<(), Box<dyn Error>> {@@ -55,6 +65,48 @@ fn main() -> Result<(), Box<dyn Error>> {frames,fade,} => render(&input, &output, frames, fade),+ Command::Api { request } => api(request),+ }+}++/// The stdin transport. Every line in is one request; every line out is+/// one JSON object, either an event or the response, flushed as written so+/// a caller reading a pipe sees progress as it happens. A line that is not+/// a request gets an error response and the loop goes on.+fn api(single: Option<String>) -> Result<(), Box<dyn Error>> {+ let mut api = concat_api::Api::new()?;+ let stdout = std::io::stdout();++ let mut serve = |line: &str| -> Result<(), Box<dyn Error>> {+ let mut out = stdout.lock();+ let response = match serde_json::from_str::<concat_api::Request>(line) {+ Ok(request) => api.dispatch(request, &mut |event| {+ // An event that cannot be written is a caller that went+ // away; the response's write will say so.+ let _ = serde_json::to_writer(&mut out, &event)+ .and_then(|()| writeln!(out).map_err(serde_json::Error::io));+ let _ = out.flush();+ }),+ Err(error) => concat_api::Response::Error(format!("not a request: {error}")),+ };+ serde_json::to_writer(&mut out, &response)?;+ writeln!(out)?;+ out.flush()?;+ Ok(())+ };++ match single {+ Some(line) => serve(&line),+ None => {+ for line in std::io::stdin().lock().lines() {+ let line = line?;+ if line.trim().is_empty() {+ continue;+ }+ serve(&line)?;+ }+ Ok(())+ }}}diff --git a/engine/crates/concat-host/Cargo.toml b/engine/crates/concat-host/Cargo.tomlindex 5df82db3..527043b1 100644--- a/engine/crates/concat-host/Cargo.toml+++ b/engine/crates/concat-host/Cargo.toml@@ -9,7 +9,7 @@ description = "What the editor window needs that is not the edit itself: session[features]# The wgpu compositor for previews and exports, off by default for the same# reason as everywhere else in the engine; the window turns it on.-gpu = ["concat-export/gpu", "dep:concat-render", "concat-render/gpu"]+gpu = ["concat-export/gpu", "concat-render/gpu"][dependencies]concat-core.workspace = true@@ -22,7 +22,9 @@ concat-text.workspace = true# runs them, and ureq because the host is what fetches them.concat-vision = { workspace = true, features = ["infer"] }ureq = "2"-concat-render = { workspace = true, optional = true }+# The CPU compositor draws the monitor's frame; the wgpu one is behind+# `gpu`, so a headless host pays for no GPU tree.+concat-render.workspace = trueserde = { version = "1", features = ["derive"] }serde_json.workspace = truediff --git a/engine/crates/concat-host/src/cutout.rs b/engine/crates/concat-host/src/cutout.rsindex 557ba5df..df5098b9 100644--- a/engine/crates/concat-host/src/cutout.rs+++ b/engine/crates/concat-host/src/cutout.rs@@ -29,7 +29,7 @@ use std::sync::{Arc, Mutex, OnceLock};use concat_core::time::{FrameRate, Rational};use concat_media::{DecodeOptions, Decoder, FrameSource};-use concat_project::model::Subject;+use concat_project::model::{MediaKind, Project, Subject};use concat_vision::segment::Kind;use concat_vision::{MASK_RATE, MaskStore, ModelId, Segmenter, mask_dir, models};@@ -152,6 +152,52 @@ impl Cutouts {}}+ /// What the active timeline's cutouts need analysed: one request per+ /// media and subject, since two clips of one file that keep different+ /// things need different masks, each covering the stretches of source+ /// its clips show. Paired with the media's id, for a caller that keys+ /// its bookkeeping by it. The window asks after every change and the+ /// API asks before an export; both get the same list.+ pub fn requests(project: &Project, project_dir: &Path) -> Vec<(String, AnalyseRequest)> {+ let mut wanted: Vec<(String, AnalyseRequest)> = Vec::new();+ for clip in &project.active().clips {+ let Some(cutout) = clip.cutout.as_ref() else {+ continue;+ };+ if !clip.kind.is_visual() {+ continue;+ }+ let Some(media) = project.media_by_id(&clip.media_id) else {+ continue;+ };+ // The source the clip shows: its in-point, for as long as it+ // runs at its speed. A curve's mean is its speed, so this+ // covers a curved clip too.+ let range = (+ clip.source_start,+ clip.source_start + clip.duration * clip.speed.max(0.0625),+ );+ match wanted+ .iter_mut()+ .find(|(id, request)| *id == media.id && request.subject == cutout.subject)+ {+ Some((_, request)) => request.ranges.push(range),+ None => wanted.push((+ media.id.clone(),+ AnalyseRequest {+ project: project_dir.to_path_buf(),+ media_path: media.path.clone(),+ media_size: (media.width.unwrap_or(0), media.height.unwrap_or(0)),+ still: media.kind == MediaKind::Image,+ subject: cutout.subject,+ ranges: vec![range],+ },+ )),+ }+ }+ wanted+ }+/// How many instants `request` still needs, without doing anything./// A store made by a model the subject would not pick now needs all/// of them again.diff --git a/engine/crates/concat-host/src/preview.rs b/engine/crates/concat-host/src/preview.rsindex c85f9f81..d73bb160 100644--- a/engine/crates/concat-host/src/preview.rs+++ b/engine/crates/concat-host/src/preview.rs@@ -14,9 +14,7 @@//! pictures go up once, the composite happens where it is shown, and no//! pixel comes back down.-use std::sync::Arc;-#[cfg(feature = "gpu")]-use std::sync::Mutex;+use std::sync::{Arc, Mutex};use concat_export::ExportClip;use concat_project::DocumentSettings;diff --git a/engine/crates/concat-project/src/model.rs b/engine/crates/concat-project/src/model.rsindex c39dce83..9f7abc4b 100644--- a/engine/crates/concat-project/src/model.rs+++ b/engine/crates/concat-project/src/model.rs@@ -729,9 +729,12 @@ pub enum TextAlign {}/// A title's styling. Sizes are fractions of the frame, so a title composed-/// against 1080p lands correctly exported at 4K.+/// against 1080p lands correctly exported at 4K. A style arriving in a+/// command needs only the fields it changes; the rest are the default's, so+/// a caller can say `{ "content": "Hello" }` and get a title that looks like+/// one the window would place.#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]-#[serde(rename_all = "camelCase")]+#[serde(rename_all = "camelCase", default)]pub struct TextStyle {/// The words themselves, newlines included. The clip's display name is/// snapshotted from the first non-empty line.diff --git a/engine/crates/concat/src/studio.rs b/engine/crates/concat/src/studio.rsindex 898f0ed8..aca5c97a 100644--- a/engine/crates/concat/src/studio.rs+++ b/engine/crates/concat/src/studio.rs@@ -3948,42 +3948,13 @@ impl Studio {return;};let project = std::path::PathBuf::from(session.path());- let mut wanted: Vec<(String, AnalyseRequest)> = Vec::new();- for clip in &self.timeline().clips {- let Some(cutout) = clip.cutout.as_ref() else {- continue;- };- if !clip.kind.is_visual() {- continue;- }- let Some(media) = self.project().media_by_id(&clip.media_id) else {- continue;- };- // One analysis per media and subject: two clips of one file- // that keep different things need different masks.- let key = Self::analysis_key(&media.id, cutout.subject);- // The source the clip shows: its in-point, for as long as it- // runs at its speed. A curve's mean is its speed, so this- // covers a curved clip too.- let range = (- clip.source_start,- clip.source_start + clip.duration * clip.speed.max(0.0625),- );- match wanted.iter_mut().find(|(id, _)| *id == key) {- Some((_, request)) => request.ranges.push(range),- None => wanted.push((- key,- AnalyseRequest {- project: project.clone(),- media_path: media.path.clone(),- media_size: (media.width.unwrap_or(0), media.height.unwrap_or(0)),- still: media.kind == model::MediaKind::Image,- subject: cutout.subject,- ranges: vec![range],- },- )),- }- }+ // One analysis per media and subject, keyed the way the job map is.+ let wanted: Vec<(String, AnalyseRequest)> = Cutouts::requests(self.project(), &project)+ .into_iter()+ .map(|(media_id, request)| {+ (Self::analysis_key(&media_id, request.subject), request)+ })+ .collect();let Some((id, request)) = wanted.into_iter().find(|(_, request)| Cutouts::outstanding(request) > 0)