Skip to content

The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.

Add the Concat API as one dispatcher crate

Jareer315ecdd52abf
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index d920884c..66ca2558 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -40,7 +40,7 @@ Two consequences of the rule shape everything below:
## 2. The map
-Everything lives in `engine/`, one Cargo workspace of thirteen crates.
+Everything lives in `engine/`, one Cargo workspace of fourteen crates.
| Crate | Owns | Lines |
|---|---|---|
@@ -54,7 +54,8 @@ Everything lives in `engine/`, one Cargo workspace of thirteen crates.
| `concat-vision` | What the engine sees in a picture: the person mask behind a cutout, brush strokes over it, the frame cut by it; the model itself behind `infer`. | 0.9k |
| `concat-host` | What a window needs that is not the edit: sessions, project folders, caches, the monitor, audio playback, the export driver, templates, job slots, app directories. | 3.8k |
| `concat-speech` | Transcription with whisper.cpp and text to speech with Kokoro, both in-process, models downloaded on demand. | 1.2k |
-| `concat-cli` | A driver for the engine without a window: `probe` and `render`. | 0.2k |
+| `concat-api` | The Concat API: one dispatcher of JSON requests over the host - projects, edits, media, the catalogue, templates, exports, frames - that every transport calls. | 0.9k |
+| `concat-cli` | A driver for the engine without a window: `probe`, `render`, and `api`, the API over stdin and stdout. | 0.3k |
| `concat` | The editor window in Slint, and the Rust that binds it to the host. | 7.2k Rust, 15.9k Slint |
| `concat-android` | The Android activity: `android_main` over the window. | 20 |
@@ -71,7 +72,8 @@ concat ──► concat-speech ──► concat-host ──► concat-export ─
├──► concat-effects (the catalogue, for the inspector's shelves)
└──► concat-vision (a press on the stage, mapped into the source)
-concat-android ──► concat concat-cli ──► concat-core, concat-media, concat-render
+concat-android ──► concat concat-api ──► concat-host, concat-export, concat-effects, concat-project
+ concat-cli ──► concat-api, concat-core, concat-media, concat-render
```
`concat-core` depends on nothing; `concat-project` is not folded into it
@@ -565,7 +567,63 @@ native libraries stay out of everything that does not speak.
---
-## 13. The window: `concat`
+## 13. The API: `concat-api`
+
+The editor driven without its window. Every way of doing that - the
+command line today, a daemon on a socket, an MCP server, a plugin - is a
+transport, and a transport is a loop: read a `Request`, hand it to
+`Api::dispatch`, write the `Response` and whatever `Event`s fired on the
+way. What a request means lives in this one crate, so two transports
+cannot disagree and a method added here reaches all of them.
+
+**The shape.** `Request` is a serde enum tagged by `method`, named
+`area.verb`: `project.create`, `project.open`, `project.close`,
+`project.list`, `project.get`, `project.document`, `project.save`,
+`project.setVideo`, `edit.apply`, `edit.undo`, `edit.redo`,
+`media.probe`, `media.import`, `catalogue.list`, `template.list`,
+`template.instantiate`, `template.save`, `export.run`, `export.cancel`,
+`preview.frame`, `version`. A response is `{"result": ...}` or
+`{"error": "..."}`, the error being the sentence the window would show.
+Events are tagged by `event`: export progress, and cutout analysis ahead
+of an export. Fields are camelCase, the document's and the command
+layer's spelling, so a caller learns one. `API_VERSION` names the
+contract and moves only when a caller written against the previous one
+would misread a reply.
+
+**The edit is not redefined.** `edit.apply` carries a `concat-project`
+`Command` exactly as the window sends one, clamps and refusals included,
+so every operation the window can perform is one a script can perform
+and a new command needs nothing added here. A `Batch` is one request and
+one undo step. A text style in a command needs only the fields it sets.
+
+**What the crate owns is the choreography** the window performs by hand.
+`media.import` is the probe and the add. `template.instantiate` probes
+every fill before anything is made, so a bad path refuses the request
+rather than leaving a folder behind. `export.run` finds every mask the
+timeline's cutouts still need (`Cutouts::requests`, the same list the
+window asks for after each change), paints the titles, then renders, so
+the file the API writes is the file the window would have written.
+`preview.frame` is the paused monitor's true frame, titles included, as
+a PNG.
+
+**One `Api`, one session per project folder**, keyed by the folder's
+canonical path, so `.` and its absolute spelling are one project.
+A method on a folder that is not open is an error, never a silent open.
+The `Api` is not thread-safe on purpose: a transport serving several
+callers owns the one `Api` and serialises through it, as the window's
+event loop does, and a long method blocks its caller. `Api::exporter` is
+the one handle that crosses threads, so a cancel can reach a running
+export.
+
+**The first transport** is `concat-cli api`: one request per line on
+stdin, one JSON object per line on stdout, events interleaved and
+flushed as they happen, so a pipe from any language edits and exports a
+project. A line that is not a request gets an error and the loop goes
+on.
+
+---
+
+## 14. The window: `concat`
**One library, three entry points.** `concat::run()` is the program.
`main.rs` calls it on the desktop and on iOS; `concat-android`'s
@@ -670,7 +728,7 @@ it; CI runs the check.
---
-## 14. Build and delivery
+## 15. Build and delivery
- **Toolchain.** Pinned in `engine/rust-toolchain.toml`. `cargo build`
needs the FFmpeg 7+ development libraries, cmake and a C++ toolchain
@@ -708,7 +766,7 @@ it; CI runs the check.
---
-## 15. Testing
+## 16. Testing
240 unit tests, all in-file `#[cfg(test)]` modules; there are no `tests/`
directories. The pure layers are where the coverage is dense: rational
@@ -719,30 +777,46 @@ fixtures (a package without fixtures fails the build), the audio graph's
shape, seek policy and LRU eviction, filter strings, all seven
transition-resolution behaviours, CPU/GPU parity, the speech model
bookkeeping, and the host's cache keys, WAV parsing, sweep ordering and
-template packing. Tests that touch FFmpeg encode a small file into the
+template packing, and the API's request shapes and project life cycle
+in a scratch folder. Tests that touch FFmpeg encode a small file into the
temporary directory first.
The window has unit tests for its formatting helpers and stage geometry.
The seams that a window drives end to end (publish, the dock tree, the
echo and commit path, playback against a device) are exercised by using
-the app, which is the work in 15.1.
+the app, which is the work in 17.1.
---
-## 16. Next work
+## 17. Next work
Ordered by how much each matters. Each is either an invariant a
contributor could break without knowing it, or a piece of work with a
known shape.
-### 16.1 Put a real project through every flow in the window
+### 17.1 Put a real project through every flow in the window
Import, place, trim, split, transitions, a title, an effect, a speed
curve, transcription, narration, a template, an export, on each renderer
and each platform. The engine's seams are tested; the window's are used.
What that pass turns up goes above everything below.
-### 16.2 A phone layout
+### 17.2 The API's other transports
+
+The dispatcher is in place and the command line speaks it. The daemon is
+next: `concat serve`, JSON-RPC over stdio or a local socket, the same
+`Request` and `Response` with an id on each, and `export.cancel` finally
+meaningful because a second message can arrive while a first blocks.
+MCP is that daemon with tool schemas derived from the request enum. Then
+the window itself listening on the same socket, so a script drives the
+live edit and the lanes redraw, which is `Shell::with` through
+`invoke_from_event_loop` and nothing new. Two decisions come with them: a
+lock file in the project folder, or a window that reloads on an external
+change, since the window autosaves and last writer wins today; and
+whether speech joins the dispatcher behind a feature, so a headless
+export never links whisper.
+
+### 17.3 A phone layout
The window builds and packages for Android and iOS, and what runs there is
the desktop's tree on a phone's screen. A phone wants one seat at a time,
@@ -754,7 +828,7 @@ asked for by name (they are linked, and the software decoder is what is
opened); and the monitor on Android, which composites on the CPU because
Slint's Android backend creates its own device.
-### 16.3 Two paths that build different chains
+### 17.4 Two paths that build different chains
On the GPU, effects with a shader leave the FFmpeg chain and travel as
passes; on the CPU they stay in the chain. A machine with an adapter and
@@ -765,7 +839,7 @@ render on one machine and not the other. Either require the chain for
`effect` kinds in validation, or teach the CPU path to refuse the package
loudly.
-### 16.4 Small things the code already knows about
+### 17.5 Small things the code already knows about
- `MoveTimeline`'s documented index is counted with the timeline removed;
the clamp is against the unremoved length.
@@ -794,12 +868,16 @@ loudly.
---
-## 17. How to change things
+## 18. How to change things
- **A new operation on the edit.** Add a `Command` variant in
`concat-project/src/commands.rs`, make its `apply` report `applied`
honestly, add it to the serde round-trip test's list, then bind a
callback in `concat/src/lib.rs` that builds it from the window's state.
+- **A new API method.** A `Request` variant in
+ `concat-api/src/message.rs` with its `method` name, a `Reply` for what
+ it returns if none fits, and its arm in `Api::run`. Every transport
+ carries it from then on; none of them changes.
- **A new effect.** Add a folder under `concat-effects/packages/` with
`effect.toml` and `fixtures.toml` (and `effect.wgsl` if it has a
shader). The build fails until the fixtures cover default, minimum and
diff --git a/engine/Cargo.lock b/engine/Cargo.lock
index d02f8f67..95af1722 100644
--- a/engine/Cargo.lock
+++ b/engine/Cargo.lock
@@ -1176,14 +1176,30 @@ dependencies = [
"slint",
]
+[[package]]
+name = "concat-api"
+version = "0.2.1"
+dependencies = [
+ "concat-effects",
+ "concat-export",
+ "concat-host",
+ "concat-project",
+ "png 0.17.16",
+ "serde",
+ "serde_json",
+ "tempfile",
+]
+
[[package]]
name = "concat-cli"
version = "0.2.1"
dependencies = [
"clap",
+ "concat-api",
"concat-core",
"concat-media",
"concat-render",
+ "serde_json",
]
[[package]]
diff --git a/engine/Cargo.toml b/engine/Cargo.toml
index d1f2da92..c5ebb9aa 100644
--- a/engine/Cargo.toml
+++ b/engine/Cargo.toml
@@ -15,6 +15,7 @@ concat-project = { path = "crates/concat-project" }
concat-render = { path = "crates/concat-render" }
concat-export = { path = "crates/concat-export" }
concat-host = { path = "crates/concat-host" }
+concat-api = { path = "crates/concat-api" }
concat-speech = { path = "crates/concat-speech" }
concat-effects = { path = "crates/concat-effects" }
concat-text = { path = "crates/concat-text" }
diff --git a/engine/README.md b/engine/README.md
index d99008e5..6e67ef9b 100644
--- a/engine/README.md
+++ b/engine/README.md
@@ -12,7 +12,8 @@ The video engine behind Concat. Rust, no GC, no hidden control flow.
| `concat-render` | Turning a timeline plus a timestamp into one finished frame. | `concat-core`, wgpu (optional) |
| `concat-effects` | Effect packages: manifests, chain templates, the catalogue. Each built-in effect is a folder under `packages/`. | `concat-project`, serde, toml |
| `concat-export` | Timeline to file: flatten, the frame-by-frame render loop, the paused monitor's true frame. | `concat-core`, `concat-media`, `concat-render`, `concat-project`, `concat-effects` |
-| `concat-cli` | A binary to drive the above. The vertical slice. | the engine crates |
+| `concat-api` | The Concat API: JSON requests in, responses and events out. Projects, edits, media, the catalogue, templates, exports, frames. | `concat-host`, `concat-export`, `concat-effects`, `concat-project` |
+| `concat-cli` | A binary to drive the above: `probe`, `render`, and `api`, the API over stdin and stdout. | `concat-api`, the engine crates |
| `concat-host` | What the window needs that is not the edit: sessions, project folders, previews, playback, templates, job slots. | `concat-media`, `concat-project`, `concat-export`, cpal |
| `concat-speech` | Transcription (whisper.cpp, in-process) and text to speech (Kokoro via sherpa-onnx). | `concat-host`, `concat-media`, whisper-rs, sherpa-onnx |
| `concat` | The editor window: every pane, dialog and primitive, in Slint. The app a user launches. | slint, `concat-host`, `concat-speech` |
@@ -24,6 +25,7 @@ The dependency arrows point one way:
concat -> concat-speech -> concat-host -> {export, project, media} -> core
-> render -> core
export -> effects -> project
+concat-cli -> concat-api -> concat-host
```
If you ever find yourself wanting `core` to depend on `media`, something has
@@ -40,6 +42,8 @@ cargo build
cargo test
cargo run -p concat-cli -- probe some-video.mp4
cargo run -p concat-cli -- render some-video.mp4 out.mp4 --frames 120
+cargo run -p concat-cli -- api '{"method":"catalogue.list","kind":"filter"}'
+cargo run -p concat-cli -- api < edits.jsonl # one request per line; see concat-api
cargo run -p concat # the editor window, debug
cargo run --profile quick -p concat # optimised, rebuilds in seconds: for trying changes
cargo build --profile app -p concat # the shipping binary: fat LTO, panic=abort, stripped
diff --git a/engine/crates/concat-api/Cargo.toml b/engine/crates/concat-api/Cargo.toml
new file mode 100644
index 00000000..5a5b70b1
--- /dev/null
+++ b/engine/crates/concat-api/Cargo.toml
@@ -0,0 +1,24 @@
+[package]
+name = "concat-api"
+version.workspace = true
+edition.workspace = true
+rust-version.workspace = true
+license.workspace = true
+description = "The Concat API: one dispatcher for projects, edits, media, effects, templates, exports and frames, which every transport - the command line, a daemon, a plugin - calls."
+
+[dependencies]
+concat-project.workspace = true
+concat-effects.workspace = true
+concat-export.workspace = true
+concat-host.workspace = true
+serde = { version = "1", features = ["derive"] }
+serde_json.workspace = true
+# The frame method hands back a PNG: the one picture format every
+# caller reads and that keeps the compositor's pixels exact.
+png = "0.17"
+
+[dev-dependencies]
+tempfile = "3"
+
+[lints]
+workspace = true
diff --git a/engine/crates/concat-api/src/lib.rs b/engine/crates/concat-api/src/lib.rs
new file mode 100644
index 00000000..9f447205
--- /dev/null
+++ b/engine/crates/concat-api/src/lib.rs
@@ -0,0 +1,809 @@
+// SPDX-License-Identifier: AGPL-3.0-or-later
+// SPDX-FileCopyrightText: 2026 Jareer and Concat contributors
+
+//! The Concat API: the one dispatcher every way of driving the editor
+//! without its window goes through.
+//!
+//! The command line, a daemon on a socket, an MCP server, a plugin: each is
+//! a transport, and a transport is a loop that reads a [`Request`], hands
+//! it to [`Api::dispatch`], and writes the [`Response`] and any [`Event`]s
+//! back. Nothing about what a request *means* lives in a transport, so two
+//! of them cannot disagree, and a method added here reaches all of them.
+//!
+//! The crate decides nothing about the edit either. Edits are
+//! `concat_project` [`Command`]s carried as they are; projects, media,
+//! templates, titles, cutouts and exports are `concat_host`'s. What this
+//! crate owns is the choreography the window performs by hand - probe then
+//! add, paint titles and find masks before rendering, save through the
+//! session - stated once so a file exported here is the file the window
+//! would have written.
+//!
+//! One [`Api`] holds one session per open project folder. It is not
+//! thread-safe by design: a transport that serves several callers owns the
+//! one `Api` and serialises through it, the way the window's event loop
+//! does, and a long method blocks its caller. [`Api::exporter`] is the one
+//! handle that crosses threads, so a cancel can reach a running export.
+
+pub mod message;
+
+use std::collections::BTreeMap;
+use std::path::{Path, PathBuf};
+use std::sync::Arc;
+
+use concat_effects::Catalogue;
+use concat_effects::manifest::Kind;
+use concat_export::ExportClip;
+use concat_host::cutout::{self, Cutouts};
+use concat_host::export::{self, Exporter};
+use concat_host::preview::{FrameSpec, Monitor};
+use concat_host::session::EditorView;
+use concat_host::templates::{self, SlotFill};
+use concat_host::{AppDirs, ProjectInfo, Session, Titles, media, projects};
+use concat_project::Command;
+use concat_project::model::VideoSettings;
+
+pub use message::{
+ API_VERSION, Dirs, Done, Event, ExportSpec, Fill, PackageInfo, ParamInfo, Reply, Request,
+ Response, VersionInfo, Written,
+};
+
+/// The export sheet's middle quality, and what an export gets when the
+/// caller says nothing.
+const DEFAULT_CRF: u8 = 20;
+/// The x264 preset every export uses unless told otherwise.
+const DEFAULT_PRESET: &str = "medium";
+
+/// The dispatcher: the open sessions and the services behind them.
+pub struct Api {
+ dirs: AppDirs,
+ /// Open projects by canonical folder path.
+ sessions: BTreeMap<String, Session>,
+ titles: Titles,
+ cutouts: Cutouts,
+ monitor: Monitor,
+ exporter: Exporter,
+}
+
+impl Api {
+ /// An API over this machine's app directories.
+ pub fn new() -> Result<Api, String> {
+ Ok(Api::with_dirs(AppDirs::locate()?))
+ }
+
+ /// An API over the given directories: a test's scratch, or an embedder
+ /// with a home of its own.
+ pub fn with_dirs(dirs: AppDirs) -> Api {
+ Api {
+ titles: Titles::new(&dirs),
+ cutouts: Cutouts::new(&dirs.data),
+ monitor: Monitor::new(),
+ exporter: Exporter::new(),
+ sessions: BTreeMap::new(),
+ dirs,
+ }
+ }
+
+ /// The directories this API works under.
+ pub fn dirs(&self) -> &AppDirs {
+ &self.dirs
+ }
+
+ /// The export slot, for a transport that needs to cancel from another
+ /// thread while [`Api::dispatch`] blocks on [`Request::ExportRun`].
+ pub fn exporter(&self) -> Exporter {
+ self.exporter.clone()
+ }
+
+ /// Runs one request. Events fire through `notify` as the work goes;
+ /// the response is what the request is worth.
+ pub fn dispatch(&mut self, request: Request, notify: &mut dyn FnMut(Event)) -> Response {
+ self.run(request, notify).into()
+ }
+
+ fn run(&mut self, request: Request, notify: &mut dyn FnMut(Event)) -> Result<Reply, String> {
+ let view = |view: EditorView| Ok(Reply::View(Box::new(view)));
+ match request {
+ Request::Version => Ok(Reply::Version(self.version())),
+ Request::ProjectCreate {
+ location,
+ name,
+ video,
+ } => view(self.create(&location, &name, video.unwrap_or_default())?),
+ Request::ProjectOpen { path } => view(self.open(&path)?),
+ Request::ProjectClose { path, save } => {
+ self.close(&path, save)?;
+ Ok(Reply::Done(Done {}))
+ }
+ Request::ProjectList => Ok(Reply::Projects(self.recents())),
+ Request::ProjectGet { path } => view(self.session(&path)?.view()),
+ Request::ProjectDocument { path } => {
+ Ok(Reply::Document(self.session(&path)?.document()))
+ }
+ Request::ProjectSave { path, name } => {
+ self.save(&path, name.as_deref())?;
+ Ok(Reply::Done(Done {}))
+ }
+ Request::ProjectSetVideo { path, video } => {
+ view(self.session_mut(&path)?.set_video(video)?)
+ }
+ Request::EditApply { path, command } => view(self.apply(&path, *command)?),
+ Request::EditUndo { path } => view(self.session_mut(&path)?.undo()),
+ Request::EditRedo { path } => view(self.session_mut(&path)?.redo()),
+ Request::MediaProbe { path } => Ok(Reply::Media(media::probe(&path)?)),
+ Request::MediaImport { path, file } => view(self.import(&path, &file)?),
+ Request::CatalogueList { kind } => Ok(Reply::Packages(catalogue(kind.as_deref())?)),
+ Request::TemplateList => Ok(Reply::Templates(templates::list(&self.dirs.config))),
+ Request::TemplateInstantiate {
+ template,
+ location,
+ name,
+ fills,
+ } => view(self.instantiate(&template, &location, &name, fills)?),
+ Request::TemplateSave { path, name } => {
+ Ok(Reply::Template(self.save_template(&path, &name)?))
+ }
+ Request::ExportRun { path, spec } => {
+ Ok(Reply::Written(self.export(&path, &spec, notify)?))
+ }
+ Request::ExportCancel => {
+ self.exporter.cancel();
+ Ok(Reply::Done(Done {}))
+ }
+ Request::PreviewFrame {
+ path,
+ time,
+ output,
+ width,
+ height,
+ } => Ok(Reply::Written(self.frame(
+ &path,
+ time,
+ &output,
+ width.zip(height),
+ )?)),
+ }
+ }
+
+ /// [`Request::Version`].
+ pub fn version(&self) -> VersionInfo {
+ VersionInfo {
+ api_version: API_VERSION.to_owned(),
+ concat: env!("CARGO_PKG_VERSION").to_owned(),
+ dirs: Dirs::from(&self.dirs),
+ }
+ }
+
+ /// [`Request::ProjectCreate`]: the folder, its manifest, and a session
+ /// on it, remembered in recents like a project the window made.
+ pub fn create(
+ &mut self,
+ location: &str,
+ name: &str,
+ video: VideoSettings,
+ ) -> Result<EditorView, String> {
+ let info = projects::create(
+ location,
+ name,
+ video.width,
+ video.height,
+ video.rate_num,
+ video.rate_den,
+ )?;
+ self.adopt(info)
+ }
+
+ /// [`Request::ProjectOpen`].
+ pub fn open(&mut self, path: &str) -> Result<EditorView, String> {
+ let key = key_of(path);
+ if let Some(session) = self.sessions.get(&key) {
+ return Ok(session.view());
+ }
+ let info = projects::open(path)?;
+ self.adopt(info)
+ }
+
+ /// Opens a session on a project the host just described and puts it at
+ /// the front of the recents list.
+ fn adopt(&mut self, info: ProjectInfo) -> Result<EditorView, String> {
+ let session = Session::open_info(&info)?;
+ // Recents are a convenience for the launch screen; a machine whose
+ // config folder cannot be written still edits.
+ let _ = projects::remember(&self.dirs.config, &info);
+ let view = session.view();
+ self.sessions.insert(key_of(&info.path), session);
+ Ok(view)
+ }
+
+ /// [`Request::ProjectClose`].
+ pub fn close(&mut self, path: &str, save: bool) -> Result<(), String> {
+ if save {
+ self.save(path, None)?;
+ }
+ self.sessions
+ .remove(&key_of(path))
+ .map(drop)
+ .ok_or_else(|| not_open(path))
+ }
+
+ /// [`Request::ProjectList`].
+ pub fn recents(&self) -> Vec<ProjectInfo> {
+ projects::list(&self.dirs.config)
+ }
+
+ /// [`Request::ProjectSave`].
+ pub fn save(&mut self, path: &str, name: Option<&str>) -> Result<(), String> {
+ self.session_mut(path)?.save(name)
+ }
+
+ /// [`Request::EditApply`].
+ pub fn apply(&mut self, path: &str, command: Command) -> Result<EditorView, String> {
+ self.session_mut(path)?.apply(command)
+ }
+
+ /// [`Request::MediaImport`]: the probe and the add, as one.
+ pub fn import(&mut self, path: &str, file: &str) -> Result<EditorView, String> {
+ let item = media::probe(file)?.to_new_media();
+ self.apply(path, Command::AddMedia { item })
+ }
+
+ /// [`Request::TemplateInstantiate`].
+ pub fn instantiate(
+ &mut self,
+ template: &str,
+ location: &str,
+ name: &str,
+ fills: Vec<Fill>,
+ ) -> Result<EditorView, String> {
+ // Every file is probed before anything is made, so a bad path
+ // refuses the whole request rather than leaving a folder behind.
+ let fills = fills
+ .into_iter()
+ .map(|fill| {
+ Ok(SlotFill {
+ media_id: fill.media_id,
+ item: media::probe(&fill.file)?.to_new_media(),
+ })
+ })
+ .collect::<Result<Vec<SlotFill>, String>>()?;
+ let info = templates::instantiate(template, location, name, fills)?;
+ self.adopt(info)
+ }
+
+ /// [`Request::TemplateSave`].
+ pub fn save_template(
+ &mut self,
+ path: &str,
+ name: &str,
+ ) -> Result<concat_host::templates::TemplateInfo, String> {
+ let session = self.session(path)?;
+ templates::save(
+ &self.dirs.config,
+ &session.document(),
+ &session.settings(),
+ session.path(),
+ name,
+ )
+ }
+
+ /// [`Request::ExportRun`]: what the window's Export sheet does, in its
+ /// order. Masks first, because the frame loop reads whatever is in the
+ /// project's cache and draws the picture as shot where there is none;
+ /// then titles, which rejoin the clip list as stills; then the render.
+ pub fn export(
+ &mut self,
+ path: &str,
+ spec: &ExportSpec,
+ notify: &mut dyn FnMut(Event),
+ ) -> Result<Written, String> {
+ let session = self.session(path)?;
+ if session.project().active().clips.is_empty() {
+ return Err("There is nothing on the timeline to export".to_owned());
+ }
+ let settings = session.settings();
+ let width = spec.width.unwrap_or(settings.width);
+ let height = spec.height.unwrap_or(settings.height);
+
+ let project_dir = PathBuf::from(session.path());
+ let project = session.project().clone();
+ self.find_masks(&project, &project_dir, notify)?;
+
+ let session = self.session(path)?;
+ let titles = self.title_clips(session, width, height);
+ let host_spec = export::ExportSpec {
+ output: spec.output.clone(),
+ crf: spec.crf.unwrap_or(DEFAULT_CRF),
+ preset: spec
+ .preset
+ .clone()
+ .unwrap_or_else(|| DEFAULT_PRESET.to_owned()),
+ };
+ let mut request = export::request(session, &host_spec, titles);
+ request.width = width;
+ request.height = height;
+ request.rate_num = spec.rate_num.unwrap_or(settings.rate_num);
+ request.rate_den = spec.rate_den.unwrap_or(settings.rate_den);
+
+ let job = self.exporter.begin()?;
+ let written = export::run(&request, job.cancel_flag(), |progress| {
+ notify(Event::from(progress));
+ })?;
+ Ok(Written {
+ path: written,
+ width,
+ height,
+ })
+ }
+
+ /// Runs every cutout analysis the timeline still needs, one after the
+ /// other, so the export that follows cuts every clip it should.
+ fn find_masks(
+ &self,
+ project: &concat_project::Project,
+ project_dir: &Path,
+ notify: &mut dyn FnMut(Event),
+ ) -> Result<(), String> {
+ for (media_id, request) in Cutouts::requests(project, project_dir) {
+ if Cutouts::outstanding(&request) == 0 {
+ continue;
+ }
+ self.cutouts.analyse(&request, &mut |progress| {
+ let (fetching, fraction) = match progress {
+ cutout::Progress::Fetching { received, total } => {
+ (true, received as f32 / total.max(1) as f32)
+ }
+ cutout::Progress::Analysing(fraction) => (false, fraction),
+ };
+ notify(Event::CutoutProgress {
+ media_id: media_id.clone(),
+ fetching,
+ fraction,
+ });
+ })?;
+ }
+ Ok(())
+ }
+
+ /// The timeline's titles painted for a `width` × `height` frame, as
+ /// the stills that stand in for them.
+ fn title_clips(&self, session: &Session, width: u32, height: u32) -> Vec<ExportClip> {
+ self.titles
+ .clips(session.project(), width, height)
+ .into_iter()
+ .map(|title| title.clip)
+ .collect()
+ }
+
+ /// [`Request::PreviewFrame`]: the paused monitor's true frame, to a
+ /// file.
+ pub fn frame(
+ &mut self,
+ path: &str,
+ time: f64,
+ output: &str,
+ size: Option<(u32, u32)>,
+ ) -> Result<Written, String> {
+ let session = self.session(path)?;
+ let settings = session.settings();
+ let (width, height) = size.unwrap_or((settings.width, settings.height));
+ if width == 0 || height == 0 {
+ return Err("A frame needs a width and a height".to_owned());
+ }
+ let mut clips = session.flattened_clips();
+ clips.extend(self.title_clips(session, width, height));
+ let pixels = self.monitor.frame(
+ Arc::new(clips),
+ &settings,
+ FrameSpec {
+ time,
+ width,
+ height,
+ },
+ )?;
+ write_png(Path::new(output), width, height, &pixels)?;
+ Ok(Written {
+ path: output.to_owned(),
+ width,
+ height,
+ })
+ }
+
+ fn session(&self, path: &str) -> Result<&Session, String> {
+ self.sessions
+ .get(&key_of(path))
+ .ok_or_else(|| not_open(path))
+ }
+
+ fn session_mut(&mut self, path: &str) -> Result<&mut Session, String> {
+ self.sessions
+ .get_mut(&key_of(path))
+ .ok_or_else(|| not_open(path))
+ }
+}
+
+/// [`Request::CatalogueList`]: the built-in packages plus whatever
+/// [`Catalogue::install`] added, in id order.
+fn catalogue(kind: Option<&str>) -> Result<Vec<PackageInfo>, String> {
+ let kind = match kind {
+ None => None,
+ Some("effect") => Some(Kind::Effect),
+ Some("filter") => Some(Kind::Filter),
+ Some("audio") => Some(Kind::Audio),
+ Some("transition") => Some(Kind::Transition),
+ Some("generator") => Some(Kind::Generator),
+ Some(other) => {
+ return Err(format!(
+ "{other:?} is not a kind: effect, filter, audio, transition or generator"
+ ));
+ }
+ };
+ let mut packages: Vec<PackageInfo> = Catalogue::builtin()
+ .packages()
+ .filter(|package| kind.is_none_or(|kind| package.kind() == kind))
+ .map(|package| {
+ let meta = &package.manifest.effect;
+ PackageInfo {
+ id: meta.id.clone(),
+ name: meta.name.clone(),
+ kind: format!("{:?}", meta.kind).to_ascii_lowercase(),
+ category: meta.category.clone(),
+ description: meta.description.clone(),
+ intensity: meta.intensity.clone(),
+ params: package
+ .manifest
+ .params
+ .iter()
+ .map(|param| ParamInfo {
+ key: param.key.clone(),
+ label: param.label.clone(),
+ kind: format!("{:?}", param.kind).to_ascii_lowercase(),
+ min: param.min,
+ max: param.max,
+ default: param.default,
+ step: param.step,
+ unit: param.unit.clone(),
+ animate: param.animate,
+ values: param.values.clone(),
+ labels: param.labels.clone(),
+ })
+ .collect(),
+ }
+ })
+ .collect();
+ packages.sort_by(|left, right| left.id.cmp(&right.id));
+ Ok(packages)
+}
+
+/// The key a project folder is held under: its canonical path where the
+/// folder exists, so `.` and an absolute spelling of it are one session,
+/// and the path as given where it does not yet.
+fn key_of(path: &str) -> String {
+ std::fs::canonicalize(path)
+ .map(|canonical| canonical.to_string_lossy().into_owned())
+ .unwrap_or_else(|_| path.to_owned())
+}
+
+fn not_open(path: &str) -> String {
+ format!("{path} is not open - open it first")
+}
+
+/// Writes RGBA pixels as a PNG, creating the folder above the file.
+fn write_png(output: &Path, width: u32, height: u32, pixels: &[u8]) -> Result<(), String> {
+ if let Some(parent) = output.parent()
+ && !parent.as_os_str().is_empty()
+ {
+ std::fs::create_dir_all(parent)
+ .map_err(|error| format!("could not create {}: {error}", parent.display()))?;
+ }
+ let file = std::fs::File::create(output)
+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;
+ let mut encoder = png::Encoder::new(std::io::BufWriter::new(file), width, height);
+ encoder.set_color(png::ColorType::Rgba);
+ encoder.set_depth(png::BitDepth::Eight);
+ let mut writer = encoder
+ .write_header()
+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;
+ writer
+ .write_image_data(pixels)
+ .map_err(|error| format!("could not write {}: {error}", output.display()))?;
+ writer
+ .finish()
+ .map_err(|error| format!("could not write {}: {error}", output.display()))
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use concat_project::commands::NewMedia;
+ use concat_project::model::MediaKind;
+ use serde_json::json;
+
+ /// An API whose config and data live in a scratch folder that goes
+ /// away with the test.
+ fn api() -> (Api, tempfile::TempDir) {
+ let scratch = tempfile::tempdir().expect("scratch");
+ let dirs = AppDirs {
+ config: scratch.path().join("config"),
+ data: scratch.path().join("data"),
+ };
+ (Api::with_dirs(dirs), scratch)
+ }
+
+ fn ok(response: Response) -> Reply {
+ match response {
+ Response::Result(reply) => reply,
+ Response::Error(error) => panic!("refused: {error}"),
+ }
+ }
+
+ fn err(response: Response) -> String {
+ match response {
+ Response::Result(_) => panic!("worked, unexpectedly"),
+ Response::Error(error) => error,
+ }
+ }
+
+ fn view(reply: Reply) -> EditorView {
+ match reply {
+ Reply::View(view) => *view,
+ _ => panic!("not a view"),
+ }
+ }
+
+ fn quiet() -> impl FnMut(Event) {
+ |_| {}
+ }
+
+ fn still(path: &str) -> NewMedia {
+ NewMedia {
+ path: path.to_owned(),
+ name: "still.png".to_owned(),
+ duration: None,
+ kind: MediaKind::Image,
+ width: Some(640),
+ height: Some(360),
+ frame_rate: None,
+ frame_rate_fraction: None,
+ video_codec: None,
+ audio_codec: None,
+ has_audio: false,
+ }
+ }
+
+ #[test]
+ fn a_request_parses_by_method_and_carries_a_command_verbatim() {
+ let request: Request = serde_json::from_value(json!({
+ "method": "edit.apply",
+ "path": "/p",
+ "command": { "op": "addTextClip", "start": 1.5 }
+ }))
+ .expect("parses");
+ assert_eq!(
+ request,
+ Request::EditApply {
+ path: "/p".to_owned(),
+ command: Box::new(Command::AddTextClip {
+ track_id: None,
+ start: 1.5,
+ style: None,
+ duration: None,
+ offset_y: None,
+ }),
+ }
+ );
+ }
+
+ #[test]
+ fn a_response_is_a_result_or_an_error_object() {
+ let worked = serde_json::to_value(Response::Result(Reply::Done(Done {}))).expect("json");
+ assert_eq!(worked, json!({ "result": {} }));
+ let refused = serde_json::to_value(Response::Error("no".to_owned())).expect("json");
+ assert_eq!(refused, json!({ "error": "no" }));
+ }
+
+ #[test]
+ fn create_edit_save_and_reopen_round_trip() {
+ let (mut api, scratch) = api();
+ let location = scratch.path().join("projects");
+ std::fs::create_dir_all(&location).expect("location");
+ let location = location.to_string_lossy().into_owned();
+
+ let created = view(ok(api.dispatch(
+ Request::ProjectCreate {
+ location: location.clone(),
+ name: "Round trip".to_owned(),
+ video: Some(VideoSettings {
+ width: 1080,
+ height: 1920,
+ rate_num: 60,
+ rate_den: 1,
+ }),
+ },
+ &mut quiet(),
+ )));
+ assert_eq!(created.settings.width, 1080);
+ assert_eq!(created.settings.height, 1920);
+ assert_eq!(created.settings.rate_num, 60);
+ let path = format!("{location}/Round trip");
+ assert!(projects::is_project(Path::new(&path)));
+
+ let added = view(ok(api.dispatch(
+ Request::EditApply {
+ path: path.clone(),
+ command: Box::new(Command::AddMedia {
+ item: still("/nowhere/still.png"),
+ }),
+ },
+ &mut quiet(),
+ )));
+ let media_id = added.created_id.expect("minted");
+ let placed = view(ok(api.dispatch(
+ Request::EditApply {
+ path: path.clone(),
+ command: Box::new(Command::AddClipAtFirstFree {
+ media_id,
+ start: 2.0,
+ }),
+ },
+ &mut quiet(),
+ )));
+ assert!(placed.can_undo);
+ assert_eq!(placed.project.active().clips.len(), 1);
+
+ ok(api.dispatch(
+ Request::ProjectClose {
+ path: path.clone(),
+ save: true,
+ },
+ &mut quiet(),
+ ));
+ assert_eq!(
+ err(api.dispatch(Request::ProjectGet { path: path.clone() }, &mut quiet())),
+ not_open(&path)
+ );
+
+ let reopened = view(ok(
+ api.dispatch(Request::ProjectOpen { path: path.clone() }, &mut quiet())
+ ));
+ assert_eq!(reopened.project.active().clips.len(), 1);
+ assert_eq!(reopened.project.active().clips[0].start, 2.0);
+ assert!(!reopened.can_undo, "history does not survive a save");
+
+ let recents = match ok(api.dispatch(Request::ProjectList, &mut quiet())) {
+ Reply::Projects(list) => list,
+ _ => panic!("not a list"),
+ };
+ assert_eq!(recents.len(), 1);
+ assert_eq!(recents[0].name, "Round trip");
+ }
+
+ #[test]
+ fn a_refusal_is_the_command_layers_sentence() {
+ let (mut api, scratch) = api();
+ let location = scratch.path().to_string_lossy().into_owned();
+ let created = view(ok(api.dispatch(
+ Request::ProjectCreate {
+ location: location.clone(),
+ name: "Refused".to_owned(),
+ video: None,
+ },
+ &mut quiet(),
+ )));
+ let track_id = created.project.active().tracks[0].id.clone();
+ let refused = err(api.dispatch(
+ Request::EditApply {
+ path: format!("{location}/Refused"),
+ command: Box::new(Command::AddClip {
+ media_id: "m999".to_owned(),
+ track_id,
+ start: 0.0,
+ }),
+ },
+ &mut quiet(),
+ ));
+ assert_eq!(refused, "That media is no longer in the bin.");
+ }
+
+ #[test]
+ fn creating_over_a_project_is_refused() {
+ let (mut api, scratch) = api();
+ let location = scratch.path().to_string_lossy().into_owned();
+ let create = || Request::ProjectCreate {
+ location: location.clone(),
+ name: "Twice".to_owned(),
+ video: None,
+ };
+ ok(api.dispatch(create(), &mut quiet()));
+ assert!(err(api.dispatch(create(), &mut quiet())).contains("already exists"));
+ }
+
+ #[test]
+ fn the_catalogue_lists_packages_with_their_knobs() {
+ let packages = catalogue(Some("filter")).expect("kind");
+ assert!(!packages.is_empty());
+ assert!(packages.iter().all(|package| package.kind == "filter"));
+ assert!(packages.windows(2).all(|pair| pair[0].id < pair[1].id));
+ assert!(catalogue(Some("look")).is_err());
+ let all = catalogue(None).expect("all");
+ assert!(all.len() > packages.len());
+ assert!(all.iter().any(|package| !package.params.is_empty()));
+ }
+
+ #[test]
+ fn a_method_on_a_closed_project_says_so() {
+ let (mut api, _scratch) = api();
+ assert_eq!(
+ err(api.dispatch(
+ Request::EditUndo {
+ path: "/never".to_owned()
+ },
+ &mut quiet()
+ )),
+ not_open("/never")
+ );
+ }
+
+ #[test]
+ fn undo_and_redo_step_the_history() {
+ let (mut api, scratch) = api();
+ let location = scratch.path().to_string_lossy().into_owned();
+ ok(api.dispatch(
+ Request::ProjectCreate {
+ location: location.clone(),
+ name: "History".to_owned(),
+ video: None,
+ },
+ &mut quiet(),
+ ));
+ let path = format!("{location}/History");
+ ok(api.dispatch(
+ Request::EditApply {
+ path: path.clone(),
+ command: Box::new(Command::AddTextClip {
+ track_id: None,
+ start: 0.0,
+ style: None,
+ duration: Some(3.0),
+ offset_y: None,
+ }),
+ },
+ &mut quiet(),
+ ));
+ let undone = view(ok(
+ api.dispatch(Request::EditUndo { path: path.clone() }, &mut quiet())
+ ));
+ assert!(undone.project.active().clips.is_empty());
+ assert!(undone.can_redo);
+ let redone = view(ok(api.dispatch(Request::EditRedo { path }, &mut quiet())));
+ assert_eq!(redone.project.active().clips.len(), 1);
+ }
+
+ #[test]
+ fn the_document_is_what_a_save_writes() {
+ let (mut api, scratch) = api();
+ let location = scratch.path().to_string_lossy().into_owned();
+ ok(api.dispatch(
+ Request::ProjectCreate {
+ location: location.clone(),
+ name: "Doc".to_owned(),
+ video: None,
+ },
+ &mut quiet(),
+ ));
+ let path = format!("{location}/Doc");
+ let document = match ok(api.dispatch(
+ Request::ProjectDocument { path: path.clone() },
+ &mut quiet(),
+ )) {
+ Reply::Document(document) => document,
+ _ => panic!("not a document"),
+ };
+ ok(api.dispatch(
+ Request::ProjectSave {
+ path: path.clone(),
+ name: None,
+ },
+ &mut quiet(),
+ ));
+ assert_eq!(projects::read_document(&path).expect("saved"), document);
+ }
+}
diff --git a/engine/crates/concat-api/src/message.rs b/engine/crates/concat-api/src/message.rs
new file mode 100644
index 00000000..b0617635
--- /dev/null
+++ b/engine/crates/concat-api/src/message.rs
@@ -0,0 +1,436 @@
+// SPDX-License-Identifier: AGPL-3.0-or-later
+// SPDX-FileCopyrightText: 2026 Jareer and Concat contributors
+
+//! What crosses the API boundary: a [`Request`] in, a [`Response`] out,
+//! and [`Event`]s along the way.
+//!
+//! All three are serde types with a fixed JSON shape, and that shape is the
+//! contract: a transport is free to carry it over stdin, a socket or a
+//! function call, but never to reinterpret it. Requests are tagged by
+//! `method`, events by `event`, and a response is `{"result": ...}` or
+//! `{"error": "..."}`. Fields are camelCase, matching the document and the
+//! command layer, so a caller learns one spelling.
+//!
+//! The edit vocabulary is not redefined here. [`Request::EditApply`] carries
+//! a `concat_project` [`Command`] as it is, so every operation the window
+//! can perform is one an API caller can perform, with the same clamps and
+//! the same refusals, and a new command needs nothing added here.
+
+use concat_host::export::Progress;
+use concat_host::media::MediaSummary;
+use concat_host::session::EditorView;
+use concat_host::templates::TemplateInfo;
+use concat_host::{AppDirs, ProjectInfo};
+use concat_project::Command;
+use concat_project::model::VideoSettings;
+use serde::{Deserialize, Serialize};
+use serde_json::Value;
+
+/// The version of this contract. Bumped when a method's shape changes in a
+/// way a caller written against the previous one would misread; adding a
+/// method or an optional field does not bump it.
+pub const API_VERSION: &str = "0.1";
+
+/// What a caller asks for. One variant per method, named `area.verb`.
+///
+/// Every method that touches a project names it by its folder, the same
+/// path [`Request::ProjectOpen`] took; the API keeps one session per folder
+/// and a method on a folder that is not open is an error, not a silent
+/// open, so a caller always knows what it is editing.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(tag = "method", rename_all_fields = "camelCase")]
+pub enum Request {
+ /// The API's version and the build behind it.
+ #[serde(rename = "version")]
+ Version,
+
+ /// Creates a project folder under `location`, named `name`, and opens
+ /// it. Refuses a folder that already holds a project.
+ #[serde(rename = "project.create")]
+ ProjectCreate {
+ /// The directory the project folder is made in.
+ location: String,
+ /// The project's name; the folder is named after it.
+ name: String,
+ /// Frame and rate; 1080p at 30 when absent.
+ #[serde(default)]
+ video: Option<VideoSettings>,
+ },
+ /// Opens a project folder. Opening one already open returns its state
+ /// as it stands, edits and all.
+ #[serde(rename = "project.open")]
+ ProjectOpen {
+ /// The project folder.
+ path: String,
+ },
+ /// Closes an open project, saving first when asked. Unsaved edits are
+ /// dropped otherwise.
+ #[serde(rename = "project.close")]
+ ProjectClose {
+ /// The project folder.
+ path: String,
+ /// Whether to write the document before closing.
+ #[serde(default)]
+ save: bool,
+ },
+ /// The projects this machine opened most recently, newest first.
+ #[serde(rename = "project.list")]
+ ProjectList,
+ /// The state of an open project.
+ #[serde(rename = "project.get")]
+ ProjectGet {
+ /// The project folder.
+ path: String,
+ },
+ /// The document exactly as a save would write it.
+ #[serde(rename = "project.document")]
+ ProjectDocument {
+ /// The project folder.
+ path: String,
+ },
+ /// Writes the document to the project folder.
+ #[serde(rename = "project.save")]
+ ProjectSave {
+ /// The project folder.
+ path: String,
+ /// A new name for the project, when renaming.
+ #[serde(default)]
+ name: Option<String>,
+ },
+ /// Sets the active timeline's frame and rate, as an undoable edit.
+ #[serde(rename = "project.setVideo")]
+ ProjectSetVideo {
+ /// The project folder.
+ path: String,
+ /// The frame and rate.
+ video: VideoSettings,
+ },
+
+ /// Applies one edit command. The command's own notes say what each
+ /// does; a refusal comes back as the sentence the window would show.
+ #[serde(rename = "edit.apply")]
+ EditApply {
+ /// The project folder.
+ path: String,
+ /// The edit. Boxed because a command is the largest thing a
+ /// request carries; the JSON is the command as it is.
+ command: Box<Command>,
+ },
+ /// Steps the history back one edit.
+ #[serde(rename = "edit.undo")]
+ EditUndo {
+ /// The project folder.
+ path: String,
+ },
+ /// Steps the history forward one edit.
+ #[serde(rename = "edit.redo")]
+ EditRedo {
+ /// The project folder.
+ path: String,
+ },
+
+ /// Reports what is inside a media file without touching any project.
+ #[serde(rename = "media.probe")]
+ MediaProbe {
+ /// The file.
+ path: String,
+ },
+ /// Probes a file and adds it to a project's bin: what dropping a file
+ /// on the window does. A path already in the bin is a no-op.
+ #[serde(rename = "media.import")]
+ MediaImport {
+ /// The project folder.
+ path: String,
+ /// The file to import.
+ file: String,
+ },
+
+ /// Every effect package the build knows, with its parameters, so a
+ /// caller can build a valid chain without reading a manifest.
+ #[serde(rename = "catalogue.list")]
+ CatalogueList {
+ /// Restrict to one kind: "effect", "filter", "audio", "transition"
+ /// or "generator".
+ #[serde(default)]
+ kind: Option<String>,
+ },
+
+ /// The template library.
+ #[serde(rename = "template.list")]
+ TemplateList,
+ /// Makes a project from a template with every slot filled, and opens
+ /// it. Every slot must be filled; a set that leaves one empty makes
+ /// nothing.
+ #[serde(rename = "template.instantiate")]
+ TemplateInstantiate {
+ /// The template bundle folder.
+ template: String,
+ /// The directory the project folder is made in.
+ location: String,
+ /// The project's name.
+ name: String,
+ /// The file for each slot.
+ fills: Vec<Fill>,
+ },
+ /// Packs an open project into a new template bundle.
+ #[serde(rename = "template.save")]
+ TemplateSave {
+ /// The project folder.
+ path: String,
+ /// The template's name.
+ name: String,
+ },
+
+ /// Renders an open project's active timeline to a file, exactly as
+ /// the window's Export does: titles painted, cutouts analysed, then the
+ /// frame loop and the mix. Blocks until the file is written, reporting
+ /// through [`Event`]s.
+ #[serde(rename = "export.run")]
+ ExportRun {
+ /// The project folder.
+ path: String,
+ /// How to render it.
+ #[serde(flatten)]
+ spec: ExportSpec,
+ },
+ /// Stops the export that is running, if one is. Meaningful only from a
+ /// transport that can speak while a dispatch blocks.
+ #[serde(rename = "export.cancel")]
+ ExportCancel,
+
+ /// Composites the true frame at one instant and writes it as a PNG.
+ #[serde(rename = "preview.frame")]
+ PreviewFrame {
+ /// The project folder.
+ path: String,
+ /// The timeline instant, in seconds.
+ time: f64,
+ /// The file to write.
+ output: String,
+ /// Frame width; the timeline's when absent.
+ #[serde(default)]
+ width: Option<u32>,
+ /// Frame height; the timeline's when absent.
+ #[serde(default)]
+ height: Option<u32>,
+ },
+}
+
+/// One template slot and the file that takes it.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct Fill {
+ /// The placeholder's media id in the template.
+ pub media_id: String,
+ /// The file to put there; probed on the way in.
+ pub file: String,
+}
+
+/// How an export is rendered. Everything but the output is optional and
+/// defaults to what the window's sheet defaults to.
+#[derive(Clone, PartialEq, Debug, Default, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct ExportSpec {
+ /// The file to write.
+ pub output: String,
+ /// Constant rate factor; lower is better and bigger. 20 when absent.
+ #[serde(default)]
+ pub crf: Option<u8>,
+ /// The x264 preset name. "medium" when absent.
+ #[serde(default)]
+ pub preset: Option<String>,
+ /// Output width; the timeline's when absent.
+ #[serde(default)]
+ pub width: Option<u32>,
+ /// Output height; the timeline's when absent.
+ #[serde(default)]
+ pub height: Option<u32>,
+ /// Frame rate numerator; the timeline's when absent.
+ #[serde(default)]
+ pub rate_num: Option<i64>,
+ /// Frame rate denominator; the timeline's when absent.
+ #[serde(default)]
+ pub rate_den: Option<i64>,
+}
+
+/// What a request hands back. Serialised as the payload alone: the variant
+/// is implied by the method, so a caller that sent `project.get` reads an
+/// editor view and nothing wraps it.
+#[derive(Clone, Serialize)]
+#[serde(untagged)]
+pub enum Reply {
+ /// [`Request::Version`].
+ Version(VersionInfo),
+ /// The project after a change, or as it stands.
+ View(Box<EditorView>),
+ /// [`Request::ProjectList`].
+ Projects(Vec<ProjectInfo>),
+ /// [`Request::ProjectDocument`].
+ Document(Value),
+ /// [`Request::MediaProbe`].
+ Media(MediaSummary),
+ /// [`Request::CatalogueList`].
+ Packages(Vec<PackageInfo>),
+ /// [`Request::TemplateList`].
+ Templates(Vec<TemplateInfo>),
+ /// [`Request::TemplateSave`].
+ Template(TemplateInfo),
+ /// [`Request::ExportRun`] and [`Request::PreviewFrame`]: the file
+ /// written.
+ Written(Written),
+ /// Methods with nothing to say beyond having worked.
+ Done(Done),
+}
+
+/// The build and the contract it speaks.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct VersionInfo {
+ /// [`API_VERSION`].
+ pub api_version: String,
+ /// The Concat build.
+ pub concat: String,
+ /// Where this machine keeps recents, templates and models.
+ pub dirs: Dirs,
+}
+
+/// The app's directories, as paths.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct Dirs {
+ /// Small state: recents, settings, the template library.
+ pub config: String,
+ /// Large state: downloaded models, painted titles.
+ pub data: String,
+}
+
+impl From<&AppDirs> for Dirs {
+ fn from(dirs: &AppDirs) -> Self {
+ Dirs {
+ config: dirs.config.to_string_lossy().into_owned(),
+ data: dirs.data.to_string_lossy().into_owned(),
+ }
+ }
+}
+
+/// A file the API wrote.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct Written {
+ /// The file's path.
+ pub path: String,
+ /// Its picture size.
+ pub width: u32,
+ /// Its picture size.
+ pub height: u32,
+}
+
+/// The empty reply, an object so every reply is one.
+#[derive(Clone, Copy, PartialEq, Debug, Default, Serialize, Deserialize)]
+pub struct Done {}
+
+/// One effect package, as a caller building a chain needs it.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct PackageInfo {
+ /// The id a clip's chain stores, `author.name`.
+ pub id: String,
+ /// What the catalogue card says.
+ pub name: String,
+ /// "effect", "filter", "audio", "transition" or "generator".
+ pub kind: String,
+ /// The shelf the card sits on.
+ pub category: String,
+ /// A sentence about it.
+ pub description: String,
+ /// The parameter the simple view shows as the one slider, if any.
+ pub intensity: Option<String>,
+ /// The knobs, in the order the inspector shows them.
+ pub params: Vec<ParamInfo>,
+}
+
+/// One parameter of a package.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(rename_all = "camelCase")]
+pub struct ParamInfo {
+ /// The key a chain entry's `params` stores it under.
+ pub key: String,
+ /// What the control is labelled.
+ pub label: String,
+ /// "float", "int", "bool", "enum", "color" or "point".
+ #[serde(rename = "type")]
+ pub kind: String,
+ /// Lowest value.
+ pub min: f64,
+ /// Highest value.
+ pub max: f64,
+ /// The value an untouched control means.
+ pub default: f64,
+ /// Slider increment; 0 means continuous.
+ pub step: f64,
+ /// Displayed after the number.
+ pub unit: String,
+ /// Whether the control can carry keyframes.
+ pub animate: bool,
+ /// For "enum": the values the document may hold.
+ pub values: Vec<f64>,
+ /// For "enum": what each value is called, in `values` order.
+ pub labels: Vec<String>,
+}
+
+/// A request's outcome. `{"result": ...}` when it worked, `{"error":
+/// "..."}` with the sentence a person would be shown when it did not.
+#[derive(Clone, Serialize)]
+#[serde(rename_all = "camelCase")]
+pub enum Response {
+ /// The reply.
+ Result(Reply),
+ /// Why there is none.
+ Error(String),
+}
+
+impl From<Result<Reply, String>> for Response {
+ fn from(result: Result<Reply, String>) -> Self {
+ match result {
+ Ok(reply) => Response::Result(reply),
+ Err(error) => Response::Error(error),
+ }
+ }
+}
+
+/// Something that happened while a request ran. A transport forwards
+/// these as they come; a caller that does not care ignores them.
+#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
+#[serde(tag = "event", rename_all_fields = "camelCase")]
+pub enum Event {
+ /// The export moved on.
+ #[serde(rename = "export.progress")]
+ ExportProgress {
+ /// Frames done.
+ frame: i64,
+ /// Frames in total.
+ total: i64,
+ /// "rendering", "mixing audio" or "muxing".
+ stage: String,
+ },
+ /// A cutout's masks are being found ahead of an export.
+ #[serde(rename = "cutout.progress")]
+ CutoutProgress {
+ /// The media being analysed.
+ media_id: String,
+ /// True while a model downloads, false while it runs.
+ fetching: bool,
+ /// How far along, `0..=1`.
+ fraction: f32,
+ },
+}
+
+impl From<Progress> for Event {
+ fn from(progress: Progress) -> Self {
+ Event::ExportProgress {
+ frame: progress.frame,
+ total: progress.total,
+ stage: progress.stage.to_owned(),
+ }
+ }
+}
diff --git a/engine/crates/concat-cli/Cargo.toml b/engine/crates/concat-cli/Cargo.toml
index a5c92fa3..20adf6ac 100644
--- a/engine/crates/concat-cli/Cargo.toml
+++ b/engine/crates/concat-cli/Cargo.toml
@@ -15,6 +15,9 @@ path = "src/main.rs"
concat-core.workspace = true
concat-media.workspace = true
concat-render.workspace = true
+# The `api` command: the Concat API over stdin and stdout.
+concat-api.workspace = true
+serde_json.workspace = true
clap.workspace = true
[lints]
diff --git a/engine/crates/concat-cli/src/main.rs b/engine/crates/concat-cli/src/main.rs
index cbbfeaaa..7f773676 100644
--- a/engine/crates/concat-cli/src/main.rs
+++ b/engine/crates/concat-cli/src/main.rs
@@ -5,9 +5,12 @@
//!
//! This exists so the engine can be exercised end to end without a UI. The
//! `render` command is the vertical slice: probe, build a timeline, plan every
-//! frame, decode, composite, encode.
+//! frame, decode, composite, encode. The `api` command is the Concat API's
+//! first transport: JSON requests in, JSON responses and events out, one per
+//! line, so a script in any language edits and exports a project.
use std::error::Error;
+use std::io::{BufRead, Write};
use std::path::PathBuf;
use clap::{Parser, Subcommand};
@@ -44,6 +47,13 @@ enum Command {
#[arg(long, default_value_t = 15)]
fade: u64,
},
+
+ /// Speak the Concat API: a JSON request per line on stdin, a response per
+ /// line on stdout, with events in between as they happen.
+ Api {
+ /// One request to run instead of reading stdin.
+ request: Option<String>,
+ },
}
fn main() -> Result<(), Box<dyn Error>> {
@@ -55,6 +65,48 @@ fn main() -> Result<(), Box<dyn Error>> {
frames,
fade,
} => render(&input, &output, frames, fade),
+ Command::Api { request } => api(request),
+ }
+}
+
+/// The stdin transport. Every line in is one request; every line out is
+/// one JSON object, either an event or the response, flushed as written so
+/// a caller reading a pipe sees progress as it happens. A line that is not
+/// a request gets an error response and the loop goes on.
+fn api(single: Option<String>) -> Result<(), Box<dyn Error>> {
+ let mut api = concat_api::Api::new()?;
+ let stdout = std::io::stdout();
+
+ let mut serve = |line: &str| -> Result<(), Box<dyn Error>> {
+ let mut out = stdout.lock();
+ let response = match serde_json::from_str::<concat_api::Request>(line) {
+ Ok(request) => api.dispatch(request, &mut |event| {
+ // An event that cannot be written is a caller that went
+ // away; the response's write will say so.
+ let _ = serde_json::to_writer(&mut out, &event)
+ .and_then(|()| writeln!(out).map_err(serde_json::Error::io));
+ let _ = out.flush();
+ }),
+ Err(error) => concat_api::Response::Error(format!("not a request: {error}")),
+ };
+ serde_json::to_writer(&mut out, &response)?;
+ writeln!(out)?;
+ out.flush()?;
+ Ok(())
+ };
+
+ match single {
+ Some(line) => serve(&line),
+ None => {
+ for line in std::io::stdin().lock().lines() {
+ let line = line?;
+ if line.trim().is_empty() {
+ continue;
+ }
+ serve(&line)?;
+ }
+ Ok(())
+ }
}
}
diff --git a/engine/crates/concat-host/Cargo.toml b/engine/crates/concat-host/Cargo.toml
index 5df82db3..527043b1 100644
--- a/engine/crates/concat-host/Cargo.toml
+++ b/engine/crates/concat-host/Cargo.toml
@@ -9,7 +9,7 @@ description = "What the editor window needs that is not the edit itself: session
[features]
# The wgpu compositor for previews and exports, off by default for the same
# reason as everywhere else in the engine; the window turns it on.
-gpu = ["concat-export/gpu", "dep:concat-render", "concat-render/gpu"]
+gpu = ["concat-export/gpu", "concat-render/gpu"]
[dependencies]
concat-core.workspace = true
@@ -22,7 +22,9 @@ concat-text.workspace = true
# runs them, and ureq because the host is what fetches them.
concat-vision = { workspace = true, features = ["infer"] }
ureq = "2"
-concat-render = { workspace = true, optional = true }
+# The CPU compositor draws the monitor's frame; the wgpu one is behind
+# `gpu`, so a headless host pays for no GPU tree.
+concat-render.workspace = true
serde = { version = "1", features = ["derive"] }
serde_json.workspace = true
diff --git a/engine/crates/concat-host/src/cutout.rs b/engine/crates/concat-host/src/cutout.rs
index 557ba5df..df5098b9 100644
--- a/engine/crates/concat-host/src/cutout.rs
+++ b/engine/crates/concat-host/src/cutout.rs
@@ -29,7 +29,7 @@ use std::sync::{Arc, Mutex, OnceLock};
use concat_core::time::{FrameRate, Rational};
use concat_media::{DecodeOptions, Decoder, FrameSource};
-use concat_project::model::Subject;
+use concat_project::model::{MediaKind, Project, Subject};
use concat_vision::segment::Kind;
use concat_vision::{MASK_RATE, MaskStore, ModelId, Segmenter, mask_dir, models};
@@ -152,6 +152,52 @@ impl Cutouts {
}
}
+ /// What the active timeline's cutouts need analysed: one request per
+ /// media and subject, since two clips of one file that keep different
+ /// things need different masks, each covering the stretches of source
+ /// its clips show. Paired with the media's id, for a caller that keys
+ /// its bookkeeping by it. The window asks after every change and the
+ /// API asks before an export; both get the same list.
+ pub fn requests(project: &Project, project_dir: &Path) -> Vec<(String, AnalyseRequest)> {
+ let mut wanted: Vec<(String, AnalyseRequest)> = Vec::new();
+ for clip in &project.active().clips {
+ let Some(cutout) = clip.cutout.as_ref() else {
+ continue;
+ };
+ if !clip.kind.is_visual() {
+ continue;
+ }
+ let Some(media) = project.media_by_id(&clip.media_id) else {
+ continue;
+ };
+ // The source the clip shows: its in-point, for as long as it
+ // runs at its speed. A curve's mean is its speed, so this
+ // covers a curved clip too.
+ let range = (
+ clip.source_start,
+ clip.source_start + clip.duration * clip.speed.max(0.0625),
+ );
+ match wanted
+ .iter_mut()
+ .find(|(id, request)| *id == media.id && request.subject == cutout.subject)
+ {
+ Some((_, request)) => request.ranges.push(range),
+ None => wanted.push((
+ media.id.clone(),
+ AnalyseRequest {
+ project: project_dir.to_path_buf(),
+ media_path: media.path.clone(),
+ media_size: (media.width.unwrap_or(0), media.height.unwrap_or(0)),
+ still: media.kind == MediaKind::Image,
+ subject: cutout.subject,
+ ranges: vec![range],
+ },
+ )),
+ }
+ }
+ wanted
+ }
+
/// How many instants `request` still needs, without doing anything.
/// A store made by a model the subject would not pick now needs all
/// of them again.
diff --git a/engine/crates/concat-host/src/preview.rs b/engine/crates/concat-host/src/preview.rs
index c85f9f81..d73bb160 100644
--- a/engine/crates/concat-host/src/preview.rs
+++ b/engine/crates/concat-host/src/preview.rs
@@ -14,9 +14,7 @@
//! pictures go up once, the composite happens where it is shown, and no
//! pixel comes back down.
-use std::sync::Arc;
-#[cfg(feature = "gpu")]
-use std::sync::Mutex;
+use std::sync::{Arc, Mutex};
use concat_export::ExportClip;
use concat_project::DocumentSettings;
diff --git a/engine/crates/concat-project/src/model.rs b/engine/crates/concat-project/src/model.rs
index c39dce83..9f7abc4b 100644
--- a/engine/crates/concat-project/src/model.rs
+++ b/engine/crates/concat-project/src/model.rs
@@ -729,9 +729,12 @@ pub enum TextAlign {
}
/// A title's styling. Sizes are fractions of the frame, so a title composed
-/// against 1080p lands correctly exported at 4K.
+/// against 1080p lands correctly exported at 4K. A style arriving in a
+/// command needs only the fields it changes; the rest are the default's, so
+/// a caller can say `{ "content": "Hello" }` and get a title that looks like
+/// one the window would place.
#[derive(Clone, PartialEq, Debug, Serialize, Deserialize)]
-#[serde(rename_all = "camelCase")]
+#[serde(rename_all = "camelCase", default)]
pub struct TextStyle {
/// The words themselves, newlines included. The clip's display name is
/// snapshotted from the first non-empty line.
diff --git a/engine/crates/concat/src/studio.rs b/engine/crates/concat/src/studio.rs
index 898f0ed8..aca5c97a 100644
--- a/engine/crates/concat/src/studio.rs
+++ b/engine/crates/concat/src/studio.rs
@@ -3948,42 +3948,13 @@ impl Studio {
return;
};
let project = std::path::PathBuf::from(session.path());
- let mut wanted: Vec<(String, AnalyseRequest)> = Vec::new();
- for clip in &self.timeline().clips {
- let Some(cutout) = clip.cutout.as_ref() else {
- continue;
- };
- if !clip.kind.is_visual() {
- continue;
- }
- let Some(media) = self.project().media_by_id(&clip.media_id) else {
- continue;
- };
- // One analysis per media and subject: two clips of one file
- // that keep different things need different masks.
- let key = Self::analysis_key(&media.id, cutout.subject);
- // The source the clip shows: its in-point, for as long as it
- // runs at its speed. A curve's mean is its speed, so this
- // covers a curved clip too.
- let range = (
- clip.source_start,
- clip.source_start + clip.duration * clip.speed.max(0.0625),
- );
- match wanted.iter_mut().find(|(id, _)| *id == key) {
- Some((_, request)) => request.ranges.push(range),
- None => wanted.push((
- key,
- AnalyseRequest {
- project: project.clone(),
- media_path: media.path.clone(),
- media_size: (media.width.unwrap_or(0), media.height.unwrap_or(0)),
- still: media.kind == model::MediaKind::Image,
- subject: cutout.subject,
- ranges: vec![range],
- },
- )),
- }
- }
+ // One analysis per media and subject, keyed the way the job map is.
+ let wanted: Vec<(String, AnalyseRequest)> = Cutouts::requests(self.project(), &project)
+ .into_iter()
+ .map(|(media_id, request)| {
+ (Self::analysis_key(&media_id, request.subject), request)
+ })
+ .collect();
let Some((id, request)) = wanted
.into_iter()
.find(|(_, request)| Cutouts::outstanding(request) > 0)