The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.
Rust ffmpegfree-video-editormodel-context-protocolnon-linear-editoroffline-firstopen-source-video-editorself-hostedvideo-editingvideo-editing-automationvideo-editing-softwarevideo-editing-toolsvideo-editorvoice-cloningyoutube-automationyoutube-shorts
docs: EFFECTS.md, how to write and share a package
Jareer18288f69489f
diff --git a/EFFECTS.md b/EFFECTS.mdnew file mode 100644index 00000000..7c3819b7--- /dev/null+++ b/EFFECTS.md@@ -0,0 +1,295 @@+# Writing effects for Concat++Every effect, filter, transition and audio effect in Concat is a **package**:+a folder with a short manifest and, beside it, a filter chain or a shader.+The ones that ship with the app are packages too, and nothing in the app's+code names any of them. A package you write goes in a folder on your+machine and shows up on the same shelves as the built-ins, and a package you+share is that folder, zipped.++## Where packages live++Concat reads every folder under its effects directory:++| Platform | Folder |+|---|---|+| macOS | `~/Library/Application Support/concat/effects` |+| Windows | `%APPDATA%\concat\effects` |+| Linux | `~/.config/concat/effects` (or `$XDG_CONFIG_HOME/concat/effects`) |+| Portable build | `portable/effects` beside the executable |++The app watches that folder while it runs. Add a package, edit its files,+or delete it, and the shelves follow within a few seconds. The **Reload+custom effects** button on the Effects and Filters pages does the same on+demand, and its notice names the folder, so you never have to remember the+path. A package that will not load is skipped, the reason appears in the+window, and every reason is in the log (Settings › About opens it).++## The smallest package++A package is a folder named after its id, holding `effect.toml`:++```+effects/+└── ada.warm-shadows/+ └── effect.toml+```++```toml+format = 1++[effect]+id = "ada.warm-shadows"+name = "Warm Shadows"+kind = "filter"+category = "Colour"+description = "A little amber in the shadows, the highlights left alone."++[[param]]+key = "amount"+label = "Amount"+min = 0+max = 100+default = 50+unit = "%"++[ffmpeg]+let = ["a = amount / 100 * 0.3"]+chain = "colorbalance=rs={fixed(a, 3)}:gs={fixed(a * 0.4, 3)}"+```++Save it, and Warm Shadows is on the Filters page under Colour, with one+slider. That is a whole package.++## The manifest++`effect.toml` is TOML. `format = 1` at the top says which package format+the file is written for; it may be left out, which means 1. A future Concat+that changes the format will bump the number, and a package written for a+newer one is refused with a message saying so rather than failing somewhere+confusing.++### `[effect]`++| Key | Meaning |+|---|---|+| `id` | `author.name`, lower-case letters, digits and hyphens. Stored in project files forever, so pick it once. The folder must be named after it. `concat.` is the built-ins' author and is refused for your own. |+| `name` | What the card says. |+| `kind` | `effect`, `filter`, `audio`, `transition` or `generator`. See below. |+| `category` | The shelf the card sits on, free text. `Imported` is where LUT imports go. |+| `order` | Sort key within the category; ties break on name. |+| `version` | Your own number, bumped when the output changes. Default 1. |+| `intensity` | For an effect: the parameter the simple view shows as its one slider. |+| `aliases` | Earlier ids this package answers to, so old projects keep rendering. |+| `description` | One sentence for the tooltip. |++### Kinds++- **filter**: a colour look, picture in, picture out. Every filter gets an+ intensity control for free. Below full intensity the host mixes your result+ with the untouched picture, so your chain or shader always renders the look+ at full strength and never has to mix itself.+- **effect**: picture in, picture out, no automatic mixing. Blur,+ distortion, texture, motion.+- **audio**: sound in, sound out. An FFmpeg chain of audio filters.+- **transition**: two pictures and a progress in, one picture out. A shader+ under `[transition]`, never a chain.+- **generator**: no picture in, a picture out. A shader.++### `[[param]]`++One table per control, in the order the inspector shows them.++| Key | Meaning |+|---|---|+| `key` | The name the backend reads and the document stores. Lower-case letters, digits and underscores. `index` and `intensity` are reserved. |+| `label` | What the control is called. |+| `type` | `float` (default), `int`, `bool`, `enum`, `color`, `point`. |+| `min`, `max`, `default`, `step` | The range, the untouched value, and the slider increment. `max` defaults to 1, `step` 0 means continuous. |+| `unit` | Shown after the number: `%`, `dB`, `K`, `s`. |+| `group` | A subhead; consecutive params with the same group share one. |+| `animate` | Whether the control can carry keyframes. |+| `values`, `labels` | For `enum`: what the document may hold, and what each is called. |++Every declared parameter must be read by the backend, and a backend may not+read one the manifest does not declare. Both are caught at load.++## The FFmpeg backend++```toml+[ffmpeg]+let = ["k = clamp(strength / 100, 0, 1)", "r = round(k * 20)"]+chain = "gblur=sigma={fixed(k * 4, 2)},unsharp=5:5:{fixed(r / 10, 2)}"+```++`chain` is FFmpeg filter syntax with `{expression}` slots. `let` names+intermediate values, evaluated in order; each may read the parameters and+the names before it. `{{` and `}}` are literal braces.++An expression is a number, a parameter, arithmetic with `+ - * /`, a+comparison with `< > <= >= == !=` (1 or 0), and these functions:++| Function | Result |+|---|---|+| `round(x)`, `floor(x)`, `ceil(x)` | A whole number, printed without a decimal point. |+| `abs(x)`, `sqrt(x)`, `pow(x, y)` | As you would expect. |+| `min(a, b)`, `max(a, b)`, `clamp(x, lo, hi)` | Bounds. |+| `if(cond, a, b)` | `a` when `cond` is not 0, else `b`. |+| `fixed(x, n)` | Text with exactly `n` decimals, which is how a filtergraph wants most numbers. |++Two names are always there: `index`, the fragment's position in the clip's+chain, which any filtergraph labels you write must embed+(`split[a{index}][b{index}]`); and `lut`, the quoted path of the table a+`[lut]` section names.++**Which filters a chain may use.** A chain runs inside the app's own+process, and FFmpeg has filters that open files, load plugins and take+commands. A package is a folder anyone can share, so a chain is held to a+list of filters that read the frame and write the frame and nothing else.+The list is in+[`src/crates/concat-effects/src/filters.rs`](src/crates/concat-effects/src/filters.rs):+the colour, blur, distortion, keying and stacking filters, `split`, `blend`+and `overlay` for graphs, `lut3d` and `lut1d` for tables, and the tone,+dynamics, delay and pitch filters for sound. A chain naming anything else is+refused at load, and a filter's name has to be spelt out, never come from a+slot. If a pure pixel or sound filter you need is missing, add it to the+list in a pull request.++## The WGSL backend++```toml+[wgsl]+entry = "effect.wgsl"+```++`effect.wgsl` beside the manifest declares a `Params` struct with one field+per parameter and the entry the host calls:++```wgsl+struct Params { amount: f32 }++fn effect(uv: vec2<f32>) -> vec4<f32> {+ let c = sample(uv);+ let warm = split_tone(c.rgb, vec3<f32>(1.0, 0.75, 0.5), vec3<f32>(1.0), params.amount / 100.0);+ return vec4<f32>(warm, c.a);+}+```++A `float`, `int`, `bool` or `enum` parameter is an `f32` field; a `point` is+a `vec2<f32>`; a `color` is a `vec4<f32>`. Every declared parameter must be+a field; a field the manifest does not declare stays zero.++The host wraps your file in its half of the contract. You get:++- `sample(uv)`: the layer's colour at `uv`, straight alpha.+- `texel()`: one pixel as a fraction of the layer.+- `luma(rgb)`: Rec. 709 luminance.+- `lut(rgb)`: the package's table applied, the identity when it ships none.+- `hash(p, seed)`: a number in 0..1 for grain and dither.+- `reveal_order(uv)`: over a title, which word is painted here, 0..1; 0+ everywhere else.+- `frame.size` in pixels, `frame.time` in seconds into the timeline,+ `frame.clip_time` in seconds since this clip began, and `frame.intensity`,+ which the host applies for you after `effect` returns.+- The grading library: `saturation`, `vibrance`, `contrast`, `s_curve`,+ `fade`, `lift_gamma_gain`, `split_tone`, `tint_midtones`, `white_balance`,+ `kelvin`, `vignette`, `mono`, `matte`, `film_curve`, `hue_rotate`,+ `hsl_band`, `halation`, `soften`, `grain_at`, `edge_at`, and the masks+ `hue_mask`, `skin_mask`, `shadows`, `highlights`, `midtones`. Their+ signatures are at the top of+ [`src/crates/concat-effects/src/shader.rs`](src/crates/concat-effects/src/shader.rs).++Two rules the loader enforces: every loop must have a `break` or a+`break if` in it, and the shader may not declare bindings of its own. Both+keep a shared shader from hanging or reaching past its inputs.++A package may carry both backends. The shader renders wherever there is a+GPU; the chain is what a machine without one gets, and what the card+preview is rendered from. The built-in looks do this.++### Transitions++```toml+[transition]+entry = "effect.wgsl"+xfade = "dissolve"+fallback = "cross-fade"+```++The shader declares `fn transition(uv: vec2<f32>, progress: f32) -> vec4<f32>`+and reads `from_at(uv)` and `to_at(uv)`, the outgoing and incoming pictures;+`sample` reads the outgoing one, so the grading helpers work on it unchanged.+`progress` runs 0 to 1 across the cut, and the shader owns the whole blend.+`xfade` names a built-in FFmpeg `xfade` transition for exports without a+GPU, and `fallback` a built-in transition id to degrade to; both are+optional.++### Tables++```toml+[lut]+file = "look.cube"+```++A `.cube` table beside the manifest. The shader reads it through `lut(rgb)`;+a chain names its path as `{lut}`, as in `lut3d=file={lut}`. Importing a+`.cube` from the Filters page writes exactly this package for you.++## Fixtures++`fixtures.toml` pins what the chain renders for given parameters, so a+change to the package, or to the expression language, that alters the+output is caught rather than shipped:++```toml+[[case]]+name = "as shot"+chain = "colorbalance=rs=0.150:gs=0.060"++[[case]]+name = "all the way"+at = "max"+chain = "colorbalance=rs=0.300:gs=0.120"++[[case]]+name = "a little"+params = { amount = 10 }+chain = "colorbalance=rs=0.030:gs=0.012"+```++`at` is `default`, `min` or `max` for every parameter, and `params` sets+some on top of it. `index` sets the fragment position. Every FFmpeg package+that ships with the app pins its default and both ends of every slider.++## The card++A package with a chain gets its card rendered for you the first time it+loads: the reference picture through the chain at its defaults, written+beside the manifest as `preview.jpg`. Put your own `preview.png` there to+use instead, which is the only way a shader-only package gets a picture.++## Checking a package++The command line does everything the app does on load, and runs the+fixtures, and says what is wrong:++```+concat-cli check ~/Library/Application\ Support/concat/effects/ada.warm-shadows+concat-cli check ~/Library/Application\ Support/concat/effects+```++One folder checks one package; a folder of folders checks them all. It+exits non-zero when any fails, so it runs in a CI job on a repository of+packages. Build it from the source tree with `cargo build -p concat-cli`+in `src/`.++## Sharing++A package is its folder. Zip it, and whoever receives it unzips it into+their effects folder. Keep the `format` line, pin fixtures for anything with+a chain, and include a `preview.png` if the package is shader-only. The+built-ins under+[`src/crates/concat-effects/packages/`](src/crates/concat-effects/packages/)+are 170-odd worked examples: `concat.sepia` is a chain and a shader,+`concat.bass` is audio, `concat.blur-dissolve` is a transition, and+`concat.adjust` shows parameters with groups and a full set of fixtures.diff --git a/README.md b/README.mdindex 18e8c2d8..766f8aa1 100644--- a/README.md+++ b/README.md@@ -89,7 +89,7 @@ Optional models download from the settings panel on first use and then never nee> [!IMPORTANT]> The best way to contribute is to grab a build from the [Releases](https://github.com/jub0t/Concat/releases) page and use it: find where it breaks, and say where it could be better.>-> Ready to write code? [CONTRIBUTING.md](./CONTRIBUTING.md) covers setup, the layout of the tree, the checks to run, and how contributions are licensed. [ROADMAP.MD](./ROADMAP.MD) says where the project is going, and [this Discussion](https://github.com/jub0t/Concat/discussions/3) is where it was announced.+> Ready to write code? [CONTRIBUTING.md](./CONTRIBUTING.md) covers setup, the layout of the tree, the checks to run, and how contributions are licensed. Want to write an effect, a filter or a transition instead? [EFFECTS.md](./EFFECTS.md) is the whole of it: a folder, a manifest, and a chain or a shader, loaded from your effects folder while the app runs. [ROADMAP.MD](./ROADMAP.MD) says where the project is going, and [this Discussion](https://github.com/jub0t/Concat/discussions/3) is where it was announced.>> Contributors are free to claim a `@Contributor` role in the Discord server, just ask for it.