Skip to content

The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.

Fix what reading the code turned up

Jareerd5abcd7a4d20
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
index 3471caa9..9b470947 100644
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -453,6 +453,10 @@ audio callback touches exactly one lock, and only with `try_lock`, because
it must never block. Position is stored *and* messaged on play and seek,
since a paused callback returns before storing.
+Every lock in playback and titles recovers from poisoning rather than
+propagating it: each guards a cache or a list that is still valid after a
+panic elsewhere, and refusing it would end audio or titles for the session.
+
**Export, templates, jobs, directories.** `Exporter` adapts a progress
closure into the engine's `Reporter` and runs under a `SingleFlight`,
which refuses a second concurrent job and gives each run its own cancel
@@ -653,43 +657,7 @@ asked for by name (they are linked, and the software decoder is what is
opened); and the monitor on Android, which composites on the CPU because
Slint's Android backend creates its own device.
-### 15.3 The intensity key and a package that declares it
-
-A filter's intensity is an undeclared key the catalogue reads as a
-percentage. `concat.vibrance` is a filter that declares a parameter with
-that name, ranged 0.1..2, so one value feeds the chain and also blends the
-result back at that many percent. Reserve the key in manifest validation
-as `index` is reserved, and rename the package's parameter.
-
-### 15.4 The GPU compositor's dead path
-
-`read_back` is typed to return `None` so a failed readback marks the
-compositor dead and every later frame goes to the CPU. No branch returns
-`None`: a failed map would panic inside the mapped range instead. Make the
-map's result the branch, so the documented fallback is reachable.
-
-### 15.5 Lock discipline in `playback.rs`
-
-Every lock outside the audio callback unwraps. A panic in a decode worker
-poisons the cache, and the next resync from any thread panics with it,
-which ends audio for the session. `titles.rs` takes the other convention
-(recover the inner value). Pick one for the crate; the recovering one is
-right for caches. In the same file: the message channel to the callback is
-unbounded and drained only by the callback, so a machine with no output
-device accumulates every set-clips message and the PCM it pins; and the
-four thread spawns `expect`.
-
-### 15.6 Cache keys that survive a toolchain
-
-`media.rs` and `playback.rs` name cache files by FNV-1a because the
-standard hasher is free to change between Rust releases, and a test pins
-the constant. `titles.rs` names its PNGs with the standard hasher. A
-toolchain bump orphans every painted title, which is self-healing but
-unbounded: the titles directory, the titles memo and a project's peaks
-have no sweep, while playback's audio cache has both a memory cap and a
-disk budget.
-
-### 15.7 Two paths that build different chains
+### 15.3 Two paths that build different chains
On the GPU, effects with a shader leave the FFmpeg chain and travel as
passes; on the CPU they stay in the chain. A machine with an adapter and
@@ -700,7 +668,7 @@ render on one machine and not the other. Either require the chain for
`effect` kinds in validation, or teach the CPU path to refuse the package
loudly.
-### 15.8 Small things the code already knows about
+### 15.4 Small things the code already knows about
- `MoveTimeline`'s documented index is counted with the timeline removed;
the clamp is against the unremoved length.
diff --git a/engine/crates/concat-effects/packages/concat.vibrance/effect.toml b/engine/crates/concat-effects/packages/concat.vibrance/effect.toml
index 255a8eaa..c21fb091 100644
--- a/engine/crates/concat-effects/packages/concat.vibrance/effect.toml
+++ b/engine/crates/concat-effects/packages/concat.vibrance/effect.toml
@@ -4,20 +4,23 @@ name = "Vibrance"
kind = "filter"
category = "Color"
order = 32
-intensity = "intensity"
+# 2: the parameter is `amount`; a project that stored `intensity` for it
+# takes the default.
+version = 2
+intensity = "amount"
aliases = ["vibrance"]
description = "Richer colour where it is muted, without blowing out what is already strong."
[[param]]
-key = "intensity"
-label = "Intensity"
+key = "amount"
+label = "Amount"
min = 0.1
max = 2
default = 0.7
step = 0.05
[ffmpeg]
-chain = "vibrance=intensity={fixed(intensity, 2)}"
+chain = "vibrance=intensity={fixed(amount, 2)}"
[wgsl]
entry = "effect.wgsl"
diff --git a/engine/crates/concat-effects/packages/concat.vibrance/effect.wgsl b/engine/crates/concat-effects/packages/concat.vibrance/effect.wgsl
index fb0611d4..7abdba85 100644
--- a/engine/crates/concat-effects/packages/concat.vibrance/effect.wgsl
+++ b/engine/crates/concat-effects/packages/concat.vibrance/effect.wgsl
@@ -1,4 +1,4 @@
-struct Params { intensity: f32 }
+struct Params { amount: f32 }
// Saturates the dull colours more than the vivid ones.
fn effect(uv: vec2<f32>) -> vec4<f32> {
@@ -7,6 +7,6 @@ fn effect(uv: vec2<f32>) -> vec4<f32> {
let mx = max(max(c.r, c.g), c.b);
let mn = min(min(c.r, c.g), c.b);
let sat = mx - mn;
- let boost = params.intensity * (1.0 - sat);
+ let boost = params.amount * (1.0 - sat);
return vec4<f32>(clamp(mix(vec3<f32>(l), c.rgb, 1.0 + boost), vec3<f32>(0.0), vec3<f32>(1.0)), c.a);
}
diff --git a/engine/crates/concat-effects/src/manifest.rs b/engine/crates/concat-effects/src/manifest.rs
index ea94b2f0..056abb20 100644
--- a/engine/crates/concat-effects/src/manifest.rs
+++ b/engine/crates/concat-effects/src/manifest.rs
@@ -258,6 +258,15 @@ impl Manifest {
if param.key == "index" {
return Err(self.invalid("`index` is reserved"));
}
+ // The key every filter answers to without declaring it: how much
+ // of the look is applied, read by the catalogue as a percent. A
+ // parameter under that name would be read twice, once as itself
+ // and once as the mix, and the two would disagree.
+ if param.key == "intensity" {
+ return Err(
+ self.invalid("`intensity` is reserved: it is the mix every filter takes")
+ );
+ }
if !seen.insert(param.key.as_str()) {
return Err(self.invalid(format!("parameter `{}` is declared twice", param.key)));
}
@@ -374,6 +383,12 @@ mod tests {
&GOOD.replace("key = \"radius\"", "key = \"index\""),
"reserved",
);
+ rejects(
+ &GOOD
+ .replace("key = \"radius\"", "key = \"intensity\"")
+ .replace("intensity = \"radius\"", "intensity = \"intensity\""),
+ "reserved",
+ );
rejects(
&GOOD.replace("unit = \"px\"", "units = \"px\""),
"unknown field",
diff --git a/engine/crates/concat-host/src/playback.rs b/engine/crates/concat-host/src/playback.rs
index 3bced280..57416cec 100644
--- a/engine/crates/concat-host/src/playback.rs
+++ b/engine/crates/concat-host/src/playback.rs
@@ -276,10 +276,21 @@ fn report(events: &dyn PlaybackEvents, message: String) {
events.error(message);
}
+/// A lock that outlives a panic on another thread. Every lock in this file
+/// guards a cache, a queue or a list, and a poisoned one still holds a valid
+/// value of that kind: recovering it costs at worst one stale entry, while
+/// refusing it would end audio for the rest of the session.
+fn locked<T>(mutex: &Mutex<T>) -> std::sync::MutexGuard<'_, T> {
+ mutex
+ .lock()
+ .unwrap_or_else(|poisoned| poisoned.into_inner())
+}
+
impl Playback {
/// Starts the audio threads: the supervised output stream, the clock
- /// publisher and the decode workers.
- pub fn start(events: Arc<dyn PlaybackEvents>) -> Arc<Playback> {
+ /// publisher and the decode workers. An error is a thread that could not
+ /// be spawned, which is the machine's condition and not the caller's.
+ pub fn start(events: Arc<dyn PlaybackEvents>) -> Result<Arc<Playback>, String> {
let (tx, rx) = mpsc::channel::<Msg>();
let shared = Arc::new(Shared {
position_micros: AtomicU64::new(0),
@@ -294,7 +305,7 @@ impl Playback {
std::thread::Builder::new()
.name("audio-output".into())
.spawn(move || supervise_stream(rx, shared, events, last_active))
- .expect("could not spawn the audio thread");
+ .map_err(|error| format!("could not spawn the audio thread: {error}"))?;
}
// Position events at ~30Hz while playing. The window interpolates
@@ -315,7 +326,7 @@ impl Playback {
}
}
})
- .expect("could not spawn the transport event thread");
+ .map_err(|error| format!("could not spawn the transport event thread: {error}"))?;
}
let playback = Arc::new(Playback {
@@ -340,10 +351,10 @@ impl Playback {
std::thread::Builder::new()
.name(format!("audio-decode-{index}"))
.spawn(move || decode_worker(&this))
- .expect("could not spawn a decode worker");
+ .map_err(|error| format!("could not spawn a decode worker: {error}"))?;
}
- playback
+ Ok(playback)
}
/// Starts playing from `position` seconds.
@@ -389,18 +400,18 @@ impl Playback {
/// decode queue and joins the mix when it lands. `project` is the
/// project folder, whose cache holds the PCM files.
pub fn set_clips(self: &Arc<Self>, project: PathBuf, specs: Vec<ClipSpec>) {
- *self.specs.lock().unwrap() = specs.clone();
- *self.project.lock().unwrap() = Some(project.clone());
+ *locked(&self.specs) = specs.clone();
+ *locked(&self.project) = Some(project.clone());
for spec in specs {
let key = decode_key(&spec);
- if self.cache.lock().unwrap().contains_key(&key) {
+ if locked(&self.cache).contains_key(&key) {
continue;
}
- if !self.decoding.lock().unwrap().insert(key.clone()) {
+ if !locked(&self.decoding).insert(key.clone()) {
continue;
}
- let mut jobs = self.queue.jobs.lock().unwrap();
+ let mut jobs = locked(&self.queue.jobs);
jobs.push_back(DecodeJob {
spec,
project: project.clone(),
@@ -417,9 +428,9 @@ impl Playback {
/// Sends the mix callback everything currently wanted *and* decoded, and
/// remembers the set so a rebuilt stream can be seeded with it.
fn resync(&self) {
- let specs = self.specs.lock().unwrap();
+ let specs = locked(&self.specs);
let now = self.tick.fetch_add(1, Ordering::Relaxed) + 1;
- let mut cache = self.cache.lock().unwrap();
+ let mut cache = locked(&self.cache);
let active: Vec<ActiveClip> = specs
.iter()
.filter_map(|spec| {
@@ -436,7 +447,7 @@ impl Playback {
})
.collect();
drop(cache);
- *self.last_active.lock().unwrap() = active.clone();
+ *locked(&self.last_active) = active.clone();
let _ = self.tx.send(Msg::SetClips(active));
}
@@ -445,8 +456,8 @@ impl Playback {
/// space and page cache, not copies - but a long session accumulates a
/// mapping per edit of every trimmed span, and those add up.
fn evict_memory(&self) {
- let live: HashSet<String> = self.specs.lock().unwrap().iter().map(decode_key).collect();
- let mut cache = self.cache.lock().unwrap();
+ let live: HashSet<String> = locked(&self.specs).iter().map(decode_key).collect();
+ let mut cache = locked(&self.cache);
while cache.len() > MEMORY_CACHE_CAP {
let doomed = cache
.iter()
@@ -471,11 +482,10 @@ impl Playback {
return;
}
let this = Arc::clone(self);
- std::thread::Builder::new()
+ let spawned = std::thread::Builder::new()
.name("audio-cache-sweep".into())
.spawn(move || {
- let live: HashSet<String> =
- this.specs.lock().unwrap().iter().map(decode_key).collect();
+ let live: HashSet<String> = locked(&this.specs).iter().map(decode_key).collect();
let directory = project.join("cache").join("audio");
if let Some(doomed) = sweep_plan(&directory, &live, DISK_CACHE_BUDGET) {
for path in doomed {
@@ -483,8 +493,12 @@ impl Playback {
}
}
this.sweeping.store(false, Ordering::Relaxed);
- })
- .expect("could not spawn the cache sweep thread");
+ });
+ // A machine that cannot spare a thread keeps its cache a while
+ // longer; the next sweep asks again.
+ if spawned.is_err() {
+ self.sweeping.store(false, Ordering::Relaxed);
+ }
}
}
@@ -493,7 +507,7 @@ impl Playback {
fn decode_worker(playback: &Arc<Playback>) {
loop {
let job = {
- let mut jobs = playback.queue.jobs.lock().unwrap();
+ let mut jobs = locked(&playback.queue.jobs);
loop {
// Newest first: pop from the back where set_clips pushes.
match jobs.pop_back() {
@@ -512,14 +526,14 @@ fn decode_worker(playback: &Arc<Playback>) {
.iter()
.any(|spec| decode_key(spec) == job.key);
if !wanted {
- playback.decoding.lock().unwrap().remove(&job.key);
+ locked(&playback.decoding).remove(&job.key);
continue;
}
match decode(&job.spec, &job.project, &job.key) {
Ok(pcm) => {
let now = playback.tick.fetch_add(1, Ordering::Relaxed) + 1;
- playback.cache.lock().unwrap().insert(
+ locked(&playback.cache).insert(
job.key.clone(),
CacheEntry {
pcm: Arc::new(pcm),
@@ -532,7 +546,7 @@ fn decode_worker(playback: &Arc<Playback>) {
format!("audio decode failed for {}: {error}", job.spec.path),
),
}
- playback.decoding.lock().unwrap().remove(&job.key);
+ locked(&playback.decoding).remove(&job.key);
playback.resync();
}
}
@@ -811,6 +825,28 @@ fn supervise_stream(
);
reported_missing = true;
}
+ // No stream means no callback to drain the channel, and
+ // every message left in it pins the PCM it names. Fold them
+ // into the seed a rebuilt stream starts from instead, so
+ // the next stream picks up where the window is and the
+ // channel stays empty.
+ if let Ok(receiver) = rx.try_lock() {
+ while let Ok(message) = receiver.try_recv() {
+ match message {
+ Msg::SetClips(next) => *locked(&last_active) = next,
+ Msg::Play(at) => {
+ shared
+ .position_micros
+ .store((at * 1_000_000.0) as u64, Ordering::Relaxed);
+ shared.playing.store(true, Ordering::Relaxed);
+ }
+ Msg::Pause => shared.playing.store(false, Ordering::Relaxed),
+ Msg::Seek(at) => shared
+ .position_micros
+ .store((at * 1_000_000.0) as u64, Ordering::Relaxed),
+ }
+ }
+ }
std::thread::sleep(backoff);
backoff = (backoff * 2).min(std::time::Duration::from_secs(30));
}
@@ -873,7 +909,7 @@ where
// Seed from the world as it is, not from zero: a stream rebuilt mid-
// playback resumes the same clips at the shared clock's position.
- let mut clips: Vec<ActiveClip> = last_active.lock().unwrap().clone();
+ let mut clips: Vec<ActiveClip> = locked(last_active).clone();
let mut playing = shared.playing.load(Ordering::Relaxed);
let mut position = shared.position_micros.load(Ordering::Relaxed) as f64 / 1_000_000.0;
diff --git a/engine/crates/concat-host/src/titles.rs b/engine/crates/concat-host/src/titles.rs
index 1651a267..b8d281db 100644
--- a/engine/crates/concat-host/src/titles.rs
+++ b/engine/crates/concat-host/src/titles.rs
@@ -16,9 +16,7 @@
//! changed is never painted twice, across sessions included. What a title
//! does *not* key on is where it sits or when it plays: moving one is free.
-use std::collections::hash_map::DefaultHasher;
use std::collections::{HashMap, HashSet};
-use std::hash::{Hash, Hasher};
use std::path::{Path, PathBuf};
use std::sync::Mutex;
@@ -59,8 +57,10 @@ pub struct Titles {
impl Titles {
/// A painter that caches under the app's data directory.
pub fn new(dirs: &AppDirs) -> Titles {
+ let dir = dirs.data.join("titles");
+ sweep(&dir, TITLES_KEPT);
Titles {
- dir: dirs.data.join("titles"),
+ dir,
fonts: Mutex::new(None),
loaded_files: Mutex::new(HashSet::new()),
memo: Mutex::new(HashMap::new()),
@@ -201,23 +201,58 @@ impl Titles {
/// Everything the pixels depend on, hashed. Placement and timing are left
/// out on purpose; see the module docs.
+///
+/// FNV-1a over the bytes, as the other caches on disk are keyed, because a
+/// file's name has to mean the same thing after a toolchain upgrade and the
+/// standard hasher makes no such promise.
fn key_of(project: &Project, style: &TextStyle, width: u32, height: u32) -> u64 {
- let mut hasher = DefaultHasher::new();
+ let mut bytes = Vec::new();
// The style as JSON: every field, in a stable order, with no need to
- // keep a Hash impl in step with the struct.
- serde_json::to_string(style)
- .unwrap_or_default()
- .hash(&mut hasher);
- width.hash(&mut hasher);
- height.hash(&mut hasher);
+ // keep a serialiser in step with the struct.
+ bytes.extend_from_slice(serde_json::to_string(style).unwrap_or_default().as_bytes());
+ bytes.push(0);
+ bytes.extend_from_slice(&width.to_le_bytes());
+ bytes.extend_from_slice(&height.to_le_bytes());
for font in &project.fonts {
- font.family.hash(&mut hasher);
- font.path.hash(&mut hasher);
+ bytes.extend_from_slice(font.family.as_bytes());
+ bytes.push(0);
+ bytes.extend_from_slice(font.path.as_bytes());
+ bytes.push(0);
}
// Bumped when the painter's output changes for the same input, so stale
// files are not mistaken for current ones.
- 1u32.hash(&mut hasher);
- hasher.finish()
+ bytes.extend_from_slice(&2u32.to_le_bytes());
+ crate::media::fnv1a(&bytes)
+}
+
+/// How many painted titles the directory keeps. Each is a frame-sized PNG,
+/// mostly transparent, so this is tens of megabytes at most; above it the
+/// oldest go, and a title that is still in use paints again.
+const TITLES_KEPT: usize = 400;
+
+/// Removes the oldest painted titles until `keep` remain. Sidecars go with
+/// their PNGs. Best effort throughout: a file that will not go is left.
+fn sweep(dir: &Path, keep: usize) {
+ let Ok(entries) = std::fs::read_dir(dir) else {
+ return;
+ };
+ let mut pngs: Vec<(std::time::SystemTime, PathBuf)> = entries
+ .flatten()
+ .map(|entry| entry.path())
+ .filter(|path| path.extension().is_some_and(|ext| ext == "png"))
+ .filter_map(|path| {
+ let modified = std::fs::metadata(&path).ok()?.modified().ok()?;
+ Some((modified, path))
+ })
+ .collect();
+ if pngs.len() <= keep {
+ return;
+ }
+ pngs.sort();
+ for (_, png) in pngs.iter().take(pngs.len() - keep) {
+ let _ = std::fs::remove_file(png);
+ let _ = std::fs::remove_file(png.with_extension("json"));
+ }
}
fn read_block(side: &Path) -> Option<(u32, u32)> {
diff --git a/engine/crates/concat-render/src/gpu.rs b/engine/crates/concat-render/src/gpu.rs
index b412367b..f5959016 100644
--- a/engine/crates/concat-render/src/gpu.rs
+++ b/engine/crates/concat-render/src/gpu.rs
@@ -850,13 +850,29 @@ impl WgpuCompositor {
/// failure the constructor's never-panic policy cannot rule out up
/// front. The caller falls back to the CPU compositor rather than
/// panicking mid-export.
+ /// `None` when the device did not deliver the pixels - a lost device,
+ /// a failed map - and the caller then marks this compositor dead. The
+ /// map's own result is what decides, not the poll's: a poll can return
+ /// without the map having been served.
fn read_back(&mut self) -> Option<Frame> {
- let target = self.target.as_ref().expect("composite rendered first");
+ let target = self.target.as_ref()?;
let (width, height, padded_row) = (target.width, target.height, target.padded_row);
let slice = target.staging.slice(..);
- slice.map_async(wgpu::MapMode::Read, |_| {});
- let _ = self.device.poll(wgpu::PollType::wait_indefinitely());
+ let (mapped_tx, mapped_rx) = std::sync::mpsc::channel();
+ slice.map_async(wgpu::MapMode::Read, move |result| {
+ let _ = mapped_tx.send(result);
+ });
+ if self
+ .device
+ .poll(wgpu::PollType::wait_indefinitely())
+ .is_err()
+ {
+ return None;
+ }
+ if !matches!(mapped_rx.try_recv(), Ok(Ok(()))) {
+ return None;
+ }
let mut frame = Frame::transparent(width, height);
{
diff --git a/engine/crates/concat/src/host.rs b/engine/crates/concat/src/host.rs
index 6c38590d..a893bf28 100644
--- a/engine/crates/concat/src/host.rs
+++ b/engine/crates/concat/src/host.rs
@@ -54,7 +54,7 @@ impl Host {
Ok(Host {
titles: concat_host::Titles::new(&dirs),
dirs,
- playback: Playback::start(Arc::new(Events)),
+ playback: Playback::start(Arc::new(Events))?,
monitor: match gpu {
Some(gpu) => Monitor::with_gpu(gpu.device, gpu.queue),
None => Monitor::new(),