The truly free, open-source, local-first, cross-platform CapCut replacement. Scriptable through a JSON-RPC API & MCP.
Rust ffmpegfree-video-editormodel-context-protocolnon-linear-editoroffline-firstopen-source-video-editorself-hostedvideo-editingvideo-editing-automationvideo-editing-softwarevideo-editing-toolsvideo-editorvoice-cloningyoutube-automationyoutube-shorts
updates
Jareer866b801df2e8
diff --git a/.github/workflows/ffmpeg.yml b/.github/workflows/ffmpeg.ymlindex a84733aa..23df4e46 100644--- a/.github/workflows/ffmpeg.yml+++ b/.github/workflows/ffmpeg.yml@@ -92,7 +92,7 @@ jobs:--enable-decoder=rawvideo,h264,hevc,vp8,vp9,av1,mpeg4,mjpeg,prores,dnxhd,png,webp,bmp,tiff,gif,aac,mp3,flac,opus,vorbis,alac,pcm_s16le,pcm_s24le,pcm_s32le,pcm_f32le \--enable-encoder=rawvideo,libx264,aac,mjpeg,png,pcm_s16le \--enable-demuxer=mov,matroska,avi,mp3,wav,flac,ogg,aac,image2,gif,rawvideo,pcm_s16le,concat \- --enable-muxer=mp4,mov,mp3,wav,ipod,mjpeg,image2,rawvideo,null \+ --enable-muxer=mp4,mov,mp3,wav,ipod,mjpeg,image2,rawvideo,pcm_s16le,null \--enable-parser=h264,hevc,vp8,vp9,av1,aac,mpegaudio,flac,vorbis,png,mjpeg \--enable-bsf=h264_mp4toannexb,hevc_mp4toannexb,aac_adtstoasc \--enable-protocol=file,pipe \@@ -141,6 +141,14 @@ jobs:# Sound: the preview render, then the full export mix and mux."$BIN/ffmpeg.exe" -hide_banner -loglevel error -y \-f s16le -ar 48000 -ac 2 -i tone.pcm -c:a aac tone.m4a++ # The playback engine's decode: a clip span straight to raw PCM.+ # See desktop/src-tauri/src/playback.rs - this needs the pcm_s16le+ # *muxer*, easy to forget because the demuxer of the same name is+ # enabled for a different reason.+ "$BIN/ffmpeg.exe" -hide_banner -loglevel error -y -ss 0 -t 1 -i tone.m4a -vn \+ -af "aresample=48000,aformat=sample_fmts=s16:channel_layouts=stereo" \+ -f s16le span.pcm"$BIN/ffmpeg.exe" -hide_banner -loglevel error -y -i tone.m4a -vn \-af "aresample=48000,asetrate=52000,aresample=48000,atempo=0.92,highpass=f=85,equalizer=f=4200:t=q:w=0.8:g=3,compand=attacks=0.01:decays=0.18:points=-80/-80|0/-1.5,deesser=i=0.3,aecho=0.8:0.82:38:0.02,alimiter=limit=0.94,aphaser=type=t:speed=1.2,flanger=delay=2:depth=4,bass=g=5,treble=g=4" \-ac 2 -ar 48000 -c:a pcm_s16le -f wav filtered.wavdiff --git a/desktop/docs/decisions/0004-audio-preview-in-the-webview.md b/desktop/docs/decisions/0004-audio-preview-in-the-webview.mdindex 152fab3a..87c424aa 100644--- a/desktop/docs/decisions/0004-audio-preview-in-the-webview.md+++ b/desktop/docs/decisions/0004-audio-preview-in-the-webview.md@@ -1,5 +1,12 @@# 0004 - Audio and video preview run in the webview, for now+> **Superseded in part by [0005](0005-audio-playback-in-the-engine.md):**+> audio playback now lives in the engine - decoded, mixed and clocked in+> `src-tauri/src/playback.rs`, with the UI following the engine's position+> events. What remains true here is the *video* half: the monitor picture is+> still the webview's `<video>` element, muted, until the engine can present+> frames.+## DecisionPlayback preview - both the picture in the monitor and the sound - uses thewebview's own `<video>` and `<audio>` elements, fed through Tauri's assetdiff --git a/desktop/docs/decisions/0005-audio-playback-in-the-engine.md b/desktop/docs/decisions/0005-audio-playback-in-the-engine.mdnew file mode 100644index 00000000..72d70a4c--- /dev/null+++ b/desktop/docs/decisions/0005-audio-playback-in-the-engine.md@@ -0,0 +1,51 @@+# 0005 - Audio playback lives in the engine++## Decision++All audible playback is decoded, mixed and clocked by the Rust host+(`src-tauri/src/playback.rs`). The webview no longer owns any part of the+audio path: it describes the audible clip set (`audio_set_clips`), drives the+transport (`transport_play` / `transport_pause` / `transport_seek`), and+follows the engine's `transport` position events. The monitor's `<video>`+element is permanently muted - it is a picture surface, nothing more.++## Why++The webview mixer (decision 0004) was built on `HTMLMediaElement` routed+through `createMediaElementSource`, corrected against a wall-clock transport.+That design hit its ceiling in practice, not in theory: WebKit's+element-through-graph pipeline drops out audibly even on a three-second file,+corrective reseeks are audible clicks, filtered clips required rendering a+whole WAV per slider adjustment, and there were three clocks - wall clock,+media element, audio context - that could only ever agree approximately.++The engine path has exactly one clock: the audio device's sample counter.++## How it works++- **Decode once, mix forever.** Each audible clip's source span is decoded by+ FFmpeg to raw 48 kHz stereo PCM, with the clip's filter chain and speed+ baked in using the same filters the exporter uses - preview and export+ cannot disagree. Volume and fades apply at mix time (the gain law mirrors+ `clipGainAt`), so gain edits never re-decode.+- **Disk-backed, memory-mapped.** PCM lands in `<project>/cache/audio/` keyed+ by everything that changes the samples, and is memory-mapped rather than+ loaded - memory stays bounded on long material, and a reopened project+ replays instantly.+- **A lock-free mix callback.** The cpal callback owns its state and drains a+ message channel; no locks are taken on the audio thread. Clip-set changes,+ play, pause and seek are messages.+- **The clock flows outward.** The callback publishes its position; a 30 Hz+ event stream carries it to the UI, which interpolates on the wall clock+ between events. The UI never tells the engine what time it is while playing.++## What this is not, yet++- **Not video presentation.** The picture is still the muted webview element+ showing the top-most clip (decision 0004's video half stands). When the+ engine can decode and present frames, the same transport already drives it.+- **Not streaming decode.** A clip's span is decoded whole before it joins+ the mix. New material is silent for the decode's duration (cached+ thereafter). Fine for editing; a capture monitor would need a ring buffer.+- **Not an undo boundary.** The mixer is a pure function of the clip set it+ is sent; it holds no editing state at all.diff --git a/desktop/src-tauri/Cargo.lock b/desktop/src-tauri/Cargo.lockindex 6d02793e..e684ea61 100644--- a/desktop/src-tauri/Cargo.lock+++ b/desktop/src-tauri/Cargo.lock@@ -32,6 +32,28 @@ dependencies = ["alloc-no-stdlib",]+[[package]]+name = "alsa"+version = "0.9.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "ed7572b7ba83a31e20d1b48970ee402d2e3e0537dcfe0a3ff4d6eb7508617d43"+dependencies = [+ "alsa-sys",+ "bitflags 2.13.1",+ "cfg-if",+ "libc",+]++[[package]]+name = "alsa-sys"+version = "0.3.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "db8fee663d06c4e303404ef5f40488a53e062f89ba8bfed81f42325aafad1527"+dependencies = [+ "libc",+ "pkg-config",+]+[[package]]name = "android_system_properties"version = "0.1.6"@@ -534,6 +556,46 @@ dependencies = ["libc",]+[[package]]+name = "coreaudio-rs"+version = "0.13.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "1aae284fbaf7d27aa0e292f7677dfbe26503b0d555026f702940805a630eac17"+dependencies = [+ "bitflags 1.3.2",+ "libc",+ "objc2-audio-toolbox",+ "objc2-core-audio",+ "objc2-core-audio-types",+ "objc2-core-foundation",+]++[[package]]+name = "cpal"+version = "0.16.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "cbd307f43cc2a697e2d1f8bc7a1d824b5269e052209e28883e5bc04d095aaa3f"+dependencies = [+ "alsa",+ "coreaudio-rs",+ "dasp_sample",+ "jni",+ "js-sys",+ "libc",+ "mach2",+ "ndk",+ "ndk-context",+ "num-derive",+ "num-traits",+ "objc2-audio-toolbox",+ "objc2-core-audio",+ "objc2-core-audio-types",+ "wasm-bindgen",+ "wasm-bindgen-futures",+ "web-sys",+ "windows 0.54.0",+]+[[package]]name = "cpufeatures"version = "0.2.17"@@ -650,6 +712,12 @@ dependencies = ["syn 2.0.119",]+[[package]]+name = "dasp_sample"+version = "0.11.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f"+[[package]]name = "dbus"version = "0.9.12"@@ -2057,6 +2125,15 @@ version = "0.4.34"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"+[[package]]+name = "mach2"+version = "0.4.3"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"+dependencies = [+ "libc",+]+[[package]]name = "markup5ever"version = "0.38.0"@@ -2074,6 +2151,15 @@ version = "2.8.3"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"+[[package]]+name = "memmap2"+version = "0.9.11"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "d1219ed1b7f229ee7104d281dd01d6802fe28bb6e95d292942c4daacdeb798c0"+dependencies = [+ "libc",+]+[[package]]name = "memoffset"version = "0.9.1"@@ -2146,6 +2232,12 @@ dependencies = ["thiserror 1.0.69",]+[[package]]+name = "ndk-context"+version = "0.1.1"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "27b02d87554356db9e9a873add8782d4ea6e3e58ea071a9adb9a2e8ddb884a8b"+[[package]]name = "ndk-sys"version = "0.6.0+11769913"@@ -2167,6 +2259,17 @@ version = "0.2.2"source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"+[[package]]+name = "num-derive"+version = "0.4.2"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "ed3955f1a9c7c0c15e092f9c887db08b1fc683305fdf6eb6684f22555355e202"+dependencies = [+ "proc-macro2",+ "quote",+ "syn 2.0.119",+]+[[package]]name = "num-traits"version = "0.2.19"@@ -2221,6 +2324,21 @@ dependencies = ["objc2-foundation",]+[[package]]+name = "objc2-audio-toolbox"+version = "0.3.2"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "6948501a91121d6399b79abaa33a8aa4ea7857fe019f341b8c23ad6e81b79b08"+dependencies = [+ "bitflags 2.13.1",+ "libc",+ "objc2",+ "objc2-core-audio",+ "objc2-core-audio-types",+ "objc2-core-foundation",+ "objc2-foundation",+]+[[package]]name = "objc2-cloud-kit"version = "0.3.2"@@ -2232,6 +2350,28 @@ dependencies = ["objc2-foundation",]+[[package]]+name = "objc2-core-audio"+version = "0.3.2"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "e1eebcea8b0dbff5f7c8504f3107c68fc061a3eb44932051c8cf8a68d969c3b2"+dependencies = [+ "dispatch2",+ "objc2",+ "objc2-core-audio-types",+ "objc2-core-foundation",+]++[[package]]+name = "objc2-core-audio-types"+version = "0.3.2"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "5a89f2ec274a0cf4a32642b2991e8b351a404d290da87bb6a9a9d8632490bd1c"+dependencies = [+ "bitflags 2.13.1",+ "objc2",+]+[[package]]name = "objc2-core-data"version = "0.3.2"@@ -2827,6 +2967,8 @@ version = "0.1.0"name = "relay-desktop"version = "0.1.0"dependencies = [+ "cpal",+ "memmap2","relay-core","relay-media","relay-render",@@ -3493,7 +3635,7 @@ dependencies = ["tao-macros","unicode-segmentation","url",- "windows",+ "windows 0.61.3","windows-core 0.61.2","windows-version","x11-dl",@@ -3565,7 +3707,7 @@ dependencies = ["webkit2gtk","webview2-com","window-vibrancy",- "windows",+ "windows 0.61.3",][[package]]@@ -3706,7 +3848,7 @@ dependencies = ["tauri-plugin","thiserror 2.0.20","url",- "windows",+ "windows 0.61.3","zbus",]@@ -3732,7 +3874,7 @@ dependencies = ["url","webkit2gtk","webview2-com",- "windows",+ "windows 0.61.3",][[package]]@@ -3757,7 +3899,7 @@ dependencies = ["url","webkit2gtk","webview2-com",- "windows",+ "windows 0.61.3","wry",]@@ -4511,7 +4653,7 @@ checksum = "7130243a7a5b33c54a444e54842e6a9e133de08b5ad7b5861cd8ed9a6a5bc96a"dependencies = ["webview2-com-macros","webview2-com-sys",- "windows",+ "windows 0.61.3","windows-core 0.61.2","windows-implement","windows-interface",@@ -4535,7 +4677,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"checksum = "381336cfffd772377d291702245447a5251a2ffa5bad679c99e61bc48bacbf9c"dependencies = ["thiserror 2.0.20",- "windows",+ "windows 0.61.3","windows-core 0.61.2",]@@ -4585,6 +4727,16 @@ dependencies = ["windows-version",]+[[package]]+name = "windows"+version = "0.54.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "9252e5725dbed82865af151df558e754e4a3c2c30818359eb17465f1346a1b49"+dependencies = [+ "windows-core 0.54.0",+ "windows-targets 0.52.6",+]+[[package]]name = "windows"version = "0.61.3"@@ -4607,6 +4759,16 @@ dependencies = ["windows-core 0.61.2",]+[[package]]+name = "windows-core"+version = "0.54.0"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "12661b9c89351d684a50a8a643ce5f608e20243b9fb84687800163429f161d65"+dependencies = [+ "windows-result 0.1.2",+ "windows-targets 0.52.6",+]+[[package]]name = "windows-core"version = "0.61.2"@@ -4688,6 +4850,15 @@ dependencies = ["windows-link 0.1.3",]+[[package]]+name = "windows-result"+version = "0.1.2"+source = "registry+https://github.com/rust-lang/crates.io-index"+checksum = "5e383302e8ec8515204254685643de10811af0ed97ea37210dc26fb0032647f8"+dependencies = [+ "windows-targets 0.52.6",+]+[[package]]name = "windows-result"version = "0.3.4"@@ -5048,7 +5219,7 @@ dependencies = ["webkit2gtk","webkit2gtk-sys","webview2-com",- "windows",+ "windows 0.61.3","windows-core 0.61.2","windows-version","x11-dl",diff --git a/desktop/src-tauri/Cargo.toml b/desktop/src-tauri/Cargo.tomlindex 8586e52f..0a0b9867 100644--- a/desktop/src-tauri/Cargo.toml+++ b/desktop/src-tauri/Cargo.toml@@ -28,6 +28,11 @@ relay-media = { path = "../../engine/crates/relay-media" }relay-render = { path = "../../engine/crates/relay-render" }rusqlite = { version = "0.40.2", features = ["bundled"] }+# Engine-owned audio playback: cpal talks to the output device, memmap2 keeps+# decoded PCM disk-backed instead of resident.+cpal = "0.16"+memmap2 = "0.9"+[profile.release]opt-level = 3lto = truediff --git a/desktop/src-tauri/src/export.rs b/desktop/src-tauri/src/export.rsindex 43046830..3951d9ce 100644--- a/desktop/src-tauri/src/export.rs+++ b/desktop/src-tauri/src/export.rs@@ -17,11 +17,11 @@ use std::path::{Path, PathBuf};use relay_core::frame::Frame;use relay_core::time::{FrameRate, Rational};-use relay_core::timeline::{Clip, ClipId, MediaRef, Timeline, Track, TrackKind};+use relay_core::timeline::{Clip, ClipId, MediaRef, Timeline, Track, TrackKind, Transform};use relay_media::{DecodeOptions, EncodeOptions, FfmpegDecoder, FfmpegEncoder, FrameSink, FrameSource,};-use relay_render::{Compositor, CpuCompositor, Layer, plan_frame};+use relay_render::{Compositor, CpuCompositor, Layer, Placement, plan_frame};use serde::{Deserialize, Serialize};use tauri::Emitter;@@ -57,6 +57,25 @@ pub struct ExportClip {/// False lets pitch rise with the rate, like tape.#[serde(default = "yes")]pub preserve_pitch: bool,+ /// Multiplier over the fitted size. 1 fills the frame, preserving aspect.+ #[serde(default = "unity")]+ pub scale: f64,+ /// Offset of the picture's centre from frame centre, frame-width fraction.+ #[serde(default)]+ pub offset_x: f64,+ /// Offset as a frame-height fraction.+ #[serde(default)]+ pub offset_y: f64,+ /// Clockwise rotation in degrees.+ #[serde(default)]+ pub rotation: f64,+ /// The source's pixel size, when the UI knows it. What makes an+ /// aspect-correct decode possible - absent, the frame is filled edge to+ /// edge the way it always was.+ #[serde(default)]+ pub media_width: Option<u32>,+ #[serde(default)]+ pub media_height: Option<u32>,}fn yes() -> bool {@@ -170,7 +189,7 @@ fn render_picture(visible: &[&ExportClip],destination: &Path,) -> Result<(), String> {- let (timeline, stills) = build_timeline(request, rate, visible);+ let (timeline, stills, decode_sizes) = build_timeline(request, rate, visible);let mut encoder = FfmpegEncoder::create(destination,@@ -197,14 +216,21 @@ fn render_picture(let time = rate.time_of_frame(index);let plan = plan_frame(&timeline, time);- let mut sources: Vec<(Frame, f32)> = Vec::with_capacity(plan.layers.len());+ let mut sources: Vec<(Frame, f32, Transform)> = Vec::with_capacity(plan.layers.len());for layer in &plan.layers {let decoder = match decoders.entry(layer.clip) {std::collections::hash_map::Entry::Occupied(entry) => entry.into_mut(),std::collections::hash_map::Entry::Vacant(entry) => {+ // Aspect-correct: decode at the source's fitted size and+ // let the compositor place it, rather than stretching to+ // the frame and losing the picture's shape.+ let (decode_width, decode_height) = decode_sizes+ .get(&layer.clip)+ .copied()+ .unwrap_or((request.width, request.height));let mut options = DecodeOptions::default().starting_at(layer.source_time)- .scaled_to(request.width, request.height)+ .scaled_to(decode_width, decode_height).at_rate(rate);// A still is a one-frame stream. Without looping it would@@ -224,13 +250,33 @@ fn render_picture(// aborting the export - a clip trimmed past its media's end is a// mistake in the edit, not a failure of the renderer.if let Some(frame) = decoder.next_frame().map_err(|error| error.to_string())? {- sources.push((frame, layer.opacity));+ sources.push((frame, layer.opacity, layer.transform));}}let layers: Vec<Layer<'_>> = sources.iter()- .map(|(frame, opacity)| Layer::new(frame).with_opacity(*opacity))+ .map(|(frame, opacity, transform)| {+ // Fitted and centred is the base; the clip's transform moves+ // it from there. The fraction-to-pixel conversion happens+ // here and nowhere else.+ let x = (i64::from(request.width) - i64::from(frame.width())) / 2;+ let y = (i64::from(request.height) - i64::from(frame.height())) / 2;+ let placement = if transform.is_identity() {+ Placement::IDENTITY+ } else {+ Placement {+ scale: transform.scale as f32,+ rotation: transform.rotation.to_radians() as f32,+ translate_x: (transform.offset_x * f64::from(request.width)) as f32,+ translate_y: (transform.offset_y * f64::from(request.height)) as f32,+ }+ };+ Layer::new(frame)+ .at(x as i32, y as i32)+ .with_opacity(*opacity)+ .with_placement(placement)+ }).collect();let composed = compositor.composite(request.width, request.height, &layers);@@ -257,9 +303,14 @@ fn build_timeline(request: &ExportRequest,rate: FrameRate,visible: &[&ExportClip],-) -> (Timeline, std::collections::HashSet<ClipId>) {+) -> (+ Timeline,+ std::collections::HashSet<ClipId>,+ HashMap<ClipId, (u32, u32)>,+) {let mut timeline = Timeline::new(request.width, request.height, rate);let mut stills = std::collections::HashSet::new();+ let mut decode_sizes: HashMap<ClipId, (u32, u32)> = HashMap::new();let lanes = visible.iter().map(|clip| clip.track).max().unwrap_or(0) + 1;let tracks: Vec<_> = (0..lanes)@@ -278,15 +329,39 @@ fn build_timeline(let mut engine_clip = Clip::new(MediaRef::new(&clip.path), start, duration);engine_clip.source_start = quantise(clip.source_start, rate);+ engine_clip.transform = Transform {+ scale: clip.scale,+ offset_x: clip.offset_x,+ offset_y: clip.offset_y,+ rotation: clip.rotation,+ };if let Some(id) = timeline.add_clip(tracks[clip.track], engine_clip) {if clip.kind == "image" {stills.insert(id);}+ if let Some(size) = fitted_size(request, clip) {+ decode_sizes.insert(id, size);+ }}}- (timeline, stills)+ (timeline, stills, decode_sizes)+}++/// The source's contain-fitted size inside the output frame, or `None` when+/// the UI never learnt the source's dimensions.+fn fitted_size(request: &ExportRequest, clip: &ExportClip) -> Option<(u32, u32)> {+ let media_width = clip.media_width.filter(|value| *value > 0)?;+ let media_height = clip.media_height.filter(|value| *value > 0)?;++ let fit = (f64::from(request.width) / f64::from(media_width))+ .min(f64::from(request.height) / f64::from(media_height));+ let width = ((f64::from(media_width) * fit).round() as u32).max(2);+ let height = ((f64::from(media_height) * fit).round() as u32).max(2);+ // Even, because a decoder asked for an odd width may round it itself and+ // then every frame read is misaligned by a pixel's worth of bytes.+ Some((width & !1, height & !1))}fn quantise(seconds: f64, rate: FrameRate) -> Rational {diff --git a/desktop/src-tauri/src/lib.rs b/desktop/src-tauri/src/lib.rsindex f137aa6f..f435b4ea 100644--- a/desktop/src-tauri/src/lib.rs+++ b/desktop/src-tauri/src/lib.rs@@ -7,6 +7,7 @@//! `relay-render` where it can be unit-tested without a window.mod export;+mod playback;mod projects;use serde::Serialize;@@ -140,6 +141,49 @@ fn read_media_bytes(path: String) -> Result<tauri::ipc::Response, String> {.map_err(|error| format!("could not read {path}: {error}"))}+/// Where one artwork file lives inside a project's cache.+///+/// The cache sits in the project folder so it travels with the project and+/// vanishes with it. The key is confined to a single flat filename - anything+/// that could walk out of the folder is refused rather than sanitised,+/// because the only caller is our own frontend and a strange key is a bug.+fn artwork_file(project: &str, key: &str) -> Result<std::path::PathBuf, String> {+ if key.is_empty()+ || !key+ .chars()+ .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-'))+ || key.starts_with('.')+ {+ return Err(format!("refusing artwork key {key:?}"));+ }+ let root = std::path::Path::new(project);+ if !root.is_dir() {+ return Err(format!("{project} is not a project folder"));+ }+ Ok(root.join("cache").join(key))+}++/// Returns one cached artwork file, or an error the frontend treats as a miss.+#[tauri::command]+fn read_artwork(project: String, key: String) -> Result<tauri::ipc::Response, String> {+ let file = artwork_file(&project, &key)?;+ std::fs::read(&file)+ .map(tauri::ipc::Response::new)+ .map_err(|error| format!("no cached artwork {key}: {error}"))+}++/// Stores one artwork file in the project's cache. Best-effort: the caller+/// fires and forgets, and a failed write only means regenerating next launch.+#[tauri::command]+fn write_artwork(project: String, key: String, bytes: Vec<u8>) -> Result<(), String> {+ let file = artwork_file(&project, &key)?;+ if let Some(parent) = file.parent() {+ std::fs::create_dir_all(parent)+ .map_err(|error| format!("could not create {}: {error}", parent.display()))?;+ }+ std::fs::write(&file, bytes).map_err(|error| format!("could not write {key}: {error}"))+}+/// Renders a strip of evenly spaced frames from a video as one JPEG.////// One image rather than N, because the timeline draws the frames as slices of@@ -176,12 +220,15 @@ fn filmstrip(path: &str, count: u32, height: u32) -> Result<Vec<u8>, String> {"-vf",// scale=-2 keeps the aspect ratio and an even width, which the// JPEG encoder requires.+ // lanczos because the default bilinear leaves 4K sources visibly+ // mushy at thumbnail sizes, and the strip is rendered once but+ // looked at constantly.&format!(- "fps={:.6},scale=-2:{height},tile={count}x1",+ "fps={:.6},scale=-2:{height}:flags=lanczos,tile={count}x1",f64::from(count) / duration),])- .args(["-frames:v", "1", "-q:v", "6", "-f", "mjpeg", "-"])+ .args(["-frames:v", "1", "-q:v", "3", "-f", "mjpeg", "-"]).output().map_err(|error| format!("could not run ffmpeg: {error}"))?;@@ -195,61 +242,33 @@ fn filmstrip(path: &str, count: u32, height: u32) -> Result<Vec<u8>, String> {Ok(output.stdout)}-/// Renders one clip's audio through a filter chain and returns it as WAV.-///-/// This is what lets the preview play filtered audio *exactly* as the export-/// will render it. The alternative - reimplementing pitch shifting, companding-/// and de-essing in Web Audio - would be a second implementation to keep in-/// step with the first, and it would be wrong in ways nobody would notice-/// until the exported file sounded different from the edit.+/// The audible clip set, replaced wholesale whenever the timeline changes.///-/// WAV because it decodes instantly in the browser and needs no container-/// seeking; the result is short-lived and never written to disk.+/// Decoding, mixing and the playback clock all live in the engine - see+/// `playback.rs`. The UI describes what should be audible and follows the+/// engine's `transport` position events.#[tauri::command]-async fn render_filtered_audio(- path: String,- source_start: f64,- duration: f64,- chain: String,-) -> Result<tauri::ipc::Response, String> {- let bytes = tauri::async_runtime::spawn_blocking(move || {- // A filter chain arrives as text and goes straight to FFmpeg, so it- // must not be able to smuggle in another argument.- if chain.contains('\n') || chain.contains('\r') {- return Err("filter chain contains a line break".to_owned());- }+fn audio_set_clips(+ state: tauri::State<'_, std::sync::Arc<playback::Playback>>,+ project: String,+ clips: Vec<playback::ClipSpec>,+) {+ state.set_clips(std::path::PathBuf::from(project), clips);+}- let mut command = std::process::Command::new(relay_media::ffmpeg());- command- .args(["-hide_banner", "-nostdin", "-loglevel", "error"])- .args(["-ss", &format!("{source_start:.6}")])- .args(["-t", &format!("{duration:.6}")])- .args(["-i", &path])- .args(["-vn", "-af", &chain])- // 16-bit stereo at 48k: what the filters run at anyway, and small- // enough to move over the bridge without a second thought.- .args(["-ac", "2", "-ar", "48000", "-c:a", "pcm_s16le"])- .args(["-f", "wav", "-"]);-- let output = command- .output()- .map_err(|error| format!("could not run ffmpeg: {error}"))?;-- if !output.status.success() {- let detail = String::from_utf8_lossy(&output.stderr);- let tail = detail.lines().rev().take(2).collect::<Vec<_>>().join(" / ");- return Err(format!("filter failed: {tail}"));- }- if output.stdout.is_empty() {- return Err("filter produced no audio".to_owned());- }+#[tauri::command]+fn transport_play(state: tauri::State<'_, std::sync::Arc<playback::Playback>>, position: f64) {+ state.play(position);+}- Ok(output.stdout)- })- .await- .map_err(|error| format!("filter task failed: {error}"))??;+#[tauri::command]+fn transport_pause(state: tauri::State<'_, std::sync::Arc<playback::Playback>>) {+ state.pause();+}- Ok(tauri::ipc::Response::new(bytes))+#[tauri::command]+fn transport_seek(state: tauri::State<'_, std::sync::Arc<playback::Playback>>, position: f64) {+ state.seek(position);}/// Creates a project folder, writes its manifest and records it as recent.@@ -406,7 +425,9 @@ pub fn run() {.setup(|app| {// Resolved here rather than before the builder, because finding// the resource directory needs the app to exist.+ use tauri::Manager;use_bundled_ffmpeg(app);+ app.manage(playback::Playback::start(app.handle().clone()));Ok(())}).plugin(tauri_plugin_opener::init())@@ -415,8 +436,13 @@ pub fn run() {probe_media,engine_version,read_media_bytes,- render_filtered_audio,+ audio_set_clips,+ transport_play,+ transport_pause,+ transport_seek,extract_filmstrip,+ read_artwork,+ write_artwork,create_project,open_project,save_project,diff --git a/desktop/src-tauri/src/playback.rs b/desktop/src-tauri/src/playback.rsnew file mode 100644index 00000000..2ce8fd9d--- /dev/null+++ b/desktop/src-tauri/src/playback.rs@@ -0,0 +1,475 @@+//! Engine-owned audio playback.+//!+//! This replaces the webview audio preview (see the superseded+//! docs/decisions/0004): the engine decodes, mixes and clocks all audible+//! clips itself, and the UI is a controller that follows.+//!+//! The shape of it:+//!+//! - Every audible clip's span is decoded **once** by FFmpeg straight to raw+//! PCM - the clip's filter chain and speed baked in with the same filters+//! the exporter uses, so the preview and the export cannot disagree. The+//! PCM lands as a file in the project's cache and is memory-mapped, so+//! memory stays bounded however long the material is, and a reopened+//! project pays nothing.+//!+//! - One cpal output stream mixes the mapped clips sample by sample, with+//! volume and fades applied at mix time (they mirror `clipGainAt` in the+//! frontend exactly). Gain-only edits therefore never re-decode.+//!+//! - The playback clock is the audio device's own sample counter. The UI+//! receives `transport` position events and interpolates between them;+//! there is no second clock to drift against.+//!+//! The mix callback owns its state and receives changes as messages, so it+//! takes no locks on the audio thread. Decodes run on worker threads and the+//! clip set is re-sent to the callback as each one lands.++use std::collections::{HashMap, HashSet};+use std::hash::{Hash, Hasher};+use std::path::{Path, PathBuf};+use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};+use std::sync::{mpsc, Arc, Mutex};++use serde::Deserialize;+use tauri::Emitter;++/// Every clip is decoded to this rate; the callback resamples to the device.+const PCM_RATE: u32 = 48_000;++/// One audible clip, as the frontend describes it.+#[derive(Clone, Deserialize)]+#[serde(rename_all = "camelCase")]+pub struct ClipSpec {+ pub path: String,+ /// Timeline seconds.+ pub start: f64,+ pub duration: f64,+ /// Seconds into the source file.+ pub source_start: f64,+ pub volume: f32,+ pub fade_in: f64,+ pub fade_out: f64,+ pub speed: f64,+ pub preserve_pitch: bool,+ /// FFmpeg audio filter chain, or empty.+ #[serde(default)]+ pub chain: String,+}++/// Decoded audio: 16-bit little-endian stereo at [`PCM_RATE`], memory-mapped+/// from the cache file so the OS decides what stays resident.+struct Pcm {+ map: memmap2::Mmap,+ frames: u64,+}++impl Pcm {+ fn open(path: &Path) -> Result<Pcm, String> {+ let file = std::fs::File::open(path)+ .map_err(|error| format!("could not open {}: {error}", path.display()))?;+ let map = unsafe { memmap2::Mmap::map(&file) }+ .map_err(|error| format!("could not map {}: {error}", path.display()))?;+ let frames = (map.len() / 4) as u64;+ Ok(Pcm { map, frames })+ }++ /// One sample as a float in [-1, 1]. Out of range reads are silence.+ fn sample(&self, frame: u64, channel: usize) -> f32 {+ if frame >= self.frames {+ return 0.0;+ }+ let at = (frame as usize * 2 + channel) * 2;+ let raw = i16::from_le_bytes([self.map[at], self.map[at + 1]]);+ f32::from(raw) / 32768.0+ }+}++/// A clip the mix callback can play: spec fields it needs, plus the audio.+struct ActiveClip {+ start: f64,+ duration: f64,+ volume: f32,+ fade_in: f64,+ fade_out: f64,+ pcm: Arc<Pcm>,+}++/// Mirror of the frontend's `clipGainAt`, so preview loudness has exactly one+/// definition even though it is computed in two languages.+fn gain_at(clip: &ActiveClip, local: f64) -> f32 {+ if local < 0.0 || local > clip.duration {+ return 0.0;+ }+ let mut gain = f64::from(clip.volume);+ if clip.fade_in > 0.0 {+ gain *= (local / clip.fade_in).min(1.0);+ }+ if clip.fade_out > 0.0 {+ gain *= ((clip.duration - local) / clip.fade_out).min(1.0);+ }+ gain.max(0.0) as f32+}++enum Msg {+ SetClips(Vec<ActiveClip>),+ Play(f64),+ Pause,+ Seek(f64),+}++struct Shared {+ /// Timeline position in microseconds, written by the mix callback.+ position_micros: AtomicU64,+ playing: AtomicBool,+}++pub struct Playback {+ tx: mpsc::Sender<Msg>,+ shared: Arc<Shared>,+ /// Decoded clips by decode key, for the lifetime of the app.+ cache: Mutex<HashMap<String, Arc<Pcm>>>,+ decoding: Mutex<HashSet<String>>,+ /// What the timeline currently wants audible.+ specs: Mutex<Vec<ClipSpec>>,+}++impl Playback {+ pub fn start(app: tauri::AppHandle) -> Arc<Playback> {+ let (tx, rx) = mpsc::channel::<Msg>();+ let shared = Arc::new(Shared {+ position_micros: AtomicU64::new(0),+ playing: AtomicBool::new(false),+ });++ {+ let shared = Arc::clone(&shared);+ std::thread::Builder::new()+ .name("audio-output".into())+ .spawn(move || audio_thread(rx, shared))+ .expect("could not spawn the audio thread");+ }++ // Position events at ~30Hz while playing. The UI interpolates between+ // them, so this cadence bounds correction error, not smoothness.+ {+ let shared = Arc::clone(&shared);+ std::thread::Builder::new()+ .name("transport-events".into())+ .spawn(move || loop {+ std::thread::sleep(std::time::Duration::from_millis(33));+ if shared.playing.load(Ordering::Relaxed) {+ let position =+ shared.position_micros.load(Ordering::Relaxed) as f64 / 1_000_000.0;+ let _ = app.emit("transport", position);+ }+ })+ .expect("could not spawn the transport event thread");+ }++ Arc::new(Playback {+ tx,+ shared,+ cache: Mutex::new(HashMap::new()),+ decoding: Mutex::new(HashSet::new()),+ specs: Mutex::new(Vec::new()),+ })+ }++ pub fn play(&self, position: f64) {+ self.shared.playing.store(true, Ordering::Relaxed);+ self.shared+ .position_micros+ .store((position.max(0.0) * 1_000_000.0) as u64, Ordering::Relaxed);+ let _ = self.tx.send(Msg::Play(position.max(0.0)));+ }++ pub fn pause(&self) {+ self.shared.playing.store(false, Ordering::Relaxed);+ let _ = self.tx.send(Msg::Pause);+ }++ pub fn seek(&self, position: f64) {+ let _ = self.tx.send(Msg::Seek(position.max(0.0)));+ }++ /// Replaces the audible clip set.+ ///+ /// Anything already decoded plays immediately; anything new decodes on a+ /// worker and joins the mix when it lands. `project` is the project+ /// folder, whose cache holds the PCM files.+ pub fn set_clips(self: &Arc<Self>, project: PathBuf, specs: Vec<ClipSpec>) {+ *self.specs.lock().unwrap() = specs.clone();++ for spec in specs {+ let key = decode_key(&spec);+ if self.cache.lock().unwrap().contains_key(&key) {+ continue;+ }+ if !self.decoding.lock().unwrap().insert(key.clone()) {+ continue;+ }++ let this = Arc::clone(self);+ let project = project.clone();+ std::thread::Builder::new()+ .name("audio-decode".into())+ .spawn(move || {+ match decode(&spec, &project, &key) {+ Ok(pcm) => {+ this.cache.lock().unwrap().insert(key.clone(), Arc::new(pcm));+ }+ Err(error) => eprintln!("audio decode failed for {}: {error}", spec.path),+ }+ this.decoding.lock().unwrap().remove(&key);+ this.resync();+ })+ .expect("could not spawn a decode thread");+ }++ self.resync();+ }++ /// Sends the mix callback everything currently wanted *and* decoded.+ fn resync(&self) {+ let specs = self.specs.lock().unwrap();+ let cache = self.cache.lock().unwrap();+ let active = specs+ .iter()+ .filter_map(|spec| {+ let pcm = cache.get(&decode_key(spec))?;+ Some(ActiveClip {+ start: spec.start,+ duration: spec.duration,+ volume: spec.volume,+ fade_in: spec.fade_in,+ fade_out: spec.fade_out,+ pcm: Arc::clone(pcm),+ })+ })+ .collect();+ let _ = self.tx.send(Msg::SetClips(active));+ }+}++/// Identity of one decoded span: everything that changes the samples.+/// Volume, fades and timeline position are deliberately absent - they apply+/// at mix time.+fn decode_key(spec: &ClipSpec) -> String {+ let mut hasher = std::collections::hash_map::DefaultHasher::new();+ spec.path.hash(&mut hasher);+ spec.source_start.to_bits().hash(&mut hasher);+ spec.duration.to_bits().hash(&mut hasher);+ spec.speed.to_bits().hash(&mut hasher);+ spec.preserve_pitch.hash(&mut hasher);+ spec.chain.hash(&mut hasher);+ format!("{:016x}", hasher.finish())+}++/// Decodes one clip span to the cache and maps it.+///+/// The FFmpeg invocation mirrors the exporter's treatment of filters and+/// speed, so what the preview mixes is what the export renders. A file+/// already in the cache is reused as is - that is the point of the cache.+fn decode(spec: &ClipSpec, project: &Path, key: &str) -> Result<Pcm, String> {+ if spec.chain.contains('\n') || spec.chain.contains('\r') {+ return Err("filter chain contains a line break".to_owned());+ }++ let directory = project.join("cache").join("audio");+ let destination = directory.join(format!("{key}.pcm"));+ if destination.is_file() {+ return Pcm::open(&destination);+ }+ std::fs::create_dir_all(&directory)+ .map_err(|error| format!("could not create {}: {error}", directory.display()))?;++ let mut filters: Vec<String> = Vec::new();+ if !spec.chain.is_empty() {+ filters.push(spec.chain.clone());+ }+ if (spec.speed - 1.0).abs() > 1e-9 {+ if spec.preserve_pitch {+ // atempo only accepts [0.5, 2]; factors outside compose.+ let mut factor = spec.speed.clamp(0.0625, 16.0);+ while factor > 2.0 {+ filters.push("atempo=2.0".to_owned());+ factor /= 2.0;+ }+ while factor < 0.5 {+ filters.push("atempo=0.5".to_owned());+ factor /= 0.5;+ }+ filters.push(format!("atempo={factor:.6}"));+ } else {+ // Tape behaviour: resample to the common rate, shift it, and land+ // back on the common rate. Pitch rides the speed.+ filters.push(format!(+ "aresample={PCM_RATE},asetrate={:.0},aresample={PCM_RATE}",+ f64::from(PCM_RATE) * spec.speed+ ));+ }+ }+ filters.push(format!("aresample={PCM_RATE}"));+ filters.push("aformat=sample_fmts=s16:channel_layouts=stereo".to_owned());++ // A sped-up clip reads further into the file than it occupies on the+ // timeline: the source window is `duration * speed`, exactly the window+ // the exporter trims.+ let window = spec.duration * spec.speed.clamp(0.0625, 16.0);++ let temporary = directory.join(format!("{key}.pcm.decoding"));+ let file = std::fs::File::create(&temporary)+ .map_err(|error| format!("could not create {}: {error}", temporary.display()))?;++ let status = std::process::Command::new(relay_media::ffmpeg())+ .args(["-hide_banner", "-nostdin", "-loglevel", "error"])+ .args(["-ss", &format!("{:.6}", spec.source_start)])+ .args(["-t", &format!("{window:.6}")])+ .args(["-i", &spec.path])+ .args(["-vn", "-af", &filters.join(",")])+ .args(["-f", "s16le", "-"])+ .stdout(file)+ .status()+ .map_err(|error| format!("could not run ffmpeg: {error}"))?;++ if !status.success() {+ let _ = std::fs::remove_file(&temporary);+ return Err(format!("ffmpeg exited with {status}"));+ }++ std::fs::rename(&temporary, &destination)+ .map_err(|error| format!("could not commit {}: {error}", destination.display()))?;+ Pcm::open(&destination)+}++/// Owns the output stream for the life of the app.+///+/// The stream's callback owns all mix state and drains the message channel+/// itself, so no lock is ever taken on the audio thread.+fn audio_thread(rx: mpsc::Receiver<Msg>, shared: Arc<Shared>) {+ use cpal::traits::{DeviceTrait, HostTrait};++ let host = cpal::default_host();+ let Some(device) = host.default_output_device() else {+ eprintln!("no audio output device; preview will be silent");+ return;+ };+ let Ok(config) = device.default_output_config() else {+ eprintln!("no default output config; preview will be silent");+ return;+ };++ let sample_format = config.sample_format();+ let config: cpal::StreamConfig = config.into();++ let result = match sample_format {+ cpal::SampleFormat::F32 => run_stream::<f32>(&device, &config, rx, shared),+ cpal::SampleFormat::I16 => run_stream::<i16>(&device, &config, rx, shared),+ cpal::SampleFormat::U16 => run_stream::<u16>(&device, &config, rx, shared),+ other => Err(format!("unsupported sample format {other:?}")),+ };+ if let Err(error) = result {+ eprintln!("audio output failed: {error}");+ }+}++fn run_stream<T>(+ device: &cpal::Device,+ config: &cpal::StreamConfig,+ rx: mpsc::Receiver<Msg>,+ shared: Arc<Shared>,+) -> Result<(), String>+where+ T: cpal::SizedSample + cpal::FromSample<f32>,+{+ use cpal::traits::{DeviceTrait, StreamTrait};++ let channels = config.channels as usize;+ let step = 1.0 / f64::from(config.sample_rate.0);++ let mut clips: Vec<ActiveClip> = Vec::new();+ let mut playing = false;+ let mut position = 0.0f64;++ let stream = device+ .build_output_stream(+ config,+ move |data: &mut [T], _| {+ while let Ok(message) = rx.try_recv() {+ match message {+ Msg::SetClips(next) => clips = next,+ Msg::Play(at) => {+ position = at;+ playing = true;+ }+ Msg::Pause => playing = false,+ Msg::Seek(at) => position = at,+ }+ }++ if !playing {+ data.fill(T::from_sample(0.0));+ return;+ }++ for frame in data.chunks_mut(channels) {+ let mut left = 0.0f32;+ let mut right = 0.0f32;++ for clip in &clips {+ let local = position - clip.start;+ if local < 0.0 || local >= clip.duration {+ continue;+ }+ let gain = gain_at(clip, local);+ if gain <= 0.0 {+ continue;+ }++ // Linear interpolation across the 48k source grid -+ // exact when the device runs at 48k, inaudibly close+ // when it does not.+ let read = local * f64::from(PCM_RATE);+ let index = read.floor() as u64;+ let fraction = (read - read.floor()) as f32;+ let pcm = &clip.pcm;+ left += gain+ * (pcm.sample(index, 0) * (1.0 - fraction)+ + pcm.sample(index + 1, 0) * fraction);+ right += gain+ * (pcm.sample(index, 1) * (1.0 - fraction)+ + pcm.sample(index + 1, 1) * fraction);+ }++ // Hard limit. A mix of boosted clips can exceed full+ // scale; wrapping would be far worse than flattening.+ frame[0] = T::from_sample(left.clamp(-1.0, 1.0));+ if channels > 1 {+ frame[1] = T::from_sample(right.clamp(-1.0, 1.0));+ }+ for extra in frame.iter_mut().skip(2) {+ *extra = T::from_sample(0.0);+ }++ position += step;+ }++ shared+ .position_micros+ .store((position * 1_000_000.0) as u64, Ordering::Relaxed);+ },+ |error| eprintln!("audio stream error: {error}"),+ None,+ )+ .map_err(|error| format!("could not build the output stream: {error}"))?;++ stream+ .play()+ .map_err(|error| format!("could not start the output stream: {error}"))?;++ // The stream lives exactly as long as this thread.+ loop {+ std::thread::sleep(std::time::Duration::from_secs(3600));+ }+}diff --git a/desktop/src-tauri/tauri.macos.conf.json b/desktop/src-tauri/tauri.macos.conf.jsonnew file mode 100644index 00000000..17f6198f--- /dev/null+++ b/desktop/src-tauri/tauri.macos.conf.json@@ -0,0 +1,20 @@+{+ "$schema": "https://schema.tauri.app/config/2",+ "app": {+ "windows": [+ {+ "title": "WolfCut",+ "width": 1440,+ "height": 900,+ "minWidth": 1024,+ "minHeight": 640,+ "resizable": true,+ "decorations": true,+ "titleBarStyle": "Overlay",+ "hiddenTitle": true,+ "backgroundColor": "#07080c",+ "dragDropEnabled": true+ }+ ]+ }+}diff --git a/desktop/src/App.tsx b/desktop/src/App.tsxindex 25e85b79..a6a36ebf 100644--- a/desktop/src/App.tsx+++ b/desktop/src/App.tsx@@ -1,5 +1,5 @@import { useCallback, useEffect, useMemo, useRef, useState } from "react";-import { isTauri } from "@tauri-apps/api/core";+import { invoke, isTauri } from "@tauri-apps/api/core";import { open as openDialog } from "@tauri-apps/plugin-dialog";import { getCurrentWebview } from "@tauri-apps/api/webview";import { getCurrentWindow } from "@tauri-apps/api/window";@@ -14,8 +14,7 @@ import { RightPanel, type RightTab } from "./components/RightPanel";import { StartScreen, type ProjectSession } from "./components/StartScreen";import { TitleBar } from "./components/TitleBar";import { TimelinePanel, resolveDrop, type Tool } from "./components/TimelinePanel";-import { createAssets, requestAssets } from "./lib/assets";-import { AudioPreview } from "./lib/audio";+import { createAssets, requestAssets, requestVideoPeaks } from "./lib/assets";import {engineVersion,loadProject,@@ -28,10 +27,10 @@ import {addClip,addMedia,addTrack,- clipGainAt,clipsAt,- clipsOnTrack,createProject,+ detachAudio,+ detachedAudioOf,findClip,firstFreeTrack,findMedia,@@ -42,6 +41,7 @@ import {moveClip,moveClips,projectDuration,+ reattachAudio,removeClips,removeFont,removeTrack,@@ -54,11 +54,10 @@ import {updateClip,trimClip,whyNotMerge,- type Clip,type MediaItem,type Project,} from "./lib/project";-import { buildChain, chainKey } from "./lib/filters";+import { buildChain } from "./lib/filters";import { familyForPath, registerFont } from "./lib/text";import { fromDocument, toDocument } from "./lib/persist";import { useTheme, type Theme } from "./lib/theme";@@ -71,7 +70,7 @@ const MAX_SECONDS_PER_PIXEL = 2;* The window: either the launch screen or an open editor.** The editor is a separate component rather than a branch inside one, so that- * every hook it owns - transport, audio, asset cache - mounts when a project+ * every hook it owns - transport, asset cache - mounts when a project* opens and unmounts when it closes. Closing a project therefore cannot leave* a stale audio element or a half-finished waveform behind.*/@@ -144,6 +143,11 @@ function Editor({const [saveState, setSaveState] = useState<"idle" | "saving" | "saved" | "failed">("idle");/** Blocks autosave until the project on disk has been read in. */const [loaded, setLoaded] = useState(false);+ /**+ * The output frame. Starts from the project settings but is editable from+ * the preview footer, and the edited value is what the document keeps.+ */+ const [frame, setFrame] = useState({ width: session.width, height: session.height });// Panel geometry.const [leftWidth, setLeftWidth] = useState(340);@@ -163,8 +167,8 @@ function Editor({const { playhead, playing } = transport;// Read by event listeners that outlive the render that installed them.- const latest = useRef({ project, secondsPerPixel, scrollLeft, trackScroll, snap, playhead });- latest.current = { project, secondsPerPixel, scrollLeft, trackScroll, snap, playhead };+ const latest = useRef({ project, secondsPerPixel, scrollLeft, trackScroll, snap, playhead, frame });+ latest.current = { project, secondsPerPixel, scrollLeft, trackScroll, snap, playhead, frame };// The project's rate is authoritative, not the first clip's. Timecode that// changes meaning when you import a 25fps file is worse than useless.@@ -177,6 +181,18 @@ function Editor({.then((document) => {const restored = fromDocument(document);if (!cancelled && restored) setProject(restored);+ // The document's frame wins over the manifest: it is where an edited+ // output size was saved.+ const video = (document as { video?: { width?: unknown; height?: unknown } }).video;+ if (+ !cancelled &&+ typeof video?.width === "number" &&+ typeof video?.height === "number" &&+ video.width > 0 &&+ video.height > 0+ ) {+ setFrame({ width: Math.round(video.width), height: Math.round(video.height) });+ }}).catch(() => undefined).finally(() => {@@ -283,21 +299,50 @@ function Editor({const save = useCallback(async () => {setSaveState("saving");try {- await saveProject(session.path, toDocument(session, latest.current.project));+ await saveProject(+ session.path,+ toDocument(+ { ...session, width: latest.current.frame.width, height: latest.current.frame.height },+ latest.current.project,+ ),+ );setSaveState("saved");+ return true;} catch (cause) {setSaveState("failed");setError(String(cause));+ return false;}}, [session]);+ // A deliberate save deserves an acknowledgement; the autosave stays silent+ // because a toast every 1.5 seconds of editing would be noise, not news.+ const [toast, setToast] = useState<{ id: number; message: string; failed: boolean } | null>(+ null,+ );+ const saveAndNotify = useCallback(() => {+ void save().then((saved) =>+ setToast({+ id: Date.now(),+ message: saved ? "Project saved" : "Save failed",+ failed: !saved,+ }),+ );+ }, [save]);++ useEffect(() => {+ if (!toast) return;+ const timer = window.setTimeout(() => setToast(null), 2200);+ return () => window.clearTimeout(timer);+ }, [toast]);+// Autosave, debounced. The edit is the thing people lose; making them// remember a keystroke to keep it is a design decision nobody wants.useEffect(() => {if (!loaded) return;const timer = window.setTimeout(() => void save(), 1500);return () => clearTimeout(timer);- }, [project, loaded, save]);+ }, [project, frame, loaded, save]);useEffect(() => {engineVersion()@@ -319,14 +364,32 @@ function Editor({setProject((current) => addMedia(current, item));setSelectedMediaId(item.id);// Fire and forget: the clip draws flat until the artwork lands.- requestAssets(assets.current, item);+ requestAssets(assets.current, item, session.path);} catch (cause) {// The engine's errors already name the file and quote FFmpeg.setError(String(cause));}}setBusy(false);- }, []);+ }, [session.path]);++ // Artwork for everything already in the project: a reopened project should+ // get its thumbnails and waveforms back without re-importing anything. The+ // disk cache makes the second launch cheap; requestAssets dedupes the rest.+ useEffect(() => {+ if (!loaded) return;+ // Audio clips cut from a video (detached, or split out) draw a waveform,+ // so those videos - and only those - need peaks decoded.+ const wantsPeaks = new Set(+ project.clips.filter((clip) => clip.kind === "audio").map((clip) => clip.mediaId),+ );+ for (const item of project.media) {+ requestAssets(assets.current, item, session.path);+ if (item.kind === "video" && wantsPeaks.has(item.id)) {+ requestVideoPeaks(assets.current, item, session.path);+ }+ }+ }, [loaded, project.media, project.clips, session.path]);// Files dropped from the OS. Tauri intercepts these before the webview sees// them, which is why this is an event subscription and not an HTML5 handler.@@ -349,47 +412,49 @@ function Editor({return () => stop?.();}, [importPaths]);- // ── audio preview ────────────────────────────────────────────────────────- const audio = useRef<AudioPreview | null>(null);- if (audio.current === null) audio.current = new AudioPreview();-- const [renderingClips, setRenderingClips] = useState<ReadonlySet<string>>(new Set());-- useEffect(() => {- const preview = audio.current;- if (preview) {- preview.onRenderChange = () => setRenderingClips(new Set(preview.pending()));- }- return () => preview?.dispose();- }, []);-- useEffect(() => {- const voices = clipsAt(project, playhead)- .filter((clip) => clip.kind === "audio")- .flatMap((clip) => {+ // ── audio playback ───────────────────────────────────────────────────────+ // The engine mixes everything audible - see src-tauri/src/playback.rs. The+ // UI's whole job is to describe the audible clip set whenever it changes:+ // audio clips, plus video clips that still own their sound. The engine+ // decodes what is new (cached across sessions) and remixes the rest.+ const audibleClips = useMemo(+ () =>+ project.clips.flatMap((clip) => {+ if (clip.kind !== "audio" && clip.kind !== "video") return [];+ if (clip.kind === "video" && clip.muted) return [];+ const track = findTrack(project, clip.trackId);+ if (!track || track.muted) return [];const media = findMedia(project, clip.mediaId);- if (!media) return [];+ if (!media || !media.hasAudio) return [];return [{- clipId: clip.id,path: media.path,- // Source advances at `speed`, so the same playhead maps further- // into the file on a sped-up clip.- time: clip.sourceStart + (playhead - clip.start) * clip.speed,+ start: clip.start,+ duration: clip.duration,+ sourceStart: clip.sourceStart,+ volume: clip.volume,+ fadeIn: clip.fadeIn,+ fadeOut: clip.fadeOut,speed: clip.speed,preservePitch: clip.preservePitch,- // A filtered clip plays a render of itself, produced by the same- // ffmpeg chain the exporter will use.- filter: chainFor(clip),- // Uncapped: the preview routes through a gain node, so what you- // hear matches what the exporter will render.- volume: clipGainAt(clip, playhead),+ chain: buildChain(clip.filters) ?? "",},];- });+ }),+ [project],+ );- audio.current?.sync(voices, playing);- }, [project, playhead, playing]);+ useEffect(() => {+ if (!isTauri()) return;+ // Debounced so a slider drag settles before its chain re-decodes; gain+ // and position changes are cheap remixes either way.+ const timer = window.setTimeout(() => {+ void invoke("audio_set_clips", { project: session.path, clips: audibleClips }).catch(+ (cause: unknown) => console.error("WolfCut: could not update the mix", cause),+ );+ }, 150);+ return () => window.clearTimeout(timer);+ }, [audibleClips, session.path]);/*** Every title live at the playhead, bottom track first.@@ -435,8 +500,6 @@ function Editor({clipId: top.id,path: media.path,time: top.sourceStart + (playhead - top.start),- muted: false,- volume: clipGainAt(top, playhead),isStill: top.kind === "image",};}, [project, playhead]);@@ -536,9 +599,6 @@ function Editor({const deleteSelected = useCallback(() => {if (selectedClipIds.length === 0) return;- // Free the audio element first: once the clip is gone the sync pass has- // nothing to match it against and it would keep sounding.- for (const clipId of selectedClipIds) audio.current?.release(clipId);setProject((current) => removeClips(current, selectedClipIds));setSelectedClipIds([]);}, [selectedClipIds]);@@ -590,7 +650,7 @@ function Editor({switch (event.code) {case "KeyS":event.preventDefault();- void save();+ saveAndNotify();break;case "KeyB":event.preventDefault();@@ -653,29 +713,7 @@ function Editor({window.addEventListener("keydown", onKey);return () => window.removeEventListener("keydown", onKey);- }, [deleteSelected, duration, frameRate, mergeSelected, save, splitAtPlayhead, transport]);-- // Filters are rendered per clip, keyed so that returning a slider to a value- // you already heard reuses the render instead of producing it again.- const chainFor = useCallback((clip: Clip) => {- const chain = buildChain(clip.filters);- if (!chain) return undefined;-- // A sped-up clip reads further into the file than it occupies on the- // timeline, so the render has to cover `duration * speed` of source - the- // same window the exporter trims. Rendering only `duration` left the- // preview seeking past the end of a short file, which is heard as the- // clip going silent partway through rather than as a filter problem.- // The window is part of the key, so changing speed re-renders.- const window = clip.duration * Math.min(8, Math.max(0.1, clip.speed));-- return {- key: chainKey(clip.mediaId, clip.sourceStart, window, clip.filters),- chain,- sourceStart: clip.sourceStart,- duration: window,- };- }, []);+ }, [deleteSelected, duration, frameRate, mergeSelected, saveAndNotify, splitAtPlayhead, transport]);// The exporter works from a flat list: the engine rebuilds a real timeline// from it, so track *order* has to survive the trip even though track@@ -700,7 +738,7 @@ function Editor({sourceStart: clip.sourceStart,track: index,hidden: !track.visible,- muted: track.muted,+ muted: track.muted || clip.muted === true,volume: clip.volume,fadeIn: clip.fadeIn,fadeOut: clip.fadeOut,@@ -766,7 +804,7 @@ function Editor({label: "Save",icon: "folder",hint: "Ctrl+S",- onSelect: () => void save(),+ onSelect: () => saveAndNotify(),},{label: "Export...",@@ -889,6 +927,8 @@ function Editor({playhead={playhead}duration={duration}frameRate={frameRate}+ frame={frame}+ onFrameChange={(width, height) => setFrame({ width, height })}onTogglePlay={transport.toggle}onStep={(frames) => transport.step(frames, frameRate)}onSeek={transport.seek}@@ -909,10 +949,13 @@ function Editor({clip={selectedClip}media={inspectorMedia}project={project}+ projectName={session.name}+ projectPath={session.path}+ frame={frame}+ duration={duration}frameRate={frameRate}onAddFont={() => void pickFont()}onRemoveFont={(family) => setProject((current) => removeFont(current, family))}- rendering={selectedClipIds.some((id) => renderingClips.has(id))}onChangeClip={(patch) => {if (selectedClipIds.length !== 1) return;setProject((current) => updateClip(current, selectedClipIds[0], patch));@@ -972,11 +1015,6 @@ function Editor({}onRemoveTrack={(trackId) =>setProject((current) => {- // Free the audio for anything about to disappear, or it keeps- // sounding with no clip left to match it against.- for (const clip of clipsOnTrack(current, trackId)) {- audio.current?.release(clip.id);- }setSelectedClipIds((ids) =>ids.filter((id) => findClip(current, id)?.trackId !== trackId),);@@ -1013,6 +1051,38 @@ function Editor({},]: []),+ // "Detach audio" and its reverse, offered only where they+ // mean something: one clip, with a link to make or unmake.+ ...(!many &&+ clip?.kind === "video" &&+ !clip.muted &&+ findMedia(project, clip.mediaId)?.hasAudio &&+ detachedAudioOf(project, clip.id).length === 0+ ? [+ {+ label: "Detach audio",+ icon: "waveform" as const,+ onSelect: () => setProject((current) => detachAudio(current, clipId)),+ },+ ]+ : []),+ ...(!many &&+ clip &&+ ((clip.kind === "video" && detachedAudioOf(project, clip.id).length > 0) ||+ (clip.kind === "audio" &&+ clip.detachedFrom &&+ findClip(project, clip.detachedFrom)))+ ? [+ {+ label: "Reattach audio",+ icon: "merge" as const,+ onSelect: () => {+ setProject((current) => reattachAudio(current, clipId));+ setSelectedClipIds([]);+ },+ },+ ]+ : []),{label: "Move to playhead",icon: "select",@@ -1030,7 +1100,6 @@ function Editor({hint: "Del",danger: true,onSelect: () => {- for (const id of target) audio.current?.release(id);setProject((current) => removeClips(current, target));setSelectedClipIds([]);},@@ -1044,12 +1113,28 @@ function Editor({{context && <ContextMenu target={context} onClose={() => setContext(null)} />}+ {toast && (+ <div+ key={toast.id}+ role="status"+ className={`surface fixed bottom-4 right-4 z-50 flex items-center gap-2 rounded-xl+ px-3 py-2 text-xs ${toast.failed ? "text-danger" : "text-primary"}`}+ >+ <Icon+ name={toast.failed ? "close" : "check"}+ size={13}+ className={toast.failed ? "" : "text-success"}+ />+ {toast.message}+ </div>+ )}+{exporting && (<ExportDialogprojectName={session.name}projectPath={session.path}- width={session.width}- height={session.height}+ width={frame.width}+ height={frame.height}rateNum={session.rateNum}rateDen={session.rateDen}duration={duration}diff --git a/desktop/src/assets/wolfcut-logo.png b/desktop/src/assets/wolfcut-logo.pngnew file mode 100644index 00000000..e740347aBinary files /dev/null and b/desktop/src/assets/wolfcut-logo.png differdiff --git a/desktop/src/components/FiltersPanel.tsx b/desktop/src/components/FiltersPanel.tsxindex afd2ab68..f6d397b1 100644--- a/desktop/src/components/FiltersPanel.tsx+++ b/desktop/src/components/FiltersPanel.tsx@@ -26,12 +26,9 @@ import { Empty } from "./Panel";*/export function FiltersPanel({clip,- rendering,onChange,}: {clip: Clip | null;- /** True while the preview is re-rendering this clip's audio. */- rendering: boolean;onChange: (filters: ClipFilter[]) => void;}) {const [category, setCategory] = useState<FilterCategory>("voice");@@ -76,11 +73,6 @@ export function FiltersPanel({<section className="mb-5"><h3 className="mb-2 flex items-center gap-2 text-[11px] font-semibold uppercase tracking-wider text-tertiary">Applied- {rendering && (- <span className="font-normal normal-case tracking-normal text-accent">- rendering...- </span>- )}</h3><ul className="flex flex-col gap-2">diff --git a/desktop/src/components/Inspector.tsx b/desktop/src/components/Inspector.tsxindex 08f172f8..26736b73 100644--- a/desktop/src/components/Inspector.tsx+++ b/desktop/src/components/Inspector.tsx@@ -1,13 +1,11 @@import type { ReactNode } from "react";-import type { Clip, MediaItem } from "../lib/project";+import type { Clip, MediaItem, Project } from "../lib/project";import { shortDuration, timecode } from "../lib/time";-import { Icon } from "./Icon";-import { Empty } from "./Panel";/*** Properties of the current selection - a clip if one is selected, otherwise- * the highlighted bin item.+ * the highlighted bin item, otherwise the project itself.** Read-only for now. When these become editable the edits go to the engine as* commands and come back as new state; the panel stays dumb. That is what@@ -17,17 +15,40 @@ export function Inspector({clip,media,frameRate,+ project,+ projectName,+ projectPath,+ frame,+ duration,}: {clip: Clip | null;media: MediaItem | null;frameRate: number;+ project: Project;+ projectName: string;+ projectPath: string;+ /** The output size, as currently set in the preview footer. */+ frame: { width: number; height: number };+ /** Timeline length in seconds. */+ duration: number;}) {return (<div>{!clip && !media ? (- <Empty icon={<Icon name="info" size={26} strokeWidth={1.5} />}>- Select a clip or a bin item.- </Empty>+ <div className="selectable px-3 py-2">+ <Section title="Project">+ <Row label="Name" value={projectName} />+ <Row label="Folder" value={projectPath} mono wrap />+ <Row label="Output" value={`${frame.width} x ${frame.height}`} mono />+ <Row label="Rate" value={`${frameRate.toFixed(2)} fps`} mono />+ <Row label="Duration" value={timecode(duration, frameRate)} mono />+ </Section>+ <Section title="Contents">+ <Row label="Media" value={`${project.media.length}`} mono />+ <Row label="Tracks" value={`${project.tracks.length}`} mono />+ <Row label="Clips" value={`${project.clips.length}`} mono />+ </Section>+ </div>) : (<div className="selectable px-3 py-2">{clip && (diff --git a/desktop/src/components/MediaBin.tsx b/desktop/src/components/MediaBin.tsxindex 8dfbd937..757a325e 100644--- a/desktop/src/components/MediaBin.tsx+++ b/desktop/src/components/MediaBin.tsx@@ -335,6 +335,7 @@ function MediaThumb({ item, assets }: { item: MediaItem; assets: MediaAssets })// most of it, and a crop reads better than bars at this size.const tile = strip.width / frames;const scale = Math.max(width / tile, height / strip.height);+ context.imageSmoothingQuality = "high";const drawWidth = tile * scale;const drawHeight = strip.height * scale;context.drawImage(diff --git a/desktop/src/components/Menu.tsx b/desktop/src/components/Menu.tsxindex 3aba2046..586650d9 100644--- a/desktop/src/components/Menu.tsx+++ b/desktop/src/components/Menu.tsx@@ -31,10 +31,13 @@ export function Menu({groups,trigger,align = "left",+ direction = "down",}: {groups: MenuOption[][];trigger: (open: boolean) => ReactNode;align?: "left" | "right";+ /** "up" hangs the menu above the trigger, for triggers near the bottom edge. */+ direction?: "down" | "up";}) {const [open, setOpen] = useState(false);const root = useRef<HTMLDivElement>(null);@@ -73,7 +76,8 @@ export function Menu({{open && (<divrole="menu"- className={`surface absolute z-50 mt-1.5 min-w-52 rounded-xl px-1.5 py-1.5 text-primary+ className={`surface absolute z-50 min-w-52 rounded-xl px-1.5 py-1.5 text-primary+ ${direction === "up" ? "bottom-full mb-1.5" : "mt-1.5"}${align === "right" ? "right-0" : "left-0"}`}>{groups.map((group, groupIndex) => (diff --git a/desktop/src/components/Preview.tsx b/desktop/src/components/Preview.tsxindex 31fa71df..e7501c58 100644--- a/desktop/src/components/Preview.tsx+++ b/desktop/src/components/Preview.tsx@@ -1,21 +1,42 @@import { useEffect, useLayoutEffect, useRef, useState } from "react";import { convertFileSrc } from "@tauri-apps/api/core";-import { connectElement } from "../lib/audio";import { timecode } from "../lib/time";import { textCss, type TextStyle } from "../lib/text";import { Icon, IconButton } from "./Icon";+import { Menu } from "./Menu";import { PANEL_SHELL } from "./Panel";+/**+ * Output frames people actually deliver to. Dimensions rather than bare+ * ratios, because a ratio alone does not say how many pixels the export has.+ */+const FRAME_PRESETS = [+ { label: "16:9", width: 1920, height: 1080 },+ { label: "16:9 · 4K", width: 3840, height: 2160 },+ { label: "9:16", width: 1080, height: 1920 },+ { label: "1:1", width: 1080, height: 1080 },+ { label: "4:3", width: 1440, height: 1080 },+ { label: "21:9", width: 2560, height: 1080 },+] as const;++/** "16:9" for a preset size, the reduced fraction for anything else. */+function ratioLabel(width: number, height: number): string {+ const preset = FRAME_PRESETS.find(+ (candidate) => candidate.width === width && candidate.height === height,+ );+ if (preset) return preset.label.split(" ")[0];+ const gcd = (a: number, b: number): number => (b === 0 ? a : gcd(b, a % b));+ const divisor = gcd(width, height) || 1;+ return `${width / divisor}:${height / divisor}`;+}+/** The video clip that should be on screen right now, if any. */export interface PreviewSource {clipId: string;path: string;/** Where in the source file the playhead sits, in seconds. */time: number;- muted: boolean;- /** Linear gain with fades applied. May exceed 1; see lib/audio.ts. */- volume: number;/** A still is shown as an image; there is nothing to seek or play. */isStill: boolean;}@@ -49,6 +70,8 @@ export function Preview({playhead,duration,frameRate,+ frame,+ onFrameChange,onTogglePlay,onStep,onSeek,@@ -60,6 +83,9 @@ export function Preview({playhead: number;duration: number;frameRate: number;+ /** The project's output size, shown and changed in the footer. */+ frame: { width: number; height: number };+ onFrameChange: (width: number, height: number) => void;onTogglePlay: () => void;onStep: (frames: number) => void;onSeek: (seconds: number) => void;@@ -118,17 +144,6 @@ export function Preview({const element = video.current;if (!element || !source || source.isStill) return;- element.muted = source.muted;-- // Through a gain node, not element.volume, which browsers cap at 1.0.- // A video clip's audio obeys the same clip gain as an audio clip's.- try {- connectElement(element).gain.value = Math.max(0, source.volume);- } catch {- // Routing can only fail if the element was already claimed by another- // graph, which cannot happen here - but silence is not worth a crash.- }-const tolerance = playing ? 0.3 : 0.03;if (element.readyState > 0 && Math.abs(element.currentTime - source.time) > tolerance) {try {@@ -162,8 +177,12 @@ export function Preview({*/}<div className="min-h-0 min-w-0 flex-1 overflow-hidden bg-stage p-4"><div ref={stage} data-preview-surface className="relative h-full w-full">+ {/* Picture only. Every drop of audio - a video clip's included -+ comes from the engine's mixer, so there is exactly one clock and+ one gain law. */}<videoref={video}+ mutedplaysInlineclassName={`absolute inset-0 h-full w-full object-contain ${source && !source.isStill ? "block" : "hidden"@@ -275,8 +294,38 @@ export function Preview({/></div>- <span className="min-w-0 truncate justify-self-end font-technical text-[10px] text-tertiary">- {frameRate.toFixed(2)} fps+ <span className="flex min-w-0 items-center gap-2 justify-self-end">+ <Menu+ align="right"+ direction="up"+ groups={[+ FRAME_PRESETS.map((preset) => ({+ label: `${preset.label} — ${preset.width} x ${preset.height}`,+ icon:+ preset.width === frame.width && preset.height === frame.height+ ? ("check" as const)+ : undefined,+ onSelect: () => onFrameChange(preset.width, preset.height),+ })),+ ]}+ trigger={(open) => (+ <span+ title="Output size"+ className={`flex items-center gap-1 rounded-md px-1.5 py-0.5 font-technical text-[10px]+ transition-colors ${+ open+ ? "bg-active text-primary"+ : "text-tertiary hover:bg-hover hover:text-secondary"+ }`}+ >+ {ratioLabel(frame.width, frame.height)} · {frame.width}x{frame.height}+ <Icon name="chevronDown" size={10} />+ </span>+ )}+ />+ <span className="truncate font-technical text-[10px] text-tertiary">+ {frameRate.toFixed(2)} fps+ </span></span></div></div>diff --git a/desktop/src/components/RightPanel.tsx b/desktop/src/components/RightPanel.tsxindex 081a2e44..131fe4fd 100644--- a/desktop/src/components/RightPanel.tsx+++ b/desktop/src/components/RightPanel.tsx@@ -8,24 +8,25 @@ import { TextPanel } from "./TextPanel";export type RightTab = "details" | "adjust" | "filters" | "text";-const TABS: { id: RightTab; label: string }[] = [- { id: "details", label: "Details" },- { id: "adjust", label: "Adjust" },- { id: "filters", label: "Filters" },-];-/**- * Text replaces Adjust and Filters when a title is selected.+ * The tab strip follows the selection.*- * A text clip has no volume, no speed and no audio filters, so offering those- * tabs would be offering three panels that can only say "nothing here". The- * tab strip therefore depends on what is selected rather than being fixed.+ * Nothing selected: only Details, showing the project itself - there is+ * nothing to edit, so no editing tabs. A media clip selected: Adjust and+ * Filters, the things that can actually be changed. A title: just Text,+ * because a text clip has no volume, no speed and no audio filters, and+ * offering those tabs would be offering panels that can only say "nothing+ * here".*/-const TEXT_TABS: { id: RightTab; label: string }[] = [- { id: "details", label: "Details" },- { id: "text", label: "Text" },+const DETAILS_TABS: { id: RightTab; label: string }[] = [{ id: "details", label: "Details" }];++const CLIP_TABS: { id: RightTab; label: string }[] = [+ { id: "adjust", label: "Adjust" },+ { id: "filters", label: "Filters" },];+const TEXT_TABS: { id: RightTab; label: string }[] = [{ id: "text", label: "Text" }];+/*** The right-hand panel.*@@ -40,33 +41,37 @@ export function RightPanel({clip,media,project,+ projectName,+ projectPath,+ frame,+ duration,frameRate,onChangeClip,onSpeedChange,onAddFont,onRemoveFont,- rendering,}: {tab: RightTab;onTab: (tab: RightTab) => void;clip: Clip | null;media: MediaItem | null;project: Project;+ projectName: string;+ projectPath: string;+ frame: { width: number; height: number };+ duration: number;frameRate: number;onChangeClip: (patch: Partial<Clip>) => void;onSpeedChange: (speed: number) => void;onAddFont: () => void;onRemoveFont: (family: string) => void;- /** True while the selected clip's filtered audio is being rendered. */- rendering: boolean;}) {const isText = clip?.kind === "text";- const tabs = isText ? TEXT_TABS : TABS;+ const tabs = clip ? (isText ? TEXT_TABS : CLIP_TABS) : DETAILS_TABS;- // Selecting a title while sitting on Filters would otherwise leave the strip- // with nothing highlighted and the panel showing a tab that is no longer- // offered, so the selection falls back to one that exists.- const active = tabs.some((entry) => entry.id === tab) ? tab : "details";+ // A selection change can leave the strip showing a tab that is no longer+ // offered, so the active tab falls back to the first one that exists.+ const active = tabs.some((entry) => entry.id === tab) ? tab : tabs[0].id;return (<Panel>@@ -90,7 +95,18 @@ export function RightPanel({</div></div>- {active === "details" && <Inspector clip={clip} media={media} frameRate={frameRate} />}+ {active === "details" && (+ <Inspector+ clip={clip}+ media={media}+ frameRate={frameRate}+ project={project}+ projectName={projectName}+ projectPath={projectPath}+ frame={frame}+ duration={duration}+ />+ )}{active === "text" && (<TextPanelclip={clip}@@ -106,7 +122,6 @@ export function RightPanel({{active === "filters" && (<FiltersPanelclip={clip}- rendering={rendering}onChange={(filters: ClipFilter[]) => onChangeClip({ filters })}/>)}diff --git a/desktop/src/components/TimelinePanel.tsx b/desktop/src/components/TimelinePanel.tsxindex a790c708..c2e0f3c4 100644--- a/desktop/src/components/TimelinePanel.tsx+++ b/desktop/src/components/TimelinePanel.tsx@@ -1272,6 +1272,8 @@ function drawFilmstrip(const drawWidth = height * (tileWidth / strip.height);if (!Number.isFinite(drawWidth) || drawWidth <= 0) return;+ context.imageSmoothingQuality = "high";+const columns = Math.ceil(width / drawWidth);for (let column = 0; column < columns; column += 1) {const fraction = columns > 1 ? column / (columns - 1) : 0;diff --git a/desktop/src/components/TitleBar.tsx b/desktop/src/components/TitleBar.tsxindex ef9ba5b5..7756187d 100644--- a/desktop/src/components/TitleBar.tsx+++ b/desktop/src/components/TitleBar.tsx@@ -3,6 +3,7 @@ import type { ReactNode } from "react";import { isTauri } from "@tauri-apps/api/core";import { getCurrentWindow } from "@tauri-apps/api/window";+import logo from "../assets/wolfcut-logo.png";import type { Theme } from "../lib/theme";import { Icon } from "./Icon";import { Menu, type MenuOption } from "./Menu";@@ -10,10 +11,16 @@ import { Menu, type MenuOption } from "./Menu";/*** The custom title bar.*- * The window is undecorated (`decorations: false`), so this strip *is* the- * title bar: it carries the app menu, the project name and the window+ * On Windows the window is undecorated (`decorations: false`), so this strip+ * *is* the title bar: it carries the app menu, the project name and the window* controls, and everything not otherwise interactive is a drag region.*+ * On macOS the native traffic lights are overlaid instead+ * (`tauri.macos.conf.json` sets `titleBarStyle: "Overlay"`), because+ * Windows-style controls on a Mac read as "ported, carelessly". The strip+ * keeps everything else but drops its own window buttons and leaves room on+ * the left for the traffic lights.+ ** `data-tauri-drag-region` is what makes an area draggable, and Tauri also* gives it double-click-to-maximise for free. It has to be on the actual* element under the pointer - a parent with the attribute does not delegate -@@ -40,6 +47,7 @@ export function TitleBar({}) {const [maximized, setMaximized] = useState(false);const native = isTauri();+ const macOS = navigator.userAgent.includes("Mac");useEffect(() => {if (!native) return;@@ -69,12 +77,12 @@ export function TitleBar({return (<headerdata-tauri-drag-region- className="flex h-8 shrink-0 items-center gap-1 border-b border-hairline bg-panel pl-2"+ className={`flex h-8 shrink-0 items-center gap-1 border-b border-hairline bg-panel ${+ macOS ? "pl-20 pr-1" : "pl-2"+ }`}><span className="flex items-center gap-1.5 pr-1" data-tauri-drag-region>- <span className="flex h-4 w-4 items-center justify-center rounded bg-accent text-on-accent">- <Icon name="film" size={11} strokeWidth={2.5} />- </span>+ <img src={logo} alt="" className="pointer-events-none h-4 w-4" draggable={false} /><span className="font-display text-[13px] font-bold tracking-tight text-primary">WolfCut</span></span>@@ -121,7 +129,7 @@ export function TitleBar({{actions && <div className="flex items-center gap-1 pr-2">{actions}</div>}- {native && (+ {native && !macOS && (<div className="flex h-full items-stretch"><WindowButton label="Minimise" onClick={() => void control("minimize")}><Icon name="winMinimize" size={14} strokeWidth={1.5} />diff --git a/desktop/src/lib/assets.ts b/desktop/src/lib/assets.tsindex dc144dcf..22380db6 100644--- a/desktop/src/lib/assets.ts+++ b/desktop/src/lib/assets.ts@@ -71,7 +71,13 @@ const BUCKETS_PER_SECOND = 200;/** Frames per filmstrip. Enough to read the shot, few enough to stay cheap. */const STRIP_FRAMES = 24;-const STRIP_HEIGHT = 72;+/**+ * Strip frames are drawn at ~32-72 CSS pixels tall, so they need the+ * display's real pixel density or a Retina screen upscales them into mush.+ * Capped at the engine's 240 limit; the ratio is read once because a strip+ * cached at one density is not worth regenerating on a monitor change.+ */+const STRIP_HEIGHT = Math.min(240, Math.round(72 * (window.devicePixelRatio || 1)));/*** Starts producing artwork for one media item if it is not already cached.@@ -86,19 +92,34 @@ export function requestAssets(path: string;kind: "video" | "audio" | "image";duration: number | null;+ hasAudio: boolean;},+ /** Project folder holding the on-disk artwork cache; null skips caching. */+ projectPath: string | null = null,): void {if (assets.pending.has(media.id)) return;- if (assets.peaks.has(media.id) || assets.strips.has(media.id)) return;++ // Video peaks are deliberately NOT loaded here. Decoding a video's audio+ // means reading the whole file and crunching every sample, which stalls the+ // main thread hard enough to make playback stutter - so it happens only on+ // demand, via requestVideoPeaks, when a detached audio clip needs drawing.+ const wantsPeaks = media.kind === "audio" && !assets.peaks.has(media.id);+ const wantsStrip = media.kind !== "audio" && !assets.strips.has(media.id);+ if (!wantsPeaks && !wantsStrip) return;assets.pending.add(media.id);- const work =- media.kind === "audio"- ? loadPeaks(media.path).then((peaks) => {- if (peaks) assets.peaks.set(media.id, peaks);- })- : media.kind === "image"+ const jobs: Promise<unknown>[] = [];+ if (wantsPeaks) {+ jobs.push(+ loadPeaks(media.path, projectPath).then((peaks) => {+ if (peaks) assets.peaks.set(media.id, peaks);+ }),+ );+ }+ if (wantsStrip) {+ jobs.push(+ media.kind === "image"? // A still is a one-frame filmstrip. Storing it that way means the// timeline and the bin draw it with the code they already have, and// a long still clip tiles the same frame instead of stretching it.@@ -108,21 +129,127 @@ export function requestAssets(assets.stripFrames.set(media.id, 1);}})- : loadStrip(media.path, media.duration).then((strip) => {+ : loadStrip(media.path, media.duration, projectPath).then((strip) => {if (strip) {assets.strips.set(media.id, strip);assets.stripFrames.set(media.id, STRIP_FRAMES);}- });+ }),+ );+ }- void work.finally(() => {+ void Promise.allSettled(jobs).then(() => {assets.pending.delete(media.id);announce(assets);});}+/**+ * Waveform for a video's audio track, on demand.+ *+ * Called when a detached audio clip exists (or is being created) for a video,+ * which is the only time a video needs peaks. Expensive on first run - the+ * whole file is read and decoded - but the result lands in the disk cache, so+ * a reopened project pays only a file read.+ */+export function requestVideoPeaks(+ assets: MediaAssets,+ media: { id: string; path: string; kind: "video" | "audio" | "image"; hasAudio: boolean },+ projectPath: string | null = null,+): void {+ if (media.kind !== "video" || !media.hasAudio) return;+ if (assets.peaks.has(media.id)) return;++ const pendingKey = `${media.id}:peaks`;+ if (assets.pending.has(pendingKey)) return;+ assets.pending.add(pendingKey);++ void loadPeaks(media.path, projectPath)+ .then((peaks) => {+ if (peaks) assets.peaks.set(media.id, peaks);+ })+ .finally(() => {+ assets.pending.delete(pendingKey);+ announce(assets);+ });+}++/**+ * The on-disk artwork cache, kept in the project's folder.+ *+ * Strips and peaks survive a relaunch there, which is the difference between+ * a reopened project showing its artwork immediately and every launch paying+ * for ffmpeg runs and full audio decodes again. Reads and writes both fail+ * soft: a miss or a failed write only means generating now, caching later.+ */+function artworkKey(path: string): string {+ // FNV-1a over the absolute path. Collisions are astronomically unlikely at+ // bin sizes, and a collision's worst case is a wrong thumbnail.+ let hash = 0x811c9dc5;+ for (let index = 0; index < path.length; index += 1) {+ hash ^= path.charCodeAt(index);+ hash = Math.imul(hash, 0x01000193) >>> 0;+ }+ return hash.toString(16).padStart(8, "0");+}++async function readArtwork(project: string | null, key: string): Promise<ArrayBuffer | null> {+ if (!project) return null;+ try {+ return await invoke<ArrayBuffer>("read_artwork", { project, key });+ } catch {+ return null;+ }+}++function writeArtwork(project: string | null, key: string, bytes: Uint8Array): void {+ if (!project) return;+ // Serialising the payload for IPC is main-thread work, and nothing about a+ // cache write is urgent - it waits for an idle moment so it never competes+ // with playback.+ const send = () =>+ void invoke("write_artwork", { project, key, bytes: Array.from(bytes) }).catch(+ () => undefined,+ );+ if (typeof window.requestIdleCallback === "function") window.requestIdleCallback(() => send());+ else window.setTimeout(send, 500);+}++/** [bucketsPerSecond f32][count u32][min f32...][max f32...], little-endian. */+function encodePeaks(peaks: Peaks): Uint8Array {+ const count = peaks.min.length;+ const buffer = new ArrayBuffer(8 + count * 8);+ const view = new DataView(buffer);+ view.setFloat32(0, peaks.bucketsPerSecond, true);+ view.setUint32(4, count, true);+ new Float32Array(buffer, 8, count).set(peaks.min);+ new Float32Array(buffer, 8 + count * 4, count).set(peaks.max);+ return new Uint8Array(buffer);+}++function decodePeaks(bytes: ArrayBuffer): Peaks | null {+ if (bytes.byteLength < 8) return null;+ const view = new DataView(bytes);+ const bucketsPerSecond = view.getFloat32(0, true);+ const count = view.getUint32(4, true);+ if (!Number.isFinite(bucketsPerSecond) || bucketsPerSecond <= 0) return null;+ if (bytes.byteLength !== 8 + count * 8) return null;+ return {+ min: new Float32Array(bytes, 8, count),+ max: new Float32Array(bytes, 8 + count * 4, count),+ bucketsPerSecond,+ };+}+/** Decodes a file and reduces it to min/max pairs. */-async function loadPeaks(path: string): Promise<Peaks | null> {+async function loadPeaks(path: string, project: string | null): Promise<Peaks | null> {+ const key = `${artworkKey(path)}-b${BUCKETS_PER_SECOND}.peaks`;+ const cached = await readArtwork(project, key);+ if (cached) {+ const peaks = decodePeaks(cached);+ if (peaks) return peaks;+ }+try {const bytes = await readMediaBytes(path);@@ -153,7 +280,9 @@ async function loadPeaks(path: string): Promise<Peaks | null> {max[bucket] = high;}- return { min, max, bucketsPerSecond: buffer.sampleRate / bucketSize };+ const peaks = { min, max, bucketsPerSecond: buffer.sampleRate / bucketSize };+ writeArtwork(project, key, encodePeaks(peaks));+ return peaks;} catch (cause) {console.warn(`WolfCut: no waveform for ${path}`, cause);return null;@@ -172,15 +301,32 @@ async function loadImage(path: string): Promise<ImageBitmap | null> {}/** Asks the engine for a strip of frames and decodes it into an ImageBitmap. */-async function loadStrip(path: string, duration: number | null): Promise<ImageBitmap | null> {+async function loadStrip(+ path: string,+ duration: number | null,+ project: string | null,+): Promise<ImageBitmap | null> {if (!duration || duration <= 0) return null;+ // The key carries the strip's shape, so a density or frame-count change+ // regenerates rather than serving yesterday's resolution.+ const key = `${artworkKey(path)}-f${STRIP_FRAMES}-h${STRIP_HEIGHT}.strip.jpg`;+ const cached = await readArtwork(project, key);+ if (cached) {+ try {+ return await createImageBitmap(new Blob([cached], { type: "image/jpeg" }));+ } catch {+ // A corrupt cache entry falls through to regeneration.+ }+ }+try {const bytes = await invoke<ArrayBuffer>("extract_filmstrip", {path,count: STRIP_FRAMES,height: STRIP_HEIGHT,});+ writeArtwork(project, key, new Uint8Array(bytes));// createImageBitmap decodes off the main thread, so a slow JPEG does not// stall the timeline.return await createImageBitmap(new Blob([bytes], { type: "image/jpeg" }));diff --git a/desktop/src/lib/audio.ts b/desktop/src/lib/audio.tsdeleted file mode 100644index b395b8c5..00000000--- a/desktop/src/lib/audio.ts+++ /dev/null@@ -1,337 +0,0 @@-/**- * Audio preview.- *- * Each audible clip gets a media element for decoding, routed through a Web- * Audio `GainNode` for level. The gain node is the whole reason this is not- * just `element.volume`: that property is hard-capped at 1.0 by every browser,- * so a boost could not be previewed at all - it could only be promised and- * applied later at export, which is a useless thing to tell someone mixing.- * A `GainNode` has no ceiling, so what you hear is what gets rendered.- *- * Audio is loaded as a blob rather than through the asset protocol, and that- * is deliberate too. `createMediaElementSource` on a cross-origin element- * yields *silence* with no error - the element plays, the graph outputs- * nothing. A blob URL is same-origin, so the graph is guaranteed to work. The- * cost is holding the file in memory, which we already do for the waveform.- *- * The known limits, so nobody mistakes this for a finished mixer:- * - Sync is corrective, not sample-accurate. Drift past a tolerance is- * fixed by reseeking, which is audible when it happens.- * - No effects and no automation beyond the clip's constant gain and fades.- * - Scrubbing repositions audio, it does not scrub it.- *- * See docs/decisions/0004-audio-preview-in-the-webview.md.- */--import { readMediaBytes, renderFilteredAudio } from "./engine";--/** One audio clip that should be sounding right now. */-export interface Voice {- clipId: string;- path: string;- /** Where in the source file the playhead currently sits, in seconds. */- time: number;- /** Linear gain, fades already applied. May exceed 1. */- volume: number;- /** Playback rate. */- speed: number;- /** False lets pitch rise with speed, like tape. */- preservePitch: boolean;- /**- * A filtered render of this clip, when it has filters.- *- * The rendered audio begins at the clip's in-point rather than the file's,- * so a position inside it is offset by `sourceStart`.- */- filter?: {- /** Identity of this exact render. A change means re-render. */- key: string;- chain: string;- sourceStart: number;- duration: number;- };-}--/**- * How far out of step a voice may drift before it is reseeked.- *- * Generous while playing: a reseek is an audible click, and browsers do not- * keep a media element perfectly locked to an external clock anyway. Tight- * while paused, where "seek" is the entire operation.- */-const TOLERANCE_PLAYING = 0.3;-const TOLERANCE_PAUSED = 0.02;--/**- * How long the filter settings must hold still before re-rendering.- *- * Without this, every step of a slider drag spawned an FFmpeg process and tore- * down the audio element mid-playback - dozens of renders for one gesture, and- * audio that cut out while you were trying to hear what you were adjusting.- */-const RENDER_DEBOUNCE_MS = 400;--let sharedContext: AudioContext | null = null;--/** The one AudioContext. Browsers cap how many a page may create. */-export function audioContext(): AudioContext {- sharedContext ??= new AudioContext();- return sharedContext;-}--/**- * A context created before any user interaction starts suspended. Resuming is- * safe to call repeatedly and only does anything the first time.- */-function wake(): void {- const context = audioContext();- if (context.state === "suspended") void context.resume().catch(() => undefined);-}--// One source node per element, forever: calling createMediaElementSource twice-// on the same element throws, and there is no way to undo the first call.-const routed = new WeakMap<HTMLMediaElement, GainNode>();--/**- * Routes a media element through a gain node and returns it.- *- * Shared with the video preview, so picture and sound obey the same clip gain- * through the same graph.- */-export function connectElement(element: HTMLMediaElement): GainNode {- const existing = routed.get(element);- if (existing) return existing;-- const context = audioContext();- const gain = context.createGain();- context.createMediaElementSource(element).connect(gain);- gain.connect(context.destination);- routed.set(element, gain);- return gain;-}--interface Loaded {- element: HTMLAudioElement;- gain: GainNode;- objectUrl?: string;- loading: boolean;- /** Which render this element holds; undefined means the unfiltered file. */- key?: string;- /** How far into the file this element's audio begins. */- offset: number;-}--export class AudioPreview {- private voices = new Map<string, Loaded>();- private rendering = new Set<string>();- /** Pending re-renders, keyed by clip. */- private timers = new Map<string, number>();- /** Which key each pending timer is waiting to render. */- private scheduled = new Map<string, string | undefined>();- /** Notified when a render finishes, so the UI can drop its spinner. */- onRenderChange: (() => void) | null = null;-- /** Clips whose filtered audio is still rendering. */- pending(): ReadonlySet<string> {- return this.rendering;- }-- /**- * Brings the audio output in line with the transport.- *- * Idempotent - calling it every animation frame is fine, and is in fact how- * drift gets corrected.- */- sync(active: Voice[], playing: boolean): void {- if (playing) wake();-- const wanted = new Set(active.map((voice) => voice.clipId));- for (const [clipId, loaded] of this.voices) {- if (!wanted.has(clipId) && !loaded.element.paused) loaded.element.pause();- }-- const tolerance = playing ? TOLERANCE_PLAYING : TOLERANCE_PAUSED;-- for (const voice of active) {- const loaded = this.load(voice);- const element = loaded.element;-- // No clamp. A boost is meant to be audible.- loaded.gain.gain.value = Math.max(0, voice.volume);-- // `preservesPitch` is exactly the toggle: on, the browser time-stretches- // and the voice stays put; off, it resamples and pitch rides the rate.- element.playbackRate = Math.max(0.0625, Math.min(16, voice.speed));- element.preservesPitch = voice.preservePitch;-- // A filtered render starts at the clip's in-point, so the same instant- // sits at a different offset inside it than in the original file.- const target = Math.max(0, voice.time - loaded.offset);-- // readyState 0 means metadata has not arrived; setting currentTime then- // throws in some engines and is discarded in others.-- if (element.readyState > 0 && Math.abs(element.currentTime - target) > tolerance) {- try {- element.currentTime = target;- } catch {- // Still opening; the next sync lands it.- }- }-- if (playing) {- if (element.paused) void element.play().catch(() => undefined);- } else if (!element.paused) {- element.pause();- }- }- }-- stopAll(): void {- for (const loaded of this.voices.values()) loaded.element.pause();- }-- /** Drops a voice, for instance when its clip is deleted. */- release(clipId: string): void {- const timer = this.timers.get(clipId);- if (timer !== undefined) {- clearTimeout(timer);- this.timers.delete(clipId);- this.scheduled.delete(clipId);- }-- const loaded = this.voices.get(clipId);- if (!loaded) return;- this.voices.delete(clipId);- this.destroy(loaded);-- if (this.rendering.delete(clipId)) this.onRenderChange?.();- }-- /** Tears down one element and its graph node. */- private destroy(loaded: Loaded): void {- loaded.element.pause();- loaded.element.removeAttribute("src");- loaded.element.load();- loaded.gain.disconnect();- if (loaded.objectUrl) URL.revokeObjectURL(loaded.objectUrl);- }-- dispose(): void {- for (const timer of this.timers.values()) clearTimeout(timer);- this.timers.clear();- this.scheduled.clear();- for (const clipId of [...this.voices.keys()]) this.release(clipId);- }-- private load(voice: Voice): Loaded {- const key = voice.filter?.key;- const existing = this.voices.get(voice.clipId);-- if (existing && existing.key === key) return existing;-- // The settings changed. Do *not* tear down what is currently audible:- // schedule the swap and keep playing the old render until the new one is- // ready, so adjusting a filter does not silence the thing you are judging.- if (existing) {- this.schedule(voice);- return existing;- }-- return this.create(voice);- }-- /** Queues a re-render, restarting the clock on every further change. */- private schedule(voice: Voice): void {- const key = voice.filter?.key;-- // `has` rather than a value comparison: removing the last filter makes the- // wanted key `undefined`, and a missing entry also reads as `undefined`.- // Comparing values there comes out equal, so the swap was never scheduled- // and the clip kept playing its filtered render for good.- if (this.timers.has(voice.clipId) && this.scheduled.get(voice.clipId) === key) return;-- const timer = this.timers.get(voice.clipId);- if (timer !== undefined) clearTimeout(timer);-- this.scheduled.set(voice.clipId, key);- this.timers.set(- voice.clipId,- window.setTimeout(() => {- this.timers.delete(voice.clipId);- this.scheduled.delete(voice.clipId);-- const previous = this.voices.get(voice.clipId);- this.voices.delete(voice.clipId);- this.create(voice);- if (previous) this.destroy(previous);- }, RENDER_DEBOUNCE_MS),- );- }-- private create(voice: Voice): Loaded {- const key = voice.filter?.key;- const element = new Audio();- element.preload = "auto";-- const loaded: Loaded = {- element,- gain: connectElement(element),- loading: true,- key,- offset: voice.filter ? voice.filter.sourceStart : 0,- };- this.voices.set(voice.clipId, loaded);-- const source = voice.filter- ? renderFilteredAudio({- path: voice.path,- sourceStart: voice.filter.sourceStart,- duration: voice.filter.duration,- chain: voice.filter.chain,- })- : readMediaBytes(voice.path);-- if (voice.filter) {- this.rendering.add(voice.clipId);- this.onRenderChange?.();- }-- // Source arrives asynchronously; until then the element is silent and the- // sync pass simply has nothing to play, which is correct.- void source- // A filter chain that FFmpeg rejects would otherwise leave this element- // with no source at all - permanently silent, and never retried because- // its key already matches. Falling back to the unfiltered file keeps the- // clip audible; the filters are simply not heard, which is a far better- // failure than a clip that has gone quiet for no visible reason.- .catch((cause) => {- if (!voice.filter) throw cause;- console.error(`WolfCut: filter render failed for ${voice.path}; playing it dry`, cause);- loaded.offset = 0;- return readMediaBytes(voice.path);- })- .then((bytes) => {- const url = URL.createObjectURL(new Blob([bytes]));-- // A render that finishes after its clip was replaced or removed has- // nothing to play into; hand the URL straight back rather than- // leaking it into a detached element.- if (this.voices.get(voice.clipId) !== loaded) {- URL.revokeObjectURL(url);- return;- }-- loaded.objectUrl = url;- loaded.element.src = url;- loaded.element.load();- })- .catch((cause) => console.error(`WolfCut: could not load audio for ${voice.path}`, cause))- .finally(() => {- loaded.loading = false;- if (this.rendering.delete(voice.clipId)) this.onRenderChange?.();- });-- return loaded;- }-}diff --git a/desktop/src/lib/engine.ts b/desktop/src/lib/engine.tsindex 631e7e20..fdb25677 100644--- a/desktop/src/lib/engine.ts+++ b/desktop/src/lib/engine.ts@@ -60,20 +60,6 @@ export async function readMediaBytes(path: string): Promise<ArrayBuffer> {return invoke<ArrayBuffer>("read_media_bytes", { path });}-/**- * Renders a clip's audio through an FFmpeg filter chain, returning WAV bytes.- *- * The same chain string the exporter uses, so the preview cannot drift from- * the rendered file.- */-export async function renderFilteredAudio(request: {- path: string;- sourceStart: number;- duration: number;- chain: string;-}): Promise<ArrayBuffer> {- return invoke<ArrayBuffer>("render_filtered_audio", request);-}/** A project on disk, as the launch screen sees it. */export interface ProjectInfo {diff --git a/desktop/src/lib/persist.ts b/desktop/src/lib/persist.tsindex 68b5bfba..45ae5bf6 100644--- a/desktop/src/lib/persist.ts+++ b/desktop/src/lib/persist.ts@@ -157,6 +157,8 @@ export function fromDocument(raw: unknown): Project | null {offsetY: number(clip.offsetY, 0),speed: Math.max(0.1, number(clip.speed, 1)),preservePitch: flag(clip.preservePitch, true),+ muted: flag(clip.muted, false) || undefined,+ detachedFrom: typeof clip.detachedFrom === "string" ? clip.detachedFrom : undefined,filters: Array.isArray(clip.filters)? clip.filters.flatMap((filter) => {if (typeof filter !== "object" || filter === null) return [];diff --git a/desktop/src/lib/project.ts b/desktop/src/lib/project.tsindex 5ffb8abf..b548ed17 100644--- a/desktop/src/lib/project.ts+++ b/desktop/src/lib/project.ts@@ -147,6 +147,18 @@ export interface Clip {*/filters: ClipFilter[];+ /**+ * True when this video clip's embedded audio is left out of preview and+ * export - the state "Detach audio" puts it in. Meaningless on other kinds.+ */+ muted?: boolean;++ /**+ * On a detached audio clip: the id of the video clip the sound came from.+ * The link is what makes "Reattach audio" possible from either side.+ */+ detachedFrom?: string;+/*** The overlay, when this is a text clip. Absent on every other kind.*@@ -475,6 +487,95 @@ export function removeClips(project: Project, clipIds: readonly string[]): Projereturn { ...project, clips: project.clips.filter((clip) => !doomed.has(clip.id)) };}+/** The audio clips holding sound detached from this video clip, if any. */+export function detachedAudioOf(project: Project, videoClipId: string): Clip[] {+ return project.clips.filter((clip) => clip.detachedFrom === videoClipId);+}++/**+ * Splits a video clip's sound onto its own audio clip - "Detach audio".+ *+ * The video clip goes quiet (`muted`) and a linked audio clip appears with the+ * same span of the same file, taking the audio filters with it - they were+ * always about the sound. The link (`detachedFrom`) is what reattachment+ * follows later. Does nothing for a clip that has no audio to give.+ */+export function detachAudio(project: Project, clipId: string): Project {+ const clip = findClip(project, clipId);+ const media = clip ? findMedia(project, clip.mediaId) : null;+ if (!clip || clip.kind !== "video" || clip.muted || !media?.hasAudio) return project;+ if (detachedAudioOf(project, clipId).length > 0) return project;++ // A lane free for the whole span, or a fresh one - never on top of a clip.+ const end = clip.start + clip.duration;+ const free = project.tracks.find(+ (track) =>+ !project.clips.some(+ (other) =>+ other.trackId === track.id &&+ other.start < end &&+ clip.start < other.start + other.duration,+ ),+ );+ let base = project;+ let trackId = free?.id ?? "";+ if (!trackId) {+ const grown = addTrack(project);+ base = grown.project;+ trackId = grown.trackId;+ }++ const sound: Clip = {+ ...clip,+ id: nextId("c"),+ trackId,+ kind: "audio",+ filters: [...clip.filters],+ detachedFrom: clip.id,+ muted: undefined,+ };++ return {+ ...base,+ clips: [+ ...base.clips.map((other) =>+ other.id === clipId ? { ...other, muted: true, filters: [] } : other,+ ),+ sound,+ ],+ };+}++/**+ * Puts detached audio back into its video clip - "Reattach audio".+ *+ * Accepts either side of the link: the muted video clip or any of its+ * detached audio clips. Every linked audio clip is absorbed (a detached clip+ * that was split leaves several), the video clip sounds again, and the audio+ * filters ride back with it. Does nothing when the other side is gone.+ */+export function reattachAudio(project: Project, clipId: string): Project {+ const clip = findClip(project, clipId);+ if (!clip) return project;++ const videoId = clip.kind === "audio" && clip.detachedFrom ? clip.detachedFrom : clip.id;+ const video = findClip(project, videoId);+ const sounds = detachedAudioOf(project, videoId);+ if (!video || sounds.length === 0) return project;++ const doomed = new Set(sounds.map((sound) => sound.id));+ return {+ ...project,+ clips: project.clips+ .filter((other) => !doomed.has(other.id))+ .map((other) =>+ other.id === videoId+ ? { ...other, muted: undefined, filters: [...sounds[0].filters] }+ : other,+ ),+ };+}+/** Cuts a clip in two at `time`. Does nothing if the cut misses the clip. */export function splitClip(project: Project, clipId: string, time: number): Project {const clip = findClip(project, clipId);diff --git a/desktop/src/lib/transport.ts b/desktop/src/lib/transport.tsindex 3ca0e20d..6f04d8a4 100644--- a/desktop/src/lib/transport.ts+++ b/desktop/src/lib/transport.ts@@ -1,17 +1,19 @@import { useCallback, useEffect, useRef, useState } from "react";+import { invoke, isTauri } from "@tauri-apps/api/core";+import { listen } from "@tauri-apps/api/event";/*** The playback clock.*- * The playhead is computed as `origin + (now - startedAt)`, never accumulated- * frame by frame. Accumulation drifts - it is the same mistake as summing- * `1/29.97` in a loop, and it is why `relay-core` has a rational time type at- * all. Here it also means a dropped frame costs nothing: the next tick reads- * the wall clock and lands in the right place.+ * The engine owns it: the position that matters is the audio device's sample+ * counter, reported through `transport` events at ~30Hz. Between events the+ * UI interpolates on the wall clock from the last reported anchor, so the+ * playhead moves at animation rate while never drifting further than one+ * event interval from the truth.*- * Playback is still paced by the browser here. That is a prototype- * compromise: the engine owns the clock once it can decode and present- * frames itself.+ * In a plain browser (`npm run dev` with no Rust host) there is no engine and+ * the anchor simply never gets corrected - the wall clock alone is the same+ * prototype behaviour this file always had there.*/export interface Transport {playhead: number;@@ -26,64 +28,87 @@ export interface Transport {}export function useTransport({ duration }: { duration: number }): Transport {+ const native = isTauri();const [playhead, setPlayhead] = useState(0);const [playing, setPlaying] = useState(false);- // Where the clock was when playback started, and when that was.- const origin = useRef(0);- const startedAt = useRef(0);- // Read inside the rAF loop so the effect does not restart on every change.+ // The last known-true position and when it was known. While playing, the+ // rendered playhead is `position + (now - at)`.+ const anchor = useRef({ position: 0, at: 0 });+ // Read inside loops and callbacks without restarting them.+ const playheadRef = useRef(0);+ playheadRef.current = playhead;+ const playingRef = useRef(false);+ playingRef.current = playing;const durationRef = useRef(duration);durationRef.current = duration;- const pause = useCallback(() => setPlaying(false), []);+ const pause = useCallback(() => {+ setPlaying(false);+ if (native) void invoke("transport_pause").catch(() => undefined);+ }, [native]);const play = useCallback(() => {- setPlayhead((current) => {- origin.current = current;- return current;- });- startedAt.current = performance.now();+ const from = playheadRef.current;+ anchor.current = { position: from, at: performance.now() };+ if (native) void invoke("transport_play", { position: from }).catch(() => undefined);setPlaying(true);- }, []);+ }, [native]);const toggle = useCallback(() => {- if (playing) pause();+ if (playingRef.current) pause();else play();- }, [pause, play, playing]);+ }, [pause, play]);- const seek = useCallback((seconds: number) => {- const next = Math.max(0, seconds);- origin.current = next;- startedAt.current = performance.now();- setPlayhead(next);- }, []);+ const seek = useCallback(+ (seconds: number) => {+ const next = Math.max(0, seconds);+ anchor.current = { position: next, at: performance.now() };+ setPlayhead(next);+ if (native) void invoke("transport_seek", { position: next }).catch(() => undefined);+ },+ [native],+ );const step = useCallback((frames: number, frameRate: number) => {// Land on an exact frame boundary rather than adding an offset to// wherever the playhead happened to stop.- setPlayhead((current) => {- const next = Math.max(0, (Math.round(current * frameRate) + frames) / frameRate);- origin.current = next;- startedAt.current = performance.now();- return next;- });+ const next = Math.max(0, (Math.round(playheadRef.current * frameRate) + frames) / frameRate);+ seek(next);},- [],+ [seek],);+ // Engine position reports re-anchor the interpolation. Ignored while+ // paused: a report can arrive just after a local pause, and rewinding the+ // playhead to it would visibly snap.+ useEffect(() => {+ if (!native) return;+ let stopped = false;+ const stopping = listen<number>("transport", (event) => {+ if (stopped || !playingRef.current) return;+ anchor.current = { position: event.payload, at: performance.now() };+ });+ return () => {+ stopped = true;+ void stopping.then((stop) => stop());+ };+ }, [native]);++ // The render loop: interpolate from the anchor while playing.useEffect(() => {if (!playing) return;let frame = 0;const tick = (now: number) => {- const next = origin.current + (now - startedAt.current) / 1000;+ const next = anchor.current.position + (now - anchor.current.at) / 1000;const end = durationRef.current;if (end > 0 && next >= end) {setPlayhead(end);setPlaying(false);+ if (native) void invoke("transport_pause").catch(() => undefined);return;}@@ -93,7 +118,7 @@ export function useTransport({ duration }: { duration: number }): Transport {frame = requestAnimationFrame(tick);return () => cancelAnimationFrame(frame);- }, [playing]);+ }, [playing, native]);return { playhead, playing, play, pause, toggle, seek, step };}diff --git a/engine/crates/relay-core/src/timeline.rs b/engine/crates/relay-core/src/timeline.rsindex c002783b..c97cdad8 100644--- a/engine/crates/relay-core/src/timeline.rs+++ b/engine/crates/relay-core/src/timeline.rs@@ -61,12 +61,56 @@ pub struct Clip {pub duration: Rational,/// Blend factor in `0.0..=1.0`, applied over whatever is beneath.pub opacity: f32,+ /// How the picture sits in the frame. Identity is fitted and centred.+ pub transform: Transform,+}++/// A clip's placement in the output frame.+///+/// Deliberately resolution-independent: `scale` is relative to the fitted+/// size and the offsets are fractions of the frame, so the same transform+/// means the same picture on a 720p export and a 4K one.+#[derive(Clone, Copy, PartialEq, Debug)]+pub struct Transform {+ /// Multiplier over the fitted size. 1 fills the frame, preserving aspect.+ pub scale: f64,+ /// Offset of the picture's centre from frame centre, as a fraction of+ /// frame width.+ pub offset_x: f64,+ /// Offset as a fraction of frame height.+ pub offset_y: f64,+ /// Clockwise rotation about the picture's centre, in degrees.+ pub rotation: f64,+}++impl Transform {+ /// Fitted, centred, unrotated.+ pub const IDENTITY: Transform =+ Transform { scale: 1.0, offset_x: 0.0, offset_y: 0.0, rotation: 0.0 };++ /// True when applying this transform would change nothing.+ pub fn is_identity(&self) -> bool {+ *self == Self::IDENTITY+ }+}++impl Default for Transform {+ fn default() -> Self {+ Self::IDENTITY+ }}impl Clip {/// A clip that starts at the beginning of its media and is fully opaque.pub fn new(media: MediaRef, start: Rational, duration: Rational) -> Self {- Self { media, source_start: Rational::ZERO, start, duration, opacity: 1.0 }+ Self {+ media,+ source_start: Rational::ZERO,+ start,+ duration,+ opacity: 1.0,+ transform: Transform::IDENTITY,+ }}/// The span of timeline this clip occupies.diff --git a/engine/crates/relay-render/src/compositor.rs b/engine/crates/relay-render/src/compositor.rsindex cebc627f..6bf2a420 100644--- a/engine/crates/relay-render/src/compositor.rs+++ b/engine/crates/relay-render/src/compositor.rs@@ -7,6 +7,40 @@use relay_core::frame::{BYTES_PER_PIXEL, Frame};+/// A layer's placement beyond its base position, in output pixels.+///+/// Applied about the layer's centre: scale first, then rotation, then the+/// translation. Pixel-space on purpose - the resolution-independent form+/// lives in `relay-core::timeline::Transform`, and whoever builds layers+/// converts exactly once.+#[derive(Clone, Copy, PartialEq, Debug)]+pub struct Placement {+ /// Multiplier over the layer's own size.+ pub scale: f32,+ /// Clockwise rotation about the layer's centre, in radians.+ pub rotation: f32,+ /// Offset of the layer's centre from its base position, in pixels.+ pub translate_x: f32,+ pub translate_y: f32,+}++impl Placement {+ /// Unscaled, unrotated, unmoved.+ pub const IDENTITY: Placement =+ Placement { scale: 1.0, rotation: 0.0, translate_x: 0.0, translate_y: 0.0 };++ /// True when applying this placement would change nothing.+ pub fn is_identity(&self) -> bool {+ *self == Self::IDENTITY+ }+}++impl Default for Placement {+ fn default() -> Self {+ Self::IDENTITY+ }+}+/// One source image, positioned and weighted.#[derive(Clone, Copy, Debug)]pub struct Layer<'a> {@@ -18,12 +52,14 @@ pub struct Layer<'a> {pub x: i32,/// Vertical offset of the layer's top-left corner. May be negative.pub y: i32,+ /// Scale, rotation and translation about the layer's centre.+ pub placement: Placement,}impl<'a> Layer<'a> {/// A layer drawn at the origin, fully opaque.pub fn new(frame: &'a Frame) -> Self {- Self { frame, opacity: 1.0, x: 0, y: 0 }+ Self { frame, opacity: 1.0, x: 0, y: 0, placement: Placement::IDENTITY }}/// Sets the blend strength.@@ -38,6 +74,12 @@ impl<'a> Layer<'a> {self.y = y;self}++ /// Sets the scale/rotation/translation.+ pub fn with_placement(mut self, placement: Placement) -> Self {+ self.placement = placement;+ self+ }}/// Blends layers into a single output frame.@@ -65,6 +107,12 @@ impl Compositor for CpuCompositor {}let source = layer.frame;++ if !layer.placement.is_identity() {+ blend_transformed(&mut output, layer, opacity);+ continue;+ }+let Some((dst_x, src_x, columns)) = overlap(layer.x, source.width(), width) else {continue;};@@ -86,6 +134,93 @@ impl Compositor for CpuCompositor {}}+/// Draws one layer through its placement: inverse-mapped, bilinearly sampled.+///+/// Each covered output pixel is carried backwards through the placement into+/// source coordinates and sampled there. Inverse mapping is what makes the+/// result hole-free at any scale or angle; bilinear is the cheapest filter+/// that does not shimmer on motion. Only the transformed bounding box is+/// visited, so a small layer stays cheap on a large frame.+fn blend_transformed(output: &mut Frame, layer: &Layer<'_>, opacity: f32) {+ let source = layer.frame;+ let placement = layer.placement;+ let scale = placement.scale.max(1e-6);+ let (sin, cos) = placement.rotation.sin_cos();++ let src_w = source.width() as f32;+ let src_h = source.height() as f32;+ // The layer's centre in output space, after translation.+ let centre_x = layer.x as f32 + src_w / 2.0 + placement.translate_x;+ let centre_y = layer.y as f32 + src_h / 2.0 + placement.translate_y;++ // Bounding box of the transformed rectangle, clamped to the output.+ let half_w = src_w * scale / 2.0;+ let half_h = src_h * scale / 2.0;+ let reach_x = (half_w * cos.abs()) + (half_h * sin.abs());+ let reach_y = (half_w * sin.abs()) + (half_h * cos.abs());++ let x_from = ((centre_x - reach_x).floor().max(0.0)) as u32;+ let y_from = ((centre_y - reach_y).floor().max(0.0)) as u32;+ let x_to = ((centre_x + reach_x).ceil().min(output.width() as f32)) as u32;+ let y_to = ((centre_y + reach_y).ceil().min(output.height() as f32)) as u32;+ if x_from >= x_to || y_from >= y_to {+ return;+ }++ let dst_stride = output.width() as usize * BYTES_PER_PIXEL;+ let src_stride = source.width() as usize * BYTES_PER_PIXEL;+ let src_pixels = source.pixels();+ let dst_pixels = output.pixels_mut();++ for y in y_from..y_to {+ for x in x_from..x_to {+ // Sample at the pixel centre, mapped back into source space:+ // untranslate, unrotate, unscale, then re-origin at the corner.+ let dx = (x as f32 + 0.5) - centre_x;+ let dy = (y as f32 + 0.5) - centre_y;+ let sx = (dx * cos + dy * sin) / scale + src_w / 2.0 - 0.5;+ let sy = (-dx * sin + dy * cos) / scale + src_h / 2.0 - 0.5;++ if sx < -0.5 || sy < -0.5 || sx > src_w - 0.5 || sy > src_h - 0.5 {+ continue;+ }++ let x0 = sx.floor().max(0.0) as usize;+ let y0 = sy.floor().max(0.0) as usize;+ let x1 = (x0 + 1).min(source.width() as usize - 1);+ let y1 = (y0 + 1).min(source.height() as usize - 1);+ let fx = (sx - x0 as f32).clamp(0.0, 1.0);+ let fy = (sy - y0 as f32).clamp(0.0, 1.0);++ let mut sample = [0.0f32; 4];+ for (corner_x, corner_y, weight) in [+ (x0, y0, (1.0 - fx) * (1.0 - fy)),+ (x1, y0, fx * (1.0 - fy)),+ (x0, y1, (1.0 - fx) * fy),+ (x1, y1, fx * fy),+ ] {+ let at = corner_y * src_stride + corner_x * BYTES_PER_PIXEL;+ for channel in 0..4 {+ sample[channel] += f32::from(src_pixels[at + channel]) * weight;+ }+ }++ let alpha = (sample[3] / 255.0) * opacity;+ if alpha <= 0.0 {+ continue;+ }++ let at = y as usize * dst_stride + x as usize * BYTES_PER_PIXEL;+ for channel in 0..3 {+ let over = sample[channel] * alpha;+ let under = f32::from(dst_pixels[at + channel]) * (1.0 - alpha);+ dst_pixels[at + channel] = (over + under).round().clamp(0.0, 255.0) as u8;+ }+ dst_pixels[at + 3] = 255;+ }+ }+}+/// Works out the visible span along one axis when a layer of `source` pixels is/// placed at `offset` inside a destination of `destination` pixels.///@@ -220,6 +355,47 @@ mod tests {assert_eq!(frame.pixel(1, 1), Some([255, 0, 0, 255]));}+ #[test]+ fn scaling_doubles_coverage() {+ let red = solid(2, 2, [255, 0, 0, 255]);+ let placement = Placement { scale: 2.0, ..Placement::IDENTITY };+ let frame =+ CpuCompositor.composite(4, 4, &[Layer::new(&red).at(1, 1).with_placement(placement)]);+ // A 2x2 layer based at (1,1) scaled x2 about its centre covers the+ // whole 4x4 output.+ assert_eq!(frame.pixel(0, 0), Some([255, 0, 0, 255]));+ assert_eq!(frame.pixel(3, 3), Some([255, 0, 0, 255]));+ }++ #[test]+ fn translation_moves_the_layer() {+ let red = solid(1, 1, [255, 0, 0, 255]);+ let placement = Placement { translate_x: 2.0, ..Placement::IDENTITY };+ let frame = CpuCompositor.composite(3, 1, &[Layer::new(&red).with_placement(placement)]);+ assert_eq!(frame.pixel(0, 0), Some([0, 0, 0, 255]));+ assert_eq!(frame.pixel(2, 0), Some([255, 0, 0, 255]));+ }++ #[test]+ fn a_half_turn_swaps_the_ends() {+ let mut strip = Frame::transparent(2, 1);+ strip.set_pixel(0, 0, [255, 0, 0, 255]);+ strip.set_pixel(1, 0, [0, 0, 255, 255]);+ let placement = Placement { rotation: std::f32::consts::PI, ..Placement::IDENTITY };+ let frame = CpuCompositor.composite(2, 1, &[Layer::new(&strip).with_placement(placement)]);+ assert_eq!(frame.pixel(0, 0), Some([0, 0, 255, 255]));+ assert_eq!(frame.pixel(1, 0), Some([255, 0, 0, 255]));+ }++ #[test]+ fn a_transformed_layer_is_clipped_at_the_frame_edge() {+ let red = solid(2, 2, [255, 0, 0, 255]);+ let placement = Placement { scale: 100.0, ..Placement::IDENTITY };+ let frame = CpuCompositor.composite(2, 2, &[Layer::new(&red).with_placement(placement)]);+ assert_eq!(frame.width(), 2);+ assert_eq!(frame.pixel(1, 1), Some([255, 0, 0, 255]));+ }+#[test]fn overlap_spans_are_correct() {assert_eq!(overlap(0, 4, 4), Some((0, 0, 4)));diff --git a/engine/crates/relay-render/src/lib.rs b/engine/crates/relay-render/src/lib.rsindex 3b249267..4a7408b1 100644--- a/engine/crates/relay-render/src/lib.rs+++ b/engine/crates/relay-render/src/lib.rs@@ -13,5 +13,5 @@pub mod compositor;pub mod plan;-pub use compositor::{Compositor, CpuCompositor, Layer};+pub use compositor::{Compositor, CpuCompositor, Layer, Placement};pub use plan::{FramePlan, PlannedLayer, plan_frame};diff --git a/engine/crates/relay-render/src/plan.rs b/engine/crates/relay-render/src/plan.rsindex 56ec67a4..da5c92f3 100644--- a/engine/crates/relay-render/src/plan.rs+++ b/engine/crates/relay-render/src/plan.rs@@ -11,7 +11,7 @@use std::path::PathBuf;use relay_core::time::Rational;-use relay_core::timeline::{ClipId, Timeline, TrackKind};+use relay_core::timeline::{ClipId, Timeline, TrackKind, Transform};/// One visible layer at one instant.#[derive(Clone, PartialEq, Debug)]@@ -24,6 +24,8 @@ pub struct PlannedLayer {pub source_time: Rational,/// Blend strength over everything beneath, in `0.0..=1.0`.pub opacity: f32,+ /// The clip's placement in the frame, resolution-independent.+ pub transform: Transform,}/// Everything needed to draw one output frame.@@ -73,6 +75,7 @@ pub fn plan_frame(timeline: &Timeline, time: Rational) -> FramePlan {media: clip.media.path.clone(),source_time,opacity: clip.opacity.clamp(0.0, 1.0),+ transform: clip.transform,});}