Skip to content

EffectCraft: An open-source motion graphics and compositing tool written by scratch in pure Rust

#1539: clarify fallback visibility and benchmark correction rounds

Kristian Myram76f82d7d552b
diff --git a/apps/effectcraft-cli/src/main.rs b/apps/effectcraft-cli/src/main.rs
index dc47671e2..69a56c696 100644
--- a/apps/effectcraft-cli/src/main.rs
+++ b/apps/effectcraft-cli/src/main.rs
@@ -1056,7 +1056,7 @@ fn bench_roto(args: &Args) -> Result<(), Failure> {
if sam2.is_none() {
note!("bench --roto: no SAM 2.1 size is installed; no SAM 2 column");
}
- // `--fixes`: also count the correction strokes each needs before every frame reaches IoU 0.9 (up to 6).
+ // `--fixes`: count correction rounds before every frame reaches IoU 0.9 (up to 6 rounds, each up to two strokes).
let fixes = args.flag("--fixes");
let measure = |sc: &bench::Scenario, run: &dyn Fn(&bench::Scenario) -> Option<bench::Run>| if fixes { bench::with_fixes(sc, 0.9, 6, run) } else { run(sc) };
let mut runs = vec![];
diff --git a/crates/track/src/roto/bench.rs b/crates/track/src/roto/bench.rs
index c086bfd57..98ada3903 100644
--- a/crates/track/src/roto/bench.rs
+++ b/crates/track/src/roto/bench.rs
@@ -46,7 +46,7 @@ pub struct Run {
/// Mean IoU over the frames after the object's last absence (re-appearance), if it had one.
pub after_return: Option<f64>,
pub per_frame: Vec<f64>,
- /// Correction strokes needed before every frame reaches the target IoU (`>N`: not within N); set by
+ /// Correction rounds needed before every frame reaches the target IoU (`>N`: not within N); set by
/// [`with_fixes`], `None` when not measured.
pub fixes: Option<String>,
/// The most memory (MB) the tracker held for its memory banks and cached frames (trackers that have none: `None`).
@@ -301,7 +301,7 @@ pub fn run_tracker(sc: &Scenario, tracker: &str, model: &dyn MaskModel) -> Optio
}
/// The correction a user would paint on frame `f` where `pred` is wrong: a short foreground stroke through
-/// the true object's centre, and a background stroke on the biggest false-positive blob's centre.
+/// the true object's centre, and a background stroke at the centroid of all false-positive pixels.
fn correction(sc: &Scenario, f: usize, pred: &[u8]) -> Vec<Stroke> {
let truth = sc.truth.get(f).map(Vec::as_slice).unwrap_or(&[]);
let centre = |keep: &dyn Fn(usize) -> bool| -> Option<[f64; 2]> {
@@ -324,8 +324,8 @@ fn correction(sc: &Scenario, f: usize, pred: &[u8]) -> Vec<Stroke> {
}
/// Repeat `run` on `sc`, each time painting the correction a user would on the first frame under `target` IoU,
-/// until every frame reaches it or `max` corrections were needed. Sets `fixes` on the first run's result: the
-/// number of corrections, or `>max`.
+/// until every frame reaches it or `max` correction rounds were needed. Sets `fixes` on the first run's result:
+/// the number of rounds (each can add foreground and background strokes), or `>max`.
pub fn with_fixes(sc: &Scenario, target: f64, max: usize, run: &dyn Fn(&Scenario) -> Option<Run>) -> Option<Run> {
let mut cur = sc.clone();
let mut first: Option<Run> = None;
diff --git a/docs/roto-brush-sam2.md b/docs/roto-brush-sam2.md
index 7078fa103..b165a29bc 100644
--- a/docs/roto-brush-sam2.md
+++ b/docs/roto-brush-sam2.md
@@ -126,8 +126,10 @@ the CPU.
Pending per-object probability maps have a separate 256 MiB budget, including their allocated capacity and frame
slots. This bounds the full-resolution results waiting to be united when objects first prompted on different frames
-report in separate passes. Reaching the budget cancels tracking and reports the reason through the existing
-`trackerNote`; propagation falls back to the classic engine. A single object's results need no pending buffer.
+report in separate passes. Reaching the budget cancels tracking and falls back to the classic engine. The reason is
+available as `trackerNote` in live job progress only. Completion removes that progress and shows a success toast, so
+`wait: true` callers can miss the fallback reason; the completion toast alone does not confirm the tracker was used.
+A single object's results need no pending buffer.
## How it works
`crates/sam2` is a Rust port (on [candle](https://github.com/huggingface/candle), native only) of the Hugging Face
@@ -189,9 +191,10 @@ loses it when it is hidden; SAM 2's streaming memory is what lets it re-acquire
textured disc), so the "move" and "light" rows mostly show all three working; the last two rows are the difference.
### Corrections needed (`--fixes`)
-`bench --roto --fixes` also counts the correction strokes each tracker needs before every frame reaches IoU 0.9: after each
-run it paints what a user would on the first frame under the target (a short foreground stroke through the true object's
-centre, and a background stroke on the biggest false-positive blob), up to 6. 16 frames, debug build:
+`bench --roto --fixes` also counts the correction rounds each tracker needs before every frame reaches IoU 0.9: after each
+run it paints on the first frame under the target, up to 6 rounds. A round can add a short foreground stroke through the
+true object's centre and a background stroke at the centroid of all false-positive pixels. With separate false-positive
+blobs, that centroid can fall between them. The counts below are rounds, not individual strokes. 16 frames, debug build:
| Clip | Classical | MobileSAM | SAM 2.1 Tiny |
|---|---|---|---|