EffectCraft: An open-source motion graphics and compositing tool written by scratch in pure Rust
S2.52: refreshed benchmark (24 frames, SAM 2 on the GPU)
Kristian Myram7f56fd87097b
diff --git a/docs/roto-brush-sam2.md b/docs/roto-brush-sam2.mdindex b3ceba2ad..d36d2086c 100644--- a/docs/roto-brush-sam2.md+++ b/docs/roto-brush-sam2.md@@ -88,18 +88,18 @@ encoder's 64 × 64 level matches PyTorch to 7e-6, the memory attention to 1.8e-5times the pieces on both devices (`EFFECTCRAFT_SAM2_PROFILE=1` prints a frame's breakdown).## Benchmark-`effectcraft-cli bench --roto [--frames N] [--sam2 model.safetensors] [--mobilesam mobile_sam.pt]` runs synthetic clips-with known mattes through all three. 12 frames, debug build, this PC; mean IoU (mean IoU after the object returns), seconds+`effectcraft-cli bench --roto [--frames N] [--sam2 model.safetensors] [--mobilesam mobile_sam.pt]` runs synthetic clips with+known mattes through all three. 24 frames, debug build, SAM 2 on the GPU; mean IoU (mean IoU after the object returns), secondsper frame:| Clip | Classical | MobileSAM | SAM 2.1 Tiny ||---|---|---|---|-| move | 0.982 | 0.992 | 0.993 |-| light | 0.981 | 0.992 | 0.992 |-| occlusion | 0.978 | 0.881 | 0.988 |-| reenter | 0.632 (0.000) | 0.499 (0.725) | 0.975 (0.967) |-| s/frame | 0.01 | 1.1 | 8.2 |+| move | 0.980 | 0.989 | 0.992 |+| light | 0.979 | 0.990 | 0.992 |+| occlusion (a bar hides the disk, then clears it) | 0.491 (0.000) | 0.494 (0.000) | 0.989 (0.988) |+| reenter (the disk leaves and comes back) | 0.618 (0.000) | 0.620 (0.000) | 0.982 (0.976) |+| s/frame | 0.01 | 0.5–1.3 | 0.8 |MobileSAM prompts every frame independently from the flow-warped matte, so it follows the object while the flow does and-loses it when it is hidden; SAM 2's streaming memory is what lets it re-acquire the object. The synthetic clips are easy-(one textured disc), so the "move" and "light" rows mostly show all three working; the re-entry row is the difference.+loses it when it is hidden; SAM 2's streaming memory is what lets it re-acquire the object. The synthetic clips are easy (one+textured disc), so the "move" and "light" rows mostly show all three working; the last two rows are the difference.