Skip to content

EffectCraft: An open-source motion graphics and compositing tool written by scratch in pure Rust

S2.0: record the SAM 2 decisions D1-D6

Kristian Myram9cb48399c364
diff --git a/docs/adr/0002-sam2-roto-tracker.md b/docs/adr/0002-sam2-roto-tracker.md
new file mode 100644
index 000000000..3807d84a2
--- /dev/null
+++ b/docs/adr/0002-sam2-roto-tracker.md
@@ -0,0 +1,19 @@
+# ADR 0002: SAM 2 video tracker for the Roto Brush
+
+Status: accepted (maintainer answers, 2026-10-15). Plan: `plan/roto-sam2.md` (maintainer-local).
+
+## Context
+The Roto Brush's classical engine loses an object that is occluded or leaves and re-enters the frame
+(`effectcraft-cli bench --roto`: IoU 0.49 on the occlusion clip, 0.62 on re-entry, 0 after the object returns).
+Meta's SAM 2.1 (Apache-2.0 code and checkpoints) has streaming memory built for this.
+
+## Decisions
+- **D1** A native-only `candle` dependency may enter the workspace, in its own crate (`effectcraft-sam2`), excluded from wasm32.
+- **D2** `effectcraft-sam2` is licensed Apache-2.0 only (it derives from Apache-2.0 Hugging Face `transformers` code); `effectcraft-segment` stays MIT OR Apache-2.0.
+- **D3** Shared candle helpers are vendored with a NOTICE for now; a shared crate with LightCraft can follow.
+- **D4** Weights are never hosted or bundled by us: direct from Hugging Face, a mirror, or a user folder, with the SHA-256 pinned in the registry.
+- **D5** SAM 2 is opt-in at first; it becomes the default for tracking only after the benchmark shows it should.
+- **D6** Hand-written kernels (the MobileSAM way) remain the Phase 6 route for AMD GPUs if candle cannot get there.
+
+## Consequences
+Without the model installed nothing changes. Every ported file carries Apache-2.0 §4(b) modified-work headers.
diff --git a/tools/requirements-sam2-reference.txt b/tools/requirements-sam2-reference.txt
new file mode 100644
index 000000000..8fab732ff
--- /dev/null
+++ b/tools/requirements-sam2-reference.txt
@@ -0,0 +1,4 @@
+# Pinned versions the SAM 2 reference fixtures were generated with (CPU).
+torch==2.7.0
+transformers==5.19.0
+safetensors==0.8.0