Skip to content

A virtual camera films an AI agent's screen while a tracker locks onto every glyph. Two fframes passes, one SkSL shader, all sound synthesized.

docs: English everywhere, new README with preview, shot sheet and shader stages

Mustafabb7ff6abdf64
diff --git a/README.md b/README.md
index da0d802..28571a1 100644
--- a/README.md
+++ b/README.md
@@ -1,41 +1,170 @@
+<div align="center">
+
# ff-tracking
-> **English:** a 6-second tracking-HUD style test made with [fframes](https://github.com/dmtrKovalenko/fframes).
-> Pass 1 (`flat`) renders an agent terminal and tracker boxes at 3840×2160. Pass 2 (`lens`) films it
-> through one SkSL shader (`lens/shaders/lens.sksl`): perspective tilt, depth of field focused on the
-> tracked glyph, LED grid, bloom, chromatic aberration, per-scene palette. A sharp camera-space
-> tracker overlay is projected with the same camera (`hud::project`), so the label coordinates are the
-> boxes' real output pixels. Sound is synthesized (`tools/sfx.py`). Run `tools/render.sh`.
-> Style inspired by [@mnowakdesign](https://x.com/mnowakdesign/status/2108253918086176899). MIT.
+**A virtual camera films an AI agent's screen while a tracker locks onto every glyph it types.**
+
+Six seconds, two [fframes](https://github.com/dmtrKovalenko/fframes) passes, one SkSL shader.
+No footage, no stock audio: every pixel and every sound comes from code.
+
+[![fframes 1.2](https://img.shields.io/badge/fframes-1.2.0-ff5a1f)](https://github.com/dmtrKovalenko/fframes)
+[![Rust 2024](https://img.shields.io/badge/rust-2024-b7410e?logo=rust)](https://www.rust-lang.org)
+[![Skia Metal](https://img.shields.io/badge/GPU-Skia%20Metal-555)](https://skia.org)
+[![License: MIT](https://img.shields.io/badge/license-MIT-2ea44f)](LICENSE)
+
+<a href="https://github.com/mrsarac/ff-tracking/releases/download/v1.0.0/tracking.mp4">
+ <img src="docs/media/hero.webp" width="960" alt="ff-tracking: tilted close-up of a terminal, hatched tracker boxes locking onto glyphs, thermal and phosphor palettes, ending on a green Done pill">
+</a>
+
+**[▶ Full 1080p with sound](https://github.com/mrsarac/ff-tracking/releases/download/v1.0.0/tracking.mp4)** ·
+[On X](https://x.com/0xsarac/status/2108515856544317489) ·
+[How it works](#how-it-works) ·
+[Quick start](#quick-start)
+
+</div>
+
+---
+
+## What you are looking at
+
+An agent thinks out loud in a terminal: `thinking…` → `reading 14 files` →
+`tool_call: render_frame()` → … → `✓ 0 problems` → `Done`. A computer-vision style tracker
+follows the newest characters. A camera that never sits still films the screen up close.
+
+- **Real coordinates.** Every `x: 1403 y: 559` label is the box's actual pixel position in
+ that frame of the output. None of them are random numbers.
+- **Focus that follows the tracker.** The focal plane is set by the box being tracked, so
+ the depth of field racks along the line as the agent types.
+- **Chain of thought, drawn.** Links run from box to box in reading order, and a dot runs
+ along every link.
+- **The lock.** Every shot ends with the newest word locking on: red for two frames, a
+ scanline tear, a beep panned to where the box is.
+- **Twelve shots, twelve looks.** A cut every 0.5 s through paper white, thermal, black
+ and white, phosphor, amber, navy and neon.
+- **A collapse and a landing.** Fourteen boxes fall into one, and that box becomes a
+ `Done` pill before the fade to black.
+
+<img src="docs/media/shots.jpg" width="100%" alt="Contact sheet of all twelve shots">
+
+## How it works
+
+```text
+ hud (shared crate: shots, tracker, camera, projection)
+ ╱ │ ╲
+ flat (pass 1) tracks ──► out/tracks.json ──► sfx.py lens (pass 2)
+ SVG, 3840×2160 │ SkSL on Skia Metal
+ │ ▼ + sharp camera overlay
+ └──────► out/pass/flat.mp4 ──────────────┴──► iChannel0 ──► out/tracking.mp4
+```
+
+**Pass 1, `flat`**, draws the screen the way a terminal would: black, white, monospace, one
+`<text>` per glyph, so the position of every character is known exactly. The tracker boxes
+are on the screen itself. Some are hatched with an SVG pattern in `mix-blend-mode:
+difference`, so the stripes turn dark where they cross a glyph. The only color is pure red,
+and it marks a locked box. The pass renders at 2× because the lens magnifies it up to 3×.
+
+**Pass 2, `lens`**, binds each frame of pass 1 as a shader input and films it. One SkSL
+shader does everything physical:
-6 saniyelik tracking-HUD stil denemesi, [fframes](https://github.com/dmtrKovalenko/fframes) ile.
-Bir ajan terminalde düşünür; ekrandaki takip kutuları yazılan harfleri izler, bir sanal kamera
-ekranı eğik açıdan, odak bulanıklığı, LED ızgarası ve sahne başına paletle çeker.
+<img src="docs/media/stages.jpg" width="100%" alt="The same frame through the six steps of the lens shader">
-Stil ilhamı: [Michael Nowak (@mnowakdesign)](https://x.com/mnowakdesign/status/2108253918086176899).
+1. **Perspective.** Every pixel shoots a ray at a tilted plane: real perspective, not an SVG skew.
+2. **Depth of field.** The circle of confusion comes from the depth difference to the tracked
+ target, with 36 golden-angle taps and highlights weighted like bokeh.
+3. **Palette and aberration.** Brightness maps through five color stops per shot. R, G and B
+ sample at radially shifted points, and the shift grows on a lock.
+4. **Bloom and LED grid.** A wide ring of taps lifts everything around bright glyphs. The
+ grid uses equal-area RGB stripes, so it adds no tint, and it fades out of focus.
+5. **Tear, grain, camera tracker.** Rows slide on a lock and right after a cut. On top, the
+ lens draws its own tracker sharp: links, labels and target brackets, projected with
+ `hud::project`, the CPU copy of the shader's camera.
-## Çalıştırma
+Both passes and the sound read the same `hud` crate, so the lens always knows where the
+glyph it focuses on actually is.
+
+**Sound**: `tools/sfx.py` reads `out/tracks.json` and synthesizes everything. It makes a
+screen hum, a bit-crushed glitch on each cut, a click per keystroke and a pentatonic beep
+per lock, panned to the box. It adds a riser into the collapse and a chime with a sub hit on
+`Done`. Loudness is measured with ffmpeg's `ebur128` and set to -14 LUFS.
+
+## Quick start
+
+Requirements: macOS with Apple silicon (Skia on Metal), Rust, ffmpeg, Python 3 with `numpy`
+and `scipy`. The README images also need Pillow and `img2webp`.
+
+```bash
+git clone https://github.com/mrsarac/ff-tracking && cd ff-tracking
+tools/render.sh # → out/tracking.mp4
+```
+
+`render.sh` builds the workspace, exports the tracks, renders pass 1, synthesizes the sound
+and renders pass 2. It takes about 40 s on an M3 Pro, plus the first build.
+
+Every pass is an fframes CLI, so the usual tools work on each of them:
```bash
-tools/render.sh # -> out/tracking.mp4 (yaklaşık 40 sn)
+target/release/lens strip all -n 12 # contact sheet of the final video
+target/release/lens frame 81 -o frames # one full-size frame
+target/release/lens preview # real-time window with sound
+target/release/lens audio analyze # loudness, true peak
+LENS_STAGE=1 target/release/lens frame 81 # stop the shader after a step (0-3), as in the image above
+python3 -I tools/readme_media.py # rebuild the images in this README
```
-Gerekenler: Rust, ffmpeg, Python 3 + numpy + scipy, macOS (Skia Metal).
+## Make it yours
+
+Every shot is one entry in `SCENE_LIST` in [`hud/src/lib.rs`](hud/src/lib.rs):
+
+```rust
+SceneDef {
+ kind: Kind::Typing,
+ text: "thinking…", // what the agent types
+ palette: Palette::Thermal, // Paper, Thermal, Mono, Phosphor, Amber, Navy, Neon
+ // tilt_x tilt_y roll zoom 0→1 pan x, y blur
+ cam: cam(0.38, -0.22, 0.04, 2.5, 2.8, 50.0, 0.0, 1.2),
+ hero: (560.0, 540.0), // where the line starts on the flat screen
+ size: 104.0, // font size
+},
+```
+
+Change the text, a palette or the camera, run `tools/render.sh`, and the tracker, focus,
+labels and beeps all follow. `cargo test -p hud` checks that every line fits the screen and
+that the camera keeps every target in frame.
+
+## Layout
+
+```text
+hud/ shots, tracker boxes, camera per frame, projection (no fframes dependency)
+flat/ pass 1: the screen, 3840×2160
+lens/ pass 2: lens/shaders/lens.sksl and the camera-space tracker
+tools/ render.sh (whole pipeline), sfx.py (sound), readme_media.py (these images)
+docs/ design spec, prompts and decisions, fframes contributions
+```
-## Yapı
+## Notes from the build
-| Klasör | Ne yapar |
-|---|---|
-| `hud/` | Sahneler, takip kutuları, kamera, projeksiyon; iki geçişin ortak kaynağı |
-| `flat/` | Geçiş 1: düz ekran + kutular, 3840×2160 → `out/pass/flat.mp4` |
-| `lens/` | Geçiş 2: `shaders/lens.sksl` ile kamera + net takip katmanı → `out/tracking.mp4` |
-| `tools/sfx.py` | Bütün sesleri sentezler → `out/pass/sfx.wav` |
+- A synced video frame is a regular image for a shader:
+ `.image("iChannel0", &frame.get_synced_video_frame(..)?.into_image())`. That is what makes
+ the two-pass camera possible.
+- `flat` is a reserved word in SkSL (a GLSL ES interpolation qualifier), so name helpers
+ something else.
+- At the `Video` level, never return `Svgr::empty()`; return a root `<svg>`. An empty root is
+ an error, and late in a Skia render it surfaces only after the encoders drain
+ ([fframes#209](https://github.com/dmtrKovalenko/fframes/issues/209)).
+- The two passes exist because a shader can't yet sample an SVG subtree of the same frame.
+ [fframes#210](https://github.com/dmtrKovalenko/fframes/issues/210) proposes that.
-Sahne metni, palet ya da kamera değiştirmek için: `hud/src/lib.rs` → `SCENE_LIST`.
+## Credits
-Tasarım: `docs/superpowers/specs/2026-10-09-tracking-hud-design.md`. Promptlar ve kararlar: `docs/PROMPTS.md`.
+- Visual style inspired by [Michael Nowak (@mnowakdesign)](https://x.com/mnowakdesign/status/2108253918086176899).
+ None of his footage is used.
+- Built on [fframes](https://github.com/dmtrKovalenko/fframes) by Dmitriy Kovalenko, rendering with
+ [Skia](https://skia.org), encoding with [FFmpeg](https://ffmpeg.org).
+- Font: [JetBrains Mono](https://github.com/JetBrains/JetBrainsMono) (SIL OFL 1.1, see
+ [`licenses/JetBrainsMono-OFL.txt`](licenses/JetBrainsMono-OFL.txt)).
+- Made in one Claude Code session. Every prompt and decision is in [`docs/PROMPTS.md`](docs/PROMPTS.md),
+ and the design is in [`docs/superpowers/specs/`](docs/superpowers/specs/2026-10-09-tracking-hud-design.md).
-## Lisans
+## License
-Kod: MIT (`LICENSE`). Yazı tipi: JetBrains Mono, SIL Open Font License 1.1 (`media/JetBrainsMono-OFL.txt`).
-Sesler `tools/sfx.py` ile sentezlenir; üçüncü taraf ses yoktur.
+[MIT](LICENSE) © 2026 Mustafa Saraç. The font keeps its own license.
diff --git a/docs/contrib/2026-10-09-fframes-taslaklar.md b/docs/contrib/fframes-issues.md
similarity index 92%
rename from docs/contrib/2026-10-09-fframes-taslaklar.md
rename to docs/contrib/fframes-issues.md
index 9508a35..1ac7a10 100644
--- a/docs/contrib/2026-10-09-fframes-taslaklar.md
+++ b/docs/contrib/fframes-issues.md
@@ -1,17 +1,20 @@
-# fframes katkı taslakları (2026-10-09)
+# Contributions to fframes (2026-10-09)
-Hedef: github.com/dmtrKovalenko/fframes, hesap @mrsarac. Durum: A gönderildi → https://github.com/dmtrKovalenko/fframes/issues/209 (2026-10-09). B gönderildi → https://github.com/dmtrKovalenko/fframes/issues/210. Repo public + MIT.
-Sıra: önce A (hata), sonra B (özellik önerisi, içinde C örnek teklifi).
-C için ff-tracking reposunun public olması ve bir lisans (MIT) alması gerekir.
+Opened from @mrsarac on [dmtrKovalenko/fframes](https://github.com/dmtrKovalenko/fframes):
+
+- **A.** [#209](https://github.com/dmtrKovalenko/fframes/issues/209), bug report.
+- **B.** [#210](https://github.com/dmtrKovalenko/fframes/issues/210), feature idea, with **C**, the offer to contribute this project as an example.
+
+The texts below are the submitted versions (hard-wrapped here, unwrapped on GitHub).
---
-## A. Issue: hata
+## A. Bug report (#209)
-**Başlık:**
+**Title:**
Skia render: an error late in the render surfaces only after unfinished segments drain their x264 lookahead (looks like a hang)
-**Gövde:**
+**Body:**
### What happens
@@ -132,12 +135,12 @@ Pro, rustc 1.91.1, Homebrew FFmpeg 9.0.1, libx264.165. On 1.0.0 I saw the same s
---
-## B. Issue: özellik önerisi (içinde C)
+## B. Feature idea (#210)
-**Başlık:**
+**Title:**
Feature idea: feed a shader with an SVG subtree rendered in the same frame (multi-pass in one video)
-**Gövde:**
+**Body:**
### Motivation
diff --git a/docs/media/hero.webp b/docs/media/hero.webp
new file mode 100644
index 0000000..6c37456
Binary files /dev/null and b/docs/media/hero.webp differ
diff --git a/docs/media/shots.jpg b/docs/media/shots.jpg
new file mode 100644
index 0000000..537e260
Binary files /dev/null and b/docs/media/shots.jpg differ
diff --git a/docs/media/stages.jpg b/docs/media/stages.jpg
new file mode 100644
index 0000000..9f59b87
Binary files /dev/null and b/docs/media/stages.jpg differ
diff --git a/docs/superpowers/specs/2026-10-09-tracking-hud-design.md b/docs/superpowers/specs/2026-10-09-tracking-hud-design.md
index 7bb295c..3327595 100644
--- a/docs/superpowers/specs/2026-10-09-tracking-hud-design.md
+++ b/docs/superpowers/specs/2026-10-09-tracking-hud-design.md
@@ -1,149 +1,129 @@
-# ff-tracking: tracking HUD stil denemesi (tasarım)
+# ff-tracking: tracking-HUD style test (design)
-Tarih: 2026-10-09
-Durum: onaylandı (brainstorming), uygulama planı bekliyor
+Date: 2026-10-09
+Status: approved (brainstorming), implemented
-## Amaç
+## Goal
-Michael Nowak'ın X paylaşımındaki görsel stili fframes ile yeniden üretmek ve geliştirmek.
-Referans: <https://x.com/mnowakdesign/status/2108253918086176899> (yerel kopya `reference/`, git dışı).
+Recreate and extend the visual style of Michael Nowak's X post with fframes.
+Reference: <https://x.com/mnowakdesign/status/2108253918086176899> (local copy in `reference/`, not in git).
-Bu bir **stil denemesidir**. İçerik ikinci planda. Başarı ölçütü: görünüm referans kadar
-"yakın çekim fiziksel ekran" hissi verir ve referansın ötesinde en az bir yeni fikir taşır.
+This is a **style test**; the content is secondary. Success: the result feels as much like
+"a close-up of a physical screen" as the reference and adds at least one idea beyond it.
-## Referansın analizi
+## Reference analysis
-- 5,5 sn, 1920×1080, sesli.
-- Arayüz metni çok yakından, eğik açıyla çekilmiş gibi. Güçlü alan derinliği (odak dışı bulanıklık).
-- LED/CRT piksel ızgarası, parlama (bloom), renk kayması (chromatic aberration).
-- Bilgisayarlı görü katmanı: taralı kutular, `x: 725 y: 475` etiketleri, kutuları bağlayan çizgiler.
-- Yaklaşık 0,5 sn'de bir kesme. Her sahnede farklı palet: kâğıt beyazı, termal mor, siyah-beyaz,
- lacivert, neon yeşil.
+- 5.5 s, 1920×1080, with sound.
+- UI text filmed very close and at an angle. Strong depth of field.
+- LED/CRT pixel grid, bloom, chromatic aberration.
+- Computer-vision overlay: hatched boxes, `x: 725 y: 475` labels, lines linking the boxes.
+- A cut about every 0.5 s. Each shot has its own palette: paper white, thermal purple,
+ black and white, navy, neon green.
-## Kararlar
+## Decisions
-| Konu | Karar |
+| Topic | Decision |
|---|---|
-| Kullanım | Stil denemesi |
-| Katmanlar | Takip katmanı, piksel ızgarası + parlama, eğik açı + bulanıklık, hızlı kurgu + palet (hepsi) |
-| Ana görüntü | Terminal / ajan ekranı; kutular yazılan karakterleri takip eder |
-| Metin | Ajan düşünce akışı, İngilizce |
-| Süre | 6 sn, 30 fps (180 kare), 12 sahne × 0,5 sn (15 kare) |
-| Ses | Kodla üretilen senkron efekt sesleri, hedef -14 LUFS |
-| Yaklaşım | İki geçiş, ikisi de fframes; ikinci geçiş Skia Metal shader |
+| Use | Style test |
+| Layers | Tracker overlay, pixel grid + bloom, tilt + blur, fast cuts + palettes (all of them) |
+| Subject | An agent's terminal; the boxes track the characters as they are typed |
+| Text | Agent thought stream, in English |
+| Length | 6 s, 30 fps (180 frames), 12 shots × 0.5 s (15 frames) |
+| Sound | Synthesized, synced sound effects, target -14 LUFS |
+| Approach | Two passes, both fframes; the second is a Skia Metal shader |
-## Sahne akışı
+## Shot list
-| # | Terminal metni | Palet | Kamera |
+| # | Terminal text | Palette | Camera |
|---|---|---|---|
-| 1 | `▍` (yanıp sönen imleç) | kâğıt beyazı | çok yakın, sola eğik |
-| 2 | `thinking…` | termal (mor→sarı) | yavaş kayma |
-| 3 | `reading 14 files` | siyah-beyaz | sağa eğik |
-| 4 | `src/render.rs` | siyah-beyaz | odak kayması |
-| 5 | `tool_call: render_frame()` | fosfor yeşili | yukarıdan bakış |
-| 6 | `attention → token 4812` | termal | hızlı yakınlaşma |
-| 7 | `not just completing —` | siyah-beyaz | yatay kayma |
-| 8 | `— composing` | kehribar | geri çekilme |
-| 9 | `verifying 180 frames` | lacivert | sabit, titreşimli |
-| 10 | `✓ 0 problems` | fosfor yeşili | eğik |
-| 11 | (bütün kutular tek noktaya toplanır) | termal | yakınlaşma |
-| 12 | `Done` | neon yeşil hap şekli | geri çekilme, siyaha kesme |
-
-### Referansın ötesindeki eklemeler
-
-1. **Gerçek koordinatlar.** Etiketlerdeki x/y değerleri, karakterin düz (geçiş 1) görüntüdeki
- gerçek piksel konumudur. Rastgele sayı yoktur.
-2. **Düşünce zinciri çizgileri.** Bağlantı çizgileri, ajanın okuduğu sırayla kutudan kutuya ilerler.
-3. **Kilitlenen odak.** Netlik düzlemi her sahnede o an takip edilen kutuya kayar (rack focus).
-4. **Kilitlenme anı.** Kutu hedefe kilitlenince 2 kare kırmızı olur; tarama çizgisi yırtılır; bip sesi çalar.
-5. **Final.** Sahne 11'de bütün kutular tek kutuya toplanır; sahne 12'de o kutu "Done" hapına dönüşür.
-
-## Mimari
+| 1 | `›` and a blinking cursor | paper white | very close, tilted left |
+| 2 | `thinking…` | thermal (purple→yellow) | slow drift |
+| 3 | `reading 14 files` | black and white | tilted right |
+| 4 | `src/render.rs` | black and white | rack focus |
+| 5 | `tool_call: render_frame()` | phosphor green | top-down |
+| 6 | `attention → token 4812` | thermal | fast push-in |
+| 7 | `not just completing —` | black and white | horizontal pan |
+| 8 | `— composing` | amber | pull back |
+| 9 | `verifying 180 frames` | navy | locked off, shaking |
+| 10 | `✓ 0 problems` | phosphor green | tilted |
+| 11 | (every box collapses into one point) | thermal | push-in |
+| 12 | `Done` | neon green pill | pull back, fade to black |
+
+### Beyond the reference
+
+1. **Real coordinates.** The x/y in each label is the box's real pixel position, not a random number.
+2. **Chain-of-thought lines.** The links run from box to box in the order the agent reads.
+3. **Locking focus.** The focal plane moves to the box being tracked in every shot (rack focus).
+4. **Lock moment.** When a box locks on, it turns red for 2 frames, the scanlines tear and a beep plays.
+5. **Ending.** In shot 11 every box collapses into one; in shot 12 that box becomes the "Done" pill.
+
+## Architecture (as built)
```
-scenes.json ──► flat (geçiş 1) ──► out/flat.mp4 + out/tracks.json
- │
-scenes.json ──────────────────────► lens (geçiş 2, Skia Metal, lens.sksl) ──► out/tracking.mp4
- ▲
- sfx (WAV üretici) ─► out/sfx/*.wav (tracks.json olaylarına göre)
+hud (shared crate: shots, tracker boxes, camera, projection)
+ ├─► flat (pass 1, 3840×2160) ──► out/pass/flat.mp4
+ ├─► tracks (bin) ──► out/tracks.json ──► tools/sfx.py ──► out/pass/sfx.wav
+ └─► lens (pass 2, Skia Metal, lens.sksl + camera overlay) ──► out/tracking.mp4
```
-### `scenes.json` (tek kaynak)
+### `hud` (single source of truth)
-Her sahne için: `text`, `palette` (ad), `camera` (eğim açıları, yakınlık, kayma yönü/hızı),
-`frames` (varsayılan 15). Paletler aynı dosyada ad → renk durakları (LUT) olarak tanımlıdır.
+`hud/src/lib.rs` holds `SCENE_LIST` (text, palette, camera per shot), the tracker
+(`track(scene, frame)`), the camera per video frame (`shot(frame)`) and `project()`, the CPU
+copy of the shader's camera. Both passes call the same functions, so they agree on every frame.
-### Geçiş 1: `flat` (fframes projesi, CPU veya Skia backend)
+### Pass 1: `flat`
-- Siyah zemin, beyaz, tek genişlikli yazı tipi. Harf harf yazım animasyonu.
-- Karakter konumu: `x = sol_kenar + sütun × karakter_genişliği`. Tek genişlikli yazı tipi
- olduğu için kesin.
-- Takip katmanı: `<pattern>` ile taralı kutular, etiketler, çizgiler. Kutular yeni yazılan
- karakterleri ve kelimeleri hedefler.
-- Çıktılar: `out/flat.mp4` (renksiz, efektsiz) ve `out/tracks.json`:
- her kare için `focus: [x, y]`, olay listesi `locks: [{frame, x, y}]`, `cuts: [frame]`.
+- Black background, white monospace text (JetBrains Mono, 0.6 em advance), typed glyph by glyph.
+- One `<text>` per glyph at `x = hero.x + column × advance`, so the tracker positions are exact.
+- Tracker boxes on the screen itself: stroked rects, some hatched with a `<pattern>` in
+ `mix-blend-mode: difference` (the stripes turn dark over a glyph, like the reference).
+- Only color: pure red (#ff0000) on a locked box; the lens keeps it red in every palette.
+- Rendered at 2× (3840×2160) because the lens magnifies it up to 3×.
-### Geçiş 2: `lens` (fframes, Skia Metal backend)
+### Pass 2: `lens`
-- `out/flat.mp4` karesini `iChannel0` olarak `lens.sksl` shader'ına verir.
-- Shader sırası: perspektif (homografi) → odak noktasına uzaklığa göre bulanıklık →
- palet LUT → parlama → renk kayması → LED ızgara maskesi → kilitlenmede yırtılma.
-- Uniform değerleri: kamera (`scenes.json`), odak ve kilitlenme (`tracks.json`), sahne içi zaman.
-- SkSL kısıtları: sabit döngü sınırları, dinamik dizi indeksi yok. Bulanıklık sabit örnek sayılı
- disk örneklemesiyle yapılır.
+- Binds the synced `flat.mp4` frame as `iChannel0` of `lens/shaders/lens.sksl`.
+- Shader order: tear → perspective (ray / tilted plane) → depth of field (circle of confusion
+ from the depth difference to the tracked target, 36 taps, chromatic aberration per channel) →
+ palette (5 stops) → bloom (16 wide taps) → LED grid (equal-area RGB stripes, fades out of
+ focus) → exposure kick on a cut, vignette, flicker, grain, final fade.
+- On top: a sharp camera-space tracker (links, a dot running along each link, coordinate
+ labels, brackets on the target), projected with `hud::project`. Label x/y = the box's
+ real pixel in the output. Reason: small labels were unreadable through the LED grid.
-### Ses: `sfx`
+### Sound: `tools/sfx.py`
-- Küçük bir script, efekt seslerini sentezler: kesmede tık/glitch, kilitlenmede kısa bip,
- sürekli düşük uğultu.
-- fframes ses haritasına `tracks.json` olay karelerine göre yerleştirilir. Hedef -14 LUFS, limiter -1 dBFS.
+Reads `out/tracks.json` and synthesizes everything: screen hum, a glitch on each cut, a
+click per keystroke, a pentatonic beep on each lock (panned to the box), a riser into the
+collapse and a chime on "Done". Loudness is measured with ffmpeg `ebur128` and corrected
+to -14 LUFS under a soft clip.
-## Riskler ve geri dönüş
+## Checks
-- **Ana risk:** geçiş 2'de video karesinin shader'a girdi olarak verilememesi.
- **Teknik test (ilk iş):** tek sahnelik `flat.mp4` → `lens` → kontrol karesi.
- Test başarısız olursa geçiş 2'yi ffmpeg filtreleriyle yaparım (`perspective`, `gblur`,
- `rgbashift`, ızgara bindirmesi) ve Patron'a bildiririm. Sahne başına palet LUT ile korunur.
-- **Performans:** shader'daki bulanıklık örnek sayısı render süresini belirler. Taslaklar yarım
- çözünürlükte render edilir.
+1. After every render: contact sheets, `inspect`, `audio analyze`.
+2. Result: `out/tracking.mp4`, 1920×1080, 6.0 s, 180 frames, -14.0 LUFS, true peak -0.9 dBTP
+ (fframes limiter at -1 dBFS).
-## Kontrol
+## Out of scope
-1. Her renderdan sonra fframes CLI ile kare özet görüntüsü (contact sheet), otomatik hata taraması
- ve ses ölçümü.
-2. Tam videodan önce Patron'a 2 sahnelik taslak gösterilir (sahne 2 ve 6: termal palet, en zor efektler).
-3. Son çıktı: `out/tracking.mp4`, 1920×1080, 6,0 sn, -14 LUFS ±1.
+- Motion tracking on real camera footage.
+- Versions longer than 6 s, music, voice-over.
-## Kapsam dışı
+## Notes from the build
-- Gerçek kamera görüntüsü üzerinde hareket takibi.
-- 6 sn'den uzun sürüm, müzik, seslendirme.
-- X'te yayınlama.
+- **The risk did not happen.** `get_synced_video_frame(..).into_image()` binds as a shader
+ child, so the ffmpeg fallback was not needed.
+- **SkSL:** `flat` is a reserved word (GLSL ES interpolation qualifier).
+- **fframes 1.0.0:** the decoder did not hand out the last frames of `flat.mp4`; the
+ workaround was 10 padding frames. A frame without a root `<svg>` stalled the render.
+- **fframes 1.2.0 (current):** the last frames arrive (#167, #172), so the padding is gone;
+ `MaxPerformance` works (lens render 33 s → 28-30 s). A frame without a root `<svg>`
+ still delays the error: the unfinished segments drain their x264 lookahead on drop first
+ (86 s instead of 28 s here), reported as fframes #209. That is why the lens draws a black
+ frame when the source frame is missing.
+- 1.0 and 1.2 output look the same (SSIM 0.95; the difference is grain and encoder noise).
-## Uygulamada değişenler (2026-10-09)
+## Run
-- **`scenes.json` yerine `hud` crate'i.** Sahne verisi, takip kutuları ve kamera ortak bir Rust
- crate'inde (`hud/src/lib.rs`). İki geçiş aynı fonksiyonları çağırır; `tracks.json` yalnızca ses
- script'i için `hud`'dan üretilir (`target/release/tracks`).
-- **İki katmanlı takip.** Kutular ve tarama deseni ekranda kalır (LED ızgarası ve bulanıklık alır).
- Etiketler, bağlantı çizgileri ve hedef köşelikleri kameranın net katmanıdır; `lens` bunları
- `hud::project` ile çizer. Etiketteki x/y, kutunun çıktı görüntüsündeki gerçek pikselidir.
- Sebep: LED ızgarasında küçük etiketler okunamıyordu.
-- **Teknik test başarılı.** `get_synced_video_frame(..).into_image()` shader'a `iChannel0` olarak
- veriliyor; ffmpeg yedeğine gerek kalmadı.
-- **Render ayarı.** `lens` tek GPU hattıyla (`OnePipeline`) çalışır. `flat` sonuna 10 yedek kare
- eklenir, çünkü çözücü son kareleri vermiyor; kare gelmezse `lens` siyah kare çizer.
-- **Ses.** Gerçekleşen değer: -14,0 LUFS, gerçek tepe -0,9 dBTP (fframes limiter -1 dBFS).
-
-## Çalıştırma
-
-`tools/render.sh` → `out/tracking.mp4` (yaklaşık 40 sn).
-
-## fframes 1.2.0'a geçiş (2026-10-09)
-
-- 1.2.0 son video karelerini veriyor (#167, #172): `flat` sonundaki 10 yedek kare kaldırıldı.
-- `MaxPerformance` 1.2.0'da kilitlenmiyor: tekrar açıldı (lens render 33 sn → 28-30 sn).
-- Eski kilitlenmenin asıl sebebi: kökünde `<svg>` olmayan bir kare (`Svgr::empty()`) tam
- render'ı hata vermeden kilitliyor. Aralık render'ı (`render 160..180`) doğru hata veriyor.
- 1.2.0'da iki modda da tekrarlandı; siyah kare yedeği bu yüzden kalıyor.
-- 1.0 ile 1.2 çıktısı gözle aynı (SSIM 0,95, fark kumlanma ve kodlama gürültüsü).
+`tools/render.sh` → `out/tracking.mp4` (about 40 s on an M3 Pro).
diff --git a/lens/shaders/lens.sksl b/lens/shaders/lens.sksl
index 85a1816..1095c97 100644
--- a/lens/shaders/lens.sksl
+++ b/lens/shaders/lens.sksl
@@ -21,6 +21,8 @@ uniform float uLock;
uniform float uCut;
uniform float uOut;
uniform float uSeed;
+// Debug: 0 perspective, 1 + depth of field, 2 + palette and aberration, 3 + bloom and LED, 4 all.
+uniform float uStage;
const float FLAT_SCALE = 2.0;
const float LW = 1920.0;
@@ -65,7 +67,7 @@ half4 main(float2 coord) {
// Tear: on a lock and right after a cut, a few rows slide sideways.
float2 px = coord;
float shake = uLock + uCut;
- if (shake > 0.0) {
+ if (shake > 0.0 && uStage >= 4.0) {
float band = floor(coord.y / 16.0);
float n = hash(float2(band, frameSeed.x * 3.7 + frameSeed.y));
float hit = step(0.93 - 0.18 * shake, n);
@@ -93,10 +95,10 @@ half4 main(float2 coord) {
float zf = (p0 + eu * fl.x + ev * fl.y).z;
float coc = clamp(uBlur * 190.0 * abs(1.0 - zf / z), 0.0, 46.0);
float logPerPx = LH / (uZoom * res.y) * z;
- float rLog = coc * logPerPx;
+ float rLog = coc * logPerPx * step(1.0, uStage);
// Chromatic aberration grows toward the edges and on a lock.
- float2 ca = s * (0.006 + 0.016 * uLock + 0.008 * uCut) * (LH / uZoom);
+ float2 ca = step(2.0, uStage) * s * (0.006 + 0.016 * uLock + 0.008 * uCut) * (LH / uZoom);
float spin = hash(coord * 0.37 + frameSeed) * 6.2831;
float3 accG = float3(0.0);
@@ -124,6 +126,7 @@ half4 main(float2 coord) {
// Red marks a locked box in the flat pass; it stays red in every palette.
float red = clamp((accG.r - accG.g) * 1.4, 0.0, 1.0);
float3 col = float3(pal(accR).r, pal(accG.g).g, pal(accB).b);
+ if (uStage < 2.0) { col = accG; }
col = mix(col, float3(1.0, 0.13, 0.10), red);
// Bloom: a wide ring of taps lifts everything around bright glyphs.
@@ -140,8 +143,9 @@ half4 main(float2 coord) {
}
glow /= float(GLOW_TAPS);
glowRed /= float(GLOW_TAPS);
- col += pal(min(1.0, glow * 1.6)) * glow * 0.75;
- col += float3(1.0, 0.1, 0.05) * glowRed * 0.9;
+ float glowOn = step(3.0, uStage);
+ col += pal(min(1.0, glow * 1.6)) * glow * 0.75 * glowOn;
+ col += float3(1.0, 0.1, 0.05) * glowRed * 0.9 * glowOn;
// LED grid on the screen surface; it fades out of focus and where cells get too small.
// Equal-area RGB stripes per cell (no tint on average) inside rounded LED rows.
@@ -153,14 +157,15 @@ half4 main(float2 coord) {
float3 led = rowm * colm * (0.4 + 1.0 * sub);
float cellPx = CELL / logPerPx;
float ledAmt = (1.0 - smoothstep(0.8, 3.5, rLog / CELL)) * smoothstep(2.2, 4.5, cellPx);
- col *= mix(float3(1.0), led * 1.3, ledAmt * 0.9);
+ col *= mix(float3(1.0), led * 1.3, ledAmt * 0.9 * glowOn);
// Exposure kick on a cut, vignette, rolling flicker, grain, final fade.
- col = col * (1.0 + uCut * 1.1) + uCut * 0.06;
+ float post = step(4.0, uStage);
+ col = col * (1.0 + uCut * 1.1 * post) + uCut * 0.06 * post;
float2 v = s * float2(0.85, 1.15);
- col *= 1.0 - 0.42 * dot(v, v);
- col *= 0.965 + 0.035 * sin(coord.y * 0.9 - iTime * 55.0);
- col += (hash(coord + frameSeed * 1.31) - 0.5) * 0.06;
+ col *= 1.0 - 0.42 * dot(v, v) * post;
+ col *= 1.0 - post * (0.035 - 0.035 * sin(coord.y * 0.9 - iTime * 55.0));
+ col += (hash(coord + frameSeed * 1.31) - 0.5) * 0.06 * post;
col *= 1.0 - uOut;
return half4(half3(clamp(col, 0.0, 1.0)), 1.0);
}
diff --git a/lens/src/lib.rs b/lens/src/lib.rs
index cd76bd3..8801f69 100644
--- a/lens/src/lib.rs
+++ b/lens/src/lib.rs
@@ -77,17 +77,23 @@ impl Video for LensVideo {
.float("uLock", if s.lock { 1.0 } else { 0.0 })
.float("uCut", s.cut)
.float("uOut", s.out)
- .float("uSeed", (si * 31 + f) as f32);
+ .float("uSeed", (si * 31 + f) as f32)
+ .float("uStage", stage());
let layer = self.shader.draw(&frame, uniforms);
fframes::svgr!(
<svg xmlns="http://www.w3.org/2000/svg" width={WIDTH} height={HEIGHT}>
<image href={layer.href()} x="0" y="0" width={WIDTH} height={HEIGHT} />
- {camera_tracker(&s, si, f)}
+ {if stage() >= 4.0 { camera_tracker(&s, si, f) } else { Svgr::empty() }}
</svg>
)
}
}
+/// `LENS_STAGE=0..3` renders the shader up to one step (see lens.sksl); unset = all.
+fn stage() -> f32 {
+ std::env::var("LENS_STAGE").ok().and_then(|v| v.parse().ok()).unwrap_or(4.0)
+}
+
const MONO: &str = "JetBrains Mono";
const LOCK_RED: &str = "#ff3b30";
const RW: f32 = WIDTH as f32;
diff --git a/media/JetBrainsMono-OFL.txt b/licenses/JetBrainsMono-OFL.txt
similarity index 100%
rename from media/JetBrainsMono-OFL.txt
rename to licenses/JetBrainsMono-OFL.txt
diff --git a/tools/readme_media.py b/tools/readme_media.py
new file mode 100644
index 0000000..185b699
--- /dev/null
+++ b/tools/readme_media.py
@@ -0,0 +1,98 @@
+"""Build the README images from rendered frames.
+
+Run from the workspace root after tools/render.sh:
+ python3 -I tools/readme_media.py
+Needs: Pillow, ffmpeg, img2webp (libwebp). Writes docs/media/{stages.jpg, shots.jpg, hero.webp}.
+"""
+
+import subprocess
+import sys
+from pathlib import Path
+
+from PIL import Image, ImageDraw, ImageFont
+
+ROOT = Path(__file__).resolve().parent.parent
+OUT = ROOT / "docs" / "media"
+WORK = ROOT / "frames" / "readme"
+FONT = ImageFont.truetype(str(ROOT / "media" / "JetBrainsMono-Bold.ttf"), 30)
+SMALL = ImageFont.truetype(str(ROOT / "media" / "JetBrainsMono-Regular.ttf"), 24)
+STAGE_FRAME = 81
+STAGES = [
+ ("flat", "1 flat pass"),
+ ("0", "2 perspective"),
+ ("1", "3 + depth of field"),
+ ("2", "4 + palette, aberration"),
+ ("3", "5 + bloom, LED grid"),
+ ("4", "6 + tear, grain, camera tracker"),
+]
+SHOTS = [
+ "›", "thinking…", "reading 14 files", "src/render.rs",
+ "tool_call: render_frame()", "attention → token 4812", "not just completing —", "— composing",
+ "verifying 180 frames", "✓ 0 problems", "collapse", "Done",
+]
+
+
+def run(cmd, env=None):
+ subprocess.run(cmd, check=True, cwd=ROOT, env=env, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+
+
+def label(img, text, font=FONT):
+ d = ImageDraw.Draw(img, "RGBA")
+ h = font.size + 30
+ d.rectangle([0, 0, img.width, h], fill=(0, 0, 0, 150))
+ d.text((22, 14), text, font=font, fill=(255, 255, 255, 255))
+ return img
+
+
+def grid(tiles, cols, w, h):
+ rows = (len(tiles) + cols - 1) // cols
+ out = Image.new("RGB", (cols * w, rows * h), "black")
+ for i, t in enumerate(tiles):
+ out.paste(t, ((i % cols) * w, (i // cols) * h))
+ return out
+
+
+def stages():
+ import os
+
+ tiles = []
+ for key, text in STAGES:
+ d = WORK / f"stage-{key}"
+ if key == "flat":
+ run(["target/release/flat", "frame", str(STAGE_FRAME), "-o", str(d)])
+ else:
+ run(["target/release/lens", "frame", str(STAGE_FRAME), "-o", str(d)], env={**os.environ, "LENS_STAGE": key})
+ img = Image.open(d / f"{STAGE_FRAME}.png").convert("RGB").resize((960, 540), Image.LANCZOS)
+ tiles.append(label(img, text))
+ grid(tiles, 3, 960, 540).save(OUT / "stages.jpg", quality=88)
+
+
+def shots():
+ frames = [s * 15 + 6 for s in range(12)]
+ run(["target/release/lens", "frame", ",".join(map(str, frames)), "-o", str(WORK / "shots")])
+ tiles = []
+ for i, f in enumerate(frames):
+ img = Image.open(WORK / "shots" / f"{f}.png").convert("RGB").resize((640, 360), Image.LANCZOS)
+ tiles.append(label(img, f"{i + 1:02d} {SHOTS[i]}", SMALL))
+ grid(tiles, 4, 640, 360).save(OUT / "shots.jpg", quality=86)
+
+
+def hero():
+ """Animated WebP (GitHub renders it inline): 960 px, 20 fps, about 4 MB."""
+ d = WORK / "hero"
+ if d.exists():
+ for p in d.iterdir():
+ p.unlink()
+ d.mkdir(parents=True, exist_ok=True)
+ run(["ffmpeg", "-y", "-i", "out/tracking.mp4", "-vf", "fps=20,scale=960:-1:flags=lanczos", str(d / "f%03d.png")])
+ frames = sorted(str(p) for p in d.glob("f*.png"))
+ run(["img2webp", "-loop", "0", "-lossy", "-q", "72", "-m", "6", "-d", "50", *frames, "-o", str(OUT / "hero.webp")])
+
+
+if __name__ == "__main__":
+ OUT.mkdir(parents=True, exist_ok=True)
+ WORK.mkdir(parents=True, exist_ok=True)
+ for step in sys.argv[1:] or ["stages", "shots", "hero"]:
+ globals()[step]()
+ for p in sorted(OUT.iterdir()):
+ print(f"{p.relative_to(ROOT)} {p.stat().st_size / 1e6:.1f} MB")