Open source CapCut alternative.
TypeScript
docs: the preview's memory ceiling
David Led60a5529cea6
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.mdindex 400632e7..db28df15 100644--- a/docs/guides/cut/README.md+++ b/docs/guides/cut/README.md@@ -60,6 +60,12 @@ Some machines cannot decode the footage as fast as it plays, and no reader can iDecoders live in a bounded pool, and leaving the pool is suspension before it is closure: an idle source gives up its decoder but keeps the parsed file and the canvases its frames land on, so the next visit wakes it for the cost of one keyframe seek. The canvases are the part that must not churn — a montage of short clips replayed for a few minutes, allocating a fresh set on every reopen, floods the GPU process until frames stop arriving at all. They also must not pile up, so the warm shelf of stood-down sources is capped by the memory it holds rather than by a count of them: a small stage keeps a montage's whole cast, a large one keeps a few, and what does not fit closes for real. A file that moves to a new address mid-play — an import landing in project storage, a signed link re-minted ahead of expiry — keeps its source: the frames already decoded carry the picture while a fresh reader opens on the new address behind them, and the walk goes on from wherever the playhead stands. Closing there instead would put a cold open in front of the very next frame, and the picture would freeze for exactly as long as the open takes.+Those caps are shares of one number. A cut is built out of expensive pictures — frames held inside the platform's decoder, canvases behind every source the pool keeps warm, the bytes of the files they were decoded from — and each of those was capped in whatever currency its own subsystem thought in: a count of decoders, a number of pixels, a number of open files. The tab's real ceiling was whatever those happened to add up to on the machine the tuning was done on. Now every cache says what it holds in bytes and asks what it may hold, and the answer is the smaller of the size it was tuned to want and its share of a quarter of the machine's memory. Where there is room the tuned size is the smaller one and the preview behaves as it was tuned to. Where there is not, the share binds first and every cache sheds together.++The ceiling is a model. The memory that runs a machine out is memory a page cannot see — decoded frames sit inside the platform's decoder, canvas backing sits in the GPU process, and the browser reports the size of neither — so the counting happens at the allocation sites, where the sizes are known exactly. What the browser does report is roughly how much memory the machine has, which is enough to tell a laptop from a workstation.++An editor left in a background tab lets go of the expensive part: every decoder and the canvases behind them, the element pictures, the mattes, and the sound decoded ahead. Standing a source down is something the frame loop does, and a hidden tab is given almost no frames, so that memory would sit out of the machine's reach for as long as the person is away doing something else. The canvas keeps showing the frame it last painted, and coming back costs one keyframe seek per visible clip. The timeline's own strips stay, because they are what the person sees the moment the tab is theirs again.+Two things follow from that. A cut is a read from a different ring rather than a handover between players, so crossing a join costs nothing and needs no clock to change hands; and the picture answers a moved playhead inside a frame, because the answer was already in memory. Sound is scheduled on one audio context and its clock is what playback runs on — the picture follows the sound rather than the two chasing each other. When a browser holds that context suspended, as it does until the page has been interacted with, wall time carries the cut and the audio clock is picked up once it agrees — and when the clock jumps instead of agreeing, a context resumed late or an output device switched mid-play, every scheduled window restarts where the clock now stands rather than playing out the moment it was placed for. A context that stops running mid-play is asked back every frame, because the picture would otherwise carry on against wall time over a cut gone silent.Each voice reads its file the way the picture's readers do: a walk held open, pulled forward about a second at a time. Reading in order is what keeps sound and picture in the same part of the file. A voice asking for a long span at once is asking for the whole interleaved stretch that span is scattered through — twenty seconds of sound is a hundred megabytes of a phone recording — so over a link that merely keeps up with playback its every read arrives after the moment it was for, and a cut that sounded for its first stretch plays out in silence. The rule about endings is the picture's rule, for the same reason: a walk reports the same ending whether it read the track's last sample or gave up on it, and an ending believed a few seconds into a long clip plays the rest of the cut in silence while the picture reads happily past it. So an ending short of the clip is treated as a walk that failed and another goes after the sound it did not reach, and a walk that gets no further than the one before it has found where the track really stops — which is often a fraction before the picture does. Nothing else about a read is final either. A read that fails, comes back empty, or stops answering leaves the voice waiting a moment and asking again, quickly at first and then on a slow cadence for as long as the failure holds, and it asks the link keeper for a fresh URL on the way — the same footing the picture's readers are on, which is what stops a blip from silencing a clip whose picture healed seconds later. Opening a walk is nearly all a walk costs — a container read out of a hundred megabytes over a link — so a voice holding sound for a moment that has gone by re-aims the reader it already has, the way the picture's readers are re-aimed. Giving the file up there would start the open again a second behind where it just was, and again after that: the voice pays the open for the rest of the play and never once catches the clock, which is the same silence arrived at from the other direction. The reader moves for a new address too — a library clip copied into the project behind the editor moves off the URL it arrived on mid-play — and the sound already read plays out across the change. A reader is opened before it is needed, on the same footing as the picture's: a clip under a parked playhead has had a decoder open on it all along, which is why the picture starts on the frame, and the sound now opens its file and takes one read where the playhead stands rather than doing all of it once the clock is already running. That is what a play straight after a seek used to wait seconds for. The first window a standing start schedules — a play begun, a seek landed, a picture-hold released — is a short one, so the sound joins the picture's first frame and the windows behind it extend the lead by whole groups; and a play held for a clip still opening primes its sound during the hold, so the picture and the sound open together and the release finds both ready.@@ -74,7 +80,7 @@ Whether any of this is true of a given build is measured rather than assumed: `nThe machine it measures on is part of the measurement. A developer desktop has decode slots to spare and never falls behind, so the preview's caps never bind there and a change that would ruin a laptop reads as a clean sheet. `--machine` puts the page on a smaller one: it takes away hardware decode slots, so decoders past the limit deliver their frames late the way software decoding does, it slows the main thread to a fraction of this one, and it paces the media over a link the way a cloud project reads it. That last one is its own axis: a reader short of bytes and a reader short of decode both show a picture drifting behind the sound, each takes its own fix, and `--net` and `--rtt` open the link up on a failing case to say which it was. The fixtures cover the shape that binding actually happens in — one long clip on the timeline, read the way a cloud project reads it, played cold and played again after the editor has been closed and reopened. `npm run eval:cut-perf-lowend` is the profile the reports come from. What the gates ask there is what a viewer would ask — did the frames arrive, did the picture hitch a quarter second behind the sound, and was the last pass as good as the first — and each machine has its own budget for those, because a number that is honest on a desktop is fantasy on a fifteen-watt laptop.-A profile it cannot model is the one someone is actually editing on, so the preview can report on itself. Settings carries a diagnostics switch, off until an account turns it on; with it on, the engine counts what its played frames did — how many arrived late, how far the picture sat behind its own sound, what the decoders and the warm shelf were holding, how long the main thread was blocked — and sends one summary per half minute of playback, alongside what the machine brought: cores, memory, the size it is decoding at, and which codecs it decodes in hardware. That last one separates two stutters that look identical from outside: a machine spending its time after the decoder, and one whose decoder refused the footage and put every stream on the CPU. A summary also carries how many walks the window started and the worst wait for a single frame off one, which separates the third: a reader whose bytes had not arrived waits seconds on one frame and nothing on the next, where a decoder that is merely slow answers every one of them in tens of milliseconds. The summaries are numbers; no frame, no sound, and nothing naming the project leaves the browser.+A profile it cannot model is the one someone is actually editing on, so the preview can report on itself. Settings carries a diagnostics switch, off until an account turns it on; with it on, the engine counts what its played frames did — how many arrived late, how far the picture sat behind its own sound, what every cache was holding and how near that came to this machine's ceiling, how long the main thread was blocked — and sends one summary per half minute of playback, alongside what the machine brought: cores, memory, the size it is decoding at, and which codecs it decodes in hardware. The record of a blocked frame carries the memory total as well, which is what separates a machine short of processor from one that stopped for a page fault — the two are identical from outside, and the second is the one a smaller ceiling fixes. That last one separates two stutters that look identical from outside: a machine spending its time after the decoder, and one whose decoder refused the footage and put every stream on the CPU. A summary also carries how many walks the window started and the worst wait for a single frame off one, which separates the third: a reader whose bytes had not arrived waits seconds on one frame and nothing on the next, where a decoder that is merely slow answers every one of them in tens of milliseconds. The summaries are numbers; no frame, no sound, and nothing naming the project leaves the browser.## Color grading