Open source CapCut alternative.
TypeScript
/**
* What the editor is allowed to hold in memory, and what it is holding.
*
* A cut is built out of expensive pictures. Decoded frames sit in the
* platform decoder, canvases sit behind every source the pool keeps warm, and
* the bytes of the files themselves sit in each reader's cache. Every one of
* those is capped somewhere, but the caps were written one subsystem at a
* time, in whatever currency that subsystem thinks in — a count of decoders, a
* number of source pixels, a number of canvas pixels, a number of open files.
* Nothing added them up, so the tab's real ceiling was whatever those numbers
* happened to sum to on the machine the change was tuned on.
*
* This is the one place the total is decided. Holders register what they are
* standing on, in bytes, and ask what they may stand on: the smaller of the
* size they were tuned to want and their share of the tab's ceiling. On a
* machine with memory to spare the tuned size is the smaller one and nothing
* about the preview changes. On a machine without, the share binds first and
* every holder sheds together, so no one subsystem eats the room the others
* needed.
*
* The ceiling is a model. The memory that actually runs a machine out is
* memory a page cannot see: decoded frames live in the platform's decoder
* process, canvas backing lives in the GPU process, and
* `measureUserAgentSpecificMemory` — which needs a cross-origin isolation this
* page does not have — reports the size of neither. So the accounting is done
* at the allocation sites, where the sizes are known exactly.
*/
/** Where the tab's memory goes. Each is a family of allocations with one
* eviction story, so a bucket is what a share can sensibly be given to. */
export type MemoryBucket =
/** Frames held inside the platform decoder, on behalf of live decoders. */
| "decoders"
/** Canvas backing: the rings frames land on, and the warm shelf. */
| "canvases"
/** File bytes held by open readers. */
| "reads"
/** Decoded sound waiting to be scheduled. */
| "audio"
/** Rasterized pictures: element bitmaps, thumbnails, mattes. */
| "pictures";
const BUCKETS: MemoryBucket[] = ["decoders", "canvases", "reads", "audio", "pictures"];
/**
* Each bucket's share of the ceiling.
*
* The split follows what the buckets actually cost at the sizes people edit
* at: a live 4K decoder holds hundreds of megabytes of frames where the
* canvases drawn from it hold tens, and the readers under both hold the file
* bytes those frames were decoded from. The shares sum to one, and a bucket
* holding less than its share does not lend the remainder — a holder that
* grows into borrowed room has to give it back mid-gesture, which is the churn
* the caps exist to prevent.
*/
const SHARES: Record<MemoryBucket, number> = {
decoders: 0.4,
canvases: 0.25,
reads: 0.25,
audio: 0.05,
pictures: 0.05,
};
/**
* The tab's share of the machine.
*
* The ceiling covers the tab and the decoder process working for it, which
* between them are one of several things the machine is running: the
* operating system, the browser's other processes, the person's other tabs.
* A quarter of what the browser reports leaves those the rest. Past that the
* machine starts swapping, and a swapping machine previews worse than one
* decoding at half the size — the picture stops for a page fault, which no
* amount of decode headroom recovers.
*
* The number the browser reports is capped at eight, so eight is the largest
* ceiling this can be a quarter of: two gigabytes, which is the figure the
* sizes below are tuned against. Taking a smaller fraction would mean no
* machine ever runs at those sizes, since none can report more.
*/
const CEILING_SHARE = 0.25;
/**
* What to assume when the browser does not say.
*
* `deviceMemory` is Chrome and Edge only; Safari and Firefox report nothing.
* What it reports is rounded down to a power of two and capped at eight, so
* eight already means "eight or more" — a laptop with that much and a
* workstation with sixty-four say the same thing. Assuming it for the
* browsers that stay silent puts them on the same footing as the machines we
* can see, which are overwhelmingly the same machines.
*/
const ASSUMED_GB = 8;
/** The smallest ceiling worth honouring: below this the editor cannot hold a
* single 1080p source open, so a machine reporting less memory still gets this
* much. */
const CEILING_MIN = 384 * 2 ** 20;
const GB = 2 ** 30;
/**
* Bytes the editor may hold.
*
* `deviceMemory` is coarse by design — it rounds to a power of two and caps at
* eight — which is enough to tell the machines that need protecting from the
* machines that do not, and it is the only figure a page is given. It is read
* each time it is asked for: the figure never moves, and the read is a
* property lookup.
*/
export function memoryCeiling(): number {
const reported =
typeof navigator === "undefined"
? 0
: ((navigator as unknown as { deviceMemory?: number }).deviceMemory ?? 0);
const gb = reported > 0 ? reported : ASSUMED_GB;
return Math.max(CEILING_MIN, Math.round(gb * GB * CEILING_SHARE));
}
/**
* Everything that stands in a bucket, and how much of the bucket each may
* stand on.
*
* The split lives here, in one table. A fraction typed into one file cannot
* see the fractions typed into the others, and the sum is the whole point: two
* caches that each ask for their bucket's share stand on twice it. Every holder is named here, the ones that only report what they
* hold included — room reserved for a cache that never asks is room the asking
* caches must not be handed. The fractions in a bucket sum to one, and a test
* says so.
*/
const HOLDERS = {
/** Frames the live decoders behind the preview are holding. */
previewDecoders: { bucket: "decoders", portion: 1 },
/** The preview pool: the ring each live source lands frames on, and the
* shelf of canvases kept for the sources that have stood down. */
previewCanvases: { bucket: "canvases", portion: 0.72 },
/** Readers a render keeps open between frames, each with its own pool of
* canvases at the source's size, and its element scratch surfaces. */
exportReaders: { bucket: "canvases", portion: 0.2 },
/** The sharpen and clarity pass's float framebuffers, at the picture's
* size while detail is in use. */
detailTargets: { bucket: "canvases", portion: 0.08 },
/** File bytes held once for every reader on a cloud file. */
chunkMemory: { bucket: "reads", portion: 0.5 },
/** What each open reader keeps of its own walk through a file. */
readerCaches: { bucket: "reads", portion: 0.5 },
/** Decoded sound waiting for the mixer to schedule it. */
mixerAudio: { bucket: "audio", portion: 1 },
/** Filmstrips and waveforms drawn for shelf entries. */
libraryPictures: { bucket: "pictures", portion: 0.25 },
/** Parsed LUT files off the Library shelf, held for the clips that use them. */
lutTables: { bucket: "pictures", portion: 0.05 },
/** The same tables held again by the LUT build worker, which bakes from
* its own copies. */
lutWorkerTables: { bucket: "pictures", portion: 0.05 },
/** Baked clip color LUTs: the lattices the compositor samples. */
gradeLuts: { bucket: "pictures", portion: 0.06 },
/** The same LUTs uploaded as 3D textures for the GPU pass. */
gradeLutTextures: { bucket: "pictures", portion: 0.04 },
/** Captured frames behind trim edges and filmstrip tiles. */
edgeFrames: { bucket: "pictures", portion: 0.3 },
/** Decoded sticker images, shared across the page's renders. */
stickers: { bucket: "pictures", portion: 0.15 },
/** Element pictures the behind-the-person pass draws from. */
overlayRasters: { bucket: "pictures", portion: 0.1 },
} as const satisfies Record<string, { bucket: MemoryBucket; portion: number }>;
/** A named cache in the budget. */
export type MemoryHolder = keyof typeof HOLDERS;
/** The table itself, for the report that names what each cache holds and for
* the test that adds each bucket's portions up. */
export const memoryHolders = HOLDERS as Record<
MemoryHolder,
{ bucket: MemoryBucket; portion: number }
>;
/**
* Bytes a holder may stand on: the smaller of the size it was tuned to want
* and its portion of its bucket's share of the ceiling.
*/
export function allowance(holder: MemoryHolder, tuned: number): number {
const { bucket, portion } = HOLDERS[holder];
const share = Math.round(memoryCeiling() * SHARES[bucket] * portion);
if (share < tuned) bound[bucket] = true;
return Math.min(tuned, share);
}
/** Buckets whose share came in under what they were tuned to want: the
* ceiling is deciding their size. Read and cleared by whoever is reporting;
* it says the machine is the constraint. */
const bound: Record<MemoryBucket, boolean> = {
decoders: false,
canvases: false,
reads: false,
audio: false,
pictures: false,
};
/** Which buckets the ceiling is currently deciding the size of, clearing the
* record. */
export function takeMemoryPressure(): MemoryBucket[] {
const hit = BUCKETS.filter((b) => bound[b]);
for (const b of hit) bound[b] = false;
return hit;
}
type Reporter = () => number;
const holders = new Map<MemoryBucket, Set<Reporter>>();
/**
* Say what a cache is standing on, so the total can be reported.
*
* Reporters are pulled: a cache that has to remember to announce every
* allocation eventually forgets one, and the number it forgot is the number a
* report needs. Returns the way to stop reporting.
*/
export function holdMemory(holder: MemoryHolder, bytes: Reporter): () => void {
const { bucket } = HOLDERS[holder];
let set = holders.get(bucket);
if (!set) holders.set(bucket, (set = new Set()));
set.add(bytes);
return () => {
set.delete(bytes);
};
}
export type MemoryUsage = Record<MemoryBucket, number> & { total: number };
/** What every registered cache is holding right now, by bucket. */
export function memoryUsage(): MemoryUsage {
const out = { decoders: 0, canvases: 0, reads: 0, audio: 0, pictures: 0, total: 0 };
for (const bucket of BUCKETS) {
let n = 0;
for (const bytes of holders.get(bucket) ?? []) n += bytes();
out[bucket] = n;
out.total += n;
}
return out;
}
/** Least time between two walks of every reporter, for callers that want a
* number rather than a peak. */
const TOTAL_TTL_MS = 1_000;
let totalAt = 0;
let total = 0;
/**
* What everything is holding, from a walk no more than a second old.
*
* Reading the total walks every cache that reports one. A caller on a path
* that is already in trouble — the record of a frame that blocked — wants the
* number without paying that walk again for a figure that cannot have moved
* far since the last one.
*/
export function memoryTotalCached(): number {
const now = Date.now();
if (now - totalAt >= TOTAL_TTL_MS) {
totalAt = now;
total = memoryUsage().total;
}
return total;
}
/**
* The JavaScript heap, where a browser reports one.
*
* It is a small part of what an editor holds — the pictures are all outside it
* — and Chrome quantizes what it returns. It is worth reporting anyway: a heap
* climbing across a session is the signature of a cache that never evicts,
* which the modeled numbers beside it cannot show.
*/
export function heapBytes(): number {
const perf = typeof performance === "undefined" ? null : performance;
const mem = (perf as unknown as { memory?: { usedJSHeapSize?: number } } | null)?.memory;
return mem?.usedJSHeapSize ?? 0;
}
const MB = 2 ** 20;
/** Bytes as whole megabytes, which is the resolution every report of this
* wants. */
export const mb = (bytes: number): number => Math.round(bytes / MB);
/** Four bytes a pixel: the canvas backing behind a picture of that size. */
export const canvasBytes = (pixels: number): number => pixels * 4;
/**
* A decoded frame of a picture that size.
*
* Hardware decoders hand back planar 4:2:0 — a full-resolution luma plane and
* two quarter-resolution chroma planes — which is where the three halves come
* from. A frame drawn at a smaller size still costs this: the decode happens
* at the file's own resolution and the scaling happens after.
*/
export const decodedFrameBytes = (pixels: number): number => Math.round(pixels * 1.5);