Skip to content

Open source CapCut alternative.

TypeScript

fix(site): a library clip lands the moment it is dropped

David Le6aff3af3ce12
diff --git a/docs/guides/cut/README.md b/docs/guides/cut/README.md
index 8eed0492..8c530455 100644
--- a/docs/guides/cut/README.md
+++ b/docs/guides/cut/README.md
@@ -58,7 +58,7 @@ Decoders live in a bounded pool, and leaving the pool is suspension before it is
Two things follow from that. A cut is a read from a different ring rather than a handover between players, so crossing a join costs nothing and needs no clock to change hands; and the picture answers a moved playhead inside a frame, because the answer was already in memory. Sound is scheduled on one audio context and its clock is what playback runs on — the picture follows the sound rather than the two chasing each other. When a browser holds that context suspended, as it does until the page has been interacted with, wall time carries the cut and the audio clock is picked up once it agrees — and when the clock jumps instead of agreeing, a context resumed late or an output device switched mid-play, every scheduled window restarts where the clock now stands rather than playing out the moment it was placed for. A context that stops running mid-play is asked back every frame, because the picture would otherwise carry on against wall time over a cut gone silent.
-Each voice reads its file the way the picture's readers do: a walk held open, pulled forward about a second at a time. Reading in order is what keeps sound and picture in the same part of the file. A voice asking for a long span at once is asking for the whole interleaved stretch that span is scattered through — twenty seconds of sound is a hundred megabytes of a phone recording — so over a link that merely keeps up with playback its every read arrives after the moment it was for, and a cut that sounded for its first stretch plays out in silence. The rule about endings is the picture's rule, for the same reason: a walk reports the same ending whether it read the track's last sample or gave up on it, and an ending believed a few seconds into a long clip plays the rest of the cut in silence while the picture reads happily past it. So an ending short of the clip is treated as a walk that failed and another goes after the sound it did not reach, and a walk that gets no further than the one before it has found where the track really stops — which is often a fraction before the picture does. Nothing else about a read is final either. A read that fails, comes back empty, or stops answering leaves the voice waiting a moment and asking again, quickly at first and then on a slow cadence for as long as the failure holds, and it asks the link keeper for a fresh URL on the way — the same footing the picture's readers are on, which is what stops a blip from silencing a clip whose picture healed seconds later. Opening a walk is nearly all a walk costs — a container read out of a hundred megabytes over a link — so a voice holding sound for a moment that has gone by re-aims the reader it already has, the way the picture's readers are re-aimed. Giving the file up there would start the open again a second behind where it just was, and again after that: the voice pays the open for the rest of the play and never once catches the clock, which is the same silence arrived at from the other direction. The reader moves for a new address too, and the sound already read plays out across the change. A reader is opened before it is needed, on the same footing as the picture's: a clip under a parked playhead has had a decoder open on it all along, which is why the picture starts on the frame, and the sound now opens its file and takes one read where the playhead stands rather than doing all of it once the clock is already running. That is what a play straight after a seek used to wait seconds for.
+Each voice reads its file the way the picture's readers do: a walk held open, pulled forward about a second at a time. Reading in order is what keeps sound and picture in the same part of the file. A voice asking for a long span at once is asking for the whole interleaved stretch that span is scattered through — twenty seconds of sound is a hundred megabytes of a phone recording — so over a link that merely keeps up with playback its every read arrives after the moment it was for, and a cut that sounded for its first stretch plays out in silence. The rule about endings is the picture's rule, for the same reason: a walk reports the same ending whether it read the track's last sample or gave up on it, and an ending believed a few seconds into a long clip plays the rest of the cut in silence while the picture reads happily past it. So an ending short of the clip is treated as a walk that failed and another goes after the sound it did not reach, and a walk that gets no further than the one before it has found where the track really stops — which is often a fraction before the picture does. Nothing else about a read is final either. A read that fails, comes back empty, or stops answering leaves the voice waiting a moment and asking again, quickly at first and then on a slow cadence for as long as the failure holds, and it asks the link keeper for a fresh URL on the way — the same footing the picture's readers are on, which is what stops a blip from silencing a clip whose picture healed seconds later. Opening a walk is nearly all a walk costs — a container read out of a hundred megabytes over a link — so a voice holding sound for a moment that has gone by re-aims the reader it already has, the way the picture's readers are re-aimed. Giving the file up there would start the open again a second behind where it just was, and again after that: the voice pays the open for the rest of the play and never once catches the clock, which is the same silence arrived at from the other direction. The reader moves for a new address too — a library clip copied into the project behind the editor moves off the URL it arrived on mid-play — and the sound already read plays out across the change. A reader is opened before it is needed, on the same footing as the picture's: a clip under a parked playhead has had a decoder open on it all along, which is why the picture starts on the frame, and the sound now opens its file and takes one read where the playhead stands rather than doing all of it once the clock is already running. That is what a play straight after a seek used to wait seconds for.
What is on screen at a given instant is still a pure function of the document and a time, and it is still the same function an export asks. The preview reaches an instant by playing toward it and an export by jumping straight there; because both draw through one compositor from one plan, the file cannot quietly disagree with what was watched.
@@ -94,7 +94,7 @@ Cloud projects follow the account: the same project opens in any browser on any
Every client call goes through a backend seam (`site/src/cut/lib/backend/`): the local driver is the engine transport unchanged, the cloud driver rewrites the same paths to `/api/cut-cloud/*` with the session cookie, and the browser driver answers storage routes in the page from origin-private storage, falling through to the cloud twin for AI. Residency is per project and fixed when the project is created, so creating one is where the user chooses. One picker makes that choice, standing beside New project in the sidebar: it names a shelf — Local or Cloud — and everything the browser creates lands there, new projects and anything added to the library alike. Local is this Mac's engine when the app has connected and this browser's own storage otherwise, so a machine without the app still has a real local shelf. The choice sticks on that browser until they change it, and every other New project button follows it. Inside a folder the folder decides, since a project files only into a folder beside it. The home lists every shelf as one listing — the Mac's from its stored listing when the app is closed — each project badged with where it lives, and opening one binds the editor to that project's backend. Capability flags on the driver hide what a backend can't do, so components never feature-detect.
-The library is the exception, because it is the user's one shelf across places: every residency keeps a library, and the Library lists them at once as one listing, newest first, with a badge saying which shelf each item is on. So a cloud project can pull a clip off this Mac, and a browser project can pull one out of the cloud. A shelf that lives in the page holds its media in the same origin-private storage its projects use, and a link imported onto it rides the cloud worker — a page can't fetch a video off a YouTube link — with the bytes coming back down and the staged cloud copy dropped after. The Mac's shelf takes that same route as a second try: sites refuse fetches on the strength of the tool making them, so a link the engine's own tools can't bring back is fetched by the worker and lands here anyway. Each shelf is remembered on its own, so the Library tab keeps listing the Mac's items with the app closed, read-only, and clicking one raises the same banner an unreachable local project does. The editor's library picker asks only for what it can reach, since an item it cannot copy into the project has no business in a picker. Each item's own shelf answers for it: previews, renames, and deletes go to the backend it came from, and folders belong to one shelf, so items file only into folders beside them. Adding to a project is the one crossing: on the project's own shelf the server copies the file without the bytes leaving it, and off it the browser carries them down and back up. A copy the server makes alone is quick enough to wait for, so the asset arrives whole — stored name, stored URL, in the saved document, with nothing to repoint once it is playing. A carry between shelves is minutes on a phone recording and too long to hold a drop for, so that one appears at once on the library's own URL and settles onto the stored file when the bytes land. Filing an item into a folder on another shelf crosses the same way — the bytes come down and go back up, a file with its cover and a template with its own media, and the copy left behind comes off.
+The library is the exception, because it is the user's one shelf across places: every residency keeps a library, and the Library lists them at once as one listing, newest first, with a badge saying which shelf each item is on. So a cloud project can pull a clip off this Mac, and a browser project can pull one out of the cloud. A shelf that lives in the page holds its media in the same origin-private storage its projects use, and a link imported onto it rides the cloud worker — a page can't fetch a video off a YouTube link — with the bytes coming back down and the staged cloud copy dropped after. The Mac's shelf takes that same route as a second try: sites refuse fetches on the strength of the tool making them, so a link the engine's own tools can't bring back is fetched by the worker and lands here anyway. Each shelf is remembered on its own, so the Library tab keeps listing the Mac's items with the app closed, read-only, and clicking one raises the same banner an unreachable local project does. The editor's library picker asks only for what it can reach, since an item it cannot copy into the project has no business in a picker. Each item's own shelf answers for it: previews, renames, and deletes go to the backend it came from, and folders belong to one shelf, so items file only into folders beside them. Adding to a project is the one crossing: on the project's own shelf the server copies the file without the bytes leaving it, and off it the browser carries them down and back up. Filing an item into a folder on another shelf crosses the same way — the bytes come down and go back up, a file with its cover and a template with its own media, and the copy left behind comes off.
The phone feeds the cloud shelf. Clips recorded in the iOS app upload themselves to it, marked as camera footage; that marker gives the desktop its Camera Roll — a tab of phone clips kept out of the main Library grid, and a locked Camera Roll folder in the editor's library panel so the footage drops into any project. Inspiration saved on the phone lands in an Inspiration folder on the same shelf, links riding the same worker import every shelf uses. What the phone put on the shelf it also mirrors: deleting a clip from the Camera Roll or the Library tombstones it, and the phone deletes its own copy on the next pass, so the roll on the phone and the roll at the desk stay one set. Notes are edited on both sides: the phone and the desktop's Notes tab work the same set, the newer edit winning, deletes carrying across. Notes file into folders of their own, on the shelf's folder table: the desktop files them by dragging onto the same folder tiles the Library uses, the phone by a folder list it opens like any other, and a folder made on either side is on the other by the next pass. Every request from the app carries a client marker, the server remembers accounts it has seen it from, and only those accounts get the phone tabs — a desktop-only account's sidebar never changes. The app's own guide covers the phone side: [The iOS App](../ios-app.md).
diff --git a/site/src/cut/components/Timeline.tsx b/site/src/cut/components/Timeline.tsx
index e38d4a73..69d811a2 100644
--- a/site/src/cut/components/Timeline.tsx
+++ b/site/src/cut/components/Timeline.tsx
@@ -1613,17 +1613,16 @@ export function Timeline() {
only?: (asset: MediaAsset) => boolean
) => {
const next = runCursor(t, atElement);
- // Each item is copied into the project before it can be placed, so a drop
- // of several starts every copy at once and lays them out in the order they
- // were dragged. A copy that fails leaves the rest of the run alone.
- const copied = await Promise.all(
- items.map((item) => importLibraryAsset(projectId, item).catch(() => null))
- );
- for (const asset of copied) {
- if (!asset || asset.type === "font") continue;
- if (only && !only(asset)) continue;
- const sticker = !!stickerOf(asset);
- placeAssetAt(asset.id, asset.type, next(asset, sticker), audioRow, place);
+ for (const item of items) {
+ try {
+ const asset = await importLibraryAsset(projectId, item);
+ if (asset.type === "font") continue;
+ if (only && !only(asset)) continue;
+ const sticker = !!stickerOf(asset);
+ placeAssetAt(asset.id, asset.type, next(asset, sticker), audioRow, place);
+ } catch {
+ // A copy that fails leaves the rest of the run alone.
+ }
}
};
diff --git a/site/src/cut/lib/library.ts b/site/src/cut/lib/library.ts
index 6bffeeaf..493a58c9 100644
--- a/site/src/cut/lib/library.ts
+++ b/site/src/cut/lib/library.ts
@@ -16,11 +16,8 @@ import {
MAX_FONT_BYTES,
presignedUpload,
probeFileMeta,
- registerStoredMedia,
- withRetries,
uploadProjectMediaTo,
} from "./media";
-import { localMediaUrl } from "./mediaSync";
import {
activeResidency,
availableResidencies,
@@ -684,57 +681,17 @@ async function copyLibraryFileToProject(
);
}
-/**
- * Register a library asset in the open project's media, without placing it on
- * the timeline. Callers choose where it lands.
- *
- * On the asset's own shelf the copy is the server's to make and no byte passes
- * through this tab, so the asset waits for it and arrives whole: the stored
- * name, the stored URL, in the saved document from the first frame. Nothing
- * about it moves afterwards, which is what a play across it needs — an asset
- * that changes address mid-play makes every reader open its file again.
- *
- * A carry between shelves is the bytes coming down here and going back up,
- * which is minutes on a phone recording and far too long to hold a drop for.
- * That one lands on the library's own URL and moves to the stored file when
- * the copy finishes.
- */
+/** Register a library asset in the open project's media, without placing it on
+ * the timeline. Callers choose where it lands. Usable immediately: it plays
+ * from the library's own route while the copy into the project runs behind
+ * the editor (server-side on the asset's own shelf, download-and-upload
+ * across residencies). */
export async function importLibraryAsset(
projectId: string,
lib: LibraryAsset,
): Promise<MediaAsset> {
if (lib.type === "font")
throw new Error("Fonts are used from the font menu, not the timeline.");
- const assetId = lib.id;
- if (assetId && lib.residency === getBackend().kind) {
- const fileName = await withRetries((opts) =>
- copyLibraryFileToProject(
- projectId,
- lib.residency,
- lib.fileName,
- assetId,
- opts,
- ),
- );
- const url =
- (await localMediaUrl(projectId, fileName)) ?? mediaUrl(projectId, fileName);
- const asset = registerStoredMedia(projectId, {
- fileName,
- url,
- name: lib.name,
- type: lib.type,
- duration: lib.duration,
- width: lib.width,
- height: lib.height,
- });
- // A stored file's route URL re-signs and redirects on every range read and
- // is on no origin the chunk cache keeps. The mint moves it onto the media
- // origin before anything plays it.
- void import("./mediaLinks")
- .then((m) => m.refreshSignedUrls(true))
- .catch(() => {});
- return asset;
- }
return importRemote(
projectId,
{
diff --git a/site/src/cut/lib/media.ts b/site/src/cut/lib/media.ts
index 72991324..2757a0c7 100644
--- a/site/src/cut/lib/media.ts
+++ b/site/src/cut/lib/media.ts
@@ -693,7 +693,7 @@ type RemoteImportInit = {
/** Copy attempts ride out transient blips before the queue marks the asset
* failed; an abort (asset deleted, project left) stops immediately. */
-export async function withRetries(
+async function withRetries(
copy: (opts?: { onProgress?: (fraction: number) => void; signal?: AbortSignal }) => Promise<string>,
opts?: { onProgress?: (fraction: number) => void; signal?: AbortSignal }
): Promise<string> {
@@ -769,56 +769,6 @@ export function importRemote(
return asset;
}
-/**
- * Register media the project already holds as an asset that is final the
- * moment it appears: its own stored name, its own URL, in the saved document,
- * and nothing to repoint out from under a play.
- *
- * The library's own shelf copies a file across server-side, without a byte
- * passing through this tab, so the asset can wait for it and arrive whole
- * rather than arriving on the library's URL and moving to the stored one
- * mid-play.
- */
-export function registerStoredMedia(
- projectId: string,
- init: {
- fileName: string;
- url: string;
- name: string;
- type: AssetType;
- duration: number;
- width?: number;
- height?: number;
- origin?: StoredAsset["origin"];
- }
-): MediaAsset {
- const asset: MediaAsset = {
- id: uid(),
- fileName: init.fileName,
- name: init.name,
- type: init.type,
- duration: init.duration,
- ...(init.width !== undefined ? { width: init.width } : {}),
- ...(init.height !== undefined ? { height: init.height } : {}),
- ...(init.origin ? { origin: init.origin } : {}),
- url: init.url,
- };
- useEditor.getState().addAsset(asset);
- void enrichAsset(asset);
- // Catalog dimensions are nominal (an aspect, not the file's pixels): read
- // the real ones behind the placement so the doc stores the truth.
- if (init.type === "video") {
- void probeMediaFile(init.url)
- .then((m) => {
- if (m.hasVideo) {
- useEditor.getState().updateAsset(asset.id, { width: m.width, height: m.height });
- }
- })
- .catch(() => {});
- }
- return asset;
-}
-
/** Store a fetchable image (a stock tile) in the project's media as a
* first-class image asset at its native resolution — no video baking — and
* register it, without placing it on the timeline. Callers choose where it