Open source CapCut alternative.
TypeScript
fix(cut): an imported link comes back playable [rebuild]
David Lef472cddf330c
diff --git a/docs/guides/cut/ai-assistant.md b/docs/guides/cut/ai-assistant.mdindex 586fca0b..9bb59568 100644--- a/docs/guides/cut/ai-assistant.md+++ b/docs/guides/cut/ai-assistant.md@@ -88,7 +88,7 @@ With a tab attached, tool calls execute in it against the same store the user is- **Watching leaves a record.** Contact sheets live in one conversation and the media trim drops the oldest of them, so the model writes each look down: a note-source call stores what a span showed — quoted on-screen text, what happens, where the cuts land — against its source seconds, on the asset, saved with the project. Notes come back in the editor snapshot as `observed`, in full from get-state, and on every watch result beside the spans still undescribed. A close note supersedes the coarse one over the same ground, and a watch of a source that still owes a note is refused, which is what makes an hour-long source workable: it is surveyed coarsely, described, then re-watched densely only where the work is.- **Speech cuts land on the words.** A recut edge is not left where the model's arithmetic put it. The audio around it is re-read, that recording's own room tone and voice level are measured, and the edge is placed a beat clear of where a word truly starts or finishes — tail and soft attack included, so a quiet take and a noisy one are cut the same way. Pace belongs to that step: fast, natural and relaxed set how big the beat is, and the fastest of them still lets every word finish. So "make it punchy" shortens the air rather than cutting into the speech, and where a pause is too short for both sides' air the two edges meet inside it instead of one clipping a word.- **Media rides a byte budget.** Contact sheets, audio, and attachment payloads all replay inline, so before each call the session is trimmed to a 16MB media budget — oldest parts leave first, with a note naming the tools that fetch them again. Text, tool calls, tool results, and thought signatures always survive, and the trim runs on a copy so the stored session keeps everything.-- **A link comes back as whatever it holds.** The chat's URL import works down a ladder: the bundled downloader for anything with a stream, X's own endpoints for a post's photos or Article, and otherwise the page itself — a content extractor (Defuddle over a parsed DOM) lifts the article out as markdown and the pictures it kept come along as assets. Words with no media are a successful import, so a link the user pasted to explain something reads fine, and that text is quoted for the user beside whatever media came with it.+- **A link comes back as whatever it holds.** The chat's URL import works down a ladder: the bundled downloader for anything with a stream, X's own endpoints for a post's photos or Article, and otherwise the page itself — a content extractor (Defuddle over a parsed DOM) lifts the article out as markdown and the pictures it kept come along as assets. Words with no media are a successful import, so a link the user pasted to explain something reads fine, and that text is quoted for the user beside whatever media came with it. Whatever comes down is H.264 by the time it lands: the downloader sorts on codec before resolution, and a source that publishes nothing Apple can decode — the VP9 and AV1 streams Instagram and YouTube serve at their top tiers — is converted on the way in, so an import plays on the phone, in the Mac's own player, and in a browser.- **Async when rendering.** Video generation outruns the two-minute tool window, so those tools start the job and return immediately; the clip files under the chat that asked — the owning thread is captured at call time — and lands on the timeline only when the user asked for that. Each render's card state (running, landed, failed) mirrors into the project doc as it changes, so the chat shows the same outcome on machines that never ran the job.- **Chat owns what it makes.** Media created by chat tools is tagged with its thread and previews on a card in the conversation. Placing it on the timeline or filing it into Media or the Library transfers ownership; deleting the thread deletes whatever it still owns.- **Chat media is the account's storage.** It is real project media: it counts toward the cloud quota and the project's size like an import, and the thread is the only place it can be seen or deleted from. So conversations travel wherever their media travels — a project moved between the Mac and the cloud, an owner's duplicate, and a share copy with chat shared all carry the threads with the files. A copy that took the files alone would hold media nobody can find, delete, or explain, and the user would still be paying for it.diff --git a/site/src/cut/server/urlDownload.ts b/site/src/cut/server/urlDownload.tsindex f058eab4..13438b66 100644--- a/site/src/cut/server/urlDownload.ts+++ b/site/src/cut/server/urlDownload.ts@@ -4,6 +4,7 @@ import { mkdir, readdir, stat, writeFile } from "node:fs/promises";import path from "node:path";import { Defuddle } from "defuddle/node";import { parseHTML } from "linkedom";+import { convertToMp4, fitsAlready, streamCodecs } from "./convert";import type { LibrarySource } from "./library";// Bring back whatever a URL points at. yt-dlp, resolved from PATH, pulls@@ -149,12 +150,12 @@ async function downloadMedia(url: string, tmp: string): Promise<Downloaded> {const sized = await Promise.all(names.map(async (n) => ({ n, size: (await stat(path.join(out, n))).size })));- const file = sized.sort((a, b) => b.size - a.size)[0].n;+ const file = await playableMedia(path.join(out, sized.sort((a, b) => b.size - a.size)[0].n));// The site's own cover for this video, so a card and the viewer have a first// frame to show before a byte of the video is decoded.const poster = await savePoster(meta.thumbnail, out, path.parse(file).name);const clip = {- file: path.join(out, file),+ file,title: (meta.title || "Imported clip").slice(0, 120),...(poster ? { poster } : {}),};@@ -180,6 +181,26 @@ const PAGE_EXTRACTORS = new Set(["generic", "html5"]);const IMAGE_URL_RE = /\.(png|jpe?g|webp|gif|avif|bmp)(?:$|\?)/i;+/** What comes back has to play where the import is going — the phone's viewer,+ * the Mac's player, a browser — and a site's best stream is often VP9 or AV1,+ * which Apple decodes nowhere. A file whose picture is already H.264 is handed+ * back untouched; anything else is re-encoded into one that plays. A+ * conversion that fails leaves the original: a file that plays in fewer places+ * beats no import at all. */+async function playableMedia(file: string): Promise<string> {+ const codecs = await streamCodecs(file);+ // Sound alone decodes everywhere the import goes.+ if (!codecs.video) return file;+ if (fitsAlready(file, codecs, { shrink: false, sdr: false })) return file;+ const out = path.join(path.dirname(file), `${path.parse(file).name}.h264.mp4`);+ try {+ await convertToMp4({ log: [], outPath: out, progress: 0, tmpDir: path.dirname(file) }, file, out);+ return out;+ } catch {+ return file;+ }+}+/** Save the source's cover image beside the media it belongs to. A cover that* won't download is no reason to fail an import, so this answers nothing and* the media lands without one. */@@ -722,10 +743,15 @@ function ytDlp(// Take the best video and the best audio and put them together. Asking for// a ready-made mp4 first would land whatever single file the site keeps// around for compatibility — on YouTube that is 360p next to a 4K original.- // Codec preference is a sort, never a filter, so an mp4-friendly stream- // wins ties without a resolution ever being traded for it.+ //+ // Codec leads the sort. Apple decodes H.264 and HEVC and nothing else, so+ // a taller VP9 or AV1 stream is a file that plays nowhere the import is+ // going: the phone's viewer, the Mac's own player, Safari. Instagram+ // serves 1440p VP9 beside 1080p H.264, and the 1080p file is the one worth+ // having. A source with no H.264 at all still comes down, and the+ // conversion below makes it playable."-f", "bestvideo*+bestaudio/best",- "-S", "res,fps,hdr:sdr,vcodec:h264,acodec:aac",+ "-S", "vcodec:h264,acodec:aac,res,fps,hdr:sdr","--merge-output-format", "mp4","-o", path.join(dir, "%(id)s.%(ext)s"),"--print-json",