Write HTML. Render video. Built for agents.
JavaScript aiai-agentsanimationclaude-codeffmpegframeworkgsaphtmlhtml-to-videomcpmotion-graphicspuppeteerrenderingtypescriptvideo-editor
fix(cli): media-treatment applies one treatment to every matched image or video (#5443)
Miguel Ángel44e5a47e09a1
diff --git a/docs/packages/cli.mdx b/docs/packages/cli.mdxindex 7e4a50d8a2..702d394a17 100644--- a/docs/packages/cli.mdx+++ b/docs/packages/cli.mdx@@ -429,7 +429,7 @@ where `u2net_human_seg` falls down, and free alternatives.### `media-treatment`-Read or edit the colour and effect treatment stored on one image or video.+Read or edit the colour and effect treatment stored on images and videos.```bash# Find out what exists, then look at one family or effect@@ -454,8 +454,8 @@ npx hyperframes media-treatment --selector '#hero' --clear| `--all` | Print the exhaustive capability catalog || `--project <dir>` | Project directory (default: current directory) || `--file <path>` | Composition file (default `index.html`) |-| `--selector <css>` | CSS selector for one `<img>` or `<video>` |-| `--selector-index <n>` | Zero-based match index when the selector is not unique |+| `--selector <css>` | CSS selector; every matched `<img>` or `<video>` gets the patch |+| `--selector-index <n>` | Zero-based match index to patch only one of several matches || `--grading <json>` | Validated colour-grading patch || `--apply` | Apply the patch || `--analyze` | Measure the selected local media and suggest a bounded correction |@@ -463,7 +463,7 @@ npx hyperframes media-treatment --selector '#hero' --clear| `--dry-run` | Validate and report without writing || `--json` | Agent-readable JSON |-`--dry-run` accepts `--grading` without `--apply` and writes nothing. JSON includes `attribute`, the exact normalized `value` that apply would write (or `null` to remove it), and `lint`, the color-grading lint verdict and findings for the proposed HTML.+`--dry-run` accepts `--grading` without `--apply` and writes nothing. JSON includes `count`, `attribute`, the exact normalized `value` that apply would write (or `null` to remove it), and `lint`, the color-grading lint verdict and findings for the proposed HTML. When the selector matches several elements, each gets the patch merged into its own current grading, and JSON lists them under `targets` (each with its own `value`) instead of a single `value`; run `--clear` first for an identical look. Matches come from the main document or, when it has none, from the first `<template>` that has any, the same scope rule as Studio's source edits.The command writes `data-color-grading` — the same contract Studio, preview, andrender all read, so the two can never disagree. It does not recognise subjectsdiff --git a/packages/cli/src/commands/media-treatment.test.ts b/packages/cli/src/commands/media-treatment.test.tsindex f22dd009e8..70ecd2da29 100644--- a/packages/cli/src/commands/media-treatment.test.ts+++ b/packages/cli/src/commands/media-treatment.test.ts@@ -18,6 +18,13 @@ import {import { CliRuntimeError } from "../utils/commandResult.js";const VIDEO = `<!doctype html><html><body><video id="hero" src="hero.mp4"></video></body></html>`;+const SEGMENTS = `<!doctype html><html><body><video class="seg" src="a.mp4"></video><video class="seg" src="b.mp4"></video></body></html>`;++function gradingsOf(html: string): Array<string | null> {+ return Array.from(parseHTML(html).document.querySelectorAll("video"), (video) =>+ video.getAttribute("data-color-grading"),+ );+}describe("applyMediaTreatmentToHtml", () => {it("provides a concise first-hop overview of the complete treatment surface", () => {@@ -176,6 +183,154 @@ describe("applyMediaTreatmentToHtml", () => {expect(result.html).toContain("data-color-grading=");});+ it("treats every video a selector matches alike", () => {+ const result = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ });++ const [first, second] = gradingsOf(result.html);+ expect(result.targets).toHaveLength(2);+ expect(result.html).toContain("data-hf-id=");+ expect(first).toContain('"preset":"warm-daylight"');+ expect(second).toBe(first);+ });++ it("merges the patch into each match's own grading", () => {+ const warm = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ selectorIndex: 1,+ grading: { preset: "warm-daylight" },+ }).html;++ const result = applyMediaTreatmentToHtml(warm, {+ selector: "video",+ grading: { intensity: 0.5 },+ });++ expect(result.targets.map(({ before }) => before)).toEqual([+ null,+ expect.objectContaining({ preset: "warm-daylight" }),+ ]);+ expect(result.targets[1]?.value).toContain('"preset":"warm-daylight"');+ expect(result.targets[0]?.value ?? "").not.toContain("warm-daylight");+ });++ it("clears every match even when the selector depends on the cleared attribute", () => {+ const graded = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ }).html;++ const result = applyMediaTreatmentToHtml(graded, {+ selector: "video[data-color-grading]",+ clear: true,+ });++ expect(gradingsOf(result.html)).toEqual([null, null]);+ });++ it("patches the matched element even when a write changes which elements a selector matches", () => {+ const four = SEGMENTS.replace(+ "</body>",+ '<video src="c.mp4"></video><video src="d.mp4"></video></body>',+ );+ const graded = applyMediaTreatmentToHtml(four, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ }).html;++ const result = applyMediaTreatmentToHtml(graded, {+ selector: "video:has(+ video[data-color-grading])",+ clear: true,+ });++ expect(result.targets).toHaveLength(3);+ expect(gradingsOf(result.html).map((value) => value === null)).toEqual([+ true,+ true,+ true,+ false,+ ]);+ });++ it("refuses a video inside svg instead of writing a tag that swallows what follows it", () => {+ const html = `<svg><foreignObject><video src="a.mp4"></video><p>caption</p></foreignObject></svg>`;++ expect(() =>+ applyMediaTreatmentToHtml(html, { selector: "video", grading: { preset: "warm-daylight" } }),+ ).toThrow("inside <svg>");+ });++ it("patches the matched element when ids in the file are duplicated", () => {+ const html = `<img data-hf-id="hf-x1" src="a.png"><video data-hf-id="hf-x1" src="b.mp4"></video>`;++ const result = applyMediaTreatmentToHtml(html, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ });++ const { document } = parseHTML(`<!doctype html><html><body>${result.html}</body></html>`);+ expect(document.querySelector("img")?.getAttribute("data-color-grading")).toBeNull();+ expect(document.querySelector("video")?.getAttribute("data-color-grading")).toContain(+ "warm-daylight",+ );+ });++ it("clears exactly the matches inside a template when a write changes the match set", () => {+ const videos = '<video src="a.mp4"></video>'.repeat(4);+ const graded = applyMediaTreatmentToHtml(`<template>${videos}</template>`, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ }).html;++ const result = applyMediaTreatmentToHtml(graded, {+ selector: "video:has(+ video[data-color-grading])",+ clear: true,+ });++ const cleared = (result.html.match(/<video(?![^>]*data-color-grading)/g) ?? []).length;+ expect(result.targets).toHaveLength(3);+ expect(cleared).toBe(3);+ });++ it("writes a later match's change when the first match is already graded", () => {+ const firstGraded = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ selectorIndex: 0,+ grading: { preset: "warm-daylight" },+ }).html;++ const result = applyMediaTreatmentToHtml(firstGraded, {+ selector: "video",+ grading: { preset: "warm-daylight" },+ });++ expect(result.targets.map(({ changed }) => changed)).toEqual([false, true]);+ expect(result.changed).toBe(true);+ expect(gradingsOf(result.html)[1]).toContain("warm-daylight");+ });++ it("treats only the --selector-index match when one is given", () => {+ const result = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ selectorIndex: 1,+ grading: { preset: "warm-daylight" },+ });++ const [first, second] = gradingsOf(result.html);+ expect(first).toBeNull();+ expect(second).toContain('"preset":"warm-daylight"');+ });++ it("rejects a selector that also matches something other than an image or video", () => {+ const html = SEGMENTS.replace("</body>", '<div class="seg"></div></body>');++ expect(() =>+ applyMediaTreatmentToHtml(html, { selector: ".seg", grading: { preset: "warm-daylight" } }),+ ).toThrow("selector matched <div>");+ });+it("normalizes and persists advanced grading on real media", () => {const result = applyMediaTreatmentToHtml(VIDEO, {selector: "#hero",@@ -388,11 +543,13 @@ describe("applyMediaTreatmentToHtml", () => {).toThrow(/Cannot merge.*unresolved whole-grade variable/);});- it("requires an unambiguous media target", () => {+ it("grades every matched image unless --selector-index picks one", () => {const source = `<img class="media" src="a.png"><img class="media" src="b.png">`;- expect(() =>- applyMediaTreatmentToHtml(source, { selector: ".media", grading: { preset: "neutral" } }),- ).toThrow(/matched 2 elements/);+ const all = applyMediaTreatmentToHtml(source, {+ selector: ".media",+ grading: { preset: "warm-daylight" },+ });+ expect((all.html.match(/data-color-grading/g) ?? []).length).toBe(2);const result = applyMediaTreatmentToHtml(source, {selector: ".media",@@ -470,6 +627,73 @@ describe("applyMediaTreatmentToHtml", () => {}});+ it("reports how many elements an --apply treated", async () => {+ const project = mkdtempSync(join(tmpdir(), "hf-media-all-"));+ const log = vi.spyOn(console, "log").mockImplementation(() => undefined);+ writeFileSync(join(project, "index.html"), SEGMENTS);+ try {+ await runCommand(mediaTreatmentCommand, {+ rawArgs: [+ "--project",+ project,+ "--selector",+ "video",+ "--grading",+ '{"preset":"warm-daylight"}',+ "--apply",+ "--json",+ ],+ });++ const applied = JSON.parse(String(log.mock.calls.at(-1)?.[0]));+ expect(applied).toMatchObject({ ok: true, count: 2, changed: true });+ expect(+ applied.targets.map(({ selectorIndex }: { selectorIndex: number }) => selectorIndex),+ ).toEqual([0, 1]);+ expect(gradingsOf(readFileSync(join(project, "index.html"), "utf8"))[1]).toContain(+ "warm-daylight",+ );+ } finally {+ log.mockRestore();+ rmSync(project, { recursive: true, force: true });+ }+ });++ it("reports apply when any match keeps a grading after the patch", async () => {+ const project = mkdtempSync(join(tmpdir(), "hf-media-action-"));+ const html = applyMediaTreatmentToHtml(SEGMENTS, {+ selector: "video",+ selectorIndex: 1,+ grading: { wheels: { shadows: { hue: 205, amount: 0.08, level: 0.02 } } },+ }).html;+ const log = vi.spyOn(console, "log").mockImplementation(() => undefined);+ writeFileSync(join(project, "index.html"), html);+ try {+ await runCommand(mediaTreatmentCommand, {+ rawArgs: [+ "--project",+ project,+ "--selector",+ "video",+ "--grading",+ '{"preset":"neutral"}',+ "--apply",+ "--json",+ ],+ });++ const applied = JSON.parse(String(log.mock.calls.at(-1)?.[0]));+ expect(applied.targets.map(({ value }: { value: string | null }) => value === null)).toEqual([+ true,+ false,+ ]);+ expect(applied.action).toBe("apply");+ } finally {+ log.mockRestore();+ rmSync(project, { recursive: true, force: true });+ }+ });+it("requires --apply for --grading while keeping --clear explicit", async () => {const project = mkdtempSync(join(tmpdir(), "hf-media-treatment-"));const file = join(project, "index.html");diff --git a/packages/cli/src/commands/media-treatment.ts b/packages/cli/src/commands/media-treatment.tsindex cb694a31ce..ca61c3dea1 100644--- a/packages/cli/src/commands/media-treatment.ts+++ b/packages/cli/src/commands/media-treatment.ts@@ -19,7 +19,11 @@ import {resolveExistingLocalAsset,} from "@hyperframes/parsers/asset-resolution";import { rewriteAssetPath } from "@hyperframes/parsers/asset-paths";-import { patchElementInHtml } from "@hyperframes/studio-server/source-mutation";+import {+ isHTMLElement,+ parseSourceDocument as parseMutableSource,+} from "@hyperframes/studio-server/source-mutation";+import { ensureHfIds } from "@hyperframes/parsers/hf-ids";import { defineCommand } from "citty";import { lintHyperframeHtml } from "@hyperframes/lint";import { parseHTML } from "linkedom";@@ -338,15 +342,20 @@ interface ApplyMediaTreatmentOptions {clear?: boolean;}-interface ApplyMediaTreatmentResult {- html: string;- changed: boolean;+interface MediaTreatmentTargetResult {+ selectorIndex: number;tag: "img" | "video";+ changed: boolean;value: string | null;before: unknown;after: unknown;}+interface ApplyMediaTreatmentResult extends Omit<MediaTreatmentTargetResult, "selectorIndex"> {+ html: string;+ targets: MediaTreatmentTargetResult[];+}+function parseSourceDocument(source: string): Document {if (/<!doctype|<html[\s>]/i.test(source)) return parseHTML(source).document;return parseHTML(`<!DOCTYPE html><html><body>${source}</body></html>`).document;@@ -432,12 +441,13 @@ function queryIncludingTemplates(root: Document | Element, selector: string): Elreturn [];}-function selectMediaElement(- source: string,+type MediaTarget = { element: Element; selectorIndex: number; tag: "img" | "video" };++function selectMediaElements(+ document: Document,selector: string,selectorIndex?: number,-): { element: Element; selectorIndex: number; tag: "img" | "video" } {- const document = parseSourceDocument(source);+): MediaTarget[] {let matches: Element[];try {matches = queryIncludingTemplates(document, selector);@@ -445,46 +455,64 @@ function selectMediaElement(throw new Error(`Invalid selector: ${selector}`);}if (matches.length === 0) throw new Error(`Selector did not match: ${selector}`);- if (selectorIndex === undefined && matches.length > 1) {- throw new Error(- `Selector matched ${matches.length} elements; use a unique selector or --selector-index`,- );+ if (selectorIndex !== undefined && !matches[selectorIndex]) {+ throw new Error(`--selector-index ${selectorIndex} is outside ${matches.length} matches`);}+ const indexes = selectorIndex === undefined ? matches.keys() : [selectorIndex];+ return Array.from(indexes, (index) => {+ const element = matches[index]!;+ const tag = element.tagName.toLowerCase();+ if (tag !== "img" && tag !== "video") {+ throw new Error(`Color grading requires an <img> or <video>; selector matched <${tag}>`);+ }+ if (!isHTMLElement(element)) {+ throw new Error(`Color grading requires an HTML <${tag}>; selector matched one inside <svg>`);+ }+ return { element, selectorIndex: index, tag };+ });+}- const resolvedIndex = selectorIndex ?? 0;- const element = matches[resolvedIndex];- if (!element) {- throw new Error(`--selector-index ${resolvedIndex} is outside ${matches.length} matches`);- }- const tag = element.tagName.toLowerCase();- if (tag !== "img" && tag !== "video") {- throw new Error(`Color grading requires an <img> or <video>; selector matched <${tag}>`);+function selectMediaElement(source: string, selector: string, selectorIndex?: number): MediaTarget {+ const targets = selectMediaElements(parseSourceDocument(source), selector, selectorIndex);+ if (targets.length > 1) {+ throw new Error(+ `Selector matched ${targets.length} elements; use a unique selector or --selector-index`,+ );}- return { element, selectorIndex: resolvedIndex, tag };+ return targets[0]!;}export function applyMediaTreatmentToHtml(source: string,options: ApplyMediaTreatmentOptions,): ApplyMediaTreatmentResult {- const { element, selectorIndex, tag } = selectMediaElement(- source,- options.selector,- options.selectorIndex,+ const { document, wrappedFragment } = parseMutableSource(source);+ const targets = selectMediaElements(document, options.selector, options.selectorIndex).map(+ ({ element, selectorIndex, tag }) => {+ const current = element.getAttribute(HF_COLOR_GRADING_ATTR);+ const before = parseStoredGrading(current);+ const value = options.clear ? null : serializeGradingPatch(before, options.grading);+ const changed = current !== value;+ if (changed) {+ if (value === null) element.removeAttribute(HF_COLOR_GRADING_ATTR);+ else element.setAttribute(HF_COLOR_GRADING_ATTR, value);+ }+ return { selectorIndex, tag, changed, value, before, after: parseStoredGrading(value) };+ },);- const before = parseStoredGrading(element.getAttribute(HF_COLOR_GRADING_ATTR));-- const value = options.clear ? null : serializeGradingPatch(before, options.grading);-- const changed = element.getAttribute(HF_COLOR_GRADING_ATTR) !== value;- const after = parseStoredGrading(value);- if (!changed) return { html: source, changed: false, tag, value, before, after };-- const patched = patchElementInHtml(source, { selector: options.selector, selectorIndex }, [- { type: "attribute", property: HF_COLOR_GRADING_ATTR, value },- ]);- if (!patched.matched) throw new Error(`Could not persist selector: ${options.selector}`);- return { html: patched.html, changed: true, tag, value, before, after };+ const changed = targets.some((target) => target.changed);+ const serialized = wrappedFragment ? document.body.innerHTML || "" : document.toString();+ const html = changed ? ensureHfIds(serialized) : source;+ const first = targets[0]!;+ return {+ html,+ changed,+ tag: first.tag,+ value: first.value,+ before: first.before,+ after: first.after,+ targets,+ };}function parseSelectorIndex(raw: string | undefined): number | undefined {@@ -657,7 +685,9 @@ async function prepareMutation(args: MediaTreatmentCommandArgs) {const lint = { ok: findings.every(({ severity }) => severity !== "error"), findings };if (result.changed && !dryRun) writeFileSync(filePath, result.html);- const action: "clear" | "apply" = result.value === null ? "clear" : "apply";+ const action: "clear" | "apply" = result.targets.every(({ value }) => value === null)+ ? "clear"+ : "apply";return {action,result,@@ -667,14 +697,19 @@ async function prepareMutation(args: MediaTreatmentCommandArgs) {action,file: relative(project.dir, filePath) || "index.html",selector,- selectorIndex: selectorIndex ?? 0,- tag: result.tag,+ count: result.targets.length,+ ...(result.targets.length === 1+ ? {+ selectorIndex: result.targets[0]!.selectorIndex,+ tag: result.tag,+ before: result.before,+ after: result.after,+ value: result.value,+ }+ : { targets: result.targets }),changed: result.changed,dryRun,- before: result.before,- after: result.after,attribute: HF_COLOR_GRADING_ATTR,- value: result.value,lint,},};@@ -705,7 +740,12 @@ async function printMutation(args: MediaTreatmentCommandArgs): Promise<void> {return;}const verb = mutationVerb(action, result.changed, payload.dryRun);- console.log(`${c.success("◇")} ${verb} media treatment on ${c.accent(selector)}`);+ const changedCount = result.targets.filter(({ changed }) => changed).length;+ const count =+ result.targets.length > 1+ ? ` (${changedCount} of ${result.targets.length} elements ${payload.dryRun ? "would change" : "changed"})`+ : "";+ console.log(`${c.success("◇")} ${verb} media treatment on ${c.accent(selector)}${count}`);}function printFailure(error: unknown, json: boolean): void {@@ -742,11 +782,11 @@ export const mediaTreatmentCommand = defineCommand({},selector: {type: "string",- description: "Unique CSS selector for one <img> or <video>",+ description: "CSS selector; every matched <img> or <video> gets the patch",},"selector-index": {type: "string",- description: "Zero-based match index when the selector is not unique",+ description: "Zero-based match index to patch only one of several matches",},grading: { type: "string", description: "Canonical color-grading JSON patch" },apply: {diff --git a/packages/cli/src/help.ts b/packages/cli/src/help.tsindex 68e2539543..b9e04f181e 100644--- a/packages/cli/src/help.ts+++ b/packages/cli/src/help.ts@@ -46,7 +46,7 @@ const GROUPS: Group[] = [["snapshot", "Capture key frames as PNG screenshots for visual verification"],["media-treatment",- "Discover, apply, or clear deterministic media treatments on one media element",+ "Discover, apply, or clear deterministic media treatments on media elements",],["normalize-audio",diff --git a/skills-manifest.json b/skills-manifest.jsonindex e3fcf72bd0..228c680a78 100644--- a/skills-manifest.json+++ b/skills-manifest.json@@ -54,7 +54,7 @@"files": 1},"media-use": {- "hash": "206565a9b0cc6ac2",+ "hash": "1a841e4288d862af","files": 110},"motion-graphics": {diff --git a/skills/media-use/references/media-treatments.md b/skills/media-use/references/media-treatments.mdindex 6651b28049..38c2facffa 100644--- a/skills/media-use/references/media-treatments.md+++ b/skills/media-use/references/media-treatments.md@@ -159,9 +159,13 @@ drops frames. Do not impose or claim a universal hard cap from one machine.GSAP only for supported animated values and Registry overlay blocks onlyfor authored dressing.```bash- hyperframes media-treatment --selector "<unique selector>" \+ hyperframes media-treatment --selector "<selector>" \--grading '<nested JSON patch>' --apply --json```+ A selector that matches several images or videos patches every match, each+ merged into its own current grading (the JSON reports `count` and each+ target); run `--clear` first for an identical look, or add+ `--selector-index` to patch only one.For a temporal reveal, use the focused capability result's `animation`contract. If it is `null`, the capability is static. Author the starting CSSproperty inline on the real media element and return temporary treatment