Record your screen, ship a demo. Free and open-source, GPU-accelerated, no watermarks, no subscriptions. Windows, macOS, Linux. Actively maintained.
TypeScript cross-platformelectronffmpegopen-sourceproduct-demorustscreen-capturescreen-recorderscreen-recordingscreen-studioscreencastspeech-to-texttypescriptvideo-editingvideo-editorwaylandwgpuwhisper
feat(effects): shader effects in the document, and the agent tools
EtienneLescot1b704ff6efc3
diff --git a/crates/compositor/src/shader_fx.rs b/crates/compositor/src/shader_fx.rsindex cda3ff6bf..50da290a7 100644--- a/crates/compositor/src/shader_fx.rs+++ b/crates/compositor/src/shader_fx.rs@@ -852,6 +852,33 @@ mod tests {assert_eq!((r.frame_reads, r.tier), (1, Tier::Light));}+ /// The examples agents read in the MCP resource (`electron/mcp/effects-contract.ts`) compile+ /// against this prelude, at the tier the resource promises.+ #[test]+ fn the_examples_of_the_mcp_contract_compile() {+ let src = include_str!("../../../electron/mcp/effects-contract.ts");+ let examples: Vec<String> = src+ .split("## Example")+ .skip(1)+ .map(|section| {+ section+ .lines()+ .filter_map(|l| l.strip_prefix(" "))+ .collect::<Vec<_>>()+ .join("\n")+ })+ .collect();+ assert_eq!(examples.len(), 2, "le ripple et la vignette");+ let ripple = [+ decl("strength", ShaderParamType::Number),+ decl("radius", ShaderParamType::Number),+ decl("duration", ShaderParamType::Number),+ ];+ let vignette = [decl("depth", ShaderParamType::Number), decl("tint", ShaderParamType::Color)];+ assert_eq!(analyze(&examples[0], &ripple).expect("ripple").report.tier, Tier::Light);+ assert_eq!(analyze(&examples[1], &vignette).expect("vignette").report.tier, Tier::Light);+ }+#[test]fn the_active_regions_and_their_progress() {let mut a = test_effect(VIGNETTE, vec![], &[]);diff --git a/docs/shader-effects.md b/docs/shader-effects.mdindex c9611976c..a26594596 100644--- a/docs/shader-effects.md+++ b/docs/shader-effects.md@@ -78,15 +78,15 @@ export const shaderEffectSchema = z.object({id: z.string().min(1), // "fx_…"name: z.string().min(1).max(60), // ce que la pilule affichedescription: z.string().max(500).default(""), // pour l'agent suivant : ce que l'effet fait- contract: z.literal(1),+ contract: z.number().int().positive(), // pas un littéral : voir C.5code: z.string().min(1).max(65_536), // plafond technique, pas un budget (C.7)params: z.array(shaderEffectParamSchema).max(32).default([]),origin: z.enum(["agent", "user"]).default("agent"),-});+ cost: z.object({ tier, frameReads }).optional(), // estimation du compositeur, en cache+}).superRefine(/* noms uniques, min ≤ default ≤ max */);export const shaderEffectRangeSchema = endGteStart(z.object({id: z.string().min(1), // "fxr_…", un par fragment- groupId: z.string().min(1).optional(), // partagé par les fragments d'une même pilulestartMs: z.number().nonnegative(), // cache dérivé de l'ancre, comme les zoomsendMs: z.number().nonnegative(),...clipAnchorShape,@@ -97,19 +97,23 @@ export const shaderEffectRangeSchema = endGteStart(z.object({}), "endMs", "startMs");// documentSchemaShape-shaderEffects: z.array(shaderEffectSchema).default([]),-shaderEffectRanges: z.array(shaderEffectRangeSchema).default([]),+shaderEffects: z.array(shaderEffectSchema).optional(),+shaderEffectRanges: z.array(shaderEffectRangeSchema).optional(),```- **`values`** ne porte que ce qui diffère du `default` de la définition. Une valeur hors bornes- est ramenée dans les bornes au parse, sans erreur, comme les réglages du look.+ est ramenée dans ses bornes à l'écriture (les outils) et à la lecture (la scène), sans erreur.+ Le parse ne le peut pas : les bornes sont dans la définition, pas dans la région.- **Les `label`** sont écrits par l'agent, dans la langue de l'utilisateur, et affichés telsquels. Ils ne passent pas par l'i18n : c'est du contenu, pas de l'interface.-- **`groupId`** reprend le principe du `trackId` des pistes audio. Une région coupée par un trim- ou une frontière de clip reste une seule pilule. Sa progression (`ctx.region`) court sur la- pilule entière, au lieu de repartir de zéro à chaque fragment.-- **Une définition que plus aucune région n'utilise est retirée** à la sauvegarde. Le document ne- garde pas de code mort.+- **Une pilule, ce sont les fragments identiques et contigus** (`regionIdentityKey`), comme pour+ les zooms. Il n'y a pas de clé de groupe : une région coupée par un trim ou une frontière de+ clip reste une seule pilule, et sa progression (`ctx.region`) court sur la pilule entière (B.5).+- **`cost`** est l'estimation du compositeur (C.7), mise en cache quand le code a été vérifié. Elle+ est dérivée, jamais une entrée : un code modifié la recalcule.+- **Une définition que plus aucune région n'utilise est retirée** par l'édition qui a retiré sa+ dernière région (`removeRegion`, `mapAllRegionCollections`). Le document ne garde pas de code+ mort.- **Les `max` du schéma sont des plafonds techniques, pas des budgets.** Ils protègent le document(64 Ko de code, 32 paramètres, 8 lanes) et sont loin au-dessus de tout effet raisonnable. Ce quel'agent doit viser, ce sont les paliers de C.7.@@ -146,9 +150,10 @@ ms brutes, sans ancre : elle restait en place pendant que les clips bougeaient.### B.4 Persistance-- **Champs additifs, sans montée de `schemaVersion`.** C'est la convention d'`audioTracks` et de- `vocabulary` : `.default([])`, et `z.object` qui ignore les clés inconnues. Un document écrit- avant cette fonctionnalité s'ouvre tel quel.+- **Champs additifs et optionnels, sans montée de `schemaVersion`.** C'est la convention de+ `vocabulary` : `.optional()`, et `z.object` qui ignore les clés inconnues. Un document écrit+ avant cette fonctionnalité s'ouvre tel quel, un document sans effet n'en porte aucune trace, et+ tout lecteur prend une liste absente pour vide.- **Le prix de ce choix.** Une build antérieure qui ouvre puis sauvegarde un projet perd seseffets en silence. Voir K, question 1.- **Pas dans les style presets.** Un preset porte le look du projet, jamais son contenu@@ -500,7 +505,7 @@ lecture :"effectDefinitions": [{ "effectId": "fx_…", "name": "Ripple au clic","description": "Une onde part de chaque clic et réfracte l'image.",- "contract": 1, "lines": 22, "regions": 1, "compiles": true,+ "contract": 1, "lines": 22, "regions": 1,"cost": { "tier": "light", "frameReads": 4 },"params": [ { "name": "strength", "type": "number", "label": "Force","default": 1, "min": 0, "max": 2 } ] }@@ -512,7 +517,8 @@ lecture :- **Le code se lit par `getEffect`,** quand l'agent veut le modifier.- **`description` est écrite par l'agent** et lue par l'agent suivant. La description de l'outill'exige : ce que l'effet fait, en une phrase, pas comment il le fait.-- **`compiles`** dit si le code compile sur cette machine. `null` signifie « pas vérifié ici ».+- **`cost`** est l'estimation mise en cache dans la définition. `null` signifie que rien n'a pu+ vérifier le code quand il a été écrit.- **`timeBaseNote`** nomme les effets parmi les régions en secondes de règle.### E.3 Les retours@@ -568,16 +574,16 @@ quatre ajouts d'une ligne, sur le modèle de `cameraSegmentation` :### E.7 Le banc (workbench)-Deux scénarios, sur une mire avec une piste curseur :+Un scénario, `effect-ripple-on-click`, sur une mire avec trois clics :-- **« ripple au clic ».** DSL attendu : une lecture du curseur, puis un `addEffect` qui couvre- les clics. Comportement attendu : l'effet compile et le DSL le relit.-- **« corrige l'effet ».** Le document porte un effet qui ne compile plus. Attendu : `getEffect`,- puis un `setEffectDefinition` qui compile.+- **Attendu :** un `addEffect` dont le code lit `ctx.clicks`, une région qui couvre chaque clic et+ la demi-seconde qui suit, et la cible `image`, puisque rien ne demande les textes.+- **Pas exigé :** lire la piste du curseur. Le compositeur donne déjà les clics placés sur la+ sortie, c'est tout l'intérêt du contrat (C.2).`documentInvariants` valide les deux nouvelles listes : chaque région pointe une définition-existante, et ses `values` sont dans les bornes. Le harnais passe un validateur factice à-`runChat`.+existante, et ses `values` sont dans les bornes. Hors ligne, rien ne compile le code : le résultat+dit `validated: false`, et les checks lisent le code comme du texte.---diff --git a/electron/ai-edition/agent-tools.test.ts b/electron/ai-edition/agent-tools.test.tsindex 9cfc65449..2d16fc3f2 100644--- a/electron/ai-edition/agent-tools.test.ts+++ b/electron/ai-edition/agent-tools.test.ts@@ -192,6 +192,7 @@ describe("the mutating-tool table", () => {"addAnnotation","addAudio","addCameraFullscreen",+ "addEffect","addSpeed","addTrim","addTrims",@@ -211,6 +212,8 @@ describe("the mutating-tool table", () => {"setAudio","setCameraFullscreen","setClipRange",+ "setEffect",+ "setEffectDefinition","setEditorSettings","setSpeed","setTrim",@@ -3734,3 +3737,178 @@ describe("clip structure (#1180)", () => {});});});++describe("shader effects", () => {+ const CODE = "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv) * params.level; }";+ const params = [+ { type: "number", name: "level", label: "Level", default: 1, min: 0, max: 2 },+ { type: "color", name: "tint", label: "Tint", default: "#000000" },+ ];+ // Stands in for the compositor's check: refuses `nope`, prices every effect at 4 reads.+ const checkShaderEffect = (code: string) =>+ code.includes("nope")+ ? { error: "error: no definition in scope for identifier: `nope`" }+ : { tier: "light" as const, frameReads: 4, iterations: 0 };+ const options = { checkShaderEffect };+ const run = (document: AxcutDocument, name: string, args: unknown) =>+ executeAgentTool(document, name, JSON.stringify(args), options);+ const add = (document: AxcutDocument, args: Record<string, unknown> = {}) =>+ run(document, "addEffect", {+ startSec: 2,+ endSec: 6,+ effect: { name: "Dim", description: "Dims the picture.", code: CODE, params },+ ...args,+ });++ it("writes the effect and poses it, anchored, with only the values that differ", () => {+ const added = add(fixtureDocument(), { values: { level: 0.5, tint: "#000000" } });+ expect(added.ok).toBe(true);+ const doc = added.document as AxcutDocument;+ const [effect] = doc.shaderEffects ?? [];+ const [range] = doc.shaderEffectRanges ?? [];+ expect(effect).toMatchObject({+ name: "Dim",+ contract: 1,+ origin: "agent",+ cost: { tier: "light", frameReads: 4 },+ });+ expect(range).toMatchObject({+ effectId: effect.id,+ startMs: 2000,+ endMs: 6000,+ clipId: "clip_1",+ values: { level: 0.5 },+ target: "image",+ lane: 0,+ });+ expect(JSON.parse(added.resultJson)).toMatchObject({+ effectRegionId: range.id,+ effectId: effect.id,+ validated: true,+ cost: { tier: "light", busiestTier: "light" },+ });+ expect(() => documentSchema.parse(doc)).not.toThrow();+ });++ it("refuses code that does not compile, with the compiler's error, and changes nothing", () => {+ const refused = add(fixtureDocument(), {+ effect: {+ name: "Broken",+ description: "",+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return nope; }",+ },+ });+ expect(refused.ok).toBe(false);+ expect(refused.document).toBeUndefined();+ expect(JSON.parse(refused.resultJson).error).toMatch(/does not compile[\s\S]*nope/);+ });++ it("takes either a new effect or one of the project's, not both or neither", () => {+ const doc = add(fixtureDocument()).document as AxcutDocument;+ const effectId = doc.shaderEffects?.[0].id;+ expect(run(doc, "addEffect", { startSec: 8, endSec: 9 }).ok).toBe(false);+ const reused = run(doc, "addEffect", { startSec: 8, endSec: 9, effectId });+ expect(reused.ok).toBe(true);+ expect(reused.document?.shaderEffects).toHaveLength(1);+ expect(reused.document?.shaderEffectRanges).toHaveLength(2);+ });++ it("keeps one lane free of overlaps, and points to another lane", () => {+ const doc = add(fixtureDocument()).document as AxcutDocument;+ const blocked = add(doc, { startSec: 4, endSec: 8 });+ expect(blocked.ok).toBe(false);+ expect(JSON.parse(blocked.resultJson).error).toMatch(/lane 0[\s\S]*another lane/);+ expect(add(doc, { startSec: 4, endSec: 8, lane: 1 }).ok).toBe(true);+ });++ it("recommends a lighter tier when a heavy effect runs long, and never refuses it", () => {+ const heavy = {+ ...options,+ checkShaderEffect: () => ({ tier: "heavy" as const, frameReads: 100 }),+ };+ const added = executeAgentTool(+ fixtureDocument(),+ "addEffect",+ JSON.stringify({+ startSec: 0,+ endSec: 40,+ effect: { name: "Blur", description: "", code: CODE, params },+ }),+ heavy,+ );+ expect(added.ok).toBe(true);+ expect(JSON.parse(added.resultJson).cost.advice).toMatch(/heavy over 40\.0 s/);+ });++ it("re-tunes, retargets and moves a region; a value back at its default is dropped", () => {+ const doc = add(fixtureDocument(), { values: { level: 0.5 } }).document as AxcutDocument;+ const regionId = doc.shaderEffectRanges?.[0].id;+ const tuned = run(doc, "setEffect", {+ effectRegionId: regionId,+ values: { level: 1, tint: "#ff0000" },+ target: "both",+ startSec: 10,+ endSec: 12,+ });+ expect(tuned.ok).toBe(true);+ expect(tuned.document?.shaderEffectRanges?.[0]).toMatchObject({+ startMs: 10000,+ endMs: 12000,+ target: "both",+ values: { tint: "#ff0000" },+ });+ });++ it("changes the definition for every region, recompiles, and drops a removed parameter's values", () => {+ const doc = add(fixtureDocument(), { values: { tint: "#ff0000" } }).document as AxcutDocument;+ const effectId = doc.shaderEffects?.[0].id;+ const refused = run(doc, "setEffectDefinition", {+ effectId,+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return nope; }",+ });+ expect(refused.ok).toBe(false);+ const changed = run(doc, "setEffectDefinition", { effectId, params: [params[0]] });+ expect(changed.ok).toBe(true);+ expect(changed.document?.shaderEffectRanges?.[0].values).toEqual({});+ expect(JSON.parse(changed.resultJson)).toMatchObject({ regions: 1, validated: true });+ });++ it("reads an effect's code by getEffect, and lists effects without it in the snapshot", () => {+ const doc = add(fixtureDocument()).document as AxcutDocument;+ const effectId = doc.shaderEffects?.[0].id;+ const read = JSON.parse(run(doc, "getEffect", { effectId }).resultJson);+ expect(read).toMatchObject({+ code: CODE,+ regions: [{ startSec: 2, endSec: 6, target: "image", lane: 0 }],+ });+ const snapshot = JSON.parse(run(doc, "getCurrentDocument", {}).resultJson);+ expect(snapshot.effects).toEqual([+ expect.objectContaining({ effectId, name: "Dim", startSec: 2, endSec: 6 }),+ ]);+ expect(snapshot.effectDefinitions[0]).toMatchObject({ effectId, lines: 1, regions: 1 });+ expect(snapshot.effectDefinitions[0].code).toBeUndefined();+ expect(snapshot.timeBaseNote).toMatch(/effects/);+ });++ it("removes a region by removeModifier, and its definition with the last one", () => {+ const doc = add(fixtureDocument()).document as AxcutDocument;+ const removed = run(doc, "removeModifier", { id: doc.shaderEffectRanges?.[0].id });+ expect(removed.ok).toBe(true);+ expect(JSON.parse(removed.resultJson).kind).toBe("shaderEffect");+ expect(removed.document?.shaderEffectRanges).toEqual([]);+ expect(removed.document?.shaderEffects).toEqual([]);+ });++ it("says when nothing could check the code", () => {+ const added = executeAgentTool(+ fixtureDocument(),+ "addEffect",+ JSON.stringify({+ startSec: 1,+ endSec: 2,+ effect: { name: "Dim", description: "", code: CODE, params },+ }),+ );+ expect(JSON.parse(added.resultJson)).toMatchObject({ validated: false, cost: null });+ });+});diff --git a/electron/ai-edition/agent-tools.ts b/electron/ai-edition/agent-tools.tsindex 3a60dd38d..4de94e670 100644--- a/electron/ai-edition/agent-tools.ts+++ b/electron/ai-edition/agent-tools.ts@@ -47,10 +47,25 @@ import { setDocumentWordText } from "../../src/lib/ai-edition/document/transcripimport {type AxcutAnnotationRegion,type AxcutDocument,+ type AxcutShaderEffect,+ type AxcutShaderEffectRange,blurDataSchema,clipCropRegionSchema,figureDataSchema,+ shaderEffectParamSchema,+ shaderEffectSchema,+ shaderEffectValueSchema,} from "../../src/lib/ai-edition/schema";+import {+ busiestShaderEffectReads,+ findShaderEffect,+ SHADER_EFFECT_CONTRACT,+ type ShaderEffectCheck,+ type ShaderEffectChecker,+ shaderEffectCostAdvice,+ shaderEffectTierOf,+ storedShaderEffectValues,+} from "../../src/lib/ai-edition/shaderEffects";import { getEditorSettings } from "../../src/lib/ai-edition/store/editorSettings";import { appendAutoZoomSuggestions } from "../../src/lib/ai-edition/timeline/apply-auto-zooms";import { hasAnyClipWithCamera } from "../../src/lib/ai-edition/timeline/camera";@@ -135,6 +150,8 @@ function modifierIds(document: AxcutDocument, kind: ModifierKind): string[] {// `removeRegion` applies the same grouping by routing this kind through// `removeAudioTrack`.return [...new Set(document.audioTracks.map(trackGroupId))];+ case "shaderEffect":+ return (document.shaderEffectRanges ?? []).map((region) => region.id);}}@@ -590,6 +607,7 @@ function modifierIdsOf(document: AxcutDocument): string[] {...document.annotations.map((r) => r.id),...speedRegions.map((r) => r.id),...cameraFullscreenRegions.map((r) => r.id),+ ...(document.shaderEffectRanges ?? []).map((r) => r.id),];}@@ -987,6 +1005,48 @@ export const addCameraFullscreenArgs = z.object({endSec: secondsSchema,});+const effectTargetSchema = z.enum(["image", "text", "both"]);+const effectLaneSchema = z.number().int().min(0).max(7);+const effectValuesSchema = z.record(z.string(), shaderEffectValueSchema);++export const addEffectArgs = z.object({+ startSec: secondsSchema,+ endSec: secondsSchema,+ effectId: z.string().min(1).optional(),+ effect: z+ .object({+ name: z.string().min(1).max(60),+ description: z.string().max(500),+ code: z.string().min(1).max(65_536),+ params: z.array(shaderEffectParamSchema).max(32).optional(),+ })+ .optional(),+ values: effectValuesSchema.optional(),+ target: effectTargetSchema.optional(),+ lane: effectLaneSchema.optional(),+});++export const setEffectArgs = z.object({+ effectRegionId: z.string().min(1),+ startSec: secondsSchema.optional(),+ endSec: secondsSchema.optional(),+ values: effectValuesSchema.optional(),+ target: effectTargetSchema.optional(),+ lane: effectLaneSchema.optional(),+});++export const setEffectDefinitionArgs = z.object({+ effectId: z.string().min(1),+ name: z.string().min(1).max(60).optional(),+ description: z.string().max(500).optional(),+ code: z.string().min(1).max(65_536).optional(),+ params: z.array(shaderEffectParamSchema).max(32).optional(),+});++export const getEffectArgs = z.object({+ effectId: z.string().min(1),+});+export const setCameraFullscreenArgs = z.object({cameraFullscreenId: z.string().min(1),startSec: secondsSchema.optional(),@@ -1062,6 +1122,10 @@ export const OPENSCREEN_TOOL_NAMES = ["setAnnotation","addCameraFullscreen","setCameraFullscreen",+ "addEffect",+ "setEffect",+ "setEffectDefinition",+ "getEffect","addAudio","setAudio","removeTrim",@@ -1140,6 +1204,9 @@ export const MUTATING_TOOL_NAMES: ReadonlySet<string> = new Set(["setAnnotation","addCameraFullscreen","setCameraFullscreen",+ "addEffect",+ "setEffect",+ "setEffectDefinition","addAudio","setAudio","removeTrim",@@ -1281,7 +1348,7 @@ export function documentSnapshotForModel(const autoFocusAll = legacy?.autoFocusAll === true;return {timeBaseNote:- "clips and trims are in source-time seconds; zooms, speedRegions, annotations, cameraFullscreenRegions and audioTracks are in virtual (edited-timeline) seconds: the editor's ruler, where a trim leaves a hole instead of shortening it and a speed region stretches nothing. The exported video's clock skips those holes and plays sped-up spans faster, so it differs from the ruler once there are any.",+ "clips and trims are in source-time seconds; zooms, speedRegions, annotations, cameraFullscreenRegions, effects and audioTracks are in virtual (edited-timeline) seconds: the editor's ruler, where a trim leaves a hole instead of shortening it and a speed region stretches nothing. The exported video's clock skips those holes and plays sped-up spans faster, so it differs from the ruler once there are any.",audioNote:"audioTracks are imported voiceover / music files laid over the recording. They are clip-anchored like every other region, so they travel with their clip through reorder and trim, and they play at 1x whatever a speed region does to the picture under them. addAudio places an EXISTING asset of kind 'audio': if the project has none, a file has to be imported first (importMedia, where it is offered), so never invent an id.",zoomNote:@@ -1408,6 +1475,29 @@ export function documentSnapshotForModel(startSec: roundSec(c.startMs),endSec: roundSec(c.endMs),})),+ // Where each effect is posed, and how. The code is not here: getEffect reads it.+ effects: coalesceForAgent(document.shaderEffectRanges ?? []).map((r) => ({+ id: r.id,+ effectId: r.effectId,+ name: findShaderEffect(document, r.effectId)?.name ?? null,+ startSec: roundSec(r.startMs),+ endSec: roundSec(r.endMs),+ target: r.target,+ lane: r.lane,+ values: r.values,+ })),+ effectDefinitions: (document.shaderEffects ?? []).map((e) => ({+ effectId: e.id,+ name: e.name,+ description: e.description,+ contract: e.contract,+ lines: e.code.split("\n").length,+ regions: coalesceForAgent(+ (document.shaderEffectRanges ?? []).filter((r) => r.effectId === e.id),+ ).length,+ cost: e.cost ?? null,+ params: e.params,+ })),// Imported audio, collapsed to the pills the ruler draws — a track ventilated// across a clip boundary is several fragments the user sees as one thing, and// the model has to name what the user sees.@@ -1432,6 +1522,71 @@ export function documentSnapshotForModel(};}+/** A definition as it will be stored: schema-checked, then compiled by the compositor when one is+ * wired, its cost cached. An error reads as the agent must act on it. */+function checkedEffectDefinition(+ candidate: unknown,+ checker: ShaderEffectChecker | undefined,+): { definition: AxcutShaderEffect; check: ShaderEffectCheck | null } | { error: string } {+ const parsed = shaderEffectSchema.safeParse(candidate);+ if (!parsed.success) {+ return {+ error: parsed.error.issues+ .map((i) => `${i.path.join(".") || "effect"}: ${i.message}`)+ .join("; "),+ };+ }+ const definition = parsed.data;+ const check = checker?.(definition.code, definition.params) ?? null;+ if (check?.error) return { error: `The effect does not compile:\n${check.error}` };+ const cost =+ check?.tier && typeof check.frameReads === "number"+ ? { tier: check.tier, frameReads: check.frameReads }+ : undefined;+ return { definition: { ...definition, cost }, check };+}++/** Two effects of one lane cannot overlap: the refusal names what is in the way, and the way out. */+function effectLaneBlocked(+ ranges: AxcutShaderEffectRange[],+ lane: number,+ startMs: number,+ endMs: number,+ except: ReadonlySet<string> = new Set(),+): string | null {+ const wall = coalesceRegionsForRuler(+ ranges.filter((r) => r.lane === lane && !except.has(r.id)),+ ).find((pill) => pill.start * 1000 < endMs && pill.end * 1000 > startMs);+ if (!wall) return null;+ return (+ `${formatSec(startMs / 1000)} – ${formatSec(endMs / 1000)} would run into effect ${wall.ids[0]} ` ++ `at ${formatSec(wall.start)} – ${formatSec(wall.end)} on lane ${lane}, and two effects of one lane ` ++ "cannot overlap. Put it on another lane (0–7, applied in order, lane 0 first) or pick a span clear of it."+ );+}++/** What an effect costs where it was placed: its tier, the busiest moment it shares with the+ * other effects, and a recommendation when its tier runs longer than it is meant to. */+function effectCostReport(+ document: AxcutDocument,+ definition: AxcutShaderEffect,+ startSec: number,+ endSec: number,+) {+ if (!definition.cost) return { cost: null };+ const busiest = busiestShaderEffectReads(document, startSec * 1000, endSec * 1000);+ const advice = shaderEffectCostAdvice(definition.cost.tier, endSec - startSec);+ return {+ cost: {+ tier: definition.cost.tier,+ frameReads: definition.cost.frameReads,+ busiestTier: shaderEffectTierOf(busiest),+ busiestFrameReads: busiest,+ ...(advice ? { advice } : {}),+ },+ };+}+function failure(message: string): AgentToolExecution {return { ok: false, resultJson: JSON.stringify({ error: message }) };}@@ -1578,6 +1733,9 @@ export interface AgentToolOptions {/** Whether this machine can cut out the camera background, asked only by the two tools that* report it. Absent where nothing can ask the compositor: reported as not checked. */cameraSegmentation?: () => SegmentationSupport;+ /** Asks the compositor to check an effect's WGSL (`checkShaderEffect`). Absent where nothing+ * can: the effect is stored unchecked, and the result says `validated: false`. */+ checkShaderEffect?: ShaderEffectChecker;/** `addAutoZooms`' plan, made by the async wrapper from the recorded clicks. Absent means* no cursor reader is wired here, which the tool reports as "could not look". */autoZooms?: AutoZoomPlan;@@ -3169,6 +3327,206 @@ export function executeAgentTool(};}+ case "addEffect": {+ const parsed = addEffectArgs.safeParse(args);+ if (!parsed.success) return failure(parsed.error.message);+ const { effectId, effect, values = {}, target = "image", lane = 0 } = parsed.data;+ if ((effectId === undefined) === (effect === undefined)) {+ return failure(+ "Pass either `effect` to write a new effect, or `effectId` to reuse one of this project's (getCurrentDocument lists them under effectDefinitions), not both.",+ );+ }+ let definition: AxcutShaderEffect;+ let check: ShaderEffectCheck | null = null;+ const created = effect !== undefined;+ if (effect) {+ const checked = checkedEffectDefinition(+ {+ id: createId("fx"),+ ...effect,+ contract: SHADER_EFFECT_CONTRACT,+ params: effect.params ?? [],+ origin: "agent",+ },+ options?.checkShaderEffect,+ );+ if ("error" in checked) return failure(checked.error);+ ({ definition, check } = checked);+ } else {+ const existing = findShaderEffect(document, effectId as string);+ if (!existing) return failure(`Unknown effect: ${effectId}.`);+ definition = existing;+ }+ const startMs = toMs(Math.min(parsed.data.startSec, parsed.data.endSec));+ const endMs = toMs(Math.max(parsed.data.startSec, parsed.data.endSec));+ const blocked = effectLaneBlocked(document.shaderEffectRanges ?? [], lane, startMs, endMs);+ if (blocked) return failure(`The effect was NOT placed: ${blocked}`);+ const region = {+ id: createId("fxr"),+ startMs,+ endMs,+ effectId: definition.id,+ values: storedShaderEffectValues(definition, values),+ target,+ lane,+ };+ const placed = anchorForAgent(region, document, "fxr");+ const landing = landingOf(placed, document);+ if (!landing.anchored) return coversNoClip("effect", startMs / 1000, endMs / 1000, document);+ const next: AxcutDocument = {+ ...document,+ shaderEffects: created+ ? [...(document.shaderEffects ?? []), definition]+ : (document.shaderEffects ?? []),+ shaderEffectRanges: [...(document.shaderEffectRanges ?? []), ...placed],+ };+ return {+ ok: true,+ document: next,+ resultJson: JSON.stringify({+ effectRegionId: landing.ids[0],+ effectId: definition.id,+ ...landingReport(landing, startMs / 1000, endMs / 1000),+ ...(created ? { validated: check !== null } : {}),+ ...effectCostReport(next, definition, landing.startSec, landing.endSec),+ }),+ summary:+ `effect "${definition.name}" ${formatSec(landing.startSec)} – ${formatSec(landing.endSec)}` ++ landingSuffix(landing, startMs / 1000, endMs / 1000),+ };+ }++ case "setEffect": {+ const parsed = setEffectArgs.safeParse(args);+ if (!parsed.success) return failure(parsed.error.message);+ const prev = document.shaderEffectRanges ?? [];+ const existing = prev.find((r) => r.id === parsed.data.effectRegionId);+ if (!existing) return failure(`Unknown effect region: ${parsed.data.effectRegionId}`);+ const definition = findShaderEffect(document, existing.effectId);+ if (!definition) return failure(`Effect ${existing.effectId} is missing from the project.`);+ const pillIds = new Set(resolvePillIds(prev, existing.id));+ const pill = coalesceRegionsForRuler(prev).find((p) => p.ids.includes(existing.id));+ const { values, target } = parsed.data;+ const lane = parsed.data.lane ?? existing.lane;+ const patched = prev.map((r) =>+ pillIds.has(r.id)+ ? {+ ...r,+ values:+ values === undefined+ ? r.values+ : storedShaderEffectValues(definition, { ...r.values, ...values }),+ target: target ?? r.target,+ lane,+ }+ : r,+ );+ const { startMs, endMs } = resolveSpanMs(+ pill+ ? { startMs: Math.round(pill.start * 1000), endMs: Math.round(pill.end * 1000) }+ : existing,+ parsed.data.startSec,+ parsed.data.endSec,+ );+ const blocked = effectLaneBlocked(patched, lane, startMs, endMs, pillIds);+ if (blocked) return failure(`The effect was NOT changed: ${blocked}`);+ const moved = parsed.data.startSec !== undefined || parsed.data.endSec !== undefined;+ const rebuilt = moved+ ? replacePillSpan(patched, existing.id, startMs, endMs, document.timeline.clips, () =>+ createId("fxr"),+ )+ : patched;+ const landing = landingAfterPillEdit(prev, rebuilt, pillIds, document);+ if (!landing.anchored) return coversNoClip("effect", startMs / 1000, endMs / 1000, document);+ const next: AxcutDocument = { ...document, shaderEffectRanges: rebuilt };+ return {+ ok: true,+ document: next,+ resultJson: JSON.stringify({+ effectRegionId: landing.ids[0] ?? existing.id,+ effectId: definition.id,+ ...landingReport(landing, startMs / 1000, endMs / 1000),+ ...effectCostReport(next, definition, landing.startSec, landing.endSec),+ }),+ summary:+ `updated effect "${definition.name}" ${formatSec(landing.startSec)} – ${formatSec(landing.endSec)}` ++ landingSuffix(landing, startMs / 1000, endMs / 1000),+ };+ }++ case "setEffectDefinition": {+ const parsed = setEffectDefinitionArgs.safeParse(args);+ if (!parsed.success) return failure(parsed.error.message);+ const { effectId, ...patch } = parsed.data;+ const existing = findShaderEffect(document, effectId);+ if (!existing) return failure(`Unknown effect: ${effectId}.`);+ const recompile = patch.code !== undefined || patch.params !== undefined;+ const candidate = {+ ...existing,+ ...Object.fromEntries(Object.entries(patch).filter(([, v]) => v !== undefined)),+ cost: recompile ? undefined : existing.cost,+ };+ let definition: AxcutShaderEffect;+ let check: ShaderEffectCheck | null = null;+ if (recompile) {+ const checked = checkedEffectDefinition(candidate, options?.checkShaderEffect);+ if ("error" in checked) return failure(checked.error);+ ({ definition, check } = checked);+ } else {+ const checked = shaderEffectSchema.safeParse(candidate);+ if (!checked.success) return failure(checked.error.message);+ definition = checked.data;+ }+ // The regions follow the new parameters: a removed one's value drops, bounds re-clamp.+ const next: AxcutDocument = {+ ...document,+ shaderEffects: (document.shaderEffects ?? []).map((e) =>+ e.id === effectId ? definition : e,+ ),+ shaderEffectRanges: (document.shaderEffectRanges ?? []).map((r) =>+ r.effectId === effectId+ ? { ...r, values: storedShaderEffectValues(definition, r.values) }+ : r,+ ),+ };+ const regions = coalesceForAgent(+ (next.shaderEffectRanges ?? []).filter((r) => r.effectId === effectId),+ );+ return {+ ok: true,+ document: next,+ resultJson: JSON.stringify({+ effectId,+ regions: regions.length,+ ...(recompile ? { validated: check !== null } : {}),+ ...(definition.cost ? { cost: definition.cost } : {}),+ }),+ summary: `updated effect "${definition.name}" (${regions.length} region${regions.length === 1 ? "" : "s"})`,+ };+ }++ case "getEffect": {+ const parsed = getEffectArgs.safeParse(args);+ if (!parsed.success) return failure(parsed.error.message);+ const effect = findShaderEffect(document, parsed.data.effectId);+ if (!effect) return failure(`Unknown effect: ${parsed.data.effectId}.`);+ const regions = coalesceForAgent(+ (document.shaderEffectRanges ?? []).filter((r) => r.effectId === effect.id),+ ).map((r) => ({+ id: r.id,+ startSec: roundSec(r.startMs),+ endSec: roundSec(r.endMs),+ target: r.target,+ lane: r.lane,+ values: r.values,+ }));+ return {+ ok: true,+ resultJson: JSON.stringify({ ...effect, regions }),+ summary: `read effect "${effect.name}"`,+ };+ }+case "setCameraFullscreen": {const parsed = setCameraFullscreenArgs.safeParse(args);if (!parsed.success) return failure(parsed.error.message);@@ -3411,9 +3769,10 @@ export function executeAgentTool(else if (speedRegions.some((s) => s.id === id)) kind = "speed";else if (cameraFullscreenRegions.some((c) => c.id === id)) kind = "cameraFullscreen";else if (document.audioTracks.some((t) => trackGroupId(t) === id)) kind = "audio";+ else if ((document.shaderEffectRanges ?? []).some((r) => r.id === id)) kind = "shaderEffect";if (!kind) {return failure(- `No zoom / speed / annotation / full-camera / audio modifier with id ${id}. ` ++ `No zoom / speed / annotation / full-camera / audio / effect modifier with id ${id}. ` +`For a trim use removeTrim; for a clip use removeClip.`,);}diff --git a/electron/ai-edition/chat-service.ts b/electron/ai-edition/chat-service.tsindex e1e55c94c..c222a8de4 100644--- a/electron/ai-edition/chat-service.ts+++ b/electron/ai-edition/chat-service.ts@@ -16,6 +16,7 @@ import {createEmptyDocument,documentSchema,} from "../../src/lib/ai-edition/schema";+import type { ShaderEffectChecker } from "../../src/lib/ai-edition/shaderEffects";import type {AiEditionChatMessage,AiEditionChatResult,@@ -322,6 +323,8 @@ export interface ChatRunEnv {cursor?: CursorTelemetryReader;/** Whether this machine can cut out the camera background, asked of the compositor. */cameraSegmentation?: () => SegmentationSupport;+ /** The compositor's check of an agent-written effect's WGSL. */+ checkShaderEffect?: ShaderEffectChecker;/** Runtime-only model retry override used by bounded measurement harnesses. */maxRetries?: number;}@@ -468,6 +471,7 @@ export async function runChat(editsAllowed,cursor: env.cursor,cameraSegmentation: env.cameraSegmentation,+ checkShaderEffect: env.checkShaderEffect,});// The project was deleted while the agent ran: its sessions are gone, so adiff --git a/electron/ai-edition/deep-agent/service.test.ts b/electron/ai-edition/deep-agent/service.test.tsindex 8fa62ea51..22d508fb7 100644--- a/electron/ai-edition/deep-agent/service.test.ts+++ b/electron/ai-edition/deep-agent/service.test.ts@@ -83,6 +83,19 @@ const ARGS: Record<string, unknown> = {setAnnotation: { annotationId: "ann_nope" },addCameraFullscreen: { startSec: 1, endSec: 2 },setCameraFullscreen: { cameraFullscreenId: "cam_nope" },+ // No checker in these tests: the effect is stored unchecked, and the result says so.+ addEffect: {+ startSec: 1,+ endSec: 2,+ effect: {+ name: "Pass-through",+ description: "Shows the frame unchanged.",+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }",+ },+ },+ setEffect: { effectRegionId: "fxr_nope" },+ setEffectDefinition: { effectId: "fx_nope" },+ getEffect: { effectId: "fx_nope" },// The fixture has no `kind: "audio"` asset, so these exercise the refusal branch —// the honest one to pin: the agent can place imported audio, never import it.addAudio: { assetId: "audio_nope", startSec: 1, endSec: 2 },@@ -389,6 +402,7 @@ describe("one description of the tools, not two", () => {"getTranscript","getTranscriptWords","getCursorTrack",+ "getEffect","getCaptions","getCaptionTranslationUnits",]);diff --git a/electron/ai-edition/deep-agent/service.ts b/electron/ai-edition/deep-agent/service.tsindex 44571c2dd..f97fa9cf6 100644--- a/electron/ai-edition/deep-agent/service.ts+++ b/electron/ai-edition/deep-agent/service.ts@@ -35,6 +35,7 @@ import {import { AUDIO_FADE_MAX_MS } from "../../../src/lib/ai-edition/document/audioTracks";import { VOCABULARY_MAX_CHARS } from "../../../src/lib/ai-edition/document/transcript";import type { AxcutDocument } from "../../../src/lib/ai-edition/schema";+import type { ShaderEffectChecker } from "../../../src/lib/ai-edition/shaderEffects";import { collectAutoZoomSuggestionsForDocument } from "../../../src/lib/ai-edition/timeline/apply-auto-zooms";// ponytail: the legend is DERIVED from the depth→scale table, not retyped. Both// tool descriptions used to assert "depth 1–6 maps to 1.0×–3.5×" — a formula@@ -55,6 +56,7 @@ import {addAudioArgs,addAutoZoomsArgs,addCameraFullscreenArgs,+ addEffectArgs,addSpeedArgs,addTrimArgs,addTrimsArgs,@@ -64,6 +66,7 @@ import {duplicateClipArgs,executeAgentTool,getCursorTrackArgs,+ getEffectArgs,getTranscriptArgs,getTranscriptWordsArgs,insertClipArgs,@@ -80,6 +83,8 @@ import {setAudioArgs,setCameraFullscreenArgs,setClipRangeArgs,+ setEffectArgs,+ setEffectDefinitionArgs,setSpeedArgs,setTrimArgs,setWordTextArgs,@@ -149,7 +154,7 @@ const BASE_SYSTEM_PROMPT = ["You can call the tools below against the live document snapshot; the runtime executes each edit and feeds the result back into the loop.","The AxcutDocument is the single source of truth. The timeline, the transcript editor, and the chat panel are all direct editors of the same document — when the user places a clip on the timeline, the document updates immediately, and when the timeline is empty, the document has no clips. Your edits operate on the live document, so preserve the user's placed clips.","",- "Time-bases (do not mix them up): clips and trims are in SOURCE-time seconds of an asset; zooms, speed regions, annotations and camera-fullscreen regions are in VIRTUAL (edited-timeline) seconds — the position on the editor's ruler, where the clips sit end to end. A trim leaves a hole in that ruler rather than shortening it, and a speed region does not stretch it, so the exported video's own clock differs from it as soon as there is a trim or a speed region: always place effects in ruler seconds. getCurrentDocument returns all of them, clearly labelled.",+ "Time-bases (do not mix them up): clips and trims are in SOURCE-time seconds of an asset; zooms, speed regions, annotations, camera-fullscreen regions and effects are in VIRTUAL (edited-timeline) seconds — the position on the editor's ruler, where the clips sit end to end. A trim leaves a hole in that ruler rather than shortening it, and a speed region does not stretch it, so the exported video's own clock differs from it as soon as there is a trim or a speed region: always place effects in ruler seconds. getCurrentDocument returns all of them, clearly labelled.","",// ponytail: these describe what each tool is FOR, deliberately without quoting// user phrasings. A phrase→tool table reads as helpful and is not: it swaps the@@ -167,6 +172,7 @@ const BASE_SYSTEM_PROMPT = ["- addAudio lays an imported voiceover or music file over a span. It plays an asset the project already has (kind 'audio'); importing or recording one is the editor's job, not a tool you have — so when the project has none, say so rather than naming an id that does not exist.","- moveClip changes the order of placed clips, one call per clip that moves, preserving ids, source ranges, trims and anchored effects. replaceTimeline rebuilds the timeline from kept intervals and sorts them, so it cannot reorder anything.","- Captions are the transcript drawn on the video, never typed in: setCaptionSettings turns them on, styles and places them, getCaptions reads them back. To caption in another language, read the phrases with getCaptionTranslationUnits, translate them yourself, store them with putCaptionTranslation, then show that language with setCaptionSettings. A translation never rewrites the transcript. For text the speaker did not say, use addAnnotation.",+ "- A visual effect the settings do not offer (a ripple on clicks, a glitch on a cut, a color grade) is written as a shader with addEffect; setEffect re-tunes or moves it, setEffectDefinition changes its code, getEffect reads its code.","- Deleting is a first-class action, not a workaround: removeTrim, removeModifier, removeClip. Never fake a deletion by re-adding an element or zeroing it out (span 0, speed 1×) — that leaves it in the document and misreports what you did.","If nothing in the list does what was asked, say so; do not approximate it with a bigger tool.","",@@ -212,7 +218,7 @@ const CUT_TRANSITIONS_RESULT = `When the result cuts a zoom's move in or out, itexport const TOOL_DESCRIPTIONS: Record<string, string> = {getCurrentDocument:- "Read a compact snapshot of the current project: assets (with durations), timeline clips and trim ranges (source-time), and the zoom / speed / annotation / camera-fullscreen / audio regions (virtual, edited-timeline time), each with the settings its writes take: a zoom's level, focus mode and hidden cursor, an annotation's type, box, look and arrow or blur data, an audio track's level and fades. It also gives the project's look under `settings`, in the shape setEditorSettings takes, with `settingsContext`: the formats, wallpapers and cursor themes on offer, and under `inactive` each setting the editor hides in the current state and why. Call this before editing if the snapshot in the system prompt may be stale. The AxcutDocument is the single source of truth — your edits should preserve the user's placed clips and any timeline state they have already set up.",+ "Read a compact snapshot of the current project: assets (with durations), timeline clips and trim ranges (source-time), and the zoom / speed / annotation / camera-fullscreen / audio / effect regions (virtual, edited-timeline time), each with the settings its writes take: a zoom's level, focus mode and hidden cursor, an annotation's type, box, look and arrow or blur data, an audio track's level and fades. It also gives the project's look under `settings`, in the shape setEditorSettings takes, with `settingsContext`: the formats, wallpapers and cursor themes on offer, and under `inactive` each setting the editor hides in the current state and why. Call this before editing if the snapshot in the system prompt may be stale. The AxcutDocument is the single source of truth — your edits should preserve the user's placed clips and any timeline state they have already set up.",getTranscript:"Read the transcript segments (speech and silence, with start/end seconds and text) for an asset. Omit assetId to read the primary asset's transcript.",getCursorTrack:@@ -256,6 +262,14 @@ export const TOOL_DESCRIPTIONS: Record<string, string> = {'Change the project\'s look: pass only the settings to change, named as getCurrentDocument returns them under `settings`. Background: wallpaper (a bundled path from settingsContext.bundledWallpapers, a CSS colour or gradient, or a base64 PNG/JPEG data URL; never a file path or web URL; an embedded image reads back as a marker, leave the key out to keep it), wallpaperMotion (moves a gradient or image only), backgroundBlur 0–1. Frame: aspectRatio ("auto", a preset, or an originalAspectRatios token; a fixed format picked for the first time fills the frame with a window that follows the cursor, which formatFollowCursor turns off when settingsContext.formatFill is "available"), padding 0–100 %, borderRadius 0–64 px, shadowIntensity 0–1, frame (none, window, laptop, phone, monitor) and frameTheme (light, dark), motionBlurAmount 0–1, depthOfField (blurs 3D-tilted zooms only). Camera: webcamLayoutPreset, webcamMirrored, webcamReactiveZoom (shrinks on zoom), webcamMaskShape (square or rectangle, the camera\'s own proportions; an untouched roundness follows it), webcamRoundness 0–1, webcamSizePreset 15–60, webcamAnchor (a corner or an edge middle), webcamCropRegion (the framing: a square share of the picture, width = height ≥ 1/3) and webcamCropPan (where that window sits, 0–1 per axis, 0.5 centred), webcamBackgroundMode (none, transparent, blur, custom: needs settingsContext.cameraSegmentation ready), webcamBlurIntensity 0–1, webcamWallpaper (as wallpaper). Cursor, under `cursor`: show, autoHide, theme (settingsContext.cursorThemes), asArrow (the recorded cursor kinds drawn as the arrow instead), size 1.5–6, smoothing 0–1, motionBlur 0–1, clickBounce 0–2, clickImpact, clickSound (only heard when the take recorded clicks) and clickSoundGainDb −24–0, model3d. Also autoFocusAll (every zoom follows the cursor) and audioGainDb −12–12 dB, the master volume. A value out of range or a key not listed is refused with the reason, and nothing changes. The result lists what was applied, including what the editor changes with it, and `inactive` for any of those the current state leaves without effect.',addCameraFullscreen:"Add a camera-fullscreen region over a span of the edited timeline (virtual seconds): the webcam fills the frame for that span. This only does something when the footage under that span comes from an asset with a linked webcam — check assets[].hasCameraTrack (or hasAnyCamera) in getCurrentDocument first. On footage with no camera the call is refused rather than storing a region that would render nothing; say so instead of retrying.",+ addEffect:+ "Write a visual effect as a shader and pose it over a span of the edited timeline (virtual seconds), or pose one of this project's effects again (effectId). Use it for a look the settings do not offer. `effect.code` is WGSL that defines `fn effect(uv: vec2<f32>) -> vec4<f32>`, called for every pixel; uv is 0..1 with (0,0) top-left. Available inside it: `frame(uv)` samples the composed picture; `params.<name>` reads each declared parameter; `ctx.time` is programme seconds; `ctx.region.x` goes 0 to 1 across the region (y its length in s, z seconds since it began); `ctx.resolution` and `ctx.aspect`; `ctx.cursor` (xy in uv, z = 1 when on screen); `ctx.clicks[0..3]`, the last four clicks, newest first (xy in uv where it happened, z seconds since, w = 1 when the slot holds one); `ctx.footage` and `ctx.camera`, the recording's and the webcam's rects (x, y, w, h in uv; camera zero-sized when absent) — all already placed through zoom and layout; `fx_px(n)`, n pixels of a 1080p frame as a fraction of the height; `fx_hash(vec2<u32>)`, a random number in [0,1) identical on every GPU. Rules: the effect is a pure function of those inputs (no state between frames), so the preview and the export show the same pixels; reason in uv or fx_px, never raw pixels; every loop is a `for` with literal bounds (`for (var i = 0; i < 16; i++)`), at most 4096 iterations per pixel — the one hard rule. `params` declares up to 8 (recommended) parameters, each `{type: number|toggle|color|point, name, label, default}` (number also `min`, `max`); declare one only when moving it visibly changes the effect over its whole range. `target`: `image` (default: the picture under the annotations and captions), `text` (the annotations and captions alone, premultiplied alpha), `both` (everything); use text or both only when asked. Effects of one `lane` (0–7) cannot overlap; lanes apply in order, lane 0 first. The result gives the cost tier (frame() reads per pixel: light ≤ 8, about 0.06 ms per 1080p frame on a fast GPU; medium ≤ 32, 0.14 ms; heavy ≤ 128, 0.54 ms; very heavy beyond) and a recommendation when the tier runs long: light anywhere, medium up to a minute, heavy up to 10 s, very heavy a few seconds. It is a recommendation, never a refusal. Code that does not compile is refused with the compiler's error at the line of your code: fix it and call again. Check the result with getFrames.",+ setEffect:+ "Move, resize or re-tune an effect region by id (virtual seconds): its span, its parameter `values`, its `target` or its `lane`. Only the fields you pass change; a value equal to the default is dropped. Refused if the new span runs into another effect of the same lane.",+ setEffectDefinition:+ "Change an effect itself by effectId — its name, description, WGSL `code` or `params` — and with it every region that uses it. New code is compiled first and refused with the compiler's error if it does not compile. A removed parameter's values drop; the bounds re-clamp the others. Read the current code with getEffect before rewriting it.",+ getEffect:+ "Read one effect of the project in full: its WGSL code, parameters, description, cost and the regions that use it. getCurrentDocument lists effects without their code.",setCameraFullscreen:"Move or resize an existing camera-fullscreen region by id (virtual-timeline seconds). Only the fields you pass are changed. Refused if the new span lands on footage with no linked webcam.",addAudio: `Lay an ALREADY-IMPORTED audio file over the recording across a span of the edited timeline (virtual seconds): a voiceover, or a music bed. assetId must name an asset whose kind is 'audio' — getCurrentDocument lists them. If there is none, a file has to be imported first (importMedia, where it is offered): never guess an id. Omit endSec to play the whole file from offsetSec. kind picks the lane ('voiceover' or 'music'). offsetSec is where in the FILE playback starts, gainDb its level (0 unchanged, negative ducks it), fadeInMs / fadeOutMs its fades (0 to ${AUDIO_FADE_MAX_MS} ms, default none; a track too short for both shares its span between them). A voiceover-lane track is also what gets transcribed, so the lane is not only cosmetic.`,@@ -263,7 +277,7 @@ export const TOOL_DESCRIPTIONS: Record<string, string> = {removeTrim:"Delete a trim range by id — the cut is undone and that span plays/exports again. This is how you 'remove a trim'; never re-add a trim to undo one.",removeModifier:- "Delete a modifier (zoom / speed / annotation / camera-fullscreen / audio) by id; the kind is resolved from the id. Touching rows with the same styling render as one pill and are deleted together, so read removedIds in the result for the complete set that disappeared. This is how you 'remove'/'delete' one — never neutralise it (span 0, speed 1×), which leaves it in the document. For a trim use removeTrim; for a clip use removeClip.",+ "Delete a modifier (zoom / speed / annotation / camera-fullscreen / audio / effect) by id; the kind is resolved from the id. Touching rows with the same styling render as one pill and are deleted together, so read removedIds in the result for the complete set that disappeared. This is how you 'remove'/'delete' one — never neutralise it (span 0, speed 1×), which leaves it in the document. For a trim use removeTrim; for a clip use removeClip.",removeClip:"Delete a placed clip by id; remaining clips close the gap and effects anchored to it are dropped. Use only when the user asks to remove a clip — to shorten one, use setClipRange.",getCaptions:@@ -310,6 +324,8 @@ export interface ToolRuntime {availableByAssetId?: Record<string, boolean>;/** Asks the compositor whether this machine can cut out the camera background. */cameraSegmentation?: () => SegmentationSupport;+ /** Asks the compositor to check an effect's WGSL. */+ checkShaderEffect?: ShaderEffectChecker;}/**@@ -393,6 +409,7 @@ export async function runDocumentTool(editsAllowed,cursorTelemetry: { availableByAssetId: runtime.availableByAssetId, load },cameraSegmentation: runtime.cameraSegmentation,+ checkShaderEffect: runtime.checkShaderEffect,autoZooms,});}@@ -504,6 +521,10 @@ export const TOOL_ARG_SCHEMAS: ReadonlyArray<readonly [string, z.ZodObject]> = [["setAnnotation", setAnnotationArgs],["addCameraFullscreen", addCameraFullscreenArgs],["setCameraFullscreen", setCameraFullscreenArgs],+ ["addEffect", addEffectArgs],+ ["setEffect", setEffectArgs],+ ["setEffectDefinition", setEffectDefinitionArgs],+ ["getEffect", getEffectArgs],["addAudio", addAudioArgs],["setAudio", setAudioArgs],["removeTrim", removeTrimArgs],@@ -556,6 +577,8 @@ export interface InvokeArgs {cursor?: CursorTelemetryReader;/** Injected the same way: whether this machine can cut out the camera background. */cameraSegmentation?: () => SegmentationSupport;+ /** Injected the same way: the compositor's check of an effect's WGSL. */+ checkShaderEffect?: ShaderEffectChecker;}/** One cheap probe per asset, run before the tools are built so the very first@@ -631,6 +654,7 @@ export async function invokeOpenScreenAgent(args: InvokeArgs): Promise<InvokeRescursor: args.cursor,availableByAssetId,cameraSegmentation: args.cameraSegmentation,+ checkShaderEffect: args.checkShaderEffect,});const agent = createAgent({model: chatModel,diff --git a/electron/cli/projectCommands.test.ts b/electron/cli/projectCommands.test.tsindex ee09a9261..e0d6ef213 100644--- a/electron/cli/projectCommands.test.ts+++ b/electron/cli/projectCommands.test.ts@@ -25,8 +25,8 @@ async function writeProject(relative: string, data: PackedProjectData): Promise</*** A project file as the editor saves it: the AxcutDocument itself, one zoom on the timeline, an- * export format a migrated v2 project left in `legacyEditor`, and a speed region where the editor- * keeps them (`legacyEditor.speedRegions`).+ * export format a migrated v2 project left in `legacyEditor`, a speed region where the editor+ * keeps them (`legacyEditor.speedRegions`), and one shader effect.*/async function writeDocument(relative: string, screenVideoPath: string): Promise<string> {const doc = migrateProjectDataToAxcutDocument({@@ -40,7 +40,19 @@ async function writeDocument(relative: string, screenVideoPath: string): Promiseconst speedRegions = [{ id: "s1", startMs: 0, endMs: 500, speed: 2 }];return make(relative,- JSON.stringify({ ...doc, legacyEditor: { ...doc.legacyEditor, speedRegions } }),+ JSON.stringify({+ ...doc,+ legacyEditor: { ...doc.legacyEditor, speedRegions },+ shaderEffects: [+ {+ id: "fx1",+ name: "Pass-through",+ contract: 1,+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }",+ },+ ],+ shaderEffectRanges: [{ id: "fxr1", startMs: 0, endMs: 500, effectId: "fx1" }],+ }),);}@@ -158,6 +170,7 @@ describe("runInfoCommand", () => {trimRegions: 1,speedRegions: 0,annotationRegions: 0,+ shaderEffectRegions: 0,exportFormat: "mp4",screenVideoExists: true,});@@ -179,6 +192,7 @@ describe("a project the editor saved", () => {zoomRegions: 1,trimRegions: 0,speedRegions: 1,+ shaderEffectRegions: 1,});});diff --git a/electron/cli/projectCommands.ts b/electron/cli/projectCommands.tsindex 6866eb92a..394c33f53 100644--- a/electron/cli/projectCommands.ts+++ b/electron/cli/projectCommands.ts@@ -164,6 +164,7 @@ export async function runInfoCommand(trimRegions: fields.trimRegions,speedRegions: fields.speedRegions,annotationRegions: fields.annotationRegions,+ shaderEffectRegions: fields.shaderEffectRegions,};if (json) {@@ -176,7 +177,7 @@ export async function runInfoCommand(`Webcam: ${summary.webcamVideoPath ?? "(none)"}`,`Cursor: ${summary.cursorCaptureMode ?? "(unknown)"}`,`Export: ${summary.exportFormat ?? "?"} / ${summary.exportQuality ?? "?"} / ${summary.aspectRatio ?? "?"}`,- `Timeline: ${summary.zoomRegions} zooms, ${summary.trimRegions} trims, ${summary.speedRegions} speed regions, ${summary.annotationRegions} annotations`,+ `Timeline: ${summary.zoomRegions} zooms, ${summary.trimRegions} trims, ${summary.speedRegions} speed regions, ${summary.annotationRegions} annotations, ${summary.shaderEffectRegions} effects`,].join("\n") + "\n",);}@@ -197,6 +198,8 @@ interface ProjectInfoFields {trimRegions: number;speedRegions: number;annotationRegions: number;+ /** Shader effect regions (stored fragments, like the other counts). A v2 project has none. */+ shaderEffectRegions: number;}const stringOrNull = (value: unknown): string | null => (typeof value === "string" ? value : null);@@ -217,6 +220,7 @@ function legacyInfoFields(data: PackedProjectData): ProjectInfoFields {trimRegions: count("trimRegions"),speedRegions: count("speedRegions"),annotationRegions: count("annotationRegions"),+ shaderEffectRegions: 0,};}@@ -242,5 +246,6 @@ function documentInfoFields(doc: ReturnType<typeof parseDocumentFile>): ProjectItrimRegions: doc.timeline.trimRanges.length,speedRegions: Array.isArray(editor.speedRegions) ? editor.speedRegions.length : 0,annotationRegions: doc.annotations.length,+ shaderEffectRegions: doc.shaderEffectRanges?.length ?? 0,};}diff --git a/electron/ipc/handlers.ts b/electron/ipc/handlers.tsindex 15f24992d..bb2a68a7b 100644--- a/electron/ipc/handlers.ts+++ b/electron/ipc/handlers.ts@@ -5062,6 +5062,7 @@ export function registerIpcHandlers(editsAllowed: () => mcpSettings.getSettings().allowEdits,cursor: agentCursorTelemetryReader,cameraSegmentation: () => loudnessService.probeSegmentation(),+ checkShaderEffect: (code, params) => loudnessService.checkShaderEffect(code, params),// The editor's own engine, in this process: `openscreen mcp` transcribes without the app.transcribe: (request) => getSttManager().transcribe(request),version: app.getVersion(),@@ -5237,6 +5238,7 @@ export function registerIpcHandlers(runChat(projectId, sessionId, message, getAiEditionLlmConfig(), document, sink, {cursor: agentCursorTelemetryReader,cameraSegmentation: () => loudnessService.probeSegmentation(),+ checkShaderEffect: (code, params) => loudnessService.checkShaderEffect(code, params),}),undoAiEditionToolBatch: (_projectId, _sessionId) => ({success: false,diff --git a/electron/mcp/effects-contract.test.ts b/electron/mcp/effects-contract.test.tsnew file mode 100644index 000000000..fce3e8b65--- /dev/null+++ b/electron/mcp/effects-contract.test.ts@@ -0,0 +1,39 @@+import { readFileSync } from "node:fs";+import path from "node:path";+import { describe, expect, it } from "vitest";+import { EFFECTS_CONTRACT } from "./effects-contract";++// The compositor's prelude is the authority on what an effect can read; the resource agents read+// must name every field of it, or an agent writes against a contract that is not the real one.+describe("the effect contract resource", () => {+ const shaderFx = readFileSync(+ path.join(__dirname, "../../crates/compositor/src/shader_fx.rs"),+ "utf8",+ );+ const prelude = shaderFx.slice(+ shaderFx.indexOf("pub const PRELUDE_V1"),+ shaderFx.indexOf("@group(0) @binding(0)"),+ );+ const fields = [...prelude.matchAll(/^\s+([a-z_]+): /gm)].map((m) => m[1]);++ it("names every field of the compositor's FxContext", () => {+ expect(fields).toEqual(+ expect.arrayContaining([+ "resolution",+ "time",+ "region",+ "cursor",+ "clicks",+ "footage",+ "camera",+ ]),+ );+ for (const field of fields) expect(EFFECTS_CONTRACT).toContain(`ctx.${field}`);+ });++ it("names every helper the prelude gives", () => {+ for (const helper of ["frame(uv)", "fx_px(n)", "fx_hash(p)"]) {+ expect(EFFECTS_CONTRACT).toContain(helper);+ }+ });+});diff --git a/electron/mcp/effects-contract.ts b/electron/mcp/effects-contract.tsnew file mode 100644index 000000000..929de027e--- /dev/null+++ b/electron/mcp/effects-contract.ts@@ -0,0 +1,131 @@+/**+ * The WGSL contract of agent-written shader effects, served as an MCP resource+ * (`openscreen://effects/contract`): too long for a tool description, and versioned with the+ * contract. The compositor's prelude (`crates/compositor/src/shader_fx.rs`) is the authority;+ * `effects-contract.test.ts` checks that every `ctx` field it declares is described here.+ * The design is in docs/shader-effects.md.+ */++export const EFFECTS_CONTRACT_URI = "openscreen://effects/contract";++export const EFFECTS_NOTE = `Before writing a shader effect with addEffect, read the resource ${EFFECTS_CONTRACT_URI}: the full WGSL contract and two examples.`;++export const EFFECTS_CONTRACT = `# OpenScreen shader effects — WGSL contract 1++An effect is WGSL that defines one function, called for every pixel of the output:++ fn effect(uv: vec2<f32>) -> vec4<f32>++\`uv\` runs from (0, 0) top-left to (1, 1) bottom-right of the output. Return the pixel's color.+Declare nothing else the compositor provides: bindings, entry points and the inputs below are+added after your code, so compiler errors point at your lines.++## Inputs++| Name | Type | Meaning |+|---|---|---|+| frame(uv) | vec4<f32> | The composed picture at uv (bilinear, edges clamped). |+| ctx.resolution | vec2<f32> | Output size in pixels. |+| ctx.aspect | f32 | Width over height, for round distances in uv. |+| ctx.time | f32 | Programme seconds, continuous through cuts, clips and speed changes. |+| ctx.region | vec4<f32> | x: progress 0 → 1 across the whole region; y: its length in s; z: seconds since it began. |+| ctx.cursor | vec4<f32> | xy: the pointer tip in uv; z: 1 when it is on screen. |+| ctx.clicks[i] | vec4<f32> | The last 4 clicks of the last 2 s, newest first. xy: where it happened, in uv; z: seconds since; w: 1 when the slot holds one. |+| ctx.footage | vec4<f32> | The recording's rect in uv (x, y, w, h), after zoom and layout; under a 3D tilt, its bounding box. |+| ctx.camera | vec4<f32> | The webcam's rect in uv; zero-sized without a camera. |+| params.<name> | per type | Your declared parameters. |+| fx_px(n) | f32 | n pixels of a 1080p output, as a fraction of the height. |+| fx_hash(p) | f32 | A random number in [0, 1) for a vec2<u32>, identical on every GPU. |++Everything in uv is already placed through zoom, layout, frame and tilt: never recompute where+the screen, the camera or the pointer sit.++## Parameters++Declare each one as \`{ type, name, label, default }\`; a number also takes \`min\` and \`max\`.++| type | WGSL | value in the document |+|---|---|---|+| number | f32 | a number between min and max |+| toggle | f32 (0 or 1) | true or false |+| color | vec4<f32> (RGBA, 0..1) | "#rrggbb" or "#rrggbbaa" |+| point | vec2<f32> (uv) | { "x": 0.5, "y": 0.5 } |++Names are snake_case. Up to 8 is recommended. Declare a parameter only when moving it visibly+changes the effect over its whole range: each one becomes a control in the editor.++## Rules++- **Pure.** The result depends only on uv, frame(), ctx and params. No state between frames, no+ clock, no frame counter: the preview jumps between frames when seeking, the export renders them+ in order, and both must show the same picture.+- **Resolution-free.** The preview renders smaller than the export: measure in uv or fx_px.+- **Random through fx_hash.** \`fract(sin(x) * 43758.5453)\` differs from one GPU to another.+- **Loops are \`for\` with literal bounds**: \`for (var i = 0; i < 16; i++)\`. Any other loop is+ refused, and so is more than 4096 iterations per pixel: a loop that never ends stalls the GPU.+ This is the one hard rule.+- **Alpha.** For the targets image and both, the returned color replaces the pixel; return+ \`frame(uv).a\` as alpha. For the target text, frame() is the text layer in premultiplied alpha,+ transparent outside the text, and the effect must return premultiplied alpha.++## Targets and lanes++- **image** (default): the picture under the annotations and captions; the text stays readable.+- **text**: the annotations and captions alone, drawn over the untouched picture afterwards.+- **both**: everything, captions included.++Use text or both only when the request asks for them. Effects of one lane (0–7) cannot overlap;+lanes apply in order, lane 0 first, so a ripple across the whole video and a glitch on a cut go on+two lanes.++## Cost++The result of addEffect gives the cost in frame() reads per pixel, counted from the code (loops+multiplied by their bound), and its tier. Measured on a fast GPU (RTX 4070 Ti) at 1080p:++| Tier | Reads | Per frame | Recommended for |+|---|---|---|---|+| light | ≤ 8 | ≤ 0.06 ms | anywhere, the whole video included |+| medium | ≤ 32 | ≤ 0.14 ms | up to a minute |+| heavy | ≤ 128 | ≤ 0.54 ms | up to 10 s: a transition, a key moment |+| very heavy | more | 2 ms at 512 | a few seconds |++An integrated GPU pays several times more. The tiers are recommendations, never refusals.++## Example: a ripple on every click++Parameters: strength (number, 0–2, default 1), radius (number, 0.05–0.5, default 0.2),+duration (number, 0.3–2, default 0.9).++ fn ripple(uv: vec2<f32>, click: vec4<f32>) -> vec2<f32> {+ if (click.w < 0.5 || click.z > params.duration) { return vec2<f32>(0.0); }+ let t = click.z / params.duration;+ let d = (uv - click.xy) * vec2<f32>(ctx.aspect, 1.0);+ let r = length(d);+ let x = params.radius * (1.0 - pow(1.0 - t, 3.0)) - r;+ let env = smoothstep(-0.008, 0.0, x) * exp(-max(x, 0.0) / 0.07);+ let wave = sin(x / 0.03 * 6.2831853) * env * (1.0 - t) * (1.0 - t) * params.strength;+ let dir = select(vec2<f32>(0.0), d / r, r > 1e-5);+ return dir * wave * 0.01 / vec2<f32>(ctx.aspect, 1.0);+ }++ fn effect(uv: vec2<f32>) -> vec4<f32> {+ var offset = vec2<f32>(0.0);+ for (var i = 0; i < 4; i++) { offset += ripple(uv, ctx.clicks[i]); }+ return frame(uv - offset);+ }++## Example: a vignette that breathes across the region++Parameters: depth (number, 0–1, default 0.6), tint (color, default "#000000").++ fn effect(uv: vec2<f32>) -> vec4<f32> {+ let c = frame(uv);+ let v = (uv - 0.5) * vec2<f32>(ctx.aspect, 1.0);+ let pulse = sin(ctx.region.x * 3.14159265);+ let k = smoothstep(0.35, 0.9, length(v)) * params.depth * pulse;+ return vec4<f32>(mix(c.rgb, params.tint.rgb, k), c.a);+ }++Check the result with getFrames at the moments that matter.+`;diff --git a/electron/mcp/openscreen-mcp-server.ts b/electron/mcp/openscreen-mcp-server.tsindex 7b0f3ec38..257bd7089 100644--- a/electron/mcp/openscreen-mcp-server.ts+++ b/electron/mcp/openscreen-mcp-server.ts@@ -54,6 +54,7 @@ import {withTranscript,} from "../../src/lib/ai-edition/document/transcript";import { type AxcutDocument, documentSchema } from "../../src/lib/ai-edition/schema";+import type { ShaderEffectChecker } from "../../src/lib/ai-edition/shaderEffects";import {segmentsFromStt,transcriptFromSegments,@@ -83,6 +84,7 @@ import {type ProjectVersion,} from "../ai-edition/document-service";import type { SttTranscribeRequest, SttTranscribeResponse } from "../stt/transcriptionContract";+import { EFFECTS_CONTRACT, EFFECTS_CONTRACT_URI, EFFECTS_NOTE } from "./effects-contract";import type { MediaTools } from "./media-tools";import {IMPORT_MEDIA_TOOL,@@ -170,6 +172,8 @@ export interface McpToolDeps extends ImportMediaDeps {cursor?: CursorTelemetryReader;/** Whether this machine can cut out the camera background; absent where nothing can ask. */cameraSegmentation?: () => SegmentationSupport;+ /** The compositor's check of an agent-written effect's WGSL; absent where nothing can ask. */+ checkShaderEffect?: ShaderEffectChecker;/** The main process's on-device Whisper (`SttManager.transcribe`). Absent: no transcribe tool. */transcribe?(request: SttTranscribeRequest): Promise<SttTranscribeResponse>;version: string;@@ -295,6 +299,7 @@ export function mcpInstructions(options: {]: []),...(options.transcribe ? [TRANSCRIBE_NOTE] : []),+ EFFECTS_NOTE,"",].join("\n");return (@@ -574,6 +579,7 @@ export function createToolRunner(deps: McpToolDeps): McpToolRunner {cursor: deps.cursor,availableByAssetId,cameraSegmentation: deps.cameraSegmentation,+ checkShaderEffect: deps.checkShaderEffect,});}@@ -925,6 +931,20 @@ export function createOpenScreenMcpServer();}media?.register(server);+ // The effect contract: too long for addEffect's description, and versioned with the contract.+ server.registerResource(+ "effects-contract",+ EFFECTS_CONTRACT_URI,+ {+ title: "Shader effect contract",+ description:+ "The WGSL contract of addEffect: the inputs an effect reads, its parameters, the rules, the cost tiers and two examples.",+ mimeType: "text/markdown",+ },+ async (uri) => ({+ contents: [{ uri: uri.href, mimeType: "text/markdown", text: EFFECTS_CONTRACT }],+ }),+ );return server;}diff --git a/electron/native-bridge/services/compositorViewService.ts b/electron/native-bridge/services/compositorViewService.tsindex ebab7e0ba..9a88969e1 100644--- a/electron/native-bridge/services/compositorViewService.ts+++ b/electron/native-bridge/services/compositorViewService.ts@@ -3,6 +3,7 @@ import { createRequire } from "node:module";import path from "node:path";import { fileURLToPath } from "node:url";import { app, sharedTexture, type WebFrameMain } from "electron";+import type { ShaderEffectCheck } from "../../../src/lib/ai-edition/shaderEffects";import {type CursorKind,readCursorAsArrow,@@ -588,6 +589,21 @@ export class CompositorViewService {return resolveSceneAssetPath(SEGMENTATION_MODEL_ASSET) ? "ready" : "no-model";}+ /** Checks an agent-written effect's WGSL in the compositor (`docs/shader-effects.md`, E.3): the+ * compile error, or the cost tier. `null` when this addon cannot answer, which the agent tools+ * report as "not checked" rather than as a compile failure. */+ checkShaderEffect(+ code: string,+ params: ReadonlyArray<{ name: string; type: string }>,+ ): ShaderEffectCheck | null {+ const addon = this.ensureAddon();+ if (!addon || typeof addon.checkShaderEffect !== "function") return null;+ return addon.checkShaderEffect(+ code,+ JSON.stringify(params.map(({ name, type }) => ({ name, type }))),+ );+ }+/** Subject mask for one camera frame, no view needed — see `segmentFrame` in the addon.* `null` when the addon (or this version of it) or the model is missing. */async segmentFrame(rgba: Uint8Array): Promise<Uint8Array | null> {diff --git a/skills/openscreen-demo/SKILL.md b/skills/openscreen-demo/SKILL.mdindex 8fff292bf..b6450dabb 100644--- a/skills/openscreen-demo/SKILL.md+++ b/skills/openscreen-demo/SKILL.md@@ -86,6 +86,7 @@ Use the document tools, always with the `projectId`. Read the project first with- **Frame the app:** when the take shows more than the app, `setClipRange` with a `cropRegion` crops the clip to it.- **Callouts:** one `addAnnotation` per beat, short, placed clear of the action. Text sits on the output frame and stays put when the picture zooms. Use its look for hierarchy: a large title with `animation: "typewriter"`, step callouts with `animation: "pop"`. An arrow, `type: "figure"`, points at a control the text names. `setAnnotation` moves or restyles one.- **Hide what is private:** an email address, a token or a name that shows on screen gets `addAnnotation` with `type: "blur"` over the span it is visible. Its x, y, width and height are percent of the recording's picture, not of the frame, so it stays on what it hides when a zoom moves the picture. Check it with `getFrames`.+- **An effect no setting offers** (a ripple on each click, a glitch on a cut, a color grade): read the resource `openscreen://effects/contract`, then write it as WGSL with `addEffect`. It reads the clicks and the pointer already placed on the frame. Code that does not compile comes back with the compiler's error at your line: fix it and call again. The result gives a cost tier; keep heavy effects to short spans. Check the effect with `getFrames` at the moments it acts.- **Music or a voiceover:** `importMedia` with the file's absolute path adds it to the project's media, then `addAudio` plays it over a span. `removeMedia` takes back a file the timeline no longer uses.- **Name:** `setProjectInfo` renames the project, if the `title` given to `stopRecording` no longer fits.- **Captions, for a narrated take** (`startRecording` with `microphone: true`):diff --git a/src/components/ai-edition/NewEditorShell.tsx b/src/components/ai-edition/NewEditorShell.tsxindex 08b52f21e..5c1974a2d 100644--- a/src/components/ai-edition/NewEditorShell.tsx+++ b/src/components/ai-edition/NewEditorShell.tsx@@ -1173,6 +1173,8 @@ export function NewEditorShell() {return;}+ // The editor has no effect lane yet, so nothing selects a shader effect to copy.+ if (sel.kind === "shaderEffect") return;const source =sel.kind === "zoom"? tl.zoomRegionsdiff --git a/src/lib/ai-edition/document/timeline.test.ts b/src/lib/ai-edition/document/timeline.test.tsindex 7f203f32a..765e07202 100644--- a/src/lib/ai-edition/document/timeline.test.ts+++ b/src/lib/ai-edition/document/timeline.test.ts@@ -1962,6 +1962,50 @@ describe("removeClip — delete a clip, close the gap, drop its pills", () => {expect(next.zoomRanges[0]).toMatchObject({ startMs: 2000, endMs: 4000 });});+ // A shader effect is a region like a zoom: it moves with its clip and goes with it. Its+ // definition goes with its last region, so no dead code stays in the document.+ it("moves a shader effect with its clip, and drops it and its definition with the clip", () => {+ const effect = {+ id: "fx",+ name: "Pass-through",+ description: "",+ contract: 1,+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }",+ params: [],+ origin: "agent" as const,+ };+ const range = (id: string, clipId: string, startMs: number) => ({+ id,+ clipId,+ sourceStartSec: 2,+ sourceEndSec: 4,+ startMs,+ endMs: startMs + 2000,+ effectId: "fx",+ values: {},+ target: "image" as const,+ lane: 0,+ });+ const before = { ...doc(), shaderEffects: [effect] };+ const onB = removeClip(+ {+ ...before,+ shaderEffectRanges: [range("fx_a", "clip_a", 2000), range("fx_b", "clip_b", 12000)],+ },+ "clip_a",+ );+ expect(onB.shaderEffectRanges).toEqual([+ expect.objectContaining({ id: "fx_b", startMs: 2000 }),+ ]);+ expect(onB.shaderEffects).toEqual([effect]);+ const gone = removeClip(+ { ...before, shaderEffectRanges: [range("fx_a", "clip_a", 2000)] },+ "clip_a",+ );+ expect(gone.shaderEffectRanges).toEqual([]);+ expect(gone.shaderEffects).toEqual([]);+ });+it("preserves a bare clipId that is not a complete source anchor", () => {const before = doc();before.zoomRanges.push(diff --git a/src/lib/ai-edition/document/timeline.ts b/src/lib/ai-edition/document/timeline.tsindex 4e1a75141..2b8366ba3 100644--- a/src/lib/ai-edition/document/timeline.ts+++ b/src/lib/ai-edition/document/timeline.ts@@ -21,6 +21,7 @@ import type {*/export type PlaybackSegment = AxcutClip;+import { pruneUnusedShaderEffects } from "../shaderEffects";import { type Interval, subtractInterval } from "../timeline/intervals";import { keptRawSpans } from "../timeline/programme-time";import {@@ -43,7 +44,14 @@ import { createId } from "./ids";* exist" has exactly one definition. `trim` is a source-time cut; the rest are pill-merged* effects (zoom / speed / annotation / camera-fullscreen). Clips are removed via* {@link removeClip}, not here — deleting a clip reflows the whole timeline. */-export type RegionKind = "zoom" | "trim" | "annotation" | "speed" | "cameraFullscreen" | "audio";+export type RegionKind =+ | "zoom"+ | "trim"+ | "annotation"+ | "speed"+ | "cameraFullscreen"+ | "audio"+ | "shaderEffect";/** Length a clip is given before its media has been probed. Lives here, in the pure* document layer, because that layer decides which clips are still waiting for a real@@ -378,12 +386,17 @@ export function mapAllRegionCollections(? (legacy?.cameraFullscreenRegions as StoredRegion[]): undefined;- return {+ // A clip edit can take a shader effect's last fragment with it: its definition goes too.+ return pruneUnusedShaderEffects({...document,zoomRanges: fn(document.zoomRanges as unknown as StoredRegion[],"zoom",) as unknown as AxcutDocument["zoomRanges"],+ shaderEffectRanges: fn(+ (document.shaderEffectRanges ?? []) as unknown as StoredRegion[],+ "fxr",+ ) as unknown as AxcutDocument["shaderEffectRanges"],annotations: fn(document.annotations as unknown as StoredRegion[],"ann",@@ -420,7 +433,7 @@ export function mapAllRegionCollections(: {}),}: document.legacyEditor,- };+ });}/** A clamped fragment this short has no surviving content — treat it as fully@@ -734,6 +747,7 @@ function anchoredRegionsOf(document: AxcutDocument): Array<{ id: string; clipId:document.annotations as unknown as StoredRegion[],(legacy?.speedRegions as StoredRegion[] | undefined) ?? [],(legacy?.cameraFullscreenRegions as StoredRegion[] | undefined) ?? [],+ (document.shaderEffectRanges ?? []) as unknown as StoredRegion[],];return collections.flat()@@ -1197,6 +1211,12 @@ export function removeRegion(document: AxcutDocument, kind: RegionKind, id: stri);return { ...document, legacyEditor: { ...legacy, cameraFullscreenRegions: prev } };}+ case "shaderEffect":+ // The definition goes with its last region: no dead code stays in the document.+ return pruneUnusedShaderEffects({+ ...document,+ shaderEffectRanges: dropPillById(document.shaderEffectRanges ?? [], id),+ });default: {// ponytail: exhaustive — TS errors here if a new RegionKind is added.const exhaustive: never = kind;@@ -1241,6 +1261,10 @@ const EDIT_REGIONS: Record<count: (d) => legacyRegionCount(d, "cameraFullscreenRegions"),clear: (d) => withLegacyRegionsCleared(d, "cameraFullscreenRegions"),},+ shaderEffect: {+ count: (d) => (d.shaderEffectRanges ?? []).length,+ clear: (d) => ({ ...d, shaderEffectRanges: [], shaderEffects: [] }),+ },};/** How many edit regions the document holds, stored rows and not pills. Pure. */diff --git a/src/lib/ai-edition/schema/index.test.ts b/src/lib/ai-edition/schema/index.test.tsindex 38aabd870..977f16ba5 100644--- a/src/lib/ai-edition/schema/index.test.ts+++ b/src/lib/ai-edition/schema/index.test.ts@@ -13,6 +13,8 @@ import {legacyEditorSchema,parseDocumentFile,rangeSchema,+ shaderEffectRangeSchema,+ shaderEffectSchema,timelineSchema,trimRangeSchema,upgradeV7DocumentToV8,@@ -1245,3 +1247,44 @@ describe("follow-cursor zooms read as orbits under auto focus", () => {expect(migrateRawDocumentToCurrent(manualOrbit)).toBe(manualOrbit);});});++describe("shader effects (docs/shader-effects.md)", () => {+ const effect = {+ id: "fx",+ name: "Dim",+ contract: 1,+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }",+ params: [{ type: "number", name: "level", default: 1, min: 0, max: 2 }],+ };++ it("fills the defaults of a definition and of a region", () => {+ expect(shaderEffectSchema.parse(effect)).toMatchObject({+ description: "",+ origin: "agent",+ params: [{ name: "level", label: "" }],+ });+ expect(+ shaderEffectRangeSchema.parse({ id: "r", startMs: 0, endMs: 1000, effectId: "fx" }),+ ).toMatchObject({ values: {}, target: "image", lane: 0 });+ });++ it("refuses a parameter declared twice, or a default out of its bounds", () => {+ const twice = { ...effect, params: [effect.params[0], effect.params[0]] };+ expect(shaderEffectSchema.safeParse(twice).success).toBe(false);+ const outside = { ...effect, params: [{ ...effect.params[0], default: 3 }] };+ expect(shaderEffectSchema.safeParse(outside).success).toBe(false);+ });++ // A newer build may write a contract this one does not know: the document must still open,+ // and the compositor leaves those regions undrawn.+ it("loads a contract version it does not know", () => {+ expect(shaderEffectSchema.safeParse({ ...effect, contract: 2 }).success).toBe(true);+ });++ it("keeps a document without effects free of both keys", () => {+ const doc = createEmptyDocument({ title: "t", projectId: "p" });+ const parsed = documentSchema.parse(doc);+ expect("shaderEffects" in parsed).toBe(false);+ expect("shaderEffectRanges" in parsed).toBe(false);+ });+});diff --git a/src/lib/ai-edition/schema/index.ts b/src/lib/ai-edition/schema/index.tsindex 05edecab0..96b2eff94 100644--- a/src/lib/ai-edition/schema/index.ts+++ b/src/lib/ai-edition/schema/index.ts@@ -528,6 +528,113 @@ export const zoomRegionSchema = endGteStart("startMs",);+// Agent-written shader effects (docs/shader-effects.md, B). A definition carries the WGSL and its+// declared parameters; a range poses it on the timeline, clip-anchored like a zoom, with its own+// parameter values. Additive: a document written before these existed loads unchanged.+const shaderEffectParamName = z.string().regex(/^[a-z][a-z0-9_]{0,31}$/);+const shaderEffectColor = z.string().regex(/^#(?:[0-9a-fA-F]{6}|[0-9a-fA-F]{8})$/);+const shaderEffectPoint = z.object({ x: z.number().min(0).max(1), y: z.number().min(0).max(1) });+// Written by the agent, in the user's language, shown as-is: content, not interface copy.+const shaderEffectLabel = z.string().max(40).default("");++export const shaderEffectParamSchema = z.discriminatedUnion("type", [+ z.object({+ type: z.literal("number"),+ name: shaderEffectParamName,+ label: shaderEffectLabel,+ default: z.number(),+ min: z.number(),+ max: z.number(),+ }),+ z.object({+ type: z.literal("toggle"),+ name: shaderEffectParamName,+ label: shaderEffectLabel,+ default: z.boolean(),+ }),+ z.object({+ type: z.literal("color"),+ name: shaderEffectParamName,+ label: shaderEffectLabel,+ default: shaderEffectColor,+ }),+ z.object({+ type: z.literal("point"),+ name: shaderEffectParamName,+ label: shaderEffectLabel,+ default: shaderEffectPoint,+ }),+]);++export const shaderEffectValueSchema = z.union([+ z.number(),+ z.boolean(),+ shaderEffectColor,+ shaderEffectPoint,+]);++export const shaderEffectSchema = z+ .object({+ id: z.string().min(1),+ // What the pill shows.+ name: z.string().min(1).max(60),+ // For the next agent: what the effect does, in one sentence.+ description: z.string().max(500).default(""),+ // The WGSL contract version. Not a literal: a version this build does not know (written by+ // a newer one) must still load; the compositor leaves its regions undrawn.+ contract: z.number().int().positive(),+ code: z.string().min(1).max(65_536),+ // The `max` here is a technical ceiling, not a budget: see the cost tiers.+ params: z.array(shaderEffectParamSchema).max(32).default([]),+ origin: z.enum(["agent", "user"]).default("agent"),+ // The compositor's cost estimate, cached when the code was last checked. Derived, never an+ // input: `frameReads` is `frame()` reads per pixel, `tier` its indicative tier.+ cost: z+ .object({+ tier: z.enum(["light", "medium", "heavy", "very-heavy"]),+ frameReads: z.number().nonnegative(),+ })+ .optional(),+ })+ .superRefine((effect, ctx) => {+ const names = new Set<string>();+ for (const [index, param] of effect.params.entries()) {+ if (names.has(param.name)) {+ ctx.addIssue({+ code: "custom",+ message: `parameter "${param.name}" is declared twice`,+ path: ["params", index],+ });+ }+ names.add(param.name);+ if (param.type === "number" && !(param.min <= param.default && param.default <= param.max)) {+ ctx.addIssue({+ code: "custom",+ message: `parameter "${param.name}": min ≤ default ≤ max`,+ path: ["params", index],+ });+ }+ }+ });++export const shaderEffectRangeSchema = endGteStart(+ z.object({+ id: z.string().min(1),+ startMs: z.number().nonnegative(),+ endMs: z.number().nonnegative(),+ ...clipAnchorShape,+ effectId: z.string().min(1),+ // Only the values that differ from the definition's defaults.+ values: z.record(z.string(), shaderEffectValueSchema).default({}),+ // What the effect transforms: the picture under the texts, the texts alone, or both.+ target: z.enum(["image", "text", "both"]).default("image"),+ // Application order, lane 0 first, closest to the content. No overlap within a lane.+ lane: z.number().int().min(0).max(7).default(0),+ }),+ "endMs",+ "startMs",+);+// External audio import (issue #350) — voiceover / BGM / SFX layered over the// programme. Unlike zoom/speed/annotation/trim, an audio track is NOT// clip-anchored: it floats over the whole timeline, addressed in RAW/document@@ -630,6 +737,11 @@ const documentSchemaShape = z.object({}),annotations: z.array(annotationRegionSchema).default([]),zoomRanges: z.array(zoomRegionSchema).default([]),+ // Agent-written shader effects (docs/shader-effects.md). Optional rather than defaulted, like+ // `vocabulary`: a document without effects carries no trace of them, and every reader treats+ // an absent list as empty. An older build drops both keys when it saves.+ shaderEffects: z.array(shaderEffectSchema).optional(),+ shaderEffectRanges: z.array(shaderEffectRangeSchema).optional(),// Imported audio tracks (issue #350). Defaulted so every document written// before this loads unchanged; an older build simply strips the key on save.audioTracks: z.array(audioTrackSchema).default([]),@@ -1265,6 +1377,10 @@ export type AxcutAnnotationRegion = z.infer<typeof annotationRegionSchema>;export type AxcutZoomRegion = z.infer<typeof zoomRegionSchema>;export type AxcutCameraTrack = z.infer<typeof cameraTrackSchema>;export type AxcutAudioTrack = z.infer<typeof audioTrackSchema>;+export type AxcutShaderEffect = z.infer<typeof shaderEffectSchema>;+export type AxcutShaderEffectParam = z.infer<typeof shaderEffectParamSchema>;+export type AxcutShaderEffectRange = z.infer<typeof shaderEffectRangeSchema>;+export type AxcutShaderEffectValue = z.infer<typeof shaderEffectValueSchema>;export type AxcutLegacyEditor = z.infer<typeof legacyEditorSchema>;export type AxcutDocument = z.infer<typeof documentSchema>;export type AxcutDocumentInput = z.input<typeof documentSchema>;diff --git a/src/lib/ai-edition/shaderEffects.ts b/src/lib/ai-edition/shaderEffects.tsnew file mode 100644index 000000000..8aa973128--- /dev/null+++ b/src/lib/ai-edition/shaderEffects.ts@@ -0,0 +1,168 @@+import type {+ AxcutDocument,+ AxcutShaderEffect,+ AxcutShaderEffectParam,+ AxcutShaderEffectRange,+ AxcutShaderEffectValue,+} from "./schema";++/**+ * Agent-written shader effects (docs/shader-effects.md): the pure helpers the agent tools, the+ * scene projection and the editor share. The WGSL itself is checked and compiled by the+ * compositor (`crates/compositor/src/shader_fx.rs`).+ */++/** The WGSL contract this build writes and draws. */+export const SHADER_EFFECT_CONTRACT = 1;++export type ShaderEffectTier = "light" | "medium" | "heavy" | "very-heavy";++/** What the compositor says of an effect's code (`checkShaderEffect` in the addon). */+export interface ShaderEffectCheck {+ error?: string | null;+ tier?: ShaderEffectTier | null;+ frameReads?: number | null;+ iterations?: number | null;+}++/** Asks the compositor about a code; `null` where nothing can answer (no addon, a test). */+export type ShaderEffectChecker = (+ code: string,+ params: ReadonlyArray<Pick<AxcutShaderEffectParam, "name" | "type">>,+) => ShaderEffectCheck | null;++export function findShaderEffect(+ document: AxcutDocument,+ effectId: string,+): AxcutShaderEffect | undefined {+ return (document.shaderEffects ?? []).find((effect) => effect.id === effectId);+}++const clamp = (value: number, min: number, max: number) => Math.min(max, Math.max(min, value));+const isColor = (value: unknown): value is string =>+ typeof value === "string" && /^#(?:[0-9a-fA-F]{6}|[0-9a-fA-F]{8})$/.test(value);+const isPoint = (value: unknown): value is { x: number; y: number } =>+ typeof value === "object" &&+ value !== null &&+ Number.isFinite((value as { x?: unknown }).x) &&+ Number.isFinite((value as { y?: unknown }).y);++/** The value a parameter takes: `value` when it has the right type, clamped to the bounds, the+ * default otherwise. */+export function resolveShaderEffectValue(+ param: AxcutShaderEffectParam,+ value: unknown,+): AxcutShaderEffectValue {+ switch (param.type) {+ case "number":+ return typeof value === "number" && Number.isFinite(value)+ ? clamp(value, param.min, param.max)+ : param.default;+ case "toggle":+ return typeof value === "boolean" ? value : param.default;+ case "color":+ return isColor(value) ? value : param.default;+ case "point":+ return isPoint(value) ? { x: clamp(value.x, 0, 1), y: clamp(value.y, 0, 1) } : param.default;+ }+}++const sameValue = (a: AxcutShaderEffectValue, b: AxcutShaderEffectValue) =>+ JSON.stringify(a) === JSON.stringify(b);++/** What a region stores: only the values that differ from the defaults, each resolved against+ * its parameter. Names the definition does not declare are dropped. */+export function storedShaderEffectValues(+ effect: Pick<AxcutShaderEffect, "params">,+ values: Record<string, unknown>,+): Record<string, AxcutShaderEffectValue> {+ const stored: Record<string, AxcutShaderEffectValue> = {};+ for (const param of effect.params) {+ if (!(param.name in values)) continue;+ const value = resolveShaderEffectValue(param, values[param.name]);+ if (!sameValue(value, param.default)) stored[param.name] = value;+ }+ return stored;+}++function hexToRgba(hex: string): [number, number, number, number] {+ const channel = (i: number) => Number.parseInt(hex.slice(1 + i * 2, 3 + i * 2), 16) / 255;+ return [channel(0), channel(1), channel(2), hex.length === 9 ? channel(3) : 1];+}++/** A parameter's value as the compositor packs it: one float, or 2 (point) or 4 (color). */+export function nativeShaderEffectValue(+ param: AxcutShaderEffectParam,+ value: unknown,+): number | number[] {+ const resolved = resolveShaderEffectValue(param, value);+ switch (param.type) {+ case "number":+ return resolved as number;+ case "toggle":+ return resolved ? 1 : 0;+ case "color":+ return hexToRgba(resolved as string);+ case "point": {+ const { x, y } = resolved as { x: number; y: number };+ return [x, y];+ }+ }+}++/** The document without the definitions no region uses: no dead code is kept. */+export function pruneUnusedShaderEffects(document: AxcutDocument): AxcutDocument {+ const used = new Set((document.shaderEffectRanges ?? []).map((range) => range.effectId));+ if ((document.shaderEffects ?? []).every((effect) => used.has(effect.id))) return document;+ return {+ ...document,+ shaderEffects: (document.shaderEffects ?? []).filter((effect) => used.has(effect.id)),+ };+}++/** The tier of a number of `frame()` reads per pixel, as the compositor computes it. */+export function shaderEffectTierOf(frameReads: number): ShaderEffectTier {+ if (frameReads <= 8) return "light";+ if (frameReads <= 32) return "medium";+ if (frameReads <= 128) return "heavy";+ return "very-heavy";+}++/** The longest span a tier is recommended for, in seconds (spec C.7). */+const TIER_SPAN_SEC: Record<ShaderEffectTier, number> = {+ light: Number.POSITIVE_INFINITY,+ medium: 60,+ heavy: 10,+ "very-heavy": 3,+};++/** The recommendation an agent gets when a tier runs longer than it is meant to, else null. */+export function shaderEffectCostAdvice(tier: ShaderEffectTier, spanSec: number): string | null {+ if (spanSec <= TIER_SPAN_SEC[tier]) return null;+ const lighter =+ tier === "very-heavy" ? "heavy or lighter" : tier === "heavy" ? "medium or light" : "light";+ return `${tier} over ${spanSec.toFixed(1)} s: this tier is meant for at most ${TIER_SPAN_SEC[tier]} s; for a region this long, aim ${lighter}. It is a recommendation, not a limit.`;+}++/** The `frame()` reads of every effect running at the busiest moment of `[startMs, endMs]`. */+export function busiestShaderEffectReads(+ document: AxcutDocument,+ startMs: number,+ endMs: number,+): number {+ const ranges = (document.shaderEffectRanges ?? []).filter(+ (r) => r.startMs < endMs && r.endMs > startMs,+ );+ const reads = (range: AxcutShaderEffectRange) =>+ findShaderEffect(document, range.effectId)?.cost?.frameReads ?? 0;+ // The load only changes at a region edge, so the busiest moment is one of the starts.+ let busiest = 0;+ for (const at of [startMs, ...ranges.map((r) => r.startMs)]) {+ if (at < startMs || at >= endMs) continue;+ const load = ranges+ .filter((r) => r.startMs <= at && r.endMs > at)+ .reduce((sum, r) => sum + reads(r), 0);+ busiest = Math.max(busiest, load);+ }+ return busiest;+}diff --git a/src/lib/ai-edition/store/useTimeline.test.ts b/src/lib/ai-edition/store/useTimeline.test.tsindex 917188529..cc12709c1 100644--- a/src/lib/ai-edition/store/useTimeline.test.ts+++ b/src/lib/ai-edition/store/useTimeline.test.ts@@ -750,6 +750,42 @@ describe("useTimeline.clearTimeline", () => {annotation: (tl) => tl.addAnnotation(),speed: (tl) => tl.addSpeed(),cameraFullscreen: (tl) => tl.addCameraFullscreen(),+ // The editor has no effect lane: agents add effects, so this one is written as they do.+ shaderEffect: async () => {+ const doc = useProjectStore.getState().document as AxcutDocument;+ const clip = doc.timeline.clips[0];+ await useProjectStore.getState().saveDocument(+ {+ ...doc,+ shaderEffects: [+ {+ id: "fx_1",+ name: "Pass-through",+ description: "",+ contract: 1,+ code: "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }",+ params: [],+ origin: "agent",+ },+ ],+ shaderEffectRanges: [+ {+ id: "fxr_1",+ startMs: 0,+ endMs: 1000,+ clipId: clip.id,+ sourceStartSec: clip.sourceStartSec,+ sourceEndSec: clip.sourceStartSec + 1,+ effectId: "fx_1",+ values: {},+ target: "image",+ lane: 0,+ },+ ],+ },+ { history: true },+ );+ },};const editKinds = Object.keys(addOf) as EditKind[];@@ -761,6 +797,7 @@ describe("useTimeline.clearTimeline", () => {annotation: doc?.annotations.length ?? 0,speed: doc ? readSpeedRegions(doc).length : 0,cameraFullscreen: legacy.cameraFullscreenRegions?.length ?? 0,+ shaderEffect: doc?.shaderEffectRanges?.length ?? 0,};};const noRegions: Record<EditKind, number> = {@@ -769,6 +806,7 @@ describe("useTimeline.clearTimeline", () => {annotation: 0,speed: 0,cameraFullscreen: 0,+ shaderEffect: 0,};// A project holding everything that is NOT an edit: two clips, a webcam, an imported@@ -928,6 +966,7 @@ describe("useTimeline.clearTimeline", () => {annotation: 1,speed: 1,cameraFullscreen: 1,+ shaderEffect: 1,});await act(async () => {@@ -967,6 +1006,7 @@ describe("useTimeline.clearTimeline", () => {annotation: 1,speed: 1,cameraFullscreen: 1,+ shaderEffect: 1,});act(() => {diff --git a/src/native/sceneDescription.test.ts b/src/native/sceneDescription.test.tsindex 0d84461b5..773df2c5e 100644--- a/src/native/sceneDescription.test.ts+++ b/src/native/sceneDescription.test.ts@@ -102,6 +102,8 @@ function makeDoc(annotations: overrides.annotations ?? [],zoomRanges: overrides.zoomRanges ?? [],audioTracks: overrides.audioTracks ?? [],+ ...(overrides.shaderEffects ? { shaderEffects: overrides.shaderEffects } : {}),+ ...(overrides.shaderEffectRanges ? { shaderEffectRanges: overrides.shaderEffectRanges } : {}),legacyEditor: overrides.legacyEditor ?? null,};}@@ -2949,3 +2951,99 @@ describe("buildSceneDescription.audioTracks", () => {expect(buildSceneDescription(doc).audioTracks).toEqual([]);});});++// --- shaderEffects -------------------------------------------------------------++describe("buildSceneDescription.shaderEffects", () => {+ const code = "fn effect(uv: vec2<f32>) -> vec4<f32> { return frame(uv); }";+ const effect = {+ id: "fx1",+ name: "Tint",+ description: "",+ contract: 1,+ code,+ origin: "agent" as const,+ params: [+ { type: "number" as const, name: "level", label: "", default: 1, min: 0, max: 2 },+ { type: "color" as const, name: "tint", label: "", default: "#000000" },+ { type: "toggle" as const, name: "on", label: "", default: false },+ ],+ };++ it("is omitted when there are none", () => {+ expect(buildSceneDescription(makeDoc()).shaderEffects).toBeUndefined();+ });++ // A trim inside the region cuts it in two fragments on the source clock, but it stays one+ // pill: both fragments carry the pill's span on the programme clock, which the trim shortened.+ it("gives every fragment the whole pill on the programme clock, and every value", () => {+ const doc = makeDoc({+ assets: [makeAsset({ id: "a", originalPath: "/a.mp4" })],+ clips: [+ makeClip({+ id: "c1",+ assetId: "a",+ sourceStartSec: 0,+ sourceEndSec: 10,+ timelineStartSec: 0,+ timelineEndSec: 10,+ }),+ ],+ timeline: {+ trimRanges: [+ { id: "t1", assetId: "a", startSec: 2, endSec: 4, reason: "", origin: "user" },+ ],+ },+ shaderEffects: [effect],+ shaderEffectRanges: [+ {+ id: "fxr1",+ startMs: 1000,+ endMs: 6000,+ effectId: "fx1",+ values: { tint: "#ff0000", on: true },+ target: "text",+ lane: 2,+ },+ ],+ });+ const { shaderEffects } = buildSceneDescription(doc);+ expect(shaderEffects?.map((e) => [e.startSec, e.endSec, e.clipIndex])).toEqual([+ [1, 2, 0],+ [4, 6, 1],+ ]);+ for (const fragment of shaderEffects ?? []) {+ expect(fragment).toMatchObject({+ programmeStartSec: 1,+ programmeEndSec: 4,+ target: "text",+ lane: 2,+ contract: 1,+ code,+ params: [+ { name: "level", type: "number" },+ { name: "tint", type: "color" },+ { name: "on", type: "toggle" },+ ],+ values: { level: 1, tint: [1, 0, 0, 1], on: 1 },+ });+ }+ });++ it("drops a region whose definition is gone", () => {+ const doc = makeDoc({+ shaderEffectRanges: [+ {+ id: "fxr1",+ startMs: 0,+ endMs: 1000,+ effectId: "fx_gone",+ values: {},+ target: "image",+ lane: 0,+ },+ ],+ });+ expect(buildSceneDescription(doc).shaderEffects).toBeUndefined();+ });+});diff --git a/src/native/sceneDescription.ts b/src/native/sceneDescription.tsindex 5cf2a8e5d..8c8727ee7 100644--- a/src/native/sceneDescription.ts+++ b/src/native/sceneDescription.ts@@ -41,12 +41,16 @@ import {resolvePlaybackSegments,} from "@/lib/ai-edition/document/timeline";import type { AxcutAudioTrack, AxcutClip, AxcutDocument } from "@/lib/ai-edition/schema";+import { nativeShaderEffectValue } from "@/lib/ai-edition/shaderEffects";import { getEditorSettings } from "@/lib/ai-edition/store/editorSettings";import { assetCameraSource } from "@/lib/ai-edition/timeline/camera";import { resolveClipSourceEndSec } from "@/lib/ai-edition/timeline/clipDuration";import { removedRawSpans } from "@/lib/ai-edition/timeline/programme-time";import { takeProgramme } from "@/lib/ai-edition/timeline/take-programme";-import { projectRegionsToSource } from "@/lib/ai-edition/timeline/timelineMap";+import {+ coalesceRegionsForRuler,+ projectRegionsToSource,+} from "@/lib/ai-edition/timeline/timelineMap";import {MAGNIFICATION_REFERENCE_PX,MAX_ZOOM_SCALE,@@ -120,6 +124,26 @@ export interface SceneZoomRegion {hideCursor?: boolean;}+/** One fragment of a shader effect region (`(document.shaderEffectRanges ?? [])`), projected onto its+ * clip's source time like the other regions, with its definition. `scene.rs`+ * `SceneShaderEffect` reads it; docs/shader-effects.md, B.5. */+export interface SceneShaderEffect {+ startSec: number;+ endSec: number;+ clipIndex?: number;+ /** The WHOLE pill on the programme clock (trims skipped, speed applied): the compositor's+ * `ctx.region` runs across it instead of restarting at each fragment. */+ programmeStartSec: number;+ programmeEndSec: number;+ target: "image" | "text" | "both";+ lane: number;+ contract: number;+ code: string;+ params: Array<{ name: string; type: "number" | "toggle" | "color" | "point" }>;+ /** Every declared parameter, defaults filled in: one float, or 2 (point) or 4 (color). */+ values: Record<string, number | number[]>;+}+/** A "Full Camera" timeline region (from `legacyEditor.cameraFullscreenRegions`). Times in seconds. */export interface SceneCameraFullscreenRegion {startSec: number;@@ -479,6 +503,8 @@ export interface SceneDescription {* source-time span after `projectRegionsToSourceTime`). Empty when none set.*/cameraFullscreenRegions: SceneCameraFullscreenRegion[];+ /** Agent-written shader effects, omitted when there are none. */+ shaderEffects?: SceneShaderEffect[];/*** Speed regions projected onto each clip's source time (one entry per* source-time span after `projectRegionsToSourceTime`). Empty when none set.@@ -1085,6 +1111,59 @@ export function buildSceneDescription(() => createId("speed"),);+ // Shader effects: projected like the zooms, and each fragment told where its WHOLE pill sits+ // on the programme clock, through the same projection the exporter plays the timeline with+ // (trims skipped, speed applied). A pill cut by a trim or a clip boundary keeps one progress.+ const effectsById = new Map((document.shaderEffects ?? []).map((effect) => [effect.id, effect]));+ const programmeSec = (rawSec: number) =>+ projectRawTimelineSecToPlayback(+ projectedClips,+ document.timeline.trimRanges,+ rawSec,+ rawSpeedRegions,+ );+ const pillSpanById = new Map<string, { programmeStartSec: number; programmeEndSec: number }>();+ for (const pill of coalesceRegionsForRuler(document.shaderEffectRanges ?? [])) {+ const span = {+ programmeStartSec: programmeSec(pill.start),+ programmeEndSec: programmeSec(pill.end),+ };+ for (const id of pill.ids) pillSpanById.set(id, span);+ }+ const projectedShaderEffects = projectRegionsToSource(+ (document.shaderEffectRanges ?? []).map((range) => ({+ ...range,+ ...pillSpanById.get(range.id),+ })),+ visibleClips,+ document.timeline.clips,+ () => createId("fxr"),+ );+ const sceneShaderEffects: SceneShaderEffect[] = projectedShaderEffects.flatMap((region) => {+ const effect = effectsById.get(region.effectId);+ if (!effect) return [];+ return [+ {+ startSec: region.startMs / 1000,+ endSec: region.endMs / 1000,+ clipIndex: region.clipIndex,+ programmeStartSec: region.programmeStartSec ?? region.startMs / 1000,+ programmeEndSec: region.programmeEndSec ?? region.endMs / 1000,+ target: region.target,+ lane: region.lane,+ contract: effect.contract,+ code: effect.code,+ params: effect.params.map(({ name, type }) => ({ name, type })),+ values: Object.fromEntries(+ effect.params.map((param) => [+ param.name,+ nativeShaderEffectValue(param, region.values[param.name]),+ ]),+ ),+ },+ ];+ });+// Webcam rect, single source of truth between preview & native :// on résout le rect AVEC LA MÊME maths que `PreviewCanvas.computeCompositeLayout` et on// l'envoie au natif dans `layout.webcamRect` (fractions du cadre de sortie). Le natif le@@ -1470,6 +1549,7 @@ export function buildSceneDescription(clipIndex: region.clipIndex,})),cropByClip,+ ...(sceneShaderEffects.length > 0 ? { shaderEffects: sceneShaderEffects } : {}),output: { ...pickOutputDims(document, settings.aspectRatio), fps: null },// Omitted rather than sent as `{mode:"none"}`: the Rust side defaults the field, and// every project without a webcam effect would otherwise carry it for nothing.diff --git a/technical-documentation/architecture/ai-agent.md b/technical-documentation/architecture/ai-agent.mdindex d36f20614..7b5f44333 100644--- a/technical-documentation/architecture/ai-agent.md+++ b/technical-documentation/architecture/ai-agent.md@@ -41,7 +41,7 @@ The agent is built with LangChain's `createAgent`, not `deepagents`' `createDeep## Tool schema-The model never free-writes the project document. It can only call the fixed set of 26 tools built by `deep-agent/service.ts#buildTools`, described by `TOOL_DESCRIPTIONS` in the same file and validated against the Zod argument schemas in `agent-tools.ts`; the executor parses JSON, validates arguments, and returns either a new schema-valid snapshot or an error. Those three surfaces (descriptions, built tools, executor cases) are pinned to one another by `deep-agent/service.test.ts` — an earlier fourth surface, `AGENT_TOOL_SPECS`, described the tools in JSON Schema for a provider it had stopped reaching, and drifted. The tools operate on the same [document model](document-model.md) as manual editing.+The model never free-writes the project document. It can only call the fixed set of tools built by `deep-agent/service.ts#buildTools` (`OPENSCREEN_TOOL_NAMES`), described by `TOOL_DESCRIPTIONS` in the same file and validated against the Zod argument schemas in `agent-tools.ts`; the executor parses JSON, validates arguments, and returns either a new schema-valid snapshot or an error. Those three surfaces (descriptions, built tools, executor cases) are pinned to one another by `deep-agent/service.test.ts` — an earlier fourth surface, `AGENT_TOOL_SPECS`, described the tools in JSON Schema for a provider it had stopped reaching, and drifted. The tools operate on the same [document model](document-model.md) as manual editing.| Tool | What it does | What it mutates ||---|---|---|@@ -64,6 +64,10 @@ The model never free-writes the project document. It can only call the fixed set| `setAnnotation` | Moves, resizes, retypes or restyles an annotation. A type change goes through `convertAnnotationKind` with the footage under each fragment (`annotationFootageRect`), as the inspector's type switch does; `figureData` / `blurData` merge only the keys sent. Arguments of another kind (width on a text, blurData on an arrow) are refused. | The corresponding clip-anchored `annotations` fragments. || `addCameraFullscreen` | Adds a camera-fullscreen region over virtual timeline time; refused when no clip under the span comes from an asset with a linked `cameraTrack`, since such a region can only render nothing. | `legacyEditor.cameraFullscreenRegions`. || `setCameraFullscreen` | Moves or resizes a camera-fullscreen pill, under the same camera requirement as `addCameraFullscreen`. | The corresponding `legacyEditor.cameraFullscreenRegions` fragments. |+| `addEffect` | Writes a shader effect (WGSL, [docs/shader-effects.md](../../docs/shader-effects.md)) and poses it over virtual timeline time, or poses one of the project's effects again by `effectId`. The compositor checks the code first (`checkShaderEffect`); code that does not compile is refused with the error at the agent's line. The result gives the cost tier and, when the tier runs long, a recommendation, never a refusal. Two effects of one lane cannot overlap. | `shaderEffects` (a new definition) and `shaderEffectRanges`. |+| `setEffect` | Moves, re-tunes (`values`), retargets or changes the lane of an effect pill. | The pill's `shaderEffectRanges` fragments. |+| `setEffectDefinition` | Changes an effect's name, description, code or parameters, and so every region using it; new code is checked first. A removed parameter's values drop. | The `shaderEffects` entry and its regions' `values`. |+| `getEffect` | Reads one effect in full, code included; `getCurrentDocument` lists effects without their code. | Read only. || `removeTrim` | Deletes a trim so its source span plays and exports again. | `timeline.trimRanges`. || `removeModifier` | Resolves and deletes a zoom, speed, annotation, or camera-fullscreen pill by ID; `removedIds` reports every touching storage row deleted with that pill. | The matching modifier collection. || `removeClip` | Deletes a placed clip, closes the gap, and drops effects anchored only to it. The result names the modifiers and trims it took with it. | Timeline clips and affected anchored modifiers. |diff --git a/technical-documentation/architecture/document-model.md b/technical-documentation/architecture/document-model.mdindex 5b9940e23..b063fa41d 100644--- a/technical-documentation/architecture/document-model.md+++ b/technical-documentation/architecture/document-model.md@@ -27,6 +27,8 @@ and anything unknown is rejected by the `z.literal(axcutSchemaVersion)` check in| `timeline` | `{ clips[], gaps[], trimRanges[], muteRanges[], speedRanges[], captionRanges[] }` | Clips carry their own in/out (`sourceStartSec`/`sourceEndSec`); trims are anchored to a clip (`clipId?`) since v7. See [timeline-model.md](timeline-model.md). || `annotations[]` | `AxcutAnnotationRegion[]` | Text/image/figure/blur overlays, anchored to a clip (`clipId?`). || `zoomRanges[]` | `AxcutZoomRegion[]` | Zoom-in effects, depth 1–6, anchored to a clip (`clipId?`). |+| `shaderEffects[]?` | `AxcutShaderEffect[]` | Agent-written shader effects: WGSL `code`, `name`, `description`, the contract version, declared `params`, and the compositor's cached `cost`. Optional: absent on a document without effects. A definition no region uses is dropped by the edit that removed its last region. See [docs/shader-effects.md](../../docs/shader-effects.md). |+| `shaderEffectRanges[]?` | `AxcutShaderEffectRange[]` | Where each effect applies: clip-anchored regions like the zooms, with the parameter `values` that differ from the defaults, a `target` (`image`, `text`, `both`) and a `lane`. Optional, like `shaderEffects`. || `audioTracks[]` | `AxcutAudioTrack[]` | Imported audio (voiceover / BGM / SFX, issue #350) mixed over the programme. NOT clip-anchored — addressed in RAW/document timeline seconds (`timelineStartSec`), with `trimStartSec`/`trimEndSec` windowing the source and `gainDb` its level. Added from the timeline toolbar; the referenced asset has `kind: "audio"`. || `legacyEditor` | OpenScreen v2 `ProjectEditorState` passthrough | Appearance/cursor settings not yet first-class in the AI-edition schema. || `agent` | `{ baseIntent?, pendingQuestions[], suggestions[], lastAppliedOperations[], lastReasoningSummary? }` | LLM agent state. |diff --git a/technical-documentation/architecture/editor-shell.md b/technical-documentation/architecture/editor-shell.mdindex 53f7fa18f..47f18c4e9 100644--- a/technical-documentation/architecture/editor-shell.md+++ b/technical-documentation/architecture/editor-shell.md@@ -104,10 +104,12 @@ selection pane.## Adding a region kindAn ordered checklist of every file that must move when a new modifier-(`"blur"`, `"captionHighlight"`, …) joins the existing five. The kind is added+(`"blur"`, `"captionHighlight"`, …) joins the existing ones. The kind is addedto `RegionKind` in `src/lib/ai-edition/document/timeline.ts` (`"zoom" |-"trim" | "annotation" | "speed" | "cameraFullscreen"`, `:18`); every site that-switches on it is checked below. Each path has been verified on this branch.+"trim" | "annotation" | "speed" | "cameraFullscreen" | "audio" | "shaderEffect"`);+every site that switches on it is checked below. `"shaderEffect"` went through+the document and agent steps but has no lane or inspector in the editor yet:+agents write effects ([docs/shader-effects.md](../../docs/shader-effects.md)). Each path has been verified on this branch.1. **Schema** — `src/lib/ai-edition/schema/index.ts`. Add a `xxxRegionSchema`Zod object that extends the modifier base (`{id, startMs, endMs,diff --git a/technical-documentation/architecture/mcp-server.md b/technical-documentation/architecture/mcp-server.mdindex 1fd28b93b..528562d92 100644--- a/technical-documentation/architecture/mcp-server.md+++ b/technical-documentation/architecture/mcp-server.md@@ -46,6 +46,10 @@ So a tool added to the agent appears over MCP with no further work, and the MCP- **The project tools**, see [The project and its media](#the-project-and-its-media).- MCP metadata: `readOnlyHint` for the reads (`!isMutatingTool`), `listProjects` and `createCheckpoint`, and `destructiveHint` for `replaceTimeline`, the three `remove*` agent tools, `removeMedia` and `restoreCheckpoint`.+### The effect contract++The server's one resource, `openscreen://effects/contract` ([`effects-contract.ts`](../../electron/mcp/effects-contract.ts)), holds the full WGSL contract of `addEffect`: the inputs an effect reads, its parameters, the rules, the cost tiers and two examples. It is too long for a tool description and versioned with the contract; the instructions point to it. `effects-contract.test.ts` checks that it names every field of the compositor's prelude, and `shader_fx::tests::the_examples_of_the_mcp_contract_compile` compiles its examples. The code an agent writes is checked by `McpToolDeps.checkShaderEffect`, the compositor's `checkShaderEffect`, wired in `registerIpcHandlers` like `cameraSegmentation`; the in-app agent gets the same check through `ChatRunEnv`.+### The project's look`setEditorSettings` ([`editor-settings-tool.ts`](../../electron/ai-edition/editor-settings-tool.ts)) writes every setting of the editor's look panes: background, frame, format, camera layout, cursor, `autoFocusAll` and `audioGainDb`. It takes a partial `EditorSettingsPatch` and applies it with `applySettingsPatch`, the editor's own write, plus what the panes do around a pick (a first fixed format fills the frame, an untouched camera roundness follows its shape, the camera framing is stored as size and pan). `setAspectRatio` is the same write for the format alone.diff --git a/technical-documentation/architecture/timeline-model.md b/technical-documentation/architecture/timeline-model.mdindex f8f293eb0..1bcd37980 100644--- a/technical-documentation/architecture/timeline-model.md+++ b/technical-documentation/architecture/timeline-model.md@@ -33,7 +33,7 @@ existing.## Clip-anchored modifiers-Every region (zoom / speed / annotation / cameraFullscreen) is stored as one or more+Every region (zoom / speed / annotation / cameraFullscreen / shader effect) is stored as one or more**clip-anchored fragments** — `{clipId, sourceStartSec, sourceEndSec, …payload}` — keyedto the clip it lives on, in that clip's source media time. A region the user drewacross a clip boundary is stored as one fragment per covered clip; the fragments carrydiff --git a/workbench/lib/editorial.ts b/workbench/lib/editorial.tsindex 9f4f28150..1523e5baa 100644--- a/workbench/lib/editorial.ts+++ b/workbench/lib/editorial.ts@@ -64,6 +64,7 @@ export function regionFamilies(document: AxcutDocument): RegionFamily[] {{ kind: "annotation", regions: document.annotations as unknown as Region[] },{ kind: "speed", regions: legacyRegions(document, "speedRegions") },{ kind: "cameraFullscreen", regions: legacyRegions(document, "cameraFullscreenRegions") },+ { kind: "shaderEffect", regions: (document.shaderEffectRanges ?? []) as unknown as Region[] },];}diff --git a/workbench/lib/oracles.ts b/workbench/lib/oracles.tsindex e25f3350d..505ab180b 100644--- a/workbench/lib/oracles.ts+++ b/workbench/lib/oracles.ts@@ -201,6 +201,33 @@ export function documentInvariants(document: AxcutDocument): Violation[] {}}+ // 3b. Shader effects: every region names a definition the document holds, and its values+ // stay inside their parameter's bounds (`storedShaderEffectValues` clamps on write).+ const effectsById = new Map((document.shaderEffects ?? []).map((e) => [e.id, e]));+ for (const range of document.shaderEffectRanges ?? []) {+ const effect = effectsById.get(range.effectId);+ if (!effect) {+ violations.push({+ rule: "effects.definition-exists",+ detail: `effect region ${range.id} names missing effect ${range.effectId}`,+ });+ continue;+ }+ for (const param of effect.params) {+ const value = range.values[param.name];+ if (+ param.type === "number" &&+ typeof value === "number" &&+ (value < param.min || value > param.max)+ ) {+ violations.push({+ rule: "effects.value-in-bounds",+ detail: `effect region ${range.id}: ${param.name}=${value} outside ${param.min}-${param.max}`,+ });+ }+ }+ }+// 4. Ids stay unique per family. `replaceTimeline` used to mint `clip_1..N`// and `trim_1..N` by hand (DSL-8) instead of going through `createId`;// it now reuses the id of any clip an interval reproduces exactly anddiff --git a/workbench/scenarios/effect-ripple-on-click.scn.ts b/workbench/scenarios/effect-ripple-on-click.scn.tsnew file mode 100644index 000000000..e1a87ce1c--- /dev/null+++ b/workbench/scenarios/effect-ripple-on-click.scn.ts@@ -0,0 +1,150 @@+// ponytail: the first scenario of agent-written shader effects (docs/shader-effects.md, E.7).+// The prompt names a visual effect no setting offers, so the one right move is to WRITE one:+// addEffect, with WGSL whose wave starts from `ctx.clicks`, over a region that covers the clicks.+// The effect does not need the cursor track: the compositor hands it every click already placed+// in output uv (contract C.2). Reading the track is therefore neither required nor penalised.+//+// Offline there is no compositor: the code is not compiled, the result says `validated: false`,+// and the checks read the code as text. A live run inside the app compiles it for real.++import { cursorTelemetry, singleClip } from "../lib/fixtures";+import { defineScenario, fail, pass } from "../lib/scenario";++/** Three clicks, one per dwell, at its middle: 3 s, 9 s and 17 s. */+const DWELLS = [+ { atSec: 3, holdSec: 1.2, cx: 0.3, cy: 0.4 },+ { atSec: 9, holdSec: 1, cx: 0.72, cy: 0.55 },+ { atSec: 17, holdSec: 0.9, cx: 0.5, cy: 0.8 },+];+const DURATION_SEC = 24.703979;+/** How long after a click its ripple must still be inside a region. */+const RIPPLE_SEC = 0.5;++const RIPPLE = [+ "fn ripple(uv: vec2<f32>, click: vec4<f32>) -> vec2<f32> {",+ " if (click.w < 0.5 || click.z > params.duration) { return vec2<f32>(0.0); }",+ " let t = click.z / params.duration;",+ " let d = (uv - click.xy) * vec2<f32>(ctx.aspect, 1.0);",+ " let r = length(d);",+ " let x = params.radius * (1.0 - pow(1.0 - t, 3.0)) - r;",+ " let env = smoothstep(-0.008, 0.0, x) * exp(-max(x, 0.0) / 0.07);",+ " let wave = sin(x / 0.03 * 6.2831853) * env * (1.0 - t) * (1.0 - t);",+ " let dir = select(vec2<f32>(0.0), d / r, r > 1e-5);",+ " return dir * wave * 0.01 / vec2<f32>(ctx.aspect, 1.0);",+ "}",+ "fn effect(uv: vec2<f32>) -> vec4<f32> {",+ " var offset = vec2<f32>(0.0);",+ " for (var i = 0; i < 4; i++) { offset += ripple(uv, ctx.clicks[i]); }",+ " return frame(uv - offset);",+ "}",+].join("\n");++export default defineScenario({+ id: "effect-ripple-on-click",+ title: "Effet shader — un ripple à chaque clic de souris",+ tags: ["effects", "dsl"],+ prompt: "Add a ripple effect on every mouse click.",+ document: () => singleClip(),+ cursorTelemetry: () => cursorTelemetry({ dwells: DWELLS, durationSec: DURATION_SEC }),+ gate: 0.6,+ reps: 3,++ behaviour: [+ {+ id: "beh.writes-an-effect",+ weight: 3,+ check: (c) =>+ c.calls("addEffect").length > 0+ ? pass()+ : fail("aucun addEffect : aucun réglage ne fait de ripple, il fallait l'écrire"),+ },+ ],++ dsl: [+ {+ id: "dsl.effect.reads-the-clicks",+ weight: 3,+ check: (c) =>+ (c.after.shaderEffects ?? []).some((e) => e.code.includes("ctx.clicks"))+ ? pass()+ : fail("aucun effet ne lit ctx.clicks : l'onde ne peut pas partir du point du clic"),+ },+ {+ id: "dsl.effect.covers-the-clicks",+ weight: 3,+ check: (c) => {+ const ranges = c.after.shaderEffectRanges ?? [];+ const missed = DWELLS.filter(+ (d) =>+ !ranges.some(+ (r) => r.startMs <= d.atSec * 1000 && r.endMs >= (d.atSec + RIPPLE_SEC) * 1000,+ ),+ );+ return missed.length === 0+ ? pass()+ : fail(+ `clics hors de toute région d'effet : ${missed.map((d) => `${d.atSec} s`).join(", ")}`,+ );+ },+ },+ {+ id: "dsl.effect.image-by-default",+ weight: 2,+ check: (c) => {+ const elsewhere = (c.after.shaderEffectRanges ?? []).filter((r) => r.target !== "image");+ return elsewhere.length === 0+ ? pass()+ : fail(`cible ${elsewhere[0].target} alors que rien ne demandait les textes`);+ },+ },+ {+ id: "dsl.turn.completed",+ weight: 2,+ check: (c) =>+ c.run.ok ? pass() : fail(`${c.classifyFailure()} : ${(c.run.error ?? "").slice(0, 200)}`),+ },+ ],++ demoScript: [+ { kind: "tools", calls: [{ name: "getCurrentDocument", args: {} }] },+ {+ kind: "tools",+ calls: [+ {+ name: "addEffect",+ args: {+ startSec: 0,+ endSec: DURATION_SEC,+ effect: {+ name: "Ripple on click",+ description: "A water ripple spreads from every click and bends the picture.",+ code: RIPPLE,+ params: [+ {+ type: "number",+ name: "radius",+ label: "Radius",+ default: 0.2,+ min: 0.05,+ max: 0.5,+ },+ {+ type: "number",+ name: "duration",+ label: "Duration",+ default: 0.9,+ min: 0.3,+ max: 2,+ },+ ],+ },+ },+ },+ ],+ },+ {+ kind: "text",+ text: "I added a ripple that spreads from every click, across the whole video.",+ },+ ],+});diff --git a/workbench/scenarios/registry.ts b/workbench/scenarios/registry.tsindex bacd869d9..629107fcd 100644--- a/workbench/scenarios/registry.ts+++ b/workbench/scenarios/registry.ts@@ -10,6 +10,7 @@ import cursorQuestion, { cursorBlind } from "./cursor-question.scn";import cutSilencesClean from "./cut-silences-clean.scn";import describeProject from "./describe-project.scn";import describeZooms, { describeZoomsMigrated } from "./describe-zooms.scn";+import effectRippleOnClick from "./effect-ripple-on-click.scn";import noInventedBounds from "./no-invented-bounds.scn";import outOfScopeStyling from "./out-of-scope-styling.scn";import realWizardEnhance, {@@ -52,6 +53,8 @@ export const SCENARIOS: Scenario[] = [realCutSilences,realZooms,realZoomGrounding,+ // Effects the agent writes: no setting offers them, so the move is to write one.+ effectRippleOnClick,// Controls: reachable, and expected green.targetRightClip,removeOneModifier,