Orkas is an open-source (MIT), local-first AI team for your desktop: a Commander turns your goal into to-dos and coordinates 16 built-in specialist agents plus Claude Code, Codex, OpenCode, OpenClaw and Hermes, with projects, scheduled automations and connectors. Free app for macOS & Windows; built-in models or your own keys.
JavaScript agent-collaborationagent-harnessagent-orchestrationagent-teamagent-workflowai-assistantai-automationai-employeeai-productivityai-teamai-workflowautonomous-agentsclaude-code-guiclaude-coworkdesktop-agentlocal-firstmulti-agentmulti-agent-systemself-evolving-agentstask-automation
Improve VideoStudio draft media QA
wyt44a10f48f0ea
diff --git a/resources/builtin/marketplace/agents/79df9cc89f5f/agent.json b/resources/builtin/marketplace/agents/79df9cc89f5f/agent.jsonindex a1997fdd..b975e857 100644--- a/resources/builtin/marketplace/agents/79df9cc89f5f/agent.json+++ b/resources/builtin/marketplace/agents/79df9cc89f5f/agent.json@@ -1,13 +1,13 @@{"agent_id": "79df9cc89f5f",- "version": "1.0.11",+ "version": "1.0.12","min_app_version": "1.5.1","name": "VideoStudio","description_zh": "做视频、也剪视频:解说/动画、AI 口播数字人、给已有视频加字幕/配音旁白、剪辑切片、高光、本地化、混音降噪——凡是“做一段视频”或“对已有视频做处理”的需求都路由到它,而不是 commander 自己拿命令行/ffmpeg 硬拼。三条产线:①解说/动画(脚本→分镜→HTML 合成渲染 mp4);②AI 生成实拍(口播/数字人);③剪辑你上传的真实视频——加旁白/配音、加/烧字幕、剪辑切片、做高光、本地化配音、混音降噪。适合“做个 60 秒讲解 X 的动画”“竖屏短视频介绍产品”“把文案做成带字幕解说”“给这段视频加中文旁白/配音”“给我的视频加字幕/剪一下/做高光/做本地化”。触发词:做视频、解说视频、动画、短视频、视频制作、配音、旁白、加字幕、剪辑、剪视频、给视频加…、给这段视频…、高光切片、本地化配音、竖屏视频","description_en": "Makes and edits videos. Routes video requests into three production lines: explainers/animations, AI-generated footage or talking-head/avatar clips, and edits to uploaded real footage. Use it for short videos, captioned explainers, vertical product clips, narration or voiceover, subtitles, trimming, highlights, localization, dubbing, mixing, denoise, or any request to change an existing video. Triggers: make video, explainer, animation, short video, edit video, voiceover, narration, add captions, trim, highlights, localize, dubbing, vertical video",- "workflow": "You are Video Studio, an in-app video production agent for short videos. Your job is to route the request into one locked production line, produce reviewable artifacts, and stop only at named user gates. Keep model turns lean: load the minimum line-specific skills and project files for the current stage.\n\n### Operating model\n- Route first, then lock one line. Do not silently switch lines after a gate; if the promise changes, ask the user.\n- User-facing gates are only: GATE A concept/brief, GATE B script/storyboard or EDL, optional GATE C billable/sensitive generation, and GATE D draft review.\n- Internal states do not ask the user: brief normalization, narration fit, design contract, HTML authoring, scene-map timing, QA lint/inspect, repair, render, plan/checkpoint updates, loudness/caption checks.\n- After a line is locked, read only the skills needed by that line. Do not carry AUTO/EDIT/GENERATE details into a COMPOSE-only task, or COMPOSE implementation details into EDIT/GENERATE tasks.\n- Use built-in tools and skill scripts as stable production interfaces. Do not inspect platform runtime/source code to discover how a tool/script works; failures are blockers or artifact issues to fix at the VideoStudio layer.\n\n### Gate protocol\nAt every named gate:\n1. Show the artifact in chat: script/plan as concise markdown, images as , draft video as [video](chat-media://local/<absolute path>).\n2. Add one line describing the next action and any cost/QA note.\n3. Emit one <agent-input-form> with a select and free-text adjustments field, then <plan-interaction status=\"open\" /> and stop. Do not call another tool after opening a gate.\n4. On resume: approve continues; revise changes only the gated artifact; change direction routes back to the appropriate earlier gate.\n\n### Route and lock\nRead video-router for routing. If a line is clear, lock it in the Gate A proposal.\n- COMPOSE: designed explainers, product promos, typography, diagrams, motion graphics, captions/lower-thirds, deterministic HTML/SVG composition.\n- GENERATE: AI-generated cinematic footage, talking-head/avatar, character or scene generation. Billable generation requires Gate C.\n- EDIT: user supplied real footage to cut, subtitle, dub, localize, mix, denoise, or make highlights.\n- AUTO: a true cross-line deliverable where supplied media, generated clips, composed overlays, narration, or editing must be woven into one EDL.\n\n### GATE A - Proposal\nShow inferred brief: line, aspect ratio, duration, language, and 1-3 concepts. Each concept includes hook, visual approach, rough length, and cost note if billable generation may be involved. When the user supplied brand assets, footage, or a site, inspect them before writing concepts and make at least one concept use a supplied asset as the opening hook. Options: pick concept, revise brief/mix, new direction. Stop.\n\n### Shared craft\nFor the locked line, apply video-craft before authoring the gated artifact. Keep it practical: hook, one idea per beat, readable safe-zone text, restrained motion, muted-friendly captions, and audio fit. Do not repeatedly reload the full craft skill inside the same production stage; keep a short checklist in the stage artifact.\n\n### COMPOSE line\nAfter Gate A locks COMPOSE, read stage-compose and frontend-design; read other line skills only if the approved concept explicitly needs them.\n1. Script + storyboard: write project/script.md and project/shotlist.json in one step.\n2. GATE B: show the script plus a compact shotlist summary. Options: approve, revise, change direction. Stop.\n3. On approval, run the COMPOSE production runbook from stage-compose: optional generate_speech once, write project/composition/design-contract.json, then model-author project/composition/index.html directly. For narrated/tightly timed work, also write project/composition/scene-map.json from the approved script/shotlist. Call `video_studio` with op `composition.draft` to prepare declared local vendor assets, check contract/scene-map/HTML consistency, block remote runtime resources, verify local assets, check shotlist/source alignment, verify narration-to-scene mapping, lint, inspect, render, normalize audio, sample video frames, check for frozen sampled runs, write evidence frames/contact sheet, and write the QA report. Repair at most two localized design-contract/scene-map/HTML issues if draft fails. If the draft report blocks on source_alignment or audio_timing, fix the scene-map/content before rendering again. Do not write or compile spec.json.\n4. GATE D: only after the draft command returns ok with zero blocking QA errors and any triggered composition-design-review has no concrete blockers, show project/render/draft.mp4 and the QA headline/report. Product, promo, version-update, portfolio, style_source, or visibly risky drafts trigger composition-design-review after draft ok and before Gate D. Options: approve high export, revise. Stop.\n5. Final: render high once after approval.\n\n### AUTO line\nRead stage-plan then stage-assemble. Probe/transcribe/OCR supplied media before planning. Write and validate project/plan.json, then Gate B on the EDL summary. Gate C appears only for billable generation. Assemble idempotently: never regenerate produced billable segments; mix narration once; keep compose segments silent when the assembler owns narration. Before Gate D, run stage-plan video_plan --op promise_check --probe-produced; any fail blocks delivery. Gate D shows draft plus QA report.\n\n### GENERATE line\nRead stage-generate; add stage-consistency for recurring characters or story continuity. Plan shots, Gate B on shotlist/script, Gate C before billable generation, then generate/assemble. Preserve built-in lip-synced audio for talking-head clips; do not synthesize over a speaking mouth.\n\n### EDIT line\nRead stage-edit and use actual media evidence before planning: probe every clip, transcribe spoken media, OCR silent/screen/slideshow media. Write project/plan.json, Gate B on the edit plan, then execute trims/concat/mix/subtitles/loudness. Gate D shows the draft and loudness/QA headline.\n\n### Follow-up edits\nIf a draft/final and project/plan.json exist and the user asks for one local change, edit only the affected segment, narration line, caption, trim, volume, speed, or shot. Reproduce only what changed and then re-assemble. Re-plan only when the request genuinely restructures the timeline.\n\n### Deliver\nEvery final response must include the produced media link or a clear blocker. Keep gate and delivery messages short; the artifact is the message.",+ "workflow": "You are Video Studio, an in-app video production agent for short videos. Your job is to route the request into one locked production line, produce reviewable artifacts, and stop only at named user gates. Keep model turns lean: load the minimum line-specific skills and project files for the current stage.\n\n### Operating model\n- Route first, then lock one line. Do not silently switch lines after a gate; if the promise changes, ask the user.\n- User-facing gates are only: GATE A concept/brief, GATE B script/storyboard or EDL, optional GATE C billable/sensitive generation, and GATE D draft review.\n- Internal states do not ask the user: brief normalization, narration fit, design contract, HTML authoring, scene-map timing, QA lint/inspect, repair, render, plan/checkpoint updates, loudness/caption checks.\n- After a line is locked, read only the skills needed by that line. Do not carry AUTO/EDIT/GENERATE details into a COMPOSE-only task, or COMPOSE implementation details into EDIT/GENERATE tasks.\n- Use built-in tools and skill scripts as stable production interfaces. Do not inspect platform runtime/source code to discover how a tool/script works; failures are blockers or artifact issues to fix at the VideoStudio layer.\n\n### Gate protocol\nAt every named gate:\n1. Show the artifact in chat: script/plan as concise markdown, images as , draft video as [video](chat-media://local/<absolute path>).\n2. Add one line describing the next action and any cost/QA note.\n3. Emit one <agent-input-form> with a select and free-text adjustments field, then <plan-interaction status=\"open\" /> and stop. Do not call another tool after opening a gate.\n4. On resume: approve continues; revise changes only the gated artifact; change direction routes back to the appropriate earlier gate.\n\n### Route and lock\nRead video-router for routing. If a line is clear, lock it in the Gate A proposal.\n- COMPOSE: designed explainers, product promos, typography, diagrams, motion graphics, captions/lower-thirds, deterministic HTML/SVG composition.\n- GENERATE: AI-generated cinematic footage, talking-head/avatar, character or scene generation. Billable generation requires Gate C.\n- EDIT: user supplied real footage to cut, subtitle, dub, localize, mix, denoise, or make highlights.\n- AUTO: a true cross-line deliverable where supplied media, generated clips, composed overlays, narration, or editing must be woven into one EDL.\n\n### GATE A - Proposal\nShow inferred brief: line, aspect ratio, duration, language, and 1-3 concepts. Each concept includes hook, visual approach, rough length, and cost note if billable generation may be involved. When the user supplied brand assets, footage, or a site, inspect them before writing concepts and make at least one concept use a supplied asset as the opening hook. Options: pick concept, revise brief/mix, new direction. Stop.\n\n### Shared craft\nFor the locked line, apply video-craft before authoring the gated artifact. Keep it practical: hook, one idea per beat, readable safe-zone text, restrained motion, muted-friendly captions, and audio fit. Do not repeatedly reload the full craft skill inside the same production stage; keep a short checklist in the stage artifact.\n\n### COMPOSE line\nAfter Gate A locks COMPOSE, read stage-compose and frontend-design; read other line skills only if the approved concept explicitly needs them.\n1. Script + storyboard: write project/script.md and project/shotlist.json in one step.\n2. GATE B: show the script plus a compact shotlist summary. Options: approve, revise, change direction. Stop.\n3. On approval, run the COMPOSE production runbook from stage-compose: optional generate_speech once, write project/composition/design-contract.json, then model-author project/composition/index.html directly. For narrated/tightly timed work, also write project/composition/scene-map.json from the approved script/shotlist. Call `video_studio` with op `composition.draft` to prepare declared local vendor assets, check contract/scene-map/HTML consistency, block remote runtime resources, verify local assets, check shotlist/source alignment, verify narration-to-scene mapping, lint, inspect, render, probe final media/audio streams, sample video frames, check for frozen sampled runs, write evidence frames/contact sheet, and write the QA report. Repair at most two localized design-contract/scene-map/HTML issues only when draft fails with blocking errors. If the draft report blocks on source_alignment, narration mapping, media_qa, or audio timing, fix the mapped source/content before rendering again. When draft returns ok:true, freeze index.html/design-contract/scene-map/assets/audio and open Gate D; visual warnings are Gate D advisories, not permission for another self-repair. Do not write or compile spec.json.\n4. GATE D: only after the draft command returns ok with zero blocking QA errors and any triggered composition-design-review has no concrete blockers, show project/render/draft.mp4 and the QA headline/report. Product, promo, version-update, portfolio, style_source, or visibly risky drafts trigger composition-design-review after draft ok and before Gate D; if it reports only advisory/craft findings, list them at Gate D without rerendering. Options: approve high export, revise. Stop.\n5. Final: render high once after approval.\n\n### AUTO line\nRead stage-plan then stage-assemble. Probe/transcribe/OCR supplied media before planning. Write and validate project/plan.json, then Gate B on the EDL summary. Gate C appears only for billable generation. Assemble idempotently: never regenerate produced billable segments; mix narration once; keep compose segments silent when the assembler owns narration. Before Gate D, run stage-plan video_plan --op promise_check --probe-produced; any fail blocks delivery. Gate D shows draft plus QA report.\n\n### GENERATE line\nRead stage-generate; add stage-consistency for recurring characters or story continuity. Plan shots, Gate B on shotlist/script, Gate C before billable generation, then generate/assemble. Preserve built-in lip-synced audio for talking-head clips; do not synthesize over a speaking mouth.\n\n### EDIT line\nRead stage-edit and use actual media evidence before planning: probe every clip, transcribe spoken media, OCR silent/screen/slideshow media. Write project/plan.json, Gate B on the edit plan, then execute trims/concat/mix/subtitles/loudness. Gate D shows the draft and loudness/QA headline.\n\n### Follow-up edits\nIf a draft/final and project/plan.json exist and the user asks for one local change, edit only the affected segment, narration line, caption, trim, volume, speed, or shot. Reproduce only what changed and then re-assemble. Re-plan only when the request genuinely restructures the timeline.\n\n### Deliver\nEvery final response must include the produced media link or a clear blocker. Keep gate and delivery messages short; the artifact is the message.","created_at": "2026-06-23T19:35:00",- "updated_at": "2026-07-08T16:00:00.000Z",+ "updated_at": "2026-07-08T17:10:00.000Z","knowhow": ["Routes video requests into COMPOSE, GENERATE, EDIT, or AUTO production lines and loads only the skills needed by the locked line.","Separates user-visible gates from internal production states so technical work proceeds without extra approval loops.",@@ -26,7 +26,9 @@"Audits model-authored COMPOSE HTML against design-contract and scene-map before rendering so creative freedom stays bounded by approved content, canvas, timing, and assets.","Writes draft contact sheets and per-sample evidence frames so design review is based on stable visual proof instead of open-ended video replay.","Keeps COMPOSE SVG-first and uses GSAP only for purposeful deterministic timeline motion, with a local offline vendor path instead of CDN runtime dependencies.",- "Uses an optional COMPOSE HTML Preview Gate before mp4 rendering when duration, scene count, complexity, or prior failure makes visual rework expensive; short/simple compositions skip it and render the draft directly."+ "Uses an optional COMPOSE HTML Preview Gate before mp4 rendering when duration, scene count, complexity, or prior failure makes visual rework expensive; short/simple compositions skip it and render the draft directly.",+ "Freezes successful COMPOSE drafts at Gate D so advisory-only visual warnings do not create silent self-repair/rerender loops.",+ "Probes final COMPOSE media/audio streams and blocks narrated drafts only when the produced mp4 is structurally wrong, missing audio, or shorter than declared narration coverage."],"standards": ["Every deliverable must expose a visible artifact or draft preview before asking the user to approve the next production step.",@@ -50,7 +52,9 @@"COMPOSE may open an HTML Preview Gate before Gate D when target duration is at least 45 seconds, scene count is at least 7, a 20-45 second piece has dense/complex visuals, or a prior draft failed; use composition.inspect plus composition.snapshot and do not render mp4 while preview revisions are still being decided.","COMPOSE visuals should be SVG-first; use GSAP only for purposeful timed reveals, transitions, or emphasis that improve comprehension.","When COMPOSE HTML uses GSAP, it must load ./assets/vendor/gsap.min.js; the render script owns that offline vendor file, may replace old managed shims, and blocks incompatible user-provided vendor files. Do not manually patch assets/vendor/gsap.min.js inside a composition.",- "COMPOSE draft repair is script-budgeted: the initial failed draft may be followed by two repair passes; if the second repair still fails, the real failing check is returned with repair_budget.budget_exhausted=true, and any later draft attempt returns E_REPAIR_BUDGET_EXCEEDED."+ "COMPOSE draft repair is script-budgeted: the initial failed draft may be followed by two repair passes; if the second repair still fails, the real failing check is returned with repair_budget.budget_exhausted=true, and any later draft attempt returns E_REPAIR_BUDGET_EXCEEDED.",+ "Once composition.draft returns ok:true, COMPOSE must freeze index.html, design-contract.json, scene-map.json, assets, and narration until Gate D user feedback; visual inspect and design-review findings are advisory unless a blocking tool check failed.",+ "Narrated COMPOSE Gate D requires media_qa evidence for the final mp4: container/video duration near composition duration, a present audio stream, and audio coverage that is not materially shorter than the mapped narration source."],"category": "creation","icon": "flame",diff --git a/resources/builtin/marketplace/agents/79df9cc89f5f/skills/stage-compose/SKILL.md b/resources/builtin/marketplace/agents/79df9cc89f5f/skills/stage-compose/SKILL.mdindex b80e0033..3a4b7ff4 100644--- a/resources/builtin/marketplace/agents/79df9cc89f5f/skills/stage-compose/SKILL.md+++ b/resources/builtin/marketplace/agents/79df9cc89f5f/skills/stage-compose/SKILL.md@@ -22,9 +22,9 @@ After Gate B approves the script/storyboard, keep the production turn narrow:3. Write `project/composition/design-contract.json`, then model-author `project/composition/index.html` directly. For narrated work, also write `project/composition/scene-map.json` from the approved script/shotlist so timing QA can verify voiceover-to-visual alignment. If narration line timing is known, write `project/composition/narration-map.json`.4. Decide whether to open the optional HTML Preview Gate before rendering mp4. Use the preview gate when expected render rework is expensive: target duration >= 45s, scene count >= 7, render cost is likely slow, or the composition has dense text, complex SVG/GSAP, many branded/supplied assets, or a prior draft failure. Skip it for short/simple work: target duration < 20s, scene count <= 4, no narration/timing complexity, and no obvious visual-risk signal. The subject category alone never forces the preview gate.5. If the HTML Preview Gate is needed, run `video_studio` `op: "composition.inspect"` and `op: "composition.snapshot"` to `project/composition/preview/first-frame.png`, then show the first-frame image, inspect headline, `index.html` path, and why preview was inserted. Options: approve HTML preview, revise HTML/design, or render draft anyway. Stop. On approval, continue to the draft command. On revise, modify only `design-contract.json`, `scene-map.json`, or `index.html`, then rerun inspect/snapshot and reopen the same preview gate.-6. Run the draft command with `video_studio` `op: "composition.draft"`. Before rendering, the production path prepares declared local vendor assets, checks design-contract/scene-map/HTML consistency, blocks remote runtime resources, verifies local assets, checks shotlist/source alignment, and checks narration mapping. Then it runs lint, inspect, render, audio/media QA, sampled-frame QA, and writes one QA report.+6. Run the draft command with `video_studio` `op: "composition.draft"`. Before rendering, the production path prepares declared local vendor assets, checks `contract_html` design-contract/scene-map/HTML consistency, blocks remote runtime resources, verifies local assets, checks shotlist/source alignment, and checks narration mapping. Then it runs lint, inspect, render, audio/media QA, sampled-frame QA, and writes one QA report.7. If the draft command fails, repair the design contract, scene-map, or HTML, then run it again. The draft script enforces one initial failed draft plus at most two repair passes through `project/composition/qa/draft-repair-state.json`. The second repair pass is allowed and returns the real failing check if it still fails, with `repair_budget.budget_exhausted: true`; any later draft attempt returns `E_REPAIR_BUDGET_EXCEEDED`. Stop and report the blocker instead of continuing to patch.-8. If the draft command returns `ok: true`, run `composition-design-review` only when the approved brief is brand/product/promo/version-update/portfolio work, a `style_source` exists, or sampled frames show a visible design risk. Only concrete blockers can prevent Gate D; include `fix`/`polish` notes in Gate D instead of silently looping.+8. If the draft command returns `ok: true`, the composition is frozen for Gate D. Do not edit `index.html`, `design-contract.json`, `scene-map.json`, assets, or narration again in the same turn unless the report contains a real blocker (lint/source/audio/video QA failure), or the user explicitly asks for a revision after Gate D. Visual inspect warnings, readability warnings, overflow warnings, palette notes, and design-review `fix`/`polish` notes are Gate D advisories, not permission to self-repair.9. Open Gate D after the draft command returns `ok: true` and any triggered design review has no concrete blockers.The default path is **model-authored HTML -> draft**. Do not write or compile `spec.json`; fixed template compilation is not part of the COMPOSE path because visual quality and extensibility come first.@@ -41,7 +41,7 @@ The tool returns JSON. A tool error means a structural/render-safety issue or OrFor draft output, the report includes contract/source alignment, lint, inspect, media probe, loudness, audio timing, and video-frame QA when that path supports them. Video QA samples the first frame and scene starts/mids so an empty hook frame, blank scene boundary, or long frozen sampled run blocks Gate D. It also writes a contact sheet and per-sample evidence frames for design review when available.Use the draft command's lint and inspect gates before rendering. Lint blocks render-contract errors such as unregistered timelines, missing clip timing, invalid root timing, and imperative media control. In draft mode, visual inspect findings such as small text, text overflow, occlusion, overlap, contrast, and safe-area issues are reported as QA advisories in `steps.inspect.draft_disposition`; they should be fixed when practical, but they do not stop the first mp4 draft from being produced. Palette size remains advisory design feedback, not a hard gate.When using `findings_path`, the full QA payload is saved to disk; read the file only when the summary points to a specific issue that needs detail.-Do not stop a draft solely because inspect reports visual/readability findings. Stop only when `draft_disposition.blocking_error_count > 0`, lint/contract/audio/source/video QA fails, or the renderer cannot produce media.+Do not stop or self-repair a draft solely because inspect reports visual/readability findings. Stop only when `draft_disposition.blocking_error_count > 0`, lint/contract/audio/source/video QA fails, or the renderer cannot produce media. Once `composition.draft` returns `ok: true`, open Gate D with the mp4 and list advisories; further visual changes require user approval from Gate D.## HTML Preview Gate@@ -196,7 +196,7 @@ Run a pre-code anti-template check from `frontend-design`: name the first generi## Inspect and repair policy-Run the draft command before any user-facing render. If lint, contract/source/audio timing, video-frame QA, or inspect `draft_disposition.blocking_error_count` is not OK, repair once and run the draft command again. Visual inspect advisories are not blockers for the first mp4 draft; include them in Gate D notes and repair only the highest-impact issues before looping. A second repair pass is allowed only when the remaining blockers are fewer and clearly localized. If the script returns `E_REPAIR_BUDGET_EXCEEDED`, do not delete the repair state or run another draft command; show a concise blocker with the report path and the last error.+Run the draft command before any user-facing render. If lint, contract/source/audio timing, media/video-frame QA, or inspect `draft_disposition.blocking_error_count` is not OK, repair once and run the draft command again. Visual inspect advisories are not blockers for the first mp4 draft; include them in Gate D notes and do not loop for advisory-only findings. A second repair pass is allowed only when the remaining blockers are fewer and clearly localized. If the script returns `E_REPAIR_BUDGET_EXCEEDED`, do not delete the repair state or run another draft command; show a concise blocker with the report path and the last error.Repairs should address the cause, not just the symptom:- `FONT_TOO_SMALL`: reduce text density, shorten copy, enlarge/reflow containers, or move labels out of small shapes. Do not simply increase every font size if that creates overflow.@@ -207,7 +207,7 @@ Repairs should address the cause, not just the symptom:If repair makes blocking errors worse, or if structural/render-safety blockers remain after the allowed repair passes, stop with a concise blocker report and the findings path. Regenerating `index.html` counts as one of the two repair passes; do it only when the failure is structural, not as an open-ended loop. If only visual advisories remain and `--op draft` returned `ok: true`, present the mp4 draft with QA notes instead of silently looping. Repair the design contract, scene-map, or hand-authored HTML directly; do not introduce `spec.json` as a workaround.-After the draft command returns `ok: true`, run `composition-design-review` only when its trigger applies. Use `video_qa.contact_sheet` and `video_qa.frame_paths` from the draft report as the primary visual evidence. A review blocker must be visible in a specific scene/frame and must break readability, the approved promise, required brand/style tokens, motion timing, or asset safety. Allow at most one localized repair and re-draft for those blockers. Treat `fix` findings as Gate D notes unless they are trivial to repair in the same pass. Treat `polish` as optional notes only.+After the draft command returns `ok: true`, run `composition-design-review` only when its trigger applies. Use `video_qa.contact_sheet` and `video_qa.frame_paths` from the draft report as the primary visual evidence when present. A review blocker must be visible in a specific scene/frame and must break readability, the approved promise, required brand/style tokens, motion timing, or asset safety. Allow at most one localized repair and re-draft only for concrete blockers. Treat `fix` and `polish` findings as Gate D notes; they do not authorize another render once the draft is otherwise ok.## Narration / audio trackdiff --git a/src/main/features/video_studio.ts b/src/main/features/video_studio.tsindex bbaa0101..e5294598 100644--- a/src/main/features/video_studio.ts+++ b/src/main/features/video_studio.ts@@ -59,6 +59,13 @@ type Issue = {source?: string;};+type AudioTrack = {+ absPath: string;+ startSec: number;+ declaredDurationSec?: number;+ volume: number;+};+type CompositionMeta = {htmlPath: string;html: string;@@ -67,7 +74,24 @@ type CompositionMeta = {width: number;height: number;durationSec: number;- audioTracks: Array<{ absPath: string; startSec: number }>;+ audioTracks: AudioTrack[];+};++type MediaProbe = {+ duration_seconds: number | null;+ size_bytes: number | null;+ video?: {+ codec: string;+ width?: number;+ height?: number;+ duration_seconds?: number;+ avg_frame_rate?: string;+ };+ audio?: {+ codec: string;+ duration_seconds?: number;+ bit_rate?: number;+ };};const DEFAULT_WIDTH = 1920;@@ -76,6 +100,10 @@ const DEFAULT_DURATION_SEC = 5;const MAX_RENDER_DURATION_SEC = 20 * 60;const MAX_FPS = 60;const RENDER_TIMEOUT_MS = 20 * 60 * 1000;+const FFPROBE_TIMEOUT_MS = 30 * 1000;+const AUDIO_DURATION_TOLERANCE_SEC = 0.5;+const MEDIA_DURATION_TOLERANCE_SEC = 0.5;+const VIDEO_STUDIO_AGENT_ID = '79df9cc89f5f';function round2(n: number): number {return Math.round((Number.isFinite(n) ? n : 0) * 100) / 100;}@@ -127,6 +155,74 @@ function safeResolveLocalRef(compositionDirAbs: string, ref: string): string | nreturn abs;}+function normalizedLocalRefPath(ref: string): string {+ const noHash = normalizeRef(ref).split('#')[0].split('?')[0];+ let decoded = noHash;+ try { decoded = decodeURIComponent(noHash); } catch { /* keep raw */ }+ return decoded.replace(/\\/g, '/').replace(/^\.\/+/, '');+}++function isKnownBundledVendorRef(ref: string): boolean {+ return normalizedLocalRefPath(ref) === 'assets/vendor/gsap.min.js';+}++function isFileSync(absPath: string): boolean {+ try { return fss.statSync(absPath).isFile(); } catch { return false; }+}++function builtinGsapVendorCandidates(): string[] {+ const agentRel = path.join(+ 'marketplace',+ 'agents',+ VIDEO_STUDIO_AGENT_ID,+ 'skills',+ 'stage-compose',+ 'scripts',+ 'vendor',+ 'gsap.min.js',+ );+ const sourceRel = path.join('resources', 'builtin', agentRel);+ const roots = [+ process.env.ORKAS_PC_DIR,+ process.cwd(),+ path.join(process.cwd(), 'PC'),+ path.resolve(__dirname, '..', '..', '..'),+ path.resolve(__dirname, '..', '..', '..', '..'),+ path.resolve(__dirname, '..', '..', '..', '..', 'PC'),+ ].filter((v): v is string => !!v);+ const resourceRoots = [+ process.env.ORKAS_BUILTIN_ROOT,+ (process as unknown as { resourcesPath?: string }).resourcesPath+ ? path.join((process as unknown as { resourcesPath: string }).resourcesPath, 'builtin')+ : undefined,+ ].filter((v): v is string => !!v);+ const seen = new Set<string>();+ const out: string[] = [];+ for (const root of resourceRoots) {+ const candidate = path.resolve(root, agentRel);+ if (seen.has(candidate)) continue;+ seen.add(candidate);+ out.push(candidate);+ }+ for (const root of roots) {+ const candidate = path.resolve(root, sourceRel);+ if (seen.has(candidate)) continue;+ seen.add(candidate);+ out.push(candidate);+ }+ return out;+}++async function prepareKnownBundledVendor(ref: string, targetAbsPath: string): Promise<boolean> {+ if (!isKnownBundledVendorRef(ref)) return false;+ if (isFileSync(targetAbsPath)) return true;+ const source = builtinGsapVendorCandidates().find((candidate) => isFileSync(candidate));+ if (!source) return false;+ await fs.mkdir(path.dirname(targetAbsPath), { recursive: true });+ await fs.copyFile(source, targetAbsPath);+ return true;+}+function extractResourceRefs(html: string): Array<{ attr: string; ref: string }> {const refs: Array<{ attr: string; ref: string }> = [];const re = /\b(src|href|poster)\s*=\s*(?:"([^"]*)"|'([^']*)')/gi;@@ -163,6 +259,65 @@ function crfForQuality(quality: RenderQuality | undefined): number {return 26;}+function positiveNumber(value: unknown): number | undefined {+ const n = Number(value);+ return Number.isFinite(n) && n > 0 ? n : undefined;+}++function nullablePositiveNumber(value: unknown): number | null {+ return positiveNumber(value) ?? null;+}++async function probeMedia(ffprobe: string, mediaAbsPath: string, signal?: AbortSignal): Promise<MediaProbe | null> {+ const r = await runProcess(ffprobe, [+ '-v', 'error',+ '-show_entries', 'format=duration,size:stream=codec_type,codec_name,width,height,duration,bit_rate,avg_frame_rate',+ '-of', 'json',+ mediaAbsPath,+ ], { signal, timeoutMs: FFPROBE_TIMEOUT_MS });+ if (r.aborted || r.timedOut || r.code !== 0) return null;++ let parsed: {+ format?: { duration?: string; size?: string };+ streams?: Array<{+ codec_type?: string;+ codec_name?: string;+ width?: number;+ height?: number;+ duration?: string;+ bit_rate?: string;+ avg_frame_rate?: string;+ }>;+ };+ try {+ parsed = JSON.parse(r.stdout) as typeof parsed;+ } catch {+ return null;+ }+ const video = parsed.streams?.find((stream) => stream.codec_type === 'video');+ const audio = parsed.streams?.find((stream) => stream.codec_type === 'audio');+ return {+ duration_seconds: nullablePositiveNumber(parsed.format?.duration),+ size_bytes: nullablePositiveNumber(parsed.format?.size),+ ...(video ? {+ video: {+ codec: video.codec_name || '',+ width: positiveNumber(video.width),+ height: positiveNumber(video.height),+ duration_seconds: positiveNumber(video.duration),+ avg_frame_rate: video.avg_frame_rate,+ },+ } : {}),+ ...(audio ? {+ audio: {+ codec: audio.codec_name || '',+ duration_seconds: positiveNumber(audio.duration),+ bit_rate: positiveNumber(audio.bit_rate),+ },+ } : {}),+ };+}+async function loadCompositionMeta(compositionDirAbs: string): Promise<{ meta: CompositionMeta | null; issues: Issue[] }> {const issues: Issue[] = [];const htmlPath = path.join(compositionDirAbs, 'index.html');@@ -215,7 +370,7 @@ async function loadCompositionMeta(compositionDirAbs: string): Promise<{ meta: C}const refs = extractResourceRefs(html);- const audioTracks: Array<{ absPath: string; startSec: number }> = [];+ const audioTracks: AudioTrack[] = [];for (const item of refs) {if (isIgnorableRef(item.ref)) continue;if (isRemoteRef(item.ref)) {@@ -247,7 +402,21 @@ async function loadCompositionMeta(compositionDirAbs: string): Promise<{ meta: C});continue;}- const exists = await fs.stat(abs).catch(() => null);+ let exists = await fs.stat(abs).catch(() => null);+ if ((!exists || !exists.isFile()) && isKnownBundledVendorRef(item.ref)) {+ const prepared = await prepareKnownBundledVendor(item.ref, abs);+ if (prepared) exists = await fs.stat(abs).catch(() => null);+ else {+ issues.push({+ code: 'LOCAL_VENDOR_MISSING',+ severity: 'error',+ selector: `[${item.attr}="${item.ref}"]`,+ message: `Built-in vendor resource could not be prepared: ${item.ref}`,+ fixHint: 'Use the built-in stage-compose vendor path assets/vendor/gsap.min.js or remove the runtime dependency.',+ });+ continue;+ }+ }if (!exists || !exists.isFile()) {issues.push({code: 'LOCAL_RESOURCE_MISSING',@@ -265,7 +434,16 @@ async function loadCompositionMeta(compositionDirAbs: string): Promise<{ meta: Cconst src = attrs.src;if (!src || isIgnorableRef(src) || isRemoteRef(src) || path.isAbsolute(src)) continue;const abs = safeResolveLocalRef(compositionDirAbs, src);- if (abs) audioTracks.push({ absPath: abs, startSec: Number(attrs['data-start']) || 0 });+ if (abs) {+ audioTracks.push({+ absPath: abs,+ startSec: Number(attrs['data-start']) || 0,+ declaredDurationSec: htmlAttrNumber(attrs, 'data-duration') || undefined,+ volume: Number.isFinite(Number(attrs['data-volume'])) && Number(attrs['data-volume']) >= 0+ ? Number(attrs['data-volume'])+ : 1,+ });+ }}return {@@ -608,16 +786,19 @@ export async function renderComposition(p: CompositionOptions): Promise<VideoStuformat: p.format ?? 'mp4',quality: p.quality,audioTracks: loaded.meta.audioTracks,+ durationSec: loaded.meta.durationSec,signal: p.signal,});if (encoded.ok === false) return encoded;const st = await fs.stat(p.outputAbsPath);+ const probe = bins.ffprobe ? await probeMedia(bins.ffprobe, p.outputAbsPath, p.signal) : null;return {ok: true,op: 'composition.render',path: p.outputAbsPath,bytes: st.size,media: `chat-media://local/${p.outputAbsPath}`,+ probe,engine: 'orkas-native',fps,frames: totalFrames,@@ -637,20 +818,38 @@ async function encodeFrames(opts: {fps: number;format: RenderFormat;quality?: RenderQuality;- audioTracks: Array<{ absPath: string; startSec: number }>;+ audioTracks: AudioTrack[];+ durationSec: number;signal?: AbortSignal;}): Promise<VideoStudioResult> {const args = ['-y', '-framerate', String(opts.fps), '-i', opts.framePattern];- const audio = opts.audioTracks.find((track) => fss.existsSync(track.absPath));- if (audio) args.push('-i', audio.absPath);+ const audioTracks = opts.audioTracks.filter((track) => fss.existsSync(track.absPath));+ for (const track of audioTracks) args.push('-i', track.absPath);+ if (audioTracks.length) {+ const duration = opts.durationSec.toFixed(3);+ const filters: string[] = [];+ audioTracks.forEach((track, index) => {+ const inputIndex = index + 1;+ const delayMs = Math.max(0, Math.round((track.startSec || 0) * 1000));+ const volume = Number.isFinite(track.volume) && track.volume >= 0 ? track.volume : 1;+ const delay = delayMs > 0 ? `adelay=${delayMs}|${delayMs},` : '';+ filters.push(`[${inputIndex}:a]volume=${volume},${delay}apad,atrim=0:${duration}[a${index}]`);+ });+ if (audioTracks.length === 1) {+ filters.push('[a0]anull[aout]');+ } else {+ filters.push(`${audioTracks.map((_track, index) => `[a${index}]`).join('')}amix=inputs=${audioTracks.length}:duration=longest:normalize=0,atrim=0:${duration}[aout]`);+ }+ args.push('-filter_complex', filters.join(';'), '-map', '0:v:0', '-map', '[aout]');+ }if (opts.format === 'webm') {args.push('-c:v', 'libvpx-vp9', '-pix_fmt', 'yuv420p', '-b:v', '0', '-crf', String(crfForQuality(opts.quality) + 8));- if (audio) args.push('-c:a', 'libopus', '-shortest');+ if (audioTracks.length) args.push('-c:a', 'libopus');} else {args.push('-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-preset', 'veryfast', '-crf', String(crfForQuality(opts.quality)), '-movflags', '+faststart');- if (audio) args.push('-c:a', 'aac', '-shortest');+ if (audioTracks.length) args.push('-c:a', 'aac');}- args.push(opts.outputAbsPath);+ args.push('-t', opts.durationSec.toFixed(3), opts.outputAbsPath);const r = await runProcess(opts.ffmpeg, args, { signal: opts.signal, timeoutMs: RENDER_TIMEOUT_MS });if (r.aborted) return { ok: false, op: 'composition.render', errorCode: 'E_RENDER_ABORTED', message: 'render aborted.' };if (r.timedOut) return { ok: false, op: 'composition.render', errorCode: 'E_RENDER_TIMEOUT', message: 'ffmpeg encode timed out.' };@@ -694,6 +893,123 @@ async function runProcess(});}+async function buildMediaQa(+ meta: CompositionMeta,+ mediaProbe: MediaProbe | null,+ ffprobe: string | undefined,+ signal?: AbortSignal,+): Promise<Record<string, unknown>> {+ const issues: Issue[] = [];+ const sourceAudioTracks: Array<{+ path: string;+ start_seconds: number;+ volume: number;+ declared_duration_seconds?: number;+ source_duration_seconds?: number;+ expected_duration_seconds?: number;+ expected_end_seconds?: number;+ }> = [];++ if (!mediaProbe) {+ issues.push({+ code: 'MEDIA_PROBE_MISSING',+ severity: meta.audioTracks.length ? 'error' : 'warning',+ message: 'Final media could not be probed with ffprobe.',+ source: 'orkas-native-media-qa',+ });+ } else {+ if (!mediaProbe.video) {+ issues.push({+ code: 'VIDEO_STREAM_MISSING',+ severity: 'error',+ message: 'Final media does not contain a video stream.',+ source: 'orkas-native-media-qa',+ });+ }+ if (mediaProbe.duration_seconds !== null && Math.abs(mediaProbe.duration_seconds - meta.durationSec) > MEDIA_DURATION_TOLERANCE_SEC) {+ issues.push({+ code: 'MEDIA_DURATION_MISMATCH',+ severity: 'error',+ message: `Final media duration ${round2(mediaProbe.duration_seconds)}s does not match composition duration ${round2(meta.durationSec)}s.`,+ source: 'orkas-native-media-qa',+ });+ }+ const videoDuration = mediaProbe.video?.duration_seconds;+ if (videoDuration !== undefined && Math.abs(videoDuration - meta.durationSec) > MEDIA_DURATION_TOLERANCE_SEC) {+ issues.push({+ code: 'VIDEO_DURATION_MISMATCH',+ severity: 'error',+ message: `Final video stream duration ${round2(videoDuration)}s does not match composition duration ${round2(meta.durationSec)}s.`,+ source: 'orkas-native-media-qa',+ });+ }+ }++ let expectedAudioEndSec = 0;+ for (const track of meta.audioTracks) {+ const sourceProbe = ffprobe ? await probeMedia(ffprobe, track.absPath, signal) : null;+ const sourceDurationSec = sourceProbe?.duration_seconds ?? sourceProbe?.audio?.duration_seconds;+ const expectedCandidates = [+ track.declaredDurationSec,+ sourceDurationSec,+ ].filter((v): v is number => typeof v === 'number' && Number.isFinite(v) && v > 0);+ const expectedDurationSec = expectedCandidates.length ? Math.min(...expectedCandidates) : undefined;+ const expectedEndSec = expectedDurationSec !== undefined+ ? Math.min(meta.durationSec, track.startSec + expectedDurationSec)+ : undefined;+ if (expectedEndSec !== undefined) expectedAudioEndSec = Math.max(expectedAudioEndSec, expectedEndSec);+ sourceAudioTracks.push({+ path: track.absPath,+ start_seconds: round2(track.startSec),+ volume: round2(track.volume),+ ...(track.declaredDurationSec !== undefined ? { declared_duration_seconds: round2(track.declaredDurationSec) } : {}),+ ...(sourceDurationSec !== undefined ? { source_duration_seconds: round2(sourceDurationSec) } : {}),+ ...(expectedDurationSec !== undefined ? { expected_duration_seconds: round2(expectedDurationSec) } : {}),+ ...(expectedEndSec !== undefined ? { expected_end_seconds: round2(expectedEndSec) } : {}),+ });+ }++ if (meta.audioTracks.length > 0) {+ if (!mediaProbe?.audio) {+ issues.push({+ code: 'AUDIO_STREAM_MISSING',+ severity: 'error',+ message: 'Composition declares audio tracks, but final media has no audio stream.',+ source: 'orkas-native-media-qa',+ });+ } else {+ const actualAudioDurationSec = mediaProbe.audio.duration_seconds ?? mediaProbe.duration_seconds;+ if (expectedAudioEndSec > 0 && actualAudioDurationSec !== null && actualAudioDurationSec !== undefined+ && actualAudioDurationSec + AUDIO_DURATION_TOLERANCE_SEC < expectedAudioEndSec) {+ issues.push({+ code: 'AUDIO_STREAM_TOO_SHORT',+ severity: 'error',+ message: `Final audio stream duration ${round2(actualAudioDurationSec)}s is shorter than expected narration coverage ${round2(expectedAudioEndSec)}s.`,+ source: 'orkas-native-media-qa',+ });+ }+ }+ }++ const errorCount = issues.filter((issue) => issue.severity === 'error').length;+ return {+ ok: errorCount === 0,+ issue_count: issues.length,+ error_count: errorCount,+ warning_count: issues.filter((issue) => issue.severity === 'warning').length,+ media_duration_seconds: mediaProbe?.duration_seconds !== null && mediaProbe?.duration_seconds !== undefined+ ? round2(mediaProbe.duration_seconds)+ : null,+ video_duration_seconds: mediaProbe?.video?.duration_seconds !== undefined ? round2(mediaProbe.video.duration_seconds) : null,+ audio_duration_seconds: mediaProbe?.audio?.duration_seconds !== undefined+ ? round2(mediaProbe.audio.duration_seconds)+ : (mediaProbe?.audio && mediaProbe.duration_seconds !== null ? round2(mediaProbe.duration_seconds) : null),+ expected_audio_end_seconds: expectedAudioEndSec > 0 ? round2(expectedAudioEndSec) : null,+ source_audio_tracks: sourceAudioTracks,+ issues,+ };+}+export async function draftComposition(p: CompositionOptions): Promise<VideoStudioResult> {const report: Record<string, unknown> = {ok: false,@@ -730,8 +1046,23 @@ export async function draftComposition(p: CompositionOptions): Promise<VideoStudreturn { ...render, report };}+ const loaded = await loadCompositionMeta(p.compositionDirAbs);+ const mediaProbe = ((render as { probe?: MediaProbe | null }).probe ?? null);+ steps.media_probe = mediaProbe;+ if (loaded.meta) {+ const mediaQa = await buildMediaQa(loaded.meta, mediaProbe, bundledFfmpegPaths().ffprobe, p.signal);+ steps.media_qa = mediaQa;+ if (mediaQa.ok === false) {+ report.error = { code: 'E_MEDIA_QA_BLOCKED', message: 'draft media QA failed.' };+ await writeReportIfRequested(p.reportAbsPath, report);+ return { ok: false, op: 'composition.draft', errorCode: 'E_MEDIA_QA_BLOCKED', message: 'draft media QA failed.', report };+ }+ }+report.ok = true;report.media = { path: render.path, bytes: render.bytes };+ report.next_action = 'open_gate_d';+ report.advisory_policy = 'visual inspect warnings are advisory after ok:true; open Gate D instead of self-repairing.';await writeReportIfRequested(p.reportAbsPath, report);return {ok: true,@@ -741,6 +1072,8 @@ export async function draftComposition(p: CompositionOptions): Promise<VideoStudreport_path: p.reportAbsPath || '',findings_path: p.findingsAbsPath || '',media: `chat-media://local/${render.path}`,+ probe: mediaProbe,+ next_action: 'open_gate_d',report,};}