Skip to content

World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.

Remove internal capability audit document

calesthio8d1920cf249b
diff --git a/docs/CAPABILITY_AUDIT.md b/docs/CAPABILITY_AUDIT.md
deleted file mode 100644
index 4a3d5374..00000000
--- a/docs/CAPABILITY_AUDIT.md
+++ /dev/null
@@ -1,402 +0,0 @@
-# OpenMontage Capability Audit
-
-> 2026-03-28 | Code-verified. Every finding traced to specific file + line.
-
-**Core question:** When an agent discovers its capabilities via the registry and reads
-the skills/manifests, does it get an accurate picture of what it can and can't do?
-
-**Verdict:** No. The agent receives a mostly-accurate tool inventory but is actively
-misled at the skill-to-tool boundary. Skills promise inputs tools don't accept, schemas
-reject structures skills tell the agent to produce, and the discovery mechanism omits
-the one thing the agent needs most: what inputs each tool takes.
-
----
-
-## 1. Discovery: What the Agent Learns at Boot
-
-The agent runs:
-```python
-from tools.tool_registry import registry
-registry.discover()
-envelope = registry.support_envelope()
-```
-
-**What it gets per tool** (via `BaseTool.get_info()`):
-- Identity: name, version, tier, capability, provider, stability
-- Status: available/unavailable/degraded, dependencies list
-- Runtime: execution_mode, determinism, runtime type, resource_profile
-- Decision hints: best_for, not_good_for, provider_matrix, fallback_tools
-- Skill links: agent_skills, related_skills
-- Side effects, resume_support, user_visible_verification
-
-**What it does NOT get:**
-| Missing Field | Why It Matters |
-|---|---|
-| `input_schema` | Agent can't know what parameters a tool accepts without reading source code |
-| `output_schema` | Agent can't know what a tool returns |
-| `artifact_schema` | Agent can't know what files a tool produces |
-| `estimate_cost()` results | Agent can't compare costs without calling each tool individually |
-
-**This is the root cause of most downstream issues.** The agent knows *what tools exist* but not *what they accept or produce*. It must rely on skills for that information — and skills get it wrong in several places.
-
-### Fix
-
-Add to `BaseTool.get_info()`:
-```python
-"input_schema": self.input_schema,
-"output_schema": self.output_schema,
-"artifact_schema": self.artifact_schema,
-```
-
-This single change would let the agent validate skill instructions against actual tool contracts at runtime.
-
----
-
-## 2. Skills Tell the Agent to Pass Inputs Tools Don't Accept
-
-These are cases where a stage-director skill instructs the agent to call a tool with
-specific parameters, but the tool's `input_schema` and `execute()` don't read them.
-The agent follows the skill, the extra fields are silently ignored, and the output
-doesn't reflect the agent's intent.
-
-### 2a. tts_selector: voice style, speaker directions, pronunciation
-
-**Skill says** (`skills/pipelines/explainer/asset-director.md` lines 54-62):
-> Apply speaker directions from the script (pace, emphasis, emotion).
-> Apply the playbook's `audio.voice_style`.
-> Include a pronunciation map in the TTS request for technical terms.
-
-**Tool accepts** (`tools/audio/tts_selector.py` input_schema):
-`text`, `preferred_provider`, `allowed_providers`, `output_path`
-
-No fields for voice_style, speaker_directions, pace, emphasis, emotion, or pronunciation.
-None of the concrete TTS tools (elevenlabs, openai, piper) accept these either — except
-elevenlabs_tts which accepts `stability`, `similarity_boost`, `style` (numeric 0-1 floats),
-not the semantic concepts the skill describes.
-
-**Agent impact:** Agent is told to express creative direction through tool parameters that
-don't exist. The narration will be generated with default voice settings regardless of what
-the script specifies.
-
-### 2b. image_selector: negative_prompt, consistency_anchors
-
-**Skill says** (`skills/pipelines/explainer/asset-director.md` lines 68-73):
-> Build the prompt: playbook.image_prompt_prefix + scene description + style cues.
-> Add negative prompt from playbook.
-> Include consistency anchors (same palette, same style across all images).
-
-**Tool accepts** (`tools/graphics/image_selector.py` input_schema):
-`prompt`, `preferred_provider`, `allowed_providers`, `output_path`
-
-No `negative_prompt` or `consistency_anchors` field. The selector passes `prompt` through
-to whichever provider it selects. Some downstream providers (flux_image) DO accept
-`negative_prompt`, but the selector doesn't forward it.
-
-**Agent impact:** Agent must flatten everything into a single `prompt` string. Negative
-prompts and consistency anchors are lost unless the agent bypasses the selector and calls
-flux_image directly — which contradicts the skill's guidance.
-
-### 2c. video_compose "render" operation: options, output_profile
-
-**Skill says** (`skills/pipelines/explainer/compose-director.md` lines 66-80):
-```json
-{
- "operation": "render",
- "edit_decisions": {...},
- "asset_manifest": {...},
- "output_profile": "youtube_landscape",
- "options": {
- "subtitle_burn": true,
- "audio_normalize": true,
- "two_pass_encode": true
- }
-}
-```
-
-**Tool accepts** (`tools/video/video_compose.py` input_schema):
-`operation`, `input_path`, `output_path`, `edit_decisions`, `subtitle_path`,
-`subtitle_style`, `overlays`, `audio_path`, `profile`, `codec`, `crf`, `preset`
-
-- `asset_manifest`: NOT in input_schema but IS read by `_render()` (line 271). Works but undocumented.
-- `output_profile`: Not accepted. The field is `profile` and only used by `_encode()`, not `_render()`.
-- `options`: Not accepted at all. `subtitle_burn`, `audio_normalize`, `two_pass_encode` are silently dropped.
-
-**Agent impact:** Agent follows the skill, passes these fields, they vanish. Output won't
-match target platform. No subtitle burn, no normalization, no two-pass encode.
-
-### 2d. audio_mixer: skill implies single call, tool requires multiple
-
-**Skill says** (`skills/pipelines/explainer/compose-director.md` lines 87-95):
-> Call the audio_mixer tool to:
-> 1. Layer narration segments in order
-> 2. Mix background music at playbook volume
-> 3. Apply ducking (music dips during narration)
-> 4. Normalize overall audio levels
-
-**Tool accepts** (`tools/audio/audio_mixer.py`):
-Three discrete operations: `mix`, `duck`, `extract`. Each is a separate `execute()` call.
-There is no single-call "mix everything together" mode.
-
-**Agent impact:** Agent attempts a single call with all requirements. The tool handles one
-operation per call. Agent must orchestrate: mix narration tracks first, then duck with
-music, then the compose step muxes audio into video. The skill doesn't explain this
-multi-step choreography.
-
----
-
-## 3. Skills Tell the Agent to Produce Artifacts Schemas Reject
-
-These are cases where a skill shows the agent an example JSON structure for an artifact,
-but the corresponding JSON Schema has `additionalProperties: false` and rejects the
-skill's fields.
-
-### 3a. edit_decisions: transforms and transitions
-
-**Skill says** (`skills/pipelines/explainer/edit-director.md` lines 38-53):
-```json
-{
- "id": "cut-1",
- "source": "img-scene-1",
- "in_seconds": 0,
- "out_seconds": 10,
- "layer": "primary",
- "transform": {
- "scale": 1.0,
- "position": "center",
- "animation": "ken-burns-slow-zoom"
- },
- "transition_in": "fade",
- "transition_out": "dissolve",
- "transition_duration": 0.4
-}
-```
-
-**Schema allows** (`schemas/artifacts/edit_decisions.schema.json` lines 12-24):
-```json
-{
- "required": ["id", "source", "in_seconds", "out_seconds"],
- "properties": {
- "id": {}, "source": {}, "in_seconds": {}, "out_seconds": {},
- "speed": {}, "reason": {}
- },
- "additionalProperties": false
-}
-```
-
-**Rejected fields:** `layer`, `transform` (and all sub-fields), `transition_in`,
-`transition_out`, `transition_duration`.
-
-**Agent impact:** Agent builds the edit_decisions artifact per skill instructions.
-Checkpoint validation rejects it. Agent is stuck between following the skill and
-passing schema validation. The schema also means `video_compose._compose()` will
-never receive transition/transform data even if it could handle it — which it can't
-(it only does hard cuts via concat).
-
-### 3b. edit_decisions: subtitle styling
-
-**Skill says** (edit-director.md lines 62-78):
-```json
-"subtitles": {
- "enabled": true,
- "style": "word-by-word",
- "font": "Inter",
- "font_size": 48,
- "color": "#FFFFFF",
- "background": "#00000088",
- "position": "bottom-center",
- "max_words_per_line": 8
-}
-```
-
-**Schema allows** (edit_decisions.schema.json lines 51-58):
-```json
-"subtitles": {
- "properties": {
- "enabled": { "type": "boolean" },
- "style": { "type": "string" },
- "source": { "type": "string" }
- }
-}
-```
-
-**Rejected fields:** `font`, `font_size`, `color`, `background`, `position`,
-`max_words_per_line`.
-
-**Agent impact:** All subtitle styling information is lost at schema validation.
-The compose stage gets `enabled: true` and `style: "word-by-word"` but no font,
-color, or positioning data. Subtitle styling must come from video_compose's
-`subtitle_style` input instead — but the edit-director skill doesn't mention this path.
-
-### 3c. edit_decisions: narration and SFX audio configuration
-
-**Skill says** (edit-director.md lines 84-110):
-```json
-"audio": {
- "narration": {
- "segments": [
- { "asset_id": "narration-s1", "start_seconds": 0 }
- ]
- },
- "music": {
- "asset_id": "...",
- "ducking": {
- "enabled": true,
- "threshold_db": -3,
- "reduction_db": -8,
- "attack_ms": 200,
- "release_ms": 500
- }
- },
- "sfx": []
-}
-```
-
-**Schema allows** (edit_decisions.schema.json lines 59-67):
-`music` with `asset_id`, `volume`, `ducking` (boolean only), `fade_in_seconds`, `fade_out_seconds`.
-
-**Rejected:** Entire `audio.narration` section, entire `audio.sfx` section, structured
-`ducking` object (schema only accepts boolean). Narration segment timing and SFX
-have no home in the validated artifact.
-
----
-
-## 4. Tools Lie About What They Can Do
-
-### 4a. video_compose._render() claims image-to-video conversion
-
-**Docstring** (`tools/video/video_compose.py` line 267):
-> "It orchestrates: image-to-video conversion, concatenation, audio mixing,
-> subtitle burn-in, and final encoding to target profile."
-
-**Actual code** (lines 280-302): Resolves asset IDs from asset_manifest to file paths,
-then calls `_compose()`. No image-to-video conversion. `_compose()` uses `ffmpeg -ss/-to`
-on each source, which fails on still images (PNG/JPG have no temporal dimension to seek into).
-
-**Agent impact:** Agent trusts that the "render" operation handles images. It doesn't.
-Explainer pipelines where most cuts are generated images will produce broken output.
-
-### 4b. subtitle_gen declares highlight_style but ignores it
-
-**Input schema** (`tools/subtitle/subtitle_gen.py`): includes `highlight_style` parameter.
-
-**Execute method:** Never reads `highlight_style`. It's a phantom parameter.
-
-**Agent impact:** Low — agent might pass highlight_style expecting highlighted captions
-and get plain ones.
-
-### 4c. video_selector declares operation parameter but ignores it
-
-**Input schema** (`tools/video/video_selector.py`): includes `operation` field.
-
-**Execute method:** Reads it but never passes it to the selected provider tool.
-
-**Agent impact:** Low — operation context is lost during delegation but most video
-providers only do one thing anyway.
-
----
-
-## 5. Phantom Tools in Pipeline Manifests
-
-`pipeline_defs/framework-smoke.yaml` references three tools that don't exist:
-
-| Tool Name | Stage | Field |
-|---|---|---|
-| `idea_explorer_llm` | research | preferred_tools |
-| `script_writer_llm` | script | preferred_tools |
-| `script_writer_template` | script | fallback_tools |
-
-`registry.discover()` will not find these. If the agent tries to resolve them, it gets
-nothing. These appear to be placeholders from an earlier design where LLM capabilities
-were modeled as tools.
-
-**Agent impact:** framework-smoke pipeline is broken. Other 10 pipelines are clean —
-all their tool references resolve to real tools.
-
----
-
-## 6. Unused Tools the Agent Doesn't Know to Use
-
-Five tools exist in `tools/` but no pipeline manifest references them:
-
-| Tool | What It Does | Why It's Missing |
-|---|---|---|
-| `bg_remove` | Background removal (rembg/U2Net) | Useful for compositing, overlays. No skill mentions it. |
-| `face_restore` | Face restoration (CodeFormer/GFPGAN) | Useful for low-res faces in talking-head. No skill mentions it. |
-| `upscale` | Image/video upscaling (Real-ESRGAN) | Useful for enhancement pass. No skill mentions it. |
-| `video_understand` | Vision-language analysis (CLIP/BLIP-2) | Could be used for quality review. No skill mentions it. |
-| `image_gen` | Legacy multi-provider image gen | Deprecated, replaced by image_selector. Expected. |
-
-The agent discovers these via `support_envelope()` but has no skill guidance on when or
-how to use them. They're invisible capabilities — the agent CAN use them but WON'T
-unless it independently decides to.
-
----
-
-## 7. The Playbook Gap: Declared Knowledge the Agent Can't Apply
-
-Style playbooks (`styles/*.yaml`) define detailed visual contracts:
-`image_prompt_prefix`, `negative_prompt`, `consistency_anchors`, `typography`,
-`motion.transition_duration_seconds`, `audio.music_mood`, `quality_rules`.
-
-Skills extensively reference these: "apply the playbook's voice_style", "use playbook's
-image_prompt_prefix", "set music volume per playbook". **30+ skill references across
-all pipelines.**
-
-But:
-- Zero tools import `playbook_loader` or read playbook data
-- The selector tools (tts_selector, image_selector) have no playbook-aware parameters
-- The agent must: load the YAML, extract values, manually inject them into tool calls
-
-This works IF the agent knows to do it. But the tools give no signal that they expect
-playbook-derived values. The playbook system is a skill-layer convention that the tool
-layer is unaware of. This is architecturally fine for an agent-first system — but the
-skills need to be explicit about HOW the agent should bridge this (e.g., "read
-`playbook.asset_generation.image_prompt_prefix` and prepend it to the `prompt` field
-when calling image_selector"). Currently skills say WHAT to apply but not HOW to
-translate playbook values into tool parameters.
-
----
-
-## 8. Missing Skill: Talking-Head Executive Producer
-
-`pipeline_defs/talking-head.yaml` declares `orchestration.skill: pipelines/talking-head/executive-producer` but the file `skills/pipelines/talking-head/executive-producer.md` does not exist. All other 9 pipelines with orchestration have their EP skill.
-
-**Agent impact:** Agent following the talking-head pipeline reads the manifest, tries to
-load the EP skill, gets a file-not-found. Must improvise orchestration without guidance.
-
----
-
-## Summary: What the Agent Gets Right vs. Wrong
-
-### Accurate (agent can trust these)
-- Tool inventory: 44/47 tool references in manifests resolve correctly (3 phantoms in framework-smoke only)
-- Tool status: `get_status()` accurately reports availability based on real dependency checks
-- Fallback chains: All `fallback_tools` references point to tools that exist
-- Tool self-description: 10/12 tools audited have accurate input_schema vs execute() behavior
-- Pipeline stage ordering and artifact flow: checkpoint system matches manifests
-- Cost estimates: Tools that implement `estimate_cost()` return realistic numbers
-
-### Inaccurate (agent will be misled by these)
-| Issue | Count | Root Cause |
-|---|---|---|
-| Skills pass inputs tools don't accept | 4 cases | Skills written to aspirational API, not actual tool schema |
-| Skills produce artifacts schemas reject | 3 cases | Schemas stricter than skills expect (additionalProperties: false) |
-| Tool overclaims capability | 1 case | video_compose._render() docstring lies about image handling |
-| Discovery omits input/output schemas | All tools | get_info() doesn't include schemas |
-| Phantom tools in manifests | 3 tools | framework-smoke has placeholder references |
-| Missing EP skill | 1 pipeline | talking-head pipeline incomplete |
-
-### Recommended Fix Priority
-
-| # | Fix | Effort | Effect |
-|---|---|---|---|
-| 1 | Add input_schema/output_schema to `get_info()` | 3 lines | Agent can self-validate against skills |
-| 2 | Align edit_decisions.schema.json with edit-director skills | Medium | Unblocks transitions, subtitle styling, narration config |
-| 3 | Add image-to-video loop in video_compose._compose() | Small | Unblocks all image-based pipelines |
-| 4 | Add negative_prompt passthrough to image_selector | Small | Enables playbook integration for image gen |
-| 5 | Add voice_id/style passthrough to tts_selector | Small | Enables creative direction for narration |
-| 6 | Fix compose-director to match video_compose's actual interface | Small | Agent stops passing phantom options |
-| 7 | Fix compose-director to document multi-call audio_mixer flow | Small | Agent knows to call mix then duck |
-| 8 | Remove phantom tools from framework-smoke.yaml | Trivial | Clean manifest |
-| 9 | Create talking-head executive-producer.md | Medium | Completes pipeline |
-| 10 | Document playbook-to-tool bridging pattern in skills | Medium | Agent knows HOW to apply playbook values |