Skip to content

armgit / hypit

Public Imported from GitHub
0

Clone any viral video with AI agents. Not just a script, the whole workflow: swap the face, the words, the B-roll, ship 100 variants in one command, and get your 100M views.

docs(skill): explain preview and semantic take composition

kashorinfc726bbbc039
diff --git a/skills/hypit/references/production/authoring.md b/skills/hypit/references/production/authoring.md
index 89e7d002..4a496711 100644
--- a/skills/hypit/references/production/authoring.md
+++ b/skills/hypit/references/production/authoring.md
@@ -91,15 +91,16 @@ Candidate for a Logical Output:
- `<file>` admits project media as a zero-input Candidate;
- `<build-record>` names one public Output from one earlier Build Result;
-- a Run Fragment may provide one or more Candidates, including the generic image, video, and silence
- stand-ins from `@hypit/stand-in`;
+- a Run Fragment may provide one or more Candidates, including a generic Card from
+ `@hypit/stand-in` or a timed composition of that Card;
- `<satisfy>` selects one declared Candidate for one Logical Output.
Candidate selection is an implementation decision for this Run. Core checks nominal Type
compatibility and plans the combined Author and Run graphs. It replaces the displaced upstream
subgraph while preserving any upstream Outputs that the selected Candidate itself consumes.
-Stand-ins carry the dimensions, duration, and clock required by downstream composition.
+The Card carries only its Canvas dimensions. When a timed video is needed, the timed composition
+passes that image to Media Pipeline with the Run's duration and clock.
[Runs and substitutes](runs.md#a-complete-layout-preview) provides a complete example with media
normalization and estimated semantic timing.
diff --git a/skills/hypit/references/production/examples/preview.svrun b/skills/hypit/references/production/examples/preview.svrun
index 36762a6e..66c99ec0 100644
--- a/skills/hypit/references/production/examples/preview.svrun
+++ b/skills/hypit/references/production/examples/preview.svrun
@@ -5,7 +5,7 @@
<import as="estimate" from="@hypit/semantic-take-estimate@1"/>
<target output="main.composition"/>
- <fragment id="card" using="stand-in:video">
+ <fragment id="card" using="stand-in:timed-card">
<input name="canvas" from="canvas"/>
<input name="duration" value="5"/>
<input name="clock" from="clock"/>
@@ -15,7 +15,7 @@
<fragment id="timing" using="estimate:semantic-take">
<input name="narrative" from="story"/>
<input name="segment" from="story.segment.opening"/>
- <input name="media" from="performance-media.media"/>
+ <input name="media" from="performance-preview-media.media"/>
<input name="policy" from="estimated.policy"/>
</fragment>
<satisfy output="opening-semantic.take" candidate="timing.take"/>
diff --git a/skills/hypit/references/production/examples/production.svml b/skills/hypit/references/production/examples/production.svml
index b9025a14..38ef559c 100644
--- a/skills/hypit/references/production/examples/production.svml
+++ b/skills/hypit/references/production/examples/production.svml
@@ -33,10 +33,12 @@
duration="5" aspect-ratio="9:16" generate-audio="true"/>
<pipeline:Normalize id="performance-media" source={performance.video} clock={clock}
video="primary-moving" audio="default" span-authority="video"/>
+ <pipeline:Normalize id="performance-preview-media" source={performance.video} clock={clock}
+ video="primary-moving" audio="none" span-authority="video"/>
<whisperx:SemanticTake id="opening-semantic" narrative={story}
segment={story.segment.opening} media={performance-media.media} language="en"/>
<estimate:SemanticTake id="estimated" narrative={story}
- segment={story.segment.opening} media={performance-media.media}
+ segment={story.segment.opening} media={performance-preview-media.media}
language="en" pace="normal" rounding="round"/>
<speech:Track id="speech" visual-frame={full}
visual-appearance={look.media.performance} visual-z="0">
diff --git a/skills/hypit/references/production/examples/wordless-take.ts b/skills/hypit/references/production/examples/wordless-take.ts
deleted file mode 100644
index 219b2bdd..00000000
--- a/skills/hypit/references/production/examples/wordless-take.ts
+++ /dev/null
@@ -1,23 +0,0 @@
-import type { SynchronizedMedia } from "hypit/media";
-import type { Narrative, NarrativeExcerpt } from "hypit/narrative";
-import { materializeSemanticTake } from "hypit/speech";
-
-// Expose this adapter through a project Producer/Surface with a take output.
-export function wordlessTake(
- narrative: Narrative,
- excerpt: NarrativeExcerpt,
- media: SynchronizedMedia,
-) {
- if (excerpt.kind !== "segment" || excerpt.tokenStart !== excerpt.tokenEndExclusive) {
- throw new Error("wordlessTake requires one Script Segment with no spoken Tokens.");
- }
- const segment = narrative.segments.find((item) => item.id === excerpt.id);
- if (segment === undefined) throw new Error(`Script Segment ${excerpt.id} is missing.`);
- return materializeSemanticTake(narrative, excerpt, media, {
- tokens: [],
- anchors: [
- { identity: segment.startAnchorId, frame: 0 },
- { identity: segment.endAnchorId, frame: media.timeline.frameCount },
- ],
- });
-}
diff --git a/skills/hypit/references/production/media.md b/skills/hypit/references/production/media.md
index 1819c731..3e55c7de 100644
--- a/skills/hypit/references/production/media.md
+++ b/skills/hypit/references/production/media.md
@@ -83,19 +83,21 @@ Use the performed language supported by the selected package and Endpoint.
For a layout study before speech exists, estimated timing can preserve the same Script identities.
[Runs and substitutes](runs.md) explains choosing it and the evidence it supplies.
+Audio-only A-roll follows the same path with audio-only media. Voice Clone produces the Segment's
+performed audio from its Script and the person's Voice Reference. Normalize that output with
+`video="none"`, `audio="default"` and `span-authority="audio"`, then align it to the same Segment.
+The resulting SemanticTake publishes semantic time and sound through Speech Track without inventing
+a visual performance; the work's Media, Typography or MG Tracks supply the picture.
+
## Wordless passages use media boundaries
An ordinary Script Segment such as `<empty></empty>` can carry a wordless passage. Its media gives
it a duration, so the resulting SemanticTake has the Segment's start/end anchors and no timed words.
It enters the same Speech Track assembly as a spoken Take.
-[wordless-take.ts](examples/wordless-take.ts) shows the adapter: it calls `materializeSemanticTake`
-from `hypit/speech` with the authored Segment, normalized media, a start at frame zero and an end at
-the media's frame count. A project Surface can expose it with `narrative`, `segment` and `media`
-inputs and one `take` output. [Track authoring](track-authoring.md) explains that package wiring.
-
-The speech estimator allocates spoken Tokens across a span; this wordless adapter uses the actual
-media boundaries directly. Action, music or visual rhythm can determine the chosen passage length.
+Use `estimate:SemanticTake` with that empty Segment and prepared media. With no spoken Tokens to
+locate, it maps the Segment start to frame zero and the Segment end to the media's frame count.
+Action, music or visual rhythm can determine the chosen passage length.
## Give a still a duration when that is its role
@@ -110,9 +112,12 @@ time-bearing clip, use StillVideo:
video="primary-moving" audio="none" span-authority="video" clock={clock}/>
```
-StillVideo produces a silent video BlobArtifact. Multiple `media:Still` children divide the authored
+StillVideo produces a video-only BlobArtifact. Multiple `media:Still` children divide the authored
duration by their optional weights. Normalization then makes that clip usable as prepared moving
-media. Choose this when the still should supply a Take's time or a video reference.
+media. Choose this when the still should supply a Take's time or a video reference. Add
+`guide="clip-time"` only when a short-lived diagnostic view benefits from a visible local timecode,
+frame count and progress ruler; the guide is one display treatment and says nothing about later Film
+placement.
## Edit bytes at an explicit point in the graph
diff --git a/skills/hypit/references/production/review.md b/skills/hypit/references/production/review.md
index 709fe2cb..8fd1e295 100644
--- a/skills/hypit/references/production/review.md
+++ b/skills/hypit/references/production/review.md
@@ -15,11 +15,12 @@ A separate review Run is useful when its Candidate selection is intentionally di
Studio renders the selected display closure with deterministic Producers and the exact capabilities
that the Runtime Provider declares safe for transient authoring. It submits no Build and no paid
generation. When that closure still needs external generation, use an existing file or Build
-Output, or explicitly select a stand-in when only shape and time are under review.
+Output. An existing image can enter through StillVideo; a Card can fill the remaining case where no
+suitable pixels exist and a component or wiring question still deserves an early look.
-A stand-in can establish layout, duration, cuts, and downstream wiring. Its visible marking reminds
-the reviewer that it cannot establish subject identity, performance, shot composition, texture,
-continuity, or the interaction between the selected production media and graphics.
+A Card can establish that the downstream graph renders, show broad geometry, and carry provisional
+clip time. Its visible marking keeps that evidence distinct from subject identity, performance,
+shot composition, texture, continuity, or the interaction between production media and graphics.
## Separate legality from judgment
@@ -43,8 +44,8 @@ Use whichever view can answer the current question:
- An Agent with browser interaction can open Studio, seek and play the ordinary Run, look at the
current frame, and discover where a layout or motion deserves closer inspection. This is usually
- the cheapest way to work on Caption, MG, Effects, and composition, with stand-ins when external
- media does not yet exist.
+ the cheapest way to work on Caption, MG, Effects, and composition; a Card can supply the minimal
+ visible unknown when no more representative media exists yet.
- Once a numeric interval is known, a range render can inspect the corresponding frames of the same
HyperFrames program. The range may come from the playhead, authored clock time, or an existing
Result; HyperFrames does not need to know whether language helped the Agent locate it.
diff --git a/skills/hypit/references/production/runs.md b/skills/hypit/references/production/runs.md
index b8111e70..fe1f74dc 100644
--- a/skills/hypit/references/production/runs.md
+++ b/skills/hypit/references/production/runs.md
@@ -66,12 +66,12 @@ which kind of value fits that position.
## Use a Fragment when the substitute needs computation
-A Run Fragment connects the computations that produce a Candidate. A generic video card, for
-example, needs the chosen Canvas, clock and duration:
+A Run Fragment connects the computations that produce a Candidate. This composition draws one
+generic Card, then asks Media Pipeline to hold it for the chosen duration with a clip-local guide:
```svrun
<import as="stand-in" from="@hypit/stand-in@1"/>
-<fragment id="card" using="stand-in:video">
+<fragment id="card" using="stand-in:timed-card">
<input name="canvas" from="canvas"/>
<input name="duration" value="5"/>
<input name="clock" from="clock"/>
@@ -88,6 +88,12 @@ The resulting Candidate is named `card.video`. Selecting it replaces the perform
retaining the preparation and composition that consume that video. A substitute can also consume
an existing computed Output; that Output remains part of the selected dependencies.
+When a usable image already exists, hold that image with `media-pipeline:still-video` or
+`media-pipeline:clip-time-still-video` instead. When usable video exists, select that video itself.
+The generic Card is useful only while no suitable pixels exist and the current component or wiring
+question is worth answering before generation completes; its role naturally disappears as real
+production media becomes available.
+
## A complete layout preview
The small [production Source](examples/production.svml), [Recipe](examples/look.svs),
@@ -104,17 +110,18 @@ Fragment to supply the Candidate for the original measured Take:
<fragment id="timing" using="estimate:semantic-take">
<input name="narrative" from="story"/>
<input name="segment" from="story.segment.opening"/>
- <input name="media" from="performance-media.media"/>
+ <input name="media" from="performance-preview-media.media"/>
<input name="policy" from="estimated.policy"/>
</fragment>
<satisfy output="opening-semantic.take" candidate="timing.take"/>
```
-Together with the card selection, this leaves card creation, inspection and normalization as the
-media work. The original generated performance and WhisperX alignment are outside this preview's
-selected graph. The estimator retains the real Segment, Token and Anchor identities and distributes
-their positions across the prepared media's frame count. Those positions show provisional rhythm;
-real alignment supplies the performance's actual word timing.
+Together with the card selection, this leaves Card drawing, StillVideo rendering, video-only
+normalization and timing estimation as the media work. The original generated performance and
+WhisperX alignment are outside this preview's selected graph. The estimator retains the real
+Segment, Token and Anchor identities and distributes their positions across the prepared media's
+frame count. Those positions show provisional rhythm; real alignment supplies the performance's
+actual word timing.
From the directory containing the copied example:
@@ -129,9 +136,10 @@ transient capabilities. If the chosen environment does not provide them for Stud
selected preview media through that environment and reuse its completed Outputs. The normal
production Run still requests generation and measured alignment, under the work's spending authority.
-For an existing production with usable footage, select its Result or file directly. A generic card
-is useful for spacing and timing questions; the actual shot is needed to judge overlap with a face,
-color relationships, performance or identity. [Studio](studio.md) explains what the preview can show.
+For an existing production with usable footage, select its Result or file directly. If only an
+authored or generated picture exists, StillVideo already gives downstream composition much more
+relevant pixels than a Card. The actual shot is needed to judge overlap with a face, color
+relationships, performance or identity. [Studio](studio.md) explains what the preview can show.
## Preserve the choices that still apply
diff --git a/skills/hypit/references/production/studio.md b/skills/hypit/references/production/studio.md
index 2cb6a972..702b5014 100644
--- a/skills/hypit/references/production/studio.md
+++ b/skills/hypit/references/production/studio.md
@@ -47,8 +47,10 @@ processing require the selected environment.
Studio creates no Build and submits no paid generation. It can execute deterministic Producers and
only those exact immediate capabilities a Provider explicitly permits for transient authoring.
If new image/video generation blocks the display closure, select an existing Output or file; use
-an explicit stand-in Candidate when the current question only concerns structure, layout or time.
-A stand-in does not establish performance, identity or the composition of future generated media.
+an existing image through StillVideo when one is available. An explicit Card Candidate can keep a
+component or wiring study moving when no suitable pixels exist yet. It supplies a visible canvas
+and provisional clip time; the eventual media establishes performance, identity, composition and
+the real relationship between subject and graphics.
[Runs and substitutes](runs.md#a-complete-layout-preview) includes a complete Source and Run with
a card Candidate, normalization and estimated semantic timing for this kind of layout study.
diff --git a/skills/hypit/references/production/track-authoring.md b/skills/hypit/references/production/track-authoring.md
index 2f5f91e7..108a9ed9 100644
--- a/skills/hypit/references/production/track-authoring.md
+++ b/skills/hypit/references/production/track-authoring.md
@@ -76,6 +76,26 @@ The interview's project emoji strip illustrates this design: one outer Window, o
items, a placeholder asset and one icon per answer. Each Moment changes its own slot. The Track does
not decide when the spoken answer occurred; it consumes the already projected event.
+## Author an alternative performance assembly
+
+The installed Speech Track expresses one particular and useful assembly: every Segment-local
+SemanticTake keeps its complete span, and Source order forms one continuous prefix-sum
+`SemanticTrack`. Its audio and picture projections follow that same assembly.
+
+A work may intentionally need interruptions, overlapping dialogue, a time-consuming transition or
+another performance-time relationship. That is a legitimate project Track when it makes the edited
+media, semantic positions, visual projection and audio ownership explicit. If its output still is the
+standard `SemanticTrack`, it must satisfy that type's continuous item semantics so existing consumers
+can trust it. A genuinely different timeline meaning belongs to a project-owned type with matching
+projection Surfaces or an explicit adapter to the standard type. Source then selects that component
+normally; neither Core nor the default Speech Track needs a special case.
+
+Design the author Surface from the intended relationship rather than exposing low-level overlap
+arithmetic by default. The implementation can accept prepared performances, edit or combine them at
+its declared boundary, and publish only the outputs it can define unambiguously. Caption, MG, Media
+and Effects receive its declared semantic projections rather than inferring overlap from picture;
+sound routing makes deliberate simultaneous speech visible rather than accidental.
+
## Make spatial decisions equally explicit
CanvasSpace identifies the coordinate system. SpatialFrame, anchors, extents and fitting describe
diff --git a/skills/hypit/references/production/tracks.md b/skills/hypit/references/production/tracks.md
index 5ed28749..41d1d2ba 100644
--- a/skills/hypit/references/production/tracks.md
+++ b/skills/hypit/references/production/tracks.md
@@ -65,17 +65,38 @@ forms that component exposes.
## Semantic Takes supply the media timeline
-For the usual spoken production, start with the performance carrying the main Script. That is the
-A-roll and the source of semantic time, even while another picture covers it. In a podcast, the
-participants' turns belong to that shared performance timeline; each visible speaker does not
-create a separate program clock. Independently generated narration can also supply the spine when
-that is the intended form. Music, sound effects and covering footage then follow the assembled work.
+Script Segments have authored order and meaning but no program positions. Each spoken Segment is
+associated with the local media performance carrying it; normalization establishes that media's
+frame envelope, and alignment locates the Segment's words and anchors inside it. That self-contained
+value is a SemanticTake. Speech Track places the Takes in Source order and translates every local
+position by the lengths before it. The resulting SemanticTrack is the program's semantic time.
+
+```text
+Segment A + its local media → SemanticTake A ┐
+Segment B + its local media → SemanticTake B ├─ Source order → SemanticTrack
+Segment C + its local media → SemanticTake C ┘
+```
+
+Speech Track is therefore not placing A-roll on an independently maintained master timeline. The
+ordered Takes create the timeline that Caption, MG, Media, Typography, Audio and Effects project
+through. The official Speech Track uses each complete local media span and joins the next Take at its
+end; it creates no gap or duration-consuming overlap. An edit that trims or retimes a performance
+belongs before its alignment. A visual handoff that does not change the performance media remains an
+ordinary visual composition on the resulting program time.
+
+For the usual spoken production, the performance carrying each Segment is the A-roll. In a podcast,
+the participants' turns belong to that shared performance timeline; each visible speaker does not
+create a separate program clock. Audio-only A-roll can also carry one Segment when that passage is
+independent-narration-led. Music, sound effects and covering footage then follow the
+assembled work. [Voice and performance](../playbooks/craft/voice-and-performance.md) owns the choice
+between visible A-roll, covered visible A-roll and audio-only A-roll.
A-roll can sit above those pictures: a circular presenter inset over a screen recording, or a moving
cutout in the corner over a product demonstration, still carries the main Script. The SemanticTake
-owns the performance and timing; the visual Track owns size, shape, placement and stacking. Media
-Track can present that same performance without changing its role to B-roll. Include the performance
-audio once and select the intended picture presentation in Film. [Compositing](../playbooks/craft/compositing.md)
+owns the performance and timing. Speech Track publishes its semantic, visual and audio contributions
+together; its visual Frame, fit, clipping, appearance, lifecycle motion and stack order present an
+inset or cutout directly. Include the wanted peer outputs in Film. This is one assembly of the same
+Take, not a Speech Track followed by a Media Track that displays it again. [Compositing](../playbooks/craft/compositing.md)
explains insets and background removal.
Declare a Clock for normalization, normalize generated or supplied media with explicit stream
@@ -93,12 +114,23 @@ those Takes in source order; global frame positions follow their actual normaliz
This is an assembly excerpt: the imports, layout, fit Recipe and SemanticTakes must already exist.
One Take may contain several speakers or generated cuts. Estimates help request a duration; they do
not supply word timestamps. Covering or reframing its picture leaves the same performance and audio.
-Speech Track's appearance is a fit Recipe; use Media Track for a separate animated presentation.
-
-Semantic structure also supports a wordless passage. A named empty Script Segment receives its
-actual boundaries from prepared media and remains addressable through the assembled SemanticTrack.
+Speech Track shares the ordinary visual appearance of a Media Item: fit and crop,
+opacity and filters, clipping, padding, border, shadow, frame paint, lifecycle motion and stack
+order. Sampling children can pan, zoom or rotate the Take's picture inside that Frame over normalized
+Segment progress. Media Track remains the owner of an independent Window, source playback and trim,
+source audio, replacement Sequences and pictures that are not the speaking performance.
+
+Semantic structure also supports a wordless passage. It has no speaking A-roll, yet a named empty
+Script Segment receives its actual boundaries from prepared media and remains addressable through
+the assembled SemanticTrack. Semantic identity here means that the work can refer to the passage and
+its boundaries; it does not classify the passage or require spoken words.
[Media preparation](media.md#wordless-passages-use-media-boundaries) owns that construction.
+Every Take contributes its Segment boundaries to the assembled semantic time. When its prepared
+media has no audio stream, Speech Track emits no audio clip for that Take; when it has no visual
+stream, it emits no picture. The available projections continue without manufacturing a missing
+stream.
+
## Coverage has a Window and a separate playback choice
```svml