Skip to content

Explainer videos and product demos made by your AI agent. Free and open source: a local voice (Kokoro), word timing (Whisper) and a canvas renderer turn a script into a narrated MP4.

Keep times in real seconds at any pace (cue plus offset now lands right), captions at the bottom of feed sizes with room in s.safe and a caption-band check for every size, hints for cue words spoken twice, content areas in the formats command, and doc notes from the first agent-made videos

Vincent Schmalbach4c190405645b
diff --git a/AGENTS.md b/AGENTS.md
index 0f3b354..a3af2c9 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -265,16 +265,35 @@ use it only when the video is not meant for those apps.
Vertical videos have little room. Stack things instead of putting them side
by side, use fewer words on screen, and place everything from `s.safe`.
+`explainroo formats` prints the content area of each size in pixels, for
+example 745 x 746 on TikTok.
+
+### Where the captions sit
+
+Captions are on by default for vertical, 4:5 and square videos. On Shorts,
+TikTok and Reels they sit just above the app's bottom band. On 4:5, square and
+wide videos they sit at the bottom, above the watermark. On a plain `9:16`
+video they sit about a quarter of the way up. In every case `s.safe` ends
+above them, so anything placed inside `s.safe` stays clear, and `check` warns
+about text in the caption band. Do not put the same sentence on screen as
+text while the captions show it.
+
+The first frame is often the cover picture on social apps. When the first
+elements draw themselves in, frame one is empty; give the title
+`enter: 'none'` or `at: 0` with a quick look, or pick the cover in the app.
## Pace
`pace` makes the whole video quicker or slower. At 1.3 the voice speaks 30%
faster, and the pauses, the time before and after each scene, the
transitions and every animation get 30% shorter. Word and marker cues follow
-the voice, and times in seconds in `scenes.js` are shortened too, so a scene
-keeps its rhythm. 1 is the normal pace; 1.15 to 1.3 suits social media and
-viewers who know the topic, above 1.4 gets hard to follow. `speed` changes
-only the voice.
+the voice, so tie pictures to words and the rhythm stays right at any pace.
+Times in seconds are real seconds of the scene, like `s.t` and `s.cue()`, so
+`at: s.cue('word') + 0.3` means 0.3 seconds after the word at any pace. A
+fixed time like `at: 4` does not move with the pace, which is one more reason
+to use cues. 1 is the normal pace; 1.15 to 1.3 suits social media and viewers
+who know the topic, above 1.4 gets hard to follow. `speed` changes only the
+voice.
## The watermark
@@ -379,7 +398,7 @@ Coordinates are pixels from the top left. `x` and
| `s.cue(word, n = 1)` | when the n-th time a word or phrase is spoken starts |
| `s.cueEnd(word, n = 1)` | when it ends |
| `s.mark(name)` | the time of a `[#name]` marker |
-| `s.time(v)` | turns a number, a word or `"#marker"` into seconds (numbers are divided by the pace) |
+| `s.time(v)` | turns a number, a word or `"#marker"` into seconds |
| `s.pace` | the video's pace (1 is normal) |
| `s.p(at, dur = 0.6, ease = 'inOut')` | 0 to 1 progress for your own animation |
| `s.since(at)`, `s.between(a, b)` | seconds since a time, or whether now is between two times |
@@ -414,7 +433,8 @@ place the next thing below or beside them.
| `at`, `out` | when it appears and when it leaves |
| `enter` | `draw`, `write`, `type`, `words`, `sync`, `pop`, `rise`, `fade`, `zoom`, `drop`, `slide-left`, `slide-right`, `slide-up`, `slide-down`, `none` |
| `exit` | `fade`, `pop`, `rise`, `drop`, `slide-left`, `slide-right`, `none` |
-| `dur` | length of the entrance in seconds |
+| `dur` | length of the entrance in seconds (shortened by the pace) |
+| `outDur` | length of the exit in seconds (0.45) |
| `id` | a name for arrows, `annotate` and `s.get` |
| `color` | a color name (`blue`, `accent`, `ink`, `muted`, ...) or any CSS color |
| `opacity`, `scale`, `rotate` | extra changes, `rotate` in degrees |
@@ -464,7 +484,8 @@ takes the same options.
`s.arrow(from, to, options)`: `from` and `to` are `[x, y]`, `{ x, y }` or the
`id` of an element drawn earlier, and then the arrow stops at its edge.
Options: `bend` (about -1 to 1), `head` (`end`, `start`, `both`, `none`),
-`label`, `labelSize`, `labelColor`, `labelOffset`, `gap`, `dashed`, `width`,
+`label`, `labelSize`, `labelColor`, `labelFont` (the handwritten font by
+default; `'body'` suits `clean`), `labelOffset`, `gap`, `dashed`, `width`,
`headSize`. `s.connect` is the same.
### Icons and images
@@ -479,7 +500,7 @@ s.image('assets/cables.png', { w: 1000, frame: 'card', at: 'cables', kenburns: t
The 1,854 icons come from Lucide. Search with `explainroo icons <word>`; old
Lucide names work too. Icon options: `size` (120), `color`, `weight` (2),
`bg` (`true`, `'circle'`, `'square'` or a color), `bgScale`, `label`,
-`labelSize`.
+`labelSize`, `labelColor`, `labelFont`.
`s.image(src, options)` draws a file from `assets/`: `w` and/or `h` (the
image keeps its shape), `fit` (`cover` or `contain`), `radius`, `frame`
@@ -590,6 +611,28 @@ roles: `display`, `body`, `hand`, `mono`.
- Randomness from anything other than `s.rand()`, `s.noise()` or
`Math.random()` inside the scene function.
- Text in AI images. Put words on screen with `s.text`.
+- A cue word that is spoken twice. `at: 'writes'` means the first time;
+ use `s.cue('writes', 2)` or a `[#marker]` for a later one. `check` lists
+ such words as hints.
+- Words that sound like other words. The speech check hears "won" as "one"
+ and "plain" as "plane", and `{shown|spoken}` cannot fix that. Reword the
+ sentence. For acronyms spelled out letter by letter, write
+ `{API|A, P, I}`, not `A.P.I.` (the dots read like a web address).
+- Growing a shape by changing `w` or `h` over time in the hand-drawn looks.
+ The outline is drawn fresh every frame and wobbles. Animate `scale` instead.
+- A scene change in the middle of one diagram. The default transition of
+ `clean` slides the whole frame; give scenes that share a diagram
+ `{transition=fade}` or `{transition=cut}`.
+- Arrows from an icon with a label. The `id` of an icon covers the icon, not
+ its label, so an arrow leaving downward crosses the label.
+- Colored text in `paper` and `clean`. Green, orange and red are light
+ enough for shapes but often too light for text; `check` warns about low
+ contrast. Use `ink`, `accent` or a darker hex color for text.
+
+`check` looks at text: text off the frame, too close to the edge (3.5%), too
+small, low contrast (against the page, or the text's own `bg`), on top of
+other text, or under the captions. It does not see shapes, arrows and icons,
+or drawings made with `s.draw`. Look at the stills for those.
## Working on explainroo itself
diff --git a/engine/layout.js b/engine/layout.js
index 2a581c7..30c884c 100644
--- a/engine/layout.js
+++ b/engine/layout.js
@@ -1,24 +1,69 @@
-// Where scene content may go. Normal sizes keep a 7% margin. Platform formats
-// (tiktok, reels, shorts, vertical) keep clear of the app's own buttons and
-// text, and with captions on, the captions sit at the bottom of that area.
+// Where scene content may go, and where the captions sit.
+//
+// Normal sizes keep a 7% margin. Platform formats (tiktok, reels, shorts,
+// vertical) keep clear of the app's own buttons and text, and put the
+// watermark in the top right corner of the part the app leaves free.
+// With captions on, `safe` ends above the caption band, so content placed from
+// `s.safe` never sits under a caption.
+const LINES = 2;
+
export function layoutAreas(config, W, H) {
const m = Math.round(Math.min(W, H) * 0.07);
const f = config.safe;
- if (!f) {
- return { safe: { x: m, y: m, w: W - 2 * m, h: H - 2 * m, left: m, top: m, right: W - m, bottom: H - m }, captions: null };
- }
- const left = Math.round(W * f.left);
- const right = W - Math.round(W * f.right);
- const appTop = Math.round(H * f.top);
- // The watermark sits in the top right corner of the free area.
- const top = config.watermark ? appTop + Math.round(Math.min(W, H) * 0.028 * 1.6) + 16 : appTop;
- const appBottom = H - Math.round(H * f.bottom);
- let bottom = appBottom;
+ const wmSize = Math.round(Math.min(W, H) * 0.028);
+ const wmBox = Math.round(wmSize * 1.6);
+ const capSize = H > W ? 58 : 44;
+ const capBoxH = Math.round(capSize * 1.3 * LINES + capSize * 0.55);
+
+ let left;
+ let right;
+ let top;
+ let bottom;
+ let app = null;
let captions = null;
- if (config.captions) {
- captions = { cx: (left + right) / 2, bottom: appBottom - 10, maxWidth: right - left };
- // Room for two caption lines (58px text) plus a gap above them.
- bottom = appBottom - 10 - Math.round(58 * 1.3 * 2 + 58 * 0.55) - 24;
+
+ if (f) {
+ left = Math.round(W * f.left);
+ right = W - Math.round(W * f.right);
+ const appTop = Math.round(H * f.top);
+ const appBottom = H - Math.round(H * f.bottom);
+ app = { left, top: appTop, right, bottom: appBottom };
+ top = config.watermark ? appTop + wmBox + 16 : appTop;
+ bottom = appBottom;
+ if (config.captions) captions = { cx: (left + right) / 2, bottom: appBottom - 10, maxWidth: right - left };
+ } else {
+ left = m;
+ right = W - m;
+ top = m;
+ bottom = H - m;
+ if (config.captions) {
+ const maxWidth = W * (H > W ? 0.8 : 0.7) + capSize * 1.1;
+ if (H >= W * 1.5) {
+ // Tall 9:16 without a platform: captions a quarter of the way up.
+ captions = { cx: W / 2, bottom: Math.round(H * 0.74 + capBoxH / 2), maxWidth };
+ } else {
+ // Wide, square and 4:5: captions at the bottom, clear of the watermark.
+ const clear = config.watermark ? wmSize + wmBox + 12 : Math.round(m * 0.6);
+ captions = { cx: W / 2, bottom: H - clear, maxWidth };
+ }
+ }
+ }
+
+ let band = null;
+ if (captions) {
+ band = {
+ left: captions.cx - captions.maxWidth / 2,
+ right: captions.cx + captions.maxWidth / 2,
+ top: captions.bottom - capBoxH,
+ bottom: captions.bottom,
+ };
+ bottom = Math.min(bottom, band.top - 24);
}
- return { safe: { x: left, y: top, w: right - left, h: bottom - top, left, top, right, bottom }, captions, app: { left, top: appTop, right, bottom: appBottom } };
+
+ return {
+ safe: { x: left, y: top, w: right - left, h: bottom - top, left, top, right, bottom },
+ captions,
+ band,
+ app,
+ };
}
diff --git a/engine/runtime.js b/engine/runtime.js
index bc8c6f1..ce53275 100644
--- a/engine/runtime.js
+++ b/engine/runtime.js
@@ -62,7 +62,8 @@ export class Engine {
const areas = layoutAreas(config, this.W, this.H);
this.safeArea = areas.safe;
this.captionArea = areas.captions;
- this.appArea = areas.app || null;
+ this.captionBand = areas.band;
+ this.appArea = areas.app;
this.fps = config.fps;
this.scale = scale;
this.theme = getTheme(config.theme);
@@ -292,6 +293,7 @@ export class Engine {
reported.add(key);
fn();
};
+ this.cueNotes = [];
for (const sc of this.timeline.scenes) {
const entrances = [];
this.recorder = { push: (e) => entrances.push(e) };
@@ -315,12 +317,14 @@ export class Engine {
const a = this.appArea;
if (tx.x0 < a.left - 2 || tx.y0 < a.top - 2 || tx.x1 > a.right + 2 || tx.y1 > a.bottom + 2) {
once(`app:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" is outside the ${this.config.format} safe area (${a.left},${a.top} to ${a.right},${a.bottom}), where the app's buttons, name or caption can cover it`));
- } else if (this.captionArea && tx.y1 > this.safeArea.bottom + 2 && tx.x1 > a.left && tx.x0 < a.right) {
- once(`cap:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" reaches into the caption band below y=${this.safeArea.bottom}; keep scene content above s.safe.bottom`));
}
} else if (tx.x0 < margin || tx.y0 < margin || tx.x1 > W - margin || tx.y1 > H - margin) {
once(`edge:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" sits closer than ${margin}px to the edge`));
}
+ const b = this.captionBand;
+ if (b && tx.y1 > b.top + 2 && tx.y0 < b.bottom - 2 && tx.x1 > b.left && tx.x0 < b.right) {
+ once(`cap:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" is in the caption band (y ${Math.round(b.top)} to ${Math.round(b.bottom)}), where captions can cover it; keep content above s.safe.bottom (${Math.round(this.safeArea.bottom)})`));
+ }
if (!tx.block && tx.size < minSize) once(`small:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" is ${Math.round(tx.size)}px, hard to read (use at least ${minSize}px)`));
const bg = opaque(tx.bg || this.theme.bg, this.theme.bg);
if (typeof tx.fg === 'string' && contrast(tx.fg, bg) < 3) once(`contrast:${sc.id}:${tx.text}`, () => add('warn', sc.id, t, `"${label}" has low contrast against its background`));
@@ -358,6 +362,12 @@ export class Engine {
}
}
}
+ const repeated = new Map();
+ for (const c of this.cueNotes) repeated.set(`${c.scene}|${c.word.toLowerCase()}`, c);
+ for (const c of repeated.values()) {
+ add('hint', c.scene, null, `cue "${c.word}" means the first of the ${c.count} times it is spoken in this scene; if you meant a later one, use s.cue("${c.word}", 2) or a [#marker]`);
+ }
+ this.cueNotes = null;
return issues;
}
}
diff --git a/engine/stage.js b/engine/stage.js
index fe2fc3a..57c0460 100644
--- a/engine/stage.js
+++ b/engine/stage.js
@@ -30,7 +30,8 @@ export class Stage {
this.platform = Boolean(engine.config.safe);
this.cx = this.platform ? this.safe.x + this.safe.w / 2 : this.W / 2;
this.cy = this.platform ? this.safe.y + this.safe.h / 2 : this.H / 2;
- // pace > 1 makes every animation, pause and numeric time shorter.
+ // pace > 1 makes animations shorter. Times stay real seconds of the
+ // scene, like s.t and the word cues, so `s.cue('word') + 0.3` keeps working.
this.pace = engine.timeline.pace || 1;
this.t = t;
this.T = scene.start + t;
@@ -70,10 +71,16 @@ export class Stage {
if (!ws.length) throw new SceneError(`cue("${word}"): scene "${this.id}" has no narration`);
const norms = ws.map((w) => normWord(w.text));
let seen = 0;
+ let found = null;
for (let i = 0; i + want.length <= norms.length; i++) {
let ok = true;
for (let k = 0; k < want.length; k++) if (norms[i + k] !== want[k]) { ok = false; break; }
- if (ok && ++seen === n) return { start: ws[i].start, end: ws[i + want.length - 1].end };
+ if (ok && ++seen === n && !found) found = { start: ws[i].start, end: ws[i + want.length - 1].end };
+ }
+ if (found) {
+ // A word spoken more than once silently means its first time; tell the check.
+ if (seen > 1 && n === 1 && this.engine.cueNotes) this.engine.cueNotes.push({ scene: this.id, word: String(word), count: seen });
+ return found;
}
const count = seen ? ` (it is spoken ${seen} time${seen > 1 ? 's' : ''})` : '';
const hint = suggest(want[0], [...new Set(norms)], 6).join(', ');
@@ -90,7 +97,7 @@ export class Stage {
// Accepts seconds, "#mark" or a spoken word/phrase.
time(v) {
- if (typeof v === 'number') return v / this.pace;
+ if (typeof v === 'number') return v;
if (typeof v === 'string') return v.startsWith('#') ? this.mark(v.slice(1)) : this.cue(v);
if (v === undefined || v === null) return 0;
throw new SceneError(`a time must be seconds, a spoken word or "#mark", not ${JSON.stringify(v)}`);
diff --git a/src/cli.js b/src/cli.js
index 9be4881..7b55ae2 100644
--- a/src/cli.js
+++ b/src/cli.js
@@ -2,6 +2,7 @@
import fs from 'node:fs';
import path from 'node:path';
import { loadProject, ProjectError, THEMES, MUSIC_STYLES, FORMATS, resolveSize, normalizeConfig } from './project.js';
+import { layoutAreas } from '../engine/layout.js';
import { ScriptError } from './script.js';
import { prepare, makeState } from './pipeline.js';
import { render } from './render.js';
@@ -181,11 +182,15 @@ export async function main(argv) {
return 0;
}
case 'formats': {
- const rows = Object.entries(FORMATS).map(([name, f]) => {
- const safe = f.safe ? ` keeps clear: top ${pct(f.safe.top)}, bottom ${pct(f.safe.bottom)}, left ${pct(f.safe.left)}, right ${pct(f.safe.right)}` : '';
- return `${name.padEnd(10)} ${`${f.size[0]}x${f.size[1]}`.padEnd(10)} ${f.use}${safe}`;
+ // The free area for content with the defaults (watermark on, captions
+ // as "auto" decides), the same box scenes get as s.safe.
+ const info = Object.entries(FORMATS).map(([name, f]) => {
+ const cfg = normalizeConfig({ size: name });
+ const { safe } = layoutAreas(cfg, cfg.width, cfg.height);
+ return { name, width: f.size[0], height: f.size[1], use: f.use, captions: cfg.captions, content: { x: safe.x, y: safe.y, w: safe.w, h: safe.h } };
});
- print(rows.join('\n') + '\n\nAlso: 16:9, 9:16, 1:1, 4:5 or WIDTHxHEIGHT. Set it with "size" in video.json or init --size.', { formats: FORMATS });
+ const rows = info.map((r) => `${r.name.padEnd(10)} ${`${r.width}x${r.height}`.padEnd(10)} ${r.use}\n${''.padEnd(22)}content area ${r.content.w}x${r.content.h} at ${r.content.x},${r.content.y}${r.captions ? ', captions below it' : ''}`);
+ print(rows.join('\n') + '\n\nAlso: 16:9, 9:16, 1:1, 4:5 or WIDTHxHEIGHT. Set it with "size" in video.json or init --size.', { formats: info });
return 0;
}
case 'icons': return icons(pos, print);
@@ -207,10 +212,6 @@ export async function main(argv) {
}
}
-function pct(x) {
- return `${Math.round(x * 100)}%`;
-}
-
function init(dir, flags, print) {
if (!dir) throw new ProjectError('usage: explainroo init <dir> [--theme paper] [--size youtube|tiktok|linkedin|...] [--pace 1.2] [--voice af_heart] [--title "..."]');
const target = path.resolve(dir);
diff --git a/test/formats-pace.test.js b/test/formats-pace.test.js
index 3312e9b..b5c1510 100644
--- a/test/formats-pace.test.js
+++ b/test/formats-pace.test.js
@@ -29,9 +29,23 @@ test('platform safe area keeps content above the captions and clear of the butto
assert.equal(safe.right, 1080 - Math.round(1080 * FORMATS.tiktok.safe.right));
const plain = layoutAreas(normalizeConfig({ size: '16:9' }), 1920, 1080);
assert.equal(plain.captions, null);
+ assert.equal(plain.band, null);
assert.equal(plain.safe.left, Math.round(1080 * 0.07));
});
+test('captions on feed sizes sit at the bottom, clear of the watermark and the content area', () => {
+ for (const size of ['linkedin', 'square', 'instagram']) {
+ const cfg = normalizeConfig({ size });
+ const { safe, band } = layoutAreas(cfg, cfg.width, cfg.height);
+ const wm = Math.round(Math.min(cfg.width, cfg.height) * 0.028);
+ assert.ok(band, size);
+ assert.ok(band.bottom <= cfg.height - wm - Math.round(wm * 1.6), `${size}: captions reach the watermark`);
+ assert.ok(safe.bottom < band.top, `${size}: content area overlaps the captions`);
+ }
+ const noCaptions = normalizeConfig({ size: 'linkedin', captions: false });
+ assert.equal(layoutAreas(noCaptions, 1080, 1350).safe.bottom, 1350 - Math.round(1080 * 0.07));
+});
+
test('pace must stay in range and must not make the voice too fast', () => {
assert.equal(normalizeConfig({}).pace, 1);
assert.equal(normalizeConfig({ pace: 1.3 }).pace, 1.3);