Skip to content

armgit / 67.js

Public Imported from GitHub
0

3D engine that AI agents drive

Merge origin/main into feat/611-render-output

Nguyen Xuan An2dd1a07ce071
diff --cc skills/render-67js-media/SKILL.md
index 7df8bf29,bba45985..adfd937f
--- a/skills/render-67js-media/SKILL.md
+++ b/skills/render-67js-media/SKILL.md
@@@ -4,59 -4,50 +4,51 @@@ description: Export a 67js scene to PN
license: MIT
---
- # Xuất ảnh và video 67js
-
- Locate the 67js checkout and read its `scripts/render-media.mjs` and `src/budget.js`.
- Read [rendering.md](references/rendering.md) for commands, backends and measurement limits.
- Use the repository pipeline rather than constructing a parallel exporter.
-
- For MCP snapshot jobs, read the checkout's `docs/agent-render-jobs.md` and verify source versus published support.
- Configure a private job store. Poll status after cancellation before resuming preview edits.
- For a video with cuts, sound or an encoder choice, use the recipe `timeline`, `settings.timeline`, `settings.audio` (a file in the server's `--audio` folder) and `settings.encoder` described in the same document; the job reports measured sync.
- For compositing output, add `settings.transparent: true` (PNG with alpha; no sky or auto ground), `settings.format` `png16` or `exr` (depth in metres, object IDs as float) and `settings.region` `{x, y, width, height}` (a crop of the whole frame, also for video); video runs up to 60 fps. These stay off unless set, so defaults are unchanged. Details in `docs/agent-render-jobs.md`, section Output options.
- For procedural motion, read `docs/agent-motion.md` and select `settings.motion.start`. Preview seeks preserve revisions and authored recipes.
-
- For imported character motion from `0.8.0`, read `docs/agent-character-motion.md`. Use independent `settings.animation` and `settings.motion` clocks.
-
- For an offline path-traced still with **Cook** (glass refraction, soft multi-bounce indirect light, subsurface random walk, many lights), read `docs/path-tracer.md` and submit an `image` job with `settings.renderer: { type: "pathTracer" }`. It is slow, bounded and offline: read `scene_capabilities` `pathTracer` for its limits and what it skips, and keep rasterizer jobs for video and passes.
-
- ## Render a scene.json world along a camera track
-
- To film a world the showcase page shows (GLB, sky, renderer options, exposure, effects), plan the path and render it headless:
-
- ```powershell
- # keys.json: { "fps": 24, "frames": [[[x, y, z], [tx, ty, tz]], ...] } (one eye and target per frame; createCameraPath keys also work)
- node scripts/render-media.mjs --kind video --scene file --config public/showcase/bistro/scene.json --camera keys.json --width 960 --height 540 --duration 5 --output artifacts/walk.mp4
- ```
-
- `--root <folder>` names the folder that `/showcase/...` paths in the config are relative to; without it they resolve next to the config. `--duration` and `--start` pick a slice of the track (default: all of it, at the track's fps). Check one frame first with `--kind image --time 2.5`.
-
- ## Choose and render
-
- 1. Check the available GPU memory before a GPU job on a shared machine.
- 2. Choose GPU rendering for speed. Choose the CPU preset when hardware WebGL is unavailable.
- 3. Choose the smallest preset that meets the output requirement.
- 4. Set width, height, samples and memory before rendering.
- 5. Keep one browser, one live scene renderer and one pending capture frame.
- 6. Stream frames to FFmpeg. Await encoder backpressure before capturing the next frame.
- 7. Stop on browser errors, graphics errors or a memory guard failure.
- 8. Inspect representative stills before the full film export.
- 9. Verify duration, frame count, dimensions, color metadata, audio and file size with FFprobe.
- 10. Inspect the output report and close the browser and encoder after the job.
-
- Use below-normal priority for heavy local exports. Do not stop another session's process.
- Use deterministic scene time. Do not use a real-time screen recording for offline export.
-
- CPU encoding and CPU rendering are separate choices.
- GPU encoding uses NVENC when supported. CPU encoding uses two threads by default.
-
- ## Report the limits
-
- The framebuffer estimate is an allocation model, not a measured total VRAM figure.
- Node RSS and the sampled browser JavaScript heap do not cover every browser allocation.
- Report the actual renderer backend, estimated target memory, observed process metrics and elapsed export time.
- Do not promise that a budget prevents every driver or system allocation failure.
- When resolution is reduced, state the actual render dimensions.
-
- Deliver the PNG or MP4, its JSON report and the reproduction command.
- For a score, record provenance and measured loudness. Say whether anyone listened to it.
+ # Export images and video from 67js over MCP
+
+ A render job is an immutable snapshot of the active scene. Edits made while it runs do not change it, and it does not
+ change the scene's revision, pose or checkpoints. The MCP server must have been started with `--jobs <abs private dir>`;
+ MP4 also needs `ffmpeg` on PATH or `--ffmpeg <abs path>`, and sound needs `--audio <abs dir>`. Without `--jobs` the
+ submit call fails: tell the user which launch option is missing.
+
+ ## Steps
+
+ 1. `scene_look` first. Never submit a job for a scene you have not looked at.
+ 2. `scene_inspect` for the current `revision`.
+ 3. `scene_render_submit` with `expectedRevision` and `settings`. Start small: one image at 960x540.
+ 4. `scene_render_status` with the `jobId` until the state is terminal (`completed`, `failed`, `cancelled`,
+ `interrupted`). Poll about every 5 to 10 seconds, and for video read the durable frame progress. Never loop tightly.
+ 5. `scene_render_result`: output paths, SHA-256 hashes and metrics. Report the actual `width` and `height`, the backend
+ and the seconds; memory limits can lower the size.
+ 6. Keep what you need, then `scene_render_release` to remove the job's files.
+ 7. `scene_render_cancel` stops a job; poll `scene_render_status` until it is `cancelled` before editing again.
+
+ ## Settings by task
+
+ | Task | Read |
+ |---|---|
+ | Image, stills, video, motion clock, shots, audio, encoder, passes, path tracer | `${CLAUDE_SKILL_DIR}/references/jobs.md` |
+ | Size, memory, GPU or CPU, time limits, verifying the file | `${CLAUDE_SKILL_DIR}/references/limits-and-checks.md` |
+ | A job failed or the result looks wrong | `${CLAUDE_SKILL_DIR}/references/errors-and-recovery.md` |
+ | Rendering in a source checkout with `scripts/render-media.mjs` (contributors only) | `${CLAUDE_SKILL_DIR}/references/rendering.md` |
+
+ ## Rules
+
+ - Smallest output that meets the need: a still before a video, a 2 second slice before the full film.
+ Every film round starts as a small, low-quality preview with sound; full quality only after sign-off.
+ - Time is deterministic. A frame depends on its time only, so a seek gives the same pixels as the frame in a video.
+ Never record the screen.
+ - Use `settings.motion: { "start": 0 }` to play recipe tracks (camera, nodes, lights, sun, sky, effects). Without it the
+ job holds the first pose, which is the usual reason a video is a still frame. A recipe with a `timeline` uses
+ `settings.timeline` and rejects `motion` and `animation`.
+ - Imported clips use `settings.animation`; the two clocks are independent.
++- For compositing, `settings.transparent`, `settings.format` (`png16`, `exr`) and `settings.region` are off by default; video runs up to 60 fps (`references/jobs.md`, Compositing output).
+ - Path tracing (`settings.renderer`) is offline and slow, for stills; video and passes use the rasterizer.
+ - Do not stop another session's process. Heavy local exports run at below-normal priority, one job at a time.
+ - A GLB for Blender is `scene_export_glb`, not a render job.
+
+ ## Deliver
+
+ The file path, its JSON metrics, the settings you sent and one frame you looked at. Say what was measured (frames,
+ duration, seconds, dimensions) and what was judged by eye. Say whether anyone listened to the audio; a score
+ needs provenance and measured loudness.
diff --cc skills/render-67js-media/references/jobs.md
index 00000000,79ac4efb..3b3c9954
mode 000000,100644..100644
--- a/skills/render-67js-media/references/jobs.md
+++ b/skills/render-67js-media/references/jobs.md
@@@ -1,0 -1,95 +1,114 @@@
+ # Render job settings
+
+ Every recipe is the arguments of `scene_render_submit`. `sessionId` may be left out. `expectedRevision` is the
+ revision `scene_inspect` returned. Required in `settings`: `kind` (`image`, `stills` or `video`), `width`, `height`
+ (64 to 1920 each; video sizes must be even). Defaults: 24 fps and one second.
+ `tests/render-skill-examples.test.js` checks that these blocks stay valid JSON with known settings fields.
+
+ ## One PNG
+
+ ```json
+ { "expectedRevision": 3, "settings": { "kind": "image", "width": 960, "height": 540 } }
+ ```
+
+ An image samples the pose at `motion.start`, so a still of a mid-animation moment is
+ `"motion": { "start": 2.5 }`. Lens, depth of field, look and fog come from the recipe, so a still matches `scene_look`.
+
+ ## Eight stills
+
+ `kind: "stills"` writes eight evenly spaced PNGs (a contact sheet of a motion). Use it to check a film before you
+ render the video.
+
+ ```json
+ { "expectedRevision": 3, "settings": { "kind": "stills", "width": 640, "height": 360, "duration": 4, "motion": { "start": 0 } } }
+ ```
+
+ ## MP4 of recipe motion
+
+ ```json
+ { "expectedRevision": 3, "settings": {
+ "kind": "video", "width": 1280, "height": 720, "fps": 24, "duration": 4, "motion": { "start": 0 }
+ } }
+ ```
+
+ Video is H.264 with one frame in flight. At most 60 s and 900 frames, so 60 s needs 15 fps or less. A nonlooping
+ motion must cover `start + duration`. The animated lights, sun, sky and effects tracks play with the same setting.
+ Motion blur (`effects.motionBlur.shutter`) needs camera tracks and this `motion` setting.
+
+ ## Shots, cuts and sound
+
+ A recipe `timeline` (see `build-67js-scene`, `film-and-animation.md`) plays its shots, so the job takes `timeline`
+ and no `motion` or `animation`. `duration` defaults to the rest of the timeline.
+
+ ```json
+ { "expectedRevision": 4, "settings": {
+ "kind": "video", "width": 1280, "height": 720, "fps": 24,
+ "timeline": { "start": 0 },
+ "audio": { "file": "score.wav", "offset": 0 }
+ } }
+ ```
+
+ `audio.file` is a plain name of a `.wav`, `.mp3`, `.ogg` or `.flac` file (32 MiB at most) in the server's `--audio`
+ folder; no path or URL. It is resampled to 48 kHz and encoded as AAC. The result reports `metrics.sync`, and a
+ difference over 50 ms fails the job. A file shorter than the video after its `offset` fails before rendering. For a film
+ longer than one job (60 s) render parts with `track: "pcm"` (lossless, no encoder padding) and join them with
+ `ffmpeg -f concat -safe 0 -i parts.txt -c copy joined.mp4`, then lay the sound once.
+
+ ## Encoder
+
+ `encoder: "cpu"` (libx264, default) or `"nvenc"` (h264_nvenc, in-page hardware encoding). A machine without NVENC
+ fails the job and does not fall back, so choose `cpu` unless you know the machine. The result reports
+ `metrics.videoEncoder`, `captureSeconds` and `renderSeconds`.
+
+ ## Render passes
+
+ `image` and `stills` jobs take `passes`, a list of `albedo`, `normal`, `emission`, `depth` and `objectId`. Each frame
+ also writes one raw PNG per pass at the lit image's size, with no grading, bloom, depth of field or antialiasing.
+
+ ```json
+ { "expectedRevision": 3, "settings": { "kind": "image", "width": 960, "height": 540, "passes": ["depth", "objectId"] } }
+ ```
+
+ Only opaque and cutout geometry appears. `scene_render_result` metrics carry `objectNames` (read the names for the IDs;
+ IDs follow draw order, not node order) and `depthRangeMetres`. Depth is metres over 100 in a 16-bit value (high byte
+ in red, low in green); background is `255, 255, 0`. Videos hold the lit image only.
+
++## Compositing output: alpha, 16-bit, EXR, region, 60 fps
++
++Four optional settings, off unless set, so defaults write the same bytes as before.
++
++```json
++{ "expectedRevision": 3, "settings": { "kind": "image", "width": 1280, "height": 720, "transparent": true, "format": "exr",
++ "passes": ["depth"], "region": { "x": 320, "y": 180, "width": 640, "height": 360 } } }
++```
++
++- `transparent: true` (image, stills): the lit PNG keeps alpha. Sky, environment background and the automatic ground are not
++ drawn, so add your own ground or a `shadowCatcher` node; volume fog is refused; passes ignore it.
++- `format` (image, stills): `png` (default), `png16` (16-bit RGB or RGBA) or `exr` (half float, premultiplied alpha). Files are
++ `render.png` or `render.exr` plus `render-<pass>.<ext>`. Depth is a 16-bit gray step in `png16` and float metres in channel `Z`
++ in `exr`; object IDs are a gray 16-bit PNG (up to 65535 objects) or float channel `id`.
++- `region: { x, y, width, height }` (pixels from the top left, each side 16 or more; even sides for video): the frame is
++ rendered whole and cropped, so the look matches the full frame.
++- `fps` up to 60 for video, inside 900 frames.
++- The path tracer refuses transparent, format and region; H.264 refuses transparent and format.
++
+ ## Path traced still
+
+ Offline Cook path tracing: glass refraction, soft multi-bounce light, subsurface, many lights. Slow and bounded, for
+ stills. Read `scene_capabilities` (`pathTracer`) for limits and what it skips.
+
+ ```json
+ { "expectedRevision": 3, "settings": {
+ "kind": "image", "width": 960, "height": 540,
+ "renderer": { "type": "pathTracer", "backend": "cpu", "samples": 64, "bounces": 8, "denoise": true, "maxSeconds": 120 }
+ } }
+ ```
+
+ `backend` is `cpu` (default, reproducible), `gpu` (needs a server started with `--backend gpu`) or `native`
+ (hardware ray queries; it falls back to `gpu` or `cpu` and `metrics.pathTracer.backendNote` says why). Start with 16
+ samples to judge the light, then raise them.
+
+ ## Imported character clips
+
+ `animation: { "clip": 0, "start": 1 }` samples an imported asset's clip and `rendererTime` sets shader time. They
+ are separate clocks from `motion`. Clip ids come from `scene_inspect`.