Skip to content

One-click character swap for videos: an open-source, local Higgsfield Genjutsu alternative built on ComfyUI and MiniMax H3, with FLUX.3 Klein relighting.

Remove the From picture and ControlNet tabs (#23)

mixbag4af1a70717ec
diff --git a/README.md b/README.md
index 718f982..2d97cee 100644
--- a/README.md
+++ b/README.md
@@ -8,7 +8,7 @@ The app is a small Next.js front end over a ComfyUI workflow. It uses [`@comfyor
- **Simple motion transfer**: available. One character replaces the main person (the one who fills the most of the frame). Runs `workflows/simple.api.json`, exported from `H3 One-Click Single Swap (DMAD 6 step).json`.
- **Advanced motion transfer**: available. 1–4 characters, each with an optional role; every tracked person is matched to a role per shot, and people without a character stay untouched. Runs `workflows/advanced.api.json`, exported from `H3 One-Click Advanced Swap (DMAD 6 step).json`; **One pass per person** (two characters) runs `workflows/advanced_maxq.api.json`.
-- **Object swap**: available. **Change** what SAM tracks (hair, a jacket, a car) into what you describe or show in a picture; **Remove** it; or change the **Background** around it (`person` stays). Only that area is regenerated. **From picture** recreates the person, outfit, setting and lighting from the uploaded picture, using a grey motion guide without ControlNet. For relighting, upload an edited frame from the selected source section. Portrait pictures can produce side bars. **ControlNet** takes both the person and the place from the picture: the whole frame is regenerated, and a depth map of the person in the video (SAM 3.1 + Depth Anything V2, through the H3 Fun ControlNet Union 2.0) keeps their framing, camera and motion. Everyone else in the video is replaced by the picture's setting. It runs as one H3 clip of up to 15 s (a longer part is cut to its first 15 s), because a continuation clip loses the depth framing. Runs `workflows/object.api.json`, exported from `H3 One-Click Object Swap (DMAD 6 step).json`; ControlNet runs `workflows/control.api.json`, exported from `H3 One-Click Picture Swap ControlNet (DMAD 6 step).json`.
+- **Object swap**: available. **Change** what SAM tracks (hair, a jacket, a car) into what you describe or show in a picture; **Remove** it; or change the **Background** around it (`person` stays). Only that area is regenerated. Runs `workflows/object.api.json`, exported from `H3 One-Click Object Swap (DMAD 6 step).json`.
- **Post-processing**: available, for a finished video. **Fix faces**: H3 draws faces poorly when the person is small in the frame, so this redraws the face of the person in a photo at close-up size and pastes it back. Runs `workflows/face.api.json`, written by `workflows/build/openjutsu_face_build.py`. **Fix colours** softens H3's harsh look without an H3 run: Image Detail Contrast takes some fine detail contrast out, then Orion4D's DCTL Tone Mapper grades every frame. Runs `workflows/color.api.json`.
## Requirements
@@ -32,7 +32,6 @@ Install these into `ComfyUI/custom_nodes` (ComfyUI Manager finds the first three
| [ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) | Fix faces: SAM loader |
| [comfyui-image-detail-contrast](https://github.com/D3voz/comfyui-image-detail-contrast) | Fix colours: Image Detail Contrast |
| [Orion4D_FXMax](https://github.com/orion4d/Orion4D_FXMax) | Fix colours: DCTL Tone Mapper and its presets |
-| [comfyui_controlnet_aux](https://github.com/Fannovel16/comfyui_controlnet_aux) | ControlNet: Depth Anything V2 (downloads `depth_anything_v2_vitb.pth` on first use) |
Restart ComfyUI after installing them.
@@ -51,7 +50,6 @@ Before uploading a job, Openjutsu checks the selected workflow against ComfyUI's
| `ultralytics/bbox`, `ultralytics/segm` | `face_yolov8m.pt`, `person_yolov8m-seg.pt` ([Bingsu/adetailer](https://huggingface.co/Bingsu/adetailer)), for Fix faces |
| `sams` | `sam_vit_b_01ec64.pth`, for Fix faces |
| `insightface/models` | `buffalo_l` (InsightFace downloads it on first use), for Fix faces |
-| `model_patches` | `minimax_h3_fun_controlnet_union_2.0_pruned_int8_convrot.safetensors` ([Kijai/MiniMax-H3-experimental](https://huggingface.co/Kijai/MiniMax-H3-experimental/tree/main/model_patches)), for ControlNet |
## Run it locally
@@ -131,7 +129,7 @@ Object swap uses the same masked edit for the tracked things; SAM 3 tracks each
Fix faces works on the face, not the frame. H3 Face Track + Crop follows the face of the person in the photo over the whole video (InsightFace picks them out when there are several) and crops it to 512×512, so a face a few dozen pixels tall fills the canvas. H3 Face Fix Chunks splits the crops into independent chunks (6.5 s by default, overlapping by at least 1 s) and outputs them as lists, so ComfyUI runs the H3 nodes once per chunk. Each chunk is a masked edit: only the SAM face mask is regenerated with the photo as reference, while the hair, ears, collar and background around it stay, which keeps the new face from sliding against the head. For Follow the mouth, a blurred copy of the crop is the motion reference. H3 Face Fix Join crossfades the chunk overlaps and lowers the per-frame paste weight where the face is large, and H3 Face Stitch pastes the face back through a slightly smaller mask with the original soundtrack. The chunks are not chained through the context loop: each one starts from the swap's own frames, so colours do not drift. `lib/face.ts` removes the blurred face for Match the photo and sets the seed and chunk length.
-The swap workflows are built by the scripts in `workflows/build/`: `oneclick_single_build.js` (Simple), `oneclick_advanced_build.js` (Advanced; set `globalThis.H3_MAX_QUALITY = true` first for One pass per person), `oneclick_object_build.js` and `oneclick_control_build.js`. To rebuild one, open ComfyUI in the browser with an empty workflow tab (each script clears the active tab), run `const app = window.comfyAPI.app.app;` in the developer console, paste the script, enable its optional groups (hat fix, No lip-sync) where the export needs them, export **Workflow (API)** and replace the matching `workflows/*.api.json`. Fix faces is written by `python workflows/build/openjutsu_face_build.py`. If node ids change, update `NODES` in `lib/simple.ts`. `npm test` checks the pruning against the exports.
+The swap workflows are built by the scripts in `workflows/build/`: `oneclick_single_build.js` (Simple), `oneclick_advanced_build.js` (Advanced; set `globalThis.H3_MAX_QUALITY = true` first for One pass per person), and `oneclick_object_build.js`. To rebuild one, open ComfyUI in the browser with an empty workflow tab (each script clears the active tab), run `const app = window.comfyAPI.app.app;` in the developer console, paste the script, enable its optional groups (hat fix, No lip-sync) where the export needs them, export **Workflow (API)** and replace the matching `workflows/*.api.json`. Fix faces is written by `python workflows/build/openjutsu_face_build.py`. If node ids change, update `NODES` in `lib/simple.ts`. `npm test` checks the pruning against the exports.
## Deploy on the Comfy Developer Platform
diff --git a/app/api/jobs/route.ts b/app/api/jobs/route.ts
index ee46161..3fef141 100644
--- a/app/api/jobs/route.ts
+++ b/app/api/jobs/route.ts
@@ -1,7 +1,6 @@
import { apiAccess, HttpError, IMAGE_SIGNATURE, readFormData, StoredFile, VIDEO_SIGNATURE } from "../../../lib/http.ts";
import { NextResponse } from "next/server";
-import { listModels, submitAdvanced, submitColor, submitControl, submitFace, submitFit, submitMask, submitMontage, submitObject, submitPeople, submitSimple } from "../../../lib/comfy.ts";
-import type { ControlSettings } from "../../../lib/control.ts";
+import { listModels, submitAdvanced, submitColor, submitFace, submitFit, submitMask, submitMontage, submitObject, submitPeople, submitSimple } from "../../../lib/comfy.ts";
import { TRACKS_FILE, type Label } from "../../../lib/montage.ts";
import { DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, type ColorSettings, type ToneMode, type ToneSettings, type ToneValue } from "../../../lib/color.ts";
import { CHUNK_SECONDS, FACE_STYLES, type FaceSettings, type FaceStyle } from "../../../lib/face.ts";
@@ -141,14 +140,7 @@ function parseColor(form: FormData): ColorSettings {
function parseSimple(form: FormData) {
const picture = parsePicture(form.get("picture"));
- const settings: SimpleSettings = { sceneFromPicture: form.get("scene_from_picture") === "true", role: String(form.get("role") ?? "").slice(0, 200), noLipSync: form.get("no_lip_sync") === "true", ...parseRun(form), ...parseMask(form) };
- return { picture, settings };
-}
-
-function parseControl(form: FormData) {
- const picture = parsePicture(form.get("picture"));
- const { resolution, seed } = parseRun(form);
- const settings: ControlSettings = { target: parseMask(form).target, noLipSync: form.get("no_lip_sync") === "true", resolution, seed };
+ const settings: SimpleSettings = { role: String(form.get("role") ?? "").slice(0, 200), noLipSync: form.get("no_lip_sync") === "true", ...parseRun(form), ...parseMask(form) };
return { picture, settings };
}
@@ -259,10 +251,6 @@ async function submitForm(form: FormData) {
const { picture, settings } = parseObject(form);
const model = parseModel(form, await listModels());
submit = () => submitObject(video, section, picture, settings, model);
- } else if (mode === "control") {
- const { picture, settings } = parseControl(form);
- const model = parseModel(form, await listModels());
- submit = () => submitControl(video, section, picture, settings, model);
} else if (mode === "face") {
const { picture, settings } = parseFace(form);
const model = parseModel(form, await listModels());
diff --git a/components/openjutsu.tsx b/components/openjutsu.tsx
index 76ace94..0146542 100644
--- a/components/openjutsu.tsx
+++ b/components/openjutsu.tsx
@@ -9,7 +9,6 @@ import { CLIP_SECONDS, DEFAULT_TARGET, MARGINS, MAX_SEED, MIN_SECTION_SECONDS, M
import { MAX_CHARACTERS } from "../lib/advanced.ts";
import { DEFAULT_MODEL, MAX_LORAS, STEPS, STRENGTH, type Lora } from "../lib/model.ts";
import type { EditMode, FitSummary, PaintKeyframe, PlacementKeyframe } from "../lib/object.ts";
-import { CONTROL_MAX_SECONDS } from "../lib/control.ts";
import { ObjectPlacement } from "./object-placement.tsx";
import { CHUNK_SECONDS, FACE_STYLES, faceChunkCount, type FaceStyle } from "../lib/face.ts";
import { DEFAULT_PRESET, DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, toneFromPreset, type ToneSettings, type ToneValue } from "../lib/color.ts";
@@ -18,7 +17,7 @@ import { appearances, renderLabels, type Appearance, type Kept, type Label, type
type Character = { file: File; url: string; role: string };
type Mode = "simple" | "advanced" | "object" | "post";
type Post = "face" | "color";
-type Kind = "simple" | "mask" | "advanced" | "advanced_maxq" | "object" | "fit" | "control" | "face" | "color" | "people" | "montage";
+type Kind = "simple" | "mask" | "advanced" | "advanced_maxq" | "object" | "fit" | "face" | "color" | "people" | "montage";
type TonePreset = { name: string; settings: Record<string, unknown> };
// frames: the video, part and crop SAM saw
type Run = { id: string; kind: Kind; margin: Margin; clips: number; clipSeconds: number; startedAt: number; frames: string };
@@ -34,7 +33,7 @@ const HISTORY_LENGTH = 20;
const KIND_LABELS: Record<Kind, string> = {
simple: "Person swap", mask: "Mask check", advanced: "Advanced swap", advanced_maxq: "Advanced swap", object: "Object swap", fit: "Fit check",
- control: "ControlNet", face: "Fix faces", color: "Fix colours", people: "Find people", montage: "Montage",
+ face: "Fix faces", color: "Fix colours", people: "Find people", montage: "Montage",
};
const STATUS_LABELS: Record<string, string> = { succeeded: "Done", failed: "Failed", canceled: "Canceled", expired: "Expired" };
@@ -52,15 +51,11 @@ const FACE_STYLE_TEXT: Record<FaceStyle, { label: string; hint: string }> = {
mouth: { label: "Follow the mouth", hint: "The face follows the video's expressions and mouth movements; a little less like the photo." },
};
-// "From picture" recreates the whole scene with a grey motion reference.
-type EditTab = EditMode | "picture" | "controlnet";
-const EDITS: { id: EditTab; label: string; target: string; what: string; to: string; toHint: string }[] = [
+const EDITS: { id: EditMode; label: string; target: string; what: string; to: string; toHint: string }[] = [
{ id: "change", label: "Change", target: "hair, top", what: "What to change", to: "short bright pink hair and a black leather jacket", toHint: "What it becomes" },
{ id: "add", label: "Add", target: "on the table", what: "Where it goes", to: "a chocolate cake on a plate", toHint: "Object to add" },
{ id: "remove", label: "Remove", target: "poster", what: "What to remove", to: "a plain wall", toHint: "What shows instead (optional)" },
{ id: "background", label: "Background", target: "person", what: "What stays", to: "a snowy mountain pass at dusk", toHint: "The new place" },
- { id: "picture", label: "From picture", target: "person", what: "Who is replaced", to: "", toHint: "" },
- { id: "controlnet", label: "ControlNet", target: "person", what: "Who is replaced", to: "", toHint: "" },
];
const isRun = (run: Run | null) => typeof run?.id === "string" && Object.hasOwn(KIND_LABELS, run.kind);
@@ -92,7 +87,7 @@ function readHistory(): PastRun[] {
type Settings = {
mode: Mode; maxQuality: boolean; montage: boolean; resolution: Resolution; customSize: Size; hatFix: boolean; noLipSync: boolean;
- margin: Margin; target: string; editMode: EditTab; editTarget: string; biggestOnly: boolean; keepOutline: boolean; post: Post;
+ margin: Margin; target: string; editMode: EditMode; editTarget: string; biggestOnly: boolean; keepOutline: boolean; post: Post;
detailOn: boolean; detail: number; toneOn: boolean; tone: ToneSettings; tonePreset: string; faceStyle: FaceStyle;
chunkSeconds: number; seed: number; clipSeconds: number; motion: MotionSize; steps: number; model: string; loras: Lora[];
};
@@ -248,7 +243,7 @@ export function Openjutsu() {
const [noLipSync, setNoLipSync] = useState(false);
const [margin, setMargin] = useState<Margin>("normal");
const [target, setTarget] = useState(DEFAULT_TARGET);
- const [editMode, setEditMode] = useState<EditTab>("change");
+ const [editMode, setEditMode] = useState<EditMode>("change");
const [editTarget, setEditTarget] = useState(EDITS[0].target);
const [changeTo, setChangeTo] = useState("");
const [action, setAction] = useState("");
@@ -463,14 +458,11 @@ export function Openjutsu() {
}
const edit = EDITS.find((e) => e.id === editMode)!;
- const fromPicture = mode === "object" && editMode === "picture";
- const controlNet = mode === "object" && editMode === "controlnet";
- const clips = !video?.duration ? 0 : faceFix ? faceChunkCount(part.length || video.duration, chunkSeconds) : controlNet ? 1 : clipCount(part.length || video.duration, clipSeconds);
- const pictureOnly = fromPicture || controlNet;
+ const clips = !video?.duration ? 0 : faceFix ? faceChunkCount(part.length || video.duration, chunkSeconds) : clipCount(part.length || video.duration, clipSeconds);
const ready = colorFix ? detailOn || toneOn : mode === "object"
- ? editMode === "remove" || (pictureOnly ? !!look : (editMode !== "add" || placement.length > 0) && (!!changeTo.trim() || !!look))
+ ? editMode === "remove" || (editMode !== "add" || placement.length > 0) && (!!changeTo.trim() || !!look)
: !!character;
- const generateKind: Kind = mode === "object" ? controlNet ? "control" : fromPicture ? "simple" : "object"
+ const generateKind: Kind = mode === "object" ? "object"
: mode === "post" ? post : mode === "simple" ? "simple" : montage ? "montage" : onePassEach ? "advanced_maxq" : "advanced";
const tab = `Tab: ${mode === "post" ? `post/${post}` : mode === "object" ? `object/${editMode}` : mode === "advanced" && montage ? "advanced/montage" : mode}`;
@@ -496,17 +488,12 @@ export function Openjutsu() {
form.set("hat_fix", String(hatFix));
form.set("no_lip_sync", String(noLipSync));
form.set("margin", margin);
- form.set("target", pictureOnly ? editTarget.trim() || edit.target : target);
- if (kind === "simple" && fromPicture && look) {
- form.set("picture", look.file);
- form.set("role", "");
- form.set("scene_from_picture", "true");
- } else if (kind === "simple" && character) {
+ form.set("target", target);
+ if (kind === "simple" && character) {
form.set("picture", character.file);
form.set("role", character.role);
}
- if (kind === "control" && look) form.set("picture", look.file);
- if (kind === "simple" || kind === "control") {
+ if (kind === "simple") {
form.set("resolution", resolutionField);
form.set("seed", String(seed));
form.set("clip_seconds", String(clipSeconds));
@@ -585,13 +572,12 @@ export function Openjutsu() {
const shared = { resolution, motion, hatFix, noLipSync, margin, seed, clipSeconds };
const selectedModel = { model, loras, steps };
let selection: SetupRequest;
- if (kind === "simple") selection = { kind, settings: { ...shared, sceneFromPicture: fromPicture, role: fromPicture ? "" : character?.role ?? "", target: String(form.get("target")) }, model: selectedModel };
+ if (kind === "simple") selection = { kind, settings: { ...shared, role: character?.role ?? "", target: String(form.get("target")) }, model: selectedModel };
else if (kind === "advanced" || kind === "advanced_maxq") selection = { kind, settings: { ...shared, roles: shown.map((c) => c.role) }, pictures: shown.length, model: selectedModel };
else if (kind === "montage") selection = { kind, settings: { ...shared, roles: shown.map((c) => c.role) }, pictures: shown.length, review: review ?? null, model: selectedModel };
else if (kind === "object") selection = { kind, settings: { ...shared, mode: editMode as EditMode, target: String(form.get("target")), changeTo, biggestOnly, keepOutline,
...(editMode === "change" ? { paint: paint.map((p) => ({ ...p, frame: p.frame - Math.round((whole ? 0 : part.start) * 24) })) } : {}),
...(editMode === "add" ? { placement: placement.map((p) => ({ ...p, frame: p.frame - Math.round((whole ? 0 : part.start) * 24) })) } : {}) }, picture: !!look && editMode !== "remove", model: selectedModel };
- else if (kind === "control") selection = { kind, settings: { target: String(form.get("target")), resolution, noLipSync, seed }, model: selectedModel };
else if (kind === "face") selection = { kind, settings: { style: faceStyle, seed, chunkSeconds }, model: selectedModel };
else if (kind === "color") selection = { kind, settings: { detail: detailOn ? detail : null, tone: toneOn ? tone : null } };
else if (kind === "mask") selection = { kind, hatFix, margin, target: String(form.get("target")) };
@@ -607,7 +593,7 @@ export function Openjutsu() {
const response = await fetch("/api/jobs", { method: "POST", body: form });
const body = await response.json();
if (!response.ok) throw new Error(body.error ?? "Could not start the job.");
- const next: Run = { id: body.id, kind, margin, clips: Math.max(clips, 1), clipSeconds: kind === "face" ? chunkSeconds : kind === "control" ? CONTROL_MAX_SECONDS : clipSeconds, startedAt: Date.now(), frames };
+ const next: Run = { id: body.id, kind, margin, clips: Math.max(clips, 1), clipSeconds: kind === "face" ? chunkSeconds : clipSeconds, startedAt: Date.now(), frames };
writeRun(next);
setHistory((list) => [next, ...list.filter((p) => p.id !== next.id)].slice(0, HISTORY_LENGTH));
setJob(body);
@@ -840,10 +826,10 @@ export function Openjutsu() {
<span>{edit.what} <small>{editMode === "add" ? "Describe the placement, for example on the table in front of the person." : "What SAM tracks, in English."}{(editMode === "change" || editMode === "remove") && " Separate several with commas."}</small></span>
<input value={editTarget} maxLength={100} placeholder={edit.target} onChange={(event) => setEditTarget(event.target.value)} />
</label>
- {!pictureOnly && <label>
+ <label>
<span>{edit.toHint}</span>
<input value={changeTo} maxLength={300} placeholder={`e.g. ${edit.to}`} onChange={(event) => setChangeTo(event.target.value)} />
- </label>}
+ </label>
{editMode === "add" && <label>
<span>What happens with it <small>Optional. Leave empty for a still object.</small></span>
<input value={action} maxLength={300} placeholder="e.g. she dances the same routine in sync with the others" onChange={(event) => setAction(event.target.value)} />
@@ -856,7 +842,7 @@ export function Openjutsu() {
) : (
<DropZone accept="image/*" onFiles={chooseLook} className="add">
<strong>+</strong>
- <span>{fromPicture ? "The picture to copy" : controlNet ? "The person in their place" : editMode === "add" ? "Picture of the object (optional)" : editMode === "background" ? "Picture of the place (optional)" : "Picture of the new look (optional)"}</span>
+ <span>{editMode === "add" ? "Picture of the object (optional)" : editMode === "background" ? "Picture of the place (optional)" : "Picture of the new look (optional)"}</span>
</DropZone>
))}
<p className="hint">
@@ -864,8 +850,6 @@ export function Openjutsu() {
{editMode === "add" && "A new object is generated inside your placement area while the scene continues. Keep the supporting surface visible and describe where the object rests."}
{editMode === "remove" && "What SAM tracks is erased and filled with what is around it. The camera, people and sound stay."}
{editMode === "background" && "Everything except what SAM tracks becomes the new place; the camera movement stays."}
- {fromPicture && "The person, background and lighting come from the picture. The video guides the motion and timing; its soundtrack stays. For relighting, upload an edited frame from the selected video section."}
- {controlNet && `The person and the place both come from the picture. A depth map of the person in the video keeps their framing, camera and motion; the sound stays. Everyone else in the video is replaced by the picture's setting. Runs as one clip of up to ${CONTROL_MAX_SECONDS} s.`}
</p>
{editMode === "change" && video && canvas && <details>
<summary>Paint edit area (optional)</summary>
@@ -1001,8 +985,8 @@ export function Openjutsu() {
<input type="checkbox" checked={hatFix} onChange={(event) => setHatFix(event.target.checked)} />
<i aria-hidden />
</label>}
- {(mode === "simple" || mode === "advanced" || pictureOnly) && <label className="setting toggle">
- <span>No lip-sync <small>For songs the people in the video do not sing or talk along to, so the new {mode === "advanced" ? "people" : "person"} do{mode === "advanced" ? "" : "es"} not lip-sync them{controlNet ? "" : " (about 30 s more per clip)"}.</small></span>
+ {(mode === "simple" || mode === "advanced") && <label className="setting toggle">
+ <span>No lip-sync <small>For songs the people in the video do not sing or talk along to, so the new {mode === "advanced" ? "people" : "person"} do{mode === "advanced" ? "" : "es"} not lip-sync them (about 30 s more per clip).</small></span>
<input type="checkbox" checked={noLipSync} onChange={(event) => setNoLipSync(event.target.checked)} />
<i aria-hidden />
</label>}
@@ -1013,7 +997,7 @@ export function Openjutsu() {
<i aria-hidden />
</label>
)}
- {mode !== "post" && !pictureOnly && <div className="setting">
+ {mode !== "post" && <div className="setting">
<span>Mask margin <small>{mode === "object"
? editMode === "background" ? "How far the new place reaches into the edge of what stays." : "Room around it for a bigger new shape and its shadow."
: <>Room around the person for a differently built character.{mode === "simple" ? " Check mask shows it." : ""}</>}</small></span>
@@ -1033,13 +1017,13 @@ export function Openjutsu() {
<span>Chunk length <small>{chunkSeconds.toFixed(1)} s of face per H3 generation; chunks overlap by 1 s and are crossfaded.</small></span>
<input type="range" min={CHUNK_SECONDS.min} max={CHUNK_SECONDS.max} step={0.5} value={chunkSeconds} onChange={(event) => setChunkSeconds(Number(event.target.value))} />
</div>
- ) : !controlNet && (
+ ) : (
<div className="setting">
<span>Clip length <small>{clipSeconds.toFixed(1)} s per H3 generation</small></span>
<input type="range" min={CLIP_SECONDS.min} max={CLIP_SECONDS.max} step={0.5} value={clipSeconds} onChange={(event) => setClipSeconds(Number(event.target.value))} />
</div>
)}
- {mode !== "post" && !controlNet && <div className="setting">
+ {mode !== "post" && <div className="setting">
<span>Motion reference <small>Size of the grey figures H3 copies the motion from. Bigger follows the source poses a bit more closely; sampling takes about a third longer at 768 px and about 2.4× as long at 1376 px.</small></span>
<div className="segmented">
{MOTION_SIZES.map((m) => (
@@ -1098,7 +1082,6 @@ export function Openjutsu() {
: clips && ready
? mode === "advanced" && montage ? `${clips > 1 ? "About " : ""}${clips} clip${clips > 1 ? "s" : ""} or more at ${canvas ? `${canvas[0]}×${canvas[1]}` : resolution}; Find people (about 3 minutes per 18 s) lets you check the picks first.`
: faceFix ? `Up to ${clips} chunk${clips > 1 ? "s" : ""} of face`
- : controlNet ? `One clip at ${canvas ? `${canvas[0]}×${canvas[1]}` : resolution}${(part.length || video!.duration) > CONTROL_MAX_SECONDS ? `: only the first ${CONTROL_MAX_SECONDS} s of this part are used` : ""}`
: `${clips > 1 ? "About " : ""}${clips} clip${clips > 1 ? "s" : ""} at ${canvas ? `${canvas[0]}×${canvas[1]}` : resolution}`
: mode === "object" && editMode === "add" && video && !placement.length ? "Draw where the new object should go."
: `Add a video and ${mode === "object" ? editMode === "add" ? "describe the object to add" : "say what it becomes" : faceFix ? "the photo of the person" : mode === "simple" ? "a character photo" : "character photos"}.`}</small>
diff --git a/lib/comfy.ts b/lib/comfy.ts
index c49ee0b..9c80ff1 100644
--- a/lib/comfy.ts
+++ b/lib/comfy.ts
@@ -10,7 +10,6 @@ import { applyModel, type ModelSettings } from "./model.ts";
import { OBJECT_PICTURE, OBJECT_VIDEO, pruneFit, pruneObject, type ObjectSettings, type PlacementKeyframe } from "./object.ts";
import { FACE_PICTURE, FACE_VIDEO, pruneFace, type FaceSettings } from "./face.ts";
import { COLOR_VIDEO, pruneColor, type ColorSettings } from "./color.ts";
-import { CONTROL_PICTURE, CONTROL_VIDEO, pruneControl, setControlSection, type ControlSettings } from "./control.ts";
import { PEOPLE_NODES, peopleRoles, pruneMontage, type Review } from "./montage.ts";
export type PublicJob = {
@@ -45,7 +44,7 @@ export function rawJobEvents(job: Job, signal: AbortSignal) {
return low().getJobEvents(job.urls.events || job.id, { signal });
}
-export const JOB_KINDS = ["simple", "mask", "advanced", "advanced_maxq", "object", "fit", "control", "face", "color", "people", "montage"] as const;
+export const JOB_KINDS = ["simple", "mask", "advanced", "advanced_maxq", "object", "fit", "face", "color", "people", "montage"] as const;
export type JobKind = (typeof JOB_KINDS)[number];
export async function loadGraph(kind: JobKind): Promise<ApiGraph> {
@@ -195,17 +194,6 @@ export async function submitFit(video: StoredFile, section: Section, placement:
return submit(comfy, workflow);
}
-export async function submitControl(video: StoredFile, section: Section, picture: StoredFile, settings: ControlSettings, model: ModelSettings) {
- const comfy = client();
- const graph = pruneControl(await loadGraph("control"), settings);
- applyModel(graph, model);
- setControlSection(graph, section);
- const workflow = comfy.workflows.fromJson(graph);
- workflow.setInput(nodeByTitle(graph, CONTROL_VIDEO), "video", upload(comfy, video));
- workflow.setInput(nodeByTitle(graph, CONTROL_PICTURE), "image", upload(comfy, picture));
- return submit(comfy, workflow);
-}
-
export async function submitFace(video: StoredFile, section: Section, picture: StoredFile, settings: FaceSettings, model: ModelSettings) {
const comfy = client();
const graph = pruneFace(await loadGraph("face"), settings);
diff --git a/lib/control.ts b/lib/control.ts
deleted file mode 100644
index 7ea692a..0000000
--- a/lib/control.ts
+++ /dev/null
@@ -1,33 +0,0 @@
-// ControlNet picture swap: the whole frame, the main person and the setting, comes from the picture. A depth video of only
-// the main person (SAM 3.1 + Depth Anything V2) holds the framing, camera and poses through the H3 Fun ControlNet Union 2.0.
-// workflows/control.api.json is exported (workflows/build/oneclick_control_build.js) with No lip-sync on; nodes are found by title.
-import { nodeByTitle } from "./advanced.ts";
-import { dropNoLipSync, setResolution, setSection, type ApiGraph, type Resolution, type Section } from "./simple.ts";
-
-export type ControlSettings = {
- target: string; // what SAM tracks; the biggest one becomes the person in the picture
- resolution: Resolution;
- noLipSync: boolean;
- seed: number;
-};
-
-// One H3 clip, the longest H3 is trained on: a continuation clip loses the depth framing where its new frames start.
-export const CONTROL_MAX_SECONDS = 15;
-
-export const CONTROL_VIDEO = "Source video (people get replaced)";
-export const CONTROL_PICTURE = "Picture 1";
-const PLAN = "Swap Plan (roles, seed, size)";
-
-export function pruneControl(source: ApiGraph, settings: ControlSettings): ApiGraph {
- const graph: ApiGraph = structuredClone(source);
- if (!settings.noLipSync) dropNoLipSync(graph);
- graph[nodeByTitle(graph, "SAM: what to replace")].inputs.text = settings.target;
- Object.assign(graph[nodeByTitle(graph, PLAN)].inputs, { seed: settings.seed, clip_seconds: CONTROL_MAX_SECONDS, roles: "" });
- setResolution(graph, nodeByTitle(graph, PLAN), nodeByTitle(graph, CONTROL_VIDEO), settings.resolution);
- return graph;
-}
-
-// The whole video, or a longer part, is cut to its first CONTROL_MAX_SECONDS.
-export function setControlSection(graph: ApiGraph, section: Section) {
- setSection(graph, nodeByTitle(graph, CONTROL_VIDEO), { start: section.start, length: Math.min(section.length || CONTROL_MAX_SECONDS, CONTROL_MAX_SECONDS) });
-}
diff --git a/lib/model.ts b/lib/model.ts
index 75d8a6a..1ad35c8 100644
--- a/lib/model.ts
+++ b/lib/model.ts
@@ -18,7 +18,7 @@ export const STRENGTH = { min: -4, max: 4 };
// Mutates graph: the steps (on the Swap Plan, or the Schedule in Fix faces, which has no plan), the chosen diffusion model,
// then the LoRAs in order into whatever read the old LoRA node.
export function applyModel(graph: ApiGraph, settings: ModelSettings) {
- const plans = Object.values(graph).filter((node) => ["H3OneClickSwapPlan", "H3AdvancedSwapPlan", "H3PictureSwapPlan", "H3SceneSwapPlan", "H3MontagePlan", "H3ObjectSwapPlan"].includes(node.class_type));
+ const plans = Object.values(graph).filter((node) => ["H3OneClickSwapPlan", "H3AdvancedSwapPlan", "H3MontagePlan", "H3ObjectSwapPlan"].includes(node.class_type));
if (plans.length > 1) throw new Error(`Expected one Swap Plan node in the workflow, found ${plans.length}.`);
(plans[0] ?? graph[nodeByTitle(graph, "Schedule")]).inputs.steps = settings.steps;
const unet = nodeByTitle(graph, "H3 Diffusion Model");
diff --git a/lib/setup.ts b/lib/setup.ts
index 7ab5b7e..bf354d4 100644
--- a/lib/setup.ts
+++ b/lib/setup.ts
@@ -1,7 +1,6 @@
import { pruneAdvanced, type AdvancedSettings } from "./advanced.ts";
import { comfyUi } from "./comfy.ts";
import { pruneColor, type ColorSettings } from "./color.ts";
-import { pruneControl, type ControlSettings } from "./control.ts";
import { pruneFace, type FaceSettings } from "./face.ts";
import { applyModel, type ModelSettings } from "./model.ts";
import { PEOPLE_NODES, pruneMontage, type Review } from "./montage.ts";
@@ -12,7 +11,6 @@ export type SetupRequest =
| { kind: "simple"; settings: SimpleSettings; model: ModelSettings }
| { kind: "advanced" | "advanced_maxq"; settings: AdvancedSettings; pictures: number; model: ModelSettings }
| { kind: "object"; settings: ObjectSettings; picture: boolean; model: ModelSettings }
- | { kind: "control"; settings: ControlSettings; model: ModelSettings }
| { kind: "face"; settings: FaceSettings; model: ModelSettings }
| { kind: "color"; settings: ColorSettings }
| { kind: "mask"; hatFix: boolean; margin: Margin; target: string }
@@ -32,7 +30,7 @@ export type NodeRegistry = Record<string, NodeInfo>;
const PACKS = [
{ name: "ComfyUI-MiniMaxH3-Context-Loop (0.7.3+)", url: "https://github.com/ethanfel/ComfyUI-MiniMaxH3-Context-Loop", nodes: ["MiniMaxH3GenerationProfile", "MiniMaxH3SourceTimeline", "MiniMaxH3ChainPlanModern", "MiniMaxH3ChainLoopStart", "MiniMaxH3ChainCurrent", "MiniMaxH3ChainContext", "MiniMaxH3ChainSegmentSave", "MiniMaxH3ChainReview", "MiniMaxH3ChainLoopEnd", "MiniMaxH3ChainAssemble", "MiniMaxH3ContexLoopSourceSlice", "MiniMaxH3ContexLoopMaskSlice", "MiniMaxH3ContexMaskedTarget", "MiniMaxH3ContexMaskGridPreview", "MiniMaxH3LoopTrim"] },
- { name: "comfyui-h3-oneclick-swap", url: "https://github.com/D3voz/comfyui-h3-oneclick-swap", nodes: ["H3OneClickSwapPlan", "H3AdvancedSwapPlan", "H3PictureSwapPlan", "H3SceneSwapPlan", "H3ObjectSwapPlan", "H3ObjectPlacementMask", "H3AddFitCheck", "H3AddMovingArea", "H3PaintedEditArea", "H3SoundtrackOrSilence", "H3PeopleMask", "H3GreyMannequins", "H3ClipShots", "H3FillShots", "H3TrackCrops", "H3TrackRoles", "H3RoleMask", "H3SubjectPrompt", "H3MontagePeople", "H3MontageTracks", "H3MontagePicks", "H3MontagePlan", "H3MontagePass", "H3FaceChunks", "H3FaceJoin"] },
+ { name: "comfyui-h3-oneclick-swap", url: "https://github.com/D3voz/comfyui-h3-oneclick-swap", nodes: ["H3OneClickSwapPlan", "H3AdvancedSwapPlan", "H3ObjectSwapPlan", "H3ObjectPlacementMask", "H3AddFitCheck", "H3AddMovingArea", "H3PaintedEditArea", "H3SoundtrackOrSilence", "H3PeopleMask", "H3GreyMannequins", "H3ClipShots", "H3FillShots", "H3TrackCrops", "H3TrackRoles", "H3RoleMask", "H3SubjectPrompt", "H3MontagePeople", "H3MontageTracks", "H3MontagePicks", "H3MontagePlan", "H3MontagePass", "H3FaceChunks", "H3FaceJoin"] },
{ name: "ComfyUI-Fantastic-MiniMaxH3-PromptBuilder", url: "https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder", nodes: ["MiniMaxH3FantasticVideoEditLatent"] },
{ name: "ComfyUI-KJNodes", url: "https://github.com/kijai/ComfyUI-KJNodes", nodes: ["GrowMaskWithBlur", "ImageAndMaskPreview"] },
{ name: "ComfyUI-VideoHelperSuite", url: "https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite", nodes: ["VHS_LoadVideo", "VHS_SelectEveryNthImage", "VHS_VideoCombine"] },
@@ -40,7 +38,6 @@ const PACKS = [
{ name: "ComfyUI-Impact-Pack", url: "https://github.com/ltdrdata/ComfyUI-Impact-Pack", nodes: ["SAMLoader"] },
{ name: "comfyui-image-detail-contrast", url: "https://github.com/D3voz/comfyui-image-detail-contrast", nodes: ["ImageDetailContrast"] },
{ name: "Orion4D_FXMax", url: "https://github.com/orion4d/Orion4D_FXMax", nodes: ["Orion4D_FXMax_DCTLToneMapper"] },
- { name: "comfyui_controlnet_aux", url: "https://github.com/Fannovel16/comfyui_controlnet_aux", nodes: ["DepthAnythingV2Preprocessor"] },
];
const CORE = { name: "ComfyUI (update required)", url: "https://github.com/Comfy-Org/ComfyUI", nodes: [] as string[] };
const PACK_HINT = "Already installed? ComfyUI may be too old for the pack, or the pack failed to load. Update ComfyUI, restart it and check its console for import errors.";
@@ -51,7 +48,6 @@ const MODEL_INPUTS: Record<string, Record<string, string>> = {
VAELoader: { vae_name: "vae" },
CheckpointLoaderSimple: { ckpt_name: "checkpoints" },
SAMLoader: { model_name: "sams" },
- ModelPatchLoader: { name: "model_patches" },
H3FaceTrackCrop: { detector: "ultralytics", fallback_detector: "ultralytics" },
};
@@ -59,7 +55,6 @@ export function setupGraph(source: ApiGraph, selection: SetupRequest): ApiGraph
let graph: ApiGraph;
if (selection.kind === "simple") graph = pruneSimple(source, selection.settings);
else if (selection.kind === "object") graph = pruneObject(source, selection.picture, selection.settings);
- else if (selection.kind === "control") graph = pruneControl(source, selection.settings);
else if (selection.kind === "face") graph = pruneFace(source, selection.settings);
else if (selection.kind === "color") graph = pruneColor(source, selection.settings);
else if (selection.kind === "mask") graph = pruneMask(source, selection.hatFix, selection.margin, selection.target);
diff --git a/lib/simple.ts b/lib/simple.ts
index 304a516..6a9bcb0 100644
--- a/lib/simple.ts
+++ b/lib/simple.ts
@@ -61,7 +61,6 @@ export function setSection(graph: ApiGraph, video: string, section: Section) {
export const DEFAULT_TARGET = "person";
export type SimpleSettings = {
- sceneFromPicture?: boolean;
role: string; // who the character replaces, "" to let the describer name the main person
resolution: Resolution;
motion: MotionSize;
@@ -160,16 +159,10 @@ export function suffixRunName(graph: ApiGraph, id: string, suffix: string, title
}
export function pruneSimple(source: ApiGraph, settings: SimpleSettings): ApiGraph {
- const graph = maskSettings(source, settings.sceneFromPicture ? false : settings.hatFix, settings.margin, settings.target, NODES);
+ const graph = maskSettings(source, settings.hatFix, settings.margin, settings.target, NODES);
if (!settings.noLipSync) dropNoLipSync(graph);
setMotionSize(graph, settings.motion);
const plan = graph[NODES.plan].inputs;
- if (settings.sceneFromPicture) {
- graph[NODES.plan].class_type = "H3SceneSwapPlan";
- graph[NODES.growMask] = { class_type: "SolidMask", inputs: { value: 1, width: 64, height: 64 }, _meta: { title: "Edit the whole frame" } };
- const mannequin = Object.values(graph).find((node) => node.class_type === "H3GreyMannequins")!;
- mannequin.inputs.neutral_background = true;
- }
setResolution(graph, NODES.plan, NODES.video, settings.resolution);
plan.seed = settings.seed;
plan.clip_seconds = settings.clipSeconds;
diff --git a/test/control.test.ts b/test/control.test.ts
deleted file mode 100644
index bd91f8c..0000000
--- a/test/control.test.ts
+++ /dev/null
@@ -1,54 +0,0 @@
-import assert from "node:assert/strict";
-import { readFileSync } from "node:fs";
-import test from "node:test";
-import { nodeByTitle } from "../lib/advanced.ts";
-import { CONTROL_MAX_SECONDS, CONTROL_PICTURE, CONTROL_VIDEO, pruneControl, setControlSection, type ControlSettings } from "../lib/control.ts";
-import { applyModel } from "../lib/model.ts";
-import { isLink, type ApiGraph } from "../lib/simple.ts";
-
-const graph: ApiGraph = JSON.parse(readFileSync(new URL("../workflows/control.api.json", import.meta.url), "utf8"));
-const settings = (noLipSync: boolean): ControlSettings => ({ target: "dancer", resolution: "480p", noLipSync, seed: 3 });
-const byClass = (g: ApiGraph, type: string) => Object.keys(g).filter((id) => g[id].class_type === type);
-const dangling = (g: ApiGraph) => Object.values(g).flatMap((node) => Object.values(node.inputs)).filter((v) => isLink(v) && !g[v[0]]);
-
-test("the export regenerates the whole frame from the picture, held by the person depth", () => {
- assert.equal(graph[nodeByTitle(graph, CONTROL_VIDEO)].class_type, "VHS_LoadVideo");
- assert.equal(graph[nodeByTitle(graph, CONTROL_PICTURE)].class_type, "LoadImage");
- const [fun] = byClass(graph, "MiniMaxH3FunControlNetApply");
- assert.deepEqual(graph[fun].inputs.control_video, [nodeByTitle(graph, "Depth of the main person only (control video)"), 0]);
- for (const id of byClass(graph, "BasicGuider").concat(byClass(graph, "BasicScheduler"))) assert.deepEqual(graph[id].inputs.model, [fun, 0]);
- const whole = nodeByTitle(graph, "Whole frame (regenerate everything)");
- for (const type of ["MiniMaxH3FantasticVideoEditLatent", "MiniMaxH3ContexMaskedTarget"]) assert.deepEqual(graph[byClass(graph, type)[0]].inputs.mask, [whole, 0]);
- const ref = graph[byClass(graph, "MiniMaxH3ReferenceToVideo")[0]].inputs;
- assert.ok(ref["ref_images.ref_image_0"] && !ref["ref_videos.ref_video_0"], "the picture only, no source video reference");
-});
-
-test("settings reach the plan and SAM; No lip-sync off rewires the audio and drops the prompt switch", () => {
- const pruned = pruneControl(graph, settings(false));
- const plan = pruned[nodeByTitle(pruned, "Swap Plan (roles, seed, size)")].inputs;
- assert.deepEqual([plan.resolution, plan.seed, plan.clip_seconds, plan.roles, plan.no_lip_sync], ["480p", 3, CONTROL_MAX_SECONDS, "", undefined]);
- assert.equal(pruned[nodeByTitle(pruned, "SAM: what to replace")].inputs.text, "dancer");
- assert.equal(byClass(pruned, "AudioAdjustVolume").length, 0);
- assert.deepEqual(dangling(pruned), []);
- assert.equal(byClass(pruneControl(graph, settings(true)), "AudioAdjustVolume").length, 2);
- assert.ok(byClass(graph, "AudioAdjustVolume").length === 2, "the source graph is not modified");
-});
-
-test("the video is cut to one clip of at most 15 s", () => {
- const frames = (section: { start: number; length: number }) => {
- const pruned = pruneControl(graph, settings(true));
- setControlSection(pruned, section);
- const loader = pruned[nodeByTitle(pruned, CONTROL_VIDEO)].inputs;
- return [loader.skip_first_frames, loader.frame_load_cap];
- };
- assert.deepEqual(frames({ start: 0, length: 0 }), [0, 360]);
- assert.deepEqual(frames({ start: 2, length: 20 }), [48, 360]);
- assert.deepEqual(frames({ start: 1, length: 8 }), [24, 192]);
-});
-
-test("the model settings apply to the picture plan", () => {
- const pruned = pruneControl(graph, settings(true));
- applyModel(pruned, { model: "other.safetensors", loras: [], steps: 9 });
- assert.equal(pruned[byClass(pruned, "H3PictureSwapPlan")[0]].inputs.steps, 9);
- assert.deepEqual(dangling(pruned), []);
-});
diff --git a/test/picture.test.ts b/test/picture.test.ts
deleted file mode 100644
index daea0d7..0000000
--- a/test/picture.test.ts
+++ /dev/null
@@ -1,32 +0,0 @@
-import assert from "node:assert/strict";
-import { readFileSync } from "node:fs";
-import test from "node:test";
-import { applyModel, DEFAULT_MODEL } from "../lib/model.ts";
-import { setupGraph } from "../lib/setup.ts";
-import { NODES, isLink, pruneSimple, type ApiGraph, type SimpleSettings } from "../lib/simple.ts";
-
-const graph: ApiGraph = JSON.parse(readFileSync(new URL("../workflows/simple.api.json", import.meta.url), "utf8"));
-const settings: SimpleSettings = { sceneFromPicture: true, role: "the dancer", resolution: "768p", motion: 512, hatFix: true, noLipSync: true, margin: "normal", target: "person", seed: 0, clipSeconds: 6 };
-
-test("From picture edits the whole frame using the picture and a neutral motion guide, without ControlNet", () => {
- const result = pruneSimple(graph, settings);
- applyModel(result, DEFAULT_MODEL);
- assert.equal(result[NODES.plan].class_type, "H3SceneSwapPlan");
- assert.deepEqual(result[NODES.growMask].inputs, { value: 1, width: 64, height: 64 });
- const ref = Object.values(result).find((n) => n.class_type === "MiniMaxH3ReferenceToVideo")!;
- assert.deepEqual(ref.inputs["ref_images.ref_image_0"], [NODES.picture, 0]);
- const motion = ref.inputs["ref_videos.ref_video_0"] as [string, number];
- assert.equal(result[motion[0]].inputs.neutral_background, true);
- assert.ok(!Object.values(result).some((n) => /ControlNet|DepthAnything/.test(n.class_type)));
- for (const node of Object.values(result)) for (const value of Object.values(node.inputs)) {
- if (isLink(value)) assert.ok(result[value[0]], `missing ${value[0]}`);
- }
- assert.deepEqual(setupGraph(graph, { kind: "simple", settings, model: DEFAULT_MODEL }), result);
-});
-
-test("Simple motion transfer retains its original mask and surroundings", () => {
- const result = pruneSimple(graph, { ...settings, sceneFromPicture: false });
- assert.equal(result[NODES.plan].class_type, "H3OneClickSwapPlan");
- assert.equal(result[NODES.growMask].class_type, "GrowMaskWithBlur");
- assert.ok(!Object.values(result).some((n) => n.inputs.neutral_background));
-});
diff --git a/test/setup.test.ts b/test/setup.test.ts
index 5a93aa8..b93cc64 100644
--- a/test/setup.test.ts
+++ b/test/setup.test.ts
@@ -55,7 +55,7 @@ test("missing pack nodes also point to an old ComfyUI or a pack that failed to l
});
test("all custom node types in the exports have a pack mapping", () => {
- const custom = /^(H3|MiniMaxH3(?!ReferenceToVideo|SigmaShift|FunControlNetApply)|VHS_|GrowMaskWithBlur$|ImageAndMaskPreview$|ImageDetailContrast$|Orion4D_|SAMLoader$)/;
+ const custom = /^(H3|MiniMaxH3(?!ReferenceToVideo|SigmaShift)|VHS_|GrowMaskWithBlur$|ImageAndMaskPreview$|ImageDetailContrast$|Orion4D_|SAMLoader$)/;
const graphs = JOB_KINDS.map(load);
const missing = inspectSetup(Object.fromEntries(graphs.flatMap((g, i) => Object.entries(g).map(([id, node]) => [`${i}_${id}`, node]))), {});
const core = missing.packs.find((p) => p.url === "https://github.com/Comfy-Org/ComfyUI")!;
diff --git a/workflows/build/oneclick_control_build.js b/workflows/build/oneclick_control_build.js
deleted file mode 100644
index eb0a261..0000000
--- a/workflows/build/oneclick_control_build.js
+++ /dev/null
@@ -1,278 +0,0 @@
-const g = app.graph;
-g.clear();
-const made = [];
-let curGroup = null;
-function add(type, title, pos, widgets = {}, mode = 0) {
- const n = LiteGraph.createNode(type);
- if (!n) throw new Error("no node " + type);
- if (title) n.title = title;
- n.pos = pos;
- g.add(n);
- for (const [k, v] of Object.entries(widgets)) {
- const w = /^\d+$/.test(k) ? n.widgets[+k] : n.widgets.find(w => w.name === k);
- if (!w) throw new Error(type + " has no widget " + k);
- w.value = v;
- w.callback?.(v, app.canvas, n);
- }
- n.mode = mode;
- made.push(n);
- curGroup?.members.push(n);
- return n;
-}
-function link(a, out, b, inp) {
- const o = typeof out === "number" ? out : a.outputs.findIndex(x => x.name === out);
- const i = b.inputs.findIndex(x => x.name === inp);
- if (o < 0 || i < 0) throw new Error(`link ${a.type}.${out} -> ${b.type}.${inp}`);
- a.connect(o, b, i);
-}
-function group(title, x, y, w, h, color) {
- const gr = new LiteGraph.LGraphGroup(title);
- gr.pos = [x, y]; gr.size = [w, h];
- if (color) gr.color = color;
- g.add(gr);
- gr.members = [];
- curGroup = gr;
- return gr;
-}
-
-const PIC = "Describe the person in this picture for a casting sheet, in one sentence of about 40 words: apparent age and gender, hair, facial hair, skin tone, build, and their complete outfit including footwear. No preamble, no background.";
-const PEOPLE = "These are frames from one video, in time order. List the people who appear most, up to four, the person with the most screen time first. Write one numbered line per person: first their role in the video in two to four words, without gender, that stays true in every shot, then \" - \" and how they look, like \"1. the guitarist - tall man with a grey beard in a denim jacket\". Each person appears once. Output only the list.";
-const GEN_OFF = {sampling_mode: "off"};
-
-// ---------------- 1 · UPLOAD ----------------
-const note = add("MarkdownNote", "START HERE", [40, 60], {0:
-`# One-click picture swap with ControlNet (H3)
-
-1. **Picture** - drop a photo of your character in the place they should be.
-2. **Source video** - upload the video. Its main person (the one on screen most) becomes the person in the picture, and the whole setting becomes the picture's; the framing, camera movement, motion and soundtrack are kept.
-3. Press **Run**.
-
-That's it. The graph then:
-- describes your picture and the people in the video, and writes a shot list per clip (Qwen3-VL 8B, local),
-- splits the video into ~6 s clips that overlap by 39 frames,
-- tracks the main person with SAM 3.1 and makes a depth video of only them (Depth Anything V2),
-- regenerates the whole frame from the picture, held to that depth by the H3 Fun ControlNet Union 2.0 (DMAD, 6 steps),
-- joins the clips with the original audio.
-
-The final MP4 lands in **output/h3_oneclick_swap/**. Each clip is saved as it finishes, so a crash resumes where it stopped when you press Run again with the same inputs.
-
-Everyone else in the video is dropped: the picture's setting and its people replace the whole background.
-**New take:** change **seed**. **Faster test:** set **resolution** to 480p.
-**Not singing along:** if the soundtrack has vocals but the person does not sing or talk, turn on **OPTIONAL - No lip-sync** so the new person does not lip-sync the song.`});
-note.size = [500, 860];
-
-const pics = [], picTexts = [];
-const files = [""];
-for (let i = 0; i < 1; i++) {
- const x = 580 + i * 380;
- const title = i === 0 ? "Picture 1" : `OPTIONAL - Picture ${i + 1}`;
- group(title, x - 20, 0, 360, 960, i === 0 ? "#3f789e" : "#88a");
- const load = add("LoadImage", `Picture ${i + 1}`, [x, 60], {image: files[i]}, i < 2 ? 0 : 4);
- load.size = [320, 420];
- const scale = add("ImageScaleToMaxDimension", "Shrink for the describer", [x, 520], {upscale_method: "area", largest_size: 768}, i < 2 ? 0 : 4);
- const gen = add("TextGenerate", `Describe picture ${i + 1}`, [x, 640], {prompt: PIC, max_length: 120, ...GEN_OFF}, i < 2 ? 0 : 4);
- gen.size = [320, 260];
- link(load, 0, scale, "image");
- link(scale, 0, gen, "image");
- pics.push(load); picTexts.push(gen);
-}
-
-const vx = 2100;
-group("Source video", vx - 20, 0, 420, 1580, "#3f789e");
-const video = add("VHS_LoadVideo", "Source video (people get replaced)", [vx, 60], {video: "", force_rate: 24, format: "None"});
-video.size = [380, 560];
-const soundtrack = add("TrimAudioDuration", "Soundtrack (whole video)", [vx, 650], {start_index: 0, duration: 36000});
-link(video, "audio", soundtrack, "audio");
-const nth = add("VHS_SelectEveryNthImage", "One frame every 2 s", [vx, 700], {select_every_nth: 48, skip_first_images: 24});
-const first20 = add("ImageFromBatch", "At most 20 frames", [vx, 800], {batch_index: 0, length: 20});
-const vscale = add("ImageScaleToMaxDimension", "Shrink for the describer", [vx, 920], {upscale_method: "area", largest_size: 640});
-const people = add("TextGenerate", "Who is in the video", [vx, 1020], {prompt: PEOPLE, max_length: 160, ...GEN_OFF});
-people.size = [380, 240];
-link(video, "IMAGE", nth, "images");
-link(nth, 0, first20, "image");
-link(first20, 0, vscale, "image");
-link(vscale, 0, people, "image");
-
-// ---------------- 2 · SETTINGS ----------------
-const sx = 2560;
-group("SETTINGS (optional)", sx - 20, 0, 460, 1300, "#a1309b");
-const swapPlan = add("H3PictureSwapPlan", "Swap Plan (roles, seed, size)", [sx, 60], {seed: 0, control_after_generate: "fixed"});
-swapPlan.size = [420, 420];
-const bypass = add("Fast Groups Bypasser (rgthree)", "ON / OFF SWITCHES", [sx, 560]);
-Object.assign(bypass.properties, {matchTitle: "OPTIONAL", sort: "position"});
-bypass.size = [420, 160];
-const qwen = add("CLIPLoader", "Describer: Qwen3-VL 8B", [sx, 800], {clip_name: "qwen3vl_8b_int8_convrot.safetensors", type: "stable_diffusion"});
-qwen.size = [420, 110];
-const qwenFree = add("Note", "", [sx, 960], {0: "The describer reads your pictures and the video once (about 40 s), then writes a shot list for every clip (about 20 s each)."});
-qwenFree.size = [420, 80];
-
-link(video, "IMAGE", swapPlan, "source_frames");
-pics.forEach((p, i) => link(p, 0, swapPlan, `picture_${i + 1}`));
-picTexts.forEach((t, i) => link(t, 0, swapPlan, `picture_${i + 1}_text`));
-link(people, 0, swapPlan, "video_people");
-for (const t of [...picTexts, people]) link(qwen, 0, t, "clip");
-
-// ---------------- 3 · ENGINE ----------------
-const ex = 3100;
-group("ENGINE - no need to touch", ex - 20, 0, 2900, 1900, "#444");
-const unet = add("UNETLoader", "H3 Diffusion Model", [ex, 60], {unet_name: "minimax_h3_ref2va_pruned_int8_convrot.safetensors"});
-const lora = add("LoraLoaderModelOnly", "DMAD LoRA", [ex, 200], {lora_name: "minimax_h3_DMAD_4step_full_lora_avg_rank_39_bf16.safetensors", strength_model: 1.0});
-const attn = add("ModelAttentionBackend", "Comfy Kitchen Attention", [ex, 340], {0: "comfy kitchen attention"});
-const shift = add("MiniMaxH3SigmaShift", "Sigma Shift", [ex, 440], {0: 12, 1: 2});
-const te = add("CLIPLoader", "H3 Text Encoder", [ex, 560], {clip_name: "qwen3vl_32b_minimax_h3_int4_convrot.safetensors", type: "minimax"});
-const vae = add("VAELoader", "Video VAE", [ex, 700], {vae_name: "minimax_h3_video_vae_int8_convrot.safetensors"});
-const avae = add("VAELoader", "Audio VAE", [ex, 800], {vae_name: "minimax_h3_audio_vae_fp32.safetensors"});
-const sampler = add("KSamplerSelect", "Sampler", [ex, 900], {sampler_name: "euler"});
-const sam = add("CheckpointLoaderSimple", "SAM 3.1", [ex, 1000], {ckpt_name: "sam3.1_multiplex_fp16.safetensors"});
-const samText = add("CLIPTextEncode", "SAM: what to replace", [ex, 1140], {text: "person"});
-samText.size = [320, 100];
-link(unet, 0, lora, "model");
-link(lora, 0, attn, "model");
-link(attn, 0, shift, "model");
-link(sam, 1, samText, "clip");
-
-const lx = ex + 400;
-const profile = add("MiniMaxH3GenerationProfile", "Generation Profile", [lx, 60], {scene_continuity: "Hard picture + protected audio", audio_profile: "Generate audio"});
-const plan = add("MiniMaxH3ChainPlanModern", "Production Plan (filled by Swap Plan)", [lx, 220], {crop: "center", default_steps: 6, segment_crf: 16, run_name: ""});
-plan.size = [360, 500];
-const timeline = add("MiniMaxH3SourceTimeline", "Source soundtrack", [lx, 760]);
-const start = add("MiniMaxH3ChainLoopStart", "Loop Start", [lx, 960]);
-const cur = add("MiniMaxH3ChainCurrent", "Current Clip", [lx, 1160]);
-link(profile, 0, plan, "chain_policy");
-link(swapPlan, "plan_json", plan, "plan_json_input");
-link(swapPlan, "width", plan, "width");
-link(swapPlan, "height", plan, "height");
-link(swapPlan, "run_name", plan, "run_name");
-link(soundtrack, 0, timeline, "source_audio");
-link(plan, 0, start, "plan");
-link(timeline, 0, start, "source_timeline");
-link(start, "state", cur, "state");
-
-const cx = lx + 420;
-const ref = add("MiniMaxH3ReferenceToVideo", "Reference Conditioning", [cx, 60], {ref_image_size: "match"});
-ref.size = [360, 300];
-const src = add("MiniMaxH3ContexLoopSourceSlice", "Source clip (frames + audio)", [cx, 420], {source_fps: 24, crop: "center"});
-const motionAll = add("ImageScaleToMaxDimension", "Shot list frames (~512 px)", [vx, 1340], {upscale_method: "area", largest_size: 512});
-const clipStart = add("ComfyMathExpression", "Clip start frame", [cx, 640], {0: "round(a * 24)"});
-const motion = add("ImageFromBatch", "Shot list frames for this clip", [cx, 740], {batch_index: 0, length: 141});
-const samAll = add("ImageScaleToMaxDimension", "SAM video (1008 px, SAM's own size)", [vx, 1460], {upscale_method: "area", largest_size: 1008});
-const samClip = add("ImageFromBatch", "SAM frames for this clip", [cx, 820], {batch_index: 0, length: 141});
-const noise = add("RandomNoise", "Clip Seed", [cx, 760], {control_after_generate: "fixed"});
-const track = add("SAM3_VideoTrack", "SAM 3.1 track every person", [cx, 900], {detection_threshold: 0.35, max_objects: 0, detect_interval: 1});
-const body = add("H3PeopleMask", "Main person, no headroom (for the depth)", [cx, 1060], {headroom: 0, hold_frames: 1, max_people: 1});
-const bodyImg = add("MaskToImage", "Main person mask as image", [cx, 1160]);
-const depth = add("DepthAnythingV2Preprocessor", "Depth of this clip (Depth Anything V2)", [cx, 1260], {ckpt_name: "depth_anything_v2_vitb.pth", resolution: 512});
-const personDepth = add("ImageBlend", "Depth of the main person only (control video)", [cx, 1360], {blend_factor: 1.0, blend_mode: "multiply"});
-const whole = add("SolidMask", "Whole frame (regenerate everything)", [cx, 1460], {value: 1.0});
-
-link(te, 0, ref, "clip"); link(vae, 0, ref, "vae"); link(avae, 0, ref, "audio_vae");
-const shotFrames = add("H3ClipShots", "Find this clip's cuts", [cx, 1500], {cut_threshold: 40});
-const shotGen = add("TextGenerate", "Write this clip's shot list (Qwen3-VL)", [cx, 1600], {max_length: 500, ...GEN_OFF});
-shotGen.size = [360, 200];
-const fill = add("H3FillShots", "Insert shot list into prompt", [cx, 1850]);
-const promptPrev = add("PreviewAny", "Clip prompt (what H3 gets)", [cx, 1950]);
-link(motion, 0, shotFrames, "images");
-link(swapPlan, "shot_prompt", shotFrames, "shot_prompt");
-link(shotFrames, 0, shotGen, "image");
-link(qwen, 0, shotGen, "clip");
-link(shotFrames, 1, shotGen, "prompt");
-link(cur, "prompt", fill, "prompt");
-link(shotGen, 0, fill, "shots");
-link(fill, 0, promptPrev, "source");
-link(fill, 0, ref, "prompt"); link(cur, "width", ref, "width"); link(cur, "height", ref, "height"); link(cur, "length", ref, "length");
-link(pics[0], 0, ref, "ref_images.ref_image_0");
-link(cur, "state", src, "state");
-link(ref, 1, src, "latent"); link(avae, 0, src, "audio_vae");
-link(video, "IMAGE", src, "source_frames");
-link(video, "IMAGE", motionAll, "image");
-link(motionAll, 0, motion, "image");
-link(cur, "audio_start", clipStart, "values.a");
-link(clipStart, 1, motion, "batch_index");
-link(cur, "length", motion, "length");
-link(cur, "width", whole, "width"); link(cur, "height", whole, "height");
-
-link(cur, "noise_seed", noise, "noise_seed");
-link(video, "IMAGE", samAll, "image");
-link(samAll, 0, samClip, "image"); link(clipStart, 1, samClip, "batch_index"); link(cur, "length", samClip, "length");
-link(samClip, 0, track, "images"); link(sam, 0, track, "model"); link(samText, 0, track, "conditioning");
-link(track, 0, body, "track_data");
-link(body, 0, bodyImg, "mask");
-link(samClip, 0, depth, "image");
-link(depth, 0, personDepth, "image1"); link(bodyImg, 0, personDepth, "image2");
-
-const mx = cx + 420;
-const funPatch = add("ModelPatchLoader", "H3 Fun ControlNet Union 2.0", [mx, 60], {name: "minimax_h3_fun_controlnet_union_2.0_pruned_int8_convrot.safetensors"});
-const fun = add("MiniMaxH3FunControlNetApply", "Apply ControlNet (person depth)", [mx, 180], {strength: 1.0});
-const depthPrev = add("PreviewImage", "Control video (person depth)", [mx, 420]);
-depthPrev.size = [360, 300];
-const edit = add("MiniMaxH3FantasticVideoEditLatent", "Edit latent (source + mask + audio)", [mx, 760], {grow: 0, keep_audio: true, megapixels: 0});
-const masked = add("MiniMaxH3ContexMaskedTarget", "Masked target", [mx, 960], {mask_meaning: "white = generate", audio_mode: "preserve source audio"});
-const ctx = add("MiniMaxH3ChainContext", "Apply Clip Context", [mx, 1140]);
-link(shift, 0, fun, "model"); link(funPatch, 0, fun, "model_patch"); link(vae, 0, fun, "vae"); link(personDepth, 0, fun, "control_video");
-link(personDepth, 0, depthPrev, "images");
-link(vae, 0, edit, "vae"); link(src, "scene_frames", edit, "images"); link(whole, 0, edit, "mask");
-link(avae, 0, edit, "audio_vae"); link(src, "scene_audio", edit, "audio");
-link(edit, 0, masked, "target_latent"); link(whole, 0, masked, "mask");
-link(cur, "state", ctx, "state"); link(ref, 0, ctx, "conditioning"); link(vae, 0, ctx, "vae");
-link(masked, 0, ctx, "latent"); link(avae, 0, ctx, "audio_vae");
-
-const kx = mx + 420;
-const guider = add("BasicGuider", "Guider", [kx, 60]);
-const sched = add("BasicScheduler", "Schedule", [kx, 160], {scheduler: "simple", denoise: 1});
-const sca = add("SamplerCustomAdvanced", "Sample Video + Audio", [kx, 320]);
-const dec = add("VAEDecode", "Decode Video", [kx, 520]);
-const deca = add("VAEDecodeAudio", "Decode Audio", [kx, 620]);
-const trim = add("MiniMaxH3LoopTrim", "Trim Overlap", [kx, 720], {trim_frames: 0, fps: 24, match_tail: true, audio_trim_mode: "sync_with_video"});
-// With the ControlNet on, H3 draws the first frame or two as the picture itself; later clips trim them with their overlap.
-const frame2 = add("ImageFromBatch", "Frame 2", [kx, 920], {batch_index: 2, length: 1});
-const twice = add("RepeatImageBatch", "Frame 2 as frames 0 and 1 (they show the picture)", [kx, 1020], {amount: 2});
-const rest = add("ImageFromBatch", "Frame 2 to the end", [kx, 1120], {batch_index: 2, length: 4096});
-const held = add("BatchImagesNode", "Clip with frames 0 and 1 replaced", [kx, 1220]);
-link(fun, 0, guider, "model"); link(ctx, 0, guider, "conditioning");
-link(fun, 0, sched, "model"); link(cur, "steps", sched, "steps");
-link(noise, 0, sca, "noise"); link(guider, 0, sca, "guider"); link(sampler, 0, sca, "sampler");
-link(sched, 0, sca, "sigmas"); link(ctx, 3, sca, "latent_image");
-link(sca, 0, dec, "samples"); link(vae, 0, dec, "vae");
-link(sca, 0, deca, "samples"); link(avae, 0, deca, "vae");
-link(dec, 0, frame2, "image"); link(frame2, 0, twice, "image"); link(dec, 0, rest, "image");
-link(twice, 0, held, "images.image0"); link(rest, 0, held, "images.image1");
-link(held, 0, trim, "images"); link(ctx, 1, trim, "trim_frames"); link(deca, 0, trim, "audio"); link(cur, "state", trim, "state");
-
-const ox = kx + 400;
-const save = add("MiniMaxH3ChainSegmentSave", "Save Clip + Checkpoint", [ox, 60]);
-const review = add("MiniMaxH3ChainReview", "Review (off = no pause)", [ox, 260], {enabled: false});
-const end = add("MiniMaxH3ChainLoopEnd", "Loop End", [ox, 600], {between_scene_cleanup: "off", execution_mode: "recursive"});
-const asm = add("MiniMaxH3ChainAssemble", "Final Video", [ox, 800], {audio_source: "source", filename: "swap_%date:yyyy-MM-dd%", copy_to_output: true, output_subfolder: "h3_oneclick_swap"});
-link(cur, "state", save, "state"); link(trim, 0, save, "images"); link(sca, 0, save, "sampled_latent");
-link(trim, 1, save, "audio"); link(trim, 2, save, "images_with_overlap");
-link(cur, "state", review, "state"); link(save, 0, review, "segment"); link(trim, 1, review, "audio");
-link(start, "flow", end, "flow"); link(cur, "state", end, "state"); link(trim, 0, end, "images");
-link(sca, 0, end, "sampled_latent"); link(review, 0, end, "segment");
-link(end, 0, asm, "manifest");
-
-// optional pieces added last keep the node ids above stable for the Openjutsu export
-group("OPTIONAL - No lip-sync (they don't sing along)", 40, 1000, 460, 560, "#88a");
-const noLip = add("PrimitiveBoolean", "No lip-sync in the prompt", [100, 1060], {value: true}, 4);
-const muteClip = add("AudioAdjustVolume", "Mute the clip audio H3 hears", [100, 1160], {volume: -100}, 4);
-const muteRef = add("AudioAdjustVolume", "Mute the reference audio H3 hears", [100, 1260], {volume: -100}, 4);
-link(noLip, 0, swapPlan, "no_lip_sync");
-link(soundtrack, 0, muteClip, "audio"); link(muteClip, 0, src, "source_audio");
-link(cur, "source_audio_slice", muteRef, "audio"); link(muteRef, 0, ref, "ref_audios.ref_audio_0");
-
-// stack each column by real node height, then fit groups around their nodes
-const cols = {};
-for (const n of made) (cols[n.pos[0]] ??= []).push(n);
-for (const col of Object.values(cols)) {
- col.sort((a, b) => a.pos[1] - b.pos[1]);
- let y = col[0].pos[1];
- for (const n of col) { n.pos[1] = y; y += n.size[1] + LiteGraph.NODE_TITLE_HEIGHT + 30; }
-}
-for (const gr of g._groups) {
- const inside = gr.members;
- if (!inside.length) continue;
- const bottom = Math.max(...inside.map(n => n.pos[1] + n.size[1]));
- const right = Math.max(...inside.map(n => n.pos[0] + n.size[0]));
- gr.size = [Math.max(gr.size[0], right - gr.pos[0] + 20), bottom - gr.pos[1] + 30];
-}
-app.graph.setDirtyCanvas(true, true);
-JSON.stringify({nodes: g._nodes.length, links: Object.keys(g.links).length ?? g.links.size});
diff --git a/workflows/control.api.json b/workflows/control.api.json
deleted file mode 100644
index c4f7328..0000000
--- a/workflows/control.api.json
+++ /dev/null
@@ -1,1311 +0,0 @@
-{
- "2": {
- "inputs": {
- "image": ""
- },
- "class_type": "LoadImage",
- "_meta": {
- "title": "Picture 1",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "3": {
- "inputs": {
- "upscale_method": "area",
- "largest_size": 768,
- "image": [
- "2",
- 0
- ]
- },
- "class_type": "ImageScaleToMaxDimension",
- "_meta": {
- "title": "Shrink for the describer",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "4": {
- "inputs": {
- "prompt": "Describe the person in this picture for a casting sheet, in one sentence of about 40 words: apparent age and gender, hair, facial hair, skin tone, build, and their complete outfit including footwear. No preamble, no background.",
- "max_length": 120,
- "sampling_mode": "off",
- "thinking": false,
- "use_default_template": true,
- "mtp": "auto",
- "clip": [
- "13",
- 0
- ],
- "image": [
- "3",
- 0
- ]
- },
- "class_type": "TextGenerate",
- "_meta": {
- "title": "Describe picture 1",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "5": {
- "inputs": {
- "video": "",
- "force_rate": 24,
- "custom_width": 0,
- "custom_height": 0,
- "frame_load_cap": 0,
- "skip_first_frames": 0,
- "select_every_nth": 1,
- "format": "None"
- },
- "class_type": "VHS_LoadVideo",
- "_meta": {
- "title": "Source video (people get replaced)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "6": {
- "inputs": {
- "start_index": 0,
- "duration": 2048,
- "audio": [
- "5",
- 2
- ]
- },
- "class_type": "TrimAudioDuration",
- "_meta": {
- "title": "Soundtrack (whole video)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "7": {
- "inputs": {
- "select_every_nth": 48,
- "skip_first_images": 24,
- "images": [
- "5",
- 0
- ]
- },
- "class_type": "VHS_SelectEveryNthImage",
- "_meta": {
- "title": "One frame every 2 s",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "8": {
- "inputs": {
- "batch_index": 0,
- "length": 20,
- "image": [
- "7",
- 0
- ]
- },
- "class_type": "ImageFromBatch",
- "_meta": {
- "title": "At most 20 frames",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "9": {
- "inputs": {
- "upscale_method": "area",
- "largest_size": 640,
- "image": [
- "8",
- 0
- ]
- },
- "class_type": "ImageScaleToMaxDimension",
- "_meta": {
- "title": "Shrink for the describer",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "10": {
- "inputs": {
- "prompt": "These are frames from one video, in time order. List the people who appear most, up to four, the person with the most screen time first. Write one numbered line per person: first their role in the video in two to four words, without gender, that stays true in every shot, then \" - \" and how they look, like \"1. the guitarist - tall man with a grey beard in a denim jacket\". Each person appears once. Output only the list.",
- "max_length": 160,
- "sampling_mode": "off",
- "thinking": false,
- "use_default_template": true,
- "mtp": "auto",
- "clip": [
- "13",
- 0
- ],
- "image": [
- "9",
- 0
- ]
- },
- "class_type": "TextGenerate",
- "_meta": {
- "title": "Who is in the video",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "11": {
- "inputs": {
- "clip_seconds": 6,
- "overlap_frames": 39,
- "resolution": "736p",
- "steps": 6,
- "seed": 0,
- "roles": "",
- "no_lip_sync": [
- "68",
- 0
- ],
- "short_edge": 0,
- "source_frames": [
- "5",
- 0
- ],
- "picture_1": [
- "2",
- 0
- ],
- "picture_1_text": [
- "4",
- 0
- ],
- "video_people": [
- "10",
- 0
- ]
- },
- "class_type": "H3PictureSwapPlan",
- "_meta": {
- "title": "Swap Plan (roles, seed, size)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "13": {
- "inputs": {
- "clip_name": "qwen3vl_8b_int8_convrot.safetensors",
- "type": "stable_diffusion",
- "device": "default"
- },
- "class_type": "CLIPLoader",
- "_meta": {
- "title": "Describer: Qwen3-VL 8B",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "15": {
- "inputs": {
- "unet_name": "minimax_h3_ref2va_pruned_int8_convrot.safetensors",
- "weight_dtype": "default"
- },
- "class_type": "UNETLoader",
- "_meta": {
- "title": "H3 Diffusion Model",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "16": {
- "inputs": {
- "lora_name": "minimax_h3_DMAD_4step_full_lora_avg_rank_39_bf16.safetensors",
- "strength_model": 1,
- "model": [
- "15",
- 0
- ]
- },
- "class_type": "LoraLoaderModelOnly",
- "_meta": {
- "title": "DMAD LoRA",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "17": {
- "inputs": {
- "attention": "comfy kitchen attention",
- "model": [
- "16",
- 0
- ]
- },
- "class_type": "ModelAttentionBackend",
- "_meta": {
- "title": "Comfy Kitchen Attention",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "18": {
- "inputs": {
- "shift_video": 12,
- "shift_audio": 2,
- "model": [
- "17",
- 0
- ]
- },
- "class_type": "MiniMaxH3SigmaShift",
- "_meta": {
- "title": "Sigma Shift",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "19": {
- "inputs": {
- "clip_name": "qwen3vl_32b_minimax_h3_int4_convrot.safetensors",
- "type": "minimax",
- "device": "default"
- },
- "class_type": "CLIPLoader",
- "_meta": {
- "title": "H3 Text Encoder",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "20": {
- "inputs": {
- "vae_name": "minimax_h3_video_vae_int8_convrot.safetensors"
- },
- "class_type": "VAELoader",
- "_meta": {
- "title": "Video VAE",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "21": {
- "inputs": {
- "vae_name": "minimax_h3_audio_vae_fp32.safetensors"
- },
- "class_type": "VAELoader",
- "_meta": {
- "title": "Audio VAE",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "22": {
- "inputs": {
- "sampler_name": "euler"
- },
- "class_type": "KSamplerSelect",
- "_meta": {
- "title": "Sampler",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "23": {
- "inputs": {
- "ckpt_name": "sam3.1_multiplex_fp16.safetensors"
- },
- "class_type": "CheckpointLoaderSimple",
- "_meta": {
- "title": "SAM 3.1",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "24": {
- "inputs": {
- "text": "person",
- "clip": [
- "23",
- 1
- ]
- },
- "class_type": "CLIPTextEncode",
- "_meta": {
- "title": "SAM: what to replace",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "25": {
- "inputs": {
- "scene_continuity": "Hard picture + protected audio",
- "audio_profile": "Generate audio"
- },
- "class_type": "MiniMaxH3GenerationProfile",
- "_meta": {
- "title": "Generation Profile",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "26": {
- "inputs": {
- "plan_json": "{\n \"shots\": [\n {\n \"id\": \"intro\",\n \"prompt\": \"Describe the opening shot.\"\n },\n {\n \"id\": \"continuation\",\n \"prompt\": \"Continue the same take.\"\n }\n ]\n}",
- "run_name": [
- "11",
- 3
- ],
- "generation_fingerprint": "",
- "width": [
- "11",
- 1
- ],
- "height": [
- "11",
- 2
- ],
- "encode_mode": "video",
- "crop": "center",
- "default_duration_seconds": 15,
- "default_steps": 6,
- "base_seed": 0,
- "segment_crf": 16,
- "video_blend_frames": 0,
- "chain_policy": [
- "25",
- 0
- ],
- "plan_json_input": [
- "11",
- 0
- ]
- },
- "class_type": "MiniMaxH3ChainPlanModern",
- "_meta": {
- "title": "Production Plan (filled by Swap Plan)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "27": {
- "inputs": {
- "video_path": "",
- "audio_path": "",
- "embedded_audio": "auto",
- "skip_first_frames": 0,
- "source_audio": [
- "6",
- 0
- ]
- },
- "class_type": "MiniMaxH3SourceTimeline",
- "_meta": {
- "title": "Source soundtrack",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "28": {
- "inputs": {
- "start_clip": 1,
- "scene_range": "",
- "verify_resume_history": true,
- "plan": [
- "26",
- 0
- ],
- "source_timeline": [
- "27",
- 0
- ]
- },
- "class_type": "MiniMaxH3ChainLoopStart",
- "_meta": {
- "title": "Loop Start",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "29": {
- "inputs": {
- "align_audio_reference": false,
- "state": [
- "28",
- 1
- ]
- },
- "class_type": "MiniMaxH3ChainCurrent",
- "_meta": {
- "title": "Current Clip",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "30": {
- "inputs": {
- "prompt": [
- "46",
- 0
- ],
- "width": [
- "29",
- 8
- ],
- "height": [
- "29",
- 9
- ],
- "length": [
- "29",
- 6
- ],
- "ref_image_size": "match",
- "clip": [
- "19",
- 0
- ],
- "vae": [
- "20",
- 0
- ],
- "audio_vae": [
- "21",
- 0
- ],
- "ref_images.ref_image_0": [
- "2",
- 0
- ],
- "ref_audios.ref_audio_0": [
- "70",
- 0
- ]
- },
- "class_type": "MiniMaxH3ReferenceToVideo",
- "_meta": {
- "title": "Reference Conditioning",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "31": {
- "inputs": {
- "source_fps": 24,
- "crop": "center",
- "state": [
- "29",
- 0
- ],
- "latent": [
- "30",
- 1
- ],
- "audio_vae": [
- "21",
- 0
- ],
- "source_frames": [
- "5",
- 0
- ],
- "source_audio": [
- "69",
- 0
- ]
- },
- "class_type": "MiniMaxH3ContexLoopSourceSlice",
- "_meta": {
- "title": "Source clip (frames + audio)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "32": {
- "inputs": {
- "upscale_method": "area",
- "largest_size": 512,
- "image": [
- "5",
- 0
- ]
- },
- "class_type": "ImageScaleToMaxDimension",
- "_meta": {
- "title": "Shot list frames (~512 px)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "33": {
- "inputs": {
- "expression": "round(a * 24)",
- "values.a": [
- "29",
- 10
- ]
- },
- "class_type": "ComfyMathExpression",
- "_meta": {
- "title": "Clip start frame",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "34": {
- "inputs": {
- "batch_index": [
- "33",
- 1
- ],
- "length": [
- "29",
- 6
- ],
- "image": [
- "32",
- 0
- ]
- },
- "class_type": "ImageFromBatch",
- "_meta": {
- "title": "Shot list frames for this clip",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "35": {
- "inputs": {
- "upscale_method": "area",
- "largest_size": 1008,
- "image": [
- "5",
- 0
- ]
- },
- "class_type": "ImageScaleToMaxDimension",
- "_meta": {
- "title": "SAM video (1008 px, SAM's own size)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "36": {
- "inputs": {
- "batch_index": [
- "33",
- 1
- ],
- "length": [
- "29",
- 6
- ],
- "image": [
- "35",
- 0
- ]
- },
- "class_type": "ImageFromBatch",
- "_meta": {
- "title": "SAM frames for this clip",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "37": {
- "inputs": {
- "noise_seed": [
- "29",
- 5
- ]
- },
- "class_type": "RandomNoise",
- "_meta": {
- "title": "Clip Seed",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "38": {
- "inputs": {
- "detection_threshold": 0.35,
- "max_objects": 0,
- "detect_interval": 1,
- "images": [
- "36",
- 0
- ],
- "model": [
- "23",
- 0
- ],
- "conditioning": [
- "24",
- 0
- ]
- },
- "class_type": "SAM3_VideoTrack",
- "_meta": {
- "title": "SAM 3.1 track every person",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "39": {
- "inputs": {
- "headroom": 0,
- "hold_frames": 1,
- "max_people": 1,
- "track_data": [
- "38",
- 0
- ]
- },
- "class_type": "H3PeopleMask",
- "_meta": {
- "title": "Main person, no headroom (for the depth)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "40": {
- "inputs": {
- "mask": [
- "39",
- 0
- ]
- },
- "class_type": "MaskToImage",
- "_meta": {
- "title": "Main person mask as image",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "41": {
- "inputs": {
- "ckpt_name": "depth_anything_v2_vitb.pth",
- "resolution": 512,
- "image": [
- "36",
- 0
- ]
- },
- "class_type": "DepthAnythingV2Preprocessor",
- "_meta": {
- "title": "Depth of this clip (Depth Anything V2)",
- "cnr_id": "comfyui_controlnet_aux",
- "ver": "a587a4c1594524c1d2de5cd1a86cd868ff448e1c"
- }
- },
- "42": {
- "inputs": {
- "blend_factor": 1,
- "blend_mode": "multiply",
- "image1": [
- "41",
- 0
- ],
- "image2": [
- "40",
- 0
- ]
- },
- "class_type": "ImageBlend",
- "_meta": {
- "title": "Depth of the main person only (control video)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "43": {
- "inputs": {
- "value": 1,
- "width": [
- "29",
- 8
- ],
- "height": [
- "29",
- 9
- ]
- },
- "class_type": "SolidMask",
- "_meta": {
- "title": "Whole frame (regenerate everything)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "44": {
- "inputs": {
- "cut_threshold": 40,
- "images": [
- "34",
- 0
- ],
- "shot_prompt": [
- "11",
- 4
- ]
- },
- "class_type": "H3ClipShots",
- "_meta": {
- "title": "Find this clip's cuts",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "45": {
- "inputs": {
- "prompt": [
- "44",
- 1
- ],
- "max_length": 500,
- "sampling_mode": "off",
- "thinking": false,
- "use_default_template": true,
- "mtp": "auto",
- "clip": [
- "13",
- 0
- ],
- "image": [
- "44",
- 0
- ]
- },
- "class_type": "TextGenerate",
- "_meta": {
- "title": "Write this clip's shot list (Qwen3-VL)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "46": {
- "inputs": {
- "prompt": [
- "29",
- 4
- ],
- "shots": [
- "45",
- 0
- ]
- },
- "class_type": "H3FillShots",
- "_meta": {
- "title": "Insert shot list into prompt",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "47": {
- "inputs": {
- "source": [
- "46",
- 0
- ]
- },
- "class_type": "PreviewAny",
- "_meta": {
- "title": "Clip prompt (what H3 gets)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "48": {
- "inputs": {
- "name": "minimax_h3_fun_controlnet_union_2.0_pruned_int8_convrot.safetensors"
- },
- "class_type": "ModelPatchLoader",
- "_meta": {
- "title": "H3 Fun ControlNet Union 2.0",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "49": {
- "inputs": {
- "strength": 1,
- "start_percent": 0,
- "end_percent": 1,
- "model": [
- "18",
- 0
- ],
- "model_patch": [
- "48",
- 0
- ],
- "vae": [
- "20",
- 0
- ],
- "control_video": [
- "42",
- 0
- ]
- },
- "class_type": "MiniMaxH3FunControlNetApply",
- "_meta": {
- "title": "Apply ControlNet (person depth)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "50": {
- "inputs": {
- "images": [
- "42",
- 0
- ]
- },
- "class_type": "PreviewImage",
- "_meta": {
- "title": "Control video (person depth)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "51": {
- "inputs": {
- "grow": 0,
- "keep_audio": true,
- "megapixels": 0,
- "vae": [
- "20",
- 0
- ],
- "images": [
- "31",
- 0
- ],
- "mask": [
- "43",
- 0
- ],
- "audio_vae": [
- "21",
- 0
- ],
- "audio": [
- "31",
- 1
- ]
- },
- "class_type": "MiniMaxH3FantasticVideoEditLatent",
- "_meta": {
- "title": "Edit latent (source + mask + audio)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "52": {
- "inputs": {
- "mask_meaning": "white = generate",
- "audio_mode": "preserve source audio",
- "mask_conversion": "H3 exact (causal/token max)",
- "target_latent": [
- "51",
- 0
- ],
- "mask": [
- "43",
- 0
- ]
- },
- "class_type": "MiniMaxH3ContexMaskedTarget",
- "_meta": {
- "title": "Masked target",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "53": {
- "inputs": {
- "state": [
- "29",
- 0
- ],
- "conditioning": [
- "30",
- 0
- ],
- "vae": [
- "20",
- 0
- ],
- "latent": [
- "52",
- 0
- ],
- "audio_vae": [
- "21",
- 0
- ]
- },
- "class_type": "MiniMaxH3ChainContext",
- "_meta": {
- "title": "Apply Clip Context",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "54": {
- "inputs": {
- "model": [
- "49",
- 0
- ],
- "conditioning": [
- "53",
- 0
- ]
- },
- "class_type": "BasicGuider",
- "_meta": {
- "title": "Guider",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "55": {
- "inputs": {
- "scheduler": "simple",
- "steps": [
- "29",
- 7
- ],
- "denoise": 1,
- "model": [
- "49",
- 0
- ]
- },
- "class_type": "BasicScheduler",
- "_meta": {
- "title": "Schedule",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "56": {
- "inputs": {
- "noise": [
- "37",
- 0
- ],
- "guider": [
- "54",
- 0
- ],
- "sampler": [
- "22",
- 0
- ],
- "sigmas": [
- "55",
- 0
- ],
- "latent_image": [
- "53",
- 3
- ]
- },
- "class_type": "SamplerCustomAdvanced",
- "_meta": {
- "title": "Sample Video + Audio",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "57": {
- "inputs": {
- "samples": [
- "56",
- 0
- ],
- "vae": [
- "20",
- 0
- ]
- },
- "class_type": "VAEDecode",
- "_meta": {
- "title": "Decode Video",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "58": {
- "inputs": {
- "samples": [
- "56",
- 0
- ],
- "vae": [
- "21",
- 0
- ]
- },
- "class_type": "VAEDecodeAudio",
- "_meta": {
- "title": "Decode Audio",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "59": {
- "inputs": {
- "trim_frames": [
- "53",
- 1
- ],
- "fps": 24,
- "match_tail": true,
- "audio_trim_mode": "sync_with_video",
- "images": [
- "63",
- 0
- ],
- "audio": [
- "58",
- 0
- ],
- "state": [
- "29",
- 0
- ]
- },
- "class_type": "MiniMaxH3LoopTrim",
- "_meta": {
- "title": "Trim Overlap",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "60": {
- "inputs": {
- "batch_index": 2,
- "length": 1,
- "image": [
- "57",
- 0
- ]
- },
- "class_type": "ImageFromBatch",
- "_meta": {
- "title": "Frame 2",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "61": {
- "inputs": {
- "amount": 2,
- "image": [
- "60",
- 0
- ]
- },
- "class_type": "RepeatImageBatch",
- "_meta": {
- "title": "Frame 2 as frames 0 and 1 (they show the picture)",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "62": {
- "inputs": {
- "batch_index": 2,
- "length": 4096,
- "image": [
- "57",
- 0
- ]
- },
- "class_type": "ImageFromBatch",
- "_meta": {
- "title": "Frame 2 to the end",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "63": {
- "inputs": {
- "images.image0": [
- "61",
- 0
- ],
- "images.image1": [
- "62",
- 0
- ]
- },
- "class_type": "BatchImagesNode",
- "_meta": {
- "title": "Clip with frames 0 and 1 replaced",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "64": {
- "inputs": {
- "state": [
- "29",
- 0
- ],
- "images": [
- "59",
- 0
- ],
- "sampled_latent": [
- "56",
- 0
- ],
- "audio": [
- "59",
- 1
- ],
- "images_with_overlap": [
- "59",
- 2
- ]
- },
- "class_type": "MiniMaxH3ChainSegmentSave",
- "_meta": {
- "title": "Save Clip + Checkpoint",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "65": {
- "inputs": {
- "enabled": false,
- "play_notification_sound": false,
- "auto_continue_timeout_minutes": 0,
- "unload_models_while_waiting": false,
- "assemble_partial_on_stop": true,
- "partial_audio_source": "checkpointed",
- "candidate_count": 1,
- "review_each_candidate": false,
- "state": [
- "29",
- 0
- ],
- "segment": [
- "64",
- 0
- ],
- "audio": [
- "59",
- 1
- ]
- },
- "class_type": "MiniMaxH3ChainReview",
- "_meta": {
- "title": "Review (off = no pause)",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "66": {
- "inputs": {
- "between_scene_cleanup": "off",
- "execution_mode": "recursive",
- "flow": [
- "28",
- 0
- ],
- "state": [
- "29",
- 0
- ],
- "images": [
- "59",
- 0
- ],
- "sampled_latent": [
- "56",
- 0
- ],
- "segment": [
- "65",
- 0
- ]
- },
- "class_type": "MiniMaxH3ChainLoopEnd",
- "_meta": {
- "title": "Loop End",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "67": {
- "inputs": {
- "audio_source": "source",
- "filename": "swap_%date:yyyy-MM-dd%",
- "audio_bitrate": 256,
- "copy_to_output": true,
- "output_subfolder": "h3_oneclick_swap",
- "blend_schedule": "plan",
- "boundary_tone_match": "off",
- "color_stabilization": "off",
- "delete_checkpoints_after_assembly": false,
- "generated_audio_join": "av_overlap",
- "manifest": [
- "66",
- 0
- ]
- },
- "class_type": "MiniMaxH3ChainAssemble",
- "_meta": {
- "title": "Final Video",
- "cnr_id": "comfyui-workflow-encrypt",
- "ver": "1.0.0"
- }
- },
- "68": {
- "inputs": {
- "value": true
- },
- "class_type": "PrimitiveBoolean",
- "_meta": {
- "title": "No lip-sync in the prompt",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "69": {
- "inputs": {
- "volume": -100,
- "audio": [
- "6",
- 0
- ]
- },
- "class_type": "AudioAdjustVolume",
- "_meta": {
- "title": "Mute the clip audio H3 hears",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- },
- "70": {
- "inputs": {
- "volume": -100,
- "audio": [
- "29",
- 12
- ]
- },
- "class_type": "AudioAdjustVolume",
- "_meta": {
- "title": "Mute the reference audio H3 hears",
- "cnr_id": "comfy-core",
- "ver": "0.42.0"
- }
- }
-}
\ No newline at end of file