Skip to content

One-click character swap for videos: an open-source, local Higgsfield Genjutsu alternative built on ComfyUI and MiniMax H3, with FLUX.3 Klein relighting.

Add relighting the video's own person to From picture

D3voz77767957ce4a
diff --git a/README.md b/README.md
index c6fb443..5f77f1c 100644
--- a/README.md
+++ b/README.md
@@ -27,7 +27,7 @@ Choose according to available VRAM and leave room for image/video context and wo
- **Simple motion transfer**: available. One character replaces the main person (the one who fills the most of the frame). Runs `workflows/simple.api.json`, exported from `H3 One-Click Single Swap (DMAD 6 step).json`.
- **Advanced motion transfer**: available. 1–4 characters, each with an optional role; every tracked person is matched to a role per shot, and people without a character stay untouched. Runs `workflows/advanced.api.json`, exported from `H3 One-Click Advanced Swap (DMAD 6 step).json`; **One pass per person** (two characters) runs `workflows/advanced_maxq.api.json`.
- **Object swap**: available. **Change** what SAM tracks (hair, a jacket, a car) into what you describe or show in a picture; **Remove** it; or change the **Background** around it (`person` stays). Only that area is regenerated. Runs `workflows/object.api.json`, exported from `H3 One-Click Object Swap (DMAD 6 step).json`.
-- **From picture** (under Object swap): FLUX prepares the character in the raw video frame's pose and body cut while keeping the character picture's background. Review or regenerate the image, then choose **Use this image** to set the video length, video and motion-reference resolutions, models, LoRAs, steps and seed. SolarRenderer defaults to four steps with a selectable rank-384 or rank-64 four-step/control pair and supports up to 15 seconds, like the native H3 renderer. SolarRenderer clips longer than six seconds can take considerably longer to render, especially at higher resolutions; see the workflow requirements for the extended-silence node fix. Both LoRA file pickers support `models/loras` and its subfolders. **Render video** reuses the saved preview with normal-map motion guidance and the source soundtrack, in one clip without a context loop. H3 rounds the requested length down to its valid frame grid. **Relight** (off by default) changes the light in the same FLUX pass: pick one of nine looks, each shown with an example picture, or describe your own light. FLUX then sees only your picture, and the Qwen 3.5 9B prompt upsampler reads the pose from the video frame. Magenta club and Teal lasers mostly recolour the background. See [the workflow requirements](workflows/picture/README.md).
+- **From picture** (under Object swap): FLUX prepares the character in the raw video frame's pose and body cut while keeping the character picture's background. Review or regenerate the image, then choose **Use this image** to set the video length, video and motion-reference resolutions, models, LoRAs, steps and seed. SolarRenderer defaults to four steps with a selectable rank-384 or rank-64 four-step/control pair and supports up to 15 seconds, like the native H3 renderer. SolarRenderer clips longer than six seconds can take considerably longer to render, especially at higher resolutions; see the workflow requirements for the extended-silence node fix. Both LoRA file pickers support `models/loras` and its subfolders. **Render video** reuses the saved preview with normal-map motion guidance and the source soundtrack, in one clip without a context loop. H3 rounds the requested length down to its valid frame grid. **Relight** (off by default) changes the light in the same FLUX pass: pick one of nine looks, each shown with an example picture, or describe your own light. FLUX then sees only your picture, and the Qwen 3.5 9B prompt upsampler reads the pose from the video frame. Magenta club and Teal lasers mostly recolour the background. **Relight the video's own person** skips the character picture: FLUX relights the video frame itself, keeping its person, pose and setting. See [the workflow requirements](workflows/picture/README.md).
- **Post-processing**: available, for a finished video. **Fix faces**: H3 draws faces poorly when the person is small in the frame, so this redraws the face of the person in a photo at close-up size and pastes it back. Runs `workflows/face.api.json`, written by `workflows/build/openjutsu_face_build.py`. **Fix colours** softens H3's harsh look without an H3 run: Image Detail Contrast takes some fine detail contrast out, then Orion4D's DCTL Tone Mapper grades every frame. Runs `workflows/color.api.json`.
## Requirements
diff --git a/app/api/picture/route.ts b/app/api/picture/route.ts
index 9792edd..8a67a82 100644
--- a/app/api/picture/route.ts
+++ b/app/api/picture/route.ts
@@ -5,7 +5,7 @@ import { join } from "node:path";
import { apiAccess, HttpError, IMAGE_SIGNATURE, readFormData, StoredFile, validateJobId, VIDEO_SIGNATURE } from "../../../lib/http.ts";
import { canvasSize, isLink, MAX_SEED, type ApiGraph } from "../../../lib/simple.ts";
import { listModels } from "../../../lib/comfy.ts";
-import { applyPictureVideoSettings, applyRelight, DEFAULT_PICTURE_VIDEO, parsePictureVideoSettings, pictureLoras, solarLoraFiles, type PictureVideoSettings } from "../../../lib/picture.ts";
+import { applyFrameRelight, applyPictureVideoSettings, applyRelight, DEFAULT_PICTURE_VIDEO, parsePictureVideoSettings, pictureLoras, solarLoraFiles, type PictureVideoSettings } from "../../../lib/picture.ts";
export const runtime = "nodejs";
const PREPARED = "OpenJutsu · prepared picture";
@@ -116,15 +116,19 @@ export async function POST(request: Request) {
graph = Object.fromEntries(Object.entries(graph).filter(([id]) => keep.has(id)));
} else {
upload = await readFormData(request, FILE_CHECKS);
- const form = upload.form, video = form.get("video"), picture = form.get("picture");
- if (!(video instanceof StoredFile) || !video.size || !(picture instanceof StoredFile) || !picture.size) throw new HttpError(400, "Choose a video and a character picture.");
+ const form = upload.form, video = form.get("video"), picture = form.get("picture"), light = form.get("light");
+ const lit = typeof light === "string" ? light.trim() : "", ownPerson = form.get("relight") === "frame";
+ if (!(video instanceof StoredFile) || !video.size) throw new HttpError(400, "Choose a video.");
+ if (ownPerson && !lit) throw new HttpError(400, "Choose a light for the video's own person.");
+ const character = !ownPerson && picture instanceof StoredFile && picture.size ? picture : null;
+ if (!ownPerson && !character) throw new HttpError(400, "Choose a character picture.");
const start = Number(form.get("start") ?? 0), seed = Number(form.get("seed") ?? 0);
const width = Number(form.get("width")), height = Number(form.get("height"));
if (!Number.isFinite(start) || start < 0 || !Number.isInteger(seed) || seed < 0 || seed > MAX_SEED) throw new HttpError(400, "Use a start time of 0 or later and a valid whole-number seed.");
if (![width, height].every((side) => Number.isFinite(side) && side > 0)) throw new HttpError(400, "Wait for the video's dimensions to load.");
const extension = video.name.match(/\.(mp4|mov|mkv|webm|avi)$/i)?.[0];
- const pictureExtension = picture.name.match(/\.(png|jpe?g|webp)$/i)?.[0];
- if (!extension || !pictureExtension) throw new HttpError(400, "Choose a supported video and PNG, JPEG or WebP picture.");
+ const pictureExtension = character?.name.match(/\.(png|jpe?g|webp)$/i)?.[0];
+ if (!extension || character && !pictureExtension) throw new HttpError(400, "Choose a supported video and PNG, JPEG or WebP picture.");
graph = await template("prepare");
const prompt = form.get("prompt");
if (typeof prompt === "string" && prompt.trim()) graph["6"].inputs.value = prompt;
@@ -132,11 +136,11 @@ export async function POST(request: Request) {
for (const id of ["3", "5"]) Object.assign(graph[id].inputs, { width: editWidth, height: editHeight });
graph["1"].inputs.video = await uploadFile(await openAsBlob(video.path), extension.toLowerCase());
graph["1"].inputs.skip_first_frames = Math.round(start * 24);
- graph["4"].inputs.image = await uploadFile(await openAsBlob(picture.path), pictureExtension.toLowerCase());
+ if (character) graph["4"].inputs.image = await uploadFile(await openAsBlob(character.path), pictureExtension!.toLowerCase());
graph["19"].inputs.seed = seed;
graph["21"]._meta = { title: PREPARED };
- const light = form.get("light");
- if (typeof light === "string" && light.trim()) applyRelight(graph, light.trim());
+ if (ownPerson) applyFrameRelight(graph, lit);
+ else if (lit) applyRelight(graph, lit);
}
const result = await (await comfy("/prompt", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ prompt: graph }) })).json();
return Response.json({ id: result.prompt_id }, { status: 202 });
diff --git a/components/from-picture.tsx b/components/from-picture.tsx
index c63cc8a..5676a16 100644
--- a/components/from-picture.tsx
+++ b/components/from-picture.tsx
@@ -3,7 +3,7 @@
import { useEffect, useRef, useState } from "react";
import { canvasSize, CUSTOM_SIZE, MAX_SEED, MIN_SECTION_SECONDS, MOTION_SIZES, PLACES, placeStart, RESOLUTIONS, timecode } from "../lib/simple.ts";
import { MAX_LORAS, STEPS, STRENGTH } from "../lib/model.ts";
-import { DEFAULT_PICTURE_VIDEO, LIGHT_LOOKS, parsePictureVideoSettings, pictureFrames, RELIGHT_PROMPT, solarLoraFiles, type PictureVideoSettings } from "../lib/picture.ts";
+import { DEFAULT_PICTURE_VIDEO, FRAME_RELIGHT_PROMPT, LIGHT_LOOKS, parsePictureVideoSettings, pictureFrames, RELIGHT_PROMPT, solarLoraFiles, type PictureVideoSettings } from "../lib/picture.ts";
const PROMPT = "change <image 1> character pose follow the <image 2> pose. keep the final image as same body cut as <image 2> . do not change the background of <image 1> and do not add anything extra in <image 1>.";
const STORAGE = "openjutsu.picture-preview";
@@ -25,6 +25,8 @@ export function FromPicture({ initialVideo, initialPicture, start: initialStart
const [look, setLook] = useState(-1);
const [ownLight, setOwnLight] = useState("Light the new photo with ");
const [relightPrompt, setRelightPrompt] = useState(RELIGHT_PROMPT);
+ const [ownPerson, setOwnPerson] = useState(false);
+ const [framePrompt, setFramePrompt] = useState(FRAME_RELIGHT_PROMPT);
const [job, setJob] = useState<Job | null>(null);
const [prepared, setPrepared] = useState("");
const [preview, setPreview] = useState<{ url: string; frame: string } | null>(null);
@@ -39,6 +41,7 @@ export function FromPicture({ initialVideo, initialPicture, start: initialStart
const busy = submitting || !!job;
const relight = look >= 0;
const light = look === LIGHT_LOOKS.length ? ownLight : LIGHT_LOOKS[look]?.light ?? "";
+ const ready = !!video && (ownPerson || !!picture) && !!size && (relight ? !!light.trim() : !ownPerson);
useEffect(() => {
try {
@@ -123,12 +126,13 @@ export function FromPicture({ initialVideo, initialPicture, start: initialStart
setPrepared(""); setPreview(null); setApproved(false); setResult(""); setError(""); remember(null, "", false);
}
async function prepare(nextSeed: number) {
- if (!video || !picture || !size) return;
+ if (!ready || !video || !size) return;
setSubmitting(true); invalidate(); setSeed(nextSeed);
try {
const form = new FormData();
- form.set("video", video); form.set("picture", picture); form.set("start", String(start)); form.set("seed", String(nextSeed));
- form.set("width", String(size[0])); form.set("height", String(size[1])); form.set("prompt", relight ? relightPrompt : prompt);
+ form.set("video", video); form.set("start", String(start)); form.set("seed", String(nextSeed));
+ form.set("width", String(size[0])); form.set("height", String(size[1])); form.set("prompt", ownPerson ? framePrompt : relight ? relightPrompt : prompt);
+ if (ownPerson) form.set("relight", "frame"); else if (picture) form.set("picture", picture);
if (relight) form.set("light", light);
const response = await fetch("/api/picture", { method: "POST", body: form });
const data = await response.json();
@@ -186,28 +190,33 @@ export function FromPicture({ initialVideo, initialPicture, start: initialStart
<input type="range" min={0} max={duration - settings.seconds} step={0.1} value={start} disabled={busy || whole} onChange={(event) => choosePart(Number(event.target.value), settings.seconds)} />
</label>
</>}
- <label>Character picture · image 1<input type="file" accept="image/png,image/jpeg,image/webp" disabled={busy} onChange={(event) => { const file = event.target.files?.[0]; if (file) { invalidate(); setPicture(file); } }} /></label>
- {pictureUrl && <img className="picture-reference" src={pictureUrl} alt="Character and background reference" />}
- <small>The part's first frame is image 2. Choose the render settings after reviewing the prepared image.</small>
+ <label className="setting toggle"><span>Relight the video's own person <small>No new character: only the light changes.</small></span><input type="checkbox" checked={ownPerson} disabled={busy} onChange={(event) => { invalidate(); setOwnPerson(event.target.checked); if (event.target.checked && look < 0) setLook(0); }} /><i /></label>
+ {!ownPerson && <>
+ <label>Character picture · image 1<input type="file" accept="image/png,image/jpeg,image/webp" disabled={busy} onChange={(event) => { const file = event.target.files?.[0]; if (file) { invalidate(); setPicture(file); } }} /></label>
+ {pictureUrl && <img className="picture-reference" src={pictureUrl} alt="Character and background reference" />}
+ </>}
+ <small>The part's first frame is {ownPerson ? "the picture FLUX relights" : "image 2"}. Choose the render settings after reviewing the prepared image.</small>
</section>
<section className="panel">
<h2><span>2</span> Prepare the first frame</h2>
- <p className="hint">Image 1 supplies the character and background. Image 2 supplies the pose and body cut.</p>
+ <p className="hint">{ownPerson ? "FLUX relights the frame itself: the same person, pose, framing and setting under a new light." : "Image 1 supplies the character and background. Image 2 supplies the pose and body cut."}</p>
<p className="hint">Missing the FLUX model files? <a href="https://huggingface.co/black-forest-labs/flux-3-image-klein-comfy" target="_blank" rel="noopener noreferrer">Download FLUX 3 Image Klein files</a>.</p>
<p className="hint"><strong>Prompt upsampler required.</strong> This workflow needs the <a href="https://huggingface.co/black-forest-labs/flux-3-image-klein-comfy/blob/main/text_encoders/qwen3.5_9b_flux3_prompt_upsampler_int8_convrot.safetensors" target="_blank" rel="noopener noreferrer">Qwen3.5 9B FLUX prompt upsampler</a> to expand the edit prompt before image generation. Download it into <code>ComfyUI/models/text_encoders</code>.</p>
<label>Relight<select className="file-name" value={look} disabled={busy} onChange={(event) => { invalidate(); setLook(Number(event.target.value)); }}>
- <option value={-1}>Off · keep the picture's light</option>
+ {!ownPerson && <option value={-1}>Off · keep the picture's light</option>}
{LIGHT_LOOKS.map((item, index) => <option key={item.name} value={index}>{item.name}</option>)}
<option value={LIGHT_LOOKS.length}>My own light</option>
</select></label>
{LIGHT_LOOKS[look] && <figure style={{ margin: 0, display: "grid", gap: 6 }}><img className="picture-reference" src={LIGHT_LOOKS[look].image} alt={`${LIGHT_LOOKS[look].name} example`} style={{ width: 200 }} /><small>Example on one portrait. Your picture and seed will differ.</small></figure>}
- {relight && <p className="hint">Poses and relights the character in one FLUX pass at 3 MP, about 2 minutes. FLUX sees only image 1; image 2 reaches it as a written pose, so nothing of the video's person leaks in. The light usually changes the background too. Results vary a lot by seed; try two or three.</p>}
+ {relight && !ownPerson && <p className="hint">Poses and relights the character in one FLUX pass at 3 MP, about 2 minutes. FLUX sees only image 1; image 2 reaches it as a written pose, so nothing of the video's person leaks in. The light usually changes the background too. Results vary a lot by seed; try two or three.</p>}
{look === LIGHT_LOOKS.length && <label>Your light<textarea className="report" rows={3} value={ownLight} disabled={busy} onChange={(event) => { invalidate(); setOwnLight(event.target.value); }} /></label>}
- {relight
+ {ownPerson
+ ? <label>FLUX relight prompt · {"{look}"} is the light<textarea className="report" rows={4} value={framePrompt} disabled={busy} onChange={(event) => { invalidate(); setFramePrompt(event.target.value); }} /></label>
+ : relight
? <label>FLUX relight prompt · {"{look}"} is the light<textarea className="report" rows={6} value={relightPrompt} disabled={busy} onChange={(event) => { invalidate(); setRelightPrompt(event.target.value); }} /></label>
: <label>FLUX edit prompt<textarea className="report" rows={6} value={prompt} disabled={busy} onChange={(event) => { invalidate(); setPrompt(event.target.value); }} /></label>}
<label>Seed<input className="target-input" type="number" min={0} max={MAX_SEED} step={1} value={seed} disabled={busy} onChange={(event) => { invalidate(); setSeed(Number(event.target.value)); }} /></label>
- <button className="primary" disabled={busy || !video || !picture || !size || (relight && !light.trim())} onClick={() => prepare(seed)}>Generate preview image</button>
+ <button className="primary" disabled={busy || !ready} onClick={() => prepare(seed)}>Generate preview image</button>
<p className="hint">This step only generates an image. After reviewing it, choose the video settings and press Render video.</p>
</section>
</div>}
@@ -216,13 +225,13 @@ export function FromPicture({ initialVideo, initialPicture, start: initialStart
{preview && !approved && <section className="panel">
<h2><span>3</span> Review before rendering</h2>
<div className="inputs picture-comparison">
- <figure><figcaption>Source frame · image 2</figcaption><img src={preview.frame} alt="Raw source video frame used for pose and body cut" /></figure>
+ <figure><figcaption>Source frame{ownPerson ? "" : " · image 2"}</figcaption><img src={preview.frame} alt="Raw source video frame used for pose and body cut" /></figure>
<figure><figcaption>FLUX prepared image</figcaption><img src={preview.url} alt="FLUX generated character in the source pose" /></figure>
</div>
<p className="hint">Check the pose, framing, character and background. The video will use this exact prepared image.</p>
<div className="row">
<a href={preview.url} download="prepared-frame.png">Download image</a>
- <button className="secondary" disabled={busy || !video || !picture || !size} onClick={() => prepare((seed + 1) % (MAX_SEED + 1))}>Generate another image</button>
+ <button className="secondary" disabled={busy || !ready} onClick={() => prepare((seed + 1) % (MAX_SEED + 1))}>Generate another image</button>
<button className="primary" disabled={busy} onClick={() => { setApproved(true); remember(null, prepared, true); }}>Use this image</button>
</div>
</section>}
diff --git a/lib/picture.ts b/lib/picture.ts
index e3f4080..ba4f999 100644
--- a/lib/picture.ts
+++ b/lib/picture.ts
@@ -106,20 +106,26 @@ export const LIGHT_LOOKS = [
{ name: "Red room", image: "/relight/red_room.webp", light: "Light the new photo with deep crimson red light filling the whole scene, dark red shadows and a red-black background, and one small bright white light at the left edge of the frame that puts a cold white highlight on the near cheek and in the eyes." },
];
+export const FRAME_RELIGHT_PROMPT = "Relight this photograph. Keep the person, face, identity, expression, pose, hair, clothing, framing and setting exactly as they are; change only the lighting and the colour of the light. {look}";
+
export const RELIGHT_PROMPT = "Create a photograph of the person from <image 1> in exactly the pose, head angle, hand and arm positions, camera angle, framing and body cut of the person in <image 2>; describe that pose fully. The person from <image 1> also makes exactly the facial expression of the person in <image 2>, with the same mouth shape, eyebrows and gaze; describe that expression instead of their usual one. Take only the pose, expression and framing from <image 2>, never its person, face shape, hair or clothing. Keep the face, identity, eye colour, hair and clothing of <image 1>. {look}";
const KLEIN_UPSAMPLER = "You are the prompt upsampler of an image-editing model. You see the reference image(s), attached in order (the first image, the second image, ...), and the user's short request. Expand it into ONE detailed, self-contained instruction: the caption the editing model receives together with the same images. Output only the instruction as one continuous English prose block, with no greetings, explanations, Markdown, JSON, coordinates, ids or angle-bracket tokens. Read a vague or conversational request as the most plausible edit of the supplied images.\n\nPick the voice by task shape. When one reference stays and gets bounded changes, write an imperative editing instruction on it (\"Remove the ...\", \"Change the ... to ...\", \"Zoom in on ...\"); a whole-image transformation opens with its scope (\"Convert the entire image into ...\", \"Apply a ... filter over the entire scene\", \"Horizontally flip the entire image so that ...\", \"Shift the camera viewpoint to ...\", then where the major elements should now be). When the references are drawn into a newly composed frame (combining subjects, a new scene, a garment, object or style donor), write a generation request (\"Create a photograph of ...\", \"Create an anime-style illustration of ...\", \"Generate a new image featuring ...\") that says what to take from each image, by ordinal, and what is newly generated.\n\nKeep every explicit edit, constraint, count, position, style cue and piece of wording, and decide what is left open as one concrete plan. Ground everything in what is actually visible: identify each referent unambiguously by its real appearance, never mention content that is not there, and use every image the user attached. State every end state concretely (which color, how large, which pose, which material) and give new content enough appearance to be generated. Cover the knock-on effects of the change (shadows, reflections, occlusion, perspective, scale, lighting) and say what fills a removed region. A zoom-in names what stays in frame and which neighbors are cropped out; an outpainting continues the scene in the same medium. Describe the change, not the whole scene, and close an editing instruction with one compact clause naming what stays unchanged; transferred people keep their identity, face and accessories unless asked otherwise. Transcribe on-image text verbatim in its original language inside double quotes, with line breaks as \\n, and match the original font, size and color when replacing it. Name recognizable public figures, brands and landmarks you can see. Directions are frame-relative; use no evaluative or quality words. Typical length is 50 to 150 words.\n\nOutput only the instruction.";
-// Pose and light in one FLUX pass, because stacked edits degrade the picture. FLUX sees only the character picture at 3 MP;
-// the raw frame reaches it through the upsampler's written pose and sets the canvas shape.
-export function applyRelight(graph: ApiGraph, light: string) {
+function relightRequest(graph: ApiGraph, light: string) {
const request = String(graph["6"].inputs.value);
graph["6"].inputs.value = request.includes("{look}") ? request.replaceAll("{look}", light) : `${request} ${light}`;
- delete graph["5"];
- graph["7"] = { class_type: "BatchImagesNode", inputs: { "images.image0": ["4", 0], "images.image1": ["3", 0] }, _meta: { title: "Character picture + raw frame for the prompt upsampler" } };
graph["25"].inputs.value = KLEIN_UPSAMPLER;
Object.assign(graph["23"].inputs, { max_length: 3584, sampling_mode: "on", "sampling_mode.temperature": 1, "sampling_mode.top_k": 1, "sampling_mode.top_p": 0.95,
"sampling_mode.min_p": 0, "sampling_mode.repetition_penalty": 1.05, "sampling_mode.seed": 0 });
+}
+
+// Pose and light in one FLUX pass, because stacked edits degrade the picture. FLUX sees only the character picture at 3 MP;
+// the raw frame reaches it through the upsampler's written pose and sets the canvas shape.
+export function applyRelight(graph: ApiGraph, light: string) {
+ relightRequest(graph, light);
+ delete graph["5"];
+ graph["7"] = { class_type: "BatchImagesNode", inputs: { "images.image0": ["4", 0], "images.image1": ["3", 0] }, _meta: { title: "Character picture + raw frame for the prompt upsampler" } };
const { positive, negative, vae } = graph["15"].inputs;
graph["15"] = { class_type: "ReferenceMultiLatent", inputs: { positive, negative, vae, resolution: "resize", "resolution.megapixels": 3, "images.image0": ["4", 0] }, _meta: { title: "Character picture only, 3 MP" } };
graph["relight_canvas"] = { class_type: "ImageScaleToTotalPixels", inputs: { image: ["3", 0], upscale_method: "lanczos", megapixels: 3, resolution_steps: 16 }, _meta: { title: "Frame shape at 3 MP" } };
@@ -128,3 +134,13 @@ export function applyRelight(graph: ApiGraph, light: string) {
graph["19"].inputs.latent_image = ["relight_latent", 0];
graph["21"].inputs.filename_prefix = "openjutsu_picture/flux3/relit_frame";
}
+
+// Relights the video's own person: FLUX edits the raw frame at 3 MP and keeps its person, pose and setting.
+export function applyFrameRelight(graph: ApiGraph, light: string) {
+ relightRequest(graph, light);
+ for (const id of ["4", "5", "7"]) delete graph[id];
+ graph["8"].inputs.image = ["2", 0];
+ const { positive, negative, vae } = graph["15"].inputs;
+ graph["15"] = { class_type: "ReferenceMultiLatent", inputs: { positive, negative, vae, resolution: "resize", "resolution.megapixels": 3, "images.image0": ["2", 0] }, _meta: { title: "Raw frame only, 3 MP" } };
+ graph["21"].inputs.filename_prefix = "openjutsu_picture/flux3/relit_video_frame";
+}
diff --git a/test/picture-pipeline.test.ts b/test/picture-pipeline.test.ts
index aed40b8..b335861 100644
--- a/test/picture-pipeline.test.ts
+++ b/test/picture-pipeline.test.ts
@@ -2,7 +2,7 @@ import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import test from "node:test";
import { isLink, type ApiGraph } from "../lib/simple.ts";
-import { applyRelight, LIGHT_LOOKS, RELIGHT_PROMPT } from "../lib/picture.ts";
+import { applyFrameRelight, applyRelight, FRAME_RELIGHT_PROMPT, LIGHT_LOOKS, RELIGHT_PROMPT } from "../lib/picture.ts";
const load = (stage: string): ApiGraph => JSON.parse(readFileSync(new URL(`../workflows/picture/from_picture_${stage}_flux3.api.json`, import.meta.url), "utf8"));
const prompt = "change <image 1> character pose follow the <image 2> pose. keep the final image as same body cut as <image 2> . do not change the background of <image 1> and do not add anything extra in <image 1>.";
@@ -58,10 +58,24 @@ test("relight gives FLUX only the character picture at 3 MP on the frame's shape
assert.ok(String(custom["6"].inputs.value).endsWith(" Light the new photo with candles."));
});
+test("relighting the video's own person gives FLUX and the upsampler only the raw frame", () => {
+ const graph = load("prepare");
+ graph["6"].inputs.value = FRAME_RELIGHT_PROMPT;
+ applyFrameRelight(graph, LIGHT_LOOKS[7].light);
+ assert.ok(String(graph["6"].inputs.value).endsWith(LIGHT_LOOKS[7].light));
+ assert.equal(graph["4"], undefined);
+ assert.deepEqual(graph["8"].inputs.image, ["2", 0]);
+ assert.deepEqual(graph["15"].inputs["images.image0"], ["2", 0]);
+ assert.equal(graph["15"].inputs["images.image1"], undefined);
+ assert.equal(graph["15"].inputs["resolution.megapixels"], 3);
+ assert.deepEqual(graph["19"].inputs.latent_image, ["15", 2]);
+});
+
test("picture pipelines have valid links and no disconnected processing nodes", () => {
- const relit = load("prepare");
+ const relit = load("prepare"), frame = load("prepare");
applyRelight(relit, LIGHT_LOOKS[0].light);
- for (const [stage, graph] of [...["prepare", "motion", "solar"].map((stage) => [stage, load(stage)] as const), ["relight", relit] as const]) {
+ applyFrameRelight(frame, LIGHT_LOOKS[0].light);
+ for (const [stage, graph] of [...["prepare", "motion", "solar"].map((stage) => [stage, load(stage)] as const), ["relight", relit] as const, ["frame relight", frame] as const]) {
const reached = new Set<string>();
function visit(id: string) {
if (reached.has(id)) return;
diff --git a/workflows/picture/README.md b/workflows/picture/README.md
index 40c4b38..19bbb35 100644
--- a/workflows/picture/README.md
+++ b/workflows/picture/README.md
@@ -46,6 +46,8 @@ The image-aware prompt upsampler sees both references and expands that request i
**Relight** (off by default) poses and relights the character in the same FLUX pass; a second edit on top of the prepared picture degrades it. Pick one of nine light looks or write your own. The app then sends the prepare workflow with these changes (`applyRelight` in `lib/picture.ts`): FLUX gets only image 1, resized to 3 MP; image 2 reaches it only through the upsampler's written pose; the canvas is image 2's shape at 3 MP; the request's `{look}` is replaced by the light. The light usually changes the background too, and results vary a lot by seed. Relit pictures save as `relit_frame_*.png`.
+**Relight the video's own person** turns the character picture off. FLUX gets only the raw frame at 3 MP, the upsampler sees the same frame, and the request asks for the same person, pose, framing and setting under the chosen light. The relit frame is saved as `relit_video_frame_*` and renders like any prepared picture.
+
Preparation defaults to a 768×1376 canvas and saves under `ComfyUI/output/openjutsu_picture/flux3/prepared_frame_*.png`. Compare the prepared picture against the raw first-frame preview before using it for video; a successful execution does not guarantee that the requested pose was followed.
The **motion** and **solar** variants wire the decoded prepared picture directly into H3. No manual re-upload is needed. They default to **141 frames / 5.875 seconds at 24 fps**, 480×864, four SolarRenderer steps or six H3/DMAD steps, and the original soundtrack (or silence if absent). Both renderers let you select up to 15 seconds, with no context loop. It rounds down to the H3 17k+5 frame grid: 6 seconds becomes 141 frames / 5.875 seconds; 15 seconds becomes 345 frames / 14.375 seconds. The duration shown in the settings is the actual render length. Models, LoRAs (including individual switches and strengths), H3 text encoder, VAEs, steps and seed can be changed before rendering. These settings persist across refreshes. Change `frame_load_cap` and H3's `length` together to select another valid H3 clip length.