Skip to content

One-click character swap for videos: an open-source, local Higgsfield Genjutsu alternative built on ComfyUI and MiniMax H3, with FLUX.3 Klein relighting.

Add Fix colours post-processing mode

D3voz44182bb3cec9
diff --git a/README.md b/README.md
index 7023d1c..78f84fc 100644
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@ The app is a small Next.js front end over a ComfyUI workflow. It uses [`@comfyor
- **Simple motion transfer**: available. One character replaces the main person (the one who fills the most of the frame). Runs `workflows/simple.api.json`, exported from `H3 One-Click Single Swap (DMAD 6 step).json`.
- **Advanced motion transfer**: available. 1–4 characters, each with an optional role; every tracked person is matched to a role per shot, and people without a character stay untouched. Runs `workflows/advanced.api.json`, exported from `H3 One-Click Advanced Swap (DMAD 6 step).json`; **One pass per person** (two characters) runs `workflows/advanced_maxq.api.json`.
- **Object swap**: available. **Change** what SAM tracks (hair, a jacket, a car) into what you describe or show in a picture; **Remove** it; or change the **Background** around it (`person` stays). Only that area is regenerated. **From picture** runs the Simple swap with the picture as the character: its face, hair and outfit, with the original background. Runs `workflows/object.api.json`, exported from `H3 One-Click Object Swap (DMAD 6 step).json`.
-- **Fix faces**: available. Post-processing for a finished video: H3 draws faces poorly when the person is small in the frame, so this redraws the face of the person in a photo at close-up size and pastes it back. Runs `workflows/face.api.json`, written by `user/default/openjutsu_face_build.py`.
+- **Post-processing**: available, for a finished video. **Fix faces**: H3 draws faces poorly when the person is small in the frame, so this redraws the face of the person in a photo at close-up size and pastes it back. Runs `workflows/face.api.json`, written by `user/default/openjutsu_face_build.py`. **Fix colours** softens H3's harsh look without an H3 run: Image Detail Contrast takes some fine detail contrast out, then Orion4D's DCTL Tone Mapper grades every frame. Runs `workflows/color.api.json`.
## Requirements
@@ -30,6 +30,8 @@ Install these into `ComfyUI/custom_nodes` (ComfyUI Manager finds the first three
| `comfyui-h3-oneclick-swap` | swap and object plans, people and role masks, grey mannequins, track labelling, cut finder, shot list, face-fix chunks (TODO: link the published pack) |
| [ComfyUI-H3-FaceRefine](https://github.com/Carasibana/ComfyUI-H3-FaceRefine) | Fix faces: face tracking and cropping, SAM face masks, pasting back |
| [ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) | Fix faces: SAM loader |
+| `comfyui-image-detail-contrast` | Fix colours: Image Detail Contrast (TODO: link the published pack) |
+| [Orion4D_FXMax](https://github.com/orion4d/Orion4D_FXMax) | Fix colours: DCTL Tone Mapper and its presets |
Restart ComfyUI after installing them.
@@ -90,6 +92,7 @@ A 6 s clip at 480p takes about 6–13 minutes on a 4060 Ti, most of it in H3 sam
- **Steps**, **Base model** and **LoRAs** (under Advanced) default to the H3 pruned INT8 model with the DMAD speed LoRA at 6 steps. LoRAs apply in order; without a speed LoRA, use 20 steps or more. Check mask ignores them.
- **Object swap:** name what SAM tracks in a word or two (`hair`, `jacket`, `poster`), or several separated by commas (`hair, top`) to change them together. Every match changes unless **Only the biggest one** is on (one of each thing). A picture of an outfit, hairstyle, product or place is copied onto it. For **Remove**, say what shows instead only when it is not obvious (`a plain wall`). Mask margin is smaller here (10, 20 or 40 px). Change keeps the outline of what it replaces: a tank top turned into a flannel shirt stays sleeveless.
- **Fix faces:** drop the finished video and the photo the person was made from. **Match the photo** redraws the face from the photo: the closest likeness, but H3 makes up the expressions and mouth movements. **Follow the mouth** keeps the video's expressions and mouth movements, with a little less likeness. Only that person's face changes (matched by face identity); faces taller than a fifth of the frame are left alone, and stretches with only such faces are skipped. A 17 s video at 768p (4 chunks) took about 28 minutes on a 4060 Ti, including model loading. The video is read at 24 fps.
+- **Fix colours:** drop the finished video. **Detail contrast** (-0.20 by default) softens crunchy fine detail; above 0 it sharpens instead. The **Tone mapper** starts from the **H3 zuanfilm** preset (Reinhard at 30% strength: lifted blacks, softer contrast and highlights, a little less saturation). The preset list is Orion's saved presets, read from ComfyUI; changing a value under Settings makes it Custom, and the values are sent with the job, so Orion's preset files are not changed. Save `H3 zuanfilm.json` in Orion's `presets/color_fx_dctl_tone_mapper` folder to have it in the list; without it the page uses its built-in copy. A 1.75 s 480p clip took 5 s. The video is read at 24 fps.
- **Advanced mode:** give roles as `the drummer`, optionally with how they look after ` - ` (`the drummer - man in a cap`); the look only helps match people and never reaches H3. **One pass per person** gives each of two characters their own H3 pass with only their photo (about twice the time). Check mask is Simple-only for now.
## How it works
diff --git a/app/api/jobs/route.ts b/app/api/jobs/route.ts
index 6cb733c..631d2e1 100644
--- a/app/api/jobs/route.ts
+++ b/app/api/jobs/route.ts
@@ -1,5 +1,6 @@
import { NextResponse } from "next/server";
-import { listModels, submitAdvanced, submitFace, submitMask, submitObject, submitSimple } from "../../../lib/comfy.ts";
+import { listModels, submitAdvanced, submitColor, submitFace, submitMask, submitObject, submitSimple } from "../../../lib/comfy.ts";
+import { DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, type ColorSettings, type ToneMode, type ToneSettings, type ToneValue } from "../../../lib/color.ts";
import { CHUNK_SECONDS, FACE_STYLES, type FaceSettings, type FaceStyle } from "../../../lib/face.ts";
import { MAX_CHARACTERS } from "../../../lib/advanced.ts";
import { MAX_LORAS, STEPS, STRENGTH, type ModelSettings } from "../../../lib/model.ts";
@@ -97,6 +98,27 @@ function parseFace(form: FormData) {
return { picture, settings };
}
+// A step without its fields is turned off.
+function parseColor(form: FormData): ColorSettings {
+ const detailEntry = form.get("detail");
+ const detail = detailEntry === null ? null : Number(detailEntry);
+ if (detail !== null && !(detail >= DETAIL.min && detail <= DETAIL.max)) throw new Error(`Detail contrast must be ${DETAIL.min} to ${DETAIL.max}.`);
+ const mode = form.get("tone_mode");
+ let tone: ToneSettings | null = null;
+ if (mode !== null) {
+ if (!TONE_MODES.includes(mode as ToneMode)) throw new Error("Choose a supported tone mapper mode.");
+ tone = { ...DEFAULT_TONE, mode: mode as ToneMode };
+ for (const key of Object.keys(TONE_RANGES) as ToneValue[]) {
+ const range = TONE_RANGES[key];
+ const value = Number(form.get(`tone_${key}`));
+ if (!(value >= range.min && value <= range.max)) throw new Error(`${range.label} must be ${range.min} to ${range.max}.`);
+ tone[key] = value;
+ }
+ }
+ if (detail === null && tone === null) throw new Error("Turn on detail contrast, the tone mapper or both.");
+ return { detail, tone };
+}
+
function parseSimple(form: FormData) {
const picture = parsePicture(form.get("picture"));
const settings: SimpleSettings = { role: String(form.get("role") ?? "").slice(0, 200), noLipSync: form.get("no_lip_sync") === "true", ...parseRun(form), ...parseMask(form) };
@@ -138,6 +160,9 @@ export async function POST(request: Request) {
const { picture, settings } = parseFace(form);
const model = parseModel(form, await listModels());
submit = () => submitFace(video, section, picture, settings, model);
+ } else if (mode === "color") {
+ const settings = parseColor(form);
+ submit = () => submitColor(video, section, settings);
} else {
throw new Error("Unknown mode.");
}
diff --git a/app/api/presets/route.ts b/app/api/presets/route.ts
new file mode 100644
index 0000000..e1dfbbc
--- /dev/null
+++ b/app/api/presets/route.ts
@@ -0,0 +1,9 @@
+import { NextResponse } from "next/server";
+import { listTonePresets } from "../../../lib/comfy.ts";
+
+export const runtime = "nodejs";
+export const dynamic = "force-dynamic";
+
+export async function GET() {
+ return NextResponse.json({ tone: await listTonePresets() });
+}
diff --git a/components/openjutsu.tsx b/components/openjutsu.tsx
index 7d0faef..1d14628 100644
--- a/components/openjutsu.tsx
+++ b/components/openjutsu.tsx
@@ -9,10 +9,13 @@ import { MAX_CHARACTERS } from "../lib/advanced.ts";
import { DEFAULT_MODEL, MAX_LORAS, STEPS, STRENGTH, type Lora } from "../lib/model.ts";
import type { EditMode } from "../lib/object.ts";
import { CHUNK_SECONDS, faceChunkCount, type FaceStyle } from "../lib/face.ts";
+import { DEFAULT_PRESET, DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, toneFromPreset, type ToneSettings, type ToneValue } from "../lib/color.ts";
type Character = { file: File; url: string; role: string };
-type Mode = "simple" | "advanced" | "object" | "face";
-type Kind = "simple" | "mask" | "advanced" | "advanced_maxq" | "object" | "face";
+type Mode = "simple" | "advanced" | "object" | "post";
+type Post = "face" | "color";
+type Kind = "simple" | "mask" | "advanced" | "advanced_maxq" | "object" | "face" | "color";
+type TonePreset = { name: string; settings: Record<string, unknown> };
type Run = { id: string; kind: Kind; margin: Margin; clips: number; clipSeconds: number; startedAt: number };
// pass counts the sampler runs started so far: Fix faces samples its chunks one after another in the same node.
type Progress = NodeInfo & { step: number | null; steps: number | null; pass: number };
@@ -24,9 +27,11 @@ const MODES = [
{ id: "simple", title: "Simple motion transfer", text: "Swap the main person in a video for your character.", ready: true },
{ id: "advanced", title: "Advanced motion transfer", text: "Several people, pick who becomes who.", ready: true },
{ id: "object", title: "Object swap", text: "Change hair, clothes, a prop or the background, or remove something.", ready: true },
- { id: "face", title: "Fix faces", text: "Post-processing: redraw a person's face in a finished video from their photo.", ready: true },
+ { id: "post", title: "Post-processing", text: "Fix the faces or the colours of a finished video.", ready: true },
] as const;
+const POSTS: { id: Post; label: string }[] = [{ id: "face", label: "Fix faces" }, { id: "color", label: "Fix colours" }];
+
const FACE_STYLE_TEXT: Record<FaceStyle, { label: string; hint: string }> = {
photo: { label: "Match the photo", hint: "The face is redrawn from the photo: the closest likeness, but H3 makes up its own expressions and mouth movements." },
mouth: { label: "Follow the mouth", hint: "The face follows the video's expressions and mouth movements; a little less like the photo." },
@@ -88,6 +93,9 @@ function clipShare(progress: Progress) {
return 0.05;
}
+// Rough share of a colour fix done when each node starts; the tone mapper runs on the CPU and takes longest.
+const COLOR_SHARE: Record<string, number> = { ImageDetailContrast: 0.15, Orion4D_FXMax_DCTLToneMapper: 0.3, VHS_VideoCombine: 0.9 };
+
// A picker over ComfyUI's files, or a text field when the list could not be read.
function FilePicker({ value, options, onChange, label }: { value: string; options: string[]; onChange: (value: string) => void; label: string }) {
if (!options.length) return <input className="file-name" aria-label={label} value={value} onChange={(event) => onChange(event.target.value)} />;
@@ -143,6 +151,14 @@ export function Openjutsu() {
const [changeTo, setChangeTo] = useState("");
const [look, setLook] = useState<{ file: File; url: string } | null>(null);
const [biggestOnly, setBiggestOnly] = useState(false);
+ const [post, setPost] = useState<Post>("face");
+ const [detailOn, setDetailOn] = useState(true);
+ const [detail, setDetail] = useState(DETAIL.default);
+ const [toneOn, setToneOn] = useState(true);
+ const [tone, setTone] = useState<ToneSettings>(DEFAULT_TONE);
+ // "" once a value is changed by hand
+ const [tonePreset, setTonePreset] = useState(DEFAULT_PRESET);
+ const [presets, setPresets] = useState<TonePreset[]>([]);
const [faceStyle, setFaceStyle] = useState<FaceStyle>("photo");
const [chunkSeconds, setChunkSeconds] = useState(CHUNK_SECONDS.default);
const [seed, setSeed] = useState(0);
@@ -166,6 +182,12 @@ export function Openjutsu() {
useEffect(() => {
fetch("/api/models").then((response) => response.json()).then(setFiles, () => {});
+ // Orion's own copy of the default preset wins over the built-in values once it has loaded.
+ fetch("/api/presets").then((response) => response.json()).then(({ tone }: { tone: TonePreset[] }) => {
+ setPresets(tone);
+ const saved = tone.find((p) => p.name === DEFAULT_PRESET);
+ if (saved) setTone(toneFromPreset(saved.settings));
+ }, () => {});
}, []);
useEffect(() => {
@@ -226,8 +248,20 @@ export function Openjutsu() {
const whole = !video?.duration || (part.start === 0 && part.length >= video.duration);
const place = whole ? "Whole" : PLACES.slice(1).find((p) => Math.abs(placeStart(p, video!.duration, part.length) - part.start) < 0.05);
+ const faceFix = mode === "post" && post === "face";
+ const colorFix = mode === "post" && post === "color";
+ // Orion's saved presets, with zuanfilm's H3 preset first when Orion does not have it.
+ const presetOptions: TonePreset[] = presets.some((p) => p.name === DEFAULT_PRESET) ? presets : [{ name: DEFAULT_PRESET, settings: DEFAULT_TONE }, ...presets];
+
+ function choosePreset(name: string) {
+ const preset = presetOptions.find((p) => p.name === name);
+ if (!preset) return;
+ setTone(toneFromPreset(preset.settings));
+ setTonePreset(name);
+ }
+
// Simple mode and Fix faces use only the first character; the others are kept for switching back.
- const single = mode === "simple" || mode === "face";
+ const single = mode === "simple" || mode === "post";
const limit = single ? 1 : MAX_CHARACTERS;
const shown = characters.slice(0, limit);
const character = characters[0] ?? null;
@@ -253,10 +287,10 @@ export function Openjutsu() {
setCharacters(characters.map((c, i) => (i === index ? { ...c, role } : c)));
}
- const clips = !video?.duration ? 0 : mode === "face" ? faceChunkCount(part.length || video.duration, chunkSeconds) : clipCount(part.length || video.duration, clipSeconds);
+ const clips = !video?.duration ? 0 : faceFix ? faceChunkCount(part.length || video.duration, chunkSeconds) : clipCount(part.length || video.duration, clipSeconds);
const edit = EDITS.find((e) => e.id === editMode)!;
const fromPicture = mode === "object" && editMode === "picture";
- const ready = mode === "object"
+ const ready = colorFix ? detailOn || toneOn : mode === "object"
? editMode === "remove" || (fromPicture ? !!look : !!changeTo.trim() || !!look)
: !!character;
@@ -310,6 +344,13 @@ export function Openjutsu() {
form.set("seed", String(seed));
form.set("clip_seconds", String(chunkSeconds));
}
+ if (kind === "color") {
+ if (detailOn) form.set("detail", String(detail));
+ if (toneOn) {
+ form.set("tone_mode", tone.mode);
+ for (const key of Object.keys(TONE_RANGES) as ToneValue[]) form.set(`tone_${key}`, String(tone[key]));
+ }
+ }
if (kind === "object") {
form.set("edit_mode", editMode);
form.set("target", editTarget.trim() || edit.target);
@@ -320,7 +361,7 @@ export function Openjutsu() {
form.set("seed", String(seed));
form.set("clip_seconds", String(clipSeconds));
}
- if (kind !== "mask") {
+ if (kind !== "mask" && kind !== "color") {
form.set("motion", String(motion));
form.set("steps", String(steps));
form.set("model", model);
@@ -438,8 +479,38 @@ export function Openjutsu() {
</div>
<div className="panel">
- <h2><span>2</span> {mode === "object" ? "What to change" : mode === "face" ? "Whose face" : mode === "simple" ? "Character" : "Characters"}</h2>
- {mode === "object" ? (
+ <h2><span>2</span> {mode === "object" ? "What to change" : colorFix ? "Colour fix" : faceFix ? "Whose face" : mode === "simple" ? "Character" : "Characters"}</h2>
+ {mode === "post" && (
+ <div className="segmented">
+ {POSTS.map((p) => (
+ <button key={p.id} className={p.id === post ? "on" : ""} onClick={() => setPost(p.id)}>{p.label}</button>
+ ))}
+ </div>
+ )}
+ {colorFix ? (<>
+ <label className="setting toggle">
+ <span>Detail contrast <small>Softens H3&apos;s crunchy fine detail and over-sharpened skin, without blurring edges.</small></span>
+ <input type="checkbox" checked={detailOn} onChange={(event) => setDetailOn(event.target.checked)} />
+ <i aria-hidden />
+ </label>
+ {detailOn && <label className="setting">
+ <span>Amount <small>{detail.toFixed(2)}: below 0 softens, above 0 sharpens.</small></span>
+ <input type="range" min={DETAIL.min} max={DETAIL.max} step={0.01} value={detail} onChange={(event) => setDetail(Number(event.target.value))} />
+ </label>}
+ <label className="setting toggle">
+ <span>Tone mapper <small>Orion4D DCTL Tone Mapper, a colour grade over every frame. {DEFAULT_PRESET} lifts the blacks, softens contrast and highlights and takes a little saturation out.</small></span>
+ <input type="checkbox" checked={toneOn} onChange={(event) => setToneOn(event.target.checked)} />
+ <i aria-hidden />
+ </label>
+ {toneOn && <div className="setting">
+ <span>Preset <small>Orion&apos;s saved presets. Change the values under Settings to make your own.</small></span>
+ <select className="file-name" aria-label="Tone mapper preset" value={tonePreset} onChange={(event) => choosePreset(event.target.value)}>
+ {!tonePreset && <option value="">Custom</option>}
+ {presetOptions.map((p) => <option key={p.name} value={p.name}>{p.name}</option>)}
+ </select>
+ </div>}
+ <p className="hint">No H3 run: the video is only filtered, so it keeps its frames and soundtrack. The detail contrast runs first, then the tone mapper.</p>
+ </>) : mode === "object" ? (
<div className="edit">
<div className="segmented">
{EDITS.map((e) => (
@@ -478,7 +549,7 @@ export function Openjutsu() {
<div className="character" key={c.url}>
<img src={c.url} alt={`Character ${i + 1}`} />
<button className="remove" onClick={() => removeCharacter(i)} aria-label={`Remove character ${i + 1}`}>×</button>
- {mode !== "face" && <label>
+ {mode !== "post" && <label>
<span>Who do they replace? (optional)</span>
<input
value={c.role}
@@ -496,7 +567,7 @@ export function Openjutsu() {
</DropZone>
)}
</div>
- {mode === "face" ? (<>
+ {faceFix ? (<>
<div className="segmented">
{(Object.keys(FACE_STYLE_TEXT) as FaceStyle[]).map((f) => (
<button key={f} className={f === faceStyle ? "on" : ""} onClick={() => setFaceStyle(f)}>{FACE_STYLE_TEXT[f].label}</button>
@@ -524,9 +595,32 @@ export function Openjutsu() {
</div>
</section>
+ {colorFix ? toneOn && (
+ <section className="panel settings">
+ <h2><span>3</span> Settings <small>{tonePreset || "Custom"}</small></h2>
+ <div className="setting">
+ <span>Tone mapper mode</span>
+ <div className="segmented">
+ {TONE_MODES.map((m) => (
+ <button key={m} className={m === tone.mode ? "on" : ""} onClick={() => { setTone({ ...tone, mode: m }); setTonePreset(""); }}>{m}</button>
+ ))}
+ </div>
+ </div>
+ {(Object.keys(TONE_RANGES) as ToneValue[]).map((key) => (
+ <label className="setting" key={key}>
+ <span>{TONE_RANGES[key].label} <small>{tone[key].toFixed(2)}</small></span>
+ <input
+ type="range" min={TONE_RANGES[key].min} max={TONE_RANGES[key].max} step={TONE_RANGES[key].step} value={tone[key]}
+ onChange={(event) => { setTone({ ...tone, [key]: Number(event.target.value) }); setTonePreset(""); }}
+ />
+ </label>
+ ))}
+ <p className="hint">Strength mixes the grade with the original: 0.3 is 30% of it. Pick the preset again to undo your changes.</p>
+ </section>
+ ) : (
<section className="panel settings">
<h2><span>3</span> Settings</h2>
- {mode !== "face" && <div className="setting">
+ {mode !== "post" && <div className="setting">
<span>Resolution</span>
<div className="segmented">
{RESOLUTIONS.map((r) => (
@@ -564,7 +658,7 @@ export function Openjutsu() {
<i aria-hidden />
</label>
)}
- {mode !== "face" && <div className="setting">
+ {mode !== "post" && <div className="setting">
<span>Mask margin <small>{mode === "object"
? editMode === "background" ? "How far the new place reaches into the edge of what stays." : "Room around it for a bigger new shape and its shadow."
: <>Room around the person for a differently built character.{mode === "simple" ? " Check mask shows it." : ""}</>}</small></span>
@@ -579,7 +673,7 @@ export function Openjutsu() {
<button onClick={() => setSeed(Math.floor(Math.random() * MAX_SEED))} aria-label="Random seed">🎲</button>
</div>
</div>
- {mode === "face" ? (
+ {faceFix ? (
<div className="setting">
<span>Chunk length <small>{chunkSeconds.toFixed(1)} s of face per H3 generation; chunks overlap by 1 s and are crossfaded.</small></span>
<input type="range" min={CHUNK_SECONDS.min} max={CHUNK_SECONDS.max} step={0.5} value={chunkSeconds} onChange={(event) => setChunkSeconds(Number(event.target.value))} />
@@ -590,7 +684,7 @@ export function Openjutsu() {
<input type="range" min={CLIP_SECONDS.min} max={CLIP_SECONDS.max} step={0.5} value={clipSeconds} onChange={(event) => setClipSeconds(Number(event.target.value))} />
</div>
)}
- {mode !== "face" && <div className="setting">
+ {mode !== "post" && <div className="setting">
<span>Motion reference <small>Size of the grey figures H3 copies the motion from. Bigger follows the source poses a bit more closely; sampling takes about a third longer at 768 px and about 2.4× as long at 1376 px.</small></span>
<div className="segmented">
{MOTION_SIZES.map((m) => (
@@ -629,18 +723,21 @@ export function Openjutsu() {
</div>
</details>
</section>
+ )}
<div className="go">
{formError && <p className="error">{formError}</p>}
<div className="row">
{mode === "simple" && <button className="secondary" disabled={!video || submitting} onClick={() => submit("mask")}>Check mask</button>}
- <button className="primary" disabled={!video || !ready || submitting} onClick={() => submit(mode === "object" ? fromPicture ? "simple" : "object" : mode === "face" ? "face" : mode === "simple" ? "simple" : onePassEach ? "advanced_maxq" : "advanced")}>
+ <button className="primary" disabled={!video || !ready || submitting} onClick={() => submit(mode === "object" ? fromPicture ? "simple" : "object" : mode === "post" ? post : mode === "simple" ? "simple" : onePassEach ? "advanced_maxq" : "advanced")}>
{submitting ? "Uploading…" : "Generate"}
</button>
</div>
- <small>{clips && ready
- ? mode === "face" ? `Up to ${clips} chunk${clips > 1 ? "s" : ""} of face` : `${clips} clip${clips > 1 ? "s" : ""} at ${resolution}`
- : `Add a video and ${mode === "object" ? "say what it becomes" : mode === "face" ? "the photo of the person" : mode === "simple" ? "a character photo" : "character photos"}.`}</small>
+ <small>{colorFix
+ ? !video ? "Add a video." : ready ? `${(part.length || video.duration).toFixed(1)} s of video` : "Turn on detail contrast, the tone mapper or both."
+ : clips && ready
+ ? faceFix ? `Up to ${clips} chunk${clips > 1 ? "s" : ""} of face` : `${clips} clip${clips > 1 ? "s" : ""} at ${resolution}`
+ : `Add a video and ${mode === "object" ? "say what it becomes" : faceFix ? "the photo of the person" : mode === "simple" ? "a character photo" : "character photos"}.`}</small>
</div>
</>
)}
@@ -672,7 +769,7 @@ function RunView({ run, job, progress, preview, streamError, now, onCancel, onRe
<video src={job.video.url} controls autoPlay playsInline />
<div className="row">
<a className="primary" href={`${job.video.url}?download`} download={job.video.name}>Download MP4</a>
- <button className="secondary" onClick={onReset}>{run.kind === "object" ? "New edit" : run.kind === "face" ? "Fix another" : "New swap"}</button>
+ <button className="secondary" onClick={onReset}>{run.kind === "object" ? "New edit" : run.kind === "face" || run.kind === "color" ? "Fix another" : "New swap"}</button>
</div>
</section>
);
@@ -683,17 +780,19 @@ function RunView({ run, job, progress, preview, streamError, now, onCancel, onRe
<section className="panel">
<h2>{status === "canceled" ? "Canceled" : status === "succeeded" ? "Finished without a video" : status === "expired" ? "Job no longer available" : "Something went wrong"}</h2>
{job?.error && <p className="error">{job.error}</p>}
- {status === "expired" && <p className="hint">ComfyUI no longer knows this job, for example after a restart. Finished videos are in ComfyUI&apos;s output/{run.kind === "object" ? "h3_oneclick_edit" : run.kind === "face" ? "openjutsu_face" : "h3_oneclick_swap"} folder.</p>}
+ {status === "expired" && <p className="hint">ComfyUI no longer knows this job, for example after a restart. Finished videos are in ComfyUI&apos;s output/{run.kind === "object" ? "h3_oneclick_edit" : run.kind === "face" ? "openjutsu_face" : run.kind === "color" ? "openjutsu_color" : "h3_oneclick_swap"} folder.</p>}
<button className="secondary" onClick={onReset}>Back</button>
</section>
);
}
const face = run.kind === "face";
+ const color = run.kind === "color";
const sampling = progress?.classType === "SamplerCustomAdvanced";
const chunk = Math.min(progress?.pass ?? 0, run.clips);
let fraction = 0;
- if (face) fraction = sampling ? 0.1 + 0.85 * (chunk - 1 + (progress!.steps ? (progress!.step ?? 0) / progress!.steps : 0)) / run.clips : chunk ? 0.97 : 0.05;
+ if (color) fraction = COLOR_SHARE[progress?.classType ?? ""] ?? 0.05;
+ else if (face) fraction = sampling ? 0.1 + 0.85 * (chunk - 1 + (progress!.steps ? (progress!.step ?? 0) / progress!.steps : 0)) / run.clips : chunk ? 0.97 : 0.05;
else if (mask) fraction = progress?.classType === "SAM3_VideoTrack" && progress.steps ? (progress.step ?? 0) / progress.steps : 0;
else if (progress?.classType === "MiniMaxH3ChainAssemble") fraction = 0.99;
else if (progress?.clip) fraction = (progress.clip - 1 + clipShare(progress)) / clips;
@@ -701,7 +800,7 @@ function RunView({ run, job, progress, preview, streamError, now, onCancel, onRe
return (
<section className="panel progress">
<div className="row">
- <h2>{status === "queued" ? `Queued${job?.queuePosition ? ` (#${job.queuePosition})` : ""}` : mask ? "Checking the mask"
+ <h2>{status === "queued" ? `Queued${job?.queuePosition ? ` (#${job.queuePosition})` : ""}` : mask ? "Checking the mask" : color ? "Fixing the colours"
: face ? (sampling ? `Chunk ${chunk} of up to ${run.clips}` : chunk ? "Pasting the face back" : "Tracking the face")
: progress?.clip ? `Clip ${progress.clip} of ${clips}` : "Preparing"}</h2>
<span className="time">{elapsed(now - run.startedAt)}</span>
@@ -713,7 +812,7 @@ function RunView({ run, job, progress, preview, streamError, now, onCancel, onRe
</p>
{preview && <img className="preview" src={preview} alt="Mask preview: white areas are regenerated" />}
{streamError && <p className="error">{streamError}</p>}
- <p className="hint">{mask ? "SAM tracks the whole video once, about a minute per 3 s of video." : face ? "The face is tracked over the whole video, then each chunk is regenerated; nothing is saved until the end." : "Each clip is saved as it finishes, so a rerun with the same inputs resumes where it stopped."} You can close this page and come back.</p>
+ <p className="hint">{mask ? "SAM tracks the whole video once, about a minute per 3 s of video." : color ? "Every frame goes through the filters, then the video is saved with its soundtrack." : face ? "The face is tracked over the whole video, then each chunk is regenerated; nothing is saved until the end." : "Each clip is saved as it finishes, so a rerun with the same inputs resumes where it stopped."} You can close this page and come back.</p>
<button className="secondary" onClick={onCancel}>Cancel</button>
</section>
);
diff --git a/lib/color.ts b/lib/color.ts
new file mode 100644
index 0000000..aaf1e1b
--- /dev/null
+++ b/lib/color.ts
@@ -0,0 +1,72 @@
+// Fix colours: post-processing for a finished video. Image Detail Contrast (F:\AI\comfyui-image-detail-contrast) softens
+// H3's crunchy fine detail, then Orion4D's DCTL Tone Mapper grades the frames: by default zuanfilm's H3 preset, which
+// lifts the blacks, softens contrast and highlights and takes a little saturation out. workflows/color.api.json is
+// hand-written; each request sets the values and drops a step that is turned off. Nodes are found by title.
+import { nodeByTitle } from "./advanced.ts";
+import type { ApiGraph } from "./simple.ts";
+
+export const COLOR_VIDEO = "Video to fix";
+
+export const TONE_MODES = ["Filmic Soft", "Filmic Strong", "ACES Approx", "Reinhard", "Cineon-ish"] as const;
+export type ToneMode = (typeof TONE_MODES)[number];
+
+export type ToneSettings = {
+ mode: ToneMode; exposure: number; contrast: number; pivot: number; highlight_rolloff: number;
+ shadow_lift: number; black_floor: number; saturation: number; strength: number;
+};
+export type ToneValue = Exclude<keyof ToneSettings, "mode">;
+
+// Orion's own ranges for each value.
+export const TONE_RANGES: Record<ToneValue, { label: string; min: number; max: number; step: number }> = {
+ exposure: { label: "Exposure", min: -5, max: 5, step: 0.01 },
+ contrast: { label: "Contrast", min: 0, max: 3, step: 0.01 },
+ pivot: { label: "Pivot", min: 0.01, max: 1, step: 0.01 },
+ highlight_rolloff: { label: "Highlight rolloff", min: 0, max: 2, step: 0.01 },
+ shadow_lift: { label: "Shadow lift", min: -1, max: 1, step: 0.01 },
+ black_floor: { label: "Black floor", min: 0, max: 0.25, step: 0.01 },
+ saturation: { label: "Saturation", min: 0, max: 3, step: 0.01 },
+ strength: { label: "Strength", min: 0, max: 2, step: 0.01 },
+};
+
+export const DEFAULT_PRESET = "H3 zuanfilm";
+export const DEFAULT_TONE: ToneSettings = {
+ mode: "Reinhard", exposure: 0.09, contrast: 0.81, pivot: 0.69, highlight_rolloff: 0.27,
+ shadow_lift: 0.33, black_floor: 0.12, saturation: 0.93, strength: 0.3,
+};
+
+export const DETAIL = { min: -1, max: 1, default: -0.2 };
+
+// null turns a step off.
+export type ColorSettings = { detail: number | null; tone: ToneSettings | null };
+
+// An Orion preset as ComfyUI returns it, with the values this page does not show left out.
+export function toneFromPreset(settings: Record<string, unknown>): ToneSettings {
+ const tone = { ...DEFAULT_TONE };
+ if (TONE_MODES.includes(settings.mode as ToneMode)) tone.mode = settings.mode as ToneMode;
+ for (const key of Object.keys(TONE_RANGES) as ToneValue[]) {
+ const value = settings[key];
+ if (typeof value === "number") tone[key] = value;
+ }
+ return tone;
+}
+
+export function pruneColor(source: ApiGraph, settings: ColorSettings): ApiGraph {
+ const graph: ApiGraph = structuredClone(source);
+ const detail = nodeByTitle(graph, "Detail contrast");
+ const tone = nodeByTitle(graph, "Tone mapper");
+ if (settings.detail === null) {
+ graph[tone].inputs.image = graph[detail].inputs.image;
+ delete graph[detail];
+ } else {
+ graph[detail].inputs.detail_contrast = settings.detail;
+ }
+ if (settings.tone === null) {
+ graph[nodeByTitle(graph, "Fixed video")].inputs.images = graph[tone].inputs.image;
+ delete graph[tone];
+ } else {
+ // Orion uses raw_settings over its preset files when the preset names match.
+ const node = graph[tone].inputs;
+ node.raw_settings = JSON.stringify({ node_type: "color_fx_dctl_tone_mapper", preset: node.preset, settings: { ...settings.tone, preserve_luminosity: false, clamp_output: true } });
+ }
+ return graph;
+}
diff --git a/lib/comfy.ts b/lib/comfy.ts
index defb60e..5c4e309 100644
--- a/lib/comfy.ts
+++ b/lib/comfy.ts
@@ -7,6 +7,7 @@ import { VIDEO, nodeByTitle, pruneAdvanced, type AdvancedSettings } from "./adva
import { applyModel, type ModelSettings } from "./model.ts";
import { OBJECT_PICTURE, OBJECT_VIDEO, pruneObject, type ObjectSettings } from "./object.ts";
import { FACE_PICTURE, FACE_VIDEO, pruneFace, type FaceSettings } from "./face.ts";
+import { COLOR_VIDEO, pruneColor, type ColorSettings } from "./color.ts";
export type PublicJob = {
id: string;
@@ -35,7 +36,7 @@ export function rawJobEvents(job: Job, signal: AbortSignal) {
return low().getJobEvents(job.urls.events || job.id, { signal });
}
-export const JOB_KINDS = ["simple", "mask", "advanced", "advanced_maxq", "object", "face"] as const;
+export const JOB_KINDS = ["simple", "mask", "advanced", "advanced_maxq", "object", "face", "color"] as const;
export type JobKind = (typeof JOB_KINDS)[number];
export async function loadGraph(kind: JobKind): Promise<ApiGraph> {
@@ -115,6 +116,15 @@ export async function submitFace(video: File, section: Section, picture: File, s
return publicJob(await comfy.submit(workflow));
}
+export async function submitColor(video: File, section: Section, settings: ColorSettings) {
+ const comfy = client();
+ const graph = pruneColor(await loadGraph("color"), settings);
+ setSection(graph, nodeByTitle(graph, COLOR_VIDEO), section);
+ const workflow = comfy.workflows.fromJson(graph);
+ workflow.setInput(nodeByTitle(graph, COLOR_VIDEO), "video", await upload(comfy, video));
+ return publicJob(await comfy.submit(workflow));
+}
+
export async function submitMask(video: File, section: Section, hatFix: boolean, margin: Margin, target: string) {
const comfy = client();
const graph = pruneMask(await loadGraph("mask"), hatFix, margin, target);
@@ -124,12 +134,13 @@ export async function submitMask(video: File, section: Section, hatFix: boolean,
return publicJob(await comfy.submit(workflow));
}
+const COMFYUI_URL = process.env.COMFYUI_URL || "http://127.0.0.1:8188";
+
// The proxy has no model listing, so the pickers read ComfyUI's own node info. COMFYUI_URL points at ComfyUI itself
// (not the proxy); without it, or when it does not answer, the page falls back to typed file names.
export async function listModels(): Promise<{ models: string[]; loras: string[] } | null> {
- const base = process.env.COMFYUI_URL || "http://127.0.0.1:8188";
const options = async (node: string, input: string) => {
- const info = await (await fetch(`${base}/object_info/${node}`, { signal: AbortSignal.timeout(5000) })).json();
+ const info = await (await fetch(`${COMFYUI_URL}/object_info/${node}`, { signal: AbortSignal.timeout(5000) })).json();
return info[node].input.required[input][0] as string[];
};
try {
@@ -140,6 +151,17 @@ export async function listModels(): Promise<{ models: string[]; loras: string[]
}
}
+// Orion4D's saved DCTL Tone Mapper presets, read from ComfyUI like the model lists; empty when it does not answer.
+export async function listTonePresets(): Promise<{ name: string; settings: Record<string, unknown> }[]> {
+ const get = async (path: string) => (await fetch(`${COMFYUI_URL}/orion4d_fxmax/${path}`, { signal: AbortSignal.timeout(5000) })).json();
+ try {
+ const { presets } = await get("presets/color_fx_dctl_tone_mapper");
+ return await Promise.all((presets as string[]).map(async (name) => ({ name, settings: (await get(`preset/color_fx_dctl_tone_mapper/${encodeURIComponent(name)}`)).settings })));
+ } catch {
+ return [];
+ }
+}
+
export function getJob(id: string) {
return client().jobs.get(id);
}
diff --git a/test/color.test.ts b/test/color.test.ts
new file mode 100644
index 0000000..1baf7bf
--- /dev/null
+++ b/test/color.test.ts
@@ -0,0 +1,43 @@
+import assert from "node:assert/strict";
+import { readFileSync } from "node:fs";
+import test from "node:test";
+import { nodeByTitle } from "../lib/advanced.ts";
+import { COLOR_VIDEO, DEFAULT_TONE, pruneColor, toneFromPreset } from "../lib/color.ts";
+import { isLink, type ApiGraph } from "../lib/simple.ts";
+
+const graph: ApiGraph = JSON.parse(readFileSync(new URL("../workflows/color.api.json", import.meta.url), "utf8"));
+const dangling = (g: ApiGraph) => Object.entries(g).flatMap(([id, node]) =>
+ Object.entries(node.inputs).filter(([, v]) => isLink(v) && !g[v[0]]).map(([name]) => `${id}.${name}`));
+const node = (g: ApiGraph, title: string) => g[nodeByTitle(g, title)];
+const video = nodeByTitle(graph, COLOR_VIDEO);
+
+test("both steps set the amount and embed the tone settings under the node's preset name", () => {
+ const pruned = pruneColor(graph, { detail: -0.35, tone: { ...DEFAULT_TONE, strength: 0.5 } });
+ assert.equal(node(pruned, "Detail contrast").inputs.detail_contrast, -0.35);
+ const tone = node(pruned, "Tone mapper").inputs;
+ const raw = JSON.parse(String(tone.raw_settings));
+ assert.equal(raw.preset, tone.preset);
+ assert.equal(raw.node_type, "color_fx_dctl_tone_mapper");
+ assert.deepEqual(raw.settings, { ...DEFAULT_TONE, strength: 0.5, preserve_luminosity: false, clamp_output: true });
+ assert.equal(node(graph, "Tone mapper").inputs.raw_settings, "", "the source graph is not modified");
+});
+
+test("a step that is off is dropped and the video goes around it", () => {
+ const toneOnly = pruneColor(graph, { detail: null, tone: DEFAULT_TONE });
+ assert.deepEqual(dangling(toneOnly), []);
+ assert.deepEqual(node(toneOnly, "Tone mapper").inputs.image, [video, 0]);
+
+ const detailOnly = pruneColor(graph, { detail: -0.2, tone: null });
+ assert.deepEqual(dangling(detailOnly), []);
+ assert.deepEqual(node(detailOnly, "Fixed video").inputs.images, [nodeByTitle(detailOnly, "Detail contrast"), 0]);
+
+ const neither = pruneColor(graph, { detail: null, tone: null });
+ assert.deepEqual(node(neither, "Fixed video").inputs.images, [video, 0]);
+ assert.deepEqual(node(neither, "Fixed video").inputs.audio, [video, 2]);
+});
+
+test("an Orion preset keeps the values the page shows and ignores the rest", () => {
+ const tone = toneFromPreset({ mode: "Filmic Strong", contrast: 1.25, saturation: 1.1, mask_blur: 3, exposure: "x" });
+ assert.deepEqual(tone, { ...DEFAULT_TONE, mode: "Filmic Strong", contrast: 1.25, saturation: 1.1 });
+ assert.equal(toneFromPreset({ mode: "Sepia" }).mode, DEFAULT_TONE.mode);
+});
diff --git a/workflows/color.api.json b/workflows/color.api.json
new file mode 100644
index 0000000..6c41465
--- /dev/null
+++ b/workflows/color.api.json
@@ -0,0 +1,75 @@
+{
+ "1": {
+ "inputs": {
+ "video": "",
+ "force_rate": 24,
+ "custom_width": 0,
+ "custom_height": 0,
+ "frame_load_cap": 0,
+ "skip_first_frames": 0,
+ "select_every_nth": 1,
+ "format": "None"
+ },
+ "class_type": "VHS_LoadVideo",
+ "_meta": {
+ "title": "Video to fix"
+ }
+ },
+ "2": {
+ "inputs": {
+ "image": [
+ "1",
+ 0
+ ],
+ "detail_contrast": -0.2,
+ "blur_radius": 24,
+ "blur_sigma": 8.0,
+ "temporal_mode": "per_frame",
+ "chunk_size": 8
+ },
+ "class_type": "ImageDetailContrast",
+ "_meta": {
+ "title": "Detail contrast"
+ }
+ },
+ "3": {
+ "inputs": {
+ "image": [
+ "2",
+ 0
+ ],
+ "preset": "Openjutsu",
+ "raw_settings": ""
+ },
+ "class_type": "Orion4D_FXMax_DCTLToneMapper",
+ "_meta": {
+ "title": "Tone mapper"
+ }
+ },
+ "4": {
+ "inputs": {
+ "images": [
+ "3",
+ 0
+ ],
+ "audio": [
+ "1",
+ 2
+ ],
+ "frame_rate": 24,
+ "loop_count": 0,
+ "filename_prefix": "openjutsu_color/color",
+ "format": "video/h264-mp4",
+ "pix_fmt": "yuv420p",
+ "crf": 16,
+ "save_metadata": false,
+ "trim_to_audio": false,
+ "pingpong": false,
+ "save_output": true
+ },
+ "class_type": "VHS_VideoCombine",
+ "_meta": {
+ "title": "Fixed video"
+ }
+ }
+}