One-click character swap for videos: an open-source, local Higgsfield Genjutsu alternative built on ComfyUI and MiniMax H3, with FLUX.3 Klein relighting.
Add setup check, password access and Add object placement
D3voz229f4f7407db
diff --git a/.env.example b/.env.exampleindex 21fdc3f..8bceb8c 100644--- a/.env.example+++ b/.env.example@@ -4,3 +4,5 @@ COMFY_BASE_URL=http://127.0.0.1:8189COMFY_API_KEY=# ComfyUI itself (not the proxy), only to fill the base model and LoRA pickers. Leave empty for 127.0.0.1:8188.COMFYUI_URL=+# Required before hosting or tunneling the app. Browser login: username openjutsu. Use HTTPS remotely.+OPENJUTSU_PASSWORD=diff --git a/CUSTOM_NODES_REPORT.md b/CUSTOM_NODES_REPORT.mdnew file mode 100644index 0000000..5c9f0c2--- /dev/null+++ b/CUSTOM_NODES_REPORT.md@@ -0,0 +1,165 @@+# Openjutsu: ComfyUI custom-node installation report++Prepared 24 October 2026 for people installing Openjutsu on their own PCs.++## Result++Openjutsu needs **five custom-node packs for its video swapping modes**, plus **four additional packs for both post-processing features**: nine packs for the complete application. Seven have public repository links in the project. Two Openjutsu-specific packs still have no public installation source documented, so a new user cannot reproduce the complete setup from the README alone.++This report examines all nine shipped API workflows and the code that changes them before submission. They contain 464 node instances and 76 distinct node types; the app can add two more core types at runtime. Across that union, 44 types come from custom packs and 34 come from ComfyUI core. These counts describe the current exported graphs, rather than every internal helper a pack may execute.++## 1. Packs to install++Install packs under the user's own `ComfyUI/custom_nodes/` directory. All paths below are relative to that user's ComfyUI installation; no particular drive letter is required.++| Pack | Needed for | Source / release requirement |+| --- | --- | --- |+| ComfyUI-MiniMaxH3-Context-Loop | Simple, Advanced, one pass per person, Object, Montage; also the masked target in Fix faces | [Public repository](https://github.com/ethanfel/ComfyUI-MiniMaxH3-Context-Loop). Openjutsu declares **0.7.3+**. The Registry/package ID retains the spelling `comfyui-minimaxh3-contex-loop`. |+| ComfyUI-Fantastic-MiniMaxH3-PromptBuilder | Masked H3 edit latent in swapping and Fix faces | [Public repository](https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder). Must include `MiniMaxH3FantasticVideoEditLatent`; an older prompt-builder-only release is insufficient. |+| ComfyUI-KJNodes | Mask growth in swapping; mask preview | [Public repository](https://github.com/kijai/ComfyUI-KJNodes). Must provide both `GrowMaskWithBlur` and `ImageAndMaskPreview`. |+| ComfyUI-VideoHelperSuite | Video loading in every mode; MP4 export in mask preview, Fix faces and Fix colours | [Public repository](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite). Must provide the three `VHS_` node IDs listed below. |+| **comfyui-h3-oneclick-swap** | Swap plans, person/role masks, shot processing, Montage analysis and face chunking | **Distribution gap.** README contains a TODO instead of a public pack link. Supply the maintained pack as a repository or versioned archive. Installing unrelated H3 packs will not supply these IDs. |+| ComfyUI-H3-FaceRefine | Fix faces only | [Public repository](https://github.com/Carasibana/ComfyUI-H3-FaceRefine). Requires face detection and identity-matching dependencies and models. |+| ComfyUI-Impact-Pack | Fix faces only: SAM loader | [Public repository](https://github.com/ltdrdata/ComfyUI-Impact-Pack). The Openjutsu graph uses `SAMLoader`. |+| **comfyui-image-detail-contrast** | Fix colours, when Detail contrast is enabled | **Distribution gap.** README contains a TODO instead of a public pack link. Supply the pack that registers `ImageDetailContrast`. |+| Orion4D_FXMax | Fix colours, when Tone mapper is enabled | [Public repository](https://github.com/orion4d/Orion4D_FXMax). Must expose `Orion4D_FXMax_DCTLToneMapper`, embedded `raw_settings`, and the preset-list routes used by Openjutsu. |++The Context Loop upstream [installation instructions](https://github.com/ethanfel/ComfyUI-MiniMaxH3-Context-Loop#install) additionally recommend [ComfyUI-H3-Motion-Context-MultiRef](https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef). This is a provider recommendation from that pack, separate from Openjutsu's nine direct pack dependencies. The inspected Context Loop implementation has an internal continuation fallback when a compatible provider is absent.++## 2. Minimum installation by feature++| Feature / shipped workflow | Required custom packs |+| --- | --- |+| Simple: `simple.api.json` | Context Loop, Fantastic PromptBuilder, KJNodes, VideoHelperSuite, Oneclick Swap |+| Advanced: `advanced.api.json` | Same five |+| One pass per person: `advanced_maxq.api.json` | Same five |+| Object Change / Remove / Background: `object.api.json` | Same five |+| Montage generation: `montage.api.json` | Same five |+| Check mask: `mask.api.json` | KJNodes, VideoHelperSuite, Oneclick Swap |+| Find people: `people.api.json` | VideoHelperSuite, Oneclick Swap |+| Fix faces: `face.api.json` | Context Loop, Fantastic PromptBuilder, VideoHelperSuite, Oneclick Swap, H3-FaceRefine, Impact Pack |+| Fix colours: `color.api.json` | VideoHelperSuite, Image Detail Contrast, Orion4D_FXMax |++Fix colours removes the detail or tone node when its corresponding step is disabled. Both packs are needed for the default colour pipeline. Turning off hat fix or No lip-sync does not eliminate any of the five packs needed for full swapping.++## 3. Exact custom-node IDs++Use these IDs to diagnose a missing-node error or verify a new installation. Display titles can differ.++### Context Loop: 14 types++```text+MiniMaxH3GenerationProfile+MiniMaxH3SourceTimeline+MiniMaxH3ChainPlanModern+MiniMaxH3ChainLoopStart+MiniMaxH3ChainCurrent+MiniMaxH3ChainContext+MiniMaxH3ChainSegmentSave+MiniMaxH3ChainReview+MiniMaxH3ChainLoopEnd+MiniMaxH3ChainAssemble+MiniMaxH3ContexLoopSourceSlice+MiniMaxH3ContexMaskedTarget+MiniMaxH3ContexMaskGridPreview+MiniMaxH3LoopTrim+```++The `Contex` spelling in these IDs is intentional.++### Oneclick Swap: 18 types++```text+H3OneClickSwapPlan+H3AdvancedSwapPlan+H3ObjectSwapPlan+H3PeopleMask+H3GreyMannequins+H3ClipShots+H3FillShots+H3TrackCrops+H3TrackRoles+H3RoleMask+H3SubjectPrompt+H3MontagePeople+H3MontageTracks+H3MontagePicks+H3MontagePlan+H3MontagePass+H3FaceChunks+H3FaceJoin+```++A distributed version must include the Montage nodes and face chunk/join nodes, not just the original Simple swap. It also needs the current `short_edge` custom-resolution inputs, Object plan `keep_outline`, and the track/role input contracts expected by the app.++### Other packs: 12 types++| Pack | Exact IDs |+| --- | --- |+| Fantastic PromptBuilder | `MiniMaxH3FantasticVideoEditLatent` |+| KJNodes | `GrowMaskWithBlur`, `ImageAndMaskPreview` |+| VideoHelperSuite | `VHS_LoadVideo`, `VHS_SelectEveryNthImage`, `VHS_VideoCombine` |+| H3-FaceRefine | `H3FaceTrackCrop`, `H3FaceMaskSAM`, `H3FaceStitch` |+| Impact Pack | `SAMLoader` |+| Image Detail Contrast | `ImageDetailContrast` |+| Orion4D_FXMax | `Orion4D_FXMax_DCTLToneMapper` |++## 4. ComfyUI core and supporting software++Openjutsu's README declares **ComfyUI 0.41.2 or newer**. The inspected reference installation reports 0.42.0; this audit does not establish the earliest working version independently. A new installation must expose native H3, SAM 3.1 and text generation, including `MiniMaxH3ReferenceToVideo`, `MiniMaxH3SigmaShift`, `SAM3_VideoTrack`, `SAM3_TrackToMask` and `TextGenerate`.++The app also uses core sampling, model loaders, audio, image, math and logic nodes. `ComfyMathExpression`, `ComfySwitchNode`, `ModelAttentionBackend`, `AudioAdjustVolume` and `TrimAudioDuration` are core dependencies in these graphs. `ImageScale` and `StringConcatenate` can be inserted by the app for custom sizing and run names. They do not require separate custom-node packs.++Install each pack's requirements into **the Python environment that actually runs ComfyUI**. Openjutsu's `.venv-proxy` is a separate environment for `comfy-api-proxy`; installing node dependencies there will not make ComfyUI import them.++For manual installation, run the equivalent of the following with ComfyUI's Python, for each installed pack that supplies a requirements file:++```text+<ComfyUI-Python> -m pip install -r <ComfyUI>/custom_nodes/<pack>/requirements.txt+```++For Fix faces, the inspected pack declares `ultralytics`, `scipy` and `insightface`. Supply an appropriate ONNX Runtime variant for InsightFace; the [FaceRefine installation notes](https://github.com/Carasibana/ComfyUI-H3-FaceRefine#requirements) explain the CPU/GPU package choice. Node registration alone does not verify these dependencies, because some imports happen only when a job runs.++Video handling requires a working FFmpeg setup. VideoHelperSuite can discover the binary supplied by `imageio-ffmpeg`; Context Loop recommends FFmpeg on `PATH` for assembly and review. Restart ComfyUI after installing the packs and their dependencies.++Openjutsu itself additionally requires Node.js 22.6+ and `comfy-api-proxy`. These are application services, separate from ComfyUI custom nodes.++## 5. Models a fresh installation also needs++Installing custom nodes does not install the model weights referenced by the workflows.++| ComfyUI model folder | Default file / directory | Used for |+| --- | --- | --- |+| `models/diffusion_models/` | `minimax_h3_ref2va_pruned_int8_convrot.safetensors` | H3 generation and Fix faces |+| `models/loras/` | `minimax_h3_DMAD_4step_full_lora_avg_rank_39_bf16.safetensors` | Default speed LoRA |+| `models/text_encoders/` | `qwen3vl_32b_minimax_h3_int4_convrot.safetensors` | H3 text encoder |+| `models/text_encoders/` | `qwen3vl_8b_int8_convrot.safetensors` | Picture, face, shot and role descriptions |+| `models/vae/` | `minimax_h3_video_vae_int8_convrot.safetensors` | Video VAE |+| `models/vae/` | `minimax_h3_audio_vae_fp32.safetensors` | Audio VAE |+| `models/checkpoints/` | `sam3.1_multiplex_fp16.safetensors` | Swapping, tracking and mask preview |+| `models/ultralytics/bbox/` | `face_yolov8m.pt` | Fix faces |+| `models/ultralytics/segm/` | `person_yolov8m-seg.pt` | Fix faces: configured fallback detector |+| `models/sams/` | `sam_vit_b_01ec64.pth` | Fix faces: SAM face mask |+| `models/insightface/models/` | `buffalo_l/` | Fix faces: identity matching |++These are the current workflow/app defaults, not verified model download links. The base model and LoRAs can be changed in the app; the remaining filenames are embedded in the exports. InsightFace may fetch `buffalo_l` on first use if it is missing, so include it explicitly when preparing an offline installation.++The README's tested hardware is an NVIDIA RTX 4060 Ti with 16 GB VRAM and 64 GB system RAM. This is a tested configuration, not a measured minimum or a promise of compatibility with other GPU backends.++## 6. Items to resolve before publishing installation instructions++1. **Publish or bundle Oneclick Swap.** It is required for every swapping mode, Check mask, Find people and Fix faces. The inspected source is a standalone `__init__.py` registering all 18 IDs; no public URL or version metadata is supplied in the project.+2. **Publish or bundle Image Detail Contrast.** It is required by the default Fix colours graph. Its inspected source is also a standalone `__init__.py` without a documented public URL or version.+3. **Pin a reproducible set of pack revisions.** The reference metadata is Context Loop 0.7.3, Fantastic PromptBuilder 1.8.1 (`01b6d8d`), KJNodes 1.5.2, VideoHelperSuite 1.7.9 (`115de7a`), H3-FaceRefine 1.0.0 (`79a97ce`), Impact Pack 8.28.3 and Orion4D_FXMax 1.0.0. These identify the inspected setup; they are not independently proven minimum versions. Version untracked packs or record archive hashes as well.+4. **Provide exact model download sources.** In particular, publish links for the INT8/INT4 ConvRot artifacts and default DMAD LoRA rather than expecting a general H3 model page to contain the exact files.+5. **Validate on a clean PC.** Restart ComfyUI, confirm the required IDs in its `/object_info` registry, then run a short job through each supported feature. Include custom resolution, Montage Find people/review, Fix faces and default Fix colours. Confirm Python dependencies, models and FFmpeg as part of that run.++The colour workflow uses an `Openjutsu` preset label, while `lib/color.ts` supplies the actual settings through `raw_settings`. A separate `Openjutsu.json` preset is not required with the inspected Orion implementation, which accepts embedded settings without a matching preset file. `H3 soft.json` is optional for displaying that saved preset in the picker; the app has built-in defaults.++Openjutsu's exported workflows do not directly require ComfyUI-GGUF, NativeAudioLock, H3 upscalers or the other unrelated packs present in a larger ComfyUI installation. ComfyUI Manager can help install public packs, but it does not replace the two missing distribution sources.++## Evidence and limits++Primary evidence: Openjutsu `README.md`, `workflows/*.api.json`, and the graph preparation code in `lib/comfy.ts`, `lib/simple.ts`, `lib/advanced.ts`, `lib/montage.ts`, `lib/object.ts`, `lib/face.ts`, `lib/color.ts` and `lib/model.ts`; custom-pack registration tables and metadata; and the linked upstream repositories. All 78 directly referenced or app-inserted IDs were present in the reference ComfyUI registry during the audit. Pack ownership was checked against source registration tables because another installed extension mislabels some registry module names.++This was a dependency and distribution audit. No clean-PC installation or end-to-end generation was performed, and availability on the reference machine does not prove that another user's installation is ready.diff --git a/README.md b/README.mdindex 1b2c966..0a813a4 100644--- a/README.md+++ b/README.md@@ -27,14 +27,16 @@ Install these into `ComfyUI/custom_nodes` (ComfyUI Manager finds the first three| [ComfyUI-Fantastic-MiniMaxH3-PromptBuilder](https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder) | masked video-edit latent || [ComfyUI-KJNodes](https://github.com/kijai/ComfyUI-KJNodes) | Grow Mask With Blur || [ComfyUI-VideoHelperSuite](https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite) | video loading |-| `comfyui-h3-oneclick-swap` | swap and object plans, people and role masks, grey mannequins, track labelling, cut finder, shot list, face-fix chunks (TODO: link the published pack) |+| [comfyui-h3-oneclick-swap](https://github.com/D3voz/comfyui-h3-oneclick-swap) | swap and object plans, people and role masks, grey mannequins, track labelling, cut finder, shot list, face-fix chunks || [ComfyUI-H3-FaceRefine](https://github.com/Carasibana/ComfyUI-H3-FaceRefine) | Fix faces: face tracking and cropping, SAM face masks, pasting back || [ComfyUI-Impact-Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) | Fix faces: SAM loader |-| `comfyui-image-detail-contrast` | Fix colours: Image Detail Contrast (TODO: link the published pack) |+| [comfyui-image-detail-contrast](https://github.com/D3voz/comfyui-image-detail-contrast) | Fix colours: Image Detail Contrast || [Orion4D_FXMax](https://github.com/orion4d/Orion4D_FXMax) | Fix colours: DCTL Tone Mapper and its presets |Restart ComfyUI after installing them.+Before uploading a job, Openjutsu checks the selected workflow against ComfyUI's node and model lists. Missing packs appear with installation links; missing model files appear with their destination folders. Disabled steps are excluded from the check. This checks registered nodes and listed model files, not Python dependencies, model integrity or available VRAM. `COMFYUI_URL` must point at ComfyUI itself (normally `http://127.0.0.1:8188`).+### Models| Folder in `ComfyUI/models` | File |@@ -75,12 +77,16 @@ On Windows, double-click `start_openjutsu.bat` (ComfyUI must be running, or star`.env.local` sets `COMFY_BASE_URL=http://127.0.0.1:8189`. Leave `COMFY_API_KEY` empty for a local proxy. The base model and LoRA pickers read ComfyUI's model lists from `COMFYUI_URL` (default `http://127.0.0.1:8188`); without it they become text fields.+Uploads are limited to 512 MB total per request, including reference pictures. One upload/submission is accepted at a time per app process; other tabs can retry once it finishes. Generation continues in ComfyUI independently.+A 6 s clip at 480p takes about 6–13 minutes on a 4060 Ti, most of it in H3 sampling. 736p is the default; 1080p is about 2.5× slower than 736p and needs much more memory. Custom sets an exact width × height from 256 to 2048 px a side (rounded to 32); the video is scaled and centre-cropped to that shape first. It needs the `short_edge` input of the H3 Swap Plan nodes in comfyui-h3-oneclick-swap.Advanced motion transfer has a **Montage** tab for edited clips where different people take turns within one shot without a hard cut (whip pans, selfie handovers, travel montages). Write roles as "who - looks" ("the dancer - white polo, jeans"): the looks help match the people, and only the part before " - " reaches H3. **Generate** tracks everyone with SAM over the whole video, matches each appearance to the roles with Qwen (a person SAM loses and finds again is matched again, so they keep their photo; Qwen sees each appearance in its frame and close up at three moments, a few appearances at a time, and is asked again about any it skips or matches to one photo twice at the same moment) and makes the video as one chain like Standard, with overlapping clips: each clip has one H3 pass per photo on screen in it (the two with the most screen time) that masks only that photo's appearances, with a prompt about that person only. Appearances on screen for under 12 frames or at under 2% of the frame stay as they are, since H3 draws them as grey figures; the result page lists them. **Find people** (about 3 minutes for 18 s of video) is the optional review: it does the tracking and matching alone, lists the appearances with a photo picker each and saves the tracks, and Generate from that list masks with exactly those tracks. The saved tracks live in ComfyUI's temp folder and are checked against the video, part and size, so after a ComfyUI restart or a change, Find people again. It needs the pack's `H3TrackCrops` and `H3TrackRoles` (which now runs Qwen itself), `H3MontagePeople`, `H3MontageTracks`, `H3MontagePicks`, `H3MontagePlan` and `H3MontagePass` nodes.## Tips+- **Add an object:** under Object swap, choose **Add**. Describe the object (for example `a chocolate cake on a plate`) and where it goes (`on the table`). Draw a box covering the whole object and its shadow in the placement editor. An optional reference picture controls its appearance. One box stays fixed on screen; for camera movement, scrub forward and draw more boxes to animate the placement. This uses manual placement keyframes, not SAM tracking. Changing the source video, selected section or resolution clears the boxes. Update `comfyui-h3-oneclick-swap` and restart ComfyUI to load the new `H3ObjectPlacementMask` node.+- Only the main person is regenerated; anyone else in the video stays as they are.- **Part to use** (under the source video) swaps only part of the video: **Whole**, or a **Length** placed at the **Start**, **Middle** or **End**, fine-tuned with **Starts at**. The sound is cut to the same part, the preview loops it, and Check mask uses it too. A shorter part renders faster (fewer clips).- **Mirrors and reflections do not work yet**, in either mode. The reflection keeps the original person, and H3 copies their look from it: hair, outfit and build come back on your character. Use videos without a mirror, window or other reflection of the person.@@ -139,7 +145,8 @@ Not tested yet. The app code does not change, only `.env.local`:2. Create a deployment from the released Build on the Developer Platform. The workflow has only been run on a 16 GB card with 64 GB of system RAM, so choose at least that.3. Set `COMFY_BASE_URL` to the deployment URL and `COMFY_API_KEY` to a key from the workspace that owns it. The key stays on the server; never put it in a `NEXT_PUBLIC_` variable.-4. Host the Next.js app anywhere that runs Node (`npm run build && npm start`). Streaming progress needs a host that keeps long responses open.+4. Before hosting or tunneling the app, set a strong `OPENJUTSU_PASSWORD` and use HTTPS. Open the page and sign in as `openjutsu` with that password; the same browser login protects uploads, job controls, progress and downloads. This is a single-owner app: everyone given the password shares access to its jobs. Never expose a password-free server, including through a reverse proxy that forwards a localhost Host header.+5. Host the Next.js app anywhere that runs Node (`npm run build && npm start`). Streaming progress needs a host that keeps long responses open. Preserve the public Host and scheme when reverse proxying so same-origin API checks work. Local password-free use remains bound to `127.0.0.1` by the supplied launch commands.Notes:diff --git a/app/api/jobs/[id]/events/route.ts b/app/api/jobs/[id]/events/route.tsindex 063cace..b11c023 100644--- a/app/api/jobs/[id]/events/route.ts+++ b/app/api/jobs/[id]/events/route.ts@@ -1,3 +1,4 @@+import { apiAccess, validateJobId } from "../../../../../lib/http.ts";import { NotFound } from "@comfyorg/sdk";import { getJob, JOB_KINDS, loadGraph, publicJob, rawJobEvents, TERMINAL, type JobKind } from "../../../../../lib/comfy.ts";import { nodeDescriber } from "../../../../../lib/progress.ts";@@ -8,7 +9,14 @@ export const dynamic = "force-dynamic";// Server-sent events for the progress view, with node ids turned into node titles and clip// numbers. One upstream connection per request; the page's EventSource reconnects.export async function GET(request: Request, { params }: { params: Promise<{ id: string }> }) {+ const denied = apiAccess(request);+ if (denied) return denied;const { id } = await params;+ try {+ validateJobId(id);+ } catch {+ return Response.json({ error: "Invalid job ID." }, { status: 400 });+ }const kind = new URL(request.url).searchParams.get("kind") ?? "simple";const describe = nodeDescriber(await loadGraph(JOB_KINDS.includes(kind as JobKind) ? (kind as JobKind) : "simple"));const encoder = new TextEncoder();diff --git a/app/api/jobs/[id]/outputs/[outputId]/route.ts b/app/api/jobs/[id]/outputs/[outputId]/route.tsindex 40ae504..99e5860 100644--- a/app/api/jobs/[id]/outputs/[outputId]/route.ts+++ b/app/api/jobs/[id]/outputs/[outputId]/route.ts@@ -1,8 +1,11 @@+import { apiAccess, HttpError } from "../../../../../../lib/http.ts";import { getOutput } from "../../../../../../lib/comfy.ts";export const runtime = "nodejs";export async function GET(request: Request, { params }: { params: Promise<{ id: string; outputId: string }> }) {+ const denied = apiAccess(request);+ if (denied) return denied;try {const { id, outputId } = await params;const { name, response } = await getOutput(id, outputId, request.headers.get("Range"));@@ -16,6 +19,6 @@ export async function GET(request: Request, { params }: { params: Promise<{ id:if (new URL(request.url).searchParams.has("download")) headers.set("Content-Disposition", `attachment; filename="${name.replace(/"/g, "")}"`);return new Response(response.body, { status: response.status, headers });} catch (error) {- return Response.json({ error: error instanceof Error ? error.message : "Unable to download the output." }, { status: 502 });+ return Response.json({ error: error instanceof Error ? error.message : "Unable to download the output." }, { status: error instanceof HttpError ? error.status : 502 });}}diff --git a/app/api/jobs/[id]/route.ts b/app/api/jobs/[id]/route.tsindex 64959f4..d8645cb 100644--- a/app/api/jobs/[id]/route.ts+++ b/app/api/jobs/[id]/route.ts@@ -1,23 +1,28 @@+import { apiAccess, HttpError } from "../../../../lib/http.ts";import { NextResponse } from "next/server";import { getJob, publicJob } from "../../../../lib/comfy.ts";export const runtime = "nodejs";-export async function GET(_: Request, { params }: { params: Promise<{ id: string }> }) {+export async function GET(request: Request, { params }: { params: Promise<{ id: string }> }) {+ const denied = apiAccess(request);+ if (denied) return denied;try {const { id } = await params;return NextResponse.json(publicJob(await getJob(id)));} catch (error) {- return NextResponse.json({ error: error instanceof Error ? error.message : "Unable to check the job." }, { status: 502 });+ return NextResponse.json({ error: error instanceof Error ? error.message : "Unable to check the job." }, { status: error instanceof HttpError ? error.status : 502 });}}-export async function DELETE(_: Request, { params }: { params: Promise<{ id: string }> }) {+export async function DELETE(request: Request, { params }: { params: Promise<{ id: string }> }) {+ const denied = apiAccess(request);+ if (denied) return denied;try {const { id } = await params;const job = await getJob(id);return NextResponse.json(publicJob(await job.cancel()));} catch (error) {- return NextResponse.json({ error: error instanceof Error ? error.message : "Unable to cancel the job." }, { status: 502 });+ return NextResponse.json({ error: error instanceof Error ? error.message : "Unable to cancel the job." }, { status: error instanceof HttpError ? error.status : 502 });}}diff --git a/app/api/jobs/route.ts b/app/api/jobs/route.tsindex 785f8fb..c5ae61a 100644--- a/app/api/jobs/route.ts+++ b/app/api/jobs/route.ts@@ -1,3 +1,4 @@+import { apiAccess, HttpError, readFormData } from "../../../lib/http.ts";import { NextResponse } from "next/server";import { listModels, submitAdvanced, submitColor, submitFace, submitMask, submitMontage, submitObject, submitPeople, submitSimple } from "../../../lib/comfy.ts";import { TRACKS_FILE, type Label } from "../../../lib/montage.ts";@@ -5,7 +6,7 @@ import { DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, type ColorSettings, typeimport { CHUNK_SECONDS, FACE_STYLES, type FaceSettings, type FaceStyle } from "../../../lib/face.ts";import { MAX_CHARACTERS } from "../../../lib/advanced.ts";import { MAX_LORAS, STEPS, STRENGTH, type ModelSettings } from "../../../lib/model.ts";-import { EDIT_MODES, OBJECT_MARGINS, type EditMode, type ObjectMargin, type ObjectSettings } from "../../../lib/object.ts";+import { EDIT_MODES, OBJECT_MARGINS, parsePlacement, type EditMode, type ObjectMargin, type ObjectSettings } from "../../../lib/object.ts";import { CLIP_SECONDS, DEFAULT_TARGET, MARGINS, MAX_SEED, MIN_SECTION_SECONDS, MOTION_SIZES, RESOLUTIONS, CUSTOM_SIZE, type Margin, type MotionSize, type Preset, type Resolution, type Section, type SimpleSettings } from "../../../lib/simple.ts";export const runtime = "nodejs";@@ -21,8 +22,8 @@ function parseVideo(form: FormData) {function parseSection(form: FormData): Section {const start = Number(form.get("start") ?? 0);const length = Number(form.get("length") ?? 0);- if (!(start >= 0)) throw new Error("The start of the part to use must be 0 s or later.");- if (!(length === 0 || length >= MIN_SECTION_SECONDS)) throw new Error(`The part to use must be at least ${MIN_SECTION_SECONDS} s long.`);+ if (!Number.isFinite(start) || !(start >= 0)) throw new Error("The start of the part to use must be 0 s or later.");+ if (!Number.isFinite(length) || !(length === 0 || length >= MIN_SECTION_SECONDS)) throw new Error(`The part to use must be at least ${MIN_SECTION_SECONDS} s long.`);return { start, length };}@@ -65,18 +66,19 @@ function parseRun(form: FormData) {function parseObject(form: FormData) {const mode = String(form.get("edit_mode"));- if (!EDIT_MODES.includes(mode as EditMode)) throw new Error("Choose change, remove or background.");+ if (!EDIT_MODES.includes(mode as EditMode)) throw new Error("Choose change, remove, background or add.");const margin = String(form.get("margin") ?? "normal");if (!Object.hasOwn(OBJECT_MARGINS, margin)) throw new Error("Choose a supported mask margin.");const target = String(form.get("target") ?? "").replace(/\s+/g, " ").trim().slice(0, 100);- if (!target) throw new Error(mode === "background" ? "Say what stays, for example person." : "Say what to change, for example hair.");+ if (!target) throw new Error(mode === "add" ? "Say where the object goes, for example on the table." : mode === "background" ? "Say what stays, for example person." : "Say what to change, for example hair.");const changeTo = String(form.get("change_to") ?? "").replace(/\s+/g, " ").trim().slice(0, 300);const entry = form.get("picture");const picture = entry instanceof File && entry.size > 0 && mode !== "remove" ? parsePicture(entry) : null;- if (mode !== "remove" && !changeTo && !picture) throw new Error("Say what it becomes, or add a picture.");+ if (mode !== "remove" && !changeTo && !picture) throw new Error(mode === "add" ? "Describe the object to add, or add a picture." : "Say what it becomes, or add a picture.");const settings: ObjectSettings = {mode: mode as EditMode, target, changeTo, biggestOnly: form.get("biggest_only") === "true", keepOutline: form.get("keep_outline") === "true",margin: margin as ObjectMargin, ...parseRun(form),+ ...(mode === "add" ? { placement: parsePlacement(form.get("placement")) } : {}),};return { picture, settings };}@@ -176,10 +178,24 @@ function parseMontage(form: FormData) {return { pictures, settings, review: { tracks: String(tracks), labels: parseLabels(form.get("labels"), pictures.length) } };}+let submitting = false;+export async function POST(request: Request) {+ const denied = apiAccess(request);+ if (denied) return denied;+ if (submitting) return NextResponse.json({ error: "Another upload is in progress. Try again shortly." }, { status: 429, headers: { "Retry-After": "5" } });+ submitting = true;+ try {+ return await submitJob(request);+ } finally {+ submitting = false;+ }+}++async function submitJob(request: Request) {let submit: () => ReturnType<typeof submitSimple>;try {- const form = await request.formData();+ const form = await readFormData(request);const mode = form.get("mode");const video = parseVideo(form);const section = parseSection(form);@@ -216,7 +232,7 @@ export async function POST(request: Request) {throw new Error("Unknown mode.");}} catch (error) {- return NextResponse.json({ error: error instanceof Error ? error.message : "Invalid request." }, { status: 400 });+ return NextResponse.json({ error: error instanceof Error ? error.message : "Invalid request." }, { status: error instanceof HttpError ? error.status : 400 });}try {diff --git a/app/api/models/route.ts b/app/api/models/route.tsindex e9e6931..6a315c1 100644--- a/app/api/models/route.ts+++ b/app/api/models/route.ts@@ -1,9 +1,12 @@+import { apiAccess } from "../../../lib/http.ts";import { NextResponse } from "next/server";import { listModels } from "../../../lib/comfy.ts";export const runtime = "nodejs";export const dynamic = "force-dynamic";-export async function GET() {+export async function GET(request: Request) {+ const denied = apiAccess(request);+ if (denied) return denied;return NextResponse.json((await listModels()) ?? { models: [], loras: [] });}diff --git a/app/api/presets/route.ts b/app/api/presets/route.tsindex e1dfbbc..4d0a5fe 100644--- a/app/api/presets/route.ts+++ b/app/api/presets/route.ts@@ -1,9 +1,12 @@+import { apiAccess } from "../../../lib/http.ts";import { NextResponse } from "next/server";import { listTonePresets } from "../../../lib/comfy.ts";export const runtime = "nodejs";export const dynamic = "force-dynamic";-export async function GET() {+export async function GET(request: Request) {+ const denied = apiAccess(request);+ if (denied) return denied;return NextResponse.json({ tone: await listTonePresets() });}diff --git a/app/api/setup/route.ts b/app/api/setup/route.tsnew file mode 100644index 0000000..3294de2--- /dev/null+++ b/app/api/setup/route.ts@@ -0,0 +1,22 @@+import { apiAccess } from "../../../lib/http.ts";+import { MAX_CHARACTERS } from "../../../lib/advanced.ts";+import { JOB_KINDS, loadGraph } from "../../../lib/comfy.ts";+import { checkSetup, setupGraph, type SetupRequest } from "../../../lib/setup.ts";++export const runtime = "nodejs";++export async function POST(request: Request) {+ const denied = apiAccess(request);+ if (denied) return denied;+ try {+ const selection: SetupRequest = await request.json();+ if (!selection || !JOB_KINDS.includes(selection.kind)) return Response.json({ error: "Unknown mode." }, { status: 400 });+ if ("pictures" in selection && (!Number.isInteger(selection.pictures) || selection.pictures < 1 || selection.pictures > MAX_CHARACTERS)) {+ return Response.json({ error: `Add 1-${MAX_CHARACTERS} character photos.` }, { status: 400 });+ }+ const graph = setupGraph(await loadGraph(selection.kind), selection);+ return Response.json(await checkSetup(graph));+ } catch {+ return Response.json({ error: "Could not check these settings. Review them and try again." }, { status: 400 });+ }+}diff --git a/app/globals.css b/app/globals.cssindex af595ed..7b31627 100644--- a/app/globals.css+++ b/app/globals.css@@ -60,6 +60,11 @@ h1 { margin: 0; font-size: 28px; letter-spacing: -0.02em; }.row { display: flex; align-items: center; justify-content: space-between; gap: 12px; flex-wrap: wrap; }.hint { margin: 0; color: var(--muted); font-size: 0.86em; }.error { margin: 0; color: var(--danger); }+.setup { border-color: var(--danger); }+.setup ul { margin: 0; padding-left: 20px; display: grid; gap: 10px; }+.setup a { color: var(--accent); }+.setup small { display: block; overflow-wrap: anywhere; }+.setup code { overflow-wrap: anywhere; }.drop {border: 1.5px dashed var(--line); border-radius: 14px; display: grid; place-items: center; text-align: center;@@ -95,6 +100,12 @@ video { width: 100%; border-radius: 12px; background: #000; max-height: 420px; }.edit input { background: var(--bg); border: 1px solid var(--line); border-radius: 8px; padding: 7px 9px; width: 100%; font-size: 1rem; }.edit input:focus { outline: none; border-color: var(--accent); }.edit .look, .edit .drop.add { width: 50%; }+.placement { display: grid; gap: 10px; min-width: 0; }+.placement canvas { width: 100%; height: auto; display: block; background: #000; border-radius: 10px; cursor: crosshair; touch-action: none; }+.placement input[type="range"] { width: 100%; }+.placement-keys { display: flex; gap: 6px; flex-wrap: wrap; }+.placement-keys button { border: 1px solid var(--line); border-radius: 8px; padding: 4px 8px; background: var(--panel-2); }+.placement-keys button.on { border-color: var(--accent); color: var(--accent); }.settings { gap: 12px; }.setting { display: flex; align-items: center; justify-content: space-between; gap: 16px; flex-wrap: wrap; }diff --git a/components/object-placement.tsx b/components/object-placement.tsxnew file mode 100644index 0000000..2feb29d--- /dev/null+++ b/components/object-placement.tsx@@ -0,0 +1,95 @@+"use client";++import { useCallback, useEffect, useRef, useState, type PointerEvent } from "react";+import { placementAt, type PlacementKeyframe } from "../lib/object.ts";++type Box = Omit<PlacementKeyframe, "frame">;++export function ObjectPlacement({ src, start, end, width, height, points, onChange }: {+ src: string; start: number; end: number; width: number; height: number;+ points: PlacementKeyframe[]; onChange: (points: PlacementKeyframe[]) => void;+}) {+ const video = useRef<HTMLVideoElement>(null);+ const canvas = useRef<HTMLCanvasElement>(null);+ const origin = useRef<{ x: number; y: number } | null>(null);+ const [time, setTime] = useState(start);+ const [draft, setDraft] = useState<Box | null>(null);+ const frame = Math.round(time * 24);+ const scale = Math.min(1, 1024 / Math.max(width, height));+ const cw = Math.max(1, Math.round(width * scale));+ const ch = Math.max(1, Math.round(height * scale));++ const draw = useCallback(() => {+ const v = video.current;+ const c = canvas.current;+ if (!v || !c || v.readyState < 2) return;+ const ctx = c.getContext("2d");+ if (!ctx) return;+ const cover = Math.max(c.width / v.videoWidth, c.height / v.videoHeight);+ const sw = c.width / cover, sh = c.height / cover;+ ctx.drawImage(v, (v.videoWidth - sw) / 2, (v.videoHeight - sh) / 2, sw, sh, 0, 0, c.width, c.height);+ const box = draft ?? placementAt(points, frame);+ if (box) {+ ctx.fillStyle = "rgba(214, 255, 63, 0.25)";+ ctx.strokeStyle = "#d6ff3f";+ ctx.lineWidth = Math.max(2, c.width / 300);+ ctx.fillRect(box.x * c.width, box.y * c.height, box.width * c.width, box.height * c.height);+ ctx.strokeRect(box.x * c.width, box.y * c.height, box.width * c.width, box.height * c.height);+ }+ }, [draft, points, frame]);++ useEffect(() => { setTime(start); }, [start, src]);+ useEffect(() => {+ if (video.current) video.current.currentTime = time;+ }, [time, src]);+ useEffect(draw, [draw, cw, ch]);++ function position(event: PointerEvent<HTMLCanvasElement>) {+ const rect = event.currentTarget.getBoundingClientRect();+ return { x: Math.max(0, Math.min(1, (event.clientX - rect.left) / rect.width)),+ y: Math.max(0, Math.min(1, (event.clientY - rect.top) / rect.height)) };+ }++ function boxAt(event: PointerEvent<HTMLCanvasElement>): Box | null {+ if (!origin.current) return null;+ const point = position(event);+ return { x: Math.min(point.x, origin.current.x), y: Math.min(point.y, origin.current.y),+ width: Math.abs(point.x - origin.current.x), height: Math.abs(point.y - origin.current.y) };+ }++ return (+ <div className="placement">+ <strong>Place the new object</strong>+ <p className="hint">Drag a box around the whole object's space, including its height above the surface and room for its shadow.</p>+ <video ref={video} src={src} preload="auto" muted playsInline hidden+ onLoadedMetadata={() => { if (video.current) video.current.currentTime = time; }} onLoadedData={draw} onSeeked={draw} />+ <canvas ref={canvas} width={cw} height={ch} aria-label="Draw the new object's placement box" role="img"+ onPointerDown={(event) => {+ event.preventDefault();+ event.currentTarget.setPointerCapture(event.pointerId);+ origin.current = position(event);+ setDraft({ ...origin.current, width: 0, height: 0 });+ }}+ onPointerMove={(event) => { const box = boxAt(event); if (box) setDraft(box); }}+ onPointerUp={(event) => {+ const box = boxAt(event);+ origin.current = null;+ setDraft(null);+ if (box && box.width > 0 && box.height > 0) onChange([...points.filter((p) => p.frame !== frame), { frame, ...box }].sort((a, b) => a.frame - b.frame));+ }}+ onPointerCancel={() => { origin.current = null; setDraft(null); }} />+ <label>Placement frame: {time.toFixed(2)} s+ <input type="range" min={start} max={Math.max(start, end - 1 / 24)} step={1 / 24} value={time}+ onChange={(event) => setTime(Number(event.target.value))} />+ </label>+ <div className="row">+ <button className="secondary" disabled={!points.some((p) => p.frame === frame)} onClick={() => onChange(points.filter((p) => p.frame !== frame))}>Remove this keyframe</button>+ <button className="secondary" disabled={!points.length} onClick={() => onChange([])}>Clear placement</button>+ </div>+ {points.length > 0 && <div className="placement-keys">{points.map((p) => (+ <button key={p.frame} className={p.frame === frame ? "on" : ""} onClick={() => setTime(Math.max(start, Math.min(end - 1 / 24, p.frame / 24)))}>{(p.frame / 24).toFixed(2)} s</button>+ ))}</div>}+ <p className="hint">One box stays fixed on screen. If the camera moves, scrub forward and draw more boxes; placement moves between them. This is manual placement, not automatic tracking. Changing the video, section or resolution clears the boxes.</p>+ </div>+ );+}diff --git a/components/openjutsu.tsx b/components/openjutsu.tsxindex e1951d1..9404f15 100644--- a/components/openjutsu.tsx+++ b/components/openjutsu.tsx@@ -2,12 +2,14 @@import { useEffect, useRef, useState, type DragEvent, type ReactNode } from "react";import type { PublicJob } from "../lib/comfy.ts";+import type { SetupRequest, SetupResult } from "../lib/setup.ts";import type { NodeInfo } from "../lib/progress.ts";import { clipCount, clipStarts } from "../lib/clips.ts";import { CLIP_SECONDS, DEFAULT_TARGET, MARGINS, MAX_SEED, MIN_SECTION_SECONDS, MOTION_SIZES, RESOLUTIONS, CUSTOM_SIZE, canvasSize, type Margin, type MotionSize, type Resolution, type Section, type Size } from "../lib/simple.ts";import { MAX_CHARACTERS } from "../lib/advanced.ts";import { DEFAULT_MODEL, MAX_LORAS, STEPS, STRENGTH, type Lora } from "../lib/model.ts";-import type { EditMode } from "../lib/object.ts";+import type { EditMode, PlacementKeyframe } from "../lib/object.ts";+import { ObjectPlacement } from "./object-placement.tsx";import { CHUNK_SECONDS, faceChunkCount, type FaceStyle } from "../lib/face.ts";import { DEFAULT_PRESET, DEFAULT_TONE, DETAIL, TONE_MODES, TONE_RANGES, toneFromPreset, type ToneSettings, type ToneValue } from "../lib/color.ts";import { appearances, renderLabels, type Appearance, type Kept, type Label, type People, type Review } from "../lib/montage.ts";@@ -43,6 +45,7 @@ const FACE_STYLE_TEXT: Record<FaceStyle, { label: string; hint: string }> = {type EditTab = EditMode | "picture";const EDITS: { id: EditTab; label: string; target: string; what: string; to: string; toHint: string }[] = [{ id: "change", label: "Change", target: "hair, top", what: "What to change", to: "short bright pink hair and a black leather jacket", toHint: "What it becomes" },+ { id: "add", label: "Add", target: "on the table", what: "Where it goes", to: "a chocolate cake on a plate", toHint: "Object to add" },{ id: "remove", label: "Remove", target: "poster", what: "What to remove", to: "a plain wall", toHint: "What shows instead (optional)" },{ id: "background", label: "Background", target: "person", what: "What stays", to: "a snowy mountain pass at dusk", toHint: "The new place" },{ id: "picture", label: "From picture", target: "person", what: "Who is replaced", to: "", toHint: "" },@@ -156,6 +159,7 @@ export function Openjutsu() {const [look, setLook] = useState<{ file: File; url: string } | null>(null);const [biggestOnly, setBiggestOnly] = useState(false);const [keepOutline, setKeepOutline] = useState(true);+ const [placement, setPlacement] = useState<PlacementKeyframe[]>([]);const [post, setPost] = useState<Post>("face");const [detailOn, setDetailOn] = useState(true);const [detail, setDetail] = useState(DETAIL.default);@@ -174,6 +178,8 @@ export function Openjutsu() {const [loras, setLoras] = useState<Lora[]>(DEFAULT_MODEL.loras);const [files, setFiles] = useState<{ models: string[]; loras: string[] }>({ models: [], loras: [] });const [submitting, setSubmitting] = useState(false);+ const [checkingSetup, setCheckingSetup] = useState(false);+ const [setup, setSetup] = useState<SetupResult | null>(null);const [formError, setFormError] = useState("");const [run, setRun] = useState<Run | null>(null);@@ -185,6 +191,8 @@ export function Openjutsu() {useEffect(() => setRun(readRun()), []);+ useEffect(() => setSetup(null), [mode, post, montage, maxQuality, resolution, motion, hatFix, noLipSync, model, loras, detailOn, toneOn, faceStyle, editMode, characters.length, !!look]);+useEffect(() => {fetch("/api/models").then((response) => response.json()).then(setFiles, () => {});// Orion's own copy of the default preset wins over the built-in values once it has loaded.@@ -237,8 +245,9 @@ export function Openjutsu() {const probe = document.createElement("video");probe.preload = "metadata";probe.onloadedmetadata = () => {- setVideo((current) => (current?.url === url ? { ...current, duration: probe.duration, width: probe.videoWidth, height: probe.videoHeight } : current));- setPart({ start: 0, length: probe.duration });+ const duration = Number.isFinite(probe.duration) ? probe.duration : 0;+ setVideo((current) => (current?.url === url ? { ...current, duration, width: probe.videoWidth, height: probe.videoHeight } : current));+ setPart({ start: 0, length: duration });};probe.src = url;}@@ -246,6 +255,7 @@ export function Openjutsu() {function choosePart(start: number, length: number) {const duration = video?.duration ?? 0;const next = { start: Math.min(Math.max(0, start), duration - length), length };+ if (next.start !== part.start || next.length !== part.length) setPlacement([]);setPart(next);if (player.current) player.current.currentTime = next.start;}@@ -294,6 +304,7 @@ export function Openjutsu() {const canvas = typeof resolution === "object" ? [resolution.width, resolution.height] : video?.width ? canvasSize(video.width, video.height, resolution) : null;const resolutionField = typeof resolution === "object" ? `${resolution.width}x${resolution.height}` : resolution;+ useEffect(() => setPlacement([]), [video?.url, resolutionField]);// A montage render masks with the tracks Find people saved, so it must use the frames Find people saw.const frames = video ? `${video.file.name}:${video.file.size} ${whole ? "whole" : `${part.start.toFixed(2)}+${part.length.toFixed(2)}`} ${typeof resolution === "object" ? resolutionField : ""}` : "";@@ -309,7 +320,7 @@ export function Openjutsu() {const edit = EDITS.find((e) => e.id === editMode)!;const fromPicture = mode === "object" && editMode === "picture";const ready = colorFix ? detailOn || toneOn : mode === "object"- ? editMode === "remove" || (fromPicture ? !!look : !!changeTo.trim() || !!look)+ ? editMode === "remove" || (fromPicture ? !!look : (editMode !== "add" || placement.length > 0) && (!!changeTo.trim() || !!look)): !!character;function chooseLook(files: File[]) {@@ -321,6 +332,8 @@ export function Openjutsu() {async function submit(kind: Kind, review?: Review) {if (!video || (kind !== "mask" && kind !== "people" && !ready)) return;setSubmitting(true);+ setCheckingSetup(true);+ setSetup(null);setFormError("");const form = new FormData();form.set("mode", kind);@@ -393,6 +406,7 @@ export function Openjutsu() {if (look && editMode !== "remove") form.set("picture", look.file);form.set("biggest_only", String(biggestOnly));form.set("keep_outline", String(keepOutline));+ if (editMode === "add") form.set("placement", JSON.stringify(placement.map((p) => ({ ...p, frame: p.frame - Math.round((whole ? 0 : part.start) * 24) }))));form.set("resolution", resolutionField);form.set("seed", String(seed));form.set("clip_seconds", String(clipSeconds));@@ -406,7 +420,26 @@ export function Openjutsu() {form.append("lora_strength", String(l.strength));}}+ const shared = { resolution, motion, hatFix, noLipSync, margin, seed, clipSeconds };+ const selectedModel = { model, loras, steps };+ let selection: SetupRequest;+ if (kind === "simple") selection = { kind, settings: { ...shared, role: fromPicture ? "" : character?.role ?? "", target: String(form.get("target")) }, model: selectedModel };+ else if (kind === "advanced" || kind === "advanced_maxq") selection = { kind, settings: { ...shared, roles: shown.map((c) => c.role) }, pictures: shown.length, model: selectedModel };+ else if (kind === "montage") selection = { kind, settings: { ...shared, roles: shown.map((c) => c.role) }, pictures: shown.length, review: review ?? null, model: selectedModel };+ else if (kind === "object") selection = { kind, settings: { ...shared, mode: editMode as EditMode, target: String(form.get("target")), changeTo, biggestOnly, keepOutline,+ ...(editMode === "add" ? { placement: placement.map((p) => ({ ...p, frame: p.frame - Math.round((whole ? 0 : part.start) * 24) })) } : {}) }, picture: !!look && editMode !== "remove", model: selectedModel };+ else if (kind === "face") selection = { kind, settings: { style: faceStyle, seed, chunkSeconds }, model: selectedModel };+ else if (kind === "color") selection = { kind, settings: { detail: detailOn ? detail : null, tone: toneOn ? tone : null } };+ else if (kind === "mask") selection = { kind, hatFix, margin, target: String(form.get("target")) };+ else selection = { kind, resolution };+try {+ const checked = await fetch("/api/setup", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(selection) });+ const result = await checked.json();+ if (!checked.ok) throw new Error(result.error ?? "Could not check setup.");+ setSetup(result);+ if (!result.ok) return;+ setCheckingSetup(false);const response = await fetch("/api/jobs", { method: "POST", body: form });const body = await response.json();if (!response.ok) throw new Error(body.error ?? "Could not start the job.");@@ -420,18 +453,27 @@ export function Openjutsu() {} catch (error) {setFormError(error instanceof Error ? error.message : "Could not start the job.");} finally {+ setCheckingSetup(false);setSubmitting(false);}}async function cancel() {if (!run) return;- const response = await fetch(`/api/jobs/${run.id}`, { method: "DELETE" });- if (response.ok) setJob(await response.json());+ setFormError("");+ try {+ const response = await fetch(`/api/jobs/${run.id}`, { method: "DELETE" });+ const body = await response.json();+ if (!response.ok) throw new Error(body.error ?? "Could not cancel the job.");+ setJob(body);+ } catch (error) {+ setFormError(error instanceof Error ? error.message : "Could not cancel the job.");+ }}function reset() {setFormError("");+ setSetup(null);writeRun(null);setRun(null);setJob(null);@@ -466,6 +508,26 @@ export function Openjutsu() {</nav>{formError && <p className="error" role="alert">{formError}</p>}+ {checkingSetup && <p className="hint" role="status">Checking required nodes and models…</p>}+ {setup && !setup.ok && (+ <section className="panel setup" role="alert">+ <h2>Setup needs attention</h2>+ {setup.error && <p className="error">{setup.error}</p>}+ {setup.packs.length > 0 && <>+ <p className="hint">Install or update the items below, then restart ComfyUI and try again.</p>+ <ul>{setup.packs.map((pack) => <li key={pack.url}>+ <a href={pack.url} target="_blank" rel="noopener noreferrer">{pack.name} ↗</a>+ <small>Missing nodes: {pack.nodes.join(", ")}</small>+ </li>)}</ul>+ </>}+ {setup.models.length > 0 && <>+ <p className="hint">Add these models to ComfyUI, or choose an installed base model or LoRA, then try again.</p>+ <ul>{setup.models.map((item) => <li key={`${item.folder}/${item.name}`}>+ <code>{item.name}</code><small>ComfyUI/models/{item.folder}</small>+ </li>)}</ul>+ </>}+ </section>+ )}{run ? (<RunViewrun={run} job={job} progress={progress} preview={preview} streamError={streamError} now={now}@@ -577,7 +639,7 @@ export function Openjutsu() {))}</div><label>- <span>{edit.what} <small>What SAM tracks, in English.{(editMode === "change" || editMode === "remove") && " Separate several with commas."}</small></span>+ <span>{edit.what} <small>{editMode === "add" ? "Describe the placement, for example on the table in front of the person." : "What SAM tracks, in English."}{(editMode === "change" || editMode === "remove") && " Separate several with commas."}</small></span><input value={editTarget} maxLength={100} placeholder={edit.target} onChange={(event) => setEditTarget(event.target.value)} /></label>{!fromPicture && <label>@@ -592,15 +654,18 @@ export function Openjutsu() {) : (<DropZone accept="image/*" onFiles={chooseLook} className="add"><strong>+</strong>- <span>{fromPicture ? "The picture to copy" : editMode === "background" ? "Picture of the place (optional)" : "Picture of the new look (optional)"}</span>+ <span>{fromPicture ? "The picture to copy" : editMode === "add" ? "Picture of the object (optional)" : editMode === "background" ? "Picture of the place (optional)" : "Picture of the new look (optional)"}</span></DropZone>))}<p className="hint">{editMode === "change" && "Only what SAM tracks is regenerated: hair, a top, a car. A picture of an outfit, hairstyle or product is copied onto it."}+ {editMode === "add" && "A new object is generated inside your placement area while the scene continues. Keep the supporting surface visible and describe where the object rests."}{editMode === "remove" && "What SAM tracks is erased and filled with what is around it. The camera, people and sound stay."}{editMode === "background" && "Everything except what SAM tracks becomes the new place; the camera movement stays."}{fromPicture && "The person in the video becomes the person in the picture: face, hair and outfit. The background, motion, camera and sound stay."}</p>+ {editMode === "add" && video && canvas && <ObjectPlacement key={`${video.url}:${resolutionField}`} src={video.url}+ start={whole ? 0 : part.start} end={whole ? video.duration : part.start + part.length} width={canvas[0]} height={canvas[1]} points={placement} onChange={setPlacement} />}</div>) : (<><div className="characters">@@ -702,7 +767,7 @@ export function Openjutsu() {</div>)}</div>}- {mode === "object" && editMode !== "background" && !fromPicture && (+ {mode === "object" && (editMode === "change" || editMode === "remove") && (<label className="setting toggle"><span>Only the biggest one <small>Off: every match SAM finds changes (everyone's hair). On: only the one that fills the most of the frame.</small></span><input type="checkbox" checked={biggestOnly} onChange={(event) => setBiggestOnly(event.target.checked)} />@@ -811,10 +876,10 @@ export function Openjutsu() {{mode === "simple" && <button className="secondary" disabled={!video || submitting} onClick={() => submit("mask")}>Check mask</button>}{mode === "advanced" && montage ? (<><button className="secondary" disabled={!video || !character || submitting} onClick={() => submit("people")}>Find people</button>- <button className="primary" disabled={!video || !character || submitting} onClick={() => submit("montage")}>{submitting ? "Uploading…" : "Generate"}</button>+ <button className="primary" disabled={!video || !character || submitting} onClick={() => submit("montage")}>{submitting ? checkingSetup ? "Checking setup…" : "Uploading…" : "Generate"}</button></>) : (<button className="primary" disabled={!video || !ready || submitting} onClick={() => submit(mode === "object" ? fromPicture ? "simple" : "object" : mode === "post" ? post : mode === "simple" ? "simple" : onePassEach ? "advanced_maxq" : "advanced")}>- {submitting ? "Uploading…" : "Generate"}+ {submitting ? checkingSetup ? "Checking setup…" : "Uploading…" : "Generate"}</button>)}</div>@@ -823,7 +888,8 @@ export function Openjutsu() {: clips && ready? mode === "advanced" && montage ? `${clips} clip${clips > 1 ? "s" : ""} or more at ${canvas ? `${canvas[0]}×${canvas[1]}` : resolution}; Find people (about 3 minutes per 18 s) lets you check the picks first.`: faceFix ? `Up to ${clips} chunk${clips > 1 ? "s" : ""} of face` : `${clips} clip${clips > 1 ? "s" : ""} at ${canvas ? `${canvas[0]}×${canvas[1]}` : resolution}`- : `Add a video and ${mode === "object" ? "say what it becomes" : faceFix ? "the photo of the person" : mode === "simple" ? "a character photo" : "character photos"}.`}</small>+ : mode === "object" && editMode === "add" && video && !placement.length ? "Draw where the new object should go."+ : `Add a video and ${mode === "object" ? editMode === "add" ? "describe the object to add" : "say what it becomes" : faceFix ? "the photo of the person" : mode === "simple" ? "a character photo" : "character photos"}.`}</small></div></>)}@@ -949,7 +1015,7 @@ function MaskCheck({ url, clipSeconds, checked, margin, onMargin, onRecheck, onG<h2>Check the mask</h2><videoref={player} src={url} muted playsInline preload="auto"- onLoadedMetadata={(event) => setFrames(Math.round(event.currentTarget.duration * 24))}+ onLoadedMetadata={(event) => setFrames(Number.isFinite(event.currentTarget.duration) ? Math.round(event.currentTarget.duration * 24) : 0)}onTimeUpdate={(event) => playing && setFrame(Math.min(frames - 1, Math.floor(event.currentTarget.currentTime * 24)))}onEnded={() => setPlaying(false)}/>diff --git a/lib/clips.ts b/lib/clips.tsindex 5fcee5f..517d574 100644--- a/lib/clips.ts+++ b/lib/clips.ts@@ -8,6 +8,7 @@ const gridCeil = (frames: number) => Math.ceil((frames - 5) / 17) * 17 + 5;function clipLengths(durationSeconds: number, clipSeconds: number) {const total = Math.floor(durationSeconds * FPS);const clipFrames = Math.floor(clipSeconds * FPS);+ if (!Number.isFinite(total) || total < 5 || !Number.isFinite(clipFrames) || gridFloor(clipFrames) <= OVERLAP) return [];const lengths = [Math.min(gridFloor(clipFrames), gridFloor(total))];let rem = total - lengths[0];const count = rem >= 17 ? Math.ceil(rem / (gridFloor(clipFrames) - OVERLAP)) : 0;diff --git a/lib/comfy.ts b/lib/comfy.tsindex de05631..8835c4e 100644--- a/lib/comfy.ts+++ b/lib/comfy.ts@@ -1,3 +1,4 @@+import { validateJobId } from "./http.ts";import { readFile } from "node:fs/promises";import { join } from "node:path";import { Comfy, COMFY_CLOUD_BASE_URL, resolveCredentials, type Job } from "@comfyorg/sdk";@@ -189,6 +190,7 @@ export async function listTonePresets(): Promise<{ name: string; settings: Recor}export function getJob(id: string) {+ validateJobId(id);return client().jobs.get(id);}diff --git a/lib/http.ts b/lib/http.tsnew file mode 100644index 0000000..34d7297--- /dev/null+++ b/lib/http.ts@@ -0,0 +1,75 @@+import { createHash, timingSafeEqual } from "node:crypto";++export class HttpError extends Error {+ status: number;++ constructor(status: number, message: string) {+ super(message);+ this.status = status;+ }+}++export function validateJobId(id: string) {+ if (!/^[A-Za-z0-9_-]+$/.test(id) || id.startsWith("http")) throw new HttpError(400, "Invalid job ID.");+}++export function authorize(request: Request) {+ const password = process.env.OPENJUTSU_PASSWORD;+ if (password) {+ const encoded = request.headers.get("Authorization")?.match(/^Basic\s+(\S+)$/i)?.[1] ?? "";+ const supplied = createHash("sha256").update(Buffer.from(encoded, "base64")).digest();+ const expected = createHash("sha256").update(`openjutsu:${password}`).digest();+ if (!timingSafeEqual(supplied, expected)) {+ return Response.json({ error: "Sign in to Openjutsu." }, {+ status: 401,+ headers: { "WWW-Authenticate": 'Basic realm="Openjutsu", charset="UTF-8"', "Cache-Control": "no-store" },+ });+ }+ } else {+ // Password-free use relies on the launcher's loopback binding. Reject other hostnames, including DNS rebinding.+ const local = /^(localhost|127\.0\.0\.1|\[::1\])(:\d+)?$/i;+ const hosts = [new URL(request.url).host, request.headers.get("Host"), request.headers.get("X-Forwarded-Host")];+ if (hosts.some((host) => host !== null && !local.test(host))) {+ return Response.json({ error: "Set OPENJUTSU_PASSWORD before exposing Openjutsu outside localhost." }, { status: 403 });+ }+ }+ return null;+}++export function apiAccess(request: Request) {+ const denied = authorize(request);+ if (denied) return denied;+ const url = new URL(request.url);+ const protocol = request.headers.get("X-Forwarded-Proto");+ const scheme = protocol === "http" || protocol === "https" ? `${protocol}:` : url.protocol;+ const expectedOrigin = new URL(`${scheme}//${request.headers.get("Host") ?? url.host}`).origin;+ const origin = request.headers.get("Origin");+ const site = request.headers.get("Sec-Fetch-Site");+ if ((origin !== null && origin !== expectedOrigin) || (site !== null && site !== "same-origin" && site !== "none")) {+ return Response.json({ error: "Cross-origin API requests are not allowed." }, { status: 403 });+ }+ return null;+}++// Total multipart body, including reference pictures and boundaries; the proxy separately caps each file at 500 MB.+export const MAX_UPLOAD_BYTES = 512 * 1024 * 1024;++export async function readFormData(request: Request, limit = MAX_UPLOAD_BYTES) {+ const tooLarge = () => new HttpError(413, `Uploads must total no more than ${Math.round(limit / 1024 / 1024)} MB.`);+ if (Number(request.headers.get("Content-Length")) > limit) throw tooLarge();+ let size = 0;+ const body = request.body?.pipeThrough(new TransformStream<Uint8Array, Uint8Array>({+ transform(chunk, controller) {+ size += chunk.byteLength;+ if (size > limit) throw tooLarge();+ controller.enqueue(chunk);+ },+ }), { signal: request.signal });+ try {+ return await new Response(body, { headers: request.headers }).formData();+ } catch (error) {+ // Multipart parsers may wrap the stream error in a TypeError.+ if (size > limit) throw tooLarge();+ throw error;+ }+}diff --git a/lib/object.ts b/lib/object.tsindex 10f447e..e954bb7 100644--- a/lib/object.ts+++ b/lib/object.ts@@ -4,13 +4,34 @@import { nodeByTitle } from "./advanced.ts";import { setMotionSize, setResolution, type ApiGraph, type MotionSize, type Resolution } from "./simple.ts";-export const EDIT_MODES = ["change", "remove", "background"] as const;+export const EDIT_MODES = ["change", "remove", "background", "add"] as const;export type EditMode = (typeof EDIT_MODES)[number];// Edit-mask grow in pixels at SAM's 1008 px canvas. For background, how far the new setting reaches into the edge of what stays.export const OBJECT_MARGINS = { tight: 10, normal: 20, loose: 40 } as const;export type ObjectMargin = keyof typeof OBJECT_MARGINS;+export type PlacementKeyframe = { frame: number; x: number; y: number; width: number; height: number };++export function parsePlacement(value: unknown): PlacementKeyframe[] {+ const points = typeof value === "string" ? JSON.parse(value) : value;+ if (!Array.isArray(points) || !points.length) throw new Error("Draw an object placement area first.");+ if (!points.every((p) => p && Number.isInteger(p.frame) && [p.x, p.y, p.width, p.height].every(Number.isFinite)+ && p.x >= 0 && p.y >= 0 && p.x < 1 && p.y < 1 && p.width > 0 && p.height > 0+ && p.x + p.width <= 1 + 1e-6 && p.y + p.height <= 1 + 1e-6)) throw new Error("Keep the placement area inside the video frame.");+ return points.map(({ frame, x, y, width, height }) => ({ frame, x, y, width, height })).sort((a, b) => a.frame - b.frame);+}++export function placementAt(points: PlacementKeyframe[], frame: number) {+ if (!points.length) return null;+ const next = points.findIndex((p) => p.frame > frame);+ const a = points[Math.max(0, next === -1 ? points.length - 1 : next - 1)];+ const b = points[next === -1 ? points.length - 1 : next];+ const t = a.frame === b.frame ? 0 : Math.max(0, Math.min(1, (frame - a.frame) / (b.frame - a.frame)));+ return { x: a.x + (b.x - a.x) * t, y: a.y + (b.y - a.y) * t,+ width: a.width + (b.width - a.width) * t, height: a.height + (b.height - a.height) * t };+}+export type ObjectSettings = {mode: EditMode;target: string; // what SAM tracks, several separated by commas; for background, what stays@@ -22,6 +43,7 @@ export type ObjectSettings = {margin: ObjectMargin;seed: number;clipSeconds: number;+ placement?: PlacementKeyframe[];};export const OBJECT_VIDEO = "Source video";@@ -33,13 +55,13 @@ export const objectTargets = (target: string) => target.split(",").map((t) => t.// Asked for "the top", it described only the tee under an open flannel shirt, so layers are named outer first.function describer(settings: ObjectSettings) {const background = settings.mode === "background";- const things = background ? ["place (the setting, its lighting and what is around)"] : objectTargets(settings.target);+ const things = background ? ["place (the setting, its lighting and what is around)"] : settings.mode === "add" ? [settings.changeTo.replace(/^(?:a|an|the)\s+/i, "") || "main object"] : objectTargets(settings.target);const named = things.length === 1 ? things[0] : things.slice(0, -1).join(", ") + " and the " + things.at(-1);const words = 30 + 20 * things.length;return {prompt: `Describe the ${named} in this picture in about ${words} words${things.length > 1 ? ", one sentence for each" : ""}: their shape, colours, `+ "materials, patterns and details. "- + (background ? "" : "Include every layer worn there, outer layer first: an open shirt or jacket over a top is part of the top. ")+ + (background || settings.mode === "add" ? "" : "Include every layer worn there, outer layer first: an open shirt or jacket over a top is part of the top. ")+ "No preamble, and nothing about anything else in the picture.",max_length: 2 * words + 40,};@@ -85,6 +107,19 @@ export function pruneObject(source: ApiGraph, picture: boolean, settings: Objectseed: settings.seed,clip_seconds: settings.clipSeconds,});+ if (settings.mode === "add") {+ const placement = JSON.stringify(parsePlacement(settings.placement));+ plan.placement = placement;+ plan.keep_outline = false;+ ref["ref_videos.ref_video_0"] = [id("Motion reference for this clip"), 0];+ graph.object_placement = { class_type: "H3ObjectPlacementMask", inputs: {+ images: [id("Source clip (frames + audio)"), 0], start_frame: [id("Clip start frame"), 1], placement,+ }, _meta: { title: "Object placement area" } };+ grow.mask = ["object_placement", 0];+ grow.expand = { tight: 0, normal: 2, loose: 4 }[settings.margin];+ for (const title of ["SAM 3.1", "SAM: what to change", "SAM video (1008 px, SAM's own size)", "SAM frames for this clip", "SAM 3.1 track it",+ "Tracked area", "Tracked area, tight (for the grey shape)", "Turn it grey (shape and movement only)"]) delete graph[id(title)];+ }setResolution(graph, id("Object Swap Plan (what to change)"), id(OBJECT_VIDEO), settings.resolution);return graph;}diff --git a/lib/setup.ts b/lib/setup.tsnew file mode 100644index 0000000..218fbbc--- /dev/null+++ b/lib/setup.ts@@ -0,0 +1,102 @@+import { pruneAdvanced, type AdvancedSettings } from "./advanced.ts";+import { pruneColor, type ColorSettings } from "./color.ts";+import { pruneFace, type FaceSettings } from "./face.ts";+import { applyModel, type ModelSettings } from "./model.ts";+import { PEOPLE_NODES, pruneMontage, type Review } from "./montage.ts";+import { pruneObject, type ObjectSettings } from "./object.ts";+import { cropSource, pruneMask, pruneSimple, type ApiGraph, type Margin, type Resolution, type SimpleSettings } from "./simple.ts";++export type SetupRequest =+ | { kind: "simple"; settings: SimpleSettings; model: ModelSettings }+ | { kind: "advanced" | "advanced_maxq"; settings: AdvancedSettings; pictures: number; model: ModelSettings }+ | { kind: "object"; settings: ObjectSettings; picture: boolean; model: ModelSettings }+ | { kind: "face"; settings: FaceSettings; model: ModelSettings }+ | { kind: "color"; settings: ColorSettings }+ | { kind: "mask"; hatFix: boolean; margin: Margin; target: string }+ | { kind: "people"; resolution: Resolution }+ | { kind: "montage"; settings: AdvancedSettings; pictures: number; review: Review | null; model: ModelSettings };++export type SetupResult = {+ ok: boolean;+ error?: string;+ packs: { name: string; url: string; nodes: string[] }[];+ models: { name: string; folder: string }[];+};++type NodeInfo = { input?: { required?: Record<string, unknown[]>; optional?: Record<string, unknown[]> } };+export type NodeRegistry = Record<string, NodeInfo>;++const PACKS = [+ { name: "ComfyUI-MiniMaxH3-Context-Loop (0.7.3+)", url: "https://github.com/ethanfel/ComfyUI-MiniMaxH3-Context-Loop", nodes: ["MiniMaxH3GenerationProfile", "MiniMaxH3SourceTimeline", "MiniMaxH3ChainPlanModern", "MiniMaxH3ChainLoopStart", "MiniMaxH3ChainCurrent", "MiniMaxH3ChainContext", "MiniMaxH3ChainSegmentSave", "MiniMaxH3ChainReview", "MiniMaxH3ChainLoopEnd", "MiniMaxH3ChainAssemble", "MiniMaxH3ContexLoopSourceSlice", "MiniMaxH3ContexMaskedTarget", "MiniMaxH3ContexMaskGridPreview", "MiniMaxH3LoopTrim"] },+ { name: "comfyui-h3-oneclick-swap", url: "https://github.com/D3voz/comfyui-h3-oneclick-swap", nodes: ["H3OneClickSwapPlan", "H3AdvancedSwapPlan", "H3ObjectSwapPlan", "H3ObjectPlacementMask", "H3PeopleMask", "H3GreyMannequins", "H3ClipShots", "H3FillShots", "H3TrackCrops", "H3TrackRoles", "H3RoleMask", "H3SubjectPrompt", "H3MontagePeople", "H3MontageTracks", "H3MontagePicks", "H3MontagePlan", "H3MontagePass", "H3FaceChunks", "H3FaceJoin"] },+ { name: "ComfyUI-Fantastic-MiniMaxH3-PromptBuilder", url: "https://github.com/Adudeguyman/ComfyUI-Fantastic-MiniMaxH3-PromptBuilder", nodes: ["MiniMaxH3FantasticVideoEditLatent"] },+ { name: "ComfyUI-KJNodes", url: "https://github.com/kijai/ComfyUI-KJNodes", nodes: ["GrowMaskWithBlur", "ImageAndMaskPreview"] },+ { name: "ComfyUI-VideoHelperSuite", url: "https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite", nodes: ["VHS_LoadVideo", "VHS_SelectEveryNthImage", "VHS_VideoCombine"] },+ { name: "ComfyUI-H3-FaceRefine", url: "https://github.com/Carasibana/ComfyUI-H3-FaceRefine", nodes: ["H3FaceTrackCrop", "H3FaceMaskSAM", "H3FaceStitch"] },+ { name: "ComfyUI-Impact-Pack", url: "https://github.com/ltdrdata/ComfyUI-Impact-Pack", nodes: ["SAMLoader"] },+ { name: "comfyui-image-detail-contrast", url: "https://github.com/D3voz/comfyui-image-detail-contrast", nodes: ["ImageDetailContrast"] },+ { name: "Orion4D_FXMax", url: "https://github.com/orion4d/Orion4D_FXMax", nodes: ["Orion4D_FXMax_DCTLToneMapper"] },+];+const CORE = { name: "ComfyUI (update required)", url: "https://github.com/Comfy-Org/ComfyUI", nodes: [] as string[] };+const MODEL_INPUTS: Record<string, Record<string, string>> = {+ UNETLoader: { unet_name: "diffusion_models" },+ LoraLoaderModelOnly: { lora_name: "loras" },+ CLIPLoader: { clip_name: "text_encoders" },+ VAELoader: { vae_name: "vae" },+ CheckpointLoaderSimple: { ckpt_name: "checkpoints" },+ SAMLoader: { model_name: "sams" },+ H3FaceTrackCrop: { detector: "ultralytics", fallback_detector: "ultralytics" },+};++export function setupGraph(source: ApiGraph, selection: SetupRequest): ApiGraph {+ let graph: ApiGraph;+ if (selection.kind === "simple") graph = pruneSimple(source, selection.settings);+ else if (selection.kind === "object") graph = pruneObject(source, selection.picture, selection.settings);+ else if (selection.kind === "face") graph = pruneFace(source, selection.settings);+ else if (selection.kind === "color") graph = pruneColor(source, selection.settings);+ else if (selection.kind === "mask") graph = pruneMask(source, selection.hatFix, selection.margin, selection.target);+ else if (selection.kind === "montage") graph = pruneMontage(source, selection.pictures, selection.settings, selection.review);+ else if (selection.kind === "people") {+ graph = structuredClone(source);+ if (typeof selection.resolution === "object") cropSource(graph, PEOPLE_NODES.video, selection.resolution);+ } else graph = pruneAdvanced(source, selection.pictures, selection.settings);+ if ("model" in selection) applyModel(graph, selection.model);+ return graph;+}++export function inspectSetup(graph: ApiGraph, registry: NodeRegistry): SetupResult {+ const packs: SetupResult["packs"] = [];+ const models: SetupResult["models"] = [];+ for (const type of new Set(Object.values(graph).map((node) => node.class_type))) {+ if (Object.hasOwn(registry, type)) continue;+ const pack = PACKS.find((p) => p.nodes.includes(type)) ?? CORE;+ let missing = packs.find((p) => p.url === pack.url);+ if (!missing) packs.push(missing = { name: pack.name, url: pack.url, nodes: [] });+ missing.nodes.push(type);+ }+ for (const node of Object.values(graph)) {+ const schema = registry[node.class_type]?.input;+ for (const [input, folder] of Object.entries(MODEL_INPUTS[node.class_type] ?? {})) {+ const name = node.inputs[input];+ const options = (schema?.required?.[input] ?? schema?.optional?.[input])?.[0];+ if (typeof name !== "string" || name === "none" || !Array.isArray(options)) continue;+ const normalized = name.replaceAll("\\", "/");+ if (!options.some((value) => typeof value === "string" && value.replaceAll("\\", "/") === normalized)+ && !models.some((m) => m.name === name && m.folder === folder)) models.push({ name, folder });+ }+ }+ return { ok: packs.length === 0 && models.length === 0, packs, models };+}++export async function checkSetup(graph: ApiGraph): Promise<SetupResult> {+ try {+ const url = process.env.COMFYUI_URL || "http://127.0.0.1:8188";+ const response = await fetch(`${url.replace(/\/+$/, "")}/object_info`, { cache: "no-store", signal: AbortSignal.timeout(8000) });+ if (!response.ok) throw new Error("Node list unavailable");+ const registry = await response.json();+ if (!registry || typeof registry !== "object" || Array.isArray(registry)) throw new Error("Invalid node list");+ return inspectSetup(graph, registry);+ } catch {+ return { ok: false, error: "Could not check ComfyUI. Make sure it is running and COMFYUI_URL points to it, then try again.", packs: [], models: [] };+ }+}diff --git a/proxy.ts b/proxy.tsnew file mode 100644index 0000000..cce483f--- /dev/null+++ b/proxy.ts@@ -0,0 +1,9 @@+import { NextResponse } from "next/server";+import { authorize } from "./lib/http.ts";++export function proxy(request: Request) {+ return authorize(request) ?? NextResponse.next();+}++// API routes enforce access themselves, so large uploads never pass through Proxy's body cloning.+export const config = { matcher: "/" };diff --git a/test/http.test.ts b/test/http.test.tsnew file mode 100644index 0000000..8db6f33--- /dev/null+++ b/test/http.test.ts@@ -0,0 +1,91 @@+import assert from "node:assert/strict";+import test from "node:test";+import { apiAccess, authorize, HttpError, readFormData, validateJobId } from "../lib/http.ts";+import { getJob } from "../lib/comfy.ts";++test("URL-shaped job IDs are rejected before the SDK can fetch", async (t) => {+ let requests = 0;+ t.mock.method(globalThis, "fetch", async () => { requests++; throw new Error("Unexpected network request"); });+ for (const id of ["http://127.0.0.1:8188/queue", "https://example.com", "/api/v2/assets", "..", "a/b", "a\\b", "a%2fb", "http-other", ""]) {+ await assert.rejects(async () => getJob(id), (error: unknown) => error instanceof HttpError && error.status === 400);+ }+ assert.equal(requests, 0);+ for (const id of ["d544f17b-2c81-40c4-9cd1-9624954038e6", "job_abc-123"]) assert.doesNotThrow(() => validateJobId(id));+});++test("local API access stays password-free; foreign hosts and browser origins are rejected", (t) => {+ const previous = process.env.OPENJUTSU_PASSWORD;+ delete process.env.OPENJUTSU_PASSWORD;+ t.after(() => { if (previous === undefined) delete process.env.OPENJUTSU_PASSWORD; else process.env.OPENJUTSU_PASSWORD = previous; });+ for (const host of ["127.0.0.1:3000", "localhost:3000", "[::1]:3000"]) {+ const url = `http://${host}`;+ assert.equal(apiAccess(new Request(`${url}/api/jobs`, { headers: { Host: host, Origin: url, "Sec-Fetch-Site": "same-origin" } })), null);+ }+ assert.equal(apiAccess(new Request("http://localhost:3000/api/jobs", { headers: { Host: "127.0.0.1:3000", Origin: "http://127.0.0.1:3000" } })), null, "Next may use its internal hostname in request.url");+ assert.equal(apiAccess(new Request("http://localhost:3000/api/jobs", { headers: { Host: "localhost:3000", Origin: "https://localhost:3000", "X-Forwarded-Proto": "https" } })), null, "reverse proxies preserve the public scheme");+ const foreignHeaders: HeadersInit[] = [+ { Origin: "https://example.com" }, { Origin: "null" }, { "Sec-Fetch-Site": "cross-site" },+ { "Sec-Fetch-Site": "same-site" }, { Host: "example.com" }, { "X-Forwarded-Host": "example.com" },+ ];+ for (const headers of foreignHeaders) assert.equal(apiAccess(new Request("http://127.0.0.1:3000/api/jobs", { headers }))?.status, 403);+ assert.equal(authorize(new Request("https://example.com"))?.status, 403);+ assert.equal(authorize(new Request("http://localhost:3000", { headers: { "Sec-Fetch-Site": "cross-site" } })), null, "ordinary page navigation is allowed");+});++test("a configured password protects local and hosted access, including media and event requests", (t) => {+ const previous = process.env.OPENJUTSU_PASSWORD;+ process.env.OPENJUTSU_PASSWORD = "review-test-password";+ t.after(() => { if (previous === undefined) delete process.env.OPENJUTSU_PASSWORD; else process.env.OPENJUTSU_PASSWORD = previous; });+ const authorization = `Basic ${Buffer.from("openjutsu:review-test-password").toString("base64")}`;+ for (const host of ["http://127.0.0.1:3000", "https://openjutsu.example"]) {+ for (const path of ["/", "/api/models", "/api/jobs/job_123/events", "/api/jobs/job_123/outputs/asset_1"]) {+ const url = `${host}${path}`;+ assert.equal(authorize(new Request(url))?.status, 401);+ assert.equal(apiAccess(new Request(url, { headers: { Authorization: authorization, "Sec-Fetch-Site": "same-origin" } })), null);+ assert.equal(authorize(new Request(url, { headers: { Authorization: "Basic !!!" } }))?.status, 401);+ }+ }+ assert.equal(apiAccess(new Request("http://localhost:3000/api/jobs", { headers: { Authorization: authorization, Host: "openjutsu.example", Origin: "https://openjutsu.example", "X-Forwarded-Proto": "https" } })), null, "public origin must not inherit the internal port");+ assert.equal(apiAccess(new Request("http://localhost:3000/api/jobs", { headers: { Authorization: authorization, Host: "openjutsu.example", Origin: "https://openjutsu.example:3000", "X-Forwarded-Proto": "https" } }))?.status, 403);+ assert.match(authorize(new Request("http://localhost:3000"))!.headers.get("WWW-Authenticate")!, /^Basic /);+ assert.equal(apiAccess(new Request("https://openjutsu.example/api/jobs", { headers: { Authorization: authorization, Origin: "https://attacker.example" } }))?.status, 403);+});++test("bounded multipart parsing preserves video bytes, duplicate pictures and roles", async () => {+ const form = new FormData();+ form.set("video", new File([new Uint8Array([0, 1, 255])], "clip.mp4", { type: "video/mp4" }));+ for (const name of ["one", "two"]) {+ form.append("picture", new File([name], `${name}.png`, { type: "image/png" }));+ form.append("role", name);+ }+ const parsed = await readFormData(new Request("http://localhost/api/jobs", { method: "POST", body: form }), 4096);+ assert.deepEqual(parsed.getAll("role"), ["one", "two"]);+ assert.deepEqual(parsed.getAll("picture").map((file) => (file as File).name), ["one.png", "two.png"]);+ assert.deepEqual(new Uint8Array(await (parsed.get("video") as File).arrayBuffer()), new Uint8Array([0, 1, 255]));+});++test("declared oversize bodies are rejected without reading them", async () => {+ let read = false;+ const body = new ReadableStream({ pull() { read = true; } }, { highWaterMark: 0 });+ const init = { method: "POST", body, duplex: "half", headers: { "Content-Length": "2048" } };+ await assert.rejects(readFormData(new Request("http://localhost/api/jobs", init), 1024), (e: unknown) => e instanceof HttpError && e.status === 413);+ assert.equal(read, false);+});++test("chunked uploads cannot bypass the limit with a missing or false Content-Length", async () => {+ for (const length of [null, "1"]) {+ let pulls = 0;+ let canceled = false;+ const body = new ReadableStream<Uint8Array>({+ pull(controller) { pulls++; controller.enqueue(new Uint8Array(512)); },+ cancel() { canceled = true; },+ }, { highWaterMark: 0 });+ const headers = new Headers({ "Content-Type": "multipart/form-data; boundary=test" });+ if (length !== null) headers.set("Content-Length", length);+ const init = { method: "POST", body, duplex: "half", headers };+ await assert.rejects(readFormData(new Request("http://localhost/api/jobs", init), 1024), (e: unknown) => e instanceof HttpError && e.status === 413);+ await new Promise<void>((resolve) => setImmediate(resolve));+ assert.equal(canceled, true);+ assert.equal(pulls, 3);+ }+});diff --git a/test/object.test.ts b/test/object.test.tsindex 2c1e7be..e7338a5 100644--- a/test/object.test.ts+++ b/test/object.test.ts@@ -3,7 +3,7 @@ import { readFileSync } from "node:fs";import test from "node:test";import { nodeByTitle } from "../lib/advanced.ts";import { applyModel, DEFAULT_MODEL } from "../lib/model.ts";-import { OBJECT_PICTURE, OBJECT_VIDEO, pruneObject, type ObjectSettings } from "../lib/object.ts";+import { OBJECT_PICTURE, OBJECT_VIDEO, parsePlacement, placementAt, pruneObject, type ObjectSettings } from "../lib/object.ts";import { isLink, type ApiGraph } from "../lib/simple.ts";const graph: ApiGraph = JSON.parse(readFileSync(new URL("../workflows/object.api.json", import.meta.url), "utf8"));@@ -78,3 +78,47 @@ test("the model settings reach the Object Swap Plan", () => {assert.equal(node(pruned, "Object Swap Plan (what to change)").inputs.steps, 9);assert.deepEqual(dangling(pruned), []);});++test("adding an object uses placement keyframes and the original video, without SAM or grey references", () => {+ const placement = [{ frame: 0, x: 0.3, y: 0.4, width: 0.2, height: 0.25 }];+ for (const picture of [false, true]) {+ const pruned = pruneObject(graph, picture, { ...settings("add"), target: "on the table", changeTo: "a cake", placement });+ assert.deepEqual(dangling(pruned), []);+ assert.ok(!Object.values(pruned).some((n) => ["SAM3_VideoTrack", "H3PeopleMask", "H3GreyMannequins", "CheckpointLoaderSimple"].includes(n.class_type)));+ const ref = node(pruned, "Reference Conditioning").inputs;+ assert.deepEqual(ref["ref_videos.ref_video_0"], [nodeByTitle(pruned, "Motion reference for this clip"), 0]);+ assert.equal(!!ref["ref_images.ref_image_0"], picture);+ const plan = node(pruned, "Object Swap Plan (what to change)").inputs;+ assert.equal(plan.mode, "add");+ assert.equal(plan.keep_outline, false);+ assert.equal(plan.placement, JSON.stringify(placement));+ assert.equal(pruned.object_placement.class_type, "H3ObjectPlacementMask");+ assert.equal(pruned.object_placement.inputs.placement, plan.placement);+ assert.deepEqual(pruned.object_placement.inputs.images, [nodeByTitle(pruned, "Source clip (frames + audio)"), 0]);+ assert.deepEqual(pruned.object_placement.inputs.start_frame, [nodeByTitle(pruned, "Clip start frame"), 1]);+ assert.deepEqual(node(pruned, "Grow mask").inputs.mask, ["object_placement", 0]);+ }+});++test("add placements use the selected canvas and the model settings still reach generation", () => {+ const pruned = pruneObject(graph, false, { ...settings("add"), placement: [{ frame: -24, x: 0.2, y: 0.4, width: 0.3, height: 0.2 }], resolution: { width: 640, height: 960 } });+ applyModel(pruned, { ...DEFAULT_MODEL, steps: 8 });+ assert.deepEqual(dangling(pruned), []);+ assert.equal(node(pruned, "Object Swap Plan (what to change)").inputs.steps, 8);+ assert.deepEqual(node(pruned, "Source clip (frames + audio)").inputs.source_frames, ["custom_size", 0]);+ assert.deepEqual(node(pruned, "Motion reference video (~512 px)").inputs.image, ["custom_size", 0]);+});++test("placement validation rejects missing or invalid geometry and sorts valid keyframes", () => {+ for (const placement of [null, [], "{", [{ frame: 0, x: -0.1, y: 0.2, width: 0.2, height: 0.2 }],+ [{ frame: 0, x: 0.9, y: 0.2, width: 0.2, height: 0.2 }], [{ frame: 0, x: 0.2, y: 0.2, width: 0, height: 0.2 }]]) {+ assert.throws(() => parsePlacement(placement));+ }+ const points = [{ frame: 48, x: 0.6, y: 0.2, width: 0.2, height: 0.2 }, { frame: 0, x: 0.2, y: 0.4, width: 0.2, height: 0.2 }];+ const sorted = parsePlacement(JSON.stringify(points));+ assert.deepEqual(sorted.map((p) => p.frame), [0, 48]);+ assert.deepEqual(placementAt(sorted, -24), { x: 0.2, y: 0.4, width: 0.2, height: 0.2 });+ const middle = placementAt(sorted, 24)!;+ assert.ok(Math.abs(middle.x - 0.4) < 1e-6 && Math.abs(middle.y - 0.3) < 1e-6);+ assert.deepEqual(placementAt(sorted, 96), { x: 0.6, y: 0.2, width: 0.2, height: 0.2 });+});diff --git a/test/setup.test.ts b/test/setup.test.tsnew file mode 100644index 0000000..7d1abdf--- /dev/null+++ b/test/setup.test.ts@@ -0,0 +1,162 @@+import assert from "node:assert/strict";+import { readFileSync } from "node:fs";+import test from "node:test";+import { POST } from "../app/api/setup/route.ts";+import { DEFAULT_TONE } from "../lib/color.ts";+import { JOB_KINDS } from "../lib/comfy.ts";+import { DEFAULT_MODEL } from "../lib/model.ts";+import { checkSetup, inspectSetup, setupGraph, type NodeRegistry, type SetupRequest } from "../lib/setup.ts";+import type { ApiGraph, SimpleSettings } from "../lib/simple.ts";++const load = (kind: string): ApiGraph => JSON.parse(readFileSync(new URL(`../workflows/${kind}.api.json`, import.meta.url), "utf8"));+const settings: SimpleSettings = { role: "the drummer", resolution: "480p", motion: 512, hatFix: false, noLipSync: false, margin: "normal", target: "person", seed: 0, clipSeconds: 6 };+const simple: SetupRequest = { kind: "simple", settings, model: DEFAULT_MODEL };++function registryFor(...graphs: ApiGraph[]): NodeRegistry {+ const registry: NodeRegistry = {};+ for (const graph of graphs) for (const node of Object.values(graph)) {+ const required = (registry[node.class_type] ??= { input: { required: {} } }).input!.required!;+ for (const [key, value] of Object.entries(node.inputs)) if (typeof value === "string") {+ const options = (required[key] ??= [[]])[0] as string[];+ if (!options.includes(value)) options.push(value);+ }+ }+ return registry;+}++test("all nine workflows pass with their nodes and model filenames registered", () => {+ for (const kind of JOB_KINDS) {+ const graph = load(kind);+ assert.deepEqual(inspectSetup(graph, registryFor(graph)), { ok: true, packs: [], models: [] }, kind);+ }+});++test("missing nodes are grouped by their source pack, including both published D3voz repos", () => {+ const graph = { ...load("simple"), ...Object.fromEntries(Object.entries(load("color")).map(([id, node]) => [`color_${id}`, node])) };+ const registry = registryFor(graph);+ for (const name of ["H3PeopleMask", "H3ClipShots", "ImageDetailContrast", "VHS_LoadVideo"]) delete registry[name];+ const result = inspectSetup(graph, registry);+ assert.equal(result.ok, false);+ assert.equal(result.packs.length, 3);+ assert.deepEqual(result.packs.find((p) => p.url === "https://github.com/D3voz/comfyui-h3-oneclick-swap")?.nodes.sort(), ["H3ClipShots", "H3PeopleMask"]);+ assert.equal(result.packs.find((p) => p.nodes.includes("ImageDetailContrast"))?.url, "https://github.com/D3voz/comfyui-image-detail-contrast");+});++test("all custom node types in the exports have a pack mapping", () => {+ const custom = /^(H3|MiniMaxH3(?!ReferenceToVideo|SigmaShift)|VHS_|GrowMaskWithBlur$|ImageAndMaskPreview$|ImageDetailContrast$|Orion4D_|SAMLoader$)/;+ const graphs = JOB_KINDS.map(load);+ const missing = inspectSetup(Object.fromEntries(graphs.flatMap((g, i) => Object.entries(g).map(([id, node]) => [`${i}_${id}`, node]))), {});+ const core = missing.packs.find((p) => p.url === "https://github.com/Comfy-Org/ComfyUI")!;+ assert.ok(core.nodes.includes("SAM3_VideoTrack"));+ assert.ok(core.nodes.every((name) => !custom.test(name)), core.nodes.join(", "));+});++test("disabled colour steps and unrelated features do not block a job", () => {+ for (const selected of [{ detail: null, tone: DEFAULT_TONE }, { detail: -0.2, tone: null }]) {+ const graph = setupGraph(load("color"), { kind: "color", settings: selected });+ const registry = registryFor(graph);+ delete registry[selected.detail === null ? "ImageDetailContrast" : "Orion4D_FXMax_DCTLToneMapper"];+ assert.equal(inspectSetup(graph, registry).ok, true);+ }+ const graph = setupGraph(load("simple"), simple);+ const registry = registryFor(graph);+ delete registry.H3FaceTrackCrop;+ delete registry.SAM3_TrackToMask;+ assert.equal(inspectSetup(graph, registry).ok, true, "the hat and face-tracking branches are not used");+});++test("adding an object checks the new placement node but does not require SAM", () => {+ const selection: SetupRequest = { kind: "object", picture: false, model: DEFAULT_MODEL, settings: {+ mode: "add", target: "on the table", changeTo: "a cake", biggestOnly: false, keepOutline: false,+ resolution: "480p", motion: 512, margin: "normal", seed: 0, clipSeconds: 6,+ placement: [{ frame: 0, x: 0.3, y: 0.4, width: 0.2, height: 0.2 }],+ } };+ const graph = setupGraph(load("object"), selection);+ const registry = registryFor(graph);+ assert.equal(inspectSetup(graph, registry).ok, true);+ delete registry.H3ObjectPlacementMask;+ const missing = inspectSetup(graph, registry);+ assert.equal(missing.packs.length, 1);+ assert.equal(missing.packs[0].url, "https://github.com/D3voz/comfyui-h3-oneclick-swap");+ assert.deepEqual(missing.packs[0].nodes, ["H3ObjectPlacementMask"]);+});++test("the selected base model and LoRAs replace the workflow defaults in the check", () => {+ const graph = setupGraph(load("simple"), { ...simple, model: { model: "other.safetensors", loras: [{ name: "style.safetensors", strength: 0.8 }], steps: 8 } });+ const registry = registryFor(graph);+ assert.equal(inspectSetup(graph, registry).ok, true);+ registry.UNETLoader.input!.required!.unet_name = [[]];+ registry.LoraLoaderModelOnly.input!.required!.lora_name = [[]];+ assert.deepEqual(inspectSetup(graph, registry).models, [+ { name: "other.safetensors", folder: "diffusion_models" },+ { name: "style.safetensors", folder: "loras" },+ ]);+ const noLora = setupGraph(load("simple"), { ...simple, model: { ...DEFAULT_MODEL, loras: [] } });+ const noLoraRegistry = registryFor(noLora);+ delete noLoraRegistry.LoraLoaderModelOnly;+ assert.equal(inspectSetup(noLora, noLoraRegistry).ok, true);+});++test("model checks deduplicate files, normalize path separators, and ignore upload placeholders and preset names", () => {+ const graph: ApiGraph = {+ a: { class_type: "CLIPLoader", inputs: { clip_name: "missing.safetensors" } },+ b: { class_type: "CLIPLoader", inputs: { clip_name: "missing.safetensors" } },+ c: { class_type: "H3FaceTrackCrop", inputs: { detector: "bbox\\face.pt", fallback_detector: "none" } },+ d: { class_type: "LoadImage", inputs: { image: "example.png" } },+ e: { class_type: "Orion4D_FXMax_DCTLToneMapper", inputs: { preset: "Openjutsu" } },+ };+ const registry = registryFor(graph);+ registry.CLIPLoader.input!.required!.clip_name = [[]];+ registry.H3FaceTrackCrop.input!.required!.detector = [["bbox/face.pt"]];+ registry.LoadImage.input!.required!.image = [[]];+ registry.Orion4D_FXMax_DCTLToneMapper.input!.required!.preset = [["Other"]];+ assert.deepEqual(inspectSetup(graph, registry).models, [{ name: "missing.safetensors", folder: "text_encoders" }]);+});++test("setup reads the registry once and never uploads or submits a job", async (t) => {+ const graph = setupGraph(load("simple"), simple);+ const urls: string[] = [];+ t.mock.method(globalThis, "fetch", async (url: string) => {+ urls.push(url);+ return Response.json(registryFor(graph));+ });+ assert.equal((await checkSetup(graph)).ok, true);+ assert.equal(urls.length, 1);+ assert.ok(urls[0].endsWith("/object_info"));+});++test("offline, HTTP failure and invalid registry responses give an actionable error", async (t) => {+ for (const response of [new Response("offline", { status: 503 }), Response.json(null)]) {+ const mock = t.mock.method(globalThis, "fetch", async () => response);+ const result = await checkSetup(load("mask"));+ assert.equal(result.ok, false);+ assert.match(result.error!, /COMFYUI_URL/);+ mock.mock.restore();+ }+ t.mock.method(globalThis, "fetch", async () => { throw new Error("connection refused"); });+ assert.equal((await checkSetup(load("mask"))).ok, false);+});++test("setup route rejects invalid modes and foreign origins before contacting ComfyUI", async (t) => {+ let requests = 0;+ t.mock.method(globalThis, "fetch", async () => { requests++; throw new Error("Unexpected request"); });+ const request = (body: unknown, origin = "http://127.0.0.1:3000") => new Request("http://127.0.0.1:3000/api/setup", {+ method: "POST", headers: { Origin: origin, "Content-Type": "application/json" }, body: JSON.stringify(body),+ });+ assert.equal((await POST(request({ kind: "../../secret" }))).status, 400);+ assert.equal((await POST(request({ kind: "advanced", pictures: 500 }))).status, 400);+ assert.equal((await POST(request(simple, "https://example.com"))).status, 403);+ assert.equal(requests, 0);+});++test("setup route returns missing packs for the chosen workflow without submitting", async (t) => {+ const graph = setupGraph(load("simple"), simple);+ const registry = registryFor(graph);+ delete registry.H3OneClickSwapPlan;+ t.mock.method(globalThis, "fetch", async () => Response.json(registry));+ const response = await POST(new Request("http://127.0.0.1:3000/api/setup", { method: "POST", body: JSON.stringify(simple) }));+ const result = await response.json();+ assert.equal(response.status, 200);+ assert.equal(result.ok, false);+ assert.equal(result.packs[0].name, "comfyui-h3-oneclick-swap");+});diff --git a/test/simple.test.ts b/test/simple.test.tsindex 06440b9..304153f 100644--- a/test/simple.test.ts+++ b/test/simple.test.ts@@ -105,6 +105,14 @@ test("node ids map to titles and clip numbers", () => {assert.equal(describe(NODES.final)?.clip, null);});+test("unknown duration and invalid clip lengths do not enter the planning loop", () => {+ for (const duration of [Infinity, NaN, -1, 0]) {+ assert.equal(clipCount(duration, 6), 0);+ assert.deepEqual(clipStarts(duration, 6), [0]);+ }+ for (const length of [Infinity, NaN, 0, 39 / 24]) assert.equal(clipCount(12, length), 0);+});+test("clip count follows the Swap Plan grid", () => {assert.equal(clipCount(6, 6), 1);assert.equal(clipCount(16, 6), 4);