Skip to content

[SIGGRAPH Asia 2026] 4DAnyone: Create Anyone in 4D from a Casual Monocular Video

Release: simplify clip outputs and Nerfstudio reconstruction (#32)

Yudong Jin3d0342acd785
diff --git a/README.md b/README.md
index 5d21b8e..60551e8 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,7 @@
>
> - generates dozens of synchronized, view-consistent videos from a single monocular video.
> - requires **22 GB** of peak CUDA memory, enabling inference on consumer GPUs.
-> - generates a 121-frame video in **27 seconds** on a single RTX 4090.
+> - averages **27 seconds per 121-frame video** on a single RTX 4090.
## News
@@ -57,6 +57,7 @@ A compact 360° layout for basic coverage. Start here for an initial test.
```bash
python inference.py \
--video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
+ --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
--views_per_layer 6
```
@@ -69,6 +70,7 @@ A dense 360° layout with broad angular coverage, suitable for 4DGS reconstructi
```bash
python inference.py \
--video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
+ --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
--views_per_layer 24
```
@@ -81,6 +83,7 @@ This layout distributes views across three pitch rings for broader coverage, ena
```bash
python inference.py \
--video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
+ --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
--views_per_layer 16 --layer_pitches '[-10,15,35]'
```
@@ -93,6 +96,7 @@ A two-layer layout for dense coverage across the frontal 180° arc.
```bash
python inference.py \
--video_path "data/source/pexels/2785536-uhd_2160_3840_25fps.mp4" \
+ --output_dir "data/fdanyone/pexels/2785536-uhd_2160_3840_25fps" \
--views_per_layer 12 --layer_pitches '[0,30]' --start_yaw -90 --yaw_span 180
```
@@ -102,6 +106,8 @@ python inference.py \
Run `python inference.py --help` for the full list.
+- `video_path`: path to the source video.
+- `output_dir`: output directory for the current clip. Defaults to `data/fdanyone/<clip>`.
- `views_per_layer`: number of evenly spaced views per pitch layer. It must be divisible by 4 or 6.
- `layer_pitches`: pitch angles in degrees, one per layer. Positive values place cameras above the subject. Total views are `views_per_layer × len(layer_pitches)`.
- `start_yaw`: horizontal angle of the first view, in degrees. Yaw `0` is the front view.
@@ -111,20 +117,23 @@ Run `python inference.py --help` for the full list.
### Output
-With the default `--data_dir data`, results follow this layout. See the [output documentation](docs/output.md) for the complete format.
-
-```text
-data/
-├── gvhmr/results/<clip>/ # reusable motion-recovery result
-└── fdanyone/<clip>/
- ├── metadata.json # run settings, timings, resources
- ├── cameras.json # the final N-camera rig
- ├── skeletons/00.mp4 ... <N-1>.mp4
- └── videos/
- ├── sparse/{00,04,09,12,14,19}.mp4 # default 24-view RCP proposals
- └── dense/00.mp4 ... <N-1>.mp4 # generated target views
+The output directory contains:
+
+```bash
+<clip>/ # input filename without its extension
+├── metadata.json # run settings, timings, resources
+├── cameras.json # intrinsics and poses for N target views
+├── gvhmr/ # reusable motion recovery
+│ ├── motion.json # source timeline and motion metadata
+│ └── motion.safetensors # motion tensors
+├── skeletons/00.mp4 ... <N-1>.mp4 # pose conditioning for each target view
+└── videos/
+ ├── sparse/{00,04,09,12,14,19}.mp4 # RCP videos
+ └── dense/00.mp4 ... <N-1>.mp4 # target videos
```
+Completed outputs are never overwritten. After a failed or interrupted run, rerun with the same `--output_dir` to reuse completed motion recovery and restart generation.
+
### Custom Data
Use an input video with:
diff --git a/docs/nerfstudio.md b/docs/nerfstudio.md
index 14037c7..9892a29 100644
--- a/docs/nerfstudio.md
+++ b/docs/nerfstudio.md
@@ -17,14 +17,15 @@ Run the exporter in the 4DAnyone inference environment:
```bash
conda activate 4danyone
python scripts/export_nerfstudio.py \
- --result_dir data/fdanyone/<clip> \
+ --data_dir data/fdanyone/pexels/2785536-uhd_2160_3840_25fps \
+ --output_dir data/ns_data/pexels/2785536-uhd_2160_3840_25fps/frame_000 \
--frame_index 0
```
-The exported dataset is written to:
+The exported Nerfstudio data is written to:
```bash
-data/nerfstudio/<clip>/frame_000/
+data/ns_data/pexels/<clip>/frame_000/
├── transforms.json
├── sparse_pcd.ply # visual-hull initialization
├── images/00.png ... <N-1>.png
@@ -37,7 +38,8 @@ Standard Splatfacto:
```bash
ns-train splatfacto \
- --data data/nerfstudio/<clip>/frame_000 \
+ --data data/ns_data/pexels/2785536-uhd_2160_3840_25fps/frame_000 \
+ --output-dir data/ns_outputs/pexels/2785536-uhd_2160_3840_25fps/frame_000 \
--pipeline.model.background-color random
```
@@ -45,7 +47,8 @@ Splatfacto with perceptual loss:
```bash
python scripts/train_nerfstudio.py splatfacto-perceptual \
- --data data/nerfstudio/<clip>/frame_000 \
+ --data data/ns_data/pexels/2785536-uhd_2160_3840_25fps/frame_000 \
+ --output-dir data/ns_outputs/pexels/2785536-uhd_2160_3840_25fps/frame_000 \
--pipeline.model.background-color random \
--pipeline.model.perceptual-loss-weight 0.4 \
--pipeline.model.perceptual-compute-dtype bfloat16
@@ -55,11 +58,11 @@ If the GPU supports `bfloat16`, we recommend enabling it to accelerate training.
## View
-Use the wrapper so the custom method is registered before Nerfstudio loads its saved config:
+Launch the viewer with the config path printed by training:
```bash
python scripts/view_nerfstudio.py \
- --load-config outputs/<clip>/splatfacto-perceptual/<timestamp>/config.yml \
+ --load-config <training-output>/config.yml \
--viewer.websocket-port 7007
```
diff --git a/docs/output.md b/docs/output.md
deleted file mode 100644
index 4d6d0fc..0000000
--- a/docs/output.md
+++ /dev/null
@@ -1,34 +0,0 @@
-# Output
-
-With the default `--data_dir data`, each inference run writes:
-
-```text
-data/
-├── gvhmr/results/<clip>/
-│ ├── motion.json
-│ └── motion.safetensors
-└── fdanyone/<clip>/
- ├── metadata.json
- ├── cameras.json
- ├── skeletons/00.mp4 ... <N-1>.mp4
- └── videos/
- ├── sparse/*.mp4 # RCP proposal views, when used
- └── dense/00.mp4 ... <N-1>.mp4 # final target views
-```
-
-`<clip>` is the input filename without its extension, and `N` is the number of target views.
-
-## Files
-
-- `videos/dense/` contains the generated target-view videos.
-- `videos/sparse/` contains intermediate proposal views for larger camera layouts.
-- `skeletons/` contains pose-conditioning videos aligned with the target views.
-- `cameras.json` contains camera intrinsics and poses for every target view.
-- `metadata.json` records the input, denoising profile, model identities, timings, and worker metrics when target denoising uses multiple GPUs.
-- `gvhmr/results/` caches the recovered input motion for reuse.
-
-All output videos contain 121 frames at 704×1280, use the selected output FPS, and contain no audio.
-
-## Nerfstudio export
-
-See the [3DGS reconstruction guide](nerfstudio.md) for details.
diff --git a/fdanyone/config.py b/fdanyone/config.py
index d203898..297543a 100644
--- a/fdanyone/config.py
+++ b/fdanyone/config.py
@@ -85,7 +85,7 @@ class ForegroundConfig:
@dataclass(frozen=True)
class FramingConfig:
- """Sequence-level camera solve matching the current GVHMR demo."""
+ """Clip-wide camera solve matching the current GVHMR demo."""
reference_radius: float = 3.0
reference_target_height: float = 1.0
diff --git a/fdanyone/geometry/framing.py b/fdanyone/geometry/framing.py
index fad54a1..5526fcd 100644
--- a/fdanyone/geometry/framing.py
+++ b/fdanyone/geometry/framing.py
@@ -64,7 +64,7 @@ class FocalSolve:
@dataclass(frozen=True)
-class SequenceFraming:
+class ClipFraming:
radius: float
target_height: float
focal_normalized: float
@@ -509,19 +509,19 @@ def _cutoff_ratio(points: np.ndarray, cameras: Sequence[Camera], focal: float, p
return float(np.percentile(positions[np.isfinite(positions)], percentile))
-def solve_sequence_framing(
+def solve_clip_framing(
keypoints: np.ndarray,
names: Sequence[str],
profile: InputFraming,
camera_factory: Callable[[float, float], Sequence[Camera]],
aspect_ratio: float,
spec: FramingConfig = FRAMING,
-) -> SequenceFraming:
+) -> ClipFraming:
radius_result = solve_radius(keypoints, names, camera_factory, aspect_ratio, spec)
applied = profile.confidence >= spec.input_min_confidence
thresholds = adaptive_thresholds(profile) if applied else None
if not applied or thresholds is None or thresholds.closeup_strength <= 0:
- return SequenceFraming(
+ return ClipFraming(
radius_result.radius,
spec.reference_target_height,
spec.reference_focal_normalized,
@@ -565,7 +565,7 @@ def solve_sequence_framing(
else:
upper = midpoint
- return SequenceFraming(
+ return ClipFraming(
radius_result.radius,
target,
value[0].focal_normalized,
diff --git a/fdanyone/io.py b/fdanyone/io.py
index 556c591..463b640 100644
--- a/fdanyone/io.py
+++ b/fdanyone/io.py
@@ -3,13 +3,14 @@
from __future__ import annotations
import errno
+import fcntl
import hashlib
import json
import os
import shutil
import time
import uuid
-from contextlib import AbstractContextManager
+from contextlib import AbstractContextManager, contextmanager
from pathlib import Path
from fdanyone.errors import FourDAnyoneError
@@ -17,6 +18,34 @@ from fdanyone.errors import FourDAnyoneError
_RETRYABLE_TREE_ERRORS = {errno.EBUSY, errno.ENOTEMPTY, errno.ESTALE}
+def resolve_output_path(path: str | Path) -> Path:
+ """Normalize an output path without following a possibly dangling leaf symlink."""
+
+ expanded = Path(path).expanduser()
+ return expanded.parent.resolve() / expanded.name
+
+
+@contextmanager
+def lock_output(path: Path):
+ """Reserve an output through a stable, writable sidecar file.
+
+ Keep the empty lock file outside the output directory and never unlink it: replacing
+ its inode could let concurrent writers acquire different locks. The OS
+ releases the lock on process exit, including SIGKILL.
+ """
+
+ lock_path = path.with_name(f".{path.name}.lock")
+ descriptor = os.open(lock_path, os.O_CREAT | os.O_RDWR | os.O_NOFOLLOW, 0o666)
+ try:
+ try:
+ fcntl.flock(descriptor, fcntl.LOCK_EX | fcntl.LOCK_NB)
+ except BlockingIOError as exc:
+ raise FourDAnyoneError(f"Another inference run is using {path}.") from exc
+ yield
+ finally:
+ os.close(descriptor)
+
+
def sha256_file(path: str | Path, *, chunk_size: int = 8 * 1024 * 1024) -> str:
"""Hash a file without materializing it in memory."""
@@ -66,11 +95,7 @@ class AtomicResultDirectory(AbstractContextManager[Path]):
"""Build beside the destination and rename only after all validation passes."""
def __init__(self, destination: str | Path):
- expanded = Path(destination).expanduser()
- # Resolve the parent for a stable absolute location, but preserve the
- # leaf itself so a dangling output symlink cannot be followed and
- # mistaken for a nonexistent destination.
- self.destination = expanded.parent.resolve() / expanded.name
+ self.destination = resolve_output_path(destination)
self.working = self.destination.with_name(f".{self.destination.name}.work-{uuid.uuid4().hex[:10]}")
self._committed = False
diff --git a/fdanyone/nerfstudio/exporter.py b/fdanyone/nerfstudio/exporter.py
index 68ab974..d23d8f5 100644
--- a/fdanyone/nerfstudio/exporter.py
+++ b/fdanyone/nerfstudio/exporter.py
@@ -23,6 +23,7 @@ from fdanyone.nerfstudio.visual_hull import (
build_sparse_point_cloud,
write_sparse_point_cloud,
)
+from fdanyone.output_directory import read_output_metadata
# Nerfstudio casts mask pixels directly to bool, so soft BiRefNet predictions
# must cross a real decision boundary before serialization.
@@ -162,37 +163,38 @@ def _transforms(cameras: list[dict]) -> dict:
def export_nerfstudio(
- result_dir: str,
- frame_index: int = 0,
+ data_dir: str,
output_dir: str | None = None,
+ frame_index: int = 0,
model_dir: str = "models",
device: str = "cuda:0",
) -> dict:
"""Export one masked multi-view timestamp and visual hull for Nerfstudio.
Args:
- result_dir: A completed data/fdanyone/<clip> result.
+ data_dir: Completed 4DAnyone output directory for one clip.
+ output_dir: Dataset directory. Defaults to data/ns_data/<clip>/frame_NNN.
frame_index: Synchronized frame index from 0 through 120.
- output_dir: Destination; defaults to data/nerfstudio/<clip>/frame_NNN.
model_dir: Model root containing (or receiving) the pinned BiRefNet files.
device: CUDA device used for foreground segmentation.
"""
- result = Path(result_dir).expanduser().resolve()
+ result = Path(data_dir).expanduser().resolve()
if not result.is_dir():
raise FourDAnyoneError(f"4DAnyone result does not exist: {result}")
if not 0 <= frame_index < INFERENCE.num_frames:
raise FourDAnyoneError(f"frame_index must be in [0, {INFERENCE.num_frames - 1}], got {frame_index}.")
+ read_output_metadata(result)
cameras = _camera_records(_read_cameras(result))
transforms = _transforms(cameras)
videos = _dense_video_paths(result, cameras)
if output_dir is None:
- data_root = result.parent.parent if result.parent.name == "fdanyone" else result.parent
- destination = data_root / "nerfstudio" / result.name / f"frame_{frame_index:03d}"
+ destination = Path("data/ns_data") / result.name / f"frame_{frame_index:03d}"
else:
- destination = Path(output_dir).expanduser().resolve()
+ destination = Path(output_dir)
atomic = AtomicResultDirectory(destination)
+ destination = atomic.destination
if os.path.lexists(atomic.destination):
raise FourDAnyoneError(f"Nerfstudio dataset already exists: {atomic.destination}")
@@ -207,7 +209,6 @@ def export_nerfstudio(
masks = predict_foreground_masks(images, foreground_model, device)
binary_masks = _write_masks(masks, images, work / "masks")
_write_images(images, binary_masks, work / "images")
- del foreground_model
points, colors = build_sparse_point_cloud(images, binary_masks, cameras, device)
write_sparse_point_cloud(work / NERFSTUDIO_POINT_CLOUD, points, colors)
write_json(work / "transforms.json", transforms, sort_keys=False)
diff --git a/fdanyone/output_directory.py b/fdanyone/output_directory.py
new file mode 100644
index 0000000..0693191
--- /dev/null
+++ b/fdanyone/output_directory.py
@@ -0,0 +1,129 @@
+"""Output staging, reusable motion, and a single forward-only completion protocol."""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+from contextlib import contextmanager
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+from fdanyone.errors import ConfigurationError, FourDAnyoneError
+from fdanyone.io import lock_output, remove_tree, resolve_output_path
+
+if TYPE_CHECKING:
+ from fdanyone.motion.result import MotionResult
+
+LOGGER = logging.getLogger("fdanyone")
+_GENERATED = ("cameras.json", "skeletons", "videos", "metadata.json")
+_DIRECTORIES = {"gvhmr", "skeletons", "videos", ".inference"}
+
+
+def read_output_metadata(directory: str | Path) -> dict:
+ """Require the final commit marker before any reader consumes the output."""
+
+ path = Path(directory) / "metadata.json"
+ try:
+ metadata = json.loads(path.read_text())
+ except (OSError, ValueError) as exc:
+ raise FourDAnyoneError(f"Output is incomplete or has invalid metadata: {path}.") from exc
+ if not isinstance(metadata, dict):
+ raise FourDAnyoneError(f"Output metadata must contain a JSON object: {path}.")
+ return metadata
+
+
+class OutputDirectory:
+ """Stage output under an OS lock, retaining motion and unfinished work on failure.
+
+ The private workspace records ownership and the start of publication, not
+ process liveness. A retry holding the directory lock can discard that run's
+ partial generated artifacts. Publication only moves forward, with metadata
+ last. Nothing removes published artifacts after that commit point.
+ """
+
+ def __init__(self, destination: str | Path):
+ self.destination = resolve_output_path(destination)
+ self.motion_dir = self.destination / "gvhmr"
+ self.working = self.destination / ".inference"
+ self._owner = self.working / ".4danyone"
+ self._publishing = self.working / ".publishing"
+
+ def validate_available(self) -> None:
+ """Protect completed results and unrelated files before preparing a retry."""
+
+ if not os.path.lexists(self.destination):
+ return
+ if self.destination.is_symlink() or not self.destination.is_dir():
+ raise ConfigurationError(f"Output path already exists: {self.destination}. Choose a new --output_dir.")
+ if os.path.lexists(self.destination / "metadata.json"):
+ raise ConfigurationError(f"4DAnyone output already exists: {self.destination}. Choose a new --output_dir.")
+ allowed = {"gvhmr", ".inference"}
+ if self._owner.is_file() and self._publishing.is_file():
+ allowed.update(_GENERATED[:-1])
+ self._check_entries(self.destination, allowed)
+ if self.working.exists():
+ self._check_entries(self.working, {*_GENERATED, "gvhmr", ".4danyone", ".publishing"})
+ if any(self.working.iterdir()) and not self._owner.is_file():
+ raise ConfigurationError(f"Unrecognized staging directory: {self.working}. Choose a new --output_dir.")
+
+ @staticmethod
+ def _check_entries(directory: Path, allowed: set[str]) -> None:
+ for path in directory.iterdir():
+ expected_type = path.is_dir() if path.name in _DIRECTORIES else path.is_file()
+ if path.name not in allowed or path.is_symlink() or not expected_type:
+ raise ConfigurationError(f"Unrecognized output entry: {path}. Choose a new --output_dir.")
+
+ def _prepare(self) -> None:
+ # Only a marked, interrupted publication owns artifacts outside staging.
+ if self._publishing.is_file():
+ for name in _GENERATED[:-1]:
+ path = self.destination / name
+ if name in _DIRECTORIES:
+ remove_tree(path)
+ else:
+ path.unlink(missing_ok=True)
+ self.working.mkdir(exist_ok=True)
+ self._owner.touch(exist_ok=True)
+ # Ownership survives an interruption in cleanup itself.
+ for path in self.working.iterdir():
+ if path == self._owner:
+ continue
+ if path.is_dir():
+ remove_tree(path)
+ else:
+ path.unlink()
+
+ def save_motion(self, motion: MotionResult) -> None:
+ """Publish the complete motion pair once, independently of generation."""
+
+ if os.path.lexists(self.motion_dir):
+ raise ConfigurationError(f"Motion output already exists: {self.motion_dir}.")
+ staged = motion.save(self.working / "gvhmr")
+ staged.rename(self.motion_dir)
+
+ def _publish(self) -> None:
+ self._check_entries(self.working, {*_GENERATED, ".4danyone"})
+ if {path.name for path in self.working.iterdir()} != {*_GENERATED, ".4danyone"}:
+ raise ConfigurationError("Output is incomplete. Refusing to publish partial results.")
+ read_output_metadata(self.working)
+ for name in _GENERATED:
+ if os.path.lexists(self.destination / name):
+ raise ConfigurationError(f"Output appeared during inference: {self.destination / name}.")
+ self._publishing.touch(exist_ok=False)
+ for name in _GENERATED:
+ (self.working / name).rename(self.destination / name)
+
+ @contextmanager
+ def stage(self):
+ self.validate_available()
+ self.destination.mkdir(parents=True, exist_ok=True)
+ with lock_output(self.destination):
+ self.validate_available()
+ self._prepare()
+ yield self.working
+ self._publish()
+ try:
+ remove_tree(self.working)
+ except OSError as exc:
+ LOGGER.warning("Output is complete, but staging cleanup failed at %s: %s", self.working, exc)
diff --git a/fdanyone/output.py b/fdanyone/output_writer.py
similarity index 93%
rename from fdanyone/output.py
rename to fdanyone/output_writer.py
index c31e614..7b39f68 100644
--- a/fdanyone/output.py
+++ b/fdanyone/output_writer.py
@@ -1,4 +1,4 @@
-"""Publish generated videos and their camera metadata."""
+"""Write generated videos, cameras, and run metadata into output staging."""
from __future__ import annotations
@@ -14,7 +14,6 @@ from typing import TYPE_CHECKING
from fdanyone.config import INFERENCE
from fdanyone.device import CUDA_ALLOCATOR_CONF
from fdanyone.errors import FourDAnyoneError
-from fdanyone.io import write_json
if TYPE_CHECKING:
from fdanyone.model.inference import GeneratedViews
@@ -100,7 +99,7 @@ def _target_cameras(payload: object, expected_count: int) -> list[dict]:
return cameras
-def export_result(
+def write_output(
*,
clip: CanonicalClip,
conditioning: Conditioning,
@@ -110,7 +109,7 @@ def export_result(
model_identity: dict,
pipeline_started: float,
) -> dict:
- """Publish proposal, target, skeleton, camera, and metadata artifacts."""
+ """Write proposal, target, skeleton, camera, and metadata artifacts into staging."""
root = Path(destination).expanduser().resolve()
attention_backend = generated.attention_backend
@@ -205,8 +204,13 @@ def export_result(
},
"runtime": _runtime_metadata(generated.device),
}
- write_json(root / "cameras.json", _camera_rig_payload(camera_payload, camera_records))
- write_json(root / "metadata.json", metadata)
+ # Staging is private. Output publication, not each JSON write, owns the
+ # commit boundary and interruption recovery.
+ for name, payload in (
+ ("cameras.json", _camera_rig_payload(camera_payload, camera_records)),
+ ("metadata.json", metadata),
+ ):
+ (root / name).write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n")
return {
"attention_backend": attention_backend,
"num_rcp_videos": len(output_sparse),
diff --git a/fdanyone/pipeline.py b/fdanyone/pipeline.py
index ec52e8f..f304701 100644
--- a/fdanyone/pipeline.py
+++ b/fdanyone/pipeline.py
@@ -27,11 +27,13 @@ from fdanyone.config import BASE24, INFERENCE, RANK64_DELTA4
from fdanyone.device import CUDA_ALLOCATOR_CONF, select_cuda_devices
from fdanyone.download import ensure_example_video, ensure_models, ensure_smplx
from fdanyone.errors import ConfigurationError
-from fdanyone.io import AtomicResultDirectory, remove_tree, write_json
+from fdanyone.io import remove_tree, resolve_output_path, write_json
from fdanyone.motion.gvhmr import validate_gvhmr
from fdanyone.motion.result import MotionResult
+from fdanyone.output_directory import OutputDirectory
from fdanyone.video import (
decode_canonical_clip,
+ validate_clip_options,
validate_required_video_codecs,
verify_lossless_video,
write_gvhmr_video,
@@ -41,14 +43,11 @@ from fdanyone.views import ViewPlan, resolve_view_plan
LOGGER = logging.getLogger("fdanyone")
-def _data_paths(data_dir: str, video_path: str) -> tuple[Path, Path, Path]:
- data_root = Path(data_dir).expanduser().resolve()
- run_name = Path(video_path).stem
- return (
- data_root,
- data_root / "gvhmr" / "results" / run_name,
- data_root / "fdanyone" / run_name,
- )
+def _resolve_output_dir(output_dir: str | None, video_path: str) -> Path:
+ """Resolve the output directory without following its final path component."""
+
+ target = Path("data/fdanyone") / Path(video_path).stem if output_dir is None else Path(output_dir)
+ return resolve_output_path(target)
def _discard_scratch(path: Path) -> None:
@@ -128,16 +127,16 @@ def _run_motion(
def _build_conditioning(
*,
+ working_video: Path,
+ clip_metadata: Path,
+ motion_result_dir: Path,
+ view_plan: ViewPlan,
+ output_dir: Path,
regressor: Path,
foreground_model: Path,
gvhmr_root: Path,
- output_dir: Path,
device: str,
worker_python: str,
- working_video: Path,
- clip_metadata: Path,
- motion_result_dir: Path,
- view_plan: ViewPlan,
):
from fdanyone.skeleton.pipeline import Conditioning
@@ -175,10 +174,9 @@ def _build_conditioning(
def run_pipeline(
*,
video_path: str,
- data_dir: str,
+ output_dir: str | None,
model_dir: str,
checkpoint_path: str | None,
- enable_turbo: bool,
mhr70_regressor_path: str | None,
gvhmr_root: str,
gpu_ids: list[int] | None,
@@ -193,8 +191,9 @@ def run_pipeline(
views_per_group: int | str,
enable_rcp: bool,
enable_tcr: bool,
+ enable_turbo: bool,
) -> dict:
- """Execute inference and publish reusable GVHMR plus 4DAnyone results."""
+ """Execute inference for one clip, retaining reusable motion."""
pipeline_started = time.monotonic()
logging.basicConfig(level=logging.INFO, format="%(asctime)s | %(levelname)s | %(message)s")
@@ -212,15 +211,16 @@ def run_pipeline(
enable_rcp=enable_rcp,
enable_tcr=enable_tcr,
)
- data_root, motion_dir, result_dir = _data_paths(data_dir, video_path)
- run_name = Path(video_path).stem
- atomic = AtomicResultDirectory(result_dir)
- # Fail before asset resolution or video decode; the context manager
- # checks again later in case another process creates the path.
- if os.path.lexists(atomic.destination):
- raise ConfigurationError(
- f"4DAnyone result already exists: {atomic.destination}. Choose a new --data_dir or input filename."
- )
+ canonical_fps = validate_clip_options(
+ start_time=start_time,
+ fps=None if str(target_fps).lower() == "auto" else target_fps,
+ )
+ checkpoint = resolve_checkpoint(checkpoint_path, model_dir=model_dir) if checkpoint_path is not None else None
+ destination = _resolve_output_dir(output_dir, video_path)
+ clip_name = Path(video_path).stem
+ output = OutputDirectory(destination)
+ # Check before downloads and decoding, then again when locking the output.
+ output.validate_available()
validate_required_video_codecs()
devices = select_cuda_devices(gpu_ids)
device = devices[0]
@@ -242,35 +242,29 @@ def run_pipeline(
gvhmr_root, gvhmr_revision = validate_gvhmr(gvhmr_root)
worker_python = os.path.abspath(sys.executable)
+ if checkpoint is None:
+ checkpoint = resolve_checkpoint(model_dir=model_dir)
+ base_assets = resolve_base_assets(model_dir)
regressor = resolve_regressor(mhr70_regressor_path, model_dir=model_dir)
foreground_model = resolve_foreground_model(model_dir)
- canonical_fps = None if str(target_fps).lower() == "auto" else target_fps
clip = decode_canonical_clip(
video_path,
num_frames=INFERENCE.num_frames,
start_time=start_time,
fps=canonical_fps,
)
- data_root.mkdir(parents=True, exist_ok=True)
- scratch = Path(tempfile.mkdtemp(prefix=f".{run_name}.scratch-", dir=data_root))
+ destination.parent.mkdir(parents=True, exist_ok=True)
+ scratch = Path(tempfile.mkdtemp(prefix=f".{clip_name}.scratch-", dir=destination.parent))
try:
clip_metadata = scratch / "canonical_clip.json"
clip.write_metadata(clip_metadata)
working_video = write_gvhmr_video(clip, scratch / "canonical_clip.mp4")
- if os.path.lexists(motion_dir):
- if motion_dir.is_symlink() or not motion_dir.is_dir():
- raise ConfigurationError(f"GVHMR result path is not a regular directory: {motion_dir}")
- motion = MotionResult.load(motion_dir)
- if motion.gvhmr_revision != gvhmr_revision:
- raise ConfigurationError(
- f"Existing GVHMR result at {motion_dir} was produced by "
- f"GVHMR@{motion.gvhmr_revision}, not GVHMR@{gvhmr_revision}."
- )
- motion.validate_against_clip(clip)
- LOGGER.info("Reusing validated GVHMR result at %s", motion_dir)
- else:
- with AtomicResultDirectory(motion_dir) as motion_work:
+ with output.stage() as work:
+ if output.motion_dir.exists():
+ LOGGER.info("Reusing GVHMR motion from %s", output.motion_dir)
+ motion = MotionResult.load(output.motion_dir)
+ else:
motion = _run_motion(
working_video=working_video,
output_dir=scratch / "gvhmr",
@@ -279,41 +273,44 @@ def run_pipeline(
worker_python=worker_python,
clip_metadata=clip_metadata,
)
- motion.validate_against_clip(clip)
- motion.save(motion_work)
+ if motion.gvhmr_revision != gvhmr_revision:
+ raise ConfigurationError(
+ f"GVHMR motion has revision {motion.gvhmr_revision}, expected {gvhmr_revision}. "
+ "Choose a new --output_dir to recover motion with the current GVHMR version."
+ )
+ motion.validate_against_clip(clip)
+ if not output.motion_dir.exists():
+ output.save_motion(motion)
- checkpoint = resolve_checkpoint(checkpoint_path, model_dir=model_dir)
- base_assets = resolve_base_assets(model_dir)
- # Record the published identity only for the published checkpoint; an
- # explicit override must not claim the frozen Hugging Face coordinates.
- if checkpoint_path is None:
- model_identity = {"checkpoint": CHECKPOINT, "repo_id": HF_REPO_ID, "revision": HF_REVISION}
- else:
- model_identity = {"checkpoint": checkpoint.name, "source": "local_override"}
- if turbo_lora is not None:
- model_identity["turbo_lora"] = {
- "name": TURBO_LORA_NAME,
- "file": TURBO_LORA,
- "sha256": TURBO_LORA_SHA256,
- }
+ # Record the published identity only for the published checkpoint; an
+ # explicit override must not claim the frozen Hugging Face coordinates.
+ if checkpoint_path is None:
+ model_identity = {"checkpoint": CHECKPOINT, "repo_id": HF_REPO_ID, "revision": HF_REVISION}
+ else:
+ model_identity = {"checkpoint": checkpoint.name, "source": "local_override"}
+ if turbo_lora is not None:
+ model_identity["turbo_lora"] = {
+ "name": TURBO_LORA_NAME,
+ "file": TURBO_LORA,
+ "sha256": TURBO_LORA_SHA256,
+ }
- with atomic as work:
# Heavy rendering and generation are imported only after the motion
# contract has been materialized, keeping CLI/help and CPU tests light.
from fdanyone.model.inference import generate_views
- from fdanyone.output import export_result
+ from fdanyone.output_writer import write_output
conditioning = _build_conditioning(
+ working_video=working_video,
+ clip_metadata=clip_metadata,
+ motion_result_dir=output.motion_dir,
+ view_plan=view_plan,
+ output_dir=scratch / "conditioning",
regressor=regressor,
foreground_model=foreground_model,
gvhmr_root=gvhmr_root,
- output_dir=scratch / "conditioning",
device=device,
worker_python=worker_python,
- working_video=working_video,
- clip_metadata=clip_metadata,
- motion_result_dir=motion_dir,
- view_plan=view_plan,
)
if conditioning.num_frames != len(clip.frames) or (
conditioning.fps_num,
@@ -337,7 +334,7 @@ def run_pipeline(
devices=devices,
seed=seed,
)
- summary = export_result(
+ summary = write_output(
clip=clip,
conditioning=conditioning,
generated=generated,
@@ -348,6 +345,5 @@ def run_pipeline(
)
finally:
_discard_scratch(scratch)
- summary["result_dir"] = str(result_dir)
- summary["motion_dir"] = str(motion_dir)
+ summary["output_dir"] = str(destination)
return summary
diff --git a/fdanyone/skeleton/pipeline.py b/fdanyone/skeleton/pipeline.py
index 9984180..211fb8d 100644
--- a/fdanyone/skeleton/pipeline.py
+++ b/fdanyone/skeleton/pipeline.py
@@ -25,7 +25,7 @@ from fdanyone.geometry.cameras import (
reference_intrinsics,
)
from fdanyone.geometry.crop import Crop, center_crop, crop_from_bounds, mask_bounds, transform_intrinsics
-from fdanyone.geometry.framing import analyze_input_framing, solve_sequence_framing
+from fdanyone.geometry.framing import analyze_input_framing, solve_clip_framing
from fdanyone.io import write_json
from fdanyone.motion.gvhmr import gvhmr_imports, validate_gvhmr
from fdanyone.motion.result import MotionResult
@@ -157,8 +157,7 @@ def _video_tensor(path: Path, num_frames: int, *, crop: Crop | None = None):
return torch.stack(output_frames, dim=1).unsqueeze(0).contiguous()
-def _safe_regressor_metadata(raw_metadata: object, support_shape: tuple[int, ...]) -> dict[str, int | str]:
- del raw_metadata
+def _safe_regressor_metadata(support_shape: tuple[int, ...]) -> dict[str, int | str]:
return {
"format": "sparse_vertex_regressor",
"num_keypoints": int(support_shape[0]),
@@ -177,7 +176,7 @@ def _load_regressor(path: Path, device):
raise AssetError(f"Unexpected MHR70 regressor shapes: support={support.shape}, weights={weights.shape}.")
if names != KEYPOINT_NAMES:
raise AssetError("MHR70 regressor keypoint order does not match the frozen Goliath70 schema.")
- return support, weights, _safe_regressor_metadata(data.get("metadata"), tuple(support.shape))
+ return support, weights, _safe_regressor_metadata(tuple(support.shape))
@contextmanager
@@ -379,7 +378,7 @@ def build_skeleton_conditioning(
)
projection_height, projection_width = _projection_shape(clip.height, clip.width)
- framing = solve_sequence_framing(
+ framing = solve_clip_framing(
geometry.keypoints_world,
KEYPOINT_NAMES,
input_framing,
diff --git a/fdanyone/video.py b/fdanyone/video.py
index 6af9bcc..99ae159 100644
--- a/fdanyone/video.py
+++ b/fdanyone/video.py
@@ -229,6 +229,26 @@ def _decode_frames(
)
+def validate_clip_options(
+ *,
+ start_time: float,
+ fps: str | int | float | Fraction | None,
+) -> Fraction | None:
+ """Validate clip options without video I/O and normalize the requested FPS."""
+
+ if not math.isfinite(start_time) or start_time < 0:
+ raise VideoContractError(f"start_time must be non-negative, got {start_time}.")
+ if fps is None:
+ return None
+ try:
+ rate = Fraction(str(fps))
+ except (ValueError, ZeroDivisionError) as exc:
+ raise VideoContractError(f"Cannot parse target fps {fps!r} as a rational frame rate.") from exc
+ if rate <= 0:
+ raise VideoContractError(f"Target fps must be positive, got {rate}.")
+ return rate
+
+
def decode_canonical_clip(
video_path: str | Path,
*,
@@ -243,8 +263,7 @@ def decode_canonical_clip(
raise VideoContractError(f"Input video does not exist: {path}")
if num_frames <= 0:
raise VideoContractError(f"num_frames must be positive, got {num_frames}.")
- if not math.isfinite(start_time) or start_time < 0:
- raise VideoContractError(f"start_time must be non-negative, got {start_time}.")
+ fps = validate_clip_options(start_time=start_time, fps=fps)
source_stat = path.stat()
@@ -253,12 +272,7 @@ def decode_canonical_clip(
raise VideoContractError(f"Input has no video stream: {path}")
stream = container.streams.video[0]
input_rate = _stream_rate(stream)
- try:
- output_rate = choose_canonical_fps(input_rate) if fps is None else Fraction(str(fps))
- except (ValueError, ZeroDivisionError) as exc:
- raise VideoContractError(f"Cannot parse target fps {fps!r} as a rational frame rate.") from exc
- if output_rate <= 0:
- raise VideoContractError(f"Target fps must be positive, got {output_rate}.")
+ output_rate = choose_canonical_fps(input_rate) if fps is None else fps
metadata_rotation = _rotation_degrees(stream)
decoded = _decode_frames(container, stream, metadata_rotation)
try:
diff --git a/inference.py b/inference.py
index 2c3a7df..ee81ed8 100644
--- a/inference.py
+++ b/inference.py
@@ -11,6 +11,7 @@ from fdanyone.errors import FourDAnyoneError
def inference(
video_path: str,
+ output_dir: str | None = None,
views_per_layer: int = 24,
layer_pitches: list[int] = [15], # noqa: B006 - normalized without mutation
start_yaw: int = 0,
@@ -19,7 +20,6 @@ def inference(
enable_rcp: bool = True,
enable_tcr: bool = True,
enable_turbo: bool = True,
- data_dir: str = "data",
model_dir: str = "models",
checkpoint_path: str | None = None,
mhr70_regressor_path: str | None = None,
@@ -34,6 +34,7 @@ def inference(
Args:
video_path: Input video; it must contain at least 121 usable frames.
+ output_dir: Output directory for this clip. Defaults to data/fdanyone/<clip>.
views_per_layer: Number of evenly spaced yaw views at each pitch. It
must be divisible by 4 or 6.
layer_pitches: Camera pitch for each layer in degrees, for example
@@ -50,7 +51,6 @@ def inference(
enable_tcr: Shift view groups cyclically between denoising steps.
enable_turbo: Whether to use 4DAnyone-Turbo for accelerated denoising.
Disable it to use the base 4DAnyone model.
- data_dir: Root for reusable GVHMR motion and final 4DAnyone outputs.
model_dir: Model root; missing public checkpoints download here.
checkpoint_path: Local 4DAnyone checkpoint override.
mhr70_regressor_path: Local SMPL-X-to-MHR70 regressor override.
@@ -78,6 +78,7 @@ def inference(
return run_pipeline(
video_path=video_path,
+ output_dir=output_dir,
views_per_layer=views_per_layer,
layer_pitches=layer_pitches,
start_yaw=start_yaw,
@@ -86,7 +87,6 @@ def inference(
enable_rcp=enable_rcp,
enable_tcr=enable_tcr,
enable_turbo=enable_turbo,
- data_dir=data_dir,
model_dir=model_dir,
checkpoint_path=checkpoint_path,
mhr70_regressor_path=mhr70_regressor_path,