Skip to content

PRINTFILM: AI short-video marketing and AI short-drama creation platform

feat(short-video): 完善参考图生图与分阶段视频制作

yif60ae2253101
diff --git a/README.md b/README.md
index 064d305..3d942b6 100644
--- a/README.md
+++ b/README.md
@@ -7,7 +7,7 @@
> **把故事做成能播的片子** — AI 漫剧与 AI 短视频,从文案到成片。
-分步短视频创作:创作设置 → 文案与分镜 → 画面与配音 → 成片预览。默认 3D 动画画风、9:16 竖屏、约 60 秒,各阶段检查与编辑后再继续制作。
+分步短视频创作:创作设置 → 文案与分镜 → 参考图生图 → 图生视频 → 合成预览。新建默认 AI 视频、3D 动画画风、9:16 竖屏、约 60 秒,也可选择静图成片,各阶段检查与编辑后再继续制作。
**源码**:[github.com/yi1108/printfilm](https://github.com/yi1108/printfilm) · **版本** 0.2.0
@@ -50,8 +50,9 @@
- 精选四款非真人风格:3D 动画、二维动漫、绘本插画、国风水墨
- 输入想法生成文案,或粘贴已有文案;文案可编辑并保存草稿
-- 确认文案后生成分镜卡片,逐镜编辑旁白与画面,再生成图片和外部 TTS 配音
-- 图片和配音准备好后手动合成;FFmpeg 添加图片运镜和字幕,支持预览、下载
+- 确认文案后生成分镜,逐镜编辑旁白与画面;每镜使用固定画风参考图,并参考前镜图片延续角色与视觉风格
+- AI 视频模式把每镜图片生成为带声音的镜头视频,支持单镜预览和重生;选择模型与清晰度后分阶段生成,再手动合成成片
+- 静图模式生成图片和外部 TTS 配音,FFmpeg 添加图片运镜和字幕;两种模式都支持预览、下载
- 制作中可停止,失败时继续补齐缺失素材;离开页面后可回到项目恢复进度
- 旧对话项目在任务停止并结算后移交分步编辑,各历史版本保留原素材
@@ -68,8 +69,9 @@
```
选择画风、输入想法或已有文案
→ 编辑与保存文案 → 确认后生成分镜
- → 检查分镜卡片 → 生成图片与外部 TTS 配音
- → 检查画面、试听配音 → 合成运镜、字幕与旁白
+ → 检查分镜卡片 → 使用画风参考图生成图片
+ → AI 视频:图片生成镜头视频与声音 → 检查并合成
+ 静图成片:生成外部 TTS 配音 → 合成运镜、字幕与旁白
→ 预览与下载成片
```
diff --git a/README_EN.md b/README_EN.md
index af1b8ea..c7ee35b 100644
--- a/README_EN.md
+++ b/README_EN.md
@@ -7,7 +7,7 @@
> **Turn stories into videos you can ship** — AI short drama and AI short video, from script to final cut.
-Step-by-step short-video creation: settings → copy and storyboard → images and narration → final preview. Defaults: 3D animation, 9:16 portrait and approximately 60 seconds. Review and edit each stage before continuing.
+Step-by-step short-video creation: settings → copy and storyboard → reference-guided images → image-to-video → final preview. New projects default to AI video, 3D animation, 9:16 portrait and approximately 60 seconds; still-image production is also available. Review and edit each stage before continuing.
**Source:** [github.com/yi1108/printfilm](https://github.com/yi1108/printfilm) · **Version** 0.2.0
@@ -50,8 +50,9 @@ Manage short videos as project cards at `/history`, then start a new project at
- Four curated styles: 3D animation, 2D anime, picture-book illustration, and ink painting
- Generate copy from an idea or paste an existing script; edit and save it as a draft
-- Confirm the copy to generate editable storyboard cards, then generate images and external TTS narration
-- Review images and audition narration before composing camera motion, captions and audio; preview and download the result
+- Confirm the copy to generate editable storyboards; use a fixed style image and previous-shot references to guide each image
+- AI video mode generates clips and sound from each shot image, with selectable models, resolution, individual previews and retries before final composition
+- Still-image mode retains external TTS, camera motion and captions; both modes support preview and download
- Stop production or resume missing media after a failure; saved projects and tasks recover after leaving the page
- Existing conversations transfer to the step editor after their producers stop and settle; historical versions retain their media
@@ -68,8 +69,9 @@ Users, orders, templates, task center, and model routing.
```
Choose a style and enter an idea or existing copy
→ Edit and save copy → Confirm to generate storyboard cards
- → Review storyboard → Generate images and external TTS
- → Review images and audition narration → Compose motion and captions
+ → Review storyboard → Generate images using style references
+ → AI video: Generate clips and sound from images → Review and compose
+ Still images: Generate external TTS → Compose motion and captions
→ Preview and download the final video
```
diff --git a/backend/app/api/projects.py b/backend/app/api/projects.py
index e70221d..6ac9cc2 100644
--- a/backend/app/api/projects.py
+++ b/backend/app/api/projects.py
@@ -53,10 +53,10 @@ from app.services.kepu_projects import list_project_page, project_runtime_view
router = APIRouter(tags=["projects"])
-def _demote_after_edit(project: Project) -> None:
+def _demote_after_edit(project: Project, *, recheck_stage: bool = False) -> None:
"""改镜头/配乐后作废成片,并按素材把终态打回 script/assets/videos/compose。"""
project.final_video_url = None
- if project.status not in {
+ if not recheck_stage and project.status not in {
ProjectStatus.DONE,
ProjectStatus.AUDITING,
ProjectStatus.COMPOSING,
@@ -470,7 +470,7 @@ async def get_project(
"/projects/{project_id}",
response_model=ProjectOut,
summary="修改科普短片项目",
- description="需要登录,按请求中提供的字段更新本人的项目。生成期间及对话创作管理的项目不允许在此修改;更改配乐或字幕预设会使现有成片失效。",
+ description="需要登录,按请求中提供的字段更新本人的项目。生成期间及对话创作管理的项目不允许在此修改;更改制作模式、配乐或字幕预设会使现有成片失效。制作模式变更后沿用已有素材,按新模式回到待生成或待合成阶段。",
response_description="更新后的项目详情",
)
async def update_project(
@@ -526,10 +526,12 @@ async def update_project(
data["bgm_lock"] = str(data["bgm_lock"] or "").strip()
if "subtitle_preset" in data:
data["subtitle_preset"] = str(data["subtitle_preset"] or "").strip()
- if "bgm_lock" in data or "subtitle_preset" in data:
- _demote_after_edit(project)
+ # 按新模式判定下一阶段;保留图片/视频/配音,只作废原模式的成片。
+ mode_changed = "pipeline_mode" in data and data["pipeline_mode"] != (project.pipeline_mode or "full")
for k, v in data.items():
setattr(project, k, v)
+ if mode_changed or "bgm_lock" in data or "subtitle_preset" in data:
+ _demote_after_edit(project, recheck_stage=mode_changed)
await db.commit()
return await _get_owned_project(db, project_id, user)
diff --git a/backend/app/services/ark.py b/backend/app/services/ark.py
index 3b5eecf..2101d32 100644
--- a/backend/app/services/ark.py
+++ b/backend/app/services/ark.py
@@ -2149,11 +2149,13 @@ class ArkGateway:
# Seedance 2.0: 需要公网 https;本地 /static 先同步上 OSS
local = storage.local_path_from_url(raw)
if local and local.exists():
- public = storage.republish_url(raw, sync=True)
+ from app.services.kepu_image_references import public_image_reference
+
+ public = await asyncio.to_thread(public_image_reference, raw)
if public and str(public).startswith("https://"):
return str(public)
raise RuntimeError(
- "Seedance 需要公网可访问的图片 URL(请启用 OSS 并确保参考图已上传),"
+ "Seedance 需要公网可访问的图片 URL(请配置 HTTPS 公网访问地址或启用 OSS 并上传参考图),"
"本地 /static 图无法被方舟拉取"
)
if raw.startswith("data:"):
diff --git a/backend/app/services/kepu_continuity.py b/backend/app/services/kepu_continuity.py
index 88eb0c1..71b8d5c 100644
--- a/backend/app/services/kepu_continuity.py
+++ b/backend/app/services/kepu_continuity.py
@@ -41,11 +41,9 @@ def _usable_seedream_url(raw: str | None, *, allow_republish: bool = False) -> s
if is_tokenfree_image_url(url):
return None
if allow_republish and not (url.startswith("http://") or url.startswith("https://")):
- from app.services import storage
+ from app.services.kepu_image_references import public_image_reference
- url = str(storage.republish_url(url, sync=True) or "").strip()
- if is_tokenfree_image_url(url):
- return None
+ return public_image_reference(url)
ok = seedream_ref_urls(url)
return ok[0] if ok else None
@@ -75,6 +73,14 @@ def _any_shot_image(shot: Any) -> str | None:
return None
+def video_image_ref(shot: Any) -> str:
+ """本镜图生视频首图优先公网/本地落盘图,跳过需要上游鉴权的产物 URL。"""
+ for raw in (getattr(shot, "image_url", None), getattr(shot, "image_ark_url", None)):
+ if url := _usable_seedream_url(raw):
+ return url
+ return _any_shot_image(shot) or ""
+
+
def shot_last_frame_ref(shot: Any | None) -> str | None:
"""上一镜尾帧;TokenFree 任务 URL 跳过,避免下游无鉴权拉不到。"""
if shot is None:
diff --git a/backend/app/services/kepu_image_references.py b/backend/app/services/kepu_image_references.py
new file mode 100644
index 0000000..f0eca3b
--- /dev/null
+++ b/backend/app/services/kepu_image_references.py
@@ -0,0 +1,89 @@
+"""短视频画风与主体参考:每镜固定画风图,云端只接收可访问图片。"""
+from __future__ import annotations
+
+from typing import Any
+from ipaddress import ip_address
+from urllib.parse import urlsplit
+
+from app.config import get_settings
+from app.services import storage
+from app.services.style_lock import seedream_ref_urls
+from app.services.tokenfree_image import is_tokenfree_image_url
+
+
+def _public_reference(url: str) -> str | None:
+ """过滤私网、回环与需要 TokenFree 鉴权的图片地址。"""
+ if is_tokenfree_image_url(url) or not seedream_ref_urls(url, limit=1):
+ return None
+ host = urlsplit(url).hostname or ""
+ try:
+ if not ip_address(host).is_global:
+ return None
+ except ValueError:
+ if host.endswith(".localhost"):
+ return None
+ return url
+
+
+def public_image_reference(raw: str | None) -> str | None:
+ """本地图片同步发布或使用站点公网地址,拒绝鉴权产物与越界路径。"""
+ url = str(raw or "").strip()
+ if not url or is_tokenfree_image_url(url):
+ return None
+ if public := _public_reference(url):
+ return public
+ if not storage.is_local_static_url(url):
+ return None
+ local = storage.local_path_from_url(url)
+ if not local or not local.resolve().is_relative_to(storage.STATIC_ROOT.resolve()) or not local.is_file():
+ return None
+ published = str(storage.republish_url(url, sync=True) or "")
+ return _public_reference(published) or _public_reference(storage.to_public_url(url))
+
+
+def project_image_reference_inputs(project: Any) -> tuple[list[str], list[str]]:
+ """只读取主体与画风引用槽位;预占计费时不上传文件。"""
+ template = getattr(project, "template", None)
+ config = getattr(template, "seedream_config", None) or {}
+ config = config if isinstance(config, dict) else {}
+ styles = config.get("style_ref_urls") or []
+ styles = [url.strip() for url in styles if isinstance(url, str) and url.strip()] if isinstance(styles, list) else []
+ legacy_refs = config.get("ref_images") or []
+ legacy_refs = [url.strip() for url in legacy_refs if isinstance(url, str) and url.strip()] if isinstance(legacy_refs, list) else []
+ cover = str(getattr(template, "preview_cover", None) or "").strip()
+ if not cover:
+ from app.services.creation_templates import CREATION_TEMPLATE_IDS
+
+ template_id = getattr(template, "id", None)
+ if template_id in CREATION_TEMPLATE_IDS:
+ cover = f"/static/templates/covers/{template_id}.png"
+ if not styles and cover:
+ styles = [cover]
+ subject = str(getattr(project, "ref_image_url", None) or "").strip()
+ subjects = list(dict.fromkeys(([subject] if subject else []) + legacy_refs))[:2]
+ return subjects, styles
+
+
+def project_image_references(project: Any) -> tuple[list[str], list[str]]:
+ """解析用户主体图与独立画风图;真实生成不可静默退回纯文字模式。"""
+ subjects, styles = project_image_reference_inputs(project)
+ settings = get_settings()
+ mocked = bool(settings.ark_mock or settings.local_media_mock)
+
+ def resolve(url: str) -> str | None:
+ """离线模拟不上传;真实任务仅传公网图。"""
+ if mocked and url.startswith("/static/") and ".." not in url.split("/"):
+ return url
+ return public_image_reference(url)
+
+ subject_refs = [value for url in subjects if (value := resolve(url))]
+ style_refs = []
+ for url in styles:
+ if value := resolve(url):
+ style_refs.append(value)
+ break
+ if subjects and not subject_refs:
+ raise ValueError("主体参考图无法访问,请重新上传参考图,或检查公网访问地址和 OSS 配置")
+ if styles and not style_refs:
+ raise ValueError("画风参考图无法访问,请检查公网访问地址或 OSS 配置后重试")
+ return subject_refs, style_refs
diff --git a/backend/app/services/media_models/planning.py b/backend/app/services/media_models/planning.py
index e2103c1..981828c 100644
--- a/backend/app/services/media_models/planning.py
+++ b/backend/app/services/media_models/planning.py
@@ -22,6 +22,50 @@ def reference_params(payload: dict) -> dict:
return {key: payload[key] for key in REFERENCE_FIELDS if key in payload}
+def _kepu_stored_reference(*candidates: str | None) -> str | None:
+ """Inspect saved refs without publishing files or forwarding private artifacts."""
+ from app.services.tokenfree_image import is_tokenfree_image_url
+ from app.services.tokenfree_video import is_tokenfree_content_url
+
+ for raw in candidates:
+ url = str(raw or "").strip()
+ if url and not is_tokenfree_image_url(url) and not is_tokenfree_content_url(url):
+ return url
+ return None
+
+
+def _kepu_stored_image_ref(shot) -> str | None:
+ return _kepu_stored_reference(getattr(shot, "image_url", None), getattr(shot, "image_ark_url", None))
+
+
+def _kepu_video_refs(shot, previous, pending_video_ids: set[int]) -> list[str]:
+ """Reserve future tail slots; the worker supplies their actual URLs later."""
+ refs = [_kepu_stored_image_ref(shot) or f"pending://shot-{shot.shot_no}-image"]
+ if previous is None:
+ return refs
+ tail = _kepu_stored_reference(getattr(previous, "last_frame_url", None))
+ if previous.id in pending_video_ids or (getattr(previous, "video_url", None) and not tail):
+ tail = f"pending://shot-{previous.shot_no}-last-frame"
+ extra = tail or _kepu_stored_image_ref(previous)
+ if extra:
+ refs.append(extra)
+ return refs
+
+
+def _kepu_image_refs(project, shot, predecessors, pending_image_ids: set[int]) -> dict:
+ """Reserve the fixed style board, user image and previous shot without uploading."""
+ from app.services.kepu_image_references import project_image_reference_inputs
+
+ subjects, styles = project_image_reference_inputs(project)
+ previous = next((item for item in reversed(predecessors)
+ if item.id in pending_image_ids or _kepu_stored_image_ref(item)), None)
+ if previous is not None:
+ ref = (f"pending://shot-{previous.shot_no}-image" if previous.id in pending_image_ids
+ else _kepu_stored_image_ref(previous))
+ subjects = [ref, *subjects]
+ return {"reference_images": list(dict.fromkeys(subjects))[:2], "style_ref_urls": styles[:1]}
+
+
async def canvas_video_reference_content(db, task, payload: dict) -> list[dict]:
"""Build the same reference slots as the worker without publishing or generating media."""
from app.models_drama import DramaProject
@@ -80,22 +124,44 @@ async def plan_items(db, task) -> list[dict]:
if not project:
return []
from app.services.kepu_stages import normalize_kepu_pipeline_phase, shot_image_ready, shot_video_ready
- from app.services.pipeline import _image_size_for, _project_output_ratio
+ from app.services.pipeline import _image_size_for, _kepu_seedance_generate_audio, _project_output_ratio
+ from app.services import seedance_segments as segplan
phase = normalize_kepu_pipeline_phase(str(p.get("phase") or "script"), project)
- shots = project.shots
+ all_shots = sorted(project.shots, key=lambda shot: shot.shot_no)
+ shots = all_shots
if kind != "project_pipeline":
shots = [x for x in shots if x.id == task.shot_id]
phase = "assets" if kind == "shot_regen_image" else "videos"
+ pending_video_ids = {
+ shot.id for shot in shots
+ if phase == "videos" and (kind != "project_pipeline" or not shot_video_ready(shot))
+ }
+ pending_image_ids = {
+ shot.id for shot in shots
+ if phase == "assets" and (kind != "project_pipeline" or not shot_image_ready(shot))
+ }
for shot in shots:
if phase == "assets" and (kind != "project_pipeline" or not shot_image_ready(shot)):
+ predecessors = [previous for previous in all_shots if previous.shot_no < shot.shot_no]
add("image", {"size": _image_size_for(project) or settings.ark_image_size,
- "aspect_ratio": _project_output_ratio(project) or "1:1"},
+ "aspect_ratio": _project_output_ratio(project) or "1:1",
+ **_kepu_image_refs(project, shot, predecessors, pending_image_ids)},
model=project.image_model, target=shot.id, shot_no=shot.shot_no)
if phase == "videos" and (kind != "project_pipeline" or not shot_video_ready(shot)):
from app.services.seedance_resolutions import project_video_resolution
resolution = project_video_resolution(project, settings)
- add("video", {"duration": int(shot.duration or 5), "resolution": resolution,
- "aspect_ratio": _project_output_ratio(project) or "9:16"},
+ script = (getattr(shot, "segment_script", "") or getattr(shot, "video_prompt", "") or "").strip()
+ duration = segplan.resolve_api_duration(script, fallback=shot.duration or 5,
+ lo=settings.seedance_duration_min, hi=settings.seedance_duration_max)
+ predecessors = [previous for previous in all_shots if previous.shot_no < shot.shot_no]
+ if kind == "shot_regen_video":
+ predecessors = [previous for previous in predecessors if _kepu_stored_image_ref(previous)
+ or _kepu_stored_reference(getattr(previous, "last_frame_url", None))]
+ previous = predecessors[-1] if predecessors else None
+ add("video", {"duration": duration, "resolution": resolution,
+ "aspect_ratio": _project_output_ratio(project) or settings.ark_video_ratio,
+ "generate_audio": _kepu_seedance_generate_audio(project),
+ "reference_images": _kepu_video_refs(shot, previous, pending_video_ids)},
model=project.video_model, target=shot.id, shot_no=shot.shot_no)
return items
if kind in {"asset_image", "tool_image"}:
diff --git a/backend/app/services/pipeline.py b/backend/app/services/pipeline.py
index b42a9ad..ae096ac 100644
--- a/backend/app/services/pipeline.py
+++ b/backend/app/services/pipeline.py
@@ -30,7 +30,6 @@ from app.services import storage
from app.services.style_lock import (
build_locked_image_prompt,
merge_negative,
- seedream_ref_urls,
strip_lock_blocks,
template_allow_source_names,
template_consistency_mode,
@@ -41,8 +40,10 @@ from app.services.kepu_continuity import (
image_refs_for_shot,
persist_last_frame_from_video,
previous_usable_shot,
+ video_image_ref,
video_extra_refs_for_shot,
)
+from app.services.kepu_image_references import project_image_references
from app.services.voices import resolve_speaker
from app.services import seedance_segments as segplan
from app.services.bgm import clip_shot_bgm, resolve_bgm_path
@@ -186,7 +187,7 @@ def clear_pipeline_cancellation(project_id: int) -> None:
def _is_image_text(project: Project) -> bool:
- return (project.pipeline_mode or "full") == "image_text"
+ return (getattr(project, "pipeline_mode", None) or "full") == "image_text"
def _kepu_seedance_native_audio(project: Project | None = None) -> bool:
@@ -866,14 +867,6 @@ def _project_image_negative(project: Project) -> str:
)
-def _project_base_refs(project: Project) -> list[str]:
- tpl = project.template
- refs = list((tpl.seedream_config or {}).get("ref_images") or []) if tpl else []
- if project.ref_image_url:
- refs = [project.ref_image_url, *refs]
- return seedream_ref_urls(*refs)
-
-
def _locked_shot_prompt(project: Project, img_prompt: str) -> str:
consist = template_consistency_mode(project.template)
return build_locked_image_prompt(
@@ -941,7 +934,6 @@ async def _parallel_image_and_audio(project_id: int) -> None:
photoreal = template_is_photoreal(tpl)
consist = template_consistency_mode(tpl)
lock_character = consist == "character"
- base_refs = _project_base_refs(project)
image_size = _image_size_for(project)
negative = _project_image_negative(project)
voice = _project_voice(project)
@@ -951,11 +943,18 @@ async def _parallel_image_and_audio(project_id: int) -> None:
if not shot_meta:
return
+ # 同步发布参考图可能等待 OSS,必须释放数据库连接后执行。
+ base_refs, style_refs = (
+ await asyncio.to_thread(project_image_references, project)
+ if any(not meta["has_image"] for meta in shot_meta) else ([], [])
+ )
+
img_sem = asyncio.Semaphore(max(1, cfg.pipeline_image_concurrency))
done_img = 0
progress_lock = asyncio.Lock()
# 静图成片仍走外部 TTS;full + Seedance 内置口播则跳过
- need_audio = (not _kepu_seedance_native_audio(project)) and (
+ native_audio = _kepu_seedance_native_audio(project)
+ need_audio = (not native_audio) and (
not _continuous_audio_ok(project_id)
)
@@ -975,7 +974,8 @@ async def _parallel_image_and_audio(project_id: int) -> None:
project.status = ProjectStatus.IMAGING
await db.commit()
msg = f"出图 {done_img}/{total}" + (
- " · 整片配音生成中…" if need_audio else " · 配音已就绪"
+ " · 整片配音生成中…" if need_audio else
+ " · 声音将在视频阶段生成" if native_audio else " · 配音已就绪"
)
await publish_progress(
project_id,
@@ -1036,12 +1036,13 @@ async def _parallel_image_and_audio(project_id: int) -> None:
size=image_size,
model=image_model,
aspect_ratio=output_ratio or None,
+ style_ref_urls=style_refs or None,
)
await persist_image(meta, img)
from app.services.media_models.runtime import current_snapshot
if (current_snapshot.get() or {}).get("mode") != "fixed_media":
await _record_seedream_usage(project_id, image_result=img,
- model=get_settings().model_image, shot_id=meta["id"])
+ model=image_model or get_settings().model_image, shot_id=meta["id"])
return SimpleNamespace(image_ark_url=img.remote_url, image_url=img.local_url)
async def run_images() -> None:
@@ -1055,7 +1056,7 @@ async def _parallel_image_and_audio(project_id: int) -> None:
)
await bump_images()
continue
- refs = image_refs_for_shot(prev_proxy, base_refs)
+ refs = await asyncio.to_thread(image_refs_for_shot, prev_proxy, base_refs)
proxy = await one_image(meta, refs)
if proxy:
prev_proxy = proxy
@@ -1128,7 +1129,7 @@ async def _parallel_image_and_audio(project_id: int) -> None:
"stage": "ASSETS_READY",
"percent": 70 if image_text else 50,
"message": (
- "分镜图与整片配音已完成"
+ "分镜图已完成,下一步生成视频与声音" if native_audio else "分镜图与整片配音已完成"
),
},
)
@@ -1169,7 +1170,7 @@ async def _parallel_videos(project_id: int) -> None:
"video_prompt": s.video_prompt,
"segment_script": getattr(s, "segment_script", "") or s.video_prompt or "",
"camera": s.camera,
- "image_ref": s.image_ark_url or s.image_url or "",
+ "image_ref": video_image_ref(s),
"image_ark_url": s.image_ark_url,
"image_url": s.image_url,
"last_frame_url": getattr(s, "last_frame_url", None),
@@ -1594,14 +1595,18 @@ async def regen_shot_image(project_id: int, shot_id: int) -> None:
shot = next((s for s in project.shots if s.id == shot_id), None)
if not shot:
raise ValueError("shot not found")
- # 生成前先备好参考,避免长等待占着同一条 DB 连接
+ # 先读齐元数据;参考图发布与模型请求都在数据库连接释放后进行。
shot_no = shot.shot_no
- ref_urls = image_refs_for_shot(previous_usable_shot(list(project.shots), shot_no), _project_base_refs(project))
+ shot_rows = list(project.shots)
negative = _project_image_negative(project)
prompt = _locked_shot_prompt(project, shot.img_prompt)
image_size = _image_size_for(project)
image_model = (getattr(project, "image_model", None) or "").strip()
aspect_ratio = _project_output_ratio(project) or None
+ base_refs, style_refs = await asyncio.to_thread(project_image_references, project)
+ ref_urls = await asyncio.to_thread(
+ lambda: image_refs_for_shot(previous_usable_shot(shot_rows, shot_no), base_refs)
+ )
img = await ark.gen_image(
prompt,
negative,
@@ -1611,6 +1616,7 @@ async def regen_shot_image(project_id: int, shot_id: int) -> None:
size=image_size,
model=image_model,
aspect_ratio=aspect_ratio,
+ style_ref_urls=style_refs or None,
)
cancelled = False
async with AsyncSessionLocal() as db:
@@ -1635,7 +1641,7 @@ async def regen_shot_image(project_id: int, shot_id: int) -> None:
await _record_seedream_usage(
project_id,
image_result=img,
- model=s.model_image,
+ model=image_model or s.model_image,
shot_id=shot_id,
delivered=not cancelled,
)
@@ -1685,7 +1691,7 @@ async def regen_shot_video(project_id: int, shot_id: int) -> None:
hi=cfg.seedance_duration_max,
)
shot_no = shot.shot_no
- image_ref = shot.image_ark_url or shot.image_url or ""
+ image_ref = video_image_ref(shot)
extra_refs = video_extra_refs_for_shot(previous_usable_shot(list(project.shots), shot_no))
video_model = (getattr(project, "video_model", None) or "").strip()
ratio = _project_output_ratio(project) or cfg.ark_video_ratio
diff --git a/backend/tests/test_kepu_full_media_e2e.py b/backend/tests/test_kepu_full_media_e2e.py
new file mode 100644
index 0000000..cab62dd
--- /dev/null
+++ b/backend/tests/test_kepu_full_media_e2e.py
@@ -0,0 +1,140 @@
+"""离线完整模式验收:参考图生图 → 图片生有声视频 → 真正的 FFmpeg 成片。"""
+from __future__ import annotations
+
+import json
+import shutil
+import subprocess
+from contextlib import asynccontextmanager
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, Mock
+
+import httpx
+import pytest
+
+from app.config import get_settings
+from app.models import Project, ProjectStatus, Shot, ShotStatus
+from app.services import ffmpeg_compose, pipeline, storage
+
+
+def _run(command):
+ """运行本机媒体工具;失败时保留简短诊断。"""
+ result = subprocess.run(command, capture_output=True, text=True, encoding="utf-8", errors="replace", timeout=60)
+ assert result.returncode == 0, result.stderr[-3000:]
+ return result.stdout
+
+
+async def test_full_reference_images_to_native_audio_videos_and_playable_vertical_final(monkeypatch, tmp_path):
+ """全链路保留画风/主体/镜间参考;素材、音轨和成片均为临时目录内真实文件。"""
+ settings = get_settings()
+ ffmpeg = shutil.which(settings.ffmpeg_path) or shutil.which("ffmpeg")
+ ffprobe = shutil.which(settings.ffprobe_path) or shutil.which("ffprobe")
+ if not ffmpeg or not ffprobe or not ffmpeg_compose._find_cjk_font():
+ pytest.skip("real full-media acceptance requires FFmpeg, ffprobe and a Chinese font")
+ for key, value in {"ark_mock": True, "kepu_seedance_native_audio": True, "ffmpeg_path": ffmpeg,
+ "ffprobe_path": ffprobe, "public_base_url": "https://media.example.com"}.items():
+ monkeypatch.setattr(settings, key, value)
+ monkeypatch.setattr(storage, "STATIC_ROOT", tmp_path / "static")
+ monkeypatch.setattr(storage, "GENERATED_ROOT", tmp_path / "static/generated")
+ monkeypatch.setattr("app.services.oss.oss_enabled", lambda: False)
+ network = AsyncMock(side_effect=AssertionError("full-media acceptance must stay offline"))
+ monkeypatch.setattr(httpx.AsyncClient, "send", network)
+ monkeypatch.setattr(pipeline, "resolve_bgm_path", lambda _mood: None)
+ monkeypatch.setattr(pipeline, "_ensure_not_cancelled", AsyncMock())
+ for name in ("publish_progress", "_record_seedream_usage", "_record_seedance_usage"):
+ monkeypatch.setattr(pipeline, name, AsyncMock())
+ external_tts = AsyncMock(side_effect=AssertionError("native-audio mode must not synthesize external TTS"))
+ monkeypatch.setattr(pipeline, "_synthesize_continuous_audio", external_tts)
+
+ style = storage.project_dir(98784) / "style.png"
+ _run([ffmpeg, "-y", "-f", "lavfi", "-i", "testsrc2=size=144x256:rate=1", "-frames:v", "1", str(style)])
+ style_url = storage.publish_local(style)
+ template = SimpleNamespace(
+ id="anim_3d", style_prefix="统一三维卡通画风", preview_cover=style_url, negative_prompt="模糊",
+ seedream_config={"consistency_mode": "character"}, seedance_config={"character_consistency": True},
+ audio_config={}, subtitle_config={}, default_ratio="9:16",
+ )
+ shots = [SimpleNamespace(
+ id=no, shot_no=no, duration=4, narration=text, img_prompt="三维角色观察天空",
+ video_prompt="@duration:4 同一个角色讲解", segment_script="@duration:4 同一个角色讲解",
+ image_url=None, image_ark_url=None, audio_url=None, video_url=None, last_frame_url=None,
+ video_skip_reason=None, camera="push-in", bgm_mood="", status=ShotStatus.PENDING,
+ ) for no, text in enumerate(("阳光遇到空气。", "天空呈现蓝色。"), 1)]
+ subject = storage.to_public_url(style_url)
+ project = SimpleNamespace(
+ id=98784, user_id=1, pipeline_mode="full", template=template, shots=shots,
+ ref_image_url=subject, character_bible="同一个三维卡通角色", style_prompt="统一三维卡通画风",
+ voice_id="", image_model="seedream-5.0-pro", video_model="seedance-2.5", image_resolution="2K",
+ video_resolution="720p", output_ratio="9:16", resolution_mode="preview", progress=15,
+ status=ProjectStatus.SCRIPT_READY, final_video_url=None,
+ )
+ db = SimpleNamespace(execute=AsyncMock(return_value=SimpleNamespace(scalar_one=lambda: project)),
+ commit=AsyncMock(), add=Mock())
+
+ async def get_row(model, row_id):
+ """持久化替身仅回读本用例的项目与分镜。"""
+ return project if model is Project else next(shot for shot in shots if shot.id == row_id)
+
+ @asynccontextmanager
+ async def session():
+ """隔离数据库,同时让实际阶段逻辑写回同一组素材。"""
+ yield db
+
+ db.get = AsyncMock(side_effect=get_row)
+ monkeypatch.setattr(pipeline, "AsyncSessionLocal", session)
+ image_inputs, video_inputs = [], []
+
+ async def image(prompt, negative, refs, **kwargs):
+ """本地图片替代生图模型,并记录画风、主体和镜间参考。"""
+ image_inputs.append((prompt, refs, kwargs))
+ path = storage.project_dir(project.id) / f"frame_{kwargs['shot_no']}.png"
+ shutil.copyfile(style, path)
+ return SimpleNamespace(local_url=storage.publish_local(path), remote_url=None)
+
+ async def video(image_ref, prompt, duration, **kwargs):
+ """从本镜静帧做缩放动画并合成有声音轨,替代云端图生视频。"""
+ video_inputs.append((image_ref, kwargs))
+ path = storage.project_dir(project.id) / f"video_{kwargs['shot_no']}.mp4"
+ _run([ffmpeg, "-y", "-loop", "1", "-framerate", "12", "-i", str(storage.local_path_from_url(image_ref)),
+ "-f", "lavfi", "-i", f"sine=frequency={440 + kwargs['shot_no'] * 100}:sample_rate=24000",
+ "-vf", "zoompan=z='min(zoom+0.004,1.5)':d=1:s=144x256:fps=12", "-t", str(duration),
+ "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", "-c:a", "aac", str(path)])
+ tail = path.with_name(f"tail_{kwargs['shot_no']}.png")
+ _run([ffmpeg, "-y", "-sseof", "-0.1", "-i", str(path), "-frames:v", "1", str(tail)])
+ return storage.publish_local(path), SimpleNamespace(last_frame_url=storage.publish_local(tail))
+
+ gateway = SimpleNamespace(gen_image=image, gen_and_wait_video=video,
+ tts=AsyncMock(side_effect=AssertionError("external TTS forbidden")))
+ monkeypatch.setattr(pipeline, "get_ark", lambda: gateway)
+
+ await pipeline._parallel_image_and_audio(project.id)
+ assert project.status == ProjectStatus.IMAGE_READY
+ assert all(call[2]["style_ref_urls"] == [style_url] for call in image_inputs)
+ assert all(subject in call[1] and "三维卡通" in call[0] for call in image_inputs)
+ assert storage.to_public_url(shots[0].image_url) in image_inputs[1][1]
+ await pipeline._parallel_videos(project.id)
+ assert project.status == ProjectStatus.VIDEO_READY
+ assert [storage.local_path_from_url(call[0]) for call in video_inputs] == [
+ storage.local_path_from_url(shot.image_url) for shot in shots
+ ]
+ assert all(call[1]["generate_audio"] is True for call in video_inputs)
+ assert video_inputs[1][1]["extra_image_urls"] == [shots[0].last_frame_url]
+ await pipeline._compose_stage(project.id)
+
+ final = storage.local_path_from_url(project.final_video_url)
+ assert project.status == ProjectStatus.DONE and final.is_file() and final.stat().st_size > 2000
+ probe = json.loads(_run([ffprobe, "-v", "error", "-show_streams", "-show_format", "-of", "json", str(final)]))
+ visual = next(stream for stream in probe["streams"] if stream["codec_type"] == "video")
+ audio = next(stream for stream in probe["streams"] if stream["codec_type"] == "audio")
+ assert visual["codec_name"] == "h264" and audio["codec_name"] == "aac"
+ assert visual["width"] * 16 == visual["height"] * 9
+ assert 7.8 <= float(probe["format"]["duration"]) <= 8.5
+ assert abs(float(visual["duration"]) - float(audio["duration"])) < 0.2
+ assert not ffmpeg_compose.is_near_silent_audio(final)
+ frames = _run([ffmpeg, "-v", "error", "-i", str(final), "-an", "-vf", "select='eq(n,0)+eq(n,25)'",
+ "-fps_mode", "vfr", "-frames:v", "2", "-f", "framemd5", "-"])
+ hashes = [line.rsplit(",", 1)[1].strip() for line in frames.splitlines() if line and not line.startswith("#")]
+ assert len(hashes) == 2 and hashes[0] != hashes[1]
+ assert all(shot.audio_url is None for shot in shots)
+ external_tts.assert_not_awaited()
+ gateway.tts.assert_not_awaited()
+ network.assert_not_awaited()
diff --git a/backend/tests/test_kepu_image_references.py b/backend/tests/test_kepu_image_references.py
new file mode 100644
index 0000000..c59f927
--- /dev/null
+++ b/backend/tests/test_kepu_image_references.py
@@ -0,0 +1,232 @@
+"""短视频生图始终携带画风参考,并保留用户和镜间参考。"""
+from types import SimpleNamespace
+from contextlib import asynccontextmanager
+from unittest.mock import AsyncMock
+
+import pytest
+
+from app.config import get_settings
+from app.services import storage
+from app.models import Project, ProjectStatus, Shot, ShotStatus, Template
+from app.services import pipeline
+from app.services.ark import ArkGateway
+from app.models_tasks import TaskRun
+from app.services.media_models import MediaValidationError, get_strategy, planning
+from app.services.kepu_image_references import (
+ project_image_reference_inputs,
+ project_image_references,
+ public_image_reference,
+)
+
+
+def _project(**overrides):
+ """构造具有默认画风封面的短视频项目。"""
+ data = {"template": SimpleNamespace(seedream_config={}, preview_cover="/static/templates/covers/anim_3d.png"),
+ "ref_image_url": None}
+ return SimpleNamespace(**{**data, **overrides})
+
+
+def real_references(monkeypatch, tmp_path):
+ """使用本地媒体和公网地址模拟真实参考图发布,不访问网络。"""
+ settings = get_settings()
+ monkeypatch.setattr(settings, "ark_mock", False)
+ monkeypatch.setattr(settings, "local_media_mock", False)
+ monkeypatch.setattr(settings, "public_base_url", "https://media.example.com")
+ monkeypatch.setattr(storage, "STATIC_ROOT", tmp_path)
+ monkeypatch.setattr(storage, "republish_url", lambda url, **_: url)
+ cover = tmp_path / "templates/covers/anim_3d.png"
+ cover.parent.mkdir(parents=True)
+ cover.write_bytes(b"reference image")
+ return tmp_path
+
+
+def test_first_shot_gets_template_cover_as_dedicated_style_reference(real_references):
+ """模板没配 ref_images 时仍以所选画风封面作参考。"""
+ subjects, styles = project_image_references(_project())
+ assert subjects == []
+ assert styles == ["https://media.example.com/static/templates/covers/anim_3d.png"]
+
+
+def test_builtin_legacy_empty_cover_uses_bundled_style_board(real_references):
+ """旧模板行封面为空时,内置画风仍使用随代码发布的参考图。"""
+ project = _project(template=SimpleNamespace(id="anim_3d", preview_cover="", seedream_config={}))
+ assert project_image_references(project)[1] == ["https://media.example.com/static/templates/covers/anim_3d.png"]
+
+
+def test_user_reference_does_not_replace_style_reference(real_references):
+ """用户图用于主体,画风图独立保留。"""
+ subjects, styles = project_image_references(_project(ref_image_url="https://media.example.com/person.png"))
+ assert subjects == ["https://media.example.com/person.png"]
+ assert styles == ["https://media.example.com/static/templates/covers/anim_3d.png"]
+
+
+def test_configured_style_board_takes_priority_over_cover(real_references):
+ """后台配置的画风参考优先于展示封面。"""
+ project = _project(template=SimpleNamespace(
+ seedream_config={"style_ref_urls": ["https://media.example.com/style.png"]},
+ preview_cover="/static/templates/covers/anim_3d.png"))
+ assert project_image_references(project)[1] == ["https://media.example.com/style.png"]
+
+
+def test_legacy_subject_images_are_preserved_alongside_explicit_style_board(real_references):
+ """新增画风槽位不能吞掉旧模板的角色与产品参考。"""
+ project = _project(template=SimpleNamespace(
+ seedream_config={"ref_images": ["https://media.example.com/person.png", "https://media.example.com/product.png"],
+ "style_ref_urls": ["https://media.example.com/style.png"]}, preview_cover=""))
+ assert project_image_references(project) == (
+ ["https://media.example.com/person.png", "https://media.example.com/product.png"],
+ ["https://media.example.com/style.png"],
+ )
+
+
+def test_local_reference_prefers_synchronous_publishing(real_references, monkeypatch):
+ """中间图显式同步发布,不能把本地 URL 直接交给云端。"""
+ published = []
+ def publish(url, *, sync):
+ """模拟 OSS 发布。"""
+ published.append((url, sync))
+ return "https://oss.example.com/style.png"
+ monkeypatch.setattr(storage, "republish_url", publish)
+ assert project_image_references(_project())[1] == ["https://oss.example.com/style.png"]
+ assert published == [("/static/templates/covers/anim_3d.png", True)]
+
+
+def test_inaccessible_style_fails_instead_of_silently_using_text_only(real_references, monkeypatch):
+ """真实生成没有可访问参考图时,明确失败而非退回纯提示词。"""
+ monkeypatch.setattr(get_settings(), "public_base_url", "http://127.0.0.1:8000")
+ with pytest.raises(ValueError, match="画风参考图"):
+ project_image_references(_project())
+
+
+def test_local_reference_cannot_fall_back_to_metadata_server(real_references, monkeypatch):
+ """本地图片转绝对地址时同样拒绝私网地址。"""
+ monkeypatch.setattr(get_settings(), "public_base_url", "https://169.254.169.254")
+ assert public_image_reference("/static/templates/covers/anim_3d.png") is None
+
+
+async def test_video_gateway_accepts_public_local_image_without_oss(real_references):
+ """图生视频与生图共用可访问的本地图,无 OSS 时也可使用公网 HTTPS 地址。"""
+ gateway = object.__new__(ArkGateway)
+ assert await gateway._resolve_image_ref("/static/templates/covers/anim_3d.png", prefer_https=True) == (
+ "https://media.example.com/static/templates/covers/anim_3d.png"
+ )
+
+
+async def test_video_gateway_rejects_local_only_reference(real_references, monkeypatch):
+ """云端无法读取本机图片时,在提交付费请求前报错。"""
+ monkeypatch.setattr(get_settings(), "public_base_url", "http://127.0.0.1:8000")
+ gateway = object.__new__(ArkGateway)
+ with pytest.raises(RuntimeError, match="公网可访问"):
+ await gateway._resolve_image_ref("/static/templates/covers/anim_3d.png", prefer_https=True)
+
+
+def test_mock_reference_does_not_publish_or_require_media_files(monkeypatch):
+ """离线生成允许本地画风图,不触发 OSS 或 HTTP。"""
+ monkeypatch.setattr(get_settings(), "ark_mock", True)
+ monkeypatch.setattr(storage, "republish_url", lambda *_args, **_kwargs: pytest.fail("mock must not upload"))
+ assert project_image_references(_project())[1] == ["/static/templates/covers/anim_3d.png"]
+
+
+def test_admission_reads_reference_slots_without_publishing(monkeypatch):
+ """预占阶段仅采集引用,不上传也不发模型请求。"""
+ monkeypatch.setattr(storage, "republish_url", lambda *_args, **_kwargs: pytest.fail("admission must not upload"))
+ subjects, styles = project_image_reference_inputs(_project(ref_image_url="/static/person.png"))
+ assert subjects == ["/static/person.png"]
+ assert styles == ["/static/templates/covers/anim_3d.png"]
+
+
+ "https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content",
+ "/static/../secret.png", "file:///private/image.png", "https://127.0.0.1/style.png",
+ "https://169.254.169.254/style.png", "https://127.0.0.2/style.png",
+])
+def test_reference_rejects_private_provider_or_unsafe_sources(real_references, url):
+ """鉴权产物地址、本地越界路径与私网地址不能泄给下游。"""
+ assert public_image_reference(url) is None
+
+
[email protected]("regenerate", [False, True], ids=["batch", "single-redraw"])
+async def test_image_generation_receives_fixed_style_and_previous_subject(real_references, monkeypatch, regenerate):
+ """批量生成与单镜重绘都真实向网关传入固定画风、用户图和上一镜图。"""
+ template = Template(id="anim_3d", name="3D 动画", style_prefix="圆润三维动画",
+ preview_cover="/static/templates/covers/anim_3d.png", seedream_config={},
+ seedance_config={}, audio_config={})
+ shots = [Shot(id=no, shot_no=no, img_prompt=f"同一角色观察天空 {no}", duration=5,
+ status=ShotStatus.PENDING, version=1) for no in (1, 2)]
+ project = Project(id=991, user_id=1, template_id=template.id, template=template,
+ pipeline_mode="full", output_ratio="9:16", image_model="seedream-5.0-pro",
+ image_resolution="2K", ref_image_url="https://media.example.com/character.png",
+ character_bible="蓝色外套的男孩", shots=shots, status=ProjectStatus.SCRIPT_READY)
+ db = SimpleNamespace(execute=AsyncMock(return_value=SimpleNamespace(scalar_one=lambda: project)),
+ commit=AsyncMock())
+
+ async def get_row(model, row_id):
+ """回读当前项目和分镜。"""
+ return project if model is Project else next((shot for shot in shots if shot.id == row_id), None)
+
+ @asynccontextmanager
+ async def sessions():
+ """离线数据库替身。"""
+ yield db
+
+ async def gen_image(_prompt, _negative, _refs, **kwargs):
+ """保存图片以供下一镜参考,无模型或网络调用。"""
+ path = storage.project_dir(project.id) / f"shot_{kwargs['shot_no']}.png"
+ path.write_bytes(b"generated image")
+ return SimpleNamespace(local_url=storage.rel_static_url(path),
+ remote_url="https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content")
+
+ generate = AsyncMock(side_effect=gen_image)
+ db.get = AsyncMock(side_effect=get_row)
+ monkeypatch.setattr(storage, "GENERATED_ROOT", real_references / "generated")
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(pipeline, "AsyncSessionLocal", sessions)
+ monkeypatch.setattr(pipeline, "get_ark", lambda: SimpleNamespace(gen_image=generate))
+ for method in ("_ensure_not_cancelled", "_set_status", "publish_progress", "_record_seedream_usage"):
+ monkeypatch.setattr(pipeline, method, AsyncMock())
+ if regenerate:
+ shots[0].image_url = "https://media.example.com/previous.png"
+ shots[1].video_url = "/static/old.mp4"
+ project.final_video_url = "/static/old-final.mp4"
+ await pipeline.regen_shot_image(project.id, shots[1].id)
+ calls = generate.await_args_list
+ assert calls[0].args[2] == [shots[0].image_url, project.ref_image_url]
+ assert shots[1].video_url is None and project.final_video_url is None
+ else:
+ await pipeline._parallel_image_and_audio(project.id)
+ calls = generate.await_args_list
+ assert len(calls) == 2
+ assert calls[0].args[2] == [project.ref_image_url]
+ assert calls[1].args[2] == ["https://media.example.com/static/generated/p991/shot_1.png", project.ref_image_url]
+ for call in calls:
+ assert call.kwargs["style_ref_urls"] == ["https://media.example.com/static/templates/covers/anim_3d.png"]
+ assert call.kwargs["model"] == project.image_model
+ assert "三维" in call.args[0] and "蓝色外套" in call.args[0]
+
+
[email protected]("reference_cap", [3, 2])
+async def test_image_admission_reserves_style_subject_and_previous_slots_without_upload(monkeypatch, reference_cap):
+ """图片引用数量在受理时核验,后镜必需的画风名额不能被忽略。"""
+ template = SimpleNamespace(id="anim_3d", seedream_config={}, preview_cover="/static/style.png")
+ shots = [SimpleNamespace(id=no, shot_no=no, image_url=None, image_ark_url=None) for no in (1, 2)]
+ project = SimpleNamespace(id=1, template=template, ref_image_url="/static/subject.png", shots=shots,
+ image_model="seedream-5.0-pro", image_resolution="2K", output_ratio="9:16")
+ db = SimpleNamespace(execute=AsyncMock(return_value=SimpleNamespace(scalar_one_or_none=lambda: project)))
+ strategy = get_strategy("seedream-5.0-pro").with_reference_limits({"images": reference_cap, "videos": 0, "audios": 0})
+ monkeypatch.setattr(planning, "resolve_strategy", lambda *_: strategy)
+ monkeypatch.setattr(planning, "pin_route", lambda *_: {})
+ monkeypatch.setattr(storage, "republish_url", lambda *_args, **_kwargs: pytest.fail("admission must not upload"))
+ task = TaskRun(domain="kepu", task_type="project_pipeline", project_id=1, payload={"phase": "assets"})
+ if reference_cap == 2:
+ with pytest.raises(MediaValidationError, match="参考图片"):
+ await planning.plan_items(db, task)
+ return
+ items = await planning.plan_items(db, task)
+ assert items[0]["params"]["reference_images"] == ["/static/subject.png"]
+ assert items[1]["params"]["reference_images"] == ["pending://shot-1-image", "/static/subject.png"]
+ assert all(item["params"]["style_ref_urls"] == ["/static/style.png"] for item in items)
diff --git a/backend/tests/test_kepu_video_media_planning.py b/backend/tests/test_kepu_video_media_planning.py
new file mode 100644
index 0000000..e132e9a
--- /dev/null
+++ b/backend/tests/test_kepu_video_media_planning.py
@@ -0,0 +1,171 @@
+"""Short-video admission reserves the same video request as the worker, offline."""
+from types import SimpleNamespace
+from unittest.mock import AsyncMock, Mock
+
+import httpx
+import pytest
+
+from app.config import get_settings
+from app.models_tasks import TaskRun
+from app.services import pipeline, storage
+from app.services.media_models import planning, runtime
+
+
+def _shot(no, **changes):
+ return SimpleNamespace(**{
+ "id": no, "shot_no": no, "duration": 5, "segment_script": "",
+ "video_prompt": "", "image_url": f"/static/generated/p1/shot_{no:03d}.png",
+ "image_ark_url": None, "video_url": None, "last_frame_url": None,
+ "video_skip_reason": None, **changes,
+ })
+
+
+def _project(shots, **changes):
+ return SimpleNamespace(**{
+ "id": 1, "pipeline_mode": "full", "image_model": "seedream-5.0-pro",
+ "video_model": "seedance-2.5", "output_ratio": "9:16", "resolution_mode": "sd",
+ "video_resolution": "720p", "template": None, "shots": shots, **changes,
+ })
+
+
+def _db(project):
+ return SimpleNamespace(execute=AsyncMock(return_value=Mock(scalar_one_or_none=lambda: project)))
+
+
+def _task(kind="project_pipeline", **changes):
+ return TaskRun(domain="kepu", task_type=kind, project_id=1, payload={"phase": "videos"}, **changes)
+
+
+def offline(monkeypatch):
+ settings = get_settings()
+ monkeypatch.setattr(settings, "seedance_duration_min", 4)
+ monkeypatch.setattr(settings, "seedance_duration_max", 30)
+ monkeypatch.setattr(settings, "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(settings, "kepu_seedance_sfx_audio", False)
+ monkeypatch.setattr(planning, "pin_route", lambda capability, model: {
+ "capability": capability, "logical_model_id": model, "upstream_model": model,
+ "channel_id": "__runtime__", "channel_name": "Runtime", "base_url": settings.ark_base_url,
+ "protocol": "openai", "api_format": "openai",
+ })
+ forbidden = Mock(side_effect=AssertionError("admission must not generate or publish media"))
+ monkeypatch.setattr(storage, "publish_local", forbidden)
+ monkeypatch.setattr(storage, "republish_url", forbidden)
+ monkeypatch.setattr(pipeline, "get_ark", forbidden)
+ network = AsyncMock(side_effect=AssertionError("admission must stay offline"))
+ monkeypatch.setattr(httpx.AsyncClient, "send", network)
+ return forbidden, network
+
+
+async def test_video_snapshot_pins_script_duration_audio_and_output_settings(offline, monkeypatch):
+ shot = _shot(1, duration=5, segment_script="@duration:4 观察天空\n@duration:6 解释散射")
+ project = _project([shot], output_ratio="16:9", video_resolution="1080p")
+ task = _task()
+ monkeypatch.setattr(planning, "active_book", AsyncMock(return_value=None))
+ await planning.prepare_task_media(_db(project), task)
+ saved = task.billing_snapshot["items"][0]
+ assert saved["params"]["duration"] == 10
+ assert saved["params"]["generate_audio"] is True
+ assert saved["params"]["resolution"] == "1080p"
+ assert saved["params"]["aspect_ratio"] == "16:9"
+ token = runtime.current_snapshot.set(task.billing_snapshot)
+ op_token = runtime.current_operation.set(None)
+ try:
+ _, actual, _ = runtime.execution_params("video", {
+ "duration": 5, "resolution": "480p", "aspect_ratio": "9:16",
+ "generate_audio": True, "reference_images": [shot.image_url],
+ }, shot_no=1, model=project.video_model)
+ assert actual["duration"] == 10
+ assert actual["resolution"] == "1080p" and actual["aspect_ratio"] == "16:9"
+ finally:
+ runtime.current_snapshot.reset(token)
+ runtime.current_operation.reset(op_token)
+ offline[0].assert_not_called()
+ offline[1].assert_not_awaited()
+
+
[email protected]("changes,expected", [
+ ({"video_prompt": "@duration:6 第一段\n@duration:6 第二段"}, 12),
+ ({"duration": 2}, 4),
+ ({"duration": 80}, 30),
+ ({"duration": 8.4}, 8),
+])
+async def test_video_snapshot_uses_legacy_script_and_worker_fallback_clamp(offline, changes, expected):
+ items = await planning.plan_items(_db(_project([_shot(1, **changes)])), _task())
+ assert items[0]["params"]["duration"] == expected
+
+
[email protected]("native,sfx,expected", [(True, False, True), (False, True, True), (False, False, False)])
+async def test_video_snapshot_tracks_native_and_effect_audio(offline, monkeypatch, native, sfx, expected):
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", native)
+ monkeypatch.setattr(get_settings(), "kepu_seedance_sfx_audio", sfx)
+ items = await planning.plan_items(_db(_project([_shot(1)])), _task())
+ assert items[0]["params"]["generate_audio"] is expected
+
+
+async def test_video_references_prefer_saved_image_and_reserve_generated_previous_tail(offline):
+ first = _shot(1, image_ark_url="https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content")
+ second = _shot(2)
+ items = await planning.plan_items(_db(_project([second, first])), _task())
+ assert [item["shot_no"] for item in items] == [1, 2]
+ assert items[0]["params"]["reference_images"] == [first.image_url]
+ refs = items[1]["params"]["reference_images"]
+ assert refs[0] == second.image_url and len(refs) == 2
+ assert refs[1].startswith("pending://")
+ assert not any("tokenfree.com" in ref for item in items for ref in item["params"]["reference_images"])
+ offline[0].assert_not_called()
+
+
+async def test_video_references_reuse_ready_previous_tail_without_regenerating_it(offline):
+ first = _shot(1, video_url="/static/generated/p1/shot_001.mp4", last_frame_url="/static/generated/p1/tail.jpg")
+ second = _shot(2)
+ items = await planning.plan_items(_db(_project([first, second])), _task())
+ assert len(items) == 1
+ assert items[0]["params"]["reference_images"] == [second.image_url, first.last_frame_url]
+
+
+async def test_video_references_reserve_frame_extraction_without_reading_existing_video(offline):
+ first = _shot(1, video_url="/static/generated/p1/shot_001.mp4")
+ second = _shot(2)
+ items = await planning.plan_items(_db(_project([first, second])), _task())
+ assert len(items) == 1
+ refs = items[0]["params"]["reference_images"]
+ assert refs[0] == second.image_url and len(refs) == 2 and refs[1].startswith("pending://")
+ offline[0].assert_not_called()
+
+
+async def test_video_references_never_snapshot_tokenfree_authenticated_urls(offline):
+ private = "https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content"
+ first = _shot(1, image_url=private, image_ark_url="https://cdn.example.com/first.png")
+ second = _shot(2, image_url=private, image_ark_url=private)
+ items = await planning.plan_items(_db(_project([first, second])), _task())
+ assert items[0]["params"]["reference_images"] == [first.image_ark_url]
+ assert items[1]["params"]["reference_images"][0].startswith("pending://")
+ assert not any("tokenfree.com" in ref for item in items for ref in item["params"]["reference_images"])
+
+
+async def test_video_regen_reserves_prior_usable_still_and_replaces_only_requested_shot(offline):
+ first = _shot(1, video_skip_reason="privacy", image_ark_url="https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content")
+ empty = _shot(2, image_url=None)
+ third = _shot(3, video_url="/static/generated/p1/old.mp4")
+ items = await planning.plan_items(_db(_project([third, empty, first])), _task("shot_regen_video", shot_id=3))
+ assert len(items) == 1 and items[0]["target_id"] == 3
+ assert items[0]["params"]["reference_images"] == [third.image_url, first.image_url]
+
+
+async def test_video_reference_cap_rejected_before_provider_or_publication(offline, monkeypatch):
+ base_resolve = planning.resolve_strategy
+ monkeypatch.setattr(planning, "resolve_strategy", lambda model: base_resolve(model).with_reference_limits({"images": 1, "audios": 0, "videos": 0}))
+ with pytest.raises(ValueError, match="reference_images"):
+ await planning.plan_items(_db(_project([_shot(1), _shot(2)])), _task())
+ offline[0].assert_not_called()
+ offline[1].assert_not_awaited()
diff --git a/backend/tests/test_kepu_video_workflow.py b/backend/tests/test_kepu_video_workflow.py
new file mode 100644
index 0000000..2614918
--- /dev/null
+++ b/backend/tests/test_kepu_video_workflow.py
@@ -0,0 +1,330 @@
+"""分步短视频模式的阶段、模式切换与单镜生成契约(全程离线)。"""
+from __future__ import annotations
+
+from contextlib import asynccontextmanager
+from types import SimpleNamespace
+from unittest.mock import AsyncMock
+
+import pytest
+from fastapi import HTTPException
+from sqlalchemy import select
+
+from app.api import projects
+from app.config import get_settings
+from app.models import Project, ProjectStatus, Shot, ShotStatus, Template
+from app.models_tasks import TaskRun
+from app.schemas import ProjectCreate, ProjectUpdate
+from app.services import pipeline
+from app.services.billing.settlement import freeze_for_task
+from app.services.media_models.prices import active_book, publish_book
+from tests.conftest import make_user
+
+
+def _shot(**overrides):
+ """构造已有首帧、尚未生成视频的分镜。"""
+ values = dict(
+ id=11, shot_no=1, duration=8, narration="同一个三维卡通角色观察天空。",
+ img_prompt="三维卡通角色", image_url="/static/shot.png", image_ark_url=None,
+ audio_url=None, video_url=None, last_frame_url=None, video_skip_reason=None,
+ video_prompt="@duration:8 缓慢推近", segment_script="@duration:8 缓慢推近",
+ camera="push-in", status=ShotStatus.IMAGE_READY, version=1,
+ )
+ return SimpleNamespace(**{**values, **overrides})
+
+
+def _project(**overrides):
+ """构造可编辑的分步项目并保留模板生成参数。"""
+ values = dict(
+ id=1, user_id=7, creation_session_id=None, pipeline_mode="full",
+ status=ProjectStatus.DONE, progress=100, final_video_url="/static/final.mp4",
+ error_msg=None, active_tasks=[], shots=[_shot()], output_ratio="9:16",
+ resolution_mode="hd", image_resolution="2K", video_resolution="720p",
+ style_prompt="统一三维卡通画风", video_model="seedance-2-5",
+ template=SimpleNamespace(style_prefix="三维卡通", default_ratio="16:9",
+ seedance_config={"motion_bias": "平稳", "character_consistency": True}),
+ )
+ return SimpleNamespace(**{**values, **overrides})
+
+
+def _api_context(monkeypatch, project):
+ """隔离拥有者查询与数据库提交,仅记录实际路由创建的任务。"""
+ monkeypatch.setattr(projects, "_get_owned_project", AsyncMock(return_value=project))
+ db = SimpleNamespace(commit=AsyncMock(), rollback=AsyncMock())
+ user = SimpleNamespace(id=project.user_id)
+ return db, user
+
+
[email protected]("before,after,ready_audio,expected", [
+ ("image_text", "full", False, ProjectStatus.IMAGE_READY),
+ ("full", "image_text", False, ProjectStatus.SCRIPT_READY),
+ ("full", "image_text", True, ProjectStatus.VIDEO_READY),
+])
+async def test_mode_change_invalidates_final_and_resolves_new_stage(
+ monkeypatch, before, after, ready_audio, expected,
+):
+ """切换制作模式沿用已有素材,但成片与就绪状态必须按新模式重算。"""
+ project = _project(pipeline_mode=before)
+ image = project.shots[0].image_url
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr("app.services.kepu_stages.continuous_narration_ok", lambda _id: ready_audio)
+ monkeypatch.setattr("app.services.kepu_stages.shot_audio_file_ok", lambda _shot: ready_audio)
+ db, user = _api_context(monkeypatch, project)
+
+ updated = await projects.update_project(project.id, ProjectUpdate(pipeline_mode=after), db, user)
+
+ assert updated.pipeline_mode == after
+ assert updated.final_video_url is None
+ assert updated.status == expected
+ assert updated.shots[0].image_url == image
+ db.commit.assert_awaited_once()
+
+
+async def test_unchanged_mode_preserves_completed_output(monkeypatch):
+ """重复保存同一模式不会让已完成作品失效。"""
+ project = _project()
+ db, user = _api_context(monkeypatch, project)
+ await projects.update_project(project.id, ProjectUpdate(pipeline_mode="full"), db, user)
+ assert project.status == ProjectStatus.DONE
+ assert project.final_video_url == "/static/final.mp4"
+
+
+async def test_mode_and_bgm_change_rechecks_new_mode_even_from_image_ready(monkeypatch):
+ """素材就绪状态和同时保存的配乐不能掩盖新模式所缺的配音阶段。"""
+ project = _project(status=ProjectStatus.IMAGE_READY)
+ monkeypatch.setattr("app.services.kepu_stages.continuous_narration_ok", lambda _id: False)
+ monkeypatch.setattr("app.services.kepu_stages.shot_audio_file_ok", lambda _shot: False)
+ db, user = _api_context(monkeypatch, project)
+ await projects.update_project(
+ project.id, ProjectUpdate(pipeline_mode="image_text", bgm_lock="轻快"), db, user,
+ )
+ assert project.pipeline_mode == "image_text" and project.bgm_lock == "轻快"
+ assert project.status == ProjectStatus.SCRIPT_READY
+ assert project.final_video_url is None
+
+
+async def test_mode_change_rejects_active_generation(monkeypatch):
+ """视频生成执行期间不能改模式而让计费与实际阶段分离。"""
+ project = _project(status=ProjectStatus.VIDEOING)
+ db, user = _api_context(monkeypatch, project)
+ with pytest.raises(HTTPException) as error:
+ await projects.update_project(project.id, ProjectUpdate(pipeline_mode="image_text"), db, user)
+ assert error.value.status_code == 409
+ assert project.pipeline_mode == "full"
+ db.commit.assert_not_awaited()
+
+
+async def test_full_generate_routes_script_assets_and_video_before_compose(monkeypatch):
+ """完整模式每次只入队一个付费阶段,全部视频就绪后改走专用合成任务。"""
+ project = _project(status=ProjectStatus.DRAFT, shots=[], final_video_url=None)
+ db, user = _api_context(monkeypatch, project)
+ create_task = AsyncMock()
+ monkeypatch.setattr(projects, "create_task", create_task)
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(projects, "list_active_tasks_for_owner", AsyncMock(return_value=[]))
+
+ await projects.generate_project(project.id, db=db, user=user)
+ assert create_task.await_args.args[2].payload["phase"] == "script"
+ assert create_task.await_args.args[2].payload["pipeline_mode"] == "full"
+ assert project.status == ProjectStatus.SCRIPTING
+
+ project.status = ProjectStatus.SCRIPT_READY
+ project.shots = [_shot(image_url=None)]
+ await projects.generate_project(project.id, db=db, user=user)
+ assert create_task.await_args.args[2].payload["phase"] == "assets"
+ assert project.status == ProjectStatus.IMAGING
+
+ project.status = ProjectStatus.IMAGE_READY
+ project.shots[0].image_url = "/static/shot.png"
+ await projects.generate_project(project.id, db=db, user=user)
+ assert create_task.await_args.args[2].payload["phase"] == "videos"
+ assert project.status == ProjectStatus.VIDEOING
+
+ project.status = ProjectStatus.VIDEO_READY
+ project.shots[0].video_url = "/static/shot.mp4"
+ with pytest.raises(HTTPException, match="409"):
+ await projects.generate_project(project.id, db=db, user=user)
+ assert create_task.await_count == 3
+ await projects.compose_only(project.id, db=db, user=user)
+ assert create_task.await_args.args[2].task_type == "project_compose_only"
+ assert project.status == ProjectStatus.COMPOSING
+
+
[email protected]("mode,has_image,allowed", [
+ ("full", True, True), ("full", False, False), ("image_text", True, False),
+])
+async def test_regen_video_requires_full_mode_and_a_starting_image(monkeypatch, mode, has_image, allowed):
+ """视频重生成只允许完整模式、且已有首帧的分镜。"""
+ project = _project(pipeline_mode=mode, shots=[_shot(image_url="/static/shot.png" if has_image else None)])
+ db, user = _api_context(monkeypatch, project)
+ create_task = AsyncMock()
+ monkeypatch.setattr(projects, "_create_kepu_task", create_task)
+ if allowed:
+ await projects.regen_video(project.id, 11, db, user)
+ assert create_task.await_args.kwargs["task_type"] == "shot_regen_video"
+ assert project.status == ProjectStatus.VIDEOING
+ else:
+ with pytest.raises(HTTPException) as error:
+ await projects.regen_video(project.id, 11, db, user)
+ assert error.value.status_code == 400
+ create_task.assert_not_awaited()
+ db.commit.assert_not_awaited()
+
+
+def test_full_create_contract_preserves_user_media_choices():
+ """创建参数支持视频模式、参考图、选定的模型和输出清晰度。"""
+ request = ProjectCreate(
+ template_id="anim_3d", source_text="讲解天空为什么是蓝色。", pipeline_mode="full",
+ ref_image_url="/static/reference.png", image_model="gpt-image-1.5", image_resolution="2K",
+ video_model="seedance-2-5", video_resolution="720p", target_duration_sec=60,
+ )
+ assert request.pipeline_mode == "full"
+ assert request.ref_image_url == "/static/reference.png"
+ assert request.image_model == "gpt-image-1.5" and request.image_resolution == "2K"
+ assert request.video_model == "seedance-2-5" and request.video_resolution == "720p"
+
+
[email protected]("upstream_image,expected_first_frame", [
+ ("https://media.example/start-1.png", "https://media.example/start-1.png"),
+ ("https://www.tokenfree.com/v1/tasks/t1/artifacts/image-0/content", "/static/shot.png"),
+])
+async def test_full_video_generation_uses_own_image_and_previous_tail_with_native_audio(
+ monkeypatch, upstream_image, expected_first_frame,
+):
+ """图生视频沿用本镜首帧、前镜尾帧、脚本时长和项目模型,完整模式保留内置口播。"""
+ shot1 = _shot(image_ark_url=upstream_image, duration=3,
+ segment_script="@duration:4 观察天空\n@duration:4 蓝光向四周散射")
+ shot2 = _shot(id=12, shot_no=2, image_url="/static/start-2.png", duration=6,
+ segment_script="@duration:6 同一个角色展示蓝色天空")
+ project = _project(shots=[shot2, shot1], final_video_url=None, status=ProjectStatus.IMAGE_READY)
+ db = SimpleNamespace(
+ execute=AsyncMock(return_value=SimpleNamespace(scalar_one=lambda: project)), commit=AsyncMock(),
+ )
+
+ async def get_row(model, row_id):
+ """模拟项目与两镜的持久化回读。"""
+ return project if model is Project else {shot1.id: shot1, shot2.id: shot2}.get(row_id)
+
+ @asynccontextmanager
+ async def session():
+ """为每次分镜写回提供同一离线数据库替身。"""
+ yield db
+
+ db.get = AsyncMock(side_effect=get_row)
+ generate_video = AsyncMock(side_effect=[
+ ("/static/video-1.mp4", SimpleNamespace(last_frame_url="/static/tail-1.png")),
+ ("/static/video-2.mp4", SimpleNamespace(last_frame_url="/static/tail-2.png")),
+ ])
+ record_usage = AsyncMock()
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(pipeline, "AsyncSessionLocal", session)
+ monkeypatch.setattr(pipeline, "get_ark", lambda: SimpleNamespace(gen_and_wait_video=generate_video))
+ monkeypatch.setattr(pipeline, "_ensure_not_cancelled", AsyncMock())
+ monkeypatch.setattr(pipeline, "_set_status", AsyncMock())
+ monkeypatch.setattr(pipeline, "publish_progress", AsyncMock())
+ monkeypatch.setattr(pipeline, "_record_seedance_usage", record_usage)
+ monkeypatch.setattr(pipeline, "persist_last_frame_from_video", lambda *args, **kwargs: kwargs.get("preferred_url"))
+
+ await pipeline._parallel_videos(project.id)
+
+ first, second = generate_video.await_args_list
+ assert first.args[0] == expected_first_frame
+ assert second.args[0] == "/static/start-2.png"
+ assert first.args[2] == 8 and second.args[2] == 6
+ assert first.kwargs["extra_image_urls"] is None
+ assert second.kwargs["extra_image_urls"] == ["/static/tail-1.png"]
+ for call in (first, second):
+ assert call.kwargs["generate_audio"] is True
+ assert call.kwargs["model"] == "seedance-2-5"
+ assert call.kwargs["resolution"] == "720p" and call.kwargs["ratio"] == "9:16"
+ assert call.kwargs["character_consistency"] is True
+ assert "三维卡通" in call.args[1]
+ assert shot1.video_url == "/static/video-1.mp4" and shot2.video_url == "/static/video-2.mp4"
+ assert project.status == ProjectStatus.VIDEO_READY and project.progress == 88
+ assert [call.kwargs["fallback_duration_sec"] for call in record_usage.await_args_list] == [8, 6]
+ assert all(call.kwargs["billing_key"] == "seedance2:video0" for call in record_usage.await_args_list)
+
+
+async def test_full_project_video_admission_reserves_only_remaining_video_cost(db_session, monkeypatch):
+ """真实创建/入队/冻结链路只为尚缺的视频预扣,不重复计入已有视频与图片。"""
+ db = db_session
+ user = await make_user(db, balance_fen=100_000)
+ template = await db.get(Template, "anim_3d")
+ if template is None:
+ db.add(Template(id="anim_3d", name="3D动画", style_prefix="三维卡通"))
+ await db.flush()
+ book = await active_book(db)
+ await publish_book(
+ db, prices={"seedance-2.5": {"720p": "0.5000"}},
+ expected_version=book["version"] if book else None, user_id=user.id,
+ )
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(get_settings(), "ark_mock", True)
+ project = await projects.create_project(ProjectCreate(
+ template_id="anim_3d", source_text="讲解天空为什么是蓝色。", pipeline_mode="full",
+ video_model="seedance-2.5", video_resolution="720p", output_ratio="9:16",
+ ), db, user)
+ project.shots = [
+ Shot(shot_no=1, duration=8, image_url="/static/first.png", segment_script="@duration:8 观察天空"),
+ Shot(shot_no=2, duration=6, image_url="/static/second.png", video_url="/static/kept.mp4"),
+ ]
+ project.status = ProjectStatus.IMAGE_READY
+ await db.commit()
+
+ generated = await projects.generate_project(project.id, db=db, user=user)
+ task = (await db.execute(select(TaskRun).where(TaskRun.project_id == project.id))).scalar_one()
+ assert generated.pipeline_mode == "full" and generated.status == ProjectStatus.VIDEOING
+ assert task.payload["phase"] == "videos"
+ assert task.billing_snapshot["mode"] == "fixed_media"
+ items = task.billing_snapshot["items"]
+ assert len(items) == 1 and items[0]["target_id"] == project.shots[0].id
+ assert items[0]["params"]["duration"] == 8
+ assert items[0]["model_id"] == "seedance-2.5" and items[0]["params"]["resolution"] == "720p"
+ before = user.balance_fen
+ reserved = await freeze_for_task(db, task)
+ assert reserved == 400
+ assert user.balance_fen == before - reserved and user.frozen_fen == reserved
+ assert task.billing_status == "frozen"
+
+
[email protected]("upstream_image,expected_first_frame", [
+ ("https://media.example/start-2.png", "https://media.example/start-2.png"),
+ ("https://www.tokenfree.com/v1/tasks/t2/artifacts/image-0/content", "/static/start-2.png"),
+])
+async def test_single_video_regeneration_preserves_image_duration_and_audio_contract(
+ monkeypatch, upstream_image, expected_first_frame,
+):
+ """单镜重生成使用与批量生成一致的首帧选择、前镜尾帧和内置口播参数。"""
+ first = _shot(last_frame_url="/static/previous-tail.png", video_url="/static/kept.mp4")
+ shot = _shot(id=12, shot_no=2, duration=3, image_ark_url=upstream_image,
+ image_url="/static/start-2.png", segment_script="@duration:9 同一个角色继续讲解")
+ project = _project(shots=[first, shot], status=ProjectStatus.VIDEO_READY)
+ db = SimpleNamespace(execute=AsyncMock(return_value=SimpleNamespace(scalar_one=lambda: project)),
+ commit=AsyncMock())
+
+ async def get_row(model, row_id):
+ """为单镜重生成提供对应的项目与分镜。"""
+ return project if model is Project else shot
+
+ @asynccontextmanager
+ async def session():
+ """用内存替身隔离重生成的写回。"""
+ yield db
+
+ db.get = AsyncMock(side_effect=get_row)
+ generate_video = AsyncMock(return_value=("/static/regenerated.mp4", SimpleNamespace(last_frame_url="/static/new-tail.png")))
+ monkeypatch.setattr(get_settings(), "kepu_seedance_native_audio", True)
+ monkeypatch.setattr(pipeline, "AsyncSessionLocal", session)
+ monkeypatch.setattr(pipeline, "get_ark", lambda: SimpleNamespace(gen_and_wait_video=generate_video))
+ monkeypatch.setattr(pipeline, "_record_seedance_usage", AsyncMock())
+ monkeypatch.setattr(pipeline, "persist_last_frame_from_video", lambda *args, **kwargs: kwargs.get("preferred_url"))
+
+ await pipeline.regen_shot_video(project.id, shot.id)
+
+ call = generate_video.await_args
+ assert call.args[0] == expected_first_frame and call.args[2] == 9
+ assert call.kwargs["extra_image_urls"] == ["/static/previous-tail.png"]
+ assert call.kwargs["generate_audio"] is True and call.kwargs["character_consistency"] is True
+ assert call.kwargs["model"] == "seedance-2-5"
+ assert call.kwargs["resolution"] == "720p" and call.kwargs["ratio"] == "9:16"
+ assert shot.video_url == "/static/regenerated.mp4" and shot.last_frame_url == "/static/new-tail.png"
+ assert project.final_video_url is None
diff --git a/docs/API.md b/docs/API.md
index 033e7c5..ee2b37d 100644
--- a/docs/API.md
+++ b/docs/API.md
@@ -53,6 +53,21 @@ Authorization: Bearer <login-jwt>
当前新建短片采用 `/api/projects` 分步创作流程:准备文案、创建或更新项目、生成分镜与素材,再由用户确认后合成成片。
+`POST /api/projects` 与 `PATCH /api/projects/{project_id}` 使用 `pipeline_mode` 区分制作方式:
+
+| 模式 | 素材与生成顺序 |
+| --- | --- |
+| `full` | 分镜脚本 → 参考图模式生成分镜图片 → 图片生成镜头视频 → 合成成片 |
+| `image_text` | 分镜脚本 → 参考图模式生成分镜图片与配音 → 静图合成成片 |
+
+当前新建页面默认选择 `full`,仍可选择 `image_text`。图片沿用选定模板的画风参考图、项目 `ref_image_url` 及已有镜头参考,锁定画风和角色;镜头视频以本镜图片作为首帧,并参考上一镜尾帧保持衔接。`image_model` / `video_model` 分别选择图片与视频模型,`image_resolution` / `video_resolution` 保存相应清晰度,`target_duration_sec` 为 15~180 秒的目标时长。完整视频模式沿用原有视频内置口播逻辑;静图模式保留外部配音。
+
+画风图优先取模板 `seedream_config.style_ref_urls`,未配置时取模板封面;四个内置画风的旧模板行即使封面为空,也使用随代码提供的封面图。历史 `ref_images` 继续作为主体参考,用户图优先;后镜另加上一镜图片,画风参考独立传递。真实生成时,本地图片需同步发布到 OSS 或通过公网 `PUBLIC_BASE_URL/static/...` 访问,无法访问时返回明确错误,避免静默变成纯文字生图。镜头图片与视频参考均跳过需要鉴权的 TokenFree 产物地址,使用已保存的媒体。
+
+调用 `POST /api/projects/{project_id}/generate` 每次只提交当前待执行的一段任务,阶段由后端素材就绪状态判定:`script`、`assets`、`videos`。素材阶段完成后,完整模式再次调用生成接口才进入视频阶段;静图模式跳过视频阶段。全部素材就绪时生成接口返回 409,改用 `POST /api/projects/{project_id}/compose` 合成。各阶段仅预扣本次尚未完成的媒体费用,已有图片和视频会复用。
+
+单镜调整可使用 `POST /api/projects/{project_id}/shots/{shot_id}/regen-image` 与 `.../regen-video`;后者仅支持 `full`,并要求该镜已有图片。生成期间不能切换制作模式;通过 PATCH 切换模式会保留原分镜素材,作废原成片并按新模式回到待生成或待合成阶段。若要重新拆分镜,可调用生成接口并设置 `restart=true`,这会清除原分镜和素材。
+
旧版对话创作会话的创建、列表、详情、消息、取消、重试与 SSE 接口已删除,访问这些旧入口返回 404。其前端会话状态与轮询代码也已移除。
仅保留 `POST /api/creation-sessions/{session_id}/open-editor`,归入“科普项目与作品”分组,用于将历史项目转入当前工作台。该接口会先等待旧任务停止并结算,再交接原项目及素材;历史任务执行与恢复代码、数据库记录保留。
diff --git a/docs/BILLING.md b/docs/BILLING.md
index 5f8f108..1dcd7ca 100644
--- a/docs/BILLING.md
+++ b/docs/BILLING.md
@@ -88,6 +88,8 @@ billing_expr USD sell = official_CNY / 7.3 * 1.5
## TaskRun 计费流程
+AI 短视频的图片与镜头视频分阶段预占,只计尚未交付的素材。视频快照与执行共用分镜 `segment_script` 的段落时长、站点时长边界、声音开关、清晰度及画幅;不再只按旧 `shot.duration` 截整后预占。图片任务同时预留固定画风图、用户图与前镜引用槽位,视频任务预留本镜图片与前镜尾帧槽位,超过模型参考图上限时在受理阶段拒绝,不等付费调用后才发现参数不兼容。
+
任务中心分别展示「预估」「冻结状态」「实扣」「退款」与用量计费依据;未结算时实扣应显示未结算,而不是已消费 0 元。冻结是任务的预留金额,不是额外收费;总费用不能把冻结和实扣相加。取消/失败后按既有结算逻辑释放冻结,并保留真实已发生的上游用量。
上游请求记录使用 `task_events` 的 `upstream.request.started/accepted/failed/poll` 事件,与 `billing_scope` 内的任务关联。记录真实 HTTP 方法、路径、模型、上游任务 ID、HTTP 响应码与耗时,以及脱敏后的请求/响应;内部任务参数不是实际 HTTP 请求体。轮询按状态变化采样,不按每轮重复刷事件。没有历史记录时显示「未记录」,不能事后伪造。请求日志不替代 `usage_events` 的真实计费依据,也不因日志写入失败重新提交上游。
diff --git a/frontend/src/api.ts b/frontend/src/api.ts
index 36328f8..cfa8ade 100644
--- a/frontend/src/api.ts
+++ b/frontend/src/api.ts
@@ -370,6 +370,10 @@ export const api = {
style_prompt?: string
character_prompt?: string
extra_prompt?: string
+ image_model?: string
+ video_model?: string
+ image_resolution?: string
+ video_resolution?: string
}) {
return request<Project>('/api/projects', { method: 'POST', body: JSON.stringify(body) })
},
diff --git a/frontend/src/components/layout/SiteNav.tsx b/frontend/src/components/layout/SiteNav.tsx
index b7046a3..3e4e0ec 100644
--- a/frontend/src/components/layout/SiteNav.tsx
+++ b/frontend/src/components/layout/SiteNav.tsx
@@ -1,7 +1,7 @@
import { Link, useLocation, useNavigate } from 'react-router-dom'
import { useCallback, useEffect, useRef, useState } from 'react'
import {
- ChevronRight, CircleHelp, Clapperboard, CreditCard, House,
+ ChevronRight, CircleHelp, Clapperboard, CodeXml, CreditCard, House,
Menu, Plus, Store,
Video, Workflow, X,
} from 'lucide-react'
@@ -11,6 +11,7 @@ import BrandMark from '../BrandMark'
import CreateChoiceModal from '../ui/CreateChoiceModal'
import { USER_UPDATED_EVENT } from '../../lib/userEvents'
import { loadSessionUser } from '../../lib/sessionUser'
+import { CODE_VIDEO_URL } from '../../lib/siteLinks'
import { HOME_NAV, CREATION_NAV, RESOURCE_NAV, PERSONAL_NAV, resolveNavActive, type NavActive } from '../../lib/siteNavigation'
import { useI18n } from '../../i18n'
import LanguageSwitch from './LanguageSwitch'
@@ -187,6 +188,13 @@ export default function SiteNav({ active }: Props) {
</Link>
}
+ function renderCodeVideo(expanded = false) {
+ return <a href={CODE_VIDEO_URL} target="_blank" rel="noopener noreferrer" className={`pf-topbar-link${expanded ? ' pf-topbar-menu-link' : ''}`} onClick={() => setMenuOpen(false)}>
+ {expanded && <CodeXml size={18} strokeWidth={1.8} aria-hidden />}
+ <span>{t('nav.codeVideo')}</span>
+ </a>
+ }
+
/** 市场入口遵从服务端能力开关,进入个人市场时清理团队请求范围。 */
function renderMarketplace(expanded = false) {
if (!marketplace?.enabled) return null
@@ -204,6 +212,7 @@ export default function SiteNav({ active }: Props) {
{renderLink(HOME_NAV)}
{CREATION_NAV.map((item) => renderLink(item))}
{renderMarketplace()}
+ {renderCodeVideo()}
{RESOURCE_NAV.map((item) => renderLink(item))}
</nav>
<div className="pf-topbar-actions">
@@ -221,7 +230,7 @@ export default function SiteNav({ active }: Props) {
<nav aria-label={t('nav.main')}>
<div className="pf-topbar-menu-home">{renderLink(HOME_NAV, true)}</div>
<div className="pf-topbar-menu-group"><span>{t('nav.creation')}</span><div>{CREATION_NAV.map((item) => renderLink(item, true))}</div></div>
- <div className="pf-topbar-menu-group"><div>{renderMarketplace(true)}{RESOURCE_NAV.map((item) => renderLink(item, true))}</div></div>
+ <div className="pf-topbar-menu-group"><div>{renderMarketplace(true)}{renderCodeVideo(true)}{RESOURCE_NAV.map((item) => renderLink(item, true))}</div></div>
</nav>
<div className="pf-topbar-menu-footer">
<LanguageSwitch />
diff --git a/frontend/src/i18n/locales/en/shell.ts b/frontend/src/i18n/locales/en/shell.ts
index 62b1752..817febd 100644
--- a/frontend/src/i18n/locales/en/shell.ts
+++ b/frontend/src/i18n/locales/en/shell.ts
@@ -67,6 +67,7 @@ export const enShell = {
accountManagement: 'Account',
balanceBilling: 'Balance & billing',
assets: 'Assets',
+ codeVideo: 'Code to Video',
pricing: 'Pricing',
help: 'Help',
helpCenter: 'Help center',
diff --git a/frontend/src/i18n/locales/zh/shell.ts b/frontend/src/i18n/locales/zh/shell.ts
index ee64b09..11d318e 100644
--- a/frontend/src/i18n/locales/zh/shell.ts
+++ b/frontend/src/i18n/locales/zh/shell.ts
@@ -66,6 +66,7 @@ export const zhShell = {
accountManagement: '账户管理',
balanceBilling: '余额与账单',
assets: '资产',
+ codeVideo: '代码生视频',
pricing: '定价',
help: '帮助',
helpCenter: '帮助中心',
diff --git a/frontend/src/lib/shortVideoSettings.ts b/frontend/src/lib/shortVideoSettings.ts
new file mode 100644
index 0000000..4cbe94d
--- /dev/null
+++ b/frontend/src/lib/shortVideoSettings.ts
@@ -0,0 +1,66 @@
+import type { MediaModelOption, MediaModelsCatalog, PipelineMode } from '../api'
+
+type MediaSettings = {
+ imageModel: string
+ videoModel: string
+ imageResolution: string
+ videoResolution: string
+ ratio: string
+}
+
+const RATIOS = ['9:16', '16:9', '1:1', '4:3', '21:9']
+
+/** New projects use image-to-video; explicit still-image links keep their existing mode. */
+export function shortVideoPipelineMode(value?: string | null): PipelineMode {
+ return value === 'image_text' ? 'image_text' : 'full'
+}
+
+/** Reserve style/subject and continuity slots while accepting legacy catalogs without limits. */
+function referenceModels(models: MediaModelOption[], imageReferences: number) {
+ return models.filter(model => model.available !== false && (model.reference_limits === undefined || model.reference_limits.images >= imageReferences))
+}
+
+/** Only blank selections may use defaults; an unavailable explicit model must be replaced by the user. */
+function selectModel(models: MediaModelOption[], current: string, preferred: string) {
+ if (current) return models.find(model => model.id === current)
+ return models.find(model => model.id === preferred)
+ || models[0]
+}
+
+/** Priced resolutions take priority over the model's general capability list. */
+function resolutions(model?: MediaModelOption) {
+ return model?.available_resolutions ?? model?.allowed_resolutions ?? []
+}
+
+/** Keep supported quality choices and clamp incompatible choices after a model change. */
+function selectResolution(model: MediaModelOption | undefined, current: string) {
+ const allowed = resolutions(model)
+ return allowed.includes(current) ? current
+ : allowed.find(value => value === model?.defaults?.resolution) || allowed[0] || ''
+}
+
+/** Both stages use reference images; incompatible model parameters cannot start production. */
+export function resolveShortVideoMediaSettings(catalog: MediaModelsCatalog | null, mode: PipelineMode, current: MediaSettings, imageReferenceCount = 2) {
+ const imageOptions = referenceModels(catalog?.image_models ?? [], imageReferenceCount)
+ const videoOptions = referenceModels(catalog?.video_models ?? [], 2)
+ const image = selectModel(imageOptions, current.imageModel, catalog?.defaults.image_model || '')
+ const video = selectModel(videoOptions, current.videoModel, catalog?.defaults.video_model || '')
+ const imageRatios = image?.allowed_aspect_ratios ?? RATIOS
+ const videoRatios = video?.allowed_aspect_ratios ?? RATIOS
+ const ratios = RATIOS.filter(ratio => imageRatios.includes(ratio) && (mode === 'image_text' || videoRatios.includes(ratio)))
+ const imageResolution = image ? selectResolution(image, current.imageResolution) : current.imageResolution
+ const videoResolution = video ? selectResolution(video, current.videoResolution) : current.videoResolution
+ const ratio = ratios.includes(current.ratio) ? current.ratio : ratios[0] || ''
+ return {
+ imageOptions, videoOptions, ratios,
+ imageResolutions: resolutions(image), videoResolutions: resolutions(video),
+ imageModel: current.imageModel || image?.id || '',
+ videoModel: current.videoModel || video?.id || '',
+ imageSelectionUnavailable: Boolean(catalog && current.imageModel && !image),
+ videoSelectionUnavailable: Boolean(catalog && current.videoModel && !video),
+ imageResolution: catalog ? imageResolution : current.imageResolution,
+ videoResolution: catalog ? videoResolution : current.videoResolution,
+ ratio: catalog ? ratio : current.ratio,
+ ready: Boolean(catalog && image && imageResolution && ratio && (mode === 'image_text' || (video && videoResolution))),
+ }
+}
diff --git a/frontend/src/lib/siteLinks.ts b/frontend/src/lib/siteLinks.ts
index 1d664d4..cf58a9f 100644
--- a/frontend/src/lib/siteLinks.ts
+++ b/frontend/src/lib/siteLinks.ts
@@ -2,3 +2,4 @@
export const GITHUB_REPO_URL = 'https://github.com/yi1108/printfilm'
export const FILM_SKILLS_URL = 'https://www.gitcc.com/ai1/film-skills'
+export const CODE_VIDEO_URL = 'https://fog.printfilm.com/'
diff --git a/frontend/src/lib/status.ts b/frontend/src/lib/status.ts
index 3abc816..6a34c35 100644
--- a/frontend/src/lib/status.ts
+++ b/frontend/src/lib/status.ts
@@ -213,7 +213,7 @@ export const KEPU_STEPS = [
{ key: 'topic', label: '选题' },
{ key: 'style', label: '风格' },
{ key: 'confirm', label: '确认分镜' },
- { key: 'assets', label: '画面与配音' },
+ { key: 'assets', label: '参考图画面' },
{ key: 'videos', label: '镜头视频' },
{ key: 'compose', label: '合成预览' },
]
@@ -310,7 +310,7 @@ export function kepuPhaseHint(project: {
}
const phase = kepuBillingPhase(project)
if (phase === 'script') return '先在风格页点「生成故事板」,本步只拆分镜脚本(预扣文字模型)。'
- if (phase === 'assets') return '确认旁白与画面后开始生成:按镜头依次出图(后镜参考上一镜)+ 整片配音。'
- if (phase === 'videos') return '画面与配音已齐。下一步按镜头依次出视频,后镜参考上一镜尾帧。'
- return '素材已齐。拼接成片走后期合成(叠旁白字幕与配乐,扣费很少)。'
+ if (phase === 'assets') return '确认分镜后,使用画风参考图逐镜生成图片,并参考前镜画面保持角色与风格一致。'
+ if (phase === 'videos') return '图片已齐。使用每镜图片生成视频与声音,后镜参考上一镜尾帧延续画面。'
+ return '镜头视频已齐。检查各镜头画面与声音后,即可合成成片。'
}
diff --git a/frontend/src/pages/HomePage.tsx b/frontend/src/pages/HomePage.tsx
index 7fb4093..462870c 100644
--- a/frontend/src/pages/HomePage.tsx
+++ b/frontend/src/pages/HomePage.tsx
@@ -10,7 +10,7 @@ import { useI18n } from '../i18n'
import { getDramaImageStylePreviewUrl } from '../lib/dramaImageStylePreviews'
import { FILM_SKILLS_URL, GITHUB_REPO_URL } from '../lib/siteLinks'
import { api, type Template } from '../api'
-import { SHORT_VIDEO_LIST_PATH, shortVideoCreatePath } from '../lib/shortVideoRoutes'
+import { SHORT_VIDEO_LIST_PATH } from '../lib/shortVideoRoutes'
export default function HomePage() {
const nav = useNavigate()
@@ -18,14 +18,13 @@ export default function HomePage() {
const loggedIn = Boolean(localStorage.getItem('token'))
const [createOpen, setCreateOpen] = useState(false)
const [templates, setTemplates] = useState<Template[]>([])
- const [templatesError, setTemplatesError] = useState(false)
useEffect(() => {
let active = true
api.templates().then((items) => {
if (active) setTemplates(items)
}).catch(() => {
- if (active) setTemplatesError(true)
+ if (active) setTemplates([])
})
return () => { active = false }
}, [])
@@ -146,31 +145,6 @@ export default function HomePage() {
</button>
</section>
- <section className="pf-land-caps" aria-labelledby="pf-land-templates-title">
- <header className="pf-land-section-head">
- <p className="pf-land-kicker">Styles</p>
- <h2 id="pf-land-templates-title">{t('home.templatesTitle')}</h2>
- <p className="pf-muted">{t('home.templatesHint')}</p>
- </header>
- <div className="pf-template-grid">
- {templates.map((template) => (
- <button
- key={template.id}
- type="button"
- className="pf-template-card"
- onClick={() => goAuthOr(shortVideoCreatePath(template.id))}
- >
- <img src={api.assetUrl(template.preview_cover)} alt="" loading="lazy" />
- <div className="body">
- <h3>{template.name}</h3>
- <p>{template.description}</p>
- </div>
- </button>
- ))}
- </div>
- {templatesError ? <p className="pf-muted">{t('home.templatesLoadFailed')}</p> : null}
- </section>
-
<section className="pf-land-pipeline" aria-labelledby="pf-land-pipeline-title">
<header className="pf-land-section-head">
<p className="pf-land-kicker">{t('home.howKicker')}</p>
diff --git a/frontend/src/pages/studio/CreateProjectPage.tsx b/frontend/src/pages/studio/CreateProjectPage.tsx
index 4125695..8f154a3 100644
--- a/frontend/src/pages/studio/CreateProjectPage.tsx
+++ b/frontend/src/pages/studio/CreateProjectPage.tsx
@@ -1,14 +1,16 @@
import { useEffect, useRef, useState, type FormEvent } from 'react'
import { Link, useNavigate, useSearchParams } from 'react-router-dom'
import { ArrowLeft, ArrowRight, Check, Clapperboard, Clock3, LoaderCircle, Save, Sparkles } from 'lucide-react'
-import { api, type Project, type Template, type VoicePreset } from '../../api'
+import { api, type PipelineMode, type Project, type Template, type VoicePreset } from '../../api'
import AppShell from '../../components/layout/AppShell'
import BillingErrorNotice from '../../components/billing/BillingErrorNotice'
import Button from '../../components/ui/Button'
import Stepper from '../../components/ui/Stepper'
+import { MediaCostNotice } from '../../components/billing/MediaCostNotice'
+import { useMediaModelsCatalog } from '../../hooks/useMediaModelsCatalog'
import { creationTemplateId } from '../../lib/creationTemplates'
-import { IMAGE_TEXT_STEPS } from '../../lib/imageTextStoryboard'
-import { isRunning } from '../../lib/status'
+import { resolveShortVideoMediaSettings, shortVideoPipelineMode } from '../../lib/shortVideoSettings'
+import { isRunning, kepuSteps } from '../../lib/status'
import { dialog } from '../../lib/dialog'
import './shortVideoCreate.css'
@@ -19,11 +21,19 @@ export default function CreateProjectPage() {
const [params] = useSearchParams()
const editId = Number(params.get('project')) || null
const requestedTemplate = creationTemplateId(params.get('template'))
+ const requestedPipelineMode = shortVideoPipelineMode(params.get('mode'))
+ const returnPath = `/studio/new${params.size ? `?${params}` : ''}`
const [templates, setTemplates] = useState<Template[]>([])
const [voices, setVoices] = useState<VoicePreset[]>([])
const [project, setProject] = useState<Project | null>(null)
const [templateId, setTemplateId] = useState(requestedTemplate)
const [ratio, setRatio] = useState('9:16')
+ const [pipelineMode, setPipelineMode] = useState<PipelineMode>(requestedPipelineMode)
+ const [imageModel, setImageModel] = useState('')
+ const [videoModel, setVideoModel] = useState('')
+ const [imageResolution, setImageResolution] = useState('')
+ const [videoResolution, setVideoResolution] = useState('')
+ const mediaCatalog = useMediaModelsCatalog()
const [duration, setDuration] = useState(60)
const [voice, setVoice] = useState('')
const [mode, setMode] = useState<'idea' | 'copy'>('idea')
@@ -46,13 +56,13 @@ export default function CreateProjectPage() {
useEffect(() => {
let disposed = false
if (!localStorage.getItem('token')) {
- nav(`/auth?next=${encodeURIComponent('/studio/new' + (editId ? `?project=${editId}` : ''))}`, { replace: true })
+ nav(`/auth?next=${encodeURIComponent(returnPath)}`, { replace: true })
return
}
api.templates().then(rows => { if (!disposed) setTemplates(rows) }).catch(cause => { if (!disposed) setError(cause.message || '模板加载失败') })
api.voices().then(rows => { if (!disposed) setVoices(rows) }).catch(() => { if (!disposed) setVoiceError(true) })
return () => { disposed = true }
- }, [nav, editId])
+ }, [nav, editId, returnPath])
useEffect(() => {
let disposed = false
if (!editId) {
@@ -61,6 +71,11 @@ export default function CreateProjectPage() {
setLoading(false)
setTemplateId(requestedTemplate)
setRatio('9:16')
+ setPipelineMode(requestedPipelineMode)
+ setImageModel('')
+ setVideoModel('')
+ setImageResolution('')
+ setVideoResolution('')
setDuration(60)
setVoice('')
setMode('idea')
@@ -83,6 +98,11 @@ export default function CreateProjectPage() {
setProject(value)
setTemplateId(creationTemplateId(value.template_id))
setRatio(value.output_ratio || '9:16')
+ setPipelineMode(shortVideoPipelineMode(value.pipeline_mode))
+ setImageModel(value.image_model || '')
+ setVideoModel(value.video_model || '')
+ setImageResolution(value.image_resolution || '')
+ setVideoResolution(value.video_resolution || '')
setDuration(value.target_duration_sec || 60)
setVoice(value.voice_id || '')
setTitle(value.title)
@@ -95,24 +115,30 @@ export default function CreateProjectPage() {
}).catch(cause => { if (!disposed) setError(cause.message || '项目加载失败') })
.finally(() => { if (!disposed) setLoading(false) })
return () => { disposed = true }
- }, [editId, requestedTemplate, nav])
+ }, [editId, requestedTemplate, requestedPipelineMode, nav])
const selectedTemplate = templates.find(item => item.id === templateId)
+ const mediaSettings = resolveShortVideoMediaSettings(mediaCatalog, pipelineMode, { imageModel, videoModel, imageResolution, videoResolution, ratio }, project?.ref_image_url ? 3 : 2)
const running = Boolean(project && (isRunning(project.status) || project.active_tasks?.some(task => ['pending', 'leased', 'running', 'awaiting_poll', 'awaiting_review', 'cancel_requested'].includes(task.status))))
const disabled = Boolean(busy || loading || running)
function payload(text: string, name: string) {
return {
template_id: templateId, title: name.trim().slice(0, 200) || text.trim().slice(0, 30),
source_type: 'script' as const, source_text: text.trim(),
- pipeline_mode: 'image_text' as const, resolution_mode: 'hd' as const,
- output_ratio: ratio, target_duration_sec: duration, voice_id: voice,
+ pipeline_mode: pipelineMode, resolution_mode: 'hd' as const,
+ output_ratio: mediaSettings.ratio || ratio, target_duration_sec: duration, voice_id: voice,
+ image_model: mediaSettings.imageModel, video_model: mediaSettings.videoModel,
+ image_resolution: mediaSettings.imageResolution, video_resolution: mediaSettings.videoResolution,
}
}
const draftFields = payload(copy, title)
const draftChanged = Boolean(project && (
project.title !== draftFields.title || project.source_text !== draftFields.source_text ||
- project.template_id !== templateId || project.output_ratio !== ratio ||
- (project.target_duration_sec || 60) !== duration || (project.voice_id || '') !== voice
+ project.template_id !== templateId || project.output_ratio !== draftFields.output_ratio ||
+ shortVideoPipelineMode(project.pipeline_mode) !== pipelineMode ||
+ (project.target_duration_sec || 60) !== duration || (project.voice_id || '') !== voice ||
+ (project.image_model || '') !== draftFields.image_model || (project.video_model || '') !== draftFields.video_model ||
+ (project.image_resolution || '') !== draftFields.image_resolution || (project.video_resolution || '') !== draftFields.video_resolution
))
const canSaveDraft = !project?.shots.length && !project?.final_video_url
async function createDraft(text: string, name: string) {
@@ -173,11 +199,15 @@ export default function CreateProjectPage() {
}
async function generateStoryboard() {
if (!copy.trim() || disabled || working.current || !selectedTemplate) return
+ if (!mediaSettings.ready) {
+ setError('当前模型不可用或参考图数量不足,请重新选择图片和视频模型、清晰度与画面比例。')
+ return
+ }
working.current = true
try {
const old = savedProject.current
if (old?.shots.length) {
- const confirmed = await dialog.confirm({ title: '重新生成分镜', message: '新的文案与设置将重新生成分镜,并替换本项目已有的图片、配音和成片。', confirmText: '重新生成', tone: 'danger' })
+ const confirmed = await dialog.confirm({ title: '重新生成分镜', message: '新的文案与设置将重新生成分镜,并替换本项目已有的图片、镜头视频、配音和成片。', confirmText: '重新生成', tone: 'danger' })
if (!confirmed || !mounted.current) return
}
setBusy('正在保存并生成分镜…')
@@ -200,7 +230,7 @@ export default function CreateProjectPage() {
<div className="sv-create-page">
<header className="sv-create-header">
<div><Link className="sv-back" to="/history"><ArrowLeft size={16} /> 我的短片</Link><h1>{editId ? '编辑短片' : '创建短片'}</h1><p>从一个想法开始,每一步都可以查看和调整。</p></div>
- <Stepper steps={IMAGE_TEXT_STEPS} current={step} />
+ <Stepper steps={kepuSteps(pipelineMode)} current={pipelineMode === 'full' && step === 1 ? 2 : step} />
</header>
{error && <BillingErrorNotice message={error} />}
{loading ? <div className="sv-loading"><LoaderCircle className="sv-spin" size={20} /> 正在加载项目…</div> : <>
@@ -209,7 +239,7 @@ export default function CreateProjectPage() {
<main>
{step === 0 ? <form onSubmit={prepareCopy}>
<section className="sv-panel">
- <div className="sv-section-heading"><span>01</span><div><h2>选择画面风格</h2><p>四种动画与插画风格,整个短片保持统一。</p></div></div>
+ <div className="sv-section-heading"><span>01</span><div><h2>选择画面风格</h2><p>四种动画与插画风格,生图时使用画风参考图,逐镜延续角色与视觉风格。</p></div></div>
<div className="sv-style-grid" role="group" aria-label="画面风格">
{templates.map(item => <button key={item.id} type="button" className={`sv-style-card${item.id === templateId ? ' is-selected' : ''}`} disabled={disabled} aria-pressed={item.id === templateId} onClick={() => setTemplateId(item.id)}>
<img src={api.assetUrl(item.preview_cover)} alt="" /><div><strong>{item.name}</strong><p>{item.description}</p></div>
@@ -234,12 +264,22 @@ export default function CreateProjectPage() {
</main>
<aside className="sv-panel sv-settings">
<h2>短片设置</h2><p>先用默认设置,也可以按需要调整。</p>
- <label className="sv-field"><span>画面比例</span><select value={ratio} onChange={event => setRatio(event.target.value)} disabled={disabled}><option value="9:16">9:16 竖屏</option><option value="16:9">16:9 横屏</option><option value="1:1">1:1 方形</option></select></label>
+ <div className="sv-field"><span>成片方式</span><div className="sv-output-modes" role="group" aria-label="成片方式">{([{ id: 'full', label: 'AI 视频', desc: '参考图生图 → 图生视频 → 合成' }, { id: 'image_text', label: '静图成片', desc: '参考图生图 → 图片运镜与配音' }] as const).map(item => <button key={item.id} type="button" className={pipelineMode === item.id ? 'is-selected' : ''} aria-pressed={pipelineMode === item.id} disabled={disabled} onClick={() => setPipelineMode(item.id)}><strong>{item.label}</strong><span>{item.desc}</span>{pipelineMode === item.id && <Check size={14} />}</button>)}</div></div>
+ <label className="sv-field"><span>画面比例</span><select value={mediaSettings.ratio} onChange={event => setRatio(event.target.value)} disabled={disabled || !mediaSettings.ratios.length}>{!mediaSettings.ratios.length && <option value="">暂无共同支持的画幅</option>}{mediaSettings.ratios.map(value => <option key={value} value={value}>{value}{value === '9:16' ? ' 竖屏' : value === '16:9' ? ' 横屏' : value === '1:1' ? ' 方形' : ''}</option>)}</select></label>
<label className="sv-field"><span>目标时长</span><select value={duration} onChange={event => setDuration(Number(event.target.value))} disabled={disabled}>{[...new Set([30, 60, 90, 120, 180, duration])].sort((a, b) => a - b).map(value => <option key={value} value={value}>约 {value} 秒</option>)}</select></label>
- <label className="sv-field"><span>旁白音色</span><select value={voice} onChange={event => setVoice(event.target.value)} disabled={disabled}><option value="">默认普通话音色</option>{voice && !voices.some(item => (item.speaker || item.id) === voice) && <option value={voice}>项目原有音色</option>}{voices.map(item => <option key={item.id} value={item.speaker || item.id}>{item.label}</option>)}</select></label>
- {voiceError && <p className="sv-saved-note">音色列表暂未加载,仍可使用默认配音。</p>}
- <div className="sv-setting-summary"><Clapperboard size={17} /><div><strong>{selectedTemplate?.name || '3D 动画'}</strong><span>图片运镜 · 普通话旁白 · 中文字幕</span></div></div>
- <div className="sv-workflow-hint"><Clock3 size={17} /><p>检查分镜后生成图片和配音,素材准备好后再合成视频。</p></div>
+ {pipelineMode === 'image_text' ? <>
+ <label className="sv-field"><span>旁白音色</span><select value={voice} onChange={event => setVoice(event.target.value)} disabled={disabled}><option value="">默认普通话音色</option>{voice && !voices.some(item => (item.speaker || item.id) === voice) && <option value={voice}>项目原有音色</option>}{voices.map(item => <option key={item.id} value={item.speaker || item.id}>{item.label}</option>)}</select></label>
+ {voiceError && <p className="sv-saved-note">音色列表暂未加载,仍可使用默认配音。</p>}
+ </> : <p className="sv-saved-note">视频模型生成旁白与声音。</p>}
+ <details className="sv-model-settings" open={pipelineMode === 'full'}><summary>生成模型与清晰度</summary>
+ <label className="sv-field"><span>参考图生图模型</span><select value={mediaSettings.imageModel} disabled={disabled || !mediaSettings.imageOptions.length} onChange={event => setImageModel(event.target.value)}>{mediaSettings.imageSelectionUnavailable && <option value={mediaSettings.imageModel} disabled>{mediaSettings.imageModel}(当前不可用)</option>}{!mediaSettings.imageOptions.length && !mediaSettings.imageSelectionUnavailable && <option value="">{mediaCatalog ? '暂无可用参考图模型' : '正在加载模型…'}</option>}{mediaSettings.imageOptions.map(item => <option key={item.id} value={item.id}>{item.label}</option>)}</select></label>
+ <label className="sv-field"><span>图片清晰度</span><select value={mediaSettings.imageResolution} disabled={disabled || !mediaSettings.imageResolutions.length} onChange={event => setImageResolution(event.target.value)}>{!mediaSettings.imageResolutions.length && <option value={mediaSettings.imageResolution}>{mediaSettings.imageResolution || '请选择可用模型'}</option>}{mediaSettings.imageResolutions.map(value => <option key={value} value={value}>{value}</option>)}</select><MediaCostNotice modelId={mediaSettings.imageModel} resolution={mediaSettings.imageResolution} /></label>
+ {pipelineMode === 'full' && <><label className="sv-field"><span>图生视频模型</span><select value={mediaSettings.videoModel} disabled={disabled || !mediaSettings.videoOptions.length} onChange={event => setVideoModel(event.target.value)}>{mediaSettings.videoSelectionUnavailable && <option value={mediaSettings.videoModel} disabled>{mediaSettings.videoModel}(当前不可用)</option>}{!mediaSettings.videoOptions.length && !mediaSettings.videoSelectionUnavailable && <option value="">{mediaCatalog ? '暂无可用图生视频模型' : '正在加载模型…'}</option>}{mediaSettings.videoOptions.map(item => <option key={item.id} value={item.id}>{item.label}</option>)}</select></label>
+ <label className="sv-field"><span>视频清晰度</span><select value={mediaSettings.videoResolution} disabled={disabled || !mediaSettings.videoResolutions.length} onChange={event => setVideoResolution(event.target.value)}>{!mediaSettings.videoResolutions.length && <option value={mediaSettings.videoResolution}>{mediaSettings.videoResolution || '请选择可用模型'}</option>}{mediaSettings.videoResolutions.map(value => <option key={value} value={value}>{value}</option>)}</select><MediaCostNotice modelId={mediaSettings.videoModel} resolution={mediaSettings.videoResolution} seconds={1} /></label></>}
+ {mediaCatalog && !mediaSettings.ready && <p className="sv-model-error" role="status">请重新选择可用模型,生图需支持 {project?.ref_image_url ? 3 : 2} 张参考图{pipelineMode === 'full' ? ',视频需支持 2 张参考图' : ''},并确认清晰度和画幅。</p>}
+ </details>
+ <div className="sv-setting-summary"><Clapperboard size={17} /><div><strong>{selectedTemplate?.name || '3D 动画'}</strong><span>{pipelineMode === 'full' ? '图生视频' : '图片运镜'} · 普通话旁白 · 中文字幕</span><span>使用画风参考图,并参考前镜画面保持一致。</span></div></div>
+ <div className="sv-workflow-hint"><Clock3 size={17} /><p>{pipelineMode === 'full' ? '确认分镜后先生成参考图画面,再将图片逐镜生成为视频,检查镜头后合成成片。' : '检查分镜后生成图片和配音,素材准备好后再合成视频。'}</p></div>
</aside>
</div>
</>}
diff --git a/frontend/src/pages/studio/StoryboardPage.tsx b/frontend/src/pages/studio/StoryboardPage.tsx
index 1a15fe9..696c041 100644
--- a/frontend/src/pages/studio/StoryboardPage.tsx
+++ b/frontend/src/pages/studio/StoryboardPage.tsx
@@ -256,12 +256,12 @@ export default function StoryboardPage() {
: shots.length === 0
? '去风格页生成分镜'
: needsScriptConfirm
- ? '确认分镜,逐镜出图与配音'
+ ? '确认分镜,生成参考图画面'
: needsVideos
- ? '逐镜出视频'
+ ? '使用图片,逐镜生成视频'
: '继续生成'
- // 工作台进度:完整模式要镜头视频 + 外部 TTS,静图模式只配音合成
+ // 完整模式的默认旁白由视频模型生成;仅实际存在外部配音时展示 TTS 进度。
const progressItems = useMemo(() => {
if (!project) return []
const list = project.shots || []
@@ -275,13 +275,13 @@ export default function StoryboardPage() {
{ label: '主题解析', done: true },
{ label: '分镜脚本', done: list.length > 0 || !['DRAFT', 'SCRIPTING'].includes(project.status) },
{
- label: `画面生成 (${imgs}/${list.length || 0})`,
+ label: `参考图画面 (${imgs}/${list.length || 0})`,
done: list.length > 0 && imgs === list.length,
},
]
if (full) {
items.push({
- label: `镜头视频 (${vids}/${list.length || 0})`,
+ label: `镜头视频与声音 (${vids}/${list.length || 0})`,
done:
list.length > 0 &&
(vids === list.length ||
@@ -290,12 +290,14 @@ export default function StoryboardPage() {
pct: stage === 'VIDEOING' ? project.progress : undefined,
})
}
- items.push({
- label: `配音合成 (${auds}/${list.length || 0})`,
- done: list.length > 0 && auds === list.length,
- run: stage === 'AUDIOING',
- pct: stage === 'AUDIOING' ? project.progress : undefined,
- })
+ if (!full || auds > 0 || stage === 'AUDIOING') {
+ items.push({
+ label: `配音合成 (${auds}/${list.length || 0})`,
+ done: list.length > 0 && auds === list.length,
+ run: stage === 'AUDIOING',
+ pct: stage === 'AUDIOING' ? project.progress : undefined,
+ })
+ }
items.push({
label: full ? '镜头拼接' : '成片渲染',
done: Boolean(project.final_video_url) || project.status === 'DONE',
@@ -964,7 +966,7 @@ title="用当前镜头重新拼接"
<div className="pf-prompt-panel">
<h4>内置提示词</h4>
<p className="pf-muted" style={{ fontSize: '0.72rem', margin: '0 0 0.45rem' }}>
- 画风来自模板。角色是否出镜由 AI 按模板与主题决定;只有这里改过才覆盖本项目。
+ 生图使用模板画风参考图,并参考前镜画面延续角色、服装与色彩;每镜图片再用于生成视频。
</p>
{(
[
@@ -1246,6 +1248,11 @@ title="用当前镜头重新拼接"
{shot.video_url ? '重生视频' : '出视频'}
</button>
) : null}
+ {shot.video_url ? (
+ <button type="button" className="op" onClick={() => setPreview({ kind: 'shot', shotNo: shot.shot_no, imageUrl: shot.image_url, videoUrl: shot.video_url, audioUrl: shot.audio_url, caption: shotCaption(shot) })}>
+ 预览视频
+ </button>
+ ) : null}
<button
type="button"
className="more"
@@ -1283,8 +1290,8 @@ title="用当前镜头重新拼接"
)}
<div className="pf-shot-footer">
<span className="pf-muted" style={{ fontSize: '0.82rem' }}>
- 总时长: {formatMmSs(totalDuration)} | 画面: {project.shots.length} 个 | 音频:{' '}
- {project.shots.filter((s) => s.audio_url).length} 段 | 分辨率: 预览
+ 总时长: {formatMmSs(totalDuration)} | 画面: {project.shots.filter((s) => s.image_url).length}/{project.shots.length} 个 | 视频:{' '}
+ {project.shots.filter((s) => s.video_url).length}/{project.shots.length} 段 | 清晰度: {project.video_resolution || '默认'}
</span>
<div className="pf-toolbar">
<button
diff --git a/frontend/src/pages/studio/shortVideoCreate.css b/frontend/src/pages/studio/shortVideoCreate.css
index 90bc4a8..87f82c3 100644
--- a/frontend/src/pages/studio/shortVideoCreate.css
+++ b/frontend/src/pages/studio/shortVideoCreate.css
@@ -35,6 +35,20 @@
.sv-actions > span, .sv-saved-note { font-size: 12px; color: #929c87; line-height: 1.6; }
.sv-settings { position: sticky; top: 20px; padding: 24px; }
.sv-settings > p { margin-bottom: 24px; }
+.sv-output-modes { display: grid; gap: 8px; }
+.sv-output-modes button { position: relative; border: 1px solid #dfe5d9; border-radius: 10px; background: #fbfcf9; padding: 12px 30px 12px 12px; text-align: left; color: #293328; cursor: pointer; }
+.sv-output-modes button.is-selected { border-color: #93bf36; background: #f8fcee; box-shadow: 0 0 0 2px #c9ed844a; }
+.sv-output-modes strong, .sv-output-modes span { display: block; }
+.sv-output-modes strong { font-size: 13px; }
+.sv-output-modes span { color: #7a867c; font-size: 11px; margin-top: 5px; line-height: 1.6; }
+.sv-output-modes svg { position: absolute; right: 12px; top: 13px; color: #607e22; }
+.sv-model-settings { border-top: 1px solid #eaf0e0; margin-top: 22px; padding-top: 18px; }
+.sv-model-settings summary { font-size: 13px; font-weight: 600; cursor: pointer; margin-bottom: 18px; }
+.sv-model-settings .sv-field { margin-bottom: 16px; }
+.sv-model-settings .media-cost-notice { font-size: 11px; font-weight: 400; line-height: 1.7; color: #7a867c; }
+.sv-model-error { color: #a76232; font-size: 12px; line-height: 1.7; }
+.sv-output-modes button:focus-visible, .sv-model-settings summary:focus-visible { outline: 2px solid #a6c95e; outline-offset: 3px; }
+.sv-output-modes button:disabled { cursor: default; opacity: .6; }
.sv-setting-summary, .sv-workflow-hint { border-top: 1px solid #eaf0e0; padding-top: 20px; display: flex; gap: 10px; color: #697956; margin-top: 24px; }
.sv-setting-summary strong, .sv-setting-summary span { display: block; }
.sv-setting-summary strong { font-size: 13px; }
diff --git a/frontend/tests/short-video-settings.test.mjs b/frontend/tests/short-video-settings.test.mjs
new file mode 100644
index 0000000..9231144
--- /dev/null
+++ b/frontend/tests/short-video-settings.test.mjs
@@ -0,0 +1,93 @@
+import assert from 'node:assert/strict'
+import test from 'node:test'
+import { resolveShortVideoMediaSettings, shortVideoPipelineMode } from '../src/lib/shortVideoSettings.ts'
+
+const image = { id: 'image-ref', label: 'Reference images', available: true, reference_limits: { images: 4 }, allowed_aspect_ratios: ['9:16', '16:9', '1:1'], available_resolutions: ['2K', '4K'], defaults: { resolution: '2K' } }
+const video = { id: 'video-ref', label: 'Image to video', available: true, reference_limits: { images: 2 }, allowed_aspect_ratios: ['16:9', '9:16'], available_resolutions: ['720p', '1080p'], defaults: { resolution: '720p' } }
+const catalog = { image_models: [image], video_models: [video], defaults: { image_model: image.id, video_model: video.id } }
+const settings = { imageModel: image.id, videoModel: video.id, imageResolution: '2K', videoResolution: '720p', ratio: '9:16' }
+
+test('new short videos default to image-to-video while explicit still-image projects retain their mode', () => {
+ assert.equal(shortVideoPipelineMode(null), 'full')
+ assert.equal(shortVideoPipelineMode('full'), 'full')
+ assert.equal(shortVideoPipelineMode('image_text'), 'image_text')
+ assert.equal(shortVideoPipelineMode('unknown'), 'full')
+})
+
+test('video settings require reference-capable models and a shared supported aspect ratio', () => {
+ const result = resolveShortVideoMediaSettings(catalog, 'full', settings)
+ assert.equal(result.ready, true)
+ assert.deepEqual(result.ratios, ['9:16', '16:9'])
+ assert.equal(result.imageModel, image.id)
+ assert.equal(result.videoModel, video.id)
+ assert.equal(result.ratio, '9:16')
+})
+
+test('new projects choose compatible defaults instead of unavailable or text-only models', () => {
+ const textOnly = { ...image, id: 'text-only', reference_limits: { images: 0 } }
+ const unavailable = { ...video, id: 'offline', available: false }
+ const result = resolveShortVideoMediaSettings({ ...catalog, image_models: [textOnly, image], video_models: [unavailable, video], defaults: { image_model: textOnly.id, video_model: unavailable.id } }, 'full', { ...settings, imageModel: '', videoModel: '' })
+ assert.equal(result.imageModel, image.id)
+ assert.equal(result.videoModel, video.id)
+ assert.deepEqual(result.imageOptions.map(model => model.id), [image.id])
+ assert.deepEqual(result.videoOptions.map(model => model.id), [video.id])
+ assert.equal(result.ready, true)
+})
+
+test('explicit unavailable or removed models remain selected and block generation until replaced', () => {
+ for (const imageModel of ['removed-image', 'offline-image', 'text-only']) {
+ const result = resolveShortVideoMediaSettings({ ...catalog, image_models: [image, { ...image, id: 'offline-image', available: false }, { ...image, id: 'text-only', reference_limits: { images: 0 } }] }, 'full', { ...settings, imageModel, imageResolution: 'saved-quality' })
+ assert.equal(result.imageModel, imageModel)
+ assert.equal(result.imageResolution, 'saved-quality')
+ assert.equal(result.imageSelectionUnavailable, true)
+ assert.equal(result.ready, false)
+ }
+ const result = resolveShortVideoMediaSettings(catalog, 'full', { ...settings, videoModel: 'removed-video', videoResolution: 'saved-video-quality' })
+ assert.equal(result.videoModel, 'removed-video')
+ assert.equal(result.videoResolution, 'saved-video-quality')
+ assert.equal(result.videoSelectionUnavailable, true)
+ assert.equal(result.ready, false)
+})
+
+test('continuity requires two image slots per model and reserves a third image slot for an existing subject reference', () => {
+ const capOne = { ...image, id: 'one-image-slot', reference_limits: { images: 1 } }
+ const capTwo = { ...image, id: 'two-image-slots', reference_limits: { images: 2 } }
+ const capOneVideo = { ...video, id: 'one-video-slot', reference_limits: { images: 1 } }
+ const selection = { ...settings, imageModel: '', videoModel: '' }
+ const result = resolveShortVideoMediaSettings({ ...catalog, image_models: [capOne, capTwo, image], video_models: [capOneVideo, video] }, 'full', selection)
+ assert.deepEqual(result.imageOptions.map(model => model.id), [capTwo.id, image.id])
+ assert.deepEqual(result.videoOptions.map(model => model.id), [video.id])
+ const subject = resolveShortVideoMediaSettings({ ...catalog, image_models: [capTwo, image] }, 'full', selection, 3)
+ assert.deepEqual(subject.imageOptions.map(model => model.id), [image.id])
+ assert.equal(subject.ready, true)
+ assert.equal(resolveShortVideoMediaSettings({ ...catalog, image_models: [capOne] }, 'image_text', selection).ready, false)
+})
+
+test('model changes replace unsupported resolution and aspect settings with supported defaults', () => {
+ const result = resolveShortVideoMediaSettings(catalog, 'full', { ...settings, imageResolution: '8K', videoResolution: '480p', ratio: '1:1' })
+ assert.equal(result.imageResolution, '2K')
+ assert.equal(result.videoResolution, '720p')
+ assert.equal(result.ratio, '9:16')
+})
+
+test('still-image projects do not need a video model and keep their wider set of image ratios', () => {
+ const result = resolveShortVideoMediaSettings({ ...catalog, video_models: [] }, 'image_text', { ...settings, ratio: '1:1' })
+ assert.equal(result.ready, true)
+ assert.equal(result.ratio, '1:1')
+ assert.equal(resolveShortVideoMediaSettings({ ...catalog, video_models: [] }, 'full', settings).ready, false)
+})
+
+test('missing catalogs preserve saved draft choices but cannot start media generation', () => {
+ const result = resolveShortVideoMediaSettings(null, 'full', settings)
+ assert.equal(result.ready, false)
+ for (const [key, value] of Object.entries(settings)) assert.equal(result[key], value)
+ assert.equal(resolveShortVideoMediaSettings({ ...catalog, image_models: [] }, 'full', settings).ready, false)
+})
+
+test('models with no shared ratio are blocked; legacy catalog reference limits remain compatible', () => {
+ const incompatible = resolveShortVideoMediaSettings({ ...catalog, video_models: [{ ...video, allowed_aspect_ratios: ['21:9'] }] }, 'full', settings)
+ assert.equal(incompatible.ready, false)
+ assert.equal(incompatible.ratio, '')
+ const legacy = resolveShortVideoMediaSettings({ ...catalog, image_models: [{ ...image, reference_limits: undefined }], video_models: [{ ...video, reference_limits: undefined }] }, 'full', settings)
+ assert.equal(legacy.ready, true)
+})