Skip to content

World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.

feat: bundle a native-node ACE-Step v1 workflow for comfyui_music

Ntsakocf1a13a722df
diff --git a/.agents/skills/comfyui/SKILL.md b/.agents/skills/comfyui/SKILL.md
index ea89cc3e..05bec8c2 100644
--- a/.agents/skills/comfyui/SKILL.md
+++ b/.agents/skills/comfyui/SKILL.md
@@ -58,7 +58,9 @@ Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_musi
## Music (`comfyui_music`)
-- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional.
-- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow.
-- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
-- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here.
+- Bundled default is ACE-Step v1 (3.5B) text-to-audio, built from ComfyUI's *native* `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` nodes (core, not a third-party pack) -- unlike ACE-Step 1.5 or other custom node packs, v1's interface is standardized enough to bundle safely.
+- `prompt` maps to the bundled workflow's `tags` field (style/genre/mood, e.g. `"upbeat electronic pop, female vocals"`), matching the same "prompt = music description" convention `suno_music` uses. `lyrics` is a separate optional field -- leave empty for instrumental, or use `[verse]`/`[chorus]`/`[bridge]` structure tags and `[zh]`/`[ja]`/`[ko]`-style language-code prefixes for non-English lines.
+- `duration_seconds`, `steps`, `cfg`, `lyrics_strength`, and `seed` are patchable on the bundled workflow. Missing `ace_step_v1_3.5b.safetensors` surfaces through the same `data.missing_models[]` contract as image/video.
+- Need ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model? Fall back to `workflow_json`/`workflow_path` + `output_node`, exactly like a custom image/video workflow -- in that mode `prompt` becomes provenance/logging only again and must already be baked into the graph.
+- `output_node` (bundled or custom) should be the node that writes the final audio -- the bundled workflow's is `SaveAudioMP3`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
+- For custom workflows, provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow.
diff --git a/docs/comfyui-adapter-plan.md b/docs/comfyui-adapter-plan.md
index c3d3884f..0e29d601 100644
--- a/docs/comfyui-adapter-plan.md
+++ b/docs/comfyui-adapter-plan.md
@@ -296,24 +296,33 @@ not promote ComfyUI for an operation whose bundled models are missing.
---
-### `comfyui_music` -- Music Generation (shipped, custom-workflow-only)
+### `comfyui_music` -- Music Generation (shipped, with a native-node bundled workflow)
`tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`.
-Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation
-described below is real and unsolved, so instead of picking one pack and
-breaking for everyone else, the tool always requires a caller-supplied
-`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video
-tools' *optional* override path, just mandatory here. `prompt` is accepted
-for provenance/logging only and is never injected into the workflow --
-tags/lyrics must already be baked into the graph before calling, same
-convention as image/video custom workflows.
-
-Originally not shipped because: the ComfyUI node interface for ACE-Step is
-not standardized -- there are multiple custom node packs with different
-class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.).
-Shipping a workflow that only works with one specific custom node pack would
-break for most users. The custom-workflow-only design sidesteps this
-entirely: whichever node pack is installed, the caller exports it themselves.
+
+**Bundled default:** ACE-Step v1 (3.5B) text-to-audio, via `tools/_comfyui/workflows/ace-step-1-t2a.json`.
+The node-pack fragmentation that originally blocked this tool (`AceStepModelLoader`
+vs native `TextEncodeAceStepAudio`, etc.) turned out to be moot for ACE-Step v1:
+ComfyUI ships `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` as **native core
+nodes** (`comfy_extras/nodes_ace.py`), not a third-party pack, and Comfy-Org's own
+[`workflow_templates`](https://github.com/Comfy-Org/workflow_templates) repo bundles
+an official ACE-Step-v1 template built entirely from those native nodes plus
+long-stable core nodes (`CheckpointLoaderSimple`, `KSampler`, `ModelSamplingSD3`,
+`VAEDecodeAudio`, `SaveAudioMP3`). Every node's `class_type` and input names in
+`ace-step-1-t2a.json` were cross-checked against ComfyUI's own source
+(`comfy_extras/nodes_ace.py`, `nodes_audio.py`, `nodes_latent.py`, `nodes.py`) --
+not guessed from the UI export -- since the UI-format template Comfy-Org ships
+isn't directly usable as the API-format JSON this client submits.
+
+`prompt` maps to ACE-Step's `tags` field (style/genre/mood description, matching
+the "prompt = description of desired music" convention `suno_music` already uses).
+`lyrics` is a separate optional field (empty for instrumental). `duration_seconds`,
+`steps`, `cfg`, `lyrics_strength`, and `seed` are all patchable; `shift` and the
+tonemap `multiplier` stay at the official template's defaults.
+
+Newer/different setups aren't locked out: `workflow_json`/`workflow_path` +
+`output_node` still works exactly like the image/video tools' override path --
+for ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model entirely.
**Selector integration:** no dedicated `music_selector` exists in OpenMontage
(unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are
@@ -323,17 +332,16 @@ do. `fallback_tools = ["suno_music", "music_gen"]`.
**Audio artifact schema:** `ToolResult.data` follows the same shape as the
image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`),
-plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded
-file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no
-other reliable way to know actual output duration ahead of time.
+plus `lyrics` and `duration_seconds` -- the latter a best-effort `ffprobe` probe
+of the downloaded file (`None` if `ffprobe` isn't on PATH), since even the bundled
+workflow doesn't report actual rendered duration back through `/history`.
**Workflow/output-node contract:** identical to image/video -- `output_node`
-must be the ID of the node that writes the final artifact (typically ComfyUI's
-native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now
-also checks the `"audio"` output key (previously only `"images"`/`"gifs"`),
-which is what `SaveAudio` writes to in ComfyUI's `/history` response --
-this is the one part of the contract that *is* standardized regardless of
-which ACE-Step loader pack sits upstream of it.
+must be the ID of the node that writes the final artifact (the bundled workflow's
+is `SaveAudioMP3`, ComfyUI's native audio saver). `ComfyUIClient.generate()`'s
+artifact extraction now also checks the `"audio"` output key (previously only
+`"images"`/`"gifs"`), which is what `SaveAudioMP3`/`SaveAudio` write to in
+ComfyUI's `/history` response.
---
@@ -535,11 +543,12 @@ pipeline definition, or any schema.
override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents
it for the setup-offer surfacing in `provider_menu()`.
-4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.**
+4. ~~**Music generation:**~~ **Resolved -- shipped with a bundled ACE-Step v1 workflow.**
`comfyui_music` is a real tool now (not a hidden image/video override), routed
through the existing `registry.get_by_capability("music_generation")` path
- like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack
- fragmentation that originally blocked this is real, so the tool always
- requires caller-supplied `workflow_json`/`workflow_path` + `output_node`
- rather than betting on one pack. See the `comfyui_music` section above for
- the artifact schema and workflow/output-node contract.
+ like `suno_music`/`music_gen`. The node-pack fragmentation that originally
+ blocked this turned out not to apply to ACE-Step v1: its ComfyUI nodes are
+ native core nodes, not a third-party pack, so `ace-step-1-t2a.json` ships as
+ the default, verified node-by-node against ComfyUI's own source. Custom
+ `workflow_json`/`workflow_path` + `output_node` remains available for other
+ versions/packs. See the `comfyui_music` section above for the full contract.
diff --git a/tests/contracts/test_comfyui_tools.py b/tests/contracts/test_comfyui_tools.py
index 27e32f7c..ac533f30 100644
--- a/tests/contracts/test_comfyui_tools.py
+++ b/tests/contracts/test_comfyui_tools.py
@@ -137,6 +137,7 @@ EXPECTED_WORKFLOWS = [
"flux2-txt2img.json",
"wan22-i2v-4step.json",
"wan22-t2v-4step.json",
+ "ace-step-1-t2a.json",
]
@@ -651,6 +652,11 @@ class TestModelRequirements:
assert len(_REQUIRED_MODELS_T2V) > 0
assert any("t2v" in m.lower() for m in _REQUIRED_MODELS_T2V)
+ def test_music_tool_has_required_models(self):
+ from tools.audio.comfyui_music import _REQUIRED_MODELS
+ assert len(_REQUIRED_MODELS) > 0
+ assert any("ace_step" in m.lower() for m in _REQUIRED_MODELS)
+
# ------------------------------------------------------------------
# Custom workflow contract and provenance
@@ -863,16 +869,23 @@ class TestComfyUIMusic:
assert tool.capability == "music_generation"
assert tool.provider == "comfyui"
- def test_requires_workflow_json_or_path(self):
+ def test_bundled_path_requires_no_workflow_json_or_output_node(self, tmp_path):
+ """Without workflow_json/workflow_path it should attempt the bundled
+ ACE-Step workflow, not demand a custom one."""
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
+ tool._client.check_models = lambda required: (list(required), [])
+ tool._client.generate = lambda workflow, output_node, dest, **kwargs: [Path(dest)]
- result = tool.execute({"prompt": "ambient pad", "output_node": "9"})
+ result = tool.execute({
+ "prompt": "ambient pad",
+ "output_path": str(tmp_path / "music.mp3"),
+ })
- assert result.success is False
- assert "workflow_json" in result.error or "workflow_path" in result.error
+ assert result.success is True
+ assert result.data["workflow_provenance"]["source"] == "bundled"
- def test_requires_output_node(self):
+ def test_custom_workflow_without_output_node_errors(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
@@ -884,6 +897,52 @@ class TestComfyUIMusic:
assert result.success is False
assert "output_node" in result.error
+ def test_bundled_missing_models_returns_structured_payload(self):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+ tool._client.check_models = lambda required: ([], list(required))
+
+ result = tool.execute({"prompt": "ambient pad"})
+
+ assert result.success is False
+ assert result.data["missing_models"][0]["name"] == "ace_step_v1_3.5b.safetensors"
+ assert result.data["missing_models"][0]["download_url"]
+
+ def test_bundled_generation_patches_tags_lyrics_and_seed(self, tmp_path):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+ tool._client.check_models = lambda required: (list(required), [])
+ seen = {}
+
+ def fake_generate(workflow, output_node, dest, **kwargs):
+ seen["workflow"] = workflow
+ seen["output_node"] = output_node
+ return [Path(dest)]
+
+ tool._client.generate = fake_generate
+
+ result = tool.execute({
+ "prompt": "lofi hip hop, chill, rain sounds",
+ "lyrics": "[verse]\nquiet streets",
+ "duration_seconds": 45,
+ "seed": 777,
+ "output_path": str(tmp_path / "music.mp3"),
+ })
+
+ assert result.success is True
+ assert seen["output_node"] == "10"
+ assert seen["workflow"]["2"]["inputs"]["tags"] == "lofi hip hop, chill, rain sounds"
+ assert seen["workflow"]["2"]["inputs"]["lyrics"] == "[verse]\nquiet streets"
+ assert seen["workflow"]["4"]["inputs"]["seconds"] == 45
+ assert seen["workflow"]["8"]["inputs"]["seed"] == 777
+ assert result.data["model"] == "ace-step-v1-3.5b"
+
+ def test_get_status_degraded_when_model_missing(self):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+ tool._client.check_models = lambda required: ([], list(required))
+ assert tool.get_status() == ToolStatus.DEGRADED
+
def test_unavailable_server_reports_unavailable_reason(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: False
diff --git a/tools/_comfyui/metadata.py b/tools/_comfyui/metadata.py
index 3c0ee13b..084fbeb3 100644
--- a/tools/_comfyui/metadata.py
+++ b/tools/_comfyui/metadata.py
@@ -184,6 +184,17 @@ BUNDLED_MODEL_STACKS: dict[str, list[dict[str, Any]]] = {
),
},
],
+ "ace-step-1-t2a": [
+ {
+ "role": "checkpoint",
+ "name": "ace_step_v1_3.5b.safetensors",
+ "destination_hint": "ComfyUI/models/checkpoints/",
+ "download_url": (
+ "https://huggingface.co/Comfy-Org/ACE-Step_ComfyUI_repackaged/"
+ "blob/main/all_in_one/ace_step_v1_3.5b.safetensors"
+ ),
+ },
+ ],
}
diff --git a/tools/_comfyui/workflows/ace-step-1-t2a.json b/tools/_comfyui/workflows/ace-step-1-t2a.json
new file mode 100644
index 00000000..fdeba405
--- /dev/null
+++ b/tools/_comfyui/workflows/ace-step-1-t2a.json
@@ -0,0 +1,80 @@
+{
+ "1": {
+ "class_type": "CheckpointLoaderSimple",
+ "inputs": {
+ "ckpt_name": "ace_step_v1_3.5b.safetensors"
+ }
+ },
+ "2": {
+ "class_type": "TextEncodeAceStepAudio",
+ "inputs": {
+ "clip": ["1", 1],
+ "tags": "",
+ "lyrics": "",
+ "lyrics_strength": 0.99
+ }
+ },
+ "3": {
+ "class_type": "ConditioningZeroOut",
+ "inputs": {
+ "conditioning": ["2", 0]
+ }
+ },
+ "4": {
+ "class_type": "EmptyAceStepLatentAudio",
+ "inputs": {
+ "seconds": 120,
+ "batch_size": 1
+ }
+ },
+ "5": {
+ "class_type": "ModelSamplingSD3",
+ "inputs": {
+ "model": ["1", 0],
+ "shift": 5.0
+ }
+ },
+ "6": {
+ "class_type": "LatentOperationTonemapReinhard",
+ "inputs": {
+ "multiplier": 1.0
+ }
+ },
+ "7": {
+ "class_type": "LatentApplyOperationCFG",
+ "inputs": {
+ "model": ["5", 0],
+ "operation": ["6", 0]
+ }
+ },
+ "8": {
+ "class_type": "KSampler",
+ "inputs": {
+ "model": ["7", 0],
+ "positive": ["2", 0],
+ "negative": ["3", 0],
+ "latent_image": ["4", 0],
+ "seed": 0,
+ "steps": 50,
+ "cfg": 5.0,
+ "sampler_name": "euler",
+ "scheduler": "simple",
+ "denoise": 1.0
+ }
+ },
+ "9": {
+ "class_type": "VAEDecodeAudio",
+ "inputs": {
+ "samples": ["8", 0],
+ "vae": ["1", 2]
+ }
+ },
+ "10": {
+ "class_type": "SaveAudioMP3",
+ "inputs": {
+ "audio": ["9", 0],
+ "filename_prefix": "openmontage",
+ "quality": "V0"
+ }
+ }
+}
diff --git a/tools/audio/comfyui_music.py b/tools/audio/comfyui_music.py
index 1d006f77..9751ad46 100644
--- a/tools/audio/comfyui_music.py
+++ b/tools/audio/comfyui_music.py
@@ -1,13 +1,11 @@
"""ComfyUI music generation via a local or remote ComfyUI server.
-No bundled workflow: ACE-Step's ComfyUI node interface is not standardized
-across custom node packs (``AceStepModelLoader`` vs native
-``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for
-most installs. This tool always runs a caller-supplied ``workflow_json`` or
-``workflow_path`` -- the same override contract ``comfyui_image``/
-``comfyui_video`` offer as an alternative to their bundled workflow, just
-mandatory here instead of optional. See the ``comfyui`` skill for how to
-convert a community ACE-Step workflow into a call.
+Default workflow: ACE-Step v1 (3.5B) text-to-audio using ComfyUI's native
+``TextEncodeAceStepAudio``/``EmptyAceStepLatentAudio`` nodes (built into
+ComfyUI core, not a third-party pack). Custom workflows are still accepted
+via ``workflow_json``/``workflow_path`` for other ACE-Step node packs, other
+versions (e.g. ACE-Step 1.5), or entirely different audio models -- the same
+override contract ``comfyui_image``/``comfyui_video`` offer.
"""
from __future__ import annotations
@@ -32,12 +30,23 @@ from tools.base_tool import (
ToolTier,
)
from tools._comfyui.client import ComfyUIClient, ComfyUIError
-from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash
+from tools._comfyui.metadata import (
+ BUNDLED_MODEL_STACKS,
+ COMFYUI_SETUP_OFFER,
+ missing_models_payload,
+ model_stack,
+ workflow_hash,
+)
+
+_WORKFLOWS = Path(__file__).resolve().parent.parent / "_comfyui" / "workflows"
+
+# Model required by the bundled ACE-Step v1 workflow
+_REQUIRED_MODELS = ["ace_step_v1_3.5b.safetensors"]
class ComfyUIMusic(BaseTool):
name = "comfyui_music"
- version = "0.1.0"
+ version = "0.2.0"
tier = ToolTier.GENERATE
capability = "music_generation"
provider = "comfyui"
@@ -49,10 +58,10 @@ class ComfyUIMusic(BaseTool):
dependencies = [] # checked at runtime via server health
setup_offer = COMFYUI_SETUP_OFFER
install_instructions = (
- "Start a ComfyUI server with ACE-Step installed (any node pack) and "
- "set COMFYUI_SERVER_URL (default http://localhost:8188).\n"
- "There is no bundled workflow for this tool -- export your ACE-Step "
- "graph in API format and pass it as workflow_json/workflow_path.\n"
+ "Start a ComfyUI server and set COMFYUI_SERVER_URL "
+ "(default http://localhost:8188).\n"
+ "Requires ace_step_v1_3.5b.safetensors in ComfyUI's checkpoints "
+ "directory for the bundled workflow.\n"
"Running a separate ComfyUI instance for music? Set "
"COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over "
"COMFYUI_SERVER_URL for this tool only."
@@ -62,59 +71,73 @@ class ComfyUIMusic(BaseTool):
capabilities = ["generate_background_music", "generate_song", "generate_instrumental"]
supports = {
"seed": True,
+ "lyrics": True,
"custom_workflow": True,
"custom_output_node": True,
"offline": True,
}
best_for = [
- "local GPU music generation without API costs, using whatever ACE-Step node pack is installed",
- "full control over sampling via custom ComfyUI workflows",
+ "local GPU music generation without API costs",
+ "instrumentals and songs with lyrics via the bundled ACE-Step v1 workflow",
+ "full control over sampling or other ACE-Step versions/node packs via custom ComfyUI workflows",
]
not_good_for = [
"setups without a running ComfyUI server",
- "quick generation without first exporting/adapting an ACE-Step workflow",
"CPU-only machines",
]
fallback_tools = ["suno_music", "music_gen"]
input_schema = {
"type": "object",
- "required": ["prompt", "output_node"],
+ "required": ["prompt"],
"properties": {
"prompt": {
"type": "string",
"description": (
- "Description of the desired music, for provenance/logging only. "
- "Not injected into the workflow -- bake the actual tags/lyrics "
- "into workflow_json/workflow_path before calling."
+ "Style/mood/genre description (ACE-Step 'tags'), e.g. "
+ "'upbeat electronic pop, female vocals, driving bassline'. "
+ "Comma-separated tags work best. Not injected for custom workflows."
+ ),
+ },
+ "lyrics": {
+ "type": "string",
+ "default": "",
+ "description": (
+ "Optional lyrics. Leave empty for instrumental. Supports structure "
+ "tags like [verse]/[chorus]/[bridge] and language-code prefixes "
+ "(e.g. [zh], [ja]) for non-English lines."
),
},
+ "duration_seconds": {"type": "number", "default": 120.0},
+ "steps": {"type": "integer", "default": 50},
+ "cfg": {"type": "number", "default": 5.0},
+ "lyrics_strength": {"type": "number", "default": 0.99},
"seed": {"type": "integer", "description": "Random if omitted"},
"output_path": {"type": "string", "description": "Where to save the audio"},
"workflow_json": {
"type": "string",
- "description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.",
+ "description": "Optional full ComfyUI workflow JSON. Requires output_node.",
},
"workflow_path": {
"type": "string",
- "description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.",
+ "description": "Optional path to a ComfyUI workflow JSON file. Requires output_node.",
},
"output_node": {
"type": "string",
- "description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.",
+ "description": "ComfyUI output node ID for custom workflow_json/workflow_path.",
},
"workflow_name": {
"type": "string",
- "description": "Optional human-readable provenance label for the workflow.",
+ "description": "Optional human-readable provenance label for a custom workflow.",
},
"workflow_model": {
"type": "string",
- "description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').",
+ "description": "Optional model/provenance label for a custom workflow.",
},
"workflow_model_stack": {
"type": "array",
"description": (
- "Optional provenance metadata for workflow dependencies. "
+ "Optional provenance metadata for custom workflow dependencies. "
"Items should include name, role, and node-pack origin when known."
),
"items": {"type": "object"},
@@ -134,7 +157,7 @@ class ComfyUIMusic(BaseTool):
cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False,
)
retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"])
- idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"]
+ idempotency_key_fields = ["prompt", "lyrics", "duration_seconds", "seed"]
side_effects = ["writes audio file to output_path"]
user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"]
@@ -145,21 +168,21 @@ class ComfyUIMusic(BaseTool):
def get_status(self) -> ToolStatus:
if not self._client.is_available():
return ToolStatus.UNAVAILABLE
+ _, missing = self._client.check_models(_REQUIRED_MODELS)
+ if missing:
+ return ToolStatus.DEGRADED
return ToolStatus.AVAILABLE
def estimate_cost(self, inputs: dict[str, Any]) -> float:
return 0.0
def estimate_runtime(self, inputs: dict[str, Any]) -> float:
- # Actual runtime depends entirely on the caller's custom workflow
- # (steps, duration, sampler); this is a conservative flat estimate.
- return 180.0
+ return float(inputs.get("steps", 50)) * 2.0
def get_info(self) -> dict[str, Any]:
info = super().get_info()
info["setup_offer"] = self.setup_offer
- info["bundled_workflow"] = None
- info["custom_workflow_required"] = True
+ info["bundled_model_stack"] = BUNDLED_MODEL_STACKS["ace-step-1-t2a"]
return info
def _log_progress(self, data: dict) -> None:
@@ -173,35 +196,63 @@ class ComfyUIMusic(BaseTool):
print(f"[comfyui_music] step {value}/{max_value}")
def execute(self, inputs: dict[str, Any]) -> ToolResult:
- if not (inputs.get("workflow_json") or inputs.get("workflow_path")):
+ custom_workflow = bool(inputs.get("workflow_json") or inputs.get("workflow_path"))
+ if custom_workflow and not inputs.get("output_node"):
return ToolResult(
success=False,
error=(
- "comfyui_music requires workflow_json or workflow_path -- there "
- "is no bundled default. ACE-Step's ComfyUI node interface isn't "
- "standardized across custom node packs, so a hardcoded template "
- "would break for most installs. Export the ACE-Step workflow "
- "you actually have installed (API format) and pass it in."
+ "Custom ComfyUI workflows require output_node so OpenMontage "
+ "knows which ComfyUI node to download artifacts from."
),
)
- if not inputs.get("output_node"):
- return ToolResult(
- success=False,
- error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.",
- )
-
if not self._client.is_available():
return ToolResult(success=False, error=self._client.unavailable_reason())
+ if not custom_workflow:
+ _, missing = self._client.check_models(_REQUIRED_MODELS)
+ if missing:
+ return ToolResult(
+ success=False,
+ data=missing_models_payload(
+ missing,
+ workflow_key="ace-step-1-t2a",
+ workflow_name="ace-step-1-t2a.json",
+ ),
+ error=(
+ f"ComfyUI server is running but missing required models: "
+ f"{', '.join(missing)}.\n"
+ f"See data.missing_models for destination hints and download URLs."
+ ),
+ )
+
start = time.time()
seed = inputs.get("seed") or ComfyUIClient.random_seed()
output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3"))
- output_node = str(inputs["output_node"])
try:
- workflow = self._load_custom_workflow(inputs)
- provenance = self._workflow_provenance(inputs, output_node, workflow)
+ if custom_workflow:
+ workflow = self._load_custom_workflow(inputs)
+ output_node = str(inputs["output_node"])
+ else:
+ workflow = ComfyUIClient.load_workflow(_WORKFLOWS / "ace-step-1-t2a.json")
+ workflow = ComfyUIClient.patch_workflow(workflow, {
+ "2": {
+ "tags": inputs["prompt"],
+ "lyrics": inputs.get("lyrics", ""),
+ "lyrics_strength": inputs.get("lyrics_strength", 0.99),
+ },
+ "4": {"seconds": inputs.get("duration_seconds", 120.0)},
+ "8": {
+ "seed": seed,
+ "steps": inputs.get("steps", 50),
+ "cfg": inputs.get("cfg", 5.0),
+ },
+ "10": {"filename_prefix": output_path.stem},
+ })
+ output_node = "10"
+
+ provenance = self._workflow_provenance(inputs, custom_workflow, output_node, workflow)
paths = self._client.generate(
workflow,
output_node=output_node,
@@ -229,13 +280,14 @@ class ComfyUIMusic(BaseTool):
return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}")
duration = self._probe_duration(paths[0])
- model_name = self._model_name(inputs)
+ model_name = self._model_name(inputs, custom_workflow)
return ToolResult(
success=True,
data={
"provider": "comfyui",
"model": model_name,
"prompt": inputs["prompt"],
+ "lyrics": inputs.get("lyrics", ""),
"duration_seconds": duration,
"output": str(paths[0]),
"format": paths[0].suffix.lstrip("."),
@@ -255,7 +307,9 @@ class ComfyUIMusic(BaseTool):
return ComfyUIClient.load_workflow(Path(inputs["workflow_path"]))
@staticmethod
- def _model_name(inputs: dict[str, Any]) -> str:
+ def _model_name(inputs: dict[str, Any], custom_workflow: bool) -> str:
+ if not custom_workflow:
+ return "ace-step-v1-3.5b"
return (
inputs.get("workflow_model")
or inputs.get("model")
@@ -265,8 +319,19 @@ class ComfyUIMusic(BaseTool):
@staticmethod
def _workflow_provenance(
- inputs: dict[str, Any], output_node: str, workflow: dict[str, Any]
+ inputs: dict[str, Any],
+ custom_workflow: bool,
+ output_node: str,
+ workflow: dict[str, Any],
) -> dict[str, Any]:
+ if not custom_workflow:
+ return {
+ "source": "bundled",
+ "workflow": "ace-step-1-t2a.json",
+ "workflow_hash_sha256": workflow_hash(workflow),
+ "model_stack": model_stack("ace-step-1-t2a", inputs),
+ "output_node": output_node,
+ }
stack = inputs.get("workflow_model_stack")
return {
"source": "user_supplied",