Skip to content

World's first open-source, agentic video production system. 12 production pipelines, 100+ tools, 700+ agent skill and production-knowledge files. Turn your AI coding assistant into a full video production studio.

feat: ship comfyui_music as a custom-workflow-only ACE-Step tool

Ntsako172acca6ea4e
diff --git a/.agents/skills/comfyui/SKILL.md b/.agents/skills/comfyui/SKILL.md
index 64950870..ea89cc3e 100644
--- a/.agents/skills/comfyui/SKILL.md
+++ b/.agents/skills/comfyui/SKILL.md
@@ -1,17 +1,19 @@
---
name: comfyui
-description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports.
+description: Use when working with ComfyUI workflows in OpenMontage, including comfyui_image/comfyui_video/comfyui_music, custom workflow_json/workflow_path inputs, output_node selection, missing model setup, LoRAs, low-VRAM workflow choices, and community workflow imports.
---
# ComfyUI Workflows in OpenMontage
-Use this skill before calling `comfyui_image` or `comfyui_video`, and when converting a community ComfyUI workflow into an OpenMontage tool call.
+Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_music`, and when converting a community ComfyUI workflow into an OpenMontage tool call.
## Server Contract
- ComfyUI must be running before the tool can generate. The default server is `http://localhost:8188`; override it with `COMFYUI_SERVER_URL`.
+- Running separate ComfyUI instances per capability (different GPU, different model set)? `COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL` each override `COMFYUI_SERVER_URL` for that one tool only. Optional -- a single-server setup needs none of these.
- Health and hardware status come from `GET /system_stats`.
- Jobs are submitted to `POST /prompt`, completed outputs are read from `GET /history/{prompt_id}`, and artifact bytes are downloaded with `GET /view`.
+- Long waits (video, music) prefer ComfyUI's websocket feed for immediate completion/error detection and transparently fall back to REST polling if `websocket-client` isn't installed. Either way, a timeout is recoverable: pass the error's `prompt_id` back in as `resume_prompt_id` to resume waiting on the same job instead of resubmitting it.
- Export workflows with ComfyUI's API-format JSON, not the UI layout format. If a downloaded workflow will not submit, re-export it from ComfyUI with API format enabled.
## Choosing a Workflow
@@ -52,4 +54,11 @@ Use this skill before calling `comfyui_image` or `comfyui_video`, and when conve
- If the server is unavailable, surface the structured setup offer. Starting ComfyUI or setting `COMFYUI_SERVER_URL` is the first fix.
- If models are missing, read `data.missing_models[]`; each item should include the file name, role, destination hint, and download URL when OpenMontage knows it.
- If custom nodes are missing, ask the user to install them through ComfyUI Manager or the workflow author's documented install path, then restart ComfyUI.
-- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt.
+- If a long render times out locally, check ComfyUI history before retrying from scratch; the server may still have completed the prompt -- or just call again with `resume_prompt_id` set to the `prompt_id` from the timeout error.
+
+## Music (`comfyui_music`)
+
+- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional.
+- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow.
+- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
+- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here.
diff --git a/docs/comfyui-adapter-plan.md b/docs/comfyui-adapter-plan.md
index c8eef2e3..c3d3884f 100644
--- a/docs/comfyui-adapter-plan.md
+++ b/docs/comfyui-adapter-plan.md
@@ -296,19 +296,44 @@ not promote ComfyUI for an operation whose bundled models are missing.
---
-### `comfyui_music` -- Music Generation (not shipped)
-
-We explored adding a `comfyui_music` tool using the ACE-Step 3.5B model.
-The model runs well in ComfyUI, but the ComfyUI node interface for
-ACE-Step is not standardized -- there are multiple custom node packs with
-different class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`,
-etc.). Shipping a workflow that only works with one specific custom node
-pack would break for most users.
-
-**Future path:** ACE-Step support should be revisited once OpenMontage decides
-the music-generation routing shape and a portable ComfyUI audio workflow
-contract. Current image/video workflow overrides are intentionally scoped to
-image and video artifacts, not arbitrary audio workflows.
+### `comfyui_music` -- Music Generation (shipped, custom-workflow-only)
+
+`tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`.
+Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation
+described below is real and unsolved, so instead of picking one pack and
+breaking for everyone else, the tool always requires a caller-supplied
+`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video
+tools' *optional* override path, just mandatory here. `prompt` is accepted
+for provenance/logging only and is never injected into the workflow --
+tags/lyrics must already be baked into the graph before calling, same
+convention as image/video custom workflows.
+
+Originally not shipped because: the ComfyUI node interface for ACE-Step is
+not standardized -- there are multiple custom node packs with different
+class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.).
+Shipping a workflow that only works with one specific custom node pack would
+break for most users. The custom-workflow-only design sidesteps this
+entirely: whichever node pack is installed, the caller exports it themselves.
+
+**Selector integration:** no dedicated `music_selector` exists in OpenMontage
+(unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are
+already routed directly via `registry.get_by_capability("music_generation")`,
+and `comfyui_music` participates in that the same way `suno_music`/`music_gen`
+do. `fallback_tools = ["suno_music", "music_gen"]`.
+
+**Audio artifact schema:** `ToolResult.data` follows the same shape as the
+image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`),
+plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded
+file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no
+other reliable way to know actual output duration ahead of time.
+
+**Workflow/output-node contract:** identical to image/video -- `output_node`
+must be the ID of the node that writes the final artifact (typically ComfyUI's
+native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now
+also checks the `"audio"` output key (previously only `"images"`/`"gifs"`),
+which is what `SaveAudio` writes to in ComfyUI's `/history` response --
+this is the one part of the contract that *is* standardized regardless of
+which ACE-Step loader pack sits upstream of it.
---
@@ -373,15 +398,17 @@ COMFYUI_POLL_TIMEOUT=600 # max wait for image gen
COMFYUI_VIDEO_TIMEOUT=900 # max wait for video gen
```
-**Multi-server (optional):** point `comfyui_image` and `comfyui_video` at
-separate ComfyUI instances -- e.g. one GPU running FLUX 2, another running
-WAN 2.2 -- by setting a per-capability override. Each takes priority over
-`COMFYUI_SERVER_URL` for its own tool only; leave both unset and everything
-still talks to the single shared server.
+**Multi-server (optional):** point `comfyui_image`, `comfyui_video`, and
+`comfyui_music` at separate ComfyUI instances -- e.g. one GPU running FLUX 2,
+another running WAN 2.2, another running ACE-Step -- by setting a
+per-capability override. Each takes priority over `COMFYUI_SERVER_URL` for
+its own tool only; leave all three unset and everything talks to the single
+shared server.
```bash
COMFYUI_IMAGE_SERVER_URL=http://gpu-a:8188
COMFYUI_VIDEO_SERVER_URL=http://gpu-b:8188
+COMFYUI_MUSIC_SERVER_URL=http://gpu-c:8188
```
**For Docker Compose setups** (ComfyUI in a container):
@@ -496,20 +523,23 @@ pipeline definition, or any schema.
`poll()` REST loop when it isn't installed or the connection fails —
`resume_prompt_id` recovery behaves identically either way.
-3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video")`
+3. ~~**Multi-server:**~~ **Resolved.** `ComfyUIClient(capability="image"|"video"|"music")`
resolves its server URL from a per-capability env var first
- (`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL`), then the shared
- `COMFYUI_SERVER_URL`, then the `http://localhost:8188` default. `comfyui_image`
- and `comfyui_video` pass their capability at construction, so image and video
- generation can point at different ComfyUI instances (different GPUs, different
- model sets) with zero code changes -- single-server setups need no extra
- configuration since both env vars are optional. `client.capability`/
+ (`COMFYUI_IMAGE_SERVER_URL` / `COMFYUI_VIDEO_SERVER_URL` / `COMFYUI_MUSIC_SERVER_URL`),
+ then the shared `COMFYUI_SERVER_URL`, then the `http://localhost:8188` default.
+ All three tools pass their capability at construction, so image, video, and
+ music generation can each point at different ComfyUI instances (different GPUs,
+ different model sets) with zero code changes -- single-server setups need no extra
+ configuration since all three env vars are optional. `client.capability`/
`client.is_default_url`/`client.unavailable_reason()` all account for the
override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents
it for the setup-offer surfacing in `provider_menu()`.
-4. **Music generation:** ACE-Step works in ComfyUI but OpenMontage needs a
- dedicated music-generation routing contract before adding `comfyui_music`.
- The follow-up should decide selector integration, audio artifact schemas, and
- a portable workflow/output-node contract rather than treating music as a
- hidden image/video workflow override.
+4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.**
+ `comfyui_music` is a real tool now (not a hidden image/video override), routed
+ through the existing `registry.get_by_capability("music_generation")` path
+ like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack
+ fragmentation that originally blocked this is real, so the tool always
+ requires caller-supplied `workflow_json`/`workflow_path` + `output_node`
+ rather than betting on one pack. See the `comfyui_music` section above for
+ the artifact schema and workflow/output-node contract.
diff --git a/tests/contracts/test_comfyui_tools.py b/tests/contracts/test_comfyui_tools.py
index 44c451c0..27e32f7c 100644
--- a/tests/contracts/test_comfyui_tools.py
+++ b/tests/contracts/test_comfyui_tools.py
@@ -17,13 +17,14 @@ from tools.base_tool import (
ToolStatus,
ToolTier,
)
+from tools.audio.comfyui_music import ComfyUIMusic
from tools.graphics.comfyui_image import ComfyUIImage
from tools.graphics.image_selector import ImageSelector
from tools.tool_registry import ToolRegistry
from tools.video.video_selector import VideoSelector
from tools.video.comfyui_video import ComfyUIVideo
-TOOLS = [ComfyUIImage, ComfyUIVideo]
+TOOLS = [ComfyUIImage, ComfyUIVideo, ComfyUIMusic]
WORKFLOW_DIR = Path(__file__).resolve().parent.parent.parent / "tools" / "_comfyui" / "workflows"
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
@@ -323,6 +324,23 @@ class TestClientHelpers:
)
assert paths == [tmp_path / "out.png"]
+ def test_generate_reads_audio_key_from_savaudio_node(self, monkeypatch, tmp_path):
+ """The native SaveAudio node writes outputs under "audio", not
+ "images"/"gifs" -- comfyui_music depends on this being handled."""
+ from tools._comfyui.client import ComfyUIClient
+
+ client = ComfyUIClient("http://comfy.test")
+ monkeypatch.setattr(client, "submit", lambda workflow: "p1")
+ monkeypatch.setattr(client, "poll", lambda prompt_id, **kwargs: {
+ "outputs": {"9": {"audio": [{
+ "filename": "track.flac", "subfolder": "", "type": "output",
+ }]}}
+ })
+ monkeypatch.setattr(client, "download", lambda filename, subfolder, dest, folder_type="output": Path(dest))
+
+ paths = client.generate({"9": {"inputs": {}}}, "9", tmp_path / "out.flac")
+ assert paths == [tmp_path / "out.flac"]
+
def test_is_default_url_when_env_not_set(self, monkeypatch):
from tools._comfyui.client import ComfyUIClient
monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False)
@@ -838,6 +856,142 @@ class TestCustomWorkflowContract:
assert any(item["role"] == "vae" for item in provenance["model_stack"])
+class TestComfyUIMusic:
+
+ def test_capability_and_provider(self):
+ tool = ComfyUIMusic()
+ assert tool.capability == "music_generation"
+ assert tool.provider == "comfyui"
+
+ def test_requires_workflow_json_or_path(self):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+
+ result = tool.execute({"prompt": "ambient pad", "output_node": "9"})
+
+ assert result.success is False
+ assert "workflow_json" in result.error or "workflow_path" in result.error
+
+ def test_requires_output_node(self):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+
+ result = tool.execute({
+ "prompt": "ambient pad",
+ "workflow_json": json.dumps({"9": {"inputs": {}}}),
+ })
+
+ assert result.success is False
+ assert "output_node" in result.error
+
+ def test_unavailable_server_reports_unavailable_reason(self):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: False
+ tool._client.unavailable_reason = lambda: "no server here"
+
+ result = tool.execute({
+ "prompt": "ambient pad",
+ "workflow_json": json.dumps({"9": {"inputs": {}}}),
+ "output_node": "9",
+ })
+
+ assert result.success is False
+ assert result.error == "no server here"
+
+ def test_successful_generation_returns_provenance_and_duration(self, tmp_path, monkeypatch):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+
+ dest_file = tmp_path / "music.mp3"
+
+ def fake_generate(workflow, output_node, dest, **kwargs):
+ Path(dest).write_bytes(b"fake-audio-bytes")
+ return [Path(dest)]
+
+ tool._client.generate = fake_generate
+ monkeypatch.setattr("shutil.which", lambda name: None) # no ffprobe in test env
+
+ result = tool.execute({
+ "prompt": "upbeat synthwave",
+ "workflow_json": json.dumps({"9": {"inputs": {}}}),
+ "output_node": "9",
+ "output_path": str(dest_file),
+ "workflow_name": "my-ace-step-graph",
+ "workflow_model": "ace-step-v1-3.5b",
+ })
+
+ assert result.success is True
+ assert result.data["provider"] == "comfyui"
+ assert result.data["model"] == "ace-step-v1-3.5b"
+ assert result.data["output"] == str(dest_file)
+ assert result.data["format"] == "mp3"
+ assert result.data["duration_seconds"] is None # ffprobe unavailable
+ provenance = result.data["workflow_provenance"]
+ assert provenance["source"] == "user_supplied"
+ assert provenance["output_node"] == "9"
+ assert provenance["workflow_hash_sha256"]
+
+ def test_timeout_surfaces_resumable_prompt_id(self, tmp_path):
+ from tools._comfyui.client import ComfyUIError
+
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+
+ def fake_generate(workflow, output_node, dest, **kwargs):
+ raise ComfyUIError("timed out", prompt_id="music-prompt-id")
+
+ tool._client.generate = fake_generate
+
+ result = tool.execute({
+ "prompt": "ambient pad",
+ "workflow_json": json.dumps({"9": {"inputs": {}}}),
+ "output_node": "9",
+ "output_path": str(tmp_path / "music.mp3"),
+ })
+
+ assert result.success is False
+ assert result.data["prompt_id"] == "music-prompt-id"
+ assert "resume_prompt_id" in result.error
+
+ def test_passes_timeout_and_resume_prompt_id_through(self, tmp_path):
+ tool = ComfyUIMusic()
+ tool._client.is_available = lambda: True
+ seen = {}
+
+ def fake_generate(workflow, output_node, dest, **kwargs):
+ seen.update(kwargs)
+ return [Path(dest)]
+
+ tool._client.generate = fake_generate
+
+ tool.execute({
+ "prompt": "ambient pad",
+ "workflow_json": json.dumps({"9": {"inputs": {}}}),
+ "output_node": "9",
+ "output_path": str(tmp_path / "music.mp3"),
+ "timeout_seconds": 3600,
+ "resume_prompt_id": "already-running-id",
+ })
+
+ assert seen["timeout"] == 3600
+ assert seen["resume_prompt_id"] == "already-running-id"
+
+ def test_registry_discovers_comfyui_music_under_music_generation(self):
+ registry = ToolRegistry()
+ tool = ComfyUIMusic()
+ registry.register(tool)
+ registry._discovered_packages.add("tools")
+
+ by_capability = registry.get_by_capability("music_generation")
+ assert any(t.name == "comfyui_music" for t in by_capability)
+
+ def test_uses_music_capability_env_var_for_multi_server(self, monkeypatch):
+ monkeypatch.delenv("COMFYUI_SERVER_URL", raising=False)
+ monkeypatch.setenv("COMFYUI_MUSIC_SERVER_URL", "http://music-gpu:8188")
+ tool = ComfyUIMusic()
+ assert tool._client.server_url == "http://music-gpu:8188"
+
+
class TestComfyUISetupOffer:
def test_provider_menu_summary_includes_structured_setup_offer(self):
diff --git a/tools/_comfyui/client.py b/tools/_comfyui/client.py
index 6632bfa6..e280bea6 100644
--- a/tools/_comfyui/client.py
+++ b/tools/_comfyui/client.py
@@ -404,8 +404,13 @@ class ComfyUIClient:
outputs = entry.get("outputs", {})
node_output = outputs.get(output_node, {})
- # ComfyUI stores images and videos under the "images" key
- items = node_output.get("images", []) or node_output.get("gifs", [])
+ # ComfyUI stores images/video frames under "images", legacy GIFs
+ # under "gifs", and the native SaveAudio node's output under "audio".
+ items = (
+ node_output.get("images", [])
+ or node_output.get("gifs", [])
+ or node_output.get("audio", [])
+ )
if not items:
raise ComfyUIError(
f"No output artifacts on node {output_node}. "
diff --git a/tools/_comfyui/metadata.py b/tools/_comfyui/metadata.py
index 75707868..3c0ee13b 100644
--- a/tools/_comfyui/metadata.py
+++ b/tools/_comfyui/metadata.py
@@ -24,6 +24,7 @@ COMFYUI_SETUP_OFFER: dict[str, Any] = {
"per_capability_env_var_overrides": {
"comfyui_image": "COMFYUI_IMAGE_SERVER_URL",
"comfyui_video": "COMFYUI_VIDEO_SERVER_URL",
+ "comfyui_music": "COMFYUI_MUSIC_SERVER_URL",
},
}
diff --git a/tools/audio/comfyui_music.py b/tools/audio/comfyui_music.py
new file mode 100644
index 00000000..1d006f77
--- /dev/null
+++ b/tools/audio/comfyui_music.py
@@ -0,0 +1,300 @@
+"""ComfyUI music generation via a local or remote ComfyUI server.
+
+No bundled workflow: ACE-Step's ComfyUI node interface is not standardized
+across custom node packs (``AceStepModelLoader`` vs native
+``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for
+most installs. This tool always runs a caller-supplied ``workflow_json`` or
+``workflow_path`` -- the same override contract ``comfyui_image``/
+``comfyui_video`` offer as an alternative to their bundled workflow, just
+mandatory here instead of optional. See the ``comfyui`` skill for how to
+convert a community ACE-Step workflow into a call.
+"""
+
+from __future__ import annotations
+
+import json
+import shutil
+import subprocess
+import time
+from pathlib import Path
+from typing import Any
+
+from tools.base_tool import (
+ BaseTool,
+ Determinism,
+ ExecutionMode,
+ ResourceProfile,
+ RetryPolicy,
+ ToolResult,
+ ToolRuntime,
+ ToolStability,
+ ToolStatus,
+ ToolTier,
+)
+from tools._comfyui.client import ComfyUIClient, ComfyUIError
+from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash
+
+
+class ComfyUIMusic(BaseTool):
+ name = "comfyui_music"
+ version = "0.1.0"
+ tier = ToolTier.GENERATE
+ capability = "music_generation"
+ provider = "comfyui"
+ stability = ToolStability.EXPERIMENTAL
+ execution_mode = ExecutionMode.SYNC
+ determinism = Determinism.SEEDED
+ runtime = ToolRuntime.LOCAL_GPU
+
+ dependencies = [] # checked at runtime via server health
+ setup_offer = COMFYUI_SETUP_OFFER
+ install_instructions = (
+ "Start a ComfyUI server with ACE-Step installed (any node pack) and "
+ "set COMFYUI_SERVER_URL (default http://localhost:8188).\n"
+ "There is no bundled workflow for this tool -- export your ACE-Step "
+ "graph in API format and pass it as workflow_json/workflow_path.\n"
+ "Running a separate ComfyUI instance for music? Set "
+ "COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over "
+ "COMFYUI_SERVER_URL for this tool only."
+ )
+ agent_skills = ["comfyui"]
+
+ capabilities = ["generate_background_music", "generate_song", "generate_instrumental"]
+ supports = {
+ "seed": True,
+ "custom_workflow": True,
+ "custom_output_node": True,
+ "offline": True,
+ }
+ best_for = [
+ "local GPU music generation without API costs, using whatever ACE-Step node pack is installed",
+ "full control over sampling via custom ComfyUI workflows",
+ ]
+ not_good_for = [
+ "setups without a running ComfyUI server",
+ "quick generation without first exporting/adapting an ACE-Step workflow",
+ "CPU-only machines",
+ ]
+ fallback_tools = ["suno_music", "music_gen"]
+
+ input_schema = {
+ "type": "object",
+ "required": ["prompt", "output_node"],
+ "properties": {
+ "prompt": {
+ "type": "string",
+ "description": (
+ "Description of the desired music, for provenance/logging only. "
+ "Not injected into the workflow -- bake the actual tags/lyrics "
+ "into workflow_json/workflow_path before calling."
+ ),
+ },
+ "seed": {"type": "integer", "description": "Random if omitted"},
+ "output_path": {"type": "string", "description": "Where to save the audio"},
+ "workflow_json": {
+ "type": "string",
+ "description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.",
+ },
+ "workflow_path": {
+ "type": "string",
+ "description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.",
+ },
+ "output_node": {
+ "type": "string",
+ "description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.",
+ },
+ "workflow_name": {
+ "type": "string",
+ "description": "Optional human-readable provenance label for the workflow.",
+ },
+ "workflow_model": {
+ "type": "string",
+ "description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').",
+ },
+ "workflow_model_stack": {
+ "type": "array",
+ "description": (
+ "Optional provenance metadata for workflow dependencies. "
+ "Items should include name, role, and node-pack origin when known."
+ ),
+ "items": {"type": "object"},
+ },
+ "timeout_seconds": {
+ "type": "integer",
+ "description": "How long to wait for the ComfyUI job before giving up. Default 1800s (30min).",
+ },
+ "resume_prompt_id": {
+ "type": "string",
+ "description": "A prompt_id from a previous timed-out call. Skips resubmission and resumes waiting/downloading.",
+ },
+ },
+ }
+
+ resource_profile = ResourceProfile(
+ cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False,
+ )
+ retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"])
+ idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"]
+ side_effects = ["writes audio file to output_path"]
+ user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"]
+
+ def __init__(self) -> None:
+ self._client = ComfyUIClient(capability="music")
+ self._last_progress_log = 0.0
+
+ def get_status(self) -> ToolStatus:
+ if not self._client.is_available():
+ return ToolStatus.UNAVAILABLE
+ return ToolStatus.AVAILABLE
+
+ def estimate_cost(self, inputs: dict[str, Any]) -> float:
+ return 0.0
+
+ def estimate_runtime(self, inputs: dict[str, Any]) -> float:
+ # Actual runtime depends entirely on the caller's custom workflow
+ # (steps, duration, sampler); this is a conservative flat estimate.
+ return 180.0
+
+ def get_info(self) -> dict[str, Any]:
+ info = super().get_info()
+ info["setup_offer"] = self.setup_offer
+ info["bundled_workflow"] = None
+ info["custom_workflow_required"] = True
+ return info
+
+ def _log_progress(self, data: dict) -> None:
+ """Throttled progress line (see comfyui_video for rationale)."""
+ now = time.monotonic()
+ if now - self._last_progress_log < 10:
+ return
+ self._last_progress_log = now
+ value, max_value = data.get("value"), data.get("max")
+ if value is not None and max_value:
+ print(f"[comfyui_music] step {value}/{max_value}")
+
+ def execute(self, inputs: dict[str, Any]) -> ToolResult:
+ if not (inputs.get("workflow_json") or inputs.get("workflow_path")):
+ return ToolResult(
+ success=False,
+ error=(
+ "comfyui_music requires workflow_json or workflow_path -- there "
+ "is no bundled default. ACE-Step's ComfyUI node interface isn't "
+ "standardized across custom node packs, so a hardcoded template "
+ "would break for most installs. Export the ACE-Step workflow "
+ "you actually have installed (API format) and pass it in."
+ ),
+ )
+
+ if not inputs.get("output_node"):
+ return ToolResult(
+ success=False,
+ error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.",
+ )
+
+ if not self._client.is_available():
+ return ToolResult(success=False, error=self._client.unavailable_reason())
+
+ start = time.time()
+ seed = inputs.get("seed") or ComfyUIClient.random_seed()
+ output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3"))
+ output_node = str(inputs["output_node"])
+
+ try:
+ workflow = self._load_custom_workflow(inputs)
+ provenance = self._workflow_provenance(inputs, output_node, workflow)
+ paths = self._client.generate(
+ workflow,
+ output_node=output_node,
+ dest=output_path,
+ timeout=inputs.get("timeout_seconds", 1800),
+ interval=10,
+ resume_prompt_id=inputs.get("resume_prompt_id"),
+ on_progress=self._log_progress,
+ )
+
+ except ComfyUIError as exc:
+ data = {"prompt_id": exc.prompt_id} if exc.prompt_id else {}
+ if exc.prompt_id:
+ error_msg = (
+ f"{exc}\n\nThis job was NOT cancelled and is very likely still "
+ f"running server-side. To recover it without resubmitting, call "
+ f"execute() again with resume_prompt_id={exc.prompt_id!r} "
+ f"(and a longer timeout_seconds if it needs more time), or poll "
+ f"GET {{COMFYUI_SERVER_URL}}/history/{exc.prompt_id} directly."
+ )
+ else:
+ error_msg = str(exc)
+ return ToolResult(success=False, error=error_msg, data=data)
+ except Exception as exc:
+ return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}")
+
+ duration = self._probe_duration(paths[0])
+ model_name = self._model_name(inputs)
+ return ToolResult(
+ success=True,
+ data={
+ "provider": "comfyui",
+ "model": model_name,
+ "prompt": inputs["prompt"],
+ "duration_seconds": duration,
+ "output": str(paths[0]),
+ "format": paths[0].suffix.lstrip("."),
+ "workflow_provenance": provenance,
+ },
+ artifacts=[str(p) for p in paths],
+ cost_usd=0.0,
+ duration_seconds=round(time.time() - start, 2),
+ seed=seed,
+ model=model_name,
+ )
+
+ @staticmethod
+ def _load_custom_workflow(inputs: dict[str, Any]) -> dict:
+ if inputs.get("workflow_json"):
+ return json.loads(inputs["workflow_json"])
+ return ComfyUIClient.load_workflow(Path(inputs["workflow_path"]))
+
+ @staticmethod
+ def _model_name(inputs: dict[str, Any]) -> str:
+ return (
+ inputs.get("workflow_model")
+ or inputs.get("model")
+ or inputs.get("workflow_name")
+ or "custom-comfyui-workflow"
+ )
+
+ @staticmethod
+ def _workflow_provenance(
+ inputs: dict[str, Any], output_node: str, workflow: dict[str, Any]
+ ) -> dict[str, Any]:
+ stack = inputs.get("workflow_model_stack")
+ return {
+ "source": "user_supplied",
+ "workflow_name": inputs.get("workflow_name"),
+ "workflow_path": inputs.get("workflow_path"),
+ "model": inputs.get("workflow_model") or inputs.get("model"),
+ "workflow_hash_sha256": workflow_hash(workflow),
+ "model_stack": stack if isinstance(stack, list) else [],
+ "model_stack_source": "caller_supplied" if stack else "unknown_custom_workflow",
+ "output_node": output_node,
+ }
+
+ @staticmethod
+ def _probe_duration(path: Path) -> float | None:
+ """Best-effort track duration via ffprobe; None if unavailable."""
+ if shutil.which("ffprobe") is None:
+ return None
+ try:
+ out = subprocess.run(
+ [
+ "ffprobe", "-v", "error",
+ "-show_entries", "format=duration",
+ "-of", "default=noprint_wrappers=1:nokey=1",
+ str(path),
+ ],
+ capture_output=True, text=True, timeout=15, check=True,
+ )
+ value = out.stdout.strip()
+ return round(float(value), 2) if value else None
+ except (subprocess.SubprocessError, ValueError):
+ return None