Skip to content

💫 Toolkit to help you get started with SDD or any other process!

feat(workflows): add opt-in output capture to command and prompt steps (#4957)

Mehrdad Bozorgmehrfc93fb8e5f6b
diff --git a/docs/reference/workflows.md b/docs/reference/workflows.md
index 9187265a8..8b95c6918 100644
--- a/docs/reference/workflows.md
+++ b/docs/reference/workflows.md
@@ -1054,6 +1054,115 @@ or paused command is resumed, the complete dispatch configuration is re-resolved
from the current inputs, so values supplied with `workflow resume --input` take
effect consistently. A resume without updated inputs reproduces the same values.
+### Capturing Command and Prompt Output
+
+`command` and `prompt` steps stream the integration CLI's output to the
+terminal, so `steps.<id>.output.stdout` and `steps.<id>.output.stderr` are empty
+strings. Set `capture: true` to retain that output in the step result instead,
+where later steps can read it:
+
+```yaml
+- id: verdict
+ type: prompt
+ prompt: "Reply with exactly READY or BLOCKED for {{ inputs.spec }}"
+ capture: true
+ capture_max_bytes: 4096 # Optional: per-stream limit (default 1048576)
+ timeout: 120 # Optional: seconds before the CLI is killed
+
+- id: branch
+ type: if
+ condition: "{{ steps.verdict.output.stdout | contains('READY') }}"
+ then:
+ - id: plan
+ command: speckit.plan
+```
+
+| Field | Applies to | Meaning |
+| ------------------- | ------------------- | --------------------------------------------------------------------------------------------------- |
+| `capture` | `command`, `prompt` | `true` captures stdout and stderr instead of streaming them. Must be a literal boolean; default `false`. |
+| `capture_max_bytes` | `command`, `prompt` | Limit for each captured stream: a positive integer, or `-1` for no limit. Default `1048576` (1 MiB). Requires `capture: true`. |
+| `timeout` | `command`, `prompt` | Seconds before the CLI is killed; with `capture: true`, its process group (POSIX) or job object (Windows) is killed, which includes the processes it started unless they detached from it, as a daemonising helper can. For `command` it applies only with `capture: true` (default `600`); a streaming `command` step has no timeout. For `prompt` it always applies (default `300`). |
+
+A captured step prints nothing while it runs. Its output gains these fields:
+
+| Output field | Meaning |
+| ------------------ | -------------------------------------------------------------------------- |
+| `stdout`, `stderr` | The captured text. |
+| `stdout_truncated` | `true` when stdout exceeded `capture_max_bytes` and the rest was discarded. |
+| `stderr_truncated` | The same for stderr. |
+| `timed_out` | `true` when the CLI and its processes were killed at `timeout`; `exit_code` is `-1`. |
+| `interrupted` | `true` when the run was interrupted (Ctrl+C); `exit_code` is `130`. |
+
+**What the text is.** Capture requests the integration CLI's normal text
+output, the same output a streaming step prints. It is not a structured format:
+it can include progress or tool-use lines as well as the final answer, it
+differs between integrations, and it is valid JSON only if the agent printed
+nothing but JSON.
+
+**Limits and encoding.** `capture_max_bytes` counts raw bytes on each stream
+before decoding. Bytes past the limit are read and discarded so the CLI is never
+blocked, and the matching `*_truncated` field is set. Output is decoded as
+UTF-8: invalid bytes become U+FFFD, and a character split by the limit, a
+timeout, or an interruption is dropped rather than replaced. Line endings are
+normalised to `\n`.
+
+With `capture_max_bytes: -1` nothing is discarded and both `*_truncated` fields
+stay `false`. The whole output is then held in memory and written to run state,
+so an agent that prints a large amount grows both without bound. Use it only
+for output you know is small.
+
+**Check completeness before using the output.** Filters such as `from_json` do
+not look at these fields; they parse whatever text is there. Output is
+incomplete when `stdout_truncated` is `true`, and also when `timed_out` or
+`interrupted` is `true` — a stopped CLI leaves partial output even with no size
+limit. Guard the use:
+
+```yaml
+- id: parse
+ type: if
+ condition: "{{ steps.verdict.output.exit_code == 0 and not steps.verdict.output.stdout_truncated }}"
+ then:
+ - id: use
+ type: shell
+ run: "echo parsed"
+```
+
+**Failures.** A non-zero exit, a timeout, or an interruption fails the step and
+keeps whatever was captured in `output.stdout` and `output.stderr`. The step's
+error is a short summary (`Command exited with code 2`,
+`Prompt timed out after 120 seconds.`), never the captured text. A failed step
+stops the run, so set `continue_on_error: true` on the capturing step when a
+later step needs to inspect the failure:
+
+```yaml
+- id: verdict
+ type: prompt
+ prompt: "Reply with exactly READY or BLOCKED"
+ capture: true
+ timeout: 120
+ continue_on_error: true
+
+- id: on-timeout
+ type: if
+ condition: "{{ steps.verdict.output.timed_out }}"
+ then:
+ - id: review
+ type: gate
+ message: "The verdict step timed out. Approve to continue without it."
+```
+
+> ⚠️ **Captured output is stored in plain text and is not redacted.** It is
+> written to the run's `state.json` so that `specify workflow resume` can replay
+> it, and it stays there after the run. Anything the agent prints — including
+> secrets it read from files or the environment — is persisted. Captured text is
+> not printed to the terminal and is not written to `log.jsonl`. Do not enable
+> capture on a step whose output may contain credentials unless the run
+> directory is protected accordingly.
+
+Captured text is agent output and therefore untrusted; see
+[Interpolation and shell safety](#interpolation-and-shell-safety) before using
+it in a `shell` step.
+
## Expressions
Steps can reference inputs and previous step outputs using `{{ expression }}` syntax:
@@ -1114,7 +1223,7 @@ Expressions are resolved by **plain string substitution** — the value of `{{ .
If an interpolated value can contain characters like `;`, `|`, `&`, `$( )`, backticks, or quotes, it can change or extend the command that actually runs. This matters most when the value is not fully under the workflow author's control:
- **Workflow `inputs.*`** — supplied by whoever runs the workflow.
-- **A prior step's output**, e.g. `{{ steps.plan.output.stdout }}` — for a `prompt` step this is **text produced by the AI agent**, which can in turn be influenced by files, tickets, or web content the agent read. Treat agent output as untrusted when it flows into a `shell` step.
+- **A prior step's output**, e.g. `{{ steps.plan.output.stdout }}` — for a `command` or `prompt` step with [`capture: true`](#capturing-command-and-prompt-output) this is **text produced by the AI agent**, which can in turn be influenced by files, tickets, or web content the agent read. Treat agent output as untrusted when it flows into a `shell` step.
There is **no shell-escaping filter** in the expression language and **no sandbox** around a `shell` step, so none of the practices below can be treated as a guarantee that a hostile value is neutralised. The only reliable control is to constrain what an interpolated value *can* be, and to keep values you cannot constrain out of `run` fields entirely. Scrutinise every `run` field that interpolates a value you do not control, and at minimum:
@@ -1153,7 +1262,10 @@ Shell steps automatically receive the following environment variables:
Each workflow run persists its state at `.specify/workflows/runs/<run_id>/`:
-- `state.json` — current run state and step progress
+- `state.json` — current run state and step progress, including each step's
+ output. Output captured with
+ [`capture: true`](#capturing-command-and-prompt-output) is stored here in
+ plain text.
- `inputs.json` — resolved input values
- `log.jsonl` — step-by-step execution log
diff --git a/src/specify_cli/integrations/_capture.py b/src/specify_cli/integrations/_capture.py
new file mode 100644
index 000000000..93d8a10fb
--- /dev/null
+++ b/src/specify_cli/integrations/_capture.py
@@ -0,0 +1,536 @@
+"""Bounded capture of an integration CLI's stdout and stderr.
+
+``subprocess.run(capture_output=True)`` buffers everything a process writes,
+decodes it with the locale encoding, and loses the output on a timeout. Captured
+workflow dispatch needs the opposite on all three: a per-stream byte cap, a
+fixed UTF-8 decoding, and whatever was read before a timeout or interruption.
+"""
+
+from __future__ import annotations
+
+import codecs
+import os
+import signal
+import subprocess
+import threading
+import time
+from collections.abc import Iterator, Sequence
+from contextlib import contextmanager
+from dataclasses import dataclass
+from typing import Any
+
+DEFAULT_CAPTURE_MAX_BYTES = 1024 * 1024
+UNLIMITED_CAPTURE = -1
+
+_READ_CHUNK_BYTES = 64 * 1024
+# After a kill, a descendant that inherited the pipes can keep them open; wait
+# this long for end-of-file before returning what has been read so far.
+_DRAIN_GRACE_SECONDS = 2.0
+
+
+def capture_max_bytes_is_valid(value: Any) -> bool:
+ """Return whether *value* is a positive int or ``-1`` (unlimited).
+
+ bool is a subclass of int, but ``capture_max_bytes: true`` is a config
+ error rather than a one-byte limit, so it is rejected explicitly.
+ """
+ if isinstance(value, bool) or not isinstance(value, int):
+ return False
+ return value > 0 or value == UNLIMITED_CAPTURE
+
+
+@dataclass(frozen=True)
+class CapturedProcess:
+ """Outcome of one captured process.
+
+ ``stdout_truncated``/``stderr_truncated`` report size-based truncation
+ only. Output can also be incomplete because the process was stopped;
+ ``timed_out`` and ``interrupted`` report that separately.
+ """
+
+ exit_code: int
+ stdout: str
+ stderr: str
+ stdout_truncated: bool = False
+ stderr_truncated: bool = False
+ timed_out: bool = False
+ interrupted: bool = False
+
+ def as_dispatch_result(self) -> dict[str, Any]:
+ """Return the ``dispatch_command`` result dict for this process."""
+ return {
+ "exit_code": self.exit_code,
+ "stdout": self.stdout,
+ "stderr": self.stderr,
+ "stdout_truncated": self.stdout_truncated,
+ "stderr_truncated": self.stderr_truncated,
+ "timed_out": self.timed_out,
+ "interrupted": self.interrupted,
+ }
+
+
+class _Sink:
+ """Retain at most *limit* bytes of a stream; count the rest as truncated."""
+
+ def __init__(self, limit: int) -> None:
+ self._limit = limit
+ self._data = bytearray()
+ self._truncated = False
+ self._lock = threading.Lock()
+
+ def feed(self, chunk: bytes) -> None:
+ with self._lock:
+ if self._limit != UNLIMITED_CAPTURE:
+ room = self._limit - len(self._data)
+ if len(chunk) > room:
+ self._truncated = True
+ chunk = chunk[:room]
+ self._data += chunk
+
+ def snapshot(self) -> tuple[bytes, bool]:
+ with self._lock:
+ return bytes(self._data), self._truncated
+
+
+def _drain(pipe: Any, sink: _Sink) -> None:
+ """Read *pipe* to end-of-file, including past the sink's limit.
+
+ Reading continues after the cap so the process never blocks writing to a
+ full pipe; the sink discards what it does not retain.
+ """
+ try:
+ while chunk := pipe.read1(_READ_CHUNK_BYTES):
+ sink.feed(chunk)
+ except (OSError, ValueError):
+ pass
+ finally:
+ try:
+ pipe.close()
+ except OSError:
+ pass
+
+
+# How often a reader join checks whether the capture was cancelled.
+_JOIN_POLL_SECONDS = 0.1
+
+
+def _join(
+ readers: Sequence[threading.Thread],
+ deadline: float | None,
+ cancelled: Any = None,
+) -> bool:
+ """Join *readers* until *deadline*; return whether all reached end-of-file.
+
+ When *cancelled* is given, it is polled while waiting, and the join stops
+ early, returning False, once it returns true.
+ """
+ for reader in readers:
+ while reader.is_alive():
+ if cancelled is not None and cancelled():
+ return False
+ if deadline is None:
+ wait = None if cancelled is None else _JOIN_POLL_SECONDS
+ else:
+ remaining = deadline - time.monotonic()
+ if remaining <= 0:
+ return False
+ wait = (
+ remaining
+ if cancelled is None
+ else min(remaining, _JOIN_POLL_SECONDS)
+ )
+ reader.join(wait)
+ return True
+
+
+# Windows process creation flag; not exported by ``subprocess``.
+_CREATE_SUSPENDED = 0x00000004
+
+
+class _ProcessTree:
+ """Run a process so that it and its process group or job can be killed.
+
+ Killing only the CLI would leave the tools it started running, still
+ changing the project and holding the pipes open after the step has
+ failed. On POSIX the CLI leads a new session and its process group is
+ killed; on Windows it runs inside a job object that is terminated. The
+ boundary is that group or job, not every descendant: a process that
+ moves to another session or process group (POSIX) or breaks away from
+ the job (Windows), as a daemonising helper can, survives the kill.
+ A separate session also keeps the terminal's Ctrl+C from reaching the CLI
+ directly: the interrupt is handled here, by killing the group or job.
+ """
+
+ def __init__(self, args: Sequence[str], cwd: str | None) -> None:
+ isolation: dict[str, Any] = (
+ {"creationflags": subprocess.CREATE_NEW_PROCESS_GROUP | _CREATE_SUSPENDED}
+ if os.name == "nt"
+ else {"start_new_session": True}
+ )
+ self.proc = subprocess.Popen(
+ args,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ cwd=cwd,
+ **isolation,
+ )
+ self._job: Any = None
+ if os.name == "nt":
+ try:
+ self._job = _start_in_job(self.proc)
+ except BaseException:
+ # Fail closed: a CLI whose descendants could not be killed
+ # never runs.
+ self.proc.kill()
+ self.proc.wait()
+ for pipe in (self.proc.stdout, self.proc.stderr):
+ if pipe is not None:
+ pipe.close()
+ raise
+
+ self._lock = threading.Lock()
+ # Set when another thread kills the tree through interrupt().
+ self.interrupted = False
+
+ def _send_kill(self) -> None:
+ if self._job is not None:
+ _windows().TerminateJobObject(self._job, 1)
+ elif os.name != "nt":
+ try:
+ os.killpg(self.proc.pid, signal.SIGKILL)
+ except OSError:
+ pass
+ try:
+ self.proc.kill()
+ except OSError:
+ pass
+
+ def interrupt(self) -> None:
+ """Kill the process group or job from another thread.
+
+ The thread running the capture sees the process exit and reports
+ the capture as interrupted.
+ """
+ with self._lock:
+ self.interrupted = True
+ self._send_kill()
+
+ def kill(self) -> None:
+ """Kill the process group or job, then reap the process."""
+ with self._lock:
+ self._send_kill()
+ self.proc.wait()
+
+ def close(self) -> None:
+ with self._lock:
+ if self._job is not None:
+ _windows().CloseHandle(self._job)
+ self._job = None
+
+
+# Captures running in any thread, so an interrupt of the main thread can
+# reach the ones running in worker threads; see cancel_captures().
+_active_lock = threading.Lock()
+_active: set[_ProcessTree] = set()
+_cancelling = 0
+
+
+def _register(tree: _ProcessTree) -> None:
+ with _active_lock:
+ _active.add(tree)
+ cancelling = _cancelling > 0
+ if cancelling:
+ tree.interrupt()
+
+
+def _unregister(tree: _ProcessTree) -> None:
+ with _active_lock:
+ _active.discard(tree)
+
+
+@contextmanager
+def cancel_captures() -> Iterator[None]:
+ """Kill every running capture, and any that starts, until the block ends.
+
+ Ctrl+C raises ``KeyboardInterrupt`` in the main thread only, and a
+ captured CLI runs in its own session or job, so it does not receive the
+ terminal's Ctrl+C either. When the main thread is interrupted while
+ worker threads run captures, wait for those workers inside this block:
+ their captures are killed and report ``interrupted``, and a capture that
+ starts meanwhile is killed as soon as it is registered.
+ """
+ global _cancelling
+ with _active_lock:
+ _cancelling += 1
+ trees = list(_active)
+ try:
+ for tree in trees:
+ tree.interrupt()
+ yield
+ finally:
+ with _active_lock:
+ _cancelling -= 1
+
+
+@contextmanager
+def _deferred_interrupt() -> Iterator[list[int]]:
+ """Record Ctrl+C instead of raising it until cleanup is in place.
+
+ A ``KeyboardInterrupt`` raised after ``Popen`` returned but before the
+ code that kills the process on interruption would leave the CLI running
+ in its own session (POSIX) or suspended outside its job (Windows).
+ Within this block a SIGINT is appended to the yielded list; the caller
+ treats a non-empty list as an interruption. Only the main thread
+ receives SIGINT, and only Python's default handler is replaced, so a
+ custom or ignoring handler keeps its behaviour.
+ """
+ pending: list[int] = []
+ if (
+ threading.current_thread() is not threading.main_thread()
+ or signal.getsignal(signal.SIGINT) is not signal.default_int_handler
+ ):
+ yield pending
+ return
+ previous = signal.signal(
+ signal.SIGINT, lambda signum, frame: pending.append(signum)
+ )
+ try:
+ yield pending
+ finally:
+ signal.signal(signal.SIGINT, previous)
+
+
+_kernel32: Any = None
+
+
+def _windows() -> Any:
+ """Return kernel32 with the signatures the job-object helpers use."""
+ global _kernel32
+ if _kernel32 is None:
+ import ctypes
+ from ctypes import wintypes
+
+ kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
+ signatures = {
+ "CreateJobObjectW": ((wintypes.LPVOID, wintypes.LPCWSTR), wintypes.HANDLE),
+ "AssignProcessToJobObject": (
+ (wintypes.HANDLE, wintypes.HANDLE),
+ wintypes.BOOL,
+ ),
+ "TerminateJobObject": ((wintypes.HANDLE, wintypes.UINT), wintypes.BOOL),
+ "CreateToolhelp32Snapshot": (
+ (wintypes.DWORD, wintypes.DWORD),
+ wintypes.HANDLE,
+ ),
+ "Thread32First": ((wintypes.HANDLE, ctypes.c_void_p), wintypes.BOOL),
+ "Thread32Next": ((wintypes.HANDLE, ctypes.c_void_p), wintypes.BOOL),
+ "OpenThread": (
+ (wintypes.DWORD, wintypes.BOOL, wintypes.DWORD),
+ wintypes.HANDLE,
+ ),
+ "ResumeThread": ((wintypes.HANDLE,), wintypes.DWORD),
+ "CloseHandle": ((wintypes.HANDLE,), wintypes.BOOL),
+ }
+ for name, (argtypes, restype) in signatures.items():
+ function = getattr(kernel32, name)
+ function.argtypes = argtypes
+ function.restype = restype
+ _kernel32 = kernel32
+ return _kernel32
+
+
+def _start_in_job(proc: subprocess.Popen) -> Any:
+ """Put the suspended *proc* in a new job object, then resume it.
+
+ *proc* was created suspended, so it cannot start a descendant before it
+ is in the job; every process it starts afterwards joins the job too.
+ Returns the job handle. Raises OSError if any step fails.
+ """
+ import ctypes
+
+ kernel32 = _windows()
+ job = kernel32.CreateJobObjectW(None, None)
+ if not job:
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ if not kernel32.AssignProcessToJobObject(job, int(proc._handle)):
+ raise ctypes.WinError(ctypes.get_last_error())
+ _resume_threads(proc.pid)
+ except BaseException:
+ # The CLI may already be running if the failure or interruption came
+ # after a thread was resumed: kill everything in the job before the
+ # handle goes, because closing it does not.
+ kernel32.TerminateJobObject(job, 1)
+ kernel32.CloseHandle(job)
+ raise
+ return job
+
+
+def _resume_threads(pid: int) -> None:
+ """Resume the threads of the suspended process *pid*.
+
+ ``subprocess`` closes the main thread's handle, so the thread is found
+ through a thread snapshot.
+ """
+ import ctypes
+ from ctypes import wintypes
+
+ class THREADENTRY32(ctypes.Structure):
+ _fields_ = [
+ ("dwSize", wintypes.DWORD),
+ ("cntUsage", wintypes.DWORD),
+ ("th32ThreadID", wintypes.DWORD),
+ ("th32OwnerProcessID", wintypes.DWORD),
+ ("tpBasePri", wintypes.LONG),
+ ("tpDeltaPri", wintypes.LONG),
+ ("dwFlags", wintypes.DWORD),
+ ]
+
+ th32cs_snapthread = 0x00000004
+ thread_suspend_resume = 0x0002
+ resume_failed = 0xFFFFFFFF
+ invalid_handle = wintypes.HANDLE(-1).value
+
+ kernel32 = _windows()
+ snapshot = kernel32.CreateToolhelp32Snapshot(th32cs_snapthread, 0)
+ if not snapshot or snapshot == invalid_handle:
+ raise ctypes.WinError(ctypes.get_last_error())
+ resumed = 0
+ try:
+ entry = THREADENTRY32()
+ entry.dwSize = ctypes.sizeof(THREADENTRY32)
+ more = kernel32.Thread32First(snapshot, ctypes.byref(entry))
+ while more:
+ if entry.th32OwnerProcessID == pid:
+ thread = kernel32.OpenThread(
+ thread_suspend_resume, False, entry.th32ThreadID
+ )
+ if not thread:
+ raise ctypes.WinError(ctypes.get_last_error())
+ try:
+ if kernel32.ResumeThread(thread) == resume_failed:
+ raise ctypes.WinError(ctypes.get_last_error())
+ finally:
+ kernel32.CloseHandle(thread)
+ resumed += 1
+ more = kernel32.Thread32Next(snapshot, ctypes.byref(entry))
+ finally:
+ kernel32.CloseHandle(snapshot)
+ if not resumed:
+ raise OSError(f"Could not find the threads of process {pid} to resume.")
+
+
+def _decode(data: bytes, *, complete: bool) -> str:
+ """Decode captured bytes as UTF-8, replacing invalid sequences with U+FFFD.
+
+ When the capture is not *complete* (cut at the byte limit, or the process
+ was stopped), a trailing partial multibyte sequence is dropped rather than
+ replaced, so a cut never invents a replacement character. Line endings are
+ normalised to ``\\n`` as ``subprocess`` text mode does.
+ """
+ decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
+ text = decoder.decode(data, final=complete)
+ return text.replace("\r\n", "\n").replace("\r", "\n")
+
+
+def run_captured(
+ args: Sequence[str],
+ *,
+ cwd: str | None = None,
+ timeout: float | None = None,
+ max_bytes: int = DEFAULT_CAPTURE_MAX_BYTES,
+) -> CapturedProcess:
+ """Run *args* and capture stdout and stderr, each capped at *max_bytes*.
+
+ *max_bytes* counts raw bytes before decoding; ``-1`` disables the cap. A
+ timeout or ``KeyboardInterrupt`` kills the process and is reported on the
+ result (exit code ``-1`` or ``130``) together with the output read so far,
+ instead of being raised, after the process's group (POSIX) or job
+ (Windows) is killed; a descendant that left it is not killed. ``OSError``
+ from starting the process propagates.
+ """
+ if not capture_max_bytes_is_valid(max_bytes):
+ raise ValueError(
+ f"max_bytes must be a positive integer or -1, got {max_bytes!r}."
+ )
+
+ tree: _ProcessTree | None = None
+ sinks = (_Sink(max_bytes), _Sink(max_bytes))
+ readers: list[threading.Thread] = []
+ deadline = None if timeout is None else time.monotonic() + timeout
+ timed_out = interrupted = False
+ try:
+ try:
+ with _deferred_interrupt() as pending_interrupt:
+ tree = _ProcessTree(args, cwd)
+ _register(tree)
+ for pipe, sink in zip((tree.proc.stdout, tree.proc.stderr), sinks):
+ reader = threading.Thread(
+ target=_drain, args=(pipe, sink), daemon=True
+ )
+ reader.start()
+ readers.append(reader)
+ # From here a Ctrl+C raises KeyboardInterrupt inside this try.
+ if pending_interrupt:
+ raise KeyboardInterrupt
+ try:
+ tree.proc.wait(timeout=timeout)
+ except subprocess.TimeoutExpired:
+ timed_out = True
+ else:
+ # The process can exit while a descendant still holds the
+ # pipes open; the same deadline bounds the wait for
+ # end-of-file.
+ # cancel_captures() kills the CLI, but a detached helper can
+ # still hold the pipes, so this wait also stops on it.
+ timed_out = not _join(readers, deadline, lambda: tree.interrupted)
+ except KeyboardInterrupt:
+ interrupted = True
+ if tree is None:
+ # Setup was interrupted, and the constructor has already
+ # killed the process it started.
+ return CapturedProcess(
+ exit_code=130, stdout="", stderr="", interrupted=True
+ )
+ except BaseException:
+ if tree is not None:
+ tree.kill()
+ raise
+ if tree.interrupted:
+ # Killed through cancel_captures() while this thread waited.
+ interrupted, timed_out = True, False
+
+ if timed_out or interrupted:
+ # Killing the group or job closes the write ends of the pipes its
+ # processes hold, so the readers reach end-of-file. Only a
+ # descendant that left the group or job could keep them open; the
+ # grace period bounds that wait.
+ tree.kill()
+ _join(readers, time.monotonic() + _DRAIN_GRACE_SECONDS)
+ # A pipe whose reader never started is closed here instead.
+ for pipe in (tree.proc.stdout, tree.proc.stderr)[len(readers) :]:
+ pipe.close()
+ finally:
+ if tree is not None:
+ _unregister(tree)
+ tree.close()
+
+ (stdout, stdout_truncated), (stderr, stderr_truncated) = (
+ sink.snapshot() for sink in sinks
+ )
+ stopped = timed_out or interrupted
+ if interrupted:
+ exit_code = 130
+ elif timed_out:
+ exit_code = -1
+ else:
+ exit_code = tree.proc.returncode
+ return CapturedProcess(
+ exit_code=exit_code,
+ stdout=_decode(stdout, complete=not (stopped or stdout_truncated)),
+ stderr=_decode(stderr, complete=not (stopped or stderr_truncated)),
+ stdout_truncated=stdout_truncated,
+ stderr_truncated=stderr_truncated,
+ timed_out=timed_out,
+ interrupted=interrupted,
+ )
diff --git a/src/specify_cli/integrations/base.py b/src/specify_cli/integrations/base.py
index 6eee0513d..b3dd0abc3 100644
--- a/src/specify_cli/integrations/base.py
+++ b/src/specify_cli/integrations/base.py
@@ -32,6 +32,7 @@ from .._invocation_style import get_invocation_prefix, is_dollar_skills_agent
from .._toml_string import escape_toml_basic as _escape_toml_basic
from .._toml_string import has_illegal_toml_control as _has_illegal_toml_control
from ..events import install_integration_events, remove_integration_events
+from ._capture import run_captured
if TYPE_CHECKING:
from .manifest import IntegrationManifest
@@ -413,6 +414,8 @@ class IntegrationBase(ABC):
stream: bool = True,
integration_args: Sequence[str] | None = None,
integration_options: Mapping[str, Any] | None = None,
+ output_json: bool | None = None,
+ capture_max_bytes: int | None = None,
) -> dict[str, Any]:
"""Dispatch a Spec Kit command through this integration's CLI.
@@ -429,20 +432,40 @@ class IntegrationBase(ABC):
piped directly to the terminal so the user sees live output.
When ``False``, output is captured and returned in the dict.
+ *output_json* selects the CLI's output format independently of
+ *stream*. ``None`` (the default) keeps the historical coupling:
+ text when streaming, structured output when capturing. Pass
+ ``False`` with ``stream=False`` to capture the CLI's textual
+ stdout.
+
+ *capture_max_bytes* selects bounded capture and requires
+ ``stream=False``: each stream is capped at that many raw bytes
+ (``-1`` for no cap) and decoded as UTF-8. The result then also
+ carries ``stdout_truncated``, ``stderr_truncated``, ``timed_out``,
+ and ``interrupted``, and a timeout or interruption is reported
+ there with the output read so far instead of being raised.
+ ``None`` (the default) keeps the unbounded capture, which raises
+ ``subprocess.TimeoutExpired``.
+
Returns a dict with ``exit_code``, ``stdout``, and ``stderr``.
Raises ``NotImplementedError`` if the integration does not
support CLI dispatch.
"""
import subprocess
+ if stream and capture_max_bytes is not None:
+ raise ValueError("capture_max_bytes requires stream=False.")
+
self.validate_runtime_config(integration_args, integration_options)
prompt = self._build_dispatch_prompt(command_name, args, project_root)
# When streaming to the terminal, request text output so the
# user sees readable output instead of raw JSONL events.
+ if output_json is None:
+ output_json = not stream
exec_args = self.build_exec_args(
prompt,
model=model,
- output_json=not stream,
+ output_json=output_json,
integration_args=integration_args,
integration_options=integration_options,
project_root=project_root,
@@ -489,6 +512,14 @@ class IntegrationBase(ABC):
"stderr": "",
}
+ if capture_max_bytes is not None:
+ return run_captured(
+ exec_args,
+ cwd=cwd,
+ timeout=timeout,
+ max_bytes=capture_max_bytes,
+ ).as_dispatch_result()
+
result = subprocess.run(
exec_args,
capture_output=True,
diff --git a/src/specify_cli/integrations/copilot/__init__.py b/src/specify_cli/integrations/copilot/__init__.py
index d997db85d..c41f55e87 100644
--- a/src/specify_cli/integrations/copilot/__init__.py
+++ b/src/specify_cli/integrations/copilot/__init__.py
@@ -29,6 +29,7 @@ from typing import Any
import typer
+from .._capture import run_captured
from ..base import IntegrationBase, IntegrationOption, SkillsIntegration
from ..manifest import IntegrationManifest
@@ -368,6 +369,8 @@ class CopilotIntegration(IntegrationBase):
stream: bool = True,
integration_args: Sequence[str] | None = None,
integration_options: Mapping[str, Any] | None = None,
+ output_json: bool | None = None,
+ capture_max_bytes: int | None = None,
) -> dict[str, Any]:
"""Dispatch via ``--agent speckit.<stem>`` instead of slash-commands.
@@ -380,6 +383,11 @@ class CopilotIntegration(IntegrationBase):
"""
import subprocess
+ if stream and capture_max_bytes is not None:
+ raise ValueError("capture_max_bytes requires stream=False.")
+ if output_json is None:
+ output_json = not stream
+
self.validate_runtime_config(integration_args, integration_options)
stem = command_name
@@ -412,7 +420,7 @@ class CopilotIntegration(IntegrationBase):
cli_args.append("--yolo")
if model:
cli_args.extend(["--model", model])
- if not stream:
+ if output_json:
cli_args.extend(["--output-format", "json"])
cwd = str(project_root) if project_root else None
@@ -436,6 +444,14 @@ class CopilotIntegration(IntegrationBase):
"stderr": "",
}
+ if capture_max_bytes is not None:
+ return run_captured(
+ cli_args,
+ cwd=cwd,
+ timeout=timeout,
+ max_bytes=capture_max_bytes,
+ ).as_dispatch_result()
+
result = subprocess.run(
cli_args,
capture_output=True,
diff --git a/src/specify_cli/workflows/_execution.py b/src/specify_cli/workflows/_execution.py
index 6d2a2e89d..7a880632b 100644
--- a/src/specify_cli/workflows/_execution.py
+++ b/src/specify_cli/workflows/_execution.py
@@ -14,6 +14,8 @@ from typing import Any
import yaml
+from specify_cli.integrations._capture import cancel_captures
+
from .base import StepContext, StepResult, StepStatus
from .composition import (
CallError,
@@ -396,6 +398,30 @@ def result_view(node):
return result
+def _shutdown_workers(
+ pool: ThreadPoolExecutor, pending: Any, *, interrupted: bool
+) -> None:
+ """Cancel *pending* fan-out futures and wait for the running workers.
+
+ Ctrl+C raises ``KeyboardInterrupt`` in this thread only, and captured CLIs
+ run in their own session or job, so a worker's capture would run on while
+ the pool waits for it. After an interruption, before the wait or during
+ it, the workers' captures are killed and the wait is repeated.
+ """
+ for future in list(pending):
+ future.cancel()
+ if not interrupted:
+ try:
+ pool.shutdown(wait=True)
+ return
+ except KeyboardInterrupt:
+ with cancel_captures():
+ pool.shutdown(wait=True)
+ raise
+ with cancel_captures():
+ pool.shutdown(wait=True)
+
+
@dataclass(frozen=True)
class Occurrence:
"""One invocation, never stored on a shared step implementation."""
@@ -944,21 +970,25 @@ class Execution:
halted.set()
raise
- with ThreadPoolExecutor(max_workers=workers) as pool:
- futures = {i: pool.submit(run_item_guarded, i) for i in range(workers)}
+ pool = ThreadPoolExecutor(max_workers=workers)
+ futures = {}
+ interrupted = False
+ try:
+ futures.update(
+ (i, pool.submit(run_item_guarded, i)) for i in range(workers)
+ )
for index in range(len(items)):
- try:
- outcome, error, value = futures.pop(index).result()
- except BaseException:
- for future in futures.values():
- future.cancel()
- raise
+ outcome, error, value = futures.pop(index).result()
results.append(value)
if outcome in HALTING:
- for future in futures.values():
- future.cancel()
return outcome, error, results
following = index + workers
if following < len(items) and not halted.is_set():
futures[following] = pool.submit(run_item_guarded, following)
+ except KeyboardInterrupt:
+ # Anywhere in submission or collection, not only in result().
+ interrupted = True
+ raise
+ finally:
+ _shutdown_workers(pool, futures.values(), interrupted=interrupted)
return "completed", None, results
diff --git a/src/specify_cli/workflows/step/_capture.py b/src/specify_cli/workflows/step/_capture.py
new file mode 100644
index 000000000..b67700ed5
--- /dev/null
+++ b/src/specify_cli/workflows/step/_capture.py
@@ -0,0 +1,114 @@
+"""Shared ``capture`` contract for the ``command`` and ``prompt`` steps.
+
+Both steps stream the integration CLI's output to the terminal by default.
+``capture: true`` retains stdout and stderr in the step output instead, capped
+per stream by ``capture_max_bytes``. This module holds the parts of that
+contract the two steps must agree on: validation, the output fields, and the
+failure summary.
+"""
+
+from __future__ import annotations
+
+import math
+from typing import Any
+
+from specify_cli.integrations._capture import (
+ DEFAULT_CAPTURE_MAX_BYTES,
+ capture_max_bytes_is_valid,
+)
+
+
+def capture_errors(config: dict[str, Any], label: str) -> list[str]:
+ """Return errors for the ``capture``/``capture_max_bytes`` fields.
+
+ Shared by execute() and validate() so both paths reject the same values
+ with the same message. *label* names the step kind (``"Command"``).
+ """
+ step_id = config.get("id", "?")
+ errors: list[str] = []
+ capture = config.get("capture", False)
+ if "capture" in config and not isinstance(capture, bool):
+ errors.append(
+ f"{label} step {step_id!r}: 'capture' must be true or false, "
+ f"got {capture!r}."
+ )
+ if "capture_max_bytes" in config:
+ max_bytes = config["capture_max_bytes"]
+ if not capture_max_bytes_is_valid(max_bytes):
+ errors.append(
+ f"{label} step {step_id!r}: 'capture_max_bytes' must be a "
+ f"positive integer or -1 (no limit), got {max_bytes!r}."
+ )
+ elif capture is not True:
+ # A limit on a step that streams would silently do nothing.
+ errors.append(
+ f"{label} step {step_id!r}: 'capture_max_bytes' requires "
+ f"'capture: true'."
+ )
+ return errors
+
+
+def capture_limit(config: dict[str, Any]) -> int | None:
+ """Return the per-stream byte limit, or ``None`` when the step streams.
+
+ Call only after :func:`capture_errors` returned no errors.
+ """
+ if config.get("capture", False) is not True:
+ return None
+ return config.get("capture_max_bytes", DEFAULT_CAPTURE_MAX_BYTES)
+
+
+def timeout_error(config: dict[str, Any], label: str) -> str | None:
+ """Return an error message if ``config['timeout']`` is invalid, else None.
+
+ Mirrors the shell and prompt steps: bool is rejected although it is an int
+ subclass, and non-finite or oversized numbers are rejected because they
+ would raise inside the subprocess wait.
+ """
+ if "timeout" not in config:
+ return None
+ timeout = config["timeout"]
+ try:
+ valid = (
+ not isinstance(timeout, bool)
+ and isinstance(timeout, (int, float))
+ and math.isfinite(timeout)
+ and timeout > 0
+ )
+ except OverflowError:
+ valid = False
+ if valid:
+ return None
+ return (
+ f"{label} step {config.get('id', '?')!r}: 'timeout' must be a "
+ f"positive number of seconds, got {timeout!r}."
+ )
+
+
+def record_capture(output: dict[str, Any], dispatch_result: dict[str, Any]) -> None:
+ """Add the capture-only completeness fields to a step's *output*.
+
+ ``*_truncated`` report the byte limit only; ``timed_out`` and
+ ``interrupted`` report a stopped process, whose output is partial even
+ without a limit.
+ """
+ for field in ("stdout_truncated", "stderr_truncated", "timed_out", "interrupted"):
+ output[field] = bool(dispatch_result.get(field, False))
+
+
+def capture_failure(
+ dispatch_result: dict[str, Any],
+ *,
+ subject: str,
+ timeout: float,
+) -> str:
+ """Return the error summary for a failed captured dispatch.
+
+ The step error is written to ``log.jsonl`` and printed, so it never
+ carries captured stderr; that text stays in ``output.stderr``.
+ """
+ if dispatch_result.get("interrupted"):
+ return "Interrupted by user"
+ if dispatch_result.get("timed_out"):
+ return f"{subject} timed out after {timeout} seconds."
+ return f"{subject} exited with code {dispatch_result['exit_code']}"
diff --git a/src/specify_cli/workflows/step/command/__init__.py b/src/specify_cli/workflows/step/command/__init__.py
index f836e9192..770757e98 100644
--- a/src/specify_cli/workflows/step/command/__init__.py
+++ b/src/specify_cli/workflows/step/command/__init__.py
@@ -2,6 +2,7 @@
from __future__ import annotations
+import inspect
import shutil
from pathlib import Path
from typing import Any
@@ -9,6 +10,15 @@ from typing import Any
from specify_cli.integrations.installer import project_dispatch
from specify_cli.workflows.base import StepBase, StepContext, StepResult, StepStatus
from specify_cli.workflows.expressions import evaluate_expression
+from specify_cli.workflows.step._capture import (
+ capture_errors,
+ capture_failure,
+ capture_limit,
+ record_capture,
+ timeout_error,
+)
+
+_DEFAULT_CAPTURE_TIMEOUT = 600
class CommandStep(StepBase):
@@ -24,7 +34,11 @@ class CommandStep(StepBase):
CLI output is streamed to the terminal for live progress.
``output.exit_code`` is always captured and can be referenced
by later steps (e.g. ``{{ steps.specify.output.exit_code }}``).
- Full ``stdout``/``stderr`` capture is a planned enhancement.
+ ``capture: true`` retains the CLI's textual stdout and stderr in
+ ``output.stdout``/``output.stderr`` instead of streaming them,
+ capped per stream by ``capture_max_bytes`` and bounded by
+ ``timeout`` (default 600 seconds). ``timeout`` applies only to a
+ captured step; a streaming step has none.
"""
type_key = "command"
@@ -134,6 +148,23 @@ class CommandStep(StepBase):
return StepResult(status=StepStatus.FAILED, error=runtime_config)
integration_args, integration_options = runtime_config
+ # Same rationale as the guards above: an unvalidated run must fail the
+ # step on a malformed capture/timeout rather than dispatch with a
+ # coerced default or crash inside the subprocess wait. 'timeout' only
+ # applies to a captured step, so it is only checked for one.
+ capture_config_errors = capture_errors(config, "Command")
+ if capture_config_errors:
+ return StepResult(
+ status=StepStatus.FAILED, error=capture_config_errors[0]
+ )
+ capture_max_bytes = capture_limit(config)
+ timeout = _DEFAULT_CAPTURE_TIMEOUT
+ if capture_max_bytes is not None:
+ invalid_timeout = timeout_error(config, "Command")
+ if invalid_timeout is not None:
+ return StepResult(status=StepStatus.FAILED, error=invalid_timeout)
+ timeout = config.get("timeout", _DEFAULT_CAPTURE_TIMEOUT)
+
args_str = str(resolved_input.get("args", ""))
output: dict[str, Any] = {
"command": command,
@@ -157,6 +188,8 @@ class CommandStep(StepBase):
context,
integration_args,
integration_options,
+ capture_max_bytes=capture_max_bytes,
+ timeout=timeout,
)
except ValueError as exc:
output["exit_code"] = 1
@@ -172,11 +205,22 @@ class CommandStep(StepBase):
output["stdout"] = dispatch_result["stdout"]
output["stderr"] = dispatch_result["stderr"]
output["dispatched"] = True
+ if capture_max_bytes is not None:
+ record_capture(output, dispatch_result)
if dispatch_result["exit_code"] != 0:
+ if capture_max_bytes is not None:
+ error = capture_failure(
+ dispatch_result, subject="Command", timeout=timeout
+ )
+ else:
+ error = (
+ dispatch_result["stderr"]
+ or f"Command exited with code {dispatch_result['exit_code']}"
+ )
return StepResult(
status=StepStatus.FAILED,
output=output,
- error=dispatch_result["stderr"] or f"Command exited with code {dispatch_result['exit_code']}",
+ error=error,
)
return StepResult(
status=StepStatus.COMPLETED,
@@ -209,6 +253,9 @@ class CommandStep(StepBase):
context: StepContext,
integration_args: list[str],
integration_options: dict[str, Any],
+ *,
+ capture_max_bytes: int | None = None,
+ timeout: float = _DEFAULT_CAPTURE_TIMEOUT,
) -> dict[str, Any] | None:
"""Invoke *command* by name through the integration CLI.
@@ -217,9 +264,14 @@ class CommandStep(StepBase):
markdown agents, ``/speckit-specify`` for skills agents),
then executes the CLI non-interactively.
+ With *capture_max_bytes* set, the CLI's textual output is captured
+ (bounded by that limit and by *timeout*) instead of streamed.
+
Returns the dispatch result dict, or ``None`` if dispatch is
not possible (integration not found, CLI not installed, or
- dispatch not supported).
+ dispatch not supported). Raises ``ValueError`` if capture is
+ requested from an integration whose ``dispatch_command`` predates
+ the capture keywords.
"""
if not integration_key or not isinstance(integration_key, str):
# A non-string integration (a list/dict/expression that resolved to
@@ -257,6 +309,27 @@ class CommandStep(StepBase):
if cli_path is None and fallback_cli_path is None:
return None
+ capture_kwargs: dict[str, Any] = {}
+ if capture_max_bytes is not None:
+ capture_kwargs = {
+ "stream": False,
+ "output_json": False,
+ "timeout": timeout,
+ "capture_max_bytes": capture_max_bytes,
+ }
+ # An installed integration may override dispatch_command() with
+ # the signature from before these keywords existed; calling it
+ # would raise a raw TypeError and take down the run.
+ parameters = inspect.signature(impl.dispatch_command).parameters
+ if not any(
+ parameter.kind is inspect.Parameter.VAR_KEYWORD
+ for parameter in parameters.values()
+ ) and not all(name in parameters for name in capture_kwargs):
+ raise ValueError(
+ f"integration {integration_key!r} does not support "
+ f"'capture: true'."
+ )
+
try:
return impl.dispatch_command(
command,
@@ -265,6 +338,7 @@ class CommandStep(StepBase):
model=model,
integration_args=integration_args,
integration_options=integration_options,
+ **capture_kwargs,
)
except (NotImplementedError, OSError):
return None
@@ -386,4 +460,11 @@ class CommandStep(StepBase):
f"Command step {config.get('id', '?')!r}: 'model' must be a "
f"string, got {type(model).__name__}."
)
+ errors.extend(capture_errors(config, "Command"))
+ # 'timeout' is only applied to a captured step, so a streaming step's
+ # value is left unchecked as it always was.
+ if config.get("capture") is True:
+ invalid_timeout = timeout_error(config, "Command")
+ if invalid_timeout is not None:
+ errors.append(invalid_timeout)
return errors
diff --git a/src/specify_cli/workflows/step/prompt/__init__.py b/src/specify_cli/workflows/step/prompt/__init__.py
index 1d6b988c5..0177260c6 100644
--- a/src/specify_cli/workflows/step/prompt/__init__.py
+++ b/src/specify_cli/workflows/step/prompt/__init__.py
@@ -7,9 +7,16 @@ import shutil
from pathlib import Path
from typing import Any
+from specify_cli.integrations._capture import run_captured
from specify_cli.integrations.installer import project_dispatch
from specify_cli.workflows.base import StepBase, StepContext, StepResult, StepStatus
from specify_cli.workflows.expressions import evaluate_expression
+from specify_cli.workflows.step._capture import (
+ capture_errors,
+ capture_failure,
+ capture_limit,
+ record_capture,
+)
class PromptStep(StepBase):
@@ -25,8 +32,9 @@ class PromptStep(StepBase):
CLI output is streamed to the terminal for live progress.
``output.exit_code`` is always captured and can be referenced
- by later steps. Full response text capture is a planned
- enhancement.
+ by later steps. ``capture: true`` retains the CLI's textual
+ stdout and stderr in ``output.stdout``/``output.stderr`` instead
+ of streaming them, capped per stream by ``capture_max_bytes``.
Example YAML::
@@ -99,10 +107,24 @@ class PromptStep(StepBase):
if timeout_error is not None:
return StepResult(status=StepStatus.FAILED, error=timeout_error)
+ # Same rationale: fail the step on a malformed capture config rather
+ # than silently streaming or capturing with a coerced limit.
+ capture_config_errors = capture_errors(config, "Prompt")
+ if capture_config_errors:
+ return StepResult(
+ status=StepStatus.FAILED, error=capture_config_errors[0]
+ )
+ capture_max_bytes = capture_limit(config)
+
# Attempt CLI dispatch
timeout = config.get("timeout", 300)
dispatch_result = self._try_dispatch(
- prompt, integration, model, context, timeout=timeout
+ prompt,
+ integration,
+ model,
+ context,
+ timeout=timeout,
+ capture_max_bytes=capture_max_bytes,
)
output: dict[str, Any] = {
@@ -116,14 +138,22 @@ class PromptStep(StepBase):
output["stdout"] = dispatch_result["stdout"]
output["stderr"] = dispatch_result["stderr"]
output["dispatched"] = True
+ if capture_max_bytes is not None:
+ record_capture(output, dispatch_result)
if dispatch_result["exit_code"] != 0:
+ if capture_max_bytes is not None:
+ error = capture_failure(
+ dispatch_result, subject="Prompt", timeout=timeout
+ )
+ else:
+ error = (
+ dispatch_result["stderr"]
+ or f"Prompt exited with code {dispatch_result['exit_code']}"
+ )
return StepResult(
status=StepStatus.FAILED,
output=output,
- error=(
- dispatch_result["stderr"]
- or f"Prompt exited with code {dispatch_result['exit_code']}"
- ),
+ error=error,
)
return StepResult(
status=StepStatus.COMPLETED,
@@ -189,8 +219,13 @@ class PromptStep(StepBase):
model: str | None,
context: StepContext,
timeout: int = 300,
+ capture_max_bytes: int | None = None,
) -> dict[str, Any] | None:
- """Dispatch *prompt* directly through the integration CLI."""
+ """Dispatch *prompt* directly through the integration CLI.
+
+ With *capture_max_bytes* set, stdout and stderr are captured (bounded
+ by that limit) and returned instead of streamed to the terminal.
+ """
if not integration_key or not isinstance(integration_key, str) or not prompt:
# A non-string integration would raise TypeError: unhashable type
# from get_integration's dict lookup and abort the run; treat it as
@@ -242,6 +277,17 @@ class PromptStep(StepBase):
import subprocess
+ if capture_max_bytes is not None:
+ try:
+ return run_captured(
+ exec_args,
+ cwd=str(project_root),
+ timeout=timeout,
+ max_bytes=capture_max_bytes,
+ ).as_dispatch_result()
+ except OSError:
+ return None
+
try:
result = subprocess.run(
exec_args,
@@ -309,4 +355,5 @@ class PromptStep(StepBase):
timeout_error = self._timeout_error(config)
if timeout_error is not None:
errors.append(timeout_error)
+ errors.extend(capture_errors(config, "Prompt"))
return errors
diff --git a/tests/integrations/test_capture.py b/tests/integrations/test_capture.py
new file mode 100644
index 000000000..5c0bc2af8
--- /dev/null
+++ b/tests/integrations/test_capture.py
@@ -0,0 +1,575 @@
+"""Bounded subprocess capture and the captured ``dispatch_command`` contract."""
+
+from __future__ import annotations
+
+import os
+import signal
+import subprocess
+import sys
+import threading
+import time
+
+import pytest
+
+from specify_cli.integrations import get_integration
+from specify_cli.integrations import _capture
+from specify_cli.integrations._capture import (
+ DEFAULT_CAPTURE_MAX_BYTES,
+ CapturedProcess,
+ cancel_captures,
+ run_captured,
+)
+
+
+def _python(source: str, *args: str) -> list[str]:
+ return [sys.executable, "-u", "-c", source, *args]
+
+
+# Writes to the heartbeat file every 0.1s until it is killed.
+_HEARTBEAT = (
+ "import sys, time\n"
+ "while True:\n"
+ " with open(sys.argv[1], 'a') as f: f.write('.')\n"
+ " time.sleep(0.1)\n"
+)
+
+
+def _spawn_descendant(heartbeat, then: str) -> list[str]:
+ """Return a CLI that starts a heartbeat descendant, then runs *then*.
+
+ The descendant inherits stdout and stderr, so it also holds the pipes.
+ """
+ return _python(
+ "import subprocess, sys, time\n"
+ "subprocess.Popen([sys.executable, '-c', sys.argv[1], sys.argv[2]])\n"
+ "while not __import__('os').path.exists(sys.argv[2]): time.sleep(0.05)\n"
+ "print('started')\n" + then,
+ _HEARTBEAT,
+ str(heartbeat),
+ )
+
+
+def _assert_stopped(heartbeat) -> None:
+ """Assert the heartbeat descendant is dead and no reader is left running."""
+ time.sleep(0.5)
+ size = heartbeat.stat().st_size
+ time.sleep(1)
+ assert heartbeat.stat().st_size == size, "descendant still running"
+ assert not [
+ thread
+ for thread in threading.enumerate()
+ if thread.name.endswith("(_drain)")
+ ], "reader threads still running"
+
+
+class TestRunCaptured:
+ def test_captures_both_streams_and_exit_code(self):
+ result = run_captured(
+ _python(
+ "import sys; print('out'); sys.stderr.write('err'); sys.exit(3)"
+ )
+ )
+ assert result == CapturedProcess(exit_code=3, stdout="out\n", stderr="err")
+
+ def test_default_limit_is_one_mebibyte(self):
+ assert DEFAULT_CAPTURE_MAX_BYTES == 1024 * 1024
+
+ def test_finite_limit_truncates_only_the_stream_that_exceeds_it(self):
+ # 3 MB is far beyond any pipe buffer: the process only exits 0 if the
+ # capped stream keeps being drained after the limit is reached.
+ result = run_captured(
+ _python(
+ "import sys; sys.stdout.write('a' * 3_000_000); "
+ "sys.stderr.write('e' * 10)"
+ ),
+ max_bytes=1000,
+ timeout=60,
+ )
+ assert result.exit_code == 0
+ assert result.stdout == "a" * 1000
+ assert result.stdout_truncated is True
+ assert result.stderr == "e" * 10
+ assert result.stderr_truncated is False
+ assert result.timed_out is False
+
+ def test_output_exactly_at_the_limit_is_not_truncated(self):
+ result = run_captured(
+ _python("import sys; sys.stdout.write('a' * 64)"), max_bytes=64
+ )
+ assert result.stdout == "a" * 64
+ assert result.stdout_truncated is False
+
+ def test_unlimited_capture_retains_everything_without_flags(self):
+ result = run_captured(
+ _python("import sys; sys.stdout.write('a' * 3_000_000)"),
+ max_bytes=-1,
+ timeout=60,
+ )
+ assert len(result.stdout) == 3_000_000
+ assert result.stdout_truncated is False
+ assert result.stderr_truncated is False
+
+ def test_limit_counts_raw_bytes_and_drops_a_split_character(self):
+ # Two 3-byte characters with a 4-byte limit: the second is cut after
+ # its first byte and dropped, not turned into a replacement character.
+ result = run_captured(
+ _python("import sys; sys.stdout.buffer.write('€€'.encode('utf-8'))"),
+ max_bytes=4,
+ )
+ assert result.stdout == "€"
+ assert result.stdout_truncated is True
+
+ def test_invalid_utf8_is_replaced(self):
+ result = run_captured(
+ _python(r"import sys; sys.stdout.buffer.write(b'ok\xff\xfeend')")
+ )
+ assert result.stdout == "ok��end"
+ assert result.stdout_truncated is False
+
+ def test_complete_output_ending_mid_character_is_replaced_not_dropped(self):
+ result = run_captured(
+ _python("import sys; sys.stdout.buffer.write('€'.encode('utf-8')[:2])")
+ )
+ assert result.stdout == "�"
+
+ def test_line_endings_are_normalised(self):
+ result = run_captured(
+ _python(r"import sys; sys.stdout.buffer.write(b'a\r\nb\rc\n')")
+ )
+ assert result.stdout == "a\nb\nc\n"
+
+ def test_timeout_keeps_partial_output_and_is_not_size_truncation(self):
+ started = time.monotonic()
+ result = run_captured(
+ _python("import time; print('partial'); time.sleep(60)"),
+ timeout=1,
+ max_bytes=-1,
+ )
+ assert time.monotonic() - started < 30
+ assert result.timed_out is True
+ assert result.exit_code == -1
+ assert result.stdout == "partial\n"
+ assert result.stdout_truncated is False
+ assert result.interrupted is False
+
+ def test_interrupt_kills_the_process_and_keeps_partial_output(
+ self, monkeypatch
+ ):
+ class InterruptedOnce(subprocess.Popen):
+ def wait(self, timeout=None):
+ if not getattr(self, "_interrupt_raised", False):
+ self._interrupt_raised = True
+ # Let the child write before the simulated Ctrl+C.
+ time.sleep(1)
+ raise KeyboardInterrupt
+ return super().wait(timeout)
+
+ monkeypatch.setattr(_capture.subprocess, "Popen", InterruptedOnce)
+ started = time.monotonic()
+ result = run_captured(
+ _python("import time; print('partial'); time.sleep(60)")
+ )
+ assert time.monotonic() - started < 30
+ assert result.interrupted is True
+ assert result.exit_code == 130
+ assert result.stdout == "partial\n"
+ assert result.timed_out is False
+
+ def test_timeout_kills_descendants_and_stops_readers(self, tmp_path):
+ heartbeat = tmp_path / "heartbeat"
+ result = run_captured(
+ _spawn_descendant(heartbeat, "time.sleep(60)"), timeout=5
+ )
+ assert result.timed_out is True
+ assert result.stdout == "started\n"
+ _assert_stopped(heartbeat)
+
+ def test_descendant_holding_the_pipes_after_exit_is_killed(self, tmp_path):
+ """The CLI exits, but its descendant keeps the pipes open."""
+ heartbeat = tmp_path / "heartbeat"
+ started = time.monotonic()
+ result = run_captured(_spawn_descendant(heartbeat, ""), timeout=5)
+ assert time.monotonic() - started < 30
+ assert result.timed_out is True
+ assert result.stdout == "started\n"
+ _assert_stopped(heartbeat)
+
+ def test_interrupt_kills_descendants_and_stops_readers(
+ self, tmp_path, monkeypatch
+ ):
+ class InterruptedOnce(subprocess.Popen):
+ def wait(self, timeout=None):
+ if not getattr(self, "_interrupt_raised", False):
+ self._interrupt_raised = True
+ while not heartbeat.exists():
+ time.sleep(0.05)
+ raise KeyboardInterrupt
+ return super().wait(timeout)
+
+ heartbeat = tmp_path / "heartbeat"
+ monkeypatch.setattr(_capture.subprocess, "Popen", InterruptedOnce)
+ result = run_captured(_spawn_descendant(heartbeat, "time.sleep(60)"))
+ assert result.interrupted is True
+ assert result.exit_code == 130
+ _assert_stopped(heartbeat)
+
+ def test_interrupt_while_starting_the_readers_kills_descendants(
+ self, tmp_path, monkeypatch
+ ):
+ class InterruptedStart(threading.Thread):
+ def start(self):
+ # Interrupt once the CLI is running and has a descendant.
+ while not heartbeat.exists():
+ time.sleep(0.05)
+ raise KeyboardInterrupt
+
+ heartbeat = tmp_path / "heartbeat"
+ monkeypatch.setattr(_capture.threading, "Thread", InterruptedStart)
+ result = run_captured(_spawn_descendant(heartbeat, "time.sleep(60)"))
+ assert result.interrupted is True
+ assert result.exit_code == 130
+ _assert_stopped(heartbeat)
+
+ def test_interrupt_during_process_setup_is_reported_not_raised(
+ self, monkeypatch
+ ):
+ class InterruptedSetup:
+ def __init__(self, args, cwd):
+ raise KeyboardInterrupt
+
+ monkeypatch.setattr(_capture, "_ProcessTree", InterruptedSetup)
+ result = run_captured(_python("pass"))
+ assert result == CapturedProcess(
+ exit_code=130, stdout="", stderr="", interrupted=True
+ )
+
+ def test_interrupt_right_after_the_process_starts_kills_it(self, monkeypatch):
+ """Ctrl+C after Popen returns, before cleanup is set up, still kills."""
+ spawned: list[subprocess.Popen] = []
+
+ class InterruptedAfterSpawn(subprocess.Popen):
+ def __init__(self, *args, **kwargs):
+ super().__init__(*args, **kwargs)
+ spawned.append(self)
+ signal.raise_signal(signal.SIGINT)
+
+ monkeypatch.setattr(_capture.subprocess, "Popen", InterruptedAfterSpawn)
+ started = time.monotonic()
+ result = run_captured(_python("import time; time.sleep(60)"))
+ assert time.monotonic() - started < 30
+ assert result.interrupted is True
+ assert result.exit_code == 130
+ (proc,) = spawned
+ assert proc.returncode is not None, "the CLI was left running"
+ assert signal.getsignal(signal.SIGINT) is signal.default_int_handler
+
+ def test_completed_process_is_not_killed(self, monkeypatch):
+ def fail(self):
+ raise AssertionError("a completed process must not be killed")
+
+ monkeypatch.setattr(_capture._ProcessTree, "kill", fail)
+ result = run_captured(_python("print('done')"), timeout=30)
+ assert result == CapturedProcess(exit_code=0, stdout="done\n", stderr="")
+
+ @pytest.mark.skipif(sys.platform != "win32", reason="Windows job objects")
+ def test_windows_cli_does_not_run_before_it_is_in_the_job(
+ self, tmp_path, monkeypatch
+ ):
+ """The CLI starts suspended, so it cannot spawn outside the job."""
+ marker = tmp_path / "ran"
+ start_in_job = _capture._start_in_job
+ ran_before_job: list[bool] = []
+
+ def observe(proc):
+ # A running CLI would create the marker well within this delay.
+ time.sleep(1)
+ ran_before_job.append(marker.exists())
+ return start_in_job(proc)
+
+ monkeypatch.setattr(_capture, "_start_in_job", observe)
+ result = run_captured(
+ _python(
+ "import sys; open(sys.argv[1], 'w').close(); print('ran')",
+ str(marker),
+ ),
+ timeout=60,
+ )
+ assert ran_before_job == [False]
+ assert result.stdout == "ran\n"
+ assert marker.exists()
+
+ @pytest.mark.skipif(sys.platform != "win32", reason="Windows job objects")
+ def test_windows_interrupt_after_resume_kills_the_job(
+ self, tmp_path, monkeypatch
+ ):
+ """Setup interrupted after the CLI was resumed still fails closed."""
+ resume_threads = _capture._resume_threads
+
+ def resume_then_interrupt(pid):
+ resume_threads(pid)
+ while not heartbeat.exists():
+ time.sleep(0.05)
+ raise KeyboardInterrupt
+
+ heartbeat = tmp_path / "heartbeat"
+ monkeypatch.setattr(_capture, "_resume_threads", resume_then_interrupt)
+ result = run_captured(_spawn_descendant(heartbeat, "time.sleep(60)"))
+ assert result.interrupted is True
+ assert result.exit_code == 130
+ _assert_stopped(heartbeat)
+
+ @pytest.mark.skipif(sys.platform != "win32", reason="Windows job objects")
+ def test_windows_cli_never_runs_without_a_job(self, tmp_path, monkeypatch):
+ def fail(proc):
+ raise OSError("job assignment failed")
+
+ marker = tmp_path / "ran"
+ monkeypatch.setattr(_capture, "_start_in_job", fail)
+ with pytest.raises(OSError, match="job assignment failed"):
+ run_captured(
+ _python("import sys; open(sys.argv[1], 'w').close()", str(marker))
+ )
+ time.sleep(1)
+ assert not marker.exists()
+
+ @pytest.mark.parametrize("max_bytes", [0, -2, True, False, 1.5, "10", None])
+ def test_invalid_limit_is_rejected_before_starting_a_process(
+ self, max_bytes, monkeypatch
+ ):
+ def fail(*args, **kwargs):
+ raise AssertionError("process must not start")
+
+ monkeypatch.setattr(_capture.subprocess, "Popen", fail)
+ with pytest.raises(ValueError, match="positive integer or -1"):
+ run_captured(_python("pass"), max_bytes=max_bytes)
+
+ def test_missing_executable_raises_oserror(self, tmp_path):
+ with pytest.raises(OSError):
+ run_captured([str(tmp_path / "no-such-cli")])
+
+
+class TestDispatchCommandCapture:
+ """``output_json`` and ``capture_max_bytes`` on ``dispatch_command``."""
+
+ @pytest.fixture
+ def claude(self, monkeypatch):
+ impl = get_integration("claude")
+ monkeypatch.setattr(
+ type(impl),
+ "build_exec_args",
+ lambda self, prompt, *, output_json=True, **kwargs: _python(
+ "import sys; print(sys.argv[1], sys.argv[2])",
+ "json" if output_json else "text",
+ prompt,
+ ),
+ )
+ return impl
+
+ def test_bounded_capture_returns_text_and_completeness_fields(self, claude):
+ result = claude.dispatch_command(
+ "speckit.plan",
+ stream=False,
+ output_json=False,
+ capture_max_bytes=1024,
+ timeout=60,
+ )
+ assert result == {
+ "exit_code": 0,
+ "stdout": "text /speckit-plan\n",
+ "stderr": "",
+ "stdout_truncated": False,
+ "stderr_truncated": False,
+ "timed_out": False,
+ "interrupted": False,
+ }
+
+ def test_existing_captured_callers_keep_structured_output(self, claude):
+ """``stream=False`` alone still requests JSON and returns three keys."""
+ result = claude.dispatch_command("speckit.plan", stream=False, timeout=60)
+ assert result == {
+ "exit_code": 0,
+ "stdout": "json /speckit-plan\n",
+ "stderr": "",
+ }
+
+ def test_existing_captured_callers_still_raise_on_timeout(self, monkeypatch):
+ impl = get_integration("claude")
+ monkeypatch.setattr(
+ type(impl),
+ "build_exec_args",
+ lambda self, prompt, **kwargs: _python("import time; time.sleep(60)"),
+ )
+ with pytest.raises(subprocess.TimeoutExpired):
+ impl.dispatch_command("speckit.plan", stream=False, timeout=1)
+
+ def test_bounded_capture_reports_timeout_instead_of_raising(self, monkeypatch):
+ impl = get_integration("claude")
+ monkeypatch.setattr(
+ type(impl),
+ "build_exec_args",
+ lambda self, prompt, **kwargs: _python(
+ "import time; print('partial'); time.sleep(60)"
+ ),
+ )
+ result = impl.dispatch_command(
+ "speckit.plan", stream=False, capture_max_bytes=-1, timeout=1
+ )
+ assert result["timed_out"] is True
+ assert result["exit_code"] == -1
+ assert result["stdout"] == "partial\n"
+
+ def test_streaming_dispatch_still_requests_text(self, claude, capfd):
+ result = claude.dispatch_command("speckit.plan")
+ assert result == {"exit_code": 0, "stdout": "", "stderr": ""}
+ printed = capfd.readouterr().out.replace("\r\n", "\n")
+ assert printed == "text /speckit-plan\n"
+
+ def test_capture_limit_with_streaming_is_rejected(self, claude):
+ with pytest.raises(ValueError, match="requires stream=False"):
+ claude.dispatch_command("speckit.plan", capture_max_bytes=1024)
+
+
+class TestCopilotDispatchCapture:
+ """Copilot overrides ``dispatch_command`` and builds its own argv."""
+
+ @pytest.fixture
+ def recorded(self, monkeypatch):
+ from specify_cli.integrations import copilot as copilot_module
+
+ calls: list[tuple[list[str], dict]] = []
+
+ def fake_run_captured(args, **kwargs):
+ calls.append((list(args), kwargs))
+ return CapturedProcess(exit_code=0, stdout="text", stderr="")
+
+ def fake_run(args, **kwargs):
+ calls.append((list(args), kwargs))
+ return subprocess.CompletedProcess(args, 0, stdout="{}", stderr="")
+
+ monkeypatch.setattr(copilot_module, "run_captured", fake_run_captured)
+ monkeypatch.setattr("subprocess.run", fake_run)
+ return calls
+
+ def test_bounded_text_capture_omits_json_format(self, recorded):
+ result = get_integration("copilot").dispatch_command(
+ "speckit.plan",
+ stream=False,
+ output_json=False,
+ capture_max_bytes=2048,
+ timeout=7,
+ )
+ (args, kwargs), = recorded
+ assert "--output-format" not in args
+ assert kwargs["max_bytes"] == 2048
+ assert kwargs["timeout"] == 7
+ assert result["stdout"] == "text"
+ assert result["stdout_truncated"] is False
+
+ def test_existing_captured_callers_keep_json_format(self, recorded):
+ result = get_integration("copilot").dispatch_command(
+ "speckit.plan", stream=False
+ )
+ (args, kwargs), = recorded
+ assert args[args.index("--output-format") + 1] == "json"
+ assert kwargs["capture_output"] is True
+ assert result == {"exit_code": 0, "stdout": "{}", "stderr": ""}
+
+ def test_streaming_omits_json_format(self, recorded):
+ get_integration("copilot").dispatch_command("speckit.plan")
+ (args, kwargs), = recorded
+ assert "--output-format" not in args
+ assert "capture_output" not in kwargs
+
+ def test_capture_limit_with_streaming_is_rejected(self, recorded):
+ with pytest.raises(ValueError, match="requires stream=False"):
+ get_integration("copilot").dispatch_command(
+ "speckit.plan", capture_max_bytes=1024
+ )
+ assert recorded == []
+
+
+class TestCancelCaptures:
+ """Interrupting captures that run in worker threads."""
+
+ @staticmethod
+ def _in_thread(args):
+ results: list[CapturedProcess] = []
+ worker = threading.Thread(
+ target=lambda: results.append(run_captured(args, timeout=60))
+ )
+ worker.start()
+ return worker, results
+
+ def test_running_capture_is_killed_and_reports_interrupted(self, tmp_path):
+ heartbeat = tmp_path / "heartbeat"
+ worker, results = self._in_thread(
+ _spawn_descendant(heartbeat, "time.sleep(60)")
+ )
+ while not heartbeat.exists():
+ time.sleep(0.05)
+ started = time.monotonic()
+ with cancel_captures():
+ worker.join(30)
+ assert not worker.is_alive()
+ assert time.monotonic() - started < 30
+ (result,) = results
+ assert result.interrupted is True
+ assert result.timed_out is False
+ assert result.exit_code == 130
+ _assert_stopped(heartbeat)
+
+ @pytest.mark.skipif(
+ sys.platform == "win32", reason="leaving the job needs breakaway rights"
+ )
+ def test_cancel_stops_waiting_for_a_detached_helper(self, tmp_path):
+ """The CLI exited, but a helper outside its group holds the pipes."""
+ pid_file = tmp_path / "helper.pid"
+ cli = _python(
+ "import os, subprocess, sys\n"
+ "helper = subprocess.Popen(\n"
+ " [sys.executable, '-c', 'import time; time.sleep(60)'],\n"
+ " start_new_session=True)\n"
+ "open(sys.argv[1] + '.tmp', 'w').write(str(helper.pid))\n"
+ "os.replace(sys.argv[1] + '.tmp', sys.argv[1])\n",
+ str(pid_file),
+ )
+ results: list[CapturedProcess] = []
+ worker = threading.Thread(
+ target=lambda: results.append(run_captured(cli, timeout=20))
+ )
+ worker.start()
+ try:
+ while not pid_file.exists():
+ time.sleep(0.05)
+ # Let the CLI exit, so the capture is waiting on its readers.
+ time.sleep(0.5)
+ started = time.monotonic()
+ with cancel_captures():
+ worker.join(30)
+ assert time.monotonic() - started < 10, "the join ignored cancellation"
+ (result,) = results
+ assert result.interrupted is True
+ assert result.timed_out is False
+ finally:
+ if pid_file.exists():
+ try:
+ os.kill(int(pid_file.read_text()), signal.SIGKILL)
+ except ProcessLookupError:
+ pass
+ worker.join(30)
+
+ def test_capture_started_while_cancelling_is_killed(self, tmp_path):
+ with cancel_captures():
+ worker, results = self._in_thread(
+ _python("import time; time.sleep(60)")
+ )
+ worker.join(30)
+ assert not worker.is_alive()
+ (result,) = results
+ assert result.interrupted is True
+
+ def test_captures_after_the_block_run_normally(self):
+ with cancel_captures():
+ pass
+ result = run_captured(_python("print('ok')"), timeout=30)
+ assert result == CapturedProcess(exit_code=0, stdout="ok\n", stderr="")
diff --git a/tests/workflows/test_step_capture.py b/tests/workflows/test_step_capture.py
new file mode 100644
index 000000000..101039da5
--- /dev/null
+++ b/tests/workflows/test_step_capture.py
@@ -0,0 +1,646 @@
+"""``capture: true`` on the ``command`` and ``prompt`` workflow steps."""
+
+from __future__ import annotations
+
+import json
+import signal
+import sys
+import threading
+import time
+
+import pytest
+import yaml
+
+from specify_cli.integrations import get_integration
+from specify_cli.integrations._capture import CapturedProcess
+from specify_cli.integrations.base import IntegrationBase
+from specify_cli.workflows.base import StepContext, StepStatus
+from specify_cli.workflows.engine import WorkflowEngine
+from specify_cli.workflows.step.command import CommandStep
+from specify_cli.workflows.step.prompt import PromptStep
+
+STEPS = {
+ "command": (CommandStep, {"command": "speckit.plan"}),
+ "prompt": (PromptStep, {"type": "prompt", "prompt": "Summarise the plan"}),
+}
+LABELS = {"command": "Command", "prompt": "Prompt"}
+CAPTURE_FIELDS = ("stdout_truncated", "stderr_truncated", "timed_out", "interrupted")
+
+
+def _config(kind: str, **fields) -> dict:
+ return {"id": "ask", **STEPS[kind][1], **fields}
+
+
+def _execute(kind: str, project_dir, **fields):
+ context = StepContext(default_integration="claude", project_root=str(project_dir))
+ return STEPS[kind][0]().execute(_config(kind, **fields), context)
+
+
+def fake_cli(monkeypatch, project_dir):
+ """Make the ``claude`` integration run a Python script as its CLI.
+
+ The script receives the requested output format (``json``/``text``) as
+ ``sys.argv[1]`` and counts its invocations in ``calls.txt``.
+ """
+
+ def install(body: str) -> None:
+ script = project_dir / "fake_cli.py"
+ script.write_text(
+ "import sys, time\n"
+ f"open({str(project_dir / 'calls.txt')!r}, 'a').write('x')\n" + body,
+ encoding="utf-8",
+ )
+ monkeypatch.setattr(
+ type(get_integration("claude")),
+ "build_exec_args",
+ lambda self, prompt, *, output_json=True, **kwargs: [
+ sys.executable,
+ "-u",
+ str(script),
+ "json" if output_json else "text",
+ ],
+ )
+
+ install.calls = lambda: len((project_dir / "calls.txt").read_text())
+ return install
+
+
[email protected]("kind", ["command", "prompt"])
+class TestCaptureValidation:
+ @pytest.mark.parametrize(
+ "fields",
+ [
+ {},
+ {"capture": False},
+ {"capture": True},
+ {"capture": True, "capture_max_bytes": 1},
+ {"capture": True, "capture_max_bytes": -1},
+ {"capture": True, "timeout": 30},
+ ],
+ )
+ def test_valid_configs_pass(self, kind, fields):
+ assert STEPS[kind][0]().validate(_config(kind, **fields)) == []
+
+ @pytest.mark.parametrize("capture", ["true", 1, 0, None, []])
+ def test_non_boolean_capture_is_rejected(self, kind, capture):
+ errors = STEPS[kind][0]().validate(_config(kind, capture=capture))
+ assert errors == [
+ f"{LABELS[kind]} step 'ask': 'capture' must be true or false, "
+ f"got {capture!r}."
+ ]
+
+ @pytest.mark.parametrize("max_bytes", [0, -2, True, False, 1.5, "1024", None])
+ def test_invalid_limit_is_rejected(self, kind, max_bytes):
+ errors = STEPS[kind][0]().validate(
+ _config(kind, capture=True, capture_max_bytes=max_bytes)
+ )
+ assert errors == [
+ f"{LABELS[kind]} step 'ask': 'capture_max_bytes' must be a positive "
+ f"integer or -1 (no limit), got {max_bytes!r}."
+ ]
+
+ @pytest.mark.parametrize("fields", [{}, {"capture": False}])
+ def test_limit_without_capture_is_rejected(self, kind, fields):
+ errors = STEPS[kind][0]().validate(
+ _config(kind, capture_max_bytes=1024, **fields)
+ )
+ assert errors == [
+ f"{LABELS[kind]} step 'ask': 'capture_max_bytes' requires "
+ f"'capture: true'."
+ ]
+
+ @pytest.mark.parametrize(
+ "fields",
+ [
+ {"capture": "yes"},
+ {"capture": True, "capture_max_bytes": 0},
+ {"capture_max_bytes": 1024},
+ {"capture": True, "timeout": -1},
+ ],
+ )
+ def test_unvalidated_execute_fails_the_step_without_dispatching(
+ self, kind, fields, project_dir, fake_cli
+ ):
+ fake_cli("print('ran')\n")
+ result = _execute(kind, project_dir, **fields)
+ assert result.status == StepStatus.FAILED
+ assert "'ask'" in result.error
+ assert not (project_dir / "calls.txt").exists()
+
+
+class TestCommandTimeoutValidation:
+ @pytest.mark.parametrize("timeout", [0, -5, True, "60", float("inf")])
+ def test_invalid_timeout_is_rejected_when_capturing(self, timeout):
+ errors = CommandStep().validate(
+ _config("command", capture=True, timeout=timeout)
+ )
+ assert errors == [
+ f"Command step 'ask': 'timeout' must be a positive number of "
+ f"seconds, got {timeout!r}."
+ ]
+
+ def test_streaming_command_timeout_stays_unchecked(self):
+ """A streaming command never applied 'timeout'; validation is unchanged."""
+ assert CommandStep().validate(_config("command", timeout="5m")) == []
+
+
[email protected]("kind", ["command", "prompt"])
+class TestCaptureExecution:
+ def test_captures_text_output_without_printing_it(
+ self, kind, project_dir, fake_cli, capfd
+ ):
+ fake_cli("print('format=' + sys.argv[1]); sys.stderr.write('note')\n")
+ result = _execute(kind, project_dir, capture=True)
+
+ assert result.status == StepStatus.COMPLETED
+ assert result.error is None
+ assert result.output["exit_code"] == 0
+ assert result.output["stdout"] == "format=text\n"
+ assert result.output["stderr"] == "note"
+ assert result.output["dispatched"] is True
+ assert {field: result.output[field] for field in CAPTURE_FIELDS} == dict.fromkeys(
+ CAPTURE_FIELDS, False
+ )
+ printed = capfd.readouterr()
+ assert "format=text" not in printed.out + printed.err
+
+ @pytest.mark.parametrize("fields", [{}, {"capture": False}])
+ def test_streaming_is_unchanged_without_capture(
+ self, kind, fields, project_dir, fake_cli, capfd
+ ):
+ fake_cli("print('format=' + sys.argv[1]); sys.stderr.write('note')\n")
+ result = _execute(kind, project_dir, **fields)
+
+ assert result.status == StepStatus.COMPLETED
+ assert result.output["stdout"] == ""
+ assert result.output["stderr"] == ""
+ assert not set(CAPTURE_FIELDS) & set(result.output)
+ printed = capfd.readouterr()
+ # Windows writes \r\n to the terminal.
+ assert printed.out.replace("\r\n", "\n") == "format=text\n"
+ assert printed.err == "note"
+
+ def test_non_zero_exit_keeps_output_and_summarises_the_error(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("print('half done'); sys.stderr.write('token=SECRET'); sys.exit(7)\n")
+ result = _execute(kind, project_dir, capture=True)
+
+ assert result.status == StepStatus.FAILED
+ assert result.error == f"{LABELS[kind]} exited with code 7"
+ assert result.output["exit_code"] == 7
+ assert result.output["stdout"] == "half done\n"
+ assert result.output["stderr"] == "token=SECRET"
+ assert result.output["timed_out"] is False
+
+ def test_streaming_non_zero_exit_error_is_unchanged(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.exit(7)\n")
+ result = _execute(kind, project_dir)
+ assert result.status == StepStatus.FAILED
+ assert result.error == f"{LABELS[kind]} exited with code 7"
+ assert result.output["stderr"] == ""
+
+ def test_finite_limit_truncates_and_flags(self, kind, project_dir, fake_cli):
+ fake_cli("sys.stdout.write('a' * 500_000); sys.stderr.write('e' * 4)\n")
+ result = _execute(kind, project_dir, capture=True, capture_max_bytes=16)
+
+ assert result.status == StepStatus.COMPLETED
+ assert result.output["stdout"] == "a" * 16
+ assert result.output["stdout_truncated"] is True
+ assert result.output["stderr"] == "eeee"
+ assert result.output["stderr_truncated"] is False
+
+ def test_unlimited_capture_never_flags_truncation(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.stdout.write('a' * 2_000_000)\n")
+ result = _execute(kind, project_dir, capture=True, capture_max_bytes=-1)
+
+ assert result.status == StepStatus.COMPLETED
+ assert len(result.output["stdout"]) == 2_000_000
+ assert result.output["stdout_truncated"] is False
+
+ def test_non_utf8_output_is_decoded_with_replacement(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.stdout.buffer.write('caf\\u00e9 '.encode('utf-8') + b'\\xff')\n")
+ result = _execute(kind, project_dir, capture=True)
+ assert result.output["stdout"] == "café \ufffd"
+
+ def test_timeout_fails_the_step_with_partial_output(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("print('partial'); sys.stderr.write('slow'); time.sleep(60)\n")
+ result = _execute(
+ kind, project_dir, capture=True, capture_max_bytes=-1, timeout=1
+ )
+
+ assert result.status == StepStatus.FAILED
+ assert result.error == f"{LABELS[kind]} timed out after 1 seconds."
+ assert result.output["exit_code"] == -1
+ assert result.output["timed_out"] is True
+ assert result.output["stdout"] == "partial\n"
+ assert result.output["stderr"] == "slow"
+ # Partial because the process was stopped, not because of the limit.
+ assert result.output["stdout_truncated"] is False
+
+ def test_interruption_fails_the_step_with_partial_output(
+ self, kind, project_dir, fake_cli, monkeypatch
+ ):
+ fake_cli("print('unused')\n")
+ interrupted = CapturedProcess(
+ exit_code=130, stdout="partial", stderr="noise", interrupted=True
+ )
+ for module in (
+ "specify_cli.integrations.base",
+ "specify_cli.workflows.step.prompt",
+ ):
+ monkeypatch.setattr(
+ f"{module}.run_captured", lambda *args, **kwargs: interrupted
+ )
+ result = _execute(kind, project_dir, capture=True)
+
+ assert result.status == StepStatus.FAILED
+ assert result.error == "Interrupted by user"
+ assert result.output["exit_code"] == 130
+ assert result.output["interrupted"] is True
+ assert result.output["stdout"] == "partial"
+ assert result.output["stderr"] == "noise"
+
+
+class TestCommandCaptureDispatch:
+ def test_default_timeout_is_600_seconds(self, project_dir, fake_cli, monkeypatch):
+ fake_cli("print('unused')\n")
+ seen: dict = {}
+
+ def fake_run_captured(args, **kwargs):
+ seen.update(kwargs)
+ return CapturedProcess(exit_code=0, stdout="", stderr="")
+
+ monkeypatch.setattr(
+ "specify_cli.integrations.base.run_captured", fake_run_captured
+ )
+ _execute("command", project_dir, capture=True)
+ assert seen["timeout"] == 600
+ assert seen["max_bytes"] == 1024 * 1024
+
+ def test_integration_without_capture_keywords_fails_cleanly(
+ self, project_dir, fake_cli, monkeypatch
+ ):
+ """An installed integration may still override the older signature."""
+ fake_cli("print('ran')\n")
+
+ def legacy_dispatch(
+ self,
+ command_name,
+ args="",
+ *,
+ project_root=None,
+ model=None,
+ timeout=600,
+ stream=True,
+ integration_args=None,
+ integration_options=None,
+ ):
+ return IntegrationBase.dispatch_command(
+ self, command_name, args, project_root=project_root, stream=stream
+ )
+
+ monkeypatch.setattr(
+ type(get_integration("claude")), "dispatch_command", legacy_dispatch
+ )
+ captured = _execute("command", project_dir, capture=True)
+ assert captured.status == StepStatus.FAILED
+ assert captured.error == (
+ "Command step 'ask': integration 'claude' does not support "
+ "'capture: true'."
+ )
+ assert not (project_dir / "calls.txt").exists()
+
+ streamed = _execute("command", project_dir)
+ assert streamed.status == StepStatus.COMPLETED
+
+
+def _install_workflow(project_dir, steps: list[dict], inputs: dict | None = None):
+ definition = {
+ "schema_version": "1.0",
+ "workflow": {
+ "id": "wf",
+ "name": "WF",
+ "version": "1.0.0",
+ "integration": "claude",
+ },
+ "steps": steps,
+ }
+ if inputs:
+ definition["inputs"] = inputs
+ workflow_dir = project_dir / ".specify" / "workflows" / "wf"
+ workflow_dir.mkdir(parents=True, exist_ok=True)
+ (workflow_dir / "workflow.yml").write_text(
+ yaml.safe_dump(definition), encoding="utf-8"
+ )
+ engine = WorkflowEngine(project_dir)
+ return engine, engine.load_workflow("wf")
+
+
+def _run_dir(project_dir, state):
+ return project_dir / ".specify" / "workflows" / "runs" / state.run_id
+
+
[email protected]("kind", ["command", "prompt"])
+class TestCaptureInWorkflows:
+ """The composition gap from issue #4613, before and after ``capture``."""
+
+ @staticmethod
+ def _steps(kind: str, **fields) -> list[dict]:
+ return [
+ _config(kind, **fields),
+ {
+ "id": "use",
+ "type": "shell",
+ "run": "echo verdict={{ steps.ask.output.stdout }}",
+ },
+ ]
+
+ def test_without_capture_downstream_sees_no_output(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.stdout.write('READY')\n")
+ engine, definition = _install_workflow(project_dir, self._steps(kind))
+ state = engine.execute(definition, {})
+
+ assert state.status.value == "completed"
+ assert state.step_results["use"]["output"]["stdout"].strip() == "verdict="
+
+ def test_with_capture_downstream_consumes_the_output(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.stdout.write('READY')\n")
+ engine, definition = _install_workflow(
+ project_dir, self._steps(kind, capture=True)
+ )
+ state = engine.execute(definition, {})
+
+ assert state.status.value == "completed"
+ assert state.step_results["use"]["output"]["stdout"].strip() == "verdict=READY"
+
+ def test_captured_output_survives_resume_without_redispatch(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("sys.stdout.write('READY')\n")
+ steps = self._steps(kind, capture=True)
+ steps.insert(
+ 1,
+ {
+ "id": "review",
+ "type": "gate",
+ "message": "Continue?",
+ "options": ["approve", "reject"],
+ "verdict_input": "verdict",
+ },
+ )
+ engine, definition = _install_workflow(
+ project_dir, steps, inputs={"verdict": {"type": "string", "default": ""}}
+ )
+ paused = engine.execute(definition, {})
+ assert paused.status.value == "paused"
+ assert "use" not in paused.step_results
+
+ resumed = WorkflowEngine(project_dir).resume(
+ paused.run_id, inputs={"verdict": "approve"}
+ )
+ assert resumed.status.value == "completed"
+ assert resumed.step_results["use"]["output"]["stdout"].strip() == "verdict=READY"
+ assert fake_cli.calls() == 1
+
+ def test_captured_stderr_is_persisted_in_state_but_kept_out_of_the_log(
+ self, kind, project_dir, fake_cli, capfd
+ ):
+ fake_cli("sys.stderr.write('token=SECRET'); sys.exit(2)\n")
+ engine, definition = _install_workflow(
+ project_dir, self._steps(kind, capture=True)
+ )
+ state = engine.execute(definition, {})
+
+ assert state.status.value == "failed"
+ run_dir = _run_dir(project_dir, state)
+ saved = json.loads((run_dir / "state.json").read_text(encoding="utf-8"))
+ assert saved["step_results"]["ask"]["output"]["stderr"] == "token=SECRET"
+ assert "SECRET" not in (run_dir / "log.jsonl").read_text(encoding="utf-8")
+ printed = capfd.readouterr()
+ assert "SECRET" not in printed.out + printed.err
+
+ def test_continue_on_error_lets_downstream_inspect_a_failed_capture(
+ self, kind, project_dir, fake_cli
+ ):
+ fake_cli("print('partial'); time.sleep(60)\n")
+ steps = [
+ _config(kind, capture=True, timeout=1, continue_on_error=True),
+ {
+ "id": "recover",
+ "type": "if",
+ "condition": "{{ steps.ask.output.timed_out }}",
+ "then": [
+ {
+ "id": "report",
+ "type": "shell",
+ "run": "echo partial={{ steps.ask.output.stdout }}",
+ }
+ ],
+ },
+ ]
+ engine, definition = _install_workflow(project_dir, steps)
+ state = engine.execute(definition, {})
+
+ assert state.status.value == "completed"
+ assert state.step_results["ask"]["output"]["exit_code"] == -1
+ report = state.step_results["report"]["output"]["stdout"]
+ assert report.strip() == "partial=partial"
+
+
+ not hasattr(signal, "pthread_kill"), reason="needs a SIGINT to the main thread"
+)
+def test_ctrl_c_during_concurrent_fan_out_kills_the_captures(project_dir, fake_cli):
+ """Ctrl+C reaches only the main thread; worker captures must still stop."""
+ fake_cli(
+ "import os\n"
+ f"beat = os.path.join({str(project_dir)!r}, f'beat-{{os.getpid()}}')\n"
+ "while True:\n"
+ " open(beat, 'a').write('.')\n"
+ " time.sleep(0.1)\n"
+ )
+ engine, definition = _install_workflow(
+ project_dir,
+ [
+ {
+ "id": "fan",
+ "type": "fan-out",
+ "items": [1, 2],
+ "max_concurrency": 2,
+ "step": {**_config("prompt", capture=True, timeout=20)},
+ }
+ ],
+ )
+ main = threading.main_thread().ident
+
+ def interrupt_once_both_run():
+ while len(list(project_dir.glob("beat-*"))) < 2:
+ time.sleep(0.05)
+ signal.pthread_kill(main, signal.SIGINT)
+
+ threading.Thread(target=interrupt_once_both_run, daemon=True).start()
+ started = time.monotonic()
+ try:
+ engine.execute(definition, {})
+ except KeyboardInterrupt:
+ pass
+ # Without cancellation the pool waits for both captures' 20s timeout.
+ assert time.monotonic() - started < 15
+
+ beats = list(project_dir.glob("beat-*"))
+ assert len(beats) == 2
+ time.sleep(0.5)
+ sizes = [beat.stat().st_size for beat in beats]
+ time.sleep(1)
+ assert [beat.stat().st_size for beat in beats] == sizes, "a capture kept running"
+
+
+ not hasattr(signal, "pthread_kill"), reason="needs a SIGINT to the main thread"
+)
+def test_ctrl_c_while_fan_out_shuts_down_after_a_failure_kills_the_captures(
+ project_dir, monkeypatch
+):
+ """Item 1 fails at once; Ctrl+C arrives while the pool waits for item 2."""
+ script = project_dir / "cli.py"
+ script.write_text(
+ "import os, sys, time\n"
+ "if 'item=1' in sys.argv[1]:\n"
+ " sys.exit(1)\n"
+ f"beat = os.path.join({str(project_dir)!r}, f'beat-{{os.getpid()}}')\n"
+ "while True:\n"
+ " open(beat, 'a').write('.')\n"
+ " time.sleep(0.1)\n",
+ encoding="utf-8",
+ )
+ monkeypatch.setattr(
+ type(get_integration("claude")),
+ "build_exec_args",
+ lambda self, prompt, **kwargs: [sys.executable, "-u", str(script), prompt],
+ )
+ engine, definition = _install_workflow(
+ project_dir,
+ [
+ {
+ "id": "fan",
+ "type": "fan-out",
+ "items": [1, 2],
+ "max_concurrency": 2,
+ "step": {
+ "id": "ask",
+ "type": "prompt",
+ "prompt": "item={{ item }}",
+ "capture": True,
+ "timeout": 20,
+ },
+ }
+ ],
+ )
+ main = threading.main_thread().ident
+
+ def interrupt_during_shutdown():
+ while not list(project_dir.glob("beat-*")):
+ time.sleep(0.05)
+ # Item 1 has failed by now and the fan-out is waiting for item 2.
+ time.sleep(1)
+ signal.pthread_kill(main, signal.SIGINT)
+
+ threading.Thread(target=interrupt_during_shutdown, daemon=True).start()
+ started = time.monotonic()
+ try:
+ engine.execute(definition, {})
+ except KeyboardInterrupt:
+ pass
+ assert time.monotonic() - started < 15
+
+ (beat,) = project_dir.glob("beat-*")
+ time.sleep(0.5)
+ size = beat.stat().st_size
+ time.sleep(1)
+ assert beat.stat().st_size == size, "the capture kept running"
+
+
+def test_ctrl_c_while_fan_out_submits_an_item_kills_the_running_captures(
+ project_dir, monkeypatch
+):
+ """Ctrl+C lands in submit(), not result(), while item 2's capture runs."""
+ from concurrent.futures import ThreadPoolExecutor
+
+ from specify_cli.workflows import _execution
+
+ script = project_dir / "cli.py"
+ script.write_text(
+ "import os, sys, time\n"
+ "if 'item=1' in sys.argv[1]:\n"
+ " sys.exit(0)\n"
+ f"beat = os.path.join({str(project_dir)!r}, f'beat-{{os.getpid()}}')\n"
+ "while True:\n"
+ " open(beat, 'a').write('.')\n"
+ " time.sleep(0.1)\n",
+ encoding="utf-8",
+ )
+ monkeypatch.setattr(
+ type(get_integration("claude")),
+ "build_exec_args",
+ lambda self, prompt, **kwargs: [sys.executable, "-u", str(script), prompt],
+ )
+
+ class InterruptedThirdSubmit(ThreadPoolExecutor):
+ submitted = 0
+
+ def submit(self, *args, **kwargs):
+ type(self).submitted += 1
+ if type(self).submitted == 3:
+ # Item 1 has completed and item 2's capture is running.
+ while not list(project_dir.glob("beat-*")):
+ time.sleep(0.05)
+ raise KeyboardInterrupt
+ return super().submit(*args, **kwargs)
+
+ monkeypatch.setattr(_execution, "ThreadPoolExecutor", InterruptedThirdSubmit)
+ engine, definition = _install_workflow(
+ project_dir,
+ [
+ {
+ "id": "fan",
+ "type": "fan-out",
+ "items": [1, 2, 3],
+ "max_concurrency": 2,
+ "step": {
+ "id": "ask",
+ "type": "prompt",
+ "prompt": "item={{ item }}",
+ "capture": True,
+ "timeout": 20,
+ },
+ }
+ ],
+ )
+ started = time.monotonic()
+ try:
+ engine.execute(definition, {})
+ except KeyboardInterrupt:
+ pass
+ # Without cancellation the pool waits for item 2's 20s timeout.
+ assert time.monotonic() - started < 15
+ assert InterruptedThirdSubmit.submitted == 3
+
+ (beat,) = project_dir.glob("beat-*")
+ time.sleep(0.5)
+ size = beat.stat().st_size
+ time.sleep(1)
+ assert beat.stat().st_size == size, "the capture kept running"
diff --git a/workflows/README.md b/workflows/README.md
index 642aa6ebb..c71010892 100644
--- a/workflows/README.md
+++ b/workflows/README.md
@@ -151,6 +151,32 @@ Send an arbitrary inline prompt to an integration CLI (no command file needed):
integration: claude
```
+### Capturing Command and Prompt Output
+
+Command and prompt steps stream the CLI's output to the terminal, so their
+`output.stdout` and `output.stderr` are empty. Set `capture: true` to retain the
+output for later steps instead:
+
+```yaml
+- id: verdict
+ type: prompt
+ prompt: "Reply with exactly READY or BLOCKED for {{ inputs.spec }}"
+ capture: true
+ capture_max_bytes: 4096 # Optional: per-stream limit (default 1 MiB, -1 = no limit)
+ timeout: 120 # Optional: seconds before the CLI is killed
+```
+
+A captured step prints nothing while it runs. Output past the limit is
+discarded and reported in `output.stdout_truncated` / `output.stderr_truncated`;
+a timeout or interruption fails the step, sets `output.timed_out` or
+`output.interrupted`, and keeps the partial output. Check those fields before
+parsing the text.
+
+> ⚠️ Captured output is stored unredacted in the run's `state.json`, and it is
+> untrusted agent text. See
+> [Capturing Command and Prompt Output](../docs/reference/workflows.md#capturing-command-and-prompt-output)
+> for the full contract.
+
### Shell Steps
Run a shell command and capture output: