PostgreSQL backup tool with Point-In-Time-Recovery and restore verification
TypeScript backupbackupsdatabasedatabase-backupdevopsdockergolangkubernetesmariadbmongodbmysqlpgpostgrespostgresqls3self-hostedsystem-administrationtoolsweb-ui
FEATURE (agents): Run the test commands a change names in its Change verification section during the full implementation review, add a tests reviewer and the testing skill, and start the hooks through mise with pinned Python and OpenSpec so a missing tool stops the agent instead of skipping the review
Rostislav Dugine775683e3eba
diff --git a/.agent-hooks/common-prompt.md b/.agent-hooks/common-prompt.mdindex eab9a902..ed9698e8 100644--- a/.agent-hooks/common-prompt.md+++ b/.agent-hooks/common-prompt.md@@ -15,8 +15,9 @@ Before judging, read completely:| `assets/readme/**` | `assets/readme/AGENTS.md` || `adr/**` | `adr/AGENTS.md` |-4. The ADRs in `adr/` that the work relies on or contradicts.-5. When the repository has `openspec/`: the main specs under `openspec/specs` that relate to the task, and the artifacts of active changes under `openspec/changes`.+4. When the work writes, changes, runs or reviews tests: `.agents/skills/testing/SKILL.md`.+5. The ADRs in `adr/` that the work relies on or contradicts.+6. When the repository has `openspec/`: the main specs under `openspec/specs` that relate to the task, and the artifacts of active changes under `openspec/changes`.Paths that were already changed before the latest request belong to unfinished work of the user or another agent. Do not review them and do not block on them unless the candidate touched them within this request.diff --git a/.agent-hooks/diff-prompt.md b/.agent-hooks/diff-prompt.mdindex 13184b10..fe824222 100644--- a/.agent-hooks/diff-prompt.md+++ b/.agent-hooks/diff-prompt.md@@ -7,7 +7,7 @@ Use `git status` and the full `git diff HEAD` including staged changes, and read- Check that the diff is focused, technically sound, complete, and matches the latest user request and the accepted plan, if there was one.- Look for bugs: wrong behavior, missed edge cases, broken error handling, races, resource leaks, security problems.- Check the material claims of the candidate's response against the repository, including claims that tests and linters ran and passed. A claim that cannot be true for this diff is a problem.-- Check tests against the testing rules of the module docs: new behavior has a test at the level the module doc prefers, test names describe the scenario, and test data is cleaned up.+- Check tests against `.agents/skills/testing/SKILL.md` and the testing rules of the module docs: new behavior has a test at the level the module doc prefers, test names follow the module doc, tests wait for conditions instead of fixed pauses, and test data is cleaned up.- Check that code, comments, log messages, API strings and documentation are in English, except the places "Language in code" in `AGENTS.md` lists.- Check documentation consistency per the section "Documentation consistency" in `AGENTS.md`: if the change touches a command, rule, path or order of steps that `AGENTS.md`, the module docs, `adr/`, `docs/`, the repository's own skills or a README describe, those places are updated too. Search the repository for mentions of what changed. `openspec/` is outside this check.- Check edits to agent instructions per the same section. If an edit adds to `AGENTS.md` or an own skill an inventory of what exists, the state of the code, a description of how something works or a retelling of another document, return `ok: false` and name the place to move it to from that section's table. Do not block an edit that adds or changes a rule of agent behavior under this point.diff --git a/.agent-hooks/implementation-prompt.md b/.agent-hooks/implementation-prompt.mdindex eea6c2e1..177dcfa1 100644--- a/.agent-hooks/implementation-prompt.md+++ b/.agent-hooks/implementation-prompt.md@@ -1,6 +1,6 @@## Full implementation review-The candidate is the implementation of an OpenSpec change whose tasks in `tasks.md` are all checked. Read the change's `proposal.md`, `design.md`, `tasks.md` and delta specs. The hook already ran the linters of the touched modules, and the hook tests when the hooks changed, and they passed. The hook does not run the Go test suites: check the agent's claims about tests against the repository.+The candidate is the implementation of an OpenSpec change whose tasks in `tasks.md` are all checked. Read the change's `proposal.md`, `design.md`, `tasks.md` and delta specs. The hook already ran the linters of the touched modules, the hook tests when the hooks changed, and the commands of the "Change verification" section at the end of `tasks.md`, and they all passed. Tests that no command in that section runs did not run here: check the agent's claims about them against the repository.An implementation usually spans several requests, so the shared rule about paths changed before the latest request does not apply here: review everything that belongs to the change. Part of the implementation may already be committed; find it with `git log` and `git show` over the paths from the specs and `design.md`. Do not review edits unrelated to the change.@@ -28,6 +28,18 @@ Review only the requirements of your delta spec.- Check documentation consistency per the section "Documentation consistency" in `AGENTS.md`: `AGENTS.md`, the module docs, `adr/`, `docs/`, the repository's own skills, the READMEs and code comments agree with each other and with the code. Search the repository for mentions of what the change changed.- Check the edits to agent instructions the change made, per the same section. Report an inventory of what exists, the state of the code, a description of how something works or a retelling of another document in `AGENTS.md` or an own skill as a `docs` finding, and name the place to move it to from that section's table. A rule of agent behavior that is written nowhere else is not a finding.+### Role: tests++Read `.agents/skills/testing/SKILL.md` and the testing rules of the module doc for each area whose tests the change touched. Review every test the change added or changed:++- The test fails without the behavior it pins. A test that would pass with that behavior removed is a `missing_test` finding.+- A bug fix has a test that reproduces the bug and fails on the code before the fix. Without one, return a `missing_test` finding.+- The test covers the cases the module doc asks for. A missing case is a `missing_test` finding.+- The test waits for a condition up to a limit instead of a fixed pause, and it cleans up the data it created. A fixed pause, a flaky assertion or leaked data is a `bug` finding that names the condition to wait for or the data to clean up.+- The commands of the "Change verification" section run the tests of every backend package, verification agent package and frontend test file in which the change added or changed tests. A package or file left out is a `missing_test` finding that names it. Tests that no allowed command form runs and that a module check already runs, such as the hook tests in `.agent-hooks/`, count as covered, so a change with only such tests needs no section.++Do not ask for the full backend suite (`just test`) to be run.+### Role: repeat reviewThe previous review of this change blocked the turn, and the agent fixed the findings. The previous findings and the paths changed after the previous review are passed below.diff --git a/.agent-hooks/proposal-prompt.md b/.agent-hooks/proposal-prompt.mdindex 5fed01bb..96952b72 100644--- a/.agent-hooks/proposal-prompt.md+++ b/.agent-hooks/proposal-prompt.md@@ -1,6 +1,6 @@## Full proposal review-The candidate is an OpenSpec proposal whose artifacts are all written and none of whose tasks in `tasks.md` is checked. No code exists for it yet. Read the change's `proposal.md`, `design.md`, `tasks.md` and delta specs, and the main specs under `openspec/specs` the change touches. The hook already ran `openspec validate --strict`, and it passed.+The candidate is an OpenSpec proposal whose artifacts are all written and none of whose tasks in `tasks.md` is checked. No code exists for it yet. Read the change's `proposal.md`, `design.md`, `tasks.md` and delta specs, and the main specs under `openspec/specs` the change touches. The hook already checked the format of the "Change verification" section at the end of `tasks.md` and ran `openspec validate --strict`, and both passed.A proposal is often written over several requests, so the shared rule about paths changed before the latest request does not apply here: review every artifact of the change. Do not review other files in the working tree.@@ -37,6 +37,7 @@ Review only the requirements of your delta spec.- Check that every task has a verification: a test, a command or a repository search.- Check that every task closes without the user and without the full backend suite. Local tests of the touched packages and of the packages that test the same feature are enough. A task that runs `just test` or `just test-filesystem`, or waits on the user, CI or a push, is a `tasks` finding. Never ask for the full suite to be added.- Check that the task order can be followed: no task relies on something that appears only in a later one.+- Check that a change whose tasks add or change tests in a backend package, a verification agent package or the frontend has a `## Change verification` section, and that its commands run the tests of every such package the tasks' verifications name. A missing section, or a package the section leaves out, is a `tasks` finding that names the package. Tests that no allowed command form runs and that a module check already runs, such as the hook tests in `.agent-hooks/`, need no section, and neither does a change without tests.### Role: repeat reviewdiff --git a/.agent-hooks/review_gate.py b/.agent-hooks/review_gate.pyindex 5c51213d..c209aca4 100755--- a/.agent-hooks/review_gate.py+++ b/.agent-hooks/review_gate.py@@ -2,6 +2,10 @@"""Review hooks for Claude Code and Codex, wired in .claude/settings.json and .codex/hooks.json.Usage: review_gate.py <action> --client <claude|codex>+ review_gate.py doctor++The clients start it through run.sh, which runs it with the Python and OpenSpec versions pinned+in mise.toml and stops the agent with the reason when the hook cannot start or crashes.Actions:session-start SessionStart: tells the agent to load the communication skill in a fresh context.@@ -14,17 +18,24 @@ Actions:and the module docs. Runs when a .go, .ts or .tsx file under backend/, agent/,frontend/src/ or website/ changed.implementation Full review of an OpenSpec change whose tasks are all checked: first the read-only- checks of the touched modules, then parallel reviewers by spec and for documentation,+ checks of the touched modules, then the commands of the "Change verification"+ section of tasks.md, then parallel reviewers by spec, for documentation and for tests,after a block a repeat over the previous findings only.- proposal Full review of a finished OpenSpec proposal with no checked task: first+ proposal Full review of a finished OpenSpec proposal with no checked task: first the format+ check of the "Change verification" section without running it, thenopenspec validate --strict, then parallel reviewers by spec, for proposal and designand for tasks. While it runs, necessity, diff, architecture and naming skip a turnthat only wrote the artifacts.+ doctor Reports whether Python, git, openspec and a reviewer CLI are found, one line per tool,+ and exits with a non-zero code when one is missing. `just doctor` runs it.++Both full reviews run within FULL_REVIEW_BUDGET_SECONDS from the hook's start: the checks share+one total limit, and a reviewer starts only when the budget still holds its whole limit.Every review runs a separate read-only process of the same agent.The review prompt lives in .agent-hooks/<action>-prompt.md, the shared part in common-prompt.md:they say what the reviewer reads and checks.-A review that could not run never accepts the result.+A review that could not run never accepts the result, and a missing openspec stops the agent."""import argparse@@ -36,9 +47,13 @@ import jsonimport osfrom pathlib import Pathimport re+import shlex+import shutil+import signalimport subprocessimport sysimport tempfile+import timeAUTOMATIC_REVIEW_PREFIX = "AUTOMATIC REVIEW:"@@ -46,6 +61,13 @@ MAX_REVISIONS = 3MAX_FULL_REVIEW_REVISIONS = 2MAX_PARALLEL_REVIEWERS = 6CHECK_TIMEOUT_SECONDS = 900+CHECKS_TOTAL_TIMEOUT_SECONDS = 1800+KILLED_CHECK_OUTPUT_TIMEOUT_SECONDS = 10+# Both full review hooks have a 7200-second client timeout. The budget leaves 600 seconds for Git,+# openspec status, prompts and result handling, and still fits the checks' total limit plus two+# waves of reviewers, so the client never kills a review without a message.+FULL_REVIEW_BUDGET_SECONDS = 6600+BUDGET_EXHAUSTED_ERROR = "the review time budget ran out before this reviewer started"CHECK_OUTPUT_LINES = 80OPENSPEC_TIMEOUT_SECONDS = 60REVIEWER_ACTIVE_ENV = "AGENT_REVIEW_HOOK_ACTIVE"@@ -57,6 +79,12 @@ CODE_SUFFIXES = (".go", ".ts", ".tsx")TASK_PATTERN = re.compile(r"^\s*[-*] \[([ xX])\]", re.MULTILINE)+class MissingToolError(Exception):+ def __init__(self, tool):+ super().__init__(tool)+ self.tool = tool++@dataclass(frozen=True)class Reviewer:claude_model: str@@ -115,7 +143,8 @@ class Check:# The hook only runs checks that leave files untouched: a formatter would rewrite files a parallel-# agent is editing. `just test` resets the test containers other agents share, so it never runs here.+# agent is editing. Tests run only through the change verification commands a change names, and+# their forms leave out `just test`, which resets the test containers other agents share.MODULE_CHECKS = (("backend/", (Check("backend", ("golangci-lint", "run", "./cmd/...", "./internal/...")),)),(@@ -135,6 +164,140 @@ MODULE_CHECKS = ()+class ChangeVerificationError(Exception):+ pass+++@dataclass(frozen=True)+class VerificationForm:+ prefix: tuple+ directory: str+ usage: str+ min_arguments: int+ max_arguments: int | None+ is_package_list: bool+++CHANGE_VERIFICATION_HEADING = "## Change verification"+VERIFICATION_FORMS = (+ VerificationForm(+ prefix=("just", "test-package"),+ directory="backend",+ usage="just test-package <package>...",+ min_arguments=1,+ max_arguments=None,+ is_package_list=True,+ ),+ VerificationForm(+ prefix=("just", "--justfile", "agent/verification/Justfile", "test"),+ directory=".",+ usage="just --justfile agent/verification/Justfile test",+ min_arguments=0,+ max_arguments=0,+ is_package_list=False,+ ),+ VerificationForm(+ prefix=("pnpm", "--dir", "frontend", "test"),+ directory=".",+ usage="pnpm --dir frontend test [<argument>...]",+ min_arguments=0,+ max_arguments=None,+ is_package_list=False,+ ),+)+# The commands run outside the agent's permission prompts, and both runners pass the arguments+# through a shell: `just test-package` interpolates them into a bash recipe. The set leaves out+# every shell operator, redirection and substitution character, so `./internal/a;rm` is refused+# although it holds no separate operator word.+VERIFICATION_ARGUMENT_PATTERN = re.compile(r"[A-Za-z0-9._/=:@-]+")+ALLOWED_VERIFICATION_FORMS = (+ "Allowed forms, one command per line in one ```sh block:\n"+ "- `just test-package <package>...`, every package starting with `./`, runs in `backend/`\n"+ "- `just --justfile agent/verification/Justfile test`, with nothing after it\n"+ "- `pnpm --dir frontend test [<argument>...]`\n"+ "Arguments may hold only letters, digits and `.`, `/`, `_`, `-`, `=`, `:`, `@`."+)+++def read_change_verification_block(tasks_text):+ lines = tasks_text.splitlines()+ try:+ start = next(index for index, line in enumerate(lines) if line.strip() == CHANGE_VERIFICATION_HEADING)+ except StopIteration:+ return None++ # A shell comment inside a code block looks like a heading, so only a heading outside blocks ends the section.+ is_inside_other_block = False+ for index in range(start + 1, len(lines)):+ line = lines[index].strip()+ if line == "```sh" and not is_inside_other_block:+ return read_code_block(lines[index + 1 :])+ if line.startswith("```"):+ is_inside_other_block = not is_inside_other_block+ elif not is_inside_other_block and re.match(r"#{1,2} ", line):+ break+ raise ChangeVerificationError("the section has no ```sh code block")+++def read_code_block(lines):+ for index, line in enumerate(lines):+ if line.strip() == "```":+ return lines[:index]+ raise ChangeVerificationError("the ```sh code block is not closed")+++def parse_verification_command(line):+ try:+ words = shlex.split(line)+ except ValueError as error:+ raise ChangeVerificationError(f"`{line}` cannot be split into words: {error}")++ for form in VERIFICATION_FORMS:+ if tuple(words[: len(form.prefix)]) != form.prefix:+ continue++ arguments = words[len(form.prefix) :]+ for argument in arguments:+ if not VERIFICATION_ARGUMENT_PATTERN.fullmatch(argument):+ raise ChangeVerificationError(f"the argument `{argument}` in `{line}` has a forbidden character")+ if len(arguments) < form.min_arguments or (form.max_arguments is not None and len(arguments) > form.max_arguments):+ raise ChangeVerificationError(f"`{line}` does not match the form `{form.usage}`")+ if form.is_package_list and not all(argument.startswith("./") for argument in arguments):+ raise ChangeVerificationError(f"every package in `{line}` must start with `./`")+ return Check(form.directory, tuple(words))+ raise ChangeVerificationError(f"`{line}` is not an allowed command")+++def parse_change_verification(change_dir):+ try:+ tasks_text = (change_dir / "tasks.md").read_text()+ except OSError:+ return []++ try:+ block = read_change_verification_block(tasks_text)+ if block is None:+ return []+ commands = [line.strip() for line in block if line.strip() and not line.strip().startswith("#")]+ if not commands:+ raise ChangeVerificationError("the ```sh code block holds no command")+ return [parse_verification_command(line) for line in commands]+ except ChangeVerificationError as error:+ raise ChangeVerificationError(+ f"The \"Change verification\" section of `{change_dir.name}/tasks.md` is not valid: {error}.\n\n"+ f"{ALLOWED_VERIFICATION_FORMS}"+ ) from None+++def get_change_verification_checks(targets):+ checks = []+ for target in targets:+ for check in parse_change_verification(target.change_dir):+ if check not in checks:+ checks.append(check)+ return checks++def read_payload():try:value = json.load(sys.stdin)@@ -670,7 +833,8 @@ def is_unstarted(change_dir):# The CLI decides whether the artifacts are complete: it knows the change's schema, an optional-# design.md and skip_specs. Without the CLI the proposal review does not run, and the quick checks do.+# design.md and skip_specs. A CLI that cannot start raises MissingToolError, because skipping+# the proposal review would let an unreviewed proposal pass silently.def is_planning_complete(root, change_name):try:process = subprocess.run(@@ -682,8 +846,13 @@ def is_planning_complete(root, change_name):stderr=subprocess.PIPE,timeout=OPENSPEC_TIMEOUT_SECONDS,)+ except OSError as error:+ raise MissingToolError("openspec") from error+ except subprocess.TimeoutExpired:+ return False+ try:status = json.loads(process.stdout)- except (OSError, subprocess.TimeoutExpired, json.JSONDecodeError):+ except json.JSONDecodeError:return Falsereturn process.returncode == 0 and isinstance(status, dict) and status.get("isPlanningComplete") is True@@ -749,26 +918,54 @@ def get_output_tail(output):return "\n".join((output or "").rstrip().splitlines()[-CHECK_OUTPUT_LINES:]) or "(no output)"-def run_checks(root, checks):+# `just` starts bash, bash starts `go test`, and `go test` starts test binaries that hold a test slot+# database and containers. Each check runs in its own process group, so a timeout kills them all.+def stop_process_group(process):+ try:+ os.killpg(process.pid, signal.SIGKILL)+ except ProcessLookupError:+ pass+ try:+ return process.communicate(timeout=KILLED_CHECK_OUTPUT_TIMEOUT_SECONDS)[0]+ except subprocess.TimeoutExpired as error:+ # A descendant that left the group still holds the pipe, so the output read so far is all there is.+ # TimeoutExpired carries it as bytes even for a text-mode process; get_output_tail decodes it.+ process.stdout.close()+ process.wait()+ return error.output+++def run_checks(root, checks, check_timeout_seconds=CHECK_TIMEOUT_SECONDS, total_timeout_seconds=CHECKS_TOTAL_TIMEOUT_SECONDS):+ deadline = time.monotonic() + total_timeout_secondsfor check in checks:command = f"`{' '.join(check.arguments)}` in `{check.directory}`"+ remaining_seconds = max(deadline - time.monotonic(), 0)+ if check_timeout_seconds <= remaining_seconds:+ timeout_seconds, exceeded_limit = check_timeout_seconds, f"its limit of {check_timeout_seconds} seconds per check"+ else:+ timeout_seconds, exceeded_limit = remaining_seconds, f"the total limit of {total_timeout_seconds} seconds for all checks"+try:- process = subprocess.run(+ process = subprocess.Popen(list(check.arguments),cwd=root / check.directory,text=True,errors="replace",stdout=subprocess.PIPE,stderr=subprocess.STDOUT,- timeout=CHECK_TIMEOUT_SECONDS,+ start_new_session=True,)- except subprocess.TimeoutExpired as error:- return f"{command} did not finish in {CHECK_TIMEOUT_SECONDS} seconds.\n\nEnd of output:\n{get_output_tail(error.output)}"except OSError as error:return f"{command} could not start: {error}"+ try:+ output = process.communicate(timeout=timeout_seconds)[0]+ except subprocess.TimeoutExpired:+ output = stop_process_group(process)+ return f"{command} did not finish within {exceeded_limit} and was stopped with every process it started.\n\nEnd of output:\n{get_output_tail(output)}"+if process.returncode != 0:- return f"{command} exited with code {process.returncode}.\n\nEnd of output:\n{get_output_tail(process.stdout)}"+ return f"{command} exited with code {process.returncode}.\n\nEnd of output:\n{get_output_tail(output)}"return None@@ -892,7 +1089,9 @@ def parse_findings(full_review, value):return [{"category": finding["category"], "file": finding["file"], "summary": finding["summary"]} for finding in findings]-def run_review_job(root, client, full_review, job):+def run_review_job(root, client, full_review, job, deadline):+ if deadline - time.monotonic() < full_review.reviewer.timeout_seconds:+ return [], describe_reviewer_error(BUDGET_EXHAUSTED_ERROR)try:return parse_findings(full_review, run_reviewer(root, job.prompt, client, full_review.reviewer)), Noneexcept (OSError, ValueError, RuntimeError, subprocess.TimeoutExpired) as error:@@ -916,14 +1115,26 @@ def block_full_review(session, full_review, report, instruction):emit({"decision": "block", "reason": f"{AUTOMATIC_REVIEW_PREFIX} {full_review.gate}\n{report}\n\n{instruction}"})-def gate_full_review(session, client, full_review):+def gate_full_review(session, client, full_review, deadline):root = session.roottargets = full_review.select_targets(session)if not targets:emit({})return- check_failure = run_checks(root, full_review.get_checks(root, targets))+ try:+ checks = full_review.get_checks(root, targets)+ except ChangeVerificationError as error:+ block_full_review(+ session,+ full_review,+ f"{error}\n\nNo check or reviewer ran.",+ "Fix the section. " + FULL_REVIEW_REPLY_INSTRUCTION,+ )+ return++ checks_timeout_seconds = min(CHECKS_TOTAL_TIMEOUT_SECONDS, max(deadline - time.monotonic(), 0))+ check_failure = run_checks(root, checks, total_timeout_seconds=checks_timeout_seconds)if check_failure:block_full_review(session,@@ -935,7 +1146,7 @@ def gate_full_review(session, client, full_review):jobs = [job for target in targets for job in build_review_jobs(session, full_review, target)]with ThreadPoolExecutor(max_workers=MAX_PARALLEL_REVIEWERS) as executor:- results = list(executor.map(lambda job: run_review_job(root, client, full_review, job), jobs))+ results = list(executor.map(lambda job: run_review_job(root, client, full_review, job, deadline), jobs))reports = []has_blocking = False@@ -1029,6 +1240,19 @@ def select_proposal_targets(session):return targets+# Every section is parsed before the first check, so a malformed one blocks the turn before anything runs.+def get_implementation_checks(root, targets):+ verification_checks = get_change_verification_checks(targets)+ return [*get_module_checks(root), *verification_checks]+++# The proposal review checks the section's format only: no code exists yet for its commands to test.+def get_proposal_checks(root, targets):+ for target in targets:+ parse_change_verification(target.change_dir)+ return [Check(".", ("openspec", "validate", target.change_dir.name, "--strict")) for target in targets]++IMPLEMENTATION_REVIEW = FullReview(gate="implementation",title="Full implementation review",@@ -1038,10 +1262,10 @@ IMPLEMENTATION_REVIEW = FullReview(non_blocking_categories=("minor",),summary_categories=(),category_notes={"minor": " (optional)"},- first_roles=("documentation and tasks",),+ first_roles=("documentation and tasks", "tests"),files_title="Paths changed relative to HEAD",select_targets=select_implementation_targets,- get_checks=lambda root, targets: get_module_checks(root),+ get_checks=get_implementation_checks,block_instruction="Resolve every finding yourself, without asking the user. Fix a bug in the code. ""On a spec mismatch, pick the right side and fix the other one, including the change's delta spec. ""Reject a false finding. Minor findings are optional. "@@ -1064,9 +1288,7 @@ PROPOSAL_REVIEW = FullReview(first_roles=("proposal and design", "tasks"),files_title="Change files",select_targets=select_proposal_targets,- get_checks=lambda root, targets: [- Check(".", ("openspec", "validate", target.change_dir.name, "--strict")) for target in targets- ],+ get_checks=get_proposal_checks,block_instruction="Resolve every gap, mismatch, tasks and simplify finding yourself, without asking the user: ""fix the change's artifacts, and reject a false finding. ""Do not apply scope findings: cutting the feature needs the user's consent, and the user sees them in the summary. "@@ -1081,34 +1303,72 @@ PROPOSAL_REVIEW = FullReview(FULL_REVIEWS = {full_review.gate: full_review for full_review in (IMPLEMENTATION_REVIEW, PROPOSAL_REVIEW)}+def check_environment():+ tool_paths = {+ "python": sys.executable,+ "git": shutil.which("git"),+ "openspec": shutil.which("openspec"),+ "claude": shutil.which("claude"),+ "codex": shutil.which("codex"),+ }+ for tool, path in tool_paths.items():+ print(f"{tool}: {path or 'not found'}")++ is_missing_tool = not all(tool_paths[tool] for tool in ("python", "git", "openspec"))+ is_reviewer_missing = not (tool_paths["claude"] or tool_paths["codex"])+ if is_reviewer_missing:+ print("no reviewer CLI found: the review hooks need claude or codex")+ return 1 if is_missing_tool or is_reviewer_missing else 0+++def stop_for_missing_tool(error):+ message = (+ f"The review hooks could not find `{error.tool}`, so the review did not run. "+ "Run `mise install` to install the pinned tools and `just doctor` to check the environment."+ )+ emit({"continue": False, "stopReason": message, "systemMessage": message})++def main():+ deadline = time.monotonic() + FULL_REVIEW_BUDGET_SECONDSparser = argparse.ArgumentParser()- parser.add_argument("action", choices=("session-start", "snapshot", *GATES))- parser.add_argument("--client", choices=("claude", "codex"), required=True)+ parser.add_argument("action", choices=("session-start", "snapshot", "doctor", *GATES))+ parser.add_argument("--client", choices=("claude", "codex"))arguments = parser.parse_args()+ if arguments.action == "doctor":+ sys.exit(check_environment())+ if arguments.client is None:+ parser.error(f"--client is required for {arguments.action}")++ try:+ run_action(arguments.action, arguments.client, deadline)+ except MissingToolError as error:+ stop_for_missing_tool(error)++def run_action(action, client, deadline):payload = read_payload()if os.environ.get(REVIEWER_ACTIVE_ENV) == "1":emit({})return- if arguments.action == "session-start":+ if action == "session-start":start_session()returnsession = Session(find_root(payload), payload)- if arguments.action == "snapshot":+ if action == "snapshot":snapshot(session)return- if arguments.action in FULL_REVIEWS:- gate_full_review(session, arguments.client, FULL_REVIEWS[arguments.action])+ if action in FULL_REVIEWS:+ gate_full_review(session, client, FULL_REVIEWS[action], deadline)returnevent = get_field(payload, "hook_event_name", "hookEventName")if event == "PreToolUse":- gate_plan_tool(session, arguments.action, arguments.client)+ gate_plan_tool(session, action, client)return- gate_stop(session, arguments.action, arguments.client)+ gate_stop(session, action, client)if __name__ == "__main__":diff --git a/.agent-hooks/review_gate_test.py b/.agent-hooks/review_gate_test.pyindex 6ae7db97..c40a3b41 100644--- a/.agent-hooks/review_gate_test.py+++ b/.agent-hooks/review_gate_test.py@@ -1,11 +1,19 @@+from dataclasses import replace+import ioimport jsonimport os+import shutil+import signalfrom pathlib import Pathimport subprocessimport sysimport tempfileimport textwrap+import timeimport unittest+from unittest import mock++import review_gateHOOK_PATH = Path(__file__).resolve().parent / "review_gate.py"@@ -16,13 +24,14 @@ STUB_NAMES = ("claude", "codex", "openspec", "golangci-lint", "pnpm", "npm", "ju# The stub stands in for every external command: it takes its answer from <stubs>/<name>.json# and records each call in <stubs>/calls, so a test sees the arguments, the working directory# and the prompt. openspec answers by its first argument (openspec.json["status"]),-# npm by its script name (npm.json["build"]).+# npm by its script name (npm.json["build"]). sleep_seconds delays the answer.STUB_SOURCE = textwrap.dedent("""import jsonimport osfrom pathlib import Pathimport sys+ import timeimport uuidstubs_dir = Path(os.environ["REVIEW_GATE_STUBS"])@@ -39,6 +48,7 @@ STUB_SOURCE = textwrap.dedent("cwd": os.getcwd(),"prompt": prompt,"effort_env": os.environ.get("CLAUDE_CODE_EFFORT_LEVEL"),+ "started_at": time.time_ns(),}))@@ -50,6 +60,7 @@ STUB_SOURCE = textwrap.dedent(elif name == "npm":config = config.get(arguments[-1], {})+ time.sleep(config.get("sleep_seconds", 0))if config.get("exit_code"):sys.stdout.write(config.get("output", ""))sys.exit(config["exit_code"])@@ -88,8 +99,63 @@ FAILING_HOOK_TEST = textwrap.dedent()+# The stub stands in for mise in the launcher tests: it records the subcommand, its arguments,+# the environment variable that forbids installs and its stdin to <stubs>/mise-calls.json, then+# answers from <stubs>/mise.json by subcommand (mise.json["ls"]). Like real mise, `ls --missing`+# prints nothing when every pinned version is installed, and `exec` runs even when one is missing.+MISE_STUB_SOURCE = textwrap.dedent(+ """+ import json+ import os+ from pathlib import Path+ import sys++ stubs_dir = Path(os.environ["REVIEW_GATE_STUBS"])+ subcommand = sys.argv[3]+ calls_path = stubs_dir / "mise-calls.json"+ calls = json.loads(calls_path.read_text()) if calls_path.exists() else []+ calls.append(+ {+ "subcommand": subcommand,+ "arguments": sys.argv[1:],+ "exec_auto_install": os.environ.get("MISE_EXEC_AUTO_INSTALL"),+ "stdin": sys.stdin.read() if subcommand == "exec" else "",+ }+ )+ calls_path.write_text(json.dumps(calls))++ config_path = stubs_dir / "mise.json"+ config = json.loads(config_path.read_text()) if config_path.exists() else {}+ answer = config.get(subcommand, {})+ sys.stdout.write(answer.get("stdout", "{}" if subcommand == "exec" else ""))+ sys.stderr.write(answer.get("stderr", ""))+ sys.exit(answer.get("exit_code", 0))+ """+)+# The check starts a child that outlives it and writes the child's pid to the path in its argument.+CHECK_WITH_CHILD_SOURCE = textwrap.dedent(+ """+ from pathlib import Path+ import subprocess+ import sys++ child = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(60)"])+ Path(sys.argv[1]).write_text(str(child.pid))+ child.wait()+ """+)+# The same, but the child starts its own session, so it leaves the check's process group and keeps the output pipe open.+CHECK_WITH_ESCAPED_CHILD_SOURCE = CHECK_WITH_CHILD_SOURCE.replace(+ '"import time; time.sleep(60)"])', '"import time; time.sleep(60)"], start_new_session=True)'+)+LAUNCHER_PATH = HOOK_PATH.parent / "run.sh"+LAUNCHER_COMMANDS = ("bash", "mktemp", "tail", "tr", "rm")++class HookRepository:- def __init__(self, test_case):+ # A missing tool or an isolated PATH takes the machine's PATH away, so a copy installed there+ # stays out of reach; only git is linked in.+ def __init__(self, test_case, missing_tools=(), is_path_isolated=False):temporary_dir = tempfile.TemporaryDirectory(prefix="review-gate-test-")test_case.addCleanup(temporary_dir.cleanup)base = Path(temporary_dir.name)@@ -101,6 +167,8 @@ class HookRepository:directory.mkdir()for name in STUB_NAMES:+ if name in missing_tools:+ continuestub = bin_dir / namestub.write_text(f"#!{sys.executable}\n{STUB_SOURCE}")stub.chmod(0o755)@@ -110,7 +178,13 @@ class HookRepository:for name, value in os.environ.items()if not name.startswith("GIT_") and name != "AGENT_REVIEW_HOOK_ACTIVE"}- self.environment["PATH"] = f"{bin_dir}{os.pathsep}{os.environ['PATH']}"+ if missing_tools or is_path_isolated:+ system_dir = base / "system-bin"+ system_dir.mkdir()+ (system_dir / "git").symlink_to(shutil.which("git"))+ self.environment["PATH"] = f"{bin_dir}{os.pathsep}{system_dir}"+ else:+ self.environment["PATH"] = f"{bin_dir}{os.pathsep}{os.environ['PATH']}"self.environment["REVIEW_GATE_STUBS"] = str(self.stubs_dir)self.environment["PYTHONDONTWRITEBYTECODE"] = "1"@@ -141,8 +215,8 @@ class HookRepository:def write_proposal(self, spec_names=("gadgets",), tasks="- [ ] 1.1 Build\n- [ ] 1.2 Verify\n"):self.write_change(spec_names=spec_names, tasks=tasks, name=PROPOSAL_NAME)- def complete_tasks(self, name=CHANGE_NAME):- self.write(f"openspec/changes/{name}/tasks.md", "# Tasks\n\n- [x] 1.1 Build\n- [x] 1.2 Verify\n")+ def complete_tasks(self, name=CHANGE_NAME, verification_section=""):+ self.write(f"openspec/changes/{name}/tasks.md", f"# Tasks\n\n- [x] 1.1 Build\n- [x] 1.2 Verify\n{verification_section}")def set_reviewer_result(self, result):for name in ("claude", "codex"):@@ -184,6 +258,36 @@ class HookRepository:)return json.loads(process.stdout) if process.stdout.strip() else None+ def run_doctor(self):+ return subprocess.run(+ [sys.executable, str(HOOK_PATH), "doctor"],+ cwd=self.root,+ env=self.environment,+ stdin=subprocess.DEVNULL,+ text=True,+ capture_output=True,+ )++ # Runs the hook in this process, so a test can shrink the time limits.+ def stop_in_process(self, action, budget_seconds, reviewer_timeout_seconds, checks_timeout_seconds=None):+ payload = {"hook_event_name": "Stop", "session_id": "session-1", "last_assistant_message": "Done.", "cwd": str(self.root)}+ full_review = review_gate.FULL_REVIEWS[action]+ shrunk_review = replace(full_review, reviewer=replace(full_review.reviewer, timeout_seconds=reviewer_timeout_seconds))+ stdout = io.StringIO()+ with (+ mock.patch.dict(os.environ, self.environment, clear=True),+ mock.patch.dict(review_gate.FULL_REVIEWS, {action: shrunk_review}),+ mock.patch.object(review_gate, "FULL_REVIEW_BUDGET_SECONDS", budget_seconds),+ mock.patch.object(+ review_gate, "CHECKS_TOTAL_TIMEOUT_SECONDS", checks_timeout_seconds or review_gate.CHECKS_TOTAL_TIMEOUT_SECONDS+ ),+ mock.patch.object(sys, "argv", ["review_gate.py", action, "--client", "claude"]),+ mock.patch.object(sys, "stdin", io.StringIO(json.dumps(payload))),+ mock.patch.object(sys, "stdout", stdout),+ ):+ review_gate.main()+ return json.loads(stdout.getvalue())+def submit_prompt(self, prompt, session_id="session-1", client="claude"):self.run_hook("snapshot", {"hook_event_name": "UserPromptSubmit", "session_id": session_id, "prompt": prompt}, client)@@ -196,6 +300,43 @@ def finding(category, summary, file="src/widgets.py:10"):return {"category": category, "file": file, "summary": summary}+def verification_section(*lines, fence="sh"):+ body = "".join(f"{line}\n" for line in lines)+ return f"\n## Change verification\n\n```{fence}\n{body}```\n"+++def is_process_gone(pid):+ try:+ os.kill(pid, 0)+ except ProcessLookupError:+ return True+ try:+ return Path(f"/proc/{pid}/stat").read_text().split(")")[-1].split()[0] == "Z"+ except OSError:+ return True+++def wait_until_process_gone(pid, timeout_seconds=5):+ deadline = time.monotonic() + timeout_seconds+ while time.monotonic() < deadline:+ if is_process_gone(pid):+ return True+ time.sleep(0.05)+ return False+++# With these limits a wave of reviewers that sleep 5.5 seconds fits the budget twice but not a third time,+# with about 2 seconds of margin on each side for the hook's own setup.+THIRD_WAVE_BUDGET_SECONDS = 14+THIRD_WAVE_REVIEWER_TIMEOUT_SECONDS = 6+THIRD_WAVE_REVIEWER_SECONDS = 5.5+# These keep the production relationship, budget >= checks total + 2 x reviewer limit, while the check+# and the reviewers use almost all of their limits.+TWO_WAVES_CHECKS_TIMEOUT_SECONDS = 2+TWO_WAVES_REVIEWER_TIMEOUT_SECONDS = 4+TWO_WAVES_BUDGET_SECONDS = TWO_WAVES_CHECKS_TIMEOUT_SECONDS + 2 * TWO_WAVES_REVIEWER_TIMEOUT_SECONDS + 2++class ShortReplyAssertions:def assert_asks_for_short_reply(self, reason):self.assertIn("one short line about your fixes that names every rejected finding with the reason", reason)@@ -219,6 +360,367 @@ class SessionStartTest(unittest.TestCase):self.assertIn(".agents/skills/humanizer/SKILL.md", context)+class Launcher:+ def __init__(self, test_case):+ temporary_dir = tempfile.TemporaryDirectory(prefix="review-gate-launcher-test-")+ test_case.addCleanup(temporary_dir.cleanup)+ base = Path(temporary_dir.name)++ self.home = base / "home"+ self.stubs_dir = base / "stubs"+ self.bin_dir = base / "bin"+ for directory in (self.home, self.stubs_dir, self.bin_dir):+ directory.mkdir()++ # PATH holds only what the launcher needs before mise, so a mise installed on this machine stays out of reach.+ for name in LAUNCHER_COMMANDS:+ (self.bin_dir / name).symlink_to(shutil.which(name))++ self.environment = {+ "PATH": str(self.bin_dir),+ "HOME": str(self.home),+ "REVIEW_GATE_STUBS": str(self.stubs_dir),+ }+ self.mise_answers = {}++ def install_mise(self, directory):+ directory.mkdir(parents=True, exist_ok=True)+ stub = directory / "mise"+ stub.write_text(f"#!{sys.executable}\n{MISE_STUB_SOURCE}")+ stub.chmod(0o755)++ def set_mise_answer(self, subcommand, **answer):+ self.mise_answers[subcommand] = answer+ (self.stubs_dir / "mise.json").write_text(json.dumps(self.mise_answers))++ def mise_calls(self, subcommand):+ calls_path = self.stubs_dir / "mise-calls.json"+ calls = json.loads(calls_path.read_text()) if calls_path.exists() else []+ return [call for call in calls if call["subcommand"] == subcommand]++ def run(self, *arguments, stdin=""):+ return subprocess.run(+ [str(LAUNCHER_PATH), *arguments],+ env=self.environment,+ input=stdin,+ text=True,+ capture_output=True,+ )+++MISSING_OPENSPEC_LINE = "npm:@fission-ai/openspec 1.11.0 (missing) mise.toml 1.11.0"+++class LauncherTest(unittest.TestCase):+ def assert_stops_agent(self, process, *fragments):+ self.assertEqual(process.returncode, 0)+ response = json.loads(process.stdout)+ self.assertIs(response["continue"], False)+ for fragment in fragments:+ self.assertIn(fragment, response["stopReason"])+ self.assertIn(fragment, response["systemMessage"])+ return response++ def test_runs_hook_through_mise_from_home_local_bin_without_installs_when_path_has_no_mise(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.home / ".local" / "bin")+ payload = json.dumps({"hook_event_name": "Stop", "session_id": "session-1"})++ process = launcher.run("naming", "--client", "claude", stdin=payload)++ self.assertEqual(process.returncode, 0)+ self.assertEqual(json.loads(process.stdout), {})+ root = str(HOOK_PATH.parent.parent)+ [missing_check] = launcher.mise_calls("ls")+ self.assertEqual(+ missing_check["arguments"],+ ["-C", root, "ls", "--current", "--missing", "--no-truncate", "python", "npm:@fission-ai/openspec"],+ )+ [call] = launcher.mise_calls("exec")+ self.assertEqual(call["arguments"], ["-C", root, "exec", "--", "python3", str(HOOK_PATH), "naming", "--client", "claude"])+ self.assertEqual(call["exec_auto_install"], "false")+ self.assertEqual(call["stdin"], payload)++ def test_stops_agent_without_running_hook_when_pinned_version_is_missing(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("ls", stdout=f"{MISSING_OPENSPEC_LINE}\n")++ process = launcher.run("proposal", "--client", "claude")++ self.assert_stops_agent(process, "not installed", MISSING_OPENSPEC_LINE, "mise install", "just doctor")+ self.assertEqual(launcher.mise_calls("exec"), [])++ def test_stops_agent_without_running_hook_when_missing_tools_check_fails(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("ls", stdout="", stderr="mise ERROR Config file mise.toml is not trusted\n", exit_code=1)++ process = launcher.run("diff", "--client", "claude")++ self.assert_stops_agent(process, "mise exited with code 1", "is not trusted", "just doctor")+ self.assertEqual(launcher.mise_calls("exec"), [])++ def test_stops_agent_when_version_manager_is_missing(self):+ launcher = Launcher(self)++ process = launcher.run("diff", "--client", "claude")++ self.assert_stops_agent(process, "could not find the version manager mise", "just doctor")++ def test_stops_agent_with_action_and_exit_code_when_hook_crashes(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("exec", stdout="", stderr="Traceback: KeyError\n", exit_code=3)++ process = launcher.run("implementation", "--client", "codex")++ self.assert_stops_agent(process, "`implementation`", "exit code 3", "just doctor")++ def test_keeps_stop_message_valid_json_when_error_output_has_quotes_and_control_characters(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("exec", stdout="", stderr='KeyError: "baseline"\\ \t\x1b[31m\r\n', exit_code=1)++ process = launcher.run("diff", "--client", "claude")++ response = self.assert_stops_agent(process)+ self.assertIn('KeyError: "baseline"\\', response["stopReason"])++ def test_doctor_exits_with_code_of_environment_check(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("exec", stdout="openspec: not found\n", exit_code=1)++ process = launcher.run("doctor")++ self.assertEqual(process.returncode, 1)+ self.assertEqual(process.stdout, "openspec: not found\n")+ [call] = launcher.mise_calls("exec")+ self.assertEqual(call["arguments"][-2:], [str(HOOK_PATH), "doctor"])++ def test_doctor_exits_with_code_1_when_pinned_version_is_missing(self):+ launcher = Launcher(self)+ launcher.install_mise(launcher.bin_dir)+ launcher.set_mise_answer("ls", stdout=f"{MISSING_OPENSPEC_LINE}\n")++ process = launcher.run("doctor")++ self.assertEqual(process.returncode, 1)+ self.assertIn(MISSING_OPENSPEC_LINE, process.stderr)+ self.assertEqual(launcher.mise_calls("exec"), [])++ def test_doctor_exits_with_code_1_when_version_manager_is_missing(self):+ launcher = Launcher(self)++ process = launcher.run("doctor")++ self.assertEqual(process.returncode, 1)+ self.assertIn("could not find the version manager mise", process.stderr)+++class MissingToolTest(unittest.TestCase):+ def test_stops_agent_naming_openspec_and_doctor_when_openspec_is_missing(self):+ repository = HookRepository(self, missing_tools=("openspec",))+ repository.submit_prompt("Write a spec for gadgets")+ repository.write_proposal()++ response = repository.stop("proposal")++ self.assertIs(response["continue"], False)+ self.assertIn("could not find `openspec`", response["stopReason"])+ self.assertIn("just doctor", response["stopReason"])+ self.assertIn("just doctor", response["systemMessage"])+ self.assertEqual(repository.reviewer_calls(), [])+++class DoctorTest(unittest.TestCase):+ def test_reports_path_of_every_tool_and_exits_with_code_0(self):+ repository = HookRepository(self, is_path_isolated=True)+ bin_dir = repository.root.parent / "bin"++ process = repository.run_doctor()++ self.assertEqual(process.returncode, 0, process.stdout)+ self.assertEqual(+ process.stdout.splitlines(),+ [+ f"python: {sys.executable}",+ f"git: {repository.root.parent / 'system-bin' / 'git'}",+ f"openspec: {bin_dir / 'openspec'}",+ f"claude: {bin_dir / 'claude'}",+ f"codex: {bin_dir / 'codex'}",+ ],+ )++ def test_exits_with_code_0_when_only_one_reviewer_cli_is_present(self):+ repository = HookRepository(self, missing_tools=("codex",))++ process = repository.run_doctor()++ self.assertEqual(process.returncode, 0, process.stdout)+ self.assertIn("codex: not found", process.stdout.splitlines())++ def test_reports_missing_reviewer_cli_and_exits_with_nonzero_code(self):+ repository = HookRepository(self, missing_tools=("claude", "codex"))++ process = repository.run_doctor()++ self.assertNotEqual(process.returncode, 0)+ self.assertIn("no reviewer CLI found", process.stdout)+++class RunChecksTest(unittest.TestCase):+ def setUp(self):+ temporary_dir = tempfile.TemporaryDirectory(prefix="review-gate-checks-test-")+ self.addCleanup(temporary_dir.cleanup)+ self.root = Path(temporary_dir.name)++ def read_pid(self, name):+ return int((self.root / name).read_text())++ def test_stops_check_with_every_child_process_when_check_exceeds_its_limit(self):+ check = review_gate.Check(".", (sys.executable, "-c", CHECK_WITH_CHILD_SOURCE, "child.pid"))++ failure = review_gate.run_checks(self.root, [check], check_timeout_seconds=1, total_timeout_seconds=10)++ self.assertIn("did not finish within its limit of 1 seconds per check", failure)+ self.assertTrue(wait_until_process_gone(self.read_pid("child.pid")))++ def test_stops_running_check_and_names_total_limit_when_checks_together_exceed_it(self):+ checks = [+ review_gate.Check(".", (sys.executable, "-c", "import time; time.sleep(1.5)")),+ review_gate.Check(".", (sys.executable, "-c", CHECK_WITH_CHILD_SOURCE, "child.pid")),+ ]++ failure = review_gate.run_checks(self.root, checks, check_timeout_seconds=2, total_timeout_seconds=3)++ self.assertIn("did not finish within the total limit of 3 seconds for all checks", failure)+ self.assertIn("child.wait()", failure)+ self.assertTrue(wait_until_process_gone(self.read_pid("child.pid")))++ def test_names_exceeded_limit_when_child_outside_process_group_keeps_output_open(self):+ check = review_gate.Check(".", (sys.executable, "-c", CHECK_WITH_ESCAPED_CHILD_SOURCE, "child.pid"))+ self.addCleanup(lambda: is_process_gone(self.read_pid("child.pid")) or os.kill(self.read_pid("child.pid"), signal.SIGKILL))++ with mock.patch.object(review_gate, "KILLED_CHECK_OUTPUT_TIMEOUT_SECONDS", 0.5):+ failure = review_gate.run_checks(self.root, [check], check_timeout_seconds=1, total_timeout_seconds=10)++ self.assertIn("did not finish within its limit of 1 seconds per check", failure)+ self.assertIn("End of output:\n(no output)", failure)+++class FullReviewBudgetTest(unittest.TestCase):+ def get_hook_timeouts(self, config_path, gate):+ config = json.loads((HOOK_PATH.parent.parent / config_path).read_text())+ return [+ hook["timeout"]+ for groups in config["hooks"].values()+ for group in groups+ for hook in group["hooks"]+ if f'run.sh" {gate} --client' in hook["command"]+ ]++ def test_budget_fits_checks_and_two_reviewer_waves(self):+ for full_review in review_gate.FULL_REVIEWS.values():+ self.assertGreaterEqual(+ review_gate.FULL_REVIEW_BUDGET_SECONDS,+ review_gate.CHECKS_TOTAL_TIMEOUT_SECONDS + 2 * full_review.reviewer.timeout_seconds,+ )++ def test_budget_ends_before_full_review_hook_timeouts_of_both_clients(self):+ for config_path in (".claude/settings.json", ".codex/hooks.json"):+ for gate in review_gate.FULL_REVIEWS:+ [timeout_seconds] = self.get_hook_timeouts(config_path, gate)+ self.assertLess(review_gate.FULL_REVIEW_BUDGET_SECONDS, timeout_seconds, f"{config_path} {gate}")+++class ChangeVerificationParsingTest(unittest.TestCase):+ def setUp(self):+ temporary_dir = tempfile.TemporaryDirectory(prefix="review-gate-verification-test-")+ self.addCleanup(temporary_dir.cleanup)+ self.change_dir = Path(temporary_dir.name) / CHANGE_NAME+ self.change_dir.mkdir()++ def parse(self, section):+ (self.change_dir / "tasks.md").write_text(f"# Tasks\n\n- [x] 1.1 Build\n{section}")+ return review_gate.parse_change_verification(self.change_dir)++ def assert_rejected(self, section, reason):+ with self.assertRaises(review_gate.ChangeVerificationError) as context:+ self.parse(section)+ message = str(context.exception)+ self.assertIn(reason, message)+ self.assertIn(f"`{CHANGE_NAME}/tasks.md`", message)+ self.assertIn(review_gate.ALLOWED_VERIFICATION_FORMS, message)++ def test_rejects_full_backend_suite(self):+ self.assert_rejected(verification_section("just test"), "`just test` is not an allowed command")++ def test_rejects_shell_operator_as_forbidden_character(self):+ self.assert_rejected(+ verification_section("just test-package ./internal/features/intervals/... && rm -rf /tmp/x"),+ "the argument `&&`",+ )++ def test_rejects_argument_with_forbidden_character(self):+ self.assert_rejected(+ verification_section("just test-package ./internal/a;rm"),+ "the argument `./internal/a;rm` in `just test-package ./internal/a;rm` has a forbidden character",+ )++ def test_runs_backend_package_tests_in_backend_directory(self):+ checks = self.parse(verification_section("just test-package ./internal/features/intervals/..."))++ self.assertEqual(checks, [review_gate.Check("backend", ("just", "test-package", "./internal/features/intervals/..."))])++ def test_accepts_every_allowed_form_and_skips_empty_and_comment_lines(self):+ checks = self.parse(+ verification_section(+ "# backend",+ "just test-package ./internal/features/intervals/... ./internal/features/backups/...",+ "",+ "just --justfile agent/verification/Justfile test",+ "pnpm --dir frontend test src/features/widgets --reporter=dot",+ )+ )++ self.assertEqual(+ checks,+ [+ review_gate.Check(+ "backend", ("just", "test-package", "./internal/features/intervals/...", "./internal/features/backups/...")+ ),+ review_gate.Check(".", ("just", "--justfile", "agent/verification/Justfile", "test")),+ review_gate.Check(".", ("pnpm", "--dir", "frontend", "test", "src/features/widgets", "--reporter=dot")),+ ],+ )++ def test_rejects_backend_package_tests_without_package(self):+ self.assert_rejected(verification_section("just test-package"), "does not match the form `just test-package <package>...`")++ def test_rejects_package_path_without_leading_dot_slash(self):+ self.assert_rejected(verification_section("just test-package internal/features/intervals/..."), "must start with `./`")++ def test_rejects_arguments_after_verification_agent_form(self):+ self.assert_rejected(+ verification_section("just --justfile agent/verification/Justfile test ./internal/..."),+ "does not match the form `just --justfile agent/verification/Justfile test`",+ )++ def test_rejects_section_with_only_bash_fenced_block(self):+ self.assert_rejected(verification_section("pnpm --dir frontend test", fence="bash"), "the section has no ```sh code block")++ def test_rejects_unclosed_block(self):+ self.assert_rejected("\n## Change verification\n\n```sh\npnpm --dir frontend test\n", "the ```sh code block is not closed")++ def test_rejects_block_without_commands(self):+ self.assert_rejected(verification_section("# nothing yet", ""), "the ```sh code block holds no command")++ def test_returns_no_commands_without_section(self):+ self.assertEqual(self.parse(""), [])++class PlanGateTest(unittest.TestCase):def exit_plan_mode(self, repository, gate, plan):payload = {@@ -471,7 +973,7 @@ class ImplementationGateStartTest(unittest.TestCase):response = repository.stop("implementation")self.assertEqual(response, {})- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)def test_skips_review_when_tasks_remain_unchecked(self):repository = HookRepository(self)@@ -517,7 +1019,7 @@ class ImplementationGateStartTest(unittest.TestCase):repository.stop("implementation")- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)def test_skips_review_when_prompt_has_openspec_planning_command(self):repository = HookRepository(self)@@ -615,7 +1117,7 @@ class ImplementationGateChecksTest(ShortReplyAssertions, unittest.TestCase):repository.stop("implementation")self.assertEqual(repository.check_calls(), [])- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)def test_blocks_turn_with_output_tail_when_linter_fails(self):repository = HookRepository(self)@@ -632,21 +1134,87 @@ class ImplementationGateChecksTest(ShortReplyAssertions, unittest.TestCase):self.assert_asks_for_short_reply(response["reason"])self.assertEqual(repository.reviewer_calls(), [])- def test_never_runs_formatters_or_just(self):+ def test_never_runs_formatters_and_runs_just_only_from_change_verification_block(self):repository = HookRepository(self)repository.submit_prompt("Continue")repository.write("backend/internal/features/widgets/service.go", "package widgets\n")repository.write("frontend/src/pages/widgets/ui/WidgetsPage.tsx", "export {};\n")- repository.complete_tasks()+ repository.complete_tasks(verification_section=verification_section("just test-package ./internal/features/widgets/..."))response = repository.stop("implementation")self.assertEqual(response, {})- self.assertEqual(repository.calls("just"), [])+ self.assertEqual([call["arguments"] for call in repository.calls("just")], [["test-package", "./internal/features/widgets/..."]])arguments = [" ".join(call["arguments"]) for call in repository.check_calls()]- self.assertEqual(len(arguments), 2)+ self.assertEqual(len(arguments), 3)self.assertFalse(any("fmt" in argument or "format" in argument for argument in arguments))- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)++ def test_blocks_turn_with_command_and_output_tail_when_package_test_fails(self):+ repository = HookRepository(self)+ repository.set_stub("just", {"exit_code": 1, "output": "--- FAIL: Test_CreateInterval_WhenOverlapping_ReturnsConflict\n"})+ repository.submit_prompt("Continue")+ repository.write("backend/internal/features/intervals/service.go", "package intervals\n")+ repository.complete_tasks(verification_section=verification_section("just test-package ./internal/features/intervals/..."))++ response = repository.stop("implementation")++ self.assertEqual(response["decision"], "block")+ self.assertIn("`just test-package ./internal/features/intervals/...` in `backend` exited with code 1", response["reason"])+ self.assertIn("--- FAIL: Test_CreateInterval_WhenOverlapping_ReturnsConflict", response["reason"])+ self.assert_asks_for_short_reply(response["reason"])+ self.assertEqual(repository.reviewer_calls(), [])++ def test_runs_backend_package_tests_in_backend_directory_after_linter(self):+ repository = HookRepository(self)+ repository.submit_prompt("Continue")+ repository.write("backend/internal/features/intervals/service.go", "package intervals\n")+ repository.complete_tasks(verification_section=verification_section("just test-package ./internal/features/intervals/..."))++ repository.stop("implementation")++ [linter_call] = repository.calls("golangci-lint")+ [test_call] = repository.calls("just")+ self.assertEqual(test_call["arguments"], ["test-package", "./internal/features/intervals/..."])+ self.assertEqual(Path(test_call["cwd"]), repository.root / "backend")+ self.assertLess(linter_call["started_at"], test_call["started_at"])++ def test_runs_command_named_by_two_changes_once(self):+ repository = HookRepository(self)+ repository.submit_prompt("Continue")+ repository.complete_tasks(verification_section=verification_section("pnpm --dir frontend test"))+ repository.write_change(spec_names=("gizmos",), tasks="- [x] 1.1 Build\n" + verification_section("pnpm --dir frontend test"), name="add-gizmos")++ repository.stop("implementation")++ self.assertEqual([call["arguments"] for call in repository.calls("pnpm")], [["--dir", "frontend", "test"]])++ def test_starts_reviewers_without_test_run_when_change_has_no_verification_section(self):+ repository = HookRepository(self)+ repository.submit_prompt("Continue")+ repository.write("frontend/src/pages/widgets/ui/WidgetsPage.tsx", "export {};\n")+ repository.complete_tasks()++ response = repository.stop("implementation")++ self.assertEqual(response, {})+ self.assertEqual([call["arguments"] for call in repository.check_calls()], [["--dir", "frontend", "lint"]])+ self.assertEqual(len(repository.reviewer_calls()), 3)++ def test_blocks_turn_before_any_check_when_verification_section_is_malformed(self):+ repository = HookRepository(self)+ repository.submit_prompt("Continue")+ repository.write("backend/internal/features/intervals/service.go", "package intervals\n")+ repository.complete_tasks(verification_section=verification_section("just test"))++ response = repository.stop("implementation")++ self.assertEqual(response["decision"], "block")+ self.assertIn("`just test` is not an allowed command", response["reason"])+ self.assertIn(review_gate.ALLOWED_VERIFICATION_FORMS, response["reason"])+ self.assertIn("No check or reviewer ran.", response["reason"])+ self.assertEqual(repository.check_calls(), [])+ self.assertEqual(repository.reviewer_calls(), [])def test_runs_hook_tests_when_change_touches_hooks(self):repository = HookRepository(self)@@ -657,7 +1225,7 @@ class ImplementationGateChecksTest(ShortReplyAssertions, unittest.TestCase):response = repository.stop("implementation")self.assertEqual(response, {})- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)def test_blocks_turn_when_hook_tests_fail(self):repository = HookRepository(self)@@ -682,7 +1250,7 @@ class ImplementationGateReviewTest(ShortReplyAssertions, unittest.TestCase):repository.stop("implementation")calls = repository.calls("claude")- self.assertEqual(len(calls), 3)+ self.assertEqual(len(calls), 4)for call in calls:self.assertIn("--model opus --effort medium", " ".join(call["arguments"]))self.assertEqual(call["effort_env"], "medium")@@ -693,6 +1261,7 @@ class ImplementationGateReviewTest(ShortReplyAssertions, unittest.TestCase):f"Role: delta spec `openspec/changes/{CHANGE_NAME}/specs/gadgets/spec.md`",f"Role: delta spec `openspec/changes/{CHANGE_NAME}/specs/widgets/spec.md`","Role: documentation and tasks",+ "Role: tests",],)@@ -705,7 +1274,7 @@ class ImplementationGateReviewTest(ShortReplyAssertions, unittest.TestCase):repository.stop("implementation", client="codex")calls = repository.calls("codex")- self.assertEqual(len(calls), 3)+ self.assertEqual(len(calls), 4)for call in calls:self.assertIn("--model gpt-6.1-sol --config model_reasoning_effort=\"high\"", " ".join(call["arguments"]))@@ -718,9 +1287,64 @@ class ImplementationGateReviewTest(ShortReplyAssertions, unittest.TestCase):[call] = [call for call in repository.reviewer_calls() if "\nRole: documentation and tasks\n" in call["prompt"]]prompt = call["prompt"]- docs_role = prompt[prompt.index("### Role: documentation and tasks") : prompt.index("### Role: repeat review")]+ docs_role = prompt[prompt.index("### Role: documentation and tasks") : prompt.index("### Role: tests")]self.assertIn("Check the edits to agent instructions the change made", docs_role)+ def test_tests_reviewer_prompt_requires_failing_tests_reproduced_bugs_waits_and_covered_packages(self):+ repository = HookRepository(self)+ repository.write_change(spec_names=("widgets", "gadgets"), tasks="- [x] 1.1 Build\n")+ repository.submit_prompt("Continue")+ repository.complete_tasks()++ repository.stop("implementation")++ self.assertEqual(len(repository.reviewer_calls()), 4)+ [call] = [call for call in repository.reviewer_calls() if "\nRole: tests\n" in call["prompt"]]+ prompt = call["prompt"]+ tests_role = prompt[prompt.index("### Role: tests") : prompt.index("### Role: repeat review")]+ self.assertIn("`.agents/skills/testing/SKILL.md`", tests_role)+ self.assertIn("The test fails without the behavior it pins.", tests_role)+ self.assertIn("A bug fix has a test that reproduces the bug and fails on the code before the fix.", tests_role)+ self.assertIn("waits for a condition up to a limit instead of a fixed pause", tests_role)+ self.assertIn(+ "run the tests of every backend package, verification agent package and frontend test file in which the change added or changed tests",+ tests_role,+ )+ self.assertIn("such as the hook tests in `.agent-hooks/`, count as covered", tests_role)++ def test_blocks_turn_with_budget_reason_when_third_wave_of_reviewers_does_not_fit(self):+ repository = HookRepository(self)+ repository.set_stub("claude", {"sleep_seconds": THIRD_WAVE_REVIEWER_SECONDS})+ repository.write_change(spec_names=("widgets", *(f"spec{index}" for index in range(10))), tasks="- [x] 1.1 Build\n")+ repository.submit_prompt("Continue")+ repository.complete_tasks()++ response = repository.stop_in_process("implementation", THIRD_WAVE_BUDGET_SECONDS, THIRD_WAVE_REVIEWER_TIMEOUT_SECONDS)++ self.assertEqual(response["decision"], "block")+ self.assertIn(review_gate.BUDGET_EXHAUSTED_ERROR, response["reason"])+ self.assertEqual(len(repository.reviewer_calls()), 12)++ def test_runs_two_waves_of_reviewers_after_checks_use_their_total_limit(self):+ repository = HookRepository(self)+ repository.set_stub("golangci-lint", {"sleep_seconds": TWO_WAVES_CHECKS_TIMEOUT_SECONDS - 0.1})+ repository.set_stub("claude", {"sleep_seconds": TWO_WAVES_REVIEWER_TIMEOUT_SECONDS - 0.2})+ repository.write_change(spec_names=("widgets", *(f"spec{index}" for index in range(4))), tasks="- [x] 1.1 Build\n")+ repository.submit_prompt("Continue")+ repository.write("backend/internal/features/widgets/service.go", "package widgets\n")+ repository.complete_tasks()++ response = repository.stop_in_process(+ "implementation",+ TWO_WAVES_BUDGET_SECONDS,+ TWO_WAVES_REVIEWER_TIMEOUT_SECONDS,+ checks_timeout_seconds=TWO_WAVES_CHECKS_TIMEOUT_SECONDS,+ )++ self.assertEqual(response, {})+ self.assertEqual(len(repository.calls("golangci-lint")), 1)+ self.assertEqual(len(repository.reviewer_calls()), 7)+def test_blocks_turn_and_requires_agent_to_resolve_findings_when_reviewer_finds_bug(self):repository = HookRepository(self)repository.set_reviewer_result({"findings": [finding("bug", "Race between the check and the write.")]})@@ -819,7 +1443,7 @@ class ImplementationGateRepeatTest(SummaryAssertions, unittest.TestCase):repository.clear_calls()repository.stop("implementation")- self.assertEqual(len(repository.reviewer_calls()), 2)+ self.assertEqual(len(repository.reviewer_calls()), 3)def test_asks_for_summary_without_review_once_after_repeat_accepts_change(self):repository = HookRepository(self)@@ -1009,6 +1633,20 @@ class ProposalGateValidationTest(unittest.TestCase):self.assertIn("must include at least one scenario", response["reason"])self.assertEqual(repository.reviewer_calls(), [])+ def test_blocks_turn_without_running_command_when_verification_section_has_forbidden_command(self):+ repository = HookRepository(self)+ repository.submit_prompt("Write a spec")+ repository.write_proposal(tasks="- [ ] 1.1 Build\n" + verification_section("just test"))++ response = repository.stop("proposal")++ self.assertEqual(response["decision"], "block")+ self.assertIn("`just test` is not an allowed command", response["reason"])+ self.assertIn(review_gate.ALLOWED_VERIFICATION_FORMS, response["reason"])+ self.assertEqual(repository.calls("just"), [])+ self.assertEqual([call["arguments"][0] for call in repository.calls("openspec")], ["status"])+ self.assertEqual(repository.reviewer_calls(), [])+class ProposalGateReviewTest(ShortReplyAssertions, SummaryAssertions, unittest.TestCase):def test_runs_reviewer_per_spec_and_for_proposal_and_tasks_on_claude_with_medium_effort(self):@@ -1057,6 +1695,53 @@ class ProposalGateReviewTest(ShortReplyAssertions, SummaryAssertions, unittest.Tself.assertIn("every scenario of the delta specs is covered by a task with a test", call["prompt"])self.assertIn("the task order can be followed", call["prompt"])+ def test_tasks_reviewer_prompt_requires_verification_section_for_tested_packages_except_hook_tests(self):+ repository = HookRepository(self)+ repository.submit_prompt("Write a spec")+ repository.write_proposal()++ repository.stop("proposal")++ [call] = [call for call in repository.reviewer_calls() if "\nRole: tasks\n" in call["prompt"]]+ prompt = call["prompt"]+ tasks_role = prompt[prompt.index("### Role: tasks") : prompt.index("### Role: repeat review")]+ self.assertIn(+ "tasks add or change tests in a backend package, a verification agent package or the frontend has a `## Change verification` section",+ tasks_role,+ )+ self.assertIn("run the tests of every such package the tasks' verifications name", tasks_role)+ self.assertIn("a package the section leaves out, is a `tasks` finding that names the package", tasks_role)+ self.assertIn("such as the hook tests in `.agent-hooks/`, need no section", tasks_role)++ def test_blocks_turn_with_budget_reason_when_third_wave_of_reviewers_does_not_fit(self):+ repository = HookRepository(self)+ repository.set_stub("claude", {"sleep_seconds": THIRD_WAVE_REVIEWER_SECONDS})+ repository.submit_prompt("Write a spec")+ repository.write_proposal(spec_names=tuple(f"spec{index}" for index in range(11)))++ response = repository.stop_in_process("proposal", THIRD_WAVE_BUDGET_SECONDS, THIRD_WAVE_REVIEWER_TIMEOUT_SECONDS)++ self.assertEqual(response["decision"], "block")+ self.assertIn(review_gate.BUDGET_EXHAUSTED_ERROR, response["reason"])+ self.assertEqual(len(repository.reviewer_calls()), 12)++ def test_runs_two_waves_of_reviewers_after_validation_uses_checks_total_limit(self):+ repository = HookRepository(self)+ repository.set_stub("openspec", {"validate": {"sleep_seconds": TWO_WAVES_CHECKS_TIMEOUT_SECONDS - 0.1}})+ repository.set_stub("claude", {"sleep_seconds": TWO_WAVES_REVIEWER_TIMEOUT_SECONDS - 0.2})+ repository.submit_prompt("Write a spec")+ repository.write_proposal(spec_names=tuple(f"spec{index}" for index in range(5)))++ response = repository.stop_in_process(+ "proposal",+ TWO_WAVES_BUDGET_SECONDS,+ TWO_WAVES_REVIEWER_TIMEOUT_SECONDS,+ checks_timeout_seconds=TWO_WAVES_CHECKS_TIMEOUT_SECONDS,+ )++ self.assertEqual(response, {})+ self.assertEqual(len(repository.reviewer_calls()), 7)+def test_blocks_turn_and_requires_agent_to_resolve_simplify_finding_without_asking_user(self):repository = HookRepository(self)repository.set_reviewer_result(diff --git a/.agent-hooks/run.sh b/.agent-hooks/run.shnew file mode 100755index 00000000..dd345191--- /dev/null+++ b/.agent-hooks/run.sh@@ -0,0 +1,92 @@+#!/usr/bin/env bash+# Starts review_gate.py with the Python and OpenSpec versions pinned in mise.toml, so a client+# started from an IDE without the mise shims on PATH runs the same hook as a terminal one.+# A hook that cannot run stops the agent with the reason instead of letting the turn pass unreviewed.+# Only bash builtins and coreutils run before mise, because nothing else is guaranteed on PATH yet.++set -uo pipefail++hooks_dir="$(cd "${BASH_SOURCE[0]%/*}" && pwd)"+root="${hooks_dir%/*}"+action="${1:-}"+environment_check_hint="Run \`mise install\` to install the pinned tools and \`just doctor\` to check the environment."++find_mise() {+ if command -v mise; then+ return 0+ fi+ if [[ -x "$HOME/.local/bin/mise" ]]; then+ printf '%s\n' "$HOME/.local/bin/mise"+ return 0+ fi+ return 1+}++escape_json() {+ local value="$1"+ value="${value//\\/\\\\}"+ value="${value//\"/\\\"}"+ value="${value//$'\n'/\\n}"+ value="${value//$'\t'/\\t}"+ printf '%s' "$value"+}++stop_agent() {+ local message+ message="$(escape_json "$1")"+ printf '{"continue": false, "stopReason": "%s", "systemMessage": "%s"}\n' "$message" "$message"+ exit 0+}++if ! mise_path="$(find_mise)"; then+ reason="The review hooks could not find the version manager mise on PATH or in \$HOME/.local/bin."+ if [[ "$action" == "doctor" ]]; then+ printf '%s\n' "$reason" >&2+ exit 1+ fi+ stop_agent "$reason Install mise, then $environment_check_hint"+fi++# mise exec runs a command even when a pinned version is missing: python3 and openspec then resolve+# to whatever else PATH holds. So the launcher refuses to start until both are installed.+missing_tools="$("$mise_path" -C "$root" ls --current --missing --no-truncate python npm:@fission-ai/openspec 2>&1)"+missing_check_exit_code=$?+if [[ $missing_check_exit_code -ne 0 || -n "$missing_tools" ]]; then+ if [[ $missing_check_exit_code -ne 0 ]]; then+ reason="The review hooks could not check the pinned tools: mise exited with code $missing_check_exit_code."+ else+ reason="The pinned tools the review hooks need are not installed."+ fi+ reason="$reason+$missing_tools"+ if [[ "$action" == "doctor" ]]; then+ printf '%s\n' "$reason" >&2+ exit 1+ fi+ stop_agent "$reason+$environment_check_hint"+fi++# An installation would not fit the hook timeout.+export MISE_EXEC_AUTO_INSTALL=false+command=("$mise_path" -C "$root" exec -- python3 "$hooks_dir/review_gate.py" "$@")++if [[ "$action" == "doctor" ]]; then+ exec "${command[@]}"+fi++error_log="$(mktemp)"+trap 'rm -f "$error_log"' EXIT++output="$("${command[@]}" 2>"$error_log")"+exit_code=$?+if [[ $exit_code -eq 0 ]]; then+ printf '%s\n' "$output"+ exit 0+fi++error_tail="$(tail -n 20 "$error_log" | tr -d '\000-\010\013-\037')"+stop_agent "The review hook \`${action:-(none)}\` failed with exit code $exit_code, so the review did not run. $environment_check_hint++End of its error output:+${error_tail:-(no output)}"diff --git a/.agents/skills/testing/SKILL.md b/.agents/skills/testing/SKILL.mdnew file mode 100644index 00000000..4af78fac--- /dev/null+++ b/.agents/skills/testing/SKILL.md@@ -0,0 +1,47 @@+---+name: testing+description: Write, change, run and review tests in Databasus by the rules shared by every module. Use when writing, changing, running or reviewing tests, when fixing a bug, and when choosing which test commands a change runs.+---++# Testing++These rules apply in every module. The module docs add the rules of their stack: test names, the level of test they prefer, where tests live and how they clean up. Read the one for the area you work in:++- [`backend/AGENTS.md`, "Testing"](../../../backend/AGENTS.md#testing)+- [`agent/verification/AGENTS.md`, "Testing"](../../../agent/verification/AGENTS.md#testing)++## Run what you wrote++Run every test you wrote or changed and see it pass before you hand the work over. A package that compiles has not passed its tests: when you only built it or ran `go vet`, say so in the report instead of calling it tested.++## Start a bug fix with a failing test++A bug fix starts with a test that reproduces the bug. Run it on the code before the fix and see it fail for the reason the bug describes, then fix the code and see it pass. A test that passes on the broken code proves nothing about the fix.++## Test what a caller observes++Test the behavior a caller sees, at the level the module doc prefers: through the HTTP API in the backend, through exported functions and CLI commands in the verification agent. A test that pins private helpers breaks on every refactoring and still misses the contract.++Mock only external systems the test cannot run, such as a third-party API, and failures that are unsafe to reproduce, such as a full disk. A database, a container or a file system the test fixtures provide stays real.++## Wait for a condition, not for time++A test that needs something to happen waits for that condition up to a limit and fails with a message when the limit passes. A fixed pause is either too short on a loaded machine or wastes time on a fast one. A negative check, such as "no notification was sent", first waits for a sign that the action was processed, so it does not pass merely because it looked too early.++## Run the tests the change touched++Which tests to run, and which command never to run, is set in [Several agents, one set of containers](../../../AGENTS.md#several-agents-one-set-of-containers), and the commands in the table of its parent section, [Development environment](../../../AGENTS.md#development-environment).++When package tests fail because the shared test fixtures are down, such as a refused database connection or a missing container, tell the user. Do not start the fixtures through `just test`: the developer starts the first full run.++## Clean up after an interrupted run++After a test run you stopped or that crashed, remove only what that run left behind: its containers, processes and temporary directories. Never remove all containers, prune Docker or stop processes you did not start; another agent's run may be using them.++## Before handing the work over++- Every test you wrote or changed ran and passed, and the report says which commands ran.+- A bug fix has a test that failed before the fix.+- No test sleeps for a fixed time to wait for a result.+- Every test removes the data, files and processes it created, as the module doc describes.+- An OpenSpec change whose tasks add or change tests lists their commands in the "Change verification" section of its `tasks.md`, as the tasks rule in [`openspec/config.yaml`](../../../openspec/config.yaml) describes.diff --git a/.claude/settings.json b/.claude/settings.jsonindex 81bfd97b..266b95a7 100644--- a/.claude/settings.json+++ b/.claude/settings.json@@ -5,7 +5,7 @@"hooks": [{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" session-start --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" session-start --client claude","timeout": 10}]@@ -16,7 +16,7 @@"hooks": [{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" snapshot --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" snapshot --client claude","timeout": 30}]@@ -28,19 +28,19 @@"hooks": [{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" plan --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" plan --client claude","timeout": 600,"statusMessage": "Reviewing the plan"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" necessity --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" necessity --client claude","timeout": 600,"statusMessage": "Checking whether the task is needed"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" architecture --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" architecture --client claude","timeout": 600,"statusMessage": "Checking architecture"}@@ -52,38 +52,38 @@"hooks": [{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" diff --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" diff --client claude","timeout": 600,"statusMessage": "Reviewing the diff"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" necessity --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" necessity --client claude","timeout": 600,"statusMessage": "Checking whether the task is needed"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" architecture --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" architecture --client claude","timeout": 600,"statusMessage": "Checking architecture"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" naming --client claude",+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" naming --client claude","timeout": 600,"statusMessage": "Checking names"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" implementation --client claude",- "timeout": 3600,+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" implementation --client claude",+ "timeout": 7200,"statusMessage": "Full implementation review"},{"type": "command",- "command": "python3 \"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" proposal --client claude",- "timeout": 3600,+ "command": "\"$(git -C \"${CLAUDE_PROJECT_DIR}\" rev-parse --show-toplevel)/.agent-hooks/run.sh\" proposal --client claude",+ "timeout": 7200,"statusMessage": "Full proposal review"}]diff --git a/.claude/skills/testing/SKILL.md b/.claude/skills/testing/SKILL.mdnew file mode 120000index 00000000..fcbefaef--- /dev/null+++ b/.claude/skills/testing/SKILL.md@@ -0,0 +1 @@+../../../.agents/skills/testing/SKILL.md\ No newline at end of filediff --git a/.codex/hooks.json b/.codex/hooks.jsonindex c17a2adc..967b9259 100644--- a/.codex/hooks.json+++ b/.codex/hooks.json@@ -6,7 +6,7 @@"hooks": [{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" session-start --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" session-start --client codex","timeout": 10}]@@ -17,7 +17,7 @@"hooks": [{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" snapshot --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" snapshot --client codex","timeout": 30}]@@ -28,44 +28,44 @@"hooks": [{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" plan --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" plan --client codex","timeout": 600,"statusMessage": "Reviewing the plan"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" necessity --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" necessity --client codex","timeout": 600,"statusMessage": "Checking whether the task is needed"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" diff --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" diff --client codex","timeout": 600,"statusMessage": "Reviewing the diff"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" architecture --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" architecture --client codex","timeout": 600,"statusMessage": "Checking architecture"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" naming --client codex",+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" naming --client codex","timeout": 600,"statusMessage": "Checking names"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" implementation --client codex",- "timeout": 3600,+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" implementation --client codex",+ "timeout": 7200,"statusMessage": "Full implementation review"},{"type": "command",- "command": "python3 \"$(git rev-parse --show-toplevel)/.agent-hooks/review_gate.py\" proposal --client codex",- "timeout": 3600,+ "command": "\"$(git rev-parse --show-toplevel)/.agent-hooks/run.sh\" proposal --client codex",+ "timeout": 7200,"statusMessage": "Full proposal review"}]diff --git a/.gitignore b/.gitignoreindex a617607c..d361ec3d 100644--- a/.gitignore+++ b/.gitignore@@ -21,6 +21,7 @@ node_modules/.claude/skills/*!.claude/skills/git-commits/!.claude/skills/humanizer/+!.claude/skills/testing/backend/cleanup_test_db.exeplans/diff --git a/AGENTS.md b/AGENTS.mdindex 6fc6424f..e49832e0 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -39,6 +39,7 @@ Work happens on a Fedora host; any other host distro is on its own. Tool version| `just pull-testcontainers` | Pull the database images the backend tests use || `just lint` | Check every module without changing files || `just format` | Format every module |+| `just doctor` | Check that the review hooks find the tools they need || `goose -dir backend/migrations create <name> sql` | Create a migration || `goose -dir backend/migrations up` (or `down`), with `.env` exported | Apply or roll back migrations on the dev database; goose reads `GOOSE_DRIVER` and `GOOSE_DBSTRING` from the environment || `swag init -d . -g cmd/main.go -o swagger`, run in `backend/` | Regenerate the API docs |@@ -58,6 +59,12 @@ When a failure makes no sense — a container that vanished, a database that emp---+## Tests++When you write, change, run or review tests, load the [testing skill](.agents/skills/testing/SKILL.md) first. It holds the rules every module shares; the module docs add the rules of their stack.++---+## Language in code**English only in code, comments, identifiers, log messages, API strings, test assertions, and commit messages.** No other language inside `backend/` or `agent/` at all, and none inside `frontend/src/` outside the interface dictionaries named below — not even for user-facing fallback copy or error messages.diff --git a/Justfile b/Justfileindex 75c7f221..f87baae6 100644--- a/Justfile+++ b/Justfile@@ -16,6 +16,9 @@ lint:pnpm --dir frontend lintpnpm --dir frontend typecheck+doctor:+ .agent-hooks/run.sh doctor+format:just --justfile backend/Justfile formatjust --justfile agent/verification/Justfile formatdiff --git a/agent/verification/AGENTS.md b/agent/verification/AGENTS.mdindex dff30eae..8293e26a 100644--- a/agent/verification/AGENTS.md+++ b/agent/verification/AGENTS.md@@ -138,7 +138,7 @@ func (h *Heartbeater) Run(ctx context.Context) {## Testing-**Always run tests after writing them and verify they pass.**+The [testing skill](../../.agents/skills/testing/SKILL.md) holds the rules every module shares; this section adds the verification agent's own.### Namingdiff --git a/backend/AGENTS.md b/backend/AGENTS.mdindex c74f93d6..99a71c17 100644--- a/backend/AGENTS.md+++ b/backend/AGENTS.md@@ -301,9 +301,7 @@ Generate the migration stub with the goose create command from the [command tabl## Testing-**Always run tests after writing them and verify they pass.**--Run the packages you touched and the ones that test the same feature (`just test-package ./internal/features/<feature>/...`). Never run `just test` — see [Several agents, one set of containers](../AGENTS.md#several-agents-one-set-of-containers).+The [testing skill](../.agents/skills/testing/SKILL.md) holds the rules every module shares; this section adds the backend's own.### Namingdiff --git a/mise.toml b/mise.tomlindex f4c044ee..38d0109d 100644--- a/mise.toml+++ b/mise.toml@@ -8,3 +8,5 @@ process-compose = "1.122.0""aqua:pressly/goose" = "3.27.3"swag = "1.16.4"golangci-lint = "2.11.3"+python = "3.14.7"+"npm:@fission-ai/openspec" = "1.11.0"diff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/.openspec.yaml b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/.openspec.yamlnew file mode 100644index 00000000..5c881ba7--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/.openspec.yaml@@ -0,0 +1,2 @@+schema: spec-driven+created: 2026-10-18diff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/design.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/design.mdnew file mode 100644index 00000000..00d0df2e--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/design.md@@ -0,0 +1,117 @@+## Context++See proposal.md for the motivation and the delta specs for the behavior contract.++How the hooks start and check today:++- `.claude/settings.json` and `.codex/hooks.json` run `python3 .../.agent-hooks/review_gate.py <action>` with whatever `python3` the client's `PATH` holds. `mise.toml` pins neither Python nor the OpenSpec CLI. On this machine `python3` resolves through the global mise shims to 3.14.7. `openspec --version` reports 1.11.0, but that copy is `/usr/local/bin/openspec`, outside mise: mise itself has only 1.13.2 and 1.14.1 installed, and the generated `openspec-*` skills were written by 1.11.0.+- `is_planning_complete` (`.agent-hooks/review_gate.py:660-674`) treats a missing `openspec` like any other error and returns `False`. The proposal review then never starts, and the quick checks do. The comment at `:658-659` states this as intended.+- `MODULE_CHECKS` (`review_gate.py:112-128`) lists read-only checks per touched module. The comment above it says `just test` "never runs here" because it resets the shared test containers. `run_checks` (`:738-758`) runs each check with `subprocess.run(timeout=900)`. On timeout that kills only the direct child, and there is no limit across checks.+- The implementation review reviewers are one per delta spec plus `documentation and tasks` (`review_gate.py:1027`). They run at most six at a time (`MAX_PARALLEL_REVIEWERS`), each with a 1800-second limit (`review_gate.py:87`). Both full review hooks have a 3600-second timeout in both client configs, so two waves of reviewers already reach it, and nothing stops a third wave.+- `implementation-prompt.md:3` tells reviewers the hook does not run the Go test suites. `proposal-prompt.md` has the task reviewer reject any task that runs `just test` or `just test-filesystem`.+- `just test-package` (`backend/Justfile`) is a shebang recipe that interpolates its arguments into `go test -count=1 {{ packages }}` inside bash. `pnpm --dir frontend test` runs the `test` script of `frontend/package.json` (`vitest run`) through a shell. `just test` in `agent/verification/Justfile` runs `go test -race -count=1 -failfast ./internal/...`.+- Test rules live in `backend/AGENTS.md` ("Testing") and `agent/verification/AGENTS.md` ("Testing"). Both open with the same sentence, "Always run tests after writing them and verify they pass."++Constraints from the root `AGENTS.md`: agents never run `just test` ("Several agents, one set of containers"), the hook never modifies the working tree, and one rule is written in one place ("Documentation consistency"). From `backend/AGENTS.md`: test names follow `Test_WhatWeDo_WhichConditions_WhatWeExpect`, controller tests are preferred, and `just test-package` is the way to run backend packages.++## Goals / Non-Goals++**Goals:**++- The implementation review sees real test results for the packages the change names before any reviewer starts.+- Nothing in a task list can make the hook run an arbitrary command.+- A hook that cannot run is visible to the user.++**Non-Goals:**++- Choosing tests from the changed paths automatically.+- Making the full backend suite safe for parallel agents.++## Decisions++### The change names its test commands in a "Change verification" section++`tasks.md` may end with a section `## Change verification` that holds one `sh` code block with one command per line. The section has no checkboxes, so the apply workflow does not track it as a task. The implementation review reads it, de-duplicates commands across the changes it reviews, and runs them after `MODULE_CHECKS`. The proposal review parses it before `openspec validate --strict` and runs nothing from it. `openspec/config.yaml` gets a tasks rule that describes the section and its allowed forms.++The section is optional for the parser. Whether a change needs it is a judgment about the change, so the reviewers decide it. The proposal task reviewer asks for the section when the tasks add or change tests that an allowed form runs: backend packages, verification agent packages and frontend tests. The implementation tests reviewer checks that it covers every such package. The hook tests are exempt: no allowed form runs them, and the module check runs them whenever `.agent-hooks/` changes. The website has no test runner, only its lint and build module checks.++Rejected: selecting tests from the changed files, as the sibling project's `just test-affected` does. It needs an import-graph selector, and its fallback for unmapped paths is the full suite, which agents here must not run. Rejected: parsing the "Verify with" sentence of each task. Those sentences are prose, and a parser for them would guess. Rejected: requiring the section in the parser. A documentation-only change would need a fake command.++### Three allowed command forms, run without a shell, with a safe character set for arguments++The parser reads the first code block fenced as `sh` under the heading; a `bash`, `shell` or bare fence does not count, so there is one spelling to parse and test. It accepts:++| Form | Runs in |+|---|---|+| `just test-package <package>...`, at least one package starting with `./` | `backend/` |+| `just --justfile agent/verification/Justfile test`, nothing after it | the repository root; `just` switches to the justfile's directory |+| `pnpm --dir frontend test [<argument>...]` | the repository root |++The hook splits a line with `shlex` and runs the word list directly, without a shell. Every word after the fixed prefix must match `[A-Za-z0-9._/=:@-]+`. That set leaves out every shell operator, redirection and substitution character, so no separate operator check is needed.++The character check exists because the hook runs outside the agent's permission prompts, and both runners pass arguments through a shell. `just test-package ./internal/a;rm` contains no separate operator word, but the recipe would expand it to `go test -count=1 ./internal/a;rm` inside bash. The sibling project checks operator words only, which misses this case.++Rejected: `just test` with a directory rule for the verification agent. At the root the same text means the full backend suite, and both the parser and the reviewers would have to tell the two apart by context. Rejected: allowing `go test` directly. It would bypass the libedit shim that `just test-package` applies on Fedora.++### One time budget per full review, with limits for checks and process groups++Both full review hooks get a 7200-second timeout in both client configs. A full review runs inside a budget of 6600 seconds from the hook's start, which leaves 600 seconds for Git, `openspec status`, prompt building and result handling.++- `run_checks` starts each check with `Popen(start_new_session=True)`. On timeout it kills the whole process group with `os.killpg`. A check keeps its 900-second limit, and all checks of one review share a total limit of 1800 seconds. The failure message names the limit that was exceeded.+- A reviewer job starts only when the budget still holds its whole 1800-second limit. A job that cannot start returns the error "the review time budget ran out before this reviewer started", which blocks the turn like any reviewer that could not run.++The worst case that still runs is 1800 seconds of checks plus two waves of reviewers, 5400 seconds in all. Two waves happen when a change has more than four delta specs, because the docs and tests roles bring the job count above six. A third wave could not start after slow checks, and it is reported instead of being cut off by the client.++Rejected: keeping `subprocess.run(timeout=...)`. `just` starts bash, bash starts `go test`, and `go test` starts test binaries that hold a test slot database and testcontainers. Killing only `just` leaves the rest running. Rejected: no total limit for checks. Several 900-second checks would outlast the hook timeout, and the client would kill the hook with no message. Rejected: raising the hook timeout without a reviewer budget. A change with many delta specs would still outgrow any fixed timeout.++### The tests reviewer is a role of the existing full review++`IMPLEMENTATION_REVIEW.first_roles` gains `tests`, and `implementation-prompt.md` gets a "Role: tests" section. The role reads the testing skill and the module testing rules, reviews every test the change added or changed, and checks that the change verification commands cover the packages of those tests that an allowed form runs. Hook tests count as covered by their module check. It reports with the existing categories:++- `missing_test` when a test is missing or proves nothing;+- `bug` when a test is wrong or flaky, for example a fixed pause or leaked data;+- `minor` for the rest.++The delta spec role keeps matching scenarios to tests. The documentation role keeps the rule not to ask for the full suite.++Rejected: a separate Stop gate for tests. It would run on every turn and repeat the diff check's test point. Rejected: a new finding category. `missing_test` and `bug` already block the turn, and a new category would change the result schema and the block logic for no new behavior.++### A `testing` skill holds the rules shared by every module++`.agents/skills/testing/SKILL.md` holds the cross-module rules:++- run every test you wrote or changed and see it pass, and in the report tell a compile-only check from a test run;+- a bug fix starts with a test that reproduces the bug and fails;+- test what a caller observes, at the level the module doc prefers;+- mock only external systems and failures unsafe to reproduce;+- wait for a condition up to a limit instead of a fixed pause, and let a negative check first wait for a sign that the action was processed;+- run the tests the change touched with the commands from the root `AGENTS.md` table, and never `just test`;+- when package tests fail because the shared fixtures are down, tell the user;+- after an interrupted run clean up only your own leftovers, never all containers;+- a checklist before handing work over.++`.claude/skills/testing/SKILL.md` is a relative symlink to it. The root `AGENTS.md` gets a short "Tests" section that tells the agent to load the skill when it writes, changes, runs or reviews tests. `backend/AGENTS.md` and `agent/verification/AGENTS.md` replace their shared opening sentence with a link to the skill and keep their stack-specific rules. `common-prompt.md` adds a reading step for the skill when the work involves tests, and `diff-prompt.md` checks tests against it.++Rejected: copying the sibling project's skill whole. Its scenario-prose test names contradict `backend/AGENTS.md` and ADR-0013. Its template database per test does not match the slot databases here. Rejected: putting the shared rules in the root `AGENTS.md`. They matter only for test work, and the root file is read on every turn.++### Hooks start through `run.sh` with `mise exec`, and fail closed++`.agent-hooks/run.sh` finds `mise` on `PATH` or in `$HOME/.local/bin`, where the Fedora install puts it ("Development environment" in `AGENTS.md` names Fedora as the only supported host). It sets `MISE_EXEC_AUTO_INSTALL=false` and runs `mise -C <root> exec -- python3 review_gate.py "$@"`. When `mise` is missing or the hook exits with a non-zero code, it prints `{"continue": false, "stopReason": ..., "systemMessage": ...}` and exits with code 0. The message names the failure and says that `mise install` installs the pinned tools and `just doctor` checks the environment. `mise exec` with `MISE_EXEC_AUTO_INSTALL=false` still runs the command when a pinned version is missing, and `python3` and `openspec` then resolve to whatever else `PATH` holds. So before `mise exec`, `run.sh` runs `mise -C <root> ls --current --missing python npm:@fission-ai/openspec`, and any output or a non-zero exit code lands in the same stop message; for `doctor` it exits with code 1. The message is escaped for JSON before it is printed; the sibling script interpolates it raw.++The `doctor` action is the exception. `run.sh doctor` passes the exit code of `review_gate.py doctor` through with `exec`. When `mise` is missing, it prints the reason and exits with code 1. That way `just doctor` fails exactly when a tool is missing.++`mise.toml` pins `python = "3.14.7"` and `"npm:@fission-ai/openspec" = "1.11.0"`. 1.11.0 is the CLI the generated skills came from, so they keep matching it. `review_gate.py` raises a `MissingToolError` when `openspec` cannot start in `is_planning_complete`, and `main` turns it into the same stop message. A new `doctor` action prints one line per tool and exits non-zero when one is missing. The tools are Python, `git`, `openspec`, and `claude` or `codex`. The root `Justfile` gets `doctor: .agent-hooks/run.sh doctor`, and the root `AGENTS.md` command table gets the row. Every hook in `.claude/settings.json` and `.codex/hooks.json` runs through `run.sh`.++Rejected: searching the macOS Homebrew paths for `mise`, as the sibling script does. Other hosts are on their own here. Rejected: keeping `python3` from `PATH` and checking tools inside the hook. A client started from an IDE may not have the mise shims on `PATH`, so the hook would not start at all and could report nothing. That is how the sibling project lost its full reviews before it moved to `mise exec`. Rejected: upgrading OpenSpec to 1.13.2 while pinning. The upgrade regenerates the `openspec-*` skills, which is out of scope.++## Risks / Trade-offs++- [The shared test fixtures are down, so every `just test-package` run fails] → The turn is blocked with the connection error in the output. The testing skill tells the agent to report it to the user instead of starting the fixtures through `just test`. "Several agents, one set of containers" already says the developer starts the first full run.+- [The turn that checks the last task takes up to 30 more minutes] → Only the commands the change names run, and only on that turn and its repeats.+- [A hook test run claims a test slot database while an agent runs its own package tests] → Same contention as two agents running package tests today. Slots are claimed with advisory locks and wait up to 60 seconds (`backend/internal/config/config.go:46-53`).+- [`run.sh` stops the agent on every turn when mise is broken or a pinned tool is missing] → Intended: a silent skip is worse. The stop message names `mise install` and `just doctor`.+- [A reviewer asks for the section in a change without tests] → The proposal prompt limits the request to tasks that add or change tests.++## Migration Plan++Existing active changes have no "Change verification" section, so the hook runs no tests for them, as today. OpenSpec 1.11.0 is not installed in mise on this machine, so every worktree needs `mise install` before its first turn after this change lands; until then each hook stops the agent with the message that names it. A developer without `mise` on `PATH` or in `$HOME/.local/bin` gets the same stop message. Rollback is a revert of the change. The old hook commands in the client configs come back with it.diff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/proposal.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/proposal.mdnew file mode 100644index 00000000..43968e31--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/proposal.md@@ -0,0 +1,48 @@+## Why++The full implementation review never sees a test run. The hook runs only the linters of the touched modules, and its prompt tells the reviewers to "check the agent's claims about tests against the repository" (`.agent-hooks/implementation-prompt.md:3`). A reviewer can read a test, but it cannot tell whether the test passes, and nobody reviews the tests themselves as a whole: the delta spec reviewer only matches each scenario to a test. Test rules are also scattered across the module docs, and some of them are duplicated there, so a reviewer has no single standard to check tests against.++The hook can also fail silently. It starts through `python3` from the user's shell (`.claude/settings.json`), and it finds `openspec` the same way. When `openspec` is missing, the proposal review quietly stops running (`.agent-hooks/review_gate.py:660-674`). A check that times out leaves the processes it spawned running, because only the direct child is killed (`review_gate.py:738-758`).++The sibling project the same team works in solved the launch and timeout problems and lets a change name test commands the hook runs. This change brings that here. It keeps this repository's rule that agents never run the full backend suite, because that suite resets the test containers other agents share.++## Governing docs++This change answers to the root [`AGENTS.md`](../../../AGENTS.md). It edits the testing sections of [`backend/AGENTS.md`](../../../backend/AGENTS.md) and [`agent/verification/AGENTS.md`](../../../agent/verification/AGENTS.md), so it answers to those as well. It touches no product code.++## What Changes++- A change's `tasks.md` can end with a "Change verification" section: a shell code block with one test command per line. The full implementation review runs these commands after the linters and before the reviewers, and blocks the turn with the end of the output when one fails. Only three command forms are allowed, and their arguments are checked against a safe character set: backend package tests, the verification agent's tests and the frontend tests. The full backend suite is not among them. The full proposal review checks the section's format before validation.+- The full implementation review gains a tests reviewer. It checks the tests the change added or changed against the testing skill and the module docs. A test must fail without the behavior it pins, and a bug fix needs a test that reproduces the bug. It also checks that the change verification commands cover the packages whose tests the change touched.+- The task reviewer of the full proposal review checks that a change whose tasks add or change tests that an allowed form runs has the "Change verification" section, and that the section covers the packages named in the tasks' verifications. The hook tests are exempt, because the module check already runs them.+- A new `testing` skill holds the test rules that apply to every module: run what you wrote, start a bug fix with a failing test, wait for conditions instead of sleeping, clean up only your own leftovers after an interrupted run. The module docs keep their stack-specific rules and link to the skill. Reviewers load the skill when the work involves tests.+- The hooks start through a wrapper script that runs the hook with the Python and OpenSpec versions pinned in `mise.toml`. When the wrapper or a tool is missing, the hook stops the agent with the reason instead of skipping the review. A new `just doctor` command checks the tools the hooks need.+- A check that times out is stopped together with every process it spawned. Each full review runs inside a time budget that ends before the hook timeout: the checks share one total limit, and a reviewer starts only when its whole limit still fits, otherwise the hook reports it. Both full review hook timeouts grow from 3600 to 7200 seconds.++### Out of scope++- Choosing tests automatically from the changed files. The change names its commands.+- Running the full backend suite (`just test`) or the Docker filesystem suite from the hook or from a task.+- Upgrading the OpenSpec CLI or regenerating the `openspec-*` skills. `mise.toml` pins the version already in use.+- Moving the backend naming rules, controller test rules or the canonical controller test out of `backend/AGENTS.md`. They are stack-specific.++## Capabilities++### New Capabilities++- `review-hook-launch`: how the hooks start with the pinned tools, what happens when a tool is missing, and how the user checks the environment.++### Modified Capabilities++- `implementation-review`: the checks before the reviewers also run the change's verification commands, under a shared time limit and with process groups stopped on timeout. A tests reviewer joins the parallel reviewers. New requirements cover which verification commands are allowed and the time budget that keeps the review inside the hook timeout.+- `proposal-review`: the checks before the reviewers also validate the format of the "Change verification" section. The task reviewer checks that the section exists when the tasks add or change tests an allowed form runs. A new requirement covers the same time budget.++## Impact++- `.agent-hooks/`: new `run.sh`; `review_gate.py`, `review_gate_test.py`, `common-prompt.md`, `diff-prompt.md`, `implementation-prompt.md` and `proposal-prompt.md` change.+- `.claude/settings.json`, `.codex/hooks.json`: every hook runs through `run.sh`, and both full review hook timeouts become 7200 seconds.+- `mise.toml`: pins `python` and `npm:@fission-ai/openspec`. Every worktree needs `mise install` before its first turn after the change lands.+- `Justfile`: a `doctor` recipe.+- `.agents/skills/testing/SKILL.md` and a symlink to it in `.claude/skills/testing/`.+- `AGENTS.md`, `backend/AGENTS.md`, `agent/verification/AGENTS.md`: the command table gains `just doctor`, a pointer to the testing skill, and duplicated test rules replaced by links.+- `openspec/config.yaml`: a tasks rule describing the "Change verification" section.diff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/implementation-review/spec.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/implementation-review/spec.mdnew file mode 100644index 00000000..3aa11996--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/implementation-review/spec.md@@ -0,0 +1,117 @@+## MODIFIED Requirements++### Requirement: Read-only checks of the touched modules run first+Before starting reviewers the hook SHALL run the checks of every module the working tree changes relative to the last commit: the Go linter for the backend and for the verification agent, the linter for the frontend, the linter and the build for the website, and the hook tests when the hooks changed. After them the hook SHALL run the change verification commands of every change it reviews, each command once even when several changes name it. The checks MUST NOT modify files in the working tree, so formatters do not run. The hook MUST NOT run the full backend suite or the Docker filesystem suite. When a check fails, the hook SHALL block the turn and pass the agent the command and the end of its output. Reviewers MUST NOT start in that case.++Each check SHALL have its own time limit, and all checks of one review SHALL share a total time limit, both within the time budget of the requirement "The full review ends within the hook timeout". When a check exceeds either limit, the hook SHALL stop the check together with every process it started, block the turn and name the limit that was exceeded.++#### Scenario: The backend linter fails+- **WHEN** the change touches the backend and its linter fails+- **THEN** the turn is blocked, the agent gets the end of the linter output, and no reviewer starts++#### Scenario: Only the frontend changed+- **WHEN** the change touches only the frontend and has no change verification commands+- **THEN** the hook runs the frontend linter and no backend, agent or website check++#### Scenario: The website changed+- **WHEN** the change touches the website+- **THEN** the hook runs both the website linter and the website build++#### Scenario: A package test fails+- **WHEN** the change verification commands run the tests of a backend package and one of them fails+- **THEN** the turn is blocked, the agent gets the command and the end of the test output, and no reviewer starts++#### Scenario: A test hangs+- **WHEN** a change verification command runs longer than its time limit+- **THEN** the hook stops the command and every process it started, blocks the turn and names the exceeded limit++#### Scenario: The checks together exceed the total limit+- **WHEN** each check stays under its own time limit, but together they run past the total limit+- **THEN** the hook stops the running check and every process it started, blocks the turn and names the total limit++#### Scenario: The linters pass and the change has no verification commands+- **WHEN** the change has no "Change verification" section and the linters pass+- **THEN** the reviewers start without any test run++### Requirement: What the full implementation review checks+The full implementation review SHALL run several read-only reviewers in parallel: one per delta spec of the change, one for documentation and the task list, and one for tests. The delta spec reviewer SHALL find the code and the test for every requirement and scenario, check that the test actually proves the scenario, check the design decisions against the code, and look for bugs. The documentation reviewer SHALL check that everything each task states was done, including its verification, and that documentation agrees with itself and with the code. The documentation reviewer SHALL also check edits to agent instructions made by the change against the requirement "What belongs in agent instructions" of the `agent-instructions` capability.++The tests reviewer SHALL check every test the change added or changed against the repository's testing skill and the testing rules of the module docs: the test fails without the behavior it pins, a bug fix has a test that reproduces the bug, the cases the module doc asks for are covered, the test waits for a condition instead of a fixed pause, and it cleans up its data. The tests reviewer SHALL also check that the change verification commands run the tests of every backend package, verification agent package and frontend test file the change added or changed tests in. Tests that no allowed command form runs and that a module check already runs, such as the hook tests, count as covered.++#### Scenario: A scenario without a test+- **WHEN** a delta spec scenario has no test, or its test passes without the described behavior+- **THEN** the reviewer returns a `missing_test` finding with the path to the test or code++#### Scenario: Documentation lags behind+- **WHEN** the change adds an environment variable and the configuration documentation does not mention it+- **THEN** the documentation reviewer returns a `docs` finding++#### Scenario: The change retells a document in AGENTS.md+- **WHEN** the change adds a list of a module doc's sections to the root `AGENTS.md`+- **THEN** the documentation reviewer returns a `docs` finding that suggests naming the doc's topic instead++#### Scenario: The change adds a rule to AGENTS.md+- **WHEN** the change adds a rule to `AGENTS.md` that changes agent behavior and is written nowhere else+- **THEN** the documentation reviewer returns no finding under the instruction content criterion++#### Scenario: A change with two delta specs+- **WHEN** the full implementation review starts for a change with two delta specs+- **THEN** the hook starts four reviewers in parallel: one per delta spec, one for documentation and tasks, and one for tests++#### Scenario: A test that waits with a fixed pause+- **WHEN** a test the change added sleeps for a fixed time before checking that a backup finished+- **THEN** the tests reviewer returns a finding that asks to wait for the condition instead++#### Scenario: A bug fix without a reproducing test+- **WHEN** a change fixes a bug and no test it added or changed fails on the code before the fix+- **THEN** the tests reviewer returns a `missing_test` finding++#### Scenario: Changed tests outside the verification commands+- **WHEN** the change adds tests to a backend package that none of its change verification commands runs+- **THEN** the tests reviewer returns a `missing_test` finding that names the package++#### Scenario: A change with only hook tests+- **WHEN** the change adds tests only to the hook tests and has no "Change verification" section+- **THEN** the tests reviewer returns no finding about the missing section, because the module check runs the hook tests++## ADDED Requirements++### Requirement: Which change verification commands the hook runs+The change verification commands SHALL be the lines of the first code block fenced as `sh` under the "Change verification" heading of the change's task list. A block fenced as `bash`, `shell` or without a language does not count. Empty lines and comment lines are skipped. The hook SHALL accept only these command forms:++- backend package tests: `just test-package` followed by one or more package paths, run in the backend directory;+- the verification agent's tests: `just --justfile agent/verification/Justfile test`, with no further arguments;+- frontend tests: `pnpm --dir frontend test`, optionally followed by test file paths or filters.++The hook SHALL run each command directly, without a shell. Every argument after the fixed prefix MUST consist only of letters, digits and the characters `.`, `/`, `_`, `-`, `=`, `:` and `@`, and a package path MUST start with `./`. This character set leaves out every shell operator and substitution. A line that is not one of the allowed forms or has an argument outside the allowed characters, a section without a closed `sh` block, and a block without commands SHALL each block the turn with the reason and the list of allowed forms. No check or reviewer runs in that case.++#### Scenario: A full suite run in the block+- **WHEN** the change verification block contains `just test`+- **THEN** the turn is blocked with the reason that the command is not allowed, and nothing runs++#### Scenario: A shell operator+- **WHEN** a line reads `just test-package ./internal/features/intervals/... && rm -rf /tmp/x`+- **THEN** the turn is blocked with the reason that the argument `&&` has a forbidden character, and nothing runs++#### Scenario: An argument with a forbidden character+- **WHEN** a line reads `just test-package ./internal/a;rm`+- **THEN** the turn is blocked with the reason that an argument has a forbidden character, and nothing runs++#### Scenario: Two changes name the same command+- **WHEN** the hook reviews two changes whose blocks both contain `pnpm --dir frontend test`+- **THEN** the command runs once++#### Scenario: Backend package tests+- **WHEN** a block contains `just test-package ./internal/features/intervals/...`+- **THEN** the hook runs that command in the backend directory++### Requirement: The full review ends within the hook timeout+The full implementation review SHALL work within a time budget that ends before the hook timeout of the client configuration, with a margin for the hook's own Git, OpenSpec and file work. The checks SHALL run inside that budget. A reviewer SHALL start only when the budget still holds its whole time limit. A reviewer that cannot start for that reason SHALL count as a reviewer that could not run: the review is not accepted, and the agent gets the reason. The hook MUST NOT be killed by the client timeout without a message.++#### Scenario: Too many reviewers for the budget+- **WHEN** a change has so many delta specs that a third wave of reviewers would not fit in the remaining budget+- **THEN** the reviewers of that wave do not start, the turn is blocked, and the agent gets the reason that the review time budget ran out++#### Scenario: Two waves of reviewers after slow checks+- **WHEN** the checks use their whole total limit and the change needs two waves of reviewers+- **THEN** both waves run and the hook answers before the client timeoutdiff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/proposal-review/spec.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/proposal-review/spec.mdnew file mode 100644index 00000000..7f36c827--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/proposal-review/spec.md@@ -0,0 +1,46 @@+## MODIFIED Requirements++### Requirement: OpenSpec validation runs first+Before starting reviewers the hook SHALL check the format of the change's "Change verification" section by the rules the full implementation review applies to it, without running its commands, and then run strict OpenSpec validation of the change. When the section's format is wrong or validation fails, the hook SHALL block the turn and pass the agent the reason or the end of the validation output. Reviewers MUST NOT start in that case.++#### Scenario: A requirement without a scenario+- **WHEN** a delta spec has a requirement without a scenario and validation fails+- **THEN** the turn is blocked, the agent gets the validation output, and no reviewer starts++#### Scenario: A forbidden command in the verification section+- **WHEN** the change's "Change verification" section contains `just test`+- **THEN** the turn is blocked with the reason and the list of allowed forms, the command does not run, and no reviewer starts++### Requirement: What the full proposal review checks+The full proposal review SHALL run several read-only reviewers in parallel, on the same models as the full implementation review: one per delta spec of the change, one for the proposal and design, and one for the task list.++The delta spec reviewer SHALL check that requirements are complete and testable, do not contradict the main specs, the proposal or the design, and that claims about existing code are true. The proposal and design reviewer SHALL look for a noticeably simpler way to reach the same result, following the repository rule that the best code is unwritten, check the design decisions against the module architecture rules and accepted ADRs, and check that decisions agree with each other. The task reviewer SHALL check that a task with a test covers every scenario, that every task has a verification, and that the task order is feasible. The task reviewer SHALL also check that a change whose tasks add or change tests in a backend package, a verification agent package or the frontend has a "Change verification" section, and that the section runs the tests of every such package the tasks' verifications name. Tests that no allowed command form runs and that a module check already runs, such as the hook tests, need no section.++#### Scenario: A change with two delta specs+- **WHEN** the full proposal review starts for a change with two delta specs+- **THEN** the hook starts four reviewers in parallel: one per delta spec, one for proposal and design, and one for tasks++#### Scenario: The task reviewer's role+- **WHEN** the hook starts the task reviewer+- **THEN** its prompt names the role and requires checking that a task with a test covers every scenario, every task has a verification, the task order is feasible, and the "Change verification" section covers the tested packages++#### Scenario: Tests without a verification section+- **WHEN** the tasks add tests to a backend package and the change has no "Change verification" section+- **THEN** the task reviewer returns a `tasks` finding that asks for the section with that package++#### Scenario: A section that leaves out a package+- **WHEN** a task's verification runs the tests of two backend packages and the "Change verification" section runs only one of them+- **THEN** the task reviewer returns a `tasks` finding that names the missing package++#### Scenario: A change with only hook tests+- **WHEN** the tasks add tests only to the hook tests and the change has no "Change verification" section+- **THEN** the task reviewer returns no finding about the missing section++## ADDED Requirements++### Requirement: The full proposal review ends within the hook timeout+The full proposal review SHALL work within the same kind of time budget as the full implementation review: the budget ends before the hook timeout with a margin, validation runs inside it, and a reviewer starts only when the budget still holds its whole time limit. A reviewer that cannot start for that reason SHALL count as a reviewer that could not run. The hook MUST NOT be killed by the client timeout without a message.++#### Scenario: Too many reviewers for the budget+- **WHEN** a proposal has so many delta specs that a third wave of reviewers would not fit in the remaining budget+- **THEN** the reviewers of that wave do not start, the turn is blocked, and the agent gets the reason that the review time budget ran outdiff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/review-hook-launch/spec.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/review-hook-launch/spec.mdnew file mode 100644index 00000000..6bdda608--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/specs/review-hook-launch/spec.md@@ -0,0 +1,46 @@+## Purpose++Makes the review hooks start the same way in every environment and refuse to stay silent: a hook that cannot run stops the agent with a reason, so a review is never skipped without the user knowing.++## ADDED Requirements++### Requirement: Hooks run with the repository's pinned tools+Every review hook, in Claude Code and in Codex, SHALL start through one launcher that runs the hook with the Python and OpenSpec versions pinned in the repository's tool versions file. The launcher SHALL find the version manager even when the client was started outside a login shell, for example from an IDE. The launcher MUST NOT install a missing tool itself, because an installation would not fit the hook timeout.++#### Scenario: A client started from an IDE+- **WHEN** Claude Code was started from an IDE whose environment does not activate the version manager, and the agent finishes a turn+- **THEN** the hooks run with the pinned Python and OpenSpec versions++#### Scenario: A pinned version is not installed+- **WHEN** the version manager is present but the pinned OpenSpec version is not installed, and the agent finishes a turn+- **THEN** no installation starts, the agent stops, and the user sees that the hooks did not start, that `mise install` installs the pinned tools and that `just doctor` checks the environment++### Requirement: A hook that cannot run stops the agent+When the launcher cannot find the version manager, when the hook exits with an error, or when a tool a gate needs is missing, the hook SHALL stop the agent and show the user the reason and the command that checks the environment. The hook MUST NOT let the turn pass as if the review had run.++#### Scenario: OpenSpec is missing+- **WHEN** the OpenSpec CLI cannot be found and a turn finishes that wrote the artifacts of a change+- **THEN** the agent stops, and the user sees that the hooks could not find `openspec` and that `just doctor` checks the environment++#### Scenario: The version manager is missing+- **WHEN** the launcher cannot find the version manager+- **THEN** the agent stops, and the user sees that the version manager was not found and that `just doctor` checks the environment++#### Scenario: The hook crashes+- **WHEN** the hook process exits with a non-zero code+- **THEN** the agent stops, and the user sees which hook action failed, its exit code, and that `just doctor` checks the environment++### Requirement: The environment check reports every tool the hooks need+The repository SHALL have a command that reports, one line per tool, whether the hooks can find Python, git, OpenSpec and the CLI of at least one reviewer client, Claude Code or Codex. The command SHALL run through the same launcher as the hooks, and SHALL exit with a non-zero code when any of them is missing or the version manager cannot be found.++#### Scenario: Every tool is present+- **WHEN** the user runs `just doctor` with every tool installed+- **THEN** each tool is reported with its path and the command exits with code 0++#### Scenario: No reviewer CLI+- **WHEN** neither the Claude Code CLI nor the Codex CLI is installed and the user runs `just doctor`+- **THEN** the command reports that no reviewer CLI was found and exits with a non-zero code++#### Scenario: No version manager+- **WHEN** the version manager cannot be found and the user runs `just doctor`+- **THEN** the command reports that the version manager was not found and exits with a non-zero codediff --git a/openspec/changes/archive/2026-10-18-run-change-tests-in-review/tasks.md b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/tasks.mdnew file mode 100644index 00000000..f9f76c1e--- /dev/null+++ b/openspec/changes/archive/2026-10-18-run-change-tests-in-review/tasks.md@@ -0,0 +1,75 @@+The hook test command below runs from the repository root: `python3 -m unittest discover -s .agent-hooks -p '*_test.py'`.++## 1. Hook launch++- [x] 1.1 Pin `python = "3.14.7"` and `"npm:@fission-ai/openspec" = "1.11.0"` in `mise.toml`. Verify with `mise install`, then `mise exec -- python3 --version` printing `Python 3.14.7` and `mise exec -- openspec --version` printing `1.11.0`.+- [x] 1.2 Add `.agent-hooks/run.sh` per design.md. It finds `mise` on `PATH` or in `$HOME/.local/bin`, refuses to start when `mise ls --current --missing` reports the pinned `python` or `npm:@fission-ai/openspec` missing, sets `MISE_EXEC_AUTO_INSTALL=false` and runs `review_gate.py` through `mise exec`. When `mise` is missing, a pinned tool is missing or the hook fails, it prints an escaped stop JSON with exit code 0. For `doctor` it passes the exit code through and exits with code 1 when `mise` or a pinned tool is missing. Add `LauncherTest` to `review_gate_test.py`. It runs the script with a stub `mise`, which records its arguments and environment, and a temporary `HOME`:+ - with no `mise` on `PATH` but a stub in `$HOME/.local/bin`, the hook runs through the stub (`review-hook-launch` scenario "A client started from an IDE");+ - the stub sees `MISE_EXEC_AUTO_INSTALL=false`, and when its `ls --missing` reports a missing pinned tool while its `exec` still succeeds, like real `mise`, the output is valid JSON with `continue: false` that names `mise install` and `just doctor`, and `exec` never runs ("A pinned version is not installed");+ - with no `mise` anywhere, the output is valid JSON with `continue: false` that names the version manager and `just doctor` ("The version manager is missing");+ - with a stub that exits with code 3, the output is valid JSON that names the action, the code and `just doctor` ("The hook crashes");+ - a message containing a double quote stays valid JSON;+ - `run.sh doctor` exits with the code of the stub, and with code 1 when no `mise` is found ("No version manager").++ Verify with the hook test command.+- [x] 1.3 In `review_gate.py`, raise `MissingToolError` when `openspec` cannot start in `is_planning_complete`, and turn it into a stop message that names the tool and `just doctor`. Add the `doctor` action, which reports Python, `git`, `openspec` and the reviewer CLIs, and keep `--client` required for every other action. Update the module docstring and the comment above `is_planning_complete`. Add these tests:+ - `MissingToolTest`, covering "OpenSpec is missing" with `openspec` removed from the test `PATH`;+ - `DoctorTest`, covering "Every tool is present" and "No reviewer CLI".++ Verify with the hook test command.+- [x] 1.4 Add the `doctor` recipe (`.agent-hooks/run.sh doctor`) to the root `Justfile` and its row to the command table in the root `AGENTS.md`. Verify with `just doctor` exiting with code 0, and with `PATH=/usr/bin HOME=$(mktemp -d) .agent-hooks/run.sh doctor` exiting with a non-zero code.+- [x] 1.5 Switch every hook in `.claude/settings.json` and `.codex/hooks.json` to `run.sh`, and set the timeouts of the `implementation` and `proposal` hooks to 7200 in both files. Verify with `python3 -m json.tool` on both files, and with `echo '{"hook_event_name":"Stop","session_id":"manual-check","cwd":"'$PWD'"}' | .agent-hooks/run.sh naming --client claude` printing `{}`.++## 2. Checks, time budget and change verification commands++- [x] 2.1 Rewrite `run_checks` per design.md: start every check in its own process group, kill the group on timeout, and enforce the shared 1800-second total limit on top of the 900-second limit per check. The failure message names the exceeded limit. Make the limits parameters so tests can shrink them. Add `RunChecksTest`:+ - a check whose child process outlives the timeout leaves no process behind (`implementation-review` scenario "A test hangs");+ - two checks that each stay under their own limit but together pass the total limit fail with a message naming the total limit, and the second check's process group is gone ("The checks together exceed the total limit").++ Verify with the hook test command.+- [x] 2.2 Add the time budget of both full reviews per design.md: 6600 seconds from the hook's start, with checks inside it, and a reviewer job that starts only when the budget still holds its whole limit. A job that cannot start returns the "review time budget ran out" error. Make the budget a parameter. Add tests to `ImplementationGateReviewTest` and `ProposalGateReviewTest` with a shrunk budget and reviewer limit:+ - a third wave of reviewers does not start, and the turn is blocked with the budget reason (both "Too many reviewers for the budget" scenarios);+ - two waves that fit both run ("Two waves of reviewers after slow checks").++ Verify with the hook test command.+- [x] 2.3 Add the change verification parser per the `implementation-review` requirement "Which change verification commands the hook runs" and design.md. It reads the first `sh`-fenced block under `## Change verification`, skips empty and comment lines, accepts the three allowed forms with their directories, and checks every argument against the character set. Every error message carries the list of allowed forms. Add `ChangeVerificationParsingTest` with one test per scenario of that requirement except "Two changes name the same command", plus tests for:+ - `just test-package` with no package;+ - a package path without `./`;+ - extra arguments after the verification agent's form;+ - a `bash`-fenced block, which does not count;+ - an unclosed block;+ - an empty block;+ - a missing section, which yields no commands.++ Verify with the hook test command.+- [x] 2.4 Make the implementation review parse the sections of every change it reviews before any check, run the de-duplicated commands after the module checks, and block on a parse error or a failing command. Update `test_never_runs_formatters_or_just` so that `just` runs only from a change verification block. Add tests to `ImplementationGateChecksTest` for these `implementation-review` scenarios:+ - "A package test fails": no reviewer runs and the output tail is passed;+ - "Backend package tests": the command runs in `backend/`;+ - "Two changes name the same command": the command runs once;+ - "The linters pass and the change has no verification commands": reviewers start and no test runs;+ - a malformed section, after which no check and no reviewer runs.++ Verify with the hook test command.+- [x] 2.5 Make the proposal review parse the section of each change before `openspec validate --strict`, without running its commands, and block on a parse error. Add a test to `ProposalGateValidationTest` for the `proposal-review` scenario "A forbidden command in the verification section": the turn is blocked, the stub `just` is never called and no reviewer runs. Verify with the hook test command.++## 3. Reviewers and prompts++- [x] 3.1 Add the `tests` role to the implementation review's `first_roles` and a "Role: tests" section to `implementation-prompt.md` per design.md, including the exemption for tests that a module check runs. Rewrite the prompt's opening so it says the hook ran the linters and the change verification commands. Raise by one the reviewer counts that the existing implementation review tests assert (`review_gate_test.py:447`, `:493`, `:591`, `:621`, `:632`, `:657`, `:680`). Add tests to `ImplementationGateReviewTest`:+ - a change with two delta specs starts four reviewers (`implementation-review` scenario "A change with two delta specs");+ - the `tests` role's prompt requires checking that a test fails without its behavior, that a bug fix has a reproducing test, that tests wait for conditions instead of fixed pauses, and that the change verification commands cover every package with changed tests that an allowed form runs, with hook tests counted as covered. This covers "A test that waits with a fixed pause", "A bug fix without a reproducing test", "Changed tests outside the verification commands" and "A change with only hook tests".++ Verify with the hook test command.+- [x] 3.2 Extend the tasks role in `proposal-prompt.md`. A change whose tasks add or change tests in a backend package, a verification agent package or the frontend needs a `## Change verification` section that covers every such package the tasks' verifications name. Hook tests need no section. Add a test to `ProposalGateReviewTest` that the task reviewer's prompt requires this and states the exemption. It covers the `proposal-review` scenarios "The task reviewer's role", "Tests without a verification section", "A section that leaves out a package" and "A change with only hook tests". Verify with the hook test command.+- [x] 3.3 Add the testing skill as a reading step to `common-prompt.md` for work that involves tests, and point the tests line of `diff-prompt.md` at the skill and the module docs. Verify with `grep -n "skills/testing/SKILL.md" .agent-hooks/common-prompt.md .agent-hooks/diff-prompt.md` finding both.++## 4. Testing skill and documentation++- [x] 4.1 Write `.agents/skills/testing/SKILL.md` with the rules design.md lists, in the repository's skill format with `name` and `description` front matter, linking the module docs for stack-specific rules. Create `.claude/skills/testing/SKILL.md` as a relative symlink to `../../../.agents/skills/testing/SKILL.md`. Verify with `readlink .claude/skills/testing/SKILL.md` printing that target and `head -4 .claude/skills/testing/SKILL.md` printing the front matter.+- [x] 4.2 Add the "Tests" section to the root `AGENTS.md`. Replace the shared opening sentence of the "Testing" sections in `backend/AGENTS.md` and `agent/verification/AGENTS.md` with a link to the skill. Add the tasks rule for the `## Change verification` section to `openspec/config.yaml`: last section, no checkboxes, one `sh`-fenced block, one command per line, the three allowed forms, and no section needed for hook tests. Rewrite the comment above `MODULE_CHECKS` to match the new behavior. Verify with these searches:+ - `grep -rn "Always run tests after writing them" AGENTS.md backend/AGENTS.md agent/verification/AGENTS.md` returns nothing;+ - `grep -n "Change verification" openspec/config.yaml` finds the rule.+- [x] 4.3 Search the repository for statements the change made untrue and bring them in line per "Documentation consistency". Look for the hook not running tests, the 3600-second full review timeouts, and `python3` as the hook command. Verify with `grep -rn "does not run the Go test suites\|never runs here" .agent-hooks AGENTS.md backend/AGENTS.md` and `grep -n "review_gate.py" .claude/settings.json .codex/hooks.json` both returning nothing.++## 5. Change validation++- [x] 5.1 Verify with `openspec validate run-change-tests-in-review --strict` passing, the hook test command passing and `just doctor` exiting with code 0.diff --git a/openspec/config.yaml b/openspec/config.yamlindex b123d9e7..78f5e8e8 100644--- a/openspec/config.yaml+++ b/openspec/config.yaml@@ -56,6 +56,7 @@ rules:- Each task's verification is a local command the agent runs itself, never "should work". Valid checks are the tests of the touched packages and of the packages that test the same feature (just test-package ./internal/features/<feature>/... in backend/), just lint at the root or in a module, just build and just test in agent/verification, pnpm lint, pnpm typecheck, pnpm test, pnpm build, or an automated test that pins the behavior.- Every task is one the agent can complete and verify on its own. Every task closes without the user stepping in and without the full backend suite. Keep these out of tasks and out of other tasks' verification, including a final verification task - the full backend suite (just test at the root), the Docker filesystem suite (just test-filesystem), deploys, CI pipeline runs, any check that needs a push, and manual app runs or browser clicking. The full backend suite resets the test containers other agents share, so it is left to the developer and CI.- Order by dependency, smallest slice first when the work is repetitive.+ - When the tasks add or change tests in a backend package, a verification agent package or the frontend, end tasks.md with a "## Change verification" section that runs every such package the tasks' verifications name. It is the last section, has no checkboxes and holds one code block fenced as sh, with one command per line. The full implementation review runs these commands, and only three forms are allowed - just test-package followed by package paths starting with ./ (runs in backend/), just --justfile agent/verification/Justfile test with nothing after it, and pnpm --dir frontend test with optional test paths or filters. Arguments hold only letters, digits, dots, slashes, underscores, hyphens, equals signs, at signs and colons. The hook tests in .agent-hooks/ need no section, because the hook's module check runs them.- End the task list at the pre-commit boundary. The change is complete when implementation, required documentation, verification and mandatory review are finished, and the working tree is ready to commit.- Never add tasks for committing, pushing, creating or updating a pull request, requesting pull request review, merging, or any later Git or repository-hosting workflow. Completion status updates must be ready for inclusion in the same commit as the implementation.diff --git a/openspec/specs/implementation-review/spec.md b/openspec/specs/implementation-review/spec.mdindex 6a01a5b5..2fa867a3 100644--- a/openspec/specs/implementation-review/spec.md+++ b/openspec/specs/implementation-review/spec.md@@ -29,22 +29,42 @@ The hook that runs when the agent finishes a turn, in Claude Code and in Codex,- **THEN** the full implementation review starts again### Requirement: Read-only checks of the touched modules run first-Before starting reviewers the hook SHALL run the checks of every module the working tree changes relative to the last commit: the Go linter for the backend and for the verification agent, the linter for the frontend, the linter and the build for the website, and the hook tests when the hooks changed. The checks MUST NOT modify files, so formatters do not run. The hook MUST NOT run the backend test suites. When a check fails, the hook SHALL block the turn and pass the agent the command and the end of its output. Reviewers MUST NOT start in that case.+Before starting reviewers the hook SHALL run the checks of every module the working tree changes relative to the last commit: the Go linter for the backend and for the verification agent, the linter for the frontend, the linter and the build for the website, and the hook tests when the hooks changed. After them the hook SHALL run the change verification commands of every change it reviews, each command once even when several changes name it. The checks MUST NOT modify files in the working tree, so formatters do not run. The hook MUST NOT run the full backend suite or the Docker filesystem suite. When a check fails, the hook SHALL block the turn and pass the agent the command and the end of its output. Reviewers MUST NOT start in that case.++Each check SHALL have its own time limit, and all checks of one review SHALL share a total time limit, both within the time budget of the requirement "The full review ends within the hook timeout". When a check exceeds either limit, the hook SHALL stop the check together with every process it started, block the turn and name the limit that was exceeded.#### Scenario: The backend linter fails- **WHEN** the change touches the backend and its linter fails- **THEN** the turn is blocked, the agent gets the end of the linter output, and no reviewer starts#### Scenario: Only the frontend changed-- **WHEN** the change touches only the frontend+- **WHEN** the change touches only the frontend and has no change verification commands- **THEN** the hook runs the frontend linter and no backend, agent or website check#### Scenario: The website changed- **WHEN** the change touches the website- **THEN** the hook runs both the website linter and the website build+#### Scenario: A package test fails+- **WHEN** the change verification commands run the tests of a backend package and one of them fails+- **THEN** the turn is blocked, the agent gets the command and the end of the test output, and no reviewer starts++#### Scenario: A test hangs+- **WHEN** a change verification command runs longer than its time limit+- **THEN** the hook stops the command and every process it started, blocks the turn and names the exceeded limit++#### Scenario: The checks together exceed the total limit+- **WHEN** each check stays under its own time limit, but together they run past the total limit+- **THEN** the hook stops the running check and every process it started, blocks the turn and names the total limit++#### Scenario: The linters pass and the change has no verification commands+- **WHEN** the change has no "Change verification" section and the linters pass+- **THEN** the reviewers start without any test run+### Requirement: What the full implementation review checks-The full implementation review SHALL run several read-only reviewers in parallel: one per delta spec of the change and one for documentation and the task list. The delta spec reviewer SHALL find the code and the test for every requirement and scenario, check that the test actually proves the scenario, check the design decisions against the code, and look for bugs. The documentation reviewer SHALL check that everything each task states was done, including its verification, and that documentation agrees with itself and with the code. The documentation reviewer SHALL also check edits to agent instructions made by the change against the requirement "What belongs in agent instructions" of the `agent-instructions` capability.+The full implementation review SHALL run several read-only reviewers in parallel: one per delta spec of the change, one for documentation and the task list, and one for tests. The delta spec reviewer SHALL find the code and the test for every requirement and scenario, check that the test actually proves the scenario, check the design decisions against the code, and look for bugs. The documentation reviewer SHALL check that everything each task states was done, including its verification, and that documentation agrees with itself and with the code. The documentation reviewer SHALL also check edits to agent instructions made by the change against the requirement "What belongs in agent instructions" of the `agent-instructions` capability.++The tests reviewer SHALL check every test the change added or changed against the repository's testing skill and the testing rules of the module docs: the test fails without the behavior it pins, a bug fix has a test that reproduces the bug, the cases the module doc asks for are covered, the test waits for a condition instead of a fixed pause, and it cleans up its data. The tests reviewer SHALL also check that the change verification commands run the tests of every backend package, verification agent package and frontend test file the change added or changed tests in. Tests that no allowed command form runs and that a module check already runs, such as the hook tests, count as covered.#### Scenario: A scenario without a test- **WHEN** a delta spec scenario has no test, or its test passes without the described behavior@@ -62,6 +82,26 @@ The full implementation review SHALL run several read-only reviewers in parallel- **WHEN** the change adds a rule to `AGENTS.md` that changes agent behavior and is written nowhere else- **THEN** the documentation reviewer returns no finding under the instruction content criterion+#### Scenario: A change with two delta specs+- **WHEN** the full implementation review starts for a change with two delta specs+- **THEN** the hook starts four reviewers in parallel: one per delta spec, one for documentation and tasks, and one for tests++#### Scenario: A test that waits with a fixed pause+- **WHEN** a test the change added sleeps for a fixed time before checking that a backup finished+- **THEN** the tests reviewer returns a finding that asks to wait for the condition instead++#### Scenario: A bug fix without a reproducing test+- **WHEN** a change fixes a bug and no test it added or changed fails on the code before the fix+- **THEN** the tests reviewer returns a `missing_test` finding++#### Scenario: Changed tests outside the verification commands+- **WHEN** the change adds tests to a backend package that none of its change verification commands runs+- **THEN** the tests reviewer returns a `missing_test` finding that names the package++#### Scenario: A change with only hook tests+- **WHEN** the change adds tests only to the hook tests and has no "Change verification" section+- **THEN** the tests reviewer returns no finding about the missing section, because the module check runs the hook tests+### Requirement: Full review modelsThe full review reviewers SHALL run on Opus with `medium` effort in Claude Code and on `gpt-6.1-sol` with `high` effort in Codex. The quick checks SHALL run on Opus with `low` effort in Claude Code and on `gpt-6.1-sol` with `low` effort in Codex.@@ -109,3 +149,43 @@ While a change has an unaccepted full implementation review, the necessity check#### Scenario: The agent fixes a delta spec after a finding- **WHEN** after a `spec_mismatch` finding the agent edits the change's delta spec and finishes the turn- **THEN** the necessity check does not start, and the full review checks the edit++### Requirement: Which change verification commands the hook runs+The change verification commands SHALL be the lines of the first code block fenced as `sh` under the "Change verification" heading of the change's task list. A block fenced as `bash`, `shell` or without a language does not count. Empty lines and comment lines are skipped. The hook SHALL accept only these command forms:++- backend package tests: `just test-package` followed by one or more package paths, run in the backend directory;+- the verification agent's tests: `just --justfile agent/verification/Justfile test`, with no further arguments;+- frontend tests: `pnpm --dir frontend test`, optionally followed by test file paths or filters.++The hook SHALL run each command directly, without a shell. Every argument after the fixed prefix MUST consist only of letters, digits and the characters `.`, `/`, `_`, `-`, `=`, `:` and `@`, and a package path MUST start with `./`. This character set leaves out every shell operator and substitution. A line that is not one of the allowed forms or has an argument outside the allowed characters, a section without a closed `sh` block, and a block without commands SHALL each block the turn with the reason and the list of allowed forms. No check or reviewer runs in that case.++#### Scenario: A full suite run in the block+- **WHEN** the change verification block contains `just test`+- **THEN** the turn is blocked with the reason that the command is not allowed, and nothing runs++#### Scenario: A shell operator+- **WHEN** a line reads `just test-package ./internal/features/intervals/... && rm -rf /tmp/x`+- **THEN** the turn is blocked with the reason that the argument `&&` has a forbidden character, and nothing runs++#### Scenario: An argument with a forbidden character+- **WHEN** a line reads `just test-package ./internal/a;rm`+- **THEN** the turn is blocked with the reason that an argument has a forbidden character, and nothing runs++#### Scenario: Two changes name the same command+- **WHEN** the hook reviews two changes whose blocks both contain `pnpm --dir frontend test`+- **THEN** the command runs once++#### Scenario: Backend package tests+- **WHEN** a block contains `just test-package ./internal/features/intervals/...`+- **THEN** the hook runs that command in the backend directory++### Requirement: The full review ends within the hook timeout+The full implementation review SHALL work within a time budget that ends before the hook timeout of the client configuration, with a margin for the hook's own Git, OpenSpec and file work. The checks SHALL run inside that budget. A reviewer SHALL start only when the budget still holds its whole time limit. A reviewer that cannot start for that reason SHALL count as a reviewer that could not run: the review is not accepted, and the agent gets the reason. The hook MUST NOT be killed by the client timeout without a message.++#### Scenario: Too many reviewers for the budget+- **WHEN** a change has so many delta specs that a third wave of reviewers would not fit in the remaining budget+- **THEN** the reviewers of that wave do not start, the turn is blocked, and the agent gets the reason that the review time budget ran out++#### Scenario: Two waves of reviewers after slow checks+- **WHEN** the checks use their whole total limit and the change needs two waves of reviewers+- **THEN** both waves run and the hook answers before the client timeoutdiff --git a/openspec/specs/proposal-review/spec.md b/openspec/specs/proposal-review/spec.mdindex c5305892..266f7e30 100644--- a/openspec/specs/proposal-review/spec.md+++ b/openspec/specs/proposal-review/spec.md@@ -29,16 +29,20 @@ The hook that runs when the agent finishes a turn, in Claude Code and in Codex,- **THEN** the full proposal review starts again### Requirement: OpenSpec validation runs first-Before starting reviewers the hook SHALL run strict OpenSpec validation of the change. When validation fails, the hook SHALL block the turn and pass the agent the end of its output. Reviewers MUST NOT start in that case.+Before starting reviewers the hook SHALL check the format of the change's "Change verification" section by the rules the full implementation review applies to it, without running its commands, and then run strict OpenSpec validation of the change. When the section's format is wrong or validation fails, the hook SHALL block the turn and pass the agent the reason or the end of the validation output. Reviewers MUST NOT start in that case.#### Scenario: A requirement without a scenario- **WHEN** a delta spec has a requirement without a scenario and validation fails- **THEN** the turn is blocked, the agent gets the validation output, and no reviewer starts+#### Scenario: A forbidden command in the verification section+- **WHEN** the change's "Change verification" section contains `just test`+- **THEN** the turn is blocked with the reason and the list of allowed forms, the command does not run, and no reviewer starts+### Requirement: What the full proposal review checksThe full proposal review SHALL run several read-only reviewers in parallel, on the same models as the full implementation review: one per delta spec of the change, one for the proposal and design, and one for the task list.-The delta spec reviewer SHALL check that requirements are complete and testable, do not contradict the main specs, the proposal or the design, and that claims about existing code are true. The proposal and design reviewer SHALL look for a noticeably simpler way to reach the same result, following the repository rule that the best code is unwritten, check the design decisions against the module architecture rules and accepted ADRs, and check that decisions agree with each other. The task reviewer SHALL check that a task with a test covers every scenario, that every task has a verification, and that the task order is feasible.+The delta spec reviewer SHALL check that requirements are complete and testable, do not contradict the main specs, the proposal or the design, and that claims about existing code are true. The proposal and design reviewer SHALL look for a noticeably simpler way to reach the same result, following the repository rule that the best code is unwritten, check the design decisions against the module architecture rules and accepted ADRs, and check that decisions agree with each other. The task reviewer SHALL check that a task with a test covers every scenario, that every task has a verification, and that the task order is feasible. The task reviewer SHALL also check that a change whose tasks add or change tests in a backend package, a verification agent package or the frontend has a "Change verification" section, and that the section runs the tests of every such package the tasks' verifications name. Tests that no allowed command form runs and that a module check already runs, such as the hook tests, need no section.#### Scenario: A change with two delta specs- **WHEN** the full proposal review starts for a change with two delta specs@@ -46,7 +50,19 @@ The delta spec reviewer SHALL check that requirements are complete and testable,#### Scenario: The task reviewer's role- **WHEN** the hook starts the task reviewer-- **THEN** its prompt names the role and requires checking that a task with a test covers every scenario, every task has a verification, and the task order is feasible+- **THEN** its prompt names the role and requires checking that a task with a test covers every scenario, every task has a verification, the task order is feasible, and the "Change verification" section covers the tested packages++#### Scenario: Tests without a verification section+- **WHEN** the tasks add tests to a backend package and the change has no "Change verification" section+- **THEN** the task reviewer returns a `tasks` finding that asks for the section with that package++#### Scenario: A section that leaves out a package+- **WHEN** a task's verification runs the tests of two backend packages and the "Change verification" section runs only one of them+- **THEN** the task reviewer returns a `tasks` finding that names the missing package++#### Scenario: A change with only hook tests+- **WHEN** the tasks add tests only to the hook tests and the change has no "Change verification" section+- **THEN** the task reviewer returns no finding about the missing section### Requirement: The agent resolves findings itself, except cutting the featureA reviewer SHALL return findings, each with a category, a path and a description. The categories are `gap` (a requirement is incomplete or a case is missing), `mismatch` (artifacts contradict each other, the main specs, the code or the architecture rules), `tasks` (a scenario without a task with a test, a task without verification, an infeasible order), `simplify` (the same result for the user, noticeably simpler), `scope` (an option that cuts the feature) and `minor`. The hook SHALL block the turn when there is a `gap`, `mismatch`, `tasks` or `simplify` finding, and require the agent to resolve each one without asking the user. `scope` and `minor` findings MUST NOT block the turn. The agent MUST NOT apply a `scope` finding itself: cutting the task is the user's decision.@@ -92,3 +108,10 @@ When only files of changes that have every artifact and no checked task changed#### Scenario: An unfinished proposal- **WHEN** the agent wrote only the proposal and finishes the turn with a question to the user- **THEN** the full proposal review does not start, and the quick necessity check reviews the proposal++### Requirement: The full proposal review ends within the hook timeout+The full proposal review SHALL work within the same kind of time budget as the full implementation review: the budget ends before the hook timeout with a margin, validation runs inside it, and a reviewer starts only when the budget still holds its whole time limit. A reviewer that cannot start for that reason SHALL count as a reviewer that could not run. The hook MUST NOT be killed by the client timeout without a message.++#### Scenario: Too many reviewers for the budget+- **WHEN** a proposal has so many delta specs that a third wave of reviewers would not fit in the remaining budget+- **THEN** the reviewers of that wave do not start, the turn is blocked, and the agent gets the reason that the review time budget ran outdiff --git a/openspec/specs/review-hook-launch/spec.md b/openspec/specs/review-hook-launch/spec.mdnew file mode 100644index 00000000..9c01710d--- /dev/null+++ b/openspec/specs/review-hook-launch/spec.md@@ -0,0 +1,48 @@+# review-hook-launch Specification++## Purpose++Makes the review hooks start the same way in every environment and refuse to stay silent: a hook that cannot run stops the agent with a reason, so a review is never skipped without the user knowing.++## Requirements++### Requirement: Hooks run with the repository's pinned tools+Every review hook, in Claude Code and in Codex, SHALL start through one launcher that runs the hook with the Python and OpenSpec versions pinned in the repository's tool versions file. The launcher SHALL find the version manager even when the client was started outside a login shell, for example from an IDE. The launcher MUST NOT install a missing tool itself, because an installation would not fit the hook timeout.++#### Scenario: A client started from an IDE+- **WHEN** Claude Code was started from an IDE whose environment does not activate the version manager, and the agent finishes a turn+- **THEN** the hooks run with the pinned Python and OpenSpec versions++#### Scenario: A pinned version is not installed+- **WHEN** the version manager is present but the pinned OpenSpec version is not installed, and the agent finishes a turn+- **THEN** no installation starts, the agent stops, and the user sees that the hooks did not start, that `mise install` installs the pinned tools and that `just doctor` checks the environment++### Requirement: A hook that cannot run stops the agent+When the launcher cannot find the version manager, when the hook exits with an error, or when a tool a gate needs is missing, the hook SHALL stop the agent and show the user the reason and the command that checks the environment. The hook MUST NOT let the turn pass as if the review had run.++#### Scenario: OpenSpec is missing+- **WHEN** the OpenSpec CLI cannot be found and a turn finishes that wrote the artifacts of a change+- **THEN** the agent stops, and the user sees that the hooks could not find `openspec` and that `just doctor` checks the environment++#### Scenario: The version manager is missing+- **WHEN** the launcher cannot find the version manager+- **THEN** the agent stops, and the user sees that the version manager was not found and that `just doctor` checks the environment++#### Scenario: The hook crashes+- **WHEN** the hook process exits with a non-zero code+- **THEN** the agent stops, and the user sees which hook action failed, its exit code, and that `just doctor` checks the environment++### Requirement: The environment check reports every tool the hooks need+The repository SHALL have a command that reports, one line per tool, whether the hooks can find Python, git, OpenSpec and the CLI of at least one reviewer client, Claude Code or Codex. The command SHALL run through the same launcher as the hooks, and SHALL exit with a non-zero code when any of them is missing or the version manager cannot be found.++#### Scenario: Every tool is present+- **WHEN** the user runs `just doctor` with every tool installed+- **THEN** each tool is reported with its path and the command exits with code 0++#### Scenario: No reviewer CLI+- **WHEN** neither the Claude Code CLI nor the Codex CLI is installed and the user runs `just doctor`+- **THEN** the command reports that no reviewer CLI was found and exits with a non-zero code++#### Scenario: No version manager+- **WHEN** the version manager cannot be found and the user runs `just doctor`+- **THEN** the command reports that the version manager was not found and exits with a non-zero code