:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(hogli): add a verification ladder skill and hogli verify --changed (#122352)
Georges-Antoine Assi7745f1da27ac
diff --git a/.agents/skills/verifying-changes/SKILL.md b/.agents/skills/verifying-changes/SKILL.mdnew file mode 100644index 000000000000..019630be64ae--- /dev/null+++ b/.agents/skills/verifying-changes/SKILL.md@@ -0,0 +1,72 @@+---+name: verifying-changes+description: >+ Picks the cheapest check that is enough to confirm a change works, and says when a slower one is needed.+ Use before you report a task done, before you start Storybook, the dev stack or a+ Playwright run to check your own work, and when you are unsure how much verification a diff needs.+ Maps each kind of change, such as pure logic, DRF serializer or viewset, migration, kea logic, React component,+ CSS, Django template, CI workflow, SQL or HogQL, or docs and skills, to the layer that is enough and the layer that is wasted.+ `hogli verify --changed` runs the cheap layers for the current diff and names the slower ones left to run yourself.+ Trigger terms: verify, how do I test this, is this enough testing, start the stack, render it, run e2e.+---++# Verifying changes++Verification has six layers, and each one costs roughly ten times the one before it.+Start with the cheapest, and move up only when the current layer cannot see what the diff changes.++| Layer | Check | Command |+| ----- | ---------------------------- | ------------------------------------------------------------------------------------------------ |+| 1 | Lint, format, typecheck | `hogli verify --changed`, or `ruff check` and `pnpm --filter=@posthog/frontend typescript:check` |+| 2 | The nearest unit test | `hogli test <file>`, `hogli test --changed` |+| 3 | Serializer and OpenAPI regen | `hogli up -d`, `hogli build:openapi`, then commit the generated diff |+| 4 | Storybook render | `pnpm storybook`, open the affected story, compare before and after |+| 5 | The running stack | `hogli up -d`, `hogli wait`, then exercise the flow |+| 6 | Playwright or a recorded run | `hogli test playwright/<spec>.spec.ts` against a running stack |++`hogli verify --changed` classifies the diff against master, runs layers 1 and 2, and lists any higher layer you still have to run yourself.+It does not report the diff as verified when a check was skipped, when a check only printed advice, or when no test covers the changed files.+mypy is one of those advice-only checks. It runs repo-wide, so `hogli verify` prints its command instead of running it.+Use `--plan` to see the classification without running anything.++## Decision table++| Change | Enough | Wasted |+| -------------------------- | --------------------------------------------------- | ---------------------------------------- |+| Pure logic, Python or TS | 2: nearest unit test | 5: stack |+| DRF serializer or viewset | 3: OpenAPI regen, plus the endpoint test at layer 2 | 6: Playwright |+| Migration | 2: migration test, plus `hogli migrations:check` | 4: Storybook |+| Kea logic | 2: logic test, plus typecheck | 4: Storybook |+| React component | 4: Storybook render | 5: stack, unless the data wiring changed |+| CSS only | 4: Storybook render | 2: unit tests |+| CI workflow | 1: `hogli lint:workflows`, `hogli ci:plan` | 2: unit tests |+| SQL or HogQL | 2: the ClickHouse-backed query test | 5: stack |+| Docs or skills | 1: format, `hogli lint:skills` | 2 and above |+| Django template | 5: render the page in the running stack | 6: Playwright |+| Settings, URLs, middleware | 5: running stack | 6: Playwright |++A diff that mixes kinds needs the highest layer among them, and only for the files of that kind.++## Other rules still apply++- Reading the code does not confirm visual or UX work, as the root `AGENTS.md` says. A component or CSS change always needs at least a Storybook render.+- Layer 2 says where a test runs. Use `/writing-tests` to decide whether to add or change one. When a regression is hard to reach cheaply, extract the logic and test it at a lower layer.+- `hogli ci:preflight` stays the pre-push check, see `/running-ci-preflight`. `hogli verify` reuses its lint and format checks, so a pass here usually means a pass there too.++## When the stack is required++Start the stack only when the behavior spans more than one process and no layer 2 test runs both sides:++- Settings, URL routing, middleware, or the Content Security Policy, because only the browser applies them.+- A Django template, because only Django renders it.+- A change that a person has to see in the app: a new scene, a navigation change, or data flowing from ingestion to a UI.+- A Celery task, Temporal workflow, or plugin-server path whose wiring changed. A change to its logic alone stays at layer 2.+- A Playwright spec you changed, because it runs against the stack.++Everything else can skip the stack.+Django and ClickHouse-backed tests need the Postgres and ClickHouse services but not the web app, so `hogli up -d` plus a scoped `hogli test` still counts as layer 2.++## Report what you ran++Say which layer you stopped at and why the next one was not needed.+If you could not run a required layer, say so and name the command, so a reviewer knows what is unverified.diff --git a/AGENTS.md b/AGENTS.mdindex 166696bbcc59..a3a750ccae60 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -314,6 +314,7 @@ ALWAYS invoke the matching skill **first** — do not skip it, and do not attemp- `/writing-dataclasses` — adding or changing a Python dataclass, replacing a tuple or `dict[str, Any]` payload, or passing a dataclass or facade contract through internal layers- `/announcing-behavior-changes` — shipping a fix that changes what an existing user sees (a metric moves, a count drops, a range resolves differently), or adding, reviewing, or removing an in-app change notice+- `/verifying-changes` — choosing how to confirm a change works before reporting it done, or before starting Storybook, the dev stack, or a Playwright run to check your own work- `/merging-prs` — merging a PR, or babysitting one through the Trunk merge queue- `/stacking-prs` — creating, restacking, adopting, or landing a stack of PRs (`gh stack`)- `/implementing-mcp-tools` — adding/modifying endpoints or `tools.yaml`diff --git a/hogli.yaml b/hogli.yamlindex 83eb87dcdc19..981c9f0d0afc 100644--- a/hogli.yaml+++ b/hogli.yaml@@ -800,6 +800,9 @@ tests:test:click: hogli_commands.test_runner:test_commanddescription: Auto-detect test type (Python, Jest, Playwright, Rust, Go) and run it+ verify:+ click: hogli_commands.verify_layers:verify+ description: Run the cheapest checks that cover your diff (lint, typecheck, nearest tests), then name the next potential layertest:python:cmd: pytestdescription: Run Python test suitediff --git a/tools/hogli-commands/hogli_commands/test_runner.py b/tools/hogli-commands/hogli_commands/test_runner.pyindex 64371a27d4fb..248447104afb 100644--- a/tools/hogli-commands/hogli_commands/test_runner.py+++ b/tools/hogli-commands/hogli_commands/test_runner.py@@ -726,8 +726,19 @@ def _run_grouped(detected: list[tuple[str, TestRunConfig]], extra_args: list[str_run(cfg.command + extra_args, env=cfg.env_or_none, cwd=cfg.cwd)-def _run_changed(extra_args: list[str]) -> None:- """Find changed test files and tests for changed source files, then run them."""+@dataclass(frozen=True, kw_only=True, slots=True)+class ChangedTests:+ directly_changed: set[str]+ discovered: set[str]+ skipped_playwright_specs: set[str]++ @property+ def all_files(self) -> list[str]:+ return sorted(self.directly_changed | self.discovered)+++def changed_test_files(*, skip_playwright: bool = False) -> ChangedTests:+ """The changed test files, and the test files found for changed source files."""changed = _get_changed_files()directly_changed = {f for f in changed if _is_test_file(f)}@@ -737,7 +748,26 @@ def _run_changed(extra_args: list[str]) -> None:if not _is_test_file(f):discovered.update(_find_test_files_for_source(f))- all_test_files = sorted(directly_changed | discovered)+ playwright_specs: set[str] = set()+ if skip_playwright:+ playwright_specs = {f for f in directly_changed | discovered if f.startswith("playwright/")}+ directly_changed -= playwright_specs+ discovered -= playwright_specs++ return ChangedTests(+ directly_changed=directly_changed, discovered=discovered, skipped_playwright_specs=playwright_specs+ )+++def _run_changed(extra_args: list[str], skip_playwright: bool = False) -> None:+ """Find changed test files and tests for changed source files, then run them."""+ tests = changed_test_files(skip_playwright=skip_playwright)+ if tests.skipped_playwright_specs:+ click.secho(f"Skipping {len(tests.skipped_playwright_specs)} Playwright spec(s).", fg="yellow")+ directly_changed = tests.directly_changed+ discovered = tests.discovered++ all_test_files = tests.all_filesif not all_test_files:click.secho("No test files found for changes on this branch.", fg="yellow")@@ -828,16 +858,21 @@ def _run_watch(file_path: str, extra_args: list[str]) -> None:)@click.argument("file_path", required=False, type=click.Path())@click.option("--changed", is_flag=True, help="Run tests changed on this branch, plus tests for changed source files")+ "--skip-playwright",+ is_flag=True,+ help="With --changed, leave out Playwright specs, which need a running stack",+)@click.option("--watch", is_flag=True, help="Re-run tests on file changes (Python and Jest)")@click.pass_context-def test_command(ctx: click.Context, file_path: str | None, changed: bool, watch: bool) -> None:+def test_command(ctx: click.Context, file_path: str | None, changed: bool, skip_playwright: bool, watch: bool) -> None:"""Auto-detect test type and run the correct test runner."""if changed:if file_path:raise click.UsageError("Cannot combine --changed with a file path.")if watch:raise click.UsageError("Cannot combine --changed with --watch.")- _run_changed(list(ctx.args))+ _run_changed(list(ctx.args), skip_playwright=skip_playwright)returnif not file_path:diff --git a/tools/hogli-commands/hogli_commands/tests/test_test_runner.py b/tools/hogli-commands/hogli_commands/tests/test_test_runner.pyindex 8f4880b37879..642209f42fa6 100644--- a/tools/hogli-commands/hogli_commands/tests/test_test_runner.py+++ b/tools/hogli-commands/hogli_commands/tests/test_test_runner.py@@ -515,6 +515,17 @@ class TestRunChanged:command = mock_run.call_args[0][0]assert command.count("posthog/api/test/test_comments.py") == 1+ @patch("hogli_commands.test_runner._run")+ @patch("hogli_commands.test_runner._get_changed_files")+ def test_skip_playwright_leaves_out_specs(self, mock_changed, mock_run) -> None:+ mock_changed.return_value = ["playwright/e2e/dashboards.spec.ts", "posthog/api/test/test_user.py"]+ _run_changed([], skip_playwright=True)++ commands = [call[0][0] for call in mock_run.call_args_list]+ assert len(commands) == 1+ assert commands[0][0] == "pytest"+ assert "playwright/e2e/dashboards.spec.ts" not in commands[0]+@patch("hogli_commands.test_runner._get_changed_files")def test_no_changed_files_exits(self, mock_changed) -> None:mock_changed.return_value = ["posthog/api/views.py"]diff --git a/tools/hogli-commands/hogli_commands/tests/test_verify_layers.py b/tools/hogli-commands/hogli_commands/tests/test_verify_layers.pynew file mode 100644index 000000000000..f97c35e6ea6c--- /dev/null+++ b/tools/hogli-commands/hogli_commands/tests/test_verify_layers.py@@ -0,0 +1,97 @@+from __future__ import annotations++import pytest+from unittest.mock import patch++from click.testing import CliRunner+from hogli_commands.test_runner import ChangedTests+from hogli_commands.verify_layers import (+ ChangeKind,+ ComplexityLayer,+ _print_manual_layers,+ classify,+ plan_verification,+ verify,+)+++class TestClassify:+ @pytest.mark.parametrize(+ "path,expected_kind",+ [+ ("posthog/api/test/test_insight.py", ChangeKind.TEST),+ ("posthog/api/test/__snapshots__/test_insight.ambr", ChangeKind.TEST),+ ("frontend/src/scenes/insights/Insight.test.tsx", ChangeKind.TEST),+ ("playwright/e2e/dashboards.spec.ts", ChangeKind.END_TO_END_TEST),+ ("posthog/api/insight.py", ChangeKind.API_SCHEMA),+ ("products/surveys/backend/api/survey.py", ChangeKind.API_SCHEMA),+ ("posthog/migrations/0999_add_field.py", ChangeKind.MIGRATION),+ ("frontend/src/scenes/insights/insightLogic.tsx", ChangeKind.KEA_LOGIC),+ ("frontend/src/scenes/insights/Insight.tsx", ChangeKind.REACT_COMPONENT),+ ("frontend/src/scenes/insights/Insight.stories.tsx", ChangeKind.REACT_COMPONENT),+ ("frontend/src/styles/base.scss", ChangeKind.CSS),+ ("posthog/hogql_queries/insights/trends/trends_query_runner.py", ChangeKind.SQL_OR_HOGQL),+ (".github/workflows/ci-backend.yml", ChangeKind.CI_WORKFLOW),+ (".agents/skills/verifying-changes/SKILL.md", ChangeKind.DOCS_OR_SKILLS),+ ("products/surveys/skills/creating-surveys/references/guide.md", ChangeKind.DOCS_OR_SKILLS),+ ("posthog/settings/web.py", ChangeKind.APP_WIRING),+ ("products/surveys/backend/routes.py", ChangeKind.APP_WIRING),+ ("posthog/csp_middleware.py", ChangeKind.APP_WIRING),+ ("posthog/models/team/team.py", ChangeKind.PURE_LOGIC),+ ("docs/onboarding/web-analytics/react.tsx", ChangeKind.REACT_COMPONENT),+ (".agents/skills/verifying-changes/scripts/check.py", ChangeKind.PURE_LOGIC),+ (".github/actions/paths-filter/action.yml", ChangeKind.CI_WORKFLOW),+ (".github/actions/paths-filter/__tests__/filter.test.ts", ChangeKind.TEST),+ (".github/actions/paths-filter/src/filter.ts", ChangeKind.PURE_LOGIC),+ ("posthog/templates/authorize_and_link.html", ChangeKind.DJANGO_TEMPLATE),+ ],+ )+ def test_kind(self, path: str, expected_kind: ChangeKind) -> None:+ assert classify(path).kind == expected_kind+++class TestPlanVerification:+ @pytest.mark.parametrize(+ "paths,runs_unit_tests,manual_layers",+ [+ (["docs/internal/monorepo-layout.md", ".github/workflows/ci-backend.yml"], False, ()),+ (["posthog/models/team/team.py", "posthog/api/test/test_team.py"], True, ()),+ (+ ["frontend/src/scenes/insights/Insight.tsx", "posthog/api/insight.py", "posthog/settings/web.py"],+ True,+ (ComplexityLayer.OPENAPI, ComplexityLayer.STORYBOOK, ComplexityLayer.STACK),+ ),+ ],+ )+ def test_layers(self, paths: list[str], runs_unit_tests: bool, manual_layers: tuple[ComplexityLayer, ...]) -> None:+ plan = plan_verification(paths)+ assert plan.runs_unit_tests is runs_unit_tests+ assert plan.manual_layers == manual_layers++ @pytest.mark.parametrize(+ "outstanding,claims_enough",+ [((), True), (("[markdown-format] was skipped: needs node. Set up what it needs.",), False)],+ )+ def test_outstanding_check_does_not_claim_enough(+ self, capsys: pytest.CaptureFixture[str], outstanding: tuple[str, ...], claims_enough: bool+ ) -> None:+ _print_manual_layers(plan_verification(["docs/internal/monorepo-layout.md"]), outstanding)+ assert ("are enough for this diff" in capsys.readouterr().out) is claims_enough+++class TestVerifyCommand:+ def test_an_advisory_check_and_no_tests_leave_the_diff_unverified(self) -> None:+ no_tests = ChangedTests(directly_changed=set(), discovered=set(), skipped_playwright_specs=set())+ with (+ patch("hogli_commands.verify_layers.changed_files", return_value=["posthog/models/team/team.py"]),+ patch("hogli_commands.verify_layers._run_diff_check", return_value=("advisory", "run mypy")),+ patch("hogli_commands.verify_layers.changed_test_files", return_value=no_tests),+ patch("hogli_commands.verify_layers.subprocess.run") as run,+ ):+ result = CliRunner().invoke(verify, ["--changed"])++ assert result.exit_code == 0+ assert "[type-check] did not run: run mypy." in result.output+ assert "No tests were found for the changed files" in result.output+ assert "are enough for this diff" not in result.output+ run.assert_not_called()diff --git a/tools/hogli-commands/hogli_commands/verify_layers.py b/tools/hogli-commands/hogli_commands/verify_layers.pynew file mode 100644index 000000000000..de904e1676f6--- /dev/null+++ b/tools/hogli-commands/hogli_commands/verify_layers.py@@ -0,0 +1,278 @@+"""``hogli verify``: run the cheapest checks that are enough for the current diff.++The layers and the decision table come from ``.agents/skills/verifying-changes/SKILL.md``.+Keep the two in sync.+"""++from __future__ import annotations++import sys+import subprocess+import dataclasses+from collections.abc import Callable, Iterable+from dataclasses import dataclass+from enum import IntEnum, StrEnum++import click+from hogli.manifest import REPO_ROOT++from hogli_commands.build import TRIGGERS as BUILD_TRIGGERS+from hogli_commands.change_detection import changed_files, matches_globs+from hogli_commands.ci_preflight import (+ _COLOR,+ _ICON,+ DIFF_CHECKS,+ DiffCheck,+ Outcome,+ Scope,+ _capability_met,+ _run_diff_check,+)+from hogli_commands.test_runner import changed_test_files+++class ComplexityLayer(IntEnum):+ STATIC = 1+ UNIT_TEST = 2+ OPENAPI = 3+ STORYBOOK = 4+ STACK = 5+ END_TO_END = 6+++class ChangeKind(StrEnum):+ DOCS_OR_SKILLS = "docs or skills"+ CI_WORKFLOW = "CI workflow"+ MIGRATION = "migration"+ END_TO_END_TEST = "Playwright spec"+ TEST = "test"+ API_SCHEMA = "DRF serializer or viewset"+ SQL_OR_HOGQL = "SQL or HogQL"+ KEA_LOGIC = "kea logic"+ CSS = "CSS"+ REACT_COMPONENT = "React component"+ DJANGO_TEMPLATE = "Django template"+ APP_WIRING = "settings, URLs or middleware"+ PURE_LOGIC = "pure logic"+++@dataclass(frozen=True, kw_only=True, slots=True)+class KindRule:+ kind: ChangeKind+ globs: tuple[str, ...]+ sufficient: ComplexityLayer+++# First match wins, so a narrow kind sits above a broad one that shares its files.+KIND_RULES: tuple[KindRule, ...] = (+ KindRule(+ kind=ChangeKind.DOCS_OR_SKILLS,+ globs=("*.md", "*.mdx"),+ sufficient=ComplexityLayer.STATIC,+ ),+ KindRule(+ kind=ChangeKind.CI_WORKFLOW,+ globs=(".github/workflows/*", ".github/actions/*.yml", ".github/actions/*.yaml"),+ sufficient=ComplexityLayer.STATIC,+ ),+ KindRule(kind=ChangeKind.MIGRATION, globs=("*/migrations/*.py",), sufficient=ComplexityLayer.UNIT_TEST),+ KindRule(kind=ChangeKind.END_TO_END_TEST, globs=("playwright/*",), sufficient=ComplexityLayer.END_TO_END),+ KindRule(+ kind=ChangeKind.TEST,+ globs=("test_*.py", "*/test_*.py", "*.test.ts", "*.test.tsx", "*/test/*", "*/tests/*", "*/__snapshots__/*"),+ sufficient=ComplexityLayer.UNIT_TEST,+ ),+ KindRule(kind=ChangeKind.API_SCHEMA, globs=BUILD_TRIGGERS["build:openapi"], sufficient=ComplexityLayer.OPENAPI),+ KindRule(+ kind=ChangeKind.SQL_OR_HOGQL,+ globs=("*.sql", "posthog/hogql/*", "posthog/hogql_queries/*", "products/*/backend/hogql_queries/*"),+ sufficient=ComplexityLayer.UNIT_TEST,+ ),+ KindRule(kind=ChangeKind.KEA_LOGIC, globs=("*Logic.ts", "*Logic.tsx"), sufficient=ComplexityLayer.UNIT_TEST),+ KindRule(kind=ChangeKind.CSS, globs=("*.css", "*.scss"), sufficient=ComplexityLayer.STORYBOOK),+ KindRule(kind=ChangeKind.REACT_COMPONENT, globs=("*.tsx",), sufficient=ComplexityLayer.STORYBOOK),+ KindRule(kind=ChangeKind.DJANGO_TEMPLATE, globs=("*/templates/*.html",), sufficient=ComplexityLayer.STACK),+ KindRule(+ kind=ChangeKind.APP_WIRING,+ globs=(+ "posthog/settings/*",+ "posthog/urls.py",+ "products/*/backend/routes.py",+ "posthog/middleware.py",+ "posthog/csp_middleware.py",+ "docker-compose*.yml",+ ),+ sufficient=ComplexityLayer.STACK,+ ),+)++FALLBACK_RULE = KindRule(kind=ChangeKind.PURE_LOGIC, globs=("*",), sufficient=ComplexityLayer.UNIT_TEST)++COMPLEXITY_LAYERS: dict[ComplexityLayer, str] = {+ ComplexityLayer.OPENAPI: "Start the services with `hogli up -d`, run `hogli build:openapi`, and commit the generated types.",+ ComplexityLayer.STORYBOOK: "Render the affected stories with `pnpm storybook` and compare them before and after.",+ ComplexityLayer.STACK: "Start the stack with `hogli up -d` and `hogli wait`, then exercise the change in the app.",+ ComplexityLayer.END_TO_END: "Run the changed Playwright specs with `hogli test <spec>` against a running stack.",+}++STATIC_CHECK_KEYS = (+ "ruff-lint",+ "ruff-format",+ "type-check",+ "frontend-format",+ "markdown-format",+ "desktop-biome",+ "agent-biome",+ "workflow-lint",+ "migrations",+)+++def _streamed(command: list[str]) -> Callable[[Scope], Outcome]:+ # Preflight keeps three lines of a failure, and the pnpm banner fills them, so the compiler+ # errors and the typecheck wrapper's recovery steps print straight to the terminal instead.+ def run(_scope: Scope) -> Outcome:+ if not _capability_met("node"):+ return "skipped", "needs node"+ returncode = subprocess.run(command, cwd=REPO_ROOT).returncode+ return ("pass", "ok") if returncode == 0 else ("fail", f"exit {returncode}, see the output above")++ return run+++FRONTEND_TS = ("frontend/src/*.ts", "frontend/src/*.tsx", "products/*/frontend/*.ts", "products/*/frontend/*.tsx")++VERIFY_ONLY_CHECKS: tuple[DiffCheck, ...] = (+ DiffCheck(+ key="oxlint",+ label="frontend lint (oxlint)",+ triggers=list(FRONTEND_TS),+ verify=["pnpm", "exec", "oxlint", "--quiet"],+ requires=("node",),+ takes_files=True,+ ),+ DiffCheck(+ key="typescript",+ label="frontend typecheck (tsgo)",+ triggers=list(FRONTEND_TS),+ verify=None,+ run=_streamed(["pnpm", "--filter=@posthog/frontend", "typescript:check"]),+ ),+ DiffCheck(+ key="skills-lint",+ label="agent skill sources",+ triggers=[".agents/skills/*", "products/*/skills/*"],+ verify=["hogli", "lint:skills"],+ requires=("python-env",),+ ),+)+++@dataclass(frozen=True, kw_only=True, slots=True)+class VerifyPlan:+ files_by_kind: dict[ChangeKind, list[str]]+ runs_unit_tests: bool+ manual_layers: tuple[ComplexityLayer, ...]+++def classify(path: str) -> KindRule:+ return next((rule for rule in KIND_RULES if matches_globs(path, rule.globs)), FALLBACK_RULE)+++def plan_verification(paths: Iterable[str]) -> VerifyPlan:+ files_by_kind: dict[ChangeKind, list[str]] = {}+ sufficient: set[ComplexityLayer] = set()+ for path in paths:+ rule = classify(path)+ files_by_kind.setdefault(rule.kind, []).append(path)+ sufficient.add(rule.sufficient)+ return VerifyPlan(+ files_by_kind=files_by_kind,+ runs_unit_tests=any(layer >= ComplexityLayer.UNIT_TEST for layer in sufficient),+ manual_layers=tuple(sorted(layer for layer in sufficient if layer in COMPLEXITY_LAYERS)),+ )+++def _static_checks(files: list[str]) -> list[DiffCheck]:+ preflight_checks = [check for check in DIFF_CHECKS if check.key in STATIC_CHECK_KEYS]+ triggered = []+ for check in [*preflight_checks, *VERIFY_ONLY_CHECKS]:+ matched = [path for path in files if matches_globs(path, check.triggers)]+ if matched:+ triggered.append(dataclasses.replace(check, matched=matched))+ return triggered+++def _print_plan(plan: VerifyPlan) -> None:+ for kind, paths in plan.files_by_kind.items():+ click.echo(f" {kind}: {len(paths)} file(s)")+ click.echo()+++def _print_manual_layers(plan: VerifyPlan, outstanding: tuple[str, ...] = ()) -> None:+ covered = "Lint, format, typecheck and the nearest tests" if plan.runs_unit_tests else "Lint, format and typecheck"+ if outstanding:+ click.secho(" This diff is not verified yet. Do these, then run `hogli verify --changed` again:", fg="yellow")+ for step in outstanding:+ click.echo(f" - {step}")+ if not plan.manual_layers:+ if not outstanding:+ click.secho(+ f" {covered} are enough for this diff. Slower checks would not find anything more.", fg="green"+ )+ return+ click.secho(f" {covered} are not enough for this diff. Run these yourself, cheapest first:", bold=True)+ for layer in plan.manual_layers:+ click.echo(f" {layer.value}. {COMPLEXITY_LAYERS[layer]}")+++ name="verify",+ help="Run the cheapest checks that cover your diff (lint, typecheck, nearest tests), then name the next potential layer",+)[email protected]("--changed", is_flag=True, help="Verify the files changed on this branch, plus uncommitted work.")[email protected]("--plan", "plan_only", is_flag=True, help="Show how the diff is classified without running checks.")+def verify(changed: bool, plan_only: bool) -> None:+ if not changed:+ raise click.UsageError("Pass --changed to verify the files changed on this branch.")+ files = changed_files()+ if not files:+ click.secho("No changed files on this branch.", fg="yellow")+ return+ plan = plan_verification(files)+ click.secho(f"\n hogli verify: {len(files)} changed file(s)\n", bold=True)+ _print_plan(plan)+ if plan_only:+ _print_manual_layers(plan)+ return++ click.secho(" Layer 1: lint, format, typecheck", bold=True)+ failures = 0+ outstanding: list[str] = []+ for check in _static_checks(files):+ status, detail = _run_diff_check(check, do_fix=False, against=None, strict=False, changed=files)+ failures += status == "fail"+ if status == "skipped":+ outstanding.append(f"[{check.key}] was skipped: {detail}. Set up what it needs.")+ elif status == "advisory":+ outstanding.append(f"[{check.key}] did not run: {detail}.")+ click.secho(f" {_ICON[status]} [{check.key}] {check.label}", fg=_COLOR[status])+ click.echo(f" {detail}")+ click.echo()+ if failures:+ click.secho(" Fix the failures above, then run `hogli verify --changed` again.", fg="red")+ sys.exit(1)++ if plan.runs_unit_tests and not changed_test_files(skip_playwright=True).all_files:+ outstanding.append(+ "No tests were found for the changed files. Pick the nearest test and run `hogli test <file>`."+ )+ elif plan.runs_unit_tests:+ click.secho(" Layer 2: hogli test --changed --skip-playwright", bold=True)+ test_command = [sys.executable, "-m", "hogli", "test", "--changed", "--skip-playwright"]+ if subprocess.run(test_command, cwd=REPO_ROOT).returncode != 0:+ click.secho("\n Tests failed. Fix them before you run slower checks.", fg="red")+ sys.exit(1)+ click.echo()++ _print_manual_layers(plan, tuple(outstanding))