Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
feat(ci): write doc-drift and security findings as a GitLab Code Quality report (#2643)
Raghav Chamadiyae1bfb32758e4
diff --git a/ci/gitlab/repowise.gitlab-ci.yml b/ci/gitlab/repowise.gitlab-ci.ymlindex fcf1f8426..ae59465c8 100644--- a/ci/gitlab/repowise.gitlab-ci.yml+++ b/ci/gitlab/repowise.gitlab-ci.yml@@ -6,8 +6,11 @@# - remote: https://raw.githubusercontent.com/repowise-dev/repowise/main/ci/gitlab/repowise.gitlab-ci.yml## Each gate is its own job, so one failing does not hide the others. The-# markdown report is printed to the job log and kept as an artifact. Exit codes-# are the gates': 1 failed, 2 could not evaluate; both fail the job.+# markdown report is printed to the job log and kept as an artifact. Doc drift+# and security also write a Code Quality report, so their findings show in the+# merge request's Code Quality widget; repowise-code-quality publishes the+# default branch's report the widget compares against. Exit codes are the+# gates': 1 failed, 2 could not evaluate; both fail the job.## Variables (set them in your .gitlab-ci.yml or the project's CI settings):# REPOWISE_VERSION Version to install; empty installs the latest.@@ -49,26 +52,73 @@ repowise-coverage:repowise-doc-drift:extends: .repowisescript:- - >- repowise doc-drift --check- ${REPOWISE_DOC_DRIFT_BASELINE:+--baseline "$REPOWISE_DOC_DRIFT_BASELINE"}- --format markdown > repowise-doc-drift.md+ # The gate runs once per format (one run writing both would halve the cost).+ # The Code Quality run goes even when the gate fails, and the job exits with+ # the gate's code. Its output is kept only when it parses as JSON, so a usage+ # error or a crash leaves `[]` rather than an invalid artifact.+ - |+ status=0+ repowise doc-drift --check \+ ${REPOWISE_DOC_DRIFT_BASELINE:+--baseline "$REPOWISE_DOC_DRIFT_BASELINE"} \+ --format markdown > repowise-doc-drift.md || status=$?+ repowise doc-drift --check \+ ${REPOWISE_DOC_DRIFT_BASELINE:+--baseline "$REPOWISE_DOC_DRIFT_BASELINE"} \+ --format gitlab > gl.tmp || true+ python -m json.tool gl.tmp > /dev/null && mv gl.tmp gl-code-quality-doc-drift.json \+ || echo '[]' > gl-code-quality-doc-drift.json+ exit "$status"after_script:- cat repowise-doc-drift.mdartifacts:when: alwayspaths: [repowise-doc-drift.md]+ reports:+ codequality: gl-code-quality-doc-drift.jsonrepowise-security:extends: .repowisescript:- - >- repowise security check "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME...HEAD"- --fail-on "${REPOWISE_SECURITY_FAIL_ON:-high}"- ${REPOWISE_SECURITY_BASELINE:+--baseline "$REPOWISE_SECURITY_BASELINE"}- --format markdown > repowise-security.md+ # Two runs and a guarded report, as in repowise-doc-drift.+ - |+ status=0+ repowise security check "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME...HEAD" \+ --fail-on "${REPOWISE_SECURITY_FAIL_ON:-high}" \+ ${REPOWISE_SECURITY_BASELINE:+--baseline "$REPOWISE_SECURITY_BASELINE"} \+ --format markdown > repowise-security.md || status=$?+ repowise security check "origin/$CI_MERGE_REQUEST_TARGET_BRANCH_NAME...HEAD" \+ --fail-on "${REPOWISE_SECURITY_FAIL_ON:-high}" \+ ${REPOWISE_SECURITY_BASELINE:+--baseline "$REPOWISE_SECURITY_BASELINE"} \+ --format gitlab > gl.tmp || true+ python -m json.tool gl.tmp > /dev/null && mv gl.tmp gl-code-quality-security.json \+ || echo '[]' > gl-code-quality-security.json+ exit "$status"after_script:- cat repowise-security.mdartifacts:when: alwayspaths: [repowise-security.md]+ reports:+ codequality: gl-code-quality-security.json++# The Code Quality widget compares a merge request's report with the one from+# the target branch's latest pipeline, so this job publishes that one on the+# default branch. It never fails the pipeline. Only doc drift has a base report:+# security judges only what a change adds, and the default branch has no change+# to judge, so every security issue a merge request raises shows as new.+repowise-code-quality:+ image: python:3.12+ allow_failure: true+ script:+ - |+ python -m pip install --quiet "repowise${REPOWISE_VERSION:+==$REPOWISE_VERSION}"+ repowise doc-drift --check \+ ${REPOWISE_DOC_DRIFT_BASELINE:+--baseline "$REPOWISE_DOC_DRIFT_BASELINE"} \+ --format gitlab > gl.tmp || true+ python -m json.tool gl.tmp > /dev/null && mv gl.tmp gl-code-quality-doc-drift.json \+ || echo '[]' > gl-code-quality-doc-drift.json+ artifacts:+ when: always+ reports:+ codequality: gl-code-quality-doc-drift.json+ rules:+ - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCHdiff --git a/docs/layers/DOC_DRIFT.md b/docs/layers/DOC_DRIFT.mdindex e54808ca4..bef6a0d5b 100644--- a/docs/layers/DOC_DRIFT.md+++ b/docs/layers/DOC_DRIFT.md@@ -98,7 +98,10 @@ repowise doc-drift --check --format github`--format github` annotates each finding on the document line and writes asummary to the job page. `--format sarif` produces a file for GitHub code-scanning; `--format markdown` is for posting a comment yourself. Full history+scanning, `--format gitlab` a GitLab Code Quality report for the merge request+widget (baselined findings left out), and `--format markdown` is for posting a+comment yourself. In the Code Quality report a finding at or above+`--fail-on-confidence` is `major` and one below it `minor`. Full history(`fetch-depth: 0`) is optional and only improves the rename suggestions. TheGitHub Action and GitLab template that run it beside the other gates are in[Repowise in CI](../start/CI.md).diff --git a/docs/layers/SECURITY.md b/docs/layers/SECURITY.mdindex fc5b5a652..36807d7bf 100644--- a/docs/layers/SECURITY.md+++ b/docs/layers/SECURITY.md@@ -258,7 +258,8 @@ clone, an unreadable baseline). A git error is never read as a clean change.Every format carries the masked snippet only, because CI logs are oftenpublic: `table`, `json`, `markdown`, `github` (annotations plus the job-summary) and `sarif` (for code-scanning upload). The gate is the registry+summary), `sarif` (for code-scanning upload) and `gitlab` (a GitLab Code+Quality report for the merge request widget). The gate is the registryabove and has its limits: a pass means no pattern matched a changed line.**Baseline.** `--write-baseline FILE` records the change's findings in a@@ -274,7 +275,10 @@ different value sharing the first four characters on an identical line staysaccepted. Because the gate sees one change at a time, writing to an existingbaseline adds to its entries (and to those of `--baseline`, when given)instead of replacing them; remove an entry by deleting it from the file.-`--format sarif` marks accepted findings as suppressed.+`--format sarif` marks accepted findings as suppressed; `--format gitlab`+leaves them out, because that format has no suppression field. In the Code+Quality report a finding at or above `--fail-on` is `critical` when high and+`major` otherwise, and one below it `minor`.Without a REVSPEC the base comes from the CI's pull-request variables, elsethe remote's default branch. A shallow checkout cannot be read commit bydiff --git a/docs/reference/CLI_REFERENCE.md b/docs/reference/CLI_REFERENCE.mdindex 3b6c4c00f..4b9fcc4dd 100644--- a/docs/reference/CLI_REFERENCE.md+++ b/docs/reference/CLI_REFERENCE.md@@ -764,7 +764,7 @@ with a close match. It is a suggestion to check, never applied.| `--baseline` | With `--check`, accept the findings recorded in this file; only new ones fail || `--write-baseline` | With `--check`, record the current findings to this file and exit 0 || `--since` | With `--check`, gate only drift this change is answerable for: documents it edits, documents naming files it deletes or renames, anchors into documents it edits, and commands whose manifest it edits. A bare ref means `REF...HEAD` plus uncommitted changes; `auto` reads the target branch from CI |-| `--format` | Output: `table` (default), `json`, `markdown`, `github`, `sarif` |+| `--format` | Output: `table` (default), `json`, `markdown`, `github`, `sarif`, `gitlab` (GitLab Code Quality report) || `--repo` | In workspace mode, target a specific repo (defaults to primary) || `--no-workspace` | Force single-repo mode |@@ -777,6 +777,7 @@ repowise doc-drift --format jsonrepowise doc-drift --check # CI gate, no index neededrepowise doc-drift --check --format github # annotations + step summaryrepowise doc-drift --check --format sarif > drift.sarif # code scanning upload+repowise doc-drift --check --format gitlab > gl-code-quality-doc-drift.json # merge request widgetrepowise doc-drift --check --write-baseline .doc-drift-baseline.jsonrepowise doc-drift --check --baseline .doc-drift-baseline.jsonrepowise doc-drift --check --since auto # only drift this PR is answerable for@@ -959,12 +960,13 @@ default branch.| `--baseline` | Accept the findings recorded in this file; only new ones fail || `--write-baseline` | Add this change's findings to this file, keeping its entries and those of `--baseline`, and exit 0 || `--path` | A path inside the repository (defaults to cwd) |-| `--format` | `table` (default), `json`, `markdown`, `github`, `sarif` |+| `--format` | `table` (default), `json`, `markdown`, `github`, `sarif`, `gitlab` (GitLab Code Quality report) |```bashrepowise security check origin/main...HEADrepowise security check --format github --baseline .security-baseline.jsonrepowise security check --format sarif > security.sarif+repowise security check --format gitlab > gl-code-quality-security.jsonrepowise security check --write-baseline .security-baseline.json```diff --git a/docs/start/CI.md b/docs/start/CI.mdindex 3919440df..c50721ba1 100644--- a/docs/start/CI.md+++ b/docs/start/CI.md@@ -27,7 +27,11 @@ A setup problem never reads as a pass.script, `markdown` for a comment or a job artifact, and `github` for GitHubActions: up to ten annotations on the changed lines, a notice counting therest, and the markdown report appended to the job summary. Doc drift and-security also write `sarif` for code scanning.+security also write `sarif` for code scanning and `gitlab`, a GitLab Code+Quality report (a JSON issue list; findings a baseline accepted are left out).+When a gate cannot evaluate, `gitlab` still prints a valid list, `[]`, and the+GitLab template writes `[]` whenever the output does not parse, so the report+artifact is never an invalid file.**The change being judged.** Coverage and security take a revision range:`origin/main...HEAD` (three dots: what the branch did since it forked, the@@ -155,10 +159,34 @@ repowise-coverage:It adds one job per gate on merge request pipelines: `repowise-coverage` (onlywhen `REPOWISE_COVERAGE_REPORT` is set), `repowise-doc-drift` and`repowise-security`. Each prints its markdown report to the log and keeps it as-an artifact. Other variables: `REPOWISE_VERSION`, `REPOWISE_DOC_DRIFT_BASELINE`,+an artifact. The doc drift and security jobs also write a Code Quality report+(`gl-code-quality-doc-drift.json`, `gl-code-quality-security.json`), so their+findings show in the merge request's Code Quality widget, even when the gate+fails the job. A fourth job, `repowise-code-quality`, runs on the default+branch and never fails the pipeline: it publishes the report the widget+compares a merge request against, so only issues the merge request introduces+show as new. It publishes doc drift only: security judges only what a change+adds, and the default branch has no change to judge. Other+variables: `REPOWISE_VERSION`, `REPOWISE_DOC_DRIFT_BASELINE`,`REPOWISE_SECURITY_BASELINE`, `REPOWISE_SECURITY_FAIL_ON`. Override any job's`image`, `rules` or `needs` in your own file as usual.+GitLab can also draw line coverage in the merge request diff. That comes from+your own test job, not from Repowise: have your test runner write a Cobertura+report (for example with pytest-cov) and declare it there. The same+`coverage.xml` can be your `REPOWISE_COVERAGE_REPORT`.++```yaml+test:+ script:+ - pytest --cov --cov-report=xml:coverage.xml+ artifacts:+ reports:+ coverage_report:+ coverage_format: cobertura+ path: coverage.xml+```+## Other CI systemsThe gates are plain commands. Fetch the target branch, run them, and let thediff --git a/packages/cli/src/repowise/cli/ci.py b/packages/cli/src/repowise/cli/ci.pyindex 176f6cf6e..7a1ea10c1 100644--- a/packages/cli/src/repowise/cli/ci.py+++ b/packages/cli/src/repowise/cli/ci.py@@ -21,7 +21,8 @@ EXIT_GATE_FAILED = 1#: The gate could not run: no report, unknown revision, bad config, ...EXIT_CANNOT_EVALUATE = 2-#: Output formats every gating command offers; a command may add more (SARIF).+#: Output formats every gating command offers; a command may add more (SARIF,+#: GitLab Code Quality).CI_FORMATS = ("table", "json", "markdown", "github")#: Why a diff against the base failed in CI, nearly always.@@ -42,7 +43,8 @@ def cannot_evaluate(fmt: str, code: str, message: str) -> NoReturn:"""Report why the gate could not run and exit :data:`EXIT_CANNOT_EVALUATE`.``json`` gets a ``{"error", "message"}`` document on stdout, ``github`` an- ``::error::`` line; the message always reaches stderr for a human.+ ``::error::`` line, ``gitlab`` an empty issue list so the report artifact+ stays valid JSON; the message always reaches stderr for a human."""from rich.markup import escape@@ -54,6 +56,8 @@ def cannot_evaluate(fmt: str, code: str, message: str) -> NoReturn:emit_json({"error": code, "message": message})elif fmt == "github":click.echo(error(message))+ elif fmt == "gitlab":+ click.echo("[]")if fmt != "json":err_console.print(f"[red]{escape(message)}[/red]")raise click.exceptions.Exit(EXIT_CANNOT_EVALUATE)diff --git a/packages/cli/src/repowise/cli/commands/doc_drift_cmd.py b/packages/cli/src/repowise/cli/commands/doc_drift_cmd.pyindex 669f3c2f7..ab93a32b5 100644--- a/packages/cli/src/repowise/cli/commands/doc_drift_cmd.py+++ b/packages/cli/src/repowise/cli/commands/doc_drift_cmd.py@@ -40,7 +40,7 @@ if TYPE_CHECKING:_NO_INDEX = object()_STALE_INDEX = object()-_FORMATS = (*CI_FORMATS, "sarif")+_FORMATS = (*CI_FORMATS, "sarif", "gitlab")def _repo_path(path: str | None, repo_alias: str | None, no_workspace: bool, fmt: str) -> Path:@@ -189,6 +189,9 @@ def _emit(findings, tool_version=__version__, fail_on=fail_on, accepted=accepted))+ elif fmt == "gitlab":+ fail_on = gate.fail_on if gate is not None else HIGH_CONFIDENCE_THRESHOLD+ emit_json(render.render_gitlab(findings, fail_on=fail_on, accepted=accepted))elif fmt in ("markdown", "github"):markdown = render.render_markdown(findings,@@ -222,9 +225,14 @@ def _print_verdict(gate: GateResult) -> None:def _refuse(code: str, message: str, fmt: str, *, remedy: str, repo: str) -> NoReturn:- """Refuse and exit 1; machine formats other than json keep stdout empty."""+ """Refuse and exit 1; machine formats other than json and gitlab keep stdout empty.++ ``gitlab`` prints an empty issue list, so the report artifact stays valid.+ """if fmt in ("json", "table"):emit_refusal(code, message, fmt, remedy=remedy, repo=repo)+ elif fmt == "gitlab":+ click.echo("[]")from rich.markup import escapenotices = ci_notices(fmt)@@ -393,7 +401,10 @@ def _run_check(@click.option("--no-workspace", is_flag=True, default=False, help="Force single-repo mode.")@format_option(choices=_FORMATS,- help="Output format. 'github' prints annotations and fills the job summary.",+ help=(+ "Output format. 'github' prints annotations and fills the job summary; "+ "'sarif' is for code-scanning upload; 'gitlab' is a GitLab Code Quality report."+ ),)def doc_drift_command(path: str | None,diff --git a/packages/cli/src/repowise/cli/commands/security_cmd.py b/packages/cli/src/repowise/cli/commands/security_cmd.pyindex 56f333059..0d4a7f63b 100644--- a/packages/cli/src/repowise/cli/commands/security_cmd.py+++ b/packages/cli/src/repowise/cli/commands/security_cmd.py@@ -43,7 +43,7 @@ from repowise.cli.output import emit_json, format_option, notice_consoleif TYPE_CHECKING:from repowise.core.analysis.security_gate import ChangeScan, GateResult-_CHECK_FORMATS = (*CI_FORMATS, "sarif")+_CHECK_FORMATS = (*CI_FORMATS, "sarif", "gitlab")@click.group("security")@@ -246,7 +246,7 @@ def security_scan(choices=_CHECK_FORMATS,help="Output format. ``github`` writes annotations to stdout and the ""markdown summary to $GITHUB_STEP_SUMMARY when set; ``sarif`` is for "- "code-scanning upload.",+ "code-scanning upload; ``gitlab`` is a GitLab Code Quality report.",)def security_check(revspec: str | None,@@ -272,6 +272,7 @@ def security_check(repowise security check origin/main...HEADrepowise security check --format github --baseline .security-baseline.jsonrepowise security check --format sarif > security.sarif+ repowise security check --format gitlab > gl-code-quality-security.json"""from repowise.core.analysis.security_gate import evaluate@@ -382,6 +383,10 @@ def _emit_check(accepted=accepted,))+ elif fmt == "gitlab":+ emit_json(+ security_gate.render_gitlab(scan.findings, fail_on=gate.fail_on, accepted=accepted)+ )elif fmt in ("markdown", "github"):markdown = security_gate.render_markdown(gate,diff --git a/packages/core/src/repowise/core/analysis/doc_drift/render.py b/packages/core/src/repowise/core/analysis/doc_drift/render.pyindex 2b2bfdfed..0d866e91e 100644--- a/packages/core/src/repowise/core/analysis/doc_drift/render.py+++ b/packages/core/src/repowise/core/analysis/doc_drift/render.py@@ -1,4 +1,5 @@-"""CI renderings of documentation drift: markdown, GitHub annotations, SARIF.+"""CI renderings of documentation drift: markdown, GitHub annotations, SARIF,+GitLab Code Quality.Pure functions over finding dicts and a :class:`~.gate.GateResult`, so the CLI,the hosted platform and the PR bot print the same thing. Two honesty rules hold@@ -15,7 +16,7 @@ from __future__ import annotationsfrom collections.abc import Mapping, Sequencefrom typing import Any-from repowise.core.ci import github, sarif+from repowise.core.ci import github, gitlab, sariffrom repowise.core.ci.markdown import ROW_LIMIT, cell, more_line, pluralfrom .constants import DETECTION_BASIS, HIGH_CONFIDENCE_THRESHOLD@@ -250,3 +251,36 @@ def render_sarif())return sarif.run(SARIF_TOOL_NAME, tool_version, _sarif_rules(), results)+++# ---------------------------------------------------------------------------+# GitLab Code Quality+# ---------------------------------------------------------------------------+++def render_gitlab(+ findings: Sequence[Mapping[str, Any]],+ *,+ fail_on: float = HIGH_CONFIDENCE_THRESHOLD,+ accepted: frozenset[str] = frozenset(),+) -> list[dict]:+ """GitLab Code Quality issues: ``major`` at or above *fail_on*, ``minor`` below.++ The same line :func:`render_sarif` draws between ``error`` and ``warning``.+ *accepted* (the baseline) is applied by :func:`~repowise.core.ci.gitlab.report`.+ """+ return gitlab.report(+ SARIF_TOOL_NAME,+ (+ gitlab.issue(+ str(f["kind"]),+ "major" if float(f["confidence"]) >= fail_on else "minor",+ _message(f),+ str(f["file_path"]),+ int(f["line_number"]),+ fingerprint_of(f),+ )+ for f in _order(findings)+ ),+ accepted=accepted,+ )diff --git a/packages/core/src/repowise/core/analysis/security_gate.py b/packages/core/src/repowise/core/analysis/security_gate.pyindex d2a56fbdf..7b4c47091 100644--- a/packages/core/src/repowise/core/analysis/security_gate.py+++ b/packages/core/src/repowise/core/analysis/security_gate.py@@ -26,7 +26,7 @@ from pathlib import Path, PurePosixPathfrom typing import Anyfrom repowise.core.ci import baseline as ci_baseline-from repowise.core.ci import github, sarif+from repowise.core.ci import github, gitlab, sariffrom repowise.core.ci.markdown import ROW_LIMIT, cell, more_line, pluralfrom repowise.core.support_paths import DOC_EXTENSIONS@@ -554,3 +554,40 @@ def render_sarif())return sarif.run(SARIF_TOOL_NAME, tool_version, _sarif_rules(), results)+++def _gitlab_severity(severity: str, fail_on: str) -> str:+ if _RANK[severity] < _RANK[fail_on]:+ return "minor"+ return "critical" if severity == "high" else "major"+++def render_gitlab(+ findings: Sequence[Mapping[str, Any]],+ *,+ fail_on: str = "high",+ accepted: frozenset[str] = frozenset(),+) -> list[dict]:+ """GitLab Code Quality issues whose severity follows the gate, as SARIF's level does.++ At or above *fail_on*: ``critical`` for high, ``major`` otherwise; below+ it, ``minor``. *accepted* (the baseline) is applied by+ :func:`~repowise.core.ci.gitlab.report`. A secret found only in an earlier+ commit sits on line 1 of its file, since its line belongs to that commit;+ when the change deleted that file, the path is absent at the head.+ """+ return gitlab.report(+ SARIF_TOOL_NAME,+ (+ gitlab.issue(+ str(f["kind"]),+ _gitlab_severity(f["severity"], fail_on),+ _message(f),+ str(f["file_path"]),+ None if f["commit"] else int(f["line_number"]),+ f["fingerprint"],+ )+ for f in _order(findings)+ ),+ accepted=accepted,+ )diff --git a/packages/core/src/repowise/core/ci/__init__.py b/packages/core/src/repowise/core/ci/__init__.pyindex e6f65ee0c..b5431d512 100644--- a/packages/core/src/repowise/core/ci/__init__.py+++ b/packages/core/src/repowise/core/ci/__init__.py@@ -1,4 +1,5 @@-"""Shared CI plumbing: workflow commands, markdown pieces, the target branch.+"""Shared CI plumbing: workflow commands, markdown pieces, the target branch,+SARIF and GitLab Code Quality reports, baselines.Every CI-facing feature (patch coverage, doc drift, ...) renders its owncontent and uses these for the parts CI systems define, so escaping, caps anddiff --git a/packages/core/src/repowise/core/ci/gitlab.py b/packages/core/src/repowise/core/ci/gitlab.pynew file mode 100644index 000000000..63cf53cb3--- /dev/null+++ b/packages/core/src/repowise/core/ci/gitlab.py@@ -0,0 +1,70 @@+"""GitLab Code Quality reports: the JSON issue list a merge request's widget reads.++Features supply check names, severities and messages; this module fixes the+shape, names the tool and applies the baseline.+"""++from __future__ import annotations++from collections.abc import Iterable, Mapping+from typing import Any++#: The severities the format defines, least to most severe. Checked because+#: GitLab drops a report holding an unknown severity without saying so.+SEVERITIES = ("info", "minor", "major", "critical", "blocker")+++def issue(+ check_name: str,+ severity: str,+ description: str,+ path: str,+ line: int | None,+ fingerprint: str,+) -> dict:+ """One issue on a repo-relative *path*; *line* ``None`` places it on line 1."""+ if severity not in SEVERITIES:+ raise ValueError(f"unknown Code Quality severity {severity!r}")+ return {+ "description": description,+ "check_name": check_name,+ "fingerprint": fingerprint,+ "severity": severity,+ "location": {+ "path": path.replace("\\", "/").lstrip("/"),+ "lines": {"begin": max(1, int(line or 1))},+ },+ }+++def report(+ tool: str,+ issues: Iterable[Mapping[str, Any]],+ *,+ accepted: frozenset[str] = frozenset(),+) -> list[dict]:+ """*issues* in order as ``<tool>/<check>``, each fingerprint made unique.++ An issue whose fingerprint is in *accepted* (a baseline) is left out: the+ format has no suppression field, so the widget would list it as open.++ GitLab matches issues across pipelines by fingerprint, so two sharing one+ would read as a single issue. A repeat becomes ``<fingerprint>:<n>``, *n*+ counting from 2 in the order given, which is stable when the caller sorts.+ """+ seen: set[str] = set()+ out = []+ for item in issues:+ if item["fingerprint"] in accepted:+ continue+ item = dict(item)+ item["check_name"] = f"{tool}/{item['check_name']}"+ base = fp = item["fingerprint"]+ n = 1+ while fp in seen:+ n += 1+ fp = f"{base}:{n}"+ seen.add(fp)+ item["fingerprint"] = fp+ out.append(item)+ return outdiff --git a/tests/unit/analysis/test_security_gate.py b/tests/unit/analysis/test_security_gate.pyindex d766605c7..e36c47670 100644--- a/tests/unit/analysis/test_security_gate.py+++ b/tests/unit/analysis/test_security_gate.py@@ -18,6 +18,7 @@ from repowise.core.analysis.security_gate import (evaluate,fingerprint_of,github_annotations,+ render_gitlab,render_markdown,render_sarif,scan_change,@@ -272,6 +273,7 @@ def test_no_raw_secret_in_any_format(repo, tmp_path):render_markdown(gate, label="main...HEAD", files_scanned=2, commits_scanned=3),"\n".join(github_annotations(scan.findings, gate)),json.dumps(render_sarif(scan.findings, tool_version="0")),+ json.dumps(render_gitlab(scan.findings)),baseline.read_text(encoding="utf-8"),]for raw in (KEY, "hunter2hunter2", PEM_BODY[:20]):@@ -475,3 +477,53 @@ def test_diff_noprefix_config_does_not_misread_paths(repo):_write(repo, "b/cfg.py", "password = 'Zq8vLm2pXr7w'\n")_commit(repo, "nested")assert [f["file_path"] for f in _scan(repo).findings] == ["b/cfg.py"]+++def test_gitlab_maps_severities_drops_the_baseline_and_puts_history_on_line_one():+ accepted = _row(snippet="eval(accepted)")+ rows = [+ _row(line=4),+ _row("subprocess_shell_true", "med", line=2, snippet="run(x, shell=True)"),+ _row("weak_hash", "low", line=3, snippet="md5"),+ _row("aws_access_key", snippet="AWS = 'AKIA****'", line=9, commit="a" * 40),+ accepted,+ ]+ issues = render_gitlab(rows, accepted=frozenset({accepted["fingerprint"]}))+ by_check = {i["check_name"].removeprefix("repowise-security/"): i for i in issues}+ assert set(by_check) == {"eval_call", "subprocess_shell_true", "weak_hash", "aws_access_key"}+ assert by_check["eval_call"]["check_name"] == "repowise-security/eval_call"+ assert by_check["eval_call"]["severity"] == "critical"+ assert by_check["subprocess_shell_true"]["severity"] == "minor"+ assert by_check["weak_hash"]["severity"] == "minor"+ assert by_check["eval_call"]["location"] == {"path": "a.py", "lines": {"begin": 4}}+ assert by_check["aws_access_key"]["location"]["lines"] == {"begin": 1}+ assert "aaaaaaa" in by_check["aws_access_key"]["description"]+ assert by_check["eval_call"]["fingerprint"] == rows[0]["fingerprint"]+++ ("fail_on", "expected"),+ [+ ("high", ["critical", "minor", "minor"]),+ ("med", ["critical", "major", "minor"]),+ ("low", ["critical", "major", "major"]),+ ],+)+def test_gitlab_severity_follows_the_gate(fail_on, expected):+ rows = [+ _row(),+ _row("subprocess_shell_true", "med", snippet="run(x, shell=True)"),+ _row("weak_hash", "low", snippet="md5"),+ ]+ assert [i["severity"] for i in render_gitlab(rows, fail_on=fail_on)] == expected+++def test_gitlab_keeps_two_identical_lines_apart():+ # Two identical matched lines in one file share a fingerprint by design.+ rows = [_row(line=7), _row(line=2)]+ first, second = render_gitlab(rows)+ assert first["location"]["lines"]["begin"] == 2+ assert (first["fingerprint"], second["fingerprint"]) == (+ rows[0]["fingerprint"],+ rows[0]["fingerprint"] + ":2",+ )diff --git a/tests/unit/cli/test_doc_drift_check.py b/tests/unit/cli/test_doc_drift_check.pyindex 86a022000..d38db864a 100644--- a/tests/unit/cli/test_doc_drift_check.py+++ b/tests/unit/cli/test_doc_drift_check.py@@ -179,6 +179,29 @@ def test_sarif_is_one_json_document(monkeypatch, tmp_path):]["uri"] == "docs/a.md"+def test_gitlab_is_one_issue_list_without_the_baseline(monkeypatch, tmp_path):+ baseline = tmp_path / "baseline.json"+ _invoke(monkeypatch, tmp_path, [_finding()], ["--write-baseline", str(baseline)])+ findings = [_finding(), _finding(target="src/other.py", confidence=0.5)]+ args = ["--baseline", str(baseline), "--format", "gitlab"]+ result = _invoke(monkeypatch, tmp_path, findings, args, runner=_split_runner())+ assert result.exit_code == 0+ (issue,) = json.loads(result.stdout)+ assert issue["severity"] == "minor"+ assert issue["location"] == {"path": "docs/a.md", "lines": {"begin": 3}}+++def test_gitlab_prints_an_empty_list_when_it_cannot_evaluate(monkeypatch, tmp_path):+ bad = tmp_path / "bad.json"+ bad.write_text("{not json", encoding="utf-8")+ result = _invoke(+ monkeypatch, tmp_path, [_finding()], ["--baseline", str(bad), "--format", "gitlab"],+ runner=_split_runner(),+ )+ assert result.exit_code == 2+ assert json.loads(result.stdout) == []++def test_document_filter_narrows_the_gate(monkeypatch, tmp_path):findings = [_finding(), _finding(file_path=".github/b.md")]result = _invoke(diff --git a/tests/unit/cli/test_doc_drift_cmd.py b/tests/unit/cli/test_doc_drift_cmd.pyindex aa13e0800..6270b1ce5 100644--- a/tests/unit/cli/test_doc_drift_cmd.py+++ b/tests/unit/cli/test_doc_drift_cmd.py@@ -107,9 +107,15 @@ def test_an_unreadable_index_refuses_rather_than_reporting_zero(assert "findings" not in payload[email protected]("fmt", ["sarif", "markdown", "github"])-def test_a_refusal_under_a_machine_format_keeps_stdout_empty(monkeypatch, tmp_path, fmt):- """A refusal must not land in the file a CI step redirects stdout into."""+ ("fmt", "stdout"), [("sarif", ""), ("markdown", ""), ("github", ""), ("gitlab", "[]")]+)+def test_a_refusal_under_a_machine_format_keeps_stdout_empty(monkeypatch, tmp_path, fmt, stdout):+ """A refusal must not land in the file a CI step redirects stdout into.++ ``gitlab`` prints an empty issue list instead, so the Code Quality+ artifact stays valid JSON.+ """monkeypatch.setattr(doc_drift_cmd, "_repo_path", lambda *a, **k: tmp_path)def _run(coro):@@ -119,7 +125,7 @@ def test_a_refusal_under_a_machine_format_keeps_stdout_empty(monkeypatch, tmp_pamonkeypatch.setattr(doc_drift_cmd, "run_async", _run)result = _split_runner().invoke(doc_drift_cmd.doc_drift_command, ["--format", fmt])assert result.exit_code == 1- assert result.stdout == ""+ assert result.stdout.strip() == stdoutassert "No readable Repowise index" in result.stderr@@ -222,3 +228,5 @@ def test_min_confidence_defaults_to_showing_what_the_index_stored(monkeypatch, tCliRunner().invoke(doc_drift_cmd.doc_drift_command, [])assert captured["mc"] is None++diff --git a/tests/unit/cli/test_security_check_cmd.py b/tests/unit/cli/test_security_check_cmd.pyindex 86e37a8f8..76e56ee0c 100644--- a/tests/unit/cli/test_security_check_cmd.py+++ b/tests/unit/cli/test_security_check_cmd.py@@ -9,6 +9,7 @@ import pytestfrom click.testing import CliRunnerfrom repowise.cli.commands.security_cmd import security_command+from tests.unit.cli.test_format_json_rollout import _split_runnerKEY = "AKIA" + "QZXNRTVYWMPKLBHG"@@ -137,3 +138,40 @@ def test_sarif_suppresses_baselined_findings_below_the_threshold(repo, tmp_path)result = _check(repo, "--baseline", str(baseline), "--format", "sarif")(res,) = json.loads(result.stdout)["runs"][0]["results"]assert res["ruleId"] == "weak_hash" and res["suppressions"] == [{"kind": "external"}]+++def test_gitlab_format_is_one_issue_list(repo):+ _commit(repo, "run.py", "import os\nos.system(cmd)\n")+ result = _check(repo, "--format", "gitlab")+ assert result.exit_code == 1+ (issue,) = json.loads(result.stdout)+ assert issue["check_name"] == "repowise-security/os_system"+ assert issue["severity"] == "critical"+ assert issue["location"] == {"path": "run.py", "lines": {"begin": 2}}+++def test_gitlab_severity_follows_a_non_default_fail_on(repo):+ _commit(repo, "h.py", "import hashlib\nhashlib.md5(b'')\n")+ (default,) = json.loads(_check(repo, "--format", "gitlab").stdout)+ assert default["severity"] == "minor"+ result = _check(repo, "--fail-on", "low", "--format", "gitlab")+ assert result.exit_code == 1+ (issue,) = json.loads(result.stdout)+ assert issue["severity"] == "major"+++def test_gitlab_leaves_baselined_findings_out(repo, tmp_path):+ _commit(repo, "h.py", "import hashlib\nhashlib.md5(b'')\n")+ baseline = tmp_path / "b.json"+ assert _check(repo, "--write-baseline", str(baseline)).exit_code == 0+ result = _check(repo, "--baseline", str(baseline), "--format", "gitlab")+ assert result.exit_code == 0+ assert json.loads(result.stdout) == []+++def test_gitlab_prints_an_empty_list_when_it_cannot_evaluate(tmp_path):+ result = _split_runner().invoke(+ security_command, ["check", "--path", str(tmp_path), "--format", "gitlab"]+ )+ assert result.exit_code == 2+ assert json.loads(result.stdout) == []diff --git a/tests/unit/doc_drift/test_render.py b/tests/unit/doc_drift/test_render.pyindex ec028bd76..b2ecfeaee 100644--- a/tests/unit/doc_drift/test_render.py+++ b/tests/unit/doc_drift/test_render.py@@ -1,4 +1,4 @@-"""CI renderings: markdown summary, GitHub annotations, SARIF."""+"""CI renderings: markdown summary, GitHub annotations, SARIF, GitLab Code Quality."""from __future__ import annotations@@ -9,6 +9,7 @@ from repowise.core.analysis.doc_drift.gate import evaluate_gatefrom repowise.core.analysis.doc_drift.render import (SARIF_FINGERPRINT_KEY,render_github_annotations,+ render_gitlab,render_markdown,render_sarif,)@@ -186,3 +187,39 @@ def test_sarif_is_deterministic():for r in render_sarif(findings, tool_version="x")["runs"][0]["results"]]assert uris == [("docs/a.md", 1), ("docs/a.md", 3), ("docs/z.md", 3)]+++# -- GitLab Code Quality ------------------------------------------------------+++def test_gitlab_severity_follows_the_gate_and_drops_the_baseline():+ accepted = _f("gone.py")+ findings = [_f("a.py", suggestion="pkg/a/"), _f("R.md#x", 0.5, kind="anchor"), accepted]+ fp = derive_doc_drift_fingerprint("docs/a.md", "path", "gone.py")+ issues = render_gitlab(findings, accepted=frozenset({fp}))+ assert issues == [+ {+ "description": "Document names R.md#x, which no longer exists.",+ "check_name": "repowise-doc-drift/anchor",+ "fingerprint": derive_doc_drift_fingerprint("docs/a.md", "anchor", "R.md#x"),+ "severity": "minor",+ "location": {"path": "docs/a.md", "lines": {"begin": 3}},+ },+ {+ "description": "Document names a.py, which no longer exists. Likely now: pkg/a/.",+ "check_name": "repowise-doc-drift/path",+ "fingerprint": derive_doc_drift_fingerprint("docs/a.md", "path", "a.py"),+ "severity": "major",+ "location": {"path": "docs/a.md", "lines": {"begin": 3}},+ },+ ]+ (lowered,) = render_gitlab([_f("R.md#x", 0.5, kind="anchor")], fail_on=0.4)+ assert lowered["severity"] == "major"+++def test_gitlab_is_deterministic_and_unique():+ findings = [_f("b.py", doc="docs/z.md"), _f("a.py"), _f("a.py", line=9)]+ one = render_gitlab(findings)+ assert one == render_gitlab(list(reversed(findings)))+ fps = [i["fingerprint"] for i in one]+ assert len(set(fps)) == 3 and fps[1] == fps[0] + ":2"diff --git a/tests/unit/test_ci_action.py b/tests/unit/test_ci_action.pyindex 2ce93a1ea..a1687f5e1 100644--- a/tests/unit/test_ci_action.py+++ b/tests/unit/test_ci_action.py@@ -8,11 +8,13 @@ run here against a stand-in ``repowise`` to pin its exit-code rules.from __future__ import annotations+import jsonimport osimport reimport shutilimport statimport subprocess+import sysfrom pathlib import Pathimport pytest@@ -51,15 +53,118 @@ def test_the_gitlab_template_parses() -> None:assert {"repowise-coverage", "repowise-doc-drift", "repowise-security"} <= set(jobs)+def _gitlab_jobs() -> dict:+ return yaml.safe_load(GITLAB.read_text(encoding="utf-8"))++[email protected]("gate", ["doc-drift", "security"])+def test_the_gitlab_code_quality_report_is_the_file_the_job_writes(gate) -> None:+ job = _gitlab_jobs()[f"repowise-{gate}"]+ report = f"gl-code-quality-{gate}.json"+ assert job["artifacts"]["reports"]["codequality"] == report+ assert f"repowise-{gate}.md" in job["artifacts"]["paths"]+ (block,) = job["script"]+ assert f"mv gl.tmp {report}" in block and f"echo '[]' > {report}" in block+++def test_the_default_branch_publishes_the_reports_the_widget_compares_against() -> None:+ job = _gitlab_jobs()["repowise-code-quality"]+ assert "extends" not in job and job["allow_failure"] is True+ assert job["rules"] == [{"if": "$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH"}]+ # One path: the report type takes a single file.+ assert job["artifacts"]["reports"]["codequality"] == "gl-code-quality-doc-drift.json"+++_ISSUE = '[{"check_name": "repowise-security/eval_call"}]'+++def _run_gitlab_job(tmp_path: Path, bin_dir: Path, job_name: str, **env: str) -> int:+ """Run a job's script the way GitLab does: one shell under ``set -eo pipefail``."""+ bash = _bash()+ if bash is None:+ pytest.skip("bash is not available")+ # Out of the install line only: the fake stands in for the installed CLI.+ lines = [+ line+ for block in _gitlab_jobs()[job_name]["script"]+ for line in block.splitlines()+ if "pip install" not in line+ ]+ python_dir = str(Path(sys.executable).parent)+ full = {+ **os.environ,+ "PATH": os.pathsep.join([str(bin_dir), python_dir, os.environ["PATH"]]),+ "FAKE_LOG": str(tmp_path / "log"),+ "CI_MERGE_REQUEST_TARGET_BRANCH_NAME": "main",+ **env,+ }+ script = "\n".join(["set -eo pipefail", *lines]) + "\n"+ proc = subprocess.run([bash, "-c", script], cwd=tmp_path, env=full, capture_output=True)+ return proc.returncode+++ "case",+ [+ # (gate, markdown run's exit, gitlab run's exit, gitlab run's stdout, artifact)+ ("doc-drift", "0", "0", "[]", []),+ ("security", "1", "1", _ISSUE, json.loads(_ISSUE)), # the gate failed+ ("security", "2", "2", "[]", []), # could not evaluate+ ("doc-drift", "1", "0", _ISSUE, json.loads(_ISSUE)), # the runs disagree+ ("doc-drift", "0", "1", "Traceback (most recent call last):", []), # a crash+ ("security", "0", "2", "", []), # a usage error prints nothing+ ],+)+def test_the_gitlab_job_writes_a_valid_report_and_exits_with_the_gate(+ tmp_path, fake_repowise, case+) -> None:+ gate, markdown_code, gitlab_code, gitlab_out, report = case+ code = _run_gitlab_job(+ tmp_path,+ fake_repowise,+ f"repowise-{gate}",+ FAKE_DOC_DRIFT=markdown_code,+ FAKE_SECURITY=markdown_code,+ FAKE_GITLAB=gitlab_code,+ FAKE_GITLAB_OUT=gitlab_out,+ )+ assert code == int(markdown_code)+ first, second = (tmp_path / "log").read_text(encoding="utf-8").splitlines()+ assert first.endswith("--format markdown") and second.endswith("--format gitlab")+ assert first.removesuffix("markdown") == second.removesuffix("gitlab")+ artifact = tmp_path / f"gl-code-quality-{gate}.json"+ assert json.loads(artifact.read_text(encoding="utf-8")) == report+++def test_the_default_branch_job_writes_the_doc_drift_report(tmp_path, fake_repowise) -> None:+ code = _run_gitlab_job(+ tmp_path, fake_repowise, "repowise-code-quality", FAKE_DOC_DRIFT="1",+ FAKE_GITLAB_OUT=_ISSUE,+ )+ assert code == 0+ (call,) = (tmp_path / "log").read_text(encoding="utf-8").splitlines()+ assert call.startswith("doc-drift --check") and call.endswith("--format gitlab")+ doc_drift = tmp_path / "gl-code-quality-doc-drift.json"+ assert json.loads(doc_drift.read_text(encoding="utf-8")) == json.loads(_ISSUE)++@pytest.fixturedef fake_repowise(tmp_path: Path) -> Path:- """A ``repowise`` that logs its arguments and exits with ``$FAKE_<GATE>``."""+ """A ``repowise`` that logs its arguments and exits with ``$FAKE_<GATE>``.++ A ``--format gitlab`` run prints ``$FAKE_GITLAB_OUT`` (default ``[]``) and+ exits with ``$FAKE_GITLAB`` when that is set.+ """bin_dir = tmp_path / "bin"bin_dir.mkdir()fake = bin_dir / "repowise"fake.write_text("#!/usr/bin/env bash\n"'echo "$*" >> "$FAKE_LOG"\n'+ 'case "$*" in *"--format gitlab"*)\n'+ ' printf "%s" "${FAKE_GITLAB_OUT-[]}"\n'+ ' if [ -n "${FAKE_GITLAB:-}" ]; then exit "$FAKE_GITLAB"; fi ;;\n'+ "esac\n"'case "$1" in\n'' coverage) exit "${FAKE_COVERAGE:-0}" ;;\n'' doc-drift) exit "${FAKE_DOC_DRIFT:-0}" ;;\n'diff --git a/tests/unit/test_ci_gitlab.py b/tests/unit/test_ci_gitlab.pynew file mode 100644index 000000000..5c70599ff--- /dev/null+++ b/tests/unit/test_ci_gitlab.py@@ -0,0 +1,49 @@+"""``repowise.core.ci.gitlab``: the GitLab Code Quality issue list."""++from __future__ import annotations++import pytest++from repowise.core.ci import gitlab+++def test_issue_has_the_required_fields():+ assert gitlab.issue("path", "major", "msg", "\\docs\\a.md", 7, "abc") == {+ "description": "msg",+ "check_name": "path",+ "fingerprint": "abc",+ "severity": "major",+ "location": {"path": "docs/a.md", "lines": {"begin": 7}},+ }++[email protected]("line", [None, 0])+def test_an_issue_without_a_line_sits_on_line_one(line):+ assert gitlab.issue("r", "info", "m", "/a.py", line, "f")["location"] == {+ "path": "a.py",+ "lines": {"begin": 1},+ }+++def test_an_unknown_severity_is_refused():+ with pytest.raises(ValueError, match="high"):+ gitlab.issue("r", "high", "m", "a.py", 1, "f")+++def test_report_names_the_tool_and_makes_repeated_fingerprints_unique_in_order():+ issues = [gitlab.issue("r", "minor", str(i), "a.py", i, fp) for i, fp in enumerate("aaba")]+ out = gitlab.report("tool", issues)+ assert [i["fingerprint"] for i in out] == ["a", "a:2", "b", "a:3"]+ assert {i["check_name"] for i in out} == {"tool/r"}+ assert (issues[1]["fingerprint"], issues[1]["check_name"]) == ("a", "r") # not mutated+++def test_a_suffix_never_collides_with_a_given_fingerprint():+ issues = [gitlab.issue("r", "minor", "m", "a.py", 1, fp) for fp in ("a:2", "a", "a")]+ assert [i["fingerprint"] for i in gitlab.report("t", issues)] == ["a:2", "a", "a:3"]+++def test_report_leaves_out_what_the_baseline_accepted():+ issues = [gitlab.issue("r", "minor", "m", "a.py", 1, fp) for fp in ("a", "b", "a")]+ out = gitlab.report("t", issues, accepted=frozenset({"a"}))+ assert [i["fingerprint"] for i in out] == ["b"]