Open-source AI Security Operations Center: alert fusion, LLM-agent triage, MITRE ATT&CK investigation, and a replayable decision ledger for every agent step. Self-hostable, runs with no API keys, MIT licensed. Ships an MCP server for Claude, Cursor and Continue.
Python agentic-aiai-securityalert-triageblue-teamcybersecuritydetection-engineeringincident-responselanggraphllm-agentsmcpmitre-attackmodel-context-protocolpurple-teamsecurity-operationsself-hostedsiemsoarsocthreat-detectionthreat-intelligence
fix(release): a skipped upload must not report as a successful publish (#1014)
Beenu Aroraea282cb4b238
diff --git a/.github/release-packages.yml b/.github/release-packages.ymlnew file mode 100644index 000000000..5f40a97f7--- /dev/null+++ b/.github/release-packages.yml@@ -0,0 +1,125 @@+# Every package `release.yml` uploads, and whether it is on its registry.+#+# This file exists because a green publish job is not a published package. On+# the v12.2.0 release run all eight `npm — publish …` / `PyPI — publish …`+# jobs reported success and all eight packages returned 404: only the final+# upload step is credential-gated, so each job built the artefact, reached the+# upload, skipped it, and exited 0. A reader of the Actions page saw eight+# green publishes and nine packages that do not exist.+#+# The credential gate is correct — the blocker is an account action, described+# in docs/operations/publishing.md — so the fix is not to publish but to stop+# the signal lying. Part of that is naming the knowingly-absent set here,+# where it is reviewable in a diff, instead of leaving it to be inferred from+# a 404.+#+# `scripts/check_published_packages.py` enforces this file in three+# directions, because a gate that only runs one way passes while drift+# accumulates in the other:+#+# 1. this file <-> the npm and PyPI matrices in `.github/workflows/release.yml`+# A package the workflow builds and this file does not declare is a+# package nothing will ever check. A package declared here that the+# workflow does not build is a claim with no pipeline behind it.+#+# 2. `published: false` -> the registry must 404+# When a credential arrives and an upload succeeds, this entry becomes+# false and must be corrected. That is the point: the knowingly-absent+# set has to shrink visibly, in a commit, rather than quietly stop being+# true.+#+# 3. `published: true` -> the registry must resolve, at `version`+# This is the direction the release run cannot check itself. A job that+# claims a publish and uploaded nothing fails here.+#+# `version` is read from each package's own manifest by the gate, not copied+# here — a version in two places is a version that disagrees with itself.++packages:+ # ── npm ────────────────────────────────────────────────────────────────+ - name: aisoc+ registry: npm+ directory: packages/aisoc-lite+ install: npx aisoc+ published: false+ reason: >-+ npm configures trusted publishing on a package's settings page, and a+ package that has never been published has no settings page — so the+ first upload of each npm package needs an `NPM_TOKEN` secret and every+ upload after it does not. This repository's only secret is+ `FLY_API_TOKEN`. See docs/operations/publishing.md#one-time-setup-npm.++ - name: '@aisoc/mcp'+ registry: npm+ directory: services/mcp+ install: npx -y @aisoc/mcp+ published: false+ reason: >-+ Same first-upload token requirement as `aisoc`, and no `NPM_TOKEN` is+ configured. See docs/operations/publishing.md#one-time-setup-npm.++ - name: '@aisoc/sdk'+ registry: npm+ directory: packages/sdk-ts+ install: npm install @aisoc/sdk+ published: false+ reason: >-+ Same first-upload token requirement as `aisoc`, and no `NPM_TOKEN` is+ configured. See docs/operations/publishing.md#one-time-setup-npm.++ # ── PyPI ───────────────────────────────────────────────────────────────+ #+ # PyPI needs no token: it supports *pending* trusted publishers, so the+ # trust relationship can be registered before the project exists. The+ # blocker is only that nobody has registered them and set the+ # `AISOC_PYPI_TRUSTED_PUBLISHING` repository variable, which is what arms+ # the upload step. The repository currently has no variables at all.+ - name: aisoc-sandbox+ registry: pypi+ directory: packages/aisoc-sandbox+ install: pip install aisoc-sandbox+ published: false+ reason: >-+ No pending trusted publisher is registered on PyPI and the+ `AISOC_PYPI_TRUSTED_PUBLISHING` repository variable is unset, so the+ upload step does not run. See+ docs/operations/publishing.md#one-time-setup-pypi.++ - name: aisoc-cli+ registry: pypi+ directory: packages/aisoc-cli+ install: pip install aisoc-cli+ published: false+ reason: >-+ No pending trusted publisher is registered on PyPI and+ `AISOC_PYPI_TRUSTED_PUBLISHING` is unset. See+ docs/operations/publishing.md#one-time-setup-pypi.++ - name: aisoc-sdk+ registry: pypi+ directory: packages/sdk-py+ install: pip install aisoc-sdk+ published: false+ reason: >-+ No pending trusted publisher is registered on PyPI and+ `AISOC_PYPI_TRUSTED_PUBLISHING` is unset. See+ docs/operations/publishing.md#one-time-setup-pypi.++ - name: aisoc-plugin-sdk+ registry: pypi+ directory: packages/plugin-sdk-py+ install: pip install aisoc-plugin-sdk+ published: false+ reason: >-+ No pending trusted publisher is registered on PyPI and+ `AISOC_PYPI_TRUSTED_PUBLISHING` is unset. See+ docs/operations/publishing.md#one-time-setup-pypi.++ - name: aisoc-detections+ registry: pypi+ directory: packages/aisoc-detections+ published: false+ reason: >-+ No pending trusted publisher is registered on PyPI and+ `AISOC_PYPI_TRUSTED_PUBLISHING` is unset. See+ docs/operations/publishing.md#one-time-setup-pypi.diff --git a/.github/workflows/publish-images.yml b/.github/workflows/publish-images.ymlindex 2428ce7a5..da97866ed 100644--- a/.github/workflows/publish-images.yml+++ b/.github/workflows/publish-images.yml@@ -403,6 +403,18 @@ jobs:exit 1fimapfile -t tags < <(jq -r '.tags[]' <<< "$DOCKER_METADATA_OUTPUT_JSON")+ # Every tag above is `enable=`d on a condition, and for the demo+ # image both conditions are ref-dependent — so a dispatch from any+ # branch but `main` produces an empty tag list. The job did fail,+ # on `tags[0]: unbound variable` several lines further down and+ # after `imagetools create` had already been handed no `--tag` at+ # all. A build that produced two architectures and can name none of+ # them has done no publishing, and should say that rather than+ # leave an operator reading a shell diagnostic.+ if [ "${#tags[@]}" -eq 0 ]; then+ echo "::error::no tag is enabled for ${IMAGE} on ref ${GITHUB_REF}, so the built digests would be unreachable. Dispatch from \`main\`, or add a tag rule for this ref." >&2+ exit 1+ fidocker buildx imagetools create \"${tags[@]/#/--tag=}" \$(printf "${IMAGE}@sha256:%s " *)diff --git a/.github/workflows/readme-gates.yml b/.github/workflows/readme-gates.ymlindex 384340880..a7139540e 100644--- a/.github/workflows/readme-gates.yml+++ b/.github/workflows/readme-gates.yml@@ -157,6 +157,13 @@ jobs:with:python-version: '3.11'+ # The gate reads `.github/release-packages.yml` and `release.yml`'s+ # publish matrices, so it needs a YAML parser. Installed rather than+ # worked around: the alternative is a gate that drops its+ # manifest-versus-workflow check when the import fails, which is the+ # silent skip this whole area is being fixed for.+ - run: python -m pip install --quiet pyyaml+# `--require-network` on a hosted runner: the script skips when a# registry is unreachable so a contributor on a plane is not told the# README is wrong, but a runner that cannot reach npm has not checkeddiff --git a/.github/workflows/release.yml b/.github/workflows/release.ymlindex cbd13d79a..cbdced81c 100644--- a/.github/workflows/release.yml+++ b/.github/workflows/release.yml@@ -14,13 +14,13 @@ on:## Dispatch this from `main` so the current workflow definition runs, and# name the tag to publish; the build checks that tag out for its source.- # The GitHub Release, npm and PyPI jobs are push-only and stay skipped,- # because re-publishing a package version is not a thing registries allow- # and re-cutting a release note is not a thing anyone wants.+ # The GitHub Release stays skipped, because re-cutting a release note is not+ # a thing anyone wants, and no package is ever *uploaded* on a dispatch,+ # because re-publishing a package version is not a thing registries allow.workflow_dispatch:inputs:tag:- description: 'Existing release tag whose images should be (re)published, e.g. v11.0.0'+ description: 'Existing release tag whose images should be (re)published, e.g. v11.0.0. Ignored when packages_only is set.'required: truetype: stringpromote_stable:@@ -28,6 +28,17 @@ on:required: falsedefault: falsetype: boolean+ # A workflow change nobody can exercise is a claim. The packaging half+ # of a release otherwise runs only on a tag push, which is the worst+ # possible moment to discover it is wrong — so this runs it on demand:+ # build, pack, validate and report, with the image rebuild skipped and+ # no upload reachable. It is also how a maintainer checks that every+ # package is still publishable before cutting a tag.+ packages_only:+ description: 'Run only the packaging and publication report. Skips the image rebuild, uploads nothing, and ignores `tag`.'+ required: false+ default: false+ type: booleanpermissions:contents: write@@ -190,6 +201,33 @@ jobs:donecat checksums.txt+ # `action-gh-release` defaults `fail_on_unmatched_files` to **false**,+ # so a `files:` entry that matches nothing is dropped and the release is+ # created without it — green. The SBOM is produced by a third-party+ # action into a path this job constructs by string, which is exactly the+ # kind of pairing that silently stops matching one day. An advertised+ # asset that is absent is a claim the release page makes and cannot+ # keep, so both halves are closed: named here, and enforced by the+ # action below.+ - name: Every advertised release asset exists and is not empty+ env:+ VERSION: ${{ steps.version.outputs.VERSION }}+ run: |+ set -euo pipefail+ missing=0+ for asset in \+ "dist/aisoc-${VERSION}-source.tar.gz" \+ "dist/aisoc-sbom-${VERSION}.spdx.json" \+ "dist/checksums.txt"; do+ if [ ! -s "$asset" ]; then+ echo "::error::${asset} is missing or empty, and the release advertises it." >&2+ missing=$((missing + 1))+ else+ echo "ok: ${asset} ($(wc -c < "$asset") bytes)"+ fi+ done+ [ "$missing" -eq 0 ]+- name: Create Releaseuses: softprops/action-gh-release@3d0d9888cb7fd7b750713d6e236d1fcb99157228 # v3with:@@ -197,11 +235,81 @@ jobs:body_path: release-body.mddraft: falseprerelease: ${{ contains(github.ref_name, '-') }}+ fail_on_unmatched_files: truefiles: |dist/aisoc-${{ steps.version.outputs.VERSION }}-source.tar.gzdist/aisoc-sbom-${{ steps.version.outputs.VERSION }}.spdx.jsondist/checksums.txt+ # ── Which uploads are armed, decided once and named in the job titles ──+ #+ # The defect this job exists to fix: on the v12.2.0 release run, all eight+ # `npm — publish …` / `PyPI — publish …` jobs reported **success** and all+ # eight packages returned 404. Only the final upload step is credential-+ # gated, so each job built the artefact, reached the upload, skipped it, and+ # exited 0. The Actions page showed eight green publishes for nothing.+ #+ # The gate itself is right — the blocker is an account action, not code, and+ # the release must still complete without it, because the images and the+ # GitHub Release genuinely do publish. What was wrong is the signal.+ #+ # A job's `name:` can read `needs`, so deciding here and naming the answer+ # there makes the Actions list itself honest: `npm — publish aisoc` when an+ # upload can happen, `npm — pack only, NOT uploaded: aisoc` when it cannot.+ # It has to be a separate job because `secrets` is not available in a+ # job-level `name:` or `if:` — only in a job `env:` — so the NPM_TOKEN half+ # of the question cannot be asked where the answer is needed.+ package-credentials:+ name: Packages — which uploads are armed+ runs-on: ubuntu-latest+ if: github.event_name == 'push' || inputs.packages_only+ timeout-minutes: 5+ outputs:+ npm: ${{ steps.probe.outputs.npm }}+ pypi: ${{ steps.probe.outputs.pypi }}+ env:+ NPM_TOKEN: ${{ secrets.NPM_TOKEN }}+ steps:+ - id: probe+ env:+ NPM_TRUSTED: ${{ vars.AISOC_NPM_TRUSTED_PUBLISHING }}+ PYPI_TRUSTED: ${{ vars.AISOC_PYPI_TRUSTED_PUBLISHING }}+ # Nothing is uploaded on a dispatch even with a credential present:+ # a registry refuses a version it already holds, so a republish of+ # the package half is not a thing that can succeed.+ UPLOADS_POSSIBLE: ${{ github.event_name == 'push' }}+ run: |+ set -euo pipefail+ npm=unarmed+ pypi=unarmed+ if [ "$UPLOADS_POSSIBLE" = "true" ]; then+ if [ "$NPM_TRUSTED" = "true" ] || [ -n "$NPM_TOKEN" ]; then npm=armed; fi+ if [ "$PYPI_TRUSTED" = "true" ]; then pypi=armed; fi+ fi+ echo "npm=$npm" >> "$GITHUB_OUTPUT"+ echo "pypi=$pypi" >> "$GITHUB_OUTPUT"+ {+ echo "## Package uploads"+ echo+ echo "| Registry | This run | Why |"+ echo "| --- | --- | --- |"+ if [ "$npm" = "armed" ]; then+ echo "| npm | uploads | a credential is configured |"+ elif [ "$UPLOADS_POSSIBLE" != "true" ]; then+ echo "| npm | packs only | not a tag push; a registry will not accept a version twice |"+ else+ echo "| npm | packs only | neither trusted publishing nor \`NPM_TOKEN\` is configured |"+ fi+ if [ "$pypi" = "armed" ]; then+ echo "| PyPI | uploads | trusted publishing is enabled |"+ elif [ "$UPLOADS_POSSIBLE" != "true" ]; then+ echo "| PyPI | packs only | not a tag push; a registry will not accept a version twice |"+ else+ echo "| PyPI | packs only | \`AISOC_PYPI_TRUSTED_PUBLISHING\` is not set |"+ fi+ } >> "$GITHUB_STEP_SUMMARY"+ echo "npm=$npm pypi=$pypi"+# Build & push Docker images## Most services build from their own service directory. The `web` app is@@ -232,7 +340,9 @@ jobs:needs: release# `needs` alone would skip this whenever `release` is skipped, which is# every dispatched run — the one path that exists to publish images.- if: always() && needs.release.result != 'failure' && needs.release.result != 'cancelled'+ # `packages_only` is the one dispatch that deliberately leaves the+ # registry alone, so the whole image chain below sits out with it.+ if: always() && !inputs.packages_only && needs.release.result != 'failure' && needs.release.result != 'cancelled'timeout-minutes: 30strategy:fail-fast: false@@ -636,10 +746,16 @@ jobs:# specifiers to real versions, which npm would otherwise reject. The tarball# is then handed to `npm publish`, which is what supports `--provenance`.npm-publish:- name: npm — publish ${{ matrix.pkg }}+ # The job title states what the job did, so a reader does not have to open+ # a log to find out whether anything was uploaded. `needs` is one of the+ # few contexts available in a job `name:`, which is why the credential+ # question is answered in its own job above.+ name: npm — ${{ needs.package-credentials.outputs.npm == 'armed' && 'publish' || 'pack only, NOT uploaded:' }} ${{ matrix.pkg }}runs-on: ubuntu-latest- needs: release- if: github.event_name == 'push'+ needs: [release, package-credentials]+ # `always()` because `release` is skipped on a dispatch and a skip+ # propagates down a `needs` chain unless each job restates what it needs.+ if: always() && needs.package-credentials.result == 'success' && needs.release.result != 'failure' && needs.release.result != 'cancelled'permissions:contents: readid-token: write # npm provenance attestation@@ -701,13 +817,28 @@ jobs:# Republishing an unchanged version is a hard error on npm, and package# versions move independently of the monorepo tag, so most releases will# legitimately have nothing to push for some packages.+ #+ # `npm view … >/dev/null 2>&1` treated *every* failure as "not+ # published" and threw the reason away with it, so a registry outage, a+ # rate limit and a genuine 404 were one answer. The direction is safe —+ # an unknown becomes an attempted publish, which npm rejects if it was+ # wrong — but an operator reading the log could not tell which happened.+ # E404 is the only answer that means absent; anything else is recorded+ # as unknown, out loud.- name: Is this version already on npm?id: exists+ env:+ SPEC: ${{ matrix.pkg }}@${{ steps.pack.outputs.version }}run: |- set -euo pipefail- if npm view "${{ matrix.pkg }}@${{ steps.pack.outputs.version }}" version >/dev/null 2>&1; then+ set -uo pipefail+ if output=$(npm view "$SPEC" version 2>&1); then+ echo "$SPEC is already on npm ($output)."echo "already=true" >> "$GITHUB_OUTPUT"+ elif grep -q 'E404' <<< "$output"; then+ echo "$SPEC is not on npm yet."+ echo "already=false" >> "$GITHUB_OUTPUT"else+ echo "::warning title=npm::Could not establish whether $SPEC is published; npm said: ${output//$'\n'/ }. Treating it as absent, which means an upload will be attempted and the registry gets the last word."echo "already=false" >> "$GITHUB_OUTPUT"fi@@ -716,7 +847,7 @@ jobs:# fails with ENEEDAUTH when there is no token, so the upgrade is what# makes the token-free path reachable at all rather than a nicety.- name: Upgrade npm for trusted publishing- if: vars.AISOC_NPM_TRUSTED_PUBLISHING == 'true'+ if: needs.package-credentials.outputs.npm == 'armed' && vars.AISOC_NPM_TRUSTED_PUBLISHING == 'true'run: |set -euo pipefailnpm install -g npm@latest@@ -727,7 +858,7 @@ jobs:# unrotated in repository settings. Provenance is generated# automatically on this path, so `--provenance` is not passed.- name: Publish to npm (trusted publishing)- if: steps.exists.outputs.already == 'false' && vars.AISOC_NPM_TRUSTED_PUBLISHING == 'true'+ if: needs.package-credentials.outputs.npm == 'armed' && steps.exists.outputs.already == 'false' && vars.AISOC_NPM_TRUSTED_PUBLISHING == 'true'run: npm publish "${{ steps.pack.outputs.tarball }}" --access public# The fallback, for as long as it is needed. npm configures a trusted@@ -736,18 +867,29 @@ jobs:# very first upload of each package needs a token, and every upload# after it does not.- name: Publish to npm (token)- if: steps.exists.outputs.already == 'false' && vars.AISOC_NPM_TRUSTED_PUBLISHING != 'true' && env.NPM_TOKEN != ''+ if: needs.package-credentials.outputs.npm == 'armed' && steps.exists.outputs.already == 'false' && vars.AISOC_NPM_TRUSTED_PUBLISHING != 'true' && env.NPM_TOKEN != ''env:NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}run: npm publish "${{ steps.pack.outputs.tarball }}" --access public --provenance- - name: Explain a skipped publish- if: steps.exists.outputs.already == 'true' || (vars.AISOC_NPM_TRUSTED_PUBLISHING != 'true' && env.NPM_TOKEN == '')+ # Reached whenever nothing was uploaded, which — while no credential+ # exists — is every run. It states the outcome in the log; the job's own+ # title states it in the Actions list, and `Packages — publication+ # report` states it in the run summary. Three places, one decision.+ - name: Say plainly that nothing was uploaded+ if: needs.package-credentials.outputs.npm != 'armed' || steps.exists.outputs.already == 'true'+ env:+ ARMED: ${{ needs.package-credentials.outputs.npm }}+ ALREADY: ${{ steps.exists.outputs.already }}+ PKG: ${{ matrix.pkg }}+ VERSION: ${{ steps.pack.outputs.version }}run: |- if [ "${{ steps.exists.outputs.already }}" = "true" ]; then- echo "::notice title=npm::${{ matrix.pkg }}@${{ steps.pack.outputs.version }} is already published, nothing to do."+ if [ "$ALREADY" = "true" ]; then+ echo "::notice title=npm::${PKG}@${VERSION} is already on npm, so this release uploaded nothing for it."+ elif [ "${{ github.event_name }}" != "push" ]; then+ echo "::notice title=npm::${PKG} was built, packed and validated. Nothing was uploaded: a dispatched run never publishes, because a registry will not accept a version twice."else- echo "::warning title=npm::Neither trusted publishing nor NPM_TOKEN is configured, so ${{ matrix.pkg }} was built and packed but not uploaded. See docs/operations/publishing.md."+ echo "::warning title=npm::${PKG} was built, packed and validated, and NOT uploaded — neither trusted publishing nor NPM_TOKEN is configured. npm still has no ${PKG}. See docs/operations/publishing.md."fi# ── Publish the PyPI packages ──────────────────────────────────────────────@@ -757,10 +899,10 @@ jobs:# relationship can be registered before the project exists — that is how the# first upload of each of these works. Setup: docs/operations/publishing.md.pypi-publish:- name: PyPI — publish ${{ matrix.pkg }}+ name: PyPI — ${{ needs.package-credentials.outputs.pypi == 'armed' && 'publish' || 'pack only, NOT uploaded:' }} ${{ matrix.pkg }}runs-on: ubuntu-latest- needs: release- if: github.event_name == 'push'+ needs: [release, package-credentials]+ if: always() && needs.package-credentials.result == 'success' && needs.release.result != 'failure' && needs.release.result != 'cancelled'permissions:contents: readid-token: write # PyPI trusted publishing@@ -796,16 +938,64 @@ jobs:# `skip-existing` because package versions move independently of the# monorepo tag, so a release often has nothing new for a given package.- name: Publish to PyPI- if: vars.AISOC_PYPI_TRUSTED_PUBLISHING == 'true'+ if: needs.package-credentials.outputs.pypi == 'armed'uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2with:packages-dir: ${{ matrix.dir }}/distskip-existing: true- - name: Explain a skipped publish- if: vars.AISOC_PYPI_TRUSTED_PUBLISHING != 'true'+ - name: Say plainly that nothing was uploaded+ if: needs.package-credentials.outputs.pypi != 'armed'+ env:+ PKG: ${{ matrix.pkg }}run: |- echo "::warning title=PyPI::Trusted publishing is not enabled, so ${{ matrix.pkg }} was built and validated but not uploaded. Set the AISOC_PYPI_TRUSTED_PUBLISHING repository variable to 'true' once the pending publishers exist — see docs/operations/publishing.md."+ if [ "${{ github.event_name }}" != "push" ]; then+ echo "::notice title=PyPI::${PKG} was built and twine-checked. Nothing was uploaded: a dispatched run never publishes, because a registry will not accept a version twice."+ else+ echo "::warning title=PyPI::${PKG} was built and twine-checked, and NOT uploaded — AISOC_PYPI_TRUSTED_PUBLISHING is not set, so the upload step did not run. PyPI still has no ${PKG}. See docs/operations/publishing.md."+ fi++ # ── Ask the registries what this release actually published ────────────+ #+ # The loop the package half never closed, and the same one `docker-verify`+ # closes for images. A release is the set of artefacts it published, not the+ # set of jobs that finished, so the question has to be put to npm and PyPI+ # rather than to the workflow's own statuses — which is exactly how eight+ # green publish jobs coexisted with eight 404s.+ #+ # `.github/release-packages.yml` declares which packages are expected to be+ # on their registry and, for each one that is not, why. The gate fails when+ # a package declared published does not resolve (a job claimed an upload+ # that did not happen) and equally when one declared unpublished does (the+ # knowingly-absent set has to shrink in a commit, not in silence).+ #+ # It writes the run summary either way, so the answer is on the run page+ # without opening a log: every package, its version in the tree, whether the+ # registry holds it, and the reason for each one that was packed and not+ # uploaded.+ package-report:+ name: Packages — publication report+ runs-on: ubuntu-latest+ needs: [package-credentials, npm-publish, pypi-publish]+ # `always()` so the report still runs — and still tells the truth — when a+ # package fails to build. A report that only appears on the happy path is+ # not a report.+ if: always() && needs.package-credentials.result == 'success'+ timeout-minutes: 10+ permissions:+ contents: read+ steps:+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0+ with:+ python-version: '3.11'+ - run: python -m pip install --quiet pyyaml+ # `--require-network` because an unreachable registry during a release+ # is an unanswered question, and this is the one job whose entire+ # purpose is to answer it. Elsewhere the same gate skips offline so a+ # contributor on a train is not told the README is wrong.+ - name: Every package the release claims to have published resolves+ run: python3 scripts/check_published_packages.py --require-network --summary "$GITHUB_STEP_SUMMARY"# ── Publish the Helm chart ────────────────────────────────────────────────#@@ -831,7 +1021,19 @@ jobs:# with 17 image-missing findings and then passed unchanged on a re-run.# The gate was right both times; the ordering was not.needs: [release, docker-manifest]- if: always() && (github.event_name == 'push' || needs.release.result == 'success')+ # Was `always() && (github.event_name == 'push' || needs.release.result ==+ # 'success')`. `release` runs only on a push, so the second arm implied+ # the first and the whole condition reduced to "this is a push" — which+ # made the `Explain a packaged-but-unpushed chart` step below, whose `if`+ # is `github.event_name != 'push'`, unreachable. A branch that documents+ # what happens on a dispatch, and cannot run on one, is a comment wearing+ # a step's clothing.+ #+ # Requiring the manifests instead fixes both ends: the chart is linted,+ # packaged and checked against the registry on a dispatch too (where the+ # notice is now reachable and honest), and on a push it no longer starts+ # when the images it names were never built.+ if: always() && needs.docker-manifest.result == 'success'timeout-minutes: 15permissions:contents: readdiff --git a/CHANGELOG.md b/CHANGELOG.mdindex fbe62e6a7..642cd90b2 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -35,6 +35,68 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0because its version line is independent (0.x) while its peer tracks theStorybook major exactly — the previous hold used`update-types: semver-major`, which cannot hold a 0.x dependency at all.+### Fixed++- **Nine publish jobs reported success having uploaded nothing.** On the+ v12.2.0 release run all 59 jobs were green, eight of them named+ `npm — publish <pkg>` or `PyPI — publish <pkg>`, and all eight packages+ returned 404 from their registry. Only the final upload step is+ credential-gated, so each job built the artefact, reached the upload,+ skipped it on a missing credential, and exited 0. A reader of the Actions+ page saw eight green publishes for packages that do not exist.++ The credential gate is correct and unchanged — the blocker is an account+ action, not code, and the release must still complete without it, because+ the images and the GitHub Release genuinely do publish. What changed is+ that the decision is made once, in a new `package-credentials` job, and+ then carried to three places a reader can see: the job titles in the+ Actions list (`npm — pack only, NOT uploaded: aisoc`), a line in each job's+ log, and a run summary written by the new `package-report` job naming every+ package, whether the registry holds it, and why each skipped one was+ skipped. `secrets` is unavailable in a job-level `name:` or `if:`, which is+ why the decision needs its own job: `needs` is readable there and a secret+ is not.++- **Nothing asked the registry after a release.** `package-report` now runs+ `scripts/check_published_packages.py --require-network`, which reads a new+ committed manifest, `.github/release-packages.yml`, and enforces it in+ three directions: against `release.yml`'s publish matrices both ways, so a+ package the workflow builds and the manifest does not declare cannot go+ unchecked; `published: true` against the registry **at the version in the+ tree**, which is the direction a release run cannot check itself; and+ `published: false` against the registry, so the knowingly-not-published set+ shrinks in a commit when a credential arrives rather than quietly ceasing+ to be true. `tests/test_package_install_claims.py` now reads that same+ manifest and the same registry query instead of its own copy of both — the+ copy covered five of the eight packages and had already drifted, which is+ how `packages/aisoc-lite/README.md` came to advertise+ `npx aisoc triage --demo` with no caveat while `aisoc` 404s on npm.++- **Three more steps in the same workflows could pass without doing their+ work.** `softprops/action-gh-release` defaults `fail_on_unmatched_files` to+ false, so a release advertising a source tarball, an SBOM and a checksum+ file would have been created without any of them that stopped matching —+ green. Both halves are now closed: the assets are asserted present and+ non-empty before the release is created, and the action is told to fail on+ an unmatched file. `chart-publish`'s condition reduced to "this is a push"+ (`release` runs only on a push, so the second arm implied the first),+ which made its `Explain a packaged-but-unpushed chart` step unreachable —+ it now requires the manifests instead, so the chart is linted, packaged and+ checked against the registry on a dispatch too and the notice is reachable+ and true. And `publish-images.yml` produced an empty tag list for the demo+ image on any dispatch from a branch other than `main`, failing several+ lines later on `tags[0]: unbound variable` after `imagetools create` had+ already been handed no tag; it now says what happened.++### Added++- **`release.yml` accepts `packages_only` on `workflow_dispatch`.** Builds,+ packs and validates every package and runs the publication report, with the+ image rebuild skipped and no upload reachable — the preflight refuses to+ arm an upload on a dispatch however well credentialled the repository is,+ because a registry will not accept a version twice. The packaging half of a+ release otherwise runs only on a tag push, which is the worst possible+ moment to discover it is broken.## [12.2.0] - 2026-09-28diff --git a/docs/audit/CLAIM_TO_GATE_MATRIX.md b/docs/audit/CLAIM_TO_GATE_MATRIX.mdindex 6024db330..6f38c966f 100644--- a/docs/audit/CLAIM_TO_GATE_MATRIX.md+++ b/docs/audit/CLAIM_TO_GATE_MATRIX.md@@ -77,7 +77,7 @@ Statuses: `GATED` (a CI job fails when the claim stops being true) · `PARTIAL`| Alerts fan out to Opsgenie / email / external-SOAR destinations (SSRF-guarded) | README (destinations + SOAR) | `ci.yml` api job (`test_compliance_cspm.py` — Opsgenie priority mapping, email subject/recipients, `aisoc.handoff.v1` SOAR envelope; outbound webhook guard blocks non-http(s) schemes + private/loopback targets) | GATED | - || Per-tenant configurable data retention with tenant-scoped purge | README (data lifecycle) | `ci.yml` api job: `test_retention.py` (purge SQL well-formed — clamped window, mandatory tenant predicate, parameterised cutoff) **plus** `test_retention_worker.py`, which asserts the worker issues the delete, that a dry run reports the blast radius without deleting, that the alert delete binds its tenant explicitly (the worker runs with row security off, so relying on RLS would purge every tenant under the first tenant's window), that per-tenant windows are not shared, that tenants without an explicit policy are untouched, and that the count SQL used for the preview has the same predicate as the delete | GATED | `audit_days` is storable but not purged: the audit log is an append-only hash chain, so truncating it invalidates every later verification. Needs chain-aware truncation with a re-anchored checkpoint. || Self-service field-extraction / transform pipelines (custom parsers) | README (data lifecycle) | `ci.yml` api job (`test_pipeline_transforms.py` — the whitelisted DSL renames/maps/extracts onto OCSF with dotted paths + ReDoS-proof grok `extract` (re.escape'd literals + fixed token map, no raw user regex); validation rejects unknown ops/tokens / oversized pipelines; fail-open per op never drops an event; input never mutated) | GATED | - |-| Every package the README tells a visitor to install is published | README (Try AiSOC in 60 seconds) | `release.yml :: npm-publish` + `:: pypi-publish` (all eight distributions are built, packed and `twine check`ed on every release whether or not the upload is armed, so a package that stops being publishable fails the release rather than silently vanishing from the registry) | GATED | - |+| Every package the README tells a visitor to install either resolves, or says in the present tense that it does not | README (Try AiSOC in 60 seconds); each package README | `release.yml :: package-report` (`check_published_packages.py --require-network` asks npm and PyPI directly and writes the answer to the run summary; a package `.github/release-packages.yml` declares published that does not resolve fails the release, and so does one declared unpublished that does) + `ci.yml :: python-test` (root) runs `test_package_install_claims.py` (the same manifest, the same registry query, so the gate and the READMEs cannot disagree) and `test_release_publish_signal.py` (every upload step is guarded by the credential decision, and the job titles carry it) | GATED | The row previously read "…is published" and named `npm-publish` + `pypi-publish` as the gate. Those jobs prove a package is *packable*, never that it shipped: on v12.2.0 all eight reported success and all eight registries returned 404, because only the final upload step is credential-gated. The claim has been restated to what is true and enforced, and the gate moved to one that asks a registry instead of reading a job status || The install commands the README advertises actually run | README (on-ramp table) | `readme-gates.yml :: published-onramp` (runs `npx aisoc@latest triage --demo` and `pip install aisoc-sandbox && aisoc-sandbox demo` against the public registries on a clean runner with no checkout, Linux + macOS, daily as well as on diff; self-arming — warns while a package is unpublished, hard-fails once it exists and breaks) | GATED | - || Documentation links on the front door resolve | README / SUPPORT / CONTRIBUTING | `link-check.yml :: front-door` (hard gate over the five files a new visitor reads; `:: docs-tree` observes the wider documentation tree so pre-existing rot is burned down deliberately rather than blocking unrelated PRs) | GATED | - || The CLI makes no network call unless an endpoint is configured | `packages/aisoc-lite/TELEMETRY.md` | `ci.yml` aisoc-lite vitest (`telemetry.test.ts` — there is no default endpoint, and an enabled-but-unconfigured run is asserted to make zero `fetch` calls) | GATED | - |diff --git a/docs/operations/publishing.md b/docs/operations/publishing.mdindex 43fb89db5..ed5714d53 100644--- a/docs/operations/publishing.md+++ b/docs/operations/publishing.md@@ -6,10 +6,69 @@ one-time setup below, so a release never fails just because a registrycredential has not been configured yet.Until that setup is done the release still runs: each package is built,-packed and `twine check`ed, and the job emits a warning explaining that the-upload was skipped. That way the pipeline cannot silently rot, which is the-failure mode that let `npx aisoc` sit in the README while no such package-existed.+packed and `twine check`ed, and nothing is uploaded. That way the pipeline+cannot silently rot, which is the failure mode that let `npx aisoc` sit in the+README while no such package existed.++## Telling an upload from a skip++A green publish job is not a published package. On the v12.2.0 release run all+59 jobs reported success — eight of them named `npm — publish <pkg>` or+`PyPI — publish <pkg>` — and all eight packages returned 404. Only the final+upload step is credential-gated, so each job built the artefact, reached the+upload, skipped it, and exited 0.++Three things now carry that fact to a reader, all from one decision made in+the `Packages — which uploads are armed` job:++1. **The job title.** With no credential the job is called+ `npm — pack only, NOT uploaded: aisoc`, not `npm — publish aisoc`. Visible+ in the Actions list without opening anything.+2. **The run summary.** `Packages — publication report` writes a table of+ every package, its version in the tree, whether the registry holds it, and+ the reason for each one that was packed and not uploaded.+3. **The registry, asked directly.** That same job runs+ `scripts/check_published_packages.py --require-network`, which is the only+ check here that does not take the workflow's word for anything.++The release still completes without a credential, because the images and the+GitHub Release genuinely do publish and must not be blocked by a package that+cannot.++### `.github/release-packages.yml`++Every package the release uploads is declared there, with `published: true`+or `false` and, when false, the reason. The gate enforces it in three+directions, so none of them can drift in silence:++- against `release.yml`'s npm and PyPI matrices, both ways — a package the+ workflow builds and the file does not declare is one nothing checks;+- `published: true` must resolve on the registry, **at the version in the+ tree**. This is the direction a release run cannot check itself: a job that+ claims an upload and performed none fails here;+- `published: false` must 404. When a credential arrives and an upload+ succeeds, the entry has to be corrected in a commit — the knowingly-absent+ set shrinks visibly rather than quietly ceasing to be true.++So the sequence when you finish the setup below is: publish, then flip that+package's `published` to `true` and delete its `reason`. The gate fails until+you do, and fails again if you do it early.++### Exercising the package half without cutting a tag++```bash+gh workflow run release.yml --ref main -f tag=v0.0.0 -f packages_only=true+```++Builds, packs and validates all eight packages, runs the publication report,+and skips the image rebuild entirely. Nothing is uploaded: the preflight+refuses to arm an upload on a dispatch however well credentialled the+repository is, because a registry will not accept a version twice. `tag` is+required by the form and ignored on this path.++Use it to check that every package is still publishable before tagging —+otherwise a packaging break is discovered on a tag push, which is the worst+possible moment.## What gets published@@ -180,9 +239,15 @@ gh workflow run release.yml --ref main -f tag=vX.Y.Z```Dispatch it from `main` so the current workflow definition runs; the build-checks the named tag out for its source. The GitHub Release, npm and PyPI jobs-are push-only and stay skipped, because a registry will not accept a package-version twice.+checks the named tag out for its source. The GitHub Release stays skipped, and+no package is uploaded, because a registry will not accept a package version+twice.++A package half that only partly published has no equivalent repair: npm and+PyPI both refuse a second upload of a version that already exists. Bump the+package's version and cut a new tag. `Packages — publication report` is what+tells you it happened — it fails the release when a package declared+published does not resolve at the version in the tree.## Verifying a publishdiff --git a/packages/aisoc-lite/README.md b/packages/aisoc-lite/README.mdindex 9cc1699d0..973f36431 100644--- a/packages/aisoc-lite/README.md+++ b/packages/aisoc-lite/README.md@@ -10,6 +10,13 @@ npx aisoc triage --demo✓ AiSOC triaged 200 alerts: 12 TP, 171 FP suppressed (85.5% noise), 17 need review — in 0.1s [deterministic · no LLM]```+**`aisoc` is ready, unpublished, so the command above does not resolve today.**+`release.yml` builds and packs it on every tag, but npm configures trusted+publishing on a package's settings page and a package that has never been+published has no settings page — the first upload needs a token, which is an+account action rather than a code change. Until then, run it from a clone:+`pnpm --filter aisoc build && node packages/aisoc-lite/dist/cli.js triage --demo`.+This is the zero-install front door to [AiSOC](https://github.com/beenuar/AiSOC), the open-source, self-hostable AI SOC. The full Docker stack is a ~3.5-minute commitment; this CLI gets you a verdict in under a minute.## Why it's trustworthydiff --git a/scripts/check_published_packages.py b/scripts/check_published_packages.pyindex e1cfb1e81..9bf89ade8 100755--- a/scripts/check_published_packages.py+++ b/scripts/check_published_packages.py@@ -1,25 +1,58 @@#!/usr/bin/env python3-"""The README's publishing claim must match what the registries actually hold.+"""Ask the registries what was published, rather than the workflow.-The packaging milestone has now moved three times — v8.0 to v8.1 to v8.2 —-for the same reason each time: the blocker is registry credentials, not code,-so it cannot be scheduled by writing a version number. v9.0 stops moving the-number and states a fact instead: the pipeline builds and packs every-package on each tag, and the upload is blocked on an account action.+Why this exists+---------------+On the v12.2.0 release run all 59 jobs reported success, eight of them named+`npm — publish …` or `PyPI — publish …`, and all eight packages returned 404.+Only the final upload step is credential-gated, so each job built the+artefact, reached the upload, skipped it, and exited 0. The signal said nine+packages shipped and nothing had.-A fact can go stale in a way a promise cannot. The moment somebody adds an-`NPM_TOKEN`, "ready, unpublished" becomes false, and the README would keep-saying it — the same failure as a promise that never lands, just pointing the-other way.+The credential gate is correct — the blocker is an account action, not code —+so the answer is not to publish but to stop inferring publication from a green+job. "The job ran" and "the registry holds the package" are different facts+and only the first was ever checked. This checks the second.-So this checks the claim against the registries. Offline it skips rather than-fails, because a contributor on a plane must not be told the README is wrong-when the truth is that the network is absent. It is a scheduled check, not a-per-PR blocker, for the same reason.+What it enforces+----------------+`.github/release-packages.yml` declares every package `release.yml` uploads+and whether each is expected to be on its registry. Three directions, because+a gate that only runs one way passes while drift accumulates in the other —+the recurring defect in this tree:-Usage:+ manifest <-> workflow Both ways. A package the npm/PyPI matrices build+ and the manifest does not declare is a package+ nothing will ever check; a package declared with no+ matrix entry is a claim with no pipeline behind it.++ published: true -> registry resolves, at the version in the tree. This is+ the direction a release run cannot check itself: a job+ that claims a publish and uploaded nothing fails here.++ published: false -> registry 404s. So the knowingly-absent set shrinks in a+ commit when a credential arrives, instead of quietly+ ceasing to be true. A stale `false` is as wrong as a+ stale `true`; it is just wrong in the flattering+ direction.++It also holds the prose to the same answer: `README.md`'s maturity row and+each package README's install command. Those used to be checked by a second+hardcoded package list in `tests/test_package_install_claims.py`, which could+disagree with this one. That test now imports this module, so there is one+list, one registry query, and one verdict.++Offline it skips rather than fails, because a contributor on a plane must not+be told the README is wrong when the truth is that the network is absent. The+structural checks above run first and do not need a network, so an offline run+still catches manifest/workflow drift.++Usage+-----python3 scripts/check_published_packages.pypython3 scripts/check_published_packages.py --require-network+ python3 scripts/check_published_packages.py --summary "$GITHUB_STEP_SUMMARY"+ python3 scripts/check_published_packages.py --self-test"""from __future__ import annotations@@ -27,11 +60,15 @@ from __future__ import annotationsimport argparseimport jsonimport sys+import tomllibimport urllib.errorimport urllib.parseimport urllib.request+from dataclasses import dataclassfrom pathlib import Path+import yaml+# `scripts/` is on sys.path when this file is run as a program, but not when a# test loads it by path with importlib. gate_toolkit sits beside it either way.sys.path.insert(0, str(Path(__file__).resolve().parent))@@ -42,22 +79,29 @@ self_test_if_requested(__file__)ROOT = repo_root()README = ROOT / "README.md"+MANIFEST = ROOT / ".github" / "release-packages.yml"+WORKFLOW = ROOT / ".github" / "workflows" / "release.yml"-#: The phrase the README uses while nothing is published.+#: The phrase README.md's maturity table uses while nothing is published.UNPUBLISHED_CLAIM = "Ready, unpublished"-#: (registry, name) for every package `release.yml` would upload.-PACKAGES: tuple[tuple[str, str], ...] = (- ("npm", "aisoc"),- ("npm", "@aisoc/sdk"),- ("npm", "@aisoc/mcp"),- ("pypi", "aisoc-sandbox"),- ("pypi", "aisoc-cli"),- ("pypi", "aisoc-sdk"),- ("pypi", "aisoc-plugin-sdk"),- ("pypi", "aisoc-detections"),+#: Wording that tells a reader, in the present tense, that the install command+#: beside it will not resolve yet. Any one satisfies the gate — the point is+#: that the reader is warned, not that a particular sentence is used.+DISCLAIMERS = (+ "not yet on pypi",+ "not yet on npm",+ "ready, unpublished",+ "will not resolve",+ "does **not** resolve",+ "does not resolve",)+#: The job ids whose matrices name what gets uploaded. Ids, not display names:+#: the display names now change with the credential state, which is the whole+#: point of the change that introduced this file.+MATRIX_JOBS = {"npm-publish": "npm", "pypi-publish": "pypi"}+_TIMEOUT = 10@@ -65,69 +109,297 @@ class Offline(RuntimeError):"""The registry could not be reached, which is not the same as absent."""-def _is_published(registry: str, name: str) -> bool:- url = f"https://registry.npmjs.org/{urllib.parse.quote(name, safe='@')}" if registry == "npm" else f"https://pypi.org/pypi/{name}/json"- request = urllib.request.Request(url, headers={"Accept": "application/json"})+@dataclass(frozen=True)+class Package:+ name: str+ registry: str+ directory: str+ published: bool+ reason: str+ install: str | None++ @property+ def slug(self) -> str:+ return f"{self.registry}:{self.name}"++ def readme(self, root: Path) -> Path:+ return root / self.directory / "README.md"++ def version(self, root: Path) -> str:+ """The version the release would upload, read from the package itself.++ Never copied into the manifest: a version in two places is a version+ that disagrees with itself.+ """+ directory = root / self.directory+ if self.registry == "npm":+ data = json.loads((directory / "package.json").read_text(encoding="utf-8"))+ return str(data["version"])+ data = tomllib.loads((directory / "pyproject.toml").read_text(encoding="utf-8"))+ project = data.get("project") or data.get("tool", {}).get("poetry", {})+ return str(project["version"])+++def load_manifest(root: Path = ROOT) -> list[Package]:+ """Every declared package, or a hard error — never an empty list.++ A gate that walks zero packages finds zero violations, and the clean+ result is indistinguishable from a wrong root or a deleted manifest.+ """+ path = root / ".github" / "release-packages.yml"+ if not path.is_file():+ raise SystemExit(f"{path} is missing; there is nothing to check publication against.")+ declared = yaml.safe_load(path.read_text(encoding="utf-8")) or {}+ entries = declared.get("packages") or []+ if not entries:+ raise SystemExit(f"{path} declares no packages; refusing to report a clean tree.")+ packages = []+ for entry in entries:+ packages.append(+ Package(+ name=str(entry["name"]),+ registry=str(entry["registry"]),+ directory=str(entry["directory"]),+ published=bool(entry["published"]),+ reason=" ".join(str(entry.get("reason", "")).split()),+ install=entry.get("install"),+ )+ )+ return packages+++def workflow_matrix(root: Path = ROOT) -> set[tuple[str, str, str]]:+ """(registry, package, directory) for every upload `release.yml` declares."""+ workflow = yaml.safe_load((root / ".github" / "workflows" / "release.yml").read_text(encoding="utf-8"))+ jobs = workflow.get("jobs") or {}+ found: set[tuple[str, str, str]] = set()+ for job_id, registry in MATRIX_JOBS.items():+ job = jobs.get(job_id)+ if job is None:+ raise SystemExit(f"release.yml no longer has a `{job_id}` job; this gate is checking a workflow that moved.")+ for entry in job["strategy"]["matrix"]["include"]:+ found.add((registry, str(entry["pkg"]), str(entry["dir"])))+ return found+++def _registry_url(registry: str, name: str) -> str:+ if registry == "npm":+ return f"https://registry.npmjs.org/{urllib.parse.quote(name, safe='@')}"+ return f"https://pypi.org/pypi/{name}/json"+++def registry_versions(registry: str, name: str) -> set[str] | None:+ """Every version on the registry, or None when the distribution is absent.++ Raises Offline for anything that is not a clean answer. "Unreachable" and+ "not published" must never collapse into the same result: one of them is+ a finding and the other is a contributor's train going into a tunnel.+ """+ request = urllib.request.Request(_registry_url(registry, name), headers={"Accept": "application/json"})try:with urllib.request.urlopen(request, timeout=_TIMEOUT) as response: # noqa: S310 — fixed https hosts- json.load(response)- return True+ payload = json.load(response)except urllib.error.HTTPError as exc:if exc.code == 404:- return False+ return Noneraise Offline(f"{registry}:{name} returned {exc.code}") from exc- except (urllib.error.URLError, TimeoutError, OSError) as exc:+ except (urllib.error.URLError, TimeoutError, OSError, json.JSONDecodeError) as exc:raise Offline(f"{registry}:{name} unreachable: {exc}") from exc+ versions = payload.get("versions") if registry == "npm" else payload.get("releases")+ return set(versions or {})+++def is_published(registry: str, name: str) -> bool:+ """Whether the distribution exists at all. Used by the README-claim tests."""+ return registry_versions(registry, name) is not None+++# --------------------------------------------------------------------------+# Checks+# --------------------------------------------------------------------------+++def _check_manifest_matches_workflow(packages: list[Package], root: Path) -> list[str]:+ declared = {(p.registry, p.name, p.directory) for p in packages}+ built = workflow_matrix(root)+ failures = []+ for registry, name, directory in sorted(built - declared):+ failures.append(+ f"release.yml uploads {registry}:{name} from {directory}, and "+ f".github/release-packages.yml does not declare it — so nothing checks whether it published."+ )+ for registry, name, directory in sorted(declared - built):+ failures.append(+ f".github/release-packages.yml declares {registry}:{name} from {directory}, and "+ f"release.yml's {registry} matrix does not build it — the declaration has no pipeline behind it."+ )+ return failures++def _check_directories_exist(packages: list[Package], root: Path) -> list[str]:+ failures = []+ for package in packages:+ try:+ package.version(root)+ except (OSError, KeyError, ValueError) as exc:+ failures.append(f"{package.slug}: cannot read a version from {package.directory} ({exc}).")+ return failures+++def _check_registry_state(packages: list[Package], observed: dict[str, set[str] | None], root: Path) -> list[str]:+ failures = []+ for package in packages:+ versions = observed[package.slug]+ if package.published and versions is None:+ failures.append(+ f"{package.slug} is declared published and the registry returns 404. A release job reported success and uploaded nothing."+ )+ elif package.published and versions is not None and package.version(root) not in versions:+ failures.append(+ f"{package.slug} is declared published, but version {package.version(root)} — the version in the tree — "+ f"is not on the registry (it holds {len(versions)} other version(s)). The last release did not upload it."+ )+ elif not package.published and versions is not None:+ failures.append(+ f"{package.slug} is declared unpublished and the registry resolves it. "+ f"Set `published: true` and drop the reason: the knowingly-absent set must shrink in a commit."+ )+ return failures+++def _check_prose(packages: list[Package], observed: dict[str, set[str] | None], root: Path) -> list[str]:+ """README.md's maturity row and each package README's install command."""+ failures = []+ live = [p.slug for p in packages if observed[p.slug] is not None]+ readme = (root / "README.md").read_text(encoding="utf-8")+ claims_unpublished = UNPUBLISHED_CLAIM.lower() in readme.lower()++ if live and claims_unpublished:+ failures.append(+ f"README.md still says {UNPUBLISHED_CLAIM!r}, but these are live: {', '.join(live)}. "+ f"Update the maturity table: the claim was true and is not any more."+ )+ if not live and not claims_unpublished:+ failures.append(+ f"Nothing is on npm or PyPI, and README.md no longer says {UNPUBLISHED_CLAIM!r}. A reader following it will hit a 404."+ )++ for package in packages:+ if not package.install or observed[package.slug] is not None:+ continue # nothing advertised, or it resolves and needs no caveat+ path = package.readme(root)+ if not path.is_file():+ continue+ text = path.read_text(encoding="utf-8")+ if package.install not in text:+ continue+ if not any(d in text.lower() for d in DISCLAIMERS):+ failures.append(+ f"{path.relative_to(root)} shows `{package.install}` and {package.slug} is not on its registry (404). "+ f"Say so in the present tense next to the command."+ )+ return failures+++# --------------------------------------------------------------------------+# Reporting+# --------------------------------------------------------------------------+++def summary_markdown(packages: list[Package], observed: dict[str, set[str] | None], root: Path) -> str:+ """The table a reader sees on the run page without opening a log."""+ uploaded = [p for p in packages if observed[p.slug] is not None]+ skipped = [p for p in packages if observed[p.slug] is None]++ lines = ["## Package publication", ""]+ lines.append(+ f"**{len(uploaded)} of {len(packages)} packages are on their registry.** "+ f"Asked of npm and PyPI directly, not inferred from a job status."+ )+ lines.append("")+ lines.append("| Package | Registry | Version in tree | On the registry | Outcome |")+ lines.append("| --- | --- | --- | --- | --- |")+ for package in packages:+ versions = observed[package.slug]+ version = package.version(root)+ if versions is None:+ state, outcome = "no — 404", "packed, **not uploaded**"+ elif version in versions:+ state, outcome = f"yes — {version}", "published"+ else:+ state, outcome = f"yes, but not {version}", "**this version was not uploaded**"+ lines.append(f"| `{package.name}` | {package.registry} | {version} | {state} | {outcome} |")-def main() -> int:+ if skipped:+ lines.append("")+ lines.append("### Why these were packed and not uploaded")+ lines.append("")+ for package in skipped:+ lines.append(f"- **`{package.name}`** ({package.registry}) — {package.reason}")+ lines.append("")+ lines.append(+ "Each of these was built, validated and packed, so the packaging cannot rot unnoticed. "+ "None of them was uploaded. Declared in `.github/release-packages.yml`."+ )+ return "\n".join(lines) + "\n"+++def main(argv: list[str] | None = None) -> int:parser = argparse.ArgumentParser(description=__doc__)parser.add_argument("--require-network",action="store_true",help="fail instead of skipping when a registry cannot be reached",)- args = parser.parse_args()+ parser.add_argument(+ "--summary",+ metavar="PATH",+ help="append a markdown publication table to PATH (use $GITHUB_STEP_SUMMARY)",+ )+ args = parser.parse_args(argv)- readme = README.read_text(encoding="utf-8")- claims_unpublished = UNPUBLISHED_CLAIM.lower() in readme.lower()+ packages = load_manifest(ROOT)+ print(f"Root: {ROOT}")+ print(f"Declared packages: {len(packages)} ({sum(p.published for p in packages)} expected published)")++ # Structural first: these need no network, so an offline run still catches+ # a manifest that has drifted from the workflow it describes.+ failures = _check_manifest_matches_workflow(packages, ROOT) + _check_directories_exist(packages, ROOT)+ if failures:+ _report(failures)+ return 1- published: list[str] = []- for registry, name in PACKAGES:+ observed: dict[str, set[str] | None] = {}+ for package in packages:try:- if _is_published(registry, name):- published.append(f"{registry}:{name}")+ observed[package.slug] = registry_versions(package.registry, package.name)except Offline as exc:if args.require_network:print(f"ERROR: {exc}", file=sys.stderr)return 1- print(f"SKIP: {exc}")+ print(f"SKIP: {exc} — structural checks passed; registry state not established.")return 0- if published and claims_unpublished:- print("PUBLISHED-PACKAGES GATE FAILED:", file=sys.stderr)- print(- f" - README still says {UNPUBLISHED_CLAIM!r}, but these are live: {', '.join(published)}.",- file=sys.stderr,- )- print(- " Update the maturity table: the claim was true and is not any more.",- file=sys.stderr,- )- return 1+ failures = _check_registry_state(packages, observed, ROOT) + _check_prose(packages, observed, ROOT)- if not published and not claims_unpublished:- print("PUBLISHED-PACKAGES GATE FAILED:", file=sys.stderr)- print(- f" - Nothing is on npm or PyPI, and the README no longer says {UNPUBLISHED_CLAIM!r}. A reader following it will hit a 404.",- file=sys.stderr,- )+ if args.summary:+ with open(args.summary, "a", encoding="utf-8") as handle:+ handle.write(summary_markdown(packages, observed, ROOT))++ if failures:+ _report(failures)return 1- state = f"{len(published)}/{len(PACKAGES)} published" if published else "none published"- print(f"OK: README matches registry state ({state}).")+ live = sum(1 for p in packages if observed[p.slug] is not None)+ print(f"OK: registry state matches the manifest ({live}/{len(packages)} published).")return 0+def _report(failures: list[str]) -> None:+ print("PUBLISHED-PACKAGES GATE FAILED:", file=sys.stderr)+ for failure in failures:+ print(f" - {failure}", file=sys.stderr)++if __name__ == "__main__":raise SystemExit(main())diff --git a/tests/test_package_install_claims.py b/tests/test_package_install_claims.pyindex 8ac961303..3f230d7ed 100644--- a/tests/test_package_install_claims.py+++ b/tests/test_package_install_claims.py@@ -1,9 +1,9 @@"""A package README must not imply an install command that cannot resolve.None of the eight first-party packages has ever been uploaded. The release-workflow's publish jobs report **success** anyway, because only the final+workflow's publish jobs reported **success** anyway, because only the finalupload step is credential-gated and the repository's sole secret is-`FLY_API_TOKEN` — so the job runs, skips the upload, and goes green. Asking+`FLY_API_TOKEN` — so each job ran, skipped the upload, and went green. Askingthe registries is the only way to know: all eight return 404.That is a deliberate, documented state and not a defect. The defect is what@@ -18,82 +18,108 @@ registry, the README that advertises it must say so in the present tense. Ifa package is later published, the gate stops requiring the disclaimer for itrather than needing to be edited — so publishing cannot leave a stale "notyet" behind either.++The package list, the registry query and the accepted wording all come from+`scripts/check_published_packages.py`, which reads+`.github/release-packages.yml`. This file used to carry its own copy of all+three, covering five of the eight packages — two descriptions of the same+fact, free to drift, and drifted: `aisoc`, `@aisoc/mcp` and `aisoc-detections`+were never checked here at all, and `packages/aisoc-lite/README.md` was+advertising `npx aisoc triage --demo` with no caveat as a result."""from __future__ import annotations-import json-import pathlib+import importlib.utilimport re-import urllib.error-import urllib.request+import sys+from pathlib import Pathimport pytest-REPO = pathlib.Path(__file__).resolve().parents[1]+REPO = Path(__file__).resolve().parents[1]+_GATE = REPO / "scripts" / "check_published_packages.py"+++def _load_gate():+ spec = importlib.util.spec_from_file_location("check_published_packages", _GATE)+ module = importlib.util.module_from_spec(spec)+ # Registered before execution: `@dataclass` resolves annotations through+ # `sys.modules[cls.__module__]`, and a module that is only ever a local+ # variable is not there, so the decorator raises during import.+ sys.modules[spec.name] = module+ spec.loader.exec_module(module)+ return module+++gate = _load_gate()-#: (README, registry, distribution name, the install command it advertises).-PACKAGES: tuple[tuple[str, str, str, str], ...] = (- ("packages/sdk-py/README.md", "pypi", "aisoc-sdk", "pip install aisoc-sdk"),- ("packages/aisoc-cli/README.md", "pypi", "aisoc-cli", "pip install aisoc-cli"),- ("packages/plugin-sdk-py/README.md", "pypi", "aisoc-plugin-sdk", "pip install aisoc-plugin-sdk"),- ("packages/aisoc-sandbox/README.md", "pypi", "aisoc-sandbox", "pip install aisoc-sandbox"),- ("packages/sdk-ts/README.md", "npm", "@aisoc/sdk", "npm install @aisoc/sdk"),-)+#: Every package the release workflow uploads, straight from the manifest.+PACKAGES = gate.load_manifest(REPO)-#: Wording that tells a reader, in the present tense, that the command above-#: will not resolve yet. Any one of these satisfies the gate — the point is-#: that the reader is warned, not that a particular sentence is used.-DISCLAIMERS = ("not yet on pypi", "not yet on npm", "ready, unpublished", "will not resolve")+#: Only the ones whose README advertises a command a reader could run.+ADVERTISED = [p for p in PACKAGES if p.install]#: A label naming a release that has already shipped. It reads as availability#: rather than as a caveat, which is precisely how these went stale.STALE_LABEL = re.compile(r"v\d+\.\d+\+?\s*\(once\b", re.I)-def _published(registry: str, name: str) -> bool:- url = f"https://pypi.org/pypi/{name}/json" if registry == "pypi" else f"https://registry.npmjs.org/{name.replace('/', '%2f')}"+def _published(package) -> bool:try:- with urllib.request.urlopen(url, timeout=20) as response: # noqa: S310 - fixed https hosts- return response.status == 200 and bool(json.loads(response.read() or b"{}"))- except urllib.error.HTTPError as exc:- if exc.code == 404:- return False- pytest.skip(f"{registry} answered {exc.code} for {name}; cannot establish publication")- except OSError as exc:- pytest.skip(f"no network to {registry} ({exc}); cannot establish publication")- return False--[email protected]("relative,registry,name,command", PACKAGES, ids=[p[2] for p in PACKAGES])-def test_an_unpublished_package_says_so(relative: str, registry: str, name: str, command: str) -> None:- readme = REPO / relative- assert readme.is_file(), f"{relative} is gone but still listed here"+ return bool(gate.is_published(package.registry, package.name))+ except gate.Offline as exc:+ # `pytest.skip` raises, but the gate module is loaded by path so its+ # exception type is untyped and the handler reads as falling through.+ pytest.skip(f"{exc}; cannot establish publication")+ raise # unreachable++[email protected]("package", ADVERTISED, ids=[p.name for p in ADVERTISED])+def test_an_unpublished_package_says_so(package) -> None:+ readme = package.readme(REPO)+ assert readme.is_file(), f"{package.directory} has no README but the manifest advertises `{package.install}`"text = readme.read_text(encoding="utf-8")- if command not in text:+ if package.install not in text:return # it does not advertise the command, so there is nothing to qualify- if _published(registry, name):+ if _published(package):return # the command resolves; no disclaimer needed, and none requiredlowered = text.lower()- assert any(d in lowered for d in DISCLAIMERS), (- f"{relative} shows `{command}` but {name} is not on {registry} (404). Say so in the present tense next to the command."+ assert any(d in lowered for d in gate.DISCLAIMERS), (+ f"{readme.relative_to(REPO)} shows `{package.install}` but {package.name} is not on "+ f"{package.registry} (404). Say so in the present tense next to the command.")[email protected]("relative", [p[0] for p in PACKAGES], ids=[p[2] for p in PACKAGES])-def test_no_readme_defers_to_a_release_that_already_shipped(relative: str) -> None:[email protected]("package", PACKAGES, ids=[p.name for p in PACKAGES])+def test_no_readme_defers_to_a_release_that_already_shipped(package) -> None:"""`v8.0+ (once ... lands on PyPI)` on a v12 tree is not a caveat.Checked separately from the disclaimer because the two fail differently:a missing disclaimer says nothing, while a stale one says something falsewith more confidence than saying nothing would."""- text = (REPO / relative).read_text(encoding="utf-8")- found = STALE_LABEL.search(text)+ readme = package.readme(REPO)+ if not readme.is_file():+ return+ found = STALE_LABEL.search(readme.read_text(encoding="utf-8"))assert not found, (- f"{relative} defers publication to {found.group(0)!r}, a release that has shipped. "+ f"{readme.relative_to(REPO)} defers publication to {found.group(0)!r}, a release that has shipped. "f"The tree is at v{(REPO / 'VERSION').read_text().strip()}.")+++def test_every_package_the_release_uploads_is_covered_here() -> None:+ """The list is derived, so a new package cannot slip past unnoticed.++ The direction that actually broke. A hardcoded list in this file agreed+ with itself for as long as nobody added a package to `release.yml`, and+ three were already missing from it when this was written.+ """+ declared = {(p.registry, p.name, p.directory) for p in PACKAGES}+ assert declared == gate.workflow_matrix(REPO), (+ "the manifest and release.yml's publish matrices disagree; scripts/check_published_packages.py names the difference"+ )diff --git a/tests/test_release_publish_signal.py b/tests/test_release_publish_signal.pynew file mode 100644index 000000000..3bc3e81a6--- /dev/null+++ b/tests/test_release_publish_signal.py@@ -0,0 +1,260 @@+"""A skipped upload must not read as a successful publish.++On the v12.2.0 release run all 59 jobs reported success. Eight of them were+named `npm — publish <pkg>` or `PyPI — publish <pkg>`, and all eight packages+returned 404 from their registry. Only the final upload step is credential-+gated, so each job built the artefact, reached the upload, skipped it on a+missing credential, and exited 0. Nothing was red, nothing was uploaded, and+the Actions page said otherwise.++The credential gate is correct and stays. What changed is that the decision is+made once, in `package-credentials`, and then named in three places a reader+can see: the job titles in the Actions list, the log line in each job, and the+run summary written by `package-report`.++This file holds that shape in place. Two kinds of check, because they fail+differently:++* the decision itself is *executed* — the preflight's shell block is lifted+ out of the workflow and run under bash with each combination of event and+ credential, the same way `test_release_channel_tags.py` runs the tag block.+ A copy of the logic would agree with itself while the workflow drifted.++* the wiring is *structural* — every step that can upload a package must be+ guarded by that decision, and the job titles must vary with it. That is the+ direction the original defect came from: the gate was right and nothing+ carried its answer anywhere a reader would look.+"""++from __future__ import annotations++import os+import subprocess+import tempfile+from pathlib import Path++import pytest+import yaml++ROOT = Path(__file__).resolve().parents[1]+WORKFLOW = ROOT / ".github" / "workflows" / "release.yml"++#: The one job that decides whether an upload can happen.+DECIDER = "package-credentials"++#: The jobs whose whole purpose is to ship a package.+PUBLISH_JOBS = ("npm-publish", "pypi-publish")+++def _workflow() -> dict:+ return yaml.safe_load(WORKFLOW.read_text(encoding="utf-8"))+++def _job(job_id: str) -> dict:+ jobs = _workflow()["jobs"]+ assert job_id in jobs, f"release.yml no longer has a `{job_id}` job; this test is checking a workflow that moved"+ return jobs[job_id]+++def _probe_script() -> str:+ step = next((s for s in _job(DECIDER)["steps"] if s.get("id") == "probe"), None)+ assert step is not None, f"`{DECIDER}` no longer has a `probe` step"+ return step["run"]+++def run_probe(*, event: str, npm_token: str = "", npm_trusted: str = "", pypi_trusted: str = "") -> dict[str, str]:+ """Execute the workflow's own credential block and return its outputs."""+ with tempfile.TemporaryDirectory() as tmp:+ output = Path(tmp) / "gh-output"+ summary = Path(tmp) / "gh-summary"+ output.touch()+ summary.touch()+ env = {+ **os.environ,+ "NPM_TOKEN": npm_token,+ "NPM_TRUSTED": npm_trusted,+ "PYPI_TRUSTED": pypi_trusted,+ # The workflow renders `github.event_name == 'push'` into this.+ "UPLOADS_POSSIBLE": "true" if event == "push" else "false",+ "GITHUB_OUTPUT": str(output),+ "GITHUB_STEP_SUMMARY": str(summary),+ }+ done = subprocess.run(["bash", "-c", _probe_script()], env=env, capture_output=True, text=True)+ assert done.returncode == 0, f"the credential block failed: {done.stderr}"+ pairs = dict(line.split("=", 1) for line in output.read_text(encoding="utf-8").splitlines() if "=" in line)+ pairs["_summary"] = summary.read_text(encoding="utf-8")+ return pairs+++# --------------------------------------------------------------------------+# The decision, executed+# --------------------------------------------------------------------------+++def test_no_credential_means_no_upload_on_a_tag_push() -> None:+ """The repository's state today: one secret, `FLY_API_TOKEN`, no variables."""+ decided = run_probe(event="push")+ assert decided["npm"] == "unarmed"+ assert decided["pypi"] == "unarmed"+++def test_an_npm_token_arms_npm_alone() -> None:+ decided = run_probe(event="push", npm_token="npm_xxx")+ assert decided["npm"] == "armed"+ assert decided["pypi"] == "unarmed", "a token for one registry must not arm the other"+++def test_trusted_publishing_arms_each_registry_independently() -> None:+ assert run_probe(event="push", npm_trusted="true")["npm"] == "armed"+ assert run_probe(event="push", pypi_trusted="true")["pypi"] == "armed"++[email protected]("credentials", [{}, {"npm_token": "npm_xxx"}, {"npm_trusted": "true", "pypi_trusted": "true"}])+def test_a_dispatched_run_never_uploads_however_well_credentialled(credentials: dict) -> None:+ """A registry refuses a version it already holds, so a republish cannot work.++ Worth executing rather than reading: the packages-only dispatch exists so+ the packaging path can be exercised on demand, and an exercise that could+ reach a real upload would be a footgun rather than a test.+ """+ decided = run_probe(event="workflow_dispatch", **credentials)+ assert decided["npm"] == "unarmed"+ assert decided["pypi"] == "unarmed"+++def test_the_summary_states_the_outcome_and_the_reason() -> None:+ summary = run_probe(event="push")["_summary"]+ assert "packs only" in summary+ assert "NPM_TOKEN" in summary and "AISOC_PYPI_TRUSTED_PUBLISHING" in summary, (+ "the run summary must name what is missing, or a reader learns only that something did not happen"+ )+ assert "uploads" not in summary.replace("Package uploads", ""), "the summary claimed an upload while both registries were unarmed"+++# --------------------------------------------------------------------------+# The wiring, structurally+# --------------------------------------------------------------------------+++def _upload_steps(job: dict) -> list[dict]:+ """Steps that can put a package on a registry."""+ found = []+ for step in job["steps"]:+ run = step.get("run") or ""+ uses = step.get("uses") or ""+ if "npm publish" in run or "gh-action-pypi-publish" in uses:+ found.append(step)+ return found++[email protected]("job_id", PUBLISH_JOBS)+def test_the_job_title_changes_with_the_credential_state(job_id: str) -> None:+ """The defect in one assertion: a constant title cannot report an outcome.++ `needs` is one of the few contexts a job `name:` can read, which is the+ whole reason the decision lives in its own job — `secrets` is unavailable+ in a job-level `name:` or `if:`.+ """+ name = _job(job_id)["name"]+ assert f"needs.{DECIDER}.outputs" in name, (+ f"`{job_id}` has a fixed title, so a run that uploaded nothing is indistinguishable "+ f"in the Actions list from one that published every package"+ )+ assert "NOT uploaded" in name, f"`{job_id}`'s unarmed title does not say that nothing was uploaded"++[email protected]("job_id", PUBLISH_JOBS)+def test_every_upload_step_is_guarded_by_that_same_decision(job_id: str) -> None:+ """A second upload path added later must not bypass the signal.++ The direction that matters: the title is driven by `package-credentials`,+ so an upload reachable without consulting it would publish under a title+ saying nothing was uploaded — the original defect, inverted.+ """+ job = _job(job_id)+ steps = _upload_steps(job)+ assert steps, f"`{job_id}` has no upload step at all; it cannot publish anything"+ for step in steps:+ condition = step.get("if") or ""+ assert f"needs.{DECIDER}.outputs" in condition, f"`{job_id}` step {step.get('name')!r} can upload without consulting `{DECIDER}`"++[email protected]("job_id", PUBLISH_JOBS)+def test_a_job_that_uploaded_nothing_says_so_in_its_log(job_id: str) -> None:+ job = _job(job_id)+ explains = [s for s in job["steps"] if "nothing was uploaded" in (s.get("name") or "").lower()]+ assert explains, f"`{job_id}` has no step that states a skipped upload"+ for step in explains:+ assert f"needs.{DECIDER}.outputs" in (step.get("if") or ""), (+ f"`{job_id}`'s explanation step is not tied to the credential decision, so it can fire on a run that did publish"+ )+++def test_the_report_asks_the_registry_rather_than_the_workflow() -> None:+ job = _job("package-report")+ runs = " ".join(s.get("run") or "" for s in job["steps"])+ assert "check_published_packages.py" in runs, "the publication report does not run the registry gate"+ assert "--require-network" in runs, "the report may skip when a registry is unreachable, which is the one question it exists to answer"+ assert "GITHUB_STEP_SUMMARY" in runs, "the report does not reach the run summary, so a reader must open a log"+++def test_the_report_survives_a_failed_package_build() -> None:+ """A report that only appears on the happy path is not a report."""+ condition = _job("package-report")["if"]+ assert "always()" in condition, "the publication report is skipped whenever a package job fails"+++def test_the_packages_only_input_exists_so_the_path_can_be_exercised() -> None:+ """`!inputs.packages_only` is vacuously true when no such input is declared.++ The same shape as the `promote_stable` check in+ `test_release_channel_tags.py`: a condition naming an undeclared input is+ a branch that can never be taken, and it looks identical to one that can.+ """+ workflow = _workflow()+ # PyYAML parses a bare `on:` key as the boolean True.+ triggers = workflow.get("on", workflow.get(True))+ assert triggers, "release.yml declares no triggers at all"+ assert "packages_only" in triggers["workflow_dispatch"]["inputs"]+ assert "inputs.packages_only" in workflow["jobs"]["docker-build"]["if"], (+ "a packages-only dispatch would still rebuild and re-push every image"+ )+ for job_id in (*PUBLISH_JOBS, DECIDER):+ reachable = _job(job_id)["if"]+ assert "inputs.packages_only" in reachable or "always()" in reachable, (+ f"`{job_id}` cannot run on a packages-only dispatch, so the path cannot be exercised before a tag"+ )+++# --------------------------------------------------------------------------+# The neighbouring skips found in the same audit+# --------------------------------------------------------------------------+++def test_a_release_cannot_ship_without_the_assets_it_advertises() -> None:+ """`action-gh-release` defaults `fail_on_unmatched_files` to false.++ A `files:` entry that matches nothing is dropped and the release is+ created without it, green — so the release page can advertise an SBOM and+ a checksum file it does not carry.+ """+ steps = _job("release")["steps"]+ create = next(s for s in steps if "action-gh-release" in (s.get("uses") or ""))+ assert create["with"].get("fail_on_unmatched_files") is True, "a missing release asset would be silently dropped from the release"+ assert any("advertised release asset" in (s.get("name") or "") for s in steps), (+ "nothing checks that the advertised assets exist before the release is created"+ )+++def test_the_chart_notice_about_an_unpushed_chart_is_reachable() -> None:+ """It was not: the job's own condition reduced to "this is a push".++ `release` runs only on a push, so `github.event_name == 'push' ||+ needs.release.result == 'success'` was true exactly when the event was a+ push — leaving a step conditioned on `github.event_name != 'push'` inside+ a job that could only run on one.+ """+ job = _job("chart-publish")+ assert "github.event_name" not in job["if"], "chart-publish is gated on the event again, which makes its not-a-push branch dead"+ notice = next((s for s in job["steps"] if "unpushed chart" in (s.get("name") or "")), None)+ assert notice is not None and "!= 'push'" in notice["if"]