Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

fix(security): split the CVE audit into shipped (blocking) and extras (advisory)

Nikhie4125514bbff
diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml
index f9c23c89..0ab20d81 100644
--- a/.github/workflows/security.yml
+++ b/.github/workflows/security.yml
@@ -56,34 +56,73 @@ jobs:
with:
python-version: "3.11"
- run: pip install pip-audit
- - name: Audit declared dependencies
+ - name: Audit shipped dependencies (core + [serve], blocking)
run: |
# Audit what the package DECLARES, not a local install. Installing torch from the
# PyTorch CPU index yields a local version (2.14.0+cpu) that does not exist on PyPI,
# so pip-audit cannot resolve it and --strict fails the job on that alone.
#
- # Every declared name, not just [project.dependencies]: the extras are what an
- # install actually pulls in. `Dockerfile` ends with `pip install ".[serve]"`, so
- # fastapi/uvicorn/python-multipart and their transitive closure ship in the image,
- # and an advisory in any of them would be invisible to a core-only audit. Read the
- # tables rather than transcribing a list, so an extra added later is audited by the
- # commit that adds it and not by a follow-up that forgets.
- python - <<'PY' > requirements-audit.txt
+ # The blocking scope is what the package and the image actually ship: [project.dependencies]
+ # plus the [serve] extra -- the Dockerfile ends with `pip install ".[serve]"`, so fastapi,
+ # uvicorn and python-multipart and their transitive closure run inside the published image,
+ # and an advisory in any of them has to fail every PR. Read the tables rather than
+ # transcribing a list, so an extra added later is audited by the commit that adds it.
+ #
+ # The remaining extras (langchain, crewai, llamaindex, ...) are audited separately, see
+ # the next step: a finding there is still reported, but it does not fail unrelated PRs.
+ python - <<'PY' > requirements-audit-core.txt
import tomllib
with open("pyproject.toml", "rb") as fh:
project = tomllib.load(fh)["project"]
specs = list(project.get("dependencies", []))
- for extra in project.get("optional-dependencies", {}).values():
- specs.extend(extra)
+ specs.extend(project.get("optional-dependencies", {}).get("serve", []))
seen = set()
- for spec in specs: # `langchain` and `langgraph` repeat two of these
+ for spec in specs:
if spec not in seen:
seen.add(spec)
print(spec)
PY
- cat requirements-audit.txt
- echo "auditing $(wc -l < requirements-audit.txt) declared names: the core plus every extra"
- pip-audit --strict --desc -r requirements-audit.txt
+ cat requirements-audit-core.txt
+ echo "strictly auditing $(wc -l < requirements-audit-core.txt) shipped names: the core plus [serve]"
+ pip-audit --strict --desc -r requirements-audit-core.txt
+
+ - name: Audit optional-integration dependencies (findings advisory, failures blocking)
+ # The integration extras pull transitives that ship nowhere: no laya surface runs a Chroma
+ # server or calls nltk's model save/load, and crewai pins json-repair below the patched
+ # release (#645). Those advisories still deserve eyes, so they are reported here as an
+ # annotation, but they must not turn every PR red -- a gate that is always red teaches
+ # everyone to skim past it, including the day one of these packages ships a fix (#646).
+ #
+ # Only FINDINGS are advisory. pip-audit exits non-zero both for findings and for a broken
+ # audit -- a specifier it cannot resolve, a resolver or tool crash -- and its report always
+ # carries the findings count. So a failure WITH a report is advisory; a failure WITHOUT one
+ # is the audit itself breaking, and that still fails the job exactly as it did before.
+ run: |
+ python - <<'PY' > requirements-audit-extras.txt
+ import tomllib
+ with open("pyproject.toml", "rb") as fh:
+ project = tomllib.load(fh)["project"]
+ seen = set()
+ for extra, specs in project.get("optional-dependencies", {}).items():
+ if extra == "serve":
+ continue # already in the blocking audit above
+ for spec in specs:
+ if spec not in seen: # `langchain` and `langgraph` repeat two of these
+ seen.add(spec)
+ print(spec)
+ PY
+ cat requirements-audit-extras.txt
+ echo "advisorially auditing $(wc -l < requirements-audit-extras.txt) optional-integration names"
+ # `set -e` is on: capture the exit code without tripping it, then decide below.
+ pip-audit --strict --desc -r requirements-audit-extras.txt > pip-audit-extras.log 2>&1 && code=0 || code=$?
+ cat pip-audit-extras.log
+ if [ "$code" -ne 0 ] && ! grep -q "known vulnerabilities" pip-audit-extras.log; then
+ echo "::error::the extras audit itself failed (exit $code) -- resolver or tooling error, still blocking"
+ exit "$code"
+ fi
+ if [ "$code" -ne 0 ]; then
+ echo "::warning::advisories in optional-integration dependencies -- visible here, not blocking PRs (see #645/#646)"
+ fi
codeql:
name: CodeQL
diff --git a/tests/test_audit_scope.py b/tests/test_audit_scope.py
index 9913858e..811013d3 100644
--- a/tests/test_audit_scope.py
+++ b/tests/test_audit_scope.py
@@ -1,12 +1,18 @@
"""The CVE audit job must see every name the package declares, not just the core five.
-`.github/workflows/security.yml` builds `requirements-audit.txt` from `pyproject.toml` and hands it
-to `pip-audit`. For a while it did that with a regex anchored on `dependencies = [`, which matches
-the `[project]` array and none of `[project.optional-dependencies]` -- so the ten names behind the
+`.github/workflows/security.yml` builds its audit lists from `pyproject.toml` and hands them to
+`pip-audit`. For a while it did that with a regex anchored on `dependencies = [`, which matches the
+`[project]` array and none of `[project.optional-dependencies]` -- so the ten names behind the
`serve`, `fast`, `mcp`, `structured`, `onnx`, `langchain` and `langgraph` extras, and the 57
transitive names they pull in, were outside the audit. `Dockerfile` ends with
`pip install ".[serve]" && pip check`: the HTTP server's dependencies ship in the published image.
+Since #645/#646 the audit is split in two. The blocking scope is what ships: the core plus the
+`[serve]` extra, audited strictly, where a finding fails the job. The remaining optional
+integrations are audited in a second, advisory step: findings are reported as an annotation but do
+not fail unrelated PRs, while a failure of the audit itself -- a specifier pip-audit cannot
+resolve, a resolver or tool crash -- still does.
+
Text parsing, not tomllib: the floor is 3.10 and tomllib arrives in 3.11, same as
`tests/test_packaging.py`. The job that runs the extractor pins `python-version: "3.11"`, which is
checked here, and the lifted code is only executed where tomllib exists -- on 3.10 that check
@@ -80,80 +86,124 @@ if len(od) > 1:
check_true("pyproject/core parses", len(core) == 5, core)
check("pyproject/extra tables found", sorted(extras),
- ["crewai", "fast", "langchain", "langgraph", "llamaindex", "mcp", "onnx", "serve",
- "structured"])
+ ["crewai", "fast", "langchain", "langgraph", "llamaindex", "mcp", "onnx", "serve",
+ "structured"])
declared = list(core) + [s for names in extras.values() for s in names]
expected = sorted(set(declared))
check("pyproject/extras add names the core does not have",
- sorted({re.split(r"[<>=!;\[ ]", s)[0] for s in declared}
- - {re.split(r"[<>=!;\[ ]", s)[0] for s in core}),
- ["crewai", "fastapi", "langchain-core", "langgraph", "llama-index-core", "mcp", "onnx",
- "onnxruntime", "onnxscript", "pydantic", "python-multipart", "tilelang", "uvicorn"])
-
-# ---------------------------------------------------------------- the job's own extractor
+ sorted({re.split(r"[<>=!;\[ ]", s)[0] for s in declared}
+ - {re.split(r"[<>=!;\[ ]", s)[0] for s in core}),
+ ["crewai", "fastapi", "langchain-core", "langgraph", "llama-index-core", "mcp", "onnx",
+ "onnxruntime", "onnxscript", "pydantic", "python-multipart", "tilelang", "uvicorn"])
+
+# The two scopes: shipped (blocking) is the core plus [serve]; everything else is advisory.
+shipped = sorted(set(core) | set(extras.get("serve", [])))
+advisory = sorted(set(declared) - set(shipped))
+check_true("pyproject/scope split covers every declared name",
+ sorted(set(shipped) | set(advisory)) == expected and not set(shipped) & set(advisory),
+ "shipped and advisory must partition the declared specs")
+
+# ---------------------------------------------------------------- the job's own extractors
deps_job = "\n".join(section(read(os.path.join(".github", "workflows", "security.yml")),
" deps:"))
check_true("security.yml/has a deps job", "pip-audit" in deps_job, deps_job[:80])
-m = re.search(r"python - <<'PY' > requirements-audit\.txt\n(.*?)\n\s*PY\n", deps_job, re.S)
-check_true("security.yml/exactly one heredoc writes the audit list",
- m is not None and len(re.findall(r"<<'PY'", deps_job)) == 1,
- "the extractor has to be liftable as text for this file to check it")
-extractor = ""
-if m:
+
+def lift_extractor(filename):
+ """The heredoc that writes one audit list, dedented so it can run."""
+ m = re.search(r"python - <<'PY' > %s\n(.*?)\n\s*PY\n" % re.escape(filename), deps_job, re.S)
+ if not m:
+ return ""
body = m.group(1)
indents = [len(l) - len(l.lstrip()) for l in body.splitlines() if l.strip()]
- extractor = "\n".join(l[min(indents):] if l.strip() else "" for l in body.splitlines())
+ return "\n".join(l[min(indents):] if l.strip() else "" for l in body.splitlines())
+
-# Why the extractor is read out of the workflow instead of re-implemented here: the failure mode
+check_true("security.yml/exactly two heredocs write the audit lists",
+ len(re.findall(r"<<'PY'", deps_job)) == 2
+ and "> requirements-audit-core.txt" in deps_job
+ and "> requirements-audit-extras.txt" in deps_job,
+ "one extractor per scope; any other count means a scope silently went unaudited")
+core_extractor = lift_extractor("requirements-audit-core.txt")
+extras_extractor = lift_extractor("requirements-audit-extras.txt")
+
+# Why the extractors are read out of the workflow instead of re-implemented here: the failure mode
# this file exists for is the two surfaces drifting, so a check that carries its own copy of the
# parsing rules would drift in exactly the same direction and stay green.
-check_true("extractor/reads the tables with tomllib, not a regex on one array",
- "tomllib" in extractor and "dependencies = \\[" not in extractor, extractor[:200])
-check_true("extractor/walks optional-dependencies",
- "optional-dependencies" in extractor, extractor[:200])
+for name, extractor in (("core", core_extractor), ("extras", extras_extractor)):
+ check_true("extractor/%s reads the tables with tomllib, not a regex on one array" % name,
+ "tomllib" in extractor and "dependencies = \\[" not in extractor, extractor[:200])
+ check_true("extractor/%s walks optional-dependencies" % name,
+ "optional-dependencies" in extractor, extractor[:200])
check_true("extractor/keeps the declared-not-installed reason",
"local version" in deps_job and "+cpu" in deps_job,
"auditing an install would trip --strict on torch's 2.14.0+cpu")
-check_true("extractor/still audits the file strictly",
- re.search(r"pip-audit --strict[^\n]*-r requirements-audit\.txt", deps_job) is not None,
- deps_job[:120])
+check_true("extractor/audits both files strictly",
+ re.search(r"pip-audit --strict[^\n]*-r requirements-audit-core\.txt", deps_job)
+ and re.search(r"pip-audit --strict[^\n]*-r requirements-audit-extras\.txt", deps_job),
+ "advisorial applies to findings, not to how the audit runs")
check_true("extractor/prints the scope it audited",
- re.search(r'echo "[^"]*requirements-audit\.txt', deps_job) is not None,
- "when the job goes red the log has to say how many names were in scope")
+ re.search(r'echo "[^"]*requirements-audit-core\.txt', deps_job) is not None
+ and re.search(r'echo "[^"]*requirements-audit-extras\.txt', deps_job) is not None,
+ "when a scope goes red the log has to say how many names were in it")
pin = re.search(r"python-version:\s*[\"'](\d+)\.(\d+)[\"']", deps_job)
check_true("security.yml/the deps job pins python >= 3.11",
pin is not None and tuple(map(int, pin.groups())) >= (3, 11),
- "tomllib, which the extractor reads the tables with, is 3.11+")
+ "tomllib, which the extractors read the tables with, is 3.11+")
+
+# Findings advisory, failures blocking: no blanket continue-on-error, and the step must tell a
+# findings report (its "known vulnerabilities" count) apart from the audit itself breaking.
+check_true("security.yml/extras findings are advisory without swallowing tooling errors",
+ "continue-on-error" not in deps_job
+ and "known vulnerabilities" in deps_job
+ and "::warning::" in deps_job
+ and "::error::" in deps_job,
+ "#646: a findings report warns, a broken audit (no report) must still fail the job")
ran = False
-if sys.version_info >= (3, 11) and extractor:
+if sys.version_info >= (3, 11) and core_extractor and extras_extractor:
ran = True
- run = subprocess.run([sys.executable, "-c", extractor], cwd=ROOT,
- capture_output=True, text=True)
- check_true("extractor/runs against the real pyproject.toml",
- run.returncode == 0, run.stderr.strip()[:300])
- emitted = [l.strip() for l in run.stdout.splitlines() if l.strip()]
- check("extractor/emits every declared name, core and extras alike", sorted(set(emitted)),
- expected)
+
+ def run_extractor(source):
+ return subprocess.run([sys.executable, "-c", source], cwd=ROOT,
+ capture_output=True, text=True)
+
+ core_run = run_extractor(core_extractor)
+ check_true("extractor/core runs against the real pyproject.toml",
+ core_run.returncode == 0, core_run.stderr.strip()[:300])
+ extras_run = run_extractor(extras_extractor)
+ check_true("extractor/extras runs against the real pyproject.toml",
+ extras_run.returncode == 0, extras_run.stderr.strip()[:300])
+
+ core_emitted = [l.strip() for l in core_run.stdout.splitlines() if l.strip()]
+ extras_emitted = [l.strip() for l in extras_run.stdout.splitlines() if l.strip()]
+ check("extractor/core emits the shipped names, [serve] included", sorted(set(core_emitted)),
+ shipped)
+ check("extractor/extras emits every remaining declared name", sorted(set(extras_emitted)),
+ advisory)
# Specifiers carried through, not stripped: `fastapi` alone would audit whatever PyPI serves
# today and hide the floor the package actually supports.
check_true("extractor/preserves each version specifier",
- all(e in declared for e in emitted),
- [e for e in emitted if e not in declared])
+ all(e in declared for e in core_emitted + extras_emitted),
+ [e for e in core_emitted + extras_emitted if e not in declared])
check_true("extractor/collapses the langchain/langgraph duplicate",
- len(emitted) == len(set(emitted)),
- [s for s in emitted if emitted.count(s) > 1])
+ len(core_emitted) == len(set(core_emitted))
+ and len(extras_emitted) == len(set(extras_emitted)),
+ [s for s in core_emitted + extras_emitted
+ if (core_emitted + extras_emitted).count(s) > 1])
+ check_true("extractor/the scopes do not double-audit a name",
+ not set(core_emitted) & set(extras_emitted),
+ sorted(set(core_emitted) & set(extras_emitted)))
for name in ("fastapi", "uvicorn", "python-multipart"):
- check_true("extractor/audits %s, which the image installs" % name,
- any(re.split(r"[<>=!;\[ ]", s)[0] == name for s in emitted),
+ check_true("extractor/audits %s in the blocking scope, which the image installs" % name,
+ any(re.split(r"[<>=!;\[ ]", s)[0] == name for s in core_emitted),
"Dockerfile installs it through the `serve` extra")
else:
- NOTES.append("python %d.%d has no tomllib: the extractor was not executed here, only its "
+ NOTES.append("python %d.%d has no tomllib: the extractors were not executed here, only their "
"source was checked (the job pins 3.11+)" % sys.version_info[:2])
-# Where the interpreter can run the extractor, it must have. Without this the skip above is a
-# hole a mutant can open -- `if False` would leave the file green with seven checks missing.
+# Where the interpreter can run the extractors, it must have. Without this the skip above is a
+# hole a mutant can open -- `if False` would leave the file green with checks missing.
check_true("extractor/ran wherever tomllib exists",
ran or sys.version_info < (3, 11), "the executable checks were skipped on a 3.11+ run")