Skip to content

Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

fix(license): parse SPDX license expressions (#295)

Narendran Raghavanc61fa9715e5e
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 2b51d087..9ee8fc65 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -473,6 +473,25 @@ All notable changes to SkillEvaluator are documented in this file.
boundary prevents quadratic rescanning of long excluded addresses. Ordinary
addresses on the same line and domains that merely resemble placeholders remain detectable
([#244](https://github.com/NVIDIA/SkillEvaluator/issues/244)).
+- The license check now parses SPDX license expressions instead of splitting
+ them on operators. Expressions with unbalanced parentheses, such as
+ `(MIT OR Apache-2.0`, were allowed, and dangling or repeated operators were
+ only sent to review; both now fail with a HIGH
+ `unsupported_license_expression` finding. A malformed value without an
+ upper-case `AND`, `OR` or `WITH`, such as `mit or (apache-2.0`, is reviewed
+ as an unknown license name instead of being allowed, and `(MIT)` is allowed
+ as MIT. A blocked license still fails a value that does not parse, such as
+ `GPL-3.0 (or later)` or `(GPL-3.0 OR MIT`. The `+`, `-only` and `-or-later`
+ forms of blocked licenses, such as `GPL-3.0+`, `AGPL-3.0+` and
+ `LGPL-2.1-only`, and deprecated ids such as
+ `GPL-2.0-with-classpath-exception` are blocked instead of passing as unknown
+ licenses. `Apache-2.0 WITH LLVM-exception` is allowed through a
+ new `allowed_license_exceptions` list, while an unlisted exception such as
+ `Commons-Clause` still needs review. An allowed frontmatter expression that
+ leaves out the allowed license found in a LICENSE file is now a conflict, as
+ a single identifier already was. SPDX headers inside Markdown inline code or
+ table cells no longer keep the closing backtick or `|`, and a trailing `;` is
+ dropped. Names such as `MIT License` are matched as before.
- A skill that exceeds a Tier 2 context-deduplication work limit (more than
512 content chunks, too much pairwise comparison work, or an LLM-review limit:
diff --git a/docs/tier1-validation.mdx b/docs/tier1-validation.mdx
index c9b54d81..f403d37c 100644
--- a/docs/tier1-validation.mdx
+++ b/docs/tier1-validation.mdx
@@ -211,6 +211,8 @@ For `validate`, the default set is `schema`, `version`, `security`, `pii`, `lice
<Accordion title="license — compliance">
Looks for license evidence in frontmatter, common license files, and SPDX headers. Detected identifiers are normalized and matched against the packaged permissive allowlist and restrictive blocklist. Missing or unknown licenses are reported for review; blocked licenses fail according to the active profile (CRITICAL under the default `external` profile).
+
+ Frontmatter values and SPDX headers can be SPDX license expressions with `AND`, `OR`, `WITH`, and parentheses, such as `MIT OR Apache-2.0`. An expression passes only when every license in it is allowed, and any blocked license fails it. The `+`, `-only`, and `-or-later` forms of a blocked license, such as `GPL-3.0+` or `LGPL-2.1-only`, and deprecated ids such as `GPL-2.0-with-classpath-exception` are blocked too. A `WITH` exception keeps an allowed license allowed only when it is in the packaged `allowed_license_exceptions` list (`LLVM-exception`, `Swift-exception`); other exceptions are reported for review. An expression that uses an upper-case `AND`, `OR`, or `WITH` but has unbalanced parentheses or a missing or repeated operator fails with an `unsupported_license_expression` finding. A value without those operators, such as `MIT License` or `MIT License (MIT)`, is treated as a license name: it is matched by name as before or reported for review. A blocked license fails the value even when it does not parse, so `GPL-3.0 (or later)` and `(GPL-3.0 OR MIT` are blocked.
</Accordion>
<Accordion title="code-integrity — code risk, secrets, and hygiene">
diff --git a/src/skillevaluator/config/__init__.py b/src/skillevaluator/config/__init__.py
index 9d1667fc..ca5ec5df 100644
--- a/src/skillevaluator/config/__init__.py
+++ b/src/skillevaluator/config/__init__.py
@@ -24,6 +24,7 @@ def load_license_config() -> dict:
dict containing:
- allowed_licenses: List of SPDX identifiers for permissive licenses
- blocked_licenses: List of SPDX identifiers for restrictive licenses
+ - allowed_license_exceptions: List of SPDX exceptions allowed after WITH
- license_patterns: Dict mapping license IDs to detection patterns
- proprietary_indicators: List of strings indicating proprietary content
- spdx_detection: Config for SPDX header scanning
diff --git a/src/skillevaluator/config/license_config.yaml b/src/skillevaluator/config/license_config.yaml
index 160123b6..22c26fde 100644
--- a/src/skillevaluator/config/license_config.yaml
+++ b/src/skillevaluator/config/license_config.yaml
@@ -10,6 +10,7 @@
# Keys:
# - allowed_licenses: Permissive licenses that are safe to ingest
# - blocked_licenses: Restrictive/proprietary licenses that should be rejected
+# - allowed_license_exceptions: SPDX exceptions allowed after WITH on an allowed license
# - license_patterns: Regex patterns to detect licenses from LICENSE file content
# - spdx_header_pattern: Pattern to match SPDX-License-Identifier headers in source files
@@ -52,6 +53,18 @@ allowed_licenses:
- PSF-2.0 # Python Software Foundation
- Python-2.0
+# =============================================================================
+# ALLOWED LICENSE EXCEPTIONS
+# =============================================================================
+# SPDX exceptions only add permissions to the license they follow, so
+# "Apache-2.0 WITH LLVM-exception" is allowed. Any other exception after an
+# allowed license needs manual review, and a blocked license stays blocked
+# whatever exception follows it.
+
+allowed_license_exceptions:
+ - LLVM-exception
+ - Swift-exception
+
# =============================================================================
# BLOCKED LICENSES (Restrictive/Copyleft/Proprietary)
# =============================================================================
diff --git a/src/skillevaluator/validators/license.py b/src/skillevaluator/validators/license.py
index f271ed72..45a4564f 100644
--- a/src/skillevaluator/validators/license.py
+++ b/src/skillevaluator/validators/license.py
@@ -12,7 +12,7 @@ Validates license compliance for Skills, Rules, and Workflows using multi-tier d
from __future__ import annotations
import re
-from dataclasses import dataclass
+from dataclasses import dataclass, replace
from functools import cached_property
from itertools import islice
from pathlib import Path
@@ -34,17 +34,117 @@ logger = get_logger(__name__)
_SPDX_PATTERN = re.compile(SPDX_LICENSE_PATTERN, re.IGNORECASE)
_LICENSE_SUFFIX_PATTERN = re.compile(r"(-license|-licence)$")
_THE_PREFIX_PATTERN = re.compile(r"^(the-)?")
-_SPDX_EXPRESSION_SPLIT = re.compile(r"\s+(?:OR|AND|WITH)\s+", re.IGNORECASE)
+_SPDX_EXPRESSION_TOKEN = re.compile(r"[()]|[^\s()]+")
+_SPDX_OPERATORS = frozenset({"AND", "OR", "WITH"})
+_SPDX_VERSION_VARIANT_SUFFIX = re.compile(r"-(?:only|or-later)$")
+# Parser states where a license name or "(" comes next, and states where the
+# expression may end or continue with AND, OR or ")".
+_EXPRESSION_OPERAND_STATES = frozenset({"start", "(", "AND", "OR"})
+_EXPRESSION_COMPLETE_STATES = frozenset({"license", "exception", ")"})
-def _strip_spdx_capture(raw: str) -> str:
- """Drop inline comment terminators and trailing source after the SPDX value."""
+def _strip_spdx_capture(raw: str, line_prefix: str = "") -> str:
+ """Drop inline comment terminators and trailing source after the SPDX value.
+
+ A backtick or ``|`` ends the value only when the same delimiter opens before
+ the tag on that line (Markdown inline code or a table cell), so a header such
+ as ``MIT | GPL-3.0`` is not shortened to ``MIT``.
+ """
raw = raw.split("#", 1)[0]
- for marker in ("*/", "-->"):
+ markers = ["*/", "-->", *(delimiter for delimiter in ("`", "|") if delimiter in line_prefix)]
+ for marker in markers:
idx = raw.find(marker)
if idx != -1:
raw = raw[:idx]
- return raw.strip()
+ return raw.strip().rstrip(";").rstrip()
+
+
+@dataclass(slots=True, frozen=True)
+class _LicenseExpression:
+ """License names and ``WITH`` exceptions of a license value, or its syntax error.
+
+ ``names`` holds every name written in the value, whatever its position and
+ even when the value does not parse, so the blocked check stays fail-closed.
+ """
+
+ licenses: tuple[str, ...] = ()
+ exceptions: tuple[tuple[str, str], ...] = ()
+ error: str | None = None
+ names: tuple[str, ...] = ()
+
+
+def _parse_license_expression(value: str) -> _LicenseExpression:
+ """Check SPDX expression syntax and collect the names it references.
+
+ Supports ``AND``, ``OR`` and ``WITH`` (matched case-insensitively) and
+ parentheses. Adjacent words without an operator between them form one name,
+ so free-text values such as ``MIT License`` keep the existing name matching.
+ Misplaced operators and unbalanced parentheses are syntax errors when the
+ value uses an SPDX operator (upper case, as the SPDX specification writes
+ them); otherwise the whole value is one free-text name, such as
+ ``MIT License (MIT)``, and goes to manual review like other unknown names.
+ The names between operators and parentheses are kept in ``names`` either
+ way, so a blocked license in ``GPL-3.0 (or later)`` still blocks the value.
+ """
+ items: list[tuple[str, str]] = []
+ words: list[str] = []
+ for token in _SPDX_EXPRESSION_TOKEN.findall(value):
+ keyword = token.upper()
+ if token in ("(", ")") or keyword in _SPDX_OPERATORS:
+ if words:
+ items.append(("name", " ".join(words)))
+ words = []
+ items.append((keyword, token))
+ else:
+ words.append(token)
+ if words:
+ items.append(("name", " ".join(words)))
+
+ names = tuple(text for kind, text in items if kind == "name")
+ expression = _parse_expression_items(items)
+ if expression.error and not any(text in _SPDX_OPERATORS for _kind, text in items):
+ whole = value.strip()
+ return _LicenseExpression((whole,), names=(whole, *names))
+ return replace(expression, names=names)
+
+
+def _parse_expression_items(items: list[tuple[str, str]]) -> _LicenseExpression:
+ """Run the SPDX expression grammar over grouped names, operators and parentheses."""
+ licenses: list[str] = []
+ exceptions: list[tuple[str, str]] = []
+ depth = 0
+ state = "start"
+ for kind, text in items:
+ if kind == "name" and state == "WITH":
+ exceptions.append((licenses[-1], text))
+ state = "exception"
+ elif kind == "name" and state in _EXPRESSION_OPERAND_STATES:
+ licenses.append(text)
+ state = "license"
+ elif kind == "name" or (kind == "(" and state not in _EXPRESSION_OPERAND_STATES):
+ return _LicenseExpression(error=f"missing operator before '{text}'")
+ elif kind == "(":
+ depth += 1
+ state = "("
+ elif kind == "WITH" and state != "license":
+ return _LicenseExpression(error="WITH must follow a single license identifier")
+ elif kind == "WITH":
+ state = "WITH"
+ elif state not in _EXPRESSION_COMPLETE_STATES:
+ return _LicenseExpression(error=f"missing license identifier before '{text}'")
+ elif kind == ")" and depth == 0:
+ return _LicenseExpression(error="unbalanced parenthesis")
+ elif kind == ")":
+ depth -= 1
+ state = ")"
+ else:
+ state = kind
+ if depth:
+ return _LicenseExpression(error="unbalanced parenthesis")
+ if items and state not in _EXPRESSION_COMPLETE_STATES:
+ missing = "exception" if state == "WITH" else "license"
+ return _LicenseExpression(error=f"missing {missing} identifier after '{items[-1][1]}'")
+ return _LicenseExpression(tuple(licenses), tuple(exceptions))
# File reference indicators in license field values
@@ -92,6 +192,7 @@ class LicenseValidator(ValidatorBase):
The validator uses skillevaluator/config/license_config.yaml which defines:
- allowed_licenses: Permissive licenses that pass validation
- blocked_licenses: Restrictive licenses that fail validation
+ - allowed_license_exceptions: SPDX exceptions allowed after WITH
- license_patterns: Patterns to detect licenses from file content
- proprietary_indicators: Strings indicating proprietary licensing
"""
@@ -122,6 +223,13 @@ class LicenseValidator(ValidatorBase):
"""Pre-compute normalized blocklist for O(1) lookups."""
return frozenset(self._normalize_license_id(lic) for lic in self.config.get("blocked_licenses", []))
+ @cached_property
+ def _normalized_exception_allowlist(self) -> frozenset[str]:
+ """Pre-compute normalized SPDX exceptions that may follow an allowed license."""
+ return frozenset(
+ self._normalize_license_id(exception) for exception in self.config.get("allowed_license_exceptions", [])
+ )
+
@property
def name(self) -> str:
return "License Compliance"
@@ -305,15 +413,44 @@ class LicenseValidator(ValidatorBase):
return matches[0] if matches else None
def _license_bucket(self, license_id: str | None) -> str:
- """Classify a license id as allowed, blocked, or unknown."""
+ """Classify a license value as allowed, blocked, unknown, or unsupported syntax."""
if not license_id:
return "unknown"
+ return self._license_policy(_parse_license_expression(license_id))[0]
+
+ def _license_policy(self, expression: _LicenseExpression) -> tuple[str, str | None]:
+ """Return the policy status of a parsed expression and the name that decided it.
+
+ Any blocked name blocks the expression, even one that is not valid SPDX
+ syntax. Otherwise a syntax error makes it unsupported. It is allowed only
+ when every license is allowed and every ``WITH`` exception is listed.
+ """
+ if blocked := next((name for name in expression.names if self._is_blocked_license(name)), None):
+ return "blocked", blocked
+ if expression.error:
+ return "unsupported", None
+ for name in expression.licenses:
+ if self._normalize_license_id(name) not in self._normalized_allowlist:
+ return "unknown", name
+ for license_id, exception in expression.exceptions:
+ if self._normalize_license_id(exception) not in self._normalized_exception_allowlist:
+ return "unknown", f"{license_id} WITH {exception}"
+ return "allowed", None
+
+ def _is_blocked_license(self, license_id: str) -> bool:
+ """Return True for a blocked id or one of its SPDX variants.
+
+ Variants are the ``-only``/``-or-later``/``+`` forms and the deprecated
+ ids with a built-in exception, such as ``GPL-2.0-with-classpath-exception``.
+ """
normalized = self._normalize_license_id(license_id)
- if normalized in self._normalized_blocklist:
- return "blocked"
- if normalized in self._normalized_allowlist:
- return "allowed"
- return "unknown"
+ base, separator, exception = normalized.partition("-with-")
+ if not (separator and exception.endswith("-exception")):
+ base = normalized
+ return any(
+ candidate in self._normalized_blocklist
+ for candidate in (normalized, base, _SPDX_VERSION_VARIANT_SUFFIX.sub("", base))
+ )
def _record_unrecognized_license_file(
self,
@@ -395,18 +532,21 @@ class LicenseValidator(ValidatorBase):
)
result.add_message(f"Tier 2: Detected {chosen.license_id} from {chosen.file_path}")
return chosen
- if self._license_bucket(frontmatter.license_id) == "blocked":
+ if self._license_bucket(frontmatter.license_id) in {"blocked", "unsupported"}:
return frontmatter
file_by_id: dict[str, LicenseDetection] = {}
for item in scan.detections:
if item.license_id:
file_by_id.setdefault(self._normalize_license_id(item.license_id), item)
- fm_norm = self._normalize_license_id(frontmatter.license_id or "")
- if set(file_by_id) == {fm_norm}:
+ declared = {
+ self._normalize_license_id(name)
+ for name in _parse_license_expression(frontmatter.license_id or "").licenses
+ }
+ if set(file_by_id) <= declared:
return frontmatter
- other = next((item for key, item in file_by_id.items() if key != fm_norm), None)
+ other = next((item for key, item in file_by_id.items() if key not in declared), None)
if (
other is not None
and self._license_bucket(frontmatter.license_id) == "allowed"
@@ -530,18 +670,13 @@ class LicenseValidator(ValidatorBase):
with file_path.open(encoding="utf-8", errors="ignore") as f:
header = "".join(islice(f, LICENSE_HEADER_SCAN_LINES))
if match := _SPDX_PATTERN.search(header):
- raw = _strip_spdx_capture(match.group(1))
+ line_start = header.rfind("\n", 0, match.start()) + 1
+ raw = _strip_spdx_capture(match.group(1), header[line_start : match.start()])
return raw or None
except Exception as e:
logger.debug("Could not scan %s: %s", file_path, e)
return None
- @staticmethod
- def _expression_symbols(license_id: str) -> list[str]:
- """Split an SPDX expression into identifiers, keeping hyphenated ids intact."""
- stripped = license_id.replace("(", " ").replace(")", " ").strip()
- return [part.strip() for part in _SPDX_EXPRESSION_SPLIT.split(stripped) if part.strip()]
-
def _validate_license(self, detection: LicenseDetection, result: ValidationResult) -> None:
"""Validate detected license against allowlist/blocklist."""
license_id = detection.license_id
@@ -549,48 +684,25 @@ class LicenseValidator(ValidatorBase):
result.add_warning("License detection returned empty identifier")
return
- symbols = self._expression_symbols(license_id)
- if len(symbols) > 1:
- self._validate_license_expression(detection, symbols, result)
- return
-
- normalized = self._normalize_license_id(license_id)
-
- if normalized in self._normalized_allowlist:
- self._set_license_metadata(result, detection, "allowed")
+ # Fail closed: one blocked name blocks the value, even a malformed one;
+ # findings for a compound value name the license or exception that decided it.
+ expression = _parse_license_expression(license_id)
+ status, name = self._license_policy(expression)
+ self._set_license_metadata(result, detection, status)
+ if status == "allowed":
result.add_message(f"License: {license_id} (ALLOWED - permissive)")
- elif normalized in self._normalized_blocklist:
- self._set_license_metadata(result, detection, "blocked")
- self._add_blocked_license_finding(result, license_id, detection.file_path)
+ elif status == "blocked":
+ subject = name if name and len(expression.names) > 1 else license_id
+ if subject != license_id:
+ result.add_message(f"SPDX expression '{license_id}' includes blocked license '{subject}'")
+ if expression.error:
+ result.add_message(f"License expression '{license_id}' is not valid SPDX syntax: {expression.error}")
+ self._add_blocked_license_finding(result, subject, detection.file_path)
+ elif expression.error:
+ self._add_unsupported_license_expression_finding(result, license_id, detection.file_path, expression.error)
else:
- self._set_license_metadata(result, detection, "unknown")
- self._handle_unknown_license(result, license_id, detection.file_path)
-
- def _validate_license_expression(
- self,
- detection: LicenseDetection,
- symbols: list[str],
- result: ValidationResult,
- ) -> None:
- """Fail closed if any SPDX expression symbol is blocked or unknown when required."""
- blocked = [symbol for symbol in symbols if self._normalize_license_id(symbol) in self._normalized_blocklist]
- if blocked:
- self._set_license_metadata(result, detection, "blocked")
- result.add_message(
- f"SPDX expression '{detection.license_id}' includes blocked license '{blocked[0]}'"
- )
- self._add_blocked_license_finding(result, blocked[0], detection.file_path)
- return
- if all(self._normalize_license_id(symbol) in self._normalized_allowlist for symbol in symbols):
- self._set_license_metadata(result, detection, "allowed")
- result.add_message(f"License: {detection.license_id} (ALLOWED - permissive)")
- return
- unknown = next(
- (symbol for symbol in symbols if self._normalize_license_id(symbol) not in self._normalized_allowlist),
- detection.license_id,
- )
- self._set_license_metadata(result, detection, "unknown")
- self._handle_unknown_license(result, unknown, detection.file_path)
+ subject = name if name and (len(expression.licenses) > 1 or expression.exceptions) else license_id
+ self._handle_unknown_license(result, subject, detection.file_path)
@staticmethod
def _set_license_metadata(result: ValidationResult, detection: LicenseDetection, status: str) -> None:
@@ -621,6 +733,26 @@ class LicenseValidator(ValidatorBase):
is_error=True,
)
+ @staticmethod
+ def _add_unsupported_license_expression_finding(
+ result: ValidationResult,
+ license_id: str,
+ file_path: str | None,
+ reason: str,
+ ) -> None:
+ """Add structured finding for a license expression that does not parse."""
+ result.add_structured_finding(
+ Finding(
+ category="LICENSE",
+ severity="HIGH",
+ check_name="unsupported_license_expression",
+ message=f"License expression '{license_id}' is not valid SPDX syntax: {reason}",
+ file_path=file_path or "unknown",
+ suggestion="Use a valid SPDX license expression, such as 'MIT OR Apache-2.0'.",
+ ),
+ is_error=True,
+ )
+
def _handle_unknown_license(self, result: ValidationResult, license_id: str, file_path: str | None) -> None:
"""Handle license not in allowlist or blocklist."""
if self.strict_mode:
@@ -646,4 +778,8 @@ class LicenseValidator(ValidatorBase):
"""Normalize license ID for case-insensitive comparison."""
normalized = license_id.strip().lower().replace(" ", "-").replace("_", "-")
normalized = _THE_PREFIX_PATTERN.sub("", normalized)
- return _LICENSE_SUFFIX_PATTERN.sub("", normalized)
+ normalized = _LICENSE_SUFFIX_PATTERN.sub("", normalized)
+ # SPDX "+" means "this version or any later version" (GPL-3.0+ is GPL-3.0-or-later).
+ if normalized.endswith("+"):
+ normalized = f"{normalized[:-1]}-or-later"
+ return normalized
diff --git a/tests/validators/test_license.py b/tests/validators/test_license.py
index dbad4ee0..311060e8 100644
--- a/tests/validators/test_license.py
+++ b/tests/validators/test_license.py
@@ -268,6 +268,21 @@ license: {declared_license}
return skill_dir
+def create_skill_with_declared_license(tmp_path: Path, name: str, declared_license: str) -> Path:
+ """Create a skill whose only license signal is its SKILL.md license field."""
+ skill_dir = tmp_path / name
+ skill_dir.mkdir()
+ (skill_dir / "SKILL.md").write_text(f"""---
+name: {name}
+description: Skill with a declared license expression
+license: "{declared_license}"
+---
+
+# Declared License Expression
+""")
+ return skill_dir
+
+
# =============================================================================
# BASIC VALIDATION TESTS
# =============================================================================
@@ -511,6 +526,200 @@ in the Software without restriction.
assert any(f.check_name == "unknown_license" for f in result.findings)
assert all(f.check_name != "frontmatter_license_mismatch" for f in result.findings)
+ @pytest.mark.parametrize(
+ "declared_license",
+ [
+ "CC-BY-4.0 AND Apache-2.0",
+ "CC-BY-4.0 OR Apache-2.0",
+ "(MIT OR Apache-2.0)",
+ "(MIT OR Apache-2.0) AND BSD-3-Clause",
+ "MIT OR CC-BY-4.0 AND Apache-2.0",
+ "(MIT)",
+ "Apache-2.0 WITH LLVM-exception",
+ ],
+ )
+ def test_valid_spdx_expression_of_allowed_licenses_passes(self, tmp_path: Path, declared_license: str):
+ """Parentheses, mixed operators and listed WITH exceptions keep an allowed expression allowed."""
+ skill_dir = create_skill_with_declared_license(tmp_path, "spdx-allowed", declared_license)
+
+ result = LicenseValidator(strict_mode=True).validate(skill_dir)
+
+ assert result.passed
+ assert result.metadata.get("license") == declared_license
+ assert result.metadata.get("license_status") == "allowed"
+
+ @pytest.mark.parametrize(
+ "declared_license",
+ [
+ "GPL-2.0",
+ "LGPL-2.1",
+ "GPL-3.0+",
+ "GPL-2.0+",
+ "AGPL-3.0+",
+ "LGPL-2.1+",
+ "LGPL-2.1-only",
+ "LGPL-3.0-or-later",
+ "MPL-2.0+",
+ "MIT OR GPL-3.0+",
+ "GPL-2.0-only WITH Classpath-exception-2.0",
+ "GPL-2.0-with-classpath-exception",
+ "GPL-2.0-with-GCC-exception",
+ "GPL-3.0-with-autoconf-exception",
+ ],
+ )
+ def test_version_forms_of_blocked_licenses_are_blocked(self, tmp_path: Path, declared_license: str):
+ """`+`, `-only`, `-or-later` and deprecated `-with-*-exception` forms must not pass as unknown."""
+ skill_dir = create_skill_with_declared_license(tmp_path, "spdx-blocked", declared_license)
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "blocked"
+ assert any(f.check_name == "blocked_license" for f in result.findings)
+
+ @pytest.mark.parametrize(
+ ("declared_license", "blocked", "syntax_error"),
+ [
+ ("GPL-3.0 (or later)", "GPL-3.0", None),
+ ("LGPL-2.1 (or any later version)", "LGPL-2.1", None),
+ ("(gpl-3.0 or mit", "gpl-3.0", None),
+ ("mit or (gpl-3.0", "gpl-3.0", None),
+ ("mit or gpl-3.0)", "gpl-3.0", None),
+ ("MIT AND (GPL-3.0", "GPL-3.0", "unbalanced parenthesis"),
+ ("(GPL-3.0 OR MIT", "GPL-3.0", "unbalanced parenthesis"),
+ ],
+ )
+ def test_blocked_license_in_malformed_value_is_blocked(
+ self, tmp_path: Path, declared_license: str, blocked: str, syntax_error: str | None
+ ):
+ """A blocked license fails a value that does not parse, instead of being reviewed or unsupported."""
+ skill_dir = create_skill_with_declared_license(tmp_path, "malformed-blocked", declared_license)
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "blocked"
+ finding = next(f for f in result.findings if f.check_name == "blocked_license")
+ assert finding.message == f"License '{blocked}' is not allowed (restrictive/copyleft)"
+ assert not result.warnings
+ syntax_messages = [message for message in result.messages if "is not valid SPDX syntax" in message]
+ if syntax_error:
+ assert syntax_messages == [
+ f"License expression '{declared_license}' is not valid SPDX syntax: {syntax_error}"
+ ]
+ else:
+ assert not syntax_messages
+
+ def test_or_later_frontmatter_is_not_hidden_by_apache_license_file(self, tmp_path: Path):
+ """GPL-3.0+ next to an allowed LICENSE file is blocked, not reviewed as unknown."""
+ skill_dir = create_skill_with_declared_license_and_file(
+ tmp_path,
+ "gpl-or-later-with-apache-file",
+ "GPL-3.0+",
+ APACHE_2_LICENSE_TEXT,
+ )
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "blocked"
+ assert any(f.check_name == "blocked_license" for f in result.findings)
+
+ def test_unlisted_with_exception_needs_review(self, tmp_path: Path):
+ """An exception that is not in the allowlist does not inherit Apache-2.0's allowed status."""
+ skill_dir = create_skill_with_declared_license(
+ tmp_path,
+ "spdx-unlisted-exception",
+ "Apache-2.0 WITH Commons-Clause",
+ )
+
+ lenient = LicenseValidator().validate(skill_dir)
+ strict = LicenseValidator(strict_mode=True).validate(skill_dir)
+
+ assert lenient.metadata.get("license_status") == "unknown"
+ assert any("'Apache-2.0 WITH Commons-Clause' not in allowlist" in warning for warning in lenient.warnings)
+ assert not strict.passed
+ assert any(f.check_name == "unknown_license" for f in strict.findings)
+
+ @pytest.mark.parametrize(
+ ("declared_license", "reason"),
+ [
+ ("(MIT OR Apache-2.0", "unbalanced parenthesis"),
+ ("MIT OR Apache-2.0)", "unbalanced parenthesis"),
+ ("MIT AND", "missing license identifier after 'AND'"),
+ ("OR MIT", "missing license identifier before 'OR'"),
+ ("MIT OR OR Apache-2.0", "missing license identifier before 'OR'"),
+ ("(MIT OR Apache-2.0) BSD-3-Clause", "missing operator before 'BSD-3-Clause'"),
+ ("MIT WITH", "missing exception identifier after 'WITH'"),
+ ("(MIT OR Apache-2.0) WITH LLVM-exception", "WITH must follow a single license identifier"),
+ ],
+ )
+ def test_malformed_spdx_expression_is_rejected(self, tmp_path: Path, declared_license: str, reason: str):
+ """Malformed expressions fail instead of being allowed or sent to manual review."""
+ skill_dir = create_skill_with_declared_license(tmp_path, "spdx-malformed", declared_license)
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "unsupported"
+ finding = next(f for f in result.findings if f.check_name == "unsupported_license_expression")
+ assert finding.message == f"License expression '{declared_license}' is not valid SPDX syntax: {reason}"
+
+ @pytest.mark.parametrize(
+ ("declared_license", "status"),
+ [
+ ("MIT License", "allowed"),
+ ("The MIT License", "allowed"),
+ ("MIT XOR Apache-2.0", "unknown"),
+ ("LicenseRef-Example-with-attribution", "unknown"),
+ ("MIT License (MIT)", "unknown"),
+ ("Apache 2.0 (with exceptions)", "unknown"),
+ ("(MIT", "unknown"),
+ ("mit or (apache-2.0", "unknown"),
+ ],
+ )
+ def test_license_names_without_operators_are_not_syntax_errors(
+ self, tmp_path: Path, declared_license: str, status: str
+ ):
+ """Values without an upper-case SPDX operator stay free-text names for name matching or review."""
+ skill_dir = create_skill_with_declared_license(tmp_path, "free-text-license", declared_license)
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert result.passed
+ assert result.metadata.get("license_status") == status
+ assert all(f.check_name != "unsupported_license_expression" for f in result.findings)
+
+ def test_expression_including_license_file_id_is_consistent(self, tmp_path: Path):
+ """A LICENSE file naming one license of the declared expression is not a conflict."""
+ skill_dir = create_skill_with_declared_license_and_file(
+ tmp_path,
+ "dual-license-apache-file",
+ "MIT OR Apache-2.0",
+ APACHE_2_LICENSE_TEXT,
+ )
+
+ result = LicenseValidator(strict_mode=True).validate(skill_dir)
+
+ assert result.passed
+ assert result.metadata.get("license_status") == "allowed"
+ assert all(f.check_name != "frontmatter_license_mismatch" for f in result.findings)
+
+ def test_expression_excluding_license_file_id_fails_closed(self, tmp_path: Path):
+ """An allowed expression that omits the allowed LICENSE file's id conflicts, as a single id does."""
+ skill_dir = create_skill_with_declared_license_and_file(
+ tmp_path,
+ "mit-and-cc-vs-apache-body",
+ "MIT AND CC-BY-4.0",
+ APACHE_2_LICENSE_TEXT,
+ )
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "conflict"
+ assert any(f.check_name == "frontmatter_license_mismatch" for f in result.findings)
+
def test_workflow_frontmatter_detection(self, workflow_with_license: Path):
"""Test detection of license in workflow frontmatter."""
validator = LicenseValidator()
@@ -757,6 +966,75 @@ description: Skill whose SPDX header is MIT OR MIT-0
assert result.metadata.get("license_status") == "allowed"
assert "MIT OR MIT-0" in (result.metadata.get("license") or "")
+ @pytest.mark.parametrize(
+ ("filename", "content", "expected"),
+ [
+ (
+ "reference.md",
+ "| Check | Header |\n| --- | --- |\n| License | `// SPDX-License-Identifier: GPL-3.0` |\n",
+ "GPL-3.0",
+ ),
+ ("notes.md", "Add `SPDX-License-Identifier: MIT OR GPL-3.0` to each file.\n", "MIT OR GPL-3.0"),
+ ("script.py", "# SPDX-License-Identifier: MIT # trailing comment\n", "MIT"),
+ ("script.js", "// SPDX-License-Identifier: MIT;\n", "MIT"),
+ (
+ "script.js",
+ "/* SPDX-License-Identifier: GPL-2.0-only WITH Classpath-exception-2.0 */\n",
+ "GPL-2.0-only WITH Classpath-exception-2.0",
+ ),
+ ("script.sh", "# SPDX-License-Identifier: MIT | GPL-3.0\n", "MIT | GPL-3.0"),
+ ],
+ )
+ def test_spdx_header_value_stops_at_enclosing_delimiters(
+ self, tmp_path: Path, filename: str, content: str, expected: str
+ ):
+ """Markdown code/table delimiters and a trailing `;` end the value; a bare `|` does not."""
+ source = tmp_path / filename
+ source.write_text(content)
+
+ assert LicenseValidator._extract_spdx_from_file(source) == expected
+
+ def test_spdx_header_in_markdown_table_is_blocked(self, tmp_path: Path):
+ """A table cell's closing backtick and `|` must not hide GPL-3.0 as an unknown id."""
+ skill_dir = tmp_path / "spdx-md-table"
+ skill_dir.mkdir()
+ (skill_dir / "SKILL.md").write_text("""---
+name: spdx-md-table
+description: Skill whose SPDX header is in a Markdown table
+---
+
+# SPDX table
+""")
+ (skill_dir / "reference.md").write_text(
+ "| File | License |\n| --- | --- |\n| tool.py | `SPDX-License-Identifier: MIT OR GPL-3.0` |\n"
+ )
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "blocked"
+ assert result.metadata.get("license") == "MIT OR GPL-3.0"
+ assert any(f.check_name == "blocked_license" for f in result.findings)
+
+ def test_malformed_spdx_header_is_rejected(self, tmp_path: Path):
+ """An unbalanced header expression is not allowed after its parenthesis is dropped."""
+ skill_dir = tmp_path / "spdx-malformed-header"
+ skill_dir.mkdir()
+ (skill_dir / "SKILL.md").write_text("""---
+name: spdx-malformed-header
+description: Skill whose SPDX header has an unbalanced parenthesis
+---
+
+# SPDX malformed
+""")
+ (skill_dir / "script.py").write_text("# SPDX-License-Identifier: (MIT OR Apache-2.0\n")
+
+ result = LicenseValidator().validate(skill_dir)
+
+ assert not result.passed
+ assert result.metadata.get("license_status") == "unsupported"
+ assert any(f.check_name == "unsupported_license_expression" for f in result.findings)
+
# =============================================================================
# NO LICENSE / UNKNOWN LICENSE TESTS