Skip to content

Plan, apply and roll back changes to a production ontology, with a blast radius report and a proof an auditor can re-check.

ci: build, lint and test the optional features, and catch dark test files

Nicolas Geyssede54828b52af
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 5a949ada..4f6e51c6 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -37,3 +37,71 @@ jobs:
run: |
cargo install cargo-audit
cargo audit
+
+ # Optional features are compiled by nothing above: `default = []`, and the
+ # three cargo invocations in `build` are bare. A green run there says nothing
+ # about postgres, duckdb, sql, embeddings or causal-pywhy, nor about the
+ # dependency trees they pull in (sqlx, duckdb, tract-onnx, tokenizers,
+ # instant-distance).
+ #
+ # It is also why seven test files under tests/ had never been collected once.
+ # tests/schema_test.rs is gated on `postgres` and covers
+ # SchemaIntrospector::sql_to_xsd, which carried a real defect the whole time
+ # (#76, fixed in #80): the test existed, and it was dark.
+ #
+ # Ubuntu only. The bundled duckdb build dominates wall time and Windows would
+ # double it for no signal the Linux leg does not already give.
+ features:
+ name: features / ${{ matrix.name }}
+ runs-on: ubuntu-latest
+ strategy:
+ fail-fast: false
+ matrix:
+ include:
+ # Breadth: everything compiles and lints, including causal-pywhy,
+ # whose Rust side has no dependencies of its own.
+ - name: breadth (all-features)
+ flags: --all-features
+ test: false
+ # Depth: everything whose tests run without external services.
+ # causal-pywhy is left out here rather than everywhere — it shells out
+ # to Python/DoWhy at runtime, which is a service dependency in all but
+ # name. It is still compiled and linted by the breadth leg.
+ - name: depth (postgres,duckdb,embeddings,sql)
+ flags: --features postgres,duckdb,embeddings,sql
+ test: true
+ features_list: postgres duckdb embeddings sql
+ env:
+ CARGO_INCREMENTAL: 0
+ CARGO_PROFILE_DEV_DEBUG: 0
+ steps:
+ - uses: actions/checkout@v4
+ - name: Free disk space
+ run: |
+ sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc /usr/local/share/powershell /usr/local/share/chromium /usr/local/share/edge
+ sudo apt-get clean
+ df -h
+ - name: Install build deps (OpenSSL/libpq)
+ run: sudo apt-get update && sudo apt-get install -y pkg-config libssl-dev libpq-dev
+ - uses: dtolnay/rust-toolchain@stable
+ with:
+ components: clippy
+ - uses: Swatinem/rust-cache@v2
+ with:
+ # Keyed on the feature set: the legs build disjoint dependency trees,
+ # and sharing one cache between them would thrash it on every run.
+ key: ${{ matrix.name }}
+ - name: Check
+ run: cargo check ${{ matrix.flags }} --all-targets
+ - name: Clippy
+ # --all-targets so the gated tests are linted too. `-D warnings` over
+ # code that is never compiled is not a check.
+ run: cargo clippy ${{ matrix.flags }} --all-targets -- -D warnings
+ - name: Test
+ if: matrix.test
+ run: |
+ set -o pipefail
+ cargo test ${{ matrix.flags }} 2>&1 | tee cargo-test.log
+ - name: Verify every enabled test file collected tests
+ if: matrix.test
+ run: bash scripts/check-test-collection.sh cargo-test.log "${{ matrix.features_list }}"
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c0b780a0..cfd8f64d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,19 @@ All notable changes to Open Ontologies are documented here.
## [Unreleased]
### Added
+- **CI builds, lints and tests the optional features.** A `features` job adds a
+ breadth leg (`cargo check` + `cargo clippy --all-targets` at `--all-features`)
+ and a depth leg (`cargo test --features postgres,duckdb,embeddings,sql`), with
+ `rust-cache` keyed per feature set. `default = []`, so none of `postgres`,
+ `duckdb`, `sql`, `embeddings` or `causal-pywhy` — nor the `sqlx`, `duckdb`,
+ `tract-onnx`, `tokenizers` and `instant-distance` trees — was compiled by
+ anything in CI before this.
+
+ `scripts/check-test-collection.sh` fails the run when a test file whose gate
+ is enabled collects zero tests. Seven files under `tests/` had never been
+ collected once, 53 tests in total; a file that collects nothing reports
+ "test result: ok" and is indistinguishable from a passing one. Files whose
+ gate is off in a given leg are skipped, so partial-feature legs stay green.
- **Opt-in persistent triple store.** A new `[storage]` section selects the
backend for the main graph: `mode = "memory"` (default, unchanged behaviour)
or `mode = "persistent"`, which opens a RocksDB-backed Oxigraph store at
diff --git a/scripts/check-test-collection.sh b/scripts/check-test-collection.sh
new file mode 100644
index 00000000..99239124
--- /dev/null
+++ b/scripts/check-test-collection.sh
@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+#
+# Fail when an integration test file compiles to a target that collects no tests.
+#
+# `cargo test` prints "running 0 tests" followed by "test result: ok" for a file
+# whose `#![cfg(feature = "...")]` header is not satisfied. That is
+# indistinguishable from passing, which is the worst version of this failure:
+# a green badge over a file that has never run.
+#
+# tests/schema_test.rs sat behind `#![cfg(feature = "postgres")]` while CI ran a
+# bare `cargo test`, so its six tests were never collected once — including the
+# one asserting against `SchemaIntrospector::sql_to_xsd`, which carried a real
+# defect the whole time (IEEE 754 columns mapped to xsd:decimal, fixed in #80).
+#
+# A file whose gate is NOT in the enabled set is skipped: not being built is
+# expected there, and flagging it would make the check permanently red on any
+# partial-feature leg.
+#
+# Usage: bash scripts/check-test-collection.sh <cargo-test-log> "<enabled features>"
+#
+set -euo pipefail
+
+log=${1:?usage: check-test-collection.sh <cargo-test-log> "<enabled features>"}
+enabled=" ${2:-} "
+
+[ -f "$log" ] || { echo "::error::log file not found: $log"; exit 1; }
+
+missing=()
+empty=()
+checked=0
+
+for f in tests/*.rs; do
+ [ -e "$f" ] || continue
+
+ # File-level gate, if any: `#![cfg(feature = "x")]` in the header.
+ gate=$(sed -n '1,12p' "$f" \
+ | sed -n 's/^#!\[cfg(feature = "\([A-Za-z0-9_-]*\)")\].*/\1/p' \
+ | head -1)
+
+ if [ -n "$gate" ] && [[ "$enabled" != *" $gate "* ]]; then
+ continue
+ fi
+
+ checked=$((checked + 1))
+
+ # cargo prints: Running tests/foo.rs (target/debug/deps/foo-<hash>)
+ # then, after a blank line: running N tests
+ count=$(awk -v target="$f" '
+ index($0, "Running " target " ") { seen = 1; next }
+ seen && $1 == "running" { print $2; exit }
+ ' "$log")
+
+ if [ -z "$count" ]; then
+ missing+=("$f")
+ elif [ "$count" -eq 0 ]; then
+ empty+=("$f")
+ fi
+done
+
+status=0
+
+if [ ${#empty[@]} -gt 0 ]; then
+ for f in "${empty[@]}"; do
+ echo "::error file=$f::collected 0 tests while its feature gate is enabled"
+ done
+ status=1
+fi
+
+if [ ${#missing[@]} -gt 0 ]; then
+ for f in "${missing[@]}"; do
+ echo "::error file=$f::no test target ran for this file"
+ done
+ status=1
+fi
+
+if [ "$status" -eq 0 ]; then
+ echo "test collection OK — $checked test file(s) expected under [${2:-none}], all collected at least one test"
+else
+ echo
+ echo "A test file that collects nothing reports \"test result: ok\" and is"
+ echo "indistinguishable from a passing file. Either its gate is wrong, or the"
+ echo "feature set this leg enables no longer matches the files it should build."
+fi
+
+exit "$status"