Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

ci(evals): add a weight-free eval job

Emiliano Gandini Outedab2998bdd973a
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 11818be5..5e7f69ad 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -144,6 +144,34 @@ jobs:
tests/test_truncation_direction.py \
-q
+ # The eval harness needs numpy and laya only, so it stays a separate, weight-free job: metric
+ # math, dataset parsing, the CLI and the API guard. The real-checkpoint gate is the scheduled
+ # `evals` workflow, which needs weights and network.
+ evals:
+ name: evals (metric math, dataset, API guard)
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+ - uses: actions/setup-python@v5
+ with:
+ python-version: "3.11"
+ cache: pip
+
+ - name: Install
+ run: |
+ python -m pip install --upgrade pip
+ pip install torch --index-url https://download.pytorch.org/whl/cpu
+ pip install -e . pytest
+
+ - name: Eval harness tests
+ run: python -m pytest tests/test_evals.py -q
+
+ - name: Eval API guard
+ run: python tests/test_evals_api.py
+
+ - name: Fixture dataset validates
+ run: laya-eval validate research/evals/fixture.jsonl
+
# Every workflow in this repo ran on ubuntu-latest only, so a test that fails off Linux was
# invisible until a user hit it -- which is what happened to tests/test_download.py (#140).
#