Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(evals): converge the shifted citation onto the drift-proof file::symbol form

Aashishda38ce307714
diff --git a/laya/evals.py b/laya/evals.py
index 79ea2b85..840195c9 100644
--- a/laya/evals.py
+++ b/laya/evals.py
@@ -206,10 +206,11 @@ class Dataset:
# whose `expected` names a question `questions` does not carry. A row built in Python with
# `Example(...)` skipped all three, and every reader of it takes the two fields as mappings: a
# non-object `expected` died as an unhandled ``AttributeError``, in `evaluate` only after the
- # runner had answered (`laya/evals.py:1217`), and a non-object `questions` leaked through
- # `questions_fingerprint` (`laya/evals.py:297`). An `expected` key with no question behind it
- # was answered with "runner returned no answer for question 'x'" -- a verdict about the runner,
- # paid for with a whole inference run, for a mistake in the caller's row. The two opt-in doors
+ # runner had answered (`laya/evals.py:1205`), and a non-object `questions` leaked through
+ # `questions_fingerprint` (`laya/evals.py::questions_fingerprint`). An `expected` key with no
+ # question behind it was answered with "runner returned no answer for question 'x'" -- a
+ # verdict about the runner, paid for with a whole inference run, for a mistake in the
+ # caller's row. The two opt-in doors
# refused the same rows before calling anything, but as tracebacks
# (`laya/evals_shortlist.py:38`, `laya/evals_tournament.py:33`). Checking at the door all four
# readers share names the row and lands before a single call: a leaked traceback exits 1, the
diff --git a/tests/test_evals.py b/tests/test_evals.py
index 2d99584a..b2e82809 100644
--- a/tests/test_evals.py
+++ b/tests/test_evals.py
@@ -779,10 +779,11 @@ def test_dataset_rejects_malformed_json_and_empty(tmp_path):
# A row built in Python never passes through `Example.from_dict`, which is the only place these three
# checks were made: `questions` and `expected` must be objects, and every `expected` key must name a
# question that is actually there. Every reader takes the two fields as mappings, so a row that
-# skipped the checks reached `evaluate`'s scoring loop (`laya/evals.py:1217`), `questions_fingerprint`
-# (`laya/evals.py:297`) or an opt-in door's `_validate_dataset` as an unhandled ``AttributeError`` --
-# or got as far as the runner and came back as a verdict *about the runner*. `Dataset` is the one door
-# all four readers share, so the refusal names the row and lands before a call is paid for.
+# skipped the checks reached `evaluate`'s scoring loop (`laya/evals.py:1205`), `questions_fingerprint`
+# (`laya/evals.py::questions_fingerprint`) or an opt-in door's `_validate_dataset` as an unhandled
+# ``AttributeError`` -- or got as far as the runner and came back as a verdict *about the runner*.
+# `Dataset` is the one door all four readers share, so the refusal names the row and lands before a
+# call is paid for.
@pytest.mark.parametrize("field, value, fragment", [
("expected", True, "'expected' must be an object keyed by question id, got bool"),
("expected", "a", "'expected' must be an object keyed by question id, got str"),