Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
Merge upstream/main into ci/dotnet-parity-layout
kevin-gatimu157b325a5dea
diff --cc .github/workflows/dotnet.ymlindex 9d77a8cf,b8c1bf69..3c419ac9--- a/.github/workflows/dotnet.yml+++ b/.github/workflows/dotnet.yml@@@ -136,119 -108,61 +136,124 @@@ jobsname: parity (${{ matrix.name }})runs-on: ubuntu-latestcontinue-on-error: true # advisory: see the header- # Per cell, from the matrix (see the derivation below).- # 120, not 60, for the reason rust.yml's parity job went to 240: the suites replay every- # golden through real ONNX inference and the case count only goes up. At 60 the- # typed-decisions cell hit the wall on the 0.4.16 release commit and GitHub recorded it as- # CANCELLED, which reads like a superseded run rather than a budget failure. Its test step- # had run for 34 minutes and printed no result summary, so that cell verified nothing while- # the lane stayed green on `continue-on-error`.- #- # Measured on a478221b, where #1358 added about 50 batch cases per checkpoint:- # english 52m passed, with 8 minutes to spare- # typed-decisions 60m killed mid-test-run- # multilingual 13m passed- # router 26m failed on setup-dotnet, not on time- # english being that close is the same warning typed-decisions already acted on.- #- # Per-checkpoint sharding is already here, so the next lever if these grow again is splitting- # the replay classes off the tokenizer and shape sweeps rather than more headroom.- timeout-minutes: 120++ # Per cell, from the matrix (see the derivation below). The single-cell limit went from 60 to++ # 120 on main (b9fe351c) because the replays grew: on a478221b, where #1358 added about 50 batch++ # cases per checkpoint, english took 52m, typed-decisions was killed at 60m mid-test-run and++ # recorded as CANCELLED (which reads like a superseded run, not a budget failure), multilingual++ # 13m and router 26m. Splitting each checkpoint by layout halves the replays per cell, so each++ # cell gets a limit sized from its own measured time instead of one shared headroom number.+ timeout-minutes: ${{ matrix.timeout }}strategy:fail-fast: false- # One cell at a time. Every cell that needs weights fetches them from the same Hugging+ # FOUR at a time (was two). Every cell that needs weights fetches them from the same Hugging# Face repo, and four in parallel -- alongside `java.yml`'s parity matrix against the same- # repo in the same run -- is what provoked the throttle. `regen_golden.py` now refuses an- # incomplete snapshot at the fetch, so this is no longer the only thing between a- # truncated download and a confusing failure; not provoking the throttle is simply- # cheaper than diagnosing it, and this lane is advisory.+ # repo in the same run -- is what once provoked the throttle. Two was the answer then; it is+ # now the wrong trade, because the matrix is seven cells (a checkpoint x layout pair for each+ # checkpoint, plus the router) and at two the critical path would be half of the summed cell+ # time, about 105 minutes on a slow host, no better than the one-cell-per-checkpoint shape+ # this replaces. What changed since: the exports and the pinned Hub downloads are cached+ # (keys below), so a warm run fetches nothing; a cold run fetches only for the cells that+ # miss, `HF_TOKEN` authenticates those fetches, and `regen_golden.py` refuses an incomplete+ # snapshot at the fetch, naming the file. Four is below the seven concurrent fetches that+ # provoked the original throttle and gives, on the slow-host estimates below, a critical path+ # of about 60 minutes (about 45 with seven slots, about 105 with two). The cells are listed+ # longest first so the long ones start together and the short ones fill the slots as they+ # free up. If a cold run is throttled, lower this to 2: the cost is wall time only, and this+ # lane is advisory.+ max-parallel: 4+ # One cell per checkpoint x layout. Both cells of a checkpoint regenerate that checkpoint's+ # goldens from Python at this commit (the recording is layout independent but each cell owns+ # its copy; nothing is shared between cells and goldens are never cached).#- # TWO, not one. One was the first attempt and it cost too much: measured from this PR's own- # cells, serialising takes the parity critical path from 10:56 to 30:08. Two keeps the peak- # well below the seven concurrent fetches that provoked the throttle while costing roughly- # half of that. And serialising is no longer the load-bearing half of the fix -- the- # download sites now refuse an incomplete snapshot AT THE FETCH, naming the file, so a- # throttled fetch fails immediately and legibly instead of three minutes later inside a- # tokenizer constructor. This only reduces how often that path is taken.- max-parallel: 2+ # `layouts` is LAYA_TEST_LAYOUTS: it narrows the theory rows that exist for both layouts+ # (long and batch predictions). The test methods that exercise one layout only carry+ # `[Trait("Layout", ...)]`: `split-only` for the split-layout theories, `any` for the+ # layout-parameterised ones. The fused cell therefore runs everything except `split-only`,+ # including the tokenizer, shortlist, error-case and multilingual-only tests, which do not+ # depend on the layout and so run once; the split cell runs `split-only` and `any`. A+ # deselected test is never discovered, so it cannot show up as a skip.+ #+ # `min-passed` and `timeout`, from the committed goldens and the lane logs in+ # researcher-agent/ci-runtime.md (slow-host mode, 4 vCPU, the worse of the two modes seen):+ # * Tests per cell, counted from the theory data with LAYA_TEST_CHECKPOINTS and+ # LAYA_TEST_LAYOUTS set (all layouts together the english count is 484, the same as the+ # a478221b log):+ # english fused 222 pass (216 rows + 6 layout independent), split 130+ # multilingual fused 323 pass, split 136+ # typed-decisions fused 228 pass, split 136+ # `min-passed` is about 80% of that, rounded down: a floor against a wrong filter or a+ # dropped group, with room for the case count to shrink. A layout whose artifacts are+ # missing is caught by the skip guard, not by this floor.+ # * Time. Tests measured on a slow host with both layouts in one cell: english 33.8 min,+ # typed-decisions about 41 (its cell hit the old 60 minute cap), multilingual 8-16; regen+ # 17.1 / 19.4 / 4.5 min; the router cell 15-22 in total. A layout cell does about half of+ # the forward passes (each success case runs once per layout); take 0.55 for the fused+ # cell, which also carries the one-off tests. Volume is expected to grow by about 47% when+ # the pending option-order and lang-temperature cases land. Projected cell: install + regen+ # + 0.55 x tests x 1.47 + 1 => english about 46, typed-decisions about 54, multilingual+ # about 17, router 22. `timeout` is that with 1.5x to 2x headroom for a slower host or a+ # cold export cache (about 3 minutes per checkpoint): 75 / 90 / 40 / 45.matrix:include:- - name: english- checkpoints: english- filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests- min-passed: 50- - name: multilingual- checkpoints: multilingual- filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests- min-passed: 50- - name: typed-decisions+ - name: typed-decisions-fused+ cache: typed-decisionscheckpoints: typed-decisions- filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests- min-passed: 50- # The router end-to-end tests load english and multilingual side by side.+ layouts: fused+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-not-trait Layout=split-only+ min-passed: 180+ timeout: 90+ - name: typed-decisions-split+ cache: typed-decisions+ checkpoints: typed-decisions+ layouts: split+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-trait Layout=split-only --filter-trait Layout=any+ min-passed: 105+ timeout: 90+ - name: english-fused+ cache: english+ checkpoints: english+ layouts: fused+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-not-trait Layout=split-only+ min-passed: 175+ timeout: 75+ - name: english-split+ cache: english+ checkpoints: english+ layouts: split+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-trait Layout=split-only --filter-trait Layout=any+ min-passed: 100+ timeout: 75+ # The router end-to-end tests load english and multilingual side by side; they have+ # no layout parameter, so LAYA_TEST_LAYOUTS stays unset.- name: router+ cache: routercheckpoints: english multilingual+ layouts: ""filter: --filter-class Laya.Tests.RouterEndToEndTestsmin-passed: 1+ timeout: 45+ - name: multilingual-fused+ cache: multilingual+ checkpoints: multilingual+ layouts: fused+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-not-trait Layout=split-only+ min-passed: 255+ timeout: 40+ - name: multilingual-split+ cache: multilingual+ checkpoints: multilingual+ layouts: split+ filter: --filter-class Laya.Tests.PredictParityTests Laya.Tests.TokenizerParityTests Laya.Tests.ShapeSweepTests Laya.Tests.ShortlistEndToEndTests --filter-trait Layout=split-only --filter-trait Layout=any+ min-passed: 105+ timeout: 40steps:- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1+ # So the runner's speed can be read from the log: the same job has run at twice the+ # per-token cost on some hosts, and the CPU model and core count are what tell them apart.+ - name: Runner CPU+ run: |+ nproc+ lscpu | grep -E '^(Model name|CPU\(s\)|Thread\(s\) per core|Core\(s\) per socket|Socket\(s\)|CPU max MHz)'- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0with:python-version: "3.12"