Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
test(ts): pin predictLong parity and document scan semantics
wowec98c9230ff9
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex ed634b64..a9c14ec2 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -41,6 +41,18 @@ jobs:cache: npmcache-dependency-path: laya-ts/package-lock.json- run: npm ci+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0+ with:+ python-version: "3.11"+ cache: pip+ - name: Install Python parity oracle dependencies+ run: |+ python -m pip install numpy tokenizers==0.23.2+ python -m pip install torch --index-url https://download.pytorch.org/whl/cpu+ - name: Python long-prediction oracle drift check (no models)+ run: python scripts/predict_long_fixtures.py --check+ - name: Official tokenizer decode parity (no models)+ run: python scripts/tokenizer_decode_fixtures.py --check- name: Unit testsrun: npm test -- --run- name: Packed package end-to-end testdiff --git a/laya-ts/README.md b/laya-ts/README.mdindex 4451768c..0448a4e1 100644--- a/laya-ts/README.md+++ b/laya-ts/README.md@@ -45,6 +45,83 @@ const out = await agent.predict("charged twice", {`onnxruntime-node` / `onnxruntime-web` are optional peer deps, imported lazily behind the provider you use.+## Long documents (`predictLong`)++`Agent.predictLong` and `Router.predictLong` scan overlapping token windows locally, in Node+or the browser. This lets a question consider a long ticket or conversation beyond the single+window used by `predict`. The Router chooses and loads a checkpoint once for the document.++```ts+const scanned = await router.predictLong(longTicket, {+ refund: { type: "noul", instructions: "Does the customer request a refund?" },+ action: { type: "choice", instructions: "Choose the next action", criteria: ["refund", "investigate"] },+}, { window: 256, stride: 128, batchSize: 4 });+console.log(scanned.answers.refund, scanned.usage.windows, scanned.routing);+```++The only aggregation mode is `"auto"`: `noul` takes the largest value; `choice` and `score`+take the answer with the highest `answer_confidence`. Ties retain the first result. The+winner keeps its metadata. Its confidence is a window score, not calibrated confidence for+the whole document; scanning more windows can change which answer wins.++The window is capped by the space left by the questions in the checkpoint's configured+sequence budget. Its default is `max(64, max_len - head_max_len - 8)`, then capped; stride+defaults to half that effective window. An explicit stride cannot leave gaps. `window`,+`stride`, and `batchSize` must be positive safe integers when supplied; null/undefined use+defaults. `maxLen`, `headMaxLen`, and `minConfidence` are deliberately absent from the long+options. Window sizing and aggregation have different meanings from single-pass truncation+and confidence gating. Ordinary prediction options and result types remain unchanged.++Windows follow Python 0.4.18's decode → batch prediction → re-encode path. The official+English and multilingual tokenizers support decoding. A custom encode-only tokenizer still+supports ordinary predictions and the short-input fast path; splitting requires `decode`+and throws if it is unavailable. Decoding and re-encoding can change the token count,+particularly for multilingual text, so normal truncation can still occur. An answer's+optional `window.token_start` / `token_end` identifies its **original candidate token slice**,+not a guarantee that exactly those tokens reached the model.++Agent hooks wrap one window batch; Router hooks wrap one document. Process defaults run+once. Reordering result objects preserves identifiable origins, while rewriting/reordering+states, replacing an answer object, or editing prediction fields removes window attribution.+In-place reliability/action annotations preserve it. An end hook may uniformly remove its own+temporary questions, but cannot silently remove a caller's question. Inconsistent answer+schemas or window counts fail. A skipping hook must return exactly one document result;+the result and usage are copied, a warning is emitted, and `usage.windows` is zero.+The warning also covers short-input and Router cache hits, as agreed in #1316; Python+0.4.18 itself only warns on the multi-window Agent skip path. Hook results are validated+against the TypeScript return contract, which also rejects malformed payloads Python accepts.++If a start hook changes questions or token budgets so planned windows no longer fit, the+scan is refused **after inference**. This check is outside hook error suppression and hook+timeouts; the forward-pass cost has already been incurred. The short path checks this too.+`LongResult` / `RoutedLongResult` have separate usage types: `truncated` is a boolean on the+short path and a numeric count on the multi-window path. Numeric usage is summed, dictionaries+are shallow-merged, other values come from the last result, and `output_tokens` stays zero.++Deterministic parity fixtures are recorded directly from Python `ONNXAgent.predict_long`+at v0.4.18, with source hashes, and replayed by TypeScript tests. The generator's `--check`+detects later reference behavior changes; these checks establish control-flow parity, not+model quality or a speed improvement.++To repeat the optional real-inference comparison with an English checkpoint and its split+ONNX export (weights are not downloaded by these scripts):++```bash+# From laya-ts/, with numpy, torch, transformers and onnxruntime installed in Python.+python scripts/predict_long_onnx_parity.py --python-root /path/to/laya-v0.4.18 \+ --checkpoint /path/to/english-checkpoint --bundle /path/to/split-onnx --output python-long.json+npm run build+node scripts/predict_long_onnx_parity.mjs python-long.json comparison.json+```++The Python harness calls `ONNXAgent.predict_long` with an adapter joining the same encoder+and head graphs used by TypeScript. It records source/artifact hashes and checks all three+answer types on short, overlapping long, and structured inputs. The Node harness verifies+artifact digests and compares actual token/marker/mask inputs, decoded answers, attribution,+and usage at absolute tolerance+`1e-4`, excluding the existing runtime model-name difference. It does not measure retrieval+quality, accuracy, or performance, and does not cover browser GPU numerical differences.+## Hooks (observe or shape every decision)Port of the Python `laya.hooks` lifecycle. A hook is a `(ctx) => void` for `onPredictStart` /diff --git a/laya-ts/fixtures/predict-long.json b/laya-ts/fixtures/predict-long.jsonnew file mode 100644index 00000000..a7c3c570--- /dev/null+++ b/laya-ts/fixtures/predict-long.json@@ -0,0 +1,68 @@+{+ "schema": "laya-predict-long/1",+ "reference": {+ "pinned_tag": "v0.4.18",+ "pinned_commit": "118d361a774c9a28cd02a78d6cf4d2af0614f422",+ "commit": "118d361a774c9a28cd02a78d6cf4d2af0614f422",+ "modified_reference_files": [],+ "python": "3.11.13",+ "numpy": "2.4.6",+ "method": "laya.onnx_agent.ONNXAgent.predict_long",+ "files": {+ "laya/onnx_agent.py": "6364c072750cbaf5a5d692bc64902b4233ed8982f13f99d3eb8e0c9531479ef4",+ "laya/agent.py": "6c425362c589c2fb58a2f6fab19e1dbb0264a7216134323e4605be1b3f4b97f7",+ "laya/common.py": "1c4525db9fcd3e98ff9ad8c064b3081b8b3c4d836a73e7cb186a9b00dd32781a",+ "laya/hooks.py": "bf74bad91a559a291ee6bf2033a8b7b1dea8bed751304a1b8e0bd15bea7e9d62",+ "laya/confidence.py": "f289d9688f3f7e84ea503a40236d2e5b6dcd05180b7c2fcae5a9d4c0b0f19b55"+ }+ },+ "cases": [+ {"name":"all-types","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"short","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.1192,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.8808,"1":0.1192},"confidence":0.4729,"answer_confidence":0.8808,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":134,"output_tokens":0,"state_tokens":2,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[42,46,46],"sums":[3290,4002,3820]}],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"empty-state","state":"","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.0067,"confidence":0.9933,"answer_confidence":0.9933,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":128,"output_tokens":0,"state_tokens":0,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[40,44,44],"sums":[3061,3773,3591]}],"events":[{"phase":"start","states":[""],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"structured-state","state":{"text":"hello","ok":true},"questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775},"window":{"index":2,"token_start":16,"token_end":29,"count":3}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":3}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":2,"token_start":16,"token_end":29,"count":3}}},"usage":{"input_tokens":519,"output_tokens":0,"state_tokens":45,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":3}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4555,5267,5085,4305,5017,4835]},{"rows":3,"tokens":[53,57,57],"sums":[4216,4928,4746]}],"events":[{"phase":"start","states":["{\"text\": \"hello\""," \"hello\", \"ok\": ",", \"ok\": true}"],"questions":["route","urgent","grade"]},{"phase":"end","results":3,"error":false}],"cache_unchanged":true}},+ {"name":"conversation","state":[{"role":"user","content":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789"}],"questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9909,"b":0.0067,"c":0.0025},"confidence":0.9478,"answer_confidence":0.9909,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}},"grade":{"type":"score","score":0.0067,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9933,"1":0.0067},"confidence":0.942,"answer_confidence":0.9933,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}}},"usage":{"input_tokens":1760,"output_tokens":0,"state_tokens":160,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":10}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4542,5254,5072,4473,5185,5003]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4383,5095,4913,4307,5019,4837]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4436,5148,4966,4597,5309,5127]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4792,5504,5322,4976,5688,5506]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4750,5462,5280,4361,5073,4891]}],"events":[{"phase":"start","states":["[{\"role\": \"user\"",": \"user\", \"conte",", \"content\": \"AB","nt\": \"ABCDEGHIKL","CDEGHIKLMNOPQRST","MNOPQRSTUVXYZabc","UVXYZabcdeghikmn","deghikmnopqrstuv","opqrstuvxyz01234","xyz0123456789\"}]"],"questions":["route","urgent","grade"]},{"phase":"end","results":10,"error":false}],"cache_unchanged":true}},+ {"name":"mask-removal","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789[M]ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":12,"token_start":96,"token_end":111,"count":13}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":13}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":12,"token_start":96,"token_end":111,"count":13}}},"usage":{"input_tokens":2285,"output_tokens":0,"state_tokens":207,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":13}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4866,5578,5396,4376,5088,4906]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4182,4894,4712,4400,5112,4930]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4551,5263,5081,4740,5452,5270]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4938,5650,5468,4866,5578,5396]},{"rows":3,"tokens":[55,59,59],"sums":[4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789 ","3456789 ABCDEGHI","ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":13,"error":false}],"cache_unchanged":true}},+ {"name":"default-window-and-cap","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":26,"token_end":78,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":6}}},"usage":{"input_tokens":1653,"output_tokens":0,"state_tokens":295,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":18,"tokens":[92,96,96,92,96,96,92,96,96,92,96,96,92,96,96,75,79,79],"sums":[8134,8846,8664,8008,8720,8538,8143,8855,8673,8035,8747,8565,8152,8864,8682,6615,7327,7145]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456","deghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZ","789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123","abcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUV","456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0","XYZabcdeghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"explicit-window-capped","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":160,"stride":100,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}},"urgent":{"type":"noul","noul":0.0474,"confidence":0.9526,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}}},"usage":{"input_tokens":499,"output_tokens":0,"state_tokens":81,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":2}},"warnings":["laya: predict_long: window=160 is wider than the 52 state tokens these questions leave inside max_len=96, so every window would be truncated to 52 on the way to the model; scanning with window=52 instead","laya: predict_long: stride=100 was a 50% step for the window=160 you asked for, but the window was reduced to 52 to fit the room these questions leave; scanning with stride=26 instead"],"forwards":[{"rows":6,"tokens":[92,96,96,69,73,73],"sums":[8134,8846,8664,5994,6706,6524]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456","deghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":2,"error":false}],"cache_unchanged":true}},+ {"name":"tie-first-window","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"tie":true,"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}},"urgent":{"type":"noul","noul":0.0474,"confidence":0.9526,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"decode-expansion-truncates","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":64,"head_max_len":32},"options":{"window":64},"expansion":2,"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":4,"token_start":56,"token_end":84,"count":7}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":70,"token_end":98,"count":7}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":14,"token_end":42,"count":7}}},"usage":{"input_tokens":1344,"output_tokens":0,"state_tokens":388,"state_tokens_dropped":192,"truncated":7,"truncated_questions":["route","urgent","grade"],"options":{"grade":{"total":2,"distinct":1,"tokens_per_option":8}},"windows":7}},"warnings":["laya: predict_long: window=64 is wider than the 28 state tokens these questions leave inside max_len=64, so every window would be truncated to 28 on the way to the model; scanning with window=28 instead"],"forwards":[{"rows":21,"tokens":[64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64],"sums":[4907,5327,5375,5401,5821,5869,5983,6403,6451,4781,5201,5249,4939,5359,5407,5445,5865,5913,6017,6437,6485]}],"events":[{"phase":"start","states":["AABBCCDDEEGGHHIIKKLLMMNNOOPPQQRRSSTTUUVVXXYYZZaabbccddee","QQRRSSTTUUVVXXYYZZaabbccddeegghhiikkmmnnooppqqrrssttuuvv","gghhiikkmmnnooppqqrrssttuuvvxxyyzz00112233445566778899AA","xxyyzz00112233445566778899AABBCCDDEEGGHHIIKKLLMMNNOOPPQQ","BBCCDDEEGGHHIIKKLLMMNNOOPPQQRRSSTTUUVVXXYYZZaabbccddeegg","RRSSTTUUVVXXYYZZaabbccddeegghhiikkmmnnooppqqrrssttuuvvxx","hhiikkmmnnooppqqrrssttuuvvxxyyzz00112233445566778899"],"questions":["route","urgent","grade"]},{"phase":"end","results":7,"error":false}],"cache_unchanged":true}},+ {"name":"empty-questions","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":6}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":[]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"unsupported-aggregate","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"aggregate":"mean"},"expected":{"error":"predict_long: only aggregate='auto' is supported","warnings":[],"forwards":[],"events":[],"cache_unchanged":true}},+ {"name":"reverse-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"reverse-results","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"installed-end-reverse","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"installed-end-reverse","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"clone-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clone-results","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"annotate","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"annotate","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"shared-answer","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"shared-answer","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"reverse-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"reverse-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[55,59,59,56,60,60],"sums":[4334,5046,4864,4866,5578,5396]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4938,5650,5468,4740,5452,5270]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4551,5263,5081,4400,5112,4930]}],"events":[{"phase":"start","states":["uvxyz0123456789","mnopqrstuvxyz012","bcdeghikmnopqrst","STUVXYZabcdeghik","KLMNOPQRSTUVXYZa","ABCDEGHIKLMNOPQR"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"clone-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clone-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"rewrite-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"rewrite-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9756,"b":0.0179,"c":0.0066},"confidence":0.8825,"answer_confidence":0.9756,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.0025,"confidence":0.9975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.018,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.982,"1":0.018},"confidence":0.87,"answer_confidence":0.982,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":155,"output_tokens":0,"state_tokens":9,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[49,53,53],"sums":[4147,4859,4677]}],"events":[{"phase":"start","states":["rewritten"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"empty-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"empty-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":["laya: predict_long: a start hook left no states to score, so the call aggregated nothing and returns no answers"],"forwards":[],"events":[{"phase":"start","states":[],"questions":["route","urgent","grade"]},{"phase":"end","results":0,"error":false}],"cache_unchanged":true}},+ {"name":"remove-caller-all","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"remove-caller-all","expected":{"error":"predict_long: answer 'route' is missing from 6 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"remove-caller-some","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"remove-caller-some","expected":{"error":"predict_long: answer 'route' is missing from 1 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"temporary-remove-all","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"temporary-remove-all","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1412,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","temporary"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"temporary-remove-some","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"temporary-remove-some","expected":{"error":"predict_long: answer 'temporary' is missing from 1 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","temporary"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"rewrite-question-type","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"rewrite-question-type","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1077,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[60,60,60,60,60,60],"sums":[5112,5112,4930,5263,5263,5081]},{"rows":6,"tokens":[60,60,60,60,60,60],"sums":[5452,5452,5270,5650,5650,5468]},{"rows":6,"tokens":[60,60,60,59,59,59],"sums":[5578,5578,5396,5046,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"clear-questions","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clear-questions","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":6}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":[]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"skip","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip","expected":{"result":{"model":"cache","answers":{"cached":{"type":"noul","noul":0.8,"confidence":0.8,"answer_confidence":0.8,"action":{"act_probability":0.5}}},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":["laya: predict_long: a hook answered the state before it was scanned, so no window decided the result and none is reported"],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"skip-many","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip-many","expected":{"error":"predict_long: a start hook answered this state with 6 results; ctx.skip() takes one result for the document, not one per window","warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"usage-fields","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"usage-fields","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"extra":21,"flag":6,"details":{"0":0,"1":1,"2":2,"3":3,"4":4,"5":5},"label":"5","windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"shorten-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"shorten-results","expected":{"error":"predict_long: the state was split into 6 windows and the call returned 5 results","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"replace-batch","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"replace-batch","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[],"cache_unchanged":true}},+ {"name":"replace-batch-added-removed","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"replace-batch-added-removed","expected":{"error":"predict_long: answer 'temporary' is missing from 6 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[],"cache_unchanged":true}},+ {"name":"short-skip","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip","expected":{"result":{"model":"cache","answers":{"cached":{"type":"noul","noul":0.8,"confidence":0.8,"answer_confidence":0.8,"action":{"act_probability":0.5}}},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"stability-long","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"stability","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"reliability":{"probes":["reorder"],"option_layout":"sequential","decision":null,"soft_stability":0.49875,"stability":0.0,"rename_stability":null,"reorder_stability":0.0,"n_variants":2,"distinct_choices":["a","c"],"variant_choices":{"original":"a","reversed":"c"},"variant_support":{"original":0.9966,"reversed":0.0009}},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1388,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,56,56,60,60,56],"sums":[4400,5112,4930,4400,4551,5263,5081,4551]},{"rows":8,"tokens":[56,60,60,56,56,60,60,56],"sums":[4740,5452,5270,4740,4938,5650,5468,4938]},{"rows":8,"tokens":[56,60,60,56,55,59,59,55],"sums":[4866,5578,5396,4866,4334,5046,4864,4334]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","route::rel1"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"stability-short","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"stability","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775},"reliability":{"probes":["reorder"],"option_layout":"sequential","decision":null,"soft_stability":0.4429,"stability":0.0,"rename_stability":null,"reorder_stability":0.0,"n_variants":2,"distinct_choices":["a","c"],"variant_choices":{"original":"a","reversed":"c"},"variant_support":{"original":0.8438,"reversed":0.042}}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.1192,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.8808,"1":0.1192},"confidence":0.4729,"answer_confidence":0.8808,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":176,"output_tokens":0,"state_tokens":2,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":4,"tokens":[42,46,46,42],"sums":[3290,4002,3820,3290]}],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade","route::rel1"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"budget-after-forward","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":48,"hooksRaise":false},"behavior":"budget","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=48 head_max_len=40) leave 4 state tokens per window, but the scan was sized for 48 from the agent's config (max_len=96 head_max_len=40). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":[],"forwards":[{"rows":12,"tokens":[48,48,48,48,48,48,48,48,48,48,48,48],"sums":[3692,4079,3897,3958,4211,4029,3584,4023,3841,3858,4156,3974]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz012","bcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRS","3456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstu","TUVXYZabcdeghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":4,"error":false}],"cache_unchanged":true}},+ {"name":"short-budget-after-forward","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmno","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{},"behavior":"budget","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=48 head_max_len=40) leave 4 state tokens per window, but the scan was sized for 52 from the agent's config (max_len=96 head_max_len=40). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":[],"forwards":[{"rows":3,"tokens":[48,48,48],"sums":[3692,4079,3897]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmno"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},+ {"name":"in-place-question-budget","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y"}}},"cfg":{"max_len":64,"head_max_len":32},"options":{"hooksRaise":false},"behavior":"widen","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=64 head_max_len=32) leave 12 state tokens per window, but the scan was sized for 30 from the agent's config (max_len=64 head_max_len=32). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":["laya: predict_long: these questions leave only 30 of max_len=64 for the state (their heads take the rest), so the scan window is capped 64 -> 30 and roughly 2.1x as many windows -- each a full forward pass -- are needed to read the document. Fewer or shorter options, a larger max_len, or laya.shortlist.predict_shortlist for a large label set will all cost less than scanning at this width"],"forwards":[{"rows":7,"tokens":[64,64,64,64,64,64,64],"sums":[4573,4797,5063,4372,4643,4897,4981]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdegh","RSTUVXYZabcdeghikmnopqrstuvxyz","ikmnopqrstuvxyz0123456789ABCDE","0123456789ABCDEGHIKLMNOPQRSTUV","GHIKLMNOPQRSTUVXYZabcdeghikmno","XYZabcdeghikmnopqrstuvxyz01234","pqrstuvxyz0123456789"],"questions":["route"]},{"phase":"end","results":7,"error":false}],"cache_unchanged":true}},+ {"name":"edit-type","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"type","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"score","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"score":0.5,"legend":{"0":"low","1":"high"}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-choice","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"choice","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"edited","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-noul","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"noul","qid":"urgent","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1234,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-score","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"score","qid":"grade","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.1234,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-legend","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"legend","qid":"grade","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"edited","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-probabilities","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"probabilities","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.1234,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-confidence","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"confidence","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.1234,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},+ {"name":"edit-answer_confidence","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"answer_confidence","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.1234,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}}+ ]+}diff --git a/laya-ts/scripts/predict_long_fixtures.py b/laya-ts/scripts/predict_long_fixtures.pynew file mode 100644index 00000000..fdb696c1--- /dev/null+++ b/laya-ts/scripts/predict_long_fixtures.py@@ -0,0 +1,307 @@+#!/usr/bin/env python3+"""Record the real Python ONNXAgent.predict_long contract without downloading models.++Generate with --python-root pointing at v0.4.18 (118d361a774c9a28cd02a78d6cf4d2af0614f422).+CI uses --check against the current Python tree: provenance may change, behavior may not.+Only tokenization and the ONNX session are synthetic; sequence building, decoding,+hook dispatch, OptionStabilityHook, budget checks and aggregation are production code.+This is a deterministic control-flow oracle, not a model-quality evaluation.+Requires numpy and CPU torch (imported by laya.common); no ONNX Runtime is needed.+"""+from __future__ import annotations++import argparse+from copy import deepcopy+import hashlib+import json+from pathlib import Path+import platform+import subprocess+import sys+import threading+from types import SimpleNamespace+import warnings++import numpy as np++ROOT = Path(__file__).resolve().parents[2]+FIXTURE = ROOT / "laya-ts/fixtures/predict-long.json"+REFERENCE_COMMIT = "118d361a774c9a28cd02a78d6cf4d2af0614f422"+++class CharacterTokenizer:+ """One Unicode codepoint per token; expansion deliberately models decode drift."""++ cls_token_id, sep_token_id, mask_token_id, pad_token_id = 1, 2, 3, 0+ mask_token = "[M]"++ def __init__(self, expansion=1):+ self.expansion = expansion++ def __call__(self, text, add_special_tokens=False, truncation=False, max_length=None):+ ids = [ord(char) + 10 for char in text]+ if truncation and max_length is not None:+ ids = ids[:max_length]+ return {"input_ids": ids}++ def decode(self, ids):+ return "".join(chr(int(token) - 10) * self.expansion for token in ids)+++class RecordingSession:+ def __init__(self, tie=False):+ self.tie = tie+ self.calls = []++ def run(self, names, inputs):+ ids, attention = inputs["input_ids"], inputs["attention_mask"]+ sums = ids.sum(axis=1).tolist()+ self.calls.append({"rows": len(ids), "tokens": attention.sum(axis=1).tolist(), "sums": sums})+ width = inputs["marker_mask"].shape[1]+ logits = [[2 if self.tie else 1 + total % 5] + [-slot for slot in range(1, width)]+ for total in sums]+ return [np.asarray(logits, dtype=np.float32),+ np.tile(np.array([[0.25, 0.75]], dtype=np.float32), (len(ids), 1))]+++def bare_agent(onnx_agent, case):+ agent = onnx_agent.__new__(onnx_agent)+ agent.model_id = "fixture"+ agent.cfg = deepcopy(case["cfg"])+ agent.tok = CharacterTokenizer(case.get("expansion", 1))+ agent.temperature = [1.0, 1.0, 1.0]+ agent.temperature_by_options = {}+ agent.lang_temperatures = {}+ agent.session = RecordingSession(case.get("tie", False))+ agent.hooks = ()+ agent.hooks_raise = True+ agent.hooks_concurrent = True+ agent.hooks_timeout = None+ agent._hooks_lock = None+ agent._hooks_mutex = threading.Lock()+ return agent+++def cases():+ questions = {+ "route": {"type": "choice", "instructions": "?", "criteria": {"a": "x", "b": "y", "c": "z"}},+ "urgent": {"type": "noul", "instructions": "?"},+ "grade": {"type": "score", "instructions": "?", "criteria": [0, {"level": "high"}]},+ }+ state = "ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789"++ def make(name, **changes):+ return {"name": name, "state": state, "questions": deepcopy(questions),+ "cfg": {"max_len": 96, "head_max_len": 40},+ "options": {"window": 16, "stride": 8, "batchSize": 2}, **changes}++ out = [make("all-types"), make("short", state="hi"),+ make("empty-state", state=""), make("structured-state", state={"text": "hello", "ok": True}),+ make("conversation", state=[{"role": "user", "content": state}]),+ make("mask-removal", state=state + "[M]" + state),+ make("default-window-and-cap", state=state * 3, options={}),+ make("explicit-window-capped", options={"window": 160, "stride": 100, "batchSize": 2}),+ make("tie-first-window", tie=True),+ make("decode-expansion-truncates", state=state * 2, expansion=2,+ cfg={"max_len": 64, "head_max_len": 32}, options={"window": 64}),+ make("empty-questions", questions={}),+ make("unsupported-aggregate", options={"aggregate": "mean"})]+ for behavior in ("reverse-results", "installed-end-reverse", "clone-results", "annotate", "shared-answer",+ "reverse-states", "clone-states", "rewrite-states", "empty-states",+ "remove-caller-all", "remove-caller-some", "temporary-remove-all",+ "temporary-remove-some", "rewrite-question-type", "clear-questions",+ "skip", "skip-many", "usage-fields", "shorten-results", "replace-batch",+ "replace-batch-added-removed"):+ out.append(make(behavior, behavior=behavior))+ out += [make("short-skip", state="hi", behavior="skip"),+ make("stability-long", behavior="stability"),+ make("stability-short", state="hi", behavior="stability"),+ make("budget-after-forward", state=state * 2, behavior="budget",+ options={"window": 48, "hooksRaise": False}),+ make("short-budget-after-forward", state=state[:35], behavior="budget", options={}),+ make("in-place-question-budget", state=state * 2, behavior="widen",+ options={"hooksRaise": False}, cfg={"max_len": 64, "head_max_len": 32},+ questions={"route": {"type": "choice", "instructions": "?", "criteria": {"a": "x", "b": "y"}}})]+ for field, qid in (("type", "route"), ("choice", "route"), ("noul", "urgent"),+ ("score", "grade"), ("legend", "grade"), ("probabilities", "route"),+ ("confidence", "route"), ("answer_confidence", "route")):+ out.append(make("edit-" + field, behavior="edit", field=field, qid=qid))+ return out+++def hooks_for(case, stability_hook, events, cached):+ behavior = case.get("behavior")++ def start(ctx):+ if behavior == "reverse-states":+ ctx.states.reverse()+ elif behavior == "clone-states":+ ctx.states = deepcopy(ctx.states)+ elif behavior == "rewrite-states":+ ctx.states = ["rewritten"]+ elif behavior == "empty-states":+ ctx.states = []+ elif behavior in ("temporary-remove-all", "temporary-remove-some"):+ ctx.questions["temporary"] = {"type": "noul", "instructions": "?"}+ elif behavior == "rewrite-question-type":+ ctx.questions["route"] = {"type": "noul", "instructions": "?"}+ elif behavior == "clear-questions":+ ctx.questions = {}+ elif behavior in ("skip", "skip-many"):+ ctx.skip([cached] * (len(ctx.states) if behavior == "skip-many" else 1))+ elif behavior == "budget":+ ctx.max_len = 48+ elif behavior == "widen":+ ctx.questions["route"]["criteria"].update({"opt%02d" % i: "d" * 20 for i in range(8)})+ events.append({"phase": "start", "states": deepcopy(ctx.states), "questions": list(ctx.questions)})++ def end(ctx):+ events.append({"phase": "end", "results": len(ctx.results) if ctx.results is not None else None,+ "error": ctx.error is not None})+ if ctx.results is None:+ return+ if behavior == "reverse-results":+ ctx.results.reverse()+ elif behavior == "clone-results":+ ctx.results = deepcopy(ctx.results)+ elif behavior == "shorten-results":+ ctx.results.pop()+ elif behavior in ("remove-caller-all", "remove-caller-some", "temporary-remove-all", "temporary-remove-some"):+ qid = "temporary" if behavior.startswith("temporary") else "route"+ for row in ctx.results if behavior.endswith("all") else ctx.results[:1]:+ del row["answers"][qid]+ elif behavior == "annotate":+ for row in ctx.results:+ for answer in row["answers"].values():+ answer["reliability"] = {"note": "observer"}+ answer["action"]["note"] = "observer"+ elif behavior == "usage-fields":+ for index, row in enumerate(ctx.results):+ row["usage"].update({"extra": index + 1, "flag": True, "details": {str(index): index},+ "label": str(index), "output_tokens": 9})+ elif behavior == "edit":+ field, qid = case["field"], case["qid"]+ for row in ctx.results:+ answer = row["answers"][qid]+ if field == "type":+ # A valid end-hook rewrite: all score fields are present. The+ # captured question type still chooses the aggregation metric.+ answer.update(type="score", score=0.5, legend={"0": "low", "1": "high"})+ elif field == "choice":+ answer[field] = "edited"+ elif field in ("legend", "probabilities"):+ answer[field][next(iter(answer[field]))] = "edited" if field == "legend" else 0.1234+ else:+ answer[field] = 0.1234++ hooks = []+ if behavior == "stability":+ hooks.append(stability_hook(accept=None, escalate=None, probes="reorder",+ variants=["reversed"], seed="long-fixture"))+ return hooks, start, end+++def record(onnx_agent, stability_hook, case):+ agent = bare_agent(onnx_agent, case)+ behavior = case.get("behavior")+ if behavior == "installed-end-reverse":+ agent.hooks = [SimpleNamespace(on_predict_end=lambda ctx: ctx.results.reverse())]+ if behavior == "shared-answer":+ original = agent._infer_batch++ def shared(*args, **kwargs):+ rows = original(*args, **kwargs)+ for row in rows:+ row["answers"]["route"] = rows[0]["answers"]["route"]+ return rows++ agent._infer_batch = shared+ if behavior in ("replace-batch", "replace-batch-added-removed"):+ def replacement(states, questions, batch_size=None, lang=None, **kwargs):+ if behavior == "replace-batch-added-removed":+ questions["temporary"] = {"type": "noul", "instructions": "?"}+ rows = agent._infer_batch(states, questions, batch_size=batch_size, lang=lang)+ if behavior == "replace-batch-added-removed":+ for row in rows:+ del row["answers"]["temporary"]+ return rows++ agent.predict_batch = replacement+ events = []+ cached = {"model": "cache", "answers": {"cached": {"type": "noul", "noul": 0.8,+ "confidence": 0.8, "answer_confidence": 0.8, "action": {"act_probability": 0.5}}},+ "usage": {"input_tokens": 0, "output_tokens": 0}}+ initial_cache = deepcopy(cached)+ hooks, start, end = hooks_for(case, stability_hook, events, cached)+ options = {"batch_size" if key == "batchSize" else "hooks_raise" if key == "hooksRaise" else key: value+ for key, value in case["options"].items()}+ outcome = {}+ with warnings.catch_warnings(record=True) as captured:+ warnings.simplefilter("always")+ try:+ # Deliberately invoke the real public method, never a port or a copied helper.+ outcome["result"] = agent.predict_long(deepcopy(case["state"]), deepcopy(case["questions"]),+ hooks=hooks, on_predict_start=start,+ on_predict_end=end, **options)+ except (ValueError, TypeError) as exc:+ outcome["error"] = str(exc)+ outcome["warnings"] = [str(w.message) for w in captured]+ outcome["forwards"] = agent.session.calls+ outcome["events"] = events+ outcome["cache_unchanged"] = cached == initial_cache+ return outcome+++def provenance(root):+ files = ["laya/onnx_agent.py", "laya/agent.py", "laya/common.py", "laya/hooks.py", "laya/confidence.py"]+ commit = subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()+ dirty = subprocess.check_output(["git", "-C", str(root), "status", "--porcelain", "--", *files], text=True)+ return {"pinned_tag": "v0.4.18", "pinned_commit": REFERENCE_COMMIT, "commit": commit,+ "modified_reference_files": dirty.splitlines(), "python": platform.python_version(),+ "numpy": np.__version__, "method": "laya.onnx_agent.ONNXAgent.predict_long",+ "files": {name: hashlib.sha256((root / name).read_bytes()).hexdigest() for name in files}}+++def generate(root):+ sys.path.insert(0, str(root))+ from laya.hooks import OptionStabilityHook+ from laya.onnx_agent import ONNXAgent++ if Path(sys.modules[ONNXAgent.__module__].__file__).resolve() != root / "laya/onnx_agent.py":+ raise RuntimeError("Python imported a different laya tree than --python-root")+ return {"schema": "laya-predict-long/1", "reference": provenance(root),+ "cases": [{**case, "expected": record(ONNXAgent, OptionStabilityHook, case)} for case in cases()]}+++def main():+ parser = argparse.ArgumentParser(description=__doc__)+ parser.add_argument("--python-root", type=Path, default=ROOT)+ parser.add_argument("--fixture", type=Path, default=FIXTURE)+ parser.add_argument("--check", action="store_true")+ args = parser.parse_args()+ if not args.check:+ reference = provenance(args.python_root.resolve())+ if reference["commit"] != REFERENCE_COMMIT or reference["modified_reference_files"]:+ raise SystemExit("Recording requires a clean v0.4.18 reference at " + REFERENCE_COMMIT+ + "; use --python-root for that checkout, or --check to compare current behavior")+ result = generate(args.python_root.resolve())+ if args.check:+ expected = json.loads(args.fixture.read_text(encoding="utf-8"))+ actual = {key: value for key, value in result.items() if key != "reference"}+ wanted = {key: value for key, value in expected.items() if key != "reference"}+ if json.loads(json.dumps(actual)) != wanted:+ changed = [case["name"] for case, old in zip(result["cases"], expected["cases"]) if case != old]+ raise SystemExit("predict_long fixtures differ from this Python reference: " + ", ".join(changed))+ print("PASS: Python ONNXAgent.predict_long matches all committed fixtures")+ else:+ args.fixture.parent.mkdir(parents=True, exist_ok=True)+ header = json.dumps({key: value for key, value in result.items() if key != "cases"}, indent=2)[:-2]+ encoded = ",\n".join(" " + json.dumps(case, ensure_ascii=False, separators=(",", ":"))+ for case in result["cases"])+ args.fixture.write_text(header + ',\n "cases": [\n' + encoded + "\n ]\n}\n", encoding="utf-8")+ print(f"Wrote {args.fixture}: {len(result['cases'])} cases")+ print(json.dumps(result["reference"], sort_keys=True))+++if __name__ == "__main__":+ main()diff --git a/laya-ts/scripts/predict_long_onnx_parity.mjs b/laya-ts/scripts/predict_long_onnx_parity.mjsnew file mode 100644index 00000000..58b28fb8--- /dev/null+++ b/laya-ts/scripts/predict_long_onnx_parity.mjs@@ -0,0 +1,53 @@+import { readFile, writeFile } from 'node:fs/promises';+import { createReadStream } from 'node:fs';+import { createHash } from 'node:crypto';+import { basename, join } from 'node:path';+import { Agent } from '../dist/index.js';+const [input, output] = process.argv.slice(2);+if (!input || !output) throw new Error('usage: node scripts/predict_long_onnx_parity.mjs python.json comparison.json');+const report = JSON.parse(await readFile(input, 'utf8'));+for (const [name, expected] of Object.entries(report.sha256)) {+ if (basename(name) !== name) throw new Error(`invalid artifact name: ${name}`);+ const hash = createHash('sha256');+ for await (const chunk of createReadStream(join(report.bundle, name))) hash.update(chunk);+ if (hash.digest('hex') !== expected) throw new Error(`artifact changed since Python comparison: ${name}`);+}+const agent = await Agent.load(report.bundle, { device: 'cpu', numThreads: 2, expectedSha256: report.sha256 });+const provider = agent.provider;+const inputs = [];+agent.provider = { ...provider, async runEncoder(batch) {+ inputs.push(structuredClone({ input_ids: batch.inputIds, attention_mask: batch.attentionMask,+ marker_pos: batch.markerPos, marker_mask: batch.markerMask, qtype: batch.qtype }));+ return provider.runEncoder(batch);+} };+let maxAnswerDelta = 0;+let compared = 0;+for (const c of report.cases) {+ inputs.length = 0;+ const began = performance.now();+ c.typescript = await agent.predictLong(c.state, c.questions, c.options);+ c.typescript_inputs = structuredClone(inputs);+ c.typescript_seconds = (performance.now() - began) / 1000;+ c.differences = [];+ function compare(a,b,path) {+ if (typeof a === 'number' && typeof b === 'number') {+ const delta = Math.abs(a-b);+ if (path.startsWith('answers.') && !path.includes('.window.')) maxAnswerDelta = Math.max(maxAnswerDelta, delta);+ if (!Number.isFinite(a) || !Number.isFinite(b) || delta > 1e-4) c.differences.push({path,python:a,typescript:b,delta});+ compared++;+ } else if (a !== null && b !== null && typeof a === 'object' && typeof b === 'object') {+ for (const key of new Set([...Object.keys(a),...Object.keys(b)])) compare(a[key],b[key],path?`${path}.${key}`:key);+ } else if (a !== b && path !== 'model') c.differences.push({path,python:a,typescript:b});+ }+ compare(c.python_inputs,c.typescript_inputs,'inputs');+ compare(c.python,c.typescript,'');+ console.log(c.name, JSON.stringify(c.typescript.usage), 'differences', c.differences.length);+}+report.environment.node = process.version;+report.environment.node_onnxruntime = (await import('onnxruntime-node')).env.versions.node;+report.comparison = { tolerance: 1e-4, max_answer_absolute_delta: maxAnswerDelta, numeric_fields_compared: compared,+ model_name_difference: 'Existing TS laya-rl-agent vs Python laya-rl-agent-onnx',+ differences: report.cases.reduce((n,c)=>n+c.differences.length,0) };+await writeFile(output, JSON.stringify(report,null,2));+console.log(JSON.stringify(report.comparison));+if(report.comparison.differences) process.exitCode = 1;diff --git a/laya-ts/scripts/predict_long_onnx_parity.py b/laya-ts/scripts/predict_long_onnx_parity.pynew file mode 100644index 00000000..e1d10979--- /dev/null+++ b/laya-ts/scripts/predict_long_onnx_parity.py@@ -0,0 +1,105 @@+"""Matched split-graph ONNX parity; real Python ONNXAgent.predict_long, CPU only."""+import argparse+import hashlib+import json+import os+from pathlib import Path+import platform+import subprocess+import sys+import threading+import time++parser = argparse.ArgumentParser(description=__doc__)+parser.add_argument("--python-root", type=Path, required=True, help="Checkout of Python v0.4.18")+parser.add_argument("--checkpoint", type=Path, required=True, help="Original English checkpoint, including tokenizer/")+parser.add_argument("--bundle", type=Path, required=True, help="Split ONNX export of that exact checkpoint")+parser.add_argument("--output", type=Path, required=True)+args = parser.parse_args()+REFERENCE = args.python_root.resolve()+sys.path.insert(0, str(REFERENCE))+os.environ["HF_HUB_OFFLINE"] = "1"+os.environ["TRANSFORMERS_OFFLINE"] = "1"+import numpy as np+import onnxruntime as ort+import transformers+from transformers import AutoTokenizer+import laya+from laya.onnx_agent import ONNXAgent+from laya.common import clamp_temperature++BUNDLE = args.bundle.resolve()+CHECKPOINT = args.checkpoint.resolve()+if laya.__version__ != "0.4.18":+ raise SystemExit("--python-root must point at the pinned Python 0.4.18 reference")+reference_commit = subprocess.check_output(["git", "-C", str(REFERENCE), "rev-parse", "HEAD"], text=True).strip()+if reference_commit != "118d361a774c9a28cd02a78d6cf4d2af0614f422":+ raise SystemExit("--python-root must be checked out at v0.4.18 commit 118d361a774c9a28cd02a78d6cf4d2af0614f422")+if subprocess.check_output(["git", "-C", str(REFERENCE), "status", "--porcelain", "--", "laya"], text=True).strip():+ raise SystemExit("--python-root must have an unmodified laya/ source tree")+if (CHECKPOINT / "tokenizer/tokenizer.json").read_bytes() != (BUNDLE / "tokenizer.json").read_bytes():+ raise SystemExit("checkpoint and ONNX bundle must use the same tokenizer.json")+so = ort.SessionOptions()+so.intra_op_num_threads = 2+so.inter_op_num_threads = 1+encoder = ort.InferenceSession(str(BUNDLE / "encoder.onnx"), sess_options=so, providers=["CPUExecutionProvider"])+head = ort.InferenceSession(str(BUNDLE / "head.onnx"), sess_options=so, providers=["CPUExecutionProvider"])+calls = []+class SplitSession:+ def run(self, names, feeds):+ calls.append({key: value.tolist() for key, value in feeds.items()})+ hidden = encoder.run(None, {k: feeds[k] for k in ("input_ids", "attention_mask")})[0]+ hf = {k: feeds[k] for k in ("marker_pos", "marker_mask", "attention_mask")}+ hf.update(hidden_states=hidden, qtype=feeds["qtype"].reshape(-1, 1))+ return head.run(names, hf)++agent = ONNXAgent.__new__(ONNXAgent)+agent.cfg = json.loads((BUNDLE / "rl_agent_config.json").read_text())+agent.tok = AutoTokenizer.from_pretrained(str(CHECKPOINT / "tokenizer"), local_files_only=True)+agent.session = SplitSession()+agent.hooks = []+agent._hooks_mutex = threading.Lock()+agent.temperature_raw = agent.cfg["temperature"]+agent.temperature = [clamp_temperature(x) for x in agent.temperature_raw]+agent.temperature_by_options_raw = agent.cfg.get("temperature_by_options", {})+agent.temperature_by_options = {k: clamp_temperature(v) for k, v in agent.temperature_by_options_raw.items()}+agent.lang_temperatures = {}+agent.binning_map = None+questions = {+ "action": {"type": "choice", "instructions": "What should support do?", "criteria": ["refund", "investigate", "close"]},+ "urgent": {"type": "noul", "instructions": "Is urgent action required?"},+ "severity": {"type": "score", "instructions": "How severe is the problem?", "criteria": ["minor", "moderate", "severe"]},+}+long = "This customer has asked for an update on the account. " * 8 + "The account was charged twice. Please refund the duplicate payment immediately."+cases = [+ {"name": "short", "state": "Please refund the duplicate charge.", "questions": questions, "options": {}},+ {"name": "long-overlap-capped", "state": long, "questions": questions, "options": {"window": 64, "stride": 32, "batchSize": 2}},+ {"name": "structured", "state": {"ticket": long, "priority": "urgent"}, "questions": questions, "options": {"window": 96, "stride": 48, "batchSize": 2}},+]+for case in cases:+ calls.clear()+ opts = dict(case["options"])+ if "batchSize" in opts:+ opts["batch_size"] = opts.pop("batchSize")+ began = time.perf_counter()+ case["python"] = agent.predict_long(case["state"], case["questions"], **opts)+ case["python_inputs"] = list(calls)+ case["python_seconds"] = time.perf_counter() - began+ print(case["name"], case["python"]["usage"], flush=True)+def sha(path):+ h = hashlib.sha256()+ with path.open("rb") as f:+ for block in iter(lambda: f.read(1024 * 1024), b""):+ h.update(block)+ return h.hexdigest()+report = {+ "reference": {"tag": "v0.4.18", "commit": reference_commit,+ "sources": {name: sha(REFERENCE / name) for name in ["laya/onnx_agent.py", "laya/agent.py", "laya/common.py", "laya/hooks.py"]}},+ "checkpoint": {"path": str(CHECKPOINT), "tokenizer_sha256": sha(CHECKPOINT / "tokenizer/tokenizer.json")},+ "bundle": str(BUNDLE),+ "sha256": {p.name: sha(p) for p in BUNDLE.iterdir() if p.is_file()},+ "environment": {"python": platform.python_version(), "onnxruntime": ort.__version__, "numpy": np.__version__, "transformers": transformers.__version__, "backend": "CPUExecutionProvider", "python_threads": 2},+ "method": "Real ONNXAgent.predict_long over an adapter joining the exact encoder/head graphs consumed by TS. No fake logits; no model-quality or latency benchmark.",+ "cases": cases,+}+args.output.write_text(json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False), encoding="utf-8")diff --git a/laya-ts/tests/predict-long-fixtures.test.ts b/laya-ts/tests/predict-long-fixtures.test.tsnew file mode 100644index 00000000..35bae0f1--- /dev/null+++ b/laya-ts/tests/predict-long-fixtures.test.ts@@ -0,0 +1,231 @@+import { readFileSync } from "node:fs";+import { describe, expect, it, vi } from "vitest";+import { Agent } from "../src/agent.js";+import { OptionStabilityHook } from "../src/option-stability.js";+import type { PredictContext } from "../src/hooks.js";+import type { Batch, SessionProvider } from "../src/providers.js";++// Generated by calling Python ONNXAgent.predict_long, including its real encoder,+// decoder and hook dispatcher. The fake provider is deterministic, not a model eval.+const fixtures = JSON.parse(readFileSync(new URL("../fixtures/predict-long.json", import.meta.url), "utf8"));+const clone = <T>(value: T): T => structuredClone(value);+const sum = (values: number[]) => values.reduce((left, right) => left + right, 0);++function setup(testCase: any) {+ const forwards: unknown[] = [];+ const provider: SessionProvider = {+ async runEncoder(batch: Batch) {+ forwards.push({ rows: batch.inputIds.length, tokens: batch.attentionMask.map(sum), sums: batch.inputIds.map(sum) });+ return { lastHidden: batch.inputIds.map(() => [[0]]) };+ },+ async runHead(_hidden, batch: Batch) {+ return {+ logits: batch.inputIds.map((row, index) => batch.markerMask[index].map((_, slot) =>+ slot === 0 ? (testCase.tie ? 2 : 1 + sum(row) % 5) : -slot)),+ act: batch.inputIds.map(() => [0.25, 0.75]),+ };+ },+ };+ const agent = new Agent({+ provider,+ cfg: clone(testCase.cfg),+ tok: {+ clsId: 1, sepId: 2, maskId: 3, padId: 0, maskToken: "[M]",+ encode: (text: string) => Array.from(text, (char) => char.codePointAt(0)! + 10),+ decode: (ids: number[]) => ids.map((id) => String.fromCodePoint(id - 10).repeat(testCase.expansion ?? 1)).join(""),+ },+ });+ if (testCase.behavior === "installed-end-reverse") {+ agent.addHook({ onPredictEnd(ctx) { ctx.results!.reverse(); } });+ }+ if (testCase.behavior === "shared-answer") {+ // A custom inference adapter sharing an answer object has no unique origin.+ // Install this before the public dispatcher records answers, matching Python.+ const original = (agent as any)._systemOneMany.bind(agent);+ (agent as any)._systemOneMany = async (...args: unknown[]) => {+ const rows = await original(...args);+ for (const row of rows) row.answers.route = rows[0].answers.route;+ return rows;+ };+ }+ if (["replace-batch", "replace-batch-added-removed"].includes(testCase.behavior)) {+ agent.predictBatch = async (states, questions, options = {}) => {+ if (testCase.behavior === "replace-batch-added-removed") {+ questions.temporary = { type: "noul", instructions: "?" };+ }+ const rows = await (agent as any)._systemOneMany(states, questions, options);+ if (testCase.behavior === "replace-batch-added-removed") {+ for (const row of rows) delete row.answers.temporary;+ }+ return rows;+ };+ }+ return { agent, forwards };+}++function hooksFor(testCase: any, events: unknown[], cached: any) {+ const behavior = testCase.behavior;+ const onPredictStart = (ctx: PredictContext) => {+ if (behavior === "reverse-states") ctx.states.reverse();+ else if (behavior === "clone-states") ctx.states = clone(ctx.states);+ else if (behavior === "rewrite-states") ctx.states = ["rewritten"];+ else if (behavior === "empty-states") ctx.states = [];+ else if (["temporary-remove-all", "temporary-remove-some"].includes(behavior)) {+ ctx.questions.temporary = { type: "noul", instructions: "?" };+ } else if (behavior === "rewrite-question-type") {+ ctx.questions.route = { type: "noul", instructions: "?" };+ } else if (behavior === "clear-questions") ctx.questions = {};+ else if (["skip", "skip-many"].includes(behavior)) {+ ctx.skip(Array.from({ length: behavior === "skip-many" ? ctx.states.length : 1 }, () => cached));+ } else if (behavior === "budget") ctx.maxLen = 48;+ else if (behavior === "widen") {+ const criteria = (ctx.questions.route as any).criteria;+ for (let i = 0; i < 8; i++) criteria[`opt${String(i).padStart(2, "0")}`] = "d".repeat(20);+ }+ events.push({ phase: "start", states: clone(ctx.states), questions: Object.keys(ctx.questions) });+ };+ const onPredictEnd = (ctx: PredictContext) => {+ events.push({ phase: "end", results: ctx.results?.length ?? null, error: ctx.error !== null });+ const rows = ctx.results as any[] | null;+ if (rows === null) return;+ if (behavior === "reverse-results") rows.reverse();+ else if (behavior === "clone-results") ctx.results = clone(rows);+ else if (behavior === "shorten-results") rows.pop();+ else if (["remove-caller-all", "remove-caller-some", "temporary-remove-all", "temporary-remove-some"].includes(behavior)) {+ const qid = behavior.startsWith("temporary") ? "temporary" : "route";+ for (const row of behavior.endsWith("all") ? rows : rows.slice(0, 1)) delete row.answers[qid];+ } else if (behavior === "annotate") {+ for (const row of rows) for (const answer of Object.values(row.answers) as any[]) {+ answer.reliability = { note: "observer" };+ answer.action.note = "observer";+ }+ } else if (behavior === "usage-fields") {+ rows.forEach((row, index) => Object.assign(row.usage, {+ extra: index + 1, flag: true, details: { [String(index)]: index }, label: String(index), output_tokens: 9,+ }));+ } else if (behavior === "edit") {+ const { field, qid } = testCase;+ for (const row of rows) {+ const answer = row.answers[qid];+ if (field === "type") Object.assign(answer, { type: "score", score: 0.5, legend: { "0": "low", "1": "high" } });+ else if (field === "choice") answer[field] = "edited";+ else if (field === "legend" || field === "probabilities") {+ answer[field][Object.keys(answer[field])[0]] = field === "legend" ? "edited" : 0.1234;+ } else answer[field] = 0.1234;+ }+ }+ };+ const hooks = behavior === "stability" ? [new OptionStabilityHook({+ accept: null, escalate: null, probes: "reorder", variants: ["reversed"], seed: "long-fixture",+ })] : [];+ return { hooks, onPredictStart, onPredictEnd };+}++function diagnostic(text: string): unknown {+ const normalized = text.replaceAll("predictLong", "predict_long").replaceAll("predictBatch", "predict_batch")+ .replaceAll("maxLen", "max_len").replaceAll("headMaxLen", "head_max_len");+ // Diagnostic spelling is language-specific; compare its category and concrete+ // parameters, not camelCase versus snake_case or the suggested next step's prose.+ if (normalized.includes("after the start hooks ran")) {+ return { kind: "budget", values: normalized.match(/\d+/g)?.map(Number) };+ }+ const missing = normalized.match(/answer ['"](.*?)['"] is missing from (\d+) of (\d+) windows/);+ if (missing) return { kind: "missing-answer", qid: missing[1], missing: Number(missing[2]), total: Number(missing[3]) };+ const capPython = normalized.match(/window=(\d+) is wider than the (\d+) state tokens.*max_len=(\d+)/);+ const capDefault = normalized.match(/leave only (\d+) of max_len=(\d+).*capped (\d+) -> (\d+)/);+ const capTS = normalized.match(/questions leave (\d+) state tokens inside max_len=(\d+); capping window (\d+) -> (\d+)/);+ if (capPython) return { kind: "window-cap", requested: +capPython[1], room: +capPython[2], maxLen: +capPython[3], effective: +capPython[2] };+ if (capDefault) return { kind: "window-cap", requested: +capDefault[3], room: +capDefault[1], maxLen: +capDefault[2], effective: +capDefault[4] };+ if (capTS) return { kind: "window-cap", requested: +capTS[3], room: +capTS[1], maxLen: +capTS[2], effective: +capTS[4] };+ const stridePython = normalized.match(/stride=(\d+).*window was reduced to (\d+).*stride=(\d+) instead/);+ const strideTS = normalized.match(/window was reduced to (\d+); adjusting stride (\d+) -> (\d+)/);+ if (stridePython) return { kind: "stride", before: +stridePython[1], window: +stridePython[2], after: +stridePython[3] };+ if (strideTS) return { kind: "stride", before: +strideTS[2], window: +strideTS[1], after: +strideTS[3] };+ if (normalized.includes("a hook answered the state before it was scanned, so no window decided the result")) {+ return { kind: "skip-unattributed" };+ }+ return normalized;+}++function sameContract(actual: any, expected: any, path = "result") {+ // Python uses float32 ONNX outputs; JS computes softmax in double precision.+ // Public values are rounded to four decimals. Nothing structural is relaxed.+ if (typeof expected === "number" && !Number.isInteger(expected)) {+ expect(typeof actual, path).toBe("number");+ expect(Math.abs(actual - expected), path).toBeLessThanOrEqual(0.00010001);+ } else if (Array.isArray(expected)) {+ expect(Array.isArray(actual), path).toBe(true);+ expect(actual.length, path).toBe(expected.length);+ expected.forEach((value, index) => sameContract(actual[index], value, `${path}[${index}]`));+ } else if (expected !== null && typeof expected === "object") {+ expect(actual, path).not.toBeNull();+ expect(Object.keys(actual).sort(), path).toEqual(Object.keys(expected).sort());+ for (const key of Object.keys(expected)) sameContract(actual[key], expected[key], `${path}.${key}`);+ } else if (typeof expected === "string") {+ if (path.endsWith(".error") || path.includes(".warnings[")) {+ expect(diagnostic(actual), path).toEqual(diagnostic(expected));+ } else {+ // The published TS model identifier is unchanged by adding predictLong.+ const want = path.endsWith(".model") && expected === "laya-rl-agent-onnx" ? "laya-rl-agent" : expected;+ expect(actual, path).toBe(want);+ }+ } else {+ expect(actual, path).toBe(expected);+ }+}++describe("predictLong Python ONNX oracle", () => {+ it("records the approved clean Python reference and method", () => {+ expect(fixtures.reference.commit).toBe("118d361a774c9a28cd02a78d6cf4d2af0614f422");+ expect(fixtures.reference.modified_reference_files).toEqual([]);+ expect(fixtures.reference.method).toBe("laya.onnx_agent.ONNXAgent.predict_long");+ });++ for (const testCase of fixtures.cases) it(testCase.name, async () => {+ const { agent, forwards } = setup(testCase);+ const events: unknown[] = [];+ const cached = { model: "cache", answers: { cached: { type: "noul", noul: 0.8,+ confidence: 0.8, answer_confidence: 0.8, action: { act_probability: 0.5 } } }, usage: { input_tokens: 0, output_tokens: 0 } };+ const initialCache = clone(cached);+ const warnings: string[] = [];+ const warningSpy = vi.spyOn(console, "warn").mockImplementation((message) => warnings.push(String(message)));+ const actual: any = {};+ try {+ actual.result = await agent.predictLong(clone(testCase.state), clone(testCase.questions), {+ ...testCase.options, ...hooksFor(testCase, events, cached),+ });+ } catch (error) {+ actual.error = (error as Error).message;+ } finally {+ warningSpy.mockRestore();+ }+ actual.warnings = warnings;+ actual.forwards = forwards;+ actual.events = events;+ actual.cache_unchanged = JSON.stringify(cached) === JSON.stringify(initialCache);+ if (testCase.name === "short-skip") {+ // The approved TS contract explicitly warns on every skipped scan, including+ // this short path where Python v0.4.18 returns windows=0 without a warning.+ expect(warnings).toEqual(["laya: predictLong: a hook answered the state before it was scanned, so no window decided the result"]);+ actual.warnings = [];+ }+ sameContract(actual, testCase.expected, testCase.name);+ });++ it("the oracle actually exercises post-inference rejection and real truncation", () => {+ for (const name of ["budget-after-forward", "short-budget-after-forward", "in-place-question-budget"]) {+ const outcome = fixtures.cases.find((testCase: any) => testCase.name === name).expected;+ expect(outcome.error).toContain("after the start hooks ran");+ expect(outcome.forwards.length).toBeGreaterThan(0);+ expect(outcome.events.at(-1)).toMatchObject({ phase: "end", error: false });+ }+ const drift = fixtures.cases.find((testCase: any) => testCase.name === "decode-expansion-truncates").expected.result;+ expect(drift.usage.state_tokens_dropped).toBeGreaterThan(0);+ expect(drift.usage.truncated).toBeGreaterThan(0);+ const stability = fixtures.cases.find((testCase: any) => testCase.name === "stability-long").expected.result;+ expect(stability.answers.route.reliability.probes).toEqual(["reorder"]);+ expect(stability.answers.route.window).toBeDefined();+ const shared = fixtures.cases.find((testCase: any) => testCase.name === "shared-answer").expected.result;+ expect(shared.answers.route.window).toBeUndefined();+ });+});diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.pyindex 0a4bf045..d591a30b 100644--- a/tests/test_hooks_api.py+++ b/tests/test_hooks_api.py@@ -225,7 +225,8 @@ for label, fn in (("Agent.predict_batch", Agent.predict_batch),("Router.predict", Router.predict),("Router.predict_batch", Router.predict_batch),("Router.predict_long", Router.predict_long),- ("ONNXAgent.system_one", ONNXAgent.system_one)):+ ("ONNXAgent.system_one", ONNXAgent.system_one),+ ("ONNXAgent.predict_long", ONNXAgent.predict_long)):check_param(label, fn, "hooks", None)check_param(label, fn, "on_predict_start", None)check_param(label, fn, "on_predict_end", None)@@ -234,10 +235,16 @@ for label, fn in (("Agent.predict_batch", Agent.predict_batch),# The scan sizes its windows from the checkpoint budget, so it takes `window`/`stride` instead# of the per-call token overrides the single-window entries take.-check_param("Agent.predict_long", Agent.predict_long, "window", None)-check_param("Agent.predict_long", Agent.predict_long, "stride", None)-for param in ("max_len", "head_max_len"):- check("Agent.predict_long/%s not accepted" % param, param in sig(Agent.predict_long), False)+# The TypeScript predictLong port uses these same long-only controls. Keep the+# Python oracle's ONNX and Router surfaces pinned alongside the torch entry point.+for label, fn in (("Agent.predict_long", Agent.predict_long),+ ("ONNXAgent.predict_long", ONNXAgent.predict_long),+ ("Router.predict_long", Router.predict_long)):+ for param in ("window", "stride", "batch_size"):+ check_param(label, fn, param, None)+ check_param(label, fn, "aggregate", "auto")+ for param in ("max_len", "head_max_len", "min_confidence"):+ check("%s/%s not accepted" % (label, param), param in sig(fn), False)check_param("Agent.predict_batch", Agent.predict_batch, "batch_size", None)check_param("Agent.predict_batch", Agent.predict_batch, "sort_by_length", False)