Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

test(ts): pin predictLong parity and document scan semantics

wowec98c9230ff9
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ed634b64..a9c14ec2 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -41,6 +41,18 @@ jobs:
cache: npm
cache-dependency-path: laya-ts/package-lock.json
- run: npm ci
+ - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+ with:
+ python-version: "3.11"
+ cache: pip
+ - name: Install Python parity oracle dependencies
+ run: |
+ python -m pip install numpy tokenizers==0.23.2
+ python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
+ - name: Python long-prediction oracle drift check (no models)
+ run: python scripts/predict_long_fixtures.py --check
+ - name: Official tokenizer decode parity (no models)
+ run: python scripts/tokenizer_decode_fixtures.py --check
- name: Unit tests
run: npm test -- --run
- name: Packed package end-to-end test
diff --git a/laya-ts/README.md b/laya-ts/README.md
index 4451768c..0448a4e1 100644
--- a/laya-ts/README.md
+++ b/laya-ts/README.md
@@ -45,6 +45,83 @@ const out = await agent.predict("charged twice", {
`onnxruntime-node` / `onnxruntime-web` are optional peer deps, imported lazily behind the provider you use.
+## Long documents (`predictLong`)
+
+`Agent.predictLong` and `Router.predictLong` scan overlapping token windows locally, in Node
+or the browser. This lets a question consider a long ticket or conversation beyond the single
+window used by `predict`. The Router chooses and loads a checkpoint once for the document.
+
+```ts
+const scanned = await router.predictLong(longTicket, {
+ refund: { type: "noul", instructions: "Does the customer request a refund?" },
+ action: { type: "choice", instructions: "Choose the next action", criteria: ["refund", "investigate"] },
+}, { window: 256, stride: 128, batchSize: 4 });
+console.log(scanned.answers.refund, scanned.usage.windows, scanned.routing);
+```
+
+The only aggregation mode is `"auto"`: `noul` takes the largest value; `choice` and `score`
+take the answer with the highest `answer_confidence`. Ties retain the first result. The
+winner keeps its metadata. Its confidence is a window score, not calibrated confidence for
+the whole document; scanning more windows can change which answer wins.
+
+The window is capped by the space left by the questions in the checkpoint's configured
+sequence budget. Its default is `max(64, max_len - head_max_len - 8)`, then capped; stride
+defaults to half that effective window. An explicit stride cannot leave gaps. `window`,
+`stride`, and `batchSize` must be positive safe integers when supplied; null/undefined use
+defaults. `maxLen`, `headMaxLen`, and `minConfidence` are deliberately absent from the long
+options. Window sizing and aggregation have different meanings from single-pass truncation
+and confidence gating. Ordinary prediction options and result types remain unchanged.
+
+Windows follow Python 0.4.18's decode → batch prediction → re-encode path. The official
+English and multilingual tokenizers support decoding. A custom encode-only tokenizer still
+supports ordinary predictions and the short-input fast path; splitting requires `decode`
+and throws if it is unavailable. Decoding and re-encoding can change the token count,
+particularly for multilingual text, so normal truncation can still occur. An answer's
+optional `window.token_start` / `token_end` identifies its **original candidate token slice**,
+not a guarantee that exactly those tokens reached the model.
+
+Agent hooks wrap one window batch; Router hooks wrap one document. Process defaults run
+once. Reordering result objects preserves identifiable origins, while rewriting/reordering
+states, replacing an answer object, or editing prediction fields removes window attribution.
+In-place reliability/action annotations preserve it. An end hook may uniformly remove its own
+temporary questions, but cannot silently remove a caller's question. Inconsistent answer
+schemas or window counts fail. A skipping hook must return exactly one document result;
+the result and usage are copied, a warning is emitted, and `usage.windows` is zero.
+The warning also covers short-input and Router cache hits, as agreed in #1316; Python
+0.4.18 itself only warns on the multi-window Agent skip path. Hook results are validated
+against the TypeScript return contract, which also rejects malformed payloads Python accepts.
+
+If a start hook changes questions or token budgets so planned windows no longer fit, the
+scan is refused **after inference**. This check is outside hook error suppression and hook
+timeouts; the forward-pass cost has already been incurred. The short path checks this too.
+`LongResult` / `RoutedLongResult` have separate usage types: `truncated` is a boolean on the
+short path and a numeric count on the multi-window path. Numeric usage is summed, dictionaries
+are shallow-merged, other values come from the last result, and `output_tokens` stays zero.
+
+Deterministic parity fixtures are recorded directly from Python `ONNXAgent.predict_long`
+at v0.4.18, with source hashes, and replayed by TypeScript tests. The generator's `--check`
+detects later reference behavior changes; these checks establish control-flow parity, not
+model quality or a speed improvement.
+
+To repeat the optional real-inference comparison with an English checkpoint and its split
+ONNX export (weights are not downloaded by these scripts):
+
+```bash
+# From laya-ts/, with numpy, torch, transformers and onnxruntime installed in Python.
+python scripts/predict_long_onnx_parity.py --python-root /path/to/laya-v0.4.18 \
+ --checkpoint /path/to/english-checkpoint --bundle /path/to/split-onnx --output python-long.json
+npm run build
+node scripts/predict_long_onnx_parity.mjs python-long.json comparison.json
+```
+
+The Python harness calls `ONNXAgent.predict_long` with an adapter joining the same encoder
+and head graphs used by TypeScript. It records source/artifact hashes and checks all three
+answer types on short, overlapping long, and structured inputs. The Node harness verifies
+artifact digests and compares actual token/marker/mask inputs, decoded answers, attribution,
+and usage at absolute tolerance
+`1e-4`, excluding the existing runtime model-name difference. It does not measure retrieval
+quality, accuracy, or performance, and does not cover browser GPU numerical differences.
+
## Hooks (observe or shape every decision)
Port of the Python `laya.hooks` lifecycle. A hook is a `(ctx) => void` for `onPredictStart` /
diff --git a/laya-ts/fixtures/predict-long.json b/laya-ts/fixtures/predict-long.json
new file mode 100644
index 00000000..a7c3c570
--- /dev/null
+++ b/laya-ts/fixtures/predict-long.json
@@ -0,0 +1,68 @@
+{
+ "schema": "laya-predict-long/1",
+ "reference": {
+ "pinned_tag": "v0.4.18",
+ "pinned_commit": "118d361a774c9a28cd02a78d6cf4d2af0614f422",
+ "commit": "118d361a774c9a28cd02a78d6cf4d2af0614f422",
+ "modified_reference_files": [],
+ "python": "3.11.13",
+ "numpy": "2.4.6",
+ "method": "laya.onnx_agent.ONNXAgent.predict_long",
+ "files": {
+ "laya/onnx_agent.py": "6364c072750cbaf5a5d692bc64902b4233ed8982f13f99d3eb8e0c9531479ef4",
+ "laya/agent.py": "6c425362c589c2fb58a2f6fab19e1dbb0264a7216134323e4605be1b3f4b97f7",
+ "laya/common.py": "1c4525db9fcd3e98ff9ad8c064b3081b8b3c4d836a73e7cb186a9b00dd32781a",
+ "laya/hooks.py": "bf74bad91a559a291ee6bf2033a8b7b1dea8bed751304a1b8e0bd15bea7e9d62",
+ "laya/confidence.py": "f289d9688f3f7e84ea503a40236d2e5b6dcd05180b7c2fcae5a9d4c0b0f19b55"
+ }
+ },
+ "cases": [
+ {"name":"all-types","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"short","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.1192,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.8808,"1":0.1192},"confidence":0.4729,"answer_confidence":0.8808,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":134,"output_tokens":0,"state_tokens":2,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[42,46,46],"sums":[3290,4002,3820]}],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"empty-state","state":"","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.0067,"confidence":0.9933,"answer_confidence":0.9933,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":128,"output_tokens":0,"state_tokens":0,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[40,44,44],"sums":[3061,3773,3591]}],"events":[{"phase":"start","states":[""],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"structured-state","state":{"text":"hello","ok":true},"questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775},"window":{"index":2,"token_start":16,"token_end":29,"count":3}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":3}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":2,"token_start":16,"token_end":29,"count":3}}},"usage":{"input_tokens":519,"output_tokens":0,"state_tokens":45,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":3}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4555,5267,5085,4305,5017,4835]},{"rows":3,"tokens":[53,57,57],"sums":[4216,4928,4746]}],"events":[{"phase":"start","states":["{\"text\": \"hello\""," \"hello\", \"ok\": ",", \"ok\": true}"],"questions":["route","urgent","grade"]},{"phase":"end","results":3,"error":false}],"cache_unchanged":true}},
+ {"name":"conversation","state":[{"role":"user","content":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789"}],"questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9909,"b":0.0067,"c":0.0025},"confidence":0.9478,"answer_confidence":0.9909,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}},"grade":{"type":"score","score":0.0067,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9933,"1":0.0067},"confidence":0.942,"answer_confidence":0.9933,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":8,"token_end":24,"count":10}}},"usage":{"input_tokens":1760,"output_tokens":0,"state_tokens":160,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":10}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4542,5254,5072,4473,5185,5003]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4383,5095,4913,4307,5019,4837]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4436,5148,4966,4597,5309,5127]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4792,5504,5322,4976,5688,5506]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4750,5462,5280,4361,5073,4891]}],"events":[{"phase":"start","states":["[{\"role\": \"user\"",": \"user\", \"conte",", \"content\": \"AB","nt\": \"ABCDEGHIKL","CDEGHIKLMNOPQRST","MNOPQRSTUVXYZabc","UVXYZabcdeghikmn","deghikmnopqrstuv","opqrstuvxyz01234","xyz0123456789\"}]"],"questions":["route","urgent","grade"]},{"phase":"end","results":10,"error":false}],"cache_unchanged":true}},
+ {"name":"mask-removal","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789[M]ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":12,"token_start":96,"token_end":111,"count":13}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":13}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":12,"token_start":96,"token_end":111,"count":13}}},"usage":{"input_tokens":2285,"output_tokens":0,"state_tokens":207,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":13}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4866,5578,5396,4376,5088,4906]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4182,4894,4712,4400,5112,4930]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4551,5263,5081,4740,5452,5270]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4938,5650,5468,4866,5578,5396]},{"rows":3,"tokens":[55,59,59],"sums":[4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789 ","3456789 ABCDEGHI","ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":13,"error":false}],"cache_unchanged":true}},
+ {"name":"default-window-and-cap","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":26,"token_end":78,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":6}}},"usage":{"input_tokens":1653,"output_tokens":0,"state_tokens":295,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":18,"tokens":[92,96,96,92,96,96,92,96,96,92,96,96,92,96,96,75,79,79],"sums":[8134,8846,8664,8008,8720,8538,8143,8855,8673,8035,8747,8565,8152,8864,8682,6615,7327,7145]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456","deghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZ","789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123","abcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUV","456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0","XYZabcdeghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"explicit-window-capped","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":160,"stride":100,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}},"urgent":{"type":"noul","noul":0.0474,"confidence":0.9526,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":52,"count":2}}},"usage":{"input_tokens":499,"output_tokens":0,"state_tokens":81,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":2}},"warnings":["laya: predict_long: window=160 is wider than the 52 state tokens these questions leave inside max_len=96, so every window would be truncated to 52 on the way to the model; scanning with window=52 instead","laya: predict_long: stride=100 was a 50% step for the window=160 you asked for, but the window was reduced to 52 to fit the room these questions leave; scanning with stride=26 instead"],"forwards":[{"rows":6,"tokens":[92,96,96,69,73,73],"sums":[8134,8846,8664,5994,6706,6524]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456","deghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":2,"error":false}],"cache_unchanged":true}},
+ {"name":"tie-first-window","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"tie":true,"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9362,"b":0.0466,"c":0.0171},"confidence":0.7503,"answer_confidence":0.9362,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}},"urgent":{"type":"noul","noul":0.0474,"confidence":0.9526,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}},"grade":{"type":"score","score":0.0474,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9526,"1":0.0474},"confidence":0.7246,"answer_confidence":0.9526,"action":{"act_probability":0.3775},"window":{"index":0,"token_start":0,"token_end":16,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"decode-expansion-truncates","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":64,"head_max_len":32},"options":{"window":64},"expansion":2,"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":4,"token_start":56,"token_end":84,"count":7}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":70,"token_end":98,"count":7}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":1,"token_start":14,"token_end":42,"count":7}}},"usage":{"input_tokens":1344,"output_tokens":0,"state_tokens":388,"state_tokens_dropped":192,"truncated":7,"truncated_questions":["route","urgent","grade"],"options":{"grade":{"total":2,"distinct":1,"tokens_per_option":8}},"windows":7}},"warnings":["laya: predict_long: window=64 is wider than the 28 state tokens these questions leave inside max_len=64, so every window would be truncated to 28 on the way to the model; scanning with window=28 instead"],"forwards":[{"rows":21,"tokens":[64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64,64],"sums":[4907,5327,5375,5401,5821,5869,5983,6403,6451,4781,5201,5249,4939,5359,5407,5445,5865,5913,6017,6437,6485]}],"events":[{"phase":"start","states":["AABBCCDDEEGGHHIIKKLLMMNNOOPPQQRRSSTTUUVVXXYYZZaabbccddee","QQRRSSTTUUVVXXYYZZaabbccddeegghhiikkmmnnooppqqrrssttuuvv","gghhiikkmmnnooppqqrrssttuuvvxxyyzz00112233445566778899AA","xxyyzz00112233445566778899AABBCCDDEEGGHHIIKKLLMMNNOOPPQQ","BBCCDDEEGGHHIIKKLLMMNNOOPPQQRRSSTTUUVVXXYYZZaabbccddeegg","RRSSTTUUVVXXYYZZaabbccddeegghhiikkmmnnooppqqrrssttuuvvxx","hhiikkmmnnooppqqrrssttuuvvxxyyzz00112233445566778899"],"questions":["route","urgent","grade"]},{"phase":"end","results":7,"error":false}],"cache_unchanged":true}},
+ {"name":"empty-questions","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":6}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":[]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"unsupported-aggregate","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"aggregate":"mean"},"expected":{"error":"predict_long: only aggregate='auto' is supported","warnings":[],"forwards":[],"events":[],"cache_unchanged":true}},
+ {"name":"reverse-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"reverse-results","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"installed-end-reverse","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"installed-end-reverse","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"clone-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clone-results","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"annotate","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"annotate","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775,"note":"observer"},"reliability":{"note":"observer"},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"shared-answer","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"shared-answer","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"reverse-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"reverse-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[55,59,59,56,60,60],"sums":[4334,5046,4864,4866,5578,5396]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4938,5650,5468,4740,5452,5270]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4551,5263,5081,4400,5112,4930]}],"events":[{"phase":"start","states":["uvxyz0123456789","mnopqrstuvxyz012","bcdeghikmnopqrst","STUVXYZabcdeghik","KLMNOPQRSTUVXYZa","ABCDEGHIKLMNOPQR"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"clone-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clone-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"rewrite-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"rewrite-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9756,"b":0.0179,"c":0.0066},"confidence":0.8825,"answer_confidence":0.9756,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.0025,"confidence":0.9975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.018,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.982,"1":0.018},"confidence":0.87,"answer_confidence":0.982,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":155,"output_tokens":0,"state_tokens":9,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":3,"tokens":[49,53,53],"sums":[4147,4859,4677]}],"events":[{"phase":"start","states":["rewritten"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"empty-states","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"empty-states","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":["laya: predict_long: a start hook left no states to score, so the call aggregated nothing and returns no answers"],"forwards":[],"events":[{"phase":"start","states":[],"questions":["route","urgent","grade"]},{"phase":"end","results":0,"error":false}],"cache_unchanged":true}},
+ {"name":"remove-caller-all","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"remove-caller-all","expected":{"error":"predict_long: answer 'route' is missing from 6 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"remove-caller-some","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"remove-caller-some","expected":{"error":"predict_long: answer 'route' is missing from 1 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"temporary-remove-all","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"temporary-remove-all","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1412,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","temporary"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"temporary-remove-some","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"temporary-remove-some","expected":{"error":"predict_long: answer 'temporary' is missing from 1 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","temporary"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"rewrite-question-type","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"rewrite-question-type","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1077,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[60,60,60,60,60,60],"sums":[5112,5112,4930,5263,5263,5081]},{"rows":6,"tokens":[60,60,60,60,60,60],"sums":[5452,5452,5270,5650,5650,5468]},{"rows":6,"tokens":[60,60,60,59,59,59],"sums":[5578,5578,5396,5046,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"clear-questions","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"clear-questions","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{},"usage":{"input_tokens":0,"output_tokens":0,"windows":6}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":[]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"skip","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip","expected":{"result":{"model":"cache","answers":{"cached":{"type":"noul","noul":0.8,"confidence":0.8,"answer_confidence":0.8,"action":{"act_probability":0.5}}},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":["laya: predict_long: a hook answered the state before it was scanned, so no window decided the result and none is reported"],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"skip-many","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip-many","expected":{"error":"predict_long: a start hook answered this state with 6 results; ctx.skip() takes one result for the document, not one per window","warnings":[],"forwards":[],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"usage-fields","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"usage-fields","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"extra":21,"flag":6,"details":{"0":0,"1":1,"2":2,"3":3,"4":4,"5":5},"label":"5","windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"shorten-results","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"shorten-results","expected":{"error":"predict_long: the state was split into 6 windows and the call returned 5 results","warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"replace-batch","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"replace-batch","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[],"cache_unchanged":true}},
+ {"name":"replace-batch-added-removed","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"replace-batch-added-removed","expected":{"error":"predict_long: answer 'temporary' is missing from 6 of 6 windows after prediction hooks","warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4400,5112,4930,5112,4551,5263,5081,5263]},{"rows":8,"tokens":[56,60,60,60,56,60,60,60],"sums":[4740,5452,5270,5452,4938,5650,5468,5650]},{"rows":8,"tokens":[56,60,60,60,55,59,59,59],"sums":[4866,5578,5396,5578,4334,5046,4864,5046]}],"events":[],"cache_unchanged":true}},
+ {"name":"short-skip","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"skip","expected":{"result":{"model":"cache","answers":{"cached":{"type":"noul","noul":0.8,"confidence":0.8,"answer_confidence":0.8,"action":{"act_probability":0.5}}},"usage":{"input_tokens":0,"output_tokens":0,"windows":0}},"warnings":[],"forwards":[],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"stability-long","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"stability","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"reliability":{"probes":["reorder"],"option_layout":"sequential","decision":null,"soft_stability":0.49875,"stability":0.0,"rename_stability":null,"reorder_stability":0.0,"n_variants":2,"distinct_choices":["a","c"],"variant_choices":{"original":"a","reversed":"c"},"variant_support":{"original":0.9966,"reversed":0.0009}},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1388,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":8,"tokens":[56,60,60,56,56,60,60,56],"sums":[4400,5112,4930,4400,4551,5263,5081,4551]},{"rows":8,"tokens":[56,60,60,56,56,60,60,56],"sums":[4740,5452,5270,4740,4938,5650,5468,4938]},{"rows":8,"tokens":[56,60,60,56,55,59,59,55],"sums":[4866,5578,5396,4866,4334,5046,4864,4334]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade","route::rel1"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"stability-short","state":"hi","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"stability","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.8438,"action":{"act_probability":0.3775},"reliability":{"probes":["reorder"],"option_layout":"sequential","decision":null,"soft_stability":0.4429,"stability":0.0,"rename_stability":null,"reorder_stability":0.0,"n_variants":2,"distinct_choices":["a","c"],"variant_choices":{"original":"a","reversed":"c"},"variant_support":{"original":0.8438,"reversed":0.042}}},"urgent":{"type":"noul","noul":0.018,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.1192,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.8808,"1":0.1192},"confidence":0.4729,"answer_confidence":0.8808,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":176,"output_tokens":0,"state_tokens":2,"state_tokens_dropped":0,"truncated":false,"truncated_questions":[],"windows":1}},"warnings":[],"forwards":[{"rows":4,"tokens":[42,46,46,42],"sums":[3290,4002,3820,3290]}],"events":[{"phase":"start","states":["hi"],"questions":["route","urgent","grade","route::rel1"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"budget-after-forward","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":48,"hooksRaise":false},"behavior":"budget","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=48 head_max_len=40) leave 4 state tokens per window, but the scan was sized for 48 from the agent's config (max_len=96 head_max_len=40). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":[],"forwards":[{"rows":12,"tokens":[48,48,48,48,48,48,48,48,48,48,48,48],"sums":[3692,4079,3897,3958,4211,4029,3584,4023,3841,3858,4156,3974]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz012","bcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRS","3456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstu","TUVXYZabcdeghikmnopqrstuvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":4,"error":false}],"cache_unchanged":true}},
+ {"name":"short-budget-after-forward","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmno","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{},"behavior":"budget","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=48 head_max_len=40) leave 4 state tokens per window, but the scan was sized for 52 from the agent's config (max_len=96 head_max_len=40). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":[],"forwards":[{"rows":3,"tokens":[48,48,48],"sums":[3692,4079,3897]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmno"],"questions":["route","urgent","grade"]},{"phase":"end","results":1,"error":false}],"cache_unchanged":true}},
+ {"name":"in-place-question-budget","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y"}}},"cfg":{"max_len":64,"head_max_len":32},"options":{"hooksRaise":false},"behavior":"widen","expected":{"error":"predict_long: after the start hooks ran, the questions and token budget (max_len=64 head_max_len=32) leave 12 state tokens per window, but the scan was sized for 30 from the agent's config (max_len=64 head_max_len=32). Every window would be re-truncated and parts of the document would reach no model. Pass window=/stride= explicitly, or call predict_batch directly, if you need a start hook to change either.","warnings":["laya: predict_long: these questions leave only 30 of max_len=64 for the state (their heads take the rest), so the scan window is capped 64 -> 30 and roughly 2.1x as many windows -- each a full forward pass -- are needed to read the document. Fewer or shorter options, a larger max_len, or laya.shortlist.predict_shortlist for a large label set will all cost less than scanning at this width"],"forwards":[{"rows":7,"tokens":[64,64,64,64,64,64,64],"sums":[4573,4797,5063,4372,4643,4897,4981]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQRSTUVXYZabcdegh","RSTUVXYZabcdeghikmnopqrstuvxyz","ikmnopqrstuvxyz0123456789ABCDE","0123456789ABCDEGHIKLMNOPQRSTUV","GHIKLMNOPQRSTUVXYZabcdeghikmno","XYZabcdeghikmnopqrstuvxyz01234","pqrstuvxyz0123456789"],"questions":["route"]},{"phase":"end","results":7,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-type","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"type","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"score","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"score":0.5,"legend":{"0":"low","1":"high"}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-choice","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"choice","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"edited","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-noul","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"noul","qid":"urgent","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1234,"confidence":0.982,"answer_confidence":0.982,"action":{"act_probability":0.3775}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-score","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"score","qid":"grade","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.1234,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-legend","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"legend","qid":"grade","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"edited","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-probabilities","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"probabilities","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.1234,"b":0.0025,"c":0.0009},"confidence":0.9776,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-confidence","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"confidence","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.9966,"b":0.0025,"c":0.0009},"confidence":0.1234,"answer_confidence":0.9966,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}},
+ {"name":"edit-answer_confidence","state":"ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789","questions":{"route":{"type":"choice","instructions":"?","criteria":{"a":"x","b":"y","c":"z"}},"urgent":{"type":"noul","instructions":"?"},"grade":{"type":"score","instructions":"?","criteria":[0,{"level":"high"}]}},"cfg":{"max_len":96,"head_max_len":40},"options":{"window":16,"stride":8,"batchSize":2},"behavior":"edit","field":"answer_confidence","qid":"route","expected":{"result":{"model":"laya-rl-agent-onnx","answers":{"route":{"type":"choice","choice":"a","probabilities":{"a":0.8438,"b":0.1142,"c":0.042},"confidence":0.5228,"answer_confidence":0.1234,"action":{"act_probability":0.3775}},"urgent":{"type":"noul","noul":0.1192,"confidence":0.8808,"answer_confidence":0.8808,"action":{"act_probability":0.3775},"window":{"index":3,"token_start":24,"token_end":40,"count":6}},"grade":{"type":"score","score":0.0025,"legend":{"0":"0","1":"{\"level\": \"high\"}"},"probabilities":{"0":0.9975,"1":0.0025},"confidence":0.975,"answer_confidence":0.9975,"action":{"act_probability":0.3775},"window":{"index":5,"token_start":40,"token_end":55,"count":6}}},"usage":{"input_tokens":1053,"output_tokens":0,"state_tokens":95,"state_tokens_dropped":0,"truncated":0,"truncated_questions":[],"windows":6}},"warnings":[],"forwards":[{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4400,5112,4930,4551,5263,5081]},{"rows":6,"tokens":[56,60,60,56,60,60],"sums":[4740,5452,5270,4938,5650,5468]},{"rows":6,"tokens":[56,60,60,55,59,59],"sums":[4866,5578,5396,4334,5046,4864]}],"events":[{"phase":"start","states":["ABCDEGHIKLMNOPQR","KLMNOPQRSTUVXYZa","STUVXYZabcdeghik","bcdeghikmnopqrst","mnopqrstuvxyz012","uvxyz0123456789"],"questions":["route","urgent","grade"]},{"phase":"end","results":6,"error":false}],"cache_unchanged":true}}
+ ]
+}
diff --git a/laya-ts/scripts/predict_long_fixtures.py b/laya-ts/scripts/predict_long_fixtures.py
new file mode 100644
index 00000000..fdb696c1
--- /dev/null
+++ b/laya-ts/scripts/predict_long_fixtures.py
@@ -0,0 +1,307 @@
+#!/usr/bin/env python3
+"""Record the real Python ONNXAgent.predict_long contract without downloading models.
+
+Generate with --python-root pointing at v0.4.18 (118d361a774c9a28cd02a78d6cf4d2af0614f422).
+CI uses --check against the current Python tree: provenance may change, behavior may not.
+Only tokenization and the ONNX session are synthetic; sequence building, decoding,
+hook dispatch, OptionStabilityHook, budget checks and aggregation are production code.
+This is a deterministic control-flow oracle, not a model-quality evaluation.
+Requires numpy and CPU torch (imported by laya.common); no ONNX Runtime is needed.
+"""
+from __future__ import annotations
+
+import argparse
+from copy import deepcopy
+import hashlib
+import json
+from pathlib import Path
+import platform
+import subprocess
+import sys
+import threading
+from types import SimpleNamespace
+import warnings
+
+import numpy as np
+
+ROOT = Path(__file__).resolve().parents[2]
+FIXTURE = ROOT / "laya-ts/fixtures/predict-long.json"
+REFERENCE_COMMIT = "118d361a774c9a28cd02a78d6cf4d2af0614f422"
+
+
+class CharacterTokenizer:
+ """One Unicode codepoint per token; expansion deliberately models decode drift."""
+
+ cls_token_id, sep_token_id, mask_token_id, pad_token_id = 1, 2, 3, 0
+ mask_token = "[M]"
+
+ def __init__(self, expansion=1):
+ self.expansion = expansion
+
+ def __call__(self, text, add_special_tokens=False, truncation=False, max_length=None):
+ ids = [ord(char) + 10 for char in text]
+ if truncation and max_length is not None:
+ ids = ids[:max_length]
+ return {"input_ids": ids}
+
+ def decode(self, ids):
+ return "".join(chr(int(token) - 10) * self.expansion for token in ids)
+
+
+class RecordingSession:
+ def __init__(self, tie=False):
+ self.tie = tie
+ self.calls = []
+
+ def run(self, names, inputs):
+ ids, attention = inputs["input_ids"], inputs["attention_mask"]
+ sums = ids.sum(axis=1).tolist()
+ self.calls.append({"rows": len(ids), "tokens": attention.sum(axis=1).tolist(), "sums": sums})
+ width = inputs["marker_mask"].shape[1]
+ logits = [[2 if self.tie else 1 + total % 5] + [-slot for slot in range(1, width)]
+ for total in sums]
+ return [np.asarray(logits, dtype=np.float32),
+ np.tile(np.array([[0.25, 0.75]], dtype=np.float32), (len(ids), 1))]
+
+
+def bare_agent(onnx_agent, case):
+ agent = onnx_agent.__new__(onnx_agent)
+ agent.model_id = "fixture"
+ agent.cfg = deepcopy(case["cfg"])
+ agent.tok = CharacterTokenizer(case.get("expansion", 1))
+ agent.temperature = [1.0, 1.0, 1.0]
+ agent.temperature_by_options = {}
+ agent.lang_temperatures = {}
+ agent.session = RecordingSession(case.get("tie", False))
+ agent.hooks = ()
+ agent.hooks_raise = True
+ agent.hooks_concurrent = True
+ agent.hooks_timeout = None
+ agent._hooks_lock = None
+ agent._hooks_mutex = threading.Lock()
+ return agent
+
+
+def cases():
+ questions = {
+ "route": {"type": "choice", "instructions": "?", "criteria": {"a": "x", "b": "y", "c": "z"}},
+ "urgent": {"type": "noul", "instructions": "?"},
+ "grade": {"type": "score", "instructions": "?", "criteria": [0, {"level": "high"}]},
+ }
+ state = "ABCDEGHIKLMNOPQRSTUVXYZabcdeghikmnopqrstuvxyz0123456789"
+
+ def make(name, **changes):
+ return {"name": name, "state": state, "questions": deepcopy(questions),
+ "cfg": {"max_len": 96, "head_max_len": 40},
+ "options": {"window": 16, "stride": 8, "batchSize": 2}, **changes}
+
+ out = [make("all-types"), make("short", state="hi"),
+ make("empty-state", state=""), make("structured-state", state={"text": "hello", "ok": True}),
+ make("conversation", state=[{"role": "user", "content": state}]),
+ make("mask-removal", state=state + "[M]" + state),
+ make("default-window-and-cap", state=state * 3, options={}),
+ make("explicit-window-capped", options={"window": 160, "stride": 100, "batchSize": 2}),
+ make("tie-first-window", tie=True),
+ make("decode-expansion-truncates", state=state * 2, expansion=2,
+ cfg={"max_len": 64, "head_max_len": 32}, options={"window": 64}),
+ make("empty-questions", questions={}),
+ make("unsupported-aggregate", options={"aggregate": "mean"})]
+ for behavior in ("reverse-results", "installed-end-reverse", "clone-results", "annotate", "shared-answer",
+ "reverse-states", "clone-states", "rewrite-states", "empty-states",
+ "remove-caller-all", "remove-caller-some", "temporary-remove-all",
+ "temporary-remove-some", "rewrite-question-type", "clear-questions",
+ "skip", "skip-many", "usage-fields", "shorten-results", "replace-batch",
+ "replace-batch-added-removed"):
+ out.append(make(behavior, behavior=behavior))
+ out += [make("short-skip", state="hi", behavior="skip"),
+ make("stability-long", behavior="stability"),
+ make("stability-short", state="hi", behavior="stability"),
+ make("budget-after-forward", state=state * 2, behavior="budget",
+ options={"window": 48, "hooksRaise": False}),
+ make("short-budget-after-forward", state=state[:35], behavior="budget", options={}),
+ make("in-place-question-budget", state=state * 2, behavior="widen",
+ options={"hooksRaise": False}, cfg={"max_len": 64, "head_max_len": 32},
+ questions={"route": {"type": "choice", "instructions": "?", "criteria": {"a": "x", "b": "y"}}})]
+ for field, qid in (("type", "route"), ("choice", "route"), ("noul", "urgent"),
+ ("score", "grade"), ("legend", "grade"), ("probabilities", "route"),
+ ("confidence", "route"), ("answer_confidence", "route")):
+ out.append(make("edit-" + field, behavior="edit", field=field, qid=qid))
+ return out
+
+
+def hooks_for(case, stability_hook, events, cached):
+ behavior = case.get("behavior")
+
+ def start(ctx):
+ if behavior == "reverse-states":
+ ctx.states.reverse()
+ elif behavior == "clone-states":
+ ctx.states = deepcopy(ctx.states)
+ elif behavior == "rewrite-states":
+ ctx.states = ["rewritten"]
+ elif behavior == "empty-states":
+ ctx.states = []
+ elif behavior in ("temporary-remove-all", "temporary-remove-some"):
+ ctx.questions["temporary"] = {"type": "noul", "instructions": "?"}
+ elif behavior == "rewrite-question-type":
+ ctx.questions["route"] = {"type": "noul", "instructions": "?"}
+ elif behavior == "clear-questions":
+ ctx.questions = {}
+ elif behavior in ("skip", "skip-many"):
+ ctx.skip([cached] * (len(ctx.states) if behavior == "skip-many" else 1))
+ elif behavior == "budget":
+ ctx.max_len = 48
+ elif behavior == "widen":
+ ctx.questions["route"]["criteria"].update({"opt%02d" % i: "d" * 20 for i in range(8)})
+ events.append({"phase": "start", "states": deepcopy(ctx.states), "questions": list(ctx.questions)})
+
+ def end(ctx):
+ events.append({"phase": "end", "results": len(ctx.results) if ctx.results is not None else None,
+ "error": ctx.error is not None})
+ if ctx.results is None:
+ return
+ if behavior == "reverse-results":
+ ctx.results.reverse()
+ elif behavior == "clone-results":
+ ctx.results = deepcopy(ctx.results)
+ elif behavior == "shorten-results":
+ ctx.results.pop()
+ elif behavior in ("remove-caller-all", "remove-caller-some", "temporary-remove-all", "temporary-remove-some"):
+ qid = "temporary" if behavior.startswith("temporary") else "route"
+ for row in ctx.results if behavior.endswith("all") else ctx.results[:1]:
+ del row["answers"][qid]
+ elif behavior == "annotate":
+ for row in ctx.results:
+ for answer in row["answers"].values():
+ answer["reliability"] = {"note": "observer"}
+ answer["action"]["note"] = "observer"
+ elif behavior == "usage-fields":
+ for index, row in enumerate(ctx.results):
+ row["usage"].update({"extra": index + 1, "flag": True, "details": {str(index): index},
+ "label": str(index), "output_tokens": 9})
+ elif behavior == "edit":
+ field, qid = case["field"], case["qid"]
+ for row in ctx.results:
+ answer = row["answers"][qid]
+ if field == "type":
+ # A valid end-hook rewrite: all score fields are present. The
+ # captured question type still chooses the aggregation metric.
+ answer.update(type="score", score=0.5, legend={"0": "low", "1": "high"})
+ elif field == "choice":
+ answer[field] = "edited"
+ elif field in ("legend", "probabilities"):
+ answer[field][next(iter(answer[field]))] = "edited" if field == "legend" else 0.1234
+ else:
+ answer[field] = 0.1234
+
+ hooks = []
+ if behavior == "stability":
+ hooks.append(stability_hook(accept=None, escalate=None, probes="reorder",
+ variants=["reversed"], seed="long-fixture"))
+ return hooks, start, end
+
+
+def record(onnx_agent, stability_hook, case):
+ agent = bare_agent(onnx_agent, case)
+ behavior = case.get("behavior")
+ if behavior == "installed-end-reverse":
+ agent.hooks = [SimpleNamespace(on_predict_end=lambda ctx: ctx.results.reverse())]
+ if behavior == "shared-answer":
+ original = agent._infer_batch
+
+ def shared(*args, **kwargs):
+ rows = original(*args, **kwargs)
+ for row in rows:
+ row["answers"]["route"] = rows[0]["answers"]["route"]
+ return rows
+
+ agent._infer_batch = shared
+ if behavior in ("replace-batch", "replace-batch-added-removed"):
+ def replacement(states, questions, batch_size=None, lang=None, **kwargs):
+ if behavior == "replace-batch-added-removed":
+ questions["temporary"] = {"type": "noul", "instructions": "?"}
+ rows = agent._infer_batch(states, questions, batch_size=batch_size, lang=lang)
+ if behavior == "replace-batch-added-removed":
+ for row in rows:
+ del row["answers"]["temporary"]
+ return rows
+
+ agent.predict_batch = replacement
+ events = []
+ cached = {"model": "cache", "answers": {"cached": {"type": "noul", "noul": 0.8,
+ "confidence": 0.8, "answer_confidence": 0.8, "action": {"act_probability": 0.5}}},
+ "usage": {"input_tokens": 0, "output_tokens": 0}}
+ initial_cache = deepcopy(cached)
+ hooks, start, end = hooks_for(case, stability_hook, events, cached)
+ options = {"batch_size" if key == "batchSize" else "hooks_raise" if key == "hooksRaise" else key: value
+ for key, value in case["options"].items()}
+ outcome = {}
+ with warnings.catch_warnings(record=True) as captured:
+ warnings.simplefilter("always")
+ try:
+ # Deliberately invoke the real public method, never a port or a copied helper.
+ outcome["result"] = agent.predict_long(deepcopy(case["state"]), deepcopy(case["questions"]),
+ hooks=hooks, on_predict_start=start,
+ on_predict_end=end, **options)
+ except (ValueError, TypeError) as exc:
+ outcome["error"] = str(exc)
+ outcome["warnings"] = [str(w.message) for w in captured]
+ outcome["forwards"] = agent.session.calls
+ outcome["events"] = events
+ outcome["cache_unchanged"] = cached == initial_cache
+ return outcome
+
+
+def provenance(root):
+ files = ["laya/onnx_agent.py", "laya/agent.py", "laya/common.py", "laya/hooks.py", "laya/confidence.py"]
+ commit = subprocess.check_output(["git", "-C", str(root), "rev-parse", "HEAD"], text=True).strip()
+ dirty = subprocess.check_output(["git", "-C", str(root), "status", "--porcelain", "--", *files], text=True)
+ return {"pinned_tag": "v0.4.18", "pinned_commit": REFERENCE_COMMIT, "commit": commit,
+ "modified_reference_files": dirty.splitlines(), "python": platform.python_version(),
+ "numpy": np.__version__, "method": "laya.onnx_agent.ONNXAgent.predict_long",
+ "files": {name: hashlib.sha256((root / name).read_bytes()).hexdigest() for name in files}}
+
+
+def generate(root):
+ sys.path.insert(0, str(root))
+ from laya.hooks import OptionStabilityHook
+ from laya.onnx_agent import ONNXAgent
+
+ if Path(sys.modules[ONNXAgent.__module__].__file__).resolve() != root / "laya/onnx_agent.py":
+ raise RuntimeError("Python imported a different laya tree than --python-root")
+ return {"schema": "laya-predict-long/1", "reference": provenance(root),
+ "cases": [{**case, "expected": record(ONNXAgent, OptionStabilityHook, case)} for case in cases()]}
+
+
+def main():
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument("--python-root", type=Path, default=ROOT)
+ parser.add_argument("--fixture", type=Path, default=FIXTURE)
+ parser.add_argument("--check", action="store_true")
+ args = parser.parse_args()
+ if not args.check:
+ reference = provenance(args.python_root.resolve())
+ if reference["commit"] != REFERENCE_COMMIT or reference["modified_reference_files"]:
+ raise SystemExit("Recording requires a clean v0.4.18 reference at " + REFERENCE_COMMIT
+ + "; use --python-root for that checkout, or --check to compare current behavior")
+ result = generate(args.python_root.resolve())
+ if args.check:
+ expected = json.loads(args.fixture.read_text(encoding="utf-8"))
+ actual = {key: value for key, value in result.items() if key != "reference"}
+ wanted = {key: value for key, value in expected.items() if key != "reference"}
+ if json.loads(json.dumps(actual)) != wanted:
+ changed = [case["name"] for case, old in zip(result["cases"], expected["cases"]) if case != old]
+ raise SystemExit("predict_long fixtures differ from this Python reference: " + ", ".join(changed))
+ print("PASS: Python ONNXAgent.predict_long matches all committed fixtures")
+ else:
+ args.fixture.parent.mkdir(parents=True, exist_ok=True)
+ header = json.dumps({key: value for key, value in result.items() if key != "cases"}, indent=2)[:-2]
+ encoded = ",\n".join(" " + json.dumps(case, ensure_ascii=False, separators=(",", ":"))
+ for case in result["cases"])
+ args.fixture.write_text(header + ',\n "cases": [\n' + encoded + "\n ]\n}\n", encoding="utf-8")
+ print(f"Wrote {args.fixture}: {len(result['cases'])} cases")
+ print(json.dumps(result["reference"], sort_keys=True))
+
+
+if __name__ == "__main__":
+ main()
diff --git a/laya-ts/scripts/predict_long_onnx_parity.mjs b/laya-ts/scripts/predict_long_onnx_parity.mjs
new file mode 100644
index 00000000..58b28fb8
--- /dev/null
+++ b/laya-ts/scripts/predict_long_onnx_parity.mjs
@@ -0,0 +1,53 @@
+import { readFile, writeFile } from 'node:fs/promises';
+import { createReadStream } from 'node:fs';
+import { createHash } from 'node:crypto';
+import { basename, join } from 'node:path';
+import { Agent } from '../dist/index.js';
+const [input, output] = process.argv.slice(2);
+if (!input || !output) throw new Error('usage: node scripts/predict_long_onnx_parity.mjs python.json comparison.json');
+const report = JSON.parse(await readFile(input, 'utf8'));
+for (const [name, expected] of Object.entries(report.sha256)) {
+ if (basename(name) !== name) throw new Error(`invalid artifact name: ${name}`);
+ const hash = createHash('sha256');
+ for await (const chunk of createReadStream(join(report.bundle, name))) hash.update(chunk);
+ if (hash.digest('hex') !== expected) throw new Error(`artifact changed since Python comparison: ${name}`);
+}
+const agent = await Agent.load(report.bundle, { device: 'cpu', numThreads: 2, expectedSha256: report.sha256 });
+const provider = agent.provider;
+const inputs = [];
+agent.provider = { ...provider, async runEncoder(batch) {
+ inputs.push(structuredClone({ input_ids: batch.inputIds, attention_mask: batch.attentionMask,
+ marker_pos: batch.markerPos, marker_mask: batch.markerMask, qtype: batch.qtype }));
+ return provider.runEncoder(batch);
+} };
+let maxAnswerDelta = 0;
+let compared = 0;
+for (const c of report.cases) {
+ inputs.length = 0;
+ const began = performance.now();
+ c.typescript = await agent.predictLong(c.state, c.questions, c.options);
+ c.typescript_inputs = structuredClone(inputs);
+ c.typescript_seconds = (performance.now() - began) / 1000;
+ c.differences = [];
+ function compare(a,b,path) {
+ if (typeof a === 'number' && typeof b === 'number') {
+ const delta = Math.abs(a-b);
+ if (path.startsWith('answers.') && !path.includes('.window.')) maxAnswerDelta = Math.max(maxAnswerDelta, delta);
+ if (!Number.isFinite(a) || !Number.isFinite(b) || delta > 1e-4) c.differences.push({path,python:a,typescript:b,delta});
+ compared++;
+ } else if (a !== null && b !== null && typeof a === 'object' && typeof b === 'object') {
+ for (const key of new Set([...Object.keys(a),...Object.keys(b)])) compare(a[key],b[key],path?`${path}.${key}`:key);
+ } else if (a !== b && path !== 'model') c.differences.push({path,python:a,typescript:b});
+ }
+ compare(c.python_inputs,c.typescript_inputs,'inputs');
+ compare(c.python,c.typescript,'');
+ console.log(c.name, JSON.stringify(c.typescript.usage), 'differences', c.differences.length);
+}
+report.environment.node = process.version;
+report.environment.node_onnxruntime = (await import('onnxruntime-node')).env.versions.node;
+report.comparison = { tolerance: 1e-4, max_answer_absolute_delta: maxAnswerDelta, numeric_fields_compared: compared,
+ model_name_difference: 'Existing TS laya-rl-agent vs Python laya-rl-agent-onnx',
+ differences: report.cases.reduce((n,c)=>n+c.differences.length,0) };
+await writeFile(output, JSON.stringify(report,null,2));
+console.log(JSON.stringify(report.comparison));
+if(report.comparison.differences) process.exitCode = 1;
diff --git a/laya-ts/scripts/predict_long_onnx_parity.py b/laya-ts/scripts/predict_long_onnx_parity.py
new file mode 100644
index 00000000..e1d10979
--- /dev/null
+++ b/laya-ts/scripts/predict_long_onnx_parity.py
@@ -0,0 +1,105 @@
+"""Matched split-graph ONNX parity; real Python ONNXAgent.predict_long, CPU only."""
+import argparse
+import hashlib
+import json
+import os
+from pathlib import Path
+import platform
+import subprocess
+import sys
+import threading
+import time
+
+parser = argparse.ArgumentParser(description=__doc__)
+parser.add_argument("--python-root", type=Path, required=True, help="Checkout of Python v0.4.18")
+parser.add_argument("--checkpoint", type=Path, required=True, help="Original English checkpoint, including tokenizer/")
+parser.add_argument("--bundle", type=Path, required=True, help="Split ONNX export of that exact checkpoint")
+parser.add_argument("--output", type=Path, required=True)
+args = parser.parse_args()
+REFERENCE = args.python_root.resolve()
+sys.path.insert(0, str(REFERENCE))
+os.environ["HF_HUB_OFFLINE"] = "1"
+os.environ["TRANSFORMERS_OFFLINE"] = "1"
+import numpy as np
+import onnxruntime as ort
+import transformers
+from transformers import AutoTokenizer
+import laya
+from laya.onnx_agent import ONNXAgent
+from laya.common import clamp_temperature
+
+BUNDLE = args.bundle.resolve()
+CHECKPOINT = args.checkpoint.resolve()
+if laya.__version__ != "0.4.18":
+ raise SystemExit("--python-root must point at the pinned Python 0.4.18 reference")
+reference_commit = subprocess.check_output(["git", "-C", str(REFERENCE), "rev-parse", "HEAD"], text=True).strip()
+if reference_commit != "118d361a774c9a28cd02a78d6cf4d2af0614f422":
+ raise SystemExit("--python-root must be checked out at v0.4.18 commit 118d361a774c9a28cd02a78d6cf4d2af0614f422")
+if subprocess.check_output(["git", "-C", str(REFERENCE), "status", "--porcelain", "--", "laya"], text=True).strip():
+ raise SystemExit("--python-root must have an unmodified laya/ source tree")
+if (CHECKPOINT / "tokenizer/tokenizer.json").read_bytes() != (BUNDLE / "tokenizer.json").read_bytes():
+ raise SystemExit("checkpoint and ONNX bundle must use the same tokenizer.json")
+so = ort.SessionOptions()
+so.intra_op_num_threads = 2
+so.inter_op_num_threads = 1
+encoder = ort.InferenceSession(str(BUNDLE / "encoder.onnx"), sess_options=so, providers=["CPUExecutionProvider"])
+head = ort.InferenceSession(str(BUNDLE / "head.onnx"), sess_options=so, providers=["CPUExecutionProvider"])
+calls = []
+class SplitSession:
+ def run(self, names, feeds):
+ calls.append({key: value.tolist() for key, value in feeds.items()})
+ hidden = encoder.run(None, {k: feeds[k] for k in ("input_ids", "attention_mask")})[0]
+ hf = {k: feeds[k] for k in ("marker_pos", "marker_mask", "attention_mask")}
+ hf.update(hidden_states=hidden, qtype=feeds["qtype"].reshape(-1, 1))
+ return head.run(names, hf)
+
+agent = ONNXAgent.__new__(ONNXAgent)
+agent.cfg = json.loads((BUNDLE / "rl_agent_config.json").read_text())
+agent.tok = AutoTokenizer.from_pretrained(str(CHECKPOINT / "tokenizer"), local_files_only=True)
+agent.session = SplitSession()
+agent.hooks = []
+agent._hooks_mutex = threading.Lock()
+agent.temperature_raw = agent.cfg["temperature"]
+agent.temperature = [clamp_temperature(x) for x in agent.temperature_raw]
+agent.temperature_by_options_raw = agent.cfg.get("temperature_by_options", {})
+agent.temperature_by_options = {k: clamp_temperature(v) for k, v in agent.temperature_by_options_raw.items()}
+agent.lang_temperatures = {}
+agent.binning_map = None
+questions = {
+ "action": {"type": "choice", "instructions": "What should support do?", "criteria": ["refund", "investigate", "close"]},
+ "urgent": {"type": "noul", "instructions": "Is urgent action required?"},
+ "severity": {"type": "score", "instructions": "How severe is the problem?", "criteria": ["minor", "moderate", "severe"]},
+}
+long = "This customer has asked for an update on the account. " * 8 + "The account was charged twice. Please refund the duplicate payment immediately."
+cases = [
+ {"name": "short", "state": "Please refund the duplicate charge.", "questions": questions, "options": {}},
+ {"name": "long-overlap-capped", "state": long, "questions": questions, "options": {"window": 64, "stride": 32, "batchSize": 2}},
+ {"name": "structured", "state": {"ticket": long, "priority": "urgent"}, "questions": questions, "options": {"window": 96, "stride": 48, "batchSize": 2}},
+]
+for case in cases:
+ calls.clear()
+ opts = dict(case["options"])
+ if "batchSize" in opts:
+ opts["batch_size"] = opts.pop("batchSize")
+ began = time.perf_counter()
+ case["python"] = agent.predict_long(case["state"], case["questions"], **opts)
+ case["python_inputs"] = list(calls)
+ case["python_seconds"] = time.perf_counter() - began
+ print(case["name"], case["python"]["usage"], flush=True)
+def sha(path):
+ h = hashlib.sha256()
+ with path.open("rb") as f:
+ for block in iter(lambda: f.read(1024 * 1024), b""):
+ h.update(block)
+ return h.hexdigest()
+report = {
+ "reference": {"tag": "v0.4.18", "commit": reference_commit,
+ "sources": {name: sha(REFERENCE / name) for name in ["laya/onnx_agent.py", "laya/agent.py", "laya/common.py", "laya/hooks.py"]}},
+ "checkpoint": {"path": str(CHECKPOINT), "tokenizer_sha256": sha(CHECKPOINT / "tokenizer/tokenizer.json")},
+ "bundle": str(BUNDLE),
+ "sha256": {p.name: sha(p) for p in BUNDLE.iterdir() if p.is_file()},
+ "environment": {"python": platform.python_version(), "onnxruntime": ort.__version__, "numpy": np.__version__, "transformers": transformers.__version__, "backend": "CPUExecutionProvider", "python_threads": 2},
+ "method": "Real ONNXAgent.predict_long over an adapter joining the exact encoder/head graphs consumed by TS. No fake logits; no model-quality or latency benchmark.",
+ "cases": cases,
+}
+args.output.write_text(json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False), encoding="utf-8")
diff --git a/laya-ts/tests/predict-long-fixtures.test.ts b/laya-ts/tests/predict-long-fixtures.test.ts
new file mode 100644
index 00000000..35bae0f1
--- /dev/null
+++ b/laya-ts/tests/predict-long-fixtures.test.ts
@@ -0,0 +1,231 @@
+import { readFileSync } from "node:fs";
+import { describe, expect, it, vi } from "vitest";
+import { Agent } from "../src/agent.js";
+import { OptionStabilityHook } from "../src/option-stability.js";
+import type { PredictContext } from "../src/hooks.js";
+import type { Batch, SessionProvider } from "../src/providers.js";
+
+// Generated by calling Python ONNXAgent.predict_long, including its real encoder,
+// decoder and hook dispatcher. The fake provider is deterministic, not a model eval.
+const fixtures = JSON.parse(readFileSync(new URL("../fixtures/predict-long.json", import.meta.url), "utf8"));
+const clone = <T>(value: T): T => structuredClone(value);
+const sum = (values: number[]) => values.reduce((left, right) => left + right, 0);
+
+function setup(testCase: any) {
+ const forwards: unknown[] = [];
+ const provider: SessionProvider = {
+ async runEncoder(batch: Batch) {
+ forwards.push({ rows: batch.inputIds.length, tokens: batch.attentionMask.map(sum), sums: batch.inputIds.map(sum) });
+ return { lastHidden: batch.inputIds.map(() => [[0]]) };
+ },
+ async runHead(_hidden, batch: Batch) {
+ return {
+ logits: batch.inputIds.map((row, index) => batch.markerMask[index].map((_, slot) =>
+ slot === 0 ? (testCase.tie ? 2 : 1 + sum(row) % 5) : -slot)),
+ act: batch.inputIds.map(() => [0.25, 0.75]),
+ };
+ },
+ };
+ const agent = new Agent({
+ provider,
+ cfg: clone(testCase.cfg),
+ tok: {
+ clsId: 1, sepId: 2, maskId: 3, padId: 0, maskToken: "[M]",
+ encode: (text: string) => Array.from(text, (char) => char.codePointAt(0)! + 10),
+ decode: (ids: number[]) => ids.map((id) => String.fromCodePoint(id - 10).repeat(testCase.expansion ?? 1)).join(""),
+ },
+ });
+ if (testCase.behavior === "installed-end-reverse") {
+ agent.addHook({ onPredictEnd(ctx) { ctx.results!.reverse(); } });
+ }
+ if (testCase.behavior === "shared-answer") {
+ // A custom inference adapter sharing an answer object has no unique origin.
+ // Install this before the public dispatcher records answers, matching Python.
+ const original = (agent as any)._systemOneMany.bind(agent);
+ (agent as any)._systemOneMany = async (...args: unknown[]) => {
+ const rows = await original(...args);
+ for (const row of rows) row.answers.route = rows[0].answers.route;
+ return rows;
+ };
+ }
+ if (["replace-batch", "replace-batch-added-removed"].includes(testCase.behavior)) {
+ agent.predictBatch = async (states, questions, options = {}) => {
+ if (testCase.behavior === "replace-batch-added-removed") {
+ questions.temporary = { type: "noul", instructions: "?" };
+ }
+ const rows = await (agent as any)._systemOneMany(states, questions, options);
+ if (testCase.behavior === "replace-batch-added-removed") {
+ for (const row of rows) delete row.answers.temporary;
+ }
+ return rows;
+ };
+ }
+ return { agent, forwards };
+}
+
+function hooksFor(testCase: any, events: unknown[], cached: any) {
+ const behavior = testCase.behavior;
+ const onPredictStart = (ctx: PredictContext) => {
+ if (behavior === "reverse-states") ctx.states.reverse();
+ else if (behavior === "clone-states") ctx.states = clone(ctx.states);
+ else if (behavior === "rewrite-states") ctx.states = ["rewritten"];
+ else if (behavior === "empty-states") ctx.states = [];
+ else if (["temporary-remove-all", "temporary-remove-some"].includes(behavior)) {
+ ctx.questions.temporary = { type: "noul", instructions: "?" };
+ } else if (behavior === "rewrite-question-type") {
+ ctx.questions.route = { type: "noul", instructions: "?" };
+ } else if (behavior === "clear-questions") ctx.questions = {};
+ else if (["skip", "skip-many"].includes(behavior)) {
+ ctx.skip(Array.from({ length: behavior === "skip-many" ? ctx.states.length : 1 }, () => cached));
+ } else if (behavior === "budget") ctx.maxLen = 48;
+ else if (behavior === "widen") {
+ const criteria = (ctx.questions.route as any).criteria;
+ for (let i = 0; i < 8; i++) criteria[`opt${String(i).padStart(2, "0")}`] = "d".repeat(20);
+ }
+ events.push({ phase: "start", states: clone(ctx.states), questions: Object.keys(ctx.questions) });
+ };
+ const onPredictEnd = (ctx: PredictContext) => {
+ events.push({ phase: "end", results: ctx.results?.length ?? null, error: ctx.error !== null });
+ const rows = ctx.results as any[] | null;
+ if (rows === null) return;
+ if (behavior === "reverse-results") rows.reverse();
+ else if (behavior === "clone-results") ctx.results = clone(rows);
+ else if (behavior === "shorten-results") rows.pop();
+ else if (["remove-caller-all", "remove-caller-some", "temporary-remove-all", "temporary-remove-some"].includes(behavior)) {
+ const qid = behavior.startsWith("temporary") ? "temporary" : "route";
+ for (const row of behavior.endsWith("all") ? rows : rows.slice(0, 1)) delete row.answers[qid];
+ } else if (behavior === "annotate") {
+ for (const row of rows) for (const answer of Object.values(row.answers) as any[]) {
+ answer.reliability = { note: "observer" };
+ answer.action.note = "observer";
+ }
+ } else if (behavior === "usage-fields") {
+ rows.forEach((row, index) => Object.assign(row.usage, {
+ extra: index + 1, flag: true, details: { [String(index)]: index }, label: String(index), output_tokens: 9,
+ }));
+ } else if (behavior === "edit") {
+ const { field, qid } = testCase;
+ for (const row of rows) {
+ const answer = row.answers[qid];
+ if (field === "type") Object.assign(answer, { type: "score", score: 0.5, legend: { "0": "low", "1": "high" } });
+ else if (field === "choice") answer[field] = "edited";
+ else if (field === "legend" || field === "probabilities") {
+ answer[field][Object.keys(answer[field])[0]] = field === "legend" ? "edited" : 0.1234;
+ } else answer[field] = 0.1234;
+ }
+ }
+ };
+ const hooks = behavior === "stability" ? [new OptionStabilityHook({
+ accept: null, escalate: null, probes: "reorder", variants: ["reversed"], seed: "long-fixture",
+ })] : [];
+ return { hooks, onPredictStart, onPredictEnd };
+}
+
+function diagnostic(text: string): unknown {
+ const normalized = text.replaceAll("predictLong", "predict_long").replaceAll("predictBatch", "predict_batch")
+ .replaceAll("maxLen", "max_len").replaceAll("headMaxLen", "head_max_len");
+ // Diagnostic spelling is language-specific; compare its category and concrete
+ // parameters, not camelCase versus snake_case or the suggested next step's prose.
+ if (normalized.includes("after the start hooks ran")) {
+ return { kind: "budget", values: normalized.match(/\d+/g)?.map(Number) };
+ }
+ const missing = normalized.match(/answer ['"](.*?)['"] is missing from (\d+) of (\d+) windows/);
+ if (missing) return { kind: "missing-answer", qid: missing[1], missing: Number(missing[2]), total: Number(missing[3]) };
+ const capPython = normalized.match(/window=(\d+) is wider than the (\d+) state tokens.*max_len=(\d+)/);
+ const capDefault = normalized.match(/leave only (\d+) of max_len=(\d+).*capped (\d+) -> (\d+)/);
+ const capTS = normalized.match(/questions leave (\d+) state tokens inside max_len=(\d+); capping window (\d+) -> (\d+)/);
+ if (capPython) return { kind: "window-cap", requested: +capPython[1], room: +capPython[2], maxLen: +capPython[3], effective: +capPython[2] };
+ if (capDefault) return { kind: "window-cap", requested: +capDefault[3], room: +capDefault[1], maxLen: +capDefault[2], effective: +capDefault[4] };
+ if (capTS) return { kind: "window-cap", requested: +capTS[3], room: +capTS[1], maxLen: +capTS[2], effective: +capTS[4] };
+ const stridePython = normalized.match(/stride=(\d+).*window was reduced to (\d+).*stride=(\d+) instead/);
+ const strideTS = normalized.match(/window was reduced to (\d+); adjusting stride (\d+) -> (\d+)/);
+ if (stridePython) return { kind: "stride", before: +stridePython[1], window: +stridePython[2], after: +stridePython[3] };
+ if (strideTS) return { kind: "stride", before: +strideTS[2], window: +strideTS[1], after: +strideTS[3] };
+ if (normalized.includes("a hook answered the state before it was scanned, so no window decided the result")) {
+ return { kind: "skip-unattributed" };
+ }
+ return normalized;
+}
+
+function sameContract(actual: any, expected: any, path = "result") {
+ // Python uses float32 ONNX outputs; JS computes softmax in double precision.
+ // Public values are rounded to four decimals. Nothing structural is relaxed.
+ if (typeof expected === "number" && !Number.isInteger(expected)) {
+ expect(typeof actual, path).toBe("number");
+ expect(Math.abs(actual - expected), path).toBeLessThanOrEqual(0.00010001);
+ } else if (Array.isArray(expected)) {
+ expect(Array.isArray(actual), path).toBe(true);
+ expect(actual.length, path).toBe(expected.length);
+ expected.forEach((value, index) => sameContract(actual[index], value, `${path}[${index}]`));
+ } else if (expected !== null && typeof expected === "object") {
+ expect(actual, path).not.toBeNull();
+ expect(Object.keys(actual).sort(), path).toEqual(Object.keys(expected).sort());
+ for (const key of Object.keys(expected)) sameContract(actual[key], expected[key], `${path}.${key}`);
+ } else if (typeof expected === "string") {
+ if (path.endsWith(".error") || path.includes(".warnings[")) {
+ expect(diagnostic(actual), path).toEqual(diagnostic(expected));
+ } else {
+ // The published TS model identifier is unchanged by adding predictLong.
+ const want = path.endsWith(".model") && expected === "laya-rl-agent-onnx" ? "laya-rl-agent" : expected;
+ expect(actual, path).toBe(want);
+ }
+ } else {
+ expect(actual, path).toBe(expected);
+ }
+}
+
+describe("predictLong Python ONNX oracle", () => {
+ it("records the approved clean Python reference and method", () => {
+ expect(fixtures.reference.commit).toBe("118d361a774c9a28cd02a78d6cf4d2af0614f422");
+ expect(fixtures.reference.modified_reference_files).toEqual([]);
+ expect(fixtures.reference.method).toBe("laya.onnx_agent.ONNXAgent.predict_long");
+ });
+
+ for (const testCase of fixtures.cases) it(testCase.name, async () => {
+ const { agent, forwards } = setup(testCase);
+ const events: unknown[] = [];
+ const cached = { model: "cache", answers: { cached: { type: "noul", noul: 0.8,
+ confidence: 0.8, answer_confidence: 0.8, action: { act_probability: 0.5 } } }, usage: { input_tokens: 0, output_tokens: 0 } };
+ const initialCache = clone(cached);
+ const warnings: string[] = [];
+ const warningSpy = vi.spyOn(console, "warn").mockImplementation((message) => warnings.push(String(message)));
+ const actual: any = {};
+ try {
+ actual.result = await agent.predictLong(clone(testCase.state), clone(testCase.questions), {
+ ...testCase.options, ...hooksFor(testCase, events, cached),
+ });
+ } catch (error) {
+ actual.error = (error as Error).message;
+ } finally {
+ warningSpy.mockRestore();
+ }
+ actual.warnings = warnings;
+ actual.forwards = forwards;
+ actual.events = events;
+ actual.cache_unchanged = JSON.stringify(cached) === JSON.stringify(initialCache);
+ if (testCase.name === "short-skip") {
+ // The approved TS contract explicitly warns on every skipped scan, including
+ // this short path where Python v0.4.18 returns windows=0 without a warning.
+ expect(warnings).toEqual(["laya: predictLong: a hook answered the state before it was scanned, so no window decided the result"]);
+ actual.warnings = [];
+ }
+ sameContract(actual, testCase.expected, testCase.name);
+ });
+
+ it("the oracle actually exercises post-inference rejection and real truncation", () => {
+ for (const name of ["budget-after-forward", "short-budget-after-forward", "in-place-question-budget"]) {
+ const outcome = fixtures.cases.find((testCase: any) => testCase.name === name).expected;
+ expect(outcome.error).toContain("after the start hooks ran");
+ expect(outcome.forwards.length).toBeGreaterThan(0);
+ expect(outcome.events.at(-1)).toMatchObject({ phase: "end", error: false });
+ }
+ const drift = fixtures.cases.find((testCase: any) => testCase.name === "decode-expansion-truncates").expected.result;
+ expect(drift.usage.state_tokens_dropped).toBeGreaterThan(0);
+ expect(drift.usage.truncated).toBeGreaterThan(0);
+ const stability = fixtures.cases.find((testCase: any) => testCase.name === "stability-long").expected.result;
+ expect(stability.answers.route.reliability.probes).toEqual(["reorder"]);
+ expect(stability.answers.route.window).toBeDefined();
+ const shared = fixtures.cases.find((testCase: any) => testCase.name === "shared-answer").expected.result;
+ expect(shared.answers.route.window).toBeUndefined();
+ });
+});
diff --git a/tests/test_hooks_api.py b/tests/test_hooks_api.py
index 0a4bf045..d591a30b 100644
--- a/tests/test_hooks_api.py
+++ b/tests/test_hooks_api.py
@@ -225,7 +225,8 @@ for label, fn in (("Agent.predict_batch", Agent.predict_batch),
("Router.predict", Router.predict),
("Router.predict_batch", Router.predict_batch),
("Router.predict_long", Router.predict_long),
- ("ONNXAgent.system_one", ONNXAgent.system_one)):
+ ("ONNXAgent.system_one", ONNXAgent.system_one),
+ ("ONNXAgent.predict_long", ONNXAgent.predict_long)):
check_param(label, fn, "hooks", None)
check_param(label, fn, "on_predict_start", None)
check_param(label, fn, "on_predict_end", None)
@@ -234,10 +235,16 @@ for label, fn in (("Agent.predict_batch", Agent.predict_batch),
# The scan sizes its windows from the checkpoint budget, so it takes `window`/`stride` instead
# of the per-call token overrides the single-window entries take.
-check_param("Agent.predict_long", Agent.predict_long, "window", None)
-check_param("Agent.predict_long", Agent.predict_long, "stride", None)
-for param in ("max_len", "head_max_len"):
- check("Agent.predict_long/%s not accepted" % param, param in sig(Agent.predict_long), False)
+# The TypeScript predictLong port uses these same long-only controls. Keep the
+# Python oracle's ONNX and Router surfaces pinned alongside the torch entry point.
+for label, fn in (("Agent.predict_long", Agent.predict_long),
+ ("ONNXAgent.predict_long", ONNXAgent.predict_long),
+ ("Router.predict_long", Router.predict_long)):
+ for param in ("window", "stride", "batch_size"):
+ check_param(label, fn, param, None)
+ check_param(label, fn, "aggregate", "auto")
+ for param in ("max_len", "head_max_len", "min_confidence"):
+ check("%s/%s not accepted" % (label, param), param in sig(fn), False)
check_param("Agent.predict_batch", Agent.predict_batch, "batch_size", None)
check_param("Agent.predict_batch", Agent.predict_batch, "sort_by_length", False)