Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
feat(java): predictLong, and the rules a model cannot be asked to demonstrate
Adeshina Lasisi4164afcb1e68
diff --git a/.github/workflows/java.yml b/.github/workflows/java.ymlindex 87a19b36..cc756551 100644--- a/.github/workflows/java.yml+++ b/.github/workflows/java.yml@@ -353,12 +353,16 @@ jobs:include:# Tokenizers and configs only -- no weights, nothing traced. The three families here# are the ones whose builders read a checkpoint but never a graph.+ # `predict_long.json` is here and not in the `predict` cell on purpose: its generator+ # STUBS inference, so it needs a tokenizer and an rl_agent_config and no traced graph.+ # Putting it in the weight-bearing cell would make a 13-second check wait on a+ # 40-minute export for nothing.- name: tokenizercheckpoints: english multilingualgraph: falsefamilies: >---only tokenizer_ids.json --only sequences.json --only decode.json- --only decode_text.json --only window_plan.json+ --only decode_text.json --only window_plan.json --only predict_long.jsontests: >---tests com.convaiinnovations.laya.tokenizer.TokenizerParityTest--tests com.convaiinnovations.laya.sequence.SequenceParityTest@@ -366,7 +370,8 @@ jobs:--tests com.convaiinnovations.laya.tokenizer.PreTokenizerTablesTest--tests com.convaiinnovations.laya.tokenizer.TokenDecoderTest--tests com.convaiinnovations.laya.sequence.WindowPlanTest- # Measured locally: 444. The margin is deliberately smaller than the 8 the 183/175+ --tests com.convaiinnovations.laya.PredictLongTest+ # Measured locally: 493. The margin is deliberately smaller than the 8 the 183/175# pair used, because the smallest class in this cell is PreTokenizerTablesTest at 5# tests and a margin of 8 let that whole class be dropped with the floor still green# (measured at the time: 369 of 374, exit 0). A floor below the total minus the@@ -378,7 +383,7 @@ jobs:# slices into a single test, and a slice list that emptied would still register its# test. The slices are guarded inside the suite instead, by the `total >= 400`# assertion in TokenDecoderTest and the `cases.size() >= 23` corpus floor beside it.- min-tests: 440+ min-tests: 489timeout: 25# The one cell that needs the traced graph, and the only checkpoint the end-to-end# golden was recorded from.diff --git a/laya-java/CHANGELOG.md b/laya-java/CHANGELOG.mdindex 0d7c08e9..1722b9bb 100644--- a/laya-java/CHANGELOG.md+++ b/laya-java/CHANGELOG.md@@ -20,6 +20,14 @@ see [Installing](README.md#installing).`confidence` alongside `answerConfidence`.- Python-compatible tokenization (BPE and Metaspace, added tokens, normalizers), sequencebuilding, collation, answer decoding and calibration.+- `Agent.predictLong` and `LongPrediction`: questions answered over a state longer than the+ context window. `predict` truncates such a state to one window and drops the rest silently;+ this tokenizes it once, scans it in overlapping windows, scores them in shared graph calls and+ aggregates per question — P(true) is the **max** over windows for a `noul`, and a choice or a+ score takes the single **most-confident** window, so a localized signal is not out-voted by the+ neutral text a long document is mostly made of. Ties take the earliest window. Each answer+ carries the `Window` that decided it, because the probability reported is that window's and not+ a calibrated number for the document.- `sequence.WindowPlan`: how a scan over a state longer than one sequence is sized —`stateRoom`, `budget` and `batchCap`, ported from `laya.common`. The window is capped at theroom the questions leave, and at the *smallest* room when several are asked, because a windowdiff --git a/laya-java/README.md b/laya-java/README.mdindex 52404bb7..2ce22fb0 100644--- a/laya-java/README.md+++ b/laya-java/README.md@@ -301,6 +301,27 @@ String withSpecials = tok.decode(ids, false); // ...keptAgentConfig cfg = AgentConfig.fromModelDirectory(Path.of("./checkpoint"));```+## A state longer than the context window++```java+LongPrediction scan = agent.predictLong(wholeDocument, questions);+Answer intent = scan.answer("intent");+scan.window("intent").ifPresent(w ->+ System.out.printf("decided by window %d of %d, tokens [%d,%d)%n",+ w.index(), w.count(), w.tokenStart(), w.tokenEnd()));+```++`predict` truncates a state that exceeds `max_len` to a single window and drops the rest without+saying so. `predictLong` scans it instead: one tokenization, overlapping windows, shared graph+calls, then one answer per question. A `noul` takes the **highest** P(true) across windows — the+statement holds if any window supports it — while a choice or a score takes the **most confident**+window, which stops a localized signal being out-voted by the neutral text that makes up most of a+long document. Ties go to the earliest window.++The probability on an answer is **the deciding window's**, not a calibrated number for the whole+document, which is why `window(id)` exists: without it the number has no stated scope. The window+is absent when the state fitted one window, because then nothing was decided between windows.+`decode` follows the reference rather than tidying after it, and on the multilingual checkpointthat means it is **lossy**: `Metaspace` prepends its marker, so `decode(encode("Hello world"))` is`" Hello world"`. The English checkpoint happens to round-trip. Correcting the space would makediff --git a/laya-java/fixtures/predict_long.json b/laya-java/fixtures/predict_long.jsonnew file mode 100644index 00000000..e3685bc5--- /dev/null+++ b/laya-java/fixtures/predict_long.json@@ -0,0 +1,7630 @@+{+ "by": {+ "english": {+ "max_len": 512,+ "head_max_len": 192,+ "states": {+ "repeat-90": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2.",+ "repeat-100": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12.",+ "repeat-3": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2.",+ "repeat-120": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12. clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause112 states that item 1 was billed on day 25. clause113 states that item 2 was billed on day 26. clause114 states that item 3 was billed on day 27. clause115 states that item 4 was billed on day 28. clause116 states that item 5 was billed on day 0. clause117 states that item 6 was billed on day 1. clause118 states that item 7 was billed on day 2. clause119 states that item 8 was billed on day 3."+ },+ "splits": {+ "repeat-90|None|None": [+ "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25",+ ". clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9",+ ". clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22",+ ". clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6",+ ". clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19",+ ". clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-100|None|None": [+ "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25",+ ". clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9",+ ". clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22",+ ". clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6",+ ". clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19",+ ". clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3",+ ". clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12."+ ],+ "repeat-90|120|60": [+ "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9",+ ". clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14",+ ". clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19",+ ". clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24",+ ". clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0",+ ". clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5",+ ". clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10",+ ". clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15",+ ". clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20",+ ". clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25",+ ". clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1",+ ". clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6",+ ". clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11",+ ". clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16",+ ". clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21",+ ". clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26",+ ". clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2",+ ". clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-90|150|150": [+ "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item",+ " 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24",+ ". clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item",+ " 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20",+ ". clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item",+ " 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16",+ ". clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item",+ " 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-120|None|None": [+ "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states",+ "4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item",+ " that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was",+ " 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on",+ " billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24",+ " day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause",+ ". clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states",+ "29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item",+ " that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was",+ " 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on",+ " billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20",+ " day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause",+ ". clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states",+ "54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item",+ " that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was",+ " 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on",+ " billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16",+ " day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause",+ ". clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states",+ "79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item",+ " that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was",+ " 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on",+ " billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12",+ " day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12. clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause",+ ". clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states",+ "104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause112 states that item",+ " that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause112 states that item 1 was billed on day 25. clause113 states that item 2 was billed on day 26. clause114 states that item 3 was billed on day 27. clause115 states that item 4 was billed on day 28. clause116 states that item 5 was",+ " 1 was billed on day 25. clause113 states that item 2 was billed on day 26. clause114 states that item 3 was billed on day 27. clause115 states that item 4 was billed on day 28. clause116 states that item 5 was billed on day 0. clause117 states that item 6 was billed on day 1. clause118 states that item 7 was billed on day 2. clause119 states that item 8 was billed on day 3."+ ]+ },+ "cases": [+ {+ "case": "noul-max-wins",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.31,+ "answer_confidence": 0.31,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.5,+ "confidence": 0.77,+ "answer_confidence": 0.77,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.31,+ "answer_confidence": 0.31,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.31,+ "answer_confidence": 0.31,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "noul-tie-takes-the-first",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.3,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.95,+ "answer_confidence": 0.95,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.3,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.3,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "choice-most-confident",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.4,+ "replace": 0.6+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.55,+ "refund": 0.44999999999999996+ },+ "confidence": 0.45,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9,+ "replace": 0.09999999999999998+ },+ "confidence": 0.1,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.5,+ "refund": 0.5+ },+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.4,+ "replace": 0.6+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.55,+ "refund": 0.44999999999999996+ },+ "confidence": 0.45,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9,+ "replace": 0.09999999999999998+ },+ "confidence": 0.1,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "choice-tie-takes-the-first",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 0,+ "token_start": 0,+ "token_end": 312,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.8,+ "refund": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.8,+ "refund": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.8,+ "refund": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 0,+ "token_start": 0,+ "token_end": 312,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "score-most-confident",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "grade": {+ "type": "score",+ "criteria": [+ "low",+ "mid",+ "high"+ ],+ "instructions": "How bad?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 1.0,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 2.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.12,+ "answer_confidence": 0.88,+ "action": {+ "act_probability": 0.75+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 0.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 1.0,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 2.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.12,+ "answer_confidence": 0.88,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 0.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 2.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.12,+ "answer_confidence": 0.88,+ "action": {+ "act_probability": 0.75+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "two-questions-two-windows",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ },+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.9,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.2,+ "replace": 0.8+ },+ "confidence": 0.8,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.95,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ },+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.3,+ "refund": 0.7+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.99,+ "replace": 0.010000000000000009+ },+ "confidence": 0.01,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.9,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.2,+ "replace": 0.8+ },+ "confidence": 0.8,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.95,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.3,+ "refund": 0.7+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.99,+ "replace": 0.010000000000000009+ },+ "confidence": 0.01,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.95,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 156,+ "token_end": 468,+ "count": 6+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.99,+ "replace": 0.010000000000000009+ },+ "confidence": 0.01,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "span-clamped-at-the-end",+ "state_key": "repeat-100",+ "split_key": "repeat-100|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.99,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 7+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.99,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780,+ 936+ ],+ "budget": 312,+ "state_tokens": 1201,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.99,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 312,+ "token_end": 624,+ "count": 7+ }+ }+ },+ "usage": {+ "input_tokens": 721,+ "output_tokens": 0,+ "state_tokens": 91,+ "state_tokens_dropped": 21,+ "truncated": 3,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 7+ }+ },+ "warnings": []+ },+ {+ "case": "explicit-window-and-stride",+ "state_key": "repeat-90",+ "split_key": "repeat-90|120|60",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": 120,+ "stride": 60,+ "batch_size": null,+ "batch_cap_used": 7,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.25,+ "confidence": 0.25,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.30000000000000004,+ "confidence": 0.30000000000000004,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.4,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.45000000000000007,+ "confidence": 0.45000000000000007,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 107,+ "output_tokens": 5,+ "state_tokens": 17,+ "state_tokens_dropped": 7,+ "truncated": true,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.5,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 108,+ "output_tokens": 5,+ "state_tokens": 18,+ "state_tokens_dropped": 8,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-8",+ "options": {+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.55,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 109,+ "output_tokens": 5,+ "state_tokens": 19,+ "state_tokens_dropped": 9,+ "truncated": true,+ "truncated_questions": [+ "q9"+ ],+ "backend": "window-9"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.6,+ "confidence": 0.6,+ "answer_confidence": 0.6,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 110,+ "output_tokens": 5,+ "state_tokens": 20,+ "state_tokens_dropped": 10,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-10",+ "options": {+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.65,+ "confidence": 0.65,+ "answer_confidence": 0.65,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 111,+ "output_tokens": 5,+ "state_tokens": 21,+ "state_tokens_dropped": 11,+ "truncated": true,+ "truncated_questions": [+ "q11"+ ],+ "backend": "window-11"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.7000000000000001,+ "confidence": 0.7000000000000001,+ "answer_confidence": 0.7000000000000001,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 112,+ "output_tokens": 5,+ "state_tokens": 22,+ "state_tokens_dropped": 12,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-12",+ "options": {+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.75,+ "confidence": 0.75,+ "answer_confidence": 0.75,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 113,+ "output_tokens": 5,+ "state_tokens": 23,+ "state_tokens_dropped": 13,+ "truncated": true,+ "truncated_questions": [+ "q13"+ ],+ "backend": "window-13"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.8,+ "confidence": 0.8,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 114,+ "output_tokens": 5,+ "state_tokens": 24,+ "state_tokens_dropped": 14,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-14",+ "options": {+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.85,+ "confidence": 0.85,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 115,+ "output_tokens": 5,+ "state_tokens": 25,+ "state_tokens_dropped": 15,+ "truncated": true,+ "truncated_questions": [+ "q15"+ ],+ "backend": "window-15"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.9,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 116,+ "output_tokens": 5,+ "state_tokens": 26,+ "state_tokens_dropped": 16,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-16",+ "options": {+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9500000000000001,+ "confidence": 0.9500000000000001,+ "answer_confidence": 0.9500000000000001,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 17,+ "token_start": 1020,+ "token_end": 1081,+ "count": 18+ }+ }+ },+ "usage": {+ "input_tokens": 117,+ "output_tokens": 5,+ "state_tokens": 27,+ "state_tokens_dropped": 17,+ "truncated": true,+ "truncated_questions": [+ "q17"+ ],+ "backend": "window-17"+ }+ }+ ],+ "starts": [+ 0,+ 60,+ 120,+ 180,+ 240,+ 300,+ 360,+ 420,+ 480,+ 540,+ 600,+ 660,+ 720,+ 780,+ 840,+ 900,+ 960,+ 1020+ ],+ "budget": 120,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9500000000000001,+ "confidence": 0.9500000000000001,+ "answer_confidence": 0.9500000000000001,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 17,+ "token_start": 1020,+ "token_end": 1081,+ "count": 18+ }+ }+ },+ "usage": {+ "input_tokens": 1953,+ "output_tokens": 0,+ "state_tokens": 333,+ "state_tokens_dropped": 153,+ "truncated": 9,+ "truncated_questions": [+ "q17"+ ],+ "backend": "window-17",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 18+ }+ },+ "warnings": []+ },+ {+ "case": "explicit-stride-no-overlap",+ "state_key": "repeat-90",+ "split_key": "repeat-90|150|150",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": 150,+ "stride": 150,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.25,+ "confidence": 0.25,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.30000000000000004,+ "confidence": 0.30000000000000004,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.4,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.45000000000000007,+ "confidence": 0.45000000000000007,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 7,+ "token_start": 1050,+ "token_end": 1081,+ "count": 8+ }+ }+ },+ "usage": {+ "input_tokens": 107,+ "output_tokens": 5,+ "state_tokens": 17,+ "state_tokens_dropped": 7,+ "truncated": true,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7"+ }+ }+ ],+ "starts": [+ 0,+ 150,+ 300,+ 450,+ 600,+ 750,+ 900,+ 1050+ ],+ "budget": 150,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.45000000000000007,+ "confidence": 0.45000000000000007,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 7,+ "token_start": 1050,+ "token_end": 1081,+ "count": 8+ }+ }+ },+ "usage": {+ "input_tokens": 828,+ "output_tokens": 0,+ "state_tokens": 108,+ "state_tokens_dropped": 28,+ "truncated": 4,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 8+ }+ },+ "warnings": []+ },+ {+ "case": "fits-one-window",+ "state_key": "repeat-3",+ "split_key": null,+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": null,+ "starts": null,+ "budget": null,+ "state_tokens": null,+ "single_call": true,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.42,+ "confidence": 0.61,+ "answer_confidence": 0.61,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 11,+ "output_tokens": 3,+ "state_tokens": 7,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ },+ "warnings": []+ },+ {+ "case": "clamped-window-caps-the-batch",+ "state_key": "repeat-120",+ "split_key": "repeat-120|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "Pick one of many.",+ "criteria": {+ "opt000": "criterion 0 with several words",+ "opt001": "criterion 1 with several words",+ "opt002": "criterion 2 with several words",+ "opt003": "criterion 3 with several words",+ "opt004": "criterion 4 with several words",+ "opt005": "criterion 5 with several words",+ "opt006": "criterion 6 with several words",+ "opt007": "criterion 7 with several words",+ "opt008": "criterion 8 with several words",+ "opt009": "criterion 9 with several words",+ "opt010": "criterion 10 with several words",+ "opt011": "criterion 11 with several words",+ "opt012": "criterion 12 with several words",+ "opt013": "criterion 13 with several words",+ "opt014": "criterion 14 with several words",+ "opt015": "criterion 15 with several words",+ "opt016": "criterion 16 with several words",+ "opt017": "criterion 17 with several words",+ "opt018": "criterion 18 with several words",+ "opt019": "criterion 19 with several words",+ "opt020": "criterion 20 with several words",+ "opt021": "criterion 21 with several words",+ "opt022": "criterion 22 with several words",+ "opt023": "criterion 23 with several words",+ "opt024": "criterion 24 with several words",+ "opt025": "criterion 25 with several words",+ "opt026": "criterion 26 with several words",+ "opt027": "criterion 27 with several words",+ "opt028": "criterion 28 with several words",+ "opt029": "criterion 29 with several words",+ "opt030": "criterion 30 with several words",+ "opt031": "criterion 31 with several words",+ "opt032": "criterion 32 with several words",+ "opt033": "criterion 33 with several words",+ "opt034": "criterion 34 with several words",+ "opt035": "criterion 35 with several words",+ "opt036": "criterion 36 with several words",+ "opt037": "criterion 37 with several words",+ "opt038": "criterion 38 with several words",+ "opt039": "criterion 39 with several words",+ "opt040": "criterion 40 with several words",+ "opt041": "criterion 41 with several words",+ "opt042": "criterion 42 with several words",+ "opt043": "criterion 43 with several words",+ "opt044": "criterion 44 with several words",+ "opt045": "criterion 45 with several words",+ "opt046": "criterion 46 with several words",+ "opt047": "criterion 47 with several words",+ "opt048": "criterion 48 with several words",+ "opt049": "criterion 49 with several words",+ "opt050": "criterion 50 with several words",+ "opt051": "criterion 51 with several words",+ "opt052": "criterion 52 with several words",+ "opt053": "criterion 53 with several words",+ "opt054": "criterion 54 with several words",+ "opt055": "criterion 55 with several words",+ "opt056": "criterion 56 with several words",+ "opt057": "criterion 57 with several words",+ "opt058": "criterion 58 with several words",+ "opt059": "criterion 59 with several words",+ "opt060": "criterion 60 with several words",+ "opt061": "criterion 61 with several words",+ "opt062": "criterion 62 with several words",+ "opt063": "criterion 63 with several words",+ "opt064": "criterion 64 with several words",+ "opt065": "criterion 65 with several words",+ "opt066": "criterion 66 with several words",+ "opt067": "criterion 67 with several words",+ "opt068": "criterion 68 with several words",+ "opt069": "criterion 69 with several words",+ "opt070": "criterion 70 with several words",+ "opt071": "criterion 71 with several words",+ "opt072": "criterion 72 with several words",+ "opt073": "criterion 73 with several words",+ "opt074": "criterion 74 with several words",+ "opt075": "criterion 75 with several words",+ "opt076": "criterion 76 with several words",+ "opt077": "criterion 77 with several words",+ "opt078": "criterion 78 with several words",+ "opt079": "criterion 79 with several words",+ "opt080": "criterion 80 with several words",+ "opt081": "criterion 81 with several words",+ "opt082": "criterion 82 with several words",+ "opt083": "criterion 83 with several words",+ "opt084": "criterion 84 with several words",+ "opt085": "criterion 85 with several words",+ "opt086": "criterion 86 with several words",+ "opt087": "criterion 87 with several words",+ "opt088": "criterion 88 with several words",+ "opt089": "criterion 89 with several words",+ "opt090": "criterion 90 with several words",+ "opt091": "criterion 91 with several words",+ "opt092": "criterion 92 with several words",+ "opt093": "criterion 93 with several words",+ "opt094": "criterion 94 with several words",+ "opt095": "criterion 95 with several words",+ "opt096": "criterion 96 with several words",+ "opt097": "criterion 97 with several words",+ "opt098": "criterion 98 with several words",+ "opt099": "criterion 99 with several words"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": 9,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.1,+ "replace": 0.9+ },+ "confidence": 0.9,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.15000000000000002,+ "replace": 0.85+ },+ "confidence": 0.85,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.2,+ "replace": 0.8+ },+ "confidence": 0.8,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.25,+ "replace": 0.75+ },+ "confidence": 0.75,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.30000000000000004,+ "replace": 0.7+ },+ "confidence": 0.7,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.35,+ "replace": 0.65+ },+ "confidence": 0.65,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.4,+ "replace": 0.6+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.45000000000000007,+ "replace": 0.5499999999999999+ },+ "confidence": 0.55,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 107,+ "output_tokens": 5,+ "state_tokens": 17,+ "state_tokens_dropped": 7,+ "truncated": true,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.5,+ "replace": 0.5+ },+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 108,+ "output_tokens": 5,+ "state_tokens": 18,+ "state_tokens_dropped": 8,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-8",+ "options": {+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.55,+ "replace": 0.44999999999999996+ },+ "confidence": 0.45,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 109,+ "output_tokens": 5,+ "state_tokens": 19,+ "state_tokens_dropped": 9,+ "truncated": true,+ "truncated_questions": [+ "q9"+ ],+ "backend": "window-9"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.6,+ "replace": 0.4+ },+ "confidence": 0.4,+ "answer_confidence": 0.6,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 110,+ "output_tokens": 5,+ "state_tokens": 20,+ "state_tokens_dropped": 10,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-10",+ "options": {+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.65,+ "replace": 0.35+ },+ "confidence": 0.35,+ "answer_confidence": 0.65,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 111,+ "output_tokens": 5,+ "state_tokens": 21,+ "state_tokens_dropped": 11,+ "truncated": true,+ "truncated_questions": [+ "q11"+ ],+ "backend": "window-11"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.7000000000000001,+ "replace": 0.29999999999999993+ },+ "confidence": 0.3,+ "answer_confidence": 0.7000000000000001,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 112,+ "output_tokens": 5,+ "state_tokens": 22,+ "state_tokens_dropped": 12,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-12",+ "options": {+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.75,+ "replace": 0.25+ },+ "confidence": 0.25,+ "answer_confidence": 0.75,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 113,+ "output_tokens": 5,+ "state_tokens": 23,+ "state_tokens_dropped": 13,+ "truncated": true,+ "truncated_questions": [+ "q13"+ ],+ "backend": "window-13"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 114,+ "output_tokens": 5,+ "state_tokens": 24,+ "state_tokens_dropped": 14,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-14",+ "options": {+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.85,+ "replace": 0.15000000000000002+ },+ "confidence": 0.15,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 115,+ "output_tokens": 5,+ "state_tokens": 25,+ "state_tokens_dropped": 15,+ "truncated": true,+ "truncated_questions": [+ "q15"+ ],+ "backend": "window-15"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9,+ "replace": 0.09999999999999998+ },+ "confidence": 0.1,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 116,+ "output_tokens": 5,+ "state_tokens": 26,+ "state_tokens_dropped": 16,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-16",+ "options": {+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9500000000000001,+ "replace": 0.04999999999999993+ },+ "confidence": 0.05,+ "answer_confidence": 0.9500000000000001,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 117,+ "output_tokens": 5,+ "state_tokens": 27,+ "state_tokens_dropped": 17,+ "truncated": true,+ "truncated_questions": [+ "q17"+ ],+ "backend": "window-17"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.0,+ "replace": 0.0+ },+ "confidence": 0.0,+ "answer_confidence": 1.0,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 118,+ "output_tokens": 5,+ "state_tokens": 28,+ "state_tokens_dropped": 18,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-18",+ "options": {+ "q18": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.05,+ "replace": -0.050000000000000044+ },+ "confidence": -0.05,+ "answer_confidence": 1.05,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 119,+ "output_tokens": 5,+ "state_tokens": 29,+ "state_tokens_dropped": 19,+ "truncated": true,+ "truncated_questions": [+ "q19"+ ],+ "backend": "window-19"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.1,+ "replace": -0.10000000000000009+ },+ "confidence": -0.1,+ "answer_confidence": 1.1,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 120,+ "output_tokens": 5,+ "state_tokens": 30,+ "state_tokens_dropped": 20,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-20",+ "options": {+ "q20": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.1500000000000001,+ "replace": -0.15000000000000013+ },+ "confidence": -0.15,+ "answer_confidence": 1.1500000000000001,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 121,+ "output_tokens": 5,+ "state_tokens": 31,+ "state_tokens_dropped": 21,+ "truncated": true,+ "truncated_questions": [+ "q21"+ ],+ "backend": "window-21"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.2000000000000002,+ "replace": -0.20000000000000018+ },+ "confidence": -0.2,+ "answer_confidence": 1.2000000000000002,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 122,+ "output_tokens": 5,+ "state_tokens": 32,+ "state_tokens_dropped": 22,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-22",+ "options": {+ "q22": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.2500000000000002,+ "replace": -0.2500000000000002+ },+ "confidence": -0.25,+ "answer_confidence": 1.2500000000000002,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 123,+ "output_tokens": 5,+ "state_tokens": 33,+ "state_tokens_dropped": 23,+ "truncated": true,+ "truncated_questions": [+ "q23"+ ],+ "backend": "window-23"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.3000000000000003,+ "replace": -0.30000000000000027+ },+ "confidence": -0.3,+ "answer_confidence": 1.3000000000000003,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 124,+ "output_tokens": 5,+ "state_tokens": 34,+ "state_tokens_dropped": 24,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-24",+ "options": {+ "q24": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.35,+ "replace": -0.3500000000000001+ },+ "confidence": -0.35,+ "answer_confidence": 1.35,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 125,+ "output_tokens": 5,+ "state_tokens": 35,+ "state_tokens_dropped": 25,+ "truncated": true,+ "truncated_questions": [+ "q25"+ ],+ "backend": "window-25"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.4000000000000001,+ "replace": -0.40000000000000013+ },+ "confidence": -0.4,+ "answer_confidence": 1.4000000000000001,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 126,+ "output_tokens": 5,+ "state_tokens": 36,+ "state_tokens_dropped": 26,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-26",+ "options": {+ "q26": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.4500000000000002,+ "replace": -0.4500000000000002+ },+ "confidence": -0.45,+ "answer_confidence": 1.4500000000000002,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 27,+ "token_start": 1350,+ "token_end": 1441,+ "count": 28+ }+ }+ },+ "usage": {+ "input_tokens": 127,+ "output_tokens": 5,+ "state_tokens": 37,+ "state_tokens_dropped": 27,+ "truncated": true,+ "truncated_questions": [+ "q27"+ ],+ "backend": "window-27"+ }+ }+ ],+ "starts": [+ 0,+ 50,+ 100,+ 150,+ 200,+ 250,+ 300,+ 350,+ 400,+ 450,+ 500,+ 550,+ 600,+ 650,+ 700,+ 750,+ 800,+ 850,+ 900,+ 950,+ 1000,+ 1050,+ 1100,+ 1150,+ 1200,+ 1250,+ 1300,+ 1350+ ],+ "budget": 100,+ "state_tokens": 1441,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 1.4500000000000002,+ "replace": -0.4500000000000002+ },+ "confidence": -0.45,+ "answer_confidence": 1.4500000000000002,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 27,+ "token_start": 1350,+ "token_end": 1441,+ "count": 28+ }+ }+ },+ "usage": {+ "input_tokens": 3178,+ "output_tokens": 0,+ "state_tokens": 658,+ "state_tokens_dropped": 378,+ "truncated": 14,+ "truncated_questions": [+ "q27"+ ],+ "backend": "window-27",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q18": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q20": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q22": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q24": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q26": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 28+ }+ },+ "warnings": [+ "laya: predict_long: these questions leave only 100 of max_len=512 for the state (their heads take the rest), so the scan window is capped 312 -> 100 and roughly 3.1x as many windows -- each a full forward pass -- are needed to read the document. Fewer or shorter options, a larger max_len, or laya.shortlist.predict_shortlist for a large label set will all cost less than scanning at this width",+ "laya: predict_long: these questions leave only 100 of max_len=512 for the state (their heads take the rest), so the scan window is capped 312 -> 100 and roughly 3.1x as many windows -- each a full forward pass -- are needed to read the document. Fewer or shorter options, a larger max_len, or laya.shortlist.predict_shortlist for a large label set will all cost less than scanning at this width"+ ]+ },+ {+ "case": "explicit-batch-size",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": 2,+ "batch_cap_used": 2,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.25,+ "confidence": 0.25,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.30000000000000004,+ "confidence": 0.30000000000000004,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 5,+ "token_start": 780,+ "token_end": 1081,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 156,+ 312,+ 468,+ 624,+ 780+ ],+ "budget": 312,+ "state_tokens": 1081,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 5,+ "token_start": 780,+ "token_end": 1081,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ }+ ]+ },+ "multilingual": {+ "max_len": 1024,+ "head_max_len": 256,+ "states": {+ "repeat-90": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2.",+ "repeat-100": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12.",+ "repeat-3": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2.",+ "repeat-120": "clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12. clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause112 states that item 1 was billed on day 25. clause113 states that item 2 was billed on day 26. clause114 states that item 3 was billed on day 27. clause115 states that item 4 was billed on day 28. clause116 states that item 5 was billed on day 0. clause117 states that item 6 was billed on day 1. clause118 states that item 7 was billed on day 2. clause119 states that item 8 was billed on day 3."+ },+ "splits": {+ "repeat-90|None|None": [+ " clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed",+ "4 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was",+ " on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-100|None|None": [+ " clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed",+ "4 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was",+ " on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6",+ " billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12."+ ],+ "repeat-90|120|60": [+ " clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was",+ " item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item",+ " billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 1",+ " 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item ",+ "5. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22",+ "19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 2",+ ". clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause",+ "6 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day",+ "30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was",+ " 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item ",+ " billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45",+ "4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day ",+ " states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item",+ "19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 2",+ " 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 2",+ "6. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states",+ "2 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8.",+ " that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was",+ " clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause7",+ " billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 1",+ "4 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed",+ "9. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that",+ " on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1",+ " item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-90|150|150": [+ " clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was",+ " billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item ",+ "19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states",+ " that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was",+ " billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause",+ "47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 2",+ "6. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause6",+ "5 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause7",+ "4 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9",+ " was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2."+ ],+ "repeat-120|None|None": [+ " clause0 states that item 0 was billed on day 0. clause1 states that item 1 was billed on day 1. clause2 states that item 2 was billed on day 2. clause3 states that item 3 was billed on day 3. clause4 states that item 4 was billed on day 4. clause5 states that item 5 was billed on day 5. clause6 states that item 6 was billed on day 6. clause7 states that item 7 was billed on day 7. clause8 states that item 8 was billed on day 8. clause9 states that item 9 was billed on day 9. clause10 states that item 10 was billed on day 10. clause11 states that item 11 was billed on day 11. clause12 states that item 12 was billed on day 12. clause13 states that item 13 was billed on day 13. clause14 states that item 14 was billed on day 14. clause15 states that item 15 was billed on day 15. clause16 states that item 16 was billed on day 16. clause17 states that item 17 was billed on day 17. clause18 states that item 18 was billed on day 18. clause19 states that item 19 was billed on day 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item",+ " 19. clause20 states that item 20 was billed on day 20. clause21 states that item 21 was billed on day 21. clause22 states that item 22 was billed on day 22. clause23 states that item 23 was billed on day 23. clause24 states that item 24 was billed on day 24. clause25 states that item 25 was billed on day 25. clause26 states that item 26 was billed on day 26. clause27 states that item 27 was billed on day 27. clause28 states that item 28 was billed on day 28. clause29 states that item 29 was billed on day 0. clause30 states that item 30 was billed on day 1. clause31 states that item 31 was billed on day 2. clause32 states that item 32 was billed on day 3. clause33 states that item 33 was billed on day 4. clause34 states that item 34 was billed on day 5. clause35 states that item 35 was billed on day 6. clause36 states that item 36 was billed on day 7. clause37 states that item 0 was billed on day 8. clause38 states that item 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27",+ " 1 was billed on day 9. clause39 states that item 2 was billed on day 10. clause40 states that item 3 was billed on day 11. clause41 states that item 4 was billed on day 12. clause42 states that item 5 was billed on day 13. clause43 states that item 6 was billed on day 14. clause44 states that item 7 was billed on day 15. clause45 states that item 8 was billed on day 16. clause46 states that item 9 was billed on day 17. clause47 states that item 10 was billed on day 18. clause48 states that item 11 was billed on day 19. clause49 states that item 12 was billed on day 20. clause50 states that item 13 was billed on day 21. clause51 states that item 14 was billed on day 22. clause52 states that item 15 was billed on day 23. clause53 states that item 16 was billed on day 24. clause54 states that item 17 was billed on day 25. clause55 states that item 18 was billed on day 26. clause56 states that item 19 was billed on day 27. clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed",+ ". clause57 states that item 20 was billed on day 28. clause58 states that item 21 was billed on day 0. clause59 states that item 22 was billed on day 1. clause60 states that item 23 was billed on day 2. clause61 states that item 24 was billed on day 3. clause62 states that item 25 was billed on day 4. clause63 states that item 26 was billed on day 5. clause64 states that item 27 was billed on day 6. clause65 states that item 28 was billed on day 7. clause66 states that item 29 was billed on day 8. clause67 states that item 30 was billed on day 9. clause68 states that item 31 was billed on day 10. clause69 states that item 32 was billed on day 11. clause70 states that item 33 was billed on day 12. clause71 states that item 34 was billed on day 13. clause72 states that item 35 was billed on day 14. clause73 states that item 36 was billed on day 15. clause74 states that item 0 was billed on day 16. clause75 states that item 1 was billed on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20",+ " on day 17. clause76 states that item 2 was billed on day 18. clause77 states that item 3 was billed on day 19. clause78 states that item 4 was billed on day 20. clause79 states that item 5 was billed on day 21. clause80 states that item 6 was billed on day 22. clause81 states that item 7 was billed on day 23. clause82 states that item 8 was billed on day 24. clause83 states that item 9 was billed on day 25. clause84 states that item 10 was billed on day 26. clause85 states that item 11 was billed on day 27. clause86 states that item 12 was billed on day 28. clause87 states that item 13 was billed on day 0. clause88 states that item 14 was billed on day 1. clause89 states that item 15 was billed on day 2. clause90 states that item 16 was billed on day 3. clause91 states that item 17 was billed on day 4. clause92 states that item 18 was billed on day 5. clause93 states that item 19 was billed on day 6. clause94 states that item 20 was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12. clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause",+ " was billed on day 7. clause95 states that item 21 was billed on day 8. clause96 states that item 22 was billed on day 9. clause97 states that item 23 was billed on day 10. clause98 states that item 24 was billed on day 11. clause99 states that item 25 was billed on day 12. clause100 states that item 26 was billed on day 13. clause101 states that item 27 was billed on day 14. clause102 states that item 28 was billed on day 15. clause103 states that item 29 was billed on day 16. clause104 states that item 30 was billed on day 17. clause105 states that item 31 was billed on day 18. clause106 states that item 32 was billed on day 19. clause107 states that item 33 was billed on day 20. clause108 states that item 34 was billed on day 21. clause109 states that item 35 was billed on day 22. clause110 states that item 36 was billed on day 23. clause111 states that item 0 was billed on day 24. clause112 states that item 1 was billed on day 25. clause113 states that item 2 was billed on day 26. clause114 states that item 3 was billed on day 27. clause115 states that item 4 was billed on day 28. clause116 states that item 5 was billed on day 0. clause117 states that item 6 was billed on day 1. clause118 states that item 7 was billed on day 2. clause119 states that item 8 was billed on day 3."+ ]+ },+ "cases": [+ {+ "case": "noul-max-wins",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.31,+ "answer_confidence": 0.31,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.5,+ "confidence": 0.77,+ "answer_confidence": 0.77,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.31,+ "answer_confidence": 0.31,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "noul-tie-takes-the-first",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.3,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.95,+ "answer_confidence": 0.95,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "choice-most-confident",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.4,+ "replace": 0.6+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.55,+ "refund": 0.44999999999999996+ },+ "confidence": 0.45,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9,+ "replace": 0.09999999999999998+ },+ "confidence": 0.1,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.9,+ "replace": 0.09999999999999998+ },+ "confidence": 0.1,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "choice-tie-takes-the-first",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 0,+ "token_start": 0,+ "token_end": 760,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.8,+ "refund": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.8,+ "replace": 0.19999999999999996+ },+ "confidence": 0.2,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 0,+ "token_start": 0,+ "token_end": 760,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "score-most-confident",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "grade": {+ "type": "score",+ "criteria": [+ "low",+ "mid",+ "high"+ ],+ "instructions": "How bad?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 1.0,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 2.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.12,+ "answer_confidence": 0.88,+ "action": {+ "act_probability": 0.75+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 0.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.6,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.75+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "grade": {+ "type": "score",+ "score": 2.5,+ "legend": {+ "0": "low",+ "1": "mid",+ "2": "high"+ },+ "probabilities": {+ "0": 0.2,+ "1": 0.3,+ "2": 0.5+ },+ "confidence": 0.12,+ "answer_confidence": 0.88,+ "action": {+ "act_probability": 0.75+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "two-questions-two-windows",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ },+ "intent": {+ "type": "choice",+ "instructions": "What should we do?",+ "criteria": {+ "refund": "money back",+ "replace": "new unit"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15,+ "confidence": 0.9,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.2,+ "replace": 0.8+ },+ "confidence": 0.8,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.95,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ },+ "intent": {+ "type": "choice",+ "choice": "replace",+ "probabilities": {+ "replace": 0.3,+ "refund": 0.7+ },+ "confidence": 0.7,+ "answer_confidence": 0.3,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.99,+ "replace": 0.010000000000000009+ },+ "confidence": 0.01,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.95,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 1,+ "token_start": 380,+ "token_end": 1140,+ "count": 3+ }+ },+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.99,+ "replace": 0.010000000000000009+ },+ "confidence": 0.01,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ },+ {+ "case": "span-clamped-at-the-end",+ "state_key": "repeat-100",+ "split_key": "repeat-100|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.99,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1520,+ "count": 4+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760,+ 1140+ ],+ "budget": 760,+ "state_tokens": 1620,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.99,+ "confidence": 0.99,+ "answer_confidence": 0.99,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1520,+ "count": 4+ }+ }+ },+ "usage": {+ "input_tokens": 406,+ "output_tokens": 0,+ "state_tokens": 46,+ "state_tokens_dropped": 6,+ "truncated": 2,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 4+ }+ },+ "warnings": []+ },+ {+ "case": "explicit-window-and-stride",+ "state_key": "repeat-90",+ "split_key": "repeat-90|120|60",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": 120,+ "stride": 60,+ "batch_size": null,+ "batch_cap_used": 4,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.25,+ "confidence": 0.25,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.30000000000000004,+ "confidence": 0.30000000000000004,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.4,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.45000000000000007,+ "confidence": 0.45000000000000007,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 107,+ "output_tokens": 5,+ "state_tokens": 17,+ "state_tokens_dropped": 7,+ "truncated": true,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.5,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 108,+ "output_tokens": 5,+ "state_tokens": 18,+ "state_tokens_dropped": 8,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-8",+ "options": {+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.55,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 109,+ "output_tokens": 5,+ "state_tokens": 19,+ "state_tokens_dropped": 9,+ "truncated": true,+ "truncated_questions": [+ "q9"+ ],+ "backend": "window-9"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.6,+ "confidence": 0.6,+ "answer_confidence": 0.6,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 110,+ "output_tokens": 5,+ "state_tokens": 20,+ "state_tokens_dropped": 10,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-10",+ "options": {+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.65,+ "confidence": 0.65,+ "answer_confidence": 0.65,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 111,+ "output_tokens": 5,+ "state_tokens": 21,+ "state_tokens_dropped": 11,+ "truncated": true,+ "truncated_questions": [+ "q11"+ ],+ "backend": "window-11"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.7000000000000001,+ "confidence": 0.7000000000000001,+ "answer_confidence": 0.7000000000000001,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 112,+ "output_tokens": 5,+ "state_tokens": 22,+ "state_tokens_dropped": 12,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-12",+ "options": {+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.75,+ "confidence": 0.75,+ "answer_confidence": 0.75,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 113,+ "output_tokens": 5,+ "state_tokens": 23,+ "state_tokens_dropped": 13,+ "truncated": true,+ "truncated_questions": [+ "q13"+ ],+ "backend": "window-13"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.8,+ "confidence": 0.8,+ "answer_confidence": 0.8,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 114,+ "output_tokens": 5,+ "state_tokens": 24,+ "state_tokens_dropped": 14,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-14",+ "options": {+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.85,+ "confidence": 0.85,+ "answer_confidence": 0.85,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 115,+ "output_tokens": 5,+ "state_tokens": 25,+ "state_tokens_dropped": 15,+ "truncated": true,+ "truncated_questions": [+ "q15"+ ],+ "backend": "window-15"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9,+ "confidence": 0.9,+ "answer_confidence": 0.9,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 116,+ "output_tokens": 5,+ "state_tokens": 26,+ "state_tokens_dropped": 16,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-16",+ "options": {+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.9500000000000001,+ "confidence": 0.9500000000000001,+ "answer_confidence": 0.9500000000000001,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 117,+ "output_tokens": 5,+ "state_tokens": 27,+ "state_tokens_dropped": 17,+ "truncated": true,+ "truncated_questions": [+ "q17"+ ],+ "backend": "window-17"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.0,+ "confidence": 1.0,+ "answer_confidence": 1.0,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 118,+ "output_tokens": 5,+ "state_tokens": 28,+ "state_tokens_dropped": 18,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-18",+ "options": {+ "q18": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.05,+ "confidence": 1.05,+ "answer_confidence": 1.05,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 119,+ "output_tokens": 5,+ "state_tokens": 29,+ "state_tokens_dropped": 19,+ "truncated": true,+ "truncated_questions": [+ "q19"+ ],+ "backend": "window-19"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.1,+ "confidence": 1.1,+ "answer_confidence": 1.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 120,+ "output_tokens": 5,+ "state_tokens": 30,+ "state_tokens_dropped": 20,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-20",+ "options": {+ "q20": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.1500000000000001,+ "confidence": 1.1500000000000001,+ "answer_confidence": 1.1500000000000001,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 121,+ "output_tokens": 5,+ "state_tokens": 31,+ "state_tokens_dropped": 21,+ "truncated": true,+ "truncated_questions": [+ "q21"+ ],+ "backend": "window-21"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.2000000000000002,+ "confidence": 1.2000000000000002,+ "answer_confidence": 1.2000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 122,+ "output_tokens": 5,+ "state_tokens": 32,+ "state_tokens_dropped": 22,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-22",+ "options": {+ "q22": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.2500000000000002,+ "confidence": 1.2500000000000002,+ "answer_confidence": 1.2500000000000002,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 23,+ "token_start": 1380,+ "token_end": 1457,+ "count": 24+ }+ }+ },+ "usage": {+ "input_tokens": 123,+ "output_tokens": 5,+ "state_tokens": 33,+ "state_tokens_dropped": 23,+ "truncated": true,+ "truncated_questions": [+ "q23"+ ],+ "backend": "window-23"+ }+ }+ ],+ "starts": [+ 0,+ 60,+ 120,+ 180,+ 240,+ 300,+ 360,+ 420,+ 480,+ 540,+ 600,+ 660,+ 720,+ 780,+ 840,+ 900,+ 960,+ 1020,+ 1080,+ 1140,+ 1200,+ 1260,+ 1320,+ 1380+ ],+ "budget": 120,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 1.2500000000000002,+ "confidence": 1.2500000000000002,+ "answer_confidence": 1.2500000000000002,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 23,+ "token_start": 1380,+ "token_end": 1457,+ "count": 24+ }+ }+ },+ "usage": {+ "input_tokens": 2676,+ "output_tokens": 0,+ "state_tokens": 516,+ "state_tokens_dropped": 276,+ "truncated": 12,+ "truncated_questions": [+ "q23"+ ],+ "backend": "window-23",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q10": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q12": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q14": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q16": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q18": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q20": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q22": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 24+ }+ },+ "warnings": []+ },+ {+ "case": "explicit-stride-no-overlap",+ "state_key": "repeat-90",+ "split_key": "repeat-90|150|150",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": 150,+ "stride": 150,+ "batch_size": null,+ "batch_cap_used": 2,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.25,+ "confidence": 0.25,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.30000000000000004,+ "confidence": 0.30000000000000004,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.35,+ "confidence": 0.35,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.4,+ "confidence": 0.4,+ "answer_confidence": 0.4,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 106,+ "output_tokens": 5,+ "state_tokens": 16,+ "state_tokens_dropped": 6,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-6",+ "options": {+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.45000000000000007,+ "confidence": 0.45000000000000007,+ "answer_confidence": 0.45000000000000007,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 107,+ "output_tokens": 5,+ "state_tokens": 17,+ "state_tokens_dropped": 7,+ "truncated": true,+ "truncated_questions": [+ "q7"+ ],+ "backend": "window-7"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.5,+ "confidence": 0.5,+ "answer_confidence": 0.5,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 108,+ "output_tokens": 5,+ "state_tokens": 18,+ "state_tokens_dropped": 8,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-8",+ "options": {+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.55,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 9,+ "token_start": 1350,+ "token_end": 1457,+ "count": 10+ }+ }+ },+ "usage": {+ "input_tokens": 109,+ "output_tokens": 5,+ "state_tokens": 19,+ "state_tokens_dropped": 9,+ "truncated": true,+ "truncated_questions": [+ "q9"+ ],+ "backend": "window-9"+ }+ }+ ],+ "starts": [+ 0,+ 150,+ 300,+ 450,+ 600,+ 750,+ 900,+ 1050,+ 1200,+ 1350+ ],+ "budget": 150,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.55,+ "confidence": 0.55,+ "answer_confidence": 0.55,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 9,+ "token_start": 1350,+ "token_end": 1457,+ "count": 10+ }+ }+ },+ "usage": {+ "input_tokens": 1045,+ "output_tokens": 0,+ "state_tokens": 145,+ "state_tokens_dropped": 45,+ "truncated": 5,+ "truncated_questions": [+ "q9"+ ],+ "backend": "window-9",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q6": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q8": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 10+ }+ },+ "warnings": []+ },+ {+ "case": "fits-one-window",+ "state_key": "repeat-3",+ "split_key": null,+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": null,+ "starts": null,+ "budget": null,+ "state_tokens": null,+ "single_call": true,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.42,+ "confidence": 0.61,+ "answer_confidence": 0.61,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 11,+ "output_tokens": 3,+ "state_tokens": 7,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "windows": 1+ }+ },+ "warnings": []+ },+ {+ "case": "clamped-window-caps-the-batch",+ "state_key": "repeat-120",+ "split_key": "repeat-120|None|None",+ "questions": {+ "intent": {+ "type": "choice",+ "instructions": "Pick one of many.",+ "criteria": {+ "opt000": "criterion 0 with several words",+ "opt001": "criterion 1 with several words",+ "opt002": "criterion 2 with several words",+ "opt003": "criterion 3 with several words",+ "opt004": "criterion 4 with several words",+ "opt005": "criterion 5 with several words",+ "opt006": "criterion 6 with several words",+ "opt007": "criterion 7 with several words",+ "opt008": "criterion 8 with several words",+ "opt009": "criterion 9 with several words",+ "opt010": "criterion 10 with several words",+ "opt011": "criterion 11 with several words",+ "opt012": "criterion 12 with several words",+ "opt013": "criterion 13 with several words",+ "opt014": "criterion 14 with several words",+ "opt015": "criterion 15 with several words",+ "opt016": "criterion 16 with several words",+ "opt017": "criterion 17 with several words",+ "opt018": "criterion 18 with several words",+ "opt019": "criterion 19 with several words",+ "opt020": "criterion 20 with several words",+ "opt021": "criterion 21 with several words",+ "opt022": "criterion 22 with several words",+ "opt023": "criterion 23 with several words",+ "opt024": "criterion 24 with several words",+ "opt025": "criterion 25 with several words",+ "opt026": "criterion 26 with several words",+ "opt027": "criterion 27 with several words",+ "opt028": "criterion 28 with several words",+ "opt029": "criterion 29 with several words",+ "opt030": "criterion 30 with several words",+ "opt031": "criterion 31 with several words",+ "opt032": "criterion 32 with several words",+ "opt033": "criterion 33 with several words",+ "opt034": "criterion 34 with several words",+ "opt035": "criterion 35 with several words",+ "opt036": "criterion 36 with several words",+ "opt037": "criterion 37 with several words",+ "opt038": "criterion 38 with several words",+ "opt039": "criterion 39 with several words",+ "opt040": "criterion 40 with several words",+ "opt041": "criterion 41 with several words",+ "opt042": "criterion 42 with several words",+ "opt043": "criterion 43 with several words",+ "opt044": "criterion 44 with several words",+ "opt045": "criterion 45 with several words",+ "opt046": "criterion 46 with several words",+ "opt047": "criterion 47 with several words",+ "opt048": "criterion 48 with several words",+ "opt049": "criterion 49 with several words",+ "opt050": "criterion 50 with several words",+ "opt051": "criterion 51 with several words",+ "opt052": "criterion 52 with several words",+ "opt053": "criterion 53 with several words",+ "opt054": "criterion 54 with several words",+ "opt055": "criterion 55 with several words",+ "opt056": "criterion 56 with several words",+ "opt057": "criterion 57 with several words",+ "opt058": "criterion 58 with several words",+ "opt059": "criterion 59 with several words",+ "opt060": "criterion 60 with several words",+ "opt061": "criterion 61 with several words",+ "opt062": "criterion 62 with several words",+ "opt063": "criterion 63 with several words",+ "opt064": "criterion 64 with several words",+ "opt065": "criterion 65 with several words",+ "opt066": "criterion 66 with several words",+ "opt067": "criterion 67 with several words",+ "opt068": "criterion 68 with several words",+ "opt069": "criterion 69 with several words",+ "opt070": "criterion 70 with several words",+ "opt071": "criterion 71 with several words",+ "opt072": "criterion 72 with several words",+ "opt073": "criterion 73 with several words",+ "opt074": "criterion 74 with several words",+ "opt075": "criterion 75 with several words",+ "opt076": "criterion 76 with several words",+ "opt077": "criterion 77 with several words",+ "opt078": "criterion 78 with several words",+ "opt079": "criterion 79 with several words",+ "opt080": "criterion 80 with several words",+ "opt081": "criterion 81 with several words",+ "opt082": "criterion 82 with several words",+ "opt083": "criterion 83 with several words",+ "opt084": "criterion 84 with several words",+ "opt085": "criterion 85 with several words",+ "opt086": "criterion 86 with several words",+ "opt087": "criterion 87 with several words",+ "opt088": "criterion 88 with several words",+ "opt089": "criterion 89 with several words",+ "opt090": "criterion 90 with several words",+ "opt091": "criterion 91 with several words",+ "opt092": "criterion 92 with several words",+ "opt093": "criterion 93 with several words",+ "opt094": "criterion 94 with several words",+ "opt095": "criterion 95 with several words",+ "opt096": "criterion 96 with several words",+ "opt097": "criterion 97 with several words",+ "opt098": "criterion 98 with several words",+ "opt099": "criterion 99 with several words"+ }+ }+ },+ "window": null,+ "stride": null,+ "batch_size": null,+ "batch_cap_used": null,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.1,+ "replace": 0.9+ },+ "confidence": 0.9,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.15000000000000002,+ "replace": 0.85+ },+ "confidence": 0.85,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.2,+ "replace": 0.8+ },+ "confidence": 0.8,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.25,+ "replace": 0.75+ },+ "confidence": 0.75,+ "answer_confidence": 0.25,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 103,+ "output_tokens": 5,+ "state_tokens": 13,+ "state_tokens_dropped": 3,+ "truncated": true,+ "truncated_questions": [+ "q3"+ ],+ "backend": "window-3"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.30000000000000004,+ "replace": 0.7+ },+ "confidence": 0.7,+ "answer_confidence": 0.30000000000000004,+ "action": {+ "act_probability": 0.25+ }+ }+ },+ "usage": {+ "input_tokens": 104,+ "output_tokens": 5,+ "state_tokens": 14,+ "state_tokens_dropped": 4,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-4",+ "options": {+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.35,+ "replace": 0.65+ },+ "confidence": 0.65,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 5,+ "token_start": 1530,+ "token_end": 1967,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 105,+ "output_tokens": 5,+ "state_tokens": 15,+ "state_tokens_dropped": 5,+ "truncated": true,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5"+ }+ }+ ],+ "starts": [+ 0,+ 306,+ 612,+ 918,+ 1224,+ 1530+ ],+ "budget": 612,+ "state_tokens": 1967,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "intent": {+ "type": "choice",+ "choice": "refund",+ "probabilities": {+ "refund": 0.35,+ "replace": 0.65+ },+ "confidence": 0.65,+ "answer_confidence": 0.35,+ "action": {+ "act_probability": 0.25+ },+ "window": {+ "index": 5,+ "token_start": 1530,+ "token_end": 1967,+ "count": 6+ }+ }+ },+ "usage": {+ "input_tokens": 615,+ "output_tokens": 0,+ "state_tokens": 75,+ "state_tokens_dropped": 15,+ "truncated": 3,+ "truncated_questions": [+ "q5"+ ],+ "backend": "window-5",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q4": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 6+ }+ },+ "warnings": []+ },+ {+ "case": "explicit-batch-size",+ "state_key": "repeat-90",+ "split_key": "repeat-90|None|None",+ "questions": {+ "holds": {+ "type": "noul",+ "instructions": "Does it hold?"+ }+ },+ "window": null,+ "stride": null,+ "batch_size": 2,+ "batch_cap_used": 2,+ "per_window": [+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.1,+ "confidence": 0.1,+ "answer_confidence": 0.1,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 100,+ "output_tokens": 5,+ "state_tokens": 10,+ "state_tokens_dropped": 0,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-0",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.15000000000000002,+ "confidence": 0.15000000000000002,+ "answer_confidence": 0.15000000000000002,+ "action": {+ "act_probability": 0.5+ }+ }+ },+ "usage": {+ "input_tokens": 101,+ "output_tokens": 5,+ "state_tokens": 11,+ "state_tokens_dropped": 1,+ "truncated": true,+ "truncated_questions": [+ "q1"+ ],+ "backend": "window-1"+ }+ },+ {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 102,+ "output_tokens": 5,+ "state_tokens": 12,+ "state_tokens_dropped": 2,+ "truncated": false,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ }+ }+ }+ ],+ "starts": [+ 0,+ 380,+ 760+ ],+ "budget": 760,+ "state_tokens": 1457,+ "single_call": false,+ "result": {+ "model": "laya-rl-agent-onnx",+ "answers": {+ "holds": {+ "type": "noul",+ "noul": 0.2,+ "confidence": 0.2,+ "answer_confidence": 0.2,+ "action": {+ "act_probability": 0.5+ },+ "window": {+ "index": 2,+ "token_start": 760,+ "token_end": 1457,+ "count": 3+ }+ }+ },+ "usage": {+ "input_tokens": 303,+ "output_tokens": 0,+ "state_tokens": 33,+ "state_tokens_dropped": 3,+ "truncated": 1,+ "truncated_questions": [],+ "backend": "window-2",+ "options": {+ "q0": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ },+ "q2": {+ "total": 3,+ "distinct": 2,+ "tokens_per_option": null+ }+ },+ "windows": 3+ }+ },+ "warnings": []+ }+ ]+ }+ }+}diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.javaindex 6bd3f0d8..13778d99 100644--- a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/Agent.java@@ -6,6 +6,7 @@ import com.convaiinnovations.laya.infer.InferenceSession;import com.convaiinnovations.laya.onnx.LayaSession;import com.convaiinnovations.laya.sequence.Collator;import com.convaiinnovations.laya.sequence.SequenceBuilder;+import com.convaiinnovations.laya.sequence.WindowPlan;import com.convaiinnovations.laya.tokenizer.Tokenizer;import java.io.IOException;import java.nio.file.Path;@@ -242,6 +243,203 @@ public final class Agent implements AutoCloseable, Predictor {return List.of(out);}+ /** Scans a state longer than one sequence, with the checkpoint's own window and stride. */+ public LongPrediction predictLong(Object state, Map<String, Question> questions) {+ return predictLong(state, questions, null, null, null, 0);+ }++ /**+ * Evaluates questions over a state longer than the context window.+ *+ * <p>{@link #predict} truncates a state that exceeds {@code max_len} to a single window and+ * drops the rest silently. This tokenizes the state once, splits it into overlapping token+ * windows, decodes each back to text, scores them all through {@link #predictBatch} -- so the+ * windows share graph calls rather than costing one each -- and combines the per-window+ * answers:+ *+ * <ul>+ * <li>{@code noul}: P(true) is the MAX over windows. The statement holds if any window+ * supports it.</li>+ * <li>{@code choice} and {@code score}: the answer from the single most-confident window,+ * so a localized signal is not out-voted by the many neutral windows a long document is+ * mostly made of.</li>+ * </ul>+ *+ * <p>Ties go to the EARLIEST window, which is what the reference's {@code max} over a range+ * does. The returned probability and confidence are the deciding window's, not a calibrated+ * number for the document; {@link LongPrediction.Window} says which window that was.+ *+ * <p>A state that already fits one window is passed straight to {@link #predict}, with no+ * window attribution and {@code windows == 1}.+ *+ * @param window state tokens per window, or null for the per-question budget. Capped at+ * the room the questions leave -- see {@link WindowPlan}+ * @param stride token step between windows, or null for a 50% overlap of the EFFECTIVE+ * window+ * @param batchSize windows per graph call, or 0 to let {@link WindowPlan#batchCap} bound it+ * @throws IllegalArgumentException for a null state or questions, and for a window or stride+ * {@link WindowPlan} refuses+ */+ public LongPrediction predictLong(Object state, Map<String, Question> questions,+ Integer window, Integer stride, String language,+ int batchSize) {+ if (closed) {+ throw new IllegalStateException(+ "this agent is closed; open a new one rather than reusing it");+ }+ // Refused rather than serialised, as predict and predictBatch refuse it: a missing state+ // would otherwise be answered as a decision about the literal text "null".+ if (state == null) {+ throw new IllegalArgumentException("state must not be null; pass a string, a map or "+ + "a list");+ }+ if (questions == null) {+ throw new IllegalArgumentException("questions must not be null");+ }++ int maxLen = config.maxLen();+ int headMaxLen = config.headMaxLen();+ WindowPlan.Budget plan = WindowPlan.budget(tokenizer, List.copyOf(questions.values()),+ maxLen, headMaxLen, window, stride);++ // The mask string is replaced, not stripped: a state containing the checkpoint's own mask+ // token would otherwise add markers the head did not ask for, and the option spans+ // downstream are found BY those markers.+ String text = SequenceBuilder.serializeState(state).replace(maskToken(), " ");+ int[] stateIds = tokenizer.encode(text);++ if (stateIds.length <= plan.window()) {+ // Identical to a plain call, with no windowing overhead and no window attribution --+ // nothing decided between windows because there was only one.+ Prediction single = predict(state, questions, language);+ Map<String, LongPrediction.Windowed> answers = new LinkedHashMap<>();+ for (Map.Entry<String, Answer> entry : single.answers().entrySet()) {+ answers.put(entry.getKey(),+ new LongPrediction.Windowed(entry.getValue(), null));+ }+ Usage one = single.usage();+ // `one.outputTokens()` carried through, not forced to zero. The MULTI-window path+ // zeroes it because the reference does; the single-window path returns the plain+ // call's usage as it was, and this head emits no tokens anyway -- so the two agree+ // today, and hardcoding it would make them disagree the moment that changed.+ return new LongPrediction(single.model(), answers,+ new LongPrediction.LongUsage(one.inputTokens(), one.outputTokens(),+ one.stateTokens(), one.stateTokensDropped(),+ one.truncated() ? 1 : 0, one.truncatedQuestions(),+ one.collapsedOptions(), 1));+ }++ List<String> scanned = new ArrayList<>();+ List<Integer> starts = new ArrayList<>();+ for (int at = 0; at < stateIds.length; at += plan.stride()) {+ int end = Math.min(at + plan.window(), stateIds.length);+ // Decoded back to text so predictBatch re-tokenizes each window as an ordinary+ // state; the default 50% overlap absorbs any boundary drift on re-tokenization.+ scanned.add(tokenizer.decode(java.util.Arrays.copyOfRange(stateIds, at, end)));+ starts.add(at);+ if (end >= stateIds.length) {+ break;+ }+ }++ Integer cap = WindowPlan.batchCap(scanned.size(), plan.window(),+ Math.max(64, maxLen - headMaxLen - 8), batchSize > 0 ? batchSize : null);+ List<Prediction> results = predictBatch(scanned, questions, language,+ cap == null ? 0 : cap, false);+ if (results.size() != scanned.size()) {+ throw new IllegalStateException(String.format(+ "the state was split into %d windows and the batch returned %d results",+ scanned.size(), results.size()));+ }++ return aggregate(List.copyOf(questions.keySet()), results, starts, plan.window(),+ stateIds.length);+ }++ /**+ * Combines the per-window answers into one, naming the window that decided each.+ *+ * <p>Package-private and static so a test can drive it with recorded per-window results. The+ * rules it implements -- a tie going to the earliest window, {@code noul} chosen on P(true)+ * while a choice is chosen on confidence -- cannot be exercised through a real graph, because+ * a model cannot be asked to produce a tie or a particular deciding window on demand. Driving+ * it directly is what makes those rules testable at all.+ *+ * @param starts first token of each window, parallel to {@code results}+ * @param window the EFFECTIVE window, which is what the reported span is measured in+ * @param stateTokens the tokenized state's length, which clamps the final window's end+ */+ static LongPrediction aggregate(List<String> questionIds, List<Prediction> results,+ List<Integer> starts, int window, int stateTokens) {+ if (results.isEmpty()) {+ throw new IllegalArgumentException("a scan with no windows has nothing to aggregate");+ }+ if (starts.size() != results.size()) {+ throw new IllegalArgumentException(String.format(+ "%d window starts for %d results", starts.size(), results.size()));+ }+ Map<String, LongPrediction.Windowed> answers = new LinkedHashMap<>();+ for (String id : questionIds) {+ int best = 0;+ double bestKey = Double.NEGATIVE_INFINITY;+ for (int at = 0; at < results.size(); at++) {+ Answer candidate = results.get(at).answer(id);+ if (candidate == null) {+ throw new IllegalArgumentException(+ "window " + at + " has no answer for question " + id);+ }+ // noul is chosen on P(true) and everything else on the calibrated confidence --+ // a DIFFERENT field, not the same number under two names.+ double key = candidate instanceof Answer.Noul noul+ ? noul.noul()+ : candidate.answerConfidence();+ // Strictly greater, so a tie keeps the EARLIEST window, as the reference does.+ if (key > bestKey) {+ bestKey = key;+ best = at;+ }+ }+ int start = starts.get(best);+ answers.put(id, new LongPrediction.Windowed(results.get(best).answer(id),+ new LongPrediction.Window(best, start,+ Math.min(start + window, stateTokens), results.size())));+ }+ return new LongPrediction(Prediction.MODEL, answers, aggregateUsage(results));+ }++ /**+ * Usage across the windows of one scan.+ *+ * <p>Numeric fields SUM, which makes {@code truncated} a count of windows that truncated+ * rather than a flag. {@code collapsedOptions} is MERGED rather than replaced: it is keyed by+ * question and set only on the windows where option spans actually collapsed, so replacing it+ * left a caller holding whichever collapsing window came last -- and the deciding window is+ * the most confident one, not the last one. {@code truncatedQuestions} is the last window's+ * list, carried as the reference carries it, which is why the count can be above zero while+ * the list is empty.+ */+ private static LongPrediction.LongUsage aggregateUsage(List<Prediction> results) {+ int inputTokens = 0;+ int stateTokens = 0;+ int dropped = 0;+ int truncatedWindows = 0;+ List<String> truncatedQuestions = List.of();+ Map<String, Usage.CollapsedOptions> collapsed = new LinkedHashMap<>();+ for (Prediction result : results) {+ Usage usage = result.usage();+ inputTokens += usage.inputTokens();+ stateTokens += usage.stateTokens();+ dropped += usage.stateTokensDropped();+ truncatedWindows += usage.truncated() ? 1 : 0;+ truncatedQuestions = usage.truncatedQuestions();+ collapsed.putAll(usage.collapsedOptions());+ }+ // Zero, not a sum: this head emits no tokens, and summing zeros across windows would+ // still be zero while implying the field meant something per window.+ return new LongPrediction.LongUsage(inputTokens, 0, stateTokens, dropped,+ truncatedWindows, truncatedQuestions, collapsed, results.size());+ }+/*** Builds every question's sequence for one state, tokenizing the state once.*diff --git a/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/LongPrediction.java b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/LongPrediction.javanew file mode 100644index 00000000..863e52b0--- /dev/null+++ b/laya-java/laya-java/src/main/java/com/convaiinnovations/laya/LongPrediction.java@@ -0,0 +1,107 @@+package com.convaiinnovations.laya;++import java.util.List;+import java.util.Map;+import java.util.Optional;++/**+ * The result of scanning a state longer than one sequence.+ *+ * <p>Separate from {@link Prediction} rather than bolted onto it, because every answer here+ * carries something an ordinary answer cannot: the window that decided it. Putting that on+ * {@link Answer} would give every answer from {@link Agent#predict} a field that is meaningless+ * for it, and {@code Usage} would need a window count and a truncation COUNT where the single-call+ * shape has a truncation flag.+ *+ * <p>The probability and confidence on each answer are the DECIDING WINDOW'S, not a calibrated+ * number for the whole document. A long document is mostly neutral text, so averaging would drown+ * a localized signal; the reference takes the strongest window for {@code noul} and the most+ * confident one for a choice or a score, and reports that window's own numbers so the fields stay+ * mutually consistent. {@link Window} is what makes that readable rather than mysterious.+ */+public record LongPrediction(String model, Map<String, Windowed> answers, LongUsage usage) {++ public LongPrediction {+ answers = Map.copyOf(answers);+ }++ /**+ * An answer and the window it came from.+ *+ * @param window the deciding window, or null when no window decided -- a state short enough+ * to answer in one call has no window attribution, which is the reference's rule too+ */+ public record Windowed(Answer answer, Window window) {+ }++ /**+ * Where in the tokenized state the deciding window sat.+ *+ * @param index which window, counting from zero+ * @param tokenStart first token of the window, into the tokenized state+ * @param tokenEnd one past its last token, CLAMPED to the end of the state -- the final+ * window is shorter than the others whenever the state does not divide+ * evenly, and reporting {@code start + window} there would name tokens that+ * do not exist+ * @param count how many windows the scan ran, so a caller can tell "the strongest of 40"+ * from "the only one"+ */+ public record Window(int index, int tokenStart, int tokenEnd, int count) {++ public Window {+ if (index < 0 || tokenStart < 0 || tokenEnd < tokenStart || count <= 0) {+ throw new IllegalArgumentException(String.format(+ "a window must be a non-empty span inside a non-empty scan, got"+ + " index=%d [%d,%d) of %d", index, tokenStart, tokenEnd, count));+ }+ }++ /** How many state tokens this window covered. */+ public int tokens() {+ return tokenEnd - tokenStart;+ }+ }++ /**+ * Usage for a scan.+ *+ * <p>Not {@link Usage}: across windows the reference SUMS the numeric fields, so+ * {@code truncated} becomes a count of windows that truncated rather than a flag, and+ * {@code windows} has no counterpart at all on the single-call shape. Keeping the two types+ * apart is what stops {@code truncated} meaning two different things depending on which call+ * produced it.+ *+ * @param truncatedWindows how many windows truncated, summed. For a state answered in one+ * call this is 1 or 0, so the field means the same thing either way+ * @param truncatedQuestions the LAST window's list, carried as the reference carries it --+ * which is why {@code truncatedWindows} can be above zero while this is empty+ * @param collapsedOptions merged across windows, not replaced. It is keyed by question and+ * set only on windows where option spans actually collapsed, so replacing it would leave+ * a caller holding whichever collapsing window came last -- and the deciding window is+ * the most confident one, not the last one+ * @param windows how many windows were scored. Zero when a state was answered+ * without the model reading a window+ */+ public record LongUsage(int inputTokens, int outputTokens, int stateTokens,+ int stateTokensDropped, int truncatedWindows,+ List<String> truncatedQuestions,+ Map<String, Usage.CollapsedOptions> collapsedOptions, int windows) {++ public LongUsage {+ truncatedQuestions = List.copyOf(truncatedQuestions);+ collapsedOptions = Map.copyOf(collapsedOptions);+ }+ }++ /** The answer to one question, or null when it was not asked. */+ public Answer answer(String questionId) {+ Windowed windowed = answers.get(questionId);+ return windowed == null ? null : windowed.answer();+ }++ /** The window that decided one question, empty when no window did. */+ public Optional<Window> window(String questionId) {+ Windowed windowed = answers.get(questionId);+ return windowed == null ? Optional.empty() : Optional.ofNullable(windowed.window());+ }+}diff --git a/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictLongTest.java b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictLongTest.javanew file mode 100644index 00000000..542041a3--- /dev/null+++ b/laya-java/laya-java/src/test/java/com/convaiinnovations/laya/PredictLongTest.java@@ -0,0 +1,482 @@+package com.convaiinnovations.laya;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNotNull;+import static org.junit.jupiter.api.Assertions.assertNull;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.junit.jupiter.api.DynamicTest.dynamicTest;++import com.convaiinnovations.laya.sequence.SequenceBuilder;+import com.convaiinnovations.laya.sequence.WindowPlan;+import com.convaiinnovations.laya.tokenizer.Tokenizer;+import java.util.ArrayList;+import java.util.Arrays;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;+import org.junit.jupiter.api.Assumptions;+import org.junit.jupiter.api.DisplayName;+import org.junit.jupiter.api.DynamicTest;+import org.junit.jupiter.api.Test;+import org.junit.jupiter.api.TestFactory;++/**+ * {@code predictLong} against {@code fixtures/predict_long.json}, recorded from+ * {@code ONNXAgent.predict_long}.+ *+ * <p>The model is not what is under test here, and could not be: the aggregation rules are chosen+ * by values a case has to control. A tie between two windows has to be constructed, the deciding+ * window has to be put somewhere specific, and {@code noul} is chosen on P(true) while a choice is+ * chosen on confidence -- a DIFFERENT field, which only shows when the two disagree. So the+ * generator stubs inference, drives the real {@code predict_long} over it, and records both what+ * it handed to inference and what it made of the answers.+ *+ * <p>That splits cleanly in three, and each part is checked against the reference separately:+ *+ * <ul>+ * <li>the SPLIT -- the window texts the reference handed to inference, which pins the state+ * tokenization, the window offsets and the per-window decode together;</li>+ * <li>the AGGREGATION -- {@link Agent#aggregate}, driven with the recorded per-window results;+ * </li>+ * <li>the whole path end to end, over a stub session, which is the only one of the three that+ * can show the pieces are wired to each other.</li>+ * </ul>+ */+final class PredictLongTest {++ @SuppressWarnings("unchecked")+ private static Map<String, Object> family() {+ return (Map<String, Object>) (Map<?, ?>) Fixtures.load("predict_long.json");+ }++ @SuppressWarnings("unchecked")+ private static Map<String, Object> forCheckpoint(String name) {+ Map<String, Object> by = (Map<String, Object>) family().get("by");+ Map<String, Object> entry = (Map<String, Object>) by.get(name);+ Assumptions.assumeTrue(entry != null && !entry.containsKey("skipped"),+ "the " + name + " checkpoint was not present when the fixture was recorded");+ return entry;+ }++ private static Tokenizer open(String name) {+ java.nio.file.Path model = Fixtures.checkpoint(name);+ Assumptions.assumeTrue(model != null, Fixtures.missingCheckpoint(name));+ try {+ return Tokenizer.fromModelDirectory(model);+ } catch (java.io.IOException problem) {+ throw new IllegalStateException("cannot open the " + name + " tokenizer", problem);+ }+ }++ // ------------------------------------------------------------------ the split++ @TestFactory+ @DisplayName("the scan hands inference the windows the reference handed it")+ @SuppressWarnings("unchecked")+ List<DynamicTest> splitMatchesTheReference() {+ List<DynamicTest> tests = new ArrayList<>();+ for (String name : List.of("english", "multilingual")) {+ Map<String, Object> entry = forCheckpoint(name);+ Map<String, Object> states = (Map<String, Object>) entry.get("states");+ Map<String, Object> splits = (Map<String, Object>) entry.get("splits");+ List<Object> cases = (List<Object>) entry.get("cases");+ int maxLen = number(entry.get("max_len"));+ int headMaxLen = number(entry.get("head_max_len"));+ Tokenizer tok = open(name);+ for (Object raw : cases) {+ Map<String, Object> one = (Map<String, Object>) raw;+ if (one.get("split_key") == null) {+ continue; // answered in one call; nothing was scanned+ }+ tests.add(dynamicTest(name + " " + one.get("case"), () -> {+ String state = (String) states.get(one.get("state_key"));+ List<Object> want = (List<Object>) splits.get(one.get("split_key"));+ assertNotNull(want, "no recorded split for " + one.get("split_key"));++ List<Question> questions = questionsOf(one);+ WindowPlan.Budget plan = WindowPlan.budget(tok, questions,+ maxLen, headMaxLen,+ optional(one.get("window")), optional(one.get("stride")));+ // The budget the reference computed, before anything downstream of it is+ // compared: a split that matched with a different budget would be a+ // coincidence, not parity.+ assertEquals(number(one.get("budget")), plan.window(), "effective window");++ int[] ids = tok.encode(SequenceBuilder.serializeState(state)+ .replace(tok.maskToken(), " "));+ assertEquals(number(one.get("state_tokens")), ids.length, "state tokens");++ List<String> got = new ArrayList<>();+ List<Integer> starts = new ArrayList<>();+ for (int at = 0; at < ids.length; at += plan.stride()) {+ int end = Math.min(at + plan.window(), ids.length);+ got.add(tok.decode(Arrays.copyOfRange(ids, at, end)));+ starts.add(at);+ if (end >= ids.length) {+ break;+ }+ }+ assertEquals(want.size(), got.size(), "window count");+ for (int at = 0; at < want.size(); at++) {+ assertEquals(want.get(at), got.get(at), "window " + at);+ }+ assertEquals(ints(one.get("starts")), starts, "window starts");+ }));+ }+ }+ assertTrue(tests.size() >= 20,+ () -> "both checkpoints' scanned cases, got " + tests.size());+ return tests;+ }++ // ------------------------------------------------------------------ the aggregation++ @TestFactory+ @DisplayName("every aggregation rule matches, including the ties")+ @SuppressWarnings("unchecked")+ List<DynamicTest> aggregationMatchesTheReference() {+ List<DynamicTest> tests = new ArrayList<>();+ int withTies = 0;+ for (String name : List.of("english", "multilingual")) {+ Map<String, Object> entry = forCheckpoint(name);+ for (Object raw : (List<Object>) entry.get("cases")) {+ Map<String, Object> one = (Map<String, Object>) raw;+ if (one.get("per_window") == null) {+ continue; // answered in one call+ }+ if (((String) one.get("case")).contains("tie")) {+ withTies++;+ }+ tests.add(dynamicTest(name + " " + one.get("case"), () -> {+ List<Object> perWindow = (List<Object>) one.get("per_window");+ List<Prediction> results = new ArrayList<>();+ for (Object window : perWindow) {+ results.add(predictionOf((Map<String, Object>) window));+ }+ Map<String, Object> questions =+ (Map<String, Object>) one.get("questions");++ LongPrediction got = Agent.aggregate(List.copyOf(questions.keySet()), results,+ ints(one.get("starts")), number(one.get("budget")),+ number(one.get("state_tokens")));++ Map<String, Object> want =+ (Map<String, Object>) one.get("result");+ Map<String, Object> wantAnswers = (Map<String, Object>) want.get("answers");+ assertEquals(wantAnswers.keySet(), got.answers().keySet(), "question ids");++ for (Map.Entry<String, Object> expected : wantAnswers.entrySet()) {+ String id = expected.getKey();+ Map<String, Object> a = (Map<String, Object>) expected.getValue();+ Answer answer = got.answer(id);+ assertEquals(a.get("type"), answer.type(), id + " type");+ assertEquals(dbl(a.get("answer_confidence")), answer.answerConfidence(),+ 1e-12, id + " answer_confidence");+ assertEquals(dbl(a.get("confidence")), answer.confidence(), 1e-12,+ id + " confidence");+ if (answer instanceof Answer.Noul noul) {+ assertEquals(dbl(a.get("noul")), noul.noul(), 1e-12, id + " noul");+ } else if (answer instanceof Answer.Choice choice) {+ assertEquals(a.get("choice"), choice.choice(), id + " choice");+ } else if (answer instanceof Answer.Score score) {+ assertEquals(dbl(a.get("score")), score.score(), 1e-12, id + " score");+ }++ Map<String, Object> w = (Map<String, Object>) a.get("window");+ LongPrediction.Window window = got.window(id).orElseThrow();+ assertEquals(number(w.get("index")), window.index(), id + " window index");+ assertEquals(number(w.get("token_start")), window.tokenStart(),+ id + " token_start");+ assertEquals(number(w.get("token_end")), window.tokenEnd(),+ id + " token_end");+ assertEquals(number(w.get("count")), window.count(), id + " count");+ }++ // Usage: the fields this port models. The reference's usage dict is open and+ // carries anything `predict_batch` put there -- its `backend` key is a string+ // the last window wins -- while `LongUsage` is a closed record, so an unknown+ // field has nowhere to go and is deliberately not carried.+ Map<String, Object> wantUsage = (Map<String, Object>) want.get("usage");+ LongPrediction.LongUsage usage = got.usage();+ assertEquals(number(wantUsage.get("input_tokens")), usage.inputTokens(),+ "input_tokens summed");+ assertEquals(0, usage.outputTokens(), "output_tokens zeroed for a scan");+ assertEquals(number(wantUsage.get("state_tokens")), usage.stateTokens(),+ "state_tokens summed");+ assertEquals(number(wantUsage.get("state_tokens_dropped")),+ usage.stateTokensDropped(), "state_tokens_dropped summed");+ // A COUNT of windows that truncated, not a flag -- which is why it can be+ // above zero while truncatedQuestions is empty.+ assertEquals(number(wantUsage.get("truncated")), usage.truncatedWindows(),+ "truncated is a window count");+ assertEquals(wantUsage.get("truncated_questions"), usage.truncatedQuestions(),+ "truncated_questions is the LAST window's list");+ assertEquals(number(wantUsage.get("windows")), usage.windows(), "windows");++ Map<String, Object> wantOptions =+ (Map<String, Object>) wantUsage.get("options");+ if (wantOptions != null) {+ // MERGED across windows, not replaced: it is set only on the windows where+ // spans actually collapsed, and the deciding window is the most confident+ // one, not the last one.+ assertEquals(wantOptions.keySet(), usage.collapsedOptions().keySet(),+ "collapsed options merged across windows");+ }+ }));+ }+ }+ final int ties = withTies;+ assertTrue(ties >= 4,+ () -> "the tie cases are the ones a port can silently get backwards and there are "+ + ties);+ return tests;+ }++ // ------------------------------------------------------------------ the whole path++ @Test+ @DisplayName("a state that fits one window is answered without windowing")+ void singleWindowPath(@org.junit.jupiter.api.io.TempDir java.nio.file.Path root)+ throws Exception {+ TinyCheckpoint.write(root, 64, 24);+ TinyCheckpoint.RecordingSession session = new TinyCheckpoint.RecordingSession();+ try (Agent agent = TinyCheckpoint.agent(root, session)) {+ Map<String, Question> questions = Map.of(+ "holds", Question.noul("Does it hold?"));+ LongPrediction got = agent.predictLong("short", questions);+ assertEquals(1, got.usage().windows(), "one window, the model read all of it");+ assertTrue(got.window("holds").isEmpty(),+ "no window decided, because there was only one");+ assertNotNull(got.answer("holds"), "and it is still answered");+ assertEquals(1, session.batches.size(), "one graph call");+ }+ }++ @Test+ @DisplayName("a long state is scanned, and every window span lands inside the state")+ void multiWindowPath(@org.junit.jupiter.api.io.TempDir java.nio.file.Path root)+ throws Exception {+ TinyCheckpoint.write(root, 64, 24);+ TinyCheckpoint.RecordingSession session = new TinyCheckpoint.RecordingSession();+ try (Agent agent = TinyCheckpoint.agent(root, session)) {+ Map<String, Question> questions = new LinkedHashMap<>();+ questions.put("holds", Question.noul("Does it hold?"));+ questions.put("intent", Question.choice("What now?",+ TinyCheckpoint.ordered("refund", "money back", "replace", "a new one")));+ StringBuilder state = new StringBuilder();+ for (int i = 0; i < 200; i++) {+ state.append("clause ").append(i).append(" says something. ");+ }+ LongPrediction got = agent.predictLong(state.toString(), questions);++ assertTrue(got.usage().windows() > 1,+ () -> "a long state must be scanned, got " + got.usage().windows());++ // The window COUNT, not just "more than one". The scan steps by the STRIDE, and a+ // scan that stepped by the window instead would still produce several windows, still+ // cover the state and still pass every other assertion here -- it would just stop+ // overlapping, so a span sitting across a boundary would be read whole by nothing.+ // Measured: a mutant swapping the step survived all 49 cases before this.+ WindowPlan.Budget plan = WindowPlan.budget(agent.tokenizer(),+ List.copyOf(questions.values()), agent.config().maxLen(),+ agent.config().headMaxLen(), null, null);+ int tokens = stateTokens(agent, state.toString());+ int expected = 0;+ for (int at = 0; at < tokens; at += plan.stride()) {+ expected++;+ if (at + plan.window() >= tokens) {+ break;+ }+ }+ assertEquals(expected, got.usage().windows(),+ () -> "a scan of " + tokens + " tokens at window " + plan.window()+ + " stride " + plan.stride());+ assertTrue(plan.stride() < plan.window(),+ "the default stride overlaps, which is what lets a span near a boundary land "+ + "whole inside some window");+ assertEquals(questions.keySet(), got.answers().keySet(), "every question answered");+ for (String id : questions.keySet()) {+ LongPrediction.Window window = got.window(id).orElseThrow(+ () -> new AssertionError(id + " has no deciding window"));+ assertEquals(got.usage().windows(), window.count(), id + " window count");+ assertTrue(window.index() >= 0 && window.index() < window.count(),+ id + " index inside the scan");+ assertTrue(window.tokens() > 0, id + " span is non-empty");+ // The clamp: no span may name a token past the end of the state.+ assertTrue(window.tokenEnd() <= stateTokens(agent, state.toString()),+ id + " span ends inside the state");+ }+ }+ }++ @Test+ @DisplayName("aggregate refuses input it cannot aggregate")+ void aggregateRefusesImpossibleInput() {+ assertThrows(IllegalArgumentException.class,+ () -> Agent.aggregate(List.of("q"), List.of(), List.of(), 10, 100),+ "a scan with no windows has nothing to aggregate");+ Prediction one = new Prediction(Prediction.MODEL,+ Map.of("q", new Answer.Noul(0.5, 0.5, 0.5, 0.0)),+ new Usage(1, 0, 1, 0, false, List.of(), Map.of()));+ assertThrows(IllegalArgumentException.class,+ () -> Agent.aggregate(List.of("q"), List.of(one), List.of(0, 7), 10, 100),+ "two starts for one result is a caller error, not something to guess at");+ assertThrows(IllegalArgumentException.class,+ () -> Agent.aggregate(List.of("missing"), List.of(one), List.of(0), 10, 100),+ "a question no window answered cannot be aggregated silently");+ }++ @Test+ @DisplayName("a null state or questions map is refused")+ void nullArguments(@org.junit.jupiter.api.io.TempDir java.nio.file.Path root)+ throws Exception {+ TinyCheckpoint.write(root, 64, 24);+ try (Agent agent = TinyCheckpoint.agent(root, new TinyCheckpoint.RecordingSession())) {+ Map<String, Question> questions = Map.of("holds", Question.noul("Does it hold?"));+ assertThrows(IllegalArgumentException.class,+ () -> agent.predictLong(null, questions),+ "a null state would otherwise be answered as a decision about the literal word null");+ assertThrows(IllegalArgumentException.class,+ () -> agent.predictLong("text", null),+ "a null questions map is a caller error");+ }+ }++ // ------------------------------------------------------------------ helpers++ private static int stateTokens(Agent agent, String state) {+ return agent.tokenizer().encode(+ SequenceBuilder.serializeState(state)+ .replace(agent.tokenizer().maskToken(), " ")).length;+ }++ /** The recorded public question specs, as {@link Question} values. */+ @SuppressWarnings("unchecked")+ private static List<Question> questionsOf(Map<String, Object> one) {+ List<Question> out = new ArrayList<>();+ Map<String, Object> specs = (Map<String, Object>) one.get("questions");+ for (Object raw : specs.values()) {+ Map<String, Object> spec = (Map<String, Object>) raw;+ String instructions = (String) spec.get("instructions");+ Object criteria = spec.get("criteria");+ switch ((String) spec.get("type")) {+ case "choice" -> out.add(Question.choice(instructions,+ (Map<String, Object>) criteria));+ case "score" -> out.add(Question.score(instructions, (List<Object>) criteria));+ default -> out.add(Question.noul(instructions));+ }+ }+ return out;+ }++ /** One recorded window's result, as a {@link Prediction}. */+ @SuppressWarnings("unchecked")+ private static Prediction predictionOf(Map<String, Object> window) {+ Map<String, Answer> answers = new LinkedHashMap<>();+ for (Map.Entry<String, Object> entry+ : ((Map<String, Object>) window.get("answers")).entrySet()) {+ answers.put(entry.getKey(), answerOf((Map<String, Object>) entry.getValue()));+ }+ Map<String, Object> usage = (Map<String, Object>) window.get("usage");+ Map<String, Usage.CollapsedOptions> collapsed = new LinkedHashMap<>();+ Map<String, Object> options = (Map<String, Object>) usage.get("options");+ if (options != null) {+ for (Map.Entry<String, Object> entry : options.entrySet()) {+ Map<String, Object> o = (Map<String, Object>) entry.getValue();+ Integer per = o.get("tokens_per_option") == null ? null+ : number(o.get("tokens_per_option"));+ collapsed.put(entry.getKey(), new Usage.CollapsedOptions(+ number(o.get("total")), number(o.get("distinct")), per));+ }+ }+ return new Prediction(Prediction.MODEL, answers, new Usage(+ number(usage.get("input_tokens")), number(usage.get("output_tokens")),+ number(usage.get("state_tokens")), number(usage.get("state_tokens_dropped")),+ Boolean.TRUE.equals(usage.get("truncated")),+ (List<String>) (List<?>) usage.get("truncated_questions"), collapsed));+ }++ @SuppressWarnings("unchecked")+ private static Answer answerOf(Map<String, Object> a) {+ double confidence = dbl(a.get("confidence"));+ double answerConfidence = dbl(a.get("answer_confidence"));+ double act = dbl(((Map<String, Object>) a.get("action")).get("act_probability"));+ switch ((String) a.get("type")) {+ case "noul":+ return new Answer.Noul(dbl(a.get("noul")), confidence, answerConfidence, act);+ case "choice": {+ Map<String, Double> probabilities = new LinkedHashMap<>();+ ((Map<String, Object>) a.get("probabilities"))+ .forEach((k, v) -> probabilities.put(k, dbl(v)));+ return new Answer.Choice((String) a.get("choice"), probabilities,+ confidence, answerConfidence, act);+ }+ default: {+ Map<String, Double> probabilities = new LinkedHashMap<>();+ ((Map<String, Object>) a.get("probabilities"))+ .forEach((k, v) -> probabilities.put(k, dbl(v)));+ Map<String, String> legend = new LinkedHashMap<>();+ ((Map<String, Object>) a.get("legend"))+ .forEach((k, v) -> legend.put(k, (String) v));+ return new Answer.Score(dbl(a.get("score")), legend, probabilities,+ confidence, answerConfidence, act);+ }+ }+ }++ private static int number(Object value) {+ return ((Number) value).intValue();+ }++ private static double dbl(Object value) {+ return ((Number) value).doubleValue();+ }++ private static Integer optional(Object value) {+ return value == null ? null : ((Number) value).intValue();+ }++ @SuppressWarnings("unchecked")+ private static List<Integer> ints(Object value) {+ List<Integer> out = new ArrayList<>();+ for (Object each : (List<Object>) value) {+ out.add(((Number) each).intValue());+ }+ return out;+ }++ @Test+ @DisplayName("the recorded corpus still covers the branches this family exists for")+ @SuppressWarnings("unchecked")+ void corpusStillCoversTheBranches() {+ for (String name : List.of("english", "multilingual")) {+ Map<String, Object> entry = forCheckpoint(name);+ List<Object> cases = (List<Object>) entry.get("cases");+ assertTrue(cases.size() >= 12, () -> name + " lost cases: " + cases.size());+ int single = 0;+ int capped = 0;+ int multi = 0;+ for (Object raw : cases) {+ Map<String, Object> one = (Map<String, Object>) raw;+ if (Boolean.TRUE.equals(one.get("single_call"))) {+ single++;+ } else {+ multi++;+ }+ if (one.get("batch_cap_used") != null) {+ capped++;+ }+ }+ final int s = single;+ final int c = capped;+ final int m = multi;+ assertTrue(s >= 1, () -> name + " has no single-window case");+ assertTrue(m >= 10, () -> name + " has only " + m + " scanned cases");+ assertTrue(c >= 1, () -> name + " never reaches the batch cap");+ }+ assertFalse(family().isEmpty(), "the family is present");+ assertNull(family().get("skipped"), "and was not skipped wholesale");+ }+}diff --git a/laya-java/scripts/gen_fixtures.py b/laya-java/scripts/gen_fixtures.pyindex cfb70c72..676cc668 100644--- a/laya-java/scripts/gen_fixtures.py+++ b/laya-java/scripts/gen_fixtures.py@@ -2634,6 +2634,307 @@ def _window_batch_cap_cases(window_batch_cap):" caps from one that returns None every time")return cases+#: Per-window answers crafted to decide the aggregation, per case. Real model output cannot pin+#: these rules: the deciding window has to be chosen by values the case controls, ties have to be+#: constructed, and a document long enough to produce them would take a forward pass per window.+#:+#: (case id, state recipe, questions, kwargs, per-window answers)+PREDICT_LONG_CASES = [+ # noul: the strongest window decides, carrying its own numbers so the fields stay consistent.+ ("noul-max-wins", ("repeat", 90), {"holds": {"type": "noul",+ "instructions": "Does it hold?"}},+ {}, {"holds": [("noul", 0.20, 0.55), ("noul", 0.90, 0.31),+ ("noul", 0.50, 0.77), ("noul", 0.10, 0.99)]}),+ # A TIE on P(true). Python's `max` keeps the FIRST maximal index, which is a rule a port can+ # silently get backwards and no realistic model output would expose.+ ("noul-tie-takes-the-first", ("repeat", 90),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {}, {"holds": [("noul", 0.30, 0.10), ("noul", 0.90, 0.20),+ ("noul", 0.90, 0.95), ("noul", 0.40, 0.30)]}),+ # choice and score: the most-confident window wins, which is a DIFFERENT field from noul's.+ ("choice-most-confident", ("repeat", 90),+ {"intent": {"type": "choice", "instructions": "What should we do?",+ "criteria": {"refund": "money back", "replace": "new unit"}}},+ {}, {"intent": [("choice", "refund", 0.40), ("choice", "replace", 0.55),+ ("choice", "refund", 0.90), ("choice", "replace", 0.50)]}),+ ("choice-tie-takes-the-first", ("repeat", 90),+ {"intent": {"type": "choice", "instructions": "What should we do?",+ "criteria": {"refund": "money back", "replace": "new unit"}}},+ {}, {"intent": [("choice", "refund", 0.80), ("choice", "replace", 0.80)]}),+ ("score-most-confident", ("repeat", 90),+ {"grade": {"type": "score", "criteria": ["low", "mid", "high"],+ "instructions": "How bad?"}},+ {}, {"grade": [("score", 1.0, 0.30), ("score", 2.5, 0.88), ("score", 0.5, 0.40)]}),+ # Two questions in one scan, decided by DIFFERENT windows -- the case that catches a port+ # which picks one window for the whole document.+ ("two-questions-two-windows", ("repeat", 90),+ {"holds": {"type": "noul", "instructions": "Does it hold?"},+ "intent": {"type": "choice", "instructions": "What should we do?",+ "criteria": {"refund": "money back", "replace": "new unit"}}},+ {}, {"holds": [("noul", 0.15, 0.90), ("noul", 0.95, 0.20), ("noul", 0.40, 0.50)],+ "intent": [("choice", "refund", 0.20), ("choice", "replace", 0.30),+ ("choice", "refund", 0.99)]}),+ # The window span of the LAST window is clamped to the document, not start + budget.+ ("span-clamped-at-the-end", ("repeat", 100),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {}, {"holds": [("noul", 0.1, 0.1), ("noul", 0.2, 0.2), ("noul", 0.99, 0.99)]}),+ # An explicit window and stride change the split, so the spans and the count move with them.+ ("explicit-window-and-stride", ("repeat", 90),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {"window": 120, "stride": 60}, "auto"),+ ("explicit-stride-no-overlap", ("repeat", 90),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {"window": 150, "stride": 150}, "auto"),+ # Short enough for one window: the scan is skipped entirely and the single-call result is+ # returned, with `windows` still written.+ ("fits-one-window", ("repeat", 3),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {}, "single"),+ # A clamped window, so `batch_size=None` becomes a real cap -- recorded as the batch call+ # received it.+ ("clamped-window-caps-the-batch", ("repeat", 120),+ {"intent": {"type": "choice", "instructions": "Pick one of many.",+ "criteria": {("opt%03d" % i): "criterion %d with several words" % i+ for i in range(100)}}},+ {}, "auto"),+ # An explicit batch size is passed through untouched.+ ("explicit-batch-size", ("repeat", 90),+ {"holds": {"type": "noul", "instructions": "Does it hold?"}},+ {"batch_size": 2}, "auto"),+]+++def _long_state(recipe):+ """A deterministic document of a controlled length, in tokens-ish."""+ kind, size = recipe+ assert kind == "repeat", kind+ words = []+ for i in range(size):+ words.append("clause%d states that item %d was billed on day %d" % (i, i % 37, i % 29))+ return ". ".join(words) + "."+++def _answer(spec, index):+ """One window's answer dict, in the shape `predict_batch` returns."""+ kind = spec[0]+ if kind == "noul":+ _, noul, conf = spec+ return {"type": "noul", "noul": float(noul), "confidence": float(conf),+ "answer_confidence": float(conf), "action": {"act_probability": 0.5}}+ if kind == "choice":+ _, choice, conf = spec+ other = "replace" if choice == "refund" else "refund"+ # `confidence` deliberately DIFFERENT from `answer_confidence`, and ordered the other+ # way. A choice is chosen on `answer_confidence`; setting the two equal made a port that+ # read the wrong one indistinguishable from a correct one -- measured, a mutant swapping+ # them survived all 49 cases.+ return {"type": "choice", "choice": choice,+ "probabilities": {choice: float(conf), other: 1.0 - float(conf)},+ "confidence": round(1.0 - float(conf), 4),+ "answer_confidence": float(conf),+ "action": {"act_probability": 0.25}}+ if kind == "score":+ _, score, conf = spec+ # Same here, and for the same reason.+ return {"type": "score", "score": float(score),+ "legend": {"0": "low", "1": "mid", "2": "high"},+ "probabilities": {"0": 0.2, "1": 0.3, "2": 0.5},+ "confidence": round(1.0 - float(conf), 4),+ "answer_confidence": float(conf),+ "action": {"act_probability": 0.75}}+ raise AssertionError(kind)+++def predict_long_scan():+ """`ONNXAgent.predict_long`'s windowing and aggregation, with inference stubbed out.++ The model is not what this pins. The rules are:++ * noul -> P(true) is the MAX over windows (the statement holds if any window supports it)+ * choice -> the answer from the single most-confident window, so a localized signal is not+ out-voted by the many neutral windows a long document is mostly made of+ * score -> the level from the most-confident window, likewise++ and those are chosen by a DIFFERENT field per type (`noul` versus `answer_confidence`), with+ ties resolved to the FIRST maximal window because that is what Python's `max` does. None of+ that is observable from real model output: a tie has to be constructed, and a document long+ enough to produce four windows would cost four forward passes to record.++ So `predict_batch` and `system_one` are stubbed and the real `predict_long` is driven over+ them. That also makes the fixture record what the batch call RECEIVED -- the window texts and+ the batch cap -- which pins the state tokenization, the window offsets, the decode of every+ window and `window_batch_cap`, in one place, against the reference's own splitting code.++ Stubbing inference also means no hook chain is dispatched, so `evidence` stays empty and this+ is exactly the no-hooks path: `answered` false, `rewritten` false, and the question types+ resolved from `_to_internal`. That is the path the JVM port implements.+ """+ import io+ import os++ rig = os.environ.get("LAYA_FIXTURE_CHECKPOINTS", DEFAULT_CHECKPOINT_ROOT)+ out = {"by": {}}+ for name in ("english", "multilingual"):+ root = os.path.join(rig, name)+ directory = os.path.join(root, "tokenizer")+ if not os.path.isfile(os.path.join(directory, "tokenizer.json")):+ out["by"][name] = {"skipped": "no tokenizer/ under %s" % root}+ continue+ from transformers import AutoTokenizer++ from laya.onnx_agent import ONNXAgent++ tok = AutoTokenizer.from_pretrained(directory)+ cfg = json.load(io.open(os.path.join(root, "rl_agent_config.json"), encoding="utf-8"))++ cases, states, splits = [], {}, {}+ for cid, recipe, questions, kwargs, per_window in PREDICT_LONG_CASES:+ state = _long_state(recipe)+ seen = {}++ class Stub(object):+ """Everything `predict_long` touches on `self`, and nothing else."""++ def __init__(self):+ self.tok = tok+ self.cfg = cfg++ _to_internal = staticmethod(ONNXAgent._to_internal)++ def system_one(self, st, qs, lang=None, **kw):+ seen["system_one"] = {"state": st}+ answers = {}+ for qid, qdef in qs.items():+ kind = ONNXAgent._to_internal(qdef)["t"]+ answers[qid] = _answer(+ ("noul", 0.42, 0.61) if kind == "noul"+ else ("choice", "refund", 0.61) if kind == "choice"+ else ("score", 1.5, 0.61), 0)+ return {"model": "laya-rl-agent-onnx", "answers": answers,+ "usage": {"input_tokens": 11, "output_tokens": 3,+ "state_tokens": 7, "state_tokens_dropped": 0,+ "truncated": False, "truncated_questions": []}}++ def predict_batch(self, states, qs, batch_size=None, lang=None, **kw):+ seen["windows"] = list(states)+ seen["batch_size"] = batch_size+ # The same arithmetic `predict_long` used to produce `states`, recorded so the+ # port can be handed the identical (starts, budget, state length) rather than+ # inferring them from the window texts.+ from laya.common import encode_text, serialize_state, window_budget+ internal = [ONNXAgent._to_internal(q) for q in qs.values()]+ budget, step, _ = window_budget(+ tok, internal, cfg.get("max_len", 512), cfg.get("head_max_len", 192),+ window=kwargs.get("window"), stride=kwargs.get("stride"))+ ids = encode_text(tok, serialize_state(state).replace(tok.mask_token, " "),+ add_special_tokens=False)["input_ids"]+ starts, at = [], 0+ while at < len(ids):+ starts.append(at)+ if at + budget >= len(ids):+ break+ at += step+ seen["starts"] = starts+ seen["budget"] = budget+ seen["state_tokens"] = len(ids)+ results = []+ for j, _st in enumerate(states):+ answers = {}+ for qid, qdef in qs.items():+ kind = ONNXAgent._to_internal(qdef)["t"]+ if not isinstance(per_window, dict):+ # Values that rise with the index, so the LAST window decides and+ # the span of the final window is what gets reported.+ conf = 0.10 + 0.05 * j+ spec = (("noul", conf, conf) if kind == "noul"+ else ("choice", "refund", conf) if kind == "choice"+ else ("score", 1.0, conf))+ else:+ specs = per_window[qid]+ spec = specs[j % len(specs)]+ answers[qid] = _answer(spec, j)+ # Usage designed to exercise the merge: numerics SUM, `options` is a+ # per-question dict that must MERGE rather than be replaced, and+ # `backend` is non-numeric so the last window wins.+ usage = {"input_tokens": 100 + j, "output_tokens": 5,+ "state_tokens": 10 + j, "state_tokens_dropped": j,+ "truncated": bool(j % 2), "truncated_questions":+ (["q%d" % j] if j % 2 else []),+ "backend": "window-%d" % j}+ if j % 2 == 0:+ usage["options"] = {("q%d" % j): {"total": 3, "distinct": 2,+ "tokens_per_option": None}}+ results.append({"model": "laya-rl-agent-onnx", "answers": answers,+ "usage": usage})+ seen["results"] = results+ return results++ import warnings++ with warnings.catch_warnings(record=True) as caught:+ warnings.simplefilter("always")+ result = ONNXAgent.predict_long(+ Stub(), state, questions,+ window=kwargs.get("window"), stride=kwargs.get("stride"),+ batch_size=kwargs.get("batch_size"))+ state_key = "%s-%d" % recipe+ states[state_key] = state+ split_key = "%s|%s|%s" % (state_key, kwargs.get("window"), kwargs.get("stride"))+ scanned = seen.get("windows")+ if scanned is not None:+ previous = splits.get(split_key)+ assert previous is None or previous == scanned, (+ "two cases with the same state, window and stride were split differently,"+ " which means %s is not the key it claims to be" % split_key)+ splits[split_key] = scanned+ cases.append({+ "case": cid,+ # The state and the window split live in `states` and `splits`; a case names them.+ "state_key": state_key,+ "split_key": split_key if scanned is not None else None,+ "questions": questions,+ "window": kwargs.get("window"),+ "stride": kwargs.get("stride"),+ "batch_size": kwargs.get("batch_size"),+ "batch_cap_used": seen.get("batch_size"),+ # The per-window results the reference aggregated. Recorded because the+ # aggregation is the thing under test and a model cannot be asked to produce a+ # tie or a chosen deciding window on demand.+ "per_window": seen.get("results"),+ "starts": seen.get("starts"),+ "budget": seen.get("budget"),+ "state_tokens": seen.get("state_tokens"),+ "single_call": "system_one" in seen,+ "result": result,+ "warnings": [str(w.message) for w in caught],+ })++ # `states` and `splits` are what the cases reference. The split is the reference's own+ # window list -- the proof that the token split, the per-window decode and the batch cap+ # all match, independent of the aggregation the cases assert on.+ out["by"][name] = {"max_len": cfg.get("max_len", 512),+ "head_max_len": cfg.get("head_max_len", 192),+ "states": states, "splits": splits, "cases": cases}++ # The rules have to be REACHED. A case list that stopped constructing ties, or stopped+ # producing more than one window, would still compare clean case by case.+ for name, entry in out["by"].items():+ if "skipped" in entry:+ continue+ multi = sum(1 for c in entry["cases"] if not c["single_call"])+ split_sizes = [len(entry["splits"][c["split_key"]])+ for c in entry["cases"] if c["split_key"]]+ spans = sum(1 for c in entry["cases"]+ for a in c["result"]["answers"].values() if "window" in a)+ capped = sum(1 for c in entry["cases"] if c["batch_cap_used"])+ # ...and the scans have to produce several windows each, or the aggregation has nothing+ # to choose between.+ if multi < 8 or spans < 8 or capped < 1 or min(split_sizes or [0]) < 3:+ raise AssertionError(+ "the %s cases reach %d multi-window scans, %d window attributions and %d capped"+ " batches; this family exists to pin those" % (name, multi, spans, capped))+ return out+def _decoder_shape(tokenizer_json):"""The decoder section's type chain, flattened, so the fixture says what it was recorded on."""import io@@ -2655,6 +2956,7 @@ def _decoder_shape(tokenizer_json):FAMILIES = {+ "predict_long.json": predict_long_scan,"window_plan.json": window_plan,"lang_tables.json": lang_tables,"lang_detect.json": lang_detect,