Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(agent,onnx): say predict_long's truncation keys can disagree

Som Tripathi054e4d8eb3fc
diff --git a/laya/agent.py b/laya/agent.py
index 4cfe8be4..7cfc61fd 100644
--- a/laya/agent.py
+++ b/laya/agent.py
@@ -1176,9 +1176,11 @@ class Agent(HookRegistry):
Across several windows the truncation keys are combined like every other `usage` field:
`truncated`, `state_tokens` and `state_tokens_dropped` are summed (so `truncated` is the
number of windows that were cut, and the token counts include the overlap), and
- `truncated_questions` is the last window's list. A window is cut when it is larger than
- the room a question's head leaves, from a `window` above the default or a start hook that
- narrows `max_len` / `head_max_len`. Test `usage["truncated"] > 0` here, not `is True`.
+ `truncated_questions` is the last window's list. The two can disagree: when only an
+ earlier window was cut, `truncated` is above 0 and `truncated_questions` is empty. A
+ window is cut when it is larger than the room a question's head leaves, from a `window`
+ above the default or a start hook that narrows `max_len` / `head_max_len`. Test
+ `usage["truncated"] > 0` here, not `is True`.
"""
if aggregate != "auto":
raise ValueError("predict_long: only aggregate='auto' is supported")
diff --git a/laya/onnx_agent.py b/laya/onnx_agent.py
index e3ab8b0e..fa3bf502 100644
--- a/laya/onnx_agent.py
+++ b/laya/onnx_agent.py
@@ -418,7 +418,7 @@ class ONNXAgent(HookRegistry):
Returns a single result dict, the same shape as `system_one`, with `usage["windows"]`
added. Across several windows the truncation keys are summed or carried the same way as
in `Agent.predict_long`: `truncated` is a window count and `truncated_questions` is the
- last window's list.
+ last window's list, so `truncated` can be above 0 while the list is empty.
"""
from .agent import _start_evidence, _with_start_probe
from .hooks import aggregate_usage