Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

fix(examples): example 33 must teach the budget build_head implements

Aashish261da357698f
diff --git a/examples/33_high_cardinality_choice.py b/examples/33_high_cardinality_choice.py
index fb7537b9..dc60e6f5 100644
--- a/examples/33_high_cardinality_choice.py
+++ b/examples/33_high_cardinality_choice.py
@@ -1,30 +1,43 @@
"""Example 33 -- high-cardinality choice questions and the option token budget.
-Laya splits every sequence into an option head (`head_max_len`) and a state body
-(`max_len - head_max_len`). A choice question with many options shares one fixed head budget, so
-at some point each option gets only a few tokens. This example measures that squeeze on a
-20-option and a 77-option (Banking77) question, then raises the budgets to show the answer and
-the token budget recover.
+Laya sizes the option head against `head_max_len` and gives the state whatever `max_len` has left
+after the head it actually built -- which is not `max_len - head_max_len`, because a head built for a
+many-option question runs past the cap. A choice question with many options shares one budget, so at
+some point each option gets only a few tokens. This example measures that squeeze on a 20-option and
+a 77-option (Banking77) question, then raises the budgets to show the answer and the token budget
+recover.
"""
from _common import laya, banner, device_line, load
-from laya.common import build_sequence
+from laya.common import build_head, build_sequence
banner("33", "High-cardinality choice", """
Laya formats a call as:
[CLS] <typed question + option descriptions> [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state
- The option block lives in `head_max_len`; the state gets what is left of `max_len`. When the
- full options overflow the head, the code grants each one
+ The option block is sized against `head_max_len`; the state gets what is left of `max_len`.
+ `head_max_len` is not a hard cap, and that gap is the whole point of this page.
+
+ Each option keeps its full length while 16 or more tokens of head are still spare. Once the
+ spare drops below 16, the code cuts every option to
per = max(4, (head_max_len - 16) // N)
- tokens (`16` is the reserved minimum for the instruction prefix). At the English default
- head_max_len = 192 a 77-option Banking77 question gets `max(4, (192-16)//77) = 4` tokens per
- option -- the `[MASK]` plus three subwords -- so labels stop being distinguishable. The
- instruction is then clipped to its 8-token floor as well.
+ tokens -- `[MASK]` plus `per - 1` subwords -- and clips the instruction to
+
+ head_ids[: max(8, opt_budget)]
+
+ so the floor under the instruction is 8, while the 16 above is the threshold the spare is judged
+ against and the reserve the division keeps back before splitting what is left between N options.
+ It is that spare which triggers the cut, not a question whose options overflow the head: the
+ options are cut while they still fit the head. At the English default head_max_len = 192 a
+ 77-option Banking77 question lands on `max(4, (192-16)//77) = 4`, and 77 options x 4 tokens plus
+ the instruction build a head longer than 192, which is where the state goes. The run below prints
+ the head it built, how far past the cap it reached, the room left for the state, and how many
+ options still have a span of their own -- at 4 tokens each some share one span, so the model
+ cannot tell those labels apart.
This is the documented weak spot: raise `head_max_len`/`max_len`, or split the label space
coarse-to-fine. Notice how the answer changes here, and that confidence does not warn you.
@@ -88,14 +101,16 @@ def budget_report(agent, question, max_len, head_max_len):
"""Print the budget arithmetic and return the measured token span of each option."""
n = len(question["criteria"])
full = option_lengths(agent.tok, question)
- if head_max_len - sum(full) < 16:
+ spare = head_max_len - sum(full)
+ if spare < 16:
per = max(4, (head_max_len - 16) // n)
- print(" sum(options)=%d > head_max_len-16=%d, so per = max(4, (%d-16)//%d) = %d tokens"
- % (sum(full), head_max_len - 16, head_max_len, n, per))
+ print(" sum(options)=%d leaves %d tokens of head spare, under the 16 the cut waits for, so "
+ "per = max(4, (%d-16)//%d) = %d tokens" % (sum(full), spare, head_max_len, n, per))
else:
- print(" sum(options)=%d <= head_max_len-16=%d, so every option keeps its full length"
- % (sum(full), head_max_len - 16))
+ print(" sum(options)=%d leaves %d tokens of head spare, at least 16, so every option keeps "
+ "its full length" % (sum(full), spare))
+ head_ids, _, stats = build_head(agent.tok, internal(question), head_max_len)
ids, markers = build_sequence(agent.tok, MESSAGE, internal(question), max_len, head_max_len)
sep_id = agent.tok.sep_token_id
@@ -112,6 +127,12 @@ def budget_report(agent, question, max_len, head_max_len):
"instruction prefix %d tokens, sequence %d of max_len=%d"
% (len(markers), n, min(spans), max(spans), sum(spans) / max(1, len(spans)),
markers[0] - 2, len(ids), max_len)) # markers[0] = [CLS] + instructions + [SEP]
+ room = max(0, max_len - len(head_ids) - 1)
+ print(" the head built is %d tokens, %s head_max_len=%d, so the state has room for %d tokens "
+ "and %d of %d options still have a span of their own"
+ % (len(head_ids),
+ "%d past" % (len(head_ids) - head_max_len) if len(head_ids) > head_max_len else "within",
+ head_max_len, room, stats["options_distinct"], stats["options"]))
return spans
def ask(agent, question, label):