Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
fix(examples): example 33 must teach the budget build_head implements
Aashish261da357698f
diff --git a/examples/33_high_cardinality_choice.py b/examples/33_high_cardinality_choice.pyindex fb7537b9..dc60e6f5 100644--- a/examples/33_high_cardinality_choice.py+++ b/examples/33_high_cardinality_choice.py@@ -1,30 +1,43 @@"""Example 33 -- high-cardinality choice questions and the option token budget.-Laya splits every sequence into an option head (`head_max_len`) and a state body-(`max_len - head_max_len`). A choice question with many options shares one fixed head budget, so-at some point each option gets only a few tokens. This example measures that squeeze on a-20-option and a 77-option (Banking77) question, then raises the budgets to show the answer and-the token budget recover.+Laya sizes the option head against `head_max_len` and gives the state whatever `max_len` has left+after the head it actually built -- which is not `max_len - head_max_len`, because a head built for a+many-option question runs past the cap. A choice question with many options shares one budget, so at+some point each option gets only a few tokens. This example measures that squeeze on a 20-option and+a 77-option (Banking77) question, then raises the budgets to show the answer and the token budget+recover."""from _common import laya, banner, device_line, load-from laya.common import build_sequence+from laya.common import build_head, build_sequencebanner("33", "High-cardinality choice", """Laya formats a call as:[CLS] <typed question + option descriptions> [SEP] [MASK] opt0 [MASK] opt1 ... [SEP] state- The option block lives in `head_max_len`; the state gets what is left of `max_len`. When the- full options overflow the head, the code grants each one+ The option block is sized against `head_max_len`; the state gets what is left of `max_len`.+ `head_max_len` is not a hard cap, and that gap is the whole point of this page.++ Each option keeps its full length while 16 or more tokens of head are still spare. Once the+ spare drops below 16, the code cuts every option toper = max(4, (head_max_len - 16) // N)- tokens (`16` is the reserved minimum for the instruction prefix). At the English default- head_max_len = 192 a 77-option Banking77 question gets `max(4, (192-16)//77) = 4` tokens per- option -- the `[MASK]` plus three subwords -- so labels stop being distinguishable. The- instruction is then clipped to its 8-token floor as well.+ tokens -- `[MASK]` plus `per - 1` subwords -- and clips the instruction to++ head_ids[: max(8, opt_budget)]++ so the floor under the instruction is 8, while the 16 above is the threshold the spare is judged+ against and the reserve the division keeps back before splitting what is left between N options.+ It is that spare which triggers the cut, not a question whose options overflow the head: the+ options are cut while they still fit the head. At the English default head_max_len = 192 a+ 77-option Banking77 question lands on `max(4, (192-16)//77) = 4`, and 77 options x 4 tokens plus+ the instruction build a head longer than 192, which is where the state goes. The run below prints+ the head it built, how far past the cap it reached, the room left for the state, and how many+ options still have a span of their own -- at 4 tokens each some share one span, so the model+ cannot tell those labels apart.This is the documented weak spot: raise `head_max_len`/`max_len`, or split the label spacecoarse-to-fine. Notice how the answer changes here, and that confidence does not warn you.@@ -88,14 +101,16 @@ def budget_report(agent, question, max_len, head_max_len):"""Print the budget arithmetic and return the measured token span of each option."""n = len(question["criteria"])full = option_lengths(agent.tok, question)- if head_max_len - sum(full) < 16:+ spare = head_max_len - sum(full)+ if spare < 16:per = max(4, (head_max_len - 16) // n)- print(" sum(options)=%d > head_max_len-16=%d, so per = max(4, (%d-16)//%d) = %d tokens"- % (sum(full), head_max_len - 16, head_max_len, n, per))+ print(" sum(options)=%d leaves %d tokens of head spare, under the 16 the cut waits for, so "+ "per = max(4, (%d-16)//%d) = %d tokens" % (sum(full), spare, head_max_len, n, per))else:- print(" sum(options)=%d <= head_max_len-16=%d, so every option keeps its full length"- % (sum(full), head_max_len - 16))+ print(" sum(options)=%d leaves %d tokens of head spare, at least 16, so every option keeps "+ "its full length" % (sum(full), spare))+ head_ids, _, stats = build_head(agent.tok, internal(question), head_max_len)ids, markers = build_sequence(agent.tok, MESSAGE, internal(question), max_len, head_max_len)sep_id = agent.tok.sep_token_id@@ -112,6 +127,12 @@ def budget_report(agent, question, max_len, head_max_len):"instruction prefix %d tokens, sequence %d of max_len=%d"% (len(markers), n, min(spans), max(spans), sum(spans) / max(1, len(spans)),markers[0] - 2, len(ids), max_len)) # markers[0] = [CLS] + instructions + [SEP]+ room = max(0, max_len - len(head_ids) - 1)+ print(" the head built is %d tokens, %s head_max_len=%d, so the state has room for %d tokens "+ "and %d of %d options still have a span of their own"+ % (len(head_ids),+ "%d past" % (len(head_ids) - head_max_len) if len(head_ids) > head_max_len else "within",+ head_max_len, room, stats["options_distinct"], stats["options"]))return spansdef ask(agent, question, label):