Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
release: 0.4.25, the Router's vega lane
Nandakishord5a78081fd79
diff --git a/README.md b/README.mdindex d34bf4a4..cc79bb8b 100644--- a/README.md+++ b/README.md@@ -243,6 +243,18 @@ The shipped checkpoints work zero-shot, but fine-tuning on decisions from your o**[nandhakishorm.github.io/laya](https://nandhakishorm.github.io/laya/)**: guides for [prediction hooks](https://nandhakishorm.github.io/laya/hooks/), [schema-driven decisions](https://nandhakishorm.github.io/laya/structured/), [Docker](https://nandhakishorm.github.io/laya/docker/) and [LangChain and LangGraph](https://nandhakishorm.github.io/laya/langchain/), plus a full [API reference](https://nandhakishorm.github.io/laya/reference/).+## What's new in 0.4.25++* **The `Router` has a vega lane.** When vegaml is installed, `Router()` routes what the encoders+ cannot read to `laya.Agent(vegaml=True)`, built on first use: `image=`, `audio=` or `video=` in+ place of a state, a state past 8,192 tokens (`vega_over_tokens`), and any question the vega agent+ has a fitted head for (`Router(vega=agent)` after `agent.fit(...)`). Text the encoders can read+ keeps going to the encoders with the same decisions and reasons as before, `Router(vega=False)`+ turns the lane off, `model="vega"` selects it, and `predict_batch` stays on the encoders. The CLI+ gained `--image`, `--audio` and `--video` for the same lane; the HTTP server refuses the three+ keys with a message saying where media goes. Details in+ [the routing guide](https://nandhakishorm.github.io/laya/routing/#the-vega-lane).+## What's new in 0.4.24* **Video, through the same call: `agent.predict(video="call.mp4", questions=questions)`** on@@ -435,6 +447,7 @@ laya --batch tickets.txt --predict # score a file of requests,cat tickets.txt | laya --batch - --predict --json # stdin; one JSON line of answers per requestlaya "Where is my card" --questions intents.json # answer your own questions, written in a JSON filelaya "Refund my card" --predict --min-confidence 0.9 # mark an answer the model is unsure of+laya --image invoice.png --preset triage # a picture, a recording (--audio) or a clip (--video) on the vega lanelaya # interactive mode```@@ -656,6 +669,8 @@ downloaded models between runs.Laya ships three checkpoints. The built-in **`Router`** is the recommended entry point: it evaluates any state in any language, automatically detects scripts and languages in sub-milliseconds, and dispatches to the optimal checkpoint in a single forward pass.+> **Since 0.4.25, with vegaml installed:** the same `Router` also takes `image=`, `audio=` or `video=` in place of a state and sends them, states past 8,192 tokens, and questions with fitted heads to `laya.Agent(vegaml=True)`; everything else routes exactly as below. `Router(vega=False)` turns that off. See [the vega lane](https://nandhakishorm.github.io/laya/routing/#the-vega-lane).+### Minimal example (30 seconds)```pythondiff --git a/docs/cli-mcp.md b/docs/cli-mcp.mdindex d1767d02..a7774f00 100644--- a/docs/cli-mcp.md+++ b/docs/cli-mcp.md@@ -86,6 +86,16 @@ The main controls are:- `--task NAME` forces the typed-decisions workflow instead of detecting it.- `--device cpu|cuda|...` passes a device choice to the Router.- `--json` emits machine-readable output.+- `--image FILE`, `--audio FILE` or `--video FILE` answer the questions about a picture, a recording+ or a clip instead of a text. These go to the Router's vega lane (`laya.Agent(vegaml=True)`, on by+ default when vegaml is installed, which it is with laya 0.4.23 and newer), imply `--predict`, and+ take the place of the text argument. A recording is transcribed locally; a clip is decided on its+ speech if it has any, else on sampled frames.++```bash+laya --image invoice.png --preset triage+laya --audio call.wav --questions questions.json --json+```### Use a built-in presetdiff --git a/docs/http-api.md b/docs/http-api.mdindex 2fa55550..8912324e 100644--- a/docs/http-api.md+++ b/docs/http-api.md@@ -229,7 +229,10 @@ hook arguments `predict` also takes -- `hooks`, `on_predict_start`, `on_predict_that runs inside the server process, and the last two say how the hooks a deployment installedexecute, so no value a caller sends has a meaning here. The same five are refused client-side by aLangChain node with a `base_url` (`laya.integrations.langchain`), so a chain and a raw HTTP client-now get the same answer.+now get the same answer. `image`, `audio` and `video`, which `predict` takes since 0.4.25 to route a+picture, a recording or a clip to the vega lane, are refused with a `422` as well: this endpoint+takes text states only, and media goes through `laya.Agent(vegaml=True)` or a `Router` with the vega+lane in the process that holds the model.`model` is accepted so a Jev client can keep sending one. The public Hugging Face ids(`convaiinnovations/laya-multilingual`, `convaiinnovations/laya-typed-decisions`), the checkpointdiff --git a/docs/routing.md b/docs/routing.mdindex ff63e8fd..f52b96a9 100644--- a/docs/routing.md+++ b/docs/routing.md@@ -70,6 +70,47 @@ print(decision.reason) # why that checkpoint was selected`decision["model"]` and `decision["reason"]`. `Router.predict()` includes the same decision underthe result's `"routing"` key.+## The vega lane++Since 0.4.25 a `Router` has a fourth lane beside the three encoder checkpoints: `vega`, served by+`laya.Agent(vegaml=True)`, the frozen 0.8B universal reader with vegaml's decision engine. It is on+by default when the `vegaml` package is installed (it ships with laya 0.4.23 and newer) and built+on first use, so a Router costs nothing more to construct than before. Text the encoders can read+keeps going to the encoders, with the same decisions and the same reasons as before. The lane only+takes what the encoders cannot:++| input | routed to vega because |+|---|---|+| `image=`, `audio=` or `video=` instead of a state | the encoders have no vision or speech path |+| a state past `vega_over_tokens` (8,192, the most any encoder reads) | the encoders would truncate it; vega reads up to 73,728 tokens whole |+| a question the vega agent has a fitted head for | the head was fitted on your rows for that question |++```python+from laya import Router++router = Router() # the lane is on if vegaml is installed+router.predict(ticket, questions) # English text: english, as before+router.predict(seventy_page_contract, questions) # about 40,000 tokens: vega, read whole+router.predict(None, questions, image="invoice.png") # vega+router.predict(None, questions, audio="call.wav") # vega, transcribed then decided+router.route(long_text, questions).reason # "state is about 40000 tokens, past the 8192 ..."+```++`Router(vega=False)` turns the lane off and the Router behaves exactly as it did before, including+refusing media with a message that names the lane. `Router(vega=True)` insists on it and raises at+construction when vegaml is missing. `Router(vega="4b")` names a different reader (any name+`laya.Agent(vegaml=...)` takes), and `Router(vega=agent)` attaches an agent you already built, which+is how fitted heads reach the Router: fit on the agent, hand it over, and the questions it serves+route to it. `model="vega"` selects the lane explicitly like any other checkpoint name.++Two things stay on the encoders on purpose. `predict_batch` never routes to vega, because a batch+shares one forward pass on one checkpoint; send media and long states one at a time. And the HTTP+server refuses `image`, `audio` and `video` in a body with a message saying where media goes, since+the JSON surface has no media encoding yet.++Answers from the lane carry vega's fields on top of laya's: `prediction_set`, `abstain`, `unbound`+and `route`; see [vega inside laya](vegaml.md).+## Explain a detection resultUse `explain=True` to inspect why built-in detection reached its result, without loading adiff --git a/docs/vegaml.md b/docs/vegaml.mdindex 7b4f750b..ff6196cd 100644--- a/docs/vegaml.md+++ b/docs/vegaml.md@@ -122,6 +122,12 @@ which measures what the picture itself contributed.## Model switching+Since 0.4.25 the ordinary `laya.Router()` does this by itself when vegaml is installed: images,+speech, video, states past 8,192 tokens and questions with fitted heads go to vega, everything else+to the encoders, with `Router(vega=False)` to turn it off. See+[the vega lane](routing.md#the-vega-lane). `laya.vega.Router` below is the explicit two-model form.++```pythonfrom laya.vega import Routerdiff --git a/laya/__init__.py b/laya/__init__.pyindex 998e42fc..b795750d 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -35,7 +35,7 @@ from .presets import (from .router import DEFAULT_MODELS, RouteDecision, Routerfrom .structured import DecisionResult, decide, decide_batch-__version__ = "0.4.24"+__version__ = "0.4.25"# Routing, language detection and email cleaning are pure Python. The torch-backed names are# resolved lazily so that `import laya` -- and therefore `from laya import Router` ordiff --git a/laya/cli.py b/laya/cli.pyindex 93b3bd94..4b2f4633 100644--- a/laya/cli.py+++ b/laya/cli.py@@ -128,6 +128,15 @@ def build_parser():help="flag an answer whose calibrated confidence falls below P (0.0-1.0) as"" low-confidence; the answer itself is kept, so this asks for a warning"" rather than a refusal. Needs --predict, --preset or --questions.")+ parser.add_argument("--image", metavar="FILE",+ help="answer the questions about a picture instead of a text: routes to the vega lane "+ "(needs vegaml, which ships with laya); implies --predict")+ parser.add_argument("--audio", metavar="FILE",+ help="answer the questions about a recording (wav, flac, ogg, mp3): transcribed locally, "+ "then decided on the vega lane; implies --predict")+ parser.add_argument("--video", metavar="FILE",+ help="answer the questions about a clip: its speech if it has any, else sampled frames, "+ "on the vega lane; implies --predict")parser.add_argument("--device", help="torch device, e.g. cpu or cuda")parser.add_argument("--json", action="store_true", help="print the raw result as JSON")parser.add_argument("--explain", action="store_true",@@ -283,10 +292,31 @@ def resolve_questions(args):return laya.router_questions(), PRESET_STATE_KEYS["router"]+def media_override(args):+ """The three media arguments of `Router.predict`, as the flags gave them: at most one set,+ the others None, so every control the parser names reaches the call."""+ given = {key: (getattr(args, key, None) or None) for key in ("image", "audio", "video")}+ if sum(1 for value in given.values() if value) > 1:+ raise ValueError("pass one of --image, --audio, --video, not %s"+ % " and ".join("--" + k for k, v in given.items() if v))+ return given++def run(text, args, router=None):"""Route or predict one request; returns 0 on success, 2 on a handled error."""router = router or make_router(args)try:+ media = media_override(args)+ if any(media.values()):+ questions, _ = resolve_questions(args)+ result = router.predict(None, questions, model=args.model, task=args.task, lang=args.lang,+ lang_guess=args.lang_guess, **media,+ **budget_overrides(args), **abstention_override(args))+ if args.json:+ print(json.dumps(result, ensure_ascii=False, indent=2, default=str))+ else:+ show_answers(result)+ return 0if args.predict or args.preset or args.questions:questions, state_key = resolve_questions(args)# Each preset's instructions name the field they read -- `` `message` ``,@@ -300,7 +330,7 @@ def run(text, args, router=None):state = {state_key: text}result = router.predict(state, questions,model=args.model, task=args.task, lang=args.lang,- lang_guess=args.lang_guess,+ lang_guess=args.lang_guess, **media,**budget_overrides(args), **abstention_override(args))if args.json:print(json.dumps(result, ensure_ascii=False, indent=2, default=str))@@ -518,6 +548,11 @@ def main(argv=None):print("laya: no requests found in %s." % args.batch, file=sys.stderr)return 2return run_batch(lines, args)+ if any(getattr(args, key, None) for key in ("image", "audio", "video")):+ if text:+ print("laya: pass either a text or one of --image, --audio, --video, not both.", file=sys.stderr)+ return 2+ return run("", args)if not text:return interactive(args)return run(text, args)diff --git a/laya/mcp/remote.py b/laya/mcp/remote.pyindex 526a6032..3c80f7a0 100644--- a/laya/mcp/remote.py+++ b/laya/mcp/remote.py@@ -191,7 +191,16 @@ class RemoteRouter(Router):max_len: Optional[int] = None,head_max_len: Optional[int] = None,min_confidence: Optional[float] = None,+ image: Any = None,+ audio: Any = None,+ video: Any = None,) -> Dict[str, Any]:+ if any(value is not None for value in (image, audio, video)):+ # `Router.predict` takes these since 0.4.25 for the vega lane; laya-serve has no media+ # encoding, so the remote says so here rather than sending a file name as a state+ raise RemoteError("unsupported_remote",+ "images, recordings and clips cannot be sent to laya-serve; use "+ "laya.Agent(vegaml=True) or a local Router with the vega lane")if hooks_timeout is not None:raise RemoteError("unsupported_remote", "hooks_timeout belongs to the server's hooks")if any(value is not None for value in (hooks, on_predict_start, on_predict_end, hooks_raise)):diff --git a/laya/router.py b/laya/router.pyindex 65d48e3c..924dc2c3 100644--- a/laya/router.py+++ b/laya/router.py@@ -55,6 +55,13 @@ DEFAULT_MODELS = {"typed-decisions": (BUNDLE_REPO, "typed-decisions"),}+#: The lane for what the encoders cannot read, served by vegaml's universal reader through+#: `laya.Agent(vegaml=True)` when the Router is built with `vega` on (the default when vegaml is+#: installed): images, speech and video, states past the encoders' 8,192 tokens, and questions+#: with fitted heads. Text the encoders can read keeps going to the encoders exactly as before.+VEGA_MODEL = "vega"+VEGA_REPO = "nandakishorm/vega-U"+# The same checkpoints also live in their own repos, for anyone who prefers them.STANDALONE_MODELS = {"english": "convaiinnovations/laya",@@ -63,6 +70,19 @@ STANDALONE_MODELS = {}+def _estimate_tokens(state: Any) -> int:+ """A cheap token estimate for routing: characters over four, over the flattened state.+ Only the decision "can an encoder read this at all" rests on it, so a rough count is enough."""+ if state is None:+ return 0+ try:+ from .lang import state_text+ text = state_text(state, max_chars=1 << 30)+ except Exception:+ text = str(state)+ return len(text) // 4++def _repo_str(spec):"""Human-readable id for a model spec: 'repo' or 'repo/subfolder'."""repo, sub = _split(spec)@@ -724,6 +744,15 @@ class Router(HookRegistry):hooks_timeout: Optional[float] = None,agent_kwargs: Optional[Dict[str, Any]] = None,sha256_digests: Optional[Dict[str, Optional[Dict[str, str]]]] = None,+ # The vega lane. "auto": on when the vegaml package is installed, built on first use;+ # True: on, and an ImportError now if vegaml is missing; False or None: off, and the Router+ # behaves exactly as it did before 0.4.25; a string: the vegaml reader to use ("4b", a Hub+ # id, an attached local dir); an already built `laya.Agent(vegaml=...)`: used as is, and its+ # fitted heads route the questions they were fitted for.+ vega: Any = "auto",+ # A state estimated past this many tokens goes to vega, because no encoder checkpoint can+ # read it whole; 8,192 is `laya-multilingual` at `max_len=8192`, the most any encoder takes.+ vega_over_tokens: int = 8192,):self.hooks = normalise_hooks(hooks, on_predict_start, on_predict_end)self.hooks_raise = bool(hooks_raise)@@ -787,9 +816,49 @@ class Router(HookRegistry):self._lock = threading.RLock()self._build_lock = threading.Lock()self._loading: Dict[str, _InFlightBuild] = {}+ self.vega_over_tokens = int(vega_over_tokens)+ self._vega_spec: Any = None+ self._install_vega(vega)if preload:self.preload()+ def _install_vega(self, vega: Any) -> None:+ """Register the vega lane according to `vega`, without importing vegaml here: a lazy lane+ is only looked up by `find_spec`, so building a Router stays as light as it was."""+ if vega is None or vega is False:+ return+ if hasattr(vega, "predict") and not isinstance(vega, str):+ self._add(VEGA_MODEL, None)+ self._agents[VEGA_MODEL] = vega+ self._order.append(VEGA_MODEL)+ self.max_loaded = max(self.max_loaded, len(self._agents))+ self._vega_spec = vega+ return+ if isinstance(vega, str) and vega != "auto":+ self._add(VEGA_MODEL, None)+ self._vega_spec = vega+ return+ import importlib.util+ available = importlib.util.find_spec("vegaml") is not None+ if vega is True and not available:+ raise ImportError("Router(vega=True) needs the vegaml package: pip install -U laya "+ "(vegaml ships with laya 0.4.23 and newer)")+ if available:+ self._add(VEGA_MODEL, None)+ self._vega_spec = True++ @property+ def vega_enabled(self) -> bool:+ """Whether this Router has a vega lane to route media, long states and fitted heads to."""+ return VEGA_MODEL in self.models++ def _vega_heads(self) -> set:+ """The question ids a resident vega agent has fitted heads for, or nothing."""+ agent = self._agents.get(VEGA_MODEL)+ engine = getattr(agent, "vega", None)+ heads = getattr(engine, "heads", None)+ return set(heads) if isinstance(heads, dict) else set()+# ------------------------------------------------------------------ registrydef resolve(self, name: Any) -> str:"""Canonical key of a checkpoint this Router knows: built-in, aliased, or registered.@@ -1003,6 +1072,9 @@ class Router(HookRegistry):def _build(self, key: str):"""Construct the Agent for `key`: config is read under `_lock`, the build runs outside it."""from .agent import Agent+ if key == VEGA_MODEL and self._vega_spec is not None and isinstance(self._vega_spec, (bool, str)):+ from .vega import VegaAgent+ return VegaAgent(vegaml=self._vega_spec, device=self.device, token=self.token)with self._lock:if self.models.get(key) is None:raise ValueError("checkpoint %r has no source to load from: it was attached as a built Agent "@@ -1199,6 +1271,13 @@ class Router(HookRegistry):hint = hint(state)return _english_from_code(hint)+ def _vega_repo(self) -> str:+ spec = self._vega_spec+ if isinstance(spec, str):+ return spec+ model_id = getattr(spec, "model_id", None)+ return model_id if isinstance(model_id, str) else VEGA_REPO+# ------------------------------------------------------------------ routingdef route(self,@@ -1213,9 +1292,14 @@ class Router(HookRegistry):hooks_timeout: Optional[float] = None,*,explain: bool = False,+ media: Optional[str] = None,+ allow_vega: bool = True,) -> RouteDecision:"""Decide which checkpoint to use, then let `on_route` hooks observe or replace it.+ `media` is "image", "audio" or "video" when the caller is passing one of those instead of+ a text state; see `predict`. `allow_vega=False` keeps the decision on the encoders.+`ctx.decision` is the `RouteDecision`; a hook may replace it (for example to pin acheckpoint) and the replacement is what gets returned and used. `hooks` are per-callhooks, appended after any installed on the Router.@@ -1226,7 +1310,7 @@ class Router(HookRegistry):if questions is not None and not isinstance(questions, dict):raise TypeError("questions must be a dict of question id -> definition, got %s" % type(questions).__name__)decision = self._route(state, questions, model=model, task=task, lang=lang, lang_guess=lang_guess,- **({"explain": True} if explain else {}))+ media=media, allow_vega=allow_vega, **({"explain": True} if explain else {}))raise_errors = self.hooks_raise if hooks_raise is None else bool(hooks_raise)active = compose_hooks(self.hooks, hooks)timeout = self.hooks_timeout if hooks_timeout is None else validate_timeout(hooks_timeout)@@ -1246,11 +1330,18 @@ class Router(HookRegistry):lang_guess: Optional[Any] = None,*,explain: bool = False,+ media: Optional[str] = None,+ allow_vega: bool = True,) -> RouteDecision:"""Decide which checkpoint to use, without loading or running anything.- Precedence: explicit `model` > explicit `task` > detected workflow (opt-in) >- explicit `lang` > `lang_guess` > detected script/language > default.+ Precedence: explicit `model` > explicit `task` > the vega lane (an image, a recording or a+ clip; a question with a fitted head; a state past `vega_over_tokens`) > detected workflow+ (opt-in) > explicit `lang` > `lang_guess` > detected script/language > default.++ `media` names the kind of non-text input the caller passed ("image", "audio" or "video");+ `allow_vega=False` keeps a decision on the encoders, which is what `predict_batch` does,+ because a batch shares one forward pass on one checkpoint.`lang_guess` is an opt-in hint -- a language code or a callable taking the state --checked after an explicit `lang` and before the built-in detection. It only answers@@ -1271,6 +1362,28 @@ class Router(HookRegistry):return RouteDecision(model=key, repo=_repo_str(self.models.get(key)), reason="explicit task=%r" % task,detection=None, workflow=None)+ vega_on = allow_vega and self.vega_enabled+ if media is not None:+ if not vega_on:+ raise ValueError("an %s needs the vega lane, which this Router does not have: install "+ "vegaml (pip install -U laya) and build the Router with vega='auto' or "+ "vega=True, or pass media to laya.Agent(vegaml=True) directly" % media)+ return RouteDecision(model=VEGA_MODEL, repo=self._vega_repo(),+ reason="%s input: the encoders have no vision or speech path" % media,+ detection=None, workflow=None)+ if vega_on:+ fitted = sorted(q for q in (questions or {}) if q in self._vega_heads())+ if fitted:+ return RouteDecision(model=VEGA_MODEL, repo=self._vega_repo(),+ reason="vega has fitted heads for %s" % ", ".join(fitted),+ detection=None, workflow=None)+ n_tokens = _estimate_tokens(state)+ if n_tokens > self.vega_over_tokens:+ return RouteDecision(model=VEGA_MODEL, repo=self._vega_repo(),+ reason="state is about %d tokens, past the %d the encoders can read; "+ "vega reads it whole" % (n_tokens, self.vega_over_tokens),+ detection=None, workflow=None)+workflow = match_typed_decisions_workflow(questions or {})if workflow and self.auto_task_detection:return RouteDecision(model="typed-decisions", repo=_repo_str(self.models["typed-decisions"]),@@ -1353,16 +1466,29 @@ class Router(HookRegistry):max_len: Optional[int] = None,head_max_len: Optional[int] = None,min_confidence: Optional[float] = None,+ image: Any = None,+ audio: Any = None,+ video: Any = None,) -> Dict[str, Any]:"""Route, then answer every question in one forward pass on the chosen checkpoint.+ `image`, `audio` or `video` in place of `state` route to the vega lane, which is the only+ checkpoint that can read them; so does a state past `vega_over_tokens` and a question the+ vega agent has a fitted head for. Everything else routes exactly as before.+The result is the usual `system_one` payload plus a `routing` key recording the decision.Router-level `on_predict_start` / `on_predict_end` hooks wrap the whole route+infer calland see `ctx.decision`; see `laya.hooks`. `max_len` / `head_max_len` override the agenttoken budget for this call (a start hook may set `ctx.max_len` / `ctx.head_max_len`)."""- if state is None:- raise TypeError("state must not be None; pass a string, dict, or list")+ media_given = [k for k, v in (("image", image), ("audio", audio), ("video", video)) if v is not None]+ if len(media_given) > 1:+ raise ValueError("pass one of image=, audio=, video=; got %s" % ", ".join(media_given))+ media = media_given[0] if media_given else None+ if state is None and media is None:+ raise TypeError("state must not be None; pass a string, dict, or list (or image=, audio=, video=)")+ if state is not None and media is not None:+ raise ValueError("pass either a state or %s=, not both" % media)if not isinstance(questions, dict):raise TypeError("questions must be a dict of question id -> definition, got %s" % type(questions).__name__)mc = check_min_confidence(min_confidence) if min_confidence is not None else None@@ -1389,7 +1515,7 @@ class Router(HookRegistry):# Per-call hooks apply to the whole call, including on_route inside route().decision = self.route(state, questions, model=model, task=task, lang=lang,lang_guess=lang_guess, hooks=hooks, hooks_raise=hooks_raise,- hooks_timeout=hooks_timeout)+ hooks_timeout=hooks_timeout, media=media)ctx.decision = dict(decision)ctx.model = decision["model"]agent = self.load(decision["model"])@@ -1421,14 +1547,20 @@ class Router(HookRegistry):overrides["max_len"] = ctx.max_lenif ctx.head_max_len is not None:overrides["head_max_len"] = ctx.head_max_len- try:- result = agent.system_one(ctx.states[0], ctx.questions,- lang=effective_lang, **overrides)- except TypeError as e:- if "unexpected keyword argument 'lang'" in str(e):- result = agent.system_one(ctx.states[0], ctx.questions, **overrides)- else:- raise+ if decision["model"] == VEGA_MODEL:+ # vega's predict takes laya's shape plus the media; `max_len` and+ # `head_max_len` are recorded there and not applied+ result = agent.predict(ctx.states[0], ctx.questions, lang=effective_lang,+ image=image, audio=audio, video=video, **overrides)+ else:+ try:+ result = agent.system_one(ctx.states[0], ctx.questions,+ lang=effective_lang, **overrides)+ except TypeError as e:+ if "unexpected keyword argument 'lang'" in str(e):+ result = agent.system_one(ctx.states[0], ctx.questions, **overrides)+ else:+ raisefinally:_SKIP_DEFAULTS.reset(skip)if scan is not None:@@ -1633,6 +1765,7 @@ class Router(HookRegistry):hooks=hooks,hooks_raise=hooks_raise,hooks_timeout=hooks_timeout,+ allow_vega=False,**({"explain": True} if explain else {}),))@@ -1645,6 +1778,7 @@ class Router(HookRegistry):task=request.get("task"),lang=request.get("lang"),lang_guess=request.get("lang_guess"),+ allow_vega=False,**({"explain": True} if explain else {}),))@@ -1656,6 +1790,7 @@ class Router(HookRegistry):task=request.get("task"),lang=request.get("lang"),lang_guess=request.get("lang_guess"),+ allow_vega=False,**({"explain": True} if explain else {}),)ctx = PredictContext(diff --git a/laya/serve.py b/laya/serve.pyindex b9a3d52d..07fef247 100644--- a/laya/serve.py+++ b/laya/serve.py@@ -176,6 +176,11 @@ DEFAULT_MAX_TOKEN_BUDGET = 8192# language it reads, how many tokens it gets and where it abstains are all plain data.BODY_CONTROLS = ("model", "max_len", "head_max_len", "task", "lang", "lang_guess", "min_confidence")BODY_REFUSALS = ("hooks", "on_predict_start", "on_predict_end", "hooks_raise", "hooks_timeout")+# `image`, `audio` and `video` are `Router.predict` arguments since 0.4.25 and route to the vega+# lane; the HTTP body has no media encoding yet, so they are refused here rather than accepted as+# strings and passed on, and the response says where media goes (laya.Agent(vegaml=True) locally).+# A tuple of their own, because ``BODY_REFUSALS`` is pinned as the hook set LangChain refuses too.+MEDIA_REFUSALS = ("image", "audio", "video")# ``Router.predict_batch`` reads two different kinds of control: call-level keyword arguments that# apply to the whole batch (chunking, padding, the abstention threshold) and per-request keys that@@ -202,9 +207,12 @@ BATCH_BODY_OUT_PARAMETERS = ("diagnostics",)# Route-only requests cannot apply prediction budgets, abstention or batch controls.# Hook policy belongs to the operator, including the on_route callback itself.ROUTE_BODY_CONTROLS = ("model", "task", "lang", "lang_guess", "explain")-ROUTE_BODY_REFUSALS = BODY_REFUSALS + (+ROUTE_BODY_REFUSALS = BODY_REFUSALS + MEDIA_REFUSALS + ("on_route", "max_len", "head_max_len", "min_confidence", "batch_size","sort_by_length", "diagnostics", "states",+ # `Router.route` takes these two since 0.4.25: `media` names a non-text input and+ # `allow_vega` is how `predict_batch` keeps a batch on the encoders; neither is a body control+ "media", "allow_vega",)@@ -672,6 +680,13 @@ def _refuse_body_refusals(body: Dict[str, Any]) -> None:from fastapi import HTTPExceptiongiven = sorted(key for key in BODY_REFUSALS if key in body and body[key] is not None)+ media = sorted(key for key in MEDIA_REFUSALS if key in body and body[key] is not None)+ if media:+ raise HTTPException(+ status_code=422,+ detail="%s cannot be sent to this endpoint, which takes text states only; images, "+ "recordings and clips go through laya.Agent(vegaml=True) or a Router with the "+ "vega lane, in the process that holds the model" % ", ".join(media))if given:raise HTTPException(status_code=422,diff --git a/nix/package.nix b/nix/package.nixindex 0a7dca73..a94767f8 100644--- a/nix/package.nix+++ b/nix/package.nix@@ -12,7 +12,7 @@ let# side, where tests/test_packaging.py compares all three declarations (this one,# pyproject.toml and laya.__version__) and fails CI when they diverge. Nothing# did, which is how this sat at 0.3.4 while the package reached 0.3.20.- version = "0.4.24";+ version = "0.4.25";src = ../.;format = "setuptools";propagatedBuildInputs = with python3Packages; [diff --git a/pyproject.toml b/pyproject.tomlindex ccfa994d..2b6ce84a 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"[project]name = "laya"-version = "0.4.24"+version = "0.4.25"description = "Fast, non-autoregressive System 1 decision engine with calibrated probabilities"readme = "README.md"requires-python = ">=3.10"diff --git a/scripts/test_suites.py b/scripts/test_suites.pyindex 64b6308f..57117d3c 100644--- a/scripts/test_suites.py+++ b/scripts/test_suites.py@@ -128,6 +128,7 @@ PYTEST_SUITES = ["tests/test_compile.py","tests/test_finetune_entrypoints.py","tests/test_vega.py",+ "tests/test_router_vega.py",]diff --git a/tests/test_router_vega.py b/tests/test_router_vega.pynew file mode 100644index 00000000..15b2da56--- /dev/null+++ b/tests/test_router_vega.py@@ -0,0 +1,150 @@+"""The vega lane on `laya.Router`: media, long states and fitted heads go to vega; everything the+encoders can read keeps going to the encoders exactly as before 0.4.25.++Weight-free: a stub stands in for `laya.Agent(vegaml=True)`, so the decisions are tested, not+the engine. The lane is off when vegaml is absent and `vega` is left at "auto", which is what+the first test pins, since that is every pre-0.4.25 deployment's situation.+"""+import importlib.util+import os+import sys++import pytest++sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))+from laya.router import VEGA_MODEL, Router, _estimate_tokens # noqa: E402++HAS_VEGAML = importlib.util.find_spec("vegaml") is not None+Q = {"dept": {"type": "choice", "instructions": "Which team?",+ "criteria": {"billing": "money", "technical": "bugs"}},+ "urgent": {"type": "noul", "instructions": "Urgent?"}}+ENGLISH = "Please refund the duplicate charge on invoice 4411 today."+LONG = ("The contract between the parties states the following. " * 3000) # ~40k tokens+++class _StubEngine:+ def __init__(self, heads):+ self.heads = heads+++class _StubVega:+ """What the Router needs from a vega agent: `.vega.heads`, `predict`, `predict_long`."""+ model_id = "stub/vega"++ def __init__(self, heads=()):+ self.vega = _StubEngine({h: object() for h in heads})+ self.calls = []++ def predict(self, state=None, questions=None, lang=None, image=None, audio=None, video=None, **kw):+ self.calls.append({"state": state, "image": image, "audio": audio, "video": video, "lang": lang, **kw})+ answers = {q: {"choice": "billing", "probabilities": {"billing": 0.9, "technical": 0.1},+ "answer_confidence": 0.9, "confidence": 0.9, "route": "auto"} if d["type"] == "choice"+ else {"noul": 0.2, "answer_confidence": 0.8, "confidence": 0.8, "route": "auto"}+ for q, d in questions.items()}+ return {"answers": answers, "usage": {"input_tokens": 10, "output_tokens": 0}, "latency_ms": 1.0,+ "modality": "image" if image is not None else "audio" if audio is not None else+ "video" if video is not None else "text"}++ system_one = predict++ def predict_long(self, state, questions, **kw):+ out = self.predict(state, questions, **kw)+ out["usage"]["windows"] = 1+ return out+++# ---------------------------------------------------------------------- off by default without vegaml[email protected](HAS_VEGAML, reason="vegaml is installed here, so auto turns the lane on")+def test_without_vegaml_the_lane_is_off_and_nothing_changes():+ r = Router()+ assert not r.vega_enabled and VEGA_MODEL not in r.models+ assert r.route(ENGLISH).model == "english"+ assert r.route(LONG).model != VEGA_MODEL, "a long state stays on the encoders when there is no lane"+ with pytest.raises(ValueError, match="needs the vega lane"):+ r.route(None, Q, media="image")+ with pytest.raises(ImportError, match="pip install -U laya"):+ Router(vega=True)+++def test_vega_false_keeps_the_old_behaviour_even_with_an_agent_available():+ r = Router(vega=False)+ assert not r.vega_enabled+ assert r.route(LONG).model != VEGA_MODEL+ with pytest.raises(ValueError, match="needs the vega lane"):+ r.predict(None, Q, image=object())+++# ------------------------------------------------------------------------------- the lane itself+def test_an_attached_vega_agent_turns_the_lane_on_and_routes_what_the_encoders_cannot_read():+ stub = _StubVega(heads=("urgent",))+ r = Router(vega=stub)+ assert r.vega_enabled and r.load(VEGA_MODEL) is stub+ # text the encoders can read is untouched+ d = r.route(ENGLISH, {"dept": Q["dept"]})+ assert d.model == "english" and d.reason == "English Latin text"+ # media+ d = r.route(None, Q, media="image")+ assert d.model == VEGA_MODEL and "image input" in d.reason and d["repo"] == "stub/vega"+ # a state past the encoders' budget+ d = r.route(LONG, {"dept": Q["dept"]})+ assert d.model == VEGA_MODEL and "past the 8192" in d.reason+ # a question with a fitted head, however short the state+ d = r.route(ENGLISH, Q)+ assert d.model == VEGA_MODEL and "fitted heads for urgent" in d.reason+ # explicit model= still wins, both ways+ assert r.route(LONG, Q, model="english").model == "english"+ assert r.route(ENGLISH, {"dept": Q["dept"]}, model="vega").model == VEGA_MODEL+++def test_predict_hands_media_and_long_states_to_vega_and_stamps_the_routing():+ stub = _StubVega()+ r = Router(vega=stub)+ out = r.predict(None, Q, image="invoice.png")+ assert out["routing"]["model"] == VEGA_MODEL and out["modality"] == "image"+ assert stub.calls[-1]["image"] == "invoice.png" and stub.calls[-1]["state"] is None+ out = r.predict(LONG, Q)+ assert out["routing"]["model"] == VEGA_MODEL and stub.calls[-1]["state"] == LONG+ out = r.predict_long(LONG, Q)+ assert out["routing"]["model"] == VEGA_MODEL and out["usage"]["windows"] == 1+ with pytest.raises(ValueError, match="pass one of"):+ r.predict(None, Q, image="a", audio="b")+ with pytest.raises(ValueError, match="not both"):+ r.predict(ENGLISH, Q, video="c")+++def test_vega_over_tokens_is_the_line_and_the_estimate_is_characters_over_four():+ stub = _StubVega()+ assert _estimate_tokens("a" * 400) == 100 and _estimate_tokens({"k": "b" * 40}) == 10+ r = Router(vega=stub, vega_over_tokens=100)+ assert r.route("x" * 404, {"dept": Q["dept"]}).model == VEGA_MODEL+ assert r.route("x" * 396, {"dept": Q["dept"]}).model != VEGA_MODEL+++def test_predict_batch_never_routes_to_vega():+ stub = _StubVega(heads=("urgent",))+ r = Router(vega=stub)+ decisions = [r._route(LONG, Q, allow_vega=False), r._route(ENGLISH, Q, allow_vega=False)]+ assert all(d.model != VEGA_MODEL for d in decisions)+ with pytest.raises(ValueError, match="needs the vega lane"):+ r._route(None, Q, media="image", allow_vega=False)+++def test_a_reader_name_registers_a_lazy_lane():+ r = Router(vega="4b")+ assert r.vega_enabled and r.route(None, Q, media="audio")["repo"] == "4b"+ assert r.models[VEGA_MODEL] is None, "no repo source: the lane is built by VegaAgent, not Agent"+ assert VEGA_MODEL in r.registered++[email protected](not HAS_VEGAML, reason="needs vegaml to test the auto lane")+def test_auto_turns_the_lane_on_when_vegaml_is_present_without_importing_it():+ before = "vegaml" in sys.modules+ r = Router()+ assert r.vega_enabled+ assert ("vegaml" in sys.modules) == before, "building a Router must not import vegaml"+ assert r.route(ENGLISH, {"dept": Q["dept"]}).model == "english"+ assert r.route(LONG, {"dept": Q["dept"]}).model == VEGA_MODEL+++if __name__ == "__main__":+ sys.exit(pytest.main([__file__, "-q"]))diff --git a/tests/test_serve.py b/tests/test_serve.pyindex 7e568038..70dda790 100644--- a/tests/test_serve.py+++ b/tests/test_serve.py@@ -27,6 +27,7 @@ from laya.serve import ( # noqa: E402BATCH_BODY_ITEM_CONTROLS,BODY_CONTROLS,BODY_REFUSALS,+ MEDIA_REFUSALS,DEFAULT_BODY_READ_TIMEOUT,DEFAULT_LOG_LEVEL,DEFAULT_MAX_TOKEN_BUDGET,@@ -2920,8 +2921,8 @@ def test_every_predict_control_is_forwarded_or_refused():from laya.router import Routertaken = set(inspect.signature(Router.predict).parameters) - {"self", "state", "questions"}- declared = set(BODY_CONTROLS) | set(BODY_REFUSALS)- assert not set(BODY_CONTROLS) & set(BODY_REFUSALS)+ declared = set(BODY_CONTROLS) | set(BODY_REFUSALS) | set(MEDIA_REFUSALS)+ assert not set(BODY_CONTROLS) & set(BODY_REFUSALS) and not set(BODY_CONTROLS) & set(MEDIA_REFUSALS)assert taken == declared, "predict() takes %s; serve declares %s" % (sorted(taken), sorted(declared))@@ -3052,7 +3053,8 @@ def test_a_hook_control_is_refused_not_dropped(monkeypatch, key):says that itself instead of answering as though the control had been honoured."""value = {"hooks": [], "on_predict_start": "cache", "on_predict_end": "audit",- "hooks_raise": False, "hooks_timeout": 5.0}[key]+ "hooks_raise": False, "hooks_timeout": 5.0,+ "image": "invoice.png", "audio": "call.wav", "video": "clip.mp4"}[key]client, fake = _budget_client(monkeypatch)r = client.post("/v1/systemone", json={**REQ, key: value})assert r.status_code == 422, r.text@@ -3331,6 +3333,23 @@ def test_bad_control_on_batch_is_refused_before_inference(monkeypatch, key, valuassert not fake.batch_calls, "a refused body must not reach predict_batch"[email protected]("key", MEDIA_REFUSALS)+def test_media_is_refused_on_both_endpoints_with_a_pointer(monkeypatch, key):+ """`image`, `audio` and `video` are `Router.predict` arguments since 0.4.25, so the single+ endpoint has to say it cannot take them rather than forward a file name as a state; the batch+ endpoint refuses them for the same reason. The answer names where media goes."""+ client, fake = _budget_client(monkeypatch)+ r = client.post("/v1/systemone", json={**REQ, key: "%s.bin" % key})+ assert r.status_code == 422, r.text+ assert key in r.json()["detail"] and "vegaml=True" in r.json()["detail"]+ assert not fake.calls+ client, fake = _batch_client(monkeypatch)+ r = client.post("/v1/systemone/batch", json={**BATCH_REQ, key: "%s.bin" % key})+ assert r.status_code == 422, r.text+ assert key in r.json()["detail"] and "cannot be sent to this endpoint" in r.json()["detail"]+ assert not fake.batch_calls++@pytest.mark.parametrize("key", BODY_REFUSALS)def test_batch_hook_control_is_refused_not_dropped(monkeypatch, key):"""The batch endpoint must honour the same refusal policy the single endpoint already states.@@ -3341,7 +3360,8 @@ def test_batch_hook_control_is_refused_not_dropped(monkeypatch, key):reason they are on ``/v1/systemone``: they run inside the server process."""value = {"hooks": [], "on_predict_start": "cache", "on_predict_end": "audit",- "hooks_raise": False, "hooks_timeout": 5.0}[key]+ "hooks_raise": False, "hooks_timeout": 5.0,+ "image": "invoice.png", "audio": "call.wav", "video": "clip.mp4"}[key]client, fake = _batch_client(monkeypatch)r = client.post("/v1/systemone/batch", json={**BATCH_REQ, key: value})assert r.status_code == 422, r.text