Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
release: 0.4.26, laya[vision] and laya[audio]
Nandakishor3293116c8942
diff --git a/README.md b/README.mdindex cc79bb8b..427ab7ed 100644--- a/README.md+++ b/README.md@@ -29,7 +29,7 @@ python -m pip install layaWith [uv](https://docs.astral.sh/uv/), run `uv add laya` in a uv project or `uv pip install laya` in a virtual environment.-Python 3.10 or newer. Optional extras: `laya[serve]` (HTTP server), `laya[mcp]` (MCP server), `laya[langchain]` (LangChain and LangGraph), `laya[llamaindex]` (LlamaIndex selectors), `laya[crewai]` (CrewAI routing), `laya[onnx]` (ONNX Runtime), `laya[fast]` (TileLang GPU fast path), `laya[structured]` (pydantic models in `decide`). Step-by-step setup for each platform, CPU-only or GPU PyTorch builds, and troubleshooting are in [Installation details](#installation-details).+Python 3.10 or newer. Optional extras: `laya[serve]` (HTTP server), `laya[mcp]` (MCP server), `laya[langchain]` (LangChain and LangGraph), `laya[llamaindex]` (LlamaIndex selectors), `laya[crewai]` (CrewAI routing), `laya[onnx]` (ONNX Runtime), `laya[fast]` (TileLang GPU fast path), `laya[structured]` (pydantic models in `decide`), `laya[vision]` (Pillow and torchvision, for `predict(image=...)` and `predict(video=...)`), `laya[audio]` (soundfile, to decode audio files). Step-by-step setup for each platform, CPU-only or GPU PyTorch builds, and troubleshooting are in [Installation details](#installation-details).For TypeScript / Node.js / browser, see [`laya-ts/`](https://github.com/NandhaKishorM/laya/tree/main/laya-ts/). npm releases (`npm install laya-ts`) are published from this repository's `laya-ts-v*` release tags.@@ -107,8 +107,9 @@ The rule, in order: an image or audio goes to vega; text longer than the fast mo# text: up to 73,728 tokens in one pass; usage["truncated"] says if a state hit the limitagent.predict_long(seventy_page_contract, questions)# image: PIL image, path or HxWx3 array, any format Pillow opens; scaled to ~1 megapixel, no file-size cap+# needs pip install 'laya[vision]' (Pillow and torchvision), as does videoagent.predict(image="invoice.png", questions=questions)-# audio: path or bytes (wav, flac, ogg, mp3; needs soundfile) or a 1-D waveform with sampling_rate=;+# audio: path or bytes (wav, flac, ogg, mp3; needs pip install 'laya[audio]') or a 1-D waveform with sampling_rate=;# any length, resampled to 16 kHz, transcribed in 30 s windows, transcript counts toward the 73,728 tokensagent.predict(audio=wave, questions=questions, sampling_rate=16000)# video: path or bytes, any container ffmpeg opens, any length and resolution (needs the ffmpeg binary);@@ -243,6 +244,15 @@ The shipped checkpoints work zero-shot, but fine-tuning on decisions from your o**[nandhakishorm.github.io/laya](https://nandhakishorm.github.io/laya/)**: guides for [prediction hooks](https://nandhakishorm.github.io/laya/hooks/), [schema-driven decisions](https://nandhakishorm.github.io/laya/structured/), [Docker](https://nandhakishorm.github.io/laya/docker/) and [LangChain and LangGraph](https://nandhakishorm.github.io/laya/langchain/), plus a full [API reference](https://nandhakishorm.github.io/laya/reference/).+## What's new in 0.4.26++* **`laya[vision]` and `laya[audio]`.** Images and video through the vega lane need Pillow and+ torchvision, and audio files need soundfile; none of them is a core dependency, and a fresh+ install used to fail inside transformers with a message about an image processor backend. The+ two extras install exactly what each path needs, and `predict(image=...)`, `predict(video=...)`+ and `predict(audio=path)` now name the extra when it is missing. Text and a numpy waveform need+ nothing beyond `pip install laya`.+## What's new in 0.4.25* **The `Router` has a vega lane.** When vegaml is installed, `Router()` routes what the encodersdiff --git a/docs/routing.md b/docs/routing.mdindex f52b96a9..555a3ba6 100644--- a/docs/routing.md+++ b/docs/routing.md@@ -108,6 +108,9 @@ shares one forward pass on one checkpoint; send media and long states one at a tserver refuses `image`, `audio` and `video` in a body with a message saying where media goes, sincethe JSON surface has no media encoding yet.+Images and video through the lane need `pip install 'laya[vision]'` (Pillow and torchvision) and+audio files `pip install 'laya[audio]'` (soundfile); text and a numpy waveform need nothing more.+Answers from the lane carry vega's fields on top of laya's: `prediction_set`, `abstain`, `unbound`and `route`; see [vega inside laya](vegaml.md).diff --git a/docs/vegaml.md b/docs/vegaml.mdindex ff6196cd..49ba4ee0 100644--- a/docs/vegaml.md+++ b/docs/vegaml.md@@ -7,6 +7,8 @@ is `laya.VegaAgent`, in `laya/vega.py`.```bashpip install -U laya # vegaml ships with laya 0.4.23 and newer+pip install 'laya[vision]' # images and video: Pillow and torchvision+pip install 'laya[audio]' # audio files: soundfile; a numpy waveform needs nothing``````pythondiff --git a/laya/__init__.py b/laya/__init__.pyindex b795750d..853ef7ce 100644--- a/laya/__init__.py+++ b/laya/__init__.py@@ -35,7 +35,7 @@ from .presets import (from .router import DEFAULT_MODELS, RouteDecision, Routerfrom .structured import DecisionResult, decide, decide_batch-__version__ = "0.4.25"+__version__ = "0.4.26"# Routing, language detection and email cleaning are pure Python. The torch-backed names are# resolved lazily so that `import laya` -- and therefore `from laya import Router` ordiff --git a/laya/vega.py b/laya/vega.pyindex 6e13a7ea..994d13c3 100644--- a/laya/vega.py+++ b/laya/vega.py@@ -334,12 +334,22 @@ class VegaAgent(Agent):started = time.perf_counter()with self._lock:if image is not None:- raw = self.vega.decide_image(image, vq, mode=mode, alpha=self.alpha, risk=self.risk)+ try:+ raw = self.vega.decide_image(image, vq, mode=mode, alpha=self.alpha, risk=self.risk)+ except ImportError as exc:+ # the reader's image processor wants Pillow and torchvision, which core laya+ # does not carry; say which extra, not which transformers backend+ raise ImportError("images and video need Pillow and torchvision: "+ "pip install 'laya[vision]' (%s)" % exc) from excmodality = "image"elif audio is not None:- raw = self.vega.decide_audio(audio, vq, asr=asr, language=language,- sampling_rate=sampling_rate, mode=mode,- alpha=self.alpha, views=views)+ try:+ raw = self.vega.decide_audio(audio, vq, asr=asr, language=language,+ sampling_rate=sampling_rate, mode=mode,+ alpha=self.alpha, views=views)+ except ImportError as exc:+ raise ImportError("an audio file needs soundfile to decode: pip install 'laya[audio]'; "+ "a numpy waveform with sampling_rate= needs nothing extra (%s)" % exc) from excmodality = "audio"else:raw = self.vega.decide(state, vq, mode=mode, alpha=self.alpha, views=views)diff --git a/nix/package.nix b/nix/package.nixindex a94767f8..7ad1d4c8 100644--- a/nix/package.nix+++ b/nix/package.nix@@ -12,7 +12,7 @@ let# side, where tests/test_packaging.py compares all three declarations (this one,# pyproject.toml and laya.__version__) and fails CI when they diverge. Nothing# did, which is how this sat at 0.3.4 while the package reached 0.3.20.- version = "0.4.25";+ version = "0.4.26";src = ../.;format = "setuptools";propagatedBuildInputs = with python3Packages; [diff --git a/pyproject.toml b/pyproject.tomlindex 2b6ce84a..40b59024 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"[project]name = "laya"-version = "0.4.25"+version = "0.4.26"description = "Fast, non-autoregressive System 1 decision engine with calibrated probabilities"readme = "README.md"requires-python = ">=3.10"@@ -57,6 +57,14 @@ mcp = ["mcp>=2.2.0"]# `pip install laya[structured]` adds pydantic for the typed-model helpers (laya/structured.py).structured = ["pydantic>=2.0.0"]+# `pip install laya[vision]` is what `Agent(vegaml=True).predict(image=...)`, `predict(video=...)` and the+# Router's `--image`/`--video` need: the reader's image processor is built on Pillow and torchvision.+# Text and speech need neither, which is why these are an extra and not core.+vision = ["pillow>=9.0", "torchvision>=0.19"]+# `pip install laya[audio]` decodes audio FILES (wav, flac, ogg, mp3) for `predict(audio=path)`; a numpy+# waveform with `sampling_rate=` needs nothing extra.+audio = ["soundfile>=0.12"]+# `pip install laya[onnx]` adds ONNX Runtime for `laya.onnx_agent` and `onnxscript`, which# `torch.onnx.export` requires (`scripts/export_onnx.py`, `tests/test_onnx.py`).#diff --git a/tests/test_audit_scope.py b/tests/test_audit_scope.pyindex dd304472..dfb8e431 100644--- a/tests/test_audit_scope.py+++ b/tests/test_audit_scope.py@@ -86,15 +86,16 @@ if len(od) > 1:check_true("pyproject/core parses", len(core) == 6, core)check("pyproject/extra tables found", sorted(extras),- ["agno", "crewai", "fast", "langchain", "langgraph", "llamaindex", "mcp", "onnx", "serve",- "structured"])+ ["agno", "audio", "crewai", "fast", "langchain", "langgraph", "llamaindex", "mcp", "onnx", "serve",+ "structured", "vision"])declared = list(core) + [s for names in extras.values() for s in names]expected = sorted(set(declared))check("pyproject/extras add names the core does not have",sorted({re.split(r"[<>=!;\[ ]", s)[0] for s in declared}- {re.split(r"[<>=!;\[ ]", s)[0] for s in core}),["agno", "crewai", "fastapi", "langchain-core", "langgraph", "llama-index-core", "mcp", "onnx",- "onnxruntime", "onnxscript", "pydantic", "python-multipart", "tilelang", "uvicorn"])+ "onnxruntime", "onnxscript", "pillow", "pydantic", "python-multipart", "soundfile", "tilelang",+ "torchvision", "uvicorn"])# The two scopes: shipped (blocking) is the core plus [serve]; everything else is advisory.shipped = sorted(set(core) | set(extras.get("serve", [])))