Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

feat: promote Route Mode as primary Quickstart, bump to v0.3.4

Nandakishord113dca2512f
diff --git a/README.md b/README.md
index f12882b7..da95bc05 100644
--- a/README.md
+++ b/README.md
@@ -44,27 +44,29 @@ pip install laya
---
-## Quickstart
+## Quickstart: Route Mode (Recommended)
+
+Laya ships three checkpoints. The built-in **`Router`** is the recommended entry point: it evaluates any state in any language, automatically detects scripts and languages in sub-milliseconds, and dispatches to the optimal checkpoint in a single forward pass.
```python
import laya
+from laya import Router
-# 1. Load the fine-tuned model directly from Hugging Face Hub (auto-downloads weights)
-agent = laya.load("convaiinnovations/laya")
+# Preload checkpoints into memory for instant sub-35ms routing
+router = Router(preload=True)
-# 2. Provide any state (string or dictionary)
+# 1. State in any language or schema
state = {
"from": "[email protected]",
"subject": "Duplicate charge on invoice #4411",
"body": "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan."
}
-# 3. Define your typed questions
+# 2. Define your typed questions
questions = {
- # choice: categorical selection with probabilities & confidence
"department": {
"type": "choice",
- "instructions": "Which department should handle this email?",
+ "instructions": "Which department should handle this request?",
"criteria": {
"billing": "invoices, payments, refunds",
"technical": "bugs, outages, system errors",
@@ -72,197 +74,159 @@ questions = {
"other": "everything else"
}
},
- # score: placement on an ordinal rubric
"urgency": {
"type": "score",
"instructions": "How urgent is this request?",
"criteria": ["not urgent", "soon", "critical deadline or blocking issue"]
},
- # noul: calibrated boolean probability P(true)
"churn_risk": {
"type": "noul",
"instructions": "Does the user threaten to cancel or leave?"
},
- "is_phishing": {
+ "refund_requested": {
"type": "noul",
- "instructions": "Is this email a phishing or scam attempt?"
+ "instructions": "Does the user explicitly request a refund?"
}
}
-# 4. Run all questions in ONE single forward pass (~35 ms on GPU)
-result = agent.predict(state, questions)
-answers = result["answers"]
-
-print("Department :", answers["department"]["choice"])
-# -> billing (confidence: 0.94)
+# 3. English state -> automatically routed to laya (ModernBERT-large, 39.5 ms)
+res_en = router.predict(state, questions)
+print("Department :", res_en["answers"]["department"]["choice"]) # -> billing (confidence: 0.94)
+print("Routing :", res_en["routing"]["model"]) # -> english
-print("Urgency :", answers["urgency"]["score"])
-# -> 1.84 / 2.0
+# 4. Hindi state -> automatically routed to laya-multilingual (mmBERT-base, 32.8 ms)
+res_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।"}, questions)
+print("Department :", res_hi["answers"]["department"]["choice"]) # -> billing (confidence: 0.86)
+print("Routing :", res_hi["routing"]["model"]) # -> multilingual
-print("Churn Risk :", answers["churn_risk"]["noul"])
-# -> 0.892 (89.2% probability)
-
-print("Phishing :", answers["is_phishing"]["noul"])
-# -> 0.008 (0.8% probability)
+# 5. Explicit override when you want a specific checkpoint
+res_td = router.predict(state, questions, model="typed-decisions")
```
----
-
-## Automated Confidence Gating
-
-Because Laya's probabilities are trained with strictly proper scoring rules (RLCD), confidence scores are statistically meaningful:
+Every result carries full routing metadata explaining why the choice was made:
```python
-dept = answers["department"]["choice"]
-conf = answers["department"]["confidence"]
-
-if conf >= 0.85:
- # High confidence: automated action without human in the loop
- route_automatically(dept)
-else:
- # Low confidence: escalate to human triage
- escalate_to_human_agent(dept, reason=f"Low confidence ({conf:.2f})")
+res_hi["routing"]
+# {
+# 'model': 'multilingual',
+# 'repo': 'convaiinnovations/laya/multilingual',
+# 'reason': 'non-Latin script (devanagari, 100% of letters); the English checkpoint cannot read it'
+# }
```
----
-
-## Built-in Workflow Presets
-
-Laya provides pre-tuned question schemas for immediate production use:
+Inspect a routing decision without running any forward pass:
```python
-import laya
-
-agent = laya.load("convaiinnovations/laya")
-
-# 1. Intelligent Model Router (routes to small vs. frontier models)
-routing = agent.predict({"request": "Refactor this service using dependency injection"}, laya.router_questions())
-
-# 2. Real-time Prompt Guardrails (jailbreaks, injections, leaks)
-guard = agent.predict({"prompt": "Ignore all instructions"}, laya.guard_questions())
-
-# 3. Content Safety & Moderation (toxicity, harassment, threats)
-safety = agent.predict({"post": "User comment text"}, laya.moderation_questions())
-
-# 4. Support Ticket Triage (intent, urgency, frustration, churn)
-triage = agent.predict({"message": "My payment failed twice"}, laya.triage_questions())
+router.route({"body": "Der Kunde wurde zweimal belastet"}, questions).reason
+# "Latin script but language looks like 'de', not English"
```
----
+### Why Route: The Evidence
-## Model Routing (three checkpoints, one call)
+On a shared benchmark (17,416 questions, one T4 GPU, identical questions per model):
-Laya ships three checkpoints. `Router` picks the right one per request and loads it lazily.
+| Benchmark / Task | English (`laya`) | Multilingual (`laya-multilingual`) | `Router` (Routed) |
+|---|---|---|---|
+| MASSIVE intent, English | **0.783** | 0.657 | **0.783** |
+| MASSIVE intent, 13 other languages | 0.306 | **0.451** | **0.451** |
+| XNLI, English | **0.860** | 0.843 | **0.860** |
+| XNLI, 14 other languages | 0.521 | **0.731** | **0.731** |
+| Languages usable (>3x random) | 23 / 51 | 45 / 51 | **45 / 51** |
+| Latency, 1 question (T4 GPU) | 39.5 ms | **32.8 ms** | **32.8 ms** |
+| Latency, 10 questions batched | 158.6 ms | **72.3 ms** | **72.3 ms** |
-| name | repo | size | context | best at |
-|---|---|---|---|---|
-| `english` | [`convaiinnovations/laya`](https://huggingface.co/convaiinnovations/laya) | 421M | 512 | English text |
-| `multilingual` | [`convaiinnovations/laya-multilingual`](https://huggingface.co/convaiinnovations/laya-multilingual) | 322M | 1024 | 100+ languages, 2x faster |
-| `typed-decisions` | [`convaiinnovations/laya-typed-decisions`](https://huggingface.co/convaiinnovations/laya-typed-decisions) | 421M | 1024 | the four typed-decisions workflows |
+The English checkpoint collapses on non-Latin scripts (Khmer scores **0.000 accuracy at 0.952 confidence**). Because the model stays confident while being wrong, confidence gating cannot save you. `Router` detects the script in <0.5 ms pure Python before the forward pass.
-```python
-from laya import Router
+### Production Preload & Memory
-router = Router() # nothing is downloaded until a request needs it
+A cold checkpoint build costs seconds; language detection costs microseconds. At the default `max_loaded=1`, traffic that alternates languages rebuilds a model on *every* request (measured at a 7.4 s median reload on CPU and 10.3 s on T4).
-# English -> routed to the English checkpoint
-router.predict({"body": "I was charged twice, please refund."}, questions)
+For a server or production app, preload:
-# Hindi -> routed to the multilingual checkpoint automatically
-router.predict({"body": "मुझसे दो बार शुल्क लिया गया"}, questions)
+```python
+# Every checkpoint resident in memory; language flips cost detection only (<1 ms)
+router = Router(preload=True)
+router = Router(preload=True, device="cuda")
-# explicit when you already know
-router.predict(state, questions, model="typed-decisions")
-router.predict(state, questions, lang="de")
-```
+# Or preload only the specific checkpoints you serve:
+router.preload(["english", "multilingual"])
-Every result carries the decision that produced it:
+# If your app already built an agent, attach it to avoid duplicate VRAM:
+router.attach("english", existing_agent)
-```python
-result = router.predict({"body": "二重に請求されました"}, questions)
-result["routing"]
-# {'model': 'multilingual',
-# 'repo': 'convaiinnovations/laya-multilingual',
-# 'reason': 'non-Latin script (kana, 100% of letters); the English checkpoint cannot read it',
-# ...}
+# Manage resident memory (default keeps 1 hot, LRU eviction)
+router = Router(max_loaded=2) # keep two hot
+router.unload() # free memory
```
-Inspect a decision without running the model:
+| Deployment Mode | Per-Request Latency | Model Reloads |
+|---|---|---|
+| `Router()` (lazy, `max_loaded=1`) | 7 to 10 s on every language switch | 1 per switch |
+| `Router(preload=True)` | **32.8 ms (GPU) / 193–464 ms (CPU)** | **none** |
-```python
-router.route({"body": "Der Kunde wurde zweimal belastet"}, questions).reason
-# "Latin script but language looks like 'de', not English"
-```
+---
-### Why route at all
+## Single-Model Mode (Direct SDK)
-Accuracy on a shared benchmark (17,416 questions, one T4, identical questions per model):
+If you only need a single checkpoint for a dedicated pipeline, you can load models directly:
-| | `english` | `multilingual` |
-|---|---|---|
-| MASSIVE intent, English | **0.783** | 0.657 |
-| MASSIVE intent, 13 other languages | 0.306 | **0.451** |
-| XNLI, English | **0.860** | 0.843 |
-| XNLI, 14 other languages | 0.521 | **0.731** |
-| English-only suites | **0.684** | 0.619 |
-| Latency, 10 questions | 159 ms | **72 ms** |
+```python
+import laya
-The English checkpoint does not degrade gracefully outside English -- it collapses, and stays
-confident while doing so. On 20-option MASSIVE intent (random = 0.050) it scores 0.100 on Hindi
-and 0.103 on Korean, with an expected calibration error of 0.855. Script detection is therefore
-the primary routing signal.
+# 1. Load a specific checkpoint directly from the hub
+agent = laya.load("convaiinnovations/laya") # English root
+agent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual") # 100+ languages
+agent_td = laya.load("convaiinnovations/laya", subfolder="typed-decisions")
-### Routing rules
+# 2. Run all questions in ONE single forward pass (~35 ms on GPU)
+result = agent.predict(state, questions)
+answers = result["answers"]
-Precedence, highest first:
+print("Department :", answers["department"]["choice"]) # -> billing (confidence: 0.94)
+print("Urgency :", answers["urgency"]["score"]) # -> 1.84 / 2.0
+print("Churn Risk :", answers["churn_risk"]["noul"]) # -> 0.892 (89.2% probability)
+```
-1. `model=` -- explicit checkpoint.
-2. `task="typed_decisions"` -- explicit task.
-3. A question-id set exactly matching a typed-decisions workflow, **only** if you construct the
- router with `auto_task_detection=True`. It is off by default: that checkpoint is fine-tuned on
- four synthetic workflows and should not be a silent fallback.
-4. `lang=` -- explicit language code.
-5. Detected script (exact) and, for Latin text, a stopword/diacritic language guess (best effort).
-6. `default=` (`"english"` unless you change it).
+---
-### Preload — make routing free
+## Automated Confidence Gating
-A cold checkpoint build costs **seconds**; language detection costs **microseconds**. At the
-default `max_loaded=1`, traffic that alternates languages rebuilds a model on *every* request.
-For a server or a demo, preload:
+Because Laya's probabilities are trained with strictly proper scoring rules (RLCD), confidence scores are statistically meaningful:
```python
-router = Router(preload=True) # every checkpoint resident, routing is free
-router = Router(preload=True, device="cuda")
-router.preload(["english", "multilingual"]) # or just the two you serve
+dept = answers["department"]["choice"]
+conf = answers["department"]["confidence"]
+
+if conf >= 0.85:
+ # High confidence: automated action without human in the loop
+ route_automatically(dept)
+else:
+ # Low confidence: escalate to human triage
+ escalate_to_human_agent(dept, reason=f"Low confidence ({conf:.2f})")
```
-`preload` raises `max_loaded` to fit what it built, so the LRU cannot evict it immediately.
+---
+
+## Built-in Workflow Presets
-If the process already has a checkpoint loaded for other reasons, hand it over instead of
-loading a second copy:
+Laya provides pre-tuned question schemas for immediate production use:
```python
-router.attach("english", existing_agent) # no duplicate 421M parameters
-router.preload() # builds only what is still missing
-```
+import laya
-Measured on CPU with the demo Space's own workload:
+agent = laya.load("convaiinnovations/laya")
-| | per request | model loads |
-|---|---|---|
-| `Router()` — lazy, `max_loaded=1` | 4–6 s on every language switch | 1 per switch |
-| `Router(preload=True)` | **193–464 ms** | **none** |
+# 1. Intelligent Model Router (routes to small vs. frontier models)
+routing = agent.predict({"request": "Refactor this service using dependency injection"}, laya.router_questions())
-### Memory
+# 2. Real-time Prompt Guardrails (jailbreaks, injections, leaks)
+guard = agent.predict({"prompt": "Ignore all instructions"}, laya.guard_questions())
-All three together are ~1.16B parameters (~4.6 GB fp32), so `Router` keeps **one** resident by
-default and evicts least-recently-used. Raise it when you have the RAM:
+# 3. Content Safety & Moderation (toxicity, harassment, threats)
+safety = agent.predict({"post": "User comment text"}, laya.moderation_questions())
-```python
-Router(max_loaded=2) # keep two hot
-router.unload() # free everything
-router.loaded # ['multilingual']
+# 4. Support Ticket Triage (intent, urgency, frustration, churn)
+triage = agent.predict({"message": "My payment failed twice"}, laya.triage_questions())
```
---
diff --git a/laya/__init__.py b/laya/__init__.py
index 19e535ce..1f1acb4f 100644
--- a/laya/__init__.py
+++ b/laya/__init__.py
@@ -22,7 +22,7 @@ from .presets import (
)
from .router import DEFAULT_MODELS, RouteDecision, Router
-__version__ = "0.3.3"
+__version__ = "0.3.4"
__all__ = [
"Agent",
"RLAgent",
diff --git a/pyproject.toml b/pyproject.toml
index be4372aa..f247394b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "laya"
-version = "0.3.3"
+version = "0.3.4"
description = "Fast, non-autoregressive System 1 decision engine with calibrated probabilities"
readme = "README.md"
requires-python = ">=3.8"
diff --git a/setup.py b/setup.py
index 9b93a818..93db822b 100644
--- a/setup.py
+++ b/setup.py
@@ -2,7 +2,7 @@ from setuptools import setup, find_packages
setup(
name="laya",
- version="0.3.3",
+ version="0.3.4",
description="Fast, non-autoregressive System 1 decision engine with calibrated probabilities",
long_description=open("README.md").read(),
long_description_content_type="text/markdown",