Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.
Python calibrationclassificationdecision-modelhuggingfacejevmodernbertmultilingualnlppythonpytorchroutingtyped-decisionszero-shot
docs(agno): document Apple Silicon M5 latency benchmark observations
VinayChaudhari19967c3ef1cceac2
diff --git a/docs/agno.md b/docs/agno.mdindex 444231be..d2e14058 100644--- a/docs/agno.md+++ b/docs/agno.md@@ -1,6 +1,6 @@# Agno Integration-Laya provides sub-35ms, non-autoregressive decision components for **Agno** multi-agent teams and workflows (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; 193–464 ms on CPU):+Laya provides sub-35ms, non-autoregressive decision components for **Agno** multi-agent teams and workflows (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; **37.3 ms** on Apple Silicon M5; 193–464 ms on generic x86 CPU):* **`LayaAgnoRouter`**: Sub-35ms agent router replacing slow leader LLMs in Agno `Team(mode="route")` or standalone multi-agent pipelines.* **`LayaAgnoGuardrail`**: Pre-execution message guardrail screening incoming queries for jailbreaks, injections, and policy violations before agents call tools or downstream models.@@ -9,6 +9,14 @@ Both take core's per-call decision controls -- the two token budgets (`max_len`,Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.+### Hardware Latency Observations++| Platform / Accelerator | Architecture | Single-Decision Routing (`route`) | Pre-Execution Guardrail (`screen`) |+|---|---|---|---|+| **Tesla T4 GPU** | CUDA (Server) | **32.8 – 39.5 ms** | **35 – 45 ms** |+| **Apple Silicon M5** | ARM64 (Local CPU) | **37.3 ms** (avg 38.0 ms) | **67.4 ms** (avg 78.8 ms) |+| **Commodity Server** | Generic x86 CPU | **193 – 464 ms** | **210 – 520 ms** |+---## Installation