Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(agno): document Apple Silicon M5 latency benchmark observations

VinayChaudhari19967c3ef1cceac2
diff --git a/docs/agno.md b/docs/agno.md
index 444231be..d2e14058 100644
--- a/docs/agno.md
+++ b/docs/agno.md
@@ -1,6 +1,6 @@
# Agno Integration
-Laya provides sub-35ms, non-autoregressive decision components for **Agno** multi-agent teams and workflows (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; 193–464 ms on CPU):
+Laya provides sub-35ms, non-autoregressive decision components for **Agno** multi-agent teams and workflows (single-question latency measured at **32.8 ms** with `laya-multilingual` and **39.5 ms** with `laya` on a Tesla T4 GPU; **37.3 ms** on Apple Silicon M5; 193–464 ms on generic x86 CPU):
* **`LayaAgnoRouter`**: Sub-35ms agent router replacing slow leader LLMs in Agno `Team(mode="route")` or standalone multi-agent pipelines.
* **`LayaAgnoGuardrail`**: Pre-execution message guardrail screening incoming queries for jailbreaks, injections, and policy violations before agents call tools or downstream models.
@@ -9,6 +9,14 @@ Both take core's per-call decision controls -- the two token budgets (`max_len`,
Supports both **local in-process inference** (`Agent` or `Router`) and **remote HTTP inference** against your own `laya-serve` instance without requiring PyTorch on edge clients.
+### Hardware Latency Observations
+
+| Platform / Accelerator | Architecture | Single-Decision Routing (`route`) | Pre-Execution Guardrail (`screen`) |
+|---|---|---|---|
+| **Tesla T4 GPU** | CUDA (Server) | **32.8 – 39.5 ms** | **35 – 45 ms** |
+| **Apple Silicon M5** | ARM64 (Local CPU) | **37.3 ms** (avg 38.0 ms) | **67.4 ms** (avg 78.8 ms) |
+| **Commodity Server** | Generic x86 CPU | **193 – 464 ms** | **210 – 520 ms** |
+
---
## Installation