Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(readme): document the CLI batch mode

Aashish0aeb5019560c
diff --git a/README.md b/README.md
index 54c20cf9..3c92a31c 100644
--- a/README.md
+++ b/README.md
@@ -199,10 +199,12 @@ laya "I was charged twice, please refund" # routing decision only; wo
laya "Refactor this service" --predict # full answers (downloads the checkpoint on first use)
laya "Mein Konto wurde zweimal belastet" --lang de # force a language instead of detecting it
laya "My payment failed twice" --preset triage # answer a ready-made preset (triage, email, guard, moderation, router)
+laya --batch tickets.txt --predict # score a file of requests, one per line, in one batch
+cat tickets.txt | laya --batch - --predict --json # stdin; one JSON line of answers per request
laya # interactive mode
```
-Routing alone never downloads a checkpoint, so it returns in milliseconds. `--predict` loads the routed checkpoint, which needs network access to the Hugging Face hub the first time; if a checkpoint cannot be downloaded, the CLI says so instead of crashing.
+Routing alone never downloads a checkpoint, so it returns in milliseconds. `--predict` loads the routed checkpoint, which needs network access to the Hugging Face hub the first time; if a checkpoint cannot be downloaded, the CLI says so instead of crashing. `--batch` (with or without `--predict`) sends the whole file through `Router.predict_batch` in one process, so the requests share checkpoint loads and forward passes — measured 2.6x on 20 tickets vs looping `predict` one by one, with `--batch-size N` to bound the forward pass and `--json` for JSONL output. Batch routing (`laya --batch FILE`, no `--predict`) likewise answers with `route_batch` in one pass, still without loading anything.
---