Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

README and docs: the input limits as comments on the lines that take them

Nandakishor5024e2f653b8
diff --git a/README.md b/README.md
index ae1a9ca3..13607b42 100644
--- a/README.md
+++ b/README.md
@@ -93,13 +93,18 @@ The rule, in order: an image or audio goes to vega; text longer than the fast mo
**Images, speech, video, long documents.** The questions and the answer shape do not change with the input:
```python
-agent.predict(image="invoice.png", questions=questions) # PIL image, path or array
-agent.predict(audio=wave, questions=questions, sampling_rate=16000) # transcribed, then decided; the transcript rides along
-agent.predict(video="call.mp4", questions=questions) # speech transcribed if there is any, else frames tiled into one image
-agent.predict_long(seventy_page_contract, questions) # up to 73,728 tokens, read whole
+# text: up to 73,728 tokens in one pass; usage["truncated"] says if a state hit the limit
+agent.predict_long(seventy_page_contract, questions)
+# image: PIL image, path or HxWx3 array, any format Pillow opens; scaled to ~1 megapixel, no file-size cap
+agent.predict(image="invoice.png", questions=questions)
+# audio: path or bytes (wav, flac, ogg, mp3; needs soundfile) or a 1-D waveform with sampling_rate=;
+# any length, resampled to 16 kHz, transcribed in 30 s windows, transcript counts toward the 73,728 tokens
+agent.predict(audio=wave, questions=questions, sampling_rate=16000)
+# video: path or bytes, any container ffmpeg opens, any length and resolution (needs the ffmpeg binary);
+# speech is transcribed and decided as text, a silent clip is decided on 4 sampled frames as one image
+agent.predict(video="call.mp4", questions=questions)
```
-An image is a PIL image, a path or an HxWx3 array in any format Pillow opens; it is scaled to about one megapixel before the reader sees it, so a sharper file gains nothing, and memory is the only size bound. Audio is a path, raw file bytes (wav, flac, ogg, mp3, with `pip install soundfile`) or a 1-D numpy waveform with `sampling_rate=`, of any length: it is resampled to 16 kHz, transcribed in 30-second windows, and the transcript counts toward the 73,728-token context. The default transcriber is English-only; pass `asr="openai/whisper-tiny", language="hi"` for other languages. Video is a path or raw bytes in any container ffmpeg opens, decoded through the `ffmpeg` binary on your PATH with no Python dependency: the sound track is transcribed and, if it holds real words, decided through the text path with heads, calibration, set and route applying; a silent clip has four evenly spaced frames tiled into one image and decided through the image path, zero-shot. Any length and resolution work, and `out["video"]` says which path decided.
**Fit and calibrate on your own labels.** Seconds, no gradient step, and the route line is placed for you:
diff --git a/docs/vegaml.md b/docs/vegaml.md
index 9ac356cf..df887c0c 100644
--- a/docs/vegaml.md
+++ b/docs/vegaml.md
@@ -79,13 +79,18 @@ than the top probability.
## Images, speech, video, long documents
```python
-agent.predict(image=img, questions=questions) # PIL image, path, or array
-agent.predict(audio=wave, questions=questions, sampling_rate=16000) # transcribed, then decided
-agent.predict(video="call.mp4", questions=questions) # speech if any, else frames
-agent.predict_long(document, questions) # one pass up to 73k tokens
+# text: up to 73,728 tokens in one pass; usage["truncated"] says if a state hit the limit
+agent.predict_long(document, questions)
+# image: PIL image, path or HxWx3 array, any format Pillow opens; scaled to ~1 megapixel, no file-size cap
+agent.predict(image=img, questions=questions)
+# audio: path or bytes (wav, flac, ogg, mp3; needs soundfile) or a 1-D waveform with sampling_rate=;
+# any length, resampled to 16 kHz, transcribed in 30 s windows, transcript counts toward the 73,728 tokens
+agent.predict(audio=wave, questions=questions, sampling_rate=16000)
+# video: path or bytes, any container ffmpeg opens, any length and resolution (needs the ffmpeg binary);
+# speech is transcribed and decided as text, a silent clip is decided on 4 sampled frames as one image
+agent.predict(video="call.mp4", questions=questions)
```
-An image is a PIL image, a path or an HxWx3 array in any format Pillow opens; it is scaled to about one megapixel before the reader sees it, so a sharper file gains nothing, and memory is the only size bound. Audio is a path, raw file bytes (wav, flac, ogg, mp3, with `pip install soundfile`) or a 1-D numpy waveform with `sampling_rate=`, of any length: it is resampled to 16 kHz, transcribed in 30-second windows, and the transcript counts toward the 73,728-token context. The default transcriber is English-only; pass `asr="openai/whisper-tiny", language="hi"` for other languages. Video is a path or raw bytes in any container ffmpeg opens, decoded through the `ffmpeg` binary on your PATH with no Python dependency: the sound track is transcribed and, if it holds real words, decided through the text path with heads, calibration, set and route applying; a silent clip has four evenly spaced frames tiled into one image and decided through the image path, zero-shot. Any length and resolution work, and `out["video"]` says which path decided.
The modality is chosen by which argument you passed; the questions and the answer shape are the
same. An audio answer carries `transcript` and `asr`, because a decision made on words you never