Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

docs(env): write down the two LAYA_* names no page carried, and sweep for the rest

Aashishf7c48f52fb76
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 7a9cca5f..9234c3e6 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -110,6 +110,7 @@ jobs:
python tests/test_head_checkpointing.py
python tests/test_packaging.py
python tests/test_doc_tables.py
+ python tests/test_env_docs.py
python tests/test_tokenizer_cache.py
python tests/test_tokenizer_concurrency.py
python tests/test_lazy_import.py
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index 8344b4ba..b30ee614 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -49,6 +49,7 @@ jobs:
python tests/test_decision_model.py
python tests/test_packaging.py
python tests/test_doc_tables.py
+ python tests/test_env_docs.py
python tests/test_lang_guess.py
python tests/test_calibration_persistence.py
python tests/test_context_manager.py
diff --git a/README.md b/README.md
index 54c20cf9..22fb6da1 100644
--- a/README.md
+++ b/README.md
@@ -633,7 +633,7 @@ else:
A threshold is a policy you choose from measured accuracy at that coverage on your data, not a property of the model. Both checkpoints are over-confident as shipped and `laya-multilingual` has no fitted temperatures at all, so fit them before relying on these numbers — see [Calibration](#calibration) above, and the [fine-tuning notebook](notebooks/laya_finetune_typed_decisions_2xT4_kaggle.ipynb) for the fitting loop itself. Then pick the point where the errors you accept are ones you can live with. Confidence orders decisions; it does not establish that a decision is correct.
-A threshold also depends on the autocast dtype. On CUDA at compute capability 8 or above the runtime uses the checkpoint's `amp_dtype`, which is bf16 for all three shipped checkpoints. On the fixed set from `benchmarks/parity_fast.py` (60 states, 288 questions per checkpoint, RTX 2000 Ada) bf16 moves a probability by up to 0.073 against the fp32 forward and flips 3 of 864 argmaxes across the three checkpoints; fp16 stays within 0.019 and flips none, at the same latency. `LAYA_CUDA_AMP=fp16` selects fp16 and `LAYA_CUDA_AMP=bf16` selects bf16 (`LAYA_CPU_AMP=bf16` is the CPU counterpart). Fit and measure a threshold in the dtype you serve with.
+A threshold also depends on the autocast dtype. On CUDA at compute capability 8 or above the runtime uses the checkpoint's `amp_dtype`, which is bf16 for all three shipped checkpoints. On the fixed set from `benchmarks/parity_fast.py` (60 states, 288 questions per checkpoint, RTX 2000 Ada) bf16 moves a probability by up to 0.073 against the fp32 forward and flips 3 of 864 argmaxes across the three checkpoints; fp16 stays within 0.019 and flips none, at the same latency. `LAYA_CUDA_AMP=fp16` selects fp16 and `LAYA_CUDA_AMP=bf16` selects bf16 (`LAYA_CPU_AMP=bf16` is the CPU counterpart). MPS autocasts in fp16 too, but the overhead dominates on a single small row, so there it engages only once a call reaches `mps_amp_min_rows` rows -- 5 by default, `LAYA_MPS_AMP_MIN_ROWS` to move it; a value that does not parse falls back to 5 and anything below 1 is clamped to 1. Fit and measure a threshold in the dtype you serve with.
---
diff --git a/docs/docker.md b/docs/docker.md
index 27487236..42ee140b 100644
--- a/docs/docker.md
+++ b/docs/docker.md
@@ -225,6 +225,7 @@ These apply to the `laya-serve` service only.
| `LAYA_MODELS` | (all) | comma list to preload: `english,multilingual,typed-decisions` |
| `LAYA_THREADS` | `OMP_NUM_THREADS` | caps torch intra-op threads; keep at or below physical cores |
| `LAYA_AUTO_TASK` | `0` | `1` lets the router reach `typed-decisions` automatically |
+| `LAYA_MAX_CONCURRENT` | `16` | requests admitted at once; later ones get `503` (a value that does not parse, or is not positive, falls back to `16`) |
| `LAYA_LOG_LEVEL` | `info` | uvicorn log level |
| `LAYA_API_KEY` | (none) | when set, requires `Authorization: Bearer <key>` |
diff --git a/tests/test_env_docs.py b/tests/test_env_docs.py
new file mode 100644
index 00000000..49a32b9d
--- /dev/null
+++ b/tests/test_env_docs.py
@@ -0,0 +1,232 @@
+"""Every ``LAYA_*`` the package reads has to be written down where a user can find it.
+
+Two names drifted out of reach of the documentation and nothing noticed:
+
+ * ``LAYA_MAX_CONCURRENT`` appears only inside ``laya/serve.py``'s module docstring.
+ That table is the developer-facing copy; no page renders it, and the server table in
+ ``docs/docker.md`` lists nine variables and not this one -- so the admission-control
+ knob behind a ``503`` is invisible to the deployment that hits it.
+ * ``LAYA_MPS_AMP_MIN_ROWS`` is named nowhere at all, one sentence away from its
+ documented ``LAYA_CUDA_AMP`` / ``LAYA_CPU_AMP`` siblings, even though it changes which
+ forwards run in fp16 on Apple Silicon and therefore which probabilities come back.
+
+A review note fixes both once. The names live in three places -- the package, the
+markdown, and the compose files -- and a list written down in any one of them goes stale
+the moment an ``os.environ`` read is added in another, so this derives the set instead of
+transcribing it: it walks ``laya/`` with ``ast`` and compares what it finds against what
+the docs mention, in both directions.
+
+Why the whole string-constant sweep rather than only ``os.environ.get("LAYA_...")``: the
+narrow form misses two of the fourteen names the package actually consumes, because
+``_env_bool("LAYA_AUTO_TASK", ...)`` and ``os.environ.get(_ENV_KEY)`` hide the literal from
+a call-shape matcher. Over-catching is the safe direction -- a name the package puts in a
+string is a name a user can set.
+
+The same ``ast`` + file-walk approach as ``tests/test_doc_tables.py``: no weights, no GPU,
+no network, deterministic.
+"""
+from __future__ import annotations
+
+import ast
+import os
+import re
+import sys
+from typing import Dict, List, Set
+
+PASS: List[str] = []
+FAIL: List[str] = []
+
+ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+PACKAGE = os.path.join(ROOT, "laya")
+NAME = re.compile(r"\bLAYA_[A-Z0-9_]+\b")
+
+# Markdown a user reads for configuration. The package's own docstrings are deliberately
+# not here: rendering them is not wired up, which is how LAYA_MAX_CONCURRENT went missing.
+DOC_FILES = ("README.md", "BENCHMARKS.md", "CONTRIBUTING.md")
+SKIP_DIRS = {".git", ".github", ".cache", ".venv", "site", "__pycache__", "node_modules",
+ "dist", "build", ".pytest_cache", ".ruff_cache"}
+
+# Documented elsewhere already, by a pull request that has not merged. Each entry has to
+# stay *undocumented* to remain valid, so the row retires itself the moment the page lands
+# and cannot become a permanent hole in the sweep.
+DEFERRED = {
+ "LAYA_SHA256_DIGESTS": "#528 (docs(security)) documents digest verification",
+}
+
+
+def check(what: str, got: object, want: object) -> None:
+ if got == want:
+ PASS.append(what)
+ else:
+ FAIL.append("%s: got %r, want %r" % (what, got, want))
+
+
+def check_true(what: str, cond: bool, detail: object = "") -> None:
+ if cond:
+ PASS.append(what)
+ else:
+ FAIL.append("%s: %s" % (what, detail))
+
+
+def read(path: str) -> str:
+ # newline="" so a CRLF checkout and an LF checkout parse identically
+ with open(path, encoding="utf-8", newline="") as fh:
+ return fh.read().replace("\r\n", "\n")
+
+
+def walk(base: str, *dirs: str):
+ """Every file under base/*, with vendored and generated trees cut out."""
+ start = os.path.join(base, *dirs)
+ if os.path.isfile(start):
+ yield start
+ return
+ for dirpath, dirnames, filenames in os.walk(start):
+ dirnames[:] = sorted(d for d in dirnames if d not in SKIP_DIRS)
+ for fn in sorted(filenames):
+ yield os.path.join(dirpath, fn)
+
+
+def package_names() -> Dict[str, Set[str]]:
+ """LAYA_* names appearing in a string constant anywhere under laya/."""
+ found: Dict[str, Set[str]] = {}
+ for path in walk(ROOT, "laya"):
+ if not path.endswith(".py"):
+ continue
+ tree = ast.parse(read(path), filename=path)
+ for node in ast.walk(tree):
+ if isinstance(node, ast.Constant) and isinstance(node.value, str):
+ for name in NAME.findall(node.value):
+ found.setdefault(name, set()).add(os.path.relpath(path, ROOT))
+ return found
+
+
+def environ_call_names() -> Set[str]:
+ """The narrow capture: names passed straight to os.environ.get/[]. Only used to show
+ that it misses part of the set, which is why package_names() is the one that gates."""
+ found: Set[str] = set()
+ for path in walk(ROOT, "laya"):
+ if not path.endswith(".py"):
+ continue
+ tree = ast.parse(read(path), filename=path)
+ for node in ast.walk(tree):
+ args: List[ast.expr] = []
+ if isinstance(node, ast.Call) and isinstance(node.func, ast.Attribute) \
+ and isinstance(node.func.value, ast.Attribute) \
+ and node.func.value.attr == "environ" and node.args:
+ args = [node.args[0]]
+ elif isinstance(node, ast.Subscript) and isinstance(node.value, ast.Attribute) \
+ and node.value.attr == "environ":
+ args = [node.slice]
+ for arg in args:
+ if isinstance(arg, ast.Constant) and isinstance(arg.value, str):
+ found.update(NAME.findall(arg.value))
+ return found
+
+
+def markdown_names() -> Dict[str, Set[str]]:
+ found: Dict[str, Set[str]] = {}
+ paths = [os.path.join(ROOT, f) for f in DOC_FILES if os.path.isfile(os.path.join(ROOT, f))]
+ paths += [p for p in walk(ROOT, "docs") if p.endswith(".md")]
+ for path in paths:
+ for name in NAME.findall(read(path)):
+ found.setdefault(name, set()).add(os.path.relpath(path, ROOT))
+ return found
+
+
+def non_markdown_names() -> Dict[str, Set[str]]:
+ """Everywhere a variable can be consumed outside prose: Python, compose, Dockerfile,
+ nix, shell. A name that appears only in markdown is documented but wired to nothing.
+ `tests/` is excluded on purpose -- a test that greps the docs cannot be the reason a
+ variable exists.
+ """
+ found: Dict[str, Set[str]] = {}
+ for dirpath, dirnames, filenames in os.walk(ROOT):
+ dirnames[:] = sorted(d for d in dirnames if d not in SKIP_DIRS and d != "tests")
+ for fn in sorted(filenames):
+ path = os.path.join(dirpath, fn)
+ if fn.endswith((".md", ".pyc", ".json", ".safetensors", ".bin", ".png", ".ico")):
+ continue
+ try:
+ text = read(path)
+ except (UnicodeDecodeError, OSError):
+ continue
+ for name in NAME.findall(text):
+ found.setdefault(name, set()).add(os.path.relpath(path, ROOT))
+ return found
+
+
+def main() -> int:
+ pkg = package_names()
+ docs = markdown_names()
+ tree = non_markdown_names()
+
+ # ------------------------------------------------- the capture itself has to be working
+ # If the walk or the parser silently found nothing, every check below would pass vacuously.
+ witnesses = {"LAYA_DEVICE", "LAYA_CUDA_AMP", "LAYA_CPU_AMP", "LAYA_MAX_CONCURRENT",
+ "LAYA_MPS_AMP_MIN_ROWS", "LAYA_SHA256_DIGESTS", "LAYA_API_KEY"}
+ check_true("sweep/reads the names the package is known to consume",
+ witnesses <= set(pkg), sorted(witnesses - set(pkg)))
+ check_true("sweep/found a plausible number of names", len(pkg) >= len(witnesses), len(pkg))
+ # LAYA_AUTO_TASK goes through _env_bool() and LAYA_DEVICE through a module constant in
+ # laya/mcp/device.py, so the call-shaped matcher is not enough on its own.
+ missed = set(pkg) - environ_call_names()
+ check_true("sweep/the narrow os.environ matcher misses names a string sweep finds",
+ "LAYA_AUTO_TASK" in missed, sorted(missed))
+
+ # ------------------------------------------------- direction 1: consumed -> documented
+ missing = {n: v for n, v in pkg.items() if n not in docs}
+ retired = {n: v for n, v in missing.items() if n in DEFERRED}
+ unexplained = {n: sorted(v) for n, v in missing.items() if n not in DEFERRED}
+ check_true("every LAYA_* the package reads is documented", not unexplained, unexplained)
+ check("the only undocumented names are the deferred ones", sorted(retired), sorted(DEFERRED))
+
+ # ------------------------------------------------- direction 2: documented -> consumed
+ # A configuration page describing a variable nothing reads is worse than a gap: the
+ # reader sets it and the program ignores it. The consumer may be the package, a compose
+ # file's interpolation, the Dockerfile, or the Nix module.
+ docs_only = {}
+ for name, where in docs.items():
+ consumers = tree.get(name, set())
+ if not consumers:
+ docs_only[name] = sorted(where)
+ check_true("every documented LAYA_* is consumed somewhere in the tree",
+ not docs_only, docs_only)
+
+ # the two rows this suite was written for, pinned to the page that carries them, so a
+ # later edit that deletes one fails by name rather than only as a set difference
+ anchors = {"LAYA_MAX_CONCURRENT": "docs/docker.md",
+ "LAYA_MPS_AMP_MIN_ROWS": "README.md"}
+ for name, page in sorted(anchors.items()):
+ check_true("%s is documented on %s" % (name, page),
+ page in docs.get(name, set()), sorted(docs.get(name, set())))
+
+ # ------------------------------------------------- what this cannot check
+ unbacked = [
+ ("a documented default matches the code default",
+ "docs/docker.md documents the Compose service's default, which is deliberately not "
+ "the package default (LAYA_PRELOAD is 0 there and 1 in laya/serve.py)"),
+ ("a name only mentioned in a comment",
+ "comments are not string constants, so an undocumented read in a comment is invisible "
+ "here; the same is true of a name built by concatenation"),
+ ("the values a variable accepts",
+ "LAYA_CUDA_AMP=fp32 is silently ignored rather than rejected; checking the accepted "
+ "set against prose is a different sweep"),
+ ]
+ check_true("uncheckable claims are reported, not skipped", len(unbacked) == 3, unbacked)
+
+ print("\n%d passed, %d failed" % (len(PASS), len(FAIL)))
+ for f in FAIL:
+ print(" FAIL " + f)
+ if not FAIL:
+ print("all %d LAYA_* names the package reads appear in the docs" % len(pkg))
+ print("all %d documented names are consumed somewhere in the tree" % len(docs))
+ for name, why in sorted(DEFERRED.items()):
+ print("deferred: %s -- %s; delete the DEFERRED row once it is documented" % (name, why))
+ print("not checkable from this repository (not asserted either way):")
+ for name, why in unbacked:
+ print(" - %s: %s" % (name, why))
+ return 1 if FAIL else 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())