Skip to content

armgit / laya

Public Imported from GitHub
0

Non-autoregressive System 1 decision engine. Typed choice, score and yes/no decisions over any text in a single forward pass, in 100+ languages, with a router that picks the right checkpoint per request.

ci(java): what the review of the new lanes found

Adeshina Lasisi2266acff76dc
diff --git a/.github/workflows/java.yml b/.github/workflows/java.yml
index 8dc5dc76..e8a15095 100644
--- a/.github/workflows/java.yml
+++ b/.github/workflows/java.yml
@@ -76,7 +76,7 @@ jobs:
if: ${{ !cancelled() }}
run: >
python laya-java/scripts/check_test_results.py
- laya-java/laya-java/build/test-results/test --min-tests 440 --allow-aborted
+ laya-java/laya-java/build/test-results/test --min-tests 480 --allow-aborted
# LAST, deliberately. This gate and the tests are independent signals -- the tests compare
# against the COMMITTED fixture, this compares the committed fixture against a fresh
@@ -147,11 +147,19 @@ jobs:
# them under the hosted tool cache, which Gradle's own directory scan does not look in, so
# `fromEnv` names the variables it exports.
#
- # What makes a misconfiguration LOUD is the pair below it: `auto-download=false` plus
- # `launcherFor`, which fails the build outright -- "Cannot find a Java installation on your
- # machine matching: {languageVersion=N}" -- rather than quietly running on the compile JDK
- # and reporting a pass for the wrong JVM. `TestJvmVersionTest` then asserts the same thing
- # from inside the suite, because the JUnit XML cannot record it.
+ # These are `-P`, not `-D`, and that is not a style choice: Gradle ignores
+ # `org.gradle.java.installations.*` given as `-D`. Measured on this build --
+ # `auto-detect=false` as `-D` left all 8 locally detected toolchains in the
+ # `javaToolchains` report; as `-P` it dropped to 3. With `-D` the lane still passed, but
+ # only because setup-java makes the LAST java-version the JAVA_HOME, so Gradle's
+ # "current JVM" supplier happened to provide the right launcher -- a coincidence, and the
+ # exact fragility naming the variables was meant to remove.
+ #
+ # With `-P` the three together make a misconfiguration loud: `auto-detect=false` means the
+ # only installations are the two named, and `launcherFor` with `auto-download=false` then
+ # fails the build outright -- "Cannot find a Java installation on your machine matching:
+ # {languageVersion=N}" -- instead of quietly running on the compile JDK. Verified both
+ # ways: without `fromEnv` the build exits 1 with that message; with it, 0.
- uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
with:
distribution: temurin
@@ -163,19 +171,28 @@ jobs:
working-directory: laya-java
run: >
./gradlew --no-daemon
- -Dorg.gradle.java.installations.fromEnv=JAVA_HOME_17_${{ runner.arch }},JAVA_HOME_${{ matrix.jdk }}_${{ runner.arch }}
- -Dorg.gradle.java.installations.auto-download=false
+ -Porg.gradle.java.installations.fromEnv=JAVA_HOME_17_${{ runner.arch }},JAVA_HOME_${{ matrix.jdk }}_${{ runner.arch }}
+ -Porg.gradle.java.installations.auto-detect=false
+ -Porg.gradle.java.installations.auto-download=false
-PtestJavaVersion=${{ matrix.jdk }}
test
# That the tests ran on the RIGHT JVM is asserted inside the suite, by
# `TestJvmVersionTest` against the `laya.test.expectedJavaVersion` the build sets from
# `-PtestJavaVersion`. It cannot be checked from here: Gradle writes `<properties/>` empty
# into the JUnit XML, so the report cannot tell 17 from 24.
- - name: Model-free suite was not vacuous
+ #
+ # `--require-passed` is what stops that assertion being optional. The test is
+ # `@EnabledIfSystemProperty`, so if `-PtestJavaVersion` ever stops reaching the build it is
+ # SKIPPED rather than failed -- and `--allow-aborted`, which this lane needs because the
+ # checkpoint-backed factories abort here, would forgive it. Measured: with the flag
+ # dropped, Gradle exits 0 and the floor alone exits 0 on 488 tests, having run JDK 17 twice
+ # and reported it as two JDKs. With `--require-passed` it exits 1.
+ - name: Model-free suite was not vacuous, and ran on the right JVM
if: ${{ !cancelled() }}
run: >
python laya-java/scripts/check_test_results.py
- laya-java/laya-java/build/test-results/test --min-tests 440 --allow-aborted
+ laya-java/laya-java/build/test-results/test --min-tests 480 --allow-aborted
+ --require-passed TestJvmVersionTest
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ failure() }}
with:
@@ -244,19 +261,33 @@ jobs:
pip install "torch==2.14.0" --index-url https://download.pytorch.org/whl/cpu
pip install -r laya-dotnet/tools/requirements-regen.txt
- # The export is the slow part (download, trace, verify) and depends only on the pinned
- # revision, the exporter and the model definition, so that is the cache key.
- - name: Cache pinned Hugging Face downloads
- uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
- with:
- path: ~/.cache/huggingface/hub
- key: laya-java-hf-${{ runner.os }}-${{ hashFiles('laya-java/scripts/prepare_checkpoint.py') }}
+ # The export is the slow part (download, trace, verify) and depends on the PINNED REVISION
+ # as well as the exporter and the model definition, so `prepare_checkpoint.py` -- which is
+ # where HF_REVISION lives -- is part of the key. It was not, and that was not cosmetic:
+ # `prepare_checkpoint.py` short-circuits on an existing graph, so a revision bump changed
+ # nothing in the old key, the graph traced from the PREVIOUS revision was restored and
+ # reused, and `predict.json` was then recorded from that same stale graph. Both sides of
+ # the comparison agreed, nine tests passed, and the cell was green having never measured
+ # the pinned checkpoint -- with the drift step showing `predict.json` unchanged, which
+ # reads as reassurance. The script now also stamps the revision beside the graph and
+ # re-exports when it differs, so this key is defence in depth rather than the only guard.
+ #
+ # There is NO Hugging Face download cache, deliberately. There was one, on
+ # `~/.cache/huggingface/hub`, and it never held anything: `prepare_checkpoint.py` passes
+ # `local_dir` to `snapshot_download`, which writes the files there and not into the hub
+ # cache. Measured against the pinned huggingface_hub -- with a `revision` pinned, as here,
+ # the hub cache is not even created; without one it gets a `refs/main` pointer and zero
+ # blobs. So the step always missed on restore, saved nothing, and read like a download
+ # cost that was being avoided when it was being paid in full every run. Making it real
+ # means giving `snapshot_download` a `cache_dir`, or caching `.work/checkpoints` (which
+ # the revision stamp now makes safe) and accepting a multi-gigabyte entry for the
+ # weight-bearing cell; neither is this change's call to make.
- name: Cache the exported graph
if: ${{ matrix.graph }}
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: laya-java/.work/onnx
- key: laya-java-onnx-${{ runner.os }}-${{ hashFiles('scripts/export_onnx.py', 'laya/common.py', 'laya-dotnet/tools/requirements-regen.txt') }}
+ key: laya-java-onnx-${{ runner.os }}-${{ matrix.name }}-${{ hashFiles('laya-java/scripts/prepare_checkpoint.py', 'scripts/export_onnx.py', 'laya/common.py', 'laya-dotnet/tools/requirements-regen.txt') }}
- name: Prepare the checkpoints this cell owns
run: |
@@ -306,7 +337,7 @@ jobs:
if: ${{ !cancelled() }}
run: >
python laya-java/scripts/check_test_results.py
- laya-java/laya-java/build/test-results/test --min-tests ${{ matrix['min-tests'] }}
+ laya-java/laya-java/build/test-results/test --min-tests ${{ matrix.min-tests }}
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
if: ${{ failure() }}
diff --git a/laya-java/scripts/check_test_results.py b/laya-java/scripts/check_test_results.py
index 9fd8dce5..c3fa254b 100755
--- a/laya-java/scripts/check_test_results.py
+++ b/laya-java/scripts/check_test_results.py
@@ -34,6 +34,10 @@ def main(argv=None):
help="fail if fewer than this many tests ran")
parser.add_argument("--allow-aborted", action="store_true",
help="permit skipped/aborted tests, for a lane with no checkpoints")
+ parser.add_argument("--require-passed", action="append", default=[], metavar="CLASS.TEST",
+ help="a substring that must match at least one test that actually PASSED; "
+ "repeatable. For a guard that is conditionally disabled, where being "
+ "skipped would otherwise satisfy --allow-aborted")
args = parser.parse_args(argv)
if os.path.isdir(args.results):
@@ -53,7 +57,7 @@ def main(argv=None):
return 1
tests = failures = errors = skipped = 0
- broken, aborted = [], []
+ broken, aborted, passed = [], [], []
for path in files:
try:
root = ElementTree.parse(path).getroot()
@@ -73,6 +77,8 @@ def main(argv=None):
elif case.find("skipped") is not None:
reason = case.find("skipped").get("message") or ""
aborted.append("%s (%s)" % (name, reason.strip()[:120]))
+ else:
+ passed.append(name)
print("check_test_results: %d tests, %d failed, %d errored, %d skipped, across %d file(s)"
% (tests, failures, errors, skipped, len(files)))
@@ -100,6 +106,23 @@ def main(argv=None):
problems.append("only %d of %d tests actually ran (%d skipped), expected at least %d"
% (executed, tests, skipped, args.min_tests))
+ # A floor cannot protect a guard that is allowed to be absent. `--allow-aborted` exists so a
+ # lane without the checkpoints can still gate on its model-free tests, but it also forgives a
+ # CONDITIONALLY disabled test -- and a test disabled because the thing it checks was never
+ # switched on is indistinguishable, to this script, from one that passed. The JDK matrix is
+ # exactly that case: if `-PtestJavaVersion` stops reaching the build, no launcher override
+ # happens, the one test that would notice is skipped rather than failed, and both cells go
+ # green having run the compile JDK twice. Naming it here turns "silently absent" into a
+ # failure.
+ for wanted in args.require_passed:
+ if not any(wanted in name for name in passed):
+ where = ("it was skipped" if any(wanted in name for name in aborted)
+ else "it failed" if any(wanted in name for name in broken)
+ else "it did not run at all")
+ problems.append("no PASSING test matched %r -- %s. This lane requires that test to "
+ "run, because it is the check that the lane is not vacuous."
+ % (wanted, where))
+
if problems:
for line in problems:
print("check_test_results: " + line, file=sys.stderr)
diff --git a/laya-java/scripts/prepare_checkpoint.py b/laya-java/scripts/prepare_checkpoint.py
index 0a384e5e..c558f571 100755
--- a/laya-java/scripts/prepare_checkpoint.py
+++ b/laya-java/scripts/prepare_checkpoint.py
@@ -36,6 +36,41 @@ WANTED_LIGHT = ("rl_agent_config.json", "config.json", "tokenizer.json", "tokeni
WANTED = WANTED_LIGHT + ("model.safetensors", "encoder/*")
+STAMP = ".laya-revision"
+
+
+def stamped(path, revision, wanted_weights=None):
+ """Whether `path` was produced by THIS revision (and, if asked, with weights).
+
+ Everything cacheable here is keyed on the pinned revision, and nothing on disk otherwise
+ records which revision produced it. Without that, "the graph is already exported" and "the
+ checkpoint is already downloaded" are claims about a PATH rather than about a revision, so a
+ restored cache from before a `HF_REVISION` bump is reused and the whole lane measures the old
+ model while reporting on the new one. Both sides of the comparison come from the same stale
+ artifact, so they agree and the cell goes green having tested nothing it claims to.
+ """
+ marker = os.path.join(path, STAMP) if os.path.isdir(path) else path + STAMP
+ try:
+ with open(marker, "r", encoding="utf-8") as handle:
+ recorded = handle.read().strip().split()
+ except OSError:
+ return False
+ if not recorded or recorded[0] != revision:
+ return False
+ # A weightless checkpoint cannot be traced, so it must not satisfy a request that needs one.
+ if wanted_weights is not None:
+ return (len(recorded) > 1 and recorded[1] == "weights") == bool(wanted_weights)
+ return True
+
+
+def stamp(path, revision, with_weights=None):
+ """Record the revision `path` was produced from, once it is complete."""
+ marker = os.path.join(path, STAMP) if os.path.isdir(path) else path + STAMP
+ text = revision + ("" if with_weights is None else (" weights" if with_weights else " light"))
+ with open(marker, "w", encoding="utf-8") as handle:
+ handle.write(text + "\n")
+
+
def fetch(name, revision, destination, with_weights=True):
"""Materialise exactly the wanted files of one checkpoint at `destination`.
@@ -83,6 +118,7 @@ def fetch(name, revision, destination, with_weights=True):
os.replace(source, destination)
if os.path.exists(staging):
shutil.rmtree(staging)
+ stamp(destination, revision, with_weights)
return destination
@@ -91,7 +127,8 @@ def main(argv=None):
parser.add_argument("--checkpoint", required=True, help="english, multilingual, ...")
parser.add_argument("--revision", default=HF_REVISION)
parser.add_argument("--force", action="store_true",
- help="re-export even when the graph is already present")
+ help="re-download and re-export even when a stamp says this revision's "
+ "checkpoint and graph are already here")
parser.add_argument("--no-graph", action="store_true",
help="fetch only the tokenizer and config, and do not export a graph: "
"enough for the tokenizer and sequence fixtures, which cover both "
@@ -103,10 +140,15 @@ def main(argv=None):
graph_dir = os.path.join(work, "onnx", args.checkpoint)
graph = os.path.join(graph_dir, "laya.onnx")
- print("downloading %s at %s%s" % (args.checkpoint, args.revision[:12],
- " (tokenizer and config only)" if args.no_graph else ""),
- flush=True)
- fetch(args.checkpoint, args.revision, model_dir, with_weights=not args.no_graph)
+ wants_weights = not args.no_graph
+ if not args.force and stamped(model_dir, args.revision, wanted_weights=wants_weights):
+ print(" checkpoint already at %s for %s, not re-downloading"
+ % (os.path.relpath(model_dir, REPO), args.revision[:12]))
+ else:
+ print("downloading %s at %s%s" % (args.checkpoint, args.revision[:12],
+ " (tokenizer and config only)" if args.no_graph else ""),
+ flush=True)
+ fetch(args.checkpoint, args.revision, model_dir, with_weights=wants_weights)
size = sum(os.path.getsize(os.path.join(root, f))
for root, _dirs, files in os.walk(model_dir) for f in files)
print(" checkpoint at %s (%.1f MB)" % (os.path.relpath(model_dir, REPO), size / 1e6))
@@ -115,9 +157,10 @@ def main(argv=None):
print(" --no-graph: no export, as asked")
return 0
- if os.path.exists(graph) and os.path.getsize(graph) > 0 and not args.force:
- print(" graph already exported at %s (use --force to redo it)"
- % os.path.relpath(graph, REPO))
+ if (os.path.exists(graph) and os.path.getsize(graph) > 0 and not args.force
+ and stamped(graph, args.revision)):
+ print(" graph already exported at %s for %s (use --force to redo it)"
+ % (os.path.relpath(graph, REPO), args.revision[:12]))
return 0
os.makedirs(graph_dir, exist_ok=True)
@@ -137,6 +180,7 @@ def main(argv=None):
check=True, cwd=REPO, env=env)
if not os.path.exists(graph) or os.path.getsize(graph) == 0:
raise SystemExit("the exporter reported success but wrote no graph at %s" % graph)
+ stamp(graph, args.revision)
print(" %s (%.1f MB) in %.1f s"
% (os.path.relpath(graph, REPO), os.path.getsize(graph) / 1e6,
time.perf_counter() - started))