Build and revise procedural 3D assets with your coding agent. Local MCP server, CLI and TypeScript engine with rendered review, editable source and GLB export.
JavaScript 3d3d-assets3d-modelingagent-skillsagent-toolsai-agentsasset-pipelinecligame-assetsglbgltfmcpmcp-servermodel-context-protocolprocedural-generationprocedural-modelingtext-to-3dthreejstypescript
fix(coverage): give the ratchet a scope, and correct why it needed one (#93)
Matthew Kissingeree6d5d6effc2
diff --git a/AGENTS.md b/AGENTS.mdindex 40461b18..f4f39ba8 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -88,7 +88,8 @@ do not lower them without an explicit measured rationale. Live model tests are o`bun run test:live` and may spend money.Tests and CI pin `KILN_RENDER=cpu`. The coverage ratchet must not vary by whether the runner has a-GPU.+GPU. It is measured over `src/` alone -- the shipped engine -- so nothing you change under+`scripts/` can move it; those tests still run, and their correctness is their own assertions' job.Use strict test-driven development for behavior changes: add a focused failing test, observe theexpected failure, implement the smallest fix, and rerun both the focused and full gates. Preservediff --git a/CHANGELOG.md b/CHANGELOG.mdindex 5b67f1fd..0986736e 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -3,6 +3,35 @@Changes to `@kiln/engine`. Source and installable packages are distributed throughGitHub. The package is not published on the npm registry.+## The coverage ratchet has a scope now — 2026-09-12++- The previous entry blamed 162 lines of lost coverage slack on helper scripts being+ measured and never executed. The number was right; the mechanism was wrong. Bun+ instruments only files it loads, and `coveragePathIgnorePatterns` already dropped+ `**/*.test.mjs`, so those tools were never in the report — there is no `SF:` record+ for `dispatch-asset`, `upload-posters`, `geometry-experiments` or `harness`.+- Exactly **four** files under `scripts/` were instrumented, because tests import them:+ `evaluation/observe-shipping.mjs`, `build-runtime.mjs`, `evaluation/conditions.ts` and+ `authorship.ts`. 229 lines against the engine's 45,726 — 0.5% of the denominator, and+ worth 0.10 points. Two of the four are dense single-expression files, so reflowing+ them inflated their line counts while their covered lines stayed put. That is where+ the 162 lines went.+- So the finding is narrower and worse than "helper scripts dilute the ratchet": the+ engine's coverage contract had a **repo-only formatting input**. A number this+ repository enforces as policy could be moved by reformatting code that does not ship.+- `bunfig.toml` now ignores `scripts/**` for coverage. Those tests still run — they are+ the gates — but their sources leave the denominator, so the ratchet is a statement+ about the shipped engine and nothing else. Re-measured on the narrowed scope: **95.39%+ functions, 92.59% lines**.+- Narrowing hands back 44 lines of slack no new test earned, so `lines` goes 92 → **92.1**+ to hold the 224-line margin 13.3 chose rather than pocket it. `functions` stays at 94,+ where the margin is unchanged in practice. Tightening further is a separate decision.+- And the record carries its own scope now. `measuredBaseline` requires `measuredOver`,+ the gate refuses a baseline without it, and it prints it: `Recorded baseline:+ functions 95.39%, lines 92.59% (over src/, [email protected], 2026-09-12)`. A percentage+ without its scope is as ambiguous as a threshold without a baseline — which is what+ made the wrong explanation above plausible enough to write down.+## The repository's own gates are linted now — 2026-09-12- Adding a file to `scripts/` and running `biome check` on its path printed "No files@@ -19,13 +48,10 @@ GitHub. The package is not published on the npm registry.- Left, with the cost measured rather than guessed. The rest of `scripts/` is one-offtools, several authored as dense single-expression lines. Taking the whole directoryis 26 lint findings, a 1,530-line reflow (3,078 lines to 4,608, pure formatting), and- **162 lines of coverage slack** — `test:coverage` measures `scripts`, those tools are- never executed, so the reflow lands in the uncovered denominator and lines fall- 92.49% to 92.14% against a threshold of 92. Measured by taking the full pass and- running the gate.-- Which says something about the ratchet worth recording: line coverage over manual- helper scripts is noise inside a number this repository treats as a contract.- Narrowing what `test:coverage` measures is the real fix, and a separate decision.+ **162 lines of coverage slack** — lines fall 92.49% to 92.14% against a threshold of+ 92. Measured by taking the full pass and running the gate.+- **Corrected below**: the mechanism first given for that third cost was wrong, and it+ pointed at the wrong fix.- `render-service/` stays out for an unrelated reason: its sources are CRLF by`.gitattributes`, and the formatter would rewrite the line endings.- One incidental finding: Biome prints at most 20 diagnostics by default, so a countdiff --git a/bunfig.toml b/bunfig.tomlindex 9e579b75..5658eb27 100644--- a/bunfig.toml+++ b/bunfig.toml@@ -3,6 +3,15 @@ coverageSkipTestFiles = truecoverageReporter = ["text", "lcov"]coverageDir = "coverage"coveragePathIgnorePatterns = [+ # The ratchet is a contract about the shipped engine, so it is measured over+ # `src/` alone. `scripts/` is repo-only -- it is not in `files` or `exports`,+ # and its correctness is asserted by its own tests, which still run here. Four+ # of its files were being instrumented because a test imports them, 229 lines+ # against the engine's 45,726, and that was enough for a FORMATTING change in a+ # repo-only script to move the engine's coverage contract: reflowing two dense+ # single-expression files took lines from 92.49% to 92.14%. A number treated as+ # a contract should not have an input like that.+ "scripts/**","**/*.test.ts","**/*.test.mjs","**/__tests__/**",diff --git a/coverage-thresholds.json b/coverage-thresholds.jsonindex 73b162f5..9430470d 100644--- a/coverage-thresholds.json+++ b/coverage-thresholds.json@@ -1,12 +1,13 @@{"measuredBaseline": {- "functions": 95.27,- "lines": 92.49,+ "functions": 95.39,+ "lines": 92.59,+ "measuredOver": "src/","measuredUnder": "[email protected]","measuredOn": "2026-09-12"},"thresholds": {"functions": 94,- "lines": 92+ "lines": 92.1}}diff --git a/docs/plans/repo-size-and-r2-migration-2026-09-10.md b/docs/plans/repo-size-and-r2-migration-2026-09-10.mdindex 01a0b073..c9b8db04 100644--- a/docs/plans/repo-size-and-r2-migration-2026-09-10.md+++ b/docs/plans/repo-size-and-r2-migration-2026-09-10.md@@ -1600,7 +1600,18 @@ never by more than 41 levels, at a shadow edge, with a mean delta of 0.01--0.09.There is nothing to choose between them by eye, which is what the 1:1 crop of thepavilion's shadow edge confirms. `pcf` stays the shipped name.-**Turning shadows on is renderer work, not a preset flag.** Four things stand in the+**Decided 2026-09-12: the gallery does not cast shadows.** The owner's call, and the+measurements support it. The site's `<Canvas>` already grounds an asset with drei+`ContactShadows`, which runs its own depth pass and never consults+`shadowMap.type`, so the surface a visitor actually looks at has the cue that+matters. A directional shadow would add 2-6% of pixels at a maximum delta of+27-54/255, from a shipped 3/4 view whose camera sits six degrees from the key light+-- close to the worst angle for showing one -- in exchange for the four pieces of+silently-failing renderer work below. **This row is closed, not deferred.** The+analysis stays because it is the reason, and because anything that revisits the+question starts from these four rather than rediscovering them.++**Turning shadows on is renderer work, not a preset flag.** Four things stood in theway, and the first three were each found by the probe producing no shadow at all:1. **A receiver.** The service renders an asset against a flat background with no@@ -1668,6 +1679,7 @@ the installed tree.| 14.1 | Re-ground 7.12 / SEP-2640 against an authoritative source | **Done 2026-09-12.** Deferred still, for a stronger reason. See below || 14.2 | Establish what actually blocks the `ai` 7 family, and gate it | **Done 2026-09-12.** `@strands-agents/[email protected]` is latest, declares peer `@ai-sdk/provider: ^3.0.0`, and its `VercelModel` is typed on `LanguageModelV3` in 21 places. `@openrouter/[email protected]` requires `ai: ^7.0.0`; `[email protected]` depends on `@ai-sdk/[email protected]`. The family cannot be taken until Strands ships a release accepting the v4 spec -- no budget changes that. `scripts/peer-ranges.test.mjs` now fails on a peer range the version beside it does not meet, verified by patching the installed `@openrouter/ai-sdk-provider` manifest to peer `ai: ^7.0.0` and watching it report `installed 6.0.282` || 14.3 | Assert the prompt-cache breakpoint on the wire, offline | **Done 2026-09-12.** Both native transports captured in `src/agent/providers.test.ts`, both differential. See below |+| 14.5 | Give the coverage ratchet a scope, so repo-only code cannot move the engine's contract | **Done 2026-09-12.** Measured over `src/` alone; baseline re-measured at 95.39% functions / 92.59% lines; `lines` raised 92 to 92.1 so the narrowing does not quietly hand back slack. See below || 14.4 | Bring the repository's own gates under the lint gate, which they were never under | **Done 2026-09-12** for the gates and their tests: 416 files checked where 405 were, and the tree reports nothing. The one-off tools beside them are measured and deliberately left, below |### 14.1 -- the decisive fact is not the SEP's status@@ -1771,17 +1783,27 @@ was one 1,300-character line before this pass. Taking the whole directory costs:lines where the fix is unreviewable and the file is no more readable after.- **A 1,530-line reflow.** `scripts/` goes from 3,078 lines to 4,608, a 50% growththat is entirely formatting.-- **162 lines of coverage slack.** `test:coverage` measures `src scripts`, the- one-off tools are never executed by a test, so the reflow lands almost entirely- in the uncovered denominator: lines fall 92.49% to 92.14% against a threshold of- 92, leaving 62 lines of room where there were 224. Measured by taking the full+- **162 lines of coverage slack.** Lines fall 92.49% to 92.14% against a threshold+ of 92, leaving 62 lines of room where there were 224. Measured by taking the fullpass and running the gate, not estimated.-That third cost is the interesting one, because it says something the ratchet has-been quietly absorbing all along: line coverage over manual helper scripts is-noise in a number the repository treats as a contract. Narrowing what-`test:coverage` measures is the real fix and it is a separate decision, so it is-recorded here rather than folded in.+**Correction, 2026-09-12, to the sentence this row first carried.** It said the+one-off tools are never executed so the reflow lands in the uncovered denominator.+That number is right and that mechanism is wrong, and the wrong mechanism pointed+at the wrong fix. Bun instruments only files it loads, and+`coveragePathIgnorePatterns` already dropped `**/*.test.mjs`, so the one-off tools+were **never in the report at all**: no `SF:` record for `dispatch-asset`,+`promote-asset`, `upload-posters`, `curate-texture-library`, `geometry-experiments`+or `harness`. Exactly four files under `scripts/` were instrumented, because tests+import them -- `evaluation/observe-shipping.mjs`, `build-runtime.mjs`,+`evaluation/conditions.ts` and `authorship.ts` -- totalling 229 lines against the+engine's 45,726, which is 0.5% of the denominator. Two of those four are dense+single-expression files, and reflowing them inflated their line counts while their+covered lines stayed where they were. That is where the 162 lines went.++So the finding is not "helper scripts dilute the ratchet in bulk"; `scripts/` was+only ever dragging the total down by 0.10 points. It is narrower and worse: **the+engine's coverage contract had a repo-only formatting input at all.** Fixed in 14.5.`render-service/` stays out for an unrelated reason: its `src/**` is CRLF by`.gitattributes` and Biome's formatter would rewrite the line endings.@@ -1790,3 +1812,37 @@ One incidental finding worth keeping. Biome caps output at 20 diagnostics bydefault, so `bun run lint` reported 20 findings in `scripts/` when there were 26.`--error-on-warnings` still fails the gate, so nothing escapes -- but a count readoff that output is a floor, not a total.++### 14.5 -- the ratchet now says what it measures++The owner's call on 14.4's open question was to take the proper fix rather than the+cheap one. Measuring first changed what the proper fix was: see the correction in+14.4. The problem was never volume, it was that a number the repository enforces as+policy could be moved by reformatting code that does not ship.++`bunfig.toml` ignores `scripts/**` for coverage. Those tests still run -- they are+the gates -- but their sources leave the denominator, so the ratchet is a statement+about the shipped engine and nothing else. Re-measured on the narrowed scope:+**95.39% functions (3168/3321), 92.59% lines (42336/45726)**, both a little higher+than the blended numbers because the four instrumented `scripts/` files sat at+72.93%.++That narrowing hands back 44 lines of slack no new test earned, so `lines` goes from+92 to **92.1**, holding the 224-line margin 13.3 chose instead of pocketing it --+222 lines now. `functions` stays at 94, where the margin is unchanged in practice+(1.39 points against 1.27, 46 functions against 42). Tightening further is a+separate decision with its own friction and is not taken here.++**The record now carries its own scope.** `measuredBaseline` requires a+`measuredOver`, the gate refuses a baseline without one, and it prints it:+`Recorded baseline: functions 95.39%, lines 92.59% (over src/, [email protected],+2026-09-12)`. A percentage without its scope is as ambiguous as a threshold without+a baseline, and that ambiguity is exactly what made 14.4's first explanation+plausible enough to write down. Verified by a case in+`scripts/check-coverage.test.mjs` that omits the field, and by+`reliability.test.mjs` pinning both the raised threshold and the scope --+`bunfig.toml` losing the `scripts/**` line puts the formatting input straight back.++`AGENTS.md` states the scope where an agent reads it, beside the existing rule that+the ratchet must not vary by whether the runner has a GPU. Same class of rule: the+contract must not depend on things that are not the engine.diff --git a/scripts/check-coverage.mjs b/scripts/check-coverage.mjsindex 8f176263..6e4cd56b 100644--- a/scripts/check-coverage.mjs+++ b/scripts/check-coverage.mjs@@ -55,6 +55,14 @@ if (if (typeof baseline?.functions !== 'number' || typeof baseline?.lines !== 'number') {throw new Error('coverage thresholds must record the measuredBaseline they were ratcheted from');}+// And what it was measured OVER. A percentage without its scope is as ambiguous+// as a threshold without a baseline, and that ambiguity is not theoretical: this+// gate measured `src` plus four incidentally-imported files under `scripts/` for+// months, which was enough for reformatting a repo-only script to move the+// engine's contract -- and enough to make the cause hard to find afterwards.+if (typeof baseline?.measuredOver !== 'string' || baseline.measuredOver.length === 0) {+ throw new Error('coverage measuredBaseline must record what it was measuredOver');+}for (const metric of ['functions', 'lines']) {if (thresholds[metric] > baseline[metric]) {throw new Error(@@ -80,7 +88,9 @@ const summary = `${report('functions', 'functions')}, ${report('lines', 'lines')// Printed on the way past, not only on failure: the recorded baseline is the number// the next ratchet decision is made against, and a log that shows it beside the// current measurement makes a stale record visible without a second run.-const measuredWhere = [baseline.measuredUnder, baseline.measuredOn].filter(Boolean).join(', ');+const measuredWhere = [`over ${baseline.measuredOver}`, baseline.measuredUnder, baseline.measuredOn]+ .filter(Boolean)+ .join(', ');const provenance = [`Recorded baseline: functions ${baseline.functions.toFixed(2)}%, lines ${baseline.lines.toFixed(2)}%`,measuredWhere ? ` (${measuredWhere})` : '',diff --git a/scripts/check-coverage.test.mjs b/scripts/check-coverage.test.mjsindex 8cbc25af..b4966067 100644--- a/scripts/check-coverage.test.mjs+++ b/scripts/check-coverage.test.mjs@@ -10,7 +10,10 @@ import { fileURLToPath } from 'node:url';const checker = fileURLToPath(new URL('./check-coverage.mjs', import.meta.url));/** Two functions and two lines, half of each covered: a flat 50% in both metrics. */-async function fixture(thresholds, measuredBaseline = { functions: 100, lines: 100 }) {+async function fixture(+ thresholds,+ measuredBaseline = { functions: 100, lines: 100, measuredOver: 'src/' },+) {const directory = await mkdtemp(join(tmpdir(), 'kiln-coverage-'));await writeFile(join(directory, 'lcov.info'),@@ -54,7 +57,7 @@ test('aggregate coverage checker accepts coverage at the ratchet', async () => {expect(result.status, result.stderr || result.stdout).toBe(0);expect(result.stdout.trim().split('\n')).toEqual(['Coverage gate passed: functions 50.00% (minimum 50.00%, 0 functions of slack), lines 50.00% (minimum 50.00%, 0 lines of slack)',- 'Recorded baseline: functions 100.00%, lines 100.00%',+ 'Recorded baseline: functions 100.00%, lines 100.00% (over src/)',]);} finally {await rm(directory, { recursive: true, force: true });@@ -94,7 +97,10 @@ test('a shortfall is reported in whole functions and lines, not only in percent'// above anything ever measured fails in whatever change runs next, and reads there// as that change's regression.test('a threshold above the recorded measured baseline is rejected as self-contradictory', async () => {- const directory = await fixture({ functions: 50, lines: 50 }, { functions: 49, lines: 100 });+ const directory = await fixture(+ { functions: 50, lines: 50 },+ { functions: 49, lines: 100, measuredOver: 'src/' },+ );try {const result = run(directory);@@ -123,3 +129,20 @@ test('a thresholds file that records no measured baseline is rejected', async ()await rm(directory, { recursive: true, force: true });}});++// A percentage without its scope is as ambiguous as a threshold without a+// baseline. This gate measured `src` plus four incidentally-imported files under+// `scripts/` for months: enough for reformatting a repo-only script to move the+// engine's contract by 0.35 points, and enough to make the cause hard to find+// once it had. The scope is part of the record now, so it cannot go unstated.+test('a measured baseline that does not say what it measured is rejected', async () => {+ const directory = await fixture({ functions: 50, lines: 50 }, { functions: 100, lines: 100 });+ try {+ const result = run(directory);++ expect(result.status).not.toBe(0);+ expect(result.stderr).toContain('must record what it was measuredOver');+ } finally {+ await rm(directory, { recursive: true, force: true });+ }+});diff --git a/scripts/reliability.test.mjs b/scripts/reliability.test.mjsindex 3020982a..6ca0fd19 100644--- a/scripts/reliability.test.mjs+++ b/scripts/reliability.test.mjs@@ -92,6 +92,12 @@ describe('repository reliability contracts', () => {expect(bunfig).toContain('coverageReporter = ["text", "lcov"]');expect(bunfig).not.toContain('coverageThreshold =');expect(bunfig).toContain('coveragePathIgnorePatterns = [');+ // The ratchet is a contract about the shipped engine, measured over `src/`+ // alone. Without this line four files under `scripts/` are instrumented+ // because tests import them, and reflowing two dense ones moved the engine's+ // coverage by 0.35 points -- a formatting change in repo-only code moving a+ // number the repository treats as policy.+ expect(bunfig).toContain('"scripts/**",');// The enforced ratchets are policy: they must not move without someone// noticing, so they stay literal here. The measured baseline is an// observation, and it legitimately moves whenever code lands — so it is@@ -100,7 +106,11 @@ describe('repository reliability contracts', () => {// drift, and the copy in the test is the one that turns a stale sentence// into a red build with no idea which of the three is right.expect(Object.keys(thresholds).sort()).toEqual(['measuredBaseline', 'thresholds']);- expect(thresholds.thresholds).toEqual({ functions: 94, lines: 92 });+ expect(thresholds.thresholds).toEqual({ functions: 94, lines: 92.1 });+ // Narrowing the scope handed back 44 lines of slack that no new test earned.+ // `lines` moves 92 -> 92.1 to hold the margin 13.3 chose rather than pocket+ // it; tightening beyond that is a separate decision with its own friction.+ expect(thresholds.measuredBaseline.measuredOver).toBe('src/');const { functions, lines } = thresholds.measuredBaseline;expect(functions).toBeGreaterThanOrEqual(thresholds.thresholds.functions);expect(lines).toBeGreaterThanOrEqual(thresholds.thresholds.lines);