Skip to content

armgit / kiln

Public Imported from GitHub
0

Build and revise procedural 3D assets with your coding agent. Local MCP server, CLI and TypeScript engine with rendered review, editable source and GLB export.

Hold the test suite steady on a loaded runner (#119)

Matthew Kissinger7ce59a82cbe9
diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml
index 5418a016..9fba1762 100644
--- a/.github/workflows/pages.yml
+++ b/.github/workflows/pages.yml
@@ -79,7 +79,7 @@ jobs:
# the script `bun test ./src ./scripts` would silently drop it -- and because
# these are the only automated check on what a poster receipt asserts.
- name: Test the site scripts
- run: bun test scripts
+ run: bun test scripts --timeout 20000
working-directory: site
# Fifty programs through the evaluator, writing GLBs, thumbnails and the
diff --git a/AGENTS.md b/AGENTS.md
index e0a965fa..40ccb54d 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -123,6 +123,12 @@ cache identity, contract and preset validation -- so none of them needs a GPU or
build, which is why `--ignore-scripts` is enough. Run it whenever you change `render-service/`; CI
requires it.
+The test scripts set `--timeout 20000`. Bun's 5 s default is below what a cold process spawn or a
+first native-library call costs on a loaded CI runner: tests that take under a second locally have
+timed out at 5 s on the Windows runner while passing on re-run. A test that legitimately needs more
+than 20 s names its own budget as the third argument to `test()`, with the measured duration in a
+comment; a real hang still fails, it just takes 20 s to.
+
Tests and CI pin `KILN_RENDER=cpu`. The coverage ratchet must not vary by whether the runner has a
GPU. It is measured over `src/` alone -- the shipped engine -- so nothing you change under
`scripts/` can move it; those tests still run, and their correctness is their own assertions' job.
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e2d19b1d..fcf02f37 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -3,6 +3,18 @@
Changes to `@kiln/engine`. Source and installable packages are distributed through
GitHub. The package is not published on the npm registry.
+## Unreleased: Test suite holds under a loaded runner
+
+- The test scripts and CI set a 20 s per-test budget instead of Bun's 5 s default. Tests that take
+ under a second locally (a cold CLI spawn, the first `sharp` decode in a file) had timed out at 5 s
+ on the Windows runner and passed on re-run; a real hang still fails, twenty seconds later.
+- The tier-2 driver's process-group test no longer races node start-up. It stopped the run 50 ms
+ after spawn, so on a loaded runner the child died before it had spawned its descendant and the
+ orphan spawned afterwards survived. The test now waits for the child to announce its descendant
+ before interrupting the group, and a separate test proves the deadline reports itself. Temp
+ directories that a just-killed process tree still holds are removed with retries instead of
+ failing the run with `EBUSY`.
+
## Unreleased: Neutral view backdrop, and a named backdrop per capture
- **Every contact sheet is now painted on the neutral studio grey `#aab1bc` instead of near-black
diff --git a/package.json b/package.json
index 0e1e01ea..71bc033a 100644
--- a/package.json
+++ b/package.json
@@ -97,9 +97,9 @@
"build:runtime": "node scripts/build-viewer.mjs && node scripts/build-runtime.mjs all",
"build:viewer": "node scripts/build-viewer.mjs",
"docs:tools": "bun scripts/generate-tool-reference.ts",
- "test": "KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts",
- "test:coverage": "KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts --coverage && bun scripts/check-coverage.mjs",
- "test:live": "KILN_SPIKE_LIVE=1 bun test",
+ "test": "KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts --timeout 20000",
+ "test:coverage": "KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts --coverage --timeout 20000 && bun scripts/check-coverage.mjs",
+ "test:live": "KILN_SPIKE_LIVE=1 bun test --timeout 20000",
"test:package": "node scripts/test-plugin-package.mjs && node scripts/smoke-package.mjs",
"test:render-service": "npm --prefix render-service test",
"lint": "biome check --error-on-warnings .",
diff --git a/scripts/reliability.test.mjs b/scripts/reliability.test.mjs
index 7105f051..a736659a 100644
--- a/scripts/reliability.test.mjs
+++ b/scripts/reliability.test.mjs
@@ -46,8 +46,11 @@ describe('repository reliability contracts', () => {
const readme = (await readText('README.md')).replace(/\s+/g, ' ');
const workflow = await readText('.github/workflows/ci.yml');
+ // The 20 s per-test budget is part of the contract: Bun's 5 s default is
+ // below a cold process spawn or a first native-library call on a loaded
+ // runner, and the suite lost three Windows runs to it in one day.
expect(pkg.scripts['test:coverage']).toBe(
- 'KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts --coverage && bun scripts/check-coverage.mjs',
+ 'KILN_SPIKE_LIVE=0 KILN_RENDER=cpu bun test src scripts --coverage --timeout 20000 && bun scripts/check-coverage.mjs',
);
// A warning baseline is a number kept in prose that everyone agrees to ignore, and
// the twenty-sixth finding arrives invisible. The tree reports nothing, so the flag
diff --git a/scripts/tier2-dogfood.test.mjs b/scripts/tier2-dogfood.test.mjs
index 71f2e9fb..799283a8 100644
--- a/scripts/tier2-dogfood.test.mjs
+++ b/scripts/tier2-dogfood.test.mjs
@@ -19,7 +19,11 @@ import {
const roots = [];
afterEach(() => {
- for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true });
+ // A root can still be the working directory of a process tree the driver just
+ // killed; Windows reports that as EBUSY for a few hundred milliseconds after
+ // taskkill returns, so the removal retries instead of failing the test.
+ for (const root of roots.splice(0))
+ rmSync(root, { recursive: true, force: true, maxRetries: 10, retryDelay: 100 });
});
describe('Tier 2 blind dogfood driver', () => {
@@ -378,31 +382,63 @@ describe('Tier 2 blind dogfood driver', () => {
]);
});
- test('the deadline kills only the spawned process group', async () => {
+ test('the deadline reports itself and ends the process', async () => {
const root = mkdtempSync(join(tmpdir(), 'kiln-tier2-timeout-test-'));
roots.push(root);
+ const stdoutPath = join(root, 'stdout.jsonl');
+ const stderrPath = join(root, 'stderr.log');
+ const result = await runProcess('node', ['-e', 'setInterval(()=>{},1000)'], {
+ cwd: root,
+ env: process.env,
+ stdoutPath,
+ stderrPath,
+ timeoutMs: 50,
+ killGraceMs: 50,
+ });
+
+ expect(result.timedOut).toBe(true);
+ expect(result.interrupted).toBe(false);
+ expect(readFileSync(stderrPath, 'utf8')).toContain('deadline exceeded');
+ }, 30_000);
+
+ test('stopping the run kills the descendants the process had spawned', async () => {
+ const root = mkdtempSync(join(tmpdir(), 'kiln-tier2-timeout-test-'));
+ roots.push(root);
+ const spawned = join(root, 'descendant-spawned');
const marker = join(root, 'descendant-survived');
const stdoutPath = join(root, 'stdout.jsonl');
const stderrPath = join(root, 'stderr.log');
+ // The child announces its descendant only once it exists. Stopping the run
+ // on a fixed timer instead raced node start-up on a loaded runner: the child
+ // was killed before it had spawned anything, and the orphan it spawned
+ // afterwards survived, which is not what the driver is asked to prevent.
const childScript = [
"const {spawn}=require('node:child_process')",
- `spawn(process.execPath,['-e',${JSON.stringify(`setTimeout(()=>require('node:fs').writeFileSync(${JSON.stringify(marker)},'bad'),350)`)}],{stdio:'ignore'})`,
+ `spawn(process.execPath,['-e',${JSON.stringify(`setTimeout(()=>require('node:fs').writeFileSync(${JSON.stringify(marker)},'bad'),1500)`)}],{stdio:'ignore'})`,
+ `require('node:fs').writeFileSync(${JSON.stringify(spawned)},'ok')`,
'setInterval(()=>{},1000)',
].join(';');
- const result = await runProcess('node', ['-e', childScript], {
+ const controller = new AbortController();
+ const run = runProcess('node', ['-e', childScript], {
cwd: root,
env: process.env,
stdoutPath,
stderrPath,
- timeoutMs: 50,
+ timeoutMs: 20_000,
killGraceMs: 50,
+ signal: controller.signal,
});
- await new Promise((resolve) => setTimeout(resolve, 450));
-
- expect(result.timedOut).toBe(true);
+ while (!existsSync(spawned)) await new Promise((resolve) => setTimeout(resolve, 20));
+ controller.abort();
+ const result = await run;
+ // Outlive the descendant's own timer so a survivor would have written.
+ await new Promise((resolve) => setTimeout(resolve, 1700));
+
+ expect(result.interrupted).toBe(true);
+ expect(result.timedOut).toBe(false);
expect(existsSync(marker)).toBe(false);
- expect(readFileSync(stderrPath, 'utf8')).toContain('deadline exceeded');
- });
+ expect(readFileSync(stderrPath, 'utf8')).toContain('terminating process group');
+ }, 30_000);
test('a harness prompt can be delivered through stdin without becoming a shell argument', async () => {
const root = mkdtempSync(join(tmpdir(), 'kiln-tier2-stdin-test-'));
diff --git a/src/__tests__/cli-backdrop.test.ts b/src/__tests__/cli-backdrop.test.ts
index 45370416..ad0c2762 100644
--- a/src/__tests__/cli-backdrop.test.ts
+++ b/src/__tests__/cli-backdrop.test.ts
@@ -97,4 +97,5 @@ it('paints the CLI contact sheet on the named backdrop, with or without a captur
} finally {
await rm(directory, { recursive: true, force: true });
}
-});
+ // Three cold CLI spawns, each with a CPU render: 4.8 s locally under load.
+}, 60_000);
diff --git a/src/__tests__/community-exporter.test.ts b/src/__tests__/community-exporter.test.ts
index 13044523..c23ee688 100644
--- a/src/__tests__/community-exporter.test.ts
+++ b/src/__tests__/community-exporter.test.ts
@@ -119,7 +119,9 @@ test('headless textures retain pixels and source state across concurrent exports
expect(texture.source).toBe(source);
expect(texture.image).toBe(sourceImage);
expect(mesh.userData['privatePayload']).toBe('do not export');
-});
+ // Under 1 s locally, but this is the file's first sharp decode, so it pays
+ // libvips start-up; a loaded Windows runner has taken 5.8 s for it.
+}, 30_000);
test('candidate preserves morph and skin data and does not mutate bone references', async () => {
process.env['KILN_GLTF_EXPORTER'] = 'three';