3D engine that AI agents drive
Review fixes for the render budget: region jobs, frame rate budgets, aspect, unrounded fit, memory cap warning (#904)
Nguyen Xuan An148e721ba942
diff --git a/docs/mcp.md b/docs/mcp.mdindex c9b50362..3a6bb39a 100644--- a/docs/mcp.md+++ b/docs/mcp.md@@ -206,7 +206,7 @@ Limits of the estimate: it covers a job's own time, not the wait for a busy mach`settings.budget` lets a job choose its own renderer and size from what you can spend: `{ "secondsPerFrame": 2 }` for an offline render, `{ "framesPerSecond": 30 }` for a realtime preview, and optionally `"memoryMiB"` (64 to 65536) as a cap on the estimated peak memory. Give exactly one of `secondsPerFrame` (0.001 to 3600) and `framesPerSecond` (0.01 to 240). `width` and `height` stay the largest frame you want; `renderer` must be left out, because the budget picks it. A job that cannot go through the path tracer (`passes`, `transparent`, `region`, `multilayer`, a `format` other than png) only considers raster.-The runtime estimates these candidates with the cost model above (history correction included), best looking first, and takes the first one that fits: the path tracer at 256, 64 and 16 samples with the wavelet denoiser at full size on the runtime's backend, then the raster renderer at full size, 0.75, 0.5 and 0.25 scale (sides rounded, even for video, at least 64). The ladder is the whole choice for now: named quality tiers are a later slice of #904. A candidate fits when its estimated seconds per frame (the whole job divided by its frames, so scene build time counts) is at most 85% of the budget, because the estimate is only good to a few tens of percent, and its estimated peak memory is within `memoryMiB`. The same scene, budget and history always give the same pick.+The runtime estimates these candidates with the cost model above (history correction included), best looking first, and takes the first one that fits: the path tracer at 256, 64 and 16 samples with the wavelet denoiser at full size on the runtime's backend, then the raster renderer at full size, 0.75, 0.5 and 0.25 scale (sides rounded, even for video, at least 64). The ladder is the whole choice for now: named quality tiers are a later slice of #904. Both sides of the raster frame shrink by one factor, so the aspect holds, and the short side stays at least 64 px; a size that repeats the previous candidate is dropped. A job with `region` keeps the full frame, because the region is in full-frame pixels. A candidate that cannot be estimated is skipped and listed in `skipped`. A candidate fits when its estimated time is at most 85% of the budget, because the estimate is only good to a few tens of percent, and its estimated peak memory is within `memoryMiB`. With `secondsPerFrame` the time is the whole job divided by its frames, so startup and scene build count, and a single image can never fit a very small budget. With `framesPerSecond` the time is the per-frame cost only: it is a render-speed budget, not `settings.fps` (the video's own rate), and startup and scene build (`estimate.fixedSeconds`) are reported in `warnings` instead. The server's `--memory` render-target cap can still shrink the frame while the job runs; `warnings` says so, and also when `memoryMiB` is above that cap. `metrics.report.fallbacks` in the result lists a reduction. The same scene, budget and history always give the same pick.`scene_render_submit` returns `budget` beside `estimate`: `requested`, `picked` (label, size, renderer or `raster`, expected seconds per frame and memory), `fits`, `reason` (which better candidate was too expensive, or why nothing fits) and `considered`, every candidate with its expected cost. When nothing fits, the cheapest candidate runs anyway and `fits` is false; raise the budget or shorten the job. `scripts/measure-render-budget.mjs` submits tight, middle, loose and 30 fps budgets on two scenes and prints what each picked beside the measured time.diff --git a/scripts/agent-render-budget.mjs b/scripts/agent-render-budget.mjsindex f96578ea..1b8582dc 100644--- a/scripts/agent-render-budget.mjs+++ b/scripts/agent-render-budget.mjs@@ -3,7 +3,7 @@// would cost, so the same scene, budget and history always give the same pick./** What scene_capabilities topic jobs says about the budget. The details are in docs/mcp.md. */-export const RENDER_BUDGET_NOTE = 'settings.budget { secondsPerFrame | framesPerSecond, memoryMiB? } replaces hand-picked renderer settings: width and height become the largest frame, and the job takes the best candidate whose estimate fits (path tracer at 256, 64 or 16 samples, then raster at scale 1, 0.75, 0.5, 0.25). Time is the whole job divided by its frames, and a pick is expected to use at most 85% of it. submit returns budget { picked, fits, reason, considered }; when nothing fits the cheapest candidate runs and fits is false.';+export const RENDER_BUDGET_NOTE = 'settings.budget { secondsPerFrame | framesPerSecond, memoryMiB? } replaces hand-picked renderer settings: width and height become the largest frame, and the job takes the best candidate whose estimate fits (path tracer at 256, 64 or 16 samples, then raster at scale 1, 0.75, 0.5, 0.25). secondsPerFrame counts the whole job divided by its frames; framesPerSecond is a render-speed budget (not settings.fps, the video rate) and counts only the per-frame cost, leaving startup and scene build out. A pick is expected to use at most 85% of the time. submit returns budget { picked, fits, reason, considered }; when nothing fits the cheapest candidate runs and fits is false.';export const BUDGET_LIMITS = Object.freeze({secondsPerFrame: Object.freeze({ minimum: 0.001, maximum: 3600 }),@@ -45,9 +45,12 @@ export function parseBudget(input) {return budget;}-function scaledSide(side, scale, even) {- const unit = even ? 2 : 1;- return Math.max(MINIMUM_SIDE, Math.round((side * scale) / unit) * unit);+/** Both sides shrink by one factor so the aspect holds; the factor is raised when it would take the short side under the minimum. */+function scaledSize(base, scale) {+ const unit = base.kind === 'video' ? 2 : 1;+ const factor = Math.max(scale, MINIMUM_SIDE / Math.min(base.width, base.height));+ const side = length => Math.min(length, Math.max(MINIMUM_SIDE, Math.round((length * factor) / unit) * unit));+ return { width: side(base.width), height: side(base.height) };}/** Raw job settings (what the caller would have written by hand) for each candidate, best first, with a label. */@@ -59,10 +62,11 @@ export function budgetCandidates(base, tracerBackend) {candidates.push({ label: `path tracer, ${samples} samples, full size`, settings: { ...base, renderer: { type: 'pathTracer', backend: tracerBackend, samples, denoise: 'wavelet' } } });}}- const even = base.kind === 'video';- for (const scale of RASTER_SCALE_STEPS) {- const width = scaledSide(base.width, scale, even);- const height = scaledSide(base.height, scale, even);+ // A region is placed in pixels of the full frame, so a region job keeps the full size.+ for (const scale of base.region ? [1] : RASTER_SCALE_STEPS) {+ const { width, height } = scaledSize(base, scale);+ const previous = candidates[candidates.length - 1]?.settings;+ if (previous && !previous.renderer && previous.width === width && previous.height === height) continue;candidates.push({ label: `raster, ${scale === 1 ? 'full size' : `${scale} scale`}`, settings: { ...base, width, height } });}return candidates;@@ -76,27 +80,46 @@ function cost(estimate) {* Chooses among the candidates. `estimateOf(rawSettings)` returns an estimate for a candidate; `frameCount` turns its seconds into seconds per frame. The result holds the settings to* submit and the report of the choice: what was picked, whether it fits, why, and what each candidate was expected to cost.*/-export function chooseBudgetSettings({ base, budget, tracerBackend, frameCount, estimateOf }) {- const considered = budgetCandidates(base, tracerBackend).map(candidate => {- const estimate = estimateOf(candidate.settings);- return { ...candidate, estimate, secondsPerFrame: round(estimate.seconds / frameCount, 3) };- });- const fits = candidate => candidate.secondsPerFrame <= budget.secondsPerFrame * TIME_HEADROOM && (budget.memoryMiB === undefined || candidate.estimate.peakMemoryMiB <= budget.memoryMiB);+export function chooseBudgetSettings({ base, budget, tracerBackend, frameCount, estimateOf, serverMemoryMiB }) {+ const failures = [];+ const considered = [];+ for (const candidate of budgetCandidates(base, tracerBackend)) {+ try {+ const estimate = estimateOf(candidate.settings);+ // A frame rate budget is about render speed, so it leaves the one-time startup and scene build out.+ const timePerFrame = budget.framesPerSecond === undefined ? estimate.seconds / frameCount : estimate.frameSeconds;+ considered.push({ ...candidate, estimate, timePerFrame, secondsPerFrame: round(timePerFrame, 3) });+ } catch (error) {+ failures.push(`${candidate.label}: ${error.message}`);+ }+ }+ if (!considered.length) throw new RangeError(`budget found no candidate this job can use. ${failures.join(' ')}`);+ const fits = candidate => candidate.timePerFrame <= budget.secondsPerFrame * TIME_HEADROOM && (budget.memoryMiB === undefined || candidate.estimate.peakMemoryMiB <= budget.memoryMiB);const fitting = considered.find(fits);- const cheapest = [...considered].sort((first, second) => first.secondsPerFrame - second.secondsPerFrame || first.estimate.peakMemoryMiB - second.estimate.peakMemoryMiB)[0];+ const cheapest = [...considered].sort((first, second) => first.timePerFrame - second.timePerFrame || first.estimate.peakMemoryMiB - second.estimate.peakMemoryMiB)[0];const picked = fitting ?? cheapest;const { settings, label } = picked;const requested = { secondsPerFrame: round(budget.secondsPerFrame, 4) };if (budget.framesPerSecond !== undefined) requested.framesPerSecond = budget.framesPerSecond;if (budget.memoryMiB !== undefined) requested.memoryMiB = budget.memoryMiB;+ const warnings = [];+ if (serverMemoryMiB !== undefined && budget.memoryMiB !== undefined && budget.memoryMiB > serverMemoryMiB) {+ warnings.push(`budget.memoryMiB ${budget.memoryMiB} is above the server --memory render-target cap of ${serverMemoryMiB} MiB, which is the limit that applies.`);+ }+ if (serverMemoryMiB !== undefined) warnings.push(`The server --memory cap of ${serverMemoryMiB} MiB may still shrink the frame; metrics.report.fallbacks in the result says if it did.`);+ if (budget.framesPerSecond !== undefined && picked.estimate.fixedSeconds > 0) {+ warnings.push(`Startup and scene build take about ${picked.estimate.fixedSeconds} s once and are not counted against framesPerSecond.`);+ }return {settings,report: {requested,picked: { label, width: settings.width, height: settings.height, renderer: settings.renderer ? { backend: settings.renderer.backend, samples: settings.renderer.samples, denoise: settings.renderer.denoise } : 'raster', expected: { ...cost(picked.estimate), secondsPerFrame: picked.secondsPerFrame } },fits: Boolean(fitting),- reason: fitting ? reasonForFit(picked, considered, budget) : reasonForMiss(picked, budget),+ reason: fitting ? reasonForFit(picked, considered, budget) : reasonForMiss(picked, budget, frameCount),+ ...(warnings.length ? { warnings } : {}),considered: considered.map(candidate => ({ label: candidate.label, secondsPerFrame: candidate.secondsPerFrame, peakMemoryMiB: candidate.estimate.peakMemoryMiB, fits: fits(candidate) })),+ ...(failures.length ? { skipped: failures } : {}),},};}@@ -110,10 +133,14 @@ function reasonForFit(picked, considered, budget) {const better = considered.slice(0, considered.indexOf(picked));if (!better.length) return `The best candidate fits: about ${picked.secondsPerFrame} s per frame and ${picked.estimate.peakMemoryMiB} MiB.`;const next = better[better.length - 1];- const over = next.secondsPerFrame > budget.secondsPerFrame * TIME_HEADROOM ? `${next.label} needs about ${next.secondsPerFrame} s per frame against ${round(budget.secondsPerFrame * TIME_HEADROOM, 4)} s` : `${next.label} needs about ${next.estimate.peakMemoryMiB} MiB against ${budget.memoryMiB} MiB`;+ const over = next.timePerFrame > budget.secondsPerFrame * TIME_HEADROOM ? `${next.label} needs about ${next.secondsPerFrame} s per frame against ${round(budget.secondsPerFrame * TIME_HEADROOM, 4)} s` : `${next.label} needs about ${next.estimate.peakMemoryMiB} MiB against ${budget.memoryMiB} MiB`;return `${over}, so ${picked.label} is the best that fits.`;}-function reasonForMiss(picked, budget) {- return `Nothing fits: the cheapest candidate, ${picked.label}, needs about ${picked.secondsPerFrame} s per frame and ${picked.estimate.peakMemoryMiB} MiB against ${round(budget.secondsPerFrame, 4)} s${budget.memoryMiB === undefined ? '' : ` and ${budget.memoryMiB} MiB`}. It runs anyway; raise the budget or shorten the job.`;+function reasonForMiss(picked, budget, frameCount) {+ const fixedPerFrame = picked.estimate.fixedSeconds / frameCount;+ const fixedNote = budget.framesPerSecond === undefined && fixedPerFrame > budget.secondsPerFrame * TIME_HEADROOM+ ? ` Startup and scene build alone take about ${round(fixedPerFrame, 3)} s per frame over ${frameCount} frame${frameCount === 1 ? '' : 's'}.`+ : '';+ return `Nothing fits: the cheapest candidate, ${picked.label}, needs about ${picked.secondsPerFrame} s per frame and ${picked.estimate.peakMemoryMiB} MiB against ${round(budget.secondsPerFrame, 4)} s${budget.memoryMiB === undefined ? '' : ` and ${budget.memoryMiB} MiB`}. It runs anyway; raise the budget or lengthen the job.${fixedNote}`;}diff --git a/scripts/agent-render-estimate.mjs b/scripts/agent-render-estimate.mjsindex 3e3fc999..3a031fa3 100644--- a/scripts/agent-render-estimate.mjs+++ b/scripts/agent-render-estimate.mjs@@ -152,6 +152,8 @@ export function estimateRenderJob({ size, settings, backend, memoryBudgetMiB, hiseconds: round(modelSeconds * correction.seconds, 1),peakMemoryMiB: Math.round(modelPeakMemoryMiB * correction.memory),framesPerSecond: round(settings.frameCount / Math.max(0.001, modelSeconds * correction.seconds), 2),+ frameSeconds: round(frameSeconds * correction.seconds, 4),+ fixedSeconds: round((modelSeconds - frameSeconds * settings.frameCount) * correction.seconds, 2),basis: basisOf(correction, jobKind(settings, backend)),modelSeconds: round(modelSeconds, 1),modelPeakMemoryMiB: Math.round(modelPeakMemoryMiB),diff --git a/scripts/agent-runtime.mjs b/scripts/agent-runtime.mjsindex 5861629a..d3516063 100644--- a/scripts/agent-runtime.mjs+++ b/scripts/agent-runtime.mjs@@ -1270,7 +1270,7 @@ export function createAgentRuntime({ kernelWorkers = true, backend = 'gpu', browif (base.renderer !== undefined) throw new TypeError('budget picks the renderer: leave settings.renderer out, or leave settings.budget out to choose it yourself.');const budget = parseBudget(budgetInput);const frameCount = normalizeJobSettings(base, snapshot.clips, snapshot.recipe).frameCount;- ({ settings: jobSettings, report: budgetReport } = chooseBudgetSettings({ base, budget, tracerBackend: backend === 'cpu' ? 'cpu' : 'gpu', frameCount, estimateOf }));+ ({ settings: jobSettings, report: budgetReport } = chooseBudgetSettings({ base, budget, tracerBackend: backend === 'cpu' ? 'cpu' : 'gpu', frameCount, estimateOf, serverMemoryMiB: memory }));}const estimate = estimateOf(jobSettings);estimateForNextJob = estimate;diff --git a/tests/render-budget.test.js b/tests/render-budget.test.jsindex 4a84336b..ccd6dc7b 100644--- a/tests/render-budget.test.js+++ b/tests/render-budget.test.js@@ -77,3 +77,48 @@ test('the same scene, budget and history give the same pick', () => {const plain = choose({ secondsPerFrame: 1 }).report.considered.find(candidate => candidate.label === 'raster, full size');assert.ok(corrected.secondsPerFrame > plain.secondsPerFrame);});++test('a region job keeps the full frame instead of throwing on a smaller candidate', () => {+ const chosen = choose({ secondsPerFrame: 0.001 }, { region: { x: 100, y: 100, width: 400, height: 300 } });+ assert.equal(chosen.settings.width, 1280);+ assert.deepEqual(chosen.report.considered.map(candidate => candidate.label), ['raster, full size']);+});++test('one candidate that cannot be estimated is skipped, and none usable names the budget', () => {+ const failing = raw => {+ if (raw.renderer) throw new RangeError('tracer refused');+ return estimateRenderJob({ size, settings: normalizeJobSettings(raw, [], recipe), backend: 'gpu', history: [] });+ };+ const chosen = chooseBudgetSettings({ base, budget: parseBudget({ secondsPerFrame: 600 }), tracerBackend: 'gpu', frameCount: 1, estimateOf: failing });+ assert.equal(chosen.report.picked.label, 'raster, full size');+ assert.equal(chosen.report.skipped.length, 3);+ const none = () => { throw new RangeError('no'); };+ assert.throws(() => chooseBudgetSettings({ base, budget: parseBudget({ secondsPerFrame: 600 }), tracerBackend: 'gpu', frameCount: 1, estimateOf: none }), /budget found no candidate/);+});++test('scaling keeps the aspect, holds the 64 px floor on the short side and drops same-size duplicates', () => {+ const wide = budgetCandidates({ kind: 'image', width: 1920, height: 80 }, 'gpu').filter(candidate => !candidate.settings.renderer);+ assert.deepEqual(wide.map(candidate => [candidate.settings.width, candidate.settings.height]), [[1920, 80], [1536, 64]]);+ const small = budgetCandidates({ kind: 'image', width: 64, height: 64 }, 'gpu').filter(candidate => !candidate.settings.renderer);+ assert.equal(small.length, 1);+});++test('a frame rate budget counts the per-frame cost and leaves startup out', () => {+ const rate = choose({ framesPerSecond: 15 });+ assert.equal(rate.report.fits, true);+ assert.equal(rate.settings.renderer, undefined);+ assert.match(rate.report.warnings.join(' '), /not counted against framesPerSecond/);+ const perJob = choose({ secondsPerFrame: 1 / 30 });+ assert.equal(perJob.report.fits, false);+ assert.match(perJob.report.reason, /Startup and scene build alone take about/);+});++test('the fit test uses unrounded seconds and the server memory cap is reported', () => {+ const estimateOf = () => ({ seconds: 0.8504, frameSeconds: 0.01, fixedSeconds: 0.84, peakMemoryMiB: 100 });+ const regionBase = { ...base, region: { x: 0, y: 0, width: 64, height: 64 } };+ const edge = chooseBudgetSettings({ base: regionBase, budget: parseBudget({ secondsPerFrame: 1 }), tracerBackend: 'gpu', frameCount: 1, estimateOf });+ assert.equal(edge.report.fits, false);+ const server = chooseBudgetSettings({ base, budget: parseBudget({ secondsPerFrame: 600, memoryMiB: 4096 }), tracerBackend: 'gpu', frameCount: 1, estimateOf, serverMemoryMiB: 192 });+ assert.match(server.report.warnings.join(' '), /above the server --memory render-target cap of 192 MiB/);+ assert.match(server.report.warnings.join(' '), /may still shrink the frame/);+});