Skip to content

SolidCraft: an open-source parametric 3D CAD application built in pure Rust. Desktop, web, and automation through MCP.

Wave 353/509: Kernel rig: every registry command has a case or an exemption (kernel-cases --coverage) (and 2 more)

jyvikof0a52c36b5fb
diff --git a/skills/solidcraft-tests/SKILL.md b/skills/solidcraft-tests/SKILL.md
index 4a6113881b..63ed8d6e9e 100644
--- a/skills/solidcraft-tests/SKILL.md
+++ b/skills/solidcraft-tests/SKILL.md
@@ -25,8 +25,8 @@ A failing test without its fix cannot merge: it turns `main` red for everyone. I
## Goldens, matrices and skipped tests
- The reference for what the kernel builds is the kernel-cases ladder (`tests/kernel-cases/ladder/`, `cargo xtask kernel-cases --ladder`, `docs/kernel-ladder.md`).
-- Kernel PRs run the kernel rig gate (`kernel-rig.yml`): each area against its `KNOWN_FAILING`, the ladder against its golden. In the same PR, a fix removes its line, a new failing case adds one, a changed ladder result is re-recorded (`--ladder --record "why"`); see `tests/kernel-cases/README.md`.
+- Kernel PRs run `kernel-rig.yml`: the areas the diff reaches (`PATHS`) against `KNOWN_FAILING`, the ladder against its golden. In the same PR a fix removes its line, a new failing case adds one, a changed ladder result is re-recorded (`--record "why"`), a new command gets a case or an `EXEMPT` line; see `tests/kernel-cases/README.md`.
- Maintainers' PRs run the native, GPU path and reference tests on the affected packages; others wait for `approved-for-ci`. `release-gate.yml` runs everything in full before develop replaces main.
- The action matrix is a ratchet: record a newly passing action with `cargo xtask action-matrix --bless` (`docs/action-matrix.md`).
- Pull requests skip a few heavy tests, which run in the merge queue, nightly and on `main`. Check that validation after merging. The list is in `.config/nextest.toml` and the testing page.
-- Mutation testing finds behaviour the suites miss: `docs/mutation-testing.md`.
+- Mutation testing finds what the suites miss: `docs/mutation-testing.md`.
diff --git a/tests/kernel-cases/EXEMPT b/tests/kernel-cases/EXEMPT
new file mode 100644
index 0000000000..5bd7ed2b39
--- /dev/null
+++ b/tests/kernel-cases/EXEMPT
@@ -0,0 +1,399 @@
+# Registry commands with no kernel rig case (`cargo xtask kernel-cases --coverage`). Every other
+# command of `solidcraft-cli commands` (planned ones aside) must be run by some recipe.json under
+# tests/kernel-cases/. One line a command: `<id> <why it has no case>`; `#` starts a comment.
+# Each section is sorted, with no id twice.
+#
+# Commands with no geometry to grade: the view, rendering, appearance, materials, selection, the
+# browser, files and saving, documents, diagnostics, and component, parameter and timeline
+# bookkeeping. Each is covered by the engine's and the UI's own tests.
+
+appearance.assign appearance only: colours and textures
+appearance.library appearance only: colours and textures
+appearance.list appearance only: colours and textures
+assembly.bom a bill of materials report
+body.rename a body name
+browser.collapse_all the browser tree: order, groups and folding
+browser.expand_all the browser tree: order, groups and folding
+browser.group the browser tree: order, groups and folding
+browser.move the browser tree: order, groups and folding
+browser.order the browser tree: order, groups and folding
+browser.rename_group the browser tree: order, groups and folding
+browser.ungroup the browser tree: order, groups and folding
+canvas.calibrate reference images, and sketches traced from them (sketch only)
+canvas.decal reference images, and sketches traced from them (sketch only)
+canvas.delete reference images, and sketches traced from them (sketch only)
+canvas.edit reference images, and sketches traced from them (sketch only)
+canvas.insert reference images, and sketches traced from them (sketch only)
+canvas.to_sketch reference images, and sketches traced from them (sketch only)
+component.break_link component links
+component.copies a listing of identical copies
+component.cycle_color component colours
+component.delete removes a component: no geometry built
+component.links component links
+component.list a listing
+component.metadata component metadata
+component.metadata.set component metadata
+component.properties component properties
+component.relink component links
+component.rename a component name
+component.replace_with_link component links
+component.unlock component locking
+component.update_links component links
+contact.list a listing of contact sets
+diagnostics.crash_report crash reports
+diagnostics.crash_reports crash reports
+diagnostics.crash_reports.clear crash reports
+diagnostics.crash_test crash reports
+diagnostics.profile timing: where a command's time goes
+doc.open opening and recovering documents
+doc.recover opening and recovering documents
+doc.recovery_discard opening and recovering documents
+doc.recovery_list opening and recovering documents
+document.metadata document metadata and revisions
+document.metadata.remove document metadata and revisions
+document.metadata.set document metadata and revisions
+document.revision document metadata and revisions
+edit.checkpoint undo, redo and checkpoints: an earlier state restored, no new geometry
+edit.redo undo, redo and checkpoints: an earlier state restored, no new geometry
+edit.restore_checkpoint undo, redo and checkpoints: an earlier state restored, no new geometry
+edit.undo undo, redo and checkpoints: an earlier state restored, no new geometry
+engine.commands the command registry itself
+engine.describe the command registry itself
+explode.create exploded views: display offsets
+explode.delete exploded views: display offsets
+explode.list exploded views: display offsets
+explode.show exploded views: display offsets
+expr.evaluate expression evaluation
+file.open_in_easyprint hands the file to another program
+file.save saving the design
+file.save_as saving the design
+file.save_mesh saving a display mesh
+file.send_to_slicer hands the file to a slicer
+file.slicers a listing of slicers
+frame.profile.list a listing of the profile library
+frame.profile.properties a profile's section properties from the library
+inspect.accessibility display analysis: shading
+inspect.colored_components display: component colours
+inspect.curvature_map display analysis: shading
+inspect.environment_map display analysis: shading
+inspect.silhouette display analysis: outline
+inspect.zebra display analysis: shading
+joint.list a listing of joints
+library.apply material and appearance libraries
+library.export material and appearance libraries
+library.inspect material and appearance libraries
+material.assign materials and their properties
+material.create materials and their properties
+material.delete materials and their properties
+material.edit materials and their properties
+material.inspect materials and their properties
+material.list materials and their properties
+material.properties.directional materials and their properties
+material.properties.validate materials and their properties
+model.edges a listing of the model's topology
+model.faces a listing of the model's topology
+model.threads a listing of the model's topology
+motion.delete removes a motion study
+motion.export writes a study's results to a file
+motion.export_animation writes an animation
+motion.list a listing of motion studies
+motion.play plays a study in the view
+occurrence.properties occurrence properties
+parameters.comment parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.complete parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.delete parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.export parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.favorite parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.graph parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.import parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.list parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.move parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.rename parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parameters.users parameter bookkeeping; a value change that rebuilds is parameters.change, which cases use
+parts.library a listing of the parts library
+plastic.rules a listing of plastic rules
+render.aging.describe rendering: images, cameras, stages
+render.backends rendering: images, cameras, stages
+render.camera.fit rendering: images, cameras, stages
+render.camera.get rendering: images, cameras, stages
+render.camera.set rendering: images, cameras, stages
+render.cancel rendering: images, cameras, stages
+render.effects.describe rendering: images, cameras, stages
+render.export rendering: images, cameras, stages
+render.hdr.inspect rendering: images, cameras, stages
+render.library.appearance.apply rendering: images, cameras, stages
+render.library.describe rendering: images, cameras, stages
+render.library.environment.apply rendering: images, cameras, stages
+render.library.list rendering: images, cameras, stages
+render.library.physical.prepare rendering: images, cameras, stages
+render.library.preview rendering: images, cameras, stages
+render.library.studio.apply rendering: images, cameras, stages
+render.library.studio.prepare rendering: images, cameras, stages
+render.library.surface.bind rendering: images, cameras, stages
+render.library.surface.prepare rendering: images, cameras, stages
+render.library.surface.unbind rendering: images, cameras, stages
+render.library.targets rendering: images, cameras, stages
+render.optics.bind rendering: images, cameras, stages
+render.optics.unbind rendering: images, cameras, stages
+render.preflight rendering: images, cameras, stages
+render.preset.apply rendering: images, cameras, stages
+render.preset.resolve rendering: images, cameras, stages
+render.quality.prepare rendering: images, cameras, stages
+render.result.pin rendering: images, cameras, stages
+render.result.regrade rendering: images, cameras, stages
+render.scene rendering: images, cameras, stages
+render.settings.apply rendering: images, cameras, stages
+render.settings.capture_hdr rendering: images, cameras, stages
+render.settings.get rendering: images, cameras, stages
+render.settings.set rendering: images, cameras, stages
+render.shot.duplicate rendering: images, cameras, stages
+render.shot.remove rendering: images, cameras, stages
+render.shot.rename rendering: images, cameras, stages
+render.stage.backdrop rendering: images, cameras, stages
+render.stage.clear rendering: images, cameras, stages
+render.stage.ground rendering: images, cameras, stages
+render.start rendering: images, cameras, stages
+render.status rendering: images, cameras, stages
+select.clear the selection
+select.set the selection
+selection.delete deletes what is selected through the commands that own each kind
+sheet.rules a listing of sheet metal rules
+sketch.arc.tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.auto_constrain 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.auto_constrain_from_datum 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.break 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.break_link 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.cancel 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.centerline 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.chain 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.chamfer.distance_angle 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.chamfer.equal_distance 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.chamfer.two_distance 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.circle.three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.circle.three_tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.circle.two_point 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.circle.two_tangent 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constrainer 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.coincident 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.collinear 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.concentric 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.curvature 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.equal 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.midpoint 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.parallel 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.perpendicular 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.polygon 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.constraint.symmetry 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.construction 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.copy 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.cut 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.delete 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.dimension_text 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.drag 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.drag_end 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.draw.dimensioned 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.drawing_bounds 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.edit_text 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.export_dxf 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.extend 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.fillet 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.finish_auto_constrain 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.glyphs 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.infinite 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.insert_dxf 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.insert_svg 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.inspect 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.line.midpoint 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.loose_ends 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.loose_ends.tie 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.mirror 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.move 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.move3d 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.offset 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.offset.break 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.paste 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.pattern.rectangular 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.point3d 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.polygon.edge 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.rectangle.three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.redefine 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.scale 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.slot.arc_center 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.slot.arc_three_point 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.slot.center_point 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.snap 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.solve 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.toggle_driven 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.transform 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.trim 2D sketch only (the sketch solver and engine tests); no kernel operation
+sketch.trim.path 2D sketch only (the sketch solver and engine tests); no kernel operation
+timeline.dependents a listing of dependents
+timeline.export_script writes the history as a script
+timeline.inspect a listing of the timeline
+timeline.rename a feature name
+timeline.rename_group a group name
+view.ambient_occlusion the view: camera, layout, display
+view.apply the view: camera, layout, display
+view.back the view: camera, layout, display
+view.bottom the view: camera, layout, display
+view.delete the view: camera, layout, display
+view.fit the view: camera, layout, display
+view.front the view: camera, layout, display
+view.graphics_preset the view: camera, layout, display
+view.grid the view: camera, layout, display
+view.ground_shadow the view: camera, layout, display
+view.history_layout the view: camera, layout, display
+view.home the view: camera, layout, display
+view.isometric the view: camera, layout, display
+view.layout the view: camera, layout, display
+view.left the view: camera, layout, display
+view.list the view: camera, layout, display
+view.navigate the view: camera, layout, display
+view.navigation_bar the view: camera, layout, display
+view.normal_to the view: camera, layout, display
+view.perspective the view: camera, layout, display
+view.rename the view: camera, layout, display
+view.right the view: camera, layout, display
+view.save the view: camera, layout, display
+view.spin the view: camera, layout, display
+view.tool the view: camera, layout, display
+view.top the view: camera, layout, display
+view.visual_style the view: camera, layout, display
+
+# Commands that make or change geometry and have no case yet. This section only shrinks:
+# the pull request that adds a case for one removes its line (the check fails on a listed command
+# that a case uses), and a new geometry command gets a case, not a line here.
+[to cover]
+body.repair repair: the solid a refused or open part becomes
+component.copy_bodies assemblies: bodies copied into a component
+component.extract assemblies: bodies extracted to a design
+component.from_bodies assemblies: a component made from bodies
+component.insert_design assemblies: a design inserted
+component.move_bodies assemblies: bodies moved between components
+component.move_sketches assemblies: sketches moved between components
+component.paste_new assemblies: a pasted new component
+component.revert_position assemblies: a placement reverted
+config.activate configurations: a configured part rebuilt per row
+config.column configurations: a configured part rebuilt per row
+config.configure configurations: a configured part rebuilt per row
+config.preview configurations: a configured part rebuilt per row
+config.row configurations: a configured part rebuilt per row
+config.rule configurations: a configured part rebuilt per row
+config.rules configurations: a configured part rebuilt per row
+config.table configurations: a configured part rebuilt per row
+construct.axis.edge construction: the plane, axis or point exactly
+construct.axis.normal_to_face construction: the plane, axis or point exactly
+construct.axis.through_cylinder construction: the plane, axis or point exactly
+construct.axis.two_planes construction: the plane, axis or point exactly
+construct.plane.at_angle construction: the plane, axis or point exactly
+construct.plane.midplane construction: the plane, axis or point exactly
+construct.plane.perpendicular construction: the plane, axis or point exactly
+construct.plane.tangent construction: the plane, axis or point exactly
+construct.plane.three_points construction: the plane, axis or point exactly
+construct.plane.two_edges construction: the plane, axis or point exactly
+construct.point.along_path construction: the plane, axis or point exactly
+construct.point.center construction: the plane, axis or point exactly
+construct.point.edge_plane construction: the plane, axis or point exactly
+construct.point.three_planes construction: the plane, axis or point exactly
+construct.point.two_edges construction: the plane, axis or point exactly
+construct.point.vertex construction: the plane, axis or point exactly
+contact.check assemblies: contact sets and what they report
+contact.create assemblies: contact sets and what they report
+contact.disable_all assemblies: contact sets and what they report
+contact.edit assemblies: contact sets and what they report
+contact.enable_all assemblies: contact sets and what they report
+contact.enable_sets assemblies: contact sets and what they report
+file.export_view_dxf drawings: the projection of the model a view DXF holds
+frame.member frames: member bodies
+frame.profile.insert frames: member bodies
+inspect.center_of_mass inspect: the number or curves it reports
+inspect.clearance inspect: the number or curves it reports
+inspect.compare inspect: the number or curves it reports
+inspect.continuity inspect: the number or curves it reports
+inspect.curvature_comb inspect: the number or curves it reports
+inspect.draft inspect: the number or curves it reports
+inspect.interference inspect: the number or curves it reports
+inspect.isocurve inspect: the number or curves it reports
+inspect.minimum_radius inspect: the number or curves it reports
+inspect.parameter_sweep inspect: results rebuilt over a parameter range
+inspect.section inspect: the number or curves it reports
+joint.as_built assemblies: joint placements and values
+joint.cam assemblies: joint placements and values
+joint.create assemblies: joint placements and values
+joint.delete assemblies: joint placements and values
+joint.drive assemblies: joint placements and values
+joint.edit assemblies: joint placements and values
+joint.gear assemblies: joint placements and values
+joint.inspect assemblies: joint placements and values
+joint.limits assemblies: joint placements and values
+joint.motion_link assemblies: joint placements and values
+joint.origin assemblies: joint placements and values
+joint.rack_pinion assemblies: joint placements and values
+joint.reach assemblies: joint placements and values
+joint.rigid_group assemblies: joint placements and values
+joint.screw assemblies: joint placements and values
+joint.solve assemblies: joint placements and values
+joint.sweep assemblies: joint placements and values
+motion.interference assemblies: motion results
+motion.plot assemblies: motion results
+motion.study assemblies: motion results
+motion.trace assemblies: motion results
+motion.trace.edit assemblies: motion results
+motion.trace.keep assemblies: motion results
+occurrence.copy assemblies: an occurrence copied
+occurrence.drag assemblies: an occurrence dragged to a placement
+occurrence.reparent assemblies: an occurrence moved in the tree
+parts.fastener standard parts: a fastener body
+parts.insert standard parts: a part body
+parts.resize standard parts: a part resized
+plastic.assign_rule plastic: the feature body
+plastic.boss plastic: the feature body
+plastic.lip plastic: the feature body
+plastic.manage_rules plastic: rules that rebuild the features using them
+plastic.rest plastic: the feature body
+sheet.export_bend_table sheet metal: the bend table of a flat pattern
+sheet.export_dxf sheet metal: the flat pattern as DXF
+sheet.hem sheet metal: the feature body
+sheet.path_flange sheet metal: the feature body
+sheet.perforate sheet metal: the feature body
+sketch.arc3d sketch: a curve kind no case profiles yet
+sketch.auto_project sketch from the model: curves the kernel computes from bodies
+sketch.conic sketch: a curve kind no case profiles yet
+sketch.fit_curves_to_section sketch from the model: curves the kernel computes from bodies
+sketch.include_3d sketch from the model: curves the kernel computes from bodies
+sketch.intersect sketch from the model: curves the kernel computes from bodies
+sketch.intersection_curve sketch from the model: curves the kernel computes from bodies
+sketch.isoparametric_curve sketch from the model: curves the kernel computes from bodies
+sketch.project sketch from the model: curves the kernel computes from bodies
+sketch.project_to_surface sketch from the model: curves the kernel computes from bodies
+sketch.spline.control_point_5 sketch: a curve kind no case profiles yet
+sketch.spline3d sketch: a curve kind no case profiles yet
+sketch.spun_profile sketch from the model: curves the kernel computes from bodies
+solid.align solid modelling: the body it makes
+solid.boundary_fill solid modelling: the body it makes
+solid.bounding_solid solid modelling: the body it makes
+solid.cone solid modelling: the body it makes
+solid.pattern.path solid modelling: the body it makes
+solid.replace_face solid modelling: the body it makes
+solid.scale solid modelling: the body it makes
+solid.web solid modelling: the body it makes
+surface.blend surfaces: the surface body it makes
+surface.boundary surfaces: the surface body it makes
+surface.fair surfaces: the surface body it makes
+surface.match surfaces: the surface body it makes
+surface.offset surfaces: the surface body it makes
+surface.reverse_normal surfaces: the surface body it makes
+surface.ruled surfaces: the surface body it makes
+surface.unstitch surfaces: the surface body it makes
+surface.untrim surfaces: the surface body it makes
+timeline.compute_all timeline: the bodies after the rebuild
+timeline.delete timeline: the bodies after the rebuild
+timeline.edit timeline: the bodies after the rebuild
+timeline.group timeline: the bodies after the rebuild
+timeline.move_many timeline: the bodies after the rebuild
+timeline.move_to_group timeline: the bodies after the rebuild
+timeline.reorder timeline: the bodies after the rebuild
+timeline.reorder_many timeline: the bodies after the rebuild
+timeline.roll_to timeline: the bodies after the rebuild
+timeline.suppress timeline: the bodies after the rebuild
+timeline.ungroup timeline: the bodies after the rebuild
diff --git a/tests/kernel-cases/README.md b/tests/kernel-cases/README.md
index f130ef3238..709ec90c06 100644
--- a/tests/kernel-cases/README.md
+++ b/tests/kernel-cases/README.md
@@ -38,6 +38,8 @@ One directory per area, one per case: `tests/kernel-cases/<area>/<id>/`.
| `fillets`, `sweeps`, `threads`, `sheet-metal`, `tessellation` | history rig |
| `continuity` (designed seams of known order, G0 to G4, that prove the continuity measurement itself) | history rig |
+Each area also has its gate files: `KNOWN_FAILING` and `PATHS` (see [the gate](#the-kernel-rig-gate)).
+
`<id>` is `<source number>-<short-name>`, such as `1894-prism-cut-touching-walls`.
Each case has:
@@ -100,16 +102,12 @@ the frozen truck-geometry PRs, a `kernel-paused` PR), for the report's coverage
## The kernel rig gate
Every area is a regression suite: each case is one small case of one issue, with its exact
-answer. `.github/workflows/kernel-rig.yml` runs them all, and the ladder, on every pull request
-(not a draft) that touches the kernel surface: `crates/kernel/**`, `crates/geom/**`,
-`crates/smlib/**` (`SMLIB_REV` and `SMLIB_PREBUILT` included), `crates/step/**`,
-`crates/doc/src/eval*.rs`, `crates/engine/src/cmd/features*.rs`, `tests/kernel-cases/**` and the
-rig's own code (GitHub filters whole files, so any change to `eval*.rs` or `features*.rs` runs
-it). Pushes to main and develop on the same paths, merge groups and the release gate
-(`release-gate.yml`) run it too. One job builds `solidcraft-cli` (release, SMLib) and the rig's
-runner, and hands both to a matrix of shards, one per area plus the ladder, that run in
-parallel; the `kernel-rig` check passes when every shard does. (`fillet-rig.yml`, the earlier
-fillet-only gate, folds into it.) Each area's shard runs:
+answer. `.github/workflows/kernel-rig.yml` runs them, and the ladder, on pull requests (not
+drafts). A plan job (`scripts/ci/kernel_rig_plan.py`) picks the areas from the changed files, one
+job builds `solidcraft-cli` (release, SMLib) and the rig's runner, and a matrix of shards, one per
+selected area plus the ladder, runs them in parallel; the `kernel-rig` check passes when every
+selected shard does (and when none was selected). Pushes to main and develop, merge groups, the
+release gate (`release-gate.yml`) and a run by hand run every area. Each area's shard runs:
```sh
cargo xtask kernel-cases --area <area> --cli target/release/solidcraft-cli \
@@ -135,6 +133,42 @@ request, so the list only shrinks and never goes stale:
To check a list locally before pushing: `cargo xtask kernel-cases --area <area> --cli
target/release/solidcraft-cli --known-failing tests/kernel-cases/<area>/KNOWN_FAILING`.
+### Which areas a pull request runs
+
+- `<area>/PATHS`: the source files whose change reruns that area, one git pathspec glob a line
+ (`*` within a folder, `**` across folders; `#` starts a comment): the engine command modules,
+ doc evaluation and kernel files its commands go through. Only the area's own files: never a
+ file `PATHS-ALL` covers, nor a whole crate (the self-test fails on either).
+- `PATHS-ALL`: the shared core, whose change reruns every area: SMLib (`crates/smlib/**`, its pin
+ included) and `scripts/fetch-smlib.sh`, `crates/geom/**`, the kernel's core (bodies, extrude and
+ revolve, booleans, the SMLib bridge, tessellation, what the grader measures), timeline
+ evaluation, the solid feature commands, the grader, the runner and the workflow.
+- A change under `<area>/` (a case, its `KNOWN_FAILING`, its `PATHS`) reruns that area.
+- The ladder spans every operation: it runs whenever any area does, and on a change under
+ `ladder/`.
+
+The workflow's trigger paths are whole folders that hold every `PATHS` glob (a self-test,
+`scripts/ci/test_kernel_rig_plan.py`, checks it), so a pull request that touches none of them
+starts no job at all. To see what a change would run:
+`git diff --name-only -z origin/develop... | python3 scripts/ci/kernel_rig_plan.py --paths -`.
+
+**A new area** joins with its folder: its cases, a `KNOWN_FAILING` (empty but for its header
+when every case passes), and a `PATHS` listing the files its commands go through. The plan finds
+the areas from the folders, so the workflow needs no edit (unless a `PATHS` glob lies outside
+the trigger paths, which the self-test reports). Add a row to the table under Layout.
+
+### Every command has a case
+
+`cargo xtask kernel-cases --coverage --cli target/release/solidcraft-cli` (run in the gate's
+build job) lists every registry command (`solidcraft-cli commands`, planned ones aside) that no
+`recipe.json` here runs, and fails unless `EXEMPT` lists it: one line a command, `<id> <why it
+has no case>`, each section sorted. Lines before `[to cover]` have no geometry to grade (the
+view, rendering, appearance, files, component and parameter bookkeeping, ...). Lines after it
+make geometry and wait for a case: that section only shrinks. The check also fails on a listed
+command that a case now uses (remove its line in the same pull request) or that the registry no
+longer has. A new geometry command gets a case, not an `EXEMPT` line. The build job runs for this
+check on any change to `crates/engine/src/cmd/`, a recipe or `EXEMPT`, even when no area does.
+
## The complexity ladder
`ladder/` orders kernel operations by complexity instead of by where they went wrong: tiers from
diff --git a/tests/kernel-cases/solids/KNOWN_FAILING b/tests/kernel-cases/solids/KNOWN_FAILING
new file mode 100644
index 0000000000..242ef035c9
--- /dev/null
+++ b/tests/kernel-cases/solids/KNOWN_FAILING
@@ -0,0 +1,4 @@
+# The solids cases that fail today, one a line: `<case> <why it fails today>`. Sorted. A run with
+# `--known-failing` fails on a case that fails and is not listed here, and on a listed case that
+# passes: a fix removes its line in the same pull request.
+# Baseline: rig-cases-solids-frames 45f69c0e5 with SMLib 60b1c6a, aarch64 Linux.
diff --git a/tests/kernel-cases/solids/PATHS b/tests/kernel-cases/solids/PATHS
new file mode 100644
index 0000000000..7b538874c7
--- /dev/null
+++ b/tests/kernel-cases/solids/PATHS
@@ -0,0 +1,9 @@
+# Source files whose change must rerun the solids area (cones, scale, align, bounding solids).
+apps/solidcraft-cli/src/kernel_cases.rs
+crates/doc/src/eval.rs
+crates/doc/src/eval_more.rs
+crates/engine/src/cmd/features.rs
+crates/engine/src/cmd/features_more.rs
+crates/kernel/src/**
+crates/smlib/**
+tests/kernel-cases/solids/**
diff --git a/tests/kernel-cases/solids/cov-cone-frustum-along-x/expected.json b/tests/kernel-cases/solids/cov-cone-frustum-along-x/expected.json
new file mode 100644
index 0000000000..032d0fe1bb
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-frustum-along-x/expected.json
@@ -0,0 +1,27 @@
+{
+ "sources": [],
+ "what": "a truncated cone (frustum) R10 r5 h20 on the X axis from x=5",
+ "source": "analytic",
+ "method": "frustum: V = pi h/3 (R^2+Rr+r^2) = pi*20*175/3; A = pi(R^2+r^2) + pi(R+r) sqrt(h^2+(R-r)^2); centroid h(R^2+2Rr+3r^2)/(4(R^2+Rr+r^2)) = 7.857 from the base; faces: base, top, side = 3",
+ "body_count": 1,
+ "faces": 3,
+ "volume_mm3": 3665.191429188092,
+ "area_mm2": 1364.1829574548633,
+ "center_of_mass": [
+ 12.857142857142858,
+ 0,
+ 0
+ ],
+ "bbox": {
+ "min": [
+ 5,
+ -10,
+ -10
+ ],
+ "max": [
+ 25,
+ 10,
+ 10
+ ]
+ }
+}
diff --git a/tests/kernel-cases/solids/cov-cone-frustum-along-x/recipe.json b/tests/kernel-cases/solids/cov-cone-frustum-along-x/recipe.json
new file mode 100644
index 0000000000..515ddcf422
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-frustum-along-x/recipe.json
@@ -0,0 +1,18 @@
+{
+ "commands": [
+ {
+ "command": "solid.cone",
+ "params": {
+ "radius": 10,
+ "top_radius": 5,
+ "height": 20,
+ "base": [
+ 5,
+ 0,
+ 0
+ ],
+ "axis": "X"
+ }
+ }
+ ]
+}
diff --git a/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/expected.json b/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/expected.json
new file mode 100644
index 0000000000..bdb086425f
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/expected.json
@@ -0,0 +1,27 @@
+{
+ "sources": [],
+ "what": "a full cone (top radius 0: an apex) r6, height -12 from a base at z=10: the apex at z=-2",
+ "source": "analytic",
+ "method": "V = pi r^2 h/3 = pi*36*4; A = pi r^2 + pi r sqrt(h^2+r^2); centroid h/4 from the base toward the apex: z = 10-3; faces: base and side = 2",
+ "body_count": 1,
+ "faces": 2,
+ "volume_mm3": 452.3893421169302,
+ "area_mm2": 365.99066584669873,
+ "center_of_mass": [
+ 0,
+ 0,
+ 7
+ ],
+ "bbox": {
+ "min": [
+ -6,
+ -6,
+ -2
+ ],
+ "max": [
+ 6,
+ 6,
+ 10
+ ]
+ }
+}
diff --git a/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/recipe.json b/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/recipe.json
new file mode 100644
index 0000000000..6ba0aabab8
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-full-apex-negative-height/recipe.json
@@ -0,0 +1,16 @@
+{
+ "commands": [
+ {
+ "command": "solid.cone",
+ "params": {
+ "radius": 6,
+ "height": -12,
+ "base": [
+ 0,
+ 0,
+ 10
+ ]
+ }
+ }
+ ]
+}
diff --git a/tests/kernel-cases/solids/cov-cone-zero-height-refused/expected.json b/tests/kernel-cases/solids/cov-cone-zero-height-refused/expected.json
new file mode 100644
index 0000000000..728ba07bc0
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-zero-height-refused/expected.json
@@ -0,0 +1,8 @@
+{
+ "sources": [],
+ "what": "a cone of height 0 has no volume: refused",
+ "source": "analytic",
+ "method": "the base and the top coincide: no solid, a clean refusal naming the height",
+ "outcome": "refused",
+ "refusal": "height must not be zero"
+}
diff --git a/tests/kernel-cases/solids/cov-cone-zero-height-refused/recipe.json b/tests/kernel-cases/solids/cov-cone-zero-height-refused/recipe.json
new file mode 100644
index 0000000000..53a77ce847
--- /dev/null
+++ b/tests/kernel-cases/solids/cov-cone-zero-height-refused/recipe.json
@@ -0,0 +1,11 @@
+{
+ "commands": [
+ {
+ "command": "solid.cone",
+ "params": {
+ "radius": 5,
+ "height": 0
+ }
+ }
+ ]
+}
diff --git a/xtask/src/kernel_cases.rs b/xtask/src/kernel_cases.rs
index 7121ea2edd..ce3d2e9834 100644
--- a/xtask/src/kernel_cases.rs
+++ b/xtask/src/kernel_cases.rs
@@ -335,6 +335,10 @@ pub fn run(rest: &[&str]) -> Result<(), String> {
}
let o = options(rest, "target/kernel-cases")?;
let dir = root().join("tests/kernel-cases");
+ if rest.contains(&"--coverage") {
+ println!("{}", run_coverage(&o, &dir)?);
+ return Ok(());
+ }
let list = cases(&dir, o.area.as_deref())?;
if list.is_empty() {
return Err(format!("no cases (recipe.json + expected.json) under {}", dir.display()));
@@ -410,9 +414,145 @@ fn gate(text: &str, cases: &[String], statuses: &[&str]) -> Result<String, Strin
}
}
+/// The commands of the registry (`solidcraft-cli commands`): each id, with whether it is only
+/// planned (registered, not built yet).
+fn registry(o: &Options) -> Result<Vec<(String, bool)>, String> {
+ let mut cmd = match &o.cli {
+ Some(bin) => Command::new(bin),
+ None => {
+ let mut c = cargo();
+ c.args(["run", "-q", "--release", "-p", "solidcraft-cli", "--"]);
+ c
+ }
+ };
+ let out = cmd.current_dir(root()).arg("commands").output().map_err(|e| format!("solidcraft-cli: {e}"))?;
+ if !out.status.success() {
+ return Err(format!("solidcraft-cli commands failed:\n{}", String::from_utf8_lossy(&out.stderr)));
+ }
+ let list: Value = serde_json::from_slice(&out.stdout).map_err(|e| format!("commands JSON: {e}"))?;
+ let list = list.as_array().ok_or("commands JSON: not a list")?;
+ Ok(list.iter().filter_map(|c| Some((c["id"].as_str()?.to_string(), c["planned"].as_str().is_some_and(|p| !p.is_empty())))).collect())
+}
+
+/// Every command id the command scripts (`recipe.json`) under `dir` run, at any depth.
+fn used_commands(dir: &Path) -> Result<Vec<String>, String> {
+ let mut out = Vec::new();
+ let mut todo = vec![dir.to_path_buf()];
+ while let Some(d) = todo.pop() {
+ for e in std::fs::read_dir(&d).map_err(|e| format!("{}: {e}", d.display()))?.flatten() {
+ let p = e.path();
+ if p.is_dir() {
+ todo.push(p);
+ } else if p.file_name().is_some_and(|n| n == "recipe.json") {
+ let text = std::fs::read_to_string(&p).map_err(|e| format!("{}: {e}", p.display()))?;
+ out.extend(recipe_commands(&text).map_err(|e| format!("{}: {e}", p.display()))?);
+ }
+ }
+ }
+ out.sort();
+ out.dedup();
+ Ok(out)
+}
+
+/// The command ids a recipe runs: each `commands[].command` of a command script. An oracle
+/// recipe (no `commands`) names features, not commands, and counts none.
+fn recipe_commands(text: &str) -> Result<Vec<String>, String> {
+ let v: Value = serde_json::from_str(text).map_err(|e| e.to_string())?;
+ let Some(list) = v.get("commands") else { return Ok(Vec::new()) };
+ let list = list.as_array().ok_or("`commands` is not a list")?;
+ list.iter()
+ .enumerate()
+ .map(|(i, c)| c["command"].as_str().map(str::to_string).ok_or_else(|| format!("commands[{i}] has no `command` id")))
+ .collect()
+}
+
+/// One line of `tests/kernel-cases/EXEMPT`: a command with no case (its reason is for readers).
+#[derive(Debug, PartialEq)]
+struct Exempt<'a> {
+ id: &'a str,
+ /// Listed under `[to cover]`: a geometry command still waiting for its case.
+ to_cover: bool,
+}
+
+/// The section header of the commands that make geometry but have no case yet.
+const TO_COVER: &str = "[to cover]";
+
+/// The commands `tests/kernel-cases/EXEMPT` exempts from the coverage check: one per line,
+/// `<id> <why it has no case>`; `#` starts a comment line. The lines before `[to cover]` have no
+/// geometry to grade; those after it make geometry and wait for a case, so that section only
+/// shrinks. Each section is sorted, with no id twice in the file, and every line gives a reason.
+fn exempt(text: &str) -> Result<Vec<Exempt<'_>>, String> {
+ let mut out: Vec<Exempt> = Vec::new();
+ let mut to_cover = false;
+ let mut last: Option<&str> = None;
+ for (n, line) in text.lines().enumerate() {
+ let line = line.trim();
+ if line.is_empty() || line.starts_with('#') {
+ continue;
+ }
+ if line == TO_COVER {
+ if to_cover {
+ return Err(format!("line {}: a second {TO_COVER} section", n + 1));
+ }
+ to_cover = true;
+ last = None;
+ continue;
+ }
+ let (id, why) = line.split_once(char::is_whitespace).map_or((line, ""), |(c, w)| (c, w.trim()));
+ if why.is_empty() {
+ return Err(format!("line {}: {id} gives no reason (`<id> <why it has no case>`)", n + 1));
+ }
+ if out.iter().any(|e| e.id == id) {
+ return Err(format!("line {}: {id} is listed twice", n + 1));
+ }
+ if let Some(prev) = last
+ && prev > id
+ {
+ return Err(format!("line {}: {id} is out of order after {prev}: keep each section sorted", n + 1));
+ }
+ last = Some(id);
+ out.push(Exempt { id, to_cover });
+ }
+ Ok(out)
+}
+
+/// The coverage check: every built command of `registry` (id, planned) is in `used` (run by a
+/// case) or exempted by `exempt_text`; an exemption names a registered command that no case
+/// uses, so a case that comes removes its line.
+fn coverage(registry: &[(String, bool)], used: &[String], exempt_text: &str) -> Result<String, String> {
+ let listed = exempt(exempt_text)?;
+ let mut problems = Vec::new();
+ for e in &listed {
+ if !registry.iter().any(|(id, _)| id == e.id) {
+ problems.push(format!("EXEMPT lists {}, which is not a registry command: remove its line", e.id));
+ } else if used.iter().any(|u| u == e.id) {
+ problems.push(format!("EXEMPT lists {}, which a case now uses: remove its line", e.id));
+ }
+ }
+ let mut covered = 0;
+ for (id, _) in registry.iter().filter(|(_, planned)| !planned) {
+ if used.iter().any(|u| u == id) {
+ covered += 1;
+ } else if !listed.iter().any(|e| e.id == id) {
+ problems.push(format!("NO CASE: {id} (add a case under tests/kernel-cases/, or a line to EXEMPT)"));
+ }
+ }
+ if !problems.is_empty() {
+ return Err(format!("kernel-cases coverage:\n {}", problems.join("\n ")));
+ }
+ let to_cover = listed.iter().filter(|e| e.to_cover).count();
+ Ok(format!("kernel-cases coverage: {covered} commands have a case, {} have no geometry, {to_cover} still to cover", listed.len() - to_cover))
+}
+
+fn run_coverage(o: &Options, dir: &Path) -> Result<String, String> {
+ let file = dir.join("EXEMPT");
+ let text = std::fs::read_to_string(&file).map_err(|e| format!("{}: {e}", file.display()))?;
+ coverage(&registry(o)?, &used_commands(dir)?, &text).map_err(|e| format!("{}: {e}", file.display()))
+}
+
#[cfg(test)]
mod tests {
- use super::gate;
+ use super::{Exempt, coverage, exempt, gate, recipe_commands};
fn run(text: &str, cases: &[(&str, &str)]) -> Result<String, String> {
let names: Vec<String> = cases.iter().map(|(c, _)| c.to_string()).collect();
@@ -438,4 +578,43 @@ mod tests {
assert!(run("a x\na y\n", &[("a", "fail")]).unwrap_err().contains("a repeat of"));
assert!(run("a\n", &[("a", "fail")]).unwrap_err().contains("gives no cause"));
}
+
+ #[test]
+ fn a_recipe_names_the_commands_of_its_script() {
+ let script = r#"{"base": "../a", "commands": [{"command": "solid.box", "params": {}}, {"command": "solid.fillet"}]}"#;
+ assert_eq!(recipe_commands(script).unwrap(), ["solid.box", "solid.fillet"]);
+ assert!(recipe_commands(r#"{"id": "01-box", "features": []}"#).unwrap().is_empty());
+ assert!(recipe_commands(r#"{"commands": [{"params": {}}]}"#).unwrap_err().contains("no `command` id"));
+ assert!(recipe_commands(r#"{"commands": {}}"#).unwrap_err().contains("not a list"));
+ }
+
+ #[test]
+ fn the_exempt_list_has_two_sorted_sections_with_a_reason_each() {
+ let text = "# header\nview.fit camera only\nview.top camera only\n\n[to cover]\nsolid.cone surfaces area\n";
+ assert_eq!(
+ exempt(text).unwrap(),
+ [Exempt { id: "view.fit", to_cover: false }, Exempt { id: "view.top", to_cover: false }, Exempt { id: "solid.cone", to_cover: true },]
+ );
+ assert!(exempt("view.top x\nview.fit x\n").unwrap_err().contains("out of order"));
+ assert!(exempt("view.fit x\n[to cover]\nview.fit y\n").unwrap_err().contains("listed twice"));
+ assert!(exempt("view.fit\n").unwrap_err().contains("gives no reason"));
+ assert!(exempt("[to cover]\n[to cover]\n").unwrap_err().contains("second"));
+ }
+
+ #[test]
+ fn coverage_fails_on_a_command_with_no_case_and_on_a_stale_exemption() {
+ let registry: Vec<(String, bool)> =
+ [("solid.box", false), ("solid.cone", false), ("view.fit", false), ("surface.fillet", true)].map(|(i, p)| (i.to_string(), p)).to_vec();
+ let used = ["solid.box".to_string()];
+ let list = "view.fit camera only\n[to cover]\nsolid.cone surfaces area\n";
+ let ok = coverage(&registry, &used, list).unwrap();
+ assert!(ok.contains("1 commands have a case, 1 have no geometry, 1 still to cover"), "{ok}");
+ let missing = coverage(&registry, &used, "view.fit camera only\n").unwrap_err();
+ assert!(missing.contains("NO CASE: solid.cone"), "{missing}");
+ assert!(!missing.contains("surface.fillet"), "a planned command needs no case: {missing}");
+ let covered = coverage(&registry, &["solid.box".into(), "solid.cone".into()], list).unwrap_err();
+ assert!(covered.contains("EXEMPT lists solid.cone, which a case now uses"), "{covered}");
+ let gone = coverage(&registry, &used, "view.fit camera only\nview.gone removed\n[to cover]\nsolid.cone x\n").unwrap_err();
+ assert!(gone.contains("view.gone, which is not a registry command"), "{gone}");
+ }
}
diff --git a/xtask/src/main.rs b/xtask/src/main.rs
index 13be4f8d77..3d13136d96 100644
--- a/xtask/src/main.rs
+++ b/xtask/src/main.rs
@@ -47,7 +47,7 @@ commands:
replay the reference tests (tests/oracle/*/recipe.json), compare with each
case's expected.json and write docs/oracle.md (--cli: a solidcraft-cli
already built, instead of cargo run --release)
- kernel-cases [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--out DIR] [--known-failing FILE]
+ kernel-cases [--area AREA] [--cli BIN] [--timeout S] [--jobs N] [--out DIR] [--known-failing FILE] [--coverage]
the kernel regression rig: run every case in tests/kernel-cases/<area>/<id>/ on
SMLib (one process each, killed after --timeout, default 180 s), grade it against
the case's own answer and write docs/kernel-cases.md; DIR (default
@@ -62,6 +62,9 @@ commands:
docs/kernel-ladder.md (DIR default target/kernel-ladder); --record re-measures
the golden and re-sorts each tier by time; --gate fails on any case whose status
or counts differ from the golden, instead of the report
+ --coverage: instead of running cases, fail on a registry command (the CLI's
+ `commands`, planned ones aside) that no recipe.json under tests/kernel-cases/
+ uses and tests/kernel-cases/EXEMPT does not list, and on a stale EXEMPT line
action-matrix [--bless] [--results FILE] [--from DIR]
try every ordinary action on every face, edge, vertex and body of the design
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;