Free, open-source 3D sculpting for Linux, macOS and Windows: dense meshes, remeshing, painting, built-in AI (image/text to 3D), Send to Unreal, game-ready LODs, Gaussian splats and OpenUSD.
C++ 3d3d-sculptingcppdear-imguidigital-sculptingfbxgame-developmentgaussian-splattinggltfimage-to-3dmcpopenglopenusdsculptingtext-to-3dunreal-engine
Merge feat/sculptable
islandmed501cc6d2024a
diff --cc CHANGELOG.mdindex ad584129,06263379..f205cee4--- a/CHANGELOG.md+++ b/CHANGELOG.md@@@ -6,33 -6,29 +6,55 @@@ All notable changes to SculptForge are## [Unreleased]+ ### Make sculptable (splat capture -> sculptable, animation-ready mesh)+ - Splats palette > **From scan > Make sculptable**: one click turns the capture into a new SubTool next to the+ splats (which are hidden, not deleted): a watertight, manifold surface (opacity-weighted Gaussian density on a+ voxel grid, built in parallel z slabs with the same result on any thread count; gaps closed; floater shells+ dropped; space boxed in on 5 of 6 sides filled, which seals open bottoms; largest piece kept), QRemesh quads at+ a **Game** (4,000), **Film** (12,000) or **Sculpt** (2,000 + 2 subdivision levels with the scan projected onto+ the top) budget, and the capture's colours as polypaint or an Auto UV texture (splats inside the subject are+ faded out). A hidden "<name> Scan" SubTool keeps the dense surface for ProjectAll.+ - **Face loops**: a face is looked for on the scan from the current view (depth and colour, no ML model, no+ download): eye corners, nose tip, mouth corners and chin become QRemesh Guides (rings round the eyes and the+ orbits, the lips, the nasolabial loop), mirrored, with QRemesh symmetry. **Review** stops after the scan to+ drag the landmarks (the mirror follows) or **Place by hand**, then **Finish**.+ - **Crop Box** with draggable face handles in the viewport; options for Detail, Min Opacity, Close Gaps, Fill+ holes, All pieces, Keep scan. Progress bar and Cancel; Ctrl-hover help; F3 entries.+ - Commands: `splat_to_mesh` now runs the whole pipeline (`detail`, `preset`, `target`, `levels`, `face_guides`,+ `landmarks`, `front`, `up`, `color`, `texture_size`, `crop`, `fill_holes`, `keep_parts`, `keep_scan`, `stage`+ scan/finish, ...; `resolution` still works) and adds SubTools instead of replacing the sculpt; `face_guides`;+ `splat_test_capture` (a built-in test head or mug capture, no download).+ - 5 million splats to Film quads with face loops in about 9 s, 0.9 GB (M1 Pro).+ - Guide chapter "From scan to sculptable mesh"; UI test `make_sculptable.sfui`; UI tests: `drag_landmark` verb,+ `hover` scrolls a panel to the item as `click` does.+### Added+- Facial shapes (Tool > Facial Shapes, `face_shapes_generate`): the 52 Apple ARKit blend shapes, exactly named, as+ sculpt layers on the active SubTool, from a built-in template head (ICT-FaceKit's ICT Face Model Light, MIT):+ automatic or landmark-guided fit (similarity + non-rigid ICP), deformation transfer (Sumner and Popovic 2004) so+ the shapes follow the sculpt's proportions, mirrored Left/Right on symmetric sculpts, made on a low subdivision+ level and carried up with the detail. Eye SubTools get eyeLook rotations, teeth and tongue the lower jaw's motion,+ brows and lashes follow the skin. Face test presets and a slider per shape; `face_landmarks_detect`,+ `face_preset`.+- Live face: `face_weights json={...}` sets ARKit-named shape weights on every SubTool, re-evaluating only the+ vertices they move (no undo step, partial GPU upload; about 9 ms for all 52 on a 700k-vertex head on an M1 Pro).+- Face takes: play recorded face feeds (JSON frames, `face_take_play`), bake them (`face_take_bake`) into a glTF+ animation of the morph target weights (GLB) and USD blend shape weight time samples; saved with the project.+- `new_head`: a new scene with a life-size human head (with mouth and eye sockets) and its eyes.+- Blender bridge: sculpt layers arrive as shape keys with their names and weights (a `.shapes` file beside each+ .GoZ, which now holds the base shape when the mesh has layers).+- UI tests: `zoom` and `look` steps; `run` expands `$SCRIPT_DIR`.+- Splat training from captures (Splats > Train from capture): a phone video, a folder of photos, a phone-link+ capture or a COLMAP / nerfstudio dataset becomes a Gaussian splat that appears in the viewport after the first+ checkpoint and sharpens while it trains (worker thread; orbit freely). Progress over the canvas: step, PSNR,+ elapsed and remaining time, Good enough - stop, Cancel. Presets Fast preview and Quality. Uses Brush 0.3.0+ (Apache-2.0, WebGPU: NVIDIA, AMD, Intel, Apple), downloaded on request from its GitHub release with a pinned+ SHA-256, or a path / `SCULPTFORGE_BRUSH`; COLMAP (BSD; downloaded on request on Windows) for camera positions,+ skipped for captures with AR poses in `capture.json`; ffmpeg for video, keeping the sharpest frame of each+ stretch. The result is centred and levelled from the camera positions, saved next to the capture and copied to+ the LightBox Splats tab. Dropping a video onto the window trains it. Commands `splat_train` and+ `splat_train_cancel` (blocking in sculptforge-cli, Python and MCP). Guide chapter "Scan an object into a splat";+ a generated CC0 test capture (tools/splat/make_test_capture.py, tests/data/splat_capture).- "Fixes & suggestions" in the LightBox header (shown at startup) and the Help menu: opens the bug report andfeature request forms on GitHub.diff --cc CMakeLists.txtindex 4ac7b0c3,44e185df..a93de2ac--- a/CMakeLists.txt+++ b/CMakeLists.txt@@@ -52,7 -52,7 +52,8 @@@ add_library(sculptforge_core STATIsrc/core/Image.cppsrc/core/ImageTo3D.cppsrc/core/Splats.cpp+ src/core/SplatCapture.cpp+ src/core/Sculptable.cppsrc/core/Voxel.cppsrc/core/EnginePresets.cppsrc/core/Masking.cpp@@@ -202,7 -194,7 +203,8 @@@ if(SCULPTFORGE_BUILD_APPsrc/app/Gizmo.cppsrc/app/PanelBake.cppsrc/app/PanelRetopo.cpp+ src/app/PanelSplatTrain.cpp+ src/app/PanelSculptable.cppsrc/app/PanelPerformance.cppsrc/app/PanelGoz.cppsrc/app/PanelBlender.cppdiff --cc docs/guide/index.mdindex 53e61c7a,cdcc66ce..b9c6f9a6--- a/docs/guide/index.md+++ b/docs/guide/index.md@@@ -39,25 -38,25 +39,26 @@@ You can read it inside SculptForge (**H## AI and capture-20. [Image and text to 3D](generate-3d.md)-21. [Gaussian splats](splats.md)-22. [From scan to sculptable mesh](scan-to-sculptable.md)+21. [Image and text to 3D](generate-3d.md)+22. [Gaussian splats](splats.md)+23. [Scan an object into a splat](scan-to-splat.md)++24. [From scan to sculptable mesh](scan-to-sculptable.md)## Files and pipelines- 24. [Baking normal and displacement maps](baking.md)- 25. [Exporting, Unreal and ZBrush](exporting.md)- 26. [Working with Blender](blender.md)- 27. [Projects, autosave and QuickSave](projects.md)- 28. [Scripting and automation](scripting.md)- 29. [SculptForge for studios](studios.md)-23. [Baking normal and displacement maps](baking.md)-24. [Exporting, Unreal and ZBrush](exporting.md)-25. [Working with Blender](blender.md)-26. [Projects, autosave and QuickSave](projects.md)-27. [Scripting and automation](scripting.md)-28. [SculptForge for studios](studios.md)++25. [Baking normal and displacement maps](baking.md)++26. [Exporting, Unreal and ZBrush](exporting.md)++27. [Working with Blender](blender.md)++28. [Projects, autosave and QuickSave](projects.md)++29. [Scripting and automation](scripting.md)++30. [SculptForge for studios](studios.md)## Reference- 30. [Keyboard and mouse](shortcuts.md)- 31. [Performance and big meshes](performance.md)- 32. [Troubleshooting and FAQ](troubleshooting.md)- 33. [Glossary](glossary.md)-29. [Keyboard and mouse](shortcuts.md)-30. [Performance and big meshes](performance.md)-31. [Troubleshooting and FAQ](troubleshooting.md)-32. [Glossary](glossary.md)++31. [Keyboard and mouse](shortcuts.md)++32. [Performance and big meshes](performance.md)++33. [Troubleshooting and FAQ](troubleshooting.md)++34. [Glossary](glossary.md)## Getting helpdiff --cc docs/guide/whats-new.mdindex 2f41b637,9555d5cb..0a23833d--- a/docs/guide/whats-new.md+++ b/docs/guide/whats-new.md@@@ -7,22 -7,14 +7,30 @@@ chapter opens by itself the first time## Next version (in development)+### Facial shapes and live face++**Tool > Facial Shapes > Generate Facial Shapes** gives a sculpted head the 52 Apple ARKit blend shapes+(`eyeBlinkLeft`, `jawOpen`, `mouthSmileRight`, ...) as sculpt layers, fitted to its proportions from a template head+with those shapes (ICT-FaceKit, MIT licence). Eyes, teeth and tongue SubTools get theirs too. Fix any shape by+sculpting into its layer, check them with the **Face test** presets and sliders, and export them with their ARKit+names to GLB, USD, FBX and Blender (as shape keys). `face_weights` drives them live from a face feed, and recorded+takes play in the viewport and bake into a glTF animation or USD time samples. `new_head` starts a scene with a+human head to sculpt on. See [Facial shapes and live face](facial-shapes.md).+### Scan an object into a splat++Film an object with your phone, or take photos all around it, and SculptForge trains a Gaussian splat from it on+your own graphics card. The splat appears in the viewport after a few hundred steps and sharpens while it trains;+you can orbit around it, stop when it's good enough, or cancel. Pick **Video...** or **Photos Folder...** under+**Splats > Train from capture**, or drop a video onto the window. The free Brush trainer is offered as a one-click+download the first time. See [Scan an object into a splat](scan-to-splat.md).+ ### Make sculptable: from a scan to a production asset++ One click in the **Splats** palette turns a Gaussian-splat capture into a clean SubTool you can sculpt, rig and+ animate: a watertight surface with the floaters and background gone and open bottoms sealed, quad topology at a+ Game, Film or Sculpt budget, edge loops round the eyes and mouth and along the nasolabial folds when the capture+ is a head (mirrored across the face), and the capture's colours as polypaint or a texture. A crop box cuts away+ the table or the room; Review shows the face landmarks to nudge before remeshing. The splats stay, hidden. Five+ million splats take about ten seconds on a laptop. See [From scan to sculptable mesh](scan-to-sculptable.md).## Version 0.8.1diff --cc src/app/App.cppindex e3695794,187424c1..16ecbe44--- a/src/app/App.cpp+++ b/src/app/App.cpp@@@ -344,7 -344,7 +344,8 @@@ int App::runUiTest(int w, int h, int fppollAutoUv();pollBake();pollQRemesh();+ pollSplatTrain();+ pollSculptable();updateCameraAspect();Demo::beginFrame();@@@ -502,7 -502,7 +503,8 @@@ int App::runRecording()pollAutoUv();pollBake();pollQRemesh();+ pollSplatTrain();+ pollSculptable();updateCameraAspect();Demo::beginFrame();@@@ -570,7 -570,7 +572,8 @@@ int App::run()pollAutoUv();pollBake();pollQRemesh();+ pollSplatTrain();+ pollSculptable();int fbW = 0, fbH = 0, winW = 0, winH = 0;SDL_GetWindowSizeInPixels(window_, &fbW, &fbH);@@@ -692,7 -692,7 +695,8 @@@ void App::onMouseDown(float x, float yif (maskSelectMouseDown(x, y, mod)) return;if (gizmoMouseDown(x, y, mod)) return; // Gizmo 3D handles, pivot clicks (Gizmo.cpp)if (guideMouseDown(x, y, mod)) return; // the QRemesh Guides brush (PanelRetopo.cpp)+ if (faceMouseDown(x, y, mod)) return; // placing facial landmarks (PanelFace.cpp)+ if (sculptableMouseDown(x, y)) return; // Make sculptable: landmarks, crop box (PanelSculptable.cpp)// Alt+click on another SubTool selects it (ZBrush); Alt+drag stays an inverted stroke or a pan.// Shift+Alt+click selects and frames it (ZBrush), the active one too.const bool frameClick = (mod & SDL_KMOD_ALT) && (mod & SDL_KMOD_SHIFT) && !ctrlDown(mod);@@@ -1313,10 -1313,9 +1320,11 @@@ void App::drawUi()const ImVec2 mouse(mouseX_, mouseY_);drawGizmo(); // Gizmo 3D handles and panel (Gizmo.cpp)drawQRemeshGuides(); // QRemesh Guides curves (PanelRetopo.cpp)+ drawFaceOverlay(); // facial landmarks (PanelFace.cpp)+ drawSplatTrainOverlay(); // splat training progress (PanelSplatTrain.cpp)+ drawSculptableOverlay(); // Make sculptable: crop box, face landmarks (PanelSculptable.cpp)// In Gizmo mode (and with the QRemesh Guides brush) the brush ring shows only while Ctrl masks or Shift smooths.- const bool ringOff = gizmoActive() || (guidesMode_ && !(inputMods() & SDL_KMOD_SHIFT));+ const bool ringOff = gizmoActive() || (guidesMode_ && !(inputMods() & SDL_KMOD_SHIFT)) || sculptableHot();if (((!ImGui::GetIO().WantCaptureMouse && canvasRect().contains(mouse.x, mouse.y)) || drag_ == Drag::Stroke) &&!hideBrushCursor_ && drag_ != Drag::Select && drag_ != Drag::Guide &&(!ringOff || drag_ == Drag::Stroke || ctrlDown(inputMods())))@@@ -2228,7 -2224,7 +2235,8 @@@ void App::drawPalette_Splats()if (ImGui::Button("Save splats...", ImVec2(half, 0))) saveFileDialog(FileAction::SaveSplats);ImGui::SameLine();if (ImGui::Button("Clear##splats", ImVec2(half, 0))) runCommand("splat_clear");+ drawSplatTrainSection();+ drawSculptableSection(); // Make sculptable (PanelSculptable.cpp)}void App::drawBrushCursor() {diff --cc src/app/Config.cppindex a3b73a97,223d57ef..4c836a77--- a/src/app/Config.cpp+++ b/src/app/Config.cpp@@@ -365,9 -365,8 +365,10 @@@ void App::initActions()registerGizmoActions(); // Q W E R: Draw, and the Gizmo 3D's Move, Scale, Rotate (Gizmo.cpp)registerBakeUiActions(); // Tool > Bake (PanelBake.cpp)registerRetopoUiActions(); // Geometry > QRemesh, SubTool > ProjectAll (PanelRetopo.cpp)+ registerSplatTrainUi(); // Splats > Train from capture (PanelSplatTrain.cpp)+ registerSculptableUiActions(); // Splats > Make sculptable (PanelSculptable.cpp)registerVdmActions(); // Brush > Vector Displacement (PanelVdm.cpp)+ registerFaceUiActions(); // Tool > Facial Shapes, face_take_play / face_take_stop (PanelFace.cpp)registerPerformanceUiActions(); // Preferences > Performance (PanelPerformance.cpp)registerInputActions(); // ZBrush keys: U I O, Space, L, V, Ctrl+A / I / H / W..., the input style (Input.cpp)diff --cc src/app/UiTest.cppindex 252780a4,f755ef0c..316d3bdf--- a/src/app/UiTest.cpp+++ b/src/app/UiTest.cpp@@@ -24,10 -24,10 +24,11 @@@// move_view rotate_x rotate_y rotate_z rotate_view scale_x scale_y scale_z scale_uniform;// keep: leave the button down (a shot mid-drag), then `release`// gizmo_hover HANDLE move the mouse onto a Gizmo 3D handle (it lights up, for screenshots)+ // drag_landmark N DX DY [S] left-drag Make sculptable's face landmark N (0-7, FaceLandmarks order) by DX DY pixels// orbit DX DY right-drag on the background, in pixels (with the held modifiers)// camera YAW PITCH set the view's angles in degrees (0 0 = the front view), e.g. for guide screenshots-// run COMMANDS sculptforge-cli commands, e.g. run new_sphere subdivisions=4+// zoom FACTOR / look X Y Z move the camera FACTOR times closer / turn it about (and look at) the point X Y Z+// run COMMANDS sculptforge-cli commands, e.g. run new_sphere subdivisions=4 ($SCRIPT_DIR: the script's folder)// drop FILE as if FILE were dropped onto the window (relative to the script's folder)// env NAME VALUE set an environment variable; $SCRIPT_DIR is the script's folder// copy SRC DST copy a file or folder (SRC relative to the script's folder, DST to the working@@@ -86,8 -85,9 +87,11 @@@// center_x / center_y / center_z and size_x / size_y / size_z (active mesh bounds)// bake_running, baked_maps (maps in the last bake), bake_size, bake_files (files it wrote),// normal_map_preview (1 when the active SubTool shows its baked normal map)+// splat_training (1 while a splat trains), splat_previews (checkpoints shown), splat_train_step,+// splat_train_stage (4 done, 5 failed, 6 cancelled), splat_train_psnr, splat_train_prompt+ // sculptable_running, sculptable_review, sculptable_landmarks (8 when placed),+ // sculptable_placing (next landmark to click, -1), sculptable_crop, sculptable_crop_height,+ // show_splats, landmark_x_N / landmark_y_N / landmark_z_N (face landmark N, mesh space)// (mesh values are the active SubTool's)// snapshot remember the active mesh's vertices (for the moved* values)// remember VALUE NAME keep VALUE's current value as $NAME@@@ -264,7 -264,7 +268,8 @@@ double Demo::query(const std::string& nif (name == "active_texture_layer") return hasTexture(m) && !m.texture->layers.empty() ? double(m.texture->activeLayer) : -1.0;if (name == "auto_uv_running") return app_.autoUv_ ? 1.0 : 0.0;if (const double v = app_.bakeTestValue(name); !std::isnan(v)) return v;+ if (const double v = app_.splatTrainTestValue(name); !std::isnan(v)) return v; // splat_training, ...+ if (const double v = app_.sculptableTestValue(name); !std::isnan(v)) return v;if (name == "qremesh_running") return app_.qremesh_ ? 1.0 : 0.0;if (name == "qremesh_guides") return double(app_.doc_.qremeshGuides.size());if (name == "qremesh_guide_loops") {@@@ -867,7 -882,7 +910,7 @@@ bool Demo::parseStep(int line, const st!busy(app_.pendingUnreal_) && !busy(app_.pendingComfyCheck_) && !app_.galleryBusy() &&!app_.thumbs_.busy() && !app_.libraryScanner_.busy() && !app_.autoUv_ &&!app_.bakeBusy() && !app_.qremesh_ && !app_.voxelRemesh_ && !app_.updateJob_ &&- !app_.splatTrainBusy();- !app_.sculptableBusy();++ !app_.splatTrainBusy() && !app_.sculptableBusy();*idleFrames = idle ? *idleFrames + 1 : 0;if (*idleFrames >= 4) return true;if (idle) return false;