Free, open-source 3D sculpting for Linux, macOS and Windows: dense meshes, remeshing, painting, built-in AI (image/text to 3D), Send to Unreal, game-ready LODs, Gaussian splats and OpenUSD.
C++ 3d3d-sculptingcppdear-imguidigital-sculptingfbxgame-developmentgaussian-splattinggltfimage-to-3dmcpopenglopenusdsculptingtext-to-3dunreal-engine
Mask from the mesh border (#61): Tool > Masking > By border masks open edges plus Rings of edges inside them, optional polygroup borders and soft falloff; mask_border command with undo; a closed mesh reports it has no open edges
islandmedb8ff07473bd1
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 48a643f3..bcb92bb8 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,13 @@ All notable changes to SculptForge are listed here. The format follows## [Unreleased]+### Added+- Mask from the mesh border (#61): **Tool > Masking > By border** masks the open edges of the active SubTool and+ **Rings** (1-20) of edges inside them, so sculpting doesn't move the border (Blender: Mask from mesh+ boundary). **Polygroup borders** also masks where polygroups meet (ZBrush: Mask by polygroup border), **Soft**+ fades the mask inwards; one undo step. A closed mesh masks nothing and the status bar says it has no open edges+ (`mask_border rings=2 polygroups=true soft=true`).+## [0.10.4] - 2026-11-01### Addeddiff --git a/docs/guide/masking.md b/docs/guide/masking.mdindex 73ee9e7a..801c6318 100644--- a/docs/guide/masking.md+++ b/docs/guide/masking.md@@ -3,7 +3,7 @@A mask protects part of the model from change. Masked areas are drawn darker, and every brush, the Paintbrush and colour fills leave them alone. This chapter covers painting masks with <kbd>Ctrl</kbd>, maskingwith a lasso or rectangle, the background gestures, and the **Tool > Masking** controls for inverting, blurring, growing and building masks-from cavities or polygroups.+from cavities, polygroups or the mesh border.@@ -128,6 +128,10 @@ Open **Tool > Masking** (the Tool palette is in the right tray by default). Ever| **By cavity** | Mask concave areas (creases, folds, pores), replacing the current mask. | `mask_by_cavity` || **threshold** | How concave a spot must be for **By cavity** (-0.2 to 0.5, default 0.02). 0 masks anything concave; higher values keep only deep creases. | `threshold=` || **By polygroup** | Mask one polygroup, replacing the current mask. The number field next to it is the polygroup id. | `mask_by_polygroup` |+| **By border** | Mask the open edges of the mesh (edges with only one face) and the rings of edges inside them, replacing the current mask, so sculpting keeps the border in place. Uses **Mask Strength**. A closed mesh has no open edges, and the status bar says so. | `mask_border` |+| **Rings** | How wide **By border** masks, in rings of edges from the border (1 to 20, default 2; 1 masks the border vertices only). | `rings=` |+| **Polygroup borders** | **By border** also masks where polygroups meet (ZBrush: Mask by polygroup border). | `polygroups=true` |+| **Soft** | **By border** fades the mask from the border towards the innermost ring. | `soft=true` |Mask operations only change visible parts of the model; hidden parts keep their mask. **Clear** is theexception and frees everything.@@ -161,6 +165,18 @@ The quickest way works without knowing the group's number:group, then open the console (<kbd>`</kbd>) and read the `isolate_polygroup group=N` line. Enter that number inthe field next to **By polygroup** and click it.+### Sculpt without breaking the border++A plane, a cloth panel or a part cut from a larger model has an open border that should stay where it is, for+example to keep it lined up with the piece it joins:++1. Set **Rings** to how wide the protected strip should be.+2. Click **By border**. The open edges and the rings inside them are masked.+3. Sculpt freely; the border doesn't move. Tick **Soft** first for a gradual transition into the sculpted area.++Tick **Polygroup borders** to also protect the seams between polygroups. This is Blender's *Mask from mesh+boundary* (and, with face sets, *Mask from face set boundary*).+### A soft-edged maskPaint the mask, then click **Blur** a few times (or set **passes** to 3-5 first). **Sharpen** does the@@ -184,6 +200,7 @@ you can mask an area and fill the rest with a colour. See [Polypaint and materia| `mask_blur`, `mask_sharpen`, `mask_grow`, `mask_shrink` | `iterations` (1-100, default 1) || `mask_by_cavity` | `strength` (0-1, default 1), `threshold` (-1 to 1, default 0.02) || `mask_by_polygroup` | `group` (required), `add=true` to add to the current mask instead of replacing it |+| `mask_border` | `rings` (1-20, default 1), `polygroups=true` to also mask polygroup borders, `soft=true` to fade inwards, `strength` (0-1, default 1), `add=true` to add to the current mask || `mask_rect` | `x0 y0 x1 y1` (-1 to 1, y up), `unmask`, `strength` (0-1, default 1), optional `view_proj` matrix; without a view the rectangle spans the mesh seen from the front || `mask_lasso` | `points` (required): the outline as `"x0 y0 x1 y1 ..."`, at least 3 points in the same -1 to 1 space; `unmask`, `strength`, `view_proj` as for `mask_rect` |diff --git a/src/app/App.h b/src/app/App.hindex 295a80c1..8dfd4362 100644--- a/src/app/App.h+++ b/src/app/App.h@@ -457,6 +457,9 @@ private:int maskIterations_ = 1;float cavityThreshold_ = 0.02f;int maskGroup_ = 0;+ int maskBorderRings_ = 2; // Tool > Masking > By border: width in edge rings (#61)+ bool maskBorderGroups_ = false; // ... polygroup borders too+ bool maskBorderSoft_ = false; // ... fading inwardsfloat maskCanvasX_ = 0.0f, maskCanvasY_ = 0.0f;bool maskCanvasMoved_ = false;// Polygroups & visibility (PanelPolygroups.cpp): Ctrl+Shift click/drag.diff --git a/src/app/PanelMasking.cpp b/src/app/PanelMasking.cppindex 75462131..337e35bc 100644--- a/src/app/PanelMasking.cpp+++ b/src/app/PanelMasking.cpp@@ -211,6 +211,28 @@ void App::drawMaskingSection() {ImGui::SetNextItemWidth(-1);if (ImGui::DragInt("##maskgroup", &maskGroup_, 0.1f, 0, 1 << 30, "Group %d", ImGuiSliderFlags_AlwaysClamp))maskGroup_ = std::max(0, maskGroup_);++ // Mask from the mesh border (#61): open edges, optionally polygroup borders, a few rings wide.+ if (ImGui::Button("By border##mask", ImVec2(half, 0)))+ runCommand("mask_border", {{"rings", double(maskBorderRings_)},+ {"polygroups", maskBorderGroups_},+ {"soft", maskBorderSoft_},+ {"strength", double(maskStrength_)}});+ ui::HelpText("By border", nullptr,+ "Mask the open edges of the mesh and the edge rings inside them, so sculpting never moves the "+ "border (Blender: Mask from mesh boundary). Replaces the mask and uses the mask strength. A closed "+ "mesh has no open edges.",+ "Tool > Masking");+ ImGui::SameLine();+ ImGui::SetNextItemWidth(-1);+ tray::SliderInt("##maskborderrings", &maskBorderRings_, 1, 20, "Rings %d");+ ui::Checkbox("Polygroup borders##maskborder", &maskBorderGroups_);+ ui::HelpText("Polygroup borders", nullptr,+ "By border also masks where polygroups meet (ZBrush: Mask by polygroup border).", "Tool > Masking");+ ImGui::SameLine();+ ui::Checkbox("Soft##maskborder", &maskBorderSoft_);+ ui::HelpText("Soft", nullptr, "By border fades the mask from the border towards the innermost ring.",+ "Tool > Masking");}} // namespace sfdiff --git a/src/core/Masking.cpp b/src/core/Masking.cppindex 72857371..faabbf09 100644--- a/src/core/Masking.cpp+++ b/src/core/Masking.cpp@@ -11,6 +11,8 @@#include <cctype>#include <cmath>#include <functional>+#include <string>+#include <vector>namespace sf {@@ -180,6 +182,76 @@ bool maskByPolygroup(Mesh& mesh, uint32_t group, bool add) {});}+std::vector<uint8_t> borderVertices(const Mesh& mesh, bool polygroups) {+ const uint32_t n = static_cast<uint32_t>(mesh.vertexCount());+ std::vector<uint8_t> border(n, 0);+ poolFor(n, [&](size_t b, size_t e) {+ std::vector<uint32_t> corners;+ for (size_t i = b; i < e; ++i) {+ const uint32_t v = uint32_t(i);+ corners.clear();+ bool mixed = false, seenGroup = false;+ uint32_t group = 0;+ for (uint32_t t : mesh.vertexTriangles(v)) {+ const glm::uvec3& tri = mesh.triangles[t];+ for (int k = 0; k < 3; ++k)+ if (tri[k] != v) corners.push_back(tri[k]);+ if (!polygroups || mesh.triangleHidden(t)) continue;+ const uint32_t g = mesh.polygroupAt(t);+ if (seenGroup && g != group) mixed = true;+ group = g;+ seenGroup = true;+ }+ if (corners.empty()) continue; // a loose vertex is no border+ // Each edge v-u of a closed surface belongs to two of v's triangles, so u is listed twice; once is open.+ std::sort(corners.begin(), corners.end());+ bool open = false;+ for (size_t c = 0; c < corners.size() && !open;) {+ size_t d = c + 1;+ while (d < corners.size() && corners[d] == corners[c]) ++d;+ open = d - c == 1;+ c = d;+ }+ border[v] = (open || mixed) ? 1 : 0;+ }+ });+ return border;+}++bool maskBorder(Mesh& mesh, int rings, bool polygroups, bool soft, float strength, bool add, size_t* found) {+ const uint32_t n = static_cast<uint32_t>(mesh.vertexCount());+ if (found) *found = 0;+ if (n == 0) return false;+ rings = std::clamp(rings, 1, 20);+ strength = std::clamp(strength, 0.0f, 1.0f);+ // Ring index of each vertex from the border (0 = on it), out to rings - 1; -1 beyond.+ const std::vector<uint8_t> border = borderVertices(mesh, polygroups);+ std::vector<int8_t> ring(n, -1);+ std::vector<uint32_t> frontier, next;+ for (uint32_t v = 0; v < n; ++v)+ if (border[v]) {+ ring[v] = 0;+ frontier.push_back(v);+ }+ if (found) *found = frontier.size();+ if (frontier.empty()) return false;+ for (int r = 1; r < rings && !frontier.empty(); ++r) {+ next.clear();+ for (uint32_t v : frontier)+ for (uint32_t u : mesh.neighbors(v))+ if (ring[u] < 0) {+ ring[u] = int8_t(r);+ next.push_back(u);+ }+ frontier.swap(next);+ }+ return maskPass(mesh, [&](const std::vector<float>& m, uint32_t v) {+ float value = 0.0f;+ if (ring[v] >= 0) value = strength * (soft ? 1.0f - float(ring[v]) / float(rings) : 1.0f);+ return add ? std::max(m[v], value) : value;+ });+}+namespace {// Vertex v, or one of its mirror images, projects inside the shape.@@ -308,6 +380,37 @@ void registerMaskingCommands(CommandRegistry& registry) {return maskByPolygroup(m, uint32_t(std::max(0.0, argNumber(a, "group", 0))), argBool(a, "add", false));});+ registry.add(+ {"mask_border",+ "Mask from the mesh border (Blender: Mask from mesh boundary): mask the vertices on open edges - and, with "+ "polygroups, where polygroups meet - plus edge rings inside them, so sculpting keeps the border in place. "+ "Replaces the mask unless add is set. A closed mesh has no open edges and masks nothing.",+ {{"rings", T::Number, "Width of the mask in edge rings from the border (1-20; 1 = the border vertices only). "+ "Default 1.",+ false},+ {"polygroups", T::Bool, "Also mask polygroup borders (ZBrush: Mask by polygroup border). Default false.",+ false},+ {"soft", T::Bool, "Fade the mask from the border towards the innermost ring. Default false.", false},+ {"strength", T::Number, "Mask value on the border (0-1). Default 1.", false},+ {"add", T::Bool, "Add to the current mask instead of replacing it. Default false.", false}},+ [](Document& doc, const Args& a) -> CommandResult {+ if (doc.mesh.vertexCount() == 0) return {false, "The mesh is empty"};+ const std::string label = "Mask Border";+ const bool polygroups = argBool(a, "polygroups", false);+ UndoRecord record = UndoRecord::captureMask(doc.mesh, label);+ size_t found = 0;+ const bool changed =+ maskBorder(doc.mesh, int(argNumber(a, "rings", 1)), polygroups, argBool(a, "soft", false),+ float(argNumber(a, "strength", 1.0)), argBool(a, "add", false), &found);+ if (found == 0)+ return {true, polygroups ? "No open edges or polygroup borders: the mesh is closed and one polygroup"+ : "No open edges: the mesh is closed"};+ if (!changed) return {true, label + ": mask unchanged"};+ doc.undo.push(std::move(record));+ doc.modified = true;+ return {true, label + ": " + std::to_string(found) + " border vertices"};+ }});+// MaskRect / MaskLasso. Bad arguments are errors, unlike the "mask unchanged" of a selection that misses.const ParamSpec unmask{"unmask", T::Bool, "Unmask the inside instead (ZBrush: Alt held). Default false.", false};const ParamSpec strength{"strength", T::Number, "How much to mask or unmask (0-1). Default 1.", false};diff --git a/src/core/Masking.h b/src/core/Masking.hindex 8102d754..959fe3da 100644--- a/src/core/Masking.h+++ b/src/core/Masking.h@@ -43,6 +43,16 @@ bool maskByCavity(Mesh& mesh, float strength, float threshold);// With `add` the existing mask is kept, otherwise it is replaced.bool maskByPolygroup(Mesh& mesh, uint32_t group, bool add);+// Vertices on an open edge (an edge of only one triangle) and, with `polygroups`, where the visible triangles+// around a vertex have more than one polygroup: 1 for those, 0 elsewhere.+std::vector<uint8_t> borderVertices(const Mesh& mesh, bool polygroups);+// Mask from the mesh border (#61; Blender: Mask from mesh boundary, ZBrush: Mask by polygroup border): the+// border vertices and `rings - 1` edge rings inside them (rings 1-20) get `strength` (0-1) of mask, fading+// towards the innermost ring with `soft`. `polygroups` also masks polygroup borders. With `add` the mask+// grows (each vertex keeps the larger value), otherwise it is replaced. `found` gets the number of border+// vertices: 0 on a closed mesh (nothing to mask).+bool maskBorder(Mesh& mesh, int rings, bool polygroups, bool soft, float strength, bool add, size_t* found = nullptr);+// Mask selections (ZBrush MaskLasso / MaskRect): visible vertices that// project inside `shape` (NDC under `viewProj`, see ScreenSelect.h) gain// `strength` (0-1) of mask, or lose it with `unmask`. Vertices behind the@@ -62,8 +72,8 @@ bool shapeCoversMesh(const Mesh& mesh, const glm::mat4& viewProj, const ScreenPobool symmetryMirrorsArg(const Args& a, std::vector<glm::vec3>& mirrors, std::string& error);// Registers mask_clear, mask_all, mask_invert, mask_blur, mask_sharpen,-// mask_grow, mask_shrink, mask_by_cavity, mask_by_polygroup, mask_rect and-// mask_lasso.+// mask_grow, mask_shrink, mask_by_cavity, mask_by_polygroup, mask_border,+// mask_rect and mask_lasso.void registerMaskingCommands(CommandRegistry& registry);} // namespace sfdiff --git a/tests/test_masking.cpp b/tests/test_masking.cppindex 1f012583..b8932b33 100644--- a/tests/test_masking.cpp+++ b/tests/test_masking.cpp@@ -132,7 +132,7 @@ TEST(painted_mask_protects_from_brushes) {TEST(mask_commands_registered) {const std::string schema = registry().toolSchemaJson();for (const char* name : {"mask_clear", "mask_all", "mask_invert", "mask_blur", "mask_sharpen", "mask_grow",- "mask_shrink", "mask_by_cavity", "mask_by_polygroup", "mask_rect", "mask_lasso"}) {+ "mask_shrink", "mask_by_cavity", "mask_by_polygroup", "mask_border", "mask_rect", "mask_lasso"}) {CHECK(registry().find(name) != nullptr);CHECK(schema.find(std::string("\"") + name + "\"") != std::string::npos);}@@ -401,3 +401,81 @@ TEST(mask_rect_and_lasso_symmetry_and_clear_outside) {CHECK(registry().execute(h, "mask_rect", {{"x0", -1.0}, {"y0", -1.0}, {"x1", -0.6}, {"y1", 1.0}, {"clear_outside", true}}).ok);CHECK(countMasked(h.mesh, 0.0f) == 0);}++// #61: Mask from the mesh border.+TEST(mask_border_masks_plane_rim_and_rings_widen_it) {+ Document d = makeDoc(makePlane(8)); // 9 x 9 vertices: a rim of 32, then rings of 24, 16, 8 and the centre+ auto onRim = [](const glm::vec3& p) { return std::max(std::abs(p.x), std::abs(p.z)) > 0.99f; };+ CommandResult r = registry().execute(d, "mask_border");+ CHECK(r.ok);+ CHECK(r.message.find("32 border vertices") != std::string::npos);+ CHECK(countMasked(d.mesh) == 32);+ bool rimOnly = true;+ for (uint32_t v = 0; v < d.mesh.vertexCount(); ++v) rimOnly &= (d.mesh.mask[v] == 1.0f) == onRim(d.mesh.positions[v]);+ CHECK(rimOnly);+ CHECK(d.undo.canUndo());++ // Two rings: the rim and the ring inside it (replacing the mask), then three soft rings.+ CHECK(registry().execute(d, "mask_border", {{"rings", 2.0}}).ok);+ CHECK(countMasked(d.mesh) == 56);+ CHECK(registry().execute(d, "mask_border", {{"rings", 3.0}, {"soft", true}}).ok);+ CHECK(countMasked(d.mesh, 0.0f) == 56 + 16);+ float rim = 0.0f, inner = 1.0f;+ for (uint32_t v = 0; v < d.mesh.vertexCount(); ++v) {+ if (onRim(d.mesh.positions[v])) rim = std::max(rim, d.mesh.mask[v]);+ else if (d.mesh.mask[v] > 0.0f) inner = std::min(inner, d.mesh.mask[v]);+ }+ CHECK(rim == 1.0f && inner < 0.5f); // full on the border, fading inwards++ // Each is one undo step, back to no mask.+ CHECK(registry().execute(d, "undo").ok);+ CHECK(countMasked(d.mesh, 0.0f) == 56 && countMasked(d.mesh, 0.999f) == 56);+ CHECK(registry().execute(d, "undo").ok);+ CHECK(countMasked(d.mesh) == 32);+ CHECK(registry().execute(d, "undo").ok);+ CHECK(countMasked(d.mesh, 0.0f) == 0);++ // Add keeps the current mask; strength sets the value.+ maskHalf(d.mesh, 0.5f);+ const size_t half = countMasked(d.mesh);+ CHECK(registry().execute(d, "mask_border", {{"add", true}, {"strength", 0.25}}).ok);+ CHECK(countMasked(d.mesh) == half);+ CHECK(countMasked(d.mesh, 0.0f) > half);+ // Ring counts are clamped (to 20), which fills this small plane.+ CHECK(registry().execute(d, "mask_border", {{"rings", 1000.0}}).ok);+ CHECK(countMasked(d.mesh) == d.mesh.vertexCount());+}++TEST(mask_border_closed_mesh_masks_nothing) {+ Document d = makeDoc(makeIcosphere(3));+ CommandResult r = registry().execute(d, "mask_border", {{"rings", 3.0}});+ CHECK(r.ok);+ CHECK(r.message.find("No open edges: the mesh is closed") != std::string::npos);+ CHECK(countMasked(d.mesh, 0.0f) == 0);+ CHECK(!d.undo.canUndo());+ // Nor are there polygroup borders on a mesh of one polygroup.+ r = registry().execute(d, "mask_border", {{"polygroups", true}});+ CHECK(r.message.find("No open edges") != std::string::npos);+ CHECK(!d.undo.canUndo());+}++TEST(mask_border_polygroup_borders) {+ Document d = makeDoc(makeIcosphere(3));+ Mesh& m = d.mesh;+ m.polygroups.assign(m.triangleCount(), 0);+ for (uint32_t t = 0; t < m.triangleCount(); ++t) {+ const glm::uvec3 tri = m.triangles[t];+ m.polygroups[t] = m.positions[tri.x].y + m.positions[tri.y].y + m.positions[tri.z].y > 0.0f ? 1u : 0u;+ }+ CHECK(registry().execute(d, "mask_border", {{"polygroups", true}}).ok);+ const size_t seam = countMasked(m);+ CHECK(seam > 0 && seam < m.vertexCount() / 4);+ bool nearEquator = true;+ for (uint32_t v = 0; v < m.vertexCount(); ++v)+ if (m.mask[v] > 0.5f) nearEquator &= std::abs(m.positions[v].y) < 0.25f;+ CHECK(nearEquator);+ CHECK(registry().execute(d, "mask_border", {{"polygroups", true}, {"rings", 3.0}}).ok);+ CHECK(countMasked(m) > seam);+ // Without polygroups the closed sphere still has no border to mask.+ CHECK(registry().execute(d, "mask_border").message.find("No open edges") != std::string::npos);+}diff --git a/tests/ui/mask_border.sfui b/tests/ui/mask_border.sfuinew file mode 100644index 00000000..021b343f--- /dev/null+++ b/tests/ui/mask_border.sfui@@ -0,0 +1,43 @@+# #61: Tool > Masking > By border masks the open edges of the mesh (Blender: Mask from mesh boundary), so+# sculpting keeps the border in place. Rings widens it; a closed mesh says it has no open edges.+run new_plane resolution=32+expect_value open_edges == 128+expect_value masked == 0+camera 0 60+key f++click "Masking"+expect "By border##mask"+hold ctrl+hover "By border##mask"+expect_help "open edges"+hold none++# Default 2 rings: the rim of 128 vertices and the 120 inside it; one undo step.+click "By border##mask"+expect_status 128 border vertices+expect_value masked == 248+expect_value can_undo == 1+shot border_masked++# A stroke across the whole plane moves the inside but not the masked border.+snapshot+stroke -0.1 0.5 1.1 0.5+expect_value moved > 0+expect_value moved_masked == 0+key ctrl+z+key ctrl+z+expect_value masked == 0++# Soft fades the mask inwards.+click "Soft##maskborder"+click "By border##mask"+expect_value masked >= 128+expect_value mask_soft > 0+key ctrl+z++# A closed sphere has no open edges: nothing is masked and the status says why.+run new_sphere subdivisions=3+click "By border##mask"+expect_status No open edges: the mesh is closed+expect_value masked == 0