Free, open-source 3D sculpting for Linux, macOS and Windows: dense meshes, remeshing, painting, built-in AI (image/text to 3D), Send to Unreal, game-ready LODs, Gaussian splats and OpenUSD.
C++ 3d3d-sculptingcppdear-imguidigital-sculptingfbxgame-developmentgaussian-splattinggltfimage-to-3dmcpopenglopenusdsculptingtext-to-3dunreal-engine
Image to 3D: Remove flat background (on by default) drops the plane a picture's background becomes
islandmed084e7408d61c
diff --git a/CMakeLists.txt b/CMakeLists.txtindex 09f7b670..0a43d309 100644--- a/CMakeLists.txt+++ b/CMakeLists.txt@@ -141,6 +141,7 @@ add_library(sculptforge_core STATICsrc/core/SurfaceQuery.cppsrc/core/QRemesh.cppsrc/core/GeneratedRemesh.cpp+ src/core/Backdrop.cppsrc/core/QRemeshGuides.cppsrc/core/QRemesh2.cppsrc/core/QRemesh2Proxy.cppdiff --git a/changes/feat-img3d-drop-backdrop.md b/changes/feat-img3d-drop-backdrop.mdnew file mode 100644index 00000000..0c8ff9eb--- /dev/null+++ b/changes/feat-img3d-drop-backdrop.md@@ -0,0 +1,15 @@+---+whats_new: **Remove flat background**: Image to 3D drops the flat plane a picture's background often turns into+ (a wall behind or a ground under the subject) before the model lands. On by default in **Generate 3D**; Undo+ brings a removed plane back. Thanks @fenix.+added: **Generate 3D > Remove flat background** (on by default): a thin wall behind or ground under the subject,+ covering most of the model's bounds, is removed from the generated mesh before remeshing, as its own undo step;+ a relief or a coin on its own is kept. `image_to_3d` / `text_to_3d` take `remove_background=false`, and the new+ `remove_flat_background` command does it on the active Part. Requested by @fenix.+testers:+ - Any OS: Generate 3D from a photo with a plain wall behind the subject (Hunyuan3D 2.1); the result has no flat+ plate behind it and the status says "Removed a flat background plane". Ctrl+Z (twice with Quad Remesh) brings it back.+ - Turn **Remove flat background** off and generate again: the plane stays.+ - Tool > Relief from an emblem image, then run `remove_flat_background` in the console: "No flat background plane found".+---+Core: src/core/Backdrop.{h,cpp}. Tests: tests/test_backdrop.cpp (wall, ground, joined ground, relief, coin, plane alone).diff --git a/docs/guide/generate-3d.md b/docs/guide/generate-3d.mdindex 4861de9b..f82a8bf8 100644--- a/docs/guide/generate-3d.md+++ b/docs/guide/generate-3d.md@@ -45,7 +45,11 @@ The window has two tabs, **From image** and **From text**.<kbd>Shift</kbd>+<kbd>D</kbd>.- **Voxel remesh**: one even, watertight triangle surface. Faster (about a second).- **Off (raw mesh)**: the model exactly as the service made it.-5. Click **Generate 3D model**. A progress bar shows the queue position and progress; **Cancel** stops the+5. Leave **Remove flat background** on (the default) to drop the flat plane a picture's background often+ becomes: a thin wall behind or a ground under the subject that covers most of the model's bounds. A relief+ or a coin on its own is kept. The status line says when a plane was removed; Undo brings it back (with Remesh+ on, the first Undo steps back to the raw mesh, the second brings the plane back).+6. Click **Generate 3D model**. A progress bar shows the queue position and progress; **Cancel** stops thejob.When it finishes, the status line reports the size before and after remeshing, for example@@ -246,6 +250,9 @@ sculptforge-cli -c 'image_to_3d path=hero.png; export_glb path=hero.glb'# TRELLIS.2, best detail, with a free Hugging Face token (without one, Hunyuan3D 2.1 runs and says so)HF_TOKEN=hf_... sculptforge-cli -c 'image_to_3d path=hero.png model=trellis-2; export_glb path=hero.glb'+# keep a flat background plane the service made (removed by default; remove_flat_background does it later)+sculptforge-cli -c 'image_to_3d path=hero.png remove_background=false; export_glb path=hero.glb'+# voxel remesh instead of the default Quad Remeshsculptforge-cli -c 'image_to_3d path=hero.png remesh=voxel; export_glb path=hero.glb'diff --git a/src/ai/AiCommands.cpp b/src/ai/AiCommands.cppindex bdfffb9a..fa2ea32e 100644--- a/src/ai/AiCommands.cpp+++ b/src/ai/AiCommands.cpp@@ -8,6 +8,7 @@#include "ai/ComfyUI.h"#include "ai/ImageTo3DService.h"#include "core/EnginePresets.h"+#include "core/Backdrop.h"#include "core/GeneratedRemesh.h"#include "core/Texture.h"#include "core/Voxel.h"@@ -50,6 +51,8 @@ CommandResult importGenerated(Document& doc, const Args& a, const GeneratorResulif (!r.privateDir.empty()) std::filesystem::remove_all(std::filesystem::path(std::u8string(r.privateDir.begin(), r.privateDir.end())), ec);if (!opened.ok) return opened;doc.path.clear();+ BackdropRemoval backdrop;+ if (argBool(a, "remove_background", true)) removeFlatBackdrop(doc.mesh, &backdrop);GeneratedRemeshStats st;if (mode != GeneratedRemesh::Off) {std::string err;@@ -64,6 +67,7 @@ CommandResult importGenerated(Document& doc, const Args& a, const GeneratorResulconst std::string note = generatedRemeshNote(st, mode);return {true, "Generated a 3D model" + what + " (" + std::to_string(st.rawTriangles) + " triangles" +(note.empty() ? std::string() : ", " + note) + ") - ready to sculpt" ++ (backdrop.planes ? ". " + backdropRemovedNote(backdrop) : std::string()) +(r.note.empty() ? std::string() : " " + r.note)};}@@ -130,7 +134,9 @@ void addAiFeatureCommands(CommandRegistry& registry, std::function<AiSettings()>{"resolution", T::Number, "Remesh detail (64-512): voxels across the model; quad picks about ""resolution^2/2 quads (256: 32768). Default 256.", false},{"levels", T::Number, "quad: subdivision levels above the quad cage (0-2), projected onto the "- "model. Default 1.", false}},+ "model. Default 1.", false},+ {"remove_background", T::Bool, "Drop a flat plane the background became (a wall behind or ground "+ "under the subject; a relief or coin alone is kept). Default true.", false}},[settings](Document& doc, const Args& a) -> CommandResult {AiSettings s = settings();if (a.contains("service")) s.generatorService = argString(a, "service");@@ -173,7 +179,9 @@ void addAiFeatureCommands(CommandRegistry& registry, std::function<AiSettings()>{"resolution", T::Number, "Remesh detail (64-512): voxels across the model; quad picks about ""resolution^2/2 quads (256: 32768). Default 256.", false},{"levels", T::Number, "quad: subdivision levels above the quad cage (0-2), projected onto the "- "model. Default 1.", false}},+ "model. Default 1.", false},+ {"remove_background", T::Bool, "Drop a flat plane the background became (a wall behind or ground "+ "under the subject; a relief or coin alone is kept). Default true.", false}},[settings](Document& doc, const Args& a) -> CommandResult {AiSettings s = settings();if (a.contains("service")) s.generatorService = argString(a, "service");diff --git a/src/app/App.cpp b/src/app/App.cppindex 765f9f38..79c12aeb 100644--- a/src/app/App.cpp+++ b/src/app/App.cpp@@ -3735,24 +3735,30 @@ void App::startGenerator() {pendingGenerator_ = promise->get_future();std::thread([promise, settings, in = refImagePath_, text = textMode_ ? std::string(textPrompt_) : std::string(),out = std::string(stem.begin(), stem.end()), progress = generatorProgress_,- mode = GeneratedRemesh(generatedRemesh_), resolution = remeshResolution_] {+ mode = GeneratedRemesh(generatedRemesh_), resolution = remeshResolution_, dropBackdrop = removeBackground_] {GeneratedModel g;g.result = text.empty() ? ai::generate3D(settings, in, out, progress.get()): ai::generate3DFromText(settings, text, out, progress.get());- if (g.result.ok && mode != GeneratedRemesh::Off) {+ if (g.result.ok && (mode != GeneratedRemesh::Off || dropBackdrop)) {// Made sculptable here, so the window stays responsive; the main thread opens the file and swaps this in.Document scratch;const CommandResult opened = CommandRegistry().execute(scratch, "open", {{"path", g.result.outputPath}});- progress->percent = 0;- progress->setStatus(mode == GeneratedRemesh::Quad ? "Making it sculptable: Quad Remesh..." : "Making it sculptable: Voxel Remesh...");+ // The background plane goes first (one undo step of its own), so the remesh doesn't build it in.+ if (opened.ok && dropBackdrop && removeFlatBackdrop(scratch.mesh, &g.backdrop))+ g.withoutBackdrop = std::make_shared<Mesh>(scratch.mesh);Mesh m;- if (!opened.ok) {+ if (mode == GeneratedRemesh::Off) {+ // (opened for the background check only)+ } else if (!opened.ok) {g.remeshError = opened.message;- } else if (remeshGenerated(scratch.mesh, mode, resolution, 1, m, g.remeshError, &g.stats, [progress](float f) {- progress->percent = int(f * 100.0f);- return !progress->cancel.load();- })) {- g.remeshed = std::make_shared<Mesh>(std::move(m));+ } else {+ progress->percent = 0;+ progress->setStatus(mode == GeneratedRemesh::Quad ? "Making it sculptable: Quad Remesh..." : "Making it sculptable: Voxel Remesh...");+ if (remeshGenerated(scratch.mesh, mode, resolution, 1, m, g.remeshError, &g.stats, [progress](float f) {+ progress->percent = int(f * 100.0f);+ return !progress->cancel.load();+ }))+ g.remeshed = std::make_shared<Mesh>(std::move(m));}}promise->set_value(std::move(g));@@ -3781,12 +3787,14 @@ void App::pollGenerator() {std::filesystem::remove(std::filesystem::path(std::u8string(r.outputPath.begin(), r.outputPath.end())), ec);const size_t raw = doc_.mesh.triangleCount();const auto mode = GeneratedRemesh(generatedRemesh_);+ if (g.withoutBackdrop) doc_.replaceMeshUndoable(std::move(*g.withoutBackdrop), "Remove Flat Background");if (g.remeshed) doc_.replaceMeshUndoable(std::move(*g.remeshed), mode == GeneratedRemesh::Quad ? "Quad Remesh" : "Remesh");doc_.path.clear();const std::string remeshNote = g.remeshed ? ", " + generatedRemeshNote(g.stats, mode) : std::string();status_ = !g.remeshError.empty()? "Generated model imported (" + std::to_string(raw) + " tris, not remeshed: " + g.remeshError + ")" + note: "Generated model imported (" + std::to_string(raw) + " tris" + remeshNote + ") - ready to sculpt" + note;+ if (g.backdrop.planes) status_ += ". " + backdropRemovedNote(g.backdrop) + ", Undo brings it back";updateTitle();}@@ -4083,6 +4091,11 @@ void App::drawImageTo3D() {generatedQuadTarget(remeshResolution_));hint(quads);}+ ImGui::Checkbox(tr("Remove flat background"), &removeBackground_);+ if (ImGui::IsItemHovered(ImGuiHoveredFlags_DelayNormal))+ ImGui::SetTooltip(tr("Drops the flat plane a picture's background often becomes (a wall behind or ground under "+ "the subject) before the model lands. A relief or coin on its own is kept. Undo brings "+ "a removed plane back."));}ImGui::PopItemWidth();if (pendingGenerator_.valid()) {diff --git a/src/app/App.h b/src/app/App.hindex b4c35602..67faebfd 100644--- a/src/app/App.h+++ b/src/app/App.h@@ -32,6 +32,7 @@#include "core/Document.h"#include "core/DiscordPresence.h"#include "core/Face.h"+#include "core/Backdrop.h"#include "core/GeneratedRemesh.h"#include "core/MeshTransform.h"#include "app/Wintab.h"@@ -684,10 +685,13 @@ private:"If the mask's edge is too ragged to join up, the status line says so and nothing changes.";bool keepTexture_ = true; // Keep Texture: remeshes carry a painted texture over (core/TextureTransfer.h)int remeshResolution_ = 256;+ bool removeBackground_ = true; // Generate 3D's "Remove flat background" (core/Backdrop.h)// A generated model, made sculptable on the worker thread (Quad Remesh takes seconds).struct GeneratedModel {ai::GeneratorResult result;std::shared_ptr<Mesh> remeshed; // null: Remesh off, or it failed (remeshError)+ std::shared_ptr<Mesh> withoutBackdrop; // the raw mesh minus a flat background plane (null: none found or off)+ BackdropRemoval backdrop;GeneratedRemeshStats stats;std::string remeshError;};diff --git a/src/app/I18nSimplifiedChinese.cpp b/src/app/I18nSimplifiedChinese.cppindex eafaf004..be17b7db 100644--- a/src/app/I18nSimplifiedChinese.cpp+++ b/src/app/I18nSimplifiedChinese.cpp@@ -4208,6 +4208,10 @@ constexpr Entry kSimplifiedChinese[] = {"以 AR 查看"},{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.","视图分辨率 75%%:约只绘制一半像素,适合集成显卡和大屏幕。"},+ {"Remove flat background",+ "移除平坦背景"},+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",+ "在模型载入前,移除图片背景常变成的平面(主体后方的墙或下方的地面)。单独的浮雕或硬币会保留。撤销可找回被移除的平面。"},{"Voxel remesh","体素重新网格化"},{"Welcome to SculptForge",diff --git a/src/app/I18nSpanish.cpp b/src/app/I18nSpanish.cppindex 8a63ea2a..eda05ff7 100644--- a/src/app/I18nSpanish.cpp+++ b/src/app/I18nSpanish.cpp@@ -4645,6 +4645,10 @@ constexpr Entry kSpanish[] = {"Ver en RA"},{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.","Resolución de vista 75%%: dibuja más o menos la mitad de los píxeles, para gráficos integrados y pantallas grandes."},+ {"Remove flat background",+ "Quitar el fondo plano"},+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",+ "Quita el plano en que suele convertirse el fondo de una imagen (una pared detrás o un suelo bajo el sujeto) antes de que llegue el modelo. Un relieve o una moneda solos se conservan. Deshacer recupera un plano quitado."},{"Voxel remesh","Remallado vóxel"},{"Welcome to SculptForge",diff --git a/src/app/I18nTraditionalChinese.cpp b/src/app/I18nTraditionalChinese.cppindex 6fc3b2fa..bc3ef0d1 100644--- a/src/app/I18nTraditionalChinese.cpp+++ b/src/app/I18nTraditionalChinese.cpp@@ -4206,6 +4206,10 @@ constexpr Entry kTraditionalChinese[] = {"以 AR 檢視"},{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.","檢視解析度 75%%:約只繪製一半像素,適合內建顯示晶片與大螢幕。"},+ {"Remove flat background",+ "移除平坦背景"},+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",+ "在模型載入前,移除圖片背景常變成的平面(主體後方的牆或下方的地面)。單獨的浮雕或硬幣會保留。復原可找回被移除的平面。"},{"Voxel remesh","體素重新網格化"},{"Welcome to SculptForge",diff --git a/src/core/Backdrop.cpp b/src/core/Backdrop.cppnew file mode 100644index 00000000..a435d430--- /dev/null+++ b/src/core/Backdrop.cpp@@ -0,0 +1,349 @@+// SculptForge - open source digital sculpting+// SPDX-License-Identifier: GPL-3.0-or-later+#include "core/Backdrop.h"++#include "core/Commands.h"+#include "core/Document.h"+#include "core/FaceGroups.h"++#include <algorithm>+#include <array>+#include <cmath>+#include <limits>+#include <numeric>+#include <unordered_map>++namespace sf {+namespace {++constexpr float kMaxThickness = 0.025f; // of the plane's width+constexpr float kMinCover = 0.7f; // of the model's extent, along both plane directions+constexpr float kMaxRestArea = 0.8f; // the rest's footprint, of the plane's+constexpr float kMinStandOut = 0.05f; // how far the rest stands clear of the plane, of its width+constexpr float kMinOver = 0.5f; // share of the rest's vertices over the plane (not over a hole in it)+constexpr int kGrid = 64;++struct UnionFind {+ std::vector<uint32_t> parent;+ explicit UnionFind(size_t n) : parent(n) { std::iota(parent.begin(), parent.end(), 0u); }+ uint32_t find(uint32_t x) {+ while (parent[x] != x) x = parent[x] = parent[parent[x]];+ return x;+ }+ void join(uint32_t a, uint32_t b) { parent[find(a)] = find(b); }+};++// Vertices at the same place share an id (generated files often split them along UV seams).+std::vector<uint32_t> weldedIds(const Mesh& mesh, float cell) {+ struct Key {+ int64_t x, y, z;+ bool operator==(const Key& o) const { return x == o.x && y == o.y && z == o.z; }+ };+ struct Hash {+ size_t operator()(const Key& k) const {+ return size_t(k.x * 73856093LL) ^ size_t(k.y * 19349663LL) ^ size_t(k.z * 83492791LL);+ }+ };+ std::unordered_map<Key, uint32_t, Hash> ids;+ ids.reserve(mesh.positions.size());+ std::vector<uint32_t> id(mesh.positions.size());+ for (size_t v = 0; v < id.size(); ++v) {+ const glm::vec3 q = glm::floor(mesh.positions[v] / cell + 0.5f);+ id[v] = ids.try_emplace(Key{int64_t(q.x), int64_t(q.y), int64_t(q.z)}, uint32_t(ids.size())).first->second;+ }+ return id;+}++// Groups of the triangles with in[t] != 0 joined through shared (welded) corners, largest area first.+std::vector<std::vector<uint32_t>> groups(const Mesh& mesh, const std::vector<uint32_t>& weld,+ const std::vector<uint8_t>& in) {+ UnionFind uf(mesh.positions.size());+ std::vector<uint32_t> first(mesh.positions.size(), ~0u);+ for (uint32_t v = 0; v < weld.size(); ++v) {+ uint32_t& f = first[weld[v]];+ if (f == ~0u) f = v;+ else uf.join(v, f);+ }+ for (size_t t = 0; t < mesh.triangles.size(); ++t) {+ if (!in[t]) continue;+ const glm::uvec3& tri = mesh.triangles[t];+ uf.join(tri.x, tri.y);+ uf.join(tri.x, tri.z);+ }+ std::unordered_map<uint32_t, size_t> index;+ std::vector<std::vector<uint32_t>> out;+ std::vector<float> area;+ for (uint32_t t = 0; t < mesh.triangles.size(); ++t) {+ if (!in[t]) continue;+ const glm::uvec3& tri = mesh.triangles[t];+ const auto [it, added] = index.try_emplace(uf.find(tri.x), out.size());+ if (added) {+ out.emplace_back();+ area.push_back(0.0f);+ }+ out[it->second].push_back(t);+ area[it->second] += glm::length(glm::cross(mesh.positions[tri.y] - mesh.positions[tri.x],+ mesh.positions[tri.z] - mesh.positions[tri.x]));+ }+ std::vector<size_t> order(out.size());+ std::iota(order.begin(), order.end(), size_t(0));+ std::sort(order.begin(), order.end(), [&](size_t a, size_t b) { return area[a] > area[b]; });+ std::vector<std::vector<uint32_t>> sorted;+ sorted.reserve(out.size());+ for (size_t i : order) sorted.push_back(std::move(out[i]));+ return sorted;+}++// The direction a set of points varies least along (cyclic Jacobi on their covariance).+glm::vec3 flattestDirection(const Mesh& mesh, const std::vector<uint32_t>& tris) {+ glm::dvec3 mean(0.0);+ size_t n = 0;+ for (uint32_t t : tris)+ for (int c = 0; c < 3; ++c, ++n) mean += glm::dvec3(mesh.positions[mesh.triangles[t][c]]);+ mean /= double(std::max<size_t>(n, 1));+ double a[3][3] = {};+ for (uint32_t t : tris)+ for (int c = 0; c < 3; ++c) {+ const glm::dvec3 d = glm::dvec3(mesh.positions[mesh.triangles[t][c]]) - mean;+ for (int i = 0; i < 3; ++i)+ for (int j = 0; j < 3; ++j) a[i][j] += d[i] * d[j];+ }+ double v[3][3] = {{1, 0, 0}, {0, 1, 0}, {0, 0, 1}};+ for (int sweep = 0; sweep < 32; ++sweep) {+ const double off = a[0][1] * a[0][1] + a[0][2] * a[0][2] + a[1][2] * a[1][2];+ if (off < 1e-24) break;+ for (int p = 0; p < 2; ++p)+ for (int q = p + 1; q < 3; ++q) {+ if (std::abs(a[p][q]) < 1e-30) continue;+ const double theta = (a[q][q] - a[p][p]) / (2.0 * a[p][q]);+ const double t = (theta >= 0 ? 1.0 : -1.0) / (std::abs(theta) + std::sqrt(theta * theta + 1.0));+ const double c = 1.0 / std::sqrt(t * t + 1.0), s = t * c;+ for (int k = 0; k < 3; ++k) { // A = J^T A J+ const double akp = a[k][p], akq = a[k][q];+ a[k][p] = c * akp - s * akq;+ a[k][q] = s * akp + c * akq;+ }+ for (int k = 0; k < 3; ++k) {+ const double apk = a[p][k], aqk = a[q][k];+ a[p][k] = c * apk - s * aqk;+ a[q][k] = s * apk + c * aqk;+ }+ for (int k = 0; k < 3; ++k) {+ const double vkp = v[k][p], vkq = v[k][q];+ v[k][p] = c * vkp - s * vkq;+ v[k][q] = s * vkp + c * vkq;+ }+ }+ }+ int least = 0;+ for (int i = 1; i < 3; ++i)+ if (a[i][i] < a[least][least]) least = i;+ return glm::normalize(glm::vec3(float(v[0][least]), float(v[1][least]), float(v[2][least])));+}++struct Range {+ float lo = std::numeric_limits<float>::max(), hi = std::numeric_limits<float>::lowest();+ void add(float x) {+ lo = std::min(lo, x);+ hi = std::max(hi, x);+ }+ float size() const { return hi > lo ? hi - lo : 0.0f; }+};++// Whether the triangles in `plane` (normal `n`) are a flat backdrop to everything else in `mesh`; `area` = theirs.+bool isBackdrop(const Mesh& mesh, const std::vector<uint8_t>& removed, const std::vector<uint32_t>& plane, glm::vec3 n,+ float& area) {+ const glm::vec3 u = glm::normalize(std::abs(n.x) < 0.9f ? glm::cross(n, glm::vec3(1, 0, 0)) : glm::cross(n, glm::vec3(0, 1, 0)));+ const glm::vec3 w = glm::cross(n, u);+ std::vector<uint8_t> inPlane(mesh.triangles.size(), 0);+ for (uint32_t t : plane) inPlane[t] = 1;+ // Vertices: 1 = of the plane, 2 = of the rest (kept or not yet removed), 3 = both.+ std::vector<uint8_t> role(mesh.positions.size(), 0);+ for (size_t t = 0; t < mesh.triangles.size(); ++t) {+ if (removed[t]) continue;+ for (int c = 0; c < 3; ++c) role[mesh.triangles[t][c]] |= inPlane[t] ? 1 : 2;+ }+ Range pn, pu, pw, rn, ru, rw, tu, tw;+ size_t restCount = 0;+ for (size_t v = 0; v < role.size(); ++v) {+ if (!role[v]) continue;+ const glm::vec3& p = mesh.positions[v];+ const float dn = glm::dot(p, n), du = glm::dot(p, u), dw = glm::dot(p, w);+ tu.add(du);+ tw.add(dw);+ if (role[v] & 1) {+ pn.add(dn);+ pu.add(du);+ pw.add(dw);+ }+ if (role[v] == 2) {+ rn.add(dn);+ ru.add(du);+ rw.add(dw);+ ++restCount;+ }+ }+ const float width = std::max(pu.size(), pw.size());+ if (restCount == 0 || width <= 0.0f) return false; // never the only part+ if (pn.size() > kMaxThickness * width) return false;+ if (pu.size() < kMinCover * tu.size() || pw.size() < kMinCover * tw.size()) return false;+ if (ru.size() * rw.size() > kMaxRestArea * pu.size() * pw.size()) return false;+ const float mid = 0.5f * (pn.lo + pn.hi);+ const float below = std::max(0.0f, mid - rn.lo), above = std::max(0.0f, rn.hi - mid);+ if (std::min(below, above) > 0.1f * std::max(below, above)) return false; // through the middle, not behind+ if (std::max(below, above) < std::max(4.0f * pn.size(), kMinStandOut * width)) return false;++ // Over the plane, not in its holes (a relief's raised parts sit where its flat field isn't).+ std::vector<uint8_t> cells(size_t(kGrid) * kGrid, 0);+ const float su = (kGrid - 1e-3f) / std::max(pu.size(), 1e-12f), sw = (kGrid - 1e-3f) / std::max(pw.size(), 1e-12f);+ auto cellOf = [&](float du, float dw, int& i, int& j) {+ i = int(std::floor((du - pu.lo) * su));+ j = int(std::floor((dw - pw.lo) * sw));+ return i >= 0 && j >= 0 && i < kGrid && j < kGrid;+ };+ for (uint32_t t : plane) {+ glm::vec2 q[3];+ for (int c = 0; c < 3; ++c) {+ const glm::vec3& p = mesh.positions[mesh.triangles[t][c]];+ q[c] = {(glm::dot(p, u) - pu.lo) * su, (glm::dot(p, w) - pw.lo) * sw};+ int i, j;+ if (cellOf(glm::dot(p, u), glm::dot(p, w), i, j)) cells[size_t(j) * kGrid + size_t(i)] = 1;+ }+ const float d = (q[1].x - q[0].x) * (q[2].y - q[0].y) - (q[2].x - q[0].x) * (q[1].y - q[0].y);+ if (std::abs(d) < 1e-12f) continue;+ const int i0 = std::max(0, int(std::floor(std::min({q[0].x, q[1].x, q[2].x})))),+ i1 = std::min(kGrid - 1, int(std::ceil(std::max({q[0].x, q[1].x, q[2].x})))),+ j0 = std::max(0, int(std::floor(std::min({q[0].y, q[1].y, q[2].y})))),+ j1 = std::min(kGrid - 1, int(std::ceil(std::max({q[0].y, q[1].y, q[2].y}))));+ for (int j = j0; j <= j1; ++j)+ for (int i = i0; i <= i1; ++i) {+ const glm::vec2 c(float(i) + 0.5f, float(j) + 0.5f);+ const float b1 = ((c.x - q[0].x) * (q[2].y - q[0].y) - (q[2].x - q[0].x) * (c.y - q[0].y)) / d;+ const float b2 = ((q[1].x - q[0].x) * (c.y - q[0].y) - (c.x - q[0].x) * (q[1].y - q[0].y)) / d;+ if (b1 >= 0.0f && b2 >= 0.0f && b1 + b2 <= 1.0f) cells[size_t(j) * kGrid + size_t(i)] = 1;+ }+ }+ size_t over = 0;+ for (size_t v = 0; v < role.size(); ++v) {+ if (role[v] != 2) continue;+ int i, j;+ const glm::vec3& p = mesh.positions[v];+ if (cellOf(glm::dot(p, u), glm::dot(p, w), i, j) && cells[size_t(j) * kGrid + size_t(i)]) ++over;+ }+ if (float(over) < kMinOver * float(restCount)) return false;+ area = 0.0f;+ for (uint32_t t : plane) {+ const glm::uvec3& tri = mesh.triangles[t];+ area += glm::length(glm::cross(mesh.positions[tri.y] - mesh.positions[tri.x], mesh.positions[tri.z] - mesh.positions[tri.x]));+ }+ return true;+}++} // namespace++std::vector<uint8_t> flatBackdropTriangles(const Mesh& mesh, BackdropRemoval* found) {+ if (found) *found = {};+ if (mesh.triangles.size() < 2) return {};+ const AABB box = mesh.bounds();+ const glm::vec3 extent = box.max - box.min;+ const float largest = std::max({extent.x, extent.y, extent.z});+ if (!(largest > 0.0f) || !std::isfinite(largest)) return {};+ const std::vector<uint32_t> weld = weldedIds(mesh, largest * 1e-6f);+ std::vector<uint8_t> removed(mesh.triangles.size(), 0);+ BackdropRemoval result;+ for (int pass = 0; pass < 3; ++pass) { // a ground and a back wall, at most one more+ const std::vector<uint8_t> live = [&] {+ std::vector<uint8_t> l(removed.size());+ for (size_t t = 0; t < l.size(); ++t) l[t] = !removed[t];+ return l;+ }();+ struct Candidate {+ std::vector<uint32_t> tris;+ glm::vec3 normal;+ };+ std::vector<Candidate> candidates;+ // Separate pieces, flat in any direction.+ auto pieces = groups(mesh, weld, live);+ if (pieces.size() > 1)+ for (auto& p : pieces) candidates.push_back({std::move(p), {}});+ for (auto& c : candidates) c.normal = flattestDirection(mesh, c.tris);+ // Thin slabs at each side of the bounds (also joined to the subject).+ AABB liveBox;+ for (size_t t = 0; t < live.size(); ++t)+ if (live[t])+ for (int c = 0; c < 3; ++c) liveBox.expand(mesh.positions[mesh.triangles[t][c]]);+ const float slab = 0.02f * std::max({liveBox.max.x - liveBox.min.x, liveBox.max.y - liveBox.min.y,+ liveBox.max.z - liveBox.min.z});+ for (int axis = 0; axis < 3; ++axis)+ for (int side = 0; side < 2; ++side) {+ const float edge = side ? liveBox.max[axis] : liveBox.min[axis];+ std::vector<uint8_t> in(mesh.triangles.size(), 0);+ for (size_t t = 0; t < in.size(); ++t) {+ if (!live[t]) continue;+ const glm::uvec3& tri = mesh.triangles[t];+ in[t] = std::abs(mesh.positions[tri.x][axis] - edge) <= slab &&+ std::abs(mesh.positions[tri.y][axis] - edge) <= slab &&+ std::abs(mesh.positions[tri.z][axis] - edge) <= slab;+ }+ auto slabs = groups(mesh, weld, in);+ if (slabs.size() > 3) slabs.resize(3);+ glm::vec3 n(0.0f);+ n[axis] = 1.0f;+ for (auto& s : slabs) candidates.push_back({std::move(s), n});+ }+ int best = -1;+ float bestArea = 0.0f;+ for (size_t i = 0; i < candidates.size(); ++i) {+ float area = 0.0f;+ if (isBackdrop(mesh, removed, candidates[i].tris, candidates[i].normal, area) && area > bestArea) {+ best = int(i);+ bestArea = area;+ }+ }+ if (best < 0) break;+ for (uint32_t t : candidates[size_t(best)].tris) removed[t] = 1;+ ++result.planes;+ result.triangles += candidates[size_t(best)].tris.size();+ }+ if (result.planes == 0) return {};+ if (found) *found = result;+ std::vector<uint8_t> keep(removed.size());+ for (size_t t = 0; t < keep.size(); ++t) keep[t] = !removed[t];+ return keep;+}++bool removeFlatBackdrop(Mesh& mesh, BackdropRemoval* removed) {+ BackdropRemoval found;+ const std::vector<uint8_t> keep = flatBackdropTriangles(mesh, &found);+ if (removed) *removed = found;+ if (keep.empty()) return false;+ mesh = extractTriangles(mesh, keep);+ mesh.recomputeAllNormals();+ return true;+}++std::string backdropRemovedNote(const BackdropRemoval& r) {+ return std::string(r.planes > 1 ? "Removed " + std::to_string(r.planes) + " flat background planes"+ : std::string("Removed a flat background plane")) ++ " (" + std::to_string(r.triangles) + " triangles)";+}++void registerBackdropCommands(CommandRegistry& registry) {+ registry.add(+ {"remove_flat_background",+ "Remove the flat plane an image's background often becomes in Image to 3D: a thin wall behind or ground under "+ "the subject, covering most of the model's bounds (Image to 3D does this itself unless remove_background=false). "+ "A relief or a coin on its own is kept. One undo step.",+ {},+ [](Document& doc, const Args&) -> CommandResult {+ if (doc.mesh.triangleCount() == 0) return {false, "The Part is empty"};+ if (doc.mesh.micro) return {false, "Remove flat background: apply or remove the micro level first"};+ Mesh m = doc.mesh;+ BackdropRemoval r;+ if (!removeFlatBackdrop(m, &r)) return {false, "No flat background plane found on the active Part"};+ doc.replaceMeshUndoable(std::move(m), "Remove Flat Background");+ return {true, backdropRemovedNote(r) + " - Ctrl+Z brings it back"};+ }});+}++} // namespace sfdiff --git a/src/core/Backdrop.h b/src/core/Backdrop.hnew file mode 100644index 00000000..fc339891--- /dev/null+++ b/src/core/Backdrop.h@@ -0,0 +1,41 @@+// SculptForge - open source digital sculpting+// SPDX-License-Identifier: GPL-3.0-or-later+//+// Image to 3D often turns the picture's background into a big flat plane behind or under the subject. This finds+// such planes in a generated mesh so they can be dropped before it lands in the scene: a separate piece (any+// orientation) or a flat region at one side of the bounds (also when it is joined to the subject) that is+// - flat: thinner than 2.5% of its width,+// - large: covering most of the model's bounds in both of its directions, and larger than the rest,+// - behind or under: everything else stands out on one side of it, over it (not in holes of it).+// A relief or a coin on its own (thin, but the subject) is kept: the rest of it lies over the same area or in its+// holes, never standing clear of a larger plane. Nothing is removed when only the plane would remain.+#pragma once++#include "core/Mesh.h"++#include <cstdint>+#include <string>+#include <vector>++namespace sf {++class CommandRegistry;++struct BackdropRemoval {+ int planes = 0; // backdrops found (a ground and a back wall are two)+ size_t triangles = 0; // triangles they had+};++// keep[t] = 0 for the triangles of every flat backdrop in `mesh`, 1 for the rest; empty when there is none.+std::vector<uint8_t> flatBackdropTriangles(const Mesh& mesh, BackdropRemoval* found = nullptr);++// Removes them (unused vertices and every attribute compacted). False, `mesh` untouched, when there is none.+bool removeFlatBackdrop(Mesh& mesh, BackdropRemoval* removed = nullptr);++// "Removed a flat background plane (N triangles)" for the status line.+std::string backdropRemovedNote(const BackdropRemoval& removed);++// remove_flat_background: the same on the active Part, one undo step.+void registerBackdropCommands(CommandRegistry& registry);++} // namespace sfdiff --git a/src/core/Commands.cpp b/src/core/Commands.cppindex d7ec570c..7bd7bf99 100644--- a/src/core/Commands.cpp+++ b/src/core/Commands.cpp@@ -20,6 +20,7 @@#include "core/HardwareProfile.h"#include "core/ImageTo3D.h"#include "core/Masking.h"+#include "core/Backdrop.h"#include "core/CloseHoles.h"#include "core/Cut.h"#include "core/Multires.h"@@ -869,6 +870,7 @@ CommandRegistry::CommandRegistry() {registerMaskingCommands(*this);registerCutCommands(*this); // cut (core/Cut.h)registerCloseHolesCommands(*this); // close_holes (core/CloseHoles.h)+ registerBackdropCommands(*this); // remove_flat_background (core/Backdrop.h)registerFaceGroupCommands(*this);registerSelectionSetCommands(*this); // selection_set_* (core/SelectionSets.h)registerPartCommands(*this);diff --git a/tests/test_backdrop.cpp b/tests/test_backdrop.cppnew file mode 100644index 00000000..fb625a85--- /dev/null+++ b/tests/test_backdrop.cpp@@ -0,0 +1,130 @@+// SculptForge - open source digital sculpting+// SPDX-License-Identifier: GPL-3.0-or-later+//+// Image to 3D drops the flat plane a picture's background often becomes (core/Backdrop.h, request from @fenix):+// a back wall or a ground, apart from the subject or joined to it, goes; a relief or a coin on its own stays.+#include "TestHarness.h"++#include "ai/AiCommands.h"+#include "core/Backdrop.h"+#include "core/Commands.h"+#include "core/Document.h"+#include "core/ImageTo3D.h"+#include "core/Parts.h"+#include "core/Primitives.h"++#include <glm/glm.hpp>++using namespace sf;++namespace {++Mesh moved(Mesh m, glm::vec3 scale, glm::vec3 offset) {+ for (auto& p : m.positions) p = p * scale + offset;+ m.recomputeAllNormals();+ return m;+}++// A head-sized subject at the origin.+Mesh subject() { return makeIcosphere(3, 0.5f); }++// A wall behind it (a thin slab, 3 x 3 x 0.03) and a single-sided ground under it.+Mesh backWall() { return moved(makeCube(2, 1.0f), {3.0f, 3.0f, 0.03f}, {0.0f, 0.4f, -0.7f}); }+Mesh ground() { return moved(makePlane(6, 3.0f), {1.0f, 1.0f, 1.0f}, {0.0f, -0.55f, 0.0f}); }++// A relief: a flat field with a raised disc in the middle.+Mesh discRelief() {+ Image img;+ img.width = img.height = 64;+ img.rgba.assign(size_t(64) * 64 * 4, 255);+ for (int y = 0; y < 64; ++y)+ for (int x = 0; x < 64; ++x) {+ const bool disc = (x - 32) * (x - 32) + (y - 32) * (y - 32) < 18 * 18;+ uint8_t* p = &img.rgba[(size_t(y) * 64 + size_t(x)) * 4];+ p[0] = p[1] = p[2] = disc ? 255 : 0;+ }+ ReliefOptions o;+ o.resolution = 64;+ return makeImageRelief(img, o);+}++} // namespace++TEST(backdrop_wall_behind_a_subject_is_removed) {+ const Mesh head = subject(), wall = backWall();+ Mesh m = mergeMeshes({&head, &wall});+ BackdropRemoval r;+ CHECK(removeFlatBackdrop(m, &r));+ CHECK(r.planes == 1);+ CHECK(r.triangles == wall.triangleCount());+ CHECK(m.triangleCount() == head.triangleCount());+ CHECK(m.bounds().min.z > -0.51f);+}++TEST(backdrop_ground_under_a_subject_is_removed) {+ const Mesh head = subject(), floor = ground();+ Mesh m = mergeMeshes({&head, &floor});+ BackdropRemoval r;+ CHECK(removeFlatBackdrop(m, &r));+ CHECK(r.triangles == floor.triangleCount());+ CHECK(m.triangleCount() == head.triangleCount());++ // A ground and a back wall standing on its far edge (a corner, as a photo of a room makes): both go.+ const Mesh wall = moved(makeCube(2, 1.0f), {3.0f, 3.0f, 0.03f}, {0.0f, 0.95f, -1.5f});+ Mesh both = mergeMeshes({&head, &floor, &wall});+ CHECK(removeFlatBackdrop(both, &r));+ CHECK(r.planes == 2);+ CHECK(both.triangleCount() == head.triangleCount());+}++TEST(backdrop_ground_joined_to_the_subject_is_removed) {+ // A box standing on the ground, its bottom corners welded into the ground's grid: one connected piece.+ const Mesh box = moved(makeCube(4, 1.0f), {1.0f, 1.0f, 1.0f}, {0.0f, 0.0f, 0.0f});+ const Mesh floor = moved(makePlane(8, 4.0f), {1.0f, 1.0f, 1.0f}, {0.0f, -0.5f, 0.0f});+ Mesh m = mergeMeshes({&box, &floor});+ BackdropRemoval r;+ CHECK(removeFlatBackdrop(m, &r));+ CHECK(r.triangles >= floor.triangleCount());+ const AABB b = m.bounds();+ CHECK(b.max.x - b.min.x < 1.01f);+ CHECK(b.max.y - b.min.y > 0.99f); // the box stays, only its bottom (inside the ground) goes with it+}++TEST(backdrop_relief_and_coin_alone_are_kept) {+ Mesh relief = discRelief();+ const size_t reliefTris = relief.triangleCount();+ CHECK(!removeFlatBackdrop(relief));+ CHECK(relief.triangleCount() == reliefTris);++ // The relief on a separate back plate is still a relief.+ Mesh plate = makePlane(4, 1.0f); // XZ: turned to face +Z, just behind the relief+ for (auto& p : plate.positions) p = {p.x, -p.z, -0.01f};+ const Mesh base = discRelief();+ Mesh plated = mergeMeshes({&base, &plate});+ CHECK(flatBackdropTriangles(plated).empty());++ // A flat coin (thin, closed) on its own.+ Mesh coin = moved(makeCylinder(48, 1.0f), {2.0f, 0.06f, 2.0f}, {});+ CHECK(!removeFlatBackdrop(coin));++ // A plane alone is the only part: kept.+ Mesh plane = makePlane(4, 2.0f);+ CHECK(!removeFlatBackdrop(plane));++ // A subject alone: nothing to remove.+ Mesh head = subject();+ CHECK(!removeFlatBackdrop(head));+}++TEST(backdrop_cli_removes_it_unless_asked_not_to) {+ const Mesh head = subject(), wall = backWall();+ Document doc;+ doc.setMesh(mergeMeshes({&head, &wall}));+ CommandRegistry reg;+ CommandResult r = reg.execute(doc, "remove_flat_background", {});+ CHECK(r.ok);+ CHECK(r.message.find("Removed a flat background plane") != std::string::npos);+ CHECK(doc.mesh.triangleCount() == head.triangleCount());+ r = reg.execute(doc, "remove_flat_background", {});+ CHECK(!r.ok); // nothing left to remove+}diff --git a/tests/ui/remove_flat_background.sfui b/tests/ui/remove_flat_background.sfuinew file mode 100644index 00000000..83569b2e--- /dev/null+++ b/tests/ui/remove_flat_background.sfui@@ -0,0 +1,29 @@+# Image to 3D's "Remove flat background" (core/Backdrop.h, request from @fenix): a generated head with the picture's+# background turned into a thin wall behind it. The command Generate 3D runs on its result removes the wall, as one+# undo step; Undo brings it back. Generate 3D shows the option, on by default.+run ui_look look=classic layout=classic # written for the Classic look+# The wall: a 0.7 m square just behind the head (the lion is 0.32 x 0.44 x 0.21 m, standing on y = 0).+run new_plane resolution=4 ; transform scale=0.35 rotate=90,0,0 pivot=origin translate=0,0.22,-0.16+remember triangles wall+run part_append path=$SCRIPT_DIR/../../assets/demo/lion-head.glb ; part_merge_visible+expect_value triangles > $wall+remember triangles both+camera 30 10+key f+shot backdrop-before++run remove_flat_background+expect_status "Removed a flat background plane"+expect_value triangles == 47214+shot backdrop-after++key ctrl+z+expect_value triangles == $both+key ctrl+shift+z+expect_value triangles == 47214++# The option in Generate 3D.+drop ../../assets/demo/teddy-sketch.png+wait_for "Generate 3D" 5+expect_text Remove flat background+shot generate-remove-background