Skip to content

Free, open-source 3D sculpting for Linux, macOS and Windows: dense meshes, remeshing, painting, built-in AI (image/text to 3D), Send to Unreal, game-ready LODs, Gaussian splats and OpenUSD.

Image to 3D: Remove flat background (on by default) drops the plane a picture's background becomes

islandmed084e7408d61c
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 09f7b670..0a43d309 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -141,6 +141,7 @@ add_library(sculptforge_core STATIC
src/core/SurfaceQuery.cpp
src/core/QRemesh.cpp
src/core/GeneratedRemesh.cpp
+ src/core/Backdrop.cpp
src/core/QRemeshGuides.cpp
src/core/QRemesh2.cpp
src/core/QRemesh2Proxy.cpp
diff --git a/changes/feat-img3d-drop-backdrop.md b/changes/feat-img3d-drop-backdrop.md
new file mode 100644
index 00000000..0c8ff9eb
--- /dev/null
+++ b/changes/feat-img3d-drop-backdrop.md
@@ -0,0 +1,15 @@
+---
+whats_new: **Remove flat background**: Image to 3D drops the flat plane a picture's background often turns into
+ (a wall behind or a ground under the subject) before the model lands. On by default in **Generate 3D**; Undo
+ brings a removed plane back. Thanks @fenix.
+added: **Generate 3D > Remove flat background** (on by default): a thin wall behind or ground under the subject,
+ covering most of the model's bounds, is removed from the generated mesh before remeshing, as its own undo step;
+ a relief or a coin on its own is kept. `image_to_3d` / `text_to_3d` take `remove_background=false`, and the new
+ `remove_flat_background` command does it on the active Part. Requested by @fenix.
+testers:
+ - Any OS: Generate 3D from a photo with a plain wall behind the subject (Hunyuan3D 2.1); the result has no flat
+ plate behind it and the status says "Removed a flat background plane". Ctrl+Z (twice with Quad Remesh) brings it back.
+ - Turn **Remove flat background** off and generate again: the plane stays.
+ - Tool > Relief from an emblem image, then run `remove_flat_background` in the console: "No flat background plane found".
+---
+Core: src/core/Backdrop.{h,cpp}. Tests: tests/test_backdrop.cpp (wall, ground, joined ground, relief, coin, plane alone).
diff --git a/docs/guide/generate-3d.md b/docs/guide/generate-3d.md
index 4861de9b..f82a8bf8 100644
--- a/docs/guide/generate-3d.md
+++ b/docs/guide/generate-3d.md
@@ -45,7 +45,11 @@ The window has two tabs, **From image** and **From text**.
<kbd>Shift</kbd>+<kbd>D</kbd>.
- **Voxel remesh**: one even, watertight triangle surface. Faster (about a second).
- **Off (raw mesh)**: the model exactly as the service made it.
-5. Click **Generate 3D model**. A progress bar shows the queue position and progress; **Cancel** stops the
+5. Leave **Remove flat background** on (the default) to drop the flat plane a picture's background often
+ becomes: a thin wall behind or a ground under the subject that covers most of the model's bounds. A relief
+ or a coin on its own is kept. The status line says when a plane was removed; Undo brings it back (with Remesh
+ on, the first Undo steps back to the raw mesh, the second brings the plane back).
+6. Click **Generate 3D model**. A progress bar shows the queue position and progress; **Cancel** stops the
job.
When it finishes, the status line reports the size before and after remeshing, for example
@@ -246,6 +250,9 @@ sculptforge-cli -c 'image_to_3d path=hero.png; export_glb path=hero.glb'
# TRELLIS.2, best detail, with a free Hugging Face token (without one, Hunyuan3D 2.1 runs and says so)
HF_TOKEN=hf_... sculptforge-cli -c 'image_to_3d path=hero.png model=trellis-2; export_glb path=hero.glb'
+# keep a flat background plane the service made (removed by default; remove_flat_background does it later)
+sculptforge-cli -c 'image_to_3d path=hero.png remove_background=false; export_glb path=hero.glb'
+
# voxel remesh instead of the default Quad Remesh
sculptforge-cli -c 'image_to_3d path=hero.png remesh=voxel; export_glb path=hero.glb'
diff --git a/src/ai/AiCommands.cpp b/src/ai/AiCommands.cpp
index bdfffb9a..fa2ea32e 100644
--- a/src/ai/AiCommands.cpp
+++ b/src/ai/AiCommands.cpp
@@ -8,6 +8,7 @@
#include "ai/ComfyUI.h"
#include "ai/ImageTo3DService.h"
#include "core/EnginePresets.h"
+#include "core/Backdrop.h"
#include "core/GeneratedRemesh.h"
#include "core/Texture.h"
#include "core/Voxel.h"
@@ -50,6 +51,8 @@ CommandResult importGenerated(Document& doc, const Args& a, const GeneratorResul
if (!r.privateDir.empty()) std::filesystem::remove_all(std::filesystem::path(std::u8string(r.privateDir.begin(), r.privateDir.end())), ec);
if (!opened.ok) return opened;
doc.path.clear();
+ BackdropRemoval backdrop;
+ if (argBool(a, "remove_background", true)) removeFlatBackdrop(doc.mesh, &backdrop);
GeneratedRemeshStats st;
if (mode != GeneratedRemesh::Off) {
std::string err;
@@ -64,6 +67,7 @@ CommandResult importGenerated(Document& doc, const Args& a, const GeneratorResul
const std::string note = generatedRemeshNote(st, mode);
return {true, "Generated a 3D model" + what + " (" + std::to_string(st.rawTriangles) + " triangles" +
(note.empty() ? std::string() : ", " + note) + ") - ready to sculpt" +
+ (backdrop.planes ? ". " + backdropRemovedNote(backdrop) : std::string()) +
(r.note.empty() ? std::string() : " " + r.note)};
}
@@ -130,7 +134,9 @@ void addAiFeatureCommands(CommandRegistry& registry, std::function<AiSettings()>
{"resolution", T::Number, "Remesh detail (64-512): voxels across the model; quad picks about "
"resolution^2/2 quads (256: 32768). Default 256.", false},
{"levels", T::Number, "quad: subdivision levels above the quad cage (0-2), projected onto the "
- "model. Default 1.", false}},
+ "model. Default 1.", false},
+ {"remove_background", T::Bool, "Drop a flat plane the background became (a wall behind or ground "
+ "under the subject; a relief or coin alone is kept). Default true.", false}},
[settings](Document& doc, const Args& a) -> CommandResult {
AiSettings s = settings();
if (a.contains("service")) s.generatorService = argString(a, "service");
@@ -173,7 +179,9 @@ void addAiFeatureCommands(CommandRegistry& registry, std::function<AiSettings()>
{"resolution", T::Number, "Remesh detail (64-512): voxels across the model; quad picks about "
"resolution^2/2 quads (256: 32768). Default 256.", false},
{"levels", T::Number, "quad: subdivision levels above the quad cage (0-2), projected onto the "
- "model. Default 1.", false}},
+ "model. Default 1.", false},
+ {"remove_background", T::Bool, "Drop a flat plane the background became (a wall behind or ground "
+ "under the subject; a relief or coin alone is kept). Default true.", false}},
[settings](Document& doc, const Args& a) -> CommandResult {
AiSettings s = settings();
if (a.contains("service")) s.generatorService = argString(a, "service");
diff --git a/src/app/App.cpp b/src/app/App.cpp
index 765f9f38..79c12aeb 100644
--- a/src/app/App.cpp
+++ b/src/app/App.cpp
@@ -3735,24 +3735,30 @@ void App::startGenerator() {
pendingGenerator_ = promise->get_future();
std::thread([promise, settings, in = refImagePath_, text = textMode_ ? std::string(textPrompt_) : std::string(),
out = std::string(stem.begin(), stem.end()), progress = generatorProgress_,
- mode = GeneratedRemesh(generatedRemesh_), resolution = remeshResolution_] {
+ mode = GeneratedRemesh(generatedRemesh_), resolution = remeshResolution_, dropBackdrop = removeBackground_] {
GeneratedModel g;
g.result = text.empty() ? ai::generate3D(settings, in, out, progress.get())
: ai::generate3DFromText(settings, text, out, progress.get());
- if (g.result.ok && mode != GeneratedRemesh::Off) {
+ if (g.result.ok && (mode != GeneratedRemesh::Off || dropBackdrop)) {
// Made sculptable here, so the window stays responsive; the main thread opens the file and swaps this in.
Document scratch;
const CommandResult opened = CommandRegistry().execute(scratch, "open", {{"path", g.result.outputPath}});
- progress->percent = 0;
- progress->setStatus(mode == GeneratedRemesh::Quad ? "Making it sculptable: Quad Remesh..." : "Making it sculptable: Voxel Remesh...");
+ // The background plane goes first (one undo step of its own), so the remesh doesn't build it in.
+ if (opened.ok && dropBackdrop && removeFlatBackdrop(scratch.mesh, &g.backdrop))
+ g.withoutBackdrop = std::make_shared<Mesh>(scratch.mesh);
Mesh m;
- if (!opened.ok) {
+ if (mode == GeneratedRemesh::Off) {
+ // (opened for the background check only)
+ } else if (!opened.ok) {
g.remeshError = opened.message;
- } else if (remeshGenerated(scratch.mesh, mode, resolution, 1, m, g.remeshError, &g.stats, [progress](float f) {
- progress->percent = int(f * 100.0f);
- return !progress->cancel.load();
- })) {
- g.remeshed = std::make_shared<Mesh>(std::move(m));
+ } else {
+ progress->percent = 0;
+ progress->setStatus(mode == GeneratedRemesh::Quad ? "Making it sculptable: Quad Remesh..." : "Making it sculptable: Voxel Remesh...");
+ if (remeshGenerated(scratch.mesh, mode, resolution, 1, m, g.remeshError, &g.stats, [progress](float f) {
+ progress->percent = int(f * 100.0f);
+ return !progress->cancel.load();
+ }))
+ g.remeshed = std::make_shared<Mesh>(std::move(m));
}
}
promise->set_value(std::move(g));
@@ -3781,12 +3787,14 @@ void App::pollGenerator() {
std::filesystem::remove(std::filesystem::path(std::u8string(r.outputPath.begin(), r.outputPath.end())), ec);
const size_t raw = doc_.mesh.triangleCount();
const auto mode = GeneratedRemesh(generatedRemesh_);
+ if (g.withoutBackdrop) doc_.replaceMeshUndoable(std::move(*g.withoutBackdrop), "Remove Flat Background");
if (g.remeshed) doc_.replaceMeshUndoable(std::move(*g.remeshed), mode == GeneratedRemesh::Quad ? "Quad Remesh" : "Remesh");
doc_.path.clear();
const std::string remeshNote = g.remeshed ? ", " + generatedRemeshNote(g.stats, mode) : std::string();
status_ = !g.remeshError.empty()
? "Generated model imported (" + std::to_string(raw) + " tris, not remeshed: " + g.remeshError + ")" + note
: "Generated model imported (" + std::to_string(raw) + " tris" + remeshNote + ") - ready to sculpt" + note;
+ if (g.backdrop.planes) status_ += ". " + backdropRemovedNote(g.backdrop) + ", Undo brings it back";
updateTitle();
}
@@ -4083,6 +4091,11 @@ void App::drawImageTo3D() {
generatedQuadTarget(remeshResolution_));
hint(quads);
}
+ ImGui::Checkbox(tr("Remove flat background"), &removeBackground_);
+ if (ImGui::IsItemHovered(ImGuiHoveredFlags_DelayNormal))
+ ImGui::SetTooltip(tr("Drops the flat plane a picture's background often becomes (a wall behind or ground under "
+ "the subject) before the model lands. A relief or coin on its own is kept. Undo brings "
+ "a removed plane back."));
}
ImGui::PopItemWidth();
if (pendingGenerator_.valid()) {
diff --git a/src/app/App.h b/src/app/App.h
index b4c35602..67faebfd 100644
--- a/src/app/App.h
+++ b/src/app/App.h
@@ -32,6 +32,7 @@
#include "core/Document.h"
#include "core/DiscordPresence.h"
#include "core/Face.h"
+#include "core/Backdrop.h"
#include "core/GeneratedRemesh.h"
#include "core/MeshTransform.h"
#include "app/Wintab.h"
@@ -684,10 +685,13 @@ private:
"If the mask's edge is too ragged to join up, the status line says so and nothing changes.";
bool keepTexture_ = true; // Keep Texture: remeshes carry a painted texture over (core/TextureTransfer.h)
int remeshResolution_ = 256;
+ bool removeBackground_ = true; // Generate 3D's "Remove flat background" (core/Backdrop.h)
// A generated model, made sculptable on the worker thread (Quad Remesh takes seconds).
struct GeneratedModel {
ai::GeneratorResult result;
std::shared_ptr<Mesh> remeshed; // null: Remesh off, or it failed (remeshError)
+ std::shared_ptr<Mesh> withoutBackdrop; // the raw mesh minus a flat background plane (null: none found or off)
+ BackdropRemoval backdrop;
GeneratedRemeshStats stats;
std::string remeshError;
};
diff --git a/src/app/I18nSimplifiedChinese.cpp b/src/app/I18nSimplifiedChinese.cpp
index eafaf004..be17b7db 100644
--- a/src/app/I18nSimplifiedChinese.cpp
+++ b/src/app/I18nSimplifiedChinese.cpp
@@ -4208,6 +4208,10 @@ constexpr Entry kSimplifiedChinese[] = {
"以 AR 查看"},
{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.",
"视图分辨率 75%%:约只绘制一半像素,适合集成显卡和大屏幕。"},
+ {"Remove flat background",
+ "移除平坦背景"},
+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",
+ "在模型载入前,移除图片背景常变成的平面(主体后方的墙或下方的地面)。单独的浮雕或硬币会保留。撤销可找回被移除的平面。"},
{"Voxel remesh",
"体素重新网格化"},
{"Welcome to SculptForge",
diff --git a/src/app/I18nSpanish.cpp b/src/app/I18nSpanish.cpp
index 8a63ea2a..eda05ff7 100644
--- a/src/app/I18nSpanish.cpp
+++ b/src/app/I18nSpanish.cpp
@@ -4645,6 +4645,10 @@ constexpr Entry kSpanish[] = {
"Ver en RA"},
{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.",
"Resolución de vista 75%%: dibuja más o menos la mitad de los píxeles, para gráficos integrados y pantallas grandes."},
+ {"Remove flat background",
+ "Quitar el fondo plano"},
+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",
+ "Quita el plano en que suele convertirse el fondo de una imagen (una pared detrás o un suelo bajo el sujeto) antes de que llegue el modelo. Un relieve o una moneda solos se conservan. Deshacer recupera un plano quitado."},
{"Voxel remesh",
"Remallado vóxel"},
{"Welcome to SculptForge",
diff --git a/src/app/I18nTraditionalChinese.cpp b/src/app/I18nTraditionalChinese.cpp
index 6fc3b2fa..bc3ef0d1 100644
--- a/src/app/I18nTraditionalChinese.cpp
+++ b/src/app/I18nTraditionalChinese.cpp
@@ -4206,6 +4206,10 @@ constexpr Entry kTraditionalChinese[] = {
"以 AR 檢視"},
{"View resolution 75%%: draws about half the pixels, for integrated graphics and big screens.",
"檢視解析度 75%%:約只繪製一半像素,適合內建顯示晶片與大螢幕。"},
+ {"Remove flat background",
+ "移除平坦背景"},
+ {"Drops the flat plane a picture's background often becomes (a wall behind or ground under the subject) before the model lands. A relief or coin on its own is kept. Undo brings a removed plane back.",
+ "在模型載入前,移除圖片背景常變成的平面(主體後方的牆或下方的地面)。單獨的浮雕或硬幣會保留。復原可找回被移除的平面。"},
{"Voxel remesh",
"體素重新網格化"},
{"Welcome to SculptForge",
diff --git a/src/core/Backdrop.cpp b/src/core/Backdrop.cpp
new file mode 100644
index 00000000..a435d430
--- /dev/null
+++ b/src/core/Backdrop.cpp
@@ -0,0 +1,349 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+#include "core/Backdrop.h"
+
+#include "core/Commands.h"
+#include "core/Document.h"
+#include "core/FaceGroups.h"
+
+#include <algorithm>
+#include <array>
+#include <cmath>
+#include <limits>
+#include <numeric>
+#include <unordered_map>
+
+namespace sf {
+namespace {
+
+constexpr float kMaxThickness = 0.025f; // of the plane's width
+constexpr float kMinCover = 0.7f; // of the model's extent, along both plane directions
+constexpr float kMaxRestArea = 0.8f; // the rest's footprint, of the plane's
+constexpr float kMinStandOut = 0.05f; // how far the rest stands clear of the plane, of its width
+constexpr float kMinOver = 0.5f; // share of the rest's vertices over the plane (not over a hole in it)
+constexpr int kGrid = 64;
+
+struct UnionFind {
+ std::vector<uint32_t> parent;
+ explicit UnionFind(size_t n) : parent(n) { std::iota(parent.begin(), parent.end(), 0u); }
+ uint32_t find(uint32_t x) {
+ while (parent[x] != x) x = parent[x] = parent[parent[x]];
+ return x;
+ }
+ void join(uint32_t a, uint32_t b) { parent[find(a)] = find(b); }
+};
+
+// Vertices at the same place share an id (generated files often split them along UV seams).
+std::vector<uint32_t> weldedIds(const Mesh& mesh, float cell) {
+ struct Key {
+ int64_t x, y, z;
+ bool operator==(const Key& o) const { return x == o.x && y == o.y && z == o.z; }
+ };
+ struct Hash {
+ size_t operator()(const Key& k) const {
+ return size_t(k.x * 73856093LL) ^ size_t(k.y * 19349663LL) ^ size_t(k.z * 83492791LL);
+ }
+ };
+ std::unordered_map<Key, uint32_t, Hash> ids;
+ ids.reserve(mesh.positions.size());
+ std::vector<uint32_t> id(mesh.positions.size());
+ for (size_t v = 0; v < id.size(); ++v) {
+ const glm::vec3 q = glm::floor(mesh.positions[v] / cell + 0.5f);
+ id[v] = ids.try_emplace(Key{int64_t(q.x), int64_t(q.y), int64_t(q.z)}, uint32_t(ids.size())).first->second;
+ }
+ return id;
+}
+
+// Groups of the triangles with in[t] != 0 joined through shared (welded) corners, largest area first.
+std::vector<std::vector<uint32_t>> groups(const Mesh& mesh, const std::vector<uint32_t>& weld,
+ const std::vector<uint8_t>& in) {
+ UnionFind uf(mesh.positions.size());
+ std::vector<uint32_t> first(mesh.positions.size(), ~0u);
+ for (uint32_t v = 0; v < weld.size(); ++v) {
+ uint32_t& f = first[weld[v]];
+ if (f == ~0u) f = v;
+ else uf.join(v, f);
+ }
+ for (size_t t = 0; t < mesh.triangles.size(); ++t) {
+ if (!in[t]) continue;
+ const glm::uvec3& tri = mesh.triangles[t];
+ uf.join(tri.x, tri.y);
+ uf.join(tri.x, tri.z);
+ }
+ std::unordered_map<uint32_t, size_t> index;
+ std::vector<std::vector<uint32_t>> out;
+ std::vector<float> area;
+ for (uint32_t t = 0; t < mesh.triangles.size(); ++t) {
+ if (!in[t]) continue;
+ const glm::uvec3& tri = mesh.triangles[t];
+ const auto [it, added] = index.try_emplace(uf.find(tri.x), out.size());
+ if (added) {
+ out.emplace_back();
+ area.push_back(0.0f);
+ }
+ out[it->second].push_back(t);
+ area[it->second] += glm::length(glm::cross(mesh.positions[tri.y] - mesh.positions[tri.x],
+ mesh.positions[tri.z] - mesh.positions[tri.x]));
+ }
+ std::vector<size_t> order(out.size());
+ std::iota(order.begin(), order.end(), size_t(0));
+ std::sort(order.begin(), order.end(), [&](size_t a, size_t b) { return area[a] > area[b]; });
+ std::vector<std::vector<uint32_t>> sorted;
+ sorted.reserve(out.size());
+ for (size_t i : order) sorted.push_back(std::move(out[i]));
+ return sorted;
+}
+
+// The direction a set of points varies least along (cyclic Jacobi on their covariance).
+glm::vec3 flattestDirection(const Mesh& mesh, const std::vector<uint32_t>& tris) {
+ glm::dvec3 mean(0.0);
+ size_t n = 0;
+ for (uint32_t t : tris)
+ for (int c = 0; c < 3; ++c, ++n) mean += glm::dvec3(mesh.positions[mesh.triangles[t][c]]);
+ mean /= double(std::max<size_t>(n, 1));
+ double a[3][3] = {};
+ for (uint32_t t : tris)
+ for (int c = 0; c < 3; ++c) {
+ const glm::dvec3 d = glm::dvec3(mesh.positions[mesh.triangles[t][c]]) - mean;
+ for (int i = 0; i < 3; ++i)
+ for (int j = 0; j < 3; ++j) a[i][j] += d[i] * d[j];
+ }
+ double v[3][3] = {{1, 0, 0}, {0, 1, 0}, {0, 0, 1}};
+ for (int sweep = 0; sweep < 32; ++sweep) {
+ const double off = a[0][1] * a[0][1] + a[0][2] * a[0][2] + a[1][2] * a[1][2];
+ if (off < 1e-24) break;
+ for (int p = 0; p < 2; ++p)
+ for (int q = p + 1; q < 3; ++q) {
+ if (std::abs(a[p][q]) < 1e-30) continue;
+ const double theta = (a[q][q] - a[p][p]) / (2.0 * a[p][q]);
+ const double t = (theta >= 0 ? 1.0 : -1.0) / (std::abs(theta) + std::sqrt(theta * theta + 1.0));
+ const double c = 1.0 / std::sqrt(t * t + 1.0), s = t * c;
+ for (int k = 0; k < 3; ++k) { // A = J^T A J
+ const double akp = a[k][p], akq = a[k][q];
+ a[k][p] = c * akp - s * akq;
+ a[k][q] = s * akp + c * akq;
+ }
+ for (int k = 0; k < 3; ++k) {
+ const double apk = a[p][k], aqk = a[q][k];
+ a[p][k] = c * apk - s * aqk;
+ a[q][k] = s * apk + c * aqk;
+ }
+ for (int k = 0; k < 3; ++k) {
+ const double vkp = v[k][p], vkq = v[k][q];
+ v[k][p] = c * vkp - s * vkq;
+ v[k][q] = s * vkp + c * vkq;
+ }
+ }
+ }
+ int least = 0;
+ for (int i = 1; i < 3; ++i)
+ if (a[i][i] < a[least][least]) least = i;
+ return glm::normalize(glm::vec3(float(v[0][least]), float(v[1][least]), float(v[2][least])));
+}
+
+struct Range {
+ float lo = std::numeric_limits<float>::max(), hi = std::numeric_limits<float>::lowest();
+ void add(float x) {
+ lo = std::min(lo, x);
+ hi = std::max(hi, x);
+ }
+ float size() const { return hi > lo ? hi - lo : 0.0f; }
+};
+
+// Whether the triangles in `plane` (normal `n`) are a flat backdrop to everything else in `mesh`; `area` = theirs.
+bool isBackdrop(const Mesh& mesh, const std::vector<uint8_t>& removed, const std::vector<uint32_t>& plane, glm::vec3 n,
+ float& area) {
+ const glm::vec3 u = glm::normalize(std::abs(n.x) < 0.9f ? glm::cross(n, glm::vec3(1, 0, 0)) : glm::cross(n, glm::vec3(0, 1, 0)));
+ const glm::vec3 w = glm::cross(n, u);
+ std::vector<uint8_t> inPlane(mesh.triangles.size(), 0);
+ for (uint32_t t : plane) inPlane[t] = 1;
+ // Vertices: 1 = of the plane, 2 = of the rest (kept or not yet removed), 3 = both.
+ std::vector<uint8_t> role(mesh.positions.size(), 0);
+ for (size_t t = 0; t < mesh.triangles.size(); ++t) {
+ if (removed[t]) continue;
+ for (int c = 0; c < 3; ++c) role[mesh.triangles[t][c]] |= inPlane[t] ? 1 : 2;
+ }
+ Range pn, pu, pw, rn, ru, rw, tu, tw;
+ size_t restCount = 0;
+ for (size_t v = 0; v < role.size(); ++v) {
+ if (!role[v]) continue;
+ const glm::vec3& p = mesh.positions[v];
+ const float dn = glm::dot(p, n), du = glm::dot(p, u), dw = glm::dot(p, w);
+ tu.add(du);
+ tw.add(dw);
+ if (role[v] & 1) {
+ pn.add(dn);
+ pu.add(du);
+ pw.add(dw);
+ }
+ if (role[v] == 2) {
+ rn.add(dn);
+ ru.add(du);
+ rw.add(dw);
+ ++restCount;
+ }
+ }
+ const float width = std::max(pu.size(), pw.size());
+ if (restCount == 0 || width <= 0.0f) return false; // never the only part
+ if (pn.size() > kMaxThickness * width) return false;
+ if (pu.size() < kMinCover * tu.size() || pw.size() < kMinCover * tw.size()) return false;
+ if (ru.size() * rw.size() > kMaxRestArea * pu.size() * pw.size()) return false;
+ const float mid = 0.5f * (pn.lo + pn.hi);
+ const float below = std::max(0.0f, mid - rn.lo), above = std::max(0.0f, rn.hi - mid);
+ if (std::min(below, above) > 0.1f * std::max(below, above)) return false; // through the middle, not behind
+ if (std::max(below, above) < std::max(4.0f * pn.size(), kMinStandOut * width)) return false;
+
+ // Over the plane, not in its holes (a relief's raised parts sit where its flat field isn't).
+ std::vector<uint8_t> cells(size_t(kGrid) * kGrid, 0);
+ const float su = (kGrid - 1e-3f) / std::max(pu.size(), 1e-12f), sw = (kGrid - 1e-3f) / std::max(pw.size(), 1e-12f);
+ auto cellOf = [&](float du, float dw, int& i, int& j) {
+ i = int(std::floor((du - pu.lo) * su));
+ j = int(std::floor((dw - pw.lo) * sw));
+ return i >= 0 && j >= 0 && i < kGrid && j < kGrid;
+ };
+ for (uint32_t t : plane) {
+ glm::vec2 q[3];
+ for (int c = 0; c < 3; ++c) {
+ const glm::vec3& p = mesh.positions[mesh.triangles[t][c]];
+ q[c] = {(glm::dot(p, u) - pu.lo) * su, (glm::dot(p, w) - pw.lo) * sw};
+ int i, j;
+ if (cellOf(glm::dot(p, u), glm::dot(p, w), i, j)) cells[size_t(j) * kGrid + size_t(i)] = 1;
+ }
+ const float d = (q[1].x - q[0].x) * (q[2].y - q[0].y) - (q[2].x - q[0].x) * (q[1].y - q[0].y);
+ if (std::abs(d) < 1e-12f) continue;
+ const int i0 = std::max(0, int(std::floor(std::min({q[0].x, q[1].x, q[2].x})))),
+ i1 = std::min(kGrid - 1, int(std::ceil(std::max({q[0].x, q[1].x, q[2].x})))),
+ j0 = std::max(0, int(std::floor(std::min({q[0].y, q[1].y, q[2].y})))),
+ j1 = std::min(kGrid - 1, int(std::ceil(std::max({q[0].y, q[1].y, q[2].y}))));
+ for (int j = j0; j <= j1; ++j)
+ for (int i = i0; i <= i1; ++i) {
+ const glm::vec2 c(float(i) + 0.5f, float(j) + 0.5f);
+ const float b1 = ((c.x - q[0].x) * (q[2].y - q[0].y) - (q[2].x - q[0].x) * (c.y - q[0].y)) / d;
+ const float b2 = ((q[1].x - q[0].x) * (c.y - q[0].y) - (c.x - q[0].x) * (q[1].y - q[0].y)) / d;
+ if (b1 >= 0.0f && b2 >= 0.0f && b1 + b2 <= 1.0f) cells[size_t(j) * kGrid + size_t(i)] = 1;
+ }
+ }
+ size_t over = 0;
+ for (size_t v = 0; v < role.size(); ++v) {
+ if (role[v] != 2) continue;
+ int i, j;
+ const glm::vec3& p = mesh.positions[v];
+ if (cellOf(glm::dot(p, u), glm::dot(p, w), i, j) && cells[size_t(j) * kGrid + size_t(i)]) ++over;
+ }
+ if (float(over) < kMinOver * float(restCount)) return false;
+ area = 0.0f;
+ for (uint32_t t : plane) {
+ const glm::uvec3& tri = mesh.triangles[t];
+ area += glm::length(glm::cross(mesh.positions[tri.y] - mesh.positions[tri.x], mesh.positions[tri.z] - mesh.positions[tri.x]));
+ }
+ return true;
+}
+
+} // namespace
+
+std::vector<uint8_t> flatBackdropTriangles(const Mesh& mesh, BackdropRemoval* found) {
+ if (found) *found = {};
+ if (mesh.triangles.size() < 2) return {};
+ const AABB box = mesh.bounds();
+ const glm::vec3 extent = box.max - box.min;
+ const float largest = std::max({extent.x, extent.y, extent.z});
+ if (!(largest > 0.0f) || !std::isfinite(largest)) return {};
+ const std::vector<uint32_t> weld = weldedIds(mesh, largest * 1e-6f);
+ std::vector<uint8_t> removed(mesh.triangles.size(), 0);
+ BackdropRemoval result;
+ for (int pass = 0; pass < 3; ++pass) { // a ground and a back wall, at most one more
+ const std::vector<uint8_t> live = [&] {
+ std::vector<uint8_t> l(removed.size());
+ for (size_t t = 0; t < l.size(); ++t) l[t] = !removed[t];
+ return l;
+ }();
+ struct Candidate {
+ std::vector<uint32_t> tris;
+ glm::vec3 normal;
+ };
+ std::vector<Candidate> candidates;
+ // Separate pieces, flat in any direction.
+ auto pieces = groups(mesh, weld, live);
+ if (pieces.size() > 1)
+ for (auto& p : pieces) candidates.push_back({std::move(p), {}});
+ for (auto& c : candidates) c.normal = flattestDirection(mesh, c.tris);
+ // Thin slabs at each side of the bounds (also joined to the subject).
+ AABB liveBox;
+ for (size_t t = 0; t < live.size(); ++t)
+ if (live[t])
+ for (int c = 0; c < 3; ++c) liveBox.expand(mesh.positions[mesh.triangles[t][c]]);
+ const float slab = 0.02f * std::max({liveBox.max.x - liveBox.min.x, liveBox.max.y - liveBox.min.y,
+ liveBox.max.z - liveBox.min.z});
+ for (int axis = 0; axis < 3; ++axis)
+ for (int side = 0; side < 2; ++side) {
+ const float edge = side ? liveBox.max[axis] : liveBox.min[axis];
+ std::vector<uint8_t> in(mesh.triangles.size(), 0);
+ for (size_t t = 0; t < in.size(); ++t) {
+ if (!live[t]) continue;
+ const glm::uvec3& tri = mesh.triangles[t];
+ in[t] = std::abs(mesh.positions[tri.x][axis] - edge) <= slab &&
+ std::abs(mesh.positions[tri.y][axis] - edge) <= slab &&
+ std::abs(mesh.positions[tri.z][axis] - edge) <= slab;
+ }
+ auto slabs = groups(mesh, weld, in);
+ if (slabs.size() > 3) slabs.resize(3);
+ glm::vec3 n(0.0f);
+ n[axis] = 1.0f;
+ for (auto& s : slabs) candidates.push_back({std::move(s), n});
+ }
+ int best = -1;
+ float bestArea = 0.0f;
+ for (size_t i = 0; i < candidates.size(); ++i) {
+ float area = 0.0f;
+ if (isBackdrop(mesh, removed, candidates[i].tris, candidates[i].normal, area) && area > bestArea) {
+ best = int(i);
+ bestArea = area;
+ }
+ }
+ if (best < 0) break;
+ for (uint32_t t : candidates[size_t(best)].tris) removed[t] = 1;
+ ++result.planes;
+ result.triangles += candidates[size_t(best)].tris.size();
+ }
+ if (result.planes == 0) return {};
+ if (found) *found = result;
+ std::vector<uint8_t> keep(removed.size());
+ for (size_t t = 0; t < keep.size(); ++t) keep[t] = !removed[t];
+ return keep;
+}
+
+bool removeFlatBackdrop(Mesh& mesh, BackdropRemoval* removed) {
+ BackdropRemoval found;
+ const std::vector<uint8_t> keep = flatBackdropTriangles(mesh, &found);
+ if (removed) *removed = found;
+ if (keep.empty()) return false;
+ mesh = extractTriangles(mesh, keep);
+ mesh.recomputeAllNormals();
+ return true;
+}
+
+std::string backdropRemovedNote(const BackdropRemoval& r) {
+ return std::string(r.planes > 1 ? "Removed " + std::to_string(r.planes) + " flat background planes"
+ : std::string("Removed a flat background plane")) +
+ " (" + std::to_string(r.triangles) + " triangles)";
+}
+
+void registerBackdropCommands(CommandRegistry& registry) {
+ registry.add(
+ {"remove_flat_background",
+ "Remove the flat plane an image's background often becomes in Image to 3D: a thin wall behind or ground under "
+ "the subject, covering most of the model's bounds (Image to 3D does this itself unless remove_background=false). "
+ "A relief or a coin on its own is kept. One undo step.",
+ {},
+ [](Document& doc, const Args&) -> CommandResult {
+ if (doc.mesh.triangleCount() == 0) return {false, "The Part is empty"};
+ if (doc.mesh.micro) return {false, "Remove flat background: apply or remove the micro level first"};
+ Mesh m = doc.mesh;
+ BackdropRemoval r;
+ if (!removeFlatBackdrop(m, &r)) return {false, "No flat background plane found on the active Part"};
+ doc.replaceMeshUndoable(std::move(m), "Remove Flat Background");
+ return {true, backdropRemovedNote(r) + " - Ctrl+Z brings it back"};
+ }});
+}
+
+} // namespace sf
diff --git a/src/core/Backdrop.h b/src/core/Backdrop.h
new file mode 100644
index 00000000..fc339891
--- /dev/null
+++ b/src/core/Backdrop.h
@@ -0,0 +1,41 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Image to 3D often turns the picture's background into a big flat plane behind or under the subject. This finds
+// such planes in a generated mesh so they can be dropped before it lands in the scene: a separate piece (any
+// orientation) or a flat region at one side of the bounds (also when it is joined to the subject) that is
+// - flat: thinner than 2.5% of its width,
+// - large: covering most of the model's bounds in both of its directions, and larger than the rest,
+// - behind or under: everything else stands out on one side of it, over it (not in holes of it).
+// A relief or a coin on its own (thin, but the subject) is kept: the rest of it lies over the same area or in its
+// holes, never standing clear of a larger plane. Nothing is removed when only the plane would remain.
+#pragma once
+
+#include "core/Mesh.h"
+
+#include <cstdint>
+#include <string>
+#include <vector>
+
+namespace sf {
+
+class CommandRegistry;
+
+struct BackdropRemoval {
+ int planes = 0; // backdrops found (a ground and a back wall are two)
+ size_t triangles = 0; // triangles they had
+};
+
+// keep[t] = 0 for the triangles of every flat backdrop in `mesh`, 1 for the rest; empty when there is none.
+std::vector<uint8_t> flatBackdropTriangles(const Mesh& mesh, BackdropRemoval* found = nullptr);
+
+// Removes them (unused vertices and every attribute compacted). False, `mesh` untouched, when there is none.
+bool removeFlatBackdrop(Mesh& mesh, BackdropRemoval* removed = nullptr);
+
+// "Removed a flat background plane (N triangles)" for the status line.
+std::string backdropRemovedNote(const BackdropRemoval& removed);
+
+// remove_flat_background: the same on the active Part, one undo step.
+void registerBackdropCommands(CommandRegistry& registry);
+
+} // namespace sf
diff --git a/src/core/Commands.cpp b/src/core/Commands.cpp
index d7ec570c..7bd7bf99 100644
--- a/src/core/Commands.cpp
+++ b/src/core/Commands.cpp
@@ -20,6 +20,7 @@
#include "core/HardwareProfile.h"
#include "core/ImageTo3D.h"
#include "core/Masking.h"
+#include "core/Backdrop.h"
#include "core/CloseHoles.h"
#include "core/Cut.h"
#include "core/Multires.h"
@@ -869,6 +870,7 @@ CommandRegistry::CommandRegistry() {
registerMaskingCommands(*this);
registerCutCommands(*this); // cut (core/Cut.h)
registerCloseHolesCommands(*this); // close_holes (core/CloseHoles.h)
+ registerBackdropCommands(*this); // remove_flat_background (core/Backdrop.h)
registerFaceGroupCommands(*this);
registerSelectionSetCommands(*this); // selection_set_* (core/SelectionSets.h)
registerPartCommands(*this);
diff --git a/tests/test_backdrop.cpp b/tests/test_backdrop.cpp
new file mode 100644
index 00000000..fb625a85
--- /dev/null
+++ b/tests/test_backdrop.cpp
@@ -0,0 +1,130 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Image to 3D drops the flat plane a picture's background often becomes (core/Backdrop.h, request from @fenix):
+// a back wall or a ground, apart from the subject or joined to it, goes; a relief or a coin on its own stays.
+#include "TestHarness.h"
+
+#include "ai/AiCommands.h"
+#include "core/Backdrop.h"
+#include "core/Commands.h"
+#include "core/Document.h"
+#include "core/ImageTo3D.h"
+#include "core/Parts.h"
+#include "core/Primitives.h"
+
+#include <glm/glm.hpp>
+
+using namespace sf;
+
+namespace {
+
+Mesh moved(Mesh m, glm::vec3 scale, glm::vec3 offset) {
+ for (auto& p : m.positions) p = p * scale + offset;
+ m.recomputeAllNormals();
+ return m;
+}
+
+// A head-sized subject at the origin.
+Mesh subject() { return makeIcosphere(3, 0.5f); }
+
+// A wall behind it (a thin slab, 3 x 3 x 0.03) and a single-sided ground under it.
+Mesh backWall() { return moved(makeCube(2, 1.0f), {3.0f, 3.0f, 0.03f}, {0.0f, 0.4f, -0.7f}); }
+Mesh ground() { return moved(makePlane(6, 3.0f), {1.0f, 1.0f, 1.0f}, {0.0f, -0.55f, 0.0f}); }
+
+// A relief: a flat field with a raised disc in the middle.
+Mesh discRelief() {
+ Image img;
+ img.width = img.height = 64;
+ img.rgba.assign(size_t(64) * 64 * 4, 255);
+ for (int y = 0; y < 64; ++y)
+ for (int x = 0; x < 64; ++x) {
+ const bool disc = (x - 32) * (x - 32) + (y - 32) * (y - 32) < 18 * 18;
+ uint8_t* p = &img.rgba[(size_t(y) * 64 + size_t(x)) * 4];
+ p[0] = p[1] = p[2] = disc ? 255 : 0;
+ }
+ ReliefOptions o;
+ o.resolution = 64;
+ return makeImageRelief(img, o);
+}
+
+} // namespace
+
+TEST(backdrop_wall_behind_a_subject_is_removed) {
+ const Mesh head = subject(), wall = backWall();
+ Mesh m = mergeMeshes({&head, &wall});
+ BackdropRemoval r;
+ CHECK(removeFlatBackdrop(m, &r));
+ CHECK(r.planes == 1);
+ CHECK(r.triangles == wall.triangleCount());
+ CHECK(m.triangleCount() == head.triangleCount());
+ CHECK(m.bounds().min.z > -0.51f);
+}
+
+TEST(backdrop_ground_under_a_subject_is_removed) {
+ const Mesh head = subject(), floor = ground();
+ Mesh m = mergeMeshes({&head, &floor});
+ BackdropRemoval r;
+ CHECK(removeFlatBackdrop(m, &r));
+ CHECK(r.triangles == floor.triangleCount());
+ CHECK(m.triangleCount() == head.triangleCount());
+
+ // A ground and a back wall standing on its far edge (a corner, as a photo of a room makes): both go.
+ const Mesh wall = moved(makeCube(2, 1.0f), {3.0f, 3.0f, 0.03f}, {0.0f, 0.95f, -1.5f});
+ Mesh both = mergeMeshes({&head, &floor, &wall});
+ CHECK(removeFlatBackdrop(both, &r));
+ CHECK(r.planes == 2);
+ CHECK(both.triangleCount() == head.triangleCount());
+}
+
+TEST(backdrop_ground_joined_to_the_subject_is_removed) {
+ // A box standing on the ground, its bottom corners welded into the ground's grid: one connected piece.
+ const Mesh box = moved(makeCube(4, 1.0f), {1.0f, 1.0f, 1.0f}, {0.0f, 0.0f, 0.0f});
+ const Mesh floor = moved(makePlane(8, 4.0f), {1.0f, 1.0f, 1.0f}, {0.0f, -0.5f, 0.0f});
+ Mesh m = mergeMeshes({&box, &floor});
+ BackdropRemoval r;
+ CHECK(removeFlatBackdrop(m, &r));
+ CHECK(r.triangles >= floor.triangleCount());
+ const AABB b = m.bounds();
+ CHECK(b.max.x - b.min.x < 1.01f);
+ CHECK(b.max.y - b.min.y > 0.99f); // the box stays, only its bottom (inside the ground) goes with it
+}
+
+TEST(backdrop_relief_and_coin_alone_are_kept) {
+ Mesh relief = discRelief();
+ const size_t reliefTris = relief.triangleCount();
+ CHECK(!removeFlatBackdrop(relief));
+ CHECK(relief.triangleCount() == reliefTris);
+
+ // The relief on a separate back plate is still a relief.
+ Mesh plate = makePlane(4, 1.0f); // XZ: turned to face +Z, just behind the relief
+ for (auto& p : plate.positions) p = {p.x, -p.z, -0.01f};
+ const Mesh base = discRelief();
+ Mesh plated = mergeMeshes({&base, &plate});
+ CHECK(flatBackdropTriangles(plated).empty());
+
+ // A flat coin (thin, closed) on its own.
+ Mesh coin = moved(makeCylinder(48, 1.0f), {2.0f, 0.06f, 2.0f}, {});
+ CHECK(!removeFlatBackdrop(coin));
+
+ // A plane alone is the only part: kept.
+ Mesh plane = makePlane(4, 2.0f);
+ CHECK(!removeFlatBackdrop(plane));
+
+ // A subject alone: nothing to remove.
+ Mesh head = subject();
+ CHECK(!removeFlatBackdrop(head));
+}
+
+TEST(backdrop_cli_removes_it_unless_asked_not_to) {
+ const Mesh head = subject(), wall = backWall();
+ Document doc;
+ doc.setMesh(mergeMeshes({&head, &wall}));
+ CommandRegistry reg;
+ CommandResult r = reg.execute(doc, "remove_flat_background", {});
+ CHECK(r.ok);
+ CHECK(r.message.find("Removed a flat background plane") != std::string::npos);
+ CHECK(doc.mesh.triangleCount() == head.triangleCount());
+ r = reg.execute(doc, "remove_flat_background", {});
+ CHECK(!r.ok); // nothing left to remove
+}
diff --git a/tests/ui/remove_flat_background.sfui b/tests/ui/remove_flat_background.sfui
new file mode 100644
index 00000000..83569b2e
--- /dev/null
+++ b/tests/ui/remove_flat_background.sfui
@@ -0,0 +1,29 @@
+# Image to 3D's "Remove flat background" (core/Backdrop.h, request from @fenix): a generated head with the picture's
+# background turned into a thin wall behind it. The command Generate 3D runs on its result removes the wall, as one
+# undo step; Undo brings it back. Generate 3D shows the option, on by default.
+run ui_look look=classic layout=classic # written for the Classic look
+# The wall: a 0.7 m square just behind the head (the lion is 0.32 x 0.44 x 0.21 m, standing on y = 0).
+run new_plane resolution=4 ; transform scale=0.35 rotate=90,0,0 pivot=origin translate=0,0.22,-0.16
+remember triangles wall
+run part_append path=$SCRIPT_DIR/../../assets/demo/lion-head.glb ; part_merge_visible
+expect_value triangles > $wall
+remember triangles both
+camera 30 10
+key f
+shot backdrop-before
+
+run remove_flat_background
+expect_status "Removed a flat background plane"
+expect_value triangles == 47214
+shot backdrop-after
+
+key ctrl+z
+expect_value triangles == $both
+key ctrl+shift+z
+expect_value triangles == 47214
+
+# The option in Generate 3D.
+drop ../../assets/demo/teddy-sketch.png
+wait_for "Generate 3D" 5
+expect_text Remove flat background
+shot generate-remove-background