Skip to content

Free, open-source 3D sculpting for Linux, macOS and Windows: dense meshes, remeshing, painting, built-in AI (image/text to 3D), Send to Unreal, game-ready LODs, Gaussian splats and OpenUSD.

Focus Detail (experimental): a billion micro triangles streamed from a tile file

islandmed79d74e4530df
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a5607791..b790b557 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,6 +12,11 @@ All notable changes to SculptForge are listed here. The format follows
- Tablet Mouse mode (Preferences > Tablet): on some Windows tablets the brush stopped following the pen in 0.10.7,
because the system cursor reading stayed put while the pen moved. A reading that doesn't move with the pen is now
ignored and the pen moves the brush relatively, from where it was, with no jump (#72).
+### Added
+- Focus Detail (experimental, off by default): a billion-micro-triangle asset streamed from one compressed file.
+ Detail loads in under and ahead of the brush and wherever the camera would show it; everything else shows a
+ coarse version. Saving writes only the changed tiles. Try `sculptforge --benchmark standard-1b` or
+ `sculptforge --focus-detail FILE.sfdetail`; design and numbers in docs/design/focus-detail.md.
## [0.10.7] - 2026-11-01
diff --git a/CMakeLists.txt b/CMakeLists.txt
index dd947ce2..1088f302 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -94,6 +94,8 @@ add_library(sculptforge_core STATIC
src/core/MicroSculpt.cpp
src/core/MicroCommands.cpp
src/core/MicroStore.cpp
+ src/core/FocusDetail.cpp
+ src/core/FocusBench.cpp
src/core/MultiresCommands.cpp
src/core/AutoUv.cpp
src/core/TextureTransfer.cpp
@@ -175,6 +177,15 @@ else()
endif()
find_package(Threads REQUIRED)
target_link_libraries(sculptforge_core PUBLIC glm::glm Threads::Threads)
+# zstd for Focus Detail tiles (core/FocusDetail.h), when the system has it; without it the tiles use our own codec.
+find_package(PkgConfig QUIET)
+if(PkgConfig_FOUND)
+ pkg_check_modules(SF_ZSTD QUIET IMPORTED_TARGET libzstd)
+endif()
+if(SF_ZSTD_FOUND)
+ target_link_libraries(sculptforge_core PRIVATE PkgConfig::SF_ZSTD)
+ target_compile_definitions(sculptforge_core PRIVATE SF_HAVE_ZSTD)
+endif()
target_compile_definitions(sculptforge_core PUBLIC GLM_FORCE_XYZW_ONLY)
target_compile_definitions(sculptforge_core PRIVATE SCULPTFORGE_VERSION="${PROJECT_VERSION}") # written into projects
if(MSVC)
@@ -306,6 +317,7 @@ if(SCULPTFORGE_BUILD_APP)
src/app/PanelGeometry.cpp
src/app/Gizmo.cpp
src/app/PanelBake.cpp
+ src/app/FocusDetailApp.cpp
src/app/PanelRetopo.cpp
src/app/PanelZSpheres.cpp
src/app/PanelSplatTrain.cpp
diff --git a/docs/design/focus-detail.md b/docs/design/focus-detail.md
new file mode 100644
index 00000000..22002681
--- /dev/null
+++ b/docs/design/focus-detail.md
@@ -0,0 +1,161 @@
+# Focus Detail: a billion-polygon asset, sculpted locally at interactive rates (prototype)
+
+Branch `research/focus-detail`. Status: **prototype, off by default** (`sculptforge --focus-detail FILE.sfdetail`,
+`sculptforge --benchmark standard-1b`, `sculptforge-cli --bench-focus`). Owner of this doc: the data, memory and
+streaming side. GPU drawing and stroke preview: [perf-moonshots.md](perf-moonshots.md) (idea #8 is the GPU half
+of this design).
+
+Comparison point: the publicly described idea of "HD" geometry in other sculpting apps (only a region you pick is
+held at full resolution and you sculpt inside it). Focus Detail is our own design and differs on purpose: there is
+no region to pick, the focus follows the brush and the camera on its own, ahead of time; out-of-focus detail stays
+visible at a coarse level instead of disappearing; and the whole asset is one memory-mapped file that saves
+incrementally.
+
+## 1. Data model
+
+We already have micro levels (`core/Micro.h`): above the top subdivision level, every base triangle carries a
+regular grid of 16-bit scalar heights (`n = 2^level` segments per edge, `(n+1)(n+2)/2` points), displaced along the
+interpolated normal of the Phong-tessellated base. Micro triangles are never stored. Brushes evaluate the grid on the
+CPU (`MicroStroke`), the renderer in its vertex shader (`MicroRenderer`), and undo keeps per-tile deltas
+(`MicroUndo` journal). Focus Detail reuses all of this. **A tile is one base triangle's grid.**
+
+| | |
+|---|---|
+| Base mesh | positions, normals, triangles (62,208 triangles for standard-1b) |
+| Tile | the base triangle's micro grid: level 7 = 8,385 points = 16,384 micro triangles |
+| Quantization | int16 × `step` world units (`step` = base edge / 4096, as `microDivide`) |
+| Coarse grid | level 3 (45 points), a *subsample* of the full grid: exact at its points |
+| Compression | row-delta (each point less its row predecessor; a row's first less the point below), then zstd level 3 (or our tile codec when the build has no zstd) |
+
+### The `.sfdetail` file (v1)
+
+```
+header (128 B) magic SFDETAIL, version, codec, level, coarse level, counts, step,
+ base offset/bytes, directory offset/bytes, generation, garbage bytes, directory FNV-1a hash
+base block positions (V×12), normals (V×12), triangles (T×12)
+tile blobs one compressed full tile each
+coarse chunks 1,024 coarse grids per compressed chunk
+directory per tile: offset, bytes, exact range (max |d|), coarse error (world units);
+ per chunk: offset, bytes
+```
+
+The file is opened read-only with `mmap` (POSIX; Windows isn't done yet), so the page cache holds the
+recently read blobs. The directory carries each tile's **exact range**, so brushes find candidate triangles
+(`triRange`) correctly even for tiles that aren't loaded. It also carries the **coarse error**, the most the
+upsampled coarse grid is off from the full one, so the camera test needs no tile data.
+
+## 2. Resident states
+
+`MicroMesh::disp` keeps its full-size, contiguous layout, so every brush, the renderer and undo work unchanged.
+Each tile is in one of these states:
+
+| State | `disp` holds | Cost |
+|---|---|---|
+| **coarse** | the coarse grid upsampled: exact at the 45 coarse points, linear in between, in integer arithmetic | coarse grid always resident (5.6 MB for 62K tiles) |
+| **full** | the decompressed tile | read from the mapped file |
+| **dirty** | full and edited since the last save | |
+| *parked* | a dirty tile that was demoted: compressed in memory until saved | ~4-8 KB |
+
+**No cracks at the focus border.** A shared edge's upsampled points depend only on the edge's two coarse ends and
+give the same integer sum from either side, so coarse–coarse edges are watertight. Coarse points are a subsample,
+so corners match between coarse and full. When a tile is promoted, its real edge values are copied into coarse
+neighbours. When it is demoted, it takes its full neighbours' real edges and gives its upsampled edges to coarse
+neighbours (`shareEdges`). Tested: `focus_detail_open_is_coarse_then_full_is_exact`.
+
+## 3. The focus: predictive promotion and demotion
+
+`FocusDetail::update(mesh, view, brush, budgetMs)` runs once a frame (app: just before the GPU sync):
+
+1. **Score every tile** (parallel, ~62K tiles):
+ - **Brush:** distance from the tile's bounds to the brush's *path ahead*, from position *p* to
+ *p + v × lookahead* (0.25 s). The X-symmetry mirror is included. A tile within 2 radii gets priority 10⁶ − distance.
+ The app gets *p* from the hover pick each frame and *v* from the smoothed motion. So tiles are promoted while
+ the cursor hovers, and ahead of a moving stroke.
+ - **Camera:** a tile inside the frustum gets the score `coarseErr × focalPx / distance` (its screen-space error
+ at the coarse level) if that is above `errorPx` (0.5 px).
+2. **Promote** the wanted coarse tiles, highest score first, in parallel batches of 64, until the frame's time
+ budget runs out (4 ms). The rest wait for the next frame (`lastPending`). Each promoted tile gets `markDirty`
+ so the renderer re-uploads it.
+3. **Demote by LRU** while the full tiles exceed `budgetBytes` (512 MB, perf-moonshots' figure for the upsampled
+ copy). A demoted tile is one not wanted this frame or the last. Dirty tiles are never demoted during a stroke.
+ The brush's own tiles may go over budget.
+
+**Edits write only full tiles.** `MicroStroke::touch` (every dab and grab, before it saves undo tiles) and
+`MicroUndo::swapWith` call `ensureFull(tiles, edit)`, which promotes on the spot and marks the tiles dirty. The
+mutex makes that safe against the frame's `update()` (the predictive stroke solver runs dabs on another thread).
+If prediction worked, `ensureFull` finds nothing to load. The benchmark counts the tiles it had to load while the
+brush waited.
+
+**Undo** needs nothing new: a stroke's `MicroUndo` already holds copy-on-write deltas of exactly the tiles it
+changed. Undo and redo promote those tiles first, including from parked state. Undo of a tile that was saved and
+then evicted reads its saved value and applies the delta.
+
+**Saving** (`FocusDetail::save`) appends only the dirty and parked tiles, the coarse chunks that contain them, the
+base block if the base cage moved (Move on a micro level moves the cage), and a new directory. Then comes `fsync`, then
+the 128-byte header is rewritten in place, then `fsync` again. Until the header is written, the old directory is
+still the valid one, so a crash keeps the last save. Garbage is counted in the header; compaction isn't done yet.
+
+## 4. Prototype
+
+| Piece | Where |
+|---|---|
+| File, codec, states, focus, save | `src/core/FocusDetail.{h,cpp}` |
+| Hooks (2 lines) | `MicroStroke::touch`, `MicroUndo::swapWith`; `MicroMesh::focus` |
+| Synthetic asset | `focusWriteSyntheticHead`: the standard scenes' procedural head (`makeBenchHead`), wrinkles + pores generated per tile, written in parallel batches; the full level is never in memory |
+| standard-1b | `focusStandard1bPath()`: 62,208 base triangles × level 7 = **1,019,215,872 micro triangles**, cached in `~/.cache/sculptforge/standard-1b-v1.sfdetail` (written once, ~5 s) |
+| App | `src/app/FocusDetailApp.cpp`: open `.sfdetail`, per-frame update; `--benchmark standard-1b` (proof mode, standard stroke set) |
+| Benchmark | `sculptforge-cli --bench-focus [--bench-focus-base N --bench-micro-level L]` (`src/core/FocusBench.cpp`) |
+| Tests | `tests/test_focus_detail.cpp`: exactness, watertight borders, damaged files, brush promotion, undo/redo, incremental save + reopen, moved base, prediction ahead, LRU budget, parking |
+
+## 5. Measurements
+
+M1 Pro (10 cores, 32 GB unified memory), Asahi Linux, Release, `nice -n 19`. The results are in
+[§5.1](#51-results) (filled from `--bench-focus` and `--benchmark standard-1b`, run only at load < 2).
+
+Methodology (`--bench-focus`):
+- **Open**: map, read base + directory, decompress the 61 coarse chunks, upsample every tile, attach (`microAttach`), then BVH.
+- **Promotion latency**: 256 single tiles through `ensureFull` (decompress + edges + markDirty), one thread.
+- **Orbit + zoom**: 240 frames, a full turn at distance 2.2, then a zoom to 0.8, 1920×1080, 40°, 4 ms budget.
+- **Strokes**: a figure eight over the face at 240 Hz, X symmetry, each stroke on a cold focus (everything demoted
+ first), hover for 3 frames first. Clay at 3 sizes; Move along normals (writes tiles) and Move on the base cage
+ (the default); Clay without prediction for comparison.
+- **Save**: incremental after all strokes. **Focus off**: every tile decompressed, which is what a plain micro
+ level holds.
+
+### 5.1 Results
+
+RESULTS_PLACEHOLDER
+
+## 6. Honestly: what works, what's faked, what's missing
+
+**Works (tested, measured):**
+- A 1.02-billion-micro-triangle head opens from a 241 MB file in well under a second and sculpts with Clay and Move.
+ Tiles under and ahead of the brush come in before the brush gets there.
+- Watertight focus borders; edits, undo/redo and parking never lose detail; incremental save in tens of ms against a
+ full rewrite in seconds; crash-safe header-last commit.
+
+**Faked (LOD):**
+- Out-of-focus tiles show their **level-3 grid upsampled**: forms and wrinkles a few base edges across survive;
+ pores and finer detail are flat until the tile is promoted. Their shading also comes from the upsampled grid, so
+ it is smooth. At a framing view the coarse error is far below a pixel, so nothing is visibly lost. Zoomed in, the
+ screen-error test promotes what's visible within ~1-3 frames.
+- The renderer still draws by its own per-cluster LOD and triangle budget (`MicroRenderer`). That's what makes
+ orbiting interactive; Focus Detail doesn't change drawing.
+
+**Missing / not true yet:**
+- **CPU memory isn't sparse.** `disp` stays a dense 2 bytes per point (1.04 GB at 1B) because every consumer indexes it
+ directly. Promotion saves decompression and I/O, not RAM. At 1B this is fine (RSS ~1.2-1.5 GB). At 4B+ it isn't.
+- **GPU memory doesn't scale with the focus.** The renderer uploads every tile's full grid (6 B a point: ~3.1 GB at
+ 1B; on unified-memory machines that's system RAM). Per-tile GPU levels are perf-moonshots #8 (estimate: 3.1 GB →
+ ~0.6 GB with this focus).
+- **First shading** of all tiles in the renderer (~5 s at 1B Micro 5, docs/performance.md) still runs at open,
+ coarse or not. Shading coarse tiles at their 45 points would cut it ~180×.
+- **Not wired into projects:** a `.sfproj` doesn't reference a `.sfdetail`; the app has no Save for it yet (the core
+ `save()` is tested). Compaction of garbage, Windows mapping, masks on streamed tiles (`mask` stays dense), layers,
+ and multiple streamed Parts aren't done.
+- **Today's micro level already meets the brief's 1B targets** (docs/performance.md: 1.02B at Micro 5, RSS 1.7 GB,
+ open 0.5 s). Focus Detail at 1B wins on disk size, save time and the scaling path, not on RAM.
+
+## 7. Recommended 0.11 scope
+
+RECOMMEND_PLACEHOLDER
diff --git a/src/app/App.cpp b/src/app/App.cpp
index 08c6194c..add96262 100644
--- a/src/app/App.cpp
+++ b/src/app/App.cpp
@@ -461,6 +461,7 @@ int App::runUiTest(int w, int h, int fps) {
prewarmTexturePaint(); // dense textured meshes: ready the area under the hovering Paint brush (PanelPbr.cpp)
const auto syncStart = std::chrono::steady_clock::now();
+ updateFocusDetail(); // (a streamed micro level: promote what the camera and brush need, before the upload)
syncScene();
notePerformanceFrame(std::chrono::duration<double, std::milli>(std::chrono::steady_clock::now() - syncStart).count());
viewSamples(); // keeps Fast Navigation's view of the camera current (UI tests read view_samples)
@@ -2075,6 +2076,8 @@ void App::openPath(const std::string& path) {
installBrushFile(path);
} else if (ext == kPackExtension) { // a pack: the review screen first, nothing is installed yet
reviewPackFile(path);
+ } else if (ext == ".sfdetail") { // Focus Detail (prototype): a streamed micro level
+ openFocusDetail(path);
} else if (ext == ".exr" || ext == ".sfvdm") { // a vector displacement map for the VDM brush
runCommand("vdm_set", {{"path", path}, {"swap_yz", vdmImport_.swapYZ}, {"flip_x", vdmImport_.flipX},
{"flip_y", vdmImport_.flipY}, {"flip_z", vdmImport_.flipZ}, {"scale", double(vdmImport_.scale)}});
diff --git a/src/app/App.h b/src/app/App.h
index 34fbdfa2..636eb96e 100644
--- a/src/app/App.h
+++ b/src/app/App.h
@@ -93,7 +93,7 @@ struct AppOptions {
// --benchmark: proof mode (ProofMode.cpp), then exit. Sizes in millions of triangles (empty: 5, 20, 33 and
// 100 when memory allows); outputs default to the current folder.
bool benchmark = false;
- std::string benchmarkScene; // --benchmark standard-15m | standard-33m | standard: a standard scene (ProofMode.h)
+ std::string benchmarkScene; // --benchmark standard-15m | standard-33m | standard | standard-1b: a standard scene (ProofMode.h)
std::string benchmarkSizes, benchmarkOut, benchmarkVideo;
bool benchmarkNoVideo = false, benchmarkVsync = false;
float benchmarkStrokeSeconds = 6.0f;
@@ -1482,6 +1482,13 @@ private:
void viewAxis(float yawDegrees, float pitchDegrees); // Numpad 1 / 3 / 7: orthographic until turned
void viewOrbitStep(float yawDegrees, float pitchDegrees);
void frameActiveSubtool();
+ // Focus Detail (prototype, FocusDetailApp.cpp): open a .sfdetail file; once a frame, the camera and brush.
+ void openFocusDetail(const std::string& path);
+ void updateFocusDetail();
+ glm::vec3 focusBrushPos_{0.0f}, focusBrushVel_{0.0f};
+ float focusBrushRadius_ = 0.0f;
+ double focusBrushTime_ = 0.0;
+ bool focusBrushValid_ = false;
SDL_Keymod blenderMods(SDL_Keymod keyboard) const; // Ctrl -> Alt (invert), the mask brush -> Ctrl (mask)
bool maskBrushActive() const;
void toggleMaskBrush();
diff --git a/src/app/FocusDetailApp.cpp b/src/app/FocusDetailApp.cpp
new file mode 100644
index 00000000..2fc824de
--- /dev/null
+++ b/src/app/FocusDetailApp.cpp
@@ -0,0 +1,70 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Focus Detail in the app (prototype; core/FocusDetail.h): opening a .sfdetail file (or `--focus-detail FILE`,
+// or the `standard-1b` benchmark scene) and, once a frame, telling the focus where the camera and brush are.
+#include "app/App.h"
+
+#include "core/FocusDetail.h"
+#include "core/Micro.h"
+
+#include <chrono>
+#include <cmath>
+#include <filesystem>
+
+namespace sf {
+
+void App::openFocusDetail(const std::string& path) {
+ Mesh m;
+ std::string error;
+ if (!FocusDetail::open(path, m, error)) {
+ status_ = "Focus Detail: " + error;
+ return;
+ }
+ const size_t tris = microTriangleCount(m);
+ doc_.setMesh(std::move(m), std::filesystem::path(path).stem().string());
+ frameActiveSubtool();
+ char buf[160];
+ std::snprintf(buf, sizeof buf, "Focus Detail (experimental): %.2f billion micro triangles; detail streams in where you look and sculpt",
+ double(tris) / 1e9);
+ status_ = buf;
+}
+
+void App::updateFocusDetail() {
+ Mesh& m = doc_.mesh;
+ if (!m.micro || !m.micro->focus || solverBusy()) return;
+ FocusDetail& fd = *m.micro->focus;
+ const float h = float(canvasHeight());
+ if (!(h > 0.0f)) return;
+ FocusView view;
+ view.valid = true;
+ view.eye = camera_.position();
+ view.viewProj = camera_.projection() * camera_.view();
+ // pixels per world unit at distance 1 (ortho: as a perspective view would at the target's depth)
+ view.focalPx = glm::distance(view.eye, camera_.target) / std::max(camera_.worldPerPixel(camera_.target, h), 1e-12f);
+ const double now = std::chrono::duration<double>(std::chrono::steady_clock::now().time_since_epoch()).count();
+ FocusBrush brush;
+ RayHit hit;
+ const bool busy = (asyncStroke_ && drag_ == Drag::Stroke) || doc_.bvhDirty || previewLod_ >= 0;
+ if (!busy && pick(mouseX_, mouseY_, hit)) {
+ const float dt = float(now - focusBrushTime_);
+ const glm::vec3 v = focusBrushValid_ && dt > 0.0f && dt < 0.25f ? (hit.position - focusBrushPos_) / dt : glm::vec3(0.0f);
+ focusBrushVel_ = glm::mix(focusBrushVel_, v, 0.5f); // smoothed over a couple of frames
+ focusBrushPos_ = hit.position;
+ focusBrushRadius_ = radiusPx_ * camera_.worldPerPixel(hit.position, h);
+ focusBrushTime_ = now;
+ focusBrushValid_ = true;
+ } else if (!(drag_ == Drag::Stroke) || now - focusBrushTime_ > 1.0) {
+ focusBrushValid_ = drag_ == Drag::Stroke && focusBrushValid_; // (a stroke keeps the last brush; its dabs promote their own tiles)
+ }
+ if (focusBrushValid_) {
+ brush.present = true;
+ brush.position = focusBrushPos_;
+ brush.velocity = focusBrushVel_;
+ brush.radius = focusBrushRadius_;
+ brush.mirrorX = symmetry_.x;
+ }
+ fd.update(m, view, brush, 3.0);
+}
+
+} // namespace sf
diff --git a/src/app/ProofMode.cpp b/src/app/ProofMode.cpp
index cb494af6..9f111563 100644
--- a/src/app/ProofMode.cpp
+++ b/src/app/ProofMode.cpp
@@ -14,6 +14,9 @@
// its own thread), and a JSON result with the hardware and version is written at the end.
#include "app/ProofMode.h"
+#include "core/FocusDetail.h"
+#include "core/Micro.h"
+
#include "app/App.h"
#include "app/ShellQuote.h"
#include "app/Theme.h"
@@ -81,6 +84,19 @@ std::vector<ProofMeshSize> proofStandardScene(const std::string& name) {
if (name == "standard-15m") return {s15};
if (name == "standard-33m") return {s33};
if (name == "standard") return {s15, s33};
+ if (name == "standard-1b") { // Focus Detail (prototype): ~1.02 billion micro triangles from a detail file
+ const ProofMeshSize s1b{1019, "1B Focus Detail", [](size_t) {
+ Mesh m;
+ std::string err;
+ const std::string path = focusStandard1bPath(err);
+ if (path.empty() || !FocusDetail::open(path, m, err)) { // (main checked the asset first)
+ std::fprintf(stderr, "standard-1b: %s\n", err.c_str());
+ return makeBenchHead(size_t(1) << 20);
+ }
+ return m;
+ }, 5};
+ return {s1b};
+ }
return {};
}
@@ -782,7 +798,7 @@ void App::proofBeforeSwap(int fbW, int fbH) {
Mesh m = s.build ? s.build(tris) : makeBenchSphere(tris);
SizeResult r;
r.label = s.label;
- r.triangles = m.triangleCount();
+ r.triangles = m.micro && m.micro->focus ? microTriangleCount(m) : m.triangleCount(); // (Focus Detail: its micro triangles)
r.vertices = m.vertexCount();
doc_.setMesh(std::move(m), "Benchmark");
r.buildSeconds = nowSeconds() - t0;
diff --git a/src/app/main.cpp b/src/app/main.cpp
index bf7657ee..84da4dfe 100644
--- a/src/app/main.cpp
+++ b/src/app/main.cpp
@@ -4,6 +4,7 @@
#include "app/Bench.h"
#include "app/Calibration.h"
#include "app/CrashLog.h"
+#include "core/FocusDetail.h"
#include "core/Pack.h"
#include <SDL3/SDL_main.h>
@@ -67,6 +68,8 @@ void usage() {
" JSON result and an MP4 of the run and exits (docs/benchmark.md)\n"
" --benchmark standard-15m The standard scene: a 15M triangle head, Clay S/M/L, Move, Smooth and a 240 px\n"
" brush (standard-33m: 33M; standard: both). The numbers on sculptforge.app/benchmarks\n"
+ " --benchmark standard-1b Experimental Focus Detail: a billion micro triangle head streamed from disk\n"
+ " --focus-detail FILE Experimental: open a Focus Detail file (.sfdetail)\n"
" --benchmark-sizes 5,20 Proof mode mesh sizes in millions of triangles\n"
" --benchmark-out FILE Where the JSON result goes (default: sculptforge-benchmark-<time>.json here)\n"
" --benchmark-video FILE Where the MP4 goes; --benchmark-no-video records none\n"
@@ -166,12 +169,13 @@ int main(int argc, char** argv) {
if (i + 1 < argc && std::string(argv[i + 1]).rfind("standard", 0) == 0) {
options.benchmarkScene = argv[++i];
if (options.benchmarkScene != "standard" && options.benchmarkScene != "standard-15m" &&
- options.benchmarkScene != "standard-33m") {
- std::fprintf(stderr, "Unknown benchmark scene %s (standard-15m, standard-33m or standard)\n", argv[i]);
+ options.benchmarkScene != "standard-33m" && options.benchmarkScene != "standard-1b") {
+ std::fprintf(stderr, "Unknown benchmark scene %s (standard-15m, standard-33m, standard or standard-1b)\n", argv[i]);
return 1;
}
}
}
+ else if (a == "--focus-detail") options.openPath = next(); // a .sfdetail file (prototype)
else if (a == "--benchmark-sizes") options.benchmarkSizes = next();
else if (a == "--benchmark-out") options.benchmarkOut = next();
else if (a == "--benchmark-video") options.benchmarkVideo = next();
@@ -194,6 +198,17 @@ int main(int argc, char** argv) {
if (!sizeGiven) options.width = 1600, options.height = 960;
isolateSettings();
}
+ if (options.benchmarkScene == "standard-1b") { // the asset first (written once, a few seconds), before any window
+#if defined(_WIN32)
+ std::fprintf(stderr, "standard-1b (experimental Focus Detail) needs Linux or macOS for now\n");
+ return 1;
+#endif
+ std::string error;
+ if (sf::focusStandard1bPath(error, [](const std::string& s) { std::fprintf(stderr, "%s\n", s.c_str()); }).empty()) {
+ std::fprintf(stderr, "standard-1b: %s\n", error.c_str());
+ return 1;
+ }
+ }
if (options.benchmark && !sizeGiven) options.width = 1920, options.height = 1080;
if (options.benchmark || !options.timelapsePath.empty()) isolateSettings(); // the user's settings and files untouched
if (!options.timelapsePath.empty() && !sizeGiven) options.width = 1920, options.height = 1080;
diff --git a/src/core/Benchmark.cpp b/src/core/Benchmark.cpp
index 05adc33b..0deae298 100644
--- a/src/core/Benchmark.cpp
+++ b/src/core/Benchmark.cpp
@@ -6,6 +6,7 @@
#include "core/Commands.h"
#include "core/Document.h"
#include "core/Micro.h"
+#include "core/FocusDetail.h"
#include "core/Multires.h"
#include "core/Polypaint.h"
#include "core/Primitives.h"
@@ -328,6 +329,12 @@ bool parseBenchArg(int argc, char** argv, int& i, BenchCommandLine& cl) {
} else if (a == "--bench-micro-level") {
cl.options.micro = true;
cl.options.microLevel = std::clamp(std::atoi(next().c_str()), 1, 7);
+ } else if (a == "--bench-focus") {
+ cl.run = true;
+ cl.options.focus = true;
+ } else if (a == "--bench-focus-base") {
+ cl.options.focus = true;
+ cl.options.focusBase = size_t(std::max(100, std::atoi(next().c_str())));
} else if (a == "--bench-out") {
cl.outPath = next();
} else {
@@ -661,7 +668,8 @@ std::string runMicroBenchmark(const BenchOptions& o, const BenchGpu* gpu) {
std::string runBenchmark(const BenchOptions& o, const BenchGpu* gpu) {
if (o.dyntopo) return runDyntopoBenchmark(o, gpu);
- if (o.micro) return runMicroBenchmark(o, gpu);
+ if (o.micro && !o.focus) return runMicroBenchmark(o, gpu);
+ if (o.focus) return runFocusBenchmark(o);
auto log = [&](const std::string& s) {
if (o.log) o.log(s);
};
diff --git a/src/core/Benchmark.h b/src/core/Benchmark.h
index a337dcc6..b9532f07 100644
--- a/src/core/Benchmark.h
+++ b/src/core/Benchmark.h
@@ -50,6 +50,10 @@ struct BenchOptions {
// a base sphere at micro level `microLevel` (4: 256 micro triangles per base triangle).
bool micro = false;
int microLevel = 4;
+ // Focus Detail instead (--bench-focus, core/FocusDetail.h): the standard-1b asset (a billion micro triangles
+ // streamed from a detail file), or a head of `focusBase` base triangles at micro level `microLevel`.
+ bool focus = false;
+ size_t focusBase = 0;
std::function<void(const std::string&)> log; // progress lines (optional)
std::function<void(const std::string&)> partial; // the report so far, after each size (optional)
};
diff --git a/src/core/FocusBench.cpp b/src/core/FocusBench.cpp
new file mode 100644
index 00000000..81d4149e
--- /dev/null
+++ b/src/core/FocusBench.cpp
@@ -0,0 +1,306 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// The Focus Detail benchmark (core/FocusDetail.h: runFocusBenchmark; sculptforge-cli --bench-focus).
+#include "core/Benchmark.h"
+#include "core/BVH.h"
+#include "core/Document.h"
+#include "core/FocusDetail.h"
+#include "core/Micro.h"
+#include "core/MicroStore.h"
+#include "core/Sculpt.h"
+#include "core/ThreadPool.h"
+
+#include <glm/gtc/matrix_transform.hpp>
+
+#include <algorithm>
+#include <chrono>
+#include <cmath>
+#include <cstdio>
+#include <filesystem>
+#include <sstream>
+
+namespace sf {
+
+namespace {
+
+using Clock = std::chrono::steady_clock;
+double msSince(Clock::time_point t) { return std::chrono::duration<double, std::milli>(Clock::now() - t).count(); }
+
+std::string fmt(const char* f, double v) {
+ char b[64];
+ std::snprintf(b, sizeof b, f, v);
+ return b;
+}
+std::string mb(double bytes) { return fmt("%.0f MB", bytes / 1e6); }
+
+double percentile(std::vector<double> v, double p) {
+ if (v.empty()) return 0.0;
+ std::sort(v.begin(), v.end());
+ return v[std::min(v.size() - 1, size_t(std::ceil(p * double(v.size()))) - (p > 0 ? 1 : 0))];
+}
+
+// A 1920 x 1080 view, 40 degrees vertical, from `eye` towards `target`.
+FocusView makeView(const glm::vec3& eye, const glm::vec3& target, float errorPx = 0.5f) {
+ const float fov = glm::radians(40.0f), w = 1920.0f, h = 1080.0f;
+ FocusView v;
+ v.valid = true;
+ v.eye = eye;
+ v.viewProj = glm::perspective(fov, w / h, 0.01f, 100.0f) * glm::lookAt(eye, target, glm::vec3(0, 1, 0));
+ v.focalPx = h / (2.0f * std::tan(fov / 2.0f));
+ v.errorPx = errorPx;
+ return v;
+}
+
+struct StrokeResult {
+ std::vector<double> dab, frame; // per dab: the brush (with any promotion it waited for); dab + update
+ uint64_t syncPromoted = 0;
+ double undoMs = 0, redoMs = 0;
+ size_t undoBytes = 0;
+};
+
+// A figure eight over the face, `dabs` dabs, 240 samples a second. The focus is updated before each dab with the
+// brush's position and velocity (as the app does once a frame), unless `predict` is off (then only where the
+// brush is now, no lookahead).
+StrokeResult runStroke(Document& doc, FocusDetail& fd, BrushType type, bool move, float radius, int dabs, bool predict) {
+ StrokeResult r;
+ Mesh& m = doc.mesh;
+ Sculptor& sc = doc.sculptor;
+ sc.beginStroke(m, move ? "Move" : "Clay");
+ Symmetry sym; // X on, as the app starts
+ const FocusView view = makeView(glm::vec3(0.0f, 0.1f, 2.2f), glm::vec3(0.0f, 0.1f, 0.0f));
+ const uint64_t sync0 = fd.stats().syncPromoted;
+ const float lookahead = fd.lookaheadSeconds;
+ if (!predict) fd.lookaheadSeconds = 0.0f;
+ glm::vec3 prev(0.0f);
+ bool havePrev = false;
+ if (predict) { // the cursor hovers over the start before the pen goes down (a few frames)
+ RayHit hit;
+ if (surfaceRaycast(m, doc.bvh, glm::vec3(0.0f, 0.1f, 5.0f), glm::vec3(0, 0, -1), hit)) {
+ FocusBrush b;
+ b.present = true;
+ b.position = hit.position;
+ b.radius = radius;
+ b.mirrorX = sym.x;
+ for (int f = 0; f < 3; ++f) fd.update(m, view, b, 4.0);
+ }
+ }
+ for (int i = 0; i < dabs; ++i) {
+ const float a = float(i) / float(dabs) * 6.2831853f * 1.5f;
+ const float x = 0.32f * std::sin(a), y = 0.1f + 0.22f * std::sin(2.0f * a);
+ RayHit hit;
+ if (!surfaceRaycast(m, doc.bvh, glm::vec3(x, y, 5.0f), glm::vec3(0, 0, -1), hit)) continue;
+ FocusBrush b;
+ b.present = true;
+ b.position = hit.position;
+ b.velocity = havePrev ? (hit.position - prev) * 240.0f : glm::vec3(0.0f);
+ b.radius = radius;
+ b.mirrorX = sym.x;
+ prev = hit.position;
+ havePrev = true;
+ const auto tf = Clock::now();
+ fd.update(m, view, b, 2.0);
+ const auto td = Clock::now();
+ DabParams p;
+ p.center = hit.position;
+ p.radius = radius;
+ p.strength = move ? 0.25f : 0.35f;
+ if (move) {
+ if (i == 0) sc.beginGrab(m, doc.bvh, p, sym);
+ else sc.grab(m, doc.bvh, glm::vec3(0.0f, 0.0f, 0.0008f));
+ } else {
+ sc.dab(m, doc.bvh, type, p, sym);
+ }
+ r.dab.push_back(msSince(td));
+ r.frame.push_back(msSince(tf));
+ if (move && i == dabs / 3) break; // a Move stroke grabs once and drags: a third of the path
+ }
+ fd.lookaheadSeconds = lookahead;
+ r.syncPromoted = fd.stats().syncPromoted - sync0;
+ UndoRecord rec = sc.endStroke();
+ r.undoBytes = rec.approxBytes();
+ doc.undo.push(std::move(rec));
+ auto t = Clock::now();
+ doc.undo.undo(m, doc.bvh);
+ r.undoMs = msSince(t);
+ t = Clock::now();
+ doc.undo.redo(m, doc.bvh);
+ r.redoMs = msSince(t);
+ m.micro->clearDirty();
+ return r;
+}
+
+} // namespace
+
+std::string runFocusBenchmark(const BenchOptions& o) {
+ auto log = [&](const std::string& s) {
+ if (o.log) o.log(s);
+ };
+ std::ostringstream md;
+ namespace fs = std::filesystem;
+ std::string err, path;
+ double writeMs = 0;
+ auto tw = Clock::now();
+ if (o.focusBase == 0) {
+ path = focusStandard1bPath(err, log);
+ } else {
+ path = (fs::path(MicroStore::swapFolder()) / ("bench-focus-" + std::to_string(o.focusBase) + ".sfdetail")).string();
+ std::error_code ec;
+ fs::create_directories(MicroStore::swapFolder(), ec);
+ if (!focusWriteSyntheticHead(path, o.focusBase, o.microLevel, err)) path.clear();
+ }
+ writeMs = msSince(tw);
+ if (path.empty()) return "Focus Detail benchmark: " + err + "\n";
+ const double fileBytes = double(fs::file_size(path));
+ md << "Threads: " << ThreadPool::shared().threads() << ". Asset: `" << fs::path(path).filename().string() << "`, "
+ << mb(fileBytes) << " on disk";
+ if (writeMs > 2000) md << " (written in " << fmt("%.1f s", writeMs / 1000) << ")";
+ md << ".\n\n";
+
+ // --- Open ---
+ const size_t rss0 = resetPeakResidentBytes();
+ log("Opening " + path);
+ auto t = Clock::now();
+ Mesh m0;
+ std::shared_ptr<FocusDetail> fd = FocusDetail::open(path, m0, err);
+ if (!fd) return "Focus Detail benchmark: " + err + "\n";
+ const double openMs = msSince(t);
+ t = Clock::now();
+ Document doc;
+ doc.setMesh(std::move(m0));
+ const double docMs = msSince(t);
+ Mesh& m = doc.mesh;
+ MicroMesh& mm = *m.micro;
+ m.micro->clearDirty();
+ const size_t rssOpen = residentBytes();
+ const double points = double(mm.baseTriangles) * double(mm.samples);
+ md << "### " << fmt("%.3f", double(mm.microTriangles()) / 1e9) << " billion micro triangles (base " << mm.baseTriangles
+ << " triangles, micro level " << mm.level << ", " << fmt("%.0f", points / 1e6) << "M grid points)\n\n";
+ md << "| Step | Result |\n|---|---:|\n";
+ md << "| Open (file, base, coarse grids, upsample) | " << fmt("%.0f ms", openMs) << " |\n";
+ md << "| Scene setup (BVH, topology) | " << fmt("%.0f ms", docMs) << " |\n";
+ md << "| **Open total** | **" << fmt("%.2f s", (openMs + docMs) / 1000) << "** |\n";
+ md << "| RSS after open | " << mb(double(rssOpen)) << " (+" << mb(double(rssOpen - std::min(rssOpen, rss0))) << ") |\n";
+ md << "| Bytes on disk per micro triangle | " << fmt("%.2f", fileBytes / double(mm.microTriangles())) << " |\n";
+
+ // --- Promotion latency: one tile at a time (a brush waiting), then in parallel batches ---
+ {
+ std::vector<double> one;
+ for (uint32_t k = 0; k < 256; ++k) {
+ const uint32_t tile = uint32_t((uint64_t(k) * 2654435761u) % mm.baseTriangles);
+ if (fd->tileLevel(tile)) continue;
+ const uint32_t tt[1] = {tile};
+ t = Clock::now();
+ fd->ensureFull(mm, tt, false);
+ one.push_back(msSince(t));
+ }
+ md << "| Promote one tile (decompress + edges + markDirty), p50 / p99 | " << fmt("%.3f", percentile(one, 0.5)) << " / "
+ << fmt("%.3f ms", percentile(one, 0.99)) << " |\n";
+ }
+ m.micro->clearDirty();
+
+ // --- Orbit: a full turn close to the face, then a zoom into it; the focus follows the camera ---
+ log("Orbit");
+ fd->budgetBytes = size_t(512) << 20;
+ std::vector<double> upd;
+ const FocusStats s0 = fd->stats();
+ const int frames = 240;
+ size_t maxPending = 0;
+ for (int f = 0; f < frames; ++f) {
+ const float a = float(f) / float(frames) * 6.2831853f;
+ const float dist = f < frames / 2 ? 2.2f : 2.2f - 1.4f * float(f - frames / 2) / float(frames / 2);
+ const FocusView v = makeView(glm::vec3(dist * std::sin(a), 0.1f, dist * std::cos(a)), glm::vec3(0.0f, 0.1f, 0.0f));
+ fd->update(m, v, FocusBrush{}, 4.0);
+ upd.push_back(fd->stats().lastUpdateMs);
+ maxPending = std::max(maxPending, fd->stats().lastPending);
+ m.micro->clearDirty(); // (what the renderer's sync does)
+ }
+ const FocusStats s1 = fd->stats();
+ const double orbitTiles = double(s1.promoted - s0.promoted);
+ md << "| Orbit + zoom (" << frames << " frames, 4 ms budget): update p50 / p99 / max | " << fmt("%.2f", percentile(upd, 0.5))
+ << " / " << fmt("%.2f", percentile(upd, 0.99)) << " / " << fmt("%.2f ms", percentile(upd, 1.0)) << " |\n";
+ md << "| Orbit: tiles promoted / demoted, most waiting in a frame | " << fmt("%.0f", orbitTiles) << " / "
+ << fmt("%.0f", double(s1.demoted - s0.demoted)) << ", " << maxPending << " |\n";
+ md << "| Promotion throughput (parallel) | " << fmt("%.0f tiles/s", double(s1.promoted) / std::max(s1.promoteMs, 1e-3) * 1000.0)
+ << " = " << fmt("%.0f M micro triangles/s", double(s1.promoted) * double(mm.n * mm.n) / std::max(s1.promoteMs, 1e-3) / 1e3) << " |\n";
+ md << "| Full tiles after the zoom | " << s1.fullTiles << " of " << s1.tiles << " (" << mb(double(s1.fullBytes)) << ", budget "
+ << mb(double(fd->budgetBytes)) << ") |\n";
+ md << "| RSS, focus on | " << mb(double(residentBytes())) << " |\n\n";
+
+ // --- Strokes ---
+ md << "| Stroke (X symmetry, 240 Hz) | dabs | dab p50 | **dab p99** | max | dab + focus update p99 | tiles promoted while waiting | undo | redo | undo step |\n";
+ md << "|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|\n";
+ struct Row {
+ const char* name;
+ BrushType type;
+ bool move;
+ float radius;
+ bool predict;
+ bool alongNormals = true; // Move: the micro surface along its normals (writes tiles), else the base cage
+ };
+ const Row rows[] = {{"Clay r=0.03 (~25 px)", BrushType::Clay, false, 0.03f, true},
+ {"Clay r=0.08 (~60 px)", BrushType::Clay, false, 0.08f, true},
+ {"Clay r=0.16 (~120 px)", BrushType::Clay, false, 0.16f, true},
+ {"Move r=0.1 (~80 px), micro surface", BrushType::Move, true, 0.10f, true},
+ {"Move r=0.1, base cage (default)", BrushType::Move, true, 0.10f, true, false},
+ {"Clay r=0.08, no prediction", BrushType::Clay, false, 0.08f, false}};
+ int stroke = 0;
+ for (const Row& row : rows) {
+ if (!o.brushFilter.empty() && std::string(row.name).find(o.brushFilter) == std::string::npos) continue;
+ log(std::string(" ") + row.name);
+ // Each stroke starts on a cold region: demote everything first (a fresh part of the model).
+ fd->budgetBytes = 1;
+ fd->update(m, FocusView{}, FocusBrush{}, 1e9);
+ fd->update(m, FocusView{}, FocusBrush{}, 1e9);
+ fd->update(m, FocusView{}, FocusBrush{}, 1e9);
+ fd->budgetBytes = size_t(512) << 20;
+ m.micro->clearDirty();
+ doc.sculptor.microMoveAlongNormals = row.alongNormals;
+ const StrokeResult r = runStroke(doc, *fd, row.type, row.move, row.radius, std::max(o.dabsPerStroke, 40), row.predict);
+ md << "| " << row.name << " | " << r.dab.size() << " | " << fmt("%.2f", percentile(r.dab, 0.5)) << " | **"
+ << fmt("%.2f", percentile(r.dab, 0.99)) << "** | " << fmt("%.2f", percentile(r.dab, 1.0)) << " | "
+ << fmt("%.2f", percentile(r.frame, 0.99)) << " | " << r.syncPromoted << " | " << fmt("%.1f", r.undoMs) << " | "
+ << fmt("%.1f", r.redoMs) << " | " << fmt("%.0f KB", double(r.undoBytes) / 1e3) << " |\n";
+ ++stroke;
+ if (o.partial) o.partial(md.str());
+ }
+ md << "\n";
+
+ // --- Save: the changed tiles only ---
+ {
+ const FocusStats s = fd->stats();
+ const double before = double(fs::file_size(path));
+ t = Clock::now();
+ const bool ok = fd->save(m, err);
+ const double saveMs = msSince(t);
+ const double after = double(fs::file_size(path));
+ md << "| Save | Result |\n|---|---:|\n";
+ md << "| Incremental save (" << s.dirtyTiles << " dirty + " << s.parkedTiles << " parked tiles) | "
+ << (ok ? fmt("%.0f ms", saveMs) : "failed: " + err) << ", " << fmt("%.1f MB appended", (after - before) / 1e6) << " |\n";
+ if (writeMs > 2000) md << "| Writing the whole asset (for comparison) | " << fmt("%.1f s", writeMs / 1000) << " |\n";
+ }
+
+ // --- Focus off: every tile decompressed (what a plain micro level holds) ---
+ {
+ log("Every tile full");
+ std::vector<uint32_t> all(mm.baseTriangles);
+ for (uint32_t k = 0; k < all.size(); ++k) all[k] = k;
+ fd->budgetBytes = size_t(1) << 40;
+ t = Clock::now();
+ fd->ensureFull(mm, all, false);
+ const double allMs = msSince(t);
+ m.micro->clearDirty();
+ const size_t rssAll = residentBytes(), peak = peakResidentBytes();
+ // The renderer's GPU copy (render/MicroRenderer.h): 2 bytes of displacement + 4 of shading per grid point;
+ // with per-tile GPU levels (docs/design/perf-moonshots.md #8) coarse tiles would hold their 45 points only.
+ const double cs = double((8 + 1) * (8 + 2) / 2), full = double(s1.fullTiles);
+ const double gpuNow = points * 6.0, gpuTiled = (double(mm.baseTriangles) - full) * cs * 6.0 + full * double(mm.samples) * 6.0;
+ md << "| Every tile decompressed (focus off) | " << fmt("%.2f s", allMs / 1000) << " |\n";
+ md << "| RSS, every tile full | " << mb(double(rssAll)) << " (peak " << mb(double(peak)) << ") |\n";
+ md << "| GPU buffers (estimate), today's renderer | " << mb(gpuNow) << " |\n";
+ md << "| GPU buffers (estimate), per-tile levels with this focus | " << mb(gpuTiled) << " |\n";
+ }
+ return md.str();
+}
+
+} // namespace sf
diff --git a/src/core/FocusDetail.cpp b/src/core/FocusDetail.cpp
new file mode 100644
index 00000000..c42c849e
--- /dev/null
+++ b/src/core/FocusDetail.cpp
@@ -0,0 +1,880 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Focus Detail (core/FocusDetail.h; design: docs/design/focus-detail.md).
+#include "core/FocusDetail.h"
+
+#include "core/Benchmark.h"
+#include "core/Micro.h"
+#include "core/MicroStore.h"
+#include "core/ThreadPool.h"
+
+#include <algorithm>
+#include <chrono>
+#include <cmath>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <filesystem>
+
+#if defined(SF_HAVE_ZSTD)
+#include <zstd.h>
+#endif
+
+#if defined(__unix__) || defined(__APPLE__)
+#define SF_FOCUS_POSIX 1
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#endif
+
+namespace sf {
+
+namespace {
+
+using Clock = std::chrono::steady_clock;
+double msSince(Clock::time_point t) { return std::chrono::duration<double, std::milli>(Clock::now() - t).count(); }
+
+constexpr char kMagic[8] = {'S', 'F', 'D', 'E', 'T', 'A', 'I', 'L'};
+constexpr uint32_t kVersion = 1, kHeaderBytes = 128;
+constexpr uint32_t kCodecZstd = 1, kCodecPack = 2;
+
+uint32_t defaultCodec() {
+#if defined(SF_HAVE_ZSTD)
+ return kCodecZstd;
+#else
+ return kCodecPack;
+#endif
+}
+
+uint64_t fnv1a(const uint8_t* p, size_t n) {
+ uint64_t h = 1469598103934665603ull;
+ for (size_t i = 0; i < n; ++i) h = (h ^ p[i]) * 1099511628211ull;
+ return h;
+}
+
+// Row-delta coding of a tile (as the undo journal): each point less the one before it in its row, a row's first
+// less the point below it. Wrapping 16-bit arithmetic, so it is exact for every value.
+void deltaEncode(const int16_t* v, uint32_t n, int16_t* out) {
+ uint32_t p = 0;
+ for (uint32_t j = 0; j <= n; ++j)
+ for (uint32_t i = 0; i + j <= n; ++i, ++p) {
+ const int prev = p == 0 ? 0 : i > 0 ? v[p - 1] : v[MicroMesh::index(0, j - 1, n)];
+ out[p] = int16_t(uint16_t(uint16_t(v[p]) - uint16_t(prev)));
+ }
+}
+
+void deltaDecode(int16_t* v, uint32_t n) { // in place
+ uint32_t p = 0;
+ for (uint32_t j = 0; j <= n; ++j)
+ for (uint32_t i = 0; i + j <= n; ++i, ++p) {
+ const int prev = p == 0 ? 0 : i > 0 ? v[p - 1] : v[MicroMesh::index(0, j - 1, n)];
+ v[p] = int16_t(uint16_t(uint16_t(v[p]) + uint16_t(prev)));
+ }
+}
+
+// `count` values (`tiles` x `samples`, for the pack codec's per-tile ranges) into `out` (replaced).
+void compressValues(uint32_t codec, const int16_t* values, size_t tiles, uint32_t samples, std::vector<uint8_t>& out) {
+ out.clear();
+#if defined(SF_HAVE_ZSTD)
+ if (codec == kCodecZstd) {
+ thread_local ZSTD_CCtx* cctx = ZSTD_createCCtx();
+ const size_t bytes = tiles * samples * sizeof(int16_t);
+ out.resize(ZSTD_compressBound(bytes));
+ const size_t r = ZSTD_compressCCtx(cctx, out.data(), out.size(), values, bytes, 3);
+ out.resize(ZSTD_isError(r) ? 0 : r);
+ return;
+ }
+#endif
+ (void)codec;
+ microPackTiles(values, tiles, samples, out);
+}
+
+bool decompressValues(uint32_t codec, const uint8_t* data, size_t size, size_t tiles, uint32_t samples, int16_t* out) {
+#if defined(SF_HAVE_ZSTD)
+ if (codec == kCodecZstd) {
+ thread_local ZSTD_DCtx* dctx = ZSTD_createDCtx();
+ const size_t bytes = tiles * samples * sizeof(int16_t);
+ const size_t r = ZSTD_decompressDCtx(dctx, out, bytes, data, size);
+ return !ZSTD_isError(r) && r == bytes;
+ }
+#endif
+ if (codec != kCodecPack) return false;
+ std::fill_n(out, tiles * samples, int16_t(0));
+ return microUnpackTiles(data, size, tiles, samples, out);
+}
+
+void compressTile(uint32_t codec, const int16_t* v, uint32_t n, uint32_t samples, std::vector<uint8_t>& out) {
+ thread_local std::vector<int16_t> d;
+ d.resize(samples);
+ deltaEncode(v, n, d.data());
+ compressValues(codec, d.data(), 1, samples, out);
+}
+
+bool decompressTile(uint32_t codec, const uint8_t* data, size_t size, uint32_t n, uint32_t samples, int16_t* out) {
+ if (!decompressValues(codec, data, size, 1, samples, out)) return false;
+ deltaDecode(out, n);
+ return true;
+}
+
+// The coarse grid (cn segments) as a subsample of the full grid (n segments).
+void subsample(const int16_t* full, uint32_t n, uint32_t cn, int16_t* coarse) {
+ const uint32_t s = n / cn;
+ for (uint32_t J = 0; J <= cn; ++J)
+ for (uint32_t I = 0; I + J <= cn; ++I) coarse[MicroMesh::index(I, J, cn)] = full[MicroMesh::index(I * s, J * s, n)];
+}
+
+// The full grid from the coarse one: exact at the coarse points, linear on each coarse micro triangle in between,
+// in integers (a shared edge's points depend only on its two coarse ends, the same sum from either side).
+void upsample(const int16_t* coarse, uint32_t cn, uint32_t n, int16_t* full) {
+ const int s = int(n / cn);
+ int sh = 0;
+ while ((1 << sh) < s) ++sh;
+ uint32_t p = 0;
+ for (uint32_t j = 0; j <= n; ++j)
+ for (uint32_t i = 0; i + j <= n; ++i, ++p) {
+ const uint32_t I = i >> sh, J = j >> sh;
+ const int fi = int(i) & (s - 1), fj = int(j) & (s - 1);
+ int sum;
+ if (fi + fj <= s) {
+ sum = (s - fi - fj) * coarse[MicroMesh::index(I, J, cn)] + (fi ? fi * coarse[MicroMesh::index(I + 1, J, cn)] : 0) +
+ (fj ? fj * coarse[MicroMesh::index(I, J + 1, cn)] : 0);
+ } else {
+ sum = (fi + fj - s) * coarse[MicroMesh::index(I + 1, J + 1, cn)] + (s - fi) * coarse[MicroMesh::index(I, J + 1, cn)] +
+ (s - fj) * coarse[MicroMesh::index(I + 1, J, cn)];
+ }
+ // floor((sum + s / 2) / s), also for negative sums
+ const int q = sum + s / 2;
+ full[p] = int16_t(q >= 0 ? q >> sh : -((-q + s - 1) >> sh));
+ }
+}
+
+struct TileInfo {
+ uint16_t range = 0;
+ float coarseErr = 0;
+};
+
+// Range and coarse error of a full tile (its coarse grid into `coarse`).
+TileInfo analyse(const int16_t* full, uint32_t n, uint32_t samples, uint32_t cn, float step, int16_t* coarse) {
+ thread_local std::vector<int16_t> up;
+ up.resize(samples);
+ subsample(full, n, cn, coarse);
+ upsample(coarse, cn, n, up.data());
+ int r = 0, e = 0;
+ for (uint32_t k = 0; k < samples; ++k) {
+ r = std::max(r, std::abs(int(full[k])));
+ e = std::max(e, std::abs(int(full[k]) - int(up[k])));
+ }
+ return {uint16_t(std::min(r, 32767)), float(e) * step};
+}
+
+template <class T> void put(std::vector<uint8_t>& b, size_t at, T v) { std::memcpy(b.data() + at, &v, sizeof v); }
+template <class T> T get(const uint8_t* b, size_t at) {
+ T v;
+ std::memcpy(&v, b + at, sizeof v);
+ return v;
+}
+
+struct Header {
+ uint32_t codec = 0, level = 0, coarseLevel = 0, tris = 0, verts = 0, chunkTiles = 0;
+ float step = 0;
+ uint64_t baseOffset = 0, baseBytes = 0, dirOffset = 0, dirBytes = 0, generation = 0, garbage = 0, dirHash = 0;
+};
+
+std::vector<uint8_t> encodeHeader(const Header& h) {
+ std::vector<uint8_t> b(kHeaderBytes, 0);
+ std::memcpy(b.data(), kMagic, 8);
+ put(b, 8, kVersion);
+ put(b, 12, h.codec);
+ put(b, 16, h.level);
+ put(b, 20, h.coarseLevel);
+ put(b, 24, h.tris);
+ put(b, 28, h.verts);
+ put(b, 32, h.step);
+ put(b, 36, h.chunkTiles);
+ put(b, 40, h.baseOffset);
+ put(b, 48, h.baseBytes);
+ put(b, 56, h.dirOffset);
+ put(b, 64, h.dirBytes);
+ put(b, 72, h.generation);
+ put(b, 80, h.garbage);
+ put(b, 88, h.dirHash);
+ return b;
+}
+
+bool decodeHeader(const uint8_t* b, size_t size, Header& h, std::string& error) {
+ if (size < kHeaderBytes || std::memcmp(b, kMagic, 8) != 0) {
+ error = "Not a SculptForge detail file";
+ return false;
+ }
+ if (get<uint32_t>(b, 8) != kVersion) {
+ error = "Detail file version not supported";
+ return false;
+ }
+ h.codec = get<uint32_t>(b, 12);
+ h.level = get<uint32_t>(b, 16);
+ h.coarseLevel = get<uint32_t>(b, 20);
+ h.tris = get<uint32_t>(b, 24);
+ h.verts = get<uint32_t>(b, 28);
+ h.step = get<float>(b, 32);
+ h.chunkTiles = get<uint32_t>(b, 36);
+ h.baseOffset = get<uint64_t>(b, 40);
+ h.baseBytes = get<uint64_t>(b, 48);
+ h.dirOffset = get<uint64_t>(b, 56);
+ h.dirBytes = get<uint64_t>(b, 64);
+ h.generation = get<uint64_t>(b, 72);
+ h.garbage = get<uint64_t>(b, 80);
+ h.dirHash = get<uint64_t>(b, 88);
+ if (h.codec != kCodecZstd && h.codec != kCodecPack) {
+ error = "Detail file codec unknown";
+ return false;
+ }
+#if !defined(SF_HAVE_ZSTD)
+ if (h.codec == kCodecZstd) {
+ error = "This build can't read zstd detail files";
+ return false;
+ }
+#endif
+ if (h.level < 1 || h.level > uint32_t(MicroMesh::kMaxLevel) || h.coarseLevel >= h.level || h.chunkTiles == 0 ||
+ !(h.step > 0.0f) || !std::isfinite(h.step) || h.tris == 0 || h.verts == 0) {
+ error = "Detail file header is invalid";
+ return false;
+ }
+ return true;
+}
+
+// The edges of tile t from `from` into the tile across each, for the neighbours `copyInto` picks.
+template <class Pick>
+void shareEdges(const std::vector<glm::uvec3>& triangles, MicroMesh& mm, uint32_t t, Pick&& copyInto) {
+ const uint32_t n = mm.n, S = mm.samples;
+ const glm::uvec3 tt = triangles[t];
+ for (int ed = 0; ed < 3; ++ed) {
+ const uint32_t nb = mm.edgeNeighbor[size_t(t) * 3 + size_t(ed)];
+ if (nb == MicroMesh::kNone) continue;
+ const uint32_t u = nb >> 2;
+ const int f = int(nb & 3);
+ int dir = copyInto(u); // +1: t's edge into u, -1: u's edge into t, 0: nothing
+ if (!dir) continue;
+ int16_t* g = mm.disp.data() + size_t(t) * S;
+ int16_t* h = mm.disp.data() + size_t(u) * S;
+ const bool reversed = triangles[u][f] != tt[ed];
+ for (uint32_t k = 1; k < n; ++k) {
+ uint32_t i, j, i2, j2;
+ MicroMesh::edgePoint(ed, k, n, i, j);
+ MicroMesh::edgePoint(f, reversed ? n - k : k, n, i2, j2);
+ int16_t& a = g[MicroMesh::index(i, j, n)];
+ int16_t& c = h[MicroMesh::index(i2, j2, n)];
+ if (dir > 0) c = a;
+ else a = c;
+ }
+ mm.markDirty(dir > 0 ? u : t);
+ }
+}
+
+} // namespace
+
+// ---------------------------------------------------------------------------
+// Writing
+// ---------------------------------------------------------------------------
+
+bool FocusDetail::write(const std::string& path, const Mesh& baseIn, int level, float step, const TileFn& tile,
+ std::string& error) {
+ if (level <= kCoarseLevel || level > MicroMesh::kMaxLevel) {
+ error = "Focus Detail needs a micro level above " + std::to_string(kCoarseLevel);
+ return false;
+ }
+ Mesh base = baseIn;
+ if (base.normals.size() != base.vertexCount()) base.recomputeAllNormals();
+ const uint32_t T = uint32_t(base.triangleCount()), V = uint32_t(base.vertexCount());
+ const uint32_t n = 1u << level, S = (n + 1) * (n + 2) / 2, cn = 1u << kCoarseLevel, CS = (cn + 1) * (cn + 2) / 2;
+ std::FILE* f = std::fopen(path.c_str(), "wb");
+ if (!f) {
+ error = "Can't write " + path;
+ return false;
+ }
+ Header h;
+ h.codec = defaultCodec();
+ h.level = uint32_t(level);
+ h.coarseLevel = kCoarseLevel;
+ h.tris = T;
+ h.verts = V;
+ h.chunkTiles = kChunk;
+ h.step = step;
+ h.baseOffset = kHeaderBytes;
+ h.baseBytes = uint64_t(V) * 24 + uint64_t(T) * 12;
+ bool ok = std::fwrite(encodeHeader(h).data(), 1, kHeaderBytes, f) == kHeaderBytes;
+ ok = ok && std::fwrite(base.positions.data(), 12, V, f) == V;
+ ok = ok && std::fwrite(base.normals.data(), 12, V, f) == V;
+ ok = ok && std::fwrite(base.triangles.data(), 12, T, f) == T;
+ uint64_t offset = kHeaderBytes + h.baseBytes;
+ std::vector<Entry> dir(T);
+ const uint32_t chunks = (T + kChunk - 1) / kChunk;
+ std::vector<ChunkEntry> chunkDir(chunks);
+ // Batches of whole chunks: tiles generated and compressed in parallel, written in order.
+ const uint32_t batch = kChunk * 4;
+ std::vector<std::vector<uint8_t>> blobs(batch), chunkBlobs(4);
+ std::vector<int16_t> coarse(size_t(batch) * CS);
+ for (uint32_t b0 = 0; ok && b0 < T; b0 += batch) {
+ const uint32_t b1 = std::min(T, b0 + batch);
+ poolFor(b1 - b0, [&](size_t b, size_t e) {
+ std::vector<int16_t> v(S);
+ MicroTriangle Tr;
+ for (size_t k = b; k < e; ++k) {
+ const uint32_t t = b0 + uint32_t(k);
+ Tr.set(base, t, n);
+ std::fill(v.begin(), v.end(), int16_t(0));
+ tile(Tr, t, v.data());
+ const TileInfo info = analyse(v.data(), n, S, cn, step, coarse.data() + k * CS);
+ dir[t].range = info.range;
+ dir[t].coarseErr = info.coarseErr;
+ compressTile(h.codec, v.data(), n, S, blobs[k]);
+ }
+ }, 16);
+ for (uint32_t t = b0; ok && t < b1; ++t) {
+ std::vector<uint8_t>& blob = blobs[t - b0];
+ if (blob.empty()) ok = false;
+ dir[t].offset = offset;
+ dir[t].bytes = uint32_t(blob.size());
+ ok = ok && std::fwrite(blob.data(), 1, blob.size(), f) == blob.size();
+ offset += blob.size();
+ }
+ const uint32_t c0 = b0 / kChunk, c1 = (b1 + kChunk - 1) / kChunk;
+ poolFor(c1 - c0, [&](size_t b, size_t e) {
+ for (size_t c = b; c < e; ++c) {
+ const uint32_t first = uint32_t(c0 + c) * kChunk, count = std::min(kChunk, T - first);
+ compressValues(h.codec, coarse.data() + size_t(first - b0) * CS, count, CS, chunkBlobs[c]);
+ }
+ }, 1, 1);
+ for (uint32_t c = c0; ok && c < c1; ++c) {
+ const std::vector<uint8_t>& blob = chunkBlobs[c - c0];
+ chunkDir[c].offset = offset;
+ chunkDir[c].bytes = uint32_t(blob.size());
+ ok = std::fwrite(blob.data(), 1, blob.size(), f) == blob.size();
+ offset += blob.size();
+ }
+ }
+ std::vector<uint8_t> dirBytes(size_t(T) * sizeof(Entry) + size_t(chunks) * sizeof(ChunkEntry));
+ std::memcpy(dirBytes.data(), dir.data(), size_t(T) * sizeof(Entry));
+ std::memcpy(dirBytes.data() + size_t(T) * sizeof(Entry), chunkDir.data(), size_t(chunks) * sizeof(ChunkEntry));
+ h.dirOffset = offset;
+ h.dirBytes = dirBytes.size();
+ h.dirHash = fnv1a(dirBytes.data(), dirBytes.size());
+ ok = ok && std::fwrite(dirBytes.data(), 1, dirBytes.size(), f) == dirBytes.size();
+ ok = ok && std::fseek(f, 0, SEEK_SET) == 0 && std::fwrite(encodeHeader(h).data(), 1, kHeaderBytes, f) == kHeaderBytes;
+ ok = ok && std::fflush(f) == 0;
+#if defined(SF_FOCUS_POSIX)
+ ok = ok && ::fsync(fileno(f)) == 0;
+#endif
+ ok = std::fclose(f) == 0 && ok;
+ if (!ok) error = "Writing " + path + " failed (disk full?)";
+ return ok;
+}
+
+// ---------------------------------------------------------------------------
+// Opening
+// ---------------------------------------------------------------------------
+
+FocusDetail::~FocusDetail() { unmap(); }
+
+void FocusDetail::unmap() {
+#if defined(SF_FOCUS_POSIX)
+ if (map_) ::munmap(const_cast<uint8_t*>(map_), mapBytes_);
+ map_ = nullptr;
+ mapBytes_ = 0;
+ if (fd_ >= 0) ::close(fd_);
+ fd_ = -1;
+#endif
+}
+
+bool FocusDetail::mapFile(std::string& error) {
+#if defined(SF_FOCUS_POSIX)
+ if (map_) ::munmap(const_cast<uint8_t*>(map_), mapBytes_);
+ map_ = nullptr;
+ if (fd_ < 0) fd_ = ::open(path_.c_str(), O_RDWR);
+ struct stat st {};
+ if (fd_ < 0 || ::fstat(fd_, &st) != 0 || st.st_size < off_t(kHeaderBytes)) {
+ error = "Can't open " + path_;
+ return false;
+ }
+ mapBytes_ = size_t(st.st_size);
+ void* p = ::mmap(nullptr, mapBytes_, PROT_READ, MAP_SHARED, fd_, 0);
+ if (p == MAP_FAILED) {
+ error = "Can't map " + path_;
+ return false;
+ }
+ map_ = static_cast<const uint8_t*>(p);
+ return true;
+#else
+ error = "Focus Detail needs Linux or macOS for now (memory-mapped files)";
+ return false;
+#endif
+}
+
+std::shared_ptr<FocusDetail> FocusDetail::open(const std::string& path, Mesh& out, std::string& error) {
+ std::shared_ptr<FocusDetail> fd(new FocusDetail());
+ fd->path_ = path;
+ if (!fd->mapFile(error)) return nullptr;
+ const uint8_t* b = fd->map_;
+ Header h;
+ if (!decodeHeader(b, fd->mapBytes_, h, error)) return nullptr;
+ const uint32_t T = h.tris, V = h.verts, chunks = (T + h.chunkTiles - 1) / h.chunkTiles;
+ if (h.chunkTiles != kChunk || h.coarseLevel != uint32_t(kCoarseLevel) || h.baseBytes != uint64_t(V) * 24 + uint64_t(T) * 12 ||
+ h.baseOffset < kHeaderBytes || h.baseOffset + h.baseBytes > fd->mapBytes_ || h.dirOffset + h.dirBytes > fd->mapBytes_ ||
+ h.dirBytes != uint64_t(T) * sizeof(Entry) + uint64_t(chunks) * sizeof(ChunkEntry) ||
+ fnv1a(b + h.dirOffset, h.dirBytes) != h.dirHash) {
+ error = "Detail file is damaged";
+ return nullptr;
+ }
+ fd->level_ = int(h.level);
+ fd->n_ = 1u << h.level;
+ fd->samples_ = (fd->n_ + 1) * (fd->n_ + 2) / 2;
+ fd->coarseN_ = 1u << h.coarseLevel;
+ fd->coarseSamples_ = (fd->coarseN_ + 1) * (fd->coarseN_ + 2) / 2;
+ fd->codec_ = h.codec;
+ fd->step_ = h.step;
+ fd->generation_ = h.generation;
+ fd->garbage_ = h.garbage;
+ fd->dir_.resize(T);
+ fd->chunks_.resize(chunks);
+ std::memcpy(fd->dir_.data(), b + h.dirOffset, size_t(T) * sizeof(Entry));
+ std::memcpy(fd->chunks_.data(), b + h.dirOffset + size_t(T) * sizeof(Entry), size_t(chunks) * sizeof(ChunkEntry));
+ for (const Entry& e : fd->dir_)
+ if (e.offset < kHeaderBytes || e.offset + e.bytes > fd->mapBytes_) {
+ error = "Detail file is damaged";
+ return nullptr;
+ }
+ for (const ChunkEntry& c : fd->chunks_)
+ if (c.offset < kHeaderBytes || c.offset + c.bytes > fd->mapBytes_) {
+ error = "Detail file is damaged";
+ return nullptr;
+ }
+ // The base mesh
+ Mesh m;
+ m.positions.resize(V);
+ m.normals.resize(V);
+ m.triangles.resize(T);
+ const uint8_t* base = b + h.baseOffset;
+ std::memcpy(m.positions.data(), base, size_t(V) * 12);
+ std::memcpy(m.normals.data(), base + size_t(V) * 12, size_t(V) * 12);
+ std::memcpy(m.triangles.data(), base + size_t(V) * 24, size_t(T) * 12);
+ for (const glm::uvec3& t : m.triangles)
+ if (t.x >= V || t.y >= V || t.z >= V) {
+ error = "Detail file is damaged";
+ return nullptr;
+ }
+ m.rebuildTopology();
+ // Coarse grids, then every tile upsampled from its coarse grid.
+ const uint32_t CS = fd->coarseSamples_, S = fd->samples_;
+ fd->coarse_.assign(size_t(T) * CS, 0);
+ std::atomic<bool> bad{false};
+ poolFor(chunks, [&](size_t c0, size_t c1) {
+ for (size_t c = c0; c < c1; ++c) {
+ const uint32_t first = uint32_t(c) * kChunk, count = std::min(kChunk, T - first);
+ if (!decompressValues(fd->codec_, b + fd->chunks_[c].offset, fd->chunks_[c].bytes, count, CS,
+ fd->coarse_.data() + size_t(first) * CS))
+ bad = true;
+ }
+ }, 1, 1);
+ if (bad) {
+ error = "Detail file is damaged";
+ return nullptr;
+ }
+ MicroStore disp;
+ if (microWantsOutOfCore(size_t(T) * S)) {
+ std::string why;
+ disp.assignMapped(size_t(T) * S, why);
+ } else {
+ disp = std::vector<int16_t>(size_t(T) * S);
+ }
+ poolFor(T, [&](size_t t0, size_t t1) {
+ for (size_t t = t0; t < t1; ++t) upsample(fd->coarse_.data() + t * CS, fd->coarseN_, fd->n_, disp.data() + t * S);
+ }, 64);
+ if (!microAttach(m, fd->level_, fd->step_, std::move(disp), {}, error)) return nullptr;
+ MicroMesh& mm = *m.micro;
+ mm.rangeMax = 0;
+ for (uint32_t t = 0; t < T; ++t) {
+ mm.triRange[t] = fd->dir_[t].range; // the full tile's (brushes find triangles with it)
+ mm.rangeMax = std::max(mm.rangeMax, mm.triRange[t]);
+ }
+ fd->state_.assign(T, kCoarse);
+ fd->lastWanted_.assign(T, 0);
+ fd->parked_.resize(T);
+ fd->score_.assign(T, 0.0f);
+ fd->tris_ = m.triangles;
+ fd->baseHash_ = fnv1a(base, size_t(V) * 24);
+ fd->stats_.tiles = T;
+ mm.focus = fd;
+ out = std::move(m);
+ return fd;
+}
+
+// ---------------------------------------------------------------------------
+// Promotion and demotion
+// ---------------------------------------------------------------------------
+
+bool FocusDetail::readTile(uint32_t t, int16_t* out) const {
+ if (!parked_[t].empty()) return decompressTile(codec_, parked_[t].data(), parked_[t].size(), n_, samples_, out);
+ const Entry& e = dir_[t];
+ return map_ && decompressTile(codec_, map_ + e.offset, e.bytes, n_, samples_, out);
+}
+
+void FocusDetail::fillCoarse(MicroMesh& mm, uint32_t t) const {
+ upsample(coarse_.data() + size_t(t) * coarseSamples_, coarseN_, n_, mm.disp.data() + size_t(t) * samples_);
+}
+
+void FocusDetail::promoteLocked(MicroMesh& mm, std::span<const uint32_t> tris, bool edit, bool sync) {
+ const auto t0 = Clock::now();
+ poolFor(tris.size(), [&](size_t b, size_t e) {
+ for (size_t k = b; k < e; ++k) {
+ const uint32_t t = tris[k];
+ if (!readTile(t, mm.disp.data() + size_t(t) * samples_)) fillCoarse(mm, t); // (a damaged tile stays coarse)
+ }
+ }, 4);
+ // A promoted tile's real edges go into its coarse neighbours (no cracks along the focus border).
+ for (uint32_t t : tris) {
+ state_[t] = edit ? kDirty : kFull;
+ mm.markDirty(t);
+ ++fullTiles_;
+ }
+ if (!tris.empty()) {
+ for (uint32_t t : tris)
+ shareEdges(tris_, mm, t, [&](uint32_t u) { return state_[u] == kCoarse ? 1 : 0; });
+ }
+ ++mm.revision;
+ stats_.promoted += tris.size();
+ if (sync) stats_.syncPromoted += tris.size();
+ stats_.promoteMs += msSince(t0);
+}
+
+void FocusDetail::demoteLocked(MicroMesh& mm, uint32_t t) {
+ int16_t* g = mm.disp.data() + size_t(t) * samples_;
+ if (state_[t] == kDirty) { // keep the edit: compressed in memory until saved, coarse grid refreshed
+ const TileInfo info = analyse(g, n_, samples_, coarseN_, step_, coarse_.data() + size_t(t) * coarseSamples_);
+ dir_[t].coarseErr = info.coarseErr;
+ dir_[t].range = std::max(dir_[t].range, info.range);
+ compressTile(codec_, g, n_, samples_, parked_[t]);
+ }
+ fillCoarse(mm, t);
+ state_[t] = kCoarse;
+ --fullTiles_;
+ mm.markDirty(t);
+ // full neighbours keep their real edges: take theirs; coarse ones take this tile's upsampled edges
+ shareEdges(tris_, mm, t, [&](uint32_t u) { return state_[u] == kCoarse ? 1 : -1; });
+ ++stats_.demoted;
+}
+
+void FocusDetail::ensureFull(MicroMesh& mm, std::span<const uint32_t> tris, bool edit) {
+ std::lock_guard<std::mutex> lock(mutex_);
+ std::vector<uint32_t> need;
+ for (uint32_t t : tris) {
+ if (t >= state_.size()) continue;
+ lastWanted_[t] = frame_;
+ if (state_[t] == kCoarse) need.push_back(t);
+ else if (edit) state_[t] = kDirty;
+ }
+ if (!need.empty()) promoteLocked(mm, need, edit, true);
+}
+
+void FocusDetail::update(Mesh& m, const FocusView& view, const FocusBrush& brush, double budgetMs) {
+ if (!m.micro || m.micro->focus.get() != this) return;
+ const auto t0 = Clock::now();
+ MicroMesh& mm = *m.micro;
+ const uint32_t T = uint32_t(state_.size());
+ std::unique_lock<std::mutex> lock(mutex_);
+ ++frame_;
+ // Frustum planes (Gribb & Hartmann) for the visibility test.
+ glm::vec4 planes[6];
+ if (view.valid) {
+ const glm::mat4 M = glm::transpose(view.viewProj);
+ planes[0] = M[3] + M[0], planes[1] = M[3] - M[0], planes[2] = M[3] + M[1];
+ planes[3] = M[3] - M[1], planes[4] = M[3] + M[2], planes[5] = M[3] - M[2];
+ for (glm::vec4& p : planes) p /= glm::length(glm::vec3(p));
+ }
+ const glm::vec3 b0 = brush.position, b1 = brush.position + brush.velocity * lookaheadSeconds;
+ const float reach = brush.radius * brushReach;
+ poolFor(T, [&](size_t s, size_t e) {
+ for (size_t t = s; t < e; ++t) {
+ // (from the base as it is now: Move on a micro level moves the base cage)
+ const glm::uvec3 tr = tris_[t];
+ const glm::vec3 a = m.positions[tr.x], b = m.positions[tr.y], cc = m.positions[tr.z];
+ const glm::vec3 c = (a + b + cc) / 3.0f;
+ const glm::vec4 bs(c, std::sqrt(std::max({glm::dot(a - c, a - c), glm::dot(b - c, b - c), glm::dot(cc - c, cc - c)})) +
+ mm.bulgeMax + float(mm.triRange[t]) * mm.step);
+ float score = 0.0f;
+ if (brush.present) { // distance to the brush's path ahead (and its mirror's)
+ for (int k = 0; k < (brush.mirrorX ? 2 : 1); ++k) {
+ const glm::vec3 q = k ? glm::vec3(-c.x, c.y, c.z) : c;
+ const glm::vec3 d = b1 - b0;
+ const float len2 = glm::dot(d, d);
+ const float u = len2 > 0.0f ? std::clamp(glm::dot(q - b0, d) / len2, 0.0f, 1.0f) : 0.0f;
+ const float dist = glm::distance(q, b0 + d * u) - bs.w;
+ if (dist < reach) score = std::max(score, 1e6f - dist);
+ }
+ }
+ if (score == 0.0f && view.valid) {
+ bool in = true;
+ for (const glm::vec4& p : planes)
+ if (glm::dot(glm::vec3(p), c) + p.w < -bs.w) in = false;
+ if (in) {
+ const float px = dir_[t].coarseErr * view.focalPx / std::max(glm::distance(c, view.eye) - bs.w, 1e-4f);
+ if (px > view.errorPx) score = px;
+ }
+ }
+ score_[t] = score;
+ }
+ }, 4096);
+ scratch_.clear();
+ size_t wanted = 0;
+ for (uint32_t t = 0; t < T; ++t)
+ if (score_[t] > 0.0f) {
+ ++wanted;
+ lastWanted_[t] = frame_;
+ if (state_[t] == kCoarse) scratch_.push_back(t);
+ }
+ std::sort(scratch_.begin(), scratch_.end(), [&](uint32_t a, uint32_t b) { return score_[a] > score_[b] || (score_[a] == score_[b] && a < b); });
+ const size_t maxFull = std::max<size_t>(budgetBytes / (size_t(samples_) * 2), 1);
+ // Demotion candidates: full tiles not wanted for kLinger frames, least recently wanted first (never a dirty
+ // one while stroking: a dab on the solver thread may be writing it).
+ constexpr uint32_t kLinger = 8;
+ std::vector<uint32_t> lru;
+ auto demoteFor = [&](size_t room) {
+ if (fullTiles_ + room <= maxFull) return true;
+ if (lru.empty()) {
+ for (uint32_t t = 0; t < T; ++t)
+ if (state_[t] != kCoarse && lastWanted_[t] + kLinger < frame_ && !(mm.stroking && state_[t] == kDirty)) lru.push_back(t);
+ std::sort(lru.begin(), lru.end(), [&](uint32_t a, uint32_t b) { return lastWanted_[a] > lastWanted_[b] || (lastWanted_[a] == lastWanted_[b] && a > b); });
+ }
+ while (fullTiles_ + room > maxFull && !lru.empty()) { // (back = oldest)
+ const uint32_t t = lru.back();
+ lru.pop_back();
+ if (state_[t] != kCoarse) demoteLocked(mm, t);
+ }
+ return fullTiles_ + room <= maxFull;
+ };
+ size_t done = 0;
+ constexpr size_t kBatch = 64;
+ while (done < scratch_.size()) {
+ size_t count = std::min(kBatch, scratch_.size() - done);
+ const bool mustHave = score_[scratch_[done]] >= 1e5f; // the brush's tiles go in over the budget
+ if (!demoteFor(count) && !mustHave) {
+ count = std::min(count, maxFull - std::min(maxFull, fullTiles_));
+ if (count == 0) break;
+ }
+ promoteLocked(mm, std::span<const uint32_t>(scratch_.data() + done, count), false, false);
+ done += count;
+ if (msSince(t0) > budgetMs) break;
+ lock.unlock(); // let a brush's ensureFull in between batches
+ lock.lock();
+ }
+ demoteFor(0);
+ stats_.lastWanted = wanted;
+ stats_.lastPending = scratch_.size() - done;
+ stats_.lastUpdateMs = msSince(t0);
+}
+
+FocusStats FocusDetail::stats() const {
+ std::lock_guard<std::mutex> lock(mutex_);
+ FocusStats s = stats_;
+ s.fullTiles = fullTiles_;
+ s.dirtyTiles = size_t(std::count(state_.begin(), state_.end(), uint8_t(kDirty)));
+ for (const auto& p : parked_)
+ if (!p.empty()) ++s.parkedTiles, s.parkedBytes += p.size();
+ s.fullBytes = fullTiles_ * size_t(samples_) * 2;
+ s.fileBytes = mapBytes_;
+ return s;
+}
+
+// ---------------------------------------------------------------------------
+// Saving: changed tiles only
+// ---------------------------------------------------------------------------
+
+bool FocusDetail::save(Mesh& m, std::string& error) {
+#if defined(SF_FOCUS_POSIX)
+ if (!m.micro || m.micro->focus.get() != this) {
+ error = "Not this detail file's mesh";
+ return false;
+ }
+ MicroMesh& mm = *m.micro;
+ std::lock_guard<std::mutex> lock(mutex_);
+ const uint32_t T = uint32_t(state_.size()), S = samples_, CS = coarseSamples_;
+ std::vector<uint32_t> tiles;
+ for (uint32_t t = 0; t < T; ++t)
+ if (state_[t] == kDirty || !parked_[t].empty()) tiles.push_back(t);
+ // The base cage moved (Move on a micro level): its block goes too.
+ std::vector<uint8_t> baseBlock;
+ if (m.normals.size() == m.vertexCount()) {
+ baseBlock.resize(m.vertexCount() * 24 + tris_.size() * 12);
+ std::memcpy(baseBlock.data(), m.positions.data(), m.vertexCount() * 12);
+ std::memcpy(baseBlock.data() + m.vertexCount() * 12, m.normals.data(), m.vertexCount() * 12);
+ std::memcpy(baseBlock.data() + m.vertexCount() * 24, tris_.data(), tris_.size() * 12);
+ if (fnv1a(baseBlock.data(), m.vertexCount() * 24) == baseHash_) baseBlock.clear();
+ }
+ if (tiles.empty() && baseBlock.empty()) return true;
+ std::vector<std::vector<uint8_t>> blobs(tiles.size());
+ poolFor(tiles.size(), [&](size_t b, size_t e) {
+ for (size_t k = b; k < e; ++k) {
+ const uint32_t t = tiles[k];
+ if (state_[t] == kDirty) {
+ const int16_t* g = mm.disp.data() + size_t(t) * S;
+ const TileInfo info = analyse(g, n_, S, coarseN_, step_, coarse_.data() + size_t(t) * CS);
+ dir_[t].coarseErr = info.coarseErr;
+ dir_[t].range = info.range;
+ compressTile(codec_, g, n_, S, blobs[k]);
+ } else {
+ blobs[k] = parked_[t];
+ }
+ }
+ }, 8);
+ std::vector<uint32_t> chunkList;
+ for (uint32_t t : tiles)
+ if (chunkList.empty() || chunkList.back() != t / kChunk) chunkList.push_back(t / kChunk);
+ std::vector<std::vector<uint8_t>> chunkBlobs(chunkList.size());
+ poolFor(chunkList.size(), [&](size_t b, size_t e) {
+ for (size_t k = b; k < e; ++k) {
+ const uint32_t first = chunkList[k] * kChunk, count = std::min(kChunk, T - first);
+ compressValues(codec_, coarse_.data() + size_t(first) * CS, count, CS, chunkBlobs[k]);
+ }
+ }, 1);
+ // Append blobs, then the directory; the header last.
+ uint64_t offset = mapBytes_;
+ std::vector<Entry> dir = dir_;
+ std::vector<ChunkEntry> chunkDir = chunks_;
+ uint64_t garbage = garbage_;
+ std::vector<uint8_t> tail;
+ for (size_t k = 0; k < tiles.size(); ++k) {
+ garbage += dir[tiles[k]].bytes;
+ dir[tiles[k]].offset = offset + tail.size();
+ dir[tiles[k]].bytes = uint32_t(blobs[k].size());
+ tail.insert(tail.end(), blobs[k].begin(), blobs[k].end());
+ }
+ for (size_t k = 0; k < chunkList.size(); ++k) {
+ garbage += chunkDir[chunkList[k]].bytes;
+ chunkDir[chunkList[k]].offset = offset + tail.size();
+ chunkDir[chunkList[k]].bytes = uint32_t(chunkBlobs[k].size());
+ tail.insert(tail.end(), chunkBlobs[k].begin(), chunkBlobs[k].end());
+ }
+ Header h;
+ if (!decodeHeader(map_, mapBytes_, h, error)) return false;
+ garbage += h.dirBytes;
+ if (!baseBlock.empty()) {
+ garbage += h.baseBytes;
+ h.baseOffset = offset + tail.size();
+ tail.insert(tail.end(), baseBlock.begin(), baseBlock.end());
+ }
+ const size_t dirAt = tail.size();
+ tail.resize(dirAt + size_t(T) * sizeof(Entry) + chunkDir.size() * sizeof(ChunkEntry));
+ std::memcpy(tail.data() + dirAt, dir.data(), size_t(T) * sizeof(Entry));
+ std::memcpy(tail.data() + dirAt + size_t(T) * sizeof(Entry), chunkDir.data(), chunkDir.size() * sizeof(ChunkEntry));
+ h.dirOffset = offset + dirAt;
+ h.dirBytes = tail.size() - dirAt;
+ h.dirHash = fnv1a(tail.data() + dirAt, h.dirBytes);
+ h.generation = generation_ + 1;
+ h.garbage = garbage;
+ auto pwriteAll = [&](const uint8_t* p, size_t size, uint64_t at) {
+ while (size) {
+ const ssize_t w = ::pwrite(fd_, p, size, off_t(at));
+ if (w <= 0) return false;
+ p += w, size -= size_t(w), at += uint64_t(w);
+ }
+ return true;
+ };
+ const std::vector<uint8_t> head = encodeHeader(h);
+ if (!pwriteAll(tail.data(), tail.size(), offset) || ::fsync(fd_) != 0 || !pwriteAll(head.data(), head.size(), 0) ||
+ ::fsync(fd_) != 0) {
+ error = "Saving " + path_ + " failed (disk full?)";
+ return false;
+ }
+ dir_ = std::move(dir);
+ chunks_ = std::move(chunkDir);
+ garbage_ = garbage;
+ generation_ = h.generation;
+ if (!baseBlock.empty()) baseHash_ = fnv1a(baseBlock.data(), m.vertexCount() * 24);
+ for (uint32_t t : tiles) {
+ if (state_[t] == kDirty) state_[t] = kFull;
+ std::vector<uint8_t>().swap(parked_[t]);
+ }
+ return mapFile(error);
+#else
+ error = "Focus Detail needs Linux or macOS for now";
+ return false;
+#endif
+}
+
+// ---------------------------------------------------------------------------
+// The synthetic asset
+// ---------------------------------------------------------------------------
+
+namespace {
+
+float hash3(int x, int y, int z, uint32_t seed) {
+ uint32_t h = uint32_t(x) * 0x8da6b343u ^ uint32_t(y) * 0xd8163841u ^ uint32_t(z) * 0xcb1ab31fu ^ seed * 0x9E3779B9u;
+ h ^= h >> 13;
+ h *= 0x85EBCA6Bu;
+ h ^= h >> 16;
+ return float(h & 0xFFFFFF) / float(0xFFFFFF) * 2.0f - 1.0f;
+}
+
+float noise3(const glm::vec3& p, uint32_t seed) {
+ const glm::vec3 f = glm::floor(p), t = p - f, u = t * t * (3.0f - 2.0f * t);
+ const int x = int(f.x), y = int(f.y), z = int(f.z);
+ auto h = [&](int dx, int dy, int dz) { return hash3(x + dx, y + dy, z + dz, seed); };
+ const float a = glm::mix(glm::mix(h(0, 0, 0), h(1, 0, 0), u.x), glm::mix(h(0, 1, 0), h(1, 1, 0), u.x), u.y);
+ const float b = glm::mix(glm::mix(h(0, 0, 1), h(1, 0, 1), u.x), glm::mix(h(0, 1, 1), h(1, 1, 1), u.x), u.y);
+ return glm::mix(a, b, u.z);
+}
+
+} // namespace
+
+bool focusWriteSyntheticHead(const std::string& path, size_t baseTriangles, int level, std::string& error) {
+ Mesh base = makeBenchHead(baseTriangles);
+ if (base.normals.size() != base.vertexCount()) base.recomputeAllNormals();
+ double edge = 0.0;
+ for (const glm::uvec3& t : base.triangles)
+ edge += double(glm::distance(base.positions[t.x], base.positions[t.y]));
+ const float avgEdge = float(edge / double(std::max<size_t>(base.triangleCount(), 1)));
+ const float step = std::max(avgEdge / 4096.0f, 1e-12f);
+ // Skin: ridged wrinkles a few base edges long, pores a few dozen micro edges apart (only the full level has them).
+ const float wrinkleF = 0.8f / avgEdge, poreF = 12.0f / avgEdge, wrinkleA = 0.05f * avgEdge, poreA = 0.012f * avgEdge;
+ const uint32_t n = 1u << level;
+ return FocusDetail::write(path, base, level, step, [&](const MicroTriangle& T, uint32_t, int16_t* out) {
+ uint32_t idx = 0;
+ for (uint32_t j = 0; j <= n; ++j)
+ for (uint32_t i = 0; i + j <= n; ++i, ++idx) {
+ glm::vec3 s, nn;
+ T.eval(i, j, s, nn);
+ const float w = 1.0f - 2.0f * std::abs(noise3(s * wrinkleF, 7) * 0.7f + noise3(s * wrinkleF * 2.1f, 8) * 0.3f);
+ const float pore = noise3(s * poreF, 9);
+ const float d = wrinkleA * w - poreA * std::max(0.0f, pore) * std::max(0.0f, pore) * 2.0f;
+ out[idx] = int16_t(std::clamp(std::lround(d / step), -32767L, 32767L));
+ }
+ }, error);
+}
+
+std::string focusStandard1bPath(std::string& error, const std::function<void(const std::string&)>& log) {
+ namespace fs = std::filesystem;
+ const fs::path dir = MicroStore::swapFolder();
+ std::error_code ec;
+ fs::create_directories(dir, ec);
+ const fs::path path = dir / "standard-1b-v1.sfdetail";
+ if (std::FILE* f = std::fopen(path.string().c_str(), "rb")) { // (the header is written last: a valid one = a whole file)
+ uint8_t head[kHeaderBytes];
+ Header h;
+ std::string why;
+ const bool ok = std::fread(head, 1, kHeaderBytes, f) == kHeaderBytes && decodeHeader(head, kHeaderBytes, h, why) &&
+ h.dirOffset > 0;
+ std::fclose(f);
+ if (ok) return path.string();
+ }
+ if (log) log("Writing the standard-1b asset (once): " + path.string());
+ const fs::path tmp = fs::path(path.string() + ".tmp");
+ if (!focusWriteSyntheticHead(tmp.string(), 61035, 7, error)) return {};
+ fs::rename(tmp, path, ec);
+ if (ec) {
+ error = "Can't write " + path.string();
+ return {};
+ }
+ return path.string();
+}
+
+} // namespace sf
diff --git a/src/core/FocusDetail.h b/src/core/FocusDetail.h
new file mode 100644
index 00000000..b9bb283e
--- /dev/null
+++ b/src/core/FocusDetail.h
@@ -0,0 +1,164 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Focus Detail (prototype, off by default; docs/design/focus-detail.md): a micro level (core/Micro.h) far larger
+// than what is worth decompressing, streamed by where the brush and the camera are.
+//
+// - On disk (a .sfdetail file, memory-mapped): the base mesh, then every base triangle's micro grid ("tile") as
+// quantized 16-bit heights, row-delta coded and compressed (zstd when the build has it, else the tile codec of
+// core/MicroStore.h). A directory gives each tile's place, its exact range and how far its coarse grid is from
+// it. Coarse grids (level kCoarseLevel, a subsample of the full grid) sit in compressed chunks of kChunk tiles.
+// - In memory: MicroMesh::disp keeps its full-size layout, so brushes and the renderer work unchanged. A tile
+// outside the focus holds its coarse grid upsampled (exact at the coarse points, linear in between, so shared
+// edges stay watertight); a tile in the focus holds its full grid.
+// - The focus: the tiles under and ahead of the brush (its position plus velocity, a short time ahead) and the
+// visible tiles whose coarse grid would be more than FocusView::errorPx off on screen. Promotions run each
+// frame within a time budget, highest priority first; demotions free the least recently wanted tiles once the
+// full tiles exceed budgetBytes (a tile wanted in the last 8 frames stays). Brushes and undo never write a coarse tile: MicroStroke::touch and
+// MicroUndo::swapWith promote the tiles they are about to change first (ensureFull).
+// - Edited tiles are dirty until saved; a dirty tile that gets demoted is compressed into memory ("parked").
+// save() appends only the dirty and parked tiles, the coarse chunks holding them and a new directory, then
+// rewrites the header (the old directory stays valid until then: a crash keeps the last save).
+// Undo needs nothing new: a stroke's MicroUndo already keeps copy-on-write deltas of the tiles it touched.
+#pragma once
+
+#include "core/Mesh.h"
+
+#include <glm/glm.hpp>
+
+#include <atomic>
+#include <cstddef>
+#include <cstdint>
+#include <functional>
+#include <memory>
+#include <mutex>
+#include <span>
+#include <string>
+#include <vector>
+
+namespace sf {
+
+struct MicroMesh;
+struct MicroTriangle;
+
+struct FocusView {
+ bool valid = false;
+ glm::mat4 viewProj{1.0f};
+ glm::vec3 eye{0.0f};
+ float focalPx = 1000.0f; // pixels per world unit at distance 1 (viewport height / (2 tan(fov / 2)))
+ float errorPx = 0.5f; // a visible tile is promoted when its coarse grid is further off than this on screen
+};
+
+struct FocusBrush {
+ bool present = false; // the cursor is over the model (hovering or stroking)
+ glm::vec3 position{0.0f};
+ glm::vec3 velocity{0.0f}; // world units per second
+ float radius = 0.0f; // world units
+ bool mirrorX = false; // X symmetry: the mirrored brush too
+};
+
+struct FocusStats {
+ size_t tiles = 0, fullTiles = 0, dirtyTiles = 0, parkedTiles = 0;
+ size_t parkedBytes = 0, fullBytes = 0, fileBytes = 0;
+ uint64_t promoted = 0, demoted = 0, syncPromoted = 0; // since open
+ double promoteMs = 0; // total time promoting (decompress + upsample/copy + markDirty)
+ double lastUpdateMs = 0; // the last update() (scoring + promotions + demotions)
+ size_t lastWanted = 0; // tiles the last update() wanted full
+ size_t lastPending = 0; // of those, still coarse after it (the time budget ran out)
+};
+
+class FocusDetail {
+public:
+ static constexpr int kCoarseLevel = 3; // 8 x 8 segments, 45 points per tile
+ static constexpr uint32_t kChunk = 1024; // coarse grids per compressed chunk
+
+ // Writes a detail file for `base` (its triangles are the tiles) at micro level `level` (> kCoarseLevel) with
+ // `step` world units per integer unit; `tile(T, t, out)` fills tile t's samples (T is set for t). Tiles are
+ // generated, compressed and written in parallel batches: the full level is never in memory.
+ using TileFn = std::function<void(const MicroTriangle& T, uint32_t t, int16_t* out)>;
+ static bool write(const std::string& path, const Mesh& base, int level, float step, const TileFn& tile,
+ std::string& error);
+ // Opens a detail file: `out` becomes its base mesh with a micro level whose tiles all hold their coarse grids,
+ // and the returned manager is attached to it (MicroMesh::focus).
+ static std::shared_ptr<FocusDetail> open(const std::string& path, Mesh& out, std::string& error);
+ ~FocusDetail();
+
+ // Once a frame (main thread): scores every tile, promotes the wanted ones (most wanted first) for at most
+ // `budgetMs`, demotes the least recently wanted while over budgetBytes.
+ void update(Mesh& m, const FocusView& view, const FocusBrush& brush, double budgetMs = 4.0);
+ // Makes tiles `tris` full now (brushes, undo); `edit` marks them dirty. Thread-safe.
+ void ensureFull(MicroMesh& mm, std::span<const uint32_t> tris, bool edit);
+ // Writes the changed tiles to the file (appended), a new directory and header. Returns false with `error`.
+ bool save(Mesh& m, std::string& error);
+
+ // 0 = coarse (upsampled), 1 = full.
+ int tileLevel(uint32_t tri) const { return tri < state_.size() && state_[tri] != kCoarse ? 1 : 0; }
+ FocusStats stats() const;
+ const std::string& path() const { return path_; }
+
+ size_t budgetBytes = size_t(512) << 20; // full tiles kept (2 bytes a point) before the LRU demotes
+ float lookaheadSeconds = 0.25f; // how far ahead of the brush promotion looks
+ float brushReach = 2.0f; // promoted around the brush: this many radii
+
+private:
+ FocusDetail() = default;
+ enum : uint8_t { kCoarse = 0, kFull = 1, kDirty = 2 };
+ struct Entry {
+ uint64_t offset = 0;
+ uint32_t bytes = 0;
+ uint16_t range = 0; // the largest |value| of the full tile
+ uint16_t pad = 0;
+ float coarseErr = 0; // world units: the most the upsampled coarse grid is off
+ uint32_t pad2 = 0;
+ };
+ struct ChunkEntry {
+ uint64_t offset = 0;
+ uint32_t bytes = 0, pad = 0;
+ };
+ bool mapFile(std::string& error);
+ void unmap();
+ bool readTile(uint32_t t, int16_t* out) const; // the full grid (parked first, else the file)
+ void fillCoarse(MicroMesh& mm, uint32_t t) const; // upsamples the coarse grid into disp
+ void promoteLocked(MicroMesh& mm, std::span<const uint32_t> tris, bool edit, bool sync);
+ void demoteLocked(MicroMesh& mm, uint32_t t);
+
+ std::string path_;
+ int level_ = 0;
+ uint32_t n_ = 0, samples_ = 0, coarseN_ = 0, coarseSamples_ = 0, codec_ = 0;
+ float step_ = 0;
+ uint64_t generation_ = 0, garbage_ = 0;
+ std::vector<Entry> dir_;
+ std::vector<ChunkEntry> chunks_;
+ std::vector<int16_t> coarse_; // tiles * coarseSamples, always resident
+ std::vector<uint8_t> state_;
+ std::vector<uint32_t> lastWanted_; // per tile: the update it was last wanted in
+ uint32_t frame_ = 0;
+ std::vector<std::vector<uint8_t>> parked_; // per tile: a demoted dirty tile, compressed (or empty)
+ std::vector<glm::uvec3> tris_; // the base triangles (edges between tiles)
+ uint64_t baseHash_ = 0; // of the saved base positions and normals
+ size_t fullTiles_ = 0;
+ mutable std::mutex mutex_;
+ FocusStats stats_;
+ std::vector<uint32_t> scratch_;
+ std::vector<float> score_;
+ // The mapped file
+ int fd_ = -1;
+ const uint8_t* map_ = nullptr;
+ size_t mapBytes_ = 0;
+};
+
+// A synthetic asset for benchmarks and tests: the standard scenes' procedural head with about `baseTriangles`
+// base triangles at micro level `level`, plus skin-like detail (wrinkles and pores) generated per tile. Written
+// to `path` (nothing in memory at full resolution).
+bool focusWriteSyntheticHead(const std::string& path, size_t baseTriangles, int level, std::string& error);
+// The standard-1b asset (base ~61K triangles at level 7: just over a billion micro triangles) in the cache folder,
+// written the first time (a minute or so). Returns its path, or empty with `error`.
+std::string focusStandard1bPath(std::string& error, const std::function<void(const std::string&)>& log = {});
+
+struct BenchOptions;
+// The Focus Detail benchmark (sculptforge-cli --bench-focus; docs/design/focus-detail.md): open, memory, promotion
+// latency, an orbit close to the surface, Clay and Move strokes with and without prediction, undo, incremental
+// save, and every tile decompressed for comparison. Markdown.
+std::string runFocusBenchmark(const BenchOptions& options);
+
+} // namespace sf
diff --git a/src/core/Micro.cpp b/src/core/Micro.cpp
index 873c51e0..5fc30a5d 100644
--- a/src/core/Micro.cpp
+++ b/src/core/Micro.cpp
@@ -5,6 +5,7 @@
// shading, baking down and displacement maps. The brushes are in MicroSculpt.cpp, the commands in
// MicroCommands.cpp.
#include "core/Micro.h"
+#include "core/FocusDetail.h"
#include "core/BakeImage.h"
#include "core/BVH.h"
@@ -844,6 +845,7 @@ void MicroUndo::swapWith(Mesh& m) {
}
if (!m.micro || !microMatches(m) || m.micro->samples != samples) return;
MicroMesh& mm = mutableMicro(m);
+ if (mm.focus) mm.focus->ensureFull(mm, tris, true); // (undo writes full tiles only)
const size_t S = samples;
// Values kept in another step size (the range grew since): convert to the current one.
if (!journal && step != mm.step && step > 0.0f) {
diff --git a/src/core/Micro.h b/src/core/Micro.h
index 710a6777..22d4b262 100644
--- a/src/core/Micro.h
+++ b/src/core/Micro.h
@@ -49,6 +49,7 @@ namespace sf {
class BVH;
class CommandRegistry;
+class FocusDetail;
struct Document;
struct RayHit;
enum class BrushType;
@@ -85,6 +86,9 @@ struct MicroMesh {
// its GPU copy while the serial stays.
uint64_t serial = 0;
static uint64_t newSerial();
+ // Focus Detail (core/FocusDetail.h, prototype): set when the level streams from a detail file. Tiles outside
+ // the focus hold a coarse grid; brushes and undo make the tiles they change full first (ensureFull).
+ std::shared_ptr<FocusDetail> focus;
size_t microTriangles() const { return size_t(baseTriangles) * n * n; }
size_t microVertices() const; // unique grid points (shared edges and corners counted once)
diff --git a/src/core/MicroSculpt.cpp b/src/core/MicroSculpt.cpp
index 8f918bc2..1dd31b37 100644
--- a/src/core/MicroSculpt.cpp
+++ b/src/core/MicroSculpt.cpp
@@ -10,6 +10,7 @@
// 5. stitches: copies of a point on a shared edge or corner are averaged (they differ only where a brush read
// neighbours, like Smooth), so the surface stays closed.
#include "core/Micro.h"
+#include "core/FocusDetail.h"
#include "core/BVH.h"
#include "core/Sculpt.h"
@@ -131,6 +132,7 @@ void MicroStroke::begin(Mesh& m) {
// Saves the grids of the triangles in `tris` that this stroke hasn't touched yet.
void MicroStroke::touch(MicroMesh& mm, std::span<const uint32_t> tris) {
+ if (mm.focus) mm.focus->ensureFull(mm, tris, true); // (edits write full tiles only)
const size_t S = mm.samples, before = tiles_.size();
for (uint32_t t : tris)
if (!tileOf_[t]) {
diff --git a/tests/test_focus_detail.cpp b/tests/test_focus_detail.cpp
new file mode 100644
index 00000000..1345bae0
--- /dev/null
+++ b/tests/test_focus_detail.cpp
@@ -0,0 +1,257 @@
+// SculptForge - open source digital sculpting
+// SPDX-License-Identifier: GPL-3.0-or-later
+//
+// Focus Detail (core/FocusDetail.h): the detail file, coarse and full tiles, crack-free borders, brushes and undo
+// on streamed tiles, prediction, the LRU budget and incremental saves.
+#include "TestHarness.h"
+
+#include "core/Benchmark.h"
+#include "core/BVH.h"
+#include "core/FocusDetail.h"
+#include "core/Micro.h"
+#include "core/Sculpt.h"
+
+#include <atomic>
+#include <cstdio>
+#include <thread>
+#include <filesystem>
+#include <vector>
+
+using namespace sf;
+
+#if defined(__unix__) || defined(__APPLE__)
+
+namespace {
+
+std::string focusTempPath(const char* name) {
+ const auto dir = std::filesystem::temp_directory_path() / "sf-focus-detail-test";
+ std::filesystem::create_directories(dir);
+ return (dir / name).string();
+}
+
+// A small head (about 2,000 base triangles) at micro level 5.
+std::string writeSmall(const char* name) {
+ const std::string path = focusTempPath(name);
+ std::string err;
+ CHECK(focusWriteSyntheticHead(path, 2000, 5, err));
+ return path;
+}
+
+// Every shared edge has equal values on both sides (micro positions are computed per edge in vertex order, so
+// equal values = no crack).
+bool edgesShared(const Mesh& m) {
+ const MicroMesh& mm = *m.micro;
+ const uint32_t n = mm.n, S = mm.samples;
+ for (uint32_t t = 0; t < mm.baseTriangles; ++t)
+ for (int ed = 0; ed < 3; ++ed) {
+ const uint32_t nb = mm.edgeNeighbor[size_t(t) * 3 + size_t(ed)];
+ if (nb == MicroMesh::kNone) continue;
+ const uint32_t u = nb >> 2;
+ const int f = int(nb & 3);
+ const bool reversed = m.triangles[u][f] != m.triangles[t][ed];
+ for (uint32_t k = 0; k <= n; ++k) {
+ uint32_t i, j, i2, j2;
+ MicroMesh::edgePoint(ed, k, n, i, j);
+ MicroMesh::edgePoint(f, reversed ? n - k : k, n, i2, j2);
+ if (mm.disp[size_t(t) * S + MicroMesh::index(i, j, n)] != mm.disp[size_t(u) * S + MicroMesh::index(i2, j2, n)])
+ return false;
+ }
+ }
+ return true;
+}
+
+std::vector<uint32_t> allTiles(const Mesh& m) {
+ std::vector<uint32_t> all(m.micro->baseTriangles);
+ for (uint32_t t = 0; t < all.size(); ++t) all[t] = t;
+ return all;
+}
+
+} // namespace
+
+TEST(focus_detail_open_is_coarse_then_full_is_exact) {
+ const std::string path = writeSmall("exact.sfdetail");
+ Mesh m;
+ std::string err;
+ auto fd = FocusDetail::open(path, m, err);
+ CHECK(fd != nullptr);
+ if (!fd) return;
+ CHECK(m.micro && m.micro->level == 5);
+ CHECK(fd->stats().fullTiles == 0);
+ CHECK(fd->tileLevel(0) == 0);
+ CHECK(edgesShared(m)); // coarse everywhere: watertight
+ const std::vector<int16_t> coarse = m.micro->disp.toVector();
+ // Half the tiles full: still watertight across the border.
+ std::vector<uint32_t> half;
+ for (uint32_t t = 0; t < m.micro->baseTriangles; t += 2) half.push_back(t);
+ fd->ensureFull(*m.micro, half, false);
+ CHECK(fd->tileLevel(0) == 1 && fd->tileLevel(1) == 0);
+ CHECK(edgesShared(m));
+ // All full: the generated values (the file is watertight by construction) and the coarse points unchanged.
+ fd->ensureFull(*m.micro, allTiles(m), false);
+ CHECK(edgesShared(m));
+ const MicroMesh& mm = *m.micro;
+ const uint32_t s = mm.n >> FocusDetail::kCoarseLevel;
+ size_t differ = 0, coarseMismatch = 0;
+ for (uint32_t t = 0; t < mm.baseTriangles; ++t)
+ for (uint32_t j = 0; j <= mm.n; ++j)
+ for (uint32_t i = 0; i + j <= mm.n; ++i) {
+ const size_t k = size_t(t) * mm.samples + MicroMesh::index(i, j, mm.n);
+ if (mm.disp[k] != coarse[k]) ++differ;
+ if (i % s == 0 && j % s == 0 && mm.disp[k] != coarse[k]) ++coarseMismatch;
+ }
+ CHECK(differ > 0); // the full level has detail the coarse one hasn't
+ CHECK(coarseMismatch == 0); // the coarse grid is a subsample
+ CHECK(mm.rangeMax > 0);
+}
+
+TEST(focus_detail_damaged_file_is_refused) {
+ const std::string path = writeSmall("damaged.sfdetail");
+ std::filesystem::resize_file(path, std::filesystem::file_size(path) - 100);
+ Mesh m;
+ std::string err;
+ CHECK(FocusDetail::open(path, m, err) == nullptr);
+ CHECK(!err.empty());
+ CHECK(FocusDetail::open(focusTempPath("missing.sfdetail"), m, err) == nullptr);
+}
+
+TEST(focus_detail_brush_promotes_undo_and_incremental_save) {
+ const std::string path = writeSmall("edit.sfdetail");
+ Mesh m0;
+ std::string err;
+ auto fd = FocusDetail::open(path, m0, err);
+ CHECK(fd != nullptr);
+ if (!fd) return;
+ Document doc = makeDoc(std::move(m0));
+ Mesh& m = doc.mesh;
+ CHECK(m.micro && m.micro->focus == fd);
+ const size_t fileBefore = std::filesystem::file_size(path);
+ RayHit hit;
+ CHECK(surfaceRaycast(m, doc.bvh, glm::vec3(0.0f, 0.1f, 5.0f), glm::vec3(0, 0, -1), hit));
+ doc.sculptor.beginStroke(m, "Clay");
+ DabParams p;
+ p.center = hit.position;
+ p.radius = 0.08f;
+ p.strength = 0.8f;
+ Symmetry sym;
+ sym.x = false;
+ doc.sculptor.dab(m, doc.bvh, BrushType::Clay, p, sym);
+ const FocusStats s1 = fd->stats();
+ CHECK(s1.syncPromoted > 0); // the brush's tiles became full before it wrote
+ CHECK(s1.dirtyTiles > 0);
+ CHECK(fd->tileLevel(hit.triangle) == 1);
+ const std::vector<int16_t> edited = m.micro->disp.toVector();
+ doc.undo.push(doc.sculptor.endStroke());
+ doc.undo.undo(m, doc.bvh);
+ CHECK(m.micro->disp != edited);
+ doc.undo.redo(m, doc.bvh);
+ CHECK(m.micro->disp == edited);
+ // Save: only the changed tiles are appended.
+ CHECK(fd->save(m, err));
+ const size_t fileAfter = std::filesystem::file_size(path);
+ CHECK(fileAfter > fileBefore);
+ CHECK(fileAfter - fileBefore < fileBefore / 4);
+ CHECK(fd->stats().dirtyTiles == 0);
+ // Reopened: the edited tiles come back as saved.
+ Mesh again;
+ auto fd2 = FocusDetail::open(path, again, err);
+ CHECK(fd2 != nullptr);
+ if (!fd2) return;
+ fd2->ensureFull(*again.micro, allTiles(again), false);
+ std::vector<uint32_t> all = allTiles(m);
+ fd->ensureFull(*m.micro, all, false);
+ CHECK(again.micro->disp == m.micro->disp);
+ // Move on a micro level moves the base cage: saved too.
+ m.positions[3] += glm::vec3(0.0f, 0.01f, 0.0f);
+ CHECK(fd->save(m, err));
+ Mesh third;
+ CHECK(FocusDetail::open(path, third, err) != nullptr);
+ CHECK(third.positions.size() == m.positions.size() && third.positions[3] == m.positions[3]);
+}
+
+TEST(focus_detail_prediction_budget_and_parking) {
+ const std::string path = writeSmall("budget.sfdetail");
+ Mesh m;
+ std::string err;
+ auto fd = FocusDetail::open(path, m, err);
+ CHECK(fd != nullptr);
+ if (!fd) return;
+ MicroMesh& mm = *m.micro;
+ // The brush moving towards +x: tiles ahead of it are promoted before it gets there.
+ FocusBrush brush;
+ brush.present = true;
+ brush.position = glm::vec3(-0.3f, 0.1f, 0.9f);
+ brush.velocity = glm::vec3(2.0f, 0.0f, 0.0f); // 0.5 units in the lookahead
+ brush.radius = 0.05f;
+ fd->update(m, FocusView{}, brush, 1000.0);
+ const FocusStats s = fd->stats();
+ CHECK(s.fullTiles > 0);
+ bool ahead = false;
+ for (uint32_t t = 0; t < mm.baseTriangles; ++t) {
+ const glm::uvec3 tr = m.triangles[t];
+ const glm::vec3 c = (m.positions[tr.x] + m.positions[tr.y] + m.positions[tr.z]) / 3.0f;
+ if (fd->tileLevel(t) && c.x > 0.05f) ahead = true;
+ }
+ CHECK(ahead);
+ CHECK(edgesShared(m));
+ // An edited tile, then a budget of a few tiles elsewhere: the edit is parked (compressed), not lost.
+ std::vector<uint32_t> one{0};
+ fd->ensureFull(mm, one, true);
+ mm.disp[5] = int16_t(mm.disp[5] + 77);
+ const int16_t editedValue = mm.disp[5];
+ fd->budgetBytes = size_t(mm.samples) * 2 * 8;
+ brush.position = glm::vec3(0.3f, 0.1f, 0.9f);
+ brush.velocity = glm::vec3(0.0f);
+ for (int i = 0; i < 12; ++i) fd->update(m, FocusView{}, brush, 1000.0);
+ const FocusStats s2 = fd->stats();
+ CHECK(s2.fullTiles <= std::max<size_t>(8, s2.lastWanted)); // (the brush's own tiles go over the budget)
+ CHECK(s2.demoted > 0);
+ CHECK(fd->tileLevel(0) == 0);
+ CHECK(s2.parkedTiles == 1);
+ CHECK(edgesShared(m));
+ fd->ensureFull(mm, one, false);
+ CHECK(mm.disp[5] == editedValue);
+ // Camera: a close view promotes visible tiles whose coarse grid is off on screen, a far one doesn't.
+ fd->budgetBytes = size_t(1) << 30;
+ FocusView far;
+ far.valid = true;
+ far.eye = glm::vec3(0, 0, 400.0f);
+ far.viewProj = glm::mat4(1.0f) * 0.001f;
+ far.viewProj[3][3] = 1.0f;
+ const size_t before = fd->stats().promoted;
+ fd->update(m, far, FocusBrush{}, 1000.0);
+ CHECK(fd->stats().promoted == before);
+}
+
+TEST(focus_detail_brush_thread_and_frames_together) {
+ // A stroke's dabs (ensureFull on another thread) while frames promote and demote under a small budget.
+ const std::string path = writeSmall("threads.sfdetail");
+ Mesh m;
+ std::string err;
+ auto fd = FocusDetail::open(path, m, err);
+ CHECK(fd != nullptr);
+ if (!fd) return;
+ MicroMesh& mm = *m.micro;
+ fd->budgetBytes = size_t(mm.samples) * 2 * 64;
+ std::atomic<bool> stop{false};
+ std::thread brush([&] {
+ uint32_t k = 1;
+ while (!stop) {
+ std::vector<uint32_t> tiles;
+ for (int i = 0; i < 8; ++i) tiles.push_back((k = k * 1664525u + 1013904223u) % mm.baseTriangles);
+ fd->ensureFull(mm, tiles, false);
+ }
+ });
+ for (int f = 0; f < 60; ++f) {
+ FocusBrush b;
+ b.present = true;
+ b.position = glm::vec3(std::sin(float(f) * 0.2f), 0.1f, 0.8f);
+ b.radius = 0.1f;
+ fd->update(m, FocusView{}, b, 1000.0);
+ }
+ stop = true;
+ brush.join();
+ CHECK(fd->stats().demoted > 0);
+ CHECK(edgesShared(m));
+}
+
+#endif