Reverse engineer anything with agents, from app behavior down to native binaries.
TypeScript agent-skillsai-agentsbinary-analysisclaude-codeclicodexcordisdecompilerdeveloper-toolsdshdsh-pluginghidrahopperllmmcpmodel-context-protocolreareverse-engineeringstatic-analysistouhou
perf(model): avoid decoding omitted GGUF strings (#5781)
gensokyo-dev7d8e4e43380b
diff --git a/experiments/analysis-efficiency/README.md b/experiments/analysis-efficiency/README.mdindex b19989706..154a09249 100644--- a/experiments/analysis-efficiency/README.md+++ b/experiments/analysis-efficiency/README.md@@ -1,5 +1,13 @@# Analysis efficiency measurements+The [2026-10-21 coverage reconciliation](coverage-review-20261021.md) maps all+46 source directories to concrete workflow evidence, including original feature+PRs, historical no-benefit results and remaining verification boundaries.++The [GGUF validation checkpoint](model-utf8-validation.md) removes decoding of+omitted strings while retaining full UTF-8 validation. It records real-model+phase gains, unchanged task token costs and the higher main-task memory peak.+The [WAT framing checkpoint](wasm-string-framing.md) measures structural scanningon real SQLite and VS Code components, with unchanged Evidence and token costs,small whole-task timing gains and no memory improvement.diff --git a/experiments/analysis-efficiency/coverage-review-20261021.md b/experiments/analysis-efficiency/coverage-review-20261021.mdnew file mode 100644index 000000000..e3c641300--- /dev/null+++ b/experiments/analysis-efficiency/coverage-review-20261021.md@@ -0,0 +1,59 @@+# Workflow coverage reconciliation, 2026-10-21++The checked snapshot is main `264862bf2`, with **46 first-level `src/`+directories and 212 production stdio MCP tools**. Directory counts and tool+counts are different inventories: one application workflow can cross several+directories, and one provider can expose many independently costly operations.+This is a source/history review, not a new runtime qualification of all tools.++The search includes this experiment directory, owning contracts and source+history, original feature PRs and related open/closed issues. Missing experiment+files do not establish missing audits. In particular, the model and IL2CPP+feature PRs contain substantial real-task evidence absent from this directory.+Historical measurements retain their original revisions, platforms and limits.++| Directories / workflow | Evidence already present | Remaining coverage boundary |+| ----------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| `kuna`, `ghidra` | Earlier native entry work and [Ghidra projection](ghidra-evidence-projection.md) | Selected questions only; database reuse is a separate [research issue #3136](https://github.com/morluto/rea/issues/3136). |+| `ida` | [Windows native correctness and completed-task costs](ida-native-windows.md), including real `whoami.exe` | Warm selected-task gains do not establish cold analysis, all string formats, all operations or other IDA platforms. |+| `hopper`, `jeb` | [Hopper string records](hopper-string-records.md), [Linux runtime](hopper-linux-python-runtime.md), [real JEB CE](jeb-linux-native.md) | Preserve negative whole-task results; other editions, platforms and workflows are separate scopes. |+| `rizin`, `cutter` | Existing command/session adapters and lifecycle fixes | No comparable dedicated real completed-task performance record located in the searched records. Actual engine/GUI availability must be established before selecting a task. |+| `gdb`, `objdump` | [Real Bash command observations](command-observation-views.md) | Selected read-only disassembly is measured; other commands, large outputs and running-target observation are separate tasks. |+| `adb`, `frida`, `inspector` | [Frida native status/history experiment](frida-status-views.md) | Frida status does not qualify arbitrary scripts, ADB/device transfers, browser Inspector events or live target freshness. |+| `go`, `rust` | [Go build/dependency selection on real binaries](go-build-views.md); Rust cargo-auditable support [#2456](https://github.com/morluto/rea/pull/2456) | Go is already measured. No comparable dedicated Rust dependency-investigation cost record located in this review. |+| `javascript`, `browser`, `electron` | [Agent fact workflow](javascript-facts-agent-history.json), lifecycle experiments and [#4415](https://github.com/morluto/rea/issues/4415) | Real Joplin preparation remains exploratory. Individual JS features and browser/runtime workflows need their own evidence. |+| `android`, `apktool` | [Package summary](android-package-summary.md), [three-APK shared resource decoding](apktool-string-tables.md) | These do not qualify all JADX class/method/reference operations or physical-device workflows. #4777 is already fixed. |+| `dotnet`, `il2cpp` | [Managed summaries/pages](managed-artifact-summary.md), IL2CPP metadata [#3807](https://github.com/morluto/rea/pull/3807) and native links/seeds [#5327](https://github.com/morluto/rea/pull/5327) | IL2CPP already has real Windows/Android row comparisons, task costs and Ghidra import. Unsupported versions/generics/platforms remain separate correctness and coverage work. |+| `model` | GGUF/safetensors [#5603](https://github.com/morluto/rea/pull/5603): real models, independent readers and complete-task costs | Existing delivery is already audited. [#5774](https://github.com/morluto/rea/issues/5774) isolates omitted-string validation cost within that producer. |+| `native`, `artifacts` | [Native inventory](native-inventory-views.md), [entry views](native-entry-views.md); real NIB output reductions [#1460](https://github.com/morluto/rea/pull/1460), [#1477](https://github.com/morluto/rea/pull/1477) | Inventory does not qualify Apple UI/resource, signatures, dyld or runtime traces. NIB byte reductions alone do not establish token, time, CPU or memory savings for a completed question. |+| `firmware` | Region pilot plus real full OpenWrt extraction/publication [#4602](https://github.com/morluto/rea/pull/4602) | Extraction is already measured; noisy timings establish no general speedup. Other image layouts, extraction trees and publication filesystems remain unqualified. |+| `flutter`, `sqlite`, `wasm`, `evm`, `pcap`, `signals` | [Dart AOT](flutter-aot-summary.md), [SQLite controls](sqlite-existing-controls.md), [WAT framing](wasm-string-framing.md), [EVM](evm-interface-summary.md), [PCAP](pcap-small-captures.md), [signals](signal-selection-workflows.md) | Extend a specific question. Preserve cases where existing controls suffice and small tasks regress; these records do not qualify each format or tool exhaustively. |+| `application`, `domain`, `server`, `cli`, `contracts` | [Mixed retained investigations](retained-investigation-workflows.md), [immutable parent validation](immutable-parent-validation.md), [capture validation](retained-capture-validation.md) | Shared selected-query paths are already optimized. Reconstruction/environment composition and new fact families require their own end-to-end questions. |+| `browser` / recorded captures | [SAZ selection](saz-selection-workflows.md) and [real qlog producer/retained reads](retained-capture-validation.md) | [#3133](https://github.com/morluto/rea/issues/3133) still includes actual model/task questions; producer measurements do not close those requirements. |+| `filesystem`, `process`, `reference`, `providers`, `captures`, `windows`, `main`, `composition`, `config`, `evaluation` | Cross-workflow resource/cleanup checks, [process traces](process-trace-validation.md), [MCP profile](mcp-profile-identity.md) and Windows lifecycle checkpoints | Infrastructure coverage follows the exercised workflow. No blanket startup/resource claim; reference imports and `compare_source_to_bundle` need a dedicated completed source-tree question. |++Every one of the 46 directories appears above; repeated `browser` highlights a+distinct workflow, not another directory. The table records located evidence,+not proof that every historical PR was inspected. Open issue state by itself+does not prove missing implementation or absent measurements.++## Three bounded follow-up questions++1. **Rust dependency identity:** inspect a public cargo-auditable binary, find+ exact dependency/version edges and retain complete source Evidence. Compare+ existing delivery first; independently decode its auditable section. Include+ a small binary and account for all discovery, queries and serialization.+2. **Apple UI graph lookup:** on macOS, locate a specific control/outlet in a+ real Transmission/XLD NIB graph using existing summary/document pages.+ Compare equivalent complete questions and independently check archive+ records. Historical byte reductions alone are not the completion criterion.+3. **Reference/source comparison:** import a pinned public source tree and+ answer one source-to-bundle identity question through existing workflows.+ Independently check file manifests, symlink/unknown semantics and selected+ matches; measure repeated traversal and retained representation before+ proposing an index or cache.++Each lane stops with a measured benefit, an evidenced no-change decision, or an+explicit unavailable prerequisite. Rizin/Cutter and device/live workflows stay+visible as additional gaps; no new execution, injection or mutation authority is+implied by this review. No all-provider run or new API is justified by counts.diff --git a/experiments/analysis-efficiency/model-utf8-validation.json b/experiments/analysis-efficiency/model-utf8-validation.jsonnew file mode 100644index 000000000..7e9fddbfe--- /dev/null+++ b/experiments/analysis-efficiency/model-utf8-validation.json@@ -0,0 +1,435 @@+{+ "baseline_source": "264862bf2e3612c9cb6fcfb861843cbba4e1893f",+ "candidate_code": "c2550343173c5abd8f34e925eac1bcc12b7ee6a2",+ "toolchain": {+ "node": "24.18.0",+ "npm": "11.16.0"+ },+ "runtime_filetables_sha256": {+ "baseline": "81c293ff8997017cfe9db2a65a8b70b188e3e6d20cef07902599c69ade5d9fac",+ "candidate": "b4b654a7151f6affbb3ab8541bff74b643d9f78073d338b8feb2b095dd12b6b5"+ },+ "lock_sha256": "699a14749e3825c62430faeb18224190fee1c9347760494e38e334e180bea6ec",+ "gguf_source_sha256": {+ "baseline": "9c387d5fba10db7830e74ec74a73f4b3d2e1347cca665e948b7891881b186007",+ "candidate": "c43e5f8eebe04c0f82e0a970606f20ddf409c7a5fd0e298c88b09bcd0e7864a9"+ },+ "driver_sha256": "9732f45b8b550eef47c1fc993b435129108934f3ec3501a34df1784e6ee37b44",+ "resource_guard_sha256": {+ "resource-run.py": "7a9642ab86ba34290564f636eb080cacf38b4185a254028680b238bff172993b",+ "resource_payload.py": "6f089e645459eba80c5d73e39c21606c78173a44daf49fabc3b88c4631ce51c6"+ },+ "oracle": {+ "gguf": "0.19.0",+ "safetensors": "0.8.0",+ "receipts": [+ {+ "file": "stories15M-q4_0.gguf",+ "sha256": "66967fbece6dbe97886593fdbb73589584927e29119ec31f08090732d1861739",+ "metadata": 20,+ "tensors": 57,+ "status": "matched"+ },+ {+ "file": "stories260K.gguf",+ "sha256": "270cba1bd5109f42d03350f60406024560464db173c0e387d91f0426d3bd256d",+ "metadata": 19,+ "tensors": 48,+ "status": "matched"+ },+ {+ "file": "tiny-bert.safetensors",+ "sha256": "965f02b6a7e5520fc12f710e4e3b6132f697f1c8f648819553c5ade86752d2de",+ "metadata": 2,+ "tensors": 100,+ "status": "matched"+ }+ ]+ },+ "tokenizer": "tiktoken 0.12.0 cl100k_base; independent compact JSON chunks, catalog once + each named request/reply; not actual model usage or initialization/framing",+ "cases": [+ {+ "artifact": {+ "repo": "ggml-org/models",+ "revision": "499bc8821c6b12b4e53c5bffcb21ec206f212d81",+ "member": "tinyllamas/stories15M-q4_0.gguf",+ "file": "stories15M-q4_0.gguf",+ "url": "https://huggingface.co/ggml-org/models/resolve/499bc8821c6b12b4e53c5bffcb21ec206f212d81/tinyllamas/stories15M-q4_0.gguf",+ "bytes": 19077344+ },+ "normalized_sha256": "a894614da7ce9486b07eb2ede05ce11307d4726089722278bc4c95043feb6b66",+ "answers_sha256": "64277f01f43440bc99a94ab0787a4ddde22866ed20140d87d3942b1723cbb067",+ "catalog_sha256": "859b1433ef7c45ea23ba4073715d25ed7a76fd9df6d74285d20a5ea39e8e2898",+ "mcp": [+ {+ "pair": 1,+ "side": "baseline",+ "wall_seconds": 6.667,+ "cpu_seconds": 6.480601,+ "peak_family_mib": 769.34,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [367.272, 1936.653, 162.868, 12.072, 19.236]+ },+ {+ "pair": 1,+ "side": "candidate",+ "wall_seconds": 6.867,+ "cpu_seconds": 6.690645,+ "peak_family_mib": 816.34,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [324.806, 1822.885, 153.167, 11.988, 18.093]+ },+ {+ "pair": 2,+ "side": "baseline",+ "wall_seconds": 6.77,+ "cpu_seconds": 6.555891,+ "peak_family_mib": 798.98,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [456.467, 1823.786, 160.038, 11.296, 18.607]+ },+ {+ "pair": 2,+ "side": "candidate",+ "wall_seconds": 6.565,+ "cpu_seconds": 6.3467,+ "peak_family_mib": 813.566,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [329.862, 1767.086, 152.299, 10.948, 17.523]+ },+ {+ "pair": 3,+ "side": "baseline",+ "wall_seconds": 6.666,+ "cpu_seconds": 6.49779,+ "peak_family_mib": 771.02,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [345.895, 1854.963, 156.895, 11.476, 15.852]+ },+ {+ "pair": 3,+ "side": "candidate",+ "wall_seconds": 6.567,+ "cpu_seconds": 6.386103,+ "peak_family_mib": 818.891,+ "catalog_plus_calls_tokens": 895583,+ "calls_ms": [342.734, 1827.835, 157.363, 10.122, 17.6]+ }+ ],+ "phase": [+ {+ "pair": 1,+ "side": "baseline",+ "first_ms": 154.027,+ "repeat_median_ms": 56.14,+ "wall_seconds": 1.028,+ "cpu_seconds": 0.87953,+ "peak_family_mib": 75.867+ },+ {+ "pair": 1,+ "side": "candidate",+ "first_ms": 121.41,+ "repeat_median_ms": 35.429,+ "wall_seconds": 0.925,+ "cpu_seconds": 0.717987,+ "peak_family_mib": 77.945+ },+ {+ "pair": 2,+ "side": "baseline",+ "first_ms": 159.766,+ "repeat_median_ms": 53.575,+ "wall_seconds": 1.027,+ "cpu_seconds": 0.854739,+ "peak_family_mib": 75.586+ },+ {+ "pair": 2,+ "side": "candidate",+ "first_ms": 121.383,+ "repeat_median_ms": 38.773,+ "wall_seconds": 0.925,+ "cpu_seconds": 0.733397,+ "peak_family_mib": 75.125+ },+ {+ "pair": 3,+ "side": "baseline",+ "first_ms": 148.447,+ "repeat_median_ms": 53.58,+ "wall_seconds": 1.026,+ "cpu_seconds": 0.848173,+ "peak_family_mib": 75.363+ },+ {+ "pair": 3,+ "side": "candidate",+ "first_ms": 121.712,+ "repeat_median_ms": 37.191,+ "wall_seconds": 0.926,+ "cpu_seconds": 0.728712,+ "peak_family_mib": 77.887+ }+ ]+ },+ {+ "artifact": {+ "repo": "ggml-org/models",+ "revision": "499bc8821c6b12b4e53c5bffcb21ec206f212d81",+ "member": "tinyllamas/stories260K.gguf",+ "file": "stories260K.gguf",+ "url": "https://huggingface.co/ggml-org/models/resolve/499bc8821c6b12b4e53c5bffcb21ec206f212d81/tinyllamas/stories260K.gguf",+ "bytes": 1185376+ },+ "normalized_sha256": "15f4dbdc30dd9bd936747057031c6391c38b4c1717d56ae33ed2f0cba4184405",+ "answers_sha256": "99257428a01aa513b435c95ca7217c69f4641d9021372a6333e7cb9e2cb35d63",+ "catalog_sha256": "859b1433ef7c45ea23ba4073715d25ed7a76fd9df6d74285d20a5ea39e8e2898",+ "mcp": [+ {+ "pair": 1,+ "side": "baseline",+ "wall_seconds": 6.568,+ "cpu_seconds": 6.375324,+ "peak_family_mib": 743.887,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [223.11, 1938.73, 102.018, 10.759, 16.11]+ },+ {+ "pair": 1,+ "side": "candidate",+ "wall_seconds": 6.566,+ "cpu_seconds": 6.338327,+ "peak_family_mib": 744.852,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [248.979, 1886.809, 105.801, 10.738, 15.653]+ },+ {+ "pair": 2,+ "side": "baseline",+ "wall_seconds": 6.365,+ "cpu_seconds": 6.206216,+ "peak_family_mib": 783.84,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [248.196, 1845.737, 112.874, 10.724, 17.594]+ },+ {+ "pair": 2,+ "side": "candidate",+ "wall_seconds": 7.383,+ "cpu_seconds": 7.211823,+ "peak_family_mib": 742.281,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [233.155, 1909.721, 104.803, 11.603, 19.357]+ },+ {+ "pair": 3,+ "side": "baseline",+ "wall_seconds": 6.565,+ "cpu_seconds": 6.323702,+ "peak_family_mib": 740.84,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [233.664, 1918.301, 104.389, 14.561, 15.227]+ },+ {+ "pair": 3,+ "side": "candidate",+ "wall_seconds": 6.566,+ "cpu_seconds": 6.315785,+ "peak_family_mib": 778.133,+ "catalog_plus_calls_tokens": 895421,+ "calls_ms": [232.37, 1851.929, 78.318, 21.857, 16.438]+ }+ ],+ "phase": [+ {+ "pair": 1,+ "side": "baseline",+ "first_ms": 35.366,+ "repeat_median_ms": 7.784,+ "wall_seconds": 0.625,+ "cpu_seconds": 0.429065,+ "peak_family_mib": 62.922+ },+ {+ "pair": 1,+ "side": "candidate",+ "first_ms": 30.009,+ "repeat_median_ms": 7.934,+ "wall_seconds": 0.624,+ "cpu_seconds": 0.392848,+ "peak_family_mib": 59.664+ },+ {+ "pair": 2,+ "side": "baseline",+ "first_ms": 32.0,+ "repeat_median_ms": 8.772,+ "wall_seconds": 0.624,+ "cpu_seconds": 0.412059,+ "peak_family_mib": 60.422+ },+ {+ "pair": 2,+ "side": "candidate",+ "first_ms": 32.032,+ "repeat_median_ms": 9.36,+ "wall_seconds": 0.624,+ "cpu_seconds": 0.412412,+ "peak_family_mib": 59.441+ },+ {+ "pair": 3,+ "side": "baseline",+ "first_ms": 33.628,+ "repeat_median_ms": 9.496,+ "wall_seconds": 0.624,+ "cpu_seconds": 0.396232,+ "peak_family_mib": 60.41+ },+ {+ "pair": 3,+ "side": "candidate",+ "first_ms": 30.506,+ "repeat_median_ms": 8.172,+ "wall_seconds": 0.622,+ "cpu_seconds": 0.404398,+ "peak_family_mib": 61.672+ }+ ]+ },+ {+ "artifact": {+ "repo": "hf-internal-testing/tiny-random-bert",+ "revision": "f171d7baecaf37b5da5a3616d8833b9969753535",+ "member": "model.safetensors",+ "file": "tiny-bert.safetensors",+ "url": "https://huggingface.co/hf-internal-testing/tiny-random-bert/resolve/f171d7baecaf37b5da5a3616d8833b9969753535/model.safetensors",+ "bytes": 520212+ },+ "normalized_sha256": "8658d90261cdbaaf1f5f8d2e9408ad301b40ad3e6b7aab7ac9c5d4be71ca36d5",+ "answers_sha256": "82072404a0aa24e3b42979397421f455da2589c2ea649f28ba7279d43e6c3bc8",+ "catalog_sha256": "859b1433ef7c45ea23ba4073715d25ed7a76fd9df6d74285d20a5ea39e8e2898",+ "mcp": [+ {+ "pair": 1,+ "side": "baseline",+ "wall_seconds": 6.667,+ "cpu_seconds": 6.46146,+ "peak_family_mib": 741.602,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [251.201, 1965.785, 114.055, 10.837, 28.862]+ },+ {+ "pair": 1,+ "side": "candidate",+ "wall_seconds": 6.767,+ "cpu_seconds": 6.519606,+ "peak_family_mib": 736.367,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [255.557, 1979.227, 86.726, 10.093, 24.531]+ },+ {+ "pair": 2,+ "side": "baseline",+ "wall_seconds": 6.565,+ "cpu_seconds": 6.379249,+ "peak_family_mib": 730.273,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [245.517, 1900.802, 94.351, 10.0, 25.635]+ },+ {+ "pair": 2,+ "side": "candidate",+ "wall_seconds": 6.569,+ "cpu_seconds": 6.388131,+ "peak_family_mib": 785.746,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [250.715, 1846.568, 103.314, 21.618, 27.261]+ },+ {+ "pair": 3,+ "side": "baseline",+ "wall_seconds": 6.771,+ "cpu_seconds": 6.530512,+ "peak_family_mib": 745.098,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [255.441, 1949.189, 110.897, 10.679, 24.345]+ },+ {+ "pair": 3,+ "side": "candidate",+ "wall_seconds": 6.466,+ "cpu_seconds": 6.296937,+ "peak_family_mib": 741.348,+ "catalog_plus_calls_tokens": 893755,+ "calls_ms": [243.919, 1895.978, 113.995, 10.66, 27.269]+ }+ ],+ "phase": [+ {+ "pair": 1,+ "side": "baseline",+ "first_ms": 27.648,+ "repeat_median_ms": 8.11,+ "wall_seconds": 0.623,+ "cpu_seconds": 0.394405,+ "peak_family_mib": 57.281+ },+ {+ "pair": 1,+ "side": "candidate",+ "first_ms": 27.003,+ "repeat_median_ms": 8.082,+ "wall_seconds": 0.523,+ "cpu_seconds": 0.383678,+ "peak_family_mib": 57.266+ },+ {+ "pair": 2,+ "side": "baseline",+ "first_ms": 26.941,+ "repeat_median_ms": 8.04,+ "wall_seconds": 0.626,+ "cpu_seconds": 0.401321,+ "peak_family_mib": 59.504+ },+ {+ "pair": 2,+ "side": "candidate",+ "first_ms": 28.012,+ "repeat_median_ms": 8.006,+ "wall_seconds": 0.625,+ "cpu_seconds": 0.392541,+ "peak_family_mib": 59.012+ },+ {+ "pair": 3,+ "side": "baseline",+ "first_ms": 28.734,+ "repeat_median_ms": 8.499,+ "wall_seconds": 0.624,+ "cpu_seconds": 0.411118,+ "peak_family_mib": 59.23+ },+ {+ "pair": 3,+ "side": "candidate",+ "first_ms": 29.961,+ "repeat_median_ms": 8.632,+ "wall_seconds": 0.625,+ "cpu_seconds": 0.41875,+ "peak_family_mib": 57.5+ }+ ]+ }+ ],+ "published_equivalent_source": "2f091ccc734d7bd945bd61432d41cf5ba68be12b",+ "candidate_reproduction": "Use baseline_source plus the byte-identical src/model/GgufHeader.ts from published_equivalent_source; driver SHA-256 is pinned separately. Measured local candidate predates the integration rebase.",+ "post_review_receipt_fix": {+ "measured_driver_version": "2f091ccc734d7bd945bd61432d41cf5ba68be12b",+ "driver_sha256": "449860fb973418f2e3e51183a81be1e3111ffa6453de8208a4b175d3d410322f",+ "publication_owner_sha256": "901092ae6dc7fda1a8869d62362a0a9f560e03d5ef5667a5c6ed660db000b9a2",+ "validation": "3 new fault-injection regressions reject original driver and pass current; 11 shared receipt regressions pass. Real stories15M MCP and seven-inspection phase reruns qualify; original paired samples retain their measured driver hash."+ }+}diff --git a/experiments/analysis-efficiency/model-utf8-validation.md b/experiments/analysis-efficiency/model-utf8-validation.mdnew file mode 100644index 000000000..d0cf6594d--- /dev/null+++ b/experiments/analysis-efficiency/model-utf8-validation.md@@ -0,0 +1,147 @@+# Omitted GGUF string validation++[Issue #5774](https://github.com/morluto/rea/issues/5774) follows the existing+real-model and completed-task audit in [#5603](https://github.com/morluto/rea/pull/5603).+The [coverage reconciliation](coverage-review-20261021.md) explains why neither+model inspection nor IL2CPP, firmware extraction or IDA should be described as+wholly unaudited. This experiment changes only one model producer hot path.++## Owner and preserved semantics++GGUF arrays retain eight preview values. Previously, each omitted string still+constructed a fatal UTF-8 decoder and materialized discarded text. `GgufHeader`+now validates an omitted value of at most one existing 65,536-byte chunk with+Node's `isUtf8`. Longer values retain the streaming fatal decoder, including+multi-byte characters split across chunks. Retained values use their original+decoder, preserving BOMs and exact text.++Every admitted byte is still read, hashed and validated. Header bounds,+representation admission, cancellation checks, array-loop fairness, source+offsets, partial-progress semantics, preview omission and complete retained+Evidence are unchanged. There is no cache, new contract, selector or default.+CLI and MCP share this producer through the existing application workflow.++## Corpus and correctness++The [curated receipts](model-utf8-validation.json) pin source commits, runtime+file tables, driver/resource sources, immutable public URLs, artifact digests,+complete report/answer fingerprints and all final samples. Baseline is main+`264862bf2e3612c9cb6fcfb861843cbba4e1893f`; measured candidate code is+`c2550343173c5abd8f34e925eac1bcc12b7ee6a2`. Later documentation/integration+commits do not relabel that cohort. That measured local commit predates the+rebase: its production file is byte-identical to published+`2f091ccc734d7bd945bd61432d41cf5ba68be12b`, with both source hashes recorded.++| Artifact | Bytes | Metadata / tensors | Independent reader |+| ------------------------------------------------ | ---------: | -----------------: | ------------------ |+| ggml-org stories15M Q4_0 GGUF | 19,077,344 | 20 / 57 | gguf 0.19.0 |+| ggml-org stories260K GGUF | 1,185,376 | 19 / 48 | gguf 0.19.0 |+| hf-internal-testing tiny-random-bert safetensors | 520,212 | 2 / 100 | safetensors 0.8.0 |++Official independent readers match all exposed metadata values/array previews+and counts, tensor names, types, shapes and parameter counts. GGUF reader+offsets/lengths match every tensor; safetensors reader geometry and independent+Python JSON offsets match its file ranges. Python byte hashing checks every+header digest. No weights are executed or loaded into a model framework.+The main GGUF has 32,000 vocabulary entries; the smaller GGUF and safetensors+are controls, not evidence for larger vocabularies or other GGUF versions.++## Completed public tasks++Linux x64, Node 24.18.0/npm 11.16.0. Three alternating baseline/candidate,+candidate/baseline, baseline/candidate pairs per artifact run serially. Each+fresh production stdio MCP task discovers all 212 tools, directly selects two+metadata records, retrieves two exact tensors from retained Evidence, repeats+both calls, exports complete Evidence, validates answers and closes the+transport. OS caches are unflushed; the controller reads inputs for hashing.+Fresh process does not mean cold disk or cold model-engine analysis.++Medians of three whole tasks per side, including SDK/catalog/schema validation,+all five calls, source checks, export and teardown:++| Artifact | Wall seconds, before → after | Family CPU seconds, before → after | Peak family MiB, before → after |+| ----------- | ---------------------------: | ---------------------------------: | ------------------------------: |+| stories15M | 6.667 → 6.567 | 6.498 → 6.386 | 771.0 → 816.3 |+| stories260K | 6.565 → 6.566 | 6.324 → 6.338 | 743.9 → 744.9 |+| tiny BERT | 6.667 → 6.569 | 6.461 → 6.388 | 741.6 → 741.3 |++The primary task's observed wall reduction is about 1.5%, but the unchanged+safetensors control has similar variation. No stable whole-task speedup is+established. Main-model peak memory rises: ranges 769.3–799.0 → 813.6–818.9+MiB; its cause is unmeasured. No peak-memory improvement is claimed.++Separate fresh-process provider cohorts perform one first inspection and six+repeated inspections, comparing the complete report after every operation:++| Artifact | First / repeat median ms, before | First / repeat median ms, after |+| ----------- | -------------------------------: | ------------------------------: |+| stories15M | 154.03 / 53.58 | 121.41 / 37.19 |+| stories260K | 33.63 / 8.77 | 30.51 / 8.17 |+| tiny BERT | 27.65 / 8.11 | 28.01 / 8.08 |++For stories15M this is about 21% faster first inspection and 31% faster repeated+inspection. Median CPU for the seven-inspection process is 0.855 → 0.729 s;+peak family memory is 75.6 → 77.9 MiB. Phase gains do not include discovery+or substitute for the complete-task results above.++One catalog plus all named request/reply JSON chunks estimates median+895,583 / 895,421 / 893,755 `cl100k_base` tokens for the three artifacts in+both implementations, using tiktoken 0.12.0. This includes serialized output+schemas, not merely named input schemas. It is an SDK serialization proxy;+actual model usage, repeated host injection, negotiation and framing are+unmeasured. No token saving is claimed.++All 18 public tasks and 18 provider processes completed under one CPU, 4 GiB+hard / 3 GiB early-stop family memory, no swap and 192 tasks. Payload exit,+counters and owned cleanup are verified; no limit/OOM events occurred. Kernel+cgroup peak includes file cache. Terminal whole-scope counters after scope+removal remain unknown. Raw inputs, exports, profiles and transcripts stay+outside Git.++## Reproduction and validation boundaries++Build the baseline and the same baseline with the published GGUF source file+using the pinned toolchain; this reconstructs the measured local candidate.+Capture a complete+baseline normalized report from existing CLI/MCP Evidence using the same+absolute input path. Run `model-utf8-validation.mjs` with absolute `--runtime`,+`--artifact`, `--expected` and a new `--output` directory under `resource-run.py+--observe-payload-exit`. Add `--phase` for the seven-inspection provider probe.+The driver validates advertised named schemas through the actual SDK and+compares full exported parents, preview facts and tensor selections. A failing+receipt or cleanup invalidates a task; it is not silently retried as success.++Five new producer regressions cover empty omitted strings, exact-chunk strings,+split multi-byte characters, both byte orders, retained BOMs and rejection of+short/long malformed omitted values. Existing malformed-header, admission,+cancellation, partial-progress and public CLI/MCP tests remain required.+An initial new-test assertion incorrectly hashed alignment padding; the+corrected oracle uses the actual declared header, matching existing semantics.+The first test launcher refused built projects; the exact built-project gate+was then used. Neither attempt is a passing check. The initial resource-wrapper+copy failed source-hash lookup before launching a build; corrected guarded+builds qualified before measurement.++Integrated onto main `71b696582`: all 68 tests pass again, and three real public+CLI complete reports plus three production SDK tasks match the original+normalized reports. Typecheck, lint/module boundaries, formatting, metadata,+runtime build, docs checks (five tests) and separate architecture guards pass.+`npm run check` does **not** pass: Knip reports an unlisted `g++` invocation in+the Windows test added by main's #5772. An independent main source checkout,+with matching compiled worker artifacts and an uncached Knip run, reproduces+only that finding. It is outside this change and is left unchanged. Initial+control attempts lacked resolvable worker artifacts and are not clean controls.+Development builds/checks used separate 6/8 GiB hard guards; runtime experiments+retain their 4 GiB limit. No guard stop or OOM occurred.++Greptile identified a receipt defect in the phase driver, not the model parser:+failed inspections/assertions were not recorded before cleanup, and a close+failure could suppress publication. Each attempted phase now records its cost+and primary failure before cleanup. Phase and MCP paths share one atomic receipt+publisher; cleanup failures are independent facts and cannot replace the primary+failure. Three new fault-injection tests fail on the original driver and pass+after the correction; all 11 existing shared receipt tests pass. These stand-ins+qualify failure accounting only. Real stories15M MCP and seven-inspection phase+reruns also pass with verified counters/cleanup. The original 36 paired samples+keep their measured driver hash; the receipt records the post-review driver and+publication-owner hashes separately. No timing cohort is relabeled.diff --git a/experiments/analysis-efficiency/model-utf8-validation.mjs b/experiments/analysis-efficiency/model-utf8-validation.mjsnew file mode 100644index 000000000..fbaa13453--- /dev/null+++ b/experiments/analysis-efficiency/model-utf8-validation.mjs@@ -0,0 +1,243 @@+/** Equivalent model tasks; inputs, expected reports and raw receipts stay outside Git. */+import assert from "node:assert/strict";+import { createHash } from "node:crypto";+import { createWriteStream } from "node:fs";+import { mkdir, mkdtemp, readFile, writeFile } from "node:fs/promises";+import { tmpdir } from "node:os";+import { join, resolve } from "node:path";+import { finished } from "node:stream/promises";+import { pathToFileURL } from "node:url";+import { parseArgs } from "node:util";+import { Client } from "@modelcontextprotocol/client";+import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";+import { createRetainedQueryCaller } from "./retained-query-calls.mjs";+import {+ finalizeRetainedQueryReceipt,+ publishRetainedQueryReceipt,+} from "./retained-query-finalization.mjs";++const { values } = parseArgs({+ options: {+ ...Object.fromEntries(+ ["runtime", "artifact", "expected", "output"].map((name) => [+ name,+ { type: "string" },+ ]),+ ),+ phase: { type: "boolean", default: false },+ },+});+for (const name of ["runtime", "artifact", "expected", "output"])+ assert.ok(values[name] && values[name] === resolve(values[name]), name);+const hash = (value) => createHash("sha256").update(value).digest("hex");+const expected = JSON.parse(await readFile(values.expected, "utf8"));+assert.equal(expected.artifact.path, values.artifact);+await mkdir(values.output, { mode: 0o700 });+const report = {+ artifact_sha256: hash(await readFile(values.artifact)),+ normalized_sha256: hash(JSON.stringify(expected)),+ calls: [],+ answers: [],+};++if (values.phase) {+ const { ModelArtifactProvider } = await import(+ pathToFileURL(join(values.runtime, "dist/model/ModelArtifactProvider.js"))+ );+ const provider = new ModelArtifactProvider();+ try {+ for (let index = 0; index < 7; index++) {+ const sample = {+ phase: index === 0 ? "fresh" : "repeat",+ is_error: true,+ };+ const start = performance.now();+ try {+ const result = await provider.inspect({+ path: values.artifact,+ hash: "header",+ });+ sample.elapsed_ms = performance.now() - start;+ assert.ok(result.ok, JSON.stringify(result));+ assert.deepEqual(result.value, expected);+ sample.is_error = false;+ } catch (cause) {+ sample.elapsed_ms ??= performance.now() - start;+ sample.failure = String(cause?.stack ?? cause);+ throw cause;+ } finally {+ report.calls.push(sample);+ }+ }+ report.status = "passed";+ } catch (cause) {+ report.status = "failed";+ report.failure = String(cause?.stack ?? cause);+ process.exitCode = 1;+ } finally {+ report.cleanup_failures = [];+ try {+ await provider.close();+ } catch (cause) {+ report.cleanup_failures.push(`provider: ${String(cause)}`);+ report.status = "failed";+ }+ if (!(await publishRetainedQueryReceipt({ report, output: values.output })))+ process.exitCode = 1;+ }+} else {+ await runMcpTask();+}++async function runMcpTask() {+ const account = await mkdtemp(join(tmpdir(), "rea-model-utf8-"));+ const client = new Client({ name: "model-utf8-validation", version: "1" });+ const transport = new StdioClientTransport({+ command: process.execPath,+ args: [join(values.runtime, "scripts/rea.mjs"), "mcp"],+ env: {+ HOME: account,+ PATH: process.env.PATH,+ NODE_OPTIONS: "--max-old-space-size=2048",+ UV_THREADPOOL_SIZE: "1",+ REA_LOG_LEVEL: "silent",+ TMPDIR: account,+ },+ stderr: "pipe",+ });+ const serverLog = createWriteStream(join(values.output, "server.log"));+ const serverLogFinished = finished(serverLog);+ serverLogFinished.catch(() => {});+ transport.stderr?.pipe(serverLog);+ const transcript = [];+ const call = createRetainedQueryCaller(client, (receipt, event) => {+ report.calls.push(receipt);+ transcript.push(event);+ });+ const observe = async (name, input, phase) => {+ const reply = await call(name, input, phase);+ assert.notEqual(reply.isError, true, JSON.stringify(reply));+ assert.ok(reply.structuredContent);+ return reply.structuredContent;+ };+ try {+ await client.connect(transport);+ const catalog = await client.listTools();+ assert.equal(catalog.nextCursor, undefined);+ await writeFile(+ join(values.output, "catalog.json"),+ JSON.stringify(catalog),+ );+ report.catalog_sha256 = hash(JSON.stringify(catalog));+ report.catalog_tools = catalog.tools.length;+ const definitions = new Map(catalog.tools.map((tool) => [tool.name, tool]));+ const originalCall = client.callTool.bind(client);+ client.callTool = (input, options) =>+ originalCall(input, {+ ...options,+ toolDefinition: definitions.get(input.name),+ });+ const keys = [+ ...new Set(expected.metadata.slice(0, 2).map((row) => row.key)),+ ];+ const names = expected.tensors.slice(0, 2).map((row) => row.name);+ const fields = [+ "key",+ "type",+ "value",+ "element_count",+ "preview_complete",+ ];+ const tensorFields = [+ "name",+ "type",+ "shape",+ "file_offset",+ "byte_length",+ ];+ const select = (rows, selectedFields) =>+ rows.map((row) =>+ Object.fromEntries(+ ["index", ...selectedFields].map((key) => [key, row[key]]),+ ),+ );+ const observations = [];+ for (const phase of ["fresh", "repeat"]) {+ const metadata = await observe(+ "inspect_model_artifact",+ {+ path: values.artifact,+ view: { collection: "metadata", keys, fields },+ },+ phase,+ );+ const parentId = metadata.normalized_result.parent_evidence_id;+ assert.equal(typeof parentId, "string");+ assert.deepEqual(+ metadata.normalized_result.items,+ select(+ expected.metadata.filter((row) => keys.includes(row.key)),+ fields,+ ),+ );+ const tensors = await observe(+ "inspect_analysis_view",+ {+ source: { kind: "retained-evidence", evidence_id: parentId },+ view: {+ kind: "model-artifact",+ collection: "tensors",+ names,+ fields: tensorFields,+ },+ },+ `${phase}:tensors`,+ );+ assert.equal(tensors.normalized_result.parent_evidence_id, parentId);+ assert.deepEqual(+ tensors.normalized_result.items,+ select(expected.tensors.slice(0, 2), tensorFields),+ );+ observations.push(parentId);+ report.answers.push({+ parent_evidence_id: parentId,+ metadata: metadata.normalized_result.items,+ tensors: tensors.normalized_result.items,+ });+ }+ await observe(+ "export_evidence_bundle",+ { path: join(values.output, "evidence.json") },+ "export",+ );+ const bundle = JSON.parse(+ await readFile(join(values.output, "evidence.json"), "utf8"),+ );+ for (const id of observations) {+ const parent = bundle.records.find((record) => record.evidence_id === id);+ assert.ok(parent);+ assert.deepEqual(parent.normalized_result, expected);+ }+ assert.deepEqual(report.answers[0], report.answers[1]);+ assert.equal(hash(await readFile(values.artifact)), report.artifact_sha256);+ report.status = "passed";+ } catch (cause) {+ report.status = "failed";+ report.failure = String(cause?.stack ?? cause);+ process.exitCode = 1;+ } finally {+ if (+ !(await finalizeRetainedQueryReceipt({+ report,+ client,+ transport,+ account,+ serverLog,+ serverLogFinished,+ output: values.output,+ transcript,+ }))+ )+ process.exitCode = 1;+ }+}diff --git a/experiments/analysis-efficiency/retained-capture-validation.md b/experiments/analysis-efficiency/retained-capture-validation.mdindex 2323554d1..17631322e 100644--- a/experiments/analysis-efficiency/retained-capture-validation.md+++ b/experiments/analysis-efficiency/retained-capture-validation.md@@ -18,7 +18,7 @@ Current history and the existing evidence support these narrower conclusions:| Apktool | [#4777](https://github.com/morluto/rea/issues/4777) is closed. Shared decoding and optional primary-string omission have [three-APK evidence](apktool-string-tables.md). Do not implement another decode cache from the old issue list. || Hopper | [#5017](https://github.com/morluto/rea/pull/5017) already reuses decoded string records. Its [checkpoint](hopper-string-records.md) preserves faster warm queries and the negative whole-task result. || JEB | The [Linux CE audit](jeb-linux-native.md) verifies selected real CLI/MCP reads. It does not justify another wrapper cache; discovery and other platforms remain separate coverage gaps. |-| IDA | Existing [native Windows verification](ida-native-windows.md) qualifies selected correctness/authority paths. It does not establish a comparable full performance audit. |+| IDA | Existing [native Windows verification](ida-native-windows.md) includes selected real completed-task performance and correctness/authority checks. It does not qualify every operation or platform. || JavaScript/browser/Electron | Prior experiments and retained pages already exist. [#4415](https://github.com/morluto/rea/issues/4415) remains exploratory; no safe general Joplin preparation speedup is established. || Other static formats | Managed, Flutter, SQLite, WASM, EVM, PCAP, signals and SAZ already have checkpoints in this directory. Extend a specific measured question rather than create duplicate selectors or caches. || Other providers, languages and infrastructure | Rizin/Cutter/GDB/objdump, Go/Rust, ADB/Frida/inspector and startup/resource owners still need bounded real-workflow questions. Historical local improvements do not prove complete coverage or establish another bottleneck. |diff --git a/experiments/analysis-efficiency/retained-query-finalization.mjs b/experiments/analysis-efficiency/retained-query-finalization.mjsindex 2dd193e94..205ef3022 100644--- a/experiments/analysis-efficiency/retained-query-finalization.mjs+++ b/experiments/analysis-efficiency/retained-query-finalization.mjs@@ -58,6 +58,16 @@ export const finalizeRetainedQueryReceipt = async ({);if (report.cleanup_failures.length > 0) report.retained_workspace = account;+ return publishRetainedQueryReceipt({ report, output, transcript, stdout });+};++/** Share atomic publication and failure accounting with non-MCP phase probes. */+export const publishRetainedQueryReceipt = async ({+ report,+ output,+ transcript,+ stdout = process.stdout,+}) => {report.receipt_failures = [];if (transcript !== undefined)await attempt(report.receipt_failures, "transcript-file", () =>diff --git a/experiments/analysis-efficiency/test_model_utf8_validation.py b/experiments/analysis-efficiency/test_model_utf8_validation.pynew file mode 100644index 000000000..f6d0266d4--- /dev/null+++ b/experiments/analysis-efficiency/test_model_utf8_validation.py@@ -0,0 +1,68 @@+"""Phase receipt fault injection only; these stand-ins do not qualify a model provider."""+import json+from pathlib import Path+import subprocess+import tempfile+import unittest+++class ModelPhaseReceipts(unittest.TestCase):+ def run_phase(self, inspect_body, close_body):+ driver = Path(__file__).with_name('model-utf8-validation.mjs').resolve()+ with tempfile.TemporaryDirectory(prefix='rea-model-phase-test-') as directory:+ root = Path(directory)+ artifact = root / 'input'+ artifact.write_bytes(b'receipt fixture, not a model')+ expected = {'artifact': {'path': str(artifact)}}+ (root / 'expected.json').write_text(json.dumps(expected))+ (root / 'package.json').write_text('{"type":"module"}')+ module = root / 'dist' / 'model'+ module.mkdir(parents=True)+ (module / 'ModelArtifactProvider.js').write_text(+ 'export class ModelArtifactProvider {\n'+ 'async inspect() {' + inspect_body.replace('EXPECTED', json.dumps(expected)) + '}\n'+ 'async close() {' + close_body + '}\n}\n')+ result = subprocess.run([+ 'node', str(driver), '--phase', '--runtime', str(root),+ '--artifact', str(artifact), '--expected', str(root / 'expected.json'),+ '--output', str(root / 'output'),+ ], capture_output=True, text=True, timeout=15)+ self.assertEqual(result.returncode, 1, result.stderr)+ report = json.loads((root / 'output' / 'report.json').read_text())+ self.assertEqual(json.loads(result.stdout), report)+ self.assertEqual(report['status'], 'failed')+ self.assertEqual(report['receipt_failures'], [])+ return report++ def test_rejected_inspection_keeps_cost_and_primary_failure_after_close_failure(self):+ report = self.run_phase(+ 'throw new Error("inspection failed");',+ 'throw new Error("cleanup failed");')+ self.assertEqual(len(report['calls']), 1)+ sample = report['calls'][0]+ self.assertEqual(sample['phase'], 'fresh')+ self.assertTrue(sample['is_error'])+ self.assertGreaterEqual(sample['elapsed_ms'], 0)+ self.assertIn('inspection failed', sample['failure'])+ self.assertIn('inspection failed', report['failure'])+ self.assertEqual(report['cleanup_failures'], ['provider: Error: cleanup failed'])++ def test_mismatch_keeps_inspection_time_and_failure(self):+ report = self.run_phase('return {ok:true,value:{wrong:true}};', '')+ self.assertEqual(len(report['calls']), 1)+ self.assertTrue(report['calls'][0]['is_error'])+ self.assertGreaterEqual(report['calls'][0]['elapsed_ms'], 0)+ self.assertIn('AssertionError', report['failure'])+ self.assertEqual(report['cleanup_failures'], [])++ def test_close_failure_keeps_all_successful_samples_and_publishes_failed_receipt(self):+ report = self.run_phase('return {ok:true,value:EXPECTED};',+ 'throw new Error("cleanup failed");')+ self.assertEqual(len(report['calls']), 7)+ self.assertTrue(all(not sample['is_error'] for sample in report['calls']))+ self.assertNotIn('failure', report)+ self.assertEqual(report['cleanup_failures'], ['provider: Error: cleanup failed'])+++if __name__ == '__main__':+ unittest.main()diff --git a/src/model/GgufHeader.ts b/src/model/GgufHeader.tsindex 7f598502d..d30c10946 100644--- a/src/model/GgufHeader.ts+++ b/src/model/GgufHeader.ts@@ -1,3 +1,4 @@+import { isUtf8 } from "node:buffer";import { TextDecoder } from "node:util";import { setImmediate } from "node:timers/promises";import type { JsonValue } from "../domain/jsonValue.js";@@ -73,6 +74,13 @@ export class GgufHeader {rejectModel("GGUF name exceeds its format byte limit");if (retain) this.#budget.charge(6 * length + 2);this.reader.guarantee(length);+ // Omitted preview entries still need validation, but not decoded strings.+ // Keep the streaming decoder for values spanning multiple read chunks.+ if (!retain && length <= 65536) {+ if (!isUtf8(await this.reader.take(length)))+ rejectModel("GGUF string is not valid UTF-8");+ return "";+ }const decoder = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true });let result = "";try {diff --git a/tests/boundary/model/modelArtifact.test.ts b/tests/boundary/model/modelArtifact.test.tsindex 22c2a8373..8cbb08ac0 100644--- a/tests/boundary/model/modelArtifact.test.ts+++ b/tests/boundary/model/modelArtifact.test.ts@@ -368,6 +368,91 @@ it("validates booleans beyond an array preview without allocating the full value});});+it.each([false, true])(+ "validates omitted UTF-8 strings across chunk boundaries, big endian=%s",+ async (bigEndian) => {+ const base = ggufFixture({ bigEndian });+ const preview = Array.from({ length: 8 }, (_, n) => `\uFEFFtoken-${n} 🦙`);+ const strings = [+ ...preview,+ "",+ "a".repeat(65533) + "€",+ "a".repeat(65535) + "🦙",+ "\0tail",+ ];+ const fixture = ggufFixture({+ bigEndian,+ metadata: [+ {+ key: "custom.strings",+ type: 9,+ value: Buffer.concat([+ base.u32(8),+ base.u64(BigInt(strings.length)),+ ...strings.map(base.text),+ ]),+ },+ ],+ });+ const { path, provider } = await providerAt(fixture.bytes);+ const result = await provider.inspect({ path, hash: "header" });+ if (!result.ok) throw result.error;+ expect(result.value.metadata[0]).toMatchObject({+ value: preview,+ element_count: strings.length,+ preview_complete: false,+ });+ expect(result.value.artifact).toMatchObject({+ bytes_read: fixture.header.length,+ header_sha256: createHash("sha256").update(fixture.header).digest("hex"),+ });+ expect(result.value.tensors).toHaveLength(2);+ },+);++it.each([+ ["short", Buffer.from([0xc0, 0x80])],+ [+ "chunk-sized",+ Buffer.concat([Buffer.alloc(65535, 0x61), Buffer.from([0xe2])]),+ ],+ [+ "cross-chunk",+ Buffer.concat([+ Buffer.alloc(65535, 0x61),+ Buffer.from([0xf0, 0x9f, 0x61, 0x80]),+ ]),+ ],+] as const)(+ "rejects malformed omitted %s UTF-8 strings",+ async (_, invalid) => {+ const base = ggufFixture();+ const fixture = ggufFixture({+ metadata: [+ {+ key: "custom.strings",+ type: 9,+ value: Buffer.concat([+ base.u32(8),+ base.u64(9n),+ ...Array.from({ length: 8 }, () => base.text("valid")),+ base.u64(BigInt(invalid.length)),+ invalid,+ ]),+ },+ ],+ });+ const { path, provider } = await providerAt(fixture.bytes);+ const result = await provider.inspect({ path, hash: "header" });+ if (result.ok) throw new Error("Invalid omitted UTF-8 string was accepted");+ expect(result.error).toMatchObject({+ _tag: "BinaryTargetError",+ constraint: "file_format_rejected",+ message: expect.stringContaining("GGUF string is not valid UTF-8"),+ });+ },+);+it("refuses an oversized declared header before reading or expanding it", async () => {const bytes = Buffer.alloc(9);bytes.writeBigUInt64LE(100_000_001n);