Skip to content

armgit / rea

Public Imported from GitHub
0

Reverse engineer anything with agents, from app behavior down to native binaries.

fix(hopper): load verified FAT64 slices with native Mach-O semantics

morluto0a49d5bfb931
diff --git a/bridge/hopper_bridge.py b/bridge/hopper_bridge.py
index 0c1da8a57..993f43f23 100644
--- a/bridge/hopper_bridge.py
+++ b/bridge/hopper_bridge.py
@@ -282,7 +282,7 @@ def _original_file_offset(document, offset):
"""Translate Hopper's image-relative mapping using the observed loaded header."""
if offset > 9007199254740991:
raise CapabilityUnavailableError("Provider file offset exceeds the exact JSON integer range")
- path = document.getExecutableFilePath()
+ path = globals().get("REA_ORIGINAL_EXECUTABLE_PATH") or document.getExecutableFilePath()
if not path:
raise CapabilityUnavailableError("Hopper did not identify the original executable for file-offset verification")
try:
diff --git a/docs/installation.md b/docs/installation.md
index 459d2d1ce..2b0a97c0f 100644
--- a/docs/installation.md
+++ b/docs/installation.md
@@ -323,14 +323,21 @@ memory has no original file offset. For FAT Mach-O, offsets refer to the origina
container file. Results retain Hopper's image-relative offset, the observed slice
base, and the source executable path.
Loader selection reads the Mach-O container header, including FAT files with a
-single architecture. FAT64 uses the same derived FAT/Mach-O loader chain;
-loading depends on the installed Hopper build. Hopper 6.1.0-demo timed out on
-a FAT64 fixture while FAT32 controls and its extracted thin image loaded.
-This does not establish that other builds reject FAT64. If loading fails, use
-`lipo -archs <source-file>` to read the exact architecture names, then
-`lipo -thin <architecture> <source-file> -output <thin-file>` and open the
-resulting image. FAT64 is distinct from the CPU architecture: FAT32 containers
-can contain 64-bit executable images.
+single architecture. For FAT64, REA validates the architecture table and selected
+Mach-O header, prepares a private thin image, and loads it with Hopper's native
+Mach-O loader. It checks the entire source's SHA-256 while copying the slice;
+source identity and reported offsets still refer to the original container.
+An ambiguous architecture subtype requires an explicitly extracted thin image;
+REA does not guess. Configured loader arguments remain explicit overrides.
+FAT64 is distinct from the CPU architecture: FAT32 containers can contain 64-bit
+executable images. This preparation avoids the Raw Binary loader dialog observed
+with native FAT64 loading on Hopper 6.1.0-demo.
+The prepared image and its owned document close together, including on MCP exit;
+if document closure is unconfirmed, REA retains the backing image and reports
+`cleanup_incomplete` with its path. Ordinary documents retain their existing
+MCP-exit behavior.
+Startup deadlines report missing bridge readiness and preserve the launcher
+outcome; a successful helper exit does not prove that a loader dialog completed.
Cursor navigation returns the observed object start when Hopper snaps an interior
address; adjacent-object navigation rejects unmapped inputs and document ends.
Native API text rejects NUL characters and unpaired Unicode surrogates before
diff --git a/docs/testing.md b/docs/testing.md
index 2c2cbab54..e65cba078 100644
--- a/docs/testing.md
+++ b/docs/testing.md
@@ -217,12 +217,11 @@ The Linux demo lane remains a separate `verify:hopper:linux` command.
existing Xcode clang/lipo toolchain. It compiles arm64/x86-64 thin executables and
one- and two-slice FAT32 containers, verifies exact/interior address mappings
against bytes in the original files, checks CLI/MCP parity, and checks owned
-runtime cleanup. `verify:hopper:fat64` additionally requires the installed Hopper
-to load a compiled FAT64 container and verifies its original-file mappings. It
-fails on a native load or analysis failure rather than treating a timeout as
-proof of unsupported format. This additional lane remains unverified on Hopper
-6.1.0-demo: FAT64 timed out while FAT32 controls and the extracted thin image
-loaded successfully.
+runtime cleanup. `verify:hopper:fat64` additionally checks FAT64 preparation through
+Hopper's native Mach-O loader, source-container mappings, profile identity,
+malformed and ambiguous slice rejection, and temporary-image lifetime. Both
+lanes have been verified on Hopper 6.1.0-demo; this establishes REA's prepared
+FAT64 workflow, not native FAT64-loader support.
Cross-architecture fixture compilation is not required by `verify:hopper`.
Golden tests use immutable captured text inputs with producer/source provenance
diff --git a/scripts/verify-real-hopper-fat.mjs b/scripts/verify-real-hopper-fat.mjs
index 516229cc4..d4709e357 100644
--- a/scripts/verify-real-hopper-fat.mjs
+++ b/scripts/verify-real-hopper-fat.mjs
@@ -1,6 +1,14 @@
import assert from "node:assert/strict";
+import { createHash } from "node:crypto";
import { execFile } from "node:child_process";
-import { access, mkdtemp, readFile, realpath, rm } from "node:fs/promises";
+import {
+ access,
+ mkdtemp,
+ readFile,
+ realpath,
+ rm,
+ writeFile,
+} from "node:fs/promises";
import { join } from "node:path";
import { tmpdir } from "node:os";
import { promisify } from "node:util";
@@ -10,7 +18,7 @@ import { Client } from "@modelcontextprotocol/client";
import { StdioClientTransport } from "@modelcontextprotocol/client/stdio";
import Ajv from "ajv/dist/2020.js";
import addFormats from "ajv-formats";
-import { requireMcpResult } from "./lib/mcp-verifier-results.mjs";
+import { mcpTextValue, requireMcpResult } from "./lib/mcp-verifier-results.mjs";
import { snapshotHopperRuntime } from "./lib/real-hopper-cleanup.mjs";
import { HOPPER_TARGET_LEASE_DIRECTORY } from "../dist/hopper/HopperTargetLease.js";
import { parseConfig } from "../dist/config.js";
@@ -56,6 +64,9 @@ const fat = join(directory, "c-fat");
const fat64 = join(directory, "c-fat64");
const singleFat = join(directory, "c-single-fat");
const owned = new Set();
+const extraFat64 = [];
+const invalidFat64 = [];
+let preparedImagePath;
const before = await snapshotHopperRuntime(
"/tmp",
HOPPER_TARGET_LEASE_DIRECTORY,
@@ -114,6 +125,86 @@ try {
"-output",
singleFat,
]);
+ if (verifyFat64) {
+ const original = await readFile(fat64);
+ const swapped = Buffer.from(original);
+ swapped.writeUInt32BE(0xbfbafeca, 0);
+ swapped.writeUInt32LE(original.readUInt32BE(4), 4);
+ for (let index = 0; index < original.readUInt32BE(4); index++) {
+ const offset = 8 + index * 32;
+ for (const field of [0, 4, 24, 28])
+ swapped.writeUInt32LE(
+ original.readUInt32BE(offset + field),
+ offset + field,
+ );
+ for (const field of [8, 16])
+ swapped.writeBigUInt64LE(
+ original.readBigUInt64BE(offset + field),
+ offset + field,
+ );
+ }
+ const little = join(directory, "c-fat64-little-endian");
+ await writeFile(little, swapped);
+ extraFat64.push(little);
+ const single = join(directory, "c-single-fat64");
+ await run(toolPaths.lipo, [
+ "-create",
+ process.arch === "arm64" ? arm : intel,
+ "-fat64",
+ "-output",
+ single,
+ ]);
+ extraFat64.push(single);
+ for (const [name, reason, mutate] of [
+ [
+ "reserved",
+ /nonzero reserved field/u,
+ (bytes) => bytes.writeUInt32BE(1, 36),
+ ],
+ [
+ "range",
+ /extends beyond the file/u,
+ (bytes) => bytes.writeBigUInt64BE(BigInt(bytes.length), 24),
+ ],
+ [
+ "identity",
+ /disagrees with its Mach-O header/u,
+ (bytes) => bytes.writeUInt32BE(bytes.readUInt32BE(12) + 1, 12),
+ ],
+ [
+ "overlap",
+ /slices overlap/u,
+ (bytes) => {
+ const cpu = process.arch === "arm64" ? 0x100000c : 0x1000007;
+ const base = bytes.readUInt32BE(8) === cpu ? 8 : 40;
+ bytes.copy(bytes, base === 8 ? 40 : 8, base, base + 32);
+ },
+ ],
+ ]) {
+ const bytes = Buffer.from(original);
+ mutate(bytes);
+ const path = join(directory, `c-fat64-invalid-${name}`);
+ await writeFile(path, bytes);
+ invalidFat64.push({ path, reason });
+ }
+ const hostBytes = await readFile(process.arch === "arm64" ? arm : intel);
+ const secondOffset = Math.ceil((16384 + hostBytes.length) / 16384) * 16384;
+ const ambiguous = Buffer.alloc(secondOffset + hostBytes.length);
+ ambiguous.writeUInt32BE(0xcafebabf, 0);
+ ambiguous.writeUInt32BE(2, 4);
+ for (const [index, offset] of [16384, secondOffset].entries()) {
+ const base = 8 + index * 32;
+ ambiguous.writeUInt32BE(hostBytes.readUInt32LE(4), base);
+ ambiguous.writeUInt32BE(hostBytes.readUInt32LE(8), base + 4);
+ ambiguous.writeBigUInt64BE(BigInt(offset), base + 8);
+ ambiguous.writeBigUInt64BE(BigInt(hostBytes.length), base + 16);
+ ambiguous.writeUInt32BE(14, base + 24);
+ hostBytes.copy(ambiguous, offset);
+ }
+ const path = join(directory, "c-fat64-ambiguous");
+ await writeFile(path, ambiguous);
+ invalidFat64.push({ path, reason: /selection.*ambiguous/u });
+ }
await client.connect(transport);
assert.ok(transport.pid !== null);
owned.add(transport.pid);
@@ -135,18 +226,18 @@ try {
);
return result;
};
- let fatExpected;
- let fatAddress;
+ const cliMappings = new Map();
for (const path of [
arm,
intel,
fat,
singleFat,
- ...(verifyFat64 ? [fat64] : []),
+ ...(verifyFat64 ? [fat64, ...extraFat64] : []),
]) {
console.error(`Verifying Hopper source-file mappings for ${path}`);
- await call("open_binary", { path });
const file = await readFile(path);
+ const sourceHash = createHash("sha256").update(file).digest("hex");
+ await call("open_binary", { path });
const procedures = await call("list_procedures");
const entry = procedures.find((item) => item.value.endsWith("rea_entry"));
assert.ok(entry, "compiler fixture omitted rea_entry");
@@ -168,7 +259,12 @@ try {
.toString("hex"),
bytes.bytes_hex,
);
- if (path === fat || path === singleFat || path === fat64) {
+ if (
+ path === fat ||
+ path === singleFat ||
+ path === fat64 ||
+ extraFat64.includes(path)
+ ) {
assert.ok(
mapping.image_base_file_offset > 0,
"FAT mapping omitted its physical slice offset",
@@ -184,35 +280,109 @@ try {
"fixture did not distinguish image-relative and original-file coordinates",
);
} else assert.equal(mapping.image_base_file_offset, 0);
- if (path === fat) {
- fatExpected = mapping;
- fatAddress = address;
+ if (path === fat || path === fat64) {
+ cliMappings.set(path, { mapping, address });
}
observations.push({ path, address, ...mapping, matched_bytes: 16 });
}
+ if (path === fat64 || extraFat64.includes(path)) {
+ const session = await call("binary_session");
+ assert.equal(session.path, path);
+ assert.equal(session.sha256, sourceHash);
+ const profile = session.analysis_provider_binding.analysis_profile;
+ assert.equal(
+ profile.parameters.prepared_image.method,
+ "fat64_thin_slice",
+ );
+ assert.equal(
+ profile.parameters.prepared_image.source_offset,
+ observations.at(-1).image_base_file_offset,
+ );
+ const document = await call("current_document");
+ await call("set_comment", {
+ address: entry.address,
+ comment: "source-bound FAT64 preparation",
+ });
+ await call("open_binary", { path });
+ assert.deepEqual(
+ (await call("binary_session")).analysis_provider_binding
+ .analysis_profile,
+ profile,
+ );
+ assert.equal(await call("current_document"), document);
+ assert.equal(
+ await call("comment", { address: entry.address }),
+ "source-bound FAT64 preparation",
+ );
+ }
+ assert.equal(
+ createHash("sha256")
+ .update(await readFile(path))
+ .digest("hex"),
+ sourceHash,
+ "native analysis modified source executable bytes",
+ );
await call("close_binary");
}
- const pending = run(
- process.execPath,
- [
- "scripts/rea.mjs",
- "address-to-file-offset",
- fat,
- fatAddress,
- "--provider",
- "hopper",
- "--format",
- "json",
- ],
- {
- timeout: 180_000,
- env: { ...process.env, HOPPER_LOADER_ARGS_JSON: "[]" },
- },
- );
- assert.ok(pending.child.pid !== undefined);
- owned.add(pending.child.pid);
- const cli = JSON.parse((await pending).stdout);
- assert.deepEqual(cli.normalized_result, fatExpected);
+ for (const { path, reason } of invalidFat64) {
+ const reply = await client.callTool({
+ name: "open_binary",
+ arguments: { path },
+ });
+ assert.equal(reply.isError, true, "malformed FAT64 unexpectedly launched");
+ assert.match(mcpTextValue(reply), reason);
+ }
+ if (verifyFat64) {
+ await call("open_binary", { path: fat64 });
+ await call("list_procedures");
+ const resources = [
+ ...(await snapshotHopperRuntime(
+ "/tmp",
+ HOPPER_TARGET_LEASE_DIRECTORY,
+ owned,
+ )),
+ ].filter((path) => !before.has(path));
+ for (const root of resources) {
+ const image = join(root, "image.macho");
+ if (
+ await access(image).then(
+ () => true,
+ () => false,
+ )
+ )
+ preparedImagePath = image;
+ }
+ assert.ok(
+ preparedImagePath,
+ "prepared native image was not owned by this session",
+ );
+ await client.close();
+ await transport.close();
+ await assert.rejects(access(preparedImagePath), { code: "ENOENT" });
+ }
+ for (const [path, expected] of cliMappings) {
+ const pending = run(
+ process.execPath,
+ [
+ "scripts/rea.mjs",
+ "address-to-file-offset",
+ path,
+ expected.address,
+ "--provider",
+ "hopper",
+ "--format",
+ "json",
+ ],
+ {
+ timeout: 180_000,
+ env: { ...process.env, HOPPER_LOADER_ARGS_JSON: "[]" },
+ },
+ );
+ assert.ok(pending.child.pid !== undefined);
+ owned.add(pending.child.pid);
+ const cli = JSON.parse((await pending).stdout);
+ assert.deepEqual(cli.normalized_result, expected.mapping);
+ }
} finally {
await client.close();
await transport.close();
@@ -236,6 +406,8 @@ console.log(
observations,
cliMcpParity: true,
fat64: verifyFat64 ? "verified" : "not_run",
+ malformedFat64Rejected: invalidFat64.length,
+ preparedImageRemovedOnMcpShutdown: preparedImagePath !== undefined,
cleanShutdown: true,
},
null,
diff --git a/src/hopper/BridgeLauncher.ts b/src/hopper/BridgeLauncher.ts
index 8e4e7559b..19d50527a 100644
--- a/src/hopper/BridgeLauncher.ts
+++ b/src/hopper/BridgeLauncher.ts
@@ -23,6 +23,11 @@ import {
} from "./LinuxPrivateDisplayProbe.js";
import writeFileAtomic from "write-file-atomic";
+import {
+ prepareHopperMachOImage,
+ type HopperMachOImage,
+} from "./HopperMachOImage.js";
+
import { acquireHopperTargetLease } from "./HopperTargetLease.js";
import type { HopperTargetLease } from "./HopperTargetLease.js";
@@ -41,11 +46,13 @@ export interface BridgeSession {
export type BridgeLaunch =
| (ProviderProcessLaunch & {
readonly shutdownMode: "bridge-request";
+ readonly preparedImagePath?: string;
/** The owned helper may hand off to an external GUI application. */
readonly providerLifetime: "launcher-process" | "external-application";
})
| (Extract<ProviderProcessLaunch, { readonly ownsProcessLifetime: true }> & {
readonly shutdownMode: "process-cleanup";
+ readonly preparedImagePath?: string;
readonly providerLifetime: "launcher-process";
readonly cleanup: NonNullable<ProviderProcessLaunch["cleanup"]>;
});
@@ -69,6 +76,10 @@ interface SharedHopperApplicationLauncherOptions {
readonly targetKind: "executable" | "database";
readonly loaderArgs: readonly string[];
readonly bridgeScriptPath: string;
+ readonly preparedImage?: {
+ readonly image: HopperMachOImage;
+ readonly sourceSha256: string;
+ };
}
/** Explicit launcher contract; the Linux adapter verifies its pinned Hopper build before execution. */
@@ -125,16 +136,30 @@ export class HopperApplicationLauncher implements BridgeLauncher {
const lease = leaseResult.value;
let leaseTransferred = false;
try {
+ let loadedPath = this.options.targetPath;
+ if (this.options.preparedImage !== undefined) {
+ const prepared = await prepareHopperMachOImage(
+ this.options.targetPath,
+ this.options.preparedImage.sourceSha256,
+ this.options.preparedImage.image,
+ join(session.directory, "image.macho"),
+ options.signal,
+ );
+ if (!prepared.ok) return prepared;
+ loadedPath = prepared.value;
+ }
const ownsProcessLifetime = usesLinuxDemo(this.options);
const bootstrap = await this.#writeBootstrap(
session,
ownsProcessLifetime,
+ loadedPath,
);
if (!bootstrap.ok) return bootstrap;
const launched = await this.#launchPreparedTarget({
session,
signal: options.signal,
bootstrapPath: bootstrap.value,
+ loadedPath,
ownsProcessLifetime,
lease,
});
@@ -149,6 +174,7 @@ export class HopperApplicationLauncher implements BridgeLauncher {
readonly session: BridgeSession;
readonly signal: AbortSignal | undefined;
readonly bootstrapPath: string;
+ readonly loadedPath: string;
readonly ownsProcessLifetime: boolean;
readonly lease: HopperTargetLease | undefined;
}): Promise<
@@ -167,7 +193,7 @@ export class HopperApplicationLauncher implements BridgeLauncher {
"-Y",
bootstrapPath,
action,
- this.options.targetPath,
+ input.loadedPath,
];
try {
if (signal?.aborted === true) return err(new HopperCancelledError());
@@ -217,6 +243,9 @@ export class HopperApplicationLauncher implements BridgeLauncher {
return err(new HopperStartError({ cause }));
}
const ownedLauncher = {
+ ...(this.options.preparedImage === undefined
+ ? {}
+ : { preparedImagePath: input.loadedPath }),
process: started.process,
ownsProcessLifetime: true as const,
ownership: started.ownership,
@@ -270,12 +299,18 @@ export class HopperApplicationLauncher implements BridgeLauncher {
async #writeBootstrap(
session: BridgeSession,
ownsProcessLifetime: boolean,
+ loadedPath: string,
): Promise<Result<string, HopperStartError>> {
const bootstrapPath = `${session.directory}/bootstrap.py`;
try {
await writeFile(
bootstrapPath,
- bridgeBootstrapSource(session, this.options, ownsProcessLifetime),
+ bridgeBootstrapSource(
+ session,
+ this.options,
+ ownsProcessLifetime,
+ loadedPath,
+ ),
{ encoding: "utf8", mode: 0o600 },
);
await chmod(bootstrapPath, 0o600);
@@ -321,12 +356,14 @@ const bridgeBootstrapSource = (
session: BridgeSession,
options: SharedHopperApplicationLauncherOptions,
ownsProcessLifetime: boolean,
+ loadedPath: string,
): string =>
[
`REA_SOCKET = ${JSON.stringify(session.socketPath)}`,
`REA_TOKEN = ${JSON.stringify(session.token)}`,
`REA_RUN_ID = ${JSON.stringify(session.runId)}`,
- `REA_TARGET_PATH = ${JSON.stringify(options.targetPath)}`,
+ `REA_TARGET_PATH = ${JSON.stringify(loadedPath)}`,
+ `REA_ORIGINAL_EXECUTABLE_PATH = ${options.targetKind === "executable" ? JSON.stringify(options.targetPath) : "None"}`,
`REA_OWNS_PROCESS_LIFETIME = ${ownsProcessLifetime ? "True" : "False"}`,
`exec(compile(open(${JSON.stringify(options.bridgeScriptPath)}, 'rb').read(), ${JSON.stringify(options.bridgeScriptPath)}, 'exec'))`,
"",
diff --git a/src/hopper/HopperAnalysisProfile.ts b/src/hopper/HopperAnalysisProfile.ts
index d168e0e3d..c9590b830 100644
--- a/src/hopper/HopperAnalysisProfile.ts
+++ b/src/hopper/HopperAnalysisProfile.ts
@@ -19,6 +19,11 @@ import { ProviderAdapterError } from "../domain/providerAdapterError.js";
import { type AnalysisError } from "../domain/analysisErrorBase.js";
import { err, ok, type Result } from "../domain/result.js";
+import {
+ resolveHopperMachOImage,
+ type HopperMachOImage,
+} from "./HopperMachOImage.js";
+
interface HopperProfileOptions {
readonly launcherPath: string;
readonly loaderArgsOverride: readonly string[];
@@ -43,7 +48,20 @@ export const resolveHopperAnalysisProfile = async (
? "fat32"
: "thin";
}
- const derived = hopperLoaderArgsForTarget(target, machoContainer);
+ let preparedImage: HopperMachOImage | undefined;
+ if (
+ target.kind === "executable" &&
+ machoContainer === "fat64" &&
+ options.loaderArgsOverride.length === 0
+ ) {
+ const resolved = await resolveHopperMachOImage(target, options.signal);
+ if (!resolved.ok) return resolved;
+ preparedImage = resolved.value;
+ }
+ const derived = hopperLoaderArgsForTarget(
+ target,
+ preparedImage === undefined ? machoContainer : "thin",
+ );
if (!derived.ok) return derived;
const loaderArgs =
options.loaderArgsOverride.length === 0
@@ -70,6 +88,9 @@ export const resolveHopperAnalysisProfile = async (
...(machoContainer === undefined
? {}
: { macho_container: machoContainer }),
+ ...(preparedImage === undefined
+ ? {}
+ : { prepared_image: { ...preparedImage } }),
loader: {
source:
options.loaderArgsOverride.length === 0
diff --git a/src/hopper/HopperCleanup.test.ts b/src/hopper/HopperCleanup.test.ts
index 6e8018f9f..a26d1ac01 100644
--- a/src/hopper/HopperCleanup.test.ts
+++ b/src/hopper/HopperCleanup.test.ts
@@ -1,3 +1,8 @@
+import { access, rm, writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { createTestTempDirectory } from "../../tests/fixtures/temporaryDirectory.js";
+import { PrivateRuntimeRoot } from "../process/PrivateRuntimeRoot.js";
+
import { EventEmitter } from "node:events";
import { Socket } from "node:net";
@@ -120,3 +125,45 @@ describe("Hopper shutdown rejection diagnostics", () => {
);
});
});
+
+it("keeps a prepared backing image when native document closure cannot be confirmed", async () => {
+ const parent = await createTestTempDirectory("rea-hopper-backing-lifetime-");
+ const runtimeRoot = await PrivateRuntimeRoot.create({ parent });
+ const preparedImagePath = join(runtimeRoot.path, "image.macho");
+ await writeFile(preparedImagePath, "owned backing bytes");
+ const { logger } = loggerHarness();
+ try {
+ const launch = {
+ ...processCleanupLaunch(),
+ preparedImagePath,
+ shutdownMode: "bridge-request" as const,
+ providerLifetime: "external-application" as const,
+ };
+ const result = await cleanupHopperSession({
+ socket: new Socket(),
+ launch,
+ runtimeRoot,
+ processSupervisor: undefined,
+ activeRequest: null,
+ retainDocument: true,
+ progress: undefined,
+ logger,
+ onDiagnostic: undefined,
+ request: () => Promise.reject(failure("document closure unobserved")),
+ releaseTransport: () => undefined,
+ });
+ expect(result).toMatchObject({
+ ok: false,
+ error: {
+ cleanupResources: expect.arrayContaining([
+ "hopper-document",
+ runtimeRoot.path,
+ ]),
+ },
+ });
+ await expect(access(preparedImagePath)).resolves.toBeUndefined();
+ } finally {
+ await runtimeRoot.close();
+ await rm(parent, { recursive: true, force: true });
+ }
+});
diff --git a/src/hopper/HopperCleanup.ts b/src/hopper/HopperCleanup.ts
index 3d31884d8..a4409e295 100644
--- a/src/hopper/HopperCleanup.ts
+++ b/src/hopper/HopperCleanup.ts
@@ -65,7 +65,7 @@ export const cleanupHopperSession = async (
await report(
input.progress,
0,
- input.retainDocument
+ input.retainDocument && input.launch?.preparedImagePath === undefined
? "detaching REA from the Hopper document"
: "requesting Hopper document shutdown",
);
@@ -75,7 +75,20 @@ export const cleanupHopperSession = async (
await stopProcess(input, state);
recordUnconfirmedDocument(input, state);
await report(input.progress, 0.75, "removing Hopper private runtime files");
- await closeRuntimeRoot(input.runtimeRoot, state);
+ if (
+ input.launch?.preparedImagePath !== undefined &&
+ !state.shutdownConfirmed
+ ) {
+ const resource = input.runtimeRoot?.path ?? input.launch.preparedImagePath;
+ state.resources.add(resource);
+ state.issues.push({
+ resource,
+ reason:
+ "Prepared image retained because native document closure was not confirmed",
+ });
+ } else {
+ await closeRuntimeRoot(input.runtimeRoot, state);
+ }
return cleanupOutcome(input, state);
};
@@ -99,7 +112,8 @@ const requestShutdown = async (
return;
}
const method =
- input.retainDocument || input.launch?.shutdownMode === "process-cleanup"
+ (input.retainDocument && input.launch?.preparedImagePath === undefined) ||
+ input.launch?.shutdownMode === "process-cleanup"
? "shutdown"
: "shutdown_document";
const shutdown = await input
diff --git a/src/hopper/HopperMachOImage.ts b/src/hopper/HopperMachOImage.ts
new file mode 100644
index 000000000..0b6ac600e
--- /dev/null
+++ b/src/hopper/HopperMachOImage.ts
@@ -0,0 +1,242 @@
+import { createHash } from "node:crypto";
+import { open, type FileHandle } from "node:fs/promises";
+import { z } from "zod";
+
+import type { BinaryTarget } from "../domain/binaryTarget.js";
+import { BinaryTargetError } from "../domain/configurationErrors.js";
+import { AnalysisCancelledError } from "../domain/analysisErrorCore.js";
+import {
+ readFatSliceDeclarations,
+ validateFatSlice,
+} from "../domain/apple/machoContainer.js";
+import {
+ HopperCancelledError,
+ HopperStartError,
+} from "../domain/hopperErrors.js";
+import { err, ok, type Result } from "../domain/result.js";
+
+/** Stable source coordinates committed before preparing an owned thin image. */
+export const hopperMachOImageSchema = z.strictObject({
+ method: z.literal("fat64_thin_slice"),
+ architecture_count: z.number().int().min(1).max(128),
+ container_table_sha256: z.string().regex(/^[a-f0-9]{64}$/u),
+ source_offset: z.number().int().nonnegative().max(Number.MAX_SAFE_INTEGER),
+ byte_length: z.number().int().positive().max(Number.MAX_SAFE_INTEGER),
+ cpu_type: z.number().int().nonnegative(),
+ cpu_subtype: z.number().int().nonnegative(),
+});
+
+export type HopperMachOImage = z.infer<typeof hopperMachOImageSchema>;
+
+/** Validate the selected FAT64 slice without depending on Apple's lipo tool. */
+export const resolveHopperMachOImage = async (
+ target: BinaryTarget & { readonly kind: "executable" },
+ signal?: AbortSignal,
+): Promise<
+ Result<HopperMachOImage, BinaryTargetError | AnalysisCancelledError>
+> => {
+ const invalid = (reason: string) =>
+ err(new BinaryTargetError(target.path, reason));
+ if (signal?.aborted) return err(new AnalysisCancelledError("open_binary"));
+ let file: FileHandle;
+ try {
+ file = await open(target.path, "r");
+ } catch (cause: unknown) {
+ return invalid(
+ `Cannot read FAT64 source for Hopper image preparation: ${cause instanceof Error ? cause.message : String(cause)}`,
+ );
+ }
+ try {
+ const read = async (offset: number, length: number): Promise<Buffer> => {
+ const bytes = Buffer.alloc(length);
+ const { bytesRead } = await file.read(bytes, 0, length, offset);
+ return bytes.subarray(0, bytesRead);
+ };
+ const header = await read(0, 8);
+ const magic = header.length === 8 ? header.readUInt32BE(0) : 0;
+ if (magic !== 0xcafebabf && magic !== 0xbfbafeca)
+ return invalid(
+ "FAT64 source header changed before Hopper image preparation",
+ );
+ const littleEndian = magic === 0xbfbafeca;
+ const count = littleEndian
+ ? header.readUInt32LE(4)
+ : header.readUInt32BE(4);
+ const size = (await file.stat()).size;
+ const tableEnd = 8 + count * 32;
+ if (count === 0 || count > 128 || tableEnd > size)
+ return invalid(
+ "FAT64 architecture table is truncated or exceeds the supported 128 entries",
+ );
+ const table = await read(8, count * 32);
+ const parsed = readFatSliceDeclarations(table, {
+ count,
+ wide: true,
+ littleEndian,
+ });
+ if (parsed.status === "malformed") return invalid(parsed.reason);
+ const cpuTypes = { x86: 7, x86_64: 0x1000007, arm: 12, arm64: 0x100000c };
+ const matches = [];
+ const ranges = [];
+ for (const slice of parsed.slices) {
+ if (signal?.aborted)
+ return err(new AnalysisCancelledError("open_binary"));
+ const rangeError = validateFatSlice(slice, size, tableEnd);
+ if (rangeError !== null) return invalid(rangeError);
+ const reserved = littleEndian
+ ? table.readUInt32LE(slice.index * 32 + 28)
+ : table.readUInt32BE(slice.index * 32 + 28);
+ if (reserved !== 0)
+ return invalid(
+ `FAT64 architecture ${slice.index} has a nonzero reserved field`,
+ );
+ if (slice.size < 12)
+ return invalid(
+ `FAT64 architecture ${slice.index} has a truncated Mach-O header`,
+ );
+ const embedded = await read(slice.offset, 12);
+ if (embedded.length !== 12)
+ return invalid(`FAT64 architecture ${slice.index} became truncated`);
+ const thinMagic = embedded.readUInt32BE(0);
+ if (![0xfeedface, 0xfeedfacf, 0xcefaedfe, 0xcffaedfe].includes(thinMagic))
+ return invalid(
+ `FAT64 architecture ${slice.index} does not contain a Mach-O image`,
+ );
+ const little = thinMagic === 0xcefaedfe || thinMagic === 0xcffaedfe;
+ const identity = {
+ cpuType: little ? embedded.readUInt32LE(4) : embedded.readUInt32BE(4),
+ cpuSubtype: little
+ ? embedded.readUInt32LE(8)
+ : embedded.readUInt32BE(8),
+ };
+ const identityError = validateFatSlice(slice, size, tableEnd, identity);
+ if (identityError !== null) return invalid(identityError);
+ ranges.push(slice);
+ if (slice.cpuType === cpuTypes[target.architecture]) matches.push(slice);
+ }
+ ranges.sort((a, b) => a.offset - b.offset);
+ for (let index = 1; index < ranges.length; index++) {
+ const previous = ranges[index - 1];
+ const current = ranges[index];
+ if (
+ previous !== undefined &&
+ current !== undefined &&
+ current.offset < previous.offset + previous.size
+ )
+ return invalid("FAT64 slices overlap");
+ }
+ const selected = matches.length === 1 ? matches[0] : undefined;
+ if (selected === undefined)
+ return invalid(
+ `FAT64 selection for ${target.architecture} is ${matches.length === 0 ? "missing" : "ambiguous"}; extract the exact architecture into a thin image before opening it`,
+ );
+ return ok({
+ method: "fat64_thin_slice",
+ architecture_count: count,
+ container_table_sha256: createHash("sha256")
+ .update(header)
+ .update(table)
+ .digest("hex"),
+ source_offset: selected.offset,
+ byte_length: selected.size,
+ cpu_type: selected.cpuType,
+ cpu_subtype: selected.cpuSubtype,
+ });
+ } catch (cause: unknown) {
+ return invalid(
+ `Cannot validate FAT64 image: ${cause instanceof Error ? cause.message : String(cause)}`,
+ );
+ } finally {
+ await file.close();
+ }
+};
+
+/** Copy a selected slice and verify the entire original source in the same pass. */
+export const prepareHopperMachOImage = async (
+ sourcePath: string,
+ sourceSha256: string,
+ image: HopperMachOImage,
+ destination: string,
+ signal?: AbortSignal,
+): Promise<Result<string, HopperStartError | HopperCancelledError>> => {
+ if (signal?.aborted) return err(new HopperCancelledError());
+ try {
+ const source = await open(sourcePath, "r");
+ try {
+ const output = await open(destination, "wx", 0o600);
+ try {
+ const hash = createHash("sha256");
+ const tableHash = createHash("sha256");
+ const tableEnd = 8 + image.architecture_count * 32;
+ const thinHeader: Buffer[] = [];
+ let position = 0;
+ let copied = 0;
+ for await (const chunk of source.createReadStream({
+ autoClose: false,
+ })) {
+ if (signal?.aborted) return err(new HopperCancelledError());
+ const bytes = Buffer.from(chunk);
+ hash.update(bytes);
+ if (position < tableEnd)
+ tableHash.update(
+ bytes.subarray(0, Math.min(bytes.length, tableEnd - position)),
+ );
+ const start = Math.max(0, image.source_offset - position);
+ const end = Math.min(
+ bytes.length,
+ image.source_offset + image.byte_length - position,
+ );
+ if (end > start) {
+ if (copied < 12)
+ thinHeader.push(
+ bytes.subarray(start, Math.min(end, start + 12 - copied)),
+ );
+ await output.writeFile(bytes.subarray(start, end));
+ copied += end - start;
+ }
+ position += bytes.length;
+ }
+ const observed = hash.digest("hex");
+ const header = Buffer.concat(thinHeader);
+ const magic = header.length === 12 ? header.readUInt32BE(0) : 0;
+ const little = magic === 0xcefaedfe || magic === 0xcffaedfe;
+ const headerMatches =
+ header.length === 12 &&
+ [0xfeedface, 0xfeedfacf, 0xcefaedfe, 0xcffaedfe].includes(magic) &&
+ (little ? header.readUInt32LE(4) : header.readUInt32BE(4)) ===
+ image.cpu_type &&
+ (little ? header.readUInt32LE(8) : header.readUInt32BE(8)) ===
+ image.cpu_subtype;
+ if (
+ tableHash.digest("hex") !== image.container_table_sha256 ||
+ !headerMatches
+ )
+ return err(
+ new HopperStartError({
+ userMessage: `FAT64 source ${sourcePath} no longer agrees with its committed architecture table or selected Mach-O header. Open the stable source again.`,
+ }),
+ );
+ if (observed !== sourceSha256 || copied !== image.byte_length)
+ return err(
+ new HopperStartError({
+ userMessage: `FAT64 source ${sourcePath} changed before launch: expected SHA-256 ${sourceSha256}, observed ${observed}; copied ${copied} of ${image.byte_length} selected bytes. Open the stable source again.`,
+ }),
+ );
+ return ok(destination);
+ } finally {
+ await output.close();
+ }
+ } finally {
+ await source.close();
+ }
+ } catch (cause: unknown) {
+ return signal?.aborted
+ ? err(new HopperCancelledError())
+ : err(
+ new HopperStartError({
+ cause,
+ userMessage: `Cannot prepare the selected FAT64 image from ${sourcePath}: ${cause instanceof Error ? cause.message : String(cause)}`,
+ }),
+ );
+ }
+};
diff --git a/src/hopper/HopperProvider.ts b/src/hopper/HopperProvider.ts
index b03d73fd3..57817943f 100644
--- a/src/hopper/HopperProvider.ts
+++ b/src/hopper/HopperProvider.ts
@@ -24,6 +24,7 @@ import {
resolveHopperAnalysisProfile,
} from "./HopperAnalysisProfile.js";
import { HopperClient } from "./HopperClient.js";
+import { hopperMachOImageSchema } from "./HopperMachOImage.js";
import {
CAPABILITIES,
@@ -158,7 +159,30 @@ export class HopperProvider implements AnalysisProviderCandidate {
),
close: () => Promise.resolve(),
};
- const container = profile?.parameters.macho_container;
+ const preparation = profile?.parameters.prepared_image;
+ const parsedImage =
+ preparation === undefined
+ ? undefined
+ : hopperMachOImageSchema.safeParse(preparation);
+ if (parsedImage?.success === false)
+ return {
+ execute: (operation) =>
+ Promise.resolve(
+ err(
+ new AnalysisCapabilityUnavailableError(
+ IDENTITY.id,
+ operation,
+ "Hopper prepared-image profile is malformed; resolve the analysis profile again.",
+ ),
+ ),
+ ),
+ close: () => Promise.resolve(),
+ };
+ const preparedImage = parsedImage?.data;
+ const container =
+ preparedImage === undefined
+ ? profile?.parameters.macho_container
+ : "thin";
const derivedLoaderArgs = hopperLoaderArgsForTarget(
target,
container === "thin" || container === "fat32" || container === "fat64"
@@ -176,6 +200,14 @@ export class HopperProvider implements AnalysisProviderCandidate {
launcherPath: this.config.hopperLauncherPath,
targetPath: target.path,
targetKind: target.kind,
+ ...(preparedImage === undefined
+ ? {}
+ : {
+ preparedImage: {
+ image: preparedImage,
+ sourceSha256: target.sha256,
+ },
+ }),
loaderArgs:
this.config.hopperLoaderArgs.length > 0
? this.config.hopperLoaderArgs
@@ -206,6 +238,11 @@ export class HopperProvider implements AnalysisProviderCandidate {
value: createAnalysisExecution(result.value, executionProvider, {
...(profile === undefined ? {} : { analysisProfile: profile }),
limitations: [
+ ...(preparedImage === undefined
+ ? []
+ : [
+ "REA loaded a verified thin Mach-O slice from the original FAT64 source. Identity and file offsets refer to the original file. The owned temporary image and its Hopper document close together, including on MCP shutdown.",
+ ]),
...(CAPABILITIES.find(
(descriptor) => descriptor.operation === operation,
)?.limitations ?? []),