Skip to content

armgit / rea

Public Imported from GitHub
0

Reverse engineer anything with agents, from app behavior down to native binaries.

fix(browser): preserve capture uncertainty and frame ancestry

morluto27311f88d419
diff --git a/docs/browser-observation.md b/docs/browser-observation.md
index 9479f4b3c..9c52ae877 100644
--- a/docs/browser-observation.md
+++ b/docs/browser-observation.md
@@ -20,6 +20,17 @@ Inspect native SAZ or HAR 1.2 offline with REA, or connect Everywhere's separate
- `observe_web_session` / `rea observe-web-session` arms an observation window of the requested duration for an external user action and records ordered reload, SPA navigation, redirect, failure, lifecycle, and target-termination metadata.
- `discover_webmcp_tools` / `rea discover-webmcp-tools` uses the experimental CDP WebMCP domain to return every in-scope registration and its complete observed `input_schema` declaration and canonical `input_schema_sha256` digest inline. Declarations are untrusted JSON data, including declared types, required fields, enums, examples and reference URLs. REA does not resolve schema references. These fields replace `input_schema_shape`. Registrations with the same name and URL in distinct observed frames retain separate identities. `tool_key` includes the observed frame ID; it is not a stable identity across navigation or browser sessions. Same-frame registration updates replace that owner only, and removal leaves other owners intact. Declared `readOnly`, `untrustedContent`, `consequential`, and `autosubmit` annotation hints are reported per registration: `true`/`false` are declared values, `null` means the declaration did not provide the hint in that capture, and a missing `consequential` key means the hint was not recorded, either in a historical capture or because the observed annotation was invalid. Consult discovery completeness and exclusions to distinguish these cases. Malformed annotation values retain the registration with incomplete coverage and an `invalid_protocol_value` exclusion. Other malformed protocol payloads can fail discovery explicitly. REA never exposes `WebMCP.invokeTool`.
- `compare_web_captures` / `rea compare-web-captures` compares passive captures or exact-step-aligned browser scenarios. Scenario results commit reproducible literal normalization and expose alignment failures plus artifact-level action, screenshot, DOM, accessibility, URL, history, storage, and event differences. Missing or truncated evidence is never treated as equivalence.
+
+Passive DOM comparison checks complete node inventories and complete frame
+parent/child structure independently. Frame IDs and sibling enumeration order
+are capture-local and do not establish a change; duplicate IDs, missing parents,
+and cyclic ancestry remain unknown. A complete frame difference can still prove
+change when DOM nodes are incomplete, and vice versa. Partial DOM or metadata
+collection digests do not prove modification: a missing node or link can result
+from a protocol omission. Rejected DOM facts also mark derived URL and agent-hint
+metadata incomplete, while valid facts from other nodes and documents remain
+available in the capture.
+
- `capture_web_screenshot` / `rea capture-web-screenshot` returns a content-addressed visible-viewport PNG.
- `compare_web_screenshots` / `rea compare-web-screenshots` performs bounded local PNG pixel comparison without OCR or external services.
- The CLI comparison commands accept each capture, screenshot artifact or normalization policy as inline JSON or a JSON file path. Use files for screenshot artifacts and scenario captures, whose embedded PNG bytes can exceed the host's command-line length limit.
@@ -294,6 +305,18 @@ validation and resource bounds. Omitted script metadata makes its affected
section explicitly truncated.
Disallowed-origin entries are filtered before retention.
+Resource-tree inventories distinguish a valid empty `resources` array from a
+missing or malformed collection. Invalid resource entries are omitted with
+incomplete coverage, while valid resources from that frame and other frames
+remain available. Comparing an incomplete resource inventory cannot prove
+addition, removal, or equality; a complete empty inventory can prove removal.
+Resource records require a nonempty type and a string MIME type; absent or
+malformed required fields are recorded as omissions instead of invented defaults.
+An explicitly empty MIME type is valid. The optional content size may be absent;
+an invalid supplied size remains unknown with incomplete resource coverage while
+the resource's other facts are retained. Zero and nonnegative fractional sizes
+are preserved.
+
CDP discovery, connection, and commands wait for the peer or caller cancellation;
a stalled peer can leave an operation waiting. WebSocket messages remain subject
to the [64 MiB limit](#request-boundary). REA does not impose a message-count or
diff --git a/src/browser/CdpCaptureDocuments.test.ts b/src/browser/CdpCaptureDocuments.test.ts
index be8656e8a..d1e20c2a0 100644
--- a/src/browser/CdpCaptureDocuments.test.ts
+++ b/src/browser/CdpCaptureDocuments.test.ts
@@ -268,7 +268,9 @@ describe("CDP frame capture", () => {
it("walks deeply nested frame trees and returns every frame and resource", () => {
let frameTree: Record<string, unknown> = {
frame: { id: "frame-0", url: `${origin}/0` },
- resources: [{ url: `${origin}/0.js`, type: "Script" }],
+ resources: [
+ { url: `${origin}/0.js`, type: "Script", mimeType: "text/javascript" },
+ ],
};
for (let index = 1; index < 2_000; index += 1) {
frameTree = {
@@ -276,7 +278,13 @@ describe("CDP frame capture", () => {
id: `frame-${String(index)}`,
url: `${origin}/${String(index)}`,
},
- resources: [{ url: `${origin}/${String(index)}.js`, type: "Script" }],
+ resources: [
+ {
+ url: `${origin}/${String(index)}.js`,
+ type: "Script",
+ mimeType: "text/javascript",
+ },
+ ],
childFrames: [frameTree],
};
}
diff --git a/src/browser/CdpCaptureDocuments.ts b/src/browser/CdpCaptureDocuments.ts
index f9f97d375..4d3645f1d 100644
--- a/src/browser/CdpCaptureDocuments.ts
+++ b/src/browser/CdpCaptureDocuments.ts
@@ -14,6 +14,7 @@ import {
cdpSnapshotString,
cdpSnapshotValueString,
} from "./CdpSnapshotStrings.js";
+import type { BrowserExclusionReason } from "../domain/browserCompleteness.js";
import { exclusionReasonForUrl } from "./CdpCaptureEventHelpers.js";
import type { CdpCaptureCompleteness } from "./CdpCaptureCompleteness.js";
import {
@@ -43,8 +44,8 @@ export const captureFrames = (
return { items: [] };
}
const items: WebPageInspection["frames"] = [];
- for (const tree of walkFrameTrees(root, () =>
- completeness?.exclude("frames", "invalid_protocol_value"),
+ for (const tree of walkFrameTrees(root, (omittedCount) =>
+ completeness?.exclude("frames", "invalid_protocol_value", omittedCount),
)) {
const frame = recordValue(tree.frame);
const frameId = cdpStringValue(frame?.id);
@@ -61,9 +62,16 @@ export const captureFrames = (
continue;
}
if (maximum !== undefined && items.length >= maximum) continue;
+ const parentFrameId = cdpStringValue(frame.parentId);
+ // Only the enclosing tree's root may legitimately omit parent evidence.
+ if (
+ (tree !== root || Object.hasOwn(frame, "parentId")) &&
+ (parentFrameId === undefined || parentFrameId.length === 0)
+ )
+ completeness?.exclude("frames", "invalid_protocol_value");
items.push({
frame_id: frameId,
- parent_frame_id: cdpStringValue(frame.parentId) ?? null,
+ parent_frame_id: parentFrameId ?? null,
url: sanitized.url,
origin: sanitized.origin,
});
@@ -91,10 +99,19 @@ export const captureResources = (
return { items: [] };
}
const items: CapturedResource[] = [];
- for (const tree of walkFrameTrees(root, () =>
- completeness?.exclude("resources", "invalid_protocol_value"),
+ for (const tree of walkFrameTrees(root, (omittedCount) =>
+ completeness?.exclude("resources", "invalid_protocol_value", omittedCount),
)) {
- for (const resource of recordsValue(tree.resources)) {
+ if (!Array.isArray(tree.resources)) {
+ completeness?.exclude("resources", "invalid_protocol_value", null);
+ continue;
+ }
+ for (const entry of tree.resources) {
+ const resource = recordValue(entry);
+ if (resource === undefined) {
+ completeness?.exclude("resources", "invalid_protocol_value");
+ continue;
+ }
const url = allowedSanitizedUrl(resource.url, allowedOrigins);
if (url === undefined || url.origin === null) {
completeness?.exclude(
@@ -103,15 +120,26 @@ export const captureResources = (
);
continue;
}
- const contentSize = numberValue(resource.contentSize);
+ const type = cdpStringValue(resource.type);
+ const mimeType = cdpStringValue(resource.mimeType);
+ if (type === undefined || type.length === 0 || mimeType === undefined) {
+ completeness?.exclude("resources", "invalid_protocol_value");
+ continue;
+ }
+ const rawContentSize = numberValue(resource.contentSize);
+ const contentSize =
+ rawContentSize !== undefined && rawContentSize >= 0
+ ? rawContentSize
+ : null;
+ if (resource.contentSize !== undefined && contentSize === null)
+ completeness?.exclude("resources", "invalid_protocol_value");
items.push({
rawUrl: cdpStringValue(resource.url) ?? "",
url: url.url,
origin: url.origin,
- type: cdpStringValue(resource.type) ?? "Other",
- mime_type: cdpStringValue(resource.mimeType) ?? "",
- content_size:
- contentSize === undefined ? null : Math.max(0, contentSize),
+ type,
+ mime_type: mimeType,
+ content_size: contentSize,
});
}
}
@@ -120,7 +148,7 @@ export const captureResources = (
export const walkFrameTrees = function* (
root: UnknownRecord,
- onMalformedChild?: () => void,
+ onMalformedChild?: (omittedCount: number | null) => void,
): Generator<UnknownRecord> {
const pending = [root];
while (pending.length > 0) {
@@ -128,11 +156,11 @@ export const walkFrameTrees = function* (
if (tree === undefined) return;
const rawChildren = tree.childFrames;
if (rawChildren !== undefined && !Array.isArray(rawChildren))
- onMalformedChild?.();
+ onMalformedChild?.(null);
const children = Array.isArray(rawChildren)
? rawChildren.flatMap((child) => {
const parsed = recordValue(child);
- if (parsed === undefined) onMalformedChild?.();
+ if (parsed === undefined) onMalformedChild?.(null);
return parsed === undefined ? [] : [parsed];
})
: [];
@@ -187,14 +215,16 @@ export function* captureDom(
let excludedUrls = 0;
let total = 0;
let unresolvedSvgNodes = 0;
- for (const document of recordsValue(root.documents)) {
+ if (!Array.isArray(root.strings))
+ excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
+ for (const document of snapshotDocuments(root.documents, completeness)) {
const documentUrl = cdpSnapshotString(strings, document.documentURL);
if (documentUrl === undefined) {
- completeness?.exclude("dom", "invalid_protocol_value");
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (allowedSanitizedUrl(documentUrl, allowedOrigins) === undefined) {
- completeness?.exclude("dom", exclusionReasonForUrl(documentUrl));
+ excludeDocumentEvidence(completeness, exclusionReasonForUrl(documentUrl));
continue;
}
const baseUrlValue =
@@ -202,12 +232,15 @@ export function* captureDom(
? documentUrl
: cdpSnapshotValueString(strings, document.baseURL);
if (baseUrlValue === undefined) {
- completeness?.exclude("dom", "invalid_protocol_value");
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const baseUrl = baseUrlValue || documentUrl;
const documentNodes = recordValue(document.nodes);
- if (documentNodes === undefined) continue;
+ if (documentNodes === undefined) {
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
+ continue;
+ }
const rawNodeTypes = numberArray(documentNodes.nodeType);
const rawNodeNames = numberArray(documentNodes.nodeName);
const rawNodeValues = numberArray(documentNodes.nodeValue);
@@ -225,12 +258,12 @@ export function* captureDom(
rawNodeValues.length !== rawNodeTypes.length ||
rawParents.length !== rawNodeTypes.length
) {
- completeness?.exclude("dom", "invalid_protocol_value");
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
if (rawAttributes.length > rawNodeTypes.length)
- completeness?.exclude(
- "dom",
+ excludeDocumentEvidence(
+ completeness,
"invalid_protocol_value",
rawAttributes.length - rawNodeTypes.length,
);
@@ -266,7 +299,7 @@ export function* captureDom(
attributeIndexes === undefined ||
!cdpSnapshotAttributesResolve(strings, attributeIndexes)
) {
- completeness?.exclude("dom", "invalid_protocol_value");
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
continue;
}
const attributes = attributeIndexes as readonly number[];
@@ -346,6 +379,32 @@ export function* captureDom(
};
}
+/** Skipped document facts also make their derived URL and hint inventory incomplete. */
+const excludeDocumentEvidence = (
+ completeness: CdpCaptureCompleteness | undefined,
+ reason: BrowserExclusionReason,
+ count: number | null = 1,
+): void => {
+ completeness?.exclude("dom", reason, count);
+ completeness?.exclude("metadata", reason, count);
+};
+
+const snapshotDocuments = (
+ value: unknown,
+ completeness: CdpCaptureCompleteness | undefined,
+): readonly UnknownRecord[] => {
+ if (!Array.isArray(value) || value.length === 0) {
+ excludeDocumentEvidence(completeness, "invalid_protocol_value", null);
+ return [];
+ }
+ return value.flatMap((item) => {
+ const document = recordValue(item);
+ if (document === undefined)
+ excludeDocumentEvidence(completeness, "invalid_protocol_value");
+ return document === undefined ? [] : [document];
+ });
+};
+
function* captureDomMetadata(
pending: readonly PendingDomMetadata[],
completeness: CdpCaptureCompleteness | undefined,
diff --git a/src/domain/webCaptureDiff.ts b/src/domain/webCaptureDiff.ts
index 39e88e6d6..e062582a5 100644
--- a/src/domain/webCaptureDiff.ts
+++ b/src/domain/webCaptureDiff.ts
@@ -9,6 +9,7 @@ import {
type WebCaptureDimension,
} from "./webCaptureDiffSchemas.js";
import type { WebMcpDiscovery } from "./webMcpDiscovery.js";
+import { fingerprintWebCaptureFrames } from "./webCaptureFrameComparison.js";
type Dimension = WebCaptureDimension;
type Change = WebCaptureChange;
@@ -172,13 +173,7 @@ const compareWebCaptureDimensions = (
const before = input.before.inspection;
const after = input.after.inspection;
return {
- dom_structure: compareDimension(
- singleton("document", digestCanonicalValue(domProjection(before))),
- singleton("document", digestCanonicalValue(domProjection(after))),
- sectionsComplete(before, ["frames", "dom"]) &&
- sectionsComplete(after, ["frames", "dom"]),
- "DOM or frame capture was incomplete in at least one observation.",
- ),
+ dom_structure: domDimension(before, after),
scripts: compareDimension(
keyed(
before.scripts.items.map(scriptProjection),
@@ -200,9 +195,10 @@ const compareWebCaptureDimensions = (
"Resource inventory was incomplete in at least one observation.",
),
network: networkDimension(before, after),
- metadata: compareDimension(
- singleton("metadata", digestCanonicalValue(metadataProjection(before))),
- singleton("metadata", digestCanonicalValue(metadataProjection(after))),
+ metadata: compareAggregate(
+ "metadata",
+ metadataProjection(before),
+ metadataProjection(after),
sectionsComplete(before, ["metadata"]) &&
sectionsComplete(after, ["metadata"]),
"Safe metadata capture was incomplete in at least one observation.",
@@ -601,17 +597,64 @@ const incompleteSections = (completeness: {
...completeness.unavailable_sections,
]);
-const domProjection = (inspection: WebPageInspection) => ({
- frames: inspection.frames
- .map(({ url, origin }) => ({ url, origin }))
- .sort(
- (left, right) =>
- compareUnicodeCodePoints(left.url, right.url) ||
- compareUnicodeCodePoints(left.origin ?? "", right.origin ?? "") ||
- Number(left.origin !== null) - Number(right.origin !== null),
- ),
- nodes: inspection.dom.nodes.map(({ index: _index, ...node }) => node),
-});
+/** Whole-collection digest differences cannot prove a change across omissions. */
+const compareAggregate = (
+ identity: string,
+ before: unknown,
+ after: unknown,
+ complete: boolean,
+ reason: string,
+): Dimension =>
+ compareDimension(
+ complete ? singleton(identity, digestCanonicalValue(before)) : new Map(),
+ complete ? singleton(identity, digestCanonicalValue(after)) : new Map(),
+ complete,
+ reason,
+ );
+
+const domDimension = (
+ before: WebPageInspection,
+ after: WebPageInspection,
+): Dimension => {
+ const beforeFrames = fingerprintWebCaptureFrames(before.frames);
+ const afterFrames = fingerprintWebCaptureFrames(after.frames);
+ const nodesComplete = [before, after].every(
+ (inspection) =>
+ sectionsComplete(inspection, ["dom"]) &&
+ inspection.dom.total_nodes === inspection.dom.nodes.length,
+ );
+ const framesComplete =
+ beforeFrames !== null &&
+ afterFrames !== null &&
+ [before, after].every((inspection) =>
+ sectionsComplete(inspection, ["frames"]),
+ );
+ const changes: Change[] = [];
+ if (
+ nodesComplete &&
+ digestCanonicalValue(before.dom.nodes) !==
+ digestCanonicalValue(after.dom.nodes)
+ )
+ changes.push({ identity: "document", change: "modified" });
+ if (framesComplete && beforeFrames !== afterFrames)
+ changes.push({ identity: "frames", change: "modified" });
+ if (changes.length > 0)
+ return {
+ status: "changed",
+ total_changes: changes.length,
+ changes,
+ reason: null,
+ };
+ return nodesComplete && framesComplete
+ ? { status: "unchanged", total_changes: 0, changes: [], reason: null }
+ : {
+ status: "unknown",
+ total_changes: 0,
+ changes: [],
+ reason:
+ "DOM or frame capture was incomplete, or frame ancestry was unresolved in at least one observation.",
+ };
+};
const scriptProjection = (
script: WebPageInspection["scripts"]["items"][number],
diff --git a/src/domain/webCaptureFrameComparison.test.ts b/src/domain/webCaptureFrameComparison.test.ts
new file mode 100644
index 000000000..bd70649a9
--- /dev/null
+++ b/src/domain/webCaptureFrameComparison.test.ts
@@ -0,0 +1,62 @@
+import { describe, expect, it } from "vitest";
+
+import type { WebPageInspection } from "./browserObservationSchemas.js";
+import { fingerprintWebCaptureFrames } from "./webCaptureFrameComparison.js";
+
+const frame = (
+ id: string,
+ parent: string | null = null,
+ url = "https://app.example.test/frame",
+): WebPageInspection["frames"][number] => ({
+ frame_id: id,
+ parent_frame_id: parent,
+ url,
+ origin: "https://app.example.test",
+});
+
+describe("capture-local frame graph normalization", () => {
+ it.each([
+ [frame("child", "missing")],
+ [frame("same"), frame("same")],
+ [frame("a", "b"), frame("b", "a")],
+ [frame("self", "self")],
+ ])("keeps unresolved ancestry unknown (%#)", (...frames) => {
+ expect(fingerprintWebCaptureFrames(frames)).toBeNull();
+ });
+
+ it("preserves multiplicity while ignoring sibling order and frame IDs", () => {
+ const left = [frame("root"), frame("a", "root"), frame("b", "root")];
+ const renamed = [frame("y", "new"), frame("new"), frame("x", "new")];
+ expect(fingerprintWebCaptureFrames(left)).toBe(
+ fingerprintWebCaptureFrames(renamed),
+ );
+ expect(fingerprintWebCaptureFrames(left)).not.toBe(
+ fingerprintWebCaptureFrames(left.slice(0, 2)),
+ );
+ expect(fingerprintWebCaptureFrames(left)).not.toBe(
+ fingerprintWebCaptureFrames([
+ frame("root"),
+ frame("a", "root"),
+ frame("b", "a"),
+ ]),
+ );
+ });
+
+ it("handles deep ancestry without recursive serialization", () => {
+ const frames = Array.from({ length: 5000 }, (_, index) =>
+ frame(String(index), index === 0 ? null : String(index - 1)),
+ );
+ const renamed = frames
+ .map((item) => ({
+ ...item,
+ frame_id: "new-" + item.frame_id,
+ parent_frame_id:
+ item.parent_frame_id === null ? null : "new-" + item.parent_frame_id,
+ }))
+ .reverse();
+ expect(fingerprintWebCaptureFrames(frames)).toMatch(/^[a-f0-9]{64}$/u);
+ expect(fingerprintWebCaptureFrames(frames)).toBe(
+ fingerprintWebCaptureFrames(renamed),
+ );
+ });
+});
diff --git a/src/domain/webCaptureFrameComparison.ts b/src/domain/webCaptureFrameComparison.ts
new file mode 100644
index 000000000..e85f03e2c
--- /dev/null
+++ b/src/domain/webCaptureFrameComparison.ts
@@ -0,0 +1,49 @@
+import type { WebPageInspection } from "./browserObservationSchemas.js";
+import { digestCanonicalValue } from "./canonicalDigest.js";
+
+/** Compare complete frame forests without treating capture-local IDs as identity. */
+export const fingerprintWebCaptureFrames = (
+ frames: WebPageInspection["frames"],
+): string | null => {
+ const byId = new Map(frames.map((frame) => [frame.frame_id, frame]));
+ if (byId.size !== frames.length) return null;
+ const remaining = new Map(frames.map((frame) => [frame.frame_id, 0]));
+ for (const frame of frames) {
+ if (frame.parent_frame_id === null) continue;
+ const count = remaining.get(frame.parent_frame_id);
+ if (count === undefined) return null;
+ remaining.set(frame.parent_frame_id, count + 1);
+ }
+ const ready = frames.filter((frame) => remaining.get(frame.frame_id) === 0);
+ const children = new Map<string, string[]>();
+ const roots: string[] = [];
+ let examined = 0;
+ for (let index = 0; index < ready.length; index += 1) {
+ const frame = ready[index];
+ if (frame === undefined) throw new Error("Missing ready frame");
+ const fingerprint = digestCanonicalValue({
+ url: frame.url,
+ origin: frame.origin,
+ children: (children.get(frame.frame_id) ?? []).sort(),
+ });
+ examined += 1;
+ const parentId = frame.parent_frame_id;
+ if (parentId === null) {
+ roots.push(fingerprint);
+ continue;
+ }
+ const siblings = children.get(parentId) ?? [];
+ siblings.push(fingerprint);
+ children.set(parentId, siblings);
+ const count = (remaining.get(parentId) ?? 0) - 1;
+ remaining.set(parentId, count);
+ if (count === 0) {
+ const parent = byId.get(parentId);
+ if (parent === undefined)
+ throw new Error("Missing admitted parent frame");
+ ready.push(parent);
+ }
+ }
+ // Cyclic ancestry never reaches the leaf queue and cannot prove equality.
+ return examined === frames.length ? digestCanonicalValue(roots.sort()) : null;
+};
diff --git a/tests/boundary/browser/cdpDomComparisonCompleteness.test.ts b/tests/boundary/browser/cdpDomComparisonCompleteness.test.ts
new file mode 100644
index 000000000..d8c5386c3
--- /dev/null
+++ b/tests/boundary/browser/cdpDomComparisonCompleteness.test.ts
@@ -0,0 +1,266 @@
+import { describe, expect, it } from "vitest";
+import { writeFile } from "node:fs/promises";
+import { join } from "node:path";
+
+import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
+import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
+import type { WebPageInspection } from "../../../src/domain/browserObservationSchemas.js";
+import { compareWebCaptures } from "../../../src/domain/webCaptureDiff.js";
+import { compareWebCapturesInputSchema } from "../../../src/domain/webCaptureDiffSchemas.js";
+import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
+import { trackBrowser } from "./cdpBrowserProvider.support.js";
+import { runCliJson } from "../../fixtures/cliJsonProcess.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+
+type SnapshotMode =
+ | "valid"
+ | "omitted-node"
+ | "changed-node"
+ | "missing-nodes"
+ | "null-nodes"
+ | "missing-documents"
+ | "invalid-documents"
+ | "empty-documents"
+ | "invalid-document"
+ | "mixed-documents"
+ | "missing-strings";
+
+const snapshot = (origin: string, mode: SnapshotMode) => {
+ const document = {
+ documentURL: 0,
+ nodes: {
+ nodeType: [9, 1],
+ nodeName: [1, mode === "changed-node" ? 6 : 3],
+ nodeValue: [2, 2],
+ parentIndex: [-1, 0],
+ attributes: [[], mode === "omitted-node" ? null : [4, 5]],
+ },
+ };
+ let documents: unknown = [document];
+ if (mode === "missing-nodes") documents = [{ documentURL: 0 }];
+ if (mode === "null-nodes") documents = [{ documentURL: 0, nodes: null }];
+ if (mode === "missing-documents") documents = undefined;
+ if (mode === "invalid-documents") documents = {};
+ if (mode === "empty-documents") documents = [];
+ if (mode === "invalid-document") documents = [null];
+ if (mode === "mixed-documents") documents = [document, null];
+ return {
+ strings:
+ mode === "missing-strings"
+ ? undefined
+ : [origin + "/app", "#document", "", "A", "href", "/target", "SPAN"],
+ documents,
+ };
+};
+
+const startBrowser = async () => {
+ const state = {
+ mode: "valid" as SnapshotMode,
+ nested: false,
+ renamed: false,
+ parentOverride: undefined as { value: unknown } | undefined,
+ };
+ const browser = await startFakeCdpBrowser({
+ commandResult(command, origin) {
+ if (command.method === "DOMSnapshot.captureSnapshot")
+ return snapshot(origin, state.mode);
+ if (command.method !== "Page.getFrameTree") return undefined;
+ const id = (name: string) => (state.renamed ? "new-" + name : name);
+ const child = {
+ frame: {
+ id: id("b"),
+ parentId:
+ state.parentOverride === undefined
+ ? id(state.nested ? "a" : "main")
+ : state.parentOverride.value,
+ loaderId: "loader-b",
+ url: origin + "/b",
+ },
+ };
+ const parent = {
+ frame: {
+ id: id("a"),
+ parentId: id("main"),
+ loaderId: "loader-a",
+ url: origin + "/a",
+ },
+ childFrames: state.nested ? [child] : [],
+ };
+ return {
+ frameTree: {
+ frame: {
+ id: id("main"),
+ loaderId: "loader-main",
+ url: origin + "/app",
+ },
+ childFrames: state.nested ? [parent] : [parent, child],
+ },
+ };
+ },
+ });
+ trackBrowser(browser);
+ const inspect = async () => {
+ const result = await new CdpBrowserProvider().inspectPage(
+ inspectWebPageInputSchema.parse({
+ cdp_endpoint: browser.endpoint,
+ allowed_origins: [browser.allowedOrigin],
+ target_id: "allowed-page",
+ observation_ms: 0,
+ }),
+ );
+ if (!result.ok) throw result.error;
+ return result.value;
+ };
+ return { state, inspect };
+};
+
+const compare = (before: WebPageInspection, after: WebPageInspection) =>
+ compareWebCaptures(
+ compareWebCapturesInputSchema.parse({
+ before: { inspection: before },
+ after: { inspection: after },
+ }),
+ );
+
+describe("DOM protocol omissions at the production capture boundary", () => {
+ it.each([
+ "missing-nodes",
+ "null-nodes",
+ "missing-documents",
+ "invalid-documents",
+ "empty-documents",
+ "invalid-document",
+ "mixed-documents",
+ "missing-strings",
+ ] as const)("retains unknown coverage for %s", async (mode) => {
+ const { state, inspect } = await startBrowser();
+ state.mode = mode;
+ const capture = await inspect();
+ for (const section of ["dom", "metadata"])
+ expect(capture.completeness.unavailable_sections).toContain(section);
+ if (mode === "mixed-documents") {
+ expect(capture.dom.nodes.map(({ node_name }) => node_name)).toEqual([
+ "#document",
+ "A",
+ ]);
+ expect(capture.metadata.dom_urls).toHaveLength(1);
+ }
+ const comparison = compare(capture, capture);
+ expect(comparison.dimensions.dom_structure.status).toBe("unknown");
+ expect(comparison.dimensions.metadata.status).toBe("unknown");
+ });
+
+ it("does not turn a rejected node and its URL into a page change", async () => {
+ const { state, inspect } = await startBrowser();
+ const valid = await inspect();
+ state.mode = "omitted-node";
+ const partial = await inspect();
+ expect(partial.dom.total_nodes).toBe(2);
+ expect(partial.dom.nodes).toHaveLength(1);
+ for (const [before, after] of [
+ [valid, partial],
+ [partial, valid],
+ ]) {
+ if (before === undefined || after === undefined)
+ throw new Error("Missing pair");
+ const comparison = compare(before, after);
+ expect(comparison.dimensions.dom_structure.status).toBe("unknown");
+ expect(comparison.dimensions.metadata.status).toBe("unknown");
+ expect(comparison.overall_status).toBe("unknown");
+ }
+ });
+});
+
+describe("complete frame and DOM comparison", () => {
+ it.each([undefined, null, 17, ""])(
+ "keeps missing or invalid child parent evidence unknown (%#)",
+ async (parent) => {
+ const { state, inspect } = await startBrowser();
+ const before = await inspect();
+ state.parentOverride = { value: parent };
+ const after = await inspect();
+ expect(after.frames).toHaveLength(3);
+ expect(after.completeness.unavailable_sections).toContain("frames");
+ expect(compare(before, after).dimensions.dom_structure.status).toBe(
+ "unknown",
+ );
+ expect(compare(after, after).dimensions.dom_structure.status).toBe(
+ "unknown",
+ );
+ },
+ );
+ it("retains a complete observed node change", async () => {
+ const { state, inspect } = await startBrowser();
+ const before = await inspect();
+ expect(compare(before, before).dimensions.dom_structure.status).toBe(
+ "unchanged",
+ );
+ state.mode = "changed-node";
+ expect(
+ compare(before, await inspect()).dimensions.dom_structure.status,
+ ).toBe("changed");
+ });
+
+ it("compares frame ancestry independently of volatile IDs", async () => {
+ const { state, inspect } = await startBrowser();
+ const before = await inspect();
+ state.renamed = true;
+ expect(
+ compare(before, await inspect()).dimensions.dom_structure.status,
+ ).toBe("unchanged");
+ state.nested = true;
+ const nested = await inspect();
+ expect(nested.frames[2]?.parent_frame_id).toBe("new-a");
+ expect(compare(before, nested).dimensions.dom_structure.status).toBe(
+ "changed",
+ );
+ });
+
+ it("retains proven frame changes when DOM node evidence is incomplete", async () => {
+ const { state, inspect } = await startBrowser();
+ const before = await inspect();
+ state.mode = "omitted-node";
+ state.nested = true;
+ const comparison = compare(before, await inspect());
+ expect(comparison.dimensions.dom_structure.status).toBe("changed");
+ expect(comparison.dimensions.metadata.status).toBe("unknown");
+ });
+});
+
+describe("DOM evidence through the public CLI", () => {
+ it("preserves omissions and independently proven frame changes", async () => {
+ const { state, inspect } = await startBrowser();
+ const root = await createTestTempDirectory("rea-dom-comparison-");
+ const before = join(root, "before.json");
+ const after = join(root, "after.json");
+ await writeFile(before, JSON.stringify({ inspection: await inspect() }));
+ state.mode = "omitted-node";
+ for (const nested of [false, true]) {
+ state.nested = nested;
+ await writeFile(after, JSON.stringify({ inspection: await inspect() }));
+ expect(
+ await runCliJson(["compare-web-captures", before, after, "--json"]),
+ ).toMatchObject({
+ operation: "compare_web_captures",
+ normalized_result: {
+ overall_status: nested ? "changed" : "unknown",
+ dimensions: {
+ dom_structure: { status: nested ? "changed" : "unknown" },
+ metadata: { status: "unknown" },
+ },
+ },
+ });
+ }
+ state.mode = "missing-nodes";
+ await writeFile(after, JSON.stringify({ inspection: await inspect() }));
+ expect(
+ await runCliJson(["compare-web-captures", after, after, "--json"]),
+ ).toMatchObject({
+ operation: "compare_web_captures",
+ normalized_result: {
+ overall_status: "unknown",
+ dimensions: { dom_structure: { status: "unknown" } },
+ },
+ });
+ });
+});
diff --git a/tests/boundary/browser/cdpResourceCompleteness.test.ts b/tests/boundary/browser/cdpResourceCompleteness.test.ts
new file mode 100644
index 000000000..d40e838cc
--- /dev/null
+++ b/tests/boundary/browser/cdpResourceCompleteness.test.ts
@@ -0,0 +1,322 @@
+import { writeFile } from "node:fs/promises";
+import { join } from "node:path";
+import { describe, expect, it } from "vitest";
+
+import { CdpBrowserProvider } from "../../../src/browser/CdpBrowserProvider.js";
+import { inspectWebPageInputSchema } from "../../../src/domain/browserObservation.js";
+import type { WebPageInspection } from "../../../src/domain/browserObservationSchemas.js";
+import { compareWebCaptures } from "../../../src/domain/webCaptureDiff.js";
+import { compareWebCapturesInputSchema } from "../../../src/domain/webCaptureDiffSchemas.js";
+import { startFakeCdpBrowser } from "../../fixtures/fakeCdpBrowser.js";
+import { runCliJson } from "../../fixtures/cliJsonProcess.js";
+import { createTestTempDirectory } from "../../fixtures/temporaryDirectory.js";
+import { trackBrowser } from "./cdpBrowserProvider.support.js";
+
+type ResourceMode =
+ | "valid"
+ | "missing"
+ | "null"
+ | "wrong-type"
+ | "invalid-entry"
+ | "child-missing"
+ | "child-invalid-entry"
+ | "child-wrong-type"
+ | "empty";
+
+const startBrowser = async () => {
+ const state = {
+ mode: "valid" as ResourceMode,
+ peerFields: undefined as Record<string, unknown> | undefined,
+ };
+ const browser = await startFakeCdpBrowser({
+ commandResult(command, origin) {
+ const malformedChildren =
+ state.mode === "child-invalid-entry" ||
+ state.mode === "child-wrong-type";
+ if (command.method === "Page.getFrameTree" && malformedChildren)
+ return {
+ frameTree: {
+ frame: { id: "frame-main", url: origin + "/app" },
+ childFrames:
+ state.mode === "child-wrong-type"
+ ? {}
+ : [
+ {
+ frame: {
+ id: "child",
+ parentId: "frame-main",
+ url: origin + "/child",
+ },
+ },
+ null,
+ ],
+ },
+ };
+ if (command.method !== "Page.getResourceTree") return undefined;
+ const resource = (name: string) => ({
+ url: origin + "/" + name,
+ type: "Script",
+ mimeType: "text/javascript",
+ contentSize: 128,
+ ...(name === "peer.js" ? state.peerFields : undefined),
+ });
+ let resources: unknown = [resource("keep.js"), resource("peer.js")];
+ if (state.mode === "missing") resources = undefined;
+ if (state.mode === "null") resources = null;
+ if (state.mode === "wrong-type") resources = {};
+ if (state.mode === "invalid-entry")
+ resources = [resource("keep.js"), null];
+ if (state.mode === "empty") resources = [];
+ const malformedChildFrames =
+ state.mode === "child-wrong-type"
+ ? {}
+ : state.mode === "child-invalid-entry"
+ ? [
+ {
+ frame: {
+ id: "child",
+ parentId: "frame-main",
+ url: origin + "/child",
+ },
+ resources: [resource("child.js")],
+ },
+ null,
+ ]
+ : undefined;
+ return {
+ frameTree: {
+ frame: { id: "frame-main", url: origin + "/app" },
+ resources,
+ ...(malformedChildFrames === undefined
+ ? {}
+ : { childFrames: malformedChildFrames }),
+ ...(state.mode === "child-missing"
+ ? {
+ childFrames: [
+ {
+ frame: {
+ id: "child",
+ parentId: "frame-main",
+ url: origin + "/child",
+ },
+ },
+ ],
+ }
+ : {}),
+ },
+ };
+ },
+ });
+ trackBrowser(browser);
+ const inspect = async () => {
+ const result = await new CdpBrowserProvider().inspectPage(
+ inspectWebPageInputSchema.parse({
+ cdp_endpoint: browser.endpoint,
+ allowed_origins: [browser.allowedOrigin],
+ target_id: "allowed-page",
+ observation_ms: 0,
+ }),
+ );
+ if (!result.ok) throw result.error;
+ return result.value;
+ };
+ return { state, inspect, origin: browser.allowedOrigin };
+};
+
+const compareResources = (
+ before: WebPageInspection,
+ after: WebPageInspection,
+) =>
+ compareWebCaptures(
+ compareWebCapturesInputSchema.parse({
+ before: { inspection: before },
+ after: { inspection: after },
+ }),
+ ).dimensions.resources;
+
+describe("resource inventory omissions through CDP", () => {
+ it.each([
+ { fields: { type: undefined }, retained: false },
+ { fields: { type: null }, retained: false },
+ { fields: { type: 17 }, retained: false },
+ { fields: { type: "" }, retained: false },
+ { fields: { mimeType: undefined }, retained: false },
+ { fields: { mimeType: null }, retained: false },
+ { fields: { mimeType: 17 }, retained: false },
+ { fields: { contentSize: -1 }, retained: true },
+ { fields: { contentSize: null }, retained: true },
+ { fields: { contentSize: "128" }, retained: true },
+ ])(
+ "keeps malformed resource fields uncertain: $fields",
+ async ({ fields, retained }) => {
+ const { state, inspect, origin } = await startBrowser();
+ const before = await inspect();
+ state.peerFields = fields;
+ const after = await inspect();
+ for (const [left, right] of [
+ [before, after],
+ [after, before],
+ [after, after],
+ ] as const)
+ expect(compareResources(left, right)).toMatchObject({
+ status: "unknown",
+ total_changes: 0,
+ changes: [],
+ });
+ expect(after.completeness.excluded).toContainEqual({
+ section: "resources",
+ reason: "invalid_protocol_value",
+ count: 1,
+ });
+ expect(after.resources.map(({ url }) => url).sort()).toEqual(
+ retained
+ ? [origin + "/keep.js", origin + "/peer.js"]
+ : [origin + "/keep.js"],
+ );
+ if (retained)
+ expect(
+ after.resources.find(({ url }) => url.endsWith("/peer.js")),
+ ).toMatchObject({
+ type: "Script",
+ mime_type: "text/javascript",
+ content_size: null,
+ });
+ },
+ );
+
+ it.each([
+ {
+ fields: { type: "Other", mimeType: "", contentSize: undefined },
+ size: null,
+ },
+ { fields: { contentSize: 0 }, size: 0 },
+ { fields: { contentSize: 1.5 }, size: 1.5 },
+ ])(
+ "accepts valid optional and empty fields: $fields",
+ async ({ fields, size }) => {
+ const { state, inspect } = await startBrowser();
+ state.peerFields = fields;
+ const capture = await inspect();
+ expect(capture.resources).toHaveLength(2);
+ expect(
+ capture.resources.find(({ url }) => url.endsWith("/peer.js"))
+ ?.content_size,
+ ).toBe(size);
+ expect(capture.completeness.unavailable_sections).not.toContain(
+ "resources",
+ );
+ expect(compareResources(capture, capture).status).toBe("unchanged");
+ },
+ );
+
+ it.each([
+ "missing",
+ "null",
+ "wrong-type",
+ "invalid-entry",
+ "child-missing",
+ "child-invalid-entry",
+ "child-wrong-type",
+ ] as const)(
+ "does not claim a resource removal or equality for %s",
+ async (mode) => {
+ const { state, inspect, origin } = await startBrowser();
+ const before = await inspect();
+ expect(before.resources).toHaveLength(2);
+ expect(before.completeness.unavailable_sections).not.toContain(
+ "resources",
+ );
+ state.mode = mode;
+ const after = await inspect();
+ for (const [left, right] of [
+ [before, after],
+ [after, before],
+ [after, after],
+ ] as const)
+ expect(compareResources(left, right)).toMatchObject({
+ status: "unknown",
+ total_changes: 0,
+ changes: [],
+ });
+ expect(after.completeness.unavailable_sections).toContain("resources");
+ expect(after.completeness.excluded).toContainEqual({
+ section: "resources",
+ reason: "invalid_protocol_value",
+ count: mode === "invalid-entry" ? 1 : null,
+ });
+ if (mode === "child-invalid-entry" || mode === "child-wrong-type") {
+ expect(after.completeness.excluded).toContainEqual({
+ section: "frames",
+ reason: "invalid_protocol_value",
+ count: null,
+ });
+ expect(after.frames.map(({ frame_id }) => frame_id)).toContain(
+ "frame-main",
+ );
+ if (mode === "child-invalid-entry")
+ expect(after.frames.map(({ frame_id }) => frame_id)).toContain(
+ "child",
+ );
+ }
+ const expected =
+ mode === "invalid-entry"
+ ? [origin + "/keep.js"]
+ : mode === "child-invalid-entry"
+ ? [origin + "/child.js", origin + "/keep.js", origin + "/peer.js"]
+ : mode === "child-missing"
+ ? [origin + "/keep.js", origin + "/peer.js"]
+ : mode === "child-wrong-type"
+ ? [origin + "/keep.js", origin + "/peer.js"]
+ : [];
+ expect(after.resources.map(({ url }) => url).sort()).toEqual(expected);
+ },
+ );
+
+ it("distinguishes a complete empty inventory from an unavailable collection", async () => {
+ const { state, inspect } = await startBrowser();
+ const before = await inspect();
+ state.mode = "empty";
+ const after = await inspect();
+ expect(after.resources).toEqual([]);
+ expect(after.completeness.unavailable_sections).not.toContain("resources");
+ expect(compareResources(before, after)).toMatchObject({
+ status: "changed",
+ total_changes: 2,
+ changes: [{ change: "removed" }, { change: "removed" }],
+ });
+ expect(compareResources(after, after).status).toBe("unchanged");
+ });
+});
+
+describe("resource uncertainty in public CLI Evidence", () => {
+ it("preserves partial membership while complete inventories still prove removal", async () => {
+ const { state, inspect } = await startBrowser();
+ const root = await createTestTempDirectory("rea-resource-comparison-");
+ const before = join(root, "before.json");
+ const after = join(root, "after.json");
+ await writeFile(before, JSON.stringify({ inspection: await inspect() }));
+ for (const { mode, fields } of [
+ { mode: "invalid-entry", fields: undefined },
+ { mode: "valid", fields: { type: undefined } },
+ { mode: "valid", fields: { contentSize: -1 } },
+ { mode: "empty", fields: undefined },
+ ] as const) {
+ state.mode = mode;
+ state.peerFields = fields;
+ await writeFile(after, JSON.stringify({ inspection: await inspect() }));
+ expect(
+ await runCliJson(["compare-web-captures", before, after, "--json"]),
+ ).toMatchObject({
+ operation: "compare_web_captures",
+ normalized_result: {
+ dimensions: {
+ resources: {
+ status: mode === "empty" ? "changed" : "unknown",
+ total_changes: mode === "empty" ? 2 : 0,
+ },
+ },
+ },
+ });
+ }
+ });
+});