High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
fix: capture boundingBox returns the content box while /box and annotate return the border box, so overlays under-cover every bordered element (PIN-119)
Luigi Agosti65d95c154e98
diff --git a/docs/endpoints.md b/docs/endpoints.mdindex f4e5f22b..ed75d434 100644--- a/docs/endpoints.md+++ b/docs/endpoints.md@@ -316,9 +316,14 @@ Capture query parameters:`none`; reserved for a future `document.readyState` gate.- `withBounds=true|false` — default `true`. When on, every snapshot nodewith a non-zero backend node id gets a `boundingBox` field and a- `visible` flag. Each bounded node costs one `DOM.getBoxModel` round trip- (~5ms); for the typical interactive-filter snapshot the budget is under- 250ms. Pass `withBounds=false` to skip the per-node work.+ `visible` flag. `boundingBox` is the element's **border box** — the+ painted edge, the same rectangle `GET /box`, `screenshot?annotate=true`+ and `getBoundingClientRect` report, so a box can be cross-checked against+ any of them. It is not the content box: an element with a border or+ padding would otherwise report a rectangle inset from the edge a viewer+ identifies the control by. Each bounded node costs one `DOM.getBoxModel`+ round trip (~5ms); for the typical interactive-filter snapshot the budget+ is under 250ms. Pass `withBounds=false` to skip the per-node work.- `beyondViewport=true|false` — default `false`. When on, the image spansthe full document instead of just the visible viewport. The responsesets `image.coordinateSpace` to `"document"` and bounding boxes are@@ -332,7 +337,10 @@ Capture query parameters:The response carries `image.coordinateSpace`, `image.devicePixelRatio`,and `image.viewport` ( `w`, `h`, `scrollX`, `scrollY` in CSS pixels atcapture time) so clients can translate between image pixels and-`boundingBox` values without guessing.+`boundingBox` values without guessing. Two axes have to be pinned for that+to hold: the coordinate ORIGIN, which `image.coordinateSpace` names+(`viewport` or `document`), and the box-model EDGE, which is always the+border box.The response carries an `epoch.domEpoch` token cached on the tab's ref-cache.Future client work can pass `expectedEpoch` to action endpoints to detectdiff --git a/docs/reference/capture.md b/docs/reference/capture.mdindex 216aa2b6..eb3eb378 100644--- a/docs/reference/capture.md+++ b/docs/reference/capture.md@@ -87,8 +87,15 @@ an `expectedEpoch` query param to reject stale refs at use time.When `withBounds=true` (the default), each snapshot node with a non-zerobackend node id gets a `boundingBox` and a `visible` flag, and a node that-cannot be measured gets neither. The coordinate space depends on `selector`-and `beyondViewport`:+cannot be measured gets neither.++`boundingBox` is the **border box** — the painted edge of the element, the+same rectangle `pinchtab box`, `screenshot?annotate=true` and+`getBoundingClientRect` report. Overlays and crop rectangles drawn from it+cover the control a viewer sees, and a box from `/capture` can be compared+directly with one from any of those surfaces.++The coordinate space depends on `selector` and `beyondViewport`:- **`viewport`** (default): boxes are viewport-relative CSS pixels. Theimage is the visible viewport. `image.devicePixelRatio` tells you the@@ -125,7 +132,7 @@ box query failed. Treat a missing `visible` as unknown and a present| `depth` | Snapshot tree depth limit || `output` | `file` (default), `inline` (base64 in JSON), or `raw` (bytes only — drops the snapshot) || `wait` | `stable` (default) waits for `Page.lifecycleEvent` quiescence (250ms silence / 750ms ceiling); `load` polls `document.readyState` until `complete` (2s ceiling); `none` skips the wait |-| `withBounds` | `true` (default) — populate `boundingBox` + `visible` on every measurable snapshot node; `false` omits both keys everywhere |+| `withBounds` | `true` (default) — populate `boundingBox` (the border box) + `visible` on every measurable snapshot node; `false` omits both keys everywhere || `beyondViewport` | `true` — capture the full scrollable document; coordinate space becomes `document` || `scale` | Rescale the output bitmap. Default `1`. `0.5` halves each axis (quarter the pixels) || `requirePair` | `true` returns 409 if `pairing.navigated` would be true |diff --git a/internal/bridge/observe/bounds.go b/internal/bridge/observe/bounds.goindex 6736d972..af89de4b 100644--- a/internal/bridge/observe/bounds.go+++ b/internal/bridge/observe/bounds.go@@ -78,7 +78,7 @@ func AnnotateBounds(ctx context.Context, nodes []A11yNode, pageCoords bool, vp Vif nodes[i].NodeID == 0 {continue}- box, ok := getBoxAABB(ctx, nodes[i].NodeID)+ box, ok := ElementBorderBox(ctx, nodes[i].NodeID)if !ok {continue}@@ -93,28 +93,21 @@ func AnnotateBounds(ctx context.Context, nodes []A11yNode, pageCoords bool, vp Vreturn nil}-func getBoxAABB(ctx context.Context, backendNodeID int64) (BoundingBox, bool) {- return boxModelAABB(ctx, backendNodeID, boxQuadContent)-}-// ElementBorderBox is the border-box rectangle of a node in viewport-relative// CSS coordinates, the space getBoxModel reports and the space /box and// ScrollIntoViewAndGetBox hand to their callers unchanged. It is the// cross-frame-correct alternative to evaluating getBoundingClientRect in the// element's own context, which is relative to the document the element lives in// and therefore frame-relative for anything inside an iframe.+//+// Every bounds consumer goes through here. AnnotateBounds used to take+// getBoxModel's CONTENT quad, which meant /capture reported a rectangle inset by+// each element's border and padding while /box and the annotate path reported the+// border box — the same field name, the same origin, a different box-model edge.+// Overlays drawn from it sat inside the painted border. The content quad has no+// consumer through this helper: the click sites read box.Content from their own+// CDP calls in internal/bridge/input_human.go and cdpops/geometry.go.func ElementBorderBox(ctx context.Context, backendNodeID int64) (BoundingBox, bool) {- return boxModelAABB(ctx, backendNodeID, boxQuadBorder)-}--type boxQuad string--const (- boxQuadContent boxQuad = "content"- boxQuadBorder boxQuad = "border"-)--func boxModelAABB(ctx context.Context, backendNodeID int64, quad boxQuad) (BoundingBox, bool) {var result json.RawMessageerr := chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {return chromedp.FromContext(ctx).Target.Execute(ctx, "DOM.getBoxModel", map[string]any{@@ -126,17 +119,13 @@ func boxModelAABB(ctx context.Context, backendNodeID int64, quad boxQuad) (Bound}var box struct {Model struct {- Content []float64 `json:"content"`- Border []float64 `json:"border"`+ Border []float64 `json:"border"`} `json:"model"`}if err := json.Unmarshal(result, &box); err != nil {return BoundingBox{}, false}- q := box.Model.Content- if quad == boxQuadBorder {- q = box.Model.Border- }+ q := box.Model.Borderif len(q) < 8 {return BoundingBox{}, false}diff --git a/internal/bridge/observe/bounds_test.go b/internal/bridge/observe/bounds_test.goindex b3724bc4..7d7e58a2 100644--- a/internal/bridge/observe/bounds_test.go+++ b/internal/bridge/observe/bounds_test.go@@ -140,10 +140,12 @@ func (f scrollFixture) annotate(t *testing.T, pageCoords bool) (BoundingBox, booreturn *nodes[0].BoundingBox, *nodes[0].Visible, vp}-// getBoxModel reports the content box and getBoundingClientRect the border box,-// so the two differ by the button's border and padding — single digits, nothing-// like the scroll offset a wrong transform introduces.-const boxModelSlack = 12.0+// AnnotateBounds and getBoundingClientRect now report the same box-model edge —+// the border box — so the only slack left is float rounding. The 12px this used+// to allow was absorbing the content-box inset (the button's border plus+// padding), which is the defect these bounds no longer have; anything larger+// than a rounding error is a real disagreement.+const boxModelSlack = 1.0func assertNear(t *testing.T, label string, got, want float64) {t.Helper()diff --git a/internal/bridge/observe/box_quad_test.go b/internal/bridge/observe/box_quad_test.gonew file mode 100644index 00000000..3aa9aa16--- /dev/null+++ b/internal/bridge/observe/box_quad_test.go@@ -0,0 +1,181 @@+package observe++import (+ "context"+ "encoding/json"+ "math"+ "testing"++ "github.com/chromedp/chromedp"+)++// Two buttons that differ ONLY in border and padding. The plain one is the+// control: with no border and no padding every box-model quad coincides, which+// is why this defect survived — every existing fixture element was unstyled.+//+// box-sizing:border-box makes the declared 120x50 the BORDER box, so the content+// box is inset by border+padding on each side and is a different rectangle in+// both origin and size. A border-only fixture would not discriminate: it still+// passes against a padding-box implementation, which is why the padding is here.+const boxQuadFixtureHTML = `<body style="margin:0">+<button id="target" style="position:absolute;left:60px;top:50px;width:120px;height:50px;border:0;padding:0;margin:0;box-sizing:border-box">target</button>+<button id="thick" style="position:absolute;left:60px;top:150px;width:120px;height:50px;border:10px solid #000;padding:8px;margin:0;box-sizing:border-box">thick</button>+</body>`++// contentBoxAABB is the content quad, read through the test's OWN+// DOM.getBoxModel call. It is deliberately not a production helper: the content+// box has no consumer through this package, and reintroducing one here as a+// convenience is how the second geometry path gets written.+func contentBoxAABB(t *testing.T, ctx context.Context, backendNodeID int64) BoundingBox {+ t.Helper()++ var result json.RawMessage+ if err := chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {+ return chromedp.FromContext(ctx).Target.Execute(ctx, "DOM.getBoxModel", map[string]any{+ "backendNodeId": backendNodeID,+ }, &result)+ })); err != nil {+ t.Fatal(err)+ }+ var box struct {+ Model struct {+ Content []float64 `json:"content"`+ } `json:"model"`+ }+ if err := json.Unmarshal(result, &box); err != nil {+ t.Fatal(err)+ }+ q := box.Model.Content+ if len(q) < 8 {+ t.Fatalf("content quad has %d values, want 8", len(q))+ }+ return BoundingBox{X: q[0], Y: q[1], W: q[2] - q[0], H: q[5] - q[1]}+}++// buttonNodeID resolves one of the fixture's buttons by accessible name.+func buttonNodeID(t *testing.T, ctx context.Context, name string) int64 {+ t.Helper()++ rawNodes, err := FetchAXTree(ctx)+ if err != nil {+ t.Fatal(err)+ }+ nodes, _ := BuildSnapshot(rawNodes, "", -1)+ for _, n := range nodes {+ if n.Role == "button" && n.Name == name && n.NodeID != 0 {+ return n.NodeID+ }+ }+ t.Fatalf("no button named %q in the snapshot (%d nodes)", name, len(nodes))+ return 0+}++// clientRectOf is the browser's own answer, and the oracle the annotate path+// uses: internal/cdptk/annotate.go builds its overlay rects from+// getBoundingClientRect, so agreeing with it here is agreeing with+// screenshot?annotate=true.+func clientRectOf(t *testing.T, ctx context.Context, id string) BoundingBox {+ t.Helper()++ var rect struct {+ X float64 `json:"x"`+ Y float64 `json:"y"`+ W float64 `json:"width"`+ H float64 `json:"height"`+ }+ script := `(() => { const r = document.getElementById('` + id + `').getBoundingClientRect(); return {x: r.x, y: r.y, width: r.width, height: r.height}; })()`+ if err := chromedp.Run(ctx, chromedp.Evaluate(script, &rect)); err != nil {+ t.Fatal(err)+ }+ return BoundingBox{X: rect.X, Y: rect.Y, W: rect.W, H: rect.H}+}++func annotatedBox(t *testing.T, ctx context.Context, nodeID int64) BoundingBox {+ t.Helper()++ vp, err := FetchLayout(ctx)+ if err != nil {+ t.Fatal(err)+ }+ nodes := []A11yNode{{NodeID: nodeID}}+ if err := AnnotateBounds(ctx, nodes, false, vp); err != nil {+ t.Fatal(err)+ }+ if nodes[0].BoundingBox == nil {+ t.Fatal("AnnotateBounds produced no bounding box")+ }+ return *nodes[0].BoundingBox+}++func assertSameBox(t *testing.T, label string, got, want BoundingBox) {+ t.Helper()++ const tolerance = 0.5+ off := func(a, b float64) bool { return math.Abs(a-b) > tolerance }+ if off(got.X, want.X) || off(got.Y, want.Y) || off(got.W, want.W) || off(got.H, want.H) {+ t.Errorf("%s = {x %.0f y %.0f w %.0f h %.0f}, want {x %.0f y %.0f w %.0f h %.0f}",+ label, got.X, got.Y, got.W, got.H, want.X, want.Y, want.W, want.H)+ }+}++// The card's defect: three surfaces describing the same element, two agreeing+// and /capture reporting a rectangle inset by the border. capture bounds come+// from AnnotateBounds, /box from ElementBorderBox, and annotate rects from+// getBoundingClientRect — so all three conventions are compared here, on a+// bordered+padded element and on an unstyled control.+func TestCaptureBoundsAgreeWithBoxAndClientRect(t *testing.T) {+ fixture := newScrollFixtureFrom(t, boxQuadFixtureHTML, 0, 0)++ for _, tc := range []struct {+ name string+ id string+ }{+ {"unstyled control", "target"},+ {"bordered and padded", "thick"},+ } {+ t.Run(tc.name, func(t *testing.T) {+ nodeID := buttonNodeID(t, fixture.ctx, tc.id)+ clientRect := clientRectOf(t, fixture.ctx, tc.id)++ assertSameBox(t, "capture boundingBox (AnnotateBounds)", annotatedBox(t, fixture.ctx, nodeID), clientRect)++ borderBox, ok := ElementBorderBox(fixture.ctx, nodeID)+ if !ok {+ t.Fatal("ElementBorderBox reported no box")+ }+ assertSameBox(t, "/box (ElementBorderBox)", borderBox, clientRect)+ })+ }+}++// Without this the test above could pass against a content-box implementation+// that happens to agree — it would mean the fixture cannot see the defect. The+// plain button is the other half: its quads coincide, so a change here must move+// the bordered element and leave the unstyled one exactly where it was.+func TestBoxQuadFixtureDistinguishesContentFromBorder(t *testing.T) {+ fixture := newScrollFixtureFrom(t, boxQuadFixtureHTML, 0, 0)++ thick := buttonNodeID(t, fixture.ctx, "thick")+ border, ok := ElementBorderBox(fixture.ctx, thick)+ if !ok {+ t.Fatal("ElementBorderBox reported no box for the bordered element")+ }+ content := contentBoxAABB(t, fixture.ctx, thick)++ if content.X == border.X && content.Y == border.Y {+ t.Fatalf("content and border boxes share an origin (%v); the fixture cannot tell the two quads apart", border)+ }+ if content.W == border.W || content.H == border.H {+ t.Fatalf("content %v and border %v share a dimension; a padding-box implementation would pass unnoticed", content, border)+ }+ if content.W >= border.W || content.H >= border.H {+ t.Fatalf("content %v is not inside border %v", content, border)+ }++ plain := buttonNodeID(t, fixture.ctx, "target")+ plainBorder, ok := ElementBorderBox(fixture.ctx, plain)+ if !ok {+ t.Fatal("ElementBorderBox reported no box for the unstyled element")+ }+ assertSameBox(t, "unstyled control content quad", contentBoxAABB(t, fixture.ctx, plain), plainBorder)+}diff --git a/tests/manual/capture-drift-scenarios.md b/tests/manual/capture-drift-scenarios.mdindex ddae4ac8..adc87fba 100644--- a/tests/manual/capture-drift-scenarios.md+++ b/tests/manual/capture-drift-scenarios.md@@ -23,6 +23,13 @@ Expect:- response prints `epoch: ep_...`, `pairing: navigated=false`.- the snapshot's button "Known Position Button" has a `boundingBox` near`{x: 100, y: 200, w: 240, h: 48}` (within 1 CSS pixel × `devicePixelRatio`).+- the button "Bordered Padded Button" has a `boundingBox` near+ `{x: 100, y: 400, w: 240, h: 48}` — the BORDER box. Its content box is+ `{x: 118, y: 418, w: 204, h: 12}`, so reading `{118, 418, 204, 12}` here means+ the bounds path has gone back to the content quad and every overlay drawn from+ it will sit inside the painted border.+- cross-check the same ref against `pinchtab box` and+ `GET /screenshot?annotate=true`: all three report the same rectangle.- `image.coordinateSpace == "viewport"`.## 2. wait:stable for deferred mountdiff --git a/tests/tools/fixtures/capture-known-bounds.html b/tests/tools/fixtures/capture-known-bounds.htmlindex 57eabbcc..62bcae6c 100644--- a/tests/tools/fixtures/capture-known-bounds.html+++ b/tests/tools/fixtures/capture-known-bounds.html@@ -17,6 +17,27 @@border: none;font-size: 16px;}+ /* #known sets no border, but a button still carries the UA stylesheet's+ padding, so even here the content box {x: 106, y: 201, w: 228, h: 46} is+ not the border box {x: 100, y: 200, w: 240, h: 48} — a 6px inset that is+ easy to read as rounding. #bordered makes the difference unmissable:+ with box-sizing:border-box (the UA default for form controls) its+ declared 240x48 IS the border box, and its content box is inset by+ border+padding to {x: 118, y: 418, w: 204, h: 12}. Both sets of numbers+ are measured, not derived. */+ #bordered {+ position: absolute;+ left: 100px;+ top: 400px;+ width: 240px;+ height: 48px;+ box-sizing: border-box;+ border: 10px solid #1e3a8a;+ padding: 8px;+ background: #2563eb;+ color: white;+ font-size: 16px;+ }#below-fold {position: absolute;left: 50px;@@ -35,7 +56,18 @@ref's boundingBox should report<code>{x: 100, y: 200, w: 240, h: 48}</code> within 1 CSS pixel.</p>+ <p>+ boundingBox is the <strong>border box</strong> — the same edge+ <code>/box</code>, <code>screenshot?annotate=true</code> and+ <code>getBoundingClientRect</code> report. The button #bordered is the+ element that makes a wrong edge unmissable: it is at (100, 400) with a+ border-box size of (240, 48), so its boundingBox is+ <code>{x: 100, y: 400, w: 240, h: 48}</code>, while its content box is+ <code>{x: 118, y: 418, w: 204, h: 12}</code> — a different origin AND a+ different size, so a padding-box reading cannot pass either.+ </p><button id="known">Known Position Button</button>+ <button id="bordered">Bordered Padded Button</button><button id="below-fold">Below Fold</button></main></body>