Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

fix: capture boundingBox returns the content box while /box and annotate return the border box, so overlays under-cover every bordered element (PIN-119)

Luigi Agosti65d95c154e98
diff --git a/docs/endpoints.md b/docs/endpoints.md
index f4e5f22b..ed75d434 100644
--- a/docs/endpoints.md
+++ b/docs/endpoints.md
@@ -316,9 +316,14 @@ Capture query parameters:
`none`; reserved for a future `document.readyState` gate.
- `withBounds=true|false` — default `true`. When on, every snapshot node
with a non-zero backend node id gets a `boundingBox` field and a
- `visible` flag. Each bounded node costs one `DOM.getBoxModel` round trip
- (~5ms); for the typical interactive-filter snapshot the budget is under
- 250ms. Pass `withBounds=false` to skip the per-node work.
+ `visible` flag. `boundingBox` is the element's **border box** — the
+ painted edge, the same rectangle `GET /box`, `screenshot?annotate=true`
+ and `getBoundingClientRect` report, so a box can be cross-checked against
+ any of them. It is not the content box: an element with a border or
+ padding would otherwise report a rectangle inset from the edge a viewer
+ identifies the control by. Each bounded node costs one `DOM.getBoxModel`
+ round trip (~5ms); for the typical interactive-filter snapshot the budget
+ is under 250ms. Pass `withBounds=false` to skip the per-node work.
- `beyondViewport=true|false` — default `false`. When on, the image spans
the full document instead of just the visible viewport. The response
sets `image.coordinateSpace` to `"document"` and bounding boxes are
@@ -332,7 +337,10 @@ Capture query parameters:
The response carries `image.coordinateSpace`, `image.devicePixelRatio`,
and `image.viewport` ( `w`, `h`, `scrollX`, `scrollY` in CSS pixels at
capture time) so clients can translate between image pixels and
-`boundingBox` values without guessing.
+`boundingBox` values without guessing. Two axes have to be pinned for that
+to hold: the coordinate ORIGIN, which `image.coordinateSpace` names
+(`viewport` or `document`), and the box-model EDGE, which is always the
+border box.
The response carries an `epoch.domEpoch` token cached on the tab's ref-cache.
Future client work can pass `expectedEpoch` to action endpoints to detect
diff --git a/docs/reference/capture.md b/docs/reference/capture.md
index 216aa2b6..eb3eb378 100644
--- a/docs/reference/capture.md
+++ b/docs/reference/capture.md
@@ -87,8 +87,15 @@ an `expectedEpoch` query param to reject stale refs at use time.
When `withBounds=true` (the default), each snapshot node with a non-zero
backend node id gets a `boundingBox` and a `visible` flag, and a node that
-cannot be measured gets neither. The coordinate space depends on `selector`
-and `beyondViewport`:
+cannot be measured gets neither.
+
+`boundingBox` is the **border box** — the painted edge of the element, the
+same rectangle `pinchtab box`, `screenshot?annotate=true` and
+`getBoundingClientRect` report. Overlays and crop rectangles drawn from it
+cover the control a viewer sees, and a box from `/capture` can be compared
+directly with one from any of those surfaces.
+
+The coordinate space depends on `selector` and `beyondViewport`:
- **`viewport`** (default): boxes are viewport-relative CSS pixels. The
image is the visible viewport. `image.devicePixelRatio` tells you the
@@ -125,7 +132,7 @@ box query failed. Treat a missing `visible` as unknown and a present
| `depth` | Snapshot tree depth limit |
| `output` | `file` (default), `inline` (base64 in JSON), or `raw` (bytes only — drops the snapshot) |
| `wait` | `stable` (default) waits for `Page.lifecycleEvent` quiescence (250ms silence / 750ms ceiling); `load` polls `document.readyState` until `complete` (2s ceiling); `none` skips the wait |
-| `withBounds` | `true` (default) — populate `boundingBox` + `visible` on every measurable snapshot node; `false` omits both keys everywhere |
+| `withBounds` | `true` (default) — populate `boundingBox` (the border box) + `visible` on every measurable snapshot node; `false` omits both keys everywhere |
| `beyondViewport` | `true` — capture the full scrollable document; coordinate space becomes `document` |
| `scale` | Rescale the output bitmap. Default `1`. `0.5` halves each axis (quarter the pixels) |
| `requirePair` | `true` returns 409 if `pairing.navigated` would be true |
diff --git a/internal/bridge/observe/bounds.go b/internal/bridge/observe/bounds.go
index 6736d972..af89de4b 100644
--- a/internal/bridge/observe/bounds.go
+++ b/internal/bridge/observe/bounds.go
@@ -78,7 +78,7 @@ func AnnotateBounds(ctx context.Context, nodes []A11yNode, pageCoords bool, vp V
if nodes[i].NodeID == 0 {
continue
}
- box, ok := getBoxAABB(ctx, nodes[i].NodeID)
+ box, ok := ElementBorderBox(ctx, nodes[i].NodeID)
if !ok {
continue
}
@@ -93,28 +93,21 @@ func AnnotateBounds(ctx context.Context, nodes []A11yNode, pageCoords bool, vp V
return nil
}
-func getBoxAABB(ctx context.Context, backendNodeID int64) (BoundingBox, bool) {
- return boxModelAABB(ctx, backendNodeID, boxQuadContent)
-}
-
// ElementBorderBox is the border-box rectangle of a node in viewport-relative
// CSS coordinates, the space getBoxModel reports and the space /box and
// ScrollIntoViewAndGetBox hand to their callers unchanged. It is the
// cross-frame-correct alternative to evaluating getBoundingClientRect in the
// element's own context, which is relative to the document the element lives in
// and therefore frame-relative for anything inside an iframe.
+//
+// Every bounds consumer goes through here. AnnotateBounds used to take
+// getBoxModel's CONTENT quad, which meant /capture reported a rectangle inset by
+// each element's border and padding while /box and the annotate path reported the
+// border box — the same field name, the same origin, a different box-model edge.
+// Overlays drawn from it sat inside the painted border. The content quad has no
+// consumer through this helper: the click sites read box.Content from their own
+// CDP calls in internal/bridge/input_human.go and cdpops/geometry.go.
func ElementBorderBox(ctx context.Context, backendNodeID int64) (BoundingBox, bool) {
- return boxModelAABB(ctx, backendNodeID, boxQuadBorder)
-}
-
-type boxQuad string
-
-const (
- boxQuadContent boxQuad = "content"
- boxQuadBorder boxQuad = "border"
-)
-
-func boxModelAABB(ctx context.Context, backendNodeID int64, quad boxQuad) (BoundingBox, bool) {
var result json.RawMessage
err := chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {
return chromedp.FromContext(ctx).Target.Execute(ctx, "DOM.getBoxModel", map[string]any{
@@ -126,17 +119,13 @@ func boxModelAABB(ctx context.Context, backendNodeID int64, quad boxQuad) (Bound
}
var box struct {
Model struct {
- Content []float64 `json:"content"`
- Border []float64 `json:"border"`
+ Border []float64 `json:"border"`
} `json:"model"`
}
if err := json.Unmarshal(result, &box); err != nil {
return BoundingBox{}, false
}
- q := box.Model.Content
- if quad == boxQuadBorder {
- q = box.Model.Border
- }
+ q := box.Model.Border
if len(q) < 8 {
return BoundingBox{}, false
}
diff --git a/internal/bridge/observe/bounds_test.go b/internal/bridge/observe/bounds_test.go
index b3724bc4..7d7e58a2 100644
--- a/internal/bridge/observe/bounds_test.go
+++ b/internal/bridge/observe/bounds_test.go
@@ -140,10 +140,12 @@ func (f scrollFixture) annotate(t *testing.T, pageCoords bool) (BoundingBox, boo
return *nodes[0].BoundingBox, *nodes[0].Visible, vp
}
-// getBoxModel reports the content box and getBoundingClientRect the border box,
-// so the two differ by the button's border and padding — single digits, nothing
-// like the scroll offset a wrong transform introduces.
-const boxModelSlack = 12.0
+// AnnotateBounds and getBoundingClientRect now report the same box-model edge —
+// the border box — so the only slack left is float rounding. The 12px this used
+// to allow was absorbing the content-box inset (the button's border plus
+// padding), which is the defect these bounds no longer have; anything larger
+// than a rounding error is a real disagreement.
+const boxModelSlack = 1.0
func assertNear(t *testing.T, label string, got, want float64) {
t.Helper()
diff --git a/internal/bridge/observe/box_quad_test.go b/internal/bridge/observe/box_quad_test.go
new file mode 100644
index 00000000..3aa9aa16
--- /dev/null
+++ b/internal/bridge/observe/box_quad_test.go
@@ -0,0 +1,181 @@
+package observe
+
+import (
+ "context"
+ "encoding/json"
+ "math"
+ "testing"
+
+ "github.com/chromedp/chromedp"
+)
+
+// Two buttons that differ ONLY in border and padding. The plain one is the
+// control: with no border and no padding every box-model quad coincides, which
+// is why this defect survived — every existing fixture element was unstyled.
+//
+// box-sizing:border-box makes the declared 120x50 the BORDER box, so the content
+// box is inset by border+padding on each side and is a different rectangle in
+// both origin and size. A border-only fixture would not discriminate: it still
+// passes against a padding-box implementation, which is why the padding is here.
+const boxQuadFixtureHTML = `<body style="margin:0">
+<button id="target" style="position:absolute;left:60px;top:50px;width:120px;height:50px;border:0;padding:0;margin:0;box-sizing:border-box">target</button>
+<button id="thick" style="position:absolute;left:60px;top:150px;width:120px;height:50px;border:10px solid #000;padding:8px;margin:0;box-sizing:border-box">thick</button>
+</body>`
+
+// contentBoxAABB is the content quad, read through the test's OWN
+// DOM.getBoxModel call. It is deliberately not a production helper: the content
+// box has no consumer through this package, and reintroducing one here as a
+// convenience is how the second geometry path gets written.
+func contentBoxAABB(t *testing.T, ctx context.Context, backendNodeID int64) BoundingBox {
+ t.Helper()
+
+ var result json.RawMessage
+ if err := chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {
+ return chromedp.FromContext(ctx).Target.Execute(ctx, "DOM.getBoxModel", map[string]any{
+ "backendNodeId": backendNodeID,
+ }, &result)
+ })); err != nil {
+ t.Fatal(err)
+ }
+ var box struct {
+ Model struct {
+ Content []float64 `json:"content"`
+ } `json:"model"`
+ }
+ if err := json.Unmarshal(result, &box); err != nil {
+ t.Fatal(err)
+ }
+ q := box.Model.Content
+ if len(q) < 8 {
+ t.Fatalf("content quad has %d values, want 8", len(q))
+ }
+ return BoundingBox{X: q[0], Y: q[1], W: q[2] - q[0], H: q[5] - q[1]}
+}
+
+// buttonNodeID resolves one of the fixture's buttons by accessible name.
+func buttonNodeID(t *testing.T, ctx context.Context, name string) int64 {
+ t.Helper()
+
+ rawNodes, err := FetchAXTree(ctx)
+ if err != nil {
+ t.Fatal(err)
+ }
+ nodes, _ := BuildSnapshot(rawNodes, "", -1)
+ for _, n := range nodes {
+ if n.Role == "button" && n.Name == name && n.NodeID != 0 {
+ return n.NodeID
+ }
+ }
+ t.Fatalf("no button named %q in the snapshot (%d nodes)", name, len(nodes))
+ return 0
+}
+
+// clientRectOf is the browser's own answer, and the oracle the annotate path
+// uses: internal/cdptk/annotate.go builds its overlay rects from
+// getBoundingClientRect, so agreeing with it here is agreeing with
+// screenshot?annotate=true.
+func clientRectOf(t *testing.T, ctx context.Context, id string) BoundingBox {
+ t.Helper()
+
+ var rect struct {
+ X float64 `json:"x"`
+ Y float64 `json:"y"`
+ W float64 `json:"width"`
+ H float64 `json:"height"`
+ }
+ script := `(() => { const r = document.getElementById('` + id + `').getBoundingClientRect(); return {x: r.x, y: r.y, width: r.width, height: r.height}; })()`
+ if err := chromedp.Run(ctx, chromedp.Evaluate(script, &rect)); err != nil {
+ t.Fatal(err)
+ }
+ return BoundingBox{X: rect.X, Y: rect.Y, W: rect.W, H: rect.H}
+}
+
+func annotatedBox(t *testing.T, ctx context.Context, nodeID int64) BoundingBox {
+ t.Helper()
+
+ vp, err := FetchLayout(ctx)
+ if err != nil {
+ t.Fatal(err)
+ }
+ nodes := []A11yNode{{NodeID: nodeID}}
+ if err := AnnotateBounds(ctx, nodes, false, vp); err != nil {
+ t.Fatal(err)
+ }
+ if nodes[0].BoundingBox == nil {
+ t.Fatal("AnnotateBounds produced no bounding box")
+ }
+ return *nodes[0].BoundingBox
+}
+
+func assertSameBox(t *testing.T, label string, got, want BoundingBox) {
+ t.Helper()
+
+ const tolerance = 0.5
+ off := func(a, b float64) bool { return math.Abs(a-b) > tolerance }
+ if off(got.X, want.X) || off(got.Y, want.Y) || off(got.W, want.W) || off(got.H, want.H) {
+ t.Errorf("%s = {x %.0f y %.0f w %.0f h %.0f}, want {x %.0f y %.0f w %.0f h %.0f}",
+ label, got.X, got.Y, got.W, got.H, want.X, want.Y, want.W, want.H)
+ }
+}
+
+// The card's defect: three surfaces describing the same element, two agreeing
+// and /capture reporting a rectangle inset by the border. capture bounds come
+// from AnnotateBounds, /box from ElementBorderBox, and annotate rects from
+// getBoundingClientRect — so all three conventions are compared here, on a
+// bordered+padded element and on an unstyled control.
+func TestCaptureBoundsAgreeWithBoxAndClientRect(t *testing.T) {
+ fixture := newScrollFixtureFrom(t, boxQuadFixtureHTML, 0, 0)
+
+ for _, tc := range []struct {
+ name string
+ id string
+ }{
+ {"unstyled control", "target"},
+ {"bordered and padded", "thick"},
+ } {
+ t.Run(tc.name, func(t *testing.T) {
+ nodeID := buttonNodeID(t, fixture.ctx, tc.id)
+ clientRect := clientRectOf(t, fixture.ctx, tc.id)
+
+ assertSameBox(t, "capture boundingBox (AnnotateBounds)", annotatedBox(t, fixture.ctx, nodeID), clientRect)
+
+ borderBox, ok := ElementBorderBox(fixture.ctx, nodeID)
+ if !ok {
+ t.Fatal("ElementBorderBox reported no box")
+ }
+ assertSameBox(t, "/box (ElementBorderBox)", borderBox, clientRect)
+ })
+ }
+}
+
+// Without this the test above could pass against a content-box implementation
+// that happens to agree — it would mean the fixture cannot see the defect. The
+// plain button is the other half: its quads coincide, so a change here must move
+// the bordered element and leave the unstyled one exactly where it was.
+func TestBoxQuadFixtureDistinguishesContentFromBorder(t *testing.T) {
+ fixture := newScrollFixtureFrom(t, boxQuadFixtureHTML, 0, 0)
+
+ thick := buttonNodeID(t, fixture.ctx, "thick")
+ border, ok := ElementBorderBox(fixture.ctx, thick)
+ if !ok {
+ t.Fatal("ElementBorderBox reported no box for the bordered element")
+ }
+ content := contentBoxAABB(t, fixture.ctx, thick)
+
+ if content.X == border.X && content.Y == border.Y {
+ t.Fatalf("content and border boxes share an origin (%v); the fixture cannot tell the two quads apart", border)
+ }
+ if content.W == border.W || content.H == border.H {
+ t.Fatalf("content %v and border %v share a dimension; a padding-box implementation would pass unnoticed", content, border)
+ }
+ if content.W >= border.W || content.H >= border.H {
+ t.Fatalf("content %v is not inside border %v", content, border)
+ }
+
+ plain := buttonNodeID(t, fixture.ctx, "target")
+ plainBorder, ok := ElementBorderBox(fixture.ctx, plain)
+ if !ok {
+ t.Fatal("ElementBorderBox reported no box for the unstyled element")
+ }
+ assertSameBox(t, "unstyled control content quad", contentBoxAABB(t, fixture.ctx, plain), plainBorder)
+}
diff --git a/tests/manual/capture-drift-scenarios.md b/tests/manual/capture-drift-scenarios.md
index ddae4ac8..adc87fba 100644
--- a/tests/manual/capture-drift-scenarios.md
+++ b/tests/manual/capture-drift-scenarios.md
@@ -23,6 +23,13 @@ Expect:
- response prints `epoch: ep_...`, `pairing: navigated=false`.
- the snapshot's button "Known Position Button" has a `boundingBox` near
`{x: 100, y: 200, w: 240, h: 48}` (within 1 CSS pixel × `devicePixelRatio`).
+- the button "Bordered Padded Button" has a `boundingBox` near
+ `{x: 100, y: 400, w: 240, h: 48}` — the BORDER box. Its content box is
+ `{x: 118, y: 418, w: 204, h: 12}`, so reading `{118, 418, 204, 12}` here means
+ the bounds path has gone back to the content quad and every overlay drawn from
+ it will sit inside the painted border.
+- cross-check the same ref against `pinchtab box` and
+ `GET /screenshot?annotate=true`: all three report the same rectangle.
- `image.coordinateSpace == "viewport"`.
## 2. wait:stable for deferred mount
diff --git a/tests/tools/fixtures/capture-known-bounds.html b/tests/tools/fixtures/capture-known-bounds.html
index 57eabbcc..62bcae6c 100644
--- a/tests/tools/fixtures/capture-known-bounds.html
+++ b/tests/tools/fixtures/capture-known-bounds.html
@@ -17,6 +17,27 @@
border: none;
font-size: 16px;
}
+ /* #known sets no border, but a button still carries the UA stylesheet's
+ padding, so even here the content box {x: 106, y: 201, w: 228, h: 46} is
+ not the border box {x: 100, y: 200, w: 240, h: 48} — a 6px inset that is
+ easy to read as rounding. #bordered makes the difference unmissable:
+ with box-sizing:border-box (the UA default for form controls) its
+ declared 240x48 IS the border box, and its content box is inset by
+ border+padding to {x: 118, y: 418, w: 204, h: 12}. Both sets of numbers
+ are measured, not derived. */
+ #bordered {
+ position: absolute;
+ left: 100px;
+ top: 400px;
+ width: 240px;
+ height: 48px;
+ box-sizing: border-box;
+ border: 10px solid #1e3a8a;
+ padding: 8px;
+ background: #2563eb;
+ color: white;
+ font-size: 16px;
+ }
#below-fold {
position: absolute;
left: 50px;
@@ -35,7 +56,18 @@
ref's boundingBox should report
<code>{x: 100, y: 200, w: 240, h: 48}</code> within 1 CSS pixel.
</p>
+ <p>
+ boundingBox is the <strong>border box</strong> — the same edge
+ <code>/box</code>, <code>screenshot?annotate=true</code> and
+ <code>getBoundingClientRect</code> report. The button #bordered is the
+ element that makes a wrong edge unmissable: it is at (100, 400) with a
+ border-box size of (240, 48), so its boundingBox is
+ <code>{x: 100, y: 400, w: 240, h: 48}</code>, while its content box is
+ <code>{x: 118, y: 418, w: 204, h: 12}</code> — a different origin AND a
+ different size, so a padding-box reading cannot pass either.
+ </p>
<button id="known">Known Position Button</button>
+ <button id="bordered">Bordered Padded Button</button>
<button id="below-fold">Below Fold</button>
</main>
</body>