Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

fix: align capture bounds and defaults

Luigi Agosti974034e36a24
diff --git a/SPEC_DEVICE_SCALE_FACTOR.md b/SPEC_DEVICE_SCALE_FACTOR.md
deleted file mode 100644
index f56fe7c3..00000000
--- a/SPEC_DEVICE_SCALE_FACTOR.md
+++ /dev/null
@@ -1,152 +0,0 @@
-# Spec — tab-level `deviceScaleFactor` emulation
-
-Status: draft. Not implemented. Tracking for a separate PR after
-`feat/add-capture-endpoint` lands.
-
-## Problem
-
-PinchTab today runs against the host's real Chrome + real display, so
-`window.devicePixelRatio` is whatever the host gives you (1 on most Linux
-servers, 2 on retina Macs, 1.5 on common Windows hardware, …). Screenshots
-come back at that native DPR. Two consequences for agent workflows:
-
-1. **Non-portable image dimensions.** The same `pinchtab capture` against
- the same fixture returns different pixel sizes on different hosts —
- the LLM has to figure out scale from `image.devicePixelRatio` every
- time. Most agent frameworks pin DPR so this just stops mattering.
-2. **Bounding-box ↔ image-pixel translation lives in the client.** We
- already return `image.devicePixelRatio` so a vision agent can rescale
- if it wants, but the math is busywork the daemon could do upstream.
-
-The standard fix used by browser-use, Playwright, and Puppeteer is to set
-`deviceScaleFactor` once at browser-context creation. Every subsequent
-screenshot inherits it. No per-call knob.
-
-## Proposed contract
-
-A new tab-level setting `deviceScaleFactor: float` controls the DPR used
-for all rendering and capture within that tab. Default = native (no
-emulation).
-
-When set to N > 0:
-
-- `Emulation.setDeviceMetricsOverride({width, height, deviceScaleFactor: N, mobile: false})`
- is called at tab attach, before the first navigation, against the tab
- context.
-- `window.devicePixelRatio` returns N inside the page.
-- `/screenshot`, `/capture`, screencast frames are all rasterized at N CSS
- pixels → N image pixels per axis.
-- Bounding boxes from `DOM.getBoxModel` stay in CSS pixels (DPR doesn't
- change CSS coordinates).
-- Click, type, navigation are unchanged (CSS-pixel based).
-
-`Emulation.clearDeviceMetricsOverride` reverts.
-
-## What it does NOT change
-
-- Bounding boxes (`A11yNode.BoundingBox`): always CSS pixels.
-- Click / type / scroll coordinates: always CSS pixels.
-- Accessibility tree: unaffected.
-- The `image.devicePixelRatio` field in `/capture` response: stays
- accurate — reflects the emulated DPR when override is active, native
- DPR otherwise.
-
-## API surface
-
-### Tab creation
-
-```bash
-pinchtab tab new --device-scale-factor 1
-```
-
-```http
-POST /tab
-{
- "action": "new",
- "url": "...",
- "deviceScaleFactor": 1
-}
-```
-
-### Runtime control on an existing tab
-
-```bash
-pinchtab tab emulate --tab-id <x> --device-scale-factor 2
-pinchtab tab emulate --tab-id <x> --clear
-```
-
-```http
-POST /tabs/{id}/emulate
-{ "deviceScaleFactor": 2, "width": 1440, "height": 900 }
-
-DELETE /tabs/{id}/emulate
-```
-
-`width` and `height` are required by CDP for the override call. If
-omitted, the implementation reads the current viewport and reuses it.
-
-### Profile / instance default
-
-Set in profile config so all tabs spun up for an instance inherit:
-
-```yaml
-deviceScaleFactor: 1
-```
-
-Tabs adopted from a real user profile keep the user's native DPR unless
-the instance config overrides.
-
-## Stealth posture
-
-`window.devicePixelRatio` is fingerprintable. Emulating DPR=1 on retina
-hardware is a divergence sites can detect. The stealth implications:
-
-- Default = no emulation (preserve real-profile property).
-- Agent-instance profiles set explicitly when predictability matters
- more than stealth.
-- Tabs marked for human handoff should never have an override active —
- the human sees a re-rasterized page.
-
-If the override is set mid-session it triggers a reflow, fires
-`IntersectionObserver` callbacks, and may resolve `loading="lazy"`
-images. Same hazard class as `--beyond-viewport`. Recommended pattern:
-set at tab init only; runtime control reserved for explicit operator
-action.
-
-## Out of scope (separate work)
-
-- Per-call DPR override on `/screenshot` / `/capture`. Use `?scale=` for
- bitmap rescale (already shipped); use tab-level emulation when you want
- the page to *render* at a different DPR.
-- Touch / mobile emulation. `Emulation.setDeviceMetricsOverride` accepts
- a `mobile` flag and viewport-orientation params — those belong in a
- separate "mobile emulation" spec.
-- Network throttling, locale, timezone. Same emulation domain but
- different concerns.
-
-## Implementation footprint
-
-- `bridge.TabPolicyState` or profile-config struct: add
- `DeviceScaleFactor float64` (zero = no override).
-- `bridge/tab_manager.go` tab-attach path: when set, call
- `Emulation.setDeviceMetricsOverride` before first navigation.
-- New handler `POST /tabs/{id}/emulate` and `DELETE
- /tabs/{id}/emulate`.
-- CLI: `pinchtab tab new --device-scale-factor`, `pinchtab tab emulate`.
-- MCP tool: extend `pinchtab_list_tabs` response with the active
- emulation, optional `pinchtab_emulate` tool for setting it.
-
-Estimated ~150 LOC plus tests. Independent of any other in-flight work.
-
-## Open questions
-
-- Should we surface DPR as a column in `pinchtab tabs` output? Probably
- yes — agents need to know what they're capturing into.
-- Default for `pinchtab tab new` when invoked without flags: native DPR
- or 1? Native DPR preserves backwards compat. 1 makes agent workflows
- predictable. Lean native; agent operators set 1 explicitly.
-- Interaction with `--beyond-viewport`: should we re-call
- `setDeviceMetricsOverride` to grow the layout viewport, or let
- `captureBeyondViewport` handle it as today? Today's mechanism is
- already a layout expansion — they're orthogonal but combining them
- needs a test.
diff --git a/cmd/pinchtab/cmd_cli_register.go b/cmd/pinchtab/cmd_cli_register.go
index 67dde2c2..5c199a5f 100644
--- a/cmd/pinchtab/cmd_cli_register.go
+++ b/cmd/pinchtab/cmd_cli_register.go
@@ -248,7 +248,7 @@ func configureBrowserFlags() {
captureCmd.Flags().StringP("output", "o", "", "Save the captured image to this local file path (default: capture-<ts>.jpg)")
captureCmd.Flags().StringP("selector", "s", "", "Scope: clips screenshot and filters snapshot subtree to the same element")
- captureCmd.Flags().String("filter", "", "Snapshot filter: 'interactive' or 'all' (default: all)")
+ captureCmd.Flags().String("filter", "", "Snapshot filter: 'interactive' or 'all' (default: interactive)")
captureCmd.Flags().String("format", "", "Image format: 'jpeg' (default) or 'png'")
captureCmd.Flags().StringP("quality", "q", "", "JPEG quality (0-100)")
captureCmd.Flags().String("depth", "", "Snapshot max depth (-1 for full)")
diff --git a/docs/reference/capture.md b/docs/reference/capture.md
index 73f5d318..599e4d7e 100644
--- a/docs/reference/capture.md
+++ b/docs/reference/capture.md
@@ -21,7 +21,8 @@ pinchtab capture --require-pair
# Full-document image; bounding boxes in page coords
pinchtab capture --beyond-viewport
-# Scope to one element: image clips to it, snapshot subtree filters to it
+# Scope to one element: image clips to it, snapshot subtree filters to it.
+# Bounding boxes are relative to the clipped image origin.
pinchtab capture -s "#checkout-form"
```
@@ -81,11 +82,14 @@ an `expectedEpoch` query param to reject stale refs at use time.
When `withBounds=true` (the default), each snapshot node with a
non-zero backend node id gets a `boundingBox` and a `visible` flag. The
-coordinate space depends on `beyondViewport`:
+coordinate space depends on `selector` and `beyondViewport`:
- **`viewport`** (default): boxes are viewport-relative CSS pixels. The
image is the visible viewport. `image.devicePixelRatio` tells you the
ratio of image pixels to CSS pixels.
+- **`clip`** (when `selector` is set): boxes are relative to the cropped
+ image origin. The response also includes `image.clip` with the original
+ document-relative clip rectangle.
- **`document`** (when `beyondViewport=true`): boxes use page coordinates
(`box.x` and `box.y` include scroll offset). The image is the full
document.
@@ -106,7 +110,7 @@ viewport — a cheap heuristic, not a strict occlusion check.
| `quality` | JPEG quality 0-100 |
| `depth` | Snapshot tree depth limit |
| `output` | `file` (default), `inline` (base64 in JSON), or `raw` (bytes only — drops the snapshot) |
-| `wait` | `stable` (default) waits for `Page.lifecycleEvent` quiescence (250ms silence / 750ms ceiling); `load` and `none` skip the wait |
+| `wait` | `stable` (default) waits for `Page.lifecycleEvent` quiescence (250ms silence / 750ms ceiling); `load` polls `document.readyState` until `complete` (2s ceiling); `none` skips the wait |
| `withBounds` | `true` (default) — populate `boundingBox` + `visible` on every snapshot node |
| `beyondViewport` | `true` — capture the full scrollable document; coordinate space becomes `document` |
| `scale` | Rescale the output bitmap. Default `1`. `0.5` halves each axis (quarter the pixels) |
diff --git a/docs/reference/mcp-tools.md b/docs/reference/mcp-tools.md
index a57fb4d5..0a6b9915 100644
--- a/docs/reference/mcp-tools.md
+++ b/docs/reference/mcp-tools.md
@@ -27,7 +27,7 @@ Structured semantic locators are matched by the semantic engine; CSS, XPath, ref
| `pinchtab_snapshot` | `tabId`, `interactive`, `compact`, `format`, `diff`, `selector`, `maxTokens`, `depth`, `noAnimations` | `selector` scopes the snapshot; `format` is limited to `compact` or `text` |
| `pinchtab_frame` | `tabId`, `target` | Get or set the frame scope for selector-based actions on the tab; `target` accepts `main`, a snapshot ref, an iframe selector, or a frame name/URL |
| `pinchtab_screenshot` | `tabId`, `selector`, `scale`, `format`, `quality`, `annotate`, `beyondViewport` | `selector` captures a specific element in current frame scope; `scale` rescales the output bitmap (e.g. `0.5` = half size); `format` is `jpeg` or `png`; `annotate=true` overlays numbered ref boxes and populates the annotations envelope; `beyondViewport=true` captures the full scrollable document (ignored when `selector` is set) — box coords are document-relative in that mode |
-| `pinchtab_capture` | `tabId`, `selector`, `filter`, `format`, `quality`, `depth`, `wait`, `withBounds`, `beyondViewport`, `requirePair`, `noAnimations` | Paired screenshot + accessibility snapshot from the same DOM epoch. Returns an image content block plus a JSON envelope with `epoch`, `pairing.navigated`, per-node `boundingBox`, and `image.coordinateSpace`. Use when the model reads pixels AND acts on refs in the same turn. |
+| `pinchtab_capture` | `tabId`, `selector`, `filter`, `format`, `quality`, `depth`, `scale`, `wait`, `withBounds`, `beyondViewport`, `requirePair`, `noAnimations` | Paired screenshot + accessibility snapshot from the same DOM epoch. Returns an image content block plus a JSON envelope with `epoch`, `pairing.navigated`, per-node `boundingBox`, and `image.coordinateSpace` (`viewport`, `document`, or selector `clip`). Use when the model reads pixels AND acts on refs in the same turn. |
| `pinchtab_get_text` | `tabId`, `raw`, `format`, `maxChars` | `raw=true` maps to `/text?mode=raw`; `format=text/plain` returns plain text; inherits the current `pinchtab_frame` scope for that tab |
## Interaction
diff --git a/internal/bridge/capture.go b/internal/bridge/capture.go
index cd614a73..02440d35 100644
--- a/internal/bridge/capture.go
+++ b/internal/bridge/capture.go
@@ -22,10 +22,9 @@ type CaptureOpts struct {
DisableAnimations bool
// Wait controls the lifecycle wait before the capture window opens.
- // Empty (or "none") skips the wait. "stable" waits for
- // Page.lifecycleEvent quiescence — 250ms of silence or 750ms ceiling.
- // "load" is currently a no-op alias for "none"; reserved for a future
- // document.readyState gate.
+ // Empty (or "none") skips the wait. "load" polls document.readyState
+ // until it reaches "complete" (2s ceiling). "stable" waits for
+ // Page.lifecycleEvent quiescence — 250ms of silence, 750ms ceiling.
Wait string
// WithBounds populates BoundingBox + Visible on every snapshot node that
@@ -59,10 +58,12 @@ type PairedResult struct {
ImageFormat string // "jpeg" or "png"
// Viewport metadata captured alongside the image. CoordinateSpace is
- // "viewport" by default and "document" when ImageOpts.BeyondViewport is
- // true — bounding boxes are expressed in the named space.
+ // "viewport" by default, "document" when ImageOpts.BeyondViewport is true,
+ // and "clip" for selector-clipped captures. Bounding boxes are expressed in
+ // the named space.
Viewport ViewportInfo
CoordinateSpace string
+ Clip *page.Viewport
Filter string
Nodes []A11yNode
@@ -74,10 +75,10 @@ type PairedResult struct {
// navigation between the two CDP calls" — checked by comparing the main
// frame's loaderId before and after the capture window. opts.Wait == "stable"
// adds a Page.lifecycleEvent quiet-window wait before the window opens.
-// opts.WithBounds populates a viewport- or document-relative BoundingBox per
-// snapshot node via DOM.getBoxModel. Residual risk: in-document churn (React
-// re-renders, IntersectionObserver mutations) is not detected — wait:stable
-// reduces but does not eliminate it.
+// opts.WithBounds populates a viewport-, document-, or clip-relative
+// BoundingBox per snapshot node via DOM.getBoxModel. Residual risk:
+// in-document churn (React re-renders, IntersectionObserver mutations) is not
+// detected — wait:stable reduces but does not eliminate it.
func PairedCapture(ctx context.Context, opts CaptureOpts) (*PairedResult, error) {
start := time.Now()
res := &PairedResult{
@@ -92,11 +93,12 @@ func PairedCapture(ctx context.Context, opts CaptureOpts) (*PairedResult, error)
}
}
- if opts.Wait == WaitStable {
- // Errors here are non-fatal — a failed wait should still produce a
- // capture, just without the quiet-window guarantee. The duration is
- // captured for diagnostics but not currently surfaced in PairedResult.
+ // Wait errors are non-fatal — degrade rather than fail the capture.
+ switch opts.Wait {
+ case WaitStable:
_, _ = WaitForQuietWindow(ctx, 250*time.Millisecond, 750*time.Millisecond)
+ case WaitLoad:
+ _, _ = WaitForReadyState(ctx, 2*time.Second)
}
// Pre-capture frame info — root frame id + loader id.
@@ -147,7 +149,12 @@ func PairedCapture(ctx context.Context, opts CaptureOpts) (*PairedResult, error)
)
pageCoords := opts.Image.BeyondViewport
- if pageCoords {
+ if opts.Image.Clip != nil {
+ res.CoordinateSpace = "clip"
+ clip := *opts.Image.Clip
+ res.Clip = &clip
+ pageCoords = true
+ } else if pageCoords {
res.CoordinateSpace = "document"
} else {
res.CoordinateSpace = "viewport"
@@ -155,6 +162,9 @@ func PairedCapture(ctx context.Context, opts CaptureOpts) (*PairedResult, error)
if opts.WithBounds {
_ = AnnotateBounds(ctx, res.Nodes, pageCoords, res.Viewport)
+ if opts.Image.Clip != nil {
+ projectBoundsToClip(res.Nodes, *opts.Image.Clip)
+ }
}
// Post-capture frame info. Compare root frame id + loader id to detect
@@ -175,6 +185,16 @@ func imageFormatString(f page.CaptureScreenshotFormat) string {
return string(f)
}
+func projectBoundsToClip(nodes []A11yNode, clip page.Viewport) {
+ for i := range nodes {
+ if nodes[i].BoundingBox == nil {
+ continue
+ }
+ nodes[i].BoundingBox.X -= clip.X
+ nodes[i].BoundingBox.Y -= clip.Y
+ }
+}
+
// filterAXNodesByFrame mirrors handlers.scopeSnapshotNodesByFrame: drop any
// AX node whose FrameID does not match the active frame scope. Lives here so
// PairedCapture can honor /frame state without the handler having to
diff --git a/internal/bridge/observe/bounds.go b/internal/bridge/observe/bounds.go
index bb174a5f..15136744 100644
--- a/internal/bridge/observe/bounds.go
+++ b/internal/bridge/observe/bounds.go
@@ -65,11 +65,10 @@ func FetchLayout(ctx context.Context) (ViewportInfo, error) {
// non-zero. Each node costs one DOM.getBoxModel round trip; for the typical
// FilterInteractive snapshot of <50 nodes the total budget is ~250ms.
//
-// pageCoords=true translates viewport-relative boxes into document
-// (page) coordinates by adding the current scroll offset — the form needed
-// by beyondViewport captures whose image spans the whole document. With
-// pageCoords=false the boxes are left in viewport coords, matching the
-// default viewport-only screenshot.
+// DOM.getBoxModel returns document-relative CSS coordinates. pageCoords=true
+// leaves boxes in that space for beyondViewport/clip captures. pageCoords=false
+// projects boxes into viewport coordinates by subtracting the current scroll
+// offset, matching the default viewport-only screenshot.
//
// Visibility heuristic: a node is Visible if its rect has non-zero area and
// intersects the viewport. The check is intentionally cheap — strict
@@ -83,12 +82,13 @@ func AnnotateBounds(ctx context.Context, nodes []A11yNode, pageCoords bool, vp V
if !ok {
continue
}
- if pageCoords {
- box.X += vp.ScrollX
- box.Y += vp.ScrollY
+ visible := isVisible(box, true, vp)
+ if !pageCoords {
+ box.X -= vp.ScrollX
+ box.Y -= vp.ScrollY
}
nodes[i].BoundingBox = &box
- nodes[i].Visible = isVisible(box, pageCoords, vp)
+ nodes[i].Visible = visible
}
return nil
}
diff --git a/internal/bridge/observe/bounds_test.go b/internal/bridge/observe/bounds_test.go
new file mode 100644
index 00000000..3b02c1ed
--- /dev/null
+++ b/internal/bridge/observe/bounds_test.go
@@ -0,0 +1,14 @@
+package observe
+
+import "testing"
+
+func TestIsVisibleWithDocumentCoordinates(t *testing.T) {
+ vp := ViewportInfo{Width: 800, Height: 600, ScrollX: 0, ScrollY: 1000}
+
+ if !isVisible(BoundingBox{X: 20, Y: 1050, W: 100, H: 40}, true, vp) {
+ t.Fatal("box inside scrolled viewport should be visible")
+ }
+ if isVisible(BoundingBox{X: 20, Y: 50, W: 100, H: 40}, true, vp) {
+ t.Fatal("box above scrolled viewport should not be visible")
+ }
+}
diff --git a/internal/bridge/observe/lifecycle.go b/internal/bridge/observe/lifecycle.go
index 81648408..30afcb8e 100644
--- a/internal/bridge/observe/lifecycle.go
+++ b/internal/bridge/observe/lifecycle.go
@@ -10,6 +10,34 @@ import (
"github.com/chromedp/chromedp"
)
+// WaitForReadyState polls document.readyState until "complete" or ceiling.
+// Lighter than WaitForQuietWindow — use when "load event fired" is enough.
+func WaitForReadyState(ctx context.Context, ceiling time.Duration) (string, error) {
+ if ceiling <= 0 {
+ ceiling = 2 * time.Second
+ }
+ deadline := time.Now().Add(ceiling)
+ const poll = 50 * time.Millisecond
+
+ var state string
+ for {
+ if err := chromedp.Run(ctx, chromedp.Evaluate(`document.readyState`, &state)); err != nil {
+ return state, err
+ }
+ if state == "complete" {
+ return state, nil
+ }
+ if !time.Now().Before(deadline) {
+ return state, nil
+ }
+ select {
+ case <-ctx.Done():
+ return state, ctx.Err()
+ case <-time.After(poll):
+ }
+ }
+}
+
// WaitForQuietWindow blocks until either no Page.lifecycleEvent has been
// observed for `quiet` duration, or `ceiling` has elapsed since the call
// started. Returns the duration actually waited.
diff --git a/internal/bridge/observe/snapshot.go b/internal/bridge/observe/snapshot.go
index a407b386..5ba5750d 100644
--- a/internal/bridge/observe/snapshot.go
+++ b/internal/bridge/observe/snapshot.go
@@ -41,8 +41,9 @@ type A11yNode struct {
// BoundingBox is a CSS-pixel rectangle for a snapshot node. Populated only by
// PairedCapture when CaptureOpts.WithBounds is true. Coordinate space depends
-// on PairedResult.CoordinateSpace — viewport-relative by default, document
-// (page) coordinates when BeyondViewport is on.
+// on PairedResult.CoordinateSpace: viewport-relative by default, document
+// coordinates when BeyondViewport is on, and clip-relative when selector
+// scoping crops the image.
type BoundingBox struct {
X float64 `json:"x"`
Y float64 `json:"y"`
diff --git a/internal/bridge/observe_facade.go b/internal/bridge/observe_facade.go
index 469936b7..118a0e05 100644
--- a/internal/bridge/observe_facade.go
+++ b/internal/bridge/observe_facade.go
@@ -51,6 +51,10 @@ func WaitForQuietWindow(ctx context.Context, quiet, ceiling time.Duration) (time
return bridgeobserve.WaitForQuietWindow(ctx, quiet, ceiling)
}
+func WaitForReadyState(ctx context.Context, ceiling time.Duration) (string, error) {
+ return bridgeobserve.WaitForReadyState(ctx, ceiling)
+}
+
type BoundingBox = bridgeobserve.BoundingBox
type ViewportInfo = bridgeobserve.ViewportInfo
diff --git a/internal/bridge/screenshot.go b/internal/bridge/screenshot.go
index e45c702b..8572f364 100644
--- a/internal/bridge/screenshot.go
+++ b/internal/bridge/screenshot.go
@@ -8,6 +8,27 @@ import (
"github.com/chromedp/chromedp"
)
+// MinScale / MaxScale bound the bitmap rescale factor. Anything outside is
+// almost certainly a mistake or a DoS — 50× would ask the renderer for a
+// multi-gigapixel image.
+const (
+ MinScale = 0.05
+ MaxScale = 4.0
+)
+
+func ClampScale(scale float64) float64 {
+ if scale <= 0 {
+ return 1
+ }
+ if scale < MinScale {
+ return MinScale
+ }
+ if scale > MaxScale {
+ return MaxScale
+ }
+ return scale
+}
+
// fetchViewportSize returns window.innerWidth/innerHeight via one
// Runtime.evaluate round trip. Falls back to (0, 0) on any failure so the
// caller can decide whether to bail or skip the rescale.
@@ -75,8 +96,10 @@ func CaptureScreenshot(ctx context.Context, opts ScreenshotOpts) ([]byte, error)
clip = &c
} else {
// Synthesize a viewport-covering clip so CDP's clip.scale applies.
- // Prefer caller-supplied dims; otherwise fetch innerWidth/Height
- // via a single Runtime.evaluate.
+ //
+ // Known issue: two back-to-back /capture?scale=<n!=1> on the same
+ // tab without nav between can hang on the second call. Workaround:
+ // nav between captures (see e2e cli/capture-basic.sh).
w, h := opts.ViewportWidth, opts.ViewportHeight
if w == 0 || h == 0 {
w, h = fetchViewportSize(ctx)
diff --git a/internal/bridge/screenshot_test.go b/internal/bridge/screenshot_test.go
new file mode 100644
index 00000000..7be92132
--- /dev/null
+++ b/internal/bridge/screenshot_test.go
@@ -0,0 +1,51 @@
+package bridge
+
+import (
+ "testing"
+
+ "github.com/chromedp/cdproto/page"
+)
+
+func TestClampScale(t *testing.T) {
+ cases := []struct {
+ in, want float64
+ }{
+ {0, 1},
+ {-1, 1},
+ {0.01, 0.05},
+ {0.05, 0.05},
+ {0.5, 0.5},
+ {1, 1},
+ {2, 2},
+ {4, 4},
+ {4.5, 4},
+ {1000, 4},
+ }
+ for _, c := range cases {
+ got := ClampScale(c.in)
+ if got != c.want {
+ t.Errorf("ClampScale(%v) = %v, want %v", c.in, got, c.want)
+ }
+ }
+}
+
+func TestProjectBoundsToClip(t *testing.T) {
+ nodes := []A11yNode{
+ {
+ Ref: "e1",
+ BoundingBox: &BoundingBox{X: 125, Y: 240, W: 30, H: 20},
+ },
+ {Ref: "e2"},
+ }
+ projectBoundsToClip(nodes, page.Viewport{X: 100, Y: 200, Width: 300, Height: 200})
+
+ if nodes[0].BoundingBox.X != 25 || nodes[0].BoundingBox.Y != 40 {
+ t.Fatalf("projected box = (%v,%v), want (25,40)", nodes[0].BoundingBox.X, nodes[0].BoundingBox.Y)
+ }
+ if nodes[0].BoundingBox.W != 30 || nodes[0].BoundingBox.H != 20 {
+ t.Fatalf("projected size = %vx%v, want 30x20", nodes[0].BoundingBox.W, nodes[0].BoundingBox.H)
+ }
+ if nodes[1].BoundingBox != nil {
+ t.Fatal("node without bounding box should remain unchanged")
+ }
+}
diff --git a/internal/cli/actions/actions_capture.go b/internal/cli/actions/actions_capture.go
index 0398fdef..f70f390d 100644
--- a/internal/cli/actions/actions_capture.go
+++ b/internal/cli/actions/actions_capture.go
@@ -24,6 +24,7 @@ func Capture(client *http.Client, base, token string, cmd *cobra.Command) {
outFile, _ := cmd.Flags().GetString("output")
jsonOutput, _ := cmd.Flags().GetBool("json")
+ outFileExplicit := cmd.Flags().Changed("output")
format, _ := cmd.Flags().GetString("format")
if format == "" {
@@ -77,30 +78,31 @@ func Capture(client *http.Client, base, token string, cmd *cobra.Command) {
return
}
- // Always materialize the image locally — both terse and --json callers
- // need the file. --json mode still prints the base64 envelope so
- // pipelines that prefer inline bytes can keep using it.
+ // --json without an explicit --output skips the file write so piping JSON
+ // doesn't leave a stray image in pwd. Base64 stays in the envelope.
ext := ".jpg"
if resp.Image.Format == "png" {
ext = ".png"
}
- if outFile == "" {
- outFile = fmt.Sprintf("capture-%s%s", time.Now().Format("20060102-150405"), ext)
- }
- img, err := base64.StdEncoding.DecodeString(resp.Image.Base64)
- if err != nil {
- output.Error("capture", fmt.Sprintf("decode image bytes: %v", err), output.ExitRuntime)
- return
- }
- if err := os.WriteFile(outFile, img, 0600); err != nil {
- output.Error("capture", fmt.Sprintf("write image: %v", err), output.ExitRuntime)
- return
+ writeImage := !jsonOutput || outFileExplicit
+ var img []byte
+ if writeImage {
+ if outFile == "" {
+ outFile = fmt.Sprintf("capture-%s%s", time.Now().Format("20060102-150405"), ext)
+ }
+ decoded, err := base64.StdEncoding.DecodeString(resp.Image.Base64)
+ if err != nil {
+ output.Error("capture", fmt.Sprintf("decode image bytes: %v", err), output.ExitRuntime)
+ return
+ }
+ img = decoded
+ if err := os.WriteFile(outFile, img, 0600); err != nil {
+ output.Error("capture", fmt.Sprintf("write image: %v", err), output.ExitRuntime)
+ return
+ }
}
if jsonOutput {
- // Echo what the server sent, pretty-printed. The base64 stays in
- // place for callers who want inline bytes; outFile is where we
- // wrote the decoded copy.
var pretty map[string]any
if err := json.Unmarshal(raw, &pretty); err == nil {
output.JSON(pretty)
@@ -124,7 +126,7 @@ func Capture(client *http.Client, base, token string, cmd *cobra.Command) {
fmt.Println()
filter := resp.Snapshot.Filter
if filter == "" {
- filter = "all"
+ filter = "interactive"
}
output.Value(fmt.Sprintf("snapshot: %d nodes (%s)", resp.Snapshot.NodeCount, filter))
for _, n := range resp.Snapshot.Nodes {
diff --git a/internal/handlers/capture.go b/internal/handlers/capture.go
index 01652290..45be4d61 100644
--- a/internal/handlers/capture.go
+++ b/internal/handlers/capture.go
@@ -28,7 +28,7 @@ import (
//
// @Param tabId string query Tab ID (optional, defaults to current)
// @Param selector string query Optional scope (clips screenshot and filters snapshot subtree)
-// @Param filter string query Snapshot filter: "interactive" or "all" (default "all")
+// @Param filter string query Snapshot filter: "interactive" or "all" (default "interactive")
// @Param depth int query Snapshot max depth (default -1 for full)
// @Param format string query Image format: "jpeg" or "png" (default "jpeg")
// @Param quality int query JPEG quality 1-100 (default 80)
@@ -67,6 +67,9 @@ func (h *Handlers) HandleCapture(w http.ResponseWriter, r *http.Request) {
selector := q.Get("selector")
filter := q.Get("filter")
+ if filter == "" {
+ filter = bridge.FilterInteractive
+ }
output := q.Get("output")
if output == "" {
output = "file"
@@ -75,12 +78,11 @@ func (h *Handlers) HandleCapture(w http.ResponseWriter, r *http.Request) {
reqNoAnim := q.Get("noAnimations") == "true"
beyondViewport := q.Get("beyondViewport") == "true" || q.Get("beyondViewport") == "1"
- // scale rescales the rendered bitmap. Default 1 (native). 0.5 halves
- // each axis (quarter of the pixels).
+ // Clamp so ?scale=10000 can't ask CDP for a gigapixel image.
scale := 1.0
if s := q.Get("scale"); s != "" {
- if sf, err := strconv.ParseFloat(s, 64); err == nil && sf > 0 {
- scale = sf
+ if sf, err := strconv.ParseFloat(s, 64); err == nil {
+ scale = bridge.ClampScale(sf)
}
}
@@ -93,9 +95,9 @@ func (h *Handlers) HandleCapture(w http.ResponseWriter, r *http.Request) {
withBounds = false
}
- // Default to wait=stable: vision-grounded callers are the primary user and
- // they want a quiesced page. Callers that want raw timing pass wait=none
- // (or the alias wait=load — reserved for a future readyState gate).
+ // Default to wait=stable: vision-grounded callers are the primary user
+ // and they want a quiesced page. wait=load polls document.readyState;
+ // wait=none skips the wait entirely.
wait := q.Get("wait")
if wait == "" {
wait = bridge.WaitStable
@@ -212,6 +214,14 @@ func (h *Handlers) HandleCapture(w http.ResponseWriter, r *http.Request) {
"scrollY": result.Viewport.ScrollY,
},
}
+ if result.Clip != nil {
+ imageInfo["clip"] = map[string]any{
+ "x": result.Clip.X,
+ "y": result.Clip.Y,
+ "w": result.Clip.Width,
+ "h": result.Clip.Height,
+ }
+ }
switch output {
case "file":
diff --git a/internal/handlers/capture_test.go b/internal/handlers/capture_test.go
index d73e5052..28cb787e 100644
--- a/internal/handlers/capture_test.go
+++ b/internal/handlers/capture_test.go
@@ -20,17 +20,12 @@ func TestHandleCapture_NoTab(t *testing.T) {
}
func TestHandleCapture_UnknownOutput(t *testing.T) {
- // Bad output value must 400 before we ever try to capture — so failTab is
- // irrelevant; the check happens after tab resolution but before the CDP
- // round trip.
h := New(&mockBridge{failTab: true}, &config.RuntimeConfig{}, nil, nil, nil)
req := httptest.NewRequest("GET", "/capture?output=carrier-pigeon", nil)
w := httptest.NewRecorder()
h.HandleCapture(w, req)
- // Tab resolution fails first, so we get 404. This test exists to confirm
- // the unknown-output branch does not panic during query parsing.
if w.Code != http.StatusNotFound {
- t.Errorf("expected 404 (tab not found), got %d", w.Code)
+ t.Errorf("expected 404, got %d", w.Code)
}
}
@@ -56,9 +51,6 @@ func TestHandleTabCapture_NoTab(t *testing.T) {
}
func TestHandleCapture_RouteMounted(t *testing.T) {
- // The route table is the source of truth for the API surface; make sure
- // /capture is registered alongside /screenshot and /snapshot so that
- // future route refactors notice.
h := New(&mockBridge{failTab: true}, &config.RuntimeConfig{}, nil, nil, nil)
mux := http.NewServeMux()
h.RegisterRoutes(mux, nil)
@@ -67,12 +59,10 @@ func TestHandleCapture_RouteMounted(t *testing.T) {
req := httptest.NewRequest("GET", path, nil)
w := httptest.NewRecorder()
mux.ServeHTTP(w, req)
- // 404 is fine (tab not found); 404 with body "tab not found" proves
- // the route hit the handler. A bare 404 with empty body would mean
- // the mux did not match the path.
if w.Code != http.StatusNotFound {
t.Errorf("%s: expected 404 from handler, got %d body=%s", path, w.Code, w.Body.String())
}
+ // Empty body would mean the mux didn't match the path.
if !strings.Contains(w.Body.String(), "tab") {
t.Errorf("%s: expected handler-shaped 404 body, got %q", path, w.Body.String())
}
@@ -80,9 +70,6 @@ func TestHandleCapture_RouteMounted(t *testing.T) {
}
func TestHandleCapture_WaitParamParses(t *testing.T) {
- // The wait param accepts known string values; the parse must not crash
- // even when the tab lookup fails. Mirrors
- // TestHandleScreenshot_BeyondViewportRequestParses.
for _, v := range []string{"stable", "load", "none", ""} {
h := New(&mockBridge{failTab: true}, &config.RuntimeConfig{}, nil, nil, nil)
req := httptest.NewRequest("GET", "/capture?wait="+v, nil)
@@ -95,9 +82,6 @@ func TestHandleCapture_WaitParamParses(t *testing.T) {
}
func TestHandleCapture_BoundsAndBeyondViewportParse(t *testing.T) {
- // withBounds, beyondViewport, and scale must parse without crashing —
- // the handler should at minimum validate them before hitting the tab
- // lookup.
cases := []string{
"/capture?withBounds=true&beyondViewport=true",
"/capture?withBounds=false",
diff --git a/internal/handlers/screenshot.go b/internal/handlers/screenshot.go
index 8ae7f126..8fac6593 100644
--- a/internal/handlers/screenshot.go
+++ b/internal/handlers/screenshot.go
@@ -140,8 +140,8 @@ func (h *Handlers) HandleScreenshot(w http.ResponseWriter, r *http.Request) {
scale := 1.0
if s := r.URL.Query().Get("scale"); s != "" {
- if sf, err := strconv.ParseFloat(s, 64); err == nil && sf > 0 {
- scale = sf
+ if sf, err := strconv.ParseFloat(s, 64); err == nil {
+ scale = bridge.ClampScale(sf)
}
}
buf, err = bridge.CaptureScreenshot(tCtx, bridge.ScreenshotOpts{
diff --git a/internal/mcp/handlers_navigation.go b/internal/mcp/handlers_navigation.go
index 41eb01dc..3fa3473a 100644
--- a/internal/mcp/handlers_navigation.go
+++ b/internal/mcp/handlers_navigation.go
@@ -237,6 +237,9 @@ func handleCapture(c *Client) func(context.Context, mcp.CallToolRequest) (*mcp.C
if quality, ok := optFloat(r, "quality"); ok {
q.Set("quality", fmt.Sprintf("%d", int(quality)))
}
+ if scale, ok := optFloat(r, "scale"); ok && scale > 0 {
+ q.Set("scale", fmt.Sprintf("%g", scale))
+ }
if v := optNumber(r, "depth"); v > 0 {
q.Set("depth", formatInt(v))
}
@@ -283,6 +286,7 @@ func captureResult(body []byte) (*mcp.CallToolResult, error) {
CoordinateSpace string `json:"coordinateSpace"`
DPR float64 `json:"devicePixelRatio"`
Viewport json.RawMessage `json:"viewport"`
+ Clip json.RawMessage `json:"clip,omitempty"`
} `json:"image"`
Snapshot json.RawMessage `json:"snapshot"`
}
@@ -299,6 +303,16 @@ func captureResult(body []byte) (*mcp.CallToolResult, error) {
mimeType = "image/png"
}
+ imagePayload := map[string]any{
+ "format": format,
+ "bytes": env.Image.Bytes,
+ "coordinateSpace": env.Image.CoordinateSpace,
+ "devicePixelRatio": env.Image.DPR,
+ "viewport": env.Image.Viewport,
+ }
+ if len(env.Image.Clip) > 0 {
+ imagePayload["clip"] = env.Image.Clip
+ }
textPayload := map[string]any{
"status": env.Status,
"tabId": env.TabID,
@@ -307,14 +321,8 @@ func captureResult(body []byte) (*mcp.CallToolResult, error) {
"capturedAt": env.CapturedAt,
"epoch": env.Epoch,
"pairing": env.Pairing,
- "image": map[string]any{
- "format": format,
- "bytes": env.Image.Bytes,
- "coordinateSpace": env.Image.CoordinateSpace,
- "devicePixelRatio": env.Image.DPR,
- "viewport": env.Image.Viewport,
- },
- "snapshot": env.Snapshot,
+ "image": imagePayload,
+ "snapshot": env.Snapshot,
}
encoded, err := json.Marshal(textPayload)
if err != nil {
diff --git a/internal/mcp/tools.go b/internal/mcp/tools.go
index 16639841..23bf6e71 100644
--- a/internal/mcp/tools.go
+++ b/internal/mcp/tools.go
@@ -40,14 +40,15 @@ func allTools() []mcp.Tool {
mcp.WithBoolean("beyondViewport", mcp.Description("Capture the entire scrollable document, not just the visible viewport. Box coordinates in the annotations envelope are document-relative when this is true. Ignored when selector is set.")),
),
mcp.NewTool("pinchtab_capture",
- mcp.WithDescription("Paired screenshot + accessibility snapshot from the same DOM epoch. Returns an image content block plus a JSON envelope `{url, title, epoch:{frameId,loaderId,domEpoch}, pairing:{navigated,captureDurationMs}, image:{format,bytes,coordinateSpace,devicePixelRatio,viewport}, snapshot:{filter,nodeCount,nodes}}`. Each node in `snapshot.nodes` may carry a `boundingBox:{x,y,w,h}` and `visible:bool` so a vision-capable model can overlay refs on pixels. Coordinate space is 'viewport' by default and 'document' when beyondViewport=true. Prefer this over a separate snapshot+screenshot pair whenever the model needs both image and refs."),
+ mcp.WithDescription("Paired screenshot + accessibility snapshot from the same DOM epoch. Returns an image content block plus a JSON envelope `{url, title, epoch:{frameId,loaderId,domEpoch}, pairing:{navigated,captureDurationMs}, image:{format,bytes,coordinateSpace,devicePixelRatio,viewport,clip}, snapshot:{filter,nodeCount,nodes}}`. Each node in `snapshot.nodes` may carry a `boundingBox:{x,y,w,h}` and `visible:bool` so a vision-capable model can overlay refs on pixels. Coordinate space is 'viewport' by default, 'document' when beyondViewport=true, and 'clip' when selector is set. Prefer this over a separate snapshot+screenshot pair whenever the model needs both image and refs."),
mcp.WithString("tabId", mcp.Description("Target tab ID")),
mcp.WithString("selector", mcp.Description("Scope: clips screenshot and filters snapshot subtree to the same element. Wins over beyondViewport.")),
mcp.WithString("filter", mcp.Description("Snapshot filter: 'interactive' (default) or 'all'")),
mcp.WithString("format", mcp.Description("Image format: 'jpeg' (default) or 'png'")),
mcp.WithNumber("quality", mcp.Description("JPEG quality 0-100 (JPEG only)")),
mcp.WithNumber("depth", mcp.Description("Snapshot tree depth limit (e.g. 3)")),
- mcp.WithString("wait", mcp.Description("Lifecycle wait before capture: 'stable' (default — waits for Page.lifecycleEvent quiescence), 'load', or 'none'")),
+ mcp.WithNumber("scale", mcp.Description("Rescale the output image bitmap. Default 1 (native). 0.5 halves each axis.")),
+ mcp.WithString("wait", mcp.Description("Lifecycle wait before capture: 'stable' (default — waits for Page.lifecycleEvent quiescence), 'load' (polls document.readyState until complete), or 'none'")),
mcp.WithBoolean("withBounds", mcp.Description("Populate boundingBox + visible on snapshot nodes (default true). Set false to skip per-node DOM.getBoxModel round trips when bounds are not needed.")),
mcp.WithBoolean("beyondViewport", mcp.Description("Capture the full scrollable document instead of just the visible viewport. Coordinate space becomes 'document'; bounding boxes use page coords.")),
mcp.WithBoolean("requirePair", mcp.Description("Return 409 if the main frame navigates during the capture window (loaderId changes).")),
diff --git a/internal/routes/routes_test.go b/internal/routes/routes_test.go
index 3c6fe1f3..862025c7 100644
--- a/internal/routes/routes_test.go
+++ b/internal/routes/routes_test.go
@@ -5,6 +5,32 @@ import (
"testing"
)
+func TestCaptureEndpointIsShorthandAndTabScoped(t *testing.T) {
+ wantShorthand := "GET /capture"
+ found := false
+ for _, route := range ShorthandRoutes() {
+ if route == wantShorthand {
+ found = true
+ break
+ }
+ }
+ if !found {
+ t.Fatalf("ShorthandRoutes() missing %s — proxy mux will 404 until the routes catalog entry is added next to the handlers.RegisterRoutes call", wantShorthand)
+ }
+
+ wantTab := "GET /tabs/{id}/capture"
+ found = false
+ for _, route := range TabScopedRoutes() {
+ if route == wantTab {
+ found = true
+ break
+ }
+ }
+ if !found {
+ t.Fatalf("TabScopedRoutes() missing %s — the catalog entry's TabScoped flag was likely flipped to false", wantTab)
+ }
+}
+
func TestFrameEndpointsAreShorthandAndTabScoped(t *testing.T) {
found := map[string]bool{
"GET /frame": false,
diff --git a/tests/e2e/scenarios/api/capture-basic.sh b/tests/e2e/scenarios/api/capture-basic.sh
index 56ff0941..e79bdfc4 100755
--- a/tests/e2e/scenarios/api/capture-basic.sh
+++ b/tests/e2e/scenarios/api/capture-basic.sh
@@ -1,10 +1,5 @@
#!/bin/bash
# capture-basic.sh — /capture endpoint happy paths.
-#
-# /capture returns a screenshot AND an accessibility snapshot from the
-# same DOM epoch in one HTTP call. These scenarios cover the contract
-# /screenshot + /snapshot together cannot satisfy: paired epoch, frame
-# parity, scale, bounding boxes, and the requirePair drift gate.
GROUP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${GROUP_DIR}/../../helpers/api.sh"
@@ -19,12 +14,12 @@ assert_ok "capture"
assert_json_exists "$RESULT" '.status' "response.status"
assert_json_exists "$RESULT" '.epoch.domEpoch' "epoch.domEpoch present"
-# pairing.navigated may legitimately be false on a fresh capture, so use
-# `has` rather than truthy-check.
+# jq -e treats `false` as missing — use has() since navigated may be false.
assert_json_exists "$RESULT" '.pairing | has("navigated")' "pairing.navigated present"
assert_json_exists "$RESULT" '.image.path' "image written to disk"
assert_json_exists "$RESULT" '.snapshot.nodes' "snapshot.nodes present"
assert_json_exists "$RESULT" '.image.coordinateSpace' "image.coordinateSpace present"
+assert_result_eq '.snapshot.filter' 'interactive' 'default filter=interactive'
end_test
@@ -75,9 +70,6 @@ start_test "capture: scale reduces image bytes"
pt_post /navigate -d "{\"url\":\"${FIXTURES_URL}/table.html\"}"
-# inline output puts base64 in the response — byte size of the decoded
-# image is what we want to compare. A scale of 0.25 should produce
-# materially fewer bytes than scale=1.
FULL_BYTES=$(e2e_curl -s "${E2E_SERVER}/capture?output=inline" | jq -r '.image.bytes')
HALF_BYTES=$(e2e_curl -s "${E2E_SERVER}/capture?output=inline&scale=0.25" | jq -r '.image.bytes')
@@ -97,7 +89,6 @@ pt_post /navigate -d "{\"url\":\"${FIXTURES_URL}/form.html\"}"
pt_get "/capture?withBounds=true&filter=interactive"
assert_ok "capture withBounds"
-# At least one interactive node should carry a boundingBox.
HAS_BOUNDS=$(echo "$RESULT" | jq '[.snapshot.nodes[] | select(.boundingBox != null)] | length')
if [ "$HAS_BOUNDS" -gt 0 ]; then
pass_assert "$HAS_BOUNDS nodes with boundingBox"
@@ -122,6 +113,28 @@ fi
end_test
+# ─────────────────────────────────────────────────────────────────
+start_test "capture: selector bounds are clip-relative"
+
+pt_post /navigate -d "{\"url\":\"${FIXTURES_URL}/form.html\"}"
+
+SELECTOR_ENC='%23test-form'
+pt_get "/capture?selector=${SELECTOR_ENC}&withBounds=true&filter=interactive"
+assert_ok "capture selector"
+
+assert_result_eq '.image.coordinateSpace' 'clip' 'coordinateSpace=clip under selector'
+assert_result_exists '.image.clip' 'image.clip present'
+assert_result_jq \
+ '[.snapshot.nodes[] | select(.boundingBox != null)] | length > 0' \
+ 'selector returned bounded nodes' \
+ 'selector capture returned no bounded nodes'
+assert_result_jq \
+ '[.snapshot.nodes[] | select(.boundingBox != null and .boundingBox.y < 200)] | length > 0' \
+ 'at least one selector bound near clip origin' \
+ 'selector bounds look unprojected'
+
+end_test
+
# ─────────────────────────────────────────────────────────────────
start_test "capture: epoch token is unique across calls"
@@ -151,8 +164,6 @@ end_test
# ─────────────────────────────────────────────────────────────────
start_test "/screenshot ?scale= reduces image bytes"
-# Sibling test that exercises the new --scale path on /screenshot too,
-# since the underlying bridge.CaptureScreenshot helper is shared.
pt_post /navigate -d "{\"url\":\"${FIXTURES_URL}/table.html\"}"
FULL=$(e2e_curl -s "${E2E_SERVER}/screenshot" | wc -c)
HALF=$(e2e_curl -s "${E2E_SERVER}/screenshot?scale=0.25" | wc -c)