Skip to content

Stealth browser automation for AI agents over MCP and the Chrome DevTools Protocol: navigation, network hooks, DOM extraction, and pixel-accurate UI cloning.

Scale screenshots to fit and add quality and max_width

Vibhek Soni5eb5ccca2013
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 585b85d..5281e29 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -11,6 +11,7 @@ The format is based on Keep a Changelog and adheres to Semantic Versioning where
- **Stealth benchmark** - `python -m benchmarks.stealth` runs the browser against a bundled local probe, an init script capability check, Sannysoft, Intoli, CreepJS, Cloudflare (nowsecure.nl), and the tls.peet.ws TLS and HTTP/2 fingerprint API. Each target reports PASS, DEGRADED, FAIL, or UNREACHABLE with its individual checks, and each run writes a JSON result file. A weekly GitHub Actions workflow runs it on Linux, Windows, and macOS in headless and headed mode plus the previous Chrome release, commits results to `docs/stealth-results/`, regenerates the automated section of `STEALTH_TESTS.md`, and opens a `stealth-regression` issue when a target gets worse (#25).
- **`STEALTH_BROWSER_EXECUTABLE`** - Environment variable that selects the browser executable instead of auto-detection.
+- **Screenshot images** - `take_screenshot` returns MCP image content with metadata instead of base64 text, scales large captures down to fit client limits instead of writing a file on the server, and accepts `quality` (JPEG, 1 to 100) and `max_width` to shrink images. Thanks to @bepcyc (#37).
- **Page bindings** - `create_page_binding`, `get_page_binding_calls`, `resolve_page_binding_call`, and `remove_page_binding` let page JavaScript call `window.<name>(...args)` and get a Promise that the agent answers. Calls are queued as JSON (64 KB per call, 500 queued per instance), `auto_resolve` answers immediately with a fixed value, and bindings follow every tab and navigation. Nothing runs on the server host. This replaces `create_python_binding`.
- **`BROWSER_TAB_RECYCLE_NAVIGATIONS`** - Opt-in main tab recycling. It was always on at 25 navigations before.
- **Launch flags benchmark target** - Fails when the real Chrome command line has automation, sandbox-disabling, or unsupported flags.
diff --git a/README.md b/README.md
index 2165dc0..86b65b2 100644
--- a/README.md
+++ b/README.md
@@ -352,25 +352,7 @@ client = Client(
| `get_page_content()` | Read page HTML and text |
| `take_screenshot()` | Capture screenshots |
-`take_screenshot` returns an MCP image block by default, with the correct PNG/JPEG
-MIME type and structured dimensions, byte count and SHA-256 metadata. Remote MCP
-clients can display the image without reading the server's filesystem. Inline
-images are complete and limited to 8 MiB after compression (16 MiB capture bytes
-and 16 megapixels before encoding); larger captures fail with an explicit export
-instruction instead of silently returning an inaccessible local path. For an
-explicit server-local export, pass `file_path`. This still saves the requested
-file and returns its location; making that exported file accessible to a remote
-client remains the caller's responsibility.
-
-The 8 MiB binary limit becomes about 10.67 MiB of base64, plus JSON metadata,
-on the wire; client-specific message limits may be lower.
-
-This intentionally replaces the previous inline base64 text and automatic local
-file fallback. Metadata is also included as a small text block for clients without
-structured-result support. `complete` describes the returned image bytes, not page
-readiness or full-page coverage. Limits apply after browser capture and before
-Pillow decoding/inline encoding; they are not browser/CDP memory budgets. File
-export still captures the image in the browser and is not a low-memory alternative.
+`take_screenshot` returns an MCP image with metadata (format, size, scale, quality, byte count, SHA-256), so remote clients can show it without reading the server's disk. Large captures, such as long full-page screenshots, are scaled down to fit client limits (8000 px per side, about 5 MB), and the metadata reports the original size. Pass `format="jpeg"` with a lower `quality`, or `max_width`, for smaller images, or `file_path` to save the full-resolution image on the server.
</details>
diff --git a/src/screenshot_encoder.py b/src/screenshot_encoder.py
new file mode 100644
index 0000000..ec5137b
--- /dev/null
+++ b/src/screenshot_encoder.py
@@ -0,0 +1,148 @@
+"""Fit browser screenshots into MCP image limits."""
+
+import hashlib
+import io
+from pathlib import Path
+from typing import Any, Dict, Optional, Tuple
+
+from PIL import Image
+
+INLINE_MAX_BYTES = 3_750_000
+MAX_IMAGE_SIDE = 8000
+MAX_CAPTURE_BYTES = 64 * 1024 * 1024
+MAX_DECODE_PIXELS = 100_000_000
+DEFAULT_QUALITY = 85
+FIT_ATTEMPTS = 6
+
+
+def validate_options(format: str, quality: int, max_width: Optional[int]) -> str:
+ """
+ Normalize the format and check screenshot options before capturing.
+
+ Args:
+ format (str): png, jpeg, or jpg.
+ quality (int): JPEG quality from 1 to 100.
+ max_width (Optional[int]): Largest width to return, or None.
+
+ Returns:
+ str: Normalized format, png or jpeg.
+
+ Raises:
+ ValueError: When an option is out of range.
+ """
+ normalized = (format or "png").lower()
+ if normalized == "jpg":
+ normalized = "jpeg"
+ if normalized not in ("png", "jpeg"):
+ raise ValueError(f"Unsupported screenshot format: {format}. Use 'png' or 'jpeg'.")
+ if not isinstance(quality, int) or not 1 <= quality <= 100:
+ raise ValueError(f"quality must be an integer from 1 to 100, got {quality!r}")
+ if max_width is not None and (not isinstance(max_width, int) or max_width < 1):
+ raise ValueError(f"max_width must be a positive integer, got {max_width!r}")
+ return normalized
+
+
+def _encode(image: Image.Image, format: str, quality: int) -> bytes:
+ """
+ Encode an image.
+
+ Args:
+ image (Image.Image): Image to encode.
+ format (str): png or jpeg.
+ quality (int): JPEG quality.
+
+ Returns:
+ bytes: Encoded image.
+ """
+ buffer = io.BytesIO()
+ if format == "jpeg":
+ image.save(buffer, format="JPEG", quality=quality, optimize=True)
+ else:
+ image.save(buffer, format="PNG", optimize=True)
+ return buffer.getvalue()
+
+
+def _flatten_for_jpeg(image: Image.Image) -> Image.Image:
+ """
+ Composite transparency onto white, since JPEG has no alpha channel.
+
+ Args:
+ image (Image.Image): Decoded capture.
+
+ Returns:
+ Image.Image: RGB image.
+ """
+ if image.mode == "P":
+ image = image.convert("RGBA")
+ if image.mode in ("RGBA", "LA"):
+ background = Image.new("RGB", image.size, (255, 255, 255))
+ background.paste(image, mask=image.split()[-1])
+ return background
+ return image.convert("RGB") if image.mode != "RGB" else image
+
+
+def fit_screenshot(
+ capture_path: Path,
+ format: str,
+ quality: int = DEFAULT_QUALITY,
+ max_width: Optional[int] = None,
+) -> Tuple[bytes, Dict[str, Any]]:
+ """
+ Encode a captured screenshot so it fits inline MCP image limits.
+
+ The image is scaled down, keeping its aspect ratio, when it is wider than
+ max_width, longer than MAX_IMAGE_SIDE on either side, or larger than
+ INLINE_MAX_BYTES once encoded. INLINE_MAX_BYTES keeps the base64 payload
+ near 5 MB, which is what Claude and similar clients accept.
+
+ Args:
+ capture_path (Path): File written by the browser capture.
+ format (str): png or jpeg.
+ quality (int): JPEG quality from 1 to 100, ignored for PNG.
+ max_width (Optional[int]): Largest width to return, or None.
+
+ Returns:
+ Tuple[bytes, Dict[str, Any]]: Encoded image and metadata with format,
+ width, height, original_width, original_height, scale, quality, bytes,
+ sha256, and complete.
+
+ Raises:
+ ValueError: When the capture is too large to decode safely or cannot be
+ reduced below the inline limit.
+ """
+ if capture_path.stat().st_size > MAX_CAPTURE_BYTES:
+ raise ValueError("Screenshot exceeds inline capture limit; use file_path for export.")
+ with Image.open(capture_path) as captured:
+ original_width, original_height = captured.size
+ if original_width * original_height > MAX_DECODE_PIXELS:
+ raise ValueError("Screenshot dimensions exceed the decode limit; use file_path for export.")
+ decoded = captured.copy()
+ image = _flatten_for_jpeg(decoded) if format == "jpeg" else decoded
+
+ scale = 1.0
+ if max_width and max_width < original_width:
+ scale = max_width / original_width
+ scale = min(scale, MAX_IMAGE_SIDE / max(original_width, original_height))
+
+ for _ in range(FIT_ATTEMPTS):
+ size = (max(1, round(original_width * scale)), max(1, round(original_height * scale)))
+ resized = image if size == image.size else image.resize(size, Image.LANCZOS)
+ data = _encode(resized, format, quality)
+ if len(data) <= INLINE_MAX_BYTES:
+ return data, {
+ "format": format,
+ "width": size[0],
+ "height": size[1],
+ "original_width": original_width,
+ "original_height": original_height,
+ "scale": round(size[0] / original_width, 4),
+ "quality": quality if format == "jpeg" else None,
+ "bytes": len(data),
+ "sha256": hashlib.sha256(data).hexdigest(),
+ "complete": True,
+ }
+ shrink = (INLINE_MAX_BYTES / len(data)) ** 0.5 * 0.95
+ scale *= min(0.9, max(0.5, shrink))
+ raise ValueError(
+ "Screenshot could not be reduced below the inline limit; lower quality, set max_width, or use file_path."
+ )
diff --git a/src/server.py b/src/server.py
index 8c42f93..f0670e8 100644
--- a/src/server.py
+++ b/src/server.py
@@ -10,7 +10,6 @@ import sys
import tempfile
from collections import defaultdict
from contextlib import asynccontextmanager
-from datetime import datetime
from pathlib import Path
from typing import Any, Dict, List, Optional, Union
@@ -39,6 +38,7 @@ from dynamic_hook_ai_interface import dynamic_hook_ai
from persistent_storage import persistent_storage
from progressive_element_cloner import progressive_element_cloner
from response_handler import response_handler
+from screenshot_encoder import DEFAULT_QUALITY, fit_screenshot, validate_options as validate_screenshot_options
from stdio_guard import install_stdio_guard
from platform_utils import validate_browser_environment, get_platform_info
from process_cleanup import process_cleanup
@@ -799,31 +799,32 @@ async def take_screenshot(
instance_id: str,
full_page: bool = False,
format: str = "png",
- file_path: Optional[str] = None
+ file_path: Optional[str] = None,
+ quality: int = DEFAULT_QUALITY,
+ max_width: Optional[int] = None
) -> Union[str, ToolResult]:
"""
Take a screenshot of the page.
+ The image is returned as MCP image content with metadata. It is scaled
+ down, keeping its aspect ratio, when it is wider than max_width, longer
+ than 8000 pixels on a side, or too large to send inline. The metadata
+ reports the original size and the scale used.
+
Args:
instance_id (str): Browser instance ID.
full_page (bool): Capture full page (not just viewport).
- format (str): Image format ('png' or 'jpeg').
- file_path (Optional[str]): Optional file path to save screenshot to.
+ format (str): Image format ('png' or 'jpeg'). Use jpeg with a lower quality for smaller images.
+ file_path (Optional[str]): Save the full-resolution screenshot to this path on the server instead.
+ quality (int): JPEG quality from 1 to 100. PNG is lossless and ignores it.
+ max_width (Optional[int]): Largest width in pixels to return, for a smaller image.
Returns:
- Union[str, ToolResult]: Explicit server-local file export, or an MCP image
- with format, dimensions and SHA-256 metadata. Inline images are limited
- to 8 MiB; use file_path for larger exports.
+ Union[str, ToolResult]: Server file location when file_path is given, otherwise
+ an MCP image with format, width, height, original size, scale, quality,
+ byte count, and SHA-256 metadata.
"""
- from PIL import Image
- import hashlib
- import io
-
- format = format.lower()
- if format == "jpg":
- format = "jpeg"
- if format not in ("png", "jpeg"):
- raise ValueError(f"Unsupported screenshot format: {format}. Use 'png' or 'jpeg'.")
+ format = validate_screenshot_options(format, quality, max_width)
tab = await browser_manager.get_tab(instance_id)
if not tab:
@@ -834,53 +835,21 @@ async def take_screenshot(
save_path.parent.mkdir(parents=True, exist_ok=True)
await tab.save_screenshot(save_path, format=format, full_page=full_page)
return f"Screenshot saved on the server: {str(save_path.absolute())}"
-
+
with tempfile.NamedTemporaryFile(suffix='.png', delete=False) as tmp_file:
tmp_path = Path(tmp_file.name)
-
+
try:
- await tab.save_screenshot(tmp_path, format=format, full_page=full_page)
- if tmp_path.stat().st_size > 16 * 1024 * 1024:
- raise ValueError("Screenshot exceeds inline capture limit; use file_path for export.")
-
- with Image.open(tmp_path) as img:
- if img.width * img.height > 16 * 1024 * 1024:
- raise ValueError("Screenshot dimensions exceed inline limit; use file_path.")
- if img.mode in ('RGBA', 'LA', 'P') and format.lower() == 'jpeg':
- background = Image.new('RGB', img.size, (255, 255, 255))
- if img.mode == 'P':
- img = img.convert('RGBA')
- background.paste(img, mask=img.split()[-1] if img.mode in ('RGBA', 'LA') else None)
- img = background
-
- output_buffer = io.BytesIO()
-
- if format.lower() == 'jpeg':
- img.save(output_buffer, format='JPEG', quality=85, optimize=True)
- else:
- img.save(output_buffer, format='PNG', optimize=True)
-
- compressed_bytes = output_buffer.getvalue()
-
- if len(compressed_bytes) > 8 * 1024 * 1024:
- raise ValueError("Screenshot exceeds 8 MiB inline limit; use file_path for export.")
- metadata = {
- "format": format,
- "width": img.width,
- "height": img.height,
- "bytes": len(compressed_bytes),
- "sha256": hashlib.sha256(compressed_bytes).hexdigest(),
- "complete": True,
- }
- return ToolResult(
- content=[ImageContent(
- type="image",
- mimeType=f"image/{format}",
- data=base64.b64encode(compressed_bytes).decode("ascii"),
- ), TextContent(type="text", text=json.dumps(metadata))],
- structured_content=metadata,
- )
-
+ await tab.save_screenshot(tmp_path, format="png", full_page=full_page)
+ image_bytes, metadata = fit_screenshot(tmp_path, format, quality, max_width)
+ return ToolResult(
+ content=[ImageContent(
+ type="image",
+ mimeType=f"image/{format}",
+ data=base64.b64encode(image_bytes).decode("ascii"),
+ ), TextContent(type="text", text=json.dumps(metadata))],
+ structured_content=metadata,
+ )
finally:
if tmp_path.exists():
os.unlink(tmp_path)
diff --git a/tests/test_screenshot_content.py b/tests/test_screenshot_content.py
index 539cc19..cd45072 100644
--- a/tests/test_screenshot_content.py
+++ b/tests/test_screenshot_content.py
@@ -23,6 +23,10 @@ from mcp.types import CallToolResult, ImageContent, TextContent
from PIL import Image
ROOT = Path(__file__).resolve().parents[1]
+sys.path.insert(0, str(ROOT / "src"))
+
+import screenshot_encoder
+from screenshot_encoder import DEFAULT_QUALITY, fit_screenshot, validate_options
class ScreenshotContentTests(unittest.TestCase):
@@ -63,7 +67,8 @@ class ScreenshotContentTests(unittest.TestCase):
__name__="screenshot_test", section_tool=lambda _: mcp.tool,
browser_manager=Manager(), Path=Path, Optional=Optional, Union=Union,
ToolResult=ToolResult, ImageContent=ImageContent, TextContent=TextContent,
- tempfile=tempfile, os=os, base64=base64, json=json,
+ tempfile=tempfile, os=os, base64=base64, json=json, DEFAULT_QUALITY=DEFAULT_QUALITY,
+ fit_screenshot=fit_screenshot, validate_screenshot_options=validate_options,
)
exec(compile(ast.Module(body=[function], type_ignores=[]), str(ROOT / "src/server.py"), "exec"), namespace)
return mcp
@@ -131,9 +136,10 @@ class ScreenshotContentTests(unittest.TestCase):
return await client.call_tool("take_screenshot", {"instance_id": "owned"})
result = asyncio.run(run())
image = base64.b64decode(result.content[0].data, validate=True)
- self.assertGreater(len(image), 8 * 1024 * 1024 * .98)
- self.assertLessEqual(len(image), 8 * 1024 * 1024)
- self.assertEqual(Image.open(io.BytesIO(image)).size, (1660, 1660))
+ self.assertGreater(len(image), screenshot_encoder.INLINE_MAX_BYTES * .9)
+ self.assertLessEqual(len(image), screenshot_encoder.INLINE_MAX_BYTES)
+ self.assertEqual(Image.open(io.BytesIO(image)).size, (1100, 1100))
+ self.assertEqual(result.structured_content["scale"], 1.0)
self.assertEqual(result.structured_content["sha256"], hashlib.sha256(image).hexdigest())
def test_explicit_file_export_stays_available(self):
@@ -149,8 +155,9 @@ class ScreenshotContentTests(unittest.TestCase):
self.assertEqual(self.captures, [])
def test_oversized_capture_fails_and_removes_temporary_file(self):
- self.png = b"x" * (16 * 1024 * 1024 + 1)
- result = self.call()
+ self.png = b"x" * 2048
+ with patch.object(screenshot_encoder, "MAX_CAPTURE_BYTES", 1024):
+ result = self.call()
self.assertTrue(result.isError)
self.assertIn("inline capture limit", result.content[0].text)
self.assertFalse(self.captures[0][0].exists())
@@ -158,9 +165,10 @@ class ScreenshotContentTests(unittest.TestCase):
def test_excessive_pixels_are_rejected_before_decode(self):
header = self.png[12:16] + struct.pack(">II", 4097, 4097) + self.png[24:29]
self.png = self.png[:12] + header + struct.pack(">I", zlib.crc32(header)) + self.png[33:]
- result = self.call()
+ with patch.object(screenshot_encoder, "MAX_DECODE_PIXELS", 16 * 1024 * 1024):
+ result = self.call()
self.assertTrue(result.isError)
- self.assertIn("dimensions exceed inline limit", result.content[0].text)
+ self.assertIn("dimensions exceed the decode limit", result.content[0].text)
self.assertFalse(self.captures[0][0].exists())
def test_capture_failure_removes_temporary_file(self):
@@ -184,10 +192,57 @@ class ScreenshotContentTests(unittest.TestCase):
# A synthetic encoder result tests the precise post-compression boundary.
# Input remains a valid small capture; only output encoding is mocked.
self.png = ScreenshotContentTests.small_png()
- with patch.object(Image.Image, "save", lambda _, stream, **kw: stream.write(b"x" * (8 * 1024 * 1024 + 1))):
- self.assertTrue(self.call().isError)
+ oversized = b"x" * (screenshot_encoder.INLINE_MAX_BYTES + 1)
+ with patch.object(Image.Image, "save", lambda _, stream, **kw: stream.write(oversized)):
+ result = self.call()
+ self.assertTrue(result.isError)
+ self.assertIn("could not be reduced", result.content[0].text)
self.assertFalse(self.captures[0][0].exists())
+ def test_tall_full_page_is_scaled_instead_of_failing(self):
+ buffer = io.BytesIO()
+ Image.new("RGB", (400, 20000), (200, 10, 10)).save(buffer, format="PNG")
+ self.png = buffer.getvalue()
+ result = self.call(full_page=True)
+ self.assertFalse(result.isError)
+ meta = result.structuredContent
+ self.assertEqual((meta["width"], meta["height"]), (160, screenshot_encoder.MAX_IMAGE_SIDE))
+ self.assertEqual((meta["original_width"], meta["original_height"]), (400, 20000))
+ self.assertEqual(meta["scale"], 0.4)
+ image = Image.open(io.BytesIO(base64.b64decode(result.content[0].data)))
+ self.assertEqual(image.size, (160, 8000))
+
+ def test_large_encoding_is_scaled_to_fit_byte_limit(self):
+ buffer = io.BytesIO()
+ Image.frombytes("RGB", (600, 600), os.urandom(600 * 600 * 3)).save(buffer, format="PNG")
+ self.png = buffer.getvalue()
+ with patch.object(screenshot_encoder, "INLINE_MAX_BYTES", 300_000):
+ result = self.call()
+ self.assertFalse(result.isError)
+ self.assertLessEqual(result.structuredContent["bytes"], 300_000)
+ self.assertLess(result.structuredContent["scale"], 1.0)
+
+ def test_max_width_and_quality(self):
+ buffer = io.BytesIO()
+ Image.frombytes("RGB", (800, 400), os.urandom(800 * 400 * 3)).save(buffer, format="PNG")
+ self.png = buffer.getvalue()
+ small = self.call(format="jpeg", quality=20, max_width=200).structuredContent
+ large = self.call(format="jpeg", quality=95, max_width=200).structuredContent
+ self.assertEqual((small["width"], small["height"], small["scale"]), (200, 100, 0.25))
+ self.assertEqual((small["quality"], large["quality"]), (20, 95))
+ self.assertLess(small["bytes"], large["bytes"])
+ self.assertIsNone(self.call(format="png").structuredContent["quality"])
+ self.assertEqual(self.call(max_width=5000).structuredContent["scale"], 1.0)
+
+ def test_capture_is_lossless_before_encoding(self):
+ self.call(format="jpeg")
+ self.assertEqual(self.captures[0][1]["format"], "png")
+
+ def test_invalid_quality_or_width_fails_before_capture(self):
+ for options in ({"quality": 0}, {"quality": 101}, {"max_width": 0}):
+ self.assertTrue(self.call(**options).isError, options)
+ self.assertEqual(self.captures, [])
+
@staticmethod
def small_png():
buffer = io.BytesIO()
@@ -201,7 +256,7 @@ if __name__ == "__main__":
fixture.setUp()
if sys.argv[1] == "--serve-large-fixture":
buffer = io.BytesIO()
- Image.frombytes("RGB", (1660, 1660), os.urandom(1660 * 1660 * 3)).save(buffer, format="PNG")
+ Image.frombytes("RGB", (1100, 1100), os.urandom(1100 * 1100 * 3)).save(buffer, format="PNG")
fixture.png = buffer.getvalue()
try:
fixture.server().run(show_banner=False)