Skip to content

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

release: Crawl4AI v0.9.2

ntohidi055e2ecdc702
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 1eadb852..c2926ec8 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -58,9 +58,11 @@ We would like to thank the following people for their contributions to Crawl4AI:
- [microHoffman](https://github.com/microHoffman) - feat: add --json-ensure-ascii CLI flag for Unicode handling [#1668](https://github.com/unclecode/crawl4ai/pull/1668)
- [fstark96](https://github.com/fstark96) - fix: skip channel='chromium' on Windows to prevent Chrome exit code 0 [#2051](https://github.com/unclecode/crawl4ai/pull/2051)
- [TobiasWallura-xitaso](https://github.com/TobiasWallura-xitaso) - fix: use writable directories for supervisord pid and redis data in Docker [#2047](https://github.com/unclecode/crawl4ai/pull/2047)
-- [harshmathurx](https://github.com/harshmathurx) - fix: authenticate rate limit Redis storage in Docker [#2043](https://github.com/unclecode/crawl4ai/pull/2043)
+- [harshmathurx](https://github.com/harshmathurx) - fix: authenticate rate limit Redis storage in Docker [#2043](https://github.com/unclecode/crawl4ai/pull/2043), fix: add non-free apt source for nvidia-cuda-toolkit GPU builds [#2057](https://github.com/unclecode/crawl4ai/pull/2057)
- [RajanChavada](https://github.com/RajanChavada) - chore: remove dead normalize_url definitions and accidental adaptive_crawler copy [#2042](https://github.com/unclecode/crawl4ai/pull/2042)
- [bibi-creator](https://github.com/bibi-creator) - docs: split sponsors into Strategic Partners and Enterprise Sponsors [#2056](https://github.com/unclecode/crawl4ai/pull/2056), fix: update sponsor logos [#2052](https://github.com/unclecode/crawl4ai/pull/2052)
+- [reallav0](https://github.com/reallav0) - fix: clean up MemoryAdaptiveDispatcher tasks on stream close [#2072](https://github.com/unclecode/crawl4ai/pull/2072), fix: include Playwright headless shell in Docker image [#2067](https://github.com/unclecode/crawl4ai/pull/2067)
+- [Pitchfork-and-Torch](https://github.com/Pitchfork-and-Torch) - fix: Docker playground config type and monitor WebSocket auth [#2063](https://github.com/unclecode/crawl4ai/pull/2063)
#### Feb-Alpha-1
- [sufianuddin](https://github.com/sufianuddin) - fix: [Documentation for JsonCssExtractionStrategy](https://github.com/unclecode/crawl4ai/issues/651)
diff --git a/Dockerfile b/Dockerfile
index 92f1404f..3b39e307 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,7 +1,7 @@
FROM python:3.12-slim-bookworm AS build
# C4ai version
-ARG C4AI_VER=0.9.1
+ARG C4AI_VER=0.9.2
ENV C4AI_VERSION=$C4AI_VER
LABEL c4ai.version=$C4AI_VER
diff --git a/README.md b/README.md
index 20367f5f..705f42ac 100644
--- a/README.md
+++ b/README.md
@@ -37,9 +37,9 @@ Limited slots._
Crawl4AI turns the web into clean, LLM ready Markdown for RAG, agents, and data pipelines. Fast, controllable, battle tested by a 50k+ star community.
-[✨ Check out latest update v0.9.1](#-recent-updates)
+[✨ Check out latest update v0.9.2](#-recent-updates)
-✨ **New in v0.9.1**: Patch release with 12 bug fixes across Docker, browser, and core. Adds `preserve_classes`/`preserve_tags` whitelist for PruningContentFilter, fixes Windows browser crash, Docker auth gate UI, HTTP timeout unit mismatch, and more. [Release notes →](https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.9.1.md)
+✨ **New in v0.9.2**: Maintenance patch release. Fixes a `MemoryAdaptiveDispatcher` task/page leak when a streaming crawl is closed, Docker Playground "Advanced Config" and Monitor WebSocket auth, Playwright headless-shell packaging, and GPU (`ENABLE_GPU=true`) Docker builds. [Release notes →](https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.9.2.md)
✨ Recent v0.9.0: Major secure-by-default release of the Docker API server. Auth is on by default, the server binds loopback unless given a token, and the request body is now an untrusted trust boundary. [Release notes →](https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.9.0.md)
@@ -565,6 +565,21 @@ async def test_news_crawl():
## ✨ Recent Updates
<details open>
+<summary><strong>Version 0.9.2 Release Highlights - Maintenance Bug Fixes</strong></summary>
+
+A maintenance patch release with bug fixes across the dispatcher, Docker, and GPU builds. `MemoryAdaptiveDispatcher` no longer leaks crawl tasks and browser pages when a streaming crawl is closed. Docker fixes cover the Playground "Advanced Config" 400, the Monitor WebSocket 500 under JWT auth, and Playwright headless-shell packaging. `ENABLE_GPU=true` Docker builds no longer fail on the CUDA toolkit.
+
+No new features, no breaking changes.
+
+```bash
+pip install -U crawl4ai
+```
+
+[Full v0.9.2 Release Notes →](https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.9.2.md)
+
+</details>
+
+<details>
<summary><strong>Version 0.9.1 Release Highlights - Bug Fixes & PruningContentFilter Whitelist</strong></summary>
A patch release with 12 bug fixes and one new feature. The new `preserve_classes` / `preserve_tags` parameters for `PruningContentFilter` let you whitelist CSS classes or HTML tags that should never be pruned — useful for protecting short metadata elements like author names and timestamps.
diff --git a/crawl4ai/__version__.py b/crawl4ai/__version__.py
index 6dac9465..505ca152 100644
--- a/crawl4ai/__version__.py
+++ b/crawl4ai/__version__.py
@@ -1,7 +1,7 @@
# crawl4ai/__version__.py
# This is the version that will be used for stable releases
-__version__ = "0.9.1"
+__version__ = "0.9.2"
# For nightly builds, this gets set during build process
__nightly_version__ = None
diff --git a/deploy/docker/README.md b/deploy/docker/README.md
index a711c720..7ce0dcd0 100644
--- a/deploy/docker/README.md
+++ b/deploy/docker/README.md
@@ -59,10 +59,10 @@ Pull and run images directly from Docker Hub without building locally.
#### 1. Pull the Image
-Our latest stable release is `0.8.6`. Images are built with multi-arch manifests, so Docker automatically pulls the correct version for your system.
+Our latest stable release is `0.9.2`. Images are built with multi-arch manifests, so Docker automatically pulls the correct version for your system.
```bash
-# Pull the latest stable version (0.8.6)
+# Pull the latest stable version (0.9.2)
docker pull unclecode/crawl4ai:0.8.6
# Or use the latest tag
diff --git a/docs/blog/release-v0.9.2.md b/docs/blog/release-v0.9.2.md
new file mode 100644
index 00000000..95a47aa5
--- /dev/null
+++ b/docs/blog/release-v0.9.2.md
@@ -0,0 +1,67 @@
+# Crawl4AI v0.9.2: Maintenance Bug Fixes
+
+*July 2026 - 2 min read*
+
+---
+
+I'm releasing Crawl4AI v0.9.2, a small maintenance patch that cleans up a dispatcher resource leak and fixes a handful of Docker and GPU build issues. No new features, no breaking changes.
+
+If you're on v0.9.1, upgrade freely.
+
+## What's new at a glance
+
+- **Dispatcher cleanup**: `MemoryAdaptiveDispatcher` no longer leaves crawl tasks and browser pages running after a streaming crawl is closed
+- **Docker Playground**: "Advanced Config" no longer returns a 400
+- **Docker Monitor**: the dashboard WebSocket no longer returns a 500 under JWT auth
+- **Docker image**: Playwright headless shell is now included
+- **GPU builds**: `ENABLE_GPU=true` Docker builds no longer fail on the CUDA toolkit
+
+## Bug fixes
+
+### Dispatcher (1 fix)
+
+- **Streaming cleanup leak**: When the async generator from `MemoryAdaptiveDispatcher.run_urls_stream()` was closed or cancelled mid-stream, its per-URL crawl tasks kept running in the background. A later `arun_many()` on the same crawler could then overlap the old tasks while they still held the same browser, contexts, and pages — leaking pages and emitting stray Playwright `TargetClosedError`s. Closing the stream now cancels and awaits every in-flight task and drains any queued-but-unstarted URLs before returning. (#2071, thanks @reallav0)
+
+### Docker (3 fixes)
+
+- **Playground "Advanced Config" 400**: `/config/dump` requires a `type`, but the playground's `pyConfigToJson` only sent `code`. The request now includes the config type, aligns the stream fallback with the dump shape, and teaches `shouldUseStream` both nestings. (#2059, thanks @Pitchfork-and-Torch)
+- **Monitor WebSocket 500**: the monitor dashboard's `/monitor/ws` endpoint 500'd because a router-level `token_dep` is HTTP-request-only and can't inject into WebSocket scopes. Auth is now enforced solely by `AuthGateMiddleware`; admin routes keep `require_admin`. (#2060, thanks @Pitchfork-and-Torch)
+- **Headless shell packaging**: the Playwright headless shell is now installed in the Docker image. (#2067, thanks @reallav0)
+
+### GPU / Build (1 fix)
+
+- **CUDA toolkit on Bookworm**: `ENABLE_GPU=true` builds failed with "Package has no installation candidate" because `nvidia-cuda-toolkit` lives in Debian Bookworm's `non-free` component, which the `python:3.12-slim-bookworm` base image omits. The Dockerfile now adds the `non-free` (and contrib) apt sources before the GPU install block. (#2020, thanks @harshmathurx)
+
+## Tests
+
+- Regression test added for the dispatcher stream-close cleanup, asserting in-flight tasks are cancelled, the queue is drained, and session count returns to zero.
+- Updated the monitor auth test for the WebSocket-safe router config, and corrected a regex group name and the Playwright cache-copy target in the Docker tests.
+
+## Breaking changes
+
+None.
+
+## Upgrade
+
+```bash
+pip install -U crawl4ai
+crawl4ai-doctor # verify installation
+```
+
+Docker users: pull the latest image once the Docker release workflow finishes.
+
+## Verification
+
+```bash
+python docs/releases_review/demo_v0.9.2.py
+```
+
+## Acknowledgments
+
+Thanks to the community contributors who made this release possible: @reallav0 (#2071, #2067), @Pitchfork-and-Torch (#2059, #2060), @harshmathurx (#2020).
+
+## Support & Resources
+
+- [Documentation](https://docs.crawl4ai.com)
+- [GitHub Issues](https://github.com/unclecode/crawl4ai/issues)
+- [Discord Community](https://discord.gg/crawl4ai)
diff --git a/docs/md_v2/blog/index.md b/docs/md_v2/blog/index.md
index 4d3dd1b5..30a9dbe1 100644
--- a/docs/md_v2/blog/index.md
+++ b/docs/md_v2/blog/index.md
@@ -20,6 +20,20 @@ Ever wondered why your AI coding assistant struggles with your library despite c
## Latest Release
+### [Crawl4AI v0.9.2 – Maintenance Bug Fixes](../blog/release-v0.9.2.md)
+*July 2026*
+
+Crawl4AI v0.9.2 is a maintenance patch release with bug fixes across the dispatcher, Docker, and GPU builds.
+
+Key highlights:
+- **🧹 Dispatcher Cleanup**: `MemoryAdaptiveDispatcher` no longer leaks crawl tasks and browser pages when a streaming crawl is closed
+- **🐳 Docker Fixes**: Playground "Advanced Config" 400, Monitor WebSocket auth 500, and Playwright headless-shell packaging
+- **🖥️ GPU Builds**: `ENABLE_GPU=true` Docker builds fixed (CUDA toolkit `non-free` apt source)
+
+[Read full release notes →](../blog/release-v0.9.2.md)
+
+## Recent Releases
+
### [Crawl4AI v0.9.1 – Bug Fixes & PruningContentFilter Whitelist](../blog/release-v0.9.1.md)
*July 2026*
@@ -32,8 +46,6 @@ Key highlights:
[Read full release notes →](../blog/release-v0.9.1.md)
-## Recent Releases
-
### [Crawl4AI v0.8.5 – Anti-Bot Detection, Shadow DOM & 60+ Bug Fixes](../blog/release-v0.8.5.md)
*March 2026*
diff --git a/docs/md_v2/blog/releases/v0.9.2.md b/docs/md_v2/blog/releases/v0.9.2.md
new file mode 100644
index 00000000..95a47aa5
--- /dev/null
+++ b/docs/md_v2/blog/releases/v0.9.2.md
@@ -0,0 +1,67 @@
+# Crawl4AI v0.9.2: Maintenance Bug Fixes
+
+*July 2026 - 2 min read*
+
+---
+
+I'm releasing Crawl4AI v0.9.2, a small maintenance patch that cleans up a dispatcher resource leak and fixes a handful of Docker and GPU build issues. No new features, no breaking changes.
+
+If you're on v0.9.1, upgrade freely.
+
+## What's new at a glance
+
+- **Dispatcher cleanup**: `MemoryAdaptiveDispatcher` no longer leaves crawl tasks and browser pages running after a streaming crawl is closed
+- **Docker Playground**: "Advanced Config" no longer returns a 400
+- **Docker Monitor**: the dashboard WebSocket no longer returns a 500 under JWT auth
+- **Docker image**: Playwright headless shell is now included
+- **GPU builds**: `ENABLE_GPU=true` Docker builds no longer fail on the CUDA toolkit
+
+## Bug fixes
+
+### Dispatcher (1 fix)
+
+- **Streaming cleanup leak**: When the async generator from `MemoryAdaptiveDispatcher.run_urls_stream()` was closed or cancelled mid-stream, its per-URL crawl tasks kept running in the background. A later `arun_many()` on the same crawler could then overlap the old tasks while they still held the same browser, contexts, and pages — leaking pages and emitting stray Playwright `TargetClosedError`s. Closing the stream now cancels and awaits every in-flight task and drains any queued-but-unstarted URLs before returning. (#2071, thanks @reallav0)
+
+### Docker (3 fixes)
+
+- **Playground "Advanced Config" 400**: `/config/dump` requires a `type`, but the playground's `pyConfigToJson` only sent `code`. The request now includes the config type, aligns the stream fallback with the dump shape, and teaches `shouldUseStream` both nestings. (#2059, thanks @Pitchfork-and-Torch)
+- **Monitor WebSocket 500**: the monitor dashboard's `/monitor/ws` endpoint 500'd because a router-level `token_dep` is HTTP-request-only and can't inject into WebSocket scopes. Auth is now enforced solely by `AuthGateMiddleware`; admin routes keep `require_admin`. (#2060, thanks @Pitchfork-and-Torch)
+- **Headless shell packaging**: the Playwright headless shell is now installed in the Docker image. (#2067, thanks @reallav0)
+
+### GPU / Build (1 fix)
+
+- **CUDA toolkit on Bookworm**: `ENABLE_GPU=true` builds failed with "Package has no installation candidate" because `nvidia-cuda-toolkit` lives in Debian Bookworm's `non-free` component, which the `python:3.12-slim-bookworm` base image omits. The Dockerfile now adds the `non-free` (and contrib) apt sources before the GPU install block. (#2020, thanks @harshmathurx)
+
+## Tests
+
+- Regression test added for the dispatcher stream-close cleanup, asserting in-flight tasks are cancelled, the queue is drained, and session count returns to zero.
+- Updated the monitor auth test for the WebSocket-safe router config, and corrected a regex group name and the Playwright cache-copy target in the Docker tests.
+
+## Breaking changes
+
+None.
+
+## Upgrade
+
+```bash
+pip install -U crawl4ai
+crawl4ai-doctor # verify installation
+```
+
+Docker users: pull the latest image once the Docker release workflow finishes.
+
+## Verification
+
+```bash
+python docs/releases_review/demo_v0.9.2.py
+```
+
+## Acknowledgments
+
+Thanks to the community contributors who made this release possible: @reallav0 (#2071, #2067), @Pitchfork-and-Torch (#2059, #2060), @harshmathurx (#2020).
+
+## Support & Resources
+
+- [Documentation](https://docs.crawl4ai.com)
+- [GitHub Issues](https://github.com/unclecode/crawl4ai/issues)
+- [Discord Community](https://discord.gg/crawl4ai)
diff --git a/docs/releases_review/demo_v0.9.2.py b/docs/releases_review/demo_v0.9.2.py
new file mode 100644
index 00000000..aa028d91
--- /dev/null
+++ b/docs/releases_review/demo_v0.9.2.py
@@ -0,0 +1,194 @@
+#!/usr/bin/env python3
+"""
+Crawl4AI v0.9.2 Release Demo - Fix Verification Tests
+======================================================
+
+This demo ACTUALLY RUNS and VERIFIES the key changes in v0.9.2.
+v0.9.2 is a maintenance patch release with no new features, so each test
+exercises a bug-fix path end-to-end.
+
+Fixes Verified (Python-testable):
+1. MemoryAdaptiveDispatcher cleans up tasks when a stream is closed (#2071)
+2. Basic real-crawl smoke test (the release build crawls end-to-end)
+
+Not covered here (validated by CI image builds, not this script):
+- Docker Playground "Advanced Config" 400 (#2059)
+- Docker Monitor WebSocket auth 500 (#2060)
+- Playwright headless-shell packaging (#2067)
+- GPU ENABLE_GPU=true build / CUDA toolkit non-free apt source (#2020)
+
+Usage:
+ python docs/releases_review/demo_v0.9.2.py
+"""
+
+import asyncio
+import sys
+from dataclasses import dataclass
+from types import SimpleNamespace
+
+
+@dataclass
+class TestResult:
+ name: str
+ feature: str
+ passed: bool
+ message: str
+ skipped: bool = False
+
+
+results: list[TestResult] = []
+
+
+def print_header(title: str):
+ print(f"\n{'=' * 70}")
+ print(f"{title}")
+ print(f"{'=' * 70}")
+
+
+def print_test(name: str, feature: str):
+ print(f"\n[TEST] {name} ({feature})")
+ print("-" * 50)
+
+
+def record_result(name: str, feature: str, passed: bool, message: str, skipped: bool = False):
+ results.append(TestResult(name, feature, passed, message, skipped))
+ status = "SKIP" if skipped else ("PASS" if passed else "FAIL")
+ print(f" [{status}] {message}")
+
+
+# ── Test 1: Dispatcher stream-close cleanup (#2071) ───────────────────
+
+async def test_dispatcher_stream_cleanup():
+ """Closing a streaming MemoryAdaptiveDispatcher must cancel in-flight
+ per-URL tasks, drain queued URLs, and restore concurrent_sessions."""
+ print_test("Dispatcher stream-close cleanup", "MemoryAdaptiveDispatcher #2071")
+
+ from crawl4ai import CrawlerRunConfig, MemoryAdaptiveDispatcher
+
+ class BlockingCrawler:
+ """Return one result immediately; block all other URLs forever."""
+
+ async def arun(self, url, config=None, session_id=None):
+ if url == "fast":
+ return SimpleNamespace(success=True, status_code=200, error_message="")
+ await asyncio.Event().wait()
+
+ class TrackingDispatcher(MemoryAdaptiveDispatcher):
+ def __init__(self):
+ super().__init__(max_session_permit=3)
+ self.tasks = {}
+
+ async def crawl_url(self, url, config, task_id, retry_count=0):
+ self.tasks[url] = asyncio.current_task()
+ return await super().crawl_url(url, config, task_id, retry_count)
+
+ dispatcher = TrackingDispatcher()
+ stream = dispatcher.run_urls_stream(
+ ["fast", "blocked-1", "blocked-2", "queued"],
+ BlockingCrawler(),
+ CrawlerRunConfig(),
+ )
+
+ try:
+ first = await asyncio.wait_for(stream.__anext__(), timeout=5)
+ assert first.url == "fast", f"expected first result 'fast', got {first.url!r}"
+
+ # Close the stream while blocked-1 / blocked-2 are still running.
+ await asyncio.wait_for(stream.aclose(), timeout=5)
+
+ started = set(dispatcher.tasks)
+ all_done = all(t.done() for t in dispatcher.tasks.values())
+ blocked_cancelled = (
+ dispatcher.tasks["blocked-1"].cancelled()
+ and dispatcher.tasks["blocked-2"].cancelled()
+ )
+
+ if (
+ started == {"fast", "blocked-1", "blocked-2"}
+ and all_done
+ and blocked_cancelled
+ and dispatcher.concurrent_sessions == 0
+ and dispatcher.task_queue.empty()
+ ):
+ record_result(
+ "dispatcher_stream_cleanup", "dispatcher", True,
+ "aclose() cancelled in-flight tasks, drained queue, sessions=0",
+ )
+ else:
+ record_result(
+ "dispatcher_stream_cleanup", "dispatcher", False,
+ f"leaked state: started={started}, all_done={all_done}, "
+ f"blocked_cancelled={blocked_cancelled}, "
+ f"sessions={dispatcher.concurrent_sessions}, "
+ f"queue_empty={dispatcher.task_queue.empty()}",
+ )
+ finally:
+ # Belt-and-suspenders: never let the demo itself leak tasks.
+ for t in dispatcher.tasks.values():
+ if not t.done():
+ t.cancel()
+ await asyncio.gather(*dispatcher.tasks.values(), return_exceptions=True)
+
+
+# ── Test 2: Basic real-crawl smoke test ───────────────────────────────
+
+async def test_basic_crawl_smoke():
+ """The release build should crawl a real page end-to-end and produce
+ markdown. Skips gracefully if no browser/network is available."""
+ print_test("Basic crawl smoke test", "arun end-to-end")
+
+ from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig
+
+ try:
+ async with AsyncWebCrawler(config=BrowserConfig(headless=True)) as crawler:
+ result = await crawler.arun(
+ url="https://example.com",
+ config=CrawlerRunConfig(),
+ )
+ if result.success and result.markdown and len(str(result.markdown)) > 0:
+ record_result(
+ "basic_crawl_smoke", "core", True,
+ f"crawled example.com, markdown length={len(str(result.markdown))}",
+ )
+ else:
+ record_result(
+ "basic_crawl_smoke", "core", False,
+ f"crawl failed: success={result.success}, error={result.error_message}",
+ )
+ except Exception as e:
+ record_result(
+ "basic_crawl_smoke", "core", False,
+ f"browser/network unavailable — skipped: {e}", skipped=True,
+ )
+
+
+# ── Main ──────────────────────────────────────────────────────────────
+
+async def main():
+ print_header("Crawl4AI v0.9.2 Release Verification")
+
+ await test_dispatcher_stream_cleanup()
+ await test_basic_crawl_smoke()
+
+ # Summary
+ print_header("RESULTS SUMMARY")
+ total = len(results)
+ passed = sum(1 for r in results if r.passed and not r.skipped)
+ failed = sum(1 for r in results if not r.passed and not r.skipped)
+ skipped = sum(1 for r in results if r.skipped)
+
+ for r in results:
+ status = "SKIP" if r.skipped else ("PASS" if r.passed else "FAIL")
+ print(f" [{status}] {r.name}: {r.message}")
+
+ print(f"\nTotal: {total} | Passed: {passed} | Failed: {failed} | Skipped: {skipped}")
+
+ if failed > 0:
+ print("\n⚠️ Some tests FAILED. Review before release.")
+ sys.exit(1)
+ else:
+ print("\n✅ All tests passed!")
+
+
+if __name__ == "__main__":
+ asyncio.run(main())