Skip to content

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

2.0 KB
Raw Blame

Crawl4AI

Crawl4AI Cloud is the hosted web crawler API for LLMs and AI agents. No servers to run. Start free at crawl4ai.com. Prefer to self-host? This image is the open-source Crawl4AI server (Apache-2.0).

GitHub Stars Downloads Discord


Run it

The server needs a token. Without one it answers only inside its container.

export CRAWL4AI_API_TOKEN="$(openssl rand -hex 32)"
docker run -d -p 11235:11235 --name crawl4ai --shm-size=1g \
  -e CRAWL4AI_API_TOKEN="$CRAWL4AI_API_TOKEN" \
  unclecode/crawl4ai:latest

Test it (allow about 10 seconds for the start):

curl -s http://localhost:11235/md \
  -H "Authorization: Bearer $CRAWL4AI_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://news.ycombinator.com"}' | jq -r .markdown

The dashboard is at http://localhost:11235/dashboard, the playground at http://localhost:11235/playground.

Images are multi-arch (amd64, arm64). Tags: latest, or a version such as 0.9.5.


Docs

Want this without the ops work? Try Crawl4AI Cloud.


Star on GitHub · Follow @unclecode on X · Discord