Skip to content

Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.

331 B
Raw Blame
type: "llm"
provider: "openai/gpt-4o-mini"
api_token: "env:OPENAI_API_KEY"
instruction: "Extract all articles with their titles, authors, publication dates and main topics in a structured format"
params:
  chunk_token_threshold: 4096
  overlap_rate: 0.1
  word_token_rate: 0.75
  temperature: 0.3
  max_tokens: 1000
  verbose: true