Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
feat: add support for parsing comma-separated list environment variables (#2765)
4e50c47 František Nesveda committed 1 year ago
docs: bump `typedoc-api` plugin to mitigate stray React errors (#2762)
ba80c87 Jindřich Bär committed 1 year ago
chore(release): update internal dependencies [skip ci]
29e2589 Apify Release Bot committed 1 year ago
fix(social): support new URL formats for Facebook, YouTube and X (#2758)
4c95847 Kristýna Lhoťanová committed 1 year ago
docs: format blog How to scrape Google search results with Python (#2760)
cf19158 Max Bohomolov committed 1 year ago
docs: blog How to scrape Google search results with Python (#2739)
0b37625 Max Bohomolov committed 1 year ago