Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Fixed version of apify-client dependency, bumped package.json version
123e885 Jan Curn committed 9 years ago
More robust Apifier.getInput() and setOutput(), new unit tests, updated README.md, bumped package.json version
d13be96 Jan Curn committed 9 years ago
Bugfix: key-value store API uses "recordKey" instead of "key", bumped package.json version
3638414 Jan Curn committed 9 years ago
Minor fix in use of APIFY_API_BASE_URL, bumped package.json version
c143722 Jan Curn committed 9 years ago
Better description of the package, bumped package.json version
13e423e Jan Curn committed 9 years ago
Removed old code and dependencies, fixed package.json & bumped version, added new unit tests
53db128 Jan Curn committed 9 years ago