Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Apify.getValue() and setValue() now can use local dir if used with the APIFY_DEV_KEY_VALUE_STORE_DIR env variable to simplify local development, added unit tests
07d111c Jan Curn committed 9 years ago
Merge branch 'develop' of github.com:Apifier/apify-runtime-js into develop
2311bf3 Marek Trunkat committed 9 years ago
Fixing getValue() and removing unneeded param from setValue()
2edd432 Marek Trunkat committed 9 years ago
Apify.call - wrapping input to input field and adding some tests
dc136b2 Marek Trunkat committed 9 years ago
Merge branch 'master' of github.com:Apifier/apify-runtime-js into develop
15ff07c Marek Trunkat committed 9 years ago
Another fix in publish.sh, bumped package.json version for test
163eab8 Jan Curn committed 9 years ago
Another fix in publish.sh, bumped package.json version for test
ab37045 Jan Curn committed 9 years ago
Another fix in publish.sh, bumped package.json version for test
d87c105 Jan Curn committed 9 years ago
Fixed publish.sh to avoid "-bash: [: '0.4.12': binary operator expected" error
222c559 Jan Curn committed 9 years ago
Disabled squid cache in proxy_chain.js, bumped package.json version to 0.4.11
542e250 Jan Curn committed 9 years ago