Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Update changelog, bump apify-shared
Ondra Urban6527b30c0f75
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 3d6a84011..6ceefb022 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,4 +1,4 @@-0.21.6 / 2020/09/01+0.21.6 / 2020/09/02====================It appears that `CheerioCrawler` was correctly retiring sessions on timeoutsand blocked status codes (401, 403, 429), whereas `PuppeteerCrawler` did not.@@ -6,6 +6,7 @@ Apologies for the omission, this release fixes the problem.- Fix sessions not being retired on blocked status codes in `PuppeteerCrawler`.- Fix sessions not being marked bad on navigation timeouts in `PuppeteerCrawler`.+- Update `apify-shared` to version `0.5.0`.0.21.5 / 2020/09/30====================diff --git a/package.json b/package.jsonindex 24fb031b1..b108b0483 100644--- a/package.json+++ b/package.json@@ -60,7 +60,7 @@"@types/socket.io": "^2.1.4","@types/tough-cookie": "^4.0.0","apify-client": "^0.6.0",- "apify-shared": "^0.4.0",+ "apify-shared": "^0.5.0","cheerio": "^1.0.0-rc.3","content-type": "^1.0.4","express": "^4.17.1",