Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Update changelog
Ondra Urban89caa53d9b28
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex dd2361391..149d71809 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -1,10 +1,17 @@-NEXT+0.15.4 / 2019-08-02====================+- **DEPRECATED**: `dataset.delete()`, `keyValueStore.delete()` and `requestQueue.delete()` methods have been+ deprecated in favor of `*.drop()` methods, because the `drop` name more clearly communicates the fact+ that those methods drop / delete the storage itself, not individual elements in the storage.+- Added `Apify.utils.requestAsBrowser()` helper function that enables you to make HTTP(S) requests disguising+ as a browser (Firefox). This may help in overcoming certain anti-scraping and anti-bot protections.- Added `options.gotoTimeoutSecs` to `PuppeteerCrawler` to enable easier setting of navigation timeouts.- `PuppeteerPool` options that were deprecated from the `PuppeteerCrawler` constructor were finally removed.Please use `maxOpenPagesPerInstance`, `retireInstanceAfterRequestCount`, `instanceKillerIntervalSecs`,`killInstanceAfterSecs` and `proxyUrls` via the `puppeteerPoolOptions` object.- On the Apify Platform a warning will now be printed when using an outdated `apify` package version.+- `Apify.utils.puppeteer.enqueueLinksByClickingElements()` will now print a warning when the nodes it+ tries to click become modified (detached from DOM). This is useful to debug unexpected behavior.0.15.3 / 2019-07-29====================