Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Update changelog

Ondra Urban109642ae1a30
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 640d731eb..926b2a5fc 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,9 @@
+xxxxxxxxxxxxxxxxxxx
+===================
+- Added `getInfo()` method to Dataset to get meta-information about a dataset.
+- Added CheerioCrawler, a specialized class for crawling the web using `cheerio`
+- Added `keepDuplicateUrls` option to RequestList to allow duplicate URLs
+
0.6.1 / 2018-08-17
===================
- Replaced git-hosted version of our fork of ps-tree with @apify/ps-tree package