Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updating README
Marek Trunkat031c17ad7d8c
diff --git a/README.md b/README.mdindex 42650a22e..0657dcf99 100644--- a/README.md+++ b/README.md@@ -117,7 +117,7 @@ The Apify SDK package provides the following tools:like list of products or real estate offers.</li><li>- <a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#Dataset">KeyValueStore</a>+ <a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#KeyValueStore">KeyValueStore</a>- A simple key-value store for arbitrary data records or files, along with their MIME content type.It is ideal for saving screenshots of web pages, PDFs or any downloaded files.The data is stored on local filesystem or in the cloud.@@ -660,7 +660,10 @@ Apify.main(async () => {## Storage-Each actor run at Apify platform has assigned its default storages (key-value store, request queue and dataset) which are available+Each actor run at Apify platform has assigned its default storages+(<a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#KeyValueStore">key-value store</a>,+<a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#RequestQueue">request queue</a> and+<a href="https://www.apify.com/docs/sdk/apify-runtime-js/latest#Dataset">dataset</a>) which are availablevia API and helper functions such as `Apify.setValue()`, `Apify.pushData()`, etc.. If you are running actor locally then the dataget stored in the directory defined by <code>APIFY_LOCAL_EMULATION_DIR</code> environment variable and its subdirectories based onfollowing environment variables: