Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated DEFAULT_USER_AGENT
Jan Curn0b8d37e1f72e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex bbe8567be..9715f2c7a 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -2,6 +2,7 @@===================- Added `Apify.utils.htmlToText()` function to convert HTML to text and removed unncessary `html-to-text` dependency.The new function is now used in `Apify.utils.social.parseHandlesFromHtml()`.+- Updated `DEFAULT_USER_AGENT`0.10.0 / 2018-12-19===================diff --git a/src/constants.js b/src/constants.jsindex c8a85516c..b9a33d343 100644--- a/src/constants.js+++ b/src/constants.js@@ -2,9 +2,9 @@ import { ACTOR_EVENT_NAMES } from 'apify-shared/consts';/*** The default user agent used by `Apify.launchPuppeteer`.- * Last updated on 2018-09-10.+ * Last updated on 2018-12-30.*/-export const DEFAULT_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36'; // eslint-disable-line max-len+export const DEFAULT_USER_AGENT = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'; // eslint-disable-line max-len/*** Exit codes for the actor process.