Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated docs
Jan Curn2a3318b53cb0
diff --git a/docs/api/social.md b/docs/api/social.mdnew file mode 100644index 000000000..bdd0d8aa6--- /dev/null+++ b/docs/api/social.md@@ -0,0 +1,17 @@+---+id: social+title: social+---+<a name="social"></a>++A namespace that contains various utilities to help you extract social handles+from text, URLs and and HTML documents.++**Example usage:**++```javascript+const Apify = require('apify');++const emails = Apify.utils.social.emailsFromText('[email protected] [email protected]');+```+diff --git a/docs/api/utils.md b/docs/api/utils.mdindex 2dd1421fb..f1abe4c24 100644--- a/docs/api/utils.md+++ b/docs/api/utils.md@@ -28,13 +28,13 @@ await Apify.utils.sleep(1500);<a name="utils.URL_NO_COMMAS_REGEX"></a>-## `utils.URL_NO_COMMAS_REGEX`+## `utils.URL\_NO\_COMMAS\_REGEX`Default regular expression to match URLs in a string that may be plain text, JSON, CSV or other. It supports common URL charactersand does not support URLs containing commas or spaces. The URLs also may contain Unicode letters (not symbols).<a name="utils.URL_WITH_COMMAS_REGEX"></a>-## `utils.URL_WITH_COMMAS_REGEX`+## `utils.URL\_WITH\_COMMAS\_REGEX`Regular expression that, in addition to the default regular expression `URL_NO_COMMAS_REGEX`, supports matching commas in URL path and query.Note, however, that this may prevent parsing URLs from comma delimited lists, or the URLs may become malformed.diff --git a/src/utils_social.js b/src/utils_social.jsindex aed1cbf77..1b73500a9 100644--- a/src/utils_social.js+++ b/src/utils_social.js@@ -134,7 +134,7 @@ const SKIP_PHONE_REGEX = new RegExp(`^(${SKIP_PHONE_REGEXS.join('|')})$`, 'i');* @param {String} text Text to search the phone numbers in.* @return {String[]} Array of phone numbers found.* If no phone numbers are found, the function returns an empty array.- * @memberOf utils+ * @memberOf utils.social*/const phonesFromText = (text) => {if (!_.isString(text)) return [];@@ -162,7 +162,7 @@ const phonesFromText = (text) => {* @param {String[]} urls Array of URLs.* @return {String[]} Array of phone numbers found.* If no phone numbers are found, the function returns an empty array.- * @memberOf utils+ * @memberOf utils.social*/const phonesFromUrls = (urls) => {if (!Array.isArray(urls)) throw new Error('The "urls" parameter must be an array');@@ -205,7 +205,7 @@ const FACEBOOK_PROFILE_REGEX_STRING = `(?<!\\w)(?:http(?:s)?:\\/\\/)?(?:www.)?(?** Example usage:* ```- * xx+ * TODO* ```* @type {RegExp}* @memberOf utils.social