Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Fix StealthOptions caps that broke the build
Ondra Urbanfa9c9e08cf39
diff --git a/.travis/check-docs.js b/.travis/check-docs.jsindex a5c4fa9cd..1dac68edb 100644--- a/.travis/check-docs.js+++ b/.travis/check-docs.js@@ -5,11 +5,10 @@ execSync('npm run build-docs', { stdio: 'inherit'});execSync('npm run build-readme', { stdio: 'inherit'});console.log('Checking if all changes are committed.');-const gitStatus = execSync('git status', { encoding: 'utf8' });+const gitStatus = execSync('git status --porcelain --branch', { encoding: 'utf8' });const statusLines = gitStatus.split('\n');console.log(gitStatus);if (statusLines.length > 1 && statusLines[1].length) {console.error('You have uncommitted changes in the documentation. Please build the docs, review and commit changes.');- console.error(gitStatus);process.exit(1);}diff --git a/docs/typedefs/stealthOptions.md b/docs/typedefs/stealthOptions.mddeleted file mode 100644index 4e3d311de..000000000--- a/docs/typedefs/stealthOptions.md+++ /dev/null@@ -1,69 +0,0 @@-----id: stealthoptions-title: StealthOptions-----<a name="StealthOptions"></a>--Configuration of stealth tricks for a proper hiding effect all of them should be set to true.-These tricks are applied only when the `stealth` option is set to `true`.--**Properties**-<table>-<thead>-<tr>-<th>Param</th><th>Type</th><th>Default</th>-</tr>-</thead>-<tbody>-<tr>-<td><code>[addPlugins]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>If plugins should be added to the navigator.</p>-</td></tr><tr>-<td><code>[emulateWindowFrame]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Emulates window Iframe.</p>-</td></tr><tr>-<td><code>[emulateWebGL]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Emulates graphic card.</p>-</td></tr><tr>-<td><code>[emulateConsoleDebug]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Emulates console.debug to return null.</p>-</td></tr><tr>-<td><code>[addLanguage]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Adds languages to the navigator.</p>-</td></tr><tr>-<td><code>[hideWebDriver]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Hides the webdriver by changing the navigator proto.</p>-</td></tr><tr>-<td><code>[hackPermissions]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Fakes interaction with permissions.</p>-</td></tr><tr>-<td><code>[mockChrome]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Adds the chrome runtime properties.</p>-</td></tr><tr>-<td><code>[mockChromeInIframe]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Adds the chrome runtime properties inside the every newly created iframe.</p>-</td></tr><tr>-<td><code>[mockDeviceMemory]</code></td><td><code>boolean</code></td><td><code>true</code></td>-</tr>-<tr>-<td colspan="3"><p>Sets device memory to other value than 0.</p>-</td></tr></tbody>-</table>