Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
add LaunchOptions for both contexts (#959)
Paulo Cesarea2bed79f4f9
diff --git a/docs/guides/typescript_actor.md b/docs/guides/typescript_actor.mdindex 778f3cc67..50bbd05c9 100644--- a/docs/guides/typescript_actor.md+++ b/docs/guides/typescript_actor.md@@ -48,17 +48,22 @@ To use TypeScript in your actors, you'll need the following prerequisites.```json{"compilerOptions": {- "target": "es2018",+ "target": "es2019","module": "commonjs","moduleResolution": "node","strict": true,"noImplicitAny": false,"strictNullChecks": false,"lib": [- "es2018",- "es2018.asynciterable",- "dom",- "dom.iterable"+ "DOM",+ "DOM.Iterable",+ "ES2015",+ "ES2016",+ "ES2018",+ "ES2019.Object",+ "ES2018.AsyncIterable",+ "ES2020.String",+ "ES2019.Array"],"rootDir": "src/","outDir": "build/"@@ -91,7 +96,7 @@ return types, in the Apify SDK.Caveats========-As of version 0.20, the generated typings, due to JSDoc limitations, have some properties+As of version 1.0+, the generated typings, due to JSDoc limitations, have some propertiesand parameters annotated with `any` type, therefore the settings `noImplicitAny` and `strictNullChecks`, set to `true`, may not be advised. You may try enabling them, but it might hinder development because of the need for typecasts to be able to compile, your mileage may vary.Besides the _implicit any_ errors that might occur in the code when writing in TypeScript, the@@ -104,7 +109,7 @@ interface MySchema {expectedParam2?: number;}-const input: MySchema = await Apify.getInput(); // getInput returns Promise<any> here+const input: MySchema = (await Apify.getInput()) as any; // getInput returns Promise<Object<string, *>|string|Buffer|null> hereif (!input?.expectedParam1) { // input is MySchema now and you can check in a type-safe waythrow new Error('Missing expectedParam1');@@ -125,16 +130,3 @@ await dataset.forEach((item: ExpectedShape) => {// otherwise item is "any"})```--When using `launchPuppeteer()`, puppeteer's `LaunchOptions` needs to be-provided as an intersection using `&`, with the `LaunchPuppeteerOptions`,-so you can have all the options available:--```typescript-// manually import the types from puppeteer-import { PuppeteerOptions } from 'puppeteer'-import { LaunchPuppeteerOptions } from 'apify'--Apify.launchPuppeteer({ } as PuppeteerOptions & LaunchPuppeteerOptions)-// should show all available options intersected-```diff --git a/src/browser_launchers/playwright_launcher.js b/src/browser_launchers/playwright_launcher.jsindex d1a044f5e..a130de2fb 100644--- a/src/browser_launchers/playwright_launcher.js+++ b/src/browser_launchers/playwright_launcher.js@@ -1,4 +1,5 @@import ow from 'ow';+import { LaunchOptions } from 'playwright'; // eslint-disable-line no-unused-vars,import/namedimport { PlaywrightPlugin } from 'browser-pool';import BrowserLauncher from './browser_launcher';@@ -24,7 +25,7 @@ import BrowserLauncher from './browser_launcher';* ```** @typedef PlaywrightLaunchContext- * @property {Object<string, *>} [launchOptions]+ * @property {LaunchOptions} [launchOptions]* `browserType.launch` [options](https://playwright.dev/docs/api/class-browsertype?_highlight=launch#browsertypelaunchoptions)* @property {string} [proxyUrl]* URL to a HTTP proxy server. It must define the port number,diff --git a/src/browser_launchers/puppeteer_launcher.js b/src/browser_launchers/puppeteer_launcher.jsindex 08d8ec745..dda32b90c 100644--- a/src/browser_launchers/puppeteer_launcher.js+++ b/src/browser_launchers/puppeteer_launcher.js@@ -1,4 +1,5 @@import ow from 'ow';+import { LaunchOptions } from 'puppeteer'; // eslint-disable-line no-unused-vars,import/namedimport { PuppeteerPlugin } from 'browser-pool';import BrowserLauncher from './browser_launcher';import { isAtHome } from '../utils';@@ -32,7 +33,7 @@ const LAUNCH_PUPPETEER_DEFAULT_VIEWPORT = {* ```** @typedef PuppeteerLaunchContext- * @property {Object<string, *>} [launchOptions]+ * @property {LaunchOptions} [launchOptions]* `puppeteer.launch` [options](https://pptr.dev/#?product=Puppeteer&version=v5.5.0&show=api-puppeteerlaunchoptions)* @property {string} [proxyUrl]* URL to a HTTP proxy server. It must define the port number,diff --git a/tsconfig.json b/tsconfig.jsonindex e4ef78ad5..5e29b2922 100644--- a/tsconfig.json+++ b/tsconfig.json@@ -10,7 +10,9 @@"ES2016","ES2018","ES2019.Object",- "ES2018.AsyncIterable"+ "ES2018.AsyncIterable",+ "ES2020.String",+ "ES2019.Array"],"baseUrl": "./","allowJs": true,