Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Merge pull request #550 from apifytech/fix/cookie-parsing
Ondra Urban0235036bbdc0
diff --cc docs/api/PuppeteerCrawler.mdindex 696ebdb71,ac621f924..dbc9535dc--- a/docs/api/PuppeteerCrawler.md+++ b/docs/api/PuppeteerCrawler.md@@@ -93,11 -93,12 +93,12 @@@ await crawler.run()response: Response,page: Page,puppeteerPool: PuppeteerPool,- autoscaledPool: AutoscaledPool+ autoscaledPool: AutoscaledPool,+ session: Session,}</code></pre><p> <code>request</code> is an instance of the <a href="request"><code>Request</code></a> object with details about the URL to open, HTTP method etc.- <code>response</code> is an instance of the <code>Puppeteer</code>- <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a><code>page</code> is an instance of the <code>Puppeteer</code>+ <a href="https://pptr.dev/#?product=Puppeteer&show=api-class-page" target="_blank"><code>Page</code></a>.+ <code>response</code> is an instance of the <code>Puppeteer</code><a href="https://pptr.dev/#?product=Puppeteer&show=api-class-response" target="_blank"><code>Response</code></a>,which is the main resource response as returned by <code>page.goto(request.url)</code>.<code>puppeteerPool</code> is an instance of the <a href="puppeteerpool"><code>PuppeteerPool</code></a> used by this <code>PuppeteerCrawler</code>.</p>