Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

1005 B
Raw Blame

@crawlee/impit-client

This package provides a Crawlee-compliant HttpClient interface for the impit package.

To use the impit package directly without Crawlee, check out impit on NPM.

Example usage

Simply pass the ImpitHttpClient instance to the httpClient option of the crawler constructor:

import { CheerioCrawler, Dictionary } from '@crawlee/cheerio';
import { ImpitHttpClient, Browser } from '@crawlee/impit-client';

const crawler = new CheerioCrawler({
    httpClient: new ImpitHttpClient({
        browser: Browser.Firefox,
        http3: true,
        ignoreTlsErrors: true,
    }),
    async requestHandler({ $, request }) {
        // Extract the title of the page.
        const title = $('title').text();
        console.log(`Title of the page ${request.url}: ${title}`);
    },
});

crawler.run([
    'http://www.example.com/page-1',
    'http://www.example.com/page-2',
]);