Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

5.5 KB
Raw Blame
---
id: impit-http-client
title: Impit HTTP Client
description: Browser impersonation for HTTP requests using the Impit library
---

import CodeBlock from '@theme/CodeBlock';

import BasicUsageSource from '!!raw-loader!./basic-usage.ts';
import CheerioCrawlerSource from '!!raw-loader!./cheerio-crawler.ts';
import HttpCrawlerSource from '!!raw-loader!./http-crawler.ts';
import AdvancedConfigSource from '!!raw-loader!./advanced-config.ts';

The `ImpitHttpClient` is an HTTP client implementation based on the [Impit](https://github.com/apify/impit) library. It enables browser impersonation for HTTP requests, helping you bypass bot detection systems without running an actual browser.

:::info Successor to got-scraping

Impit is the successor to `got-scraping`, which is no longer actively maintained. We recommend using `ImpitHttpClient` for all new projects. Impit provides better anti-bot evasion through TLS fingerprinting and HTTP/3 support, while maintaining a smaller package size.

**Impit will become the default HTTP client in the next major version of Crawlee.**

:::

### Why use Impit?

Websites increasingly use sophisticated bot detection that analyzes:

- **HTTP fingerprints**: User-Agent strings, header ordering, HTTP/2 pseudo-header sequences
- **TLS fingerprints**: Cipher suites, TLS extensions, and cryptographic details in the ClientHello message

Standard HTTP clients like `fetch` or `axios` are easily detected because their fingerprints don't match real browsers. Unlike `got-scraping` which only handles HTTP-level fingerprinting, Impit also mimics TLS fingerprints, making requests appear to come from real browsers.

## Installation

Install the `@crawlee/impit-client` package:

```bash npm2yarn
npm install @crawlee/impit-client
```

:::note

The `impit` package includes native binaries and supports Windows, macOS (including ARM), and Linux out of the box.

:::

## Basic usage

Pass the `ImpitHttpClient` instance to the `httpClient` option of any Crawlee crawler:

<CodeBlock language="ts">{BasicUsageSource}</CodeBlock>

## Usage with different crawlers

### CheerioCrawler

<CodeBlock language="ts">{CheerioCrawlerSource}</CodeBlock>

### HttpCrawler

<CodeBlock language="ts">{HttpCrawlerSource}</CodeBlock>

## Configuration options

The `ImpitHttpClient` constructor accepts the following options:

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `browser` | `'chrome'` \| `'firefox'` \| `...` | `undefined` | Browser to impersonate. Affects TLS fingerprint and default headers. |
| `http3` | `boolean` | `false` | Enable HTTP/3 (QUIC) protocol support. |
| `ignoreTlsErrors` | `boolean` | `false` | Ignore TLS certificate errors. Useful for testing or self-signed certificates. |


### Available fingerprints

Impit bundles several realistic browser fingerprints.
Using version-specific fingerprints can improve success rates against sophisticated bot detection systems that track browser versions and flag outdated signatures.

When using generic fingerprints (`chrome` or `firefox`), Impit automatically selects an appropriate version. For most use cases, the generic options are sufficient.
However, if you're targeting a site with particularly strict bot detection, or need to match a specific browser environment, you can specify an exact version:

```ts
import { ImpitHttpClient } from '@crawlee/impit-client';

// Use a generic Chrome fingerprint
const chromeClient = new ImpitHttpClient({ browser: 'chrome' });

// Use a specific Chrome version
const chrome131Client = new ImpitHttpClient({ browser: 'chrome131' });

// Or a specific Firefox version
const firefox144Client = new ImpitHttpClient({ browser: 'firefox144' });
```

### Advanced configuration

<CodeBlock language="ts">{AdvancedConfigSource}</CodeBlock>

## Proxy support

Proxies are configured per-request through Crawlee's proxy management system, not on the `ImpitHttpClient` itself. Use `ProxyConfiguration` as you normally would:

```ts
import { CheerioCrawler, ProxyConfiguration } from 'crawlee';
import { ImpitHttpClient, Browser } from '@crawlee/impit-client';

const proxyConfiguration = new ProxyConfiguration({
    proxyUrls: ['http://proxy1.example.com:8080', 'http://proxy2.example.com:8080'],
});

const crawler = new CheerioCrawler({
    httpClient: new ImpitHttpClient({ browser: Browser.Chrome }),
    proxyConfiguration,
    async requestHandler({ $, request }) {
        console.log(`Scraped ${request.url}`);
    },
});
```

## How it works

Impit achieves browser impersonation at two levels:

1. **HTTP level**: Mimics browser-specific header ordering, HTTP/2 settings, and pseudo-header sequences that antibot services analyze.

2. **TLS level**: Uses a patched version of `rustls` to replicate the exact TLS ClientHello message that browsers send, including cipher suites and extensions.

This dual-layer approach makes requests appear to come from a real browser, significantly reducing blocks from bot detection systems.

## Comparison with other solutions

| Feature | got-scraping | curl-impersonate | Impit |
|---------|--------------|------------------|-------|
| TLS fingerprinting | No | Yes | Yes |
| HTTP/3 support | No | Yes | Yes |
| Native Node.js package | Yes | No (child process) | Yes |
| Windows/macOS ARM | Yes | No | Yes |
| Package size | ~10 MB | ~20 MB | ~8 MB |

**Related links**

- [Impit GitHub repository](https://github.com/apify/impit)
- [Custom HTTP Client guide](./custom-http-client)
- [Proxy Management guide](./proxy-management)
- [Avoiding blocking guide](./avoid-blocking)