Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

803 B
Raw Blame
import { expect, getActorTestDir, initialize, runActor, skipTest } from '../tools.mjs';

// The actor spins up an HTTP server on 127.0.0.1; that works inside the
// in-process LOCAL/MEMORY worker but is unreachable from the Apify platform
// container, so the run never finishes. Base-href handling is pure parsing
// logic in @crawlee/utils that doesn't depend on the storage backend, so
// LOCAL+MEMORY coverage is sufficient.
if (process.env.STORAGE_IMPLEMENTATION === 'PLATFORM') {
    await skipTest('localhost fixture is not reachable from the Apify platform');
}

const testActorDirname = getActorTestDir(import.meta.url);
await initialize(testActorDirname);

const { datasetItems } = await runActor(testActorDirname);

await expect(datasetItems.length === 4, `Enqueueing respects <base href>`);