Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Merge branch 'develop' into feature/copywriting

Marek Trunkátadb012ea2418
diff --cc website/pages/en/index.js
index 10359eae7,ece22a03b..8be6681d5
--- a/website/pages/en/index.js
+++ b/website/pages/en/index.js
@@@ -165,11 -165,11 +165,11 @@@ const EasyCrawling = () =>
<Block background="light" gridBlockAlign="left">
{[
{
- content: 'There are three main class that you can use to start crawling web in no time. ' +
- 'Do you need to crawl just the plain HTML? Use the **blazing fast** [`CheerioCrawler`](docs/examples/cheeriocrawler).\n' +
+ content: 'There are three main class that you can use to start crawling the web in no time. ' +
- 'Need to crawl plain HTML? Use the **blazing fast** [`CheerioCrawler`](examples/cheeriocrawler).\n' +
++ 'Need to crawl plain HTML? Use the **blazing fast** [`CheerioCrawler`](docs/examples/cheeriocrawler).\n' +
'For complex websites that use **React**, **Vue** or other front-end javascript libraries and require JavaScript execution, ' +
- 'spawn a headless browser with [`PuppeteerCrawler`](examples/puppeteercrawler).\n' +
- 'To **control of all aspects** of your crawling, just use the bare bones [`BasicCrawler`](examples/basiccrawler)',
+ 'spawn a headless browser with [`PuppeteerCrawler`](docs/examples/puppeteercrawler).\n' +
+ 'To **control of all aspects** of your crawling, just use the bare bones [`BasicCrawler`](docs/examples/basiccrawler)',
image: imgUrl('chrome_scrape.gif'),
imageAlign: 'right',
title: 'Easy crawling',
@@@ -182,9 -182,9 +182,9 @@@ const PowerfulTools = () =>
<Block gridBlockAlign="left">
{[
{
- content: 'All the crawlers are automatically **scaled** based on available system resources using the [`AutoscaledPool`](docs/api/AutoscaledPool) class. ' +
+ content: 'All the crawlers are automatically **scaled** based on available system resources using the [`AutoscaledPool`](docs/api/autoscaledpool) class. ' +
'When you run your code on the [Apify Cloud](https://my.apify.com/actors), you can also take advantage of a pool of **proxies** to avoid detection. ' +
- 'For the data storage, you can use the [`Dataset`](docs/api/dataset) and [`Key-value store`](docs/api/keyvaluestore) classes.',
+ 'For data storage, you can use the [`Dataset`](docs/api/dataset) and [`Key-value store`](docs/api/keyvaluestore) classes.',
image: imgUrl('source_code.png'),
imageAlign: 'left',
title: 'Powerful tools',