Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Better description in package.json and siteConfig.js

Jan Curn5b703a0c7d0b
diff --git a/package.json b/package.json
index 5080b921d..af11dc845 100644
--- a/package.json
+++ b/package.json
@@ -1,7 +1,7 @@
{
"name": "apify",
"version": "0.8.17",
- "description": "The scalable web crawling and scraping library for JavaScript",
+ "description": "The scalable web crawling and scraping library for JavaScript. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.",
"engines": {
"node": ">=8.0.0"
},
diff --git a/website/siteConfig.js b/website/siteConfig.js
index cd4a4282e..36f48d612 100644
--- a/website/siteConfig.js
+++ b/website/siteConfig.js
@@ -24,7 +24,10 @@ const repoUrl = 'https://github.com/apifytech/apify-js';
const siteConfig = {
title: 'Apify SDK', // Title for your website.
- tagline: 'The scalable web crawling and scraping library for JavaScript.',
+ // This is also used as page meta description for SEO, so write it carefully.
+ // TODO: Take this from package.json
+ // eslint-disable-next-line max-len
+ tagline: 'The scalable web crawling and scraping library for JavaScript. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.',
url: 'https://sdk.apify.com', // Your website URL
cname: 'sdk.apify.com',
baseUrl: '/', // Base URL for your project */