Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

docs: add meta title and description (#2382)

Saurav Jain2fd0d8f8c987
diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js
index f1cb8899a..17051d1dd 100644
--- a/website/docusaurus.config.js
+++ b/website/docusaurus.config.js
@@ -71,6 +71,10 @@ module.exports = {
return `https://github.com/apify/crawlee/edit/master/website/${doc.versionDocsDirPath}/${doc.docPath}`;
},
},
+ blog: {
+ blogTitle: 'Crawlee Blog - learn how to build better scrapers',
+ blogDescription: 'Guides and tutorials on using Crawlee, the most reliable open-source web scraping and browser automation library for JavaScript and Node.js developers.',
+ },
theme: {
customCss: '/src/css/custom.css',
},