Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

docs: remove debugging code to fix build

Martin Adámek485d83d6fd2c
diff --git a/CHANGELOG.md b/CHANGELOG.md
index bc36a368f..c012b4460 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -420,12 +420,6 @@ const response = await promise;
Previously, you were able to have a browser pool that would mix Puppeteer and Playwright plugins (or even your own custom plugins if you've built any). As of this version, that is no longer allowed, and creating such a browser pool will cause an error to be thrown (it's expected that all plugins that will be used are of the same type).
-:::info Confused?
-
-As an example, this change disallows a pool to mix Puppeteer with Playwright. You can still create pools that use multiple Playwright plugins, each with a different launcher if you want!
-
-:::
-
### Handling requests outside of browser
One small feature worth mentioning is the ability to handle requests with browser crawlers outside the browser. To do that, we can use a combination of `Request.skipNavigation` and `context.sendRequest()`.
@@ -490,14 +484,14 @@ await Actor.main(async () => {
#### Events
-Apify SDK (v2) exports `Apify.events`, which is an `EventEmitter` instance. With Crawlee, the events are managed by <ApiLink to="core/class/EventManager">`EventManager`</ApiLink> class instead. We can either access it via `Actor.eventManager` getter, or use `Actor.on` and `Actor.off` shortcuts instead.
+Apify SDK (v2) exports `Apify.events`, which is an `EventEmitter` instance. With Crawlee, the events are managed by [`EventManager`](https://crawlee.dev/api/core/class/EventManager) class instead. We can either access it via `Actor.eventManager` getter, or use `Actor.on` and `Actor.off` shortcuts instead.
```diff
-Apify.events.on(...);
+Actor.on(...);
```
-> We can also get the <ApiLink to="core/class/EventManager">`EventManager`</ApiLink> instance via `Configuration.getEventManager()`.
+> We can also get the [`EventManager`](https://crawlee.dev/api/core/class/EventManager) instance via `Configuration.getEventManager()`.
In addition to the existing events, we now have an `exit` event fired when calling `Actor.exit()` (which is called at the end of `Actor.main()`). This event allows you to gracefully shut down any resources when `Actor.exit` is called.
diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js
index bd07676d1..1ee48c0d6 100644
--- a/website/docusaurus.config.js
+++ b/website/docusaurus.config.js
@@ -144,24 +144,22 @@ module.exports = {
docId: 'quick-start/quick-start',
label: 'Docs',
position: 'left',
- activeBaseRegex: 'docs/quick-start',
},
{
type: 'doc',
docId: 'examples/examples',
label: 'Examples',
position: 'left',
- activeBaseRegex: 'docs/examples',
},
{
type: 'custom-api',
to: 'core',
label: 'API',
position: 'left',
- activeBaseRegex: 'api/(?!core/changelog)',
},
{
- to: 'api/core/changelog',
+ type: 'custom-api',
+ to: 'core/changelog',
label: 'Changelog',
position: 'left',
className: 'changelog',
diff --git a/website/src/theme/NavbarItem/ComponentTypes.js b/website/src/theme/NavbarItem/ComponentTypes.js
index ff0e5058a..d871a4b22 100644
--- a/website/src/theme/NavbarItem/ComponentTypes.js
+++ b/website/src/theme/NavbarItem/ComponentTypes.js
@@ -6,15 +6,14 @@ import HtmlNavbarItem from '@theme/NavbarItem/HtmlNavbarItem';
import DocSidebarNavbarItem from '@theme/NavbarItem/DocSidebarNavbarItem';
import DocsVersionNavbarItem from '@theme/NavbarItem/DocsVersionNavbarItem';
import DocsVersionDropdownNavbarItem from '@theme/NavbarItem/DocsVersionDropdownNavbarItem';
-import React from 'react';
import { useActiveDocContext } from '@docusaurus/plugin-content-docs/client';
import { useDocsVersion, useLayoutDoc } from '@docusaurus/theme-common/internal';
import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
+import React from 'react';
-const pkg = require('../../../../packages/crawlee/package.json');
+const versions = require('../../../versions.json');
-const [v1, v2] = pkg.version.split('.');
-const stable = [v1, v2].join('.');
+const stable = versions[0];
function DocNavbarItem({
docId,
@@ -41,17 +40,16 @@ function DocNavbarItem({
function ApiNavbarItem(ctx) {
const { activeDoc, activeVersion } = useActiveDocContext();
- const version = useDocsVersion();
+ let version = {};
+
+ try {
+ // eslint-disable-next-line react-hooks/rules-of-hooks
+ version = useDocsVersion();
+ } catch {
+ version.version = stable;
+ }
+
const { siteConfig } = useDocusaurusContext();
- console.log(ctx, activeDoc, activeVersion, activeVersion?.path.replace(/^\/docs/, '/api'));
- console.log(window.location.href, version);
- // const { activeDoc } = useActiveDocContext(docsPluginId);
- // const doc = useLayoutDoc(docId, docsPluginId);
- // console.log(activeDoc, doc, (!!activeDoc?.sidebar && activeDoc.sidebar === doc.sidebar));
- // Draft items are not displayed in the navbar.
- // if (doc === null) {
- // return null;
- // }
if (siteConfig.presets[0][1].docs.disableVersioning || version.version === stable) {
return (
diff --git a/website/static/js/custom.js b/website/static/js/custom.js
index 10f43f191..bdb2a7fc4 100644
--- a/website/static/js/custom.js
+++ b/website/static/js/custom.js
@@ -25,24 +25,6 @@ function load() {
});
el.classList.add('api-version-bound');
}
-
- // const navbarLinks = document.querySelectorAll('.navbar a.navbar__link');
- //
- // for (const el of [...navbarLinks].slice(1, 3)) {
- // if (el.classList.contains('api-version-bound')) {
- // continue;
- // }
- //
- // const url = new URL(el.href);
- // const parts = url.pathname.split('/');
- // parts.splice(2, 0, 'next');
- // url.pathname = parts.join('/');
- // el.href = url.href;
- // el.classList.add('api-version-bound');
- // el.addEventListener('click', (e) => {
- // e.stopPropagation();
- // });
- // }
}
setInterval(() => {