Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

docs: explain session pool allocation flow (#1572)

Lukáš Křivka1f022f55f09c
diff --git a/packages/core/src/session_pool/session_pool.ts b/packages/core/src/session_pool/session_pool.ts
index 8b86ab2b0..957eea698 100644
--- a/packages/core/src/session_pool/session_pool.ts
+++ b/packages/core/src/session_pool/session_pool.ts
@@ -114,6 +114,13 @@ export interface SessionPoolOptions {
* session3.retire()
*
* ```
+ *
+ * **Default session allocation flow:*
+ * 1. Until the `SessionPool` reaches `maxPoolSize`, new sessions are created, provided to the user and added to the pool
+ * 2. Blocked/retired sessions stay in the pool but are never provided to the user
+ * 3. Once the pool is full (live plus blocked session count reaches `maxPoolSize`), a random session from the pool is provided.
+ * 4. If a blocked session would be picked, instead all blocked sessions are evicted from the pool and a new session is created and provided
+ *
* @category Scaling
*/
export class SessionPool extends EventEmitter {