🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
docs: use a better looking architecture image instead of mermaid
Karim shoair1c2c7c55bf7d
diff --git a/docs/assets/spider_architecture.png b/docs/assets/spider_architecture.pngnew file mode 100644index 0000000..7368b7bBinary files /dev/null and b/docs/assets/spider_architecture.png differdiff --git a/docs/spiders/architecture.md b/docs/spiders/architecture.mdindex a4d4bb1..249f91c 100644--- a/docs/spiders/architecture.md+++ b/docs/spiders/architecture.md@@ -13,39 +13,7 @@ If you're familiar with Scrapy, you'll feel right at home. If not, don't worryThe diagram below shows how data flows through the spider system when a crawl is running:-```mermaid-graph TB- Spider["Spider"]- Scheduler["Scheduler"]- Engine["Crawler Engine"]- SessionMgr["Session Manager"]- Output["Output"]- Checkpoint["Checkpoint"]-- Spider -- "1. Initial Requests" --> Scheduler- Scheduler -- "2. Next Request" --> Engine- Engine -- "3. Fetch" --> SessionMgr- SessionMgr -- "4. Response" --> Engine- Engine -- "5. Response to callback" --> Spider- Spider -- "6. New Requests" --> Scheduler- Spider -- "7. Items" --> Output- Engine -. "Save" .-> Checkpoint- Checkpoint -. "Resume" .-> Scheduler-- classDef spiderClass fill:#7c4dff,stroke:#333,color:#fff- classDef engineClass fill:#00897b,stroke:#333,color:#fff- classDef schedulerClass fill:#1565c0,stroke:#333,color:#fff- classDef sessionClass fill:#ef6c00,stroke:#333,color:#fff- classDef outputClass fill:#2e7d32,stroke:#333,color:#fff- classDef checkpointClass fill:#6d4c41,stroke:#333,color:#fff-- class Spider spiderClass- class Engine engineClass- class Scheduler schedulerClass- class SessionMgr sessionClass- class Output outputClass- class Checkpoint checkpointClass-```+<img src="../assets/spider_architecture.png" title="Spider architecture diagram by @TrueSkills" alt="Spider architecture diagram by @TrueSkills" style="width: 70%;"/>Here's what happens step by step when you run a spider without many details: