🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
docs: add new api references
Karim shoairdddb179c50a2
diff --git a/docs/api-reference/proxy-rotation.md b/docs/api-reference/proxy-rotation.mdnew file mode 100644index 0000000..61a8fad--- /dev/null+++ b/docs/api-reference/proxy-rotation.md@@ -0,0 +1,18 @@+---+search:+ exclude: true+---++# Proxy Rotation++The `ProxyRotator` class provides thread-safe proxy rotation for any fetcher or session.++You can import it directly like below:++```python+from scrapling.fetchers import ProxyRotator+```++## ::: scrapling.engines.toolbelt.proxy_rotation.ProxyRotator+ handler: python+ :docstring:diff --git a/docs/api-reference/response.md b/docs/api-reference/response.mdnew file mode 100644index 0000000..26f6895--- /dev/null+++ b/docs/api-reference/response.md@@ -0,0 +1,18 @@+---+search:+ exclude: true+---++# Response Class++The `Response` class wraps HTTP responses returned by all fetchers, providing access to status, headers, body, cookies, and a `Selector` for parsing.++You can import the `Response` class like below:++```python+from scrapling.engines.toolbelt.custom import Response+```++## ::: scrapling.engines.toolbelt.custom.Response+ handler: python+ :docstring:diff --git a/docs/api-reference/spiders.md b/docs/api-reference/spiders.mdnew file mode 100644index 0000000..0a69abb--- /dev/null+++ b/docs/api-reference/spiders.md@@ -0,0 +1,42 @@+---+search:+ exclude: true+---++# Spider Classes++Here's the reference information for the spider framework classes' parameters, attributes, and methods.++You can import them directly like below:++```python+from scrapling.spiders import Spider, Request, CrawlResult, SessionManager, Response+```++## ::: scrapling.spiders.Spider+ handler: python+ :docstring:++## ::: scrapling.spiders.Request+ handler: python+ :docstring:++## Result Classes++## ::: scrapling.spiders.result.CrawlResult+ handler: python+ :docstring:++## ::: scrapling.spiders.result.CrawlStats+ handler: python+ :docstring:++## ::: scrapling.spiders.result.ItemList+ handler: python+ :docstring:++## Session Management++## ::: scrapling.spiders.session.SessionManager+ handler: python+ :docstring:diff --git a/zensical.toml b/zensical.tomlindex b19a6ab..231d82f 100644--- a/zensical.toml+++ b/zensical.toml@@ -54,7 +54,10 @@ nav = [{Selector = "api-reference/selector.md"},{Fetchers = "api-reference/fetchers.md"},{"MCP Server" = "api-reference/mcp-server.md"},- {"Custom Types" = "api-reference/custom-types.md"}+ {"Custom Types" = "api-reference/custom-types.md"},+ {Response = "api-reference/response.md"},+ {Spiders = "api-reference/spiders.md"},+ {"Proxy Rotation" = "api-reference/proxy-rotation.md"}]},{"Writing your retrieval system" = "development/adaptive_storage_system.md"},{"Using Scrapling's custom types" = "development/scrapling_custom_types.md"}