🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
docs: make the contribution rules clearer
Karim shoairb9fa6b62e606
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex e972abc..3967c1d 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -63,6 +63,37 @@ Please follow these coding conventions as we do when writing code for Scrapling:> Please don’t put your name in the code you contribute; git provides enough metadata to identify the author of the code.## Development++### Getting started++1. Fork the repository and clone your fork:+ ```bash+ git clone https://github.com/<your-username>/Scrapling.git+ cd Scrapling+ git checkout dev+ ```++2. Create a virtual environment and install dependencies:+ ```bash+ python -m venv .venv+ source .venv/bin/activate # On Windows: .venv\Scripts\activate+ pip install -e ".[all]"+ pip install -r tests/requirements.txt+ ```++3. Install browser dependencies:+ ```bash+ scrapling install+ ```++4. Set up pre-commit hooks:+ ```bash+ pip install pre-commit+ pre-commit install+ ```++### Tips+Setting the scrapling logging level to `debug` makes it easier to know what's happening in the background.```pythonimport logging@@ -73,15 +104,6 @@ Bonus: You can install the beta of the upcoming update from the dev branch as fopip3 install git+https://github.com/D4Vinci/Scrapling.git@dev```-## Building Documentation-Documentation is built using [Zensical](https://zensical.org/). You can build it locally using the following commands:-```bash-pip install zensical-pip install -r docs/requirements.txt-zensical build --clean # Build the static site-zensical serve # Local preview-```-## TestsScrapling includes a comprehensive test suite that can be executed with pytest. However, first, you need to install all libraries and `pytest-plugins` listed in `tests/requirements.txt`. Then, running the tests will result in an output like this:```bash@@ -93,16 +115,34 @@ Scrapling includes a comprehensive test suite that can be executed with pytest.configfile: pytest.iniplugins: asyncio-1.2.0, anyio-4.11.0, xdist-3.8.0, httpbin-2.1.0, cov-7.0.0asyncio: mode=Mode.STRICT, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function- 10 workers [515 items]- scheduling tests via LoadScheduling-+ 10 workers [515 items]+ scheduling tests via LoadScheduling+...<shortened>...-+=============================== 271 passed in 52.68s ==============================```-Hence, we used `-n auto` in the command above to run tests in threads to increase speed.+Here, `-n auto` runs tests in parallel across multiple processes to increase speed.++**Note:** You may need to run browser tests sequentially (`DynamicFetcher`/`StealthyFetcher`) to avoid conflicts. To run non-browser tests in parallel and browser tests separately:+```bash+# Non-browser tests (parallel)+pytest tests/ -k "not (DynamicFetcher or StealthyFetcher)" -n auto++# Browser tests (sequential)+pytest tests/ -k "DynamicFetcher or StealthyFetcher"+```Bonus: You can also see the test coverage with the `pytest` plugin below```bashpytest --cov=scrapling tests/```++## Building Documentation+Documentation is built using [Zensical](https://zensical.org/). You can build it locally using the following commands:+```bash+pip install zensical+pip install -r docs/requirements.txt+zensical build --clean # Build the static site+zensical serve # Local preview+```