🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
Updating contribution rules
Karim shoair48da61d1ee85
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex e896ab7..ec5342c 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -2,7 +2,7 @@Thank you for your interest in contributing to Scrapling!-Everybody is invited and welcome to contribute to Scrapling.+Everyone is welcome to contribute to Scrapling.Minor changes are more likely to be included promptly. Adding unit tests for new features or test cases for bugs you've fixed helps us ensure that the Pull Request (PR) is acceptable.@@ -11,20 +11,22 @@ There are many ways to contribute to Scrapling. Here are some of them:- Report bugs and request features using the [GitHub issues](https://github.com/D4Vinci/Scrapling/issues). Please follow the issue template to help us resolve your issue quickly.- Blog about Scrapling. Tell the world how you’re using Scrapling. This will help newcomers with more examples and increase the Scrapling project's visibility.- Join the [Discord community](https://discord.gg/EMgGbDceNQ) and share your ideas on how to improve Scrapling. We’re always open to suggestions.-- If you are not a developer, perhaps you would like to help with translating the [documentation](https://github.com/D4Vinci/Scrapling/tree/dev/docs)?+- If you are not a developer, perhaps you would like to help with reviewing the translation of the [documentation](https://github.com/D4Vinci/Scrapling/tree/dev/docs)? We are not aiming to add new languages right now.## Making a Pull RequestTo ensure that your PR gets accepted, please make sure that your PR is based on the latest changes from the dev branch and that it satisfies the following requirements:-- **The PR must be made against the [**dev**](https://github.com/D4Vinci/Scrapling/tree/dev) branch of Scrapling. Any PR made against the main branch will be rejected.**-- **The code should be passing all available tests. We use tox with GitHub's CI to run the current tests on all supported Python versions for every code-related commit.**-- **The code should be passing all code quality checks like `mypy` and `pyright`. We are using GitHub's CI to enforce code style checks as well.**+- **The PR must be made against the [**dev**](https://github.com/D4Vinci/Scrapling/tree/dev) branch of Scrapling. We will reject any PR made against the main branch.**+- **The code should pass all available tests. We use tox with GitHub's CI to run the current tests on all supported Python versions for every code-related commit.**+- **The code should pass all code quality checks like `mypy` and `pyright`. We are using GitHub's CI to enforce code style checks as well.**- **Read our [AI Contribution Policy](AI_POLICY.md) before submitting. Any contribution made with AI assistance must disclose that in the PR or issue.**- **Make your changes, keep the code clean with an explanation of any part that might be vague, and remember to create a separate virtual environment for this project.**- If you are adding a new feature, please add tests for it.- If you are fixing a bug, please add code with the PR that reproduces the bug.- Spider platform templates are welcome when the platform exposes a uniform structure across many independent domains; single-site scrapers never belong in the library.- Please follow the rules and coding style rules we explain below.+- Make sure there's already no other PR that solves the issue unless you think your PR is better. In that case, you should state why it's better.+- Try to avoid changing our design choices or open a discussion about it first. A general rule of thumb: too much hand-holding is bad, since most of our users are developers.## Finding work@@ -32,15 +34,15 @@ To ensure that your PR gets accepted, please make sure that your PR is based onIf you have decided to make a contribution to Scrapling, but you do not know what to contribute, here are some ways to find pending work:- Check out the [contribution](https://github.com/D4Vinci/Scrapling/contribute) GitHub page, which lists open issues tagged as `good first issue`. These issues provide a good starting point.-- There are also the [help wanted](https://github.com/D4Vinci/Scrapling/issues?q=is%3Aissue%20label%3A%22help%20wanted%22%20state%3Aopen) issues, but know that some may require familiarity with the Scrapling code base first. You can also target any other issue, provided it is not tagged as `invalid`, `wontfix`, or similar tags.+- There are also the [help wanted](https://github.com/D4Vinci/Scrapling/issues?q=is%3Aissue%20label%3A%22help%20wanted%22%20state%3Aopen) issues, but know that some may require familiarity with the Scrapling codebase first. You can also target any other issue, as long as it isn't tagged `invalid`, `wontfix`, or similar.- If you enjoy writing automated tests, you can work on increasing our test coverage. Currently, the test coverage is around 90–92%.- Join the [Discord community](https://discord.gg/EMgGbDceNQ) and ask questions in the `#help` channel.## Coding stylePlease follow these coding conventions as we do when writing code for Scrapling:- We use [pre-commit](https://pre-commit.com/) to automatically address simple code issues before every commit, so please install it and run `pre-commit install` to set it up. This will install hooks to run [ruff](https://docs.astral.sh/ruff/), [bandit](https://github.com/PyCQA/bandit), and [vermin](https://github.com/netromdk/vermin) on every commit. We are currently using a workflow to automatically run these tools on every PR, so if your code doesn't pass these checks, the PR will be rejected.-- We use type hints for better code clarity and [pyright](https://github.com/microsoft/pyright)/[mypy](https://github.com/python/mypy) for static type checking. If your code isn't acceptable by those tools, your PR won't pass the code quality rule.-- We use the conventional commit messages format as [here](https://gist.github.com/qoomon/5dfcdf8eec66a051ecd85625518cfd13#types), so for example, we use the following prefixes for commit messages:+- We use type hints for better code clarity and [pyright](https://github.com/microsoft/pyright)/[mypy](https://github.com/python/mypy) for static type checking. If your code isn't acceptable to those tools, your PR won't pass the code quality rule.+- We use the conventional commit message format as [here](https://gist.github.com/qoomon/5dfcdf8eec66a051ecd85625518cfd13#types), so for example, we use the following prefixes for commit messages:| Prefix | When to use it ||-------------|--------------------------|@@ -51,7 +53,7 @@ Please follow these coding conventions as we do when writing code for Scrapling:| `refactor:` | Code refactoring || `chore:` | Maintenance tasks |- Then include the details of the change in the commit message body/description.+ Then include the change details in the commit message body/description.Example:```@@ -107,7 +109,7 @@ pip3 install git+https://github.com/D4Vinci/Scrapling.git@dev```## Tests-Scrapling includes a comprehensive test suite that can be executed with pytest. However, first, you need to install all libraries and `pytest-plugins` listed in `tests/requirements.txt`. Then, running the tests will result in an output like this:+Scrapling includes a comprehensive test suite you can run with pytest. First, install all libraries and `pytest-plugins` listed in `tests/requirements.txt`. Then, running the tests will result in an output like this:```bash$ pytest tests -n auto=============================== test session starts ===============================