🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
Initial commit
Karim shoairab0e2779f9c7
diff --git a/.gitignore b/.gitignorenew file mode 100644index 0000000..82f9275--- /dev/null+++ b/.gitignore@@ -0,0 +1,162 @@+# Byte-compiled / optimized / DLL files+__pycache__/+*.py[cod]+*$py.class++# C extensions+*.so++# Distribution / packaging+.Python+build/+develop-eggs/+dist/+downloads/+eggs/+.eggs/+lib/+lib64/+parts/+sdist/+var/+wheels/+share/python-wheels/+*.egg-info/+.installed.cfg+*.egg+MANIFEST++# PyInstaller+# Usually these files are written by a python script from a template+# before PyInstaller builds the exe, so as to inject date/other infos into it.+*.manifest+*.spec++# Installer logs+pip-log.txt+pip-delete-this-directory.txt++# Unit test / coverage reports+htmlcov/+.tox/+.nox/+.coverage+.coverage.*+.cache+nosetests.xml+coverage.xml+*.cover+*.py,cover+.hypothesis/+.pytest_cache/+cover/++# Translations+*.mo+*.pot++# Django stuff:+*.log+local_settings.py+db.sqlite3+db.sqlite3-journal++# Flask stuff:+instance/+.webassets-cache++# Scrapy stuff:+.scrapy++# Sphinx documentation+docs/_build/++# PyBuilder+.pybuilder/+target/++# Jupyter Notebook+.ipynb_checkpoints++# IPython+profile_default/+ipython_config.py++# pyenv+# For a library or package, you might want to ignore these files since the code is+# intended to run in multiple environments; otherwise, check them in:+# .python-version++# pipenv+# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.+# However, in case of collaboration, if having platform-specific dependencies or dependencies+# having no cross-platform support, pipenv may install dependencies that don't work, or not+# install all needed dependencies.+#Pipfile.lock++# poetry+# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.+# This is especially recommended for binary packages to ensure reproducibility, and is more+# commonly ignored for libraries.+# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control+#poetry.lock++# pdm+# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.+#pdm.lock+# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it+# in version control.+# https://pdm.fming.dev/latest/usage/project/#working-with-version-control+.pdm.toml+.pdm-python+.pdm-build/++# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm+__pypackages__/++# Celery stuff+celerybeat-schedule+celerybeat.pid++# SageMath parsed files+*.sage.py++# Environments+.env+.venv+env/+venv/+ENV/+env.bak/+venv.bak/++# Spyder project settings+.spyderproject+.spyproject++# Rope project settings+.ropeproject++# mkdocs documentation+/site++# mypy+.mypy_cache/+.dmypy.json+dmypy.json++# Pyre type checker+.pyre/++# pytype static type analyzer+.pytype/++# Cython debug symbols+cython_debug/++# PyCharm+# JetBrains specific template is maintained in a separate JetBrains.gitignore that can+# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore+# and can be added to the global gitignore or merged into this file. For a more nuclear+# option (not recommended) you can uncomment the following to ignore the entire idea folder.+#.idea/diff --git a/LICENSE b/LICENSEnew file mode 100644index 0000000..41615aa--- /dev/null+++ b/LICENSE@@ -0,0 +1,28 @@+BSD 3-Clause License++Copyright (c) 2024, Karim shoair++Redistribution and use in source and binary forms, with or without+modification, are permitted provided that the following conditions are met:++1. Redistributions of source code must retain the above copyright notice, this+ list of conditions and the following disclaimer.++2. Redistributions in binary form must reproduce the above copyright notice,+ this list of conditions and the following disclaimer in the documentation+ and/or other materials provided with the distribution.++3. Neither the name of the copyright holder nor the names of its+ contributors may be used to endorse or promote products derived from+ this software without specific prior written permission.++THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.