🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Python aiai-scrapingautomationcrawlercrawlingcrawling-pythondatadata-extractionmcpmcp-serverplaywrightpythonscrapingselectorsstealthweb-scraperweb-scrapingweb-scraping-pythonwebscrapingxpath
build: pump up version, browsers and deps
Karim shoair4ced6e4ac268
diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.ymlindex f82605d..930d0a0 100644--- a/.github/workflows/tests.yml+++ b/.github/workflows/tests.yml@@ -73,7 +73,7 @@ jobs:- name: Install all browsers dependenciesrun: |python3 -m pip install --upgrade pip- python3 -m pip install playwright==1.59.0 patchright==1.59.1+ python3 -m pip install playwright==1.60.0 patchright==1.60.1- name: Get Playwright versionid: playwright-versiondiff --git a/agent-skill/Scrapling-Skill.zip b/agent-skill/Scrapling-Skill.zipindex 22fbb32..1a8d15b 100644Binary files a/agent-skill/Scrapling-Skill.zip and b/agent-skill/Scrapling-Skill.zip differdiff --git a/agent-skill/Scrapling-Skill/SKILL.md b/agent-skill/Scrapling-Skill/SKILL.mdindex 27c31c0..ab57268 100644--- a/agent-skill/Scrapling-Skill/SKILL.md+++ b/agent-skill/Scrapling-Skill/SKILL.md@@ -1,7 +1,7 @@---name: scrapling-officialdescription: Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; or write spiders.-version: "0.4.8"+version: "0.4.9"license: Complete terms in LICENSE.txtmetadata:homepage: "https://scrapling.readthedocs.io/en/latest/index.html"@@ -40,7 +40,7 @@ Blazing fast crawls with real-time stats and streaming. Built by Web Scrapers foCreate a virtual Python environment through any way available, like `venv`, then inside the environment do:-`pip install "scrapling[all]>=0.4.8"`+`pip install "scrapling[all]>=0.4.9"`Then do this to download all the browsers' dependencies:diff --git a/agent-skill/Scrapling-Skill/examples/README.md b/agent-skill/Scrapling-Skill/examples/README.mdindex d0f9a2b..85de486 100644--- a/agent-skill/Scrapling-Skill/examples/README.md+++ b/agent-skill/Scrapling-Skill/examples/README.md@@ -9,7 +9,7 @@ All examples collect **all 100 quotes across 10 pages**.Make sure Scrapling is installed:```bash-pip install "scrapling[all]>=0.4.8"+pip install "scrapling[all]>=0.4.9"scrapling install --force```diff --git a/pyproject.toml b/pyproject.tomlindex 8cb4768..9c390c0 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"[project]name = "scrapling"# Static version instead of a dynamic version so we can get better layer caching while building docker, check the docker file to understand-version = "0.4.8"+version = "0.4.9"description = "Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!"readme = {file = "README.md", content-type = "text/markdown"}license = {file = "LICENSE"}@@ -61,7 +61,7 @@ classifiers = ["Typing :: Typed",]dependencies = [- "lxml>=6.1.0",+ "lxml>=6.1.1","cssselect>=1.4.0","orjson>=3.11.8","tld>=0.13.2",@@ -73,8 +73,8 @@ dependencies = [fetchers = ["click>=8.3.0","curl_cffi>=0.15.0",- "playwright==1.59.0",- "patchright==1.59.1",+ "playwright==1.60.0",+ "patchright==1.60.1","browserforge>=1.2.4","apify-fingerprint-datapoints>=0.13.0","msgspec>=0.21.1",diff --git a/scrapling/__init__.py b/scrapling/__init__.pyindex 3420e25..a5ff5eb 100644--- a/scrapling/__init__.py+++ b/scrapling/__init__.py@@ -1,5 +1,5 @@__author__ = "Karim Shoair ([email protected])"-__version__ = "0.4.8"+__version__ = "0.4.9"__copyright__ = "Copyright (c) 2024 Karim Shoair"from typing import Any, TYPE_CHECKINGdiff --git a/scrapling/engines/toolbelt/fingerprints.py b/scrapling/engines/toolbelt/fingerprints.pyindex f4fdfc1..0c02002 100644--- a/scrapling/engines/toolbelt/fingerprints.py+++ b/scrapling/engines/toolbelt/fingerprints.py@@ -13,8 +13,8 @@ from scrapling.core._types import Dict, Literal, Tuple__OS_NAME__ = platform_system()OSName = Literal["linux", "macos", "windows"]# Current versions hardcoded for now (Playwright doesn't allow to know the version of a browser without launching it)-chromium_version = 147-chrome_version = 147+chromium_version = 148+chrome_version = 148@lru_cache(1, typed=True)diff --git a/server.json b/server.jsonindex e4e813e..deccdda 100644--- a/server.json+++ b/server.json@@ -14,12 +14,12 @@"mimeType": "image/png"}],- "version": "0.4.8",+ "version": "0.4.9","packages": [{"registryType": "pypi","identifier": "scrapling",- "version": "0.4.8",+ "version": "0.4.9","runtimeHint": "uvx","packageArguments": [{diff --git a/setup.cfg b/setup.cfgindex 1c967c8..4e3d8b6 100644--- a/setup.cfg+++ b/setup.cfg@@ -1,6 +1,6 @@[metadata]name = scrapling-version = 0.4.8+version = 0.4.9author = Karim Shoairauthor_email = [email protected]description = Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!diff --git a/tests/requirements.txt b/tests/requirements.txtindex 83aa3dc..7c9c3ff 100644--- a/tests/requirements.txt+++ b/tests/requirements.txt@@ -1,6 +1,6 @@pytest>=2.8.0,<9pytest-cov-playwright==1.59.0+playwright==1.60.0werkzeug<3.0.0pytest-httpbin==2.1.0pytest-asynciodiff --git a/tox.ini b/tox.iniindex e740705..799ceae 100644--- a/tox.ini+++ b/tox.ini@@ -10,8 +10,8 @@ envlist = pre-commit,py{310,311,312,313}usedevelop = Truechangedir = testsdeps =- playwright==1.59.0- patchright==1.59.1+ playwright==1.60.0+ patchright==1.60.1-r{toxinidir}/tests/requirements.txtextras = ai,shellcommands =