Skip to content

Bulk asynchronous downloader for multiple file hosts

docs: changelog and wiki updates for v8.1.0 release (#1346)

NTFSvolume9a6a09ff8d48
diff --git a/CHANGELOG.md b/CHANGELOG.md
index ed6a1f3b4..6788fe39f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -22,6 +22,28 @@ All notable changes to this project will be documented here. For more details, v
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+## [8.1.0] - 2025-09-27
+
+### Added
+
+- imglike support
+- Support for channel photos (xVideos)
+
+### Changed
+
+- Remove SimpCity disclaimer on first time startup
+- Tiktok now has a hardcoded rate limit of 0.5 req/s
+
+### Fixed
+
+- `str object cannot be interpreted as an integer` error when using any retry option
+- Date parsing (pixeldrain)
+- Date parsing (bunkr)
+- Channel's pagination (xVideos)
+- Http errors being reported as `Not Implemented` (Pixeldrain, MediaFire, GoFile)
+- Http errors causing CDL to crash while downloading galleries (xVideos)
+- CDL getting stuck after 100 TCP connections
+
## [8.0.0] - 2025-09-23
### Added
@@ -41,7 +63,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Support for `pd.cybar.xyz` URLs to bypass pixeldrain daily limit
- The `--separate-post` and `--separate-post-format` options now work for tiktok posts
- CDL can now download private TikTok posts when using cookies
-- New config option to download tiktok videos in original quality: <https://script-ware.gitbook.io/cyberdrop-dl/reference/configuration-options/settings/download_options#download-tiktok-src-quality-videos>
+- New config option to download tiktok videos in original quality: <https://script-ware.gitbook.io/cyberdrop-dl/reference/cli-arguments#download-tiktok-src-quality-videos>
- New `--max-thread-folder-depth` config option: <https://script-ware.gitbook.io/cyberdrop-dl/reference/configuration-options/settings/download_options#maximum_thread_folder_depth>
- New `--ignore-coomer-post-content` config option: <https://script-ware.gitbook.io/cyberdrop-dl/reference/configuration-options/settings/ignore_options#ignore_coomer_post_content>
- The `--ignore-coomer-ads` config option now affects all Kemono based crawlers (Coomer, Kemono & Nekohouse)
diff --git a/cyberdrop_dl/crawlers/_chevereto.py b/cyberdrop_dl/crawlers/_chevereto.py
index e7dc74cdf..0be522f85 100644
--- a/cyberdrop_dl/crawlers/_chevereto.py
+++ b/cyberdrop_dl/crawlers/_chevereto.py
@@ -37,6 +37,7 @@ class CheveretoCrawler(Crawler, is_generic=True):
"/album/<id>",
"/album/<name>.<id>",
),
+ "Category": "/category/<name>",
"Image": (
"/img/<id>",
"/img/<name>.<id>",
@@ -97,18 +98,26 @@ class CheveretoCrawler(Crawler, is_generic=True):
scrape_item.setup_as_profile(title)
self._process_page(scrape_item, soup)
+ async def _get_final_album_url(self, url: AbsoluteHttpURL) -> AbsoluteHttpURL:
+ if "category" in url.parts:
+ return url
+
+ # We need the full URL (aka "/<name>.<id>") to fetch sub albums
+ if "." not in url.name:
+ url = await self._get_redirect_url(url)
+
+ # The first redirect may have only added a trailing slash, try again
+ if not url.name and "." not in url.parts[-2]:
+ url = await self._get_redirect_url(url)
+
+ return url
+
@error_handling_wrapper
async def album(self, scrape_item: ScrapeItem, album_id: str) -> None:
results = await self.get_album_results(album_id)
title: str = ""
- # We need the full URL (aka "/<name>.<id>") to fetch sub albums
- if "." not in scrape_item.url.name:
- scrape_item.url = await self._get_redirect_url(scrape_item.url)
-
- # The first redirect may have only added a trailing slash, try again
- if not scrape_item.url.name and "." not in scrape_item.url.parts[-2]:
- scrape_item.url = await self._get_redirect_url(scrape_item.url)
+ scrape_item.url = await self._get_final_album_url(scrape_item.url)
async for soup in self.web_pager(_sort_by_new(scrape_item.url), trim=False):
if not title:
@@ -116,7 +125,8 @@ class CheveretoCrawler(Crawler, is_generic=True):
await self._unlock_password_protected_album(scrape_item)
return await self.album(scrape_item, album_id)
- title = open_graph.get_title(soup) or open_graph.get("description", soup)
+ title = open_graph.get_title(soup) or open_graph.get("description", soup) or ""
+ assert title
title = self.create_title(title, album_id)
scrape_item.setup_as_album(title, album_id=album_id)
self._process_page(scrape_item, soup, results)
diff --git a/docs/reference/cli-arguments.md b/docs/reference/cli-arguments.md
index 3c088fbaf..8b0bfa168 100644
--- a/docs/reference/cli-arguments.md
+++ b/docs/reference/cli-arguments.md
@@ -117,13 +117,19 @@ Download TikTok audios from posts and save them as separate files
| ---------- | ------- | ------------ |
| `BoolFlag` | `False` | `store_true` |
-Download TikTok videos in source quality.
+By default, CDL will download the "optimized for streaming" version of tiktok videos. Setting this option to `True` will download videos in original (source) quality.
+
+`_original` will be added as a prefix to their filename.
{% hint style="warning" %}
-Enabling this option will make downloads several orders of magnitude slower.
+This will make video downloads several times slower
When it is set to `False` (the default) CDL can download 50 videos with a single request.
When it is set to `True` , CDL needs to make at least 3 requests _per_ video to download them.
+
+There's also a daily limit of the API CDL uses: 5000 requests per day
+
+Setting this option to `True` will consume the daily limit faster
{% endhint %}
### `max-items-retry`
@@ -266,9 +272,9 @@ media_duration_limits:
--minimum-audio-duration MINIMUM_AUDIO_DURATION
files:
- -d DOWNLOAD_FOLDER, --download-folder DOWNLOAD_FOLDER
+ -d, --download-folder DOWNLOAD_FOLDER
-j, --dump-json, --no-dump-json
- -i INPUT_FILE, --input-file INPUT_FILE
+ -i, --input-file INPUT_FILE
--save-pages-html, --no-save-pages-html
ignore_options:
diff --git a/docs/reference/supported-websites.md b/docs/reference/supported-websites.md
index d5237ccca..1f9ff28ce 100644
--- a/docs/reference/supported-websites.md
+++ b/docs/reference/supported-websites.md
@@ -30,7 +30,7 @@ Supported domains via Real-Debrid include `mega.nz`, `rapidgator`, `google drive
## Supported sites
-List of sites supported by cyberdrop-dl-patched as of version 8.0.0
+List of sites supported by cyberdrop-dl-patched as of version 8.1.0
| Site | Primary URL | Supported Domains | Supported Paths |
|---------------------------|---------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
@@ -49,7 +49,7 @@ List of sites supported by cyberdrop-dl-patched as of version 8.0.0
| Camwhores.tv | https://www.camwhores.tv | camwhores.tv | Categories: <br> - `/categories/...`<br>Members: <br> - `/members/<member_id>`<br>Search: <br> - `/search/...`<br>Tags: <br> - `/tags/...`<br>Videos: <br> - `/videos/...`<br> |
| Catbox | https://catbox.moe | files.catbox.moe<br>litter.catbox.moe | Direct Links<br> |
| CelebForum | https://celebforum.to | celebforum.* | Attachments: <br> - `/(attachments\|data\|uploads)/...`<br>Threads: <br> - `/(thread\|topic\|tema\|threads\|topics\|temas)/<thread_name_and_id>`<br> - `/goto/<post_id>`<br> - `/posts/<post_id>`<br><br><br>**NOTES**<br> - base crawler: Xenforo<br> |
-| Chevereto | ::GENERIC CRAWLER:: | | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br>Video: <br> - `/video/<id>`<br> - `/video/<name>.<id>`<br> - `/videos/<id>`<br> - `/videos/<name>.<id>`<br> |
+| Chevereto | ::GENERIC CRAWLER:: | | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Category: <br> - `/category/<name>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br>Video: <br> - `/video/<id>`<br> - `/video/<name>.<id>`<br> - `/videos/<id>`<br> - `/videos/<name>.<id>`<br> |
| CloudflareStream | https://cloudflarestream.com | cloudflarestream.com<br>videodelivery.net | Public Video: <br> - `/<video_uid>`<br> - `/<video_uid>/iframe`<br> - `/<video_uid>/watch`<br> - `/embed/___.js?video=<video_uid>`<br>Restricted Access Video: <br> - `/<jwt_access_token>`<br> - `/<jwt_access_token>/iframe`<br> - `/<jwt_access_token>/watch`<br> - `/embed/___.js?video=<jwt_access_token>`<br> |
| Coomer | https://coomer.st | coomer.party<br>coomer.st<br>coomer.su | Direct links: <br> - `/data/...`<br> - `/thumbnail/...`<br>Favorites: <br> - `/favorites/<user_id>`<br>Individual Post: <br> - `/<service>/user/<user_id>/post/<post_id>`<br>Model: <br> - `/<service>/user/<user_id>`<br>Search: <br> - `/search?q=...`<br> |
| Cyberdrop | https://cyberdrop.me | cyberdrop.* | Album: <br> - `/a/...`<br>Direct Links<br>File: <br> - `/f/...`<br> |
@@ -78,14 +78,15 @@ List of sites supported by cyberdrop-dl-patched as of version 8.0.0
| HotPic | https://hotpic.cc | 2385290.xyz<br>hotpic.* | Album: <br> - `/album/...`<br>Image: <br> - `/i/...`<br> |
| Iceyfile | https://iceyfile.com | iceyfile.* | Files: <br> - `/<file_id>`<br> - `/<file_id>/<file_name>`<br>Public Folders: <br> - `/folder/<folder_id>`<br> - `/folder/<folder_id>/<folder_name>`<br>Shared folders: <br> - `/shared/<share_key>`<br> |
| ImageBam | https://www.imagebam.com | imagebam.* | Gallery: <br> - `/gallery/...`<br>Gallery or Image: <br> - `/view/...`<br>Image: <br> - `/image/...`<br>Thumbnails: <br> - `thumbs<x>.imagebam.com/...`<br> |
-| ImagePond | https://imagepond.net | imagepond.net | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br>Video: <br> - `/video/<id>`<br> - `/video/<name>.<id>`<br> - `/videos/<id>`<br> - `/videos/<name>.<id>`<br> |
+| ImagePond | https://imagepond.net | imagepond.net | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Category: <br> - `/category/<name>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br>Video: <br> - `/video/<id>`<br> - `/video/<name>.<id>`<br> - `/videos/<id>`<br> - `/videos/<name>.<id>`<br> |
| ImgBB | https://ibb.co | ibb.co<br>imgbb.co | Album: <br> - `/album/...`<br>Image: <br> - `/...`<br> |
| ImgBox | https://imgbox.com | imgbox.* | Album: <br> - `/g/...`<br>Direct Links<br>Image: <br> - `/...`<br> |
+| ImgLike | https://imglike.com | imglike.com | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Category: <br> - `/category/<name>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br>Video: <br> - `/video/<id>`<br> - `/video/<name>.<id>`<br> - `/videos/<id>`<br> - `/videos/<name>.<id>`<br> |
| Imgur | https://imgur.com | imgur.* | Album: <br> - `/a/...`<br>Direct Links<br>Gallery: <br> - `/gallery/...`<br>Image: <br> - `/...`<br> |
| Imx.to | https://imx.to | imx.to | Image: <br> - `/i/...`<br>Thumbnail: <br> - `/t/...`<br> |
| IncestFlix | https://www.incestflix.com | incestflix.* | Tag: <br> - `/tag/...`<br>Video: <br> - `/watch/...`<br> |
| InfluencerBitches | https://influencerbitches.com | influencerbitches.* | Model: <br> - `/model/...`<br> |
-| JPG5 | https://jpg6.su | host.church<br>jpeg.pet<br>jpg.church<br>jpg.fish<br>jpg.fishing<br>jpg.homes<br>jpg.pet<br>jpg1.su<br>jpg2.su<br>jpg3.su<br>jpg4.su<br>jpg5.su<br>jpg6.su<br>selti-delivery.ru | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br> |
+| JPG5 | https://jpg6.su | host.church<br>jpeg.pet<br>jpg.church<br>jpg.fish<br>jpg.fishing<br>jpg.homes<br>jpg.pet<br>jpg1.su<br>jpg2.su<br>jpg3.su<br>jpg4.su<br>jpg5.su<br>jpg6.su<br>selti-delivery.ru | Album: <br> - `/a/<id>`<br> - `/a/<name>.<id>`<br> - `/album/<id>`<br> - `/album/<name>.<id>`<br>Category: <br> - `/category/<name>`<br>Direct Links<br>Image: <br> - `/image/<id>`<br> - `/image/<name>.<id>`<br> - `/img/<id>`<br> - `/img/<name>.<id>`<br>Profiles: <br> - `/<user_name>`<br> |
| Kemono | https://kemono.cr | kemono.cr<br>kemono.party<br>kemono.su | Direct links: <br> - `/data/...`<br> - `/thumbnail/...`<br>Discord Server: <br> - `/discord/<server_id>`<br>Discord Server Channel: <br> - `/discord/server/<server_id>/<channel_id>#...`<br>Favorites: <br> - `/favorites/<user_id>`<br>Individual Post: <br> - `/<service>/user/<user_id>/post/<post_id>`<br>Model: <br> - `/<service>/user/<user_id>`<br>Search: <br> - `/search?q=...`<br> |
| LeakedModels | https://leakedmodels.com/forum | leakedmodels.* | Attachments: <br> - `/(attachments\|data\|uploads)/...`<br>Threads: <br> - `/(thread\|topic\|tema\|threads\|topics\|temas)/<thread_name_and_id>`<br> - `/goto/<post_id>`<br> - `/posts/<post_id>`<br><br><br>**NOTES**<br> - base crawler: Xenforo<br> |
| LeakedZone | https://leakedzone.com | leakedzone.* | Model: <br> - `/<model_id>`<br>Video: <br> - `/<model_id>/video/<video_id>`<br> |