Bulk asynchronous downloader for multiple file hosts
description: These are options for enable/disable hashing and auto dupe deletion
cyberdrop-dl maintains an internal database of all downloaded files, indexed by their hashes.
This can be used to automatically delete newly downloaded files if they were downloaded before. To enable auto dupe cleanup:
- Set
hashingtoIN_PLACEorPOST_DOWNLOAD - Set
dedupetotrue
mode
| Type | Default |
|---|---|
OFF,IN_PLACE or POST_DOWNLOAD |
IN_PLACE |
OFF: Do not compute any checksum of downloaded filesIN_PLACE: compute checksums after each download finishesPOST_DOWNLOAD: Wait for all downloads to finish and compute checksums for all files at once
hashing:
mode: in_place
algorithms
| Type | Default |
|---|---|
list of xxh128, md5 or sha256 |
[xxh128, md5,sha256] |
List of checksum algorithms to compute for new downloads. xxh128 is used for file deduplication and will always be automatically added by CDL if missing from the list.
The additional algorithms are used to skip downloads before they even begin. Some file hosts provide these checksums beforehand.
ex: Gofile provides md5 and pixeldrain provides sha256
hashing:
algorithms:
- xxh128
- md5
- sha256
dedupe
enabled
| Type | Default |
|---|---|
bool |
false |
Delete duplicated downloads automatically. Needs hashing to be enabled (IN_PLACE or POST_DOWNLOAD)
Files with matching known hashes from the database are automatically deleted. Only the oldest copy of the files will be kept.
Deletion will only happen if two or more matching files are found during the database search.
{% hint style="warning" %} dedupe will delete files if you ever downloaded them before, even if the original download no longer exists on disk {% endhint %}
hashing:
dedupe:
enabled: true
use_trash_bin
| Type | Default |
|---|---|
bool |
false |
Deduped files are sent to the trash bin instead of being deleted.
{% hint style="warning" %} Files on a network drive can't be sent to the trasbin. They will always be permanently deleted {% endhint %}
hashing:
dedupe:
use_trash_bin: true