Bulk asynchronous downloader for multiple file hosts
All setting on this section go at the root of the config file
download_folder
| Type | Default |
|---|---|
Path |
downloads/cyberdrop-dl |
The path to the folder you want cyberdrop-dl to download files to.
download_folder: downloads/cyberdrop-dl
dump_json
| Type | Default |
|---|---|
bool |
False |
If enabled, CDL will created a json lines files with the information about every file downloaded in the current run.
The path to this file will be the same as --log-file but with the extension .results.jsonl
Each line in the file will contain the following details:
{
"url": "https://store9.gofile.io/download/web/7c88c147-ABCD-4e4d-9a6c-12345678/a_video.mp4",
"referer": "https://gofile.io/d/ABC123",
"download_folder": "downloads/cyberdrop-dl/test_album (GoFile)",
"filename": "0hxte0li0o931lwgcrzbz_source.mp4",
"original_filename": "a_video.mp4",
"download_filename": "0hxte0li0o931lwgcrzbz_source.mp4",
"filesize": 12054723,
"ext": ".mp4",
"debrid_link": null,
"duration": null,
"album_id": "ABC123",
"datetime": "2025-01-22T11:00:07",
"parents": ["https://a_forum.com/threads/<name>.54321/post-123123"],
"parent_threads": ["https://a_forum.com/threads/<name>.54321"],
"partial_file": "downloads/cyberdrop-dl/test_album (GoFile)/a_video.mp4.part",
"complete_file": "downloads/cyberdrop-dl/test_album (GoFile)/a_video.mp4",
"hash": "xxh128:53ee56b7bfafa31b8780a572e9783df3",
"downloaded": true,
"attempts": 1
}
dump_json: false
{% hint style="warning" %} The schema of this JSON output is NOT stable may change without notice, even on minor version updates {% endhint %}
max_file_name_length
| Type | Default |
|---|---|
PositiveInt |
95 |
Maximum number of characters a filename should have. CDL will truncate filenames longer that this.
max_file_name_length: 95
max_folder_name_length
| Type | Default |
|---|---|
PositiveInt |
60 |
Maximum number of characters a folder should have. CDL will truncate folders longer that this.
max_folder_name_length: 60
restrict_path
| Type | Default |
|---|---|
list of windows, unix, ascii or no_emoji |
[] |
Sanitize file/folder names according to the options in the list. An empty list (the default) will use the default config according to the OS.
Due to compatibility reasons, the default on Windows and macOS is [windows, no_emoji]. The default on any other system is [windows]
restrict_path: []
windows
remove all characters invalid on Windows: \\, \, | ,/, <, >, :, ?, *
unix
remove all characters invalid on Unix: /
ascii
remove all characters except ascii letters, underscores, and dots
no_emoji
remove all unicode characters except letters, numbers and marks (categories: C, N and M). A few hardcoded exceptions are still allowed:
- literal space
U+0020 - Simbols (category:
S):^,~,$,+,= - Punctuation (category:
P):.,-,_,!,#,%,',(,),,,;,@,[,],{,}
min_free_space
| Type | Default | Restrictions |
|---|---|---|
ByteSize |
5GiB |
>=512MiB |
This is the minimum amount of free space require to start new downloads.
min_free_space: 5.0GiB
{% hint style="info" %}
Values lower than 512MB will always be replaced with 512MB
{% endhint %}
cookies
| Type | Default |
|---|---|
Path or null |
null |
Path to a file/folder with Netscape cookies. All cookie files must have a .txt extension. If the path is a folder, all .txt in the folder are read (non recursive).
These can be used for websites that require login or to pass DDoS-Guard challenges.
You can extract the cookies from your browser using tools like cookie-editor and save them as a .txt file.
The file must be a Netscape formatted cookie file. You can use any name for the file as long as it has a .txt extension.
See: How to extract cookies (DDoSGuard or login errors) #839 for detailed instructions
{% hint style="warning" %}
The user-agent config value MUST match the user-agent of the browser from which you imported the cookies. If they do not match, the cookies will not work
{% endhint %}
deep_scrape
| Type | Default |
|---|---|
bool |
false |
cyberdrop-dl uses a some tricks to try to reduce the number of requests it needs to make while scraping a site. However, this may cause a few links to be skipped.
Use --deep-scrape to disable this and always make a new requests if required.
deep_scrape: false
{% hint style="warning" %} Use this option only if when absolutely necessary, as it will significantly increase the number of requests being made.
For example, scraping an album normally takes one single request.
With --deep-scrape, CDL will make n requests per album, where n is the total number of items in the album
{% endhint %}
delete_partial_files
| Type | Default |
|---|---|
bool |
false |
Files downloaded by CDL have a .part extension (or .cdl_hls for HLS segments) that will be replaced with the original extension when the download reaches 100%.
This allows CDL to resume downloads on subsequent runs.
Setting this to true will delete any .part and .cdl_hls files in the download folder at the end of a session.
delete_partial_files: false
ignore_hashes
| Type | Default |
|---|---|
bool |
false |
Download files even if their hash matches a file on the database.
{% hint style="info" %} Not all file hosts provide hash information before download {% endhint %}
This does not affect auto dedupe. Duplicates are still deleted after the download. See Retries and Download History.
ignore_hashes: false
ignore_history
| Type | Default |
|---|---|
bool |
false |
By default, the program tracks your downloads in a database to prevent downloading the same file multiple times, to save time and reduce load on the servers you're downloading from.
Setting this to true to disable it, ignoring the database and allowing you to re-download files.
This only stops CDL from reading the database. New downloads are still recorded, and files that already exist on disk with the expected size are still skipped. It also disables auto dedupe for the run.
On most sites CDL skips a page entirely if it already downloaded the file it points to. This option disables that too, so pages you have scraped before are requested again.
See Retries and Download History for every check CDL makes.
ignore_history: false
delete_empty_folders
| Type | Default |
|---|---|
bool |
false |
Check (and remove) any empty files and folders in the --download-folder and --sort.input_folder at the end of a session.
delete_empty_folders: true
mtime
| Type | Default |
|---|---|
bool |
True |
CDL does it's absolute best to extract the upload date of files.
By default, this date will be set as the last modified and last accessed date on the downloaded file.
On Windows and macOS, it will also try to set the created date.
Change this to false and those dates will be the current datetime instead
mtime: true
max_thread_depth
| Type | Default |
|---|---|
NonNegativeInt |
0 |
Restricts how many levels deep the scraper is allowed to go while scraping a thread
A value of 0 means only the top level thread will be scraped
max_thread_depth: 0
{% hint style="info" %}
This setting is hardcoded to 0 for Discourse sites
{% endhint %}
{% hint style="warning" %}
It is not recommended to set this above the default value of 0, as there is a high chance of infinite nesting in certain cases.
For example, when dealing with Megathreads, if a Megathread is linked to another Megathread, you could end up scraping an undesirable amount of data. {% endhint %}
Example
Consider CDL finds the following sub-threads while scraping an input URL:
└── thread_01
├── thread_02
├── thread_03
│ ├── thread_09
│ ├── thread_10
│ └── thread_11
├── thread_04
├── thread_05
├── thread_06
├── thread_07
│ └── thread_12
└── thread_08
- With
max_thread_depth= 0, CDL will only download files inthread_01, all the other threads will be ignored - With
max_thread_depth= 1, CDL will only download files inthread_01tothread_08. All threads fromthread_09tothread_12will be ignored - With
max_thread_depth>= 2, CDL will download files from all the threads in this case
max_thread_folder_depth
| Type | Default |
|---|---|
NonNegativeInt or null |
null |
Restricts the max number of nested folders CDL will create when max_thread_depth is greater that 0
Values:
null: Create as many nested folders as required (AKA, the same number asmax_thread_depthallows)0: Do not create subfolders, use a flat structure for any nested thread.1+: Create a max ofnfolders
max_thread_folder_depth: null
Example
- With
max_thread_folder_depth=null:
└── thread_01
├── thread_02
├── thread_03
│ ├── thread_09
│ ├── thread_10
│ └── thread_11
├── thread_04
├── thread_05
├── thread_06
├── thread_07
│ └── thread_12
└── thread_08
- With
max_thread_folder_depth= 0:
├── thread_01
├── thread_02
├── thread_03
├── thread_09
├── thread_10
├── thread_11
├── thread_04
├── thread_05
├── thread_06
├── thread_07
├── thread_12
└── thread_08
- With
max_thread_folder_depth= 1:
└── thread_01
├── thread_02
├── thread_03
├── thread_09
├── thread_10
├── thread_11
├── thread_04
├── thread_05
├── thread_06
├── thread_07
├── thread_12
└── thread_08
update_check
| Type | Default |
|---|---|
bool |
True |
Check for updates at the end of the session