Bulk asynchronous downloader for multiple file hosts
level
| Type | Default |
|---|---|
DEBUG, INFO, WARNING, ERROR, CRITICAL |
DEBUG |
Only log messages of this level or higher to the main log file, according to Python logging levels.
| Level | Value | Description |
|---|---|---|
DEBUG |
10 | Offers detailed information, typically useful to troubleshoot problems |
INFO |
20 | Provides general information about scrape and download progress |
WARNING |
30 | Potential issues or something that might need attention (e.g. Login wasn't provided for <FORUM>) |
ERROR |
40 | Serious problem preventing cyberdrop-dl to execute some function |
CRITICAL |
50 | Fatal error that causes cyberdrop-dl to exit immediately |
logs:
level: DEBUG
{% hint style="info" %}
Using anything other that DEBUG makes troubleshooting issues harder. Practically speaking, you should only change this if you expect CDL to run for an extended period
(with a large number of input URLs) to minimize the log files sizes
{% endhint %}
console_level
| Type | Default |
|---|---|
DEBUG, INFO, WARNING, ERROR, CRITICAL or null |
null |
Only log messages of this level or higher to the console. A null value will use the same level as logs.level
logs:
level: null
http_traffic
| Type | Default |
|---|---|
bool |
true |
Log HTTP requests and responses (at INFO level)
{% hint style="warning" %} These requests may have personally identifiable information like your IP or login credentials for sites in cookies/headers. {% endhint %}
logs:
http_traffic: true
expire_after
| Type | Default |
|---|---|
timedelta or null |
null |
if logs.rotate is enabled, this setting specifies the retention period for log files before they are deleted.
- A
timedeltainput is expected to be a valid ISO 8601 timespan, ex:P10DT2H30M10S - An
intinput is assumed to be seconds - A
strinput is expected to be in the format;<value> <unit>, ex:10 days. - A
nullvalue means disabled (never delete any logs)
{% hint style="warning" %}
Any .log or .csv file within logs.folder will be deleted, even if CDL did not create them
{% endhint %}
{% hint style="info" %}
Log files with an absolute path not relative to logs.folder will never be deleted
{% endhint %}
logs:
expire_after: null
folder
| Type | Default |
|---|---|
Path or null |
null |
The path to the location you want cyberdrop-dl to store logs in. A null values will use the platform's default
- Windows:
%AppData%/cyberdrop-dl/Logs - macOS/Linux/Android:
${XDG_STATE_HOME}/cyberdrop-dl/logsor~/.local/state/cyberdrop-dl/logs
logs:
folder: null
rotate
| Type | Default |
|---|---|
bool |
false |
If enabled, cyberdrop-dl will add the current datetime to each log file, in the format YYMMDD_HHMMSS.
Every log file will be created inside a sub folder with the current date
This will prevent overriding old log files
logs:
rotate: false
rotate_date_position
| Type | Default |
|---|---|
prefix or suffix |
suffix |
Ignored unless --logs.rotate is enabled
logs:
rotate_date_position: "suffix"
Files
main
| Type | Default | Restrictions |
|---|---|---|
Path |
downloader.log |
extension will always be overridden to .log |
Path of main log file. For relative paths, the final path will be logs.folder / logs.files.main
logs:
files:
main: downloader.log
download_errors
| Type | Default | Restrictions |
|---|---|---|
Path |
download_errors.csv |
extension will always be overridden to .csv |
Path of the download error log. For relative paths, the final path will be logs.folder / logs.files.download_errors
cyberdrop-dl will output the links it fails to download, the reason and their origin in CSV format.
logs:
files:
download_errors: download_errors.csv
scrape_errors
| Type | Default | Restrictions |
|---|---|---|
Path |
scrape_errors.csv |
extension will always be overridden to .csv |
Path of the errors errors log file. For relative paths, the final path will be logs.folder / logs.files.scrape_errors
cyberdrop-dl will save to the file the links it fails to scrape, the reason and their origin in CSV format.
logs:
files:
scrape_errors: scrape_errors.csv
unsupported
| Type | Default | Restrictions |
|---|---|---|
Path |
unsupported.csv |
extension will always be overridden to .csv |
Path of the unsupported log file. For relative paths, the final path will be logs.folder / logs.files.unsupported
cyberdrop-dl will output links it can't download to this file.
logs:
files:
unsupported: unsupported.csv
last_forum_post
| Type | Default | Restrictions |
|---|---|---|
Path |
last_forum_post.csv |
extension will always be overridden to .csv |
Save the URL of the last scraped post from each thread to this file. For relative paths, the final path will be logs.folder / logs.files.last_forum_post
logs:
files:
unsupported: last_forum_post.csv
dedupe
| Type | Default | Restrictions |
|---|---|---|
Path |
dedupe.csv |
extension will always be overridden to .csv |
Path of the dedupe log file. For relative paths, the final path will be logs.folder / logs.files.dedupe
Every duplicate that the deduper deletes is saved here, together with the file it matched and their shared hash. The file is only created if at least one duplicate was deleted.
logs:
files:
dedupe: dedupe.csv