Skip to content

Find, verify, and analyze leaked credentials

6.7 KB
Raw Blame

Filesystem Source

Overview

The Filesystem source lets TruffleHog scan files and directories on local disk for secrets, credentials, and sensitive data. It can scan a single file, walk a whole directory tree, or follow symlinks to files and directories outside the given path.

Filesystem Fundamentals

What does this source scan?

This source scans paths on the machine running TruffleHog. A path can point to a single file, a directory, or a symlink. When a directory is given, TruffleHog walks every file inside it, including files in subdirectories.

Key Terminology

Term Description
Path A file or directory location to scan. You can list more than one
Symlink A file that points to another file or directory instead of holding data itself
Symlink Depth How many symlinks in a row TruffleHog will follow before it stops, to avoid following a loop forever
Include Paths File A file with a list of patterns. Only files matching one of these patterns are scanned
Exclude Paths File A file with a list of patterns. Files matching one of these patterns are skipped
Binary File A file that is not plain text, such as an image or a compiled program
Resume Info Progress information saved during a scan so a long scan can continue where it left off after a restart

Features

  • File and Directory Scanning: Scan a single file or an entire directory tree
  • Symlink Support: Optionally follow symlinks to files and directories, up to a configured depth
  • Path Filtering: Include or exclude files using pattern files
  • Binary File Skipping: Optionally skip files that are not plain text
  • Concurrent Processing: Files within a directory are scanned in parallel
  • Resumable Scans: A scan that stops partway through a large directory can pick up from where it stopped

Configuration

Scanning Paths

CLI Usage:

trufflehog filesystem /var/log /home/user/project/config.yaml

YAML Configuration:

sources:
- connection:
    '@type': type.googleapis.com/sources.Filesystem
    paths:
    - /var/log
    - /home/user/project/config.yaml
  name: local-scan
  type: SOURCE_TYPE_FILESYSTEM
  verify: true

You can also pass directories using the --directory flag, which can be repeated:

trufflehog filesystem --directory /var/log --directory /etc

Following Symlinks

By default, symlinks are not followed. Set a maximum depth to allow TruffleHog to follow them.

CLI Usage:

trufflehog filesystem /home/user/project --max-symlink-depth 5

YAML Configuration:

sources:
- connection:
    '@type': type.googleapis.com/sources.Filesystem
    paths:
    - /home/user/project
    max_symlink_depth: 5
  name: local-scan
  type: SOURCE_TYPE_FILESYSTEM
  verify: true

The highest depth allowed is 40. Setting a higher number will cause the scan to fail with an error at startup.

Skipping Binary Files

YAML Configuration:

sources:
- connection:
    '@type': type.googleapis.com/sources.Filesystem
    paths:
    - /home/user/project
    skip_binaries: true
  name: local-scan
  type: SOURCE_TYPE_FILESYSTEM
  verify: true

Path Filtering

Use plain text files with one pattern per line to include or exclude files from a scan.

CLI Usage:

trufflehog filesystem /home/user/project --include-paths /home/user/include-patterns.txt --exclude-paths /home/user/exclude-patterns.txt

YAML Configuration:

sources:
- connection:
    '@type': type.googleapis.com/sources.Filesystem
    paths:
    - /home/user/project
    include_paths_file: /home/user/include-patterns.txt
    exclude_paths_file: /home/user/exclude-patterns.txt
  name: local-scan
  type: SOURCE_TYPE_FILESYSTEM
  verify: true

How Scanning Works

Scanning Process

  1. Path Check: For each configured path, TruffleHog checks whether it is a file, a directory, or a symlink
  2. Directory Walk: If it is a directory, every entry inside is visited. Subdirectories are visited the same way, going deeper until every file is reached
  3. Symlink Handling: If an entry is a symlink and following symlinks is turned on, TruffleHog follows it to the file or directory it points to, up to the configured depth
  4. File Filtering: Each file is checked against the include and exclude patterns before it is scanned
  5. Binary Check: If skipping binaries is turned on, files that look like binary files are skipped
  6. Content Scanning: The file is opened and its content is sent to the detection engine as chunks
  7. Progress Tracking: The path of the last file scanned in each directory is saved, so a scan that stops can resume from that point

What Gets Scanned

  • Regular files at the given paths
  • Files inside directories at the given paths, including subdirectories
  • Files reached by following a symlink, when symlink following is turned on

What Doesn't Get Scanned

  • Symlinks, when symlink following is turned off
  • Files that go past the configured maximum symlink depth
  • Files matching an exclude pattern, or not matching an include pattern, when pattern files are set
  • Binary files, when skipping binaries is turned on
  • Files that are not regular files, such as device files or sockets

Usage Examples

Scanning a Single File

trufflehog filesystem /home/user/project/config.yaml

Scanning a Directory

trufflehog filesystem /home/user/project

Scanning Multiple Paths

trufflehog filesystem /home/user/project /var/log/app.log

Scanning a Directory and Following Symlinks

trufflehog filesystem /home/user/project --max-symlink-depth 10

Scanning While Skipping Binary Files

sources:
- connection:
    '@type': type.googleapis.com/sources.Filesystem
    paths:
    - /home/user/project
    skip_binaries: true
  name: local-scan
  type: SOURCE_TYPE_FILESYSTEM
  verify: true

Troubleshooting

Common Issues

Issue: Symlinks are not being scanned Solution: Set max_symlink_depth (or --max-symlink-depth) to a value greater than zero. Symlinks are skipped by default.


Issue: specified symlink depth exceeds the allowed max Solution: The maximum allowed symlink depth is 40. Use a smaller value.


Issue: Some files are missing from scan results Solution: Check the include and exclude pattern files for a pattern that matches those files, and confirm skip_binaries is not turned on for text files that look like binary data.


Issue: A restarted scan seems to repeat or skip files Solution: Resume tracking saves the last file scanned inside each directory. If the directory contents changed between runs, the resume point may no longer line up with the same files.