Skip to content

armgit / peel

Public Imported from GitHub
0

Uninstall Mac apps with every file they leave behind, and see why each belongs. Free up space, fine-tune macOS, and set up Terminal.

Judge Duplicates' copies by more of what they are

Țugui Dragoș962adb47760f
diff --git a/GUIDE.md b/GUIDE.md
index e4398fe4..fe1311a3 100644
--- a/GUIDE.md
+++ b/GUIDE.md
@@ -120,9 +120,11 @@ Space shows what fills your disk, area by area, and what each area is.
it looks in Desktop, Documents, Downloads, Pictures, Movies, and Music. Files under 100 KB, and folders holding
less, are left out unless you choose a smaller size, in the app and in `peel duplicates` alike. A copy with Finder
tags, a Finder comment, or a resource fork the others lack is marked Stays and never moved, since moving it would
- lose them, and the others can all go. After a scan you start, Peel selects every copy in each group but the first,
- which it suggests keeping: one on the disk your home folder is on, then one outside Downloads, then one whose name
- doesn't end like a copy's, such as `photo copy.jpg` or `photo (1).jpg`, then the oldest. Look before you
+ lose them, and the others can all go. A group where each copy has something the others lack isn't listed. After a
+ scan you start, Peel selects every copy in each group but the one it suggests keeping: one on the disk your home
+ folder is on, then one outside `/Users/Shared`, then one outside Downloads, then one whose name doesn't end like a
+ copy's, such as `photo copy.jpg` or `photo (1).jpg`, then the oldest. A copy in `/Users/Shared` isn't selected for
+ you, since every account may use it. Look before you
move them: one copy is always kept. A copy in a folder that syncs to your other devices, such as iCloud Drive or
what Desktop and Documents hold when they are in iCloud, is listed and never moved, since the Trash would take it
from your other devices too: beside a copy on this Mac it is marked Stays, and a group whose every copy syncs is
diff --git a/Packages/PeelCore/Sources/PeelCore/Developer/ProjectArtifacts.swift b/Packages/PeelCore/Sources/PeelCore/Developer/ProjectArtifacts.swift
index ddf8453b..92bfa680 100644
--- a/Packages/PeelCore/Sources/PeelCore/Developer/ProjectArtifacts.swift
+++ b/Packages/PeelCore/Sources/PeelCore/Developer/ProjectArtifacts.swift
@@ -962,11 +962,7 @@ public enum ProjectArtifacts {
/// inside it says more.
static func holdingBackWhatIsShared(_ artifacts: [ProjectArtifact]) -> [ProjectArtifact] {
artifacts.map { artifact in
- let path = PathPattern.comparablePath(of: artifact.url).lowercased()
- let isShared = ["/users/shared", "/system/volumes/data/users/shared"].contains {
- PathComponents.isPath(path, atOrInside: $0)
- }
- guard artifact.heldBack == nil, isShared else { return artifact }
+ guard artifact.heldBack == nil, FileAccess.isSharedWithEveryone(artifact.url) else { return artifact }
var held = artifact
held.heldBack = .sharedWithEveryone
return held
diff --git a/Packages/PeelCore/Sources/PeelCore/Duplicates/DuplicateFinder.swift b/Packages/PeelCore/Sources/PeelCore/Duplicates/DuplicateFinder.swift
index 1ba55e26..762242e9 100644
--- a/Packages/PeelCore/Sources/PeelCore/Duplicates/DuplicateFinder.swift
+++ b/Packages/PeelCore/Sources/PeelCore/Duplicates/DuplicateFinder.swift
@@ -149,6 +149,12 @@ public struct DuplicateFinder: Sendable {
// Lightroom Classic's preview caches beside a catalog:
// helpx.adobe.com/lightroom-classic/desktop/kb/preference-file-and-other-file-locations.html
"lrdata",
+ // Parallels Desktop, "Show package content": kb.parallels.com/4680
+ "pvm",
+ // VMware Fusion, "a bundle format (also known as a package)": knowledge.broadcom.com/external/article/303392
+ "vmwarevm",
+ // UTM, conforming to `com.apple.package`: Platform/macOS/Info.plist in github.com/utmapp/UTM
+ "utm",
]
/// Finds folders whose whole contents are identical, then identical files. Files are compared by size, then
@@ -442,7 +448,7 @@ public struct DuplicateFinder: Sendable {
)
}
let group = DuplicateGroup(id: members[0].digest.hexadecimal, files: files)
- return group.movableFiles.isEmpty && !group.everyCopySyncs ? nil : group
+ return group.isWorthListing ? group : nil
}
/// Which copy is kept, from the home folder: its Downloads, and the disk it is on.
@@ -456,14 +462,15 @@ public struct DuplicateFinder: Sendable {
homeDevice = stat(home.path(percentEncoded: false), &info) == 0 ? info.st_dev : nil
}
- /// Ranks a copy for keeping, lower being better: on the disk the home folder is on, outside Downloads, without
- /// a copy suffix, created earlier, less deeply nested, then with a shorter name. The path breaks any tie that
- /// is left.
+ /// Ranks a copy for keeping, lower being better: on the disk the home folder is on, outside `/Users/Shared`
+ /// and Downloads, without a copy suffix, created earlier, less deeply nested, then with a shorter name. The
+ /// path breaks any tie that is left.
func rank(of url: URL, identity: FileIdentity) -> (Int, Int, Int, Int, String) {
let path = DuplicateFinder.path(of: url)
let name = url.deletingPathExtension().lastPathComponent
let flags = [
- identity.link.device != homeDevice, path.isInside(downloads), DuplicateFinder.hasCopySuffix(name),
+ identity.link.device != homeDevice, FileAccess.isSharedWithEveryone(url), path.isInside(downloads),
+ DuplicateFinder.hasCopySuffix(name),
]
return (
// Each flag outweighs those after it.
@@ -492,21 +499,31 @@ public struct DuplicateFinder: Sendable {
return Set(managedFolders.map { home + "/" + $0 })
}
- /// Whether `name` ends with a suffix that Finder, a browser, or the Keep Both option adds to a copy: " copy",
- /// " copy 2", " (3)", or " 2", in any letter case, with "copy" in any language listed in `copyWords`. It works
- /// word by word rather than with a regular expression, to stay fast: it runs for every file in every group.
+ /// Whether `name` carries what Finder, a browser, the Keep Both option or AppKit (`"%@ copy"` in its `Common`
+ /// table) adds to a copy: " copy", " copy 2", " (3)", or " 2", in any letter case, with "copy" in any language
+ /// listed in `copyWords`, "%@ (kopie)", the forms joined to the name, and the forms before it. It works word by
+ /// word rather than with a regular expression, to stay fast: it runs for every file in every group.
static func hasCopySuffix(_ name: String) -> Bool {
var words = name.split(separator: " ", omittingEmptySubsequences: false)
- guard words.count > 1, let last = words.popLast() else { return false }
+ if words.count == 1 {
+ return joinedCopyEndings.contains { name.count > $0.count && name.lowercased().hasSuffix($0) }
+ }
+ if words.count > 2, words[0].lowercased() == "copie", words[1].lowercased() == "de" { return true }
+ if words[0].lowercased() == "копия" { return true }
+ guard let last = words.popLast() else { return false }
if isNumber(last) {
return last.count <= 2 || words.count > 1 && copyWords.contains(words[words.count - 1].lowercased())
}
if last.count > 2, last.first == "(", last.last == ")" {
- return isNumber(last.dropFirst().dropLast())
+ let inside = last.dropFirst().dropLast()
+ return isNumber(inside) || copyWords.contains(inside.lowercased())
}
return copyWords.contains(last.lowercased())
}
+ /// What AppKit joins to a name with no space between them, in Japanese, Chinese, and Polish.
+ private static let joinedCopyEndings = ["のコピー", "_副本", "副本", "-kopia"]
+
private static let copyWords: Set<String> = [
"copy", "kopie", "copie", "copia", "cópia", "kopia", "копия", "копія", "kopyası", "のコピー", "副本", "拷貝", "복사본",
]
diff --git a/Packages/PeelCore/Sources/PeelCore/Duplicates/Duplicates.swift b/Packages/PeelCore/Sources/PeelCore/Duplicates/Duplicates.swift
index 8e0375c2..0147cc6a 100644
--- a/Packages/PeelCore/Sources/PeelCore/Duplicates/Duplicates.swift
+++ b/Packages/PeelCore/Sources/PeelCore/Duplicates/Duplicates.swift
@@ -53,6 +53,11 @@ public struct DuplicateFile: Sendable, Hashable, Identifiable {
public var id: URL { url }
+ /// True for a copy in `/Users/Shared`, which every account on this Mac may use, so it isn't suggested for moving.
+ public var isSharedWithEveryone: Bool {
+ FileAccess.isSharedWithEveryone(url)
+ }
+
public var modificationDate: Date {
identity.modificationDate
}
@@ -92,9 +97,25 @@ public struct DuplicateGroup: Sendable, Hashable, Identifiable {
}
/// The copies Peel suggests moving: every one that may move but the first, which it keeps, or every one when a
- /// copy stays for what the others lack.
+ /// copy stays for what the others lack, and never one every account on this Mac may use.
public var suggestedFiles: [DuplicateFile] {
- holdsACopyWithWhatOthersLack ? movableFiles : Array(movableFiles.dropFirst())
+ (holdsACopyWithWhatOthersLack ? movableFiles : Array(movableFiles.dropFirst()))
+ .filter { !$0.isSharedWithEveryone }
+ }
+
+ /// True when moving the suggested copies frees less than they take, since something else holds the same blocks.
+ public var sharesStorage: Bool {
+ suggestedFiles.contains(where: \.sharesStorage)
+ }
+
+ /// True when the copies are on more than one disk, so what moving them frees is not all on one.
+ public var isOnSeveralDisks: Bool {
+ Set(files.map(\.identity.link.device)).count > 1
+ }
+
+ /// A scan lists a group while a copy of it may move, or while every copy syncs, which the page says apart.
+ var isWorthListing: Bool {
+ !movableFiles.isEmpty || everyCopySyncs
}
/// What moving the suggested copies would free.
@@ -274,10 +295,19 @@ public struct DuplicateScan: Sendable {
})
}
+ /// The copies no longer there in the groups holding any of `urls`, where another page may have taken one this page
+ /// never had selected, even the one it keeps.
+ public func copiesGone(inGroupsHolding urls: Set<URL>) -> Set<URL> {
+ let files = groups.map { $0.files.map(\.url) }
+ let folders = folderGroups.map { $0.folders.map(\.url) }
+ return Set((files + folders).filter { $0.contains(where: urls.contains) }.joined().filter(\.isMissing))
+ }
+
public func removing(_ urls: Set<URL>) -> DuplicateScan {
let remaining = groups.compactMap { group -> DuplicateGroup? in
let files = group.files.filter { !urls.contains($0.url) }
- return files.count > 1 ? DuplicateGroup(id: group.id, files: files) : nil
+ let left = DuplicateGroup(id: group.id, files: files)
+ return files.count > 1 && left.isWorthListing ? left : nil
}
let remainingFolders = folderGroups.compactMap { group -> DuplicateFolderGroup? in
let folders = group.folders.filter { !urls.contains($0.url) }
diff --git a/Packages/PeelCore/Sources/PeelCore/Support/FileAccess.swift b/Packages/PeelCore/Sources/PeelCore/Support/FileAccess.swift
index e4b7ce5c..f3766862 100644
--- a/Packages/PeelCore/Sources/PeelCore/Support/FileAccess.swift
+++ b/Packages/PeelCore/Sources/PeelCore/Support/FileAccess.swift
@@ -1,5 +1,6 @@
import Darwin
public import Foundation
+internal import PeelPrivileged
public enum FileAccess {
/// True when the current user can't move `url` to the Trash without administrator rights.
@@ -49,6 +50,14 @@ public enum FileAccess {
return info.st_flags & (UInt32(UF_IMMUTABLE) | UInt32(SF_IMMUTABLE)) != 0
}
+ /// Whether `url` is in `/Users/Shared`, which belongs to every account on this Mac.
+ static func isSharedWithEveryone(_ url: URL) -> Bool {
+ let path = PathPattern.comparablePath(of: url).lowercased()
+ return ["/users/shared", "/system/volumes/data/users/shared"].contains {
+ PathComponents.isPath(path, atOrInside: $0)
+ }
+ }
+
/// Whether `url` is marked so that nothing removes or renames it (`SF_NOUNLINK`, `sys/stat.h`), so it can't go to
/// the Trash, whoever asks.
static func cannotBeRemoved(_ url: URL) -> Bool {
diff --git a/Packages/PeelCore/Tests/PeelCoreTests/DuplicateFinderTests.swift b/Packages/PeelCore/Tests/PeelCoreTests/DuplicateFinderTests.swift
index d3c662c5..b87e5b33 100644
--- a/Packages/PeelCore/Tests/PeelCoreTests/DuplicateFinderTests.swift
+++ b/Packages/PeelCore/Tests/PeelCoreTests/DuplicateFinderTests.swift
@@ -660,6 +660,108 @@ struct DuplicateFinderTests {
#expect(!DuplicateFinder.hasCopySuffix("Invoice (3a)"))
}
+ @Test func recognizesTheNamesAppKitGivesADuplicateInEveryLanguage() {
+ for name in [
+ "契約書のコピー", "合同_副本", "合約副本", "Umowa-kopia", "Smlouva (kopie)", "Avtal (kopia)",
+ "Copie de Contrat", "Копия Договор", "Contract - copie",
+ ] {
+ #expect(DuplicateFinder.hasCopySuffix(name), "\(name)")
+ }
+ for name in ["コピー", "副本", "Copie de", "Копия", "Kopia", "Copies of Contract", "Contrat Copie de", "ab-kopiat"] {
+ #expect(!DuplicateFinder.hasCopySuffix(name), "\(name)")
+ }
+ }
+
+ @Test func aVirtualMachineIsADocumentEvenWithItsAppGone() async throws {
+ let directory = try TemporaryDirectory()
+ let disk = randomData(count: 200_000)
+ for machine in ["Windows.pvm", "Windows copy.pvm", "Fedora.vmwarevm", "Fedora copy.vmwarevm", "Debian.utm", "Debian copy.utm"] {
+ try directory.file("home/Documents/\(machine)/disk.hds", contents: disk)
+ try directory.file("home/Documents/\(machine)/\(machine.count).txt", contents: Data(machine.utf8))
+ }
+
+ let result = try await scan(directory)
+
+ #expect(result.groups.isEmpty)
+ #expect(result.folderGroups.isEmpty)
+ }
+
+ @Test func findsTheCopiesAnotherPageTookInAGroupItShows() async throws {
+ let directory = try TemporaryDirectory()
+ let bytes = randomData(count: 200_000)
+ let kept = try directory.file("home/Documents/Film.mov", contents: bytes)
+ try directory.file("home/Downloads/Film.mov", contents: bytes)
+ let other = randomData(count: 200_000)
+ try directory.file("home/Documents/Other.mov", contents: other)
+ try directory.file("home/Downloads/Other.mov", contents: other)
+ let result = try await scan(directory)
+ try FileManager.default.removeItem(at: kept)
+ let film = try #require(result.groups.first { $0.files[0].url.lastPathComponent == "Film.mov" })
+ let selected = try #require(film.files.first { $0.url.path(percentEncoded: false).contains("/Downloads/") })
+
+ let gone = result.copiesGone(inGroupsHolding: [selected.url])
+
+ #expect(gone.map(\.lastPathComponent) == ["Film.mov"])
+ #expect(gone.allSatisfy { $0.path(percentEncoded: false).contains("/Documents/") })
+ #expect(result.removing(gone).groups.map { $0.files[0].url.lastPathComponent } == ["Other.mov"])
+ }
+
+ @Test func aMoveLeavesNoGroupOfCopiesThatCanNoLongerMove() {
+ func file(_ name: String, heldBack: HoldBack? = nil) -> DuplicateFile {
+ DuplicateFile(
+ url: URL(filePath: "/Users/me/\(name)"), size: 100, reclaimableSize: 100, allocatedSize: 100,
+ identity: FileIdentity(stat()), heldBack: heldBack
+ )
+ }
+ let local = file("local")
+ let group = DuplicateGroup(id: "a", files: [
+ file("tagged", heldBack: .keepsWhatTheOthersLack), file("synced", heldBack: .inTheCloud), local,
+ ])
+ let synced = DuplicateGroup(id: "b", files: [
+ file("one", heldBack: .inTheCloud), file("two", heldBack: .inTheCloud), file("three", heldBack: .inTheCloud),
+ ])
+ let scan = DuplicateScan(
+ groups: [group, synced], folderGroups: [], unreadableLocations: [], skippedLocations: [],
+ needsFullDiskAccess: false
+ )
+
+ #expect(scan.removing([local.url]).groups.map(\.id) == ["b"])
+ }
+
+ @Test func aGroupSharesStorageOnlyInWhatItSuggestsMoving() {
+ func file(_ name: String, reclaimable: Int64, heldBack: HoldBack? = nil) -> DuplicateFile {
+ DuplicateFile(
+ url: URL(filePath: "/Users/me/\(name)"), size: 100, reclaimableSize: reclaimable, allocatedSize: 100,
+ identity: FileIdentity(stat()), heldBack: heldBack
+ )
+ }
+ let clone = file("clone", reclaimable: 0)
+ let plain = file("plain", reclaimable: 100)
+ let tagged = file("tagged", reclaimable: 100, heldBack: .keepsWhatTheOthersLack)
+
+ #expect(DuplicateGroup(id: "a", files: [clone, tagged, plain]).sharesStorage, "every copy may go here")
+ #expect(!DuplicateGroup(id: "b", files: [clone, plain]).sharesStorage, "the first copy stays")
+ #expect(DuplicateGroup(id: "c", files: [plain, clone]).sharesStorage)
+ }
+
+ @Test func prefersACopyOutsideTheFolderEveryAccountSharesAndNeverSuggestsOneThere() async throws {
+ let order = DuplicateFinder.KeepOrder(home: URL(filePath: "/Users/me", directoryHint: .isDirectory))
+ let identity = FileIdentity(stat())
+ let shared = order.rank(of: URL(filePath: "/Users/Shared/Film.mov"), identity: identity)
+ let own = order.rank(of: URL(filePath: "/Users/me/Downloads/Film.mov"), identity: identity)
+ #expect(own < shared)
+ func file(_ path: String) -> DuplicateFile {
+ DuplicateFile(
+ url: URL(filePath: path), size: 100, reclaimableSize: 100, allocatedSize: 100, identity: identity
+ )
+ }
+ let group = DuplicateGroup(id: "a", files: [
+ file("/Users/me/Documents/Film.mov"), file("/Users/Shared/Film.mov"), file("/Users/me/Downloads/Film.mov"),
+ ])
+ #expect(group.suggestedFiles.map(\.url.lastPathComponent) == ["Film.mov"])
+ #expect(group.suggestedFiles.first?.url.path(percentEncoded: false) == "/Users/me/Downloads/Film.mov")
+ }
+
/// `hasCopySuffix` is asked once for every file in every group, so it has to stay fast on a large scan.
@Test func recognizesCopySuffixesQuickly() {
var copies = 0
diff --git a/Peel/Views/Duplicates/DuplicateGroupView.swift b/Peel/Views/Duplicates/DuplicateGroupView.swift
index 64ecee3f..69bfb57c 100644
--- a/Peel/Views/Duplicates/DuplicateGroupView.swift
+++ b/Peel/Views/Duplicates/DuplicateGroupView.swift
@@ -105,10 +105,8 @@ struct DuplicateGroupView: View {
.accessibilityHidden(true)
}
- /// True when a copy other than the first, which Peel suggests keeping, shares its blocks with something else.
- /// This checks blocks, not sizes: a compressed or sparse file is smaller on disk without sharing anything.
private var sharesStorage: Bool {
- group.files.dropFirst().contains { $0.sharesStorage }
+ group.sharesStorage
}
}
diff --git a/README.md b/README.md
index 307c115c..f9e22dba 100644
--- a/README.md
+++ b/README.md
@@ -137,7 +137,7 @@ anything, and `peel help <command>` lists every option. With a Homebrew install,
| `peel orphans` | Lists files left by apps that are no longer installed. `--remove <group>` moves one group. |
| `peel caches` | Lists the caches of developer tools. `--remove` moves the ones Peel suggests. |
| `peel projects <folders>` | Lists what builds left in your projects. `--remove` moves the ones Peel suggests. |
-| `peel duplicates [folders]` | Finds files and folders with identical contents. `--remove` keeps one copy of each and moves the rest, but never a copy that syncs to your other devices. |
+| `peel duplicates [folders]` | Finds files and folders with identical contents. `--remove` keeps one copy of each and moves the rest, but never a copy that syncs to your other devices, one with Finder tags, a comment, or a resource fork the others lack, or one in `/Users/Shared`. |
| `peel search` | Searches Spotlight by name, kind, or size, and by age along with one of them. |
| `peel updates` | Checks your apps for updates. |
| `peel history` | Lists what Peel moved to the Trash, from the app and from Terminal alike. `--refused` lists what it was asked to move and wouldn't, and why. |
diff --git a/SAFETY.md b/SAFETY.md
index f155f4ea..0459122e 100644
--- a/SAFETY.md
+++ b/SAFETY.md
@@ -488,15 +488,16 @@ Peel doesn't recommend; Return answers Select Recommended, or Cancel when nothin
still shows its reason beside it.
In Duplicates, one copy of every group always stays, and a file that changed since the scan is refused. When a scan you
-started ends, Duplicates selects every copy but the first of each group, the one it suggests keeping: check which copy
-stays before you move the others. Nothing inside a repository, a project, `node_modules`, a photo or music library, an
+started ends, Duplicates selects every copy of each group but the one it suggests keeping, and never one in
+`/Users/Shared`, which every account may use: check which copy stays before you move the others. Nothing inside a repository, a project, `node_modules`, a photo or music library, an
app, or another package is ever offered, nor a hidden file or folder, such as a tool's settings in `~/.config`. An app,
another package, or anything hidden counts only toward the folder around it, a document its app keeps as a package among
them, such as a Logic Pro or Scrivener project, even once the app is gone, while a folder holding a repository, a
project, `node_modules`, or a library is never compared, nor any folder above it. A copy with Finder tags, a Finder
comment, or a resource fork the others lack is listed and never selected or moved, since moving it would take that away;
-it keeps the bytes for the others, which can all go. Folders that differ in them inside aren't listed together, and a
-folder kept only in the cloud is never looked into. A copy in a folder that syncs to your other devices, iCloud Drive or
+it keeps the bytes for the others, which can all go, and a group where each copy has something the others lack isn't
+listed. Folders that differ in them, inside or on the folder itself, aren't listed together, and a folder kept only in
+the cloud is never looked into. A copy in a folder that syncs to your other devices, iCloud Drive or
one a sync app keeps in step, is listed and never selected or moved: its row reads Stays, and a group whose every copy
syncs is listed under Synced to Other Devices.