Skip to content

armgit / plexo

Public Imported from GitHub
0

Speed up downloads by combining multiple network connections in parallel

Publish partial downloads without hard-link dependency

Anmol Kapil98e58962dc66
diff --git a/README.md b/README.md
index 7193850..2a78504 100644
--- a/README.md
+++ b/README.md
@@ -157,9 +157,7 @@ Work distribution is dynamically proportional to each interface's real-time thro
Each worker writes its assigned byte range at its final offset in one staging file beside the chosen destination. Workers use separate file handles and explicit byte positions, so non-overlapping ranges can be written in parallel.
-While downloading, the folder contains `<filename>.plexo` and no empty file under the final name. Once every range is complete, Plexo flushes the partial file, publishes the final name without overwriting an existing file, and removes the `.plexo` name. There is no full-file assembly copy, so the destination needs approximately one file's worth of space.
-
-Publication uses a same-volume hard link. On a volume without hard-link support, Plexo leaves the complete `.plexo` file intact and reports a finalization error rather than copying the file or risking an overwrite.
+While downloading, the folder contains `<filename>.plexo` and no empty file under the final name. Once every range is complete, Plexo flushes the partial file, checks for a filename collision, and renames it into place in the same folder. If the final name is taken, Plexo chooses a numbered name. This works on drives such as exFAT without a full-file assembly copy, so the destination needs approximately one file's worth of space.
---
diff --git a/src/main/download/downloadFile.ts b/src/main/download/downloadFile.ts
index 16d5fef..f08eeec 100644
--- a/src/main/download/downloadFile.ts
+++ b/src/main/download/downloadFile.ts
@@ -1,5 +1,5 @@
import { createWriteStream, type WriteStream } from 'node:fs'
-import { link, lstat, open, rm, stat } from 'node:fs/promises'
+import { lstat, open, rename, rm, stat } from 'node:fs/promises'
import { basename, dirname, extname, join } from 'node:path'
/** The only large file owned by a download. It lives beside the final file so publishing it
@@ -88,17 +88,24 @@ export class DownloadFile {
if (partialExists) continue
}
await beforeAttempt(candidate)
+ // Check immediately before the rename, after the recovery intent is saved.
+ // Node has no portable no-replace rename: another process could still
+ // claim this name in the small interval between these two operations.
+ const destinationExists = await lstat(candidate).then(
+ () => true,
+ (error: NodeJS.ErrnoException) => {
+ if (error.code === 'ENOENT') return false
+ throw error
+ }
+ )
+ if (destinationExists) continue
try {
- // Hard-linking is an atomic, no-overwrite publication on the same volume.
- // Removing the partial name afterwards frees no extra file-sized space.
- await link(this.path, candidate)
+ // The partial and final names are in one directory, so this moves the
+ // completed bytes into place without copying the file.
+ await rename(this.path, candidate)
return candidate
} catch (error) {
- const code = (error as NodeJS.ErrnoException).code
- if (code === 'EEXIST') continue
- if (code === 'ENOTSUP' || code === 'EOPNOTSUPP' || code === 'ENOSYS') {
- throw new Error('This destination does not support safe one-copy finalization')
- }
+ if ((error as NodeJS.ErrnoException).code === 'EEXIST') continue
throw error
}
}