Skip to content

Send files directly between devices over the internet - no cloud storage, no size limits.

Removed legacy hyperdrive

Denis Lupookovbde2429f339e
diff --git a/README.md b/README.md
index 202b5d1..1020252 100644
--- a/README.md
+++ b/README.md
@@ -196,7 +196,7 @@ Desktop and mobile share translation catalogs through `@altersend/locales`, curr
### Tech stack
-[Electron](https://electronjs.org) · [React Native](https://reactnative.dev) · [Expo](https://expo.dev) · [Bare](https://bare.pears.com) · [Hyperswarm](https://github.com/holepunchto/hyperswarm) · [Protomux](https://github.com/holepunchto/protomux) · [Hyperdrive](https://github.com/holepunchto/hyperdrive) · [React Strict DOM](https://github.com/facebook/react-strict-dom) · [Tailwind](https://tailwindcss.com) · [Zustand](https://github.com/pmndrs/zustand)
+[Electron](https://electronjs.org) · [React Native](https://reactnative.dev) · [Expo](https://expo.dev) · [Bare](https://bare.pears.com) · [Hyperswarm](https://github.com/holepunchto/hyperswarm) · [Protomux](https://github.com/holepunchto/protomux) · [React Strict DOM](https://github.com/facebook/react-strict-dom) · [Tailwind](https://tailwindcss.com) · [Zustand](https://github.com/pmndrs/zustand)
### Crash reporting
diff --git a/apps/desktop/package.json b/apps/desktop/package.json
index ecb8cf2..d5dfffe 100644
--- a/apps/desktop/package.json
+++ b/apps/desktop/package.json
@@ -41,10 +41,8 @@
"@sentry/electron": "^7.13.0",
"b4a": "^1.8.0",
"bare-rpc": "^1.2.0",
- "corestore": "^7.9.1",
"electron-squirrel-startup": "^1.0.1",
"hypercore-crypto": "^3.6.1",
- "hyperdrive": "^13.3.0",
"hyperswarm": "^4.17.0",
"lucide-react": "^1.8.0",
"paparam": "^1.10.0",
diff --git a/apps/desktop/src/types.d.ts b/apps/desktop/src/types.d.ts
index e988b88..5ab65ea 100644
--- a/apps/desktop/src/types.d.ts
+++ b/apps/desktop/src/types.d.ts
@@ -1,7 +1,5 @@
declare module 'hyperswarm'
declare module 'hypercore-crypto'
-declare module 'hyperdrive'
-declare module 'corestore'
declare module 'b4a'
declare module 'pear-runtime'
declare module 'which-runtime'
diff --git a/docs/architecture.md b/docs/architecture.md
index 7699a92..fa942da 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -45,11 +45,10 @@ The protocol layer, running entirely inside a **Bare worklet** (a lightweight JS
- `worklet/index.ts` — entrypoint; wires Bare IPC → RPC server → orchestrator
- `worklet/transfer/orchestrator.ts` — top-level coordinator; owns session lifecycle + state, composes the subsystems below
-- `worklet/transfer/swarm.ts` — `TransferSwarm`: peer connectivity, Corestore replication, per-peer control channels
-- `worklet/transfer/drive.ts` — the `@altersend/drive` chunk channel over Protomux; the preferred transfer path
+- `worklet/transfer/swarm.ts` — `TransferSwarm`: peer connectivity, per-peer control channels
+- `worklet/transfer/drive.ts` — the `@altersend/drive` chunk channel over Protomux; the only transfer path
- `worklet/transfer/control-channel.ts` — per-peer control messages (offers, requests, progress, cancel)
-- `worklet/transfer/storage.ts` — `TransferStorage`: the ephemeral Corestore + Hyperdrive for a session (wiped on disconnect)
-- `worklet/transfer/sender.ts` / `receiver.ts` — stage files into the writable Hyperdrive (legacy path); receiver picks drive channel per file, else replicated Hyperdrive
+- `worklet/transfer/sender.ts` / `receiver.ts` — sender describes files on disk and serves chunks from them; receiver opens a drive channel per file and writes chunks straight to the target
- `worklet/transfer/topic-auth.ts` — the join-code proof (see [Topic authentication](#topic-authentication))
- `worklet/relay/config.ts` / `conf.ts` — relay state + `relayThrough`; relay list from a signed DHT record (see [Relay fallback](#relay-fallback))
- `worklet/relay/announce.ts` / `upgradeWebRelay.ts` — present a signed cap token to a relay (see [Relay fallback](#relay-fallback))
@@ -84,12 +83,12 @@ Locale metadata, preference resolution, i18next setup, and bundled catalogs for
1. **Sender** opens the share screen → the worklet generates a single-use topic: a random 32-byte key, hex-encoded to a 64-char join code, displayed as QR / `com.altersend.mobile://join/<topic>`.
2. **Receiver** scans or types the code → domain validates and extracts the topic, passes it to core.
-3. Both sides join the Hyperswarm topic; on connection the shared Corestore replicates over the noise-encrypted socket.
+3. Both sides join the Hyperswarm topic; on connection they open control and drive channels over the noise-encrypted socket.
4. **Topic authentication** — the sender challenges the receiver to prove it holds the join code before releasing any offers; a wrong proof is rejected, while a peer that never proves it is flagged (not dropped). See [Topic authentication](#topic-authentication).
-5. Sender broadcasts a `transfer-ready` message with file offers (each carrying the drive key).
-6. Receiver requests each file over a `@altersend/drive` chunk channel, streaming chunks straight to disk (falling back to the replicated Hyperdrive for older peers). Progress flows back over the control channel → RPC → domain → UI.
+5. Sender broadcasts a `transfer-ready` message with file offers.
+6. Receiver requests each file over a `@altersend/drive` chunk channel, streaming chunks straight to disk. Progress flows back over the control channel → RPC → domain → UI.
-Step 6 is mid-migration: `@altersend/drive` is the path when both sides support it, Hyperdrive is only the older-peer fallback. New transfer work belongs in `packages/drive`.
+A peer that does not speak the drive protocol (AlterSend 1.6 or older) fails the download with an "update AlterSend" message. New transfer work belongs in `packages/drive`.
A transfer is capped at **10,000 files** (`MAX_FILES_PER_TRANSFER`, validated in `control-validation.ts`); the send UI blocks earlier with a "zip them" hint. The whole offer list goes in one message on the single worklet thread, so huge counts would choke it.
@@ -106,7 +105,7 @@ A valid proof releases the offers (even if late) and is required to serve a file
Most transfers connect directly via hole-punching. When two peers can't reach each other — typically both behind symmetric NAT (e.g. both on a VPN) — the transfer falls back to a **blind relay**: a public server that pairs the peers and forwards their already-encrypted UDX stream, never holding the keys to decrypt it.
- **Engagement** — `relay/config.ts` exposes `relayThrough` in "eager" mode: when enabled it always offers the relay, so hyperdht races a relayed path against a direct punch and upgrades to direct if the punch lands.
-- **Discovery** — the relay key isn't baked in. `relay/conf.ts` reads the relay list from a signed DHT **mutable record** (public key injected at build via `--relay-conf-pubkey`), so relays rotate with no app release. Fetched lazily once the relay is enabled, with bounded retry (the worklet wipes its Corestore on startup, so a hypercore won't do).
+- **Discovery** — the relay key isn't baked in. `relay/conf.ts` reads the relay list from a signed DHT **mutable record** (public key injected at build via `--relay-conf-pubkey`), so relays rotate with no app release. Fetched lazily once the relay is enabled, with bounded retry (the worklet keeps no persistent core storage, so a hypercore won't do).
- **Classification** — `TransferSwarm.classifyConnection` matches a peer's `remoteHost` against known relay hosts and emits a per-peer `connection-type` (`direct` / `relay`), keyed per sender; the UI shows **Connected** vs **Connected via relay**.
- **Caps** — a relay caps how much a session forwards. A sender raises its own cap with a short-lived token signed by the entitlement service and verified against the relay's public key: `relay/announce.ts` sends it over a protomux `altersend-pro` channel, and `config.ts` releases it only while sending and only to a key from the signed relay list. A browser receiver asks its sender for the upgrade via the `web-relay` control message; `upgradeWebRelay.ts` dials the relay to announce it. Enforced relay-side.
@@ -134,7 +133,7 @@ You can **pair** devices you trust to send to them later without a code. This li
- **Pairing** — the QR opens a pairing swarm on a fresh topic; both sides exchange a signed `pairing-info` (pubkey + name, signed over the noise handshake so it can't be relayed) and vote to remember each other, deriving a shared rendezvous topic. A live transfer peer can pair via the "Pair" button (`RememberCoordinator`).
- **Discovery & invites** — the discovery swarm uses the device keypair, firewalled to remembered pubkeys, keeping background links to paired devices. To send without a code you "invite" one: the worklet joins its rendezvous topic and sends an invite it can accept.
- **Recognition (privacy)** — to badge an already-paired peer without revealing identity, each side sends only a **signature** over the handshake (no pubkey/name); the receiver matches it against its own remembered devices. A non-paired peer learns nothing.
-- **Persistence** — the transfer Corestore/Hyperdrive is wiped on every disconnect; device identity and the remembered-peer list persist.
+- **Persistence** — a transfer keeps nothing on disk beyond the file being written; device identity and the remembered-peer list persist.
## IPC bridge (desktop)
diff --git a/knip.json b/knip.json
index e5ecf35..07d2d86 100644
--- a/knip.json
+++ b/knip.json
@@ -41,9 +41,7 @@
"ignoreDependencies": [
"b4a",
"bare-rpc",
- "corestore",
"hypercore-crypto",
- "hyperdrive",
"hyperswarm",
"pear-runtime-updater",
"@electron-forge/maker-dmg",
diff --git a/package-lock.json b/package-lock.json
index fbe88ac..943027c 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -42,10 +42,8 @@
"@sentry/electron": "^7.13.0",
"b4a": "^1.8.0",
"bare-rpc": "^1.2.0",
- "corestore": "^7.9.1",
"electron-squirrel-startup": "^1.0.1",
"hypercore-crypto": "^3.6.1",
- "hyperdrive": "^13.3.0",
"hyperswarm": "^4.17.0",
"lucide-react": "^1.8.0",
"paparam": "^1.10.0",
@@ -25428,14 +25426,10 @@
"bare-tls": "^3.1.5",
"bare-zlib": "^1.3.4",
"compact-encoding": "^2.19.0",
- "corestore": "^7.9.1",
"hypercore-crypto": "^3.6.1",
"hyperdb": "^6.7.0",
"hyperdht": "^6.32.0",
- "hyperdrive": "^13.3.0",
"hyperswarm": "^4.17.0",
- "localdrive": "^2.2.0",
- "mirror-drive": "^1.14.1",
"protomux": "^3.7.0"
},
"devDependencies": {
diff --git a/packages/core/package.json b/packages/core/package.json
index a846974..ec2d53f 100644
--- a/packages/core/package.json
+++ b/packages/core/package.json
@@ -43,14 +43,10 @@
"bare-tls": "^3.1.5",
"bare-zlib": "^1.3.4",
"compact-encoding": "^2.19.0",
- "corestore": "^7.9.1",
"hypercore-crypto": "^3.6.1",
"hyperdb": "^6.7.0",
"hyperdht": "^6.32.0",
- "hyperdrive": "^13.3.0",
"hyperswarm": "^4.17.0",
- "localdrive": "^2.2.0",
- "mirror-drive": "^1.14.1",
"protomux": "^3.7.0"
},
"devDependencies": {
diff --git a/packages/core/src/types.d.ts b/packages/core/src/types.d.ts
index e6d83e4..25e40a1 100644
--- a/packages/core/src/types.d.ts
+++ b/packages/core/src/types.d.ts
@@ -39,71 +39,6 @@ declare module 'hyperdb' {
export default HyperDB
}
-declare module 'corestore' {
- export default class Corestore {
- constructor(storage: string)
- namespace(name: string): Corestore
- replicate(socket: unknown, opts?: { live?: boolean }): unknown
- close(): Promise<void>
- }
-}
-
-declare module 'hyperdrive' {
- import Corestore from 'corestore'
-
- export interface HyperdriveBlobInfo {
- byteLength: number
- }
- export interface HyperdriveEntry {
- value?: { blob?: HyperdriveBlobInfo } | null
- }
- export interface HyperdriveDownload {
- done(): Promise<void>
- destroy(): void
- }
-
- export default class Hyperdrive {
- constructor(corestore: Corestore, key?: Uint8Array)
- ready(): Promise<void>
- readonly writable: boolean
- readonly key: Uint8Array
- readonly core?: { writable: boolean }
- entry(path: string): Promise<HyperdriveEntry | null>
- update(opts?: { wait?: boolean }): Promise<void>
- close(): Promise<void>
- download(path: string): HyperdriveDownload
- // Returns a Readable (not just a ReadableStream) so callers can destroy() on abort.
- createReadStream(path: string): import('stream').Readable
- }
-}
-
-declare module 'localdrive' {
- export interface LocaldriveBlobInfo {
- byteLength: number
- }
- export interface LocaldriveEntry {
- value?: { blob?: LocaldriveBlobInfo } | null
- }
-
- export default class Localdrive {
- constructor(dir: string)
- ready(): Promise<void>
- entry(path: string): Promise<LocaldriveEntry | null>
- close(): Promise<void>
- createWriteStream(path: string): import('stream').Writable
- }
-}
-
-declare module 'mirror-drive' {
- interface DriveLike {
- entry(path: string): Promise<unknown>
- }
- export default class MirrorDrive {
- constructor(source: DriveLike, dest: DriveLike, opts?: { entries?: string[]; prune?: boolean })
- done(): Promise<void>
- }
-}
-
declare module 'hyperswarm' {
import { EventEmitter } from 'events'
@@ -225,6 +160,10 @@ declare module 'bare-fs' {
interface MkdirOptions {
recursive?: boolean
}
+ interface Stats {
+ size: number
+ isFile(): boolean
+ }
export function rmSync(path: string, opts?: RmOptions): void
export function readFileSync(path: string, encoding: BufferEncoding): string
@@ -240,6 +179,7 @@ declare module 'bare-fs' {
writeFile(path: string, data: string, encoding: BufferEncoding): Promise<void>
rename(from: string, to: string): Promise<void>
mkdir(path: string, opts?: MkdirOptions): Promise<void>
+ stat(path: string): Promise<Stats>
}
const _default: {
diff --git a/packages/core/src/worklet/index.ts b/packages/core/src/worklet/index.ts
index cfdc208..5cb0d8b 100644
--- a/packages/core/src/worklet/index.ts
+++ b/packages/core/src/worklet/index.ts
@@ -82,7 +82,7 @@ if (relayKeyArg || relayEnabledArg) {
})
}
-const orchestrator = new TransferOrchestrator(sendTransferEvent, storageRoot, identityRoot, {
+const orchestrator = new TransferOrchestrator(sendTransferEvent, identityRoot, {
displayName,
deviceType
})
diff --git a/packages/core/src/worklet/peers/pairing-coordinator.ts b/packages/core/src/worklet/peers/pairing-coordinator.ts
index d54a261..5c8d6ef 100644
--- a/packages/core/src/worklet/peers/pairing-coordinator.ts
+++ b/packages/core/src/worklet/peers/pairing-coordinator.ts
@@ -37,8 +37,6 @@ export class PairingCoordinator {
this.emit = deps.emit
this.swarm = new TransferSwarm(
{
- onReady: () => Promise.resolve(),
- onReplicate: () => {},
onPeerConnected: (session) => this.onPeerConnected(session),
onPeerDisconnected: (peerKey) => this.onPeerDisconnected(peerKey),
onControlMessage: (message, session) => this.onControlMessage(message, session)
diff --git a/packages/core/src/worklet/rpc/protocol.ts b/packages/core/src/worklet/rpc/protocol.ts
index dcf3cb4..8c63896 100644
--- a/packages/core/src/worklet/rpc/protocol.ts
+++ b/packages/core/src/worklet/rpc/protocol.ts
@@ -26,7 +26,6 @@ export type { DeviceSecretInit }
export interface DownloadFileRequest {
transferId: string
fileId: string
- driveKey: string
path: string
name?: string
size?: number
diff --git a/packages/core/src/worklet/transfer/control-channel.ts b/packages/core/src/worklet/transfer/control-channel.ts
index 25450c5..c45f980 100644
--- a/packages/core/src/worklet/transfer/control-channel.ts
+++ b/packages/core/src/worklet/transfer/control-channel.ts
@@ -13,7 +13,6 @@ export interface FileOffer {
name: string
path: string
size: number
- driveKey: string
kind: 'file'
}
diff --git a/packages/core/src/worklet/transfer/control-validation.test.ts b/packages/core/src/worklet/transfer/control-validation.test.ts
index 1af2908..da27fa2 100644
--- a/packages/core/src/worklet/transfer/control-validation.test.ts
+++ b/packages/core/src/worklet/transfer/control-validation.test.ts
@@ -10,7 +10,6 @@ const validFileOffer = {
name: 'photo.jpg',
path: '/photo.jpg',
size: 1024,
- driveKey: 'abc123',
kind: 'file'
}
diff --git a/packages/core/src/worklet/transfer/control-validation.ts b/packages/core/src/worklet/transfer/control-validation.ts
index 7f9b44f..7a5f07c 100644
--- a/packages/core/src/worklet/transfer/control-validation.ts
+++ b/packages/core/src/worklet/transfer/control-validation.ts
@@ -53,7 +53,6 @@ function isValidFileOffer(x: unknown): x is FileOffer {
isSafeFileName(o.name) &&
isBoundedString(o.path, MAX_PATH_LEN) &&
isNonNegativeInteger(o.size) &&
- isBoundedString(o.driveKey, MAX_ID_LEN) &&
o.kind === 'file'
)
}
diff --git a/packages/core/src/worklet/transfer/legacy/downloader.ts b/packages/core/src/worklet/transfer/legacy/downloader.ts
deleted file mode 100644
index e9a1226..0000000
--- a/packages/core/src/worklet/transfer/legacy/downloader.ts
+++ /dev/null
@@ -1,218 +0,0 @@
-import Hyperdrive from 'hyperdrive'
-import type Corestore from 'corestore'
-import b4a from 'b4a'
-import fs from 'bare-fs'
-import { firstFreePath } from '@altersend/drive'
-import {
- AbortError,
- type AbortLike,
- getChunkSize,
- getDirname,
- getFileName,
- isValidHexKey,
- onAbort
-} from '../utils'
-import { fileExists } from '../fs-utils'
-import {
- DownloadReporter,
- type DownloaderCallbacks,
- type DownloadFileOutcome
-} from '../download-events'
-import type { DownloadFileRequest } from '../../rpc/protocol'
-
-export class LegacyHyperdriveDownloader {
- private readonly driveStore: Corestore
- private readonly remoteDrives = new Map<string, Hyperdrive>()
-
- constructor(driveStore: Corestore) {
- this.driveStore = driveStore
- }
-
- async download(
- file: DownloadFileRequest,
- targetPath: string,
- callbacks: DownloaderCallbacks,
- signal?: AbortLike
- ): Promise<DownloadFileOutcome> {
- const remoteDrive = await this.getRemoteDrive(file.driveKey)
- const resolvedName = file.name ?? getFileName(file.path)
- const announced = new DownloadReporter({
- file,
- callbacks,
- targetPath,
- totalBytes: file.size ?? 0
- })
-
- const entry = await remoteDrive.entry(file.path)
- if (!entry?.value?.blob) {
- return announced.failed(`Could not find remote file: ${resolvedName}`)
- }
-
- const actualBytes = entry.value.blob.byteLength
- if (typeof file.size === 'number' && file.size !== actualBytes) {
- return announced.failed(`Sender claimed ${file.size} bytes but file is ${actualBytes} bytes`)
- }
-
- const events = new DownloadReporter({
- file,
- callbacks,
- targetPath,
- totalBytes: actualBytes
- })
- events.started()
-
- const savedTo = await this.writeToDisk(
- remoteDrive,
- file.path,
- targetPath,
- actualBytes,
- (bytesTransferred) => events.progressed(bytesTransferred),
- signal,
- file.overwrite === true
- )
-
- events.completed(savedTo)
- return { ok: true, savedTo }
- }
-
- async evictDrive(driveKey: string): Promise<void> {
- const cached = this.remoteDrives.get(driveKey)
- if (!cached) return
- this.remoteDrives.delete(driveKey)
- try {
- await cached.close()
- } catch (err) {
- console.warn('LegacyHyperdrive: failed to close evicted drive', err)
- }
- }
-
- async closeAll(): Promise<void> {
- const drives = Array.from(this.remoteDrives.values())
- this.remoteDrives.clear()
- for (const drive of drives) {
- try {
- await drive.close()
- } catch (err) {
- console.warn('LegacyHyperdrive: close drive failed', err)
- }
- }
- }
-
- private async getRemoteDrive(driveKey: string): Promise<Hyperdrive> {
- if (!isValidHexKey(driveKey)) {
- throw new Error(`Invalid drive key format: expected 64 hex chars`)
- }
- const existingDrive = this.remoteDrives.get(driveKey)
- if (existingDrive) {
- try {
- await existingDrive.update({ wait: true })
- } catch (err) {
- console.warn('LegacyHyperdrive: update on cached drive failed, using cached state', err)
- }
- return existingDrive
- }
-
- const remoteDrive = new Hyperdrive(this.driveStore, b4a.from(driveKey, 'hex'))
- await remoteDrive.ready()
- try {
- await remoteDrive.update({ wait: true })
- } catch (err) {
- try {
- await remoteDrive.close()
- } catch {}
- throw new Error(
- `Could not sync with peer drive: ${err instanceof Error ? err.message : String(err)}`
- )
- }
- this.remoteDrives.set(driveKey, remoteDrive)
- return remoteDrive
- }
-
- private freeTargetPath(targetPath: string): Promise<string> {
- return firstFreePath(targetPath, async (candidate) => !(await fileExists(candidate)))
- }
-
- private async writeToDisk(
- remoteDrive: Hyperdrive,
- sourcePath: string,
- targetPath: string,
- totalBytes: number,
- onProgress: (bytesTransferred: number) => void,
- signal?: AbortLike,
- overwrite = false
- ): Promise<string> {
- const { default: Localdrive } = await import('localdrive')
- const destination = new Localdrive(getDirname(targetPath))
- const partName = `${getFileName(targetPath)}.part`
- const partPath = `${targetPath}.part`
-
- const readStream = remoteDrive.createReadStream(sourcePath)
- const writeStream = destination.createWriteStream(`/${partName}`)
-
- let release = () => {}
-
- try {
- await new Promise<void>((resolve, reject) => {
- let settled = false
- let bytesTransferred = 0
- let lastReportedBytes = 0
-
- const destroyStreams = () => {
- try {
- readStream.destroy()
- } catch {}
- try {
- writeStream.destroy()
- } catch {}
- }
-
- const finish = (err?: Error) => {
- if (settled) return
- settled = true
- if (!err) {
- resolve()
- return
- }
- destroyStreams()
- reject(err)
- }
-
- release = onAbort(signal, () => finish(new AbortError()))
-
- readStream.on('data', (chunk: unknown) => {
- bytesTransferred += getChunkSize(chunk)
- const nextBytes =
- totalBytes > 0 ? Math.min(bytesTransferred, totalBytes) : bytesTransferred
- if (nextBytes === totalBytes || nextBytes - lastReportedBytes >= 64 * 1024) {
- lastReportedBytes = nextBytes
- onProgress(nextBytes)
- }
- })
- readStream.once('error', finish)
- writeStream.once('error', finish)
- writeStream.once('close', () => {
- if (totalBytes > 0 && lastReportedBytes < totalBytes) {
- onProgress(totalBytes)
- }
- finish()
- })
- readStream.pipe(writeStream)
- })
- } catch (err) {
- try {
- await fs.promises.unlink(partPath)
- } catch (cleanupErr) {
- if ((cleanupErr as NodeJS.ErrnoException)?.code !== 'ENOENT') {
- console.warn('LegacyHyperdrive: failed to delete partial file', partPath, cleanupErr)
- }
- }
- throw err
- } finally {
- release()
- }
-
- const finalPath = overwrite ? targetPath : await this.freeTargetPath(targetPath)
- await fs.promises.rename(partPath, finalPath)
- return finalPath
- }
-}
diff --git a/packages/core/src/worklet/transfer/legacy/staging.ts b/packages/core/src/worklet/transfer/legacy/staging.ts
deleted file mode 100644
index 5d81f22..0000000
--- a/packages/core/src/worklet/transfer/legacy/staging.ts
+++ /dev/null
@@ -1,80 +0,0 @@
-import type Hyperdrive from 'hyperdrive'
-import type Localdrive from 'localdrive'
-import MirrorDrive from 'mirror-drive'
-import b4a from 'b4a'
-import { AbortError } from '../utils'
-import type { ScannedFile } from '../sender'
-
-export class LegacyPeerStaging {
- private readonly drive: Hyperdrive
- private pending: ScannedFile[] = []
- private running: Promise<void> | null = null
-
- constructor(drive: Hyperdrive) {
- this.drive = drive
- }
-
- get driveKey(): string {
- return b4a.toString(this.drive.key, 'hex')
- }
-
- stagedEntry(sourcePath: string) {
- return this.drive.entry(sourcePath)
- }
-
- setPendingFiles(files: ScannedFile[]): void {
- this.pending = files
- }
-
- stage(
- onStaging: (file: ScannedFile) => void,
- onDrained: () => Promise<void>,
- signal?: AbortSignal
- ): Promise<void> {
- if (!this.running) {
- const run: Promise<void> = this.run(onStaging, onDrained, signal).catch((err) => {
- if (this.running === run) this.running = null
- throw err
- })
- this.running = run
- }
- return this.running
- }
-
- reset(): void {
- this.pending = []
- this.running = null
- }
-
- private async run(
- onStaging: (file: ScannedFile) => void,
- onDrained: () => Promise<void>,
- signal?: AbortSignal
- ): Promise<void> {
- const files = this.pending
- const superseded = () => this.pending !== files
-
- try {
- for (const file of files) {
- if (signal?.aborted) throw new AbortError()
- onStaging(file)
- if (!file.alreadyStaged) await this.importToDrive(file.sourceDrive, file.sourcePath)
- }
- } catch (err) {
- if (superseded()) return
- throw err
- }
-
- if (superseded()) return
- this.pending = []
- await onDrained()
- }
-
- private async importToDrive(sourceDrive: Localdrive, sourcePath: string): Promise<void> {
- const mirror = new MirrorDrive(sourceDrive, this.drive, {
- entries: [sourcePath],
- prune: false
- })
- await mirror.done()
- }
-}
diff --git a/packages/core/src/worklet/transfer/orchestrator.ts b/packages/core/src/worklet/transfer/orchestrator.ts
index 054fe10..e530ef4 100644
--- a/packages/core/src/worklet/transfer/orchestrator.ts
+++ b/packages/core/src/worklet/transfer/orchestrator.ts
@@ -57,11 +57,11 @@ import {
} from './download-events'
import type { DownloadLifecycleEvent, DownloaderCallbacks } from './download-events'
import type { DriveChannel } from '@altersend/drive'
-import type { PeerDrive } from './drive'
import { PeerIdentityStore } from './peer-identity-store'
-import { TransferStorage } from './storage'
+import { TransferSender } from './sender'
+import { TransferReceiver } from './receiver'
import { TransferSwarm, type PeerSession } from './swarm'
-import { AbortError, isValidHexKey } from './utils'
+import { isValidHexKey } from './utils'
import {
DeviceIdentityStore,
type DeviceIdentityDefaults,
@@ -108,7 +108,8 @@ const LIFECYCLE_TEARDOWN_TIMEOUT_MS = 5000
export class TransferOrchestrator implements TransferRPC {
private readonly emitIPC: (message: TransferIPCMessage | PeerControlMessage) => void
- private readonly storage: TransferStorage
+ private readonly sender = new TransferSender()
+ private readonly receiver = new TransferReceiver()
private readonly offerPeers = new Map<string, string>()
private readonly swarm: TransferSwarm
@@ -123,7 +124,6 @@ export class TransferOrchestrator implements TransferRPC {
private suspended: boolean = false
private lifecycleQueue: Promise<unknown> = Promise.resolve()
private readonly inflight = new Set<AbortController>()
- private stagingAbort: AbortController | null = null
private readonly deviceIdentityStore: DeviceIdentityStore
private readonly rememberedStore: RememberedPeerStore
@@ -135,22 +135,15 @@ export class TransferOrchestrator implements TransferRPC {
constructor(
emitIPC: (message: TransferIPCMessage | PeerControlMessage) => void,
- storageRoot: string,
identityRoot: string,
identityDefaults: DeviceIdentityDefaults = {}
) {
this.emitIPC = emitIPC
- this.storage = new TransferStorage(storageRoot)
this.deviceIdentityStore = new DeviceIdentityStore(identityRoot, identityDefaults)
this.rememberedStore = new RememberedPeerStore(identityRoot)
const identityStore = new PeerIdentityStore(identityRoot)
this.swarm = new TransferSwarm(
{
- onReady: () => this.storage.ready(),
- onReplicate: (socket) => {
- if (this.suspended) return
- this.storage.replicate(socket)
- },
onPeerConnected: (session) => {
if (this.suspended) return
this.onPeerConnected(session)
@@ -381,7 +374,7 @@ export class TransferOrchestrator implements TransferRPC {
}
if (!session.drive) return
session.drive
- .serve(message.fileId, message.fileName, this.storage.sender.localPath(message.fileId))
+ .serve(message.fileId, message.fileName, this.sender.localPath(message.fileId))
.catch((err) =>
console.warn(
'TransferOrchestrator: drive send stopped',
@@ -391,17 +384,16 @@ export class TransferOrchestrator implements TransferRPC {
)
}
- private async driveChannelFor(file: DownloadFileRequest): Promise<DriveChannel | null> {
+ private async driveChannelFor(file: DownloadFileRequest): Promise<DriveChannel> {
const peerKey = this.offerPeers.get(file.transferId)
const drive = peerKey ? this.swarm.getSession(peerKey)?.drive : null
- if (!drive || !(await this.speaksDrive(drive))) return null
+ if (!drive) throw new Error('Lost the connection to the sender')
+ if (!(await drive.supported.catch(() => false))) {
+ throw new Error('Sender is running an older version of AlterSend and needs to update')
+ }
return drive.session(file.fileId)
}
- private speaksDrive(drive: PeerDrive | null | undefined): Promise<boolean> {
- return drive ? drive.supported.catch(() => false) : Promise.resolve(false)
- }
-
private setRole(role: TransferRole | null): void {
if (this.role === role) return
this.role = role
@@ -466,7 +458,6 @@ export class TransferOrchestrator implements TransferRPC {
}
async host(): Promise<HostReply> {
- await this.storage.ready()
const topic = this.swarm.generateKey()
this.currentTopic = topic
return { topic }
@@ -491,7 +482,6 @@ export class TransferOrchestrator implements TransferRPC {
throw new BadRequestError('Already in a session. Disconnect first.')
}
- await this.storage.ready()
this.setRole('receiver')
this.sendStatus('joining')
try {
@@ -518,16 +508,13 @@ export class TransferOrchestrator implements TransferRPC {
await this.disconnect()
}
- await this.storage.ready()
- this.stagingAbort?.abort()
- this.stagingAbort = null
- await this.storage.sender.reset()
+ await this.sender.reset()
try {
const fileRequests = requests.filter((r) => r.kind !== 'text')
const textRequests = requests.filter((r) => r.kind === 'text')
- const { files, totalBytes, errors } = await this.storage.sender.scanFiles(fileRequests)
+ const { files, totalBytes, errors } = await this.sender.scanFiles(fileRequests)
for (const error of errors) this.sendError(error)
if (files.length === 0 && textRequests.length === 0) {
@@ -548,7 +535,7 @@ export class TransferOrchestrator implements TransferRPC {
this.swarm.broadcast(this.activeTransfer)
this.sendStatus('sharing', { transferId })
- const offers: TransferOffer[] = this.storage.sender.buildOffers(files, transferId)
+ const offers: TransferOffer[] = this.sender.buildOffers(files, transferId)
for (const req of textRequests) {
if (!req.content || req.content.trim() === '') continue
@@ -600,33 +587,7 @@ export class TransferOrchestrator implements TransferRPC {
private offerTo(session: PeerSession): void {
if (!this.authedPeers.has(session.peerKey)) return
- const ready = this.activeTransferReady
- if (!ready) return
-
- this.stageForLegacyPeer(session)
- .then(() => {
- if (this.activeTransferReady !== ready) return
- session.controlChannel.send(ready)
- })
- .catch((err) => {
- if (err instanceof AbortError) return
- console.warn('TransferOrchestrator: staging for peer failed', err)
- this.sendError(err instanceof Error ? err.message : String(err))
- })
- }
-
- private async stageForLegacyPeer(session: PeerSession): Promise<void> {
- if (await this.speaksDrive(session.drive)) return
-
- const transferId = this.activeTransferReady?.transferId
- const controller = this.stagingAbort ?? new AbortController()
- this.stagingAbort = controller
-
- await this.storage.sender.stageForLegacyPeers(
- (file) =>
- this.sendStatus('sharing', { file: file.fileName, path: file.inputPath, transferId }),
- controller.signal
- )
+ if (this.activeTransferReady) session.controlChannel.send(this.activeTransferReady)
}
async downloadFiles(files: DownloadFileRequest[]): Promise<DownloadFilesReply> {
@@ -640,11 +601,11 @@ export class TransferOrchestrator implements TransferRPC {
const controller = new AbortController()
this.inflight.add(controller)
try {
- const results = await this.storage.receiver.downloadFiles(
+ const results = await this.receiver.downloadFiles(
files,
this.getDownloaderCallbacks(),
- controller.signal,
- (file) => this.driveChannelFor(file)
+ (file) => this.driveChannelFor(file),
+ controller.signal
)
return { files: results }
} finally {
@@ -656,8 +617,8 @@ export class TransferOrchestrator implements TransferRPC {
if (typeof fileId !== 'string' || fileId.length === 0) {
throw new BadRequestError('Missing fileId')
}
- const active = this.storage.receiver.activeDownloadIds()
- if (!this.storage.receiver.pause(fileId)) {
+ const active = this.receiver.activeDownloadIds()
+ if (!this.receiver.pause(fileId)) {
throw new BadRequestError(
`pauseDownload: no active download for "${fileId}". Active: [${active.join(', ')}]`
)
@@ -666,9 +627,6 @@ export class TransferOrchestrator implements TransferRPC {
abortInFlight(): void {
for (const session of this.swarm.sessions) session.drive?.cancel()
- this.stagingAbort?.abort()
- this.stagingAbort = null
-
for (const controller of this.inflight) controller.abort()
}
@@ -684,9 +642,8 @@ export class TransferOrchestrator implements TransferRPC {
this.offerPeers.clear()
await this.swarm.endSession()
- await this.storage.sender.reset()
- await this.storage.receiver.reset()
- await this.storage.wipeAndReinit()
+ await this.sender.reset()
+ await this.receiver.reset()
this.setRole(null)
this.sendStatus('disconnected')
@@ -703,7 +660,7 @@ export class TransferOrchestrator implements TransferRPC {
this.currentTopic = null
this.offerPeers.clear()
await this.swarm.endSession()
- await this.storage.receiver.discardPartials()
+ await this.receiver.discardPartials()
}
async setRelayConfig(input: SetRelayConfigInput): Promise<SetRelayConfigReply> {
@@ -779,6 +736,7 @@ export class TransferOrchestrator implements TransferRPC {
await tryAsync('pairing.destroy', () => this.pairing.destroy())
await tryAsync('swarm.destroy', () => this.swarm.destroy())
await tryAsync('rememberedStore.close', () => this.rememberedStore.close())
- await tryAsync('storage.destroy', () => this.storage.destroy())
+ await tryAsync('sender.reset', () => this.sender.reset())
+ await tryAsync('receiver.destroy', () => this.receiver.destroy())
}
}
diff --git a/packages/core/src/worklet/transfer/receiver.ts b/packages/core/src/worklet/transfer/receiver.ts
index 321fad5..815526f 100644
--- a/packages/core/src/worklet/transfer/receiver.ts
+++ b/packages/core/src/worklet/transfer/receiver.ts
@@ -1,4 +1,3 @@
-import type Corestore from 'corestore'
import {
receiveFile,
discardPartial,
@@ -18,7 +17,6 @@ import {
withDisplayName
} from './utils'
import { fileExists } from './fs-utils'
-import { LegacyHyperdriveDownloader } from './legacy/downloader'
import type { DownloadFileRequest, DownloadFileResult } from '../rpc/protocol'
import {
DownloadReporter,
@@ -38,21 +36,16 @@ function toMessage(error: unknown): string {
}
export class TransferReceiver {
- private readonly legacy: LegacyHyperdriveDownloader
private readonly partials = new Map<string, PartialDownload>()
private readonly active = new Map<string, AbortController>()
private readonly queued = new Set<string>()
private readonly paused = new Set<string>()
- constructor(driveStore: Corestore) {
- this.legacy = new LegacyHyperdriveDownloader(driveStore)
- }
-
async downloadFiles(
files: DownloadFileRequest[],
callbacks: DownloaderCallbacks,
- signal?: AbortLike,
- driveFor?: (file: DownloadFileRequest) => Promise<DriveChannel | null>
+ driveFor: (file: DownloadFileRequest) => Promise<DriveChannel>,
+ signal?: AbortLike
): Promise<DownloadFileResult[]> {
const results: DownloadFileResult[] = []
for (const file of files) {
@@ -111,15 +104,10 @@ export class TransferReceiver {
const releaseOuter = onAbort(signal, () => perFile.abort())
try {
- const channel = (await driveFor?.(file)) ?? null
- record(
- channel
- ? await this.downloadViaDrive(file, targetPath, callbacks, channel, perFile.signal)
- : await this.legacy.download(file, targetPath, callbacks, perFile.signal)
- )
+ const channel = await driveFor(file)
+ record(await this.downloadViaDrive(file, targetPath, callbacks, channel, perFile.signal))
} catch (error) {
const cancelled = error instanceof Error && error.name === 'AbortError'
- if (!cancelled) await this.legacy.evictDrive(file.driveKey)
record(
new DownloadReporter({
@@ -247,7 +235,6 @@ export class TransferReceiver {
this.queued.clear()
this.paused.clear()
await this.discardPartials()
- await this.legacy.closeAll()
}
async destroy(): Promise<void> {
diff --git a/packages/core/src/worklet/transfer/resume.test.ts b/packages/core/src/worklet/transfer/resume.test.ts
index 2b1faba..16024ad 100644
--- a/packages/core/src/worklet/transfer/resume.test.ts
+++ b/packages/core/src/worklet/transfer/resume.test.ts
@@ -4,7 +4,6 @@ import { tmpdir } from 'node:os'
import { join } from 'node:path'
import SecretStream from '@hyperswarm/secret-stream'
import { Duplex } from 'streamx'
-import type Corestore from 'corestore'
import type { PeerSocket } from 'hyperswarm'
import { PeerDrive } from './drive'
import { TransferReceiver } from './receiver'
@@ -30,17 +29,6 @@ function socketPair(): [PeerSocket, PeerSocket] {
]
}
-function unusableStore(): Corestore {
- return new Proxy(
- {},
- {
- get(_target, prop) {
- throw new Error(`Corestore.${String(prop)} was used on a drive-only transfer`)
- }
- }
- ) as unknown as Corestore
-}
-
function collect(): { callbacks: DownloaderCallbacks; errors: DownloadLifecycleEvent[] } {
const errors: DownloadLifecycleEvent[] = []
return {
@@ -72,14 +60,13 @@ describe('drive resume through TransferReceiver', () => {
const request: DownloadFileRequest = {
transferId: 't1',
fileId: 'f1',
- driveKey: 'a'.repeat(64),
path: '/in.bin',
name: 'in.bin',
size: input.length,
targetPath: join(dir, 'out.bin')
}
- const receiver = new TransferReceiver(unusableStore())
+ const receiver = new TransferReceiver()
const attempt = async (abortAfterFirstChunk: boolean) => {
const [socketA, socketB] = socketPair()
@@ -103,8 +90,8 @@ describe('drive resume through TransferReceiver', () => {
const [result] = await receiver.downloadFiles(
[request],
callbacks,
- controller.signal,
- async () => receiverSide.session(request.fileId)
+ async () => receiverSide.session(request.fileId),
+ controller.signal
)
return { result, errors, receivedBytes }
}
@@ -138,14 +125,13 @@ describe('pausing one download', () => {
const request: DownloadFileRequest = {
transferId: 't1',
fileId: 'f1',
- driveKey: 'a'.repeat(64),
path: '/in.bin',
name: 'in.bin',
size: input.length,
targetPath: dst
}
- const receiver = new TransferReceiver(unusableStore())
+ const receiver = new TransferReceiver()
const run = async (pauseOnFirstChunk: boolean) => {
const [socketA, socketB] = socketPair()
@@ -163,7 +149,7 @@ describe('pausing one download', () => {
callbacks.onFileProgress = () => receiver.pause(request.fileId)
}
- const [result] = await receiver.downloadFiles([request], callbacks, undefined, async () =>
+ const [result] = await receiver.downloadFiles([request], callbacks, async () =>
receiverSide.session(request.fileId)
)
return { result, errors }
@@ -212,7 +198,7 @@ describe('target naming', () => {
.serve(request.fileId, 'in.bin', src)
.catch((err) => console.error('test setup failed', err))
}
- const [result] = await receiver.downloadFiles([request], callbacks, undefined, async () =>
+ const [result] = await receiver.downloadFiles([request], callbacks, async () =>
receiverSide.session(request.fileId)
)
return result
@@ -221,7 +207,6 @@ describe('target naming', () => {
const baseRequest = (dst: string): DownloadFileRequest => ({
transferId: 't1',
fileId: 'f1',
- driveKey: 'a'.repeat(64),
path: '/in.bin',
name: 'in.bin',
size: 64,
@@ -235,7 +220,7 @@ describe('target naming', () => {
const dst = join(dir, 'out.bin')
await writeFile(`${dst}.part`, new Uint8Array(999))
- const result = await download(new TransferReceiver(unusableStore()), baseRequest(dst), src)
+ const result = await download(new TransferReceiver(), baseRequest(dst), src)
expect(result.ok).toBe(true)
expect(result.savedTo).toBe(dst)
expect(Buffer.from(await readFile(dst))).toEqual(Buffer.from(new Uint8Array(64).fill(6)))
@@ -249,7 +234,7 @@ describe('target naming', () => {
const dst = join(dir, 'out.bin')
await writeFile(dst, 'PRE-EXISTING')
- const result = await download(new TransferReceiver(unusableStore()), baseRequest(dst), src)
+ const result = await download(new TransferReceiver(), baseRequest(dst), src)
expect(result.ok).toBe(true)
expect(result.savedTo).toBe(join(dir, 'out (1).bin'))
expect(await readFile(dst, 'utf8')).toBe('PRE-EXISTING')
@@ -265,7 +250,6 @@ describe('pausing a file that has not started yet', () => {
const make = (id: string): DownloadFileRequest => ({
transferId: 't1',
fileId: id,
- driveKey: 'a'.repeat(64),
path: '/in.bin',
name: `${id}.bin`,
size: 4 * 1024 * 1024,
@@ -277,7 +261,7 @@ describe('pausing a file that has not started yet', () => {
const receiverSide = PeerDrive.create(socketB)!
await senderSide.supported
- const receiver = new TransferReceiver(unusableStore())
+ const receiver = new TransferReceiver()
const started: string[] = []
const { callbacks } = collect()
callbacks.onFileStart = (event) => {
@@ -293,7 +277,6 @@ describe('pausing a file that has not started yet', () => {
const results = await receiver.downloadFiles(
[make('f1'), make('f2')],
callbacks,
- undefined,
async (file) => receiverSide.session(file.fileId)
)
diff --git a/packages/core/src/worklet/transfer/sender.test.ts b/packages/core/src/worklet/transfer/sender.test.ts
index cc611ef..1a9fa08 100644
--- a/packages/core/src/worklet/transfer/sender.test.ts
+++ b/packages/core/src/worklet/transfer/sender.test.ts
@@ -1,50 +1,32 @@
-import { describe, it, expect, vi } from 'vitest'
-import { mkdtemp, writeFile, stat, rm } from 'node:fs/promises'
+import { describe, it, expect } from 'vitest'
+import { mkdtemp, writeFile, stat, rm, mkdir } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
-import type Hyperdrive from 'hyperdrive'
-import type Localdrive from 'localdrive'
-import { LegacyPeerStaging } from './legacy/staging'
import { TransferSender, type ScannedFile } from './sender'
-function untouchableDrive(): Hyperdrive {
- const key = new Uint8Array(32).fill(7)
- return new Proxy(
- { key },
- {
- get(target, prop) {
- if (prop === 'key') return target.key
- throw new Error(`Hyperdrive.${String(prop)} was used, so the file got copied`)
- }
- }
- ) as unknown as Hyperdrive
-}
-
function scanned(name: string, size: number): ScannedFile {
return {
fileName: name,
displayName: name,
inputPath: `/src/${name}`,
sourcePath: `/${name}`,
- sourceDrive: {} as Localdrive,
size
}
}
describe('TransferSender.buildOffers', () => {
- it('describes files without copying them into the Hyperdrive', () => {
- const sender = new TransferSender(untouchableDrive())
+ it('describes files without copying them anywhere', () => {
+ const sender = new TransferSender()
const offers = sender.buildOffers([scanned('a.bin', 10), scanned('b.bin', 20)], 't1')
expect(offers.map((offer) => offer.name)).toEqual(['a.bin', 'b.bin'])
expect(offers.map((offer) => offer.size)).toEqual([10, 20])
expect(offers.every((offer) => offer.transferId === 't1')).toBe(true)
- expect(offers.every((offer) => offer.driveKey === '07'.repeat(32))).toBe(true)
})
it('maps each offer back to the file on disk so drive can read it', () => {
- const sender = new TransferSender(untouchableDrive())
+ const sender = new TransferSender()
const [offer] = sender.buildOffers([scanned('a.bin', 10)], 't1')
@@ -53,7 +35,7 @@ describe('TransferSender.buildOffers', () => {
})
it('gives every offer a distinct id', () => {
- const sender = new TransferSender(untouchableDrive())
+ const sender = new TransferSender()
const offers = sender.buildOffers([scanned('a.bin', 1), scanned('a.bin', 1)], 't1')
@@ -61,7 +43,7 @@ describe('TransferSender.buildOffers', () => {
})
it('sends the picker display name on the wire, not the on-disk cache file name', () => {
- const sender = new TransferSender(untouchableDrive())
+ const sender = new TransferSender()
const [offer] = sender.buildOffers(
[
@@ -74,82 +56,49 @@ describe('TransferSender.buildOffers', () => {
})
})
-describe('TransferSender.reset', () => {
- it('deletes temporary files even when a drive peer meant nothing was staged', async () => {
- const dir = await mkdtemp(join(tmpdir(), 'sender-temp-'))
- const picked = join(dir, 'pic.jpg')
- await writeFile(picked, 'x')
+describe('TransferSender.scanFiles', () => {
+ it('measures files on disk and keeps the folder structure in the offer path', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'sender-scan-'))
+ await mkdir(join(dir, 'photos'))
+ const picked = join(dir, 'photos', 'pic.jpg')
+ await writeFile(picked, 'hello')
- const sender = new TransferSender(untouchableDrive())
- sender.buildOffers([{ ...scanned('pic.jpg', 1), inputPath: picked, isTemporary: true }], 't1')
+ const sender = new TransferSender()
+ const result = await sender.scanFiles([{ path: picked, relativePath: 'photos/pic.jpg' }])
- await sender.reset()
+ expect(result.errors).toEqual([])
+ expect(result.totalBytes).toBe(5)
+ expect(result.files[0].sourcePath).toBe('/photos/pic.jpg')
+ expect(result.files[0].size).toBe(5)
- await expect(stat(picked)).rejects.toThrow()
await rm(dir, { recursive: true, force: true })
})
-})
-describe('TransferSender.stageForLegacyPeers', () => {
- it('runs again for a later peer after a run was aborted', async () => {
- const sender = new TransferSender(untouchableDrive())
- sender.buildOffers([{ ...scanned('a.bin', 1), alreadyStaged: true }], 't1')
+ it('reports files it cannot read instead of offering them', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'sender-missing-'))
+
+ const sender = new TransferSender()
+ const result = await sender.scanFiles([{ path: join(dir, 'gone.bin') }, { path: dir }])
- const cancelled = new AbortController()
- cancelled.abort()
- await expect(sender.stageForLegacyPeers(() => {}, cancelled.signal)).rejects.toThrow()
+ expect(result.files).toEqual([])
+ expect(result.errors).toHaveLength(2)
- const onStaging = vi.fn()
- await sender.stageForLegacyPeers(onStaging, new AbortController().signal)
- expect(onStaging).toHaveBeenCalledTimes(1)
+ await rm(dir, { recursive: true, force: true })
})
+})
- it('reports every file, including ones already in the drive', async () => {
- const sender = new TransferSender(untouchableDrive())
- sender.buildOffers(
- [
- { ...scanned('a.bin', 1), alreadyStaged: true },
- { ...scanned('b.bin', 2), alreadyStaged: true }
- ],
- 't1'
- )
+describe('TransferSender.reset', () => {
+ it('deletes temporary files', async () => {
+ const dir = await mkdtemp(join(tmpdir(), 'sender-temp-'))
+ const picked = join(dir, 'pic.jpg')
+ await writeFile(picked, 'x')
- const onStaging = vi.fn()
- await sender.stageForLegacyPeers(onStaging, new AbortController().signal)
+ const sender = new TransferSender()
+ sender.buildOffers([{ ...scanned('pic.jpg', 1), inputPath: picked, isTemporary: true }], 't1')
- expect(onStaging.mock.calls.map(([file]) => file.fileName)).toEqual(['a.bin', 'b.bin'])
- })
-})
+ await sender.reset()
-describe('TransferSender staging races', () => {
- it('leaves a newer share alone when an older run finishes importing', async () => {
- let letImportFinish!: () => void
- const stuck = new Promise<void>((resolve) => {
- letImportFinish = resolve
- })
- const importing = vi
- .spyOn(
- LegacyPeerStaging.prototype as unknown as { importToDrive: () => Promise<void> },
- 'importToDrive'
- )
- .mockImplementation(() => stuck)
-
- try {
- const sender = new TransferSender(untouchableDrive())
- sender.buildOffers([scanned('old.bin', 1)], 'old')
- const old = sender.stageForLegacyPeers(() => {}, new AbortController().signal)
-
- await sender.reset()
- sender.buildOffers([scanned('new.bin', 1)], 'new')
-
- letImportFinish()
- await old
-
- const seen = vi.fn()
- await sender.stageForLegacyPeers(seen, new AbortController().signal)
- expect(seen.mock.calls.map(([file]) => file.fileName)).toEqual(['new.bin'])
- } finally {
- importing.mockRestore()
- }
+ await expect(stat(picked)).rejects.toThrow()
+ await rm(dir, { recursive: true, force: true })
})
})
diff --git a/packages/core/src/worklet/transfer/sender.ts b/packages/core/src/worklet/transfer/sender.ts
index 27c83cb..fecc18d 100644
--- a/packages/core/src/worklet/transfer/sender.ts
+++ b/packages/core/src/worklet/transfer/sender.ts
@@ -1,9 +1,6 @@
import crypto from 'hypercore-crypto'
import fs from 'bare-fs'
-import Localdrive from 'localdrive'
-import type Hyperdrive from 'hyperdrive'
-import { getDirname, getFileName, toRelativePath, toSafeFileName } from './utils'
-import { LegacyPeerStaging } from './legacy/staging'
+import { getFileName, toRelativePath, toSafeFileName } from './utils'
import type { FileOffer } from './control-channel'
export interface ScannedFile {
@@ -11,10 +8,8 @@ export interface ScannedFile {
displayName: string
inputPath: string
sourcePath: string
- sourceDrive: Localdrive
size: number
isTemporary?: boolean
- alreadyStaged?: boolean
}
export interface ScanResult {
@@ -27,43 +22,36 @@ function createFileId(): string {
return crypto.randomBytes(12).toString('hex')
}
-function resolveSource(
+function resolveRelativePath(
path: string,
requestedRelativePath: string | undefined,
fileName: string
-): { root: string; relativePath: string } {
+): string {
const normalizedPath = path.replace(/\\/g, '/')
const relativePath = toRelativePath(requestedRelativePath ?? fileName)
- const suffix = `/${relativePath}`
- if (relativePath && normalizedPath.endsWith(suffix)) {
- return {
- root: normalizedPath.slice(0, normalizedPath.length - suffix.length) || '/',
- relativePath
- }
- }
+ if (relativePath && normalizedPath.endsWith(`/${relativePath}`)) return relativePath
+ return fileName
+}
- return { root: getDirname(path), relativePath: fileName }
+async function readableSize(path: string): Promise<number | null> {
+ try {
+ const stats = await fs.promises.stat(path)
+ return stats.isFile() ? stats.size : null
+ } catch (err) {
+ console.warn(`TransferSender: cannot stat ${path}`, err)
+ return null
+ }
}
export class TransferSender {
- readonly legacy: LegacyPeerStaging
- private readonly scanDrives: Set<Localdrive> = new Set()
private readonly sourcePaths = new Map<string, string>()
private readonly temporaryPaths = new Set<string>()
- constructor(drive: Hyperdrive) {
- this.legacy = new LegacyPeerStaging(drive)
- }
-
localPath(fileId: string): string | null {
return this.sourcePaths.get(fileId) ?? null
}
- get driveKey(): string {
- return this.legacy.driveKey
- }
-
async scanFiles(
requests: { path: string; name?: string; relativePath?: string; isTemporary?: boolean }[]
): Promise<ScanResult> {
@@ -71,52 +59,22 @@ export class TransferSender {
const errors: string[] = []
let totalBytes = 0
- const driveByDir = new Map<string, Localdrive>()
-
for (const req of requests) {
- const path = req.path
- const fileName = getFileName(path)
+ const fileName = getFileName(req.path)
const displayName = req.name ? toSafeFileName(req.name, fileName) : fileName
- const { root, relativePath } = resolveSource(path, req.relativePath, fileName)
- const sourcePath = `/${relativePath}`
- let sourceDrive = driveByDir.get(root)
- if (!sourceDrive) {
- sourceDrive = new Localdrive(root)
- driveByDir.set(root, sourceDrive)
- this.scanDrives.add(sourceDrive)
- await sourceDrive.ready()
- }
- const entry = await sourceDrive.entry(sourcePath)
-
- if (!entry?.value?.blob) {
- const existingEntry = await this.legacy.stagedEntry(sourcePath)
- if (existingEntry?.value?.blob) {
- const size = existingEntry.value.blob.byteLength
- totalBytes += size
- files.push({
- fileName,
- displayName,
- inputPath: path,
- sourcePath,
- sourceDrive,
- size,
- isTemporary: req.isTemporary,
- alreadyStaged: true
- })
- continue
- }
+ const size = await readableSize(req.path)
+
+ if (size === null) {
errors.push(`Could not read file: ${fileName}`)
continue
}
- const size = entry.value.blob.byteLength
totalBytes += size
files.push({
fileName,
displayName,
- inputPath: path,
- sourcePath,
- sourceDrive,
+ inputPath: req.path,
+ sourcePath: `/${resolveRelativePath(req.path, req.relativePath, fileName)}`,
size,
isTemporary: req.isTemporary
})
@@ -125,24 +83,10 @@ export class TransferSender {
return { files, totalBytes, errors }
}
- async closeSourceDrives(): Promise<void> {
- const drives = Array.from(this.scanDrives)
- this.scanDrives.clear()
- for (const drive of drives) {
- try {
- await drive.close()
- } catch (err) {
- console.warn('TransferSender: close source drive failed', err)
- }
- }
- }
-
buildOffers(files: ScannedFile[], transferId: string): FileOffer[] {
- this.legacy.setPendingFiles(files)
-
return files.map((file) => {
const id = createFileId()
- if (!file.alreadyStaged) this.sourcePaths.set(id, file.inputPath)
+ this.sourcePaths.set(id, file.inputPath)
if (file.isTemporary) this.temporaryPaths.add(file.inputPath)
return {
id,
@@ -150,20 +94,13 @@ export class TransferSender {
name: file.displayName,
path: file.sourcePath,
size: file.size,
- driveKey: this.driveKey,
kind: 'file'
}
})
}
- stageForLegacyPeers(onStaging: (file: ScannedFile) => void, signal?: AbortSignal): Promise<void> {
- return this.legacy.stage(onStaging, () => this.closeSourceDrives(), signal)
- }
-
async reset(): Promise<void> {
this.sourcePaths.clear()
- this.legacy.reset()
- await this.closeSourceDrives()
const temporaries = Array.from(this.temporaryPaths)
this.temporaryPaths.clear()
diff --git a/packages/core/src/worklet/transfer/storage.ts b/packages/core/src/worklet/transfer/storage.ts
deleted file mode 100644
index 73f3808..0000000
--- a/packages/core/src/worklet/transfer/storage.ts
+++ /dev/null
@@ -1,68 +0,0 @@
-import Corestore from 'corestore'
-import fs from 'bare-fs'
-import Hyperdrive from 'hyperdrive'
-import type { PeerSocket } from 'hyperswarm'
-import { TransferSender } from './sender'
-import { TransferReceiver } from './receiver'
-
-async function tryAsync(label: string, op: () => Promise<unknown>): Promise<void> {
- try {
- await op()
- } catch (err) {
- console.warn(`TransferStorage: ${label} failed`, err)
- }
-}
-
-export class TransferStorage {
- private readonly root: string
- private coreStore!: Corestore
- private drive!: Hyperdrive
- private readyPromise!: Promise<void>
- private _sender!: TransferSender
- private _receiver!: TransferReceiver
-
- constructor(root: string) {
- this.root = root
- this.init()
- }
-
- get sender(): TransferSender {
- return this._sender
- }
-
- get receiver(): TransferReceiver {
- return this._receiver
- }
-
- ready(): Promise<void> {
- return this.readyPromise
- }
-
- replicate(socket: PeerSocket): void {
- this.coreStore.replicate(socket, { live: true })
- }
-
- async wipeAndReinit(): Promise<void> {
- await this.wipe()
- this.init()
- }
-
- async destroy(): Promise<void> {
- await tryAsync('receiver.destroy', () => this._receiver.destroy())
- await this.wipe()
- }
-
- private init(): void {
- this.coreStore = new Corestore(this.root)
- this.drive = new Hyperdrive(this.coreStore.namespace('outgoing-drive'))
- this.readyPromise = this.drive.ready()
- this._sender = new TransferSender(this.drive)
- this._receiver = new TransferReceiver(this.coreStore.namespace('incoming-drives'))
- }
-
- private async wipe(): Promise<void> {
- await tryAsync('drive.close', () => this.drive.close())
- await tryAsync('coreStore.close', () => this.coreStore.close())
- await tryAsync('storage rm', () => fs.promises.rm(this.root, { recursive: true, force: true }))
- }
-}
diff --git a/packages/core/src/worklet/transfer/swarm.ts b/packages/core/src/worklet/transfer/swarm.ts
index c01adc3..040341c 100644
--- a/packages/core/src/worklet/transfer/swarm.ts
+++ b/packages/core/src/worklet/transfer/swarm.ts
@@ -20,8 +20,6 @@ export interface PeerSession {
}
export interface TransferSwarmCallbacks {
- onReady: () => Promise<void>
- onReplicate: (socket: PeerSocket) => void
onPeerConnected: (session: PeerSession) => void
onPeerDisconnected: (peerKey: string | null, remainingCount: number) => void
onControlMessage: (message: PeerControlMessage, session: PeerSession) => void
@@ -80,10 +78,6 @@ export class TransferSwarm {
private async handleConnection(socket: PeerSocket, info: PeerInfo): Promise<void> {
const peerKey = b4a.toString(info.publicKey, 'hex')
- await this.callbacks.onReady()
-
- this.callbacks.onReplicate(socket)
-
let session: PeerSession | null = null
const controlChannel = PeerControlChannel.create(socket, (message) => {
if (!session) return
diff --git a/packages/core/src/worklet/transfer/utils.ts b/packages/core/src/worklet/transfer/utils.ts
index 3115fa4..ef33f68 100644
--- a/packages/core/src/worklet/transfer/utils.ts
+++ b/packages/core/src/worklet/transfer/utils.ts
@@ -109,16 +109,3 @@ export function onAbort(signal: AbortLike | undefined, abort: () => void): () =>
signal.addEventListener('abort', abort)
return () => signal.removeEventListener('abort', abort)
}
-
-export function getChunkSize(chunk: unknown): number {
- if (typeof chunk === 'string') return Buffer.byteLength(chunk)
- if (
- chunk &&
- typeof chunk === 'object' &&
- 'byteLength' in chunk &&
- typeof (chunk as { byteLength: unknown }).byteLength === 'number'
- ) {
- return (chunk as { byteLength: number }).byteLength
- }
- return 0
-}
diff --git a/packages/domain/src/receive/downloadModel.test.ts b/packages/domain/src/receive/downloadModel.test.ts
index 2714ea0..999d977 100644
--- a/packages/domain/src/receive/downloadModel.test.ts
+++ b/packages/domain/src/receive/downloadModel.test.ts
@@ -30,7 +30,6 @@ function offer(path: string, overrides: Partial<FileOffer> = {}): FileOffer {
name,
path,
size: 100,
- driveKey: 'd1',
...overrides
}
}
diff --git a/packages/domain/src/receive/downloadModel.ts b/packages/domain/src/receive/downloadModel.ts
index 4a4aacf..891ef43 100644
--- a/packages/domain/src/receive/downloadModel.ts
+++ b/packages/domain/src/receive/downloadModel.ts
@@ -563,7 +563,6 @@ export function createSingleDownloadRequest(
return {
transferId: file.transferId,
fileId: file.id,
- driveKey: file.driveKey,
path: file.path,
name: file.name,
size: file.size,
@@ -581,7 +580,6 @@ export function createDirectoryDownloadRequests(
.map((file) => ({
transferId: file.transferId,
fileId: file.id,
- driveKey: file.driveKey,
path: file.path,
name: file.name,
size: file.size,
diff --git a/packages/domain/src/transfer/activity.test.ts b/packages/domain/src/transfer/activity.test.ts
index ec474db..a7d96cf 100644
--- a/packages/domain/src/transfer/activity.test.ts
+++ b/packages/domain/src/transfer/activity.test.ts
@@ -17,7 +17,6 @@ const offer = (id: string, size = 1000): IncomingFileOffer => ({
name: `${id}.bin`,
path: `/files/${id}.bin`,
size,
- driveKey: 'drive-1',
kind: 'file'
})
diff --git a/packages/domain/src/transfer/binding.test.ts b/packages/domain/src/transfer/binding.test.ts
index 563a17c..9b12f66 100644
--- a/packages/domain/src/transfer/binding.test.ts
+++ b/packages/domain/src/transfer/binding.test.ts
@@ -13,7 +13,6 @@ const OFFER = {
name: 'a.bin',
path: 'a.bin',
size: 1000,
- driveKey: 'drive-1',
kind: 'file'
} as const
diff --git a/packages/domain/src/transfer/reducer.test.ts b/packages/domain/src/transfer/reducer.test.ts
index ff334fb..a60c75a 100644
--- a/packages/domain/src/transfer/reducer.test.ts
+++ b/packages/domain/src/transfer/reducer.test.ts
@@ -18,7 +18,6 @@ const offer = (id: string, name: string): IncomingFileOffer => ({
name,
path: `/files/${name}`,
size: 1024,
- driveKey: 'drive-1',
kind: 'file'
})