One storage API for Google Drive, Nextcloud, and S3-compatible clouds, in TypeScript and Python.
TypeScript byoccloud-storagecloudflare-r2encryptiongoogle-drivenextcloudoauth2pkcepythons3sdkstoragetypescriptwebdav
feat: local and in-memory providers, and a complete operation surface
Ajay Ramineni6f25571ce4ab
diff --git a/.gitignore b/.gitignoreindex 4224537..29cbe37 100644--- a/.gitignore+++ b/.gitignore@@ -51,3 +51,6 @@ yarn-error.log*delete/npm/pypi/++# Created by the runnable quickstart examples+byoc-demo-storage/diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7187e36..e7c0eba 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,57 @@ version is below `1.0.0`, the public API may change between minor releases.---+## [0.3.0] - 2026-08-27++The release that makes BYOC runnable without an account, and closes a set of+operations that existed in one SDK but not the other.++### Highlights++**Two providers that need no credentials.** `@byoc/local` and `@byoc/memory` on npm, `LocalFileSystemProvider` and `MemoryProvider` in the Python package. Both pass the same behavioural suite as the network-backed adapters and both certify clean against the provider compliance harness. Evaluating BYOC, developing offline, and unit-testing code that talks to storage no longer require signing up for anything.++**`copy()` is reachable.** It was implemented on all three TypeScript adapters in 0.2.x and exposed by neither client, so no caller could reach any of it. Both clients now expose it, and the Python S3 and Google Drive adapters have real implementations rather than a capability flag with nothing behind it.++**A conformance fixture for the operation surface.** The existing fixtures pin bytes, and an adapter missing a method still writes identical bytes for everything it does implement. `spec/fixtures/provider-operations.json` pins which operations and capabilities each adapter has, in both SDKs, so this class of divergence fails CI instead of shipping.++### Added++- `@byoc/local` / `LocalFileSystemProvider`: files on a local disk or mounted volume, with paths confined to the configured root+- `@byoc/memory` / `MemoryProvider`: an in-memory test double with `snapshot()`, `clear()`, and a configurable stream chunk size+- `copy()` on both clients, capability-gated like `move()`+- `walk()` on both clients: every object beneath a path, descending into folders+- `delete_tree()`: recursive delete, children before parents, failures collected rather than aborting+- `delete_many()`: concurrent batch delete returning per-path outcomes, because a partial delete is the common real case+- `signed_url()` / `getSignedUrl()` on both clients, gated on `public_urls`. It existed only on the S3 adapter before+- `copy()` and `move()` on the Python S3 adapter, using `x-amz-copy-source`+- `copy()` and `move()` on the Python Google Drive adapter, using `files.copy` and a parent-list `PATCH`+- `spec/fixtures/provider-operations.json`, and the conformance suites that read it in both SDKs+- Both new adapters accept an async iterator as upload input, making good on what `StorageInput` has always declared++### Fixed++- **S3 server-side copy truncated the copy source at a `#` or `?`.** `copyObject` built the `x-amz-copy-source` header with `encodeURI`, which leaves both characters intact, so copying `draft#2.pdf` asked S3 for `draft` and reported the object missing. The header is now RFC 3986 encoded per segment, the same rule the object URL already followed. Verified against MinIO in both SDKs.+- **The Python client dropped a provider that defined `__len__` while it was empty.** Registration tested the adapter for truthiness rather than `is not None`, so an empty `MemoryProvider` failed to register and the client reported that no provider had been supplied.+- **`S3CompatibleProvider` and `GoogleDriveProvider` in Python declared `server_side_copy=True` with no `copy` method.** Callers who feature-detected on the capability got an `AttributeError` instead of a clean `CapabilityUnsupportedError`.+- **The in-memory provider did not report nested keys as folders**, unlike S3, which returns them as `CommonPrefixes`. Anything below the first level was invisible to a tree walk, so `delete_tree()` reported complete success while leaving those objects in place. Caught before release by running the two providers through one shared suite.++### Changed++- Adapter manifests report `adapterVersion` / `adapter_version` `0.3.0`+- The Python S3 adapter's `presigned_url()` is now `signed_url()`, matching what the client calls. The old name still works as an alias.+- `AuthType` gained no new members: the `"local"` variant already existed and is now used++### Known limitations++- **E2EE still buffers the whole payload**, so client-side encryption and multi-gigabyte resumable uploads remain mutually exclusive. The design for a framed envelope is in [`docs/design/streaming-and-chunked-e2ee.md`](./docs/design/streaming-and-chunked-e2ee.md); the transport work is not in this release.+- **Streaming uploads are still buffered by the three network adapters.** Only the local and in-memory providers consume an async iterator without collecting it first.+- **`move` and `copy` of a filename containing `?` fail against wsgidav.** BYOC sends a correctly percent-encoded `Destination` header; wsgidav decodes it and then re-parses the result as a URL, truncating at the `?`. Confirmed with `curl` against the server directly, and reproducible in both SDKs. Other WebDAV servers may or may not share the defect.+- **The Python SDK is async only.** A synchronous facade for Celery, Django, and scripts is planned.+- **Two instances of the same provider type cannot be registered on one client**, since the registry is keyed by manifest id.+- **Google Drive is not covered by CI**, as it needs a real account and a browser consent step.++---+## [0.2.1] - 2026-08-26A packaging fix for the TypeScript SDK. No runtime behaviour changed, and the@@ -104,6 +155,7 @@ Initial release: the TypeScript storage abstraction, with Google Drive, S3-compa---+[0.3.0]: https://github.com/Ajayvarmaramineni/BYOC/releases/tag/v0.3.0[0.2.1]: https://github.com/Ajayvarmaramineni/BYOC/releases/tag/v0.2.1[0.2.0]: https://github.com/Ajayvarmaramineni/BYOC/releases/tag/v0.2.0[0.1.0]: https://github.com/Ajayvarmaramineni/BYOC/releases/tag/v0.1.0diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex bca4ea4..f234bf0 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -11,6 +11,7 @@ By participating you agree to the [Code of Conduct](./CODE_OF_CONDUCT.md).```textpackages/ TypeScript SDK, published to npm as @byoc/*+ (local/ and memory/ need no credentials -- develop against those)python/ Python SDK, published to PyPI as byoc-storagespec/fixtures/ conformance vectors shared by both SDKsdocs/ setup and integration guides@@ -49,8 +50,8 @@ Without activating the virtualenv, call the binaries directly: `.venv/bin/pytest## Running the full suite```-TypeScript 193 tests npm test && npm run typecheck-Python 224 tests pytest && mypy src && ruff check .+TypeScript 374 tests npm test && npm run typecheck+Python 361 tests pytest && mypy src && ruff check .```Integration and interop tests skip automatically when no server is reachable, so@@ -106,6 +107,14 @@ external state:| `provider-metadata.json` | metadata key names written into provider storage || `sigv4.json` | AWS request signing || `pkce.json` | the OAuth challenge derivation |+| `provider-operations.json` | which operations and capabilities each adapter has |++`provider-operations.json` is the odd one out: it pins no bytes. The byte-level+fixtures cannot catch an adapter that is simply missing a method, because the+bytes it writes for everything else are still identical. That blind spot is how+0.2.x shipped `copy()` on three TypeScript adapters and no Python ones, with+neither client exposing it at all. Adding an operation or an adapter means+updating that fixture and both SDKs together.**Changing an expectation in a fixture is a breaking change.** It can makepreviously written files unreadable. Bump the format version, asdiff --git a/README.md b/README.mdindex b10ac0b..a18e121 100644--- a/README.md+++ b/README.md@@ -11,7 +11,7 @@[](https://github.com/Ajayvarmaramineni/BYOC/actions)[](https://www.npmjs.com/package/@byoc/core)[](https://pypi.org/project/byoc-storage/)-[](#testing-and-verification)+[](#testing-and-verification)[](#testing-and-verification)[](LICENSE)@@ -57,6 +57,8 @@ BYOC ships two peer SDKs with the same capabilities. Pick your language, or use```bashnpm install @byoc/core+npm install @byoc/local+npm install @byoc/memorynpm install @byoc/google-drivenpm install @byoc/s3-compatiblenpm install @byoc/webdav@@ -81,6 +83,68 @@ All adapters are included.## Quick start+No account, no network, no credentials. The local and in-memory adapters are a+one-line install on npm and are bundled in the Python package, so you can run+BYOC before deciding whether you want it.++<table>+<tr><th>TypeScript</th><th>Python</th></tr>+<tr valign="top">+<td>++```ts+import { BYOC } from "@byoc/core";+import { LocalFileSystemProvider }+ from "@byoc/local";++const storage = new BYOC({+ provider: new LocalFileSystemProvider({+ rootDirectory: "./storage"+ })+});++await storage.connect();+await storage.writeText("hello.md", "# Hi");++console.log(+ await storage.readText("hello.md")+);+```++</td>+<td>++```python+from byoc import (+ AsyncBYOC,+ LocalFileSystemProvider,+)++storage = AsyncBYOC(+ provider=LocalFileSystemProvider(+ "./storage"+ )+)++async with storage:+ await storage.write_text(+ "hello.md", "# Hi"+ )+ print(+ await storage.read_text("hello.md")+ )+```++</td>+</tr>+</table>++Swap `LocalFileSystemProvider` for `S3CompatibleProvider`, `WebDAVProvider`, or+`GoogleDriveProvider` and none of the calling code changes. That substitution is+the entire point of BYOC, and it is the first thing you can check for yourself.++### Against a real provider+<table><tr><th>TypeScript</th><th>Python</th></tr><tr valign="top">@@ -157,6 +221,8 @@ The Python SDK is idiomatic Python, not a transliteration: `snake_case`, excepti| Provider | Ownership model | TypeScript | Python | Status || :--- | :--- | :--- | :--- | :--- |+| **Local filesystem** | Local disk / mounted volume | [`@byoc/local`](./packages/local) | `byoc.LocalFileSystemProvider` | Live-verified |+| **In-memory** | Test double | [`@byoc/memory`](./packages/memory) | `byoc.MemoryProvider` | Live-verified || **Google Drive** | Personal cloud | [`@byoc/google-drive`](./packages/google-drive) | `byoc.providers.gdrive` | Live-verified || **Cloudflare R2 / AWS S3 / MinIO / Wasabi** | Developer cloud | [`@byoc/s3-compatible`](./packages/s3-compatible) | `byoc.providers.s3` | Live-verified || **Nextcloud / ownCloud / WebDAV / Synology** | Self-hosted | [`@byoc/webdav`](./packages/webdav) | `byoc.providers.webdav` | Live-verified |@@ -221,11 +287,13 @@ FastAPI / Django / Next.js / Express- Virtual POSIX paths (`users/123/report.pdf`) resolved to each provider's native addressing, including Google Drive's opaque file IDs- Multi-provider registry with runtime switching, and stream-piped migration between any two providers- Paginated listing that follows continuation tokens past provider page caps+- Recursive `walk()` and `delete_tree()`, and a concurrent `delete_many()` that reports per-path outcomes instead of failing the batch+- Server-side `copy()` and `move()` on every adapter, so the bytes never travel through your process**Uploads**- Resumable chunked uploads with 256 KiB alignment, progress callbacks, and network-failure resumption- Automatic MIME detection, with per-upload overrides-- Presigned URLs for S3-compatible backends, so a browser can fetch without proxying bytes through you+- Signed URLs for S3-compatible backends, so a browser can fetch without proxying bytes through you**Security**- OAuth 2.0 with PKCE (RFC 7636) and CSRF state, using the non-restricted `drive.file` scope so you never enter Google's Restricted Scope assessment@@ -244,8 +312,9 @@ FastAPI / Django / Next.js / Express## Testing and verification```-TypeScript 193 tests tsc --strict-Python 224 tests mypy --strict, ruff+TypeScript 374 tests tsc --strict+Python 361 tests mypy --strict, ruff+Integration 38 tests live MinIO and WebDAV serversInterop 13 tests both SDKs, same live servers```@@ -254,6 +323,7 @@ Every adapter is exercised against a real server rather than a mock. Each of the- Object keys containing `#` or `?` were silently truncated, so `draft#2.pdf` and `draft#3.pdf` overwrote each other- SigV4 signatures were rejected when a caller passed a canonically-cased header- Google Drive queries broke on any filename containing an apostrophe+- S3 server-side copy truncated the copy source at a `#`, so `copy("draft#2.pdf", ...)` reported the object missing<table><tr><th>TypeScript</th><th>Python</th></tr>@@ -305,6 +375,8 @@ byoc/│ ├── google-drive/ # OAuth PKCE, virtual paths, resumable uploads│ ├── s3-compatible/ # SigV4 signer, R2 / S3 / MinIO client│ ├── webdav/ # Nextcloud, ownCloud, Synology adapter+│ ├── local/ # local filesystem, no credentials required+│ ├── memory/ # in-memory test double│ └── provider-sdk/ # certification harness for custom adapters├── python/ # Python SDK, published to PyPI as byoc-storage│ ├── src/byoc/ # client, paths, encryption, retry, providers/diff --git a/docs/design/streaming-and-chunked-e2ee.md b/docs/design/streaming-and-chunked-e2ee.mdnew file mode 100644index 0000000..b316593--- /dev/null+++ b/docs/design/streaming-and-chunked-e2ee.md@@ -0,0 +1,263 @@+# Design: streaming I/O and chunked E2EE++**Status:** proposed, targeting v0.3.0+**Supersedes:** the buffered upload path and the `BYOC_E2EE_V2` envelope++---++## The problem++BYOC cannot handle a file larger than available memory.++```+Python adapters reject streams outright: "Streaming uploads are not yet supported"+TypeScript S3 accepts Readable, but WebDAV and Drive still buffer+Python download buffers response.content, then hands back a one-chunk iterator+E2EE encrypt() takes the whole payload and returns the whole envelope+```++`StorageInput` in Python already *declares* `AsyncIterator[bytes]` and documents it+as streaming without buffering. That is currently untrue: the type is a promise the+adapters break. This design makes it true.++The consequence is not theoretical. A 150 MB lecture recording costs roughly 300 MB+of resident memory to encrypt, because the plaintext and the envelope both live in+memory at once. A 4 GB video is simply impossible.++Two independent problems hide behind one symptom:++1. **Transport buffering.** Adapters read the whole body before sending it.+2. **Cryptographic buffering.** AES-GCM produces one authentication tag over one+ message, so the whole plaintext must be processed before the tag is known.++The second is why `EncryptedStorageWrapper` reports `resumableUploads: false`.+Fixing transport alone would leave encryption unusable on large files.++---++## Part 1: the `BYOC_E2EE_V3` envelope++### Why the format has to change++AES-GCM authenticates one message with one tag. To decrypt a stream incrementally+you need many independently authenticated frames. That is a format change, not an+implementation change, so the magic header moves to `V3`.++Framing introduces three attacks that a single-message envelope does not have, and+the design has to close all three:++| Attack | Defence |+| :--- | :--- |+| Reorder frames | Frame index is bound as AAD |+| Drop trailing frames (truncation) | A final-frame flag is bound as AAD |+| Swap the header between files | The whole header is bound as AAD in every frame |++### Layout++```+Header, 44 bytes, cleartext but authenticated:++ offset size field+ ------ ---- ---------------------------------------------+ 0 12 MAGIC "BYOC_E2EE_V3"+ 12 4 ITERATIONS uint32 BE, PBKDF2 work factor+ 16 4 FRAME_SIZE uint32 BE, plaintext bytes per frame+ 20 16 SALT random, per file+ 36 8 NONCE_BASE random, per file+ ------------------------------------------------------+ 44 end of header++Then one or more frames:++ offset size field+ ------ ---- ---------------------------------------------+ 0 4 FRAME_LEN uint32 BE, ciphertext byte count+ 4 16 TAG GCM authentication tag+ 20 n CIPHERTEXT FRAME_LEN bytes+```++Per-frame overhead is 20 bytes. At the default 256 KiB frame that is 0.008%.++### Key and nonce derivation++```+key = PBKDF2-HMAC-SHA256(passphrase, SALT, ITERATIONS, dkLen=32)+nonce[i] = NONCE_BASE (8 bytes) || uint32_BE(i) -> 12 bytes+aad[i] = HEADER (44 bytes) || uint32_BE(i) || is_final (1 byte)+```++The key is derived once per file and reused across frames. That is safe because the+nonce is unique per frame; it is also what makes streaming affordable, since a+600,000-iteration PBKDF2 per frame would be unusable.++`NONCE_BASE` is defence in depth. The salt is random per file, so the key already+differs per file and a bare counter would be sufficient. The extra 8 random bytes+mean that an implementation which incorrectly caches a derived key across files+still does not reuse a nonce. Nonce reuse under GCM is catastrophic, so the 8 bytes+are cheap insurance against a plausible future bug.++### Truncation detection++The last frame sets `is_final = 1`; every other frame sets `0`. Because the flag is+bound as AAD, an attacker who deletes trailing frames leaves a stream whose last+delivered frame authenticates as non-final. The reader treats "input ended without a+final frame" as tampering, not as a clean end of file.++**An empty payload still emits exactly one frame**, containing zero plaintext bytes+and `is_final = 1`. Without it, a zero-frame envelope would have nothing+authenticating the header.++### Reading untrusted input++`FRAME_LEN` comes from storage the attacker may control and drives an allocation.+This is the same class of bug as the iteration count in V2, where a 4-byte edit could+force minutes of PBKDF2. Both fields are range-checked before they are used:++```+ITERATIONS 10_000 .. 2_000_000 (unchanged from V2)+FRAME_SIZE 4_096 .. 8_388_608 (4 KiB .. 8 MiB)+FRAME_LEN 0 .. FRAME_SIZE + 16 (a frame cannot exceed its declared size)+```++A violation raises `CORRUPTED_DATA` before any key derivation or allocation.++### Compatibility++`decrypt` dispatches on the magic header and keeps reading V1 and V2. `encrypt`+always writes V3. Nothing previously written becomes unreadable.++The buffered `encrypt_sync` / `decrypt_sync` entry points stay, implemented as thin+wrappers that feed a single-shot iterator through the streaming path. Small payloads+keep the simple API.++---++## Part 2: streaming transport++### The layers are independent++This matters and is easy to get wrong:++```+plaintext -> E2EE frames (256 KiB) -> provider chunks (8 MiB) -> HTTP+```++Frame boundaries and upload-chunk boundaries have nothing to do with each other.+Google Drive requires resumable chunks to be 256 KiB-aligned; ciphertext frames are+256 KiB *plus 20 bytes*, so they never align. That is fine. The uploader chunks the+ciphertext byte stream without knowing frames exist.++Keeping these layers separate is what lets encryption and resumable upload compose+at all, which is exactly what V2 could not do.++### Per-provider work++**WebDAV** is straightforward. A `PUT` with a chunked request body. `httpx` and+`undici` both accept an async iterator directly.++**Google Drive** already chunks for resumable uploads. The change is feeding+`upload_chunks` from an iterator rather than slicing a `bytes` object, and dropping+the `len(payload)` dependency by tracking offsets as chunks are consumed.++**S3 needs a decision, because SigV4 hashes the body.**++Signature Version 4 signs `sha256(body)`. You cannot compute that without reading+the entire body, which defeats streaming. There are two ways out:++| Option | Cost |+| :--- | :--- |+| `STREAMING-AWS4-HMAC-SHA256-PAYLOAD` | Per-chunk signatures, a second framing layer inside the request body, meaningful complexity |+| `UNSIGNED-PAYLOAD` | One header change; the body is not covered by the signature |++**Recommendation: `UNSIGNED-PAYLOAD`, for streaming uploads only.** The request is+still authenticated, and over TLS the body is still confidential and integrity-protected+by the transport. This is what most S3 SDKs do for streams. Buffered uploads keep+signing the real body hash, so nothing regresses for existing callers.++This is a security-relevant tradeoff and belongs in `SECURITY.md`, not buried in an+adapter. Callers who need body-level signing can pass `bytes` and get the current+behaviour.++### Downloads++`StorageOutput.stream()` becomes genuinely lazy. Today Python reads+`response.content` up front and hands back an iterator over one chunk, so the+"streaming" accessor buffers. The response context has to stay open until the caller+finishes consuming, which means `download()` returns an object that owns the+response rather than one built after the fact.++`read()` and `text()` keep working by draining the stream.++### Migration++Python's migration currently does `await output.read()`. With streaming downloads it+pipes source to target directly, matching what TypeScript already does. That removes+the last place a whole file is held in memory.++---++## Part 3: conformance++The current fixtures pin byte formats. They cannot catch the divergence that already+exists, where TypeScript streams and Python does not, because both produce the same+stored bytes. Streaming needs behavioural coverage.++**`spec/fixtures/e2ee-envelope.json` gains V3 vectors.** With `SALT` and `NONCE_BASE`+fixed, the envelope is byte-reproducible, so both SDKs can assert an exact hex match+as they do for V2 today. Add vectors for: an empty payload, a payload smaller than+one frame, and a payload spanning three frames.++**New rejection cases**, each of which must raise before any allocation:++- final frame removed (truncation)+- frames 1 and 2 swapped (reordering)+- `FRAME_LEN` set to `0xFFFFFFFF` (oversized allocation)+- `FRAME_SIZE` set to 16 (below the floor)+- one header byte flipped (header tampering, caught by AAD)++**New `spec/fixtures/streaming.json`.** The invariant is that transport shape must not+change stored bytes:++```+upload(path, bytes) -> object A+upload(path, iterator over same bytes) -> object B+assert A == B, byte for byte+```++Run in both SDKs, against MinIO and WebDAV, with chunk sizes chosen to fall on and+off frame boundaries. This is the test that would have caught the current divergence.++---++## Sequencing++Each step lands green and independently useful.++| Step | Work | Why this order |+| :--- | :--- | :--- |+| 1 | V3 envelope in Python, buffered API on top | Format first, no transport changes to confuse failures |+| 2 | V3 in TypeScript, cross-SDK vectors | Locks the format before anything depends on it |+| 3 | Streaming download, both SDKs | Simpler than upload, no signing questions |+| 4 | Streaming upload: WebDAV, then Drive, then S3 | Increasing difficulty; S3 last because of the SigV4 decision |+| 5 | `streaming.json` and interop coverage | Proves the SDKs did not drift |+| 6 | Wire E2EE into the streaming path | Only now can `resumableUploads` stop being forced to `false` |+| 7 | Python migration streams | Removes the last full-file buffer |++Steps 1 and 2 are the risky ones. Step 6 is the payoff: `EncryptedStorageWrapper`+stops overriding `resumableUploads` to `false`, and encryption composes with+multi-gigabyte uploads.++## Open questions++**Default frame size.** 256 KiB matches Drive's alignment and keeps overhead at+0.008%. Larger frames mean less overhead and more memory held per frame. The default+should be set from measurements against representative media files, not chosen from+the alignment constant alone.++**Does `FRAME_SIZE` belong in the header at all?** It could be implied by the first+frame's length. Storing it explicitly costs 4 bytes and lets a reader validate every+frame against a declared bound, which is worth more than the bytes.++**Progress reporting for streams.** `UploadProgress.total_bytes` is currently derived+from `len(payload)`. A stream of unknown length cannot provide it. The field becomes+`None`, and `percentage` with it. Callers relying on a percentage need to know that.diff --git a/examples/node-quickstart/index.ts b/examples/node-quickstart/index.tsindex c682c0b..2a478a4 100644--- a/examples/node-quickstart/index.ts+++ b/examples/node-quickstart/index.ts@@ -8,10 +8,31 @@ import {} from "@byoc/google-drive";import { S3CompatibleProvider } from "@byoc/s3-compatible";import { WebDAVProvider } from "@byoc/webdav";+import { LocalFileSystemProvider } from "@byoc/local";+import { MemoryProvider } from "@byoc/memory";async function main() {console.log("=== BYOC (Bring Your Own Cloud) Ecosystem Demo ===\n");+ // 0. A real round trip, with no account and no network.+ // Everything below this block uses placeholder credentials and only+ // demonstrates shapes; this part genuinely reads and writes.+ console.log("0. Real storage round trip, no credentials required:");+ const scratch = new BYOC({ provider: new MemoryProvider() });+ await scratch.connect();++ await scratch.writeText("reports/q3.md", "# Q3 results");+ await scratch.copy("reports/q3.md", "reports/q3-backup.md");+ console.log(" read back: ", await scratch.readText("reports/q3-backup.md"));++ const walked: string[] = [];+ for await (const item of scratch.walk("reports")) walked.push(item.path);+ console.log(" walk: ", walked.join(", "));++ const removed = await scratch.deleteTree("reports");+ console.log(` deleteTree: removed ${removed.deleted.length}, failed ${removed.failed.length}`);+ console.log(" Swap MemoryProvider for LocalFileSystemProvider and it writes real files.\n");+// 1. Generate PKCE Security Parametersconst codeVerifier = generateCodeVerifier(64);const codeChallenge = await generateCodeChallenge(codeVerifier);@@ -54,9 +75,12 @@ async function main() {rootFolder: "MyApp"});- // 3. Initialize Universal BYOC Client with all 3 Clouds+ // Local disk: no account, no network, no credentials.+ const localDisk = new LocalFileSystemProvider({ rootDirectory: "./byoc-demo-storage" });++ // 3. Initialize Universal BYOC Client with every ownership modelconst storage = new BYOC({- providers: [googleDrive, cloudflareR2, nextcloud],+ providers: [googleDrive, cloudflareR2, nextcloud, localDisk],defaultProviderId: "google-drive"});@@ -79,6 +103,9 @@ async function main() {storage.useProvider("webdav");console.log(" Switched active provider to:", storage.manifest().name, `(${storage.manifest().category})`);+ storage.useProvider("local");+ console.log(" Switched active provider to:", storage.manifest().name, `(${storage.manifest().category})`);+console.log("\n5. Ready to migrate data between user clouds and developer clouds with zero code changes!");}diff --git a/examples/node-quickstart/package.json b/examples/node-quickstart/package.jsonindex b2d5c7b..954c685 100644--- a/examples/node-quickstart/package.json+++ b/examples/node-quickstart/package.json@@ -1,6 +1,6 @@{"name": "example-node-quickstart",- "version": "0.2.1",+ "version": "0.3.0","private": true,"description": "Quickstart example consuming @byoc/core and @byoc/google-drive","main": "index.ts",@@ -9,9 +9,11 @@"build": "tsc -b"},"dependencies": {- "@byoc/core": "0.2.1",- "@byoc/google-drive": "0.2.1",- "@byoc/s3-compatible": "0.2.1",- "@byoc/webdav": "0.2.1"+ "@byoc/core": "0.3.0",+ "@byoc/google-drive": "0.3.0",+ "@byoc/local": "0.3.0",+ "@byoc/memory": "0.3.0",+ "@byoc/s3-compatible": "0.3.0",+ "@byoc/webdav": "0.3.0"}}diff --git a/examples/python-quickstart/main.py b/examples/python-quickstart/main.pyindex a3b1c4f..2717458 100644--- a/examples/python-quickstart/main.py+++ b/examples/python-quickstart/main.py@@ -20,7 +20,7 @@ from __future__ import annotationsimport asyncioimport os-from byoc import AsyncBYOC, E2EECrypto+from byoc import AsyncBYOC, E2EECrypto, LocalFileSystemProvider, MemoryProviderfrom byoc.providers.gdrive import (GoogleDriveProvider,generate_code_challenge,@@ -41,6 +41,8 @@ def build_providers() -> list[object]:storage call is actually made."""return [+ # No account, no network, no credentials.+ LocalFileSystemProvider("./byoc-demo-storage"),GoogleDriveProvider(client_id="your-client-id.apps.googleusercontent.com",redirect_uri="http://localhost:8765/callback",@@ -65,6 +67,23 @@ def build_providers() -> list[object]:async def main() -> None:print("=== BYOC (Bring Your Own Cloud) Python quickstart ===\n")+ # 0. A real round trip, with no account and no network. Everything below+ # this block uses placeholder credentials and only demonstrates shapes;+ # this part genuinely reads and writes.+ print("0. Real storage round trip, no credentials required:")+ scratch = AsyncBYOC(provider=MemoryProvider())+ async with scratch:+ await scratch.write_text("reports/q3.md", "# Q3 results")+ await scratch.copy("reports/q3.md", "reports/q3-backup.md")+ print(f" read back: {await scratch.read_text('reports/q3-backup.md')!r}")++ walked = [item.path async for item in scratch.walk("reports")]+ print(f" walk: {', '.join(walked)}")++ removed = await scratch.delete_tree("reports")+ print(f" delete_tree: removed {len(removed.deleted)}, failed {len(removed.failed)}")+ print(" Swap MemoryProvider for LocalFileSystemProvider and it writes real files.\n")+# 1. PKCE, the handshake that protects the OAuth authorization code.verifier = generate_code_verifier(64)print("1. PKCE security handshake:")@@ -81,7 +100,7 @@ async def main() -> None:# 3. Capabilities are declared, so callers feature-detect instead of guessing.print("3. Capabilities differ per provider, and BYOC reports them honestly:")- for provider_id in ("google-drive", "s3-compatible", "webdav"):+ for provider_id in ("google-drive", "s3-compatible", "webdav", "local"):caps = storage.use_provider(provider_id).capabilities()print(f" {provider_id:16} folders={caps.folders!s:5} "@@ -91,7 +110,7 @@ async def main() -> None:# 4. Switching backends changes nothing about the calling code.print("4. Runtime provider switching:")- for provider_id in ("s3-compatible", "webdav", "google-drive"):+ for provider_id in ("s3-compatible", "webdav", "local", "google-drive"):storage.use_provider(provider_id)print(f" active: {storage.manifest().name}")print()diff --git a/package-lock.json b/package-lock.jsonindex 14d050a..a584523 100644--- a/package-lock.json+++ b/package-lock.json@@ -1,12 +1,12 @@{"name": "byoc-monorepo",- "version": "0.2.1",+ "version": "0.3.0","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "byoc-monorepo",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","workspaces": ["packages/*",@@ -23,12 +23,14 @@},"examples/node-quickstart": {"name": "example-node-quickstart",- "version": "0.2.1",+ "version": "0.3.0","dependencies": {- "@byoc/core": "0.2.1",- "@byoc/google-drive": "0.2.1",- "@byoc/s3-compatible": "0.2.1",- "@byoc/webdav": "0.2.1"+ "@byoc/core": "0.3.0",+ "@byoc/google-drive": "0.3.0",+ "@byoc/local": "0.3.0",+ "@byoc/memory": "0.3.0",+ "@byoc/s3-compatible": "0.3.0",+ "@byoc/webdav": "0.3.0"}},"node_modules/@byoc/core": {@@ -39,6 +41,14 @@"resolved": "packages/google-drive","link": true},+ "node_modules/@byoc/local": {+ "resolved": "packages/local",+ "link": true+ },+ "node_modules/@byoc/memory": {+ "resolved": "packages/memory",+ "link": true+ },"node_modules/@byoc/provider-sdk": {"resolved": "packages/provider-sdk","link": true@@ -1716,7 +1726,7 @@},"packages/core": {"name": "@byoc/core",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","dependencies": {"@types/node": ">=20"@@ -1727,22 +1737,45 @@},"packages/google-drive": {"name": "@byoc/google-drive",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","dependencies": {- "@byoc/core": "^0.2.1",+ "@byoc/core": "^0.3.0","@types/node": ">=20"},"engines": {"node": ">=20"}},+ "packages/local": {+ "name": "@byoc/local",+ "version": "0.3.0",+ "license": "Apache-2.0",+ "dependencies": {+ "@byoc/core": "^0.3.0",+ "@types/node": ">=20"+ },+ "engines": {+ "node": ">=20"+ }+ },+ "packages/memory": {+ "name": "@byoc/memory",+ "version": "0.3.0",+ "license": "Apache-2.0",+ "dependencies": {+ "@byoc/core": "^0.3.0"+ },+ "engines": {+ "node": ">=20"+ }+ },"packages/provider-sdk": {"name": "@byoc/provider-sdk",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"engines": {"node": ">=20"@@ -1750,10 +1783,10 @@},"packages/s3-compatible": {"name": "@byoc/s3-compatible",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"engines": {"node": ">=20"@@ -1761,10 +1794,10 @@},"packages/webdav": {"name": "@byoc/webdav",- "version": "0.2.1",+ "version": "0.3.0","license": "Apache-2.0","dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"engines": {"node": ">=20"diff --git a/package.json b/package.jsonindex 1ac7a2b..2846bc3 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "byoc-monorepo",- "version": "0.2.1",+ "version": "0.3.0","private": true,"description": "BYOC (Bring Your Own Cloud) \u2014 Universal Storage Abstraction Layer","license": "Apache-2.0",@@ -16,7 +16,7 @@"build": "npm run build --workspaces --if-present","test": "vitest run","test:watch": "vitest",- "typecheck": "tsc -b packages/core packages/google-drive packages/s3-compatible packages/webdav packages/provider-sdk examples/node-quickstart",+ "typecheck": "tsc -b packages/core packages/google-drive packages/s3-compatible packages/webdav packages/local packages/memory packages/provider-sdk examples/node-quickstart","clean": "rm -rf node_modules packages/*/node_modules packages/*/dist examples/*/node_modules"},"devDependencies": {diff --git a/packages/core/package.json b/packages/core/package.jsonindex 32aaecf..307e768 100644--- a/packages/core/package.json+++ b/packages/core/package.json@@ -1,6 +1,6 @@{"name": "@byoc/core",- "version": "0.2.1",+ "version": "0.3.0","description": "Core universal storage abstraction for BYOC (Bring Your Own Cloud)","type": "module","main": "./dist/index.js",diff --git a/packages/core/src/client/byoc.ts b/packages/core/src/client/byoc.tsindex 6b241d1..6713910 100644--- a/packages/core/src/client/byoc.ts+++ b/packages/core/src/client/byoc.ts@@ -9,7 +9,10 @@ import type {StorageOutput,StorageQuota,UploadOptions,- BackupOptions+ BackupOptions,+ BatchDeleteReport,+ BatchFailure,+ SignedUrlOptions} from "../types/storage.js";import { normalizeVirtualPath } from "../paths/resolver.js";import { BYOCErrorCode } from "../errors/codes.js";@@ -339,6 +342,24 @@ export class BYOC {/*** Move an object from source to destination virtual path.*/+ /**+ * Copies an object. Every current adapter performs this server-side, so the+ * bytes never travel through this process.+ */+ public async copy(source: string, destination: string): Promise<void> {+ if (!this.currentProvider.copy) {+ throw new StorageError({+ code: BYOCErrorCode.CAPABILITY_UNSUPPORTED,+ message: `Provider '${this.manifest().name}' does not support native copy operations.`,+ provider: this.manifest().id,+ retryable: false+ });+ }+ const normSource = normalizeVirtualPath(source);+ const normDest = normalizeVirtualPath(destination);+ return this.currentProvider.copy(normSource, normDest);+ }+public async move(source: string, destination: string): Promise<void> {if (!this.currentProvider.move) {throw new StorageError({@@ -369,6 +390,133 @@ export class BYOC {return this.currentProvider.quota();}+ /**+ * A time-limited URL a browser can use directly, without proxying bytes.+ *+ * Only providers reporting `publicUrls` can issue one. Google Drive and+ * WebDAV cannot, so they throw rather than returning a URL that would need+ * the caller's credentials to be useful.+ */+ public async getSignedUrl(path: string, options?: SignedUrlOptions): Promise<string> {+ const capabilities = await this.capabilities();+ const signer = (this.currentProvider as { getSignedUrl?: unknown }).getSignedUrl;++ if (!capabilities.publicUrls || typeof signer !== "function") {+ throw new StorageError({+ code: BYOCErrorCode.CAPABILITY_UNSUPPORTED,+ message: `Provider '${this.manifest().name}' cannot issue signed URLs.`,+ provider: this.manifest().id,+ retryable: false+ });+ }++ return (signer as (p: string, o?: SignedUrlOptions) => string).call(+ this.currentProvider,+ normalizeVirtualPath(path),+ options+ );+ }++ /**+ * Yields every object beneath `path`, descending into folders.+ *+ * `list` returns one level, which is right for rendering a file browser and+ * wrong for "everything under here". This walks the tree breadth-first,+ * yielding folders before their contents.+ *+ * It is built on `list`, so it costs one call per folder.+ */+ public async *walk(path?: string): AsyncGenerator<StorageObject> {+ const pending: (string | undefined)[] = [normalizeVirtualPath(path ?? "") || undefined];++ while (pending.length > 0) {+ const current = pending.shift();+ for (const item of await this.currentProvider.list(current)) {+ yield item;+ if (item.type === "folder") pending.push(item.path);+ }+ }+ }++ /**+ * Deletes everything under `path`, then `path` itself.+ *+ * Children are deleted before their parents, so a provider that refuses to+ * remove a non-empty folder still ends up with an empty one to remove.+ * Failures are collected rather than aborting the walk.+ */+ public async deleteTree(path: string): Promise<BatchDeleteReport> {+ const normalized = normalizeVirtualPath(path);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Delete tree requires a valid non-empty path.",+ provider: this.manifest().id,+ retryable: false+ });+ }++ const descendants: StorageObject[] = [];+ for await (const item of this.walk(normalized)) descendants.push(item);++ // Deepest first: a folder must be emptied before it can be removed.+ descendants.sort(+ (a, b) => b.path.split("/").length - a.path.split("/").length+ );++ return this.deleteMany([...descendants.map((item) => item.path), normalized], 1);+ }++ /**+ * Deletes several paths, reporting per-path outcomes.+ *+ * One failure does not abort the rest: a locked or already-removed object is+ * the common case, and the caller needs to know which paths survived rather+ * than losing the whole batch.+ */+ public async deleteMany(paths: string[], concurrency = 8): Promise<BatchDeleteReport> {+ if (concurrency < 1) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "deleteMany concurrency must be at least 1.",+ provider: this.manifest().id,+ retryable: false+ });+ }++ const deleted: string[] = [];+ const failed: BatchFailure[] = [];+ const queue = [...paths];++ const worker = async (): Promise<void> => {+ for (;;) {+ const target = queue.shift();+ if (target === undefined) return;+ try {+ await this.delete(target);+ deleted.push(target);+ } catch (error) {+ failed.push({+ path: target,+ error: (error as Error).message,+ code: (error as StorageError).code ?? BYOCErrorCode.PROVIDER_UNAVAILABLE+ });+ }+ }+ };++ await Promise.all(+ Array.from({ length: Math.min(concurrency, queue.length || 1) }, worker)+ );++ return {+ deleted,+ failed,+ total: deleted.length + failed.length,+ allSucceeded: failed.length === 0+ };+ }+/*** High-level backup helper to export data to the user's storage.*/diff --git a/packages/core/src/index.ts b/packages/core/src/index.tsindex 9ff8b52..6735320 100644--- a/packages/core/src/index.ts+++ b/packages/core/src/index.ts@@ -17,7 +17,10 @@ export type {StorageQuota,UploadOptions,UploadProgress,- BackupOptions+ BackupOptions,+ BatchDeleteReport,+ BatchFailure,+ SignedUrlOptions} from "./types/storage.js";// Errorsdiff --git a/packages/core/src/types/storage.ts b/packages/core/src/types/storage.tsindex be31320..debf812 100644--- a/packages/core/src/types/storage.ts+++ b/packages/core/src/types/storage.ts@@ -91,3 +91,30 @@ export interface StorageQuota {/** Remaining available storage in bytes (if known) */readonly available?: number;}++/** One path a batch operation could not complete, and why. */+export interface BatchFailure {+ readonly path: string;+ readonly error: string;+ readonly code: string;+}++/**+ * Outcome of a multi-path delete.+ *+ * A batch is reported rather than throwing on the first failure, because a+ * partial delete is the common real case: one object is locked or already+ * gone while the rest succeed, and the caller needs to know which.+ */+export interface BatchDeleteReport {+ readonly deleted: string[];+ readonly failed: BatchFailure[];+ readonly total: number;+ readonly allSucceeded: boolean;+}++/** Options for a time-limited signed URL. */+export interface SignedUrlOptions {+ readonly method?: string;+ readonly expiresInSeconds?: number;+}diff --git a/packages/google-drive/package.json b/packages/google-drive/package.jsonindex f898367..4803169 100644--- a/packages/google-drive/package.json+++ b/packages/google-drive/package.json@@ -1,6 +1,6 @@{"name": "@byoc/google-drive",- "version": "0.2.1",+ "version": "0.3.0","description": "Google Drive storage provider adapter for BYOC (Bring Your Own Cloud)","type": "module","main": "./dist/index.js",@@ -23,7 +23,7 @@"clean": "rm -rf dist tsconfig.tsbuildinfo"},"dependencies": {- "@byoc/core": "^0.2.1",+ "@byoc/core": "^0.3.0","@types/node": ">=20"},"keywords": [diff --git a/packages/google-drive/src/adapter.ts b/packages/google-drive/src/adapter.tsindex 414ed7f..9220d36 100644--- a/packages/google-drive/src/adapter.ts+++ b/packages/google-drive/src/adapter.ts@@ -72,7 +72,7 @@ export class GoogleDriveProvider implements BYOCProvider {category: "personal-cloud",authentication: "oauth2",supportsUserOwnedStorage: true,- adapterVersion: "0.2.0"+ adapterVersion: "0.3.0"};}diff --git a/packages/local/LICENSE b/packages/local/LICENSEnew file mode 100644index 0000000..45654bb--- /dev/null+++ b/packages/local/LICENSE@@ -0,0 +1,175 @@+ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work.++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONSdiff --git a/packages/local/README.md b/packages/local/README.mdnew file mode 100644index 0000000..438cfeb--- /dev/null+++ b/packages/local/README.md@@ -0,0 +1,78 @@+# @byoc/local++Local filesystem provider adapter for **BYOC (Bring Your Own Cloud)**.++> Part of the [BYOC monorepo](https://github.com/Ajayvarmaramineni/BYOC). Requires `@byoc/core`.+>+> **Status: v0.3.0.** Unit tested and certified against the provider compliance suite.++The provider that needs no account, no network, and no credentials. Use it to+evaluate BYOC before signing up for anything, to develop offline, and to back a+self-hosted deployment with a mounted volume as first-class storage rather than+a special case.++## Install++```bash+npm install @byoc/core @byoc/local+```++## Usage++```ts+import { BYOC } from "@byoc/core";+import { LocalFileSystemProvider } from "@byoc/local";++const storage = new BYOC({+ provider: new LocalFileSystemProvider({ rootDirectory: "./storage" })+});++await storage.connect();++await storage.writeText("reports/q3.md", "# Q3");+console.log(await storage.readText("reports/q3.md"));+console.log(await storage.list("reports"));+```++Swap in `S3CompatibleProvider`, `WebDAVProvider`, or `GoogleDriveProvider`+later and none of the calling code changes.++## Configuration++| Option | Type | Default | Meaning |+| :--- | :--- | :--- | :--- |+| `rootDirectory` | `string` | required | Directory that backs this provider |+| `createRoot` | `boolean` | `true` | Set `false` to require the directory to already exist |++## Capabilities++| Capability | Supported | Notes |+| :--- | :--- | :--- |+| `folders` | yes | Real directories |+| `serverSideCopy` | yes | `fs.cp` / `fs.rename`, no bytes through your process |+| `quota` | yes | Reports the backing filesystem, not the directory |+| `publicUrls` | no | A local path is not reachable by a browser |+| `sharing` | no | |+| `resumableUploads` | no | |+| `versioning` | no | |++## Path safety++Every path is confined to `rootDirectory`. `..` segments are rejected before+they reach the adapter, and the resolved target is rechecked against the+resolved root afterwards, so a **symlink pointing outside the root cannot be+read or written through** — a case traversal filtering alone does not catch.++## Custom metadata++The filesystem cannot store a MIME type or arbitrary key-value metadata on a+file. When you supply either, the adapter writes a sidecar under a single+`.byoc/` directory at the root. That directory is hidden from `list()`, so it+never appears as content.++If you never pass metadata, no sidecar is written and the directory holds+nothing but your own files.++## License++Apache-2.0diff --git a/packages/local/package.json b/packages/local/package.jsonnew file mode 100644index 0000000..2b313f2--- /dev/null+++ b/packages/local/package.json@@ -0,0 +1,55 @@+{+ "name": "@byoc/local",+ "version": "0.3.0",+ "description": "Local filesystem storage adapter for BYOC. No account, no network, no credentials.",+ "type": "module",+ "main": "./dist/index.js",+ "types": "./dist/index.d.ts",+ "exports": {+ ".": {+ "types": "./dist/index.d.ts",+ "import": "./dist/index.js",+ "default": "./dist/index.js"+ }+ },+ "files": [+ "dist",+ "!dist/tsconfig.tsbuildinfo",+ "README.md",+ "LICENSE"+ ],+ "scripts": {+ "build": "tsc -b",+ "clean": "rm -rf dist tsconfig.tsbuildinfo"+ },+ "dependencies": {+ "@byoc/core": "^0.3.0",+ "@types/node": ">=20"+ },+ "keywords": [+ "byoc",+ "storage",+ "filesystem",+ "local",+ "self-hosted",+ "node"+ ],+ "author": "Ajay Ramineni",+ "license": "Apache-2.0",+ "repository": {+ "type": "git",+ "url": "git+https://github.com/Ajayvarmaramineni/BYOC.git",+ "directory": "packages/local"+ },+ "publishConfig": {+ "access": "public"+ },+ "homepage": "https://github.com/Ajayvarmaramineni/BYOC/tree/main/packages/local#readme",+ "bugs": {+ "url": "https://github.com/Ajayvarmaramineni/BYOC/issues"+ },+ "engines": {+ "node": ">=20"+ },+ "sideEffects": false+}diff --git a/packages/local/src/adapter.ts b/packages/local/src/adapter.tsnew file mode 100644index 0000000..d075b39--- /dev/null+++ b/packages/local/src/adapter.ts@@ -0,0 +1,514 @@+import { createReadStream } from "node:fs";+import { constants as fsConstants } from "node:fs";+import * as fs from "node:fs/promises";+import * as path from "node:path";+import { Readable } from "node:stream";++import {+ BYOCErrorCode,+ StorageError,+ getBasename,+ normalizeVirtualPath,+ type BYOCProvider,+ type ProviderCapabilities,+ type ProviderManifest,+ type StorageInput,+ type StorageObject,+ type StorageOutput,+ type StorageQuota,+ type UploadOptions+} from "@byoc/core";++export const PROVIDER_ID = "local";++/**+ * Sidecar metadata lives under a single dotted directory at the root, so a+ * caller's own files are never shadowed and listings stay clean.+ */+const SIDECAR_DIR = ".byoc";++export interface LocalFileSystemProviderConfig {+ /** Directory that backs this provider. Created on connect unless disabled. */+ rootDirectory: string;+ /** Set `false` to require the directory to already exist. */+ createRoot?: boolean;+}++interface Sidecar {+ mimeType?: string;+ metadata?: Record<string, unknown>;+}++/**+ * Stores objects as real files under `rootDirectory`.+ *+ * The provider that needs no account, no network, and no credentials. It+ * exists so BYOC can be evaluated, tested, and developed against before+ * anyone signs up for anything, and so a self-hosted deployment can use a+ * mounted volume as first-class storage rather than a special case.+ *+ * Every path is confined to `rootDirectory`: paths are resolved before use+ * and rechecked afterwards, so neither `..` nor a symlink can escape.+ */+export class LocalFileSystemProvider implements BYOCProvider {+ private readonly configuredRoot: string;+ private readonly createRoot: boolean;+ private resolvedRoot?: string;++ constructor(config: LocalFileSystemProviderConfig) {+ this.configuredRoot = path.resolve(config.rootDirectory);+ this.createRoot = config.createRoot ?? true;+ }++ public manifest(): ProviderManifest {+ return {+ id: PROVIDER_ID,+ name: "Local Filesystem",+ category: "self-hosted",+ authentication: "local",+ supportsUserOwnedStorage: true,+ adapterVersion: "0.3.0"+ };+ }++ public capabilities(): ProviderCapabilities {+ // No sharing and no public URLs: a local path is not reachable by a+ // browser, and pretending otherwise would break feature detection.+ return {+ folders: true,+ sharing: false,+ publicUrls: false,+ resumableUploads: false,+ versioning: false,+ quota: true,+ serverSideCopy: true+ };+ }++ public async connect(): Promise<void> {+ if (this.createRoot) {+ await fs.mkdir(this.configuredRoot, { recursive: true });+ } else {+ const stat = await fs.stat(this.configuredRoot).catch(() => undefined);+ if (!stat?.isDirectory()) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: `Local storage root does not exist: ${this.configuredRoot}`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ }+ this.resolvedRoot = await fs.realpath(this.configuredRoot);+ }++ public async disconnect(): Promise<void> {+ // Nothing to release: there is no connection and no cached handle.+ this.resolvedRoot = undefined;+ }++ // -- path handling ------------------------------------------------------++ private root(): string {+ // Resolve lazily so a caller who forgets connect() still gets correct+ // behaviour rather than a confusing undefined.+ return this.resolvedRoot ?? this.configuredRoot;+ }++ /**+ * Virtual path -> absolute filesystem path, confined to the root.+ *+ * `normalizeVirtualPath` already rejects `..`. This adds the check that+ * survives symlinks, which traversal filtering alone cannot catch: the+ * resolved target must still sit inside the resolved root.+ */+ private async toLocal(virtualPath: string): Promise<string> {+ const normalized = normalizeVirtualPath(virtualPath);+ const root = this.root();+ const candidate = normalized ? path.join(root, normalized) : root;++ // realpath the deepest existing ancestor, so this works for paths being+ // created as well as existing ones while still following symlinks.+ let existing = candidate;+ const trailing: string[] = [];+ for (;;) {+ const found = await fs+ .access(existing, fsConstants.F_OK)+ .then(() => true)+ .catch(() => false);+ if (found) break;+ const parent = path.dirname(existing);+ if (parent === existing) break;+ trailing.unshift(path.basename(existing));+ existing = parent;+ }++ const realExisting = await fs.realpath(existing).catch(() => existing);+ const resolved = trailing.length ? path.join(realExisting, ...trailing) : realExisting;++ const relative = path.relative(root, resolved);+ const escapes = relative.startsWith("..") || path.isAbsolute(relative);+ if (resolved !== root && escapes) {+ throw new StorageError({+ code: BYOCErrorCode.PERMISSION_DENIED,+ message: `Path "${virtualPath}" resolves outside the storage root.`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ return resolved;+ }++ // -- sidecar metadata ---------------------------------------------------++ private sidecarFor(normalized: string): string {+ return path.join(this.root(), SIDECAR_DIR, `${normalized}.json`);+ }++ private async readSidecar(normalized: string): Promise<Sidecar> {+ try {+ return JSON.parse(await fs.readFile(this.sidecarFor(normalized), "utf-8")) as Sidecar;+ } catch {+ // A missing or unreadable sidecar is not an error: the file itself is+ // the source of truth, and metadata is supplementary.+ return {};+ }+ }++ private async writeSidecar(normalized: string, sidecar: Sidecar): Promise<void> {+ if (!sidecar.mimeType && !sidecar.metadata) return;+ const target = this.sidecarFor(normalized);+ await fs.mkdir(path.dirname(target), { recursive: true });+ await fs.writeFile(target, JSON.stringify(sidecar), "utf-8");+ }++ // -- object construction ------------------------------------------------++ private async toObject(localPath: string, normalized: string): Promise<StorageObject> {+ const stat = await fs.stat(localPath);+ const isDir = stat.isDirectory();+ const sidecar = isDir ? {} : await this.readSidecar(normalized);++ return {+ id: `local_${normalized}`,+ path: normalized,+ name: getBasename(normalized) || path.basename(localPath),+ provider: PROVIDER_ID,+ providerId: localPath,+ type: isDir ? "folder" : "file",+ size: isDir ? undefined : stat.size,+ mimeType: isDir ? undefined : (sidecar.mimeType ?? "application/octet-stream"),+ createdAt: stat.birthtime,+ updatedAt: stat.mtime,+ metadata: sidecar.metadata+ };+ }++ private static mapError(error: unknown, target: string): StorageError {+ const code = (error as NodeJS.ErrnoException)?.code;+ if (code === "ENOENT") {+ return new StorageError({+ code: BYOCErrorCode.OBJECT_NOT_FOUND,+ message: `Local file not found: ${target}`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ if (code === "EACCES" || code === "EPERM" || code === "EISDIR") {+ return new StorageError({+ code: BYOCErrorCode.PERMISSION_DENIED,+ message: `Local filesystem refused the operation on ${target} (${code}).`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ return new StorageError({+ code: BYOCErrorCode.PROVIDER_UNAVAILABLE,+ message: `Local filesystem error on ${target}: ${String(error)}`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ /** Collapses every accepted input shape to bytes. */+ private static async toBytes(data: StorageInput): Promise<Uint8Array> {+ if (typeof data === "string") return new TextEncoder().encode(data);+ if (data instanceof Uint8Array) return data;+ if (data instanceof ArrayBuffer) return new Uint8Array(data);+ if (typeof Blob !== "undefined" && data instanceof Blob) {+ return new Uint8Array(await data.arrayBuffer());+ }++ const chunks: Uint8Array[] = [];+ if (typeof (data as ReadableStream).getReader === "function") {+ const reader = (data as ReadableStream<Uint8Array>).getReader();+ for (;;) {+ const { done, value } = await reader.read();+ if (done) break;+ if (value) chunks.push(value);+ }+ } else if (typeof (data as AsyncIterable<unknown>)[Symbol.asyncIterator] === "function") {+ for await (const chunk of data as AsyncIterable<Uint8Array | string>) {+ chunks.push(typeof chunk === "string" ? new TextEncoder().encode(chunk) : chunk);+ }+ } else {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Unsupported upload payload type for the local provider.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const total = chunks.reduce((sum, chunk) => sum + chunk.byteLength, 0);+ const joined = new Uint8Array(total);+ let offset = 0;+ for (const chunk of chunks) {+ joined.set(chunk, offset);+ offset += chunk.byteLength;+ }+ return joined;+ }++ // -- core operations ----------------------------------------------------++ public async upload(+ virtualPath: string,+ data: StorageInput,+ options?: UploadOptions+ ): Promise<StorageObject> {+ const normalized = normalizeVirtualPath(virtualPath);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Upload requires a file path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const target = await this.toLocal(normalized);+ try {+ await fs.mkdir(path.dirname(target), { recursive: true });++ let written: number;+ if (+ typeof data !== "string" &&+ !(data instanceof Uint8Array) &&+ !(data instanceof ArrayBuffer) &&+ (data instanceof Readable ||+ typeof (data as ReadableStream).getReader === "function" ||+ typeof (data as unknown as AsyncIterable<unknown>)[Symbol.asyncIterator] === "function")+ ) {+ // Pipe streams straight to disk rather than buffering them first.+ written = await this.writeStream(target, data);+ } else {+ const payload = await LocalFileSystemProvider.toBytes(data);+ await fs.writeFile(target, payload);+ written = payload.byteLength;+ }++ await this.writeSidecar(normalized, {+ mimeType: options?.mimeType,+ metadata: options?.metadata+ });++ options?.onProgress?.({+ bytesUploaded: written,+ totalBytes: written,+ percentage: 100+ });+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, normalized);+ }++ return this.toObject(target, normalized);+ }++ private async writeStream(target: string, data: StorageInput): Promise<number> {+ const handle = await fs.open(target, "w");+ let written = 0;+ try {+ const source =+ data instanceof Readable+ ? data+ : typeof (data as ReadableStream).getReader === "function"+ ? Readable.fromWeb(data as never)+ : Readable.from(data as unknown as AsyncIterable<Uint8Array>);++ for await (const chunk of source) {+ const bytes =+ typeof chunk === "string" ? new TextEncoder().encode(chunk) : (chunk as Uint8Array);+ await handle.write(bytes);+ written += bytes.byteLength;+ }+ } finally {+ await handle.close();+ }+ return written;+ }++ public async download(virtualPath: string): Promise<StorageOutput> {+ const normalized = normalizeVirtualPath(virtualPath);+ const target = await this.toLocal(normalized);++ const stat = await fs.stat(target).catch(() => undefined);+ if (!stat?.isFile()) {+ throw new StorageError({+ code: BYOCErrorCode.OBJECT_NOT_FOUND,+ message: `Local file not found: ${normalized}`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const metadata = await this.toObject(target, normalized);+ return {+ stream: createReadStream(target),+ arrayBuffer: async () => {+ const buffer = await fs.readFile(target);+ return buffer.buffer.slice(+ buffer.byteOffset,+ buffer.byteOffset + buffer.byteLength+ ) as ArrayBuffer;+ },+ text: () => fs.readFile(target, "utf-8"),+ metadata+ };+ }++ public async delete(virtualPath: string): Promise<void> {+ const normalized = normalizeVirtualPath(virtualPath);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Delete requires a path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ const target = await this.toLocal(normalized);+ // Deletion is idempotent, matching every other adapter.+ await fs.rm(target, { recursive: true, force: true });+ await fs.rm(this.sidecarFor(normalized), { force: true });+ }++ public async exists(virtualPath: string): Promise<boolean> {+ const normalized = normalizeVirtualPath(virtualPath);+ if (!normalized) return false;+ const target = await this.toLocal(normalized);+ return fs+ .access(target, fsConstants.F_OK)+ .then(() => true)+ .catch(() => false);+ }++ public async metadata(virtualPath: string): Promise<StorageObject> {+ const normalized = normalizeVirtualPath(virtualPath);+ const target = await this.toLocal(normalized);+ try {+ return await this.toObject(target, normalized);+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, normalized);+ }+ }++ public async list(virtualPath?: string): Promise<StorageObject[]> {+ const normalized = normalizeVirtualPath(virtualPath ?? "");+ const target = await this.toLocal(normalized);++ let entries: string[];+ try {+ entries = await fs.readdir(target);+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, normalized);+ }++ const results: StorageObject[] = [];+ for (const entry of entries.sort()) {+ // The sidecar store is an implementation detail, not content.+ if (entry === SIDECAR_DIR && target === this.root()) continue;+ const child = normalized ? `${normalized}/${entry}` : entry;+ results.push(await this.toObject(path.join(target, entry), child));+ }+ return results;+ }++ // -- capability-gated operations ----------------------------------------++ public async createFolder(virtualPath: string): Promise<StorageObject> {+ const normalized = normalizeVirtualPath(virtualPath);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Create folder requires a path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ const target = await this.toLocal(normalized);+ try {+ await fs.mkdir(target, { recursive: true });+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, normalized);+ }+ return this.toObject(target, normalized);+ }++ public async copy(source: string, destination: string): Promise<void> {+ const [sourceNorm, destNorm] = LocalFileSystemProvider.requirePair(source, destination, "Copy");+ const src = await this.toLocal(sourceNorm);+ const dst = await this.toLocal(destNorm);++ try {+ await fs.mkdir(path.dirname(dst), { recursive: true });+ await fs.cp(src, dst, { recursive: true });+ const sidecar = await this.readSidecar(sourceNorm);+ await this.writeSidecar(destNorm, sidecar);+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, sourceNorm);+ }+ }++ public async move(source: string, destination: string): Promise<void> {+ const [sourceNorm, destNorm] = LocalFileSystemProvider.requirePair(source, destination, "Move");+ const src = await this.toLocal(sourceNorm);+ const dst = await this.toLocal(destNorm);++ try {+ await fs.mkdir(path.dirname(dst), { recursive: true });+ await fs.rename(src, dst);+ const sidecar = await this.readSidecar(sourceNorm);+ await this.writeSidecar(destNorm, sidecar);+ await fs.rm(this.sidecarFor(sourceNorm), { force: true });+ } catch (error) {+ throw LocalFileSystemProvider.mapError(error, sourceNorm);+ }+ }++ private static requirePair(+ source: string,+ destination: string,+ operation: string+ ): [string, string] {+ const sourceNorm = normalizeVirtualPath(source);+ const destNorm = normalizeVirtualPath(destination);+ if (!sourceNorm || !destNorm) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: `${operation} requires both a source and a destination path.`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ return [sourceNorm, destNorm];+ }++ /** Reports the backing filesystem's usage, not this directory's. */+ public async quota(): Promise<StorageQuota> {+ const stat = await fs.statfs(this.root());+ const total = stat.blocks * stat.bsize;+ const available = stat.bavail * stat.bsize;+ return { used: total - stat.bfree * stat.bsize, total, available };+ }+}diff --git a/packages/local/src/index.ts b/packages/local/src/index.tsnew file mode 100644index 0000000..1195429--- /dev/null+++ b/packages/local/src/index.ts@@ -0,0 +1,5 @@+export {+ LocalFileSystemProvider,+ PROVIDER_ID,+ type LocalFileSystemProviderConfig+} from "./adapter.js";diff --git a/packages/local/tsconfig.json b/packages/local/tsconfig.jsonnew file mode 100644index 0000000..134f8ff--- /dev/null+++ b/packages/local/tsconfig.json@@ -0,0 +1,13 @@+{+ "extends": "../../tsconfig.base.json",+ "compilerOptions": {+ "rootDir": "src",+ "outDir": "dist",+ "composite": true,+ "tsBuildInfoFile": "dist/tsconfig.tsbuildinfo"+ },+ "references": [+ { "path": "../core" }+ ],+ "include": ["src/**/*"]+}diff --git a/packages/memory/LICENSE b/packages/memory/LICENSEnew file mode 100644index 0000000..45654bb--- /dev/null+++ b/packages/memory/LICENSE@@ -0,0 +1,175 @@+ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work.++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding those notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONSdiff --git a/packages/memory/README.md b/packages/memory/README.mdnew file mode 100644index 0000000..2eba0c4--- /dev/null+++ b/packages/memory/README.md@@ -0,0 +1,82 @@+# @byoc/memory++In-memory provider adapter for **BYOC (Bring Your Own Cloud)**.++> Part of the [BYOC monorepo](https://github.com/Ajayvarmaramineni/BYOC). Requires `@byoc/core`.+>+> **Status: v0.3.0.** Unit tested and certified against the provider compliance suite.++A test double that behaves like a real provider. Use it to unit-test code that+talks to BYOC without a disk, a network, or a credential, and to run the same+suite in CI that you run against a live backend.++## Install++```bash+npm install @byoc/core @byoc/memory+```++## Usage++```ts+import { BYOC } from "@byoc/core";+import { MemoryProvider } from "@byoc/memory";++const provider = new MemoryProvider();+const storage = new BYOC({ provider });++await storage.connect();+await storage.writeText("reports/q3.md", "# Q3");++// Inspect what your code stored, without mocking anything.+expect(provider.size).toBe(1);+expect(Buffer.from(provider.snapshot()["reports/q3.md"]).toString()).toBe("# Q3");++provider.clear(); // between test cases+```++## Configuration++| Option | Type | Default | Meaning |+| :--- | :--- | :--- | :--- |+| `quotaBytes` | `number` | none | Total capacity to report. Omit to report usage only |+| `streamChunkSize` | `number` | `65536` | Bytes per chunk from `download().stream`. Lower it to exercise a caller's chunk handling |++## Test helpers++| Member | Returns | Use |+| :--- | :--- | :--- |+| `size` | `number` | How many objects are stored |+| `snapshot()` | `Record<string, Uint8Array>` | Every stored path and its bytes |+| `clear()` | `void` | Drop everything, between test cases |++## It models a flat object store++Like S3 and R2, this provider has **no real folders**. It reports+`folders: false`, and `createFolder` raises `CAPABILITY_UNSUPPORTED` rather+than faking a folder with a key prefix. Paths still nest: `list("reports")`+returns everything one level under `reports/`.++That is deliberate. A test double whose capabilities differ from production+lets bugs through, so this one commits to the most common real shape. If your+production provider has real folders, use [`@byoc/local`](../local) as the+double instead — it reports `folders: true` and is backed by real directories.++## Capabilities++| Capability | Supported |+| :--- | :--- |+| `serverSideCopy` | yes |+| `quota` | yes |+| `folders` | no |+| `publicUrls` | no |+| `sharing` | no |+| `resumableUploads` | no |+| `versioning` | no |++Nothing is persisted: every instance starts empty and is discarded with the+process.++## License++Apache-2.0diff --git a/packages/memory/package.json b/packages/memory/package.jsonnew file mode 100644index 0000000..e387dcf--- /dev/null+++ b/packages/memory/package.json@@ -0,0 +1,54 @@+{+ "name": "@byoc/memory",+ "version": "0.3.0",+ "description": "In-memory BYOC storage adapter: a test double that behaves like a real provider.",+ "type": "module",+ "main": "./dist/index.js",+ "types": "./dist/index.d.ts",+ "exports": {+ ".": {+ "types": "./dist/index.d.ts",+ "import": "./dist/index.js",+ "default": "./dist/index.js"+ }+ },+ "files": [+ "dist",+ "!dist/tsconfig.tsbuildinfo",+ "README.md",+ "LICENSE"+ ],+ "scripts": {+ "build": "tsc -b",+ "clean": "rm -rf dist tsconfig.tsbuildinfo"+ },+ "dependencies": {+ "@byoc/core": "^0.3.0"+ },+ "keywords": [+ "byoc",+ "storage",+ "in-memory",+ "testing",+ "test-double",+ "mock"+ ],+ "author": "Ajay Ramineni",+ "license": "Apache-2.0",+ "repository": {+ "type": "git",+ "url": "git+https://github.com/Ajayvarmaramineni/BYOC.git",+ "directory": "packages/memory"+ },+ "publishConfig": {+ "access": "public"+ },+ "homepage": "https://github.com/Ajayvarmaramineni/BYOC/tree/main/packages/memory#readme",+ "bugs": {+ "url": "https://github.com/Ajayvarmaramineni/BYOC/issues"+ },+ "engines": {+ "node": ">=20"+ },+ "sideEffects": false+}diff --git a/packages/memory/src/adapter.ts b/packages/memory/src/adapter.tsnew file mode 100644index 0000000..c01fcf0--- /dev/null+++ b/packages/memory/src/adapter.ts@@ -0,0 +1,367 @@+import {+ BYOCErrorCode,+ StorageError,+ getBasename,+ normalizeVirtualPath,+ type BYOCProvider,+ type ProviderCapabilities,+ type ProviderManifest,+ type StorageInput,+ type StorageObject,+ type StorageOutput,+ type StorageQuota,+ type UploadOptions+} from "@byoc/core";++export const PROVIDER_ID = "memory";++const DEFAULT_STREAM_CHUNK_SIZE = 64 * 1024;++/** One object's bytes plus the metadata a real provider would return. */+interface StoredObject {+ data: Uint8Array;+ mimeType: string;+ createdAt: Date;+ updatedAt: Date;+ metadata: Record<string, unknown>;+}++export interface MemoryProviderConfig {+ /** Total capacity to report. Omit to report usage only. */+ quotaBytes?: number;+ /**+ * Bytes per chunk yielded by `download().stream`. Small values are useful+ * for exercising a caller's chunk handling.+ */+ streamChunkSize?: number;+}++/**+ * Keeps every object in a Map.+ *+ * A test double that behaves like a real provider: use it to unit-test code+ * that talks to BYOC without a disk, a network, or a credential, and to run+ * the same suite in CI that you run against a live backend.+ *+ * It models a flat key-value object store, the shape S3 and R2 have, so code+ * verified against it behaves the same way against those. Like S3 it has no+ * real folders, so it reports `folders: false` and offers no `createFolder`.+ * Paths still nest: `list("reports")` returns everything one level under+ * `reports/`.+ *+ * Nothing is persisted; every instance starts empty.+ */+export class MemoryProvider implements BYOCProvider {+ private readonly objects = new Map<string, StoredObject>();+ private readonly quotaBytes?: number;+ private readonly streamChunkSize: number;++ constructor(config: MemoryProviderConfig = {}) {+ if (config.streamChunkSize !== undefined && config.streamChunkSize < 1) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "streamChunkSize must be at least 1 byte.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ this.quotaBytes = config.quotaBytes;+ this.streamChunkSize = config.streamChunkSize ?? DEFAULT_STREAM_CHUNK_SIZE;+ }++ public manifest(): ProviderManifest {+ return {+ id: PROVIDER_ID,+ name: "In-Memory",+ category: "self-hosted",+ authentication: "local",+ supportsUserOwnedStorage: false,+ adapterVersion: "0.3.0"+ };+ }++ public capabilities(): ProviderCapabilities {+ return {+ folders: false,+ sharing: false,+ publicUrls: false,+ resumableUploads: false,+ versioning: false,+ quota: true,+ serverSideCopy: true+ };+ }++ public async connect(): Promise<void> {+ // Nothing to connect to.+ }++ public async disconnect(): Promise<void> {+ // Stored objects survive, so a reconnect sees the same state.+ }++ // -- test helpers -------------------------------------------------------++ /** Drops every stored object. Convenient between test cases. */+ public clear(): void {+ this.objects.clear();+ }++ /** Every stored path and its bytes, for assertions in tests. */+ public snapshot(): Record<string, Uint8Array> {+ return Object.fromEntries(+ [...this.objects].map(([path, stored]) => [path, stored.data])+ );+ }++ /** How many objects are stored. */+ public get size(): number {+ return this.objects.size;+ }++ // -- internals ----------------------------------------------------------++ private toObject(path: string, stored: StoredObject): StorageObject {+ return {+ id: `memory_${path}`,+ path,+ name: getBasename(path),+ provider: PROVIDER_ID,+ providerId: path,+ type: "file",+ size: stored.data.byteLength,+ mimeType: stored.mimeType,+ createdAt: stored.createdAt,+ updatedAt: stored.updatedAt,+ metadata: stored.metadata+ };+ }++ private require(path: string, operation: string): [string, StoredObject] {+ const normalized = normalizeVirtualPath(path);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: `${operation} requires a path.`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ const stored = this.objects.get(normalized);+ if (!stored) {+ throw new StorageError({+ code: BYOCErrorCode.OBJECT_NOT_FOUND,+ message: `Object not found in memory storage: ${normalized}`,+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ return [normalized, stored];+ }++ /** Collapses every accepted input shape to bytes. */+ private static async toBytes(data: StorageInput): Promise<Uint8Array> {+ if (typeof data === "string") return new TextEncoder().encode(data);+ if (data instanceof Uint8Array) return data;+ if (data instanceof ArrayBuffer) return new Uint8Array(data);+ if (typeof Blob !== "undefined" && data instanceof Blob) {+ return new Uint8Array(await data.arrayBuffer());+ }++ const chunks: Uint8Array[] = [];+ if (typeof (data as ReadableStream).getReader === "function") {+ const reader = (data as ReadableStream<Uint8Array>).getReader();+ for (;;) {+ const { done, value } = await reader.read();+ if (done) break;+ if (value) chunks.push(value);+ }+ } else if (typeof (data as AsyncIterable<Uint8Array>)[Symbol.asyncIterator] === "function") {+ for await (const chunk of data as AsyncIterable<Uint8Array | string>) {+ chunks.push(typeof chunk === "string" ? new TextEncoder().encode(chunk) : chunk);+ }+ } else {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Unsupported upload payload type for the in-memory provider.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const total = chunks.reduce((sum, chunk) => sum + chunk.byteLength, 0);+ const joined = new Uint8Array(total);+ let offset = 0;+ for (const chunk of chunks) {+ joined.set(chunk, offset);+ offset += chunk.byteLength;+ }+ return joined;+ }++ // -- core operations ----------------------------------------------------++ public async upload(+ path: string,+ data: StorageInput,+ options?: UploadOptions+ ): Promise<StorageObject> {+ const normalized = normalizeVirtualPath(path);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Upload requires a file path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const payload = await MemoryProvider.toBytes(data);+ const now = new Date();+ const previous = this.objects.get(normalized);++ const stored: StoredObject = {+ data: payload,+ mimeType: options?.mimeType ?? "application/octet-stream",+ // Overwriting keeps the original creation time, as object stores do.+ createdAt: previous?.createdAt ?? now,+ updatedAt: now,+ metadata: { ...(options?.metadata ?? {}) }+ };+ this.objects.set(normalized, stored);++ options?.onProgress?.({+ bytesUploaded: payload.byteLength,+ totalBytes: payload.byteLength,+ percentage: 100+ });++ return this.toObject(normalized, stored);+ }++ public async download(path: string): Promise<StorageOutput> {+ const [normalized, stored] = this.require(path, "Download");+ const payload = stored.data;+ const chunkSize = this.streamChunkSize;++ const stream = new ReadableStream<Uint8Array>({+ start(controller) {+ for (let offset = 0; offset < payload.byteLength; offset += chunkSize) {+ controller.enqueue(payload.subarray(offset, offset + chunkSize));+ }+ controller.close();+ }+ });++ return {+ stream,+ arrayBuffer: async () =>+ payload.buffer.slice(+ payload.byteOffset,+ payload.byteOffset + payload.byteLength+ ) as ArrayBuffer,+ text: async () => new TextDecoder().decode(payload),+ metadata: this.toObject(normalized, stored)+ };+ }++ public async delete(path: string): Promise<void> {+ const normalized = normalizeVirtualPath(path);+ if (!normalized) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Delete requires a path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }+ // Idempotent, matching every other adapter.+ this.objects.delete(normalized);+ }++ public async exists(path: string): Promise<boolean> {+ const normalized = normalizeVirtualPath(path);+ return Boolean(normalized) && this.objects.has(normalized);+ }++ public async metadata(path: string): Promise<StorageObject> {+ const [normalized, stored] = this.require(path, "Metadata lookup");+ return this.toObject(normalized, stored);+ }++ public async list(path?: string): Promise<StorageObject[]> {+ const prefix = normalizeVirtualPath(path ?? "");+ const scope = prefix ? `${prefix}/` : "";++ const results: StorageObject[] = [];+ const seenPrefixes = new Set<string>();++ for (const [storedPath, stored] of this.objects) {+ if (!storedPath.startsWith(scope)) continue;++ const remainder = storedPath.slice(scope.length);+ if (!remainder.includes("/")) {+ results.push(this.toObject(storedPath, stored));+ continue;+ }++ // Deeper keys surface as a synthetic folder for their first segment,+ // which is what S3 does with CommonPrefixes. Without it a caller walking+ // the tree cannot discover anything nested, and a recursive delete would+ // silently leave objects behind.+ const child = `${scope}${remainder.split("/")[0]}`;+ if (seenPrefixes.has(child)) continue;+ seenPrefixes.add(child);+ results.push({+ id: `memory_${child}`,+ path: child,+ name: getBasename(child),+ provider: PROVIDER_ID,+ providerId: child,+ type: "folder"+ });+ }+ return results.sort((a, b) => a.path.localeCompare(b.path));+ }++ // -- capability-gated operations ----------------------------------------++ public async copy(source: string, destination: string): Promise<void> {+ const [, stored] = this.require(source, "Copy");+ const destNorm = normalizeVirtualPath(destination);+ if (!destNorm) {+ throw new StorageError({+ code: BYOCErrorCode.INVALID_INPUT,+ message: "Copy requires a destination path.",+ provider: PROVIDER_ID,+ retryable: false+ });+ }++ const now = new Date();+ this.objects.set(destNorm, {+ data: stored.data,+ mimeType: stored.mimeType,+ createdAt: now,+ updatedAt: now,+ metadata: { ...stored.metadata }+ });+ }++ public async move(source: string, destination: string): Promise<void> {+ await this.copy(source, destination);+ await this.delete(source);+ }++ public async quota(): Promise<StorageQuota> {+ let used = 0;+ for (const stored of this.objects.values()) used += stored.data.byteLength;++ if (this.quotaBytes === undefined) return { used };+ return {+ used,+ total: this.quotaBytes,+ available: Math.max(this.quotaBytes - used, 0)+ };+ }+}diff --git a/packages/memory/src/index.ts b/packages/memory/src/index.tsnew file mode 100644index 0000000..23d78a8--- /dev/null+++ b/packages/memory/src/index.ts@@ -0,0 +1 @@+export { MemoryProvider, PROVIDER_ID, type MemoryProviderConfig } from "./adapter.js";diff --git a/packages/memory/tests/providers.test.ts b/packages/memory/tests/providers.test.tsnew file mode 100644index 0000000..9a39187--- /dev/null+++ b/packages/memory/tests/providers.test.ts@@ -0,0 +1,432 @@+/**+ * Behavioural suite for the two credential-free providers.+ *+ * Both are driven through the same assertions, because the point of shipping+ * them is that code written against one BYOC provider behaves the same against+ * any other. A divergence here is a bug in whichever one differs.+ *+ * It lives in the memory package because that one has no Node-only imports and+ * so can host the shared cases for both.+ */+import os from "node:os";+import fsp from "node:fs/promises";+import path from "node:path";+import { describe, it, expect, beforeEach, afterEach } from "vitest";++import { BYOC, BYOCErrorCode, type BYOCProvider, type StorageError } from "@byoc/core";+import { LocalFileSystemProvider } from "../../local/src/adapter.js";+import { MemoryProvider } from "../src/adapter.js";++/**+ * Filenames that have historically broken adapters: `#` and `?` truncated an+ * object key, `+` was mis-decoded as a space, and non-ASCII broke signing.+ */+const AWKWARD_NAMES = ["draft#2.pdf", "q?x.txt", "sp ace.txt", "ümlaut.txt", "a+b.txt", "100%.txt"];++interface Subject {+ readonly label: string;+ readonly hasFolders: boolean;+ create(): Promise<BYOCProvider>;+}++const tempRoots: string[] = [];++async function makeTempRoot(): Promise<string> {+ const root = await fsp.mkdtemp(path.join(os.tmpdir(), "byoc-suite-"));+ tempRoots.push(root);+ return root;+}++const SUBJECTS: Subject[] = [+ {+ label: "MemoryProvider",+ hasFolders: false,+ create: async () => new MemoryProvider({ quotaBytes: 1024 * 1024 })+ },+ {+ label: "LocalFileSystemProvider",+ hasFolders: true,+ create: async () =>+ new LocalFileSystemProvider({ rootDirectory: path.join(await makeTempRoot(), "root") })+ }+];++afterEach(async () => {+ await Promise.all(tempRoots.splice(0).map((root) => fsp.rm(root, { recursive: true, force: true })));+});++async function errorCodeOf(run: () => Promise<unknown>): Promise<string | undefined> {+ try {+ await run();+ return undefined;+ } catch (error) {+ return (error as StorageError).code;+ }+}++for (const subject of SUBJECTS) {+ describe(subject.label, () => {+ let storage: BYOC;++ beforeEach(async () => {+ storage = new BYOC({ provider: await subject.create() });+ await storage.connect();+ });++ afterEach(async () => {+ await storage.disconnect();+ });++ // -- round trips ------------------------------------------------------++ it("round-trips text through write and read", async () => {+ const written = await storage.writeText("docs/report.md", "# Hello");++ expect(written.path).toBe("docs/report.md");+ expect(written.size).toBe(7);+ expect(await storage.readText("docs/report.md")).toBe("# Hello");+ });++ it("distinguishes present from missing objects", async () => {+ await storage.writeText("a.txt", "x");++ expect(await storage.exists("a.txt")).toBe(true);+ expect(await storage.exists("missing.txt")).toBe(false);+ // The root is a container, never a file.+ expect(await storage.exists("")).toBe(false);+ });++ it("reports accurate size in metadata", async () => {+ await storage.writeText("notes/readme.md", "hello world");+ const found = await storage.metadata("notes/readme.md");++ expect(found.size).toBe(11);+ expect(found.name).toBe("readme.md");+ expect(found.path).toBe("notes/readme.md");+ });++ it("raises OBJECT_NOT_FOUND for a missing object", async () => {+ expect(await errorCodeOf(() => storage.metadata("nope.txt"))).toBe(+ BYOCErrorCode.OBJECT_NOT_FOUND+ );+ expect(await errorCodeOf(() => storage.download("nope.txt"))).toBe(+ BYOCErrorCode.OBJECT_NOT_FOUND+ );+ });++ it("preserves custom metadata across a round trip", async () => {+ await storage.upload("tagged.bin", Buffer.from("payload"), {+ mimeType: "application/x-custom",+ metadata: { owner: "alex" }+ });+ const found = await storage.metadata("tagged.bin");++ expect(found.mimeType).toBe("application/x-custom");+ expect(found.metadata?.owner).toBe("alex");+ });++ // -- listing ----------------------------------------------------------++ it("lists one level only", async () => {+ await storage.writeText("top.txt", "a");+ await storage.writeText("docs/one.txt", "b");+ await storage.writeText("docs/deep/two.txt", "c");++ const names = (await storage.list("docs")).map((item) => item.name);++ expect(names).toContain("one.txt");+ // `deep/two.txt` must not appear as a flattened child.+ expect(names).not.toContain("two.txt");+ });++ it("returns paths that can be fed straight back in", async () => {+ await storage.writeText("docs/one.txt", "content");++ const files = (await storage.list("docs")).filter((item) => item.type !== "folder");+ expect(files.length).toBeGreaterThan(0);++ for (const item of files) {+ expect(await storage.readText(item.path)).toBe("content");+ }+ });++ // -- awkward filenames ------------------------------------------------++ it.each(AWKWARD_NAMES)("round-trips the filename %s", async (name) => {+ await storage.writeText(`odd/${name}`, `payload::${name}`);++ expect(await storage.readText(`odd/${name}`)).toBe(`payload::${name}`);+ expect((await storage.list("odd")).map((item) => item.name)).toContain(name);+ });++ it("keeps similarly named files apart", async () => {+ // The 0.1.0 bug: `draft#2.pdf` and `draft#3.pdf` both wrote to `draft`.+ await storage.writeText("draft#2.pdf", "two");+ await storage.writeText("draft#3.pdf", "three");++ expect(await storage.readText("draft#2.pdf")).toBe("two");+ expect(await storage.readText("draft#3.pdf")).toBe("three");+ });++ // -- copy, move, delete -----------------------------------------------++ it("leaves the source in place on copy", async () => {+ await storage.writeText("src.txt", "content");+ await storage.copy("src.txt", "nested/dst.txt");++ expect(await storage.readText("nested/dst.txt")).toBe("content");+ expect(await storage.exists("src.txt")).toBe(true);+ });++ it("removes the source on move", async () => {+ await storage.writeText("src.txt", "content");+ await storage.move("src.txt", "nested/dst.txt");++ expect(await storage.readText("nested/dst.txt")).toBe("content");+ expect(await storage.exists("src.txt")).toBe(false);+ });++ it("treats delete as idempotent", async () => {+ await storage.writeText("gone.txt", "x");+ await storage.delete("gone.txt");+ // A second delete is a no-op, not an error, on every adapter.+ await storage.delete("gone.txt");++ expect(await storage.exists("gone.txt")).toBe(false);+ });++ // -- streaming --------------------------------------------------------++ it("streams a download in several chunks", async () => {+ const payload = Buffer.alloc(200_000, "x");+ await storage.upload("big.bin", payload);++ const output = await storage.download("big.bin");+ const chunks: Buffer[] = [];+ for await (const chunk of output.stream as AsyncIterable<Uint8Array>) {+ chunks.push(Buffer.from(chunk));+ }++ expect(Buffer.concat(chunks).equals(payload)).toBe(true);+ expect(chunks.length).toBeGreaterThan(1);+ });++ it("accepts an async iterator as upload input", async () => {+ async function* chunks(): AsyncGenerator<Buffer> {+ yield Buffer.from("str");+ yield Buffer.from("eam");+ yield Buffer.from("ed!");+ }++ await storage.upload("streamed.bin", chunks() as never);++ expect(await storage.readText("streamed.bin")).toBe("streamed!");+ });++ // -- recursive and batch operations -----------------------------------++ it("walks objects at every depth", async () => {+ // Regression: on a flat provider `walk` returned only the top level.+ // MemoryProvider did not emit synthetic folder entries for deeper keys,+ // so there was nothing to descend into and everything nested was invisible.+ for (const p of ["docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"]) {+ await storage.writeText(p, p);+ }++ const found: string[] = [];+ for await (const item of storage.walk("docs")) found.push(item.path);++ expect(found).toEqual(+ expect.arrayContaining(["docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"])+ );+ });++ it("keeps the walk inside the requested subtree", async () => {+ await storage.writeText("docs/inside.txt", "a");+ await storage.writeText("other/outside.txt", "b");++ const found: string[] = [];+ for await (const item of storage.walk("docs")) found.push(item.path);++ expect(found).toContain("docs/inside.txt");+ expect(found).not.toContain("other/outside.txt");+ });++ it("removes every descendant on deleteTree", async () => {+ // Regression: this reported success while leaving nested objects behind.+ const nested = ["docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"];+ for (const p of nested) await storage.writeText(p, p);+ await storage.writeText("keep.txt", "untouched");++ const report = await storage.deleteTree("docs");++ expect(report.allSucceeded, JSON.stringify(report.failed)).toBe(true);+ for (const p of nested) {+ expect(await storage.exists(p), `${p} survived deleteTree`).toBe(false);+ }+ expect(await storage.readText("keep.txt")).toBe("untouched");+ });++ it("refuses to delete the tree at an empty path", async () => {+ // An empty path is the storage root; wiping it must be deliberate.+ expect(await errorCodeOf(() => storage.deleteTree(""))).toBe(BYOCErrorCode.INVALID_INPUT);+ });++ it("reports each path from deleteMany", async () => {+ await storage.writeText("a.txt", "a");+ await storage.writeText("b.txt", "b");++ const report = await storage.deleteMany(["a.txt", "b.txt", "never-existed.txt"]);++ // Deletion is idempotent everywhere, so a missing path is a success.+ expect([...report.deleted].sort()).toEqual(["a.txt", "b.txt", "never-existed.txt"]);+ expect(report.failed).toEqual([]);+ expect(report.total).toBe(3);+ expect(report.allSucceeded).toBe(true);+ });++ it("rejects a zero concurrency on deleteMany", async () => {+ expect(await errorCodeOf(() => storage.deleteMany(["a.txt"], 0))).toBe(+ BYOCErrorCode.INVALID_INPUT+ );+ });++ it("refuses a signed URL without the capability", async () => {+ // Neither provider can hand a browser a working URL, so both must say so.+ expect((await storage.capabilities()).publicUrls).toBe(false);++ await storage.writeText("a.txt", "x");+ expect(await errorCodeOf(() => storage.getSignedUrl("a.txt"))).toBe(+ BYOCErrorCode.CAPABILITY_UNSUPPORTED+ );+ });++ // -- capabilities -----------------------------------------------------++ it("backs every declared capability with a working method", async () => {+ const caps = await storage.capabilities();+ expect(caps.folders).toBe(subject.hasFolders);++ if (caps.folders) {+ const created = await storage.createFolder("newdir/nested");+ expect(created.type).toBe("folder");+ expect(await storage.exists("newdir/nested")).toBe(true);+ } else {+ expect(await errorCodeOf(() => storage.createFolder("newdir"))).toBe(+ BYOCErrorCode.CAPABILITY_UNSUPPORTED+ );+ }++ if (caps.serverSideCopy) {+ await storage.writeText("cap.txt", "x");+ await storage.copy("cap.txt", "cap-copy.txt");+ expect(await storage.exists("cap-copy.txt")).toBe(true);+ }++ if (caps.quota) {+ expect((await storage.getQuota()).used).toBeGreaterThanOrEqual(0);+ }+ });++ // -- safety -----------------------------------------------------------++ it.each([+ "../escape.txt",+ "docs/../../escape.txt",+ "../../../../../../etc/byoc-escape",+ "docs/../../../escape.txt"+ ])("refuses the traversal attempt %s", async (attack) => {+ const code = await errorCodeOf(() => storage.writeText(attack, "PWNED"));++ expect([BYOCErrorCode.INVALID_INPUT, BYOCErrorCode.PERMISSION_DENIED]).toContain(code);+ });+ });+}++describe("LocalFileSystemProvider specifics", () => {+ it("refuses to follow a symlink out of the root", async () => {+ // Traversal filtering alone does not catch this: the path has no `..`.+ const base = await makeTempRoot();+ const root = path.join(base, "root");+ const outside = path.join(base, "outside");+ await fsp.mkdir(root);+ await fsp.mkdir(outside);+ await fsp.writeFile(path.join(outside, "secret.txt"), "classified");+ await fsp.symlink(outside, path.join(root, "backdoor"));++ const storage = new BYOC({ provider: new LocalFileSystemProvider({ rootDirectory: root }) });+ await storage.connect();+ try {+ expect(await errorCodeOf(() => storage.readText("backdoor/secret.txt"))).toBe(+ BYOCErrorCode.PERMISSION_DENIED+ );+ expect(await errorCodeOf(() => storage.writeText("backdoor/planted.txt", "PWNED"))).toBe(+ BYOCErrorCode.PERMISSION_DENIED+ );+ } finally {+ await storage.disconnect();+ }++ await expect(fsp.access(path.join(outside, "planted.txt"))).rejects.toThrow();+ });++ it("never writes outside its root", async () => {+ const base = await makeTempRoot();+ const root = path.join(base, "root");++ const storage = new BYOC({ provider: new LocalFileSystemProvider({ rootDirectory: root }) });+ await storage.connect();+ try {+ await storage.writeText("a/b/c.txt", "inside");+ } finally {+ await storage.disconnect();+ }++ const entries = await fsp.readdir(base);+ expect(entries).toEqual(["root"]);+ });++ it("hides its sidecar store from listings", async () => {+ const root = path.join(await makeTempRoot(), "root");+ const storage = new BYOC({ provider: new LocalFileSystemProvider({ rootDirectory: root }) });+ await storage.connect();+ try {+ await storage.upload("a.txt", Buffer.from("x"), { metadata: { k: "v" } });+ const names = (await storage.list()).map((item) => item.name);++ expect(names).toContain("a.txt");+ expect(names).not.toContain(".byoc");+ } finally {+ await storage.disconnect();+ }+ });++ it("can require the root to already exist", async () => {+ const missing = path.join(await makeTempRoot(), "does-not-exist");+ const provider = new LocalFileSystemProvider({ rootDirectory: missing, createRoot: false });++ expect(await errorCodeOf(() => provider.connect())).toBe(BYOCErrorCode.INVALID_INPUT);+ });+});++describe("MemoryProvider specifics", () => {+ it("exposes stored state through its test helpers", async () => {+ const provider = new MemoryProvider();+ const storage = new BYOC({ provider });+ await storage.connect();++ await storage.writeText("a.txt", "one");+ await storage.writeText("b.txt", "two");++ expect(provider.size).toBe(2);+ expect(Buffer.from(provider.snapshot()["a.txt"]!).toString()).toBe("one");++ provider.clear();+ expect(provider.size).toBe(0);+ expect(await storage.exists("a.txt")).toBe(false);+ });++ it("rejects a stream chunk size below one byte", () => {+ expect(() => new MemoryProvider({ streamChunkSize: 0 })).toThrow();+ });+});diff --git a/packages/memory/tsconfig.json b/packages/memory/tsconfig.jsonnew file mode 100644index 0000000..134f8ff--- /dev/null+++ b/packages/memory/tsconfig.json@@ -0,0 +1,13 @@+{+ "extends": "../../tsconfig.base.json",+ "compilerOptions": {+ "rootDir": "src",+ "outDir": "dist",+ "composite": true,+ "tsBuildInfoFile": "dist/tsconfig.tsbuildinfo"+ },+ "references": [+ { "path": "../core" }+ ],+ "include": ["src/**/*"]+}diff --git a/packages/provider-sdk/package.json b/packages/provider-sdk/package.jsonindex 4c43805..1f86c11 100644--- a/packages/provider-sdk/package.json+++ b/packages/provider-sdk/package.json@@ -1,6 +1,6 @@{"name": "@byoc/provider-sdk",- "version": "0.2.1",+ "version": "0.3.0","description": "BYOC Provider SDK \u2014 Certification suite and developer tools for building BYOC storage adapters","type": "module","main": "./dist/index.js",@@ -17,7 +17,7 @@"clean": "rm -rf dist tsconfig.tsbuildinfo"},"dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"keywords": ["byoc",diff --git a/packages/provider-sdk/tests/operations.conformance.test.ts b/packages/provider-sdk/tests/operations.conformance.test.tsnew file mode 100644index 0000000..2177d99--- /dev/null+++ b/packages/provider-sdk/tests/operations.conformance.test.ts@@ -0,0 +1,167 @@+/**+ * Conformance: the operation surface both SDKs must expose.+ *+ * Pins which operations exist, not which bytes they write. The byte-level+ * fixtures cannot catch a missing method, because an adapter that lacks+ * `copy()` still writes identical bytes for everything it does implement.+ * That gap is how three working `copy()` implementations shipped in 0.2.x+ * with no way for a caller to reach any of them.+ *+ * The fixture names operations in snake_case as the neutral form. Method+ * naming is deliberately not part of the cross-SDK contract, so this file+ * camelCases them on the way in.+ */+import fs from "node:fs";+import path from "node:path";+import { fileURLToPath } from "node:url";+import { describe, it, expect } from "vitest";++import { BYOC, type BYOCProvider } from "@byoc/core";+import { GoogleDriveProvider } from "../../google-drive/src/adapter.js";+import { S3CompatibleProvider } from "../../s3-compatible/src/adapter.js";+import { WebDAVProvider } from "../../webdav/src/adapter.js";+import { LocalFileSystemProvider } from "../../local/src/adapter.js";+import { MemoryProvider } from "../../memory/src/adapter.js";++const HERE = path.dirname(fileURLToPath(import.meta.url));+const FIXTURE_PATH = path.resolve(HERE, "../../../spec/fixtures/provider-operations.json");++interface ProviderSpec {+ operations: string[];+ capabilities: Record<string, boolean>;+}++interface Fixture {+ client_operations: { required: string[] };+ provider_operations: { required: string[]; optional: string[] };+ capability_contracts: { contracts: { capability: string; requires_operation: string }[] };+ providers: Record<string, ProviderSpec>;+}++const fixture = JSON.parse(fs.readFileSync(FIXTURE_PATH, "utf-8")) as Fixture;++/**+ * Neutral operation names whose idiomatic TypeScript spelling is not a+ * mechanical case transform. `bytes` is the natural Python word; `Buffer` is+ * the natural Node one. Method naming is deliberately outside the cross-SDK+ * contract, so the mapping lives here in the SDK's own test rather than in+ * the shared fixture.+ */+const NAME_OVERRIDES: Record<string, string> = {+ read_bytes: "readBuffer",+ write_bytes: "writeBuffer",+ signed_url: "getSignedUrl"+};++/** snake_case neutral form -> this SDK's camelCase convention. */+function toCamel(name: string): string {+ return (+ NAME_OVERRIDES[name] ??+ name.replace(/_([a-z])/g, (_, letter: string) => letter.toUpperCase())+ );+}++function has(target: object, operation: string): boolean {+ return typeof (target as Record<string, unknown>)[toCamel(operation)] === "function";+}++// Constructed with placeholder credentials: nothing here performs I/O, and+// every assertion is about the shape of the adapter rather than its behaviour.+const ADAPTERS: Record<string, BYOCProvider> = {+ "google-drive": new GoogleDriveProvider({+ auth: {+ clientId: "test.apps.googleusercontent.com",+ redirectUri: "http://localhost/callback"+ }+ }),+ "s3-compatible": new S3CompatibleProvider({+ endpoint: "http://127.0.0.1:9000",+ bucket: "test",+ region: "us-east-1",+ accessKeyId: "key",+ secretAccessKey: "secret"+ }),+ webdav: new WebDAVProvider({+ endpoint: "http://127.0.0.1:8080",+ username: "user",+ password: "pass"+ }),+ local: new LocalFileSystemProvider({ rootDirectory: "/tmp/byoc-conformance-shape-only" }),+ memory: new MemoryProvider()+};++const PROVIDER_IDS = Object.keys(fixture.providers)+ .filter((name) => !name.startsWith("$"))+ .sort();++describe("operation-surface conformance", () => {+ it.each(fixture.client_operations.required)(+ "the client exposes the required operation %s",+ (operation) => {+ // An adapter method the client does not expose is unreachable by callers.+ const client = new BYOC({ provider: new MemoryProvider() });++ expect(+ has(client, operation),+ `BYOC is missing the required operation '${toCamel(operation)}'. An operation ` ++ "implemented on adapters but absent from the client cannot be called by anyone."+ ).toBe(true);+ }+ );++ describe.each(PROVIDER_IDS)("adapter %s", (providerId) => {+ const adapter = ADAPTERS[providerId]!;+ const spec = fixture.providers[providerId]!;++ it.each(fixture.provider_operations.required)("implements required %s", (operation) => {+ expect(+ has(adapter, operation),+ `Adapter '${providerId}' is missing required operation '${toCamel(operation)}'.`+ ).toBe(true);+ });++ it("has an optional-operation surface matching the fixture", () => {+ // Catches an adapter drifting ahead of or behind its peer SDK.+ const actual = fixture.provider_operations.optional+ .filter((operation) => has(adapter, operation))+ .sort();++ expect(+ actual,+ `Adapter '${providerId}' optional operations do not match the fixture. Either ` ++ "implement the missing operation or update " ++ "spec/fixtures/provider-operations.json in both SDKs."+ ).toEqual([...spec.operations].sort());+ });++ it("declares the capabilities the fixture pins", async () => {+ const capabilities = (await adapter.capabilities()) as unknown as Record<string, boolean>;++ for (const [neutralFlag, expected] of Object.entries(spec.capabilities)) {+ const flag = toCamel(neutralFlag);+ expect(+ capabilities[flag],+ `Adapter '${providerId}' declares ${flag}=${capabilities[flag]}, but the ` ++ `cross-SDK fixture pins ${expected}.`+ ).toBe(expected);+ }+ });++ it.each(fixture.capability_contracts.contracts)(+ "backs a declared $capability with a real method",+ async ({ capability, requires_operation: operation }) => {+ // Declaring a capability without the method is a lie to feature detection.+ const capabilities = (await adapter.capabilities()) as unknown as Record<string, boolean>;+ if (!capabilities[toCamel(capability)]) return;++ expect(+ has(adapter, operation),+ `Adapter '${providerId}' declares ${capability}=true but has no ` ++ `'${toCamel(operation)}' method. Callers feature-detect on the capability ` ++ "and would hit an undefined-is-not-a-function instead of a clean " ++ "CAPABILITY_UNSUPPORTED error."+ ).toBe(true);+ }+ );+ });+});diff --git a/packages/s3-compatible/package.json b/packages/s3-compatible/package.jsonindex b391843..c3d27e1 100644--- a/packages/s3-compatible/package.json+++ b/packages/s3-compatible/package.json@@ -1,6 +1,6 @@{"name": "@byoc/s3-compatible",- "version": "0.2.1",+ "version": "0.3.0","description": "Universal S3-compatible storage adapter for Cloudflare R2, AWS S3, MinIO, and Wasabi","type": "module","main": "./dist/index.js",@@ -23,7 +23,7 @@"clean": "rm -rf dist tsconfig.tsbuildinfo"},"dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"keywords": ["byoc",diff --git a/packages/s3-compatible/src/adapter.ts b/packages/s3-compatible/src/adapter.tsindex 8abb1fa..d3c7115 100644--- a/packages/s3-compatible/src/adapter.ts+++ b/packages/s3-compatible/src/adapter.ts@@ -48,7 +48,7 @@ export class S3CompatibleProvider implements BYOCProvider {category: "developer-cloud",authentication: "access-key",supportsUserOwnedStorage: false,- adapterVersion: "0.2.0"+ adapterVersion: "0.3.0"};}diff --git a/packages/webdav/package.json b/packages/webdav/package.jsonindex 2b22a6b..3de1895 100644--- a/packages/webdav/package.json+++ b/packages/webdav/package.json@@ -1,6 +1,6 @@{"name": "@byoc/webdav",- "version": "0.2.1",+ "version": "0.3.0","description": "Universal WebDAV storage adapter for Nextcloud, ownCloud, and Synology NAS","type": "module","main": "./dist/index.js",@@ -23,7 +23,7 @@"clean": "rm -rf dist tsconfig.tsbuildinfo"},"dependencies": {- "@byoc/core": "^0.2.1"+ "@byoc/core": "^0.3.0"},"keywords": ["byoc",diff --git a/packages/webdav/src/adapter.ts b/packages/webdav/src/adapter.tsindex d2f98e3..97f45fb 100644--- a/packages/webdav/src/adapter.ts+++ b/packages/webdav/src/adapter.ts@@ -50,7 +50,7 @@ export class WebDAVProvider implements BYOCProvider {category: "self-hosted",authentication: this.config.token ? "oauth2" : "basic",supportsUserOwnedStorage: true,- adapterVersion: "0.2.0"+ adapterVersion: "0.3.0"};}diff --git a/python/README.md b/python/README.mdindex 22c8a90..593d076 100644--- a/python/README.md+++ b/python/README.md@@ -2,7 +2,7 @@[](https://pypi.org/project/byoc-storage/)[](https://pypi.org/project/byoc-storage/)-[](https://github.com/Ajayvarmaramineni/BYOC)+[](https://github.com/Ajayvarmaramineni/BYOC)[](https://github.com/Ajayvarmaramineni/BYOC)[](https://github.com/Ajayvarmaramineni/BYOC/blob/main/LICENSE)@@ -45,6 +45,23 @@ import byoc## Usage+Start with no credentials at all. `LocalFileSystemProvider` and+`MemoryProvider` are bundled, need no account and no network, and behave like+every other provider:++```python+from byoc import AsyncBYOC, LocalFileSystemProvider++storage = AsyncBYOC(provider=LocalFileSystemProvider("./storage"))++async with storage:+ await storage.write_text("documents/welcome.md", "# Hello from BYOC!")+ content = await storage.read_text("documents/welcome.md")+```++Swap in `S3CompatibleProvider`, `WebDAVProvider`, or `GoogleDriveProvider` and+none of the calling code changes:+```pythonimport osfrom byoc import AsyncBYOC@@ -65,6 +82,25 @@ async with storage:content = await storage.read_text("documents/welcome.md")```+### Testing without a cloud++`MemoryProvider` is a test double that behaves like a real provider, so your+tests exercise the same code path production does:++```python+from byoc import AsyncBYOC, MemoryProvider++async def test_report_is_archived():+ provider = MemoryProvider()+ await archive_report(AsyncBYOC(provider=provider), report)++ assert provider.snapshot() == {"reports/q3.pdf": b"..."}+```++Like S3, it models a flat object store and reports `folders=False`. If your+production provider has real folders, use `LocalFileSystemProvider` against a+`tmp_path` instead.+### Multiple providers and migration```pythondiff --git a/python/pyproject.toml b/python/pyproject.tomlindex 66b8ebb..dde8a9c 100644--- a/python/pyproject.toml+++ b/python/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "hatchling.build"[project]name = "byoc-storage"-version = "0.2.0"+version = "0.3.0"description = "BYOC (Bring Your Own Cloud) — universal storage abstraction for storage your users already own"readme = "README.md"requires-python = ">=3.10"diff --git a/python/src/byoc/__init__.py b/python/src/byoc/__init__.pyindex df88bb0..79a1eee 100644--- a/python/src/byoc/__init__.py+++ b/python/src/byoc/__init__.py@@ -46,6 +46,8 @@ from .providers.gdrive import (GoogleDriveProvider,GoogleDriveScope,)+from .providers.local import LocalFileSystemProvider+from .providers.memory import MemoryProviderfrom .retry import with_retryfrom .types import (AuthType,@@ -64,7 +66,7 @@ from .types import (UploadProgress,)-__version__ = "0.2.0"+__version__ = "0.3.0"__all__ = ["AsyncBYOC",@@ -83,6 +85,8 @@ __all__ = ["GoogleDriveProvider","GoogleDriveScope","InvalidInputError",+ "LocalFileSystemProvider",+ "MemoryProvider","MigrationFileResult","MigrationProgress","MigrationReport",diff --git a/python/src/byoc/client.py b/python/src/byoc/client.pyindex 0b9b374..66139c3 100644--- a/python/src/byoc/client.py+++ b/python/src/byoc/client.py@@ -7,16 +7,19 @@ collections -- never surface here.from __future__ import annotations+import asyncioimport mimetypes-from collections.abc import Sequence+from collections.abc import AsyncIterator, Sequencefrom datetime import datetime, timezone-from .errors import CapabilityUnsupportedError, InvalidInputError+from .errors import CapabilityUnsupportedError, InvalidInputError, StorageErrorfrom .logging import BYOCLogger, SilentLoggerfrom .migration import MigrationReport, ProgressCallback, migratefrom .paths import normalize_virtual_pathfrom .types import (BackupOptions,+ BatchDeleteReport,+ BatchFailure,BYOCProvider,ConflictStrategy,ProviderCapabilities,@@ -62,7 +65,10 @@ class AsyncBYOC:registry: dict[str, BYOCProvider] = {}ordered: list[BYOCProvider] = []- for adapter in [*( [provider] if provider else [] ), *(providers or [])]:+ # `is not None`, not truthiness: an adapter is free to define __len__+ # or __bool__, and an empty one must still register.+ supplied: list[BYOCProvider] = [] if provider is None else [provider]+ for adapter in [*supplied, *(providers or [])]:ordered.append(adapter)registry[adapter.manifest().id] = adapter@@ -261,6 +267,22 @@ class AsyncBYOC:self._require_path(source, "Move"), self._require_path(destination, "Move"))+ async def copy(self, source: str, destination: str) -> None:+ """Copy an object, if the active provider supports it natively.++ Every current adapter copies server-side, so the bytes never travel+ through this process.+ """+ copier = getattr(self._current, "copy", None)+ if copier is None:+ raise CapabilityUnsupportedError(+ f"Provider '{self.manifest().name}' does not support native copy operations.",+ provider=self.manifest().id,+ )+ await copier(+ self._require_path(source, "Copy"), self._require_path(destination, "Copy")+ )+async def get_quota(self) -> StorageQuota:"""Report storage quota, if the active provider exposes it."""quota = getattr(self._current, "quota", None)@@ -272,6 +294,96 @@ class AsyncBYOC:result: StorageQuota = await quota()return result+ async def signed_url(+ self, path: str, *, method: str = "GET", expires_in_seconds: int = 3600+ ) -> str:+ """A time-limited URL a browser can use directly, without proxying bytes.++ Only providers reporting ``public_urls`` can issue one. Google Drive+ and WebDAV cannot, so they raise rather than returning a URL that+ would need the caller's credentials to be useful.+ """+ signer = getattr(self._current, "signed_url", None)+ if not self.capabilities().public_urls or signer is None:+ raise CapabilityUnsupportedError(+ f"Provider '{self.manifest().name}' cannot issue signed URLs.",+ provider=self.manifest().id,+ )+ url: str = signer(+ self._require_path(path, "Signed URL"),+ method=method,+ expires_in_seconds=expires_in_seconds,+ )+ return url++ # -- recursive and batch operations ------------------------------------++ async def walk(self, path: str | None = None) -> AsyncIterator[StorageObject]:+ """Yield every object beneath ``path``, descending into folders.++ ``list`` returns one level, which is right for rendering a file+ browser and wrong for "everything under here". This walks the tree+ breadth-first, yielding folders before their contents.++ It is built on ``list``, so it costs one call per folder. On a flat+ provider like S3 or the in-memory one there are no folders to descend+ into, so it is a single call.+ """+ pending = [normalize_virtual_path(path) or None]++ while pending:+ current = pending.pop(0)+ for item in await self._current.list(current):+ yield item+ if item.type == "folder":+ pending.append(item.path)++ async def delete_tree(self, path: str) -> BatchDeleteReport:+ """Delete everything under ``path``, then ``path`` itself.++ Children are deleted before their parents, so a provider that refuses+ to remove a non-empty folder still ends up with an empty one to+ remove. Failures are collected rather than aborting the walk.+ """+ normalized = self._require_path(path, "Delete tree")++ # Deepest first: a folder must be emptied before it can be removed.+ descendants = [item async for item in self.walk(normalized)]+ descendants.sort(key=lambda item: item.path.count("/"), reverse=True)++ targets = [item.path for item in descendants] + [normalized]+ return await self.delete_many(targets, concurrency=1)++ async def delete_many(+ self, paths: Sequence[str], *, concurrency: int = 8+ ) -> BatchDeleteReport:+ """Delete several paths, reporting per-path outcomes.++ One failure does not abort the rest: a locked or already-removed+ object is the common case, and the caller needs to know which paths+ survived rather than losing the whole batch.+ """+ if concurrency < 1:+ raise InvalidInputError(+ "delete_many concurrency must be at least 1.", provider=self.manifest().id+ )++ report = BatchDeleteReport()+ limit = asyncio.Semaphore(concurrency)++ async def remove(target: str) -> None:+ async with limit:+ try:+ await self.delete(target)+ report.deleted.append(target)+ except StorageError as error:+ report.failed.append(+ BatchFailure(path=target, error=str(error), code=error.code)+ )++ await asyncio.gather(*(remove(target) for target in paths))+ return report+# -- high-level helpers -------------------------------------------------async def backup(diff --git a/python/src/byoc/providers/gdrive/_http.py b/python/src/byoc/providers/gdrive/_http.pyindex ab0e2f0..1f83b3f 100644--- a/python/src/byoc/providers/gdrive/_http.py+++ b/python/src/byoc/providers/gdrive/_http.py@@ -217,6 +217,46 @@ class DriveHttpClient:raise self.map_error(response)return str(response.json()["id"])+ async def copy_file(+ self, file_id: str, new_name: str, parent_id: str+ ) -> dict[str, Any]:+ """Server-side copy. Drive duplicates the blob without re-uploading."""+ body = {"name": new_name, "parents": [parent_id]}+ response = await self._http().post(+ f"{API_BASE}/files/{file_id}/copy",+ params={"fields": FILE_FIELDS},+ headers=await self._headers({"Content-Type": "application/json"}),+ content=json.dumps(body).encode("utf-8"),+ )+ if response.is_error:+ raise self.map_error(response)+ resource: dict[str, Any] = response.json()+ return resource++ async def move_file(+ self, file_id: str, new_parent_id: str, old_parent_id: str, new_name: str+ ) -> dict[str, Any]:+ """Reparent and rename in one PATCH.++ Drive models a move as an edit to the file's parent list rather than a+ rename, so this is instantaneous regardless of file size.+ """+ params = {+ "addParents": new_parent_id,+ "removeParents": old_parent_id,+ "fields": FILE_FIELDS,+ }+ response = await self._http().patch(+ f"{API_BASE}/files/{file_id}",+ params=params,+ headers=await self._headers({"Content-Type": "application/json"}),+ content=json.dumps({"name": new_name}).encode("utf-8"),+ )+ if response.is_error:+ raise self.map_error(response)+ resource: dict[str, Any] = response.json()+ return resource+async def delete_file(self, file_id: str) -> None:response = await self._http().delete(f"{API_BASE}/files/{file_id}", headers=await self._headers()diff --git a/python/src/byoc/providers/gdrive/adapter.py b/python/src/byoc/providers/gdrive/adapter.pyindex c6fa19a..2fe4eae 100644--- a/python/src/byoc/providers/gdrive/adapter.py+++ b/python/src/byoc/providers/gdrive/adapter.py@@ -89,7 +89,7 @@ class GoogleDriveProvider:category="personal-cloud",authentication="oauth2",supports_user_owned_storage=True,- adapter_version="0.2.0",+ adapter_version="0.3.0",)def capabilities(self) -> ProviderCapabilities:@@ -233,6 +233,41 @@ class GoogleDriveProvider:type="folder",)+ async def copy(self, source: str, destination: str) -> None:+ """Server-side copy into ``destination``, creating parents as needed."""+ source_norm = normalize_virtual_path(source)+ dest_norm = normalize_virtual_path(destination)+ if not source_norm or not dest_norm:+ raise InvalidInputError(+ "Copy requires both a source and a destination path.", provider=PROVIDER_ID+ )++ source_id = await self.resolver.resolve_file_id(source_norm)+ dest_parent_id = await self.resolver.resolve_parent_folder_id(dest_norm, create=True)++ copied = await self.http.copy_file(source_id, get_basename(dest_norm), dest_parent_id)+ self.resolver.cache.set(dest_norm, str(copied["id"]))++ async def move(self, source: str, destination: str) -> None:+ """Reparent the file server-side, so no bytes are transferred."""+ source_norm = normalize_virtual_path(source)+ dest_norm = normalize_virtual_path(destination)+ if not source_norm or not dest_norm:+ raise InvalidInputError(+ "Move requires both a source and a destination path.", provider=PROVIDER_ID+ )++ source_id = await self.resolver.resolve_file_id(source_norm)+ old_parent_id = await self.resolver.resolve_parent_folder_id(source_norm, create=False)+ new_parent_id = await self.resolver.resolve_parent_folder_id(dest_norm, create=True)++ await self.http.move_file(+ source_id, new_parent_id, old_parent_id, get_basename(dest_norm)+ )+ # The old path no longer resolves; the new one now points at the same id.+ self.resolver.invalidate(source_norm)+ self.resolver.cache.set(dest_norm, source_id)+async def quota(self) -> StorageQuota:return await self.http.get_quota()diff --git a/python/src/byoc/providers/local.py b/python/src/byoc/providers/local.pynew file mode 100644index 0000000..169400f--- /dev/null+++ b/python/src/byoc/providers/local.py@@ -0,0 +1,433 @@+"""Local filesystem storage adapter.++The provider that needs no account, no network, and no credentials. It exists+so BYOC can be evaluated, tested, and developed against before anyone signs up+for anything, and so a self-hosted deployment can use a mounted volume as+first-class storage rather than a special case.++Blocking file I/O runs in a worker thread, so this adapter is safe to use on an+event loop alongside the network-backed ones.+"""++from __future__ import annotations++import asyncio+import json+import mimetypes+import os+import shutil+from collections.abc import AsyncIterator, Mapping+from datetime import datetime, timezone+from pathlib import Path+from typing import Any++from ..errors import (+ InvalidInputError,+ ObjectNotFoundError,+ PermissionDeniedError,+ StorageError,+)+from ..paths import get_basename, normalize_virtual_path+from ..types import (+ ProviderCapabilities,+ ProviderManifest,+ StorageInput,+ StorageObject,+ StorageOutput,+ StorageQuota,+ UploadOptions,+ UploadProgress,+)++PROVIDER_ID = "local"++# Sidecar metadata lives under a single dotted directory at the root, so a+# caller's own files are never shadowed and listings stay clean.+SIDECAR_DIR = ".byoc"++READ_CHUNK_SIZE = 64 * 1024+++class LocalFileSystemProvider:+ """Stores objects as real files under ``root_directory``.++ Args:+ root_directory: Directory that backs this provider. Created if missing.+ create_root: Set ``False`` to require the directory to already exist.++ Every path is confined to ``root_directory``. Paths are resolved before use+ and rechecked afterwards, so neither ``..`` nor a symlink can escape.+ """++ def __init__(self, root_directory: str | os.PathLike[str], *, create_root: bool = True) -> None:+ self.root = Path(root_directory).expanduser()+ self._create_root = create_root+ self._resolved_root: Path | None = None++ # -- identity ----------------------------------------------------------++ def manifest(self) -> ProviderManifest:+ return ProviderManifest(+ id=PROVIDER_ID,+ name="Local Filesystem",+ category="self-hosted",+ authentication="local",+ supports_user_owned_storage=True,+ adapter_version="0.3.0",+ )++ def capabilities(self) -> ProviderCapabilities:+ # No sharing and no public URLs: a local path is not reachable by a+ # browser, and pretending otherwise would break feature detection.+ return ProviderCapabilities(+ folders=True,+ sharing=False,+ public_urls=False,+ resumable_uploads=False,+ versioning=False,+ quota=True,+ server_side_copy=True,+ )++ # -- lifecycle ---------------------------------------------------------++ async def connect(self) -> None:+ def prepare() -> Path:+ if self._create_root:+ self.root.mkdir(parents=True, exist_ok=True)+ elif not self.root.is_dir():+ raise InvalidInputError(+ f"Local storage root does not exist: {self.root}", provider=PROVIDER_ID+ )+ return self.root.resolve()++ self._resolved_root = await asyncio.to_thread(prepare)++ async def disconnect(self) -> None:+ # Nothing to release: there is no connection and no cached handle.+ self._resolved_root = None++ # -- path handling -----------------------------------------------------++ def _root_or_raise(self) -> Path:+ if self._resolved_root is None:+ # Resolve lazily so a caller who forgets connect() still gets+ # correct behaviour rather than a confusing AttributeError.+ self._resolved_root = self.root.expanduser().resolve()+ return self._resolved_root++ def _to_local(self, virtual_path: str) -> Path:+ """Virtual path -> absolute filesystem path, confined to the root.++ ``normalize_virtual_path`` already rejects ``..``. This adds the check+ that survives symlinks, which traversal filtering alone cannot catch:+ the resolved target must still sit inside the resolved root.+ """+ normalized = normalize_virtual_path(virtual_path)+ root = self._root_or_raise()+ candidate = (root / normalized) if normalized else root++ # `strict=False` so this works for paths being created, not just+ # existing ones. Symlinks along the way are still followed.+ resolved = candidate.resolve()+ if resolved != root and root not in resolved.parents:+ raise PermissionDeniedError(+ f'Path "{virtual_path}" resolves outside the storage root.',+ provider=PROVIDER_ID,+ )+ return resolved++ def _to_virtual(self, local_path: Path) -> str:+ return local_path.relative_to(self._root_or_raise()).as_posix()++ # -- sidecar metadata --------------------------------------------------++ def _sidecar_for(self, normalized: str) -> Path:+ root = self._root_or_raise()+ return root / SIDECAR_DIR / f"{normalized}.json"++ def _read_sidecar(self, normalized: str) -> dict[str, Any]:+ sidecar = self._sidecar_for(normalized)+ try:+ loaded: dict[str, Any] = json.loads(sidecar.read_text("utf-8"))+ return loaded+ except (OSError, ValueError):+ # A missing or unreadable sidecar is not an error: the file itself+ # is the source of truth, and metadata is supplementary.+ return {}++ def _write_sidecar(+ self, normalized: str, mime_type: str | None, metadata: Mapping[str, str]+ ) -> None:+ if not mime_type and not metadata:+ return+ sidecar = self._sidecar_for(normalized)+ sidecar.parent.mkdir(parents=True, exist_ok=True)+ sidecar.write_text(+ json.dumps({"mime_type": mime_type, "metadata": dict(metadata)}), "utf-8"+ )++ def _delete_sidecar(self, normalized: str) -> None:+ self._sidecar_for(normalized).unlink(missing_ok=True)++ # -- object construction -----------------------------------------------++ def _to_object(self, local_path: Path, normalized: str) -> StorageObject:+ stat = local_path.stat()+ is_dir = local_path.is_dir()+ sidecar = {} if is_dir else self._read_sidecar(normalized)++ mime = sidecar.get("mime_type")+ if not mime and not is_dir:+ mime = mimetypes.guess_type(local_path.name)[0] or "application/octet-stream"++ return StorageObject(+ id=f"local_{normalized or ''}",+ path=normalized,+ name=get_basename(normalized) or local_path.name,+ provider=PROVIDER_ID,+ provider_id=str(local_path),+ type="folder" if is_dir else "file",+ size=None if is_dir else stat.st_size,+ mime_type=None if is_dir else mime,+ created_at=datetime.fromtimestamp(stat.st_ctime, tz=timezone.utc),+ updated_at=datetime.fromtimestamp(stat.st_mtime, tz=timezone.utc),+ metadata=sidecar.get("metadata", {}),+ )++ @staticmethod+ def _map_os_error(error: OSError, path: str) -> StorageError:+ if isinstance(error, FileNotFoundError):+ return ObjectNotFoundError(f"Local file not found: {path}", provider=PROVIDER_ID)+ if isinstance(error, (PermissionError, IsADirectoryError, NotADirectoryError)):+ return PermissionDeniedError(+ f"Local filesystem refused the operation on {path}: {error.strerror}",+ provider=PROVIDER_ID,+ )+ return StorageError(+ f"Local filesystem error on {path}: {error.strerror or error}",+ provider=PROVIDER_ID,+ )++ # -- core operations ---------------------------------------------------++ async def upload(+ self, path: str, data: StorageInput, options: UploadOptions | None = None+ ) -> StorageObject:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError("Upload requires a file path.", provider=PROVIDER_ID)++ target = self._to_local(normalized)+ resolved = options or UploadOptions()++ payload: bytes | None+ if isinstance(data, str):+ payload = data.encode("utf-8")+ elif isinstance(data, (bytes, bytearray, memoryview)):+ payload = bytes(data)+ else:+ payload = None # async iterator: written chunk by chunk below++ def write_bytes(body: bytes) -> None:+ target.parent.mkdir(parents=True, exist_ok=True)+ target.write_bytes(body)++ try:+ if payload is not None:+ await asyncio.to_thread(write_bytes, payload)+ written = len(payload)+ else:+ written = await self._write_stream(target, data) # type: ignore[arg-type]++ await asyncio.to_thread(+ self._write_sidecar, normalized, resolved.mime_type, resolved.metadata+ )+ except OSError as error:+ raise self._map_os_error(error, normalized) from error++ if resolved.on_progress:+ resolved.on_progress(+ UploadProgress(bytes_uploaded=written, total_bytes=written, percentage=100.0)+ )++ return await asyncio.to_thread(self._to_object, target, normalized)++ async def _write_stream(self, target: Path, chunks: AsyncIterator[bytes]) -> int:+ """Write an async iterator to disk without buffering the whole payload."""++ def open_target() -> Any:+ target.parent.mkdir(parents=True, exist_ok=True)+ return target.open("wb")++ handle = await asyncio.to_thread(open_target)+ written = 0+ try:+ async for chunk in chunks:+ await asyncio.to_thread(handle.write, chunk)+ written += len(chunk)+ finally:+ await asyncio.to_thread(handle.close)+ return written++ async def download(self, path: str) -> StorageOutput:+ normalized = normalize_virtual_path(path)+ target = self._to_local(normalized)++ if not await asyncio.to_thread(target.is_file):+ raise ObjectNotFoundError(+ f"Local file not found: {normalized}", provider=PROVIDER_ID+ )++ metadata = await asyncio.to_thread(self._to_object, target, normalized)++ async def stream() -> AsyncIterator[bytes]:+ handle = await asyncio.to_thread(target.open, "rb")+ try:+ while True:+ chunk = await asyncio.to_thread(handle.read, READ_CHUNK_SIZE)+ if not chunk:+ return+ yield chunk+ finally:+ await asyncio.to_thread(handle.close)++ async def read() -> bytes:+ return await asyncio.to_thread(target.read_bytes)++ return StorageOutput(metadata=metadata, stream=stream, read=read)++ async def delete(self, path: str) -> None:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError("Delete requires a path.", provider=PROVIDER_ID)+ target = self._to_local(normalized)++ def remove() -> None:+ if target.is_dir():+ shutil.rmtree(target)+ else:+ # Deletion is idempotent, matching every other adapter.+ target.unlink(missing_ok=True)+ self._delete_sidecar(normalized)++ try:+ await asyncio.to_thread(remove)+ except OSError as error:+ raise self._map_os_error(error, normalized) from error++ async def exists(self, path: str) -> bool:+ normalized = normalize_virtual_path(path)+ if not normalized:+ return False+ return await asyncio.to_thread(self._to_local(normalized).exists)++ async def metadata(self, path: str) -> StorageObject:+ normalized = normalize_virtual_path(path)+ target = self._to_local(normalized)++ if not await asyncio.to_thread(target.exists):+ raise ObjectNotFoundError(+ f"Local file not found: {normalized}", provider=PROVIDER_ID+ )+ return await asyncio.to_thread(self._to_object, target, normalized)++ async def list(self, path: str | None = None) -> list[StorageObject]:+ normalized = normalize_virtual_path(path)+ target = self._to_local(normalized)++ def scan() -> list[StorageObject]:+ if not target.is_dir():+ raise ObjectNotFoundError(+ f"Local folder not found: {normalized}", provider=PROVIDER_ID+ )+ results: list[StorageObject] = []+ for entry in sorted(target.iterdir(), key=lambda item: item.name):+ # The sidecar store is an implementation detail, not content.+ if entry.name == SIDECAR_DIR and entry.parent == self._root_or_raise():+ continue+ child = f"{normalized}/{entry.name}" if normalized else entry.name+ results.append(self._to_object(entry, child))+ return results++ try:+ return await asyncio.to_thread(scan)+ except OSError as error:+ raise self._map_os_error(error, normalized) from error++ # -- capability-gated operations ---------------------------------------++ async def create_folder(self, path: str) -> StorageObject:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError("Create folder requires a path.", provider=PROVIDER_ID)+ target = self._to_local(normalized)++ try:+ await asyncio.to_thread(lambda: target.mkdir(parents=True, exist_ok=True))+ except OSError as error:+ raise self._map_os_error(error, normalized) from error++ return await asyncio.to_thread(self._to_object, target, normalized)++ async def copy(self, source: str, destination: str) -> None:+ src_norm = normalize_virtual_path(source)+ dst_norm = normalize_virtual_path(destination)+ if not src_norm or not dst_norm:+ raise InvalidInputError(+ "Copy requires both a source and a destination path.", provider=PROVIDER_ID+ )+ src, dst = self._to_local(src_norm), self._to_local(dst_norm)++ def do_copy() -> None:+ dst.parent.mkdir(parents=True, exist_ok=True)+ if src.is_dir():+ shutil.copytree(src, dst, dirs_exist_ok=True)+ else:+ shutil.copy2(src, dst)+ sidecar = self._read_sidecar(src_norm)+ if sidecar:+ self._write_sidecar(+ dst_norm, sidecar.get("mime_type"), sidecar.get("metadata", {})+ )++ try:+ await asyncio.to_thread(do_copy)+ except OSError as error:+ raise self._map_os_error(error, src_norm) from error++ async def move(self, source: str, destination: str) -> None:+ src_norm = normalize_virtual_path(source)+ dst_norm = normalize_virtual_path(destination)+ if not src_norm or not dst_norm:+ raise InvalidInputError(+ "Move requires both a source and a destination path.", provider=PROVIDER_ID+ )+ src, dst = self._to_local(src_norm), self._to_local(dst_norm)++ def do_move() -> None:+ dst.parent.mkdir(parents=True, exist_ok=True)+ shutil.move(str(src), str(dst))+ sidecar = self._read_sidecar(src_norm)+ if sidecar:+ self._write_sidecar(+ dst_norm, sidecar.get("mime_type"), sidecar.get("metadata", {})+ )+ self._delete_sidecar(src_norm)++ try:+ await asyncio.to_thread(do_move)+ except OSError as error:+ raise self._map_os_error(error, src_norm) from error++ async def quota(self) -> StorageQuota:+ """Report the backing filesystem's usage, not this directory's."""++ def usage() -> StorageQuota:+ total, used, free = shutil.disk_usage(self._root_or_raise())+ return StorageQuota(used=used, total=total, available=free)++ return await asyncio.to_thread(usage)+++__all__ = ["PROVIDER_ID", "LocalFileSystemProvider"]diff --git a/python/src/byoc/providers/memory.py b/python/src/byoc/providers/memory.pynew file mode 100644index 0000000..3612fe3--- /dev/null+++ b/python/src/byoc/providers/memory.py@@ -0,0 +1,293 @@+"""In-memory storage adapter.++A test double that behaves like a real provider. Use it to unit-test code that+talks to BYOC without touching a disk, a network, or a credential, and to run+the same suite in CI that you run against a live backend.++It models a flat key-value object store, the shape S3 and R2 have, so code+verified against it behaves the same way against those. Nothing is persisted:+every instance starts empty and is discarded with the process.+"""++from __future__ import annotations++import asyncio+import mimetypes+from collections.abc import AsyncIterator, Mapping+from dataclasses import dataclass, field+from datetime import datetime, timezone++from ..errors import InvalidInputError, ObjectNotFoundError+from ..paths import get_basename, normalize_virtual_path+from ..types import (+ ProviderCapabilities,+ ProviderManifest,+ StorageInput,+ StorageObject,+ StorageOutput,+ StorageQuota,+ UploadOptions,+ UploadProgress,+)++PROVIDER_ID = "memory"++DEFAULT_STREAM_CHUNK_SIZE = 64 * 1024+++@dataclass+class _StoredObject:+ """One object's bytes and the metadata a real provider would return."""++ data: bytes+ mime_type: str+ created_at: datetime+ updated_at: datetime+ metadata: Mapping[str, str] = field(default_factory=dict)+++class MemoryProvider:+ """Keeps every object in a dictionary.++ Args:+ quota_bytes: Total capacity to report. ``None`` reports usage only.+ stream_chunk_size: Bytes per chunk yielded by ``download().stream()``.+ Small values are useful for exercising a caller's chunk handling.++ Like S3, this provider has no real folders, so it reports+ ``folders=False`` and ``create_folder`` is not offered. Paths still nest:+ ``list("reports")`` returns everything one level under ``reports/``.+ """++ def __init__(+ self,+ *,+ quota_bytes: int | None = None,+ stream_chunk_size: int = DEFAULT_STREAM_CHUNK_SIZE,+ ) -> None:+ if stream_chunk_size < 1:+ raise InvalidInputError(+ "stream_chunk_size must be at least 1 byte.", provider=PROVIDER_ID+ )+ self._objects: dict[str, _StoredObject] = {}+ self._quota_bytes = quota_bytes+ self._stream_chunk_size = stream_chunk_size++ # -- identity ----------------------------------------------------------++ def manifest(self) -> ProviderManifest:+ return ProviderManifest(+ id=PROVIDER_ID,+ name="In-Memory",+ category="self-hosted",+ authentication="local",+ supports_user_owned_storage=False,+ adapter_version="0.3.0",+ )++ def capabilities(self) -> ProviderCapabilities:+ return ProviderCapabilities(+ folders=False,+ sharing=False,+ public_urls=False,+ resumable_uploads=False,+ versioning=False,+ quota=True,+ server_side_copy=True,+ )++ # -- lifecycle ---------------------------------------------------------++ async def connect(self) -> None:+ """No-op: there is nothing to connect to."""++ async def disconnect(self) -> None:+ """No-op. Stored objects survive, so a reconnect sees the same state."""++ # -- test helpers ------------------------------------------------------++ def clear(self) -> None:+ """Drop every stored object. Convenient between test cases."""+ self._objects.clear()++ def snapshot(self) -> dict[str, bytes]:+ """Every stored path and its bytes, for assertions in tests."""+ return {path: stored.data for path, stored in self._objects.items()}++ def __len__(self) -> int:+ return len(self._objects)++ # -- internals ---------------------------------------------------------++ def _to_object(self, path: str, stored: _StoredObject) -> StorageObject:+ return StorageObject(+ id=f"memory_{path}",+ path=path,+ name=get_basename(path),+ provider=PROVIDER_ID,+ provider_id=path,+ type="file",+ size=len(stored.data),+ mime_type=stored.mime_type,+ created_at=stored.created_at,+ updated_at=stored.updated_at,+ metadata=stored.metadata,+ )++ def _require(self, path: str, operation: str) -> tuple[str, _StoredObject]:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError(f"{operation} requires a path.", provider=PROVIDER_ID)+ stored = self._objects.get(normalized)+ if stored is None:+ raise ObjectNotFoundError(+ f"Object not found in memory storage: {normalized}", provider=PROVIDER_ID+ )+ return normalized, stored++ # -- core operations ---------------------------------------------------++ async def upload(+ self, path: str, data: StorageInput, options: UploadOptions | None = None+ ) -> StorageObject:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError("Upload requires a file path.", provider=PROVIDER_ID)++ if isinstance(data, str):+ payload = data.encode("utf-8")+ elif isinstance(data, (bytes, bytearray, memoryview)):+ payload = bytes(data)+ else:+ collected = bytearray()+ async for chunk in data:+ collected.extend(chunk)+ payload = bytes(collected)++ resolved = options or UploadOptions()+ now = datetime.now(timezone.utc)+ previous = self._objects.get(normalized)++ self._objects[normalized] = _StoredObject(+ data=payload,+ mime_type=resolved.mime_type+ or mimetypes.guess_type(normalized)[0]+ or "application/octet-stream",+ # Overwriting keeps the original creation time, as object stores do.+ created_at=previous.created_at if previous else now,+ updated_at=now,+ metadata=dict(resolved.metadata),+ )++ if resolved.on_progress:+ resolved.on_progress(+ UploadProgress(+ bytes_uploaded=len(payload), total_bytes=len(payload), percentage=100.0+ )+ )++ return self._to_object(normalized, self._objects[normalized])++ async def download(self, path: str) -> StorageOutput:+ normalized, stored = self._require(path, "Download")+ payload = stored.data+ chunk_size = self._stream_chunk_size++ async def stream() -> AsyncIterator[bytes]:+ for offset in range(0, len(payload), chunk_size):+ # Yield to the loop so callers see real interleaving, as they+ # would against a network-backed provider.+ await asyncio.sleep(0)+ yield payload[offset : offset + chunk_size]++ async def read() -> bytes:+ return payload++ return StorageOutput(+ metadata=self._to_object(normalized, stored), stream=stream, read=read+ )++ async def delete(self, path: str) -> None:+ normalized = normalize_virtual_path(path)+ if not normalized:+ raise InvalidInputError("Delete requires a path.", provider=PROVIDER_ID)+ # Idempotent, matching every other adapter.+ self._objects.pop(normalized, None)++ async def exists(self, path: str) -> bool:+ normalized = normalize_virtual_path(path)+ return bool(normalized) and normalized in self._objects++ async def metadata(self, path: str) -> StorageObject:+ normalized, stored = self._require(path, "Metadata lookup")+ return self._to_object(normalized, stored)++ async def list(self, path: str | None = None) -> list[StorageObject]:+ prefix = normalize_virtual_path(path)+ scope = f"{prefix}/" if prefix else ""++ results: list[StorageObject] = []+ seen_prefixes: set[str] = set()++ for stored_path, stored in self._objects.items():+ if not stored_path.startswith(scope):+ continue++ remainder = stored_path[len(scope) :]+ if "/" not in remainder:+ results.append(self._to_object(stored_path, stored))+ continue++ # Deeper keys surface as a synthetic folder for their first+ # segment, which is what S3 does with CommonPrefixes. Without it a+ # caller walking the tree cannot discover anything nested, and a+ # recursive delete would silently leave objects behind.+ child = f"{scope}{remainder.split('/', 1)[0]}"+ if child in seen_prefixes:+ continue+ seen_prefixes.add(child)+ results.append(+ StorageObject(+ id=f"memory_{child}",+ path=child,+ name=get_basename(child),+ provider=PROVIDER_ID,+ provider_id=child,+ type="folder",+ )+ )++ return sorted(results, key=lambda item: item.path)++ # -- capability-gated operations ---------------------------------------++ async def copy(self, source: str, destination: str) -> None:+ src_norm, stored = self._require(source, "Copy")+ dst_norm = normalize_virtual_path(destination)+ if not dst_norm:+ raise InvalidInputError("Copy requires a destination path.", provider=PROVIDER_ID)++ now = datetime.now(timezone.utc)+ self._objects[dst_norm] = _StoredObject(+ data=stored.data,+ mime_type=stored.mime_type,+ created_at=now,+ updated_at=now,+ metadata=dict(stored.metadata),+ )+ _ = src_norm++ async def move(self, source: str, destination: str) -> None:+ await self.copy(source, destination)+ await self.delete(source)++ async def quota(self) -> StorageQuota:+ used = sum(len(stored.data) for stored in self._objects.values())+ if self._quota_bytes is None:+ return StorageQuota(used=used)+ return StorageQuota(+ used=used, total=self._quota_bytes, available=max(self._quota_bytes - used, 0)+ )+++__all__ = ["PROVIDER_ID", "MemoryProvider"]diff --git a/python/src/byoc/providers/s3.py b/python/src/byoc/providers/s3.pyindex d876708..c7cf84d 100644--- a/python/src/byoc/providers/s3.py+++ b/python/src/byoc/providers/s3.py@@ -103,7 +103,7 @@ class S3CompatibleProvider:category="developer-cloud",authentication="access-key",supports_user_owned_storage=False,- adapter_version="0.2.0",+ adapter_version="0.3.0",)def capabilities(self) -> ProviderCapabilities:@@ -165,7 +165,7 @@ class S3CompatibleProvider:"""return f"{self._bucket_base_url()}/{encode_path_segments(key)}"- def presigned_url(+ def signed_url(self, path: str, *, method: str = "GET", expires_in_seconds: int = 3600) -> str:"""Time-limited URL a browser can use directly, without proxying bytes."""@@ -178,6 +178,9 @@ class S3CompatibleProvider:expires_in_seconds=expires_in_seconds,)+ # Retained under its 0.2.x name; `signed_url` is what the client calls.+ presigned_url = signed_url+def _sign(self,method: str,@@ -331,6 +334,29 @@ class S3CompatibleProvider:await with_retry(send)+ async def copy(self, source: str, destination: str) -> None:+ """Server-side copy. The bytes never travel through this process."""+ source_key = self._to_key(source)+ dest_url = self.object_url(self._to_key(destination))++ # S3 wants the copy source as `/bucket/key`, URL-encoded. Encoding it+ # per segment is what keeps a `#` or `?` in the filename from+ # truncating the source key, exactly as it does for the object URL.+ copy_source = f"/{self.bucket}/{encode_path_segments(source_key)}"++ async def send() -> None:+ signed = self._sign("PUT", dest_url, {"x-amz-copy-source": copy_source})+ response = await self._http().request("PUT", dest_url, headers=signed)+ if response.is_error:+ raise self._map_error(response, response.text)++ await with_retry(send)++ async def move(self, source: str, destination: str) -> None:+ """Copy then delete. S3 has no atomic rename, so this is two calls."""+ await self.copy(source, destination)+ await self.delete(source)+async def metadata(self, path: str) -> StorageObject:key = self._to_key(path)url = self.object_url(key)diff --git a/python/src/byoc/providers/webdav.py b/python/src/byoc/providers/webdav.pyindex 0615552..1b18446 100644--- a/python/src/byoc/providers/webdav.py+++ b/python/src/byoc/providers/webdav.py@@ -131,7 +131,7 @@ class WebDAVProvider:category="self-hosted",authentication="oauth2" if self.token else "basic",supports_user_owned_storage=True,- adapter_version="0.2.0",+ adapter_version="0.3.0",)def capabilities(self) -> ProviderCapabilities:diff --git a/python/src/byoc/types.py b/python/src/byoc/types.pyindex 3a136c1..7f15f4d 100644--- a/python/src/byoc/types.py+++ b/python/src/byoc/types.py@@ -93,6 +93,36 @@ class BackupOptions:on_progress: ProgressCallback | None = None+@dataclass(frozen=True, slots=True)+class BatchFailure:+ """One path a batch operation could not complete, and why."""++ path: str+ error: str+ code: str+++@dataclass(frozen=True, slots=True)+class BatchDeleteReport:+ """Outcome of a multi-path delete.++ A batch is reported rather than raising on the first failure, because a+ partial delete is the common real case: one object is locked or already+ gone while the rest succeed, and the caller needs to know which.+ """++ deleted: list[str] = field(default_factory=list)+ failed: list[BatchFailure] = field(default_factory=list)++ @property+ def total(self) -> int:+ return len(self.deleted) + len(self.failed)++ @property+ def all_succeeded(self) -> bool:+ return not self.failed++@dataclass(frozen=True, slots=True)class ProviderManifest:"""Static metadata identifying a provider adapter."""diff --git a/python/tests/test_conformance_operations.py b/python/tests/test_conformance_operations.pynew file mode 100644index 0000000..30b8b81--- /dev/null+++ b/python/tests/test_conformance_operations.py@@ -0,0 +1,124 @@+"""Conformance: the operation surface both SDKs must expose.++Pins which operations exist, not which bytes they write. The byte-level+fixtures cannot catch a missing method, because an adapter that lacks+``copy()`` still writes identical bytes for everything it does implement.+That gap is how the Python S3 and Google Drive adapters shipped 0.2.x+declaring ``server_side_copy=True`` with no ``copy()`` behind it.+"""++from __future__ import annotations++from typing import Any++import pytest+from conftest import load_fixture++from byoc import AsyncBYOC, LocalFileSystemProvider, MemoryProvider+from byoc.providers.gdrive import GoogleDriveProvider+from byoc.providers.s3 import S3CompatibleProvider+from byoc.providers.webdav import WebDAVProvider+from byoc.types import BYOCProvider++FIXTURE = load_fixture("provider-operations.json")++# Constructed with placeholder credentials: nothing here performs I/O, and+# every assertion is about the shape of the adapter rather than its behaviour.+ADAPTERS: dict[str, BYOCProvider] = {+ "google-drive": GoogleDriveProvider(+ client_id="test.apps.googleusercontent.com",+ redirect_uri="http://localhost/callback",+ ),+ "s3-compatible": S3CompatibleProvider(+ endpoint="http://127.0.0.1:9000",+ bucket="test",+ region="us-east-1",+ access_key_id="key",+ secret_access_key="secret",+ ),+ "webdav": WebDAVProvider(+ endpoint="http://127.0.0.1:8080", username="user", password="pass"+ ),+ "local": LocalFileSystemProvider("/tmp/byoc-conformance-shape-only"),+ "memory": MemoryProvider(),+}++PROVIDER_SPECS: dict[str, Any] = {+ name: spec for name, spec in FIXTURE["providers"].items() if not name.startswith("$")+}+++def _has(target: object, operation: str) -> bool:+ return callable(getattr(target, operation, None))++[email protected]("operation", FIXTURE["client_operations"]["required"])+def test_client_exposes_every_required_operation(operation: str) -> None:+ """An adapter method the client does not expose is unreachable by callers."""+ client = AsyncBYOC(provider=MemoryProvider())++ assert _has(client, operation), (+ f"AsyncBYOC is missing the required operation '{operation}'. "+ "An operation implemented on adapters but absent from the client "+ "cannot be called by anyone."+ )++[email protected]("provider_id", sorted(PROVIDER_SPECS))[email protected]("operation", FIXTURE["provider_operations"]["required"])+def test_every_adapter_implements_the_required_operations(+ provider_id: str, operation: str+) -> None:+ assert _has(ADAPTERS[provider_id], operation), (+ f"Adapter '{provider_id}' is missing required operation '{operation}'."+ )++[email protected]("provider_id", sorted(PROVIDER_SPECS))+def test_adapter_operation_surface_matches_the_fixture(provider_id: str) -> None:+ """Catches an adapter drifting ahead of or behind its peer SDK."""+ adapter = ADAPTERS[provider_id]+ expected = set(PROVIDER_SPECS[provider_id]["operations"])+ optional = set(FIXTURE["provider_operations"]["optional"])++ actual = {operation for operation in optional if _has(adapter, operation)}++ assert actual == expected, (+ f"Adapter '{provider_id}' optional operations {sorted(actual)} do not match "+ f"the fixture's {sorted(expected)}. Either implement the missing operation "+ "or update spec/fixtures/provider-operations.json in both SDKs."+ )++[email protected]("provider_id", sorted(PROVIDER_SPECS))+def test_declared_capabilities_match_the_fixture(provider_id: str) -> None:+ capabilities = ADAPTERS[provider_id].capabilities()++ for flag, expected in PROVIDER_SPECS[provider_id]["capabilities"].items():+ assert getattr(capabilities, flag) is expected, (+ f"Adapter '{provider_id}' declares {flag}={getattr(capabilities, flag)}, "+ f"but the cross-SDK fixture pins {expected}."+ )++[email protected]("provider_id", sorted(PROVIDER_SPECS))+ "contract",+ FIXTURE["capability_contracts"]["contracts"],+ ids=lambda contract: str(contract["capability"]),+)+def test_a_declared_capability_is_backed_by_a_real_method(+ provider_id: str, contract: dict[str, str]+) -> None:+ """Declaring a capability without the method is a lie to feature detection."""+ adapter = ADAPTERS[provider_id]+ capability, operation = contract["capability"], contract["requires_operation"]++ if not getattr(adapter.capabilities(), capability):+ pytest.skip(f"{provider_id} does not declare {capability}")++ assert _has(adapter, operation), (+ f"Adapter '{provider_id}' declares {capability}=True but has no "+ f"'{operation}' method. Callers feature-detect on the capability and "+ "would hit an AttributeError instead of a clean CapabilityUnsupportedError."+ )diff --git a/python/tests/test_local_memory_providers.py b/python/tests/test_local_memory_providers.pynew file mode 100644index 0000000..3791a0b--- /dev/null+++ b/python/tests/test_local_memory_providers.py@@ -0,0 +1,476 @@+"""Behavioural suite for the two credential-free providers.++Both are driven through the same assertions, because the point of shipping+them is that code written against one BYOC provider behaves the same against+any other. A divergence here is a bug in whichever one differs.+"""++from __future__ import annotations++from collections.abc import AsyncIterator, Callable+from pathlib import Path++import pytest++from byoc import AsyncBYOC, LocalFileSystemProvider, MemoryProvider+from byoc.errors import (+ CapabilityUnsupportedError,+ InvalidInputError,+ ObjectNotFoundError,+ PermissionDeniedError,+)+from byoc.types import BYOCProvider, UploadOptions++# Filenames that have historically broken adapters: `#` and `?` truncated an+# object key, `+` was mis-decoded as a space, and non-ASCII broke signing.+AWKWARD_NAMES = ["draft#2.pdf", "q?x.txt", "sp ace.txt", "ümlaut.txt", "a+b.txt", "100%.txt"]+++ProviderFactory = Callable[[Path], BYOCProvider]+++def _local(tmp: Path) -> BYOCProvider:+ return LocalFileSystemProvider(tmp / "root")+++def _memory(_: Path) -> BYOCProvider:+ return MemoryProvider(quota_bytes=1024 * 1024)+++ALL_PROVIDERS = [+ pytest.param(_local, True, id="local"),+ pytest.param(_memory, False, id="memory"),+]+++async def storage(+ request: pytest.FixtureRequest, tmp_path: Path+) -> AsyncIterator[AsyncBYOC]:+ """A connected client for whichever provider the test is parametrized over."""+ factory: ProviderFactory = request.param+ client = AsyncBYOC(provider=factory(tmp_path))+ await client.connect()+ try:+ yield client+ finally:+ await client.disconnect()+++def _parametrize(func: object) -> object:+ return pytest.mark.parametrize(+ ("storage", "has_folders"), ALL_PROVIDERS, indirect=["storage"]+ )(func)+++# -- round trips ------------------------------------------------------------+++@_parametrize+async def test_write_then_read_round_trip(storage: AsyncBYOC, has_folders: bool) -> None:+ written = await storage.write_text("docs/report.md", "# Hello")++ assert written.path == "docs/report.md"+ assert written.size == 7+ assert await storage.read_text("docs/report.md") == "# Hello"+ assert await storage.read_bytes("docs/report.md") == b"# Hello"+++@_parametrize+async def test_exists_distinguishes_present_from_missing(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ await storage.write_text("a.txt", "x")++ assert await storage.exists("a.txt") is True+ assert await storage.exists("missing.txt") is False+ # The root is a container, never a file.+ assert await storage.exists("") is False+++@_parametrize+async def test_metadata_reports_size_and_mime(storage: AsyncBYOC, has_folders: bool) -> None:+ await storage.write_text("notes/readme.md", "hello world")+ found = await storage.metadata("notes/readme.md")++ assert found.size == 11+ assert found.mime_type is not None and "markdown" in found.mime_type+ assert found.name == "readme.md"+ assert found.path == "notes/readme.md"+++@_parametrize+async def test_metadata_on_missing_object_raises(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ with pytest.raises(ObjectNotFoundError):+ await storage.metadata("nope.txt")+++@_parametrize+async def test_download_missing_object_raises(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ with pytest.raises(ObjectNotFoundError):+ await storage.download("nope.txt")+++@_parametrize+async def test_custom_metadata_survives_a_round_trip(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ await storage.upload(+ "tagged.bin",+ b"payload",+ UploadOptions(mime_type="application/x-custom", metadata={"owner": "alex"}),+ )+ found = await storage.metadata("tagged.bin")++ assert found.mime_type == "application/x-custom"+ assert found.metadata.get("owner") == "alex"+++# -- listing ----------------------------------------------------------------+++@_parametrize+async def test_list_returns_one_level_only(storage: AsyncBYOC, has_folders: bool) -> None:+ await storage.write_text("top.txt", "a")+ await storage.write_text("docs/one.txt", "b")+ await storage.write_text("docs/deep/two.txt", "c")++ names = sorted(item.name for item in await storage.list("docs"))++ assert "one.txt" in names+ # `deep/two.txt` must not appear as a flattened child.+ assert "two.txt" not in names+++@_parametrize+async def test_listing_paths_can_be_fed_back_in(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """A returned path must be usable as an argument without translation."""+ await storage.write_text("docs/one.txt", "content")++ listed = [item for item in await storage.list("docs") if item.type != "folder"]+ assert listed, "expected at least one file"++ for item in listed:+ assert await storage.read_text(item.path) == "content"+++# -- awkward filenames ------------------------------------------------------+++@_parametrize[email protected]("name", AWKWARD_NAMES)+async def test_awkward_filenames_round_trip(+ storage: AsyncBYOC, has_folders: bool, name: str+) -> None:+ await storage.write_text(f"odd/{name}", f"payload::{name}")++ assert await storage.read_text(f"odd/{name}") == f"payload::{name}"+ assert name in {item.name for item in await storage.list("odd")}+++@_parametrize+async def test_similar_names_do_not_collide(storage: AsyncBYOC, has_folders: bool) -> None:+ """The 0.1.0 bug: `draft#2.pdf` and `draft#3.pdf` both wrote to `draft`."""+ await storage.write_text("draft#2.pdf", "two")+ await storage.write_text("draft#3.pdf", "three")++ assert await storage.read_text("draft#2.pdf") == "two"+ assert await storage.read_text("draft#3.pdf") == "three"+++# -- copy, move, delete -----------------------------------------------------+++@_parametrize+async def test_copy_leaves_the_source_in_place(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ await storage.write_text("src.txt", "content")+ await storage.copy("src.txt", "nested/dst.txt")++ assert await storage.read_text("nested/dst.txt") == "content"+ assert await storage.exists("src.txt") is True+++@_parametrize+async def test_move_removes_the_source(storage: AsyncBYOC, has_folders: bool) -> None:+ await storage.write_text("src.txt", "content")+ await storage.move("src.txt", "nested/dst.txt")++ assert await storage.read_text("nested/dst.txt") == "content"+ assert await storage.exists("src.txt") is False+++@_parametrize+async def test_delete_is_idempotent(storage: AsyncBYOC, has_folders: bool) -> None:+ await storage.write_text("gone.txt", "x")+ await storage.delete("gone.txt")+ # A second delete is a no-op, not an error, on every adapter.+ await storage.delete("gone.txt")++ assert await storage.exists("gone.txt") is False+++# -- streaming --------------------------------------------------------------+++@_parametrize+async def test_download_stream_yields_the_whole_payload(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ payload = b"x" * 200_000+ await storage.write_bytes("big.bin", payload)++ output = await storage.download("big.bin")+ chunks = [chunk async for chunk in output.stream()]++ assert b"".join(chunks) == payload+ assert len(chunks) > 1, "a 200 KB payload should arrive in several chunks"+++@_parametrize+async def test_upload_accepts_an_async_iterator(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """``StorageInput`` advertises async iterators, so they must work."""++ async def chunks() -> AsyncIterator[bytes]:+ for part in (b"str", b"eam", b"ed!"):+ yield part++ await storage.upload("streamed.bin", chunks())++ assert await storage.read_bytes("streamed.bin") == b"streamed!"+++# -- recursive and batch operations ----------------------------------------+++@_parametrize+async def test_walk_finds_objects_at_every_depth(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """Regression: on a flat provider ``walk`` returned only the top level.++ ``MemoryProvider`` did not emit synthetic folder entries for deeper keys,+ so there was nothing for the walk to descend into and everything nested+ was invisible.+ """+ for path in ("docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"):+ await storage.write_text(path, path)++ found = {item.path async for item in storage.walk("docs")}++ assert {"docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"} <= found+++@_parametrize+async def test_walk_stays_within_the_requested_subtree(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ await storage.write_text("docs/inside.txt", "a")+ await storage.write_text("other/outside.txt", "b")++ found = {item.path async for item in storage.walk("docs")}++ assert "docs/inside.txt" in found+ assert "other/outside.txt" not in found+++@_parametrize+async def test_delete_tree_removes_every_descendant(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """Regression: this reported success while leaving nested objects behind."""+ for path in ("docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"):+ await storage.write_text(path, path)+ await storage.write_text("keep.txt", "untouched")++ report = await storage.delete_tree("docs")++ assert report.all_succeeded, report.failed+ for path in ("docs/one.txt", "docs/deep/two.txt", "docs/deep/deeper/three.txt"):+ assert await storage.exists(path) is False, f"{path} survived delete_tree"+ assert await storage.read_text("keep.txt") == "untouched"+++@_parametrize+async def test_delete_tree_requires_a_path(storage: AsyncBYOC, has_folders: bool) -> None:+ # An empty path is the storage root; wiping it must be deliberate.+ with pytest.raises(InvalidInputError):+ await storage.delete_tree("")+++@_parametrize+async def test_delete_many_reports_each_path(storage: AsyncBYOC, has_folders: bool) -> None:+ await storage.write_text("a.txt", "a")+ await storage.write_text("b.txt", "b")++ report = await storage.delete_many(["a.txt", "b.txt", "never-existed.txt"])++ # Deletion is idempotent everywhere, so a missing path is a success.+ assert sorted(report.deleted) == ["a.txt", "b.txt", "never-existed.txt"]+ assert report.failed == []+ assert report.total == 3+ assert report.all_succeeded is True+++@_parametrize+async def test_delete_many_rejects_a_zero_concurrency(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ with pytest.raises(InvalidInputError):+ await storage.delete_many(["a.txt"], concurrency=0)+++@_parametrize+async def test_signed_url_is_refused_without_the_capability(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """Neither provider can hand a browser a working URL, so both must say so."""+ assert storage.capabilities().public_urls is False++ await storage.write_text("a.txt", "x")+ with pytest.raises(CapabilityUnsupportedError):+ await storage.signed_url("a.txt")+++# -- capabilities -----------------------------------------------------------+++@_parametrize+async def test_declared_capabilities_match_reality(+ storage: AsyncBYOC, has_folders: bool+) -> None:+ """A declared capability must be backed by a working method."""+ caps = storage.capabilities()++ assert caps.folders is has_folders+ if caps.folders:+ created = await storage.create_folder("newdir/nested")+ assert created.type == "folder"+ assert await storage.exists("newdir/nested") is True+ else:+ with pytest.raises(CapabilityUnsupportedError):+ await storage.create_folder("newdir")++ if caps.server_side_copy:+ await storage.write_text("cap.txt", "x")+ await storage.copy("cap.txt", "cap-copy.txt")+ assert await storage.exists("cap-copy.txt") is True++ if caps.quota:+ assert (await storage.get_quota()).used >= 0+++# -- safety -----------------------------------------------------------------+++@_parametrize+ "attack",+ [+ "../escape.txt",+ "docs/../../escape.txt",+ "../../../../../../etc/byoc-escape",+ "docs/../../../escape.txt",+ ],+)+async def test_traversal_cannot_escape_the_root(+ storage: AsyncBYOC, has_folders: bool, attack: str+) -> None:+ with pytest.raises((InvalidInputError, PermissionDeniedError)):+ await storage.write_text(attack, "PWNED")+++async def test_symlink_cannot_escape_the_local_root(tmp_path: Path) -> None:+ """Traversal filtering alone does not catch this: the path has no `..`."""+ root = tmp_path / "root"+ outside = tmp_path / "outside"+ root.mkdir()+ outside.mkdir()+ (outside / "secret.txt").write_text("classified")+ (root / "backdoor").symlink_to(outside)++ client = AsyncBYOC(provider=LocalFileSystemProvider(root))+ await client.connect()+ try:+ with pytest.raises(PermissionDeniedError):+ await client.read_text("backdoor/secret.txt")+ with pytest.raises(PermissionDeniedError):+ await client.write_text("backdoor/planted.txt", "PWNED")+ finally:+ await client.disconnect()++ assert not (outside / "planted.txt").exists()+++async def test_local_provider_never_writes_outside_its_root(tmp_path: Path) -> None:+ root = tmp_path / "root"+ client = AsyncBYOC(provider=LocalFileSystemProvider(root))+ await client.connect()+ try:+ await client.write_text("a/b/c.txt", "inside")+ finally:+ await client.disconnect()++ written = {p for p in tmp_path.rglob("*") if p.is_file()}+ assert written, "expected the file to be written"+ assert all(root in p.parents for p in written)+++# -- provider-specific behaviour -------------------------------------------+++async def test_memory_provider_helpers_expose_state() -> None:+ provider = MemoryProvider()+ client = AsyncBYOC(provider=provider)+ await client.connect()++ await client.write_text("a.txt", "one")+ await client.write_text("b.txt", "two")++ assert len(provider) == 2+ assert provider.snapshot() == {"a.txt": b"one", "b.txt": b"two"}++ provider.clear()+ assert len(provider) == 0+ assert await client.exists("a.txt") is False+++async def test_an_empty_provider_still_registers() -> None:+ """Regression: the client used truthiness, so a provider defining+ ``__len__`` was silently dropped while empty."""+ provider = MemoryProvider()+ assert len(provider) == 0++ client = AsyncBYOC(provider=provider)++ assert client.manifest().id == "memory"+++async def test_local_provider_hides_its_sidecar_store(tmp_path: Path) -> None:+ client = AsyncBYOC(provider=LocalFileSystemProvider(tmp_path))+ await client.connect()+ try:+ await client.upload("a.txt", b"x", UploadOptions(metadata={"k": "v"}))+ listed = {item.name for item in await client.list()}+ finally:+ await client.disconnect()++ assert "a.txt" in listed+ assert ".byoc" not in listed+++async def test_local_provider_can_require_an_existing_root(tmp_path: Path) -> None:+ missing = tmp_path / "does-not-exist"+ provider = LocalFileSystemProvider(missing, create_root=False)++ with pytest.raises(InvalidInputError):+ await provider.connect()diff --git a/spec/fixtures/provider-operations.json b/spec/fixtures/provider-operations.jsonnew file mode 100644index 0000000..e947520--- /dev/null+++ b/spec/fixtures/provider-operations.json@@ -0,0 +1,110 @@+{+ "$comment": "BYOC operation-surface contract. Unlike the other fixtures, this one pins NO wire bytes -- it pins which operations exist. Both SDKs write identical bytes whether or not an adapter implements copy(), so no byte-level vector can catch an adapter that is simply missing a method. That is exactly how the TypeScript SDK came to implement copy() on all three network adapters while the Python S3 and Google Drive adapters had none, and how both clients ended up with no copy() at all, leaving three working implementations unreachable. Operation names below are given in snake_case as the neutral form; each SDK maps them to its own convention (TypeScript camelCases them). Method NAMING is deliberately not part of the contract -- only which operations exist.",+ "version": 1,++ "client_operations": {+ "$comment": "Operations the universal client must expose in every SDK. An operation implemented on an adapter but absent from the client is unreachable by callers and does not count as shipped.",+ "required": [+ "connect",+ "disconnect",+ "upload",+ "download",+ "delete",+ "list",+ "exists",+ "metadata",+ "read_text",+ "read_bytes",+ "write_text",+ "write_bytes",+ "create_folder",+ "move",+ "copy",+ "get_quota",+ "signed_url",+ "walk",+ "delete_tree",+ "delete_many",+ "capabilities",+ "has_capability",+ "manifest",+ "use_provider",+ "get_providers",+ "migrate",+ "backup"+ ]+ },++ "provider_operations": {+ "$comment": "Every adapter must implement the required set. Optional operations are only required when the matching capability is declared true -- see capability_contracts.",+ "required": [+ "connect",+ "disconnect",+ "upload",+ "download",+ "delete",+ "list",+ "exists",+ "metadata",+ "manifest",+ "capabilities"+ ],+ "optional": ["create_folder", "move", "copy", "quota", "signed_url"]+ },++ "capability_contracts": {+ "$comment": "A declared capability is a promise to the caller, who feature-detects on it instead of catching an exception. Declaring a capability without the backing method is a lie that surfaces as an AttributeError or undefined-is-not-a-function at the call site. Python's S3 and Google Drive adapters both declared server_side_copy=true with no copy() method until 0.3.0.",+ "contracts": [+ {+ "capability": "folders",+ "requires_operation": "create_folder",+ "$comment": "A provider without real folders must report false and let create_folder raise CAPABILITY_UNSUPPORTED, rather than faking folders with key prefixes."+ },+ {+ "capability": "server_side_copy",+ "requires_operation": "copy"+ },+ {+ "capability": "quota",+ "requires_operation": "quota"+ },+ {+ "capability": "public_urls",+ "requires_operation": "signed_url",+ "$comment": "Only S3-compatible backends can hand a browser a URL that works without the caller's credentials. Drive and WebDAV report false rather than returning a link that would 401."+ }+ ]+ },++ "providers": {+ "$comment": "The operation surface each shipped adapter is expected to have, in both SDKs. Adding an adapter means adding it here.",+ "google-drive": {+ "operations": ["create_folder", "move", "copy", "quota"],+ "capabilities": { "folders": true, "quota": true, "server_side_copy": true, "public_urls": false }+ },+ "s3-compatible": {+ "operations": ["move", "copy", "signed_url"],+ "capabilities": {+ "folders": false,+ "quota": false,+ "server_side_copy": true,+ "public_urls": true+ },+ "$comment": "S3 has no native folders and reports no account quota. Both stay false rather than being faked."+ },+ "webdav": {+ "operations": ["create_folder", "move", "copy", "quota"],+ "capabilities": { "folders": true, "quota": true, "server_side_copy": true, "public_urls": false }+ },+ "local": {+ "operations": ["create_folder", "move", "copy", "quota"],+ "capabilities": { "folders": true, "quota": true, "server_side_copy": true, "public_urls": false },+ "$comment": "Quota reports the backing filesystem's usage, not the root directory's."+ },+ "memory": {+ "operations": ["move", "copy", "quota"],+ "capabilities": { "folders": false, "quota": true, "server_side_copy": true, "public_urls": false },+ "$comment": "Models a flat key-value object store, the shape S3 and R2 have, so it has no real folders either."+ }+ }+}