Skip to content

"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files

s3: add Neon Object Storage provider

Rishi Raj Jain1a319f219101
diff --git a/README.md b/README.md
index 312f57284..e73292ed4 100644
--- a/README.md
+++ b/README.md
@@ -93,6 +93,7 @@ directories to and from different cloud storage providers.
- Microsoft Azure Files Storage [:page_facing_up:](https://rclone.org/azurefiles/)
- Microsoft OneDrive [:page_facing_up:](https://rclone.org/onedrive/)
- Minio [:page_facing_up:](https://rclone.org/s3/#minio)
+- Neon Object Storage [:page_facing_up:](https://rclone.org/s3/#neon)
- Nextcloud [:page_facing_up:](https://rclone.org/webdav/#nextcloud)
- Blomp Cloud Storage [:page_facing_up:](https://rclone.org/swift/)
- OpenDrive [:page_facing_up:](https://rclone.org/opendrive/)
diff --git a/backend/s3/provider/Neon.yaml b/backend/s3/provider/Neon.yaml
new file mode 100644
index 000000000..ba295ffe5
--- /dev/null
+++ b/backend/s3/provider/Neon.yaml
@@ -0,0 +1,11 @@
+name: Neon
+description: Neon Object Storage
+region:
+ us-east-1: US East (N. Virginia)
+ us-east-2: US East (Ohio)
+ eu-central-1: Europe (Frankfurt)
+ ap-southeast-1: Asia Pacific (Singapore)
+endpoint: {}
+quirks:
+ force_path_style: true # virtual-hosted style requests are not supported
+ object_lock_supported: false # object lock config is stored but enforcement is limited
diff --git a/docs/content/_index.md b/docs/content/_index.md
index 707734c0b..9adeb7a3f 100644
--- a/docs/content/_index.md
+++ b/docs/content/_index.md
@@ -184,6 +184,7 @@ WebDAV or S3, that work out of the box.)
{{< provider name="Microsoft Azure Files Storage" home="https://azure.microsoft.com/en-us/services/storage/files/" config="/azurefiles/" >}}
{{< provider name="Microsoft OneDrive" home="https://onedrive.live.com/" config="/onedrive/" >}}
{{< provider name="Minio" home="https://www.minio.io/" config="/s3/#minio" >}}
+{{< provider name="Neon Object Storage" home="https://neon.com/docs/storage/overview" config="/s3/#neon" >}}
{{< provider name="Nextcloud" home="https://nextcloud.com/" config="/webdav/#nextcloud" >}}
{{< provider name="Blomp Cloud Storage" home="https://rclone.org/swift/" config="/swift/" >}}
{{< provider name="OpenDrive" home="https://www.opendrive.com/" config="/opendrive/" >}}
diff --git a/docs/content/s3.md b/docs/content/s3.md
index 4f41e83eb..e26320760 100644
--- a/docs/content/s3.md
+++ b/docs/content/s3.md
@@ -43,6 +43,7 @@ The S3 backend can be used with a number of different providers:
{{< provider name="Magalu Object Storage" home="https://magalu.cloud/object-storage/" config="/s3/#magalu" >}}
{{< provider name="MEGA S4 Object Storage" home="https://mega.io/objectstorage" config="/s3/#mega" >}}
{{< provider name="Minio" home="https://www.minio.io/" config="/s3/#minio" >}}
+{{< provider name="Neon Object Storage" home="https://neon.com/docs/storage/overview" config="/s3/#neon" >}}
{{< provider name="Outscale" home="https://en.outscale.com/storage/outscale-object-storage/" config="/s3/#outscale" >}}
{{< provider name="OVHcloud" home="https://www.ovhcloud.com/en/public-cloud/object-storage/" config="/s3/#ovhcloud" >}}
{{< provider name="Petabox" home="https://petabox.io/" config="/s3/#petabox" >}}
@@ -8104,6 +8105,157 @@ So once set up, for example, to copy files into a bucket
rclone copy /path/to/files minio:bucket
```
+### Neon Object Storage {#neon}
+
+[Neon Object Storage](https://neon.com/docs/storage/overview) is S3
+compatible object storage in a Neon project. Each branch of the project
+has its own copy of the buckets and its own S3 endpoint. The endpoint
+looks like `https://<branch_id>.storage.<cell>.<region>.aws.neon.tech`.
+
+In the Neon Console, click **Connect**, open the **Storage** tab and
+choose **Parameters only**. Use `AWS_ACCESS_KEY_ID`,
+`AWS_SECRET_ACCESS_KEY`, `AWS_ENDPOINT_URL_S3` and `AWS_REGION` as
+`access_key_id`, `secret_access_key`, `endpoint` and `region`. To make
+a separate credential, use **Settings** > **Credentials** or run
+`neon credentials create --scope storage:read --scope storage:write`.
+
+A credential works on the branch it was made on and on that branch's
+child branches. To use a child branch, change only `endpoint`.
+
+Neon also has these limits (see the
+[S3 compatibility docs](https://neon.com/docs/storage/s3-compatibility)):
+
+- The maximum object size is 5 GiB for a single upload and 10 GiB for
+ a multipart upload.
+- Neon rejects object keys containing control characters with
+ `400 InvalidURI`. To store files with these names, add `Ctl,Del` to
+ the encoding, for example `encoding = Slash,InvalidUtf8,Dot,Ctl,Del`.
+- Neon doesn't support ACLs over the S3 API. Leave `acl` and
+ `bucket_acl` empty and set bucket access (`private` or `public_read`)
+ in Neon.
+- Neon stores versioning and lifecycle configurations but doesn't
+ enforce them.
+- Neon may throttle heavy use with `503 SlowDown` errors. rclone retries
+ these.
+
+Here is an example of making a Neon Object Storage configuration. First run:
+
+```console
+rclone config
+```
+
+This will guide you through an interactive setup process.
+
+```text
+No remotes found, make a new one?
+n) New remote
+s) Set configuration password
+q) Quit config
+n/s/q> n
+
+Enter name for new remote.
+name> neon
+
+Option Storage.
+Type of storage to configure.
+Choose a number from below, or type in your own value.
+[snip]
+XX / Amazon S3 Compliant Storage Providers including AWS, ... Neon, ...
+ \ (s3)
+[snip]
+Storage> s3
+
+Option provider.
+Choose your S3 provider.
+Choose a number from below, or type in your own value.
+Press Enter to leave empty.
+[snip]
+XX / Neon Object Storage
+ \ (Neon)
+[snip]
+provider> Neon
+
+Option env_auth.
+Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
+Only applies if access_key_id and secret_access_key is blank.
+Choose a number from below, or type in your own boolean value (true or false).
+Press Enter for the default (false).
+ 1 / Enter AWS credentials in the next step.
+ \ (false)
+ 2 / Get AWS credentials from the environment (env vars or IAM).
+ \ (true)
+env_auth>
+
+Option access_key_id.
+AWS Access Key ID.
+Leave blank for anonymous access or runtime credentials.
+Enter a value. Press Enter to leave empty.
+access_key_id> NEON_ACCESS_KEY_ID
+
+Option secret_access_key.
+AWS Secret Access Key (password).
+Leave blank for anonymous access or runtime credentials.
+Enter a value. Press Enter to leave empty.
+secret_access_key> NEON_SECRET_ACCESS_KEY
+
+Option region.
+Region to connect to.
+Leave blank if you are using an S3 clone and you don't have a region.
+Choose a number from below, or type in your own value.
+Press Enter to leave empty.
+ 1 / US East (N. Virginia)
+ \ (us-east-1)
+ 2 / US East (Ohio)
+ \ (us-east-2)
+ 3 / Europe (Frankfurt)
+ \ (eu-central-1)
+ 4 / Asia Pacific (Singapore)
+ \ (ap-southeast-1)
+region> us-east-2
+
+Option endpoint.
+Endpoint for S3 API.
+Required when using an S3 clone.
+Enter a value. Press Enter to leave empty.
+endpoint> https://<branch_id>.storage.<cell>.<region>.aws.neon.tech
+
+Option bucket_object_lock_enabled.
+Enable Object Lock when creating new buckets.
+Enter a boolean value (true or false). Press Enter for the default (false).
+bucket_object_lock_enabled>
+
+Edit advanced config?
+y) Yes
+n) No (default)
+y/n> n
+
+Configuration complete.
+Options:
+- type: s3
+- provider: Neon
+- access_key_id: NEON_ACCESS_KEY_ID
+- secret_access_key: NEON_SECRET_ACCESS_KEY
+- region: us-east-2
+- endpoint: https://<branch_id>.storage.<cell>.<region>.aws.neon.tech
+Keep this "neon" remote?
+y) Yes this is OK (default)
+e) Edit this remote
+d) Delete this remote
+y/e/d> y
+```
+
+This will leave the config file looking like this.
+
+```ini
+[neon]
+type = s3
+provider = Neon
+access_key_id = NEON_ACCESS_KEY_ID
+secret_access_key = NEON_SECRET_ACCESS_KEY
+region = us-east-2
+endpoint = https://<branch_id>.storage.<cell>.<region>.aws.neon.tech
+```
+
### Netease NOS
For Netease NOS configure as per the configurator `rclone config`