"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Go azure-blobazure-blob-storageazure-filesbackblaze-b2cloud-storagedropboxencryptionftpfuse-filesystemgogolanggoogle-cloud-storagegoogle-driveonedriveopenstack-swiftrclones3sftpsyncwebdav
onedrive: make --onedrive-delta quicker when not listing from the drive root
Nick Craig-Wood3ac301eff1f7
diff --git a/backend/onedrive/onedrive.go b/backend/onedrive/onedrive.goindex 4673947e7..0df28a265 100644--- a/backend/onedrive/onedrive.go+++ b/backend/onedrive/onedrive.go@@ -438,21 +438,12 @@ Setting this flag speeds up these things greatly:rclone size onedrive:rclone rc vfs/refresh recursive=true-**However** the delta listing API **only** works at the root of the-drive. If you use it not at the root then it recurses from the root-and discards all the data that is not under the directory you asked-for. So it will be correct but may not be very efficient.--This is why this flag is not set as the default.--As a rule of thumb if nearly all of your data is under rclone's root-directory (the |root/directory| in |onedrive:root/directory|) then-using this flag will be a big performance win. If your data is-mostly not under the root then using this flag will be a big-performance loss.--It is recommended if you are mounting your onedrive at the root-(or near the root when using crypt) and using rclone |rc vfs/refresh|.+Rclone asks for the delta listing of the directory being listed. If+the drive only supports delta listings at the root of the drive (as+some older OneDrive for Business and SharePoint drives do) then+rclone lists from the root and discards all the data that is not+under the directory you asked for. So it will be correct but may not+be very efficient.`, "|", "`"),Advanced: true,}, {@@ -1511,10 +1502,6 @@ func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (return err}- // ListR only works at the root of a onedrive, not on a folder- // So we have to filter things outside of the root which is- // inefficient.-list := list.NewHelper(callback)// list a folder conventionally - used for shared folders@@ -1590,17 +1577,26 @@ func (f *Fs) ListR(ctx context.Context, dir string, callback fs.ListRCallback) (return nil}- opts := rest.Opts{- Method: "GET",- Path: "/root/delta",- Parameters: map[string][]string{- // "token": {token},+ listDelta := func(opts rest.Opts) error {+ opts.Parameters = url.Values{"$top": {fmt.Sprintf("%d", f.opt.ListChunk)},- },+ }+ var result api.DeltaResponse+ return f._listAll(ctx, "", false, false, fn, &opts, &result, &result.Value, &result.NextLink)}- var result api.DeltaResponse- err = f._listAll(ctx, "", false, false, fn, &opts, &result, &result.Value, &result.NextLink)+ err = listDelta(f.newOptsCall(directoryID, "GET", "/delta"))+ // Some drives only support delta listings at the root of the+ // drive, in which case list the whole drive and filter out the+ // items outside dir.+ var apiErr *api.Error+ if err != nil && len(seen) == 0 && errors.As(err, &apiErr) {+ fs.Debugf(f, "Delta listing of directory failed, listing from the root of the drive instead: %v", err)+ err = listDelta(rest.Opts{+ Method: "GET",+ Path: "/root/delta",+ })+ }if err != nil {return err}diff --git a/docs/content/onedrive.md b/docs/content/onedrive.mdindex 007e741ca..05f608bd2 100644--- a/docs/content/onedrive.md+++ b/docs/content/onedrive.md@@ -326,10 +326,11 @@ This can be useful with `rclone mount` and [rclone rc vfs/refreshrecursive=true](/rc/#vfs-refresh)) to very quickly fill the mount withinformation about all the files.-The API used for the recursive listing (`ListR`) only supports listing-from the root of the drive. This will become increasingly inefficient-the further away you get from the root as rclone will have to discard-files outside of the directory you are using.+Rclone asks for the delta listing of the directory being listed. Some+drives only support delta listings from the root of the drive, in+which case rclone lists the whole drive and discards the files outside+of the directory you are using, which is increasingly inefficient the+further away from the root you get.Some commands (like `rclone lsf -R`) will use `ListR` by default - youcan turn this off with `--disable ListR` if you need to.