"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Go azure-blobazure-blob-storageazure-filesbackblaze-b2cloud-storagedropboxencryptionftpfuse-filesystemgogolanggoogle-cloud-storagegoogle-driveonedriveopenstack-swiftrclones3sftpsyncwebdav
rc: add options/info call to enumerate options
Nick Craig-Wood8fbb25909165
diff --git a/docs/content/rc.md b/docs/content/rc.mdindex d0cec5cc8..b6e6f7e44 100644--- a/docs/content/rc.md+++ b/docs/content/rc.md@@ -400,6 +400,76 @@ call and taken by the [options/set](#options-set) calls as well as the- `BandwidthSpec` - this will be set and returned as a string, eg"1M".+### Option blocks {#option-blocks}++The calls [options/info](#options-info) (for the main config) and+[config/providers](#config-providers) (for the backend config) may be+used to get information on the rclone configuration options. This can+be used to build user interfaces for displaying and setting any rclone+option.++These consist of arrays of `Option` blocks. These have the following+format. Each block describes a single option.++| Field | Type | Optional | Description |+|-------|------|----------|-------------|+| Name | string | N | name of the option in snake_case |+| FieldName | string | N | name of the field used in the rc - if blank use Name |+| Help | string | N | help, started with a single sentence on a single line |+| Groups | string | Y | groups this option belongs to - comma separated string for options classification |+| Provider | string | Y | set to filter on provider |+| Default | any | N | default value, if set (and not to nil or "") then Required does nothing |+| Value | any | N | value to be set by flags |+| Examples | Examples | Y | predefined values that can be selected from list (multiple-choice option) |+| ShortOpt | string | Y | the short command line option for this |+| Hide | Visibility | N | if non zero, this option is hidden from the configurator or the command line |+| Required | bool | N | this option is required, meaning value cannot be empty unless there is a default |+| IsPassword | bool | N | set if the option is a password |+| NoPrefix | bool | N | set if the option for this should not use the backend prefix |+| Advanced | bool | N | set if this is an advanced config option |+| Exclusive | bool | N | set if the answer can only be one of the examples (empty string allowed unless Required or Default is set) |+| Sensitive | bool | N | set if this option should be redacted when using `rclone config redacted` |++An example of this might be the `--log-level` flag. Note that the+`Name` of the option becomes the command line flag with `_` replaced+with `-`.++```+{+ "Advanced": false,+ "Default": 5,+ "DefaultStr": "NOTICE",+ "Examples": [+ {+ "Help": "",+ "Value": "EMERGENCY"+ },+ {+ "Help": "",+ "Value": "ALERT"+ },+ ...+ ],+ "Exclusive": true,+ "FieldName": "LogLevel",+ "Groups": "Logging",+ "Help": "Log level DEBUG|INFO|NOTICE|ERROR",+ "Hide": 0,+ "IsPassword": false,+ "Name": "log_level",+ "NoPrefix": true,+ "Required": true,+ "Sensitive": false,+ "Type": "LogLevel",+ "Value": null,+ "ValueStr": "NOTICE"+},+```++Note that the `Help` may be multiple lines separated by `\n`. The+first line will always be a short sentence and this is the sentence+shown when running `rclone help flags`.+## Specifying remotes to work onRemotes are specified with the `fs=`, `srcFs=`, `dstFs=`@@ -638,7 +708,12 @@ See the [config paths](/commands/rclone_config_paths/) command for more informatReturns a JSON object:- providers - array of objects-See the [config providers](/commands/rclone_config_providers/) command for more information on the above.+See the [config providers](/commands/rclone_config_providers/) command+for more information on the above.++Note that the Options blocks are in the same format as returned by+"options/info". They are described in the+[option blocks](#option-blocks) section.**Authentication is required for this call.**@@ -1647,6 +1722,14 @@ set in _config then use options/config and for _filter use options/filter.This shows the internal names of the option within rclone which shouldmap to the external options very easily with a few exceptions.+### options/info: Get info about all the global options {#options-info}++Returns an object where keys are option block names and values are an+array of objects with info about each options.++These objects are in the same format as returned by "config/providers". They are+described in the [option blocks](#option-blocks) section.+### options/local: Get the currently active config for this call {#options-local}Returns an object with the keys "config" and "filter".diff --git a/fs/config/rc.go b/fs/config/rc.goindex 114dcffd1..ba75b8f7b 100644--- a/fs/config/rc.go+++ b/fs/config/rc.go@@ -91,7 +91,12 @@ func init() {Returns a JSON object:- providers - array of objects-See the [config providers](/commands/rclone_config_providers/) command for more information on the above.+See the [config providers](/commands/rclone_config_providers/) command+for more information on the above.++Note that the Options blocks are in the same format as returned by+"options/info". They are described in the+[option blocks](#option-blocks) section.`,})}diff --git a/fs/fs_test.go b/fs/fs_test.goindex 15e78853d..e0099e96a 100644--- a/fs/fs_test.go+++ b/fs/fs_test.go@@ -234,10 +234,8 @@ func TestOptionMarshalJSON(t *testing.T) {"Name": "case_insensitive","FieldName": "","Help": "",-"Provider": "","Default": false,"Value": true,-"ShortOpt": "","Hide": 0,"Required": false,"IsPassword": false,diff --git a/fs/rc/config.go b/fs/rc/config.goindex 1f538472d..5fd243914 100644--- a/fs/rc/config.go+++ b/fs/rc/config.go@@ -12,19 +12,6 @@ import ("github.com/rclone/rclone/fs/filter")-// AddOption adds an option set-func AddOption(name string, option interface{}) {- // FIXME remove this function when conversion to options is complete- fs.RegisterGlobalOptions(fs.OptionsInfo{Name: name, Opt: option})-}--// AddOptionReload adds an option set with a reload function to be-// called when options are changed-func AddOptionReload(name string, option interface{}, reload func(context.Context) error) {- // FIXME remove this function when conversion to options is complete- fs.RegisterGlobalOptions(fs.OptionsInfo{Name: name, Opt: option, Reload: reload})-}-func init() {Add(Call{Path: "options/blocks",@@ -73,6 +60,29 @@ func rcOptionsGet(ctx context.Context, in Params) (out Params, err error) {return out, nil}+func init() {+ Add(Call{+ Path: "options/info",+ Fn: rcOptionsInfo,+ Title: "Get info about all the global options",+ Help: `Returns an object where keys are option block names and values are an+array of objects with info about each options.++These objects are in the same format as returned by "config/providers". They are+described in the [option blocks](#option-blocks) section.+`,+ })+}++// Show the info of all the option blocks+func rcOptionsInfo(ctx context.Context, in Params) (out Params, err error) {+ out = make(Params)+ for _, opt := range fs.OptionsRegistry {+ out[opt.Name] = opt.Options+ }+ return out, nil+}+func init() {Add(Call{Path: "options/local",diff --git a/fs/rc/config_test.go b/fs/rc/config_test.goindex de9c38070..1f12bed82 100644--- a/fs/rc/config_test.go+++ b/fs/rc/config_test.go@@ -20,6 +20,16 @@ func clearOptionBlock() func() {}}+var testInfo = fs.Options{{+ Name: "string",+ Default: "str",+ Help: "It is a string",+}, {+ Name: "int",+ Default: 17,+ Help: "It is an int",+}}+var testOptions = struct {String stringInt int@@ -28,10 +38,18 @@ var testOptions = struct {Int: 42,}+func registerTestOptions() {+ fs.RegisterGlobalOptions(fs.OptionsInfo{Name: "potato", Opt: &testOptions, Options: testInfo})+}++func registerTestOptionsReload(reload func(context.Context) error) {+ fs.RegisterGlobalOptions(fs.OptionsInfo{Name: "potato", Opt: &testOptions, Options: testInfo, Reload: reload})+}+func TestAddOption(t *testing.T) {defer clearOptionBlock()()assert.Equal(t, len(fs.OptionsRegistry), 0)- AddOption("potato", &testOptions)+ registerTestOptions()assert.Equal(t, len(fs.OptionsRegistry), 1)assert.Equal(t, &testOptions, fs.OptionsRegistry["potato"].Opt)}@@ -40,7 +58,7 @@ func TestAddOptionReload(t *testing.T) {defer clearOptionBlock()()assert.Equal(t, len(fs.OptionsRegistry), 0)reload := func(ctx context.Context) error { return nil }- AddOptionReload("potato", &testOptions, reload)+ registerTestOptionsReload(reload)assert.Equal(t, len(fs.OptionsRegistry), 1)assert.Equal(t, &testOptions, fs.OptionsRegistry["potato"].Opt)assert.Equal(t, fmt.Sprintf("%p", reload), fmt.Sprintf("%p", fs.OptionsRegistry["potato"].Reload))@@ -48,7 +66,7 @@ func TestAddOptionReload(t *testing.T) {func TestOptionsBlocks(t *testing.T) {defer clearOptionBlock()()- AddOption("potato", &testOptions)+ registerTestOptions()call := Calls.Get("options/blocks")require.NotNil(t, call)in := Params{}@@ -60,7 +78,7 @@ func TestOptionsBlocks(t *testing.T) {func TestOptionsGet(t *testing.T) {defer clearOptionBlock()()- AddOption("potato", &testOptions)+ registerTestOptions()call := Calls.Get("options/get")require.NotNil(t, call)in := Params{}@@ -76,8 +94,8 @@ func TestOptionsGetMarshal(t *testing.T) {ci := fs.GetConfig(ctx)// Add some real options- AddOption("main", ci)- AddOption("rc", &Opt)+ fs.RegisterGlobalOptions(fs.OptionsInfo{Name: "main", Opt: ci, Options: nil})+ fs.RegisterGlobalOptions(fs.OptionsInfo{Name: "rc", Opt: &Opt, Options: nil})// get themcall := Calls.Get("options/get")@@ -92,11 +110,23 @@ func TestOptionsGetMarshal(t *testing.T) {require.NoError(t, err)}+func TestOptionsInfo(t *testing.T) {+ defer clearOptionBlock()()+ registerTestOptions()+ call := Calls.Get("options/info")+ require.NotNil(t, call)+ in := Params{}+ out, err := call.Fn(context.Background(), in)+ require.NoError(t, err)+ require.NotNil(t, out)+ assert.Equal(t, Params{"potato": testInfo}, out)+}+func TestOptionsSet(t *testing.T) {defer clearOptionBlock()()var reloaded int- AddOptionReload("potato", &testOptions, func(ctx context.Context) error {- if reloaded > 0 {+ registerTestOptionsReload(func(ctx context.Context) error {+ if reloaded > 1 {return errors.New("error while reloading")}reloaded++@@ -114,8 +144,8 @@ func TestOptionsSet(t *testing.T) {require.NoError(t, err)require.Nil(t, out)assert.Equal(t, 50, testOptions.Int)- assert.Equal(t, "hello", testOptions.String)- assert.Equal(t, 1, reloaded)+ assert.Equal(t, "str", testOptions.String)+ assert.Equal(t, 2, reloaded)// error from reload_, err = call.Fn(context.Background(), in)diff --git a/fs/registry.go b/fs/registry.goindex 22d424284..2c46194cb 100644--- a/fs/registry.go+++ b/fs/registry.go@@ -195,11 +195,11 @@ type Option struct {FieldName string // name of the field used in the rc JSON - will be auto filled normallyHelp string // help, start with a single sentence on a single line that will be extracted for command line helpGroups string `json:",omitempty"` // groups this option belongs to - comma separated string for options classification- Provider string // set to filter on provider+ Provider string `json:",omitempty"` // set to filter on providerDefault interface{} // default value, nil => "", if set (and not to nil or "") then Required does nothingValue interface{} // value to be set by flagsExamples OptionExamples `json:",omitempty"` // predefined values that can be selected from list (multiple-choice option)- ShortOpt string // the short option for this if required+ ShortOpt string `json:",omitempty"` // the short option for this if requiredHide OptionVisibility // set this to hide the config from the configurator or the command lineRequired bool // this option is required, meaning value cannot be empty unless there is a defaultIsPassword bool // set if the option is a password@@ -348,7 +348,7 @@ func (os OptionExamples) Sort() { sort.Sort(os) }type OptionExample struct {Value stringHelp string- Provider string+ Provider string `json:",omitempty"`}// Register a filesystem@@ -417,6 +417,7 @@ var OptionsRegistry = map[string]OptionsInfo{}//// Packages which need global options should use this in an init() functionfunc RegisterGlobalOptions(oi OptionsInfo) {+ oi.Options.setValues()OptionsRegistry[oi.Name] = oiif oi.Opt != nil && oi.Options != nil {err := oi.Check()@@ -429,7 +430,10 @@ func RegisterGlobalOptions(oi OptionsInfo) {var optionName = regexp.MustCompile(`^[a-z0-9_]+$`)// Check ensures that for every element of oi.Options there is a field-// in oi.Opt that matches it+// in oi.Opt that matches it.+//+// It also sets Option.FieldName to be the name of the field for use+// in JSON.func (oi *OptionsInfo) Check() error {errCount := errcount.New()items, err := configstruct.Items(oi.Opt)@@ -471,6 +475,8 @@ func (oi *OptionsInfo) Check() error {//errCount.Add(err)Errorf(nil, "%s", err)}+ // Set FieldName+ option.FieldName = item.Field}return errCount.Err(fmt.Sprintf("internal error: options block %q", oi.Name))}