Skip to content

Your lightweight open-source personal agent.

3.5 KB
Raw Blame

How to Configure Model Fallback in nanobot

Model fallback lets nanobot try a primary model first, then fall back to one or more named presets when the primary provider fails or rate-limits.

What you will build

  • two or more modelPresets
  • a primary agents.defaults.modelPreset
  • an ordered agents.defaults.fallbackModels chain

When to use this

Use fallback when you want better reliability across rate limits, provider outages, local model downtime, or cost-sensitive routing.

Before you start

Complete Quick Start if nanobot is not installed or cannot reply yet. If you already have a working WebUI or CLI session, continue here using that instance; no reinstall or second gateway is needed.

Verify each provider works before adding it as a fallback.

Minimal working example

In Settings → Models, create the primary and backup presets, then use Enable preset and arrange their order. The first enabled preset is primary; later entries are tried as fallbacks. Save and test each preset before relying on the chain.

Manual configuration

Merge this shape into ~/.nanobot/config.json and replace provider/model names with ones you control:

{
  "modelPresets": {
    "Fast": {
      "provider": "primary-provider",
      "model": "primary-model-id",
      "maxTokens": 4096,
      "contextWindowTokens": 65536,
      "temperature": 0.1
    },
    "Deep": {
      "provider": "fallback-provider",
      "model": "fallback-model-id",
      "maxTokens": 4096,
      "contextWindowTokens": 200000,
      "temperature": 0.1
    }
  },
  "agents": {
    "defaults": {
      "modelPreset": "Fast",
      "fallbackModels": ["Deep"]
    }
  }
}

String entries in fallbackModels are preset names, not raw model IDs. Replace the placeholder model IDs with currently supported model IDs from your provider. The Provider Cookbook has concrete recipes for common providers.

Production notes

  • In the WebUI, a reply produced by a named fallback preset shows its provider logo and preset name next to the reply timestamp. Hover, focus, or click the badge for an explanation. The composer still shows your selected preset; primary replies have no fallback badge. Saved replies retain the name used for that invocation, even after a preset is renamed or deleted. Older replies without recorded attribution and unnamed inline fallback objects are not labeled.
  • Keep fallback context windows realistic; smaller fallback windows constrain how much context can fit.
  • Put cheaper or faster fallbacks before expensive ones when acceptable.
  • Use /model <preset> for runtime switching without editing config.
  • Keep preset names human-readable; the same name appears in the WebUI and /model.

Security notes

  • Different providers may have different data handling policies.
  • Do not put provider keys directly in shared config files.
  • Confirm fallback models can safely receive the same prompts and files.

Troubleshooting

  • If a fallback never triggers, confirm the primary error is treated as retryable/fallbackable.
  • If startup fails, check that each fallback string matches a key under modelPresets.
  • If output is truncated after fallback, review maxTokens and contextWindowTokens.

Related nanobot docs