RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.
Go agent-harnessagent-native-databaseagentic-aiagentic-retrievalagentic-searchaiai-agentscontext-enginecontext-engineeringcontext-managementharness-engineeringknowledge-compilationragretrieval-augmented-generationretrieval-harnesssearch-harness
docs: update migration (#20476)
YanZhangf8b1a178a3d2
diff --git a/docs/administrator/configurations/configurations.md b/docs/administrator/configurations/configurations.mdindex f7f62960fd..508e780d93 100644--- a/docs/administrator/configurations/configurations.md+++ b/docs/administrator/configurations/configurations.md@@ -201,7 +201,7 @@ s3:- `region_name`: Must be `auto`.- `endpoint_url`: `https://t3.storage.dev`, or `https://fly.storage.tigris.dev` on Fly.io.- `addressing_style`: Must be `virtual`.-- `bucket` / `prefix_path`: Optional. Enables single-bucket mode — see [Migrate from multi-bucket to single-bucket mode](../migration/backup_and_migration.md#migrate-from-multi-bucket-to-single-bucket-mode).+- `bucket` / `prefix_path`: Optional. Enables single-bucket mode. Before changing the storage layout, create a recovery point by following [Backup and Restore (v1.x)](../migration/backup_and_restore_v1.md).When using an external storage backend, you can remove the `minio` service from `docker-compose-base.yml`.diff --git a/docs/administrator/migration/_category_.json b/docs/administrator/migration/_category_.jsonindex 394ddfcb4b..a32ac45921 100644--- a/docs/administrator/migration/_category_.json+++ b/docs/administrator/migration/_category_.json@@ -2,7 +2,7 @@"label": "Migration","position": 2,"link": {- "type": "generated-index",- "description": "Guides for data migration, official and third-party."+ "type": "doc",+ "id": "administrator/migration/migration_overview"}}diff --git a/docs/administrator/migration/backup_and_migration.md b/docs/administrator/migration/backup_and_migration.mddeleted file mode 100644index fffb4ea9fb..0000000000--- a/docs/administrator/migration/backup_and_migration.md+++ /dev/null@@ -1,103 +0,0 @@-----sidebar_position: 2-title: "Go Backup and Migration"-sidebar_label: "Go Backup and Migration"-slug: /migration-sidebar_custom_props: {- categoryIcon: LucideLocateFixed-}------# Go Backup and Migration--This guide explains how to move the default Go Docker deployment to another host.--The bundled `docker/migration.sh` covers only the MySQL, MinIO, Redis, and Elasticsearch volumes and is not a complete backup method for the Go Docker deployment. It does not include Kvrocks or optional service volumes. Use the volume procedure below to back up a Go deployment.--## Move a Go deployment to another host--### 1. Identify all persistent data--From the repository root, inspect the running deployment and its Docker volumes:--```bash-docker ps-docker volume ls-```--Record the Compose project name and the volumes actually mounted by its running containers before stopping them. For each relevant container shown by `ps`, check its mounts with `docker inspect <container-name> --format '{{range .Mounts}}{{println .Name .Destination}}{{end}}'`. `docker volume ls` also shows unrelated and unused volumes, so do not treat that list as the backup set. The default project name is usually `docker`; if you started Compose with `-p ragflow`, volume names normally start with `ragflow_`. Use the same `-p` value with every Compose command below.--The default Go stack can use these volumes, depending on the enabled services:--| Service or data | Typical volume |-|---------------------------------------------|-----------------------------|-| Metadata database | `<project>_mysql_data` |-| Uploaded objects in bundled MinIO | `<project>_minio_data` |-| Search index with Elasticsearch | `<project>_esdata01` |-| Go cache and persistent Redis-protocol data | `<project>_kvrocks_data` |-| NATS JetStream data | `<project>_nats_data` |-| ClickHouse analytics data | `<project>_clickhouse_data` |--Also retain `docker/.env`, the configuration template, and any custom certificates or mounted files. Protect the backup because these files may contain credentials.--### 2. Stop writers and archive the volumes--Stop the Go deployment and any other process that writes to the same stores. For a Compose project named `docker`:--```bash-docker compose --env-file docker/.env -f docker/docker-compose.yml down-mkdir backup-go-```--Create a new, empty `backup-go` directory for this backup; do not reuse a directory containing earlier archives. Repeat the following command for **each volume you recorded**. Replace `docker_mysql_data` with its exact name; the example creates `backup-go/docker_mysql_data.tar.gz`:--```bash-volume_name=docker_mysql_data-if docker volume inspect "$volume_name" >/dev/null 2>&1; then- if docker run --rm -v "$volume_name":/source:ro -v "$PWD/backup-go":/backup alpine:3.20 \- tar czf "/backup/$volume_name.tar.gz" -C /source .; then- tar tzf "backup-go/$volume_name.tar.gz" > /dev/null- else- echo "Backup failed: $volume_name"- fi-else- echo "Missing volume: $volume_name"-fi-```--Keep the volume name in each archive filename and copy the entire `backup-go` directory to the target host. Verify that every selected volume has a readable archive.--### 3. Restore on the target host---Install the matching Go deployment and configuration on the target host. Keep services stopped and put `backup-go` in the repository root.--For each archived Docker volume, restore into a **new, empty volume**. Replace both names in this example and repeat. Keep the source name when reading the archive; use the target Compose project's name when creating its volume:--```bash-source_volume=docker_mysql_data-target_volume=docker_mysql_data-if ! test -f "backup-go/$source_volume.tar.gz"; then- echo "Missing archive: backup-go/$source_volume.tar.gz"-elif docker volume inspect "$target_volume" >/dev/null 2>&1; then- echo "Target volume already exists; inspect it before restoring: $target_volume"-else- if docker volume create "$target_volume"; then- docker run --rm -v "$target_volume":/target -v "$PWD/backup-go":/backup:ro alpine:3.20 \- tar xzf "/backup/$source_volume.tar.gz" -C /target- fi-fi-```--If the target Compose project has a different name, set `target_volume` to that project's volume name while retaining `source_volume` from the archive. Do not restore over a populated volume: extracting an archive does not remove files already present. Keep the metadata database, object data, and search index from the same stopped deployment together.--### 4. Start and verify--Start the Go stack with the target configuration and the same Compose project name used for its volumes:--```bash-docker compose --env-file docker/.env -f docker/docker-compose.yml up -d-docker ps-```--The Go image's entrypoint runs the standalone database migration before starting its enabled server modes. Check the container logs for migration errors before using the service. Confirm that an existing dataset can list and open files and that search still returns its documents.diff --git a/docs/administrator/migration/backup_and_restore_v0.md b/docs/administrator/migration/backup_and_restore_v0.mdnew file mode 100644index 0000000000..1d5849f807--- /dev/null+++ b/docs/administrator/migration/backup_and_restore_v0.md@@ -0,0 +1,181 @@+---+sidebar_position: 2+title: Backup and Restore (v0.x)+sidebar_label: Backup and Restore (v0.x)+slug: /backup_and_restore_v0+sidebar_custom_props: {+ categoryIcon: LucideLocateFixed+}+---++# Backup and Restore (v0.x)++:::info Version scope+This document applies to RAGFlow `v0.x` Docker deployments. For `v1.0.0-rc1` and later, see [Backup and Restore (v1.x)](./backup_and_restore_v1.md).+:::++:::warning This guide does not upgrade RAGFlow+This guide creates or restores a same-version recovery point and can move a deployment to another host. To upgrade from `v0.x` to `v1.x`, complete this backup first and then follow [Upgrade from v0.x to v1.x](./upgrade_from_v0_to_v1.md). The upgrade must pass through `v0.27.2` and then `v1.0.0-rc1`; that step is irreversible.+:::++Choose the steps for your task:++- **Create an upgrade recovery point:** Complete [steps 1–3](#1-find-the-data-to-back-up), verify every archive, and keep the backup unchanged.+- **Move to another host:** Complete all six steps, including transfer, restore, and verification.++## 1. Find the data to back up++From the repository root, first list the containers in the RAGFlow Compose project. This shows which services are enabled:++```bash+docker compose -p docker -f docker/docker-compose.yml ps --all+```++Display the mounts of every container in the Compose project:++```bash+for container_id in $(docker compose -p docker -f docker/docker-compose.yml ps --all -q); do+ docker inspect "$container_id" --format '{{.Name}}{{range .Mounts}}{{println "" .Type .Source .Name .Destination}}{{end}}'+done+```++The `docker` project commonly uses these volumes:++| Data | Typical volume |+|------|----------------|+| Metadata database | `docker_mysql_data` |+| Uploaded objects in bundled MinIO | `docker_minio_data` |+| Redis data | `docker_redis_data` |+| Elasticsearch index | `docker_esdata01` |++Use these four default volumes as the starting point. If the deployment uses Infinity instead of Elasticsearch, include its persistent data. Add every named volume and persistent host directory shown by the mount check.++If any RAGFlow data is stored outside the Docker volumes listed above, back it up at the same time.++For a non-standard storage layout, identify the required data by following the Docker documentation for [volumes](https://docs.docker.com/engine/storage/volumes/) and [bind mounts](https://docs.docker.com/engine/storage/bind-mounts/).++Also retain `docker/.env`, the configuration template, custom certificates, and custom-mounted files. Protect these files because they can contain credentials.++Before stopping the deployment, run `df -h` and `docker system df`. Make sure the host has enough free space for the volume archives and temporary backup files.++For an offline host, cache the image required by the chosen method before stopping RAGFlow: `alpine` for `docker/migration.sh`, or `alpine:3.20` for the manual commands below.++## 2. Stop RAGFlow++Stop RAGFlow and its bundled services. This keeps the database, object storage, and search index at the same point in time while the backup is created:++```bash+docker compose -p docker -f docker/docker-compose.yml down+```++Wait until the command finishes before creating the archives.++## 3. Back up the persistent volumes++Create a timestamped backup directory:++```bash+backup_dir="backup-ragflow-v0-$(date +%Y%m%d-%H%M%S)"+mkdir "$backup_dir"+```++The `date` command adds the current time to the directory name so that files from different backups are not mixed.++Choose one backup method:++| Deployment | Backup method |+|------------|---------------|+| `v0.20.1` or later with the four default MySQL, MinIO, Redis, and Elasticsearch volumes | Use the bundled script. |+| `v0.20.0`, Infinity, or any non-default volume set | Use the manual procedure for every recorded volume. |++For a deployment with the four default volumes, create the archives with:++```bash+bash docker/migration.sh backup "$backup_dir"+```++Otherwise, do not use the script. Repeat the following commands for **every exact volume name recorded in step 1**:++```bash+volume_name=docker_mysql_data+docker volume inspect "$volume_name" >/dev/null &&+ docker run --rm -v "$volume_name":/source:ro -v "$PWD/$backup_dir":/backup alpine:3.20 \+ tar czf "/backup/$volume_name.tar.gz" -C /source .+```++After all required volumes have been processed, list its contents:++```bash+for archive in "$backup_dir"/*.tar.gz; do tar tzf "$archive" > /dev/null || exit 1; done+ls -lh "$backup_dir"+```++Confirm that every recorded volume has a readable archive and keep all files in the same backup set.++## 4. Transfer the backup++Copy the complete backup directory and retained configuration files to the target host. Keep archive filenames unchanged. Use a transfer method appropriate for your environment.++On the target host, list the transferred directory and confirm that its files and sizes match the source. Keep the source deployment and the backup available until the restored deployment passes the checks in step 6.++## 5. Restore on the target host++Install the same RAGFlow release on the target host and place the backup directory in the repository root. Keep all target services stopped.++Set `backup_dir` to the transferred directory name:++```bash+backup_dir="<backup-directory-name>"+```++Restore only into target volumes that do not yet exist. Both the bundled script and the manual procedure extract files without removing files already present in a volume. Restoring into a used volume can therefore mix old and restored data.++Use the bundled restore command only when the backup was created by a compatible version of the script and contains all four default archives:++```bash+bash docker/migration.sh restore "$backup_dir"+```++If the backup was created manually, set `volume_name` to each saved volume and repeat the following procedure:++```bash+volume_name=docker_mysql_data+if ! test -f "$backup_dir/$volume_name.tar.gz"; then+ echo "Missing archive: $backup_dir/$volume_name.tar.gz"+elif docker volume inspect "$volume_name" > /dev/null 2>&1; then+ echo "Target volume already exists; stop and inspect it: $volume_name"+else+ docker volume create "$volume_name" &&+ docker run --rm -v "$volume_name":/target -v "$PWD/$backup_dir":/backup:ro alpine:3.20 \+ tar xzf "/backup/$volume_name.tar.gz" -C /target+fi+```++If the deployment uses Infinity, restore it from its saved volume or with its storage-specific procedure. Restore any additional volumes and external services from the same backup set before continuing.++If a manual restore fails, do not use the incomplete target volume. After confirming that it contains no data that must be retained, replace it with a new empty volume before retrying the restore.++If the restore script reports that a target volume already exists, enter `n` and stop. Restore only after replacing it with a confirmed empty volume.++## 6. Start and verify the restored release++Start the same RAGFlow version that created the backup. The first command starts the containers; the second displays their current state:++```bash+docker compose -p docker -f docker/docker-compose.yml up -d+docker compose -p docker -f docker/docker-compose.yml ps+```++Wait until the services report a running or healthy state. Then confirm that:++- Existing users can sign in.+- Knowledge bases and files are present.+- Existing files can be opened.+- Retrieval returns content from previously parsed documents.+- A new document can be uploaded, parsed, and retrieved.+- The configured model providers and default models are available.+- Agents and enabled data-source synchronization still work.++When this backup is part of a version upgrade, first verify the restored source version. Then follow [Upgrade from v0.x to v1.x](./upgrade_from_v0_to_v1.md), starting and validating every intermediate version in the standard route.++After all checks pass, the backup and restore are complete.diff --git a/docs/administrator/migration/backup_and_restore_v1.md b/docs/administrator/migration/backup_and_restore_v1.mdnew file mode 100644index 0000000000..04aabc11fe--- /dev/null+++ b/docs/administrator/migration/backup_and_restore_v1.md@@ -0,0 +1,147 @@+---+sidebar_position: 3+title: "Backup and Restore (v1.x)"+sidebar_label: "Backup and Restore (v1.x)"+slug: /backup_and_restore_v1+sidebar_custom_props: {+ categoryIcon: LucideLocateFixed+}+---++# Backup and Restore (v1.x)++:::info Backup and host migration only+This guide creates or restores a same-version recovery point and can move a deployment to another host. To upgrade an existing v1 deployment to a later release, create this backup first and then follow [Upgrade from v1.x to later releases](./upgrade_from_v1_to_later_releases.md).+:::++Choose the steps for your task:++- **Create an upgrade recovery point:** Complete [steps 1–2](#1-find-the-data-to-back-up), verify every archive, and keep the backup unchanged.+- **Move to another host:** Complete all four steps, including restore and verification.++## 1. Find the data to back up++List all containers in the `docker` Compose project to identify the services that are enabled:++```bash+docker compose -p docker -f docker/docker-compose.yml ps --all+```++Display the mounts of every container in the Compose project:++```bash+for container_id in $(docker compose -p docker -f docker/docker-compose.yml ps --all -q); do+ docker inspect "$container_id" --format '{{.Name}}{{range .Mounts}}{{println "" .Type .Source .Name .Destination}}{{end}}'+done+```++Write down every RAGFlow volume name and every persistent host directory shown in the output. Use the container mounts as the backup list. RAGFlow volume names normally start with `docker_`.++Use the following table to check the list you recorded. Include each service enabled in your deployment, together with any additional or externally managed storage used in your environment.++| Service or data | Persistent location |+|-----------------|---------------------|+| Metadata database in bundled MySQL | `docker_mysql_data` |+| Uploaded objects in bundled MinIO | `docker_minio_data` |+| Elasticsearch index | `docker_esdata01` |+| Infinity index | `docker_infinity_data` |+| Kvrocks cache and checkpoint data | `docker_kvrocks_data` |+| NATS JetStream data | `docker_nats_data` |+| Kibana data, when Kibana is enabled | `docker_kibana_data` |+| RAGFlow logs | `docker/ragflow-logs` bind mount |++This table is a checklist. Always use the mounts reported by `docker inspect` as the final backup scope.++For a non-standard storage layout, identify the required data by following the Docker documentation for [volumes](https://docs.docker.com/engine/storage/volumes/) and [bind mounts](https://docs.docker.com/engine/storage/bind-mounts/).++RAGFlow logs are retained for troubleshooting. Missing logs do not affect the restoration of business data.+If you need the logs for troubleshooting, copy `docker/ragflow-logs` with the backup.++If any RAGFlow data is stored outside the Docker volumes listed above, back it up at the same time.++Also retain `docker/.env`, the configuration template, and any custom certificates or mounted files. Protect the backup because these files may contain credentials.++Before stopping the deployment, run `df -h` and `docker system df`. Make sure the host has enough free space for the volume archives and temporary backup files.++For an offline host, make sure that the `alpine:3.20` image is available before stopping RAGFlow.++## 2. Stop RAGFlow and create the backup++First, stop RAGFlow and its bundled services. This prevents the database, object storage, and search index from changing while their archives are being created:++```bash+docker compose -p docker -f docker/docker-compose.yml down+```++After all containers have stopped, create a new directory for this backup. The volume archives created below will be stored in this directory:++```bash+backup_dir="backup-ragflow-v1-$(date +%Y%m%d-%H%M%S)"+mkdir "$backup_dir"+```++The `date` command adds the current time to the directory name so that files from different backups are not mixed. Then repeat the following commands for **each volume you recorded**. Set `volume_name` to the exact Docker volume name:++```bash+volume_name=docker_mysql_data+docker volume inspect "$volume_name" >/dev/null &&+ docker run --rm -v "$volume_name":/source:ro -v "$PWD/$backup_dir":/backup alpine:3.20 \+ tar czf "/backup/$volume_name.tar.gz" -C /source . &&+ tar tzf "$backup_dir/$volume_name.tar.gz" >/dev/null+```++A successful command returns without an error and creates a readable archive.++After processing all volumes, list the archives:++```bash+ls -lh "$backup_dir"+```++Confirm that every recorded volume has a corresponding `.tar.gz` file. Keep the entire backup directory together with `docker/.env`, the configuration template, certificates, and other custom-mounted files.++## 3. Restore the backup on the target host++Copy the complete backup to the target host and install the same RAGFlow release. Restore the saved configuration, put the backup directory in the repository root, and keep all services stopped. Set `backup_dir` to the transferred directory name:++```bash+backup_dir="<backup-directory-name>"+```++For each archive, set `volume_name` to the volume name contained in the archive filename and run:++```bash+volume_name=docker_mysql_data+if ! test -f "$backup_dir/$volume_name.tar.gz"; then+ echo "Missing archive: $backup_dir/$volume_name.tar.gz"+elif docker volume inspect "$volume_name" >/dev/null 2>&1; then+ echo "Target volume already exists; inspect it before restoring: $volume_name"+else+ docker volume create "$volume_name" &&+ docker run --rm -v "$volume_name":/target -v "$PWD/$backup_dir":/backup:ro alpine:3.20 \+ tar xzf "/backup/$volume_name.tar.gz" -C /target+fi+```++If extraction fails, do not use the incomplete target volume. After confirming that it contains no data that must be retained, replace it with a new empty volume before retrying the restore.++Repeat this operation until every archive has been restored. Restore the metadata database, object data, search index, and other service data from the same backup set.++Restore every saved bind-mounted directory to the same location used on the source host.++## 4. Start and verify++Start RAGFlow with the restored volumes:++```bash+docker compose -p docker -f docker/docker-compose.yml up -d+```++Check the service states and the RAGFlow application logs:++```bash+docker compose -p docker -f docker/docker-compose.yml ps --all+docker compose -p docker -f docker/docker-compose.yml logs --tail=200 ragflow-cpu+```++Finally, sign in and confirm that existing knowledge bases and files are present, files can be opened, and retrieval returns content from previously parsed documents. After these checks pass, the backup and restore are complete.diff --git a/docs/administrator/migration/database_schema_and_migration.md b/docs/administrator/migration/database_schema_and_migration.mddeleted file mode 100644index 07d86e0869..0000000000--- a/docs/administrator/migration/database_schema_and_migration.md+++ /dev/null@@ -1,48 +0,0 @@-----sidebar_position: 1-title: Database Schema and Migration-sidebar_label: Database Schema and Migration-slug: /database_schema_and_migration-sidebar_custom_props: {- categoryIcon: LucideLocateFixed-}------# Database Schema and Migration--For a manually started Go backend, run database migration as a standalone step before starting any server process:--```bash-./bin/ragflow_server --migrate-```--The command loads the same database configuration as the Go server, applies schema and data changes, then exits. Use `-f` or `--config` to select a configuration file when needed. Do not combine `--migrate` with `--admin`, `--api`, `--ingestor`, or `--syncer`. Allow enough time for data backfills and schema changes on large databases.--The migration process does not start a server mode or initialize the document engine, Kvrocks cache, object storage, or NATS JetStream message queue. It needs access to the configured metadata database and local configuration files. Run it where `conf/models` is available: database initialization loads the model provider definitions from that relative path and fails if the directory cannot be read. The model data migration also tries to read `conf/llm_factories.json`; if the file is absent or unreadable, that input is skipped. Keep the matching configuration files with the new binary.--## Where migrations are defined--Go migration logic lives in `internal/dao/`. `database.go` runs the migration sequence: manual changes in `migration.go` and related files, GORM `AutoMigrate` for the Go entities in `internal/entity/`, and the conversation history backfill. Model provider data migration is in `model_migration.go`; its version handling is in `migration_version.go`. These are Go source files, not a directory of generated, numbered SQL migration files.--The standalone command runs the full sequence. Ordinary server startup connects to the database and may create or update a limited set of runtime tables, but does not run the full manual and data migration sequence. Do not rely on server startup to perform an upgrade.--## Database version and startup check--The Go migration marker is stored in `system_settings` under the fixed key `mysql_migration.database.version`. The key name is retained for database compatibility and does not select another migration implementation. The Go implementation writes `v0.26.0` after the base tenant-model step and `v0.27.2` after the follow-up model-data step. When the stored version is below `v1.0.0-rc1`, the conversation-history migration writes that value after it completes, even if the old history columns or rows are absent and no data needs copying. A step already covered by the stored version is skipped. The last completed step therefore determines the value that remains in the row. This marker is **not** a complete version number for every table or schema change. A missing or unparseable marker does not prove that the database is up to date.--Each Go server process compares its code version with this marker after database initialization. That initialization may already have updated runtime tables, indexes, or built-in templates before the check runs. The comparison uses the release-number portion of each version; development suffixes are not used to establish ordering. If the database version is newer, startup fails with `Refusing to start: database was migrated by a newer version`. If the marker is absent or the versions cannot be compared, this downgrade check does not block startup. The check does not apply pending migrations, and the standalone `--migrate` command does not run the downgrade check. Verify the binary and target database before invoking it.--## Upgrade procedure--1. Back up the metadata database and verify that you can restore it. See [Backup and Migration](./backup_and_migration.md).-2. Stop or drain all Go server processes that use the database.-3. Deploy the new Go binary and its matching configuration. For a manually started deployment, run `./bin/ragflow_server --migrate` against the target database and wait for it to exit before starting servers. In a deployment with multiple replicas, coordinate a single migration job and wait for it to finish before starting the replicas. The Go-only Docker entrypoint runs this command before its enabled Go server modes; account for that behavior when planning a separate migration job.-4. Start `--admin` first, then `--api` and `--ingestor`; start `--syncer` if file synchronization is enabled.--Check the migration logs as well as the exit status before starting services. Some schema conflicts are logged and skipped, and built-in template seeding failures are logged as warnings; a successful exit alone does not verify every schema change or template row. The migration command can be rerun after a failure, but do not assume every database DDL operation is transactional. Check the error and database state before retrying. The Go backend has no supported automatic rollback command or generated reverse migration. To return to an older release, restore a compatible database backup along with the older binary.--## Development mode--`RAGFLOW_DEV_MODE=true` disables only the code-versus-database downgrade check for Go server processes. It does not run migrations, reverse schema changes, or make an older binary compatible with a newer database. This is especially relevant to development builds: the conversation-history migration records `v1.0.0-rc1.dev1` even when the checkout still reports a `v0.27.x` release. Use it only for a development database in that situation. Set it for each affected server process; keep it unset in production.--This migration procedure applies to the default MySQL metadata database.diff --git a/docs/administrator/migration/migration_overview.md b/docs/administrator/migration/migration_overview.mdnew file mode 100644index 0000000000..4c124fe886--- /dev/null+++ b/docs/administrator/migration/migration_overview.md@@ -0,0 +1,72 @@+---+sidebar_position: 0+title: Migration Guide+sidebar_label: Start Here+slug: /migration_overview+sidebar_custom_props: {+ categoryIcon: LucideLocateFixed+}+---++# Migration Guide++Use this page to identify the correct path, prepare a recoverable backup, and then open the procedure for your task.++:::danger Mandatory and irreversible v0.x to v1.x upgrade+Every `v0.x` deployment must successfully start and pass verification on `v0.27.2`, and must then be upgraded to `v1.0.0-rc1`. Do not skip either version or upgrade directly from `v0.x` to a later `v1.x` release.++The `v0.27.2` to `v1.0.0-rc1` migration is irreversible. Before starting it, stop all data-writing tasks and create a complete, verified `v0.27.2` recovery point. If migration fails, do not retry against the partially migrated data; restore the complete recovery point before trying again.+:::++:::note Command assumptions+The commands run from the repository root, use the `docker` Compose project, and specify `docker/docker-compose.yml`. Docker Compose therefore loads `docker/.env` automatically. If your files use a different layout, adjust the commands by following the [Docker Compose environment-file rules](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/).+:::++## 1. Document map: choose your task++| Current version | Goal | Follow these guides |+|-----------------|------|---------------------|+| `v0.x` | Back up, restore, or move the deployment to another host | [Backup and Restore (v0.x)](./backup_and_restore_v0.md) |+| `v0.x` | Upgrade to the current latest version | Create and verify an upgrade recovery point by following [Backup and Restore (v0.x)](./backup_and_restore_v0.md), then follow [Upgrade from v0.x to v1.x](./upgrade_from_v0_to_v1.md) |+| `v1.0.0-rc1` or later | Back up, restore, or move the deployment to another host | [Backup and Restore (v1.x)](./backup_and_restore_v1.md) |+| `v1.0.0-rc1` or later | Upgrade to a later release | First complete [Backup and Restore (v1.x)](./backup_and_restore_v1.md), then follow [Upgrade from v1.x to later releases](./upgrade_from_v1_to_later_releases.md) |++## 2. Standard upgrade route from v0.x++Use this route for all v0.x upgrades:++```text+v0.20.0 -> v0.22.1 -> v0.25.6 -> v0.26.4 -> v0.27.2 -> v1.0.0-rc1 -> latest v1.x release+```++The key checkpoints are:++| Checkpoint | Why it is included |+|------------|--------------------|+| [`v0.22.1`](./upgrade_from_v0_to_v1.md#21-v0221) | Handles the bundled embedding model transition. |+| [`v0.25.6`](./upgrade_from_v0_to_v1.md#22-v0256) | Required compatibility checkpoint before the model-provider migration. |+| [`v0.26.4`](./upgrade_from_v0_to_v1.md#23-v0264) | Runs and verifies the base model-provider migration. |+| [`v0.27.2`](./upgrade_from_v0_to_v1.md#24-v0272) | Completes the required v0.x migration work. |+| [`v1.0.0-rc1`](./upgrade_from_v0_to_v1.md#3-upgrade-from-v0272-to-v100-rc1) | Mandatory, irreversible boundary between v0.x and v1.x. |++Start with the first checkpoint that is newer than your current version. Never downgrade to an earlier checkpoint or start an older RAGFlow version against data already used by a newer version.++If the current deployment is older than `v0.20.0`, first upgrade it to `v0.20.0` using the files supplied with that release. Agents created before `v0.20.0` are incompatible with `v0.20.0` and later; retain their definitions only as a reference, then rebuild and test them after reaching `v0.20.0`.++## 3. Prepare before changing versions++1. Record the current RAGFlow version, enabled services, whether Elasticsearch or Infinity is used, storage backend, and custom-mounted files.+2. Stop document imports, parsing jobs, Agents, synchronization, and other data-writing tasks.+3. Back up the configuration and every persistent volume from the same stopped deployment.+4. Restore the backup in a separate test environment and verify that the restored version can start and retrieve existing documents.+5. Keep the original backup unchanged until that version has passed acceptance testing.++Use a separate test host so the test deployment cannot connect to production volumes or data stores.++## 4. Complete the migration safely++For a version upgrade, start and verify every checkpoint in the route. Merely changing image tags or copying volumes does not run the checkpoint's data migration.++At each checkpoint, verify sign-in, existing files, retrieval, a new document upload and parse, model configuration, and a representative Agent run. Continue only after the current checkpoint passes these checks.++The `v0.27.2` to `v1.0.0-rc1` migration is irreversible. If it fails, stop `v1.0.0-rc1` and restore the application version, configuration, and all persistent data from the same `v0.27.2` recovery point. Do not retry against partially migrated data.diff --git a/docs/administrator/migration/upgrade_from_v0_to_v1.md b/docs/administrator/migration/upgrade_from_v0_to_v1.mdnew file mode 100644index 0000000000..1e657e8e08--- /dev/null+++ b/docs/administrator/migration/upgrade_from_v0_to_v1.md@@ -0,0 +1,223 @@+---+sidebar_position: 1+title: Upgrade from v0.x to v1.x+sidebar_label: Upgrade from v0.x to v1.x+slug: /upgrade_from_v0_to_v1+sidebar_custom_props: {+ categoryIcon: LucideLocateFixed+}+---++# Upgrade from v0.x to v1.x++This guide provides the step-by-step procedure for upgrading a MySQL-based Docker deployment from RAGFlow `v0.20.0` or later to the current latest version.++Before continuing, review the standard route in [Migration Guide](./migration_overview.md) and complete the backup and preparation checklist.++:::danger Do not skip the v0.x to v1.x boundary+Every `v0.x` deployment must successfully start and pass verification on `v0.27.2`, and must then be upgraded to `v1.0.0-rc1`. Do not upgrade directly from an earlier `v0.x` release to `v1.0.0-rc1` or a later `v1.x` release.++The `v0.27.2` to `v1.0.0-rc1` migration is irreversible. Create and verify a complete `v0.27.2` recovery point before starting it. If migration fails, restore that recovery point before trying again.+:::++## Quick navigation++- [Upgrade within v0.x releases](#1-upgrade-within-between-v0x-releases)+- [Special instructions for each v0.x checkpoint](#2-handle-each-key-release)+- [Upgrade from v0.27.2 to v1.0.0-rc1](#3-upgrade-from-v0272-to-v100-rc1)+- [Upgrade from v1.0.0-rc1 to a later v1.x release](#4-upgrade-from-v100-rc1-to-a-later-v1x-release)++For every v0.x checkpoint below, first follow the [v0.x-to-v0.x upgrade steps](#1-upgrade-between-v0x-releases), and then complete the linked version-specific instructions.++**Upgrade route:**++[`v0.22.1`](#21-v0221) → [`v0.25.6`](#22-v0256) → [`v0.26.4`](#23-v0264) → [`v0.27.2`](#24-v0272) → [`v1.0.0-rc1`](#3-upgrade-from-v0272-to-v100-rc1) → [latest v1.x release](#4-upgrade-from-v100-rc1-to-a-later-v1x-release)++Start with the first checkpoint that is newer than your current version. Never downgrade to an earlier checkpoint.++After the final upgrade, complete [Verify the upgraded deployment](#5-verify-the-upgraded-deployment). If any step fails, follow [Recover from a failed upgrade](#6-recover-from-a-failed-upgrade).++## 1. Upgrade within v0.x releases++Treat each arrow in the route as a separate upgrade. You **must start and verify every target release in the route**. Changing the image tag repeatedly, or copying volumes without starting the intermediate release, does not execute that release's database and data migrations.++### 1.1. Stop writers and create a checkpoint++From the current release directory, stop the deployment:++```bash+docker compose -p docker -f docker/docker-compose.yml down+```++For critical deployments, create a new backup before each version change so that every checkpoint has its own recovery point.++### 1.2. Deploy the target release files++Create a separate working directory for the next checkpoint. Set `target_version` to the next version in the route:++```bash+target_version=v0.22.1+git fetch --tags+git worktree add "../ragflow-$target_version" "$target_version"+cd "../ragflow-$target_version"+```++Change `target_version=v0.22.1` to the version required for the current checkpoint.++If the worktree command fails because the directory or tag already exists, see the [Git worktree documentation](https://git-scm.com/docs/git-worktree.html).++Copy the required settings into the new `docker/.env` without replacing the file, then confirm that `RAGFLOW_IMAGE` uses the target version:++```bash+grep '^RAGFLOW_IMAGE=' docker/.env+```++### 1.3. Start one deployment and monitor migration++Start the target release:++```bash+docker compose -p docker -f docker/docker-compose.yml up -d+docker compose -p docker -f docker/docker-compose.yml ps --all+```++From the `SERVICE` column, copy the RAGFlow application service name. Replace `<service-name>` below with that value:++```bash+docker compose -p docker -f docker/docker-compose.yml logs -f "<service-name>"+```++Keep one application replica running and do not resume data-writing tasks until the migration logs show no errors or unresolved warnings.++### 1.4. Verify the checkpoint++At every checkpoint, confirm that:++- Existing users can sign in and see their knowledge bases.+- Existing files can be listed and opened.+- Retrieval returns content from previously parsed documents.+- A new document can be uploaded, parsed, and retrieved.+- Model providers, credentials, model types, API endpoints, and tenant default models are correct.+- Existing Agents open and complete a representative run.+- The selected Elasticsearch or Infinity service and object storage are healthy.++Do not continue to the next checkpoint until failures at the current release have been resolved.++## 2. Handle each key release++These sections contain only the additional checks for each release. Use them together with the [v0.x-to-v0.x upgrade steps](#1-upgrade-between-v0x-releases).++### 2.1. v0.22.1++Use `v0.22.1` rather than `v0.22.0`. If a knowledge base used the embedding model from the former `full` image, select an available replacement model. If the compatibility check fails, use another compatible model or re-parse the existing chunks with the new model.++### 2.2. v0.25.6++This checkpoint is required. Start `v0.25.6`, complete the standard verification, and continue only after all checks pass.++### 2.3. v0.26.4++Keep `--init-model-provider-tables` enabled on the first start. After startup, check the migration logs and verify the providers, credentials, models, and tenant default models in the UI. Test each model type used by the deployment before continuing.++### 2.4. v0.27.2++Keep `--init-model-provider-tables` enabled on the first start. Confirm that the migration logs report success, then test model configuration, existing retrieval, and a new document upload and parse.++Do not continue until `v0.27.2` passes all checks. The older GraphRAG and RAPTOR configuration pages are no longer available, but previously generated content remains searchable.++## 3. Upgrade from v0.27.2 to v1.0.0-rc1++This mandatory step is different from the earlier checkpoints. It introduces the v1 runtime and performs an irreversible data migration.++### 3.1. Stop and back up v0.27.2++Make sure that document parsing, synchronization, and Agent tasks have finished. From the `v0.27.2` repository root, stop the deployment:++```bash+docker compose -p docker -f docker/docker-compose.yml down+```++If the deployment uses the four default MySQL, MinIO, Redis, and Elasticsearch volumes, create and verify the backup with:++```bash+backup_dir="backup-v0.27.2-$(date +%Y%m%d-%H%M%S)"+bash docker/migration.sh backup "$backup_dir"+for archive in mysql_backup.tar.gz minio_backup.tar.gz redis_backup.tar.gz es_backup.tar.gz; do+ tar tzf "$backup_dir/$archive" >/dev/null || exit 1+done+```++If the deployment uses Infinity or any non-default volume set, use the manual method in [Backup and Restore (v0.x)](./backup_and_restore_v0.md) for every recorded volume. Do not combine script-created and manually created volume archives.++- Continue only if all required archives pass the checks, then keep the backup unchanged for rollback.+- Also save `docker/.env`, custom certificates, mounted configuration files, and any additional volumes.+- Keep the Redis or Valkey backup for rollback only; do not restore it to v1.++### 3.2. Prepare the v1.0.0-rc1 deployment++`v1.0.0-rc1` includes several changes outside the database:++- Deprecated HTTP API aliases are removed.+- NATS replaces Redis as the message queue.+- Kvrocks stores cache and checkpoints.+- The local sandbox and the earlier Team/Me permission behavior are not supported in this release.++Redis or Valkey data files are not compatible with Kvrocks or NATS storage. Restore or reuse MySQL, object storage, and the same Elasticsearch or Infinity data, but start Kvrocks, NATS, and ClickHouse with new empty volumes. Do not mount the `v0.27.2` Redis or Valkey volume into any of these services. Finish or stop queued work before the backup, and submit any incomplete jobs again after the upgrade.++Prepare a separate repository directory at the `v1.0.0-rc1` tag. Use its Compose files and `.env` template, then copy the required values from the old configuration into the new template. Set the release image to:++```dotenv+RAGFLOW_IMAGE=infiniflow/ragflow:v1.0.0-rc1+```++When reusing compatible volumes on the same host, keep the `docker` volume-name prefix. On a new host, restore the MySQL, MinIO, and Elasticsearch or Infinity archives with their original names. Let Compose create new Kvrocks, NATS, and ClickHouse volumes.++Keep parallel application replicas stopped until one migration attempt has completed. For a custom orchestrated deployment, use that platform's documentation to reduce and restore the replica count. Large databases need enough startup time for schema changes and conversation-history backfills.++### 3.3. Start v1.0.0-rc1 and wait for migration++From the `v1.0.0-rc1` repository root, start the deployment:++```bash+docker compose -p docker -f docker/docker-compose.yml up -d+docker compose -p docker -f docker/docker-compose.yml logs -f ragflow-cpu+```++Keep the logs open until the migration finishes and RAGFlow starts. Press `Ctrl+C` to exit the log view without stopping the containers.++Check the container state and the HTTP health endpoint:++```bash+docker compose -p docker -f docker/docker-compose.yml ps+curl -f http://127.0.0.1/api/v1/system/healthz+```++The health-check command uses the default web port `80`. If `SVR_WEB_HTTP_PORT` has been changed in `docker/.env`, use that port in the URL.++If the logs report a migration error, stop the v1 deployment and preserve the logs. Do not retry against the partially migrated data. Restore the complete `v0.27.2` recovery point, resolve the cause in a separate test environment, and retry the upgrade only from the restored recovery point.++The migration is irreversible. Never start `v0.27.2` against a database that has been partially or fully migrated by `v1.0.0-rc1`.++## 4. Upgrade from v1.0.0-rc1 to a later v1.x release++First verify that `v1.0.0-rc1` started successfully and that its migration completed. If the target version is newer, create a new v1 backup and then follow [Upgrade from v1.x to later releases](./upgrade_from_v1_to_later_releases.md). Do not skip directly from `v0.27.2` to the later release.++## 5. Verify the upgraded deployment++Sign in to RAGFlow and confirm that existing data and configuration are present, then test document retrieval, a new document upload, one Chat, and one Agent run.++Keep the `v0.27.2` backup until the upgraded deployment has completed acceptance testing and operated successfully under normal workload.++## 6. Recover from a failed upgrade++Stop the target release and preserve its logs. Do not continue to the next checkpoint, rerun migrations blindly, or start an older application image against a database already modified by a newer release.++Restore the application and all persistent stores from the same checkpoint backup:++- The metadata database.+- Object storage and Elasticsearch or Infinity data.+- Cache, queue, analytics, and other enabled persistent volumes.+- Configuration files, certificates, and custom-mounted files.+- The matching RAGFlow image and Compose files.++RAGFlow does not provide an automatic reverse migration. Mixing an older application with newer database or service data is not a rollback.diff --git a/docs/administrator/migration/upgrade_from_v1_to_later_releases.md b/docs/administrator/migration/upgrade_from_v1_to_later_releases.mdnew file mode 100644index 0000000000..fc5af1fe66--- /dev/null+++ b/docs/administrator/migration/upgrade_from_v1_to_later_releases.md@@ -0,0 +1,90 @@+---+sidebar_position: 4+title: Upgrade from v1.x to later releases+sidebar_label: Upgrade from v1.x to later releases+slug: /upgrade_from_v1_to_later_releases+sidebar_custom_props: {+ categoryIcon: LucideLocateFixed+}+---++# Upgrade from v1.x to later releases++:::info Version scope+This document applies to MySQL-based RAGFlow `v1.0.0-rc1` and later Docker deployments. To upgrade from an earlier release, first follow [Upgrade from v0.x to v1.x](./upgrade_from_v0_to_v1.md).+:::++The commands below start one application instance. For a custom multi-replica deployment, use the orchestration platform's documentation to reduce it to one application instance before migration and restore the remaining instances only after migration succeeds.++## Before upgrading++1. Stop document imports, parsing jobs, data-source synchronization, and other processes that can write data.+2. Back up the metadata database and all other persistent data from the same stopped deployment. Verify that the backup can be restored. See [Backup and Restore (v1.x)](./backup_and_restore_v1.md).+3. Use the Docker Compose files, environment template, and image shipped with the target release. Copy the required settings from the existing deployment into the new template.+4. Keep enough free time in the maintenance window for schema changes and data backfills. Large databases can take longer to migrate.++## Run the migration++### 1. Stop the current deployment++Stop the existing deployment before installing the target release files:++```bash+docker compose -p docker -f docker/docker-compose.yml down+```++### 2. Start the target release++Install the target release files, merge the required settings into its `docker/.env` template, and start the deployment:++```bash+docker compose -p docker -f docker/docker-compose.yml up -d+```++For a MySQL metadata database, RAGFlow runs the required migration during startup.++### 3. Monitor the migration++Follow the RAGFlow application logs:++```bash+docker compose -p docker -f docker/docker-compose.yml logs -f ragflow-cpu+```++Keep the log view open until the migration finishes and the application starts. Press `Ctrl+C` to leave the log view; the containers continue running. Review warnings as well as errors before allowing users, ingestion workers, or synchronization jobs to write data.++If the logs report a migration failure, stop the deployment, preserve the logs, and follow [Roll back after a failed migration](#roll-back-after-a-failed-migration).++### 4. Check the deployment status++Display the state of the containers:++```bash+docker compose -p docker -f docker/docker-compose.yml ps+```++Wait until the expected services report a running or healthy state, then complete the checks below.++## Verify the upgraded deployment++After migration, confirm that:++- All expected containers are running and healthy.+- Existing users can sign in.+- Existing knowledge bases and files are visible.+- Existing files can be opened and retrieved.+- A new document can be uploaded, parsed, and retrieved.+- Existing conversation history is available.+- Model providers and tenant default models are configured correctly.+- Enabled ingestion and file-synchronization services can complete new jobs.++## Roll back after a failed migration++Rollback is performed by restoring the complete pre-upgrade backup together with its matching RAGFlow release and configuration.++To return to the previous release:++1. Stop the failed deployment.+2. Restore the metadata database, object storage, search index, cache, queue, and other persistent data from the same pre-upgrade backup point.+3. Restore the matching Docker Compose files and configuration.+4. Start the previous RAGFlow release and repeat the verification checks.