OpenProject is the leading open source project management software for product, project and portfolio management. A powerful Jira alternative with agile planning, issue tracking, roadmaps, Gantt charts, time tracking, collaboration features, and more. Available on premises or in the cloud. ⭐ Star us on GitHub
Ruby agile-developmentangularbcfboardsbug-trackerganttgantt-chartifcissue-trackerjira-alternativekanbanopenprojectproduct-managementproject-managementproject-planningroadmaprubyscrumtimelineworkflows
[#65106] Document postgres 17 update for OpenProject 16.2 via dump & restore workflow
Tizian Rößler91fa03d0ee16
diff --git a/docs/installation-and-operations/misc/migration-to-postgresql17/README.md b/docs/installation-and-operations/misc/migration-to-postgresql17/README.mdindex 7a27036ceaf..e4488d84064 100644--- a/docs/installation-and-operations/misc/migration-to-postgresql17/README.md+++ b/docs/installation-and-operations/misc/migration-to-postgresql17/README.md@@ -1,240 +1,304 @@# Migrating your OpenProject installation to PostgreSQL 17-OpenProject version 16+ will default to PostgreSQL 17. If you have an existing OpenProject installation, please follow the guide below to upgrade your PostgreSQL version.+OpenProject version 16+ will default to PostgreSQL 17. If you have an existing OpenProject installation, please follow the guide below to upgrade your PostgreSQL version.+For the time being, using an older Postgres version is still possible, but not recommended.+This documentation shows how to upgrade PostgreSQL via a SQL dump.+If you prefer doing the upgrade using the [in-place method](https://www.postgresql.org/docs/current/pgupgrade.html), you are free to do so.-## Package-based installation-> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/packaged/).+## Docker Compose++> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/docker/).> Before attempting the upgrade, please ensure you have performed a backup of your installation by following the [backup guide](../../operation/backing-up/).-Please first check whether this guide applies to you at all. Only PostgreSQL installations that were installed by the OpenProject package are applicable to this guide.+### 1. Backup the Current PostgreSQL Database-To do that, please run the following command:+Create a backup of your current PostgreSQL database. Run the following command from your OpenProject project directory:-```shell-sudo cat /etc/openproject/installer.dat | grep postgres/autoinstall+```bash+docker compose exec -it -u postgres db pg_dump -d openproject -x -O > openproject.sql```-And verify that it outputs: postgres/autoinstall **install**.+This creates a backup named `openproject.sql`.-If that is not the case, you are likely using a self-provisioned database or a remote database. In this case, please follow the instructions from your provider or use generic PostgreSQL upgrade guides. A guide we can recommend for Debian/Ubuntu based servers is [this one](https://gorails.com/guides/upgrading-postgresql-version-on-ubuntu-server). Please adapt that guide or the following steps to your distribution.+### 2. Stop OpenProject Services-In the following, we assume that you initially let OpenProject setup your PostgreSQL installation, using a local database.+Shut down all running containers:-NOTE: RedHat and CentOS are slightly different, depending on which PostgreSQL package/repository will be used.-For the documentation parts titled RedHat/CentOS RedHat Enterprise Linux 8 was used.+```bash+docker compose down+```-1. First, connect to your server and make sure your local version is PostgreSQL v13:+### 3. Prepare for PostgreSQL 17-For Debian/Ubuntu:+To upgrade to PostgreSQL 17, you need to override the default database image.-```shell-sudo cat /var/lib/postgresql/13/main/PG_VERSION-13-```+#### Create `docker-compose.override.yml`-For RedHat/CentOS:+Create a file named `docker-compose.override.yml` in the docker compose directory.+Replace the default PostgreSQL version with 17.+Also, define a new volume for the upgraded database to avoid overwriting the existing one.-```shell-sudo cat /var/lib/pgsql/13/data/PG_VERSION-13-```+Here’s an example configuration:-2. Install the new version of PostgreSQL:+```yaml+volumes:+ pgdata17:-For Debian/Ubuntu:+services:+ db:+ image: postgres:17+ volumes:+ - pgdata17:/var/lib/postgresql/data-```shell-sudo apt-get update-sudo apt-get install postgresql-17-sudo pg_createcluster 17 main --start```-For RedHat/CentOS:+### 4. Start the New Database Container-```shell-sudo yum install postgresql17 postgresql17-contrib-sudo /usr/bin/postgresql-17-setup initdb+With your override file in place, start only the database container:++```bash+docker compose up db -d```-3. Stop the PostgreSQL servers:+This starts a clean PostgreSQL 17 container with an empty data directory.-For Debian/Ubuntu:+### 5. Restore the Backup to PostgreSQL 17-```shell-sudo su - postgres -c "/usr/lib/postgresql/13/bin/pg_ctl stop --wait --pgdata=/var/lib/postgresql/13/main"-sudo su - postgres -c "/usr/lib/postgresql/17/bin/pg_ctl stop --wait --pgdata=/var/lib/postgresql/17/main"+Now that the new database is running, restore your backup:++```bash+docker compose exec -T -u postgres db psql -d openproject < openproject.sql```-For RedHat/CentOS:+This will import your data into the new PostgreSQL 17 container.-```shell-sudo su - postgres -c "/usr/pgsql-13/bin/pg_ctl stop --wait --pgdata=/var/lib/pgsql/13/data"-sudo su - postgres -c "/usr/pgsql-17/bin/pg_ctl stop --wait --pgdata=/var/lib/pgsql/17/data"+### 6. Start the Full OpenProject Stack++With your data restored, bring up the rest of the OpenProject services:++```bash+docker compose up -d```-4. Migrate your data to PostgreSQL 17:+### Confirmation-For Debian/Ubuntu:+You now have OpenProject running with PostgreSQL 17.+Verify everything works correctly by visiting your OpenProject instance in the browser.-```shell-sudo su - postgres <<CMD-/usr/lib/postgresql/17/bin/pg_upgrade \- --old-bindir=/usr/lib/postgresql/13/bin \- --new-bindir=/usr/lib/postgresql/17/bin \- --old-datadir=/var/lib/postgresql/13/main \- --new-datadir=/var/lib/postgresql/17/main \- --old-options '-c config_file=/etc/postgresql/13/main/postgresql.conf' \- --new-options '-c config_file=/etc/postgresql/17/main/postgresql.conf'-CMD++## Docker All-in-One++> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/docker/#all-in-one-container).+> Before attempting the upgrade, please ensure you have performed a backup of your installation by following the [backup guide](../../operation/backing-up/).++This only works if you are using OpenProject >= 16.2 because older versions have PostgreSQL 13 as the default database.++### 1. Backup the Existing Database+++Create a PostgreSQL dump using:++```bash+docker exec -it $OP_CONTAINER_NAME su - postgres -c 'pg_dump -d openproject -x -O' > openproject.sql```-For RedHat/CentOS:+This command connects to the running container and exports the database into a SQL file on your host machine.-```shell-sudo su - postgres <<CMD-/usr/pgsql-17/bin/pg_upgrade \- --old-bindir=/usr/pgsql-13/bin \- --new-bindir=/usr/pgsql-17/bin \- --old-datadir=/var/lib/pgsql/13/data \- --new-datadir=/var/lib/pgsql/17/data \- --old-options '-c config_file=/var/lib/pgsql/13/data/postgresql.conf' \- --new-options '-c config_file=/var/lib/pgsql/17/data/postgresql.conf'-CMD+### 2. Stop the OpenProject container++```bash+docker stop $OP_CONTAINER_NAME```-5. Make PostgreSQL v17 the new default server to run on port 45432:+### 3. Start a New PostgreSQL 17 Container-For Debian/Ubuntu:+Run a fresh PostgreSQL 17 container using a new volume:-```shell-sudo su - postgres -c "cp /etc/postgresql/{13,17}/main/conf.d/custom.conf"-sudo su - postgres -c "sed -i 's|45432|45433|' /etc/postgresql/13/main/conf.d/custom.conf"-sudo su - postgres -c "cp /etc/postgresql/13/main/pg_hba.conf /etc/postgresql/17/main/pg_hba.conf"-sudo su - postgres -c "/usr/lib/postgresql/17/bin/pg_ctl start --wait --pgdata=/var/lib/postgresql/17/main -o '-c config_file=/etc/postgresql/17/main/postgresql.conf'"+```bash+docker run --rm -d --name postgres \+-e POSTGRES_PASSWORD=postgres \+-e LANG=C.UTF-8 \+-e LC_ALL=C.UTF-8 \+-v /var/lib/openproject/pgdata17:/var/lib/postgresql/data \+postgres:17```-For RedHat/CentOS:+### 4. Create OpenProject user-```shell-sudo su - postgres -c "mkdir -p /var/lib/pgsql/17/data/conf.d"+Connect to the new PostgreSQL 17 container and drop the `openproject` database:-sudo su - postgres -c "vi /var/lib/pgsql/17/data/postgresql.conf"-# at the section CONFIG FILE INCLUDES, please add the include directory conf.d-include_dir = 'conf.d'+```bash+echo "CREATE USER openproject WITH PASSWORD 'openproject';" | docker exec -i postgres psql -U postgres+```-sudo su - postgres -c "cp -p /var/lib/pgsql/13/data/conf.d/custom.conf /var/lib/pgsql/17/data/conf.d/custom.conf"-sudo su - postgres -c "sed -i 's|45432|45433|' /var/lib/pgsql/13/data/conf.d/custom.conf"-sudo su - postgres -c "/usr/pgsql-17/bin/pg_ctl start --wait --pgdata=/var/lib/pgsql/17/data -o '-c config_file=/var/lib/pgsql/17/data/postgresql.conf'"+### 5. Create a New Database-# Getting the password for the PostgreSQL database from the configuration-sudo openproject config:get DATABASE_URL-postgres://openproject:[CRYPTICAL-PASSWORD-STRING]@127.0.0.1:45432/openproject+Now create a fresh `openproject` database:-# Alter the password for user openproject in pgsql17-[root@openproject ~]# su - postgres-[postgres@openproject ~]$ psql --port 45432-psql (17.6)-Type "help" for help.-postgres=# ALTER USER openproject WITH PASSWORD '[CRYPTICAL-PASSWORD-STRING]';-ALTER ROLE-postgres=# \q-[postgres@openproject ~]$ logout+```bash+echo "CREATE DATABASE openproject OWNER openproject;" | docker exec -i postgres psql -U postgres```-6. Check your OpenProject installation on the GUI. A version higher than `17.0` should be displayed for `PostgreSQL version` in the "Administration > Information" section.+### 6. Restore the Database from the Dump++Restore your data:-7. If everything is fine, you can then remove your older PostgreSQL installation:+```bash+docker exec -i postgres psql -U openprojet -d openproject < openproject.sql+```-For Debian/Ubuntu:+This imports your backup into the newly created database.-```shell-sudo rm -rf /var/lib/postgresql/13/main-sudo apt-get purge postgresql-13+### 7. Stop PostgreSQL container++```bash+docker stop postgres```-For RedHat/CentOS:+### 8. Relaunch OpenProject with PostgreSQL 17-```shell-sudo rm -rf /var/lib/pgsql/13/data-sudo yum remove postgresql13+You can now run a new OpenProject container connected to your upgraded PostgreSQL 17 data volume:++```bash+docker run -d -p 8080:80 --name openproject \+ -e OPENPROJECT_HOST__NAME=openproject.example.com \+ -e OPENPROJECT_SECRET_KEY_BASE=secret \+ -v /var/lib/openproject/pgdata17:/var/openproject/pgdata \+ -v /var/lib/openproject/assets:/var/openproject/assets \+ openproject/openproject:16```-## Compose-based docker installation+Make sure the environment variables and version match your setup.-> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/docker/).-> Before attempting the upgrade, please ensure you have performed a backup of your installation by following the [backup guide](../../operation/backing-up/).+### Confirmation-You can find the upgrade instructions for your docker-compose setup in the [openproject-docker-compose](https://github.com/opf/openproject-docker-compose/blob/stable/16/control/README.md#upgrade) repository.+Visit your OpenProject instance to confirm everything works as expected.-Remember that you need to have checked out that repository and work in the `compose` directory for the instructions to work.-## All-in-one docker installation-> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/docker/#all-in-one-container).+## Package-Based Installation++> Please follow this section only if you have installed OpenProject using [this procedure](../../installation/packaged/).> Before attempting the upgrade, please ensure you have performed a backup of your installation by following the [backup guide](../../operation/backing-up/).-The newer version of OpenProject includes an utility to automatically perform the upgrade for you. Assuming you followed the standard installation procedure, the folder (within the docker container) containing your PostgreSQL data will be located at `/var/openproject/pgdata`.+### 1. Stop OpenProject-Then the goal is to take this folder, and apply `pg_upgrade` on it. This will generate an upgraded cluster in another folder. We can finally switch the old postgres folder with the upgraded one and restart the container.+```bash+sudo service openproject stop+```-First, ensure that you have stopped your container:+### 2. Backup the Database-```shell-docker stop openproject+```bash+pg_dump $(sudo openproject config:get DATABASE_URL) -x -O > openproject.sql```-Once the docker has stopped, you are ready to run the upgrade command. In this case, we assume that your existing PostgreSQL data is stored on the host at `/var/lib/openproject/pgdata`. We will also map a local folder named `/var/lib/openproject/pgdata-next` to a special volume in the container, named `/var/openproject/pgdata-next`. This volume will contain the upgraded cluster:+### 3. Stop Existing PostgreSQL-```shell-docker run --rm -it \- -v /var/lib/openproject/pgdata:/var/openproject/pgdata \- -v /var/lib/openproject/pgdata-next:/var/openproject/pgdata-next \- openproject/openproject:16 root ./docker/prod/postgres-db-upgrade+#### On Debian/Ubuntu:+```bash+sudo pg_ctlcluster 13 main stop```-If everything goes well, the process should end with a message as follows:+#### On CentOS/RHEL and SLES:+```bash+sudo systemctl stop postgresql-13+```-```text-Upgrade Complete------------------Optimizer statistics are not transferred by pg_upgrade so,-once you start the new server, consider running:- ./analyze_new_cluster.sh+### 4. Install PostgreSQL 17-Running this script will delete the old cluster's data files:- ./delete_old_cluster.sh+#### On Debian/Ubuntu:+```bash+sudo apt update+sudo apt install postgresql-17+sudo pg_createcluster 17 main --start```-You can then perform the following operation to switch the upgraded PostgreSQL with the older version:+#### On CentOS/RHEL:+```bash+sudo dnf install -y postgresql17-server postgresql17-contrib+sudo /usr/pgsql-17/bin/postgresql-17-setup initdb+sudo systemctl enable --now postgresql-17+```-```shell-sudo mv /var/lib/openproject/pgdata /var/lib/openproject/pgdata-prev-sudo mv /var/lib/openproject/pgdata-next /var/lib/openproject/pgdata+#### On SLES:+```bash+TODO: sudo zypper addrepo https://download.postgresql.org/pub/repos/zypp/17/suse/sles-15.5-x86_64/ openSUSE-PostgreSQL-17+TODO: sudo zypper install --repo openSUSE-PostgreSQL-17 postgresql17 postgresql17-server postgresql17-libs postgresql17-contrib+#TODO:? sudo su - postgres -c '/usr/lib/postgresql17/bin/initdb -D /var/lib/pgsql/17/data'+sudo systemctl enable postgresql+sudo systemctl start postgresql```-Finally, you can restart OpenProject with the same command that you used before. For instance:-docker run -d -p 8080:80 --name openproject -e SECRET_KEY_BASE=secret \- -v /var/lib/openproject/pgdata:/var/openproject/pgdata \- -v /var/lib/openproject/assets:/var/openproject/assets \- [...]- openproject/openproject:16+### 5. Copy Configuration Files-If your new installation looks fine, you can then choose to remove `/var/lib/openproject/pgdata-prev`:+#### On Debian/Ubuntu:+```bash+sudo su - postgres -c "cp /etc/postgresql/13/main/pg_hba.conf /etc/postgresql/17/main/pg_hba.conf"+sudo su - postgres -c "cp /etc/postgresql/13/main/conf.d/custom.conf /etc/postgresql/17/main/conf.d/custom.conf"+sudo pg_ctlcluster 17 main restart+```-```shell-sudo rm -rf /var/lib/openproject/pgdata-prev+#### On CentOS/RHEL:+```bash+sudo su - postgres -c "cp /var/lib/pgsql/13/data/pg_hba.conf /var/lib/pgsql/17/data/pg_hba.conf"+sudo su - postgres -c "cp -r /var/lib/pgsql/13/data/conf.d /var/lib/pgsql/17/data/"+sudo su - postgres -c "cp -r /var/lib/pgsql/13/data/postgresql.conf /var/lib/pgsql/17/data/postgresql.conf"+sudo service postgresql-17 restart```-If you encounter an issue, you can switch back to the previous PostgreSQL folder by reverting the folder switch:+#### On SLES:+```bash+sudo su - postgres -c "cp /var/lib/pgsql/13/data/pg_hba.conf /var/lib/pgsql/data/pg_hba.conf"+sudo su - postgres -c "cp -r /var/lib/pgsql/13/data/conf.d /var/lib/pgsql/data/"+sudo su - postgres -c "cp -r /var/lib/pgsql/13/data/postgresql.conf /var/lib/pgsql/data/postgresql.conf"+sudo systemctl restart postgresql+```-```shell-sudo mv /var/lib/openproject/pgdata /var/lib/openproject/pgdata-next-sudo mv /var/lib/openproject/pgdata-prev /var/lib/openproject/pgdata+### 6. Remove Old PostgreSQL++#### On Debian/Ubuntu:+```bash+sudo apt remove --purge postgresql-13+```++#### On CentOS/RHEL:+```bash+sudo dnf remove postgresql13-server+```++#### On SLES:+```bash+sudo zypper remove postgresql13-server+```+++### 7. Recreate the OpenProject User and Database++```bash+sudo su - postgres -c "psql -p 45432 -c \"create user openproject with password '$(sudo openproject config:get DATABASE_URL | sed -n 's|.*://[^:]*:\([^@]*\)@.*|\1|p')'\""+sudo su - postgres -c "psql -p 45432 -c 'create database openproject owner openproject'"```-And then restart OpenProject.+### 8. Restore the Database++```bash+psql $(sudo openproject config:get DATABASE_URL) < openproject.sql+```+++### 9. Restart OpenProject++```bash+sudo openproject restart+```++### Confirmation++Visit your OpenProject instance in the browser to confirm everything works as expected.+## Helm Chart installation@@ -252,7 +316,7 @@ kubectl exec -it <postgresql-pod-name> -- bashCreate a PostgreSQL dump of the database and save it to the persistent directory:```shell-pg_dumpall -U postgres > /bitnami/postgresql/backup.sql+PGPASSWORD=$(cat "${POSTGRES_POSTGRES_PASSWORD_FILE:-/dev/null}" && echo "$POSTGRES_POSTGRES_PASSWORD") pg_dumpall -U postgres > /bitnami/postgresql/backup.sql```3. Prepare the Upgrade by renaming the current PostgreSQL data directory to ensure it is preserved in case of issues after the upgrade:@@ -274,7 +338,7 @@ kubectl exec -it <new-postgresql-pod-name> -- bashRestore the backup by running the following command:```shell-psql -U postgres -h localhost -f /bitnami/postgresql/backup.sql+PGPASSWORD=$(cat "${POSTGRES_POSTGRES_PASSWORD_FILE:-/dev/null}" && echo "$POSTGRES_POSTGRES_PASSWORD") psql -U postgres -h localhost -f /bitnami/postgresql/backup.sql```6. Restore Frontend Availability by starting the frontend or scaling it up again.