Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(scripts): make restore work on the compose volume and Helm PVC

Yudhi Armyndharisa307ad940a8c
diff --git a/docs/11-operational-runbooks.md b/docs/11-operational-runbooks.md
index 089117a2e..6d737e66f 100644
--- a/docs/11-operational-runbooks.md
+++ b/docs/11-operational-runbooks.md
@@ -433,7 +433,7 @@ curl -H "X-API-Key: $API_KEY" \
# Started with docker-compose.dev.yml (the README Quick Start)? Add `-f docker-compose.dev.yml`
# to every docker compose command in this runbook, the Rollback block included, and write `openwa`
-# wherever a command names the `openwa-api` service (steps 6 and 7, rollback step 2).
+# wherever a command names the `openwa-api` service (steps 6 and 7, rollback steps 2 and 3).
# 4. Stop services
docker compose down
@@ -503,15 +503,20 @@ curl -H "X-API-Key: $API_KEY" \
# 1. Stop services
docker compose down
-# 2. Check out the previous release and rebuild the image
+# 2. Restore from the pre-upgrade backup (both DBs + sessions). The archive upgrade step 2 produced is
+# "$BACKUP_DIR/openwa-backup-<timestamp>.tar.gz". The databases in place still hold the failed
+# upgrade's data, so the restore refuses to touch them without --force. It runs in the image
+# because the data lives in the openwa-data volume (see Runbook: Restore from Backup), and before
+# the checkout below because an image older than 0.23.7 cannot move its safety snapshot off the
+# read-only container root
+docker compose run --rm --no-deps --entrypoint /app/scripts/restore.sh \
+ -v "$BACKUP_DIR:/backups" -e OPENWA_RESTORE_SNAPSHOT_DIR=/backups -e TMPDIR=/backups \
+ openwa-api /backups/openwa-backup-<timestamp>.tar.gz --force
+
+# 3. Check out the previous release and rebuild the image
git checkout v<old-version>
docker compose build openwa-api
-# 3. Restore from the pre-upgrade backup (both DBs + sessions) — the archive step 2 produced is
-# "$BACKUP_DIR/openwa-backup-<timestamp>.tar.gz". The databases in place still hold the failed
-# upgrade's data, so the restore refuses to touch them without --force
-./scripts/restore.sh "$BACKUP_DIR/openwa-backup-<timestamp>.tar.gz" --force
-
# 4. Start with old version
docker compose up -d
@@ -668,6 +673,67 @@ docker compose up -d
curl -s -X POST -H "X-API-Key: <an-existing-key>" http://localhost:2785/api/auth/validate
```
+> Step 2 as written restores into `./data` in the checkout. The app reads that directory only on a
+> bare-metal install or under `docker-compose.dev.yml`, which bind-mounts it. The production compose
+> file keeps the data in the **named volume** `openwa-data` and the Helm chart in a PVC, so a host run
+> there fills a directory the container never reads and still reports success. Run the script from the
+> image against the volume instead, in place of step 2. Both mount the container root read-only, so
+> `OPENWA_RESTORE_SNAPSHOT_DIR` (0.23.7 or later) puts the pre-restore snapshot of the data dir on a
+> writable, persistent path, and `TMPDIR` keeps the extracted archive there too; allow free space for
+> about twice the data plus the archive. `--force` is included because the volume of an existing
+> install still holds its databases:
+>
+> ```bash
+> # Compose: the entrypoint override runs the script as root, which can read the archive and write
+> # the volume; the next start hands the restored files back to the app user.
+> docker compose run --rm --no-deps --entrypoint /app/scripts/restore.sh \
+> -v "$PWD/backups:/backups" -e OPENWA_RESTORE_SNAPSHOT_DIR=/backups -e TMPDIR=/backups \
+> openwa-api /backups/openwa-backup-<timestamp>.tar.gz --force
+>
+> # Helm, for a release named openwa (`kubectl get statefulset,configmap,pvc` shows the names of
+> # another): stop the pod, then run the script in a helper pod on the same PVC, with the release's image.
+> kubectl scale statefulset/openwa --replicas=0
+> kubectl wait --for=delete pod/openwa-0 --timeout=120s
+> kubectl apply -f - <<'EOF'
+> apiVersion: v1
+> kind: Pod
+> metadata:
+> name: openwa-restore
+> spec:
+> restartPolicy: Never
+> containers:
+> - name: restore
+> image: ghcr.io/rmyndharis/openwa:<version>
+> command: ['sleep', 'infinity']
+> envFrom:
+> - configMapRef:
+> name: openwa
+> volumeMounts:
+> - { name: data, mountPath: /app/data }
+> - { name: work, mountPath: /restore }
+> volumes:
+> - name: data
+> persistentVolumeClaim:
+> claimName: data-openwa-0
+> - name: work
+> emptyDir: {}
+> EOF
+> kubectl wait --for=condition=Ready pod/openwa-restore --timeout=300s
+> kubectl cp ./backups/openwa-backup-<timestamp>.tar.gz openwa-restore:/restore/backup.tar.gz
+> # The image sets HOME=/app/data, and the script refuses a data dir that is the home directory, so
+> # HOME is moved off it here (the compose files already set HOME=/tmp).
+> kubectl exec openwa-restore -- env HOME=/tmp OPENWA_RESTORE_SNAPSHOT_DIR=/restore TMPDIR=/restore \
+> ./scripts/restore.sh /restore/backup.tar.gz --force
+> # The emptyDir goes away with the pod: copy off the snapshot the script named first.
+> kubectl cp openwa-restore:/restore/data.pre-restore-<ts> ./backups/data.pre-restore-<ts>
+> kubectl delete pod openwa-restore
+> kubectl scale statefulset/openwa --replicas=1
+> ```
+>
+> The PostgreSQL import in step 3 then reads the dump from the archive, because `./data` on the host
+> is not the volume: `tar -xzOf ./backups/openwa-backup-<timestamp>.tar.gz ./database.sql | psql "$DATABASE_URL"`.
+> On Helm, run the step 4 check through `kubectl port-forward` to the release's Service.
+
> **PostgreSQL restores are read as UTC.** From 0.23.6 the data connection binds, parses and defaults
> every timestamp in UTC, and refuses to boot when its session is not on UTC
> ([05 - Database Design](./05-database-design.md#timestamps-on-postgresql-are-utc)). A `database.sql`
diff --git a/scripts/restore.sh b/scripts/restore.sh
index c5b336934..19baf04b3 100755
--- a/scripts/restore.sh
+++ b/scripts/restore.sh
@@ -24,6 +24,9 @@
# OPENWA_DATA_DIR data directory to restore non-DB state into (default: ./data)
# SESSION_DATA_PATH, BAILEYS_AUTH_DIR, STORAGE_LOCAL_PATH, PLUGINS_DIR
# override the corresponding state directories
+# OPENWA_RESTORE_SNAPSHOT_DIR
+# where the data-dir safety snapshot goes (default: next to the data dir);
+# needed when that parent is read-only, as in the shipped container
#
# Stop the OpenWA app before restoring. A snapshot of the current data dir is taken
# first so a bad restore can be undone.
@@ -273,10 +276,14 @@ if [ "$FORCE" -ne 1 ]; then
fi
fi
-# Safety snapshot of whatever is there now.
+# Safety snapshot of whatever is there now, next to the data dir unless OPENWA_RESTORE_SNAPSHOT_DIR
+# names another directory. The shipped compose file and Helm chart mount the data dir as a volume
+# under a read-only root, where that sibling cannot be written.
if [ -d "$DATA_DIR" ] && [ -n "$(ls -A "$DATA_DIR" 2>/dev/null || true)" ]; then
- SAFETY="${DATA_DIR%/}.pre-restore-$RESTORE_TIMESTAMP"
+ SAFETY_DIR="${OPENWA_RESTORE_SNAPSHOT_DIR:-$(dirname "$DATA_DIR")}"
+ SAFETY="${SAFETY_DIR%/}/$(basename "$DATA_DIR").pre-restore-$RESTORE_TIMESTAMP"
log "Snapshotting current data dir -> $SAFETY"
+ mkdir -p "$SAFETY_DIR"
cp -pR "$DATA_DIR" "$SAFETY"
fi
diff --git a/scripts/smoke-test-backup-restore.sh b/scripts/smoke-test-backup-restore.sh
index f30ba762e..0242d9a12 100755
--- a/scripts/smoke-test-backup-restore.sh
+++ b/scripts/smoke-test-backup-restore.sh
@@ -15,6 +15,8 @@
# (i) the data-store half of that guard refuses on its own
# (j) a probe that fails or prints no usable count leaves the target counted as live
# (k) an operator's sqlite3 rc file changes neither answer of the guard (skipped without sqlite3)
+# (l) an unwritable BACKUP_DIR fails before anything is staged (skipped as root)
+# (m) OPENWA_RESTORE_SNAPSHOT_DIR takes the data-dir snapshot off a read-only parent (skipped as root)
#
# Usage: ./scripts/smoke-test-backup-restore.sh
# Requires: bash, tar, node (restore.sh path resolution). sqlite3 is optional (see (c) and (k)).
@@ -24,7 +26,7 @@ set -euo pipefail
# would aim a case at a real install, and restore replaces the state directories wholesale, so every
# case starts from none of them and sets exactly the paths it uses.
unset OPENWA_DATA_DIR BACKUP_DIR DATABASE_TYPE MAIN_DATABASE_NAME DATABASE_NAME SESSION_DATA_PATH \
- BAILEYS_AUTH_DIR STORAGE_LOCAL_PATH PLUGINS_DIR PLUGIN_STATE_DIR
+ BAILEYS_AUTH_DIR STORAGE_LOCAL_PATH PLUGINS_DIR PLUGIN_STATE_DIR OPENWA_RESTORE_SNAPSHOT_DIR
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
BACKUP="$REPO_ROOT/scripts/backup.sh"
@@ -500,5 +502,34 @@ else
echo "SKIP: (l) running as root, which ignores the permission bits this case relies on"
fi
+echo ""
+echo "==> (m) OPENWA_RESTORE_SNAPSHOT_DIR takes the data-dir snapshot off a read-only parent"
+# The shipped compose file and Helm chart mount the data dir as a volume under a read-only root, so
+# the snapshot's default place next to it cannot be written and the restore stopped there.
+if [ "$(id -u)" -ne 0 ]; then
+ M="$WORK/m"
+ mkdir -p "$M/root/data" "$M/snapshots"
+ printf 'mike-before\n' >"$M/root/data/.api-key"
+ chmod a-w "$M/root"
+ set +e
+ OUT_M="$(cd "$M" && MAIN_DATABASE_NAME="$M/root/data/main.sqlite" DATABASE_NAME="$M/root/data/openwa.sqlite" \
+ OPENWA_DATA_DIR="$M/root/data" OPENWA_RESTORE_SNAPSHOT_DIR="$M/snapshots" "$RESTORE" "$ARCHIVE_H" 2>&1)"
+ RC_M=$?
+ set -e
+ chmod u+w "$M/root"
+ if [ "$RC_M" -ne 0 ]; then
+ fail "(m) restore failed with a writable OPENWA_RESTORE_SNAPSHOT_DIR: $OUT_M"
+ fi
+ if [ "$(cat "$M"/snapshots/data.pre-restore-*/.api-key 2>/dev/null || true)" != "mike-before" ]; then
+ fail "(m) the data-dir snapshot is not under OPENWA_RESTORE_SNAPSHOT_DIR"
+ fi
+ if [ "$(db_fingerprint "$M/root/data/main.sqlite")" != "hotel-archive-main" ]; then
+ fail "(m) the restore did not put the archived main DB in place"
+ fi
+ pass "(m) data-dir snapshot written under OPENWA_RESTORE_SNAPSHOT_DIR, restore completes"
+else
+ echo "SKIP: (m) running as root, which ignores the permission bits this case relies on"
+fi
+
echo ""
echo "All smoke tests passed!"