🐸 Rybbit - open-source and privacy-friendly alternative to Google Analytics that is 10x more intuitive.
Configure standalone ClickHouse native backups
Bill Yang1babe5fee6d3
diff --git a/clickhouse/docker-compose.clickhouse.yml b/clickhouse/docker-compose.clickhouse.ymlindex 1fe5b5572..69f5922c4 100644--- a/clickhouse/docker-compose.clickhouse.yml+++ b/clickhouse/docker-compose.clickhouse.yml@@ -7,6 +7,8 @@ services:image: clickhouse/clickhouse-server:26.3.17.4volumes:- clickhouse-data:/var/lib/clickhouse+ - /var/backups/rybbit/clickhouse:/var/lib/clickhouse/backups+ - /etc/rybbit-backup/clickhouse-backup-disk.xml:/etc/clickhouse-server/config.d/backup-disk.xml:roenvironment:- CLICKHOUSE_DB=${CLICKHOUSE_DB:-analytics}- CLICKHOUSE_USER=${CLICKHOUSE_USER:-default}diff --git a/ops/backups/README.md b/ops/backups/README.mdindex 11876556f..37236bfad 100644--- a/ops/backups/README.md+++ b/ops/backups/README.md@@ -37,10 +37,10 @@ sudo rclone configsudo chmod 600 /root/.config/rclone/rclone.conf```-The examples use a Backblaze remote named `b2`. Data is protected in transit by-TLS and at rest by B2 server-side encryption. The direct FrogStats ClickHouse-backup is written through ClickHouse's S3-compatible client. Its B2 credentials-live in a ClickHouse named collection rather than the backup SQL.+The examples use a Backblaze remote named `backblaze`. Data is protected in+transit by TLS and at rest by B2 server-side encryption. The direct FrogStats+ClickHouse backup is written through ClickHouse's S3-compatible client. Its B2+credentials live in a ClickHouse named collection rather than the backup SQL.## Install on each host@@ -135,25 +135,30 @@ The FrogStats environment example selects this user for backups.## Standalone ClickHouse staging diskThe standalone host creates one native backup and uploads the same completed-snapshot to both Hetzner and B2. Add these mounts to its ClickHouse service:+snapshot to both Hetzner and B2. Install the disk configuration outside the+repository and prepare the host staging directory:++```bash+sudo install -d -m 0750 /etc/rybbit-backup+sudo install -m 0640 -o root -g 101 \+ ops/backups/clickhouse/backup-disk.xml \+ /etc/rybbit-backup/clickhouse-backup-disk.xml+sudo install -d -m 0750 -o 101 -g 101 /var/backups/rybbit/clickhouse+```++Then add these mounts to its ClickHouse service:```yamlservices:clickhouse:volumes:- /var/backups/rybbit/clickhouse:/var/lib/clickhouse/backups- - ./ops/backups/clickhouse/backup-disk.xml:/etc/clickhouse-server/config.d/backup-disk.xml:ro-```--Prepare the host directory before recreating the container. UID/GID 101 is the-default ClickHouse user in the official image; confirm it with `docker exec`-if the image changes:--```bash-sudo install -d -m 0750 -o 101 -g 101 /var/backups/rybbit/clickhouse+ - /etc/rybbit-backup/clickhouse-backup-disk.xml:/etc/clickhouse-server/config.d/backup-disk.xml:ro```-Restart ClickHouse after adding the disk configuration, then confirm it exists:+UID/GID 101 is the default ClickHouse user in the official image; confirm it if+the image changes. Restart ClickHouse after adding the disk configuration, then+confirm it exists:```bashdocker exec clickhouse clickhouse-client \diff --git a/ops/backups/config/clickhouse.env.example b/ops/backups/config/clickhouse.env.exampleindex 578c46a0b..8cc80779c 100644--- a/ops/backups/config/clickhouse.env.example+++ b/ops/backups/config/clickhouse.env.example@@ -12,7 +12,7 @@ CLICKHOUSE_KEEP_LOCAL_BACKUP=false# The same completed native snapshot is copied to both providers. Configure# these names in root's /root/.config/rclone/rclone.conf.-CLICKHOUSE_RCLONE_REMOTES=hetzner:clickhouse-backups,b2:rybbit-db-backups/clickhouse/analytics+CLICKHOUSE_RCLONE_REMOTES=hetzner:clickhouse-backups,backblaze:rybbit-backups/clickhouse/analytics# Hetzner does not have B2 lifecycle rules, so prune it from the host. Do not# include the Object-Locked B2 remote here; let B2 lifecycle rules expire it.diff --git a/ops/backups/config/frogstats.env.example b/ops/backups/config/frogstats.env.exampleindex 634b00c09..2b11b7536 100644--- a/ops/backups/config/frogstats.env.example+++ b/ops/backups/config/frogstats.env.example@@ -6,7 +6,7 @@ BACKUP_HOST=frogstatsPOSTGRES_CONTAINER=postgresPOSTGRES_DATABASE=analyticsPOSTGRES_STAGING_ROOT=/var/backups/rybbit/postgres-POSTGRES_RCLONE_REMOTES=b2:rybbit-db-backups/frogstats/postgres+POSTGRES_RCLONE_REMOTES=backblaze:rybbit-backups/frogstats/postgres# ClickHouse: direct native backup avoids staging ~58 GiB on this host.CLICKHOUSE_CONTAINER=clickhousediff --git a/ops/backups/config/rybbit.env.example b/ops/backups/config/rybbit.env.exampleindex 8420664c2..817598d8e 100644--- a/ops/backups/config/rybbit.env.example+++ b/ops/backups/config/rybbit.env.example@@ -5,7 +5,7 @@ BACKUP_HOST=rybbitPOSTGRES_CONTAINER=postgresPOSTGRES_DATABASE=analyticsPOSTGRES_STAGING_ROOT=/var/backups/rybbit/postgres-POSTGRES_RCLONE_REMOTES=b2:rybbit-db-backups/rybbit/postgres+POSTGRES_RCLONE_REMOTES=backblaze:rybbit-backups/rybbit/postgres# No ClickHouse backup is configured here. The backend uses the standalone# ClickHouse host rather than this host's effectively empty local volume.