PostgreSQL backup tool with Point-In-Time-Recovery and restore verification
TypeScript backupbackupsdatabasedatabase-backupdevopsdockergolangkubernetesmariadbmongodbmysqlpgpostgrespostgresqls3self-hostedsystem-administrationtoolsweb-ui
FEATURE (docker): Test filesystem compatibility
Rostislav Dugin564fd07af2dc
diff --git a/.github/workflows/ci-release.yml b/.github/workflows/ci-release.ymlindex bfc435f0..9b6c8235 100644--- a/.github/workflows/ci-release.yml+++ b/.github/workflows/ci-release.yml@@ -348,6 +348,37 @@ jobs:# the 15m the Makefile's `test` target allows on a developer machine.go test -p=8 -count=1 -failfast -timeout 40m ./internal/...+ test-filesystems:+ runs-on: ubuntu-latest+ timeout-minutes: 45+ steps:+ - name: Check out code+ uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1++ - name: Set up Docker Buildx+ uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0++ - name: Install network filesystem clients+ run: sudo apt-get update && sudo apt-get install -y cifs-utils nfs-common++ - name: Build amd64 storage candidate and load locally+ uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5.4.0+ with:+ context: .+ push: false+ load: true+ platforms: linux/amd64+ cache-to: type=gha,mode=max,scope=storage-build-amd64+ cache-from: type=gha,scope=storage-build-amd64+ build-args: |+ APP_VERSION=storage-${{ github.sha }}+ tags: databasus-storage-test:${{ github.sha }}++ - name: Run filesystem suite+ env:+ DATABASUS_IMAGE: databasus-storage-test:${{ github.sha }}+ run: make test-filesystems+build-dev-image:runs-on: ubuntu-latesttimeout-minutes: 45@@ -459,7 +490,7 @@ jobs:push-dev-image:runs-on: ${{ matrix.runner }}timeout-minutes: 60- needs: [build-dev-image]+ needs: [build-dev-image, test-filesystems]if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/develop' }}permissions:contents: read@@ -688,7 +719,7 @@ jobs:determine-version:runs-on: ubuntu-latesttimeout-minutes: 10- needs: [test-backend, test-frontend, test-verification-agent, e2e-verification-agent]+ needs: [test-backend, test-frontend, test-verification-agent, e2e-verification-agent, test-filesystems]if: ${{ github.ref == 'refs/heads/main' && !contains(github.event.head_commit.message, '[skip-release]') }}outputs:should_release: ${{ steps.version_bump.outputs.should_release }}diff --git a/Dockerfile b/Dockerfileindex 1e9ac8ef..8116a240 100644--- a/Dockerfile+++ b/Dockerfile@@ -156,28 +156,25 @@ RUN --mount=type=bind,source=assets/tools,target=/ctx/tools,readonly \/app/assets/tools/*/mariadb/*/bin/* \/app/assets/tools/*/mongodb/bin/*-# postgres owns PGDATA inside the user's mounted /databasus-data volume, so its UID has-# to stay stable across releases instead of whatever the postgresql-17 package assigns.-# Hosts that need a different owner set PUID/PGID, which start.sh applies.RUN set -eux; \groupmod -g 999 postgres; \usermod -u 999 postgres; \- # Renumbering rewrites the account, not the directories the package already created, so- # they would keep IDs that resolve to nothing until some later install reuses them.chown -R postgres:postgres \/var/lib/postgresql /etc/postgresql /var/log/postgresql /run/postgresql; \mkdir -p /databasus-data/pgdata; \chown -R postgres:postgres /databasus-data/pgdata-# databasus owns everything in /databasus-data except PGDATA, so like postgres above it needs-# IDs that stay stable across releases for hosts that bind-mount the volume. postgres joins-# the group as a supplementary member rather than sharing a primary group, because it has to-# traverse the volume root to reach PGDATA and PUID/PGID renumber its primary group at startup.+# PostgreSQL traverses the shared data root through the Databasus group.RUN set -eux; \groupadd -g 65532 databasus; \useradd -r -s /usr/sbin/nologin -u 65532 -g databasus databasus; \usermod -aG databasus postgres+ENV DATABASUS_PUID=65532 \+ DATABASUS_PGID=65532 \+ POSTGRES_PUID=999 \+ POSTGRES_PGID=999+WORKDIR /app# Copy Goose from build stage@@ -210,15 +207,15 @@ COPY .env.example /.envCOPY <<EOF /app/start.sh#!/bin/bashset -e+export TMPDIR=/tmp-# Check for legacy postgresus-data volume mountif [ -d "/postgresus-data" ] && [ "\$(ls -A /postgresus-data 2>/dev/null)" ]; thenecho ""echo "=========================================="echo "ERROR: Legacy volume detected!"echo "=========================================="echo ""- echo "You are using the \`postgresus-data\` folder. It seems you changed the image name from Postgresus to Databasus without changing the volume."+ echo "You are using the /postgresus-data folder. It seems you changed the image name from Postgresus to Databasus without changing the volume."echo ""echo "Please either:"echo " 1. Switch back to image rostislavdugin/postgresus:latest (supported until ~Dec 2026)"@@ -228,28 +225,11 @@ if [ -d "/postgresus-data" ] && [ "\$(ls -A /postgresus-data 2>/dev/null)" ]; thexit 1fi-# ========= Adjust postgres user UID/GID =========-PUID=\${PUID:-999}-PGID=\${PGID:-999}--CURRENT_UID=\$(id -u postgres)-CURRENT_GID=\$(id -g postgres)-PUID_OWNER=\$(getent passwd "\$PUID" | cut -d: -f1 || true)--if [ -n "\$PUID_OWNER" ] && [ "\$PUID_OWNER" != "postgres" ]; then- echo "ERROR: PUID \$PUID is already assigned to user \$PUID_OWNER. Choose an unused UID for postgres."- exit 1-fi--if [ "\$CURRENT_GID" != "\$PGID" ]; then- echo "Adjusting postgres group GID from \$CURRENT_GID to \$PGID..."- groupmod -o -g "\$PGID" postgres-fi--if [ "\$CURRENT_UID" != "\$PUID" ]; then- echo "Adjusting postgres user UID from \$CURRENT_UID to \$PUID..."- usermod -o -u "\$PUID" postgres-fi+# ========= Configure service identities =========+[ "\$(id -g postgres)" = "\$POSTGRES_PGID" ] || groupmod -g "\$POSTGRES_PGID" postgres+[ "\$(id -g databasus)" = "\$DATABASUS_PGID" ] || groupmod -g "\$DATABASUS_PGID" databasus+[ "\$(id -u postgres)" = "\$POSTGRES_PUID" ] || usermod -u "\$POSTGRES_PUID" postgres+[ "\$(id -u databasus)" = "\$DATABASUS_PUID" ] || usermod -u "\$DATABASUS_PUID" databasus# PostgreSQL 17 binary pathsPG_BIN="/usr/lib/postgresql/17/bin"@@ -287,22 +267,13 @@ fi# Ensure proper ownership of data directoryecho "Setting up data directory permissions..."-mkdir -p /databasus-data/pgdata-mkdir -p /databasus-data/pgsocket-mkdir -p /databasus-data/temp-mkdir -p /databasus-data/backups-chown databasus:databasus /databasus-data-# postgres reaches pgdata through this directory as a member of the databasus group, and a-# bind-mounted host directory can arrive without a group traversal bit.-chmod g+x /databasus-data-chown -R postgres:postgres /databasus-data/pgdata-chown postgres:postgres /databasus-data/pgsocket-chown -R databasus:databasus /databasus-data/temp /databasus-data/backups-# Upgrade path: secret.key and instance.json may be owned by root or postgres-# from older images. Re-own them so the non-root main process can read/write.+mkdir -p /databasus-data/{pgdata,pgsocket,temp,backups}+chown databasus:databasus /databasus-data 2>/dev/null || true+chmod g+x /databasus-data 2>/dev/null || true+chown postgres:postgres /databasus-data/pgdata /databasus-data/pgsocket 2>/dev/null || true+chown databasus:databasus /databasus-data/temp /databasus-data/backups 2>/dev/null || truechown databasus:databasus /databasus-data/secret.key /databasus-data/instance.json 2>/dev/null || true-chmod 700 /databasus-data/temp-chmod 700 /databasus-data/pgsocket+chmod 700 /databasus-data/temp /databasus-data/pgsocket 2>/dev/null || true# ========= Start Valkey (internal cache) =========echo "Configuring Valkey cache..."@@ -442,9 +413,6 @@ if [ -z "\${DATABASE_DSN+x}" ]; thenexport DATABASE_DSN="host=localhost user=postgres password=\$INTERNAL_POSTGRES_PASSWORD dbname=databasus port=5437 sslmode=disable"fi-# ========= Refuse to start if legacy WAL backup data exists =========-# The agent-based WAL_V1 backup type was removed. Existing installs with-# WAL-mode databases must downgrade, manually remove them and then upgrade.echo "Checking for legacy WAL backup configuration..."WAL_CHECK_COL=\$(gosu databasus env PGPASSWORD="\$INTERNAL_POSTGRES_PASSWORD" \\\$PG_BIN/psql -h localhost -p 5437 -U postgres -d databasus -tA \\diff --git a/Makefile b/Makefilenew file mode 100644index 00000000..b646b0a2--- /dev/null+++ b/Makefile@@ -0,0 +1,14 @@+.PHONY: build-docker-storage-test-image test-filesystems++DATABASUS_STORAGE_TEST_IMAGE ?= databasus-storage-test:local++build-docker-storage-test-image:+ docker build --tag $(DATABASUS_STORAGE_TEST_IMAGE) .++test-filesystems:+ @if [ -z "$(DATABASUS_IMAGE)" ]; then \+ $(MAKE) build-docker-storage-test-image; \+ fi+ @DATABASUS_IMAGE="$${DATABASUS_IMAGE:-$(DATABASUS_STORAGE_TEST_IMAGE)}" \+ CASE="$(CASE)" \+ bash e2e/docker-storage/run.shdiff --git a/deploy/helm/README.md b/deploy/helm/README.mdindex c2e84655..fa085791 100644--- a/deploy/helm/README.md+++ b/deploy/helm/README.md@@ -414,7 +414,7 @@ extraVolumeMounts:```> **Not supported by design:** `runAsNonRoot: true` and dropping **all**-> capabilities. The entrypoint must start as root to handle PUID/PGID remapping,+> capabilities. The entrypoint must start as root to remap service accounts,> volume `chown` and PostgreSQL `initdb`, then drops to a non-root user with> `gosu` (which requires `SETUID`/`SETGID`). See the note in `values.yaml`> next to `podSecurityContext`.diff --git a/e2e/docker-storage/run.sh b/e2e/docker-storage/run.shnew file mode 100755index 00000000..b43c10f8--- /dev/null+++ b/e2e/docker-storage/run.sh@@ -0,0 +1,458 @@+#!/usr/bin/env bash+set -Eeuo pipefail++readonly harness_label_key="com.databasus.storage-test"+run_id="${RANDOM}-$$-$(date +%s)"+readonly run_id+readonly harness_label="${harness_label_key}=${run_id}"+readonly selected_case="${CASE:-all}"+readonly databasus_image="${DATABASUS_IMAGE:?DATABASUS_IMAGE must be set}"+readonly samba_fixture_image="dockurr/samba@sha256:dbd8ebd6392cc1e8e98312d96d163f0039b644b84aee741d89efdd0abd3d32bf"+readonly nfs_fixture_image="itsthenetwork/nfs-server-alpine@sha256:7fa99ae65c23c5af87dd4300e543a86b119ed15ba61422444207efc7abd0ba20"++case_root=""++cleanup() {+ local container_id+ local container_name+ local volume_name+ local volume_type+ local -a client_container_ids=()+ local -a server_container_ids=()+ local -a volume_names=()+ local -a network_ids=()++ while read -r container_id container_name; do+ [[ -n "${container_id}" ]] || continue+ if [[ "${container_name}" == *-server-* ]]; then+ server_container_ids+=("${container_id}")+ else+ client_container_ids+=("${container_id}")+ fi+ done < <(docker ps -a --filter "label=${harness_label}" --format '{{.ID}} {{.Names}}' 2>/dev/null)++ if (("${#client_container_ids[@]}")); then+ docker rm -f "${client_container_ids[@]}" >/dev/null 2>&1 || true+ fi++ mapfile -t volume_names < <(docker volume ls -q --filter "label=${harness_label}" 2>/dev/null)+ for volume_name in "${volume_names[@]}"; do+ volume_type="$(docker volume inspect --format '{{index .Options "type"}}' "${volume_name}" 2>/dev/null || true)"+ if [[ "${volume_type}" == "cifs" || "${volume_type}" == "nfs" ]]; then+ docker volume rm -f "${volume_name}" >/dev/null 2>&1 || true+ fi+ done++ if (("${#server_container_ids[@]}")); then+ docker rm -f "${server_container_ids[@]}" >/dev/null 2>&1 || true+ fi++ mapfile -t network_ids < <(docker network ls -q --filter "label=${harness_label}" 2>/dev/null)+ if (("${#network_ids[@]}")); then+ docker network rm "${network_ids[@]}" >/dev/null 2>&1 || true+ fi++ mapfile -t volume_names < <(docker volume ls -q --filter "label=${harness_label}" 2>/dev/null)+ if (("${#volume_names[@]}")); then+ docker volume rm -f "${volume_names[@]}" >/dev/null 2>&1 || true+ fi++ if [[ "${case_root}" == /tmp/databasus-storage-test.* ]]; then+ docker run --rm --label "${harness_label}" --entrypoint /bin/chmod \+ --volume "${case_root}:/case-root" "${databasus_image}" \+ -R 0777 /case-root >/dev/null 2>&1 || true+ rm -rf -- "${case_root}" || true+ fi+}++trap cleanup EXIT+case_root="$(mktemp -d /tmp/databasus-storage-test.XXXXXXXX)"++wait_for_health() {+ local container_name="$1"+ local health_status++ for ((readiness_attempt = 1; readiness_attempt <= 90; readiness_attempt++)); do+ health_status="$(docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "${container_name}")"+ if [[ "${health_status}" == "healthy" ]]; then+ return+ fi+ if [[ "$(docker inspect --format '{{.State.Running}}' "${container_name}")" != "true" ]]; then+ docker logs "${container_name}" >&2+ return 1+ fi+ sleep 1+ done++ docker logs "${container_name}" >&2+ echo "container did not become healthy" >&2+ return 1+}++wait_for_log() {+ local container_name="$1"+ local required_log_line="$2"++ for ((log_attempt = 1; log_attempt <= 60; log_attempt++)); do+ if docker logs "${container_name}" 2>&1 | grep -Fq "${required_log_line}"; then+ return+ fi+ sleep 1+ done++ docker logs "${container_name}" >&2+ echo "container did not report readiness: ${required_log_line}" >&2+ return 1+}++wait_for_exit() {+ local container_name="$1"++ for ((exit_attempt = 1; exit_attempt <= 100; exit_attempt++)); do+ if [[ "$(docker inspect --format '{{.State.Running}}' "${container_name}")" != "true" ]]; then+ [[ "$(docker inspect --format '{{.State.ExitCode}}' "${container_name}")" != "0" ]]+ return+ fi+ sleep 0.1+ done++ echo "container did not stop" >&2+ return 1+}++start_databasus() {+ local container_name="$1"+ shift++ docker run --detach \+ --name "${container_name}" \+ --label "${harness_label}" \+ "$@" \+ "${databasus_image}" >/dev/null+ wait_for_health "${container_name}"+}++probe_path() {+ local container_name="$1"+ local account_name="$2"+ local mounted_path="$3"++ docker exec --user "${account_name}" "${container_name}" /bin/sh -eu -c '+ probe_path="$1/.databasus-filesystem-probe-$$"+ printf filesystem-probe > "$probe_path"+ [ "$(cat "$probe_path")" = filesystem-probe ]+ rm "$probe_path"+ [ ! -e "$probe_path" ]+ ' sh "${mounted_path}"+}++assert_service_boundaries() {+ local container_name="$1"++ docker exec --user databasus "${container_name}" test ! -r /databasus-data/pgdata/PG_VERSION+ docker exec --user databasus "${container_name}" test ! -x /databasus-data/pgsocket+}++probe_service_paths() {+ local container_name="$1"++ probe_path "${container_name}" databasus /databasus-data/temp+ probe_path "${container_name}" databasus /databasus-data/backups+ probe_path "${container_name}" postgres /databasus-data/pgdata+ assert_service_boundaries "${container_name}"+}++run_default_ids() {+ local container_name="databasus-storage-default-ids-${run_id}"+ local volume_name="databasus-storage-default-ids-${run_id}"+ local rejected_remap_command="${case_root}/reject-identity-remap"+ local image_environment+ local container_environment+ local identity_setting++ image_environment="$(docker image inspect --format '{{range .Config.Env}}{{println .}}{{end}}' "${databasus_image}")"+ for identity_setting in DATABASUS_PUID=65532 DATABASUS_PGID=65532 POSTGRES_PUID=999 POSTGRES_PGID=999; do+ grep -Fxq "${identity_setting}" <<<"${image_environment}"+ done++ printf '#!/bin/sh\nexit 1\n' > "${rejected_remap_command}"+ chmod 0755 "${rejected_remap_command}"+ docker volume create --label "${harness_label}" "${volume_name}" >/dev/null+ start_databasus "${container_name}" \+ --mount "type=bind,source=${rejected_remap_command},target=/usr/sbin/groupmod,readonly" \+ --mount "type=bind,source=${rejected_remap_command},target=/usr/sbin/usermod,readonly" \+ --volume "${volume_name}:/databasus-data"++ container_environment="$(docker exec "${container_name}" env)"+ for identity_setting in DATABASUS_PUID=65532 DATABASUS_PGID=65532 POSTGRES_PUID=999 POSTGRES_PGID=999; do+ grep -Fxq "${identity_setting}" <<<"${container_environment}"+ done++ [[ "$(docker exec "${container_name}" id -u databasus)" == "65532" ]]+ [[ "$(docker exec "${container_name}" id -g databasus)" == "65532" ]]+ [[ "$(docker exec "${container_name}" id -u postgres)" == "999" ]]+ [[ "$(docker exec "${container_name}" id -g postgres)" == "999" ]]+ probe_service_paths "${container_name}"+}++run_custom_ids() {+ local container_name="databasus-storage-custom-ids-${run_id}"+ local volume_name="databasus-storage-custom-ids-${run_id}"++ docker volume create --label "${harness_label}" "${volume_name}" >/dev/null+ start_databasus "${container_name}" \+ --env DATABASUS_PUID=25001 \+ --env DATABASUS_PGID=25002 \+ --env POSTGRES_PUID=26001 \+ --env POSTGRES_PGID=26002 \+ --volume "${volume_name}:/databasus-data"++ [[ "$(docker exec "${container_name}" id -u databasus)" == "25001" ]]+ [[ "$(docker exec "${container_name}" id -g databasus)" == "25002" ]]+ [[ "$(docker exec "${container_name}" id -u postgres)" == "26001" ]]+ [[ "$(docker exec "${container_name}" id -g postgres)" == "26002" ]]+ probe_service_paths "${container_name}"+}++run_bind_root() {+ local container_name="databasus-storage-bind-${run_id}"+ local data_root="${case_root}/bind-root"++ mkdir -p "${data_root}"+ chmod 0770 "${data_root}"+ start_databasus "${container_name}" --volume "${data_root}:/databasus-data"+ probe_service_paths "${container_name}"+}++run_named_volume() {+ local container_name="databasus-storage-volume-${run_id}"+ local volume_name="databasus-storage-volume-${run_id}"++ docker volume create --label "${harness_label}" "${volume_name}" >/dev/null+ start_databasus "${container_name}" --volume "${volume_name}:/databasus-data"+ probe_service_paths "${container_name}"+}++run_split_mounts() {+ local container_name="databasus-storage-split-${run_id}"+ local data_volume_name="databasus-storage-split-data-${run_id}"+ local backup_volume_name="databasus-storage-split-backup-${run_id}"+ local pgdata_volume_name="databasus-storage-split-pgdata-${run_id}"+ local temporary_device_id+ local backup_device_id++ docker volume create --label "${harness_label}" "${data_volume_name}" >/dev/null+ docker volume create --label "${harness_label}" "${backup_volume_name}" >/dev/null+ docker volume create --label "${harness_label}" "${pgdata_volume_name}" >/dev/null+ start_databasus "${container_name}" \+ --volume "${data_volume_name}:/databasus-data" \+ --tmpfs /databasus-data/temp:rw,nosuid,size=64m,mode=0700,uid=65532,gid=65532 \+ --volume "${backup_volume_name}:/databasus-data/backups" \+ --volume "${pgdata_volume_name}:/databasus-data/pgdata"++ temporary_device_id="$(docker exec "${container_name}" stat -c %d /databasus-data/temp)"+ backup_device_id="$(docker exec "${container_name}" stat -c %d /databasus-data/backups)"+ [[ "${temporary_device_id}" != "${backup_device_id}" ]]+ probe_service_paths "${container_name}"+}++run_read_only() {+ local container_name="databasus-storage-read-only-${run_id}"+ local data_root="${case_root}/read-only"++ mkdir -p "${data_root}/pgdata" "${data_root}/pgsocket" "${data_root}/temp" "${data_root}/backups"+ docker run --rm --label "${harness_label}" --entrypoint /bin/sh \+ --volume "${data_root}:/data" "${databasus_image}" -c \+ 'chown 65532:65532 /data /data/temp /data/backups && chown 999:999 /data/pgdata /data/pgsocket && chmod 0770 /data /data/backups && chmod 0700 /data/pgdata /data/pgsocket /data/temp'+ docker run --detach \+ --name "${container_name}" \+ --label "${harness_label}" \+ --mount "type=bind,source=${data_root},target=/databasus-data,readonly" \+ "${databasus_image}" >/dev/null+ wait_for_exit "${container_name}"+ docker logs "${container_name}" 2>&1 | grep -Fqi "Read-only file system"+}++run_inaccessible_path() {+ local container_name="databasus-storage-inaccessible-${run_id}"+ local volume_name="databasus-storage-inaccessible-${run_id}"++ docker volume create --label "${harness_label}" "${volume_name}" >/dev/null+ start_databasus "${container_name}" --volume "${volume_name}:/databasus-data"+ docker exec "${container_name}" chmod 000 /databasus-data/backups+ if probe_path "${container_name}" databasus /databasus-data/backups >/dev/null 2>&1; then+ echo "Databasus wrote to an inaccessible backup path" >&2+ return 1+ fi+}++run_network_prerequisites() {+ [[ "$(uname -s)" == "Linux" ]]+ command -v mount.cifs >/dev/null || {+ echo "mount.cifs is required; install cifs-utils on the Docker host" >&2+ return 1+ }+ command -v mount.nfs >/dev/null || {+ echo "mount.nfs is required; install nfs-common on the Docker host" >&2+ return 1+ }++ docker image inspect "${samba_fixture_image}" >/dev/null 2>&1 || docker pull "${samba_fixture_image}" >/dev/null+ docker image inspect "${nfs_fixture_image}" >/dev/null 2>&1 || docker pull "${nfs_fixture_image}" >/dev/null+}++run_cifs() {+ local network_name="databasus-storage-cifs-${run_id}"+ local server_container_name="databasus-storage-cifs-server-${run_id}"+ local client_container_name="databasus-storage-cifs-client-${run_id}"+ local server_volume_name="databasus-storage-cifs-server-${run_id}"+ local data_volume_name="databasus-storage-cifs-data-${run_id}"+ local backup_volume_name="databasus-storage-cifs-backup-${run_id}"+ local temporary_volume_name="databasus-storage-cifs-temp-${run_id}"+ local server_address++ run_network_prerequisites+ docker network create --label "${harness_label}" "${network_name}" >/dev/null+ docker volume create --label "${harness_label}" "${server_volume_name}" >/dev/null+ docker volume create --label "${harness_label}" "${data_volume_name}" >/dev/null+ docker run --rm --label "${harness_label}" --entrypoint /bin/sh \+ --volume "${server_volume_name}:/shared" "${databasus_image}" -c \+ 'chown 25001:25002 /shared && chmod 0770 /shared'+ docker run --detach \+ --name "${server_container_name}" \+ --label "${harness_label}" \+ --network "${network_name}" \+ --health-interval 1s \+ --env NAME=Storage \+ --env USER=storage-test \+ --env PASS=storage-test-password \+ --env UID=25001 \+ --env GID=25002 \+ --volume "${server_volume_name}:/shared" \+ "${samba_fixture_image}" >/dev/null+ wait_for_health "${server_container_name}"++ server_address="$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "${server_container_name}")"+ for volume_name in "${backup_volume_name}" "${temporary_volume_name}"; do+ docker volume create \+ --label "${harness_label}" \+ --driver local \+ --opt type=cifs \+ --opt "device=//${server_address}/Storage" \+ --opt "o=username=storage-test,password=storage-test-password,vers=3.0,uid=25001,gid=25002,file_mode=0660,dir_mode=0770" \+ "${volume_name}" >/dev/null+ done++ start_databasus "${client_container_name}" \+ --env DATABASUS_PUID=25001 \+ --env DATABASUS_PGID=25002 \+ --env TMPDIR=/databasus-data/temp \+ --volume "${data_volume_name}:/databasus-data" \+ --volume "${temporary_volume_name}:/databasus-data/temp" \+ --volume "${backup_volume_name}:/databasus-data/backups"++ probe_path "${client_container_name}" databasus /databasus-data/temp+ probe_path "${client_container_name}" databasus /databasus-data/backups+ docker exec --user databasus "${client_container_name}" /bin/sh -c \+ "tr '\\0' '\\n' < /proc/1/environ | grep -Fxq TMPDIR=/tmp"+ assert_service_boundaries "${client_container_name}"+}++run_nfs_root_squash() {+ local network_name="databasus-storage-nfs-${run_id}"+ local server_container_name="databasus-storage-nfs-server-${run_id}"+ local client_container_name="databasus-storage-nfs-client-${run_id}"+ local server_volume_name="databasus-storage-nfs-server-${run_id}"+ local data_volume_name="databasus-storage-nfs-data-${run_id}"+ local client_volume_name="databasus-storage-nfs-client-${run_id}"+ local server_address++ run_network_prerequisites+ docker network create --label "${harness_label}" "${network_name}" >/dev/null+ docker volume create --label "${harness_label}" "${server_volume_name}" >/dev/null+ docker volume create --label "${harness_label}" "${data_volume_name}" >/dev/null+ docker run --rm --label "${harness_label}" --entrypoint /bin/sh \+ --volume "${server_volume_name}:/nfsshare" "${databasus_image}" -c \+ 'chown 65532:65532 /nfsshare && chmod 0770 /nfsshare'+ docker run --detach \+ --name "${server_container_name}" \+ --label "${harness_label}" \+ --network "${network_name}" \+ --privileged \+ --env SHARED_DIRECTORY=/nfsshare \+ --volume "${server_volume_name}:/nfsshare" \+ --entrypoint /bin/bash \+ "${nfs_fixture_image}" -c \+ "mount -t nfsd nfsd /proc/fs/nfsd && printf 10 > /proc/fs/nfsd/nfsv4gracetime && sed -i 's/no_root_squash/root_squash/' /etc/exports && exec /usr/bin/nfsd.sh" >/dev/null+ wait_for_log "${server_container_name}" "Startup successful."++ server_address="$(docker inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "${server_container_name}")"+ docker volume create \+ --label "${harness_label}" \+ --driver local \+ --opt type=nfs \+ --opt "o=addr=${server_address},rw,nfsvers=4" \+ --opt device=:/ \+ "${client_volume_name}" >/dev/null++ if docker run --rm --label "${harness_label}" --entrypoint /bin/sh \+ --volume "${client_volume_name}:/export" "${databasus_image}" -c \+ 'chown 1:1 /export' >/dev/null 2>&1; then+ echo "root-squashed NFS accepted a client-root ownership change" >&2+ return 1+ fi+ start_databasus "${client_container_name}" \+ --volume "${data_volume_name}:/databasus-data" \+ --volume "${client_volume_name}:/databasus-data/backups"+ probe_service_paths "${client_container_name}"+ docker rm -f "${client_container_name}" >/dev/null+ start_databasus "${client_container_name}" \+ --volume "${data_volume_name}:/databasus-data" \+ --volume "${client_volume_name}:/databasus-data/backups"+ probe_service_paths "${client_container_name}"+}++run_case() {+ local case_name="$1"++ echo "Running Docker filesystem case: ${case_name}"+ case "${case_name}" in+ build-only) docker image inspect "${databasus_image}" >/dev/null ;;+ default-ids) run_default_ids ;;+ custom-ids) run_custom_ids ;;+ bind-root) run_bind_root ;;+ named-volume) run_named_volume ;;+ split-mounts) run_split_mounts ;;+ read-only) run_read_only ;;+ inaccessible-path) run_inaccessible_path ;;+ network-prerequisites) run_network_prerequisites ;;+ cifs) run_cifs ;;+ nfs-root-squash) run_nfs_root_squash ;;+ *)+ echo "unknown Docker filesystem case: ${case_name}" >&2+ return 2+ ;;+ esac+}++# Issue coverage:+# #45, #64 and #224: bind and split mounts.+# #141, #478 and #738: default and custom service IDs without default remapping.+# #274: CIFS forced modes plus a private process TMPDIR.+# #751: PostgreSQL traversal through the bind-mounted data root.+# #763: CIFS numeric ownership.+# #96 and #199 require an arbitrary container user, which cannot start both services.+# #401 concerns application-level transfer policy, not container permissions.+# #431 exercises the existing cross-device move fallback, not container permissions.++if [[ "${selected_case}" == all ]]; then+ for case_name in \+ build-only default-ids custom-ids bind-root named-volume split-mounts \+ read-only inaccessible-path cifs nfs-root-squash; do+ run_case "${case_name}"+ cleanup+ case_root="$(mktemp -d /tmp/databasus-storage-test.XXXXXXXX)"+ done+else+ run_case "${selected_case}"+fidiff --git a/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/.openspec.yaml b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/.openspec.yamlnew file mode 100644index 00000000..1d9aeef9--- /dev/null+++ b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/.openspec.yaml@@ -0,0 +1,2 @@+schema: spec-driven+created: 2026-09-04diff --git a/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/design.md b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/design.mdnew file mode 100644index 00000000..e5a9e198--- /dev/null+++ b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/design.md@@ -0,0 +1,74 @@+## Context++The image runs Databasus and PostgreSQL under separate operating-system accounts and stores their data below `/databasus-data`. A mount can behave differently from the image filesystem because numeric ownership, root squash, CIFS mapping, and filesystem boundaries are host concerns.++The test boundary is the shipped container. The backend local-storage model does not need a new diagnostic command or a second file-publication abstraction to test that boundary.++## Goals / Non-Goals++**Goals:**++- Run the same real-container filesystem cases locally and in CI.+- Cover open and closed Docker filesystem and permission reports with executable cases or a precise exclusion.+- Keep service identity defaults in the image and allow explicit overrides.+- Fix only defects reproduced by the container matrix.++**Non-Goals:**++- Changing the backend local-storage model, connection checks, or backup publication.+- Adding a storage diagnostic command to the application.+- Moving the Docker entrypoint into a separate production script.+- Building ownership migration state or version-to-version upgrade fixtures.+- Requiring exact ZFS, Proxmox LXC, or user-namespace hosts in GitHub Actions.++## Decisions++### 1. Use one direct container harness++`make test-filesystems` runs `e2e/docker-storage/run.sh`. The harness accepts `DATABASUS_IMAGE`; otherwise the Make target builds `databasus-storage-test:local` from the root Dockerfile.++Each positive case starts the real image, waits for health, and uses `docker exec --user` to create, write, read, and remove a unique file under the account and path being tested. The harness does not call backend storage code. A trap removes containers, volumes, networks, mounts, and temporary directories.++Cases stay sequential because CIFS and NFS fixtures share host mount support and cleanup must remain deterministic.++### 2. Keep the matrix about Docker filesystems and permissions++The portable matrix covers:++- the default data-root bind mount;+- a Docker named volume;+- separate mounts for application files, temporary files, backups, and PostgreSQL data;+- default IDs and free custom IDs;+- readable, writable, read-only, and inaccessible paths;+- a real CIFS share with numeric group mapping;+- a real root-squashed NFS export mounted as application backup storage.++The issue inventory includes open and closed reports. A report belongs in the matrix when it can be reproduced at the shipped container boundary. Reports about remote database grants, remote-storage protocol features, or application behavior outside the container mount boundary get an explicit exclusion.++### 3. Keep production changes small++The Dockerfile remains the production entrypoint source. It declares the four default IDs in image metadata and reads the same four variables when an operator overrides them.++A failing case may justify a narrow entrypoint change, such as avoiding an unnecessary recursive ownership rewrite or accepting a pre-owned root-squashed mount. It does not justify a general ownership state machine, new backend commands, or a new local-storage abstraction.++Default startup must not remap accounts because the image accounts already use the declared IDs. Custom IDs may remap the two service accounts before services start. The Databasus account must not gain access to PostgreSQL private paths.++Filesystem setup keeps the existing `/postgresus-data` and `WAL_V1` startup guards. It also repairs ownership only for the known root-level application files instead of traversing mounted trees.++### 4. Run the candidate locally in CI++The `test-filesystems` job builds and loads one amd64 candidate with `push: false`. It runs independently of backend tests and image smoke jobs. Release and development image publication depend on this job.++The job installs CIFS and NFS clients, uses digest-pinned fixture images, and never publishes the candidate or transfers it between jobs.++### 5. Keep identity documentation in Advanced Configuration++The Dockerfile owns the defaults. `.env.example`, Compose, Helm values, the Helm README, root README files, and installation pages do not repeat them.++The English and five localized Advanced Configuration pages list the four variables, defaults, accepted numeric range, and override use. They describe the current configuration without compatibility history.++## Risks / Trade-offs++- GitHub-hosted runners cannot reproduce every ZFS, Proxmox, or namespace layout. Portable cases cover the numeric ownership and mount behavior behind those reports.+- Network filesystem fixtures are slower than bind mounts and named volumes. They remain in one sequential job with bounded timeouts and cleanup.+- Direct shell operations do not test backend backup publication. Issues #401 and #431 concern application-level transfer policy rather than container permissions, so this change does not alter or duplicate that code.diff --git a/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/proposal.md b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/proposal.mdnew file mode 100644index 00000000..30263535--- /dev/null+++ b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/proposal.md@@ -0,0 +1,24 @@+## Why++Docker storage permission changes can work on one mount type and fail on another. The project needs a real-container test suite for bind mounts, Docker volumes, CIFS, NFS, split mounts, and numeric ownership so these failures are caught before an image is published.++## What Changes++- Add one Bash harness behind `make test-filesystems`. It builds or accepts a local candidate image and runs the same cases locally and in GitHub Actions.+- Test file creation, writing, reading, and removal directly inside the running container under the Databasus and PostgreSQL accounts.+- Cover default and custom account IDs, bind mounts, named volumes, separate mounts, CIFS, NFS root squash, and rejected permission layouts.+- Declare `DATABASUS_PUID=65532`, `DATABASUS_PGID=65532`, `POSTGRES_PUID=999`, and `POSTGRES_PGID=999` in the Docker image. Operators can override them when mounted storage requires other numeric owners.+- Keep identity settings out of ordinary configuration. Document them only on the six Advanced Configuration pages.+- Change production code only when a container case reproduces a Docker entrypoint or image defect. Keep the inline entrypoint and backend local-storage model simple.++## Capabilities++### New Capabilities++- `docker-storage-compatibility`: Defines the real-container filesystem matrix, Docker account variables, local execution, CI execution, and cleanup.++## Impact++The change affects the Docker image, one filesystem test harness, the CI workflow, and Advanced Configuration documentation. Backend local-storage behavior, backup file publication, and connection-test behavior remain unchanged.++The suite tests the current candidate image only. It has no version-to-version upgrade fixtures and does not require a maintained ZFS, Proxmox, or user-namespace runner.diff --git a/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/specs/docker-storage-compatibility/spec.md b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/specs/docker-storage-compatibility/spec.mdnew file mode 100644index 00000000..0e7687b5--- /dev/null+++ b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/specs/docker-storage-compatibility/spec.md@@ -0,0 +1,93 @@+## Purpose++Defines the supported Docker account settings and the real-container filesystem tests that prevent storage permission changes from breaking common installation layouts.++## ADDED Requirements++### Requirement: Docker service IDs have image defaults++The Docker image SHALL declare `DATABASUS_PUID=65532`, `DATABASUS_PGID=65532`, `POSTGRES_PUID=999`, and `POSTGRES_PGID=999`. A container SHALL inherit these values without ordinary installation configuration. Operators SHALL be able to override each service account with free numeric IDs accepted by the image.++#### Scenario: Container uses image defaults++- **WHEN** an operator starts the image without identity overrides+- **THEN** the container environment contains the four image defaults+- **AND** the Databasus and PostgreSQL accounts use those IDs++#### Scenario: Operator supplies free custom IDs++- **WHEN** an operator supplies free numeric IDs through the four supported variables+- **THEN** the container starts with each service account using its configured user and primary group IDs++### Requirement: Service accounts remain isolated++The Databasus and PostgreSQL accounts SHALL have distinct user IDs and primary groups. PostgreSQL MAY receive access needed to traverse the shared data root. The Databasus account SHALL NOT gain access to PostgreSQL private data or its private socket.++#### Scenario: Databasus account reads PostgreSQL private paths++- **WHEN** the Databasus account attempts to read the embedded database tree or private socket+- **THEN** filesystem permissions reject the operation++### Requirement: Container preserves incompatible-data startup guards++The container SHALL refuse startup when it detects data at the deprecated Postgresus mount location or a metadata database containing a PostgreSQL configuration with backup type `WAL_V1`.++#### Scenario: Postgresus data directory is mounted++- **WHEN** the deprecated Postgresus mount location exists and contains data+- **THEN** the container exits before starting its services+- **AND** its log identifies the unsupported directory++#### Scenario: WAL_V1 configuration exists++- **WHEN** the metadata database contains a PostgreSQL configuration whose backup type is `WAL_V1`+- **THEN** the container exits before starting the Databasus application+- **AND** its log identifies the unsupported backup type++### Requirement: One filesystem suite runs locally and in CI++The repository SHALL provide `make test-filesystems` as the single entry point for the Docker filesystem suite. It SHALL build or accept a local candidate image and run the same cases locally and in a dedicated GitHub Actions job. The job SHALL build and load its own candidate without publishing or transferring it.++Each positive case SHALL start the real container and perform create, write, read, and remove operations directly under the responsible service account. Each negative case SHALL assert the container or filesystem operation failure. Every case SHALL clean its containers, volumes, networks, mounts, and temporary files.++#### Scenario: Developer runs the complete suite++- **WHEN** a developer runs `make test-filesystems` without an image reference+- **THEN** the target builds a local candidate and tests it++#### Scenario: CI tests its candidate++- **WHEN** the filesystem job builds and loads its candidate+- **THEN** the suite tests that local image without pushing or pulling a candidate image++#### Scenario: A case allocates Docker resources++- **WHEN** the case succeeds or fails after resource allocation+- **THEN** the harness removes every resource created by that case++### Requirement: The matrix covers supported mount behavior++The suite SHALL cover a bind-mounted data root, a Docker named volume, separate application, temporary, backup, and PostgreSQL mounts, default and custom service IDs, rejected permission layouts, CIFS numeric ownership, and NFS root squash. It SHALL test only the current candidate image and SHALL NOT include version-to-version upgrade fixtures.++The repository SHALL keep an inventory of open and closed Docker filesystem and permission reports. Each report that crosses the shipped container boundary SHALL map to an executable case. Each exclusion SHALL state why the report does not exercise that boundary.++#### Scenario: Local Docker storage layouts++- **WHEN** the candidate uses a bind mount, named volume, or separate mounts+- **THEN** the container becomes healthy and both service accounts can perform their required file operations++#### Scenario: CIFS maps numeric ownership++- **WHEN** a real CIFS fixture presents a mounted path with the configured Databasus identity+- **THEN** the container becomes healthy and the Databasus account can perform its required file operations++#### Scenario: NFS backup storage denies ownership changes to container root++- **WHEN** a root-squashed NFS fixture is already writable by the configured service identities+- **THEN** the container becomes healthy with its backup path on NFS+- **AND** both accounts can perform their required file operations on their assigned mounts++#### Scenario: Mounted path denies required access++- **WHEN** a mounted path does not permit an operation required by its service account+- **THEN** the container or the direct account-level probe failsdiff --git a/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/tasks.md b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/tasks.mdnew file mode 100644index 00000000..ecab396b--- /dev/null+++ b/openspec/changes/archive/2026-09-05-harden-docker-storage-compatibility/tasks.md@@ -0,0 +1,40 @@+## 1. Keep the implementation boundary small++- [x] 1.1 Keep the backend local-storage model and PostgreSQL credential code unchanged. Do not add application diagnostic commands or test-only binaries to the production image.+ Evidence: `git diff --exit-code -- backend` passes, and the image contains no new diagnostic binary.+- [x] 1.2 Keep the production entrypoint inline in the Dockerfile. Do not add external entrypoint scripts, ownership migration state, or version-to-version upgrade fixtures.+ Evidence: the entrypoint remains the `COPY <<EOF /app/start.sh` block in `Dockerfile`; the change adds one test script and no upgrade fixtures. The existing `/postgresus-data` and `WAL_V1` startup guards remain in place.++## 2. Filesystem harness++- [x] 2.1 Add `make build-docker-storage-test-image` and `make test-filesystems`. Make the harness accept `DATABASUS_IMAGE`, select one case through `CASE`, label every Docker resource uniquely, and clean all resources on success or failure.+ Evidence: `Makefile` exposes both targets; `e2e/docker-storage/run.sh` accepts both variables and labels resources with a per-run value. Docker container, volume, and network queries returned no labeled resources after the full run.+- [x] 2.2 Probe mounted paths directly with `docker exec --user databasus` and `docker exec --user postgres`. Verify unique-file create, write, read, and remove operations without calling backend storage code.+ Evidence: `probe_path` performs the four operations through `docker exec --user`; the complete matrix passes.++## 3. Image identities++- [x] 3.1 Declare `DATABASUS_PUID=65532`, `DATABASUS_PGID=65532`, `POSTGRES_PUID=999`, and `POSTGRES_PGID=999` in Docker image metadata. Verify the image environment, running-container environment, effective IDs, default no-remap behavior, and free explicit overrides.+ Evidence: the `default-ids` case also mounts failing commands over `groupmod` and `usermod`, so a redundant default remap stops the container. The `default-ids` and `custom-ids` cases verify image metadata, container environment, and effective account IDs. Both pass.+- [x] 3.2 Keep the four variables out of `.env.example`, default Compose configuration, Helm values, the Helm README, root README files, and installation pages. Document them only in all six Advanced Configuration pages.+ Evidence: repository search finds the four variable names only in `Dockerfile`, the filesystem harness, OpenSpec, and all six Advanced Configuration pages.++## 4. Filesystem and permission matrix++- [x] 4.1 Add healthy cases for a bind-mounted data root, a Docker named volume, and separate application, temporary, backup, and PostgreSQL mounts.+ Evidence: `bind-root`, `named-volume`, and `split-mounts` pass in the complete matrix.+- [x] 4.2 Add rejected cases for read-only and inaccessible paths. Assert container status and a relevant startup or operation failure.+ Evidence: `read-only` observes a nonzero container exit; `inaccessible-path` observes the Databasus account's expected permission denial.+- [x] 4.3 Add real CIFS and root-squashed NFS cases with digest-pinned fixtures and a prerequisite check for host client support.+ Evidence: `cifs` and `nfs-root-squash` pass with digest-pinned server images. `run_network_prerequisites` checks host helpers before either case.+- [x] 4.4 Audit open and closed Docker filesystem and permission issues. Map every container-boundary report to a case and record a precise reason for each exclusion.+ Evidence: the issue inventory in `e2e/docker-storage/run.sh` maps #45, #64, #141, #224, #274, #478, #738, #751, and #763. It excludes #96 and #199 because an arbitrary container user cannot start both services, and excludes #401 and #431 because they concern application transfer behavior rather than container permissions.+- [x] 4.5 Fix only Dockerfile or inline-entrypoint defects reproduced by these cases. Do not change backend local-storage behavior.+ Evidence: the Dockerfile declares the four IDs, skips default remapping, uses shallow best-effort ownership preparation for pre-owned mounts and the known root-level application files, and fixes process `TMPDIR` at `/tmp`. No backend file changed.++## 5. CI and final verification++- [x] 5.1 Add an independent `test-filesystems` GitHub Actions job that builds and loads its own candidate without publishing it. Pin every Action to a full commit SHA with a version comment, retain top-level `permissions: contents: read`, and require the job before release and development image publication. Verify the workflow with Actionlint and direct permission and dependency inspection.+ Evidence: Actionlint passes. The job uses full-SHA Actions, `push: false`, `load: true`, and its own local tag. Top-level permissions remain `contents: read`; both publication paths depend on the job.+- [x] 5.2 Run the targeted cases and the complete `DATABASUS_IMAGE=databasus-storage-test:local make test-filesystems` matrix. Run Dockerfile, shell, Compose, Helm, workflow, website, and OpenSpec validation, confirm zero labeled Docker resources remain, and obtain final reviewer `PASS`.+ Evidence: the rebuilt image passes the full matrix and the direct `/postgresus-data`, root-file ownership, and `WAL_V1` checks. The test runs leave no labeled resources. Bash syntax, ShellCheck, Docker build checks, Compose, Helm, Actionlint, website lint/build, OpenSpec validation, and the final reviewer all pass.diff --git a/openspec/specs/docker-storage-compatibility/spec.md b/openspec/specs/docker-storage-compatibility/spec.mdnew file mode 100644index 00000000..c6aa50a1--- /dev/null+++ b/openspec/specs/docker-storage-compatibility/spec.md@@ -0,0 +1,95 @@+# Docker Storage Compatibility Specification++## Purpose++Defines the supported Docker account settings and the real-container filesystem tests that prevent storage permission changes from breaking common installation layouts.++## Requirements++### Requirement: Docker service IDs have image defaults++The Docker image SHALL declare `DATABASUS_PUID=65532`, `DATABASUS_PGID=65532`, `POSTGRES_PUID=999`, and `POSTGRES_PGID=999`. A container SHALL inherit these values without ordinary installation configuration. Operators SHALL be able to override each service account with free numeric IDs accepted by the image.++#### Scenario: Container uses image defaults++- **WHEN** an operator starts the image without identity overrides+- **THEN** the container environment contains the four image defaults+- **AND** the Databasus and PostgreSQL accounts use those IDs++#### Scenario: Operator supplies free custom IDs++- **WHEN** an operator supplies free numeric IDs through the four supported variables+- **THEN** the container starts with each service account using its configured user and primary group IDs++### Requirement: Service accounts remain isolated++The Databasus and PostgreSQL accounts SHALL have distinct user IDs and primary groups. PostgreSQL MAY receive access needed to traverse the shared data root. The Databasus account SHALL NOT gain access to PostgreSQL private data or its private socket.++#### Scenario: Databasus account reads PostgreSQL private paths++- **WHEN** the Databasus account attempts to read the embedded database tree or private socket+- **THEN** filesystem permissions reject the operation++### Requirement: Container preserves incompatible-data startup guards++The container SHALL refuse startup when it detects data at the deprecated Postgresus mount location or a metadata database containing a PostgreSQL configuration with backup type `WAL_V1`.++#### Scenario: Postgresus data directory is mounted++- **WHEN** the deprecated Postgresus mount location exists and contains data+- **THEN** the container exits before starting its services+- **AND** its log identifies the unsupported directory++#### Scenario: WAL_V1 configuration exists++- **WHEN** the metadata database contains a PostgreSQL configuration whose backup type is `WAL_V1`+- **THEN** the container exits before starting the Databasus application+- **AND** its log identifies the unsupported backup type++### Requirement: One filesystem suite runs locally and in CI++The repository SHALL provide `make test-filesystems` as the single entry point for the Docker filesystem suite. It SHALL build or accept a local candidate image and run the same cases locally and in a dedicated GitHub Actions job. The job SHALL build and load its own candidate without publishing or transferring it.++Each positive case SHALL start the real container and perform create, write, read, and remove operations directly under the responsible service account. Each negative case SHALL assert the container or filesystem operation failure. Every case SHALL clean its containers, volumes, networks, mounts, and temporary files.++#### Scenario: Developer runs the complete suite++- **WHEN** a developer runs `make test-filesystems` without an image reference+- **THEN** the target builds a local candidate and tests it++#### Scenario: CI tests its candidate++- **WHEN** the filesystem job builds and loads its candidate+- **THEN** the suite tests that local image without pushing or pulling a candidate image++#### Scenario: A case allocates Docker resources++- **WHEN** the case succeeds or fails after resource allocation+- **THEN** the harness removes every resource created by that case++### Requirement: The matrix covers supported mount behavior++The suite SHALL cover a bind-mounted data root, a Docker named volume, separate application, temporary, backup, and PostgreSQL mounts, default and custom service IDs, rejected permission layouts, CIFS numeric ownership, and NFS root squash. It SHALL test only the current candidate image and SHALL NOT include version-to-version upgrade fixtures.++The repository SHALL keep an inventory of open and closed Docker filesystem and permission reports. Each report that crosses the shipped container boundary SHALL map to an executable case. Each exclusion SHALL state why the report does not exercise that boundary.++#### Scenario: Local Docker storage layouts++- **WHEN** the candidate uses a bind mount, named volume, or separate mounts+- **THEN** the container becomes healthy and both service accounts can perform their required file operations++#### Scenario: CIFS maps numeric ownership++- **WHEN** a real CIFS fixture presents a mounted path with the configured Databasus identity+- **THEN** the container becomes healthy and the Databasus account can perform its required file operations++#### Scenario: NFS backup storage denies ownership changes to container root++- **WHEN** a root-squashed NFS fixture is already writable by the configured service identities+- **THEN** the container becomes healthy with its backup path on NFS+- **AND** both accounts can perform their required file operations on their assigned mounts++#### Scenario: Mounted path denies required access++- **WHEN** a mounted path does not permit an operation required by its service account+- **THEN** the container or the direct account-level probe failsdiff --git a/openspec/specs/local-storage-file-lifecycle/spec.md b/openspec/specs/local-storage-file-lifecycle/spec.mdnew file mode 100644index 00000000..a2064f7b--- /dev/null+++ b/openspec/specs/local-storage-file-lifecycle/spec.md@@ -0,0 +1,85 @@+# Local Storage File Lifecycle Specification++## Purpose++Defines how local storage writes, publishes, reads, deletes, and probes backup files across one or more filesystems.++## Requirements++### Requirement: Local storage stages files before publication++Local storage SHALL write an incoming backup to the configured temporary folder. It SHALL sync and close the temporary file before publishing it at the corresponding path in the configured data folder. It SHALL create missing parent directories for both paths.++#### Scenario: Backup file is saved++- **WHEN** local storage receives a backup stream+- **THEN** it writes the stream to the temporary folder+- **AND** it syncs and closes the temporary file before publishing the final path++#### Scenario: Context is canceled during a write++- **WHEN** the operation context is canceled before or while local storage copies the stream+- **THEN** the save returns the context error+- **AND** it does not publish the temporary path through the normal completion path++### Requirement: Local storage supports separate temporary and data filesystems++Local storage SHALL publish a completed temporary file with a filesystem rename when the source and destination support it. When the temporary and data folders are on different filesystems, local storage SHALL copy the temporary file to the final path, sync the destination, and remove the temporary file after the copy succeeds.++#### Scenario: Temporary and final paths share a filesystem++- **WHEN** the completed temporary file can be renamed to the final path+- **THEN** local storage publishes it with that rename++#### Scenario: Temporary and final paths use different filesystems++- **WHEN** the temporary and final paths are on different filesystems+- **THEN** local storage copies the file to the final path and syncs it+- **AND** it removes the temporary file after the copy succeeds++#### Scenario: Rename fails for another reason++- **WHEN** the rename fails for a reason other than a filesystem boundary+- **THEN** local storage returns that failure without attempting the cross-filesystem copy path++### Requirement: Local storage reads published files++Local storage SHALL open files from the configured data folder. It SHALL return a file-not-found error when the requested path does not exist.++#### Scenario: Published file exists++- **WHEN** a caller requests an existing file+- **THEN** local storage returns a readable handle for the path in the data folder++#### Scenario: Published file is missing++- **WHEN** a caller requests a path that does not exist+- **THEN** local storage returns a file-not-found error++### Requirement: Local storage deletion is idempotent++Local storage SHALL remove the requested path from the configured data folder. A missing path SHALL be treated as successfully deleted.++#### Scenario: Published file is deleted++- **WHEN** a caller deletes an existing file+- **THEN** local storage removes it from the data folder++#### Scenario: Missing file is deleted++- **WHEN** a caller deletes a path that does not exist+- **THEN** local storage returns success++### Requirement: Local storage connection tests exercise the temporary folder++The local-storage connection test SHALL create, close, and remove a probe file in the configured temporary folder. It SHALL return an error when any required operation fails.++#### Scenario: Temporary folder permits the required operations++- **WHEN** the connection test can create, close, and remove its probe file+- **THEN** the connection test succeeds++#### Scenario: Temporary folder rejects a required operation++- **WHEN** probe creation, close, or removal fails+- **THEN** the connection test returns the operation errordiff --git a/website/app/(en)/advanced-config/page.tsx b/website/app/(en)/advanced-config/page.tsxindex db2797d0..bc2c17aa 100644--- a/website/app/(en)/advanced-config/page.tsx+++ b/website/app/(en)/advanced-config/page.tsx@@ -398,6 +398,34 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Docker storage permissions</h2>++ <p>+ Most installations should keep the baked IDs. Set these variables+ only when a bind mount, CIFS share or NFS export requires specific+ numeric ownership. Values must be base-10 integers from{" "}+ <code>1</code> through <code>4294967294</code>; empty values are+ rejected.+ </p>++ <table>+ <thead><tr><th>Variable</th><th>Default</th><th>Account</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Databasus user</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Databasus primary group</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>PostgreSQL user</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>PostgreSQL primary group</td></tr>+ </tbody>+ </table>++ <p>+ Keep application-owned mounts separate from pgdata. The entrypoint+ starts as namespace root, prepares both accounts, then drops+ privileges with <code>gosu</code>. Use rootless Docker or a user+ namespace for host isolation; arbitrary Docker <code>user:</code>{" "}+ overrides are unsupported.+ </p>+<h2 id="telemetry">Telemetry</h2><p>diff --git a/website/app/[lang]/advanced-config/content/es.tsx b/website/app/[lang]/advanced-config/content/es.tsxindex 2ffbf7ba..a63ff7bc 100644--- a/website/app/[lang]/advanced-config/content/es.tsx+++ b/website/app/[lang]/advanced-config/content/es.tsx@@ -411,6 +411,34 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Permisos del almacenamiento Docker</h2>++ <p>+ La mayoría de las instalaciones debe conservar los ID de la imagen.+ Cámbielos solo si un bind mount o un recurso CIFS o NFS exige un+ propietario numérico concreto. Se aceptan enteros decimales de{" "}+ <code>1</code> a <code>4294967294</code>; los valores vacíos se+ rechazan.+ </p>++ <table>+ <thead><tr><th>Variable</th><th>Valor predeterminado</th><th>Cuenta</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Usuario de Databasus</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Grupo principal de Databasus</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>Usuario de PostgreSQL</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>Grupo principal de PostgreSQL</td></tr>+ </tbody>+ </table>++ <p>+ Mantenga los mounts de la aplicación separados de pgdata. El+ entrypoint empieza como root dentro de su namespace, prepara+ ambas cuentas y reduce privilegios con <code>gosu</code>. Use+ Docker rootless o un user namespace para aislar el host; no se+ admiten valores arbitrarios de Docker <code>user:</code>.+ </p>+<h2 id="telemetry">Telemetría</h2><p>diff --git a/website/app/[lang]/advanced-config/content/fr.tsx b/website/app/[lang]/advanced-config/content/fr.tsxindex 90e010de..632f4013 100644--- a/website/app/[lang]/advanced-config/content/fr.tsx+++ b/website/app/[lang]/advanced-config/content/fr.tsx@@ -414,6 +414,34 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Permissions du stockage Docker</h2>++ <p>+ La plupart des installations doivent conserver les ID intégrés à+ l'image. Modifiez-les uniquement si un bind mount ou un+ partage CIFS ou NFS impose un propriétaire numérique précis. Les+ entiers décimaux de <code>1</code> à <code>4294967294</code> sont+ acceptés ; les valeurs vides sont refusées.+ </p>++ <table>+ <thead><tr><th>Variable</th><th>Valeur par défaut</th><th>Compte</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Utilisateur Databasus</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Groupe principal Databasus</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>Utilisateur PostgreSQL</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>Groupe principal PostgreSQL</td></tr>+ </tbody>+ </table>++ <p>+ Séparez les mounts de l'application de pgdata. L'entrypoint+ démarre comme root dans son namespace, prépare les deux comptes,+ puis réduit ses privilèges avec <code>gosu</code>. Utilisez Docker+ rootless ou un user namespace pour isoler l'hôte ; les valeurs+ Docker <code>user:</code> arbitraires ne sont pas prises en charge.+ </p>+<h2 id="telemetry">Télémétrie</h2><p>diff --git a/website/app/[lang]/advanced-config/content/pt.tsx b/website/app/[lang]/advanced-config/content/pt.tsxindex 27510e50..627a0aed 100644--- a/website/app/[lang]/advanced-config/content/pt.tsx+++ b/website/app/[lang]/advanced-config/content/pt.tsx@@ -404,6 +404,34 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Permissões do armazenamento Docker</h2>++ <p>+ A maioria das instalações deve manter os IDs da imagem. Altere-os+ apenas quando um bind mount ou compartilhamento CIFS ou NFS exigir+ propriedade numérica específica. São aceitos inteiros decimais de{" "}+ <code>1</code> a <code>4294967294</code>; valores vazios são+ rejeitados.+ </p>++ <table>+ <thead><tr><th>Variável</th><th>Padrão</th><th>Conta</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Usuário do Databasus</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Grupo principal do Databasus</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>Usuário do PostgreSQL</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>Grupo principal do PostgreSQL</td></tr>+ </tbody>+ </table>++ <p>+ Mantenha os mounts do aplicativo separados do pgdata. O entrypoint+ começa como root dentro do namespace, prepara as duas contas e+ reduz privilégios com <code>gosu</code>. Use Docker rootless ou+ um user namespace para isolar o host; valores arbitrários de+ Docker <code>user:</code> não são aceitos.+ </p>+<h2 id="telemetry">Telemetria</h2><p>diff --git a/website/app/[lang]/advanced-config/content/ru.tsx b/website/app/[lang]/advanced-config/content/ru.tsxindex 34bee4c7..63f9d4f6 100644--- a/website/app/[lang]/advanced-config/content/ru.tsx+++ b/website/app/[lang]/advanced-config/content/ru.tsx@@ -401,6 +401,34 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Права Docker-хранилища</h2>++ <p>+ Для большинства установок подходят ID из образа. Меняйте их,+ только если bind mount, CIFS или NFS требуют конкретного числового+ владельца. Допустимы десятичные значения от <code>1</code> до{" "}+ <code>4294967294</code>; пустые значения отклоняются.+ </p>++ <table>+ <thead><tr><th>Переменная</th><th>По умолчанию</th><th>Учетная запись</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Пользователь Databasus</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Основная группа Databasus</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>Пользователь PostgreSQL</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>Основная группа PostgreSQL</td></tr>+ </tbody>+ </table>++ <p>+ Не объединяйте точки монтирования приложения с pgdata. Сценарий+ запуска работает с правами root внутри пространства имен,+ настраивает обе учетные записи и сбрасывает привилегии через{" "}+ <code>gosu</code>. Для изоляции на хосте используйте Docker без+ прав root или пространство имен пользователей; произвольное+ значение Docker <code>user:</code> не поддерживается.+ </p>+<h2 id="telemetry">Телеметрия</h2><p>diff --git a/website/app/[lang]/advanced-config/content/zh.tsx b/website/app/[lang]/advanced-config/content/zh.tsxindex c5fa68b1..a194a872 100644--- a/website/app/[lang]/advanced-config/content/zh.tsx+++ b/website/app/[lang]/advanced-config/content/zh.tsx@@ -383,6 +383,29 @@ export default function AdvancedConfigPage() {</tbody></table>+ <h2 id="docker-storage-permissions">Docker 存储权限</h2>++ <p>+ 大多数安装应保留镜像内置 ID。只有 bind mount、CIFS 或 NFS 共享要求特定数字所有者时才需要修改。可用值是{" "}+ <code>1</code> 到 <code>4294967294</code> 的十进制整数;空值会被拒绝。+ </p>++ <table>+ <thead><tr><th>变量</th><th>默认值</th><th>账户</th></tr></thead>+ <tbody>+ <tr><td><code>DATABASUS_PUID</code></td><td><code>65532</code></td><td>Databasus 用户</td></tr>+ <tr><td><code>DATABASUS_PGID</code></td><td><code>65532</code></td><td>Databasus 主组</td></tr>+ <tr><td><code>POSTGRES_PUID</code></td><td><code>999</code></td><td>PostgreSQL 用户</td></tr>+ <tr><td><code>POSTGRES_PGID</code></td><td><code>999</code></td><td>PostgreSQL 主组</td></tr>+ </tbody>+ </table>++ <p>+ 请将应用 mount 与 pgdata 分开。Entrypoint 会先以 namespace 内的 root 身份启动,准备两个账户,再通过{" "}+ <code>gosu</code> 降低权限。若要隔离宿主机,请使用 rootless Docker 或 user namespace;不支持任意 Docker{" "}+ <code>user:</code> 覆盖。+ </p>+<h2 id="telemetry">遥测</h2><p>