PostgreSQL backup tool with Point-In-Time-Recovery and restore verification
TypeScript backupbackupsdatabasedatabase-backupdevopsdockergolangkubernetesmariadbmongodbmysqlpgpostgrespostgresqls3self-hostedsystem-administrationtoolsweb-ui
1. Reproduce the failure
- 1.1 Add
e2e/docker-storage/no-unix-sockets.json, a seccomp profile that allows every system call exceptsocketwith address familyAF_UNIX(value1), which returns errno97. Add ano-unix-socketscase toe2e/docker-storage/run.shthat starts the candidate with--security-opt seccomp=<profile>on a named volume and callsassert_running_container. Add the case to thealllist. Verify thatdocker build --tag databasus-storage-test:local .andCASE=no-unix-sockets DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shfail, and that the container log containscould not create any Unix-domain sockets.
2. Bootstrap without a socket
- 2.1 In
docker/start.sh, drop/databasus-data/pgsocketfromcreate_required_data_directories,normalize_data_permissionsandverify_service_directories. Verify thatgrep -n pgsocket docker/start.shlists only the lines that tasks 2.2 to 2.4 replace. - 2.2 Replace
write_bootstrap_postgresql_authenticationandactivate_runtime_postgresql_authenticationwith one function. It writes the finalpg_hba.conf, with SCRAM on127.0.0.1/32and::1/128, reject for replication and nolocalrules, and an emptypg_ident.conf, both with mode0600, before any PostgreSQL process starts. Delete thepg_ctl reloadcall. Verify thatgrep -nE 'peer|pg_ctl|databasus_bootstrap' docker/start.shprints nothing. - 2.3 Add the single-user open step from design decision 2. It runs
postgres --singlewith empty input underdatabasus, captures its output and prints that output only on failure. Move thepg_resetwalrecovery fromstart_postgresql_with_recoveryaround this step, and remove a stalepostmaster.pidbeforepg_resetwalruns. Verify after task 2.5 through the recovery case of task 3.4. - 2.4 Add the single-user password step from design decisions 2 and 3. It pipes
ALTER ROLE postgres WITH PASSWORD '<generated>'from theprintfbuiltin intopostgres --singlewith-c exit_on_error=on,-c log_min_error_statement=panicand-c password_encryption=scram-sha-256, and stops startup on a non-zero exit. Verify thatgrep -nE 'env PGPASSWORD|-v internal_postgres_password' docker/start.shprints nothing once task 2.5 has also converted the remainingpsqlcalls. - 2.5 Start the regular server with
-c unix_socket_directories=''and-c max_wal_senders=0and pollpg_isready -h localhost -p 5437. Without a reset, a failed start stops the container. Create the metadata database throughpsql -h localhostwith the existing\gexecstatement. PassPGPASSWORDas a shell assignment prefix in front ofgosuin everypsqlcall, includingreject_legacy_wal_configuration. Delete the socket-login check fromverify_postgresql_runtime_authenticationand keep the loopback check. Verify thatdocker build --tag databasus-storage-test:local .succeeds and thatCASE=no-unix-sockets DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shpasses. Then verify thatgrep -n pgsocket docker/start.shprints nothing.
3. Filesystem suite
- 3.1 In
assert_postgresql_runtime_authentication, delete the socket-login probe and thelocal all all rejectcheck. Add a check that/proc/net/unixinside the container lists no.s.PGSQLentry. Add a check that neither thedocker logsoutput nor any/proc/*/cmdlineinside the container contains the password from/dev/shm/databasus-database-dsn. Add a probe that runspsql "host=localhost port=5437 user=postgres dbname=postgres replication=database"with the generated password from/dev/shm/databasus-database-dsnand asserts that PostgreSQL rejects it withnumber of requested standby connections exceeds. Keep the loopback probe without a password and the0600mode checks. Verify thatCASE=bind-root DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shand theno-unix-socketscase pass. - 3.2 Stop creating
pgsocketinprepare_bind_treeand in the NFS share setup. Delete both--tmpfs /databasus-data/pgsocketmounts from the NFS case. Verify thatgrep -n pgsocket e2e/docker-storage/run.shprints nothing, and that theautomatic-ids,split-mountsandnfs-root-squashcases pass one at a time throughCASE=<case> DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.sh. - 3.3 Add
v3.60.0torun_upgrades. Add a check torun_upgradethat records whether/databasus-data/pgsocketexists before the update and, when it did, asserts that it still exists afterwards in the healthy container. Onlyv3.56.0and later create that directory, so for those releases the check requires it to exist before the update. Verify thatCASE=upgrades DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shpasses. - 3.4 Add a
postgresql-recoverycase with five parts, and add it to thealllist. In the first part, read the password from/dev/shm/databasus-database-dsn, write a marker row, stop the container withdocker kill --signal KILL, start it again and assert that it is healthy, the marker exists and the log has nopg_resetwalline. Assert also that a loopbackpsql -h localhostlogin with the old password is refused and one with the newly published password succeeds. In the second part, stop the container cleanly, remove the segments underpgdata/pg_wal, start it again and assert that the log reports a write-ahead log reset and that the container is healthy. In the third part, write16intopgdata/PG_VERSION, which makes both the open step andpg_resetwalfail, start the container and assert that it exits with an error, the application never starts and the log reports the failed recovery. Run the remaining parts on fresh volumes. In the fourth part, appendlisten_addresses = '192.0.2.1'topostgresql.conf, start the container and assert that it exits with an error, that the log reports the password step without its failure,Starting PostgreSQL...andERROR: PostgreSQL did not start., and that it has nopg_resetwalline. In the fifth part, appenddefault_transaction_read_only = ontopostgresql.conf, start the container and assert four things: it exits with a non-zero code, the log reportsERROR: PostgreSQL rejected the internal password change.and has noStarting PostgreSQL...line, the application never starts, anddocker logscontains neitherALTER ROLE postgres WITH PASSWORD, norPASSWORD ', nor any 64-character hexadecimal string. Verify thatCASE=postgresql-recovery DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shpasses.
4. Specs and review
- 4.1 Verify that
openspec validate remove-embedded-postgresql-socket --strictpasses. - 4.2 Run the mandatory full implementation review from
AGENTS.mdand resolve its blocking findings. Then rerun theno-unix-sockets,bind-root,upgradesandpostgresql-recoverycases throughCASE=<case> DATABASUS_IMAGE=databasus-storage-test:local bash e2e/docker-storage/run.shafter rebuilding the image, and verify that they pass.