PostgreSQL backup tool with Point-In-Time-Recovery and restore verification
TypeScript backupbackupsdatabasedatabase-backupdevopsdockergolangkubernetesmariadbmongodbmysqlpgpostgrespostgresqls3self-hostedsystem-administrationtoolsweb-ui
REFACTOR (docs): Plan restoring console command access to the metadata database
Rostislav Dugin09c0ccf3aa3b
diff --git a/openspec/changes/restore-console-command-database-access/.openspec.yaml b/openspec/changes/restore-console-command-database-access/.openspec.yamlnew file mode 100644index 00000000..cbd245e4--- /dev/null+++ b/openspec/changes/restore-console-command-database-access/.openspec.yaml@@ -0,0 +1,2 @@+schema: spec-driven+created: 2026-09-20diff --git a/openspec/changes/restore-console-command-database-access/design.md b/openspec/changes/restore-console-command-database-access/design.mdnew file mode 100644index 00000000..586481cb--- /dev/null+++ b/openspec/changes/restore-console-command-database-access/design.md@@ -0,0 +1,154 @@+## Context++See proposal.md — Why for the motivation. The mechanics behind it:++`configure_application_database_dsn` (`docker/start.sh:528-534`) exports+`DATABASE_DSN` with the password generated a few steps earlier in+`configure_postgresql_database` (`docker/start.sh:466-483`). `main()` ends with+`exec gosu databasus ./main` (`docker/start.sh:595`), so the application+inherits that environment in the same process, which becomes PID 1.++A process started with `docker exec` is not a child of PID 1. The container+runtime builds its environment from the image configuration and the flags given+to `docker run`, not from the running process, so `DATABASE_DSN` is absent+there. Configuration loading then reaches `godotenv.Load`+(`backend/internal/config/config.go:126-133`), which reads `/.env`. That file is+`.env.example` baked into the image (`Dockerfile:193`), and its `DATABASE_DSN`+still carries the fixed password from before rotation (`.env.example:9`).+`godotenv.Load` does not overwrite variables already present in the+environment, so this fallback is invisible to PID 1 and decisive for every+other process.++The failure surfaces during migrations because a console command runs the full+startup path before reaching its own work: `resetPasswordIfRequested` is called+at `backend/cmd/main.go:115`, well after the database is opened.++Two constraints shape the options. The image runs PostgreSQL and the+application under one unprivileged account created in `Dockerfile:157-165`, and+`configure_runtime_identity` (`docker/start.sh:121-136`) can renumber that+account from `PUID`/`PGID` before the database is bootstrapped. Mounting a new+filesystem inside the container is unavailable: that needs `CAP_SYS_ADMIN`,+which an ordinary container does not have.++Module rules: the root `AGENTS.md` engineering philosophy and+`backend/AGENTS.md`, whose comment rule means the reason for reading a second+configuration source belongs in a short *why* comment or in a name, not in a+narration of the mechanism.++## Goals / Non-Goals++**Goals:**++- One credential resolution path, producing the same answer in the application+ and in a console command.+- The credential never outlives the container it belongs to, so it cannot go+ stale against a rotated password.+- A missing configuration fails with a message that names the missing+ configuration.++**Non-Goals:**++- Reworking how the password is generated or how often.+- Making `/.env` a supported way to point the container at an external+ metadata database. It never worked for PID 1, because the exported+ environment variable wins, and this design keeps that precedence.+- A general secret-sharing mechanism for other values.++## Decisions++### Publish the credential in a memory-backed file at `/dev/shm`++Startup writes the generated connection string to a file under `/dev/shm`, mode+`0600`, owned by the runtime account. Configuration loading reads it when the+environment does not already carry `DATABASE_DSN`.++`/dev/shm` is a tmpfs the container runtime always provides, so its contents+live in memory and disappear when the container stops. That lifetime is the+point: the password is regenerated on every start+(`docker/start.sh:466-483`), and a channel that dies with the container cannot+present yesterday's password to tomorrow's process. Ownership and mode confine+it to the account that already holds the database, which is also the account a+privileged `docker exec` runs as or above.++Alternatives rejected:++- **A file on the `/databasus-data` volume.** It survives restarts, so a start+ that fails between rotating the password and rewriting the file leaves a+ stale credential behind — the same class of failure being fixed here. It also+ puts the internal password on the operator's disk permanently.+- **Reading `/proc/1/environ`.** It needs no new artifact, but it only works+ while PID 1 is the application. Adding an init process or a supervisor to the+ entrypoint would silently break it, it is unavailable when the application+ has exited, and it cannot be exercised outside a container.+- **Mounting a dedicated tmpfs for the file.** Requires `CAP_SYS_ADMIN`, so it+ would force operators to grant the container privileges it does not otherwise+ need.+- **Baking a fixed password back into the image.** Reverts the security+ property established by the archived `protect-internal-postgresql-access`+ change.++### Resolution order: environment, then the published file, then the baked defaults++An explicit `DATABASE_DSN` in the process environment wins. Otherwise the+published file is used. Only then do the remaining baked defaults apply.++This mirrors what PID 1 already experiences. When the operator supplies a+connection string with `docker run -e`, it is part of the container+configuration, so `docker exec` inherits it too, and+`configure_application_database_dsn` (`docker/start.sh:529`) skips publishing+anything — both processes agree on the external database. When the operator+supplies nothing, both read the generated value.++Alternative rejected: **file first, environment second.** It would let the+embedded credential override an operator's deliberate choice of an external+metadata database, inverting the precedence the current startup already+guarantees.++### Delete `DATABASE_DSN` from the baked `/.env`, keep it in `.env.example`++`.env.example` stays as it is, because the repository's Docker Compose stack+and local backend runs depend on it. The copy that reaches the image+(`Dockerfile:189-193`) has the `DATABASE_DSN` line removed during the build.++Leaving it would keep a connection string that cannot authenticate as the last+fallback, which is precisely what turns "nothing configured" into "password+authentication failed" and sent the reporter of the issue looking in the wrong+place. `DatabaseDsn` is already `required:"true"`+(`backend/internal/config/config.go:49`), so removing the line produces the+existing empty-value diagnostic (`backend/internal/config/config.go:178`)+instead of an authentication error.++Alternatives rejected:++- **Removing `DATABASE_DSN` from `.env.example` itself.** Breaks local+ development, which has no startup script to publish a credential.+- **Maintaining a separate container-only example file.** A second file to keep+ in sync for the sake of one line.++## Risks / Trade-offs++- **`--ipc=host` shares the host's `/dev/shm`** → The published file would land+ in a namespace the host and other containers with that flag can read. This+ only applies to a deliberate, uncommon flag, and anyone with host root can+ already read the application's environment. Documenting it is enough; a+ startup check that refuses to publish is available later if it proves+ necessary.+- **tmpfs pages can be swapped to disk by the host** → The password could touch+ the host's swap. The same is true of the application's own memory, where the+ connection string lives regardless.+- **A container image that changes its entrypoint or runtime account** → The+ file's owner is resolved from the account after `configure_runtime_identity`+ (`docker/start.sh:121-136`) has applied `PUID`/`PGID`, so publishing must stay+ ordered after it. `bootstrap_postgresql` already runs later in `main()`+ (`docker/start.sh:586-595`), so the required order is the existing one.+- **Silent regression** → The failure is invisible to normal startup: PID 1+ keeps working even when publishing is broken. An automated check that runs a+ console command against a started container is the only thing that catches+ it, which is why it belongs in the task list rather than in manual testing.++## Migration Plan++No data migration and no operator action. A container built from the new image+publishes the credential at its next start, and the console commands work from+that moment. Rolling back to the previous image restores the previous+behavior, including the broken console commands.diff --git a/openspec/changes/restore-console-command-database-access/proposal.md b/openspec/changes/restore-console-command-database-access/proposal.mdnew file mode 100644index 00000000..9db78abe--- /dev/null+++ b/openspec/changes/restore-console-command-database-access/proposal.md@@ -0,0 +1,98 @@+## Why++The console commands documented in the README, such as the password reset+`docker exec -it databasus ./main --new-password="..." --email="..."`, stopped+working once the container began generating a random password for its embedded+metadata database at every startup. Startup hands that password to the+application through the environment of the first container process, and+`docker exec` does not inherit that environment. A command started this way+falls back to the connection string baked into the image, which still carries+the previously published fixed password, and PostgreSQL rejects it with+`password authentication failed for user "postgres"`.++This breaks the documented recovery path for an operator locked out of their+own instance, and it breaks nightly pipelines that reset the password on a+fresh container. It was reported in issue #784 and affects every console+command, not only the password reset, because all of them read the metadata+database.++## What Changes++- Startup makes the generated connection string readable by any process in the+ container, instead of leaving it only in the first process's environment. The+ value lives in memory for the lifetime of the container and never reaches a+ mounted volume or the image.+- A console command started after the application resolves the connection+ string from that shared location when the operator has not supplied one+ explicitly.+- The stale connection string baked into the image as a default is deleted. A+ command that finds no connection string anywhere now fails with a message+ naming the missing configuration, rather than with an authentication error+ that points at the wrong cause.+- Precedence becomes explicit and identical for every process: an explicitly+ supplied connection string wins, then the value startup generated, then what+ remains of the baked defaults.++No BREAKING changes. The deleted default could not authenticate against the+embedded database in any supported configuration, so nothing that works today+stops working. An operator who points the container at an external metadata+database supplies that connection string as a container environment variable,+which `docker exec` already inherits; that path is unchanged.++## Capabilities++### New Capabilities++None.++### Modified Capabilities++- `internal-postgresql-protection`: the requirement covering startup-generated+ internal credentials gains the obligation to make the credential reachable by+ later processes in the same container, states that the credential is never+ written to persistent storage, and fixes the precedence between an+ operator-supplied connection string and the generated one. A new requirement+ covers the console commands' own observable behavior: they reach the metadata+ database on a running container, and they report missing configuration+ distinguishably from rejected credentials.++## Impact++Affected code:++- `docker/start.sh`: the step that generates the password and exports the+ connection string also publishes it for the rest of the container.+- `backend/internal/config/config.go`: configuration loading gains the shared+ location as a source, ordered between the process environment and the baked+ defaults.+- `Dockerfile`: the baked default file no longer carries a metadata-database+ connection string.+- `.env.example`: keeps its local-development connection string, which the+ repository's Docker Compose stack still needs.++Affected behavior: the documented password-reset command works again on a+running container; the planned `--disable-2fa` command and the console command+from `replace-seeded-admin-with-first-signup` inherit the same fix rather than+shipping broken.++Module rules this change answers to: the root `AGENTS.md` and+`backend/AGENTS.md`.++### Out of scope++- Changing password rotation. The password keeps changing at every container+ start.+- Storing the credential on the data volume or anywhere else that survives a+ container restart.+- The other development values that reach the image through the same baked+ defaults file, such as the test mail server address. They are a separate+ problem with a separate fix.+- Adding, removing or renaming console commands, and changing the documented+ command syntax in the README translations.+- Operator-facing documentation stating that an external metadata database is+ supplied as a container environment variable and that a mounted `/.env` is+ not a supported place for it. That restriction predates this change and is+ unchanged by it, and documenting it means editing the English page plus five+ locale copies under the website's sync rule. It belongs in its own change.+- Publishing the embedded PostgreSQL port or relaxing any of its+ authentication rules.diff --git a/openspec/changes/restore-console-command-database-access/specs/internal-postgresql-protection/spec.md b/openspec/changes/restore-console-command-database-access/specs/internal-postgresql-protection/spec.mdnew file mode 100644index 00000000..d0cca8c2--- /dev/null+++ b/openspec/changes/restore-console-command-database-access/specs/internal-postgresql-protection/spec.md@@ -0,0 +1,99 @@+## MODIFIED Requirements++### Requirement: Internal PostgreSQL credentials are generated at startup++When Databasus uses its embedded metadata database, the container SHALL generate a cryptographically random PostgreSQL password at startup, set it on the internal role, and supply it to the application without logging it. A container restart SHALL replace the previous password. An explicitly supplied external metadata-database connection string SHALL remain unchanged.++The generated connection string SHALL also be readable by processes that start later in the same container, so that a command run against a running instance authenticates with the same credential as the application. That value SHALL exist only for the lifetime of the running container: it SHALL NOT be written to storage that survives a container stop, and it SHALL NOT be part of the distributed image.++When the operator supplies a metadata-database connection string, startup SHALL neither generate nor publish one, and every process in the container SHALL use the supplied value. When the operator supplies none, every process SHALL use the generated one. No process SHALL fall back to a connection string shipped as a default inside the image.++An operator-supplied connection string that points back to the embedded database with the previously published fixed password is incompatible with rotation and SHALL NOT be treated as a supported embedded-database configuration.++#### Scenario: Fresh container startup++- **WHEN** a container starts without an explicit metadata-database connection string+- **THEN** it configures the embedded role and the application with the same generated password+- **AND** the application becomes healthy without printing that password++#### Scenario: Container restart++- **WHEN** the same container restarts+- **THEN** the embedded role receives a different generated password+- **AND** the application reconnects with the new password+- **AND** a command started after the restart authenticates with the new password rather than the previous one++#### Scenario: Previously published fixed password++- **WHEN** a client attempts loopback TCP authentication with the old fixed image password+- **THEN** PostgreSQL rejects the connection++#### Scenario: External metadata database++- **WHEN** the operator explicitly supplies a connection string for an external metadata database+- **THEN** startup preserves that value rather than replacing it with the embedded database connection+- **AND** startup publishes no generated connection string for other processes to pick up++#### Scenario: Explicit connection string points to the embedded database++- **WHEN** the operator explicitly supplies the previously published fixed connection string for the embedded database+- **THEN** the application cannot authenticate after startup rotates the internal role's password+- **AND** the operator must remove that explicit value so startup can supply the generated credential++## ADDED Requirements++### Requirement: Console commands reach the metadata database of a running container++A console command executed inside an already-running container SHALL resolve the metadata-database connection the same way the application did, and SHALL connect successfully whenever the application itself is connected. This SHALL hold for a container using the embedded database and for one pointed at an external metadata database.++An operator-supplied connection string SHALL take precedence over the one startup generated. A console command SHALL NOT need any argument, file or environment value beyond what the operator already supplied to the container.++#### Scenario: Password reset on a fresh instance++- **WHEN** an operator runs the documented password-reset command inside a running container that uses the embedded database+- **THEN** the command connects to the metadata database and sets the new password+- **AND** the operator can sign in with it++#### Scenario: Console command against an external metadata database++- **WHEN** the operator supplied a connection string for an external metadata database and runs a console command+- **THEN** the command connects to that external database++#### Scenario: Console command while the application is running++- **WHEN** a console command runs while the application holds its own connection to the embedded database+- **THEN** both connect with the same credential and neither disturbs the other++### Requirement: Missing metadata-database configuration is reported as such++When no metadata-database connection string is available from any source, a starting process SHALL report that the configuration is missing and SHALL NOT attempt a connection with a default credential. A rejected credential and an absent configuration SHALL produce distinguishable messages.++#### Scenario: No connection string anywhere++- **WHEN** a process starts in a container where startup published no connection string and the operator supplied none+- **THEN** it reports the metadata-database configuration as missing+- **AND** the message does not describe an authentication failure++#### Scenario: Wrong credential supplied++- **WHEN** a process starts with a connection string whose password the database rejects+- **THEN** it reports an authentication failure rather than a missing configuration++### Requirement: The shared internal credential stays inside the container++The published connection string SHALL be readable only by the account that runs Databasus and by the container's privileged account. Any other account inside the container SHALL be refused. The value SHALL NOT appear in logs, in the image, or on any volume the operator mounts.++#### Scenario: Another account in the container++- **WHEN** a process running under an operating-system account other than the Databasus runtime account attempts to read the published connection string+- **THEN** the read is refused++#### Scenario: Operator inspects the mounted data volume++- **WHEN** the operator inspects the mounted data directory after the container has started+- **THEN** the generated password is absent from it++#### Scenario: Operator inspects a stopped container's image++- **WHEN** the operator inspects the distributed image+- **THEN** it contains no metadata-database connection string usable against the embedded databasediff --git a/openspec/changes/restore-console-command-database-access/tasks.md b/openspec/changes/restore-console-command-database-access/tasks.mdnew file mode 100644index 00000000..836cb6e2--- /dev/null+++ b/openspec/changes/restore-console-command-database-access/tasks.md@@ -0,0 +1,32 @@+## 1. Resolve the credential inside the backend++- [ ] 1.1 Add a resolution step in `backend/internal/config` that reads the published connection string when `DATABASE_DSN` is absent from the process environment, ordered before the `/.env` load at `backend/internal/config/config.go:126-133`; verify with a new test in `backend/internal/config/config_test.go` that the published value is used when the environment is empty+- [ ] 1.2 Cover the precedence rule with a test asserting that a `DATABASE_DSN` already set in the environment wins over a published file holding a different value; verify with `cd backend && make test` for that package+- [ ] 1.3 Cover the absent case with a test asserting that no published file and no environment value leaves `DatabaseDsn` empty, so the existing required-field diagnostic at `backend/internal/config/config.go:178` is what the operator sees; verify with `cd backend && make test` for that package+- [ ] 1.4 Run `cd backend && make lint` and fix every finding in the touched files++## 2. Publish the credential at startup++- [ ] 2.1 Extend `configure_application_database_dsn` (`docker/start.sh:528-534`) to write the generated connection string to the memory-backed location with mode `0600` owned by the runtime account, keeping the existing early return that preserves an operator-supplied value; verify by starting a built container and confirming the file exists with the expected owner and mode+- [ ] 2.2 Confirm the ordering constraint holds by starting a container with `PUID` and `PGID` set to unused non-default ids and checking that the published file is owned by the renumbered account++## 3. Remove the stale default from the image++- [ ] 3.1 Drop `DATABASE_DSN` from the copy of `.env.example` that lands at `/.env` during the build (`Dockerfile:189-193`), leaving `.env.example` itself untouched, and correct the comment above that copy (`Dockerfile:189-192`), which currently explains the baked defaults and the precedence rule without the new source; verify with `docker run --rm --entrypoint sh <image> -c 'grep -c DATABASE_DSN /.env'` returning no match+- [ ] 3.2 Confirm local development still works after the change by running the backend against the repository Docker Compose stack++## 4. Verify the reported scenario end to end++- [ ] 4.1 Build the image, start a fresh container with no `DATABASE_DSN` supplied, and run `docker exec <container> ./main --new-password="..." --email="..."`; verify the command completes without an authentication error and the new password signs in+- [ ] 4.2 Restart that container and run the same command again; verify it succeeds with the rotated password rather than the one from the previous run+- [ ] 4.3 Start a container with an explicit `DATABASE_DSN` pointing at an external metadata database, then run a console command; verify it connects to that external database and that no credential file was published+- [ ] 4.4 Start a container whose startup published nothing and whose environment carries no connection string, then run a console command; verify the failure names the missing configuration instead of reporting an authentication failure++## 5. Guard against silent regression++- [ ] 5.1 Add an automated check that runs a console command against a started container, so a future change that breaks publishing fails a check rather than only breaking `docker exec`; verify the check fails when the publishing step is removed and passes when it is restored++## 6. Review++- [ ] 6.1 Complete the mandatory compliance review for the finished diff and resolve every CHANGES REQUIRED finding.+- [ ] 6.2 Mark the completed tasks in this file so the status update is part of the same commit as the implementation