Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.
Python ai-agentsaiopsdashboarddockerfastapihomelabincident-managementllmmonitoringopenaireactroot-cause-analysisself-hosteduptime-monitoring
docs(traefik): cover upgrading from another folder and use placeholders for addresses (#177)
Karlo Kajba Šimanić359edaf55795
diff --git a/docs/adding-servers.md b/docs/adding-servers.mdindex 8ca9b24..dac17cc 100644--- a/docs/adding-servers.md+++ b/docs/adding-servers.md@@ -39,7 +39,7 @@ services:collector:image: ghcr.io/…/collector@sha256:…environment:- INFRADECK_API_URL: http://10.8.0.1:8081/api+ INFRADECK_API_URL: http://<vpn-address>:8081/apiINFRADECK_JOIN_TOKEN: hlj_…INFRADECK_DOCKER_HOST: /run/docker-proxy/docker.sockvolumes:diff --git a/docs/configuration.md b/docs/configuration.mdindex 2af40ee..b956752 100644--- a/docs/configuration.md+++ b/docs/configuration.md@@ -35,7 +35,7 @@ Every setting below is listed, with no values, in [`example.env`](../example.env| `LOGIN_RATE_LIMIT` / `LOGIN_RATE_WINDOW_SECONDS` | `5` / `60` | Login attempts per IP and per email. || `REGISTER_RATE_LIMIT` / `REGISTER_RATE_WINDOW_SECONDS` | `3` / `3600` | Registrations per IP. || `COLLECTOR_BIND` / `COLLECTOR_PORT` | `127.0.0.1` / `8081` | Server file: address and port where the collector gate is published. Set the address to your VPN's. |-| `COLLECTOR_PUBLIC_URL` | empty | What **Connect a server** tells new machines to use, e.g. `http://10.8.0.1:8081/api`. |+| `COLLECTOR_PUBLIC_URL` | empty | What **Connect a server** tells new machines to use, e.g. `http://<vpn-address>:8081/api`. || `COLLECTOR_IMAGE` / `DOCKER_PROXY_IMAGE` | empty | Digest-pinned images remote machines install. **Connect a server** needs both. || `JOIN_TOKEN_TTL_SECONDS` | `900` | How long a generated join token stays valid. || `COLLECTOR_ACCESS` | `open` (dev), `strict` (server file) | `strict` makes the backend answer collector endpoints only when they came through the collector gate. |diff --git a/docs/deploy-traefik.md b/docs/deploy-traefik.mdindex 8f5657e..55d128f 100644--- a/docs/deploy-traefik.md+++ b/docs/deploy-traefik.md@@ -47,11 +47,7 @@ Find the subnet of Traefik's network. Replace `traefik_default` if your networkdocker network inspect traefik_default --format '{{range .IPAM.Config}}{{.Subnet}}{{end}}'```-It prints one range, for example:--```text-172.18.0.0/16-```+It prints one range, an address followed by `/` and a number. That is `<traefik-subnet>` below.Open `.env` and set these lines. Leave every other line empty.@@ -60,13 +56,15 @@ COMPOSE_FILE=docker-compose.server.yml:deploy/traefik.ymlPOSTGRES_PASSWORD=<output of: openssl rand -hex 32>REDIS_PASSWORD=<output of: openssl rand -hex 24>DASHBOARD_DOMAIN=<your-domain, without https://>-TRUSTED_PROXIES=<the subnet printed above>+TRUSTED_PROXIES=<traefik-subnet>TRAEFIK_ENTRYPOINT=<your HTTPS entrypoint, or empty for websecure>TRAEFIK_CERT_RESOLVER=<your certificate resolver, or empty for letsencrypt>TRAEFIK_NETWORK=<Traefik's network, or empty for traefik_default>```- `COMPOSE_FILE` makes every plain `docker compose ...` command use both files.+- `TRUSTED_PROXIES` is the range printed above, not the network's name: a name stops the+ frontend from starting.- `TRUSTED_PROXIES` lets Traefik tell Infradeck each visitor's real address. Without it all visitors shareTraefik's address, and one person's failed sign-ins lock everyone out for a minute. Every container on thatnetwork is trusted too, so keep only containers you trust on Traefik's network.@@ -161,20 +159,38 @@ Then open **Connect a server** in the dashboard and follow [Adding servers](addi## Upgrade an existing install-Back up the database first. The backend updates the schema when it starts:+Find the folder the running stack was started from:++```bash+docker compose ls+```++```text+NAME STATUS CONFIG FILES+infradeck running(8) /home/<user>/infradeck/docker-compose.server.yml,/home/<user>/infradeck/deploy/traefik.yml+```++Run every command below in that folder. Run from another clone, `docker compose` starts a second project with+empty volumes instead of upgrading yours. To move the install to another folder, see+[Move an install to another folder](#move-an-install-to-another-folder).++Back up the database. The backend updates the schema when it starts:```bashdocker compose exec -T postgres pg_dump -U infradeck -d infradeck -Fc > infradeck-before-upgrade.dump+docker compose exec -T postgres pg_restore --list < infradeck-before-upgrade.dump | head -3```-Use your `POSTGRES_USER` and `POSTGRES_DB` instead of `infradeck` if you set them.+The second command reads the backup back; it prints `; Archive created at ...`. Use your `POSTGRES_USER` and+`POSTGRES_DB` instead of `infradeck` if you set them. Keep the `-T`: with a terminal attached (`docker exec -t`)+the dump is corrupted.**From a version with the Traefik labels in `docker-compose.server.yml`:** the labels moved to`deploy/traefik.yml`. Set these in `.env` before you pull, or Traefik stops serving the dashboard:```bashCOMPOSE_FILE=docker-compose.server.yml:deploy/traefik.yml-TRUSTED_PROXIES=<subnet of Traefik's network, see step 2>+TRUSTED_PROXIES=<traefik-subnet, see step 2>```The dashboard's local port is `127.0.0.1:5173` again. Set `FRONTEND_PORT` if something on the server relies on@@ -212,6 +228,44 @@ docker compose exec -T postgres pg_restore -U infradeck -d infradeck --clean --idocker compose up -d```+### Move an install to another folder++The volumes of a Compose project are named after it, for example `homelab-agent_postgres_data` for a stack+started from a folder named `homelab-agent`. Docker can't rename volumes, so copy them into volumes for the new+folder. The old volumes stay as they are, for rolling back.++1. In the new clone, make `.env` match the old one: the same `POSTGRES_*`, `REDIS_PASSWORD` and `SECRETS_KEY`,+ or the database refuses the backend and two-factor sign-in stops working. Add the project name, so it no+ longer depends on the folder's name, and the two Traefik settings from the upgrade above:+ ```bash+ COMPOSE_PROJECT_NAME=infradeck+ COMPOSE_FILE=docker-compose.server.yml:deploy/traefik.yml+ TRUSTED_PROXIES=<traefik-subnet, see step 2>+ ```+2. Back up the database in the old folder, as in [Upgrade an existing install](#upgrade-an-existing-install).+3. Stop the old stack. The dashboard is down until step 5. Don't add `-v`, which deletes the volumes:+ ```bash+ cd <old-folder>+ docker compose down+ ```+4. Copy the four data volumes, with `<old-project>` the name `docker compose ls` showed. The Docker proxy's+ socket volume needs no copy.+ ```bash+ for v in postgres_data redis_data collector_state system_events; do+ docker volume create --label com.docker.compose.project=infradeck --label com.docker.compose.volume=$v infradeck_$v+ docker run --rm -v <old-project>_$v:/from:ro -v infradeck_$v:/to --entrypoint sh postgres:16-alpine -c 'cp -a /from/. /to/'+ done+ ```+ The labels mark the new volumes as the `infradeck` project's own, so Compose uses them without a warning.+5. In the new folder, start the stack:+ ```bash+ docker compose up -d --build+ ```+ Check it as in [Verify](#verify). Accounts, history and the dashboard host's server are all still there.++To go back, run `docker compose down` in the new folder and `docker compose up -d` in the old one. Delete the+old volumes (`docker volume rm <old-project>_postgres_data ...`) only once the new install has run for a while.+## Troubleshooting| Symptom | Cause | Fix |@@ -222,7 +276,8 @@ docker compose up -d| Certificate warning in the browser | `TRAEFIK_CERT_RESOLVER` doesn't match a resolver in Traefik, so Traefik serves its default certificate. | Fix the name, then `docker compose up -d`. Traefik's log shows any ACME error. || Signed out right after signing in | The dashboard was opened over `http://`, or `SESSION_COOKIE_SECURE` was set to `false`. | Use `https://<your-domain>` and leave `SESSION_COOKIE_SECURE` empty. || Everyone is locked out after a few failed sign-ins | `TRUSTED_PROXIES` is empty or wrong, so all visitors share Traefik's address. | Set it as in step 2, then `docker compose up -d`. |-| The frontend restarts over and over | `TRUSTED_PROXIES` contains something other than IP addresses and CIDR ranges. | `docker compose logs frontend` names the bad value. Fix it in `.env`. |+| The frontend restarts over and over | `TRUSTED_PROXIES` contains something other than IP addresses and CIDR ranges, such as the network's name. | `docker compose logs frontend` names the bad value. Put the subnet from step 2 in `.env`, then `docker compose up -d`. |+| `service "postgres" is not running`, although the dashboard works | The stack runs from another folder, so this folder is a different Compose project. | Find the folder with `docker compose ls` and run the command there. || The frontend restarts over and over with `mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)` | The stack was updated without `--build`, so an old frontend image runs under the new settings. | `docker compose up -d --build`. |More symptoms: [Troubleshooting](troubleshooting.md).diff --git a/docs/deployment.md b/docs/deployment.mdindex a588c02..c87b0fd 100644--- a/docs/deployment.md+++ b/docs/deployment.md@@ -94,9 +94,9 @@ Collectors reach the **collector gate**, not the public domain. In `.env`:| Setting | Example | Meaning ||---|---|---|-| `COLLECTOR_BIND` | `10.8.0.1` | Your VPN server's address on the dashboard machine (for OpenVPN, the `tun` address). The gate's port is published on this address only. |+| `COLLECTOR_BIND` | `<vpn-address>` | Your VPN server's address on the dashboard machine (for OpenVPN, the `tun` address). The gate's port is published on this address only. || `COLLECTOR_PORT` | `8081` | Port of the gate on that address. |-| `COLLECTOR_PUBLIC_URL` | `http://10.8.0.1:8081/api` | What **Connect a server** tells new machines to use. |+| `COLLECTOR_PUBLIC_URL` | `http://<vpn-address>:8081/api` | What **Connect a server** tells new machines to use. || `COLLECTOR_IMAGE`, `DOCKER_PROXY_IMAGE` | `ghcr.io/…@sha256:…` | The images remote machines install; see below. |Things to get right:diff --git a/example.env b/example.envindex a5f5677..a6ea815 100644--- a/example.env+++ b/example.env@@ -68,7 +68,7 @@ SECRETS_KEY=# (both) The address or network of the reverse proxy in front of the dashboard (Traefik, Caddy, nginx,# HAProxy, a tunnel connector...), so sign-in and registration limits count each client by its real address.# Only these addresses may say who the client is (X-Forwarded-For); anyone else could pretend to be a new-# client on every request. Several values are separated by commas: 172.18.0.0/16,10.8.0.1+# client on every request. Several values are separated by commas, e.g. <proxy-subnet>,<proxy-address># - Proxy in Docker (Traefik, Caddy, ...): the subnet of the network it shares with the dashboard:# docker network inspect <network> --format '{{range .IPAM.Config}}{{.Subnet}}{{end}}'# - Proxy on the host itself: the gateway of the dashboard's network (traffic from the host arrives from it):@@ -168,12 +168,12 @@ JOIN_TOKEN_TTL_SECONDS=# =============================================================================# (server) Where the collector gate is published. Bind it to your VPN address only (for OpenVPN the-# server's tun address, e.g. 10.8.0.1): Docker's published ports bypass ufw and similar firewalls.+# server's tun address): Docker's published ports bypass ufw and similar firewalls.# Defaults: 127.0.0.1 and 8081COLLECTOR_BIND=COLLECTOR_PORT=-# (both) The address "Connect a server" tells new machines to report to, e.g. http://10.8.0.1:8081/api+# (both) The address "Connect a server" tells new machines to report to, e.g. http://<vpn-address>:8081/apiCOLLECTOR_PUBLIC_URL=# (both) Images remote machines install, pinned by digest. Push a version tag (git tag v0.1.0 &&@@ -262,7 +262,7 @@ ENABLE_API_DOCS=# Collector-only install (collector-compose.yml on a machine without the dashboard)# =============================================================================-# REQUIRED for collector-compose.yml. The dashboard's API address, e.g. http://10.8.0.1:8081/api+# REQUIRED for collector-compose.yml. The dashboard's API address, e.g. http://<vpn-address>:8081/apiINFRADECK_API_URL=# Image to run instead of building one locally. Default: infradeck-collector:latest