Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.
Python ai-agentsaiopsdashboarddockerfastapihomelabincident-managementllmmonitoringopenaireactroot-cause-analysisself-hosteduptime-monitoring
Troubleshooting
| Symptom | Check |
|---|---|
| A new machine never shows up under New servers detected | docker logs infradeck-collector on that machine. "Could not reach …" means the address in the command isn't reachable from it (firewall, localhost, wrong IP/domain). |
| Collector log says "Backend no longer accepts server …" | The server was deleted on the dashboard; it is asking to join again, so click Add or Dismiss. |
network traefik_default declared as external, but could not be found |
You're using deploy/traefik.yml on a machine without that Traefik network. Set TRAEFIK_NETWORK in .env to the network Traefik is really on, or, without Traefik, drop deploy/traefik.yml from COMPOSE_FILE. |
Frontend keeps restarting; docker compose logs frontend shows mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied) |
Its image is older than the compose file. Run docker compose pull && docker compose up -d (when building from source with deploy/build.yml: docker compose up -d --build). |
| Dashboard shows old behaviour after an update or a code change | After checking out a new release: docker compose pull && docker compose up -d. After a code change (development stack, or deploy/build.yml on a server): rebuild all services with docker compose up -d --build. Then hard-refresh the browser (Ctrl+Shift+R). |
| "The machine running the dashboard isn't reporting yet" at the top | The stack's own collector hasn't checked in: add it under New servers detected, then wait one check interval. If it is reporting, its image is outdated: docker exec infradeck-collector grep -c dashboard_host /app/collector.py prints 0. Pull the current one (docker compose pull collector && docker compose up -d collector), or with deploy/build.yml rebuild it (docker compose build --no-cache collector && docker compose up -d collector). |
A server shows only / and no Storage bars for other disks |
Its collector predates per-mount reporting. Install a newer collector image (run a freshly generated command, or docker compose -f collector-compose.yml up -d --build). A disk mounted after the collector started appears after a collector restart. |
| Logged out after opening the dashboard at a new address | Login cookies belong to the address they were set on; log in again. |
Login fails behind HTTPS with SESSION_COOKIE_SECURE=true |
The cookie is only sent over HTTPS; open the https:// address, not http://…:5173. |
| Backend started outside the compose files: login "works" but you're signed out at once, and collectors get 404 | The backend's defaults are the safe ones (SESSION_COOKIE_SECURE=true, COLLECTOR_ACCESS=strict). Serve it over HTTPS behind the collector gate, or for a plain-HTTP, single-listener setup set SESSION_COOKIE_SECURE=false and COLLECTOR_ACCESS=open. |
| Collector log says "Registration refused (HTTP 403): The join token is invalid, expired or already used" | The token in the command has expired, was already used, or is wrong. Click Generate command again and run the new command. |
| Connect a server says no collector images are configured | Set COLLECTOR_IMAGE and DOCKER_PROXY_IMAGE in .env (see Publishing the collector images) and rebuild. |
| The register page asks for a setup token | The install has no accounts yet. Use docker compose logs backend | grep "setup token", or ADMIN_BOOTSTRAP_TOKEN. |
| "Password must be at least 16 characters long" | New passwords need 16 or more characters (at most 72 bytes). A few random words work well. |
| "Cross-site request refused" (403) on a button or form | The request came from another origin. Open the dashboard at the address you normally use. If a proxy in front rewrites the Host header, add your dashboard URL to CORS_ORIGINS. |
| The System row shows unknown for last shutdown or has no kernel messages | The optional system helper isn't running. Start it with docker compose --profile system-events up -d. Unknown for reboot-required on non-Debian hosts is expected. |
Error response … cannot assign requested address for the collector gate |
COLLECTOR_BIND is set to an address that doesn't exist yet (VPN not up). Start the VPN first, or fix the address. |
| The Network row says no config found for the firewall | None of the known firewall files exist on that host (/etc/ufw, /etc/nftables.conf, /etc/iptables/rules.v4). The check reads configuration, not the live ruleset, so a host that configures its firewall some other way shows this. |
| The Network row says not checked for the internet | The check is opt-in. Set INFRADECK_CONNECTIVITY_TARGETS on that collector (for example 1.1.1.1:443). |
| Listeners show unknown or the Network row is empty | The collector couldn't read the host's network tables (older collector, or a table with more than 500,000 sockets). An unreadable report never raises a "port closed" alert. |
| "Invalid code." although the code looks right | Codes depend on the clock: check the phone's time is automatic. A code also can't be used twice, so wait for the next one. Five wrong codes lock the code step for 5 minutes. |
| Two-factor setup says it is not available | SECRETS_KEY is missing or invalid in .env. Generate one (see Two-factor sign-in) and restart the backend. The backend log shows a SECURITY WARNING while the key is missing. |
| Lost the authenticator and the recovery codes | Run docker compose exec backend python src/backend/reset_2fa.py <email> on the host, then sign in with the password and set it up again. |
Authenticator codes stopped working after a restore or .env change |
SECRETS_KEY changed or was lost, so the stored secret can't be decrypted. Restore the old key (rotation: new,old), or use a recovery code and the reset command. |
| A collector log says "INFRADECK_API_URL redirects to …; set it to the final address" | Something in front of the dashboard's API answers with a redirect: an http:// address that redirects to https://, or a sign-in page (Authelia, Authentik …) in front of /api. The collector never follows it, so its token can't go anywhere else. Set INFRADECK_API_URL to the address it names (with /api), or let collectors reach /api without the sign-in page. |
| Server marked offline | It missed 3 check-ins in a row. Check the machine, its collector log and its route to the dashboard. The incident resolves on the next report. |
Set REDIS_PASSWORD in .env when starting the stack |
Add a REDIS_PASSWORD to .env (echo "REDIS_PASSWORD=$(openssl rand -hex 24)" >> .env). Redis holds the login sessions, so it has a password; you are signed out once after adding it. |
The backend log says AuthenticationError or invalid password for Redis, or /health shows Redis down |
REDIS_PASSWORD in .env doesn't match the password Redis was started with, or contains characters that don't fit in a URL (@, :, /, %). Use letters and digits only, and recreate both containers: docker compose up -d --force-recreate redis backend. |
| A collector log says "Check-ins are arriving faster than this server's interval allows" (HTTP 429) | The collector reports more often than a quarter of the server's check interval. It retries by itself; check that only one collector uses that server's token, or adjust INGEST_MIN_SPACING_RATIO. |
| Enrolling gives HTTP 429 "Too many attempts" | The enrollment key is limited to ENROLL_RATE_LIMIT (30) enrollments per address per hour. If many collectors reach the dashboard from one address (NAT), raise it in .env. |
A new server shows up as name (2) |
Another server already had that name (often a default such as raspberrypi). The number tells the two apart. Give one machine a different hostname (or INFRADECK_SERVER_NAME) if you want to avoid it. |
| "This install already has the most servers it allows" | MAX_SERVERS (default 200) is reached. Remove servers you no longer need, or raise it in .env. |
| A warning "The credentials of were replaced" | Someone used the enrollment key on a machine whose ID is already known, which signs the previous collector out. Expected after you reinstall that machine with the key; if it wasn't you, rotate the key under Connect a server → Advanced and inspect the machine. |
| "Enter your current password to set up two-factor." | Turning on two-factor from a normal session asks for your password again. Type it in the field that appears. |