Skip to content

Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.

fix(collector): never follow a redirect with the token and cap the dashboard's answers (#218)

Karlo Kajba Šimanić60b5d172f1ef
diff --git a/SECURITY.md b/SECURITY.md
index 76d4deb..0d89e90 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -322,6 +322,9 @@ check results) don't exist on the public side at all.
without them. The GHCR packages must be public for hosts to pull them without credentials.
- **The VPN is yours to run.** Collector traffic over the VPN listener is plain HTTP inside the
tunnel. Collectors authenticate with a per-server token that is stored hashed on the dashboard.
+ A collector never follows a redirect from the dashboard's address, so a sign-in page or another proxy in
+ front of the API can't make it send its token somewhere else; it logs the address it was sent to and
+ waits for `INFRADECK_API_URL` to be fixed. It reads at most 1 MiB of any answer.
What this still depends on:
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
index 6f1ab8b..0efb268 100644
--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@@ -27,6 +27,7 @@
| Two-factor setup says it is not available | `SECRETS_KEY` is missing or invalid in `.env`. Generate one (see [Two-factor sign-in](accounts.md#two-factor-sign-in)) and restart the backend. The backend log shows a SECURITY WARNING while the key is missing. |
| Lost the authenticator and the recovery codes | Run `docker compose exec backend python src/backend/reset_2fa.py <email>` on the host, then sign in with the password and set it up again. |
| Authenticator codes stopped working after a restore or `.env` change | `SECRETS_KEY` changed or was lost, so the stored secret can't be decrypted. Restore the old key (rotation: `new,old`), or use a recovery code and the reset command. |
+| A collector log says "INFRADECK_API_URL redirects to …; set it to the final address" | Something in front of the dashboard's API answers with a redirect: an `http://` address that redirects to `https://`, or a sign-in page (Authelia, Authentik …) in front of `/api`. The collector never follows it, so its token can't go anywhere else. Set `INFRADECK_API_URL` to the address it names (with `/api`), or let collectors reach `/api` without the sign-in page. |
| Server marked offline | It missed 3 check-ins in a row. Check the machine, its collector log and its route to the dashboard. The incident resolves on the next report. |
| `Set REDIS_PASSWORD in .env` when starting the stack | Add a `REDIS_PASSWORD` to `.env` (`echo "REDIS_PASSWORD=$(openssl rand -hex 24)" >> .env`). Redis holds the login sessions, so it has a password; you are signed out once after adding it. |
| The backend log says `AuthenticationError` or `invalid password` for Redis, or `/health` shows Redis down | `REDIS_PASSWORD` in `.env` doesn't match the password Redis was started with, or contains characters that don't fit in a URL (`@`, `:`, `/`, `%`). Use letters and digits only, and recreate both containers: `docker compose up -d --force-recreate redis backend`. |
diff --git a/src/collector/collector.py b/src/collector/collector.py
index 3e3c7ce..abffd6e 100644
--- a/src/collector/collector.py
+++ b/src/collector/collector.py
@@ -1756,6 +1756,32 @@ def run_check(check: dict[str, Any]) -> dict[str, Any]:
return {"check_id": check["id"], "time": datetime.now(timezone.utc).isoformat(), **{key: value for key, value in outcome.items() if value is not None or key == "ok"}}
+class _ApiRedirect(urllib.error.URLError):
+ """The dashboard's address answered with a redirect, which calls to it never follow."""
+
+
+class _NoRedirects(urllib.request.HTTPRedirectHandler):
+ def redirect_request(self, req, fp, code, msg, headers, newurl): # noqa: ANN001
+ target = urllib.parse.urlsplit(urllib.parse.urljoin(req.full_url, newurl))
+ raise _ApiRedirect(f"INFRADECK_API_URL redirects to {target.scheme}://{target.netloc}; set it to the final address")
+
+
+# Calls to the dashboard carry this collector's token, so they never follow a redirect: a proxy in front of the API
+# (a sign-in page, an http-to-https redirect) would otherwise get the token sent to wherever it points (#205).
+_API_OPENER = urllib.request.build_opener(_NoRedirects())
+
+
+def _api(request: urllib.request.Request) -> dict[str, Any]:
+ """Send ``request`` to the dashboard and read its JSON answer, at most MAX_BODY_BYTES of it."""
+ if urllib.parse.urlsplit(request.full_url).scheme not in ("http", "https"):
+ raise ValueError("INFRADECK_API_URL must start with http:// or https://")
+ with _API_OPENER.open(request, timeout=15) as response:
+ body = response.read(MAX_BODY_BYTES + 1)
+ if len(body) > MAX_BODY_BYTES:
+ raise ValueError("The dashboard's answer is too large")
+ return json.loads(body.decode())
+
+
def send_check_results(api_url: str, server_id: int, token: str, results: list[dict[str, Any]], tests: list[dict[str, Any]] | None = None) -> dict[str, Any]:
request = urllib.request.Request(
f"{api_url.rstrip('/')}/monitoring/metrics/checks",
@@ -1763,8 +1789,7 @@ def send_check_results(api_url: str, server_id: int, token: str, results: list[d
headers={"Content-Type": "application/json", "Authorization": f"Bearer {token}"},
method="POST",
)
- with urllib.request.urlopen(request, timeout=15) as response:
- return json.loads(response.read(MAX_BODY_BYTES).decode())
+ return _api(request)
MAX_TESTS = 5
@@ -1879,8 +1904,7 @@ def send_payload(api_url: str, server_id: int, token: str, payload: dict[str, An
headers={"Content-Type": "application/json", "Authorization": f"Bearer {token}"},
method="POST",
)
- with urllib.request.urlopen(request, timeout=15) as response:
- return json.loads(response.read().decode())
+ return _api(request)
def _machine_id() -> str | None:
@@ -1939,8 +1963,7 @@ def enroll(api_url: str, enroll_key: str) -> tuple[int, str]:
headers={"Content-Type": "application/json"},
method="POST",
)
- with urllib.request.urlopen(request, timeout=15) as response:
- data = json.loads(response.read().decode())
+ data = _api(request)
return int(data["server_id"]), str(data["collector_token"])
@@ -1949,8 +1972,7 @@ def _post_json(url: str, payload: dict[str, Any], bearer: str | None = None) ->
if bearer:
headers["Authorization"] = f"Bearer {bearer}"
request = urllib.request.Request(url, data=json.dumps(payload).encode(), headers=headers, method="POST")
- with urllib.request.urlopen(request, timeout=15) as response:
- return json.loads(response.read().decode())
+ return _api(request)
def _error_detail(exc: urllib.error.HTTPError) -> str:
diff --git a/tests/collectors/test_collector_checks.py b/tests/collectors/test_collector_checks.py
index 60afe4d..29cb030 100644
--- a/tests/collectors/test_collector_checks.py
+++ b/tests/collectors/test_collector_checks.py
@@ -321,3 +321,72 @@ def test_validate_check_rejects_credentials_in_a_url():
base = {"id": 1, "interval": 60, "retry_interval": 20, "timeout": 10, "accepted_codes": ["200-299"]}
assert collector.validate_check({**base, "type": "http", "target": "https://user:pass@host/"}) is None
assert collector.validate_check({**base, "type": "http", "target": "https://host/?token=x"}) is not None
+
+
+# --- calls to the dashboard (#205) ------------------------------------------------------------------------------
+
+def _serve(handler):
+ server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), handler)
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ return server, f"http://127.0.0.1:{server.server_address[1]}"
+
+
+def test_a_call_to_the_dashboard_never_follows_a_redirect_with_the_token():
+ seen = []
+
+ class Elsewhere(http.server.BaseHTTPRequestHandler):
+ def do_GET(self): # noqa: N802
+ seen.append(self.headers.get("Authorization"))
+ self.send_response(200)
+ self.end_headers()
+
+ do_POST = do_GET
+
+ def log_message(self, *args):
+ pass
+
+ elsewhere, elsewhere_url = _serve(Elsewhere)
+
+ class SignInRedirect(http.server.BaseHTTPRequestHandler):
+ def do_POST(self): # noqa: N802
+ self.send_response(302)
+ self.send_header("Location", f"{elsewhere_url}/login?rd=secret-return-path")
+ self.end_headers()
+
+ def log_message(self, *args):
+ pass
+
+ proxy, proxy_url = _serve(SignInRedirect)
+ try:
+ with pytest.raises(urllib.error.URLError) as raised:
+ collector.send_payload(proxy_url, 1, "collector-token", {})
+ assert seen == [] # the token never reached the other host
+ assert str(raised.value.reason) == f"INFRADECK_API_URL redirects to {elsewhere_url}; set it to the final address" # no path or query
+ finally:
+ proxy.shutdown()
+ elsewhere.shutdown()
+
+
+def test_a_huge_answer_from_the_dashboard_is_refused(monkeypatch):
+ class Huge(http.server.BaseHTTPRequestHandler):
+ def do_POST(self): # noqa: N802
+ body = b"[" + b"0," * collector.MAX_BODY_BYTES + b"0]"
+ self.send_response(200)
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ try:
+ self.wfile.write(body)
+ except OSError: # the collector stops reading at its limit
+ pass
+
+ def log_message(self, *args):
+ pass
+
+ server, url = _serve(Huge)
+ try:
+ with pytest.raises(ValueError, match="too large"):
+ collector.send_payload(url, 1, "collector-token", {})
+ with pytest.raises(ValueError, match="http:// or https://"):
+ collector.send_payload("file:///etc", 1, "collector-token", {})
+ finally:
+ server.shutdown()