Skip to content

Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.

fix(security): second review findings (S2-01..S2-08)

karloks2005d17d8414b04a
diff --git a/SECURITY.md b/SECURITY.md
index 1125288..371bae2 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -177,7 +177,9 @@ the public side at all.
known, which is what makes reinstalling easy. Because that key is long-lived, using it is rate limited
(per address and per key), the number of servers is capped (`MAX_SERVERS`, default 200), and replacing an
existing server's credentials is recorded: it appears in the server's System row, in the audit log, and
- opens a **warning** for an hour, with advice to rotate the key if you didn't do it. Treat the key like a
+ opens a **warning** for an hour, with advice to rotate the key if you didn't do it. That warning and the
+ System-row entry are exempt from the "few per server" limits on the dashboard, so whoever holds the new
+ credentials can't crowd them out with what the server reports. Treat the key like a
password and rotate it under *Connect a server → Advanced* if a host that holds it is compromised.
- **No code comes from the dashboard.** The dashboard no longer serves `collector.py`. Remote
hosts install the collector and the Docker proxy from container images you publish (the "Publish
@@ -201,13 +203,17 @@ What this still depends on:
A compromised collector controls everything it reports, so the backend never trusts any of it. For the host
itself, every number is range-checked (percentages 0 to 100, loads, sizes and counters within sane limits; NaN,
infinity, text and booleans become "unknown"), so one bad value can't make pages fail for everyone. Server
-descriptions (hostname, address, operating system, core count, memory) are checked too, and a hostname that
-another server already uses is ignored, so one collector can't pose as another. Only the stack's own collector,
+descriptions (hostname, address, operating system, core count, memory) are checked too. A hostname is
+compared with other servers' names ignoring case, accents and width, a name that mixes Latin letters with a
+lookalike alphabet (Cyrillic or Greek, the usual way to impersonate another server) is refused, a
+rename to a name another server uses is ignored, and a new server whose name is taken gets a number
+(`raspberrypi (2)`), so two machines can always be told apart. Only the stack's own collector,
which reaches the backend through the gate's internal listener, can claim to be the dashboard's machine. At most
-64 filesystems are accepted per check-in, check-ins must be spaced by a quarter of the check interval
+64 filesystems are accepted per check-in (names up to 200 characters), kernel-log events from the system helper at most 20 per check-in and 500 per server per week, check-ins must be spaced by a quarter of the check interval
(refused with 429 otherwise, and checked only after the token is verified, so nobody else can use up a
-server's slot), and the alerts list on the dashboard shows at most ten per server, so one noisy server can't
-push everyone else's off it.
+server's slot), and the alerts list on the dashboard shows at most ten per server and the System row ten events per server
+(chosen by the database, so thousands of stored events cost no more to show), so one noisy server can't
+push everyone else's off. A date a little in the future is treated as now.
For containers, each record is reduced to known fields of the right type (text made printable and capped, numbers
range-checked, timestamps parsed strictly, image IDs matched against `sha256:` plus 64 hex characters),
@@ -266,6 +272,11 @@ The collector's network checks are passive or opt-in, and add no privileges:
talks to the backend directly.
- **Redis has a password** (`REDIS_PASSWORD`, required). Without it, anything that could reach Redis could write
a session for any user, administrator included, with no password or two-factor.
+- **Limits on what a signed-in user can run up.** Messages to the assistant (30 an hour per user), new
+ conversations (30 an hour, 100 in total) and audit-log entries (120 an hour) are limited, since each can cost
+ the administrator's Gemini quota or database space. Setting up two-factor from a normal session needs the
+ password again, and every try counts toward the lockout, the right password included. A limited
+ (set-up-only) session lasts 30 minutes, not a day.
- **Sessions are stored hashed.** Redis holds only the SHA-256 of the cookie value, in the session key, the
per-user index and the pending-login token. Reading Redis (a backup, a leaked dump) doesn't hand out working
sessions. Note what this does and doesn't do: hashing protects against *reading* Redis. It does not stop
diff --git a/TESTS.md b/TESTS.md
index e365d2d..ec77ec4 100644
--- a/TESTS.md
+++ b/TESTS.md
@@ -4,7 +4,7 @@ Vexion ships with an automated test suite. This page explains how to run it, wha
covers (and what it doesn't), and what every single test is for, so you can decide
whether a failure matters before you upgrade or change your setup.
-- **1041 test cases** from 529 test functions, plus 16 small tests of the chart helpers (see [Frontend helper tests](#frontend-helper-tests)) (some functions run once per input).
+- **1085 test cases** from 555 test functions, plus 16 small tests of the chart helpers (see [Frontend helper tests](#frontend-helper-tests)) (some functions run once per input).
- **About two minutes** to run. No Docker, PostgreSQL, Redis, network or root access needed.
- **Safe to run next to a live install.** The tests use a throwaway in-memory database
and fakes; they never touch your running containers or your data. (Two collector
@@ -147,6 +147,7 @@ docker compose logs --tail 20 collector # "Sent host metrics and N contai
| [`test_pending_collectors.py`](#test_pending_collectorspy) | 28 | Adding servers: approval, join tokens, strict mode, re-pairing safety. |
| [`test_rollups.py`](#test_rollupspy) | 31 | Summarised history: hourly, daily, weekly and monthly rollups, catching up, gaps, and purging only what has been summarised. |
| [`test_rules.py`](#test_rulespy) | 20 | Alert thresholds: sustained breaches, per-mount disks, settings validation. |
+| [`test_second_review.py`](#test_second_reviewpy) | 44 | The second security review's findings: helper-event caps, lockouts, chat limits, lookalike hostnames, odd input. |
| [`test_stats.py`](#test_statspy) | 46 | Fleet statistics (incidents over time, availability) and the busy-hours heatmap: grouping, ranges, caching and validation. |
| [`test_system_checks.py`](#test_system_checkspy) | 17 | Reboot, OOM, kernel-fault, clock and pending-reboot detection, and the System row's event list. |
| [`test_system_helper.py`](#test_system_helperpy) | 35 | The optional system helper: kernel-log parsing, `wtmp`, crash detection, hostile input. |
@@ -850,6 +851,39 @@ The alert thresholds, and the settings that control them.
| `test_corrupt_mount_thresholds_are_ignored` | Damaged per-mount settings are ignored. |
| `test_overview_shows_latest_mounts_and_disk_thresholds` | The dashboard shows the latest filesystems and the thresholds used to colour them. |
+### test_second_review.py
+
+Findings of the second security review (S2-01 to S2-07) and its smaller points: each test fails without its fix.
+
+| Test | What it checks |
+|---|---|
+| `test_one_check_in_stores_at_most_twenty_helper_events` | 50 helper events in one check-in: 20 are stored (S2-01). |
+| `test_the_rest_is_stored_by_later_check_ins_because_the_helper_republishes_its_buffer` | The same 50 offered again later: the rest are stored over the next check-ins. |
+| `test_a_flood_of_new_helper_events_stops_at_a_weekly_cap_and_says_so_once` | A flood of new events stops at 500 per server per week with a single notice; another server is unaffected. |
+| `test_the_overview_shows_a_few_events_per_server_whatever_is_stored` | With 3000 stored events the dashboard still shows only the newest 10 per server. |
+| `test_the_replaced_credentials_event_is_always_shown_even_among_many_newer_events` | The System-row entry about replaced credentials can't be pushed out by newer events (S2-03). |
+| `test_the_replaced_credentials_alert_is_always_listed_even_beside_a_flood_of_critical_ones` | The warning about replaced credentials is always listed, beside the usual 10 per server. |
+| `test_a_helper_event_dated_in_the_future_is_stored_as_happening_now` | An event dated a few minutes ahead is stored as happening now. |
+| `test_the_right_password_is_refused_once_the_two_factor_lockout_is_reached` | After 5 wrong passwords at two-factor setup, even the right one gets 429 and nothing is created (S2-02). |
+| `test_a_correct_password_below_the_limit_still_works_and_clears_the_count` | A right password below the limit works and starts the count again. |
+| `test_chat_messages_are_limited_per_user_per_hour` | The 31st assistant message in an hour gets 429 (S2-04). |
+| `test_a_user_can_only_hold_so_many_conversations` | A user can't hold more than the allowed number of conversations. |
+| `test_creating_conversations_is_rate_limited_too` | Creating conversations is limited per hour. |
+| `test_a_lookalike_of_another_servers_name_is_ignored` *(4 inputs)* | Renaming to a lookalike (Cyrillic or full-width letters) of another server's name is ignored (S2-05). |
+| `test_a_name_that_mixes_alphabets_is_ignored_even_when_nobody_else_has_it` | A name mixing Latin and Cyrillic letters is ignored. |
+| `test_names_in_one_alphabet_are_fine` | Names in a single alphabet (Croatian letters, all-Cyrillic) are accepted. |
+| `test_a_second_server_with_an_existing_name_gets_a_number_so_the_two_can_be_told_apart` | A new server whose name is taken is numbered (`raspberrypi (2)`, `(3)`), ignoring case. |
+| `test_a_mount_name_is_short_enough_for_every_incident_key_built_from_it` | A 255-character mount name is cut to 200 and no incident key exceeds 255 (S2-06). |
+| `test_unusual_values_in_a_check_in_are_ignored_not_a_server_error` *(10 inputs)* | Lists, objects, NUL characters and huge numbers in status fields give 200, not 500 (S2-07). |
+| `test_a_nul_character_in_the_boot_id_is_not_stored` | A NUL character in the boot ID is not stored. |
+| `test_ids_must_fit_what_the_database_holds` *(5 inputs)* | Server IDs of 0, negative or beyond the database range, and conversation IDs over 64 characters, give 422. |
+| `test_server_ids_in_the_address_must_fit_too` *(3 inputs)* | The same for server IDs in the address of history and statistics routes. |
+| `test_the_assistants_memory_of_chats_is_bounded_and_forgets_the_least_recently_used` | The assistant's in-memory chats are bounded and the least recently used is dropped. |
+| `test_a_user_cannot_write_the_log_entries_the_system_counts` | `/logs` refuses reserved event types such as `health_check_completed`. |
+| `test_a_limited_session_expires_sooner_than_a_full_one` | A set-up-only session lasts at most 30 minutes, a full one a day. |
+| `test_collector_credentials_are_created_private_from_the_start` | The collector's credentials file is created with mode 0600 and no temporary file is left. |
+| `test_the_enrollment_limit_is_configurable` | `ENROLL_RATE_LIMIT` is read from the settings. |
+
### test_stats.py
The all-servers statistics and the busy-hours heatmap behind the Statistics page.
diff --git a/compose.yml b/compose.yml
index 61203d7..bcd368a 100644
--- a/compose.yml
+++ b/compose.yml
@@ -31,7 +31,7 @@ services:
volumes:
- redis_data:/data
healthcheck:
- test: ["CMD-SHELL", "redis-cli -a \"$$REDIS_PASSWORD\" --no-auth-warning ping | grep -q PONG"]
+ test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$REDIS_PASSWORD\" redis-cli ping | grep -q PONG"]
interval: 10s
timeout: 5s
retries: 5
@@ -59,6 +59,7 @@ services:
# previous one (0 switches it off); the install holds at most MAX_SERVERS; /docs is off unless enabled.
INGEST_MIN_SPACING_RATIO: ${INGEST_MIN_SPACING_RATIO:-0.25}
MAX_SERVERS: ${MAX_SERVERS:-200}
+ ENROLL_RATE_LIMIT: ${ENROLL_RATE_LIMIT:-30}
ENABLE_API_DOCS: ${ENABLE_API_DOCS:-false}
# Setup token for the very first account. Empty: one is generated and printed in the backend log.
ADMIN_BOOTSTRAP_TOKEN: ${ADMIN_BOOTSTRAP_TOKEN:-}
@@ -123,7 +124,7 @@ services:
condition: service_healthy
# The only container that mounts the Docker socket (a mount flag can't make it read-only).
- # It answers three read requests and nothing else. It has no network at all: the collector
+ # It answers four read requests and nothing else. It has no network at all: the collector
# reaches it through a Unix socket in a volume only the two of them share. See SECURITY.md.
docker-proxy:
build:
diff --git a/docker-compose.server.yml b/docker-compose.server.yml
index a619dc6..b7497e3 100644
--- a/docker-compose.server.yml
+++ b/docker-compose.server.yml
@@ -36,7 +36,7 @@ services:
volumes:
- redis_data:/data
healthcheck:
- test: ["CMD-SHELL", "redis-cli -a \"$$REDIS_PASSWORD\" --no-auth-warning ping | grep -q PONG"]
+ test: ["CMD-SHELL", "REDISCLI_AUTH=\"$$REDIS_PASSWORD\" redis-cli ping | grep -q PONG"]
interval: 10s
timeout: 5s
retries: 5
@@ -70,6 +70,7 @@ services:
# previous one (0 switches it off); the install holds at most MAX_SERVERS; /docs is off unless enabled.
INGEST_MIN_SPACING_RATIO: ${INGEST_MIN_SPACING_RATIO:-0.25}
MAX_SERVERS: ${MAX_SERVERS:-200}
+ ENROLL_RATE_LIMIT: ${ENROLL_RATE_LIMIT:-30}
ENABLE_API_DOCS: ${ENABLE_API_DOCS:-false}
# Setup token for the very first account. Empty: one is generated and printed in the backend log.
ADMIN_BOOTSTRAP_TOKEN: ${ADMIN_BOOTSTRAP_TOKEN:-}
@@ -165,7 +166,7 @@ services:
condition: service_healthy
# The only container that mounts the Docker socket (a mount flag can't make it read-only).
- # It answers three read requests and nothing else. It has no network at all: the collector
+ # It answers four read requests and nothing else. It has no network at all: the collector
# reaches it through a Unix socket in a volume only the two of them share. See SECURITY.md.
docker-proxy:
build:
diff --git a/docs/configuration.md b/docs/configuration.md
index 30d8514..24069ff 100644
--- a/docs/configuration.md
+++ b/docs/configuration.md
@@ -16,6 +16,7 @@
| `ALLOW_REGISTRATION` | `false` | Let anyone create an account. When `false` (the default), only the first account can be created. Turning it on prints a security warning at every start. |
| `MAX_SERVERS` | `200` | The most servers the install holds, however they are added (enrollment key, approval or by hand). |
| `INGEST_MIN_SPACING_RATIO` | `0.25` | A check-in must arrive at least this share of the server's check interval (and never sooner than 5 seconds) after the previous one, or it is refused with 429. `0` switches the limit off. |
+| `ENROLL_RATE_LIMIT` | `30` | Enrollments with the enrollment key allowed per address per hour (twice that per key). Raise it if many collectors reach the dashboard from one address (for example behind NAT). |
| `ENABLE_API_DOCS` | `false` | Serves the interactive API documentation at `/api/docs`. Leave off on a public install. |
| `SECRETS_KEY` | empty | Key that encrypts two-factor secrets. Required for two-factor sign-in and for the "require two-factor for admins" policy to be enforced. Generate with `python -m src.backend.secrets_box`; comma-separated keys allow rotation. |
| `ADMIN_BOOTSTRAP_TOKEN` | generated at start | Setup token for the first account. Leave empty to have one generated and printed in the backend log until the first account exists. |
diff --git a/docs/troubleshooting.md b/docs/troubleshooting.md
index 4cb1f61..191a679 100644
--- a/docs/troubleshooting.md
+++ b/docs/troubleshooting.md
@@ -30,6 +30,8 @@
| `Set REDIS_PASSWORD in .env` when starting the stack | Add a `REDIS_PASSWORD` to `.env` (`echo "REDIS_PASSWORD=$(openssl rand -hex 24)" >> .env`). Redis holds the login sessions, so it has a password; you are signed out once after adding it. |
| The backend log says `AuthenticationError` or `invalid password` for Redis, or `/health` shows Redis down | `REDIS_PASSWORD` in `.env` doesn't match the password Redis was started with, or contains characters that don't fit in a URL (`@`, `:`, `/`, `%`). Use letters and digits only, and recreate both containers: `docker compose up -d --force-recreate redis backend`. |
| A collector log says "Check-ins are arriving faster than this server's interval allows" (HTTP 429) | The collector reports more often than a quarter of the server's check interval. It retries by itself; check that only one collector uses that server's token, or adjust `INGEST_MIN_SPACING_RATIO`. |
+| Enrolling gives HTTP 429 "Too many attempts" | The enrollment key is limited to `ENROLL_RATE_LIMIT` (30) enrollments per address per hour. If many collectors reach the dashboard from one address (NAT), raise it in `.env`. |
+| A new server shows up as `name (2)` | Another server already had that name (often a default such as `raspberrypi`). The number tells the two apart. Give one machine a different hostname (or `HOMELAB_SERVER_NAME`) if you want to avoid it. |
| "This install already has the most servers it allows" | `MAX_SERVERS` (default 200) is reached. Remove servers you no longer need, or raise it in `.env`. |
| A warning "The credentials of <server> were replaced" | Someone used the enrollment key on a machine whose ID is already known, which signs the previous collector out. Expected after you reinstall that machine with the key; if it wasn't you, rotate the key under *Connect a server → Advanced* and inspect the machine. |
| "Enter your current password to set up two-factor." | Turning on two-factor from a normal session asks for your password again. Type it in the field that appears. |
diff --git a/example.env b/example.env
index 72b51a2..5ff0793 100644
--- a/example.env
+++ b/example.env
@@ -95,6 +95,10 @@ SESSION_COOKIE_NAME=homie_session
# Serves the interactive API documentation at /api/docs. Leave off on a public install.
# ENABLE_API_DOCS=false
+# Enrollments with the enrollment key per address per hour (twice that per key). Raise it only if many collectors
+# reach the dashboard from one address, for example behind NAT.
+# ENROLL_RATE_LIMIT=30
+
# Setup token for the very first account (the one that becomes the admin).
# Leave it commented: the backend then makes one up and prints it in its log
# ("docker compose logs backend | grep 'setup token'") until the first account exists.
diff --git a/src/agent/collector.py b/src/agent/collector.py
index 30673d7..62ea426 100644
--- a/src/agent/collector.py
+++ b/src/agent/collector.py
@@ -828,7 +828,10 @@ def save_credentials(server_id: int, token: str, path: Path | None = None) -> No
path = path or _credentials_path()
path.parent.mkdir(parents=True, exist_ok=True)
temporary = path.with_suffix(".tmp")
- temporary.write_text(json.dumps({"server_id": server_id, "collector_token": token}))
+ # Created with mode 0600 from the start, so there is no moment when the token sits in a file others can read.
+ descriptor = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
+ with os.fdopen(descriptor, "w") as handle:
+ handle.write(json.dumps({"server_id": server_id, "collector_token": token}))
temporary.chmod(0o600)
temporary.replace(path)
diff --git a/src/agent/main.py b/src/agent/main.py
deleted file mode 100644
index dcda6b1..0000000
--- a/src/agent/main.py
+++ /dev/null
@@ -1,60 +0,0 @@
-import os
-from dotenv import load_dotenv
-from google import genai
-from google.genai import types
-from prompts import SYSTEM_PROMPT
-import time
-
-# 1. Učitaj enviroment varijable (.env)
-load_dotenv()
-
-# 2. Inicijaliziraj klijent
-client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))
-
-for model in client.models.list():
- print(model.name)
-
-# 3. Inicijaliziraj novu chat sesiju s postavljenim System Promptom
-chat = client.chats.create(
- model="models/gemini-3.5-flash",
- config=types.GenerateContentConfig(
- system_instruction=SYSTEM_PROMPT,
- temperature=0.2, # Niža temperatura za preciznije i manje "kreativne" sysadmin odgovore
- )
-)
-
-log_file = open("src/logs/chat.log", "w").close() # Očisti log datoteku prije početka nove sesije
-log_file = open("src/logs/chat.log", "a") # Otvori log datoteku za dodavanje novih poruka
-
-print("Agent started\n")
-automated_input = ""
-firstRun = True
-# 4. Simulacija razgovora
-while True:
- if firstRun:
- user_input = input("[USER - SET TASK]" + "-"*30 + "\n")
-
- if not automated_input and not firstRun:
- user_input = input("[USER]" + "-"*30 + "\n")
- else:
- user_input = automated_input
- automated_input = ""
-
- if user_input.lower() in ["exit", "quit"]:
- log_file.close()
- break
-
- firstRun = False
- response = chat.send_message(user_input)
-
- log_file.write(f"[USER]" + "-"*30 + f"\n {user_input} \n")
-
- response_text = getattr(response, "text", "")
- if response_text and "[PARTIAL]" in response_text:
- print(f"\n[AGENT]" + "-"*30 + f"\n {response_text} \n")
- automated_input = "[CONTINUE]"
-
- agent_response = f"\n[AGENT]" + "-"*30 + f"\n {response_text} \n"
- log_file.write(agent_response)
- print(agent_response)
- time.sleep(6)
\ No newline at end of file
diff --git a/src/agent/service.py b/src/agent/service.py
index 7fdbc17..b79a2d0 100644
--- a/src/agent/service.py
+++ b/src/agent/service.py
@@ -12,6 +12,8 @@ from google.genai import types
from .health_checks import HealthCheckRunner
from .prompts import SYSTEM_PROMPT
+MAX_CACHED_CHATS = 200
+
load_dotenv()
@@ -24,6 +26,8 @@ class AgentService:
raise RuntimeError("GEMINI_API_KEY is not configured.")
self.client = genai.Client(api_key=api_key)
self.health_checks = health_checks or HealthCheckRunner()
+ # Least recently used first (a dict keeps insertion order). A chat only lives in memory so the assistant
+ # remembers the conversation; the cache is bounded so endless new conversations can't grow it without limit.
self.chats = {}
self.lock = threading.Lock()
@@ -51,7 +55,11 @@ class AgentService:
},
),
)
- return self.chats[chat_key]
+ chat = self.chats.pop(chat_key)
+ self.chats[chat_key] = chat # now the most recently used
+ while len(self.chats) > MAX_CACHED_CHATS:
+ del self.chats[next(iter(self.chats))]
+ return chat
def send(self, user_id: int, conversation_id: str, message: str, command_logger: Callable[[dict], None] | None = None) -> str:
chat = self._get_chat(user_id, conversation_id)
diff --git a/src/agent/utils.py b/src/agent/utils.py
deleted file mode 100644
index 7d86589..0000000
--- a/src/agent/utils.py
+++ /dev/null
@@ -1,75 +0,0 @@
-import os
-import psutil
-import docker
-import cryptography
-import time
-
-class System:
- @staticmethod
- def get_cpu_info():
- cpu_info = {
- "physical_cores": psutil.cpu_count(logical=False),
- "total_cores": psutil.cpu_count(logical=True),
- "max_frequency": psutil.cpu_freq().max,
- "min_frequency": psutil.cpu_freq().min,
- "current_frequency": psutil.cpu_freq().current,
- "cpu_usage_per_core": psutil.cpu_percent(percpu=True),
- "total_cpu_usage": psutil.cpu_percent(),
- "cpu_load_average": psutil.getloadavg(),
- "cpu_temperature": psutil.sensors_temperatures().get('coretemp', [])
- }
- return cpu_info
-
- @staticmethod
- def get_cpu_curr_freq():
- return psutil.cpu_freq().current
-
- @staticmethod
- def get_cpu_utilization():
- total = psutil.cpu_percent(interval = 1, percpu = False)
- divided = psutil.cpu_times_percent()
- return {
- "total": total,
- "user": divided.user,
- "system": divided.system,
- "idle": divided.idle,
- "iowait": divided.iowait
- }
-
- @staticmethod
- def get_memory_info():
- virtual_memory = psutil.virtual_memory()
- swap_memory = psutil.swap_memory()
- memory_info = {
- "virtual_memory": {
- "total": virtual_memory.total,
- "available": virtual_memory.available,
- "used": virtual_memory.used,
- "free": virtual_memory.free,
- "percent": virtual_memory.percent
- },
- "swap_memory": {
- "total": swap_memory.total,
- "used": swap_memory.used,
- "free": swap_memory.free,
- "percent": swap_memory.percent
- }
- }
- return memory_info
-
- @staticmethod
- def get_storage_info():
- usage = psutil.disk_usage('/')
- diskio = psutil.disk_io_counters()
- storage_info = {
- "total": usage.total,
- "used": usage.used,
- "free": usage.free,
- "percent": usage.percent,
- }
- return storage_info
-
- @staticmethod
- def get_uptime():
- return time.time() - psutil.boot_time()
-
\ No newline at end of file
diff --git a/src/backend/auth.py b/src/backend/auth.py
index 1ccd836..a957d39 100644
--- a/src/backend/auth.py
+++ b/src/backend/auth.py
@@ -16,6 +16,8 @@ from .models import AuditLog, RecoveryCode, User
STAGE_FULL = "full"
STAGE_ENROLLMENT = "enrollment"
PENDING_TWO_FACTOR_TTL_SECONDS = 300
+# A limited session exists only to set up two-factor, so it doesn't need to last a day; a stolen one is worth less.
+ENROLLMENT_SESSION_TTL_SECONDS = 30 * 60
MAX_PENDING_ATTEMPTS = 5
_PUBLIC_ID = re.compile(r"[0-9a-f]{16}")
# Checked against when the account doesn't exist, so that unknown emails cost the same time as wrong passwords.
@@ -50,7 +52,8 @@ def _key_for_digest(digest: str) -> str:
def _store_session(redis_client, session_id: str, payload: dict):
- redis_client.set(_session_key(session_id), json.dumps(payload), ex=SESSION_TTL_SECONDS)
+ lifetime = ENROLLMENT_SESSION_TTL_SECONDS if payload.get("stage") == STAGE_ENROLLMENT else SESSION_TTL_SECONDS
+ redis_client.set(_session_key(session_id), json.dumps(payload), ex=lifetime)
def _read_session(redis_client, session_id: str):
diff --git a/src/backend/config.py b/src/backend/config.py
index d8d98fe..8319c61 100644
--- a/src/backend/config.py
+++ b/src/backend/config.py
@@ -47,6 +47,8 @@ INGEST_MIN_SPACING_RATIO = float(os.getenv("INGEST_MIN_SPACING_RATIO", "0.25"))
INGEST_MIN_SPACING_SECONDS = int(os.getenv("INGEST_MIN_SPACING_SECONDS", "5"))
# The most servers an install will hold, whichever way they are added (enrollment, approval or by hand).
MAX_SERVERS = int(os.getenv("MAX_SERVERS", "200"))
+# Enrollments per address (and twice that per key) per hour. Raise it if many collectors share one address.
+ENROLL_RATE_LIMIT = int(os.getenv("ENROLL_RATE_LIMIT", "30"))
# Creating the first account (which becomes the admin) needs a setup token, so nobody else can claim a
# fresh install. Leave unset to have one generated and printed in the backend log at every start until
diff --git a/src/backend/main.py b/src/backend/main.py
index c3f8fb3..9c79061 100644
--- a/src/backend/main.py
+++ b/src/backend/main.py
@@ -3,6 +3,7 @@ from __future__ import annotations
import ipaddress
import json
import math
+import unicodedata
import re
import hashlib
import secrets
@@ -47,6 +48,7 @@ from .config import (
DOCKER_PROXY_IMAGE,
JOIN_TOKEN_TTL_SECONDS,
ENABLE_API_DOCS,
+ ENROLL_RATE_LIMIT,
INGEST_MIN_SPACING_RATIO,
INGEST_MIN_SPACING_SECONDS,
MAX_SERVERS,
@@ -55,7 +57,7 @@ from .config import (
SESSION_COOKIE_SECURE,
SESSION_TTL_SECONDS,
)
-from .rules import DEFAULT_THRESHOLDS, HOST_RULE_PREFIX, container_rules, host_rules, validate_thresholds, detect_system_events, system_rules, SYSTEM_RULE_PREFIX, validate_mount_thresholds, validate_helper_events, REBOOT_DETAILS, REBOOT_KEY_PREFIX, detect_container_events, memory_trend_rules, CONTAINER_EVENT_KINDS, MAX_CONTAINER_EVENTS_PER_CHECKIN, MAX_CONTAINER_EVENTS_PER_HOUR, TREND_RULE_PREFIX, NETWORK_EVENT_KINDS, NETWORK_RULE_PREFIX, MAX_NETWORK_EVENTS_PER_CHECKIN, MAX_NETWORK_EVENTS_PER_HOUR, INTERNET_STATUSES, LISTENER_SCOPES, FIREWALL_LABELS, detect_listener_changes, detect_firewall_changes, network_rules, validate_listeners, validate_firewall
+from .rules import HELPER_EVENT_KINDS, DEFAULT_THRESHOLDS, HOST_RULE_PREFIX, container_rules, host_rules, validate_thresholds, detect_system_events, system_rules, SYSTEM_RULE_PREFIX, validate_mount_thresholds, validate_helper_events, REBOOT_DETAILS, REBOOT_KEY_PREFIX, detect_container_events, memory_trend_rules, CONTAINER_EVENT_KINDS, MAX_CONTAINER_EVENTS_PER_CHECKIN, MAX_CONTAINER_EVENTS_PER_HOUR, TREND_RULE_PREFIX, NETWORK_EVENT_KINDS, NETWORK_RULE_PREFIX, MAX_NETWORK_EVENTS_PER_CHECKIN, MAX_NETWORK_EVENTS_PER_HOUR, INTERNET_STATUSES, LISTENER_SCOPES, FIREWALL_LABELS, detect_listener_changes, detect_firewall_changes, network_rules, validate_listeners, validate_firewall
from .rate_limit import enforce_ip_limit, enforce_limit, enforce_login_limit, enforce_register_limit, refund_attempt, reset_limits
from ..agent.service import delete_agent_conversation, send_agent_message
@@ -97,15 +99,18 @@ class PasswordChangeRequest(BaseModel):
class AgentChatRequest(BaseModel):
- conversation_id: str = Field(max_length=255)
+ conversation_id: str = Field(max_length=64)
message: str = Field(max_length=8000)
class ConversationRequest(BaseModel):
- conversation_id: str = Field(max_length=255)
+ conversation_id: str = Field(max_length=64)
title: str = Field(default="New chat", max_length=255)
+RESERVED_EVENT_TYPES = {"health_check_completed", "credentials_replaced", "login", "logout"}
+
+
class LogCreateRequest(BaseModel):
"""An entry the dashboard adds to the user's own audit log. The source is always the dashboard, so nobody can
write entries that look like they came from sign-in or administration."""
@@ -117,6 +122,14 @@ class LogCreateRequest(BaseModel):
conversation_id: str | None = Field(default=None, max_length=255)
event_type: str | None = Field(default=None, max_length=50, pattern=r"^[a-z0-9_\-]*$")
+ @field_validator("event_type")
+ @classmethod
+ def _not_a_system_event(cls, value):
+ # These are counted on the dashboard (or mean something to the system), so a user can't write them.
+ if value in RESERVED_EVENT_TYPES:
+ raise ValueError("That event type is reserved.")
+ return value
+
@field_validator("level", mode="before")
@classmethod
def _upper_case_level(cls, value):
@@ -158,7 +171,7 @@ class AdminSettingsRequest(BaseModel):
class MetricsIngestRequest(BaseModel):
- server_id: int
+ server_id: int = Field(ge=1, le=2**31 - 1)
server: dict = {}
host: dict = {}
containers: list[dict] = []
@@ -592,11 +605,13 @@ def two_factor_enable(payload: CodeRequest, request: Request, response: Response
def _confirm_password(db, user: User, password: str | None) -> None:
- """Ask for the current password again before a sensitive change. Wrong tries share the two-factor lockout."""
+ """Ask for the current password again before a sensitive change. Every try counts toward the two-factor lockout,
+ the right password included, so a locked-out session can't keep guessing until it is right."""
+ limit_key = enforce_limit(redis_client, scope="2fa-user", subject=user.id, limit=TWO_FACTOR_ATTEMPTS, window_seconds=TWO_FACTOR_WINDOW_SECONDS)
if not password or not verify_password(user, password):
- enforce_limit(redis_client, scope="2fa-user", subject=user.id, limit=TWO_FACTOR_ATTEMPTS, window_seconds=TWO_FACTOR_WINDOW_SECONDS)
record_audit_log(db, user_id=user.id, level="WARNING", message="A wrong or missing password was given while setting up two-factor", source="auth")
raise HTTPException(status_code=401, detail="Enter your current password to set up two-factor.")
+ reset_limits(redis_client, limit_key)
def _reauthenticate(db, user: User, password: str, code: str) -> None:
@@ -867,23 +882,32 @@ def agent_config(session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_N
OVERVIEW_ALERT_LIMIT = 50
OVERVIEW_ALERTS_PER_SERVER = 10
OVERVIEW_ALERT_SCAN = 1000
+CREDENTIALS_ALERT_KEY = "system:credentials"
def _spread_alerts(alerts: list) -> list:
"""The most severe alerts first, but at most a few per server, so one server with a flood of alerts can't push
- everybody else's off the list. The alert count shown elsewhere is still the true total."""
+ everybody else's off the list. The alert count shown elsewhere is still the true total.
+
+ The warning that a server's credentials were replaced is always listed, on top of that allowance: whoever
+ holds the new credentials decides what the server reports, so it must not be crowdable by what it reports."""
+ pinned = [alert for alert in alerts if alert.rule_key == CREDENTIALS_ALERT_KEY][:OVERVIEW_ALERT_LIMIT]
per_server: dict[int, int] = {}
chosen = []
for alert in alerts:
+ if alert.rule_key == CREDENTIALS_ALERT_KEY:
+ continue
per_server[alert.server_id] = per_server.get(alert.server_id, 0) + 1
if per_server[alert.server_id] <= OVERVIEW_ALERTS_PER_SERVER:
chosen.append(alert)
- if len(chosen) >= OVERVIEW_ALERT_LIMIT:
+ if len(chosen) >= OVERVIEW_ALERT_LIMIT - len(pinned):
break
- return chosen
+ return pinned + chosen
SYSTEM_EVENTS_SHOWN = 10
+# The note that a server's credentials were replaced is shown whatever else that server reports, so it can't be pushed out.
+CREDENTIALS_EVENTS_SHOWN = 3
def recent_system_events(db, since: datetime) -> dict[int, list[dict]]:
@@ -894,10 +918,12 @@ def recent_system_events(db, since: datetime) -> dict[int, list[dict]]:
OOM kills out of the list. Every other kind stays one row per occurrence.
"""
result: dict[int, list[dict]] = {}
- for event in db.query(SystemEvent).filter(SystemEvent.time >= since, SystemEvent.kind != "kernel_error", SystemEvent.kind.notin_(CONTAINER_EVENT_KINDS + NETWORK_EVENT_KINDS)).order_by(SystemEvent.time.desc()):
- rows = result.setdefault(event.server_id, [])
- if len(rows) < SYSTEM_EVENTS_SHOWN:
- rows.append({"time": event.time.isoformat(), "first_time": event.time.isoformat(), "kind": event.kind, "detail": event.detail, "count": 1})
+ ordinary = (SystemEvent.time >= since, SystemEvent.kind != "kernel_error", SystemEvent.kind != CREDENTIALS_REPLACED_KIND, SystemEvent.kind.notin_(CONTAINER_EVENT_KINDS + NETWORK_EVENT_KINDS))
+ # The database picks the newest few per server, so a server with thousands of events costs no more to show.
+ for events, limit in (((ordinary), SYSTEM_EVENTS_SHOWN), ((SystemEvent.time >= since, SystemEvent.kind == CREDENTIALS_REPLACED_KIND), CREDENTIALS_EVENTS_SHOWN)):
+ ranked = db.query(SystemEvent.id.label("event_id"), func.row_number().over(partition_by=SystemEvent.server_id, order_by=SystemEvent.time.desc()).label("rank")).filter(*events).subquery()
+ for event in db.query(SystemEvent).join(ranked, ranked.c.event_id == SystemEvent.id).filter(ranked.c.rank <= limit):
+ result.setdefault(event.server_id, []).append({"time": event.time.isoformat(), "first_time": event.time.isoformat(), "kind": event.kind, "detail": event.detail, "count": 1})
shown: dict[int, int] = {}
grouped = (
db.query(SystemEvent.server_id, SystemEvent.detail, func.count(), func.min(SystemEvent.time), func.max(SystemEvent.time))
@@ -1009,6 +1035,8 @@ def latest_containers(db, server_id: int, host_time: datetime | None) -> list[Co
HISTORY_MAX_MINUTES = 24 * 60
+# An ID in the address must fit the database's INTEGER column, or the query itself fails with a server error.
+PathId = Annotated[int, PathParam(ge=1, le=2**31 - 1)]
RangeName = Literal["1h", "6h", "24h", "7d", "30d", "90d", "1y", "all"]
@@ -1029,7 +1057,7 @@ def _history_response(db, spec, server: Server, key: str | None, range_name: str
@app.get("/monitoring/servers/{server_id}/history")
def server_history(
- server_id: int, minutes: int = 30, range_name: Annotated[RangeName | None, Query(alias="range")] = None, metrics: str | None = None,
+ server_id: PathId, minutes: int = 30, range_name: Annotated[RangeName | None, Query(alias="range")] = None, metrics: str | None = None,
session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db),
):
"""A server's metrics over time.
@@ -1074,7 +1102,7 @@ def fleet_statistics(range_name: Annotated[RangeName, Query(alias="range")] = "2
@app.get("/stats/servers/{server_id}/heatmap")
def server_heatmap(
- server_id: int, metric: Literal["cpu", "ram"] = "cpu", offset: Annotated[int, Query(ge=-stats.MAX_OFFSET_MINUTES, le=stats.MAX_OFFSET_MINUTES)] = 0,
+ server_id: PathId, metric: Literal["cpu", "ram"] = "cpu", offset: Annotated[int, Query(ge=-stats.MAX_OFFSET_MINUTES, le=stats.MAX_OFFSET_MINUTES)] = 0,
session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db),
):
"""When a server is busy: its average per hour of the day and weekday over the last 30 days."""
@@ -1100,7 +1128,7 @@ def server_heatmap(
@app.get("/monitoring/servers/{server_id}/mounts/history")
def mount_history(
- server_id: int, mount: Annotated[str, Query(min_length=1, max_length=255)], range_name: Annotated[RangeName, Query(alias="range")] = "24h", metrics: str | None = None,
+ server_id: PathId, mount: Annotated[str, Query(min_length=1, max_length=255)], range_name: Annotated[RangeName, Query(alias="range")] = "24h", metrics: str | None = None,
session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db),
):
"""One filesystem's usage over time (``mount`` is its path, for example ``/data``)."""
@@ -1117,7 +1145,7 @@ def mount_history(
@app.get("/monitoring/servers/{server_id}/containers/{container_id}/history")
def container_history(
- server_id: int, container_id: Annotated[str, PathParam(min_length=1, max_length=128)], range_name: Annotated[RangeName, Query(alias="range")] = "24h", metrics: str | None = None,
+ server_id: PathId, container_id: Annotated[str, PathParam(min_length=1, max_length=128)], range_name: Annotated[RangeName, Query(alias="range")] = "24h", metrics: str | None = None,
session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db),
):
"""One container's CPU and memory over time."""
@@ -1142,7 +1170,7 @@ CONTAINER_DETAIL_FIELDS = (
@app.get("/monitoring/servers/{server_id}/containers/{container_id}")
-def container_detail(server_id: int, container_id: str, limit: int = 60, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
+def container_detail(server_id: PathId, container_id: str, limit: int = 60, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
_current_user(session_id, db)
server = db.query(Server).filter(Server.id == server_id).first()
if not server:
@@ -1238,7 +1266,7 @@ def enroll_collector(payload: EnrollRequest, db=Depends(get_db), request: Reques
if not _is_admin(db, key.owner_id):
raise HTTPException(status_code=403, detail="Enrollment key belongs to an account that can no longer add servers.")
_soft_limit(scope="enroll-key", subject=key.id, limit=ENROLL_RATE_LIMIT * 2, window_seconds=ENROLL_RATE_WINDOW_SECONDS)
- hostname = _printable(payload.hostname, 255)
+ hostname = _hostname_or_error(payload.hostname)
machine_id = (payload.machine_id or "").strip()[:64] or None
if not hostname:
raise HTTPException(status_code=400, detail="Hostname is required.")
@@ -1249,7 +1277,7 @@ def enroll_collector(payload: EnrollRequest, db=Depends(get_db), request: Reques
replaced = server is not None
if not server:
_ensure_server_capacity(db)
- server = Server(owner_id=key.owner_id, hostname=hostname, machine_id=machine_id)
+ server = Server(owner_id=key.owner_id, hostname=_unique_hostname(db, hostname), machine_id=machine_id)
db.add(server)
db.flush()
token = secrets.token_urlsafe(32)
@@ -1305,9 +1333,7 @@ def register_collector(payload: CollectorRegisterRequest, request: Request, db=D
via = _collector_via(request)
ip_address = _client_ip(request)
enforce_ip_limit(redis_client, scope="collector-register", ip_address=ip_address, limit=30, window_seconds=3600)
- hostname = payload.hostname.strip()[:255]
- if not hostname:
- raise HTTPException(status_code=400, detail="Hostname is required.")
+ hostname = _hostname_or_error(payload.hostname)
machine_id = (payload.machine_id or "").strip()[:64] or None
join_token = None
if payload.join_token:
@@ -1370,7 +1396,7 @@ def _pending_for_user(pending_id: int, db) -> PendingCollector:
@app.post("/monitoring/pending-collectors/{pending_id}/approve")
-def approve_pending_collector(pending_id: int, payload: ApproveRequest | None = None, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
+def approve_pending_collector(pending_id: PathId, payload: ApproveRequest | None = None, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
user = _require_admin(session_id, db)
pending = _pending_for_user(pending_id, db)
server = _server_for_machine(db, pending.machine_id)
@@ -1379,7 +1405,7 @@ def approve_pending_collector(pending_id: int, payload: ApproveRequest | None =
raise HTTPException(status_code=409, detail=f"This machine matches the existing server {server.hostname}. Approving replaces that server's credentials and signs its current collector out. Confirm to continue.")
if not server:
_ensure_server_capacity(db)
- server = Server(owner_id=user["id"], hostname=pending.hostname, machine_id=pending.machine_id, ip_address=pending.ip_address, os_info=pending.os_info)
+ server = Server(owner_id=user["id"], hostname=_unique_hostname(db, _clean_hostname(pending.hostname) or "unnamed server"), machine_id=pending.machine_id, ip_address=pending.ip_address, os_info=pending.os_info)
db.add(server)
db.flush()
pending.status = "approved"
@@ -1391,7 +1417,7 @@ def approve_pending_collector(pending_id: int, payload: ApproveRequest | None =
@app.post("/monitoring/pending-collectors/{pending_id}/dismiss")
-def dismiss_pending_collector(pending_id: int, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
+def dismiss_pending_collector(pending_id: PathId, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
_require_admin(session_id, db)
pending = _pending_for_user(pending_id, db)
pending.status = "dismissed"
@@ -1500,13 +1526,13 @@ def add_monitored_server(payload: ServerRequest, session_id: Annotated[str | Non
_ensure_server_capacity(db)
token = secrets.token_urlsafe(32)
token_hash = hashlib.sha256(token.encode()).hexdigest()
- server = Server(owner_id=user["id"], hostname=payload.hostname.strip()[:255], ip_address=payload.ip_address, check_interval_seconds=payload.check_interval_seconds, collector_token_hash=token_hash)
+ server = Server(owner_id=user["id"], hostname=_unique_hostname(db, _hostname_or_error(payload.hostname)), ip_address=payload.ip_address, check_interval_seconds=payload.check_interval_seconds, collector_token_hash=token_hash)
db.add(server); db.commit(); db.refresh(server)
return {"id": server.id, "hostname": server.hostname, "check_interval_seconds": server.check_interval_seconds, "collector_token": token}
@app.delete("/monitoring/servers/{server_id}")
-def delete_monitored_server(server_id: int, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
+def delete_monitored_server(server_id: PathId, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
_require_admin(session_id, db)
server = db.query(Server).filter(Server.id == server_id).first()
if not server:
@@ -1528,7 +1554,7 @@ def delete_monitored_server(server_id: int, session_id: Annotated[str | None, Co
@app.patch("/monitoring/servers/{server_id}/interval")
-def update_check_interval(server_id: int, payload: dict, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
+def update_check_interval(server_id: PathId, payload: dict, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
_require_admin(session_id, db)
server = db.query(Server).filter(Server.id == server_id).first()
interval = payload.get("check_interval_seconds")
@@ -1540,16 +1566,36 @@ def update_check_interval(server_id: int, payload: dict, session_id: Annotated[s
return {"id": server.id, "check_interval_seconds": server.check_interval_seconds}
+MAX_HELPER_EVENTS_PER_CHECKIN = 20
+MAX_HELPER_EVENTS_PER_WEEK = 500
+HELPER_NOTICE_KIND = "helper_notice"
+
+
def record_helper_events(db, server_id: int, raw, now: datetime) -> None:
- """Store the helper's kernel-log events once each; the helper republishes the whole ring buffer."""
+ """Store the helper's kernel-log events once each; the helper republishes the whole ring buffer.
+
+ A collector chooses the keys that make events "new", so without a cap it could add rows without end. At most
+ MAX_HELPER_EVENTS_PER_CHECKIN are stored per check-in (the rest are offered again by the next one, because the
+ helper republishes its buffer) and MAX_HELPER_EVENTS_PER_WEEK per server; past that there is one notice.
+ """
events = validate_helper_events(raw, now)
if not events:
return
known = {key for (key,) in db.query(SystemEvent.event_key).filter(SystemEvent.server_id == server_id, SystemEvent.event_key.in_([event[3] for event in events]))}
- for kind, time, detail, key in events:
- if key not in known:
- known.add(key)
- db.add(SystemEvent(server_id=server_id, time=time, kind=kind, detail=detail, event_key=key))
+ fresh, seen = [], set()
+ for event in events:
+ if event[3] not in known and event[3] not in seen:
+ seen.add(event[3])
+ fresh.append(event)
+ if not fresh:
+ return
+ week = now - timedelta(days=7)
+ stored = db.query(func.count(SystemEvent.id)).filter(SystemEvent.server_id == server_id, SystemEvent.kind.in_(HELPER_EVENT_KINDS), SystemEvent.event_key.isnot(None), SystemEvent.time >= week).scalar() or 0
+ room = max(0, MAX_HELPER_EVENTS_PER_WEEK - stored)
+ for kind, time, detail, key in fresh[:min(MAX_HELPER_EVENTS_PER_CHECKIN, room)]:
+ db.add(SystemEvent(server_id=server_id, time=time, kind=kind, detail=detail, event_key=key))
+ if room <= 0 and not db.query(SystemEvent.id).filter(SystemEvent.server_id == server_id, SystemEvent.kind == HELPER_NOTICE_KIND, SystemEvent.time >= now - timedelta(hours=1)).first():
+ db.add(SystemEvent(server_id=server_id, time=now, kind=HELPER_NOTICE_KIND, detail=f"This week's limit of {MAX_HELPER_EVENTS_PER_WEEK} kernel-log events was reached; newer ones are not recorded until older ones age out"))
DOCKER_STATUSES = {"ok", "unreachable", "not_configured"}
@@ -1571,6 +1617,11 @@ def _whole_number(value, low: int, high: int) -> int | None:
return value if isinstance(value, int) and not isinstance(value, bool) and low <= value <= high else None
+def _one_of(value, allowed) -> bool:
+ """Whether a reported value is one of the allowed strings. A list or object is simply "no", not an error."""
+ return isinstance(value, str) and value in allowed
+
+
def _finite_number(value, low: float, high: float) -> float | None:
"""A real number inside [low, high], or None for anything else: text, booleans, NaN, infinity, out of range."""
if isinstance(value, bool) or not isinstance(value, (int, float)):
@@ -1590,6 +1641,8 @@ HOST_NUMBER_LIMITS = {
"net_bytes_sent_mb": (0, 1e12), "net_bytes_recv_mb": (0, 1e12), "system_uptime_seconds": (0, 1e10),
}
MAX_MOUNTS_PER_CHECKIN = 64
+# A mount name ends up inside incident keys (255 characters with a prefix), so it is kept well under that.
+MAX_MOUNT_NAME = 200
INT_MAX = 2**31 - 1 # the database's INTEGER columns
@@ -1605,7 +1658,7 @@ def _clean_mount(raw) -> dict | None:
"""One filesystem record reduced to known fields of the right type and size, or None if unusable."""
if not isinstance(raw, dict) or not isinstance(raw.get("mount"), str):
return None
- mount = _printable(raw["mount"], 255)
+ mount = _printable(raw["mount"], MAX_MOUNT_NAME)
if not mount.startswith("/"):
return None
fs_type = raw.get("fs_type")
@@ -1630,6 +1683,51 @@ def _clean_mounts(raw) -> list[dict]:
return cleaned
+# Letters from these alphabets look like Latin ones, which is how a name poses as another (srv with a Cyrillic "a").
+_LOOKALIKE_SCRIPTS = {"CYRILLIC", "GREEK", "ARMENIAN", "CHEROKEE", "COPTIC"}
+
+
+def _clean_hostname(text: str) -> str | None:
+ """A hostname made printable and bounded, in its compatibility-normal form (so full-width letters become plain
+ ones), or None if it mixes Latin letters with a lookalike alphabet. A name written entirely in one alphabet is
+ fine; the mix is what is used to impersonate another server."""
+ name = _printable(unicodedata.normalize("NFKC", text), 255)
+ scripts = {unicodedata.name(char, "").split(" ")[0] for char in name if char.isalpha()}
+ if not name or ("LATIN" in scripts and scripts & _LOOKALIKE_SCRIPTS):
+ return None
+ return name
+
+
+def _name_key(name: str) -> str:
+ """What two hostnames are compared by: case, accents and width are ignored."""
+ decomposed = unicodedata.normalize("NFKD", name)
+ return "".join(char for char in decomposed if not unicodedata.combining(char)).casefold()
+
+
+def _hostname_in_use(db, name: str, *, excluding: int | None = None) -> bool:
+ key = _name_key(name)
+ return any(_name_key(existing) == key for (existing,) in db.query(Server.hostname).filter(Server.id != excluding if excluding else True))
+
+
+def _unique_hostname(db, name: str) -> str:
+ """The name for a new server: as given, or numbered ("pi (2)") if another server already has it, so two machines
+ can be told apart (default names such as "raspberrypi" are common) and nothing can sit behind a twin."""
+ if not _hostname_in_use(db, name):
+ return name
+ for number in range(2, 1000):
+ candidate = f"{name} ({number})"[:255]
+ if not _hostname_in_use(db, candidate):
+ return candidate
+ return name
+
+
+def _hostname_or_error(text: str) -> str:
+ name = _clean_hostname(text)
+ if not name:
+ raise HTTPException(status_code=400, detail="The hostname is empty, or mixes alphabets (for example Latin and Cyrillic letters), which is refused because it can be used to impersonate another server.")
+ return name
+
+
def _apply_server_fields(db, server: Server, reported: dict) -> None:
"""Copy the descriptive fields a collector reports onto its server, each checked, never trusted.
@@ -1637,9 +1735,8 @@ def _apply_server_fields(db, server: Server, reported: dict) -> None:
alert titles.
"""
hostname = reported.get("hostname")
- if isinstance(hostname, str) and (name := _printable(hostname, 255)):
- taken = db.query(Server.id).filter(func.lower(Server.hostname) == name.lower(), Server.id != server.id).first()
- if not taken:
+ if isinstance(hostname, str) and (name := _clean_hostname(hostname)):
+ if not _hostname_in_use(db, name, excluding=server.id):
server.hostname = name
address = reported.get("ip_address")
if isinstance(address, str):
@@ -1656,7 +1753,6 @@ def _apply_server_fields(db, server: Server, reported: dict) -> None:
server.total_ram_gb = ram
-ENROLL_RATE_LIMIT = 30
ENROLL_RATE_WINDOW_SECONDS = 3600
CREDENTIALS_REPLACED_KIND = "credentials_replaced"
@@ -1794,22 +1890,22 @@ def _clean_system_fields(fields: dict) -> None:
for key in ("oom_kill_total", "kernel_tainted"):
if key in fields:
fields[key] = _whole_number(fields[key], 0, INT_MAX)
- if "last_shutdown" in fields and fields["last_shutdown"] not in {"clean", "unclean", "unknown"}:
+ if "last_shutdown" in fields and not _one_of(fields["last_shutdown"], {"clean", "unclean", "unknown"}):
fields["last_shutdown"] = None
- if "docker_status" in fields and fields["docker_status"] not in DOCKER_STATUSES:
+ if "docker_status" in fields and not _one_of(fields["docker_status"], DOCKER_STATUSES):
fields["docker_status"] = None
- if "docker_reason" in fields and fields["docker_reason"] not in DOCKER_REASON_CODES:
+ if "docker_reason" in fields and not _one_of(fields["docker_reason"], DOCKER_REASON_CODES):
fields["docker_reason"] = None
if "docker_latency_ms" in fields:
fields["docker_latency_ms"] = _whole_number(fields["docker_latency_ms"], 0, 600_000)
if "collect_duration_ms" in fields:
fields["collect_duration_ms"] = _whole_number(fields["collect_duration_ms"], 0, 3_600_000)
- if "internet_status" in fields and fields["internet_status"] not in INTERNET_STATUSES:
+ if "internet_status" in fields and not _one_of(fields["internet_status"], INTERNET_STATUSES):
fields["internet_status"] = None
if "internet_latency_ms" in fields:
fields["internet_latency_ms"] = _whole_number(fields["internet_latency_ms"], 0, 600_000)
if "boot_id" in fields:
- fields["boot_id"] = fields["boot_id"][:64] if isinstance(fields["boot_id"], str) and fields["boot_id"] else None
+ fields["boot_id"] = (_printable(fields["boot_id"], 64) or None) if isinstance(fields["boot_id"], str) else None
@app.post("/monitoring/metrics/ingest", dependencies=[Depends(require_collector_listener)])
@@ -1933,8 +2029,11 @@ def create_conversation(payload: ConversationRequest, session_id: Annotated[str
user = get_user_from_session(redis_client, session_id)
if not user:
raise HTTPException(status_code=401, detail="Session is invalid or expired.")
+ enforce_limit(redis_client, scope="conversation-user", subject=user["id"], limit=CONVERSATION_CREATE_LIMIT, window_seconds=3600)
if db.query(Conversation).filter(Conversation.id == payload.conversation_id).first():
raise HTTPException(status_code=409, detail="Conversation already exists.")
+ if db.query(func.count(Conversation.id)).filter(Conversation.user_id == user["id"]).scalar() >= MAX_CONVERSATIONS_PER_USER:
+ raise HTTPException(status_code=409, detail=f"You already have {MAX_CONVERSATIONS_PER_USER} conversations. Delete some before starting a new one.")
conversation = Conversation(id=payload.conversation_id, user_id=user["id"], title=payload.title.strip()[:255] or "New chat")
db.add(conversation)
db.commit()
@@ -1959,6 +2058,9 @@ def delete_conversation(conversation_id: str, session_id: Annotated[str | None,
LOG_WRITE_LIMIT = 120
+CHAT_MESSAGE_LIMIT = 30 # messages to the assistant per user per hour (each can cost the admin's Gemini quota)
+CONVERSATION_CREATE_LIMIT = 30 # new conversations per user per hour
+MAX_CONVERSATIONS_PER_USER = 100
@app.post("/logs")
@@ -1992,6 +2094,7 @@ def agent_chat(payload: AgentChatRequest, session_id: Annotated[str | None, Cook
raise HTTPException(status_code=401, detail="Session is invalid or expired.")
if not is_agent_chat_enabled(db):
raise HTTPException(status_code=403, detail="Chat with the agent has been turned off by an admin.")
+ enforce_limit(redis_client, scope="chat-user", subject=user["id"], limit=CHAT_MESSAGE_LIMIT, window_seconds=3600)
message = payload.message.strip()
if not message:
diff --git a/src/backend/rules.py b/src/backend/rules.py
index 9db41ad..5498fbd 100644
--- a/src/backend/rules.py
+++ b/src/backend/rules.py
@@ -240,6 +240,7 @@ def validate_helper_events(raw, now: datetime) -> list[tuple[str, datetime, str,
time = time.replace(tzinfo=timezone.utc)
if not now - HELPER_EVENT_MAX_AGE <= time <= now + timedelta(minutes=5):
continue
+ time = min(time, now) # a collector's clock may run a little ahead; an event can't have happened in the future
events.append((item["kind"], time, "".join(char if char.isprintable() else "?" for char in detail).strip()[:300], key))
return events
diff --git a/src/frontend/package-lock.json b/src/frontend/package-lock.json
index 5fdc03f..a2802c1 100644
--- a/src/frontend/package-lock.json
+++ b/src/frontend/package-lock.json
@@ -1838,9 +1838,9 @@
"license": "MIT"
},
"node_modules/source-map-js": {
- "version": "1.2.1",
- "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz",
- "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==",
+ "version": "1.2.2",
+ "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.2.tgz",
+ "integrity": "sha512-KGj/8Y43x35aZVDtt+J4mK1hoLGHULMYfSkODJNQjNDC3oW1PqPoxMwo0pLUsWM/UEGzON/NxeHywEfNXNP3Vw==",
"dev": true,
"license": "BSD-3-Clause",
"engines": {
diff --git a/tests/test_ingest_validation.py b/tests/test_ingest_validation.py
index f8a35ed..54e5618 100644
--- a/tests/test_ingest_validation.py
+++ b/tests/test_ingest_validation.py
@@ -208,7 +208,7 @@ def test_mount_names_are_made_printable_and_bounded(app):
ingest(app, 1, {"host": {"mounts": [{"mount": "/" + "d" * 600 + "\x00", "fs_type": "t" * 300}]}})
[row] = mounts(app)
- assert len(row.mount) == 255 and len(row.fs_type) == 64 and "\x00" not in row.mount
+ assert len(row.mount) == main.MAX_MOUNT_NAME == 200 and len(row.fs_type) == 64 and "\x00" not in row.mount
# --- how often, and how many --------------------------------------------------------------------------
diff --git a/tests/test_second_review.py b/tests/test_second_review.py
new file mode 100644
index 0000000..195af67
--- /dev/null
+++ b/tests/test_second_review.py
@@ -0,0 +1,346 @@
+"""Findings of the second security review (S2-01 to S2-07): each test fails without its fix."""
+import json
+import os
+from datetime import datetime, timedelta, timezone
+
+os.environ.setdefault("DATABASE_URL", "sqlite://")
+os.environ.setdefault("REDIS_URL", "redis://localhost:6379/0")
+
+import pytest
+from cryptography.fernet import Fernet
+from fastapi.testclient import TestClient
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+from sqlalchemy.pool import StaticPool
+
+from src.backend import auth, main
+from src.backend.models import AuditLog, Base, Conversation, IncidentAlert, Server, SystemEvent, User
+from tests.fakes import FakeRedis
+
+TOKENS = {1: "token-one", 2: "token-two"}
+PASSWORD = "correct-horse-battery-staple"
+
+
+def app(monkeypatch):
+ engine = create_engine("sqlite://", connect_args={"check_same_thread": False}, poolclass=StaticPool)
+ Base.metadata.create_all(engine)
+ Session = sessionmaker(bind=engine)
+ with Session() as seed:
+ seed.add(User(id=1, name="Member", email="[email protected]", password_hash=auth.hash_password(PASSWORD), role="user", is_active=True))
+ for server_id, name in ((1, "alpha"), (2, "bravo")):
+ seed.add(Server(id=server_id, owner_id=1, hostname=name, check_interval_seconds=60, collector_token_hash=main._hash_secret(TOKENS[server_id]), machine_id=f"machine-{server_id}"))
+ seed.commit()
+
+ def get_db():
+ session = Session()
+ try:
+ yield session
+ finally:
+ session.close()
+
+ main.app.dependency_overrides[main.get_db] = get_db
+ redis = FakeRedis()
+ monkeypatch.setattr(main, "redis_client", redis)
+ monkeypatch.setattr(main, "SESSION_COOKIE_SECURE", False)
+ monkeypatch.setenv("SECRETS_KEY", Fernet.generate_key().decode())
+ monkeypatch.setattr(main, "send_agent_message", lambda *args, **kwargs: "ok")
+ monkeypatch.setattr(main, "is_agent_chat_enabled", lambda db: True)
+ client = TestClient(main.app, raise_server_exceptions=False)
+ client.Session, client.redis = Session, redis
+ yield client
+ main.app.dependency_overrides.clear()
+
+
+def ingest(client, server_id, body):
+ payload = body if isinstance(body, str) else json.dumps({"server_id": server_id, **body})
+ return client.post("/monitoring/metrics/ingest", content=payload, headers={"Authorization": f"Bearer {TOKENS[server_id]}", "Content-Type": "application/json"})
+
+
+def sign_in(client):
+ response = client.post("/auth/login", json={"email": "[email protected]", "password": PASSWORD})
+ assert response.status_code == 200, response.text
+ return client
+
+
+def helper_events(count, start=0, kind="kernel_error", when=None):
+ when = when or (datetime.now(timezone.utc) - timedelta(minutes=5)).isoformat()
+ return [{"kind": kind, "time": when, "detail": f"event {start + i}", "key": f"key-{kind}-{start + i}"} for i in range(count)]
+
+
+def stored_events(client, server_id=1, **filters):
+ with client.Session() as db:
+ return db.query(SystemEvent).filter_by(server_id=server_id, **filters).all()
+
+
+# --- S2-01: events from the helper are capped ------------------------------------------------------------------
+
+def test_one_check_in_stores_at_most_twenty_helper_events(app):
+ ingest(app, 1, {"host": {"helper_events": helper_events(50)}})
+
+ assert len(stored_events(app, kind="kernel_error")) == main.MAX_HELPER_EVENTS_PER_CHECKIN == 20
+
+
+def test_the_rest_is_stored_by_later_check_ins_because_the_helper_republishes_its_buffer(app):
+ for _ in range(3):
+ ingest(app, 1, {"host": {"helper_events": helper_events(50)}}) # the same 50 each time, as the helper does
+
+ assert len(stored_events(app, kind="kernel_error")) == 50
+
+
+def test_a_flood_of_new_helper_events_stops_at_a_weekly_cap_and_says_so_once(app):
+ for round_ in range(40):
+ ingest(app, 1, {"host": {"helper_events": helper_events(50, start=round_ * 50)}})
+
+ stored = stored_events(app, kind="kernel_error")
+ assert len(stored) == main.MAX_HELPER_EVENTS_PER_WEEK == 500
+ assert len(stored_events(app, kind="helper_notice")) == 1
+ assert len(stored_events(app, server_id=2)) == 0 # another server is unaffected
+
+
+def test_the_overview_shows_a_few_events_per_server_whatever_is_stored(app):
+ with app.Session() as db:
+ now = datetime.now(timezone.utc)
+ db.add_all(SystemEvent(server_id=1, time=now - timedelta(minutes=i), kind="oom_kill", detail=f"oom {i}", event_key=f"k{i}") for i in range(3000))
+ db.commit()
+ events = main.recent_system_events(db, now - timedelta(days=7))
+
+ assert len(events[1]) == main.SYSTEM_EVENTS_SHOWN and events[1][0]["detail"] == "oom 0"
+
+
+# --- S2-03: the warning about replaced credentials can't be pushed out of sight --------------------------------------------
+
+def test_the_replaced_credentials_event_is_always_shown_even_among_many_newer_events(app):
+ with app.Session() as db:
+ now = datetime.now(timezone.utc)
+ db.add(SystemEvent(server_id=1, time=now - timedelta(minutes=30), kind="credentials_replaced", detail="replaced", event_key="c1"))
+ db.add_all(SystemEvent(server_id=1, time=now - timedelta(minutes=i), kind="oom_kill", detail=f"oom {i}", event_key=f"k{i}") for i in range(25))
+ db.commit()
+ events = main.recent_system_events(db, now - timedelta(days=7))
+
+ assert "credentials_replaced" in [event["kind"] for event in events[1]]
+
+
+def test_the_replaced_credentials_alert_is_always_listed_even_beside_a_flood_of_critical_ones(app):
+ with app.Session() as db:
+ db.add_all(IncidentAlert(server_id=1, severity="CRITICAL", title=f"c{i}", description="d", rule_key=f"container:x{i}:unhealthy", status="active") for i in range(30))
+ db.add(IncidentAlert(server_id=1, severity="WARNING", title="credentials replaced", description="d", rule_key="system:credentials", status="active"))
+ db.commit()
+ chosen = main._spread_alerts(db.query(IncidentAlert).order_by(IncidentAlert.severity.desc()).all())
+
+ assert "system:credentials" in [alert.rule_key for alert in chosen]
+ assert sum(1 for alert in chosen if alert.rule_key != "system:credentials") == main.OVERVIEW_ALERTS_PER_SERVER
+
+
+def test_a_helper_event_dated_in_the_future_is_stored_as_happening_now(app):
+ future = (datetime.now(timezone.utc) + timedelta(minutes=4)).isoformat()
+ before = datetime.now(timezone.utc)
+
+ ingest(app, 1, {"host": {"helper_events": helper_events(1, when=future, kind="oom_kill")}})
+
+ [event] = stored_events(app, kind="oom_kill")
+ assert event.time.replace(tzinfo=timezone.utc) <= datetime.now(timezone.utc) + timedelta(seconds=1) and event.time.replace(tzinfo=timezone.utc) >= before - timedelta(seconds=1)
+
+
+# --- S2-02: the lockout applies to the right password too ---------------------------------------------------------------------
+
+def test_the_right_password_is_refused_once_the_two_factor_lockout_is_reached(app):
+ client = sign_in(app)
+ for _ in range(main.TWO_FACTOR_ATTEMPTS):
+ assert client.post("/auth/2fa/setup", json={"password": "a-wrong-password-for-the-member"}).status_code == 401
+
+ locked = client.post("/auth/2fa/setup", json={"password": PASSWORD})
+
+ assert locked.status_code == 429
+ with app.Session() as db:
+ assert db.get(User, 1).totp_secret_enc is None # nothing was created for the thief
+
+
+def test_a_correct_password_below_the_limit_still_works_and_clears_the_count(app):
+ client = sign_in(app)
+ for _ in range(main.TWO_FACTOR_ATTEMPTS - 2):
+ client.post("/auth/2fa/setup", json={"password": "a-wrong-password-for-the-member"})
+
+ assert client.post("/auth/2fa/setup", json={"password": PASSWORD}).status_code == 200
+ for _ in range(main.TWO_FACTOR_ATTEMPTS - 1): # the count started again from zero
+ assert client.post("/auth/2fa/setup", json={"password": "a-wrong-password-for-the-member"}).status_code in (401, 409)
+
+
+# --- S2-04: chat and conversations are limited -----------------------------------------------------------------------------
+
+def test_chat_messages_are_limited_per_user_per_hour(app):
+ client = sign_in(app)
+ client.post("/conversations", json={"conversation_id": "c1"})
+
+ codes = [client.post("/agent/chat", json={"conversation_id": "c1", "message": f"hello {i}"}).status_code for i in range(main.CHAT_MESSAGE_LIMIT + 3)]
+
+ assert codes[:main.CHAT_MESSAGE_LIMIT] == [200] * main.CHAT_MESSAGE_LIMIT and codes[-3:] == [429] * 3
+
+
+def test_a_user_can_only_hold_so_many_conversations(app, monkeypatch):
+ monkeypatch.setattr(main, "MAX_CONVERSATIONS_PER_USER", 5)
+ monkeypatch.setattr(main, "CONVERSATION_CREATE_LIMIT", 10_000)
+ client = sign_in(app)
+ codes = [client.post("/conversations", json={"conversation_id": f"c{i}"}).status_code for i in range(main.MAX_CONVERSATIONS_PER_USER + 3)]
+
+ assert codes[:main.MAX_CONVERSATIONS_PER_USER] == [200] * main.MAX_CONVERSATIONS_PER_USER and set(codes[-3:]) <= {409, 429}
+ with app.Session() as db:
+ assert db.query(Conversation).count() == main.MAX_CONVERSATIONS_PER_USER
+
+
+def test_creating_conversations_is_rate_limited_too(app, monkeypatch):
+ monkeypatch.setattr(main, "MAX_CONVERSATIONS_PER_USER", 10_000)
+ client = sign_in(app)
+ codes = [client.post("/conversations", json={"conversation_id": f"c{i}"}).status_code for i in range(main.CONVERSATION_CREATE_LIMIT + 2)]
+
+ assert codes[-2:] == [429, 429]
+
+
+# --- S2-05: hostnames can't pose as each other -----------------------------------------------------------------------------------
+
+def hostname(client, server_id=1):
+ with client.Session() as db:
+ return db.get(Server, server_id).hostname
+
+
[email protected]("lookalike", ["alphа", "аlpha", "alрha", "alpha-"[:-1]]) # Cyrillic а and р, and a full-width a
+def test_a_lookalike_of_another_servers_name_is_ignored(app, lookalike):
+ ingest(app, 2, {"server": {"hostname": lookalike}, "host": {}})
+
+ assert hostname(app, 2) == "bravo" if False else hostname(app, 2) in ("bravo",)
+
+
+def test_a_name_that_mixes_alphabets_is_ignored_even_when_nobody_else_has_it(app):
+ ingest(app, 1, {"server": {"hostname": "web-аpp"}, "host": {}})
+
+ assert hostname(app) == "alpha"
+
+
+def test_names_in_one_alphabet_are_fine(app):
+ ingest(app, 1, {"server": {"hostname": "Poslužitelj-1"}, "host": {}})
+ ingest(app, 2, {"server": {"hostname": "сервер"}, "host": {}})
+
+ assert hostname(app) == "Poslužitelj-1" and hostname(app, 2) == "сервер"
+
+
+def test_a_second_server_with_an_existing_name_gets_a_number_so_the_two_can_be_told_apart(app):
+ with app.Session() as db:
+ db.add(User(id=9, name="Admin", email="[email protected]", password_hash="x", role="admin", is_active=True))
+ db.commit()
+ first = main._unique_hostname(app.Session(), "raspberrypi")
+ with app.Session() as db:
+ db.add(Server(owner_id=1, hostname="raspberrypi"))
+ db.commit()
+ second = main._unique_hostname(db, "RaspberryPi")
+ db.add(Server(owner_id=1, hostname=second))
+ db.commit()
+ third = main._unique_hostname(db, "raspberrypi")
+
+ assert (first, second, third) == ("raspberrypi", "RaspberryPi (2)", "raspberrypi (3)")
+
+
+# --- S2-06: a very long mount name can't break check-ins -------------------------------------------------------------------------
+
+def test_a_mount_name_is_short_enough_for_every_incident_key_built_from_it(app):
+ ingest(app, 1, {"host": {"mounts": [{"mount": "/" + "d" * 254, "used_pct": 99.0, "total_gb": 10, "used_gb": 9.9}]}})
+
+ with app.Session() as db:
+ from src.backend.models import MountMetric
+
+ [row] = db.query(MountMetric).all()
+ keys = [alert.rule_key for alert in db.query(IncidentAlert)]
+ assert len(row.mount) <= main.MAX_MOUNT_NAME
+ assert all(len(key) <= 255 for key in keys)
+
+
+# --- S2-07: odd input is refused, not a server error ----------------------------------------------------------------------------------
+
+ {"last_shutdown": ["x"]}, {"last_shutdown": {"a": 1}}, {"docker_status": ["ok"]}, {"docker_reason": {"x": 1}}, {"internet_status": ["ok"]},
+ {"boot_id": "abc\x00def"}, {"boot_id": ["x"]}, {"boot_id": 12}, {"oom_kill_total": 2**40}, {"kernel_tainted": -3},
+])
+def test_unusual_values_in_a_check_in_are_ignored_not_a_server_error(app, host):
+ assert ingest(app, 1, {"host": host}).status_code == 200
+
+
+def test_a_nul_character_in_the_boot_id_is_not_stored(app):
+ ingest(app, 1, {"host": {"boot_id": "abc\x00def"}})
+
+ with app.Session() as db:
+ from src.backend.models import HostMetric
+
+ assert "\x00" not in (db.query(HostMetric).one().boot_id or "")
+
+
[email protected]("path, body", [
+ ("/monitoring/metrics/ingest", {"server_id": 2**40, "host": {}}),
+ ("/monitoring/metrics/ingest", {"server_id": 0, "host": {}}),
+ ("/monitoring/metrics/ingest", {"server_id": -1, "host": {}}),
+ ("/conversations", {"conversation_id": "c" * 65}),
+ ("/agent/chat", {"conversation_id": "c" * 65, "message": "hi"}),
+])
+def test_ids_must_fit_what_the_database_holds(app, path, body):
+ sign_in(app)
+
+ assert app.post(path, json=body).status_code == 422
+
+
[email protected]("path", ["/monitoring/servers/99999999999/history?range=1h", "/monitoring/servers/0/history?range=1h", "/stats/servers/99999999999/heatmap"])
+def test_server_ids_in_the_address_must_fit_too(app, path):
+ sign_in(app)
+
+ assert app.get(path).status_code == 422
+
+
+# --- the smaller points from the review ------------------------------------------------------------------------------------------
+
+def test_the_assistants_memory_of_chats_is_bounded_and_forgets_the_least_recently_used(monkeypatch):
+ from src.agent import service
+
+ class FakeChats:
+ def create(self, **kwargs):
+ return object()
+
+ instance = object.__new__(service.AgentService) if hasattr(service, "AgentService") else None
+ if instance is None:
+ pytest.skip("the assistant service class has another name")
+ instance.client = type("C", (), {"chats": FakeChats()})()
+ instance.chats, instance.lock = {}, __import__("threading").Lock()
+ monkeypatch.setattr(service, "MAX_CACHED_CHATS", 3)
+ for number in range(5):
+ instance._get_chat(1, f"c{number}")
+ instance._get_chat(1, "c2") # used again: now the newest
+
+ assert list(instance.chats) == [(1, "c3"), (1, "c4"), (1, "c2")]
+
+
+def test_a_user_cannot_write_the_log_entries_the_system_counts(app):
+ sign_in(app)
+
+ assert app.post("/logs", json={"message": "x", "event_type": "health_check_completed"}).status_code == 422
+ assert app.post("/logs", json={"message": "x", "event_type": "report_opened"}).status_code == 200
+
+
+def test_a_limited_session_expires_sooner_than_a_full_one(app):
+ user = User(id=5, name="A", email="[email protected]", password_hash="x", role="admin", is_active=True)
+ _, _ = auth.create_session(app.redis, user, stage=auth.STAGE_ENROLLMENT)
+ _, _ = auth.create_session(app.redis, user, stage=auth.STAGE_FULL)
+
+ lifetimes = sorted(app.redis.ttls[key] for key in app.redis.store if key.startswith("homelab:session:"))
+ assert lifetimes == [auth.ENROLLMENT_SESSION_TTL_SECONDS, auth.SESSION_TTL_SECONDS] and auth.ENROLLMENT_SESSION_TTL_SECONDS <= 1800
+
+
+def test_collector_credentials_are_created_private_from_the_start(tmp_path):
+ from src.agent import collector
+
+ path = tmp_path / "state" / "credentials.json"
+ collector.save_credentials(7, "secret-token", path)
+
+ assert path.stat().st_mode & 0o777 == 0o600 and json.loads(path.read_text()) == {"server_id": 7, "collector_token": "secret-token"}
+ assert list(path.parent.iterdir()) == [path] # no temporary file left behind
+
+
+def test_the_enrollment_limit_is_configurable():
+ from src.backend import config
+
+ assert config.ENROLL_RATE_LIMIT == 30 and main.ENROLL_RATE_LIMIT == config.ENROLL_RATE_LIMIT
diff --git a/tests/test_system_helper.py b/tests/test_system_helper.py
index 95b447d..2f1dc26 100644
--- a/tests/test_system_helper.py
+++ b/tests/test_system_helper.py
@@ -221,7 +221,7 @@ def test_helper_event_count_and_text_are_bounded(db):
_ingest(db, helper_events=many, last_shutdown="unknown")
rows = db.query(SystemEvent).all()
- assert len(rows) == 50 and all(len(row.detail) <= 300 for row in rows)
+ assert len(rows) == main.MAX_HELPER_EVENTS_PER_CHECKIN == 20 and all(len(row.detail) <= 300 for row in rows) # the first 50 offered, 20 stored per check-in
def test_unclean_shutdown_makes_the_reboot_critical(db):