Skip to content

Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.

feat(retention): keep check-ins for 7 days, hourly summaries and events for a year or longer

karloks20057575403cab79
diff --git a/SECURITY.md b/SECURITY.md
index f7b2ba0..ea8bfae 100644
--- a/SECURITY.md
+++ b/SECURITY.md
@@ -456,11 +456,11 @@ URL, a keyword or a value in a JSON reply, a TCP port, a TLS certificate, a DNS
whitelist of metric names, cap the number of points and rows they read, and cache fleet answers for a
minute under keys made only from validated values, so a request cannot make them do unbounded work or
fill the cache. Text from the server (container names, mount paths) is shown as text, never as HTML.
-- **Retention.** Raw check-in samples (host, container, mount and system-event rows) are deleted once
- they are older than the retention period, an admin setting under *Admin → Data & retention* (7, 30, 90
- or 365 days); samples not yet summarised are never deleted, whatever the setting says. The small
- hourly/daily/weekly/monthly summaries are kept, so long-range history stays available while the bulky
- raw rows don't grow without bound.
+- **Retention.** Raw check-in samples (host, container and mount rows) and availability results are
+ deleted after 7 days. Hourly summaries and events are kept for the admin setting under *Admin → Data &
+ retention* (1 year by default, 2 years or forever; nothing shorter can be chosen); daily, weekly and
+ monthly summaries are kept forever. Nothing is deleted before the coarser level that replaces it exists,
+ so long-range history stays available while the bulky raw rows don't grow without bound.
## Other exposure to be aware of
diff --git a/TESTS.md b/TESTS.md
index c187661..6a924af 100644
--- a/TESTS.md
+++ b/TESTS.md
@@ -170,6 +170,7 @@ The chart helper tests for the frontend are separate, in `src/frontend/tests/`.
| [`test_network_checks.py`](#test_network_checkspy) | 94 | The host's real network totals, listening-port inventory, optional internet check, firewall fingerprints, and the validation of all of it. |
| [`test_pending_collectors.py`](#test_pending_collectorspy) | 28 | Adding servers: approval, join tokens, strict mode, re-pairing safety. |
| [`test_rollups.py`](#test_rollupspy) | 31 | Summarised history: hourly, daily, weekly and monthly rollups, catching up, gaps, and purging only what has been summarised. |
+| [`test_retention.py`](#test_retentionpy) | 12 | How long history is kept: check-ins a week, hourly summaries and events for the setting, coarser summaries forever, in chunks. |
| [`test_rules.py`](#test_rulespy) | 20 | Alert thresholds: sustained breaches, per-mount disks, settings validation. |
| [`test_second_review.py`](#test_second_reviewpy) | 44 | The second security review's findings: helper-event caps, lockouts, chat limits, lookalike hostnames, odd input. |
| [`test_stats.py`](#test_statspy) | 46 | Fleet statistics (incidents over time, availability) and the busy-hours heatmap: grouping, ranges, caching and validation. |
@@ -289,8 +290,8 @@ shouldn't, or an admin might be locked out.
| `test_first_registered_user_is_admin_and_later_ones_are_not` | The first account created becomes the admin; the second is a regular user. |
| `test_ensure_admin_exists_promotes_oldest_user_only_when_no_admin` | Upgrading an install from before roles existed promotes the oldest account to admin, only if there is no admin yet, and running it twice changes nothing more. |
| `test_require_admin_rejects_regular_and_disabled_users` | Admin-only actions check the role in the database: regular users and disabled admins are refused with 403. |
-| `test_admin_can_change_retention_within_limits` | An admin can change how long metrics are kept; an invalid value (0 days) is rejected with 400. |
-| `test_purge_old_metrics_deletes_only_samples_past_retention` | The hourly cleanup deletes only host and container samples older than the retention period and keeps recent ones. |
+| `test_admin_can_keep_history_for_one_year_two_years_or_forever_and_nothing_shorter` | History is kept 1 year by default; an admin can choose 2 years or forever, and anything shorter (7, 30, 364 days) or longer than offered is rejected with 400. |
+| `test_the_purge_deletes_check_ins_older_than_a_week` | The hourly cleanup deletes host and container samples older than 7 days and keeps recent ones. |
| `test_regular_user_sees_servers_but_not_pending_collectors` | Everyone signed in sees the servers, but only an admin sees the **New servers detected** list. |
| `test_regular_user_cannot_manage_servers_or_collectors` *(9 inputs)* | A regular user is refused (403) for every admin action: viewing, rotating or revoking the enrollment key, approving or dismissing a pending server, adding or deleting a server, changing a check interval, and reading admin settings. |
| `test_enrollment_key_stops_working_when_owner_is_no_longer_admin` | An enrollment key created by someone who was later demoted stops working. |
@@ -546,7 +547,7 @@ validation of container data, because everything a collector reports is untruste
| `test_sustained_churn_is_capped_per_hour` | Containers constantly coming and going cannot flood the database: events are capped per hour with one notice. |
| `test_container_changes_have_their_own_list_and_do_not_crowd_the_system_row` | Container changes appear in the Containers section, not in the System row, so they cannot push reboots out of it. |
| `test_the_container_change_list_is_bounded` | The dashboard's container-change list is capped at 15 rows. |
-| `test_container_events_follow_the_normal_retention` | Container change events are deleted with the other old data after the retention period. |
+| `test_container_events_are_kept_for_the_history_retention` | Container change events are kept for the history setting: 40 days old stays, 400 days old goes. |
**Memory-leak trends**
@@ -850,7 +851,7 @@ parsers odd, huge or hostile content.
| `test_the_overview_shows_listeners_firewall_and_network_events` | The dashboard receives the listeners, firewall fingerprints, events and internet status. |
| `test_network_events_have_their_own_list_and_do_not_crowd_the_other_rows` | Network events appear in the Network row and cannot push reboots or container changes out of theirs. |
| `test_the_network_lists_are_bounded` | The dashboard's listener list is capped at 200 and the event list at 15. |
-| `test_network_events_follow_the_normal_retention` | Old network events are deleted with the other old data. |
+| `test_network_events_are_kept_for_the_history_retention` | Network events are kept for the history setting: 40 days old stays, 400 days old goes. |
| `test_crafted_event_keys_cannot_raise_incidents_for_closed_or_loopback_ports` | Malformed or misleading stored event keys cannot raise an incident for a port that isn't open or isn't exposed. |
### test_pending_collectors.py
@@ -920,13 +921,30 @@ it was kept.
| `test_after_downtime_the_job_catches_up_a_chunk_at_a_time` | After downtime the job catches up in limited chunks until nothing is left. |
| `test_a_long_empty_stretch_cannot_stall_the_job` | A long stretch with no data at all still moves the job forward. |
| `test_the_watermark_is_stored_per_tier_and_a_damaged_one_is_ignored` | The progress marker is saved per tier, and a damaged one makes the job start over harmlessly. |
-| `test_without_any_summaries_the_purge_behaves_as_before` | With no summaries yet, retention alone decides what the purge deletes (unchanged behaviour). |
+| `test_without_any_summaries_no_check_in_is_purged` | With no summaries yet, no check-in is deleted, however old. |
| `test_samples_that_were_not_summarised_yet_are_never_purged` | The purge never deletes a sample newer than the last summarised hour, even when retention says it is old enough. |
| `test_maintenance_summarises_before_it_purges` | Maintenance summarises first, then purges, so history is kept before it is deleted. |
| `test_a_failing_rollup_job_stops_the_purge` | If summarising fails, nothing is purged that run. |
| `test_between_purges_maintenance_only_summarises` | Between hourly purges, maintenance only summarises. |
| `test_deleting_a_server_deletes_its_summaries_but_not_another_servers` | Deleting a server removes its summaries and leaves other servers' alone. |
+### test_retention.py
+
+How long history is kept (`src/backend/retention.py`).
+
+| Test | What it checks |
+|---|---|
+| `test_history_is_kept_a_year_unless_the_admin_chose_otherwise` | One year by default; the admin's choice is used. |
+| `test_a_stored_value_that_is_not_a_choice_means_the_default` *(3 inputs)* | A stored value that isn't 1 year, 2 years or forever falls back to one year. |
+| `test_hourly_summaries_older_than_the_setting_go_and_coarser_ones_stay` | Hourly host, mount and container summaries past a year go; daily and monthly ones stay. |
+| `test_two_years_keeps_what_one_year_would_delete` | The two-year setting keeps a 400-day-old hour. |
+| `test_forever_deletes_no_summary_and_no_event` | Forever deletes no hourly summary, event or availability hour. |
+| `test_hourly_summaries_stay_until_their_day_is_summarised` | An hour past the setting stays while its day has no summary yet. |
+| `test_events_older_than_the_setting_go` | Events past a year go, younger ones stay. |
+| `test_availability_results_go_after_a_week_and_their_hours_after_the_setting` | Availability results go after 7 days, their hourly summaries after the setting. |
+| `test_check_ins_go_after_a_week` | Check-in samples older than 7 days go. |
+| `test_a_big_purge_goes_in_chunks_and_still_deletes_everything_due` | Deleting in small chunks still deletes every row that is due. |
+
### test_rules.py
The alert thresholds, and the settings that control them.
diff --git a/docs/accounts.md b/docs/accounts.md
index 04a0818..f7dee36 100644
--- a/docs/accounts.md
+++ b/docs/accounts.md
@@ -18,8 +18,8 @@
- Only admins can add, approve, dismiss or delete servers, change check
intervals, manage the enrollment key and open the **Admin** page.
- **Admin** page: servers, users and invitations, alert rules and thresholds (including per-mount disk
- thresholds), how long metrics are kept (default 30 days; older check-in samples and system events are deleted
- hourly, but only after the check-ins have been summarised), and the incident agent.
+ thresholds), how long history is kept (check-ins 7 days; hourly summaries and events
+ 1 year by default, 2 years or forever; daily and coarser summaries forever), and the incident agent.
- **The incident agent and roles**: admins always see the agent's diagnoses and may use its chat. Whether other
users see diagnoses is the **Show diagnoses to users** switch in Admin → Incident agent (on by default). The chat
is for admins only by default; with **Chat access: Per user**, an **Agent chat** switch next to each user in
diff --git a/docs/api.md b/docs/api.md
index de07180..f02e53e 100644
--- a/docs/api.md
+++ b/docs/api.md
@@ -27,7 +27,7 @@ All paths are relative to `/api` when going through the frontend/nginx.
| `POST /monitoring/join-tokens` | admin | Create the single-use join token for **Connect a server** (shown once). |
| `GET /monitoring/install-info` | admin | The collector address and images the panel uses. |
| `GET` / `POST` / `DELETE /monitoring/enrollment-key` | admin | Show status / create or rotate / revoke the enrollment key. |
-| `GET` / `PUT /admin/settings` | admin | Metrics retention, chat on/off, require two-factor for admins, alert thresholds and per-mount disk thresholds; the agent's model prices, monthly budget, daily chat limit, automatic diagnosis, whether users see diagnoses, and chat access (`admins_only` or `per_user`). |
+| `GET` / `PUT /admin/settings` | admin | History retention (`history_retention_days`: 365, 730 or 0 for forever), chat on/off, require two-factor for admins, alert thresholds and per-mount disk thresholds; the agent's model prices, monthly budget, daily chat limit, automatic diagnosis, whether users see diagnoses, and chat access (`admins_only` or `per_user`). |
| `GET /admin/users`, `PATCH /admin/users/{id}` | admin | Users and invitations; change someone's role, switch them off, or (with per-user chat access) allow them the agent chat (`agent_chat_enabled`). |
| `GET /agent/config` | session | Whether chat with the agent can be used: switched on, allowed for you (`may_chat`), a key configured, and the monthly budget not used up. |
| `GET /incidents`, `GET /incidents/{id}` | session | The incident inbox; one incident with its rule-based diagnosis and, under `agent`, the agent's latest report (with `watch` progress while it is open, `reused_from` when it was reused). The report is left out (`hidden: true`) for users who aren't admins while diagnoses aren't shown to users. |
diff --git a/docs/database.md b/docs/database.md
index d7c558c..5344c8c 100644
--- a/docs/database.md
+++ b/docs/database.md
@@ -34,8 +34,13 @@ rule on that server opens no new incident (unless the condition breaks again, wh
samples into `host_rollups`, `mount_rollups` and `container_rollups`: one row per server, mount or container
per hour, day, week (Monday to Sunday) and month, all in UTC, with the average and maximum of each metric.
Each tier is built from the finer one when its period ends, a period without check-ins gets no row, and the
-purge never deletes samples that have not been summarised yet. The Statistics page reads them (see [Statistics](statistics.md)). They are
-not purged yet; a retention setting for each level is planned.
+purge never deletes samples that have not been summarised yet. The Statistics page reads them (see [Statistics](statistics.md)).
+
+**Retention** (`src/backend/retention.py`, hourly): check-in samples and `check_results` are kept 7 days, and
+only deleted once summarised. Hourly rollups, `check_rollups` and `system_events` are kept for the
+`history_retention_days` setting (365 by default, 730, or 0 for forever); an hourly rollup is only deleted
+once its day is summarised. Daily, weekly and monthly rollups are never deleted. Deletes run in chunks of
+10,000 rows.
The metrics tables carry only the indexes their queries use: `time` (for the hourly purge) and
`(server_id, time)`, or `(server_id, container_id, time)` for containers. A database from before
diff --git a/docs/metrics.md b/docs/metrics.md
index ab5e86c..9e1d912 100644
--- a/docs/metrics.md
+++ b/docs/metrics.md
@@ -146,7 +146,7 @@ Passive, and only what the collector can read without privileges.
## How history is stored
-The check-in snapshots are kept for the retention period (30 days by default). A background job also
+The check-in snapshots are kept for 7 days. A background job
summarises them into one row per **hour, day, week (Monday to Sunday) and month**, all in UTC, for
every server, filesystem and container. Each row stores, for every metric:
@@ -154,6 +154,10 @@ every server, filesystem and container. Each row stores, for every metric:
- the **highest value** (maximum) seen in a check-in,
- the number of check-ins that reported it (so averages stay exact when some check-ins lack a value).
+Hourly rows are kept for 1 year by default (2 years or forever in *Admin → Data & retention*); daily,
+weekly and monthly rows are kept forever. The further back you look, the coarser the history, but a spike
+stays visible through the maximum.
+
What that means on the Statistics page:
- A line is the **average of the snapshots** in each bucket. For CPU % that is the average of
diff --git a/docs/statistics.md b/docs/statistics.md
index 7ef09d1..ee636a7 100644
--- a/docs/statistics.md
+++ b/docs/statistics.md
@@ -50,4 +50,4 @@ What to know about the statistics:
has **View as table** as an alternative.
- **Not built yet** (tracked in Milestone 8 on GitHub): CSV and JSON export, reboots per month, a
state timeline and restarts per day for containers, rankings over the chosen range, network
- traffic charts, event markers on charts, and retention settings for the summaries.
+ traffic charts and event markers on charts.
diff --git a/src/backend/main.py b/src/backend/main.py
index 6c9f652..e89b7cf 100644
--- a/src/backend/main.py
+++ b/src/backend/main.py
@@ -27,7 +27,7 @@ from sqlalchemy import case, func
from .alerts import CONTAINER_RULE_PREFIX, CRITICAL, SERVER_OFFLINE_RULE, SEVERITIES, active_rule_keys, raise_alert, resolve_alerts, sync_alerts
import contextvars
-from . import auth, availability, dashboard, history as history_service, incidents, notify, rollups, secrets_box, stats, totp
+from . import auth, availability, dashboard, history as history_service, incidents, notify, retention, rollups, secrets_box, stats, totp
from .auth import (
STAGE_ENROLLMENT, STAGE_FULL, count_pending_attempt, create_pending_two_factor, create_session, delete_pending_two_factor, delete_session, get_recent_logs,
get_session, get_user_from_session, hash_password, issue_recovery_codes, list_user_sessions, pending_two_factor_remember, read_pending_two_factor, session_lifetime, recovery_codes_left, record_audit_log,
@@ -180,7 +180,7 @@ class ApproveRequest(BaseModel):
class AdminSettingsRequest(BaseModel):
# Both optional so the admin page can save one setting at a time.
- metrics_retention_days: int | None = None
+ history_retention_days: int | None = None
agent_chat_enabled: bool | None = None
require_2fa_for_admins: bool | None = None
alert_thresholds: dict | None = None
@@ -509,12 +509,6 @@ def ensure_admin_exists(db) -> None:
db.commit()
-METRICS_RETENTION_KEY = "metrics_retention_days"
-DEFAULT_METRICS_RETENTION_DAYS = 30
-MIN_METRICS_RETENTION_DAYS = 1
-MAX_METRICS_RETENTION_DAYS = 3650
-
-
AGENT_CHAT_ENABLED_KEY = "agent_chat_enabled"
REQUIRE_ADMIN_2FA_KEY = "require_admin_2fa"
TWO_FACTOR_ATTEMPTS = 5
@@ -531,14 +525,6 @@ def _set_setting(db, key: str, value: str) -> None:
db.add(AppSetting(key=key, value=value))
-def get_metrics_retention_days(db) -> int:
- setting = db.get(AppSetting, METRICS_RETENTION_KEY)
- try:
- return int(setting.value) if setting else DEFAULT_METRICS_RETENTION_DAYS
- except ValueError:
- return DEFAULT_METRICS_RETENTION_DAYS
-
-
def require_admin_two_factor(db) -> bool:
"""The admin setting: on unless an admin has turned it off."""
setting = db.get(AppSetting, REQUIRE_ADMIN_2FA_KEY)
@@ -608,27 +594,12 @@ def get_mount_thresholds(db) -> dict:
def _admin_settings(db) -> dict:
- return {"instance_name": get_instance_name(db), "metrics_retention_days": get_metrics_retention_days(db), "agent_chat_enabled": is_agent_chat_enabled(db), "alert_thresholds": get_alert_thresholds(db), "mount_thresholds": get_mount_thresholds(db), "require_2fa_for_admins": require_admin_two_factor(db), "secrets_configured": secrets_box.available(),
+ return {"instance_name": get_instance_name(db), "history_retention_days": retention.history_days(db), "agent_chat_enabled": is_agent_chat_enabled(db), "alert_thresholds": get_alert_thresholds(db), "mount_thresholds": get_mount_thresholds(db), "require_2fa_for_admins": require_admin_two_factor(db), "secrets_configured": secrets_box.available(),
"require_2fa_for_everyone": require_two_factor_for_everyone(db), "disabled_rules": sorted(get_disabled_rules(db)),
"agent_prices": agent_usage.get_prices(db), "agent_monthly_cap_usd": agent_usage.get_cap(db), "agent_daily_chat_limit": agent_usage.get_daily_limit(db), "agent_auto_diagnose": agent_settings.auto_diagnose(db),
"agent_show_diagnoses_to_users": agent_settings.show_diagnoses_to_users(db), "agent_chat_access": agent_settings.chat_access(db)}
-def purge_old_metrics(db, now: datetime | None = None) -> int:
- """Delete host and container samples older than the retention period."""
- cutoff = (now or datetime.now(timezone.utc)) - timedelta(days=get_metrics_retention_days(db))
- # Check-in samples that haven't been summarised yet are never deleted, whatever the retention says.
- summarised_until = rollups.hour_watermark(db)
- if summarised_until is not None:
- cutoff = min(cutoff, summarised_until)
- deleted = db.query(HostMetric).filter(HostMetric.time < cutoff).delete(synchronize_session=False)
- deleted += db.query(ContainerMetric).filter(ContainerMetric.time < cutoff).delete(synchronize_session=False)
- deleted += db.query(MountMetric).filter(MountMetric.time < cutoff).delete(synchronize_session=False)
- deleted += db.query(SystemEvent).filter(SystemEvent.time < cutoff).delete(synchronize_session=False)
- db.commit()
- return deleted
-
-
@app.get("/admin/settings")
def get_admin_settings(session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
_require_admin(session_id, db)
@@ -639,12 +610,12 @@ def get_admin_settings(session_id: Annotated[str | None, Cookie(alias=SESSION_CO
def update_admin_settings(payload: AdminSettingsRequest, session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_NAME)] = None, db=Depends(get_db)):
user = _require_admin(session_id, db)
changes = []
- days = payload.metrics_retention_days
- if days is not None:
- if not MIN_METRICS_RETENTION_DAYS <= days <= MAX_METRICS_RETENTION_DAYS:
- raise HTTPException(status_code=400, detail=f"Retention must be between {MIN_METRICS_RETENTION_DAYS} and {MAX_METRICS_RETENTION_DAYS} days.")
- _set_setting(db, METRICS_RETENTION_KEY, str(days))
- changes.append(f"Changed metrics retention to {days} days")
+ days = payload.history_retention_days
+ if days is not None and days != retention.history_days(db):
+ if days not in retention.HISTORY_CHOICES:
+ raise HTTPException(status_code=400, detail="History is kept for 1 year, 2 years or forever.")
+ _set_setting(db, retention.HISTORY_KEY, str(days))
+ changes.append(f"Changed history retention to {retention.describe(days)}")
if payload.require_2fa_for_admins is not None and payload.require_2fa_for_admins != require_admin_two_factor(db):
_set_setting(db, REQUIRE_ADMIN_2FA_KEY, "true" if payload.require_2fa_for_admins else "false")
changes.append(f"{'Required' if payload.require_2fa_for_admins else 'Stopped requiring'} two-factor for administrators")
@@ -982,8 +953,7 @@ def metrics_maintenance(db, purge_due: bool) -> bool:
return False
if not purge_due:
return False
- purge_old_metrics(db)
- availability.purge(db)
+ retention.purge(db)
analyze_memory_trends(db)
return True
diff --git a/src/backend/migrations/versions/0002_history_retention.py b/src/backend/migrations/versions/0002_history_retention.py
new file mode 100644
index 0000000..a0e40b2
--- /dev/null
+++ b/src/backend/migrations/versions/0002_history_retention.py
@@ -0,0 +1,28 @@
+"""History retention: check-ins for 7 days, hourly summaries and events for the admin's setting (retention.py).
+
+The old setting, ``metrics_retention_days``, kept check-ins and events for 1 to 3650 days (30 by default). Its
+meaning is gone, so it is removed and the new setting starts at its default, one year. The two indexes serve the
+purge of availability results and their hourly summaries.
+
+Revision 0002, after 0001.
+"""
+from alembic import op
+import sqlalchemy as sa
+
+
+revision = '0002'
+down_revision = '0001'
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+ # if_not_exists: a database the baseline brought up to date may already have them from create_all.
+ op.create_index('ix_check_results_time', 'check_results', ['time'], unique=False, if_not_exists=True)
+ op.create_index('ix_check_rollups_hour', 'check_rollups', ['hour'], unique=False, if_not_exists=True)
+ op.execute(sa.text("DELETE FROM app_settings WHERE key = 'metrics_retention_days'"))
+
+
+def downgrade() -> None:
+ op.drop_index('ix_check_rollups_hour', table_name='check_rollups')
+ op.drop_index('ix_check_results_time', table_name='check_results')
diff --git a/src/backend/models.py b/src/backend/models.py
index 82588ea..561e5d8 100644
--- a/src/backend/models.py
+++ b/src/backend/models.py
@@ -580,7 +580,8 @@ class CheckResult(Base):
"""One run of a check by one server's collector."""
__tablename__ = "check_results"
- __table_args__ = (Index("ix_check_results_check_time", "check_id", "time"),)
+ # (check_id, time) serves reading one check; time alone serves the purge.
+ __table_args__ = (Index("ix_check_results_check_time", "check_id", "time"), Index("ix_check_results_time", "time"))
id: Mapped[int] = mapped_column(Integer, primary_key=True)
check_id: Mapped[int] = mapped_column(ForeignKey("checks.id", ondelete="CASCADE"), nullable=False)
@@ -598,7 +599,8 @@ class CheckRollup(Base):
1-year and 90-day views are added up from these; the current hour comes from the raw results."""
__tablename__ = "check_rollups"
- __table_args__ = (UniqueConstraint("check_id", "hour"),)
+ # hour alone serves the purge (retention.py).
+ __table_args__ = (UniqueConstraint("check_id", "hour"), Index("ix_check_rollups_hour", "hour"))
id: Mapped[int] = mapped_column(Integer, primary_key=True)
check_id: Mapped[int] = mapped_column(ForeignKey("checks.id", ondelete="CASCADE"), index=True, nullable=False)
diff --git a/src/backend/retention.py b/src/backend/retention.py
new file mode 100644
index 0000000..5725e7b
--- /dev/null
+++ b/src/backend/retention.py
@@ -0,0 +1,74 @@
+"""How long history is kept, and deleting what is older.
+
+- Every check-in (host, container and filesystem samples) and every availability result: 7 days. Charts for
+ longer ranges read the summaries (rollups.py), so nothing past a week needs the per-minute rows.
+- Hourly summaries, the hourly availability summaries and events (reboots, OOM kills, kernel messages, container
+ and network changes): the admin's setting, one year by default, two years or forever. Never less than a year.
+- Daily, weekly and monthly summaries: forever. They are a few kilobytes per server a year.
+
+Nothing is deleted before the coarser level that replaces it exists: check-ins only once their hour is summarised,
+hourly summaries only once their day is. Deletes go in chunks, so the first purge on a big table doesn't lock it
+for long.
+"""
+from __future__ import annotations
+
+from datetime import datetime, timedelta, timezone
+
+from . import availability, rollups
+from .models import AppSetting, CheckRollup, ContainerMetric, ContainerRollup, HostMetric, HostRollup, MountMetric, MountRollup, SystemEvent
+
+DETAILED_KEPT = timedelta(days=7)
+HISTORY_KEY = "history_retention_days"
+FOREVER = 0
+HISTORY_CHOICES = (365, 730, FOREVER)
+DEFAULT_HISTORY_DAYS = 365
+CHUNK = 10_000
+
+
+def history_days(db) -> int:
+ """How many days hourly summaries and events are kept; FOREVER (0) keeps them."""
+ setting = db.get(AppSetting, HISTORY_KEY)
+ try:
+ days = int(setting.value) if setting else DEFAULT_HISTORY_DAYS
+ except ValueError:
+ return DEFAULT_HISTORY_DAYS
+ return days if days in HISTORY_CHOICES else DEFAULT_HISTORY_DAYS
+
+
+def describe(days: int) -> str:
+ return "forever" if days == FOREVER else "1 year" if days == 365 else f"{days // 365} years"
+
+
+def _delete_in_chunks(db, model, *conditions) -> int:
+ total = 0
+ while True:
+ ids = [row_id for (row_id,) in db.query(model.id).filter(*conditions).limit(CHUNK)]
+ if not ids:
+ return total
+ db.query(model).filter(model.id.in_(ids)).delete(synchronize_session=False)
+ db.commit()
+ total += len(ids)
+
+
+def purge(db, now: datetime | None = None) -> int:
+ """Delete everything past its retention. Returns how many rows went."""
+ now = now or datetime.now(timezone.utc)
+ deleted = 0
+ summarised_until = rollups.hour_watermark(db)
+ if summarised_until is not None:
+ cutoff = min(now - DETAILED_KEPT, summarised_until)
+ for model in (HostMetric, ContainerMetric, MountMetric):
+ deleted += _delete_in_chunks(db, model, model.time < cutoff)
+ deleted += availability.purge(db, now)
+ days = history_days(db)
+ if days == FOREVER:
+ return deleted
+ cutoff = now - timedelta(days=days)
+ deleted += _delete_in_chunks(db, SystemEvent, SystemEvent.time < cutoff)
+ deleted += _delete_in_chunks(db, CheckRollup, CheckRollup.hour < cutoff)
+ days_until = rollups._read_watermark(db, rollups.DAY)
+ if days_until is not None:
+ cutoff = min(cutoff, days_until)
+ for model in (HostRollup, MountRollup, ContainerRollup):
+ deleted += _delete_in_chunks(db, model, model.tier == rollups.HOUR, model.bucket_start < cutoff)
+ return deleted
diff --git a/src/backend/routes/admin.py b/src/backend/routes/admin.py
index 0304864..a9c4851 100644
--- a/src/backend/routes/admin.py
+++ b/src/backend/routes/admin.py
@@ -130,7 +130,7 @@ def admin_system(session_id: Annotated[str | None, Cookie(alias=SESSION_COOKIE_N
return {
"version": APP_VERSION, "database": {"engine": engine.dialect.name, "bytes": size,
"host_rows": db.query(func.count(HostMetric.id)).scalar(), "host_rows_week": db.query(func.count(HostMetric.id)).filter(HostMetric.time >= week).scalar()},
- "retention_days": main.get_metrics_retention_days(db),
+ "history_retention_days": main.retention.history_days(db),
"agent": {"provider": "OpenAI", "model": AGENT_MODEL, "key_configured": bool(configured_key()), "budget": agent_usage.budget(db), "chat_enabled": main.is_agent_chat_enabled(db),
"incidents_this_month": db.query(IncidentAlert).filter(IncidentAlert.created_at >= month).count(),
"chat_messages_this_month": db.query(ChatMessage).filter(ChatMessage.role == "user", ChatMessage.created_at >= month).count()},
diff --git a/src/backend/routes/monitoring.py b/src/backend/routes/monitoring.py
index fc5142a..6be79a7 100644
--- a/src/backend/routes/monitoring.py
+++ b/src/backend/routes/monitoring.py
@@ -146,7 +146,7 @@ def monitoring_server_events(server_id: main.PathId, days: int = 7, session_id:
if not server:
raise main.HTTPException(status_code=404, detail="Server not found.")
days = max(1, min(days, main.EVENTS_MAX_DAYS))
- return {"server_id": server.id, "days": days, "retention_days": main.get_metrics_retention_days(db), "events": main.dashboard.server_events(db, server, days)}
+ return {"server_id": server.id, "days": days, "retention_days": main.retention.history_days(db), "events": main.dashboard.server_events(db, server, days)}
@router.get("/monitoring/servers/{server_id}/mounts/history")
def mount_history(
diff --git a/src/frontend/src/admin/AdminPage.jsx b/src/frontend/src/admin/AdminPage.jsx
index c75b640..ccf553f 100644
--- a/src/frontend/src/admin/AdminPage.jsx
+++ b/src/frontend/src/admin/AdminPage.jsx
@@ -12,7 +12,7 @@ import { Caret, Icon } from '../icons.jsx'
import { refreshOverview, useOverview } from '../overviewStore.js'
import { AppShell, serverLink } from '../shell.jsx'
import { ADMIN_GROUPS } from '../more/groups.js'
-import { Field, Notice, Outcome, Row, Section, SettingsLayout, ShownOnce, Soon, Switch, ago, useAction, useLoaded, usePhone } from '../ui.jsx'
+import { Field, Notice, Outcome, Row, Section, SettingsLayout, ShownOnce, Soon, Switch, ago, keptFor, useAction, useLoaded, usePhone } from '../ui.jsx'
const REPOSITORY = 'https://github.com/karloks2005/Infradeck'
const STATE_DOT = { critical: 'dot-signal', warn: 'dot-warn', ok: 'dot-ok', offline: '' }
@@ -442,11 +442,13 @@ function Retention({ settings, system, onSaved }) {
const database = system.database
return (
<div className="card divide-y divide-line">
- <Row title="Check-ins and their summaries" detail="Every check-in is kept for a week; hourly, daily, weekly and monthly summaries for as long as this says. Statistics reads the summaries for ranges past 7 days.">
- <select className="select" aria-label="Retention" value={settings.metrics_retention_days} disabled={state.busy} onChange={(event) => run(async () => onSaved(await put({ metrics_retention_days: Number(event.target.value) })), 'Saved.')}>
- {[...new Set([7, 30, 90, 365, settings.metrics_retention_days])].sort((a, b) => a - b).map((days) => <option key={days} value={days}>{days === 365 ? '1 year' : `${days} days`}</option>)}
+ <Row title="Every check-in" detail="Each server's report, about one a minute, and every availability result. Charts for ranges past 7 days read the summaries below."><span className="text-[13px] text-soft">7 days</span></Row>
+ <Row title="Hourly summaries and events" detail="The average and peak of every metric per hour, each check's uptime per hour, and events such as reboots, OOM kills and port changes. Older hours are deleted once their day is summarised.">
+ <select className="select" aria-label="Keep hourly summaries and events" value={settings.history_retention_days} disabled={state.busy} onChange={(event) => run(async () => onSaved(await put({ history_retention_days: Number(event.target.value) })), 'Saved.')}>
+ {[365, 730, 0].map((days) => <option key={days} value={days}>{keptFor(days)[0].toUpperCase() + keptFor(days).slice(1)}</option>)}
</select>
</Row>
+ <Row title="Daily, weekly and monthly summaries" detail="A few kilobytes per server a year, so they are never deleted."><span className="text-[13px] text-soft">Forever</span></Row>
<Row title="Incidents" detail="Kept until their server is removed."><span className="text-[13px] text-soft">Forever</span></Row>
<Row icon="db" title="Database" detail={`${database.engine}${database.host_rows != null ? ` · ${database.host_rows.toLocaleString()} check-ins stored, ${database.host_rows_week.toLocaleString()} from the last 7 days` : ''}`}>
<span className="mr-2 text-lg font-semibold text-paper">{megabytes(database.bytes)}</span>
diff --git a/src/frontend/src/dashboard/EventsTab.jsx b/src/frontend/src/dashboard/EventsTab.jsx
index a0bf9be..e8b2f1a 100644
--- a/src/frontend/src/dashboard/EventsTab.jsx
+++ b/src/frontend/src/dashboard/EventsTab.jsx
@@ -2,6 +2,7 @@
import { useMemo, useState } from 'react'
import { Link } from 'react-router-dom'
import { Icon } from '../icons.jsx'
+import { keptFor } from '../ui.jsx'
import { usePolled } from './data.js'
import { EVENT_SOURCES, dayHeading, describeEvent, describeGroup, eventCategory, eventStrip, eventsToCsv, groupRepeats } from './logic.js'
@@ -124,7 +125,7 @@ export function EventsTab({ server, pending, isAdmin, onReview }) {
</div>
)) : <p className="m-0 px-[18px] py-10 text-center text-[13px] text-muted">{events.length ? 'No events match this search.' : `Nothing happened on ${server.hostname} in the last ${DAYS} days.`}</p>}
<div className="flex flex-wrap items-center gap-x-4 gap-y-1 border-t border-line px-[18px] py-3 text-xs text-muted">
- <span>Showing {visible.length} of {groups.length} {grouped ? 'grouped ' : ''}events from the last {DAYS} days. Events are kept for {data.retention_days} days.</span>
+ <span>Showing {visible.length} of {groups.length} {grouped ? 'grouped ' : ''}events from the last {DAYS} days. Events are kept {data.retention_days === 0 ? 'forever' : `for ${keptFor(data.retention_days)}`}.</span>
{groups.length > visible.length && <button type="button" className="link" onClick={() => setLimit((value) => value + PAGE)}>Show {Math.min(PAGE, groups.length - visible.length)} more</button>}
</div>
</>
diff --git a/src/frontend/src/ui.jsx b/src/frontend/src/ui.jsx
index 86e28f9..4de04c0 100644
--- a/src/frontend/src/ui.jsx
+++ b/src/frontend/src/ui.jsx
@@ -199,6 +199,9 @@ export function ShownOnce({ value, children }) {
)
}
+// How long the history setting keeps something: 365 → "1 year", 0 → "forever".
+export const keptFor = (days) => (days === 0 ? 'forever' : days === 365 ? '1 year' : `${Math.round(days / 365)} years`)
+
export const formatDate = (time) => new Date(time).toLocaleDateString([], { day: 'numeric', month: 'short', year: 'numeric' })
// "4 s ago", "2 h ago", "6 d ago", or the date once it is more than a month back.
diff --git a/tests/accounts/test_admin.py b/tests/accounts/test_admin.py
index 30acafe..c3ff07b 100644
--- a/tests/accounts/test_admin.py
+++ b/tests/accounts/test_admin.py
@@ -63,25 +63,27 @@ def test_require_admin_rejects_regular_and_disabled_users(db, monkeypatch):
assert error.value.status_code == 403
-def test_admin_can_change_retention_within_limits(db, monkeypatch):
+def test_admin_can_keep_history_for_one_year_two_years_or_forever_and_nothing_shorter(db, monkeypatch):
db.add(User(id=1, name="Admin", email="[email protected]", password_hash="x", role="admin"))
db.commit()
_login_as(monkeypatch, 1)
- assert main.get_admin_settings(session_id="s", db=db)["metrics_retention_days"] == main.DEFAULT_METRICS_RETENTION_DAYS
- main.update_admin_settings(main.AdminSettingsRequest(metrics_retention_days=7), session_id="s", db=db)
- assert main.get_admin_settings(session_id="s", db=db)["metrics_retention_days"] == 7
+ assert main.get_admin_settings(session_id="s", db=db)["history_retention_days"] == 365
+ for days in (730, 0, 365):
+ main.update_admin_settings(main.AdminSettingsRequest(history_retention_days=days), session_id="s", db=db)
+ assert main.get_admin_settings(session_id="s", db=db)["history_retention_days"] == days
- with pytest.raises(HTTPException) as error:
- main.update_admin_settings(main.AdminSettingsRequest(metrics_retention_days=0), session_id="s", db=db)
- assert error.value.status_code == 400
+ for days in (7, 30, 364, 3650):
+ with pytest.raises(HTTPException) as error:
+ main.update_admin_settings(main.AdminSettingsRequest(history_retention_days=days), session_id="s", db=db)
+ assert error.value.status_code == 400
-def test_purge_old_metrics_deletes_only_samples_past_retention(db):
+def test_the_purge_deletes_check_ins_older_than_a_week(db):
now = datetime(2026, 9, 25, tzinfo=timezone.utc)
db.add(User(id=1, name="Owner", email="[email protected]", password_hash="x"))
db.add(Server(id=1, owner_id=1, hostname="nas"))
- db.add(AppSetting(key=main.METRICS_RETENTION_KEY, value="7"))
+ db.add(AppSetting(key="rollup_done_hour", value=now.isoformat())) # everything is summarised
db.add_all([
HostMetric(server_id=1, time=now - timedelta(days=8)),
HostMetric(server_id=1, time=now - timedelta(days=6)),
@@ -90,7 +92,7 @@ def test_purge_old_metrics_deletes_only_samples_past_retention(db):
])
db.commit()
- assert main.purge_old_metrics(db, now=now) == 2
+ assert main.retention.purge(db, now=now) == 2
assert db.query(HostMetric).count() == 1
assert db.query(ContainerMetric).count() == 1
@@ -156,7 +158,7 @@ def test_admin_can_turn_agent_chat_off_and_on(fleet, monkeypatch):
result = main.update_admin_settings(main.AdminSettingsRequest(agent_chat_enabled=False), session_id="s", db=fleet)
- assert result == {"instance_name": main.get_instance_name(fleet), "metrics_retention_days": main.DEFAULT_METRICS_RETENTION_DAYS, "agent_chat_enabled": False, "alert_thresholds": rules.DEFAULT_THRESHOLDS, "mount_thresholds": {}, "require_2fa_for_admins": True, "secrets_configured": main.secrets_box.available(), "require_2fa_for_everyone": False, "disabled_rules": [],
+ assert result == {"instance_name": main.get_instance_name(fleet), "history_retention_days": 365, "agent_chat_enabled": False, "alert_thresholds": rules.DEFAULT_THRESHOLDS, "mount_thresholds": {}, "require_2fa_for_admins": True, "secrets_configured": main.secrets_box.available(), "require_2fa_for_everyone": False, "disabled_rules": [],
"agent_prices": {}, "agent_monthly_cap_usd": None, "agent_daily_chat_limit": main.agent_usage.DEFAULT_DAILY_CHAT_LIMIT, "agent_auto_diagnose": True,
"agent_show_diagnoses_to_users": True, "agent_chat_access": "admins_only"}
_login_as(monkeypatch, 2)
diff --git a/tests/history/test_migrations.py b/tests/history/test_migrations.py
index 4a0df08..cd6e556 100644
--- a/tests/history/test_migrations.py
+++ b/tests/history/test_migrations.py
@@ -70,11 +70,14 @@ def test_a_database_from_before_migrations_keeps_its_data_and_ends_at_the_latest
Base.metadata.create_all(engine) # how the schema was made before Alembic
with engine.begin() as connection:
connection.execute(text("INSERT INTO app_settings (key, value, updated_at) VALUES ('instance_name', 'lab', CURRENT_TIMESTAMP)"))
+ connection.execute(text("INSERT INTO app_settings (key, value, updated_at) VALUES ('metrics_retention_days', '30', CURRENT_TIMESTAMP)"))
upgrade_database(engine)
assert _version(engine) == _head()
assert _differences(engine) == []
with engine.connect() as connection:
assert connection.execute(text("SELECT value FROM app_settings WHERE key = 'instance_name'")).scalar() == "lab"
+ # 0002: the old retention setting has no meaning any more
+ assert connection.execute(text("SELECT count(*) FROM app_settings WHERE key = 'metrics_retention_days'")).scalar() == 0
def test_an_old_database_missing_tables_and_columns_is_brought_up_to_date(engine):
diff --git a/tests/history/test_retention.py b/tests/history/test_retention.py
new file mode 100644
index 0000000..6518d7c
--- /dev/null
+++ b/tests/history/test_retention.py
@@ -0,0 +1,142 @@
+"""How long history is kept (retention.py): check-ins and availability results 7 days; hourly summaries, hourly
+availability summaries and events for the admin's setting; daily and coarser summaries forever."""
+import os
+from datetime import datetime, timedelta, timezone
+
+os.environ.setdefault("DATABASE_URL", "sqlite://")
+os.environ.setdefault("REDIS_URL", "redis://localhost:6379/0")
+
+import pytest
+from sqlalchemy import create_engine
+from sqlalchemy.orm import sessionmaker
+
+from src.backend import retention, rollups
+from src.backend.models import AppSetting, Base, Check, CheckResult, CheckRollup, ContainerRollup, HostMetric, HostRollup, MountRollup, Server, SystemEvent, User
+
+NOW = datetime(2027, 6, 15, 12, tzinfo=timezone.utc)
+
+
+def db():
+ engine = create_engine("sqlite://")
+ Base.metadata.create_all(engine)
+ session = sessionmaker(bind=engine)()
+ session.add(User(id=1, name="Owner", email="[email protected]", password_hash="x", role="admin"))
+ session.add(Server(id=1, owner_id=1, hostname="nas", check_interval_seconds=60))
+ session.add(Check(id=1, name="web", type="http", target="http://web.lan/"))
+ # Every tier is summarised up to now.
+ session.add_all(AppSetting(key=rollups.WATERMARK_PREFIX + tier, value=NOW.isoformat()) for tier in rollups.TIERS)
+ session.commit()
+ return session
+
+
+def _setting(db, days):
+ db.add(AppSetting(key=retention.HISTORY_KEY, value=str(days)))
+ db.commit()
+
+
+def _summaries(db, tier, *days_back):
+ for days in days_back:
+ start = NOW - timedelta(days=days)
+ db.add(HostRollup(server_id=1, tier=tier, bucket_start=start, samples=60))
+ db.add(MountRollup(server_id=1, mount="/", tier=tier, bucket_start=start, samples=60))
+ db.add(ContainerRollup(server_id=1, container_id="c", name="web", tier=tier, bucket_start=start, samples=60))
+ db.commit()
+
+
+def _ages(db, model=HostRollup, tier=rollups.HOUR):
+ return sorted((NOW - rollups._utc(row.bucket_start)).days for row in db.query(model).filter_by(tier=tier))
+
+
+def test_history_is_kept_a_year_unless_the_admin_chose_otherwise(db):
+ assert retention.history_days(db) == 365
+ _setting(db, 730)
+ assert retention.history_days(db) == 730
+
+
[email protected]("stored", ["30", "soon", "-1"])
+def test_a_stored_value_that_is_not_a_choice_means_the_default(db, stored):
+ _setting(db, stored)
+ assert retention.history_days(db) == 365
+
+
+def test_hourly_summaries_older_than_the_setting_go_and_coarser_ones_stay(db):
+ _summaries(db, rollups.HOUR, 10, 300, 400)
+ _summaries(db, rollups.DAY, 10, 400, 900)
+ _summaries(db, rollups.MONTH, 900)
+
+ retention.purge(db, NOW)
+
+ for model in (HostRollup, MountRollup, ContainerRollup):
+ assert _ages(db, model) == [10, 300]
+ assert _ages(db, model, rollups.DAY) == [10, 400, 900]
+ assert _ages(db, model, rollups.MONTH) == [900]
+
+
+def test_two_years_keeps_what_one_year_would_delete(db):
+ _setting(db, 730)
+ _summaries(db, rollups.HOUR, 400, 800)
+
+ retention.purge(db, NOW)
+
+ assert _ages(db) == [400]
+
+
+def test_forever_deletes_no_summary_and_no_event(db):
+ _setting(db, retention.FOREVER)
+ _summaries(db, rollups.HOUR, 2000)
+ db.add(SystemEvent(server_id=1, time=NOW - timedelta(days=2000), kind="reboot", detail="x"))
+ db.add(CheckRollup(check_id=1, hour=NOW - timedelta(days=2000), slots=60, down_slots=0))
+ db.commit()
+
+ retention.purge(db, NOW)
+
+ assert _ages(db) == [2000] and db.query(SystemEvent).count() == 1 and db.query(CheckRollup).count() == 1
+
+
+def test_hourly_summaries_stay_until_their_day_is_summarised(db):
+ db.get(AppSetting, rollups.WATERMARK_PREFIX + rollups.DAY).value = (NOW - timedelta(days=500)).isoformat()
+ db.commit()
+ _summaries(db, rollups.HOUR, 400, 600)
+
+ retention.purge(db, NOW)
+
+ assert _ages(db) == [400] # past the year, but its day hasn't been summarised yet
+
+
+def test_events_older_than_the_setting_go(db):
+ db.add_all(SystemEvent(server_id=1, time=NOW - timedelta(days=days), kind="reboot", detail="x") for days in (40, 364, 366))
+ db.commit()
+
+ retention.purge(db, NOW)
+
+ assert sorted((NOW - rollups._utc(row.time)).days for row in db.query(SystemEvent)) == [40, 364]
+
+
+def test_availability_results_go_after_a_week_and_their_hours_after_the_setting(db):
+ db.add_all(CheckResult(check_id=1, server_id=1, time=NOW - timedelta(days=days), ok=True) for days in (6, 8))
+ db.add_all(CheckRollup(check_id=1, hour=NOW - timedelta(days=days), slots=60, down_slots=0) for days in (8, 300, 400))
+ db.commit()
+
+ retention.purge(db, NOW)
+
+ assert [(NOW - rollups._utc(row.time)).days for row in db.query(CheckResult)] == [6]
+ assert sorted((NOW - rollups._utc(row.hour)).days for row in db.query(CheckRollup)) == [8, 300]
+
+
+def test_check_ins_go_after_a_week(db):
+ db.add_all(HostMetric(server_id=1, time=NOW - timedelta(days=days)) for days in (1, 6, 8, 30))
+ db.commit()
+
+ retention.purge(db, NOW)
+
+ assert sorted((NOW - rollups._utc(row.time)).days for row in db.query(HostMetric)) == [1, 6]
+
+
+def test_a_big_purge_goes_in_chunks_and_still_deletes_everything_due(db, monkeypatch):
+ monkeypatch.setattr(retention, "CHUNK", 2)
+ db.add_all(HostMetric(server_id=1, time=NOW - timedelta(days=10, minutes=minute)) for minute in range(5))
+ db.commit()
+
+ assert retention.purge(db, NOW) == 5
+ assert db.query(HostMetric).count() == 0
diff --git a/tests/history/test_rollups.py b/tests/history/test_rollups.py
index 2040058..de11a29 100644
--- a/tests/history/test_rollups.py
+++ b/tests/history/test_rollups.py
@@ -360,39 +360,31 @@ def test_the_watermark_is_stored_per_tier_and_a_damaged_one_is_ignored(db):
# --- purging only what has been summarised ---------------------------------------------------
-def _set_retention(db, days):
- db.add(AppSetting(key=main.METRICS_RETENTION_KEY, value=str(days)))
- db.commit()
-
-
-def test_without_any_summaries_the_purge_behaves_as_before(db):
- _set_retention(db, 1)
+def test_without_any_summaries_no_check_in_is_purged(db):
host(db, T(2026, 3, 1, 10, 0), cpu_utilization_pct=1.0)
- host(db, T(2026, 3, 5, 10, 0), cpu_utilization_pct=2.0)
+ host(db, T(2026, 3, 20, 10, 0), cpu_utilization_pct=2.0)
- main.purge_old_metrics(db, now=T(2026, 3, 5, 12, 0))
+ main.retention.purge(db, now=T(2026, 3, 20, 12, 0))
- assert db.query(HostMetric).count() == 1
+ assert db.query(HostMetric).count() == 2
def test_samples_that_were_not_summarised_yet_are_never_purged(db):
- _set_retention(db, 1)
- for day in range(1, 5):
+ for day in range(1, 10):
host(db, T(2026, 3, day, 10, 0), cpu_utilization_pct=float(day))
db.add(AppSetting(key="rollup_done_hour", value=T(2026, 3, 2, 12).isoformat())) # only the first day and a half are summarised
db.commit()
- main.purge_old_metrics(db, now=T(2026, 3, 5, 12, 0)) # retention alone would delete everything before 4 March
+ main.retention.purge(db, now=T(2026, 3, 12, 12, 0)) # the week alone would delete everything before 5 March
- # 1 and 2 March are summarised and old enough; 3 March is old enough but not summarised yet, so it stays
- assert [rollups._utc(row.time).day for row in db.query(HostMetric).order_by(HostMetric.time)] == [3, 4]
+ # 1 and 2 March are summarised and old enough; 3 and 4 March are old enough but not summarised yet, so they stay
+ assert [rollups._utc(row.time).day for row in db.query(HostMetric).order_by(HostMetric.time)] == [3, 4, 5, 6, 7, 8, 9]
def test_maintenance_summarises_before_it_purges(db):
- _set_retention(db, 1)
- for days_back, cpu in ((4, 1.0), (3, 2.0)):
+ for days_back, cpu in ((9, 1.0), (8, 2.0)):
host(db, ago(days_back), cpu_utilization_pct=cpu)
- db.add(ContainerMetric(server_id=1, time=ago(4), container_id="c", name="web", status="running"))
+ db.add(ContainerMetric(server_id=1, time=ago(9), container_id="c", name="web", status="running"))
host(db, datetime.now(timezone.utc) - timedelta(minutes=1), cpu_utilization_pct=99.0) # recent: kept
db.commit()
@@ -403,8 +395,7 @@ def test_maintenance_summarises_before_it_purges(db):
def test_a_failing_rollup_job_stops_the_purge(db, monkeypatch):
- _set_retention(db, 1)
- host(db, ago(4), cpu_utilization_pct=1.0)
+ host(db, ago(9), cpu_utilization_pct=1.0)
def broken(*args, **kwargs):
raise RuntimeError("database trouble")
diff --git a/tests/monitoring/test_container_checks.py b/tests/monitoring/test_container_checks.py
index 4198575..8d932ce 100644
--- a/tests/monitoring/test_container_checks.py
+++ b/tests/monitoring/test_container_checks.py
@@ -483,13 +483,13 @@ def test_the_container_change_list_is_bounded(db, monkeypatch):
assert len(monitoring_routes.monitoring_overview(session_id="s", db=db)["servers"][0]["container_events"]) == main.CONTAINER_EVENTS_SHOWN
-def test_container_events_follow_the_normal_retention(db):
- old = datetime.now(timezone.utc) - timedelta(days=40)
- db.add_all([SystemEvent(server_id=5, time=old, kind=kind, detail="x", event_key="x") for kind in rules.CONTAINER_EVENT_KINDS])
+def test_container_events_are_kept_for_the_history_retention(db):
+ now = datetime.now(timezone.utc)
+ db.add_all([SystemEvent(server_id=5, time=now - timedelta(days=days), kind=kind, detail="x", event_key="x") for kind in rules.CONTAINER_EVENT_KINDS for days in (40, 400)])
db.commit()
- assert main.purge_old_metrics(db) == len(rules.CONTAINER_EVENT_KINDS)
- assert db.query(SystemEvent).count() == 0
+ assert main.retention.purge(db) == len(rules.CONTAINER_EVENT_KINDS) # the ones from 400 days ago; a year is the default
+ assert db.query(SystemEvent).count() == len(rules.CONTAINER_EVENT_KINDS)
# --- backend: memory leak trends (#20) --------------------------------------------------------------
diff --git a/tests/monitoring/test_network_checks.py b/tests/monitoring/test_network_checks.py
index c575e66..ce126a6 100644
--- a/tests/monitoring/test_network_checks.py
+++ b/tests/monitoring/test_network_checks.py
@@ -619,12 +619,12 @@ def test_the_network_lists_are_bounded(db, monkeypatch):
assert len(network["listeners"]) == main.LISTENERS_SHOWN and len(network["events"]) == main.NETWORK_EVENTS_SHOWN
-def test_network_events_follow_the_normal_retention(db):
- old = datetime.now(timezone.utc) - timedelta(days=40)
- db.add_all([SystemEvent(server_id=5, time=old, kind=kind, detail="x", event_key="x") for kind in rules.NETWORK_EVENT_KINDS])
+def test_network_events_are_kept_for_the_history_retention(db):
+ now = datetime.now(timezone.utc)
+ db.add_all([SystemEvent(server_id=5, time=now - timedelta(days=days), kind=kind, detail="x", event_key="x") for kind in rules.NETWORK_EVENT_KINDS for days in (40, 400)])
db.commit()
- assert main.purge_old_metrics(db) == len(rules.NETWORK_EVENT_KINDS)
+ assert main.retention.purge(db) == len(rules.NETWORK_EVENT_KINDS) # the ones from 400 days ago; a year is the default
def test_crafted_event_keys_cannot_raise_incidents_for_closed_or_loopback_ports(db):