Self-hosted monitoring for homelabs: host and container metrics and uptime checks from all your machines in one dashboard, with an optional AI agent that diagnoses incidents and suggests fixes.
Python ai-agentsaiopsdashboarddockerfastapihomelabincident-managementllmmonitoringopenaireactroot-cause-analysisself-hosteduptime-monitoring
fix(security): run the backend, frontend, Redis and PostgreSQL containers with reduced privileges
Karlo Kajba Šimanićce96bafe20ca
diff --git a/SECURITY.md b/SECURITY.mdindex 37964b2..edcdd40 100644--- a/SECURITY.md+++ b/SECURITY.md@@ -468,6 +468,30 @@ URL, a keyword or a value in a JSON reply, a TCP port, a TLS certificate, a DNSmonthly summaries are kept forever. Nothing is deleted before the coarser level that replaces it exists,so long-range history stays available while the bulky raw rows don't grow without bound.+## Container privileges++Every container the stack builds or runs runs with `no-new-privileges` and a read-only filesystem, and drops+all Linux capabilities, adding back only the few listed here. The only exception is the optional Caddy of the+development stack's `https` profile, which uses the official image's defaults.++| Container | Runs as | Capabilities | Filesystem |+|---|---|---|---|+| backend, agent-worker | UID 10001 | none | read-only, `/tmp` in memory |+| frontend | `nginx` (port 8080 inside the container, so it needs no capability to listen) | none | read-only; cache, pid file and the `TRUSTED_PROXIES` file in memory |+| redis | `redis` | none while running | read-only, data in its volume |+| postgres | `postgres` | none while running | read-only, data in its volume, socket and `/tmp` in memory |+| collector | root | none, so it can't read other users' files (see below) | read-only |+| collector-gate | root, workers as `nginx` | `CHOWN`, `SETUID`, `SETGID` | read-only, cache and pid in memory |+| docker-proxy | root (to open the Docker socket) | none | read-only, no network |+| system-helper, port-owners (optional) | root | `SYSLOG`; `SYS_PTRACE` and `DAC_READ_SEARCH` (see their sections above) | read-only, no network |++**Exceptions, and why.** The official Redis and PostgreSQL images start as root to make their data folder+belong to their own user (and, for PostgreSQL, give it mode `0700`), then switch to that user. That start needs+`CHOWN`, `DAC_READ_SEARCH` (to walk a data folder only its owner can read), `SETUID` and `SETGID`, and+PostgreSQL also `FOWNER` (to set the mode). The database process keeps none of them: switching away from root+clears them. Without the hand-over, an existing `redis_data` volume written by an older version (when Redis+ran as root) would no longer be writable.+## Other exposure to be aware of- The collector mounts the host filesystem read-only at `/hostfs` to read disk usage and a fewdiff --git a/compose.yml b/compose.ymlindex 57046fa..e87f96b 100644--- a/compose.yml+++ b/compose.yml@@ -4,6 +4,16 @@ services:image: postgres:16-alpinecontainer_name: infradeck-postgresrestart: unless-stopped+ # The image's entrypoint prepares the data folder as root (owner and 0700 mode), then runs PostgreSQL as the+ # postgres user, which keeps none of these. Its socket and lock files live in memory.+ cap_drop: [ALL]+ cap_add: [CHOWN, DAC_READ_SEARCH, FOWNER, SETUID, SETGID]+ read_only: true+ tmpfs:+ - /var/run/postgresql:uid=70,gid=70+ - /tmp+ security_opt:+ - no-new-privileges:truenetworks: [data]environment:POSTGRES_DB: ${POSTGRES_DB:-infradeck}@@ -24,7 +34,14 @@ services:networks: [data]# Needs a password: Redis holds the login sessions, so nothing that can merely reach it on the network# may use it. Generate one with `openssl rand -hex 24` (letters and digits only, so it fits in a URL).- command: ["sh", "-c", "exec redis-server --appendonly yes --requirepass \"$$REDIS_PASSWORD\""]+ # Through the image's entrypoint, which hands /data to the redis user and drops root before Redis starts.+ command: ["sh", "-c", "exec docker-entrypoint.sh redis-server --appendonly yes --requirepass \"$$REDIS_PASSWORD\""]+ # Only what that hand-over needs: reading and chowning /data, and switching to the redis user.+ cap_drop: [ALL]+ cap_add: [CHOWN, DAC_READ_SEARCH, SETUID, SETGID]+ read_only: true+ security_opt:+ - no-new-privileges:trueenvironment:REDIS_PASSWORD: ${REDIS_PASSWORD:?Set REDIS_PASSWORD in .env (generate one with openssl rand -hex 24)}volumes:@@ -42,6 +59,13 @@ services:container_name: infradeck-backendimage: infradeck-backend:latestrestart: unless-stopped+ # Runs as an unprivileged user (see the Dockerfile) with no capabilities and a read-only filesystem.+ cap_drop: [ALL]+ read_only: true+ tmpfs:+ - /tmp+ security_opt:+ - no-new-privileges:truenetworks: [data, front]environment:DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-infradeck}:${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}@postgres:5432/${POSTGRES_DB:-infradeck}@@ -253,6 +277,16 @@ services:VITE_API_BASE_URL: /apicontainer_name: infradeck-frontendrestart: unless-stopped+ # Runs as the unprivileged nginx user (see the Dockerfile) with no capabilities and a read-only filesystem;+ # it writes only its cache, its pid file and the TRUSTED_PROXIES settings, all in memory.+ cap_drop: [ALL]+ read_only: true+ tmpfs:+ - /tmp:uid=101,gid=101+ - /var/cache/nginx:uid=101,gid=101+ - /etc/nginx/trusted-proxies:uid=101,gid=101+ security_opt:+ - no-new-privileges:trueenvironment:# Address or network of a reverse proxy in front of this one; empty trusts nobody's X-Forwarded-For.TRUSTED_PROXIES: ${TRUSTED_PROXIES:-}@@ -260,7 +294,7 @@ services:# Serves the dashboard and proxies /api to the backend. Reach it directly over# LAN/VPN, or put the `https` profile's Caddy in front for a public domain.ports:- - "${FRONTEND_BIND:-0.0.0.0}:${FRONTEND_PORT:-5173}:80"+ - "${FRONTEND_BIND:-0.0.0.0}:${FRONTEND_PORT:-5173}:8080"depends_on:backend:condition: service_healthydiff --git a/deploy/traefik.yml b/deploy/traefik.ymlindex ae7d5e1..8727860 100644--- a/deploy/traefik.yml+++ b/deploy/traefik.yml@@ -14,7 +14,7 @@ services:- traefik.http.routers.${TRAEFIK_ROUTER:-infradeck}.rule=Host(`${DASHBOARD_DOMAIN:?Set DASHBOARD_DOMAIN in .env}`)- traefik.http.routers.${TRAEFIK_ROUTER:-infradeck}.entrypoints=${TRAEFIK_ENTRYPOINT:-websecure}- traefik.http.routers.${TRAEFIK_ROUTER:-infradeck}.tls.certresolver=${TRAEFIK_CERT_RESOLVER:-letsencrypt}- - traefik.http.services.${TRAEFIK_ROUTER:-infradeck}.loadbalancer.server.port=80+ - traefik.http.services.${TRAEFIK_ROUTER:-infradeck}.loadbalancer.server.port=8080networks:traefik:diff --git a/docker-compose.server.yml b/docker-compose.server.ymlindex 8a2ca13..e27393c 100644--- a/docker-compose.server.yml+++ b/docker-compose.server.yml@@ -12,6 +12,16 @@ services:image: postgres:16-alpinecontainer_name: infradeck-postgresrestart: unless-stopped+ # The image's entrypoint prepares the data folder as root (owner and 0700 mode), then runs PostgreSQL as the+ # postgres user, which keeps none of these. Its socket and lock files live in memory.+ cap_drop: [ALL]+ cap_add: [CHOWN, DAC_READ_SEARCH, FOWNER, SETUID, SETGID]+ read_only: true+ tmpfs:+ - /var/run/postgresql:uid=70,gid=70+ - /tmp+ security_opt:+ - no-new-privileges:truenetworks: [data]environment:POSTGRES_DB: ${POSTGRES_DB:-infradeck}@@ -32,7 +42,14 @@ services:networks: [data]# Needs a password: Redis holds the login sessions, so nothing that can merely reach it on the network# may use it. Generate one with `openssl rand -hex 24` (letters and digits only, so it fits in a URL).- command: ["sh", "-c", "exec redis-server --appendonly yes --requirepass \"$$REDIS_PASSWORD\""]+ # Through the image's entrypoint, which hands /data to the redis user and drops root before Redis starts.+ command: ["sh", "-c", "exec docker-entrypoint.sh redis-server --appendonly yes --requirepass \"$$REDIS_PASSWORD\""]+ # Only what that hand-over needs: reading and chowning /data, and switching to the redis user.+ cap_drop: [ALL]+ cap_add: [CHOWN, DAC_READ_SEARCH, SETUID, SETGID]+ read_only: true+ security_opt:+ - no-new-privileges:trueenvironment:REDIS_PASSWORD: ${REDIS_PASSWORD:?Set REDIS_PASSWORD in .env (generate one with openssl rand -hex 24)}volumes:@@ -50,6 +67,13 @@ services:container_name: infradeck-backendimage: infradeck-backend:latestrestart: unless-stopped+ # Runs as an unprivileged user (see the Dockerfile) with no capabilities and a read-only filesystem.+ cap_drop: [ALL]+ read_only: true+ tmpfs:+ - /tmp+ security_opt:+ - no-new-privileges:truenetworks: [data, front]environment:DATABASE_URL: postgresql+psycopg://${POSTGRES_USER:-infradeck}:${POSTGRES_PASSWORD:?Set POSTGRES_PASSWORD in .env}@postgres:5432/${POSTGRES_DB:-infradeck}@@ -292,6 +316,16 @@ services:VITE_LIVE_GRAPH_REFRESH_INTERVAL_SECONDS: ${LIVE_GRAPH_REFRESH_INTERVAL_SECONDS:-3}container_name: infradeck-frontendrestart: unless-stopped+ # Runs as the unprivileged nginx user (see the Dockerfile) with no capabilities and a read-only filesystem;+ # it writes only its cache, its pid file and the TRUSTED_PROXIES settings, all in memory.+ cap_drop: [ALL]+ read_only: true+ tmpfs:+ - /tmp:uid=101,gid=101+ - /var/cache/nginx:uid=101,gid=101+ - /etc/nginx/trusted-proxies:uid=101,gid=101+ security_opt:+ - no-new-privileges:trueenvironment:# Address or network of your reverse proxy; empty trusts nobody's X-Forwarded-For.TRUSTED_PROXIES: ${TRUSTED_PROXIES:-}@@ -299,7 +333,7 @@ services:# over HTTPS on your domain. The plain-HTTP port is bound to localhost only# (set FRONTEND_BIND=0.0.0.0 to also allow direct LAN/VPN access).ports:- - "${FRONTEND_BIND:-127.0.0.1}:${FRONTEND_PORT:-5173}:80"+ - "${FRONTEND_BIND:-127.0.0.1}:${FRONTEND_PORT:-5173}:8080"volumes:# Public listener only: collector endpoints are refused here (see the collector-gate service).- ./docker/nginx.strict.conf:/etc/nginx/conf.d/default.conf:rodiff --git a/docker/Caddyfile b/docker/Caddyfileindex a00aff5..181e940 100644--- a/docker/Caddyfile+++ b/docker/Caddyfile@@ -5,5 +5,5 @@# Same value as docker/nginx.strict.conf. No includeSubDomains, since other apps on your domain may not be# HTTPS-only.header Strict-Transport-Security "max-age=15552000"- reverse_proxy frontend:80+ reverse_proxy frontend:8080}diff --git a/docker/backend.Dockerfile b/docker/backend.Dockerfileindex dcb180d..179e4cf 100644--- a/docker/backend.Dockerfile+++ b/docker/backend.Dockerfile@@ -7,6 +7,10 @@ RUN pip install --no-cache-dir -r requirements.txtCOPY src ./src+# Runs as an unprivileged user with a fixed UID; the code is owned by root, so it can't change it.+RUN useradd --system --uid 10001 --no-create-home --shell /usr/sbin/nologin infradeck+USER 10001+EXPOSE 8000# The API is only published on 127.0.0.1 and reached through the frontend's nginx,diff --git a/docker/frontend.Dockerfile b/docker/frontend.Dockerfileindex 08ea50e..42d7433 100644--- a/docker/frontend.Dockerfile+++ b/docker/frontend.Dockerfile@@ -15,8 +15,15 @@ RUN npm run buildFROM nginx:alpineCOPY docker/nginx.conf /etc/nginx/conf.d/default.conf-# Writes conf.d/trusted-proxies.conf from TRUSTED_PROXIES before nginx starts.+# Writes trusted-proxies/trusted-proxies.conf from TRUSTED_PROXIES before nginx starts.COPY --chmod=755 docker/trusted-proxies.sh /docker-entrypoint.d/40-trusted-proxies.shCOPY --from=build /app/dist /usr/share/nginx/html-EXPOSE 80\ No newline at end of file+# Runs as the unprivileged nginx user: no `user` switch, the pid file in /tmp, and only the folders nginx+# writes to owned by it.+RUN sed -i -e '/^user /d' -e 's#^pid .*#pid /tmp/nginx.pid;#' /etc/nginx/nginx.conf \+ && install -d -o nginx -g nginx /etc/nginx/trusted-proxies \+ && chown -R nginx:nginx /var/cache/nginx+USER nginx++EXPOSE 8080\ No newline at end of filediff --git a/docker/nginx.conf b/docker/nginx.confindex 7c30371..d4c4972 100644--- a/docker/nginx.conf+++ b/docker/nginx.conf@@ -1,5 +1,6 @@server {- listen 80;+ # 8080, not 80: nginx runs as an unprivileged user, which can't open ports below 1024 without a capability.+ listen 8080;server_name _;root /usr/share/nginx/html;@@ -15,8 +16,9 @@ server {add_header Cross-Origin-Opener-Policy "same-origin" always;add_header Content-Security-Policy "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; font-src 'self' data:; connect-src 'self'; object-src 'none'; base-uri 'self'; form-action 'self'; frame-ancestors 'none'" always;- # Whose X-Forwarded-For is believed comes from TRUSTED_PROXIES (docker/trusted-proxies.sh writes- # conf.d/trusted-proxies.conf at start). By default nobody's: $remote_addr is the address that connected.+ # Whose X-Forwarded-For is believed comes from TRUSTED_PROXIES (docker/trusted-proxies.sh writes the file+ # included here at start). By default nobody's: $remote_addr is the address that connected.+ include /etc/nginx/trusted-proxies/*.conf;# Dashboard and API share one origin: no CORS, and collectors and browsers# use the same address (http(s)://<host>/api).diff --git a/docker/nginx.strict.conf b/docker/nginx.strict.confindex 6bab987..001c480 100644--- a/docker/nginx.strict.conf+++ b/docker/nginx.strict.conf@@ -6,7 +6,8 @@# (docker/nginx.collector.conf), which is not attached to the reverse proxy's network.server {- listen 80;+ # 8080, not 80: nginx runs as an unprivileged user, which can't open ports below 1024 without a capability.+ listen 8080;server_name _;root /usr/share/nginx/html;@@ -26,8 +27,9 @@ server {add_header Strict-Transport-Security "max-age=15552000" always;# Whose X-Forwarded-For is believed comes from TRUSTED_PROXIES, the address or network of your reverse- # proxy (docker/trusted-proxies.sh writes conf.d/trusted-proxies.conf at start). By default nobody's:+ # proxy (docker/trusted-proxies.sh writes the file included here at start). By default nobody's:# $remote_addr is the address that connected.+ include /etc/nginx/trusted-proxies/*.conf;location ~ ^/api/(collector\.py$|monitoring/(collectors|metrics/)) {return 404;diff --git a/docker/trusted-proxies.sh b/docker/trusted-proxies.shindex 8231d9e..5e15684 100644--- a/docker/trusted-proxies.sh+++ b/docker/trusted-proxies.sh@@ -8,7 +8,7 @@# file to write (used by the tests).set -euf-out="${1:-/etc/nginx/conf.d/trusted-proxies.conf}"+out="${1:-/etc/nginx/trusted-proxies/trusted-proxies.conf}"ipv4='^([0-9]{1,3}\.){3}[0-9]{1,3}(/[0-9]{1,2})?$'ipv6='^[0-9A-Fa-f.]*:[0-9A-Fa-f:.]*(/[0-9]{1,3})?$'diff --git a/docs/containers.md b/docs/containers.mdindex a83c41e..a720610 100644--- a/docs/containers.md+++ b/docs/containers.md@@ -84,7 +84,8 @@ those files are copied into a plain `nginx:alpine` image. Node and the source corunning container. nginx sets the security headers (Content-Security-Policy, Permissions-Policy andothers) and proxies `/api/` to `backend:8000`. On the server stack it uses[`docker/nginx.strict.conf`](../docker/nginx.strict.conf), which additionally **refuses** the collector-endpoints: on the public side nobody can register a collector or send metrics.+endpoints: on the public side nobody can register a collector or send metrics. nginx runs as the+unprivileged `nginx` user on port 8080, with no capabilities and a read-only filesystem.**Why this approach.**- *One address for the app and the API.* The browser loads the page and calls the API on the same origin,@@ -112,6 +113,7 @@ through the gate); on the development stack the local collector talks to the baccolumns in PostgreSQL, moves old Redis keys to the current prefix, and prints the first-run setup tokenif no account exists yet. Two background threads run inside it: one marks servers offline when theircollector stops checking in, the other sends notifications for opened and resolved incidents.+It runs as an unprivileged user (UID 10001) with no capabilities and a read-only filesystem.`GET /health` reports whether PostgreSQL and Redis answer; Docker's healthcheck uses it, and the othercontainers wait for it before they start.@@ -143,7 +145,8 @@ The full list of tables is in [Database schema](database.md).The backend creates every table and upgrades the schema itself at startup (Alembic migrations in[`src/backend/migrations/`](../src/backend/migrations)). The healthcheck (`pg_isready`) holds thebackend back until the database accepts connections. The container is on the internal `data` network-only and publishes no port.+only and publishes no port. PostgreSQL itself runs as the `postgres` user with no capabilities; the+image's start script gets a few as root to prepare the data folder (see [SECURITY.md](../SECURITY.md#container-privileges)).**Why this approach.**- *Many writers at once.* Every collector checks in every minute while the background threads and@@ -179,6 +182,8 @@ Redis doesn't hand out working sessions or a list of email addresses.**How it works.** The official `redis:7-alpine` image with a password (`REDIS_PASSWORD`) and append-onlypersistence (`--appendonly yes`) into the `redis_data` volume, so people stay signed in across a restart.It is on the internal `data` network only and publishes no port; the backend connects with `REDIS_URL`.+Redis runs as the `redis` user with no capabilities and a read-only filesystem; the image's start script+hands `/data` to that user first.**Why Redis is a required part, and why not just PostgreSQL.**- *Sessions are checked on every request.* Each API call looks up its session. Redis answers that from@@ -346,7 +351,7 @@ and a read-only filesystem, and it is opt-in. Read [SECURITY.md](../SECURITY.md)**What it does.** Puts HTTPS in front of the frontend on a public domain when you don't run Traefik.**How it works.** The official `caddy:2-alpine` image with [`docker/Caddyfile`](../docker/Caddyfile):-it obtains and renews a Let's Encrypt certificate for `DOMAIN` and proxies everything to `frontend:80`.+it obtains and renews a Let's Encrypt certificate for `DOMAIN` and proxies everything to `frontend:8080`.Start it with `docker compose --profile https up -d`.**Why Caddy here.** It handles certificates with no configuration beyond the domain name. The serverdiff --git a/docs/deploy-traefik.md b/docs/deploy-traefik.mdindex 871639f..8f5657e 100644--- a/docs/deploy-traefik.md+++ b/docs/deploy-traefik.md@@ -223,6 +223,7 @@ docker compose up -d| Signed out right after signing in | The dashboard was opened over `http://`, or `SESSION_COOKIE_SECURE` was set to `false`. | Use `https://<your-domain>` and leave `SESSION_COOKIE_SECURE` empty. || Everyone is locked out after a few failed sign-ins | `TRUSTED_PROXIES` is empty or wrong, so all visitors share Traefik's address. | Set it as in step 2, then `docker compose up -d`. || The frontend restarts over and over | `TRUSTED_PROXIES` contains something other than IP addresses and CIDR ranges. | `docker compose logs frontend` names the bad value. Fix it in `.env`. |+| The frontend restarts over and over with `mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)` | The stack was updated without `--build`, so an old frontend image runs under the new settings. | `docker compose up -d --build`. |More symptoms: [Troubleshooting](troubleshooting.md).diff --git a/docs/deployment.md b/docs/deployment.mdindex ede8a10..f0f1149 100644--- a/docs/deployment.md+++ b/docs/deployment.md@@ -144,6 +144,13 @@ frontend, collector, Docker proxy and system helper each have their code builtinto their image, so a service that isn't rebuilt keeps running the old version.The database schema is updated automatically when the backend starts.+**Upgrading from a version where the frontend listened on port 80 inside its container:** it now listens on+8080 and runs without root. The compose files, `deploy/traefik.yml` and the Caddy profile already use the new+port; only a reverse proxy you configured yourself to reach `frontend:80` over a Docker network must change to+`frontend:8080`. A proxy that uses the published port (`127.0.0.1:5173`) needs no change. Update with+`docker compose up -d --build`: without `--build` the old frontend image runs under the new settings and keeps+restarting with `mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)`.+**Upgrading from a version with the Traefik labels in `docker-compose.server.yml`:** they moved to`deploy/traefik.yml`. Before you update, change `.env` to`COMPOSE_FILE=docker-compose.server.yml:deploy/traefik.yml`. Without it the dashboard starts@@ -170,7 +177,7 @@ compose project). Collectors started from `collector-compose.yml` need- **Another reverse proxy (nginx, Caddy, HAProxy, a tunnel connector, ...):** use the **server stack**(`docker-compose.server.yml`) on its own, not the development stack, and point your proxy at its dashboardport (`127.0.0.1:5173` by default). A proxy that runs in Docker can instead join the stack's `front` network- and reach `frontend:80`. The server stack is the one that keeps the collector endpoints off the+ and reach `frontend:8080`. The server stack is the one that keeps the collector endpoints off thepublic side, registration closed and the session cookie HTTPS-only. Terminate HTTPS in your proxy, have itsend `X-Forwarded-For`, and set `TRUSTED_PROXIES` to its address (see below).- **Caddy profile (`--profile https`, development stack): private use only.** It publishes the developmentdiff --git a/docs/troubleshooting.md b/docs/troubleshooting.mdindex 70038da..6f1ab8b 100644--- a/docs/troubleshooting.md+++ b/docs/troubleshooting.md@@ -7,6 +7,7 @@| A new machine never shows up under **New servers detected** | `docker logs infradeck-collector` on that machine. "Could not reach …" means the address in the command isn't reachable from it (firewall, `localhost`, wrong IP/domain). || Collector log says "Backend no longer accepts server …" | The server was deleted on the dashboard; it is asking to join again, so click **Add** or **Dismiss**. || `network traefik_default declared as external, but could not be found` | You're using `deploy/traefik.yml` on a machine without that Traefik network. Set `TRAEFIK_NETWORK` in `.env` to the network Traefik is really on, or, without Traefik, drop `deploy/traefik.yml` from `COMPOSE_FILE`. |+| Frontend keeps restarting; `docker compose logs frontend` shows `mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)` | Its image is older than the compose file: the stack was started without `--build` after an update. Run `docker compose up -d --build`. || Dashboard shows old behaviour after a code change or `git pull` | Rebuild **all** services with `docker compose up -d --build` (on a server with `COMPOSE_FILE` set in `.env`) and hard-refresh the browser (Ctrl+Shift+R). || "The machine running the dashboard isn't reporting yet" at the top | The stack's own collector hasn't checked in: add it under **New servers detected**, then wait one check interval. If it is reporting, its image is outdated: `docker exec infradeck-collector grep -c dashboard_host /app/collector.py` prints `0`. Rebuild it (`docker compose build --no-cache collector && docker compose up -d collector`). || A server shows only `/` and no **Storage** bars for other disks | Its collector predates per-mount reporting. Install a newer collector image (run a freshly generated command, or `docker compose -f collector-compose.yml up -d --build`). A disk mounted after the collector started appears after a collector restart. |