Skip to content

Open-source gym & body-weight tracker, now in the App Store (iPhone + Apple Watch) and as an Android APK. Plan routines, log workouts, see which muscles are trained, import from Strong/Hevy/FitNotes. Self-host it or use it on the phone alone.

api: a health check that never touches the disk is not checking health

Boris 'B' Kurktchiev1fc64a602914
diff --git a/api/openapi.yaml b/api/openapi.yaml
index abb279e45..d8b586471 100644
--- a/api/openapi.yaml
+++ b/api/openapi.yaml
@@ -164,11 +164,21 @@ paths:
tags: [meta]
operationId: getHealth
summary: Health check
- description: Always public. Also reports how many user accounts exist.
+ description: |
+ Always public. Also reports how many user accounts exist.
+
+ The check writes: it creates and removes one empty file under `DATA_DIR`,
+ because a server whose data directory is full or read-only has every write
+ path broken while the port still answers. It is a 503 when that fails — the
+ container healthcheck (`wget --spider`) reads the status and nothing else.
+
+ The probe runs at most once every five seconds and the answer in between is
+ the last one, so a poll can be that stale. The route is unauthenticated, and
+ one disk write per request is not something to leave open to anyone.
security: []
responses:
'200':
- description: The server is up.
+ description: The server is up and its data directory is writable.
content:
application/json:
schema:
@@ -178,7 +188,21 @@ paths:
users:
type: integer
description: Number of registered user accounts.
- example: { ok: true, users: 7 }
+ writable: { type: boolean, const: true }
+ example: { ok: true, users: 7, writable: true }
+ '503':
+ description: >-
+ The data directory could not be written — a full disk, or a mount that
+ came back read-only. Nothing that stores anything will work until it is
+ fixed; the reason is in the container log.
+ content:
+ application/json:
+ schema:
+ type: object
+ properties:
+ ok: { type: boolean, const: false }
+ writable: { type: boolean, const: false }
+ example: { ok: false, writable: false }
/api/config:
get:
diff --git a/api/server.js b/api/server.js
index debdd4840..df43c6a73 100644
--- a/api/server.js
+++ b/api/server.js
@@ -1729,8 +1729,45 @@ const mediaRoutes = {
};
/* ---------- routes ---------- */
+// A health check that only proves the port answers is the one that lies on the day it matters:
+// a full disk, or a bind mount that came back read-only, leaves every write path in here broken
+// — no sync, no sign-in, no audit line — while the process sits there perfectly alive and the
+// container stays "healthy". So the check writes: one empty file under DATA, removed again.
+// Cheap next to what it proves, and it is the same directory every real write goes to.
+// One probe per window, not one per request: this route is unauthenticated, so a create and an
+// unlink on every call is a disk write anyone who can reach the port may ask for as fast as they
+// like. The container polls every five minutes (api/Dockerfile), so a five-second window is free
+// to the check that matters and caps a flood at one probe per five seconds. The cost is that a
+// poll can be up to five seconds stale — a mount that went read-only a second ago still reads
+// healthy, and the poll after that does not.
+const DATA_PROBE_MS = 5000;
+let dataProbe = { at: 0, ok: false };
+function dataWritable() {
+ const now = Date.now();
+ if (dataProbe.at && now - dataProbe.at < DATA_PROBE_MS) return dataProbe.ok;
+ const probe = path.join(DATA, '.health-' + process.pid);
+ let ok;
+ try {
+ fs.writeFileSync(probe, '');
+ ok = true;
+ } catch (e) {
+ console.error('health: cannot write to', DATA + ':', e.message);
+ ok = false;
+ } finally {
+ try { fs.unlinkSync(probe); } catch { /* never got created */ }
+ }
+ dataProbe = { at: now, ok };
+ return ok;
+}
+
const routes = {
- 'GET /api/health': async (req, res) => json(res, 200, { ok: true, users: db.users.length }),
+ // 503 rather than a flag in a 200: the container healthcheck is `wget --spider`, which reads
+ // the status and nothing else, and an instance that cannot write its data directory is exactly
+ // what "unhealthy" is for. The 200 keeps the shape it had, plus the flag it is now asserting.
+ 'GET /api/health': async (req, res) => {
+ if (!dataWritable()) return json(res, 503, { ok: false, writable: false });
+ json(res, 200, { ok: true, users: db.users.length, writable: true });
+ },
// Public config the login screen needs before anyone is signed in. `coach` is absent unless
// the instance has both switched the Coach on and successfully connected a provider — the
diff --git a/api/test/server-health-writable.test.js b/api/test/server-health-writable.test.js
new file mode 100644
index 000000000..6fc81a351
--- /dev/null
+++ b/api/test/server-health-writable.test.js
@@ -0,0 +1,85 @@
+/* GET /api/health used to answer `{"ok":true}` as long as the process was alive — including on a
+ full disk or a bind mount that had come back read-only, where every write path in the server is
+ broken and the container's healthcheck (`wget --spider`, api/Dockerfile) still reads healthy.
+ It writes a probe file under DATA_DIR now, and answers 503 when it cannot. */
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+import crypto from 'node:crypto';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { spawn } from 'node:child_process';
+import { fileURLToPath } from 'node:url';
+import webpush from 'web-push';
+
+const API = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');
+const SECRET = crypto.randomBytes(32).toString('hex');
+
+function seedDir() {
+ const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gym-health-'));
+ fs.writeFileSync(path.join(dir, 'secret'), SECRET, { mode: 0o600 });
+ fs.writeFileSync(path.join(dir, 'db.json'), JSON.stringify({
+ users: [{ id: 'u_h_1', name: 'H', created: new Date().toISOString() }], creds: [], subs: [], invites: []
+ }));
+ // Boot writes this one if it is missing, which a read-only directory would refuse before the
+ // server ever listened — the point here is a server that is up and cannot write, not one that
+ // cannot start. The keys have to be real: web-push validates them at setVapidDetails.
+ fs.writeFileSync(path.join(dir, 'vapid.json'), JSON.stringify(webpush.generateVAPIDKeys()));
+ return dir;
+}
+
+async function startServer(t, dataDir) {
+ const child = spawn(process.execPath, ['server.js'], {
+ cwd: API, stdio: ['ignore', 'pipe', 'pipe'],
+ env: { ...process.env, PORT: '0', DATA_DIR: dataDir, ORIGIN: 'http://localhost:8080', RP_ID: 'localhost' }
+ });
+ t.after(() => { child.kill('SIGKILL'); fs.chmodSync(dataDir, 0o700); fs.rmSync(dataDir, { recursive: true, force: true }); });
+ let log = '';
+ // The boot line, not a health poll: the poll is what is under test here.
+ const port = await new Promise((resolve, reject) => {
+ const give = setTimeout(() => reject(new Error(`server never announced a port:\n${log}`)), 20000);
+ child.stdout.on('data', d => { log += d; const m = /gym-api on :(\d+)/.exec(log); if (m) { clearTimeout(give); resolve(+m[1]); } });
+ child.stderr.on('data', d => { log += d; });
+ child.on('exit', c => { clearTimeout(give); reject(new Error(`server exited (${c}):\n${log}`)); });
+ });
+ return { api: `http://127.0.0.1:${port}`, log: () => log };
+}
+
+test('a writable data directory answers 200, and says so', async t => {
+ const h = await startServer(t, seedDir());
+ const r = await fetch(`${h.api}/api/health`);
+ assert.equal(r.status, 200);
+ assert.deepEqual(await r.json(), { ok: true, users: 1, writable: true });
+});
+
+test('a data directory the server cannot write to is 503, not healthy', async t => {
+ if (typeof process.getuid === 'function' && process.getuid() === 0) {
+ t.skip('running as root: a mode that denies writes does not deny them to root');
+ return;
+ }
+ const dir = seedDir();
+ const h = await startServer(t, dir);
+
+ // Denied before the first poll, so nothing has been probed — or cached — yet.
+ fs.chmodSync(dir, 0o500); // r-x: the mount came back read-only
+ const r = await fetch(`${h.api}/api/health`);
+ assert.equal(r.status, 503, 'wget --spider reads this as unhealthy, which is the point');
+ assert.deepEqual(await r.json(), { ok: false, writable: false });
+ // The child's stderr is a pipe of its own, with no ordering guarantee against the socket the
+ // answer came back on — the line can land after the fetch resolves, and did in 1 run of 130.
+ for (let i = 0; i < 100 && !/health: cannot write to/.test(h.log()); i++) await new Promise(r => setTimeout(r, 20));
+ assert.match(h.log(), /health: cannot write to/, 'and the operator is told which directory');
+
+ // The probe is cached for five seconds, so the very next poll answers from it even though the
+ // directory is writable again — two polls, one probe, which is what keeps an unauthenticated
+ // route from being a disk write on demand.
+ fs.chmodSync(dir, 0o700);
+ assert.equal((await fetch(`${h.api}/api/health`)).status, 503, 'the second poll did not probe again');
+
+ // A window, not a latch: past it, it recovers on its own with no restart.
+ await new Promise(r => setTimeout(r, 5200));
+ assert.equal((await fetch(`${h.api}/api/health`)).status, 200);
+
+ // Nothing is left behind by the probe itself.
+ assert.deepEqual(fs.readdirSync(dir).filter(f => f.startsWith('.health-')), []);
+});
diff --git a/website/api.html b/website/api.html
index 9a2713946..14a6f1e72 100644
--- a/website/api.html
+++ b/website/api.html
@@ -303,6 +303,8 @@ main.api-main .lead { color: var(--fg-2); max-width: 58ch; margin: 14px 0 0; fon
</summary>
<div class="ep-body">
<p>Always public. Also reports how many user accounts exist.</p>
+<p>The check writes: it creates and removes one empty file under <code>DATA_DIR</code>, because a server whose data directory is full or read-only has every write path broken while the port still answers. It is a 503 when that fails — the container healthcheck (<code>wget --spider</code>) reads the status and nothing else.</p>
+<p>The probe runs at most once every five seconds and the answer in between is the last one, so a poll can be that stale. The route is unauthenticated, and one disk write per request is not something to leave open to anyone.</p>
<p class="ep-auth"><svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="4" y="11" width="16" height="10" rx="2.5"/><path d="M8 11V7a4 4 0 0 1 8 0v4"/></svg><span>Public — no authentication required.</span></p>
@@ -310,7 +312,11 @@ main.api-main .lead { color: var(--fg-2); max-width: 58ch; margin: 14px 0 0; fon
<div class="ptab">
<div class="prow rrow">
<span class="r-code s2">200</span>
- <span class="p-desc">The server is up.<span class="r-sig">{ ok: always true, users: integer }</span></span>
+ <span class="p-desc">The server is up and its data directory is writable.<span class="r-sig">{ ok: always true, users: integer, writable: always true }</span></span>
+ </div>
+ <div class="prow rrow">
+ <span class="r-code s5">503</span>
+ <span class="p-desc">The data directory could not be written — a full disk, or a mount that came back read-only. Nothing that stores anything will work until it is fixed; the reason is in the container log.</span>
</div>
</div>
</div>