Open-source gym & body-weight tracker, now in the App Store (iPhone + Apple Watch) and as an Android APK. Plan routines, log workouts, see which muscles are trained, import from Strong/Hevy/FitNotes. Self-host it or use it on the phone alone.
JavaScript apple-watchbodyweightdockerfitnessfitness-trackergymhealthiosmcpnodejspasskeysprogressive-web-applicationspwareactself-hostedself-hostingvitewebauthnweightliftingworkout-tracker
API: PUT and GET /api/media/{hash}, /api/media/missing and /api/media/sweep, the caps in /api/config, and half an hour for a slow upload
Duarte Santos154b9d88778b
diff --git a/api/Dockerfile b/api/Dockerfileindex 69b8c5198..3db39e238 100644--- a/api/Dockerfile+++ b/api/Dockerfile@@ -53,7 +53,7 @@ RUN npm ci --omit=dev --omit=optional && npm cache clean --force# Every module server.js imports from the api root has to be named here: the image carries nothing# else, and one that is missing only shows at boot, as ERR_MODULE_NOT_FOUND.-COPY server.js push-messages.js verify-error.js password.js rate-limit.js passkeys-store.js device-link.js ./+COPY server.js push-messages.js verify-error.js password.js rate-limit.js passkeys-store.js device-link.js media.js ./COPY coach ./coachENV NODE_ENV=productionENV PORT=3000diff --git a/api/openapi.yaml b/api/openapi.yamlindex bd1a26567..3cffddb10 100644--- a/api/openapi.yaml+++ b/api/openapi.yaml@@ -78,10 +78,19 @@ info:`GET /api/config`. Off by default.- `TRUST_PROXY=1` — the throttle reads the client address from proxy headers (set bythe bundled `docker-compose.yml`, where only the web container can reach the API).+ - `MEDIA_UPLOADS` (default on; `0` removes the `media` routes and the `media` block of+ `GET /api/config`), `MEDIA_QUOTA_MB` (200 per profile, `0` = no cap),+ `MEDIA_IMAGE_MAX_MB` (2), `MEDIA_GIF_MAX_MB` (8), `MEDIA_VIDEO_MAX_MB` (40),+ `MEDIA_VIDEO_MAX_SEC` (60), `MEDIA_GC_GRACE_DAYS` (14), `MEDIA_UPLOADS_PER_HOUR` (600+ per profile), `MEDIA_MIN_FREE_MB` (512, `0` = no floor) — photos and videos of custom+ exercises (tag `media`). Every MB here is 2^20 bytes.### Conventions- - Every response body is JSON with `Cache-Control: no-store`.- - Errors are always `{"error": "<human-readable message>"}`.+ - Every response body is JSON with `Cache-Control: no-store`. The one exception is+ `GET /api/media/{hash}`, which answers with the stored file itself+ (`Cache-Control: private, no-store`).+ - Errors are always `{"error": "<human-readable message>"}`. The `password`, throttle and+ `media` routes add a stable `code` for the client to word in its own language.- Unknown method+path pairs return `404 {"error":"not found"}`; an unhandledexception returns `500 {"error":"server error"}`.- Request bodies are parsed as JSON regardless of Content-Type. A body that does@@ -128,6 +137,11 @@ tags:description: Live-workout presence heartbeat- name: admindescription: Admin dashboard — requires an admin session (`ADMIN_UIDS` or `admin:true`)+ - name: media+ description: >-+ The photo, GIF or video of a user's own custom exercise. Stored per profile and named+ by the SHA-256 of its bytes; the state only carries a `MediaRef`. Every route here is a+ 404 when the instance runs with `MEDIA_UPLOADS=0`.- name: coachdescription: >-AI Coach — plans, reviews and debriefs. Every route here answers 503 unless the@@ -169,6 +183,10 @@ paths:environment variables on the server (`INVITE_ONLY`, `ALLOW_GUEST`), and so does`password_login`, which is there only when `PASSWORD_LOGIN=1`.+ `media` is there unless the instance runs with `MEDIA_UPLOADS=0`, for everybody: the+ caps are not a secret, and its absence is how the app knows this server does not+ store photos and videos at all.+A caller with a valid session also gets a `coach` key: the block when theinstance has the AI Coach switched on **and** a provider connected, and `null`when it has not. A caller with no session gets no key at all — the block names@@ -221,6 +239,8 @@ paths:Present (and true) only when the instance runs withPASSWORD_LOGIN=1 — the client then offers name-and-passwordsign-in next to passkeys.+ media:+ $ref: '#/components/schemas/MediaConfig'examples:anonymous: { value: { invite_only: true, allow_guest: false } }signedInNoCoach: { value: { invite_only: true, allow_guest: false, coach: null } }@@ -230,6 +250,7 @@ paths:allow_guest: falsecoach: { enabled: true, provider: compatible, providerLabel: "OpenAI-compatible endpoint", authMode: instance, community: false }passwordLogin: { value: { invite_only: true, allow_guest: false, password_login: true } }+ media: { value: { invite_only: false, allow_guest: true, media: { imageMB: 2, gifMB: 8, videoMB: 40, videoSec: 60, quotaMB: 200 } } }/api/me:get:@@ -1232,6 +1253,10 @@ paths:from before revisions, and deliberate replaces such as a backup import). Theserver stamps the new revision into the document as `_rev`; a `_rev` sent bythe client is ignored.++ Every accepted write also starts the grace period of each stored file (tag `media`)+ the new state no longer references, and stops it for each one it references again.+ That bookkeeping never changes the answer.requestBody:required: truecontent:@@ -1310,6 +1335,248 @@ paths:rev: { type: integer }'401': { $ref: '#/components/responses/Unauthorized' }+ /api/media/{hash}:+ parameters:+ - name: hash+ in: path+ required: true+ description: The SHA-256 of the file's bytes, 64 lowercase hex characters.+ schema: { type: string, pattern: '^[0-9a-f]{64}$' }+ get:+ tags: [media]+ operationId: getMedia+ summary: Download one of your photos or videos+ description: |+ The stored file, whole, from the caller's own folder only: another profile's file+ answers exactly like one that was never uploaded (404). The body is the file, not+ JSON, with headers that keep a browser from ever treating it as a page:+ `Content-Type` from the sniffed type (never `text/*` or SVG),+ `X-Content-Type-Options: nosniff`, `Content-Security-Policy: default-src 'none'; sandbox`,+ `Cross-Origin-Resource-Policy: same-origin`, `Content-Disposition: inline`,+ `X-Robots-Tag: noindex` and `Cache-Control: private, no-store` — the app keeps its own+ verified copy, and an HTTP-cache copy would outlive signing out.++ No `Range` and no `ETag`: the app downloads a file once, checks it against its hash and+ keeps it. A paired phone fetches with its bearer token; `<img src>` cannot carry one.+ Not throttled.+ responses:+ '200':+ description: The file.+ headers:+ Content-Length: { schema: { type: integer } }+ Content-Disposition: { schema: { type: string }, example: 'inline; filename="<hash>.webp"' }+ content:+ image/jpeg: { schema: { type: string, format: binary } }+ image/png: { schema: { type: string, format: binary } }+ image/webp: { schema: { type: string, format: binary } }+ image/gif: { schema: { type: string, format: binary } }+ video/mp4: { schema: { type: string, format: binary } }+ video/quicktime: { schema: { type: string, format: binary } }+ video/webm: { schema: { type: string, format: binary } }+ '401': { $ref: '#/components/responses/Unauthorized' }+ '404':+ description: Not in the caller's folder — never uploaded, removed by the sweep, or someone else's.+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "no such file", code: "media-missing" }+ put:+ tags: [media]+ operationId: putMedia+ summary: Upload one photo, GIF or video+ description: |+ The body is the raw file; `Content-Type` must be one of the seven types below and+ `Content-Length` is optional (chunked uploads are capped while they stream). The path+ names the SHA-256 of the bytes, and the file is stored only if they hash to it.+ Idempotent: uploading a file the server already has answers 200 with `existed: true`.++ What is stored is decided by the file's first bytes, not by the declared type. The+ declared type only has to be in the same category — a still (JPEG, PNG, WebP, GIF) or a+ video (MP4, MOV, WebM): a JPEG declared as a GIF is stored as the JPEG it is, a video+ declared as an image is refused. The cap of the kind the bytes turned out to be applies+ (`MEDIA_IMAGE_MAX_MB`, `MEDIA_GIF_MAX_MB`, `MEDIA_VIDEO_MAX_MB`), and an MP4 or MOV must+ be a well-formed file no longer than `MEDIA_VIDEO_MAX_SEC` (+1 s), read from its+ headers. HEIC/AVIF, SVG, HTML and every other type are refused; the app converts HEIC+ photos before uploading. The server never decodes or alters a file: the app has already+ re-encoded photos and removed video metadata on the device.++ Checked in this order: session (401), the hourly budget and at most two uploads at once+ per profile (429), the declared type (415), a declared length over the cap (413 before+ a byte is read), already stored (200), the quota (413 `media-quota`; files the profile+ already dropped get one hour instead of the grace to make room), free disk (507), then+ while and after streaming: size (413), hash (400), sniffed type (415), sniffed cap+ (413), video checks (415/413). No bytes for 60 s ends the upload with 408. A refused+ upload is read and discarded up to twice its cap, so the answer reaches the client.++ A new file that the profile's state does not reference yet starts its grace period+ at once, so a state push that never comes does not keep it forever.+ requestBody:+ required: true+ content:+ image/jpeg: { schema: { type: string, format: binary } }+ image/png: { schema: { type: string, format: binary } }+ image/webp: { schema: { type: string, format: binary } }+ image/gif: { schema: { type: string, format: binary } }+ video/mp4: { schema: { type: string, format: binary } }+ video/quicktime: { schema: { type: string, format: binary } }+ video/webm: { schema: { type: string, format: binary } }+ responses:+ '201':+ description: Stored.+ content:+ application/json:+ schema: { $ref: '#/components/schemas/MediaStored' }+ example: { ok: true, hash: "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", mime: "image/webp", size: 183204, existed: false }+ '200':+ description: The server already had these bytes (from this or another of your devices).+ content:+ application/json:+ schema: { $ref: '#/components/schemas/MediaStored' }+ example: { ok: true, hash: "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08", mime: "image/webp", size: 183204, existed: true }+ '400':+ description: The bytes do not hash to the name in the path (`hash-mismatch`).+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "the file does not match its name", code: "hash-mismatch" }+ '401': { $ref: '#/components/responses/Unauthorized' }+ '403':+ description: A browser request from another origin (see *CSRF*).+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "cross-origin request refused" }+ '408':+ description: No bytes arrived for 60 seconds (`timeout`). The connection is closed.+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "the upload stalled", code: "timeout" }+ '413':+ description: |+ - `media-too-large` — over the cap of its kind; `maxMB` says the cap.+ - `media-quota` — the profile's space is full; `usedMB` and `quotaMB` say how full.+ - `media-too-long` — a video longer than `maxSec` seconds.+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ examples:+ tooLarge: { value: { error: "that file is too large", code: "media-too-large", maxMB: 2 } }+ quota: { value: { error: "your space for photos and videos is full", code: "media-quota", usedMB: 199.4, quotaMB: 200 } }+ tooLong: { value: { error: "that video is too long", code: "media-too-long", maxSec: 60 } }+ '415':+ description: >-+ `media-type` — the declared type is not one of the seven, the bytes are none of+ them, or they are a video declared as an image (or the reverse). `media-invalid` —+ an MP4 or MOV whose boxes do not add up.+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ examples:+ type: { value: { error: "that file type is not accepted", code: "media-type" } }+ invalid: { value: { error: "that video could not be read", code: "media-invalid" } }+ '429':+ description: >-+ `locked` — more than `MEDIA_UPLOADS_PER_HOUR` uploads this hour; `busy` — two+ uploads of this profile are already in flight (`retryAfter` 5).+ headers:+ Retry-After: { schema: { type: integer } }+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ examples:+ locked: { value: { error: "too many uploads — try again later", code: "locked", retryAfter: 1740 } }+ busy: { value: { error: "too many uploads at once — try again in a moment", code: "busy", retryAfter: 5 } }+ '507':+ description: The server's disk has less than `MEDIA_MIN_FREE_MB` free (`storage-full`).+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "the server is running out of disk space", code: "storage-full" }++ /api/media/missing:+ post:+ tags: [media]+ operationId: mediaMissing+ summary: Which of these files the server does not have+ description: |+ The app sends every hash its state references and uploads only what comes back, so the+ same call moves a guest's files into a new account, a phone's into its pairing, a+ backup's into the server, and re-sends a file the sweep removed while a device still+ had it. Answered from the caller's own folder only.+ requestBody:+ required: true+ content:+ application/json:+ schema:+ type: object+ required: [hashes]+ properties:+ hashes:+ type: array+ maxItems: 1000+ items: { type: string, pattern: '^[0-9a-f]{64}$' }+ responses:+ '200':+ description: The subset the server does not have, in the order sent, and the profile's usage.+ content:+ application/json:+ schema:+ type: object+ properties:+ missing:+ type: array+ items: { type: string }+ usage: { $ref: '#/components/schemas/MediaUsage' }+ example: { missing: ["b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"], usage: { bytes: 52428800, count: 14, quotaBytes: 209715200 } }+ '400':+ description: '`hashes` is missing, not a list, longer than 1000, or holds something that is not a lowercase SHA-256 (`bad-request`).'+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ '401': { $ref: '#/components/responses/Unauthorized' }++ /api/media/sweep:+ post:+ tags: [media]+ operationId: mediaSweep+ summary: Delete every file your current state does not reference+ description: |+ What "Reset everything" does after pushing the empty state: every stored file of the+ caller that the stored state does not reference goes now, without the grace period.+ A state that cannot be read removes nothing. At most 10 an hour per profile (429+ `locked`). Recorded in the activity log as `media.sweep`.++ Without this call nothing is lost either: the hourly sweep removes a file once the+ state has not referenced it for `MEDIA_GC_GRACE_DAYS` days (14), and only for profiles+ whose state reads — a missing or unreadable state, or a profile missing from `db.json`,+ never deletes anything.+ requestBody:+ content:+ application/json:+ schema: { type: object }+ example: {}+ responses:+ '200':+ description: What was removed, and what is left.+ content:+ application/json:+ schema:+ type: object+ properties:+ removed: { type: integer }+ freedBytes: { type: integer }+ usage: { $ref: '#/components/schemas/MediaUsage' }+ '401': { $ref: '#/components/responses/Unauthorized' }+ '429':+ description: More than 10 sweeps this hour (`locked`).+ headers:+ Retry-After: { schema: { type: integer } }+ content:+ application/json:+ schema: { $ref: '#/components/schemas/Error' }+ example: { error: "too many uploads — try again later", code: "locked", retryAfter: 1200 }+/api/push/public-key:get:tags: [push]@@ -1675,8 +1942,9 @@ paths:summary: 'Admin: delete a user and everything of theirs'description: |Removes the account for good: the user record, their passkeys, their push- subscriptions, their live presence, their training history file and any Coach- credential stored for them. Not reversible — `/api/admin/user/disable` is the+ subscriptions, their live presence, their training history file, their uploaded+ photos and videos (`DATA_DIR/uploads/<uid>/`) and any Coach credential stored for+ them. Not reversible — `/api/admin/user/disable` is thereversible one. An admin cannot delete their own account, and the last admincannot be deleted. Audited by name, since the id stops meaning anything.requestBody:@@ -2530,8 +2798,10 @@ components:code:type: stringdescription: >-- A stable reason on the password and throttle routes, for a client to word in- its own language (`bad-credentials`, `locked`, `too-short`, …).+ A stable reason on the password, throttle and media routes, for a client to word+ in its own language (`bad-credentials`, `locked`, `too-short`, `media-quota`, …).+ Media refusals add the numbers to word it with (`maxMB`, `maxSec`, `usedMB`,+ `quotaMB`).retryAfter:type: integerdescription: 429 only — the same number of seconds as the Retry-After header.@@ -2719,7 +2989,7 @@ components:customEx:type: arraydescription: User-created exercises (id, name, muscles…).- items: { type: object, additionalProperties: true }+ items: { $ref: '#/components/schemas/CustomExercise' }exWeights:type: objectdescription: 'Per-exercise last/best weight memory: exercise id → {w, d}.'@@ -2748,6 +3018,84 @@ components:additionalProperties: trueadditionalProperties: true+ CustomExercise:+ type: object+ description: >-+ One exercise the user made (representative shape). The server stores it as sent and+ reads only `media` from it, to know which uploaded files are still referenced.+ properties:+ id: { type: string }+ n: { type: string, description: Name. }+ bp: { type: string, description: Body part. }+ custom: { type: boolean, const: true }+ media:+ $ref: '#/components/schemas/MediaRef'+ url:+ type: string+ format: uri+ maxLength: 2048+ description: >-+ One link — a video or a guide. http(s) only, no credentials. Neither the app nor+ the server ever fetches it: the app opens it in a new browsing context on a tap.+ _ts: { type: number, description: 'When this entry was last edited, ms — for merging two devices'' copies.' }+ additionalProperties: true++ MediaRef:+ type: object+ description: >-+ What the state knows of an uploaded file. The bytes are behind+ `GET /api/media/{hash}`; the server keeps a file as long as some `customEx[].media.hash`+ or `.poster.hash` of the profile's state names it, and for `MEDIA_GC_GRACE_DAYS` after.+ required: [kind, hash, mime, size, width, height, at]+ properties:+ kind: { type: string, enum: [image, gif, video], description: '`gif` means animated; a one-frame GIF is an `image`.' }+ hash: { type: string, pattern: '^[0-9a-f]{64}$', description: SHA-256 of the stored file. }+ mime: { type: string, enum: [image/webp, image/jpeg, image/png, image/gif, video/mp4, video/quicktime, video/webm] }+ size: { type: integer, minimum: 1 }+ width: { type: integer, minimum: 1, maximum: 16384 }+ height: { type: integer, minimum: 1, maximum: 16384 }+ dur: { type: number, minimum: 0, maximum: 3600, description: 'Seconds, one decimal; videos and animated GIFs, absent when unknown.' }+ codec: { type: string, enum: [avc1, hvc1, av01, vp09, vp8, vp9, other], description: Videos only. }+ poster:+ type: object+ description: A still of at most 480 px — always for images and GIFs, when possible for videos.+ properties:+ hash: { type: string, pattern: '^[0-9a-f]{64}$' }+ mime: { type: string, enum: [image/webp, image/jpeg] }+ size: { type: integer }+ width: { type: integer }+ height: { type: integer }+ at: { type: number, description: When it was attached, ms. }++ MediaConfig:+ type: object+ description: >-+ The instance's media caps (`MEDIA_*`), in MB of 2^20 bytes. Absent when the instance+ runs with `MEDIA_UPLOADS=0`.+ properties:+ imageMB: { type: number, description: 'A photo after the app re-encoded it, or a poster.' }+ gifMB: { type: number }+ videoMB: { type: number }+ videoSec: { type: number }+ quotaMB: { type: number, description: Per profile; 0 = no cap. }+ example: { imageMB: 2, gifMB: 8, videoMB: 40, videoSec: 60, quotaMB: 200 }++ MediaUsage:+ type: object+ properties:+ bytes: { type: integer, description: What the profile's stored files take. }+ count: { type: integer }+ quotaBytes: { type: integer, description: The cap; 0 = no cap. }++ MediaStored:+ type: object+ properties:+ ok: { type: boolean, const: true }+ hash: { type: string }+ mime: { type: string, description: 'What the bytes are — not necessarily what was declared.' }+ size: { type: integer }+ existed: { type: boolean, description: The server already had this file. }+Workout:type: objectdescription: One completed workout (representative shape).diff --git a/api/server.js b/api/server.jsindex 645ec29fa..4d3415bc9 100644--- a/api/server.js+++ b/api/server.js@@ -7,6 +7,7 @@ import path from 'node:path';import https from 'node:https';import dns from 'node:dns';import net from 'node:net';+import { pipeline } from 'node:stream/promises';import {generateRegistrationOptions, verifyRegistrationResponse,generateAuthenticationOptions, verifyAuthenticationResponse@@ -28,6 +29,7 @@ import {listPasskeys, addPasskeyRecord, renamePasskeyRecord, removePasskeyRecord, MAX_PASSKEYS} from './passkeys-store.js';import { createDeviceLink, findDeviceLink, burnDeviceLink, dropDeviceLinks } from './device-link.js';+import { createMediaStore, mediaLimits, mediaConfig, MediaError, HASH_RE } from './media.js';const PORT = +(process.env.PORT || 3000);const DATA = process.env.DATA_DIR || '/data';@@ -1407,6 +1409,126 @@ const passkeyRoutes = {}};+/* ---------- photos & videos of custom exercises (api/media.js) ---------- */+// One photo, GIF or short video per custom exercise, stored per profile and named by its sha256.+// The state only ever carries a small ref; the bytes arrive and leave through the routes below.+// MEDIA_UPLOADS=0 takes the routes and the /api/config block away, but the store is created+// either way: an admin deleting a profile must still remove files uploaded while it was on.+const MEDIA_LIMITS = mediaLimits(process.env);+const MEDIA_ON = MEDIA_LIMITS.enabled;+const MEDIA = createMediaStore({ dir: path.join(DATA, 'uploads'), limits: MEDIA_LIMITS, readState });+// Leftovers of uploads the previous process was receiving when it stopped.+try { MEDIA.cleanTmp(); } catch (e) { console.error('media: boot cleanup failed', e.message); }+// Per profile, not per address: every upload is signed in, and behind a proxy one address can be+// a whole household. The hourly budget is generous (an import of a backup re-uploads everything+// at once); the two-in-flight cap in media.js is what keeps one phone from holding many sockets.+const MEDIA_BURST = createWindow({ max: MEDIA_LIMITS.uploadsPerHour, windowMs: 3600000 });+// Deleting everything unreferenced is what "Reset everything" does; ten an hour is plenty.+const MEDIA_SWEEPS = createWindow({ max: 10, windowMs: 3600000 });+// Uid -> until when its throttle has already been recorded, so a client hammering a closed+// window writes one audit line per window, not one per request.+const mediaThrottleNoted = new Map();+setInterval(() => {+ MEDIA_BURST.sweep(); MEDIA_SWEEPS.sweep();+ for (const [k, until] of mediaThrottleNoted) if (until < Date.now()) mediaThrottleNoted.delete(k);+}, 600000).unref();+function mediaThrottle(req, user, win) {+ const wait = win.take(user.id);+ if (!wait) return;+ if ((mediaThrottleNoted.get(user.id) || 0) < Date.now()) {+ mediaThrottleNoted.set(user.id, Date.now() + wait * 1000);+ audit(req, 'media.throttled', { ok: false, user, msg: win === MEDIA_SWEEPS ? 'sweep' : 'upload' });+ }+ throw new MediaError(429, 'locked', { retryAfter: wait }, { 'Retry-After': String(wait) });+}+// Removes files that no profile's readable state has referenced for MEDIA_GC_GRACE_DAYS. Only+// profiles in db.json are swept, and only when their state parses (see media.js for why a+// missing anything never means "delete"). First pass a few minutes after boot, so an instance+// that is redeployed more often than hourly still gets one.+function mediaSweepAll() {+ try {+ const r = MEDIA.sweepAll({ uids: db.users.map(u => u.id) });+ if (r.removed || r.tmp) console.log(`media: swept ${r.removed} unreferenced file(s), ${(r.freedBytes / 1048576).toFixed(1)} MB, ${r.tmp} stale upload(s)`);+ } catch (e) { console.error('media: sweep failed', e); }+}+if (MEDIA_ON) {+ setInterval(mediaSweepAll, 3600000).unref();+ setTimeout(mediaSweepAll, 5 * 60000).unref();+}++// A stored file, streamed. The headers are the point: the bytes were chosen by a user, so the+// answer must never be something a browser would run. The Content-Type comes from the sniffed+// extension (never text/* or SVG), nosniff stops a browser second-guessing it, the sandbox CSP+// neuters it even if opened as a page, CORP keeps other sites from embedding it, and no-store+// keeps a copy out of the HTTP cache — the app's own media store is the only client cache, and+// an HTTP-cache copy would outlive the sign-out purge. No Range and no ETag: nothing asks.+async function sendMediaFile(res, f, hash) {+ let fd;+ try { fd = fs.openSync(f.path, 'r'); }+ catch { throw new MediaError(404, 'media-missing'); } // swept between the lookup and here+ const size = fs.fstatSync(fd).size;+ res.writeHead(200, {+ 'Content-Type': f.mime,+ 'Content-Length': String(size),+ 'Cache-Control': 'private, no-store',+ 'X-Content-Type-Options': 'nosniff',+ 'Content-Security-Policy': "default-src 'none'; sandbox",+ 'Cross-Origin-Resource-Policy': 'same-origin',+ 'Content-Disposition': `inline; filename="${hash}.${f.ext}"`,+ 'X-Robots-Tag': 'noindex',+ // The paired phone reads the length before the body to refuse an oversized answer; it is+ // CORS-safelisted already, this only spells it out for older WebViews.+ 'Access-Control-Expose-Headers': 'Content-Length'+ });+ try { await pipeline(fs.createReadStream(null, { fd }), res); }+ catch { res.destroy(); } // the client went away mid-download; nothing left to answer+}++const mediaRoutes = {+ // The raw bytes of one file, named by the sha256 the client computed. media.js checks the+ // hash, the magic bytes, the caps, the quota and the free disk; this route only adds the+ // session and the hourly budget.+ 'PUT /api/media/{hash}': async (req, res) => {+ const user = readSession(req);+ if (!user) return json(res, 401, { error: 'not signed in' });+ mediaThrottle(req, user, MEDIA_BURST);+ const r = await MEDIA.receive(user.id, req.mediaHash, req);+ json(res, r.status, r.body);+ },+ 'GET /api/media/{hash}': async (req, res) => {+ const user = readSession(req);+ if (!user) return json(res, 401, { error: 'not signed in' });+ // Only ever the caller's own folder: another profile's file is exactly as missing as one+ // that was never uploaded.+ const f = MEDIA.file(user.id, req.mediaHash);+ if (!f) throw new MediaError(404, 'media-missing');+ await sendMediaFile(res, f, req.mediaHash);+ },+ // Which of these the server does not have, so a device uploads only those. Answered from the+ // in-memory list of the caller's own folder.+ 'POST /api/media/missing': async (req, res) => {+ const user = readSession(req);+ if (!user) return json(res, 401, { error: 'not signed in' });+ const body = await readBody(req);+ const hashes = body.hashes;+ if (!Array.isArray(hashes) || hashes.length > 1000 || !hashes.every(h => typeof h === 'string' && HASH_RE.test(h))) {+ throw new MediaError(400, 'bad-request', { error: 'hashes must be a list of at most 1000 lowercase sha256 hex strings' });+ }+ json(res, 200, MEDIA.missing(user.id, hashes));+ },+ // "Reset everything": every file the caller's current state does not reference goes now,+ // without the grace. A state that cannot be read removes nothing.+ 'POST /api/media/sweep': async (req, res) => {+ const user = readSession(req);+ if (!user) return json(res, 401, { error: 'not signed in' });+ await readBody(req);+ mediaThrottle(req, user, MEDIA_SWEEPS);+ const r = MEDIA.sweep(user.id, { graceMs: 0 });+ audit(req, 'media.sweep', { user, msg: `${r.removed} file(s), ${(r.freedBytes / 1048576).toFixed(1)} MB${r.skipped ? ', state unreadable' : ''}` });+ json(res, 200, { removed: r.removed, freedBytes: r.freedBytes, usage: MEDIA.usage(user.id) });+ }+};+/* ---------- routes ---------- */const routes = {'GET /api/health': async (req, res) => json(res, 200, { ok: true, users: db.users.length }),@@ -1434,6 +1556,9 @@ const routes = {invite_only: INVITE_ONLY, allow_guest: ALLOW_GUEST,// Only when on, so an instance without passwords answers exactly as it did before (#118)....(PASSWORD_LOGIN ? { password_login: true } : {}),+ // Public like the two flags above: the caps are not a secret, and the absence of the+ // block is how the app knows this server does not take photos and videos at all.+ ...(MEDIA_ON ? { media: mediaConfig(MEDIA_LIMITS) } : {}),...(readSession(req) ? { coach: coachConfig.publicConfig() } : {})});},@@ -1725,6 +1850,12 @@ const routes = {// the old revision until some later write happened to land on a different tick. This is the// only writer of a state file in the tree, so evicting here is the whole fix.stateCache.delete(user.id);+ // Starts (or stops) the grace clock of every stored file this write stopped (or started)+ // referencing. Bookkeeping only: the state is already saved, so nothing here may turn a+ // successful write into an error.+ if (MEDIA_ON) {+ try { MEDIA.noteState(user.id, body.state); } catch (e) { console.error('media noteState', e); }+ }json(res, 200, { ok: true, ts: body.state._ts || null, rev: body.state._rev });},@@ -1907,6 +2038,8 @@ const routes = {// The training history and any Coach credential of theirs, both outside db.json.try { fs.unlinkSync(stateFile(u.id)); } catch { /* already gone */ }try { coachConfig.clearProfileAuth(u.id); } catch { /* nothing stored */ }+ // Their photos and videos — the one place a profile's folder under uploads/ is removed.+ try { MEDIA.removeUser(u.id); } catch (e) { console.error('media: could not remove uploads of', u.id, e.message); }saveDb();// Logged with the name, because the id is about to mean nothing to anyone reading this back.audit(req, 'admin.user.delete', { user: admin, msg: name });@@ -1990,7 +2123,12 @@ const routes = {// Routes live in coach/routes.js and are handed the helpers above rather than importing// them: they are closures over db and SECRET, and passing them in keeps that module free of// a cycle. Every one of them is inert while the feature is unconfigured.- ...coachRoutes({ json, readBody, readSession, requireAdmin })+ ...coachRoutes({ json, readBody, readSession, requireAdmin }),++ /* ---------- photos & videos ---------- */+ // Absent, not refusing, when MEDIA_UPLOADS=0: a 404 is what a server from before the feature+ // answers, and the client already treats that as "this server does not store them".+ ...(MEDIA_ON ? mediaRoutes : {})};/* ---------- Coach: boot recovery, notifications, scheduled reviews ---------- */@@ -2031,7 +2169,11 @@ const server = http.createServer(async (req, res) => {let url;try { url = new URL(req.url, 'http://x'); }catch { return json(res, 400, { error: 'bad request' }); }- const key = req.method + ' ' + url.pathname;+ let key = req.method + ' ' + url.pathname;+ // The one route with a parameter in its path. Mapped onto its template key here so the table+ // above stays a plain lookup, and so csrfOk and the catch-all see one name for every file.+ const mm = /^\/api\/media\/([0-9a-f]{64})$/.exec(url.pathname);+ if (mm) { key = req.method + ' /api/media/{hash}'; req.mediaHash = mm[1]; }const handler = routes[key];if (!handler) return json(res, 404, { error: 'not found' });if (!csrfOk(req, key)) {@@ -2056,6 +2198,11 @@ const server = http.createServer(async (req, res) => {if (!res.headersSent) json(res, e.status, { error: e.message });return;}+ // A refused upload or a missing file: the caller's to act on, never a logged 500.+ if (e instanceof MediaError) {+ if (!res.headersSent) json(res, e.status, { error: e.message, code: e.code, ...e.extra }, e.headers);+ return;+ }// Every scrypt slot and the short queue behind them are taken (password.js).if (e instanceof BusyError) {if (!res.headersSent) json(res, 503, { error: 'the server is busy — try again in a moment', code: 'busy' }, { 'Retry-After': '2' });@@ -2065,6 +2212,11 @@ const server = http.createServer(async (req, res) => {if (!res.headersSent) json(res, 500, { error: 'server error' });}});+// Node's default of 300 s for a whole request would answer 408 to a 40 MB video on a ~1 Mbit/s+// uplink. Half an hour covers that; a stalled upload is cut much sooner by its own 60 s idle+// timer in media.js, and a client that never finishes its headers still meets headersTimeout.+server.requestTimeout = 30 * 60000;+server.headersTimeout = 60000;// The port is read back off the listener rather than echoed from PORT, so the line states the// port that was actually bound: with PORT=0 the OS picks one, and a caller that did not choose it// (the tests spawn the server that way, and so does anyone running two instances on one box) hasdiff --git a/api/test/dockerfile-copy.test.js b/api/test/dockerfile-copy.test.jsnew file mode 100644index 000000000..a7c96c091--- /dev/null+++ b/api/test/dockerfile-copy.test.js@@ -0,0 +1,35 @@+/* The api image carries only the files named on api/Dockerfile's COPY line (plus coach/). A+ root-level module that server.js imports and that line forgets is invisible to every test run+ from the source tree and shows up only when the published image boots — as+ ERR_MODULE_NOT_FOUND, on every instance that pulled it. So the line is checked here, for+ server.js's own imports and for whatever those modules import from the api root in turn. */+import { test } from 'node:test';+import assert from 'node:assert/strict';+import fs from 'node:fs';+import path from 'node:path';+import { fileURLToPath } from 'node:url';++const API = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');+const read = f => fs.readFileSync(path.join(API, f), 'utf8');++// Every `./name.js` a file imports, statically or dynamically — root-level only; coach/ is+// copied as a whole directory.+const rootImports = src => [...src.matchAll(/(?:from\s+|import\s*\(\s*)['"]\.\/([^'"/]+\.js)['"]/g)].map(m => m[1]);++test('every root-level module server.js needs is on the Dockerfile COPY line', () => {+ const line = read('Dockerfile').split('\n').find(l => /^COPY\s+server\.js\b/.test(l));+ assert.ok(line, 'the COPY line that carries server.js');+ const copied = new Set(line.trim().split(/\s+/).slice(1, -1));++ const needed = new Set(['server.js']);+ const queue = ['server.js'];+ while (queue.length) {+ for (const dep of rootImports(read(queue.shift()))) {+ if (!needed.has(dep)) { needed.add(dep); queue.push(dep); }+ }+ }+ assert.ok(needed.has('media.js'), 'the walk finds the media store');+ const missing = [...needed].filter(f => !copied.has(f));+ assert.deepEqual(missing, [], `add ${missing.join(', ')} to the COPY line in api/Dockerfile`);+ for (const f of copied) assert.ok(fs.existsSync(path.join(API, f)), `${f} is copied but does not exist`);+});diff --git a/api/test/server-config-session.test.js b/api/test/server-config-session.test.jsindex e4b2a7fc5..5f0a4da65 100644--- a/api/test/server-config-session.test.js+++ b/api/test/server-config-session.test.js@@ -31,9 +31,11 @@ async function startServer(t, env = {}) {// The fixture provider is "connected" by definition (coach/config.js isConnected), which is all// publicConfig() needs to produce a block.fs.writeFileSync(path.join(dataDir, 'coach.json'), JSON.stringify({ enabled: true, provider: 'fixture' }));+ // MEDIA_UPLOADS=0: the answers below are compared whole, and they are about the login flags+ // and the Coach block. The public media block has its own tests in server-media.test.js.const child = spawn(process.execPath, ['server.js'], {cwd: API, stdio: ['ignore', 'pipe', 'pipe'],- env: { ...process.env, PORT: '0', DATA_DIR: dataDir, ORIGIN: 'http://localhost:8080', RP_ID: 'localhost', INVITE_ONLY: '1', ALLOW_GUEST: '0', COACH_DISABLED: '', ...env }+ env: { ...process.env, PORT: '0', DATA_DIR: dataDir, ORIGIN: 'http://localhost:8080', RP_ID: 'localhost', INVITE_ONLY: '1', ALLOW_GUEST: '0', COACH_DISABLED: '', MEDIA_UPLOADS: '0', ...env }});t.after(() => { child.kill('SIGKILL'); fs.rmSync(dataDir, { recursive: true, force: true }); });let log = '';diff --git a/api/test/server-media.test.js b/api/test/server-media.test.jsnew file mode 100644index 000000000..674b6ee7a--- /dev/null+++ b/api/test/server-media.test.js@@ -0,0 +1,437 @@+/* The media routes on a real server.js in a child process: sessions, CSRF, CORS, the throttle,+ the headers a stored file is served with, the config block, the PUT /api/data hook, admin+ delete, and what is left on disk after each refusal. Caps are lowered through the MEDIA_*+ variables where a test needs to cross one. media-receive.test.js and media-gc.test.js cover+ the store itself in-process. */+import { test } from 'node:test';+import assert from 'node:assert/strict';+import crypto from 'node:crypto';+import fs from 'node:fs';+import http from 'node:http';+import os from 'node:os';+import path from 'node:path';+import { spawn } from 'node:child_process';+import { fileURLToPath } from 'node:url';+import { boundPort } from './helpers.mjs';+import * as M from './media-samples.mjs';++const API = path.join(path.dirname(fileURLToPath(import.meta.url)), '..');+const ORIGIN = 'http://localhost:8080';+const SECRET = crypto.randomBytes(32).toString('hex');+const mint = uid => {+ const payload = `${uid}:${Date.now() + 86400000}:0`;+ return payload + '.' + crypto.createHmac('sha256', SECRET).update(payload).digest('base64url');+};+const U1 = 'u_media_1', U2 = 'u_media_2', ADMIN = 'u_media_adm';+const cookie = uid => ({ Cookie: `gymsid=${mint(uid)}`, Origin: ORIGIN });+const bearer = uid => ({ Authorization: `Bearer ${mint(uid)}` });+const refState = (...hashes) => ({+ unit: 'kg', routines: [], workouts: [],+ customEx: hashes.map((h, i) => ({ id: 'cx' + i, n: 'Sandbag ' + i, bp: 'back', custom: true, media: { kind: 'image', hash: h, mime: 'image/jpeg', size: 1, width: 1, height: 1, at: 1 } }))+});++async function start(t, env = {}) {+ const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gym-media-'));+ fs.writeFileSync(path.join(dataDir, 'secret'), SECRET, { mode: 0o600 });+ fs.writeFileSync(path.join(dataDir, 'db.json'), JSON.stringify({+ users: [+ { id: U1, name: 'Ana', created: new Date().toISOString() },+ { id: U2, name: 'Bo', created: new Date().toISOString() },+ { id: ADMIN, name: 'Admin', created: new Date().toISOString(), admin: true }+ ],+ creds: [], subs: [], invites: []+ }));+ const child = spawn(process.execPath, ['server.js'], {+ cwd: API, stdio: ['ignore', 'pipe', 'pipe'],+ env: { ...process.env, PORT: '0', DATA_DIR: dataDir, ORIGIN, RP_ID: 'localhost', MEDIA_MIN_FREE_MB: '1', ...env }+ });+ const h = { log: '', dataDir, uploads: path.join(dataDir, 'uploads') };+ child.stdout.on('data', d => h.log += d);+ child.stderr.on('data', d => h.log += d);+ t.after(() => { child.kill('SIGKILL'); fs.rmSync(dataDir, { recursive: true, force: true }); });+ h.port = await boundPort(child, () => h.log);+ h.api = `http://127.0.0.1:${h.port}`;+ h.put = (bytes, { uid = U1, declare = 'image/jpeg', hash = M.sha(bytes), headers } = {}) =>+ fetch(`${h.api}/api/media/${hash}`, { method: 'PUT', headers: { ...(headers || cookie(uid)), 'Content-Type': declare }, body: bytes });+ h.get = (hash, uid = U1, headers) => fetch(`${h.api}/api/media/${hash}`, { headers: headers || cookie(uid) });+ h.post = (route, body, uid = U1) => fetch(`${h.api}${route}`, { method: 'POST', headers: { ...cookie(uid), 'Content-Type': 'application/json' }, body: JSON.stringify(body) });+ h.pushState = (state, uid = U1) => fetch(`${h.api}/api/data`, { method: 'PUT', headers: { ...cookie(uid), 'Content-Type': 'application/json' }, body: JSON.stringify({ state }) });+ h.files = uid => { try { return fs.readdirSync(path.join(h.uploads, uid)).filter(n => !n.startsWith('.')).sort(); } catch { return []; } };+ h.tmp = uid => { try { return fs.readdirSync(path.join(h.uploads, uid, '.tmp')); } catch { return []; } };+ h.stackFrames = () => h.log.split('\n').filter(l => /^\s+at /.test(l)).length;+ return h;+}++/* A request written by hand, for what fetch cannot do: stop halfway, stream without a length,+ or claim a length it never sends. Resolves `response` with {status, headers, body}. */+function rawPut(h, hash, { headers = {}, declare = 'image/jpeg', length } = {}) {+ const hd = { ...cookie(U1), 'Content-Type': declare, ...headers };+ if (length != null) hd['Content-Length'] = String(length);+ const req = http.request(`${h.api}/api/media/${hash}`, { method: 'PUT', headers: hd });+ const response = new Promise((resolve, reject) => {+ req.on('response', res => {+ let body = '';+ res.setEncoding('utf8');+ res.on('data', d => body += d);+ res.on('end', () => { let j = null; try { j = JSON.parse(body); } catch { /* not json */ } resolve({ status: res.statusCode, headers: res.headers, body: j }); });+ });+ req.on('error', reject);+ });+ return { req, response };+}+const sleep = ms => new Promise(r => setTimeout(r, ms));++test('every media route wants a session', async t => {+ const h = await start(t);+ const hash = 'a'.repeat(64);+ const noAuth = { Origin: ORIGIN };+ assert.equal((await fetch(`${h.api}/api/media/${hash}`, { method: 'PUT', headers: { ...noAuth, 'Content-Type': 'image/jpeg' }, body: M.jpeg() })).status, 401);+ assert.equal((await fetch(`${h.api}/api/media/${hash}`)).status, 401);+ assert.equal((await fetch(`${h.api}/api/media/missing`, { method: 'POST', headers: noAuth, body: '{"hashes":[]}' })).status, 401);+ assert.equal((await fetch(`${h.api}/api/media/sweep`, { method: 'POST', headers: noAuth, body: '{}' })).status, 401);+ assert.equal(fs.existsSync(h.uploads), false);+});++test('an upload is served back whole, with the headers that keep it from ever being a page', async t => {+ const h = await start(t);+ const bytes = M.webp(5000);+ const up = await h.put(bytes, { declare: 'image/webp' });+ assert.equal(up.status, 201);+ assert.deepEqual(await up.json(), { ok: true, hash: M.sha(bytes), mime: 'image/webp', size: 5000, existed: false });++ const r = await h.get(M.sha(bytes));+ assert.equal(r.status, 200);+ assert.equal(r.headers.get('content-type'), 'image/webp');+ assert.equal(r.headers.get('content-length'), '5000');+ assert.equal(r.headers.get('cache-control'), 'private, no-store');+ assert.equal(r.headers.get('x-content-type-options'), 'nosniff');+ assert.equal(r.headers.get('content-security-policy'), "default-src 'none'; sandbox");+ assert.equal(r.headers.get('cross-origin-resource-policy'), 'same-origin');+ assert.equal(r.headers.get('content-disposition'), `inline; filename="${M.sha(bytes)}.webp"`);+ assert.equal(r.headers.get('x-robots-tag'), 'noindex');+ assert.equal(r.headers.get('etag'), null);+ assert.equal(r.headers.get('accept-ranges'), null);+ assert.ok(Buffer.from(await r.arrayBuffer()).equals(bytes));++ // Again: already there, counted once.+ const again = await h.put(bytes, { declare: 'image/webp' });+ assert.equal(again.status, 200);+ assert.equal((await again.json()).existed, true);+ const miss = await (await h.post('/api/media/missing', { hashes: [M.sha(bytes)] })).json();+ assert.deepEqual(miss, { missing: [], usage: { bytes: 5000, count: 1, quotaBytes: 200 * 1048576 } });+ assert.equal(h.stackFrames(), 0, h.log);+});++test("another profile's file is exactly as missing as one that was never uploaded", async t => {+ const h = await start(t);+ const bytes = M.jpeg();+ assert.equal((await h.put(bytes)).status, 201);+ const r = await h.get(M.sha(bytes), U2);+ assert.equal(r.status, 404);+ assert.deepEqual(await r.json(), { error: 'no such file', code: 'media-missing' });+ const never = await h.get('0'.repeat(64), U2);+ assert.equal(never.status, 404);+ assert.deepEqual((await (await h.post('/api/media/missing', { hashes: [M.sha(bytes)] }, U2)).json()).missing, [M.sha(bytes)]);+ // A path that is not a hash is no media route at all.+ assert.equal((await h.get('A'.repeat(64))).status, 404);+ assert.equal((await fetch(`${h.api}/api/media/..%2Fstate-${U1}.json`, { headers: cookie(U1) })).status, 404);+});++test('refusals: a wrong hash, SVG, a video posing as an image — each leaves nothing on disk', async t => {+ const h = await start(t);+ const bytes = M.jpeg();+ let r = await h.put(bytes, { hash: 'f'.repeat(64) });+ assert.equal(r.status, 400);+ assert.deepEqual(await r.json(), { error: 'the file does not match its name', code: 'hash-mismatch' });++ r = await h.put(M.svg(), { declare: 'image/svg+xml' });+ assert.deepEqual([r.status, (await r.json()).code], [415, 'media-type']);+ r = await h.put(M.svg(), { declare: 'image/png' });+ assert.deepEqual([r.status, (await r.json()).code], [415, 'media-type']);+ r = await h.put(M.html(), { declare: 'image/jpeg' });+ assert.deepEqual([r.status, (await r.json()).code], [415, 'media-type']);+ r = await h.put(M.mp4(), { declare: 'image/jpeg' });+ assert.deepEqual([r.status, (await r.json()).code], [415, 'media-type']);++ assert.deepEqual(h.files(U1), []);+ assert.deepEqual(h.tmp(U1), []);+ assert.equal(h.stackFrames(), 0, `a refusal is not a server error:\n${h.log}`);+});++test('a declared length over the cap is answered before the body is sent', async t => {+ const h = await start(t);+ const hash = 'c'.repeat(64);+ const { req, response } = rawPut(h, hash, { length: 100 * 1024 * 1024 });+ req.write(M.jpeg(1024)); // and never the other 99.99 MB+ const r = await response;+ req.destroy();+ assert.equal(r.status, 413);+ assert.deepEqual(r.body, { error: 'that file is too large', code: 'media-too-large', maxMB: 2 });+ assert.deepEqual(h.tmp(U1), []);+});++test('the cap of what the bytes are applies, and the quota says how full it is', async t => {+ // 10 KB images, 1 MB GIFs, a 25 KB quota.+ const h = await start(t, { MEDIA_IMAGE_MAX_MB: '0.009765625', MEDIA_GIF_MAX_MB: '1', MEDIA_QUOTA_MB: '0.0244140625' });+ const jpegAsGif = M.jpeg(20000);+ let r = await h.put(jpegAsGif, { declare: 'image/gif' });+ assert.equal(r.status, 413);+ assert.deepEqual(await r.json(), { error: 'that file is too large', code: 'media-too-large', maxMB: 0.009765625 });++ assert.equal((await h.put(M.jpeg(9000), { uid: U2 })).status, 201);+ assert.equal((await h.put(M.jpeg(9000), { uid: U2 })).status, 201);+ r = await h.put(M.jpeg(9000), { uid: U2 });+ assert.equal(r.status, 413);+ assert.deepEqual(await r.json(), { error: 'your space for photos and videos is full', code: 'media-quota', usedMB: 0, quotaMB: 0.0244140625 });+ assert.equal(h.files(U2).length, 2);+});++test('a video longer than MEDIA_VIDEO_MAX_SEC is refused, and a broken one too', async t => {+ const h = await start(t);+ let r = await h.put(M.mp4({ seconds: 90 }), { declare: 'video/mp4' });+ assert.deepEqual([r.status, await r.json()], [413, { error: 'that video is too long', code: 'media-too-long', maxSec: 60 }]);+ r = await h.put(Buffer.concat([M.ftyp(), M.box('mdat', Buffer.alloc(64))]), { declare: 'video/mp4' });+ assert.deepEqual([r.status, (await r.json()).code], [415, 'media-invalid']);+ r = await h.put(M.mp4({ brand: 'qt ', seconds: 30, moovAtEnd: true }), { declare: 'video/quicktime' });+ assert.deepEqual([r.status, (await r.json()).mime], [201, 'video/quicktime']);+});++test('an upload without Content-Length works, and is cut off at the cap while it streams', async t => {+ const h = await start(t, { MEDIA_IMAGE_MAX_MB: '0.1' });+ const small = M.jpeg(50000);+ let { req, response } = rawPut(h, M.sha(small));+ req.write(small.subarray(0, 20000));+ await sleep(20);+ req.end(small.subarray(20000));+ let r = await response;+ assert.equal(r.status, 201, JSON.stringify(r.body));++ const big = M.jpeg(1024 * 1024);+ ({ req, response } = rawPut(h, M.sha(big)));+ let answered = false;+ response.then(() => { answered = true; }, () => { answered = true; });+ let sent = 0;+ while (!answered && sent < big.length) {+ req.write(big.subarray(sent, sent + 16384));+ sent += 16384;+ await sleep(2);+ }+ r = await response;+ req.destroy();+ assert.equal(r.status, 413, 'the client gets the answer, not a reset');+ assert.equal(r.body.code, 'media-too-large');+ assert.ok(sent < big.length, `the answer came while the upload was still going (${sent} bytes sent)`);+ assert.deepEqual(h.files(U1), [`${M.sha(small)}.jpg`]);+ assert.deepEqual(h.tmp(U1), []);+});++test('the same file from two devices at once: one file, counted once', async t => {+ const h = await start(t);+ const bytes = M.jpeg(1024 * 1024);+ const rs = await Promise.all([h.put(bytes), h.put(bytes, { headers: bearer(U1) })]);+ assert.deepEqual(rs.map(r => r.status).sort(), [200, 201]);+ assert.deepEqual(h.files(U1), [`${M.sha(bytes)}.jpg`]);+ const miss = await (await h.post('/api/media/missing', { hashes: [M.sha(bytes)] })).json();+ assert.deepEqual(miss.usage, { bytes: 1024 * 1024, count: 1, quotaBytes: 200 * 1048576 });+});++test('a cross-site browser upload is refused; a paired phone with its bearer token is not', async t => {+ const h = await start(t);+ const bytes = M.jpeg();+ let r = await h.put(bytes, { headers: { ...cookie(U1), 'Sec-Fetch-Site': 'cross-site' } });+ assert.equal(r.status, 403);+ r = await h.put(bytes, { headers: { Cookie: `gymsid=${mint(U1)}`, Origin: 'https://evil.example' } });+ assert.equal(r.status, 403);+ assert.deepEqual(h.files(U1), []);++ // The Android WebView's origin; the token is what authenticates, the origin is reflected.+ const phone = { ...bearer(U1), Origin: 'https://localhost' };+ r = await h.put(bytes, { headers: phone });+ assert.equal(r.status, 201);+ assert.equal(r.headers.get('access-control-allow-origin'), 'https://localhost');+ assert.equal(r.headers.get('access-control-allow-credentials'), null);+ const g = await h.get(M.sha(bytes), U1, phone);+ assert.equal(g.status, 200);+ assert.equal(g.headers.get('access-control-allow-origin'), 'https://localhost');+ assert.ok(Buffer.from(await g.arrayBuffer()).equals(bytes));++ // The preflight a WebView sends before that PUT.+ const pre = await fetch(`${h.api}/api/media/${M.sha(bytes)}`, {+ method: 'OPTIONS',+ headers: { Origin: 'https://localhost', 'Access-Control-Request-Method': 'PUT', 'Access-Control-Request-Headers': 'content-type,authorization' }+ });+ assert.equal(pre.status, 204);+ assert.match(pre.headers.get('access-control-allow-methods'), /\bPUT\b/);+ assert.match(pre.headers.get('access-control-allow-headers'), /Content-Type/);+ assert.match(pre.headers.get('access-control-allow-headers'), /Authorization/);+ assert.equal(pre.headers.get('access-control-allow-origin'), 'https://localhost');+});++test('the hourly budget answers 429 with Retry-After, and is recorded once', async t => {+ const h = await start(t, { MEDIA_UPLOADS_PER_HOUR: '2' });+ assert.equal((await h.put(M.jpeg())).status, 201);+ assert.equal((await h.put(M.jpeg())).status, 201);+ for (let i = 0; i < 3; i++) {+ const r = await h.put(M.jpeg());+ assert.equal(r.status, 429);+ const retry = +r.headers.get('retry-after');+ assert.ok(retry > 3000 && retry <= 3600, `retry-after ${retry}`);+ assert.deepEqual(await r.json(), { error: 'too many uploads — try again later', code: 'locked', retryAfter: retry });+ }+ assert.equal((await h.put(M.jpeg(), { uid: U2 })).status, 201, 'per profile, not per instance');+ const audit = fs.readFileSync(path.join(h.dataDir, 'audit.log'), 'utf8').trim().split('\n').map(l => JSON.parse(l));+ assert.equal(audit.filter(e => e.ev === 'media.throttled').length, 1);+});++test('a third upload while two are in flight is refused as busy', async t => {+ const h = await start(t);+ const a = rawPut(h, 'a'.repeat(64), { length: 1000000 });+ const b = rawPut(h, 'b'.repeat(64), { length: 1000000 });+ a.req.write(M.jpeg(100));+ b.req.write(M.jpeg(100));+ a.response.catch(() => {}); b.response.catch(() => {});+ await sleep(150);+ const r = await h.put(M.jpeg());+ assert.equal(r.status, 429);+ assert.equal(r.headers.get('retry-after'), '5');+ assert.deepEqual(await r.json(), { error: 'too many uploads at once — try again in a moment', code: 'busy', retryAfter: 5 });+ a.req.destroy(); b.req.destroy();+ await sleep(150);+ assert.equal((await h.put(M.jpeg())).status, 201, 'the slots come back when the two go away');+ assert.deepEqual(h.tmp(U1), []);+});++test('507 when the disk has less free than MEDIA_MIN_FREE_MB', async t => {+ const h = await start(t, { MEDIA_MIN_FREE_MB: String(1024 * 1024 * 1024) });+ const r = await h.put(M.jpeg());+ assert.equal(r.status, 507);+ assert.deepEqual(await r.json(), { error: 'the server is running out of disk space', code: 'storage-full' });+});++test('/missing answers from the caller\'s own folder and refuses what is not a list of hashes', async t => {+ const h = await start(t);+ const a = M.jpeg(), b = M.png();+ await h.put(a);+ const r = await (await h.post('/api/media/missing', { hashes: [M.sha(b), M.sha(a), M.sha(b)] })).json();+ assert.deepEqual(r, { missing: [M.sha(b)], usage: { bytes: 1000, count: 1, quotaBytes: 200 * 1048576 } });+ for (const body of [{}, { hashes: 'x' }, { hashes: ['A'.repeat(64)] }, { hashes: [1] }, { hashes: Array(1001).fill('a'.repeat(64)) }]) {+ const res = await h.post('/api/media/missing', body);+ assert.equal(res.status, 400, JSON.stringify(body).slice(0, 60));+ assert.equal((await res.json()).code, 'bad-request');+ }+});++test('a state push starts the grace of a dropped file, and /sweep removes it at once', async t => {+ const h = await start(t);+ const a = M.jpeg(), b = M.jpeg(2000);+ await h.put(a);+ await h.put(b);+ assert.equal((await h.pushState(refState(M.sha(a), M.sha(b)))).status, 200);+ const gc = () => JSON.parse(fs.readFileSync(path.join(h.uploads, U1, '.gc.json'), 'utf8'));+ assert.deepEqual(gc(), {}, 'both referenced: no marks');+ assert.equal((await h.pushState(refState(M.sha(a)))).status, 200);+ assert.deepEqual(Object.keys(gc()), [M.sha(b)]);++ const r = await h.post('/api/media/sweep', {});+ assert.equal(r.status, 200);+ assert.deepEqual(await r.json(), { removed: 1, freedBytes: 2000, usage: { bytes: 1000, count: 1, quotaBytes: 200 * 1048576 } });+ assert.equal((await h.get(M.sha(b))).status, 404);+ assert.equal((await h.get(M.sha(a))).status, 200);+ const audit = fs.readFileSync(path.join(h.dataDir, 'audit.log'), 'utf8');+ assert.match(audit, /"ev":"media\.sweep"/);+});++test('/sweep with a state that does not parse removes nothing', async t => {+ const h = await start(t);+ const a = M.jpeg();+ await h.put(a, { uid: U2 });+ fs.writeFileSync(path.join(h.dataDir, `state-${U2}.json`), '{"customEx": [tor');+ const r = await h.post('/api/media/sweep', {}, U2);+ assert.equal(r.status, 200);+ assert.equal((await r.json()).removed, 0);+ assert.deepEqual(h.files(U2), [`${M.sha(a)}.jpg`]);+ // And with no state file at all.+ fs.unlinkSync(path.join(h.dataDir, `state-${U2}.json`));+ assert.equal((await (await h.post('/api/media/sweep', {}, U2)).json()).removed, 0);+ assert.deepEqual(h.files(U2), [`${M.sha(a)}.jpg`]);+});++test('the sweep route has its own budget of 10 an hour', async t => {+ const h = await start(t);+ for (let i = 0; i < 10; i++) assert.equal((await h.post('/api/media/sweep', {})).status, 200);+ const r = await h.post('/api/media/sweep', {});+ assert.equal(r.status, 429);+ assert.equal((await r.json()).code, 'locked');+});++test('PUT /api/data still answers 200 when the media bookkeeping fails', async t => {+ const h = await start(t);+ const a = M.jpeg();+ await h.put(a);+ const gcPath = path.join(h.uploads, U1, '.gc.json');+ fs.rmSync(gcPath, { force: true });+ fs.mkdirSync(gcPath); // a write to it now throws+ const r = await h.pushState(refState());+ assert.equal(r.status, 200);+ assert.equal((await r.json()).ok, true);+ assert.match(h.log, /media noteState/);+ const saved = JSON.parse(fs.readFileSync(path.join(h.dataDir, `state-${U1}.json`), 'utf8'));+ assert.equal(saved._rev, 1, 'the state itself was saved');+});++test('folders are 0700 and files 0600, and an admin deleting the profile removes its folder', async t => {+ const h = await start(t);+ const a = M.jpeg();+ await h.put(a);+ await h.put(M.jpeg(), { uid: U2 });+ const mode = p => fs.statSync(p).mode & 0o777;+ assert.equal(mode(h.uploads), 0o700);+ assert.equal(mode(path.join(h.uploads, U1)), 0o700);+ assert.equal(mode(path.join(h.uploads, U1, '.tmp')), 0o700);+ assert.equal(mode(path.join(h.uploads, U1, `${M.sha(a)}.jpg`)), 0o600);+ assert.equal(mode(path.join(h.uploads, U1, '.gc.json')), 0o600);++ const del = await fetch(`${h.api}/api/admin/user/delete`, { method: 'POST', headers: { ...cookie(ADMIN), 'Content-Type': 'application/json' }, body: JSON.stringify({ id: U1 }) });+ assert.equal(del.status, 200);+ assert.equal(fs.existsSync(path.join(h.uploads, U1)), false);+ assert.equal(fs.existsSync(path.join(h.uploads, U2)), true, 'nobody else is touched');+});++test('/api/config carries the caps; MEDIA_UPLOADS=0 takes the block and every route away', async t => {+ const on = await start(t, { MEDIA_QUOTA_MB: '100', MEDIA_VIDEO_MAX_SEC: '30' });+ const cfg = await (await fetch(`${on.api}/api/config`)).json();+ assert.deepEqual(cfg.media, { imageMB: 2, gifMB: 8, videoMB: 40, videoSec: 30, quotaMB: 100 });++ const off = await start(t, { MEDIA_UPLOADS: '0' });+ const cfgOff = await (await fetch(`${off.api}/api/config`)).json();+ assert.equal('media' in cfgOff, false);+ const bytes = M.jpeg();+ assert.equal((await off.put(bytes)).status, 404);+ assert.equal((await off.get(M.sha(bytes))).status, 404);+ assert.equal((await off.post('/api/media/missing', { hashes: [] })).status, 404);+ assert.equal((await off.post('/api/media/sweep', {})).status, 404);+ assert.equal(fs.existsSync(off.uploads), false);+});++test('boot removes the temp files of uploads the previous process was receiving', async t => {+ const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gym-media-boot-'));+ t.after(() => fs.rmSync(dataDir, { recursive: true, force: true }));+ const tmp = path.join(dataDir, 'uploads', U1, '.tmp');+ fs.mkdirSync(tmp, { recursive: true });+ fs.writeFileSync(path.join(tmp, 'half-an-upload'), 'x');+ fs.writeFileSync(path.join(dataDir, 'secret'), SECRET);+ const child = spawn(process.execPath, ['server.js'], { cwd: API, stdio: ['ignore', 'pipe', 'pipe'], env: { ...process.env, PORT: '0', DATA_DIR: dataDir } });+ t.after(() => child.kill('SIGKILL'));+ await boundPort(child);+ assert.deepEqual(fs.readdirSync(tmp), []);+});++test('server.js gives a slow upload half an hour, not node\'s default five minutes', () => {+ const src = fs.readFileSync(path.join(API, 'server.js'), 'utf8');+ const m = /^server\.requestTimeout\s*=\s*([\d_\s*]+);/m.exec(src);+ assert.ok(m, 'server.requestTimeout is set');+ const ms = m[1].split('*').reduce((a, x) => a * Number(x.replace(/[_\s]/g, '')), 1);+ assert.ok(ms >= 30 * 60000, `requestTimeout is ${ms} ms`);+ const hm = /^server\.headersTimeout\s*=\s*([\d_\s*]+);/m.exec(src);+ assert.ok(!hm || hm[1].split('*').reduce((a, x) => a * Number(x.replace(/[_\s]/g, '')), 1) <= 60000, 'headers still have to arrive within a minute');+});diff --git a/scripts/build-api-docs.mjs b/scripts/build-api-docs.mjsindex 6c11a2734..fdb5b6809 100644--- a/scripts/build-api-docs.mjs+++ b/scripts/build-api-docs.mjs@@ -128,6 +128,7 @@ const TAGS = {data: { title: 'Data', side: 'State sync' },push: { title: 'Push', side: 'Notifications & rest timer' },activity: { title: 'Activity', side: 'Live presence' },+ media: { title: 'Media', side: 'Photos & videos of your own exercises' },// A tag missing from this map renders nowhere at all, silently — so every tag in the// spec needs a line here.coach: { title: 'AI Coach', side: 'Plans, reviews, debriefs' },