Skip to content

Open-source gym & body-weight tracker, now in the App Store (iPhone + Apple Watch) and as an Android APK. Plan routines, log workouts, see which muscles are trained, import from Strong/Hevy/FitNotes. Self-host it or use it on the phone alone.

Media: the MP4 scrub blanks uuid boxes inside moov and a fragment's moof/traf, and refuses a fragmented file with a metadata track

Duarte Santos034be9819e0c
diff --git a/docs/SELF_HOSTING.md b/docs/SELF_HOSTING.md
index cf92f9662..a008c9989 100644
--- a/docs/SELF_HOSTING.md
+++ b/docs/SELF_HOSTING.md
@@ -384,7 +384,8 @@ folder whose profile is not in `db.json` is left alone and logged once. A device
file the server removed uploads it again.
**Privacy.** The app re-encodes photos on the device, so no EXIF or GPS data survives, and blanks
-the metadata boxes and the GPS/telemetry tracks of MP4 and MOV videos. WebM videos are uploaded as
+the metadata boxes and the GPS/telemetry tracks of MP4 and MOV videos (a fragmented MP4 that
+carries such a track is refused rather than uploaded with it). WebM videos are uploaded as
recorded. The server never decodes or changes a file. Like everything else in `./data`, the files
are not encrypted at rest — whoever can read that folder can see them.
diff --git a/frontend/src/lib/media-ingest.js b/frontend/src/lib/media-ingest.js
index 3de826803..16ac0688d 100644
--- a/frontend/src/lib/media-ingest.js
+++ b/frontend/src/lib/media-ingest.js
@@ -11,8 +11,9 @@
* GIF of one frame is a photo and goes down that path; an animation is held to the photo's
* pixel guard on its logical screen.
* - An MP4/MOV keeps its video and sound; its metadata boxes and the samples of any track that is
- * neither (timed metadata, GPS telemetry) are zeroed in place (media-sniff scrubMp4). A WebM is
- * kept as it is — its scrub is deferred, and the docs say so.
+ * neither (timed metadata, GPS telemetry) are zeroed in place (media-sniff scrubMp4), and a
+ * fragmented one with such a track is refused. A WebM is kept as it is — its scrub is
+ * deferred, and the docs say so.
* Every video is then opened in a hidden <video> once (probeVideo) for its size, its length when
* the file does not say, and a poster frame. A video this browser cannot play is still accepted
* — another device may play it — with a warning and no poster.
diff --git a/frontend/src/lib/media-sniff.js b/frontend/src/lib/media-sniff.js
index a0e807572..e3556afe8 100644
--- a/frontend/src/lib/media-sniff.js
+++ b/frontend/src/lib/media-sniff.js
@@ -459,15 +459,23 @@ function sampleRanges(b, stbl) {
return out
}
+// Boxes that hold what a camera or an editor says about the recording rather than the recording:
+// QuickTime's user data, ISO metadata, and vendor boxes (a Canon keeps its EXIF, GPS included, in
+// a uuid box inside moov; XMP travels in one too).
+const METADATA_BOXES = new Set(['udta', 'meta', 'uuid'])
+
/**
* Zeroes, IN PLACE, what in an MP4/MOV can say where and by whom it was filmed, and returns the
- * same array. (a) Every udta and meta box under moov, and every top-level meta and uuid box,
- * becomes a 'free' box of the same size with a zeroed body: QuickTime's location and the
- * phone's make and model live there. (b) Every sample of a track that is neither picture nor
- * sound (timed metadata, an action camera's GPS and sensor tracks) is zeroed where it sits in
- * mdat; the track itself stays. No size changes, so stco/co64 stay right and the video and
- * sound bytes are untouched. Throws on a file that does not parse or whose sample tables point
- * outside it — such a file is refused rather than kept with its metadata.
+ * same array. (a) Every udta, meta and uuid box — at the top level, anywhere under moov, and
+ * directly under a fragment's moof and traf — becomes a 'free' box of the same size with a zeroed
+ * body: QuickTime's location, the phone's make and model and a camera's EXIF live there.
+ * (b) Every sample of a track that is neither picture nor sound (timed metadata, an action
+ * camera's GPS and sensor tracks, a subtitle track) is zeroed where it sits in mdat; the track
+ * itself stays. No size changes, so stco/co64 stay right and the video and sound bytes are
+ * untouched. Throws on a file that does not parse or whose sample tables point outside it — such
+ * a file is refused rather than kept with its metadata — and on a fragmented file (moof) that has
+ * such a track: its samples sit in fragments the sample table does not list, and the scrub does
+ * not walk trun, so it refuses rather than keep them.
*/
export function scrubMp4(bytes) {
const b = bytes
@@ -477,13 +485,32 @@ export function scrubMp4(bytes) {
b[box.start + 4] = 0x66; b[box.start + 5] = 0x72; b[box.start + 6] = 0x65; b[box.start + 7] = 0x65 // 'free'
b.fill(0, box.body, box.end)
}
+ // The direct children of a box, for moof and traf (not containers the inspector descends into).
+ const kids = box => {
+ const out = []
+ for (let pos = box.body; pos < box.end;) {
+ const c = boxAt(b, pos, box.end)
+ if (!c) throw new Error('mp4: box overruns its parent')
+ out.push(c)
+ pos = c.end
+ }
+ return out
+ }
const traks = []
const counter = { n: 0 }
+ const fragmented = top.some(x => x.type === 'moof')
for (const box of top) {
- if (box.type === 'meta' || box.type === 'uuid') { blank(box); continue }
+ if (METADATA_BOXES.has(box.type)) { blank(box); continue }
+ if (box.type === 'moof') {
+ for (const c of kids(box)) {
+ if (METADATA_BOXES.has(c.type)) blank(c)
+ else if (c.type === 'traf') for (const g of kids(c)) if (METADATA_BOXES.has(g.type)) blank(g)
+ }
+ continue
+ }
if (box.type !== 'moov') continue
walkBoxes(b, box.body, box.end, c => {
- if (c.type === 'udta' || c.type === 'meta') { blank(c); return false }
+ if (METADATA_BOXES.has(c.type)) { blank(c); return false }
if (c.type === 'trak') traks.push(c)
return true
}, counter, 1, box)
@@ -493,7 +520,9 @@ export function scrubMp4(bytes) {
const doomed = []
for (const trak of traks) {
const t = readTrak(b, trak)
- if (KEEP_HANDLERS.has(t.handler) || !t.stbl) continue
+ if (KEEP_HANDLERS.has(t.handler)) continue
+ if (fragmented) throw new Error('mp4: fragmented file with a track that is neither picture nor sound')
+ if (!t.stbl) continue
doomed.push(...sampleRanges(b, t.stbl))
}
for (const [off, size] of doomed) b.fill(0, off, off + size)
diff --git a/frontend/src/lib/media-sniff.test.js b/frontend/src/lib/media-sniff.test.js
index 186528e00..72df27686 100644
--- a/frontend/src/lib/media-sniff.test.js
+++ b/frontend/src/lib/media-sniff.test.js
@@ -139,6 +139,9 @@ describe('inspectMp4', () => {
})
})
+// A box shaped like a full box (version, flags, one field), enough for a walker to step over.
+const fullBoxLike = type => box(type, fill(4, 0), fill(4, 0))
+
describe('scrubMp4', () => {
const GPS = latin1('+47.3769+008.5417/ GPS FIX')
const build = () => mp4({
@@ -184,6 +187,28 @@ describe('scrubMp4', () => {
expect(file).toEqual(before)
})
+ it('zeroes a uuid box inside moov or a trak (a Canon keeps its EXIF and GPS there)', () => {
+ const { file } = mp4({
+ tracks: [{ handler: 'vide', fourcc: 'avc1', w: 640, h: 360, payload: fill(64, 0x77), extra: [box('uuid', fill(16, 0xab), latin1('CNTH GPSLatitude 48.8577'))] }],
+ moovExtra: [box('uuid', fill(16, 0x85), latin1('CMT4 GPSLatitude 48.8577'))]
+ })
+ const out = scrubMp4(file)
+ expect(has(out, 'GPSLatitude')).toBe(false)
+ expect(has(out, 'uuid')).toBe(false)
+ expect(inspectMp4(out)).toMatchObject({ width: 640, height: 360, codec: 'avc1', handlers: ['vide'] })
+ })
+
+ it('a fragmented file: zeroes the uuid and meta boxes of its fragments, and refuses one with a track that is neither picture nor sound', () => {
+ const frag = (...extra) => [box('moof', fullBoxLike('mfhd'), box('traf', fullBoxLike('tfhd'), ...extra)), box('mdat', fill(16, 0x77))]
+ const av = mp4({ top: frag(box('uuid', fill(16, 0x6d), latin1('SECRETGPS'))) }).file
+ expect(has(scrubMp4(av), 'SECRETGPS')).toBe(false)
+ const withText = mp4({
+ tracks: [{ handler: 'vide', fourcc: 'avc1', w: 640, h: 360, payload: fill(64, 0x77) }, { handler: 'text', fourcc: 'tx3g', payload: bytes() }],
+ top: [box('moof', fullBoxLike('mfhd'), box('traf', fullBoxLike('tfhd'))), box('mdat', latin1('SECRETGPS'))]
+ }).file
+ expect(() => scrubMp4(withText)).toThrow(/fragmented/)
+ })
+
it('refuses what is not an MP4', () => {
expect(() => scrubMp4(png())).toThrow()
})