Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(ai-research-v4): read data uri images as a browser does (#128032)

Robbieca79566b15ed
diff --git a/products/ai_training/docs/replay-data.md b/products/ai_training/docs/replay-data.md
index 2fac8b1ea1b7..59f3367ad117 100644
--- a/products/ai_training/docs/replay-data.md
+++ b/products/ai_training/docs/replay-data.md
@@ -464,13 +464,13 @@ The recorder streams each block into the encoder. Read a block as a stream: a Br
- posthog-js compresses some fields of an event and marks the event `cv: "2024-10"`. The recorder expands those fields to plain JSON and removes the `cv` key, as the replay player does, because the gzip strings do not compress again. An event that does not decode stays exactly as it arrived.
- An inline image becomes `image:v4:{team_id}:{YYYY-MM}:{hash}` in the block, where the month is the UTC month of the write.
-- The recorder takes PNG, JPEG, GIF and WebP data URIs in any letter case and with parameters, under the v3 size limits. It decodes the JSON escapes of each string before it looks for data URIs, so an escaped letter does not hide one. It reads a base64 payload across line breaks and escaped slashes, and without its padding. It reads a payload that is not base64 as percent-encoded bytes. The image bytes name the format, not the data URI's label, so the image index `mime` is always one of these four.
+- The recorder takes PNG, JPEG, GIF and WebP data URIs in any letter case and with parameters, under the v3 size limits. It decodes the JSON escapes of each string before it looks for data URIs, so an escaped letter does not hide one. It also reads the CSS escapes in the scheme, the media type and the parameters, skips the tabs and line breaks there, and reads parameters of any length, as a browser does. Only CSS reads a CSS escape as its character. In an attribute such as `src` it is text, so a data URI there whose header holds a CSS escape becomes `image:v4:declined:undecodable`, up to the end of its string. It reads a payload as base64 only when `;base64` ends the header, as a browser does. It reads a base64 payload across line breaks, escaped slashes and percent escapes, and without its padding. It reads a payload that is not base64 as percent-encoded bytes. The image bytes name the format, not the data URI's label, so the image index `mime` is always one of these four.
- An inline image that the recorder does not store becomes a marker that names the reason, and its bytes are stored nowhere. A srcset loses such a candidate. The recorder counts these as `ai_research_v4_recorder_removed_images_total{reason}`.
- `image:v4:declined:avif`: the label or the bytes name AVIF. The dataset keeps no AVIF image.
- `image:v4:declined:over_limit`: the image is larger than 900 KiB, or its message already holds 4,096 images or 64 MiB of them.
- - `image:v4:declined:undecodable`: the payload does not decode, or it holds an escape that the recorder does not read, for example a CSS escape.
+ - `image:v4:declined:undecodable`: the payload does not decode, or it holds an escape that the recorder does not read, for example a CSS escape in the payload.
- `image:v4:declined:unsupported`: the bytes are not PNG, JPEG, GIF or WebP, for example a font labeled `image/ttf`, an icon or a bitmap.
-- An SVG data URI stays inline, because SVG is text that the text scrub reads in place. A data URI with an empty payload holds no image, so it stays as it is.
+- An SVG data URI whose payload starts with markup (`<`) stays inline, because SVG is text that the text scrub reads in place. Any other payload under an SVG label follows the rules for raster images, because a browser shows raster bytes as an image whatever the label says. A data URI with an empty payload holds no image, so it stays as it is.
- The recorder reads every argument of a canvas mutation, not only the frames of the posthog-js canvas recorder. A buffer becomes a reference when its bytes are PNG, JPEG, GIF or WebP, and a marker when they are not. The pixels of an `ImageData`, and the numbers of a `Uint8Array` or a `Uint8ClampedArray`, become `image:v4:declined:unsupported`. A data URI in the `src` of an `HTMLImageElement` moves out like any data URI, and a remote URL there stays as it is.
- A remote image URL becomes `imageurl:v4:{team_id}:{YYYY-MM}:{hash}` in the block and in its fetch job, where the month is the UTC month of the write. The block and the job always carry the same reference, so a reader opens the fetched image directly.
- A URL that the URL policy refuses becomes `imageurl:v4:declined:{reason}` in the block, for example `declined:credential`, `declined:non_public_host` or `declined:tracking_beacon`. The original URL cannot be recovered. The recorder counts rewrites and refusals in `ai_research_v4_recorder_image_url_rewrites_total`.
diff --git a/rust/ai-research/replay-core-v4/src/scan/images.rs b/rust/ai-research/replay-core-v4/src/scan/images.rs
index f5d9c9987d53..bf4567a27ece 100644
--- a/rust/ai-research/replay-core-v4/src/scan/images.rs
+++ b/rust/ai-research/replay-core-v4/src/scan/images.rs
@@ -2,7 +2,8 @@
//! go to the flush's image pack. An image the image scrub cannot read, or one over a cap, becomes an [`ImageDecline`]
//! marker.
//!
-//! The scan is one pass over the colons, for `data:image/` in any case. A data URI looks the same in an attribute, a CSS `url()`, a
+//! The scan looks for `data:image/` in any case, at each colon and near each CSS escape, because a stylesheet can write
+//! any character of a data URI's header as an escape. A data URI looks the same in an attribute, a CSS `url()`, a
//! `srcset` candidate or stylesheet text, so the scan needs no knowledge of where it sits. Every other byte is
//! copied as it is. An SVG data URI stays inline, because SVG is text that the text scrub reads in place.
@@ -27,9 +28,8 @@ const FORGIVING_BASE64: GeneralPurpose = GeneralPurpose::new(
.with_decode_allow_trailing_bits(true),
);
-/// The longest media type and parameters a taken data URI has before its comma. A raster type with parameters, such
-/// as `image/png;charset=utf-8;base64`, fits. The bound keeps the scan linear.
-const MAX_META: usize = 64;
+/// The payload bytes read past whitespace and a byte order mark to find the `<` that starts SVG markup.
+const SNIFFED_PAYLOAD_BYTES: usize = 64;
#[derive(Debug, Clone, Copy)]
pub struct ImageLimits {
@@ -98,11 +98,16 @@ impl<'k> ImageSink<'k> {
}
}
+ /// Takes base64 that may hold `%` escapes. It checks the size first, so an oversized payload is never copied.
pub(crate) fn take_base64(&mut self, payload: &[u8]) -> String {
- if payload.len() > self.limits.max_image_bytes.div_ceil(3) * 4 {
+ let escapes = payload
+ .windows(3)
+ .filter(|w| w[0] == b'%' && w[1].is_ascii_hexdigit() && w[2].is_ascii_hexdigit())
+ .count();
+ if payload.len() - 2 * escapes > self.limits.max_image_bytes.div_ceil(3) * 4 {
return self.decline(ImageDecline::OverLimit);
}
- match FORGIVING_BASE64.decode(payload) {
+ match FORGIVING_BASE64.decode(percent_decoded_base64(payload)) {
Ok(bytes) => self.take(bytes),
Err(_) => self.decline(ImageDecline::Undecodable),
}
@@ -214,7 +219,7 @@ fn continues_past(bytes: &[u8], end: usize) -> bool {
Some(&byte) => {
// Wrapped base64 breaks at a line break, so a descriptor counts only after plain spaces.
return position > end
- && is_base64(byte)
+ && (is_base64(byte) || byte == b'%')
&& (line_break || !is_descriptor(bytes, position));
}
None => return false,
@@ -222,6 +227,38 @@ fn continues_past(bytes: &[u8], end: usize) -> bool {
}
}
+/// No raster format starts with `<`, and a browser shows raster bytes as an image whatever the label says, so an SVG label keeps a data URI inline only when this is true.
+fn payload_is_markup(bytes: &[u8], payload_start: usize, base64: bool) -> bool {
+ let mut chars = Vec::new();
+ let mut rest = &bytes[payload_start..];
+ while chars.len() < SNIFFED_PAYLOAD_BYTES * 3 {
+ let (byte, tail) = match rest {
+ [b'\\', b'u', b'0', b'0', b'2', b'f' | b'F', tail @ ..] | [b'\\', b'/', tail @ ..] => {
+ (b'/', tail)
+ }
+ [b'\\', b't' | b'n' | b'r' | b'f', tail @ ..] => (b' ', tail),
+ [b'"' | b'\\', ..] | [] => break,
+ [byte, tail @ ..] => (*byte, tail),
+ };
+ if !byte.is_ascii_whitespace() {
+ chars.push(byte);
+ }
+ rest = tail;
+ }
+ let mut decoded: Vec<u8> = percent_encoding::percent_decode(&chars).collect();
+ if base64 {
+ decoded.retain(|byte| !byte.is_ascii_whitespace());
+ let run = decoded
+ .iter()
+ .position(|&byte| !is_base64(byte))
+ .unwrap_or(decoded.len());
+ decoded.truncate(run / 4 * 4);
+ decoded = FORGIVING_BASE64.decode(&decoded).unwrap_or_default();
+ }
+ let text = decoded.strip_prefix(b"\xEF\xBB\xBF").unwrap_or(&decoded);
+ text.trim_ascii_start().first() == Some(&b'<')
+}
+
enum DataUri {
NotImage,
Svg,
@@ -265,10 +302,7 @@ fn wrapped_end(bytes: &[u8], mut end: usize) -> (usize, bool) {
_ => break,
}
}
- end += bytes[end..]
- .iter()
- .position(|&b| !is_base64(b))
- .unwrap_or(bytes.len() - end);
+ end = base64_run_end(bytes, end);
if end == before {
return (end, false);
}
@@ -276,6 +310,28 @@ fn wrapped_end(bytes: &[u8], mut end: usize) -> (usize, bool) {
(end, readable)
}
+/// A browser percent-decodes a data URI before it decodes base64, so a run of base64 takes `%` escapes too.
+fn base64_run_end(bytes: &[u8], from: usize) -> usize {
+ let mut at = from;
+ loop {
+ match bytes.get(at..) {
+ Some([b'%', high, low, ..]) if high.is_ascii_hexdigit() && low.is_ascii_hexdigit() => {
+ at += 3
+ }
+ Some([byte, ..]) if is_base64(*byte) => at += 1,
+ _ => return at,
+ }
+ }
+}
+
+fn percent_decoded_base64(raw: &[u8]) -> Cow<'_, [u8]> {
+ if raw.contains(&b'%') {
+ Cow::Owned(percent_encoding::percent_decode(raw).collect())
+ } else {
+ Cow::Borrowed(raw)
+ }
+}
+
fn wrapped_payload(bytes: &[u8], start: usize, end: usize) -> Vec<u8> {
let mut base64 = Vec::with_capacity(end - start);
let mut rest = &bytes[start..end];
@@ -338,66 +394,144 @@ fn percent_decoded(raw: &[u8]) -> Option<Vec<u8>> {
Some(percent_encoding::percent_decode(text.as_bytes()).collect())
}
-enum Meta {
- Read(String, usize),
- NoPayload,
- TooLong(String),
+/// Reads the header through CSS escapes and JSON escapes, and skips the tabs and line breaks that a browser removes from a URL, however they are written.
+fn header_char(bytes: &[u8], at: usize, double_quoted: bool) -> Option<(char, usize)> {
+ let mut removed = 0;
+ loop {
+ let (ch, len) = url_char(bytes, at + removed, double_quoted)?;
+ if !matches!(ch, '\t' | '\n' | '\r') {
+ return Some((ch, removed + len));
+ }
+ removed += len;
+ }
}
-/// A JSON writer may escape a `/` as `\/` or `\u002F`, so both read as `/`.
-fn read_meta(bytes: &[u8], start: usize) -> Meta {
- let mut meta = String::new();
- let mut at = start;
- let mut whitespace = false;
- while at - start <= MAX_META {
- match bytes.get(at..) {
- Some([b',', ..]) => return Meta::Read(meta, at),
- Some([b'\\', b'/', ..]) => {
- meta.push('/');
- at += 2;
- }
- Some([b'\\', b'u', b'0', b'0', b'2', b'f' | b'F', ..]) => {
- meta.push('/');
- at += 6;
- }
- Some([b'"' | b'\\', ..] | []) | None => return Meta::NoPayload,
- Some([byte, ..]) => {
- whitespace |= byte.is_ascii_whitespace();
- meta.push(char::from(byte.to_ascii_lowercase()));
- at += 1;
- }
+/// A browser keeps every character but the end of the JSON string, and in a double-quoted CSS url an escaped quote, which ends the URL there.
+fn url_char(bytes: &[u8], at: usize, double_quoted: bool) -> Option<(char, usize)> {
+ match bytes.get(at..)? {
+ [b'\\', b'\\', escaped @ ..] => css_escape(escaped).map(|(ch, len)| (ch, 2 + len)),
+ [b'\\', b'"', ..] if double_quoted => None,
+ [b'\\', escaped @ ..] => json_escape(escaped).map(|(ch, len)| (ch, 1 + len)),
+ [b'"', ..] | [] => None,
+ [byte, ..] => Some((char::from(*byte), 1)),
+ }
+}
+
+/// The length counts from after the backslash.
+fn json_escape(escaped: &[u8]) -> Option<(char, usize)> {
+ match escaped {
+ [b'u', hex @ ..] => {
+ let code = std::str::from_utf8(hex.get(..4)?)
+ .ok()
+ .and_then(|hex| u32::from_str_radix(hex, 16).ok())?;
+ Some((
+ char::from_u32(code).unwrap_or(char::REPLACEMENT_CHARACTER),
+ 5,
+ ))
}
+ [b'b', ..] => Some(('\u{8}', 1)),
+ [b'f', ..] => Some(('\u{c}', 1)),
+ [b'n', ..] => Some(('\n', 1)),
+ [b'r', ..] => Some(('\r', 1)),
+ [b't', ..] => Some(('\t', 1)),
+ [byte @ (b'"' | b'/' | b'\\'), ..] => Some((char::from(*byte), 1)),
+ _ => None,
}
- if whitespace {
- Meta::NoPayload
- } else {
- Meta::TooLong(meta)
+}
+
+/// The length counts from after the backslash, and a hex escape takes the one whitespace after it, as CSS reads it.
+fn css_escape(escaped: &[u8]) -> Option<(char, usize)> {
+ let digits = escaped
+ .iter()
+ .take(6)
+ .take_while(|b| b.is_ascii_hexdigit())
+ .count();
+ if digits == 0 {
+ return match escaped {
+ [b'\\', rest @ ..] => json_escape(rest).map(|(ch, len)| (ch, 1 + len)),
+ [b'"', ..] | [] => None,
+ [byte, ..] => Some((char::from(*byte), 1)),
+ };
}
+ let code = escaped[..digits].iter().fold(0u32, |code, &digit| {
+ code * 16 + char::from(digit).to_digit(16).unwrap_or_default()
+ });
+ let whitespace = match &escaped[digits..] {
+ [b'\\', b'r', b'\\', b'n', ..] => 4,
+ [b'\\', b'n' | b'r' | b't' | b'f', ..] => 2,
+ [b' ', ..] => 1,
+ _ => 0,
+ };
+ let ch = char::from_u32(code).unwrap_or(char::REPLACEMENT_CHARACTER);
+ Some((ch, digits + whitespace))
}
-/// Reads the data URI whose `data:` starts at `start`, with the rules of v3 `collectable_data_uri_bytes`: the media
-/// type and `base64` in any case, and any parameters before the comma.
-fn read_data_uri(bytes: &[u8], start: usize, sink: &mut ImageSink<'_>) -> DataUri {
- let meta_start = start + b"data:".len();
- let (meta, comma) = match read_meta(bytes, meta_start) {
- Meta::Read(meta, comma) if meta.starts_with("image/") => (meta, comma),
- // A browser reads parameters of any length, so the URI can hold an image whose payload the scan cannot find.
- Meta::TooLong(meta) if meta.starts_with("image/") => {
- let end = uri_end(bytes, start, meta_start);
- return DataUri::Replaced {
- replacement: sink.decline(ImageDecline::Undecodable),
- end,
+/// A browser removes tabs and line breaks inside the scheme too, but the first letter is read as it is, so a read that starts at an escaped tab stops there and the search stays linear.
+fn after_data_scheme(bytes: &[u8], start: usize) -> Option<usize> {
+ let (first, len) = url_char(bytes, start, true)?;
+ if !first.eq_ignore_ascii_case(&'d') {
+ return None;
+ }
+ let mut at = start + len;
+ for expected in "ata:".chars() {
+ let (ch, len) = header_char(bytes, at, true)?;
+ if ch.to_ascii_lowercase() != expected {
+ return None;
+ }
+ at += len;
+ }
+ Some(at)
+}
+
+enum Header {
+ Image { meta: String, payload_start: usize },
+ Other { read_to: usize },
+}
+
+/// A browser reads parameters of any length, so the header has no length limit.
+fn read_header(bytes: &[u8], meta_start: usize, double_quoted: bool) -> Header {
+ const IMAGE: &[u8] = b"image/";
+ let mut meta = String::new();
+ let mut at = meta_start;
+ while let Some((ch, len)) = header_char(bytes, at, double_quoted) {
+ let ch = ch.to_ascii_lowercase();
+ if meta.len() < IMAGE.len() && Some(ch) != IMAGE.get(meta.len()).copied().map(char::from) {
+ return Header::Other { read_to: at };
+ }
+ if ch == ',' {
+ return Header::Image {
+ meta,
+ payload_start: at + len,
};
}
- _ => return DataUri::NotImage,
- };
+ meta.push(ch);
+ at += len;
+ }
+ Header::Other { read_to: at }
+}
+
+/// Reads the data URI whose `data:` starts at `start`, with the rules of v3 `collectable_data_uri_bytes`: the media
+/// type and `base64` in any case, and any parameters before the comma. Unlike v3, `base64` counts only at the end.
+fn read_data_uri(
+ bytes: &[u8],
+ start: usize,
+ meta: &str,
+ payload_start: usize,
+ sink: &mut ImageSink<'_>,
+) -> DataUri {
+ if holds_css_escape(&bytes[start..payload_start]) && css_opener(bytes, start).is_none() {
+ return DataUri::Replaced {
+ replacement: sink.decline(ImageDecline::Undecodable),
+ end: json_string_end(bytes, payload_start),
+ };
+ }
let media_type = meta.split(';').next().unwrap_or_default();
let subtype = &media_type["image/".len()..];
- if subtype.starts_with("svg") {
+ let base64 = ends_with_base64(meta);
+ if subtype.starts_with("svg") && payload_is_markup(bytes, payload_start, base64) {
return DataUri::Svg;
}
- let payload_start = comma + 1;
- if !meta.contains("base64") {
+ if !base64 {
let end = uri_end(bytes, start, payload_start);
let replacement = match &bytes[payload_start..end] {
[] => return DataUri::NotImage,
@@ -409,10 +543,7 @@ fn read_data_uri(bytes: &[u8], start: usize, sink: &mut ImageSink<'_>) -> DataUr
};
return DataUri::Replaced { replacement, end };
}
- let run_end = bytes[payload_start..]
- .iter()
- .position(|&b| !is_base64(b))
- .map_or(bytes.len(), |length| payload_start + length);
+ let run_end = base64_run_end(bytes, payload_start);
let (end, readable) = if continues_past(bytes, run_end) {
wrapped_end(bytes, run_end)
} else {
@@ -434,25 +565,152 @@ fn read_data_uri(bytes: &[u8], start: usize, sink: &mut ImageSink<'_>) -> DataUr
DataUri::Replaced { replacement, end }
}
+/// A browser decodes the payload as base64 only when `;base64` ends the header.
+fn ends_with_base64(meta: &str) -> bool {
+ meta.trim_ascii()
+ .strip_suffix("base64")
+ .is_some_and(|rest| rest.trim_end_matches(' ').ends_with(';'))
+}
+
+fn holds_css_escape(bytes: &[u8]) -> bool {
+ bytes.windows(2).any(|pair| pair == br"\\")
+}
+
+/// CSS reads an escape as its character, but an attribute such as `src` holds it as text, so outside CSS a browser can
+/// read the header another way, for example past an escaped comma to a later one. Whitespace before the URI, in any
+/// spelling, does not change where it sits.
+fn css_opener(bytes: &[u8], start: usize) -> Option<u8> {
+ let mut start = start;
+ loop {
+ start = match &bytes[..start] {
+ [.., b' '] => start - 1,
+ [.., b'\\', b't' | b'n' | b'r' | b'f'] => start - 2,
+ _ => match css_hex_escape_before(bytes, start) {
+ Some((escape_start, 0x9 | 0xa | 0xc | 0xd | 0x20)) => escape_start,
+ _ => break,
+ },
+ };
+ }
+ let opener = start.checked_sub(1).map(|at| bytes[at])?;
+ let escaped_quote = || {
+ bytes[..start - 1]
+ .iter()
+ .rev()
+ .take_while(|&&b| b == b'\\')
+ .count()
+ % 2
+ == 1
+ };
+ match opener {
+ b'(' | b'\'' => Some(opener),
+ b'"' if escaped_quote() => Some(opener),
+ _ => None,
+ }
+}
+
+fn json_string_end(bytes: &[u8], from: usize) -> usize {
+ let mut at = from;
+ while let Some(&byte) = bytes.get(at) {
+ match byte {
+ b'"' => return at,
+ b'\\' => at += 2,
+ _ => at += 1,
+ }
+ }
+ bytes.len()
+}
+
+/// A six-digit CSS escape such as `\000020` ends at its last digit, so the letter after it starts a word.
+fn starts_inside_a_word(bytes: &[u8], start: usize) -> bool {
+ start
+ .checked_sub(1)
+ .is_some_and(|before| bytes[before].is_ascii_alphanumeric())
+ && css_hex_escape_before(bytes, start)
+ .is_none_or(|(escape_start, _)| start - escape_start != 8)
+}
+
+/// The CSS hex escape that ends right before `at`, as where its doubled backslash starts and the code it names.
+fn css_hex_escape_before(bytes: &[u8], at: usize) -> Option<(usize, u32)> {
+ let digits = bytes[..at]
+ .iter()
+ .rev()
+ .take(6)
+ .take_while(|b| b.is_ascii_hexdigit())
+ .count();
+ let escape_start = at.checked_sub(digits + 2)?;
+ if digits == 0 || bytes[escape_start..escape_start + 2] != *br"\\" {
+ return None;
+ }
+ let code = std::str::from_utf8(&bytes[at - digits..at])
+ .ok()
+ .and_then(|hex| u32::from_str_radix(hex, 16).ok())?;
+ Some((escape_start, code))
+}
+
+/// Each `data:` that an escape or a removed tab or line break spells, as where it starts and where its media type
+/// starts, in order. Before its first backslash such a scheme holds only literal letters of `data`, so it starts at
+/// most four letters before a backslash, or at a CSS escape.
+fn escaped_schemes(bytes: &[u8]) -> impl Iterator<Item = (usize, usize)> + '_ {
+ memchr::memchr_iter(b'\\', bytes).flat_map(move |backslash| {
+ let literal_starts = (1..=b"data".len())
+ .rev()
+ .filter_map(move |letters| backslash.checked_sub(letters))
+ .filter(move |&start| {
+ bytes[start..backslash].eq_ignore_ascii_case(&b"data"[..backslash - start])
+ });
+ let escape_start = (bytes.get(backslash + 1) == Some(&b'\\')).then_some(backslash);
+ literal_starts
+ .chain(escape_start)
+ .filter(move |&start| !starts_inside_a_word(bytes, start))
+ .filter_map(move |start| {
+ after_data_scheme(bytes, start).map(|meta_start| (start, meta_start))
+ })
+ })
+}
+
+fn data_schemes(bytes: &[u8]) -> impl Iterator<Item = (usize, usize)> + '_ {
+ let mut plain = memchr::memchr_iter(b':', bytes)
+ .filter_map(|colon| {
+ let start = colon.checked_sub(b"data".len())?;
+ (bytes[start..colon].eq_ignore_ascii_case(b"data")
+ && !starts_inside_a_word(bytes, start))
+ .then_some((start, colon + 1))
+ })
+ .peekable();
+ let mut escaped = escaped_schemes(bytes).peekable();
+ std::iter::from_fn(move || match (plain.peek(), escaped.peek()) {
+ (Some(plain_scheme), Some(escaped_scheme)) if escaped_scheme < plain_scheme => {
+ escaped.next()
+ }
+ (Some(_), _) => plain.next(),
+ (None, _) => escaped.next(),
+ })
+}
+
/// Replaces every inline image in `bytes` but SVG with its reference or a marker. Returns the input unchanged when it holds none.
-/// The scan matches `data:image/` byte for byte, so a JSON escape of one of its letters hides the image. Give it bytes
-/// that [`canonical_strings`](crate::bytescan::canonical_strings) wrote.
+/// A JSON escape of a letter hides an image from the scan, so give it bytes that [`canonical_strings`](crate::bytescan::canonical_strings) wrote.
pub fn extract_images<'b>(bytes: &'b [u8], sink: &mut ImageSink<'_>) -> Cow<'b, [u8]> {
let mut out: Option<Vec<u8>> = None;
let mut copied_to = 0;
- for colon in memchr::memchr_iter(b':', bytes) {
- let Some(start) = colon.checked_sub(b"data".len()) else {
- continue;
- };
- if start < copied_to
- || !bytes[start..colon].eq_ignore_ascii_case(b"data")
- || start
- .checked_sub(1)
- .is_some_and(|before| bytes[before].is_ascii_alphanumeric())
- {
+ // A header can run to the end of its JSON string, so the scan reads each header once and takes a `data:` inside it as part of that URI, to stay linear. That `data:` ends at the same comma and the same `;base64`, so it reads the payload the same way.
+ let mut headers_read_to = 0;
+ for (start, meta_start) in data_schemes(bytes) {
+ if start < copied_to.max(headers_read_to) {
continue;
}
- match read_data_uri(bytes, start, sink) {
+ let (meta, payload_start) =
+ match read_header(bytes, meta_start, css_opener(bytes, start) == Some(b'"')) {
+ Header::Image {
+ meta,
+ payload_start,
+ } => (meta, payload_start),
+ Header::Other { read_to } => {
+ headers_read_to = read_to;
+ continue;
+ }
+ };
+ headers_read_to = payload_start;
+ match read_data_uri(bytes, start, &meta, payload_start, sink) {
DataUri::NotImage => {}
DataUri::Svg => sink.left_inline += 1,
DataUri::Replaced { replacement, end } => {
@@ -544,7 +802,7 @@ mod tests {
}
#[test]
- fn takes_wrapped_escaped_unpadded_and_percent_encoded_payloads() {
+ fn takes_escaped_wrapped_unpadded_percent_encoded_and_long_data_uris() {
let image = png(&[0xfb; 62]);
let base64 = STANDARD.encode(&image);
assert!(base64.contains('/') && base64.ends_with('='));
@@ -585,6 +843,14 @@ mod tests {
format!("data:image/png;base64,{}", base64.trim_end_matches('=')),
r#""}"#,
),
+ (
+ r#"{"src":""#,
+ format!(
+ "data:image/png;base64,%69{}",
+ base64[1..].replace('/', "%2F").replace('=', "%3D")
+ ),
+ r#""}"#,
+ ),
(r#"{"src":""#, format!("data:image/png,{percent}"), r#""}"#),
(
r#"{"style":"background:url("#,
@@ -601,6 +867,89 @@ mod tests {
format!("data:image/png,{percent}"),
r#" 2x, https://example.com/a.png 1x"}"#,
),
+ // CSS escapes in the header, with each backslash doubled by JSON.
+ (
+ r#"{"style":"background:url("#,
+ format!(r"d\\61ta:image/png;base64,{base64}"),
+ r#")"}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url(\""#,
+ format!(r"data\\3a image\\2F png;base64,{base64}"),
+ r#"\")}"}"#,
+ ),
+ (
+ r#"{"style":"background:url("#,
+ format!(r"\\000044 ata\\:image/png;base64,{base64}"),
+ r#")"}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url(\"\\000020"#,
+ format!(r"d\\61ta:image/png;base64,{base64}"),
+ r#"\")}"}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url(\""#,
+ format!(r"d\\9 ata:image/png;base64,{base64}"),
+ r#"\")}"}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url(\""#,
+ format!(r"d{}\\61ta:image/png;base64,{base64}", r"\t".repeat(25)),
+ r#"\")}"}"#,
+ ),
+ (
+ r#"{"style":"background:url("#,
+ format!(r"data:image/png\\3b base64\\2c {base64}"),
+ r#")"}"#,
+ ),
+ (
+ r#"{"src":""#,
+ format!("data:image/png;name=a photo of the harbour at dusk before the rain came in.png;base64,{base64}"),
+ r#""}"#,
+ ),
+ (
+ r#"{"src":""#,
+ format!("data:image/png;name={};base64,{base64}", "a".repeat(80)),
+ r#""}"#,
+ ),
+ (
+ r#"{"src":""#,
+ format!(r"data:image/png;\tbase64,{base64}"),
+ r#""}"#,
+ ),
+ (
+ r#"{"src":""#,
+ format!(r"data:image/png;name=a\fb;base64,{base64}"),
+ r#""}"#,
+ ),
+ (
+ r#"{"src":""#,
+ format!(r#"data:image/png;name=a\"b;base64,{base64}"#),
+ r#""}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url('"#,
+ format!(r#"data:image/png;name=a\"b;base64,{base64}"#),
+ r#"')}"}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url(\""#,
+ format!(r"data:i\\9mage/png;base64,{base64}"),
+ r#"\")}"}"#,
+ ),
+ // Raster bytes under an SVG label, which a browser shows as an image.
+ (r#"{"src":""#, format!("data:image/svg+xml,{percent}"), r#""}"#),
+ (
+ r#"{"src":""#,
+ format!("data:image/svg+xml;base64, {base64}"),
+ r#""}"#,
+ ),
+ (
+ r#"{"textContent":".a{background:url("#,
+ format!("data:image/svg+xml)}}.b{{background:url(data:image/png;base64,{base64}"),
+ r#")}"}"#,
+ ),
];
for (before, uri, after) in cases {
let event = format!("{before}{uri}{after}");
@@ -614,6 +963,16 @@ mod tests {
"{event}"
);
}
+ let event = format!(
+ r#"{{"textContent":"/*data:image/png;base64*/.a{{background:url(data:image/png,{percent})}}"}}"#
+ );
+ let mut images = sink(ImageLimits::default());
+ let out = extract_images(event.as_bytes(), &mut images);
+ assert_eq!(images.images.len(), 1, "{event}");
+ assert!(
+ !String::from_utf8_lossy(&out).contains(&percent[..12]),
+ "{event}"
+ );
}
#[test]
@@ -662,15 +1021,6 @@ mod tests {
limits,
ImageDecline::Undecodable,
),
- (
- format!(
- "data:image/png;name={};base64,{}",
- "a".repeat(80),
- STANDARD.encode(png(b"x"))
- ),
- limits,
- ImageDecline::Undecodable,
- ),
(
data_uri("image/ttf", b"\x00\x01\x00\x00font"),
limits,
@@ -696,6 +1046,43 @@ mod tests {
limits,
ImageDecline::Unsupported,
),
+ (
+ format!(
+ r"data:image/svg+xml;base64,\\69{}",
+ &STANDARD.encode(png(&[7; 52]))[1..]
+ ),
+ limits,
+ ImageDecline::Undecodable,
+ ),
+ (
+ format!(
+ r"data:image/png;name=base64;x=\\00002cQQ;base64,{}",
+ STANDARD.encode(png(&[7; 52]))
+ ),
+ limits,
+ ImageDecline::Undecodable,
+ ),
+ // A browser reads a CSS escape in an attribute as text, so the label and `base64` are not what CSS reads.
+ (
+ format!(
+ r"data:image/sv\\67+xml;base64,{}",
+ STANDARD.encode(png(&[7; 52]))
+ ),
+ limits,
+ ImageDecline::Undecodable,
+ ),
+ (
+ format!(
+ r"data:image/png;name={};\\000062ase64,{}",
+ "a".repeat(80),
+ png(&[7; 52])
+ .iter()
+ .map(|b| format!("%{b:02X}"))
+ .collect::<String>()
+ ),
+ limits,
+ ImageDecline::Undecodable,
+ ),
];
for (uri, limits, reason) in cases {
let event = format!(r#"{{"src":"{uri}","alt":"x"}}"#);
@@ -719,6 +1106,16 @@ mod tests {
for event in [
format!(r#"{{"src":"{}"}}"#, data_uri("image/svg+xml", b"<svg/>")),
r#"{"src":"data:image/svg+xml,%3Csvg%2F%3E"}"#.to_owned(),
+ format!(
+ r#"{{"src":"data:image/svg+xml;base64, {}"}}"#,
+ STANDARD.encode(b"<svg/>")
+ ),
+ r#"{"src":"data:image/svg+xml;base64,P%48N2Zy8+"}"#.to_owned(),
+ format!(
+ r#"{{"src":"data:image/svg+xml;base64,{}{}"}}"#,
+ " ".repeat(200),
+ STANDARD.encode(b"<svg/>")
+ ),
r#"{"src":"data:image/gif;base64,"}"#.to_owned(),
// A URI without a comma before its string ends: the next comma separates JSON fields.
r#"{"src":"data:image/png;base64","alt":"x"}"#.to_owned(),
@@ -843,10 +1240,16 @@ mod tests {
));
let unbroken = format!("{};", "data:image/".repeat(200_000));
let mut images = sink(ImageLimits::default());
- assert_eq!(
- extract_images(unbroken.as_bytes(), &mut images).as_ref(),
- ImageDecline::Undecodable.marker().as_bytes()
- );
+ assert!(matches!(
+ extract_images(unbroken.as_bytes(), &mut images),
+ Cow::Borrowed(_)
+ ));
+ let escaped_tabs = format!("d{}x\"", r"\\9 ".repeat(200_000));
+ let mut images = sink(ImageLimits::default());
+ assert!(matches!(
+ extract_images(escaped_tabs.as_bytes(), &mut images),
+ Cow::Borrowed(_)
+ ));
assert!(started.elapsed() < std::time::Duration::from_secs(5));
}
}