Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(ai-research-v4): name the css image urls an offline render needs (#126386)

Robbiefeadc25ebcde
diff --git a/products/ai_training/docs/replay-data.md b/products/ai_training/docs/replay-data.md
index 93a477ddb78f..8ee212f9e1c3 100644
--- a/products/ai_training/docs/replay-data.md
+++ b/products/ai_training/docs/replay-data.md
@@ -465,10 +465,13 @@ The recorder streams each block into the encoder. Read a block as a stream: a Br
- The dataset keeps no AVIF image. A base64 data URI that its label or its bytes name as AVIF becomes `image:v4:declined:avif`, and a srcset loses such a candidate. The recorder counts these as `ai_research_v4_recorder_removed_images_total{format="avif"}`. A payload broken by an escape the scan does not read stays inline. It also takes the image frames of the posthog-js canvas recorder. Anything else stays inline in the sealed block, for example SVG or line-wrapped base64.
- A remote image URL becomes `imageurl:v4:{team_id}:{YYYY-MM}:{hash}` in the block and in its fetch job, where the month is the UTC month of the write. The block and the job always carry the same reference, so a reader opens the fetched image directly.
- A URL that the URL policy refuses becomes `imageurl:v4:declined:{reason}` in the block, for example `declined:credential`, `declined:non_public_host` or `declined:tracking_beacon`. The original URL cannot be recovered. The recorder counts rewrites and refusals in `ai_research_v4_recorder_image_url_rewrites_total`.
-- The rewrite covers every remote URL that the recorder reads from an image slot: image sources, `srcset`, SVG image links, video posters, `<picture>` sources and CSS `url(...)` in style attributes, style sheets and style rules. It also covers `src`, `srcset`, `href` and `poster` in an attribute change on a node the recorder does not know, for example after a restart. `srcset` keeps its descriptors and CSS keeps its `url(...)` syntax. It is not limited by the 512 URLs per message that get fetch jobs.
+- The rewrite covers every remote URL that the recorder reads from an image slot: image sources, `srcset`, SVG image links, video posters, `<picture>` sources and CSS `url(...)` in style attributes, style sheets and style rules, in custom properties too, because a page passes a custom property to an image property through `var()`. It also covers `src`, `srcset`, `href` and `poster` in an attribute change on a node the recorder does not know, for example after a restart. `srcset` keeps its descriptors and CSS keeps its `url(...)` syntax. A URL is also replaced where a selector holds it with CSSOM's escapes, by the reference with the same escapes, so a class such as `bg-[url(https://cdn.example.com/a.png)]` and its rule `.bg-\[url\(https\:\/\/cdn\.example\.com\/a\.png\)\]` still match within a block. A reader that replaces references by text must replace the escaped form as well. It is not limited by the 512 URLs per message that get fetch jobs.
- A srcset in an image slot keeps only the candidates the recorder stores. A candidate the URL policy refuses leaves the list instead of becoming a `declined` marker, and the rest keep their descriptors. A `<picture>` source whose `type` names a media type the image fetcher does not store, such as `image/avif`, gets no fetch job and loses its remote candidates, unless a source of a stored type names the same URL. Its inline images, such as SVG data URIs, stay. The writer rebuilds each srcset that the inspector read in an image slot, in the bytes it stores, after the inline images left and the escapes were written again. It separates the candidates with a comma and a space. A `srcset` key elsewhere, for example in a custom event's payload, stays as it is. A browser skips a source with no candidates, so the replay falls back to the next source or the `img`. The recorder counts these as `ai_research_v4_recorder_srcset_rewrites_total{outcome="trimmed"|"emptied"}`.
+- The writer rewrites CSS text before it replaces the URLs, so that an offline render loads only images the block refers to. An `image-set()` keeps one candidate, because a renderer picks the candidate that fits its pixel ratio and does not fall back when that candidate fails. It keeps the highest resolution of a type the image fetcher stores among the candidates that can load: inline, accepted by the URL policy, or root-relative on a known page. Only that candidate gets a fetch job, and the other remote candidates get references without one, in case the writer does not find the set's text. A set that holds an inline image keeps all its candidates, because the writer moves the inline image out first.
+- A root-relative CSS `url(/…)` resolves against the origin of its window's latest page URL, from a Meta event or a custom event that names a page, such as `$url_changed`, with `https` in place of the page's scheme. It gets the reference of the absolute URL, or a `declined` marker. A path-relative `url(…)` stays as the page wrote it, so that no part of a page path, which can hold a token, reaches a fetch. rrweb writes snapshot CSS with absolute URLs, but sends rule text, adopted style sheets and style declarations as the page wrote them. A text that resolves two ways in one block, as in windows on two origins, or that one window resolves and another cannot, keeps no rewrite. An event resolves at most 1,024 relative URLs, a flush keeps at most 64 MiB of resolved URLs, and a page URL longer than 2,048 characters resolves none.
+- The writer rewrites only whole `url()` and `image-set()` functions, found as the inspector finds them, inside JSON strings that hold CSS: the values of `_cssText`, `style` (style diffs included), `textContent`, `rule`, `replace`, `replaceSync` and `value`. So a class name such as `bg-[url(/a.png)]`, a quoted string or a comment keeps its text, and the class's rule still matches. A text node that shows the same function text gets the rewrite too. The recorder counts these as `ai_research_v4_recorder_css_rewrites_total{kind="image_set"|"relative_url", outcome="rewritten"|"unfound"}`.
- The recorder reads a protocol-relative URL, such as `//cdn.example.com/a.png`, as `https`. It gets the same reference or `declined` marker as the `https` form, and the rewrite replaces the text as the block holds it. A browser reads a backslash as a slash there, so `\\cdn.example.com/a.png` counts too, and so does a URL such as `https:\\cdn.example.com/a.png` with any mix of slashes and backslashes after the scheme, or none: on a page of another scheme the browser loads `https:cdn.example.com/a.png` from that host.
-- Some URLs stay as they arrived: relative URLs, remote URLs outside image slots (links, frames, text, CSS custom properties and other CSS properties), the `image-set()` candidates the recorder does not choose, URLs inside a compressed value that did not decode or that the decode budget skipped, and URL text written in another form than the recorder read it, for example with a CSS escape. The recorder counts the last kind as `outcome="unrewritten"`.
+- Some URLs stay as they arrived: relative URLs outside CSS, path-relative CSS URLs, root-relative CSS URLs that resolve two ways in a block, go past a budget, or come before this process saw their window's page URL (after a restart or an eviction), the candidates of an `image-set()` that does not parse, remote URLs outside image slots (links, frames, text and CSS properties that do not show an image), URLs inside a compressed value that did not decode or that the decode budget skipped, and URL text written in another form than the recorder read it, for example with a CSS hex escape. The recorder counts the last kind as `outcome="unrewritten"`.
- Before the rewrite, the recorder writes again any JSON string that holds an escape capture would not write, for example `\/`, so an escaped URL is found too. A scheme matches in any letter case.
- If a refused URL is still in a block after the rewrite, the recorder writes the block again instead of dropping it. The second pass replaces the refused URL wherever its bytes are, also inside a longer URL. For a refused URL the stored bytes do not hold as the recorder read it, for example behind a CSS escape, it replaces every URL token that holds the URL's host with `imageurl:v4:declined:unchecked`. The recorder counts this as `url_fallbacks_total{reason="declined_kept"|"declined_unfound"}`. It drops the block, as `dropped_blocks_total{outcome="declined_unrewritten"}`, only when the second pass still leaves a refused URL, so no refused URL is stored.
- The recorder reads object keys as the player does: it writes an escaped key, such as `"\u0073rc"`, in its plain form before it inspects the event, and it takes the last copy of a repeated key.
@@ -645,7 +648,7 @@ Some limits apply:
### Remote images
The recorder sends each remote image URL that the URL policy accepts to the `ai_research_v4_image_fetch` topic in the v3 job shape, keyed by registrable domain. The job carries the reference that replaces the URL in the block.
-It collects URLs from image sources, `srcset`, SVG image links, CSS `url(...)` values and attribute mutations, and skips hidden or 1×1 images as v3 does.
+It collects URLs from image sources, `srcset`, SVG image links, CSS `url(...)` values and attribute mutations, and skips hidden or 1×1 images as v3 does. Unlike v3, it also collects CSS custom properties and relative CSS URLs, and fetches one `image-set()` candidate of a stored type.
The topics hold jobs for 24 hours, because jobs carry raw URLs.
The v4 image fetcher is its own copy of the v3 fetcher under `ai-research/image-fetch-v4/`, with the v3 fetch policy, its own consumer groups, its own `AI_RESEARCH_V4_IMAGE_FETCH_*` settings and a share of the v3 per-host limits. It shares no code or table with v3.
It applies the recorder's URL policy through `@posthog/ai-research-v4-native`, an addon over `replay_core_v4::url_policy`, so it canonicalizes a URL, groups it by registrable domain and refuses private hosts exactly as the recorder does.
diff --git a/rust/ai-research/replay-core-v4/src/inspect/css.rs b/rust/ai-research/replay-core-v4/src/inspect/css.rs
index c0e07444d5c2..7ebd48d81532 100644
--- a/rust/ai-research/replay-core-v4/src/inspect/css.rs
+++ b/rust/ai-research/replay-core-v4/src/inspect/css.rs
@@ -1,26 +1,196 @@
-//! Remote image URLs in CSS, found with the rules of v3 `css.rs`: the `url()` and the best `image-set()` candidate in the
-//! value of a property that shows an image. Custom properties and other properties do not count.
+//! Image URLs in CSS: the `url()`s of an image property, or of a custom property that reaches one through `var()`.
-/// Calls `found` with each image URL candidate in `css`: a `style` attribute, `<style>` text or `_cssText`, which v3
-/// reads the same way.
-pub fn image_urls(css: &str, found: &mut impl FnMut(&str)) {
- for (start, end, collect) in declaration_value_ranges(css) {
- if collect {
- value_urls(&css[start..end], found);
+use std::sync::atomic::{AtomicUsize, Ordering};
+
+use url::Url;
+
+use crate::url_policy::{
+ is_fetched_image_type, is_protocol_relative, is_remote, try_canonicalize, MAX_URL_LEN,
+};
+
+/// Each resolution copies the page's origin, so one event resolves at most this many relative URLs.
+pub const MAX_RESOLVED_URLS: usize = 1024;
+/// Far more than a real `image-set()` lists. A longer set gets no rewrite, so the inspector never holds its candidates.
+const MAX_IMAGE_SET_CANDIDATES: usize = 32;
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub enum CssUrl<'a> {
+ Image(&'a str),
+ Unshown(&'a str),
+ /// A resolved root-relative URL. The block holds only the relative text, which its rewrite replaces.
+ Resolved(&'a str),
+ /// A root-relative `url()` that did not resolve. Its text keeps no rewrite, which another window would apply here.
+ Unresolved(&'a str),
+ Rewrite(CssRewrite),
+}
+
+#[derive(Debug, Clone, PartialEq, Eq)]
+pub struct CssRewrite {
+ pub text: String,
+ pub replacement: String,
+ pub dropped: Vec<String>,
+ /// The URL that `replacement` holds in place of a relative one, for the writer to replace with its reference.
+ pub resolved: Option<String>,
+}
+
+/// The origin that a root-relative URL on `page_url` resolves against, as `https` because the fetcher loads nothing else.
+pub fn page_base(page_url: &str) -> Option<Url> {
+ if page_url.len() > MAX_URL_LEN {
+ return None;
+ }
+ let mut base = Url::parse(page_url).ok()?;
+ if base.scheme() != "https" {
+ base.set_scheme("https").ok()?;
+ }
+ base.set_path("/");
+ base.set_query(None);
+ base.set_fragment(None);
+ Some(base)
+}
+
+#[derive(Default)]
+pub struct Resolver<'p> {
+ base: Option<&'p Url>,
+ remaining: usize,
+ budget: Option<&'p AtomicUsize>,
+}
+
+impl<'p> Resolver<'p> {
+ pub fn new(base: Option<&'p Url>, budget: Option<&'p AtomicUsize>) -> Self {
+ Self {
+ base,
+ remaining: MAX_RESOLVED_URLS,
+ budget,
+ }
+ }
+
+ /// Only a root-relative URL, as `/a.png`, resolves. It needs the page's origin and nothing else, so no part of a page
+ /// path, which can hold a token, reaches a fetch, and a `<base>` or a style sheet's own URL cannot change it.
+ fn resolves(&self, source: &str) -> bool {
+ self.base.is_some() && self.remaining > 0 && is_root_relative(source)
+ }
+
+ fn resolve(&mut self, source: &str) -> Option<String> {
+ if !self.resolves(source) {
+ return None;
+ }
+ let absolute = String::from(self.base?.join(source).ok()?);
+ let within_budget = self.budget.is_none_or(|budget| {
+ budget
+ .fetch_update(Ordering::Relaxed, Ordering::Relaxed, |left| {
+ left.checked_sub(absolute.len())
+ })
+ .is_ok()
+ });
+ if !within_budget {
+ self.remaining = 0;
+ return None;
+ }
+ self.remaining -= 1;
+ Some(absolute)
+ }
+}
+
+/// `css` with each `url()` and `image-set()` that `replacement` names replaced, found as the inspector finds them, so
+/// that a quoted string, a comment or an escaped selector never matches. `None` when nothing changed.
+pub fn rewrite_functions(
+ css: &str,
+ replacement: &mut impl FnMut(&str) -> Option<String>,
+) -> Option<String> {
+ let bytes = css.as_bytes();
+ let mut out: Option<String> = None;
+ let mut copied_to = 0;
+ let mut position = 0;
+ while position < bytes.len() {
+ if starts_with(bytes, position, b"/*") {
+ position = skip_comment(bytes, position).unwrap_or(bytes.len());
+ continue;
+ }
+ if bytes[position] == b'\\' {
+ position = skip_escape(bytes, position);
+ continue;
+ }
+ if matches!(bytes[position], b'\'' | b'"') {
+ position = skip_quoted(bytes, position).unwrap_or(bytes.len());
+ continue;
+ }
+ let image_set = if function_at(bytes, position, b"image-set") {
+ Some(b"image-set".len())
+ } else if function_at(bytes, position, b"-webkit-image-set") {
+ Some(b"-webkit-image-set".len())
+ } else {
+ None
+ };
+ let end = if let Some(name_len) = image_set {
+ // The inspector stops at an unterminated set, and each later opener would scan to the end again.
+ let Some(end) = matching_paren(bytes, position + name_len) else {
+ break;
+ };
+ end
+ } else if function_at(bytes, position, b"url") {
+ scan_url_function(css, position).0
+ } else {
+ position += 1;
+ continue;
+ };
+ if let Some(replaced) = replacement(&css[position..end]) {
+ let out = out.get_or_insert_with(|| String::with_capacity(css.len()));
+ out.push_str(&css[copied_to..position]);
+ out.push_str(&replaced);
+ copied_to = end;
}
+ position = end;
+ }
+ let mut out = out?;
+ out.push_str(&css[copied_to..]);
+ Some(out)
+}
+
+/// Calls `found` with each image URL and rewrite in `css`: a `style` attribute, `<style>` text or `_cssText`.
+pub fn image_urls(css: &str, resolver: &mut Resolver<'_>, found: &mut impl FnMut(CssUrl<'_>)) {
+ for (start, end) in image_value_ranges(css) {
+ value_urls(&css[start..end], resolver, found);
+ }
+}
+
+pub fn declaration_image_urls(
+ property: &str,
+ value: &str,
+ resolver: &mut Resolver<'_>,
+ found: &mut impl FnMut(CssUrl<'_>),
+) {
+ if can_hold_image(property.trim()) {
+ value_urls(value, resolver, found);
}
}
-pub fn declaration_image_urls(property: &str, value: &str, found: &mut impl FnMut(&str)) {
- if is_image_property(property.trim()) {
- value_urls(value, found);
+/// `text` as CSSOM writes it inside an identifier, as a selector holds the URL of a class like `bg-[url(https://…)]`.
+pub fn escape_in_identifier(text: &str) -> String {
+ let mut escaped = String::with_capacity(text.len() * 2);
+ for character in text.chars() {
+ match character {
+ '\0' => escaped.push('\u{fffd}'),
+ '\u{1}'..='\u{1f}' | '\u{7f}' => {
+ escaped.push_str(&format!("\\{:x} ", u32::from(character)));
+ }
+ 'a'..='z' | 'A'..='Z' | '0'..='9' | '-' | '_' => escaped.push(character),
+ _ if !character.is_ascii() => escaped.push(character),
+ _ => {
+ escaped.push('\\');
+ escaped.push(character);
+ }
+ }
}
+ escaped
+}
+
+fn can_hold_image(property: &str) -> bool {
+ property.starts_with("--") || is_image_property(property)
}
-/// The value ranges of the declarations in `css`, and whether each is an image property.
-fn declaration_value_ranges(css: &str) -> Vec<(usize, usize, bool)> {
+fn image_value_ranges(css: &str) -> Vec<(usize, usize)> {
let bytes = css.as_bytes();
- let mut ranges: Vec<(usize, usize, bool)> = Vec::new();
+ let mut ranges: Vec<(usize, usize)> = Vec::new();
// Where the text of the current declaration starts, after the whitespace that leads it.
let mut segment_start: Option<usize> = None;
let mut brace_depth = 0usize;
@@ -64,12 +234,10 @@ fn declaration_value_ranges(css: &str) -> Vec<(usize, usize, bool)> {
{
// Each colon trims only the whitespace before it, so a long selector list stays linear.
let property = css[segment_start.unwrap_or(position)..position].trim_end();
- let image = is_image_property(property);
- if image || property.starts_with("--") {
+ if can_hold_image(property) {
ranges.push((
position + 1,
declaration_end(bytes, position + 1, brace_depth),
- image,
));
}
}
@@ -142,7 +310,7 @@ fn is_image_property(property: &str) -> bool {
)
}
-fn value_urls(value: &str, found: &mut impl FnMut(&str)) {
+fn value_urls(value: &str, resolver: &mut Resolver<'_>, found: &mut impl FnMut(CssUrl<'_>)) {
let bytes = value.as_bytes();
let mut position = 0;
while position < bytes.len() {
@@ -166,22 +334,19 @@ fn value_urls(value: &str, found: &mut impl FnMut(&str)) {
None
};
if let Some(name_len) = image_set {
- let open = position + name_len;
- let Some(end) = matching_paren(bytes, open) else {
+ let Some(end) = matching_paren(bytes, position + name_len) else {
return;
};
- if let Some(source) = select_image_set_candidate(&value[open + 1..end - 1]) {
- collect(&source, found);
- }
+ image_set_urls(&value[position..end], name_len, resolver, found);
position = end;
continue;
}
if function_at(bytes, position, b"url") {
let (resume_at, source) = scan_url_function(value, position);
- position = resume_at;
- if let Some(source) = source {
- collect(&source, found);
+ if let Some(source) = source.as_deref().and_then(collectable) {
+ url_function_urls(&value[position..resume_at], source, resolver, found);
}
+ position = resume_at;
continue;
}
position += 1;
@@ -189,49 +354,157 @@ fn value_urls(value: &str, found: &mut impl FnMut(&str)) {
}
/// v3 collects neither a fragment nor a source with an escape in it.
-fn collect(source: &str, found: &mut impl FnMut(&str)) {
+fn collectable(source: &str) -> Option<&str> {
let source = source.trim();
- if !source.starts_with('#') && !source.contains('\\') {
- found(source);
+ (!source.starts_with('#') && !source.contains('\\')).then_some(source)
+}
+
+fn url_function_urls(
+ function: &str,
+ source: &str,
+ resolver: &mut Resolver<'_>,
+ found: &mut impl FnMut(CssUrl<'_>),
+) {
+ match resolver.resolve(source) {
+ Some(absolute) => {
+ found(CssUrl::Resolved(&absolute));
+ found(CssUrl::Rewrite(CssRewrite {
+ text: function.to_owned(),
+ replacement: url_function(&absolute),
+ dropped: Vec::new(),
+ resolved: Some(absolute),
+ }));
+ }
+ None if is_root_relative(source) => found(CssUrl::Unresolved(function)),
+ None => found(CssUrl::Image(source)),
}
}
-fn scan_url_function(value: &str, start: usize) -> (usize, Option<String>) {
- let bytes = value.as_bytes();
- let Some(resume_at) = matching_paren(bytes, start + 3) else {
- return (bytes.len(), None);
+fn is_root_relative(source: &str) -> bool {
+ source.starts_with('/') && !is_protocol_relative(source)
+}
+
+/// A serialized URL percent-encodes `"`, so the quotes hold any URL that [`Resolver`] returns.
+fn url_function(url: &str) -> String {
+ format!("url(\"{url}\")")
+}
+
+/// A renderer picks the candidate for its pixel ratio and fails offline on any the fetcher did not take, so the set keeps
+/// one: the highest resolution of a stored type. Fetching one candidate per set limits the load on the image's host.
+fn image_set_urls(
+ function: &str,
+ name_len: usize,
+ resolver: &mut Resolver<'_>,
+ found: &mut impl FnMut(CssUrl<'_>),
+) {
+ let contents = &function[name_len + 1..function.len() - 1];
+ let mut candidates = Vec::new();
+ let mut count = 0;
+ let mut readable = true;
+ let split = split_image_set(contents, &mut |text| {
+ count += 1;
+ if count <= MAX_IMAGE_SET_CANDIDATES {
+ match image_set_candidate(text) {
+ Some(candidate) => candidates.push(candidate),
+ None => readable = false,
+ }
+ }
+ });
+ if split.is_none() || !readable {
+ return;
+ }
+ if count > MAX_IMAGE_SET_CANDIDATES {
+ split_image_set(contents, &mut |text| {
+ if let Some(source) = image_set_candidate(text).and_then(|candidate| candidate.source) {
+ if is_remote(&source) {
+ found(CssUrl::Unshown(&source));
+ }
+ }
+ });
+ return;
+ }
+ let loads = |source: &str| {
+ is_data_url(source)
+ || (is_remote(source) && try_canonicalize(source).is_ok())
+ || resolver.resolves(source)
};
- let contents_end = resume_at - 1;
- let mut position = start + 4;
- skip_whitespace(bytes, &mut position);
- let source = if matches!(bytes.get(position), Some(b'\'' | b'"')) {
- let content_start = position + 1;
- let Some(end) = skip_quoted(bytes, position) else {
- return (resume_at, None);
- };
- position = end;
- skip_whitespace(bytes, &mut position);
- if position != contents_end {
- return (resume_at, None);
+ let kept = candidates
+ .iter()
+ .enumerate()
+ .filter_map(|(index, candidate)| {
+ let source = candidate.source.as_deref().filter(|source| loads(source))?;
+ Some((index, source, candidate.resolution_dppx?))
+ })
+ .fold(None, |best: Option<(usize, &str, f64)>, candidate| {
+ if best.is_none_or(|best| candidate.2 > best.2) {
+ Some(candidate)
+ } else {
+ best
+ }
+ });
+ let mut dropped = Vec::new();
+ for (index, candidate) in candidates.iter().enumerate() {
+ let source = candidate.source.as_deref().unwrap_or_default();
+ if kept.is_none_or(|kept| kept.0 != index) && is_remote(source) {
+ found(CssUrl::Unshown(source));
+ dropped.push(source.to_owned());
}
- &value[content_start..end - 1]
- } else {
- &value[position..contents_end]
+ }
+ let Some((index, source, _)) = kept else {
+ return;
};
- let source = source.trim();
- (
- resume_at,
- (!source.is_empty() && !source.contains('\\')).then(|| source.to_owned()),
- )
+ let kept = &candidates[index];
+ let resolved = resolver.resolve(source);
+ match &resolved {
+ Some(absolute) => found(CssUrl::Resolved(absolute)),
+ None => found(CssUrl::Image(source)),
+ }
+ // A candidate kept as the page wrote it holds its URL as the writer's URL pass looks for it.
+ let candidate = match &resolved {
+ Some(absolute) => format!("{} {}", url_function(absolute), kept.descriptors.trim())
+ .trim_end()
+ .to_owned(),
+ None => kept.text.to_owned(),
+ };
+ // The writer moves an inline image out of the block before it rewrites CSS, so a set that holds one never matches.
+ let holds_data_url = candidates
+ .iter()
+ .any(|candidate| candidate.source.as_deref().is_some_and(is_data_url));
+ if !holds_data_url && (candidates.len() > 1 || resolved.is_some()) {
+ found(CssUrl::Rewrite(CssRewrite {
+ text: function.to_owned(),
+ replacement: format!("{}({candidate})", &function[..name_len]),
+ dropped,
+ resolved,
+ }));
+ }
+}
+
+fn is_data_url(source: &str) -> bool {
+ source
+ .get(..5)
+ .is_some_and(|scheme| scheme.eq_ignore_ascii_case("data:"))
}
-fn select_image_set_candidate(contents: &str) -> Option<String> {
+struct ImageSetCandidate<'a> {
+ text: &'a str,
+ source: Option<String>,
+ descriptors: &'a str,
+ resolution_dppx: Option<f64>,
+}
+
+fn split_image_set<'a>(contents: &'a str, each: &mut impl FnMut(&'a str)) -> Option<()> {
let bytes = contents.as_bytes();
- let mut best: Option<(f64, String)> = None;
let mut start = 0;
let mut position = 0;
let mut depth = 0usize;
- while position < bytes.len() {
+ while position <= bytes.len() {
+ if position == bytes.len() || (bytes[position] == b',' && depth == 0) {
+ each(&contents[start..position]);
+ start = position + 1;
+ position += 1;
+ continue;
+ }
if bytes[position] == b'\\' {
position = skip_escape(bytes, position);
continue;
@@ -247,75 +520,113 @@ fn select_image_set_candidate(contents: &str) -> Option<String> {
match bytes[position] {
b'(' => depth += 1,
b')' => depth = depth.saturating_sub(1),
- b',' if depth == 0 => {
- keep_better_candidate(&mut best, &contents[start..position])?;
- start = position + 1;
- }
_ => {}
}
position += 1;
}
- keep_better_candidate(&mut best, &contents[start..])?;
- best.map(|(_, source)| source)
+ Some(())
}
-fn keep_better_candidate(best: &mut Option<(f64, String)>, candidate: &str) -> Option<()> {
+fn image_set_candidate(candidate: &str) -> Option<ImageSetCandidate<'_>> {
let candidate = candidate.trim();
if candidate.is_empty() {
return None;
}
- let (source, remainder) = parse_image_set_source(candidate)?;
- let resolution = parse_resolution(remainder)?;
- if best
- .as_ref()
- .is_none_or(|(best_resolution, _)| resolution > *best_resolution)
- {
- *best = Some((resolution, source));
- }
- Some(())
-}
-
-fn parse_image_set_source(candidate: &str) -> Option<(String, &str)> {
let bytes = candidate.as_bytes();
- if function_at(bytes, 0, b"url") {
+ let (source, descriptors) = if function_at(bytes, 0, b"url") {
let (resume_at, source) = scan_url_function(candidate, 0);
- return Some((source?, &candidate[resume_at..]));
- }
- if matches!(bytes.first(), Some(b'\'' | b'"')) {
+ (source, &candidate[resume_at..])
+ } else if matches!(bytes[0], b'\'' | b'"') {
let end = skip_quoted(bytes, 0)?;
- let source = candidate[1..end - 1].to_owned();
- if source.contains('\\') {
- return None;
- }
- return Some((source, &candidate[end..]));
- }
- None
+ (Some(candidate[1..end - 1].to_owned()), &candidate[end..])
+ } else {
+ (None, "")
+ };
+ Some(ImageSetCandidate {
+ text: candidate,
+ source: source.as_deref().and_then(collectable).map(str::to_owned),
+ descriptors,
+ resolution_dppx: parse_descriptors(descriptors),
+ })
}
-fn parse_resolution(remainder: &str) -> Option<f64> {
+fn parse_descriptors(descriptors: &str) -> Option<f64> {
+ let bytes = descriptors.as_bytes();
let mut resolution = None;
- for token in remainder.split_ascii_whitespace() {
- let parsed = if let Some(value) = token.strip_suffix("dppx") {
- value.parse::<f64>().ok()
- } else if let Some(value) = token.strip_suffix("dpi") {
- value.parse::<f64>().ok().map(|value| value / 96.0)
- } else if let Some(value) = token.strip_suffix("dpcm") {
- value.parse::<f64>().ok().map(|value| value * 2.54 / 96.0)
- } else if let Some(value) = token.strip_suffix('x') {
- value.parse::<f64>().ok()
- } else if token.starts_with("type(") {
+ let mut position = 0;
+ loop {
+ skip_whitespace(bytes, &mut position);
+ if position == bytes.len() {
+ break;
+ }
+ if function_at(bytes, position, b"type") {
+ let end = matching_paren(bytes, position + 4)?;
+ let media_type = descriptors[position + 5..end - 1]
+ .trim()
+ .trim_matches(['"', '\'']);
+ if !is_fetched_image_type(media_type) {
+ return None;
+ }
+ position = end;
continue;
- } else {
- return None;
- };
- if resolution.replace(parsed?).is_some() {
+ }
+ let token_end = bytes[position..]
+ .iter()
+ .position(u8::is_ascii_whitespace)
+ .map_or(bytes.len(), |length| position + length);
+ if resolution
+ .replace(parse_resolution(&descriptors[position..token_end])?)
+ .is_some()
+ {
return None;
}
+ position = token_end;
}
let value = resolution.unwrap_or(1.0);
(value.is_finite() && value > 0.0).then_some(value)
}
+fn parse_resolution(token: &str) -> Option<f64> {
+ if let Some(value) = token.strip_suffix("dppx") {
+ value.parse::<f64>().ok()
+ } else if let Some(value) = token.strip_suffix("dpi") {
+ value.parse::<f64>().ok().map(|value| value / 96.0)
+ } else if let Some(value) = token.strip_suffix("dpcm") {
+ value.parse::<f64>().ok().map(|value| value * 2.54 / 96.0)
+ } else {
+ token.strip_suffix('x')?.parse::<f64>().ok()
+ }
+}
+
+fn scan_url_function(value: &str, start: usize) -> (usize, Option<String>) {
+ let bytes = value.as_bytes();
+ let Some(resume_at) = matching_paren(bytes, start + 3) else {
+ return (bytes.len(), None);
+ };
+ let contents_end = resume_at - 1;
+ let mut position = start + 4;
+ skip_whitespace(bytes, &mut position);
+ let source = if matches!(bytes.get(position), Some(b'\'' | b'"')) {
+ let content_start = position + 1;
+ let Some(end) = skip_quoted(bytes, position) else {
+ return (resume_at, None);
+ };
+ position = end;
+ skip_whitespace(bytes, &mut position);
+ if position != contents_end {
+ return (resume_at, None);
+ }
+ &value[content_start..end - 1]
+ } else {
+ &value[position..contents_end]
+ };
+ let source = source.trim();
+ (
+ resume_at,
+ (!source.is_empty() && !source.contains('\\')).then(|| source.to_owned()),
+ )
+}
+
fn function_at(bytes: &[u8], start: usize, name: &[u8]) -> bool {
bytes
.get(start..start + name.len())
@@ -406,13 +717,35 @@ fn starts_with(bytes: &[u8], start: usize, needle: &[u8]) -> bool {
mod tests {
use super::*;
+ fn found(css: &str, page_url: Option<&str>) -> Vec<String> {
+ let base = page_url.and_then(page_base);
+ let mut found = Vec::new();
+ image_urls(css, &mut Resolver::new(base.as_ref(), None), &mut |url| {
+ found.push(match url {
+ CssUrl::Image(url) => format!("image {url}"),
+ CssUrl::Unshown(url) => format!("unshown {url}"),
+ CssUrl::Resolved(url) => format!("resolved {url}"),
+ CssUrl::Unresolved(text) => format!("unresolved {text}"),
+ CssUrl::Rewrite(rewrite) => format!(
+ "rewrite {} => {} dropping {:?}",
+ rewrite.text, rewrite.replacement, rewrite.dropped
+ ),
+ })
+ });
+ found
+ }
+
#[test]
fn reads_past_escapes_outside_strings() {
// Tailwind escapes brackets, parentheses and quotes in its arbitrary value class names.
let css = r".bg-\[url\(\'\/hero\.png\'\)\]{background-image:url('/hero.png')}.a{background:url(https://cdn.example.com/a.png)}";
- let mut found = Vec::new();
- image_urls(css, &mut |url| found.push(url.to_owned()));
- assert_eq!(found, ["/hero.png", "https://cdn.example.com/a.png"]);
+ assert_eq!(
+ found(css, None),
+ [
+ "unresolved url('/hero.png')",
+ "image https://cdn.example.com/a.png"
+ ]
+ );
}
#[test]
@@ -421,8 +754,82 @@ mod tests {
"{} a:hover {{ background: url(https://cdn.example.com/a.png) }}",
"a:hover, b:focus-visible,\n".repeat(50_000)
);
- let mut found = Vec::new();
- image_urls(&css, &mut |url| found.push(url.to_owned()));
- assert_eq!(found, ["https://cdn.example.com/a.png"]);
+ assert_eq!(found(&css, None), ["image https://cdn.example.com/a.png"]);
+ }
+
+ #[test]
+ fn rewrites_what_an_offline_render_cannot_load() {
+ let page = Some("http://app.example.com/shop/item?id=1");
+ for (css, expected) in [
+ (
+ ":root{--hero:url(https://cdn.example.com/hero.png)}",
+ vec!["image https://cdn.example.com/hero.png"],
+ ),
+ (
+ ".a{background:url(/img/a.png)}",
+ vec![
+ "resolved https://app.example.com/img/a.png",
+ r#"rewrite url(/img/a.png) => url("https://app.example.com/img/a.png") dropping []"#,
+ ],
+ ),
+ (
+ ".a{background:url(../img/a.png)}",
+ vec!["image ../img/a.png"],
+ ),
+ (
+ r#".a{background-image:image-set("https://cdn.example.com/1x.png" 1x, url(https://cdn.example.com/2x.png) 2x)}"#,
+ vec![
+ "unshown https://cdn.example.com/1x.png",
+ "image https://cdn.example.com/2x.png",
+ r#"rewrite image-set("https://cdn.example.com/1x.png" 1x, url(https://cdn.example.com/2x.png) 2x) => image-set(url(https://cdn.example.com/2x.png) 2x) dropping ["https://cdn.example.com/1x.png"]"#,
+ ],
+ ),
+ (
+ r#".a{background:-webkit-image-set(url(/a.avif) type("image/avif") 2x, url(/a.jpg) type("image/jpeg") 1x, linear-gradient(red, blue) 3x)}"#,
+ vec![
+ "resolved https://app.example.com/a.jpg",
+ r#"rewrite -webkit-image-set(url(/a.avif) type("image/avif") 2x, url(/a.jpg) type("image/jpeg") 1x, linear-gradient(red, blue) 3x) => -webkit-image-set(url("https://app.example.com/a.jpg") type("image/jpeg") 1x) dropping []"#,
+ ],
+ ),
+ (
+ ".a{background:image-set(url(https://cdn.example.com/a.png))}",
+ vec!["image https://cdn.example.com/a.png"],
+ ),
+ ] {
+ assert_eq!(found(css, page), expected, "{css}");
+ }
+ assert_eq!(found(".a{background:url(a.png)}", None), ["image a.png"]);
+ let base = page_base("https://app.example.com/shop/item");
+ let budget = AtomicUsize::new("https://app.example.com/img/a.png".len());
+ let mut named = Vec::new();
+ image_urls(
+ ".a{background:url(/img/a.png)} .b{background:url(/img/b.png)}",
+ &mut Resolver::new(base.as_ref(), Some(&budget)),
+ &mut |url| {
+ if let CssUrl::Resolved(url) | CssUrl::Unresolved(url) = url {
+ named.push(url.to_owned());
+ }
+ },
+ );
+ assert_eq!(
+ named,
+ ["https://app.example.com/img/a.png", "url(/img/b.png)"]
+ );
+ let rewritten = rewrite_functions("url(/a.png) image-set( url(/a.png)", &mut |function| {
+ (function == "url(/a.png)").then(|| "url(x)".to_owned())
+ });
+ assert_eq!(rewritten.as_deref(), Some("url(x) image-set( url(/a.png)"));
+ let many: Vec<String> = (1..=MAX_IMAGE_SET_CANDIDATES + 1)
+ .map(|i| format!("url(https://cdn.example.com/{i}.png) {i}x"))
+ .collect();
+ let named = found(
+ &format!(".a{{background:image-set({})}}", many.join(", ")),
+ page,
+ );
+ assert_eq!(named.len(), many.len());
+ assert!(
+ named.iter().all(|url| url.starts_with("unshown ")),
+ "{named:?}"
+ );
}
}
diff --git a/rust/ai-research/replay-core-v4/src/inspect/dom.rs b/rust/ai-research/replay-core-v4/src/inspect/dom.rs
index 91cadf985c69..8cf5689af0dc 100644
--- a/rust/ai-research/replay-core-v4/src/inspect/dom.rs
+++ b/rust/ai-research/replay-core-v4/src/inspect/dom.rs
@@ -3,11 +3,15 @@
use std::collections::{HashMap, HashSet};
use std::ops::Range;
+use std::sync::atomic::AtomicUsize;
+use std::sync::Arc;
+
+use url::Url;
use crate::bytescan::{self as scan, Span};
use crate::url_policy::{is_fetched_image_type, is_remote, MAX_URL_LEN};
-use super::css;
+use super::css::{self, CssRewrite, CssUrl, Resolver};
use super::events::{field, fields, number, text};
use super::rrweb::{
ADOPTED_STYLE_SHEET, FULL_SNAPSHOT, INCREMENTAL_SNAPSHOT, MUTATION, STYLE_DECLARATION,
@@ -174,6 +178,8 @@ struct WindowDom {
/// or the window was evicted and came back.
unverifiable: bool,
last_used: u64,
+ /// From the window's own page URL, because another window of the session can be on another page.
+ page_base: Option<Arc<Url>>,
}
/// Why the inspector stopped naming a block's remote image URLs. The writer replaces only the URLs it is given, so it
@@ -209,6 +215,15 @@ pub struct RemoteUrls {
/// in `urls` on its own.
/// Boxed, because the session state counts this struct's size and most blocks hold no srcset.
srcsets: Option<Box<SrcsetValues>>,
+ /// Past its count and byte budget this keeps no more, which is safe: each URL a rewrite drops is in `urls` too, and a
+ /// relative URL stays as the page wrote it.
+ css_rewrites: Option<Box<CssRewrites>>,
+}
+
+#[derive(Debug, Default)]
+struct CssRewrites {
+ values: HashMap<String, Option<CssRewrite>>,
+ bytes: usize,
}
#[derive(Debug, Default)]
@@ -261,6 +276,32 @@ impl RemoteUrls {
}
}
+ /// A text that resolves two ways, as a root-relative URL in windows on two origins, keeps no rewrite, so that no
+ /// window shows another's image. Its URL then stays relative, as the page wrote it.
+ fn insert_css_rewrite(&mut self, text: String, rewrite: Option<CssRewrite>) {
+ let rewrites = self.css_rewrites.get_or_insert_with(Box::default);
+ if let Some(known) = rewrites.values.get_mut(&text) {
+ if known.as_ref().map(|known| &known.replacement)
+ != rewrite.as_ref().map(|rewrite| &rewrite.replacement)
+ {
+ *known = None;
+ }
+ return;
+ }
+ let bytes = text.len()
+ + rewrite.as_ref().map_or(0, |rewrite| {
+ rewrite.replacement.len()
+ + rewrite.dropped.iter().map(String::len).sum::<usize>()
+ + rewrite.resolved.as_ref().map_or(0, String::len)
+ });
+ if rewrites.values.len() < MAX_BLOCK_REMOTE_URLS
+ && rewrites.bytes + bytes <= MAX_BLOCK_REMOTE_URL_BYTES
+ {
+ rewrites.bytes += bytes;
+ rewrites.values.insert(text, rewrite);
+ }
+ }
+
pub fn extend(&mut self, other: RemoteUrls) {
// The URLs go in before the other's limit, so that a part that hit a limit still names what it holds.
for url in &other.urls {
@@ -270,6 +311,19 @@ impl RemoteUrls {
for (raw, (text, unfetched_type)) in other.srcsets.iter().flat_map(|s| &s.values) {
self.insert_srcset(raw, text, *unfetched_type);
}
+ for (text, rewrite) in other.css_rewrites.into_iter().flat_map(|r| r.values) {
+ self.insert_css_rewrite(text, rewrite);
+ }
+ }
+
+ pub fn take_css_rewrites(&mut self) -> Vec<CssRewrite> {
+ self.css_rewrites
+ .take()
+ .map(|rewrites| rewrites.values)
+ .unwrap_or_default()
+ .into_values()
+ .flatten()
+ .collect()
}
pub fn take_srcsets(&mut self) -> Vec<Srcset> {
@@ -311,6 +365,10 @@ fn fetchable(url: &str) -> Option<&str> {
impl Urls {
fn push(&mut self, url: &str) {
self.push_remote(url);
+ self.push_fetch(url);
+ }
+
+ fn push_fetch(&mut self, url: &str) {
let Some(url) = fetchable(url) else { return };
if self.in_message.len() < MAX_IMAGE_URLS && !self.in_message.contains(url) {
self.in_message.insert(url.to_owned());
@@ -326,10 +384,22 @@ impl Urls {
}
}
+ fn push_css(&mut self, found: CssUrl<'_>) {
+ match found {
+ CssUrl::Image(url) => self.push(url),
+ CssUrl::Unshown(url) => self.push_remote(url),
+ CssUrl::Resolved(url) => self.push_fetch(url),
+ CssUrl::Unresolved(text) => self.remote.insert_css_rewrite(text.to_owned(), None),
+ CssUrl::Rewrite(rewrite) => self
+ .remote
+ .insert_css_rewrite(rewrite.text.clone(), Some(rewrite)),
+ }
+ }
+
/// Takes the URLs one walk found.
fn merge(&mut self, walk: Urls) {
for url in &walk.urls {
- self.push(url);
+ self.push_fetch(url);
}
self.remote.extend(walk.remote);
}
@@ -358,7 +428,8 @@ pub struct Observed {
/// What one walk over serialized nodes found.
#[derive(Default)]
-struct Walk {
+struct Walk<'p> {
+ resolver: Resolver<'p>,
/// Each node and its parent. A root's parent is `None`.
ids: Vec<(i64, Option<i64>)>,
/// The walk met more nodes than the node cap, so `ids` stops short.
@@ -380,7 +451,7 @@ struct Walk {
root_sources: Vec<(Option<i64>, Option<Span>, Option<Span>)>,
}
-impl Walk {
+impl Walk<'_> {
fn push_id(&mut self, id: i64, parent: Option<i64>) {
if self.ids.len() < MAX_TRACKED_NODES {
self.ids.push((id, parent));
@@ -441,7 +512,9 @@ impl Walk {
fn push_css(&mut self, bytes: &[u8], value: Option<Span>) {
let Some(value) = value else { return };
if let Some(text) = text(bytes, Some(value)) {
- css::image_urls(&text, &mut |url| self.push_url_text(url));
+ css::image_urls(&text, &mut self.resolver, &mut |found| {
+ self.image_urls.push_css(found)
+ });
} else if scan::is_object(bytes, value) {
let Ok(entries) = scan::object_entries(bytes, value) else {
return;
@@ -450,9 +523,12 @@ impl Walk {
let property = std::str::from_utf8(&bytes[entry.key.0..entry.key.1]);
let declared = text(bytes, style_property_value(bytes, entry.value));
if let (Ok(property), Some(declared)) = (property, declared) {
- css::declaration_image_urls(property, &declared, &mut |url| {
- self.image_urls.push(url)
- });
+ css::declaration_image_urls(
+ property,
+ &declared,
+ &mut self.resolver,
+ &mut |found| self.image_urls.push_css(found),
+ );
}
}
}
@@ -928,16 +1004,21 @@ fn finish_node(bytes: &[u8], node: NodeFrame, parent: Option<&mut NodeFrame>, wa
}
/// What a full snapshot does to its window.
-enum Snapshot {
+enum Snapshot<'p> {
/// The player drops it, because its data or its node is not an object.
Dropped,
/// It does not decode or is too deep to walk, and the player may still build a DOM from a payload this deep.
Unreadable,
- Walked(Box<Walk>),
+ Walked(Box<Walk<'p>>),
}
/// Walks the DOM in a full snapshot's data, decoded when the event is compressed.
-fn walk_snapshot(inner: &[u8], data: Span, expansion: &Expansion) -> Snapshot {
+fn walk_snapshot<'p>(
+ inner: &[u8],
+ data: Span,
+ expansion: &Expansion,
+ resolver: Resolver<'p>,
+) -> Snapshot<'p> {
let (bytes, root) = if scan::is_string(inner, data) {
match expansion.json(data) {
Some(Some(json)) => match scan::locate_value(json, 0) {
@@ -954,7 +1035,10 @@ fn walk_snapshot(inner: &[u8], data: Span, expansion: &Expansion) -> Snapshot {
else {
return Snapshot::Dropped;
};
- let mut walk = Walk::default();
+ let mut walk = Walk {
+ resolver,
+ ..Walk::default()
+ };
if walk_nodes(bytes, node, &mut walk) {
Snapshot::Walked(Box::new(walk))
} else {
@@ -1127,6 +1211,10 @@ impl WindowDom {
+ table_bytes::<(i64, HiddenSources)>(self.hidden_sources.capacity())
+ self.hidden_source_bytes
+ table_bytes::<i64>(self.style_texts.capacity())
+ + self
+ .page_base
+ .as_ref()
+ .map_or(0, |base| std::mem::size_of::<Url>() + base.as_str().len())
}
}
@@ -1170,6 +1258,8 @@ impl DomInspector {
inner: &[u8],
event: &Event,
window: &str,
+ event_page_url: Option<&str>,
+ resolve_budget: Option<&AtomicUsize>,
stats: &mut BlockStats,
) -> Observed {
self.clock += 1;
@@ -1180,13 +1270,18 @@ impl DomInspector {
self.clock,
window,
);
+ if let Some(page_url) = event_page_url {
+ dom.page_base = css::page_base(page_url).map(Arc::new);
+ }
+ let page_base = dom.page_base.clone();
+ let resolver = Resolver::new(page_base.as_deref(), resolve_budget);
let mut observed = Observed::default();
match (event.kind, event.source) {
(Some(FULL_SNAPSHOT), _) => {
let expansion = cv::expansion(inner, event);
let last_used = dom.last_used;
match event.data.map_or(Snapshot::Dropped, |data| {
- walk_snapshot(inner, data, &expansion)
+ walk_snapshot(inner, data, &expansion, resolver)
}) {
Snapshot::Walked(mut walk) => {
// A full snapshot replaces the window's DOM.
@@ -1194,6 +1289,7 @@ impl DomInspector {
has_snapshot: true,
unverifiable: walk.truncated,
last_used,
+ page_base: page_base.clone(),
..WindowDom::default()
};
for &(id, parent) in &walk.ids {
@@ -1220,6 +1316,7 @@ impl DomInspector {
*dom = WindowDom {
unverifiable: true,
last_used,
+ page_base: page_base.clone(),
..WindowDom::default()
}
}
@@ -1231,7 +1328,7 @@ impl DomInspector {
if let Some(data) = event.data {
let expansion = cv::expansion(inner, event);
let (orphans, changed, undecoded) =
- observe_mutation(inner, data, &expansion, dom, &mut self.urls);
+ observe_mutation(inner, data, &expansion, dom, resolver, &mut self.urls);
stats.orphan_mutations += orphans;
stats.decode_failures += u32::from(undecoded);
observed.changed = changed;
@@ -1246,7 +1343,14 @@ impl DomInspector {
if matches!(*expansion, Expansion::Failed | Expansion::Skipped) {
stats.decode_failures += 1;
}
- style_sheet_urls(inner, data, event.source, &expansion, &mut self.urls);
+ style_sheet_urls(
+ inner,
+ data,
+ event.source,
+ &expansion,
+ resolver,
+ &mut self.urls,
+ );
}
}
_ => {}
@@ -1374,6 +1478,7 @@ fn observe_mutation(
data: Span,
expansion: &Expansion,
dom: &mut WindowDom,
+ resolver: Resolver<'_>,
urls: &mut Urls,
) -> (u32, u32, bool) {
// The recorder did not decode this mutation, so the window's DOM is not known after it.
@@ -1386,7 +1491,10 @@ fn observe_mutation(
return (0, 0, true);
}
let mut undecoded = false;
- let mut walk = Walk::default();
+ let mut walk = Walk {
+ resolver,
+ ..Walk::default()
+ };
let mut adds = Vec::new();
if let Some(list) = lists.adds {
let mut sources = Vec::new();
@@ -1561,9 +1669,11 @@ fn style_sheet_urls(
data: Span,
source: Option<u8>,
expansion: &Expansion,
+ mut resolver: Resolver<'_>,
urls: &mut Urls,
) {
- let mut push = |url: &str| urls.push(url);
+ let mut push = |found: CssUrl<'_>| urls.push_css(found);
+ let resolver = &mut resolver;
match source {
Some(STYLE_SHEET_RULE) => {
let [adds, replace, replace_sync] =
@@ -1571,13 +1681,13 @@ fn style_sheet_urls(
if let Some(adds) = adds.and_then(|adds| decoded_list(inner, adds, expansion)) {
for item in items(adds) {
if let Some(rule) = text(adds.bytes, field(adds.bytes, item, b"rule")) {
- css::image_urls(&rule, &mut push);
+ css::image_urls(&rule, resolver, &mut push);
}
}
}
for sheet in [replace, replace_sync] {
if let Some(sheet) = text(inner, sheet) {
- css::image_urls(&sheet, &mut push);
+ css::image_urls(&sheet, resolver, &mut push);
}
}
}
@@ -1586,7 +1696,7 @@ fn style_sheet_urls(
let [property, value] =
set.map_or([None; 2], |set| fields(inner, set, [b"property", b"value"]));
if let (Some(property), Some(value)) = (text(inner, property), text(inner, value)) {
- css::declaration_image_urls(&property, &value, &mut push);
+ css::declaration_image_urls(&property, &value, resolver, &mut push);
}
}
Some(ADOPTED_STYLE_SHEET) => {
@@ -1605,7 +1715,7 @@ fn style_sheet_urls(
span: rules,
}) {
if let Some(rule) = text(inner, field(inner, rule, b"rule")) {
- css::image_urls(&rule, &mut push);
+ css::image_urls(&rule, resolver, &mut push);
}
}
}
@@ -1692,7 +1802,14 @@ mod tests {
let message = events(json!([event]));
let mut stats = BlockStats::default();
let mut dom = DomInspector::default();
- dom.observe(&message.inner, &message.events[0], "w", &mut stats);
+ dom.observe(
+ &message.inner,
+ &message.events[0],
+ "w",
+ None,
+ None,
+ &mut stats,
+ );
// Remote image URLs go to the fetcher. The inline image in the srcset is the image scan's to move.
assert_eq!(
dom.take_image_urls(),
@@ -1741,12 +1858,14 @@ mod tests {
let mut dom = DomInspector::default();
let mut stats = BlockStats::default();
for event in &message.events {
- dom.observe(&message.inner, event, "w", &mut stats);
+ dom.observe(&message.inner, event, "w", None, None, &mut stats);
}
assert_eq!(stats.events_before_first_full_snapshot, 1);
assert_eq!(stats.orphan_mutations, 2);
}
+ const PAGE_URL: &str = "http://app.example.com/shop/item";
+
fn mutation(adds: Value, removes: Value, texts: Value, attributes: Value) -> Value {
json!({"type": 3, "timestamp": 2, "data": {"source": 0, "adds": adds, "removes": removes, "texts": texts, "attributes": attributes}})
}
@@ -1759,7 +1878,14 @@ mod tests {
let mut stats = BlockStats::default();
dom.start_message();
for event in &message.events {
- dom.observe(&message.inner, event, window, &mut stats);
+ dom.observe(
+ &message.inner,
+ event,
+ window,
+ Some(PAGE_URL),
+ None,
+ &mut stats,
+ );
}
(dom, stats)
}
@@ -1907,6 +2033,7 @@ mod tests {
mutation(json!([]), json!([]), json!([]), json!([{"id": 20, "attributes": {"alt": "x"}}])),
mutation(json!([]), json!([]), json!([]), json!([{"id": 20, "attributes": {"hidden": null}}])),
mutation(json!([]), json!([]), json!([]), json!([{"id": 6, "attributes": {"style": {"background-image": [format!("url({})", url("style-object")), "important"], "background": format!("url({})", url("style-object-plain")), "color": "red", "mask-image": false}}}])),
+ mutation(json!([]), json!([]), json!([]), json!([{"id": 6, "attributes": {"style": {"background-image": "url(\"/img/relative-style.png\")"}}}])),
// A `source` added on its own under a `picture`, one the page has or one the same mutation adds.
mutation(json!([{"parentId": 13, "node": element(40, "source", json!({"srcset": format!("{} 2x", url("lone-source"))}), json!([]))},
{"parentId": 41, "node": element(42, "source", json!({"srcset": url("same-mutation-source")}), json!([]))},
@@ -1918,7 +2045,7 @@ mod tests {
// New text in a `style` is CSS. Text elsewhere is not.
mutation(json!([]), json!([]), json!([{"id": 4, "value": format!(".a{{background:url({})}}", url("style-text-change"))}, {"id": 6, "value": format!("background:url({})", url("not-css"))}]), json!([])),
// Rules and declarations that a page sets through CSSOM, compressed or not.
- {"type": 3, "timestamp": 3, "cv": "2024-10", "data": {"source": 8, "id": 3, "adds": sdk_cv(&json!([{"rule": format!(".c{{background:url({})}}", url("sheet-rule")), "index": 0}]))}},
+ {"type": 3, "timestamp": 3, "cv": "2024-10", "data": {"source": 8, "id": 3, "adds": sdk_cv(&json!([{"rule": format!(".c{{background:url({})}} .e{{background:url(/img/relative-rule.png)}}", url("sheet-rule")), "index": 0}]))}},
{"type": 3, "timestamp": 3, "data": {"source": 8, "id": 3, "replaceSync": format!(".r{{cursor:url({}),auto}}", url("sheet-replace"))}},
{"type": 3, "timestamp": 3, "data": {"source": 13, "id": 3, "index": [0], "set": {"property": "background-image", "value": format!("url({})", url("declaration"))}}},
{"type": 3, "timestamp": 3, "data": {"source": 13, "id": 3, "index": [0], "set": {"property": "--x", "value": format!("url({})", url("custom-declaration"))}}},
@@ -1926,15 +2053,45 @@ mod tests {
]));
let (mut dom, _) = observe_all(&message, "w");
// A hidden or one-pixel img is not fetched, but the block holds its URL, so the writer replaces it too.
- let remote = dom.take_remote_urls().into_urls();
- for name in ["hidden", "display-none", "pixel", "shown-later", "lazy"] {
+ let mut remote = dom.take_remote_urls();
+ let mut rewrites: Vec<String> = remote
+ .take_css_rewrites()
+ .into_iter()
+ .map(|rewrite| rewrite.text)
+ .collect();
+ let remote = remote.into_urls();
+ for name in [
+ "hidden",
+ "display-none",
+ "pixel",
+ "shown-later",
+ "lazy",
+ "set-1x",
+ ] {
assert!(remote.contains(&url(name)), "{name}");
}
+ assert!(!remote
+ .iter()
+ .any(|url| url.ends_with("/img/relative-style.png")));
+ rewrites.sort();
+ assert_eq!(
+ rewrites,
+ [
+ format!(
+ "image-set(url({}) 1x, url({}) 2x)",
+ url("set-1x"),
+ url("set-2x")
+ ),
+ "url(\"/img/relative-style.png\")".to_owned(),
+ "url(/img/relative-rule.png)".to_owned(),
+ ]
+ );
let mut found = dom.take_image_urls();
found.sort();
let mut expected: Vec<String> = [
"before-snapshot",
"style-text",
+ "custom-property",
"set-2x",
"style-attribute",
"svg-href",
@@ -1950,6 +2107,7 @@ mod tests {
"sheet-rule",
"sheet-replace",
"declaration",
+ "custom-declaration",
"adopted",
"w_640,q_auto/srcset-comma",
"srcset-second",
@@ -1963,6 +2121,10 @@ mod tests {
]
.into_iter()
.map(url)
+ .chain([
+ "https://app.example.com/img/relative-style.png".to_owned(),
+ "https://app.example.com/img/relative-rule.png".to_owned(),
+ ])
.collect();
expected.sort();
assert_eq!(found, expected);
@@ -2104,19 +2266,42 @@ mod tests {
&message.inner,
&message.events[0],
&window.to_string(),
+ None,
+ None,
&mut stats,
);
}
assert!(!stats.node_cap_reached);
- dom.observe(&message.inner, &message.events[1], "0", &mut stats);
+ dom.observe(
+ &message.inner,
+ &message.events[1],
+ "0",
+ None,
+ None,
+ &mut stats,
+ );
assert!(stats.node_cap_reached);
for window in 0..MAX_WINDOWS {
let other = format!("other-{window}");
- dom.observe(&message.inner, &message.events[1], &other, &mut stats);
+ dom.observe(
+ &message.inner,
+ &message.events[1],
+ &other,
+ None,
+ None,
+ &mut stats,
+ );
}
let mut stats = BlockStats::default();
- dom.observe(&message.inner, &message.events[1], "0", &mut stats);
+ dom.observe(
+ &message.inner,
+ &message.events[1],
+ "0",
+ None,
+ None,
+ &mut stats,
+ );
assert!(stats.node_cap_reached);
}
diff --git a/rust/ai-research/replay-core-v4/src/inspect/session.rs b/rust/ai-research/replay-core-v4/src/inspect/session.rs
index 9b009decdf6d..8f2ee7d6dcb5 100644
--- a/rust/ai-research/replay-core-v4/src/inspect/session.rs
+++ b/rust/ai-research/replay-core-v4/src/inspect/session.rs
@@ -1,10 +1,12 @@
//! Everything the recorder learns about one session, block by block.
+use std::sync::atomic::AtomicUsize;
use std::sync::Arc;
use super::clicks::{ClickKind, ClickTracker};
+use super::css::CssRewrite;
use super::dom::{DomInspector, RemoteUrls, Srcset, UrlLimit};
-use super::events::{number, Activity, DataFields, EventCounter};
+use super::events::{href_from, number, Activity, DataFields, EventCounter};
use super::features::{BlockFeatures, FeatureRecorder, SurfacingFeatures};
use super::rrweb::{
is_click, CUSTOM, FULL_SNAPSHOT, INCREMENTAL_SNAPSHOT, META, MOUSE_INTERACTION, MUTATION,
@@ -52,6 +54,7 @@ pub struct BlockInspection {
image_urls: Vec<String>,
remote_urls: RemoteUrls,
events: u32,
+ resolve_budget: Option<Arc<AtomicUsize>>,
}
/// What the inspector learned about one block.
@@ -68,9 +71,18 @@ pub struct Inspected {
pub url_limit: Option<UrlLimit>,
/// The image-slot srcset values of the block, which the writer rebuilds from the candidates it keeps.
pub srcsets: Vec<Srcset>,
+ pub css_rewrites: Vec<CssRewrite>,
}
impl BlockInspection {
+ /// A block whose relative CSS URLs draw their resolved bytes from `budget`, which other blocks can share.
+ pub fn with_resolve_budget(budget: Arc<AtomicUsize>) -> Self {
+ Self {
+ resolve_budget: Some(budget),
+ ..Self::default()
+ }
+ }
+
pub fn finish(mut self) -> Inspected {
let features = self.features.finish();
let mut stats = self.counter.finish();
@@ -82,6 +94,7 @@ impl BlockInspection {
image_urls: self.image_urls,
url_limit: self.remote_urls.limit,
srcsets: self.remote_urls.take_srcsets(),
+ css_rewrites: self.remote_urls.take_css_rewrites(),
remote_urls: self.remote_urls.into_urls(),
}
}
@@ -148,6 +161,9 @@ impl SessionInspector {
block.events += 1;
let navigations = block.counter.stats.navigations;
let data = DataFields::read(inner, event);
+ let page_url = matches!(event.kind, Some(META | CUSTOM))
+ .then(|| href_from(inner, event.kind, &data))
+ .flatten();
let carried_href =
block
.counter
@@ -162,9 +178,14 @@ impl SessionInspector {
let navigated = block.counter.stats.navigations > navigations;
let waiting = block.counter.stats.events_before_first_full_snapshot;
- let observed = self
- .dom
- .observe(inner, event, window, &mut block.counter.stats);
+ let observed = self.dom.observe(
+ inner,
+ event,
+ window,
+ page_url.as_deref(),
+ block.resolve_budget.as_deref(),
+ &mut block.counter.stats,
+ );
block.image_urls.extend(self.dom.take_image_urls());
block.remote_urls.extend(self.dom.take_remote_urls());
if observed.full_snapshot {
@@ -304,6 +325,33 @@ mod tests {
assert!(stats.moments_truncated);
}
+ #[test]
+ fn resolves_relative_css_against_its_own_window() {
+ let mut session = SessionInspector::default();
+ let mut block = BlockInspection::default();
+ let meta = |host: &str| json!([{"type": 4, "timestamp": 1, "data": {"href": format!("https://{host}/shop/item")}}]);
+ let rule = json!([{"type": 3, "timestamp": 2, "data": {"source": 8, "id": 3, "replaceSync": ".a{background:url(/img/a.png)}"}}]);
+ session.observe_message(&mut block, &message_in(Some("a"), meta("a.example.com")));
+ session.observe_message(&mut block, &message_in(Some("b"), meta("b.example.com")));
+ session.observe_message(&mut block, &message_in(Some("a"), rule.clone()));
+ session.observe_message(&mut block, &message_in(Some("b"), rule));
+ let inspected = block.finish();
+ assert_eq!(
+ inspected.image_urls,
+ [
+ "https://a.example.com/img/a.png",
+ "https://b.example.com/img/a.png"
+ ]
+ );
+ assert!(inspected.css_rewrites.is_empty());
+
+ let mut block = BlockInspection::default();
+ let rule = json!([{"type": 3, "timestamp": 3, "data": {"source": 8, "id": 3, "replaceSync": ".b{background:url(/img/b.png)}"}}]);
+ session.observe_message(&mut block, &message_in(Some("unseen"), rule.clone()));
+ session.observe_message(&mut block, &message_in(Some("a"), rule));
+ assert!(block.finish().css_rewrites.is_empty());
+ }
+
#[test]
fn keeps_the_first_window_ids_a_block_sees() {
let moved = json!([{"type": 3, "timestamp": 1, "data": {"source": 1, "positions": []}}]);
diff --git a/rust/ai-research/replay-recorder-v4/src/batch.rs b/rust/ai-research/replay-recorder-v4/src/batch.rs
index 5dbdd24d6abb..a5f791928dc0 100644
--- a/rust/ai-research/replay-recorder-v4/src/batch.rs
+++ b/rust/ai-research/replay-recorder-v4/src/batch.rs
@@ -8,6 +8,7 @@ use std::time::{Duration, Instant};
use futures::stream::{self, StreamExt};
use metrics::counter;
use replay_core_v4::compression::{GZIP_MAGIC, MAX_DECOMPRESSED_BYTES};
+use replay_core_v4::inspect::css::CssRewrite;
use replay_core_v4::inspect::dom::{Srcset, UrlLimit};
use replay_core_v4::inspect::session::{BlockInspection, Moment, SessionInspector};
use replay_core_v4::inspect::stats::BlockStats;
@@ -98,6 +99,7 @@ pub struct Block {
pub url_limit: Option<UrlLimit>,
/// The image-slot srcset values of the block, which the writer rebuilds from the candidates it keeps.
pub srcsets: Vec<Srcset>,
+ pub css_rewrites: Vec<CssRewrite>,
/// The distinct pages of the block's surfacing features, which the writer stores as fingerprints.
pub visited_urls: Vec<String>,
/// The earliest Kafka timestamp of the block's messages, for the capture watermark of the index topic.
@@ -119,10 +121,14 @@ const SESSION_EXPAND_BUDGET: usize = 256 * 1024 * 1024;
/// memory parsed than in its payload, so a message that does not fit is skipped.
const FLUSH_EVENTS_BUDGET: usize = 1024 * 1024 * 1024;
+/// The bytes of resolved CSS URLs one flush keeps. Each carries its page's origin, which many short URLs can outgrow.
+const FLUSH_RESOLVED_BUDGET: usize = 64 * 1024 * 1024;
+
struct DecodeBudget {
kept: AtomicUsize,
expanded: AtomicUsize,
events: AtomicUsize,
+ resolved: Arc<AtomicUsize>,
}
impl DecodeBudget {
@@ -131,6 +137,7 @@ impl DecodeBudget {
kept: AtomicUsize::new(kept),
expanded: AtomicUsize::new(expanded),
events: AtomicUsize::new(FLUSH_EVENTS_BUDGET),
+ resolved: Arc::new(AtomicUsize::new(FLUSH_RESOLVED_BUDGET)),
}
}
}
@@ -484,8 +491,10 @@ fn inspect_session(
}
// A session can have messages on the main and the overflow topic in one flush. The inspector takes them in replay
// order, so that a mutation never comes before the nodes it changes. Each still counts toward its partition's block.
- let mut inspections: Vec<BlockInspection> =
- decoded.iter().map(|_| BlockInspection::default()).collect();
+ let mut inspections: Vec<BlockInspection> = decoded
+ .iter()
+ .map(|_| BlockInspection::with_resolve_budget(budget.resolved.clone()))
+ .collect();
let times: Vec<Vec<f64>> = decoded
.iter()
.map(|(_, messages, _)| messages.iter().map(replay_time).collect())
@@ -513,6 +522,7 @@ fn inspect_session(
remote_urls: inspected.remote_urls,
url_limit: inspected.url_limit,
srcsets: inspected.srcsets,
+ css_rewrites: inspected.css_rewrites,
visited_urls: inspected.features.visited_urls,
captured_at_ms,
}
diff --git a/rust/ai-research/replay-recorder-v4/src/consumer.rs b/rust/ai-research/replay-recorder-v4/src/consumer.rs
index 8fa8e2234e60..c552ceb3c7f2 100644
--- a/rust/ai-research/replay-recorder-v4/src/consumer.rs
+++ b/rust/ai-research/replay-recorder-v4/src/consumer.rs
@@ -842,6 +842,7 @@ mod tests {
remote_urls: Vec::new(),
url_limit: None,
srcsets: Vec::new(),
+ css_rewrites: Vec::new(),
visited_urls: Vec::new(),
captured_at_ms: None,
})
diff --git a/rust/ai-research/replay-recorder-v4/src/rewrite.rs b/rust/ai-research/replay-recorder-v4/src/rewrite.rs
index 55ad67164aeb..b18a88c9b006 100644
--- a/rust/ai-research/replay-recorder-v4/src/rewrite.rs
+++ b/rust/ai-research/replay-recorder-v4/src/rewrite.rs
@@ -9,13 +9,16 @@
//! were written again, so the rebuild never depends on a prediction of those bytes.
use std::borrow::Cow;
+use std::cell::Cell;
use std::collections::{BTreeMap, HashMap, HashSet};
use aho_corasick::{AhoCorasick, AhoCorasickKind, MatchKind};
use metrics::counter;
-use replay_core_v4::bytescan::{unescape, write_json_string};
+use replay_core_v4::bytescan::{unescape, write_json_string, Span};
+use replay_core_v4::inspect::css::{escape_in_identifier, rewrite_functions, CssRewrite};
use replay_core_v4::inspect::dom::{for_each_srcset_candidate, Srcset};
use replay_core_v4::scan::images::{extract_images, ImageLimits, ImageSink, AVIF_MARKER};
+use replay_core_v4::scan::tokens::{Token, Tokens};
use replay_core_v4::seal::{image_content_key, image_url_key};
use replay_core_v4::url_policy::is_remote;
@@ -117,12 +120,164 @@ fn ends_with_escape(bytes: &[u8], at: usize) -> bool {
const SRCSET_KEY: &[u8] = br#""srcset":"#;
+/// Runs a stage of `apply` on bytes that an earlier stage may own, and keeps those bytes when the stage changes nothing.
+macro_rules! then_stage {
+ ($bytes:expr, $writer:ident . $stage:ident) => {
+ match $bytes {
+ Cow::Borrowed(bytes) => $writer.$stage(bytes),
+ Cow::Owned(owned) => {
+ let changed = match $writer.$stage(&owned) {
+ Cow::Owned(changed) => Some(changed),
+ Cow::Borrowed(_) => None,
+ };
+ Cow::Owned(changed.unwrap_or(owned))
+ }
+ }
+ };
+}
+/// A CSS-escaped `:/` or `//` as a JSON string holds it, which starts a URL that a selector names. PHP-style JSON holds
+/// `:\/\/` too, so a `//` right after a plain colon does not count.
+const ESCAPED_URL_STARTS: [&[u8]; 2] = [br"\\:\\/", br"\\/\\/"];
+
+fn holds_escaped_url(starts: &AhoCorasick, bytes: &[u8]) -> bool {
+ starts.find_iter(bytes).any(|found| {
+ found.pattern().as_usize() == 0 || found.start() == 0 || bytes[found.start() - 1] != b':'
+ })
+}
+
+/// The block's URLs as a selector holds them. Most events hold none, so these are built when one first does.
+struct EscapedMatchers {
+ urls: Option<AhoCorasick>,
+ declined: Option<AhoCorasick>,
+ replacements: Vec<Vec<u8>>,
+}
+
+impl EscapedMatchers {
+ fn new(targets: &[Target]) -> Self {
+ let build = |targets: &mut dyn Iterator<Item = &Target>| {
+ let patterns: Vec<Vec<u8>> = targets
+ .map(|target| json_string_contents(&escape_in_identifier(&target.url)))
+ .collect();
+ (!patterns.is_empty())
+ .then(|| {
+ AhoCorasick::builder()
+ .kind(Some(AhoCorasickKind::ContiguousNFA))
+ .match_kind(MatchKind::LeftmostLongest)
+ .build(patterns)
+ .ok()
+ })
+ .flatten()
+ };
+ Self {
+ urls: build(&mut targets.iter()),
+ declined: build(&mut targets.iter().filter(|target| target.declined.is_some())),
+ replacements: targets
+ .iter()
+ .map(|target| {
+ json_string_contents(&escape_in_identifier(&String::from_utf8_lossy(
+ &target.replacement,
+ )))
+ })
+ .collect(),
+ }
+ }
+}
+
+fn json_string_contents(text: &str) -> Vec<u8> {
+ let mut escaped = Vec::with_capacity(text.len() + 2);
+ write_json_string(text, &mut escaped);
+ escaped.pop();
+ escaped.remove(0);
+ escaped
+}
+
+struct CssTarget {
+ /// With its URLs already replaced, so that a long page URL resolved for many short relative URLs never fills memory.
+ replacement: String,
+ /// The indexes in `targets` of the URLs that the rewrite removes or replaces, which count as found when it applies.
+ covers: Vec<usize>,
+ kind: &'static str,
+ found: bool,
+}
+
+/// The keys whose string value is CSS text. Every string under one counts too, such as a declaration in a style diff.
+const CSS_KEYS: [&[u8]; 7] = [
+ b"_cssText",
+ b"style",
+ b"textContent",
+ b"rule",
+ b"replace",
+ b"replaceSync",
+ b"value",
+];
+
+struct CssStrings<'b> {
+ bytes: &'b [u8],
+ tokens: Tokens<'b>,
+ containers_in_css: Vec<bool>,
+ key: Option<Span>,
+}
+
+impl<'b> CssStrings<'b> {
+ fn new(bytes: &'b [u8]) -> Self {
+ Self {
+ bytes,
+ tokens: Tokens::new(bytes, (0, bytes.len())),
+ containers_in_css: Vec::new(),
+ key: None,
+ }
+ }
+}
+
+impl Iterator for CssStrings<'_> {
+ type Item = Span;
+
+ fn next(&mut self) -> Option<Span> {
+ for token in self.tokens.by_ref() {
+ let in_css = self.containers_in_css.last() == Some(&true)
+ || self
+ .key
+ .is_some_and(|key| CSS_KEYS.contains(&&self.bytes[key.0..key.1]));
+ match token {
+ Token::ObjectStart | Token::ArrayStart => {
+ self.containers_in_css.push(in_css);
+ self.key = None;
+ }
+ Token::ObjectEnd | Token::ArrayEnd => {
+ self.containers_in_css.pop();
+ self.key = None;
+ }
+ Token::Key { span, escaped } => self.key = (!escaped).then_some(span),
+ Token::Value(span) => {
+ self.key = None;
+ if in_css && self.bytes.get(span.0) == Some(&b'"') {
+ return Some(span);
+ }
+ }
+ }
+ }
+ None
+ }
+}
+
+/// The metric label of a rewrite: the inspector rewrites a whole `url()` only to resolve a relative URL.
+fn css_rewrite_kind(text: &str) -> &'static str {
+ if text
+ .get(..4)
+ .is_some_and(|start| start.eq_ignore_ascii_case("url("))
+ {
+ "relative_url"
+ } else {
+ "image_set"
+ }
+}
+
struct Target {
url: String,
replacement: Vec<u8>,
/// The policy's reason, for a declined URL.
declined: Option<&'static str>,
- found: bool,
+ found: Cell<bool>,
}
pub struct UrlRewrite {
@@ -134,6 +289,11 @@ pub struct UrlRewrite {
/// Bytes that `apply` returned still hold a declined URL.
declined_kept: bool,
srcset_key: AhoCorasick,
+ escaped_url_starts: AhoCorasick,
+ escaped: Option<EscapedMatchers>,
+ css_matcher: Option<AhoCorasick>,
+ css_targets: Vec<CssTarget>,
+ css_texts: HashMap<String, usize>,
/// Remote URLs that the block names only in picture sources of a type the fetcher does not store.
unfetched: HashSet<String>,
/// Each inspected srcset value as the stored block holds it. Only these are rebuilt, so a srcset key outside an
@@ -156,6 +316,7 @@ pub struct UrlRewrite {
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct PassCounters {
srcset_rewrites: BTreeMap<&'static str, u64>,
+ css_rewrites: BTreeMap<(&'static str, &'static str), u64>,
url_rewrites: BTreeMap<(&'static str, &'static str), u64>,
pub removed_avif: u64,
}
@@ -166,6 +327,14 @@ impl PassCounters {
counter!("ai_research_v4_recorder_srcset_rewrites_total", "outcome" => outcome)
.increment(count);
}
+ for (&(kind, outcome), &count) in &self.css_rewrites {
+ counter!(
+ "ai_research_v4_recorder_css_rewrites_total",
+ "kind" => kind,
+ "outcome" => outcome,
+ )
+ .increment(count);
+ }
for (&(outcome, reason), &count) in &self.url_rewrites {
counter!(
"ai_research_v4_recorder_image_url_rewrites_total",
@@ -209,6 +378,7 @@ impl UrlRewrite {
pub fn new(
urls: &[String],
srcsets: &[Srcset],
+ css_rewrites: &[CssRewrite],
team_id: i64,
key_month: &str,
team_key: &[u8; 32],
@@ -218,9 +388,7 @@ impl UrlRewrite {
let mut targets = Vec::with_capacity(urls.len() + srcsets.len());
for url in urls {
// The block holds the URL as a JSON string holds it.
- let mut escaped = Vec::with_capacity(url.len() + 2);
- write_json_string(url, &mut escaped);
- patterns.push(escaped[1..escaped.len() - 1].to_vec());
+ patterns.push(json_string_contents(url));
let (replacement, declined) = match url_reference(url, team_id, key_month, &url_key) {
Ok((_, reference)) => (reference, None),
Err(decline) => (declined_marker(decline.label()), Some(decline.label())),
@@ -229,7 +397,7 @@ impl UrlRewrite {
url: url.clone(),
replacement: replacement.into_bytes(),
declined,
- found: false,
+ found: Cell::new(false),
});
}
let url_targets: HashMap<String, usize> = urls
@@ -237,6 +405,19 @@ impl UrlRewrite {
.enumerate()
.map(|(index, url)| (url.clone(), index))
.collect();
+ let css_matcher = (!css_rewrites.is_empty())
+ .then(|| {
+ AhoCorasick::builder()
+ .kind(Some(AhoCorasickKind::ContiguousNFA))
+ .match_kind(MatchKind::LeftmostLongest)
+ .build(
+ css_rewrites
+ .iter()
+ .map(|rewrite| json_string_contents(&rewrite.text)),
+ )
+ .ok()
+ })
+ .flatten();
// A URL that a source of a stored type also names stays, as the inspector merges identical srcset values.
let mut source_types: HashMap<&str, bool> = HashMap::new();
for srcset in srcsets {
@@ -273,6 +454,56 @@ impl UrlRewrite {
.ok()
})
.flatten();
+ let css_targets: Vec<CssTarget> = css_rewrites
+ .iter()
+ .map(|rewrite| {
+ let mut covers = Vec::new();
+ let resolved = match &rewrite.resolved {
+ Some(absolute) => {
+ let reference = match url_reference(absolute, team_id, key_month, &url_key)
+ {
+ Ok((_, reference)) => reference,
+ Err(decline) => declined_marker(decline.label()),
+ };
+ rewrite.replacement.replacen(
+ &format!("\"{absolute}\""),
+ &format!("\"{reference}\""),
+ 1,
+ )
+ }
+ None => rewrite.replacement.clone(),
+ };
+ let replacement = match &matcher {
+ Some(matcher) => {
+ let quoted =
+ [b"\"".as_slice(), &json_string_contents(&resolved), b"\""].concat();
+ let replaced = replace_targets(
+ matcher,
+ &targets,
+ false,
+ None,
+ &quoted,
+ &mut |index| covers.push(index),
+ );
+ unescape(&replaced, (0, replaced.len()))
+ .map_or_else(|_| resolved.clone(), Cow::into_owned)
+ }
+ None => resolved,
+ };
+ covers.extend(
+ rewrite
+ .dropped
+ .iter()
+ .filter_map(|url| url_targets.get(url).copied()),
+ );
+ CssTarget {
+ replacement,
+ covers,
+ kind: css_rewrite_kind(&rewrite.text),
+ found: false,
+ }
+ })
+ .collect();
let declined: Vec<&[u8]> = patterns
.iter()
.zip(&targets)
@@ -293,6 +524,16 @@ impl UrlRewrite {
targets,
declined_kept: false,
srcset_key: AhoCorasick::new([SRCSET_KEY]).expect("one literal pattern builds"),
+ escaped_url_starts: AhoCorasick::new(ESCAPED_URL_STARTS)
+ .expect("two literal patterns build"),
+ escaped: None,
+ css_matcher,
+ css_targets,
+ css_texts: css_rewrites
+ .iter()
+ .enumerate()
+ .map(|(index, rewrite)| (rewrite.text.clone(), index))
+ .collect(),
unfetched,
stored_srcsets,
url_targets,
@@ -342,7 +583,7 @@ impl UrlRewrite {
continue;
};
for url in covers {
- self.targets[url].found = true;
+ self.targets[url].found.set(true);
}
*self.counters.srcset_rewrites.entry(outcome).or_default() += 1;
let out = out.get_or_insert_with(|| Vec::with_capacity(bytes.len()));
@@ -359,17 +600,70 @@ impl UrlRewrite {
}
}
+ /// Rewrites only whole `url()` and `image-set()` functions inside JSON strings that hold CSS, found as the inspector
+ /// finds them. So a class such as `bg-[url(/a.png)]`, a quoted string or a comment keeps its text.
+ fn rewrite_css<'b>(&mut self, bytes: &'b [u8]) -> Cow<'b, [u8]> {
+ let Some(matcher) = self.css_matcher.clone() else {
+ return Cow::Borrowed(bytes);
+ };
+ let mut css_strings: Option<std::iter::Peekable<CssStrings<'_>>> = None;
+ let mut out: Option<Vec<u8>> = None;
+ let mut copied_to = 0;
+ let mut scanned_to = 0;
+ for found in matcher.find_iter(bytes) {
+ let start = found.start();
+ if start < scanned_to {
+ continue;
+ }
+ let strings = css_strings.get_or_insert_with(|| CssStrings::new(bytes).peekable());
+ while strings.next_if(|string| string.1 <= start).is_some() {}
+ let Some(&string) = strings.peek().filter(|string| string.0 < start) else {
+ continue;
+ };
+ scanned_to = string.1;
+ let Ok(css) = unescape(bytes, string) else {
+ continue;
+ };
+ let rewritten = rewrite_functions(&css, &mut |function| {
+ let &index = self.css_texts.get(function)?;
+ let target = &mut self.css_targets[index];
+ target.found = true;
+ for &url in &target.covers {
+ self.targets[url].found.set(true);
+ }
+ Some(target.replacement.clone())
+ });
+ if let Some(rewritten) = rewritten {
+ let out = out.get_or_insert_with(|| Vec::with_capacity(bytes.len()));
+ out.extend_from_slice(&bytes[copied_to..string.0]);
+ write_json_string(&rewritten, out);
+ copied_to = string.1;
+ }
+ }
+ match out {
+ Some(mut out) => {
+ out.extend_from_slice(&bytes[copied_to..]);
+ Cow::Owned(out)
+ }
+ None => Cow::Borrowed(bytes),
+ }
+ }
+
/// The bytes with each URL replaced, and in a DOM event each inspected srcset rebuilt. Returns the input when it
/// holds none. Only a full snapshot or a mutation holds image slots, so a srcset in any other event, such as a
/// custom event's payload with the same value, is left to the rewrite of each URL.
pub fn apply<'b>(&mut self, bytes: &'b [u8], dom_event: bool) -> Cow<'b, [u8]> {
- let replaced = if !dom_event {
- self.replace_urls(bytes)
+ let rebuilt = if dom_event {
+ self.rebuild_srcsets(bytes)
} else {
- match self.rebuild_srcsets(bytes) {
- Cow::Borrowed(bytes) => self.replace_urls(bytes),
- Cow::Owned(rebuilt) => Cow::Owned(self.replace_urls(&rebuilt).into_owned()),
- }
+ Cow::Borrowed(bytes)
+ };
+ let rewritten = then_stage!(rebuilt, self.rewrite_css);
+ let replaced = then_stage!(rewritten, self.replace_urls);
+ let replaced = if holds_escaped_url(&self.escaped_url_starts, replaced.as_ref()) {
+ then_stage!(replaced, self.replace_escaped_urls)
+ } else {
+ replaced
};
let (scrubbed, count) = self.scrubber.apply(&replaced);
if count == 0 {
@@ -379,39 +673,45 @@ impl UrlRewrite {
Cow::Owned(scrubbed.into_owned())
}
- fn replace_urls<'b>(&mut self, bytes: &'b [u8]) -> Cow<'b, [u8]> {
- let Some(matcher) = &self.matcher else {
+ fn replace_escaped_urls<'b>(&mut self, bytes: &'b [u8]) -> Cow<'b, [u8]> {
+ let escaped = self
+ .escaped
+ .get_or_insert_with(|| EscapedMatchers::new(&self.targets));
+ let Some(matcher) = &escaped.urls else {
return Cow::Borrowed(bytes);
};
- let mut out: Option<Vec<u8>> = None;
- let mut copied_to = 0;
- for found in matcher.find_iter(bytes) {
- let (start, end) = (found.start(), found.end());
- let index = found.pattern().as_usize();
- // A declined URL inside a longer URL is replaced too when the fallback asks, so that it cannot stay.
- let overwrite =
- self.fallback.overwrite_declined && self.targets[index].declined.is_some();
- if !overwrite
- && (start.checked_sub(1).is_some_and(|before| {
- continues_url(bytes[before]) && !ends_with_escape(bytes, start)
- }) || bytes.get(end).is_some_and(|&after| continues_url(after)))
- {
- continue;
- }
- let target = &mut self.targets[index];
- target.found = true;
- let out = out.get_or_insert_with(|| Vec::with_capacity(bytes.len()));
- out.extend_from_slice(&bytes[copied_to..start]);
- out.extend_from_slice(&target.replacement);
- copied_to = end;
+ let targets = &self.targets;
+ let out = replace_targets(
+ matcher,
+ targets,
+ self.fallback.overwrite_declined,
+ Some(&escaped.replacements),
+ bytes,
+ &mut |index| targets[index].found.set(true),
+ );
+ if escaped
+ .declined
+ .as_ref()
+ .is_some_and(|declined| declined.is_match(out.as_ref()))
+ {
+ self.declined_kept = true;
}
- let out = match out {
- Some(mut out) => {
- out.extend_from_slice(&bytes[copied_to..]);
- Cow::Owned(out)
- }
- None => Cow::Borrowed(bytes),
+ out
+ }
+
+ fn replace_urls<'b>(&mut self, bytes: &'b [u8]) -> Cow<'b, [u8]> {
+ let Some(matcher) = &self.matcher else {
+ return Cow::Borrowed(bytes);
};
+ let targets = &self.targets;
+ let out = replace_targets(
+ matcher,
+ targets,
+ self.fallback.overwrite_declined,
+ None,
+ bytes,
+ &mut |index| targets[index].found.set(true),
+ );
if let Some(declined) = &self.declined {
if let Some(kept) = declined.find(out.as_ref()) {
self.declined_kept = true;
@@ -438,7 +738,7 @@ impl UrlRewrite {
let mut unfound_hosts = Some(Vec::new());
let mut safe = !self.declined_kept;
for target in &self.targets {
- if !target.found && target.declined.is_some() && !covered(&target.url) {
+ if !target.found.get() && target.declined.is_some() && !covered(&target.url) {
safe = false;
match (url_host(&target.url), &mut unfound_hosts) {
(Some(host), Some(hosts)) => hosts.push(host),
@@ -452,8 +752,15 @@ impl UrlRewrite {
hosts.dedup();
}
let mut counters = self.counters;
+ for target in self.css_targets {
+ let outcome = if target.found { "rewritten" } else { "unfound" };
+ *counters
+ .css_rewrites
+ .entry((target.kind, outcome))
+ .or_default() += 1;
+ }
for target in self.targets {
- let outcome = match (target.found, target.declined) {
+ let outcome = match (target.found.get(), target.declined) {
(false, _) => "unrewritten",
(true, None) => "rewritten",
(true, Some(_)) => "declined",
@@ -474,6 +781,54 @@ impl UrlRewrite {
}
}
+fn replace_targets<'b>(
+ matcher: &AhoCorasick,
+ targets: &[Target],
+ overwrite_declined: bool,
+ escaped_replacements: Option<&[Vec<u8>]>,
+ bytes: &'b [u8],
+ replaced: &mut dyn FnMut(usize),
+) -> Cow<'b, [u8]> {
+ let mut out: Option<Vec<u8>> = None;
+ let mut copied_to = 0;
+ for found in matcher.find_iter(bytes) {
+ let (start, end) = (found.start(), found.end());
+ let index = found.pattern().as_usize();
+ // An escaped URL goes on with an escaped character, which a JSON string holds as `\\` and the character.
+ let continues_after = bytes.get(end).is_some_and(|&after| continues_url(after))
+ || (escaped_replacements.is_some()
+ && bytes.get(end..end + 2) == Some(br"\\")
+ && bytes
+ .get(end + 2)
+ .is_some_and(|&after| continues_url(after)));
+ // A declined URL inside a longer URL is replaced too when the fallback asks, so that it cannot stay.
+ let overwrite = overwrite_declined && targets[index].declined.is_some();
+ if !overwrite
+ && (start.checked_sub(1).is_some_and(|before| {
+ continues_url(bytes[before]) && !ends_with_escape(bytes, start)
+ }) || continues_after)
+ {
+ continue;
+ }
+ replaced(index);
+ let out = out.get_or_insert_with(|| Vec::with_capacity(bytes.len()));
+ out.extend_from_slice(&bytes[copied_to..start]);
+ out.extend_from_slice(match escaped_replacements {
+ Some(replacements) => &replacements[index],
+ None => &targets[index].replacement,
+ });
+ copied_to = end;
+ }
+ let out = match out {
+ Some(mut out) => {
+ out.extend_from_slice(&bytes[copied_to..]);
+ Cow::Owned(out)
+ }
+ None => Cow::Borrowed(bytes),
+ };
+ out
+}
+
/// The key of the JSON object entry whose string value holds `at`, when it is a plain short key.
fn enclosing_key(bytes: &[u8], at: usize) -> Option<&str> {
let mut open = at;
@@ -578,7 +933,7 @@ mod tests {
fn rewrite(urls: &[&str], bytes: &str) -> String {
let urls: Vec<String> = urls.iter().map(|u| (*u).to_owned()).collect();
- let mut rewrite = UrlRewrite::new(&urls, &[], 7, "2026-10", &TEAM_KEY);
+ let mut rewrite = UrlRewrite::new(&urls, &[], &[], 7, "2026-10", &TEAM_KEY);
String::from_utf8(rewrite.apply(bytes.as_bytes(), true).into_owned()).unwrap()
}
@@ -613,6 +968,7 @@ mod tests {
let mut rewrite = UrlRewrite::new(
&[a.to_owned(), b.to_owned(), declined.to_owned()],
&[],
+ &[],
7,
"2026-10",
&TEAM_KEY,
@@ -632,7 +988,7 @@ mod tests {
let url = "https://cdn.example.com/a.png?token=secret";
let hidden = r#"{"src":"https:\/\/cdn.example.com\/a.png?token=secr\u0065t","alt":"a\nb"}"#;
let canonical = canonical_strings(hidden.as_bytes());
- let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);
+ let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], &[], 7, "2026-10", &TEAM_KEY);
assert_eq!(
String::from_utf8(rewrite.apply(&canonical, true).into_owned()).unwrap(),
format!(
@@ -648,7 +1004,7 @@ mod tests {
Cow::Borrowed(_)
));
// A declined URL the block still holds fails the block.
- let rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);
+ let rewrite = UrlRewrite::new(&[url.to_owned()], &[], &[], 7, "2026-10", &TEAM_KEY);
assert!(!rewrite.finish().safe);
}
@@ -675,7 +1031,7 @@ mod tests {
}
// A second copy inside a longer URL stays as it is, so the block fails.
let url = "https://bucket.s3.amazonaws.com/a.png?X-Amz-Signature=abc123";
- let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);
+ let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], &[], 7, "2026-10", &TEAM_KEY);
rewrite.apply(
format!(r#"{{"src":"{url}","alt":"x{url}"}}"#).as_bytes(),
true,
diff --git a/rust/ai-research/replay-recorder-v4/src/urls.rs b/rust/ai-research/replay-recorder-v4/src/urls.rs
index 00475a835a1d..070d23dcea15 100644
--- a/rust/ai-research/replay-recorder-v4/src/urls.rs
+++ b/rust/ai-research/replay-recorder-v4/src/urls.rs
@@ -208,6 +208,7 @@ mod tests {
remote_urls: Vec::new(),
url_limit: None,
srcsets: Vec::new(),
+ css_rewrites: Vec::new(),
visited_urls: Vec::new(),
captured_at_ms: None,
};
diff --git a/rust/ai-research/replay-recorder-v4/src/write.rs b/rust/ai-research/replay-recorder-v4/src/write.rs
index 9d8e9168dcf9..601cda4e5f56 100644
--- a/rust/ai-research/replay-recorder-v4/src/write.rs
+++ b/rust/ai-research/replay-recorder-v4/src/write.rs
@@ -125,6 +125,7 @@ fn prepare_block(
let mut rewrite = UrlRewrite::new(
&block.remote_urls,
&block.srcsets,
+ &block.css_rewrites,
block.session.team_id,
key_month,
team_key,
@@ -785,6 +786,7 @@ mod tests {
remote_urls: Vec::new(),
url_limit: None,
srcsets: Vec::new(),
+ css_rewrites: Vec::new(),
visited_urls: Vec::new(),
captured_at_ms: None,
}
@@ -801,7 +803,10 @@ mod tests {
/// The block's stored lines, after the inspector named its remote URLs.
fn stored_lines(events: &str, extra: &[&str]) -> String {
- let mut inspected = block_of("kept", events);
+ stored(block_of("kept", events), extra)
+ }
+
+ fn stored(mut inspected: Block, extra: &[&str]) -> String {
let mut inspector = replay_core_v4::inspect::session::SessionInspector::default();
let mut inspection = replay_core_v4::inspect::session::BlockInspection::default();
for message in &inspected.messages {
@@ -811,6 +816,7 @@ mod tests {
inspected.remote_urls = found.remote_urls;
inspected.url_limit = found.url_limit;
inspected.srcsets = found.srcsets;
+ inspected.css_rewrites = found.css_rewrites;
inspected
.remote_urls
.extend(extra.iter().map(|url| (*url).to_owned()));
@@ -937,6 +943,100 @@ mod tests {
assert!(!lines.contains("https://"));
}
+ #[test]
+ fn leaves_css_only_images_an_offline_render_can_load() {
+ let compressed: String = replay_core_v4::compression::gzip(
+ json!([{"rule": ".a{background:url(/img/a.png)} .q{content:\"url(/img/a.png)\"} .b{background-image:image-set(url(https://cdn.example.com/1x.png) 1x, url(https://cdn.example.com/2x.png) 2x)} .c{background:url(/c.png?X-Amz-Signature=abc123)} .bg-\\[url\\(https\\:\\/\\/cdn\\.example\\.com\\/t\\.png\\)\\]{background-image:url(https://cdn.example.com/t.png)} .s-\\[url\\(https\\:\\/\\/cdn\\.example\\.com\\/s\\.png\\?X-Amz-Signature\\=abc123\\)\\]{background-image:url(https://cdn.example.com/s.png?X-Amz-Signature=abc123)}", "index": 0}])
+ .to_string()
+ .as_bytes(),
+ )
+ .unwrap()
+ .iter()
+ .map(|&b| char::from(b))
+ .collect();
+ let events = json!([
+ {"type": 4, "timestamp": 1, "data": {"href": "http://app.example.com/shop/item", "width": 800, "height": 600}},
+ {"type": 3, "timestamp": 2, "cv": "2024-10", "data": {"source": 8, "id": 3, "adds": compressed}},
+ {"type": 3, "timestamp": 3, "data": {"source": 0, "adds": [], "removes": [], "texts": [], "attributes": [{"id": 9, "attributes": {"class": "bg-[url(/img/a.png)]"}}, {"id": 10, "attributes": {"class": "bg-[url(https://cdn.example.com/t.png)]"}}]}},
+ {"type": 3, "timestamp": 4, "data": {"source": 8, "id": 3, "replaceSync": ".\\/\\/cdn\\.example\\.com\\/p\\.png\\?X-Amz-Signature\\=abc123{background:url(//cdn.example.com/p.png?X-Amz-Signature=abc123)}"}},
+ ]);
+ let lines = stored_lines(&events.to_string(), &[]);
+ let reference = |url: &str| {
+ crate::urls::url_reference(
+ url,
+ 7,
+ "2026-10",
+ &replay_core_v4::seal::image_url_key(&[6; 32]),
+ )
+ .unwrap()
+ .1
+ };
+ for expected in [
+ format!(
+ r#"url(\"{}\")"#,
+ reference("https://app.example.com/img/a.png")
+ ),
+ format!(
+ r#"image-set(url({}) 2x)"#,
+ reference("https://cdn.example.com/2x.png")
+ ),
+ format!(
+ r#"url(\"{}\")"#,
+ crate::rewrite::declined_marker("credential")
+ ),
+ ] {
+ assert!(lines.contains(&expected), "{expected} in {lines}");
+ }
+ for kept in [
+ r#""class":"bg-[url(/img/a.png)]""#,
+ r#".q{content:\"url(/img/a.png)\"}"#,
+ ] {
+ assert!(lines.contains(kept), "{kept} in {lines}");
+ }
+ let class_url = reference("https://cdn.example.com/t.png");
+ for expected in [
+ format!(r#""class":"bg-[url({class_url})]""#),
+ format!(r".bg-\\[url\\({}\\)\\]", class_url.replace(':', r"\\:")),
+ ] {
+ assert!(lines.contains(&expected), "{expected} in {lines}");
+ }
+ for gone in [":url(/img/a.png)", "1x.png", "1x,", "Signature"] {
+ assert!(!lines.contains(gone), "{gone} in {lines}");
+ }
+ }
+
+ #[test]
+ fn leaves_a_relative_url_that_resolves_two_ways_as_the_page_wrote_it() {
+ let window = |id: &str, href: &str| {
+ let mut message = block_of(
+ "kept",
+ &json!([
+ {"type": 4, "timestamp": 1, "data": {"href": href, "width": 800, "height": 600}},
+ {"type": 3, "timestamp": 2, "data": {"source": 8, "id": 3, "replaceSync": ".a{background:url(/a.png)}"}},
+ ])
+ .to_string(),
+ )
+ .messages
+ .remove(0);
+ message.window_id = Some(Arc::from(id));
+ message
+ };
+ let mut block = block_of("kept", "[]");
+ block.messages = vec![
+ window("w1", "http://10.0.0.5/admin"),
+ window("w2", "https://app.example.com/shop"),
+ ];
+ let lines = stored(block, &[]);
+ // A private host declines its resolved URL, which must not send the block to the host scrub.
+ for kept in [
+ r#""href":"http://10.0.0.5/admin""#,
+ r#""href":"https://app.example.com/shop""#,
+ ] {
+ assert!(lines.contains(kept), "{kept} in {lines}");
+ }
+ assert_eq!(lines.matches("url(/a.png)").count(), 2, "{lines}");
+ }
+
#[test]
fn leaves_a_picture_only_the_sources_that_can_load() {
let (avif, webp, signed, fallback) = (