:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(ai-research-v4): keep only loadable candidates in image srcsets (#124841)
Nicholas Waltz75960fdb597b
diff --git a/products/ai_training/docs/replay-data.md b/products/ai_training/docs/replay-data.mdindex 30ad40bec97d..2202acd7b70d 100644--- a/products/ai_training/docs/replay-data.md+++ b/products/ai_training/docs/replay-data.md@@ -465,6 +465,7 @@ The recorder streams each block into the encoder. Read a block as a stream: a Br- A remote image URL becomes `imageurl:v4:{team_id}:{YYYY-MM}:{hash}` in the block and in its fetch job, where the month is the UTC month of the write. The block and the job always carry the same reference, so a reader opens the fetched image directly.- A URL that the URL policy refuses becomes `imageurl:v4:declined:{reason}` in the block, for example `declined:credential`, `declined:non_public_host` or `declined:tracking_beacon`. The original URL cannot be recovered. The recorder counts rewrites and refusals in `ai_research_v4_recorder_image_url_rewrites_total`.- The rewrite covers every remote URL that the recorder reads from an image slot: image sources, `srcset`, SVG image links, video posters, `<picture>` sources and CSS `url(...)` in style attributes, style sheets and style rules. It also covers `src`, `srcset`, `href` and `poster` in an attribute change on a node the recorder does not know, for example after a restart. `srcset` keeps its descriptors and CSS keeps its `url(...)` syntax. It is not limited by the 512 URLs per message that get fetch jobs.+- A srcset in an image slot keeps only the candidates the recorder stores. A candidate the URL policy refuses leaves the list instead of becoming a `declined` marker, and the rest keep their descriptors. A `<picture>` source whose `type` names a media type the image fetcher does not store, such as `image/avif`, gets no fetch job and loses its remote candidates. Its inline images, such as SVG data URIs, stay. A browser skips a source with no candidates, so the replay falls back to the next source or the `img`. The recorder counts these as `ai_research_v4_recorder_srcset_rewrites_total{outcome="trimmed"|"emptied"}`. A srcset that an inline image extraction changed falls back to the rewrite of each URL.- The recorder reads a protocol-relative URL, such as `//cdn.example.com/a.png`, as `https`. It gets the same reference or `declined` marker as the `https` form, and the rewrite replaces the text as the block holds it. A browser reads a backslash as a slash there, so `\\cdn.example.com/a.png` counts too, and so does a URL such as `https:\\cdn.example.com/a.png` with any mix of slashes and backslashes after the scheme, or none: on a page of another scheme the browser loads `https:cdn.example.com/a.png` from that host.- Some URLs stay as they arrived: relative URLs, remote URLs outside image slots (links, frames, text, CSS custom properties and other CSS properties), the `image-set()` candidates the recorder does not choose, URLs inside a compressed value that did not decode or that the decode budget skipped, and URL text written in another form than the recorder read it, for example with a CSS escape. The recorder counts the last kind as `outcome="unrewritten"`.- Before the rewrite, the recorder writes again any JSON string that holds an escape capture would not write, for example `\/`, so an escaped URL is found too. A scheme matches in any letter case.diff --git a/rust/ai-research/replay-core-v4/src/inspect/dom.rs b/rust/ai-research/replay-core-v4/src/inspect/dom.rsindex 1ae79a06beaf..2eaeaddca6c0 100644--- a/rust/ai-research/replay-core-v4/src/inspect/dom.rs+++ b/rust/ai-research/replay-core-v4/src/inspect/dom.rs@@ -5,7 +5,7 @@ use std::collections::{HashMap, HashSet};use std::ops::Range;use crate::bytescan::{self as scan, Span};-use crate::url_policy::{self, MAX_URL_LEN};+use crate::url_policy::{is_fetched_image_type, is_remote, MAX_URL_LEN};use super::css;use super::events::{field, fields, number, text};@@ -82,7 +82,10 @@ enum Media {},SvgImage,Video,- PictureSource,+ /// A `source` under a `picture`. Its `type` can name a media type the fetcher does not store, such as AVIF.+ PictureSource {+ unfetched_type: bool,+ },/// A `picture`, so that a `source` added under it later is known to be a picture source.Picture,}@@ -204,6 +207,26 @@ pub struct RemoteUrls {urls: HashSet<String>,bytes: usize,pub limit: Option<UrlLimit>,+ /// Each image-slot srcset value, and whether it belongs to a source of a type the fetcher does not store. The+ /// writer rebuilds each from the candidates it keeps, and it builds one matcher over them as over `urls`, so they+ /// share the byte budget of `urls`. Past it this keeps no more, which is safe: the writer still replaces each URL+ /// in `urls` on its own.+ /// Boxed, because the session state counts this struct's size and most blocks hold no srcset.+ srcsets: Option<Box<SrcsetValues>>,+}++#[derive(Debug, Default)]+struct SrcsetValues {+ texts: HashMap<String, bool>,+ bytes: usize,+}++/// An image-slot srcset value as the event holds it.+#[derive(Debug, Clone, PartialEq, Eq)]+pub struct Srcset {+ pub text: String,+ /// The value belongs to a `<source>` whose `type` the fetcher does not store, so none of its images is stored.+ pub unfetched_type: bool,}impl RemoteUrls {@@ -224,11 +247,40 @@ impl RemoteUrls {};}+ /// A value seen on a source of a stored type and on one of another type keeps its candidates.+ fn insert_srcset(&mut self, text: &str, unfetched_type: bool) {+ let srcsets = self.srcsets.get_or_insert_with(Box::default);+ if let Some(known) = srcsets.texts.get_mut(text) {+ *known &= unfetched_type;+ } else if srcsets.texts.len() < MAX_BLOCK_REMOTE_URLS+ && srcsets.bytes + text.len() <= MAX_BLOCK_REMOTE_URL_BYTES+ {+ srcsets.bytes += text.len();+ srcsets.texts.insert(text.to_owned(), unfetched_type);+ }+ }+pub fn extend(&mut self, other: RemoteUrls) {self.limit = self.limit.or(other.limit);for url in &other.urls {self.insert(url);}+ for (text, unfetched_type) in other.srcsets.iter().flat_map(|s| &s.texts) {+ self.insert_srcset(text, *unfetched_type);+ }+ }++ pub fn take_srcsets(&mut self) -> Vec<Srcset> {+ self.srcsets+ .take()+ .map(|srcsets| srcsets.texts)+ .unwrap_or_default()+ .into_iter()+ .map(|(text, unfetched_type)| Srcset {+ text,+ unfetched_type,+ })+ .collect()}pub fn into_urls(self) -> Vec<String> {@@ -253,22 +305,6 @@ fn fetchable(url: &str) -> Option<&str> {(url.len() <= MAX_URL_LEN && is_remote(url)).then_some(url)}-/// A URL scheme is not case sensitive, so `HTTPS://` loads as `https://` does. A browser reads any run of slashes and-/// backslashes after `http:` or `https:` as `//`, and none at all on a page of another scheme, so `https:\\host` and-/// `https:host` can load from the host too. A protocol-relative URL does as well. The writer must replace each of them-/// like any other remote URL.-fn is_remote(url: &str) -> bool {- let after_scheme = ["https:", "http:"].iter().find_map(|scheme| {- url.get(..scheme.len())- .filter(|start| start.eq_ignore_ascii_case(scheme))- .map(|_| &url[scheme.len()..])- });- match after_scheme {- Some(rest) => !rest.is_empty(),- None => url_policy::is_protocol_relative(url),- }-}-impl Urls {fn push(&mut self, url: &str) {self.push_remote(url);@@ -338,7 +374,7 @@ struct Walk {hidden: Vec<(i64, HiddenSources)>,/// `source` elements that are a walk's root, with their srcset. Whether each is a picture source depends on the/// parent that the mutation names.- root_sources: Vec<(Option<i64>, Option<Span>)>,+ root_sources: Vec<(Option<i64>, Option<Span>, Option<Span>)>,}impl Walk {@@ -380,6 +416,25 @@ impl Walk {}}+ fn push_srcset_value(&mut self, bytes: &[u8], value: Option<Span>, unfetched_type: bool) {+ if let Some(srcset) = text(bytes, value) {+ self.image_urls+ .remote+ .insert_srcset(&srcset, unfetched_type);+ }+ }++ /// A picture source of a type the fetcher does not store gets no fetch job, and the writer empties its srcset, so+ /// the browser falls back to the next source or the img.+ fn push_source(&mut self, bytes: &[u8], srcset: Option<Span>, unfetched_type: bool) {+ if unfetched_type {+ self.push_remote(bytes, srcset, true);+ } else {+ self.push_srcset(bytes, srcset);+ }+ self.push_srcset_value(bytes, srcset, unfetched_type);+ }+fn push_css(&mut self, bytes: &[u8], value: Option<Span>) {let Some(value) = value else { return };if let Some(text) = text(bytes, Some(value)) {@@ -401,9 +456,18 @@ impl Walk {}}-/// The URLs of a srcset, parsed as HTML parses one: a URL is a run of non-whitespace, so it can hold a comma. A comma-/// ends a candidate only at the end of its URL or after its descriptors.-fn srcset_urls(srcset: &str, found: &mut impl FnMut(&str)) {+/// The candidates of a srcset as (URL, descriptors), parsed as HTML parses one: a URL is a run of non-whitespace, so+/// it can hold a comma. A comma ends a candidate only at the end of its URL or after its descriptors.+pub fn srcset_candidates(srcset: &str) -> Vec<(&str, &str)> {+ let mut candidates = Vec::new();+ for_each_srcset_candidate(srcset, |url, descriptors| {+ candidates.push((url, descriptors))+ });+ candidates+}++/// Streams the candidates of [`srcset_candidates`], so that a message-sized srcset needs no list of them.+fn for_each_srcset_candidate<'s>(srcset: &'s str, mut found: impl FnMut(&'s str, &'s str)) {let bytes = srcset.as_bytes();let mut position = 0;loop {@@ -424,23 +488,33 @@ fn srcset_urls(srcset: &str, found: &mut impl FnMut(&str)) {position += 1;}let url = srcset[start..position].trim_end_matches(',');- found(url);if url.len() < position - start {+ found(url, "");continue;}+ let descriptors = position;+ let mut end = bytes.len();let mut in_parens = false;while let Some(&byte) = bytes.get(position) {position += 1;match byte {b'(' => in_parens = true,b')' => in_parens = false,- b',' if !in_parens => break,+ b',' if !in_parens => {+ end = position - 1;+ break;+ }_ => {}}}+ found(url, srcset[descriptors..end].trim());}}+fn srcset_urls(srcset: &str, found: &mut impl FnMut(&str)) {+ for_each_srcset_candidate(srcset, |url, _| found(url));+}+fn style_property_value(bytes: &[u8], span: Span) -> Option<Span> {if scan::is_array(bytes, span) {scan::array_items(bytes, span).ok()?.next()?.ok()@@ -464,6 +538,7 @@ struct Attributes {hidden: Option<Span>,width: Option<Span>,height: Option<Span>,+ source_type: Option<Span>,}impl Attributes {@@ -481,6 +556,7 @@ impl Attributes {b"hidden" => &mut self.hidden,b"width" => &mut self.width,b"height" => &mut self.height,+ b"type" => &mut self.source_type,_ => return None,})}@@ -503,6 +579,7 @@ impl Attributes {walk.push_remote(bytes, self.src, false);walk.push_remote(bytes, self.rr_src, false);walk.push_remote(bytes, self.srcset, true);+ walk.push_srcset_value(bytes, self.srcset, false);walk.push_remote(bytes, self.href, false);walk.push_remote(bytes, self.xlink_href, false);walk.push_remote(bytes, self.poster, false);@@ -515,18 +592,22 @@ impl Attributes {walk.push_url(bytes, self.src);walk.push_url(bytes, self.rr_src);walk.push_srcset(bytes, self.srcset);+ walk.push_srcset_value(bytes, self.srcset, false);}Some(Media::Img { .. }) => {walk.push_remote(bytes, self.src, false);walk.push_remote(bytes, self.rr_src, false);walk.push_remote(bytes, self.srcset, true);+ walk.push_srcset_value(bytes, self.srcset, false);}Some(Media::SvgImage) => {walk.push_url(bytes, self.href);walk.push_url(bytes, self.xlink_href);}Some(Media::Video) => walk.push_url(bytes, self.poster),- Some(Media::PictureSource) => walk.push_srcset(bytes, self.srcset),+ Some(Media::PictureSource { unfetched_type }) => {+ walk.push_source(bytes, self.srcset, unfetched_type)+ }_ => {}}walk.push_css(bytes, self.style);@@ -534,6 +615,11 @@ impl Attributes {}}+/// An empty `type` gives the browser no hint, as a missing one does.+fn is_unfetched_type(bytes: &[u8], source_type: Option<Span>) -> bool {+ text(bytes, source_type).is_some_and(|t| !t.trim().is_empty() && !is_fetched_image_type(&t))+}+/// A CSS or HTML length read as pixels, as in v3: `1`, `1px` and `1.0` are one pixel, other units are unknown.fn px_length(text: &str) -> Option<f64> {let text = text.trim().to_ascii_lowercase();@@ -650,7 +736,7 @@ struct NodeFrame {/// Child text, and child `source` elements with their srcset, which count only under a `style` or a `picture`./// They wait here because the tag can come after the children.child_texts: Vec<(Option<i64>, Span)>,- child_sources: Vec<(Option<i64>, Option<Span>)>,+ child_sources: Vec<(Option<i64>, Option<Span>, Option<Span>)>,}#[derive(Clone, Copy, PartialEq, Eq)]@@ -748,9 +834,11 @@ fn finish_node(bytes: &[u8], node: NodeFrame, parent: Option<&mut NodeFrame>, wa}}if node.tag == Tag::Picture {- for &(id, srcset) in &node.child_sources {- walk.media.extend(id.map(|id| (id, Media::PictureSource)));- walk.push_srcset(bytes, srcset);+ for &(id, srcset, source_type) in &node.child_sources {+ let unfetched_type = is_unfetched_type(bytes, source_type);+ walk.media+ .extend(id.map(|id| (id, Media::PictureSource { unfetched_type })));+ walk.push_source(bytes, srcset, unfetched_type);}}match node.id {@@ -814,7 +902,11 @@ fn finish_node(bytes: &[u8], node: NodeFrame, parent: Option<&mut NodeFrame>, wa.extend(node.text.map(|text| (node.id, text)));}if node.tag == Tag::Source {- parent.child_sources.push((node.id, node.attributes.srcset));+ parent.child_sources.push((+ node.id,+ node.attributes.srcset,+ node.attributes.source_type,+ ));}}None => {@@ -822,7 +914,11 @@ fn finish_node(bytes: &[u8], node: NodeFrame, parent: Option<&mut NodeFrame>, wawalk.push_id(id, None);}if node.tag == Tag::Source {- walk.root_sources.push((node.id, node.attributes.srcset));+ walk.root_sources.push((+ node.id,+ node.attributes.srcset,+ node.attributes.source_type,+ ));}}}@@ -1318,11 +1414,13 @@ fn observe_mutation(.filter(|(_, media)| *media == Media::Picture).map(|&(id, _)| id).collect();- for (parent, (id, srcset)) in sources {+ for (parent, (id, srcset, source_type)) in sources {let Some(parent) = parent else { continue };if pictures.contains(&parent) || dom.media.get(&parent) == Some(&Media::Picture) {- walk.media.extend(id.map(|id| (id, Media::PictureSource)));- walk.push_srcset(list.bytes, srcset);+ let unfetched_type = is_unfetched_type(list.bytes, source_type);+ walk.media+ .extend(id.map(|id| (id, Media::PictureSource { unfetched_type })));+ walk.push_source(list.bytes, srcset, unfetched_type);}}}@@ -1360,6 +1458,11 @@ fn observe_mutation(visibility.update(attributes.bytes, &values);hidden = Some((id, was_hidden, visibility.hides()));}+ if let (Media::PictureSource { unfetched_type }, Some(_)) =+ (&mut *known, values.source_type)+ {+ *unfetched_type = is_unfetched_type(attributes.bytes, values.source_type);+ }media = Some(*known);}match hidden {@@ -1915,6 +2018,10 @@ mod tests {] {assert_eq!(limit(&mut urls.into_iter()), expected);}+ // Srcset values share the byte budget, past which the writer falls back to replacing each URL.+ let mut remote = RemoteUrls::default();+ (0..=most_long).for_each(|i| remote.insert_srcset(&url(i, longest), false));+ assert_eq!(remote.take_srcsets().len(), most_long);// A mutation past the attribute cap is not read to its end.let attributes = |count: usize| {diff --git a/rust/ai-research/replay-core-v4/src/inspect/session.rs b/rust/ai-research/replay-core-v4/src/inspect/session.rsindex f177c7bc0a11..436326e4c446 100644--- a/rust/ai-research/replay-core-v4/src/inspect/session.rs+++ b/rust/ai-research/replay-core-v4/src/inspect/session.rs@@ -3,7 +3,7 @@use std::sync::Arc;use super::clicks::{ClickKind, ClickTracker};-use super::dom::{DomInspector, RemoteUrls, UrlLimit};+use super::dom::{DomInspector, RemoteUrls, Srcset, UrlLimit};use super::events::{fields, number, Activity, EventCounter};use super::rrweb::{is_click, CUSTOM, FULL_SNAPSHOT, INCREMENTAL_SNAPSHOT, META, MOUSE_INTERACTION, MUTATION,@@ -62,15 +62,18 @@ pub struct Inspected {pub remote_urls: Vec<String>,/// Set when `remote_urls` stops short, so that the block must not be stored.pub url_limit: Option<UrlLimit>,+ /// The image-slot srcset values of the block, which the writer rebuilds from the candidates it keeps.+ pub srcsets: Vec<Srcset>,}impl BlockInspection {- pub fn finish(self) -> Inspected {+ pub fn finish(mut self) -> Inspected {Inspected {stats: self.counter.finish(),moments: self.moments,image_urls: self.image_urls,url_limit: self.remote_urls.limit,+ srcsets: self.remote_urls.take_srcsets(),remote_urls: self.remote_urls.into_urls(),}}diff --git a/rust/ai-research/replay-core-v4/src/url_policy.rs b/rust/ai-research/replay-core-v4/src/url_policy.rsindex 870675ad1539..f39f95ba33fc 100644--- a/rust/ai-research/replay-core-v4/src/url_policy.rs+++ b/rust/ai-research/replay-core-v4/src/url_policy.rs@@ -346,6 +346,33 @@ pub fn is_protocol_relative(raw: &str) -> bool {matches!(raw.trim().as_bytes(), [b'/' | b'\\', b'/' | b'\\', ..])}+/// Whether the browser loads the URL from a host. A URL scheme is not case sensitive, so `HTTPS://` loads as+/// `https://` does. A browser reads any run of slashes and backslashes after `http:` or `https:` as `//`, and none at+/// all on a page of another scheme, so `https:\\host` and `https:host` can load from the host too. A+/// protocol-relative URL does as well.+pub fn is_remote(url: &str) -> bool {+ let after_scheme = ["https:", "http:"].iter().find_map(|scheme| {+ url.get(..scheme.len())+ .filter(|start| start.eq_ignore_ascii_case(scheme))+ .map(|_| &url[scheme.len()..])+ });+ match after_scheme {+ Some(rest) => !rest.is_empty(),+ None => is_protocol_relative(url),+ }+}++/// The media types the image fetcher stores, the `ALLOWED_CONTENT_TYPES` of `image-fetch-v4/image-fetcher.ts`.+pub const FETCHED_IMAGE_TYPES: [&str; 4] = ["image/png", "image/jpeg", "image/gif", "image/webp"];++/// Whether a `<source type>` names a type the fetcher stores, ignoring parameters such as `codecs`.+pub fn is_fetched_image_type(source_type: &str) -> bool {+ let essence = source_type.split(';').next().unwrap_or_default().trim();+ FETCHED_IMAGE_TYPES+ .iter()+ .any(|fetched| fetched.eq_ignore_ascii_case(essence))+}+pub fn try_canonicalize(raw: &str) -> Result<CanonicalUrl, Decline> {if raw.len() > MAX_URL_LEN {return Err(Decline::TooLong);diff --git a/rust/ai-research/replay-recorder-v4/src/batch.rs b/rust/ai-research/replay-recorder-v4/src/batch.rsindex 995d6c137a91..dab9ac95a210 100644--- a/rust/ai-research/replay-recorder-v4/src/batch.rs+++ b/rust/ai-research/replay-recorder-v4/src/batch.rs@@ -8,7 +8,7 @@ use std::time::{Duration, Instant};use futures::stream::{self, StreamExt};use metrics::counter;use replay_core_v4::compression::{GZIP_MAGIC, MAX_DECOMPRESSED_BYTES};-use replay_core_v4::inspect::dom::UrlLimit;+use replay_core_v4::inspect::dom::{Srcset, UrlLimit};use replay_core_v4::inspect::session::{BlockInspection, Moment, SessionInspector};use replay_core_v4::inspect::stats::BlockStats;use replay_core_v4::scan::cv::{Expansion, MESSAGE_DECODE_BUDGET};@@ -96,6 +96,8 @@ pub struct Block {pub remote_urls: Vec<String>,/// Set when the inspector stopped short of naming every remote URL, so that the block must not be stored.pub url_limit: Option<UrlLimit>,+ /// The image-slot srcset values of the block, which the writer rebuilds from the candidates it keeps.+ pub srcsets: Vec<Srcset>,/// The earliest Kafka timestamp of the block's messages, for the capture watermark of the index topic.pub captured_at_ms: Option<i64>,}@@ -508,6 +510,7 @@ fn inspect_session(image_urls: inspected.image_urls,remote_urls: inspected.remote_urls,url_limit: inspected.url_limit,+ srcsets: inspected.srcsets,captured_at_ms,}})diff --git a/rust/ai-research/replay-recorder-v4/src/consumer.rs b/rust/ai-research/replay-recorder-v4/src/consumer.rsindex e117d139674e..2c93ed8d8736 100644--- a/rust/ai-research/replay-recorder-v4/src/consumer.rs+++ b/rust/ai-research/replay-recorder-v4/src/consumer.rs@@ -841,6 +841,7 @@ mod tests {image_urls: Vec::new(),remote_urls: Vec::new(),url_limit: None,+ srcsets: Vec::new(),captured_at_ms: None,})};diff --git a/rust/ai-research/replay-recorder-v4/src/rewrite.rs b/rust/ai-research/replay-recorder-v4/src/rewrite.rsindex 70a4485d61c0..0830e9c75599 100644--- a/rust/ai-research/replay-recorder-v4/src/rewrite.rs+++ b/rust/ai-research/replay-recorder-v4/src/rewrite.rs@@ -1,13 +1,21 @@//! Replaces the remote image URLs of a stored block with their `imageurl:v4` references, as v3 replaced them. A URL the//! policy declines becomes a marker that names the reason, so that no credential, signature or private host stays in//! the block. The inspector names the URLs: those in image slots, CSS included.+//!+//! A srcset is rebuilt instead: a declined candidate leaves the list, and so does each remote candidate of a picture+//! source whose type the fetcher does not store. An inline image, such as an SVG data URI, stays. A browser skips a+//! source with no candidates, so it falls back to a source or an img that can load, where a marker in the list would+//! show a broken image.use std::borrow::Cow;+use std::collections::HashMap;use aho_corasick::{AhoCorasick, AhoCorasickKind, MatchKind};use metrics::counter;use replay_core_v4::bytescan::{unescape, write_json_string};+use replay_core_v4::inspect::dom::{srcset_candidates, Srcset};use replay_core_v4::seal::image_url_key;+use replay_core_v4::url_policy::is_remote;use crate::urls::url_reference;@@ -104,11 +112,16 @@ fn ends_with_escape(bytes: &[u8], at: usize) -> bool {short || unicode}+const SRCSET_KEY: &[u8] = br#""srcset":"#;+struct Target {replacement: Vec<u8>,/// The policy's reason, for a declined URL.declined: Option<&'static str>,found: bool,+ /// For a srcset, which is a whole JSON string with its quotes: how it was rebuilt, and the URL targets its value+ /// holds. Those URLs leave the block with the old value, so a match counts as theirs.+ srcset: Option<(&'static str, Vec<usize>)>,}pub struct UrlRewrite {@@ -123,10 +136,16 @@ pub struct UrlRewrite {impl UrlRewrite {/// `team_key` is the team's month key for `key_month`, the write month, as for the fetch jobs.- pub fn new(urls: &[String], team_id: i64, key_month: &str, team_key: &[u8; 32]) -> Self {+ pub fn new(+ urls: &[String],+ srcsets: &[Srcset],+ team_id: i64,+ key_month: &str,+ team_key: &[u8; 32],+ ) -> Self {let url_key = image_url_key(team_key);- let mut patterns = Vec::with_capacity(urls.len());- let mut targets = Vec::with_capacity(urls.len());+ let mut patterns = Vec::with_capacity(urls.len() + srcsets.len());+ let mut targets = Vec::with_capacity(urls.len() + srcsets.len());for url in urls {// The block holds the URL as a JSON string holds it.let mut escaped = Vec::with_capacity(url.len() + 2);@@ -140,6 +159,31 @@ impl UrlRewrite {replacement: replacement.into_bytes(),declined,found: false,+ srcset: None,+ });+ }+ let url_targets: HashMap<&str, usize> = urls+ .iter()+ .enumerate()+ .map(|(index, url)| (url.as_str(), index))+ .collect();+ for srcset in srcsets {+ let Some((value, outcome, covers)) =+ rebuilt_srcset(srcset, &url_targets, team_id, key_month, &url_key)+ else {+ continue;+ };+ // The key keeps the pattern to srcset attributes, so a `src` or `href` with the same value stays.+ let mut pattern = SRCSET_KEY.to_vec();+ write_json_string(&srcset.text, &mut pattern);+ patterns.push(pattern);+ let mut replacement = SRCSET_KEY.to_vec();+ write_json_string(&value, &mut replacement);+ targets.push(Target {+ replacement,+ declined: None,+ found: false,+ srcset: Some((outcome, covers)),});}let matcher = (!patterns.is_empty())@@ -183,18 +227,27 @@ impl UrlRewrite {let mut copied_to = 0;for found in matcher.find_iter(bytes) {let (start, end) = (found.start(), found.end());- if start.checked_sub(1).is_some_and(|before| {- continues_url(bytes[before]) && !ends_with_escape(bytes, start)- }) || bytes.get(end).is_some_and(|&after| continues_url(after))+ let index = found.pattern().as_usize();+ // A srcset pattern holds its quotes, so it matches only a whole JSON string.+ let whole_string = self.targets[index].srcset.is_some();+ if !whole_string+ && (start.checked_sub(1).is_some_and(|before| {+ continues_url(bytes[before]) && !ends_with_escape(bytes, start)+ }) || bytes.get(end).is_some_and(|&after| continues_url(after))){continue;}- let target = &mut self.targets[found.pattern().as_usize()];+ let target = &mut self.targets[index];target.found = true;let out = out.get_or_insert_with(|| Vec::with_capacity(bytes.len()));out.extend_from_slice(&bytes[copied_to..start]);out.extend_from_slice(&target.replacement);copied_to = end;+ if let Some((_, covers)) = self.targets[index].srcset.clone() {+ for url in covers {+ self.targets[url].found = true;+ }+ }}let out = match out {Some(mut out) => {@@ -215,6 +268,14 @@ impl UrlRewrite {pub fn finish(self) -> bool {let mut safe = !self.declined_kept;for target in self.targets {+ if let Some((outcome, _)) = target.srcset {+ counter!(+ "ai_research_v4_recorder_srcset_rewrites_total",+ "outcome" => if target.found { outcome } else { "unrewritten" },+ )+ .increment(1);+ continue;+ }safe &= target.found || target.declined.is_none();let outcome = match (target.found, target.declined) {(false, _) => "unrewritten",@@ -232,6 +293,50 @@ impl UrlRewrite {}}+/// The srcset with only the candidates the block keeps, each remote URL as its reference, how it was rebuilt, and the+/// URL targets of the remote URLs it held. `None` when every candidate stays, because the URL targets then replace it.+fn rebuilt_srcset(+ srcset: &Srcset,+ url_targets: &HashMap<&str, usize>,+ team_id: i64,+ key_month: &str,+ url_key: &[u8; 32],+) -> Option<(String, &'static str, Vec<usize>)> {+ let mut kept = Vec::new();+ let mut covers = Vec::new();+ let mut removed = false;+ for (url, descriptors) in srcset_candidates(&srcset.text) {+ if !is_remote(url) {+ kept.push((Cow::Borrowed(url), descriptors));+ continue;+ }+ covers.extend(url_targets.get(url));+ match url_reference(url, team_id, key_month, url_key) {+ Ok((_, reference)) if !srcset.unfetched_type => {+ kept.push((Cow::Owned(reference), descriptors))+ }+ _ => removed = true,+ }+ }+ if !removed {+ return None;+ }+ let outcome = if kept.is_empty() {+ "emptied"+ } else {+ "trimmed"+ };+ let value = kept+ .iter()+ .map(|(url, descriptors)| match *descriptors {+ "" => url.to_string(),+ descriptors => format!("{url} {descriptors}"),+ })+ .collect::<Vec<_>>()+ .join(", ");+ Some((value, outcome, covers))+}+#[cfg(test)]mod tests {use super::*;@@ -240,7 +345,7 @@ mod tests {fn rewrite(urls: &[&str], bytes: &str) -> String {let urls: Vec<String> = urls.iter().map(|u| (*u).to_owned()).collect();- let mut rewrite = UrlRewrite::new(&urls, 7, "2026-10", &TEAM_KEY);+ let mut rewrite = UrlRewrite::new(&urls, &[], 7, "2026-10", &TEAM_KEY);String::from_utf8(rewrite.apply(bytes.as_bytes()).into_owned()).unwrap()}@@ -274,6 +379,7 @@ mod tests {let event = format!(r#"{{"srcset":"{a} 1x,\n{b} 2x,\t{declined} 3x"}}"#);let mut rewrite = UrlRewrite::new(&[a.to_owned(), b.to_owned(), declined.to_owned()],+ &[],7,"2026-10",&TEAM_KEY,@@ -293,7 +399,7 @@ mod tests {let url = "https://cdn.example.com/a.png?token=secret";let hidden = r#"{"src":"https:\/\/cdn.example.com\/a.png?token=secr\u0065t","alt":"a\nb"}"#;let canonical = canonical_strings(hidden.as_bytes());- let mut rewrite = UrlRewrite::new(&[url.to_owned()], 7, "2026-10", &TEAM_KEY);+ let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);assert_eq!(String::from_utf8(rewrite.apply(&canonical).into_owned()).unwrap(),format!(@@ -309,7 +415,7 @@ mod tests {Cow::Borrowed(_)));// A declined URL the block still holds fails the block.- let rewrite = UrlRewrite::new(&[url.to_owned()], 7, "2026-10", &TEAM_KEY);+ let rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);assert!(!rewrite.finish());}@@ -336,7 +442,7 @@ mod tests {}// A second copy inside a longer URL stays as it is, so the block fails.let url = "https://bucket.s3.amazonaws.com/a.png?X-Amz-Signature=abc123";- let mut rewrite = UrlRewrite::new(&[url.to_owned()], 7, "2026-10", &TEAM_KEY);+ let mut rewrite = UrlRewrite::new(&[url.to_owned()], &[], 7, "2026-10", &TEAM_KEY);rewrite.apply(format!(r#"{{"src":"{url}","alt":"x{url}"}}"#).as_bytes());assert!(!rewrite.finish());}diff --git a/rust/ai-research/replay-recorder-v4/src/urls.rs b/rust/ai-research/replay-recorder-v4/src/urls.rsindex 99143d4bd782..d014a5a7ba10 100644--- a/rust/ai-research/replay-recorder-v4/src/urls.rs+++ b/rust/ai-research/replay-recorder-v4/src/urls.rs@@ -207,6 +207,7 @@ mod tests {image_urls: urls.iter().map(|u| (*u).to_owned()).collect(),remote_urls: Vec::new(),url_limit: None,+ srcsets: Vec::new(),captured_at_ms: None,};let mut stats = BlockStats::default();diff --git a/rust/ai-research/replay-recorder-v4/src/write.rs b/rust/ai-research/replay-recorder-v4/src/write.rsindex b8c7b1a6319f..451a16ce382b 100644--- a/rust/ai-research/replay-recorder-v4/src/write.rs+++ b/rust/ai-research/replay-recorder-v4/src/write.rs@@ -127,6 +127,7 @@ pub fn prepare_block(let content_key = image_content_key(team_key);let mut rewrite = UrlRewrite::new(&block.remote_urls,+ &block.srcsets,block.session.team_id,key_month,team_key,@@ -683,6 +684,7 @@ mod tests {image_urls: Vec::new(),remote_urls: Vec::new(),url_limit: None,+ srcsets: Vec::new(),captured_at_ms: None,}}@@ -702,6 +704,7 @@ mod tests {let found = inspection.finish();inspected.remote_urls = found.remote_urls;inspected.url_limit = found.url_limit;+ inspected.srcsets = found.srcsets;inspected.remote_urls.extend(extra.iter().map(|url| (*url).to_owned()));@@ -793,6 +796,82 @@ mod tests {assert!(!stored_lines(&event, &[public]).is_empty());}+ #[test]+ fn leaves_a_picture_only_the_sources_that_can_load() {+ let (avif, webp, signed, fallback) = (+ "https://cdn.example.com/a.avif",+ "https://cdn.example.com/a.webp",+ "https://bucket.s3.amazonaws.com/a.webp?X-Amz-Signature=abc123",+ "https://cdn.example.com/a.png",+ );+ let source = |id: i64, attributes: serde_json::Value| json!({"type": 2, "tagName": "source", "attributes": attributes, "childNodes": [], "id": id});+ // The scan leaves an SVG inline, so the browser can load it from the block.+ let (bare, typeless) = (+ "https://cdn.example.com/b.avif",+ "https://cdn.example.com/c.webp",+ );+ let svg =+ "data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%2F%3E";+ let picture = json!({"type": 2, "tagName": "picture", "attributes": {}, "id": 2, "childNodes": [+ source(6, json!({"type": "image/svg+xml", "srcset": format!("{svg} 1x")})),+ source(3, json!({"type": "image/avif", "srcset": format!("{avif} 1x")})),+ // A bare srcset equal to the img's src below: only the srcset empties.+ source(8, json!({"type": "image/avif", "srcset": bare})),+ // An empty type gives no hint, so the source keeps its candidate.+ source(9, json!({"type": "", "srcset": format!("{typeless} 1x")})),+ source(4, json!({"type": "image/webp", "srcset": format!("{signed} 2x, {webp} 1x")})),+ {"type": 2, "tagName": "img", "attributes": {"src": fallback}, "childNodes": [], "id": 5},+ ]});+ let img = json!({"type": 2, "tagName": "img", "attributes": {"src": bare}, "childNodes": [], "id": 10});+ let events = json!([{"type": 2, "timestamp": 1, "data": {"node": {"type": 0, "id": 1, "childNodes": [picture, img]}}}]).to_string();++ let block = block_of("kept", &events);+ let mut inspector = replay_core_v4::inspect::session::SessionInspector::default();+ let mut inspection = replay_core_v4::inspect::session::BlockInspection::default();+ for message in &block.messages {+ inspector.observe_message(&mut inspection, message);+ }+ // The fetcher stores no AVIF, so the AVIF source gets no fetch job.+ assert!(!inspection.finish().image_urls.iter().any(|u| u == avif));++ let lines = stored_lines(&events, &[]);+ let reference = |url: &str| {+ crate::urls::url_reference(+ url,+ 7,+ "2026-10",+ &replay_core_v4::seal::image_url_key(&[6; 32]),+ )+ .unwrap()+ .1+ };+ // The AVIF source has no candidate left and the declined one leaves the list, so the browser takes the WebP.+ assert!(lines.contains(r#""srcset":"""#), "{lines}");+ assert!(+ lines.contains(&format!(r#""srcset":"{svg} 1x""#)),+ "{lines}"+ );+ assert!(+ lines.contains(&format!(r#""srcset":"{} 1x""#, reference(webp))),+ "{lines}"+ );+ assert!(lines.contains(&reference(fallback)), "{lines}");+ assert!(+ lines.contains(&format!(r#""src":"{}""#, reference(bare))),+ "{lines}"+ );+ assert!(+ lines.contains(&format!(r#""srcset":"{} 1x""#, reference(typeless))),+ "{lines}"+ );+ assert!(+ !lines.contains("a.avif")+ && !lines.contains("X-Amz-Signature")+ && !lines.contains("declined"),+ "{lines}"+ );+ }+#[test]fn keeps_the_sealed_urls_of_a_row_under_the_message_limit() {let page = |i: usize| format!("https://app.example.com/{i}/{}", "p".repeat(1000));