:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(ai-research): fetch one v4 srcset candidate, the one shown at 1x (#127233)
Robbiefc7529c58ce7
diff --git a/products/ai_training/docs/replay-data.md b/products/ai_training/docs/replay-data.mdindex 15b5b16328d3..f3b35a9611bc 100644--- a/products/ai_training/docs/replay-data.md+++ b/products/ai_training/docs/replay-data.md@@ -473,7 +473,15 @@ The recorder streams each block into the encoder. Read a block as a stream: a Br- A URL that the URL policy refuses becomes `imageurl:v4:declined:{reason}` in the block, for example `declined:credential`, `declined:non_public_host` or `declined:tracking_beacon`. The original URL cannot be recovered. The recorder counts rewrites and refusals in `ai_research_v4_recorder_image_url_rewrites_total`.- The `v4_image_refs` module of the `posthog-replay-anonymizer` crate defines each reference and marker shape. The recorder writes them with it. A reader that must tell them apart from page text uses its `parse` function, which accepts only the exact strings the recorder writes.- The rewrite covers every remote URL that the recorder reads from an image slot: image sources, `srcset`, SVG image links, video posters, `<picture>` sources and CSS `url(...)` in style attributes, style sheets and style rules, in custom properties too, because a page passes a custom property to an image property through `var()`. It also covers `src`, `srcset`, `href` and `poster` in an attribute change on a node the recorder does not know, for example after a restart. `srcset` keeps its descriptors and CSS keeps its `url(...)` syntax. A URL is also replaced where a selector holds it with CSSOM's escapes, by the reference with the same escapes, so a class such as `bg-[url(https://cdn.example.com/a.png)]` and its rule `.bg-\[url\(https\:\/\/cdn\.example\.com\/a\.png\)\]` still match within a block. A reader that replaces references by text must replace the escaped form as well. It is not limited by the 512 URLs per message that get fetch jobs.-- A srcset in an image slot keeps only the candidates the recorder stores. A candidate the URL policy refuses leaves the list instead of becoming a `declined` marker, and the rest keep their descriptors. A `<picture>` source whose `type` names a media type the image fetcher does not store, such as `image/avif`, gets no fetch job and loses its remote candidates, unless a source of a stored type names the same URL. Its inline images, such as SVG data URIs, stay. The writer rebuilds each srcset that the inspector read in an image slot, in the bytes it stores, after the inline images left and the escapes were written again. It separates the candidates with a comma and a space. A `srcset` key elsewhere, for example in a custom event's payload, stays as it is. A browser skips a source with no candidates, so the replay falls back to the next source or the `img`. The recorder counts these as `ai_research_v4_recorder_srcset_rewrites_total{outcome="trimmed"|"emptied"}`.+- A srcset in an image slot keeps one candidate: the one that Chromium shows at a device pixel ratio of 1, which is the ratio the screenshotter renders at. Of `x` descriptors, that is the smallest density at or above 1x, or the largest density when all are below 1x. A `w` descriptor gives a density only against the layout width, which the recorder does not know, so of `w` descriptors the recorder keeps the v3 choice: the largest at or below 1024w, or the smallest when all are wider. The first candidate wins a tie. Only that candidate gets a fetch job, because the renderer has no network access and shows only fetched images. A srcset that mixes `w` and `x` descriptors, or that v3 would read as malformed, keeps no candidate.+- The kept candidate stays only when it can show offline: a URL that the URL policy accepts, or an inline image that the image scan keeps. Otherwise the srcset is empty, and no other candidate takes its place, because no other candidate has a fetch job. A candidate that leaves becomes no `declined` marker.+- HTML adds the non-empty `src` of an `img` as a 1x candidate when its srcset has no 1x candidate and no `w` descriptor. At a device pixel ratio of 1 the `img` then shows its `src`, so the recorder fetches the `src` and empties the srcset.+- A visible `img` that shows its kept candidate gets no fetch job for its `src` or `rr_src`, as in v3. They still become references. Any other `img` with a `src` gets a fetch job for it. The recorder decides this on the srcset and the `src` as the event holds them, before the inline images leave, and keeps the same candidate in the stored block.+- An attribute change that sets the srcset but not the `src` does not hold the `src`, so the recorder decides as for an `img` with no `src`. If the `img` has a `src` that wins over the new srcset, the replay shows that `src`, which has a fetch job only if an earlier event gave it one.+- When an attribute change shows a hidden `img`, the recorder decides this on the srcset and the `src` after the change.+- The image scan can still decline an inline kept candidate when its message is past the caps. The srcset is then empty, and the `img` shows no image.+- An inline image in a candidate that leaves still moves to the image pack, where no reference names it.+- A `<picture>` source whose `type` names a media type the image fetcher does not store, such as `image/avif`, gets no fetch job and loses its remote candidates. Its kept candidate stays only when it is inline, such as an SVG data URI. Srcsets with the same remote URLs in the same order share one decision, so such a source keeps the candidate of a source of a stored type with the same remote URLs, and an `img` that shows its `src` keeps the candidate of an `img` that shows the candidate. That `img` still shows its `src`. The writer rebuilds each srcset that the inspector read in an image slot, in the bytes it stores, after the inline images left and the escapes were written again. A `srcset` key elsewhere, for example in a custom event's payload, stays as it is. A browser skips a source with no candidates, so the replay falls back to the next source or the `img`. The recorder counts these as `ai_research_v4_recorder_srcset_rewrites_total{outcome="trimmed"|"emptied"}`.- The writer rewrites CSS text before it replaces the URLs, so that an offline render loads only images the block refers to. An `image-set()` keeps one candidate, because a renderer picks the candidate that fits its pixel ratio and does not fall back when that candidate fails. It keeps the highest resolution of a type the image fetcher stores among the candidates that can load: inline, accepted by the URL policy, or root-relative on a known page. Only that candidate gets a fetch job, and the other remote candidates get references without one, in case the writer does not find the set's text. A set that holds an inline image keeps all its candidates, because the writer moves the inline image out first.- A root-relative CSS `url(/…)` resolves against the origin of its window's latest page URL, from a Meta event or a custom event that names a page, such as `$url_changed`, with `https` in place of the page's scheme. It gets the reference of the absolute URL, or a `declined` marker. A path-relative `url(…)` stays as the page wrote it, so that no part of a page path, which can hold a token, reaches a fetch. rrweb writes snapshot CSS with absolute URLs, but sends rule text, adopted style sheets and style declarations as the page wrote them. A text that resolves two ways in one block, as in windows on two origins, or that one window resolves and another cannot, keeps no rewrite. An event resolves at most 1,024 relative URLs, a flush keeps at most 64 MiB of resolved URLs, and a page URL longer than 2,048 characters resolves none.- The writer rewrites only whole `url()` and `image-set()` functions, found as the inspector finds them, inside JSON strings that hold CSS: the values of `_cssText`, `style` (style diffs included), `textContent`, `rule`, `replace`, `replaceSync` and `value`. So a class name such as `bg-[url(/a.png)]`, a quoted string or a comment keeps its text, and the class's rule still matches. A text node that shows the same function text gets the rewrite too. The recorder counts these as `ai_research_v4_recorder_css_rewrites_total{kind="image_set"|"relative_url", outcome="rewritten"|"unfound"}`.@@ -656,7 +664,7 @@ Some limits apply:### Remote imagesThe recorder sends each remote image URL that the URL policy accepts to the `ai_research_v4_image_fetch` topic in the v3 job shape, keyed by registrable domain. The job carries the reference that replaces the URL in the block.-It collects URLs from image sources, `srcset`, SVG image links, CSS `url(...)` values and attribute mutations, and skips hidden or 1×1 images as v3 does. Unlike v3, it also collects CSS custom properties and relative CSS URLs, and fetches one `image-set()` candidate of a stored type.+It collects URLs from image sources, the kept candidate of each `srcset`, SVG image links, CSS `url(...)` values and attribute mutations, and skips hidden or 1×1 images as v3 does. Unlike v3, it also collects CSS custom properties and relative CSS URLs, and fetches one `image-set()` candidate of a stored type.The topics hold jobs for 24 hours, because jobs carry raw URLs.The v4 image fetcher is its own copy of the v3 fetcher under `ai-research/image-fetch-v4/`, with the v3 fetch policy, its own consumer groups, its own `AI_RESEARCH_V4_IMAGE_FETCH_*` settings and a share of the v3 per-host limits. It shares no code or table with v3.It applies the recorder's URL policy through `@posthog/ai-research-v4-native`, an addon over `replay_core_v4::url_policy`, so it canonicalizes a URL, groups it by registrable domain and refuses private hosts exactly as the recorder does.diff --git a/rust/ai-research/replay-core-v4/src/inspect/dom.rs b/rust/ai-research/replay-core-v4/src/inspect/dom.rsindex 8cf5689af0dc..fd93af338d1c 100644--- a/rust/ai-research/replay-core-v4/src/inspect/dom.rs+++ b/rust/ai-research/replay-core-v4/src/inspect/dom.rs@@ -17,6 +17,7 @@ use super::rrweb::{ADOPTED_STYLE_SHEET, FULL_SNAPSHOT, INCREMENTAL_SNAPSHOT, MUTATION, STYLE_DECLARATION,STYLE_SHEET_RULE,};+use super::srcset::{self, SrcsetSlot};use super::stats::BlockStats;use crate::scan::cv::{self, Expansion};use crate::scan::message::Event;@@ -97,7 +98,10 @@ enum Media {struct HiddenSources {src: Option<String>,rr_src: Option<String>,- srcset: Vec<String>,+ fetchable_srcset_candidate: Option<String>,+ srcset_usable: bool,+ srcset_yields_to_img_src: bool,+ has_src: bool,}impl HiddenSources {@@ -112,50 +116,55 @@ impl HiddenSources {|span: Span| text(bytes, Some(span)).and_then(|url| fetchable(&url).map(str::to_owned));if let Some(span) = attributes.src {self.src = url(span);+ self.has_src = is_non_empty_string(bytes, span);}if let Some(span) = attributes.rr_src {self.rr_src = url(span);}if let Some(span) = attributes.srcset {- self.srcset.clear();- if let Some(srcset) = text(bytes, Some(span)) {- // One message sends at most MAX_IMAGE_URLS URLs, so the img that shows needs no more.- srcset_urls(&srcset, &mut |url| {- if self.srcset.len() < MAX_IMAGE_URLS {- self.srcset.extend(fetchable(url).map(str::to_owned));- }- });- }+ let srcset = text(bytes, Some(span));+ let usable = srcset.as_deref().and_then(srcset::usable_candidate);+ self.srcset_usable = usable.is_some();+ self.srcset_yields_to_img_src = usable.is_some_and(|kept| kept.yields_to_img_src);+ self.fetchable_srcset_candidate =+ usable.and_then(|kept| fetchable(kept.url).map(str::to_owned));}}+ /// As [`srcset::shown_candidate`] decides, from what the window kept of the srcset and the `src`.+ fn shows_srcset(&self) -> bool {+ self.srcset_usable && !(self.has_src && self.srcset_yields_to_img_src)+ }+/// Whether the attributes set a source, which an update then changes.fn changes(attributes: &Attributes) -> bool {attributes.src.is_some() || attributes.rr_src.is_some() || attributes.srcset.is_some()}fn is_empty(&self) -> bool {- self.src.is_none() && self.rr_src.is_none() && self.srcset.is_empty()+ self.src.is_none() && self.rr_src.is_none() && self.fetchable_srcset_candidate.is_none()}fn heap_bytes(&self) -> usize {let string = |s: &Option<String>| s.as_ref().map_or(0, String::capacity);- string(&self.src)- + string(&self.rr_src)- + self.srcset.capacity() * std::mem::size_of::<String>()- + self.srcset.iter().map(String::capacity).sum::<usize>()+ string(&self.src) + string(&self.rr_src) + string(&self.fetchable_srcset_candidate)}- fn push_revealed(self, set: &Attributes, walk: &mut Walk) {- for (url, replaced) in [(self.src, set.src), (self.rr_src, set.rr_src)] {- if let Some(url) = url.filter(|_| replaced.is_none()) {- walk.push_url_text(&url);+ fn push_revealed(mut self, bytes: &[u8], set: &Attributes, walk: &mut Walk) {+ self.update(bytes, set);+ let shows_srcset = self.shows_srcset();+ if !shows_srcset {+ for (url, replaced) in [(self.src, set.src), (self.rr_src, set.rr_src)] {+ if let Some(url) = url.filter(|_| replaced.is_none()) {+ walk.push_url_text(&url);+ }}}- if set.srcset.is_none() {- for url in &self.srcset {- walk.push_url_text(url);- }+ if let Some(url) = self+ .fetchable_srcset_candidate+ .filter(|_| shows_srcset && set.srcset.is_none())+ {+ walk.push_url_text(&url);}}}@@ -209,10 +218,9 @@ pub struct RemoteUrls {urls: HashSet<String>,bytes: usize,pub limit: Option<UrlLimit>,- /// Each image-slot srcset value, and whether it belongs to a source of a type the fetcher does not store. The- /// writer rebuilds each from the candidates it keeps, and it builds one matcher over them as over `urls`, so they- /// share the byte budget of `urls`. Past it this keeps no more, which is safe: the writer still replaces each URL- /// in `urls` on its own.+ /// Each image-slot srcset value, and the slot that holds it. The writer rebuilds each from the candidates it keeps,+ /// and it builds one matcher over them as over `urls`, so they share the byte budget of `urls`. Past it this keeps no+ /// more, which is safe: the writer still replaces each URL in `urls` on its own./// Boxed, because the session state counts this struct's size and most blocks hold no srcset.srcsets: Option<Box<SrcsetValues>>,/// Past its count and byte budget this keeps no more, which is safe: each URL a rewrite drops is in `urls` too, and a@@ -228,7 +236,7 @@ struct CssRewrites {#[derive(Debug, Default)]struct SrcsetValues {- values: HashMap<Vec<u8>, (String, bool)>,+ values: HashMap<Vec<u8>, (String, Vec<SrcsetSlot>)>,bytes: usize,}@@ -238,8 +246,7 @@ pub struct Srcset {pub text: String,/// The value as a JSON string with its quotes and escapes, as the event holds it.pub raw: Vec<u8>,- /// The value belongs to a `<source>` whose `type` the fetcher does not store, so none of its images is stored.- pub unfetched_type: bool,+ pub slot: SrcsetSlot,}impl RemoteUrls {@@ -260,19 +267,22 @@ impl RemoteUrls {};}- /// A value seen on a source of a stored type and on one of another type keeps its candidates.- /// Keyed by the value as the event holds it, so the writer finds that value again after it stores the event.- fn insert_srcset(&mut self, raw: &[u8], text: &str, unfetched_type: bool) {+ /// A value in several kinds of slot keeps each kind, so that the writer keeps the candidates that some slot shows+ /// and no other. Keyed by the value as the event holds it, so the writer finds that value again after it stores the+ /// event.+ fn insert_srcset(&mut self, raw: &[u8], text: &str, slot: SrcsetSlot) {let srcsets = self.srcsets.get_or_insert_with(Box::default);- if let Some((_, known)) = srcsets.values.get_mut(raw) {- *known &= unfetched_type;+ if let Some((_, slots)) = srcsets.values.get_mut(raw) {+ if !slots.contains(&slot) {+ slots.push(slot);+ }} else if srcsets.values.len() < MAX_BLOCK_REMOTE_URLS&& srcsets.bytes + raw.len() + text.len() <= MAX_BLOCK_REMOTE_URL_BYTES{srcsets.bytes += raw.len() + text.len();srcsets.values- .insert(raw.to_vec(), (text.to_owned(), unfetched_type));+ .insert(raw.to_vec(), (text.to_owned(), vec![slot]));}}@@ -308,8 +318,10 @@ impl RemoteUrls {self.insert(url);}self.limit = self.limit.or(other.limit);- for (raw, (text, unfetched_type)) in other.srcsets.iter().flat_map(|s| &s.values) {- self.insert_srcset(raw, text, *unfetched_type);+ for (raw, (text, slots)) in other.srcsets.iter().flat_map(|s| &s.values) {+ for &slot in slots {+ self.insert_srcset(raw, text, slot);+ }}for (text, rewrite) in other.css_rewrites.into_iter().flat_map(|r| r.values) {self.insert_css_rewrite(text, rewrite);@@ -332,10 +344,12 @@ impl RemoteUrls {.map(|srcsets| srcsets.values).unwrap_or_default().into_iter()- .map(|(raw, (text, unfetched_type))| Srcset {- text,- raw,- unfetched_type,+ .flat_map(|(raw, (text, slots))| {+ slots.into_iter().map(move |slot| Srcset {+ text: text.clone(),+ raw: raw.clone(),+ slot,+ })}).collect()}@@ -484,29 +498,38 @@ impl Walk<'_> {}}- fn push_srcset(&mut self, bytes: &[u8], value: Option<Span>) {- if let Some(srcset) = text(bytes, value) {- srcset_urls(&srcset, &mut |url| self.push_url_text(url));+ /// Only the shown candidate gets a fetch job. The block holds the others until the writer drops them, so the writer+ /// needs their URLs too. Returns whether the slot shows a srcset candidate.+ fn push_srcset(&mut self, bytes: &[u8], value: Option<Span>, slot: SrcsetSlot) -> bool {+ let Some(srcset) = text(bytes, value) else {+ return false;+ };+ srcset_urls(&srcset, &mut |url| self.image_urls.push_remote(url));+ let shown = srcset::shown_candidate(&srcset, slot);+ if let Some(kept) = shown {+ self.push_url_text(kept.url);}+ shown.is_some()}- fn push_srcset_value(&mut self, bytes: &[u8], value: Option<Span>, unfetched_type: bool) {+ fn push_srcset_value(&mut self, bytes: &[u8], value: Option<Span>, slot: SrcsetSlot) {if let (Some(span), Some(srcset)) = (value, text(bytes, value)) {self.image_urls.remote- .insert_srcset(&bytes[span.0..span.1], &srcset, unfetched_type);+ .insert_srcset(&bytes[span.0..span.1], &srcset, slot);}}/// A picture source of a type the fetcher does not store gets no fetch job, and the writer empties its srcset, so/// the browser falls back to the next source or the img.fn push_source(&mut self, bytes: &[u8], srcset: Option<Span>, unfetched_type: bool) {- if unfetched_type {- self.push_remote(bytes, srcset, true);+ let slot = if unfetched_type {+ SrcsetSlot::UnfetchedTypeSource} else {- self.push_srcset(bytes, srcset);- }- self.push_srcset_value(bytes, srcset, unfetched_type);+ SrcsetSlot::Alone+ };+ self.push_srcset(bytes, srcset, slot);+ self.push_srcset_value(bytes, srcset, slot);}fn push_css(&mut self, bytes: &[u8], value: Option<Span>) {@@ -658,26 +681,43 @@ impl Attributes {walk.push_remote(bytes, self.src, false);walk.push_remote(bytes, self.rr_src, false);walk.push_remote(bytes, self.srcset, true);- walk.push_srcset_value(bytes, self.srcset, false);+ walk.push_srcset_value(bytes, self.srcset, SrcsetSlot::Alone);walk.push_remote(bytes, self.href, false);walk.push_remote(bytes, self.xlink_href, false);walk.push_remote(bytes, self.poster, false);}+ /// A mutation that does not set `src` leaves it unknown, so the img's srcset counts as alone.+ fn img_srcset_slot(&self, bytes: &[u8]) -> SrcsetSlot {+ if self+ .src+ .is_some_and(|span| is_non_empty_string(bytes, span))+ {+ SrcsetSlot::BesideImgSrc+ } else {+ SrcsetSlot::Alone+ }+ }+/// The image URLs v3 fetches from these attributes on an element of this kind, and the URLs in its CSS.fn push_urls(&self, bytes: &[u8], media: Option<Media>, walk: &mut Walk) {match media {Some(Media::Img { visibility }) if !visibility.hides() => {- walk.push_url(bytes, self.src);- walk.push_url(bytes, self.rr_src);- walk.push_srcset(bytes, self.srcset);- walk.push_srcset_value(bytes, self.srcset, false);+ let slot = self.img_srcset_slot(bytes);+ if walk.push_srcset(bytes, self.srcset, slot) {+ walk.push_remote(bytes, self.src, false);+ walk.push_remote(bytes, self.rr_src, false);+ } else {+ walk.push_url(bytes, self.src);+ walk.push_url(bytes, self.rr_src);+ }+ walk.push_srcset_value(bytes, self.srcset, slot);}Some(Media::Img { .. }) => {walk.push_remote(bytes, self.src, false);walk.push_remote(bytes, self.rr_src, false);walk.push_remote(bytes, self.srcset, true);- walk.push_srcset_value(bytes, self.srcset, false);+ walk.push_srcset_value(bytes, self.srcset, self.img_srcset_slot(bytes));}Some(Media::SvgImage) => {walk.push_url(bytes, self.href);@@ -694,6 +734,10 @@ impl Attributes {}}+fn is_non_empty_string(bytes: &[u8], span: Span) -> bool {+ scan::is_string(bytes, span) && span.1 - span.0 > 2+}+/// An empty `type` gives the browser no hint, as a missing one does.fn is_unfetched_type(bytes: &[u8], source_type: Option<Span>) -> bool {text(bytes, source_type).is_some_and(|t| !t.trim().is_empty() && !is_fetched_image_type(&t))@@ -1585,7 +1629,7 @@ fn observe_mutation(}Some((id, true, false)) => {if let Some(sources) = dom.set_hidden_sources(id, None) {- sources.push_revealed(&values, &mut walk);+ sources.push_revealed(attributes.bytes, &values, &mut walk);}}_ => {}@@ -1810,14 +1854,10 @@ mod tests {None,&mut stats,);- // Remote image URLs go to the fetcher. The inline image in the srcset is the image scan's to move.+ // The srcset has no 1x candidate, so the img shows its `src`, which HTML adds as one.assert_eq!(dom.take_image_urls(),- [- "https://cdn.example.com/a.png",- "https://cdn.example.com/[email protected]",- "//cdn.example.com/p.jpg"- ]+ ["https://cdn.example.com/a.png", "//cdn.example.com/p.jpg"]);assert_eq!(@@ -2018,8 +2058,11 @@ mod tests {element(17, "img", json!({"data-src": url("lazy")}), json!([])),element(18, "iframe", json!({"src": "https://example.com/frame"}), json!([])),element(19, "img", json!({"srcset": format!("{} 640w,{} 2x", url("w_640,q_auto/srcset-comma"), url("srcset-second"))}), json!([])),- element(20, "img", json!({"src": url("shown-later"), "srcset": format!("{} 2x", url("shown-later-2x")), "hidden": ""}), json!([])),+ element(20, "img", json!({"src": url("shown-later"), "srcset": format!("{} 1x, {} 2x", url("shown-later-1x"), url("shown-later-2x")), "hidden": ""}), json!([])),element(21, "img", json!({"src": format!("https://cdn.example.com/{}.png", "a".repeat(MAX_URL_LEN))}), json!([])),+ element(22, "img", json!({"src": url("srcset-removed"), "srcset": format!("{} 1x", url("removed-1x")), "hidden": ""}), json!([])),+ element(23, "img", json!({"src": url("src-over-2x"), "srcset": format!("{} 2x", url("yields-2x")), "hidden": ""}), json!([])),+ element(24, "img", json!({"src": url("src-under-1x"), "srcset": format!("{} 1x, {} 2x", url("srcset-1x"), url("srcset-2x"))}), json!([])),]))], "id": 1}});let message = events(json!([// URLs do not wait for the first full snapshot.@@ -2032,6 +2075,10 @@ mod tests {// An update that sets no source keeps the hidden img's sources.mutation(json!([]), json!([]), json!([]), json!([{"id": 20, "attributes": {"alt": "x"}}])),mutation(json!([]), json!([]), json!([]), json!([{"id": 20, "attributes": {"hidden": null}}])),+ // The srcset leaves as the img shows, so the img shows its `src`.+ mutation(json!([]), json!([]), json!([]), json!([{"id": 22, "attributes": {"hidden": null, "srcset": null}}])),+ // A srcset with no 1x candidate loses to the img's `src`, which HTML adds as one.+ mutation(json!([]), json!([]), json!([]), json!([{"id": 23, "attributes": {"hidden": null}}])),mutation(json!([]), json!([]), json!([]), json!([{"id": 6, "attributes": {"style": {"background-image": [format!("url({})", url("style-object")), "important"], "background": format!("url({})", url("style-object-plain")), "color": "red", "mask-image": false}}}])),mutation(json!([]), json!([]), json!([]), json!([{"id": 6, "attributes": {"style": {"background-image": "url(\"/img/relative-style.png\")"}}}])),// A `source` added on its own under a `picture`, one the page has or one the same mutation adds.@@ -2067,6 +2114,8 @@ mod tests {"shown-later","lazy","set-1x",+ "w_640,q_auto/srcset-comma",+ "srcset-second",] {assert!(remote.contains(&url(name)), "{name}");}@@ -2109,10 +2158,10 @@ mod tests {"declaration","custom-declaration","adopted",- "w_640,q_auto/srcset-comma",- "srcset-second",- "shown-later",- "shown-later-2x",+ "shown-later-1x",+ "srcset-removed",+ "src-over-2x",+ "srcset-1x","style-object","style-object-plain","lone-source",@@ -2131,10 +2180,10 @@ mod tests {}#[test]- fn caps_the_sources_a_hidden_img_keeps() {+ fn keeps_one_srcset_candidate_of_a_hidden_img() {let srcset = |count: usize| {(0..count)- .map(|i| format!("https://cdn.example.com/{i}.png {}w", i + 1))+ .map(|i| format!("https://cdn.example.com/{i:05}.png {}w", i + 1)).collect::<Vec<_>>().join(", ")};@@ -2146,8 +2195,7 @@ mod tests {);dom.retained_bytes()};- assert!(retained(1) < retained(MAX_IMAGE_URLS));- assert_eq!(retained(MAX_IMAGE_URLS), retained(10 * MAX_IMAGE_URLS));+ assert_eq!(retained(1), retained(10 * MAX_IMAGE_URLS));}#[test]@@ -2189,7 +2237,7 @@ mod tests {// A value counts its raw and its unescaped form.(0..=most_long).for_each(|i| {let value = url(i, longest);- remote.insert_srcset(value.as_bytes(), &value, false)+ remote.insert_srcset(value.as_bytes(), &value, SrcsetSlot::Alone)});assert_eq!(remote.take_srcsets().len(), most_long / 2);diff --git a/rust/ai-research/replay-core-v4/src/inspect/mod.rs b/rust/ai-research/replay-core-v4/src/inspect/mod.rsindex 2cdb26f1e535..a30406d8786d 100644--- a/rust/ai-research/replay-core-v4/src/inspect/mod.rs+++ b/rust/ai-research/replay-core-v4/src/inspect/mod.rs@@ -7,4 +7,5 @@ pub mod events;pub mod features;pub mod rrweb;pub mod session;+pub mod srcset;pub mod stats;diff --git a/rust/ai-research/replay-core-v4/src/inspect/srcset.rs b/rust/ai-research/replay-core-v4/src/inspect/srcset.rsnew file mode 100644index 000000000000..5c071bd2006e--- /dev/null+++ b/rust/ai-research/replay-core-v4/src/inspect/srcset.rs@@ -0,0 +1,331 @@+//! The one `srcset` candidate the recorder keeps. A renderer picks a candidate for its viewport and shows nothing offline+//! for one the fetcher did not take, and each fetched image costs a scrub, so a srcset keeps one candidate: the one the+//! screenshotter's Chromium shows, at the device pixel ratio of 1 that it renders at.++use crate::scan::images::keeps_inline_image;+use crate::url_policy::{is_remote, try_canonicalize};++const MAX_SRCSET_WIDTH: f64 = 1024.0;+const SCREENSHOT_DEVICE_PIXEL_RATIO: f64 = 1.0;++#[derive(Clone, Copy, PartialEq, Eq)]+enum DescriptorKind {+ Density,+ Width,+}++struct Candidate<'a> {+ index: usize,+ url: &'a str,+ kind: DescriptorKind,+ value: f64,+}++impl Candidate<'_> {+ /// Of density descriptors, Chromium's choice at the screenshot's device pixel ratio: the smallest density at or+ /// above it, or the largest when all are below it (`SelectionLogic` in Blink's `html_srcset_parser.cc`). A width+ /// gives a density only against the layout width, which the recorder does not know, so of width descriptors this+ /// keeps the v3 choice (`rust/replay-anonymizer/src/srcset.rs`): the largest at or below 1024w, or the smallest+ /// when all are wider. A tie keeps `self`, so the first candidate wins it.+ fn loses_to(&self, other: &Candidate<'_>) -> bool {+ let (best, candidate) = (self.value, other.value);+ match self.kind {+ DescriptorKind::Density => {+ let target = SCREENSHOT_DEVICE_PIXEL_RATIO;+ match (best >= target, candidate >= target) {+ (false, true) => true,+ (true, true) => candidate < best,+ (false, false) => candidate > best,+ (true, false) => false,+ }+ }+ DescriptorKind::Width => {+ match (best <= MAX_SRCSET_WIDTH, candidate <= MAX_SRCSET_WIDTH) {+ (false, true) => true,+ (true, true) => candidate > best,+ (false, false) => candidate < best,+ (true, false) => false,+ }+ }+ }+ }+}++#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub struct KeptCandidate<'s> {+ pub index: usize,+ pub url: &'s str,+ /// The srcset has no 1x candidate and no width descriptor, so HTML adds an img's non-empty `src` as a 1x candidate,+ /// which wins at the screenshot's device pixel ratio.+ pub yields_to_img_src: bool,+}++#[derive(Debug, Clone, Copy, PartialEq, Eq)]+pub enum SrcsetSlot {+ /// Nothing else competes with the candidate: an img with no `src`, an img whose `src` the event does not hold, or+ /// a picture source of a type the fetcher stores.+ Alone,+ BesideImgSrc,+ /// A picture source of a type the fetcher does not store. A remote candidate there gets no fetch job, so only an+ /// inline one can show.+ UnfetchedTypeSource,+}++/// `None` for a malformed srcset, or one that mixes width and density descriptors.+pub fn kept_candidate(srcset: &str) -> Option<KeptCandidate<'_>> {+ let mut first_kind = None;+ let mut mixed = false;+ let mut kept: Option<Candidate<'_>> = None;+ for_each_candidate(srcset, |candidate| {+ let kind = *first_kind.get_or_insert(candidate.kind);+ mixed |= candidate.kind != kind;+ kept = Some(match kept.take() {+ Some(best) if !best.loses_to(&candidate) => best,+ _ => candidate,+ });+ })?;+ kept.filter(|_| !mixed).map(|candidate| KeptCandidate {+ index: candidate.index,+ url: candidate.url,+ yields_to_img_src: candidate.kind == DescriptorKind::Density+ && candidate.value != SCREENSHOT_DEVICE_PIXEL_RATIO,+ })+}++/// The kept candidate when it can show offline, as a URL the fetcher can take or an inline image the image scan keeps.+pub fn usable_candidate(srcset: &str) -> Option<KeptCandidate<'_>> {+ kept_candidate(srcset).filter(|kept| {+ if is_remote(kept.url) {+ try_canonicalize(kept.url).is_ok()+ } else {+ keeps_inline_image(kept.url)+ }+ })+}++/// The inspector fetches this candidate and the writer keeps only it, so that the two agree on what the slot shows.+pub fn shown_candidate(srcset: &str, slot: SrcsetSlot) -> Option<KeptCandidate<'_>> {+ usable_candidate(srcset).filter(|kept| match slot {+ SrcsetSlot::Alone => true,+ SrcsetSlot::BesideImgSrc => !kept.yields_to_img_src,+ SrcsetSlot::UnfetchedTypeSource => !is_remote(kept.url),+ })+}++/// Streams the candidates, so that a message-sized srcset needs no list of them. `None` for a malformed srcset.+fn for_each_candidate<'s>(srcset: &'s str, mut found: impl FnMut(Candidate<'s>)) -> Option<()> {+ let bytes = srcset.as_bytes();+ let mut index = 0;+ let mut position = 0;+ while position < bytes.len() {+ while matches!(bytes.get(position), Some(byte) if byte.is_ascii_whitespace()) {+ position += 1;+ }+ if position == bytes.len() {+ break;+ }+ if bytes[position] == b',' {+ return None;+ }++ let url_start = position;+ while matches!(bytes.get(position), Some(byte) if !byte.is_ascii_whitespace()) {+ position += 1;+ }+ let mut url_end = position;+ while bytes.get(url_end.wrapping_sub(1)) == Some(&b',') && url_end > url_start {+ url_end -= 1;+ }+ if url_end == url_start {+ return None;+ }+ if position - url_end > 1 {+ return None;+ }+ let url = &srcset[url_start..url_end];+ if !url.starts_with("data:") && url.contains(',') {+ return None;+ }+ let ended_with_comma = url_end < position;++ while matches!(bytes.get(position), Some(byte) if byte.is_ascii_whitespace()) {+ position += 1;+ }+ let descriptor_start = position;+ if !ended_with_comma {+ while matches!(bytes.get(position), Some(byte) if *byte != b',') {+ position += 1;+ }+ }+ let descriptor = srcset[descriptor_start..position].trim();+ let separated = if bytes.get(position) == Some(&b',') {+ position += 1;+ true+ } else {+ ended_with_comma+ };++ let (kind, value) = parse_descriptor(descriptor)?;+ found(Candidate {+ index,+ url,+ kind,+ value,+ });+ index += 1;+ if separated {+ let mut next = position;+ while matches!(bytes.get(next), Some(byte) if byte.is_ascii_whitespace()) {+ next += 1;+ }+ if next == bytes.len() || bytes[next] == b',' {+ return None;+ }+ }+ }+ Some(())+}++fn parse_descriptor(descriptor: &str) -> Option<(DescriptorKind, f64)> {+ if descriptor.is_empty() {+ return Some((DescriptorKind::Density, 1.0));+ }+ if descriptor.split_ascii_whitespace().count() != 1 {+ return None;+ }+ if let Some(width) = descriptor.strip_suffix('w') {+ let value = width.parse::<u32>().ok()?;+ return (value > 0).then_some((DescriptorKind::Width, f64::from(value)));+ }+ let density = descriptor.strip_suffix('x')?.parse::<f64>().ok()?;+ (density.is_finite() && density > 0.0).then_some((DescriptorKind::Density, density))+}++#[cfg(test)]+mod tests {+ use base64::engine::general_purpose::STANDARD;+ use base64::Engine;++ use super::*;++ fn kept_url(srcset: &str) -> Option<&str> {+ kept_candidate(srcset).map(|kept| kept.url)+ }++ #[test]+ fn keeps_the_density_chromium_shows_at_1x_and_the_v3_width() {+ for (srcset, expected) in [+ (+ "https://example.com/a.png 1x, https://example.com/b.png 2x",+ "https://example.com/a.png",+ ),+ (+ "https://example.com/a.png 320w, https://example.com/b.png 1280w",+ "https://example.com/a.png",+ ),+ (+ "large.png 3840w, medium.png 960w, small.png 320w, extra.png 1920w",+ "medium.png",+ ),+ (+ "small.png 960w, exact.png 1024w, large.png 2048w",+ "exact.png",+ ),+ (+ "large.png 3840w, small.png 1280w, medium.png 1920w",+ "small.png",+ ),+ ("large.png 4x, medium.png 2x, small.png 1x", "small.png"),+ ("medium.png 1.5x, large.png 3x", "medium.png"),+ ("large.png 4x, small.png 3x", "small.png"),+ ("tiny.png 0.5x, large.png 2x", "large.png"),+ ("tiny.png 0.5x, small.png 0.75x", "small.png"),+ ("default.png, large.png 3x", "default.png"),+ ("only.png 4096w", "only.png"),+ (+ "data:image/png;base64,AAAA 1x, data:image/png;base64,BBBB 2x",+ "data:image/png;base64,AAAA",+ ),+ ] {+ assert_eq!(kept_url(srcset), Some(expected), "{srcset}");+ }+ assert_eq!(+ kept_candidate("c.png 3x, b.png 2x, a.png 1x").map(|kept| kept.index),+ Some(2)+ );+ }++ #[test]+ fn an_img_src_wins_over_a_srcset_with_no_1x_or_width_candidate() {+ let (a, b) = (+ "https://cdn.example.com/a.png",+ "https://cdn.example.com/b.png",+ );+ for (srcset, shown_beside_src) in [+ (format!("{a} 1x, {b} 2x"), Some(a)),+ (format!("{b} 2x, {a} 1.0x"), Some(a)),+ (format!("{a} 320w, {b} 1280w"), Some(a)),+ (format!("{b} 2x"), None),+ (format!("{a} 0.5x"), None),+ ] {+ let alone = shown_candidate(&srcset, SrcsetSlot::Alone).map(|kept| kept.url);+ let beside_src =+ shown_candidate(&srcset, SrcsetSlot::BesideImgSrc).map(|kept| kept.url);+ assert!(alone.is_some(), "{srcset}");+ assert_eq!(beside_src, shown_beside_src, "{srcset}");+ }+ }++ #[test]+ fn keeps_no_candidate_of_a_mixed_or_malformed_srcset() {+ for srcset in [+ "https://example.com/a.png 1x, https://example.com/b.png 1280w",+ "https://example.com/a.png 0x",+ "https://example.com/a.png 1x 2x",+ ", https://example.com/a.png 1x",+ "https://example.com/a.png 1x,",+ "https://example.com/a.png 1x,, https://example.com/b.png 2x",+ "https://example.com/a.png,https://example.com/b.png",+ ] {+ assert_eq!(kept_candidate(srcset), None, "{srcset}");+ }+ }++ #[test]+ fn the_first_candidate_wins_a_tie() {+ for descriptor in ["0.5x", "1x", "2x", "3x", "1024w", "2048w"] {+ assert_eq!(+ kept_url(&format!("first.png {descriptor}, second.png {descriptor}")),+ Some("first.png")+ );+ }+ }++ #[test]+ fn a_srcset_is_usable_when_its_kept_candidate_can_show_offline() {+ let inline =+ |mime: &str, bytes: &[u8]| format!("data:{mime};base64,{} 1x", STANDARD.encode(bytes));+ for (srcset, usable) in [+ ("https://cdn.example.com/a.png 1x".to_owned(), true),+ ("//cdn.example.com/a.png 1x".to_owned(), true),+ (inline("image/png", b"\x89PNG\r\n\x1a\nbytes"), true),+ ("data:image/svg+xml,%3Csvg%2F%3E 1x".to_owned(), true),+ (+ inline(+ "image/avif",+ b"\x00\x00\x00\x1cftypavif\x00\x00\x00\x00avifmif1miaf",+ ),+ false,+ ),+ (inline("image/bmp", b"BMbitmap"), false),+ ("/relative/a.png 1x".to_owned(), false),+ ("https://10.0.0.7/a.png 1x".to_owned(), false),+ (+ "https://cdn.example.com/a.png 1x, b.png 1280w".to_owned(),+ false,+ ),+ ] {+ assert_eq!(usable_candidate(&srcset).is_some(), usable, "{srcset}");+ }+ }+}diff --git a/rust/ai-research/replay-core-v4/src/scan/images.rs b/rust/ai-research/replay-core-v4/src/scan/images.rsindex a1247472bb56..8416b50b8a2b 100644--- a/rust/ai-research/replay-core-v4/src/scan/images.rs+++ b/rust/ai-research/replay-core-v4/src/scan/images.rs@@ -472,6 +472,17 @@ pub fn extract_images<'b>(bytes: &'b [u8], sink: &mut ImageSink<'_>) -> Cow<'b,}}+/// Whether the scan keeps the image of this data URI, as a reference or inline as an SVG, by the rules for one image.+/// A message past its caps can still decline it.+pub fn keeps_inline_image(data_uri: &str) -> bool {+ let mut json = Vec::with_capacity(data_uri.len() + 2);+ scan::write_json_string(data_uri, &mut json);+ let mut sink = ImageSink::new(&[0; 32], 0, "", ImageLimits::default());+ extract_images(&json, &mut sink);+ sink.declined == [0; ImageDecline::ALL.len()]+ && (!sink.images.is_empty() || sink.left_inline > 0)+}+#[cfg(test)]mod tests {use base64::engine::general_purpose::STANDARD;diff --git a/rust/ai-research/replay-recorder-v4/src/rewrite.rs b/rust/ai-research/replay-recorder-v4/src/rewrite.rsindex b7f89da8d7c0..fba47bb0814f 100644--- a/rust/ai-research/replay-recorder-v4/src/rewrite.rs+++ b/rust/ai-research/replay-recorder-v4/src/rewrite.rs@@ -2,27 +2,28 @@//! policy declines becomes a marker that names the reason, so that no credential, signature or private host stays in//! the block. The inspector names the URLs: those in image slots, CSS included.//!-//! A srcset is rebuilt instead: a declined candidate leaves the list, and so does each remote candidate of a picture-//! source whose type the fetcher does not store. An inline image, such as an SVG data URI, stays. A browser skips a-//! source with no candidates, so it falls back to a source or an img that can load, where a marker in the list would-//! show a broken image. The writer rebuilds each srcset in the bytes it stores, after escapes were written again and-//! inline images left, so the rebuild never depends on a prediction of those bytes.+//! A srcset is rebuilt instead: it keeps only the candidate that the inspector fetched, by [`shown_candidate`] in the+//! value the inspector read and its slot. It keeps none when the slot shows no candidate of that value: the inspector+//! then fetches the img's `src`, and a browser skips a source with no candidates, so it falls back to a source or an img+//! that can load, where a marker in the list would show a broken image. The writer rebuilds each srcset in the bytes it+//! stores, after escapes were written again and inline images left. Moving an inline image out can change how a srcset+//! parses, so the writer finds the inspector's candidate by its [`CandidatePosition`].use std::borrow::Cow;use std::cell::Cell;-use std::collections::{BTreeMap, HashMap, HashSet};+use std::collections::{BTreeMap, HashMap};use aho_corasick::{AhoCorasick, AhoCorasickKind, MatchKind};use metrics::counter;use posthog_replay_anonymizer::v4_image_refs::{url_declined_marker, ImageDecline, IMAGE_DECLINED_PREFIX,};-use replay_core_v4::bytescan::{canonical_strings, unescape, write_json_string, Span};+use replay_core_v4::bytescan::{unescape, write_json_string, Span};use replay_core_v4::inspect::css::{escape_in_identifier, rewrite_functions, CssRewrite};use replay_core_v4::inspect::dom::{for_each_srcset_candidate, Srcset};-use replay_core_v4::scan::images::{extract_images, ImageLimits, ImageSink};+use replay_core_v4::inspect::srcset::shown_candidate;use replay_core_v4::scan::tokens::{Token, Tokens};-use replay_core_v4::seal::{image_content_key, image_url_key};+use replay_core_v4::seal::image_url_key;use replay_core_v4::url_policy::is_remote;use crate::scrub::{Scrub, Scrubber};@@ -230,11 +231,10 @@ pub struct UrlRewrite {css_matcher: Option<AhoCorasick>,css_targets: Vec<CssTarget>,css_texts: HashMap<String, usize>,- /// Remote URLs that the block names only in picture sources of a type the fetcher does not store.- unfetched: HashSet<String>,- /// Each inspected srcset value as the stored block holds it, under [`without_image_ids`]. Only these are rebuilt, so- /// a srcset key outside an image slot, such as in a custom event's payload, stays as it is.- stored_srcsets: HashSet<Vec<u8>>,+ /// The shown candidate of each inspected srcset value, by the [`remote_urls`] of the value. Only a stored srcset with+ /// the same remote URLs is rebuilt, so a srcset key outside an image slot, such as in a custom event's payload, stays+ /// as it is. Values with the same remote URLs share an entry, so each keeps the candidate of every one of them.+ kept_srcset_candidates: HashMap<Vec<String>, Vec<CandidatePosition>>,url_targets: HashMap<String, usize>,team_id: i64,key_month: String,@@ -356,33 +356,16 @@ impl UrlRewrite {.ok()}).flatten();- // A URL that a source of a stored type also names stays, as the inspector merges identical srcset values.- let mut source_types: HashMap<&str, bool> = HashMap::new();+ let mut kept_srcset_candidates: HashMap<Vec<String>, Vec<CandidatePosition>> =+ HashMap::new();for srcset in srcsets {- for_each_srcset_candidate(&srcset.text, |url, _| {- if is_remote(url) {- *source_types.entry(url).or_insert(true) &= srcset.unfetched_type;- }- });+ let kept = kept_srcset_candidates+ .entry(remote_urls(&srcset.text))+ .or_default();+ if let Some(position) = shown_position(srcset).filter(|p| !kept.contains(p)) {+ kept.push(position);+ }}- let unfetched = source_types- .into_iter()- .filter(|&(_, unfetched)| unfetched)- .map(|(url, _)| url.to_owned())- .collect();- // The writer writes the escapes of the raw event again and then takes its inline images out, so each value- // goes through the same two steps.- let content_key = image_content_key(team_key);- let stored_srcsets = srcsets- .iter()- .map(|srcset| {- let mut sink =- ImageSink::new(&content_key, team_id, key_month, ImageLimits::default());- let canonical = canonical_strings(&srcset.raw);- let stored = extract_images(&canonical, &mut sink);- without_image_ids(&stored).into_owned()- })- .collect();let matcher = (!patterns.is_empty()).then(|| {// A DFA over long URLs can take hundreds of times their bytes. A contiguous NFA takes about ten.@@ -473,8 +456,7 @@ impl UrlRewrite {.enumerate().map(|(index, rewrite)| (rewrite.text.clone(), index)).collect(),- unfetched,- stored_srcsets,+ kept_srcset_candidates,url_targets,team_id,key_month: key_month.to_owned(),@@ -505,18 +487,15 @@ impl UrlRewrite {let Some(end) = string_end(bytes, start) else {continue;};- if !self- .stored_srcsets- .contains(without_image_ids(&bytes[start..end]).as_ref())- {- continue;- }let Ok(text) = unescape(bytes, (start, end)) else {continue;};+ let Some(kept) = self.kept_srcset_candidates.get(&remote_urls(&text)) else {+ continue;+ };let Some((value, outcome, covers)) = rebuilt_srcset(&text,- &self.unfetched,+ kept,&self.url_targets,self.team_id,&self.key_month,@@ -819,43 +798,63 @@ fn string_end(bytes: &[u8], start: usize) -> Option<usize> {None}-/// The bytes with each image reference and declined marker cut to its `image:v4:` prefix. The image caps of a message-/// decide whether an image becomes a reference or `over_limit`, and the srcset prediction in [`UrlRewrite::new`]-/// cannot see the other images of that message.-fn without_image_ids(bytes: &[u8]) -> Cow<'_, [u8]> {- const PREFIX: &[u8] = b"image:v4:";- let find = |bytes: &[u8]| {- bytes- .windows(PREFIX.len())- .position(|window| window == PREFIX)- };- let Some(mut at) = find(bytes) else {- return Cow::Borrowed(bytes);- };- let mut out = Vec::with_capacity(bytes.len());- let mut rest = bytes;- loop {- out.extend_from_slice(&rest[..at + PREFIX.len()]);- rest = &rest[at + PREFIX.len()..];- let id_length = rest- .iter()- .position(|&b| !(b.is_ascii_alphanumeric() || matches!(b, b'_' | b'-' | b':')))- .unwrap_or(rest.len());- rest = &rest[id_length..];- match find(rest) {- Some(next) => at = next,- None => break,+/// The remote URLs of a srcset, in order. The writer moves only inline images out of a srcset, so these name the+/// srcset the same way in the event the inspector read and in the block the writer stores.+fn remote_urls(srcset: &str) -> Vec<String> {+ let mut urls = Vec::new();+ for_each_srcset_candidate(srcset, |url, _| {+ if is_remote(url) {+ urls.push(url.to_owned());}- }- out.extend_from_slice(rest);- Cow::Owned(out)+ });+ urls+}++/// A srcset candidate by its kind and its index among the candidates of that kind. The writer moves only inline images+/// out of a srcset, so a position names the same candidate in the event the inspector read and in the block the writer+/// stores.+#[derive(Debug, Clone, Copy, PartialEq, Eq)]+enum CandidatePosition {+ Remote(usize),+ Inline(usize),+}++fn for_each_positioned_candidate<'s>(+ srcset: &'s str,+ mut found: impl FnMut(CandidatePosition, &'s str, &'s str),+) {+ let (mut remote, mut inline) = (0, 0);+ for_each_srcset_candidate(srcset, |url, descriptors| {+ let position = if is_remote(url) {+ remote += 1;+ CandidatePosition::Remote(remote - 1)+ } else {+ inline += 1;+ CandidatePosition::Inline(inline - 1)+ };+ found(position, url, descriptors);+ });+}++fn shown_position(srcset: &Srcset) -> Option<CandidatePosition> {+ // `shown_candidate` reads only a well-formed srcset, whose candidates `for_each_srcset_candidate` reads in the same order.+ let shown_index = shown_candidate(&srcset.text, srcset.slot)?.index;+ let (mut index, mut shown) = (0, None);+ for_each_positioned_candidate(&srcset.text, |position, _, _| {+ if index == shown_index {+ shown = Some(position);+ }+ index += 1;+ });+ shown}-/// The srcset with only the candidates the block keeps, each remote URL as its reference, how it was rebuilt, and the-/// URL targets of the remote URLs it held. `None` when every candidate stays, because the URL targets then replace it.+/// The srcset with only the candidate the block keeps, a remote URL as its reference, how it was rebuilt, and the URL+/// targets of the remote URLs it held. `None` when the srcset keeps its one candidate, because the URL targets then+/// replace it.fn rebuilt_srcset(text: &str,- unfetched: &HashSet<String>,+ kept: &[CandidatePosition],url_targets: &HashMap<String, usize>,team_id: i64,key_month: &str,@@ -875,17 +874,20 @@ fn rebuilt_srcset(value.push_str(descriptors);}};- for_each_srcset_candidate(text, |url, descriptors| {- if url.starts_with(IMAGE_DECLINED_PREFIX) {- removed = true;- } else if !is_remote(url) {- keep(url, descriptors);- } else {- covers.extend(url_targets.get(url));- match url_reference(url, team_id, key_month, url_key) {- Ok((_, reference)) if !unfetched.contains(url) => keep(&reference, descriptors),- _ => removed = true,+ for_each_positioned_candidate(text, |position, url, descriptors| {+ let is_kept = kept.contains(&position);+ match position {+ CandidatePosition::Remote(_) => {+ covers.extend(url_targets.get(url));+ match url_reference(url, team_id, key_month, url_key) {+ Ok((_, reference)) if is_kept => keep(&reference, descriptors),+ _ => removed = true,+ }}+ CandidatePosition::Inline(_) if is_kept && !url.starts_with(IMAGE_DECLINED_PREFIX) => {+ keep(url, descriptors)+ }+ CandidatePosition::Inline(_) => removed = true,}});if !removed {@@ -903,6 +905,8 @@ fn rebuilt_srcset(mod tests {use base64::engine::general_purpose::STANDARD;use base64::Engine;+ use replay_core_v4::bytescan::canonical_strings;+ use replay_core_v4::inspect::srcset::SrcsetSlot;use super::*;@@ -960,28 +964,83 @@ mod tests {}#[test]- fn rebuilds_a_srcset_whose_image_the_message_cap_declined() {+ fn keeps_only_the_srcset_candidate_the_inspector_fetched() {let png = format!("data:image/png;base64,{}",STANDARD.encode(b"\x89PNG\r\n\x1a\nbytes"));let svg = "data:image/svg+xml,%3Csvg%2F%3E";- let text = format!("{png} 1x, {svg} 2x");- let srcset = Srcset {- raw: serde_json::to_vec(&text).unwrap(),- text,- unfetched_type: false,- };- let mut rewrite = UrlRewrite::new(&[], &[srcset], &[], 7, "2026-10", &TEAM_KEY);- // A message past its image cap stores the PNG as `over_limit`, where a fresh sink takes it.- let stored = format!(- r#"{{"srcset":"{} 1x, {svg} 2x"}}"#,- ImageDecline::OverLimit.marker()- );- assert_eq!(- String::from_utf8(rewrite.apply(stored.as_bytes(), true).into_owned()).unwrap(),- format!(r#"{{"srcset":"{svg} 2x"}}"#)+ let (a, b, private) = (+ "https://cdn.example.com/a.png",+ "https://cdn.example.com/b.png",+ "https://10.0.0.7/b.png",);+ let over_limit = ImageDecline::OverLimit.marker();+ for (slot, inspected, stored, expected) in [+ // A message past its image cap stores the PNG as `over_limit`, where the inspector read a data URI.+ (+ SrcsetSlot::Alone,+ format!("{png} 2x, {svg} 1x"),+ format!("{over_limit} 2x, {svg} 1x"),+ format!("{svg} 1x"),+ ),+ (+ SrcsetSlot::Alone,+ format!("{a} 1x, {b} 2x"),+ format!("{a} 1x, {b} 2x"),+ format!("{} 1x", reference(a)),+ ),+ (+ SrcsetSlot::BesideImgSrc,+ format!("{a} 1x, {b} 2x"),+ format!("{a} 1x, {b} 2x"),+ format!("{} 1x", reference(a)),+ ),+ // The img shows its `src`, which HTML adds as a 1x candidate.+ (+ SrcsetSlot::BesideImgSrc,+ format!("{b} 2x"),+ format!("{b} 2x"),+ String::new(),+ ),+ // The 2x candidate has no fetch job, so it cannot stand in for the declined 1x one.+ (+ SrcsetSlot::Alone,+ format!("{private} 1x, {b} 2x"),+ format!("{private} 1x, {b} 2x"),+ String::new(),+ ),+ (+ SrcsetSlot::Alone,+ format!("{a} 1x, {b} 640w"),+ format!("{a} 1x, {b} 640w"),+ String::new(),+ ),+ // The image scan moves out a data URI in any case, where the inspector read a malformed srcset and fetched+ // the img's `src` instead.+ (+ SrcsetSlot::Alone,+ format!(+ "DATA:image/png;base64,{} 1x, {b} 2x",+ STANDARD.encode(b"\x89PNG\r\n\x1a\nbytes")+ ),+ format!("image:v4:7:2026-10:0123abcd 1x, {b} 2x"),+ String::new(),+ ),+ ] {+ let srcset = Srcset {+ raw: serde_json::to_vec(&inspected).unwrap(),+ text: inspected.clone(),+ slot,+ };+ let mut rewrite = UrlRewrite::new(&[], &[srcset], &[], 7, "2026-10", &TEAM_KEY);+ let stored = format!(r#"{{"srcset":"{stored}"}}"#);+ assert_eq!(+ String::from_utf8(rewrite.apply(stored.as_bytes(), true).into_owned()).unwrap(),+ format!(r#"{{"srcset":"{expected}"}}"#),+ "{inspected}"+ );+ }}#[test]diff --git a/rust/ai-research/replay-recorder-v4/src/write.rs b/rust/ai-research/replay-recorder-v4/src/write.rsindex 1eb6852c69a5..84880cddfc2d 100644--- a/rust/ai-research/replay-recorder-v4/src/write.rs+++ b/rust/ai-research/replay-recorder-v4/src/write.rs@@ -1050,7 +1050,16 @@ mod tests {let svg ="data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%2F%3E";let remote_svg = "https://cdn.example.com/a.svg";+ let avif_typed_svg = "data:image/svg+xml,%3Csvg%2F%3E";let shared = "https://cdn.example.com/shared.webp";+ let (beside_img_src, img_src) = (+ "https://cdn.example.com/beside-img-src.png",+ "https://cdn.example.com/img-src.png",+ );+ let (also_in_webp, webp_1x) = (+ "https://cdn.example.com/also-in-webp.png",+ "https://cdn.example.com/webp-1x.webp",+ );let other_avif = format!("data:image/avif;base64,{}",STANDARD.encode(b"\x00\x00\x00\x1cftypavif\x00\x00\x00\x00avifmif1other")@@ -1065,20 +1074,28 @@ mod tests {source(6, json!({"type": "image/svg+xml", "srcset": format!("{svg} 1x")})),source(7, json!({"type": "image/svg+xml", "srcset": format!("{remote_svg} 1x")})),source(3, json!({"type": "image/avif", "srcset": format!("{avif} 1x, {inline_avif} 2x")})),+ // An inline candidate needs no fetch job, so it stays whatever the type.+ source(15, json!({"type": "image/avif", "srcset": format!("{avif} 2x, {avif_typed_svg} 1x")})),// A bare srcset equal to the img's src below: only the srcset empties.source(8, json!({"type": "image/avif", "srcset": bare})),// Values that read the same once their inline AVIFs leave merge their policies, so the WebP source keeps// the shared candidate.source(11, json!({"type": "image/avif", "srcset": format!("{shared} 1x, {inline_avif} 2x")})),source(12, json!({"type": "image/webp", "srcset": format!("{shared} 1x, {other_avif} 2x")})),+ // The WebP source keeps its other candidate, so no source fetches the URL the two share.+ source(13, json!({"type": "image/avif", "srcset": format!("{also_in_webp} 1x")})),+ source(14, json!({"type": "image/webp", "srcset": format!("{also_in_webp} 2x, {webp_1x} 1x")})),+ // Neither this source nor the img below with the same srcset shows the candidate, so neither keeps it.+ source(16, json!({"type": "image/avif", "srcset": format!("{beside_img_src} 2x")})),// An empty type gives no hint, so the source keeps its candidate.source(9, json!({"type": "", "srcset": format!("{typeless} 1x")})),- // An inline image the image scrub cannot read leaves the list as a declined URL does.- source(4, json!({"type": "image/webp", "srcset": format!("{signed} 2x, {webp} 1x, {bitmap} 3x")})),+ // Only the kept candidate stays, so a declined URL and an inline image the image scrub cannot read leave too.+ source(4, json!({"type": "image/webp", "srcset": format!("{webp} 1x, {signed} 2x, {bitmap} 3x")})),{"type": 2, "tagName": "img", "attributes": {"src": fallback}, "childNodes": [], "id": 5},]});let img = json!({"type": 2, "tagName": "img", "attributes": {"src": bare}, "childNodes": [], "id": 10});- let events = json!([{"type": 2, "timestamp": 1, "data": {"node": {"type": 0, "id": 1, "childNodes": [picture, img]}}}]).to_string();+ let img_with_src = json!({"type": 2, "tagName": "img", "attributes": {"src": img_src, "srcset": format!("{beside_img_src} 2x")}, "childNodes": [], "id": 17});+ let events = json!([{"type": 2, "timestamp": 1, "data": {"node": {"type": 0, "id": 1, "childNodes": [picture, img, img_with_src]}}}]).to_string();let block = block_of("kept", &events);let mut inspector = replay_core_v4::inspect::session::SessionInspector::default();@@ -1087,7 +1104,12 @@ mod tests {inspector.observe_message(&mut inspection, message);}// The fetcher stores no AVIF, so the AVIF source gets no fetch job.- assert!(!inspection.finish().image_urls.iter().any(|u| u == avif));+ let fetched = inspection.finish().image_urls;+ assert!(!fetched+ .iter()+ .any(|u| u == avif || u == also_in_webp || u == beside_img_src));+ assert!(fetched.iter().any(|u| u == img_src));+ assert!(fetched.iter().any(|u| u == webp_1x));let lines = stored_lines(&events, &[]);let reference = |url: &str| {@@ -1100,10 +1122,11 @@ mod tests {.unwrap().1};- // The AVIF source has no candidate left and the declined one leaves the list, so the browser takes the WebP.+ // The AVIF source has no candidate left, so the browser takes the WebP.assert!(lines.contains(r#""srcset":"""#), "{lines}");assert!(- lines.contains(&format!(r#""srcset":"{svg} 1x""#)),+ lines.contains(&format!(r#""srcset":"{svg} 1x""#))+ && lines.contains(&format!(r#""srcset":"{avif_typed_svg} 1x""#)),"{lines}");assert!(@@ -1119,6 +1142,12 @@ mod tests {lines.contains(&format!(r#""srcset":"{} 1x""#, reference(shared))),"{lines}");+ assert!(+ lines.contains(&format!(r#""srcset":"{} 1x""#, reference(webp_1x)))+ && !lines.contains(&reference(also_in_webp))+ && !lines.contains(&reference(beside_img_src)),+ "{lines}"+ );assert!(lines.contains(&format!(r#""src":"{}""#, reference(bare))),"{lines}"