:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
//! Domain layer: the static analysis of a run's pinned conditions, the census over it, and the
//! per-chunk projection derived from it. Depends on `condition`, `ids`, `plan`, `projection`, and
//! `cohort-core`.
//!
//! Two consumers read the same analysis. The scan path narrows each chunk's SELECT through
//! [`ConditionAnalyses::projection`]; the census reports per run what real catalogs look like, which
//! is what says whether the narrowing can pay on a given team's event mix.
//!
//! The census is shaped around the question a later projection pass has to answer, which is
//! per event name rather than per condition. A scan selects columns for a whole event, so one
//! condition on `$pageview` that cannot be narrowed forces full columns for every `$pageview` row,
//! however many of its siblings are narrow. Counting conditions alone would report that event as
//! mostly projectable and overstate what a projection can do — on exactly the fat events where it
//! matters.
//!
//! Being a pure function of the pinned payload is what makes it safe to run per run rather than per
//! chunk: a re-validated run classifies identically, so a chunk retried on another replica cannot
//! disagree with the first attempt.
use std::collections::{BTreeMap, BTreeSet, HashMap};
use std::sync::Arc;
use cohort_core::filters::TeamFilters;
use cohort_core::hogvm::analysis::{
analyze_condition_within, event_equalities, AnalysisBudget, ColumnExactness, ConditionAnalysis,
EvaluationClass, EventEqualities, Exactness, FullColumnsReason, GlobalRoot, Projection,
ReadPath,
};
use super::condition::{EventNameSet, PinnedCondition};
use super::ids::ConditionHash;
use super::plan::ActiveConditions;
use super::projection::{ChunkProjection, ColumnExactKeys};
use super::row_filter::ScanRowFilter;
/// What one condition turned out to need. Ordered so a census renders the same way every time.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
pub enum ConditionClass {
/// Nothing but an event-name equality, which a scan's event filter already decides.
EventOnly,
/// Reads a named, narrow set of globals.
Projectable,
/// Understood, but reads a whole object, so every column is needed.
FullColumns,
/// Not understood, so it falls back to every column.
Unanalyzable,
}
impl ConditionClass {
pub const ALL: [Self; 4] = [
Self::EventOnly,
Self::Projectable,
Self::FullColumns,
Self::Unanalyzable,
];
pub const fn as_str(self) -> &'static str {
match self {
Self::EventOnly => "event_only",
Self::Projectable => "projectable",
Self::FullColumns => "full_columns",
Self::Unanalyzable => "unanalyzable",
}
}
/// Whether this class forces a scan to select every column for its event name.
const fn is_unprojectable(self) -> bool {
matches!(self, Self::FullColumns | Self::Unanalyzable)
}
fn of(analysis: &ConditionAnalysis) -> Self {
match (&analysis.evaluation, &analysis.projection) {
// An event-name equality needs no per-row evaluation, so its projection is irrelevant —
// except when the projection itself failed, which means the two passes disagree about
// what the program is. Classifying by the projection there fails wide rather than
// narrow, which is the direction a wrong answer has to fall.
(EvaluationClass::EventOnly { .. }, Projection::Reads(_)) => Self::EventOnly,
// Exhaustive rather than defaulted, like every other reason mapping in this chain. A
// new `FullColumnsReason` landing in `FullColumns` by default would be reported as an
// understood outcome: only `Unanalyzable` reaches the reason counter, so the census
// would describe a brand new failure mode as expected.
(EvaluationClass::EventOnly { .. }, Projection::FullColumns(reason))
| (EvaluationClass::General, Projection::FullColumns(reason)) => match reason {
FullColumnsReason::Unanalyzable(_) => Self::Unanalyzable,
FullColumnsReason::BarePropertiesRoot
| FullColumnsReason::BarePersonRoot
| FullColumnsReason::RepresentationSensitiveCall => Self::FullColumns,
},
(EvaluationClass::General, Projection::Reads(_)) => Self::Projectable,
}
}
}
/// The label for a pinned condition whose hash the frozen catalog has no bytecode for. Not an
/// analysis outcome: the condition never reached the analyzer.
pub const MISSING_BYTECODE_REASON: &str = "missing_bytecode";
/// How many blocked event names the per-run log line names before it starts counting instead.
/// Sized to hold a large team's whole blocked set, so the cap is a ceiling rather than a routine
/// truncation.
const MAX_RENDERED_BLOCKED_EVENTS: usize = 50;
/// How many worst-case conditions one run's census may cost, as one budget its conditions share.
///
/// The analyzer bounds each condition, which leaves `conditions × ceiling` for a run — and nothing
/// caps behavioral conditions per run, so that product is unbounded. Sharing one budget bounds the
/// run instead. Past it the remaining conditions report the budget and fall back to every column,
/// which the census counts under its own reason rather than hiding.
///
/// A budget rather than a deadline because [`ConditionAnalyses::build`] has to stay a pure function
/// of the pinned payload: two replicas re-validating one run must classify it identically, and a
/// wall clock would not.
const CENSUS_WORST_CASE_CONDITIONS: usize = 8;
/// One analysis per unique pinned condition hash.
///
/// Person runs are deliberately absent. Their conditions read the small person-scope globals rather
/// than the event globals this analysis models, so classifying them here would report against the
/// wrong vocabulary.
#[derive(Debug, Default)]
pub struct ConditionAnalyses {
by_hash: HashMap<ConditionHash, ConditionAnalysis>,
/// Not budgeted, because matching is linear in the program.
equalities: HashMap<ConditionHash, EventEqualities>,
}
impl ConditionAnalyses {
/// Analyze every unique hash among `conditions` whose bytecode the frozen catalog carries,
/// under one budget they share — see [`CENSUS_WORST_CASE_CONDITIONS`].
///
/// `conditions` is walked in order, so the budget is spent the same way every time and the
/// result stays a pure function of the pinned payload.
pub fn build(conditions: &[PinnedCondition], filters: &TeamFilters) -> Self {
let mut budget = AnalysisBudget::for_conditions(CENSUS_WORST_CASE_CONDITIONS);
let mut by_hash = HashMap::new();
let mut equalities = HashMap::new();
for condition in conditions {
if by_hash.contains_key(&condition.hash) {
continue;
}
let Some(program) = filters
.by_condition_to_program
.get(&condition.hash.as_bytes())
else {
continue;
};
by_hash.insert(
condition.hash,
analyze_condition_within(program.tokens(), &mut budget),
);
equalities.insert(condition.hash, event_equalities(program.tokens()));
}
Self {
by_hash,
equalities,
}
}
pub fn row_filter(
&self,
event_names: &EventNameSet,
filters: &TeamFilters,
active: &ActiveConditions,
) -> ScanRowFilter {
ScanRowFilter::derive(event_names, filters, active, &self.equalities)
}
pub fn row_filtered_conditions(&self) -> usize {
self.equalities
.values()
.filter(|equalities| equalities.row_filter.is_some())
.count()
}
/// One inexact read by any active condition keeps a key on the blob.
pub fn column_exact_keys(&self, active: &ActiveConditions) -> ColumnExactKeys {
active
.iter()
.map(|hash| self.column_exactness(hash))
.collect::<Option<ColumnExactness>>()
.map(ColumnExactKeys::new)
.unwrap_or_default()
}
/// `None` when the condition's reads are unknown.
fn column_exactness(&self, hash: &ConditionHash) -> Option<ColumnExactness> {
let Projection::Reads(paths) = &self.by_hash.get(hash)?.projection else {
return None;
};
let decided = self
.equalities
.get(hash)
.and_then(|equalities| equalities.column_exactness.as_ref());
paths
.iter()
.filter(|path| path.root == GlobalRoot::Properties)
.map(|path| {
let key = path.segments.first()?;
let exactness = decided
.and_then(|keys| keys.get(key))
.unwrap_or(Exactness::Inexact);
Some((key.as_str(), exactness))
})
.collect()
}
/// What a scan over the conditions active on one chunk has to select.
///
/// Takes the active set rather than the run's whole condition list: a window that has slid past
/// the chunk's day contributes nothing to that chunk's evaluation, so widening the scan for it
/// would read columns no condition on that day reads. An active hash with no analysis fails
/// closed inside [`ChunkProjection::derive`].
pub fn projection(&self, active: &ActiveConditions) -> ChunkProjection {
ChunkProjection::derive(active.iter().map(|hash| self.by_hash.get(hash)))
}
/// Count the conditions by class, and account for each event name what a projection over it
/// would have to supply.
///
/// Counted per unique hash and event name, not per pinned condition: the same condition shared
/// by several cohorts is one piece of work, and counting it twice would overstate the
/// projectable share.
///
/// The event name is part of the key rather than the hash alone, because the two arrive as
/// independent fields of the Django payload. Deduplicating on the hash would skip a repeated
/// hash before it reached its second event name, and that event would then be accounted from
/// its other conditions alone — projection-eligible on paper while carrying a condition that
/// needs every column.
pub fn census(&self, conditions: &[PinnedCondition]) -> ConditionCensus {
let mut census = ConditionCensus::default();
let mut seen: BTreeSet<(ConditionHash, &str)> = BTreeSet::new();
for condition in conditions {
if !seen.insert((condition.hash, condition.event_name.as_str())) {
continue;
}
let event = census
.per_event
.entry(condition.event_name.clone())
.or_default();
let Some(analysis) = self.by_hash.get(&condition.hash) else {
// Unreachable today: `resolve_conditions` drops any hash the frozen catalog has no
// bytecode for, so every surviving condition has one. Kept because the alternative
// to a label here is a condition that silently leaves no trace in the census.
*census
.by_class
.entry(ConditionClass::Unanalyzable)
.or_default() += 1;
*census
.unanalyzable_reasons
.entry(UnanalyzableLabel::missing_bytecode())
.or_default() += 1;
event.unprojectable += 1;
event.blockers.insert(MISSING_BYTECODE_REASON.to_owned());
continue;
};
let class = ConditionClass::of(analysis);
*census.by_class.entry(class).or_default() += 1;
match class {
ConditionClass::EventOnly => event.event_only += 1,
ConditionClass::Projectable => event.projectable += 1,
_ => event.unprojectable += 1,
}
if let Projection::FullColumns(reason) = &analysis.projection {
let label = UnanalyzableLabel::of(reason);
if class.is_unprojectable() {
event.blockers.insert(label.render());
}
if class == ConditionClass::Unanalyzable {
*census.unanalyzable_reasons.entry(label).or_default() += 1;
}
}
if let Projection::Reads(paths) = &analysis.projection {
event.reads.extend(paths.iter().map(ReadPath::render));
}
}
census
}
}
/// A reason label pair: the coarse reason, plus the opcode when one is what stopped the analysis.
///
/// Both halves are closed vocabularies — the reason by construction, the opcode because
/// `Operation` is a 57-variant enum — so the pair is safe as a metric dimension. The opcode is
/// carried because "unsupported_op" alone cannot tell one fixable compiler template apart from a
/// program nothing will ever narrow, which is the difference the census exists to report.
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct UnanalyzableLabel {
pub reason: &'static str,
/// The opcode's variant name, from its `Debug` rendering. `Arc<str>` because the metric layer
/// clones a label value per emission and the repo's convention is a refcount bump rather than
/// an allocation.
pub op: Option<Arc<str>>,
}
impl UnanalyzableLabel {
fn of(reason: &FullColumnsReason) -> Self {
Self {
reason: reason.as_str(),
op: reason.op().map(|op| Arc::from(format!("{op:?}").as_str())),
}
}
fn missing_bytecode() -> Self {
Self {
reason: MISSING_BYTECODE_REASON,
op: None,
}
}
/// The label as one string, for the log line and the per-event blocker set.
pub fn render(&self) -> String {
match &self.op {
Some(op) => format!("{}:{op}", self.reason),
None => self.reason.to_owned(),
}
}
}
/// What the conditions on one event name would cost a projection over it.
///
/// The counts are per unique condition hash on that event name. `unprojectable` is the one that
/// decides: a scan selects columns for the whole event, so one unprojectable condition forces full
/// columns for every row of it.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct EventCensus {
pub event_only: u64,
pub projectable: u64,
pub unprojectable: u64,
/// The union of what the narrow conditions on this event name read, rendered dotted.
pub reads: BTreeSet<String>,
/// Why the unprojectable ones are, as rendered [`UnanalyzableLabel`]s.
pub blockers: BTreeSet<String>,
}
impl EventCensus {
/// Whether a projection over this event name is possible at all.
pub const fn is_projection_eligible(&self) -> bool {
self.unprojectable == 0
}
}
/// The per-run shape report. Every collection is ordered, so two runs over the same catalog log the
/// same text and a census can be compared across replicas.
#[derive(Debug, Default, PartialEq, Eq)]
pub struct ConditionCensus {
pub by_class: BTreeMap<ConditionClass, u64>,
pub unanalyzable_reasons: BTreeMap<UnanalyzableLabel, u64>,
/// One entry per event name any pinned condition references.
pub per_event: BTreeMap<String, EventCensus>,
}
impl ConditionCensus {
pub fn total(&self) -> u64 {
self.by_class.values().sum()
}
pub fn count(&self, class: ConditionClass) -> u64 {
self.by_class.get(&class).copied().unwrap_or(0)
}
/// The share of *property-filtered* conditions a projection could narrow.
///
/// Event-only conditions are excluded from both halves rather than counted as wins. They are
/// the majority of conditions on a real catalog and a small minority of the rows scanned, so
/// including them produces a high number that says nothing about the work a projection would
/// save — which is the ranking error this census exists to avoid repeating. Zero when there is
/// nothing to classify, so an empty run reports no misleading fraction.
pub fn property_projectable_fraction(&self) -> f64 {
let projectable = self.count(ConditionClass::Projectable);
let denominator = projectable
+ self.count(ConditionClass::FullColumns)
+ self.count(ConditionClass::Unanalyzable);
if denominator == 0 {
return 0.0;
}
projectable as f64 / denominator as f64
}
/// The event names a projection could narrow, meaning every condition on them is narrow.
pub fn projection_eligible_event_names(&self) -> Vec<&str> {
self.per_event
.iter()
.filter(|(_, event)| event.is_projection_eligible())
.map(|(name, _)| name.as_str())
.collect()
}
/// The event names a projection could not narrow, each with why. This is the list to join
/// against per-event row volume: an event here costs full columns however many of its
/// conditions are narrow.
pub fn blocked_event_names(&self) -> Vec<(&str, &BTreeSet<String>)> {
self.per_event
.iter()
.filter(|(_, event)| !event.is_projection_eligible())
.map(|(name, event)| (name.as_str(), &event.blockers))
.collect()
}
/// The blocked event names and their reasons as one compact line, for the per-run log.
///
/// Capped at [`MAX_RENDERED_BLOCKED_EVENTS`], with the remainder counted rather than named. The
/// names are customer-defined and a catalog's event vocabulary has no ceiling, so an uncapped
/// line is a log entry whose size the operator does not control. The head is what gets acted
/// on; [`Self::render_all_blocked_events`] is where the elided names are.
pub fn render_blocked_events(&self) -> String {
let blocked = self.blocked_event_names();
let mut rendered =
render_event_sets(blocked.iter().take(MAX_RENDERED_BLOCKED_EVENTS).copied());
if let Some(elided) = blocked.len().checked_sub(MAX_RENDERED_BLOCKED_EVENTS) {
if elided > 0 {
rendered.push_str(&format!(" (+{elided} more)"));
}
}
rendered
}
/// Every blocked event name with its reasons, uncapped, for the per-run debug log.
///
/// This is the list to join against per-event row volume, so a run whose blocked set outgrows
/// the capped line still has its whole set somewhere. The debug level is what makes the
/// uncapped size acceptable.
pub fn render_all_blocked_events(&self) -> String {
render_event_sets(self.blocked_event_names())
}
/// The read sets as one compact line per event name, for the per-run debug log. Events that
/// read nothing are absent; a blocked one is named by [`Self::render_all_blocked_events`].
pub fn render_reads(&self) -> String {
render_event_sets(
self.per_event
.iter()
.filter(|(_, event)| !event.reads.is_empty())
.map(|(event_name, event)| (event_name.as_str(), &event.reads)),
)
}
}
/// `name=[value,value] name=[value]` — the shape both per-event log lines render in.
fn render_event_sets<'a>(
events: impl IntoIterator<Item = (&'a str, &'a BTreeSet<String>)>,
) -> String {
events
.into_iter()
.map(|(event_name, values)| {
format!(
"{event_name}=[{}]",
values.iter().cloned().collect::<Vec<_>>().join(",")
)
})
.collect::<Vec<_>>()
.join(" ")
}
#[cfg(test)]
mod tests {
use cohort_core::filters::{CohortId, TeamFiltersBuilder, TeamId};
use serde_json::json;
use super::*;
use crate::domain::{BlobSource, Lookback, PropertiesSource};
fn hash(value: &str) -> ConditionHash {
ConditionHash::parse(value).expect("test hashes are 16 ASCII bytes")
}
fn condition(hash_value: &str, event_name: &str) -> PinnedCondition {
PinnedCondition {
cohort_id: CohortId(1),
hash: hash(hash_value),
event_name: event_name.to_owned(),
lookback: Lookback::SlidingDays(7),
}
}
/// Build a frozen catalog holding one behavioral leaf per `(hash, event, bytecode)` triple.
fn filters(leaves: &[(&str, &str, serde_json::Value)]) -> TeamFilters {
let values = leaves
.iter()
.map(|(hash_value, event_name, bytecode)| {
json!({
"type": "behavioral",
"value": "performed_event",
"key": event_name,
"conditionHash": hash_value,
"time_value": 7,
"time_interval": "day",
"bytecode": bytecode,
})
})
.collect::<Vec<_>>();
let mut builder = TeamFiltersBuilder::default();
builder
.add_cohort(
CohortId(1),
TeamId(2),
&json!({ "properties": { "type": "AND", "values": values } }),
)
.expect("the test catalog parses");
builder.freeze(chrono_tz::UTC)
}
fn event_equality(event_name: &str) -> serde_json::Value {
json!(["_H", 1, 32, event_name, 32, "event", 1, 1, 11])
}
/// `properties.plan == 'paid' AND event == 'purchase'`, in the compiler's emission order.
fn property_condition() -> serde_json::Value {
json!([
"_H",
1,
32,
"paid",
32,
"plan",
32,
"properties",
1,
2,
11,
32,
"purchase",
32,
"event",
1,
1,
11,
3,
2
])
}
/// `properties.<key> == <literal> AND event == 'purchase'`.
fn property_equality(key: &str, literal: &str) -> serde_json::Value {
json!([
"_H",
1,
32,
literal,
32,
key,
32,
"properties",
1,
2,
11,
32,
"purchase",
32,
"event",
1,
1,
11,
3,
2
])
}
/// A bare `properties` root: understood, but every column is needed.
fn bare_properties() -> serde_json::Value {
json!(["_H", 1, 32, "properties", 1, 1, 35])
}
/// A `CALLABLE`, which introduces a frame the model refuses.
fn unanalyzable() -> serde_json::Value {
json!(["_H", 1, 52, "f", 0, 0, 0])
}
#[test]
fn each_bytecode_shape_lands_in_its_class_with_its_read_set() {
let conditions = [
condition("eventonly0000000", "purchase"),
condition("projectable00000", "signup"),
condition("fullcolumns00000", "checkout"),
condition("unanalyzable0000", "refund"),
];
let catalog = filters(&[
("eventonly0000000", "purchase", event_equality("purchase")),
("projectable00000", "signup", property_condition()),
("fullcolumns00000", "checkout", bare_properties()),
("unanalyzable0000", "refund", unanalyzable()),
]);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(census.count(ConditionClass::EventOnly), 1);
assert_eq!(census.count(ConditionClass::Projectable), 1);
assert_eq!(census.count(ConditionClass::FullColumns), 1);
assert_eq!(census.count(ConditionClass::Unanalyzable), 1);
assert_eq!(census.total(), 4);
// One projectable against one bare-root and one refused: the event-only condition is not a
// property filter and takes no part in the ratio.
assert_eq!(census.property_projectable_fraction(), 1.0 / 3.0);
assert_eq!(
census.unanalyzable_reasons,
BTreeMap::from([(
UnanalyzableLabel {
reason: "unsupported_op",
op: Some(Arc::from("Callable")),
},
1
)])
);
assert_eq!(
census.projection_eligible_event_names(),
["purchase", "signup"]
);
assert_eq!(
census.render_blocked_events(),
"checkout=[bare_properties_root] refund=[unsupported_op:Callable]"
);
// The event-only condition is still a narrow read, so its `event` read is reported too.
assert_eq!(
census.render_reads(),
"purchase=[event] signup=[event,properties.plan]"
);
}
/// One narrow and one wide condition on the same event name. A scan selects columns for the
/// whole event, so that event is not projection-eligible however narrow its other condition is.
/// Counting conditions alone would report it as half projectable and overstate the win on
/// exactly the busy events that carry several conditions.
#[test]
fn an_event_name_with_one_wide_condition_is_not_projection_eligible() {
let conditions = [
condition("narrow0000000000", "$pageview"),
condition("wide000000000000", "$pageview"),
];
let catalog = filters(&[
("narrow0000000000", "$pageview", property_condition()),
("wide000000000000", "$pageview", bare_properties()),
]);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(census.count(ConditionClass::Projectable), 1);
assert_eq!(census.count(ConditionClass::FullColumns), 1);
let event = &census.per_event["$pageview"];
assert_eq!(event.projectable, 1);
assert_eq!(event.unprojectable, 1);
assert!(!event.is_projection_eligible());
assert_eq!(census.projection_eligible_event_names(), Vec::<&str>::new());
assert_eq!(
census.blocked_event_names(),
[(
"$pageview",
&BTreeSet::from(["bare_properties_root".to_owned()])
)]
);
}
/// The opcode that stopped the analysis is carried alongside the coarse reason. Without it,
/// every branch, frame, and heap write reads as one "unsupported_op" bucket, and the census
/// cannot say whether the blocked conditions are one fixable compiler template or many.
#[test]
fn an_unsupported_opcode_is_reported_with_the_opcode() {
let conditions = [
condition("callable00000000", "a"),
condition("incohort00000000", "b"),
];
let catalog = filters(&[
("callable00000000", "a", unanalyzable()),
// `IN_COHORT` reads membership state that is not in the globals dict at all.
("incohort00000000", "b", json!(["_H", 1, 29, 29, 27])),
]);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(
census.unanalyzable_reasons,
BTreeMap::from([
(
UnanalyzableLabel {
reason: "unsupported_op",
op: Some(Arc::from("Callable")),
},
1
),
(
UnanalyzableLabel {
reason: "unsupported_op",
op: Some(Arc::from("InCohort")),
},
1
),
])
);
assert_eq!(
census.render_blocked_events(),
"a=[unsupported_op:Callable] b=[unsupported_op:InCohort]"
);
}
/// A hash the frozen catalog carries no bytecode for never reached the analyzer, so it is
/// counted under its own reason rather than blamed on the analysis.
#[test]
fn a_condition_without_bytecode_is_counted_as_missing_rather_than_analyzed() {
let conditions = [condition("nobytecode000000", "purchase")];
let census = ConditionAnalyses::build(&conditions, &filters(&[])).census(&conditions);
assert_eq!(census.count(ConditionClass::Unanalyzable), 1);
assert_eq!(
census.unanalyzable_reasons,
BTreeMap::from([(UnanalyzableLabel::missing_bytecode(), 1)])
);
assert!(!census.per_event["purchase"].is_projection_eligible());
}
/// One condition shared by several cohorts is one piece of work. Counting it per pinned entry
/// would overstate the projectable share on exactly the catalogs where sharing is common.
#[test]
fn a_hash_shared_across_cohorts_is_counted_once() {
let shared = "shared0000000000";
let mut first = condition(shared, "purchase");
first.cohort_id = CohortId(1);
let mut second = condition(shared, "purchase");
second.cohort_id = CohortId(2);
let conditions = [first, second];
let catalog = filters(&[(shared, "purchase", event_equality("purchase"))]);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(census.total(), 1);
assert_eq!(census.count(ConditionClass::EventOnly), 1);
assert_eq!(census.per_event["purchase"].event_only, 1);
}
/// The same hash under two event names has to account against both. The hash and the event name
/// are independent fields of the Django payload, so nothing structural pairs them; a census that
/// skipped the repeat would leave the second event with no row, and an event whose only other
/// conditions are narrow would then read as projection-eligible while carrying a wide one.
#[test]
fn a_hash_reused_under_two_event_names_accounts_against_both() {
let shared = "shared0000000000";
let conditions = [condition(shared, "purchase"), condition(shared, "signup")];
let catalog = filters(&[(shared, "purchase", bare_properties())]);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(
census.blocked_event_names(),
[
(
"purchase",
&BTreeSet::from(["bare_properties_root".to_owned()])
),
(
"signup",
&BTreeSet::from(["bare_properties_root".to_owned()])
),
]
);
}
/// Past the cap the info line counts the rest instead of naming them, and the uncapped
/// renderer still names every one. An event blocked only by a wide condition reads nothing, so
/// it is absent from the read dump; without the uncapped list it would appear in no output at
/// all, and the blocked set is the census's whole deliverable.
#[test]
fn blocked_events_past_the_cap_are_counted_in_the_info_line_and_named_in_full() {
let over_cap = MAX_RENDERED_BLOCKED_EVENTS + 3;
// Zero-padded so the rendered order matches the generated one.
let hashes = (0..over_cap)
.map(|index| format!("wide{index:012}"))
.collect::<Vec<_>>();
let names = (0..over_cap)
.map(|index| format!("event_{index:03}"))
.collect::<Vec<_>>();
let conditions = hashes
.iter()
.zip(&names)
.map(|(hash, name)| condition(hash, name))
.collect::<Vec<_>>();
let leaves = hashes
.iter()
.zip(&names)
.map(|(hash, name)| (hash.as_str(), name.as_str(), bare_properties()))
.collect::<Vec<_>>();
let catalog = filters(&leaves);
let census = ConditionAnalyses::build(&conditions, &catalog).census(&conditions);
assert_eq!(census.blocked_event_names().len(), over_cap);
let capped = census.render_blocked_events();
assert!(capped.ends_with(" (+3 more)"), "{capped}");
assert!(!capped.contains(&names[over_cap - 1]), "{capped}");
let full = census.render_all_blocked_events();
for name in &names {
assert!(full.contains(name), "{name} is named nowhere");
}
assert!(!full.contains("more)"), "the uncapped line does not elide");
// Nothing narrow was found, so the read dump cannot carry these names either.
assert_eq!(census.render_reads(), "");
}
/// A chunk's projection is derived from the conditions active on its day, not from the run's
/// whole list. A window that has slid past the day cannot affect its evaluation, so the wide
/// read of a condition outside it must not cost that chunk its narrow scan.
#[test]
fn a_chunks_projection_ignores_conditions_that_are_not_active_on_it() {
let conditions = [
condition("narrow0000000000", "signup"),
condition("wide000000000000", "checkout"),
];
let catalog = filters(&[
("narrow0000000000", "signup", property_condition()),
("wide000000000000", "checkout", bare_properties()),
]);
let analyses = ConditionAnalyses::build(&conditions, &catalog);
let narrow_only = ActiveConditions::new([hash("narrow0000000000")]);
let ChunkProjection::Projected(plan) = analyses.projection(&narrow_only) else {
panic!("the narrow condition alone must project");
};
match &plan.properties {
PropertiesSource::Blob(BlobSource::Keys(keys)) => {
assert_eq!(keys.iter().collect::<Vec<_>>(), ["plan"]);
}
other => panic!("expected key-filtered properties, got {other:?}"),
}
let both = ActiveConditions::new([hash("narrow0000000000"), hash("wide000000000000")]);
assert_eq!(analyses.projection(&both), ChunkProjection::FullColumns);
}
#[test]
fn a_key_is_column_exact_only_when_every_active_condition_reading_it_is() {
let conditions = [
condition("exactplan0000000", "purchase"),
condition("inexactplan00000", "purchase"),
condition("exacturl00000000", "purchase"),
condition("wide000000000000", "checkout"),
];
let catalog = filters(&[
(
"exactplan0000000",
"purchase",
property_equality("plan", "paid"),
),
(
"inexactplan00000",
"purchase",
property_equality("plan", "true"),
),
(
"exacturl00000000",
"purchase",
property_equality("$current_url", "https://example.com/"),
),
("wide000000000000", "checkout", bare_properties()),
]);
let analyses = ConditionAnalyses::build(&conditions, &catalog);
let exact_on = |active: &[&str]| {
analyses.column_exact_keys(&ActiveConditions::new(
active.iter().map(|value| hash(value)),
))
};
let exact_keys = |active: &[&str]| {
exact_on(active)
.exact_keys()
.map(str::to_owned)
.collect::<Vec<_>>()
};
assert_eq!(
exact_keys(&["exactplan0000000", "inexactplan00000", "exacturl00000000"]),
["$current_url"]
);
assert_eq!(
exact_keys(&["exactplan0000000", "exacturl00000000"]),
["$current_url", "plan"]
);
assert_eq!(
exact_keys(&["exacturl00000000", "wide000000000000"]),
Vec::<String>::new(),
"a condition reading the whole object keeps every key on the blob"
);
}
/// An active hash the frozen catalog carried no bytecode for never reached the analyzer, so its
/// reads are unknown and the chunk has to take every column.
#[test]
fn an_active_hash_without_an_analysis_projects_full_columns() {
let conditions = [condition("nobytecode000000", "purchase")];
let analyses = ConditionAnalyses::build(&conditions, &filters(&[]));
let active = ActiveConditions::new([hash("nobytecode000000")]);
assert_eq!(analyses.projection(&active), ChunkProjection::FullColumns);
}
#[test]
fn a_run_with_no_conditions_reports_an_empty_census() {
let census = ConditionAnalyses::default().census(&[]);
assert_eq!(census, ConditionCensus::default());
assert_eq!(census.total(), 0);
assert_eq!(census.property_projectable_fraction(), 0.0);
assert_eq!(census.render_reads(), "");
assert_eq!(census.render_blocked_events(), "");
}
}