Apache DataFusion SQL Query Engine
docs: add HashJoinExec metrics to user guide (#24055)
Yuwei Zhao9a83691aadaa
diff --git a/datafusion/physical-plan/src/joins/utils.rs b/datafusion/physical-plan/src/joins/utils.rsindex 20467a7ec5..ecf056560e 100644--- a/datafusion/physical-plan/src/joins/utils.rs+++ b/datafusion/physical-plan/src/joins/utils.rs@@ -1754,6 +1754,7 @@ fn append_probe_indices_in_order(/// Metrics for build & probe joins#[derive(Clone, Debug)]pub(crate) struct BuildProbeJoinMetrics {+ // Keep these metric descriptions in sync with docs/source/user-guide/metrics.md.pub(crate) baseline: BaselineMetrics,/// Total time for collecting build-side of joinpub(crate) build_time: metrics::Time,@@ -1763,15 +1764,17 @@ pub(crate) struct BuildProbeJoinMetrics {pub(crate) build_input_rows: metrics::Count,/// Memory used by build-side in bytespub(crate) build_mem_used: metrics::Gauge,- /// Total time for joining probe-side batches to the build-side batches+ /// Total time for join processing after build-side collectionpub(crate) join_time: metrics::Time,/// Number of batches consumed by probe-side of this operatorpub(crate) input_batches: metrics::Count,/// Number of rows consumed by probe-side this operatorpub(crate) input_rows: metrics::Count,- /// Fraction of probe rows that found more than one match+ /// Fraction of probe rows with at least one build-side join-key match before+ /// applying any join filterpub(crate) probe_hit_rate: metrics::RatioMetrics,- /// Average number of build matches per matched probe row+ /// Average number of build-side join-key matches per matched probe row before+ /// applying any join filterpub(crate) avg_fanout: metrics::RatioMetrics,}diff --git a/docs/source/user-guide/metrics.md b/docs/source/user-guide/metrics.mdindex 7e0363f4ce..306094c438 100644--- a/docs/source/user-guide/metrics.md+++ b/docs/source/user-guide/metrics.md@@ -42,6 +42,25 @@ DataFusion operators expose runtime metrics so you can understand where time is| ----------- | ----------------------------------------------------------------- || selectivity | Selectivity of the filter, calculated as output_rows / input_rows |+### HashJoinExec++`HashJoinExec` also exposes the common `BaselineMetrics`. Its+`elapsed_compute` metric is the sum of the build-side collection time and the+subsequent join processing time.++| Metric | Description |+| ----------------------- | --------------------------------------------------------------------------------------------------------- |+| build_time | Total time spent collecting and building the build side of the join. |+| build_input_batches | Number of input batches consumed from the build side. |+| build_input_rows | Number of input rows consumed from the build side. |+| build_mem_used | Peak tracked memory used by the build side, in bytes. |+| join_time | Total time spent processing the join after build-side collection. |+| input_batches | Number of input batches consumed from the probe side. |+| input_rows | Number of input rows consumed from the probe side. |+| probe_hit_rate | Fraction of probe-side rows with a build-side join-key match before applying any join filter. |+| avg_fanout | Average number of build-side join-key matches per matched probe-side row before applying any join filter. |+| array_map_created_count | Number of times `HashJoinExec` created an `ArrayMap` for perfect hash join lookup execution. |+## TODOAdd metrics for the remaining operators