Skip to content

Apache DataFusion SQL Query Engine

chore: Add TPCDS benchmark comparison for PR (#19552)

Oleks V987b94ca47ab
diff --git a/benchmarks/README.md b/benchmarks/README.md
index 0b71628b2d..7e9818aef2 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -157,7 +157,7 @@ To get data in `DATA_DIR` for TPCDS, please follow instructions in `./benchmarks
DATA_DIR=../../datafusion-benchmarks/tpcds/data/sf1/ ./benchmarks/compare_tpcds.sh main mybranch
```
-Alternatively you can compare manually followng the example velor
+Alternatively, you can compare manually following the example below
```shell
git checkout main
@@ -240,6 +240,23 @@ Benchmark tpch_mem.json
└──────────────┴──────────────┴──────────────┴───────────────┘
```
+## Comparing performance of main and a PR
+
+### TPCDS
+
+Considering you already have TPCDS data locally
+
+```shell
+export DATA_DIR=../../datafusion-benchmarks/tpcds/data/sf1/
+export PR_NUMBER=19464
+git fetch upstream pull/$PR_NUMBER/head:pr-$PR_NUMBER
+git checkout main
+git pull
+./benchmarks/compare_tpcds.sh main pr-$PR_NUMBER
+```
+
+Note: if `gh` is installed, you can also run `gh pr checkout $PR_NUMBER` instead of `git fetch upstream pull/$PR_NUMBER/head:pr-$PR_NUMBER`
+
### Running Benchmarks Manually
Assuming data is in the `data` directory, the `tpch` benchmark can be run with a command like this:
diff --git a/benchmarks/bench.sh b/benchmarks/bench.sh
index d5fa52d7f0..6679405623 100755
--- a/benchmarks/bench.sh
+++ b/benchmarks/bench.sh
@@ -684,7 +684,7 @@ run_tpch_mem() {
# Runs the tpcds benchmark
run_tpcds() {
- TPCDS_DIR="${DATA_DIR}/tpcds_sf1"
+ TPCDS_DIR="${DATA_DIR}"
# Check if TPCDS data directory and representative file exists
if [ ! -f "${TPCDS_DIR}/web_site.parquet" ]; then
diff --git a/datafusion/expr/src/udf.rs b/datafusion/expr/src/udf.rs
index 26d7fc99cb..28a07ad761 100644
--- a/datafusion/expr/src/udf.rs
+++ b/datafusion/expr/src/udf.rs
@@ -24,10 +24,10 @@ use crate::sort_properties::{ExprProperties, SortProperties};
use crate::udf_eq::UdfEq;
use crate::{ColumnarValue, Documentation, Expr, Signature};
use arrow::datatypes::{DataType, Field, FieldRef};
+#[cfg(debug_assertions)]
+use datafusion_common::assert_or_internal_err;
use datafusion_common::config::ConfigOptions;
-use datafusion_common::{
- ExprSchema, Result, ScalarValue, assert_or_internal_err, not_impl_err,
-};
+use datafusion_common::{ExprSchema, Result, ScalarValue, not_impl_err};
use datafusion_expr_common::dyn_eq::{DynEq, DynHash};
use datafusion_expr_common::interval_arithmetic::Interval;
use std::any::Any;