Skip to content

Apache DataFusion SQL Query Engine

chore: Add TPCDS benchmarks (#19138)

Oleks Ve8384fbbf591
diff --git a/benchmarks/README.md b/benchmarks/README.md
index 13b73074e0..bf9b50028b 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -119,7 +119,6 @@ You can also invoke the helper directly if you need to customise arguments furth
./benchmarks/compile_profile.py --profiles dev release --data /path/to/tpch_sf1
```
-
## Benchmark with modified configurations
### Select join algorithm
@@ -147,6 +146,19 @@ To verify that datafusion picked up your configuration, run the benchmarks with
## Comparing performance of main and a branch
+For TPC-H
+```shell
+./benchmarks/compare_tpch.sh main mybranch
+```
+
+For TPC-DS.
+To get data in `DATA_DIR` for TPCDS, please follow instructions in `./benchmarks/bench.sh data tcpds`
+```shell
+DATA_DIR=../../datafusion-benchmarks/tpcds/data/sf1/ ./benchmarks/compare_tpcds.sh main mybranch
+```
+
+Alternatively you can compare manually followng the example velor
+
```shell
git checkout main
@@ -299,7 +311,6 @@ This will produce output like:
└──────────────┴──────────────┴──────────────┴───────────────┘
```
-
# Benchmark Runner
The `dfbench` program contains subcommands to run the various
@@ -339,24 +350,28 @@ FLAGS:
```
# Profiling Memory Stats for each benchmark query
+
The `mem_profile` program wraps benchmark execution to measure memory usage statistics, such as peak RSS. It runs each benchmark query in a separate subprocess, capturing the child process’s stdout to print structured output.
Subcommands supported by mem_profile are the subset of those in `dfbench`.
-Currently supported benchmarks include: Clickbench, H2o, Imdb, SortTpch, Tpch
+Currently supported benchmarks include: Clickbench, H2o, Imdb, SortTpch, Tpch, TPCDS
Before running benchmarks, `mem_profile` automatically compiles the benchmark binary (`dfbench`) using `cargo build`. Note that the build profile used for `dfbench` is not tied to the profile used for running `mem_profile` itself. We can explicitly specify the desired build profile using the `--bench-profile` option (e.g. release-nonlto). By prebuilding the binary and running each query in a separate process, we can ensure accurate memory statistics.
Currently, `mem_profile` only supports `mimalloc` as the memory allocator, since it relies on `mimalloc`'s API to collect memory statistics.
-Because it runs the compiled binary directly from the target directory, make sure your working directory is the top-level datafusion/ directory, where the target/ is also located.
+Because it runs the compiled binary directly from the target directory, make sure your working directory is the top-level datafusion/ directory, where the target/ is also located.
+
+The benchmark subcommand (e.g., `tpch`) and all following arguments are passed directly to `dfbench`. Be sure to specify `--bench-profile` before the benchmark subcommand.
-The benchmark subcommand (e.g., `tpch`) and all following arguments are passed directly to `dfbench`. Be sure to specify `--bench-profile` before the benchmark subcommand.
+Example:
-Example:
```shell
datafusion$ cargo run --profile release-nonlto --bin mem_profile -- --bench-profile release-nonlto tpch --path benchmarks/data/tpch_sf1 --partitions 4 --format parquet
```
+
Example Output:
+
```
Query Time (ms) Peak RSS Peak Commit Major Page Faults
----------------------------------------------------------------
@@ -385,19 +400,21 @@ Query Time (ms) Peak RSS Peak Commit Major Page Faults
```
## Reported Metrics
+
When running benchmarks, `mem_profile` collects several memory-related statistics using the mimalloc API:
-- Peak RSS (Resident Set Size):
-The maximum amount of physical memory used by the process.
-This is a process-level metric collected via OS-specific mechanisms and is not mimalloc-specific.
+- Peak RSS (Resident Set Size):
+ The maximum amount of physical memory used by the process.
+ This is a process-level metric collected via OS-specific mechanisms and is not mimalloc-specific.
- Peak Commit:
-The peak amount of memory committed by the allocator (i.e., total virtual memory reserved).
-This is mimalloc-specific. It gives a more allocator-aware view of memory usage than RSS.
+ The peak amount of memory committed by the allocator (i.e., total virtual memory reserved).
+ This is mimalloc-specific. It gives a more allocator-aware view of memory usage than RSS.
- Major Page Faults:
-The number of major page faults triggered during execution.
-This metric is obtained from the operating system and is not mimalloc-specific.
+ The number of major page faults triggered during execution.
+ This metric is obtained from the operating system and is not mimalloc-specific.
+
# Writing a new benchmark
## Creating or downloading data outside of the benchmark
@@ -586,6 +603,34 @@ This benchmarks is derived from the [TPC-H][1] version
[2]: https://github.com/databricks/tpch-dbgen.git,
[2.17.1]: https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-h_v2.17.1.pdf
+## TPCDS
+
+Run the tpcds benchmark.
+
+For data please clone `datafusion-benchmarks` repo which contains the predefined parquet data with SF1.
+
+```shell
+git clone https://github.com/apache/datafusion-benchmarks
+```
+
+Then run the benchmark with the following command:
+
+```shell
+DATA_DIR=../../datafusion-benchmarks/tpcds/data/sf1/ ./benchmarks/bench.sh run tpcds
+```
+
+Alternatively benchmark the specific query
+
+```shell
+DATA_DIR=../../datafusion-benchmarks/tpcds/data/sf1/ ./benchmarks/bench.sh run tpcds 30
+```
+
+More help
+
+```shell
+cargo run --release --bin dfbench -- tpcds --help
+```
+
## External Aggregation
Run the benchmark for aggregations with limited memory.
diff --git a/benchmarks/bench.sh b/benchmarks/bench.sh
index 8c0a925232..6b94dadeb9 100755
--- a/benchmarks/bench.sh
+++ b/benchmarks/bench.sh
@@ -87,6 +87,9 @@ tpch10: TPCH inspired benchmark on Scale Factor (SF) 10 (~10GB),
tpch_csv10: TPCH inspired benchmark on Scale Factor (SF) 10 (~10GB), single csv file per table, hash join
tpch_mem10: TPCH inspired benchmark on Scale Factor (SF) 10 (~10GB), query from memory
+# TPC-DS Benchmarks
+tpcds: TPCDS inspired benchmark on Scale Factor (SF) 1 (~1GB), single parquet file per table, hash join
+
# Extended TPC-H Benchmarks
sort_tpch: Benchmark of sorting speed for end-to-end sort queries on TPC-H dataset (SF=1)
sort_tpch10: Benchmark of sorting speed for end-to-end sort queries on TPC-H dataset (SF=10)
@@ -220,6 +223,9 @@ main() {
tpch_csv10)
data_tpch "10" "csv"
;;
+ tpcds)
+ data_tpcds
+ ;;
clickbench_1)
data_clickbench_1
;;
@@ -388,6 +394,7 @@ main() {
run_external_aggr
run_nlj
run_hj
+ run_tpcds
;;
tpch)
run_tpch "1" "parquet"
@@ -407,6 +414,9 @@ main() {
tpch_mem10)
run_tpch_mem "10"
;;
+ tpcds)
+ run_tpcds
+ ;;
cancellation)
run_cancellation
;;
@@ -601,6 +611,24 @@ data_tpch() {
exit 1
}
+# Points to TPCDS data generation instructions
+data_tpcds() {
+ TPCDS_DIR="${DATA_DIR}"
+
+ # Check if TPCDS data directory exists
+ if [ ! -d "${TPCDS_DIR}" ]; then
+ echo ""
+ echo "For TPC-DS data generation, please clone the datafusion-benchmarks repository:"
+ echo " git clone https://github.com/apache/datafusion-benchmarks"
+ echo ""
+ return 1
+ fi
+
+ echo ""
+ echo "TPC-DS data already exists in ${TPCDS_DIR}"
+ echo ""
+}
+
# Runs the tpch benchmark
run_tpch() {
SCALE_FACTOR=$1
@@ -634,6 +662,37 @@ run_tpch_mem() {
debug_run $CARGO_COMMAND --bin dfbench -- tpch --iterations 5 --path "${TPCH_DIR}" --prefer_hash_join "${PREFER_HASH_JOIN}" -m --format parquet -o "${RESULTS_FILE}" ${QUERY_ARG}
}
+# Runs the tpcds benchmark
+run_tpcds() {
+ TPCDS_DIR="${DATA_DIR}"
+
+ # Check if TPCDS data directory exists
+ if [ ! -d "${TPCDS_DIR}" ]; then
+ echo "Error: TPC-DS data directory does not exist: ${TPCDS_DIR}" >&2
+ echo "" >&2
+ echo "Please prepare TPC-DS data first by following instructions:" >&2
+ echo " ./bench.sh data tpcds" >&2
+ echo "" >&2
+ exit 1
+ fi
+
+ # Check if directory contains parquet files
+ if ! find "${TPCDS_DIR}" -name "*.parquet" -print -quit | grep -q .; then
+ echo "Error: TPC-DS data directory exists but contains no parquet files: ${TPCDS_DIR}" >&2
+ echo "" >&2
+ echo "Please prepare TPC-DS data first by following instructions:" >&2
+ echo " ./bench.sh data tpcds" >&2
+ echo "" >&2
+ exit 1
+ fi
+
+ RESULTS_FILE="${RESULTS_DIR}/tpcds_sf1.json"
+ echo "RESULTS_FILE: ${RESULTS_FILE}"
+ echo "Running tpcds benchmark..."
+
+ debug_run $CARGO_COMMAND --bin dfbench -- tpcds --iterations 5 --path "${TPCDS_DIR}" --query_path "../datafusion/core/tests/tpc-ds" --prefer_hash_join "${PREFER_HASH_JOIN}" -o "${RESULTS_FILE}" ${QUERY_ARG}
+}
+
# Runs the compile profile benchmark helper
run_compile_profile() {
local profiles=("$@")
diff --git a/benchmarks/compare_tpcds.sh b/benchmarks/compare_tpcds.sh
new file mode 100755
index 0000000000..48331a7c75
--- /dev/null
+++ b/benchmarks/compare_tpcds.sh
@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Compare TPC-DS benchmarks between two branches
+
+set -e
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+usage() {
+ echo "Usage: $0 <branch1> <branch2>"
+ echo ""
+ echo "Example: $0 main dev2"
+ echo ""
+ echo "Note: TPC-DS benchmarks are not currently implemented in bench.sh"
+ exit 1
+}
+
+BRANCH1=${1:-""}
+BRANCH2=${2:-""}
+
+if [ -z "$BRANCH1" ] || [ -z "$BRANCH2" ]; then
+ usage
+fi
+
+# Store current branch
+CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
+
+echo "Comparing TPC-DS benchmarks: ${BRANCH1} vs ${BRANCH2}"
+
+# Run benchmark on first branch
+git checkout "$BRANCH1"
+./benchmarks/bench.sh run tpcds
+
+# Run benchmark on second branch
+git checkout "$BRANCH2"
+./benchmarks/bench.sh run tpcds
+
+# Compare results
+./benchmarks/bench.sh compare "$BRANCH1" "$BRANCH2"
+
+# Return to original branch
+git checkout "$CURRENT_BRANCH"
\ No newline at end of file
diff --git a/benchmarks/compare_tpch.sh b/benchmarks/compare_tpch.sh
new file mode 100755
index 0000000000..85e8da29ce
--- /dev/null
+++ b/benchmarks/compare_tpch.sh
@@ -0,0 +1,56 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Compare TPC-H benchmarks between two branches
+
+set -e
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+usage() {
+ echo "Usage: $0 <branch1> <branch2>"
+ echo ""
+ echo "Example: $0 main dev2"
+ exit 1
+}
+
+BRANCH1=${1:-""}
+BRANCH2=${2:-""}
+
+if [ -z "$BRANCH1" ] || [ -z "$BRANCH2" ]; then
+ usage
+fi
+
+# Store current branch
+CURRENT_BRANCH=$(git rev-parse --abbrev-ref HEAD)
+
+echo "Comparing TPC-H benchmarks: ${BRANCH1} vs ${BRANCH2}"
+
+# Run benchmark on first branch
+git checkout "$BRANCH1"
+./benchmarks/bench.sh run tpch
+
+# Run benchmark on second branch
+git checkout "$BRANCH2"
+./benchmarks/bench.sh run tpch
+
+# Compare results
+./benchmarks/bench.sh compare "$BRANCH1" "$BRANCH2"
+
+# Return to original branch
+git checkout "$CURRENT_BRANCH"
\ No newline at end of file
diff --git a/benchmarks/src/bin/dfbench.rs b/benchmarks/src/bin/dfbench.rs
index 68dea188dc..6fc382822d 100644
--- a/benchmarks/src/bin/dfbench.rs
+++ b/benchmarks/src/bin/dfbench.rs
@@ -34,7 +34,7 @@ static ALLOC: snmalloc_rs::SnMalloc = snmalloc_rs::SnMalloc;
static ALLOC: mimalloc::MiMalloc = mimalloc::MiMalloc;
use datafusion_benchmarks::{
- cancellation, clickbench, h2o, hj, imdb, nlj, sort_tpch, tpch,
+ cancellation, clickbench, h2o, hj, imdb, nlj, sort_tpch, tpcds, tpch,
};
#[derive(Debug, StructOpt)]
@@ -48,6 +48,7 @@ enum Options {
Nlj(nlj::RunOpt),
SortTpch(sort_tpch::RunOpt),
Tpch(tpch::RunOpt),
+ Tpcds(tpcds::RunOpt),
}
// Main benchmark runner entrypoint
@@ -64,5 +65,6 @@ pub async fn main() -> Result<()> {
Options::Nlj(opt) => opt.run().await,
Options::SortTpch(opt) => opt.run().await,
Options::Tpch(opt) => Box::pin(opt.run()).await,
+ Options::Tpcds(opt) => Box::pin(opt.run()).await,
}
}
diff --git a/benchmarks/src/lib.rs b/benchmarks/src/lib.rs
index 07cffa5ae4..d885ec94a3 100644
--- a/benchmarks/src/lib.rs
+++ b/benchmarks/src/lib.rs
@@ -23,5 +23,6 @@ pub mod hj;
pub mod imdb;
pub mod nlj;
pub mod sort_tpch;
+pub mod tpcds;
pub mod tpch;
pub mod util;
diff --git a/benchmarks/src/tpcds/mod.rs b/benchmarks/src/tpcds/mod.rs
new file mode 100644
index 0000000000..4829eb9fd3
--- /dev/null
+++ b/benchmarks/src/tpcds/mod.rs
@@ -0,0 +1,19 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+mod run;
+pub use run::RunOpt;
diff --git a/benchmarks/src/tpcds/run.rs b/benchmarks/src/tpcds/run.rs
new file mode 100644
index 0000000000..3b7e4ad135
--- /dev/null
+++ b/benchmarks/src/tpcds/run.rs
@@ -0,0 +1,356 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+use std::fs;
+use std::path::PathBuf;
+use std::sync::Arc;
+
+use crate::util::{print_memory_stats, BenchmarkRun, CommonOpt, QueryResult};
+
+use arrow::record_batch::RecordBatch;
+use arrow::util::pretty::{self, pretty_format_batches};
+use datafusion::datasource::file_format::parquet::ParquetFormat;
+use datafusion::datasource::listing::{
+ ListingOptions, ListingTable, ListingTableConfig, ListingTableUrl,
+};
+use datafusion::datasource::{MemTable, TableProvider};
+use datafusion::error::Result;
+use datafusion::physical_plan::display::DisplayableExecutionPlan;
+use datafusion::physical_plan::{collect, displayable};
+use datafusion::prelude::*;
+use datafusion_common::instant::Instant;
+use datafusion_common::utils::get_available_parallelism;
+use datafusion_common::{plan_err, DEFAULT_PARQUET_EXTENSION};
+
+use log::info;
+use structopt::StructOpt;
+
+// hack to avoid `default_value is meaningless for bool` errors
+type BoolDefaultTrue = bool;
+pub const TPCDS_QUERY_START_ID: usize = 1;
+pub const TPCDS_QUERY_END_ID: usize = 99;
+
+pub const TPCDS_TABLES: &[&str] = &[
+ "call_center",
+ "customer_address",
+ "household_demographics",
+ "promotion",
+ "store_sales",
+ "web_page",
+ "catalog_page",
+ "customer_demographics",
+ "income_band",
+ "reason",
+ "store",
+ "web_returns",
+ "catalog_returns",
+ "customer",
+ "inventory",
+ "ship_mode",
+ "time_dim",
+ "web_sales",
+ "catalog_sales",
+ "date_dim",
+ "item",
+ "store_returns",
+ "warehouse",
+ "web_site",
+];
+
+/// Get the SQL statements from the specified query file
+pub fn get_query_sql(base_query_path: &str, query: usize) -> Result<Vec<String>> {
+ if query > 0 && query < 100 {
+ let filename = format!("{base_query_path}/{query}.sql");
+ let mut errors = vec![];
+ match fs::read_to_string(&filename) {
+ Ok(contents) => {
+ return Ok(contents
+ .split(';')
+ .map(|s| s.trim())
+ .filter(|s| !s.is_empty())
+ .map(|s| s.to_string())
+ .collect());
+ }
+ Err(e) => errors.push(format!("{filename}: {e}")),
+ };
+
+ plan_err!("invalid query. Could not find query: {:?}", errors)
+ } else {
+ plan_err!("invalid query. Expected value between 1 and 99")
+ }
+}
+
+/// Run the tpcds benchmark.
+#[derive(Debug, StructOpt, Clone)]
+#[structopt(verbatim_doc_comment)]
+pub struct RunOpt {
+ /// Query number. If not specified, runs all queries
+ #[structopt(short, long)]
+ pub query: Option<usize>,
+
+ /// Common options
+ #[structopt(flatten)]
+ common: CommonOpt,
+
+ /// Path to data files
+ #[structopt(parse(from_os_str), required = true, short = "p", long = "path")]
+ path: PathBuf,
+
+ /// Path to query files
+ #[structopt(parse(from_os_str), required = true, short = "Q", long = "query_path")]
+ query_path: PathBuf,
+
+ /// Load the data into a MemTable before executing the query
+ #[structopt(short = "m", long = "mem-table")]
+ mem_table: bool,
+
+ /// Path to machine readable output file
+ #[structopt(parse(from_os_str), short = "o", long = "output")]
+ output_path: Option<PathBuf>,
+
+ /// Whether to disable collection of statistics (and cost based optimizations) or not.
+ #[structopt(short = "S", long = "disable-statistics")]
+ disable_statistics: bool,
+
+ /// If true then hash join used, if false then sort merge join
+ /// True by default.
+ #[structopt(short = "j", long = "prefer_hash_join", default_value = "true")]
+ prefer_hash_join: BoolDefaultTrue,
+
+ /// If true then Piecewise Merge Join can be used, if false then it will opt for Nested Loop Join
+ /// False by default.
+ #[structopt(
+ short = "w",
+ long = "enable_piecewise_merge_join",
+ default_value = "false"
+ )]
+ enable_piecewise_merge_join: BoolDefaultTrue,
+
+ /// Mark the first column of each table as sorted in ascending order.
+ /// The tables should have been created with the `--sort` option for this to have any effect.
+ #[structopt(short = "t", long = "sorted")]
+ sorted: bool,
+}
+
+impl RunOpt {
+ pub async fn run(self) -> Result<()> {
+ println!("Running benchmarks with the following options: {self:?}");
+ let query_range = match self.query {
+ Some(query_id) => query_id..=query_id,
+ None => TPCDS_QUERY_START_ID..=TPCDS_QUERY_END_ID,
+ };
+
+ let mut benchmark_run = BenchmarkRun::new();
+ let mut config = self
+ .common
+ .config()?
+ .with_collect_statistics(!self.disable_statistics);
+ config.options_mut().optimizer.prefer_hash_join = self.prefer_hash_join;
+ config.options_mut().optimizer.enable_piecewise_merge_join =
+ self.enable_piecewise_merge_join;
+ let rt_builder = self.common.runtime_env_builder()?;
+ let ctx = SessionContext::new_with_config_rt(config, rt_builder.build_arc()?);
+ // register tables
+ self.register_tables(&ctx).await?;
+
+ for query_id in query_range {
+ benchmark_run.start_new_case(&format!("Query {query_id}"));
+ let query_run = self.benchmark_query(query_id, &ctx).await;
+ match query_run {
+ Ok(query_results) => {
+ for iter in query_results {
+ benchmark_run.write_iter(iter.elapsed, iter.row_count);
+ }
+ }
+ Err(e) => {
+ benchmark_run.mark_failed();
+ eprintln!("Query {query_id} failed: {e}");
+ }
+ }
+ }
+ benchmark_run.maybe_write_json(self.output_path.as_ref())?;
+ benchmark_run.maybe_print_failures();
+ Ok(())
+ }
+
+ async fn benchmark_query(
+ &self,
+ query_id: usize,
+ ctx: &SessionContext,
+ ) -> Result<Vec<QueryResult>> {
+ let mut millis = vec![];
+ // run benchmark
+ let mut query_results = vec![];
+
+ let sql = &get_query_sql(self.query_path.to_str().unwrap(), query_id)?;
+
+ if self.common.debug {
+ println!("=== SQL for query {query_id} ===\n{}\n", sql.join(";\n"));
+ }
+
+ for i in 0..self.iterations() {
+ let start = Instant::now();
+
+ // query 15 is special, with 3 statements. the second statement is the one from which we
+ // want to capture the results
+ let mut result = vec![];
+
+ for query in sql {
+ result = self.execute_query(ctx, query).await?;
+ }
+
+ let elapsed = start.elapsed();
+ let ms = elapsed.as_secs_f64() * 1000.0;
+ millis.push(ms);
+ info!("output:\n\n{}\n\n", pretty_format_batches(&result)?);
+ let row_count = result.iter().map(|b| b.num_rows()).sum();
+ println!(
+ "Query {query_id} iteration {i} took {ms:.1} ms and returned {row_count} rows"
+ );
+ query_results.push(QueryResult { elapsed, row_count });
+ }
+
+ let avg = millis.iter().sum::<f64>() / millis.len() as f64;
+ println!("Query {query_id} avg time: {avg:.2} ms");
+
+ // Print memory stats using mimalloc (only when compiled with --features mimalloc_extended)
+ print_memory_stats();
+
+ Ok(query_results)
+ }
+
+ async fn register_tables(&self, ctx: &SessionContext) -> Result<()> {
+ for table in TPCDS_TABLES {
+ let table_provider = { self.get_table(ctx, table).await? };
+
+ if self.mem_table {
+ println!("Loading table '{table}' into memory");
+ let start = Instant::now();
+ let memtable =
+ MemTable::load(table_provider, Some(self.partitions()), &ctx.state())
+ .await?;
+ println!(
+ "Loaded table '{}' into memory in {} ms",
+ table,
+ start.elapsed().as_millis()
+ );
+ ctx.register_table(*table, Arc::new(memtable))?;
+ } else {
+ ctx.register_table(*table, table_provider)?;
+ }
+ }
+ Ok(())
+ }
+
+ async fn execute_query(
+ &self,
+ ctx: &SessionContext,
+ sql: &str,
+ ) -> Result<Vec<RecordBatch>> {
+ let debug = self.common.debug;
+ let plan = ctx.sql(sql).await?;
+ let (state, plan) = plan.into_parts();
+
+ if debug {
+ println!("=== Logical plan ===\n{plan}\n");
+ }
+
+ let plan = state.optimize(&plan)?;
+ if debug {
+ println!("=== Optimized logical plan ===\n{plan}\n");
+ }
+ let physical_plan = state.create_physical_plan(&plan).await?;
+ if debug {
+ println!(
+ "=== Physical plan ===\n{}\n",
+ displayable(physical_plan.as_ref()).indent(true)
+ );
+ }
+ let result = collect(physical_plan.clone(), state.task_ctx()).await?;
+ if debug {
+ println!(
+ "=== Physical plan with metrics ===\n{}\n",
+ DisplayableExecutionPlan::with_metrics(physical_plan.as_ref())
+ .indent(true)
+ );
+ if !result.is_empty() {
+ // do not call print_batches if there are no batches as the result is confusing
+ // and makes it look like there is a batch with no columns
+ pretty::print_batches(&result)?;
+ }
+ }
+ Ok(result)
+ }
+
+ async fn get_table(
+ &self,
+ ctx: &SessionContext,
+ table: &str,
+ ) -> Result<Arc<dyn TableProvider>> {
+ let path = self.path.to_str().unwrap();
+ let target_partitions = self.partitions();
+
+ // Obtain a snapshot of the SessionState
+ let state = ctx.state();
+ let path = format!("{path}/{table}.parquet");
+
+ // Check if the file exists
+ if !std::path::Path::new(&path).exists() {
+ eprintln!("Warning registering {table}: Table file does not exist: {path}");
+ }
+
+ let format = ParquetFormat::default()
+ .with_options(ctx.state().table_options().parquet.clone());
+
+ let table_path = ListingTableUrl::parse(path)?;
+ let options = ListingOptions::new(Arc::new(format))
+ .with_file_extension(DEFAULT_PARQUET_EXTENSION)
+ .with_target_partitions(target_partitions)
+ .with_collect_stat(state.config().collect_statistics());
+ let schema = options.infer_schema(&state, &table_path).await?;
+
+ if self.common.debug {
+ println!(
+ "Inferred schema from {table_path} for table '{table}':\n{schema:#?}\n"
+ );
+ }
+
+ let options = if self.sorted {
+ let key_column_name = schema.fields()[0].name();
+ options
+ .with_file_sort_order(vec![vec![col(key_column_name).sort(true, false)]])
+ } else {
+ options
+ };
+
+ let config = ListingTableConfig::new(table_path)
+ .with_listing_options(options)
+ .with_schema(schema);
+
+ Ok(Arc::new(ListingTable::try_new(config)?))
+ }
+
+ fn iterations(&self) -> usize {
+ self.common.iterations
+ }
+
+ fn partitions(&self) -> usize {
+ self.common
+ .partitions
+ .unwrap_or_else(get_available_parallelism)
+ }
+}
diff --git a/benchmarks/src/tpch/run.rs b/benchmarks/src/tpch/run.rs
index 63a1cd58d4..84dfc53a5e 100644
--- a/benchmarks/src/tpch/run.rs
+++ b/benchmarks/src/tpch/run.rs
@@ -94,7 +94,11 @@ pub struct RunOpt {
/// If true then Piecewise Merge Join can be used, if false then it will opt for Nested Loop Join
/// False by default.
- #[structopt(long = "enable_piecewise_merge_join", default_value = "false")]
+ #[structopt(
+ short = "w",
+ long = "enable_piecewise_merge_join",
+ default_value = "false"
+ )]
enable_piecewise_merge_join: BoolDefaultTrue,
/// Mark the first column of each table as sorted in ascending order.
diff --git a/datafusion/core/tests/tpc-ds/30.sql b/datafusion/core/tests/tpc-ds/30.sql
index 78f34b807e..80624f4900 100644
--- a/datafusion/core/tests/tpc-ds/30.sql
+++ b/datafusion/core/tests/tpc-ds/30.sql
@@ -14,7 +14,7 @@ with customer_total_return as
,ca_state)
select c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag
,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address
- ,c_last_review_date_sk,ctr_total_return
+ ,c_last_review_date,ctr_total_return
from customer_total_return ctr1
,customer_address
,customer
@@ -26,7 +26,7 @@ with customer_total_return as
and ctr1.ctr_customer_sk = c_customer_sk
order by c_customer_id,c_salutation,c_first_name,c_last_name,c_preferred_cust_flag
,c_birth_day,c_birth_month,c_birth_year,c_birth_country,c_login,c_email_address
- ,c_last_review_date_sk,ctr_total_return
+ ,c_last_review_date,ctr_total_return
limit 100;
diff --git a/test-utils/src/tpcds.rs b/test-utils/src/tpcds.rs
index ce5bac5bfd..a12ae8ceae 100644
--- a/test-utils/src/tpcds.rs
+++ b/test-utils/src/tpcds.rs
@@ -299,7 +299,7 @@ pub fn tpcds_schemas() -> Vec<TableDef> {
Field::new("c_birth_country", DataType::Utf8, false),
Field::new("c_login", DataType::Utf8, false),
Field::new("c_email_address", DataType::Utf8, false),
- Field::new("c_last_review_date_sk", DataType::Utf8, false),
+ Field::new("c_last_review_date", DataType::Utf8, false),
]),
),
TableDef::new(