Skip to content

Apache DataFusion SQL Query Engine

Add sorted TopK TPC-H benchmark target (#23003)

Geoffrey Claude88fb94d88251
diff --git a/benchmarks/README.md b/benchmarks/README.md
index d143de662e..53a43755f4 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -558,7 +558,15 @@ Test performance of end-to-end sort SQL queries. (While the `Sort` benchmark foc
Sort integration benchmark runs whole table sort queries on TPCH `lineitem` table, with different characteristics. For example, different number of sort keys, different sort key cardinality, different number of payload columns, etc.
-If the TPCH tables have been converted as sorted on their first column (see [Sorted Conversion](#sorted-conversion)), you can use the `--sorted` flag to indicate that the input data is pre-sorted, allowing DataFusion to leverage that order during query execution.
+The `--sorted` flag does not sort or rewrite the input files. It declares that the `lineitem` Parquet input is already sorted ascending by its first column (`l_orderkey`). DataFusion can then leverage that ordering during query execution.
+
+To generate the expected TPC-H SF=1 Parquet input for this benchmark, run:
+
+```bash
+./bench.sh data tpch
+```
+
+For the `lineitem` table used by `sort-tpch`, this uses `tpchgen-cli` to generate Parquet data that is already ordered by `l_orderkey`. If you use a different input directory, only pass `--sorted` when the `lineitem` files already have that ordering.
Additionally, an optional `--limit` flag is available for the sort benchmark. When specified, this flag appends a `LIMIT n` clause to the SQL query, effectively converting the query into a TopK query. Combining the `--sorted` and `--limit` options enables benchmarking of TopK queries on pre-sorted inputs.
@@ -578,7 +586,7 @@ See [`sort_tpch.rs`](src/sort_tpch.rs) for more details.
cargo run --release --bin dfbench -- sort-tpch -p './datafusion/benchmarks/data/tpch_sf1' -o '/tmp/sort_tpch.json' --query 2
```
-3. Run all queries as TopK queries on presorted data:
+3. Run all queries as TopK queries on already sorted data:
```bash
cargo run --release --bin dfbench -- sort-tpch --sorted --limit 10 -p './datafusion/benchmarks/data/tpch_sf1' -o '/tmp/sort_tpch.json'
@@ -598,6 +606,14 @@ In addition, topk_tpch is available from the bench.sh script:
./bench.sh run topk_tpch
```
+To benchmark TopK queries on TPC-H `lineitem` input ordered by `l_orderkey`, use:
+
+```bash
+./bench.sh run topk_sorted_tpch
+```
+
+This runs `dfbench sort-tpch --sorted --limit 100` through the benchmark script, using `--sorted` to declare the existing `l_orderkey` ordering.
+
## IMDB
Run Join Order Benchmark (JOB) on IMDB dataset.
diff --git a/benchmarks/bench.sh b/benchmarks/bench.sh
index 0e28beadf8..52b78c844a 100755
--- a/benchmarks/bench.sh
+++ b/benchmarks/bench.sh
@@ -99,6 +99,7 @@ tpcds: TPCDS inspired benchmark on Scale Factor (SF) 1 (~1GB),
sort_tpch: Benchmark of sorting speed for end-to-end sort queries on TPC-H dataset (SF=1)
sort_tpch10: Benchmark of sorting speed for end-to-end sort queries on TPC-H dataset (SF=10)
topk_tpch: Benchmark of top-k (sorting with limit) queries on TPC-H dataset (SF=1)
+topk_sorted_tpch: Benchmark of top-k queries on TPC-H lineitem ordered by l_orderkey (SF=1)
push_down_topk: Benchmark of ORDER BY ... LIMIT over outer joins on TPC-H dataset (SF=1) — exercises pushing TopK through a join
external_aggr: External aggregation benchmark on TPC-H dataset (SF=1)
wide_schema: Small-projection queries on a wide synthetic dataset (1024 cols × 256 files) — measures per-file metadata overhead
@@ -346,7 +347,7 @@ main() {
# same data as for tpch10
data_tpch "10" "parquet"
;;
- topk_tpch)
+ topk_tpch|topk_sorted_tpch)
# same data as for tpch
data_tpch "1" "parquet"
;;
@@ -577,6 +578,9 @@ main() {
topk_tpch)
run_topk_tpch
;;
+ topk_sorted_tpch)
+ run_topk_sorted_tpch
+ ;;
push_down_topk)
run_push_down_topk
;;
@@ -1506,6 +1510,16 @@ run_topk_tpch() {
$CARGO_COMMAND --bin dfbench -- sort-tpch --iterations 5 --path "${TPCH_DIR}" -o "${RESULTS_FILE}" --limit 100 ${QUERY_ARG} ${LATENCY_ARG}
}
+# Runs the sorted sort tpch integration benchmark with limit 100 (topk)
+run_topk_sorted_tpch() {
+ TPCH_DIR="${DATA_DIR}/tpch_sf1"
+ RESULTS_FILE="${RESULTS_DIR}/run_topk_sorted_tpch.json"
+ echo "RESULTS_FILE: ${RESULTS_FILE}"
+ echo "Running sorted topk tpch benchmark..."
+
+ $CARGO_COMMAND --bin dfbench -- sort-tpch --iterations 5 --path "${TPCH_DIR}" -o "${RESULTS_FILE}" --sorted --limit 100 ${QUERY_ARG} ${LATENCY_ARG}
+}
+
# Runs the nlj benchmark
run_nlj() {
RESULTS_FILE="${RESULTS_DIR}/nlj.json"
diff --git a/benchmarks/src/sort_tpch.rs b/benchmarks/src/sort_tpch.rs
index 338afec0e8..2182d1a383 100644
--- a/benchmarks/src/sort_tpch.rs
+++ b/benchmarks/src/sort_tpch.rs
@@ -64,8 +64,8 @@ pub struct RunOpt {
#[arg(short = 'm', long = "mem-table")]
mem_table: bool,
- /// Mark the first column of each table as sorted in ascending order.
- /// The tables should have been created with the `--sort` option for this to have any effect.
+ /// Declare that the first column of the input table is already sorted in ascending order.
+ /// This flag only attaches ordering metadata; it does not sort the input files.
#[arg(short = 't', long = "sorted")]
sorted: bool,