Apache DataFusion SQL Query Engine
Sort pushdown SQL benchmarks
Run the existing sort pushdown queries through benchmark_runner. This suite
uses the same Parquet files, with queries inlined in each .benchmark file.
The dfbench implementation and its commands remain available for comparison.
From benchmarks/, prepare data with the existing generators:
./bench.sh data sort_pushdown
./bench.sh data sort_pushdown_inexact
The second command also generates the overlap dataset and requires PyArrow. The SQL suite does not generate or rewrite data.
| SQL subgroup | Existing benchmark | Dataset directory | Queries |
|---|---|---|---|
baseline |
sort_pushdown |
sort_pushdown |
1–8 |
sorted |
sort_pushdown_sorted |
sort_pushdown |
1–8 |
inexact |
sort_pushdown_inexact |
sort_pushdown_inexact |
1–4 |
inexact_unsorted |
sort_pushdown_inexact_unsorted |
sort_pushdown_inexact |
1–6 |
inexact_overlap |
sort_pushdown_inexact_overlap |
sort_pushdown_inexact_overlap |
1–4 |
Run the standalone SQL suite directly:
cargo run --release --bin benchmark_runner -- sort_pushdown --iterations 5
Use --subgroup and --query to select a variant and original query number:
cargo run --release --bin benchmark_runner -- sort_pushdown \
--subgroup sorted --query 2 --iterations 5
Omit --subgroup to run all 30 cases. --path points to the parent directory
containing the dataset directories in the table above; it defaults to
benchmarks/data. Each dataset must contain a lineitem/ directory.
The suite forces statistics collection, as the legacy runner does. The three
inexact subgroups also enable Parquet filter pushdown, matching their bench.sh
commands. The ordered subgroups declare l_orderkey ASC NULLS LAST, including
for queries that request descending output. Other settings use the common
runner options and environment variables.