Skip to content

Apache DataFusion SQL Query Engine

docs: Add point queries blog to Good Reads and Tuning Guide (#26751)

Andrew Lamb80dda10eef47
diff --git a/dev/update_config_docs.sh b/dev/update_config_docs.sh
index 5bba39625f..f08ad673ee 100755
--- a/dev/update_config_docs.sh
+++ b/dev/update_config_docs.sh
@@ -172,12 +172,14 @@ cat <<'EOF' >> "$TARGET_FILE"
# Tuning Guide
-## Short Queries
+## Short Queries and Point Lookups
By default DataFusion will attempt to maximize parallelism and use all cores --
For example, if you have 32 cores, each plan will split the data into 32
-partitions. However, if your data is small, the overhead of splitting the data
-to enable parallelization can dominate the actual computation.
+partitions. However, if your data is small, or the query only reads a few rows
+(for example, a point lookup such as `SELECT * FROM t WHERE id = 42`), the
+overhead of planning, opening files, and splitting the data to enable
+parallelization can dominate the actual computation.
You can find out how many cores are being used via the [`EXPLAIN`] command and look
at the number of partitions in the plan.
@@ -190,7 +192,7 @@ does not apply to user defined data sources and only works when DataFusion has a
If you know your data is small, you can set the `datafusion.execution.target_partitions`
option to a smaller number to reduce the overhead of repartitioning. For very small datasets (e.g. less
-than 1MB), we recommend setting `target_partitions` to 1 to avoid repartitioning altogether.
+than 1MB) or point lookups, we recommend setting `target_partitions` to 1 to avoid repartitioning altogether.
```sql
SET datafusion.execution.target_partitions = '1';
@@ -198,6 +200,41 @@ SET datafusion.execution.target_partitions = '1';
[`ListingTable`]: https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html
+For point lookups, the following techniques also help:
+
+- Sort the data by the lookup key so the data that is fetched is physically nearby (clustered).
+ This makes techniques such as Parquet min/max metadata pruning more effective.
+- Write Parquet files with smaller row groups and data pages than the default
+ so that less data needs to be read and decoded for each lookup.
+- Enable `datafusion.execution.parquet.pushdown_filters` so filters are
+ evaluated while decoding Parquet.
+- Set the Parquet metadata cache size via
+ `datafusion.runtime.metadata_cache_limit` so it is large enough to hold all the metadata for the
+ files you query repeatedly.
+- Reuse physical plans to avoid re-planning each query.
+
+Here is an example of how to write Parquet files with smaller row groups and
+data pages:
+
+```sql
+COPY (SELECT * FROM t) TO 't.parquet'
+OPTIONS (
+ MAX_ROW_GROUP_SIZE 131072,
+ DATA_PAGE_ROW_COUNT_LIMIT 2048
+);
+```
+
+Here is how to enable pushdown filters:
+
+```sql
+SET datafusion.execution.parquet.pushdown_filters = true;
+```
+
+See the [Best Practices for Single Row Lookups (Point Queries)] blog post for
+more details.
+
+[Best Practices for Single Row Lookups (Point Queries)]: https://datafusion.apache.org/blog/2026/10/24/point-queries/
+
## Memory-limited Queries
When executing a memory-consuming query under a tight memory limit, DataFusion
diff --git a/docs/source/user-guide/concepts-readings-events.md b/docs/source/user-guide/concepts-readings-events.md
index 2024dc6474..5d7a066b3e 100644
--- a/docs/source/user-guide/concepts-readings-events.md
+++ b/docs/source/user-guide/concepts-readings-events.md
@@ -37,6 +37,8 @@
This is a list of DataFusion related blog posts, articles, and other resources. Please open a PR to add any new resources you create or find
+- **2026-10-24** [Blog: Best Practices for Single Row Lookups (Point Queries)](https://datafusion.apache.org/blog/2026/10/24/point-queries/)
+
- **2026-10-01** [Blog: How we extended Apache DataFusion to execute one query across many machines](https://www.datadoghq.com/blog/engineering/distributed-datafusion/)
- **2026-07-29** [Blog: Bringing DuckLake to DataFusion](https://ducklake.select/2026/07/29/bringing-ducklake-to-datafusion/)
diff --git a/docs/source/user-guide/configs.md b/docs/source/user-guide/configs.md
index 5029e0ac80..fe163e4add 100644
--- a/docs/source/user-guide/configs.md
+++ b/docs/source/user-guide/configs.md
@@ -259,12 +259,14 @@ The following runtime configuration settings are available:
# Tuning Guide
-## Short Queries
+## Short Queries and Point Lookups
By default DataFusion will attempt to maximize parallelism and use all cores --
For example, if you have 32 cores, each plan will split the data into 32
-partitions. However, if your data is small, the overhead of splitting the data
-to enable parallelization can dominate the actual computation.
+partitions. However, if your data is small, or the query only reads a few rows
+(for example, a point lookup such as `SELECT * FROM t WHERE id = 42`), the
+overhead of planning, opening files, and splitting the data to enable
+parallelization can dominate the actual computation.
You can find out how many cores are being used via the [`EXPLAIN`] command and look
at the number of partitions in the plan.
@@ -277,7 +279,7 @@ does not apply to user defined data sources and only works when DataFusion has a
If you know your data is small, you can set the `datafusion.execution.target_partitions`
option to a smaller number to reduce the overhead of repartitioning. For very small datasets (e.g. less
-than 1MB), we recommend setting `target_partitions` to 1 to avoid repartitioning altogether.
+than 1MB) or point lookups, we recommend setting `target_partitions` to 1 to avoid repartitioning altogether.
```sql
SET datafusion.execution.target_partitions = '1';
@@ -285,6 +287,41 @@ SET datafusion.execution.target_partitions = '1';
[`listingtable`]: https://docs.rs/datafusion/latest/datafusion/datasource/listing/struct.ListingTable.html
+For point lookups, the following techniques also help:
+
+- Sort the data by the lookup key so the data that is fetched is physically nearby (clustered).
+ This makes techniques such as Parquet min/max metadata pruning more effective.
+- Write Parquet files with smaller row groups and data pages than the default
+ so that less data needs to be read and decoded for each lookup.
+- Enable `datafusion.execution.parquet.pushdown_filters` so filters are
+ evaluated while decoding Parquet.
+- Set the Parquet metadata cache size via
+ `datafusion.runtime.metadata_cache_limit` so it is large enough to hold all the metadata for the
+ files you query repeatedly.
+- Reuse physical plans to avoid re-planning each query.
+
+Here is an example of how to write Parquet files with smaller row groups and
+data pages:
+
+```sql
+COPY (SELECT * FROM t) TO 't.parquet'
+OPTIONS (
+ MAX_ROW_GROUP_SIZE 131072,
+ DATA_PAGE_ROW_COUNT_LIMIT 2048
+);
+```
+
+Here is how to enable pushdown filters:
+
+```sql
+SET datafusion.execution.parquet.pushdown_filters = true;
+```
+
+See the [Best Practices for Single Row Lookups (Point Queries)] blog post for
+more details.
+
+[best practices for single row lookups (point queries)]: https://datafusion.apache.org/blog/2026/10/24/point-queries/
+
## Memory-limited Queries
When executing a memory-consuming query under a tight memory limit, DataFusion