Apache DataFusion SQL Query Engine
Update 53 upgrade guide to note release, other changes (#21449)
Andrew Lamb8d91fb0e233b
diff --git a/docs/source/library-user-guide/upgrading/53.0.0.md b/docs/source/library-user-guide/upgrading/53.0.0.mdindex f616220778..f572b2a3f7 100644--- a/docs/source/library-user-guide/upgrading/53.0.0.md+++ b/docs/source/library-user-guide/upgrading/53.0.0.md@@ -21,13 +21,6 @@## DataFusion 53.0.0-**Note:** DataFusion `53.0.0` has not been released yet. The information provided-*in this section pertains to features and changes that have already been merged-*to the main branch and are awaiting release in this version. See [#19692] for-\*more details.--[#19692]: https://github.com/apache/datafusion/issues/19692-### Upgrade arrow/parquet to 58.0.0 and object_store to 0.13.0DataFusion 53.0.0 uses `arrow` and `parquet` 58.0.0, and `object_store` 0.13.0.@@ -39,6 +32,39 @@ See the [Arrow 58.0.0 release notes] and the [object_store 0.13.0 upgrade guide][arrow 58.0.0 release notes]: https://github.com/apache/arrow-rs/releases/tag/58.0.0[object_store 0.13.0 upgrade guide]: https://github.com/apache/arrow-rs-object-store/blob/v0.13.0/CHANGELOG.md+### `ExecutionPlan::statistics` removed++The deprecated `ExecutionPlan::statistics()` method has been removed. If you+implement custom `ExecutionPlan`s, remove that method from your impl and+implement `partition_statistics()` instead.++**Before:**++```rust,ignore+impl ExecutionPlan for MyExec {+ // ...++ fn statistics(&self) -> Result<Statistics> {+ Ok(Statistics::new_unknown(&self.schema()))+ }+}+```++**After:**++```rust,ignore+impl ExecutionPlan for MyExec {+ // ...++ fn partition_statistics(&self, _partition: Option<usize>) -> Result<Statistics> {+ Ok(Statistics::new_unknown(&self.schema()))+ }+}+```++If you do not have partition-specific statistics, return the same value for+`None` and for any partition index.+### `ExecutionPlan::properties` now returns `&Arc<PlanProperties>`Now `ExecutionPlan::properties()` returns `&Arc<PlanProperties>` instead of a@@ -125,6 +151,15 @@ let sub_plan = self.query_to_plan(subquery, planner_context)?;planner_context.pop_outer_query_schema();```+### `HashJoinExec::try_new` adds `null_aware`++`HashJoinExec::try_new` now takes an extra `null_aware: bool` argument. This+flag is used for null-aware anti joins, such as plans generated for `NOT IN`+subqueries.++Most callers should pass `false`, the previous behavior. Pass `true` only for null-aware+`JoinType::LeftAnti` joins.+### `FileSinkConfig` adds `file_output_mode``FileSinkConfig` now includes a `file_output_mode: FileOutputMode` field to control