Apache DataFusion SQL Query Engine
docs: move misplaced upgrade notes to DataFusion 55 (#25331)
Huaijin16ecdc9bdff1
diff --git a/docs/source/library-user-guide/upgrading/54.0.0.md b/docs/source/library-user-guide/upgrading/54.0.0.mdindex f8e7ac93c0..ce4d7c22a2 100644--- a/docs/source/library-user-guide/upgrading/54.0.0.md+++ b/docs/source/library-user-guide/upgrading/54.0.0.md@@ -335,44 +335,6 @@ This produces two user-visible changes:`ScalarSubqueryExpr` expression. Code that walks or transforms `LogicalPlan` /`ExecutionPlan` trees, as well as `EXPLAIN` output, may need updating.-### Filter predicate evaluation order may differ from query text--The logical optimizer now reorders filters so that cheap predicates (most binary-comparisons, `IS NULL`, `Between`, `InList`, etc.) evaluate before expensive-ones (`LIKE`, regex, scalar function calls, subqueries). For example,-`WHERE col LIKE '%foo%' AND col2 = 5` may evaluate `col2 = 5` before-`col LIKE '%foo%'`.--**Evaluation order has never been guaranteed to match the order written in the-query.** The SQL standard explicitly allows implementations to evaluate operands-in any order; major engines (PostgreSQL, SQL Server, Oracle, MySQL) document the-same. Queries should not rely on left-to-right evaluation or short-circuit-semantics for `AND` or `OR`. Previous versions of DataFusion already reordered-predicates (e.g., as part of expression simplification or predicate pushdown);-the new reordering pass just increases the scenarios where the optimizer will-change predicate evaluation order.--**Fallible-predicate patterns are particularly affected.** For example:--```sql-WHERE s ~ '^[0-9]+$' AND CAST(s AS INT) > 0-```--The intent is likely to filter non-numeric strings before the `CAST` runs,-but this depends on evaluation-order behavior the SQL standard does not-guaranteed. The new reorder makes this kind of pattern more likely to fail-at runtime if the optimizer moves the `CAST` ahead of the regex. To force-conditional evaluation, rewrite using `CASE`, which has standardized-short-circuit semantics:--```sql-WHERE CASE WHEN s ~ '^[0-9]+$' THEN CAST(s AS INT) > 0 ELSE false END-```--Volatile expressions (`random()`, `now()`, etc.) are exempt — their position-in the conjunct list is preserved so the number of times they evaluate per-query does not change.-### `datafusion-proto`: expression deserialization now takes a `TaskContext``Serializeable::from_bytes_with_registry` is renamed to `from_bytes_with_ctx`@@ -928,8 +890,3 @@ match register_function {RegisterFunction::Table(name, table) => {},}```--### New `Dialect::Spark` variant--The `Dialect` enum in `datafusion_common::config` now includes a `Spark` variant.-If you match exhaustively on `Dialect`, add a `Dialect::Spark` arm.diff --git a/docs/source/library-user-guide/upgrading/55.0.0.md b/docs/source/library-user-guide/upgrading/55.0.0.mdindex f8b66d5f2c..abd4d0020e 100644--- a/docs/source/library-user-guide/upgrading/55.0.0.md+++ b/docs/source/library-user-guide/upgrading/55.0.0.md@@ -21,6 +21,44 @@## DataFusion 55.0.0+### Filter predicate evaluation order may differ from query text++The logical optimizer now reorders filters so that cheap predicates (most binary+comparisons, `IS NULL`, `Between`, `InList`, etc.) evaluate before expensive+ones (`LIKE`, regex, scalar function calls, subqueries). For example,+`WHERE col LIKE '%foo%' AND col2 = 5` may evaluate `col2 = 5` before+`col LIKE '%foo%'`.++**Evaluation order has never been guaranteed to match the order written in the+query.** The SQL standard explicitly allows implementations to evaluate operands+in any order; major engines (PostgreSQL, SQL Server, Oracle, MySQL) document the+same. Queries should not rely on left-to-right evaluation or short-circuit+semantics for `AND` or `OR`. Previous versions of DataFusion already reordered+predicates (e.g., as part of expression simplification or predicate pushdown);+the new reordering pass just increases the scenarios where the optimizer will+change predicate evaluation order.++**Fallible-predicate patterns are particularly affected.** For example:++```sql+WHERE s ~ '^[0-9]+$' AND CAST(s AS INT) > 0+```++The intent is likely to filter non-numeric strings before the `CAST` runs,+but this depends on evaluation-order behavior the SQL standard does not+guaranteed. The new reorder makes this kind of pattern more likely to fail+at runtime if the optimizer moves the `CAST` ahead of the regex. To force+conditional evaluation, rewrite using `CASE`, which has standardized+short-circuit semantics:++```sql+WHERE CASE WHEN s ~ '^[0-9]+$' THEN CAST(s AS INT) > 0 ELSE false END+```++Volatile expressions (`random()`, `now()`, etc.) are exempt — their position+in the conjunct list is preserved so the number of times they evaluate per+query does not change.+### Map casts with nested Structs use schema-evolution rulesCasts of a `Map` whose key or value recursively contains a `Struct` now adapt@@ -211,6 +249,11 @@ Use `temp_file.size()` instead of [`RefCountedTempFile::current_disk_usage`].[`refcountedtempfile::current_disk_usage`]: https://docs.rs/datafusion-execution/latest/datafusion_execution/disk_manager/struct.RefCountedTempFile.html#method.current_disk_usage[`refcountedtempfile::update_disk_usage`]: https://docs.rs/datafusion-execution/latest/datafusion_execution/disk_manager/struct.RefCountedTempFile.html#method.update_disk_usage+### New `Dialect::Spark` variant++The `Dialect` enum in `datafusion_common::config` now includes a `Spark` variant.+If you match exhaustively on `Dialect`, add a `Dialect::Spark` arm.+### `Dialect::AVAILABLE` replaced by `Dialect::available()``datafusion_common::config::Dialect::AVAILABLE` has been removed. Use