Skip to content

Apache DataFusion SQL Query Engine

4.1 KB
Raw Blame

Apache DataFusion 54.1.0 Changelog

This release consists of 19 commits from 9 contributors. See credits at the end of this changelog for more information.

See the upgrade guide for information on how to upgrade from previous versions.

Documentation updates:

  • [branch-54] Add datafusion.execution.enable_file_stream_work_stealing config #23296 (andygrove)

Other:

  • [branch-54] fix: preserve null_aware on logical JoinNode proto round-trip (backport #22104) #22785 (mithuncy)
  • [branch-54]: backport #22811 (bugfix: changed return type of spark's width_bucket to i64) #23087 (mbutrovich)
  • [branch-54] backport #22857 (Skip loading Parquet page index when row-group statistics already prove it cannot prune) #23088 (mbutrovich)
  • [branch-54] backport #23192 array_compact handle edge case with NULLs #23196 (comphead)
  • [branch-54] fix: Avoid panicing when stats are not available for a file group split (backport #23277) #23340 (mkleen)
  • [branch-54] fix: approx_distinct over-counts for utf8view (backport #22815, adapted) #23576 (mbutrovich)
  • [branch-54] fix: isolate anonymous file statistics cache (backport #22950, adapted) #23573 (mbutrovich)
  • [branch-54] fix: = ANY (SELECT ...) / <> ALL (SELECT ...) schema error (backport #22915) #23575 (mbutrovich)
  • [branch-54] fix: NestedLoopJoinExec emits spurious unmatched-left rows with multiple probe partitions (backport #22791) #23577 (mbutrovich)
  • [branch-54] fix: regex simplification of anchored patterns produces wrong results (backport #22727) #23578 (mbutrovich)
  • [branch-54] fix: Correctly compute nullability in recursive CTE schemas (backport #22552) #23579 (mbutrovich)
  • [branch-54] fix: handle IS TRUE correctly in EliminateOuterJoin (backport #22444) #23580 (mbutrovich)
  • [branch-54] fix: preserve no-filter SMJ matches across pending outer batches (backport #23049) #23574 (mbutrovich)
  • [branch-54] perf: avoid intermediate slice allocation in Spark slice function (backport #23481) #23582 (mbutrovich)
  • [branch-54] fix: don't duplicate volatile expressions when pushing projection into file scan (backport #23395, adapted) #23585 (fordN)
  • [branch-54] chore: fix cargo audit #23607 (alamb)
  • [branch-54] Resolve lost wakeup in SpillPoolReader with multiple conc… #23654 (pepijnve)
  • [branch-54] Handle nulls in type coercion of higher-order UDFs, map_extract, spark array_repeat (backport #23071) #23629 (gstvg)

Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

    11	Matt Butrovich
     1	Andrew Lamb
     1	Andy Grove
     1	Ford
     1	Michael Kleen
     1	Mithun Chicklore Yogendra
     1	Oleks V
     1	Pepijn Van Eeckhoudt
     1	gstvg

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.