Skip to content

Apache DataFusion SQL Query Engine

feat: rewrite inner joins against a single constant row into a filter (#26404)

xudong.w2c2ecae08bd4
diff --git a/datafusion/core/src/optimizer_rule_reference.md b/datafusion/core/src/optimizer_rule_reference.md
index 298a01edd6..13e7f62e72 100644
--- a/datafusion/core/src/optimizer_rule_reference.md
+++ b/datafusion/core/src/optimizer_rule_reference.md
@@ -73,22 +73,23 @@ in multiple phases.
| ----- | ------------------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------------ |
| 1 | `OutputRequirements` | add phase | Adds helper nodes so output requirements survive later physical rewrites. |
| 2 | `aggregate_statistics` | - | Uses exact source statistics to answer some aggregates without scanning data. |
-| 3 | `join_selection` | - | Chooses join implementation, build side, and partition mode from statistics and stream properties. |
-| 4 | `LimitedDistinctAggregation` | - | Pushes limit hints into grouped distinct-style aggregations when only a small result is needed. |
-| 5 | `FilterPushdown` | pre-optimization phase | Pushes supported physical filters down toward data sources before distribution and sorting are enforced. |
-| 6 | `WindowTopN` | - | Replaces eligible row-number window and filter patterns with per-partition TopK execution. |
-| 7 | `EnsureRequirements` | - | Enforces both distribution and sorting requirements in a single idempotent rule. |
-| 8 | `CombinePartialFinalAggregate` | - | Collapses adjacent partial and final aggregates when the distributed shape makes them redundant. |
-| 9 | `OptimizeAggregateOrder` | - | Updates aggregate expressions to use the best ordering once sort requirements are known. |
-| 10 | `ProjectionPushdown` | early pass | Pushes projections toward inputs before later physical rewrites add more limit and TopK structure. |
-| 11 | `OutputRequirements` | remove phase | Removes the temporary output-requirement helper nodes after requirement-sensitive planning is done. |
-| 12 | `LimitAggregation` | - | Passes a limit hint into eligible aggregations so they can keep fewer accumulator buckets. |
-| 13 | `LimitPushPastWindows` | - | Pushes fetch limits through bounded window operators when doing so keeps the result correct. |
-| 14 | `HashJoinBuffering` | - | Adds buffering on the probe side of hash joins so probing can start before build completion. |
-| 15 | `LimitPushdown` | - | Moves physical limits into child operators or fetch-enabled variants to cut data early. |
-| 16 | `TopKRepartition` | - | Pushes TopK below hash repartition when the partition key is a prefix of the sort key. |
-| 17 | `ProjectionPushdown` | late pass | Runs projection pushdown again after limit and TopK rewrites expose new pruning opportunities. |
-| 18 | `PushdownSort` | - | Pushes sort requirements into data sources that can already return sorted output. |
-| 19 | `EnsureCooperative` | - | Wraps non-cooperative plan parts so long-running tasks yield fairly. |
-| 20 | `FilterPushdown(Post)` | post-optimization phase | Pushes dynamic filters at the end of optimization, after plan references stop moving. |
-| 21 | `SanityCheckPlan` | - | Validates that the final physical plan meets ordering, distribution, and infinite-input safety requirements. |
+| 3 | `EliminateConstantJoin` | - | Replaces inner joins against a single row of literals with a filter that keeps the other input's order. |
+| 4 | `join_selection` | - | Chooses join implementation, build side, and partition mode from statistics and stream properties. |
+| 5 | `LimitedDistinctAggregation` | - | Pushes limit hints into grouped distinct-style aggregations when only a small result is needed. |
+| 6 | `FilterPushdown` | pre-optimization phase | Pushes supported physical filters down toward data sources before distribution and sorting are enforced. |
+| 7 | `WindowTopN` | - | Replaces eligible row-number window and filter patterns with per-partition TopK execution. |
+| 8 | `EnsureRequirements` | - | Enforces both distribution and sorting requirements in a single idempotent rule. |
+| 9 | `CombinePartialFinalAggregate` | - | Collapses adjacent partial and final aggregates when the distributed shape makes them redundant. |
+| 10 | `OptimizeAggregateOrder` | - | Updates aggregate expressions to use the best ordering once sort requirements are known. |
+| 11 | `ProjectionPushdown` | early pass | Pushes projections toward inputs before later physical rewrites add more limit and TopK structure. |
+| 12 | `OutputRequirements` | remove phase | Removes the temporary output-requirement helper nodes after requirement-sensitive planning is done. |
+| 13 | `LimitAggregation` | - | Passes a limit hint into eligible aggregations so they can keep fewer accumulator buckets. |
+| 14 | `LimitPushPastWindows` | - | Pushes fetch limits through bounded window operators when doing so keeps the result correct. |
+| 15 | `HashJoinBuffering` | - | Adds buffering on the probe side of hash joins so probing can start before build completion. |
+| 16 | `LimitPushdown` | - | Moves physical limits into child operators or fetch-enabled variants to cut data early. |
+| 17 | `TopKRepartition` | - | Pushes TopK below hash repartition when the partition key is a prefix of the sort key. |
+| 18 | `ProjectionPushdown` | late pass | Runs projection pushdown again after limit and TopK rewrites expose new pruning opportunities. |
+| 19 | `PushdownSort` | - | Pushes sort requirements into data sources that can already return sorted output. |
+| 20 | `EnsureCooperative` | - | Wraps non-cooperative plan parts so long-running tasks yield fairly. |
+| 21 | `FilterPushdown(Post)` | post-optimization phase | Pushes dynamic filters at the end of optimization, after plan references stop moving. |
+| 22 | `SanityCheckPlan` | - | Validates that the final physical plan meets ordering, distribution, and infinite-input safety requirements. |
diff --git a/datafusion/core/tests/physical_optimizer/eliminate_constant_join.rs b/datafusion/core/tests/physical_optimizer/eliminate_constant_join.rs
new file mode 100644
index 0000000000..04b66ce7e4
--- /dev/null
+++ b/datafusion/core/tests/physical_optimizer/eliminate_constant_join.rs
@@ -0,0 +1,201 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Integration tests for the [`EliminateConstantJoin`] physical optimizer rule.
+//! Cases that SQL can express are in `eliminate_constant_join.slt`.
+
+use std::collections::HashMap;
+use std::sync::Arc;
+
+use arrow::array::{Int64Array, RecordBatch};
+use arrow::compute::concat_batches;
+use arrow::datatypes::{DataType, Field, Schema};
+use arrow::util::pretty::pretty_format_batches;
+use datafusion::datasource::MemTable;
+use datafusion::datasource::memory::MemorySourceConfig;
+use datafusion::execution::session_state::SessionStateBuilder;
+use datafusion::prelude::{SessionConfig, SessionContext};
+use datafusion_common::config::ConfigOptions;
+use datafusion_common::{JoinSide, JoinType, Result};
+use datafusion_expr::Operator;
+use datafusion_physical_expr::expressions::{BinaryExpr, Column, lit};
+use datafusion_physical_optimizer::PhysicalOptimizerRule;
+use datafusion_physical_optimizer::eliminate_constant_join::EliminateConstantJoin;
+use datafusion_physical_plan::joins::NestedLoopJoinExec;
+use datafusion_physical_plan::joins::utils::{ColumnIndex, JoinFilter};
+use datafusion_physical_plan::placeholder_row::PlaceholderRowExec;
+use datafusion_physical_plan::projection::ProjectionExec;
+use datafusion_physical_plan::{ExecutionPlan, collect};
+
+/// A placeholder emits one row per partition, so a two-partition constant input
+/// matches every row twice.
+#[tokio::test]
+async fn keeps_join_with_multi_row_constant() -> Result<()> {
+ let schema = Arc::new(Schema::new(vec![Field::new("t", DataType::Int64, true)]));
+ let batch = RecordBatch::try_new(
+ Arc::clone(&schema),
+ vec![Arc::new(Int64Array::from(vec![10, 30]))],
+ )?;
+ let events = MemorySourceConfig::try_new_exec(&[vec![batch]], schema, None)?;
+ let constant = Arc::new(ProjectionExec::try_new(
+ [(lit(25i64), "cutoff".to_string())],
+ Arc::new(PlaceholderRowExec::new(Arc::new(Schema::empty())).with_partitions(2)),
+ )?);
+ let filter = JoinFilter::new(
+ Arc::new(BinaryExpr::new(
+ Arc::new(Column::new("t", 0)),
+ Operator::Lt,
+ Arc::new(Column::new("cutoff", 1)),
+ )),
+ vec![
+ ColumnIndex {
+ index: 0,
+ side: JoinSide::Left,
+ },
+ ColumnIndex {
+ index: 0,
+ side: JoinSide::Right,
+ },
+ ],
+ Arc::new(Schema::new(vec![
+ Field::new("t", DataType::Int64, true),
+ Field::new("cutoff", DataType::Int64, false),
+ ])),
+ );
+ let join: Arc<dyn ExecutionPlan> = Arc::new(NestedLoopJoinExec::try_new(
+ events,
+ constant,
+ Some(filter),
+ &JoinType::Inner,
+ None,
+ )?);
+
+ let optimized = EliminateConstantJoin::new()
+ .optimize(Arc::clone(&join), &ConfigOptions::new())?;
+ assert_eq!(rows(optimized).await?, rows(join).await?);
+ Ok(())
+}
+
+/// Lambda variables are numbered after the join filter's own columns, so a filter
+/// over the wider input would read the wrong columns.
+#[tokio::test]
+async fn keeps_join_with_lambda_filter() -> Result<()> {
+ // Plan without optimizer rules: projection pushdown cannot yet plan lambdas in
+ // nested-loop join filters.
+ let config =
+ SessionConfig::new().set_str("datafusion.sql_parser.dialect", "databricks");
+ let ctx = SessionContext::new_with_state(
+ SessionStateBuilder::new()
+ .with_config(config)
+ .with_default_features()
+ .with_physical_optimizer_rules(vec![])
+ .build(),
+ );
+ ctx.sql(
+ "CREATE TABLE lists AS \
+ SELECT * FROM (VALUES ([1, 30], 'a', 7), ([40], 'b', 8)) AS v(l, k, n)",
+ )
+ .await?
+ .collect()
+ .await?;
+ let join = ctx
+ .sql(
+ "SELECT s.k, s.n FROM (SELECT 25 AS cutoff) c \
+ JOIN lists s ON array_any_match(s.l, x -> x < c.cutoff)",
+ )
+ .await?
+ .create_physical_plan()
+ .await?;
+
+ let optimized = EliminateConstantJoin::new()
+ .optimize(Arc::clone(&join), &ConfigOptions::new())?;
+ assert_eq!(rows(optimized).await?, rows(join).await?);
+ Ok(())
+}
+
+/// The rewritten plan keeps the schema metadata that came from the constant input;
+/// otherwise the optimizer's schema check fails the query.
+#[tokio::test]
+async fn keeps_metadata_of_constant_input() -> Result<()> {
+ // With `*`, no projection above the join keeps the metadata.
+ let sql = "SELECT * FROM (SELECT count(*) AS n FROM tagged) c \
+ JOIN events e ON e.t >= c.n";
+ let expected = rows(
+ metadata_session(false)?
+ .sql(sql)
+ .await?
+ .create_physical_plan()
+ .await?,
+ )
+ .await?;
+ let ctx = metadata_session(true)?;
+ let plan = ctx.sql(sql).await?.create_physical_plan().await?;
+ assert_eq!(rows(plan).await?, expected);
+ Ok(())
+}
+
+/// Registers `events` and `tagged`, a table whose schema carries metadata, with the
+/// default rules, without [`EliminateConstantJoin`] unless `eliminate` is set.
+fn metadata_session(eliminate: bool) -> Result<SessionContext> {
+ let mut state = SessionStateBuilder::new().with_default_features().build();
+ if !eliminate {
+ let rules = state
+ .physical_optimizers()
+ .iter()
+ .filter(|rule| rule.name() != EliminateConstantJoin::new().name())
+ .cloned()
+ .collect();
+ state = SessionStateBuilder::new_from_existing(state)
+ .with_physical_optimizer_rules(rules)
+ .build();
+ }
+ let ctx = SessionContext::new_with_state(state);
+ for (name, metadata) in [
+ ("events", HashMap::new()),
+ (
+ "tagged",
+ HashMap::from([("source".to_string(), "tagged".to_string())]),
+ ),
+ ] {
+ let schema = Arc::new(Schema::new_with_metadata(
+ vec![Field::new("t", DataType::Int64, true)],
+ metadata,
+ ));
+ let batch = RecordBatch::try_new(
+ Arc::clone(&schema),
+ vec![Arc::new(Int64Array::from(vec![1, 2, 3]))],
+ )?;
+ ctx.register_table(
+ name,
+ Arc::new(MemTable::try_new(schema, vec![vec![batch]])?),
+ )?;
+ }
+ Ok(ctx)
+}
+
+/// Runs `plan` and formats its rows in sorted order.
+async fn rows(plan: Arc<dyn ExecutionPlan>) -> Result<Vec<String>> {
+ let schema = plan.schema();
+ let batches = collect(plan, Arc::new(Default::default())).await?;
+ let mut rows = pretty_format_batches(&[concat_batches(&schema, &batches)?])?
+ .to_string()
+ .lines()
+ .map(str::to_string)
+ .collect::<Vec<_>>();
+ rows.sort_unstable();
+ Ok(rows)
+}
diff --git a/datafusion/core/tests/physical_optimizer/mod.rs b/datafusion/core/tests/physical_optimizer/mod.rs
index f89d86074b..43b5771802 100644
--- a/datafusion/core/tests/physical_optimizer/mod.rs
+++ b/datafusion/core/tests/physical_optimizer/mod.rs
@@ -20,6 +20,7 @@
#[expect(clippy::needless_pass_by_value)]
mod aggregate_statistics;
mod combine_partial_final_agg;
+mod eliminate_constant_join;
#[expect(clippy::needless_pass_by_value)]
mod enforce_distribution;
mod enforce_sorting;
diff --git a/datafusion/physical-optimizer/src/eliminate_constant_join.rs b/datafusion/physical-optimizer/src/eliminate_constant_join.rs
new file mode 100644
index 0000000000..d445cb9395
--- /dev/null
+++ b/datafusion/physical-optimizer/src/eliminate_constant_join.rs
@@ -0,0 +1,243 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+//! Replace inner joins against a single row of literals with a filter.
+
+use std::sync::Arc;
+
+use arrow::datatypes::SchemaRef;
+use datafusion_common::Result;
+use datafusion_common::config::ConfigOptions;
+use datafusion_common::tree_node::{Transformed, TransformedResult, TreeNode};
+use datafusion_common::{JoinSide, JoinType, internal_datafusion_err};
+use datafusion_physical_expr::expressions::{Column, LambdaExpr, Literal};
+use datafusion_physical_expr::{PhysicalExpr, PhysicalExprSimplifier};
+use datafusion_physical_plan::filter::FilterExec;
+use datafusion_physical_plan::joins::utils::{ColumnIndex, JoinFilter};
+use datafusion_physical_plan::joins::{CrossJoinExec, NestedLoopJoinExec};
+use datafusion_physical_plan::placeholder_row::PlaceholderRowExec;
+use datafusion_physical_plan::projection::{ProjectionExec, ProjectionExpr};
+use datafusion_physical_plan::{ExecutionPlan, ExecutionPlanProperties};
+
+use crate::PhysicalOptimizerRule;
+
+/// Rewrites an inner [`NestedLoopJoinExec`] or a [`CrossJoinExec`] whose input
+/// provably produces one row of literals into a [`FilterExec`] over the other
+/// input, with the literals substituted into the join filter, under a
+/// [`ProjectionExec`] that restores the join's output.
+///
+/// For example, [`AggregateStatistics`] folds the scalar subquery in
+///
+/// ```sql
+/// SELECT e.* FROM events e CROSS JOIN (SELECT min(ts) AS first_ts FROM live) l
+/// WHERE e.ts < l.first_ts
+/// ```
+///
+/// into literals over a [`PlaceholderRowExec`], leaving a join that only filters
+/// `events`. A nested-loop join preserves neither input's order, while the filter
+/// and projection preserve the other input's ordering and partitioning. This lets
+/// later operators, such as an aggregation grouped by a sort key of `events`,
+/// use that ordering.
+///
+/// [`AggregateStatistics`]: crate::aggregate_statistics::AggregateStatistics
+#[derive(Debug, Default)]
+pub struct EliminateConstantJoin {}
+
+impl EliminateConstantJoin {
+ #[expect(missing_docs)]
+ pub fn new() -> Self {
+ Self {}
+ }
+}
+
+impl PhysicalOptimizerRule for EliminateConstantJoin {
+ fn optimize(
+ &self,
+ plan: Arc<dyn ExecutionPlan>,
+ _config: &ConfigOptions,
+ ) -> Result<Arc<dyn ExecutionPlan>> {
+ plan.transform_up(|plan| {
+ let rewritten = if let Some(join) = plan.downcast_ref::<NestedLoopJoinExec>()
+ {
+ eliminate_nested_loop_join(join)?
+ } else if let Some(join) = plan.downcast_ref::<CrossJoinExec>() {
+ eliminate(join.left(), join.right(), None, None, &join.schema())?
+ } else {
+ None
+ };
+ Ok(match rewritten {
+ Some(rewritten) => Transformed::yes(rewritten),
+ None => Transformed::no(plan),
+ })
+ })
+ .data()
+ }
+
+ fn name(&self) -> &str {
+ "EliminateConstantJoin"
+ }
+
+ fn schema_check(&self) -> bool {
+ true
+ }
+}
+
+/// Rewrites `join` if it is an inner join against a constant row and its filter
+/// has no lambdas.
+fn eliminate_nested_loop_join(
+ join: &NestedLoopJoinExec,
+) -> Result<Option<Arc<dyn ExecutionPlan>>> {
+ if join.join_type() != &JoinType::Inner {
+ return Ok(None);
+ }
+ if let Some(filter) = join.filter()
+ // Lambda variables are numbered after the filter's own columns, not the
+ // other input's.
+ && filter
+ .expression()
+ .exists(|expr| Ok(expr.is::<LambdaExpr>()))?
+ {
+ return Ok(None);
+ }
+ eliminate(
+ join.left(),
+ join.right(),
+ join.filter(),
+ join.projection().as_deref(),
+ &join.schema(),
+ )
+}
+
+/// Rewrites an inner join of `left_input` and `right_input` over its non-constant
+/// input, if the other input is a constant row.
+fn eliminate(
+ left_input: &Arc<dyn ExecutionPlan>,
+ right_input: &Arc<dyn ExecutionPlan>,
+ filter: Option<&JoinFilter>,
+ projection: Option<&[usize]>,
+ schema: &SchemaRef,
+) -> Result<Option<Arc<dyn ExecutionPlan>>> {
+ // Each input's columns as expressions over the non-constant input.
+ let (left, right, input) = if let Some(row) = constant_row(left_input) {
+ (row, columns(right_input), right_input)
+ } else if let Some(row) = constant_row(right_input) {
+ (columns(left_input), row, left_input)
+ } else {
+ return Ok(None);
+ };
+
+ let mut input = Arc::clone(input);
+ if let Some(filter) = filter {
+ let predicate = Arc::clone(filter.expression())
+ .transform_up(|expr| {
+ let Some(column) = expr.downcast_ref::<Column>() else {
+ return Ok(Transformed::no(expr));
+ };
+ // Filter columns index the filter's own schema, which
+ // `column_indices` maps back to the join inputs.
+ let ColumnIndex { index, side } =
+ filter.column_indices().get(column.index()).ok_or_else(|| {
+ internal_datafusion_err!("Unmapped join filter column")
+ })?;
+ let value = match side {
+ JoinSide::Left => left.get(*index),
+ JoinSide::Right => right.get(*index),
+ JoinSide::None => None,
+ }
+ .ok_or_else(|| {
+ internal_datafusion_err!("Join filter column out of range")
+ })?;
+ Ok(Transformed::yes(Arc::clone(value)))
+ })
+ .data()?;
+ input = Arc::new(FilterExec::try_new(predicate, input)?);
+ }
+
+ let mut outputs = left.into_iter().chain(right).collect::<Vec<_>>();
+ if let Some(projection) = projection {
+ outputs = projection
+ .iter()
+ .map(|&index| outputs.get(index).cloned())
+ .collect::<Option<_>>()
+ .ok_or_else(|| internal_datafusion_err!("Join projection out of range"))?;
+ }
+ let exprs = outputs
+ .into_iter()
+ .zip(schema.fields())
+ .map(|(expr, field)| ProjectionExpr {
+ expr,
+ alias: field.name().clone(),
+ });
+ // Keep the join's metadata, which may come from the constant input.
+ Ok(Some(Arc::new(
+ ProjectionExec::try_new_with_schema_metadata(exprs, input, schema)?,
+ )))
+}
+
+/// Returns the literal values of the one row `plan` produces, if it is a stack of
+/// projections over a single-partition [`PlaceholderRowExec`] whose expressions
+/// evaluate to literals at planning time.
+fn constant_row(plan: &Arc<dyn ExecutionPlan>) -> Option<Vec<Arc<dyn PhysicalExpr>>> {
+ let projection = plan.downcast_ref::<ProjectionExec>()?;
+ let input = projection.input();
+ let input_row = if input.is::<PlaceholderRowExec>() {
+ // A placeholder emits one row per partition, and its columns are not
+ // literals.
+ if input.output_partitioning().partition_count() != 1 {
+ return None;
+ }
+ Vec::new()
+ } else {
+ constant_row(input)?
+ };
+ // The logical simplifier already folds constant expressions, but those over a
+ // value `AggregateStatistics` folded, such as `min(ts) + 1`, are only constant
+ // now. Volatile expressions and ones that fail to evaluate are not folded, and
+ // are rejected: the filter would evaluate them for every row instead of once.
+ let schema = input.schema();
+ let simplifier = PhysicalExprSimplifier::new(&schema);
+ projection
+ .expr()
+ .iter()
+ .map(|ProjectionExpr { expr, .. }| {
+ let expr = Arc::clone(expr)
+ .transform_up(|expr| {
+ let value = expr
+ .downcast_ref::<Column>()
+ .and_then(|column| input_row.get(column.index()));
+ Ok(match value {
+ Some(value) => Transformed::yes(Arc::clone(value)),
+ None => Transformed::no(expr),
+ })
+ })
+ .data()
+ .and_then(|expr| simplifier.simplify(expr))
+ .ok()?;
+ expr.is::<Literal>().then_some(expr)
+ })
+ .collect()
+}
+
+/// Returns a column reference for each field of `plan`.
+fn columns(plan: &Arc<dyn ExecutionPlan>) -> Vec<Arc<dyn PhysicalExpr>> {
+ plan.schema()
+ .fields()
+ .iter()
+ .enumerate()
+ .map(|(index, field)| Arc::new(Column::new(field.name(), index)) as _)
+ .collect()
+}
diff --git a/datafusion/physical-optimizer/src/lib.rs b/datafusion/physical-optimizer/src/lib.rs
index b9eb248f6e..25c9452915 100644
--- a/datafusion/physical-optimizer/src/lib.rs
+++ b/datafusion/physical-optimizer/src/lib.rs
@@ -27,6 +27,7 @@
pub mod aggregate_statistics;
pub mod combine_partial_final_agg;
+pub mod eliminate_constant_join;
pub mod ensure_coop;
pub mod ensure_requirements;
// `enforce_distribution` and `enforce_sorting` are now internal implementation
diff --git a/datafusion/physical-optimizer/src/optimizer.rs b/datafusion/physical-optimizer/src/optimizer.rs
index f81555fcc2..eb4b707a74 100644
--- a/datafusion/physical-optimizer/src/optimizer.rs
+++ b/datafusion/physical-optimizer/src/optimizer.rs
@@ -24,6 +24,7 @@ use std::sync::Arc;
use crate::aggregate_statistics::AggregateStatistics;
use crate::combine_partial_final_agg::CombinePartialFinalAggregate;
+use crate::eliminate_constant_join::EliminateConstantJoin;
use crate::ensure_coop::EnsureCooperative;
use crate::ensure_requirements::EnsureRequirements;
use crate::filter_pushdown::FilterPushdown;
@@ -200,6 +201,10 @@ impl PhysicalOptimizer {
// this information is not lost across different rules during optimization.
Arc::new(OutputRequirements::new_add_mode()),
Arc::new(AggregateStatistics::new()),
+ // Runs after AggregateStatistics, which can fold a scalar subquery into a
+ // row of literals, and before EnsureRequirements, so that the filter
+ // replacing a join against such a row keeps the other input's ordering.
+ Arc::new(EliminateConstantJoin::new()),
// Statistics-based join selection will change the Auto mode to a real join implementation,
// like collect left, or hash join, or future sort merge join, which will influence the
// EnsureRequirements rule as it decides whether to add additional repartitioning and
diff --git a/datafusion/sqllogictest/test_files/cte.slt b/datafusion/sqllogictest/test_files/cte.slt
index fe02e39e3b..e0f8a4d4da 100644
--- a/datafusion/sqllogictest/test_files/cte.slt
+++ b/datafusion/sqllogictest/test_files/cte.slt
@@ -1155,10 +1155,8 @@ logical_plan
04)----Projection: Int64(1)
05)------EmptyRelation: rows=1
physical_plan
-01)CrossJoinExec
+01)ProjectionExec: expr=[a@0 as a, 1 as Int64(1)]
02)--DataSourceExec: partitions=1, partition_sizes=[0]
-03)--ProjectionExec: expr=[1 as Int64(1)]
-04)----PlaceholderRowExec
statement count 0
drop table j1;
diff --git a/datafusion/sqllogictest/test_files/eliminate_constant_join.slt b/datafusion/sqllogictest/test_files/eliminate_constant_join.slt
new file mode 100644
index 0000000000..ac441794b6
--- /dev/null
+++ b/datafusion/sqllogictest/test_files/eliminate_constant_join.slt
@@ -0,0 +1,178 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Inner joins against a single row of literals become a filter over the other
+# input, which keeps that input's ordering (EliminateConstantJoin).
+
+statement ok
+set datafusion.execution.target_partitions = 1;
+
+statement ok
+set datafusion.explain.physical_plan_only = true;
+
+# Events sorted by account
+query I
+COPY (SELECT * FROM (VALUES ('a', 10), ('a', 30), ('b', 20), ('b', NULL), ('c', 5)) AS v(acct, t))
+TO 'test_files/scratch/eliminate_constant_join/events.parquet'
+STORED AS PARQUET;
+----
+5
+
+statement ok
+CREATE EXTERNAL TABLE events
+STORED AS PARQUET
+WITH ORDER (acct ASC NULLS LAST)
+LOCATION 'test_files/scratch/eliminate_constant_join/events.parquet';
+
+query I
+COPY (SELECT * FROM (VALUES (25), (40)) AS v(ts))
+TO 'test_files/scratch/eliminate_constant_join/live.parquet'
+STORED AS PARQUET;
+----
+2
+
+statement ok
+CREATE EXTERNAL TABLE live
+STORED AS PARQUET
+LOCATION 'test_files/scratch/eliminate_constant_join/live.parquet';
+
+# min(ts) is answered from Parquet statistics, so the join only filters the
+# events, and the aggregation keeps their account order
+query TT
+EXPLAIN SELECT e.acct, count(*) AS n
+FROM events e CROSS JOIN (SELECT min(ts) AS first_ts FROM live) l
+WHERE e.t < l.first_ts
+GROUP BY e.acct;
+----
+physical_plan
+01)ProjectionExec: expr=[acct@0 as acct, count(Int64(1))@1 as n]
+02)--AggregateExec: mode=Single, gby=[acct@0 as acct], aggr=[count(Int64(1))], group_clustering_mode=Full
+03)----FilterExec: 25 > t@1, projection=[acct@0]
+04)------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, t], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet, predicate=25 > t@1, pruning_predicate=t_null_count@1 != row_count@2 AND t_min@0 < 25, required_guarantees=[]
+
+query TI
+SELECT e.acct, count(*) AS n
+FROM events e CROSS JOIN (SELECT min(ts) AS first_ts FROM live) l
+WHERE e.t < l.first_ts
+GROUP BY e.acct
+ORDER BY e.acct;
+----
+a 1
+b 1
+c 1
+
+# Expressions over a value folded from statistics are evaluated at planning time
+query TT
+EXPLAIN SELECT e.acct
+FROM events e CROSS JOIN (SELECT min(ts) + 1 AS cutoff FROM live) l
+WHERE e.t < l.cutoff;
+----
+physical_plan
+01)FilterExec: 26 > t@1, projection=[acct@0]
+02)--DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, t], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet, predicate=26 > t@1, pruning_predicate=t_null_count@1 != row_count@2 AND t_min@0 < 26, required_guarantees=[]
+
+query T rowsort
+SELECT e.acct
+FROM events e CROSS JOIN (SELECT min(ts) + 1 AS cutoff FROM live) l
+WHERE e.t < l.cutoff;
+----
+a
+b
+c
+
+# A cross join with a constant row keeps the other input's ordering
+query TT
+EXPLAIN SELECT e.acct, l.first_ts, count(*) AS n
+FROM events e CROSS JOIN (SELECT min(ts) AS first_ts FROM live) l
+GROUP BY e.acct, l.first_ts;
+----
+physical_plan
+01)ProjectionExec: expr=[acct@0 as acct, first_ts@1 as first_ts, count(Int64(1))@2 as n]
+02)--AggregateExec: mode=Single, gby=[acct@0 as acct, first_ts@1 as first_ts], aggr=[count(Int64(1))], group_clustering_mode=Full
+03)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, 25 as first_ts], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet
+
+query TII
+SELECT e.acct, l.first_ts, count(*) AS n
+FROM events e CROSS JOIN (SELECT min(ts) AS first_ts FROM live) l
+GROUP BY e.acct, l.first_ts
+ORDER BY e.acct;
+----
+a 25 2
+b 25 2
+c 25 1
+
+# The constant row's columns stay in the output
+query TT
+EXPLAIN SELECT e.acct, e.t, c.cutoff
+FROM (SELECT 25 AS cutoff) c JOIN events e ON e.t < c.cutoff;
+----
+physical_plan
+01)ProjectionExec: expr=[acct@0 as acct, t@1 as t, 25 as cutoff]
+02)--FilterExec: t@1 < 25
+03)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, t], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet, predicate=t@1 < 25, pruning_predicate=t_null_count@1 != row_count@2 AND t_min@0 < 25, required_guarantees=[]
+
+query TII rowsort
+SELECT e.acct, e.t, c.cutoff
+FROM (SELECT 25 AS cutoff) c JOIN events e ON e.t < c.cutoff;
+----
+a 10 25
+b 20 25
+c 5 25
+
+# A left join keeps the constant row when nothing matches, so it stays a join
+query TT
+EXPLAIN SELECT c.cutoff, e.acct
+FROM (SELECT 25 AS cutoff) c LEFT JOIN events e ON e.t > c.cutoff + 100;
+----
+physical_plan
+01)ProjectionExec: expr=[cutoff@0 as cutoff, acct@1 as acct]
+02)--NestedLoopJoinExec: join_type=Left, filter=t@0 > join_proj_push_down_1@1, projection=[cutoff@0, acct@2, t@3]
+03)----ProjectionExec: expr=[cutoff@0 as cutoff, cutoff@0 + 100 as join_proj_push_down_1]
+04)------ProjectionExec: expr=[25 as cutoff]
+05)--------PlaceholderRowExec
+06)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, t], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet
+
+query IT
+SELECT c.cutoff, e.acct
+FROM (SELECT 25 AS cutoff) c LEFT JOIN events e ON e.t > c.cutoff + 100;
+----
+25 NULL
+
+# The join evaluates random() once; a filter would evaluate it for every row
+query TT
+EXPLAIN SELECT e.acct
+FROM (SELECT random() AS r) c JOIN events e ON e.t < c.r * 100;
+----
+physical_plan
+01)ProjectionExec: expr=[acct@1 as acct]
+02)--NestedLoopJoinExec: join_type=Inner, filter=join_proj_push_down_2@1 < join_proj_push_down_1@0, projection=[r@0, acct@2, t@3]
+03)----ProjectionExec: expr=[r@0 as r, r@0 * 100 as join_proj_push_down_1]
+04)------ProjectionExec: expr=[random() as r]
+05)--------PlaceholderRowExec
+06)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/eliminate_constant_join/events.parquet]]}, projection=[acct, t, CAST(t@1 AS Float64) as join_proj_push_down_2], output_ordering=[acct@0 ASC NULLS LAST], file_type=parquet
+
+statement ok
+DROP TABLE events;
+
+statement ok
+DROP TABLE live;
+
+statement ok
+set datafusion.execution.target_partitions = 4;
+
+statement ok
+RESET datafusion.explain.physical_plan_only;
diff --git a/datafusion/sqllogictest/test_files/explain.slt b/datafusion/sqllogictest/test_files/explain.slt
index 6c910b9681..1cf1a29558 100644
--- a/datafusion/sqllogictest/test_files/explain.slt
+++ b/datafusion/sqllogictest/test_files/explain.slt
@@ -236,6 +236,7 @@ physical_plan after OutputRequirements
01)OutputRequirementExec: order_by=[], dist_by=Unspecified
02)--DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/core/tests/data/example.csv]]}, projection=[a, b, c], file_type=csv, has_header=true
physical_plan after aggregate_statistics SAME TEXT AS ABOVE
+physical_plan after EliminateConstantJoin SAME TEXT AS ABOVE
physical_plan after join_selection SAME TEXT AS ABOVE
physical_plan after LimitedDistinctAggregation SAME TEXT AS ABOVE
physical_plan after FilterPushdown SAME TEXT AS ABOVE
@@ -351,6 +352,7 @@ physical_plan after OutputRequirements
02)--GlobalLimitExec: skip=0, fetch=10, statistics=[Rows=Exact(8), Bytes=Absent, [(Col[0]: ScanBytes=Exact(32)),(Col[1]: ScanBytes=Inexact(24)),(Col[2]: ScanBytes=Exact(32)),(Col[3]: ScanBytes=Exact(32)),(Col[4]: ScanBytes=Exact(32)),(Col[5]: ScanBytes=Exact(64)),(Col[6]: ScanBytes=Exact(32)),(Col[7]: ScanBytes=Exact(64)),(Col[8]: ScanBytes=Inexact(88)),(Col[9]: ScanBytes=Inexact(49)),(Col[10]: ScanBytes=Exact(64))]]
03)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/parquet-testing/data/alltypes_plain.parquet]]}, projection=[id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col], limit=10, file_type=parquet, statistics=[Rows=Exact(8), Bytes=Absent, [(Col[0]: ScanBytes=Exact(32)),(Col[1]: ScanBytes=Inexact(24)),(Col[2]: ScanBytes=Exact(32)),(Col[3]: ScanBytes=Exact(32)),(Col[4]: ScanBytes=Exact(32)),(Col[5]: ScanBytes=Exact(64)),(Col[6]: ScanBytes=Exact(32)),(Col[7]: ScanBytes=Exact(64)),(Col[8]: ScanBytes=Inexact(88)),(Col[9]: ScanBytes=Inexact(49)),(Col[10]: ScanBytes=Exact(64))]]
physical_plan after aggregate_statistics SAME TEXT AS ABOVE
+physical_plan after EliminateConstantJoin SAME TEXT AS ABOVE
physical_plan after join_selection SAME TEXT AS ABOVE
physical_plan after LimitedDistinctAggregation SAME TEXT AS ABOVE
physical_plan after FilterPushdown SAME TEXT AS ABOVE
@@ -397,6 +399,7 @@ physical_plan after OutputRequirements
02)--GlobalLimitExec: skip=0, fetch=10
03)----DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/parquet-testing/data/alltypes_plain.parquet]]}, projection=[id, bool_col, tinyint_col, smallint_col, int_col, bigint_col, float_col, double_col, date_string_col, string_col, timestamp_col], limit=10, file_type=parquet
physical_plan after aggregate_statistics SAME TEXT AS ABOVE
+physical_plan after EliminateConstantJoin SAME TEXT AS ABOVE
physical_plan after join_selection SAME TEXT AS ABOVE
physical_plan after LimitedDistinctAggregation SAME TEXT AS ABOVE
physical_plan after FilterPushdown SAME TEXT AS ABOVE
@@ -652,6 +655,7 @@ physical_plan after OutputRequirements
01)OutputRequirementExec: order_by=[], dist_by=Unspecified
02)--DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/core/tests/data/example.csv]]}, projection=[a, b, c], file_type=csv, has_header=true
physical_plan after aggregate_statistics SAME TEXT AS ABOVE
+physical_plan after EliminateConstantJoin SAME TEXT AS ABOVE
physical_plan after join_selection SAME TEXT AS ABOVE
physical_plan after LimitedDistinctAggregation SAME TEXT AS ABOVE
physical_plan after FilterPushdown SAME TEXT AS ABOVE
diff --git a/datafusion/sqllogictest/test_files/joins.slt b/datafusion/sqllogictest/test_files/joins.slt
index 6f498a9925..9d0541d1bb 100644
--- a/datafusion/sqllogictest/test_files/joins.slt
+++ b/datafusion/sqllogictest/test_files/joins.slt
@@ -4843,10 +4843,8 @@ logical_plan
04)----Projection: Int64(1)
05)------EmptyRelation: rows=1
physical_plan
-01)CrossJoinExec
+01)ProjectionExec: expr=[j1_string@0 as j1_string, j1_id@1 as j1_id, 1 as Int64(1)]
02)--DataSourceExec: partitions=1, partition_sizes=[0]
-03)--ProjectionExec: expr=[1 as Int64(1)]
-04)----PlaceholderRowExec
statement count 0
drop table j1;