Skip to content

Apache DataFusion SQL Query Engine

docs: document DELETE and UPDATE for SQL users and table provider authors (#24567)

mckac37adf5a909
diff --git a/datafusion/session/src/table.rs b/datafusion/session/src/table.rs
index cd824938c0..112d281741 100644
--- a/datafusion/session/src/table.rs
+++ b/datafusion/session/src/table.rs
@@ -359,8 +359,38 @@ pub trait TableProvider: Any + Debug + Sync + Send {
/// Delete rows matching the filter predicates.
///
- /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64).
- /// Empty `filters` deletes all rows.
+ /// Returns an [`ExecutionPlan`] producing one row in a single non-null
+ /// `UInt64` column named `count`, containing the number of deleted rows.
+ /// The default implementation returns a "not implemented" error.
+ ///
+ /// # Filters
+ ///
+ /// `filters` contains logical [`Expr`] predicates on the target table.
+ /// The planner collects them from filters and pushed-down table scan
+ /// predicates, splits `AND` conjunctions, removes table qualifiers, and
+ /// deduplicates them. For example, `t.id = 1 AND t.value > 15` becomes
+ /// separate `id = 1` and `value > 15` expressions.
+ ///
+ /// Delete a row only when every predicate evaluates to true. SQL
+ /// three-valued logic applies: a false or `NULL` predicate leaves the row
+ /// unchanged. Empty `filters` deletes all rows, as for `DELETE FROM t`
+ /// without a `WHERE` clause.
+ ///
+ /// # Execution
+ ///
+ /// This method is called during physical planning, including for `EXPLAIN`.
+ /// Perform mutations when the returned plan executes, rather than while
+ /// constructing it, so planning does not change the table.
+ ///
+ /// # Limitations
+ ///
+ /// The method receives no row limit ([#24998]). Subqueries in DML
+ /// expressions are not fully supported ([#24654]); in particular, a
+ /// subquery rewritten into a join can cause predicates to be lost before
+ /// this method is called.
+ ///
+ /// [#24998]: https://github.com/apache/datafusion/issues/24998
+ /// [#24654]: https://github.com/apache/datafusion/issues/24654
// Hand-written `#[async_trait]` expansion to reduce compile time. See
// <https://github.com/apache/datafusion/issues/13814#issuecomment-5292709677>
fn delete_from<'life0, 'life1, 'async_trait>(
@@ -381,8 +411,43 @@ pub trait TableProvider: Any + Debug + Sync + Send {
/// Update rows matching the filter predicates.
///
- /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64).
- /// Empty `filters` updates all rows.
+ /// Returns an [`ExecutionPlan`] producing one row in a single non-null
+ /// `UInt64` column named `count`, containing the number of affected rows.
+ /// The default implementation returns a "not implemented" error.
+ ///
+ /// # Filters
+ ///
+ /// `filters` follows the same conventions as [`Self::delete_from`]:
+ /// predicates are combined with `AND`, table qualifiers are removed, and
+ /// only rows for which every predicate is true are updated. A false or
+ /// `NULL` predicate leaves the row unchanged. Empty `filters` updates all
+ /// rows.
+ ///
+ /// # Assignments
+ ///
+ /// `assignments` contains `(column_name, Expr)` pairs from the `SET`
+ /// clause. The planner removes identity assignments and strips table
+ /// qualifiers from the expressions. Leave columns without an assignment
+ /// unchanged.
+ ///
+ /// Evaluate every assignment against the row values from before the
+ /// statement, and only for matching rows. For example, `SET a = b, b = a`
+ /// exchanges the two values; an expression such as `100 / divisor` must
+ /// not be evaluated on rows excluded by the filters.
+ ///
+ /// # Execution
+ ///
+ /// Like [`Self::delete_from`], this method is called during physical
+ /// planning, including for `EXPLAIN`. Perform mutations when the returned
+ /// plan executes so planning does not change the table.
+ ///
+ /// # Limitations
+ ///
+ /// Subqueries have the same limitations as in [`Self::delete_from`]
+ /// ([#24654]). `UPDATE ... FROM` is not supported ([#19950]).
+ ///
+ /// [#24654]: https://github.com/apache/datafusion/issues/24654
+ /// [#19950]: https://github.com/apache/datafusion/issues/19950
// Hand-written `#[async_trait]` expansion to reduce compile time. See
// <https://github.com/apache/datafusion/issues/13814#issuecomment-5292709677>
fn update<'life0, 'life1, 'async_trait>(
diff --git a/docs/source/library-user-guide/custom-table-providers.md b/docs/source/library-user-guide/custom-table-providers.md
index dbbdf9d022..30ffb2b5cf 100644
--- a/docs/source/library-user-guide/custom-table-providers.md
+++ b/docs/source/library-user-guide/custom-table-providers.md
@@ -792,6 +792,19 @@ that a `FilterExec` is unnecessary for the `date` predicate, and the second
ensures that only the relevant directories are scanned. The actual file reading
happens later, in the stream produced by `execute()`.
+## Row-Level DML: DELETE and UPDATE
+
+A custom table provider can support `DELETE` and `UPDATE` by implementing the optional [`TableProvider::delete_from()`] and [`TableProvider::update()`] methods. Their default implementations return a "not implemented" error.
+
+Each method builds an [ExecutionPlan] that changes the matching rows and returns the number of affected rows in a `count` column. Your provider decides how to apply and persist the changes. Perform the changes when the returned plan executes: these methods run during physical planning, including for `EXPLAIN`.
+
+The method documentation describes the filters, assignments, SQL semantics, and result schema that an implementation must support. [MemTable] provides an implementation of both methods over in-memory batches.
+
+For SQL syntax and examples, see the [DML section](../user-guide/sql/dml.md) of the user guide.
+
+[`tableprovider::delete_from()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.delete_from
+[`tableprovider::update()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.update
+
## Putting It All Together
Here is a minimal but complete example of a custom table provider that generates