Apache DataFusion SQL Query Engine
docs: document DELETE and UPDATE for SQL users and table provider authors (#24567)
mckac37adf5a909
diff --git a/datafusion/session/src/table.rs b/datafusion/session/src/table.rsindex cd824938c0..112d281741 100644--- a/datafusion/session/src/table.rs+++ b/datafusion/session/src/table.rs@@ -359,8 +359,38 @@ pub trait TableProvider: Any + Debug + Sync + Send {/// Delete rows matching the filter predicates.///- /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64).- /// Empty `filters` deletes all rows.+ /// Returns an [`ExecutionPlan`] producing one row in a single non-null+ /// `UInt64` column named `count`, containing the number of deleted rows.+ /// The default implementation returns a "not implemented" error.+ ///+ /// # Filters+ ///+ /// `filters` contains logical [`Expr`] predicates on the target table.+ /// The planner collects them from filters and pushed-down table scan+ /// predicates, splits `AND` conjunctions, removes table qualifiers, and+ /// deduplicates them. For example, `t.id = 1 AND t.value > 15` becomes+ /// separate `id = 1` and `value > 15` expressions.+ ///+ /// Delete a row only when every predicate evaluates to true. SQL+ /// three-valued logic applies: a false or `NULL` predicate leaves the row+ /// unchanged. Empty `filters` deletes all rows, as for `DELETE FROM t`+ /// without a `WHERE` clause.+ ///+ /// # Execution+ ///+ /// This method is called during physical planning, including for `EXPLAIN`.+ /// Perform mutations when the returned plan executes, rather than while+ /// constructing it, so planning does not change the table.+ ///+ /// # Limitations+ ///+ /// The method receives no row limit ([#24998]). Subqueries in DML+ /// expressions are not fully supported ([#24654]); in particular, a+ /// subquery rewritten into a join can cause predicates to be lost before+ /// this method is called.+ ///+ /// [#24998]: https://github.com/apache/datafusion/issues/24998+ /// [#24654]: https://github.com/apache/datafusion/issues/24654// Hand-written `#[async_trait]` expansion to reduce compile time. See// <https://github.com/apache/datafusion/issues/13814#issuecomment-5292709677>fn delete_from<'life0, 'life1, 'async_trait>(@@ -381,8 +411,43 @@ pub trait TableProvider: Any + Debug + Sync + Send {/// Update rows matching the filter predicates.///- /// Returns an [`ExecutionPlan`] producing a single row with `count` (UInt64).- /// Empty `filters` updates all rows.+ /// Returns an [`ExecutionPlan`] producing one row in a single non-null+ /// `UInt64` column named `count`, containing the number of affected rows.+ /// The default implementation returns a "not implemented" error.+ ///+ /// # Filters+ ///+ /// `filters` follows the same conventions as [`Self::delete_from`]:+ /// predicates are combined with `AND`, table qualifiers are removed, and+ /// only rows for which every predicate is true are updated. A false or+ /// `NULL` predicate leaves the row unchanged. Empty `filters` updates all+ /// rows.+ ///+ /// # Assignments+ ///+ /// `assignments` contains `(column_name, Expr)` pairs from the `SET`+ /// clause. The planner removes identity assignments and strips table+ /// qualifiers from the expressions. Leave columns without an assignment+ /// unchanged.+ ///+ /// Evaluate every assignment against the row values from before the+ /// statement, and only for matching rows. For example, `SET a = b, b = a`+ /// exchanges the two values; an expression such as `100 / divisor` must+ /// not be evaluated on rows excluded by the filters.+ ///+ /// # Execution+ ///+ /// Like [`Self::delete_from`], this method is called during physical+ /// planning, including for `EXPLAIN`. Perform mutations when the returned+ /// plan executes so planning does not change the table.+ ///+ /// # Limitations+ ///+ /// Subqueries have the same limitations as in [`Self::delete_from`]+ /// ([#24654]). `UPDATE ... FROM` is not supported ([#19950]).+ ///+ /// [#24654]: https://github.com/apache/datafusion/issues/24654+ /// [#19950]: https://github.com/apache/datafusion/issues/19950// Hand-written `#[async_trait]` expansion to reduce compile time. See// <https://github.com/apache/datafusion/issues/13814#issuecomment-5292709677>fn update<'life0, 'life1, 'async_trait>(diff --git a/docs/source/library-user-guide/custom-table-providers.md b/docs/source/library-user-guide/custom-table-providers.mdindex dbbdf9d022..30ffb2b5cf 100644--- a/docs/source/library-user-guide/custom-table-providers.md+++ b/docs/source/library-user-guide/custom-table-providers.md@@ -792,6 +792,19 @@ that a `FilterExec` is unnecessary for the `date` predicate, and the secondensures that only the relevant directories are scanned. The actual file readinghappens later, in the stream produced by `execute()`.+## Row-Level DML: DELETE and UPDATE++A custom table provider can support `DELETE` and `UPDATE` by implementing the optional [`TableProvider::delete_from()`] and [`TableProvider::update()`] methods. Their default implementations return a "not implemented" error.++Each method builds an [ExecutionPlan] that changes the matching rows and returns the number of affected rows in a `count` column. Your provider decides how to apply and persist the changes. Perform the changes when the returned plan executes: these methods run during physical planning, including for `EXPLAIN`.++The method documentation describes the filters, assignments, SQL semantics, and result schema that an implementation must support. [MemTable] provides an implementation of both methods over in-memory batches.++For SQL syntax and examples, see the [DML section](../user-guide/sql/dml.md) of the user guide.++[`tableprovider::delete_from()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.delete_from+[`tableprovider::update()`]: https://docs.rs/datafusion/latest/datafusion/catalog/trait.TableProvider.html#method.update+## Putting It All TogetherHere is a minimal but complete example of a custom table provider that generates