Apache DataFusion SQL Query Engine
Docs: Enhance testing documentation with examples and links (#18851)
Andrew Lambe50b9398d0ca
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.mdindex dd22e12360..81ceabb646 100644--- a/docs/source/contributor-guide/testing.md+++ b/docs/source/contributor-guide/testing.md@@ -46,19 +46,41 @@ cargo nextest run## Unit testsTests for code in an individual module are defined in the same source file with a `test` module, following Rust convention.-The [test_util](https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs) module provides useful macros to write unit tests effectively, such as `assert_batches_sorted_eq` and `assert_batches_eq` for RecordBatches and `assert_contains` / `assert_not_contains` which are used extensively in the codebase.++For example, to run tests in the `datafusion` crate:++```shell+cargo test -p datafusion+```++The [test_util] module provides useful macros to write unit tests effectively, such as [`assert_batches_sorted_eq`] and [`assert_batches_eq`] for RecordBatches and [`assert_contains`] / [`assert_not_contains`] which are used extensively in the codebase.++[test_util]: https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs+[`assert_batches_sorted_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_sorted_eq.html+[`assert_batches_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_eq.html+[`assert_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_contains.html+[`assert_not_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_not_contains.html## sqllogictests Tests-DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest) which are run like other tests using `cargo test --test sqllogictests`.+DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest). You can run these tests with commands like:++```shell+# Run all tests+cargo test --profile=ci --test sqllogictests+# Run a specific test file+cargo test --profile=ci --test sqllogictests -- aggregate.slt+# Run and update expected outputs+cargo test --profile=ci --test sqllogictests -- --complete+```-`sqllogictests` tests may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated via `cargo test --test sqllogictests -- --complete`.+`sqllogictests` may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated.Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusion has chosen to trade off a slightly higher barrier to contribution for longer term maintainability.DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions.-## Snapshot testing+## Snapshot testing (`cargo insta`)[Insta](https://github.com/mitsuhiko/insta) is used for snapshot testing. Snapshots are generatedand compared on each test run. If the output changes, tests will fail.@@ -90,7 +112,7 @@ There are several public interface tests for the DataFusion library in the [testYou can run these tests individually using `cargo` as normal command such as```shell-cargo test -p datafusion --test parquet_exec+cargo test -p datafusion --test parquet_integration```## SQL "Fuzz" testing