Skip to content

Apache DataFusion SQL Query Engine

Docs: Enhance testing documentation with examples and links (#18851)

Andrew Lambe50b9398d0ca
diff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.md
index dd22e12360..81ceabb646 100644
--- a/docs/source/contributor-guide/testing.md
+++ b/docs/source/contributor-guide/testing.md
@@ -46,19 +46,41 @@ cargo nextest run
## Unit tests
Tests for code in an individual module are defined in the same source file with a `test` module, following Rust convention.
-The [test_util](https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs) module provides useful macros to write unit tests effectively, such as `assert_batches_sorted_eq` and `assert_batches_eq` for RecordBatches and `assert_contains` / `assert_not_contains` which are used extensively in the codebase.
+
+For example, to run tests in the `datafusion` crate:
+
+```shell
+cargo test -p datafusion
+```
+
+The [test_util] module provides useful macros to write unit tests effectively, such as [`assert_batches_sorted_eq`] and [`assert_batches_eq`] for RecordBatches and [`assert_contains`] / [`assert_not_contains`] which are used extensively in the codebase.
+
+[test_util]: https://github.com/apache/datafusion/tree/main/datafusion/common/src/test_util.rs
+[`assert_batches_sorted_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_sorted_eq.html
+[`assert_batches_eq`]: https://docs.rs/datafusion/latest/datafusion/macro.assert_batches_eq.html
+[`assert_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_contains.html
+[`assert_not_contains`]: https://docs.rs/datafusion/latest/datafusion/common/macro.assert_not_contains.html
## sqllogictests Tests
-DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest) which are run like other tests using `cargo test --test sqllogictests`.
+DataFusion's SQL implementation is tested using [sqllogictest](https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest). You can run these tests with commands like:
+
+```shell
+# Run all tests
+cargo test --profile=ci --test sqllogictests
+# Run a specific test file
+cargo test --profile=ci --test sqllogictests -- aggregate.slt
+# Run and update expected outputs
+cargo test --profile=ci --test sqllogictests -- --complete
+```
-`sqllogictests` tests may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated via `cargo test --test sqllogictests -- --complete`.
+`sqllogictests` may be less convenient for new contributors who are familiar with writing `.rs` tests as they require learning another tool. However, `sqllogictest` based tests are much easier to develop and maintain as they 1) do not require a slow recompile/link cycle and 2) can be automatically updated.
Like similar systems such as [DuckDB](https://duckdb.org/dev/testing), DataFusion has chosen to trade off a slightly higher barrier to contribution for longer term maintainability.
DataFusion has integrated [sqlite's test suite](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) as a supplemental test suite that is run whenever a PR is merged into DataFusion. To run it manually please refer to the [README](https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/README.md#running-tests-sqlite) file for instructions.
-## Snapshot testing
+## Snapshot testing (`cargo insta`)
[Insta](https://github.com/mitsuhiko/insta) is used for snapshot testing. Snapshots are generated
and compared on each test run. If the output changes, tests will fail.
@@ -90,7 +112,7 @@ There are several public interface tests for the DataFusion library in the [test
You can run these tests individually using `cargo` as normal command such as
```shell
-cargo test -p datafusion --test parquet_exec
+cargo test -p datafusion --test parquet_integration
```
## SQL "Fuzz" testing