Apache DataFusion SQL Query Engine
docs: Make the preference for sqllogictest over Rust unit tests more visible (#25756)
Andrew Lamb266b827f5e60
diff --git a/AGENTS.md b/AGENTS.mdindex 8fdf314ed4..c35c66a3a1 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -4,6 +4,7 @@- [Quick Start Setup](docs/source/contributor-guide/development_environment.md#quick-start)- [Testing Quick Start](docs/source/contributor-guide/testing.md#testing-quick-start)+- [Choosing What Kind of Test to Write](docs/source/contributor-guide/testing.md#choosing-what-kind-of-test-to-write)- [Before Submitting a PR](docs/source/contributor-guide/index.md#before-submitting-a-pr)- [Reviewing Pull Requests](docs/source/contributor-guide/pr_review.md)- [Contributor Guide](docs/source/contributor-guide/index.md)@@ -38,7 +39,10 @@ When creating a PR, you MUST follow the [PR template](.github/pull_request_templ## Testing-If documentation files changed then run+When adding tests, you MUST follow+[Choosing What Kind of Test to Write](docs/source/contributor-guide/testing.md#choosing-what-kind-of-test-to-write):++If documentation files changed then run```bash./ci/scripts/doc_prettier_check.sh --write --allow-dirty```diff --git a/docs/source/contributor-guide/index.md b/docs/source/contributor-guide/index.mdindex 6f1a0f1c19..d6a16680dd 100644--- a/docs/source/contributor-guide/index.md+++ b/docs/source/contributor-guide/index.md@@ -122,7 +122,12 @@ before merge.# ./dev/rust_lint.sh --write --allow-dirty```-You should also run any relevant commands from the [testing quick start](testing.md#testing-quick-start).+Please ensure your PR follows the [testing guide](testing.md). In particular:++- Prefer end-to-end Public API tests such as `sqllogictest` (`.slt`) and DataFrame API,+ over Rust unit tests where possible. See+ [Choosing What Kind of Test to Write](testing.md#choosing-what-kind-of-test-to-write).+- Run any relevant commands from the [testing quick start](testing.md#testing-quick-start).## Conventional Commits & Labeling PRsdiff --git a/docs/source/contributor-guide/pr_review.md b/docs/source/contributor-guide/pr_review.mdindex 1154ae0e96..6166a741aa 100644--- a/docs/source/contributor-guide/pr_review.md+++ b/docs/source/contributor-guide/pr_review.md@@ -133,7 +133,8 @@ Guidelines for evaluating tests:1. Prefer `sqllogictest` (`.slt`) tests or DataFrame API tests wherepossible, as they exercise **user-visible behavior** and are less coupled- to internal implementation details than unit tests.+ to internal implementation details than unit tests (see+ [Choosing What Kind of Test to Write](testing.md#choosing-what-kind-of-test-to-write)).2. Verify tests cover edge cases and common failure scenarios, not just thecommon successful path. However, it is NOT necessary to test every possibleerror path, especially if it is difficult to trigger or unlikely to occur indiff --git a/docs/source/contributor-guide/testing.md b/docs/source/contributor-guide/testing.mdindex 4c097ae498..3e39ef2eb1 100644--- a/docs/source/contributor-guide/testing.md+++ b/docs/source/contributor-guide/testing.md@@ -74,6 +74,30 @@ cargo nextest run[the book]: https://doc.rust-lang.org/book/[cargo-nextest]: https://nexte.st/+## Choosing What Kind of Test to Write++When adding tests, prefer tests that exercise **user-visible behavior** over+tests of internal implementation details. Doing so means your tests are more+likely to keep working (and keep catching regressions) while the code is+refactored. In rough order of preference:++1. Use [sqllogictest](#sqllogictests-tests) (`.slt`) tests for any behavior that can+ be expressed in SQL.++2. End-to-end tests that use public APIs such as the DataFrame API, for behavior+ that cannot be expressed in SQL. For example, DataFrame tests live in+ [`datafusion/core/tests/dataframe`] and run as part of the `core_integration`+ test binary:++ ```shell+ cargo test -p datafusion --test core_integration -- dataframe+ ```++3. Rust unit tests for logic that cannot practically be reached through+ SQL or DataFrame public APIs.++[`datafusion/core/tests/dataframe`]: https://github.com/apache/datafusion/tree/main/datafusion/core/tests/dataframe+## Unit testsTests for code in an individual module are defined in the same source file with a `test` module, following Rust convention.