Skip to content

Apache DataFusion SQL Query Engine

docs: Document the convention for internal public APIs (#25495)

Yongting Youc55319e0b630
diff --git a/datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs b/datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs
index 98e8b2d2fc..82863c0802 100644
--- a/datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs
+++ b/datafusion/physical-plan/src/joins/hash_join/partitioned_hash_eval.rs
@@ -293,9 +293,11 @@ impl HashTableLookupExpr {
/// * `random_state` - SeededRandomState for hashing
/// * `map` - Map to check membership (hash table or array map)
/// * `description` - Description for debugging
- /// # Note
- /// This is public for internal testing purposes only and is not
- /// guaranteed to be stable across versions.
+ ///
+ /// # Public Only for Internal Use:
+ /// `datafusion-proto` tests require this constructor, but it is not part of
+ /// the supported public API.
+ #[doc(hidden)]
pub fn new(
on_columns: Vec<PhysicalExprRef>,
random_state: SeededRandomState,
diff --git a/docs/source/contributor-guide/api-health.md b/docs/source/contributor-guide/api-health.md
index a20bc284cf..8737df280f 100644
--- a/docs/source/contributor-guide/api-health.md
+++ b/docs/source/contributor-guide/api-health.md
@@ -43,6 +43,35 @@ Examples of non-breaking changes include:
- Marking a function as deprecated (`#[deprecated]`)
- Adding a new function to a `trait` with a default implementation
+### DataFusion-internal Public APIs
+
+Some internal implementation details require `pub` visibility because they are
+shared across DataFusion crates. These APIs are not intended as downstream
+extension points. They are hidden from generated documentation and are not part
+of DataFusion's supported public API, so they may be changed or removed without
+notice or a deprecation period. Examples include:
+
+1. Test helpers.
+2. Operator APIs required by the optimizer to inspect or rewrite execution plans
+ across crate boundaries.
+
+For APIs intended only for internal use, add `#[doc(hidden)]` and a doc comment
+section headed `# Public Only for Internal Use:`. Name the crate or component
+that requires access and explain why the API is not intended for downstream use.
+For example:
+
+```txt
+impl HashTableLookupExpr {
+ /// ...
+ ///
+ /// # Public Only for Internal Use:
+ /// `datafusion-proto` tests require this constructor, but it is not part of
+ /// the supported public API.
+ #[doc(hidden)]
+ pub fn new(...) {...}
+}
+```
+
### What is the public SQL API and what is a breaking SQL change?
DataFusion is also used as a SQL engine, so changes to SQL semantics (the