Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Update docs

davidmezzettic50d8fd96d4c
diff --git a/docs/pipeline/data/segmentation.md b/docs/pipeline/data/segmentation.md
index 9e16b5c4..7e1bae6c 100644
--- a/docs/pipeline/data/segmentation.md
+++ b/docs/pipeline/data/segmentation.md
@@ -16,14 +16,12 @@ from txtai.pipeline import Segmentation
segment = Segmentation(sentences=True)
segment("This is a test. And another test.")
-# Segment text with Chonkie chunker (word, sentence, semantic, late etc)
+# Load third-party chunkers
segment = Segmentation(chunker="semantic")
segment("This is a test. And another test.")
```
-The Segmentation pipeline supports segmenting `sentences`, `lines`, `paragraphs` and `sections` using a rules-based approach. Each of these modes can be set when creating the pipeline.
-
-More advanced functionality is supported via a [Chonkie chunker](https://docs.chonkie.ai/chunkers/overview). The `chunker` keyword dynamically creates a Chonkie chunker. For example, `chunker='token'` creates a [TokenChunker](https://docs.chonkie.ai/chunkers/token-chunker), `chunker='semantic'` creates a [SemanticChunker](https://docs.chonkie.ai/chunkers/semantic-chunker) and so forth. Additional keyword arguments are passed to the chunker.
+The Segmentation pipeline supports segmenting `sentences`, `lines`, `paragraphs` and `sections` using a rules-based approach. Each of these modes can be set when creating the pipeline. Third-party chunkers are also supported via the `chunker` parameter.
## Configuration-driven example