Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Update documentation, closes #156

davidmezzettid78278317cfa
diff --git a/docs/pipelines/transcription.md b/docs/pipelines/audio/transcription.md
similarity index 100%
rename from docs/pipelines/transcription.md
rename to docs/pipelines/audio/transcription.md
diff --git a/docs/pipelines/segmentation.md b/docs/pipelines/data/segmentation.md
similarity index 100%
rename from docs/pipelines/segmentation.md
rename to docs/pipelines/data/segmentation.md
diff --git a/docs/pipelines/tabular.md b/docs/pipelines/data/tabular.md
similarity index 100%
rename from docs/pipelines/tabular.md
rename to docs/pipelines/data/tabular.md
diff --git a/docs/pipelines/textractor.md b/docs/pipelines/data/textractor.md
similarity index 100%
rename from docs/pipelines/textractor.md
rename to docs/pipelines/data/textractor.md
diff --git a/docs/pipelines/image/caption.md b/docs/pipelines/image/caption.md
new file mode 100644
index 00000000..b9c905dc
--- /dev/null
+++ b/docs/pipelines/image/caption.md
@@ -0,0 +1,10 @@
+# Caption
+
+A caption pipeline reads a list of images and returns a list of captions for those images.
+
+```python
+Caption()
+```
+
+::: txtai.pipeline.Caption.__init__
+::: txtai.pipeline.Caption.__call__
diff --git a/docs/pipelines/image/objects.md b/docs/pipelines/image/objects.md
new file mode 100644
index 00000000..6cfa3fc2
--- /dev/null
+++ b/docs/pipelines/image/objects.md
@@ -0,0 +1,14 @@
+# Objects
+
+A objects pipeline reads a list of images and returns a list of detected objects.
+
+```python
+# Default configuration
+Objects()
+
+# Detect objects with an image classificaton model
+Objects(classification=True, threshold=0.95)
+```
+
+::: txtai.pipeline.Objects.__init__
+::: txtai.pipeline.Objects.__call__
diff --git a/docs/pipelines/overview.md b/docs/pipelines/overview.md
index ae00c639..b344ef60 100644
--- a/docs/pipelines/overview.md
+++ b/docs/pipelines/overview.md
@@ -5,17 +5,22 @@ process various types of data. Pipelines can wrap machine learning models as wel
The following pipeline types are currently supported.
-- Hugging Face Transformers pipelines
- - [Extractive QA](../extractor)
- - [Labeling](../labels)
- - [Similarity](../similarity)
- - [Summary](../summary)
-- Hugging Face Transformers models
- - [ONNX](../onnx)
- - [Trainer](../trainer)
- - [Transcription](../transcription)
- - [Translation](../translation)
-- Data processing
- - [Segmentation](../segmentation)
- - [Tabular](../tabular)
- - [Text extraction](../textractor)
+- Audio
+ - [Transcription](../audio/transcription)
+- Data Processing
+ - [Segmentation](../data/segmentation)
+ - [Tabular](../data/tabular)
+ - [Text extraction](../data/textractor)
+- Image
+ - [Caption](../image/caption)
+ - [Objects](../image/objects)
+- Text
+ - [Extractive QA](../text/extractor)
+ - [Labeling](../text/labels)
+ - [Similarity](../text/similarity)
+ - [Summary](../text/summary)
+ - [Translation](../text/translation)
+- Training
+ - [HF ONNX](../train/hfonnx)
+ - [ML ONNX](../train/mlonnx)
+ - [Trainer](../train/trainer)
diff --git a/docs/pipelines/extractor.md b/docs/pipelines/text/extractor.md
similarity index 100%
rename from docs/pipelines/extractor.md
rename to docs/pipelines/text/extractor.md
diff --git a/docs/pipelines/labels.md b/docs/pipelines/text/labels.md
similarity index 100%
rename from docs/pipelines/labels.md
rename to docs/pipelines/text/labels.md
diff --git a/docs/pipelines/similarity.md b/docs/pipelines/text/similarity.md
similarity index 100%
rename from docs/pipelines/similarity.md
rename to docs/pipelines/text/similarity.md
diff --git a/docs/pipelines/summary.md b/docs/pipelines/text/summary.md
similarity index 100%
rename from docs/pipelines/summary.md
rename to docs/pipelines/text/summary.md
diff --git a/docs/pipelines/translation.md b/docs/pipelines/text/translation.md
similarity index 100%
rename from docs/pipelines/translation.md
rename to docs/pipelines/text/translation.md
diff --git a/docs/pipelines/onnx.md b/docs/pipelines/train/hfonnx.md
similarity index 100%
rename from docs/pipelines/onnx.md
rename to docs/pipelines/train/hfonnx.md
diff --git a/docs/pipelines/train/mlonnx.md b/docs/pipelines/train/mlonnx.md
new file mode 100644
index 00000000..20e1494b
--- /dev/null
+++ b/docs/pipelines/train/mlonnx.md
@@ -0,0 +1,6 @@
+# MLOnnx
+
+Exports a traditional machine learning model (i.e. scikit-learn) to ONNX.
+
+::: txtai.pipeline.MLOnnx.__init__
+::: txtai.pipeline.MLOnnx.__call__
diff --git a/docs/pipelines/trainer.md b/docs/pipelines/train/trainer.md
similarity index 100%
rename from docs/pipelines/trainer.md
rename to docs/pipelines/train/trainer.md
diff --git a/docs/workflows.md b/docs/workflows.md
deleted file mode 100644
index c4c5bc61..00000000
--- a/docs/workflows.md
+++ /dev/null
@@ -1,74 +0,0 @@
-# Workflows
-
-Workflows are a simple yet powerful construct that takes a callable and returns elements. Workflows don't know they are working with pipelines but enable efficient processing of pipeline data. Workflows are streaming by nature and work on data in batches, allowing large volumes of data to be processed efficiently.
-
-An example Workflow is shown below. This workflow will work with both documents and audio files. Documents will have text extracted and summarized. Audio files will be transcribed. Both results will be joined, translated into French and loaded into an Embeddings index.
-
-```python
-# file:// prefixes are required to signal to the workflow this is a file and not a text string
-files = [
- "file://txtai/article.pdf",
- "file://txtai/US_tops_5_million.wav",
- "file://txtai/Canadas_last_fully.wav",
- "file://txtai/Beijing_mobilises.wav",
- "file://txtai/The_National_Park.wav",
- "file://txtai/Maine_man_wins_1_mil.wav",
- "file://txtai/Make_huge_profits.wav"
-]
-
-data = [(x, element, None) for x, element in enumerate(files)]
-
-# Workflow that extracts text and builds a summary
-articles = Workflow([
- FileTask(textractor),
- Task(lambda x: summary([y[:1024] for y in x]))
-])
-
-# Define workflow tasks. Workflows can also be tasks!
-tasks = [
- WorkflowTask(articles, r".\.pdf$"),
- FileTask(transcribe, r"\.wav$"),
- Task(lambda x: translate(x, "fr")),
- Task(index, unpack=False)
-]
-
-# Workflow that translates text to French
-workflow = Workflow(tasks)
-for _ in workflow(data):
- pass
-```
-
-::: txtai.workflow.Workflow.__init__
-::: txtai.workflow.Workflow.__call__
-
-## Tasks
-
-::: txtai.workflow.Task.__init__
-
-### File Task
-
-Task that processes file urls
-
-### Image Task
-
-Task that processes image urls
-
-### Service Task
-
-Task that runs content against a http service
-
-::: txtai.workflow.ServiceTask.__init__
-
-### Storage Task
-
-Task that expands a local directory or cloud storage bucket into a list of URLs to process
-
-::: txtai.workflow.StorageTask.__init__
-
-### URL Task
-
-Task that processes urls
-
-### Workflow Task
-
-Task that runs a Workflow. Allows creating workflows of workflows.
diff --git a/docs/workflows/file.md b/docs/workflows/file.md
new file mode 100644
index 00000000..0815ce1b
--- /dev/null
+++ b/docs/workflows/file.md
@@ -0,0 +1,10 @@
+## File Task
+
+Task that processes file urls.
+
+```python
+workflow = Workflow([FileTask()])
+workflow(["/path/to/file", "file:///path/to/file"])
+```
+
+::: txtai.workflow.FileTask.__init__
diff --git a/docs/workflows/image.md b/docs/workflows/image.md
new file mode 100644
index 00000000..4da58a8a
--- /dev/null
+++ b/docs/workflows/image.md
@@ -0,0 +1,10 @@
+## Image Task
+
+Task that processes image urls.
+
+```python
+workflow = Workflow([ImageTask()])
+workflow(["image.jpg", "image.gif"])
+```
+
+::: txtai.workflow.ImageTask.__init__
diff --git a/docs/workflows/overview.md b/docs/workflows/overview.md
new file mode 100644
index 00000000..0df99dd5
--- /dev/null
+++ b/docs/workflows/overview.md
@@ -0,0 +1,70 @@
+# Workflows
+
+Workflows are a simple yet powerful construct that takes a callable and returns elements. Workflows operate well with pipelines but can work with any callable object. Workflows are streaming by nature and work on data in batches, allowing large volumes of data to be processed efficiently.
+
+Given that pipelines are callable objects, workflows enable efficient processing of pipeline data. Transformers models typically work with smaller batches of data, workflows are well suited to feed a series of transformers pipelines.
+
+An example of the most basic workflow:
+
+```python
+workflow = Workflow([Task(lambda x: [y * 2 for y in x])])
+list(workflow([1, 2, 3]))
+```
+
+This example simply multiplies each input value and returns a outputs via a generator.
+
+A more complex example is shown below. This workflow will work with both documents and audio files. Documents will have text extracted and summarized. Audio files will be transcribed. Both results will be joined, translated into French and loaded into an Embeddings index.
+
+```python
+# file:// prefixes are required to signal to the workflow this is a file and not a text string
+files = [
+ "file://txtai/article.pdf",
+ "file://txtai/US_tops_5_million.wav",
+ "file://txtai/Canadas_last_fully.wav",
+ "file://txtai/Beijing_mobilises.wav",
+ "file://txtai/The_National_Park.wav",
+ "file://txtai/Maine_man_wins_1_mil.wav",
+ "file://txtai/Make_huge_profits.wav"
+]
+
+data = [(x, element, None) for x, element in enumerate(files)]
+
+# Workflow that extracts text and builds a summary
+articles = Workflow([
+ FileTask(textractor),
+ Task(lambda x: summary([y[:1024] for y in x]))
+])
+
+# Define workflow tasks. Workflows can also be tasks!
+tasks = [
+ WorkflowTask(articles, r".\.pdf$"),
+ FileTask(transcribe, r"\.wav$"),
+ Task(lambda x: translate(x, "fr")),
+ Task(index, unpack=False)
+]
+
+# Workflow that translates text to French
+workflow = Workflow(tasks)
+for _ in workflow(data):
+ pass
+```
+
+Workflows can be defined using Python as described here but they can also be created as YAML configuration and run as shown below.
+
+```python
+with open("workflow.yml") as config:
+ workflow = yaml.safe_load(config)
+
+# Create and run the workflow
+app = API(workflow)
+data = list(app.workflow("workflow", ["input text"]))
+```
+
+[Read more here on creating workflow YAML](../../api).
+
+# Workflow
+
+Workflows are callable objects. Workflows take an input of iterable data elements and output iterable data elements.
+
+### ::: txtai.workflow.Workflow.__init__
+### ::: txtai.workflow.Workflow.__call__
diff --git a/docs/workflows/retrieve.md b/docs/workflows/retrieve.md
new file mode 100644
index 00000000..36eaaef2
--- /dev/null
+++ b/docs/workflows/retrieve.md
@@ -0,0 +1,10 @@
+## Retrieve Task
+
+Task that connects to a url and downloads the content locally.
+
+```python
+workflow = Workflow([RetrieveTask(directory="/tmp")])
+workflow(["https://file.to.download", "/local/file/to/copy"])
+```
+
+::: txtai.workflow.RetrieveTask.__init__
diff --git a/docs/workflows/service.md b/docs/workflows/service.md
new file mode 100644
index 00000000..be2a135b
--- /dev/null
+++ b/docs/workflows/service.md
@@ -0,0 +1,5 @@
+## Service Task
+
+Task that runs content against a http service.
+
+::: txtai.workflow.ServiceTask.__init__
diff --git a/docs/workflows/storage.md b/docs/workflows/storage.md
new file mode 100644
index 00000000..2602ada0
--- /dev/null
+++ b/docs/workflows/storage.md
@@ -0,0 +1,5 @@
+## Storage Task
+
+Task that expands a local directory or cloud storage bucket into a list of URLs to process.
+
+::: txtai.workflow.StorageTask.__init__
diff --git a/docs/workflows/task.md b/docs/workflows/task.md
new file mode 100644
index 00000000..5f2b5599
--- /dev/null
+++ b/docs/workflows/task.md
@@ -0,0 +1,61 @@
+# Tasks
+
+Workflows execute tasks. Tasks are callable objects with a number of parameters to control the processing of data at a given step.
+
+A simple task is shown below.
+
+```python
+Task(lambda x: [y * 2 for y in x])
+```
+
+The task above executes the function above for all input elements.
+
+Tasks work well with pipelines, since pipelines are callable objects. The example below will summarize each input element.
+
+```python
+summary = Summary()
+Task(summary)
+```
+
+Tasks can operate independently but work best with workflows, as workflows add large-scale stream processing.
+
+```python
+summary = Summary()
+task = Task(summary)
+task(["Very long text here"])
+
+workflow = Workflow([task])
+list(workflow(["Very long text here"]))
+```
+
+::: txtai.workflow.Task.__init__
+
+## Multi-action task merges
+
+Multi-action tasks will generate parallel outputs for the input data. The task output can be merged together in a couple different ways.
+
+### ::: txtai.workflow.Task.hstack
+### ::: txtai.workflow.Task.vstack
+### ::: txtai.workflow.Task.concat
+
+## Extract task output columns
+
+With column-wise merging, each output row will be a tuple of output values for each task action. This can be fed as input to a downstream task and that task can have separate tasks work with each element.
+
+A simple example:
+
+```python
+workflow = Workflow([Task(lambda x: [y * 3 for y in x], unpack=False, column=0)])
+list(workflow([(2, 8)]))
+```
+
+For the example input tuple of (2, 2), the workflow will only select the first element (2) and run the task against that element.
+
+```python
+workflow = Workflow([Task([lambda x: [y * 3 for y in x],
+ lambda x: [y - 1 for y in x]],
+ unpack=False, column={0:0, 1:1})])
+list(workflow([(2, 8)]))
+```
+
+The example above applies a separate action to each input column. This simple construct can help build extremely powerful workflow graphs!
diff --git a/docs/workflows/url.md b/docs/workflows/url.md
new file mode 100644
index 00000000..2928625a
--- /dev/null
+++ b/docs/workflows/url.md
@@ -0,0 +1,10 @@
+## Url Task
+
+Task that processes urls.
+
+```python
+workflow = Workflow([UrlTask()])
+workflow(["https://file.to.download", "file:////local/file/to/copy"])
+```
+
+::: txtai.workflow.UrlTask.__init__
diff --git a/docs/workflows/workflow.md b/docs/workflows/workflow.md
new file mode 100644
index 00000000..8bb0d56d
--- /dev/null
+++ b/docs/workflows/workflow.md
@@ -0,0 +1,3 @@
+## Workflow Task
+
+Task that runs a Workflow. Allows creating workflows of workflows.
diff --git a/mkdocs.yml b/mkdocs.yml
index 83932527..13126cfd 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -35,17 +35,35 @@ nav:
- Embeddings: embeddings.md
- Pipelines:
- Overview: pipelines/overview.md
- - Extractor: pipelines/extractor.md
- - Labels: pipelines/labels.md
- - ONNX: pipelines/onnx.md
- - Segmentation: pipelines/segmentation.md
- - Similarity: pipelines/similarity.md
- - Summary: pipelines/summary.md
- - Tabular: pipelines/tabular.md
- - Textractor: pipelines/textractor.md
- - Trainer: pipelines/trainer.md
- - Transcription: pipelines/transcription.md
- - Translation: pipelines/translation.md
- - Workflows: workflows.md
+ - Audio:
+ - Transcription: pipelines/audio/transcription.md
+ - Data:
+ - Segmentation: pipelines/data/segmentation.md
+ - Tabular: pipelines/data/tabular.md
+ - Textractor: pipelines/data/textractor.md
+ - Image:
+ - Caption: pipelines/image/caption.md
+ - Objects: pipelines/image/objects.md
+ - Text:
+ - Extractor: pipelines/text/extractor.md
+ - Labels: pipelines/text/labels.md
+ - Similarity: pipelines/text/similarity.md
+ - Summary: pipelines/text/summary.md
+ - Translation: pipelines/text/translation.md
+ - Train:
+ - HF ONNX: pipelines/train/hfonnx.md
+ - ML ONNX: pipelines/train/mlonnx.md
+ - Trainer: pipelines/train/trainer.md
+ - Workflows:
+ - Overview: workflows/overview.md
+ - Task: workflows/task.md
+ - File: workflows/file.md
+ - Image: workflows/image.md
+ - Retrieve: workflows/retrieve.md
+ - Service: workflows/service.md
+ - Storage: workflows/storage.md
+ - Url: workflows/url.md
+ - Workflow: workflows/workflow.md
+
- API: api.md
- Further Reading: articles.md