Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

Add documentation on local API instances, closes #209

davidmezzettic6ecf2cb3c6d
diff --git a/docs/api/index.md b/docs/api/index.md
index 0c69444d..49b5df1a 100644
--- a/docs/api/index.md
+++ b/docs/api/index.md
@@ -7,7 +7,7 @@ txtai has a full-featured API, backed by [FastAPI](https://github.com/tiangolo/f
The following is an example configuration and startup script for the API.
-Note: that this configuration file enables all functionality. It is suggested that separate processes are used for each instance of a txtai component.
+Note: This configuration file enables all functionality. For memory-bound systems, splitting pipelines into multiple instances is a best practice.
```yaml
# Index file path
@@ -89,9 +89,26 @@ curl \
-d '{"name":"sumfrench", "elements": ["https://github.com/neuml/txtai"]}'
```
+## Local instance
+
+A local API instance can be instantiated. In this case, the API runs internally, without any network connections, providing the same consolidated functionality. This enables running txtai in Python with configuration.
+
+The configuration above can be run in Python with:
+
+```python
+from txtai.api import API
+
+app = API(index.yml)
+
+# Run action
+app.workflow("sumfrench", ["https://github.com/neuml/txtai"])
+```
+
+See this [link for a full list of methods](./methods).
+
## Supported language bindings
-The following programming languages have txtai bindings:
+The following programming languages have bindings with the txtai API:
- [JavaScript](https://github.com/neuml/txtai.js)
- [Java](https://github.com/neuml/txtai.java)
diff --git a/docs/api/methods.md b/docs/api/methods.md
new file mode 100644
index 00000000..70eae734
--- /dev/null
+++ b/docs/api/methods.md
@@ -0,0 +1,12 @@
+# Methods
+
+::: txtai.api.API
+ selection:
+ filters:
+ - "!__del__"
+ - "!flows"
+ - "!function"
+ - "!limit"
+ - "!pipes"
+ - "!read"
+ - "!resolve"
diff --git a/docs/embeddings/methods.md b/docs/embeddings/methods.md
index 73a535ef..7cdb6df9 100644
--- a/docs/embeddings/methods.md
+++ b/docs/embeddings/methods.md
@@ -1,3 +1,10 @@
# Methods
::: txtai.embeddings.Embeddings
+ selection:
+ filters:
+ - "!checkarchive"
+ - "!configure"
+ - "!createdatabase"
+ - "!loadvectors"
+ - "!normalize"
diff --git a/docs/pipeline/audio/transcription.md b/docs/pipeline/audio/transcription.md
index bda61d9d..ec48c2a1 100644
--- a/docs/pipeline/audio/transcription.md
+++ b/docs/pipeline/audio/transcription.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/data/segmentation.md b/docs/pipeline/data/segmentation.md
index 1caf6400..e3931a84 100644
--- a/docs/pipeline/data/segmentation.md
+++ b/docs/pipeline/data/segmentation.md
@@ -19,7 +19,7 @@ segment("This is a test. And another test.")
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/data/tabular.md b/docs/pipeline/data/tabular.md
index 852b08d3..a72c14ff 100644
--- a/docs/pipeline/data/tabular.md
+++ b/docs/pipeline/data/tabular.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/data/textractor.md b/docs/pipeline/data/textractor.md
index 6f10ee6e..3a377caa 100644
--- a/docs/pipeline/data/textractor.md
+++ b/docs/pipeline/data/textractor.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/image/caption.md b/docs/pipeline/image/caption.md
index 0eacec3a..dcfc2607 100644
--- a/docs/pipeline/image/caption.md
+++ b/docs/pipeline/image/caption.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/image/objects.md b/docs/pipeline/image/objects.md
index 77b39b59..43c05d6d 100644
--- a/docs/pipeline/image/objects.md
+++ b/docs/pipeline/image/objects.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/index.md b/docs/pipeline/index.md
index 2afec18f..6e28a4ec 100644
--- a/docs/pipeline/index.md
+++ b/docs/pipeline/index.md
@@ -5,7 +5,7 @@
txtai provides a generic pipeline processing framework with the only interface requirement being a `__call__` method. Pipelines are flexible and process various types of data. Pipelines can wrap machine learning models as well as other processes.
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../workflow/#configuration-driven-example) or the [API](../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../workflow/#configuration-driven-example) or the [API](../api#local-instance).
## List of pipelines
diff --git a/docs/pipeline/text/entity.md b/docs/pipeline/text/entity.md
index d6c56141..f0c3eaf6 100644
--- a/docs/pipeline/text/entity.md
+++ b/docs/pipeline/text/entity.md
@@ -26,7 +26,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/text/extractor.md b/docs/pipeline/text/extractor.md
index 968b9510..d8e104e4 100644
--- a/docs/pipeline/text/extractor.md
+++ b/docs/pipeline/text/extractor.md
@@ -32,7 +32,7 @@ See the links below for more detailed examples.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/text/labels.md b/docs/pipeline/text/labels.md
index c4449cac..14a284d1 100644
--- a/docs/pipeline/text/labels.md
+++ b/docs/pipeline/text/labels.md
@@ -28,7 +28,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/text/similarity.md b/docs/pipeline/text/similarity.md
index 56d87df1..53dbfce5 100644
--- a/docs/pipeline/text/similarity.md
+++ b/docs/pipeline/text/similarity.md
@@ -30,7 +30,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/text/summary.md b/docs/pipeline/text/summary.md
index b24916b1..f2496d17 100644
--- a/docs/pipeline/text/summary.md
+++ b/docs/pipeline/text/summary.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/docs/pipeline/text/translation.md b/docs/pipeline/text/translation.md
index e8597a43..288ecef7 100644
--- a/docs/pipeline/text/translation.md
+++ b/docs/pipeline/text/translation.md
@@ -25,7 +25,7 @@ See the link below for a more detailed example.
## Configuration-driven example
-Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api).
+Pipelines are run with Python or configuration. Pipelines can be instantiated in [configuration](../../../api/configuration/#pipeline) using the lower case name of the pipeline. Configuration-driven pipelines are run with [workflows](../../../workflow/#configuration-driven-example) or the [API](../../../api#local-instance).
### config.yml
```yaml
diff --git a/mkdocs.yml b/mkdocs.yml
index 7761ffda..d5c25c00 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -90,7 +90,8 @@ nav:
- Workflow: workflow/task/workflow.md
- API:
- api/index.md
- - Configuration: api/configuration.md
- Cluster: api/cluster.md
+ - Configuration: api/configuration.md
- Docker: api/docker.md
+ - Methods: api/methods.md
- Further Reading: further.md