Skip to content

armgit / txtai

Public Imported from GitHub
0

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

706 B
Raw Blame

Storage Task

task task

The Storage Task expands a local directory or cloud storage bucket into a list of URLs to process.

Example

The following shows a simple example using this task as part of a workflow.

from txtai.workflow import StorageTask, Workflow

workflow = Workflow([StorageTask()])
workflow(["s3://path/to/bucket", "local://local/directory"])

Configuration-driven example

This task can also be created with workflow configuration.

workflow:
  name:
    tasks:
      - task: storage

Methods

Python documentation for the task.

::: txtai.workflow.StorageTask.init