Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

7.6 KB
Raw Blame

generated by https://github.com/hashicorp/terraform-plugin-docs

page_title: "litellm_vector_store Resource - terraform-provider-litellm" subcategory: "" description: |- Manages a LiteLLM vector store for storing and retrieving vector embeddings.

litellm_vector_store (Resource)

Manages a LiteLLM vector store for storing and retrieving vector embeddings. Vector stores enable semantic search and retrieval-augmented generation (RAG) capabilities using officially supported providers including AWS Bedrock Knowledge Bases, OpenAI Vector Stores, Azure Vector Stores, Vertex AI RAG Engine, and PG Vector.

Example Usage

AWS Bedrock Knowledge Base

resource "litellm_credential" "bedrock_cred" {
  credential_name = "bedrock-knowledge-base"
  
  credential_info = {
    provider = "bedrock"
    region   = "us-east-1"
  }
  
  credential_values = {
    aws_access_key_id     = var.aws_access_key_id
    aws_secret_access_key = var.aws_secret_access_key
    aws_region            = "us-east-1"
  }
}

resource "litellm_vector_store" "bedrock_kb" {
  vector_store_name        = "bedrock-litellm-website-knowledgebase"
  custom_llm_provider      = "bedrock"
  litellm_credential_name  = litellm_credential.bedrock_cred.credential_name
  
  vector_store_description = "Bedrock vector store for the LiteLLM website knowledgebase"
  
  vector_store_metadata = {
    source = "https://www.litellm.com/docs"
  }
  
  litellm_params = {
    vector_store_id = "T37J8R4WTM"
  }
}

OpenAI Vector Store

resource "litellm_credential" "openai_cred" {
  credential_name = "openai-vector-store"
  
  credential_info = {
    provider = "openai"
  }
  
  credential_values = {
    api_key = var.openai_api_key
  }
}

resource "litellm_vector_store" "openai_store" {
  vector_store_name        = "openai-knowledge-base"
  custom_llm_provider      = "openai"
  litellm_credential_name  = litellm_credential.openai_cred.credential_name
  
  vector_store_description = "OpenAI vector store for document search"
  
  vector_store_metadata = {
    environment = "production"
    purpose     = "file-search"
  }
  
  litellm_params = {
    vector_store_id = "vs_687ae3b2439881918b433cb99d10662e"
  }
}

Azure Vector Store

resource "litellm_credential" "azure_cred" {
  credential_name = "azure-vector-store"
  
  credential_info = {
    provider = "azure"
  }
  
  credential_values = {
    api_key      = var.azure_openai_key
    api_base     = var.azure_openai_endpoint
    api_version  = "2023-12-01-preview"
  }
}

resource "litellm_vector_store" "azure_store" {
  vector_store_name        = "azure-knowledge-base"
  custom_llm_provider      = "azure"
  litellm_credential_name  = litellm_credential.azure_cred.credential_name
  
  vector_store_description = "Azure vector store for enterprise search"
  
  vector_store_metadata = {
    environment = "production"
    team        = "enterprise"
  }
  
  litellm_params = {
    vector_store_id = "vs_azure_example_id"
  }
}

Vertex AI RAG Engine

resource "litellm_credential" "vertex_cred" {
  credential_name = "vertex-rag-engine"
  
  credential_info = {
    provider = "vertex_ai"
    project  = "your-gcp-project"
  }
  
  credential_values = {
    service_account_key = var.gcp_service_account_key
  }
}

resource "litellm_vector_store" "vertex_rag" {
  vector_store_name        = "vertex-rag-corpus"
  custom_llm_provider      = "vertex_ai"
  litellm_credential_name  = litellm_credential.vertex_cred.credential_name
  
  vector_store_description = "Vertex AI RAG Engine for enterprise knowledge"
  
  vector_store_metadata = {
    project     = "your-gcp-project"
    environment = "production"
  }
  
  litellm_params = {
    vector_store_id = "6917529027641081856"
  }
}

PG Vector Store

resource "litellm_credential" "pgvector_cred" {
  credential_name = "pgvector-store"
  
  credential_info = {
    provider = "pgvector"
    host     = "your-pgvector-host.com"
  }
  
  credential_values = {
    api_key  = var.pgvector_api_key
    api_base = "https://your-pgvector-host.com"
  }
}

resource "litellm_vector_store" "pgvector_store" {
  vector_store_name        = "postgres-vector-store"
  custom_llm_provider      = "pgvector"
  litellm_credential_name  = litellm_credential.pgvector_cred.credential_name
  
  vector_store_description = "PostgreSQL vector store with pgvector extension"
  
  vector_store_metadata = {
    database    = "vector_db"
    table       = "embeddings"
    environment = "production"
  }
  
  litellm_params = {
    api_base = "https://your-pgvector-host.com"
  }
}

Argument Reference

The following arguments are supported:

  • vector_store_name - (Required) Name of the vector store.
  • custom_llm_provider - (Required) The vector store provider. Supported values: "bedrock", "openai", "azure", "vertex_ai", "pgvector".
  • vector_store_description - (Optional) Description of the vector store.
  • vector_store_metadata - (Optional) Map of metadata associated with the vector store.
  • litellm_credential_name - (Optional) Name of the LiteLLM credential to use for authentication.
  • litellm_params - (Optional, Sensitive) Map of additional parameters specific to the vector store provider. Do not put API keys or other secrets here; this map is stored unencrypted in state, and terraform plan warns when it sees a secret-looking key such as api_key or aws_secret_access_key. Store secrets in a litellm_credential (credential_values_wo) and reference it via litellm_credential_name.

Attributes Reference

In addition to all arguments above, the following attributes are exported:

  • vector_store_id - The unique identifier of the vector store.
  • created_at - Timestamp when the vector store was created.
  • updated_at - Timestamp when the vector store was last updated.

Supported Providers

The following vector store providers are officially supported by LiteLLM:

  • AWS Bedrock Knowledge Bases - Managed knowledge bases on AWS Bedrock
  • OpenAI Vector Stores - OpenAI's native vector store service
  • Azure Vector Stores - Azure OpenAI vector store integration
  • Vertex AI RAG Engine - Google Cloud's RAG API for vector search
  • PG Vector - PostgreSQL with pgvector extension

Provider-Specific Parameters

AWS Bedrock Knowledge Base

litellm_params = {
  vector_store_id = "T37J8R4WTM"  # Your Bedrock Knowledge Base ID
}

OpenAI Vector Store

litellm_params = {
  vector_store_id = "vs_687ae3b2439881918b433cb99d10662e"  # Your OpenAI Vector Store ID
}

Azure Vector Store

litellm_params = {
  vector_store_id = "vs_azure_example_id"  # Your Azure Vector Store ID
}

Vertex AI RAG Engine

litellm_params = {
  vector_store_id = "6917529027641081856"  # Your Vertex AI RAG Engine ID
}

PG Vector

litellm_params = {
  api_base = "https://your-pgvector-host.com"
}

Import

Vector stores can be imported using their ID:

terraform import litellm_vector_store.example "vector-store-id"

Notes

  • Vector stores require appropriate credentials for the chosen provider.
  • The litellm_params field allows provider-specific configuration.
  • Some providers may require additional setup outside of Terraform (e.g., creating Knowledge Bases in AWS Bedrock, Vector Stores in OpenAI).
  • Ensure your vector store provider is properly configured and accessible from your LiteLLM instance.
  • Only the officially supported providers listed above are guaranteed to work with LiteLLM's vector store integration.
  • For the most up-to-date list of supported providers, refer to the LiteLLM documentation.