Skip to content

Hypergraph is more powerful. Transform unstructured text into structured knowledge with LLMs. Graphs, hypergraphs, and spatio-temporal extractions — with one command.

feat: unified provider system with string shorthand and updated docs

yifanfeng9727fe374c60f2
diff --git a/README.md b/README.md
index 76e9969..c540086 100644
--- a/README.md
+++ b/README.md
@@ -264,6 +264,14 @@ display:
| CLI Tool | ✅ | ❌ | ❌ | ❌ | ✅ |
| Multi-language | ✅ | ❌ | ❌ | ❌ | ✅ |
+## 🤖 Model Compatibility
+
+Hyper-Extract relies on the model's structured output capability (`json_schema` or Function Calling).
+
+**Verified compatible**: OpenAI GPT series, Bailian qwen-plus / qwen-turbo, local vLLM (Qwen3.5-9B GPTQ-Marlin), and more.
+
+> For the full model compatibility list, see [Provider System & Local Model Support](https://yifanfeng97.github.io/Hyper-Extract/latest/concepts/provider-system/).
+
## 📚 Related Documentation
- [Documentation](https://yifanfeng97.github.io/Hyper-Extract/latest/) - Complete documentation site
diff --git a/README_ZH.md b/README_ZH.md
index e32c313..a617c8d 100644
--- a/README_ZH.md
+++ b/README_ZH.md
@@ -264,6 +264,13 @@ display:
| 交互式 CLI | ✅ | ❌ | ❌ | ❌ | ✅ |
| 多语言支持 | ✅ | ❌ | ❌ | ❌ | ✅ |
+## 🤖 模型兼容性
+
+Hyper-Extract 依赖模型的结构化输出能力(`json_schema` 或 Function Calling)。
+
+**已验证兼容**:OpenAI GPT 系列、百炼 qwen-plus / qwen-turbo、本地 vLLM(Qwen3.5-9B GPTQ-Marlin)等。
+
+> 完整模型兼容性列表请参阅 [Provider 系统与本地模型支持](https://yifanfeng97.github.io/Hyper-Extract/latest/zh/concepts/provider-system/)。
## 📚 相关文档
diff --git a/docs/en/cli/commands/config.md b/docs/en/cli/commands/config.md
index be92f67..93942de 100644
--- a/docs/en/cli/commands/config.md
+++ b/docs/en/cli/commands/config.md
@@ -14,7 +14,7 @@ he config [COMMAND] [OPTIONS]
| Command | Description |
|---------|-------------|
-| `init` | Initialize configuration (lazy defaults: `gpt-4o-mini` + `text-embedding-3-small`) |
+| `init` | Initialize configuration (uses provider preset to auto-fill model and URL) |
| `show` | Display current configuration |
| `llm` | Configure LLM settings |
| `embedder` | Configure embedder settings |
@@ -23,10 +23,10 @@ he config [COMMAND] [OPTIONS]
## he config init
-Initialize configuration. This is the **lazy one-step setup** — if you pass `-k`, it skips all prompts and uses the built-in defaults:
+Initialize configuration. This is the **lazy one-step setup** — if you pass `-p` and `-k`, it skips all prompts and uses built-in presets:
-- **LLM**: `gpt-4o-mini`
-- **Embedder**: `text-embedding-3-small`
+- **OpenAI preset**: `gpt-4o-mini` + `text-embedding-3-small`
+- **Bailian preset**: `qwen3.6-plus` + `text-embedding-v4`
```bash
he config init [OPTIONS]
@@ -36,6 +36,7 @@ he config init [OPTIONS]
| Option | Short | Description |
|--------|-------|-------------|
+| `--provider` | `-p` | Provider preset (`openai` / `bailian` / `vllm`) |
| `--api-key` | `-k` | API key for both LLM and embedder |
| `--base-url` | `-u` | Custom API base URL (optional) |
@@ -44,15 +45,19 @@ he config init [OPTIONS]
#### Lazy one-step setup (recommended)
```bash
-he config init -k sk-your-api-key-here
+# OpenAI
+he config init -p openai -k sk-your-api-key-here
+
+# Bailian (Alibaba Cloud)
+he config init -p bailian -k sk-your-api-key-here
```
-This saves `gpt-4o-mini` and `text-embedding-3-small` automatically.
+This saves the preset's default model and API URL automatically.
#### With custom base URL
```bash
-he config init -k sk-your-key -u https://api.openai.com/v1
+he config init -p openai -k sk-your-key -u https://api.openai.com/v1
```
#### Interactive setup
@@ -75,14 +80,14 @@ he config show
**Output:**
```
-┌─────────────────────────────────────────────────────────┐
-│ Hyper-Extract Configuration │
-├──────────┬─────────────────────┬─────────────┬──────────┤
-│ Service │ Model │ API Key │ Base URL │
-├──────────┼─────────────────────┼─────────────┼──────────┤
-│ LLM │ gpt-4o-mini │ sk-xxxxx... │ (default)│
-│ Embedder │ text-embedding-3... │ sk-xxxxx... │ (default)│
-└──────────┴─────────────────────┴─────────────┴──────────┘
+┌──────────────────────────────────────────────────────────────────┐
+│ Hyper-Extract Configuration │
+├──────────┬──────────┬─────────────────────┬──────────┬───────────┤
+│ Service │ Provider │ Model │ API Key │ Base URL │
+├──────────┼──────────┼─────────────────────┼──────────┼───────────┤
+│ LLM │ bailian │ qwen3.6-plus │ sk-xx... │ dashsc... │
+│ Embedder │ bailian │ text-embedding-v4 │ sk-xx... │ dashsc... │
+└──────────┴──────────┴─────────────────────┴──────────┴───────────┘
```
---
@@ -111,11 +116,17 @@ he config llm [OPTIONS]
# View LLM config
he config llm --show
-# Update LLM model
-he config llm --model gpt-4o
+# Update LLM model (OpenAI)
+he config llm -p openai --model gpt-4o
+
+# Update LLM model (Bailian)
+he config llm -p bailian --model qwen-plus
-# Update LLM API key and endpoint
-he config llm --api-key sk-your-key --base-url https://api.openai.com/v1
+# Configure local vLLM
+he config llm -p vllm \
+ --api-key dummy \
+ --base-url http://localhost:8000/v1 \
+ --model Qwen/Qwen3.5-9B
# Reset LLM config
he config llm --unset
@@ -147,8 +158,14 @@ he config embedder [OPTIONS]
# View embedder config
he config embedder --show
-# Use a different embedding model
-he config embedder --model text-embedding-3-large
+# Use a larger OpenAI embedding model
+he config embedder -p openai --model text-embedding-3-large
+
+# Configure local vLLM embedder
+he config embedder -p vllm \
+ --api-key dummy \
+ --base-url http://localhost:8001/v1 \
+ --model BAAI/bge-m3
# Reset embedder config
he config embedder --unset
@@ -163,19 +180,39 @@ Configuration is stored at:
- **Linux/macOS**: `~/.he/config.toml`
- **Windows**: `%USERPROFILE%\.he\config.toml`
-### Example
-
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-your-api-key"
-base_url = "https://api.openai.com/v1"
+### Examples
-[embedder]
-model = "text-embedding-3-small"
-api_key = "" # Empty inherits from llm.api_key
-base_url = "" # Empty inherits from llm.base_url
-```
+=== "Bailian (Alibaba Cloud)"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1"
+
+ [embedder]
+ provider = "bailian"
+ model = "text-embedding-v4"
+ api_key = ""
+ base_url = ""
+ ```
+
+=== "Local vLLM"
+
+ ```toml
+ [llm]
+ provider = "vllm"
+ model = "Qwen/Qwen3.5-9B"
+ api_key = "dummy"
+ base_url = "http://localhost:8000/v1"
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
## Environment Variables
diff --git a/docs/en/cli/configuration.md b/docs/en/cli/configuration.md
index c1b9521..912fdee 100644
--- a/docs/en/cli/configuration.md
+++ b/docs/en/cli/configuration.md
@@ -1,55 +1,85 @@
# CLI Configuration Reference
-Configuration guide for Hyper-Extract CLI, from quick start to advanced settings.
+Configuration guide for Hyper-Extract CLI, supporting **OpenAI**, **Alibaba Cloud Bailian**, and **Local vLLM** deployments.
---
## Quick Start
-Complete configuration in 3 steps.
+Choose your deployment method and follow the steps.
-### 1. Initialize Configuration
+=== "OpenAI"
-```bash
-he config init -k YOUR_API_KEY
-```
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
-This creates `~/.he/config.toml` with the following defaults:
-- **LLM**: `gpt-4o-mini`
-- **Embedder**: `text-embedding-3-small`
+ This creates `~/.he/config.toml` with OpenAI presets:
+ - **LLM**: `gpt-4o-mini`
+ - **Embedder**: `text-embedding-3-small`
-### 2. Verify Configuration
+=== "Bailian (Alibaba Cloud)"
-```bash
-he config show
-```
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
-You should see output like this:
+ This creates `~/.he/config.toml` with Bailian presets:
+ - **LLM**: `qwen3.6-plus`
+ - **Embedder**: `text-embedding-v4`
-```
-┌─────────────────────────────────────────────────────────┐
-│ Hyper-Extract Configuration │
-├──────────┬─────────────────────┬─────────────┬──────────┤
-│ Service │ Model │ API Key │ Base URL │
-├──────────┼─────────────────────┼─────────────┼──────────┤
-│ LLM │ gpt-4o-mini │ sk-xxxxx... │ (default)│
-│ Embedder │ text-embedding-3... │ sk-xxxxx... │ (default)│
-└──────────┴─────────────────────┴─────────────┴──────────┘
-```
+=== "Local vLLM"
-### 3. Modify Configuration (Optional)
+ Start the LLM and Embedding services first, then configure separately:
-```bash
-# Change LLM model
-he config llm --model gpt-4o
+ ```bash
+ # LLM service
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
-# Use different embedder model
-he config embedder --model text-embedding-3-large
+ # Embedding service
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
-# Configure custom API endpoint
-he config llm --base-url https://your-api-endpoint.com/v1
+---
+
+## Verify Configuration
+
+```bash
+he config show
```
+=== "Cloud API (OpenAI / Bailian)"
+
+ ```
+ ┌─────────────────────────────────────────────────────────────┐
+ │ Hyper-Extract Configuration │
+ ├──────────┬──────────┬─────────────────────┬────────────┬────┤
+ │ Service │ Provider │ Model │ API Key │ ...│
+ ├──────────┼──────────┼─────────────────────┼────────────┼────┤
+ │ LLM │ bailian │ qwen3.6-plus │ sk-xxxx... │ ...│
+ │ Embedder │ bailian │ text-embedding-v4 │ sk-xxxx... │ ...│
+ └──────────┴──────────┴─────────────────────┴────────────┴────┘
+ ```
+
+=== "Local vLLM"
+
+ ```
+ ┌──────────────────────────────────────────────────────────────────┐
+ │ Hyper-Extract Configuration │
+ ├──────────┬──────────┬─────────────────────┬──────────┬───────────┤
+ │ Service │ Provider │ Model │ API Key │ Base URL │
+ ├──────────┼──────────┼─────────────────────┼──────────┼───────────┤
+ │ LLM │ vllm │ Qwen/Qwen3.5-9B │ dummy... │ localhost…│
+ │ Embedder │ vllm │ BAAI/bge-m3 │ dummy... │ localhost…│
+ └──────────┴──────────┴─────────────────────┴──────────┴───────────┘
+ ```
+
---
## CLI Command Reference
@@ -58,35 +88,28 @@ he config llm --base-url https://your-api-endpoint.com/v1
| Command | Common Flags | Description |
|---------|-------------|-------------|
-| `he config init` | `-k, --api-key` — API key<br>`-u, --base-url` — Custom API base URL | Initialize configuration (first time) |
-| `he config llm` | `-m, --model` — Model name<br>`-k, --api-key` — API key<br>`-u, --base-url` — Custom API base URL<br>`--show` — Show current config<br>`--unset` — Clear configuration | Configure LLM |
-| `he config embedder` | `-m, --model` — Model name<br>`-k, --api-key` — API key<br>`-u, --base-url` — Custom API base URL<br>`--show` — Show current config<br>`--unset` — Clear configuration | Configure embedder |
-| `he config show` | — | View complete configuration |
+| `he config init` | `-p, --provider` — Provider preset<br>`-k, --api-key` — API key<br>`-u, --base-url` — Custom API endpoint | Initialize configuration (first time) |
+| `he config llm` | `-p, --provider` — Provider<br>`-m, --model` — Model name<br>`-k, --api-key` — API key<br>`-u, --base-url` — Custom API endpoint<br>`--show` — Show current config<br>`--unset` — Clear configuration | Configure LLM |
+| `he config embedder` | `-p, --provider` — Provider<br>`-m, --model` — Model name<br>`-k, --api-key` — API key<br>`-u, --base-url` — Custom API endpoint<br>`--show` — Show current config<br>`--unset` — Clear configuration | Configure embedder |
+| `he config show` | — | View full configuration |
### Common Usage Examples
**Interactive initialization (recommended for first-time users):**
-```bash
-he config init
-# Follow prompts to enter model name, API key, etc.
-```
-**Non-interactive initialization (for scripts):**
```bash
-he config init -k sk-your-api-key
+he config init
+# Follow prompts to select provider, enter model name and API key
```
**View LLM configuration:**
-```bash
-he config llm --show
-```
-**Configure embedder separately (using different key):**
```bash
-he config embedder --api-key sk-embedder-key --model text-embedding-3-large
+he config llm --show
```
**Clear configuration (reset to defaults):**
+
```bash
he config llm --unset
he config embedder --unset
@@ -94,6 +117,65 @@ he config embedder --unset
---
+## Per-Platform Configuration Examples
+
+=== "OpenAI"
+
+ ```bash
+ # One-click setup
+ he config init -p openai -k sk-your-key
+
+ # Or switch model
+ he config llm -p openai -k sk-your-key -m gpt-4o
+ ```
+
+=== "Bailian (Alibaba Cloud)"
+
+ ```bash
+ # One-click setup (default qwen3.6-plus + text-embedding-v4)
+ he config init -p bailian -k sk-your-key
+
+ # Or switch model
+ he config llm -p bailian -k sk-your-key -m qwen-plus
+ ```
+
+=== "Local vLLM"
+
+ ```bash
+ # Start services first, then configure separately
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
+
+=== "Mixed Deployment"
+
+ LLM and Embedder can use different providers:
+
+ ```bash
+ # LLM via Bailian, Embedding via local vLLM
+ he config llm -p bailian -k sk-your-key
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+
+ # Or LLM via local, Embedding via Bailian
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+ he config embedder -p bailian -k sk-your-key
+ ```
+
+---
+
## Configuration File Details
Running `he config init` automatically creates `~/.he/config.toml`.
@@ -105,41 +187,68 @@ Running `he config init` automatically creates `~/.he/config.toml`.
### Configuration Format
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-your-api-key"
-base_url = "https://api.openai.com/v1"
-
-[embedder]
-model = "text-embedding-3-small"
-api_key = "" # Empty means inherit from llm.api_key
-base_url = "" # Empty means inherit from llm.base_url
-```
+=== "Cloud API — Bailian"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1"
+
+ [embedder]
+ provider = "bailian"
+ model = "text-embedding-v4"
+ api_key = ""
+ base_url = ""
+ ```
+
+=== "Local vLLM"
+
+ ```toml
+ [llm]
+ provider = "vllm"
+ model = "Qwen/Qwen3.5-9B"
+ api_key = "dummy"
+ base_url = "http://localhost:8000/v1"
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
+
+=== "Mixed Deployment"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = ""
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
### Configuration Options
| Section | Key | Description | Default |
|---------|-----|-------------|---------|
-| `[llm]` | `model` | LLM model name | `gpt-4o-mini` |
-| `[llm]` | `api_key` | OpenAI API key | Required |
-| `[llm]` | `base_url` | API base URL | `https://api.openai.com/v1` |
-| `[embedder]` | `model` | Embedding model name | `text-embedding-3-small` |
+| `[llm]` | `provider` | Provider preset | — |
+| `[llm]` | `model` | LLM model name | Depends on provider preset |
+| `[llm]` | `api_key` | API key | Required |
+| `[llm]` | `base_url` | API base URL | Depends on provider preset |
+| `[embedder]` | `provider` | Provider preset | — |
+| `[embedder]` | `model` | Embedding model name | Depends on provider preset |
| `[embedder]` | `api_key` | API key (empty inherits from llm) | — |
| `[embedder]` | `base_url` | API base URL (empty inherits from llm) | — |
-### Supported Models
-
-**LLM Models:**
-- `gpt-4o-mini` — Fast, cost-effective (default)
-- `gpt-4o` — High quality
-- `gpt-4-turbo` — Best quality
-- Other OpenAI-compatible models
-
-**Embedding Models:**
-- `text-embedding-3-small` — Fast, cost-effective (default)
-- `text-embedding-3-large` — Better quality
-- `text-embedding-ada-002` — Legacy model
+> See the [Provider System](../concepts/provider-system.md) compatibility table for default models per provider.
---
@@ -161,89 +270,52 @@ The following environment variables can be used as configuration fallback:
---
-## Advanced Configuration
-
-### Using OpenAI-Compatible APIs
-
-For Azure, local models, and other OpenAI-compatible APIs:
-
-```toml
-[llm]
-model = "your-model-name"
-api_key = "your-key"
-base_url = "https://your-api-endpoint.com/v1"
-
-[embedder]
-model = "your-embedding-model"
-```
-
-### Different Keys and Base URLs for LLM and Embedder
-
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-llm-key"
-base_url = "https://api.openai.com/v1"
-
-[embedder]
-model = "text-embedding-3-large"
-api_key = "sk-embedder-key"
-base_url = "https://your-embedder-provider.com/v1"
-```
-
----
-
## Troubleshooting
-### "API key not found"
-
-**Cause**: API key not configured
+=== "Cloud API Issues"
-**Solution**:
-
-```bash
-he config init -k YOUR_API_KEY
-```
+ **"API key not found"**
-### "Failed to connect to API"
+ ```bash
+ he config init -p bailian -k YOUR_API_KEY
+ ```
-**Cause**: Network issues or incorrect base_url
+ **"The model does not exist" / 404**
-**Solution**:
+ Check that the configured model name is available on the platform. See available models in [Provider System](../concepts/provider-system.md).
-```bash
-# Check configuration
-he config llm --show
+ **"Failed to connect to API"**
-# Reset base_url
-he config llm --base-url https://api.openai.com/v1
-```
+ ```bash
+ # Check configuration
+ he config llm --show
-### Permission Denied
+ # Reset base_url
+ he config llm --base-url https://dashscope.aliyuncs.com/compatible-mode/v1
+ ```
-**Solution**:
+=== "Local vLLM Issues"
-```bash
-chmod 600 ~/.he/config.toml
-```
+ **"Connection refused"**
-### Invalid TOML Format
+ vLLM service is not running or has stopped:
-**Validate configuration**:
+ ```bash
+ # Check if services are running
+ curl http://localhost:8000/v1/models
+ curl http://localhost:8001/v1/models
-```bash
-python3 -c "import tomllib; tomllib.load(open('$HOME/.he/config.toml', 'rb'))"
-```
+ # Restart services
+ ```
-**Common issues**:
+ **"CUDA out of memory"**
-- Missing quotes around strings
-- Trailing commas
-- Incorrect section headers
+ Lower `--gpu-memory-utilization` or use a smaller model / quantized version.
---
## See Also
- [`he config`](commands/config.md) — Detailed configuration command reference
+- [Provider System](../concepts/provider-system.md) — Full model compatibility list
- [Installation Guide](../getting-started/installation.md) — Initial setup steps
diff --git a/docs/en/cli/index.md b/docs/en/cli/index.md
index e360ce6..88ec668 100644
--- a/docs/en/cli/index.md
+++ b/docs/en/cli/index.md
@@ -81,9 +81,45 @@ flowchart TB
### 1. Configure API Key
-```bash
-he config init -k YOUR_OPENAI_API_KEY
-```
+=== "OpenAI"
+
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
+
+=== "Bailian (Alibaba Cloud)"
+
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
+
+=== "Local vLLM"
+
+ First install [vLLM](https://docs.vllm.ai/) and start both services:
+
+ ```bash
+ # Start LLM service (~8GB VRAM)
+ vllm serve Qwen/Qwen3.5-9B --port 8000 --api-key dummy
+
+ # Start Embedding service (~2GB VRAM)
+ vllm serve BAAI/bge-m3 --task embed --port 8001
+ ```
+
+ Then configure Hyper-Extract:
+
+ ```bash
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
+
+ > Full deployment options (quantization, Docker, etc.) see [Provider System](../concepts/provider-system.md).
### 2. Extract Knowledge
diff --git a/docs/en/concepts/provider-system.md b/docs/en/concepts/provider-system.md
new file mode 100644
index 0000000..2cc908e
--- /dev/null
+++ b/docs/en/concepts/provider-system.md
@@ -0,0 +1,119 @@
+# Provider System
+
+Hyper-Extract supports three ways to connect to LLMs: **OpenAI**, **Alibaba Bailian**, and **local vLLM**. All use the same `create_client()` interface — only the first line changes.
+
+---
+
+## Verified Model Compatibility
+
+### Cloud API
+
+| Platform | Model | `json_schema` Support | Compatible | Notes |
+|----------|-------|:---------------------:|:----------:|-------|
+| **OpenAI** | gpt-4o / gpt-4o-mini / gpt-5 | ✅ | ✅ | Native support, recommended |
+| **Alibaba Bailian** | qwen-plus / qwen-turbo / qwen3.6-plus / deepseek-r1 | ✅ | ✅ | Works out of the box |
+| **Alibaba Bailian** | qwen-max / deepseek-v3 | ❌ | ❌ | Only `json_object`; `json_schema` not supported |
+
+> **Bailian users**: Both qwen-max and deepseek-v3 do not support `json_schema`. If you hit `messages must contain the word 'json'` or get non-JSON output, switch to qwen-plus, qwen-turbo, or deepseek-r1. See [Issue #24](https://github.com/yifanfeng97/Hyper-Extract/issues/24).
+
+### Local Deployment
+
+| Service | Model | Quantization | VRAM | Verified |
+|---------|-------|-------------|------|----------|
+| LLM | **Qwen3.5-9B** | GPTQ-Marlin 4bit | ~8GB | ✅ AutoList / AutoGraph |
+| Embedding | **BAAI/bge-m3** | None | ~2GB | ✅ Semantic search |
+
+### Cloud Embedding
+
+| Platform | Model | Dimensions | Verified |
+|----------|-------|------------|----------|
+| **OpenAI** | text-embedding-3-small | 1536 | ✅ |
+| **Alibaba Bailian** | text-embedding-v4 | 1024 | ✅ |
+
+---
+
+## Quick Start by Platform
+
+```python
+from hyperextract import create_client
+
+# OpenAI
+llm, emb = create_client("openai", api_key="sk-xxx")
+
+# Bailian
+llm, emb = create_client("bailian", api_key="sk-xxx")
+
+# Local vLLM
+llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+)
+```
+
+Full configuration options: [Provider Configuration Guide](../python/guides/provider-configuration.md).
+
+---
+
+## vLLM Deployment
+
+### Start LLM Service
+
+```bash
+vllm serve /path/to/qwen3.5-9b-gptq-marlin \
+ --served-model-name Qwen/Qwen3.5-9B \
+ --trust-remote-code \
+ --quantization gptq_marlin \
+ --dtype bfloat16 \
+ --max-model-len 8192 \
+ --gpu-memory-utilization 0.90 \
+ --default-chat-template-kwargs '{"enable_thinking": false}' \
+ --port 8000 \
+ --api-key dummy
+```
+
+### Start Embedding Service
+
+```bash
+vllm serve BAAI/bge-m3 \
+ --task embed \
+ --dtype float16 \
+ --max-model-len 8192 \
+ --port 8001
+```
+
+### Docker
+
+```bash
+docker run --runtime nvidia --gpus all \
+ --ipc=host \
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
+ -p 8000:8000 \
+ vllm/vllm-openai:latest \
+ --model Qwen/Qwen3.5-9B \
+ --trust-remote-code
+```
+
+---
+
+## Recommendations
+
+**Use non-thinking models for local vLLM deployment.**
+
+When deploying locally, thinking models (e.g. Qwen3.5 thinking mode) output `<think>...</think>` tags, which conflict with constrained decoding's requirement for JSON from the first token. Disable thinking mode when deploying Qwen3.5-9B:
+
+```bash
+--default-chat-template-kwargs '{"enable_thinking": false}'
+```
+
+**DeepSeek-R1 via Bailian is verified to work** — Bailian filters out `<think>` tags on the backend, so AutoGraph and `with_structured_output` work correctly. If accessing DeepSeek-R1 through other channels, thinking output may still cause issues.
+
+**Prefer GPTQ-Marlin over AWQ** for quantization — AWQ has compatibility issues with vLLM 0.21.0.
+
+---
+
+## References
+
+- [vLLM Structured Outputs](https://docs.vllm.ai/en/latest/features/structured_outputs/)
+- [XGrammar Paper](https://arxiv.org/abs/2411.15100)
+- [Issue #21: Domestic Model Support](https://github.com/yifanfeng97/Hyper-Extract/issues/21)
diff --git a/docs/en/getting-started/cli-quickstart.md b/docs/en/getting-started/cli-quickstart.md
index fb87e67..077cc53 100644
--- a/docs/en/getting-started/cli-quickstart.md
+++ b/docs/en/getting-started/cli-quickstart.md
@@ -13,11 +13,33 @@ Get your first knowledge extraction running in 5 minutes using the terminal.
## Step 1: Configure API Key
-Run the following command to configure your API key:
+Choose your deployment method and run the corresponding configuration command:
-```bash
-he config init -k YOUR_OPENAI_API_KEY
-```
+=== "OpenAI"
+
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
+
+=== "Bailian (Alibaba Cloud)"
+
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
+
+=== "Local vLLM"
+
+ ```bash
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
This creates a configuration file at `~/.he/config.toml`. You only need to do this once.
diff --git a/docs/en/getting-started/python-quickstart.md b/docs/en/getting-started/python-quickstart.md
index 3b709b1..1153639 100644
--- a/docs/en/getting-started/python-quickstart.md
+++ b/docs/en/getting-started/python-quickstart.md
@@ -28,7 +28,9 @@ pip install hyperextract
---
-## Step 2: Configure API Key
+## Step 2: Configure Your Model
+
+**Option A — OpenAI (default)**
Create a `.env` file:
@@ -36,6 +38,10 @@ Create a `.env` file:
echo "OPENAI_API_KEY=your-api-key" > .env
```
+**Option B — Bailian or vLLM**
+
+See the [Provider Configuration Guide](../python/guides/provider-configuration.md) for platform-specific setup.
+
---
## Step 3: Create Your First Extraction Script
diff --git a/docs/en/index.md b/docs/en/index.md
index 3f1878f..a766e9d 100644
--- a/docs/en/index.md
+++ b/docs/en/index.md
@@ -16,8 +16,8 @@
# 1. Install CLI tool
uv tool install hyperextract
- # 2. Configure API Key
- he config init -k YOUR_OPENAI_API_KEY
+ # 2. Configure API Key (Bailian example)
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
# 3. Extract knowledge from a document
he parse tesla.md -t general/biography_graph -o ./output/ -l en
diff --git a/docs/en/python/guides/provider-configuration.md b/docs/en/python/guides/provider-configuration.md
new file mode 100644
index 0000000..0baada3
--- /dev/null
+++ b/docs/en/python/guides/provider-configuration.md
@@ -0,0 +1,98 @@
+# Provider Configuration Guide
+
+Configure Hyper-Extract to work with OpenAI, Bailian (Alibaba Cloud), or local vLLM deployments.
+
+---
+
+## Unified Example
+
+All three platforms run the **same extraction task** below. Only the client setup (first 3 lines) changes.
+
+### OpenAI
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("openai", api_key="sk-xxx")
+```
+
+### Alibaba Bailian
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("bailian", api_key="sk-xxx")
+# Or override model: create_client("bailian:qwen3.6-plus", api_key="sk-xxx")
+```
+
+### Local vLLM
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+)
+```
+
+### Extraction Task (same for all)
+
+```python
+graph = AutoGraph(
+ instruction="Extract people and their relationships",
+ llm_client=llm,
+ embedder=emb,
+ node_key_extractor=lambda n: n.name,
+ edge_key_extractor=lambda e: (e.source, e.target, e.type),
+ nodes_in_edge_extractor=lambda e: (e.source, e.target),
+)
+
+text = "Zhang San founded ByteDance. Li Si serves as CEO."
+graph.parse(text)
+print(f"Nodes: {len(graph.nodes)}, Edges: {len(graph.edges)}")
+```
+
+---
+
+## CLI Equivalents
+
+| Platform | Command |
+|----------|---------|
+| **OpenAI** | `he config init -p openai -k sk-xxx` |
+| **Bailian** | `he config init -p bailian -k sk-xxx` |
+| **vLLM** | `he config init` → select "local vLLM" |
+| **Mixed** (LLM=Bailian, Embedder=vLLM) | `he config llm -p bailian -k sk-xxx` + `he config embedder -p vllm -u http://localhost:8001/v1 -k dummy` |
+
+---
+
+## String Shorthand Format
+
+`create_client()` supports a compact string syntax for quick configuration:
+
+| Format | Example | Result |
+|--------|---------|--------|
+| `provider` | `"bailian"` | Uses preset defaults for LLM + embedder |
+| `provider:model` | `"bailian:qwen3.6-plus"` | Overrides LLM model, keeps preset embedder |
+| `provider:model@url` | `"vllm:Qwen3.5-9B@localhost:8000/v1"` | Full manual specification |
+
+---
+
+## Using Config File Instead
+
+If you prefer file-based configuration, run `he config init` once and then use `Template.create()` or `get_client()`:
+
+```python
+from hyperextract import get_client, AutoGraph
+
+llm, emb = get_client() # Reads ~/.he/config.toml
+graph = AutoGraph(..., llm_client=llm, embedder=emb)
+```
+
+---
+
+## See Also
+
+- [Provider System & Model Compatibility](../../concepts/provider-system.md) — Full compatibility table and vLLM deployment guide
+- [CLI Configuration Reference](../../cli/configuration.md) — Complete `he config` command reference
diff --git a/docs/en/python/guides/working-with-autotypes.md b/docs/en/python/guides/working-with-autotypes.md
index 86baa02..d1d0d36 100644
--- a/docs/en/python/guides/working-with-autotypes.md
+++ b/docs/en/python/guides/working-with-autotypes.md
@@ -65,6 +65,8 @@ class Collaboration(BaseModel):
year: int = Field(description="Year of collaboration")
# Step 2: Configure LLM clients
+# You can use any OpenAI-compatible endpoint by setting base_url
+# Or use create_client() for simplified configuration
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
embedder = OpenAIEmbeddings(model="text-embedding-3-small")
diff --git a/docs/zh/cli/commands/config.md b/docs/zh/cli/commands/config.md
index f885c91..5c3a58c 100644
--- a/docs/zh/cli/commands/config.md
+++ b/docs/zh/cli/commands/config.md
@@ -14,7 +14,7 @@ he config [COMMAND] [OPTIONS]
| 命令 | 描述 |
|---------|-------------|
-| `init` | 初始化配置(懒人模式默认使用 `gpt-4o-mini` + `text-embedding-3-small`) |
+| `init` | 初始化配置(使用 provider preset 自动填充模型和地址) |
| `show` | 显示当前配置 |
| `llm` | 配置 LLM 设置 |
| `embedder` | 配置嵌入模型设置 |
@@ -23,10 +23,10 @@ he config [COMMAND] [OPTIONS]
## he config init
-初始化配置。这是**懒人一键配置** —— 只要传入 `-k`,就会自动使用内置默认值,无需任何交互:
+初始化配置。这是**懒人一键配置** —— 只要传入 `-p` 和 `-k`,就会自动使用内置 preset 默认值,无需任何交互:
-- **LLM**: `gpt-4o-mini`
-- **嵌入模型**: `text-embedding-3-small`
+- **OpenAI preset**: `gpt-4o-mini` + `text-embedding-3-small`
+- **百炼 preset**: `qwen3.6-plus` + `text-embedding-v4`
```bash
he config init [OPTIONS]
@@ -36,6 +36,7 @@ he config init [OPTIONS]
| 选项 | 简写 | 描述 |
|--------|-------|-------------|
+| `--provider` | `-p` | 提供商 preset (`openai` / `bailian` / `vllm`) |
| `--api-key` | `-k` | LLM 和嵌入模型的 API 密钥 |
| `--base-url` | `-u` | 自定义 API 基础 URL(可选) |
@@ -44,15 +45,19 @@ he config init [OPTIONS]
#### 懒人一键配置(推荐)
```bash
-he config init -k sk-your-api-key-here
+# OpenAI
+he config init -p openai -k sk-your-api-key-here
+
+# 百炼(阿里云)
+he config init -p bailian -k sk-your-api-key-here
```
-执行后会自动保存默认模型 `gpt-4o-mini` 和 `text-embedding-3-small`。
+执行后会自动保存对应 preset 的默认模型和 API 地址。
#### 使用自定义基础 URL
```bash
-he config init -k sk-your-key -u https://api.openai.com/v1
+he config init -p openai -k sk-your-key -u https://api.openai.com/v1
```
#### 交互式初始化
@@ -75,14 +80,14 @@ he config show
**输出示例:**
```
-┌─────────────────────────────────────────────────────────┐
-│ Hyper-Extract Configuration │
-├──────────┬─────────────────────┬─────────────┬──────────┤
-│ Service │ Model │ API Key │ Base URL │
-├──────────┼─────────────────────┼─────────────┼──────────┤
-│ LLM │ gpt-4o-mini │ sk-xxxxx... │ (default)│
-│ Embedder │ text-embedding-3... │ sk-xxxxx... │ (default)│
-└──────────┴─────────────────────┴─────────────┴──────────┘
+┌──────────────────────────────────────────────────────────────────┐
+│ Hyper-Extract Configuration │
+├──────────┬──────────┬─────────────────────┬──────────┬───────────┤
+│ Service │ Provider │ Model │ API Key │ Base URL │
+├──────────┼──────────┼─────────────────────┼──────────┼───────────┤
+│ LLM │ bailian │ qwen3.6-plus │ sk-xx... │ dashsc... │
+│ Embedder │ bailian │ text-embedding-v4 │ sk-xx... │ dashsc... │
+└──────────┴──────────┴─────────────────────┴──────────┴───────────┘
```
---
@@ -111,11 +116,17 @@ he config llm [OPTIONS]
# 查看 LLM 配置
he config llm --show
-# 修改 LLM 模型
-he config llm --model gpt-4o
+# 修改 LLM 模型(OpenAI)
+he config llm -p openai --model gpt-4o
+
+# 修改 LLM 模型(百炼)
+he config llm -p bailian --model qwen-plus
-# 修改 LLM API 密钥和接口地址
-he config llm --api-key sk-your-key --base-url https://api.openai.com/v1
+# 配置本地 vLLM
+he config llm -p vllm \
+ --api-key dummy \
+ --base-url http://localhost:8000/v1 \
+ --model Qwen/Qwen3.5-9B
# 重置 LLM 配置
he config llm --unset
@@ -147,8 +158,14 @@ he config embedder [OPTIONS]
# 查看嵌入模型配置
he config embedder --show
-# 使用更大的嵌入模型
-he config embedder --model text-embedding-3-large
+# 使用 OpenAI 更大的嵌入模型
+he config embedder -p openai --model text-embedding-3-large
+
+# 配置本地 vLLM 嵌入
+he config embedder -p vllm \
+ --api-key dummy \
+ --base-url http://localhost:8001/v1 \
+ --model BAAI/bge-m3
# 重置嵌入模型配置
he config embedder --unset
@@ -165,17 +182,37 @@ he config embedder --unset
### 配置示例
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-your-api-key"
-base_url = "https://api.openai.com/v1"
-
-[embedder]
-model = "text-embedding-3-small"
-api_key = "" # 空字符串表示继承 llm.api_key
-base_url = "" # 空字符串表示继承 llm.base_url
-```
+=== "百炼 (阿里云)"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1"
+
+ [embedder]
+ provider = "bailian"
+ model = "text-embedding-v4"
+ api_key = ""
+ base_url = ""
+ ```
+
+=== "本地 vLLM"
+
+ ```toml
+ [llm]
+ provider = "vllm"
+ model = "Qwen/Qwen3.5-9B"
+ api_key = "dummy"
+ base_url = "http://localhost:8000/v1"
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
## 环境变量
diff --git a/docs/zh/cli/configuration.md b/docs/zh/cli/configuration.md
index 5dfb22a..71a2b77 100644
--- a/docs/zh/cli/configuration.md
+++ b/docs/zh/cli/configuration.md
@@ -1,55 +1,85 @@
# CLI 配置参考
-Hyper-Extract CLI 的配置指南,从快速入门到高级设置。
+Hyper-Extract CLI 的配置指南,支持 **OpenAI**、**阿里云百炼** 和 **本地 vLLM** 三种部署方式。
---
## 快速开始
-只需 3 步完成配置。
+选择你的部署方式,按步骤完成配置。
-### 1. 初始化配置
+=== "OpenAI"
-```bash
-he config init -k YOUR_API_KEY
-```
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
-这会创建 `~/.he/config.toml` 配置文件,使用以下默认值:
-- **LLM**: `gpt-4o-mini`
-- **嵌入模型**: `text-embedding-3-small`
+ 这会创建 `~/.he/config.toml`,使用 OpenAI 预设:
+ - **LLM**: `gpt-4o-mini`
+ - **嵌入模型**: `text-embedding-3-small`
-### 2. 验证配置
+=== "百炼 (阿里云)"
-```bash
-he config show
-```
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
-看到类似输出即配置成功:
+ 这会创建 `~/.he/config.toml`,使用百炼预设:
+ - **LLM**: `qwen3.6-plus`
+ - **嵌入模型**: `text-embedding-v4`
-```
-┌─────────────────────────────────────────────────────────┐
-│ Hyper-Extract Configuration │
-├──────────┬─────────────────────┬─────────────┬──────────┤
-│ Service │ Model │ API Key │ Base URL │
-├──────────┼─────────────────────┼─────────────┼──────────┤
-│ LLM │ gpt-4o-mini │ sk-xxxxx... │ (default)│
-│ Embedder │ text-embedding-3... │ sk-xxxxx... │ (default)│
-└──────────┴─────────────────────┴─────────────┴──────────┘
-```
+=== "本地 vLLM"
-### 3. 修改配置(如需)
+ 需先启动 LLM 和 Embedding 两个服务,然后分别配置:
-```bash
-# 更换 LLM 模型
-he config llm --model gpt-4o
+ ```bash
+ # LLM 服务
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
-# 使用不同的嵌入模型
-he config embedder --model text-embedding-3-large
+ # Embedding 服务
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
-# 配置自定义 API 端点
-he config llm --base-url https://your-api-endpoint.com/v1
+---
+
+## 验证配置
+
+```bash
+he config show
```
+=== "云 API (OpenAI / 百炼)"
+
+ ```
+ ┌─────────────────────────────────────────────────────────────┐
+ │ Hyper-Extract Configuration │
+ ├──────────┬──────────┬─────────────────────┬────────────┬────┤
+ │ Service │ Provider │ Model │ API Key │ ...│
+ ├──────────┼──────────┼─────────────────────┼────────────┼────┤
+ │ LLM │ bailian │ qwen3.6-plus │ sk-xxxx... │ ...│
+ │ Embedder │ bailian │ text-embedding-v4 │ sk-xxxx... │ ...│
+ └──────────┴──────────┴─────────────────────┴────────────┴────┘
+ ```
+
+=== "本地 vLLM"
+
+ ```
+ ┌──────────────────────────────────────────────────────────────────┐
+ │ Hyper-Extract Configuration │
+ ├──────────┬──────────┬─────────────────────┬──────────┬───────────┤
+ │ Service │ Provider │ Model │ API Key │ Base URL │
+ ├──────────┼──────────┼─────────────────────┼──────────┼───────────┤
+ │ LLM │ vllm │ Qwen/Qwen3.5-9B │ dummy... │ localhost…│
+ │ Embedder │ vllm │ BAAI/bge-m3 │ dummy... │ localhost…│
+ └──────────┴──────────┴─────────────────────┴──────────┴───────────┘
+ ```
+
---
## CLI 命令参考
@@ -58,35 +88,28 @@ he config llm --base-url https://your-api-endpoint.com/v1
| 命令 | 常用参数 | 描述 |
|------|---------|------|
-| `he config init` | `-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址 | 初始化配置(首次使用) |
-| `he config llm` | `-m, --model` — 模型名称<br>`-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址<br>`--show` — 查看当前配置<br>`--unset` — 清除配置 | 配置 LLM |
-| `he config embedder` | `-m, --model` — 模型名称<br>`-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址<br>`--show` — 查看当前配置<br>`--unset` — 清除配置 | 配置嵌入模型 |
+| `he config init` | `-p, --provider` — 提供商 preset<br>`-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址 | 初始化配置(首次使用) |
+| `he config llm` | `-p, --provider` — 提供商<br>`-m, --model` — 模型名称<br>`-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址<br>`--show` — 查看当前配置<br>`--unset` — 清除配置 | 配置 LLM |
+| `he config embedder` | `-p, --provider` — 提供商<br>`-m, --model` — 模型名称<br>`-k, --api-key` — API 密钥<br>`-u, --base-url` — 自定义 API 地址<br>`--show` — 查看当前配置<br>`--unset` — 清除配置 | 配置嵌入模型 |
| `he config show` | — | 查看完整配置 |
### 常见用法示例
**交互式初始化(推荐首次使用):**
-```bash
-he config init
-# 按提示输入模型名、API 密钥等信息
-```
-**非交互式初始化(脚本中使用):**
```bash
-he config init -k sk-your-api-key
+he config init
+# 按提示选择提供商、输入模型名和 API 密钥
```
**查看 LLM 配置:**
-```bash
-he config llm --show
-```
-**单独配置嵌入模型(使用不同密钥):**
```bash
-he config embedder --api-key sk-embedder-key --model text-embedding-3-large
+he config llm --show
```
**清除配置(重置为默认值):**
+
```bash
he config llm --unset
he config embedder --unset
@@ -94,9 +117,68 @@ he config embedder --unset
---
+## 分平台配置示例
+
+=== "OpenAI"
+
+ ```bash
+ # 一键配置
+ he config init -p openai -k sk-your-key
+
+ # 或换模型
+ he config llm -p openai -k sk-your-key -m gpt-4o
+ ```
+
+=== "百炼 (阿里云)"
+
+ ```bash
+ # 一键配置(默认 qwen3.6-plus + text-embedding-v4)
+ he config init -p bailian -k sk-your-key
+
+ # 或换模型
+ he config llm -p bailian -k sk-your-key -m qwen-plus
+ ```
+
+=== "本地 vLLM"
+
+ ```bash
+ # 需先启动服务,然后分别配置
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
+
+=== "混合部署"
+
+ LLM 和 Embedder 可以使用不同提供商:
+
+ ```bash
+ # LLM 用百炼,Embedding 用本地 vLLM
+ he config llm -p bailian -k sk-your-key
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+
+ # 或 LLM 用本地,Embedding 用百炼
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+ he config embedder -p bailian -k sk-your-key
+ ```
+
+---
+
## 配置文件详解
-当你运行 `he config init` 时,会自动创建 `~/.he/config.toml` 文件。
+运行 `he config init` 时会自动创建 `~/.he/config.toml`。
### 文件位置
@@ -105,41 +187,68 @@ he config embedder --unset
### 配置格式
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-your-api-key"
-base_url = "https://api.openai.com/v1"
-
-[embedder]
-model = "text-embedding-3-small"
-api_key = "" # 空字符串表示继承 llm.api_key
-base_url = "" # 空字符串表示继承 llm.base_url
-```
+=== "云 API — 百炼"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = "https://dashscope.aliyuncs.com/compatible-mode/v1"
+
+ [embedder]
+ provider = "bailian"
+ model = "text-embedding-v4"
+ api_key = ""
+ base_url = ""
+ ```
+
+=== "本地 vLLM"
+
+ ```toml
+ [llm]
+ provider = "vllm"
+ model = "Qwen/Qwen3.5-9B"
+ api_key = "dummy"
+ base_url = "http://localhost:8000/v1"
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
+
+=== "混合部署"
+
+ ```toml
+ [llm]
+ provider = "bailian"
+ model = "qwen3.6-plus"
+ api_key = "sk-your-api-key"
+ base_url = ""
+
+ [embedder]
+ provider = "vllm"
+ model = "BAAI/bge-m3"
+ api_key = "dummy"
+ base_url = "http://localhost:8001/v1"
+ ```
### 配置项说明
| 节 | 键 | 描述 | 默认值 |
|---|-----|------|--------|
-| `[llm]` | `model` | LLM 模型名称 | `gpt-4o-mini` |
-| `[llm]` | `api_key` | OpenAI API 密钥 | 必填 |
-| `[llm]` | `base_url` | API 基础 URL | `https://api.openai.com/v1` |
-| `[embedder]` | `model` | 嵌入模型名称 | `text-embedding-3-small` |
+| `[llm]` | `provider` | 提供商 preset | — |
+| `[llm]` | `model` | LLM 模型名称 | 取决于 provider preset |
+| `[llm]` | `api_key` | API 密钥 | 必填 |
+| `[llm]` | `base_url` | API 基础 URL | 取决于 provider preset |
+| `[embedder]` | `provider` | 提供商 preset | — |
+| `[embedder]` | `model` | 嵌入模型名称 | 取决于 provider preset |
| `[embedder]` | `api_key` | API 密钥(留空继承 llm) | — |
| `[embedder]` | `base_url` | API 基础 URL(留空继承 llm) | — |
-### 支持的模型
-
-**LLM 模型:**
-- `gpt-4o-mini` — 快速、成本效益高(默认)
-- `gpt-4o` — 高质量
-- `gpt-4-turbo` — 最佳质量
-- 其他 OpenAI 兼容模型
-
-**嵌入模型:**
-- `text-embedding-3-small` — 快速、成本效益高(默认)
-- `text-embedding-3-large` — 更好的质量
-- `text-embedding-ada-002` — 传统模型
+> 不同 provider 的默认模型见 [Provider 系统](../concepts/provider-system.md) 兼容性表格。
---
@@ -161,89 +270,52 @@ base_url = "" # 空字符串表示继承 llm.base_url
---
-## 高级配置
-
-### 使用 OpenAI 兼容 API
-
-对于 Azure、本地模型等 OpenAI 兼容 API:
-
-```toml
-[llm]
-model = "your-model-name"
-api_key = "your-key"
-base_url = "https://your-api-endpoint.com/v1"
-
-[embedder]
-model = "your-embedding-model"
-```
-
-### LLM 和嵌入器使用不同密钥和不同 base_url
-
-```toml
-[llm]
-model = "gpt-4o-mini"
-api_key = "sk-llm-key"
-base_url = "https://api.openai.com/v1"
-
-[embedder]
-model = "text-embedding-3-large"
-api_key = "sk-embedder-key"
-base_url = "https://your-embedder-provider.com/v1"
-```
-
----
-
## 故障排除
-### "API key not found"
-
-**原因**: 未配置 API 密钥
+=== "云 API 问题"
-**解决**:
-
-```bash
-he config init -k YOUR_API_KEY
-```
+ **"API key not found"**
-### "Failed to connect to API"
+ ```bash
+ he config init -p bailian -k YOUR_API_KEY
+ ```
-**原因**: 网络问题或 base_url 配置错误
+ **"The model does not exist" / 404**
-**解决**:
+ 检查配置的模型名是否在对应平台可用。百炼可用模型见 [Provider 系统](../concepts/provider-system.md)。
-```bash
-# 检查配置
-he config llm --show
+ **"Failed to connect to API"**
-# 重新设置 base_url
-he config llm --base-url https://api.openai.com/v1
-```
+ ```bash
+ # 检查配置
+ he config llm --show
-### 权限被拒绝
+ # 重新设置 base_url
+ he config llm --base-url https://dashscope.aliyuncs.com/compatible-mode/v1
+ ```
-**解决**:
+=== "本地 vLLM 问题"
-```bash
-chmod 600 ~/.he/config.toml
-```
+ **"Connection refused"**
-### 无效的 TOML 格式
+ vLLM 服务未启动或服务已停止:
-**验证配置**:
+ ```bash
+ # 检查服务是否运行
+ curl http://localhost:8000/v1/models
+ curl http://localhost:8001/v1/models
-```bash
-python3 -c "import tomllib; tomllib.load(open('$HOME/.he/config.toml', 'rb'))"
-```
+ # 重新启动服务
+ ```
-**常见问题**:
+ **"CUDA out of memory"**
-- 字符串缺少引号
-- 多余的逗号
-- 错误的节标题(如使用中文标点)
+ 降低 `--gpu-memory-utilization` 或换用更小的模型/量化版本。
---
## 另请参见
- [`he config`](commands/config.md) — 详细的配置命令说明
+- [Provider 系统](../concepts/provider-system.md) — 完整模型兼容性列表
- [安装指南](../getting-started/installation.md) — 初始安装步骤
diff --git a/docs/zh/cli/index.md b/docs/zh/cli/index.md
index 320b00b..ec320f8 100644
--- a/docs/zh/cli/index.md
+++ b/docs/zh/cli/index.md
@@ -81,9 +81,45 @@ flowchart TB
### 1. 配置 API 密钥
-```bash
-he config init -k YOUR_OPENAI_API_KEY
-```
+=== "OpenAI"
+
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
+
+=== "百炼 (阿里云)"
+
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
+
+=== "本地 vLLM"
+
+ 需先安装 [vLLM](https://docs.vllm.ai/) 并分别启动 LLM 和 Embedding 服务:
+
+ ```bash
+ # 启动 LLM 服务(约需 8GB 显存)
+ vllm serve Qwen/Qwen3.5-9B --port 8000 --api-key dummy
+
+ # 启动 Embedding 服务(约需 2GB 显存)
+ vllm serve BAAI/bge-m3 --task embed --port 8001
+ ```
+
+ 然后配置 Hyper-Extract:
+
+ ```bash
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
+
+ > 完整部署参数(量化、Docker 等)见 [Provider 系统](../concepts/provider-system.md)。
### 2. 提取知识
diff --git a/docs/zh/concepts/provider-system.md b/docs/zh/concepts/provider-system.md
new file mode 100644
index 0000000..8eef61b
--- /dev/null
+++ b/docs/zh/concepts/provider-system.md
@@ -0,0 +1,119 @@
+# Provider 系统
+
+Hyper-Extract 支持三种接入方式:**OpenAI**、**阿里云百炼**、**本地 vLLM**。统一使用 `create_client()` 接口,仅需修改第一行。
+
+---
+
+## 已验证模型兼容性
+
+### 云端 API
+
+| 平台 | 模型 | `json_schema` 支持 | 兼容性 | 备注 |
+|------|------|:------------------:|:------:|------|
+| **OpenAI** | gpt-4o / gpt-4o-mini / gpt-5 | ✅ | ✅ | 官方原生支持,推荐 |
+| **阿里云百炼** | qwen-plus / qwen-turbo / qwen3.6-plus / deepseek-r1 | ✅ | ✅ | 直接使用,无需修改 |
+| **阿里云百炼** | qwen-max / deepseek-v3 | ❌ | ❌ | 仅支持 `json_object`,不支持 `json_schema` |
+
+> **百炼用户注意**:qwen-max 和 deepseek-v3 均不支持 `json_schema`,若遇到 `messages must contain the word 'json'` 错误或不返回 JSON,请切换到 qwen-plus、qwen-turbo 或 deepseek-r1。详见 [Issue #24](https://github.com/yifanfeng97/Hyper-Extract/issues/24)。
+
+### 本地部署
+
+| 服务 | 模型 | 量化方式 | 显存占用 | 验证状态 |
+|------|------|---------|---------|---------|
+| LLM | **Qwen3.5-9B** | GPTQ-Marlin 4bit | ~8GB | ✅ AutoList / AutoGraph |
+| Embedding | **BAAI/bge-m3** | 无 | ~2GB | ✅ 语义搜索 |
+
+### 云端 Embedding
+
+| 平台 | 模型 | 维度 | 验证状态 |
+|------|------|------|---------|
+| **OpenAI** | text-embedding-3-small | 1536 | ✅ |
+| **阿里云百炼** | text-embedding-v4 | 1024 | ✅ |
+
+---
+
+## 分平台快速启动
+
+```python
+from hyperextract import create_client
+
+# OpenAI
+llm, emb = create_client("openai", api_key="sk-xxx")
+
+# 百炼
+llm, emb = create_client("bailian", api_key="sk-xxx")
+
+# 本地 vLLM
+llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+)
+```
+
+完整配置选项见 [Provider 配置指南](../python/guides/provider-configuration.md)。
+
+---
+
+## vLLM 部署
+
+### 启动 LLM 服务
+
+```bash
+vllm serve /path/to/qwen3.5-9b-gptq-marlin \
+ --served-model-name Qwen/Qwen3.5-9B \
+ --trust-remote-code \
+ --quantization gptq_marlin \
+ --dtype bfloat16 \
+ --max-model-len 8192 \
+ --gpu-memory-utilization 0.90 \
+ --default-chat-template-kwargs '{"enable_thinking": false}' \
+ --port 8000 \
+ --api-key dummy
+```
+
+### 启动 Embedding 服务
+
+```bash
+vllm serve BAAI/bge-m3 \
+ --task embed \
+ --dtype float16 \
+ --max-model-len 8192 \
+ --port 8001
+```
+
+### Docker 部署
+
+```bash
+docker run --runtime nvidia --gpus all \
+ --ipc=host \
+ -v ~/.cache/huggingface:/root/.cache/huggingface \
+ -p 8000:8000 \
+ vllm/vllm-openai:latest \
+ --model Qwen/Qwen3.5-9B \
+ --trust-remote-code
+```
+
+---
+
+## 使用建议
+
+**本地 vLLM 部署推荐使用非 Thinking 模型。**
+
+本地部署时,Thinking 模型(如 Qwen3.5 thinking 模式)会输出 `<think>...</think>` 标签,与约束解码要求的从首个 token 开始 JSON 合规冲突。部署 Qwen3.5-9B 时请关闭 thinking 模式:
+
+```bash
+--default-chat-template-kwargs '{"enable_thinking": false}'
+```
+
+**百炼渠道的 DeepSeek-R1 已验证可用** — 百炼平台在后端过滤了 `<think>` 标签输出,AutoGraph 和 `with_structured_output` 均可正常工作。若通过其他渠道接入 DeepSeek-R1,仍可能受 thinking 输出影响。
+
+**量化模型推荐 GPTQ-Marlin**,AWQ 在 vLLM 0.21.0 中存在兼容性问题。
+
+---
+
+## 参考
+
+- [vLLM Structured Outputs 文档](https://docs.vllm.ai/en/latest/features/structured_outputs/)
+- [XGrammar 论文](https://arxiv.org/abs/2411.15100)
+- [Issue #21: 国内模型支持讨论](https://github.com/yifanfeng97/Hyper-Extract/issues/21)
diff --git a/docs/zh/getting-started/cli-quickstart.md b/docs/zh/getting-started/cli-quickstart.md
index a7f3977..6c53952 100644
--- a/docs/zh/getting-started/cli-quickstart.md
+++ b/docs/zh/getting-started/cli-quickstart.md
@@ -13,11 +13,33 @@
## 第 1 步:配置 API 密钥
-运行以下命令配置 API 密钥:
+选择你的部署方式并运行对应的配置命令:
-```bash
-he config init -k YOUR_OPENAI_API_KEY
-```
+=== "OpenAI"
+
+ ```bash
+ he config init -p openai -k YOUR_OPENAI_API_KEY
+ ```
+
+=== "百炼 (阿里云)"
+
+ ```bash
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
+ ```
+
+=== "本地 vLLM"
+
+ ```bash
+ he config llm -p vllm \
+ -u http://localhost:8000/v1 \
+ -k dummy \
+ -m Qwen/Qwen3.5-9B
+
+ he config embedder -p vllm \
+ -u http://localhost:8001/v1 \
+ -k dummy \
+ -m BAAI/bge-m3
+ ```
这会在 `~/.he/config.toml` 创建配置文件,只需配置一次。
diff --git a/docs/zh/getting-started/python-quickstart.md b/docs/zh/getting-started/python-quickstart.md
index efa1d4c..b8e064f 100644
--- a/docs/zh/getting-started/python-quickstart.md
+++ b/docs/zh/getting-started/python-quickstart.md
@@ -28,7 +28,9 @@ pip install hyperextract
---
-## 第 2 步:配置 API 密钥
+## 第 2 步:配置模型
+
+**方式 A — OpenAI(默认)**
创建 `.env` 文件:
@@ -36,6 +38,10 @@ pip install hyperextract
echo "OPENAI_API_KEY=your-api-key" > .env
```
+**方式 B — 百炼或 vLLM**
+
+平台特定配置见 [Provider 配置指南](../python/guides/provider-configuration.md)。
+
---
## 第 3 步:创建首个提取脚本
diff --git a/docs/zh/index.md b/docs/zh/index.md
index ceceb88..c48f445 100644
--- a/docs/zh/index.md
+++ b/docs/zh/index.md
@@ -16,8 +16,8 @@
# 1. 安装 CLI 工具
uv tool install hyperextract
- # 2. 配置 API 密钥
- he config init -k YOUR_OPENAI_API_KEY
+ # 2. 配置 API 密钥(以百炼为例)
+ he config init -p bailian -k YOUR_BAILIAN_API_KEY
# 3. 从文档中提取知识
he parse sushi.md -t general/biography_graph -o ./output/ -l zh
diff --git a/docs/zh/python/guides/provider-configuration.md b/docs/zh/python/guides/provider-configuration.md
new file mode 100644
index 0000000..7cc753c
--- /dev/null
+++ b/docs/zh/python/guides/provider-configuration.md
@@ -0,0 +1,98 @@
+# Provider 配置指南
+
+将 Hyper-Extract 配置为使用 OpenAI、阿里云百炼或本地 vLLM 部署。
+
+---
+
+## 统一示例
+
+以下三个平台运行**相同的提取任务**,仅客户端配置(前 3 行)不同。
+
+### OpenAI
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("openai", api_key="sk-xxx")
+```
+
+### 阿里云百炼
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("bailian", api_key="sk-xxx")
+# 或换模型: create_client("bailian:qwen3.6-plus", api_key="sk-xxx")
+```
+
+### 本地 vLLM
+
+```python
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+)
+```
+
+### 提取任务(三个平台共用)
+
+```python
+graph = AutoGraph(
+ instruction="提取文本中的人物及其关系",
+ llm_client=llm,
+ embedder=emb,
+ node_key_extractor=lambda n: n.name,
+ edge_key_extractor=lambda e: (e.source, e.target, e.type),
+ nodes_in_edge_extractor=lambda e: (e.source, e.target),
+)
+
+text = "张三创立了字节跳动,李四担任 CEO。"
+graph.parse(text)
+print(f"节点: {len(graph.nodes)}, 关系: {len(graph.edges)}")
+```
+
+---
+
+## CLI 等价配置
+
+| 平台 | 命令 |
+|------|------|
+| **OpenAI** | `he config init -p openai -k sk-xxx` |
+| **百炼** | `he config init -p bailian -k sk-xxx` |
+| **vLLM** | `he config init` → 选择「本地 vLLM」 |
+| **混合部署**(LLM=百炼 + Embedding=本地) | `he config llm -p bailian -k sk-xxx` + `he config embedder -p vllm -u http://localhost:8001/v1 -k dummy` |
+
+---
+
+## 字符串简写格式
+
+`create_client()` 支持紧凑的字符串语法,快速配置:
+
+| 格式 | 示例 | 结果 |
+|------|------|------|
+| `provider` | `"bailian"` | 使用预设的 LLM + Embedding 默认值 |
+| `provider:model` | `"bailian:qwen3.6-plus"` | 覆盖 LLM 模型,保留预设 Embedding |
+| `provider:model@url` | `"vllm:Qwen3.5-9B@localhost:8000/v1"` | 完整手动指定 |
+
+---
+
+## 使用配置文件
+
+如果你偏好基于文件的配置,运行一次 `he config init`,然后使用 `Template.create()` 或 `get_client()`:
+
+```python
+from hyperextract import get_client, AutoGraph
+
+llm, emb = get_client() # 读取 ~/.he/config.toml
+graph = AutoGraph(..., llm_client=llm, embedder=emb)
+```
+
+---
+
+## 另请参阅
+
+- [Provider 系统与本地模型支持](../../concepts/provider-system.md) — 完整兼容性表格和 vLLM 部署指南
+- [CLI 配置参考](../../cli/configuration.md) — 完整的 `he config` 命令参考
diff --git a/examples/README.md b/examples/README.md
index c2916cd..2581add 100644
--- a/examples/README.md
+++ b/examples/README.md
@@ -4,17 +4,30 @@ Examples directory with bilingual demos (English & Chinese).
## Quick Start
-### English Demos
+### Provider Setup (choose one)
```bash
cd examples
-python en/autotypes/graph_demo.py
+
+# OpenAI
+python providers/openai_demo.py
+
+# Bailian (Alibaba Cloud)
+python providers/bailian_demo.py
+
+# Local vLLM
+python providers/vllm_demo.py
```
-### 中文示例
+### Full Auto-Type Demos
```bash
cd examples
+
+# English
+python en/autotypes/graph_demo.py
+
+# Chinese
python zh/autotypes/graph_demo.py
```
@@ -23,6 +36,10 @@ python zh/autotypes/graph_demo.py
```
examples/
├── README.md
+├── providers/ # Provider configuration demos
+│ ├── openai_demo.py # OpenAI setup
+│ ├── bailian_demo.py # Bailian (Alibaba Cloud) setup
+│ └── vllm_demo.py # Local vLLM setup
├── en/ # English demos
│ ├── tesla.md # Tesla biography data
│ ├── tesla_question.md # Query questions
diff --git a/examples/README_ZH.md b/examples/README_ZH.md
index 8e8e3fa..718cd5c 100644
--- a/examples/README_ZH.md
+++ b/examples/README_ZH.md
@@ -4,17 +4,30 @@
## 快速开始
-### 英文演示
+### Provider 配置(三选一)
```bash
cd examples
-python en/autotypes/graph_demo.py
+
+# OpenAI
+python providers/openai_demo.py
+
+# 百炼(阿里云)
+python providers/bailian_demo.py
+
+# 本地 vLLM
+python providers/vllm_demo.py
```
-### 中文示例
+### 完整 Auto-Type 演示
```bash
cd examples
+
+# 英文
+python en/autotypes/graph_demo.py
+
+# 中文
python zh/autotypes/graph_demo.py
```
@@ -24,6 +37,10 @@ python zh/autotypes/graph_demo.py
examples/
├── README.md
├── README_ZH.md # 本文件
+├── providers/ # Provider 配置示例
+│ ├── openai_demo.py # OpenAI 配置
+│ ├── bailian_demo.py # 百炼(阿里云)配置
+│ └── vllm_demo.py # 本地 vLLM 配置
├── en/ # 英文演示
│ ├── tesla.md # 特斯拉传记数据
│ ├── tesla_question.md # 查询问题
diff --git a/examples/providers/bailian_demo.py b/examples/providers/bailian_demo.py
new file mode 100644
index 0000000..562e6f2
--- /dev/null
+++ b/examples/providers/bailian_demo.py
@@ -0,0 +1,28 @@
+"""
+Bailian (Alibaba Cloud) Provider Demo
+
+Extract entities and relationships using Alibaba Cloud Bailian API.
+
+Usage:
+ python examples/providers/bailian_demo.py
+"""
+
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("bailian", api_key="sk-xxx")
+
+graph = AutoGraph(
+ instruction="Extract people and their relationships",
+ llm_client=llm,
+ embedder=emb,
+ node_key_extractor=lambda n: n.name,
+ edge_key_extractor=lambda e: (e.source, e.target, e.type),
+ nodes_in_edge_extractor=lambda e: (e.source, e.target),
+)
+
+text = "张三创立了字节跳动,李四担任 CEO。"
+graph.parse(text)
+
+print(f"节点: {len(graph.nodes)}, 关系: {len(graph.edges)}")
+for n in graph.nodes:
+ print(f" - {n.name} ({n.type})")
diff --git a/examples/providers/openai_demo.py b/examples/providers/openai_demo.py
new file mode 100644
index 0000000..8733299
--- /dev/null
+++ b/examples/providers/openai_demo.py
@@ -0,0 +1,29 @@
+"""
+OpenAI Provider Demo
+
+Extract entities and relationships using OpenAI API.
+
+Usage:
+ export OPENAI_API_KEY=sk-xxx
+ python examples/providers/openai_demo.py
+"""
+
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client("openai", api_key="sk-xxx")
+
+graph = AutoGraph(
+ instruction="Extract people and their relationships",
+ llm_client=llm,
+ embedder=emb,
+ node_key_extractor=lambda n: n.name,
+ edge_key_extractor=lambda e: (e.source, e.target, e.type),
+ nodes_in_edge_extractor=lambda e: (e.source, e.target),
+)
+
+text = "Zhang San founded ByteDance. Li Si serves as CEO."
+graph.parse(text)
+
+print(f"Nodes: {len(graph.nodes)}, Edges: {len(graph.edges)}")
+for n in graph.nodes:
+ print(f" - {n.name} ({n.type})")
diff --git a/examples/providers/vllm_demo.py b/examples/providers/vllm_demo.py
new file mode 100644
index 0000000..b4f76da
--- /dev/null
+++ b/examples/providers/vllm_demo.py
@@ -0,0 +1,35 @@
+"""
+Local vLLM Provider Demo
+
+Extract entities and relationships using locally deployed vLLM services.
+Requires two running services:
+ - LLM: http://localhost:8000/v1
+ - Embedder: http://localhost:8001/v1
+
+Usage:
+ python examples/providers/vllm_demo.py
+"""
+
+from hyperextract import create_client, AutoGraph
+
+llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+)
+
+graph = AutoGraph(
+ instruction="Extract people and their relationships",
+ llm_client=llm,
+ embedder=emb,
+ node_key_extractor=lambda n: n.name,
+ edge_key_extractor=lambda e: (e.source, e.target, e.type),
+ nodes_in_edge_extractor=lambda e: (e.source, e.target),
+)
+
+text = "Zhang San founded ByteDance. Li Si serves as CEO."
+graph.parse(text)
+
+print(f"Nodes: {len(graph.nodes)}, Edges: {len(graph.edges)}")
+for n in graph.nodes:
+ print(f" - {n.name} ({n.type})")
diff --git a/hyperextract/__init__.py b/hyperextract/__init__.py
index 2acccb9..6a673ac 100644
--- a/hyperextract/__init__.py
+++ b/hyperextract/__init__.py
@@ -37,10 +37,12 @@ from .types import (
# Template engine API
from .utils.template_engine import Template
+# Client factory
+from .utils.client import create_client, create_llm, create_embedder, get_client
+
# Logging utilities
from .utils.logging import configure_logging, get_logger, set_log_level
-
from importlib.metadata import version
__version__ = version("hyperextract")
@@ -62,6 +64,11 @@ __all__ = [
"AutoSpatioTemporalGraph",
# Template engine
"Template",
+ # Client factory
+ "create_client",
+ "create_llm",
+ "create_embedder",
+ "get_client",
# Logging utilities
"configure_logging",
"get_logger",
diff --git a/hyperextract/cli/commands/config.py b/hyperextract/cli/commands/config.py
index 5dcac8f..1bf93b4 100644
--- a/hyperextract/cli/commands/config.py
+++ b/hyperextract/cli/commands/config.py
@@ -106,22 +106,25 @@ def _show_config():
cfg = config.show()
table = Table(title="Hyper-Extract Configuration")
- table.add_column("Service", style="cyan", width=15)
- table.add_column("Model", style="yellow", width=30)
- table.add_column("API Key", style="magenta", width=25)
- table.add_column("Base URL", style="green", width=25)
+ table.add_column("Service", style="cyan", width=12)
+ table.add_column("Provider", style="blue", width=12)
+ table.add_column("Model", style="yellow", width=28)
+ table.add_column("API Key", style="magenta", width=20)
+ table.add_column("Base URL", style="green", width=30)
llm_cfg = cfg["llm"]
emb_cfg = cfg["embedder"]
table.add_row(
"LLM",
+ llm_cfg.get("provider", "-") or "-",
llm_cfg["model"],
llm_cfg["api_key"][:10] + "..." if llm_cfg["api_key"] else "(not set)",
llm_cfg["base_url"] or "(default)",
)
table.add_row(
"Embedder",
+ emb_cfg.get("provider", "-") or "-",
emb_cfg["model"],
emb_cfg["api_key"][:10] + "..." if emb_cfg["api_key"] else "(not set)",
emb_cfg["base_url"] or "(default)",
@@ -141,6 +144,12 @@ def show(
@app.command(name="llm")
def llm(
+ provider: Optional[str] = typer.Option(
+ None,
+ "--provider",
+ "-p",
+ help="Provider preset: openai, bailian, vllm",
+ ),
api_key: Optional[str] = typer.Option(
None,
"--api-key",
@@ -171,6 +180,7 @@ def llm(
table = Table(title="LLM Configuration", show_header=False)
table.add_column("Key", style="cyan")
table.add_column("Value", style="green")
+ table.add_row("Provider", cfg.provider or "(not set)")
table.add_row("Model", cfg.model)
table.add_row(
"API Key", cfg.api_key[:10] + "..." if cfg.api_key else "(not set)"
@@ -185,6 +195,7 @@ def llm(
return
config.set_llm(
+ provider=provider,
model=model,
api_key=api_key,
base_url=base_url,
@@ -194,6 +205,12 @@ def llm(
@app.command(name="embedder")
def embedder(
+ provider: Optional[str] = typer.Option(
+ None,
+ "--provider",
+ "-p",
+ help="Provider preset: openai, bailian, vllm",
+ ),
api_key: Optional[str] = typer.Option(
None,
"--api-key",
@@ -226,6 +243,7 @@ def embedder(
table = Table(title="Embedder Configuration", show_header=False)
table.add_column("Key", style="cyan")
table.add_column("Value", style="green")
+ table.add_row("Provider", cfg.provider or "(not set)")
table.add_row("Model", cfg.model)
table.add_row(
"API Key", cfg.api_key[:10] + "..." if cfg.api_key else "(not set)"
@@ -240,6 +258,7 @@ def embedder(
return
config.set_embedder(
+ provider=provider,
api_key=api_key,
model=model,
base_url=base_url,
@@ -249,6 +268,12 @@ def embedder(
@app.command(name="init")
def init(
+ provider: Optional[str] = typer.Option(
+ None,
+ "--provider",
+ "-p",
+ help="Provider preset: openai, bailian, vllm",
+ ),
api_key: Optional[str] = typer.Option(
None,
"--api-key",
@@ -263,16 +288,54 @@ def init(
),
):
"""Initialize configuration interactively."""
- logger.info("command=config-init api_key_provided=%s", api_key is not None)
+ logger.info("command=config-init provider=%s api_key_provided=%s", provider, api_key is not None)
config = ConfigManager()
- if api_key:
+ # Quick mode: provider + api_key provided
+ if provider and api_key:
+ from hyperextract.utils.client import PROVIDER_PRESETS
+ preset = PROVIDER_PRESETS.get(provider, {})
+ llm_model = preset.get("default_llm") or "gpt-4o-mini"
+ emb_model = preset.get("default_embedder") or "text-embedding-3-small"
+ preset_url = preset.get("base_url") or ""
+ resolved_base = base_url or preset_url
+
config.set_llm(
+ provider=provider,
+ model=llm_model,
+ api_key=api_key,
+ base_url=resolved_base,
+ )
+ if emb_model:
+ config.set_embedder(
+ provider=provider,
+ model=emb_model,
+ api_key=api_key,
+ base_url=resolved_base,
+ )
+ else:
+ console.print("[yellow]Warning: Provider '{}' has no default embedder. Please configure embedder separately.[/yellow]".format(provider))
+
+ console.print("[bold green]Configuration saved successfully![/bold green]")
+ console.print()
+ console.print("[bold]Current settings:[/bold]")
+ console.print(f" [cyan]Provider:[/cyan] {provider}")
+ console.print(f" [cyan]LLM Model:[/cyan] {llm_model}")
+ if emb_model:
+ console.print(f" [cyan]Embedder Model:[/cyan] {emb_model}")
+ console.print(f" [cyan]Base URL:[/cyan] {resolved_base or '(default)'}")
+ return
+
+ # Legacy quick mode: only api_key provided (OpenAI defaults)
+ if api_key and not provider:
+ config.set_llm(
+ provider="openai",
model="gpt-4o-mini",
api_key=api_key,
base_url=base_url,
)
config.set_embedder(
+ provider="openai",
model="text-embedding-3-small",
api_key=api_key,
base_url=base_url,
@@ -280,6 +343,7 @@ def init(
console.print("[bold green]Configuration saved successfully![/bold green]")
console.print()
console.print("[bold]Current settings:[/bold]")
+ console.print(" [cyan]Provider:[/cyan] openai")
console.print(" [cyan]LLM Model:[/cyan] gpt-4o-mini")
console.print(" [cyan]Embedder Model:[/cyan] text-embedding-3-small")
console.print(" [cyan]API Key:[/cyan] " + api_key[:10] + "...")
@@ -287,50 +351,86 @@ def init(
console.print(f" [cyan]Base URL:[/cyan] {base_url}")
return
+ # Interactive mode
console.print("[bold blue]Hyper-Extract Configuration Setup[/bold blue]")
console.print()
- console.print("[bold]Step 1: LLM Configuration[/bold]")
- model = console.input(" Model (default: gpt-4o-mini): ") or "gpt-4o-mini"
+ from hyperextract.utils.client import PROVIDER_PRESETS
+
+ console.print("[bold]Step 1: Choose Provider[/bold]")
+ providers = [
+ ("openai", "OpenAI", "https://api.openai.com/v1"),
+ ("bailian", "阿里云百炼", "https://dashscope.aliyuncs.com/compatible-mode/v1"),
+ ("vllm", "本地 vLLM", "自定义地址"),
+ ("custom", "其他 OpenAI 兼容接口", "自定义地址"),
+ ]
+ for i, (key, name, url) in enumerate(providers, 1):
+ console.print(f" [{i}] {name:<20} ({url})")
+
+ choice = console.input("\n请选择 [1-4]: ").strip()
+ try:
+ selected = providers[int(choice) - 1][0] if choice.isdigit() else "openai"
+ except (IndexError, ValueError):
+ selected = "openai"
+
+ preset = PROVIDER_PRESETS.get(selected, {})
+ preset_url = preset.get("base_url") or ""
+ default_llm = preset.get("default_llm") or "gpt-4o-mini"
+ default_emb = preset.get("default_embedder") or "text-embedding-3-small"
+
+ console.print()
+ console.print(f"[bold]Step 2: LLM Configuration (Provider: {selected})[/bold]")
+
+ if selected == "vllm":
+ llm_model = console.input(f" LLM Model: ").strip()
+ llm_base_url = console.input(f" LLM Base URL (e.g. http://localhost:8000/v1): ").strip()
+ else:
+ llm_model = console.input(f" Model (default: {default_llm}): ").strip() or default_llm
+ llm_base_url = console.input(f" Base URL (default: {preset_url}, press Enter to skip): ").strip() or preset_url
llm_api_key = None
while not llm_api_key:
- llm_api_key = console.input(" API Key: ")
+ llm_api_key = console.input(" API Key: ").strip()
if not llm_api_key:
- console.print(
- " [red]API Key is required. Please enter your API key.[/red]"
- )
-
- llm_base_url = console.input(" Base URL (optional, press Enter to skip): ") or None
+ if selected == "vllm":
+ llm_api_key = "dummy"
+ console.print(" [dim]Using 'dummy' for vLLM[/dim]")
+ break
+ console.print(" [red]API Key is required. Please enter your API key.[/red]")
config.set_llm(
- model=model,
+ provider=selected,
+ model=llm_model,
api_key=llm_api_key,
- base_url=llm_base_url,
+ base_url=llm_base_url or None,
)
console.print()
- console.print("[bold]Step 2: Embedder Configuration[/bold]")
- emb_model = (
- console.input(" Model (default: text-embedding-3-small): ")
- or "text-embedding-3-small"
- )
+ console.print("[bold]Step 3: Embedder Configuration[/bold]")
+
+ if selected == "vllm":
+ emb_model = console.input(" Embedder Model (e.g. bge-m3): ").strip()
+ emb_base_url = console.input(" Embedder Base URL (e.g. http://localhost:8001/v1): ").strip()
+ else:
+ emb_model = console.input(f" Model (default: {default_emb}): ").strip() or default_emb
+ emb_base_url = console.input(f" Base URL (default: {preset_url}, press Enter to skip): ").strip() or preset_url
emb_api_key = None
while not emb_api_key:
- emb_api_key = console.input(" API Key: ")
+ emb_api_key = console.input(" API Key: ").strip()
if not emb_api_key:
- console.print(
- " [red]API Key is required. Please enter your API key.[/red]"
- )
-
- emb_base_url = console.input(" Base URL (optional, press Enter to skip): ") or None
+ if selected == "vllm":
+ emb_api_key = "dummy"
+ console.print(" [dim]Using 'dummy' for vLLM[/dim]")
+ break
+ console.print(" [red]API Key is required. Please enter your API key.[/red]")
config.set_embedder(
+ provider=selected,
model=emb_model,
api_key=emb_api_key,
- base_url=emb_base_url,
+ base_url=emb_base_url or None,
)
console.print()
diff --git a/hyperextract/cli/config.py b/hyperextract/cli/config.py
index c8f3438..a171925 100644
--- a/hyperextract/cli/config.py
+++ b/hyperextract/cli/config.py
@@ -12,15 +12,36 @@ from dataclasses import dataclass
DEFAULT_CONFIG_DIR = Path.home() / ".he"
DEFAULT_CONFIG_FILE = DEFAULT_CONFIG_DIR / "config.toml"
+# Provider presets: base_url and default models for each provider
+PROVIDER_PRESETS: Dict[str, Dict[str, str | None]] = {
+ "openai": {
+ "base_url": "https://api.openai.com/v1",
+ "default_llm": "gpt-4o-mini",
+ "default_embedder": "text-embedding-3-small",
+ },
+ "bailian": {
+ "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
+ "default_llm": "qwen3.6-plus",
+ "default_embedder": "text-embedding-v4",
+ },
+ "vllm": {
+ "base_url": None,
+ "default_llm": None,
+ "default_embedder": None,
+ },
+}
+
@dataclass
class LLMConfig:
+ provider: str = ""
model: str = "gpt-4o-mini"
api_key: str = ""
base_url: str = ""
def to_dict(self) -> Dict[str, Any]:
return {
+ "provider": self.provider,
"model": self.model,
"api_key": self.api_key,
"base_url": self.base_url,
@@ -29,6 +50,7 @@ class LLMConfig:
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "LLMConfig":
return cls(
+ provider=data.get("provider", ""),
model=data.get("model", "gpt-4o-mini"),
api_key=data.get("api_key", ""),
base_url=data.get("base_url", ""),
@@ -37,12 +59,14 @@ class LLMConfig:
@dataclass
class EmbedderConfig:
+ provider: str = ""
model: str = "text-embedding-3-small"
api_key: str = ""
base_url: str = ""
def to_dict(self) -> Dict[str, Any]:
return {
+ "provider": self.provider,
"model": self.model,
"api_key": self.api_key,
"base_url": self.base_url,
@@ -51,6 +75,7 @@ class EmbedderConfig:
@classmethod
def from_dict(cls, data: Dict[str, Any]) -> "EmbedderConfig":
return cls(
+ provider=data.get("provider", ""),
model=data.get("model", "text-embedding-3-small"),
api_key=data.get("api_key", ""),
base_url=data.get("base_url", ""),
@@ -91,31 +116,56 @@ class ConfigManager:
with open(self.config_path, "wb") as f:
tomli_w.dump(data, f)
+ def _resolve_base_url(self, provider: str, explicit_base_url: str) -> str:
+ """Resolve base_url from provider preset. Explicit value takes precedence."""
+ if explicit_base_url:
+ return explicit_base_url
+ if provider in PROVIDER_PRESETS:
+ preset_url = PROVIDER_PRESETS[provider].get("base_url")
+ if preset_url is None:
+ raise ValueError(
+ f"Provider '{provider}' requires explicit base_url. "
+ f"Please set it via config or environment variable."
+ )
+ return preset_url
+ return ""
+
def get_llm_config(self) -> LLMConfig:
"""Get LLM config with environment variable fallback."""
config = LLMConfig(
+ provider=self.llm.provider,
model=self.llm.model,
api_key=self.llm.api_key or os.environ.get("OPENAI_API_KEY", ""),
- base_url=self.llm.base_url or os.environ.get("OPENAI_BASE_URL", ""),
+ base_url=self._resolve_base_url(
+ self.llm.provider,
+ self.llm.base_url or os.environ.get("OPENAI_BASE_URL", ""),
+ ),
)
return config
def get_embedder_config(self) -> EmbedderConfig:
"""Get Embedder config with environment variable fallback."""
config = EmbedderConfig(
+ provider=self.embedder.provider,
model=self.embedder.model,
api_key=self.embedder.api_key or os.environ.get("OPENAI_API_KEY", ""),
- base_url=self.embedder.base_url or os.environ.get("OPENAI_BASE_URL", ""),
+ base_url=self._resolve_base_url(
+ self.embedder.provider,
+ self.embedder.base_url or os.environ.get("OPENAI_BASE_URL", ""),
+ ),
)
return config
def set_llm(
self,
+ provider: Optional[str] = None,
model: Optional[str] = None,
api_key: Optional[str] = None,
base_url: Optional[str] = None,
) -> None:
"""Set LLM configuration."""
+ if provider is not None:
+ self.llm.provider = provider
if model:
self.llm.model = model
if api_key is not None:
@@ -126,11 +176,14 @@ class ConfigManager:
def set_embedder(
self,
+ provider: Optional[str] = None,
model: Optional[str] = None,
api_key: Optional[str] = None,
base_url: Optional[str] = None,
) -> None:
"""Set Embedder configuration."""
+ if provider is not None:
+ self.embedder.provider = provider
if model:
self.embedder.model = model
if api_key is not None:
@@ -161,13 +214,20 @@ class ConfigManager:
llm_config = self.get_llm_config()
embedder_config = self.get_embedder_config()
- if not llm_config.api_key:
+ # vLLM mode: api_key can be empty or dummy, but base_url is required
+ if llm_config.provider == "vllm":
+ if not llm_config.base_url:
+ return False, "vLLM provider requires base_url."
+ elif not llm_config.api_key:
return (
False,
"LLM API key is not configured. Run 'he config llm --api-key YOUR_KEY'",
)
- if not embedder_config.api_key:
+ if embedder_config.provider == "vllm":
+ if not embedder_config.base_url:
+ return False, "vLLM embedder requires base_url."
+ elif not embedder_config.api_key:
return (
False,
"Embedder API key is not configured. Run 'he config embedder --api-key YOUR_KEY'",
diff --git a/hyperextract/utils/client.py b/hyperextract/utils/client.py
index 352171b..bb4c759 100644
--- a/hyperextract/utils/client.py
+++ b/hyperextract/utils/client.py
@@ -1,9 +1,20 @@
-"""Client Factory - Create OpenAI LLM and Embedder clients from config."""
+"""Client Factory - Create OpenAI LLM and Embedder clients from config.
+
+Provides three levels of API:
+ - create_client(): Unified creation for both LLM and Embedder
+ - create_llm() / create_embedder(): Separate creation for advanced use
+ - get_client(): Read from config.toml (backward compatible)
+
+String shorthand format: provider:model@url
+ - "bailian" → provider only, use preset defaults
+ - "bailian:qwen-plus" → provider + model, use preset URL
+ - "vllm:Qwen3.5-9B@http://localhost:8000/v1" → full specification
+"""
import os
import logging
from pathlib import Path
-from typing import Any, List, Optional, Tuple
+from typing import Any, Dict, List, Optional, Tuple
from langchain_core.embeddings import Embeddings
from langchain_core.language_models.chat_models import BaseChatModel
@@ -16,6 +27,25 @@ DEFAULT_CONFIG_FILE = DEFAULT_CONFIG_DIR / "config.toml"
# Official OpenAI API base URL — only this endpoint accepts pre-tokenized input
OPENAI_API_URL = "https://api.openai.com/v1"
+# Provider presets: base_url and default models for each provider
+PROVIDER_PRESETS: Dict[str, Dict[str, str | None]] = {
+ "openai": {
+ "base_url": "https://api.openai.com/v1",
+ "default_llm": "gpt-4o-mini",
+ "default_embedder": "text-embedding-3-small",
+ },
+ "bailian": {
+ "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
+ "default_llm": "qwen3.6-plus",
+ "default_embedder": "text-embedding-v4",
+ },
+ "vllm": {
+ "base_url": None,
+ "default_llm": None,
+ "default_embedder": None,
+ },
+}
+
class CompatibleEmbeddings(Embeddings):
"""Embeddings for OpenAI-compatible providers that only accept string input.
@@ -134,9 +164,223 @@ class CompatibleEmbeddings(Embeddings):
return self.embed_documents([text])[0]
+def _parse_client_spec(
+ spec: str | dict,
+ *,
+ api_key: str = "",
+ default_kind: str = "llm",
+) -> Dict[str, Any]:
+ """Parse a client specification string or dict into a config dict.
+
+ String formats:
+ - "bailian" → provider only, use preset defaults
+ - "bailian:qwen-plus" → provider + model, use preset URL
+ - "vllm:Qwen3.5-9B@http://localhost:8000/v1" → full specification
+
+ Args:
+ spec: String shorthand or dict config.
+ api_key: Fallback API key.
+ default_kind: "llm" or "embedder", used for default model selection.
+
+ Returns:
+ Dict with keys: provider, model, base_url, api_key
+ """
+ if isinstance(spec, dict):
+ return {
+ "provider": spec.get("provider", ""),
+ "model": spec.get("model", ""),
+ "base_url": spec.get("base_url", ""),
+ "api_key": spec.get("api_key") or api_key,
+ **{k: v for k, v in spec.items() if k not in ("provider", "model", "base_url", "api_key")},
+ }
+
+ # Parse string shorthand: provider:model@url
+ provider = ""
+ model = ""
+ base_url = ""
+
+ parts = spec.split(":", 1)
+ provider = parts[0].strip()
+
+ if len(parts) > 1:
+ rest = parts[1].strip()
+ if "@" in rest:
+ model, base_url = rest.split("@", 1)
+ model = model.strip()
+ base_url = base_url.strip()
+ else:
+ model = rest.strip()
+
+ # Fill defaults from preset
+ preset = PROVIDER_PRESETS.get(provider, {})
+ if not base_url:
+ preset_url = preset.get("base_url")
+ if preset_url is not None:
+ base_url = preset_url
+ if not model:
+ default_key = f"default_{default_kind}"
+ model = preset.get(default_key) or ""
+
+ return {
+ "provider": provider,
+ "model": model,
+ "base_url": base_url,
+ "api_key": api_key,
+ }
+
+
+def create_llm(
+ spec: str | dict,
+ *,
+ api_key: str = "",
+ **kwargs: Any,
+) -> BaseChatModel:
+ """Create an LLM client from a specification string or dict.
+
+ Args:
+ spec: String shorthand (e.g. "bailian:qwen-plus") or dict config.
+ api_key: API key fallback.
+ **kwargs: Extra args forwarded to ChatOpenAI (e.g. temperature).
+
+ Returns:
+ Configured ChatOpenAI instance.
+
+ Examples:
+ >>> llm = create_llm("bailian:qwen-plus", api_key="sk-xxx")
+ >>> llm = create_llm("vllm:Qwen3.5-9B@localhost:8000/v1", api_key="dummy")
+ >>> llm = create_llm({"provider": "bailian", "model": "qwen-plus", "temperature": 0.5})
+ """
+ config = _parse_client_spec(spec, api_key=api_key, default_kind="llm")
+ config.update(kwargs)
+
+ from langchain_openai import ChatOpenAI
+
+ return ChatOpenAI(
+ model=config["model"],
+ api_key=config["api_key"] or os.environ.get("OPENAI_API_KEY", ""),
+ base_url=config.get("base_url") or None,
+ temperature=config.get("temperature", 0),
+ )
+
+
+def create_embedder(
+ spec: str | dict,
+ *,
+ api_key: str = "",
+ **kwargs: Any,
+) -> Embeddings:
+ """Create an Embedder client from a specification string or dict.
+
+ Args:
+ spec: String shorthand (e.g. "bailian:text-embedding-v4") or dict config.
+ api_key: API key fallback.
+ **kwargs: Extra args forwarded to the embedder class.
+
+ Returns:
+ Configured OpenAIEmbeddings or CompatibleEmbeddings instance.
+
+ Examples:
+ >>> emb = create_embedder("bailian", api_key="sk-xxx")
+ >>> emb = create_embedder("vllm:bge-m3@localhost:8001/v1", api_key="dummy")
+ """
+ config = _parse_client_spec(spec, api_key=api_key, default_kind="embedder")
+
+ base_url = config.get("base_url", "")
+ uses_custom = bool(base_url and base_url.rstrip("/") != OPENAI_API_URL)
+
+ if uses_custom:
+ return CompatibleEmbeddings(
+ model=config["model"],
+ api_key=config["api_key"] or os.environ.get("OPENAI_API_KEY", ""),
+ base_url=base_url,
+ **kwargs,
+ )
+ else:
+ from langchain_openai import OpenAIEmbeddings
+
+ return OpenAIEmbeddings(
+ model=config["model"],
+ api_key=config["api_key"] or os.environ.get("OPENAI_API_KEY", ""),
+ **kwargs,
+ )
+
+
+def create_client(
+ llm: str | dict | None = None,
+ embedder: str | dict | None = None,
+ *,
+ provider: str = "",
+ api_key: str = "",
+ **kwargs: Any,
+) -> Tuple[BaseChatModel, Embeddings]:
+ """Create both LLM and Embedder clients in one call.
+
+ Supports three patterns:
+
+ **Pattern A: Single provider string (cloud API, simplest)**
+ >>> llm, emb = create_client("bailian", api_key="sk-xxx")
+ # Uses preset defaults: qwen3.6-plus + text-embedding-v4
+
+ **Pattern B: Separate specs (vLLM, two independent services)**
+ >>> llm, emb = create_client(
+ ... llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ ... embedder="vllm:bge-m3@http://localhost:8001/v1",
+ ... api_key="dummy",
+ ... )
+
+ **Pattern C: Mixed deployment**
+ >>> llm = create_client(
+ ... llm="bailian:qwen-plus",
+ ... embedder="vllm:bge-m3@localhost:8001/v1",
+ ... api_key="sk-xxx",
+ ... )
+
+ Args:
+ llm: LLM spec string/dict, or None to auto-infer from provider.
+ embedder: Embedder spec string/dict, or None to auto-infer from provider.
+ provider: Fallback provider when llm/embedder are bare strings without provider prefix.
+ api_key: Shared API key for both services.
+ **kwargs: Extra args forwarded to ChatOpenAI (e.g. temperature).
+
+ Returns:
+ (llm_client, embedder_client) tuple.
+ """
+ # Pattern A: Single provider shorthand
+ if llm is None and embedder is None:
+ if not provider:
+ raise ValueError(
+ "Must provide llm=, embedder=, or provider= argument.\n"
+ "Examples:\n"
+ ' create_client("bailian", api_key="...")\n'
+ ' create_client(llm="bailian:qwen-plus", embedder="bailian", api_key="...")\n'
+ ' create_client(llm="vllm:Qwen3.5-9B@localhost:8000", '
+ 'embedder="vllm:bge-m3@localhost:8001", api_key="dummy")'
+ )
+ # Use provider preset defaults for both
+ llm = provider
+ embedder = provider
+
+ # Parse llm config
+ llm_config = _parse_client_spec(llm or provider, api_key=api_key, default_kind="llm")
+ llm_config.update(kwargs)
+
+ # Parse embedder config
+ # For cloud providers, embedder defaults to llm's provider
+ embedder_spec = embedder or llm_config.get("provider", "")
+ emb_config = _parse_client_spec(embedder_spec, api_key=api_key, default_kind="embedder")
+
+ # Build clients
+ llm_client = create_llm(llm_config, api_key=api_key)
+ embedder_client = create_embedder(emb_config, api_key=api_key)
+
+ return llm_client, embedder_client
+
+
def get_client(config_path: str | Path = None) -> Tuple[BaseChatModel, Embeddings]:
"""Get OpenAI LLM client and Embedder from config.
+ Backward-compatible: reads ~/.he/config.toml.
+
Args:
config_path: Config file path, default ~/.he/config.toml
@@ -157,34 +401,24 @@ def get_client(config_path: str | Path = None) -> Tuple[BaseChatModel, Embedding
llm_config = manager.get_llm_config()
emb_config = manager.get_embedder_config()
- from langchain_openai import ChatOpenAI, OpenAIEmbeddings
-
- llm_client = ChatOpenAI(
- model=llm_config.model,
- api_key=llm_config.api_key or os.environ.get("OPENAI_API_KEY", ""),
- base_url=llm_config.base_url or None,
- temperature=0,
+ # Build LLM client
+ llm_client = create_llm(
+ {
+ "provider": llm_config.provider,
+ "model": llm_config.model,
+ "base_url": llm_config.base_url,
+ "api_key": llm_config.api_key,
+ }
)
- # When using a non-OpenAI endpoint (e.g. Ollama, Azure, LiteLLM),
- # use CompatibleEmbeddings which always sends strings (not pre-tokenized
- # integer lists) to the API. Most OpenAI-compatible endpoints don't
- # support tokenized input and will return "invalid input type" errors.
- uses_custom_base_url = (
- emb_config.base_url and emb_config.base_url.rstrip("/") != OPENAI_API_URL
+ # Build embedder client
+ embedder_client = create_embedder(
+ {
+ "provider": emb_config.provider,
+ "model": emb_config.model,
+ "base_url": emb_config.base_url,
+ "api_key": emb_config.api_key,
+ }
)
- if uses_custom_base_url:
- embedder = CompatibleEmbeddings(
- model=emb_config.model,
- api_key=emb_config.api_key or os.environ.get("OPENAI_API_KEY", ""),
- base_url=emb_config.base_url,
- )
- else:
- embedder = OpenAIEmbeddings(
- model=emb_config.model,
- api_key=emb_config.api_key or os.environ.get("OPENAI_API_KEY", ""),
- base_url=emb_config.base_url or None,
- )
-
- return llm_client, embedder
+ return llm_client, embedder_client
diff --git a/mkdocs.yml b/mkdocs.yml
index f250d84..e1dfa42 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -89,6 +89,7 @@ plugins:
- python/guides/using-templates.md
- python/guides/using-methods.md
- python/guides/working-with-autotypes.md
+ - python/guides/provider-configuration.md
- python/guides/custom-templates.md
- python/guides/search-and-chat.md
- python/guides/incremental-updates.md
@@ -106,6 +107,7 @@ plugins:
- Concepts:
- concepts/index.md
- concepts/architecture.md
+ - concepts/provider-system.md
- concepts/autotypes.md
- concepts/methods.md
- concepts/templates-format.md
@@ -177,6 +179,7 @@ plugins:
- zh/python/guides/using-templates.md
- zh/python/guides/using-methods.md
- zh/python/guides/working-with-autotypes.md
+ - zh/python/guides/provider-configuration.md
- zh/python/guides/custom-templates.md
- zh/python/guides/search-and-chat.md
- zh/python/guides/incremental-updates.md
@@ -194,6 +197,7 @@ plugins:
- 核心概念:
- zh/concepts/index.md
- zh/concepts/architecture.md
+ - zh/concepts/provider-system.md
- zh/concepts/autotypes.md
- zh/concepts/methods.md
- zh/concepts/templates-format.md
diff --git a/tests/utils/__init__.py b/tests/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/tests/utils/test_client.py b/tests/utils/test_client.py
new file mode 100644
index 0000000..7f25481
--- /dev/null
+++ b/tests/utils/test_client.py
@@ -0,0 +1,339 @@
+"""Tests for client factory functions."""
+
+import os
+from pathlib import Path
+from typing import Any
+from unittest.mock import patch, MagicMock
+
+import pytest
+
+from hyperextract.utils.client import (
+ _parse_client_spec,
+ create_llm,
+ create_embedder,
+ create_client,
+ CompatibleEmbeddings,
+ get_client,
+ PROVIDER_PRESETS,
+)
+
+
+# =============================================================================
+# _parse_client_spec
+# =============================================================================
+
+class TestParseClientSpec:
+ """Tests for _parse_client_spec string parser."""
+
+ def test_provider_only(self):
+ """Provider string only — use all defaults."""
+ result = _parse_client_spec("bailian", api_key="sk-test")
+ assert result["provider"] == "bailian"
+ assert result["model"] == "qwen3.6-plus" # default_llm preset
+ assert result["base_url"] == "https://dashscope.aliyuncs.com/compatible-mode/v1"
+ assert result["api_key"] == "sk-test"
+
+ def test_provider_and_model(self):
+ """Provider:model format — override model, keep preset URL."""
+ result = _parse_client_spec("bailian:qwen-plus", api_key="sk-test")
+ assert result["provider"] == "bailian"
+ assert result["model"] == "qwen-plus"
+ assert result["base_url"] == "https://dashscope.aliyuncs.com/compatible-mode/v1"
+
+ def test_full_spec(self):
+ """Provider:model@url format — full manual specification."""
+ result = _parse_client_spec(
+ "vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ api_key="dummy",
+ )
+ assert result["provider"] == "vllm"
+ assert result["model"] == "Qwen3.5-9B"
+ assert result["base_url"] == "http://localhost:8000/v1"
+ assert result["api_key"] == "dummy"
+
+ def test_embedder_defaults(self):
+ """Embedder default kind uses embedder preset."""
+ result = _parse_client_spec("bailian", api_key="sk-test", default_kind="embedder")
+ assert result["model"] == "text-embedding-v4" # default_embedder preset
+
+ def test_dict_input(self):
+ """Dict input is passed through with api_key fallback."""
+ result = _parse_client_spec(
+ {"provider": "custom", "model": "my-model", "base_url": "http://test/v1"},
+ api_key="fallback-key",
+ )
+ assert result["provider"] == "custom"
+ assert result["model"] == "my-model"
+ assert result["base_url"] == "http://test/v1"
+ assert result["api_key"] == "fallback-key"
+
+ def test_unknown_provider(self):
+ """Unknown provider falls through without defaults."""
+ result = _parse_client_spec("unknown", api_key="sk-test")
+ assert result["provider"] == "unknown"
+ assert result["model"] == ""
+ assert result["base_url"] == ""
+
+ def test_vllm_no_defaults(self):
+ """vLLM provider has None defaults — no URL or model auto-filled."""
+ result = _parse_client_spec("vllm", api_key="dummy")
+ assert result["provider"] == "vllm"
+ assert result["model"] == ""
+ assert result["base_url"] == ""
+
+
+# =============================================================================
+# create_llm / create_embedder
+# =============================================================================
+
+class TestCreateLLM:
+ """Tests for create_llm factory."""
+
+ def test_create_llm_bailian(self):
+ """Create LLM with bailian preset."""
+ llm = create_llm("bailian", api_key="sk-test")
+ assert llm.model_name == "qwen3.6-plus"
+
+ def test_create_llm_openai(self):
+ """Create LLM with openai preset."""
+ llm = create_llm("openai", api_key="sk-test")
+ assert llm.model_name == "gpt-4o-mini"
+
+ def test_create_llm_custom_model(self):
+ """Override model via string shorthand."""
+ llm = create_llm("bailian:qwen-plus", api_key="sk-test")
+ assert llm.model_name == "qwen-plus"
+
+ def test_create_llm_from_dict(self):
+ """Create LLM from dict config."""
+ llm = create_llm(
+ {"provider": "custom", "model": "gpt-4", "base_url": "http://test/v1"},
+ api_key="sk-test",
+ temperature=0.5,
+ )
+ assert llm.model_name == "gpt-4"
+
+
+class TestCreateEmbedder:
+ """Tests for create_embedder factory."""
+
+ def test_create_embedder_openai(self):
+ """OpenAI embedder uses native OpenAIEmbeddings."""
+ emb = create_embedder("openai", api_key="sk-test")
+ from langchain_openai import OpenAIEmbeddings
+
+ assert isinstance(emb, OpenAIEmbeddings)
+
+ def test_create_embedder_bailian(self):
+ """Bailian embedder uses CompatibleEmbeddings (custom base_url)."""
+ emb = create_embedder("bailian", api_key="sk-test")
+ assert isinstance(emb, CompatibleEmbeddings)
+ assert emb._model == "text-embedding-v4"
+
+ def test_create_embedder_vllm(self):
+ """vLLM embedder uses CompatibleEmbeddings."""
+ emb = create_embedder(
+ "vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+ )
+ assert isinstance(emb, CompatibleEmbeddings)
+ assert emb._model == "bge-m3"
+
+
+# =============================================================================
+# create_client (unified API)
+# =============================================================================
+
+class TestCreateClient:
+ """Tests for create_client unified factory."""
+
+ def test_pattern_a_single_provider(self):
+ """Pattern A: Single provider string for both LLM and embedder."""
+ llm, emb = create_client("bailian", api_key="sk-test")
+ assert llm.model_name == "qwen3.6-plus"
+ assert isinstance(emb, CompatibleEmbeddings)
+ assert emb._model == "text-embedding-v4"
+
+ def test_pattern_b_separate_specs(self):
+ """Pattern B: Separate llm and embedder specs (vLLM)."""
+ llm, emb = create_client(
+ llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="dummy",
+ )
+ assert llm.model_name == "Qwen3.5-9B"
+ assert isinstance(emb, CompatibleEmbeddings)
+ assert emb._model == "bge-m3"
+
+ def test_pattern_c_mixed(self):
+ """Pattern C: Mixed deployment (Bailian LLM + local embedder)."""
+ llm, emb = create_client(
+ llm="bailian:qwen-plus",
+ embedder="vllm:bge-m3@http://localhost:8001/v1",
+ api_key="sk-test",
+ )
+ assert llm.model_name == "qwen-plus"
+ assert isinstance(emb, CompatibleEmbeddings)
+ assert emb._model == "bge-m3"
+
+ def test_no_args_raises(self):
+ """Calling with no arguments raises ValueError."""
+ with pytest.raises(ValueError, match="Must provide"):
+ create_client()
+
+ def test_temperature_forwarded(self):
+ """Extra kwargs like temperature are forwarded to LLM."""
+ llm, _ = create_client("bailian", api_key="sk-test", temperature=0.5)
+ assert llm.temperature == 0.5
+
+
+# =============================================================================
+# CompatibleEmbeddings
+# =============================================================================
+
+class TestCompatibleEmbeddings:
+ """Tests for CompatibleEmbeddings wrapper."""
+
+ @pytest.fixture
+ def mock_openai_client(self):
+ """Mock OpenAI client that returns deterministic embeddings."""
+ mock = MagicMock()
+ mock.embeddings.create.return_value = MagicMock(
+ data=[MagicMock(embedding=[0.1, 0.2, 0.3])]
+ )
+ return mock
+
+ def test_embed_query(self, mock_openai_client):
+ """embed_query sends string input and returns vector."""
+ emb = CompatibleEmbeddings(
+ model="test-model",
+ api_key="sk-test",
+ base_url="http://test/v1",
+ )
+ with patch.object(emb, "_client", mock_openai_client):
+ result = emb.embed_query("hello world")
+ assert len(result) == 3
+ assert result == [0.1, 0.2, 0.3]
+
+ def test_embed_documents(self, mock_openai_client):
+ """embed_documents sends batch string input."""
+ emb = CompatibleEmbeddings(
+ model="test-model",
+ api_key="sk-test",
+ base_url="http://test/v1",
+ )
+ mock_openai_client.embeddings.create.return_value = MagicMock(
+ data=[
+ MagicMock(embedding=[0.1, 0.2]),
+ MagicMock(embedding=[0.3, 0.4]),
+ ]
+ )
+ with patch.object(emb, "_client", mock_openai_client):
+ result = emb.embed_documents(["hello", "world"])
+ assert len(result) == 2
+ assert result[0] == [0.1, 0.2]
+ assert result[1] == [0.3, 0.4]
+
+ def test_empty_texts(self):
+ """Empty input returns empty list."""
+ emb = CompatibleEmbeddings(
+ model="test-model",
+ api_key="sk-test",
+ base_url="http://test/v1",
+ )
+ assert emb.embed_documents([]) == []
+
+ def test_chunking(self, mock_openai_client):
+ """Long texts are split into chunks that fit token limits."""
+ emb = CompatibleEmbeddings(
+ model="test-model",
+ api_key="sk-test",
+ base_url="http://test/v1",
+ chunk_size=1, # Force batching into single-item calls
+ )
+ mock_openai_client.embeddings.create.return_value = MagicMock(
+ data=[MagicMock(embedding=[0.5, 0.5])]
+ )
+ with patch.object(emb, "_client", mock_openai_client):
+ result = emb.embed_documents(["short", "also short"])
+ assert len(result) == 2
+ # Should have been called twice due to chunk_size=1
+ assert mock_openai_client.embeddings.create.call_count == 2
+
+
+# =============================================================================
+# get_client (config file)
+# =============================================================================
+
+class TestGetClient:
+ """Tests for get_client reading from config file."""
+
+ def test_get_client_from_file(self, tmp_path: Path):
+ """Read client config from TOML file."""
+ config_file = tmp_path / "config.toml"
+ config_file.write_text(
+ '[llm]\n'
+ 'provider = "bailian"\n'
+ 'model = "qwen-plus"\n'
+ 'api_key = "sk-from-file"\n'
+ 'base_url = ""\n'
+ '[embedder]\n'
+ 'provider = "bailian"\n'
+ 'model = "text-embedding-v4"\n'
+ 'api_key = "sk-from-file"\n'
+ 'base_url = ""\n'
+ )
+ llm, emb = get_client(config_file)
+ assert llm.model_name == "qwen-plus"
+ assert isinstance(emb, CompatibleEmbeddings)
+
+ def test_get_client_missing_file(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch):
+ """Missing config file returns default configs."""
+ # Clear env vars that could interfere with base_url resolution
+ monkeypatch.delenv("OPENAI_BASE_URL", raising=False)
+ config_file = tmp_path / "nonexistent.toml"
+ llm, emb = get_client(config_file)
+ assert llm.model_name == "gpt-4o-mini" # default fallback
+ from langchain_openai import OpenAIEmbeddings
+
+ assert isinstance(emb, OpenAIEmbeddings)
+
+
+# =============================================================================
+# PROVIDER_PRESETS consistency
+# =============================================================================
+
+class TestProviderPresets:
+ """Tests for PROVIDER_PRESETS data consistency."""
+
+ def test_bailian_defaults(self):
+ """Bailian preset has expected defaults."""
+ preset = PROVIDER_PRESETS["bailian"]
+ assert preset["base_url"] == "https://dashscope.aliyuncs.com/compatible-mode/v1"
+ assert preset["default_llm"] == "qwen3.6-plus"
+ assert preset["default_embedder"] == "text-embedding-v4"
+
+ def test_openai_defaults(self):
+ """OpenAI preset has expected defaults."""
+ preset = PROVIDER_PRESETS["openai"]
+ assert preset["base_url"] == "https://api.openai.com/v1"
+ assert preset["default_llm"] == "gpt-4o-mini"
+ assert preset["default_embedder"] == "text-embedding-3-small"
+
+ def test_vllm_no_defaults(self):
+ """vLLM preset has None defaults (must be specified explicitly)."""
+ preset = PROVIDER_PRESETS["vllm"]
+ assert preset["base_url"] is None
+ assert preset["default_llm"] is None
+ assert preset["default_embedder"] is None
+
+ def test_no_deepseek_provider(self):
+ """DeepSeek should not have its own preset (accessed via Bailian)."""
+ assert "deepseek" not in PROVIDER_PRESETS
+
+ def test_all_presets_have_base_url_or_none(self):
+ """Every preset has either a base_url or None (for vLLM)."""
+ for name, preset in PROVIDER_PRESETS.items():
+ assert "base_url" in preset
+ assert "default_llm" in preset
+ assert "default_embedder" in preset