Skip to content

Same agent Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.

feat: independent model management, pricing recalculation, and public pricing API (#1123)

Aman Agarwald0bcb6cc7420
diff --git a/docs/docs.json b/docs/docs.json
index 4d2bce71..c7ef8b8f 100644
--- a/docs/docs.json
+++ b/docs/docs.json
@@ -129,6 +129,13 @@
"latest/openlit/chat"
]
},
+ {
+ "group": "Pricing & Models",
+ "pages": [
+ "latest/openlit/pricing/manage-models",
+ "latest/openlit/pricing/cost-recalculation"
+ ]
+ },
{
"group": "Prompts and Experiments",
"pages": [
diff --git a/docs/latest/openlit/pricing/cost-recalculation.mdx b/docs/latest/openlit/pricing/cost-recalculation.mdx
new file mode 100644
index 00000000..aca1b110
--- /dev/null
+++ b/docs/latest/openlit/pricing/cost-recalculation.mdx
@@ -0,0 +1,121 @@
+---
+title: 'Cost Recalculation'
+description: 'Recalculate the cost of LLM traces using current model pricing — manually or on a schedule'
+sidebarTitle: 'Cost Recalculation'
+---
+
+The Pricing feature recalculates the `gen_ai.usage.cost` attribute on existing LLM traces using the per-model pricing stored in [Manage Models](/latest/openlit/pricing/manage-models). This is useful when:
+
+- Your SDK didn't send cost data (or sent `0`).
+- Model pricing has changed and you want historical traces to reflect the new prices.
+- You added a custom model after traces were already ingested.
+
+## How it works
+
+For each trace, OpenLIT:
+
+1. Reads `gen_ai.system` (provider) and `gen_ai.request.model` from the span attributes.
+2. Looks up the matching model in the `openlit_provider_models` ClickHouse table.
+3. Computes the cost:
+ ```
+ cost = (input_tokens / 1,000,000) × input_price_per_m_token
+ + (output_tokens / 1,000,000) × output_price_per_m_token
+ ```
+4. Writes the result back to the trace:
+ ```sql
+ ALTER TABLE otel_traces
+ UPDATE SpanAttributes = mapUpdate(SpanAttributes, map('gen_ai.usage.cost', '<cost>'))
+ WHERE SpanId = '<span_id>'
+ ```
+
+<Info>
+Traces with missing provider, model, or zero tokens are **skipped** — they are not counted as errors.
+If the model is not found in Manage Models, the trace is also skipped. Add the model first, then recalculate.
+</Info>
+
+## Manual recalculation
+
+Recalculate the cost for a single trace directly from the trace detail panel.
+
+<Steps>
+ <Step title="Open a trace">
+ Navigate to **Requests** and click on any LLM trace to open the detail panel.
+ </Step>
+ <Step title="Click the recalculate icon">
+ In the **Cost** metric card at the top of the panel, click the refresh icon (↻).
+ - If the cost is missing or zero, a **pinging dot** draws attention to the icon.
+ - If the cost already exists, clicking recalculates it using the latest pricing.
+ </Step>
+ <Step title="See the result">
+ A toast notification shows the updated cost. The panel refreshes automatically to display the new value.
+ </Step>
+</Steps>
+
+**API**: `POST /api/pricing/{spanId}` — returns `{ success, data: { spanId, cost } }` or `{ success: false, err: "..." }`.
+
+## Auto recalculation (cron-based)
+
+Automatically recompute pricing on a schedule for all newly ingested LLM traces.
+
+<Steps>
+ <Step title="Open the Pricing page">
+ Navigate to **Pricing** in the sidebar under the Configuration section.
+ </Step>
+ <Step title="Enable Auto Pricing">
+ Toggle the **Enable Auto Pricing** switch.
+ </Step>
+ <Step title="Set the cron schedule">
+ Enter a standard cron expression. For example:
+ - `*/15 * * * *` — every 15 minutes
+ - `0 * * * *` — every hour
+ - `0 0 * * *` — once a day at midnight
+ </Step>
+ <Step title="Save">
+ Click **Save** (or **Update** if editing). OpenLIT installs a cron job that runs the pricing recalculation on the schedule you defined.
+ </Step>
+</Steps>
+
+### How auto pricing selects traces
+
+On each cron tick:
+
+| Step | What happens |
+|------|-------------|
+| **1. Select LLM spans** | Queries `otel_traces` for spans whose `gen_ai.operation.name` matches supported LLM operations (e.g. `chat`). |
+| **2. Incremental window** | Only spans with `Timestamp >= last successful cron run` are processed. The first run processes all historical LLM spans. |
+| **3. Look up model** | For each span, reads `gen_ai.system` (provider) and `gen_ai.request.model`, then finds the matching row in `openlit_provider_models`. |
+| **4. Compute cost** | `(input_tokens / 1M) × input_price + (output_tokens / 1M) × output_price`. |
+| **5. Write back** | Updates `SpanAttributes['gen_ai.usage.cost']` on the trace via `ALTER TABLE otel_traces UPDATE`. |
+| **6. Log run** | Records `totalSpans`, `totalUpdated`, `totalFailed`, `totalSkipped` in the cron log with a status of `SUCCESS`, `PARTIAL_SUCCESS`, or `FAILURE`. |
+
+<Note>
+Spans with missing provider/model/tokens or an unknown model are **skipped** (not counted as errors). Add the model to Manage Models and the next cron run will pick it up.
+</Note>
+
+### Cron restoration on restart
+
+When the OpenLIT server restarts (e.g. after a container redeployment), all active auto-pricing cron jobs are automatically restored from the database. No manual re-setup is needed.
+
+## API reference
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/pricing/config` | `GET` | Get the current pricing config (auto flag, cron schedule) |
+| `/api/pricing/config` | `POST` | Create or update pricing config |
+| `/api/pricing/{spanId}` | `POST` | Manually recalculate cost for a single trace |
+| `/api/pricing/auto` | `POST` | Trigger auto pricing run (called by cron; allowlisted in middleware) |
+| `/api/openground/models/export` | `GET` | Export all model pricing as downloadable JSON (authenticated) |
+| `/api/pricing/export/{dbConfigId}` | `GET` | Public pricing URL for SDK `pricing_json` (no auth) |
+| `/api/openground/models/import` | `POST` | Bulk-import models from JSON, skipping duplicates |
+
+## Configuration
+
+The pricing config is stored in the SQLite `PricingConfigs` table (managed by Prisma):
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `id` | String | Config ID (auto-generated) |
+| `databaseConfigId` | String | Links to the ClickHouse database config |
+| `auto` | Boolean | Whether auto pricing is enabled |
+| `recurringTime` | String | Cron schedule (e.g. `*/15 * * * *`) |
+| `meta` | String (JSON) | Internal metadata including `cronJobId` |
diff --git a/docs/latest/openlit/pricing/manage-models.mdx b/docs/latest/openlit/pricing/manage-models.mdx
new file mode 100644
index 00000000..f1d85c70
--- /dev/null
+++ b/docs/latest/openlit/pricing/manage-models.mdx
@@ -0,0 +1,191 @@
+---
+title: 'Manage Models'
+description: 'View, edit, and add LLM models with per-model pricing across all providers'
+sidebarTitle: 'Manage Models'
+---
+
+Manage Models is an independent feature for managing LLM provider metadata and per-model pricing. It is used across OpenLIT — including Chat, Evaluations, OpenGround, and Pricing — to look up model details and compute costs.
+
+## Overview
+
+Every ClickHouse database connected to OpenLIT automatically gets a pre-populated set of **14 providers and 45+ models** with default pricing. All models are fully editable — you can update pricing, add new models, or delete ones you don't use.
+
+### Supported providers (out of the box)
+
+| Provider | Example Models |
+|----------|---------------|
+| OpenAI | GPT-4o, GPT-4o Mini, GPT-4 Turbo, GPT-3.5 Turbo |
+| Anthropic | Claude 3.5 Sonnet, Claude 3 Haiku, Claude 3 Sonnet |
+| Google AI | Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 1.0 Pro |
+| Mistral AI | Mistral Large, Mistral Medium, Mistral Small |
+| Groq | Llama 3.3 70B, Llama 3.1 8B, Mixtral 8x7B |
+| Perplexity | Sonar Pro, Sonar, Sonar Reasoning |
+| Azure OpenAI | GPT-4o, GPT-4o Mini, GPT-4 Turbo, GPT-3.5 Turbo |
+| Cohere | Command R+, Command R, Command, Command Light |
+| Together AI | Llama 3.1 70B Turbo, Llama 3.1 8B Turbo, Mixtral 8x7B |
+| Fireworks AI | Llama 3.1 70B, Llama 3.1 8B, Mixtral 8x7B |
+| DeepSeek | DeepSeek Chat, DeepSeek Coder |
+| xAI | Grok Beta, Grok 2, Grok 3, Grok 3 Mini |
+| Hugging Face | Llama 3 70B, Mistral 7B, Gemma 2 9B |
+| Replicate | Llama 3 70B, Llama 3 8B, Mixtral 8x7B |
+
+## Get started
+
+<Steps>
+ <Step title="Open Manage Models">
+ Navigate to **Manage Models** in the sidebar under the Configuration section.
+ </Step>
+ <Step title="Browse providers and models">
+ The left sidebar lists all providers. Click a provider to expand its models. Each model shows its context window and input price.
+ - **Default** badge — a model that was pre-populated by OpenLIT.
+ - **Custom** badge — a model you added manually.
+ </Step>
+ <Step title="Edit a model">
+ Click any model to open the editor panel on the right. You can change:
+ - **Display Name** — the human-readable name shown in dropdowns.
+ - **Model Type** — chat, embeddings, images, or audio.
+ - **Context Window** — the maximum token context length.
+ - **Input / Output Price per 1M Tokens** — used by the Pricing feature to compute costs.
+ - **Capabilities** — comma-separated list (e.g. `function-calling, vision, streaming`).
+ </Step>
+ <Step title="Add a new model">
+ Click the **+** button next to any provider header to add a new model. Fill in the Model ID (the identifier your SDK sends, e.g. `gpt-4o-2024-08-06`), display name, and pricing.
+ </Step>
+ <Step title="Delete a model">
+ Open a model in the editor and click the **Delete** button. Deleted models can be re-added at any time.
+ </Step>
+</Steps>
+
+## Export pricing JSON
+
+`GET /api/openground/models/export` returns all models in an SDK-compatible `pricing.json` format, grouped by model type.
+
+This is useful for feeding custom pricing into the OpenLIT Python or TypeScript SDKs when the default pricing data doesn't include your models.
+
+```json Example response
+{
+ "chat": {
+ "gpt-4o": {
+ "promptPrice": 0.0025,
+ "completionPrice": 0.01
+ },
+ "claude-3-5-sonnet-20240620": {
+ "promptPrice": 0.003,
+ "completionPrice": 0.015
+ }
+ },
+ "embeddings": {
+ "text-embedding-3-small": 0.00002
+ }
+}
+```
+
+<Tip>
+Click the **Export Pricing** button in the top-right corner of the Manage Models page to download this file directly.
+</Tip>
+
+## Public SDK Pricing URL
+
+Each database config has a **public, unauthenticated** pricing URL that you can pass directly to the OpenLIT SDK. This lets your SDK load pricing from the same models you manage in the UI — no manual JSON files needed.
+
+The URL is shown in a bar below the header on the Manage Models page. Click **Copy** to copy it to your clipboard.
+
+<Tip>
+Click the **SDK Usage** button in the header to open a dialog with ready-to-paste code snippets for Python, TypeScript, and Go. Each snippet pre-fills the public URL and includes a copy button.
+</Tip>
+
+<CodeGroup>
+```python Python
+import openlit
+
+openlit.init(
+ otlp_endpoint="http://127.0.0.1:4318",
+ pricing_json="http://your-openlit-host/api/pricing/export/<your-db-config-id>",
+)
+```
+
+```typescript TypeScript
+import openlit from "openlit";
+
+openlit.init({
+ otlpEndpoint: "http://127.0.0.1:4318",
+ pricingJson: "http://your-openlit-host/api/pricing/export/<your-db-config-id>",
+});
+```
+
+```go Go
+package main
+
+import openlit "github.com/openlit/openlit/sdk/go"
+
+func main() {
+ openlit.Init(openlit.Config{
+ OtlpEndpoint: "http://127.0.0.1:4318",
+ PricingJson: "http://your-openlit-host/api/pricing/export/<your-db-config-id>",
+ })
+}
+```
+</CodeGroup>
+
+**Endpoint:** `GET /api/pricing/export/{dbConfigId}`
+
+- No authentication required — safe to embed in application configs.
+- Returns the same SDK-compatible pricing JSON format as the export button.
+- Responses are cached for 5 minutes (`Cache-Control: public, max-age=300`).
+
+<Warning>
+The URL contains your database config ID. While the endpoint only returns pricing data (no secrets or traces), share it only with systems that should access your model pricing.
+</Warning>
+
+<Note>
+The SDK fetches this URL on startup. If you update a model's price in Manage Models, restart your app (or wait for the SDK's refresh interval) to pick up the new pricing.
+</Note>
+
+## Import pricing JSON
+
+You can bulk-import models from a JSON file. This is useful for syncing pricing across environments or importing from a colleague's export.
+
+<Steps>
+ <Step title="Open the import dialog">
+ Click the **Import Pricing JSON** button in the header of the Manage Models page.
+ </Step>
+ <Step title="Paste JSON">
+ Paste either the **structured format** or the **SDK pricing_json format**.
+
+ **Structured format** (recommended — includes provider and metadata):
+ ```json
+ {
+ "models": [
+ {
+ "provider": "openai",
+ "model_id": "gpt-4o-2024-08-06",
+ "displayName": "GPT-4o (Aug 2024)",
+ "inputPricePerMToken": 2.5,
+ "outputPricePerMToken": 10
+ }
+ ]
+ }
+ ```
+
+ **SDK pricing_json format** (provider defaults to `"unknown"` — edit after import):
+ ```json
+ {
+ "chat": {
+ "gpt-4o": { "promptPrice": 0.0025, "completionPrice": 0.01 }
+ }
+ }
+ ```
+ </Step>
+ <Step title="Import">
+ Click **Import Pricing JSON**. Models that already exist (same `provider` + `model_id`) are automatically **skipped** — no duplicates are created.
+ </Step>
+</Steps>
+
+**Endpoint:** `POST /api/openground/models/import` (authenticated)
+
+## Architecture
+
+- **ClickHouse tables**: `openlit_providers` (API key configs) and `openlit_provider_models` (all models with pricing).
+- **No `database_config_id` column** — each ClickHouse instance is its own scope. When you connect a new ClickHouse database, the migration automatically creates the tables and seeds all default models.
+- **Platform library**: `src/lib/platform/providers/` — `provider-registry.ts`, `models-service.ts`, `config.ts`, `default-models.ts`.
+- **API routes**: `GET/POST/DELETE /api/openground/models`, `GET /api/openground/models/export`, `GET /api/openground/providers`, `POST /api/openground/models/import`, `GET /api/pricing/export/{dbConfigId}` (public).
diff --git a/src/client/package-lock.json b/src/client/package-lock.json
index a0671539..67661385 100644
--- a/src/client/package-lock.json
+++ b/src/client/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "openlit",
- "version": "1.17.3",
+ "version": "1.18.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "openlit",
- "version": "1.17.3",
+ "version": "1.18.0",
"dependencies": {
"@ai-sdk/anthropic": "^3.0.6",
"@ai-sdk/cohere": "^3.0.3",
diff --git a/src/client/package.json b/src/client/package.json
index ba204f5c..6135847b 100644
--- a/src/client/package.json
+++ b/src/client/package.json
@@ -1,6 +1,6 @@
{
"name": "openlit",
- "version": "1.17.3",
+ "version": "1.18.0",
"private": true,
"scripts": {
"dev": "next dev",
diff --git a/src/client/prisma/migrations/20260410120000_pricing_config/migration.sql b/src/client/prisma/migrations/20260410120000_pricing_config/migration.sql
new file mode 100644
index 00000000..0d5dd9a5
--- /dev/null
+++ b/src/client/prisma/migrations/20260410120000_pricing_config/migration.sql
@@ -0,0 +1,11 @@
+-- CreateTable
+CREATE TABLE "PricingConfigs" (
+ "id" TEXT NOT NULL PRIMARY KEY,
+ "database_config_id" TEXT NOT NULL,
+ "auto" BOOLEAN NOT NULL DEFAULT false,
+ "recurringTime" TEXT NOT NULL DEFAULT '',
+ "meta" TEXT NOT NULL DEFAULT '{}'
+);
+
+-- CreateIndex
+CREATE UNIQUE INDEX "PricingConfigs_database_config_id_key" ON "PricingConfigs"("database_config_id");
diff --git a/src/client/prisma/schema.prisma b/src/client/prisma/schema.prisma
index 7c1b306d..38338588 100644
--- a/src/client/prisma/schema.prisma
+++ b/src/client/prisma/schema.prisma
@@ -220,3 +220,11 @@ model EvaluationConfigs {
recurringTime String
meta String
}
+
+model PricingConfigs {
+ id String @id @default(cuid())
+ databaseConfigId String @unique @map("database_config_id")
+ auto Boolean @default(false)
+ recurringTime String @default("")
+ meta String @default("{}")
+}
diff --git a/src/client/scripts/pricing/auto.js b/src/client/scripts/pricing/auto.js
new file mode 100644
index 00000000..a3453420
--- /dev/null
+++ b/src/client/scripts/pricing/auto.js
@@ -0,0 +1,55 @@
+// Environment variables for auto pricing
+const PRICING_CONFIG_ID = process.env.PRICING_CONFIG_ID;
+const CRON_ID = process.env.CRON_ID;
+const API_URL = process.env.API_URL;
+
+// Validate required environment variables
+if (!PRICING_CONFIG_ID || !CRON_ID || !API_URL) {
+ console.error("[ERROR] Missing required environment variables");
+ process.exit(1);
+}
+
+// Prepare request payload
+const payload = JSON.stringify({
+ pricingConfigId: PRICING_CONFIG_ID,
+ cronId: CRON_ID,
+});
+
+async function callApi() {
+ const startTime = new Date().toISOString();
+ console.log(`[${startTime}] Starting auto pricing update`);
+
+ try {
+ // Make API request
+ const response = await fetch(`${API_URL}/api/pricing/auto`, {
+ method: "POST",
+ body: payload,
+ headers: {
+ "Content-Type": "application/json",
+ "X-CRON-JOB": "true",
+ },
+ });
+
+ if (!response.ok) {
+ throw new Error(`HTTP ${response.status}`);
+ }
+
+ const { err, success } = await response.json();
+ if (success) {
+ const endTime = new Date().toISOString();
+ console.log(`[${endTime}] Auto pricing update completed successfully`);
+ }
+
+ if (err) {
+ throw new Error(err);
+ }
+ } catch (error) {
+ const errorTime = new Date().toISOString();
+ console.error(
+ `[${errorTime}] Auto pricing update saw some error: ${error.message}`
+ );
+ }
+}
+
+// Execute auto pricing update
+callApi();
diff --git a/src/client/src/__tests__/lib/platform/pricing/pricing-config.test.ts b/src/client/src/__tests__/lib/platform/pricing/pricing-config.test.ts
new file mode 100644
index 00000000..5bfe5233
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/pricing/pricing-config.test.ts
@@ -0,0 +1,304 @@
+jest.mock('@/constants/messages', () => ({
+ __esModule: true,
+ default: jest.fn(() => ({
+ DATABASE_CONFIG_NOT_FOUND: 'DB config not found',
+ OPERATION_FAILED: 'Operation failed',
+ CRON_JOB_UPDATION_ERROR: 'Cron job error',
+ })),
+}));
+jest.mock('@/lib/db-config', () => ({
+ getDBConfigByUser: jest.fn(),
+}));
+jest.mock('@/lib/prisma', () => ({
+ __esModule: true,
+ default: {
+ pricingConfigs: {
+ findFirst: jest.fn(),
+ findMany: jest.fn(),
+ update: jest.fn(),
+ create: jest.fn(),
+ },
+ },
+}));
+jest.mock('@/utils/asaw', () => jest.fn());
+jest.mock('@/utils/error', () => ({
+ throwIfError: jest.fn((condition: boolean, msg: string) => {
+ if (condition) throw new Error(msg);
+ }),
+}));
+jest.mock('@/helpers/server/cron', () => ({
+ __esModule: true,
+ default: jest.fn().mockImplementation(() => ({
+ validateCronSchedule: jest.fn(),
+ updateCrontab: jest.fn().mockResolvedValue(undefined),
+ deleteCronJob: jest.fn().mockResolvedValue(undefined),
+ })),
+}));
+jest.mock('@/utils/json', () => ({
+ jsonParse: jest.fn((v: string) => {
+ try { return JSON.parse(v); } catch { return {}; }
+ }),
+ jsonStringify: jest.fn((v: unknown) => JSON.stringify(v)),
+}));
+jest.mock('lodash', () => ({
+ merge: jest.fn((a: any, b: any) => ({ ...a, ...b })),
+}));
+jest.mock('crypto', () => ({
+ randomUUID: jest.fn(() => 'test-cron-uuid'),
+}));
+jest.mock('path', () => ({
+ join: jest.fn((...args: string[]) => args.join('/')),
+}));
+
+import { getPricingConfig, setPricingConfig, restorePricingCronJobs } from '@/lib/platform/pricing/config';
+import { getDBConfigByUser } from '@/lib/db-config';
+import prisma from '@/lib/prisma';
+import asaw from '@/utils/asaw';
+import { throwIfError } from '@/utils/error';
+import Cron from '@/helpers/server/cron';
+import getMessage from '@/constants/messages';
+import { randomUUID } from 'crypto';
+import { jsonParse } from '@/utils/json';
+
+const mockDBConfig = { id: 'db-1', name: 'test-db' };
+
+beforeEach(() => {
+ jest.resetAllMocks();
+
+ (getMessage as jest.Mock).mockReturnValue({
+ DATABASE_CONFIG_NOT_FOUND: 'DB config not found',
+ OPERATION_FAILED: 'Operation failed',
+ CRON_JOB_UPDATION_ERROR: 'Cron job error',
+ });
+
+ (throwIfError as jest.Mock).mockImplementation((condition: boolean, msg: string) => {
+ if (condition) throw new Error(msg);
+ });
+
+ (randomUUID as jest.Mock).mockReturnValue('test-cron-uuid');
+
+ (Cron as jest.Mock).mockImplementation(() => ({
+ validateCronSchedule: jest.fn(),
+ updateCrontab: jest.fn().mockResolvedValue(undefined),
+ deleteCronJob: jest.fn().mockResolvedValue(undefined),
+ }));
+
+ (jsonParse as jest.Mock).mockImplementation((v: string) => {
+ try { return JSON.parse(v); } catch { return {}; }
+ });
+
+ // Default asaw: wrap a promise into [err, data]
+ (asaw as jest.Mock).mockImplementation(async (promise: Promise<any>) => {
+ try {
+ const data = await promise;
+ return [null, data];
+ } catch (err) {
+ return [err, null];
+ }
+ });
+
+ // Re-apply path.join mock
+ const path = require('path');
+ (path.join as jest.Mock).mockImplementation((...args: string[]) => args.join('/'));
+});
+
+describe('getPricingConfig', () => {
+ it('returns config when it exists', async () => {
+ const mockConfig = { id: 'pc-1', auto: false, recurringTime: '', meta: '{}', databaseConfigId: 'db-1' };
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.findFirst as jest.Mock).mockResolvedValue(mockConfig);
+
+ const result = await getPricingConfig();
+
+ expect(result).toEqual(mockConfig);
+ });
+
+ it('returns null when no config exists', async () => {
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.findFirst as jest.Mock).mockResolvedValue(null);
+
+ const result = await getPricingConfig();
+
+ expect(result).toBeNull();
+ });
+
+ it('returns null when no dbConfig', async () => {
+ (getDBConfigByUser as jest.Mock).mockRejectedValue(new Error('no db'));
+
+ const result = await getPricingConfig();
+
+ expect(result).toBeNull();
+ });
+});
+
+describe('setPricingConfig', () => {
+ it('creates a new config with cron when auto=true', async () => {
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.create as jest.Mock).mockResolvedValue({ id: 'pc-new' });
+
+ await setPricingConfig(
+ { auto: true, recurringTime: '*/15 * * * *', meta: '{}' },
+ 'http://localhost:3000'
+ );
+
+ expect(prisma.pricingConfigs.create).toHaveBeenCalledWith(
+ expect.objectContaining({
+ data: expect.objectContaining({
+ auto: true,
+ recurringTime: '*/15 * * * *',
+ databaseConfigId: 'db-1',
+ }),
+ })
+ );
+
+ const cronInstance = (Cron as jest.Mock).mock.results[0].value;
+ expect(cronInstance.updateCrontab).toHaveBeenCalledWith(
+ expect.objectContaining({
+ cronId: 'test-cron-uuid',
+ cronSchedule: '*/15 * * * *',
+ cronScriptPath: expect.stringContaining('scripts/pricing/auto.js'),
+ })
+ );
+ });
+
+ it('updates existing config', async () => {
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.findFirst as jest.Mock).mockResolvedValue({
+ id: 'pc-1', auto: false, recurringTime: '', meta: '{}', databaseConfigId: 'db-1',
+ });
+ (prisma.pricingConfigs.update as jest.Mock).mockResolvedValue({ id: 'pc-1' });
+
+ await setPricingConfig(
+ { id: 'pc-1', auto: false, recurringTime: '', meta: '{}' },
+ 'http://localhost:3000'
+ );
+
+ expect(prisma.pricingConfigs.update).toHaveBeenCalled();
+ });
+
+ it('throws when cron updateCrontab fails (auto=true)', async () => {
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.create as jest.Mock).mockResolvedValue({ id: 'pc-new' });
+ (Cron as jest.Mock).mockImplementation(() => ({
+ validateCronSchedule: jest.fn(),
+ updateCrontab: jest.fn().mockRejectedValue(new Error('cron failed')),
+ deleteCronJob: jest.fn(),
+ }));
+
+ await expect(
+ setPricingConfig(
+ { auto: true, recurringTime: '*/15 * * * *', meta: '{}' },
+ 'http://localhost:3000'
+ )
+ ).rejects.toThrow('cron failed');
+ });
+
+ it('deletes cron job when auto=false', async () => {
+ (getDBConfigByUser as jest.Mock).mockResolvedValue(mockDBConfig);
+ (prisma.pricingConfigs.findFirst as jest.Mock).mockResolvedValue({
+ id: 'pc-1', auto: true, recurringTime: '* * * * *',
+ meta: JSON.stringify({ cronJobId: 'existing-cron-id' }),
+ databaseConfigId: 'db-1',
+ });
+ (prisma.pricingConfigs.update as jest.Mock).mockResolvedValue({ id: 'pc-1' });
+
+ await setPricingConfig(
+ { id: 'pc-1', auto: false, recurringTime: '', meta: '{}' },
+ 'http://localhost:3000'
+ );
+
+ const cronInstance = (Cron as jest.Mock).mock.results[0].value;
+ expect(cronInstance.deleteCronJob).toHaveBeenCalledWith('existing-cron-id');
+ });
+});
+
+describe('restorePricingCronJobs', () => {
+ it('restores cron jobs for all active configs', async () => {
+ (prisma.pricingConfigs.findMany as jest.Mock).mockResolvedValue([
+ {
+ id: 'pc-1',
+ auto: true,
+ recurringTime: '*/10 * * * *',
+ meta: JSON.stringify({ cronJobId: 'cron-1' }),
+ },
+ ]);
+
+ await restorePricingCronJobs('http://localhost:3000');
+
+ const cronInstance = (Cron as jest.Mock).mock.results[0].value;
+ expect(cronInstance.updateCrontab).toHaveBeenCalledWith(
+ expect.objectContaining({
+ cronId: 'cron-1',
+ cronSchedule: '*/10 * * * *',
+ })
+ );
+ });
+
+ it('skips configs without cronJobId', async () => {
+ (prisma.pricingConfigs.findMany as jest.Mock).mockResolvedValue([
+ { id: 'pc-1', auto: true, recurringTime: '* * * * *', meta: '{}' },
+ ]);
+
+ await restorePricingCronJobs('http://localhost:3000');
+
+ // Cron is instantiated but updateCrontab is never called (no cronJobId)
+ const cronInstance = (Cron as jest.Mock).mock.results[0]?.value;
+ if (cronInstance) {
+ expect(cronInstance.updateCrontab).not.toHaveBeenCalled();
+ }
+ });
+
+ it('does nothing when no auto configs exist', async () => {
+ (prisma.pricingConfigs.findMany as jest.Mock).mockResolvedValue([]);
+
+ await restorePricingCronJobs('http://localhost:3000');
+
+ // Cron may or may not be instantiated, but updateCrontab should not run
+ const cronInstance = (Cron as jest.Mock).mock.results[0]?.value;
+ if (cronInstance) {
+ expect(cronInstance.updateCrontab).not.toHaveBeenCalled();
+ }
+ });
+
+ it('catches per-config errors and continues', async () => {
+ (prisma.pricingConfigs.findMany as jest.Mock).mockResolvedValue([
+ {
+ id: 'pc-1',
+ auto: true,
+ recurringTime: '*/10 * * * *',
+ meta: JSON.stringify({ cronJobId: 'cron-1' }),
+ },
+ {
+ id: 'pc-2',
+ auto: true,
+ recurringTime: '*/10 * * * *',
+ meta: JSON.stringify({ cronJobId: 'cron-2' }),
+ },
+ ]);
+ let call = 0;
+ (Cron as jest.Mock).mockImplementation(() => ({
+ validateCronSchedule: jest.fn(),
+ updateCrontab: jest.fn().mockImplementation(() => {
+ call++;
+ if (call === 1) throw new Error('cron-1 failed');
+ }),
+ deleteCronJob: jest.fn(),
+ }));
+
+ // Should not throw — per-config errors are caught
+ await expect(
+ restorePricingCronJobs('http://localhost:3000')
+ ).resolves.toBeUndefined();
+ });
+
+ it('catches top-level findMany errors', async () => {
+ (prisma.pricingConfigs.findMany as jest.Mock).mockRejectedValue(
+ new Error('DB down')
+ );
+
+ // Should not throw — top-level error is caught
+ await expect(
+ restorePricingCronJobs('http://localhost:3000')
+ ).resolves.toBeUndefined();
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/pricing/pricing.test.ts b/src/client/src/__tests__/lib/platform/pricing/pricing.test.ts
new file mode 100644
index 00000000..cf2e1fac
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/pricing/pricing.test.ts
@@ -0,0 +1,405 @@
+jest.mock('@/constants/messages', () => ({
+ __esModule: true,
+ default: jest.fn(() => ({
+ UNAUTHORIZED_USER: 'Unauthorized',
+ TRACE_NOT_FOUND: 'Trace not found',
+ DATABASE_CONFIG_NOT_FOUND: 'DB config not found',
+ TRACE_FETCHING_ERROR: 'Trace fetching error',
+ })),
+}));
+jest.mock('@/lib/session', () => ({
+ getCurrentUser: jest.fn(),
+}));
+jest.mock('@/lib/platform/common', () => ({
+ dataCollector: jest.fn(),
+ OTEL_TRACES_TABLE_NAME: 'otel_traces',
+}));
+jest.mock('@/lib/platform/request', () => ({
+ getRequestViaSpanId: jest.fn(),
+}));
+jest.mock('@/lib/platform/providers/provider-registry', () => ({
+ ProviderRegistry: {
+ getModel: jest.fn(),
+ },
+}));
+jest.mock('@/helpers/server/trace', () => ({
+ getTraceMappingKeyFullPath: jest.fn((key: string) => {
+ const map: Record<string, string> = {
+ cost: 'gen_ai.usage.cost',
+ model: 'gen_ai.request.model',
+ provider: 'gen_ai.system',
+ promptTokens: 'gen_ai.usage.input_tokens',
+ completionTokens: 'gen_ai.usage.output_tokens',
+ type: 'gen_ai.operation.name',
+ };
+ return map[key] || key;
+ }),
+}));
+jest.mock('@/constants/traces', () => ({
+ SUPPORTED_EVALUATION_OPERATIONS: ['chat'],
+}));
+jest.mock('@/utils/sanitizer', () => ({
+ __esModule: true,
+ default: { sanitizeValue: jest.fn((v: string) => v) },
+}));
+jest.mock('@/utils/error', () => ({
+ throwIfError: jest.fn((condition: boolean, msg: string) => {
+ if (condition) throw new Error(msg);
+ }),
+}));
+jest.mock('@/utils/asaw', () => jest.fn());
+jest.mock('@/lib/platform/pricing/config', () => ({
+ getPricingConfigById: jest.fn(),
+}));
+jest.mock('@/lib/platform/cron-log', () => ({
+ getLastRunCronLogByCronId: jest.fn().mockResolvedValue(null),
+ insertCronLog: jest.fn().mockResolvedValue({ err: null }),
+}));
+jest.mock('@/lib/db-config', () => ({
+ getDBConfigById: jest.fn(),
+ getDBConfigByUser: jest.fn(),
+}));
+jest.mock('date-fns', () => ({
+ differenceInSeconds: jest.fn(() => 1),
+}));
+// Mock dynamic import of prisma used inside setPricingForSpanId
+jest.mock('@/lib/prisma', () => ({
+ __esModule: true,
+ default: {
+ pricingConfigs: {
+ findFirst: jest.fn().mockResolvedValue({ databaseConfigId: 'db-1' }),
+ },
+ },
+}));
+
+import { setPricingForSpanId, autoUpdatePricing } from '@/lib/platform/pricing';
+import { getCurrentUser } from '@/lib/session';
+import { dataCollector } from '@/lib/platform/common';
+import { getRequestViaSpanId } from '@/lib/platform/request';
+import { ProviderRegistry } from '@/lib/platform/providers/provider-registry';
+import { getPricingConfigById } from '@/lib/platform/pricing/config';
+import { getDBConfigById } from '@/lib/db-config';
+import { insertCronLog } from '@/lib/platform/cron-log';
+import getMessage from '@/constants/messages';
+import { throwIfError } from '@/utils/error';
+import asaw from '@/utils/asaw';
+
+beforeEach(() => {
+ jest.resetAllMocks();
+
+ (getMessage as jest.Mock).mockReturnValue({
+ UNAUTHORIZED_USER: 'Unauthorized',
+ TRACE_NOT_FOUND: 'Trace not found',
+ DATABASE_CONFIG_NOT_FOUND: 'DB config not found',
+ TRACE_FETCHING_ERROR: 'Trace fetching error',
+ });
+
+ (throwIfError as jest.Mock).mockImplementation((condition: boolean, msg: string) => {
+ if (condition) throw new Error(msg);
+ });
+
+ // Re-apply insertCronLog mock
+ (insertCronLog as jest.Mock).mockResolvedValue({ err: null });
+
+ // Re-apply prisma mock for setPricingForSpanId's dynamic import
+ const prisma = require('@/lib/prisma').default;
+ prisma.pricingConfigs.findFirst.mockResolvedValue({ databaseConfigId: 'db-1' });
+});
+
+describe('setPricingForSpanId', () => {
+ const mockTrace = {
+ SpanId: 'span-1',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {
+ 'gen_ai.system': 'openai',
+ 'gen_ai.request.model': 'gpt-4o',
+ 'gen_ai.usage.input_tokens': '100',
+ 'gen_ai.usage.output_tokens': '200',
+ },
+ };
+
+ it('computes and writes cost successfully', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({ record: mockTrace });
+ (ProviderRegistry.getModel as jest.Mock).mockResolvedValue({
+ id: 'gpt-4o',
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ });
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(true);
+ expect(result.data).toBeDefined();
+ expect(result.data!.cost).toBeCloseTo(
+ (100 / 1_000_000) * 2.5 + (200 / 1_000_000) * 10.0
+ );
+ // Should call ALTER TABLE to update the trace
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining('mapUpdate'),
+ }),
+ 'exec',
+ expect.any(String)
+ );
+ });
+
+ it('returns error when model not found', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({ record: mockTrace });
+ (ProviderRegistry.getModel as jest.Mock).mockResolvedValue(null);
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(false);
+ expect(result.err).toContain('not found');
+ });
+
+ it('returns error when trace has no provider', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({
+ record: {
+ SpanId: 'span-1',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {},
+ },
+ });
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(false);
+ expect(result.err).toContain('Missing');
+ });
+
+ it('throws when user is not authenticated', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue(null);
+
+ await expect(setPricingForSpanId('span-1')).rejects.toThrow('Unauthorized');
+ });
+
+ it('returns error when zero tokens', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({
+ record: {
+ SpanId: 'span-1',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {
+ 'gen_ai.system': 'openai',
+ 'gen_ai.request.model': 'gpt-4o',
+ 'gen_ai.usage.input_tokens': '0',
+ 'gen_ai.usage.output_tokens': '0',
+ },
+ },
+ });
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(false);
+ expect(result.err).toContain('zero tokens');
+ });
+
+ it('returns error when writeCostToTrace fails', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({ record: mockTrace });
+ (ProviderRegistry.getModel as jest.Mock).mockResolvedValue({
+ id: 'gpt-4o',
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ });
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'write failed' });
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(false);
+ expect(result.err).toBe('write failed');
+ });
+
+ it('returns DB config not found when no config + no fallback', async () => {
+ (getCurrentUser as jest.Mock).mockResolvedValue({ id: 'user-1' });
+ (getRequestViaSpanId as jest.Mock).mockResolvedValue({ record: mockTrace });
+ const prisma = require('@/lib/prisma').default;
+ prisma.pricingConfigs.findFirst.mockResolvedValue(null);
+ // The fallback uses asaw(getDBConfigByUser(true)) — return null dbConfig
+ (asaw as jest.Mock).mockResolvedValue([null, null]);
+
+ const result = await setPricingForSpanId('span-1');
+
+ expect(result.success).toBe(false);
+ expect(result.err).toBe('DB config not found');
+ });
+});
+
+describe('autoUpdatePricing', () => {
+ it('processes traces and writes costs', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue({
+ id: 'pc-1',
+ databaseConfigId: 'db-1',
+ });
+ (asaw as jest.Mock).mockResolvedValue([null, { id: 'db-1' }]);
+ (dataCollector as jest.Mock)
+ // First call: SELECT traces
+ .mockResolvedValueOnce({
+ data: [
+ {
+ SpanId: 'span-1',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {
+ 'gen_ai.system': 'openai',
+ 'gen_ai.request.model': 'gpt-4o',
+ 'gen_ai.usage.input_tokens': '100',
+ 'gen_ai.usage.output_tokens': '200',
+ },
+ },
+ ],
+ })
+ // Second call: ALTER TABLE UPDATE
+ .mockResolvedValueOnce({ err: null });
+
+ (ProviderRegistry.getModel as jest.Mock).mockResolvedValue({
+ id: 'gpt-4o',
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ });
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'pc-1',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(true);
+ expect(insertCronLog).toHaveBeenCalledWith(
+ expect.objectContaining({
+ meta: expect.objectContaining({ totalUpdated: 1 }),
+ }),
+ 'db-1'
+ );
+ });
+
+ it('returns error when pricing config not found', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue(null);
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'nonexistent',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(false);
+ expect(result.err).toContain('not found');
+ });
+
+ it('returns error when DB config not found', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue({
+ id: 'pc-1',
+ databaseConfigId: 'db-missing',
+ });
+ (asaw as jest.Mock).mockResolvedValue([null, null]);
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'pc-1',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(false);
+ expect(result.err).toBe('DB config not found');
+ });
+
+ it('logs FAILURE on trace fetch error', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue({
+ id: 'pc-1',
+ databaseConfigId: 'db-1',
+ });
+ (asaw as jest.Mock).mockResolvedValue([null, { id: 'db-1' }]);
+ (dataCollector as jest.Mock).mockResolvedValueOnce({ err: 'fetch failed' });
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'pc-1',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(false);
+ expect(insertCronLog).toHaveBeenCalledWith(
+ expect.objectContaining({
+ runStatus: 'FAILURE',
+ }),
+ 'db-1'
+ );
+ });
+
+ it('counts errors when writeCostToTrace fails', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue({
+ id: 'pc-1',
+ databaseConfigId: 'db-1',
+ });
+ (asaw as jest.Mock).mockResolvedValue([null, { id: 'db-1' }]);
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({
+ data: [
+ {
+ SpanId: 'span-1',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {
+ 'gen_ai.system': 'openai',
+ 'gen_ai.request.model': 'gpt-4o',
+ 'gen_ai.usage.input_tokens': '100',
+ 'gen_ai.usage.output_tokens': '200',
+ },
+ },
+ ],
+ })
+ .mockResolvedValueOnce({ err: 'write failed' });
+
+ (ProviderRegistry.getModel as jest.Mock).mockResolvedValue({
+ id: 'gpt-4o',
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ });
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'pc-1',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(true);
+ expect(insertCronLog).toHaveBeenCalledWith(
+ expect.objectContaining({
+ meta: expect.objectContaining({
+ totalUpdated: 0,
+ totalFailed: 1,
+ }),
+ }),
+ 'db-1'
+ );
+ });
+
+ it('skips traces without provider/model', async () => {
+ (getPricingConfigById as jest.Mock).mockResolvedValue({
+ id: 'pc-1',
+ databaseConfigId: 'db-1',
+ });
+ (asaw as jest.Mock).mockResolvedValue([null, { id: 'db-1' }]);
+ (dataCollector as jest.Mock).mockResolvedValueOnce({
+ data: [
+ {
+ SpanId: 'span-no-model',
+ Timestamp: '2026-01-01',
+ SpanAttributes: {},
+ },
+ ],
+ });
+
+ const result = await autoUpdatePricing({
+ pricingConfigId: 'pc-1',
+ cronId: 'cron-1',
+ });
+
+ expect(result.success).toBe(true);
+ // No ALTER TABLE calls — trace was skipped
+ expect(insertCronLog).toHaveBeenCalledWith(
+ expect.objectContaining({
+ meta: expect.objectContaining({ totalSkipped: 1, totalUpdated: 0 }),
+ }),
+ 'db-1'
+ );
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/config.test.ts b/src/client/src/__tests__/lib/platform/providers/config.test.ts
new file mode 100644
index 00000000..d5672722
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/config.test.ts
@@ -0,0 +1,281 @@
+jest.mock('@/lib/platform/common', () => ({
+ dataCollector: jest.fn(),
+}));
+jest.mock('@/utils/sanitizer', () => ({
+ __esModule: true,
+ default: { sanitizeValue: jest.fn((v: string) => v) },
+}));
+jest.mock('@/constants/messages', () => ({
+ __esModule: true,
+ default: jest.fn(() => ({
+ OPERATION_FAILED: 'Operation failed',
+ })),
+}));
+
+import {
+ getOpenGroundConfigs,
+ getOpenGroundConfigWithSecret,
+ upsertOpenGroundConfig,
+ deleteOpenGroundConfig,
+ toggleOpenGroundConfigStatus,
+ getActiveProviders,
+} from '@/lib/platform/providers/config';
+import { dataCollector } from '@/lib/platform/common';
+import getMessage from '@/constants/messages';
+
+const Sanitizer = require('@/utils/sanitizer').default;
+
+const mockConfig = {
+ id: 'cfg-1',
+ userId: 'user-1',
+ provider: 'openai',
+ vaultId: 'vault-1',
+ modelId: 'gpt-4o',
+ isActive: true,
+ createdAt: new Date('2026-01-01'),
+ updatedAt: new Date('2026-01-01'),
+};
+
+const mockVaultRecord = { key: 'OPENAI_API_KEY', value: 'sk-xxx' };
+
+beforeEach(() => {
+ jest.resetAllMocks();
+ Sanitizer.sanitizeValue.mockImplementation((v: string) => v);
+ (getMessage as jest.Mock).mockReturnValue({
+ OPERATION_FAILED: 'Operation failed',
+ });
+});
+
+describe('getOpenGroundConfigs', () => {
+ it('returns configs for the user', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [mockConfig] });
+
+ const result = await getOpenGroundConfigs('user-1', 'db-1');
+
+ expect(result.data).toEqual([mockConfig]);
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining("user_id = 'user-1'"),
+ }),
+ 'query',
+ 'db-1'
+ );
+ });
+
+ it('returns error on DB failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'DB error' });
+
+ const result = await getOpenGroundConfigs('user-1', 'db-1');
+
+ expect(result.err).toBe('Operation failed');
+ });
+
+ it('catches thrown exceptions', async () => {
+ (dataCollector as jest.Mock).mockRejectedValue(new Error('boom'));
+
+ const result = await getOpenGroundConfigs('user-1', 'db-1');
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
+
+describe('getOpenGroundConfigWithSecret', () => {
+ it('returns config + apiKey from vault', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [mockConfig] }) // config query
+ .mockResolvedValueOnce({ data: [mockVaultRecord] }); // vault query
+
+ const result = await getOpenGroundConfigWithSecret('openai', 'user-1', 'db-1');
+
+ expect(result.data).toBeDefined();
+ expect(result.data!.apiKey).toBe('sk-xxx');
+ expect(result.data!.vaultKey).toBe('OPENAI_API_KEY');
+ });
+
+ it('returns error when no active config found', async () => {
+ (dataCollector as jest.Mock).mockResolvedValueOnce({ data: [] });
+
+ const result = await getOpenGroundConfigWithSecret('openai', 'user-1', 'db-1');
+
+ expect(result.err).toContain('No active configuration');
+ });
+
+ it('returns error when vault lookup fails', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [mockConfig] })
+ .mockResolvedValueOnce({ data: [] });
+
+ const result = await getOpenGroundConfigWithSecret('openai', 'user-1', 'db-1');
+
+ expect(result.err).toContain('API key not found');
+ });
+
+ it('catches thrown exceptions', async () => {
+ (dataCollector as jest.Mock).mockRejectedValue(new Error('boom'));
+
+ const result = await getOpenGroundConfigWithSecret('openai', 'user-1', 'db-1');
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
+
+describe('upsertOpenGroundConfig', () => {
+ it('inserts new config when none exists', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [] }) // exists check — none
+ .mockResolvedValueOnce({ err: null }) // insert
+ .mockResolvedValueOnce({ data: [mockConfig] }); // re-fetch
+
+ const result = await upsertOpenGroundConfig({
+ provider: 'openai',
+ vaultId: 'vault-1',
+ modelId: 'gpt-4o',
+ userId: 'user-1',
+ databaseConfigId: 'db-1',
+ });
+
+ expect(result.data).toEqual(mockConfig);
+ // Second call should be the INSERT
+ expect(dataCollector).toHaveBeenNthCalledWith(
+ 2,
+ expect.objectContaining({
+ table: 'openlit_providers',
+ values: [
+ expect.objectContaining({
+ user_id: 'user-1',
+ provider: 'openai',
+ }),
+ ],
+ }),
+ 'insert',
+ 'db-1'
+ );
+ });
+
+ it('updates existing config', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [{ id: 'cfg-1' }] }) // exists check
+ .mockResolvedValueOnce({ err: null }) // update
+ .mockResolvedValueOnce({ data: [mockConfig] }); // getOpenGroundConfigById
+
+ const result = await upsertOpenGroundConfig({
+ provider: 'openai',
+ vaultId: 'vault-2',
+ userId: 'user-1',
+ databaseConfigId: 'db-1',
+ });
+
+ expect(result.data).toBeDefined();
+ expect(dataCollector).toHaveBeenNthCalledWith(
+ 2,
+ expect.objectContaining({
+ query: expect.stringContaining('ALTER TABLE'),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('returns error when insert fails', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [] })
+ .mockResolvedValueOnce({ err: 'insert failed' });
+
+ const result = await upsertOpenGroundConfig({
+ provider: 'openai',
+ vaultId: 'v',
+ userId: 'user-1',
+ databaseConfigId: 'db-1',
+ });
+
+ expect(result.err).toBe('Operation failed');
+ });
+
+ it('returns error when update fails', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ data: [{ id: 'cfg-1' }] })
+ .mockResolvedValueOnce({ err: 'update failed' });
+
+ const result = await upsertOpenGroundConfig({
+ provider: 'openai',
+ vaultId: 'v',
+ userId: 'user-1',
+ databaseConfigId: 'db-1',
+ });
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
+
+describe('deleteOpenGroundConfig', () => {
+ it('deletes a config', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await deleteOpenGroundConfig('cfg-1', 'user-1', 'db-1');
+
+ expect(result.data).toContain('deleted');
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining('DELETE WHERE'),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('returns error on DB failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'DB error' });
+
+ const result = await deleteOpenGroundConfig('cfg-1', 'user-1', 'db-1');
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
+
+describe('toggleOpenGroundConfigStatus', () => {
+ it('toggles status and returns updated config', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ err: null }) // ALTER UPDATE
+ .mockResolvedValueOnce({ data: [mockConfig] }); // re-fetch
+
+ const result = await toggleOpenGroundConfigStatus('cfg-1', 'user-1', 'db-1', false);
+
+ expect(result.data).toBeDefined();
+ });
+
+ it('returns error on toggle failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'fail' });
+
+ const result = await toggleOpenGroundConfigStatus('cfg-1', 'user-1', 'db-1', true);
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
+
+describe('getActiveProviders', () => {
+ it('returns distinct active provider names', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({
+ data: [{ provider: 'openai' }, { provider: 'anthropic' }],
+ });
+
+ const result = await getActiveProviders('user-1', 'db-1');
+
+ expect(result.data).toEqual(['openai', 'anthropic']);
+ });
+
+ it('returns empty array when no providers', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [] });
+
+ const result = await getActiveProviders('user-1', 'db-1');
+
+ expect(result.data).toEqual([]);
+ });
+
+ it('returns error on DB failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'fail' });
+
+ const result = await getActiveProviders('user-1', 'db-1');
+
+ expect(result.err).toBe('Operation failed');
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/default-models.test.ts b/src/client/src/__tests__/lib/platform/providers/default-models.test.ts
new file mode 100644
index 00000000..6566049f
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/default-models.test.ts
@@ -0,0 +1,50 @@
+import { DEFAULT_MODELS_BY_PROVIDER } from '@/lib/platform/providers/default-models';
+
+describe('default-models', () => {
+ it('exports a non-empty record of providers', () => {
+ expect(typeof DEFAULT_MODELS_BY_PROVIDER).toBe('object');
+ const providers = Object.keys(DEFAULT_MODELS_BY_PROVIDER);
+ expect(providers.length).toBeGreaterThan(0);
+ });
+
+ it('includes the expected core providers', () => {
+ const expectedProviders = [
+ 'openai', 'anthropic', 'google', 'mistral', 'groq',
+ 'perplexity', 'azure', 'cohere', 'together', 'fireworks',
+ 'deepseek', 'xai', 'huggingface', 'replicate',
+ ];
+ for (const provider of expectedProviders) {
+ expect(DEFAULT_MODELS_BY_PROVIDER).toHaveProperty(provider);
+ }
+ });
+
+ it('each provider has at least one model', () => {
+ for (const [provider, models] of Object.entries(DEFAULT_MODELS_BY_PROVIDER)) {
+ expect(models.length).toBeGreaterThan(0);
+ }
+ });
+
+ it('each model has the required fields', () => {
+ for (const [provider, models] of Object.entries(DEFAULT_MODELS_BY_PROVIDER)) {
+ for (const model of models) {
+ expect(model).toHaveProperty('id');
+ expect(model).toHaveProperty('displayName');
+ expect(model).toHaveProperty('contextWindow');
+ expect(typeof model.contextWindow).toBe('number');
+ expect(model).toHaveProperty('inputPricePerMToken');
+ expect(typeof model.inputPricePerMToken).toBe('number');
+ expect(model).toHaveProperty('outputPricePerMToken');
+ expect(typeof model.outputPricePerMToken).toBe('number');
+ }
+ }
+ });
+
+ it('model IDs are non-empty strings', () => {
+ for (const models of Object.values(DEFAULT_MODELS_BY_PROVIDER)) {
+ for (const model of models) {
+ expect(typeof model.id).toBe('string');
+ expect(model.id.length).toBeGreaterThan(0);
+ }
+ }
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/models-service.test.ts b/src/client/src/__tests__/lib/platform/providers/models-service.test.ts
new file mode 100644
index 00000000..7438a661
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/models-service.test.ts
@@ -0,0 +1,283 @@
+jest.mock('@/lib/platform/common', () => ({
+ dataCollector: jest.fn(),
+}));
+jest.mock('@/utils/sanitizer', () => ({
+ __esModule: true,
+ default: { sanitizeValue: jest.fn((v: string) => v) },
+}));
+
+import {
+ getCustomModels,
+ getCustomModelsGroupedByProvider,
+ createCustomModel,
+ updateCustomModel,
+ deleteCustomModel,
+ getCustomModelsForProvider,
+} from '@/lib/platform/providers/models-service';
+import { dataCollector } from '@/lib/platform/common';
+
+const mockModel = {
+ id: 'uuid-1',
+ model_id: 'gpt-4o-custom',
+ provider: 'openai',
+ displayName: 'GPT-4o Custom',
+ modelType: 'chat',
+ contextWindow: 128000,
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ capabilities: ['streaming'],
+ isDefault: false,
+};
+
+const Sanitizer = require('@/utils/sanitizer').default;
+
+beforeEach(() => {
+ jest.resetAllMocks();
+ // Re-apply sanitizer mock after resetAllMocks
+ Sanitizer.sanitizeValue.mockImplementation((v: string) => v);
+});
+
+describe('getCustomModels', () => {
+ it('returns all models when no provider filter', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [mockModel] });
+
+ const result = await getCustomModels('user-1', 'db-1');
+
+ expect(result.data).toHaveLength(1);
+ expect(result.data![0].model_id).toBe('gpt-4o-custom');
+ // Query should NOT have a WHERE clause for provider
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.not.stringContaining("provider = "),
+ }),
+ 'query',
+ 'db-1'
+ );
+ });
+
+ it('filters by provider when specified', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [mockModel] });
+
+ await getCustomModels('user-1', 'db-1', 'openai');
+
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining("provider = 'openai'"),
+ }),
+ 'query',
+ 'db-1'
+ );
+ });
+
+ it('returns error on DB failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'DB error' });
+
+ const result = await getCustomModels('user-1', 'db-1');
+ expect(result.err).toBe('DB error');
+ });
+});
+
+describe('getCustomModelsGroupedByProvider', () => {
+ it('groups models by provider', async () => {
+ const models = [
+ { ...mockModel, provider: 'openai' },
+ { ...mockModel, id: 'uuid-2', provider: 'anthropic', model_id: 'claude-3' },
+ ];
+ (dataCollector as jest.Mock).mockResolvedValue({ data: models });
+
+ const result = await getCustomModelsGroupedByProvider('user-1', 'db-1');
+
+ expect(result.data).toBeDefined();
+ expect(Object.keys(result.data!)).toContain('openai');
+ expect(Object.keys(result.data!)).toContain('anthropic');
+ expect(result.data!['openai']).toHaveLength(1);
+ });
+
+ it('returns error when underlying getCustomModels fails', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'fail' });
+
+ const result = await getCustomModelsGroupedByProvider('user-1', 'db-1');
+
+ expect(result.err).toBe('fail');
+ });
+
+ it('handles empty data', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [] });
+
+ const result = await getCustomModelsGroupedByProvider('user-1', 'db-1');
+
+ expect(result.data).toEqual({});
+ });
+});
+
+describe('createCustomModel', () => {
+ it('validates required fields', async () => {
+ const result = await createCustomModel('user-1', 'db-1', {
+ provider: '',
+ model_id: '',
+ displayName: '',
+ });
+ expect(result.err).toContain('required');
+ expect(dataCollector).not.toHaveBeenCalled();
+ });
+
+ it('inserts and returns the new model', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ err: null }) // insert
+ .mockResolvedValueOnce({ data: [mockModel] }); // select
+
+ const result = await createCustomModel('user-1', 'db-1', {
+ provider: 'openai',
+ model_id: 'gpt-4o-custom',
+ displayName: 'GPT-4o Custom',
+ });
+
+ expect(result.data).toBeDefined();
+ expect(result.data!.model_id).toBe('gpt-4o-custom');
+ // INSERT call should use 'insert' type
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({ table: 'openlit_provider_models' }),
+ 'insert',
+ 'db-1'
+ );
+ });
+
+ it('returns error when insert fails', async () => {
+ (dataCollector as jest.Mock).mockResolvedValueOnce({ err: 'insert failed' });
+
+ const result = await createCustomModel('user-1', 'db-1', {
+ provider: 'openai',
+ model_id: 'm',
+ displayName: 'M',
+ });
+
+ expect(result.err).toBe('insert failed');
+ });
+
+ it('returns error when post-insert select fails', async () => {
+ (dataCollector as jest.Mock)
+ .mockResolvedValueOnce({ err: null }) // insert succeeds
+ .mockResolvedValueOnce({ err: 'select failed' });
+
+ const result = await createCustomModel('user-1', 'db-1', {
+ provider: 'openai',
+ model_id: 'm',
+ displayName: 'M',
+ });
+
+ expect(result.err).toBe('select failed');
+ });
+});
+
+describe('updateCustomModel', () => {
+ it('builds an ALTER TABLE UPDATE query', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await updateCustomModel('user-1', 'db-1', 'gpt-4o-custom', {
+ displayName: 'Renamed Model',
+ contextWindow: 256000,
+ });
+
+ expect(result.data).toBe(true);
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining('ALTER TABLE'),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('still runs update when only updated_at changes (empty input)', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await updateCustomModel('user-1', 'db-1', 'id', {});
+ // updated_at = now() is always appended, so the query runs even with empty input
+ expect(result.data).toBe(true);
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining('updated_at = now()'),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('updates all optional fields together', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await updateCustomModel('user-1', 'db-1', 'id', {
+ displayName: 'New',
+ modelType: 'chat',
+ contextWindow: 8000,
+ inputPricePerMToken: 1.5,
+ outputPricePerMToken: 3.0,
+ capabilities: ['streaming', 'vision'],
+ });
+
+ expect(result.data).toBe(true);
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringMatching(
+ /display_name.*model_type.*context_window.*capabilities/s
+ ),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('returns error when ALTER UPDATE fails', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'alter failed' });
+
+ const result = await updateCustomModel('user-1', 'db-1', 'id', {
+ displayName: 'X',
+ });
+
+ expect(result.err).toBe('alter failed');
+ });
+});
+
+describe('deleteCustomModel', () => {
+ it('deletes by model id', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: null });
+
+ const result = await deleteCustomModel('user-1', 'db-1', 'gpt-4o-custom');
+ expect(result.data).toBe(true);
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({
+ query: expect.stringContaining('DELETE FROM'),
+ }),
+ 'exec',
+ 'db-1'
+ );
+ });
+
+ it('returns error on DELETE failure', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'delete failed' });
+
+ const result = await deleteCustomModel('user-1', 'db-1', 'id');
+
+ expect(result.err).toBe('delete failed');
+ });
+});
+
+describe('getCustomModelsForProvider', () => {
+ it('returns models in ModelMetadata format', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [mockModel] });
+
+ const result = await getCustomModelsForProvider('user-1', 'db-1', 'openai');
+
+ expect(result.data).toHaveLength(1);
+ expect(result.data![0].id).toBe('gpt-4o-custom');
+ expect(result.data![0]).not.toHaveProperty('model_id');
+ });
+
+ it('returns error when underlying getCustomModels fails', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ err: 'fail' });
+
+ const result = await getCustomModelsForProvider('user-1', 'db-1', 'openai');
+
+ expect(result.err).toBe('fail');
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/provider-registry.test.ts b/src/client/src/__tests__/lib/platform/providers/provider-registry.test.ts
new file mode 100644
index 00000000..5bcd4257
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/provider-registry.test.ts
@@ -0,0 +1,157 @@
+jest.mock('@/lib/platform/common', () => ({
+ dataCollector: jest.fn(),
+}));
+jest.mock('@/utils/sanitizer', () => ({
+ __esModule: true,
+ default: { sanitizeValue: jest.fn((v: string) => v) },
+}));
+
+import { ProviderRegistry } from '@/lib/platform/providers/provider-registry';
+import { dataCollector } from '@/lib/platform/common';
+
+const mockModels = [
+ {
+ provider: 'openai',
+ id: 'gpt-4o',
+ displayName: 'GPT-4o',
+ modelType: 'chat',
+ contextWindow: 128000,
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ capabilities: ['streaming'],
+ },
+ {
+ provider: 'anthropic',
+ id: 'claude-3-5-sonnet-20240620',
+ displayName: 'Claude 3.5 Sonnet',
+ modelType: 'chat',
+ contextWindow: 200000,
+ inputPricePerMToken: 3.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ['streaming', 'vision'],
+ },
+];
+
+beforeEach(() => {
+ jest.resetAllMocks();
+});
+
+describe('ProviderRegistry', () => {
+ describe('getProviderMetadata', () => {
+ it('returns static metadata for a known provider', () => {
+ const meta = ProviderRegistry.getProviderMetadata('openai');
+ expect(meta).not.toBeNull();
+ expect(meta!.providerId).toBe('openai');
+ expect(meta!.displayName).toBe('OpenAI');
+ expect(meta).toHaveProperty('configSchema');
+ expect(meta).toHaveProperty('requiresVault');
+ });
+
+ it('returns null for an unknown provider', () => {
+ expect(ProviderRegistry.getProviderMetadata('nonexistent')).toBeNull();
+ });
+ });
+
+ describe('getAllProviderMetadata', () => {
+ it('returns all static providers', () => {
+ const all = ProviderRegistry.getAllProviderMetadata();
+ expect(all.length).toBeGreaterThan(10);
+ const ids = all.map((p) => p.providerId);
+ expect(ids).toContain('openai');
+ expect(ids).toContain('anthropic');
+ expect(ids).toContain('google');
+ });
+ });
+
+ describe('getAvailableProviders', () => {
+ it('returns providers with models from DB', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: mockModels });
+
+ const providers = await ProviderRegistry.getAvailableProviders('db-1');
+
+ expect(dataCollector).toHaveBeenCalledWith(
+ expect.objectContaining({ query: expect.stringContaining('openlit_provider_models') }),
+ 'query',
+ 'db-1'
+ );
+
+ const openai = providers.find((p) => p.providerId === 'openai');
+ expect(openai).toBeDefined();
+ expect(openai!.supportedModels.length).toBeGreaterThanOrEqual(1);
+ expect(openai!.supportedModels[0].id).toBe('gpt-4o');
+ });
+
+ it('returns providers with empty models when DB returns no data', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [] });
+
+ const providers = await ProviderRegistry.getAvailableProviders('db-1');
+ const openai = providers.find((p) => p.providerId === 'openai');
+ expect(openai).toBeDefined();
+ expect(openai!.supportedModels).toEqual([]);
+ });
+
+ it('returns providers with empty models on DB error', async () => {
+ (dataCollector as jest.Mock).mockRejectedValue(new Error('DB down'));
+
+ const providers = await ProviderRegistry.getAvailableProviders('db-1');
+ expect(providers.length).toBeGreaterThan(0);
+ providers.forEach((p) => expect(p.supportedModels).toEqual([]));
+ });
+ });
+
+ describe('getProviderById', () => {
+ it('returns the provider with models when found', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({
+ data: [mockModels[0]],
+ });
+
+ const provider = await ProviderRegistry.getProviderById('openai', 'db-1');
+ expect(provider).not.toBeNull();
+ expect(provider!.providerId).toBe('openai');
+ expect(provider!.supportedModels).toHaveLength(1);
+ });
+
+ it('returns null for an unknown provider', async () => {
+ const provider = await ProviderRegistry.getProviderById('nonexistent', 'db-1');
+ expect(provider).toBeNull();
+ expect(dataCollector).not.toHaveBeenCalled();
+ });
+ });
+
+ describe('getModel', () => {
+ it('returns a specific model', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({
+ data: [mockModels[0]],
+ });
+
+ const model = await ProviderRegistry.getModel('openai', 'gpt-4o', 'db-1');
+ expect(model).not.toBeNull();
+ expect(model!.id).toBe('gpt-4o');
+ expect(model!.inputPricePerMToken).toBe(2.5);
+ });
+
+ it('returns null when model not found', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [] });
+
+ const model = await ProviderRegistry.getModel('openai', 'gpt-99', 'db-1');
+ expect(model).toBeNull();
+ });
+ });
+
+ describe('searchProviders', () => {
+ it('filters providers by name', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: mockModels });
+
+ const results = await ProviderRegistry.searchProviders('groq', 'db-1');
+ expect(results.length).toBe(1);
+ expect(results[0].providerId).toBe('groq');
+ });
+
+ it('returns empty array when nothing matches', async () => {
+ (dataCollector as jest.Mock).mockResolvedValue({ data: [] });
+
+ const results = await ProviderRegistry.searchProviders('zzzzz', 'db-1');
+ expect(results).toEqual([]);
+ });
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/provider-service.test.ts b/src/client/src/__tests__/lib/platform/providers/provider-service.test.ts
new file mode 100644
index 00000000..da773fb9
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/provider-service.test.ts
@@ -0,0 +1,104 @@
+jest.mock('@/lib/platform/providers/provider-registry', () => ({
+ ProviderRegistry: {
+ getAvailableProviders: jest.fn(),
+ getProviderById: jest.fn(),
+ searchProviders: jest.fn(),
+ },
+}));
+
+import {
+ getAllProvidersWithCustomModels,
+ getProviderByIdWithCustomModels,
+ searchProvidersWithCustomModels,
+} from '@/lib/platform/providers/provider-service';
+import { ProviderRegistry } from '@/lib/platform/providers/provider-registry';
+
+const mockProviders = [
+ {
+ providerId: 'openai',
+ displayName: 'OpenAI',
+ description: 'GPT models',
+ requiresVault: true,
+ configSchema: {},
+ supportedModels: [{ id: 'gpt-4o', displayName: 'GPT-4o' }],
+ },
+];
+
+beforeEach(() => {
+ jest.resetAllMocks();
+});
+
+describe('getAllProvidersWithCustomModels', () => {
+ it('returns providers from registry', async () => {
+ (ProviderRegistry.getAvailableProviders as jest.Mock).mockResolvedValue(
+ mockProviders
+ );
+
+ const result = await getAllProvidersWithCustomModels('user-1', 'db-1');
+
+ expect(result.data).toEqual(mockProviders);
+ expect(ProviderRegistry.getAvailableProviders).toHaveBeenCalledWith('db-1');
+ });
+
+ it('returns error on registry failure', async () => {
+ (ProviderRegistry.getAvailableProviders as jest.Mock).mockRejectedValue(
+ new Error('boom')
+ );
+
+ const result = await getAllProvidersWithCustomModels('user-1', 'db-1');
+
+ expect(result.err).toBe('Failed to load providers');
+ });
+});
+
+describe('getProviderByIdWithCustomModels', () => {
+ it('returns the provider when found', async () => {
+ (ProviderRegistry.getProviderById as jest.Mock).mockResolvedValue(
+ mockProviders[0]
+ );
+
+ const result = await getProviderByIdWithCustomModels('openai', 'user-1', 'db-1');
+
+ expect(result.data).toEqual(mockProviders[0]);
+ expect(ProviderRegistry.getProviderById).toHaveBeenCalledWith('openai', 'db-1');
+ });
+
+ it('returns error when provider not found', async () => {
+ (ProviderRegistry.getProviderById as jest.Mock).mockResolvedValue(null);
+
+ const result = await getProviderByIdWithCustomModels('unknown', 'user-1', 'db-1');
+
+ expect(result.err).toBe('Provider not found');
+ });
+
+ it('returns error on registry failure', async () => {
+ (ProviderRegistry.getProviderById as jest.Mock).mockRejectedValue(
+ new Error('boom')
+ );
+
+ const result = await getProviderByIdWithCustomModels('openai', 'user-1', 'db-1');
+
+ expect(result.err).toBe('Failed to load provider');
+ });
+});
+
+describe('searchProvidersWithCustomModels', () => {
+ it('delegates to registry searchProviders', async () => {
+ (ProviderRegistry.searchProviders as jest.Mock).mockResolvedValue(mockProviders);
+
+ const result = await searchProvidersWithCustomModels('openai', 'user-1', 'db-1');
+
+ expect(result.data).toEqual(mockProviders);
+ expect(ProviderRegistry.searchProviders).toHaveBeenCalledWith('openai', 'db-1');
+ });
+
+ it('returns error on registry failure', async () => {
+ (ProviderRegistry.searchProviders as jest.Mock).mockRejectedValue(
+ new Error('boom')
+ );
+
+ const result = await searchProvidersWithCustomModels('q', 'user-1', 'db-1');
+
+ expect(result.err).toBe('Failed to search providers');
+ });
+});
diff --git a/src/client/src/__tests__/lib/platform/providers/table-details.test.ts b/src/client/src/__tests__/lib/platform/providers/table-details.test.ts
new file mode 100644
index 00000000..fd009156
--- /dev/null
+++ b/src/client/src/__tests__/lib/platform/providers/table-details.test.ts
@@ -0,0 +1,14 @@
+import {
+ OPENLIT_PROVIDERS_TABLE_NAME,
+ OPENLIT_PROVIDER_MODELS_TABLE_NAME,
+} from '@/lib/platform/providers/table-details';
+
+describe('providers table-details', () => {
+ it('exports the correct providers table name', () => {
+ expect(OPENLIT_PROVIDERS_TABLE_NAME).toBe('openlit_providers');
+ });
+
+ it('exports the correct provider models table name', () => {
+ expect(OPENLIT_PROVIDER_MODELS_TABLE_NAME).toBe('openlit_provider_models');
+ });
+});
diff --git a/src/client/src/app/(playground)/evaluations/settings/page.tsx b/src/client/src/app/(playground)/evaluations/settings/page.tsx
index c07b8621..305b1965 100644
--- a/src/client/src/app/(playground)/evaluations/settings/page.tsx
+++ b/src/client/src/app/(playground)/evaluations/settings/page.tsx
@@ -218,7 +218,7 @@ export default function EvaluationSettingsPage() {
<div className="grid gap-3">
<div className="flex items-center justify-between">
<Label>{getMessage().EVALUATION_PROVIDER_LABEL}</Label>
- <Link href="/settings/manage-models">
+ <Link href="/manage-models">
<Button variant="ghost" size="sm" className="h-7 text-xs px-2">
<SettingsIcon className="h-3 w-3 mr-1" />
{getMessage().OPENGROUND_MANAGE_MODELS}
diff --git a/src/client/src/app/(playground)/manage-models/page.tsx b/src/client/src/app/(playground)/manage-models/page.tsx
new file mode 100644
index 00000000..76bae54b
--- /dev/null
+++ b/src/client/src/app/(playground)/manage-models/page.tsx
@@ -0,0 +1,351 @@
+"use client";
+
+import { useState, useEffect, useMemo } from "react";
+import { usePostHog } from "posthog-js/react";
+import { CLIENT_EVENTS } from "@/constants/events";
+import { Button } from "@/components/ui/button";
+import {
+ SettingsIcon,
+ Download,
+ Upload,
+ Copy,
+ Link as LinkIcon,
+ Code2,
+} from "lucide-react";
+import getMessage from "@/constants/messages";
+import ModelListSidebar from "@/components/(playground)/openground/model-list-sidebar";
+import ModelEditorPanel from "@/components/(playground)/openground/model-editor-panel";
+import SdkUsageDialog from "@/components/(playground)/openground/sdk-usage-dialog";
+import { ProviderMetadata, ModelMetadata } from "@/types/openground";
+import useFetchWrapper from "@/utils/hooks/useFetchWrapper";
+import {
+ Dialog,
+ DialogContent,
+ DialogDescription,
+ DialogFooter,
+ DialogHeader,
+ DialogTitle,
+} from "@/components/ui/dialog";
+import { Textarea } from "@/components/ui/textarea";
+import { toast } from "sonner";
+import { useRootStore } from "@/store";
+import { getDatabaseConfigList } from "@/selectors/database-config";
+import { Input } from "@/components/ui/input";
+import copy from "copy-to-clipboard";
+import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
+
+interface CustomModel extends ModelMetadata {
+ id: string;
+ customId?: string;
+ model_id: string;
+ provider?: string;
+ modelType?: string;
+}
+
+export default function ManageModelsPage() {
+ const m = getMessage();
+ const posthog = usePostHog();
+ const [selectedProvider, setSelectedProvider] = useState<string | null>(null);
+ const [selectedModel, setSelectedModel] = useState<ModelMetadata | null>(null);
+ const [isCustomModel, setIsCustomModel] = useState(false);
+ const [isAddingNew, setIsAddingNew] = useState(false);
+ const [showImport, setShowImport] = useState(false);
+ const [showSdkUsage, setShowSdkUsage] = useState(false);
+ const [importJson, setImportJson] = useState("");
+
+ // Pull the active database config from the zustand store
+ const databaseConfigList = useRootStore(getDatabaseConfigList) || [];
+ const activeDbConfig = useMemo(
+ () => databaseConfigList.find((item) => !!item.isCurrent),
+ [databaseConfigList]
+ );
+ const dbConfigId = activeDbConfig?.id || "";
+
+ const {
+ data: providers,
+ fireRequest: fetchProviders,
+ isLoading: loadingProviders,
+ } = useFetchWrapper<ProviderMetadata[]>();
+ const {
+ data: customModels,
+ fireRequest: fetchCustomModels,
+ isLoading: loadingCustom,
+ } = useFetchWrapper<Record<string, CustomModel[]>>();
+ const { fireRequest: fireImport, isLoading: importing } = useFetchWrapper();
+
+ useEffect(() => {
+ posthog?.capture(CLIENT_EVENTS.SETTINGS_MANAGE_MODELS_PAGE_VISITED);
+ }, []);
+
+ useEffect(() => {
+ loadProviders();
+ loadAllCustomModels();
+ }, []);
+
+ const loadProviders = () => {
+ fetchProviders({ requestType: "GET", url: "/api/openground/providers" });
+ };
+
+ const loadAllCustomModels = () => {
+ fetchCustomModels({ requestType: "GET", url: "/api/openground/models" });
+ };
+
+ const handleSelectModel = (
+ model: ModelMetadata,
+ provider: string,
+ _isCustom: boolean
+ ) => {
+ setSelectedModel(model);
+ setSelectedProvider(provider);
+ setIsCustomModel(true);
+ setIsAddingNew(false);
+ };
+
+ const handleAddNew = (provider: string) => {
+ setSelectedProvider(provider);
+ setSelectedModel(null);
+ setIsCustomModel(true);
+ setIsAddingNew(true);
+ };
+
+ const handleModelSaved = () => {
+ loadAllCustomModels();
+ setIsAddingNew(false);
+ };
+
+ const handleModelDeleted = () => {
+ loadAllCustomModels();
+ setSelectedModel(null);
+ setSelectedProvider(null);
+ setIsCustomModel(false);
+ };
+
+ // Build the public pricing URL from the active db config
+ const pricingUrl = dbConfigId
+ ? `${typeof window !== "undefined" ? window.location.origin : ""}/api/pricing/export/${dbConfigId}`
+ : "";
+
+ const copyUrl = () => {
+ if (!pricingUrl) return;
+ copy(pricingUrl);
+ toast.success(m.MANAGE_MODELS_PRICING_URL_COPIED, { id: "url-copy" });
+ };
+
+ const handleImport = () => {
+ let parsed: any;
+ try {
+ parsed = JSON.parse(importJson);
+ } catch {
+ toast.error(m.MANAGE_MODELS_INVALID_JSON, { id: "import" });
+ return;
+ }
+
+ fireImport({
+ requestType: "POST",
+ url: "/api/openground/models/import",
+ body: JSON.stringify(parsed),
+ successCb: (res: any) => {
+ toast.success(
+ `${m.MANAGE_MODELS_IMPORT_SUCCESS}: ${res?.imported ?? 0} imported, ${res?.skipped ?? 0} skipped`,
+ { id: "import" }
+ );
+ setShowImport(false);
+ setImportJson("");
+ loadAllCustomModels();
+ loadProviders();
+ },
+ failureCb: (err?: string) => {
+ toast.error(err || m.MANAGE_MODELS_IMPORT_FAILED, { id: "import" });
+ },
+ });
+ };
+
+ return (
+ <div className="flex flex-col h-full w-full overflow-hidden">
+ {/* Header */}
+ <div className="flex items-center justify-between p-4 border-b border-stone-200 dark:border-stone-800 shrink-0">
+ <div className="flex flex-col gap-2">
+ <h1 className="text-lg font-semibold text-stone-900 dark:text-stone-100">
+ {m.OPENGROUND_MANAGE_MODELS}
+ </h1>
+ <p className="text-sm text-stone-500 dark:text-stone-400 mt-0.5">
+ {m.OPENGROUND_MANAGE_MODELS_DESCRIPTION}
+ </p>
+ </div>
+ <div className="flex items-center gap-2">
+ {pricingUrl && (
+
+ <Tooltip>
+ <TooltipTrigger>
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-2"
+ onClick={() => setShowSdkUsage(true)}
+ >
+ <Code2 className="h-4 w-4" />
+ </Button>
+ </TooltipTrigger>
+ <TooltipContent>
+ {m.MANAGE_MODELS_SDK_USAGE}
+ </TooltipContent>
+ </Tooltip>
+ )}
+ <Tooltip>
+ <TooltipTrigger>
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-2"
+ onClick={() => setShowImport(true)}
+ >
+ <Upload className="h-4 w-4" />
+ </Button>
+ </TooltipTrigger>
+ <TooltipContent>
+ {m.MANAGE_MODELS_IMPORT_PRICING}
+ </TooltipContent>
+ </Tooltip>
+ <Tooltip>
+ <TooltipTrigger>
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-2"
+ onClick={() => {
+ window.open("/api/openground/models/export", "_blank");
+ }}
+ >
+ <Download className="h-4 w-4" />
+ </Button>
+ </TooltipTrigger>
+ <TooltipContent>
+ {m.MANAGE_MODELS_EXPORT_PRICING}
+ </TooltipContent>
+ </Tooltip>
+ </div>
+ </div>
+
+ {/* SDK Pricing URL bar */}
+ {pricingUrl && (
+ <div className="flex items-center gap-3 px-4 py-2.5 bg-stone-50 dark:bg-stone-950 border-b border-stone-200 dark:border-stone-800 shrink-0">
+ <LinkIcon className="h-3.5 w-3.5 text-stone-400 shrink-0" />
+ <span className="text-xs font-medium text-stone-500 dark:text-stone-400 shrink-0">
+ {m.MANAGE_MODELS_PRICING_URL_LABEL}
+ </span>
+ <Input
+ readOnly
+ value={pricingUrl}
+ onFocus={(e) => e.target.select()}
+ className="font-mono text-xs h-8 flex-1 min-w-0"
+ />
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-1.5 shrink-0 h-8"
+ onClick={copyUrl}
+ >
+ <Copy className="h-3.5 w-3.5" />
+ {m.COPY}
+ </Button>
+ </div>
+ )}
+
+ {/* Main Content - Split Layout */}
+ <div className="flex flex-1 overflow-hidden min-h-0">
+ <ModelListSidebar
+ providers={providers || []}
+ customModels={customModels || {}}
+ loading={loadingProviders || loadingCustom}
+ selectedModel={selectedModel}
+ selectedProvider={selectedProvider}
+ selectedIsCustom={isCustomModel}
+ onSelectModel={handleSelectModel}
+ onAddNew={handleAddNew}
+ />
+
+ <div className="flex-1 overflow-auto bg-stone-50 dark:bg-stone-950">
+ {selectedModel || isAddingNew ? (
+ <ModelEditorPanel
+ model={selectedModel}
+ provider={selectedProvider}
+ isCustomModel={isCustomModel}
+ isAddingNew={isAddingNew}
+ onSave={handleModelSaved}
+ onDelete={handleModelDeleted}
+ onCancel={() => {
+ setSelectedModel(null);
+ setSelectedProvider(null);
+ setIsAddingNew(false);
+ }}
+ />
+ ) : (
+ <div className="flex flex-col items-center justify-center h-full p-8 text-center">
+ <SettingsIcon className="h-16 w-16 text-stone-300 dark:text-stone-700 mb-4" />
+ <h3 className="text-lg font-medium text-stone-900 dark:text-stone-100 mb-2">
+ {m.OPENGROUND_SELECT_MODEL_TO_VIEW}
+ </h3>
+ <p className="text-sm text-stone-500 dark:text-stone-400 max-w-md">
+ {m.OPENGROUND_SELECT_MODEL_TO_VIEW_DESCRIPTION}
+ </p>
+ {pricingUrl && (
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-2 mt-6"
+ onClick={() => setShowSdkUsage(true)}
+ >
+ <Code2 className="h-4 w-4" />
+ {m.MANAGE_MODELS_SDK_USAGE}
+ </Button>
+ )}
+ </div>
+ )}
+ </div>
+ </div>
+
+ {/* Import Dialog */}
+ <Dialog open={showImport} onOpenChange={setShowImport}>
+ <DialogContent className="sm:max-w-lg">
+ <DialogHeader>
+ <DialogTitle>{m.MANAGE_MODELS_IMPORT_DIALOG_TITLE}</DialogTitle>
+ <DialogDescription>
+ {m.MANAGE_MODELS_IMPORT_DIALOG_DESCRIPTION}
+ </DialogDescription>
+ </DialogHeader>
+ <Textarea
+ placeholder={`{
+ "models": [
+ {
+ "provider": "openai",
+ "model_id": "gpt-4o-2024-08-06",
+ "displayName": "GPT-4o (Aug 2024)",
+ "inputPricePerMToken": 2.5,
+ "outputPricePerMToken": 10
+ }
+ ]
+}`}
+ value={importJson}
+ onChange={(e) => setImportJson(e.target.value)}
+ className="min-h-[200px] text-xs dark:text-stone-100 text-stone-800"
+ />
+ <DialogFooter>
+ <Button variant="outline" onClick={() => setShowImport(false)}>
+ {m.CANCEL}
+ </Button>
+ <Button onClick={handleImport} disabled={importing || !importJson.trim()}>
+ {importing ? m.SAVING : m.MANAGE_MODELS_IMPORT_PRICING}
+ </Button>
+ </DialogFooter>
+ </DialogContent>
+ </Dialog>
+
+ {/* SDK Usage Dialog */}
+ <SdkUsageDialog
+ open={showSdkUsage}
+ onOpenChange={setShowSdkUsage}
+ pricingUrl={pricingUrl}
+ />
+ </div>
+ );
+}
diff --git a/src/client/src/app/(playground)/openground/models/page.tsx b/src/client/src/app/(playground)/openground/models/page.tsx
index f755e82a..d7e0757b 100644
--- a/src/client/src/app/(playground)/openground/models/page.tsx
+++ b/src/client/src/app/(playground)/openground/models/page.tsx
@@ -11,7 +11,7 @@ export default function OpengroundModelsPage() {
useEffect(() => {
posthog?.capture(CLIENT_EVENTS.OPENGROUND_MODELS_PAGE_VISITED);
- router.replace("/settings/manage-models");
+ router.replace("/manage-models");
}, []);
return null;
diff --git a/src/client/src/app/(playground)/pricing/page.tsx b/src/client/src/app/(playground)/pricing/page.tsx
new file mode 100644
index 00000000..5877712a
--- /dev/null
+++ b/src/client/src/app/(playground)/pricing/page.tsx
@@ -0,0 +1,231 @@
+"use client";
+
+import { Button } from "@/components/ui/button";
+import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
+import { Input } from "@/components/ui/input";
+import { Label } from "@/components/ui/label";
+import { Switch } from "@/components/ui/switch";
+import getMessage from "@/constants/messages";
+import useFetchWrapper from "@/utils/hooks/useFetchWrapper";
+import {
+ CircleDollarSign,
+ Clock,
+ Info,
+ Play,
+ Workflow,
+ Zap,
+} from "lucide-react";
+import Link from "next/link";
+import { useEffect, useState } from "react";
+import { toast } from "sonner";
+
+interface PricingConfig {
+ id?: string;
+ auto?: boolean;
+ recurringTime?: string;
+ meta?: string;
+}
+
+const PRICING_TOAST_ID = "pricing-config";
+
+export default function PricingPage() {
+ const m = getMessage();
+ const [autoPricing, setAutoPricing] = useState(false);
+ const [recurringTime, setRecurringTime] = useState("");
+
+ const {
+ fireRequest: getConfig,
+ data: config,
+ isLoading: isLoadingConfig,
+ } = useFetchWrapper<PricingConfig>();
+ const { fireRequest: saveConfig, isLoading: isSaving } = useFetchWrapper();
+
+ useEffect(() => {
+ getConfig({
+ requestType: "GET",
+ url: "/api/pricing/config",
+ responseDataKey: "data",
+ });
+ }, []);
+
+ useEffect(() => {
+ if (config) {
+ setAutoPricing(config.auto || false);
+ setRecurringTime(config.recurringTime || "");
+ }
+ }, [config]);
+
+ const handleSave = () => {
+ toast.loading(m.SAVING, { id: PRICING_TOAST_ID });
+
+ saveConfig({
+ body: JSON.stringify({
+ id: config?.id,
+ auto: autoPricing,
+ recurringTime: recurringTime || "* * * * *",
+ meta: config?.meta || "{}",
+ }),
+ requestType: "POST",
+ url: "/api/pricing/config",
+ responseDataKey: "data",
+ successCb: () => {
+ toast.success(m.PRICING_CONFIG_SAVED, { id: PRICING_TOAST_ID });
+ getConfig({
+ requestType: "GET",
+ url: "/api/pricing/config",
+ responseDataKey: "data",
+ });
+ },
+ failureCb: (err?: string) => {
+ toast.error(err || m.OPERATION_FAILED, {
+ id: PRICING_TOAST_ID,
+ });
+ },
+ });
+ };
+
+ if (isLoadingConfig && !config) {
+ return (
+ <div className="flex flex-1 h-full w-full p-6">
+ <div className="animate-pulse grid gap-4 w-full max-w-3xl">
+ <div className="h-40 bg-stone-100 dark:bg-stone-900 rounded-xl" />
+ </div>
+ </div>
+ );
+ }
+
+ return (
+ <div className="flex flex-col flex-1 h-full w-full p-6 overflow-auto gap-6">
+ {/* Page header */}
+ <div className="space-y-1">
+ <h1 className="text-lg font-semibold text-stone-900 dark:text-stone-100 flex items-center gap-2">
+ <CircleDollarSign className="size-5" />
+ {m.PRICING_TITLE}
+ </h1>
+ <p className="text-sm text-stone-500 dark:text-stone-400">
+ {m.PRICING_PAGE_DESCRIPTION}
+ </p>
+ </div>
+
+ {/* Single info bar (replaces the large left description card) */}
+ <Card className="border-blue-200 dark:border-blue-900/50 bg-blue-50/50 dark:bg-blue-950/20">
+ <CardContent className="py-3">
+ <div className="flex items-start gap-3">
+ <Info className="size-4 text-blue-600 dark:text-blue-500 shrink-0 mt-0.5" />
+ <p className="text-xs text-stone-600 dark:text-stone-400 leading-relaxed">
+ {m.PRICING_INFO_BAR}{" "}
+ <Link
+ href="/manage-models"
+ className="font-medium text-stone-700 dark:text-stone-300 underline"
+ >
+ {m.OPENGROUND_MANAGE_MODELS}
+ </Link>
+ .
+ </p>
+ </div>
+ </CardContent>
+ </Card>
+
+ {/* Auto + Manual side-by-side to save vertical space */}
+ <div className="grid grid-cols-1 lg:grid-cols-2 gap-4">
+ <Card className="border-stone-200 dark:border-stone-800 shadow-sm flex flex-col">
+ <CardHeader className="pb-4">
+ <CardTitle className="text-base flex items-center gap-2">
+ <Zap className="size-4" />
+ {m.PRICING_AUTO_TITLE}
+ </CardTitle>
+ <p className="text-sm text-stone-500 dark:text-stone-400 font-normal">
+ {m.PRICING_AUTO_DESCRIPTION}
+ </p>
+ </CardHeader>
+ <CardContent className="space-y-4 flex-1">
+ <div className="flex items-center justify-between rounded-lg border border-stone-200 dark:border-stone-700 p-4">
+ <div className="space-y-0.5">
+ <Label>{m.PRICING_AUTO_ENABLE_LABEL}</Label>
+ <p className="text-xs text-stone-500 dark:text-stone-400">
+ {m.PRICING_AUTO_ENABLE_HINT}
+ </p>
+ </div>
+ <Switch
+ checked={autoPricing}
+ onCheckedChange={setAutoPricing}
+ />
+ </div>
+ {autoPricing && (
+ <div className="grid gap-3">
+ <Label className="flex items-center gap-2">
+ <Clock className="size-3.5" />
+ {m.PRICING_AUTO_CRON_LABEL}
+ </Label>
+ <Input
+ placeholder={m.PRICING_AUTO_CRON_PLACEHOLDER}
+ value={recurringTime}
+ onChange={(e) => setRecurringTime(e.target.value)}
+ />
+ <p className="text-xs text-stone-500 dark:text-stone-400">
+ {m.PRICING_AUTO_CRON_HELP}
+ </p>
+ </div>
+ )}
+ <Button
+ onClick={handleSave}
+ disabled={isSaving}
+ className="bg-primary dark:bg-primary text-white dark:text-white hover:bg-primary/90 dark:hover:bg-primary/90"
+ >
+ {isSaving
+ ? m.SAVING
+ : config?.id
+ ? m.PRICING_UPDATE
+ : m.PRICING_SAVE}
+ </Button>
+ </CardContent>
+ </Card>
+
+ <Card className="border-stone-200 dark:border-stone-800 shadow-sm flex flex-col">
+ <CardHeader className="pb-4">
+ <CardTitle className="text-base flex items-center gap-2">
+ <Play className="size-4" />
+ {m.PRICING_MANUAL_TITLE}
+ </CardTitle>
+ <p className="text-sm text-stone-500 dark:text-stone-400 font-normal">
+ {m.PRICING_MANUAL_DESCRIPTION}
+ </p>
+ </CardHeader>
+ <CardContent className="space-y-4 flex-1">
+ <ol className="list-decimal list-inside space-y-2 text-sm text-stone-600 dark:text-stone-400">
+ <li>{m.PRICING_MANUAL_STEP_1}</li>
+ <li>{m.PRICING_MANUAL_STEP_2}</li>
+ <li>{m.PRICING_MANUAL_STEP_3}</li>
+ </ol>
+ <Link href="/requests" className="mt-3 inline-block">
+ <Button variant="default">{m.PRICING_GO_TO_REQUESTS}</Button>
+ </Link>
+ </CardContent>
+ </Card>
+ </div>
+
+ {/* How auto pricing works */}
+ <Card className="border-stone-200 dark:border-stone-800 shadow-sm">
+ <CardHeader className="pb-3">
+ <CardTitle className="text-base flex items-center gap-2">
+ <Workflow className="size-4" />
+ {m.PRICING_HOW_AUTO_WORKS_TITLE}
+ </CardTitle>
+ <p className="text-sm text-stone-500 dark:text-stone-400 font-normal">
+ {m.PRICING_HOW_AUTO_WORKS_DESCRIPTION}
+ </p>
+ </CardHeader>
+ <CardContent>
+ <ol className="list-decimal list-inside space-y-2 text-sm text-stone-600 dark:text-stone-400 marker:text-stone-400">
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_1}</li>
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_2}</li>
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_3}</li>
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_4}</li>
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_5}</li>
+ <li>{m.PRICING_HOW_AUTO_WORKS_STEP_6}</li>
+ </ol>
+ </CardContent>
+ </Card>
+ </div>
+ );
+}
diff --git a/src/client/src/app/(playground)/settings/layout.tsx b/src/client/src/app/(playground)/settings/layout.tsx
index d090c8a6..6e9099b8 100644
--- a/src/client/src/app/(playground)/settings/layout.tsx
+++ b/src/client/src/app/(playground)/settings/layout.tsx
@@ -26,7 +26,6 @@ export default function SettingsLayout({
path: "/settings/database-config",
},
{ value: "api-keys", label: "API Keys", path: "/settings/api-keys" },
- { value: "manage-models", label: "Manage Models", path: "/settings/manage-models" },
];
const currentTab =
@@ -47,7 +46,7 @@ export default function SettingsLayout({
className="w-full flex flex-col h-full"
>
<div className="flex items-center justify-between gap-4 mb-4">
- <TabsList className="p-0 h-[30px] grid grid-cols-5 self-start border border-stone-200 dark:border-stone-800">
+ <TabsList className="p-0 h-[30px] grid grid-cols-4 self-start border border-stone-200 dark:border-stone-800">
{tabs.map((tab) => (
<TabsTrigger key={tab.value} value={tab.value} className="py-1.5 text-xs rounded-md">
{tab.label}
diff --git a/src/client/src/app/(playground)/settings/manage-models/page.tsx b/src/client/src/app/(playground)/settings/manage-models/page.tsx
deleted file mode 100644
index af7d32d4..00000000
--- a/src/client/src/app/(playground)/settings/manage-models/page.tsx
+++ /dev/null
@@ -1,154 +0,0 @@
-"use client";
-
-import { useState, useEffect } from "react";
-import { usePostHog } from "posthog-js/react";
-import { CLIENT_EVENTS } from "@/constants/events";
-import { Card } from "@/components/ui/card";
-import { Button } from "@/components/ui/button";
-import { ArrowLeftIcon, SettingsIcon, Download } from "lucide-react";
-import { useRouter } from "next/navigation";
-import getMessage from "@/constants/messages";
-import ModelListSidebar from "@/components/(playground)/openground/model-list-sidebar";
-import ModelEditorPanel from "@/components/(playground)/openground/model-editor-panel";
-import { ProviderMetadata, ModelMetadata } from "@/types/openground";
-import useFetchWrapper from "@/utils/hooks/useFetchWrapper";
-
-interface CustomModel extends ModelMetadata {
- id: string;
- customId?: string;
- model_id: string;
- provider?: string;
- modelType?: string;
-}
-
-export default function SettingsManageModelsPage() {
- const posthog = usePostHog();
- const router = useRouter();
- const [selectedProvider, setSelectedProvider] = useState<string | null>(null);
- const [selectedModel, setSelectedModel] = useState<ModelMetadata | null>(null);
- const [isCustomModel, setIsCustomModel] = useState(false);
- const [isAddingNew, setIsAddingNew] = useState(false);
-
- const { data: providers, fireRequest: fetchProviders, isLoading: loadingProviders } =
- useFetchWrapper<ProviderMetadata[]>();
- const { data: customModels, fireRequest: fetchCustomModels, isLoading: loadingCustom } =
- useFetchWrapper<Record<string, CustomModel[]>>();
-
- useEffect(() => {
- posthog?.capture(CLIENT_EVENTS.SETTINGS_MANAGE_MODELS_PAGE_VISITED);
- }, []);
-
- useEffect(() => {
- loadProviders();
- loadAllCustomModels();
- }, []);
-
- const loadProviders = () => {
- fetchProviders({
- requestType: "GET",
- url: "/api/openground/providers",
- });
- };
-
- const loadAllCustomModels = () => {
- fetchCustomModels({
- requestType: "GET",
- url: "/api/openground/models",
- });
- };
-
- const handleSelectModel = (model: ModelMetadata, provider: string, isCustom: boolean) => {
- setSelectedModel(model);
- setSelectedProvider(provider);
- setIsCustomModel(isCustom);
- setIsAddingNew(false);
- };
-
- const handleAddNew = (provider: string) => {
- setSelectedProvider(provider);
- setSelectedModel(null);
- setIsCustomModel(true);
- setIsAddingNew(true);
- };
-
- const handleModelSaved = () => {
- loadAllCustomModels();
- setIsAddingNew(false);
- };
-
- const handleModelDeleted = () => {
- loadAllCustomModels();
- setSelectedModel(null);
- setSelectedProvider(null);
- setIsCustomModel(false);
- };
-
- return (
- <div className="flex flex-col h-full w-full overflow-hidden">
- {/* Header */}
- <div className="flex items-center justify-between p-4 border-b border-stone-200 dark:border-stone-800 shrink-0">
- <div className="flex flex-col gap-2">
- <h1 className="text-lg font-semibold text-stone-900 dark:text-stone-100">
- {getMessage().OPENGROUND_MANAGE_MODELS}
- </h1>
- <p className="text-sm text-stone-500 dark:text-stone-400 mt-0.5">
- {getMessage().OPENGROUND_MANAGE_MODELS_DESCRIPTION}
- </p>
- </div>
- <Button
- variant="outline"
- size="sm"
- className="gap-2"
- onClick={() => {
- window.open("/api/openground/models/export", "_blank");
- }}
- >
- <Download className="h-4 w-4" />
- Export Pricing
- </Button>
- </div>
-
- {/* Main Content - Split Layout */}
- <div className="flex flex-1 overflow-hidden min-h-0">
- <ModelListSidebar
- providers={providers || []}
- customModels={customModels || {}}
- loading={loadingProviders || loadingCustom}
- selectedModel={selectedModel}
- selectedProvider={selectedProvider}
- selectedIsCustom={isCustomModel}
- onSelectModel={handleSelectModel}
- onAddNew={handleAddNew}
- />
-
- <div className="flex-1 overflow-auto bg-stone-50 dark:bg-stone-950">
- {selectedModel || isAddingNew ? (
- <ModelEditorPanel
- model={selectedModel}
- provider={selectedProvider}
- isCustomModel={isCustomModel}
- isAddingNew={isAddingNew}
- onSave={handleModelSaved}
- onDelete={handleModelDeleted}
- onCancel={() => {
- setSelectedModel(null);
- setSelectedProvider(null);
- setIsAddingNew(false);
- }}
- />
- ) : (
- <div className="flex flex-col items-center justify-center h-full p-8 text-center">
- <SettingsIcon className="h-16 w-16 text-stone-300 dark:text-stone-700 mb-4" />
- <h3 className="text-lg font-medium text-stone-900 dark:text-stone-100 mb-2">
- {getMessage().OPENGROUND_SELECT_MODEL_TO_VIEW}
- </h3>
- <p className="text-sm text-stone-500 dark:text-stone-400 max-w-md">
- {getMessage().OPENGROUND_SELECT_MODEL_TO_VIEW_DESCRIPTION}
- </p>
- </div>
- )}
- </div>
- </div>
- </div>
- );
-}
diff --git a/src/client/src/app/api/openground/config/route.ts b/src/client/src/app/api/openground/config/route.ts
index 45a6c4d0..51a387cb 100644
--- a/src/client/src/app/api/openground/config/route.ts
+++ b/src/client/src/app/api/openground/config/route.ts
@@ -9,7 +9,7 @@ import {
upsertOpenGroundConfig,
deleteOpenGroundConfig,
toggleOpenGroundConfigStatus,
-} from "@/lib/platform/openground/config";
+} from "@/lib/platform/providers/config";
import * as messages from "@/constants/messages/en";
/**
diff --git a/src/client/src/app/api/openground/models/cleanup/route.ts b/src/client/src/app/api/openground/models/cleanup/route.ts
index 497619f7..d1df25bb 100644
--- a/src/client/src/app/api/openground/models/cleanup/route.ts
+++ b/src/client/src/app/api/openground/models/cleanup/route.ts
@@ -4,7 +4,7 @@ import { authOptions } from "@/app/auth";
import { dataCollector } from "@/lib/platform/common";
import { getDBConfigByUser } from "@/lib/db-config";
import getMessage from "@/constants/messages";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "@/lib/platform/openground/table-details";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
import asaw from "@/utils/asaw";
import Sanitizer from "@/utils/sanitizer";
@@ -40,9 +40,7 @@ export async function POST(request: NextRequest) {
// First, get all models to see which ones have invalid UUIDs
const selectQuery = `
SELECT toString(id) as id, model_id, display_name
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${session.user.id}'
- AND database_config_id = '${dbConfig.id}'
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
`;
const { data: allModels, err: selectErr } = await dataCollector(
@@ -72,24 +70,12 @@ export async function POST(request: NextRequest) {
});
}
- // Use separate parameters to prevent log injection
- console.log('Found', invalidModels.length, 'models with invalid UUIDs:', invalidModels);
-
- // Sanitize user inputs to prevent SQL injection
- const sanitizedUserId = Sanitizer.sanitizeValue(session.user.id);
- const sanitizedDbConfigId = Sanitizer.sanitizeValue(dbConfig.id);
-
- // Delete all rows with invalid UUIDs using a simpler approach
- // We'll delete all and rely on the user to recreate valid ones
+ // Delete all rows with invalid UUIDs
const deleteQuery = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- DELETE WHERE created_by_user_id = '${sanitizedUserId}'
- AND database_config_id = '${sanitizedDbConfigId}'
- AND NOT match(toString(id), '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$')
+ ALTER TABLE ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ DELETE WHERE NOT match(toString(id), '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$')
`;
- console.log("Cleanup query:", deleteQuery);
-
const { err: deleteErr } = await dataCollector(
{ query: deleteQuery },
"exec",
diff --git a/src/client/src/app/api/openground/models/debug/route.ts b/src/client/src/app/api/openground/models/debug/route.ts
index 8dd31d42..7ea9d0be 100644
--- a/src/client/src/app/api/openground/models/debug/route.ts
+++ b/src/client/src/app/api/openground/models/debug/route.ts
@@ -4,7 +4,7 @@ import { authOptions } from "@/app/auth";
import { dataCollector } from "@/lib/platform/common";
import { getDBConfigByUser } from "@/lib/db-config";
import getMessage from "@/constants/messages";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "@/lib/platform/openground/table-details";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
import asaw from "@/utils/asaw";
// Extend the session type to include id
@@ -41,7 +41,7 @@ export async function GET(request: NextRequest) {
SELECT name
FROM system.tables
WHERE database = currentDatabase()
- AND name = '${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}'
+ AND name = '${OPENLIT_PROVIDER_MODELS_TABLE_NAME}'
`;
const { data: tableData, err: tableErr } = await dataCollector(
@@ -51,7 +51,7 @@ export async function GET(request: NextRequest) {
);
// Try to get table schema
- const schemaQuery = `DESCRIBE TABLE ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}`;
+ const schemaQuery = `DESCRIBE TABLE ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}`;
const { data: schemaData, err: schemaErr } = await dataCollector(
{ query: schemaQuery },
"query",
@@ -59,7 +59,7 @@ export async function GET(request: NextRequest) {
);
// Try to count rows
- const countQuery = `SELECT count() as count FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}`;
+ const countQuery = `SELECT count() as count FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}`;
const { data: countData, err: countErr } = await dataCollector(
{ query: countQuery },
"query",
@@ -67,7 +67,7 @@ export async function GET(request: NextRequest) {
);
// Get actual data to see what's being stored
- const dataQuery = `SELECT toString(id) as id, provider, model_id, display_name FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME} LIMIT 5`;
+ const dataQuery = `SELECT toString(id) as id, provider, model_id, display_name FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME} LIMIT 5`;
const { data: sampleData, err: dataQueryErr } = await dataCollector(
{ query: dataQuery },
"query",
@@ -75,7 +75,7 @@ export async function GET(request: NextRequest) {
);
return NextResponse.json({
- tableName: OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME,
+ tableName: OPENLIT_PROVIDER_MODELS_TABLE_NAME,
databaseConfigId: dbConfig.id,
userId: session.user.id,
checks: {
diff --git a/src/client/src/app/api/openground/models/export/route.ts b/src/client/src/app/api/openground/models/export/route.ts
index 0553a88f..8e6d4531 100644
--- a/src/client/src/app/api/openground/models/export/route.ts
+++ b/src/client/src/app/api/openground/models/export/route.ts
@@ -5,7 +5,7 @@ import { dataCollector } from "@/lib/platform/common";
import { getDBConfigByUser } from "@/lib/db-config";
import getMessage from "@/constants/messages";
import Sanitizer from "@/utils/sanitizer";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "@/lib/platform/openground/table-details";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
import asaw from "@/utils/asaw";
interface SessionWithId {
@@ -40,9 +40,7 @@ export async function GET(request: NextRequest) {
model_type,
input_price_per_m_token as inputPrice,
output_price_per_m_token as outputPrice
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
ORDER BY model_type, model_id
`;
diff --git a/src/client/src/app/api/openground/models/import/route.ts b/src/client/src/app/api/openground/models/import/route.ts
new file mode 100644
index 00000000..c0fc5bf7
--- /dev/null
+++ b/src/client/src/app/api/openground/models/import/route.ts
@@ -0,0 +1,187 @@
+import { NextRequest, NextResponse } from "next/server";
+import { getServerSession } from "next-auth";
+import { authOptions } from "@/app/auth";
+import { dataCollector } from "@/lib/platform/common";
+import { getDBConfigByUser } from "@/lib/db-config";
+import getMessage from "@/constants/messages";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
+import asaw from "@/utils/asaw";
+
+interface SessionWithId {
+ user?: { id?: string };
+}
+
+/**
+ * POST /api/openground/models/import
+ *
+ * Bulk-import models from a pricing JSON (same format as the export endpoint
+ * or the SDK pricing_json). Skips models that already exist (same provider +
+ * model_id combination).
+ *
+ * Body: {
+ * models: Array<{
+ * provider: string;
+ * model_id: string;
+ * displayName: string;
+ * modelType?: string;
+ * contextWindow?: number;
+ * inputPricePerMToken?: number;
+ * outputPricePerMToken?: number;
+ * capabilities?: string[];
+ * }>
+ * }
+ *
+ * OR the SDK pricing_json format:
+ * {
+ * chat: { "model-id": { promptPrice, completionPrice } },
+ * embeddings: { "model-id": price },
+ * ...
+ * }
+ */
+export async function POST(request: NextRequest) {
+ const session = (await getServerSession(authOptions)) as SessionWithId;
+
+ if (!session?.user?.id) {
+ return NextResponse.json(
+ { error: getMessage().UNAUTHORIZED_USER },
+ { status: 401 }
+ );
+ }
+
+ const [, dbConfig] = await asaw(getDBConfigByUser(true));
+ if (!dbConfig?.id) {
+ return NextResponse.json(
+ { error: getMessage().DATABASE_CONFIG_NOT_FOUND },
+ { status: 500 }
+ );
+ }
+
+ const body = await request.json();
+
+ // Normalize input — accept both structured models array and SDK pricing_json format
+ let modelsToImport: Array<{
+ provider: string;
+ model_id: string;
+ display_name: string;
+ model_type: string;
+ context_window: number;
+ input_price_per_m_token: number;
+ output_price_per_m_token: number;
+ capabilities: string[];
+ is_default: boolean;
+ created_by_user_id: string;
+ }> = [];
+
+ if (Array.isArray(body.models)) {
+ // Structured format
+ modelsToImport = body.models.map((m: any) => ({
+ provider: m.provider || "unknown",
+ model_id: m.model_id || m.id,
+ display_name: m.displayName || m.display_name || m.model_id || m.id,
+ model_type: m.modelType || m.model_type || "chat",
+ context_window: m.contextWindow || m.context_window || 4096,
+ input_price_per_m_token: m.inputPricePerMToken || m.input_price_per_m_token || 0,
+ output_price_per_m_token: m.outputPricePerMToken || m.output_price_per_m_token || 0,
+ capabilities: m.capabilities || [],
+ is_default: false,
+ created_by_user_id: session.user!.id!,
+ }));
+ } else {
+ // SDK pricing_json format: { chat: { "model-id": { promptPrice, completionPrice } }, ... }
+ for (const [modelType, models] of Object.entries(body)) {
+ if (modelType === "models") continue; // skip if mixed format
+ if (typeof models !== "object" || models === null) continue;
+
+ for (const [modelId, pricing] of Object.entries(models as Record<string, any>)) {
+ let inputPrice = 0;
+ let outputPrice = 0;
+
+ if (typeof pricing === "number") {
+ // embeddings / audio format
+ inputPrice = pricing * 1000; // per-K to per-M
+ } else if (typeof pricing === "object" && pricing !== null) {
+ if ("promptPrice" in pricing) {
+ inputPrice = (pricing.promptPrice || 0) * 1000;
+ outputPrice = (pricing.completionPrice || 0) * 1000;
+ } else if ("standard" in pricing) {
+ // images format
+ const firstRes = Object.values(pricing.standard || {})[0];
+ inputPrice = ((firstRes as number) || 0) * 1000;
+ }
+ }
+
+ modelsToImport.push({
+ provider: body.provider || "unknown",
+ model_id: modelId,
+ display_name: modelId,
+ model_type: modelType,
+ context_window: 4096,
+ input_price_per_m_token: inputPrice,
+ output_price_per_m_token: outputPrice,
+ capabilities: [],
+ is_default: false,
+ created_by_user_id: session.user!.id!,
+ });
+ }
+ }
+ }
+
+ if (modelsToImport.length === 0) {
+ return NextResponse.json(
+ { error: "No models to import. Provide a 'models' array or SDK pricing_json format." },
+ { status: 400 }
+ );
+ }
+
+ // Fetch existing models to skip duplicates
+ const existingQuery = `
+ SELECT provider, model_id
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ `;
+ const { data: existingRows } = await dataCollector(
+ { query: existingQuery },
+ "query",
+ dbConfig.id
+ );
+
+ const existingKeys = new Set(
+ ((existingRows as any[]) || []).map((r: any) => `${r.provider}::${r.model_id}`)
+ );
+
+ const toInsert = modelsToImport.filter(
+ (m) => !existingKeys.has(`${m.provider}::${m.model_id}`)
+ );
+
+ const skipped = modelsToImport.length - toInsert.length;
+
+ if (toInsert.length === 0) {
+ return NextResponse.json({
+ success: true,
+ imported: 0,
+ skipped,
+ message: "All models already exist",
+ });
+ }
+
+ const { err } = await dataCollector(
+ {
+ table: OPENLIT_PROVIDER_MODELS_TABLE_NAME,
+ values: toInsert,
+ },
+ "insert",
+ dbConfig.id
+ );
+
+ if (err) {
+ return NextResponse.json(
+ { error: err || getMessage().OPERATION_FAILED },
+ { status: 500 }
+ );
+ }
+
+ return NextResponse.json({
+ success: true,
+ imported: toInsert.length,
+ skipped,
+ });
+}
diff --git a/src/client/src/app/api/openground/models/migrate/route.ts b/src/client/src/app/api/openground/models/migrate/route.ts
deleted file mode 100644
index 51dbee3d..00000000
--- a/src/client/src/app/api/openground/models/migrate/route.ts
+++ /dev/null
@@ -1,61 +0,0 @@
-import { NextRequest, NextResponse } from "next/server";
-import { getServerSession } from "next-auth";
-import { authOptions } from "@/app/auth";
-import { getDBConfigByUser } from "@/lib/db-config";
-import getMessage from "@/constants/messages";
-import asaw from "@/utils/asaw";
-import CreateOpengroundCustomModelsMigration from "@/clickhouse/migrations/create-openground-custom-models-migration";
-
-// Extend the session type to include id
-interface SessionWithId {
- user?: {
- id?: string;
- name?: string | null;
- email?: string | null;
- image?: string | null;
- };
-}
-
-// POST: Manually trigger the custom models table migration
-export async function POST(request: NextRequest) {
- const session = (await getServerSession(authOptions)) as SessionWithId;
-
- if (!session?.user?.id) {
- return NextResponse.json(
- { error: getMessage().UNAUTHORIZED_USER },
- { status: 401 }
- );
- }
-
- const [, dbConfig] = await asaw(getDBConfigByUser(true));
- if (!dbConfig?.id) {
- return NextResponse.json(
- { error: getMessage().DATABASE_CONFIG_NOT_FOUND },
- { status: 500 }
- );
- }
-
- // Run the migration
- const result = await CreateOpengroundCustomModelsMigration(dbConfig.id);
-
- if (result.err) {
- console.error("Migration error:", result.err);
- return NextResponse.json(
- { error: result.err },
- { status: 500 }
- );
- }
-
- if (result.migrationExist) {
- return NextResponse.json({
- message: "Migration already exists",
- alreadyRan: true,
- });
- }
-
- return NextResponse.json({
- message: "Migration completed successfully",
- success: true,
- data: result.data,
- });
-}
diff --git a/src/client/src/app/api/openground/models/route.ts b/src/client/src/app/api/openground/models/route.ts
index e1705e43..8df02861 100644
--- a/src/client/src/app/api/openground/models/route.ts
+++ b/src/client/src/app/api/openground/models/route.ts
@@ -7,7 +7,7 @@ import { getDBConfigByUser } from "@/lib/db-config";
import getMessage from "@/constants/messages";
import PostHogServer from "@/lib/posthog";
import Sanitizer from "@/utils/sanitizer";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "@/lib/platform/openground/table-details";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
import asaw from "@/utils/asaw";
// Extend the session type to include id
@@ -56,11 +56,10 @@ export async function GET(request: NextRequest) {
context_window as contextWindow,
input_price_per_m_token as inputPricePerMToken,
output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'
- ORDER BY provider, created_at DESC
+ capabilities,
+ is_default as isDefault
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ ORDER BY provider, is_default DESC, created_at DESC
`;
const { data, err } = await dataCollector(
@@ -104,15 +103,15 @@ export async function GET(request: NextRequest) {
provider,
model_id as id,
display_name as displayName,
+ model_type as modelType,
context_window as contextWindow,
input_price_per_m_token as inputPricePerMToken,
output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'
- AND provider = '${Sanitizer.sanitizeValue(provider)}'
- ORDER BY created_at DESC
+ capabilities,
+ is_default as isDefault
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ WHERE provider = '${Sanitizer.sanitizeValue(provider)}'
+ ORDER BY is_default DESC, created_at DESC
`;
const { data, err } = await dataCollector(
@@ -179,7 +178,7 @@ export async function POST(request: NextRequest) {
.join(", ");
const updateQuery = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
+ ALTER TABLE ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
UPDATE
display_name = '${Sanitizer.sanitizeValue(model.displayName)}',
model_type = '${Sanitizer.sanitizeValue(model.modelType || "chat")}',
@@ -190,8 +189,6 @@ export async function POST(request: NextRequest) {
updated_at = now()
WHERE model_id = '${Sanitizer.sanitizeValue(modelId)}'
AND provider = '${Sanitizer.sanitizeValue(provider)}'
- AND created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'
`;
const { err } = await dataCollector(
@@ -221,7 +218,7 @@ export async function POST(request: NextRequest) {
// Insert new model - Use dataCollector's insert type to let ClickHouse handle UUID generation
const { err } = await dataCollector(
{
- table: OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME,
+ table: OPENLIT_PROVIDER_MODELS_TABLE_NAME,
values: [
{
provider: provider,
@@ -232,8 +229,8 @@ export async function POST(request: NextRequest) {
input_price_per_m_token: model.inputPricePerMToken || 0,
output_price_per_m_token: model.outputPricePerMToken || 0,
capabilities: model.capabilities || [],
+ is_default: false,
created_by_user_id: session.user.id,
- database_config_id: dbConfig.id,
},
],
},
@@ -259,14 +256,14 @@ export async function POST(request: NextRequest) {
provider,
model_id as id,
display_name as displayName,
+ model_type as modelType,
context_window as contextWindow,
input_price_per_m_token as inputPricePerMToken,
output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'
- AND provider = '${Sanitizer.sanitizeValue(provider)}'
+ capabilities,
+ is_default as isDefault
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ WHERE provider = '${Sanitizer.sanitizeValue(provider)}'
AND model_id = '${Sanitizer.sanitizeValue(modelId)}'
ORDER BY created_at DESC
LIMIT 1
@@ -322,10 +319,7 @@ export async function DELETE(request: NextRequest) {
}
// Build WHERE clause — prefer model_id+provider match, fall back to UUID id
- const whereConditions = [
- `created_by_user_id = '${Sanitizer.sanitizeValue(session.user.id)}'`,
- `database_config_id = '${Sanitizer.sanitizeValue(dbConfig.id)}'`,
- ];
+ const whereConditions: string[] = [];
if (modelId && provider) {
whereConditions.push(`model_id = '${Sanitizer.sanitizeValue(modelId)}'`);
@@ -335,7 +329,7 @@ export async function DELETE(request: NextRequest) {
}
const deleteQuery = `
- DELETE FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
+ DELETE FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
WHERE ${whereConditions.join(" AND ")}
`;
diff --git a/src/client/src/app/api/openground/providers/route.ts b/src/client/src/app/api/openground/providers/route.ts
index 353c28e1..1b7e28b5 100644
--- a/src/client/src/app/api/openground/providers/route.ts
+++ b/src/client/src/app/api/openground/providers/route.ts
@@ -9,7 +9,7 @@ import {
getAllProvidersWithCustomModels,
getProviderByIdWithCustomModels,
searchProvidersWithCustomModels,
-} from "@/lib/platform/openground/provider-service";
+} from "@/lib/platform/providers/provider-service";
/**
* GET /api/openground/providers
diff --git a/src/client/src/app/api/pricing/[spanId]/route.ts b/src/client/src/app/api/pricing/[spanId]/route.ts
new file mode 100644
index 00000000..f7f04477
--- /dev/null
+++ b/src/client/src/app/api/pricing/[spanId]/route.ts
@@ -0,0 +1,16 @@
+import { setPricingForSpanId } from "@/lib/platform/pricing";
+import asaw from "@/utils/asaw";
+
+export async function POST(
+ _request: Request,
+ { params }: { params: { spanId: string } }
+) {
+ const { spanId } = params;
+ const [err, res] = await asaw(setPricingForSpanId(spanId));
+
+ if (err) {
+ return Response.json({ success: false, err: err.message || String(err) }, { status: 400 });
+ }
+
+ return Response.json(res);
+}
diff --git a/src/client/src/app/api/pricing/auto/route.ts b/src/client/src/app/api/pricing/auto/route.ts
new file mode 100644
index 00000000..062b4ef5
--- /dev/null
+++ b/src/client/src/app/api/pricing/auto/route.ts
@@ -0,0 +1,8 @@
+import { autoUpdatePricing } from "@/lib/platform/pricing";
+import { NextRequest } from "next/server";
+
+export async function POST(request: NextRequest) {
+ const formData = await request.json();
+ const result = await autoUpdatePricing(formData);
+ return Response.json(result);
+}
diff --git a/src/client/src/app/api/pricing/config/route.ts b/src/client/src/app/api/pricing/config/route.ts
new file mode 100644
index 00000000..749cdcd9
--- /dev/null
+++ b/src/client/src/app/api/pricing/config/route.ts
@@ -0,0 +1,32 @@
+import {
+ getPricingConfig,
+ setPricingConfig,
+ PricingConfigInput,
+} from "@/lib/platform/pricing/config";
+import { NextRequest } from "next/server";
+import asaw from "@/utils/asaw";
+
+export async function GET(_: NextRequest) {
+ const config = await getPricingConfig();
+ return Response.json({ data: config });
+}
+
+export async function POST(request: NextRequest) {
+ const formData = await request.json();
+ const pricingConfig: PricingConfigInput = {
+ id: formData.id,
+ auto: !!formData.auto,
+ recurringTime: formData.recurringTime || "",
+ meta: formData.meta || "{}",
+ };
+
+ const [err, data] = await asaw(
+ setPricingConfig(pricingConfig, request.nextUrl.origin)
+ );
+
+ if (err) {
+ return Response.json(err, { status: 400 });
+ }
+
+ return Response.json({ data });
+}
diff --git a/src/client/src/app/api/pricing/export/[dbConfigId]/route.ts b/src/client/src/app/api/pricing/export/[dbConfigId]/route.ts
new file mode 100644
index 00000000..46d4c83d
--- /dev/null
+++ b/src/client/src/app/api/pricing/export/[dbConfigId]/route.ts
@@ -0,0 +1,96 @@
+import { NextRequest, NextResponse } from "next/server";
+import { dataCollector } from "@/lib/platform/common";
+import { getDBConfigById } from "@/lib/db-config";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "@/lib/platform/providers/table-details";
+import asaw from "@/utils/asaw";
+
+/**
+ * GET /api/pricing/export/[dbConfigId]
+ *
+ * Public (no auth required) endpoint that returns all model pricing in the
+ * OpenLIT SDK `pricing_json` format. The URL is unique per database config
+ * and can be shared / used in SDK init:
+ *
+ * openlit.init(pricing_json="http://localhost:3000/api/pricing/export/<dbConfigId>")
+ */
+export async function GET(
+ _request: NextRequest,
+ { params }: { params: { dbConfigId: string } }
+) {
+ const { dbConfigId } = params;
+
+ if (!dbConfigId) {
+ return NextResponse.json(
+ { error: "Database config ID is required" },
+ { status: 400 }
+ );
+ }
+
+ // Validate the dbConfigId exists
+ const [err, dbConfig] = await asaw(getDBConfigById({ id: dbConfigId }));
+
+ if (err || !dbConfig?.id) {
+ return NextResponse.json(
+ { error: "Database config not found" },
+ { status: 404 }
+ );
+ }
+
+ const query = `
+ SELECT
+ model_id,
+ model_type,
+ input_price_per_m_token as inputPrice,
+ output_price_per_m_token as outputPrice
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ ORDER BY model_type, model_id
+ `;
+
+ const { data, err: queryErr } = await dataCollector(
+ { query },
+ "query",
+ dbConfig.id
+ );
+
+ if (queryErr) {
+ return NextResponse.json(
+ { error: "Failed to fetch model pricing" },
+ { status: 500 }
+ );
+ }
+
+ const models = (data as any[]) || [];
+
+ // Build SDK-compatible pricing.json format
+ const pricing: Record<string, any> = {};
+
+ for (const model of models) {
+ const type = model.model_type || "chat";
+ if (!pricing[type]) {
+ pricing[type] = {};
+ }
+
+ if (type === "chat") {
+ pricing[type][model.model_id] = {
+ promptPrice: model.inputPrice / 1000,
+ completionPrice: model.outputPrice / 1000,
+ };
+ } else if (type === "embeddings") {
+ pricing[type][model.model_id] = model.inputPrice / 1000;
+ } else if (type === "audio") {
+ pricing[type][model.model_id] = model.inputPrice / 1000;
+ } else if (type === "images") {
+ pricing[type][model.model_id] = {
+ standard: {
+ "1024x1024": model.inputPrice / 1000,
+ },
+ };
+ }
+ }
+
+ return NextResponse.json(pricing, {
+ headers: {
+ "Cache-Control": "public, max-age=300",
+ },
+ });
+}
diff --git a/src/client/src/clickhouse/migrations/add-model-type-migration.ts b/src/client/src/clickhouse/migrations/add-model-type-migration.ts
deleted file mode 100644
index aec4fe7d..00000000
--- a/src/client/src/clickhouse/migrations/add-model-type-migration.ts
+++ /dev/null
@@ -1,20 +0,0 @@
-import migrationHelper from "./migration-helper";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "@/lib/platform/openground/table-details";
-
-const MIGRATION_ID = "add-model-type-column";
-
-export default async function AddModelTypeMigration(
- databaseConfigId?: string
-) {
- const queries = [
- `ALTER TABLE ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME} ADD COLUMN IF NOT EXISTS model_type String DEFAULT 'chat'`,
- ];
-
- const { migrationExist, queriesRun } = await migrationHelper({
- clickhouseMigrationId: MIGRATION_ID,
- databaseConfigId,
- queries,
- });
-
- return { migrationExist, queriesRun };
-}
diff --git a/src/client/src/clickhouse/migrations/create-openground-custom-models-migration.ts b/src/client/src/clickhouse/migrations/create-openground-custom-models-migration.ts
deleted file mode 100644
index bc328ae2..00000000
--- a/src/client/src/clickhouse/migrations/create-openground-custom-models-migration.ts
+++ /dev/null
@@ -1,75 +0,0 @@
-import { dataCollector } from "@/lib/platform/common";
-import getMessage from "@/constants/messages";
-import prisma from "@/lib/prisma";
-import asaw from "@/utils/asaw";
-import { getDBConfigById, getDBConfigByUser } from "@/lib/db-config";
-
-export const OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME = "openlit_openground_custom_models";
-
-export default async function CreateOpengroundCustomModelsMigration(databaseConfigId?: string) {
- const [, dbConfig] = await asaw(
- databaseConfigId
- ? getDBConfigById({ id: databaseConfigId })
- : getDBConfigByUser(true)
- );
-
- if (!dbConfig?.id) return { err: getMessage().DATABASE_CONFIG_NOT_FOUND };
-
- // Check if migration already ran
- const [, migrationExist] = await asaw(
- prisma.clickhouseMigrations.findFirst({
- where: {
- AND: {
- databaseConfigId: dbConfig.id,
- clickhouseMigrationId: "create-openground-custom-models-table",
- },
- },
- })
- );
-
- if (migrationExist?.id) {
- return { migrationExist: true };
- }
-
- // Create custom models table
- const customModelsTableQuery = `
- CREATE TABLE IF NOT EXISTS ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME} (
- id UUID DEFAULT generateUUIDv4(),
- provider String, -- Provider ID (e.g., 'openai', 'anthropic')
- model_id String, -- Model identifier
- display_name String, -- Human-readable name
- context_window UInt32 DEFAULT 4096, -- Context window size
- input_price_per_m_token Float64 DEFAULT 0, -- Input price per million tokens
- output_price_per_m_token Float64 DEFAULT 0, -- Output price per million tokens
- capabilities Array(String) DEFAULT [], -- Array of capabilities
-
- created_by_user_id String, -- User who created
- database_config_id String, -- Multi-tenancy
- created_at DateTime DEFAULT now(),
- updated_at DateTime DEFAULT now()
- ) ENGINE = MergeTree()
- PRIMARY KEY (database_config_id, created_by_user_id, provider, id)
- ORDER BY (database_config_id, created_by_user_id, provider, id, created_at);
- `;
-
- const { err } = await dataCollector(
- { query: customModelsTableQuery },
- "exec",
- dbConfig.id
- );
-
- if (err) {
- console.error("Migration error:", err);
- return { err: getMessage().OPERATION_FAILED };
- }
-
- // Record migration success
- await prisma.clickhouseMigrations.create({
- data: {
- databaseConfigId: dbConfig.id,
- clickhouseMigrationId: "create-openground-custom-models-table",
- },
- });
-
- return { data: "Migration successful" };
-}
diff --git a/src/client/src/clickhouse/migrations/create-providers-migration.ts b/src/client/src/clickhouse/migrations/create-providers-migration.ts
new file mode 100644
index 00000000..f13a4c93
--- /dev/null
+++ b/src/client/src/clickhouse/migrations/create-providers-migration.ts
@@ -0,0 +1,205 @@
+import { dataCollector } from "@/lib/platform/common";
+import getMessage from "@/constants/messages";
+import prisma from "@/lib/prisma";
+import asaw from "@/utils/asaw";
+import { getDBConfigById, getDBConfigByUser } from "@/lib/db-config";
+import {
+ OPENLIT_PROVIDERS_TABLE_NAME,
+ OPENLIT_PROVIDER_MODELS_TABLE_NAME,
+} from "@/lib/platform/providers/table-details";
+import {
+ OPENLIT_OPENGROUND_CONFIG_TABLE_NAME,
+ OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME,
+} from "@/lib/platform/openground/table-details";
+import { DEFAULT_MODELS_BY_PROVIDER } from "@/lib/platform/providers/default-models";
+import { consoleLog } from "@/utils/log";
+
+const MIGRATION_ID = "create-providers-and-provider-models-tables";
+
+/**
+ * Creates the new independent provider management tables:
+ * - openlit_providers — provider API key configs
+ * - openlit_provider_models — ALL models (seeded defaults + user-created)
+ *
+ * Each ClickHouse instance is its own scope, so no database_config_id column
+ * is needed inside these tables — the dataCollector routes to the right
+ * ClickHouse via the databaseConfigId argument.
+ *
+ * On first run, copies data from the legacy openground-scoped tables and seeds
+ * default (built-in) models from the static registry so all models are editable.
+ */
+export default async function CreateProvidersMigration(databaseConfigId?: string) {
+ const [, dbConfig] = await asaw(
+ databaseConfigId
+ ? getDBConfigById({ id: databaseConfigId })
+ : getDBConfigByUser(true)
+ );
+
+ if (!dbConfig?.id) return { err: getMessage().DATABASE_CONFIG_NOT_FOUND };
+
+ // Check if migration already ran
+ const [, migrationExist] = await asaw(
+ prisma.clickhouseMigrations.findFirst({
+ where: {
+ AND: {
+ databaseConfigId: dbConfig.id,
+ clickhouseMigrationId: MIGRATION_ID,
+ },
+ },
+ })
+ );
+
+ if (migrationExist?.id) {
+ return { migrationExist: true };
+ }
+
+ // 1. Create openlit_providers (provider configs / API key associations)
+ const providersTableQuery = `
+ CREATE TABLE IF NOT EXISTS ${OPENLIT_PROVIDERS_TABLE_NAME} (
+ id UUID DEFAULT generateUUIDv4(),
+ user_id String,
+ provider String,
+ vault_id String,
+ model_id Nullable(String),
+ is_active Boolean DEFAULT true,
+ created_at DateTime DEFAULT now(),
+ updated_at DateTime DEFAULT now(),
+
+ INDEX id_index (id) TYPE bloom_filter GRANULARITY 1,
+ INDEX provider_index (provider) TYPE bloom_filter GRANULARITY 1,
+ INDEX active_index (is_active) TYPE bloom_filter GRANULARITY 1
+ ) ENGINE = MergeTree()
+ ORDER BY (user_id, provider);
+ `;
+
+ // 2. Create openlit_provider_models (ALL models — seeded defaults + user-created)
+ const modelsTableQuery = `
+ CREATE TABLE IF NOT EXISTS ${OPENLIT_PROVIDER_MODELS_TABLE_NAME} (
+ id UUID DEFAULT generateUUIDv4(),
+ provider String,
+ model_id String,
+ display_name String,
+ model_type String DEFAULT 'chat',
+ context_window UInt32 DEFAULT 4096,
+ input_price_per_m_token Float64 DEFAULT 0,
+ output_price_per_m_token Float64 DEFAULT 0,
+ capabilities Array(String) DEFAULT [],
+ is_default Boolean DEFAULT false,
+
+ created_by_user_id String DEFAULT '',
+ created_at DateTime DEFAULT now(),
+ updated_at DateTime DEFAULT now()
+ ) ENGINE = MergeTree()
+ PRIMARY KEY (provider, model_id)
+ ORDER BY (provider, model_id, created_at);
+ `;
+
+ const createResults = await Promise.all([
+ dataCollector({ query: providersTableQuery }, "exec", dbConfig.id),
+ dataCollector({ query: modelsTableQuery }, "exec", dbConfig.id),
+ ]);
+
+ const createErrors = createResults.filter((res) => res.err);
+ if (createErrors.length > 0) {
+ console.error("Providers migration create errors:", createErrors);
+ return { err: getMessage().OPERATION_FAILED };
+ }
+
+ // 3. Copy existing data from legacy openground_configs -> openlit_providers
+ try {
+ const copyConfigsQuery = `
+ INSERT INTO ${OPENLIT_PROVIDERS_TABLE_NAME}
+ (id, user_id, provider, vault_id, model_id, is_active, created_at, updated_at)
+ SELECT id, user_id, provider, vault_id, model_id, is_active, created_at, updated_at
+ FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ `;
+ await dataCollector({ query: copyConfigsQuery }, "exec", dbConfig.id);
+ } catch (e) {
+ consoleLog("Legacy openground_configs not found or empty — skipping copy");
+ }
+
+ // 4. Copy existing custom models from legacy custom_models -> openlit_provider_models
+ try {
+ const copyModelsQuery = `
+ INSERT INTO ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ (id, provider, model_id, display_name, model_type, context_window,
+ input_price_per_m_token, output_price_per_m_token, capabilities,
+ is_default, created_by_user_id, created_at, updated_at)
+ SELECT
+ id, provider, model_id, display_name,
+ COALESCE(model_type, 'chat') as model_type,
+ context_window, input_price_per_m_token, output_price_per_m_token,
+ capabilities,
+ false as is_default,
+ created_by_user_id, created_at, updated_at
+ FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
+ `;
+ await dataCollector({ query: copyModelsQuery }, "exec", dbConfig.id);
+ } catch (e) {
+ consoleLog("Legacy openground_custom_models not found or empty — skipping copy");
+ }
+
+ // 5. Seed default models from the static registry.
+ // Skip any (provider, model_id) that already exists (user may have copied one in step 4).
+ const defaultModelValues: Record<string, unknown>[] = [];
+ for (const [provider, models] of Object.entries(DEFAULT_MODELS_BY_PROVIDER)) {
+ for (const model of models) {
+ defaultModelValues.push({
+ provider,
+ model_id: model.id,
+ display_name: model.displayName,
+ model_type: "chat",
+ context_window: model.contextWindow,
+ input_price_per_m_token: model.inputPricePerMToken,
+ output_price_per_m_token: model.outputPricePerMToken,
+ capabilities: model.capabilities || [],
+ is_default: true,
+ created_by_user_id: "",
+ });
+ }
+ }
+
+ const existingQuery = `
+ SELECT provider, model_id
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ `;
+ const { data: existingRows } = await dataCollector(
+ { query: existingQuery },
+ "query",
+ dbConfig.id
+ );
+
+ const existingKeys = new Set(
+ ((existingRows as any[]) || []).map((r) => `${r.provider}::${r.model_id}`)
+ );
+
+ const toSeed = defaultModelValues.filter(
+ (m) => !existingKeys.has(`${m.provider}::${m.model_id}`)
+ );
+
+ if (toSeed.length > 0) {
+ const { err: seedErr } = await dataCollector(
+ {
+ table: OPENLIT_PROVIDER_MODELS_TABLE_NAME,
+ values: toSeed,
+ },
+ "insert",
+ dbConfig.id
+ );
+
+ if (seedErr) {
+ console.error("Error seeding default models:", seedErr);
+ return { err: getMessage().OPERATION_FAILED };
+ }
+ }
+
+ // Record migration success
+ await prisma.clickhouseMigrations.create({
+ data: {
+ databaseConfigId: dbConfig.id,
+ clickhouseMigrationId: MIGRATION_ID,
+ },
+ });
+
+ return { data: "Providers migration successful" };
+}
diff --git a/src/client/src/clickhouse/migrations/drop-legacy-openground-tables-migration.ts b/src/client/src/clickhouse/migrations/drop-legacy-openground-tables-migration.ts
new file mode 100644
index 00000000..fe1b072c
--- /dev/null
+++ b/src/client/src/clickhouse/migrations/drop-legacy-openground-tables-migration.ts
@@ -0,0 +1,31 @@
+import migrationHelper from "./migration-helper";
+import {
+ OPENLIT_OPENGROUND_CONFIG_TABLE_NAME,
+ OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME,
+} from "@/lib/platform/openground/table-details";
+
+const MIGRATION_ID = "drop-legacy-openground-config-and-custom-models-tables";
+
+/**
+ * Drops the legacy openground-scoped tables once their data has been copied
+ * into the independent openlit_providers / openlit_provider_models tables.
+ *
+ * Runs AFTER create-providers-migration. `DROP TABLE IF EXISTS` makes this
+ * safe on fresh installs that never had the legacy tables.
+ */
+export default async function DropLegacyOpengroundTablesMigration(
+ databaseConfigId?: string
+) {
+ const queries = [
+ `DROP TABLE IF EXISTS ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}`,
+ `DROP TABLE IF EXISTS ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}`,
+ ];
+
+ const { migrationExist, queriesRun } = await migrationHelper({
+ clickhouseMigrationId: MIGRATION_ID,
+ databaseConfigId,
+ queries,
+ });
+
+ return { migrationExist, queriesRun };
+}
diff --git a/src/client/src/clickhouse/migrations/index.ts b/src/client/src/clickhouse/migrations/index.ts
index 517afe7b..676760ed 100644
--- a/src/client/src/clickhouse/migrations/index.ts
+++ b/src/client/src/clickhouse/migrations/index.ts
@@ -5,13 +5,14 @@ import CreatePromptMigration from "./create-prompt-migration";
import CreateVaultMigration from "./create-vault-migration";
import CreateCustomDashboardsMigration from "./create-custom-dashboards-migration";
import CreateOpengroundMigration from "./create-openground-migration";
-import CreateOpengroundCustomModelsMigration from "./create-openground-custom-models-migration";
import CreateRuleEngineMigration from "./create-rule-engine-migration";
import CreateChatMigration from "./create-chat-migration";
-import AddModelTypeMigration from "./add-model-type-migration";
+import CreateProvidersMigration from "./create-providers-migration";
+import DropLegacyOpengroundTablesMigration from "./drop-legacy-openground-tables-migration";
export default async function migrations(databaseConfigId?: string) {
- return Promise.all([
+ // Run base migrations in parallel
+ await Promise.all([
CreatePromptMigration(databaseConfigId),
CreateVaultMigration(databaseConfigId),
CreateEvaluationMigration(databaseConfigId),
@@ -19,9 +20,13 @@ export default async function migrations(databaseConfigId?: string) {
CreateCronLogMigration(databaseConfigId),
CreateCustomDashboardsMigration(databaseConfigId),
CreateOpengroundMigration(databaseConfigId),
- CreateOpengroundCustomModelsMigration(databaseConfigId),
CreateRuleEngineMigration(databaseConfigId),
CreateChatMigration(databaseConfigId),
- AddModelTypeMigration(databaseConfigId),
]);
+
+ // Create new provider/model tables, copy any legacy data, seed defaults
+ await CreateProvidersMigration(databaseConfigId);
+
+ // Drop the legacy openground config + custom-models tables
+ await DropLegacyOpengroundTablesMigration(databaseConfigId);
}
diff --git a/src/client/src/components/(playground)/chat/chat-settings-form.tsx b/src/client/src/components/(playground)/chat/chat-settings-form.tsx
index 10e578a7..31ca07e7 100644
--- a/src/client/src/components/(playground)/chat/chat-settings-form.tsx
+++ b/src/client/src/components/(playground)/chat/chat-settings-form.tsx
@@ -221,7 +221,7 @@ export default function ChatSettingsForm() {
<p className="text-xs text-stone-400 dark:text-stone-500">
{m.CHAT_SETTINGS_MODEL_HINT}{" "}
<a
- href="/settings/manage-models"
+ href="/manage-models"
className="underline hover:text-stone-600 dark:hover:text-stone-300"
>
{m.CHAT_SETTINGS_MANAGE_MODELS}
diff --git a/src/client/src/components/(playground)/openground/header.tsx b/src/client/src/components/(playground)/openground/header.tsx
index 4c5f9648..1a3995d3 100644
--- a/src/client/src/components/(playground)/openground/header.tsx
+++ b/src/client/src/components/(playground)/openground/header.tsx
@@ -24,7 +24,7 @@ export default function OpengroundHeader({
<div className={className}>
{showButton ? (
<>
- <Link href={"/settings/manage-models"}>
+ <Link href={"/manage-models"}>
<Button variant="outline" className="h-auto py-1">
<SettingsIcon className="h-4 w-4 mr-2" />
{getMessage().OPENGROUND_MANAGE_MODELS}
diff --git a/src/client/src/components/(playground)/openground/model-editor-panel.tsx b/src/client/src/components/(playground)/openground/model-editor-panel.tsx
index 9856a8c5..0881612b 100644
--- a/src/client/src/components/(playground)/openground/model-editor-panel.tsx
+++ b/src/client/src/components/(playground)/openground/model-editor-panel.tsx
@@ -62,7 +62,7 @@ export default function ModelEditorPanel({
});
useEffect(() => {
- if (model && isCustomModel) {
+ if (model && !isAddingNew) {
const customModel = model as any;
// The API returns `customId` as the UUID and `id` as the model_id.
// Map them correctly: UUID goes to `id`, model identifier goes to `model_id`.
@@ -272,27 +272,25 @@ export default function ModelEditorPanel({
</p>
</div>
- {/* Actions */}
- {(
- <div className="flex gap-2 pt-4">
- <Button onClick={handleSave} disabled={saving} className="flex-1">
- {saving ? getMessage().SAVING : getMessage().OPENGROUND_SAVE_MODEL}
- </Button>
- {isCustomModel && !isAddingNew && (
- <Button
- variant="destructive"
- onClick={() => setShowDeleteDialog(true)}
- disabled={deleting}
- >
- <Trash2Icon className="h-4 w-4 mr-2" />
- {getMessage().DELETE}
- </Button>
- )}
- <Button variant="outline" onClick={onCancel}>
- {getMessage().CANCEL}
+ {/* Actions — all models editable now */}
+ <div className="flex gap-2 pt-4">
+ <Button onClick={handleSave} disabled={saving} className="flex-1">
+ {saving ? getMessage().SAVING : getMessage().OPENGROUND_SAVE_MODEL}
+ </Button>
+ {!isAddingNew && (
+ <Button
+ variant="destructive"
+ onClick={() => setShowDeleteDialog(true)}
+ disabled={deleting}
+ >
+ <Trash2Icon className="h-4 w-4 mr-2" />
+ {getMessage().DELETE}
</Button>
- </div>
- )}
+ )}
+ <Button variant="outline" onClick={onCancel}>
+ {getMessage().CANCEL}
+ </Button>
+ </div>
</CardContent>
</Card>
diff --git a/src/client/src/components/(playground)/openground/model-list-sidebar.tsx b/src/client/src/components/(playground)/openground/model-list-sidebar.tsx
index d1d1341f..56996535 100644
--- a/src/client/src/components/(playground)/openground/model-list-sidebar.tsx
+++ b/src/client/src/components/(playground)/openground/model-list-sidebar.tsx
@@ -15,6 +15,7 @@ interface CustomModel extends ModelMetadata {
model_id: string; // Model identifier like "gpt-4o"
provider?: string;
modelType?: string;
+ isDefault?: boolean;
}
interface ModelListSidebarProps {
@@ -57,31 +58,25 @@ export default function ModelListSidebar({
const query = searchQuery.toLowerCase();
return providers.filter((provider) => {
const matchesProviderName = provider.displayName.toLowerCase().includes(query);
- const matchesModelName = provider.supportedModels.some((model) =>
- model.displayName.toLowerCase().includes(query) ||
- model.id.toLowerCase().includes(query)
- );
- const matchesCustomModel = customModels[provider.providerId]?.some((model) =>
- model.displayName.toLowerCase().includes(query) ||
- model.model_id.toLowerCase().includes(query)
+ const matchesModelName = (customModels[provider.providerId] || []).some(
+ (model) =>
+ model.displayName.toLowerCase().includes(query) ||
+ (model.model_id || "").toLowerCase().includes(query)
);
- return matchesProviderName || matchesModelName || matchesCustomModel;
+ return matchesProviderName || matchesModelName;
});
}, [providers, customModels, searchQuery]);
- const isModelSelected = (model: ModelMetadata, provider: string, isCustom: boolean) => {
+ const isModelSelected = (model: CustomModel, provider: string) => {
if (selectedProvider !== provider) return false;
if (!selectedModel) return false;
- if (!isCustom || !selectedIsCustom) return false;
-
- // Match by customId (UUID) or model_id
- const customModel = model as any;
const selectedCustomModel = selectedModel as any;
- const customModelKey = customModel.customId || customModel.id;
- const selectedKey = selectedCustomModel.customId || selectedCustomModel.id;
- return customModelKey === selectedKey;
+ const currentKey = model.customId || model.id || model.model_id;
+ const selectedKey =
+ selectedCustomModel.customId || selectedCustomModel.id || selectedCustomModel.model_id;
+ return currentKey === selectedKey;
};
return (
@@ -113,11 +108,8 @@ export default function ModelListSidebar({
<div className="p-2">
{filteredProviders.map((provider) => {
const isExpanded = expandedProviders.has(provider.providerId);
- const providerCustomModels = customModels[provider.providerId] || [];
- // Deduplicate: exclude custom model IDs from static count
- const customModelIds = new Set(providerCustomModels.map((cm) => cm.model_id || cm.id));
- const staticModels = provider.supportedModels.filter((m) => !customModelIds.has(m.id));
- const totalModels = staticModels.length + providerCustomModels.length;
+ const providerModels = customModels[provider.providerId] || [];
+ const totalModels = providerModels.length;
return (
<div key={provider.providerId} className="mb-2">
@@ -152,62 +144,46 @@ export default function ModelListSidebar({
</Button>
</button>
- {/* Models List */}
+ {/* Models List — all editable */}
{isExpanded && (
<div className="ml-4 mt-1 space-y-1">
- {/* Static Models - Display only, not editable */}
- {staticModels.map((model) => (
- <div
- key={model.id}
- className="w-full text-left p-2 rounded-md border-l-2 border-transparent opacity-75"
- >
- <div className="text-sm font-medium text-stone-700 dark:text-stone-300">{model.displayName}</div>
- <div className="flex items-center gap-2 mt-1">
- <Badge variant="outline" className="text-xs">
- {model.contextWindow.toLocaleString()} tokens
- </Badge>
- <Badge variant="outline" className="text-xs">
- ${model.inputPricePerMToken}/M
- </Badge>
- </div>
+ {providerModels.length === 0 ? (
+ <div className="text-xs text-stone-400 dark:text-stone-500 px-2 py-2">
+ {getMessage().OPENGROUND_NO_MODELS_FOUND}
</div>
- ))}
-
- {/* Custom Models Section */}
- {providerCustomModels.length > 0 && (
- <>
- <div className="text-xs font-medium text-stone-500 dark:text-stone-400 px-2 py-1 mt-2">
- {getMessage().OPENGROUND_CUSTOM_MODELS}
- </div>
- {providerCustomModels.map((model) => (
- <button
- key={model.id}
- onClick={() => onSelectModel(model, provider.providerId, true)}
- className={cn(
- "w-full text-left p-2 rounded-md transition-colors border-l-2 group/model",
- isModelSelected(model, provider.providerId, true)
- ? "bg-primary/10 dark:bg-primary/20 border-primary text-stone-900 dark:text-stone-100"
- : "border-transparent hover:bg-stone-100 dark:hover:bg-stone-800 text-stone-700 dark:text-stone-300"
- )}
- >
- <div className="flex items-center justify-between">
- <div className="flex items-center gap-2">
- <div className="text-sm font-medium">{model.displayName}</div>
+ ) : (
+ providerModels.map((model) => (
+ <button
+ key={model.customId || model.id || model.model_id}
+ onClick={() => onSelectModel(model, provider.providerId, true)}
+ className={cn(
+ "w-full text-left p-2 rounded-md transition-colors border-l-2 group/model",
+ isModelSelected(model, provider.providerId)
+ ? "bg-primary/10 dark:bg-primary/20 border-primary text-stone-900 dark:text-stone-100"
+ : "border-transparent hover:bg-stone-100 dark:hover:bg-stone-800 text-stone-700 dark:text-stone-300"
+ )}
+ >
+ <div className="flex items-center justify-between">
+ <div className="flex items-center gap-2">
+ <div className="text-sm font-medium">{model.displayName}</div>
+ {model.isDefault ? (
+ <Badge variant="outline" className="text-xs h-4">Default</Badge>
+ ) : (
<Badge className="text-xs h-4">Custom</Badge>
- </div>
- <PencilIcon className="h-3 w-3 text-stone-400 opacity-0 group-hover/model:opacity-100 transition-opacity" />
- </div>
- <div className="flex items-center gap-2 mt-1">
- <Badge variant="outline" className="text-xs">
- {model.contextWindow.toLocaleString()} tokens
- </Badge>
- <Badge variant="outline" className="text-xs">
- ${model.inputPricePerMToken}/M
- </Badge>
+ )}
</div>
- </button>
- ))}
- </>
+ <PencilIcon className="h-3 w-3 text-stone-400 opacity-0 group-hover/model:opacity-100 transition-opacity" />
+ </div>
+ <div className="flex items-center gap-2 mt-1">
+ <Badge variant="outline" className="text-xs">
+ {(model.contextWindow || 0).toLocaleString()} tokens
+ </Badge>
+ <Badge variant="outline" className="text-xs">
+ ${model.inputPricePerMToken}/M
+ </Badge>
+ </div>
+ </button>
+ ))
)}
</div>
)}
diff --git a/src/client/src/components/(playground)/openground/provider-config-dialog.tsx b/src/client/src/components/(playground)/openground/provider-config-dialog.tsx
index 5af3365f..9cb77bc6 100644
--- a/src/client/src/components/(playground)/openground/provider-config-dialog.tsx
+++ b/src/client/src/components/(playground)/openground/provider-config-dialog.tsx
@@ -25,6 +25,7 @@ import { Badge } from "@/components/ui/badge";
import useFetchWrapper from "@/utils/hooks/useFetchWrapper";
import SecretForm from "@/components/(playground)/vault/form";
import getMessage from "@/constants/messages";
+import { consoleLog } from "@/utils/log";
interface ProviderConfigDialogProps {
providerId: string;
@@ -93,7 +94,7 @@ export default function ProviderConfigDialog({
url: "/api/openground/config",
failureCb: (err?: string) => {
// Silently fail if no config exists yet
- console.log("No existing config:", err);
+ consoleLog("No existing config:", err);
},
});
};
diff --git a/src/client/src/components/(playground)/openground/sdk-usage-dialog.tsx b/src/client/src/components/(playground)/openground/sdk-usage-dialog.tsx
new file mode 100644
index 00000000..4052b950
--- /dev/null
+++ b/src/client/src/components/(playground)/openground/sdk-usage-dialog.tsx
@@ -0,0 +1,135 @@
+"use client";
+
+import { Button } from "@/components/ui/button";
+import {
+ Dialog,
+ DialogContent,
+ DialogDescription,
+ DialogFooter,
+ DialogHeader,
+ DialogTitle,
+} from "@/components/ui/dialog";
+import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
+import { Input } from "@/components/ui/input";
+import { Label } from "@/components/ui/label";
+import CodeBlock from "@/components/common/code-block";
+import { Copy } from "lucide-react";
+import getMessage from "@/constants/messages";
+import copy from "copy-to-clipboard";
+import { toast } from "sonner";
+
+interface SdkUsageDialogProps {
+ open: boolean;
+ onOpenChange: (open: boolean) => void;
+ pricingUrl: string;
+}
+
+const LANGUAGES = [
+ { value: "python", label: "Python", lang: "python" },
+ { value: "typescript", label: "TypeScript", lang: "typescript" },
+ { value: "go", label: "Go", lang: "go" },
+] as const;
+
+function buildSnippets(pricingUrl: string): Record<string, string> {
+ return {
+ python: `import openlit
+
+openlit.init(
+ otlp_endpoint="http://127.0.0.1:4318",
+ pricing_json="${pricingUrl}",
+)`,
+ typescript: `import openlit from "openlit";
+
+openlit.init({
+ otlpEndpoint: "http://127.0.0.1:4318",
+ pricingJson: "${pricingUrl}",
+});`,
+ go: `package main
+
+import openlit "github.com/openlit/openlit/sdk/go"
+
+func main() {
+ openlit.Init(openlit.Config{
+ OtlpEndpoint: "http://127.0.0.1:4318",
+ PricingJson: "${pricingUrl}",
+ })
+}`,
+ };
+}
+
+export default function SdkUsageDialog({
+ open,
+ onOpenChange,
+ pricingUrl,
+}: SdkUsageDialogProps) {
+ const m = getMessage();
+ const snippets = buildSnippets(pricingUrl);
+
+ const handleCopyUrl = () => {
+ if (!pricingUrl) return;
+ copy(pricingUrl);
+ toast.success(m.MANAGE_MODELS_PRICING_URL_COPIED, { id: "url-copy" });
+ };
+
+ return (
+ <Dialog open={open} onOpenChange={onOpenChange}>
+ <DialogContent className="sm:max-w-3xl max-h-[90vh] overflow-y-auto">
+ <DialogHeader>
+ <DialogTitle>{m.MANAGE_MODELS_SDK_USAGE_DIALOG_TITLE}</DialogTitle>
+ <DialogDescription>
+ {m.MANAGE_MODELS_SDK_USAGE_DIALOG_DESCRIPTION}
+ </DialogDescription>
+ </DialogHeader>
+
+ {/* URL block — uses Input (readonly) + Button for visual consistency */}
+ <div className="space-y-2">
+ <Label className="text-xs text-stone-500 dark:text-stone-400">
+ {m.MANAGE_MODELS_PRICING_URL_LABEL}
+ </Label>
+ <div className="flex items-center gap-2">
+ <Input
+ readOnly
+ value={pricingUrl}
+ onFocus={(e) => e.target.select()}
+ className="font-mono text-xs"
+ />
+ <Button
+ variant="outline"
+ size="sm"
+ className="gap-1.5 shrink-0"
+ onClick={handleCopyUrl}
+ >
+ <Copy className="h-3.5 w-3.5" />
+ {m.COPY}
+ </Button>
+ </div>
+ </div>
+
+ {/* Language tabs — CodeBlock has built-in copy + syntax highlighting */}
+ <Tabs defaultValue="python" className="w-full">
+ <TabsList className="grid grid-cols-3 w-full">
+ {LANGUAGES.map((l) => (
+ <TabsTrigger key={l.value} value={l.value}>
+ {l.label}
+ </TabsTrigger>
+ ))}
+ </TabsList>
+ {LANGUAGES.map((l) => (
+ <TabsContent key={l.value} value={l.value} className="mt-2">
+ <CodeBlock
+ className="text-xs rounded-lg"
+ code={snippets[l.value]}
+ language={l.lang}
+ />
+ </TabsContent>
+ ))}
+ </Tabs>
+
+ <p className="text-xs text-stone-500 dark:text-stone-400">
+ <strong>{m.MANAGE_MODELS_SDK_USAGE_NOTE_LABEL}</strong>{" "}
+ {m.MANAGE_MODELS_SDK_USAGE_NOTE}
+ </p>
+ </DialogContent>
+ </Dialog>
+ );
+}
diff --git a/src/client/src/components/(playground)/request/request-details.tsx b/src/client/src/components/(playground)/request/request-details.tsx
index e4570cc7..303c71ac 100644
--- a/src/client/src/components/(playground)/request/request-details.tsx
+++ b/src/client/src/components/(playground)/request/request-details.tsx
@@ -9,7 +9,8 @@ import {
normalizeTrace,
} from "@/helpers/client/trace";
import { ReverseTraceMapping, TraceMapping } from "@/constants/traces";
-import { ExternalLink, X, DollarSign, Zap, Clock, Cpu, ChevronLeft, ChevronRight, BarChart3 } from "lucide-react";
+import { ExternalLink, X, DollarSign, Zap, Clock, Cpu, ChevronLeft, ChevronRight, BarChart3, RefreshCw } from "lucide-react";
+import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from "@/components/ui/tooltip";
import {
Sheet,
SheetContent,
@@ -21,6 +22,7 @@ import { ValueOf } from "@/types/util";
import { ReactNode, useCallback, useEffect, useState } from "react";
import useFetchWrapper from "@/utils/hooks/useFetchWrapper";
import { toast } from "sonner";
+import getMessage from "@/constants/messages";
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs";
import HeirarchyDisplay from "./heirarchy-display";
import { TraceRow, TransformedTraceRow } from "@/types/trace";
@@ -84,6 +86,90 @@ function MetricCard({
);
}
+function CostMetricCard({
+ costValue,
+ spanId,
+ hasModel,
+ onRecalculated,
+}: {
+ costValue: string | undefined;
+ spanId: string | undefined;
+ hasModel: boolean;
+ onRecalculated: () => void;
+}) {
+ const m = getMessage();
+ const hasCost = !!costValue;
+ const canRecalculate = hasModel && !!spanId;
+
+ const { fireRequest, isLoading } = useFetchWrapper<{
+ success: boolean;
+ err?: string;
+ data?: { spanId: string; cost: number };
+ }>();
+
+ const handleRecalculate = (e: React.MouseEvent) => {
+ e.stopPropagation();
+ if (!spanId || isLoading) return;
+ fireRequest({
+ requestType: "POST",
+ url: `/api/pricing/${spanId}`,
+ successCb: (res) => {
+ if (res?.success) {
+ toast.success(
+ `${m.RECALCULATE_COST_SUCCESS}: $${(res.data?.cost ?? 0).toFixed(10)}`,
+ { id: "pricing-update" }
+ );
+ onRecalculated();
+ } else {
+ toast.error(res?.err || m.RECALCULATE_COST_FAILURE, {
+ id: "pricing-update",
+ });
+ }
+ },
+ failureCb: (err?: string) => {
+ toast.error(err || m.RECALCULATE_COST_REQUEST_FAILED, {
+ id: "pricing-update",
+ });
+ },
+ });
+ };
+
+ return (
+ <div className="relative flex flex-col gap-1 px-3 py-2 bg-stone-100 dark:bg-stone-800 rounded-lg min-w-[100px] shrink-0">
+ <div className="flex items-center gap-1.5 text-stone-500 dark:text-stone-400">
+ <DollarSign className="h-3.5 w-3.5" />
+ <span className="text-xs font-medium">Cost</span>
+ {canRecalculate && (
+ <TooltipProvider delayDuration={200}>
+ <Tooltip>
+ <TooltipTrigger asChild>
+ <button
+ onClick={handleRecalculate}
+ disabled={isLoading}
+ className="relative ml-auto p-0.5 rounded hover:bg-stone-200 dark:hover:bg-stone-700 transition-colors disabled:opacity-50"
+ >
+ <RefreshCw
+ className={`h-3 w-3 ${isLoading ? "animate-spin" : ""}`}
+ />
+ {!hasCost && !isLoading && (
+ <span className="w-2 h-2 bg-primary absolute -top-0.5 -right-0.5 rounded-full animate-ping" />
+ )}
+ </button>
+ </TooltipTrigger>
+ <TooltipContent side="bottom" className="max-w-[220px] text-xs">
+ {m.RECALCULATE_COST_TITLE}
+ </TooltipContent>
+ </Tooltip>
+ </TooltipProvider>
+ )}
+ </div>
+ <span className="text-sm font-semibold text-stone-800 dark:text-stone-200 truncate">
+ {costValue ?? "—"}
+ </span>
+ </div>
+ );
+}
+
function LoadingSkeleton() {
return (
<div className="flex flex-col gap-4 p-4 animate-pulse">
@@ -312,10 +398,11 @@ export default function RequestDetails() {
>
{/* Key metrics strip */}
<div className="shrink-0 flex items-center gap-2 px-4 py-3 overflow-x-auto bg-white dark:bg-stone-950 border-b border-stone-200 dark:border-stone-800">
- <MetricCard
- icon={<DollarSign className="h-3.5 w-3.5" />}
- label="Cost"
- value={costValue}
+ <CostMetricCard
+ costValue={costValue}
+ spanId={normalizedItem?.spanId ? String(normalizedItem.spanId) : undefined}
+ hasModel={!!modelValue}
+ onRecalculated={fetchData}
/>
<MetricCard
icon={<Zap className="h-3.5 w-3.5" />}
diff --git a/src/client/src/constants/messages/en.ts b/src/client/src/constants/messages/en.ts
index cb9ec254..d2d27454 100644
--- a/src/client/src/constants/messages/en.ts
+++ b/src/client/src/constants/messages/en.ts
@@ -258,6 +258,81 @@ export const OPENGROUND_ALL_PROVIDERS = "All Providers";
export const OPENGROUND_SEARCH_MODELS = "Search models...";
export const OPENGROUND_NO_MODELS_FOUND = "No models found";
export const OPENGROUND_CUSTOM_MODELS = "Custom Models";
+export const MANAGE_MODELS_EXPORT_PRICING = "Export Pricing";
+export const MANAGE_MODELS_IMPORT_PRICING = "Import Pricing JSON";
+export const MANAGE_MODELS_IMPORT_SUCCESS = "Import complete";
+export const MANAGE_MODELS_IMPORT_FAILED = "Import failed";
+export const MANAGE_MODELS_PRICING_URL_LABEL = "SDK Pricing URL";
+export const MANAGE_MODELS_PRICING_URL_DESCRIPTION =
+ "Use this URL in your OpenLIT SDK to load pricing from this database. The endpoint is public and does not require authentication.";
+export const MANAGE_MODELS_PRICING_URL_COPIED = "URL copied to clipboard";
+export const MANAGE_MODELS_IMPORT_DIALOG_TITLE = "Import Pricing JSON";
+export const MANAGE_MODELS_IMPORT_DIALOG_DESCRIPTION =
+ "Paste a pricing JSON (SDK format or structured models array). Existing models with the same provider + model ID are skipped.";
+export const MANAGE_MODELS_SDK_USAGE = "SDK Usage";
+export const MANAGE_MODELS_SDK_USAGE_DIALOG_TITLE =
+ "Use this pricing in your OpenLIT SDK";
+export const MANAGE_MODELS_SDK_USAGE_DIALOG_DESCRIPTION =
+ "Pass this public URL to pricing_json in openlit.init() and your SDK will use the model pricing from Manage Models. No API key required.";
+export const MANAGE_MODELS_SDK_USAGE_NOTE_LABEL = "Note:";
+export const MANAGE_MODELS_SDK_USAGE_NOTE =
+ "The SDK fetches this URL on startup. If you update a model's price here, it will be applied to future tracing.";
+export const COPY = "Copy";
+export const MANAGE_MODELS_INVALID_JSON = "Invalid JSON";
+
+// Pricing
+export const PRICING_TITLE = "Pricing";
+export const PRICING_PAGE_DESCRIPTION =
+ "Recalculate the cost of LLM traces using the current model pricing in Manage Models. Updates the `gen_ai.usage.cost` attribute on existing traces.";
+export const PRICING_INFO_BAR =
+ "Pricing is computed from the input/output token counts on the trace and the per-million-token price stored for each model in Manage Models. Edit a model there to update what auto/manual pricing computes.";
+export const PRICING_AUTO_TITLE = "Auto Pricing";
+export const PRICING_AUTO_DESCRIPTION =
+ "Recompute pricing on a schedule for new LLM traces.";
+export const PRICING_AUTO_ENABLE_LABEL = "Enable Auto Pricing";
+export const PRICING_AUTO_ENABLE_HINT =
+ "Periodically recompute and store pricing for new traces.";
+export const PRICING_AUTO_CRON_LABEL = "Cron Schedule";
+export const PRICING_AUTO_CRON_PLACEHOLDER = "*/15 * * * *";
+export const PRICING_AUTO_CRON_HELP =
+ "Standard cron expression. Example: */15 * * * * runs every 15 minutes.";
+export const PRICING_MANUAL_TITLE = "Manual Pricing";
+export const PRICING_MANUAL_DESCRIPTION =
+ "Recalculate the cost for a specific trace on demand.";
+export const PRICING_MANUAL_STEP_1 = "Open any LLM request in the trace explorer.";
+export const PRICING_MANUAL_STEP_2 =
+ 'Click "Recalculate Cost" in the trace detail panel.';
+export const PRICING_MANUAL_STEP_3 =
+ "The gen_ai.usage.cost attribute is updated using the latest model pricing.";
+export const PRICING_GO_TO_REQUESTS = "Go to Requests";
+export const PRICING_SAVE = "Save";
+export const PRICING_UPDATE = "Update";
+export const PRICING_CONFIG_SAVED = "Pricing config saved";
+export const PRICING_HOW_AUTO_WORKS_TITLE = "How Auto Pricing Works";
+export const PRICING_HOW_AUTO_WORKS_DESCRIPTION =
+ "On each cron tick, OpenLIT selects LLM spans from otel_traces, looks up each span's provider + model in openlit_provider_models, recomputes the cost from token counts, and writes it back to gen_ai.usage.cost.";
+export const PRICING_HOW_AUTO_WORKS_STEP_1 =
+ "Selects LLM spans: spans whose gen_ai.operation.name matches supported LLM operations (e.g. chat).";
+export const PRICING_HOW_AUTO_WORKS_STEP_2 =
+ "Incremental window: only spans with Timestamp >= the last successful cron run are processed. First run processes all historical LLM spans.";
+export const PRICING_HOW_AUTO_WORKS_STEP_3 =
+ "Looks up the model: for each span, reads gen_ai.system (provider) + gen_ai.request.model and finds the matching row in openlit_provider_models.";
+export const PRICING_HOW_AUTO_WORKS_STEP_4 =
+ "Computes cost: (input_tokens / 1M) × input_price_per_m_token + (output_tokens / 1M) × output_price_per_m_token.";
+export const PRICING_HOW_AUTO_WORKS_STEP_5 =
+ "Writes back: ALTER TABLE otel_traces UPDATE SpanAttributes['gen_ai.usage.cost'] for each processed span. Spans with missing provider/model/tokens or an unknown model are skipped (not errors).";
+export const PRICING_HOW_AUTO_WORKS_STEP_6 =
+ "Run is logged: totalSpans, totalUpdated, totalFailed, totalSkipped are recorded in the cron log with SUCCESS / PARTIAL_SUCCESS / FAILURE status.";
+
+// Recalculate Cost (in trace detail panel)
+export const RECALCULATE_COST_LABEL = "Recalculate Cost";
+export const RECALCULATE_COST_UPDATING = "Updating…";
+export const RECALCULATE_COST_TITLE =
+ "Recalculate cost using the model's price in Manage Models";
+export const RECALCULATE_COST_SUCCESS = "Cost updated";
+export const RECALCULATE_COST_FAILURE =
+ "Could not update cost — model not found in Manage Models or pricing unavailable";
+export const RECALCULATE_COST_REQUEST_FAILED = "Failed to recalculate cost";
// Features Title
export const FEATURE_OPENGROUND = "Openground";
diff --git a/src/client/src/constants/route.ts b/src/client/src/constants/route.ts
index f4e9b433..f2743c69 100644
--- a/src/client/src/constants/route.ts
+++ b/src/client/src/constants/route.ts
@@ -5,7 +5,12 @@ export const ALLOWED_OPENLIT_ROUTES_WITHOUT_TOKEN = [
"/api/rule-engine/evaluate",
];
-export const CRON_JOB_ROUTES = ["/api/evaluation/auto"];
+/** Routes that don't require auth, matched by prefix (for dynamic segments). */
+export const ALLOWED_OPENLIT_ROUTE_PREFIXES_WITHOUT_TOKEN = [
+ "/api/pricing/export/",
+];
+
+export const CRON_JOB_ROUTES = ["/api/evaluation/auto", "/api/pricing/auto"];
// Non-API routes that are accessible without completing onboarding
export const ONBOARDING_WHITELIST_ROUTES = ["/onboarding"];
diff --git a/src/client/src/constants/sidebar.tsx b/src/client/src/constants/sidebar.tsx
index f3d2ee6d..0a6514b2 100644
--- a/src/client/src/constants/sidebar.tsx
+++ b/src/client/src/constants/sidebar.tsx
@@ -3,6 +3,7 @@ import {
BookKey,
BookOpen,
BookText,
+ CircleDollarSign,
Component,
Home,
Key,
@@ -124,6 +125,18 @@ export const SIDEBAR_ITEMS: SidebarItemProps[] = [
link: "/evaluations",
type: "action",
},
+ {
+ icon: <SettingsIcon className={ICON_CLASSES} />,
+ text: "Manage Models",
+ link: "/manage-models",
+ type: "action",
+ },
+ {
+ icon: <CircleDollarSign className={ICON_CLASSES} />,
+ text: "Pricing",
+ link: "/pricing",
+ type: "action",
+ },
]
},
{
@@ -137,12 +150,6 @@ export const SIDEBAR_ITEMS: SidebarItemProps[] = [
link: "/settings/api-keys",
type: "action",
},
- {
- icon: <SettingsIcon className={ICON_CLASSES} />,
- text: "Manage Models",
- link: "/settings/manage-models",
- type: "action",
- },
]
},
{
diff --git a/src/client/src/helpers/server/cron.ts b/src/client/src/helpers/server/cron.ts
index ad17dab0..e9b22937 100644
--- a/src/client/src/helpers/server/cron.ts
+++ b/src/client/src/helpers/server/cron.ts
@@ -58,8 +58,15 @@ export default class Cron {
this.createLogDirectoryIfNotExists(job.cronLogPath);
+ // Resolve the absolute path to Node now (at crontab-write time) using the
+ // currently-running interpreter. Cron's PATH is minimal and often does not
+ // include node (nvm/homebrew installs especially), so `$(which node)` at
+ // run-time collapses to empty and /bin/sh tries to exec the .js directly,
+ // failing with "Permission denied".
+ const nodeBin = process.execPath;
+
// Add the new cron job entry
- const newEntry = `${job.cronSchedule} CRON_ID=${job.cronId} ${envVars} $(which node) ${job.cronScriptPath} >> ${job.cronLogPath} 2>&1`;
+ const newEntry = `${job.cronSchedule} CRON_ID=${job.cronId} ${envVars} ${nodeBin} ${job.cronScriptPath} >> ${job.cronLogPath} 2>&1`;
managedSection.push(newEntry);
// Construct the new crontab content
diff --git a/src/client/src/instrumentation.ts b/src/client/src/instrumentation.ts
index 940765a8..28e44327 100644
--- a/src/client/src/instrumentation.ts
+++ b/src/client/src/instrumentation.ts
@@ -23,12 +23,14 @@ export async function register() {
// This prevents the server from crashing if ClickHouse is temporarily unavailable
}
+ const apiURL =
+ process.env.API_URL || `http://localhost:${process.env.PORT || 3000}`;
+
try {
// Restore cron jobs for auto-evaluations (survives new image deployments)
const { restoreEvaluationCronJobs } = await import(
"@/lib/platform/evaluation/config"
);
- const apiURL = process.env.API_URL || `http://localhost:${process.env.PORT || 3000}`;
console.log("🔄 Restoring evaluation cron jobs...");
await restoreEvaluationCronJobs(apiURL);
console.log("✅ Evaluation cron jobs restored");
@@ -37,6 +39,19 @@ export async function register() {
// Don't throw - allow server to start even if cron restore fails
}
+ try {
+ // Restore cron jobs for auto-pricing
+ const { restorePricingCronJobs } = await import(
+ "@/lib/platform/pricing/config"
+ );
+ console.log("🔄 Restoring pricing cron jobs...");
+ await restorePricingCronJobs(apiURL);
+ console.log("✅ Pricing cron jobs restored");
+ } catch (error) {
+ console.error("❌ Error restoring pricing cron jobs:", error);
+ // Don't throw - allow server to start even if cron restore fails
+ }
+
console.log("✨ Server initialization complete");
}
}
diff --git a/src/client/src/lib/platform/chat/schema-context.ts b/src/client/src/lib/platform/chat/schema-context.ts
index 3412b298..a7fdf36b 100644
--- a/src/client/src/lib/platform/chat/schema-context.ts
+++ b/src/client/src/lib/platform/chat/schema-context.ts
@@ -40,7 +40,7 @@ URL mappings per entity type:
- **context**: \`/context/{id}\`
- **prompt**: \`/prompt-hub/{id}\`
- **vault**: \`/vault\` (no ID in URL)
-- **model**: \`/settings/manage-models\` (no ID in URL)
+- **model**: \`/manage-models\` (no ID in URL)
- **evaluation**: \`/evaluations\` (no ID in URL)
Rules:
diff --git a/src/client/src/lib/platform/chat/tools.ts b/src/client/src/lib/platform/chat/tools.ts
index 473a6800..7f7e16ca 100644
--- a/src/client/src/lib/platform/chat/tools.ts
+++ b/src/client/src/lib/platform/chat/tools.ts
@@ -28,7 +28,7 @@ import {
updateCustomModel,
deleteCustomModel,
getCustomModels,
-} from "../openground/custom-models-service";
+} from "@/lib/platform/providers/models-service";
/**
* Normalize a vault key name to UPPER_SNAKE_CASE.
diff --git a/src/client/src/lib/platform/openground/evaluate.ts b/src/client/src/lib/platform/openground/evaluate.ts
index 69d665a2..31aa8f67 100644
--- a/src/client/src/lib/platform/openground/evaluate.ts
+++ b/src/client/src/lib/platform/openground/evaluate.ts
@@ -1,6 +1,6 @@
import { AISdkAdapter } from "./ai-sdk-adapter";
-import { getOpenGroundConfigWithSecret } from "./config";
-import { ProviderRegistry } from "./provider-registry";
+import { getOpenGroundConfigWithSecret } from "@/lib/platform/providers/config";
+import { ProviderRegistry } from "@/lib/platform/providers/provider-registry";
import { createOpengroundEvaluation } from "@/lib/platform/openground-clickhouse";
import type { ProviderResult } from "@/lib/platform/openground-clickhouse";
import { getSpecificPrompt } from "@/lib/platform/prompt";
@@ -82,18 +82,14 @@ async function calculateCost(
provider: string,
model: string,
promptTokens: number,
- completionTokens: number
+ completionTokens: number,
+ databaseConfigId: string
): Promise<number> {
- // Get pricing from registry
- const providerMetadata = await ProviderRegistry.getProviderById(provider);
- if (!providerMetadata) {
- // Use separate parameter to prevent log injection
- console.warn('Provider metadata not found for:', provider);
- return 0;
- }
-
- const modelMetadata = providerMetadata.supportedModels.find(
- (m) => m.id === model
+ // Look up the model directly from the openlit_provider_models table
+ const modelMetadata = await ProviderRegistry.getModel(
+ provider,
+ model,
+ databaseConfigId
);
if (!modelMetadata) {
// Use separate parameters to prevent log injection
@@ -161,7 +157,8 @@ async function evaluateProvider(
provider,
model,
result.usage.promptTokens,
- result.usage.completionTokens
+ result.usage.completionTokens,
+ databaseConfigId
);
return {
diff --git a/src/client/src/lib/platform/openground/provider-registry.ts b/src/client/src/lib/platform/openground/provider-registry.ts
deleted file mode 100644
index c64e9497..00000000
--- a/src/client/src/lib/platform/openground/provider-registry.ts
+++ /dev/null
@@ -1,986 +0,0 @@
-import { ProviderMetadata, ModelMetadata, ConfigField } from "@/types/openground";
-
-// Re-export types for convenience
-export type { ProviderMetadata, ModelMetadata, ConfigField };
-
-// Provider metadata registry
-const PROVIDER_METADATA: Record<string, ProviderMetadata> = {
- openai: {
- providerId: "openai",
- displayName: "OpenAI",
- description: "GPT models from OpenAI",
- requiresVault: true,
- supportedModels: [
- {
- id: "gpt-4o",
- displayName: "GPT-4o",
- contextWindow: 128000,
- inputPricePerMToken: 2.5,
- outputPricePerMToken: 10.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-4o-mini",
- displayName: "GPT-4o Mini",
- contextWindow: 128000,
- inputPricePerMToken: 0.15,
- outputPricePerMToken: 0.6,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-4-turbo",
- displayName: "GPT-4 Turbo",
- contextWindow: 128000,
- inputPricePerMToken: 10.0,
- outputPricePerMToken: 30.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-3.5-turbo",
- displayName: "GPT-3.5 Turbo",
- contextWindow: 16385,
- inputPricePerMToken: 0.5,
- outputPricePerMToken: 1.5,
- capabilities: ["function-calling", "streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature (0 = deterministic, 2 = very random)",
- },
- maxTokens: {
- min: 1,
- max: 16000,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- anthropic: {
- providerId: "anthropic",
- displayName: "Anthropic",
- description: "Claude models from Anthropic",
- requiresVault: true,
- supportedModels: [
- {
- id: "claude-3-5-sonnet-20240620",
- displayName: "Claude 3.5 Sonnet",
- contextWindow: 200000,
- inputPricePerMToken: 3.0,
- outputPricePerMToken: 15.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "claude-3-haiku-20240307",
- displayName: "Claude 3 Haiku",
- contextWindow: 200000,
- inputPricePerMToken: 0.25,
- outputPricePerMToken: 1.25,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "claude-3-sonnet-20240229",
- displayName: "Claude 3 Sonnet",
- contextWindow: 200000,
- inputPricePerMToken: 3.0,
- outputPricePerMToken: 15.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 8096,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- google: {
- providerId: "google",
- displayName: "Google AI",
- description: "Gemini models from Google",
- requiresVault: true,
- supportedModels: [
- {
- id: "gemini-1.5-pro",
- displayName: "Gemini 1.5 Pro",
- contextWindow: 2000000,
- inputPricePerMToken: 1.25,
- outputPricePerMToken: 5.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gemini-1.5-flash",
- displayName: "Gemini 1.5 Flash",
- contextWindow: 1000000,
- inputPricePerMToken: 0.075,
- outputPricePerMToken: 0.3,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gemini-1.0-pro",
- displayName: "Gemini 1.0 Pro",
- contextWindow: 32768,
- inputPricePerMToken: 0.5,
- outputPricePerMToken: 1.5,
- capabilities: ["function-calling", "streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 8192,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.95,
- description: "Nucleus sampling threshold",
- },
- },
- },
- mistral: {
- providerId: "mistral",
- displayName: "Mistral AI",
- description: "Mistral and Mixtral models",
- requiresVault: true,
- supportedModels: [
- {
- id: "mistral-large-latest",
- displayName: "Mistral Large",
- contextWindow: 128000,
- inputPricePerMToken: 2.0,
- outputPricePerMToken: 6.0,
- capabilities: ["function-calling", "streaming"],
- },
- {
- id: "mistral-medium-latest",
- displayName: "Mistral Medium",
- contextWindow: 32000,
- inputPricePerMToken: 2.7,
- outputPricePerMToken: 8.1,
- capabilities: ["streaming"],
- },
- {
- id: "mistral-small-latest",
- displayName: "Mistral Small",
- contextWindow: 32000,
- inputPricePerMToken: 0.2,
- outputPricePerMToken: 0.6,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.7,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 8192,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- groq: {
- providerId: "groq",
- displayName: "Groq",
- description: "Ultra-fast inference for open source models",
- requiresVault: true,
- supportedModels: [
- {
- id: "llama-3.3-70b-versatile",
- displayName: "Llama 3.3 70B",
- contextWindow: 128000,
- inputPricePerMToken: 0.59,
- outputPricePerMToken: 0.79,
- capabilities: ["streaming"],
- },
- {
- id: "llama-3.1-8b-instant",
- displayName: "Llama 3.1 8B Instant",
- contextWindow: 128000,
- inputPricePerMToken: 0.05,
- outputPricePerMToken: 0.08,
- capabilities: ["streaming"],
- },
- {
- id: "mixtral-8x7b-32768",
- displayName: "Mixtral 8x7B",
- contextWindow: 32768,
- inputPricePerMToken: 0.24,
- outputPricePerMToken: 0.24,
- capabilities: ["streaming"],
- },
- {
- id: "gemma2-9b-it",
- displayName: "Gemma 2 9B",
- contextWindow: 8192,
- inputPricePerMToken: 0.2,
- outputPricePerMToken: 0.2,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 32768,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- perplexity: {
- providerId: "perplexity",
- displayName: "Perplexity",
- description: "Models with online search capabilities",
- requiresVault: true,
- supportedModels: [
- {
- id: "sonar-pro",
- displayName: "Sonar Pro",
- contextWindow: 127072,
- inputPricePerMToken: 3.0,
- outputPricePerMToken: 15.0,
- capabilities: ["streaming", "search"],
- },
- {
- id: "sonar",
- displayName: "Sonar",
- contextWindow: 127072,
- inputPricePerMToken: 1.0,
- outputPricePerMToken: 1.0,
- capabilities: ["streaming", "search"],
- },
- {
- id: "sonar-reasoning",
- displayName: "Sonar Reasoning",
- contextWindow: 127072,
- inputPricePerMToken: 5.0,
- outputPricePerMToken: 5.0,
- capabilities: ["streaming", "search", "reasoning"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 0.2,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 4096,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.9,
- description: "Nucleus sampling threshold",
- },
- },
- },
- azure: {
- providerId: "azure",
- displayName: "Azure OpenAI",
- description: "OpenAI models via Azure",
- requiresVault: true,
- supportedModels: [
- {
- id: "gpt-4o",
- displayName: "GPT-4o",
- contextWindow: 128000,
- inputPricePerMToken: 2.5,
- outputPricePerMToken: 10.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-4o-mini",
- displayName: "GPT-4o Mini",
- contextWindow: 128000,
- inputPricePerMToken: 0.15,
- outputPricePerMToken: 0.6,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-4-turbo",
- displayName: "GPT-4 Turbo",
- contextWindow: 128000,
- inputPricePerMToken: 10.0,
- outputPricePerMToken: 30.0,
- capabilities: ["function-calling", "vision", "streaming"],
- },
- {
- id: "gpt-35-turbo",
- displayName: "GPT-3.5 Turbo",
- contextWindow: 16385,
- inputPricePerMToken: 0.5,
- outputPricePerMToken: 1.5,
- capabilities: ["function-calling", "streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 16000,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- cohere: {
- providerId: "cohere",
- displayName: "Cohere",
- description: "Command models with RAG capabilities",
- requiresVault: true,
- supportedModels: [
- {
- id: "command-r-plus",
- displayName: "Command R+",
- contextWindow: 128000,
- inputPricePerMToken: 3.0,
- outputPricePerMToken: 15.0,
- capabilities: ["function-calling", "streaming", "rag"],
- },
- {
- id: "command-r",
- displayName: "Command R",
- contextWindow: 128000,
- inputPricePerMToken: 0.5,
- outputPricePerMToken: 1.5,
- capabilities: ["function-calling", "streaming", "rag"],
- },
- {
- id: "command",
- displayName: "Command",
- contextWindow: 4096,
- inputPricePerMToken: 1.0,
- outputPricePerMToken: 2.0,
- capabilities: ["streaming"],
- },
- {
- id: "command-light",
- displayName: "Command Light",
- contextWindow: 4096,
- inputPricePerMToken: 0.3,
- outputPricePerMToken: 0.6,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 5,
- step: 0.1,
- default: 0.3,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 4096,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 0.99,
- step: 0.01,
- default: 0.75,
- description: "Nucleus sampling threshold",
- },
- },
- },
- together: {
- providerId: "together",
- displayName: "Together AI",
- description: "Fast inference for open source models",
- requiresVault: true,
- supportedModels: [
- {
- id: "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
- displayName: "Llama 3.1 70B Turbo",
- contextWindow: 131072,
- inputPricePerMToken: 0.88,
- outputPricePerMToken: 0.88,
- capabilities: ["streaming"],
- },
- {
- id: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
- displayName: "Llama 3.1 8B Turbo",
- contextWindow: 131072,
- inputPricePerMToken: 0.18,
- outputPricePerMToken: 0.18,
- capabilities: ["streaming"],
- },
- {
- id: "mistralai/Mixtral-8x7B-Instruct-v0.1",
- displayName: "Mixtral 8x7B",
- contextWindow: 32768,
- inputPricePerMToken: 0.6,
- outputPricePerMToken: 0.6,
- capabilities: ["streaming"],
- },
- {
- id: "Qwen/Qwen2.5-72B-Instruct-Turbo",
- displayName: "Qwen 2.5 72B Turbo",
- contextWindow: 32768,
- inputPricePerMToken: 1.2,
- outputPricePerMToken: 1.2,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.7,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 8192,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.7,
- description: "Nucleus sampling threshold",
- },
- },
- },
- fireworks: {
- providerId: "fireworks",
- displayName: "Fireworks AI",
- description: "Production-ready LLM inference",
- requiresVault: true,
- supportedModels: [
- {
- id: "accounts/fireworks/models/llama-v3p1-70b-instruct",
- displayName: "Llama 3.1 70B",
- contextWindow: 131072,
- inputPricePerMToken: 0.9,
- outputPricePerMToken: 0.9,
- capabilities: ["streaming"],
- },
- {
- id: "accounts/fireworks/models/llama-v3p1-8b-instruct",
- displayName: "Llama 3.1 8B",
- contextWindow: 131072,
- inputPricePerMToken: 0.2,
- outputPricePerMToken: 0.2,
- capabilities: ["streaming"],
- },
- {
- id: "accounts/fireworks/models/mixtral-8x7b-instruct",
- displayName: "Mixtral 8x7B",
- contextWindow: 32768,
- inputPricePerMToken: 0.5,
- outputPricePerMToken: 0.5,
- capabilities: ["streaming"],
- },
- {
- id: "accounts/fireworks/models/qwen2p5-72b-instruct",
- displayName: "Qwen 2.5 72B",
- contextWindow: 32768,
- inputPricePerMToken: 0.9,
- outputPricePerMToken: 0.9,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 16384,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- deepseek: {
- providerId: "deepseek",
- displayName: "DeepSeek",
- description: "Advanced reasoning and coding models",
- requiresVault: true,
- supportedModels: [
- {
- id: "deepseek-chat",
- displayName: "DeepSeek Chat",
- contextWindow: 64000,
- inputPricePerMToken: 0.27,
- outputPricePerMToken: 1.1,
- capabilities: ["streaming", "reasoning"],
- },
- {
- id: "deepseek-coder",
- displayName: "DeepSeek Coder",
- contextWindow: 16000,
- inputPricePerMToken: 0.27,
- outputPricePerMToken: 1.1,
- capabilities: ["streaming", "coding"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 1,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 4096,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- xai: {
- providerId: "xai",
- displayName: "xAI",
- description: "Grok models with real-time knowledge",
- requiresVault: true,
- supportedModels: [
- {
- id: "grok-beta",
- displayName: "Grok Beta",
- contextWindow: 131072,
- inputPricePerMToken: 5.0,
- outputPricePerMToken: 15.0,
- capabilities: ["streaming", "real-time"],
- },
- {
- id: "grok-2",
- displayName: "Grok 2",
- contextWindow: 131072,
- inputPricePerMToken: 5.0,
- outputPricePerMToken: 15.0,
- capabilities: ["streaming", "real-time"],
- },
- {
- id: "grok-3",
- displayName: "Grok 3",
- contextWindow: 131072,
- inputPricePerMToken: 5.0,
- outputPricePerMToken: 15.0,
- capabilities: ["streaming", "real-time"],
- },
- {
- id: "grok-3-mini",
- displayName: "Grok 3 Mini",
- contextWindow: 131072,
- inputPricePerMToken: 1.0,
- outputPricePerMToken: 3.0,
- capabilities: ["streaming", "real-time"],
- },
- {
- id: "grok-vision-beta",
- displayName: "Grok Vision Beta",
- contextWindow: 8192,
- inputPricePerMToken: 5.0,
- outputPricePerMToken: 15.0,
- capabilities: ["streaming", "vision", "real-time"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 0,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 131072,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 1,
- description: "Nucleus sampling threshold",
- },
- },
- },
- huggingface: {
- providerId: "huggingface",
- displayName: "Hugging Face",
- description: "Open source models via Inference API",
- requiresVault: true,
- supportedModels: [
- {
- id: "meta-llama/Meta-Llama-3-70B-Instruct",
- displayName: "Llama 3 70B",
- contextWindow: 8192,
- inputPricePerMToken: 0.0,
- outputPricePerMToken: 0.0,
- capabilities: ["streaming"],
- },
- {
- id: "mistralai/Mistral-7B-Instruct-v0.3",
- displayName: "Mistral 7B",
- contextWindow: 32768,
- inputPricePerMToken: 0.0,
- outputPricePerMToken: 0.0,
- capabilities: ["streaming"],
- },
- {
- id: "google/gemma-2-9b-it",
- displayName: "Gemma 2 9B",
- contextWindow: 8192,
- inputPricePerMToken: 0.0,
- outputPricePerMToken: 0.0,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 2,
- step: 0.1,
- default: 0.7,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 2048,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.1,
- default: 0.95,
- description: "Nucleus sampling threshold",
- },
- },
- },
- replicate: {
- providerId: "replicate",
- displayName: "Replicate",
- description: "Run open source models in the cloud",
- requiresVault: true,
- supportedModels: [
- {
- id: "meta/meta-llama-3-70b-instruct",
- displayName: "Llama 3 70B",
- contextWindow: 8192,
- inputPricePerMToken: 0.65,
- outputPricePerMToken: 2.75,
- capabilities: ["streaming"],
- },
- {
- id: "meta/meta-llama-3-8b-instruct",
- displayName: "Llama 3 8B",
- contextWindow: 8192,
- inputPricePerMToken: 0.05,
- outputPricePerMToken: 0.25,
- capabilities: ["streaming"],
- },
- {
- id: "mistralai/mixtral-8x7b-instruct-v0.1",
- displayName: "Mixtral 8x7B",
- contextWindow: 32768,
- inputPricePerMToken: 0.3,
- outputPricePerMToken: 1.0,
- capabilities: ["streaming"],
- },
- ],
- configSchema: {
- temperature: {
- min: 0,
- max: 5,
- step: 0.01,
- default: 0.75,
- description: "Sampling temperature",
- },
- maxTokens: {
- min: 1,
- max: 4096,
- step: 1,
- default: 1000,
- description: "Maximum tokens to generate",
- },
- topP: {
- min: 0,
- max: 1,
- step: 0.01,
- default: 0.9,
- description: "Nucleus sampling threshold",
- },
- },
- },
-};
-
-export class ProviderRegistry {
- /**
- * Get all available providers
- */
- static async getAvailableProviders(): Promise<ProviderMetadata[]> {
- return Object.values(PROVIDER_METADATA);
- }
-
- /**
- * Get provider by ID
- */
- static async getProviderById(
- providerId: string
- ): Promise<ProviderMetadata | null> {
- return PROVIDER_METADATA[providerId] || null;
- }
-
- /**
- * Get provider by ID with custom models included
- */
- static async getProviderByIdWithCustomModels(
- providerId: string,
- userId: string,
- databaseConfigId: string
- ): Promise<ProviderMetadata | null> {
- const provider = PROVIDER_METADATA[providerId];
- if (!provider) return null;
-
- // Load custom models from database
- const customModels = await this.getCustomModels(
- providerId,
- userId,
- databaseConfigId
- );
-
- // Merge custom models with static models
- return {
- ...provider,
- supportedModels: [...provider.supportedModels, ...customModels],
- };
- }
-
- /**
- * Get all supported models across all providers
- */
- static async getAllModels(): Promise<
- Array<ModelMetadata & { providerId: string; providerName: string }>
- > {
- const models: Array<
- ModelMetadata & { providerId: string; providerName: string }
- > = [];
-
- for (const provider of Object.values(PROVIDER_METADATA)) {
- for (const model of provider.supportedModels) {
- models.push({
- ...model,
- providerId: provider.providerId,
- providerName: provider.displayName,
- });
- }
- }
-
- return models;
- }
-
- /**
- * Search providers by name or description
- */
- static async searchProviders(query: string): Promise<ProviderMetadata[]> {
- const lowerQuery = query.toLowerCase();
- return Object.values(PROVIDER_METADATA).filter(
- (provider) =>
- provider.displayName.toLowerCase().includes(lowerQuery) ||
- provider.description?.toLowerCase().includes(lowerQuery)
- );
- }
-
- /**
- * Get models for a specific provider (static models only)
- */
- static async getProviderModels(providerId: string): Promise<ModelMetadata[]> {
- const provider = PROVIDER_METADATA[providerId];
- return provider?.supportedModels || [];
- }
-
- /**
- * Get models for a specific provider including custom models
- */
- static async getProviderModelsWithCustom(
- providerId: string,
- userId: string,
- databaseConfigId: string
- ): Promise<ModelMetadata[]> {
- const provider = PROVIDER_METADATA[providerId];
- if (!provider) return [];
-
- // Load custom models from database
- const customModels = await this.getCustomModels(
- providerId,
- userId,
- databaseConfigId
- );
-
- // Merge static and custom models
- return [...provider.supportedModels, ...customModels];
- }
-
- /**
- * Load custom models from database for a specific provider
- */
- private static async getCustomModels(
- providerId: string,
- userId: string,
- databaseConfigId: string
- ): Promise<ModelMetadata[]> {
- try {
- // Import dynamically to avoid circular dependencies
- const { dataCollector } = await import("@/lib/platform/common");
- const { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } = await import(
- "@/lib/platform/openground/table-details"
- );
- const Sanitizer = (await import("@/utils/sanitizer")).default;
-
- const query = `
- SELECT
- model_id as id,
- display_name as displayName,
- context_window as contextWindow,
- input_price_per_m_token as inputPricePerMToken,
- output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
- AND provider = '${Sanitizer.sanitizeValue(providerId)}'
- ORDER BY created_at DESC
- `;
-
- const { data } = await dataCollector(
- { query },
- "query",
- databaseConfigId
- );
-
- return (data as ModelMetadata[]) || [];
- } catch (error) {
- console.error("Error loading custom models:", error);
- return [];
- }
- }
-}
diff --git a/src/client/src/lib/platform/openground/provider-service.ts b/src/client/src/lib/platform/openground/provider-service.ts
deleted file mode 100644
index 874e9d21..00000000
--- a/src/client/src/lib/platform/openground/provider-service.ts
+++ /dev/null
@@ -1,148 +0,0 @@
-import { ProviderRegistry } from "./provider-registry";
-import { getCustomModelsForProvider, getCustomModelsGroupedByProvider } from "./custom-models-service";
-import { ProviderMetadata } from "@/types/openground";
-
-/**
- * Get all available providers with custom models merged in
- */
-export async function getAllProvidersWithCustomModels(
- userId: string,
- databaseConfigId: string
-): Promise<{ data?: ProviderMetadata[]; err?: string }> {
- try {
- // Get static providers
- const providers = await ProviderRegistry.getAvailableProviders();
-
- // Get all custom models grouped by provider
- const { data: customModelsByProvider, err } = await getCustomModelsGroupedByProvider(
- userId,
- databaseConfigId
- );
-
- if (err) {
- return { err };
- }
-
- // Merge custom models into each provider
- const providersWithCustomModels = providers.map((provider) => {
- const customModels = customModelsByProvider?.[provider.providerId] || [];
-
- // Transform custom models to ModelMetadata format
- const customModelsFormatted = customModels.map((model) => ({
- id: model.model_id, // Use model_id as id for consistency
- displayName: model.displayName,
- contextWindow: model.contextWindow,
- inputPricePerMToken: model.inputPricePerMToken,
- outputPricePerMToken: model.outputPricePerMToken,
- capabilities: model.capabilities,
- }));
-
- return {
- ...provider,
- supportedModels: [
- ...provider.supportedModels,
- ...customModelsFormatted,
- ],
- };
- });
-
- return { data: providersWithCustomModels };
- } catch (error) {
- console.error("Error getting providers with custom models:", error);
- return { err: "Failed to load providers" };
- }
-}
-
-/**
- * Get a specific provider by ID with custom models merged in
- */
-export async function getProviderByIdWithCustomModels(
- providerId: string,
- userId: string,
- databaseConfigId: string
-): Promise<{ data?: ProviderMetadata; err?: string }> {
- try {
- // Get static provider
- const provider = await ProviderRegistry.getProviderById(providerId);
- if (!provider) {
- return { err: "Provider not found" };
- }
-
- // Get custom models for this provider
- const { data: customModels, err } = await getCustomModelsForProvider(
- userId,
- databaseConfigId,
- providerId
- );
-
- if (err) {
- return { err };
- }
-
- // Merge custom models with static models
- return {
- data: {
- ...provider,
- supportedModels: [
- ...provider.supportedModels,
- ...(customModels || []),
- ],
- },
- };
- } catch (error) {
- console.error("Error getting provider by ID with custom models:", error);
- return { err: "Failed to load provider" };
- }
-}
-
-/**
- * Search providers by query with custom models merged in
- */
-export async function searchProvidersWithCustomModels(
- query: string,
- userId: string,
- databaseConfigId: string
-): Promise<{ data?: ProviderMetadata[]; err?: string }> {
- try {
- // Search static providers
- const providers = await ProviderRegistry.searchProviders(query);
-
- // Get all custom models grouped by provider
- const { data: customModelsByProvider, err } = await getCustomModelsGroupedByProvider(
- userId,
- databaseConfigId
- );
-
- if (err) {
- return { err };
- }
-
- // Merge custom models into each provider
- const providersWithCustomModels = providers.map((provider) => {
- const customModels = customModelsByProvider?.[provider.providerId] || [];
-
- // Transform custom models to ModelMetadata format
- const customModelsFormatted = customModels.map((model) => ({
- id: model.model_id,
- displayName: model.displayName,
- contextWindow: model.contextWindow,
- inputPricePerMToken: model.inputPricePerMToken,
- outputPricePerMToken: model.outputPricePerMToken,
- capabilities: model.capabilities,
- }));
-
- return {
- ...provider,
- supportedModels: [
- ...provider.supportedModels,
- ...customModelsFormatted,
- ],
- };
- });
-
- return { data: providersWithCustomModels };
- } catch (error) {
- console.error("Error searching providers with custom models:", error);
- return { err: "Failed to search providers" };
- }
-}
diff --git a/src/client/src/lib/platform/openground/table-details.ts b/src/client/src/lib/platform/openground/table-details.ts
index 8a19efc4..55e1ebd1 100644
--- a/src/client/src/lib/platform/openground/table-details.ts
+++ b/src/client/src/lib/platform/openground/table-details.ts
@@ -1,4 +1,7 @@
+// Openground evaluation tables (stay in openground)
export const OPENLIT_OPENGROUND_TABLE_NAME = "openlit_openground";
export const OPENLIT_OPENGROUND_PROVIDERS_TABLE_NAME = "openlit_openground_providers";
+
+// Legacy table names (kept for migration reference only)
export const OPENLIT_OPENGROUND_CONFIG_TABLE_NAME = "openlit_openground_configs";
export const OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME = "openlit_openground_custom_models";
diff --git a/src/client/src/lib/platform/pricing/config.ts b/src/client/src/lib/platform/pricing/config.ts
new file mode 100644
index 00000000..ca009c6b
--- /dev/null
+++ b/src/client/src/lib/platform/pricing/config.ts
@@ -0,0 +1,180 @@
+import getMessage from "@/constants/messages";
+import { getDBConfigByUser } from "@/lib/db-config";
+import prisma from "@/lib/prisma";
+import asaw from "@/utils/asaw";
+import { throwIfError } from "@/utils/error";
+import { DatabaseConfig } from "@/types/database-config";
+import Cron from "@/helpers/server/cron";
+import { jsonParse, jsonStringify } from "@/utils/json";
+import { merge } from "lodash";
+import { randomUUID } from "crypto";
+import path from "path";
+
+export interface PricingConfig {
+ id: string;
+ databaseConfigId: string;
+ auto: boolean;
+ recurringTime: string;
+ meta: string;
+}
+
+export interface PricingConfigInput {
+ id?: string;
+ auto: boolean;
+ recurringTime: string;
+ meta: string;
+}
+
+/**
+ * Get the pricing config for the current user's database.
+ * Returns an empty shell (no id) if nothing exists yet — pricing is optional.
+ */
+export async function getPricingConfig(
+ dbConfig?: DatabaseConfig
+): Promise<PricingConfig | null> {
+ let updatedDBConfig: DatabaseConfig | undefined = dbConfig;
+ if (!dbConfig?.id) {
+ [, updatedDBConfig] = await asaw(getDBConfigByUser(true));
+ }
+
+ if (!updatedDBConfig?.id) return null;
+
+ const [, config] = await asaw(
+ prisma.pricingConfigs.findFirst({
+ where: { databaseConfigId: updatedDBConfig.id },
+ })
+ );
+
+ return (config as PricingConfig) || null;
+}
+
+export async function getPricingConfigById(
+ id: string
+): Promise<PricingConfig | null> {
+ const [, config] = await asaw(
+ prisma.pricingConfigs.findFirst({ where: { id } })
+ );
+ return (config as PricingConfig) || null;
+}
+
+/**
+ * Create or update pricing config, and wire up cron if auto is enabled.
+ */
+export async function setPricingConfig(
+ pricingConfig: PricingConfigInput,
+ apiURL: string
+) {
+ const [, dbConfig] = await asaw(getDBConfigByUser(true));
+ throwIfError(!dbConfig?.id, getMessage().DATABASE_CONFIG_NOT_FOUND);
+
+ const cronObject = new Cron();
+ if (pricingConfig.auto) {
+ cronObject.validateCronSchedule(pricingConfig.recurringTime);
+ }
+
+ let previousConfig: PricingConfig | null = null;
+ let cronJobId: string | undefined;
+ let pricingConfigId: string | undefined;
+ let data: any;
+ let err: any;
+
+ if (pricingConfig.id) {
+ previousConfig = await getPricingConfigById(pricingConfig.id);
+ pricingConfigId = previousConfig?.id;
+ const meta = jsonParse(previousConfig?.meta || "{}") as Record<string, any>;
+ cronJobId = meta?.cronJobId || randomUUID();
+ pricingConfig.meta = jsonStringify({ ...meta, cronJobId });
+ pricingConfig = merge(previousConfig || {}, pricingConfig) as PricingConfigInput;
+ [err, data] = await asaw(
+ prisma.pricingConfigs.update({
+ data: {
+ auto: pricingConfig.auto,
+ recurringTime: pricingConfig.recurringTime,
+ meta: pricingConfig.meta,
+ },
+ where: { id: pricingConfig.id },
+ })
+ );
+ } else {
+ cronJobId = randomUUID();
+ const meta = jsonParse(pricingConfig.meta || "{}") as Record<string, any>;
+ pricingConfig.meta = jsonStringify({ ...meta, cronJobId });
+ [err, data] = await asaw(
+ prisma.pricingConfigs.create({
+ data: {
+ auto: pricingConfig.auto,
+ recurringTime: pricingConfig.recurringTime,
+ meta: pricingConfig.meta,
+ databaseConfigId: dbConfig.id,
+ },
+ })
+ );
+ pricingConfigId = data?.id;
+ }
+
+ throwIfError(err, getMessage().OPERATION_FAILED);
+
+ try {
+ if (pricingConfig.auto) {
+ await cronObject.updateCrontab({
+ cronId: cronJobId!,
+ cronSchedule: pricingConfig.recurringTime,
+ cronEnvVars: {
+ PRICING_CONFIG_ID: pricingConfigId!,
+ API_URL: apiURL,
+ },
+ cronScriptPath: path.join(process.cwd(), "scripts/pricing/auto.js"),
+ cronLogPath: path.join(process.cwd(), "logs/pricing/auto.log"),
+ });
+ } else {
+ await cronObject.deleteCronJob(cronJobId!);
+ }
+ } catch (error) {
+ console.error(getMessage().CRON_JOB_UPDATION_ERROR, error);
+ throw error;
+ }
+
+ return data;
+}
+
+/**
+ * Restore cron jobs for pricing configs with auto=true on startup.
+ */
+export async function restorePricingCronJobs(apiURL: string) {
+ try {
+ const configs = await prisma.pricingConfigs.findMany({
+ where: { auto: true },
+ });
+
+ if (!configs?.length) {
+ console.log("No auto-pricing configs to restore");
+ return;
+ }
+
+ const cronObject = new Cron();
+
+ for (const config of configs) {
+ try {
+ const meta = jsonParse(config.meta || "{}") as Record<string, any>;
+ const cronJobId = meta?.cronJobId;
+ if (!cronJobId || !config.recurringTime) continue;
+
+ cronObject.updateCrontab({
+ cronId: cronJobId,
+ cronSchedule: config.recurringTime,
+ cronEnvVars: {
+ PRICING_CONFIG_ID: config.id,
+ API_URL: apiURL,
+ },
+ cronScriptPath: path.join(process.cwd(), "scripts/pricing/auto.js"),
+ cronLogPath: path.join(process.cwd(), "logs/pricing/auto.log"),
+ });
+ console.log(`Restored cron job for pricing config ${config.id}`);
+ } catch (e) {
+ console.error(`Failed to restore cron job for pricing config ${config.id}:`, e);
+ }
+ }
+ } catch (e) {
+ console.error("Failed to restore pricing cron jobs:", e);
+ }
+}
diff --git a/src/client/src/lib/platform/pricing/index.ts b/src/client/src/lib/platform/pricing/index.ts
new file mode 100644
index 00000000..51e373e0
--- /dev/null
+++ b/src/client/src/lib/platform/pricing/index.ts
@@ -0,0 +1,281 @@
+import getMessage from "@/constants/messages";
+import { getCurrentUser } from "@/lib/session";
+import { throwIfError } from "@/utils/error";
+import { dataCollector, OTEL_TRACES_TABLE_NAME } from "@/lib/platform/common";
+import { getTraceMappingKeyFullPath } from "@/helpers/server/trace";
+import { SUPPORTED_EVALUATION_OPERATIONS } from "@/constants/traces";
+import { getDBConfigById } from "@/lib/db-config";
+import { getRequestViaSpanId } from "@/lib/platform/request";
+import { ProviderRegistry } from "@/lib/platform/providers/provider-registry";
+import { getPricingConfigById } from "./config";
+import { getLastRunCronLogByCronId, insertCronLog } from "@/lib/platform/cron-log";
+import { CronRunStatus, CronType } from "@/types/cron";
+import { differenceInSeconds } from "date-fns";
+import Sanitizer from "@/utils/sanitizer";
+import asaw from "@/utils/asaw";
+
+const COST_KEY = getTraceMappingKeyFullPath("cost") as string; // gen_ai.usage.cost
+const MODEL_KEY = getTraceMappingKeyFullPath("model") as string; // gen_ai.request.model
+const PROVIDER_KEY = getTraceMappingKeyFullPath("provider") as string; // gen_ai.system
+const PROMPT_TOKENS_KEY = getTraceMappingKeyFullPath("promptTokens") as string; // gen_ai.usage.input_tokens
+const COMPLETION_TOKENS_KEY = getTraceMappingKeyFullPath("completionTokens") as string; // gen_ai.usage.output_tokens
+const TYPE_KEY = getTraceMappingKeyFullPath("type") as string; // gen_ai.operation.name
+
+interface TraceRow {
+ SpanId: string;
+ Timestamp: string;
+ SpanAttributes: Record<string, string>;
+}
+
+function getAttr(trace: TraceRow, key: string): string {
+ return (trace.SpanAttributes || {})[key] ?? "";
+}
+
+/**
+ * Compute the cost for a single trace by looking up the model in
+ * openlit_provider_models and applying token-based pricing.
+ * Returns null if pricing can't be determined (missing model/tokens).
+ */
+async function computeCostForTrace(
+ trace: TraceRow,
+ databaseConfigId: string
+): Promise<{ cost: number | null; reason?: string }> {
+ const provider = getAttr(trace, PROVIDER_KEY);
+ const model = getAttr(trace, MODEL_KEY);
+ const promptTokens = Number(getAttr(trace, PROMPT_TOKENS_KEY)) || 0;
+ const completionTokens = Number(getAttr(trace, COMPLETION_TOKENS_KEY)) || 0;
+
+ const allKeys = Object.keys(trace.SpanAttributes || {});
+ const genAiKeys = allKeys.filter((k) => k.startsWith("gen_ai"));
+
+ if (!provider || !model) {
+ const reason = `Missing ${!provider ? "provider" : ""}${
+ !provider && !model ? " and " : ""
+ }${!model ? "model" : ""} attribute on the trace (provider='${provider}', model='${model}')`;
+ return { cost: null, reason };
+ }
+ if (promptTokens === 0 && completionTokens === 0) {
+ const reason = `Trace has zero tokens (prompt=${promptTokens}, completion=${completionTokens})`;
+ return { cost: null, reason };
+ }
+
+ const modelMeta = await ProviderRegistry.getModel(
+ provider,
+ model,
+ databaseConfigId
+ );
+ if (!modelMeta) {
+ const reason = `Model '${model}' not found under provider '${provider}' in openlit_provider_models. Add it in Manage Models.`;
+ return { cost: null, reason };
+ }
+
+ const inputCost = (promptTokens / 1_000_000) * modelMeta.inputPricePerMToken;
+ const outputCost =
+ (completionTokens / 1_000_000) * modelMeta.outputPricePerMToken;
+
+ return { cost: inputCost + outputCost };
+}
+
+/**
+ * Update the gen_ai.usage.cost attribute on a specific span in otel_traces.
+ * Uses ClickHouse's mapUpdate (adds/replaces a key in a Map column).
+ */
+async function writeCostToTrace(
+ spanId: string,
+ cost: number,
+ databaseConfigId: string
+): Promise<{ err?: string }> {
+ const sanitizedSpanId = Sanitizer.sanitizeValue(spanId);
+ // toString keeps the attribute format consistent with how SDKs write it
+ const costString = cost.toFixed(10);
+ const query = `
+ ALTER TABLE ${OTEL_TRACES_TABLE_NAME}
+ UPDATE SpanAttributes = mapUpdate(SpanAttributes, map('${COST_KEY}', '${costString}'))
+ WHERE SpanId = '${sanitizedSpanId}'
+ `;
+
+ const { err } = await dataCollector({ query }, "exec", databaseConfigId);
+ return { err: err as string | undefined };
+}
+
+/**
+ * Manually recalculate + persist cost for a single span.
+ * Exposed via POST /api/pricing/[spanId].
+ */
+export async function setPricingForSpanId(spanId: string) {
+ const user = await getCurrentUser();
+ throwIfError(!user, getMessage().UNAUTHORIZED_USER);
+
+ const sanitizedSpanId = Sanitizer.sanitizeValue(spanId);
+ const { record: spanData, err: traceErr } = await getRequestViaSpanId(
+ sanitizedSpanId
+ );
+ throwIfError(!!traceErr, getMessage().TRACE_NOT_FOUND);
+ throwIfError(
+ !(spanData as any)?.SpanId,
+ getMessage().TRACE_NOT_FOUND
+ );
+
+ const trace = spanData as TraceRow;
+
+ // The trace itself doesn't tell us the dbConfig; fall back to the default
+ const { default: prisma } = await import("@/lib/prisma");
+ const pricingConfig = await prisma.pricingConfigs.findFirst();
+ const dbConfigId =
+ pricingConfig?.databaseConfigId ||
+ (await (async () => {
+ const { getDBConfigByUser } = await import("@/lib/db-config");
+ const [, dbc] = await asaw(getDBConfigByUser(true));
+ return dbc?.id;
+ })());
+
+ if (!dbConfigId) {
+ return { success: false, err: getMessage().DATABASE_CONFIG_NOT_FOUND };
+ }
+
+ const { cost, reason } = await computeCostForTrace(trace, dbConfigId);
+ if (cost === null) {
+ return {
+ success: false,
+ err:
+ reason ||
+ "Could not compute cost — missing provider/model/tokens or model not in openlit_provider_models",
+ };
+ }
+
+ const { err } = await writeCostToTrace(trace.SpanId, cost, dbConfigId);
+ if (err) return { success: false, err };
+
+ return { success: true, data: { spanId: trace.SpanId, cost } };
+}
+
+interface AutoPricingPayload {
+ pricingConfigId: string;
+ cronId: string;
+}
+
+/**
+ * Auto pricing: fetch traces in the window since last cron run and
+ * recompute + persist cost for each. Called by the /api/pricing/auto cron.
+ */
+export async function autoUpdatePricing(payload: AutoPricingPayload) {
+ const startedAt = new Date();
+ const cronLogObject = {
+ cronId: payload.cronId,
+ cronType: CronType.SPAN_PRICING,
+ metaProperties: { ...payload },
+ startedAt,
+ };
+
+ const pricingConfig = await getPricingConfigById(payload.pricingConfigId);
+ if (!pricingConfig) {
+ return { success: false, err: "Pricing config not found" };
+ }
+
+ const [dbConfigErr, dbConfig] = await asaw(
+ getDBConfigById({ id: pricingConfig.databaseConfigId })
+ );
+
+ if (dbConfigErr || !dbConfig?.id) {
+ return { success: false, err: getMessage().DATABASE_CONFIG_NOT_FOUND };
+ }
+
+ const lastRunTime = await getLastRunCronLogByCronId(payload.cronId);
+
+ // Only LLM-type spans with tokens recorded
+ const typeKeyPath = `SpanAttributes['${TYPE_KEY}']`;
+ const operationList = SUPPORTED_EVALUATION_OPERATIONS.map(
+ (op) => `'${op}'`
+ ).join(", ");
+
+ const query = `
+ SELECT SpanId, Timestamp, SpanAttributes
+ FROM ${OTEL_TRACES_TABLE_NAME}
+ WHERE ${typeKeyPath} IN (${operationList})
+ ${
+ lastRunTime
+ ? `AND Timestamp >= parseDateTimeBestEffort('${lastRunTime}')`
+ : ""
+ }
+ ORDER BY Timestamp
+ `;
+
+ const { data, err } = await dataCollector({ query }, "query", dbConfig.id);
+
+ if (err) {
+ const finishedAt = new Date();
+ await insertCronLog(
+ {
+ ...cronLogObject,
+ runStatus: CronRunStatus.FAILURE,
+ errorStacktrace: {
+ error: `${getMessage().TRACE_FETCHING_ERROR} : ${err}`,
+ },
+ finishedAt,
+ duration: differenceInSeconds(finishedAt, startedAt),
+ },
+ dbConfig.id
+ );
+ return { success: false, err: err as string };
+ }
+
+ const traces = (data as TraceRow[]) || [];
+ let errorCount = 0;
+ let updatedCount = 0;
+ const errorObject: Record<string, string> = {};
+
+ // Run sequentially to avoid hammering ClickHouse with many ALTER mutations
+ for (const trace of traces) {
+ try {
+ const { cost } = await computeCostForTrace(trace, dbConfig.id);
+ if (cost === null) {
+ // Skip traces we can't price (no counting as error)
+ continue;
+ }
+ const { err: writeErr } = await writeCostToTrace(
+ trace.SpanId,
+ cost,
+ dbConfig.id
+ );
+ if (writeErr) {
+ errorCount++;
+ errorObject[trace.SpanId] = writeErr;
+ } else {
+ updatedCount++;
+ }
+ } catch (e: any) {
+ errorCount++;
+ errorObject[trace.SpanId] = e.message || String(e);
+ }
+ }
+
+ const finishedAt = new Date();
+ const totalProcessed = updatedCount + errorCount;
+ const runStatus =
+ totalProcessed === 0
+ ? CronRunStatus.SUCCESS
+ : errorCount === 0
+ ? CronRunStatus.SUCCESS
+ : errorCount === totalProcessed
+ ? CronRunStatus.FAILURE
+ : CronRunStatus.PARTIAL_SUCCESS;
+
+ const { err: cronLogErr } = await insertCronLog(
+ {
+ ...cronLogObject,
+ runStatus,
+ errorStacktrace: errorObject,
+ meta: {
+ totalSpans: traces.length,
+ totalUpdated: updatedCount,
+ totalFailed: errorCount,
+ totalSkipped: traces.length - totalProcessed,
+ },
+ finishedAt,
+ duration: differenceInSeconds(finishedAt, startedAt),
+ },
+ dbConfig.id
+ );
+
+ return { success: true, err: cronLogErr };
+}
diff --git a/src/client/src/lib/platform/openground/config.ts b/src/client/src/lib/platform/providers/config.ts
similarity index 86%
rename from src/client/src/lib/platform/openground/config.ts
rename to src/client/src/lib/platform/providers/config.ts
index fa61f2cc..6ef45dd8 100644
--- a/src/client/src/lib/platform/openground/config.ts
+++ b/src/client/src/lib/platform/providers/config.ts
@@ -2,12 +2,11 @@ import { dataCollector } from "@/lib/platform/common";
import getMessage from "@/constants/messages";
import Sanitizer from "@/utils/sanitizer";
import { OPENLIT_VAULT_TABLE_NAME } from "@/lib/platform/vault/table-details";
-import { OPENLIT_OPENGROUND_CONFIG_TABLE_NAME } from "./table-details";
+import { OPENLIT_PROVIDERS_TABLE_NAME } from "./table-details";
export interface OpenGroundConfigData {
id: string;
userId: string;
- databaseConfigId: string;
provider: string;
vaultId: string;
modelId?: string;
@@ -33,16 +32,14 @@ export async function getOpenGroundConfigs(
SELECT
id,
user_id as userId,
- database_config_id as databaseConfigId,
provider,
vault_id as vaultId,
model_id as modelId,
is_active as isActive,
created_at as createdAt,
updated_at as updatedAt
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
ORDER BY created_at DESC
`;
@@ -73,22 +70,19 @@ export async function getOpenGroundConfigWithSecret(
databaseConfigId: string
): Promise<{ data?: OpenGroundConfigWithSecret; err?: string }> {
try {
- // Get the config from ClickHouse
const configQuery = `
SELECT
id,
user_id as userId,
- database_config_id as databaseConfigId,
provider,
vault_id as vaultId,
model_id as modelId,
is_active as isActive,
created_at as createdAt,
updated_at as updatedAt
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE provider = '${Sanitizer.sanitizeValue(provider)}'
AND user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
AND is_active = true
LIMIT 1
`;
@@ -105,7 +99,6 @@ export async function getOpenGroundConfigWithSecret(
const config = (configData as any[])[0];
- // Fetch the API key from Vault (ClickHouse)
const vaultQuery = `
SELECT key, value
FROM ${OPENLIT_VAULT_TABLE_NAME}
@@ -151,12 +144,10 @@ export async function upsertOpenGroundConfig(data: {
isActive?: boolean;
}): Promise<{ data?: OpenGroundConfigData; err?: string }> {
try {
- // Check if config exists
const checkQuery = `
SELECT id
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE user_id = '${Sanitizer.sanitizeValue(data.userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(data.databaseConfigId)}'
AND provider = '${Sanitizer.sanitizeValue(data.provider)}'
LIMIT 1
`;
@@ -170,10 +161,9 @@ export async function upsertOpenGroundConfig(data: {
const exists = (existingData as any[])?.length > 0;
if (exists) {
- // Update existing config
const configId = (existingData as any[])[0].id;
const updateQuery = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ ALTER TABLE ${OPENLIT_PROVIDERS_TABLE_NAME}
UPDATE
vault_id = '${Sanitizer.sanitizeValue(data.vaultId)}',
model_id = ${data.modelId ? `'${Sanitizer.sanitizeValue(data.modelId)}'` : "NULL"},
@@ -193,17 +183,14 @@ export async function upsertOpenGroundConfig(data: {
return { err: getMessage().OPERATION_FAILED };
}
- // Fetch and return the updated config
return getOpenGroundConfigById(configId, data.databaseConfigId);
} else {
- // Insert new config
const { err: insertErr } = await dataCollector(
{
- table: OPENLIT_OPENGROUND_CONFIG_TABLE_NAME,
+ table: OPENLIT_PROVIDERS_TABLE_NAME,
values: [
{
user_id: data.userId,
- database_config_id: data.databaseConfigId,
provider: data.provider,
vault_id: data.vaultId,
model_id: data.modelId || null,
@@ -220,21 +207,19 @@ export async function upsertOpenGroundConfig(data: {
return { err: getMessage().OPERATION_FAILED };
}
- // Get the inserted config
const { data: lastInsert } = await dataCollector(
{
query: `
SELECT
id,
user_id as userId,
- database_config_id as databaseConfigId,
provider,
vault_id as vaultId,
model_id as modelId,
is_active as isActive,
created_at as createdAt,
updated_at as updatedAt
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE user_id = '${Sanitizer.sanitizeValue(data.userId)}'
AND provider = '${Sanitizer.sanitizeValue(data.provider)}'
ORDER BY created_at DESC
@@ -265,14 +250,13 @@ async function getOpenGroundConfigById(
SELECT
id,
user_id as userId,
- database_config_id as databaseConfigId,
provider,
vault_id as vaultId,
model_id as modelId,
is_active as isActive,
created_at as createdAt,
updated_at as updatedAt
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE id = '${Sanitizer.sanitizeValue(configId)}'
LIMIT 1
`;
@@ -300,7 +284,7 @@ export async function deleteOpenGroundConfig(
): Promise<{ data?: string; err?: string }> {
try {
const query = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ ALTER TABLE ${OPENLIT_PROVIDERS_TABLE_NAME}
DELETE WHERE id = '${Sanitizer.sanitizeValue(configId)}'
AND user_id = '${Sanitizer.sanitizeValue(userId)}'
`;
@@ -334,7 +318,7 @@ export async function toggleOpenGroundConfigStatus(
): Promise<{ data?: OpenGroundConfigData; err?: string }> {
try {
const query = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ ALTER TABLE ${OPENLIT_PROVIDERS_TABLE_NAME}
UPDATE
is_active = ${isActive},
updated_at = now()
@@ -349,7 +333,6 @@ export async function toggleOpenGroundConfigStatus(
return { err: getMessage().OPERATION_FAILED };
}
- // Fetch and return the updated config
return getOpenGroundConfigById(configId, databaseConfigId);
} catch (error: any) {
console.error("Error toggling OpenGround config status:", error);
@@ -367,9 +350,8 @@ export async function getActiveProviders(
try {
const query = `
SELECT DISTINCT provider
- FROM ${OPENLIT_OPENGROUND_CONFIG_TABLE_NAME}
+ FROM ${OPENLIT_PROVIDERS_TABLE_NAME}
WHERE user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
AND is_active = true
`;
diff --git a/src/client/src/lib/platform/providers/default-models.ts b/src/client/src/lib/platform/providers/default-models.ts
new file mode 100644
index 00000000..3e029cc8
--- /dev/null
+++ b/src/client/src/lib/platform/providers/default-models.ts
@@ -0,0 +1,434 @@
+import { ModelMetadata } from "@/types/openground";
+
+/**
+ * Default (built-in) models that are seeded into the openlit_provider_models table.
+ * These become editable once seeded — they are not read from here at runtime.
+ * This file is the source of truth for initial seeding and for reference.
+ */
+
+export interface DefaultModelEntry extends ModelMetadata {
+ modelType?: string;
+}
+
+export const DEFAULT_MODELS_BY_PROVIDER: Record<string, DefaultModelEntry[]> = {
+ openai: [
+ {
+ id: "gpt-4o",
+ displayName: "GPT-4o",
+ contextWindow: 128000,
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-4o-mini",
+ displayName: "GPT-4o Mini",
+ contextWindow: 128000,
+ inputPricePerMToken: 0.15,
+ outputPricePerMToken: 0.6,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-4-turbo",
+ displayName: "GPT-4 Turbo",
+ contextWindow: 128000,
+ inputPricePerMToken: 10.0,
+ outputPricePerMToken: 30.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-3.5-turbo",
+ displayName: "GPT-3.5 Turbo",
+ contextWindow: 16385,
+ inputPricePerMToken: 0.5,
+ outputPricePerMToken: 1.5,
+ capabilities: ["function-calling", "streaming"],
+ },
+ ],
+ anthropic: [
+ {
+ id: "claude-3-5-sonnet-20240620",
+ displayName: "Claude 3.5 Sonnet",
+ contextWindow: 200000,
+ inputPricePerMToken: 3.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "claude-3-haiku-20240307",
+ displayName: "Claude 3 Haiku",
+ contextWindow: 200000,
+ inputPricePerMToken: 0.25,
+ outputPricePerMToken: 1.25,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "claude-3-sonnet-20240229",
+ displayName: "Claude 3 Sonnet",
+ contextWindow: 200000,
+ inputPricePerMToken: 3.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ ],
+ google: [
+ {
+ id: "gemini-1.5-pro",
+ displayName: "Gemini 1.5 Pro",
+ contextWindow: 2000000,
+ inputPricePerMToken: 1.25,
+ outputPricePerMToken: 5.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gemini-1.5-flash",
+ displayName: "Gemini 1.5 Flash",
+ contextWindow: 1000000,
+ inputPricePerMToken: 0.075,
+ outputPricePerMToken: 0.3,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gemini-1.0-pro",
+ displayName: "Gemini 1.0 Pro",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.5,
+ outputPricePerMToken: 1.5,
+ capabilities: ["function-calling", "streaming"],
+ },
+ ],
+ mistral: [
+ {
+ id: "mistral-large-latest",
+ displayName: "Mistral Large",
+ contextWindow: 128000,
+ inputPricePerMToken: 2.0,
+ outputPricePerMToken: 6.0,
+ capabilities: ["function-calling", "streaming"],
+ },
+ {
+ id: "mistral-medium-latest",
+ displayName: "Mistral Medium",
+ contextWindow: 32000,
+ inputPricePerMToken: 2.7,
+ outputPricePerMToken: 8.1,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "mistral-small-latest",
+ displayName: "Mistral Small",
+ contextWindow: 32000,
+ inputPricePerMToken: 0.2,
+ outputPricePerMToken: 0.6,
+ capabilities: ["streaming"],
+ },
+ ],
+ groq: [
+ {
+ id: "llama-3.3-70b-versatile",
+ displayName: "Llama 3.3 70B",
+ contextWindow: 128000,
+ inputPricePerMToken: 0.59,
+ outputPricePerMToken: 0.79,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "llama-3.1-8b-instant",
+ displayName: "Llama 3.1 8B Instant",
+ contextWindow: 128000,
+ inputPricePerMToken: 0.05,
+ outputPricePerMToken: 0.08,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "mixtral-8x7b-32768",
+ displayName: "Mixtral 8x7B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.24,
+ outputPricePerMToken: 0.24,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "gemma2-9b-it",
+ displayName: "Gemma 2 9B",
+ contextWindow: 8192,
+ inputPricePerMToken: 0.2,
+ outputPricePerMToken: 0.2,
+ capabilities: ["streaming"],
+ },
+ ],
+ perplexity: [
+ {
+ id: "sonar-pro",
+ displayName: "Sonar Pro",
+ contextWindow: 127072,
+ inputPricePerMToken: 3.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["streaming", "search"],
+ },
+ {
+ id: "sonar",
+ displayName: "Sonar",
+ contextWindow: 127072,
+ inputPricePerMToken: 1.0,
+ outputPricePerMToken: 1.0,
+ capabilities: ["streaming", "search"],
+ },
+ {
+ id: "sonar-reasoning",
+ displayName: "Sonar Reasoning",
+ contextWindow: 127072,
+ inputPricePerMToken: 5.0,
+ outputPricePerMToken: 5.0,
+ capabilities: ["streaming", "search", "reasoning"],
+ },
+ ],
+ azure: [
+ {
+ id: "gpt-4o",
+ displayName: "GPT-4o",
+ contextWindow: 128000,
+ inputPricePerMToken: 2.5,
+ outputPricePerMToken: 10.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-4o-mini",
+ displayName: "GPT-4o Mini",
+ contextWindow: 128000,
+ inputPricePerMToken: 0.15,
+ outputPricePerMToken: 0.6,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-4-turbo",
+ displayName: "GPT-4 Turbo",
+ contextWindow: 128000,
+ inputPricePerMToken: 10.0,
+ outputPricePerMToken: 30.0,
+ capabilities: ["function-calling", "vision", "streaming"],
+ },
+ {
+ id: "gpt-35-turbo",
+ displayName: "GPT-3.5 Turbo",
+ contextWindow: 16385,
+ inputPricePerMToken: 0.5,
+ outputPricePerMToken: 1.5,
+ capabilities: ["function-calling", "streaming"],
+ },
+ ],
+ cohere: [
+ {
+ id: "command-r-plus",
+ displayName: "Command R+",
+ contextWindow: 128000,
+ inputPricePerMToken: 3.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["function-calling", "streaming", "rag"],
+ },
+ {
+ id: "command-r",
+ displayName: "Command R",
+ contextWindow: 128000,
+ inputPricePerMToken: 0.5,
+ outputPricePerMToken: 1.5,
+ capabilities: ["function-calling", "streaming", "rag"],
+ },
+ {
+ id: "command",
+ displayName: "Command",
+ contextWindow: 4096,
+ inputPricePerMToken: 1.0,
+ outputPricePerMToken: 2.0,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "command-light",
+ displayName: "Command Light",
+ contextWindow: 4096,
+ inputPricePerMToken: 0.3,
+ outputPricePerMToken: 0.6,
+ capabilities: ["streaming"],
+ },
+ ],
+ together: [
+ {
+ id: "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
+ displayName: "Llama 3.1 70B Turbo",
+ contextWindow: 131072,
+ inputPricePerMToken: 0.88,
+ outputPricePerMToken: 0.88,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
+ displayName: "Llama 3.1 8B Turbo",
+ contextWindow: 131072,
+ inputPricePerMToken: 0.18,
+ outputPricePerMToken: 0.18,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "mistralai/Mixtral-8x7B-Instruct-v0.1",
+ displayName: "Mixtral 8x7B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.6,
+ outputPricePerMToken: 0.6,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "Qwen/Qwen2.5-72B-Instruct-Turbo",
+ displayName: "Qwen 2.5 72B Turbo",
+ contextWindow: 32768,
+ inputPricePerMToken: 1.2,
+ outputPricePerMToken: 1.2,
+ capabilities: ["streaming"],
+ },
+ ],
+ fireworks: [
+ {
+ id: "accounts/fireworks/models/llama-v3p1-70b-instruct",
+ displayName: "Llama 3.1 70B",
+ contextWindow: 131072,
+ inputPricePerMToken: 0.9,
+ outputPricePerMToken: 0.9,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "accounts/fireworks/models/llama-v3p1-8b-instruct",
+ displayName: "Llama 3.1 8B",
+ contextWindow: 131072,
+ inputPricePerMToken: 0.2,
+ outputPricePerMToken: 0.2,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "accounts/fireworks/models/mixtral-8x7b-instruct",
+ displayName: "Mixtral 8x7B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.5,
+ outputPricePerMToken: 0.5,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "accounts/fireworks/models/qwen2p5-72b-instruct",
+ displayName: "Qwen 2.5 72B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.9,
+ outputPricePerMToken: 0.9,
+ capabilities: ["streaming"],
+ },
+ ],
+ deepseek: [
+ {
+ id: "deepseek-chat",
+ displayName: "DeepSeek Chat",
+ contextWindow: 64000,
+ inputPricePerMToken: 0.27,
+ outputPricePerMToken: 1.1,
+ capabilities: ["streaming", "reasoning"],
+ },
+ {
+ id: "deepseek-coder",
+ displayName: "DeepSeek Coder",
+ contextWindow: 16000,
+ inputPricePerMToken: 0.27,
+ outputPricePerMToken: 1.1,
+ capabilities: ["streaming", "coding"],
+ },
+ ],
+ xai: [
+ {
+ id: "grok-beta",
+ displayName: "Grok Beta",
+ contextWindow: 131072,
+ inputPricePerMToken: 5.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["streaming", "real-time"],
+ },
+ {
+ id: "grok-2",
+ displayName: "Grok 2",
+ contextWindow: 131072,
+ inputPricePerMToken: 5.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["streaming", "real-time"],
+ },
+ {
+ id: "grok-3",
+ displayName: "Grok 3",
+ contextWindow: 131072,
+ inputPricePerMToken: 5.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["streaming", "real-time"],
+ },
+ {
+ id: "grok-3-mini",
+ displayName: "Grok 3 Mini",
+ contextWindow: 131072,
+ inputPricePerMToken: 1.0,
+ outputPricePerMToken: 3.0,
+ capabilities: ["streaming", "real-time"],
+ },
+ {
+ id: "grok-vision-beta",
+ displayName: "Grok Vision Beta",
+ contextWindow: 8192,
+ inputPricePerMToken: 5.0,
+ outputPricePerMToken: 15.0,
+ capabilities: ["streaming", "vision", "real-time"],
+ },
+ ],
+ huggingface: [
+ {
+ id: "meta-llama/Meta-Llama-3-70B-Instruct",
+ displayName: "Llama 3 70B",
+ contextWindow: 8192,
+ inputPricePerMToken: 0.0,
+ outputPricePerMToken: 0.0,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "mistralai/Mistral-7B-Instruct-v0.3",
+ displayName: "Mistral 7B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.0,
+ outputPricePerMToken: 0.0,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "google/gemma-2-9b-it",
+ displayName: "Gemma 2 9B",
+ contextWindow: 8192,
+ inputPricePerMToken: 0.0,
+ outputPricePerMToken: 0.0,
+ capabilities: ["streaming"],
+ },
+ ],
+ replicate: [
+ {
+ id: "meta/meta-llama-3-70b-instruct",
+ displayName: "Llama 3 70B",
+ contextWindow: 8192,
+ inputPricePerMToken: 0.65,
+ outputPricePerMToken: 2.75,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "meta/meta-llama-3-8b-instruct",
+ displayName: "Llama 3 8B",
+ contextWindow: 8192,
+ inputPricePerMToken: 0.05,
+ outputPricePerMToken: 0.25,
+ capabilities: ["streaming"],
+ },
+ {
+ id: "mistralai/mixtral-8x7b-instruct-v0.1",
+ displayName: "Mixtral 8x7B",
+ contextWindow: 32768,
+ inputPricePerMToken: 0.3,
+ outputPricePerMToken: 1.0,
+ capabilities: ["streaming"],
+ },
+ ],
+};
diff --git a/src/client/src/lib/platform/openground/custom-models-service.ts b/src/client/src/lib/platform/providers/models-service.ts
similarity index 72%
rename from src/client/src/lib/platform/openground/custom-models-service.ts
rename to src/client/src/lib/platform/providers/models-service.ts
index 6ae3dec8..172a3f9f 100644
--- a/src/client/src/lib/platform/openground/custom-models-service.ts
+++ b/src/client/src/lib/platform/providers/models-service.ts
@@ -1,33 +1,33 @@
import { dataCollector } from "@/lib/platform/common";
import Sanitizer from "@/utils/sanitizer";
-import { OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME } from "./table-details";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "./table-details";
import { CustomModel, CustomModelInput, ModelMetadata } from "@/types/openground";
/**
- * Validate UUID format
+ * Model management service for openlit_provider_models.
+ *
+ * All models (seeded defaults + user-created) live in a single table per
+ * ClickHouse instance. Every model returned by this service is editable/deletable.
*/
-function isValidUUID(id: string): boolean {
- const uuidRegex = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
- return uuidRegex.test(id);
-}
/**
- * Get all custom models for a user, optionally filtered by provider
+ * Get all models for a database config, optionally filtered by provider.
*/
export async function getCustomModels(
userId: string,
databaseConfigId: string,
provider?: string
): Promise<{ data?: CustomModel[]; err?: string }> {
- const whereConditions = [
- `created_by_user_id = '${Sanitizer.sanitizeValue(userId)}'`,
- `database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'`,
- ];
+ const whereConditions: string[] = [];
if (provider) {
whereConditions.push(`provider = '${Sanitizer.sanitizeValue(provider)}'`);
}
+ const whereClause = whereConditions.length > 0
+ ? `WHERE ${whereConditions.join(" AND ")}`
+ : "";
+
const query = `
SELECT
toString(id) as id,
@@ -38,10 +38,11 @@ export async function getCustomModels(
context_window as contextWindow,
input_price_per_m_token as inputPricePerMToken,
output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE ${whereConditions.join(" AND ")}
- ORDER BY provider, created_at DESC
+ capabilities,
+ is_default as isDefault
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ ${whereClause}
+ ORDER BY provider, is_default DESC, created_at DESC
`;
const { data, err } = await dataCollector(
@@ -54,13 +55,11 @@ export async function getCustomModels(
return { err: err as string };
}
- // Return all models — don't filter by UUID validity since ClickHouse
- // may store UUIDs in non-standard string format depending on insert method
return { data: (data as CustomModel[]) || [] };
}
/**
- * Get all custom models grouped by provider
+ * Get all models grouped by provider.
*/
export async function getCustomModelsGroupedByProvider(
userId: string,
@@ -72,7 +71,6 @@ export async function getCustomModelsGroupedByProvider(
return { err };
}
- // Group by provider
const grouped: Record<string, CustomModel[]> = {};
(models || []).forEach((model) => {
if (!grouped[model.provider!]) {
@@ -85,22 +83,20 @@ export async function getCustomModelsGroupedByProvider(
}
/**
- * Create a new custom model
+ * Create a new model.
*/
export async function createCustomModel(
userId: string,
databaseConfigId: string,
input: CustomModelInput
): Promise<{ data?: CustomModel; err?: string }> {
- // Validate required fields
if (!input.provider || !input.model_id || !input.displayName) {
return { err: "Provider, model ID, and display name are required" };
}
- // Insert new model
const { err } = await dataCollector(
{
- table: OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME,
+ table: OPENLIT_PROVIDER_MODELS_TABLE_NAME,
values: [
{
provider: input.provider,
@@ -111,8 +107,8 @@ export async function createCustomModel(
input_price_per_m_token: input.inputPricePerMToken || 0,
output_price_per_m_token: input.outputPricePerMToken || 0,
capabilities: input.capabilities || [],
+ is_default: false,
created_by_user_id: userId,
- database_config_id: databaseConfigId,
},
],
},
@@ -124,7 +120,6 @@ export async function createCustomModel(
return { err: err as string };
}
- // Fetch the newly created model
const selectQuery = `
SELECT
toString(id) as id,
@@ -135,11 +130,10 @@ export async function createCustomModel(
context_window as contextWindow,
input_price_per_m_token as inputPricePerMToken,
output_price_per_m_token as outputPricePerMToken,
- capabilities
- FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
- WHERE created_by_user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
- AND provider = '${Sanitizer.sanitizeValue(input.provider)}'
+ capabilities,
+ is_default as isDefault
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ WHERE provider = '${Sanitizer.sanitizeValue(input.provider)}'
AND model_id = '${Sanitizer.sanitizeValue(input.model_id)}'
ORDER BY created_at DESC
LIMIT 1
@@ -159,7 +153,7 @@ export async function createCustomModel(
}
/**
- * Update an existing custom model
+ * Update an existing model. Matches on the (provider, model_id) composite key.
*/
export async function updateCustomModel(
userId: string,
@@ -167,7 +161,6 @@ export async function updateCustomModel(
id: string,
input: Partial<CustomModelInput>
): Promise<{ data?: boolean; err?: string }> {
- // Build update fields
const updateFields: string[] = [];
if (input.displayName !== undefined) {
@@ -199,11 +192,9 @@ export async function updateCustomModel(
}
const updateQuery = `
- ALTER TABLE ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
+ ALTER TABLE ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
UPDATE ${updateFields.join(", ")}
WHERE (toString(id) = '${Sanitizer.sanitizeValue(id)}' OR model_id = '${Sanitizer.sanitizeValue(id)}')
- AND created_by_user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
`;
const { err } = await dataCollector(
@@ -220,7 +211,7 @@ export async function updateCustomModel(
}
/**
- * Delete a custom model
+ * Delete a model.
*/
export async function deleteCustomModel(
userId: string,
@@ -228,10 +219,8 @@ export async function deleteCustomModel(
id: string
): Promise<{ data?: boolean; err?: string }> {
const deleteQuery = `
- DELETE FROM ${OPENLIT_OPENGROUND_CUSTOM_MODELS_TABLE_NAME}
+ DELETE FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
WHERE (toString(id) = '${Sanitizer.sanitizeValue(id)}' OR model_id = '${Sanitizer.sanitizeValue(id)}')
- AND created_by_user_id = '${Sanitizer.sanitizeValue(userId)}'
- AND database_config_id = '${Sanitizer.sanitizeValue(databaseConfigId)}'
`;
const { err } = await dataCollector(
@@ -248,8 +237,7 @@ export async function deleteCustomModel(
}
/**
- * Get custom models for a provider to merge with static models
- * Returns models in the format expected by provider metadata
+ * Get models for a provider (used when merging with provider metadata).
*/
export async function getCustomModelsForProvider(
userId: string,
@@ -266,9 +254,8 @@ export async function getCustomModelsForProvider(
return { err };
}
- // Transform to ModelMetadata format (using model_id as id)
const transformed = (models || []).map((model) => ({
- id: model.model_id, // Use model_id as id for consistency with static models
+ id: model.model_id,
displayName: model.displayName,
contextWindow: model.contextWindow,
inputPricePerMToken: model.inputPricePerMToken,
diff --git a/src/client/src/lib/platform/providers/provider-registry.ts b/src/client/src/lib/platform/providers/provider-registry.ts
new file mode 100644
index 00000000..5c381da9
--- /dev/null
+++ b/src/client/src/lib/platform/providers/provider-registry.ts
@@ -0,0 +1,343 @@
+import { ProviderMetadata, ModelMetadata, ConfigField } from "@/types/openground";
+import { dataCollector } from "@/lib/platform/common";
+import Sanitizer from "@/utils/sanitizer";
+import { OPENLIT_PROVIDER_MODELS_TABLE_NAME } from "./table-details";
+
+// Re-export types for convenience
+export type { ProviderMetadata, ModelMetadata, ConfigField };
+
+/**
+ * Static provider metadata — structural information only (name, description,
+ * configSchema, vault requirement). The list of supported models is NO LONGER
+ * stored here; models live in the openlit_provider_models ClickHouse table and
+ * are fully editable via the manage-models UI.
+ */
+type StaticProviderMetadata = Omit<ProviderMetadata, "supportedModels">;
+
+const PROVIDER_METADATA: Record<string, StaticProviderMetadata> = {
+ openai: {
+ providerId: "openai",
+ displayName: "OpenAI",
+ description: "GPT models from OpenAI",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature (0 = deterministic, 2 = very random)" },
+ maxTokens: { min: 1, max: 16000, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ anthropic: {
+ providerId: "anthropic",
+ displayName: "Anthropic",
+ description: "Claude models from Anthropic",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 1, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 8096, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ google: {
+ providerId: "google",
+ displayName: "Google AI",
+ description: "Gemini models from Google",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 8192, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 0.95, description: "Nucleus sampling threshold" },
+ },
+ },
+ mistral: {
+ providerId: "mistral",
+ displayName: "Mistral AI",
+ description: "Mistral and Mixtral models",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 1, step: 0.1, default: 0.7, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 8192, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ groq: {
+ providerId: "groq",
+ displayName: "Groq",
+ description: "Ultra-fast inference for open source models",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 32768, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ perplexity: {
+ providerId: "perplexity",
+ displayName: "Perplexity",
+ description: "Models with online search capabilities",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 0.2, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 4096, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 0.9, description: "Nucleus sampling threshold" },
+ },
+ },
+ azure: {
+ providerId: "azure",
+ displayName: "Azure OpenAI",
+ description: "OpenAI models via Azure",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 16000, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ cohere: {
+ providerId: "cohere",
+ displayName: "Cohere",
+ description: "Command models with RAG capabilities",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 5, step: 0.1, default: 0.3, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 4096, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 0.99, step: 0.01, default: 0.75, description: "Nucleus sampling threshold" },
+ },
+ },
+ together: {
+ providerId: "together",
+ displayName: "Together AI",
+ description: "Fast inference for open source models",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 1, step: 0.1, default: 0.7, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 8192, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 0.7, description: "Nucleus sampling threshold" },
+ },
+ },
+ fireworks: {
+ providerId: "fireworks",
+ displayName: "Fireworks AI",
+ description: "Production-ready LLM inference",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 16384, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ deepseek: {
+ providerId: "deepseek",
+ displayName: "DeepSeek",
+ description: "Advanced reasoning and coding models",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 1, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 4096, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ xai: {
+ providerId: "xai",
+ displayName: "xAI",
+ description: "Grok models with real-time knowledge",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 0, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 131072, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 1, description: "Nucleus sampling threshold" },
+ },
+ },
+ huggingface: {
+ providerId: "huggingface",
+ displayName: "Hugging Face",
+ description: "Open source models via Inference API",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 2, step: 0.1, default: 0.7, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 2048, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.1, default: 0.95, description: "Nucleus sampling threshold" },
+ },
+ },
+ replicate: {
+ providerId: "replicate",
+ displayName: "Replicate",
+ description: "Run open source models in the cloud",
+ requiresVault: true,
+ configSchema: {
+ temperature: { min: 0, max: 5, step: 0.01, default: 0.75, description: "Sampling temperature" },
+ maxTokens: { min: 1, max: 4096, step: 1, default: 1000, description: "Maximum tokens to generate" },
+ topP: { min: 0, max: 1, step: 0.01, default: 0.9, description: "Nucleus sampling threshold" },
+ },
+ },
+};
+
+/**
+ * Load all models for a database config from openlit_provider_models,
+ * grouped by providerId.
+ */
+async function loadAllModelsFromDb(
+ databaseConfigId: string
+): Promise<Record<string, ModelMetadata[]>> {
+ try {
+ const query = `
+ SELECT
+ provider,
+ model_id as id,
+ display_name as displayName,
+ model_type as modelType,
+ context_window as contextWindow,
+ input_price_per_m_token as inputPricePerMToken,
+ output_price_per_m_token as outputPricePerMToken,
+ capabilities
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ ORDER BY provider, is_default DESC, created_at DESC
+ `;
+
+ const { data } = await dataCollector({ query }, "query", databaseConfigId);
+ const rows = (data as any[]) || [];
+
+ const grouped: Record<string, ModelMetadata[]> = {};
+ for (const row of rows) {
+ const provider = row.provider;
+ if (!grouped[provider]) grouped[provider] = [];
+ grouped[provider].push({
+ id: row.id,
+ displayName: row.displayName,
+ modelType: row.modelType,
+ contextWindow: Number(row.contextWindow) || 0,
+ inputPricePerMToken: Number(row.inputPricePerMToken) || 0,
+ outputPricePerMToken: Number(row.outputPricePerMToken) || 0,
+ capabilities: row.capabilities || [],
+ });
+ }
+ return grouped;
+ } catch (error) {
+ console.error("Error loading provider models:", error);
+ return {};
+ }
+}
+
+async function loadProviderModelsFromDb(
+ providerId: string,
+ databaseConfigId: string
+): Promise<ModelMetadata[]> {
+ try {
+ const query = `
+ SELECT
+ model_id as id,
+ display_name as displayName,
+ model_type as modelType,
+ context_window as contextWindow,
+ input_price_per_m_token as inputPricePerMToken,
+ output_price_per_m_token as outputPricePerMToken,
+ capabilities
+ FROM ${OPENLIT_PROVIDER_MODELS_TABLE_NAME}
+ WHERE provider = '${Sanitizer.sanitizeValue(providerId)}'
+ ORDER BY is_default DESC, created_at DESC
+ `;
+
+ const { data } = await dataCollector({ query }, "query", databaseConfigId);
+ return ((data as any[]) || []).map((row) => ({
+ id: row.id,
+ displayName: row.displayName,
+ modelType: row.modelType,
+ contextWindow: Number(row.contextWindow) || 0,
+ inputPricePerMToken: Number(row.inputPricePerMToken) || 0,
+ outputPricePerMToken: Number(row.outputPricePerMToken) || 0,
+ capabilities: row.capabilities || [],
+ }));
+ } catch (error) {
+ console.error("Error loading provider models for", error);
+ return [];
+ }
+}
+
+export class ProviderRegistry {
+ /**
+ * Get provider metadata only (no models). For UI lists that don't need models.
+ */
+ static getProviderMetadata(
+ providerId: string
+ ): StaticProviderMetadata | null {
+ return PROVIDER_METADATA[providerId] || null;
+ }
+
+ static getAllProviderMetadata(): StaticProviderMetadata[] {
+ return Object.values(PROVIDER_METADATA);
+ }
+
+ /**
+ * Get all providers with their models loaded from the DB.
+ */
+ static async getAvailableProviders(
+ databaseConfigId: string
+ ): Promise<ProviderMetadata[]> {
+ const modelsByProvider = await loadAllModelsFromDb(databaseConfigId);
+
+ return Object.values(PROVIDER_METADATA).map((provider) => ({
+ ...provider,
+ supportedModels: modelsByProvider[provider.providerId] || [],
+ }));
+ }
+
+ /**
+ * Get a specific provider with models from DB.
+ */
+ static async getProviderById(
+ providerId: string,
+ databaseConfigId: string
+ ): Promise<ProviderMetadata | null> {
+ const provider = PROVIDER_METADATA[providerId];
+ if (!provider) return null;
+
+ const models = await loadProviderModelsFromDb(providerId, databaseConfigId);
+ return {
+ ...provider,
+ supportedModels: models,
+ };
+ }
+
+ /**
+ * Search providers by name or description (structural search — does not search models).
+ */
+ static async searchProviders(
+ query: string,
+ databaseConfigId: string
+ ): Promise<ProviderMetadata[]> {
+ const lowerQuery = query.toLowerCase();
+ const matched = Object.values(PROVIDER_METADATA).filter(
+ (provider) =>
+ provider.displayName.toLowerCase().includes(lowerQuery) ||
+ provider.description?.toLowerCase().includes(lowerQuery)
+ );
+
+ const modelsByProvider = await loadAllModelsFromDb(databaseConfigId);
+ return matched.map((provider) => ({
+ ...provider,
+ supportedModels: modelsByProvider[provider.providerId] || [],
+ }));
+ }
+
+ /**
+ * Get models for a specific provider from DB.
+ */
+ static async getProviderModels(
+ providerId: string,
+ databaseConfigId: string
+ ): Promise<ModelMetadata[]> {
+ if (!PROVIDER_METADATA[providerId]) return [];
+ return loadProviderModelsFromDb(providerId, databaseConfigId);
+ }
+
+ /**
+ * Get a specific model (for e.g. cost calculation).
+ */
+ static async getModel(
+ providerId: string,
+ modelId: string,
+ databaseConfigId: string
+ ): Promise<ModelMetadata | null> {
+ const models = await loadProviderModelsFromDb(providerId, databaseConfigId);
+ return models.find((m) => m.id === modelId) || null;
+ }
+}
diff --git a/src/client/src/lib/platform/providers/provider-service.ts b/src/client/src/lib/platform/providers/provider-service.ts
new file mode 100644
index 00000000..4b8fb897
--- /dev/null
+++ b/src/client/src/lib/platform/providers/provider-service.ts
@@ -0,0 +1,55 @@
+import { ProviderRegistry } from "./provider-registry";
+import { ProviderMetadata } from "@/types/openground";
+
+/**
+ * Service layer for providers + their models.
+ *
+ * After the independent-provider-models refactor, all models live in the
+ * openlit_provider_models ClickHouse table. The ProviderRegistry is responsible
+ * for loading and merging them with static provider metadata. This file exists
+ * as a thin wrapper to keep the previous API shape used by the API routes.
+ */
+
+export async function getAllProvidersWithCustomModels(
+ _userId: string,
+ databaseConfigId: string
+): Promise<{ data?: ProviderMetadata[]; err?: string }> {
+ try {
+ const providers = await ProviderRegistry.getAvailableProviders(databaseConfigId);
+ return { data: providers };
+ } catch (error) {
+ console.error("Error getting providers with models:", error);
+ return { err: "Failed to load providers" };
+ }
+}
+
+export async function getProviderByIdWithCustomModels(
+ providerId: string,
+ _userId: string,
+ databaseConfigId: string
+): Promise<{ data?: ProviderMetadata; err?: string }> {
+ try {
+ const provider = await ProviderRegistry.getProviderById(providerId, databaseConfigId);
+ if (!provider) {
+ return { err: "Provider not found" };
+ }
+ return { data: provider };
+ } catch (error) {
+ console.error("Error getting provider by ID:", error);
+ return { err: "Failed to load provider" };
+ }
+}
+
+export async function searchProvidersWithCustomModels(
+ query: string,
+ _userId: string,
+ databaseConfigId: string
+): Promise<{ data?: ProviderMetadata[]; err?: string }> {
+ try {
+ const providers = await ProviderRegistry.searchProviders(query, databaseConfigId);
+ return { data: providers };
+ } catch (error) {
+ console.error("Error searching providers:", error);
+ return { err: "Failed to search providers" };
+ }
+}
diff --git a/src/client/src/lib/platform/providers/table-details.ts b/src/client/src/lib/platform/providers/table-details.ts
new file mode 100644
index 00000000..a18d2dfb
--- /dev/null
+++ b/src/client/src/lib/platform/providers/table-details.ts
@@ -0,0 +1,2 @@
+export const OPENLIT_PROVIDERS_TABLE_NAME = "openlit_providers";
+export const OPENLIT_PROVIDER_MODELS_TABLE_NAME = "openlit_provider_models";
diff --git a/src/client/src/middleware/check-auth.ts b/src/client/src/middleware/check-auth.ts
index a72eaa34..f9f941d1 100644
--- a/src/client/src/middleware/check-auth.ts
+++ b/src/client/src/middleware/check-auth.ts
@@ -9,6 +9,7 @@ import {
import {
DEFAULT_LOGGED_IN_ROUTE,
ALLOWED_OPENLIT_ROUTES_WITHOUT_TOKEN,
+ ALLOWED_OPENLIT_ROUTE_PREFIXES_WITHOUT_TOKEN,
CRON_JOB_ROUTES,
ONBOARDING_WHITELIST_ROUTES,
ONBOARDING_WHITELIST_API_ROUTES,
@@ -30,7 +31,10 @@ export default function checkAuth(next: NextMiddleware) {
const token = await getToken({ req: request });
const isAuth = !!token;
const isAllowedRequestWithoutToken =
- ALLOWED_OPENLIT_ROUTES_WITHOUT_TOKEN.includes(pathname);
+ ALLOWED_OPENLIT_ROUTES_WITHOUT_TOKEN.includes(pathname) ||
+ ALLOWED_OPENLIT_ROUTE_PREFIXES_WITHOUT_TOKEN.some((prefix) =>
+ pathname.startsWith(prefix)
+ );
const isCronJobRoute = CRON_JOB_ROUTES.includes(pathname);
const isAuthPage =
pathname.startsWith("/login") || pathname.startsWith("/register");
diff --git a/src/client/src/types/cron.ts b/src/client/src/types/cron.ts
index 0a7066ca..2e7bfe95 100644
--- a/src/client/src/types/cron.ts
+++ b/src/client/src/types/cron.ts
@@ -14,6 +14,7 @@ export enum CronRunStatus {
export enum CronType {
SPAN_EVALUATION = "SPAN_EVALUATION",
+ SPAN_PRICING = "SPAN_PRICING",
}
export interface CronLogData {
diff --git a/src/client/src/utils/breadcrumbs.ts b/src/client/src/utils/breadcrumbs.ts
index e195282c..4f335b79 100644
--- a/src/client/src/utils/breadcrumbs.ts
+++ b/src/client/src/utils/breadcrumbs.ts
@@ -171,7 +171,7 @@ export const ROUTE_CONFIGS: RouteConfig[] = [
regex: /^\/openground\/[^/]+$/,
getTitle: (pathname, params) => {
if (pathname === "/openground/new") return getMessage().OPENGROUND_CREATE_NEW_PLAYGROUND;
- if (pathname === "/openground/models" || pathname === "/settings/manage-models") return getMessage().OPENGROUND_MANAGE_MODELS;
+ if (pathname === "/openground/models" || pathname === "/manage-models") return getMessage().OPENGROUND_MANAGE_MODELS;
return params?.id ? getMessage().OPENGROUND_RUN_DETAILS : getMessage().FEATURE_OPENGROUND;
},
getBreadcrumbs: () => [