Skip to content

ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.

feat(ai): use Prompture 1.11 native gateway streaming

Juan Denisadd2f713a553
diff --git a/ROADMAP.md b/ROADMAP.md
index 61b84fce..c2ac52c9 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -621,7 +621,7 @@ Community request: D3M4L, September 1, 2026 — connect the assistant to OmniRou
- [x] Test compatible endpoints with a real bounded chat request; verify native model discovery and surface failures rather than treating driver construction as success.
- [x] Migrate existing settings and matching conversations, preserve write confirmations and RBAC, and test secret isolation, streaming, tool protocol, and fresh/resumed conversations.
-See [AI provider connections](docs/AI_PROVIDERS.md) for setup and compatibility details. ServerKit uses Prompture's existing drivers; compatible gateways use its OpenAI driver because the generic driver in 1.10 lacks native streaming and tools. Automatic cross-connection fallback remains a separate feature. OmniRoute-compatible protocol behavior is covered by SDK transport tests; a live OmniRoute instance is still needed for deployment-specific verification.
+See [AI provider connections](docs/AI_PROVIDERS.md) for setup and compatibility details. ServerKit uses Prompture 1.11.0's existing drivers, including native streaming and tool calls through its OpenAI-compatible driver. Automatic cross-connection fallback remains a separate feature. OmniRoute-compatible protocol behavior is covered by HTTP transport tests; a live OmniRoute instance is still needed for deployment-specific verification.
## Deploy Console & Live Run Logs
diff --git a/backend/app/services/ai_connections.py b/backend/app/services/ai_connections.py
index 3883ea8f..bb61c9b8 100644
--- a/backend/app/services/ai_connections.py
+++ b/backend/app/services/ai_connections.py
@@ -16,7 +16,6 @@ from app.models.system_settings import SystemSettings
# UI metadata complements Prompture's constructor maps (which are not a form
# schema). Never copy secret defaults from Prompture's environment settings.
_DEPENDENCIES = {
- 'openai_compatible': ['openai'], 'lmstudio': ['openai'],
'openai': ['openai'], 'claude': ['anthropic'], 'google': ['google.genai'],
'azure': ['openai'], 'google_vertexai': ['google.genai', 'anthropic'],
'bedrock': ['boto3'], 'groq': ['groq'], 'cohere': ['cohere'],
@@ -225,21 +224,18 @@ def build_driver(row, model=None):
if provider == 'claude' and os.environ.get('ANTHROPIC_BASE_URL', '').rstrip('/') not in ('', 'https://api.anthropic.com'):
raise ValueError('The installed native Anthropic driver inherits ANTHROPIC_BASE_URL. Remove that process override or configure an explicit compatible gateway connection.')
if provider in ('openai_compatible', 'prompture-hub', 'lmstudio'):
+ from prompture.drivers.openai_compatible_driver import OpenAICompatibleDriver
+
endpoint = config['endpoint'].removesuffix('/chat/completions')
- # Prompture's generic driver has no native tools/streaming in 1.10.
- # Its OpenAI driver accepts the same base URL and supports both, without
- # CachiBot's process-global short-model alias map changing gateway IDs.
- driver = get_driver_for_model(
- f'openai/{model or row.model}', base_url=endpoint,
+ # The generic descriptor has no credential kwarg map in 1.11. Construct
+ # its driver directly so explicit profile credentials are always passed.
+ driver = OpenAICompatibleDriver(
+ model=model or row.model, endpoint=endpoint,
api_key=config.get('api_key') or 'serverkit-no-auth',
)
- if not config.get('api_key'):
- # SDK versions differ in how they validate omitted auth headers.
- # Strip the placeholder in this client's request hook, after SDK
- # validation and before transport, including streaming requests.
- driver.client._client.event_hooks['request'].append(
- lambda request: request.headers.pop('Authorization', None))
- driver.api_key = None
+ # Block the constructor's ambient-key fallback, then remove the sentinel
+ # before any request. Keyless gateways send no Authorization header.
+ driver.api_key = config.get('api_key') or None
return driver
# Blank overrides suppress factory-level global settings resolution. Required
# credentials and endpoints above prevent constructor-level fallback as well.
diff --git a/backend/app/services/ai_service.py b/backend/app/services/ai_service.py
index 3bbbb5bf..9ba8fd93 100644
--- a/backend/app/services/ai_service.py
+++ b/backend/app/services/ai_service.py
@@ -547,7 +547,7 @@ def build_conversation(row, user, mode: str, page_context: Optional[dict],
from prompture.drivers import get_driver_for_model
driver = get_driver_for_model(resolved_model, env=build_provider_env())
- # Prompture 1.10 from_export does not accept a driver/env. Inject the already
+ # Prompture 1.11 from_export does not accept a driver/env. Inject the already
# configured instance through its constructor hook, so resume never builds
# an ambient/global driver, even temporarily. History/usage restoration stays
# owned by Prompture. Reapply current policy instead of trusting export options.
diff --git a/backend/requirements.txt b/backend/requirements.txt
index 1d518a51..ae15a353 100644
--- a/backend/requirements.txt
+++ b/backend/requirements.txt
@@ -90,7 +90,7 @@ marshmallow==3.26.2
# AI assistant (core primitive — powered by Prompture)
# Multi-provider LLM orchestration: Conversation, ToolRegistry, ask_live streaming,
-# ProviderEnvironment, guardrails. OpenAI (also compatible gateways) and Anthropic
-# SDKs ship with the panel. Other provider SDKs remain optional. The minor range
-# pins the descriptor/factory APIs and Conversation restoration used by the panel.
-prompture[openai,anthropic]>=1.10.0,<1.11
+# ProviderEnvironment, guardrails. Native OpenAI and Anthropic SDKs ship with the
+# panel; compatible gateways use Prompture's raw-HTTP driver. Other SDKs remain
+# optional. Pin the tested descriptor, streaming, and conversation APIs.
+prompture[openai,anthropic]==1.11.0
diff --git a/backend/tests/test_ai_connections.py b/backend/tests/test_ai_connections.py
index e5978076..b059658f 100644
--- a/backend/tests/test_ai_connections.py
+++ b/backend/tests/test_ai_connections.py
@@ -139,42 +139,76 @@ def test_chats_keep_connection_and_model_and_block_destination_changes(app, clie
assert client.delete('/api/v1/ai/connections/' + first['id'], headers=auth_headers).status_code == 409
-def _mock_transport(driver, handler):
- # Support both SDK transport module names; newer OpenAI SDKs use httpx2.
- module = importlib.import_module(type(driver.client._client._transport).__module__.split('.')[0])
- driver.client = driver.client.with_options(http_client=module.Client(
- transport=module.MockTransport(handler), event_hooks=driver.client._client.event_hooks), max_retries=0)
- return module
-
-
-def test_compatible_driver_streaming_and_tool_protocol_preserve_model(monkeypatch):
- from prompture.drivers.cachibot_driver import CachiBotDriver
- monkeypatch.setattr(CachiBotDriver, '_MODEL_MAP', {'auto': 'wrong/provider'})
- monkeypatch.setenv('OPENAI_API_KEY', 'ambient')
- row = SimpleNamespace(**service.prepare(draft(model='auto', config={'endpoint': 'http://gateway.local/v1'})))
[email protected]('provider', ['openai_compatible', 'prompture-hub', 'lmstudio'])
[email protected]('api_key', ['', 'explicit-gateway-key'])
+def test_compatible_driver_streaming_and_tool_protocol_preserve_model(monkeypatch, provider, api_key):
+ import requests
+ from prompture.drivers.openai_compatible_driver import OpenAICompatibleDriver
+ monkeypatch.setenv('OPENAI_COMPATIBLE_API_KEY', 'ambient-secret')
+ monkeypatch.setattr(service, '_installed', lambda module: False)
+ model = 'fireworks/team/model'
+ row = SimpleNamespace(**service.prepare(draft(provider=provider, model=model,
+ config={'endpoint': 'http://gateway.local/v1/chat/completions', 'api_key': api_key})))
driver = service.build_driver(row)
+ assert type(driver) is OpenAICompatibleDriver
calls = []
- def respond(request):
- payload = json.loads(request.content)
- calls.append((dict(request.headers), payload))
- if payload.get('stream'):
- content = 'data: ' + json.dumps({'id': 'c1', 'choices': [{'index': 0, 'delta': {'content': 'OK'}, 'finish_reason': None}]})
- return http.Response(200, content=(content + '\n\ndata: [DONE]\n\n').encode(), headers={'content-type': 'text/event-stream'})
- return http.Response(200, json={'id': 'c1', 'object': 'chat.completion', 'created': 1,
- 'model': 'auto', 'choices': [{'index': 0, 'finish_reason': 'tool_calls', 'message': {'role': 'assistant', 'content': None,
+ closed = []
+
+ class Response:
+ def __init__(self, payload):
+ self.payload = payload
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *args):
+ closed.append(True)
+
+ def raise_for_status(self):
+ pass
+
+ def json(self):
+ return {'choices': [{'finish_reason': 'tool_calls', 'message': {'role': 'assistant', 'content': None,
'tool_calls': [{'id': 'call1', 'type': 'function', 'function': {'name': 'peek', 'arguments': '{}'}}]}}],
- 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}})
- http = _mock_transport(driver, respond)
- assert driver.supports_streaming and driver.supports_tool_use
- chunks = list(driver.generate_messages_stream([{'role': 'user', 'content': 'Hi'}], {}))
+ 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}}
+
+ def iter_lines(self, **kwargs):
+ if self.payload.get('tools'):
+ deltas = [
+ {'tool_calls': [{'index': 0, 'id': 'call1', 'type': 'function', 'function': {'name': 'peek', 'arguments': '{'}}]},
+ {'tool_calls': [{'index': 0, 'function': {'arguments': '}'}}]},
+ ]
+ else:
+ deltas = [{'content': 'OK'}]
+ for delta in deltas:
+ yield 'data: ' + json.dumps({'choices': [{'index': 0, 'delta': delta, 'finish_reason': None}]})
+ yield 'data: ' + json.dumps({'choices': [{'index': 0, 'delta': {}, 'finish_reason': 'tool_calls' if self.payload.get('tools') else 'stop'}],
+ 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}})
+ yield 'data: [DONE]'
+
+ def post(url, **kwargs):
+ calls.append((url, kwargs['headers'], kwargs['json']))
+ return Response(kwargs['json'])
+
+ monkeypatch.setattr(requests, 'post', post)
+ assert driver.supports_streaming and driver.supports_tool_use and driver.supports_streaming_tool_use
+ messages = [{'role': 'user', 'content': 'Hi'}]
+ tools = [{'type': 'function', 'function': {'name': 'peek', 'parameters': {'type': 'object', 'properties': {}}}}]
+ chunks = list(driver.generate_messages_stream(messages, {}))
assert any(c.get('text') == 'OK' for c in chunks)
- result = driver.generate_messages_with_tools([{'role': 'user', 'content': 'Peek'}],
- [{'type': 'function', 'function': {'name': 'peek', 'parameters': {'type': 'object', 'properties': {}}}}], {})
+ assert chunks[-1]['meta']['total_tokens'] == 5
+ result = driver.generate_messages_with_tools(messages, tools, {})
assert result['tool_calls'][0]['name'] == 'peek'
- assert all(payload['model'] == 'auto' for _, payload in calls)
- assert all('authorization' not in headers for headers, _ in calls)
- assert calls[-1][1]['tools'][0]['function']['name'] == 'peek'
- driver.client.close()
+ events = list(driver.generate_messages_with_tools_stream(messages, tools, {}))
+ assert {'tool_use_start', 'tool_input_delta', 'tool_use_stop', 'message_stop'} <= {e.event_type for e in events}
+ assert next(e for e in events if e.event_type == 'tool_use_stop').input == {}
+ assert len(closed) == 2
+ assert all(url == 'http://gateway.local/v1/chat/completions' for url, _, _ in calls)
+ assert all(payload['model'] == model for _, _, payload in calls)
+ for _, headers, _ in calls:
+ assert headers.get('Authorization') == (f'Bearer {api_key}' if api_key else None)
+ assert 'ambient-secret' not in str(headers)
+ assert calls[-1][2]['tools'][0]['function']['name'] == 'peek'
def test_resume_restores_history_with_original_connection_not_ambient(app, monkeypatch):
diff --git a/docs/AI_PROVIDERS.md b/docs/AI_PROVIDERS.md
index 1b07cf32..ce02debd 100644
--- a/docs/AI_PROVIDERS.md
+++ b/docs/AI_PROVIDERS.md
@@ -47,10 +47,11 @@ based only on constructing a driver. Discovery does not prove streaming or tool
support; these depend on the provider and model. Error responses omit raw SDK
errors and credentials.
-Prompture 1.10's generic compatible driver lacks native streaming/tool support.
-ServerKit uses Prompture's OpenAI driver with an explicit base URL for compatible
-connections, preserving those capabilities without writing another API client.
-Unauthenticated local gateways omit the Authorization header. Native Anthropic's
+ServerKit pins Prompture 1.11.0 and uses its native OpenAI-compatible driver for
+custom gateways, Prompture Hub, and LM Studio. That driver handles text streaming,
+streamed tool calls, and ordinary tool calls without the OpenAI SDK. Explicit
+model IDs are preserved, and unauthenticated gateways omit the Authorization
+header even if a process-level compatible API key is set. Native Anthropic's
installed driver does not accept an endpoint argument: ServerKit refuses a
conflicting process-level `ANTHROPIC_BASE_URL` override instead of silently using
it. Vertex Gemini requires an explicit API key; Vertex Claude requires a project,