ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
feat(ai): use Prompture 1.11 native gateway streaming
Juan Denisadd2f713a553
diff --git a/ROADMAP.md b/ROADMAP.mdindex 61b84fce..c2ac52c9 100644--- a/ROADMAP.md+++ b/ROADMAP.md@@ -621,7 +621,7 @@ Community request: D3M4L, September 1, 2026 — connect the assistant to OmniRou- [x] Test compatible endpoints with a real bounded chat request; verify native model discovery and surface failures rather than treating driver construction as success.- [x] Migrate existing settings and matching conversations, preserve write confirmations and RBAC, and test secret isolation, streaming, tool protocol, and fresh/resumed conversations.-See [AI provider connections](docs/AI_PROVIDERS.md) for setup and compatibility details. ServerKit uses Prompture's existing drivers; compatible gateways use its OpenAI driver because the generic driver in 1.10 lacks native streaming and tools. Automatic cross-connection fallback remains a separate feature. OmniRoute-compatible protocol behavior is covered by SDK transport tests; a live OmniRoute instance is still needed for deployment-specific verification.+See [AI provider connections](docs/AI_PROVIDERS.md) for setup and compatibility details. ServerKit uses Prompture 1.11.0's existing drivers, including native streaming and tool calls through its OpenAI-compatible driver. Automatic cross-connection fallback remains a separate feature. OmniRoute-compatible protocol behavior is covered by HTTP transport tests; a live OmniRoute instance is still needed for deployment-specific verification.## Deploy Console & Live Run Logsdiff --git a/backend/app/services/ai_connections.py b/backend/app/services/ai_connections.pyindex 3883ea8f..bb61c9b8 100644--- a/backend/app/services/ai_connections.py+++ b/backend/app/services/ai_connections.py@@ -16,7 +16,6 @@ from app.models.system_settings import SystemSettings# UI metadata complements Prompture's constructor maps (which are not a form# schema). Never copy secret defaults from Prompture's environment settings._DEPENDENCIES = {- 'openai_compatible': ['openai'], 'lmstudio': ['openai'],'openai': ['openai'], 'claude': ['anthropic'], 'google': ['google.genai'],'azure': ['openai'], 'google_vertexai': ['google.genai', 'anthropic'],'bedrock': ['boto3'], 'groq': ['groq'], 'cohere': ['cohere'],@@ -225,21 +224,18 @@ def build_driver(row, model=None):if provider == 'claude' and os.environ.get('ANTHROPIC_BASE_URL', '').rstrip('/') not in ('', 'https://api.anthropic.com'):raise ValueError('The installed native Anthropic driver inherits ANTHROPIC_BASE_URL. Remove that process override or configure an explicit compatible gateway connection.')if provider in ('openai_compatible', 'prompture-hub', 'lmstudio'):+ from prompture.drivers.openai_compatible_driver import OpenAICompatibleDriver+endpoint = config['endpoint'].removesuffix('/chat/completions')- # Prompture's generic driver has no native tools/streaming in 1.10.- # Its OpenAI driver accepts the same base URL and supports both, without- # CachiBot's process-global short-model alias map changing gateway IDs.- driver = get_driver_for_model(- f'openai/{model or row.model}', base_url=endpoint,+ # The generic descriptor has no credential kwarg map in 1.11. Construct+ # its driver directly so explicit profile credentials are always passed.+ driver = OpenAICompatibleDriver(+ model=model or row.model, endpoint=endpoint,api_key=config.get('api_key') or 'serverkit-no-auth',)- if not config.get('api_key'):- # SDK versions differ in how they validate omitted auth headers.- # Strip the placeholder in this client's request hook, after SDK- # validation and before transport, including streaming requests.- driver.client._client.event_hooks['request'].append(- lambda request: request.headers.pop('Authorization', None))- driver.api_key = None+ # Block the constructor's ambient-key fallback, then remove the sentinel+ # before any request. Keyless gateways send no Authorization header.+ driver.api_key = config.get('api_key') or Nonereturn driver# Blank overrides suppress factory-level global settings resolution. Required# credentials and endpoints above prevent constructor-level fallback as well.diff --git a/backend/app/services/ai_service.py b/backend/app/services/ai_service.pyindex 3bbbb5bf..9ba8fd93 100644--- a/backend/app/services/ai_service.py+++ b/backend/app/services/ai_service.py@@ -547,7 +547,7 @@ def build_conversation(row, user, mode: str, page_context: Optional[dict],from prompture.drivers import get_driver_for_modeldriver = get_driver_for_model(resolved_model, env=build_provider_env())- # Prompture 1.10 from_export does not accept a driver/env. Inject the already+ # Prompture 1.11 from_export does not accept a driver/env. Inject the already# configured instance through its constructor hook, so resume never builds# an ambient/global driver, even temporarily. History/usage restoration stays# owned by Prompture. Reapply current policy instead of trusting export options.diff --git a/backend/requirements.txt b/backend/requirements.txtindex 1d518a51..ae15a353 100644--- a/backend/requirements.txt+++ b/backend/requirements.txt@@ -90,7 +90,7 @@ marshmallow==3.26.2# AI assistant (core primitive — powered by Prompture)# Multi-provider LLM orchestration: Conversation, ToolRegistry, ask_live streaming,-# ProviderEnvironment, guardrails. OpenAI (also compatible gateways) and Anthropic-# SDKs ship with the panel. Other provider SDKs remain optional. The minor range-# pins the descriptor/factory APIs and Conversation restoration used by the panel.-prompture[openai,anthropic]>=1.10.0,<1.11+# ProviderEnvironment, guardrails. Native OpenAI and Anthropic SDKs ship with the+# panel; compatible gateways use Prompture's raw-HTTP driver. Other SDKs remain+# optional. Pin the tested descriptor, streaming, and conversation APIs.+prompture[openai,anthropic]==1.11.0diff --git a/backend/tests/test_ai_connections.py b/backend/tests/test_ai_connections.pyindex e5978076..b059658f 100644--- a/backend/tests/test_ai_connections.py+++ b/backend/tests/test_ai_connections.py@@ -139,42 +139,76 @@ def test_chats_keep_connection_and_model_and_block_destination_changes(app, clieassert client.delete('/api/v1/ai/connections/' + first['id'], headers=auth_headers).status_code == 409-def _mock_transport(driver, handler):- # Support both SDK transport module names; newer OpenAI SDKs use httpx2.- module = importlib.import_module(type(driver.client._client._transport).__module__.split('.')[0])- driver.client = driver.client.with_options(http_client=module.Client(- transport=module.MockTransport(handler), event_hooks=driver.client._client.event_hooks), max_retries=0)- return module---def test_compatible_driver_streaming_and_tool_protocol_preserve_model(monkeypatch):- from prompture.drivers.cachibot_driver import CachiBotDriver- monkeypatch.setattr(CachiBotDriver, '_MODEL_MAP', {'auto': 'wrong/provider'})- monkeypatch.setenv('OPENAI_API_KEY', 'ambient')- row = SimpleNamespace(**service.prepare(draft(model='auto', config={'endpoint': 'http://gateway.local/v1'})))[email protected]('provider', ['openai_compatible', 'prompture-hub', 'lmstudio'])[email protected]('api_key', ['', 'explicit-gateway-key'])+def test_compatible_driver_streaming_and_tool_protocol_preserve_model(monkeypatch, provider, api_key):+ import requests+ from prompture.drivers.openai_compatible_driver import OpenAICompatibleDriver+ monkeypatch.setenv('OPENAI_COMPATIBLE_API_KEY', 'ambient-secret')+ monkeypatch.setattr(service, '_installed', lambda module: False)+ model = 'fireworks/team/model'+ row = SimpleNamespace(**service.prepare(draft(provider=provider, model=model,+ config={'endpoint': 'http://gateway.local/v1/chat/completions', 'api_key': api_key})))driver = service.build_driver(row)+ assert type(driver) is OpenAICompatibleDrivercalls = []- def respond(request):- payload = json.loads(request.content)- calls.append((dict(request.headers), payload))- if payload.get('stream'):- content = 'data: ' + json.dumps({'id': 'c1', 'choices': [{'index': 0, 'delta': {'content': 'OK'}, 'finish_reason': None}]})- return http.Response(200, content=(content + '\n\ndata: [DONE]\n\n').encode(), headers={'content-type': 'text/event-stream'})- return http.Response(200, json={'id': 'c1', 'object': 'chat.completion', 'created': 1,- 'model': 'auto', 'choices': [{'index': 0, 'finish_reason': 'tool_calls', 'message': {'role': 'assistant', 'content': None,+ closed = []++ class Response:+ def __init__(self, payload):+ self.payload = payload++ def __enter__(self):+ return self++ def __exit__(self, *args):+ closed.append(True)++ def raise_for_status(self):+ pass++ def json(self):+ return {'choices': [{'finish_reason': 'tool_calls', 'message': {'role': 'assistant', 'content': None,'tool_calls': [{'id': 'call1', 'type': 'function', 'function': {'name': 'peek', 'arguments': '{}'}}]}}],- 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}})- http = _mock_transport(driver, respond)- assert driver.supports_streaming and driver.supports_tool_use- chunks = list(driver.generate_messages_stream([{'role': 'user', 'content': 'Hi'}], {}))+ 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}}++ def iter_lines(self, **kwargs):+ if self.payload.get('tools'):+ deltas = [+ {'tool_calls': [{'index': 0, 'id': 'call1', 'type': 'function', 'function': {'name': 'peek', 'arguments': '{'}}]},+ {'tool_calls': [{'index': 0, 'function': {'arguments': '}'}}]},+ ]+ else:+ deltas = [{'content': 'OK'}]+ for delta in deltas:+ yield 'data: ' + json.dumps({'choices': [{'index': 0, 'delta': delta, 'finish_reason': None}]})+ yield 'data: ' + json.dumps({'choices': [{'index': 0, 'delta': {}, 'finish_reason': 'tool_calls' if self.payload.get('tools') else 'stop'}],+ 'usage': {'prompt_tokens': 3, 'completion_tokens': 2, 'total_tokens': 5}})+ yield 'data: [DONE]'++ def post(url, **kwargs):+ calls.append((url, kwargs['headers'], kwargs['json']))+ return Response(kwargs['json'])++ monkeypatch.setattr(requests, 'post', post)+ assert driver.supports_streaming and driver.supports_tool_use and driver.supports_streaming_tool_use+ messages = [{'role': 'user', 'content': 'Hi'}]+ tools = [{'type': 'function', 'function': {'name': 'peek', 'parameters': {'type': 'object', 'properties': {}}}}]+ chunks = list(driver.generate_messages_stream(messages, {}))assert any(c.get('text') == 'OK' for c in chunks)- result = driver.generate_messages_with_tools([{'role': 'user', 'content': 'Peek'}],- [{'type': 'function', 'function': {'name': 'peek', 'parameters': {'type': 'object', 'properties': {}}}}], {})+ assert chunks[-1]['meta']['total_tokens'] == 5+ result = driver.generate_messages_with_tools(messages, tools, {})assert result['tool_calls'][0]['name'] == 'peek'- assert all(payload['model'] == 'auto' for _, payload in calls)- assert all('authorization' not in headers for headers, _ in calls)- assert calls[-1][1]['tools'][0]['function']['name'] == 'peek'- driver.client.close()+ events = list(driver.generate_messages_with_tools_stream(messages, tools, {}))+ assert {'tool_use_start', 'tool_input_delta', 'tool_use_stop', 'message_stop'} <= {e.event_type for e in events}+ assert next(e for e in events if e.event_type == 'tool_use_stop').input == {}+ assert len(closed) == 2+ assert all(url == 'http://gateway.local/v1/chat/completions' for url, _, _ in calls)+ assert all(payload['model'] == model for _, _, payload in calls)+ for _, headers, _ in calls:+ assert headers.get('Authorization') == (f'Bearer {api_key}' if api_key else None)+ assert 'ambient-secret' not in str(headers)+ assert calls[-1][2]['tools'][0]['function']['name'] == 'peek'def test_resume_restores_history_with_original_connection_not_ambient(app, monkeypatch):diff --git a/docs/AI_PROVIDERS.md b/docs/AI_PROVIDERS.mdindex 1b07cf32..ce02debd 100644--- a/docs/AI_PROVIDERS.md+++ b/docs/AI_PROVIDERS.md@@ -47,10 +47,11 @@ based only on constructing a driver. Discovery does not prove streaming or toolsupport; these depend on the provider and model. Error responses omit raw SDKerrors and credentials.-Prompture 1.10's generic compatible driver lacks native streaming/tool support.-ServerKit uses Prompture's OpenAI driver with an explicit base URL for compatible-connections, preserving those capabilities without writing another API client.-Unauthenticated local gateways omit the Authorization header. Native Anthropic's+ServerKit pins Prompture 1.11.0 and uses its native OpenAI-compatible driver for+custom gateways, Prompture Hub, and LM Studio. That driver handles text streaming,+streamed tool calls, and ordinary tool calls without the OpenAI SDK. Explicit+model IDs are preserved, and unauthenticated gateways omit the Authorization+header even if a process-level compatible API key is set. Native Anthropic'sinstalled driver does not accept an endpoint argument: ServerKit refuses aconflicting process-level `ANTHROPIC_BASE_URL` override instead of silently usingit. Vertex Gemini requires an explicit API key; Vertex Claude requires a project,