ServerKit is a lightweight, modern server control panel for managing web applications, databases, and services on your VPS or dedicated server. Built with Python/Flask backend and React frontend.
Python astracloudflarecontrol-panelcyberpaneldnsfablehosting-control-panelhosting-panelkimi-k3server-managementserverkitubuntu-web-panelwebserver
test(ai): verify management UI and document completed integration
Juan Denis8468ab79f2da
diff --git a/docs/AI_MANAGEMENT_REVIEW.md b/docs/AI_MANAGEMENT_REVIEW.mddeleted file mode 100644index b424918a..00000000--- a/docs/AI_MANAGEMENT_REVIEW.md+++ /dev/null@@ -1,173 +0,0 @@-# ServerKit AI management: source review and integration proposal--Reviewed 2026-09-08. Primary references are the local Prompture checkout-(`d36872b`) and AgentSite checkout (`db323d4`). CachiBot is a secondary reference-for its model-slot UI. This is a source review, not a live provider benchmark.-Neither reference repository was modified.--## Main finding--ServerKit should expose more of Prompture's existing capabilities through a-small, coherent settings UI. It does not need another model-routing or token-accounting engine. However, Prompture does not supply ServerKit's user/workspace-authorization, saved-connection policy, or reporting views automatically.--ServerKit pins and has installed Prompture **1.11.0**. Normalized source-text-comparison found these local Prompture files identical to the installed package:-`agents/conversation.py`, `agents/async_agent.py`, `pipeline/resolver.py`,-`pipeline/routing.py`, and `infra/{budget,session,tracker,ledger,capabilities}.py`.-The checkout's `VERSION` file says 1.10.0; it is not evidence that the installed-runtime lacks these features. No SDK upgrade is needed for the capabilities-reviewed below. AgentSite declares `prompture[all]>=1.9.1`, not an exact pin.--## What Prompture already provides--| Capability | Existing implementation | ServerKit integration |-| --- | --- | --- |-| Model roles and inheritance | `ModelResolver`, ordered configuration layers, utility-to-default fallback | Store explicit connection/model choices for Utility, Standard, and Advanced roles; resolve request override → task role → default. |-| Routing | `ModelRouter`, cost-optimized, balanced, quality-first, fast strategies; routing explanations | Select only from administrator-approved connection/model candidates before a job starts. Strategies are heuristics, not a reliability or operational-risk classifier. |-| Catalog and capabilities | Provider descriptors, discovery, model rates, context limits, tool/vision/structured-output flags | Enrich the existing model picker with verified metadata, filters, and input/output prices per 1M tokens. Keep manual gateway aliases. Unknown capability/pricing stays unknown. |-| Generation controls | Conversation options, history cap, tool timeout/round/result limits; Agent iteration limits, guardrails, structured output | Add a validated Advanced section for supported controls. Distinguish lifetime token budget from per-response output-token limit. Do not expose arbitrary driver JSON or imply all models accept temperature/reasoning settings. |-| Budget policies | `hard_stop`, `warn_and_continue`, `degrade`, token/cost ceilings, remaining budget | Use the existing hard-stop mechanism now. Add degradation only after connection-aware fallback is implemented and tested. |-| Per-run observability | `UsageSession`, driver callbacks, tokens/cache tokens, costs, errors, latency, per-model summaries | Attach usage collection to each actual run; preserve ServerKit context and show the result in chat and reports. |-| Persistent observability | `UsageTracker` events, filters, tags/metadata, daily/monthly budget records; `ModelUsageLedger` aggregates | Reuse event collection/calculation, with a ServerKit-owned data location and access-controlled reporting. Global SDK aggregates are not per-user accounting. |-| Larger workflows | Agent/DeepAgent abstractions, guardrails, structured outputs, delegation and context hooks | Consider separate read-only diagnostic workflows later. Keep existing plugin tools, RBAC checks, and write confirmations in control of server operations. |--Sources: Prompture [resolver](../../prompture/prompture/pipeline/resolver.py),-[routing](../../prompture/prompture/pipeline/routing.py),-[Conversation](../../prompture/prompture/agents/conversation.py),-[AsyncAgent](../../prompture/prompture/agents/async_agent.py),-[capabilities](../../prompture/prompture/infra/capabilities.py),-[budget](../../prompture/prompture/infra/budget.py),-[session](../../prompture/prompture/infra/session.py),-[tracker](../../prompture/prompture/infra/tracker.py), and-[ledger](../../prompture/prompture/infra/ledger.py).--## What to borrow from AgentSite--Its [model picker](../../AgentSite/frontend/src/components/shared/ModelSelect.jsx)-and [model endpoint](../../AgentSite/agentsite/api/routes/models.py) demonstrate-searchable provider groups, context/output sizes, tools, vision, reasoning, and-pricing. The useful idea is showing enough information to choose a model,-without requiring users to memorize IDs. Port the interaction into ServerKit's-shared SCSS controls, not AgentSite's Tailwind markup.--Its [configuration](../../AgentSite/agentsite/config.py) assigns cheaper strategies-to triage and some specialist reviewers, and quality-first to the final judge.-That is a better model for ServerKit than treating every task as equally costly.-AgentSite also has [analytics endpoints](../../AgentSite/agentsite/api/routes/agents.py)-and a [routing-cost view](../../AgentSite/frontend/src/components/analytics/CostByRoutingChart.jsx).-Borrow the questions they answer: which model did the work, how much did it cost,-how often did it fail, and which task/strategy caused the spend?--There are important limits to copying its implementation:--- `route_for()` selects the first syntactically valid model in a configured- strategy pool. It does not attempt runtime failover. Searches found calls only- in tests; the production orchestrator calls `resolve_agent_model()` instead.- Strategy configuration alone is therefore not proof of routed generation in- this checkout. See [resolver](../../AgentSite/agentsite/engine/model_resolver.py)- and [orchestrator](../../AgentSite/agentsite/agents/orchestrator.py).-- AgentSite's utility resolver takes the first budget fallback model. In- ServerKit, an inexpensive task model and an outage fallback should be separate- settings; they solve different problems.-- Model-cost endpoints collapse ledger rows into a dictionary keyed only by model,- although the ledger key is model plus API-key hash. Multiple credential rows- can overwrite one another. The default ledger is also process-user-wide,- while the configured tracker uses AgentSite's data directory. ServerKit should- aggregate its own scoped run events instead of copying those endpoints.--## Recommended settings structure--Keep five sections within AI Assistant, using the shared card, field, switch,-model-picker, and footer patterns in [SETTINGS_UI.md](SETTINGS_UI.md):--1. **Connections:** credentials, endpoint, connection status, model discovery.-2. **Task models:** three labeled roles, each showing its saved connection and- model. Show inherited defaults explicitly; no empty dropdown with hidden behavior.-3. **Limits and behavior:** spend/token limits and supported generation controls.- Keep technical controls in an Advanced disclosure with sensible defaults.-4. **Routing and fallback:** opt-in strategy, approved candidates, fallback order,- and a preview explaining the selected model. Publish this only with working- consumers and connection-bound driver creation.-5. **Usage:** conversation/run totals and administrator reports by period, task- role, model, connection, user, and workspace.--| Task role | Intended work | Selection rule |-| --- | --- | --- |-| Utility / economical | Short summaries, classification, structured extraction | Prefer an explicitly configured inexpensive model with required capabilities; inherit Standard if unset. Do not add model calls for tasks already handled deterministically, such as current chat titles. |-| Standard | Everyday questions and routine tool-assisted diagnosis | Default assistant connection/model. |-| Advanced | Complex incident analysis and multi-step diagnosis | Explicit user selection or a defined task type; inherit Standard if unset. Higher capability never changes permissions. |--Persist each choice as `{connection_id, model}`. The model portion is an opaque-provider/gateway ID, including embedded slashes. Resolve before the run, record-the decision and its reason, and preserve the selected connection for resumed-chats. A selected strategy must not switch credentials midway through a tool loop.--## Integration issues to account for--**Budget enforcement was disabled in ServerKit.** `build_conversation()` supplied-`max_cost` with `budget_policy=None`, which makes Prompture's check a no-op.-This change enables `hard_stop` for a nonzero ceiling on both fresh and resumed-conversations, validates saved limits, translates budget stops, and preserves-usage even when a turn ends without an answer. Zero disables the ceiling. The-default is $0.50, matching the settings endpoint. These checks use reported-usage: the last call can overshoot and unpriced gateway aliases need a provider-spending limit.--**Routing options are not all enforcement mechanisms.** In the reviewed-`routing.py`, `max_cost_per_call` is declared/documented but is not used by the-selection logic. Preferred providers are ordering preferences, not an allowlist.-Discovery uses the global environment. ServerKit must validate candidates against-saved connections and enforce spending through an actual budget check.--**Automatic fallback needs an adapter.** Conversation's `_switch_model()` calls-`get_driver_for_model(new_model)` without forwarding the saved driver/environment.-Do not enable it across ServerKit connections as-is. An explicit driver factory-must retain connection identity, endpoint, and credentials. Failed write tools-must not be silently replayed after fallback. ServerKit's legacy `fallback_models`-setting is currently stored but not applied by its conversation factory.--**CachiBot's router argument is not portable.** Its agent construction can pass-`model_router`; the reviewed Prompture `AsyncAgent` constructor accepts no such-argument. Use the public routing API as a separate selection step, rather than-copying that constructor call.--**Usage needs run attribution.** ServerKit persists Conversation usage snapshots,-which are cumulative; its chat context currently drops that usage when loading-messages and handling completion. Summing those snapshots double-counts spend.-Capture per-call/run usage with Prompture callbacks, or explicitly calculate-turn deltas, while retaining the cumulative value for conversation limits.--**SDK tracker scopes are not tenant isolation.** The reviewed tracker-`check_budget(scope)` selects a budget record by scope but sums usage events over-the chosen period without filtering events by that scope. Do not advertise it as-a per-user/workspace quota unchanged. Scope accounting, permissions, and any-concurrent spend reservations require ServerKit integration.--## Delivery sequence and acceptance criteria--1. **Current repair:** consistent settings surfaces and controls, model discovery- picker, persistent enable switch, clear notification footer, enforced- conversation ceiling. Offline tests cover both chat modes and budget resume.-2. **Profiles and visible usage:** implement the three roles and their actual task- consumers together. Add a migration-backed run record with unique run ID,- conversation/user/workspace, requested role, actual connection/model, routing- reason, tokens, cost source, duration, outcome, and fallback attempts. Reuse- Prompture counters; keep accounting idempotent. Test inheritance, permissions,- exact gateway IDs, unknown pricing, and no double counting after resume.-3. **Routing and reports:** add approved model pools and a routing preview; show- daily/monthly totals, model/task breakdowns, latency/failure rates, and remaining- limits. Separate estimated cost from provider-reported cost. Missing prices- must read “Unknown,” not “Free.” Test that disallowed connections cannot be- selected and that report totals reconcile with recorded runs.-4. **Fallback and richer workflows:** add credential-aware failover/degradation,- bounded diagnostic workflows, and supported context/history controls after- the accounting and permission tests pass. Avoid shared response caching for- live server state; any future cache needs user/workspace/connection scoping- and an explicit freshness policy.--The task-role/routing/reporting work in steps 2–4 is a proposal, not implemented-UI. This review is an internal FYI; no upstream issue or external message was sent.-Useful upstream follow-ups are a connection-aware fallback factory, enforcement-or clarification of `max_cost_per_call`, and scoped tracker budget aggregation.diff --git a/docs/AI_PROVIDERS.md b/docs/AI_PROVIDERS.mdindex f84faeae..adfd0df3 100644--- a/docs/AI_PROVIDERS.md+++ b/docs/AI_PROVIDERS.md@@ -10,12 +10,14 @@ and refreshes the chat UI. Disabling blocks new messages in both chat endpointswhile retaining conversation history. Limits and protections have their own Saveaction; saving a connection preserves unsaved changes to those settings.-New chats offer a connection selector and an optional model override. Once a-chat starts, its connection and model stay fixed. Changing the panel default-does not reroute existing chats. You can rotate a connection's credentials, but+New chats offer a task, model role, and optional connection/model override.+The selected connection and model are saved with the conversation. Changing+the panel default does not reroute existing chats. Configured fallback or budget+degradation may switch the saved binding before a turn produces output.+You can rotate a connection's credentials, butchanging a provider or endpoint used by existing chats requires a new connection.-Delete those chats before deleting their connection, and select another default-before deleting the current default.+Delete those chats and remove task/routing/fallback references before deleting+their connection, and select another default before deleting the current default.## Native providers and gateways@@ -60,18 +62,111 @@ it. Vertex Gemini requires an explicit API key; Vertex Claude requires a projectlocation, and access token. Ambient cloud credentials are not a connection option.Existing assistant RBAC, redaction, and write-action confirmations remain active.-Gateway routing aliases may have unknown pricing; enforce authoritative spending-limits at the gateway. Automatic fallback to another connection is not enabled.+The picker includes catalog prices per million input/output tokens, context and+output limits, and tools, vision, reasoning, and structured-output capabilities.+Capability filters show only confirmed support. Missing metadata stays unknown.+Gateway aliases are not assigned an upstream model's price or capabilities.+Discovering a saved connection caches its nonsecret catalog; changing its+configuration clears that cache. Manual IDs remain supported.-The per-conversation cost ceiling uses Prompture's `hard_stop` policy for new-and resumed conversations (default $0.50; zero disables it). It checks reported-cumulative usage before each model call, so a final call can overshoot the-remaining amount. Budget stops return a clear message, and usage is persisted-even if the turn stops before producing an answer. This is not an authoritative-billing limit when provider pricing is unavailable.+## Task models and execution-See [AI management review](AI_MANAGEMENT_REVIEW.md) for the Prompture and-AgentSite source review and the proposed task profiles, routing, and reporting.+**Task models and behavior** assigns a saved connection and exact model ID to+Utility, Standard, and Advanced. Standard inherits the panel default; Utility+and Advanced inherit Standard when unset. The UI shows the inherited binding.+Prompture's `ModelResolver` resolves these slots before execution.++| Task | Default role | Runtime behavior |+| --- | --- | --- |+| Chat | Standard | Existing assistant tools and confirmations, or simple text mode |+| Summarize | Utility | Summarizes supplied text and attached context without tools |+| Extract structured facts | Utility | Prompture validates an object with `summary`, `facts`, and `warnings`; no tools |+| Read-only diagnosis | Advanced | Only tools marked read-only and authorized for the caller; bounded tool rounds |++An explicit connection/model override wins, followed by an explicit role.+Without either, enabled automatic routing uses the approved pool; otherwise+the task's role is used. Existing conversations retain their task and binding.+Advanced models never grant additional permissions.++Routing uses Prompture's cost-optimized, balanced, quality-first, or fast+strategy, with discovery restricted to administrator-approved saved bindings.+Candidate capability tiers guide its heuristics; “fast” is not a measured latency+guarantee. The routing preview runs against the unsaved draft without making+a model call. Each run records the selection strategy and explanation.++Fallback is opt-in and tries the ordered list with each candidate's own endpoint+and credentials. It only runs before any provider output has been delivered.+The unused legacy `fallback_models` settings field is rejected; fallback+candidates must include a saved connection ID in `/ai/management`.+Once output or a tool response starts, a failure ends that turn; actions are+never replayed on another connection. The adapter avoids Prompture's ambient+credential fallback path. The actual selected model appears in run reporting.++## Generation and spending controls++Generation controls are applied to actual conversations: maximum output tokens+per response, lifetime conversation tokens, history messages, tool rounds, and+tool-result characters. Temperature is forwarded only with catalog-confirmed+support; reasoning effort is forwarded only to supported OpenAI reasoning models.+Unsupported settings retain provider defaults. Arbitrary driver JSON and detached+tool timeouts are not exposed.++The conversation cost ceiling defaults to $0.50; zero disables it. Select:++- **Stop at the limit:** check cumulative cost/tokens before every provider call.+- **Warn and continue:** retain usage and show the exceeded allowance in chat.+- **Use a cheaper model near the limit, then stop:** at 80% of the cost allowance,+ switch before a turn to the first configured fallback with lower known combined+ input/output rates, then enforce the hard ceiling. Unknown prices are not compared.++Monthly allowances can apply panel-wide, per user, and per workspace. Workspace+attribution is captured from the active workspace when a chat starts and access+is rechecked on resume. All-workspace chats still use panel/user allowances.+Reservations prevent concurrent runs from all consuming the same remaining+allowance in ServerKit's single worker. Each run stops further calls once its+reservation is spent. Unknown-cost attempts conservatively consume that+reservation. A restarted worker marks abandoned runs interrupted and retains+their conservative charge.++Limits use reported/estimated usage; a final call can overshoot. Unknown gateway+pricing cannot enforce a billing ceiling. Use provider/gateway limits for+authoritative spending enforcement.++## Usage and plugin integration++Each response shows its actual model, per-run tokens and cost source, duration,+provider calls/errors, routing reason, and remaining conversation allowance.+Reloading history retains those run details. **AI usage** provides administrator+reports by period, role, model, connection, user, and workspace, with daily+breakdowns, remaining monthly allowances, and a drawer for provider attempts.+Unknown/partial pricing is labeled separately from known cost.++Prompture `UsageSession` collects each actual driver call; its cost calculation+and `BudgetState` checks are reused. ServerKit stores one migration-backed+`AiRun` per turn, with idempotent completion. Cumulative Conversation snapshots+remain for lifetime limits and are never summed as per-run costs. Run records+survive transcript deletion. Reporting starts with this migration; older+transcripts are not backfilled with invented per-call costs.++The plugin SDK `ask()` and `ask_stream()` use the same enable state, role/routing+policy, guardrails, limits, driver adapter, and ledger without creating chat+transcripts. Both accept optional `profile` and `workflow` arguments:++```python+from app.plugins_sdk import ai++summary = ai.ask(text, workflow='summarize')+facts = ai.ask(text, workflow='extract', profile='utility')+```++The implementation was checked against local Prompture `d36872b` and AgentSite+`db323d4`. AgentSite informed the model-picker/reporting interactions. Prompture+1.11.0 already supplies the relevant resolver, router, structured output, budget,+and usage machinery. ServerKit supplies credential binding, authorization,+scope-filtered accounting, persistence, and UI. In this SDK version, router+preferences are not an allowlist and tracker budget scopes do not filter usage+by tenant; ServerKit explicitly enforces both boundaries. No upstream issue+or external message was sent.## Upgrade behavior@@ -82,3 +177,7 @@ different provider cannot borrow the current connection's credentials: start anew chat using an explicitly configured connection. Resumed chats reconstructtheir Prompture driver with their saved connection, and restore history andusage without exporting credentials or using global provider configuration.++Migration `099_ai_management` adds cached model metadata, conversation task+snapshots, and the durable run ledger. Apply normal Alembic upgrades before+starting the updated backend. No SDK upgrade or manual schema creation is needed.diff --git a/docs/SETTINGS_UI.md b/docs/SETTINGS_UI.mdindex 89f05e89..85485491 100644--- a/docs/SETTINGS_UI.md+++ b/docs/SETTINGS_UI.md@@ -36,6 +36,15 @@ model discovery and exact IDs, secret preservation, assistant enable persistenceand failures, notification scope, and section action placement. CI runs it anduploads screenshots from `frontend/test-results/`.+AI management reuses `ModelBinding`/`ModelPicker`, `SegControl`, `FormField`,+`Switch`, `DataTable`, `DataTableFooter`, and `Drawer`. Task-role forms, routing+and fallback candidates, and usage reports share those controls. Keep connection+credentials in their own editor; reference saved connections everywhere else.+Inherited model roles must show their effective connection/model, and monetary+figures must show their currency. Browser coverage includes saving/reloading+roles, previewing draft routing, model metadata, usage details, chat task payloads,+per-response allowance display, and narrow-screen overflow.+Provider discovery tests do not contact external providers. Live connectivitymust be verified separately with a configured connection. Model capabilities andpricing should only be shown when supplied by a trustworthy provider catalog;diff --git a/frontend/scripts/settings-ui-browser.mjs b/frontend/scripts/settings-ui-browser.mjsindex 44445788..f604df0d 100644--- a/frontend/scripts/settings-ui-browser.mjs+++ b/frontend/scripts/settings-ui-browser.mjs@@ -22,6 +22,17 @@ try {let rejectToggle = false;let savedConnection;let savedPreferences;+ let sentChat;+ let management = {+ profiles: { utility: null, standard: null, advanced: null }, routing_enabled: false, strategy: 'balanced', pool: [],+ fallback_enabled: false, fallbacks: [], budget_policy: 'hard_stop', max_tokens: 0, max_output_tokens: 2048,+ max_tool_rounds: 6, max_tool_result_length: 4000, max_history_messages: 40, temperature: null, reasoning_effort: null,+ monthly_limit_usd: 0, user_monthly_limit_usd: 0, workspace_monthly_limit_usd: 0, reservation_usd: 0.5,+ };+ const usage = { run_id: 'run-1', model: 'openai_compatible/team/cheap', connection_id: 'connection-1', profile: 'utility', workflow: 'summarize',+ reason: 'Selected Utility task model', cost: 0.02, total_tokens: 120, cost_source: 'estimated', call_count: 1, errors: 0, duration_ms: 1200,+ budget: { cost_remaining: 0.48, tokens_remaining: null },+ attempts: [{ connection_id: 'connection-1', model: 'team/cheap', status: 'success', total_tokens: 120 }] };let connection = { id: 'connection-1', name: 'Local gateway', provider: 'openai_compatible', model: 'team/default', config: { endpoint: 'http://gateway.test/v1' }, secrets_set: ['api_key'] };await page.route('**/api/v1/**', async (route) => {const path = new URL(route.request().url()).pathname;@@ -30,6 +41,21 @@ try {if (path.endsWith('/auth/setup-status')) payload = { needs_setup: false };else if (path.endsWith('/auth/me')) payload = { user: { id: 1, username: 'operator', email: '[email protected]', role: 'admin', created_at: '2026-01-01' } };else if (path.endsWith('/ai/status')) payload = { enabled, configured: true };+ else if (path.endsWith('/ai/management/preview')) payload = { connection_id: connection.id, model: 'team/cheap', reason: 'Approved Utility candidate' };+ else if (path.endsWith('/ai/management')) {+ if (method === 'PUT') management = route.request().postDataJSON();+ payload = management;+ } else if (path.endsWith('/ai/usage')) payload = {+ totals: { cost: 0.02, tokens: 120, runs: 1, errors: 0, unknown_cost_runs: 0, average_duration_ms: 1200 },+ groups: { model: [{ label: usage.model, cost: 0.02, tokens: 120, runs: 1 }], profile: [{ label: 'utility', cost: 0.02, tokens: 120, runs: 1 }] },+ runs: [{ id: 'run-1', model: usage.model, profile: 'utility', status: 'success', cost: 0.02, reason: usage.reason, created_at: '2026-09-08T12:00:00Z', usage }],+ allowances: [{ scope: 'Panel', remaining: 9.98, limit: 10 }], has_more: false, next_offset: 50,+ };+ else if (path.endsWith('/ai/chat/stream')) {+ sentChat = route.request().postDataJSON();+ const events = [['open', { conversation_id: 'chat-1' }], ['run_start', usage], ['text_delta', { text: 'The server is healthy.' }], ['done', { conversation_id: 'chat-1', usage }]];+ return route.fulfill({ contentType: 'text/event-stream', body: events.map(([name, data]) => `event: ${name}\ndata: ${JSON.stringify(data)}\n\n`).join('') });+ }else if (path.endsWith('/ai/settings')) {if (method === 'PUT') {const body = route.request().postDataJSON();@@ -43,7 +69,8 @@ try {{ name: 'endpoint', label: 'API base URL', type: 'url', required: true },{ name: 'api_key', label: 'API key', type: 'text', secret: true },] }] };- else if (path.endsWith('/ai/connections/probe')) payload = { models: ['team/default', 'team/reasoning', 'team/fast'], source: 'live' };+ else if (path.endsWith('/ai/connections/probe')) payload = { models: ['team/default', 'team/reasoning', 'team/fast'], source: 'live',+ model_details: [{ id: 'team/default' }, { id: 'team/reasoning', reasoning: true, tools: true, context_window: 128000, max_output_tokens: 8192, pricing: { input: 1, output: 3 } }, { id: 'team/fast' }] };else if (path.endsWith('/ai/connections/connection-1')) {savedConnection = route.request().postDataJSON();connection = { ...connection, ...savedConnection, secrets_set: ['api_key'] };@@ -62,8 +89,11 @@ try {await page.locator('#ai-max-cost').fill('1.25');await page.getByRole('button', { name: 'Discover models', exact: true }).click();await page.getByRole('button', { name: 'Browse models (3)', exact: true }).click();+ await page.getByLabel('Capability', { exact: true }).selectOption('reasoning');+ await page.getByText('1 USD input / 3 USD output per 1M tokens', { exact: false }).waitFor();+ assert.equal(await page.getByRole('option').filter({ hasText: 'team/default' }).count(), 0);await page.getByPlaceholder('Search models…').fill('reasoning');- await page.getByRole('option', { name: 'team/reasoning', exact: true }).click();+ await page.getByRole('option', { name: /^team\/reasoning/ }).click();assert.equal(await page.locator('#ai-model').inputValue(), 'team/reasoning');await page.getByRole('button', { name: 'Save connection', exact: true }).click();await page.getByText('Connection saved.', { exact: false }).waitFor();@@ -83,6 +113,46 @@ try {await page.locator('[data-testid="ai-state"][data-enabled="false"][data-ready="false"]').waitFor({ state: 'attached' });await page.screenshot({ path: `test-results/settings-ai-${theme}.png`, fullPage: true });+ await page.getByRole('tab', { name: 'Task models and behavior' }).click();+ await page.locator('#ai-profile-utility-connection').selectOption('connection-1');+ await page.locator('#ai-profile-utility-model').fill('team/cheap');+ await page.getByRole('button', { name: 'Add model candidate' }).first().click();+ await page.locator('#ai-pool-0-model').fill('team/cheap');+ await page.locator('#ai-tier-0').selectOption('budget');+ await page.locator('#ai-routing_enabled').click();+ await page.locator('#ai-routing-sample').fill('Summarize server health');+ await page.getByRole('button', { name: 'Preview routing' }).click();+ await page.getByText('Approved Utility candidate').waitFor();+ await page.locator('#ai-max_output_tokens').fill('1024');+ await page.getByRole('button', { name: 'Save AI management settings' }).click();+ await page.getByText('AI management settings saved.').waitFor();+ assert.equal(management.profiles.utility.model, 'team/cheap');+ assert.equal(management.routing_enabled, true);+ assert.equal(Number(management.max_output_tokens), 1024);+ await page.screenshot({ path: `test-results/settings-ai-management-${theme}.png`, fullPage: true });+ await page.reload();+ await page.getByRole('tab', { name: 'Task models and behavior' }).click();+ assert.equal(await page.locator('#ai-profile-utility-model').inputValue(), 'team/cheap');+ await page.getByRole('tab', { name: 'AI usage', exact: true }).click();+ await page.getByRole('heading', { name: 'Run details' }).waitFor();+ await page.getByRole('button', { name: 'Details', exact: true }).click();+ await page.getByRole('dialog').getByText('Cost source: estimated').waitFor();+ await page.screenshot({ path: `test-results/settings-ai-usage-${theme}.png`, fullPage: true });+ await page.keyboard.press('Escape');+ enabled = true;+ await visit('chat');+ await page.locator('#ai-chat-workflow').selectOption('summarize');+ await page.locator('#ai-chat-profile').selectOption('utility');+ await page.getByRole('textbox', { name: 'Message the assistant' }).fill('Summarize server health');+ await page.getByRole('textbox', { name: 'Message the assistant' }).press('Enter');+ await page.locator('.sk-ai-message__usage summary').click();+ await page.locator('.sk-ai-message__usage summary').getByText('0.02 USD', { exact: false }).waitFor();+ await page.getByText('Conversation cost allowance remaining:', { exact: false }).waitFor();+ assert.equal(sentChat.workflow, 'summarize');+ assert.equal(sentChat.profile, 'utility');+ assert(!sentChat.connection_id, 'A default connection override must not bypass task roles');+ await page.screenshot({ path: `test-results/ai-chat-usage-${theme}.png`, fullPage: true });+await visit('notifications');await page.getByRole('heading', { name: 'Notification Channels' }).waitFor();assert.equal(await page.getByRole('switch', { name: 'Slack', exact: true }).count(), 0);@@ -136,6 +206,13 @@ try {await visit(pane);await page.locator('.settings-card, .settings-form').first().waitFor();assert(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth), `${pane} overflows on mobile`);+ if (pane === 'ai') {+ for (const tab of ['Task models and behavior', 'AI usage']) {+ await page.getByRole('tab', { name: tab, exact: true }).click();+ assert(await page.evaluate(() => document.documentElement.scrollWidth <= window.innerWidth), `${tab} overflows on mobile`);+ }+ await page.screenshot({ path: `test-results/settings-ai-mobile-${theme}.png`, fullPage: true });+ }if (pane === 'notifications') {await page.getByRole('button', { name: 'Save Preferences' }).waitFor();const actions = await page.locator('.settings-actions--footer > button').evaluateAll((nodes) => nodes.map((node) => ({ top: node.getBoundingClientRect().top, bottom: node.getBoundingClientRect().bottom })));diff --git a/frontend/tests/browser/settings-ui.jsx b/frontend/tests/browser/settings-ui.jsxindex 41a309ce..3dfdc34f 100644--- a/frontend/tests/browser/settings-ui.jsx+++ b/frontend/tests/browser/settings-ui.jsx@@ -5,12 +5,16 @@ import { LocaleProvider } from '../../src/contexts/LocaleContext';import { ToastProvider } from '../../src/contexts/ToastContext';import { ConfirmProvider } from '../../src/contexts/ConfirmContext';import { AIProvider } from '../../src/contexts/AIContext';+import { WorkspaceProvider } from '../../src/contexts/WorkspaceContext';import { useServerkitAI } from '../../src/contexts/useServerkitAI';import AISettingsTab from '../../src/components/settings/AISettingsTab';import ProfileTab from '../../src/components/settings/ProfileTab';import NotificationsTab from '../../src/components/settings/NotificationsTab';import ApiSettingsTab from '../../src/components/settings/ApiSettingsTab';import { FormField } from '../../src/components/FormField';+import ConnectionSelector from '../../src/components/ai/ConnectionSelector';+import Composer from '../../src/components/ai/Composer';+import Message from '../../src/components/ai/Message';import '../../src/styles/main.scss';export function Fixture() {@@ -23,6 +27,7 @@ export function Fixture() {{pane === 'profile' && <ProfileTab />}{pane === 'notifications' && <NotificationsTab />}{pane === 'api' && <ApiSettingsTab />}+ {pane === 'chat' && <div className="settings-card"><ConnectionSelector /><Composer />{ai.messages.map((message) => <Message key={message.id} message={message} />)}</div>}{pane === 'controls' && <div className="settings-card"><FormField label="Native field" htmlFor="native-field"><input id="native-field" /></FormField><FormField label="Native select" htmlFor="native-select"><select id="native-select"><option>Option</option></select></FormField>@@ -33,7 +38,7 @@ export function Fixture() {</>;}createRoot(document.getElementById('root')).render(- <MemoryRouter><LocaleProvider><AuthProvider><ToastProvider><ConfirmProvider><AIProvider>+ <MemoryRouter><LocaleProvider><AuthProvider><WorkspaceProvider><ToastProvider><ConfirmProvider><AIProvider><Fixture />- </AIProvider></ConfirmProvider></ToastProvider></AuthProvider></LocaleProvider></MemoryRouter>+ </AIProvider></ConfirmProvider></ToastProvider></WorkspaceProvider></AuthProvider></LocaleProvider></MemoryRouter>);