Soul-driven AI agent with permission-hardened tools, token budgets, and multi-channel access. Runs 24/7 from CLI, Telegram or More.
fix(providers): send images in tool results only where the API carries them
Asyfdzakya8f0a4813a44
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 0db6534e..1f7d930b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -6,6 +6,9 @@- **MCP servers: Mercury can use tools from Model Context Protocol servers.** Add a local server (`mercury mcp add files --stdio npx -- -y @modelcontextprotocol/server-filesystem ~/Documents`) or a remote one (`--http <url>`), allow the tools you want (`mercury mcp allow`), and the agent can call them next to its built-in tools. Every MCP call asks first in Ask Me mode, and Allow All does not cover MCP tools; only tools you list in `autoApprove` run without a prompt, and only those run when nobody can be asked. Secrets stay in `.env` as `${env:NAME}` references and are masked everywhere they could appear. Local servers get a minimal environment, never Mercury's API keys, and remote servers go through the SSRF guard. Check health with `mercury mcp test`, `mercury doctor --mcp`, `/mcp` in chat, or the new read-only **MCP** dashboard page. The agent's file tools now always ask before writing `mercury.yaml`, `permissions.yaml` or `.env`, even under Allow All. See [MCP Servers](https://mercuryagent.sh/docs/integrations/mcp) and ADR-029.- **Allow All in the terminal, the web dashboard and Mercury Cloud no longer skips MCP prompts.** The terminal attached to the background service (what `mercury` opens), the dashboard and Cloud sessions share one web channel, and in Allow All it answered every approval with yes on its own. So MCP tool calls, shell commands that change MCP setup, and writes to `mercury.yaml`, `permissions.yaml` or `.env` ran without asking there. They now ask in every channel, as documented. Other prompts still follow the session's mode.+### Plugins+- **Plugin images reach the model only where the provider can take them.** On Chat Completions providers (Mercury Cloud, GitHub Copilot, Ollama, LM Studio, LiteLLM, AI/ML API and other OpenAI-compatible endpoints) the AI SDK sent a plugin's screenshot as JSON text, so the model got megabytes of base64 instead of a picture. Images in tool results now go only to Anthropic and to OpenAI's Responses API; every other provider, Grok and ChatGPT Web included until they are verified, gets the usual "this model can't view images" note.+## 1.4.2 — search that finds things, and a research prompt that asks only when you mean it### Web searchdiff --git a/src/core/agent.ts b/src/core/agent.tsindex 32a5fa7e..a2dd1e3d 100644--- a/src/core/agent.ts+++ b/src/core/agent.ts@@ -3270,7 +3270,7 @@ export class Agent {// passed back with assistant turns (issue #24); other providers// get plain text.// Photos and PDFs from the chat go to models that can take them.- setPluginImagesAllowed(modelSeesImages(provider.name, provider.getModel()));+ setPluginImagesAllowed(modelSeesImages(provider.name, provider.getModel()) && provider.supportsToolResultImages());const providerMessages = withAttachments(deepseekProviderOptions ? withReasoningParts(messages) : messages, msg, provider.name, provider.getModel());// Per-provider output ceiling: a 32K request is rejected outright by// providers with smaller limits and burned a halve-and-retry round.diff --git a/src/providers/anthropic.ts b/src/providers/anthropic.tsindex b8ec40f6..8b08d173 100644--- a/src/providers/anthropic.ts+++ b/src/providers/anthropic.ts@@ -57,4 +57,8 @@ export class AnthropicProvider extends BaseProvider {getModelInstance(): any {return this.modelInstance;}++ supportsToolResultImages(): boolean {+ return true;+ }}\ No newline at end of filediff --git a/src/providers/base.ts b/src/providers/base.tsindex 6ad188a1..eabb1137 100644--- a/src/providers/base.ts+++ b/src/providers/base.ts@@ -40,4 +40,8 @@ export abstract class BaseProvider {getMaxOutputTokens(): number | undefined {return this.config.maxOutputTokens;}++ supportsToolResultImages(): boolean {+ return false;+ }}\ No newline at end of filediff --git a/src/providers/openai-compat.ts b/src/providers/openai-compat.tsindex 5f6fa56b..8c7c78b5 100644--- a/src/providers/openai-compat.ts+++ b/src/providers/openai-compat.ts@@ -34,11 +34,13 @@ export class OpenAICompatProvider extends BaseProvider {readonly model: string;private client: ReturnType<typeof createOpenAI>;private modelInstance: ReturnType<ReturnType<typeof createOpenAI>['languageModel']>;+ private readonly usesResponsesApi: boolean;constructor(config: ProviderConfig, { useChatApi }: { useChatApi?: boolean } = {}) {super(config);this.name = config.name;this.model = config.model;+ this.usesResponsesApi = !useChatApi;this.client = createOpenAI({apiKey: config.apiKey || 'no-key',@@ -122,4 +124,8 @@ export class OpenAICompatProvider extends BaseProvider {getModelInstance(): any {return this.modelInstance;}++ supportsToolResultImages(): boolean {+ return this.usesResponsesApi && this.name === 'openai';+ }}\ No newline at end of filediff --git a/src/providers/tool-result-images.test.ts b/src/providers/tool-result-images.test.tsnew file mode 100644index 00000000..87699570--- /dev/null+++ b/src/providers/tool-result-images.test.ts@@ -0,0 +1,37 @@+import { describe, expect, it } from 'vitest';+import { createProvider } from './registry.js';++const config = (name: string, model: string) => ({+ name,+ apiKey: 'test-key',+ baseUrl: 'https://example.invalid/v1',+ model,+ enabled: true,+});++describe('supportsToolResultImages', () => {+ it('allows images in tool results only where the API sends them as images', async () => {+ const expected: Array<[string, string, boolean]> = [+ ['anthropic', 'claude-sonnet-4-5', true],+ ['openai', 'gpt-5', true],+ ['grok', 'grok-4', false],+ ['chatgptWeb', 'gpt-5', false],+ ['githubCopilot', 'gpt-5', false],+ ['mercuryCloud', 'gpt-5', false],+ ['deepseek', 'deepseek-chat', false],+ ['ollamaLocal', 'llava', false],+ ['ollamaCloud', 'qwen2.5-vl', false],+ ['atomicChat', 'llava', false],+ ['aimlapi', 'gpt-4o', false],+ ['atlascloud', 'gpt-4o', false],+ ['openaiCompat', 'gpt-4o', false],+ ['litellm', 'gpt-4o', false],+ ['lmStudio', 'llava', false],+ ['mimo', 'mimo-vl', false],+ ];+ for (const [name, model, supported] of expected) {+ const provider = await createProvider(config(name, model), null);+ expect([name, provider.supportsToolResultImages()]).toEqual([name, supported]);+ }+ });+});