Skip to content

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

C++

MCP chat: opt-in mcp.image_results caches PNG/JPEG/WebP tool results beside the config, shows them in the web chat with an authenticated download and passes them to the vision reader (PR #2473 by cuijinkuo)

cuijinkuo7c9b6f3a28e8
diff --git a/.gitignore b/.gitignore
index c6cfb3367..88ad4c7b5 100644
--- a/.gitignore
+++ b/.gitignore
@@ -63,6 +63,10 @@ build-vision-cuda12.bat
/.ccache/
/.rocm-win/
+# Optional MCP raster cache
+*.tool-media/
+/.tool-media/
+
# local Windows HIP packaging output
dist-hip/
diff --git a/docs/MCP_IMAGES.md b/docs/MCP_IMAGES.md
new file mode 100644
index 000000000..610558fef
--- /dev/null
+++ b/docs/MCP_IMAGES.md
@@ -0,0 +1,25 @@
+# MCP raster images in the web chat
+
+Enable this optional feature with `"mcp": {"image_results": true}` in the run configuration (alongside existing MCP settings). The value must be a JSON boolean (`true` or `false`); strings and numbers are rejected. It is disabled by default; text-only tool output and stream event fields remain unchanged when disabled.
+
+MCP image results are cached by SHA-256 next to the run configuration, in `<config-stem>.tool-media/`. The web chat receives small same-origin references instead of embedding base64 in chat history. PNG, JPEG and WebP are supported; SVG and arbitrary file/resource URIs are not rendered. Each call displays at most four images, each with at most 24 million base64 characters (about 18 MB decoded). If the cache cannot be written, the chat retains the text result and logs the cache error.
+
+`GET /tool-media/<sha256>.<extension>` uses the existing server API authentication, validates the basename, and serves only cached raster files. The web client fetches images with its existing authorization headers, displays them outside collapsed tool details, and provides a blob download link. The cache persists across server restarts; if its files are removed, older image references are replaced with an image-omitted text note when preparing later turns. Disabling the feature or removing MCP configuration does the same, while preserving the tool text and allowing the conversation to continue. Malformed references are still rejected. A global disk-retention policy is not implemented yet.
+
+When the server has a vision encoder, tool images enter its existing image preparation path. Subsequent browser turns send `strata-tool-media:<basename>` references; the authenticated server resolves these only inside its cache. With no vision encoder, images remain visible to the user and the model receives the existing text-only fallback. This adds image transport, not a vision model.
+
+Search and image generation are supplied by configured MCP servers. Strata does not bundle a paid search provider or a ComfyUI installation. Machine-specific adapters and lifecycle hooks are not included in this change.
+
+## Validation without a GPU
+
+```powershell
+python -m unittest serve.test_mcp serve.test_mcp_media
+node --check serve/web/app.js
+git diff --check
+```
+
+The transport tests use an existing mock engine and scripted tool response. They verify authenticated binary delivery, invalid media rejection, path traversal refusal, streaming metadata, forwarding to vision preparation and subsequent-turn cache resolution. They do not establish visual understanding or generation quality.
+
+## Browser layout checks
+
+Use a configured image MCP fixture and the mock engine; no GPU is needed for the transport or layout check. Include a long tool name and long text result, and check a running call while Stop is visible. At viewport widths 320, 390, 640, 768 and 1280 pixels, the document must not be wider than the viewport and the preview must fit its chat bubble. Also expand tool details, reload the page, and check that the preview and download link return. A missing cache may remove the picture but must not prevent the next text turn.
diff --git a/serve/mcp.py b/serve/mcp.py
index 116251af2..326249ec9 100644
--- a/serve/mcp.py
+++ b/serve/mcp.py
@@ -19,6 +19,9 @@ breaking.
from __future__ import annotations
import collections
+import base64
+import hashlib
+from pathlib import Path
import json
import os
import re
@@ -34,7 +37,7 @@ import urllib.request
from serve import lifeline
PROTOCOL = "2025-06-18" # the MCP revision Strata asks for; the server's answer is used as given
-DEFAULTS = {"timeout_s": 60.0, "max_result_chars": 20000, "max_rounds": 8, "start_timeout_s": 120.0}
+DEFAULTS = {"timeout_s": 60.0, "max_result_chars": 20000, "max_rounds": 8, "start_timeout_s": 120.0, "image_results": False}
class McpError(RuntimeError):
@@ -486,6 +489,42 @@ def result_text(result: dict) -> str:
return "\n".join(parts)
+def cache_images(result, directory):
+ """Keep raster tool outputs outside chat history; never expose arbitrary tool paths."""
+ images = []
+ directory = Path(directory)
+ types = {"image/png": ("png", b"\x89PNG\r\n\x1a\n"), "image/jpeg": ("jpg", b"\xff\xd8\xff"),
+ "image/webp": ("webp", b"RIFF")}
+ for block in result.get("content") or []:
+ if not isinstance(block, dict):
+ continue
+ mime = block.get("mimeType")
+ if block.get("type") != "image" or mime not in types or len(images) >= 4:
+ continue
+ encoded = block.get("data")
+ if not isinstance(encoded, str) or len(encoded) > 24_000_000:
+ continue
+ try:
+ data = base64.b64decode(encoded, validate=True)
+ except (ValueError, TypeError):
+ continue
+ ext, magic = types[mime]
+ if not data.startswith(magic) or (ext == "webp" and data[8:12] != b"WEBP"):
+ continue
+ directory.mkdir(parents=True, exist_ok=True)
+ name = hashlib.sha256(data).hexdigest() + "." + ext
+ target = directory / name
+ if not target.exists():
+ temporary = directory / (name + "." + os.urandom(8).hex() + ".tmp")
+ try:
+ temporary.write_bytes(data)
+ os.replace(temporary, target)
+ finally:
+ temporary.unlink(missing_ok=True)
+ images.append({"url": "tool-media/" + name, "mimeType": mime, "name": "tool-image." + ext})
+ return images
+
+
def _clean(name: str) -> str:
return re.sub(r"[^A-Za-z0-9_-]", "_", str(name)) or "x"
@@ -494,6 +533,7 @@ class McpHub:
"""Every configured server, and the merged, namespaced tool list the model sees."""
def __init__(self, servers: dict[str, dict], settings: dict | None = None):
+ self.media_dir = None
self.settings = {**DEFAULTS, **(settings or {})}
self.servers = {name: McpServer(name, cfg, self.settings) for name, cfg in servers.items()}
self.threads: list[threading.Thread] = []
@@ -560,6 +600,14 @@ class McpHub:
res = server.call(tool, arguments if isinstance(arguments, dict) else {},
float(self.settings["timeout_s"]), cancel)
text, ok = result_text(res), not res.get("isError")
+ if ok and self.settings["image_results"] and self.media_dir is not None:
+ try:
+ images = cache_images(res, self.media_dir)
+ except OSError as e:
+ print(f"[strata] MCP image cache unavailable: {e}", file=sys.stderr, flush=True)
+ else:
+ if images:
+ out["images"] = images
if not ok:
text = "error: " + (text or "the tool reported an error")
except McpCancelled:
@@ -571,6 +619,10 @@ class McpHub:
if cap > 0 and full > cap:
text = text[:cap] + (f"\n\n[... truncated: the tool returned {full:,} characters; only the first "
f"{cap:,} are shown]")
+ if ok and out.get("images"):
+ out["model_content"] = [{"type": "text", "text": text.replace(": not shown to the model]", ": image attached]")}] + [
+ {"type": "image", "source": str(Path(self.media_dir) / im["url"].split("/")[-1])}
+ for im in out["images"]]
out.update(ok=ok, text=text, chars=full, truncated=cap > 0 and full > cap,
ms=int((time.monotonic() - t0) * 1000))
return out
@@ -627,11 +679,15 @@ def servers_from(block, where: str) -> dict[str, dict]:
def settings_from(cfg: dict) -> dict:
- """The run config's optional `"mcp"` block: timeout_s, max_result_chars, max_rounds, start_timeout_s."""
+ """The run config's optional `"mcp"` block: timeout_s, max_result_chars, max_rounds, start_timeout_s, image_results."""
out = {}
for key, value in (cfg.get("mcp") or {}).items():
number = isinstance(value, (int, float)) and not isinstance(value, bool)
- if key in ("timeout_s", "start_timeout_s"):
+ if key == "image_results":
+ if not isinstance(value, bool):
+ raise SystemExit(f"[strata] config mcp.{key}={value!r}: expected a boolean")
+ out[key] = value
+ elif key in ("timeout_s", "start_timeout_s"):
if not number or value <= 0:
raise SystemExit(f"[strata] config mcp.{key}={value!r}: expected a number of seconds > 0")
out[key] = float(value)
diff --git a/serve/server.py b/serve/server.py
index 2a7dc3254..334fab8ba 100644
--- a/serve/server.py
+++ b/serve/server.py
@@ -4824,6 +4824,24 @@ class Service:
prompt; with thinking, Service.run writes it once the thinking is over."""
# `cancel` (#1999): set when the client hangs up; the downloads, the wait for the turn and the encodes stop
# between pictures and raise ClientGone
+ # Cached tool images are references in browser history, never arbitrary file paths or huge base64 strings.
+ for message in messages:
+ for part in message.get("content", []) if isinstance(message.get("content"), list) else []:
+ source = part.get("source", "") if isinstance(part, dict) else ""
+ if isinstance(part, dict) and part.get("type") == "image" and isinstance(source, str) and source.startswith("strata-tool-media:"):
+ name = source[len("strata-tool-media:"):]
+ if not re.fullmatch(r"[0-9a-f]{64}\.(png|jpg|webp)", name):
+ raise ValueError("invalid tool image reference")
+ # Clients resend old tool results. A removed cache or disabled feature must not poison
+ # every later turn; keep the other text and replace only the unavailable picture.
+ media_dir = self.mcp.media_dir if self.mcp is not None else None
+ image = Path(media_dir) / name if media_dir is not None else None
+ if image is None or not image.is_file():
+ why = "tool image cache is disabled" if image is None else "cached tool image is no longer available"
+ part.clear()
+ part.update(type="text", text=f"[image omitted: {why}]")
+ else:
+ part["source"] = str(image)
fetched = self._note_unreadable_tool_images(messages, cancel)
ids = encoded_ids = self.encode_prompt(messages, tools, kwargs)
if force and kwargs.get("enable_thinking", True) is False:
@@ -5787,8 +5805,10 @@ def run_with_mcp(svc: Service, hub, messages, tools, kw, ids, thinking, max_new,
r = box["r"]
print(f"[strata] tool {c.name}: {'ok' if r['ok'] else 'error'}, {r['chars']:,} characters in "
f"{r['ms'] / 1000:.1f} s{' (truncated for the model)' if r['truncated'] else ''}", flush=True)
- results.append(r["text"])
- yield "mcp", {"event": "result", "id": c.id, **{k: r[k] for k in ("ok", "text", "chars", "truncated", "ms")}}
+ results.append(r.get("model_content", r["text"]) if svc.vision is not None else r["text"])
+ yield "mcp", {"event": "result", "id": c.id,
+ **({"images": r["images"], "vision": svc.vision is not None} if r.get("images") else {}),
+ **{k: r[k] for k in ("ok", "text", "chars", "truncated", "ms")}}
if cancel.is_set() or len(results) < len(calls):
done = {**done, "finish": "cancel"}
break
@@ -6381,6 +6401,28 @@ def make_handler(svc: Service):
def do_GET(self):
path = self.path.split("?")[0].rstrip("/")
+ if path.startswith("/tool-media/"):
+ if not self._authorized():
+ return
+ name = path[len("/tool-media/"):]
+ hub = svc.mcp
+ if not re.fullmatch(r"[0-9a-f]{64}\.(png|jpg|webp)", name) or not hub or not hub.media_dir:
+ self._json(404, {"error": {"message": "not found"}})
+ return
+ file = Path(hub.media_dir) / name
+ if not file.is_file():
+ self._json(404, {"error": {"message": "not found"}})
+ return
+ body = file.read_bytes()
+ self.send_response(200)
+ self.send_header("Content-Type", {".png": "image/png", ".jpg": "image/jpeg", ".webp": "image/webp"}[file.suffix])
+ self.send_header("Content-Length", str(len(body)))
+ self.send_header("X-Content-Type-Options", "nosniff")
+ self.send_header("Cache-Control", "private, max-age=86400")
+ self.send_header("Cross-Origin-Resource-Policy", "same-origin")
+ self.end_headers()
+ self.wfile.write(body)
+ return
if path.startswith("/fonts/"):
# the web app's font (Outfit, OFL: serve/web/fonts); the page falls back to the system font
name = path[len("/fonts/"):]
@@ -8618,6 +8660,8 @@ def main(argv=None, *, lifecycle_event=None, interrupt_exit_code: int = 0) -> in
if hub is not None:
import atexit
svc.mcp = hub
+ if hub.settings["image_results"]:
+ hub.media_dir = Path(a.config).with_suffix(".tool-media") if a.config else Path.cwd() / ".tool-media"
print(f"[strata] starting {len(hub.servers)} MCP server{'s' * (len(hub.servers) != 1)} for the web app's "
f"chat: {', '.join(hub.servers)}", flush=True)
hub.start()
diff --git a/serve/test_mcp_media.py b/serve/test_mcp_media.py
new file mode 100644
index 000000000..25a53bf31
--- /dev/null
+++ b/serve/test_mcp_media.py
@@ -0,0 +1,289 @@
+"""Raster MCP delivery: cache bounds, authenticated downloads and vision forwarding; no GPU."""
+import base64
+import json
+import tempfile
+import unittest
+import urllib.error
+import urllib.request
+from pathlib import Path
+from unittest.mock import patch
+from serve.mcp import cache_images, hub_from_config
+from serve.frontend import ChatTemplate, images_of
+from serve.server import ByteTokenizer, Service, serve
+from serve.test_mcp import ScriptedEngine, call_script
+from serve import test_mcp as fixtures
+PNG = base64.b64decode('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jB0kAAAAASUVORK5CYII=')
+
+def image(data=PNG, mime='image/png'):
+ return {'type': 'image', 'mimeType': mime, 'data': base64.b64encode(data).decode()}
+
+class Configuration(unittest.TestCase):
+ def test_documented_switch_reaches_hub_and_tool_result(self):
+ hub = hub_from_config({'mcp_servers': {'fake': {'command': 'unused'}},
+ 'mcp': {'image_results': True}})
+ self.assertTrue(hub.settings['image_results'])
+ server = hub.servers['fake']
+ server.status = 'ready'
+ server.tools = [{'name': 'image', 'inputSchema': {}}]
+ with tempfile.TemporaryDirectory() as tmp:
+ hub.media_dir = Path(tmp)
+ with patch.object(server, 'call', return_value={'content': [image()]}):
+ result = hub.call('fake__image', {})
+ self.assertEqual(len(result['images']), 1)
+ self.assertEqual(result['model_content'][1]['type'], 'image')
+
+ def test_disabled_and_omitted_switch_stay_off(self):
+ for settings in ({}, {'image_results': False}):
+ with self.subTest(settings=settings):
+ hub = hub_from_config({'mcp_servers': {'fake': {'command': 'unused'}}, 'mcp': settings})
+ self.assertFalse(hub.settings['image_results'])
+
+ def test_switch_rejects_non_booleans(self):
+ for value in ('true', 'false', 0, 1, None, [], {}):
+ with self.subTest(value=value), self.assertRaisesRegex(SystemExit, 'expected a boolean'):
+ hub_from_config({'mcp_servers': {'fake': {'command': 'unused'}},
+ 'mcp': {'image_results': value}})
+
+class Cache(unittest.TestCase):
+
+ def test_image_text_obeys_result_cap(self):
+ from serve.mcp import McpHub
+ hub = McpHub({'fake': {'command': 'unused'}}, {'max_result_chars': 1000, 'image_results': True})
+ server = hub.servers['fake']
+ server.status = 'ready'
+ server.tools = [{'name': 'image', 'inputSchema': {}}]
+ with tempfile.TemporaryDirectory() as tmp:
+ hub.media_dir = Path(tmp)
+ with patch.object(server, 'call', return_value={'content': [{'type': 'text', 'text': 'x' * 100000}, image()]}):
+ result = hub.call('fake__image', {})
+ self.assertTrue(result['truncated'])
+ self.assertLessEqual(len(result['model_content'][0]['text']), len(result['text']))
+ self.assertNotIn('x' * 1001, result['model_content'][0]['text'])
+ self.assertEqual(len(result['images']), 1)
+
+ def test_disabled_is_identical_to_text_only_result(self):
+ from serve.mcp import McpHub, result_text
+ hub = McpHub({'fake': {'command': 'unused'}})
+ server = hub.servers['fake']
+ server.status = 'ready'
+ server.tools = [{'name': 'image', 'inputSchema': {}}]
+ result = {'content': [{'type': 'text', 'text': 'hello'}, image()]}
+ with tempfile.TemporaryDirectory() as tmp:
+ hub.media_dir = Path(tmp) / 'unused'
+ with patch.object(server, 'call', return_value=result), patch('serve.mcp.cache_images') as cache:
+ output = hub.call('fake__image', {})
+ cache.assert_not_called()
+ self.assertEqual(output['text'], result_text(result))
+ self.assertNotIn('images', output)
+ self.assertNotIn('model_content', output)
+ self.assertFalse(hub.media_dir.exists())
+
+ def test_cache_failure_keeps_text_result(self):
+ from serve.mcp import McpHub
+ hub = McpHub({'fake': {'command': 'unused'}}, {'image_results': True})
+ server = hub.servers['fake']
+ server.status = 'ready'
+ server.tools = [{'name': 'image', 'inputSchema': {}}]
+ hub.media_dir = Path('unused')
+ with patch.object(server, 'call', return_value={'content': [{'type': 'text', 'text': 'hello'}, image()]}), patch('serve.mcp.cache_images', side_effect=OSError('disk full')):
+ output = hub.call('fake__image', {})
+ self.assertTrue(output['ok'])
+ self.assertIn('hello', output['text'])
+ self.assertNotIn('model_content', output)
+
+ def test_null_and_non_object_content(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ self.assertEqual(cache_images({'content': None}, tmp), [])
+ self.assertEqual(cache_images({'content': [None, False, 'bad', image()]}, tmp), cache_images({'content': [image()]}, tmp))
+
+ def test_cache_deduplicates_and_preserves_bytes(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ a = cache_images({'content': [image()]}, tmp)
+ b = cache_images({'content': [image()]}, tmp)
+ self.assertEqual(a, b)
+ self.assertEqual(len(list(Path(tmp).iterdir())), 1)
+ self.assertEqual((Path(tmp) / a[0]['url'].split('/')[-1]).read_bytes(), PNG)
+
+ def test_rejects_invalid_and_active_content(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ blocks = [image(b'not a PNG'), image(b'<svg/>', 'image/svg+xml'), {'type': 'image', 'mimeType': 'image/png', 'data': '***'}]
+ self.assertEqual(cache_images({'content': blocks}, tmp), [])
+
+ def test_caps_images_per_result(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ self.assertEqual(len(cache_images({'content': [image()] * 8}, tmp)), 4)
+
+ def test_authenticated_download_and_traversal(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ media = cache_images({'content': [image()]}, tmp)[0]
+ tok = ByteTokenizer()
+ svc = Service(ScriptedEngine(tok, ['OK']), tok, ChatTemplate(Path(__file__).parent / 'chat_template.jinja'))
+
+ class Hub:
+ media_dir = tmp
+ svc.mcp = Hub()
+ svc.api_key = 'test-key'
+ httpd = serve(svc, port=0)
+ base = f'http://127.0.0.1:{httpd.server_address[1]}/'
+ try:
+ with self.assertRaises(urllib.error.HTTPError) as error:
+ urllib.request.urlopen(base + media['url'])
+ self.assertEqual(error.exception.code, 401)
+ req = urllib.request.Request(base + media['url'], headers={'Authorization': 'Bearer test-key'})
+ with urllib.request.urlopen(req) as response:
+ self.assertEqual(response.read(), PNG)
+ self.assertEqual(response.headers['Content-Type'], 'image/png')
+ req = urllib.request.Request(base + 'tool-media/../../secret.png', headers={'Authorization': 'Bearer test-key'})
+ with self.assertRaises(urllib.error.HTTPError) as error:
+ urllib.request.urlopen(req)
+ self.assertEqual(error.exception.code, 404)
+ finally:
+ httpd.shutdown()
+ httpd.server_close()
+
+class HistoryReferences(unittest.TestCase):
+
+ def test_reference_resolves_only_to_cache(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ im = cache_images({'content': [image()]}, tmp)[0]
+ name = im['url'].split('/')[-1]
+ tok = ByteTokenizer()
+ svc = Service(ScriptedEngine(tok, ['OK']), tok, ChatTemplate(Path(__file__).parent / 'chat_template.jinja'))
+
+ class Hub:
+ media_dir = tmp
+ svc.mcp = Hub()
+ message = [{'role': 'tool', 'content': [{'type': 'image', 'source': 'strata-tool-media:' + name}]}]
+ with patch.object(svc, '_note_unreadable_tool_images', return_value={}):
+ with self.assertRaisesRegex(ValueError, 'without the vision encoder'):
+ svc.prepare(message, [], {}, 16)
+ self.assertEqual(images_of(message), [str(Path(tmp) / name)])
+
+ def test_mixed_missing_and_valid_pictures_keeps_valid_vision(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ media = cache_images({'content': [image()]}, tmp)[0]
+ source = str(Path(tmp) / media['url'].split('/')[-1])
+ rows = Path(tmp) / 'img.sve'
+ rows.write_bytes(b'rows')
+ class Vision:
+ dir = Path(tmp)
+
+ def encode(self, value):
+ if value != source:
+ raise AssertionError('Only the valid cached image may reach the encoder')
+ return rows, 3
+ tok = ByteTokenizer()
+ svc = Service(ScriptedEngine(tok, ['OK']), tok,
+ ChatTemplate(Path(__file__).parent / 'chat_template.jinja'), vision=Vision())
+ class Hub:
+ media_dir = tmp
+ svc.mcp = Hub()
+ messages = [{'role': 'tool', 'content': [
+ {'type': 'text', 'text': 'Retain this text.'},
+ {'type': 'image', 'source': 'strata-tool-media:' + '0' * 64 + '.png'},
+ {'type': 'image', 'source': 'strata-tool-media:' + media['url'].split('/')[-1]}]}]
+ try:
+ ids, _, _ = svc.prepare(messages, [], {}, 16)
+ prompt = tok.decode(ids)
+ self.assertIn('Retain this text.', prompt)
+ self.assertIn('[image omitted:', prompt)
+ self.assertEqual(images_of(messages), [source])
+ self.assertEqual(ids.count(tok.encode('<|image_pad|>', parse_special=True)[0]), 3)
+ finally:
+ if svc.embeddings.path:
+ svc.embeddings.path.unlink(missing_ok=True)
+
+ def test_bad_reference_cannot_read_other_files(self):
+ tok = ByteTokenizer()
+ svc = Service(ScriptedEngine(tok, ['OK']), tok, ChatTemplate(Path(__file__).parent / 'chat_template.jinja'))
+ with self.assertRaisesRegex(ValueError, 'invalid tool image reference'):
+ svc.prepare([{'role': 'tool', 'content': [{'type': 'image', 'source': 'strata-tool-media:../../secret.png'}]}], [], {}, 16)
+
+class ImageLoop(unittest.TestCase):
+ setUp = fixtures.ToolLoop.setUp
+ tearDown = fixtures.ToolLoop.tearDown
+ start = fixtures.ToolLoop.start
+ post = fixtures.ToolLoop.post
+ chunks = staticmethod(fixtures.ToolLoop.chunks)
+
+ def check_stale_history(self, state):
+ self.start('</think>\n\nText reply survives.')
+ with tempfile.TemporaryDirectory() as tmp:
+ self.hub.media_dir = Path(tmp) if state == 'missing' else None
+ if state == 'no_hub':
+ self.svc.mcp = None
+ history = [
+ {'role': 'user', 'content': 'Create a picture'},
+ {'role': 'assistant', 'content': '', 'tool_calls': [
+ {'id': 'old', 'type': 'function', 'function': {'name': 'fake__echo', 'arguments': '{}'}}]},
+ {'role': 'tool', 'tool_call_id': 'old', 'content': [
+ {'type': 'text', 'text': 'Keep this original tool text.'},
+ {'type': 'image_url', 'image_url': {'url': 'strata-tool-media:' + '0' * 64 + '.png'}}]},
+ {'role': 'user', 'content': 'Continue with text only'}]
+ # API clients resend the original unmodified history on every later turn.
+ for turn in range(2):
+ for stream in (False, True):
+ with self.subTest(state=state, turn=turn, stream=stream):
+ code, text = self.post({'messages': history}, stream=stream)
+ self.assertEqual(code, 200, text)
+ if stream:
+ answer = ''.join(c['choices'][0]['delta'].get('content', '') for c in self.chunks(text))
+ else:
+ answer = json.loads(text)['choices'][0]['message']['content']
+ self.assertIn('Text reply survives.', answer)
+ prompt = self.engine.prompt_text(turn * 2 + int(stream))
+ self.assertIn('Keep this original tool text.', prompt)
+ self.assertIn('[image omitted:', prompt)
+ self.assertNotIn('strata-tool-media:', prompt)
+
+ def test_missing_cache_history_continues_across_turns(self):
+ self.check_stale_history('missing')
+
+ def test_disabled_cache_history_continues_across_turns(self):
+ self.check_stale_history('disabled')
+
+ def test_removed_mcp_history_continues_across_turns(self):
+ self.check_stale_history('no_hub')
+
+ def test_malformed_history_still_rejected_over_http(self):
+ self.start('</think>\n\nShould not run.')
+ self.svc.mcp = None
+ for name in ('../../secret.png', 'a' * 64 + '.svg', 'A' * 64 + '.png', 'short.png'):
+ with self.subTest(name=name):
+ code, text = self.post({'messages': [{'role': 'tool', 'content': [
+ {'type': 'image_url', 'image_url': {'url': 'strata-tool-media:' + name}}]}]}, stream=False)
+ self.assertEqual(code, 400)
+ self.assertIn('invalid tool image reference', text)
+ self.assertEqual(self.engine.prompts, [])
+
+ def test_default_stream_has_no_image_fields(self):
+ self.start(call_script('fake__echo', text='hello'), '</think>\n\nDone.')
+ code, text = self.post({'strata_mcp': True})
+ self.assertEqual(code, 200)
+ event = [c['strata_mcp'] for c in self.chunks(text) if c.get('strata_mcp', {}).get('event') == 'result'][0]
+ self.assertEqual(set(event), {'event', 'id', 'ok', 'text', 'chars', 'truncated', 'ms'})
+
+ def test_image_result_streams_and_reaches_vision_prepare(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ self.hub.settings['image_results'] = True
+ self.hub.media_dir = Path(tmp)
+ im = cache_images({'content': [image()]}, tmp)
+ source = str(Path(tmp) / im[0]['url'].split('/')[-1])
+ self.start(call_script('fake__echo', text='picture'), '</think>\n\nImage received.')
+ self.svc.vision = object()
+ native = self.svc.prepare
+ seen = []
+
+ def prepare(messages, *args, **kwargs):
+ seen.extend(images_of(messages))
+ clean = [{**m, 'content': 'image delivered'} if isinstance(m.get('content'), list) else m for m in messages]
+ return native(clean, *args, **kwargs)
+ result = {'ok': True, 'text': 'image delivered', 'chars': 15, 'truncated': False, 'ms': 1, 'images': im, 'model_content': [{'type': 'text', 'text': 'image delivered'}, {'type': 'image', 'source': source}]}
+ with patch.object(self.hub, 'call', return_value=result), patch.object(self.svc, 'prepare', side_effect=prepare):
+ code, text = self.post({'strata_mcp': True})
+ self.assertEqual(code, 200)
+ self.assertIn(source, seen)
+ events = [c['strata_mcp'] for c in self.chunks(text) if 'strata_mcp' in c]
+ self.assertEqual(events[-1]['images'], im)
+ answer = ''.join((c['choices'][0]['delta'].get('content', '') for c in self.chunks(text)))
+ self.assertIn('Image received.', answer)
diff --git a/serve/web/app.css b/serve/web/app.css
index f9bad8ccd..de457cc70 100644
--- a/serve/web/app.css
+++ b/serve/web/app.css
@@ -197,3 +197,9 @@ main { flex: 1; min-height: 0; display: flex; }
.img-results figure{margin:0}
.img-results img{max-width:100%;border-radius:var(--st-r-lg);display:block}
.img-results figcaption{font-size:var(--st-fs-sm, 13px);color:var(--st-ink-2, inherit);margin-top:4px}
+
+/* Raster MCP results remain visible outside the collapsed tool details. */
+.tool-image { margin: 16px 0; max-width: 100%; }
+.tool-image img { display: block; width: 100%; max-width: 640px; height: auto; border-radius: var(--st-r-md); }
+.tool-image figcaption { margin-top: 10px; }
+.tool-image a:not([href]) { opacity: .5; pointer-events: none; }
diff --git a/serve/web/app.js b/serve/web/app.js
index d01ff2974..86fb16a8c 100644
--- a/serve/web/app.js
+++ b/serve/web/app.js
@@ -658,6 +658,38 @@ function toolHtml(t, k) {
`<span class="st-badge ${cls}">${esc(label)}</span>${t.ms != null && t.state !== "skipped" ? `<span class="muted small">${fmt(t.ms / 1000, 1)} s</span>` : ""}` +
`${icon("chevron", "st-icon st-icon--sm st-chev")}</summary><div class="st-collapse__body">${body}</div></details>`;
}
+// Raster tool results are same-origin cached files; credentials stay in request headers.
+const mediaBlobs = new Map();
+let mediaEpoch = 0;
+function clearToolMedia() {
+ mediaEpoch++;
+ for (const pending of mediaBlobs.values()) pending.then((url) => URL.revokeObjectURL(url), () => {});
+ mediaBlobs.clear();
+}
+window.addEventListener("pagehide", clearToolMedia);
+window.addEventListener("pageshow", (event) => { if (event.persisted) hydrateToolImages(document); });
+function safeToolMedia(url) { return typeof url === "string" && /^tool-media\/[0-9a-f]{64}\.(png|jpg|webp)$/.test(url); }
+function toolImages(t) {
+ return (t.images || []).filter((im) => safeToolMedia(im.url)).map((im) =>
+ `<figure class="tool-image"><img data-tool-image="${esc(im.url)}" alt="Tool image" loading="lazy">` +
+ `<figcaption><a class="st-btn" data-tool-download="${esc(im.url)}" download="${esc(im.name || "generated-image.png")}">Download image</a></figcaption></figure>`).join("");
+}
+function hydrateToolImages(root) {
+ root.querySelectorAll("[data-tool-image]").forEach((img) => {
+ const url = img.dataset.toolImage, epoch = mediaEpoch;
+ if (!safeToolMedia(url)) return;
+ if (!mediaBlobs.has(url)) mediaBlobs.set(url, fetch(url, {headers: headers()}).then((r) => {
+ if (!r.ok) throw new Error("Image unavailable");
+ return r.blob();
+ }).then((b) => URL.createObjectURL(b)).catch((e) => {if (epoch === mediaEpoch) mediaBlobs.delete(url); throw e;}));
+ mediaBlobs.get(url).then((blob) => {
+ if (epoch !== mediaEpoch || !img.isConnected) return;
+ img.src = blob;
+ const a = img.closest("figure").querySelector("[data-tool-download]");
+ a.href = blob;
+ }).catch(() => { img.alt = "Image unavailable; try refreshing"; });
+ });
+}
// the answer's text with the tool blocks where the model called them
function answerHtml(m) {
if (!m.tools || !m.tools.length) return markdown(m.text || "");
@@ -666,7 +698,7 @@ function answerHtml(m) {
const at = Math.min(Math.max(t.at || 0, pos), m.text.length);
if (at > pos) html += markdown(m.text.slice(pos, at));
pos = at;
- html += toolHtml(t, k);
+ html += toolHtml(t, k) + toolImages(t);
});
return html + markdown(m.text.slice(pos));
}
@@ -679,7 +711,7 @@ function onTool(m, x) {
if (x.event === "call") {
Object.assign(t, {name: x.name, server: x.server, tool: x.tool, arguments: x.arguments, round: x.round, state: "running"});
} else if (x.event === "result") {
- Object.assign(t, {result: x.text, ok: x.ok, chars: x.chars, truncated: x.truncated, ms: x.ms,
+ Object.assign(t, {result: x.text, ok: x.ok, chars: x.chars, truncated: x.truncated, ms: x.ms, images: x.images || [], vision: x.vision === true,
state: x.skipped ? "skipped" : x.ok ? "done" : "error"});
}
}
@@ -705,6 +737,7 @@ function updateAssistant(el, m, streaming) {
bubble.innerHTML = m.reasoning ? `<span class="muted cursor">Writing</span>` : `<span class="cursor"></span>`;
} else {
bubble.innerHTML = answerHtml(m);
+ hydrateToolImages(bubble);
if (streaming) bubble.classList.add("cursor"); else bubble.classList.remove("cursor");
}
el.querySelector(".meta-text").textContent = m.meta || (streaming ? "" : m.stopped ? "Stopped" : "");
@@ -805,7 +838,12 @@ function assistantMessages(m) {
const at = Math.min(Math.max(pos, calls[0].at || 0), m.text.length);
out.push({role: "assistant", content: m.text.slice(pos, at).trim(),
tool_calls: calls.map((t) => ({id: t.id, type: "function", function: {name: t.name, arguments: JSON.stringify(t.arguments || {})}}))});
- for (const t of calls) out.push({role: "tool", tool_call_id: t.id, content: t.result});
+ for (const t of calls) {
+ const pictures = t.vision ? (t.images || []).filter((im) => safeToolMedia(im.url)) : [];
+ const content = pictures.length ? [{type: "text", text: t.result}, ...pictures.map((im) =>
+ ({type: "image_url", image_url: {url: "strata-tool-media:" + im.url.split("/").pop()}}))] : t.result;
+ out.push({role: "tool", tool_call_id: t.id, content});
+ }
pos = at;
}
const rest = m.text.slice(pos).trim();
@@ -931,6 +969,7 @@ $("new-btn").onclick = () => {
if (busy) { toast("warn", "Still writing", "Stop the answer first."); return; }
if (!messages.length) return;
const backup = messages;
+ clearToolMedia();
messages = [];
saveChat();
renderChat();