Skip to content

Self-improving AI Employees for everyone

feat(data,vision): dataSource agent tool, visionAnalyze delegate, native image blocks

trohitg6820944f18ab
diff --git a/CLAUDE.md b/CLAUDE.md
index 6e80af7c..fde275a7 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -36,6 +36,7 @@ This is a React Flow-based workflow automation platform implementing n8n-inspire
| **[Sarvam AI Service](./docs-internal/sarvam_service.md)** | Sarvam AI (Indic-first). **`nodes/sarvam/` no longer exists** — every capability it served is now a *provider* inside an abstracted node: speech in `nodes/speech/`, and translate / transliterate / detect-language in `nodes/translate/`. What remains Sarvam-specific is `sarvamChatModel` (an **OpenAI-compat provider**: one `_COMPAT_PROVIDERS` entry + a JSON block, no provider subclass) under `nodes/model/`. **One stored key spans two auth styles** — the chat route takes `Authorization: Bearer` (what the openai SDK sends) while every other endpoint takes only `api-subscription-key`, so `SarvamCredential` declares the native header and every surface shares it. Introduces the generic **`supports_model_listing: false`** flag: Sarvam ships no `/v1/models` route, and unhandled that 404 becomes a `NodeUserError` that `AIService.fetch_models` re-raises *before* its curated fallback, breaking key validation and the model dropdown — so `OpenAIProvider.fetch_models` serves the curated list and probes with a one-token completion instead (defaults `true`; the other 12 providers are untouched, locked by test). Also added generic **`Connection.request(files=)`** multipart support. |
| **[Speech Provider RFC](./docs-internal/speech_provider_rfc.md)** | Provider-abstracted speech — two nodes (`textToSpeech`, `speechToText`) with a `provider` dropdown, backed by **two registries, one per direction** (registry membership IS the capability: a synthesis-only vendor simply never appears in the transcription enum). Everything vendor-specific lives in the plugin folder [`server/nodes/speech/`](./server/nodes/speech/) — protocol, registries, dispatch, per-vendor modules; only the generic `services/provider_registry.py` and the vendor-neutral `services/media/` are shared. v1 providers: OpenAI (both directions), ElevenLabs (TTS), Deepgram + Groq (STT), Sarvam (both). Capabilities are JSON (`server/config/speech_defaults.json`) with per-model overrides, so no shared code branches on a provider name. **Audio never travels as bytes** — nodes return an `AudioRef` (a ~400 B reference) because a 12 MB base64 result blows Temporal's 2 MiB blob limit and retries 3× re-billing the provider each time. Backing routes: `GET/POST /api/workspace/{workflow_id}/…` in [`routers/workspace.py`](./server/routers/workspace.py). |
| **[Media Transport](./docs-internal/media_transport.md)** | How files move through the engine, and why they move as **references rather than bytes**. `services/media/` — `FileRef` (the base; `FileKind = file\|audio\|image\|video\|document`) with `AudioRef(FileRef)` as the probed-container narrowing (~400 B, `extra="forbid"` so adding a bytes field is a `ValidationError` rather than a silent regression), `write_audio` / `resolve_media` / `coerce_file_param` (the one nodes should call — accepts a `FileRef`/`AudioRef`, a bare path, or the legacy base64 envelope), `inspect_audio` (tinytag → stdlib `wave` → PCM arithmetic, **never raises**), `limits.py`, and **`preview.py` — the single owner of the inline-vs-attachment rule**, with two consumers: `routers/workspace.py` picks its `Content-Disposition` from `serves_inline`, and the gallery listing sets each row's `preview` from `preview_kind`, so the panel can never offer a preview the route refuses to serve. Containment runs through `resolve_within`, which closed a live vulnerability: a node joined a user path onto the workspace root unchecked, so `audio_file="../../credentials.db"` read the encrypted credential store and uploaded it. **The id/slug asymmetry is the trap** — workspace dirs are named by the mutable `Workflow.slug` while a ref stores the immutable `workflow_id` so refs survive rename; the lookup lives in [`services/workspace_locator.py`](./server/services/workspace_locator.py) (`resolve_workspace_root(..., allow_default=)`), and **every mutating caller must pass `allow_default=False`** or an unresolvable id silently lands the write in the shared anonymous workspace. HTTP surface in [`routers/workspace.py`](./server/routers/workspace.py): Range support comes free from Starlette's `FileResponse`; `NEVER_INLINE` is `{image/svg+xml, text/html, text/xml, application/xhtml+xml}` because `shell` / `fileDownloader` / `fileModify` can write arbitrary files into a workspace. |
+| **[Data Node & Vision](./docs-internal/data_node.md)** | The `dataSource` agent tool (raw local data over two namespaces: workspace paths + operator-approved `mnt/<name>/...` external mounts with per-mount writable flags; typed bounded read tiers text/csv/json/pdf/html/xlsx/image-metadata/binary; write/append but **no delete**; `copy_to_workspace` imports mount files as real FileRefs), the machine-wide `data_mounts` allowlist (`services/data/mount_store.py` — refuses `DATA_DIR` overlap, drive roots, home dir), the `visionAnalyze` delegate tool (vision for every host model via openai/anthropic/gemini official SDK shapes + the `services/media/image_fit.py` visual-token budget), and the first native-image-blocks increment: `ContentBlock.source` (durable `file_ref` vs transient `bytes` — the wire codec raises on bytes), the `llm_media` tool-result opt-in, capability-gated `hydrate_image_blocks` in `run_native_llm_step`, Anthropic `tool_result` image encoding (openai/gemini encoders pending; their `vision.enabled` stays false until then). |
| **[Translate / Transliterate / Detect](./docs-internal/speech_provider_rfc.md#8-what-this-pattern-should-absorb-next)** | The second application of the multi-vendor pattern, and the one that retired `nodes/sarvam/`. Three nodes (`translateText`, `transliterateText`, `detectLanguage`) in [`server/nodes/translate/`](./server/nodes/translate/), backed by **three registries — one per capability**, because the asymmetry is sharper than speech's: DeepL translates and does nothing else, while Sarvam and LLM-backed providers do all three. Membership makes an unsupported selection unrepresentable rather than a runtime failure inside a paid call. Providers: **deepl** (`Authorization: DeepL-Auth-Key`, `text` is an array, free keys ending `:fx` route to a different host, and it returns `billed_characters` so cost is the provider's own figure), **sarvam** (all three over dedicated REST), **openai** (all three via `ChatUnifier` — proves the Protocol does not care what a provider does internally; billing deliberately NOT recorded because that path bills tokens the LLM layer already costs). Capabilities in `server/config/translate_defaults.json`. |
| **[CI/CD Pipeline](./docs-internal/ci_cd.md)** | GitHub Actions workflows, predeploy validation, release publishing, and composite setup action |
| **[Performance](./docs-internal/performance.md)** | Cold-start measurements (Application startup complete 2.90 s warm, 21.5 s cold post-`company clean` — was 71 s before the 2026-07-14 boot-delay fixes; first WS connect 8.29 s) + warm and cold per-phase timelines + optimisation history (lazy LangChain imports, lazy LLM-SDK exception refs, esbuild sidecar bundle, bytecode-at-build, PartySocket reconnect, TanStack Query auth bootstrap, `manualChunks`, Vite dep-cache preservation, off-loop Temporal build-id hash) + bottleneck inventory + reproduction commands + anti-patterns to never reintroduce (incl. eager SDK import at LLM provider registration, unconditional `.vite` wipe, synchronous `Worker()` on the event loop). Pairs with the build-time pipeline doc below. |
diff --git a/client/src/components/parameterPanel/DataPanel.tsx b/client/src/components/parameterPanel/DataPanel.tsx
new file mode 100644
index 00000000..287fd498
--- /dev/null
+++ b/client/src/components/parameterPanel/DataPanel.tsx
@@ -0,0 +1,439 @@
+import React, { useState } from 'react';
+import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';
+import {
+ File,
+ Folder,
+ HardDrive,
+ Home,
+ Loader2,
+ Plus,
+ Trash2,
+} from 'lucide-react';
+import { toast } from 'sonner';
+
+import {
+ AlertDialog,
+ AlertDialogAction,
+ AlertDialogCancel,
+ AlertDialogContent,
+ AlertDialogDescription,
+ AlertDialogFooter,
+ AlertDialogHeader,
+ AlertDialogTitle,
+} from '@/components/ui/alert-dialog';
+import { Badge } from '@/components/ui/badge';
+import { Button } from '@/components/ui/button';
+import { Checkbox } from '@/components/ui/checkbox';
+import { Input } from '@/components/ui/input';
+import { Switch } from '@/components/ui/switch';
+import { useWebSocket } from '@/contexts/WebSocketContext';
+
+const BROWSE_LIMIT = 100;
+
+/**
+ * The panel is UI only. Every decision — mount validation, containment,
+ * listing rows, breadcrumbs, writability — is made by the backend handlers
+ * (`data_list_mounts` / `data_add_mount` / `data_update_mount` /
+ * `data_remove_mount` / `data_browse`); this component renders responses
+ * verbatim and writes the node's `mounts` parameter.
+ */
+
+interface MountRow {
+ name: string;
+ root_path: string;
+ writable: boolean;
+}
+
+interface MountsResponse {
+ success?: boolean;
+ mounts?: MountRow[];
+ error?: string;
+}
+
+interface BrowseEntry {
+ name: string;
+ is_dir: boolean;
+ size_bytes?: number;
+ mime_type?: string | null;
+ /** Workspace rows carry `path`; mount rows carry `location`. */
+ path?: string;
+ location?: string;
+ writable?: boolean;
+}
+
+interface BrowseResponse {
+ success?: boolean;
+ source?: 'workspace' | 'mount';
+ path?: string;
+ crumbs?: Array<{ name: string; path: string }>;
+ entries?: BrowseEntry[];
+ truncated?: boolean;
+ error?: string;
+}
+
+interface DataPanelProps {
+ nodeId: string;
+ workflowId?: string;
+ parameters: Record<string, unknown>;
+ onParameterChange: (name: string, value: unknown) => void;
+}
+
+const formatSize = (bytes?: number): string => {
+ if (!bytes && bytes !== 0) return '';
+ if (bytes < 1024) return `${bytes} B`;
+ if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
+ return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
+};
+
+const DataPanel: React.FC<DataPanelProps> = ({
+ nodeId,
+ workflowId,
+ parameters,
+ onParameterChange,
+}) => {
+ const { sendRequest } = useWebSocket();
+ const queryClient = useQueryClient();
+ const [browsePath, setBrowsePath] = useState('');
+ const [draftName, setDraftName] = useState('');
+ const [draftPath, setDraftPath] = useState('');
+ const [draftWritable, setDraftWritable] = useState(false);
+ const [removeTarget, setRemoveTarget] = useState<string | null>(null);
+
+ const enabledMounts = Array.isArray(parameters.mounts)
+ ? (parameters.mounts as string[])
+ : [];
+
+ const mountsQuery = useQuery<MountsResponse, Error>({
+ queryKey: ['dataMounts'],
+ queryFn: () => sendRequest<MountsResponse>('data_list_mounts', {}),
+ });
+ const mounts = mountsQuery.data?.mounts ?? [];
+
+ const browseQuery = useQuery<BrowseResponse, Error>({
+ queryKey: ['dataBrowse', workflowId ?? '', nodeId, browsePath],
+ queryFn: () =>
+ sendRequest<BrowseResponse>('data_browse', {
+ workflow_id: workflowId,
+ data_node_id: nodeId,
+ path: browsePath,
+ limit: BROWSE_LIMIT,
+ }),
+ enabled: !!workflowId && !!nodeId,
+ });
+ const browse = browseQuery.data;
+
+ const invalidateMounts = async () => {
+ await queryClient.invalidateQueries({ queryKey: ['dataMounts'] });
+ await queryClient.invalidateQueries({
+ queryKey: ['dataBrowse', workflowId ?? '', nodeId],
+ });
+ };
+
+ const addMount = useMutation({
+ mutationFn: () =>
+ sendRequest<MountsResponse>('data_add_mount', {
+ name: draftName.trim(),
+ root_path: draftPath.trim(),
+ writable: draftWritable,
+ }),
+ onSuccess: async (result) => {
+ if (result.success === false) {
+ toast.error(result.error || 'Mount rejected');
+ return;
+ }
+ setDraftName('');
+ setDraftPath('');
+ setDraftWritable(false);
+ toast.success('Mount added');
+ await invalidateMounts();
+ },
+ onError: (error: Error) => toast.error(error.message),
+ });
+
+ const updateMount = useMutation({
+ mutationFn: (input: { name: string; writable: boolean }) =>
+ sendRequest<MountsResponse>('data_update_mount', input),
+ onSuccess: async (result) => {
+ if (result.success === false) {
+ toast.error(result.error || 'Update rejected');
+ return;
+ }
+ await invalidateMounts();
+ },
+ onError: (error: Error) => toast.error(error.message),
+ });
+
+ const removeMount = useMutation({
+ mutationFn: (name: string) =>
+ sendRequest<MountsResponse>('data_remove_mount', { name }),
+ onSuccess: async (result, name) => {
+ if (result.success === false) {
+ toast.error(result.error || 'Remove rejected');
+ return;
+ }
+ // Drop the removed name from this node's enabled subset too.
+ if (enabledMounts.includes(name)) {
+ onParameterChange(
+ 'mounts',
+ enabledMounts.filter((mount) => mount !== name),
+ );
+ }
+ toast.success('Mount removed');
+ await invalidateMounts();
+ },
+ onError: (error: Error) => toast.error(error.message),
+ });
+
+ const toggleEnabled = (name: string, enabled: boolean) => {
+ const next = enabled
+ ? [...enabledMounts.filter((mount) => mount !== name), name]
+ : enabledMounts.filter((mount) => mount !== name);
+ onParameterChange('mounts', next);
+ };
+
+ const navigateTo = (entry: BrowseEntry) => {
+ if (!entry.is_dir) return;
+ setBrowsePath(entry.path ?? entry.location ?? '');
+ };
+
+ return (
+ <div className="flex h-full min-h-0 flex-col gap-4 overflow-y-auto p-1">
+ {/* Sources: machine-wide mounts + this node's enabled subset */}
+ <section className="space-y-2">
+ <div className="flex items-center justify-between">
+ <h3 className="text-sm font-medium text-foreground">Sources</h3>
+ <span className="text-xs text-muted-foreground">
+ Machine-wide — shared by every Data node
+ </span>
+ </div>
+
+ <div className="flex items-center gap-2 rounded-md border border-border bg-card px-3 py-2">
+ <Home className="h-4 w-4 text-muted-foreground" />
+ <span className="flex-1 text-sm">Workspace</span>
+ <Badge variant="outline">read + write</Badge>
+ <Badge variant="secondary">always on</Badge>
+ </div>
+
+ {mountsQuery.isLoading ? (
+ <div className="flex items-center gap-2 px-3 py-2 text-sm text-muted-foreground">
+ <Loader2 className="h-4 w-4 animate-spin" /> Loading mounts…
+ </div>
+ ) : (
+ mounts.map((mount) => (
+ <div
+ key={mount.name}
+ className="flex items-center gap-2 rounded-md border border-border bg-card px-3 py-2"
+ >
+ <Checkbox
+ checked={enabledMounts.includes(mount.name)}
+ onCheckedChange={(checked) =>
+ toggleEnabled(mount.name, checked === true)
+ }
+ aria-label={`Expose ${mount.name} to this node`}
+ />
+ <HardDrive className="h-4 w-4 text-muted-foreground" />
+ <div className="min-w-0 flex-1">
+ <div className="truncate text-sm">{mount.name}</div>
+ <div
+ className="truncate text-xs text-muted-foreground"
+ title={mount.root_path}
+ >
+ {mount.root_path}
+ </div>
+ </div>
+ <div className="flex items-center gap-1">
+ <Switch
+ checked={mount.writable}
+ onCheckedChange={(checked) =>
+ updateMount.mutate({
+ name: mount.name,
+ writable: checked === true,
+ })
+ }
+ aria-label={`Writable flag for ${mount.name}`}
+ />
+ <span className="text-xs text-muted-foreground">
+ {mount.writable ? 'read + write' : 'read-only'}
+ </span>
+ </div>
+ <Button
+ variant="ghost"
+ size="icon"
+ onClick={() => setRemoveTarget(mount.name)}
+ aria-label={`Remove mount ${mount.name}`}
+ >
+ <Trash2 className="h-4 w-4 text-destructive" />
+ </Button>
+ </div>
+ ))
+ )}
+
+ <div className="flex flex-col gap-2 rounded-md border border-dashed border-border p-3">
+ <div className="flex gap-2">
+ <Input
+ value={draftName}
+ onChange={(event) => setDraftName(event.target.value)}
+ placeholder="mount name (e.g. reports)"
+ className="w-40"
+ />
+ <Input
+ value={draftPath}
+ onChange={(event) => setDraftPath(event.target.value)}
+ placeholder="absolute folder path"
+ className="flex-1"
+ />
+ </div>
+ <div className="flex items-center justify-between">
+ <label className="flex items-center gap-2 text-xs text-muted-foreground">
+ <Switch
+ checked={draftWritable}
+ onCheckedChange={(checked) =>
+ setDraftWritable(checked === true)
+ }
+ aria-label="New mount writable"
+ />
+ writable
+ </label>
+ <Button
+ size="sm"
+ onClick={() => addMount.mutate()}
+ disabled={
+ addMount.isPending || !draftName.trim() || !draftPath.trim()
+ }
+ >
+ {addMount.isPending ? (
+ <Loader2 className="mr-1 h-4 w-4 animate-spin" />
+ ) : (
+ <Plus className="mr-1 h-4 w-4" />
+ )}
+ Add mount
+ </Button>
+ </div>
+ </div>
+ </section>
+
+ {/* Read-only browser over workspace + enabled mounts */}
+ <section className="flex min-h-0 flex-1 flex-col gap-2">
+ <h3 className="text-sm font-medium text-foreground">Browse</h3>
+ <div className="flex flex-wrap items-center gap-1 text-xs">
+ <Button
+ variant={browsePath === '' ? 'secondary' : 'ghost'}
+ size="sm"
+ onClick={() => setBrowsePath('')}
+ >
+ <Home className="mr-1 h-3 w-3" /> Workspace
+ </Button>
+ {mounts
+ .filter((mount) => enabledMounts.includes(mount.name))
+ .map((mount) => (
+ <Button
+ key={mount.name}
+ variant={
+ browsePath.startsWith(`mnt/${mount.name}`)
+ ? 'secondary'
+ : 'ghost'
+ }
+ size="sm"
+ onClick={() => setBrowsePath(`mnt/${mount.name}`)}
+ >
+ <HardDrive className="mr-1 h-3 w-3" /> {mount.name}
+ </Button>
+ ))}
+ </div>
+
+ {(browse?.crumbs?.length ?? 0) > 0 && (
+ <div className="flex flex-wrap items-center gap-1 text-xs text-muted-foreground">
+ {browse?.crumbs?.map((crumb) => (
+ <React.Fragment key={crumb.path}>
+ <span>/</span>
+ <button
+ type="button"
+ className="hover:text-foreground"
+ onClick={() => setBrowsePath(crumb.path)}
+ >
+ {crumb.name}
+ </button>
+ </React.Fragment>
+ ))}
+ </div>
+ )}
+
+ <div className="min-h-0 flex-1 overflow-y-auto rounded-md border border-border">
+ {browseQuery.isLoading ? (
+ <div className="flex items-center gap-2 p-3 text-sm text-muted-foreground">
+ <Loader2 className="h-4 w-4 animate-spin" /> Loading…
+ </div>
+ ) : browse?.success === false ? (
+ <div className="p-3 text-sm text-destructive">{browse.error}</div>
+ ) : (browse?.entries?.length ?? 0) === 0 ? (
+ <div className="p-3 text-sm text-muted-foreground">
+ Empty folder
+ </div>
+ ) : (
+ <ul>
+ {browse?.entries?.map((entry) => (
+ <li key={entry.path ?? entry.location ?? entry.name}>
+ <button
+ type="button"
+ className="flex w-full items-center gap-2 px-3 py-1.5 text-left text-sm hover:bg-accent"
+ onClick={() => navigateTo(entry)}
+ disabled={!entry.is_dir}
+ >
+ {entry.is_dir ? (
+ <Folder className="h-4 w-4 shrink-0 text-muted-foreground" />
+ ) : (
+ <File className="h-4 w-4 shrink-0 text-muted-foreground" />
+ )}
+ <span className="min-w-0 flex-1 truncate">
+ {entry.name}
+ </span>
+ {!entry.is_dir && (
+ <span className="text-xs text-muted-foreground">
+ {formatSize(entry.size_bytes)}
+ </span>
+ )}
+ </button>
+ </li>
+ ))}
+ </ul>
+ )}
+ </div>
+ {browse?.truncated && (
+ <div className="text-xs text-muted-foreground">
+ Listing truncated at {BROWSE_LIMIT} entries
+ </div>
+ )}
+ </section>
+
+ <AlertDialog
+ open={removeTarget !== null}
+ onOpenChange={(open) => !open && setRemoveTarget(null)}
+ >
+ <AlertDialogContent>
+ <AlertDialogHeader>
+ <AlertDialogTitle>Remove mount?</AlertDialogTitle>
+ <AlertDialogDescription>
+ Every Data node on this machine loses access to
+ {' '}
+ <span className="font-mono">{removeTarget}</span>. Files on disk
+ are not touched.
+ </AlertDialogDescription>
+ </AlertDialogHeader>
+ <AlertDialogFooter>
+ <AlertDialogCancel>Cancel</AlertDialogCancel>
+ <AlertDialogAction
+ onClick={() => {
+ if (removeTarget) removeMount.mutate(removeTarget);
+ setRemoveTarget(null);
+ }}
+ >
+ Remove
+ </AlertDialogAction>
+ </AlertDialogFooter>
+ </AlertDialogContent>
+ </AlertDialog>
+ </div>
+ );
+};
+
+export default DataPanel;
diff --git a/client/src/components/parameterPanel/MiddleSection.tsx b/client/src/components/parameterPanel/MiddleSection.tsx
index 7fd9f75f..9740f53f 100644
--- a/client/src/components/parameterPanel/MiddleSection.tsx
+++ b/client/src/components/parameterPanel/MiddleSection.tsx
@@ -35,6 +35,7 @@ import ProcessManagerPanel from './ProcessManagerPanel';
import GalleryPanel from './GalleryPanel';
import ContextPanel from './ContextPanel';
import MemoryToolPanel from './MemoryToolPanel';
+import DataPanel from './DataPanel';
import { useAppStore } from '../../store/useAppStore';
import { useNodeStatus, useWebSocket, CompactionStats } from '../../contexts/WebSocketContext';
import { useUserSettingsQuery } from '../../hooks/useUserSettingsQuery';
@@ -192,6 +193,7 @@ const MiddleSection: React.FC<MiddleSectionProps> = ({
const isMasterSkillNode = hints.isMasterSkillEditor === true;
const isContextNode = hints.isContextPanel === true;
const isMemoryToolNode = hints.isMemoryToolPanel === true;
+ const isDataNode = hints.isDataPanel === true;
// Legacy combined Markdown/transcript panel remains available while
// normalize_workflow_graph upgrades input-memory graphs.
const isMemoryNode = hints.isMemoryPanel === true && !isMemoryToolNode;
@@ -607,6 +609,13 @@ const MiddleSection: React.FC<MiddleSectionProps> = ({
parameters={parameters}
onParameterChange={onParameterChange}
/>
+ ) : isDataNode ? (
+ <DataPanel
+ nodeId={nodeId}
+ workflowId={currentWorkflow?.id}
+ parameters={parameters}
+ onParameterChange={onParameterChange}
+ />
) : (
<>
{/* Parameters */}
diff --git a/client/src/types/INodeProperties.ts b/client/src/types/INodeProperties.ts
index 2fec235a..b40cc7d8 100644
--- a/client/src/types/INodeProperties.ts
+++ b/client/src/types/INodeProperties.ts
@@ -216,6 +216,8 @@ export interface INodeUIHints {
isContextPanel?: boolean;
/** MiddleSection: render explicit durable Memory item controls. */
isMemoryToolPanel?: boolean;
+ /** MiddleSection: render the Data node's mounts + read-only browser. */
+ isDataPanel?: boolean;
/** ParameterPanel: skip the Input section (e.g. start, skill, monitor). */
hideInputSection?: boolean;
/** ParameterPanel: skip the Output section (e.g. start, skill). */
diff --git a/docs-internal/data_node.md b/docs-internal/data_node.md
new file mode 100644
index 00000000..52473425
--- /dev/null
+++ b/docs-internal/data_node.md
@@ -0,0 +1,114 @@
+# Data Node, Vision Delegate, and Native Image Blocks
+
+Three related capabilities shipped together (worktree `data-node-multimodal`):
+the `dataSource` agent tool (raw local data), the `visionAnalyze` delegate
+tool (vision for every host model), and the first increment of native image
+blocks in the LLM provider layer. Research grounding: QwenLM/Qwen-MM-Plugins
+(budget abstraction, disk-wide/context-narrow) and the 2026 provider APIs
+(images in tool results on Anthropic/OpenAI-Responses/Gemini-3).
+
+## `dataSource` (`server/nodes/tool/data_source/`)
+
+A `ToolNode` (`tool_name="data"`, locked split schema like `simpleMemory`)
+over **two path namespaces**:
+
+- workspace-relative paths (`reports/q3.csv`) — always read+write;
+- external mounts `mnt/<mount_name>/<rel>` — the first segment `mnt` is
+ reserved; workspace ops refuse it.
+
+Operations: `list`, `read`, `search`, `metadata`, `write`, `append`,
+`copy_to_workspace`. **No delete** — deletion stays human-only in the
+gallery. `list` at the root also returns the node's enabled mounts so the
+model discovers them at runtime.
+
+**Mounts** are machine-wide rows in the `data_mounts` table
+(`services/data/mount_store.py`, owns its table like the memory tool store)
+with a per-mount `writable` flag (default read-only). Save-time validation
+refuses: relative/missing/non-dir paths, filesystem roots, the home dir
+itself, anything overlapping `DATA_DIR` in either direction (credentials.db
+/ workflow.db / workspaces), duplicate names/roots. Each node exposes a
+subset via `Params.mounts` (names); a mount must be in the node subset AND
+still in the global table at call time, so a global revoke wins instantly.
+
+**Containment** for both roots goes through the existing
+`resolve_within` / `resolve_entry_within` / `normalize_virtual_path`
+(`nodes/filesystem/_backend.py`); `ValueError` is translated to
+`NodeUserError` in `_paths.py`. External files never mint a `FileRef` —
+they get a `MountEntry` dict whose only address is the `mnt/...` virtual
+path (host paths never reach outputs, the DB, or LLM context; locked by a
+serialized-output scan in tests). `copy_to_workspace` (never overwrites;
+suffixes `-1`, `-2`, …) imports a mount file into the workspace and mints a
+real `FileRef` — the bridge to previews, drags, and vision.
+
+**Read tiers** (`_readers.py`, all bounded; `bound_result` caps the
+serialized envelope at ~200 KB because tool results have no downstream
+truncation): text (line window, encoding fallback + `encoding_guessed`),
+csv (Sniffer, 500 rows / 100 cols / 2k-char cells), json (≤5 MiB, depth 8,
+`pruned_paths`), pdf (pypdf via the `docs` extra, 20 pages/req), html
+(bs4), xlsx (openpyxl `read_only`, 500 rows), image (Pillow **metadata
+only** — and the `llm_media` vision opt-in for workspace refs), binary
+(ref + sha256 — never bytes/base64; deliberately fixes `fileRead`'s
+unbounded base64 fallback). 25 MiB absolute gate (`MEDIA_MAX_READ_BYTES`).
+
+**Panel** (`client/src/components/parameterPanel/DataPanel.tsx`, dispatched
+by the `isDataPanel` uiHint): mount CRUD (machine-wide, labeled as such),
+per-node enable checkboxes bound to `parameters.mounts`, and a read-only
+browser. Pure UI — rows/crumbs/writability come finished from the plugin's
+WS handlers (`data_list_mounts/add/update/remove`, `data_browse`), all
+`@ws_response` + external-socket + owner + saved-workflow checks (the
+memory `_handlers.py` security template).
+
+## `visionAnalyze` (`server/nodes/vision/vision_analyze/`)
+
+The vision-delegate tool (`tool_name="vision"`): gives every agent —
+including text-only host models — image understanding. `describe(image,
+question?, budget)` and `extract_text(image)` load workspace image bytes at
+the provider boundary (`read_media_bytes`, contained), fit them to a
+visual-token budget, send them to a vision model (openai / anthropic /
+gemini — official SDK request shapes, `detail` always explicit), and return
+text. Provider + `vision_model` are operator Params (hidden from the LLM by
+the split schema; the field is deliberately not named `model` — reserved
+sibling-name magic in ParameterRenderer).
+
+`services/media/image_fit.py` owns the budget abstraction (after
+Qwen-MM-Plugins): `small|normal|large` in visual tokens →
+`pixels = tokens × patch²`, `smart_resize` (aspect-preserving, patch-grid
+snap, floor AND ceiling), JPEG q90 / PNG-for-alpha.
+
+## Native image blocks (`services/llm/media.py` + protocol)
+
+Refs in durable state, bytes only at the provider boundary:
+
+- `ContentBlock.source` (protocol.py): durable
+ `{kind:"file_ref", ref:<FileRef>, detail}` (~450 B) or transient
+ `{kind:"bytes", media_type, data_b64}`. `_durable_source` in the wire
+ codec **raises** on a bytes-kind source — the never-bytes rule enforced
+ structurally, so a leak is a loud error, not a 2 MiB Temporal payload.
+- Tools opt in by returning `llm_media: [{ref, detail}]` (max 8;
+ `ref.workflow_id` required; png/jpeg/webp/gif). Producers today: the
+ data node's image tier. Both agent loops attach the blocks
+ (`agent_runtime.py` and `temporal/agent_activities.py`).
+- `hydrate_image_blocks(messages, provider, model)` runs once per LLM step
+ in `run_native_llm_step`, on deep copies — originals never mutate;
+ per-image failures degrade to text placeholders; incapable models get
+ `[Image attached: … cannot view images; metadata only]`.
+- **Capability gate** `provider_supports_vision` reads
+ `llm_defaults.json providers.<p>.vision.enabled`. Unknown ⇒ False ⇒
+ text fallback — never emit a block a model isn't confirmed to accept
+ (a rejected block would be journaled and resent every turn).
+- **Encoders**: Anthropic renders hydrated images inside
+ `tool_result.content` (the documented shape). OpenAI (Responses
+ `input_image` in `function_call_output`; Chat Completions hoist-to-user)
+ and Gemini (`inline_data` parts beside `function_response`) are the next
+ increments — their `vision.enabled` stays `false` until then, so bytes
+ are never hydrated just to be dropped. Text-only providers keep
+ `visionAnalyze` as their vision path permanently.
+
+## Tests
+
+`tests/nodes/test_data_source_node.py` (spec/validation/path-security/
+tiers/mount flows/no-host-path-leak), `tests/services/data/test_mount_store.py`
+(validation matrix), `tests/nodes/test_vision_analyze.py` (budget math +
+per-provider request shapes), `tests/llm/test_media_blocks.py` (codec
+round-trip + never-bytes raise, `llm_media` contract, gated hydration,
+Anthropic tool_result shape).
diff --git a/server/config/llm_defaults.json b/server/config/llm_defaults.json
index b042a2f0..51f9daff 100644
--- a/server/config/llm_defaults.json
+++ b/server/config/llm_defaults.json
@@ -22,6 +22,10 @@
"models_endpoint": "https://api.openai.com/v1/models",
"api_key_param": "openai_api_key",
"max_tokens_param": "max_tokens",
+ "vision": {
+ "enabled": false,
+ "_note": "Native image input is emitted only when enabled is true; unknown/absent providers get a text placeholder instead (a rejected image block would poison journaled history). Read by services/llm/media.py provider_supports_vision. False here until the OpenAI request encoder renders image blocks (Responses input_image in function_call_output; Chat Completions hoist-to-user-turn) — flipping it early would hydrate bytes the encoder silently drops. Anthropic ships the first encoder; text-only providers keep the visionAnalyze delegate tool."
+ },
"_models_note": "GPT-5.6 family (sol/terra/luna) GA 2026-07-09. gpt-5.5 / gpt-5.4 remain GA as legacy tiers. Dropped: gpt-5.2 (new requests blocked 2026-06-30), o3 (API shutdown 2026-12-11), o4-mini (API shutdown 2026-10-23) — they fall back to _default constraints if still selected before shutdown.",
"max_output_tokens": {
"gpt-5.6-sol": 128000,
@@ -62,6 +66,9 @@
"models_endpoint": "https://api.anthropic.com/v1/models",
"api_key_param": "anthropic_api_key",
"max_tokens_param": "max_tokens",
+ "vision": {
+ "enabled": true
+ },
"max_output_tokens": {
"claude-opus-5": 128000,
"claude-fable-5": 128000,
@@ -112,6 +119,10 @@
"models_endpoint": "https://generativelanguage.googleapis.com/v1beta/models",
"api_key_param": "google_api_key",
"max_tokens_param": "max_output_tokens",
+ "vision": {
+ "enabled": false,
+ "_note": "See openai.vision._note — off until the Gemini encoder renders inline_data parts beside function_response."
+ },
"_models_note": "Real (concrete) model names only — no -latest rolling aliases. The curated list (max_output_tokens keys) serves BOTH the Gemini Developer API (AIza keys) and Vertex / Agent Platform (AQ. keys); gemini-* entries live-verified on both endpoints (2026-06-12). Vertex 404s on -latest aliases and its models.list rejects API keys, which is why the dropdown is curated instead of live-fetched. Gemma exception: gemma-4-31b-it + gemma-4-26b-a4b-it are served ONLY on the Developer API (free tier, ai.google.dev/gemma/docs/core/gemma_on_gemini_api, verified 2026-07-11) — Vertex keys must not see them, hence vertex_incompatible_models below. Gemma 4 supports systemInstruction and function calling natively (unlike the gemma-3 era); max output is undocumented, so the curated 8192 is a conservative fallback (registry cache wins when present).",
"max_output_tokens": {
"gemini-3.6-flash": 65536,
diff --git a/server/config/node_allowlist.json b/server/config/node_allowlist.json
index 47e97b56..aa40ca8a 100644
--- a/server/config/node_allowlist.json
+++ b/server/config/node_allowlist.json
@@ -14,6 +14,8 @@
"vertexCloudTool",
"simpleMemory",
+ "dataSource",
+ "visionAnalyze",
"masterSkill",
"openaiChatModel",
diff --git a/server/nodes/tool/data_source/__init__.py b/server/nodes/tool/data_source/__init__.py
new file mode 100644
index 00000000..b365ec47
--- /dev/null
+++ b/server/nodes/tool/data_source/__init__.py
@@ -0,0 +1,715 @@
+"""Data node — an agent-facing raw-data reader/writer.
+
+One locked multi-operation tool (``data``) over two path namespaces: the
+per-workflow workspace and operator-approved external mounts
+(``mnt/<name>/...``). Reads dispatch to typed, bounded tiers
+(text/csv/json/pdf/html/xlsx/image-metadata/binary); binary content travels
+as references, never bytes. Writes are allowed into the workspace and into
+mounts whose operator flipped the writable flag. There is deliberately no
+delete operation — deletion stays a human action in the gallery panel.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import mimetypes
+import os
+import shutil
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Literal, Optional
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+from services.plugin import (
+ NodeContext,
+ NodeUserError,
+ Operation,
+ TaskQueue,
+ ToolNode,
+)
+
+from ._paths import (
+ MOUNT_PREFIX,
+ ResolvedTarget,
+ mount_entry,
+ owner_id,
+ resolve_data_path,
+ split_mount_path,
+)
+from ._readers import (
+ bound_result,
+ detect_tier,
+ file_sha256,
+ read_csv,
+ read_html,
+ read_image_meta,
+ read_json,
+ read_pdf,
+ read_text,
+ read_xlsx,
+ walk_mount,
+)
+
+DataOperation = Literal[
+ "list",
+ "read",
+ "search",
+ "metadata",
+ "write",
+ "append",
+ "copy_to_workspace",
+]
+
+ReadType = Literal[
+ "auto", "text", "csv", "json", "pdf", "html", "xlsx", "image", "binary"
+]
+
+_LIST_LIMIT_MAX = 500
+_COPY_SUFFIX_ATTEMPTS = 100
+
+
+class DataToolParams(BaseModel):
+ """Persisted operator configuration; never exposed as model arguments."""
+
+ mounts: list[str] = Field(
+ default_factory=list,
+ title="Enabled Mounts",
+ description=(
+ "Names of machine-wide mounts this node exposes to its agent. "
+ "Mounts are defined in the Data panel."
+ ),
+ )
+
+ model_config = ConfigDict(extra="ignore")
+
+ @model_validator(mode="before")
+ @classmethod
+ def _coerce_mounts(cls, data: Any) -> Any:
+ # The parameter panel stores "" for cleared fields regardless of
+ # declared type, and list values may arrive as JSON strings.
+ if isinstance(data, dict):
+ raw = data.get("mounts")
+ if raw == "" or raw is None:
+ data = {**data, "mounts": []}
+ elif isinstance(raw, str):
+ import json
+
+ try:
+ parsed = json.loads(raw)
+ except json.JSONDecodeError:
+ parsed = [raw]
+ data = {
+ **data,
+ "mounts": parsed if isinstance(parsed, list) else [str(parsed)],
+ }
+ return data
+
+
+class DataToolInput(BaseModel):
+ """One locked, multi-operation schema visible to the LLM."""
+
+ operation: DataOperation = Field(
+ description=(
+ "Data operation: list, read, search, metadata, write, append, "
+ "or copy_to_workspace."
+ )
+ )
+ path: str = Field(
+ default="",
+ max_length=4_096,
+ description=(
+ "Workspace-relative path (e.g. reports/q3.csv), or an external "
+ "mount path mnt/<mount_name>/<file>. Empty lists the workspace "
+ "root plus the enabled mounts."
+ ),
+ )
+ pattern: Optional[str] = Field(
+ default=None,
+ max_length=256,
+ description="search: filename glob or bare term (term matches *term*)",
+ )
+ as_type: ReadType = Field(
+ default="auto",
+ description="read: force a tier instead of extension detection",
+ )
+ offset: int = Field(default=0, ge=0, description="read: rows/lines/pages to skip")
+ limit: int = Field(default=100, ge=1, le=_LIST_LIMIT_MAX)
+ sheet: Optional[str] = Field(default=None, max_length=128)
+ encoding: Optional[str] = Field(default=None, max_length=32)
+ content: Optional[str] = Field(
+ default=None,
+ max_length=200_000,
+ description="write/append: UTF-8 text content",
+ )
+ dest: Optional[str] = Field(
+ default=None,
+ max_length=4_096,
+ description=(
+ "copy_to_workspace: workspace-relative destination "
+ "(default imports/<filename>)"
+ ),
+ )
+
+ model_config = ConfigDict(extra="forbid")
+
+ @model_validator(mode="after")
+ def _validate_operation_fields(self) -> "DataToolInput":
+ required: dict[str, tuple[str, ...]] = {
+ "read": ("path",),
+ "metadata": ("path",),
+ "write": ("path", "content"),
+ "append": ("path", "content"),
+ "search": ("pattern",),
+ "copy_to_workspace": ("path",),
+ }
+ missing = [
+ field_name
+ for field_name in required.get(self.operation, ())
+ if not getattr(self, field_name)
+ ]
+ if missing:
+ raise ValueError(f"{self.operation} requires {', '.join(missing)}")
+ return self
+
+
+class DataToolOutput(BaseModel):
+ operation: str
+ path: Optional[str] = None
+ source: Optional[str] = None
+ mount: Optional[str] = None
+ type: Optional[str] = None
+ truncated: Optional[bool] = None
+
+ model_config = ConfigDict(extra="allow")
+
+
+def _utc_iso(timestamp: Optional[float]) -> Optional[str]:
+ if timestamp is None:
+ return None
+ return datetime.fromtimestamp(timestamp, tz=timezone.utc).isoformat()
+
+
+class DataSourceNode(ToolNode):
+ type = "dataSource"
+ display_name = "Data"
+ subtitle = "Files & Raw Data"
+ group = ("tool",)
+ description = (
+ "Read, search, and write raw local data — workspace files and "
+ "operator-mounted folders — with typed bounded readers for text, "
+ "CSV, JSON, PDF, HTML, XLSX, and image metadata"
+ )
+ component_kind = "tool"
+ tool_name = "data"
+ tool_description = (
+ "Read and write raw local data. Paths are workspace-relative "
+ "(reports/q3.csv) or external-mount paths (mnt/<mount_name>/<file>). "
+ "Start with list (empty path) to discover files and enabled mounts. "
+ "read auto-detects text/csv/json/pdf/html/xlsx/image and pages with "
+ "offset/limit; metadata stats a file without reading it; write/append "
+ "store UTF-8 text (mounts only when writable); copy_to_workspace "
+ "imports a mount file into the workspace. There is no delete."
+ )
+ handles = (
+ {
+ "name": "output-tool",
+ "kind": "output",
+ "position": "top",
+ "label": "Data",
+ "role": "tools",
+ },
+ )
+ ui_hints = {
+ "isToolPanel": True,
+ "isDataPanel": True,
+ "hideInputSection": True,
+ "hideOutputSection": True,
+ "hideRunButton": True,
+ }
+ annotations = {
+ "destructive": False,
+ "readonly": False,
+ "open_world": False,
+ }
+ task_queue = TaskQueue.DEFAULT
+
+ Params = DataToolParams
+ ToolInput = DataToolInput
+ Output = DataToolOutput
+ tool_schema_locked = True
+ server_controlled_fields = frozenset({"mounts"})
+
+ @staticmethod
+ def _enabled_mounts(ctx: NodeContext, params: Any) -> list[str]:
+ config = ctx.raw.get("_tool_config")
+ if isinstance(config, DataToolParams):
+ return list(config.mounts)
+ if isinstance(params, DataToolParams):
+ return list(params.mounts)
+ return []
+
+ @Operation("data")
+ async def data(
+ self,
+ ctx: NodeContext,
+ params: DataToolInput | DataToolParams,
+ ) -> DataToolOutput:
+ # The node's Run button is hidden. Treat a framework-side execution
+ # as a harmless root listing for diagnostics.
+ args = (
+ DataToolInput(operation="list")
+ if isinstance(params, DataToolParams)
+ else params
+ )
+ mounts = self._enabled_mounts(ctx, params)
+ handlers = {
+ "list": self._op_list,
+ "read": self._op_read,
+ "search": self._op_search,
+ "metadata": self._op_metadata,
+ "write": self._op_write,
+ "append": self._op_append,
+ "copy_to_workspace": self._op_copy,
+ }
+ result = await handlers[args.operation](ctx, mounts, args)
+ result.setdefault("operation", args.operation)
+ return DataToolOutput.model_validate(bound_result(result))
+
+ # ---------------------------------------------------------------- ops
+
+ async def _op_list(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ if split_mount_path(args.path) is not None:
+ return await self._list_mount_dir(ctx, mounts, args)
+
+ from nodes.filesystem.gallery._service import list_directory
+ from services.media.workspace import workspace_root
+
+ root = workspace_root(ctx)
+ listing = await list_directory(
+ str(root),
+ path=args.path,
+ workflow_id=ctx.workflow_id,
+ limit=args.limit,
+ )
+ result: dict[str, Any] = {
+ "source": "workspace",
+ "path": listing["path"],
+ "entries": listing["entries"],
+ "count": listing["count"],
+ "truncated": listing["truncated"],
+ }
+ if not str(args.path or "").strip("/"):
+ result["mounts"] = await self._mount_summaries(ctx, mounts)
+ return result
+
+ async def _mount_summaries(
+ self, ctx: NodeContext, mounts: list[str]
+ ) -> list[dict[str, Any]]:
+ """The enabled mounts as the model discovers them at runtime."""
+ if not mounts:
+ return []
+ from services.data.mount_store import DataMountStore
+ from services.plugin.deps import get_database
+
+ rows = await DataMountStore(get_database()).list_mounts(owner_id(ctx))
+ by_name = {row["name"]: row for row in rows}
+ summaries = []
+ for name in mounts:
+ row = by_name.get(name)
+ summaries.append(
+ {
+ "name": name,
+ "path": f"{MOUNT_PREFIX}/{name}",
+ "writable": bool(row["writable"]) if row else False,
+ "available": row is not None,
+ }
+ )
+ return summaries
+
+ async def _list_mount_dir(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ target = await resolve_data_path(ctx, mounts, args.path)
+ if not target.abs_path.is_dir():
+ raise NodeUserError(
+ f"{target.virtual} is not a directory; use read for files"
+ )
+
+ def scan() -> list[dict[str, Any]]:
+ entries = []
+ with os.scandir(target.abs_path) as iterator:
+ for entry in iterator:
+ rest = (
+ f"{target.virtual.split('/', 2)[2]}/{entry.name}"
+ if target.virtual.count("/") >= 2
+ else entry.name
+ )
+ entries.append(
+ mount_entry(
+ mount_name=target.mount_name or "",
+ rel_path=rest,
+ abs_path=Path(entry.path),
+ writable=target.writable,
+ )
+ )
+ entries.sort(key=lambda row: (not row["is_dir"], row["name"].lower()))
+ return entries
+
+ rows = await asyncio.to_thread(scan)
+ return {
+ "source": "mount",
+ "mount": target.mount_name,
+ "path": target.virtual,
+ "entries": rows[: args.limit],
+ "count": min(len(rows), args.limit),
+ "truncated": len(rows) > args.limit,
+ }
+
+ async def _op_read(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ target = await self._resolve_file(ctx, mounts, args.path)
+ tier = detect_tier(target.abs_path, args.as_type)
+ common = self._common(target)
+
+ if tier == "image":
+ meta = await asyncio.to_thread(read_image_meta, target.abs_path)
+ result = {**common, **meta, **self._reference(ctx, target)}
+ # Vision opt-in (services/llm/media.py contract): workspace refs
+ # to allowlisted image types become viewable by vision-capable
+ # host models; mounts go through copy_to_workspace first.
+ ref = result.get("ref")
+ if isinstance(ref, dict):
+ from services.llm.media import IMAGE_MIME_ALLOWLIST
+
+ if ref.get("mime_type") in IMAGE_MIME_ALLOWLIST:
+ result["llm_media"] = [{"ref": ref, "detail": "auto"}]
+ return result
+ if tier == "binary":
+ sha = await asyncio.to_thread(file_sha256, target.abs_path)
+ stat = target.abs_path.stat()
+ return {
+ **common,
+ "type": "binary",
+ "mime_type": mimetypes.guess_type(target.abs_path.name)[0]
+ or "application/octet-stream",
+ "size_bytes": int(stat.st_size),
+ "sha256": sha,
+ **self._reference(ctx, target),
+ }
+
+ readers = {
+ "text": lambda: read_text(
+ target.abs_path,
+ offset=args.offset,
+ limit=args.limit,
+ encoding=args.encoding,
+ ),
+ "csv": lambda: read_csv(
+ target.abs_path,
+ offset=args.offset,
+ limit=args.limit,
+ encoding=args.encoding,
+ ),
+ "json": lambda: read_json(
+ target.abs_path, encoding=args.encoding
+ ),
+ "pdf": lambda: read_pdf(
+ target.abs_path, offset=args.offset, limit=args.limit
+ ),
+ "html": lambda: read_html(
+ target.abs_path,
+ offset=args.offset,
+ limit=args.limit,
+ encoding=args.encoding,
+ ),
+ "xlsx": lambda: read_xlsx(
+ target.abs_path,
+ sheet=args.sheet,
+ offset=args.offset,
+ limit=args.limit,
+ ),
+ }
+ payload = await asyncio.to_thread(readers[tier])
+ return {**common, **payload}
+
+ async def _op_search(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ from nodes.filesystem.gallery._service import (
+ list_matching,
+ search_to_pattern,
+ )
+
+ pattern = search_to_pattern(args.pattern or "")
+ if not pattern:
+ raise NodeUserError("search requires a non-empty pattern")
+
+ if split_mount_path(args.path) is not None:
+ target = await resolve_data_path(ctx, mounts, args.path)
+ start_rel = (
+ target.virtual.split("/", 2)[2]
+ if target.virtual.count("/") >= 2
+ else ""
+ )
+ rel_paths, truncated = await asyncio.to_thread(
+ walk_mount,
+ target.root,
+ start_rel=start_rel,
+ pattern=pattern,
+ )
+ rows = [
+ mount_entry(
+ mount_name=target.mount_name or "",
+ rel_path=rel,
+ abs_path=target.root / rel,
+ writable=target.writable,
+ )
+ for rel in rel_paths[: args.limit]
+ ]
+ return {
+ "source": "mount",
+ "mount": target.mount_name,
+ "path": target.virtual,
+ "pattern": pattern,
+ "entries": rows,
+ "count": len(rows),
+ "truncated": truncated or len(rel_paths) > args.limit,
+ }
+
+ from services.media.workspace import workspace_root
+
+ root = workspace_root(ctx)
+ listing = await list_matching(
+ str(root),
+ pattern=pattern,
+ path=args.path,
+ workflow_id=ctx.workflow_id,
+ limit=args.limit,
+ )
+ return {
+ "source": "workspace",
+ "path": listing["path"],
+ "pattern": listing["pattern"],
+ "entries": listing["entries"],
+ "count": listing["count"],
+ "truncated": listing["truncated"],
+ }
+
+ async def _op_metadata(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ target = await self._resolve_file(ctx, mounts, args.path)
+ stat = target.abs_path.stat()
+ tier = detect_tier(target.abs_path)
+ result: dict[str, Any] = {
+ **self._common(target),
+ "type": tier,
+ "name": target.abs_path.name,
+ "mime_type": mimetypes.guess_type(target.abs_path.name)[0]
+ or "application/octet-stream",
+ "size_bytes": int(stat.st_size),
+ "modified_at": _utc_iso(stat.st_mtime),
+ "sha256": await asyncio.to_thread(file_sha256, target.abs_path),
+ **self._reference(ctx, target),
+ }
+ if tier == "image":
+ try:
+ meta = await asyncio.to_thread(read_image_meta, target.abs_path)
+ result["image"] = meta["image"]
+ except NodeUserError:
+ pass
+ return result
+
+ async def _op_write(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ return await self._write(ctx, mounts, args, append=False)
+
+ async def _op_append(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ return await self._write(ctx, mounts, args, append=True)
+
+ async def _write(
+ self,
+ ctx: NodeContext,
+ mounts: list[str],
+ args: DataToolInput,
+ *,
+ append: bool,
+ ) -> dict[str, Any]:
+ from nodes.filesystem._backend import atomic_write_bytes, get_path_lock
+
+ target = await resolve_data_path(
+ ctx, mounts, args.path, for_write=True
+ )
+ payload = (args.content or "").encode("utf-8")
+ existed = target.abs_path.exists()
+ if target.abs_path.is_dir():
+ raise NodeUserError(f"{target.virtual} is a directory")
+
+ def write_sync() -> None:
+ target.abs_path.parent.mkdir(parents=True, exist_ok=True)
+ if append:
+ with open(target.abs_path, "ab") as handle:
+ handle.write(payload)
+ else:
+ atomic_write_bytes(
+ target.abs_path, payload, root_dir=target.root
+ )
+
+ async with get_path_lock(target.abs_path):
+ await asyncio.to_thread(write_sync)
+ return {
+ **self._common(target),
+ "bytes_written": len(payload),
+ "created": not existed,
+ "appended": append,
+ }
+
+ async def _op_copy(
+ self, ctx: NodeContext, mounts: list[str], args: DataToolInput
+ ) -> dict[str, Any]:
+ from nodes.filesystem._backend import (
+ get_path_lock,
+ resolve_entry_within,
+ )
+ from services.media.limits import MEDIA_MAX_READ_BYTES
+ from services.media.workspace import workspace_root
+
+ if split_mount_path(args.path) is None:
+ raise NodeUserError(
+ "copy_to_workspace imports FROM a mount: path must be "
+ "mnt/<mount_name>/<file>"
+ )
+ source = await self._resolve_file(ctx, mounts, args.path)
+ dest_rel = (
+ str(args.dest or "").strip().replace("\\", "/").strip("/")
+ or f"imports/{source.abs_path.name}"
+ )
+ if dest_rel.split("/", 1)[0] == MOUNT_PREFIX:
+ raise NodeUserError(
+ "copy_to_workspace destination is workspace-relative; it "
+ "cannot target a mount"
+ )
+ root = workspace_root(ctx).resolve()
+ dest = resolve_entry_within(root, dest_rel)
+ # Never overwrite: an import must not be able to destroy data.
+ if dest.exists():
+ stem, suffix = dest.stem, dest.suffix
+ for attempt in range(1, _COPY_SUFFIX_ATTEMPTS + 1):
+ candidate = dest.with_name(f"{stem}-{attempt}{suffix}")
+ if not candidate.exists():
+ dest = candidate
+ break
+ else:
+ raise NodeUserError(
+ f"Too many existing copies of {dest_rel} in the workspace"
+ )
+ size = source.abs_path.stat().st_size
+ if size > MEDIA_MAX_READ_BYTES:
+ raise NodeUserError(
+ f"File is {size:,} bytes; copy_to_workspace caps at "
+ f"{MEDIA_MAX_READ_BYTES:,}"
+ )
+
+ def copy_sync() -> None:
+ dest.parent.mkdir(parents=True, exist_ok=True)
+ shutil.copyfile(source.abs_path, dest)
+
+ async with get_path_lock(dest):
+ await asyncio.to_thread(copy_sync)
+ final_rel = dest.relative_to(root).as_posix()
+ return {
+ "source": "workspace",
+ "path": final_rel,
+ "copied_from": source.virtual,
+ "ref": self._workspace_ref(ctx, root, dest),
+ }
+
+ # ------------------------------------------------------------ helpers
+
+ @staticmethod
+ async def _resolve_file(
+ ctx: NodeContext, mounts: list[str], path: str
+ ) -> ResolvedTarget:
+ from services.media.limits import MEDIA_MAX_READ_BYTES
+
+ target = await resolve_data_path(ctx, mounts, path)
+ if target.abs_path.is_dir():
+ raise NodeUserError(
+ f"{target.virtual} is a directory; use list for directories"
+ )
+ if not target.abs_path.is_file():
+ raise NodeUserError(f"File not found: {target.virtual}")
+ size = target.abs_path.stat().st_size
+ if size > MEDIA_MAX_READ_BYTES:
+ raise NodeUserError(
+ f"File is {size:,} bytes; reads cap at "
+ f"{MEDIA_MAX_READ_BYTES:,}"
+ )
+ return target
+
+ @staticmethod
+ def _common(target: ResolvedTarget) -> dict[str, Any]:
+ common: dict[str, Any] = {
+ "source": target.source,
+ "path": target.virtual,
+ }
+ if target.mount_name:
+ common["mount"] = target.mount_name
+ return common
+
+ def _reference(
+ self, ctx: NodeContext, target: ResolvedTarget
+ ) -> dict[str, Any]:
+ """The handle other nodes consume: FileRef in the workspace,
+ location-only for mounts (host paths never leave the server)."""
+ if target.source == "workspace":
+ return {
+ "ref": self._workspace_ref(ctx, target.root, target.abs_path)
+ }
+ return {"location": target.virtual}
+
+ @staticmethod
+ def _workspace_ref(
+ ctx: NodeContext, root: Path, abs_path: Path
+ ) -> dict[str, Any]:
+ from nodes.filesystem.gallery._service import to_file_ref
+ from services.media.workspace import workspace_file_url
+
+ rel = abs_path.relative_to(root).as_posix()
+ stat = abs_path.stat()
+ workflow_id = ctx.workflow_id
+ row = {
+ "path": rel,
+ "name": abs_path.name,
+ "mime_type": mimetypes.guess_type(abs_path.name)[0]
+ or "application/octet-stream",
+ "size_bytes": int(stat.st_size),
+ "modified_at": _utc_iso(stat.st_mtime),
+ "url": workspace_file_url(workflow_id, rel) if workflow_id else None,
+ }
+ return to_file_ref(row, workflow_id)
+
+
+# Plugin-owned side-channel API for the Data panel.
+from services.ws_handler_registry import register_ws_handlers # noqa: E402
+
+from ._handlers import WS_HANDLERS # noqa: E402
+
+register_ws_handlers(WS_HANDLERS)
+
+
+__all__ = [
+ "WS_HANDLERS",
+ "DataOperation",
+ "DataSourceNode",
+ "DataToolInput",
+ "DataToolOutput",
+ "DataToolParams",
+]
diff --git a/server/nodes/tool/data_source/_handlers.py b/server/nodes/tool/data_source/_handlers.py
new file mode 100644
index 00000000..f5cef043
--- /dev/null
+++ b/server/nodes/tool/data_source/_handlers.py
@@ -0,0 +1,261 @@
+"""Human-facing WebSocket API for the Data panel.
+
+Mount CRUD operates on the machine-wide allowlist (owner-scoped); browsing
+resolves its scope from the persisted workflow graph and the authenticated
+WebSocket, mirroring the Simple Memory handlers. Neither the client nor the
+model can supply a mount root or namespace — only names of rows the
+operator created.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+from pathlib import Path
+from typing import Any, Awaitable, Callable, Dict
+
+from fastapi import WebSocket
+
+from services.plugin import NodeUserError
+from services.plugin.deps import get_database
+from services.plugin.ws import ws_response
+
+from ._paths import MOUNT_PREFIX, mount_entry, split_mount_path
+
+_BROWSE_LIMIT_MAX = 200
+_BROWSE_LIMIT_DEFAULT = 50
+
+
+def _authenticated_owner(websocket: WebSocket) -> str:
+ """Read server-authenticated identity without consulting request data."""
+ state = getattr(websocket, "state", None)
+ for attribute in ("user_id", "principal_id", "subject"):
+ value = getattr(state, attribute, None) if state is not None else None
+ if isinstance(value, (str, int)) and str(value).strip():
+ return str(value)
+ scope = getattr(websocket, "scope", None)
+ if isinstance(scope, dict):
+ for key in ("user_id", "principal_id", "subject"):
+ value = scope.get(key)
+ if isinstance(value, (str, int)) and str(value).strip():
+ return str(value)
+ return "owner"
+
+
+def _require_external_socket(websocket: WebSocket) -> None:
+ """The internal unauthenticated worker socket may not touch mounts."""
+ scope = getattr(websocket, "scope", {}) or {}
+ if scope.get("path") == "/ws/internal":
+ raise NodeUserError("Data access requires an authenticated client")
+
+
+def _store():
+ from services.data.mount_store import DataMountStore
+
+ return DataMountStore(get_database())
+
+
+async def _resolve_node_scope(
+ data: Dict[str, Any], websocket: WebSocket
+) -> tuple[str, str, str]:
+ """(owner_id, workflow_id, node_id) for a browse request, or raise."""
+ _require_external_socket(websocket)
+ workflow_id = str(data.get("workflow_id") or "").strip()
+ node_id = str(data.get("data_node_id") or data.get("node_id") or "").strip()
+ if not workflow_id:
+ raise NodeUserError("workflow_id required")
+ if not node_id:
+ raise NodeUserError("data_node_id required")
+ database = get_database()
+ saved = await database.get_workflow(workflow_id)
+ if saved is None:
+ raise NodeUserError("Workflow not found")
+ graph = saved.data if hasattr(saved, "data") else saved.get("data", saved)
+ owner = _authenticated_owner(websocket)
+ stored_owner = (
+ str(graph.get("owner_id") or "") if isinstance(graph, dict) else ""
+ )
+ if stored_owner and stored_owner != owner:
+ raise NodeUserError("Workflow access denied")
+ nodes = graph.get("nodes", []) if isinstance(graph, dict) else []
+ matches = [
+ node
+ for node in nodes
+ if str(node.get("id") or "") == node_id
+ and str(node.get("type") or node.get("data", {}).get("type") or "")
+ == "dataSource"
+ ]
+ if len(matches) != 1:
+ raise NodeUserError(
+ "Data node does not belong to the requested workflow"
+ )
+ return owner, workflow_id, node_id
+
+
+# ------------------------------------------------------------- mount CRUD
+
+
+@ws_response
+async def handle_data_list_mounts(
+ data: Dict[str, Any], websocket: WebSocket
+) -> Dict[str, Any]:
+ _require_external_socket(websocket)
+ mounts = await _store().list_mounts(_authenticated_owner(websocket))
+ return {"success": True, "mounts": mounts}
+
+
+@ws_response
+async def handle_data_add_mount(
+ data: Dict[str, Any], websocket: WebSocket
+) -> Dict[str, Any]:
+ from services.data.mount_store import MountStoreError
+
+ _require_external_socket(websocket)
+ try:
+ mount = await _store().add_mount(
+ _authenticated_owner(websocket),
+ name=str(data.get("name") or ""),
+ root_path=str(data.get("root_path") or ""),
+ writable=bool(data.get("writable", False)),
+ )
+ except MountStoreError as exc:
+ raise NodeUserError(str(exc)) from exc
+ return {"success": True, "mount": mount}
+
+
+@ws_response
+async def handle_data_update_mount(
+ data: Dict[str, Any], websocket: WebSocket
+) -> Dict[str, Any]:
+ from services.data.mount_store import MountStoreError
+
+ _require_external_socket(websocket)
+ try:
+ mount = await _store().update_mount(
+ _authenticated_owner(websocket),
+ name=str(data.get("name") or ""),
+ writable=bool(data.get("writable", False)),
+ )
+ except MountStoreError as exc:
+ raise NodeUserError(str(exc)) from exc
+ return {"success": True, "mount": mount}
+
+
+@ws_response
+async def handle_data_remove_mount(
+ data: Dict[str, Any], websocket: WebSocket
+) -> Dict[str, Any]:
+ from services.data.mount_store import MountStoreError
+
+ _require_external_socket(websocket)
+ try:
+ result = await _store().remove_mount(
+ _authenticated_owner(websocket),
+ name=str(data.get("name") or ""),
+ )
+ except MountStoreError as exc:
+ raise NodeUserError(str(exc)) from exc
+ return {"success": True, **result}
+
+
+# ----------------------------------------------------------------- browse
+
+
+def _crumbs(virtual: str) -> list[dict[str, str]]:
+ parts = [part for part in virtual.split("/") if part]
+ return [
+ {"name": part, "path": "/".join(parts[: index + 1])}
+ for index, part in enumerate(parts)
+ ]
+
+
+@ws_response
+async def handle_data_browse(
+ data: Dict[str, Any], websocket: WebSocket
+) -> Dict[str, Any]:
+ """Read-only listing over the node's workspace + enabled mounts."""
+ from nodes.tool.data_source import DataToolParams
+
+ owner, workflow_id, node_id = await _resolve_node_scope(data, websocket)
+ database = get_database()
+ persisted = await database.get_node_parameters(node_id) or {}
+ params = DataToolParams.model_validate(persisted)
+ path = str(data.get("path") or "").strip()
+ try:
+ limit = int(data.get("limit") or _BROWSE_LIMIT_DEFAULT)
+ except (TypeError, ValueError):
+ limit = _BROWSE_LIMIT_DEFAULT
+ limit = max(1, min(limit, _BROWSE_LIMIT_MAX))
+
+ split = split_mount_path(path)
+ if split is not None:
+ name, rest = split
+ if name not in params.mounts:
+ raise NodeUserError(f"Mount '{name}' is not enabled on this node")
+ row = await _store().get_mount(owner, name)
+ if row is None:
+ raise NodeUserError(f"Mount '{name}' is no longer defined")
+ from nodes.filesystem._backend import resolve_within
+
+ root = Path(row["root_path"])
+ target = resolve_within(root, rest) if rest else root.resolve()
+ if not target.is_dir():
+ raise NodeUserError("Not a directory")
+
+ def scan() -> list[dict[str, Any]]:
+ rows = []
+ with os.scandir(target) as iterator:
+ for entry in iterator:
+ child_rel = f"{rest}/{entry.name}" if rest else entry.name
+ rows.append(
+ mount_entry(
+ mount_name=name,
+ rel_path=child_rel,
+ abs_path=Path(entry.path),
+ writable=bool(row["writable"]),
+ )
+ )
+ rows.sort(key=lambda item: (not item["is_dir"], item["name"].lower()))
+ return rows
+
+ rows = await asyncio.to_thread(scan)
+ virtual = f"{MOUNT_PREFIX}/{name}/{rest}" if rest else f"{MOUNT_PREFIX}/{name}"
+ return {
+ "success": True,
+ "source": "mount",
+ "mount": name,
+ "path": virtual,
+ "crumbs": _crumbs(virtual),
+ "entries": rows[:limit],
+ "count": min(len(rows), limit),
+ "truncated": len(rows) > limit,
+ }
+
+ from nodes.filesystem.gallery._service import list_directory
+ from services.workspace_locator import resolve_workspace_root
+
+ root = await resolve_workspace_root(workflow_id, database)
+ listing = await list_directory(
+ str(root), path=path, workflow_id=workflow_id, limit=limit
+ )
+ return {"success": True, "source": "workspace", **listing}
+
+
+WSHandler = Callable[[Dict[str, Any], WebSocket], Awaitable[Dict[str, Any]]]
+WS_HANDLERS: Dict[str, WSHandler] = {
+ "data_list_mounts": handle_data_list_mounts,
+ "data_add_mount": handle_data_add_mount,
+ "data_update_mount": handle_data_update_mount,
+ "data_remove_mount": handle_data_remove_mount,
+ "data_browse": handle_data_browse,
+}
+
+
+__all__ = [
+ "WS_HANDLERS",
+ "handle_data_add_mount",
+ "handle_data_browse",
+ "handle_data_list_mounts",
+ "handle_data_remove_mount",
+ "handle_data_update_mount",
+]
diff --git a/server/nodes/tool/data_source/_paths.py b/server/nodes/tool/data_source/_paths.py
new file mode 100644
index 00000000..282db9f7
--- /dev/null
+++ b/server/nodes/tool/data_source/_paths.py
@@ -0,0 +1,233 @@
+"""Path resolution for the Data node's two-source namespace.
+
+One ``path`` argument addresses two kinds of roots:
+
+ reports/q3.csv -> the per-workflow workspace (as every
+ existing filesystem tool spells paths)
+ mnt/<mount_name>/<rel_path> -> an operator-approved external mount
+
+The first segment ``mnt`` is *reserved*: workspace operations refuse it, so
+the two namespaces can never collide and the model gets a hint instead of a
+silent miss. Containment for both roots runs through the same two-layer
+helpers in :mod:`nodes.filesystem._backend` (``resolve_within`` for reads,
+``resolve_entry_within`` for mutations) — they are root-agnostic and
+symlink-safe.
+
+A mount is usable only when it is BOTH in the node's persisted subset
+(``Params.mounts``) AND still present in the machine-wide allowlist at call
+time, so a globally revoked mount dies immediately even if a stale node
+still lists it.
+
+External files never leak host locations: results carry the round-trippable
+virtual path (``mnt/<name>/<rel>``), never ``root_path`` or any absolute
+path.
+"""
+
+from __future__ import annotations
+
+import mimetypes
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Literal, Optional
+
+from services.plugin import NodeContext, NodeUserError
+
+MOUNT_PREFIX = "mnt"
+
+
+@dataclass(frozen=True)
+class ResolvedTarget:
+ """A validated, contained filesystem target."""
+
+ source: Literal["workspace", "mount"]
+ mount_name: Optional[str]
+ abs_path: Path
+ root: Path
+ writable: bool
+ virtual: str # canonical round-trippable path (feed back as `path`)
+
+
+def owner_id(ctx: NodeContext) -> str:
+ """Trusted owner identity — mirrors SimpleMemoryNode._scope."""
+ return str(ctx.user_id or ctx.raw.get("user_id") or "owner")
+
+
+def _clean(path: str) -> str:
+ return str(path or "").strip().replace("\\", "/").strip("/")
+
+
+def split_mount_path(path: str) -> Optional[tuple[str, str]]:
+ """``mnt/<name>/<rest>`` -> (name, rest); None when not mount-shaped."""
+ cleaned = _clean(path)
+ if not cleaned:
+ return None
+ first, _, rest = cleaned.partition("/")
+ if first != MOUNT_PREFIX:
+ return None
+ name, _, remainder = rest.partition("/")
+ return (name, remainder)
+
+
+async def _mount_row(
+ ctx: NodeContext, enabled: list[str], name: str
+) -> dict[str, Any]:
+ from services.data.mount_store import DataMountStore
+ from services.plugin.deps import get_database
+
+ if not name:
+ raise NodeUserError(
+ "A mount path needs a name: mnt/<mount_name>/<file>. "
+ f"Mounts enabled on this node: {', '.join(enabled) or 'none'}"
+ )
+ if name not in enabled:
+ raise NodeUserError(
+ f"Mount '{name}' is not enabled on this Data node. "
+ f"Enabled mounts: {', '.join(enabled) or 'none'}"
+ )
+ row = await DataMountStore(get_database()).get_mount(owner_id(ctx), name)
+ if row is None:
+ raise NodeUserError(
+ f"Mount '{name}' is no longer defined on this machine; ask the "
+ "operator to re-add it in the Data panel"
+ )
+ return row
+
+
+async def resolve_data_path(
+ ctx: NodeContext,
+ mounts: list[str],
+ path: str,
+ *,
+ for_write: bool = False,
+) -> ResolvedTarget:
+ """Resolve ``path`` to a contained absolute target or raise.
+
+ ``for_write`` switches to :func:`resolve_entry_within` (containment
+ proved on the parent, basename appended unresolved — mutation-correct)
+ and enforces the mount's writable flag. The workspace is always
+ writable.
+ """
+ from nodes.filesystem._backend import (
+ resolve_entry_within as _resolve_entry_within,
+ resolve_within as _resolve_within,
+ )
+ from services.media.workspace import workspace_root
+
+ def resolve_within(root: Path, key: str) -> Path:
+ # The containment helpers raise ValueError; translate to the
+ # user-correctable contract so the operator log gets one WARN line
+ # and the model gets an explanation it can act on.
+ try:
+ return _resolve_within(root, key)
+ except ValueError as exc:
+ raise NodeUserError(str(exc)) from exc
+
+ def resolve_entry_within(root: Path, key: str) -> Path:
+ try:
+ return _resolve_entry_within(root, key)
+ except ValueError as exc:
+ raise NodeUserError(str(exc)) from exc
+
+ split = split_mount_path(path)
+ if split is not None:
+ name, rest = split
+ row = await _mount_row(ctx, list(mounts or []), name)
+ root = Path(row["root_path"])
+ writable = bool(row["writable"])
+ if for_write:
+ if not writable:
+ raise NodeUserError(
+ f"Mount '{name}' is read-only. Ask the operator to "
+ "flip its writable flag, or write into the workspace"
+ )
+ if not rest:
+ raise NodeUserError(
+ "Writes need a file path inside the mount, not the "
+ "mount root itself"
+ )
+ abs_path = resolve_entry_within(root, rest)
+ else:
+ abs_path = resolve_within(root, rest) if rest else root.resolve()
+ virtual = f"{MOUNT_PREFIX}/{name}/{rest}" if rest else f"{MOUNT_PREFIX}/{name}"
+ return ResolvedTarget(
+ source="mount",
+ mount_name=name,
+ abs_path=abs_path,
+ root=root.resolve(),
+ writable=writable,
+ virtual=virtual,
+ )
+
+ rel = _clean(path)
+ if rel.split("/", 1)[0] == MOUNT_PREFIX:
+ # Unreachable in practice (split_mount_path matched), kept as a
+ # guard should the split rules ever diverge.
+ raise NodeUserError(
+ "'mnt/' is reserved for external mounts: mnt/<mount_name>/<file>"
+ )
+ root = workspace_root(ctx).resolve()
+ if for_write:
+ if not rel:
+ raise NodeUserError("Writes need a file path, not the workspace root")
+ abs_path = resolve_entry_within(root, rel)
+ else:
+ abs_path = resolve_within(root, rel) if rel else root
+ return ResolvedTarget(
+ source="workspace",
+ mount_name=None,
+ abs_path=abs_path,
+ root=root,
+ writable=True,
+ virtual=rel,
+ )
+
+
+def mount_entry(
+ *,
+ mount_name: str,
+ rel_path: str,
+ abs_path: Path,
+ writable: bool,
+) -> dict[str, Any]:
+ """Listing/metadata row for an external-mount entry.
+
+ Deliberately NOT a FileRef: that type is workspace-contract-bound (its
+ ``path`` is served by the workspace HTTP route). ``location`` is the
+ only address given out, so no host path ever reaches outputs, the DB,
+ or LLM context.
+ """
+ try:
+ stat = abs_path.stat()
+ size = int(stat.st_size)
+ modified = stat.st_mtime
+ except OSError:
+ size = 0
+ modified = None
+ is_dir = abs_path.is_dir()
+ location = (
+ f"{MOUNT_PREFIX}/{mount_name}/{rel_path}"
+ if rel_path
+ else f"{MOUNT_PREFIX}/{mount_name}"
+ )
+ return {
+ "location": location,
+ "mount": mount_name,
+ "name": abs_path.name or mount_name,
+ "is_dir": is_dir,
+ "mime_type": None
+ if is_dir
+ else (mimetypes.guess_type(abs_path.name)[0] or "application/octet-stream"),
+ "size_bytes": 0 if is_dir else size,
+ "modified_at": modified,
+ "writable": writable,
+ }
+
+
+__all__ = [
+ "MOUNT_PREFIX",
+ "ResolvedTarget",
+ "mount_entry",
+ "owner_id",
+ "resolve_data_path",
+ "split_mount_path",
+]
diff --git a/server/nodes/tool/data_source/_readers.py b/server/nodes/tool/data_source/_readers.py
new file mode 100644
index 00000000..b604644a
--- /dev/null
+++ b/server/nodes/tool/data_source/_readers.py
@@ -0,0 +1,467 @@
+"""Bounded, typed read tiers for the Data node.
+
+Every reader here is synchronous (the node runs them via ``asyncio.to_thread``)
+and self-bounding: tool results have NO downstream truncation before they are
+serialized verbatim into an LLM message and the durable journal, so the caps
+in this module are the only thing standing between a 20 MB spreadsheet and a
+blown 2 MiB Temporal payload. ``bound_result`` is the final backstop that
+trims any envelope over ``DATA_RESULT_MAX_BYTES``.
+
+Binary content never leaves as bytes or base64 — the binary/image tiers
+return references/metadata only (this deliberately fixes ``fileRead``'s
+unbounded whole-file base64 fallback rather than inheriting it).
+"""
+
+from __future__ import annotations
+
+import csv
+import fnmatch
+import hashlib
+import io
+import json
+import mimetypes
+import os
+from pathlib import Path
+from typing import Any, Optional
+
+from services.plugin import NodeUserError
+
+# Serialized-envelope ceiling: ~2/5 of the 512 KiB Temporal payload WARN
+# threshold, leaving room for the JSON quoting the agent layer adds.
+DATA_RESULT_MAX_BYTES = 200_000
+TEXT_WINDOW_MAX_BYTES = 65_536
+CSV_MAX_ROWS = 500
+CSV_MAX_COLS = 100
+CSV_MAX_CELL_CHARS = 2_000
+JSON_MAX_SOURCE_BYTES = 5 * 1024 * 1024
+JSON_MAX_DEPTH = 8
+JSON_MAX_PRUNED_PATHS = 50
+PDF_MAX_PAGES = 20
+PDF_MAX_PAGE_CHARS = 20_000
+XLSX_MAX_ROWS = 500
+SEARCH_MAX_DEPTH = 12
+SEARCH_MAX_VISITED = 5_000
+SEARCH_MAX_RESULTS = 200
+
+_TEXT_EXTENSIONS = {
+ ".txt", ".md", ".markdown", ".log", ".rst", ".ini", ".cfg", ".toml",
+ ".yaml", ".yml", ".py", ".js", ".ts", ".tsx", ".jsx", ".css", ".sh",
+ ".ps1", ".bat", ".sql", ".xml", ".env", ".jsonl", ".ndjson", ".srt",
+ ".vtt", ".tex",
+}
+
+
+def detect_tier(path: Path, as_type: str = "auto") -> str:
+ """Pick a read tier. An explicit ``as_type`` always wins."""
+ if as_type and as_type != "auto":
+ return as_type
+ suffix = path.suffix.lower()
+ if suffix in {".csv", ".tsv"}:
+ return "csv"
+ if suffix == ".json":
+ return "json"
+ if suffix == ".pdf":
+ return "pdf"
+ if suffix in {".html", ".htm", ".xhtml"}:
+ return "html"
+ if suffix in {".xlsx", ".xlsm"}:
+ return "xlsx"
+ if suffix in _TEXT_EXTENSIONS:
+ return "text"
+ mime = mimetypes.guess_type(path.name)[0] or ""
+ if mime.startswith("image/"):
+ return "image"
+ if mime.startswith("text/"):
+ return "text"
+ return "binary"
+
+
+def _decode(
+ raw: bytes, encoding: Optional[str]
+) -> tuple[str, str, bool]:
+ """Decode bytes -> (text, encoding_used, guessed)."""
+ if encoding:
+ try:
+ return raw.decode(encoding), encoding, False
+ except (LookupError, UnicodeDecodeError) as exc:
+ raise NodeUserError(
+ f"File does not decode as {encoding!r}: {exc}"
+ ) from exc
+ try:
+ return raw.decode("utf-8"), "utf-8", False
+ except UnicodeDecodeError:
+ pass
+ try:
+ return raw.decode("utf-8-sig"), "utf-8-sig", True
+ except UnicodeDecodeError:
+ pass
+ # latin-1 maps every byte; the flag tells the model this was a guess.
+ return raw.decode("latin-1"), "latin-1", True
+
+
+def _window_lines(
+ text: str, offset: int, limit: int
+) -> tuple[str, int, bool]:
+ lines = text.splitlines()
+ window = lines[offset : offset + limit]
+ joined = "\n".join(window)
+ truncated = offset + limit < len(lines)
+ if len(joined.encode("utf-8", errors="replace")) > TEXT_WINDOW_MAX_BYTES:
+ encoded = joined.encode("utf-8", errors="replace")[:TEXT_WINDOW_MAX_BYTES]
+ joined = encoded.decode("utf-8", errors="replace")
+ truncated = True
+ return joined, len(lines), truncated
+
+
+def read_text(
+ path: Path, *, offset: int, limit: int, encoding: Optional[str]
+) -> dict[str, Any]:
+ text, used, guessed = _decode(path.read_bytes(), encoding)
+ window, total, truncated = _window_lines(text, offset, limit)
+ return {
+ "type": "text",
+ "text": window,
+ "lines_total": total,
+ "offset": offset,
+ "limit": limit,
+ "encoding": used,
+ "encoding_guessed": guessed,
+ "truncated": truncated,
+ }
+
+
+def read_csv(
+ path: Path, *, offset: int, limit: int, encoding: Optional[str]
+) -> dict[str, Any]:
+ text, used, guessed = _decode(path.read_bytes(), encoding)
+ sample = text[:16_384]
+ try:
+ dialect = csv.Sniffer().sniff(sample, delimiters=",;\t|")
+ delimiter = dialect.delimiter
+ except csv.Error:
+ delimiter = "\t" if path.suffix.lower() == ".tsv" else ","
+ reader = csv.reader(io.StringIO(text), delimiter=delimiter)
+
+ def clip(cell: str) -> str:
+ return cell if len(cell) <= CSV_MAX_CELL_CHARS else cell[:CSV_MAX_CELL_CHARS]
+
+ columns: list[str] = []
+ rows: list[list[str]] = []
+ data_rows = 0
+ cols_truncated = False
+ capped_limit = min(limit, CSV_MAX_ROWS)
+ for index, row in enumerate(reader):
+ if index == 0:
+ columns = [clip(cell) for cell in row[:CSV_MAX_COLS]]
+ cols_truncated = cols_truncated or len(row) > CSV_MAX_COLS
+ continue
+ data_rows += 1
+ position = index - 1
+ if position < offset or len(rows) >= capped_limit:
+ continue
+ cols_truncated = cols_truncated or len(row) > CSV_MAX_COLS
+ rows.append([clip(cell) for cell in row[:CSV_MAX_COLS]])
+ return {
+ "type": "csv",
+ "columns": columns,
+ "rows": rows,
+ "rows_total": data_rows,
+ "offset": offset,
+ "delimiter": delimiter,
+ "encoding": used,
+ "encoding_guessed": guessed,
+ "columns_truncated": cols_truncated,
+ "truncated": offset + len(rows) < data_rows,
+ }
+
+
+def _prune_json(
+ value: Any, depth: int, path: str, pruned: list[str]
+) -> Any:
+ if isinstance(value, dict):
+ if depth >= JSON_MAX_DEPTH:
+ if len(pruned) < JSON_MAX_PRUNED_PATHS:
+ pruned.append(path or "$")
+ return f"<pruned object: {len(value)} keys>"
+ return {
+ str(key): _prune_json(item, depth + 1, f"{path}.{key}", pruned)
+ for key, item in value.items()
+ }
+ if isinstance(value, list):
+ if depth >= JSON_MAX_DEPTH:
+ if len(pruned) < JSON_MAX_PRUNED_PATHS:
+ pruned.append(path or "$")
+ return f"<pruned array: {len(value)} items>"
+ return [
+ _prune_json(item, depth + 1, f"{path}[{index}]", pruned)
+ for index, item in enumerate(value)
+ ]
+ return value
+
+
+def read_json(path: Path, *, encoding: Optional[str]) -> dict[str, Any]:
+ raw = path.read_bytes()
+ if len(raw) > JSON_MAX_SOURCE_BYTES:
+ raise NodeUserError(
+ f"JSON file is {len(raw):,} bytes (limit "
+ f"{JSON_MAX_SOURCE_BYTES:,}). Read it as text with offset/limit "
+ "instead: as_type='text'"
+ )
+ text, used, guessed = _decode(raw, encoding)
+ try:
+ data = json.loads(text)
+ except json.JSONDecodeError as exc:
+ raise NodeUserError(f"File is not valid JSON: {exc}") from exc
+ pruned: list[str] = []
+ return {
+ "type": "json",
+ "data": _prune_json(data, 0, "", pruned),
+ "pruned_paths": pruned,
+ "encoding": used,
+ "encoding_guessed": guessed,
+ "truncated": bool(pruned),
+ }
+
+
+def read_pdf(path: Path, *, offset: int, limit: int) -> dict[str, Any]:
+ try:
+ from pypdf import PdfReader
+ except ImportError:
+ raise NodeUserError(
+ "PDF reading needs the 'docs' extra: uv sync --extra docs "
+ "(installs pypdf)"
+ ) from None
+ reader = PdfReader(str(path))
+ pages_total = len(reader.pages)
+ capped = min(limit, PDF_MAX_PAGES)
+ pages: list[dict[str, Any]] = []
+ for index in range(offset, min(offset + capped, pages_total)):
+ text = reader.pages[index].extract_text() or ""
+ pages.append(
+ {
+ "page": index + 1,
+ "text": text[:PDF_MAX_PAGE_CHARS],
+ "truncated": len(text) > PDF_MAX_PAGE_CHARS,
+ }
+ )
+ return {
+ "type": "pdf",
+ "pages": pages,
+ "pages_total": pages_total,
+ "offset": offset,
+ "truncated": offset + len(pages) < pages_total,
+ }
+
+
+def read_html(
+ path: Path, *, offset: int, limit: int, encoding: Optional[str]
+) -> dict[str, Any]:
+ try:
+ from bs4 import BeautifulSoup
+ except ImportError:
+ raise NodeUserError(
+ "HTML extraction needs the 'docs' extra: uv sync --extra docs "
+ "(installs beautifulsoup4). Or read the raw markup with "
+ "as_type='text'"
+ ) from None
+ markup, used, guessed = _decode(path.read_bytes(), encoding)
+ soup = BeautifulSoup(markup, "html.parser")
+ title = soup.title.get_text(strip=True) if soup.title else None
+ text = soup.get_text(separator="\n", strip=True)
+ window, total, truncated = _window_lines(text, offset, limit)
+ return {
+ "type": "html",
+ "title": title,
+ "text": window,
+ "lines_total": total,
+ "offset": offset,
+ "encoding": used,
+ "encoding_guessed": guessed,
+ "truncated": truncated,
+ }
+
+
+def read_xlsx(
+ path: Path, *, sheet: Optional[str], offset: int, limit: int
+) -> dict[str, Any]:
+ try:
+ from openpyxl import load_workbook
+ except ImportError:
+ raise NodeUserError(
+ "XLSX reading needs openpyxl (pip install openpyxl)"
+ ) from None
+ try:
+ workbook = load_workbook(str(path), read_only=True, data_only=True)
+ except Exception as exc:
+ raise NodeUserError(f"Cannot open spreadsheet: {exc}") from exc
+ try:
+ sheets = list(workbook.sheetnames)
+ if sheet:
+ if sheet not in sheets:
+ raise NodeUserError(
+ f"No sheet named {sheet!r}. Sheets: {', '.join(sheets)}"
+ )
+ worksheet = workbook[sheet]
+ else:
+ worksheet = workbook.active
+ capped = min(limit, XLSX_MAX_ROWS)
+
+ def clip(cell: Any) -> Any:
+ if isinstance(cell, str) and len(cell) > CSV_MAX_CELL_CHARS:
+ return cell[:CSV_MAX_CELL_CHARS]
+ return cell
+
+ columns: list[Any] = []
+ rows: list[list[Any]] = []
+ cols_truncated = False
+ for index, row in enumerate(worksheet.iter_rows(values_only=True)):
+ values = list(row)
+ cols_truncated = cols_truncated or len(values) > CSV_MAX_COLS
+ values = [clip(value) for value in values[:CSV_MAX_COLS]]
+ if index == 0:
+ columns = values
+ continue
+ position = index - 1
+ if position < offset:
+ continue
+ if len(rows) >= capped:
+ break
+ rows.append(values)
+ # max_row is header-inclusive metadata, cheap and close enough for
+ # paging; an exact count would stream the whole sheet.
+ rows_total = max(0, int(worksheet.max_row or 0) - 1)
+ return {
+ "type": "xlsx",
+ "sheet": worksheet.title,
+ "sheets": sheets,
+ "columns": columns,
+ "rows": rows,
+ "rows_total": rows_total,
+ "offset": offset,
+ "columns_truncated": cols_truncated,
+ "truncated": offset + len(rows) < rows_total,
+ }
+ finally:
+ workbook.close()
+
+
+def read_image_meta(path: Path) -> dict[str, Any]:
+ """Header-only probe — dimensions and format, never pixels, no OCR."""
+ try:
+ from PIL import Image
+ except ImportError:
+ raise NodeUserError(
+ "Image metadata needs Pillow (pip install pillow)"
+ ) from None
+ try:
+ with Image.open(path) as image:
+ return {
+ "type": "image",
+ "image": {
+ "width": image.width,
+ "height": image.height,
+ "format": (image.format or "").lower(),
+ "mode": image.mode,
+ },
+ }
+ except Exception as exc:
+ raise NodeUserError(f"Not a readable image: {exc}") from exc
+
+
+def file_sha256(path: Path) -> str:
+ digest = hashlib.sha256()
+ with open(path, "rb") as handle:
+ for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+ digest.update(chunk)
+ return digest.hexdigest()
+
+
+def walk_mount(
+ root: Path, *, start_rel: str, pattern: str
+) -> tuple[list[str], bool]:
+ """Bounded name-glob walk of a mount. Returns (rel_paths, truncated).
+
+ ``os.walk`` with ``followlinks=False`` (the default) so a symlinked
+ directory can't pull in an unbounded external tree; symlinked *files*
+ may appear in listings, but reading one later still goes through
+ ``resolve_within``, which rejects escapes.
+ """
+ base = (root / start_rel) if start_rel else root
+ base = base.resolve()
+ results: list[str] = []
+ visited = 0
+ truncated = False
+ base_depth = len(base.parts)
+ for current, dirs, files in os.walk(base):
+ depth = len(Path(current).parts) - base_depth
+ if depth >= SEARCH_MAX_DEPTH:
+ dirs[:] = []
+ for name in files:
+ visited += 1
+ if visited > SEARCH_MAX_VISITED:
+ truncated = True
+ dirs[:] = []
+ break
+ if fnmatch.fnmatch(name.lower(), pattern.lower()):
+ rel = os.path.relpath(
+ os.path.join(current, name), str(root)
+ ).replace(os.sep, "/")
+ results.append(rel)
+ if len(results) >= SEARCH_MAX_RESULTS:
+ truncated = True
+ dirs[:] = []
+ break
+ if truncated:
+ break
+ return results, truncated
+
+
+_SHRINKABLE_LISTS = ("rows", "pages", "entries", "matches", "items")
+
+
+def bound_result(result: dict[str, Any]) -> dict[str, Any]:
+ """Final size backstop: trim known payload keys until the envelope fits."""
+
+ def size(candidate: dict[str, Any]) -> int:
+ return len(json.dumps(candidate, default=str).encode("utf-8"))
+
+ if size(result) <= DATA_RESULT_MAX_BYTES:
+ return result
+ trimmed = dict(result)
+ trimmed["truncated"] = True
+ for key in _SHRINKABLE_LISTS:
+ value = trimmed.get(key)
+ while isinstance(value, list) and value and size(trimmed) > DATA_RESULT_MAX_BYTES:
+ value = value[: max(1, len(value) // 2)]
+ trimmed[key] = value
+ if len(value) == 1 and size(trimmed) > DATA_RESULT_MAX_BYTES:
+ break
+ if size(trimmed) > DATA_RESULT_MAX_BYTES:
+ for key in ("text", "data"):
+ if key in trimmed and size(trimmed) > DATA_RESULT_MAX_BYTES:
+ text = trimmed[key]
+ if isinstance(text, str):
+ trimmed[key] = text[: TEXT_WINDOW_MAX_BYTES // 2]
+ else:
+ trimmed[key] = "<payload removed: over result size cap>"
+ return trimmed
+
+
+__all__ = [
+ "CSV_MAX_ROWS",
+ "DATA_RESULT_MAX_BYTES",
+ "PDF_MAX_PAGES",
+ "SEARCH_MAX_RESULTS",
+ "XLSX_MAX_ROWS",
+ "bound_result",
+ "detect_tier",
+ "file_sha256",
+ "read_csv",
+ "read_html",
+ "read_image_meta",
+ "read_json",
+ "read_pdf",
+ "read_text",
+ "read_xlsx",
+ "walk_mount",
+]
diff --git a/server/nodes/tool/data_source/meta.json b/server/nodes/tool/data_source/meta.json
new file mode 100644
index 00000000..aa33b4b5
--- /dev/null
+++ b/server/nodes/tool/data_source/meta.json
@@ -0,0 +1,3 @@
+{
+ "color": "#50fa7b"
+}
diff --git a/server/nodes/vision/__init__.py b/server/nodes/vision/__init__.py
new file mode 100644
index 00000000..b71fa420
--- /dev/null
+++ b/server/nodes/vision/__init__.py
@@ -0,0 +1 @@
+"""Vision plugins — delegate image understanding to a vision-capable model."""
diff --git a/server/nodes/vision/vision_analyze/__init__.py b/server/nodes/vision/vision_analyze/__init__.py
new file mode 100644
index 00000000..31e9d9e2
--- /dev/null
+++ b/server/nodes/vision/vision_analyze/__init__.py
@@ -0,0 +1,352 @@
+"""Vision Analyze — the vision-delegate AI tool.
+
+Gives EVERY agent — including ones running text-only models — real image
+understanding: the tool loads a workspace image at the provider boundary
+(never carrying bytes in results), fits it to a visual-token budget, sends
+it to a vision-capable model, and returns text.
+
+This is deliberately the speech-node shape, not a ChatUnifier call: the
+unifier's message protocol is text-only today, and this node must work
+regardless of the host agent's provider. When native image blocks land in
+the provider layer, this tool remains the fallback rung for text-only host
+models and the specialist path (OCR-style extraction).
+
+Token cost/billing note: like the LLM-backed translate providers, usage is
+deliberately NOT recorded via the pricing service — this path bills
+provider tokens that the LLM pricing layer has no per-call meter for here.
+"""
+
+from __future__ import annotations
+
+import base64
+from typing import Any, Literal, Optional
+
+from pydantic import BaseModel, ConfigDict, Field, model_validator
+
+from services.plugin import (
+ NodeContext,
+ NodeUserError,
+ Operation,
+ TaskQueue,
+ ToolNode,
+)
+
+VisionProvider = Literal["openai", "anthropic", "gemini"]
+VisionOperation = Literal["describe", "extract_text"]
+VisionBudget = Literal["small", "normal", "large"]
+
+_MAX_OUTPUT_TOKENS = 1024
+
+_PROMPTS = {
+ "describe": "Describe this image in detail.",
+ "extract_text": (
+ "Extract all text visible in this image. Return only the text, "
+ "preserving reading order and line breaks."
+ ),
+}
+
+
+class VisionAnalyzeParams(BaseModel):
+ """Persisted operator configuration; never exposed as model arguments."""
+
+ provider: VisionProvider = Field(
+ default="openai",
+ description="Which vision-capable provider answers the delegate call.",
+ )
+ # NOT named `model`: with a sibling `provider` field the parameter panel
+ # would overwrite it with the chat-model list (reserved-name magic in
+ # ParameterRenderer.tsx).
+ vision_model: str = Field(
+ default="",
+ description=(
+ "Vision model id. Empty uses the provider's default model from "
+ "llm_defaults.json."
+ ),
+ )
+
+ model_config = ConfigDict(extra="ignore")
+
+
+class VisionAnalyzeToolInput(BaseModel):
+ """One locked schema visible to the LLM."""
+
+ operation: VisionOperation = Field(
+ description="describe the image, or extract_text (OCR-style)."
+ )
+ image: str = Field(
+ min_length=1,
+ max_length=4_096,
+ description=(
+ "Workspace-relative path to the image (e.g. imports/chart.png). "
+ "Use the data tool to discover files."
+ ),
+ )
+ question: Optional[str] = Field(
+ default=None,
+ max_length=4_000,
+ description="describe: an optional specific question about the image.",
+ )
+ budget: VisionBudget = Field(
+ default="normal",
+ description=(
+ "Visual-token budget: small (~256 tokens), normal (~1024), "
+ "large (~2048 — fine detail)."
+ ),
+ )
+
+ model_config = ConfigDict(extra="forbid")
+
+ @model_validator(mode="after")
+ def _strip(self) -> "VisionAnalyzeToolInput":
+ if not self.image.strip():
+ raise ValueError("image path required")
+ return self
+
+
+class VisionAnalyzeOutput(BaseModel):
+ operation: str
+ provider: str
+ vision_model: str
+ text: str
+ image: Optional[dict[str, Any]] = None
+
+ model_config = ConfigDict(extra="allow")
+
+
+async def _api_key(ctx: NodeContext, provider: str) -> str:
+ """Resolve the stored key through the provider's credential (the speech
+ idiom) so a missing key surfaces as the annotated credential envelope."""
+ async with ctx.connection(provider) as conn:
+ secrets = await conn.credentials()
+ api_key = str(secrets.get("api_key") or "")
+ if not api_key:
+ raise NodeUserError(
+ f"No API key stored for '{provider}'. Add one in the "
+ "Credentials modal."
+ )
+ return api_key
+
+
+def _default_model(provider: str) -> str:
+ from services.llm.config import get_provider_config
+
+ config = get_provider_config(provider)
+ default = getattr(config, "default_model", "") if config else ""
+ if not default:
+ raise NodeUserError(
+ f"No default model configured for provider '{provider}'; set "
+ "vision_model on the node"
+ )
+ return default
+
+
+class VisionAnalyzeNode(ToolNode):
+ type = "visionAnalyze"
+ display_name = "Vision Analyze"
+ subtitle = "See Images via a Vision Model"
+ group = ("tool",)
+ description = (
+ "Delegate image understanding to a vision-capable model: describe "
+ "an image or extract its text. Works with any host agent, "
+ "including text-only models"
+ )
+ component_kind = "tool"
+ tool_name = "vision"
+ tool_description = (
+ "See an image by delegating to a vision model. describe returns a "
+ "detailed description (pass question for something specific); "
+ "extract_text returns the text visible in the image. image is a "
+ "workspace-relative path; budget controls detail vs cost."
+ )
+ handles = (
+ {
+ "name": "output-tool",
+ "kind": "output",
+ "position": "top",
+ "label": "Vision",
+ "role": "tools",
+ },
+ )
+ ui_hints = {
+ "isToolPanel": True,
+ "hideInputSection": True,
+ "hideOutputSection": True,
+ "hideRunButton": True,
+ }
+ annotations = {
+ "destructive": False,
+ "readonly": True,
+ "open_world": True, # sends image content to an external provider
+ }
+ task_queue = TaskQueue.AI_HEAVY
+
+ Params = VisionAnalyzeParams
+ ToolInput = VisionAnalyzeToolInput
+ Output = VisionAnalyzeOutput
+ tool_schema_locked = True
+ server_controlled_fields = frozenset({"provider", "vision_model"})
+
+ @staticmethod
+ def _config(ctx: NodeContext, params: Any) -> VisionAnalyzeParams:
+ config = ctx.raw.get("_tool_config")
+ if isinstance(config, VisionAnalyzeParams):
+ return config
+ if isinstance(params, VisionAnalyzeParams):
+ return params
+ return VisionAnalyzeParams()
+
+ @Operation("vision")
+ async def vision(
+ self,
+ ctx: NodeContext,
+ params: VisionAnalyzeToolInput | VisionAnalyzeParams,
+ ) -> VisionAnalyzeOutput:
+ if isinstance(params, VisionAnalyzeParams):
+ raise NodeUserError(
+ "The vision tool needs an image argument; connect it to an "
+ "agent's tools and let the agent call it"
+ )
+ args = params
+ config = self._config(ctx, params)
+ provider = config.provider
+ model = config.vision_model.strip() or _default_model(provider)
+
+ # Bytes exist only inside this call — loaded through the contained
+ # media reader, resized to budget, sent, and dropped.
+ from services.media import read_media_bytes
+ from services.media.image_fit import fit_image_bytes
+ from services.media.limits import MEDIA_MAX_READ_BYTES
+
+ try:
+ filename, blob = read_media_bytes(
+ args.image, ctx=ctx, max_bytes=MEDIA_MAX_READ_BYTES
+ )
+ except ValueError as exc:
+ raise NodeUserError(str(exc)) from exc
+ try:
+ fitted, mime, (width, height) = fit_image_bytes(
+ blob, budget=args.budget
+ )
+ except Exception as exc:
+ raise NodeUserError(
+ f"'{filename}' is not a readable image: {exc}"
+ ) from exc
+
+ prompt = (
+ args.question.strip()
+ if args.operation == "describe" and args.question
+ else _PROMPTS[args.operation]
+ )
+ api_key = await _api_key(ctx, provider)
+ dispatch = {
+ "openai": self._ask_openai,
+ "anthropic": self._ask_anthropic,
+ "gemini": self._ask_gemini,
+ }
+ text = await dispatch[provider](
+ api_key=api_key,
+ model=model,
+ prompt=prompt,
+ image_bytes=fitted,
+ mime=mime,
+ )
+ return VisionAnalyzeOutput(
+ operation=args.operation,
+ provider=provider,
+ vision_model=model,
+ text=text,
+ image={"width": width, "height": height, "source": args.image},
+ )
+
+ # ------------------------------------------------ provider requests
+
+ @staticmethod
+ async def _ask_openai(
+ *, api_key: str, model: str, prompt: str, image_bytes: bytes, mime: str
+ ) -> str:
+ from openai import AsyncOpenAI
+
+ client = AsyncOpenAI(api_key=api_key)
+ data_url = (
+ f"data:{mime};base64,"
+ f"{base64.b64encode(image_bytes).decode('ascii')}"
+ )
+ response = await client.chat.completions.create(
+ model=model,
+ max_completion_tokens=_MAX_OUTPUT_TOKENS,
+ messages=[
+ {
+ "role": "user",
+ "content": [
+ {"type": "text", "text": prompt},
+ {
+ "type": "image_url",
+ # detail set explicitly — omitting it has caused
+ # order-of-magnitude token regressions elsewhere.
+ "image_url": {"url": data_url, "detail": "auto"},
+ },
+ ],
+ }
+ ],
+ )
+ return str(response.choices[0].message.content or "")
+
+ @staticmethod
+ async def _ask_anthropic(
+ *, api_key: str, model: str, prompt: str, image_bytes: bytes, mime: str
+ ) -> str:
+ from anthropic import AsyncAnthropic
+
+ client = AsyncAnthropic(api_key=api_key)
+ response = await client.messages.create(
+ model=model,
+ max_tokens=_MAX_OUTPUT_TOKENS,
+ messages=[
+ {
+ "role": "user",
+ "content": [
+ {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": mime,
+ "data": base64.b64encode(image_bytes).decode(
+ "ascii"
+ ),
+ },
+ },
+ {"type": "text", "text": prompt},
+ ],
+ }
+ ],
+ )
+ return "".join(
+ block.text
+ for block in response.content
+ if getattr(block, "type", "") == "text"
+ )
+
+ @staticmethod
+ async def _ask_gemini(
+ *, api_key: str, model: str, prompt: str, image_bytes: bytes, mime: str
+ ) -> str:
+ from google import genai
+ from google.genai import types as genai_types
+
+ client = genai.Client(api_key=api_key)
+ response = await client.aio.models.generate_content(
+ model=model,
+ contents=[
+ genai_types.Part.from_bytes(data=image_bytes, mime_type=mime),
+ prompt,
+ ],
+ )
+ return str(getattr(response, "text", "") or "")
+
+
+__all__ = [
+ "VisionAnalyzeNode",
+ "VisionAnalyzeOutput",
+ "VisionAnalyzeParams",
+ "VisionAnalyzeToolInput",
+]
diff --git a/server/nodes/vision/vision_analyze/meta.json b/server/nodes/vision/vision_analyze/meta.json
new file mode 100644
index 00000000..8f309ba5
--- /dev/null
+++ b/server/nodes/vision/vision_analyze/meta.json
@@ -0,0 +1,3 @@
+{
+ "color": "#bd93f9"
+}
diff --git a/server/nodes/visuals.json b/server/nodes/visuals.json
index 7e73ce43..1256c033 100644
--- a/server/nodes/visuals.json
+++ b/server/nodes/visuals.json
@@ -100,6 +100,20 @@
"currentTimeTool": {
"icon": "🕐"
},
+ "data": {
+ "icon": "🗂️",
+ "color": "#50fa7b"
+ },
+ "dataSource": {
+ "icon": "🗂️"
+ },
+ "vision": {
+ "icon": "👁️",
+ "color": "#bd93f9"
+ },
+ "visionAnalyze": {
+ "icon": "👁️"
+ },
"deepseekChatModel": {
"icon": "lobehub:deepseek"
},
diff --git a/server/pyproject.toml b/server/pyproject.toml
index 84b29591..da9e1176 100644
--- a/server/pyproject.toml
+++ b/server/pyproject.toml
@@ -44,6 +44,13 @@ dependencies = [
# project's MIT licence) and over pydub / ffprobe, which both require
# an ffmpeg binary on PATH that nothing installs on Windows.
"tinytag>=2.0.0",
+ # Data node read tiers (nodes/tool/data_source/_readers.py). First-class
+ # rather than an optional extra so the LLM-facing tool behaves the same
+ # on every install: openpyxl for XLSX (pure Python), Pillow for image
+ # metadata (was already shipping transitively via qrcode[pil]) — and for
+ # services/media/image_fit.py budget resizing.
+ "openpyxl>=3.1.0",
+ "pillow>=10.0.0",
# Multipart parsing for the workspace upload route. Already present
# transitively via mcp; declared explicitly so it survives that
# dependency changing.
diff --git a/server/services/agent_runtime.py b/server/services/agent_runtime.py
index c18c82b6..b407bde3 100644
--- a/server/services/agent_runtime.py
+++ b/server/services/agent_runtime.py
@@ -24,6 +24,10 @@ from typing import (
from pydantic import BaseModel, ValidationError
from core.logging import get_logger
+from services.llm.media import (
+ hydrate_image_blocks,
+ image_blocks_from_tool_result,
+)
from services.llm.messages import filter_empty_messages
from services.llm.protocol import (
LLMError,
@@ -139,13 +143,18 @@ async def run_native_llm_step(
raise RuntimeError("ChatUnifier is required for native agent execution")
definitions = [_tool_definition(tool) for tool in (tools or ())]
+ # Hydrate once, outside the retry loop, on throwaway copies — refs stay
+ # the durable form; bytes exist only for this provider call.
+ prepared = await hydrate_image_blocks(
+ filter_empty_messages(messages), provider=provider, model=model
+ )
attempts = max(0, int(explicit_max_retries)) + 1
for attempt in range(attempts):
try:
return await chat_unifier.chat(
provider=provider,
api_key=api_key,
- messages=filter_empty_messages(messages),
+ messages=prepared,
model=model,
temperature=temperature,
max_tokens=max_tokens,
@@ -486,6 +495,9 @@ async def run_native_agent_loop(
tool_call_id=call.id,
name=call.name,
)
+ # Tools opt into vision via `llm_media` refs; blocks stay ~450 B
+ # in durable state (hydration happens per provider call).
+ tool_message.blocks.extend(image_blocks_from_tool_result(result))
messages.append(tool_message)
await persist_transition(
"message.tool_result",
diff --git a/server/services/data/__init__.py b/server/services/data/__init__.py
new file mode 100644
index 00000000..b419023f
--- /dev/null
+++ b/server/services/data/__init__.py
@@ -0,0 +1,13 @@
+"""Data-node services: the machine-wide external mount allowlist."""
+
+from services.data.mount_store import (
+ DataMount,
+ DataMountStore,
+ MountStoreError,
+)
+
+__all__ = [
+ "DataMount",
+ "DataMountStore",
+ "MountStoreError",
+]
diff --git a/server/services/data/mount_store.py b/server/services/data/mount_store.py
new file mode 100644
index 00000000..c8abef4a
--- /dev/null
+++ b/server/services/data/mount_store.py
@@ -0,0 +1,284 @@
+"""Machine-wide external-folder allowlist backing the Data node.
+
+This module deliberately owns its table instead of adding mount-specific
+methods to :mod:`core.database` — the same shape as
+:mod:`services.memory.tool_store`. Importing the Data plugin registers the
+SQLModel table before ``Database.startup()`` calls ``create_all``;
+:meth:`DataMountStore.ensure_schema` covers standalone workers/tests that
+import later.
+
+A mount row grants every Data node on this machine *visibility* of one
+absolute directory; each node then exposes a subset by name. Validation at
+save time is the security boundary that keeps the allowlist from covering
+locations the platform itself depends on (``DATA_DIR`` holds
+``credentials.db``, ``workflow.db`` and every workspace), so the checks
+here are load-bearing, not cosmetic.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+import re
+from datetime import datetime, timezone
+from pathlib import Path
+from typing import Any, Optional
+
+from sqlmodel import Field, SQLModel, select
+
+from core.logging import get_logger
+
+logger = get_logger(__name__)
+
+# Mount names appear inside ``mnt/<name>/...`` virtual paths the LLM sees
+# and round-trips, so they are constrained to a URL/path-safe shape.
+MOUNT_NAME_PATTERN = re.compile(r"^[a-z0-9][a-z0-9_-]{0,63}$")
+
+
+def _utcnow() -> datetime:
+ return datetime.now(timezone.utc)
+
+
+class DataMount(SQLModel, table=True):
+ """One operator-approved external directory."""
+
+ __tablename__ = "data_mounts"
+
+ id: Optional[int] = Field(default=None, primary_key=True)
+ user_id: str = Field(default="owner", index=True, max_length=255)
+ name: str = Field(index=True, max_length=64)
+ root_path: str = Field(max_length=1024)
+ writable: bool = Field(default=False)
+ created_at: datetime = Field(default_factory=_utcnow)
+ updated_at: datetime = Field(default_factory=_utcnow)
+
+
+class MountStoreError(ValueError):
+ """User-correctable mount configuration failure."""
+
+
+def _protected_roots() -> list[Path]:
+ """Directories the allowlist must never cover (either direction)."""
+ from core.paths import opencompany_root
+
+ roots = [Path(opencompany_root()).resolve()]
+ # A pre-rebrand data dir may still hold live databases.
+ legacy = Path.home() / ".machina"
+ if legacy.exists():
+ roots.append(legacy.resolve())
+ return roots
+
+
+def _overlaps(candidate: Path, protected: Path) -> bool:
+ return candidate.is_relative_to(protected) or protected.is_relative_to(
+ candidate
+ )
+
+
+def validate_mount_root(root_path: str, *, writable: bool) -> Path:
+ """Validate and canonicalize a mount root. Raises :class:`MountStoreError`.
+
+ Filesystem probes run in the caller's thread; callers on the event loop
+ wrap this in ``asyncio.to_thread``.
+ """
+ raw = Path(str(root_path or "").strip())
+ if not str(raw):
+ raise MountStoreError("Mount path required")
+ if not raw.is_absolute():
+ raise MountStoreError("Mount path must be absolute")
+ try:
+ resolved = raw.resolve(strict=True)
+ except FileNotFoundError:
+ raise MountStoreError(f"Mount path does not exist: {raw}") from None
+ except OSError as exc:
+ raise MountStoreError(f"Mount path is not usable: {exc}") from exc
+ if not resolved.is_dir():
+ raise MountStoreError("Mount path must be a directory")
+ # A filesystem/drive root or the home directory itself is too broad to
+ # be an allowlist entry — it stops being a list at that point.
+ if resolved == Path(resolved.anchor):
+ raise MountStoreError("Refusing to mount a filesystem root")
+ if resolved == Path.home().resolve():
+ raise MountStoreError(
+ "Refusing to mount the home directory itself; mount a subfolder"
+ )
+ for protected in _protected_roots():
+ if _overlaps(resolved, protected):
+ raise MountStoreError(
+ "Refusing to mount the OpenCompany data directory or a "
+ "path containing it (credential and workflow storage)"
+ )
+ if not os.access(resolved, os.R_OK):
+ raise MountStoreError("Mount path is not readable by the server")
+ if writable and not os.access(resolved, os.W_OK):
+ raise MountStoreError(
+ "Mount path is not writable; save it read-only instead"
+ )
+ return resolved
+
+
+def _serialize(mount: DataMount) -> dict[str, Any]:
+ # root_path is shown back to the operator in the panel (they typed it).
+ # It must never be copied into node results — the node layer emits only
+ # ``mnt/<name>/...`` virtual paths.
+ return {
+ "name": mount.name,
+ "root_path": mount.root_path,
+ "writable": bool(mount.writable),
+ "created_at": mount.created_at.isoformat()
+ if mount.created_at
+ else None,
+ "updated_at": mount.updated_at.isoformat()
+ if mount.updated_at
+ else None,
+ }
+
+
+class DataMountStore:
+ """CRUD for the machine-wide mount allowlist."""
+
+ _schema_lock = asyncio.Lock()
+ _initialized_engines: set[Any] = set()
+
+ def __init__(self, database: Any) -> None:
+ self.database = database
+
+ async def ensure_schema(self) -> None:
+ engine = getattr(self.database, "engine", None)
+ if engine is None:
+ raise RuntimeError("Database is not initialized")
+ if engine in self._initialized_engines:
+ return
+ async with self._schema_lock:
+ if engine in self._initialized_engines:
+ return
+ async with engine.begin() as connection:
+ await connection.run_sync(
+ lambda sync_connection: DataMount.__table__.create(
+ sync_connection, checkfirst=True
+ )
+ )
+ self._initialized_engines.add(engine)
+
+ @staticmethod
+ def _validate_name(name: str) -> str:
+ candidate = str(name or "").strip()
+ if not MOUNT_NAME_PATTERN.fullmatch(candidate):
+ raise MountStoreError(
+ "Mount name must be 1-64 chars of lowercase letters, "
+ "digits, '-' or '_', starting with a letter or digit"
+ )
+ return candidate
+
+ async def list_mounts(self, owner_id: str) -> list[dict[str, Any]]:
+ await self.ensure_schema()
+ async with self.database.get_session() as session:
+ result = await session.execute(
+ select(DataMount)
+ .where(DataMount.user_id == owner_id)
+ .order_by(DataMount.name)
+ )
+ return [_serialize(row) for row in result.scalars().all()]
+
+ async def get_mount(
+ self, owner_id: str, name: str
+ ) -> Optional[dict[str, Any]]:
+ await self.ensure_schema()
+ async with self.database.get_session() as session:
+ row = await self._row(session, owner_id, name)
+ return _serialize(row) if row else None
+
+ async def add_mount(
+ self,
+ owner_id: str,
+ *,
+ name: str,
+ root_path: str,
+ writable: bool = False,
+ ) -> dict[str, Any]:
+ await self.ensure_schema()
+ candidate = self._validate_name(name)
+ resolved = await asyncio.to_thread(
+ validate_mount_root, root_path, writable=bool(writable)
+ )
+ async with self.database.get_session() as session:
+ existing = await session.execute(
+ select(DataMount).where(DataMount.user_id == owner_id)
+ )
+ resolved_key = os.path.normcase(str(resolved))
+ for row in existing.scalars().all():
+ if row.name == candidate:
+ raise MountStoreError(
+ f"A mount named '{candidate}' already exists"
+ )
+ if os.path.normcase(row.root_path) == resolved_key:
+ raise MountStoreError(
+ f"That folder is already mounted as '{row.name}'"
+ )
+ mount = DataMount(
+ user_id=owner_id,
+ name=candidate,
+ root_path=str(resolved),
+ writable=bool(writable),
+ )
+ session.add(mount)
+ await session.commit()
+ await session.refresh(mount)
+ logger.info(
+ "Data mount added",
+ mount=candidate,
+ writable=bool(writable),
+ )
+ return _serialize(mount)
+
+ async def update_mount(
+ self, owner_id: str, *, name: str, writable: bool
+ ) -> dict[str, Any]:
+ """Only the writable flag is mutable; path changes are remove+add."""
+ await self.ensure_schema()
+ async with self.database.get_session() as session:
+ row = await self._row(session, owner_id, name)
+ if row is None:
+ raise MountStoreError(f"No mount named '{name}'")
+ if writable:
+ await asyncio.to_thread(
+ validate_mount_root, row.root_path, writable=True
+ )
+ row.writable = bool(writable)
+ row.updated_at = _utcnow()
+ session.add(row)
+ await session.commit()
+ await session.refresh(row)
+ return _serialize(row)
+
+ async def remove_mount(self, owner_id: str, *, name: str) -> dict[str, Any]:
+ await self.ensure_schema()
+ async with self.database.get_session() as session:
+ row = await self._row(session, owner_id, name)
+ if row is None:
+ raise MountStoreError(f"No mount named '{name}'")
+ await session.delete(row)
+ await session.commit()
+ logger.info("Data mount removed", mount=name)
+ return {"removed": True, "name": name}
+
+ @staticmethod
+ async def _row(
+ session: Any, owner_id: str, name: str
+ ) -> Optional[DataMount]:
+ result = await session.execute(
+ select(DataMount).where(
+ DataMount.user_id == owner_id,
+ DataMount.name == str(name or "").strip(),
+ )
+ )
+ return result.scalars().first()
+
+
+__all__ = [
+ "DataMount",
+ "DataMountStore",
+ "MOUNT_NAME_PATTERN",
+ "MountStoreError",
+ "validate_mount_root",
+]
diff --git a/server/services/llm/media.py b/server/services/llm/media.py
new file mode 100644
index 00000000..3a32f7b6
--- /dev/null
+++ b/server/services/llm/media.py
@@ -0,0 +1,197 @@
+"""Tool-result images: refs in durable state, bytes only at the provider boundary.
+
+A tool opts in by returning ``llm_media: [{"ref": <FileRef kind=image>,
+"detail": "auto"}]``. The agent loop attaches ref-only image ContentBlocks to
+the tool message (durable, ~450 B each). ``hydrate_image_blocks`` runs once
+per LLM step on throwaway copies, loading bytes through the contained media
+reader and fitting them to a budget; provider encoders then emit the
+official wire shapes. Originals are never mutated, so nothing downstream of
+the journal ever sees bytes.
+"""
+
+from __future__ import annotations
+
+from dataclasses import replace
+from typing import Any, List, Optional, Sequence
+
+from core.logging import get_logger
+from services.llm.config import LLM_DEFAULTS
+from services.llm.protocol import ContentBlock, Message
+
+logger = get_logger(__name__)
+
+LLM_MEDIA_KEY = "llm_media"
+LLM_MEDIA_MAX_PER_RESULT = 8
+IMAGE_MIME_ALLOWLIST = {"image/png", "image/jpeg", "image/webp", "image/gif"}
+
+# detail -> visual-token budget for services.media.image_fit.
+_DETAIL_BUDGETS = {"low": "small", "auto": "normal", "high": "large"}
+
+
+def provider_supports_vision(provider: str, model: str = "") -> bool:
+ """Capability gate. Unknown means False: a rejected image block would be
+ journaled and resent on every turn, so never emit one speculatively."""
+ del model # per-model gating arrives with registry modality data
+ vision = (
+ LLM_DEFAULTS.get("providers", {}).get(provider, {}).get("vision", {})
+ )
+ return bool(vision.get("enabled"))
+
+
+def image_blocks_from_tool_result(result: Any) -> List[ContentBlock]:
+ """Parse a tool result's ``llm_media`` opt-in into ref-only image blocks.
+
+ Invalid entries are skipped with a warning, never raised — the tool
+ already did its (possibly paid) work.
+ """
+ if not isinstance(result, dict):
+ return []
+ entries = result.get(LLM_MEDIA_KEY)
+ if not isinstance(entries, list):
+ return []
+ blocks: List[ContentBlock] = []
+ for entry in entries[:LLM_MEDIA_MAX_PER_RESULT]:
+ ref = entry.get("ref") if isinstance(entry, dict) else None
+ if not isinstance(ref, dict):
+ logger.warning("llm_media entry without a ref; skipped")
+ continue
+ if ref.get("kind") not in ("image", "file"):
+ logger.warning(
+ "llm_media ref kind unsupported", kind=ref.get("kind")
+ )
+ continue
+ if ref.get("mime_type") not in IMAGE_MIME_ALLOWLIST:
+ logger.warning(
+ "llm_media mime unsupported", mime=ref.get("mime_type")
+ )
+ continue
+ if not ref.get("workflow_id") or not ref.get("path"):
+ logger.warning("llm_media ref missing workflow_id/path; skipped")
+ continue
+ detail = str(entry.get("detail") or "auto")
+ blocks.append(
+ ContentBlock(
+ type="image",
+ source={
+ "kind": "file_ref",
+ "ref": dict(ref),
+ "detail": detail if detail in _DETAIL_BUDGETS else "auto",
+ },
+ )
+ )
+ return blocks
+
+
+def _placeholder(ref: dict, reason: str) -> ContentBlock:
+ filename = ref.get("filename") or ref.get("path") or "image"
+ return ContentBlock(
+ type="text", text=f"[Image attached: {filename} — {reason}]"
+ )
+
+
+async def hydrate_image_blocks(
+ messages: Sequence[Message], *, provider: str, model: str
+) -> List[Message]:
+ """Return messages with image blocks hydrated for one provider call.
+
+ Identity (same objects, zero cost) when nothing carries an image ref.
+ Hydration works on copies; per-image failures degrade to a text
+ placeholder rather than failing an otherwise-valid turn.
+ """
+ if not any(_ref_blocks(message) for message in messages):
+ return list(messages)
+
+ capable = provider_supports_vision(provider, model)
+ hydrated: List[Message] = []
+ for message in messages:
+ if not _ref_blocks(message):
+ hydrated.append(message)
+ continue
+ blocks: List[ContentBlock] = []
+ for block in message.blocks:
+ if not _is_ref_block(block):
+ blocks.append(block)
+ continue
+ source = block.source or {}
+ ref = dict(source.get("ref") or {})
+ if not capable:
+ blocks.append(
+ _placeholder(
+ ref,
+ f"the current model ({model}) cannot view images; "
+ "metadata only",
+ )
+ )
+ continue
+ try:
+ data_b64, mime = await _hydrate_one(
+ ref, str(source.get("detail") or "auto")
+ )
+ blocks.append(
+ ContentBlock(
+ type="image",
+ source={
+ "kind": "bytes",
+ "media_type": mime,
+ "data_b64": data_b64,
+ "detail": source.get("detail") or "auto",
+ },
+ )
+ )
+ except Exception as exc:
+ logger.warning(
+ "image hydration failed", path=ref.get("path"), error=str(exc)
+ )
+ blocks.append(_placeholder(ref, f"unavailable: {exc}"))
+ hydrated.append(replace(message, blocks=blocks))
+ return hydrated
+
+
+def _is_ref_block(block: ContentBlock) -> bool:
+ return (
+ block.type == "image"
+ and isinstance(block.source, dict)
+ and block.source.get("kind") == "file_ref"
+ )
+
+
+def _ref_blocks(message: Message) -> List[ContentBlock]:
+ return [block for block in message.blocks or [] if _is_ref_block(block)]
+
+
+async def _hydrate_one(ref: dict, detail: str) -> tuple[str, str]:
+ """Resolve a FileRef to fitted base64 + mime, contained end to end."""
+ import asyncio
+ import base64
+
+ from nodes.filesystem._backend import resolve_within
+ from services.media.image_fit import fit_image_bytes
+ from services.media.limits import MEDIA_MAX_READ_BYTES
+ from services.plugin.deps import get_database
+ from services.workspace_locator import resolve_workspace_root
+
+ root = await resolve_workspace_root(
+ str(ref.get("workflow_id") or ""), get_database()
+ )
+ path = resolve_within(root, str(ref.get("path") or ""))
+ if not path.is_file():
+ raise FileNotFoundError(f"{ref.get('path')} not found in workspace")
+ if path.stat().st_size > MEDIA_MAX_READ_BYTES:
+ raise ValueError("image exceeds the read size cap")
+ budget = _DETAIL_BUDGETS.get(detail, "normal")
+
+ def fit() -> tuple[str, str]:
+ fitted, mime, _ = fit_image_bytes(path.read_bytes(), budget=budget)
+ return base64.b64encode(fitted).decode("ascii"), mime
+
+ return await asyncio.to_thread(fit)
+
+
+__all__ = [
+ "IMAGE_MIME_ALLOWLIST",
+ "LLM_MEDIA_KEY",
+ "LLM_MEDIA_MAX_PER_RESULT",
+ "hydrate_image_blocks",
+ "image_blocks_from_tool_result",
+ "provider_supports_vision",
+]
diff --git a/server/services/llm/protocol.py b/server/services/llm/protocol.py
index 68604f55..f703adfb 100644
--- a/server/services/llm/protocol.py
+++ b/server/services/llm/protocol.py
@@ -187,6 +187,10 @@ class ContentBlock:
tool_call_id: Optional[str] = None
name: Optional[str] = None
metadata: Dict[str, Any] = field(default_factory=dict)
+ # Media descriptor for ``type == "image"`` blocks, discriminated by
+ # ``kind``: durable ``file_ref`` (a FileRef dump) or transient ``bytes``
+ # (hydrated request material — the wire codec refuses to serialize it).
+ source: Optional[Dict[str, Any]] = None
@dataclass
@@ -568,6 +572,15 @@ def _tool_call_from_wire(value: Mapping[str, Any]) -> ToolCall:
)
+def _durable_source(source: Optional[Dict[str, Any]]) -> Optional[Dict[str, Any]]:
+ """Only ref-shaped image sources are durable; hydrated bytes are transient."""
+ if source is None:
+ return None
+ if source.get("kind") == "bytes":
+ raise ValueError("image bytes must not enter durable state")
+ return _json_safe(dict(source))
+
+
def _block_to_wire(block: ContentBlock) -> Dict[str, Any]:
return {
"type": block.type,
@@ -578,12 +591,14 @@ def _block_to_wire(block: ContentBlock) -> Dict[str, Any]:
"tool_call_id": block.tool_call_id,
"name": block.name,
"metadata": _json_safe(block.metadata),
+ "source": _durable_source(block.source),
}
def _block_from_wire(value: Mapping[str, Any]) -> ContentBlock:
tool_call = value.get("tool_call")
metadata = value.get("metadata")
+ source = value.get("source")
return ContentBlock(
type=str(value.get("type") or "text"),
text=str(value.get("text") or ""),
@@ -595,6 +610,7 @@ def _block_from_wire(value: Mapping[str, Any]) -> ContentBlock:
tool_call_id=_optional_str(value.get("tool_call_id")),
name=_optional_str(value.get("name")),
metadata=dict(metadata) if isinstance(metadata, Mapping) else {},
+ source=dict(source) if isinstance(source, Mapping) else None,
)
diff --git a/server/services/llm/providers/anthropic.py b/server/services/llm/providers/anthropic.py
index 44780d7a..3128386a 100644
--- a/server/services/llm/providers/anthropic.py
+++ b/server/services/llm/providers/anthropic.py
@@ -205,7 +205,7 @@ class AnthropicProvider:
{
"type": "tool_result",
"tool_use_id": tool_message.tool_call_id or "",
- "content": tool_message.content,
+ "content": self._tool_result_content(tool_message),
}
)
index += 1
@@ -216,6 +216,30 @@ class AnthropicProvider:
index += 1
return "\n\n".join(system_parts) if system_parts else None, api_msgs
+ @staticmethod
+ def _tool_result_content(m: Message) -> Any:
+ """Anthropic tool_result content: plain string, or a block list when
+ hydrated images ride along (the documented image-in-tool-result
+ shape). Ref-only blocks render nothing here — hydration either
+ produced bytes or a text placeholder before compile time."""
+ images = [
+ {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": block.source["media_type"],
+ "data": block.source["data_b64"],
+ },
+ }
+ for block in m.blocks or []
+ if block.type == "image"
+ and isinstance(block.source, dict)
+ and block.source.get("kind") == "bytes"
+ ]
+ if not images:
+ return m.content
+ return [{"type": "text", "text": m.content}, *images]
+
def _to_api_message(self, m: Message) -> Dict[str, Any]:
role = "assistant" if m.role == "assistant" else "user"
@@ -226,7 +250,7 @@ class AnthropicProvider:
{
"type": "tool_result",
"tool_use_id": m.tool_call_id or "",
- "content": m.content,
+ "content": self._tool_result_content(m),
}
],
}
diff --git a/server/services/media/image_fit.py b/server/services/media/image_fit.py
new file mode 100644
index 00000000..2b35aa0c
--- /dev/null
+++ b/server/services/media/image_fit.py
@@ -0,0 +1,116 @@
+"""Budget-based image fitting for vision-model input.
+
+The budget abstraction (after QwenLM/Qwen-MM-Plugins): callers pick
+``"small" | "normal" | "large"`` defined in *visual tokens*, converted to a
+pixel area via ``tokens x patch^2``. The agent gets a cost dial it can
+reason about, and per-provider retargeting is a parameter, not a rewrite.
+
+``smart_resize`` is the canonical NaViT-style rule (verbatim behavior from
+the Qwen-VL preprocessing): one aspect-preserving scale into the
+[min_pixels, max_pixels] area window, then snap both dimensions to patch
+multiples so no partial patches are billed. Both a floor and a ceiling are
+enforced — a tiny thumbnail is scaled UP so its budget isn't wasted.
+
+Bytes produced here are transient request material only. They must never
+enter node results, Temporal payloads, or the context journal (the
+never-bytes rule in docs-internal/media_transport.md).
+"""
+
+from __future__ import annotations
+
+import io
+import math
+from typing import Tuple
+
+IMAGE_BUDGET_TOKENS = {"small": 256, "normal": 1024, "large": 2048}
+DEFAULT_BUDGET = "normal"
+# 28 px is the Anthropic patch edge; close enough to every current
+# provider's tiling that a single default serves the delegate path.
+# Callers with provider-exact needs pass their own factor.
+DEFAULT_PATCH = 28
+_MIN_TOKENS = min(IMAGE_BUDGET_TOKENS.values())
+_JPEG_QUALITY = 90
+
+
+def budget_to_pixels(budget: str, *, patch: int = DEFAULT_PATCH) -> int:
+ tokens = IMAGE_BUDGET_TOKENS.get(budget, IMAGE_BUDGET_TOKENS[DEFAULT_BUDGET])
+ return tokens * patch * patch
+
+
+def smart_resize(
+ height: int,
+ width: int,
+ *,
+ min_pixels: int,
+ max_pixels: int,
+ factor: int = DEFAULT_PATCH,
+) -> Tuple[int, int]:
+ """Aspect-preserving (height, width) inside the pixel-area window,
+ snapped to ``factor`` multiples."""
+ if height <= 0 or width <= 0:
+ raise ValueError("image dimensions must be positive")
+ area = height * width
+ scale = 1.0
+ if area < min_pixels:
+ scale = math.sqrt(min_pixels / area)
+ elif area > max_pixels:
+ scale = math.sqrt(max_pixels / area)
+ resized_height = max(factor, round(height * scale / factor) * factor)
+ resized_width = max(factor, round(width * scale / factor) * factor)
+ return resized_height, resized_width
+
+
+def fit_image_bytes(
+ data: bytes,
+ *,
+ budget: str = DEFAULT_BUDGET,
+ patch: int = DEFAULT_PATCH,
+ max_pixels_cap: int | None = None,
+) -> Tuple[bytes, str, Tuple[int, int]]:
+ """Fit encoded image bytes to a token budget.
+
+ Returns ``(encoded_bytes, mime_type, (width, height))``. JPEG q90 for
+ opaque images (tool results accumulate; PNG at full budget is what
+ blows request caps), PNG kept when alpha must survive.
+ """
+ from PIL import Image
+
+ max_pixels = budget_to_pixels(budget, patch=patch)
+ if max_pixels_cap is not None:
+ max_pixels = min(max_pixels, max_pixels_cap)
+ min_pixels = min(_MIN_TOKENS * patch * patch, max_pixels)
+
+ with Image.open(io.BytesIO(data)) as image:
+ target_h, target_w = smart_resize(
+ image.height,
+ image.width,
+ min_pixels=min_pixels,
+ max_pixels=max_pixels,
+ factor=patch,
+ )
+ has_alpha = image.mode in ("RGBA", "LA", "PA") or (
+ image.mode == "P" and "transparency" in image.info
+ )
+ if (target_w, target_h) != (image.width, image.height):
+ resample = Image.Resampling.LANCZOS
+ image = image.resize((target_w, target_h), resample)
+ buffer = io.BytesIO()
+ if has_alpha:
+ image.convert("RGBA").save(buffer, format="PNG", optimize=True)
+ mime = "image/png"
+ else:
+ image.convert("RGB").save(
+ buffer, format="JPEG", quality=_JPEG_QUALITY
+ )
+ mime = "image/jpeg"
+ return buffer.getvalue(), mime, (target_w, target_h)
+
+
+__all__ = [
+ "DEFAULT_BUDGET",
+ "DEFAULT_PATCH",
+ "IMAGE_BUDGET_TOKENS",
+ "budget_to_pixels",
+ "fit_image_bytes",
+ "smart_resize",
+]
diff --git a/server/services/temporal/agent_activities.py b/server/services/temporal/agent_activities.py
index 514ca9bf..ab6528a6 100644
--- a/server/services/temporal/agent_activities.py
+++ b/server/services/temporal/agent_activities.py
@@ -2469,6 +2469,10 @@ async def _execute_tool_and_append(
tool_call_id=str(call.get("id") or pending.get("call_id") or ""),
name=str(call.get("name") or ""),
)
+ from services.llm.media import image_blocks_from_tool_result
+
+ # Tools opt into vision via `llm_media`; blocks journal as ~450 B refs.
+ tool_message.blocks.extend(image_blocks_from_tool_result(tool_result))
ref = await _append_event(
store,
ref,
diff --git a/server/tests/llm/test_media_blocks.py b/server/tests/llm/test_media_blocks.py
new file mode 100644
index 00000000..53a3b085
--- /dev/null
+++ b/server/tests/llm/test_media_blocks.py
@@ -0,0 +1,210 @@
+"""Image-block contracts: never-bytes codec rule, llm_media parsing,
+capability-gated hydration, and the Anthropic tool_result wire shape."""
+
+from __future__ import annotations
+
+import io
+from pathlib import Path
+
+import pytest
+from PIL import Image as PILImage
+
+from services.llm.media import (
+ LLM_MEDIA_MAX_PER_RESULT,
+ hydrate_image_blocks,
+ image_blocks_from_tool_result,
+ provider_supports_vision,
+)
+from services.llm.protocol import (
+ ContentBlock,
+ Message,
+ message_from_wire,
+ message_to_wire,
+)
+
+
+def _ref(path: str = "images/chart.png", mime: str = "image/png") -> dict:
+ return {
+ "kind": "image",
+ "path": path,
+ "workflow_id": "wf-media",
+ "filename": Path(path).name,
+ "mime_type": mime,
+ "size_bytes": 1234,
+ }
+
+
+def _ref_block(**overrides) -> ContentBlock:
+ source = {"kind": "file_ref", "ref": _ref(), "detail": "auto"}
+ source.update(overrides)
+ return ContentBlock(type="image", source=source)
+
+
+class TestWireCodec:
+ def test_ref_source_round_trips(self):
+ message = Message(role="tool", content="{}", tool_call_id="t1")
+ message.blocks.append(_ref_block())
+ wire = message_to_wire(message)
+ restored = message_from_wire(wire)
+ images = [b for b in restored.blocks if b.type == "image"]
+ assert images[0].source["kind"] == "file_ref"
+ assert images[0].source["ref"]["path"] == "images/chart.png"
+
+ def test_bytes_source_refuses_serialization(self):
+ message = Message(role="tool", content="{}", tool_call_id="t1")
+ message.blocks.append(
+ ContentBlock(
+ type="image",
+ source={"kind": "bytes", "media_type": "image/png", "data_b64": "AAAA"},
+ )
+ )
+ with pytest.raises(ValueError, match="durable state"):
+ message_to_wire(message)
+
+ def test_legacy_wire_without_source_decodes(self):
+ wire = message_to_wire(Message(role="user", content="hi"))
+ for block in wire["blocks"]:
+ block.pop("source", None)
+ assert message_from_wire(wire).content == "hi"
+
+
+class TestLlmMediaContract:
+ def test_valid_entry_becomes_ref_block(self):
+ blocks = image_blocks_from_tool_result(
+ {"llm_media": [{"ref": _ref(), "detail": "high"}]}
+ )
+ assert len(blocks) == 1
+ assert blocks[0].source["detail"] == "high"
+
+ def test_invalid_entries_skipped(self):
+ result = {
+ "llm_media": [
+ {"ref": _ref(mime="application/pdf")}, # mime not allowed
+ {"ref": {**_ref(), "workflow_id": ""}}, # scope missing
+ {"no_ref": True},
+ "not-a-dict",
+ ]
+ }
+ assert image_blocks_from_tool_result(result) == []
+
+ def test_cap_enforced(self):
+ entries = [{"ref": _ref(path=f"i/{n}.png")} for n in range(20)]
+ blocks = image_blocks_from_tool_result({"llm_media": entries})
+ assert len(blocks) == LLM_MEDIA_MAX_PER_RESULT
+
+ def test_non_dict_results_ignored(self):
+ assert image_blocks_from_tool_result("text") == []
+ assert image_blocks_from_tool_result({"result": 1}) == []
+
+
+class TestHydration:
+ async def test_identity_when_no_images(self):
+ messages = [Message(role="user", content="hi")]
+ hydrated = await hydrate_image_blocks(
+ messages, provider="anthropic", model="claude-opus-5"
+ )
+ assert hydrated[0] is messages[0]
+
+ async def test_incapable_provider_degrades_to_text(self):
+ message = Message(role="tool", content="{}", tool_call_id="t1")
+ message.blocks.append(_ref_block())
+ hydrated = await hydrate_image_blocks(
+ [message], provider="deepseek", model="deepseek-chat"
+ )
+ texts = [b.text for b in hydrated[0].blocks if b.type == "text"]
+ assert any("cannot view images" in t for t in texts)
+ assert all(b.type != "image" for b in hydrated[0].blocks)
+ # Original untouched — refs remain the durable form.
+ assert any(b.type == "image" for b in message.blocks)
+
+ async def test_capable_provider_gets_bytes(self, tmp_path, monkeypatch):
+ workspace = tmp_path / "ws"
+ (workspace / "images").mkdir(parents=True)
+ PILImage.new("RGB", (64, 64)).save(workspace / "images" / "chart.png")
+
+ async def fake_root(workflow_id, database, **kwargs):
+ return workspace
+
+ monkeypatch.setattr(
+ "services.workspace_locator.resolve_workspace_root", fake_root
+ )
+ monkeypatch.setattr(
+ "services.plugin.deps.get_database", lambda: object()
+ )
+ message = Message(role="tool", content="{}", tool_call_id="t1")
+ message.blocks.append(_ref_block())
+ hydrated = await hydrate_image_blocks(
+ [message], provider="anthropic", model="claude-opus-5"
+ )
+ images = [b for b in hydrated[0].blocks if b.type == "image"]
+ assert images[0].source["kind"] == "bytes"
+ assert images[0].source["media_type"] in ("image/jpeg", "image/png")
+ assert images[0].source["data_b64"]
+ # The hydrated copy must never serialize.
+ with pytest.raises(ValueError):
+ message_to_wire(hydrated[0])
+ # And the original still does.
+ assert message_to_wire(message)
+
+ async def test_missing_file_degrades_to_placeholder(self, monkeypatch):
+ async def fake_root(workflow_id, database, **kwargs):
+ return Path("Z:/nonexistent")
+
+ monkeypatch.setattr(
+ "services.workspace_locator.resolve_workspace_root", fake_root
+ )
+ monkeypatch.setattr(
+ "services.plugin.deps.get_database", lambda: object()
+ )
+ message = Message(role="tool", content="{}", tool_call_id="t1")
+ message.blocks.append(_ref_block())
+ hydrated = await hydrate_image_blocks(
+ [message], provider="anthropic", model="claude-opus-5"
+ )
+ texts = [b.text for b in hydrated[0].blocks if b.type == "text"]
+ assert any("unavailable" in t for t in texts)
+
+
+class TestCapabilityGate:
+ def test_anthropic_enabled(self):
+ # The only provider whose request encoder renders image blocks today.
+ assert provider_supports_vision("anthropic") is True
+
+ def test_encoderless_providers_disabled(self):
+ # openai/gemini stay off until their encoders render image blocks —
+ # enabled-without-encoder would hydrate bytes that get dropped.
+ for provider in ("openai", "gemini", "deepseek", "groq", "nope"):
+ assert provider_supports_vision(provider) is False
+
+
+class TestAnthropicToolResultShape:
+ def test_hydrated_image_rides_tool_result(self):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ message = Message(role="tool", content='{"ok":1}', tool_call_id="t1")
+ message.blocks.append(
+ ContentBlock(
+ type="image",
+ source={
+ "kind": "bytes",
+ "media_type": "image/jpeg",
+ "data_b64": "QUJD",
+ },
+ )
+ )
+ content = AnthropicProvider._tool_result_content(message)
+ assert content[0] == {"type": "text", "text": '{"ok":1}'}
+ assert content[1] == {
+ "type": "image",
+ "source": {
+ "type": "base64",
+ "media_type": "image/jpeg",
+ "data": "QUJD",
+ },
+ }
+
+ def test_plain_tool_result_stays_string(self):
+ from services.llm.providers.anthropic import AnthropicProvider
+
+ message = Message(role="tool", content='{"ok":1}', tool_call_id="t1")
+ assert AnthropicProvider._tool_result_content(message) == '{"ok":1}'
diff --git a/server/tests/nodes/test_data_source_node.py b/server/tests/nodes/test_data_source_node.py
new file mode 100644
index 00000000..5811366e
--- /dev/null
+++ b/server/tests/nodes/test_data_source_node.py
@@ -0,0 +1,479 @@
+"""Contracts for the Data node: spec invariants, locked ToolInput, path
+security across both namespaces, bounded read tiers, write safety, and the
+no-host-path-leak rule for external mounts."""
+
+from __future__ import annotations
+
+import importlib.util
+import json
+import re
+import sys
+import uuid
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+from pydantic import ValidationError
+
+from nodes.tool.data_source import (
+ DataSourceNode,
+ DataToolInput,
+ DataToolParams,
+ WS_HANDLERS,
+)
+from nodes.tool.data_source._paths import resolve_data_path, split_mount_path
+from services.plugin import NodeContext, NodeUserError
+
+def _ctx(tmp_path: Path, **overrides) -> NodeContext:
+ defaults = dict(
+ node_id="data-1",
+ node_type="dataSource",
+ workflow_id="wf-data-test",
+ workspace_dir=str(tmp_path / "workspace"),
+ )
+ defaults.update(overrides)
+ ctx = NodeContext(**defaults)
+ Path(ctx.workspace_dir).mkdir(parents=True, exist_ok=True)
+ return ctx
+
+
+HOST_PATH_PATTERN = re.compile(r"[A-Za-z]:[\\/]|/home/|/Users/")
+
+
+def _assert_no_host_paths(result: dict, *roots: Path) -> None:
+ serialized = json.dumps(result, default=str)
+ for root in roots:
+ assert str(root) not in serialized
+ assert root.as_posix() not in serialized
+ # url fields legitimately contain "/api/..."; strip them before the
+ # generic drive/home scan.
+ scrubbed = re.sub(r"/api/workspace/[^\"]*", "", serialized)
+ assert not HOST_PATH_PATTERN.search(scrubbed), scrubbed[:500]
+
+
+class TestSpecInvariants:
+ def test_node_identity(self):
+ assert DataSourceNode.type == "dataSource"
+ assert DataSourceNode.component_kind == "tool"
+ assert DataSourceNode.group == ("tool",)
+ assert DataSourceNode.tool_name == "data"
+ assert DataSourceNode.tool_schema_locked is True
+ assert DataSourceNode.Params is DataToolParams
+ assert DataSourceNode.ToolInput is DataToolInput
+ assert DataSourceNode.server_controlled_fields == frozenset({"mounts"})
+
+ def test_ui_hints(self):
+ hints = DataSourceNode.ui_hints
+ assert hints["isToolPanel"] is True
+ assert hints["isDataPanel"] is True
+ assert hints["hideRunButton"] is True
+
+ def test_handles(self):
+ assert DataSourceNode.handles == (
+ {
+ "name": "output-tool",
+ "kind": "output",
+ "position": "top",
+ "label": "Data",
+ "role": "tools",
+ },
+ )
+
+ def test_llm_schema_hides_config_and_has_no_delete(self):
+ schema = DataSourceNode.as_tool_schema()
+ assert schema["name"] == "data"
+ properties = schema["parameters"]["properties"]
+ assert "mounts" not in properties
+ operations = properties["operation"]["enum"]
+ assert "delete" not in operations
+ assert "remove" not in operations
+ assert set(operations) == {
+ "list",
+ "read",
+ "search",
+ "metadata",
+ "write",
+ "append",
+ "copy_to_workspace",
+ }
+
+ def test_operations_registry(self):
+ assert set(DataSourceNode._operations) == {"data"}
+
+ def test_ws_handlers_registered(self):
+ from services.ws_handler_registry import get_ws_handlers
+
+ assert set(WS_HANDLERS).issubset(set(get_ws_handlers()))
+
+
+class TestToolInputValidation:
+ def test_per_operation_required_fields(self):
+ for operation, payload in (
+ ("read", {}),
+ ("metadata", {}),
+ ("write", {"path": "a.txt"}),
+ ("append", {"path": "a.txt"}),
+ ("search", {}),
+ ("copy_to_workspace", {}),
+ ):
+ with pytest.raises(ValidationError):
+ DataToolInput(operation=operation, **payload)
+
+ def test_extra_fields_forbidden(self):
+ with pytest.raises(ValidationError):
+ DataToolInput(operation="list", namespace="sneaky")
+
+ def test_params_coerce_blank_and_json_strings(self):
+ assert DataToolParams.model_validate({"mounts": ""}).mounts == []
+ assert DataToolParams.model_validate(
+ {"mounts": '["docs"]'}
+ ).mounts == ["docs"]
+ assert DataToolParams.model_validate({}).mounts == []
+
+
+class TestPathSecurity:
+ async def test_traversal_rejected(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ for hostile in ("../secrets.txt", "..", "a/../../b", "~/x"):
+ with pytest.raises(NodeUserError):
+ await resolve_data_path(ctx, [], hostile)
+
+ async def test_drive_prefixed_rejected(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ with pytest.raises(NodeUserError):
+ await resolve_data_path(ctx, [], "C:/Windows/system32")
+
+ async def test_mnt_prefix_reserved(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ with pytest.raises(NodeUserError, match="not enabled|needs a name"):
+ await resolve_data_path(ctx, [], "mnt/anything/file.txt")
+
+ async def test_mount_traversal_name_rejected(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ # 'mnt/../..' parses as mount name '..' which can never be enabled.
+ with pytest.raises(NodeUserError):
+ await resolve_data_path(ctx, ["docs"], "mnt/../../etc/passwd")
+
+ def test_split_mount_path(self):
+ assert split_mount_path("mnt/docs/a/b.txt") == ("docs", "a/b.txt")
+ assert split_mount_path("mnt/docs") == ("docs", "")
+ assert split_mount_path("reports/q3.csv") is None
+ assert split_mount_path("") is None
+
+
+class TestWorkspaceOps:
+ async def test_write_read_text_roundtrip(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ write = await node._op_write(
+ ctx,
+ [],
+ DataToolInput(
+ operation="write", path="notes/hello.txt", content="line1\nline2"
+ ),
+ )
+ assert write["created"] is True
+ assert write["source"] == "workspace"
+
+ read = await node._op_read(
+ ctx, [], DataToolInput(operation="read", path="notes/hello.txt")
+ )
+ assert read["type"] == "text"
+ assert read["text"] == "line1\nline2"
+ assert read["lines_total"] == 2
+ assert read["encoding"] == "utf-8"
+ _assert_no_host_paths(read, tmp_path)
+
+ async def test_append(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ await node._op_write(
+ ctx, [], DataToolInput(operation="write", path="log.txt", content="a")
+ )
+ result = await node._op_append(
+ ctx, [], DataToolInput(operation="append", path="log.txt", content="b")
+ )
+ assert result["appended"] is True
+ assert (Path(ctx.workspace_dir) / "log.txt").read_text() == "ab"
+
+ async def test_csv_tier(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ (Path(ctx.workspace_dir) / "t.csv").write_text(
+ "name,age\nada,36\ngrace,45\n"
+ )
+ node = DataSourceNode()
+ result = await node._op_read(
+ ctx, [], DataToolInput(operation="read", path="t.csv")
+ )
+ assert result["type"] == "csv"
+ assert result["columns"] == ["name", "age"]
+ assert result["rows"] == [["ada", "36"], ["grace", "45"]]
+ assert result["rows_total"] == 2
+
+ async def test_json_tier_prunes_depth(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ nested: dict = {"leaf": 1}
+ for _ in range(12):
+ nested = {"child": nested}
+ (Path(ctx.workspace_dir) / "deep.json").write_text(json.dumps(nested))
+ node = DataSourceNode()
+ result = await node._op_read(
+ ctx, [], DataToolInput(operation="read", path="deep.json")
+ )
+ assert result["type"] == "json"
+ assert result["pruned_paths"]
+ assert result["truncated"] is True
+
+ async def test_image_tier_metadata_only(self, tmp_path):
+ from PIL import Image
+
+ ctx = _ctx(tmp_path)
+ Image.new("RGB", (12, 8)).save(Path(ctx.workspace_dir) / "pic.png")
+ node = DataSourceNode()
+ result = await node._op_read(
+ ctx, [], DataToolInput(operation="read", path="pic.png")
+ )
+ assert result["type"] == "image"
+ assert result["image"] == {
+ "width": 12,
+ "height": 8,
+ "format": "png",
+ "mode": "RGB",
+ }
+ assert result["ref"]["kind"] == "file"
+ serialized = json.dumps(result)
+ assert "base64" not in serialized
+ _assert_no_host_paths(result, tmp_path)
+
+ async def test_binary_tier_never_bytes(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ payload = bytes(range(256)) * 64
+ (Path(ctx.workspace_dir) / "blob.bin").write_bytes(payload)
+ node = DataSourceNode()
+ result = await node._op_read(
+ ctx, [], DataToolInput(operation="read", path="blob.bin")
+ )
+ assert result["type"] == "binary"
+ assert result["sha256"]
+ assert result["size_bytes"] == len(payload)
+ assert "content" not in result
+ assert len(json.dumps(result)) < 2_000
+
+ async def test_search_workspace(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ root = Path(ctx.workspace_dir)
+ (root / "reports").mkdir()
+ (root / "reports" / "q3-summary.txt").write_text("x")
+ (root / "other.txt").write_text("y")
+ node = DataSourceNode()
+ result = await node._op_search(
+ ctx, [], DataToolInput(operation="search", pattern="summary")
+ )
+ assert result["count"] == 1
+ assert result["entries"][0]["path"] == "reports/q3-summary.txt"
+
+ async def test_metadata(self, tmp_path):
+ ctx = _ctx(tmp_path)
+ (Path(ctx.workspace_dir) / "m.txt").write_text("hello")
+ node = DataSourceNode()
+ result = await node._op_metadata(
+ ctx, [], DataToolInput(operation="metadata", path="m.txt")
+ )
+ assert result["size_bytes"] == 5
+ assert result["sha256"]
+ assert result["type"] == "text"
+
+
+async def mount_env(tmp_path, monkeypatch):
+ """Real private DB + one read-only and one writable mount."""
+ module_name = f"tests._real_data_node_database_{uuid.uuid4().hex}"
+ spec = importlib.util.spec_from_file_location(
+ module_name,
+ Path(__file__).resolve().parents[2] / "core" / "database.py",
+ )
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules[module_name] = module
+ spec.loader.exec_module(module)
+
+ db_path = Path.cwd() / f".data-node-{uuid.uuid4().hex}.db"
+ database = module.Database(
+ SimpleNamespace(
+ database_url=f"sqlite+aiosqlite:///{db_path.as_posix()}",
+ database_echo=False,
+ database_pool_size=5,
+ database_max_overflow=5,
+ )
+ )
+ await database.startup()
+ monkeypatch.setattr(
+ "services.plugin.deps.get_database", lambda: database
+ )
+
+ from services.data.mount_store import DataMountStore
+
+ store = DataMountStore(database)
+ readonly_dir = tmp_path / "external-ro"
+ writable_dir = tmp_path / "external-rw"
+ readonly_dir.mkdir()
+ writable_dir.mkdir()
+ (readonly_dir / "doc.txt").write_text("external content")
+ await store.add_mount("owner", name="ro", root_path=str(readonly_dir))
+ await store.add_mount(
+ "owner", name="rw", root_path=str(writable_dir), writable=True
+ )
+ try:
+ yield SimpleNamespace(
+ database=database,
+ readonly_dir=readonly_dir,
+ writable_dir=writable_dir,
+ )
+ finally:
+ await database.shutdown()
+ sys.modules.pop(module_name, None)
+ for candidate in (
+ db_path,
+ Path(f"{db_path}-wal"),
+ Path(f"{db_path}-shm"),
+ ):
+ candidate.unlink(missing_ok=True)
+
+
+class TestMountOps:
+ async def test_read_from_mount_no_host_path_leak(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ result = await node._op_read(
+ ctx,
+ ["ro", "rw"],
+ DataToolInput(operation="read", path="mnt/ro/doc.txt"),
+ )
+ assert result["text"] == "external content"
+ assert result["source"] == "mount"
+ assert result["mount"] == "ro"
+ _assert_no_host_paths(result, tmp_path, mount_env.readonly_dir)
+
+ async def test_write_to_readonly_mount_refused(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ with pytest.raises(NodeUserError, match="read-only"):
+ await node._op_write(
+ ctx,
+ ["ro"],
+ DataToolInput(
+ operation="write", path="mnt/ro/new.txt", content="x"
+ ),
+ )
+
+ async def test_write_to_writable_mount(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ result = await node._op_write(
+ ctx,
+ ["rw"],
+ DataToolInput(
+ operation="write", path="mnt/rw/out.txt", content="hello"
+ ),
+ )
+ assert result["created"] is True
+ assert (mount_env.writable_dir / "out.txt").read_text() == "hello"
+
+ async def test_disabled_mount_refused(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ with pytest.raises(NodeUserError, match="not enabled"):
+ await node._op_read(
+ ctx,
+ ["rw"], # 'ro' exists globally but is not enabled here
+ DataToolInput(operation="read", path="mnt/ro/doc.txt"),
+ )
+
+ async def test_revoked_mount_dies_at_call_time(self, tmp_path, mount_env):
+ from services.data.mount_store import DataMountStore
+
+ await DataMountStore(mount_env.database).remove_mount(
+ "owner", name="ro"
+ )
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ with pytest.raises(NodeUserError, match="no longer defined"):
+ await node._op_read(
+ ctx,
+ ["ro"],
+ DataToolInput(operation="read", path="mnt/ro/doc.txt"),
+ )
+
+ async def test_mount_symlink_escape_rejected(self, tmp_path, mount_env):
+ try:
+ (mount_env.readonly_dir / "escape").symlink_to(
+ tmp_path, target_is_directory=True
+ )
+ except OSError:
+ pytest.skip("symlinks unavailable on this platform")
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ with pytest.raises(NodeUserError):
+ await node._op_read(
+ ctx,
+ ["ro"],
+ DataToolInput(
+ operation="read", path="mnt/ro/escape/workspace/x.txt"
+ ),
+ )
+
+ async def test_copy_to_workspace_never_overwrites(
+ self, tmp_path, mount_env
+ ):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ args = DataToolInput(
+ operation="copy_to_workspace", path="mnt/ro/doc.txt"
+ )
+ first = await node._op_copy(ctx, ["ro"], args)
+ second = await node._op_copy(ctx, ["ro"], args)
+ assert first["path"] == "imports/doc.txt"
+ assert second["path"] == "imports/doc-1.txt"
+ assert first["ref"]["kind"] == "file"
+ assert first["copied_from"] == "mnt/ro/doc.txt"
+ workspace = Path(ctx.workspace_dir)
+ assert (workspace / "imports" / "doc.txt").read_text() == (
+ "external content"
+ )
+ _assert_no_host_paths(first, mount_env.readonly_dir)
+
+ async def test_list_root_reports_mounts(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ result = await node._op_list(
+ ctx, ["ro", "rw"], DataToolInput(operation="list")
+ )
+ mounts = {row["name"]: row for row in result["mounts"]}
+ assert mounts["ro"]["writable"] is False
+ assert mounts["rw"]["writable"] is True
+ assert mounts["ro"]["available"] is True
+ assert mounts["ro"]["path"] == "mnt/ro"
+
+ async def test_list_mount_dir(self, tmp_path, mount_env):
+ ctx = _ctx(tmp_path)
+ node = DataSourceNode()
+ result = await node._op_list(
+ ctx, ["ro"], DataToolInput(operation="list", path="mnt/ro")
+ )
+ assert result["source"] == "mount"
+ names = [row["name"] for row in result["entries"]]
+ assert "doc.txt" in names
+ locations = [row["location"] for row in result["entries"]]
+ assert all(loc.startswith("mnt/ro/") for loc in locations)
+ _assert_no_host_paths(result, mount_env.readonly_dir)
+
+
+class TestHandlerSecurity:
+ async def test_internal_socket_denied(self):
+ internal = SimpleNamespace(
+ scope={"path": "/ws/internal"}, state=None
+ )
+ for name, handler in WS_HANDLERS.items():
+ result = await handler({}, internal)
+ assert result.get("success") is False, name
+ assert "authenticated" in str(result.get("error", "")), name
diff --git a/server/tests/nodes/test_vision_analyze.py b/server/tests/nodes/test_vision_analyze.py
new file mode 100644
index 00000000..958b08da
--- /dev/null
+++ b/server/tests/nodes/test_vision_analyze.py
@@ -0,0 +1,286 @@
+"""Contracts for the vision-delegate tool: budget fitting math, locked
+schema, and the exact outgoing request per provider (the speech-test idiom
+— assert the request, not the parsed result)."""
+
+from __future__ import annotations
+
+import io
+import json
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+from PIL import Image
+from pydantic import ValidationError
+
+from nodes.vision.vision_analyze import (
+ VisionAnalyzeNode,
+ VisionAnalyzeParams,
+ VisionAnalyzeToolInput,
+)
+from services.media.image_fit import (
+ IMAGE_BUDGET_TOKENS,
+ budget_to_pixels,
+ fit_image_bytes,
+ smart_resize,
+)
+from services.plugin import NodeContext, NodeUserError
+
+
+class TestSmartResize:
+ def test_aspect_ratio_preserved_and_snapped(self):
+ height, width = smart_resize(
+ 2160, 3840, min_pixels=256 * 28 * 28, max_pixels=1024 * 28 * 28
+ )
+ assert height % 28 == 0 and width % 28 == 0
+ assert height * width <= 1024 * 28 * 28 * 1.05
+ original_ratio = 3840 / 2160
+ assert abs((width / height) - original_ratio) / original_ratio < 0.1
+
+ def test_tiny_image_scales_up(self):
+ height, width = smart_resize(
+ 10, 10, min_pixels=256 * 28 * 28, max_pixels=1024 * 28 * 28
+ )
+ assert height * width >= 256 * 28 * 28 * 0.9
+
+ def test_in_window_image_only_snaps(self):
+ height, width = smart_resize(
+ 560, 560, min_pixels=28 * 28, max_pixels=2048 * 28 * 28
+ )
+ assert (height, width) == (560, 560)
+
+ def test_invalid_dims_raise(self):
+ with pytest.raises(ValueError):
+ smart_resize(0, 100, min_pixels=1, max_pixels=100)
+
+
+class TestFitImageBytes:
+ @staticmethod
+ def _png(size, mode="RGB") -> bytes:
+ buffer = io.BytesIO()
+ Image.new(mode, size).save(buffer, format="PNG")
+ return buffer.getvalue()
+
+ def test_opaque_becomes_bounded_jpeg(self):
+ fitted, mime, (width, height) = fit_image_bytes(
+ self._png((4000, 3000)), budget="normal"
+ )
+ assert mime == "image/jpeg"
+ assert width % 28 == 0 and height % 28 == 0
+ assert width * height <= budget_to_pixels("normal") * 1.05
+ assert len(fitted) < 2 * 1024 * 1024
+
+ def test_alpha_stays_png(self):
+ _, mime, _ = fit_image_bytes(self._png((100, 100), "RGBA"))
+ assert mime == "image/png"
+
+ def test_budgets_are_ordered(self):
+ assert (
+ IMAGE_BUDGET_TOKENS["small"]
+ < IMAGE_BUDGET_TOKENS["normal"]
+ < IMAGE_BUDGET_TOKENS["large"]
+ )
+
+
+class TestSpecInvariants:
+ def test_locked_split_schema(self):
+ assert VisionAnalyzeNode.tool_schema_locked is True
+ assert VisionAnalyzeNode.tool_name == "vision"
+ assert VisionAnalyzeNode.Params is VisionAnalyzeParams
+ assert VisionAnalyzeNode.ToolInput is VisionAnalyzeToolInput
+ schema = VisionAnalyzeNode.as_tool_schema()
+ assert schema["name"] == "vision"
+ properties = schema["parameters"]["properties"]
+ assert "provider" not in properties
+ assert "vision_model" not in properties
+ assert set(properties["operation"]["enum"]) == {
+ "describe",
+ "extract_text",
+ }
+
+ def test_tool_input_validation(self):
+ with pytest.raises(ValidationError):
+ VisionAnalyzeToolInput(operation="describe", image="")
+ with pytest.raises(ValidationError):
+ VisionAnalyzeToolInput(operation="describe", image="a.png", extra=1)
+
+ def test_no_model_field_name(self):
+ # Reserved sibling-name magic in ParameterRenderer wipes a field
+ # literally named `model` next to `provider`.
+ assert "model" not in VisionAnalyzeParams.model_fields
+ assert "vision_model" in VisionAnalyzeParams.model_fields
+
+
+def _ctx(tmp_path: Path) -> NodeContext:
+ workspace = tmp_path / "workspace"
+ workspace.mkdir(parents=True, exist_ok=True)
+ Image.new("RGB", (640, 480), color=(200, 40, 40)).save(
+ workspace / "chart.png"
+ )
+ return NodeContext(
+ node_id="vision-1",
+ node_type="visionAnalyze",
+ workflow_id="wf-vision",
+ workspace_dir=str(workspace),
+ )
+
+
+def no_api_key_lookup(monkeypatch):
+ async def fake_key(ctx, provider):
+ return "test-key"
+
+ monkeypatch.setattr("nodes.vision.vision_analyze._api_key", fake_key)
+
+
+class TestProviderRequestShapes:
+ async def test_openai_request_shape(
+ self, tmp_path, monkeypatch, no_api_key_lookup
+ ):
+ captured = {}
+
+ class FakeCompletions:
+ async def create(self, **kwargs):
+ captured.update(kwargs)
+ return SimpleNamespace(
+ choices=[
+ SimpleNamespace(
+ message=SimpleNamespace(content="a red chart")
+ )
+ ]
+ )
+
+ class FakeClient:
+ def __init__(self, api_key):
+ captured["api_key"] = api_key
+ self.chat = SimpleNamespace(completions=FakeCompletions())
+
+ monkeypatch.setattr("openai.AsyncOpenAI", FakeClient)
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ ctx.raw["_tool_config"] = VisionAnalyzeParams(provider="openai")
+ result = await node.vision(
+ ctx,
+ VisionAnalyzeToolInput(operation="describe", image="chart.png"),
+ )
+ assert result.text == "a red chart"
+ assert captured["api_key"] == "test-key"
+ content = captured["messages"][0]["content"]
+ assert content[0] == {
+ "type": "text",
+ "text": "Describe this image in detail.",
+ }
+ image_part = content[1]
+ assert image_part["type"] == "image_url"
+ assert image_part["image_url"]["detail"] == "auto"
+ assert image_part["image_url"]["url"].startswith(
+ "data:image/jpeg;base64,"
+ )
+ assert captured["max_completion_tokens"] == 1024
+ # The result carries text and metadata only — never image bytes.
+ serialized = json.dumps(result.model_dump(mode="json"))
+ assert "base64" not in serialized
+
+ async def test_anthropic_request_shape(
+ self, tmp_path, monkeypatch, no_api_key_lookup
+ ):
+ captured = {}
+
+ class FakeMessages:
+ async def create(self, **kwargs):
+ captured.update(kwargs)
+ return SimpleNamespace(
+ content=[SimpleNamespace(type="text", text="chart text")]
+ )
+
+ class FakeClient:
+ def __init__(self, api_key):
+ captured["api_key"] = api_key
+ self.messages = FakeMessages()
+
+ monkeypatch.setattr("anthropic.AsyncAnthropic", FakeClient)
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ ctx.raw["_tool_config"] = VisionAnalyzeParams(
+ provider="anthropic", vision_model="claude-sonnet-5"
+ )
+ result = await node.vision(
+ ctx,
+ VisionAnalyzeToolInput(
+ operation="extract_text", image="chart.png"
+ ),
+ )
+ assert result.text == "chart text"
+ assert result.vision_model == "claude-sonnet-5"
+ content = captured["messages"][0]["content"]
+ assert content[0]["type"] == "image"
+ assert content[0]["source"]["type"] == "base64"
+ assert content[0]["source"]["media_type"] == "image/jpeg"
+ assert "Extract all text" in content[1]["text"]
+
+ async def test_gemini_request_shape(
+ self, tmp_path, monkeypatch, no_api_key_lookup
+ ):
+ captured = {}
+
+ class FakeModels:
+ async def generate_content(self, **kwargs):
+ captured.update(kwargs)
+ return SimpleNamespace(text="gemini sees a chart")
+
+ class FakeClient:
+ def __init__(self, api_key):
+ captured["api_key"] = api_key
+ self.aio = SimpleNamespace(models=FakeModels())
+
+ monkeypatch.setattr("google.genai.Client", FakeClient)
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ ctx.raw["_tool_config"] = VisionAnalyzeParams(
+ provider="gemini", vision_model="gemini-3.6-flash"
+ )
+ result = await node.vision(
+ ctx,
+ VisionAnalyzeToolInput(
+ operation="describe",
+ image="chart.png",
+ question="What color is it?",
+ ),
+ )
+ assert result.text == "gemini sees a chart"
+ assert captured["model"] == "gemini-3.6-flash"
+ # Custom question wins over the canned prompt.
+ assert captured["contents"][1] == "What color is it?"
+
+
+class TestSafety:
+ async def test_traversal_rejected(self, tmp_path, no_api_key_lookup):
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ ctx.raw["_tool_config"] = VisionAnalyzeParams(provider="openai")
+ with pytest.raises(NodeUserError):
+ await node.vision(
+ ctx,
+ VisionAnalyzeToolInput(
+ operation="describe", image="../../credentials.db"
+ ),
+ )
+
+ async def test_non_image_rejected(self, tmp_path, no_api_key_lookup):
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ (Path(ctx.workspace_dir) / "notes.txt").write_text("hello")
+ ctx.raw["_tool_config"] = VisionAnalyzeParams(provider="openai")
+ with pytest.raises(NodeUserError, match="not a readable image"):
+ await node.vision(
+ ctx,
+ VisionAnalyzeToolInput(
+ operation="describe", image="notes.txt"
+ ),
+ )
+
+ async def test_direct_run_is_refused(self, tmp_path):
+ node = VisionAnalyzeNode()
+ ctx = _ctx(tmp_path)
+ with pytest.raises(NodeUserError, match="needs an image"):
+ await node.vision(ctx, VisionAnalyzeParams())
diff --git a/server/tests/services/data/test_mount_store.py b/server/tests/services/data/test_mount_store.py
new file mode 100644
index 00000000..5cea3ebd
--- /dev/null
+++ b/server/tests/services/data/test_mount_store.py
@@ -0,0 +1,174 @@
+"""Validation and CRUD contracts for the machine-wide mount allowlist."""
+
+from __future__ import annotations
+
+import importlib.util
+import os
+import sys
+import uuid
+from pathlib import Path
+from types import SimpleNamespace
+
+import pytest
+
+from services.data.mount_store import (
+ DataMountStore,
+ MountStoreError,
+ validate_mount_root,
+)
+
+
+async def mount_database():
+ # Root conftest stubs core.database for fast unit tests. Load the real
+ # implementation privately, matching test_tool_store.py.
+ module_name = f"tests._real_data_mount_database_{uuid.uuid4().hex}"
+ spec = importlib.util.spec_from_file_location(
+ module_name,
+ Path(__file__).resolve().parents[3] / "core" / "database.py",
+ )
+ assert spec is not None and spec.loader is not None
+ module = importlib.util.module_from_spec(spec)
+ sys.modules[module_name] = module
+ spec.loader.exec_module(module)
+
+ db_path = Path.cwd() / f".data-mount-{uuid.uuid4().hex}.db"
+ database = module.Database(
+ SimpleNamespace(
+ database_url=f"sqlite+aiosqlite:///{db_path.as_posix()}",
+ database_echo=False,
+ database_pool_size=5,
+ database_max_overflow=5,
+ )
+ )
+ await database.startup()
+ try:
+ yield database
+ finally:
+ await database.shutdown()
+ sys.modules.pop(module_name, None)
+ for candidate in (
+ db_path,
+ Path(f"{db_path}-wal"),
+ Path(f"{db_path}-shm"),
+ ):
+ candidate.unlink(missing_ok=True)
+
+
+class TestValidateMountRoot:
+ def test_relative_path_rejected(self):
+ with pytest.raises(MountStoreError, match="absolute"):
+ validate_mount_root("relative/folder", writable=False)
+
+ def test_missing_path_rejected(self, tmp_path):
+ with pytest.raises(MountStoreError, match="does not exist"):
+ validate_mount_root(str(tmp_path / "nope"), writable=False)
+
+ def test_file_rejected(self, tmp_path):
+ target = tmp_path / "file.txt"
+ target.write_text("x")
+ with pytest.raises(MountStoreError, match="directory"):
+ validate_mount_root(str(target), writable=False)
+
+ def test_filesystem_root_rejected(self, tmp_path):
+ anchor = Path(tmp_path.anchor)
+ with pytest.raises(MountStoreError, match="filesystem root"):
+ validate_mount_root(str(anchor), writable=False)
+
+ def test_home_directory_rejected(self):
+ with pytest.raises(MountStoreError, match="home directory"):
+ validate_mount_root(str(Path.home()), writable=False)
+
+ def test_data_dir_overlap_rejected_both_directions(
+ self, tmp_path, monkeypatch
+ ):
+ protected = tmp_path / "container" / "opencompany-data"
+ protected.mkdir(parents=True)
+ monkeypatch.setattr(
+ "core.paths.opencompany_root", lambda: protected
+ )
+ inner = protected / "workspaces"
+ inner.mkdir()
+ with pytest.raises(MountStoreError, match="data directory"):
+ validate_mount_root(str(inner), writable=False)
+ with pytest.raises(MountStoreError, match="data directory"):
+ validate_mount_root(str(tmp_path / "container"), writable=False)
+
+ def test_writable_probe(self, tmp_path, monkeypatch):
+ target = tmp_path / "readonly"
+ target.mkdir()
+ real_access = os.access
+
+ def fake_access(path, mode):
+ if mode == os.W_OK and Path(path) == target.resolve():
+ return False
+ return real_access(path, mode)
+
+ monkeypatch.setattr(os, "access", fake_access)
+ with pytest.raises(MountStoreError, match="not writable"):
+ validate_mount_root(str(target), writable=True)
+ # Read-only save of the same folder is fine.
+ assert validate_mount_root(str(target), writable=False)
+
+ def test_valid_directory_resolves(self, tmp_path):
+ target = tmp_path / "docs"
+ target.mkdir()
+ assert validate_mount_root(str(target), writable=False) == target.resolve()
+
+
+class TestMountStoreCrud:
+ async def test_crud_roundtrip(self, mount_database, tmp_path):
+ store = DataMountStore(mount_database)
+ folder = tmp_path / "reports"
+ folder.mkdir()
+
+ added = await store.add_mount(
+ "owner", name="reports", root_path=str(folder), writable=False
+ )
+ assert added["name"] == "reports"
+ assert added["writable"] is False
+
+ assert [m["name"] for m in await store.list_mounts("owner")] == [
+ "reports"
+ ]
+ assert (await store.get_mount("owner", "reports")) is not None
+ # Other owners see nothing.
+ assert await store.list_mounts("intruder") == []
+
+ updated = await store.update_mount(
+ "owner", name="reports", writable=True
+ )
+ assert updated["writable"] is True
+
+ removed = await store.remove_mount("owner", name="reports")
+ assert removed["removed"] is True
+ assert await store.get_mount("owner", "reports") is None
+
+ async def test_duplicate_name_and_root_rejected(
+ self, mount_database, tmp_path
+ ):
+ store = DataMountStore(mount_database)
+ first = tmp_path / "one"
+ second = tmp_path / "two"
+ first.mkdir()
+ second.mkdir()
+ await store.add_mount("owner", name="docs", root_path=str(first))
+ with pytest.raises(MountStoreError, match="already exists"):
+ await store.add_mount("owner", name="docs", root_path=str(second))
+ with pytest.raises(MountStoreError, match="already mounted"):
+ await store.add_mount("owner", name="other", root_path=str(first))
+
+ async def test_bad_names_rejected(self, mount_database, tmp_path):
+ store = DataMountStore(mount_database)
+ folder = tmp_path / "ok"
+ folder.mkdir()
+ for bad in ("", "Has Spaces", "UPPER", "-leading", "a" * 65, "a/b"):
+ with pytest.raises(MountStoreError, match="Mount name"):
+ await store.add_mount("owner", name=bad, root_path=str(folder))
+
+ async def test_unknown_mount_operations(self, mount_database):
+ store = DataMountStore(mount_database)
+ with pytest.raises(MountStoreError, match="No mount"):
+ await store.update_mount("owner", name="ghost", writable=True)
+ with pytest.raises(MountStoreError, match="No mount"):
+ await store.remove_mount("owner", name="ghost")
diff --git a/server/tests/test_node_spec.py b/server/tests/test_node_spec.py
index 68b84004..bfa29747 100644
--- a/server/tests/test_node_spec.py
+++ b/server/tests/test_node_spec.py
@@ -927,6 +927,7 @@ class TestNodeSpecContractInvariants:
"isMemoryPanel",
"isMemoryToolPanel",
"isContextPanel",
+ "isDataPanel",
"isToolPanel",
"requiresContext",
"systemManaged",