Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

feat(mcp-host): add governed GFS workspace transfers

Alfredo Lopez9b1c29e88efe
diff --git a/deploy/overlays/minikube/configmaps/mcp-host-config.yaml b/deploy/overlays/minikube/configmaps/mcp-host-config.yaml
index 196297ce0..969179c18 100644
--- a/deploy/overlays/minikube/configmaps/mcp-host-config.yaml
+++ b/deploy/overlays/minikube/configmaps/mcp-host-config.yaml
@@ -22,6 +22,12 @@ data:
CLERUM_AGENT_MAX_TASK_DURATION: '86400000'
CLERUM_SHELL_TIMEOUT: '1500000'
CLERUM_TOOL_TIMEOUT: '1500000'
+ MCP_HOST_GFS_MAX_FILE_BYTES: '16777216'
+ MCP_HOST_GFS_DOWNLOAD_STORAGE_BYTES: '1073741824'
+ MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES: '268435456'
+ MCP_HOST_GFS_CALLER_DOWNLOAD_MAX_FILES: '8'
+ MCP_HOST_GFS_CALLER_DOWNLOAD_CONCURRENCY: '1'
+ MCP_HOST_GFS_DOWNLOAD_TTL_HOURS: '168'
CLERUM_MCP_TOOL_TIMEOUT_MS: '1500000'
CLERUM_MCP_TOOL_MAX_TOTAL_TIMEOUT_MS: '1500000'
CLERUM_TOOL_PROGRESS_INTERVAL_MS: '5000'
diff --git a/docs/operations/gfs-large-file-host-downloads.md b/docs/operations/gfs-large-file-host-downloads.md
new file mode 100644
index 000000000..87177ffd9
--- /dev/null
+++ b/docs/operations/gfs-large-file-host-downloads.md
@@ -0,0 +1,87 @@
+# GFS large-file Host downloads
+
+## Scope
+
+MCP Host separates GFS source admission from visual model input:
+
+- GFS admission defaults to **16 MiB** and can be raised to at most **200 MiB** only by an environment-approved deployment.
+- Text at or below **8 KiB** may be returned inline.
+- Larger admitted sources are transferred as governed workspace files. Their contents are not placed in the model conversation.
+- Visual projection is decided by the effective provider/model/transport profile after local validation. It never defines the generic GFS source limit.
+- PR #806's expected Grok visual profile is not materialized in this execution base: its installed attempt contract does not export visual limits. Grok GFS pixels therefore fail closed until that contract lands; when it does, its values are a provider visual profile, not a GFS transfer ceiling. PR #932's 11 MiB inline-document boundary is a separate Desktop composer policy and is also not a GFS transfer ceiling.
+
+This fixes the reported 3,836,961-byte CSV case: the file is admitted under 16 MiB and delivered to the caller workspace for approved local processing.
+
+## Effective limits
+
+| Environment variable | Default | Ceiling | Meaning |
+| --- | ---: | ---: | --- |
+| `MCP_HOST_GFS_MAX_FILE_BYTES` | `16777216` | `209715200` | Largest GFS source admitted by MCP Host. |
+| `MCP_HOST_GFS_DOWNLOAD_STORAGE_BYTES` | `1073741824` | deployment-approved | Aggregate retained download budget. |
+| `MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES` | `268435456` | aggregate budget | Per-caller retained download budget. |
+| `MCP_HOST_GFS_CALLER_DOWNLOAD_MAX_FILES` | `8` | `64` | Retained completed files per caller. |
+| `MCP_HOST_GFS_CALLER_DOWNLOAD_CONCURRENCY` | `1` | `2` | Simultaneous active transfers per caller. |
+| `MCP_HOST_GFS_DOWNLOAD_TTL_HOURS` | `168` | `8760` | Retention period for a completed copy. |
+
+All values are parsed as positive decimal byte/count/hour integers. Empty, fractional, negative, exponent, padded, partial, non-integer, and above-ceiling values fail startup. Per-caller storage cannot exceed aggregate storage.
+
+The supported end-to-end size is the minimum of the applicable Desktop/GFS upload limit, GFSC transport limits, MCP Host admission, workspace PVC capacity, download quotas, and provider-specific visual limits. Raising only MCP Host admission does not raise any other layer.
+
+## Delivery behavior
+
+For an admitted source above 8 KiB:
+
+1. MCP Host obtains bounded metadata and validates identity, version, file kind, and size.
+2. It streams the authorized `/content` response to a Host-managed `.gfs-downloads` directory under the caller workspace.
+3. It verifies the exact byte count and SHA-256, publishes the file with private modes, and returns a receipt containing source metadata, version, size, checksum, expiry, and a path relative to that caller workspace.
+4. The tool result instructs the model to use `shell_exec` for local processing, write outputs outside `.gfs-downloads`, and return only bounded counts, aggregates, errors, or samples.
+
+If a surface has no caller-bound workspace/download capability, an admitted source is reported as `workspace_delivery_unavailable`; MCP Host does not fall back to returning an oversized body.
+
+## Approval and local processing
+
+`shell_exec` remains the trust boundary:
+
+- The working directory and `HOME` are the caller workspace. A supplied allowlisted or dynamic `HOME` cannot relocate execution outside it.
+- Each command requires live user approval.
+- Combined retained command output is bounded to 1 MiB. Live progress is bounded to 64 KiB. Exceeding the output bound terminates the process group and returns a truthful `output_limit_exceeded` result.
+- A processing lease is acquired after approval and before the child process is created. It is released only after process-group termination and output settlement. If acquisition fails, no process starts; if release fails, the result is an error and cleanup protection remains for recovery.
+- Expiry does not remove a copy protected by an admitted execution lease. Recovery conservatively retains leases until their bounded deadline rather than assuming that a child died with the Host.
+
+Unix directory modes and random directory names do not provide cross-caller OS isolation when a Host shares one UID. Approved arbitrary shell access remains a documented Stage 1 residual; stronger executor isolation is separate Stage 2 work.
+
+Shell cleanup signals and waits for the detached process group before releasing a processing lease. Operating systems may reuse a process-group identifier after the original leader has been reaped; Stage 1 narrows that window by signaling immediately on leader close, but stronger executor identity is required to eliminate it.
+
+Generic workspace tools reject direct and symlink-resolved access to `.gfs-downloads` through `file_read`, `file_write`, memory read/write, list/tree, and search. This prevents accidental dumps; it does not revoke access from an explicitly approved shell command.
+
+## Retention, quotas, and recovery
+
+- Completed copies expire after the configured TTL, seven days by default.
+- Host and caller quotas account for partial and completed files. Unknown or corrupt accounting fails closed rather than reporting zero usage.
+- Only positively identified expired entries are deleted. A cleanup failure remains charged and is observable for recovery.
+- Startup reconciles the ledger and partial files before the capability is advertised.
+- Shutdown stops new admission, drains active work where possible, and leaves unproven lease/recovery state protected.
+
+## Metrics
+
+The global `/metrics` endpoint exposes fixed-cardinality instruments:
+
+- `clerum_gfs_download_admissions_total{outcome}`
+- `clerum_gfs_download_transfers_total{outcome}`
+- `clerum_gfs_download_duration_seconds`
+- `clerum_gfs_download_active`
+- `clerum_gfs_download_quota_total{scope,reason}`
+- `clerum_gfs_download_expiry_total{outcome}`
+- `clerum_gfs_shell_output_limits_total{outcome}`
+
+Labels contain bounded enums only. Caller, resource, download, command, path, correlation IDs, filenames, credentials, and raw output are prohibited as metric labels.
+
+## Rollout and rollback
+
+1. Apply the ConfigMap environment values through the supported HCC rollout. Updating a ConfigMap alone does not update an existing Pod.
+2. Verify the new values and `/metrics` endpoints from newly created Hosts.
+3. Before a writer-policy transition, disable new downloads, drain transfers and leases, and prove old Host/child processes cannot mutate the store.
+4. To roll back, restore the prior image/config without deleting `.gfs-downloads`. A rollback-compatible Host preserves the ledger and quota charges.
+5. Cleanup or compaction during rollback requires separate operator authorization and a usage receipt. Do not treat an image rollback as permission to erase retained copies.
+
+If limits are lowered, existing retained copies remain charged and new admissions are rejected until usage falls below the new policy.
diff --git a/mcp-host/package-lock.json b/mcp-host/package-lock.json
index e27c32cf6..660e6c099 100644
--- a/mcp-host/package-lock.json
+++ b/mcp-host/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "clerum-mcp-host",
- "version": "0.1.479",
+ "version": "0.1.480",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "clerum-mcp-host",
- "version": "0.1.479",
+ "version": "0.1.480",
"license": "SEE LICENSE IN LICENSE",
"dependencies": {
"@anthropic-ai/sdk": "^0.32.0",
diff --git a/mcp-host/package.json b/mcp-host/package.json
index 566043608..8efd07c91 100644
--- a/mcp-host/package.json
+++ b/mcp-host/package.json
@@ -1,6 +1,6 @@
{
"name": "clerum-mcp-host",
- "version": "0.1.479",
+ "version": "0.1.480",
"description": "MCP Host - reads Host CRD and provides LLM access via OpenAI/Claude",
"main": "dist/main.js",
"engines": {
diff --git a/mcp-host/src/__tests__/server.metrics.contract.test.ts b/mcp-host/src/__tests__/server.metrics.contract.test.ts
index b67bcc05c..b1887ba0e 100644
--- a/mcp-host/src/__tests__/server.metrics.contract.test.ts
+++ b/mcp-host/src/__tests__/server.metrics.contract.test.ts
@@ -15,6 +15,11 @@ describe('RPCServer metrics contract', () => {
expect(body).toContain('process_cpu_user_seconds_total')
expect(body).toContain('clerum_mcp_status_heartbeat_runs_total')
expect(body).toContain('clerum_mcp_status_heartbeat_in_flight')
+ expect(body).toContain('clerum_gfs_download_admissions_total')
+ expect(body).toContain('clerum_gfs_download_transfers_total')
+ expect(body).toContain('clerum_gfs_download_active')
+ expect(body).toContain('clerum_gfs_download_quota_total')
+ expect(body).toContain('clerum_gfs_download_expiry_total')
} finally {
await server.stop()
}
diff --git a/mcp-host/src/agent/__tests__/fileReferenceResolver.test.ts b/mcp-host/src/agent/__tests__/fileReferenceResolver.test.ts
index 308bc320e..eff995bfa 100644
--- a/mcp-host/src/agent/__tests__/fileReferenceResolver.test.ts
+++ b/mcp-host/src/agent/__tests__/fileReferenceResolver.test.ts
@@ -14,6 +14,7 @@ import {
classifyBytes,
} from '@clerum/gfs-interaction-policy'
import { type GfsRuntimeEnv, GfscHttpError, createGfscClient } from '../../internalTools/gfsClient'
+import { GFS_FILE_LIMITS } from '../../internalTools/gfsFilePolicy'
import { logger } from '../../logger'
import { VISUAL_INPUT_LIMITS } from '../../visualInput/policy'
import {
@@ -225,7 +226,7 @@ describe('resolveFileReferences (#666)', () => {
})
it('reports too_large above the per-file ceiling, and available at it', async () => {
- const limit = VISUAL_INPUT_LIMITS.fileBytes
+ const limit = GFS_FILE_LIMITS.maxFileBytes
const atLimit = await resolveFileReferences(
[gfsReference({ byteLength: limit })],
client(() => view({ bytes: limit }))
diff --git a/mcp-host/src/agent/__tests__/gfsExecutionCapability.test.ts b/mcp-host/src/agent/__tests__/gfsExecutionCapability.test.ts
new file mode 100644
index 000000000..038e75324
--- /dev/null
+++ b/mcp-host/src/agent/__tests__/gfsExecutionCapability.test.ts
@@ -0,0 +1,59 @@
+import { describe, expect, it } from 'vitest'
+import type { ApprovalConfig } from '../../core/extensions/approvalTypes'
+import type { GfsDownloadStore } from '../../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../../internalTools/gfsProcessingLease'
+import { gfsWorkspaceExecutionEnabled } from '../gfsExecutionCapability'
+
+const approvalConfig: ApprovalConfig = {
+ defaultPolicy: 'channel_users',
+ channels: {},
+}
+const leases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => ({
+ leaseId: 'lease-1',
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ }),
+ releaseProcessingLease: async () => undefined,
+}
+
+function input(overrides?: Partial<Parameters<typeof gfsWorkspaceExecutionEnabled>[0]>) {
+ return {
+ approvalEnabled: true,
+ source: 'channel' as const,
+ callerIdentity: 'caller-1',
+ store: {} as GfsDownloadStore,
+ callerWorkspacePath: '/workspace/users/caller-1',
+ processingLeaseProvider: leases,
+ approvalConfig,
+ ...overrides,
+ }
+}
+
+describe('gfsWorkspaceExecutionEnabled', () => {
+ it('enables the caller-bound workspace lane on an attended live-approval surface', () => {
+ expect(gfsWorkspaceExecutionEnabled(input())).toBe(true)
+ expect(
+ gfsWorkspaceExecutionEnabled(
+ input({
+ approvalConfig: { ...approvalConfig, tools: { shell_exec: true } },
+ })
+ )
+ ).toBe(true)
+ })
+
+ it.each([
+ ['approval disabled', { approvalEnabled: false }],
+ ['cron source', { source: 'cron' as const }],
+ ['internal source', { source: 'internal' as const }],
+ [
+ 'shell tool disabled',
+ { approvalConfig: { ...approvalConfig, tools: { shell_exec: false } } },
+ ],
+ ['missing caller identity', { callerIdentity: undefined }],
+ ['missing Host store', { store: undefined }],
+ ['missing caller workspace', { callerWorkspacePath: undefined }],
+ ['missing processing lease provider', { processingLeaseProvider: undefined }],
+ ])('rejects %s', (_name, overrides) => {
+ expect(gfsWorkspaceExecutionEnabled(input(overrides))).toBe(false)
+ })
+})
diff --git a/mcp-host/src/agent/__tests__/gfsReferenceSurfaces.test.ts b/mcp-host/src/agent/__tests__/gfsReferenceSurfaces.test.ts
new file mode 100644
index 000000000..46c65ee3f
--- /dev/null
+++ b/mcp-host/src/agent/__tests__/gfsReferenceSurfaces.test.ts
@@ -0,0 +1,78 @@
+import { describe, expect, it } from 'vitest'
+import { classifyGfsReferenceSurfaces } from '../gfsReferenceSurfaces'
+
+describe('classifyGfsReferenceSurfaces', () => {
+ it('keeps inline text independent from workspace delivery', () => {
+ expect(
+ classifyGfsReferenceSurfaces(
+ { byteLength: 8192, reader: 'text', modelImageInput: 'unsupported' },
+ { workspaceFile: false, localExecutor: false, visual: false }
+ )
+ ).toEqual({
+ metadata: true,
+ inline: true,
+ workspace: false,
+ localExecutor: false,
+ visual: false,
+ })
+ })
+
+ it('separates workspace delivery from approved local execution for the incident size', () => {
+ const reference = {
+ byteLength: 3_836_961,
+ reader: 'text' as const,
+ modelImageInput: 'unsupported' as const,
+ }
+ expect(
+ classifyGfsReferenceSurfaces(reference, {
+ workspaceFile: true,
+ localExecutor: false,
+ visual: false,
+ })
+ ).toMatchObject({ inline: false, workspace: true, localExecutor: false })
+ expect(
+ classifyGfsReferenceSurfaces(reference, {
+ workspaceFile: true,
+ localExecutor: true,
+ visual: false,
+ })
+ ).toMatchObject({ workspace: true, localExecutor: true })
+ })
+
+ it('requires producer image candidacy and runtime visual support', () => {
+ const reference = {
+ byteLength: 3_836_961,
+ reader: 'none' as const,
+ modelImageInput: 'candidate' as const,
+ }
+ expect(
+ classifyGfsReferenceSurfaces(reference, {
+ workspaceFile: true,
+ localExecutor: true,
+ visual: true,
+ })
+ ).toMatchObject({ visual: true })
+ expect(
+ classifyGfsReferenceSurfaces(reference, {
+ workspaceFile: true,
+ localExecutor: true,
+ visual: false,
+ })
+ ).toMatchObject({ visual: false })
+ expect(
+ classifyGfsReferenceSurfaces(
+ { ...reference, modelImageInput: 'unsupported' as const },
+ { workspaceFile: true, localExecutor: true, visual: true }
+ )
+ ).toMatchObject({ visual: false })
+ })
+
+ it('rejects workspace delivery above the generic GFS source limit', () => {
+ expect(
+ classifyGfsReferenceSurfaces(
+ { byteLength: 16 * 1024 * 1024 + 1, reader: 'text', modelImageInput: 'unsupported' },
+ { workspaceFile: true, localExecutor: true, visual: true }
+ )
+ ).toMatchObject({ inline: false, workspace: false, localExecutor: false })
+ })
+})
diff --git a/mcp-host/src/agent/__tests__/taskExecutor.gfsLiveApproval.integration.test.ts b/mcp-host/src/agent/__tests__/taskExecutor.gfsLiveApproval.integration.test.ts
new file mode 100644
index 000000000..9f6db489c
--- /dev/null
+++ b/mcp-host/src/agent/__tests__/taskExecutor.gfsLiveApproval.integration.test.ts
@@ -0,0 +1,146 @@
+import { afterEach, beforeEach, expect, it, vi } from 'vitest'
+import { config as appConfig } from '../../config'
+import { ConversationManager } from '../../core/conversation/conversation'
+import { SimpleEventEmitter } from '../../core/orchestration/eventEmitter'
+import { type ChatMessage, FinishReason, type ToolCall } from '../../core/types'
+import type { GfsDownloadStore } from '../../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../../internalTools/gfsProcessingLease'
+import { TaskLifecycle } from '../../lifecycle/taskLifecycle'
+import type { SingleTurnProvider } from '../../llm/types'
+import type { Task } from '../../queue/types'
+import { TaskExecutor, type TaskExecutorDeps, resolveTaskSessionKey } from '../taskExecutor'
+
+const savedConfig = {
+ enableApproval: appConfig.enableApproval,
+ codexToolPresentation: appConfig.codexToolPresentation,
+ dynamicToolsEnabled: appConfig.dynamicToolsEnabled,
+ promptCacheEnabled: appConfig.promptCacheEnabled,
+}
+
+const processingLeases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => ({
+ leaseId: 'gfs-live-approval-lease',
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ }),
+ releaseProcessingLease: async () => undefined,
+}
+
+beforeEach(() => {
+ Object.assign(appConfig, {
+ enableApproval: true,
+ codexToolPresentation: 'direct',
+ dynamicToolsEnabled: false,
+ promptCacheEnabled: false,
+ })
+})
+
+afterEach(() => {
+ Object.assign(appConfig, savedConfig)
+})
+
+it('requires live shell approval for GFS processing despite wildcard auto-approval', async () => {
+ const shellCall: ToolCall = {
+ id: 'gfs-shell-call',
+ name: 'shell_exec',
+ arguments: { command: 'printf bounded-result' },
+ }
+ const providerCalls: ChatMessage[][] = []
+ const provider: SingleTurnProvider = {
+ getProviderType: () => 'codex-subscription',
+ classifyError: () => ({
+ code: 'LLM_UNKNOWN_ERROR' as never,
+ retryable: false,
+ message: 'fixture classification is never used',
+ }),
+ completeSingleTurn: async () => {
+ throw new Error('Unexpected non-tool completion')
+ },
+ completeSingleTurnWithTools: async messages => {
+ providerCalls.push(messages)
+ if (providerCalls.length === 1) {
+ return {
+ content: null,
+ tool_calls: [shellCall],
+ usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 },
+ finish_reason: FinishReason.ToolUse,
+ }
+ }
+ throw new Error('A suspended shell command must not reach another provider round-trip')
+ },
+ }
+
+ const task: Task = {
+ id: 'gfs-live-shell-approval',
+ source: 'channel',
+ status: 'pending',
+ priority: 'normal',
+ createdAt: new Date(),
+ sourceMessage: {
+ sender: 'gfs-caller',
+ content: 'Process the governed file',
+ channelType: 'rpc',
+ channelId: 'gfs-live-approval-channel',
+ messageId: 'gfs-live-approval-message',
+ timestamp: new Date().toISOString(),
+ hostRef: 'gfs-live-approval-host',
+ },
+ conversationHistory: [
+ { role: 'user', content: 'Process the governed file', timestamp: new Date() },
+ ],
+ responseCallback: vi.fn(async () => undefined),
+ }
+
+ const conversationManager = new ConversationManager()
+ const conversation = await conversationManager.getOrCreate(resolveTaskSessionKey(task), {
+ userId: task.sourceMessage!.sender,
+ channelType: task.sourceMessage!.channelType,
+ channelId: task.sourceMessage!.channelId,
+ })
+ conversation.auto_approved_tools = new Set(['*'])
+
+ const lifecycle = new TaskLifecycle()
+ lifecycle.register(task)
+ const onApprovalNeeded = vi.fn()
+ const onComplete = vi.fn()
+ const onFail = vi.fn()
+ const deps: TaskExecutorDeps = {
+ conversationManager,
+ llmProvider: provider,
+ mcpManager: null,
+ workspaceService: undefined,
+ gfsDownloadStore: {} as GfsDownloadStore,
+ gfsCallerWorkspacePath: '/tmp/gfs-live-approval-caller',
+ gfsProcessingLeaseProvider: processingLeases,
+ modelName: 'fixture-model',
+ approvalConfig: {
+ defaultPolicy: 'channel_users',
+ channels: {},
+ },
+ config: {
+ maxTaskDuration: 300_000,
+ maxToolCallsPerTask: 10,
+ autoStart: true,
+ taskDelay: 0,
+ approvalTimeout: 300_000,
+ },
+ coreEvents: new SimpleEventEmitter(),
+ cronScheduler: null,
+ taskLifecycle: lifecycle,
+ onApprovalNeeded,
+ onComplete,
+ onFail,
+ dynamicEnvProvider: () => ({}),
+ }
+
+ const executor = new TaskExecutor(task, deps)
+ await executor.run()
+
+ expect(onFail).not.toHaveBeenCalled()
+ expect(executor.executorState).toBe('waiting_approval')
+ expect(executor.pendingApproval).toMatchObject({
+ tool_name: 'shell_exec',
+ tool_call_id: shellCall.id,
+ })
+ expect(onApprovalNeeded).toHaveBeenCalledTimes(1)
+ expect(providerCalls).toHaveLength(1)
+})
diff --git a/mcp-host/src/agent/__tests__/taskExecutor.gfsOutputBounds.test.ts b/mcp-host/src/agent/__tests__/taskExecutor.gfsOutputBounds.test.ts
new file mode 100644
index 000000000..6e65e49a8
--- /dev/null
+++ b/mcp-host/src/agent/__tests__/taskExecutor.gfsOutputBounds.test.ts
@@ -0,0 +1,365 @@
+import { afterEach, beforeEach, expect, it, vi } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { config as appConfig } from '../../config'
+import { ConversationManager } from '../../core/conversation/conversation'
+import { SimpleEventEmitter } from '../../core/orchestration/eventEmitter'
+import { NoopSafety } from '../../core/safety/__tests__/noopSafety'
+import { SpilloverStorage } from '../../core/spillover/storage'
+import { type ChatMessage, FinishReason, type ToolCall } from '../../core/types'
+import type { GfsDownloadStore } from '../../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../../internalTools/gfsProcessingLease'
+import { TaskLifecycle } from '../../lifecycle/taskLifecycle'
+import type { SingleTurnProvider } from '../../llm/types'
+import { SseProgressReporter, progressReporterRegistry } from '../../progress/sseProgressReporter'
+import type { Task } from '../../queue/types'
+import { TaskExecutor, type TaskExecutorDeps, resolveTaskSessionKey } from '../taskExecutor'
+
+const savedConfig = {
+ enableApproval: appConfig.enableApproval,
+ codexToolPresentation: appConfig.codexToolPresentation,
+ dynamicToolsEnabled: appConfig.dynamicToolsEnabled,
+ promptCacheEnabled: appConfig.promptCacheEnabled,
+ toolSpilloverThresholdBytes: appConfig.toolSpilloverThresholdBytes,
+}
+
+let callerWorkspace: string
+
+const processingLeases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => ({
+ leaseId: 'gfs-output-bounds-lease',
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ }),
+ releaseProcessingLease: async () => undefined,
+}
+
+beforeEach(() => {
+ callerWorkspace = fs.mkdtempSync(path.join(os.tmpdir(), 'gfs-output-bounds-'))
+ Object.assign(appConfig, {
+ enableApproval: true,
+ codexToolPresentation: 'direct',
+ dynamicToolsEnabled: false,
+ promptCacheEnabled: false,
+ })
+})
+
+afterEach(() => {
+ Object.assign(appConfig, savedConfig)
+ fs.rmSync(callerWorkspace, { recursive: true, force: true })
+})
+
+function makeScenario(options: {
+ taskId: string
+ command: string
+ thresholdBytes: number
+ spilloverStorage?: SpilloverStorage
+}) {
+ appConfig.toolSpilloverThresholdBytes = options.thresholdBytes
+ const shellCall: ToolCall = {
+ id: `${options.taskId}-shell-call`,
+ name: 'shell_exec',
+ arguments: { command: options.command },
+ }
+ const providerRequests: ChatMessage[][] = []
+ const provider: SingleTurnProvider = {
+ getProviderType: () => 'codex-subscription',
+ classifyError: () => ({
+ code: 'LLM_UNKNOWN_ERROR' as never,
+ retryable: false,
+ message: 'fixture classification is never used',
+ }),
+ completeSingleTurn: async () => {
+ throw new Error('Unexpected non-tool completion')
+ },
+ completeSingleTurnWithTools: async messages => {
+ providerRequests.push(messages)
+ if (providerRequests.length === 1) {
+ return {
+ content: null,
+ tool_calls: [shellCall],
+ usage: { input_tokens: 1, output_tokens: 1, total_tokens: 2 },
+ finish_reason: FinishReason.ToolUse,
+ }
+ }
+ return {
+ content: 'Processed with bounded output.',
+ tool_calls: null,
+ usage: { input_tokens: 2, output_tokens: 2, total_tokens: 4 },
+ finish_reason: FinishReason.Stop,
+ }
+ },
+ }
+
+ const task: Task = {
+ id: options.taskId,
+ source: 'channel',
+ status: 'pending',
+ priority: 'normal',
+ createdAt: new Date(),
+ sourceMessage: {
+ sender: 'gfs-caller',
+ content: 'Process the governed file',
+ channelType: 'rpc',
+ channelId: `${options.taskId}-channel`,
+ messageId: `${options.taskId}-message`,
+ timestamp: new Date().toISOString(),
+ hostRef: `${options.taskId}-host`,
+ },
+ conversationHistory: [
+ { role: 'user', content: 'Process the governed file', timestamp: new Date() },
+ ],
+ responseCallback: vi.fn(async () => undefined),
+ }
+
+ const conversationManager = new ConversationManager()
+ const sessionKey = resolveTaskSessionKey(task)
+ const lifecycle = new TaskLifecycle()
+ lifecycle.register(task)
+ const progressEvents: string[] = []
+ const reporter = new SseProgressReporter(task.id, lifecycle, NoopSafety)
+ reporter.subscribe(event => {
+ if (event.type === 'tool_progress' && event.data.outputPreview) {
+ progressEvents.push(JSON.stringify(event.data.outputPreview))
+ }
+ })
+ progressReporterRegistry.set(task.id, reporter)
+
+ const onApprovalNeeded = vi.fn()
+ const onComplete = vi.fn()
+ const onFail = vi.fn()
+ const deps: TaskExecutorDeps = {
+ conversationManager,
+ llmProvider: provider,
+ mcpManager: null,
+ workspaceService: undefined,
+ gfsDownloadStore: {} as GfsDownloadStore,
+ gfsCallerWorkspacePath: callerWorkspace,
+ gfsProcessingLeaseProvider: processingLeases,
+ modelName: 'fixture-model',
+ approvalConfig: {
+ defaultPolicy: 'channel_users',
+ channels: {},
+ },
+ config: {
+ maxTaskDuration: 300_000,
+ maxToolCallsPerTask: 10,
+ autoStart: true,
+ taskDelay: 0,
+ approvalTimeout: 300_000,
+ },
+ coreEvents: new SimpleEventEmitter(),
+ cronScheduler: null,
+ taskLifecycle: lifecycle,
+ onApprovalNeeded,
+ onComplete,
+ onFail,
+ dynamicEnvProvider: () => ({}),
+ spilloverStorage: options.spilloverStorage,
+ }
+
+ return {
+ shellCall,
+ providerRequests,
+ conversationManager,
+ sessionKey,
+ progressEvents,
+ create: () => new TaskExecutor(task, deps),
+ dispose: () => {
+ progressReporterRegistry.delete(task.id)
+ reporter.dispose()
+ },
+ run: async () => {
+ const executor = new TaskExecutor(task, deps)
+ await executor.run()
+ return executor
+ },
+ }
+}
+
+function finalToolMessage(requests: ChatMessage[][]): ChatMessage {
+ const message = requests.at(-1)?.find(item => item.role === 'tool')
+ expect(message).toBeDefined()
+ return message!
+}
+
+it('keeps failed GFS shell output bounded with spillover disabled', async () => {
+ const proofFile = 'gfs-output-proof.txt'
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-output-bounds',
+ thresholdBytes: 2 * 1024 * 1024,
+ command:
+ `node -e "require('fs').writeFileSync('${proofFile}','verified');` +
+ `process.stdout.write('x'.repeat(1536 * 1024))"`,
+ })
+
+ try {
+ const executor = await scenario.run()
+ expect(executor.executorState).toBe('waiting_approval')
+ expect(fs.existsSync(path.join(callerWorkspace, proofFile))).toBe(false)
+
+ await executor.resumeAfterApproval(false)
+
+ expect(executor.executorState).toBe('completed')
+ expect(fs.readFileSync(path.join(callerWorkspace, proofFile), 'utf8')).toBe('verified')
+ expect(scenario.providerRequests).toHaveLength(2)
+ expect(
+ Buffer.byteLength(JSON.stringify(finalToolMessage(scenario.providerRequests)), 'utf8')
+ ).toBeLessThanOrEqual(1024 * 1024)
+ expect(JSON.stringify(finalToolMessage(scenario.providerRequests))).toContain(
+ 'output_limit_exceeded'
+ )
+ expect(Buffer.byteLength(scenario.progressEvents.join(''), 'utf8')).toBeLessThanOrEqual(
+ 64 * 1024
+ )
+
+ const conversation = scenario.conversationManager.getStore().get(scenario.sessionKey)
+ expect(conversation).toBeDefined()
+ expect(Buffer.byteLength(JSON.stringify(conversation!.turns), 'utf8')).toBeLessThanOrEqual(
+ 1024 * 1024 + 64 * 1024
+ )
+ expect(JSON.stringify(conversation!.turns)).not.toContain('x'.repeat(8192))
+ } finally {
+ scenario.dispose()
+ }
+})
+
+it('replaces successful oversized GFS shell output with a spillover summary', async () => {
+ const storage = new SpilloverStorage({
+ workspacePath: callerWorkspace,
+ ttlMs: 300_000,
+ thresholdBytes: 8192,
+ gcIntervalMs: 0,
+ })
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-spillover-enabled',
+ thresholdBytes: 8192,
+ spilloverStorage: storage,
+ command: `node -e "process.stdout.write('y'.repeat(16384))"`,
+ })
+
+ try {
+ const executor = await scenario.run()
+ await executor.resumeAfterApproval(false)
+
+ expect(executor.executorState).toBe('completed')
+ expect(scenario.providerRequests).toHaveLength(2)
+ const toolMessage = finalToolMessage(scenario.providerRequests)
+ expect(toolMessage.content).toContain('"spillover_ref"')
+ expect(toolMessage.content).toContain('spillover://gfs-shell-spillover-enabled/')
+ expect(toolMessage.content).not.toContain('y'.repeat(401))
+ expect(Buffer.byteLength(toolMessage.content, 'utf8')).toBeLessThanOrEqual(4096)
+ } finally {
+ scenario.dispose()
+ }
+})
+
+it('falls back to bounded inline output when spillover persistence fails', async () => {
+ const storage = new SpilloverStorage({
+ workspacePath: callerWorkspace,
+ ttlMs: 300_000,
+ thresholdBytes: 8192,
+ gcIntervalMs: 0,
+ })
+ const persistError = new Error('fixture spillover store unavailable')
+ vi.spyOn(storage, 'maybePersist').mockRejectedValue(persistError)
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-spillover-failing',
+ thresholdBytes: 8192,
+ spilloverStorage: storage,
+ command: `node -e "process.stdout.write('z'.repeat(16384))"`,
+ })
+
+ try {
+ const executor = await scenario.run()
+ await executor.resumeAfterApproval(false)
+
+ expect(executor.executorState).toBe('completed')
+ expect(scenario.providerRequests).toHaveLength(2)
+ const toolMessage = finalToolMessage(scenario.providerRequests)
+ expect(toolMessage.spillover_ref).toBeUndefined()
+ expect(toolMessage.content).toContain('z'.repeat(1024))
+ expect(Buffer.byteLength(toolMessage.content, 'utf8')).toBeLessThanOrEqual(32 * 1024)
+ } finally {
+ scenario.dispose()
+ vi.restoreAllMocks()
+ }
+})
+
+it('cancels before shell execution and performs no second provider round-trip', async () => {
+ const proofFile = 'gfs-cancel-proof.txt'
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-cancellation',
+ thresholdBytes: 2 * 1024 * 1024,
+ command:
+ `node -e "require('fs').writeFileSync('${proofFile}','cancelled');` +
+ `process.stdout.write('x'.repeat(1536 * 1024))"`,
+ })
+
+ try {
+ const executor = await scenario.run()
+ expect(executor.executorState).toBe('waiting_approval')
+ executor.abort()
+ await executor.resumeAfterApproval(false)
+
+ expect(executor.signal.aborted).toBe(true)
+ expect(fs.existsSync(path.join(callerWorkspace, proofFile))).toBe(false)
+ expect(scenario.providerRequests).toHaveLength(1)
+ } finally {
+ scenario.dispose()
+ }
+})
+
+it('denies shell approval without executing the command', async () => {
+ const proofFile = 'gfs-denial-proof.txt'
+ const responsePayloads: string[] = []
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-denial',
+ thresholdBytes: 2 * 1024 * 1024,
+ command: `node -e "require('fs').writeFileSync('${proofFile}','denied')"`,
+ })
+ try {
+ const executor = await scenario.run()
+ expect(executor.executorState).toBe('waiting_approval')
+ executor.sourceTask.responseCallback = async payload => {
+ if (payload.response) responsePayloads.push(payload.response)
+ }
+
+ await executor.deny()
+
+ expect(executor.executorState).toBe('completed')
+ expect(fs.existsSync(path.join(callerWorkspace, proofFile))).toBe(false)
+ expect(scenario.providerRequests).toHaveLength(1)
+ expect(responsePayloads.join('\n')).toContain('was denied by the user')
+ } finally {
+ scenario.dispose()
+ }
+})
+
+it('rehydrates a waiting GFS shell approval into a fresh executor', async () => {
+ const proofFile = 'gfs-cold-resume-proof.txt'
+ const scenario = makeScenario({
+ taskId: 'gfs-shell-cold-resume',
+ thresholdBytes: 2 * 1024 * 1024,
+ command: `node -e "require('fs').writeFileSync('${proofFile}','resumed')"`,
+ })
+
+ try {
+ const first = await scenario.run()
+ expect(first.executorState).toBe('waiting_approval')
+ const approval = first.pendingApproval
+ expect(approval).toBeDefined()
+ expect(approval!.task_budget).toBeDefined()
+
+ const second = scenario.create()
+ await second.rehydrateWaitingApproval(scenario.sessionKey, approval!)
+ expect(second.executorState).toBe('waiting_approval')
+ await second.resumeAfterApproval(false)
+
+ expect(second.executorState).toBe('completed')
+ expect(fs.readFileSync(path.join(callerWorkspace, proofFile), 'utf8')).toBe('resumed')
+ expect(scenario.providerRequests).toHaveLength(2)
+ } finally {
+ scenario.dispose()
+ }
+})
diff --git a/mcp-host/src/agent/fileReferenceResolver.ts b/mcp-host/src/agent/fileReferenceResolver.ts
index 14ffceb2d..4c0df0f44 100644
--- a/mcp-host/src/agent/fileReferenceResolver.ts
+++ b/mcp-host/src/agent/fileReferenceResolver.ts
@@ -12,6 +12,7 @@ import { type FileReferenceV1, parseFileReferenceV1 } from '@clerum/gfs-interact
import { FileReferenceErrorCode } from '../core/errors'
import type { TurnContextReferencedFile } from '../core/orchestration/turnContext'
import { GfscHttpError } from '../internalTools/gfsClient'
+import { GFS_FILE_LIMITS } from '../internalTools/gfsFilePolicy'
import { logger } from '../logger'
import { VISUAL_INPUT_LIMITS } from '../visualInput/policy'
@@ -286,7 +287,7 @@ async function resolveOne(
// Same version, so the same bytes: a different size is a reference that
// misstates the file, not a change to it.
if (view.bytes !== reference.byteLength) throw new ResolutionFailure('invalid', 'SizeMismatch')
- if (view.bytes > VISUAL_INPUT_LIMITS.fileBytes) return { availability: 'too_large', reference }
+ if (view.bytes > GFS_FILE_LIMITS.maxFileBytes) return { availability: 'too_large', reference }
return { availability: 'available', reference }
}
diff --git a/mcp-host/src/agent/gfsExecutionCapability.ts b/mcp-host/src/agent/gfsExecutionCapability.ts
new file mode 100644
index 000000000..ef52a13e4
--- /dev/null
+++ b/mcp-host/src/agent/gfsExecutionCapability.ts
@@ -0,0 +1,39 @@
+import type { ApprovalConfig } from '../core/extensions/approvalTypes'
+import type { GfsDownloadStore } from '../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../internalTools/gfsProcessingLease'
+import type { TaskSource } from '../queue/types'
+
+export interface GfsExecutionCapabilityInput {
+ approvalEnabled: boolean
+ source: TaskSource
+ callerIdentity: string | undefined
+ store: GfsDownloadStore | undefined
+ callerWorkspacePath: string | undefined
+ processingLeaseProvider: GfsProcessingLeaseProvider | undefined
+ approvalConfig: ApprovalConfig | undefined
+}
+
+/**
+ * GFS workspace delivery requires an attended channel caller, live approval,
+ * an explicit shell tool policy, and the Host-owned caller-bound store/root.
+ * Cron and internal tasks stay unattended; persistent approvals are handled at
+ * the live shell controller boundary, not by this static capability check.
+ */
+export function gfsWorkspaceExecutionEnabled(
+ input: GfsExecutionCapabilityInput
+): input is GfsExecutionCapabilityInput & {
+ callerIdentity: string
+ store: GfsDownloadStore
+ callerWorkspacePath: string
+ processingLeaseProvider: GfsProcessingLeaseProvider
+} {
+ return Boolean(
+ input.approvalEnabled &&
+ input.source === 'channel' &&
+ input.callerIdentity &&
+ input.store &&
+ input.callerWorkspacePath &&
+ input.processingLeaseProvider &&
+ input.approvalConfig?.tools?.shell_exec !== false
+ )
+}
diff --git a/mcp-host/src/agent/gfsReferenceSurfaces.ts b/mcp-host/src/agent/gfsReferenceSurfaces.ts
new file mode 100644
index 000000000..f9aae4d86
--- /dev/null
+++ b/mcp-host/src/agent/gfsReferenceSurfaces.ts
@@ -0,0 +1,35 @@
+import type { FileReferenceV1 } from '@clerum/gfs-interaction-policy'
+import { GFS_FILE_LIMITS } from '../internalTools/gfsFilePolicy'
+
+export interface GfsReferenceSurfaceCapability {
+ readonly metadata: true
+ readonly inline: boolean
+ readonly workspace: boolean
+ readonly localExecutor: boolean
+ readonly visual: boolean
+}
+
+export interface GfsSurfaceRuntimeCapability {
+ readonly workspaceFile: boolean
+ readonly localExecutor: boolean
+ readonly visual: boolean
+}
+
+/**
+ * Classify only surfaces proven by producer-owned byte admission and the
+ * current Host runtime. Filenames and media types are never evidence.
+ */
+export function classifyGfsReferenceSurfaces(
+ reference: Pick<FileReferenceV1, 'byteLength' | 'reader' | 'modelImageInput'>,
+ runtime: GfsSurfaceRuntimeCapability
+): GfsReferenceSurfaceCapability {
+ const withinSourceLimit = reference.byteLength <= GFS_FILE_LIMITS.maxFileBytes
+ const workspace = runtime.workspaceFile && withinSourceLimit
+ return {
+ metadata: true,
+ inline: reference.reader === 'text' && reference.byteLength <= GFS_FILE_LIMITS.inlineTextBytes,
+ workspace,
+ localExecutor: workspace && runtime.localExecutor,
+ visual: reference.modelImageInput === 'candidate' && runtime.visual && withinSourceLimit,
+ }
+}
diff --git a/mcp-host/src/agent/stateMachine.ts b/mcp-host/src/agent/stateMachine.ts
index b3f05a027..e008fb806 100644
--- a/mcp-host/src/agent/stateMachine.ts
+++ b/mcp-host/src/agent/stateMachine.ts
@@ -37,6 +37,7 @@ import type {
PendingApproval,
} from '../core/types'
import type { ReapedSession } from '../db/worker/protocol'
+import type { GfsDownloadStore } from '../internalTools/gfsDownloadStore'
import type { TaskLifecycle } from '../lifecycle/taskLifecycle'
import { isTerminal } from '../lifecycle/types'
import { SingleTurnProvider } from '../llm'
@@ -245,6 +246,8 @@ export class AgentStateMachine extends EventEmitter {
// Phase 7–8: Workspace memory & personalization
private workspaceProvider: ScopedWorkspaceProvider | undefined
+ private gfsWorkspaceProvider: ScopedWorkspaceProvider | undefined
+ private gfsDownloadStore: GfsDownloadStore | undefined
// T1.5 — Tool-result spillover store. Undefined when the feature flag is
// off; populated from main.ts and propagated into each TaskExecutor.
@@ -437,6 +440,14 @@ export class AgentStateMachine extends EventEmitter {
logger.info({ component: 'Agent' }, 'Workspace provider set')
}
+ setGfsWorkspaceProvider(provider: ScopedWorkspaceProvider | undefined): void {
+ this.gfsWorkspaceProvider = provider
+ }
+
+ setGfsDownloadStore(store: GfsDownloadStore | undefined): void {
+ this.gfsDownloadStore = store
+ }
+
/**
* T1.5 — Inject the spillover storage so each TaskExecutor can persist
* oversized tool outputs and register the `clerum__spillover_read` tool.
@@ -1508,6 +1519,9 @@ export class AgentStateMachine extends EventEmitter {
llmProvider: effective?.provider ?? this.llmProvider,
mcpManager: this.mcpManager,
workspaceService: this.workspaceProvider?.forSource(task.sourceMessage),
+ gfsDownloadStore: this.gfsDownloadStore,
+ gfsCallerWorkspacePath: this.gfsWorkspaceProvider?.forSource(task.sourceMessage)
+ ?.userRootPath,
config: this.config,
modelName: effective?.model ?? this.modelName,
contextWindowTokens: effective?.contextWindowTokens,
diff --git a/mcp-host/src/agent/taskExecutor.ts b/mcp-host/src/agent/taskExecutor.ts
index abc48c9a7..58c0fdfdd 100644
--- a/mcp-host/src/agent/taskExecutor.ts
+++ b/mcp-host/src/agent/taskExecutor.ts
@@ -60,6 +60,7 @@ import {
CAPABILITY_CONTRACT_TEXT,
DESKTOP_ENVIRONMENT_HINT,
DefaultPromptBuilder,
+ GFS_WORKSPACE_FILE_GUIDANCE_TEXT,
MCP_SERVER_SELECTION_TEXT,
MEMORY_GUIDANCE_TEXT,
TOOL_DISCOVERY_TEXT,
@@ -94,6 +95,8 @@ import type {
import { ApprovalExpiredError } from '../core/types'
import { prependTextToParts, textContentFromParts } from '../core/types'
import type { UsageContext } from '../core/types'
+import type { GfsDownloadStore } from '../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../internalTools/gfsProcessingLease'
import type { TaskLifecycle } from '../lifecycle/taskLifecycle'
import type { SingleTurnProvider } from '../llm'
import type { ImageInputResolver } from '../llm/imageInput'
@@ -115,6 +118,7 @@ import { resolveProviderWorkflowCallerContext } from '../workflow/providerWorkfl
import type { Workspace } from '../workspace/service'
import type { CronScheduler } from './cronScheduler'
import { referencedFilesForTurnContext } from './fileReferenceResolver'
+import { gfsWorkspaceExecutionEnabled } from './gfsExecutionCapability'
import {
type ProviderWorkflowAccessDenialReason,
isProviderWorkflowChannel,
@@ -172,6 +176,12 @@ export interface TaskExecutorDeps {
llmProvider: SingleTurnProvider
mcpManager: McpManager | null
workspaceService: Workspace | undefined
+ /** Host-owned store shared across tasks; task registries receive caller-bound handles. */
+ gfsDownloadStore?: GfsDownloadStore
+ /** Trusted caller workspace root, separate from memory-tool availability. */
+ gfsCallerWorkspacePath?: string
+ /** Required fail-closed lease boundary for approved local processing. */
+ gfsProcessingLeaseProvider?: GfsProcessingLeaseProvider
config: AgentConfig
modelName: string
/**
@@ -1902,6 +1912,7 @@ export class TaskExecutor {
tools.some(t => t.name === 'clerum__get_capabilities'),
tools.some(t => t.name.startsWith('desktop_') || t.name.startsWith('browser_')),
tools.some(t => t.name.startsWith('workflow_')),
+ tools.some(t => t.name === 'clerum__gfs_download'),
tools.some(t => t.name.includes('__')),
tools.some(t => t.name === 'clerum__tool_search'),
])
@@ -1930,6 +1941,7 @@ export class TaskExecutor {
// (not a strict MCP-server check) matches the legacy MCP-selection gate,
// which also fires for `clerum__get_capabilities`.
const hasWorkflowTools = tools.some(t => t.name.startsWith('workflow_'))
+ const hasGfsDownloadTool = tools.some(t => t.name === 'clerum__gfs_download')
// NOTE: this is intentionally still true when the bridge is active — the
// `clerum__tool_search/describe/call` native/bridge tools also contain `__`,
// so `MCP_SERVER_SELECTION_TEXT` still emits. That is fine: `TOOL_DISCOVERY_TEXT`
@@ -1954,6 +1966,7 @@ export class TaskExecutor {
platformHints,
capabilities,
workflowGuidance: hasWorkflowTools ? WORKFLOW_RECIPES_TEXT : '',
+ gfsWorkspaceGuidance: hasGfsDownloadTool ? GFS_WORKSPACE_FILE_GUIDANCE_TEXT : '',
mcpServerGuidance: hasMcpTools ? MCP_SERVER_SELECTION_TEXT : '',
toolDiscoveryGuidance: hasToolDiscovery ? TOOL_DISCOVERY_TEXT : '',
memoryGuidance: hasMemoryTools ? MEMORY_GUIDANCE_TEXT : '',
@@ -2053,6 +2066,23 @@ export class TaskExecutor {
appConfig,
this.deps.failover?.policy.fallbacks
)
+ const gfsCallerIdentity = this.task.sourceMessage?.sender
+ const gfsDownload = gfsWorkspaceExecutionEnabled({
+ approvalEnabled: appConfig.enableApproval,
+ source: this.task.source,
+ callerIdentity: gfsCallerIdentity,
+ store: this.deps.gfsDownloadStore,
+ callerWorkspacePath: this.deps.gfsCallerWorkspacePath,
+ processingLeaseProvider: this.deps.gfsProcessingLeaseProvider,
+ approvalConfig: this.deps.approvalConfig,
+ })
+ ? {
+ store: this.deps.gfsDownloadStore!,
+ callerIdentity: gfsCallerIdentity!,
+ callerWorkspacePath: this.deps.gfsCallerWorkspacePath!,
+ processingLeaseProvider: this.deps.gfsProcessingLeaseProvider!,
+ }
+ : undefined
const nativeRegistry = new NativeToolRegistry(
// The spillover threshold is a top-level setting; clerum__attachment_read
// states it in its description (#666).
@@ -2079,7 +2109,8 @@ export class TaskExecutor {
presentation.bridgeEnabled,
// §13 (stateless agents): the active provider's credential slot is the
// only one that survives into shell_exec's child env.
- this.deps.llmProvider.getProviderType()
+ this.deps.llmProvider.getProviderType(),
+ gfsDownload
)
await registerDesktopTools(nativeRegistry)
// Eagerly open this caller's remote OAuth partitions before the catalog is
@@ -2139,7 +2170,11 @@ export class TaskExecutor {
)
: new DefaultLoopController()
const innerController = approvalApplies
- ? new ApprovalController(this.conversation!, baseController)
+ ? new ApprovalController(
+ this.conversation!,
+ baseController,
+ gfsDownload ? new Set(['shell_exec']) : undefined
+ )
: baseController
const mcpManager = this.deps.mcpManager
diff --git a/mcp-host/src/core/adapters/__tests__/llmImageCompatibility.test.ts b/mcp-host/src/core/adapters/__tests__/llmImageCompatibility.test.ts
index 358dbffae..c05ce9188 100644
--- a/mcp-host/src/core/adapters/__tests__/llmImageCompatibility.test.ts
+++ b/mcp-host/src/core/adapters/__tests__/llmImageCompatibility.test.ts
@@ -512,10 +512,10 @@ describe('image source identity compatibility (#650)', () => {
messages,
tools: TOOLS,
})
- expect(imagePartsOf(binding.completeWithTools[0].messages)).toHaveLength(3)
+ expect(imagePartsOf(binding.completeWithTools[0].messages)).toHaveLength(4)
expect(
imagePartsOf(binding.completeWithTools[0].messages).every(
- part => part.source?.kind === 'tool'
+ part => part.source?.kind === 'tool' || part.source?.kind === 'attachment'
)
).toBe(true)
expect(messages).toEqual(snapshot)
@@ -928,7 +928,7 @@ describe('Codex V2 transport wiring (#650)', () => {
const codexImages = codexRequest.messages
.flatMap(message => message.contentParts ?? [])
.filter((part): part is CodexMessagePartImageV2 => part.type === 'image')
- expect(codexImages).toHaveLength(3)
+ expect(codexImages).toHaveLength(4)
expect(codexImages.every(part => part.source.kind === 'tool')).toBe(true)
const openaiImages = imagePartsOf(fallback.completeWithTools[0].messages)
expect(openaiImages).toHaveLength(2)
diff --git a/mcp-host/src/core/adapters/__tests__/llmPortAdapter.gfsVisualLimits.test.ts b/mcp-host/src/core/adapters/__tests__/llmPortAdapter.gfsVisualLimits.test.ts
new file mode 100644
index 000000000..3448fdea2
--- /dev/null
+++ b/mcp-host/src/core/adapters/__tests__/llmPortAdapter.gfsVisualLimits.test.ts
@@ -0,0 +1,65 @@
+import { describe, expect, it } from 'vitest'
+import type { ChatMessage, MessageContentPart } from '../../types'
+import { LlmPortAdapter } from '../llmPortAdapter'
+
+function gfsImagePart(
+ index: number,
+ width = 1,
+ height = 1
+): Extract<MessageContentPart, { type: 'image' }> {
+ const resourceId = String(index).padStart(32, '0')
+ return {
+ type: 'image',
+ mimeType: 'image/png',
+ data: 'AAAAAAAAAA==',
+ width,
+ height,
+ source: {
+ kind: 'gfs',
+ drive: 'main',
+ resourceId,
+ gfsUri: `gfs://main/${resourceId}`,
+ version: 1,
+ name: `image-${index}.png`,
+ },
+ }
+}
+
+function imageMessage(count: number, width = 1, height = 1): ChatMessage {
+ return {
+ role: 'user',
+ content: 'Compare these governed images',
+ contentParts: Array.from({ length: count }, (_, index) => gfsImagePart(index, width, height)),
+ }
+}
+
+function fittedImageCount(adapter: LlmPortAdapter, count: number, width = 1, height = 1): number {
+ const messages = [imageMessage(count, width, height)]
+ const fitted = (
+ adapter as unknown as {
+ fitGfsImages(
+ request: { messages: ChatMessage[]; tools: unknown[] },
+ providerMessages: ChatMessage[]
+ ): ChatMessage[]
+ }
+ ).fitGfsImages({ messages, tools: [] }, messages)
+ return fitted[0]!.contentParts!.filter(part => part.type === 'image').length
+}
+
+describe('LlmPortAdapter effective GFS visual limits', () => {
+ it('keeps four valid Codex images instead of applying the old three-image GFS gate', () => {
+ const adapter = new LlmPortAdapter({} as never, 'fixture-model', 'codex-subscription')
+ expect(fittedImageCount(adapter, 4)).toBe(4)
+ })
+
+ it('demotes only beyond the effective Codex image-count profile', () => {
+ const adapter = new LlmPortAdapter({} as never, 'fixture-model', 'codex-subscription')
+ expect(fittedImageCount(adapter, 21)).toBe(20)
+ })
+
+ it('demotes an image exceeding the effective Codex shape profile', () => {
+ const adapter = new LlmPortAdapter({} as never, 'fixture-model', 'codex-subscription')
+ expect(fittedImageCount(adapter, 1, 2048, 2048)).toBe(1)
+ expect(fittedImageCount(adapter, 1, 2049, 2048)).toBe(0)
+ })
+})
diff --git a/mcp-host/src/core/adapters/llmPortAdapter.ts b/mcp-host/src/core/adapters/llmPortAdapter.ts
index c372e05d1..eaf5626fb 100644
--- a/mcp-host/src/core/adapters/llmPortAdapter.ts
+++ b/mcp-host/src/core/adapters/llmPortAdapter.ts
@@ -17,6 +17,7 @@ import {
} from '../../llm/promptCacheMetrics'
import { logger } from '../../logger'
import { LlmUsageEvent, UsageReporter, newRequestId } from '../../usage/usageReporter.js'
+import { resolveVisualDeliveryLimits } from '../../visualInput/deliveryLimits'
import { gfsImageParts, projectGfsMessages } from '../../visualInput/messageProjection'
import { type ImageInputCapability, VisualInputError } from '../../visualInput/policy'
import { assertVisualRequestFits, hasGfsImageInput } from '../../visualInput/requestPolicy'
@@ -30,6 +31,7 @@ import {
ChatMessage,
CompletionRequest,
CompletionResponse,
+ MessageContentPart,
MessageRole,
ToolCompletionRequest,
ToolCompletionResponse,
@@ -289,27 +291,72 @@ export class LlmPortAdapter implements LlmPort {
): ChatMessage[] {
let messages = providerMessages
const candidates = gfsImageParts(messages)
+ if (candidates.length === 0) return messages
+
+ const limits = resolveVisualDeliveryLimits(this.providerName)
+ const allImages = (current: ChatMessage[]) =>
+ current.flatMap(message =>
+ (message.contentParts ?? []).filter(
+ (part): part is Extract<MessageContentPart, { type: 'image' }> =>
+ part.type === 'image' && !part.sourceIdentityOnly
+ )
+ )
+ const imageBytes = (part: Extract<MessageContentPart, { type: 'image' }>): number =>
+ Buffer.from(part.data, 'base64').byteLength
+ if (!limits) {
+ for (;;) {
+ try {
+ assertVisualRequestFits(messages, { ...request, messages })
+ return messages
+ } catch (error) {
+ if (
+ !(error instanceof VisualInputError) ||
+ error.code !== 'limit_exceeded' ||
+ candidates.length === 0
+ )
+ throw error
+ const selected = candidates.pop()!
+ messages = projectGfsMessages(messages, new Set([selected]), 'image_input_limit_exceeded')
+ }
+ }
+ }
+ const exceeds = (current: ChatMessage[]): boolean => {
+ const images = allImages(current)
+ const totalImageBytes = images.reduce((total, part) => total + imageBytes(part), 0)
+ const requestBytes = current.reduce(
+ (total, message) => total + Buffer.byteLength(JSON.stringify(message), 'utf8'),
+ 'tools' in request ? Buffer.byteLength(JSON.stringify(request.tools ?? []), 'utf8') : 0
+ )
+ return (
+ images.length > limits.maxImages ||
+ totalImageBytes > limits.maxTotalImageBytes ||
+ images.some(part => imageBytes(part) > limits.maxImageBytes) ||
+ images.some(
+ part =>
+ part.width !== undefined &&
+ part.height !== undefined &&
+ ((limits.maxDimension !== undefined &&
+ (part.width > limits.maxDimension || part.height > limits.maxDimension)) ||
+ (limits.maxPixels !== undefined && part.width * part.height > limits.maxPixels))
+ ) ||
+ requestBytes > limits.maxVisualRequestBytes
+ )
+ }
+
let demoted = 0
for (;;) {
- try {
- assertVisualRequestFits(messages, { ...request, messages })
+ if (!exceeds(messages)) {
if (demoted > 0)
logger.info(
{ provider: this.providerName, model: this.model, demotedImages: demoted },
'GFS images projected to references for this provider attempt'
)
return messages
- } catch (error) {
- if (
- !(error instanceof VisualInputError) ||
- error.code !== 'limit_exceeded' ||
- candidates.length === 0
- )
- throw error
- const selected = candidates.pop()!
- messages = projectGfsMessages(messages, new Set([selected]), 'image_input_limit_exceeded')
- demoted++
}
+ const selected = candidates.pop()
+ if (!selected) throw new VisualInputError('limit_exceeded')
+ messages = projectGfsMessages(messages, new Set([selected]), 'image_input_limit_exceeded')
+ demoted++
}
}
diff --git a/mcp-host/src/core/conversation/persistence/__tests__/sourceMessageResumeRoundTrip.test.ts b/mcp-host/src/core/conversation/persistence/__tests__/sourceMessageResumeRoundTrip.test.ts
index 2a9dd99b1..1c5fa3165 100644
--- a/mcp-host/src/core/conversation/persistence/__tests__/sourceMessageResumeRoundTrip.test.ts
+++ b/mcp-host/src/core/conversation/persistence/__tests__/sourceMessageResumeRoundTrip.test.ts
@@ -247,7 +247,13 @@ describe('#666 R4-M2 — source message resume round-trip', () => {
expect(listed?.content).toBe('read the notes')
expect(JSON.stringify(listed)).not.toContain(FILE_BYTES.toString('base64'))
expect([...referencedFilePins(listed?.fileReferenceResolutions).values()]).toEqual([
- { version: 3, currentVersion: 5 },
+ {
+ version: 3,
+ currentVersion: 5,
+ byteLength: 4,
+ reader: 'text',
+ modelImageInput: 'unsupported',
+ },
])
})
diff --git a/mcp-host/src/core/extensions/__tests__/approvalController.test.ts b/mcp-host/src/core/extensions/__tests__/approvalController.test.ts
index 58307127c..97c70a325 100644
--- a/mcp-host/src/core/extensions/__tests__/approvalController.test.ts
+++ b/mcp-host/src/core/extensions/__tests__/approvalController.test.ts
@@ -56,6 +56,73 @@ describe('ApprovalController', () => {
expect(spy).not.toHaveBeenCalled()
})
+ it('requires a fresh delegate decision for live-approval tools despite wildcard approval', () => {
+ const conv = makeConversation({
+ auto_approved_tools: new Set(['*', 'shell_exec']),
+ })
+ const pendingApproval: PendingApproval = {
+ request_id: 'req-live',
+ tool_name: 'other_tool',
+ parameters: {},
+ description: 'Other tool',
+ tool_call_id: 'tc_other',
+ context_snapshot: [],
+ }
+ const customDelegate = {
+ ...new DefaultLoopController(),
+ beforeTool: vi.fn().mockReturnValue({ type: 'suspend', approval: pendingApproval }),
+ shouldAccept: delegate.shouldAccept.bind(delegate),
+ onTextRejected: delegate.onTextRejected.bind(delegate),
+ onExhaustion: delegate.onExhaustion.bind(delegate),
+ refreshTools: delegate.refreshTools.bind(delegate),
+ }
+ const controller = new ApprovalController(conv, customDelegate, new Set(['shell_exec']))
+
+ const result = controller.beforeTool('shell_exec', { command: 'process-file' })
+ expect(result).toEqual({ type: 'suspend', approval: pendingApproval })
+ expect(customDelegate.beforeTool).toHaveBeenCalledWith('shell_exec', {
+ command: 'process-file',
+ })
+ })
+
+ it('consumes only the matching live one-shot approval and ignores persistent state', () => {
+ const pendingApproval: PendingApproval = {
+ request_id: 'req-live',
+ tool_name: 'shell_exec',
+ parameters: { command: 'process-file' },
+ description: 'Governed shell command',
+ tool_call_id: 'tc_shell',
+ context_snapshot: [],
+ }
+ const conv = makeConversation({
+ auto_approved_tools: new Set(['*']),
+ pending_approval: pendingApproval,
+ })
+ const customDelegate = {
+ ...new DefaultLoopController(),
+ beforeTool: vi.fn(),
+ shouldAccept: delegate.shouldAccept.bind(delegate),
+ onTextRejected: delegate.onTextRejected.bind(delegate),
+ onExhaustion: delegate.onExhaustion.bind(delegate),
+ refreshTools: delegate.refreshTools.bind(delegate),
+ }
+ const controller = new ApprovalController(conv, customDelegate, new Set(['shell_exec']))
+
+ expect(controller.beforeTool('shell_exec', pendingApproval.parameters)).toBe('proceed')
+ expect(conv.pending_approval).toBeUndefined()
+ expect(customDelegate.beforeTool).not.toHaveBeenCalled()
+
+ const secondApproval = { ...pendingApproval, request_id: 'req-live-2' }
+ customDelegate.beforeTool = vi.fn().mockReturnValue({
+ type: 'suspend',
+ approval: secondApproval,
+ })
+ expect(controller.beforeTool('shell_exec', pendingApproval.parameters)).toEqual({
+ type: 'suspend',
+ approval: secondApproval,
+ })
+ })
+
it("should propagate 'skip' from delegate", () => {
const conv = makeConversation()
const customDelegate = {
diff --git a/mcp-host/src/core/extensions/approvalController.ts b/mcp-host/src/core/extensions/approvalController.ts
index 8e2cd8853..81a6ef5d7 100644
--- a/mcp-host/src/core/extensions/approvalController.ts
+++ b/mcp-host/src/core/extensions/approvalController.ts
@@ -15,36 +15,35 @@
* Gate 2 (tool.requiresApproval() inside toolUseLoop) was REMOVED as part of
* the BUG-11 fix. All approval decisions now flow through this single gate.
*/
-
-import {
- LoopController,
-} from "../interfaces";
-import {
- ChatMessage,
- ToolDefinition,
- PendingApproval,
-} from "../types";
-import type { Conversation } from "../types";
-import { isMcpToolName, getMcpServerPrefix } from "./mcpApprovalGateController";
+import { LoopController } from '../interfaces'
+import { ChatMessage, PendingApproval, ToolDefinition } from '../types'
+import type { Conversation } from '../types'
+import { getMcpServerPrefix, isMcpToolName } from './mcpApprovalGateController'
/**
* Decorator that checks auto_approved_tools before delegating to base controller.
*/
export class ApprovalController implements LoopController {
- private readonly conversation: Conversation;
- private readonly delegate: LoopController;
+ private readonly conversation: Conversation
+ private readonly delegate: LoopController
+ private readonly liveApprovalTools: ReadonlySet<string>
- constructor(conversation: Conversation, delegate: LoopController) {
- this.conversation = conversation;
- this.delegate = delegate;
+ constructor(
+ conversation: Conversation,
+ delegate: LoopController,
+ liveApprovalTools?: ReadonlySet<string>
+ ) {
+ this.conversation = conversation
+ this.delegate = delegate
+ this.liveApprovalTools = liveApprovalTools ?? new Set<string>()
}
shouldAccept(content: string, iteration: number): boolean {
- return this.delegate.shouldAccept(content, iteration);
+ return this.delegate.shouldAccept(content, iteration)
}
onTextRejected(content: string, iteration: number): ChatMessage | null {
- return this.delegate.onTextRejected(content, iteration);
+ return this.delegate.onTextRejected(content, iteration)
}
/**
@@ -59,22 +58,36 @@ export class ApprovalController implements LoopController {
*/
beforeTool(
toolName: string,
- params: Record<string, unknown>,
- ): "proceed" | "skip" | { type: "suspend"; approval: PendingApproval } {
+ params: Record<string, unknown>
+ ): 'proceed' | 'skip' | { type: 'suspend'; approval: PendingApproval } {
+ // GFS local processing requires a live approval for every shell invocation.
+ // Wildcard and persistent approvals remain useful for other tools, but they
+ // cannot become implicit approval to execute a downloaded source.
+ if (this.liveApprovalTools.has(toolName)) {
+ if (
+ this.conversation.pending_approval &&
+ this.conversation.pending_approval.tool_name === toolName
+ ) {
+ this.conversation.pending_approval = undefined
+ return 'proceed'
+ }
+ return this.delegate.beforeTool(toolName, params)
+ }
+
// "Approve once, run all" — user approved any tool in this turn, auto-approve rest
- if (this.conversation.auto_approved_tools.has("*")) {
- return "proceed";
+ if (this.conversation.auto_approved_tools.has('*')) {
+ return 'proceed'
}
// Check individual tool name
if (this.conversation.auto_approved_tools.has(toolName)) {
- return "proceed";
+ return 'proceed'
}
// Check MCP server-level approval (e.g., "airtable-server" approves all airtable-server__* tools)
- const serverPrefix = getMcpServerPrefix(toolName);
+ const serverPrefix = getMcpServerPrefix(toolName)
if (serverPrefix && this.conversation.auto_approved_tools.has(serverPrefix)) {
- return "proceed";
+ return 'proceed'
}
// One-shot: pending_approval was granted but not yet consumed
@@ -82,20 +95,18 @@ export class ApprovalController implements LoopController {
this.conversation.pending_approval &&
this.conversation.pending_approval.tool_name === toolName
) {
- this.conversation.pending_approval = undefined;
- return "proceed";
+ this.conversation.pending_approval = undefined
+ return 'proceed'
}
- return this.delegate.beforeTool(toolName, params);
+ return this.delegate.beforeTool(toolName, params)
}
onExhaustion(iteration: number): string {
- return this.delegate.onExhaustion(iteration);
+ return this.delegate.onExhaustion(iteration)
}
- async refreshTools(
- currentTools: ToolDefinition[],
- ): Promise<ToolDefinition[]> {
- return this.delegate.refreshTools(currentTools);
+ async refreshTools(currentTools: ToolDefinition[]): Promise<ToolDefinition[]> {
+ return this.delegate.refreshTools(currentTools)
}
}
diff --git a/mcp-host/src/core/orchestration/toolUseLoopMessages.ts b/mcp-host/src/core/orchestration/toolUseLoopMessages.ts
index 0650b3747..1a6d31c83 100644
--- a/mcp-host/src/core/orchestration/toolUseLoopMessages.ts
+++ b/mcp-host/src/core/orchestration/toolUseLoopMessages.ts
@@ -94,6 +94,8 @@ function collectVisualImageParts(
mimeType,
data: attachment.dataBase64,
source: { kind: 'tool', attachmentId: attachment.id, toolCallId: result.tool_call_id },
+ ...(attachment.width === undefined ? {} : { width: attachment.width }),
+ ...(attachment.height === undefined ? {} : { height: attachment.height }),
...(!retained ? { sourceIdentityOnly: true as const } : {}),
})
}
@@ -130,6 +132,8 @@ function imagePart(attachment: Attachment, toolCallId: string): MessageContentPa
type: 'image',
mimeType: attachment.mimeType,
data: attachment.dataBase64,
+ ...(attachment.width === undefined ? {} : { width: attachment.width }),
+ ...(attachment.height === undefined ? {} : { height: attachment.height }),
...(attachment.visualSource
? { source: { ...attachment.visualSource, attachmentId: attachment.id, toolCallId } }
: {}),
diff --git a/mcp-host/src/core/reasoning/__tests__/promptBuilder.test.ts b/mcp-host/src/core/reasoning/__tests__/promptBuilder.test.ts
index cc8e06fa9..1dd7cb564 100644
--- a/mcp-host/src/core/reasoning/__tests__/promptBuilder.test.ts
+++ b/mcp-host/src/core/reasoning/__tests__/promptBuilder.test.ts
@@ -1,6 +1,10 @@
import { describe, expect, it } from 'vitest'
import { ToolDefinition } from '../../types'
-import { DefaultPromptBuilder, MEMORY_GUIDANCE_TEXT } from '../promptBuilder'
+import {
+ DefaultPromptBuilder,
+ GFS_WORKSPACE_FILE_GUIDANCE_TEXT,
+ MEMORY_GUIDANCE_TEXT,
+} from '../promptBuilder'
function tool(name: string): ToolDefinition {
return { name, description: `mock ${name}`, parameters: {} }
@@ -34,3 +38,18 @@ describe('DefaultPromptBuilder — memory guidance (P.4)', () => {
expect(occurrences).toBe(1)
})
})
+
+describe('DefaultPromptBuilder — governed GFS workspace files', () => {
+ it('includes the guidance when the explicit download tool is registered', () => {
+ const builder = new DefaultPromptBuilder()
+ const msg = builder.buildSystemPrompt([tool('clerum__gfs_download')])
+ expect(msg.content).toContain(GFS_WORKSPACE_FILE_GUIDANCE_TEXT)
+ expect(msg.content).toContain('write outputs outside `.gfs-downloads`')
+ })
+
+ it('omits the guidance when workspace download is unavailable', () => {
+ const builder = new DefaultPromptBuilder()
+ const msg = builder.buildSystemPrompt([tool('clerum__gfs_read')])
+ expect(msg.content).not.toContain(GFS_WORKSPACE_FILE_GUIDANCE_TEXT)
+ })
+})
diff --git a/mcp-host/src/core/reasoning/promptBuilder.ts b/mcp-host/src/core/reasoning/promptBuilder.ts
index 932e74f02..bca852bd1 100644
--- a/mcp-host/src/core/reasoning/promptBuilder.ts
+++ b/mcp-host/src/core/reasoning/promptBuilder.ts
@@ -57,6 +57,15 @@ export const DESKTOP_ENVIRONMENT_HINT =
'browser-based tasks (including OAuth flows), USE THESE TOOLS. You have full computer-use capabilities. ' +
"Take screenshots to see what's on screen, then use click/type/key to interact."
+/**
+ * Governed GFS workspace-file contract. Emitted whenever the explicit download
+ * tool is registered; the per-result `usage` object remains authoritative.
+ */
+export const GFS_WORKSPACE_FILE_GUIDANCE_TEXT =
+ 'GFS WORKSPACE FILES: When a GFS result says `delivery:"workspace_file"`, its `path` is relative to the caller workspace and the file is already present locally. ' +
+ 'Do not print, `file_read`, or paste the whole source. If analysis or conversion is needed, use an approved `shell_exec` command or script with that relative path, write outputs outside `.gfs-downloads`, and return only bounded counts, aggregates, errors, or small samples. ' +
+ 'Ask for shell approval with the exact command/script to execute. Do not treat expiry, checksum, version, transfer, or destination selection as model-controlled choices.'
+
/**
* Workflow-recipe behavioral contract. Emitted whenever any `workflow_*` tool
* is registered. Governs the `workflow_trigger`/`workflow_status`/`workflow_result`
@@ -166,6 +175,10 @@ export class DefaultPromptBuilder implements PromptBuilder {
sections.push(WORKFLOW_RECIPES_TEXT)
}
+ if (tools.some(t => t.name === 'clerum__gfs_download')) {
+ sections.push(GFS_WORKSPACE_FILE_GUIDANCE_TEXT)
+ }
+
// NOTE: this is intentionally still true when the bridge is active — the
// `clerum__tool_search/describe/call` native/bridge tools also contain `__`,
// so `MCP_SERVER_SELECTION_TEXT` still emits. That is fine: `TOOL_DISCOVERY_TEXT`
@@ -292,6 +305,7 @@ export class DefaultPromptBuilder implements PromptBuilder {
if (input.workflowGuidance.trim()) parts.push(input.workflowGuidance.trim())
if (input.mcpServerGuidance.trim()) parts.push(input.mcpServerGuidance.trim())
if (input.toolDiscoveryGuidance.trim()) parts.push(input.toolDiscoveryGuidance.trim())
+ if (input.gfsWorkspaceGuidance?.trim()) parts.push(input.gfsWorkspaceGuidance.trim())
if (input.memoryGuidance.trim()) parts.push(input.memoryGuidance.trim())
return parts.join('\n\n')
}
diff --git a/mcp-host/src/core/reasoning/systemPrompt.ts b/mcp-host/src/core/reasoning/systemPrompt.ts
index e1712453c..c6a52860d 100644
--- a/mcp-host/src/core/reasoning/systemPrompt.ts
+++ b/mcp-host/src/core/reasoning/systemPrompt.ts
@@ -75,6 +75,11 @@ export interface BuilderInput {
* `context` tier alongside `mcpServerGuidance` without breaking the cache hash.
*/
toolDiscoveryGuidance: string
+ /**
+ * Governed GFS workspace-file guidance. Empty when `clerum__gfs_download`
+ * is not registered. The per-tool `usage` receipt remains authoritative.
+ */
+ gfsWorkspaceGuidance?: string
/**
* Memory guidance (P.4). Empty string when the session does not have
* `memory_*` tools registered (P1-009): if memory tools light up
diff --git a/mcp-host/src/core/tools/__tests__/nativeToolRegistry.gfsReadPin.integration.test.ts b/mcp-host/src/core/tools/__tests__/nativeToolRegistry.gfsReadPin.integration.test.ts
index c6dd2a8a5..948a95918 100644
--- a/mcp-host/src/core/tools/__tests__/nativeToolRegistry.gfsReadPin.integration.test.ts
+++ b/mcp-host/src/core/tools/__tests__/nativeToolRegistry.gfsReadPin.integration.test.ts
@@ -227,12 +227,28 @@ describe('referencedFilePins (#666)', () => {
{ availability: 'stale', reference: gfsReference(3), resolvedVersion: 4 },
{ availability: 'unsupported', reference: attachment.value },
])
- expect([...pins]).toEqual([[`main/${RID}`, { version: 3, currentVersion: 4 }]])
+ expect([...pins]).toEqual([
+ [
+ `main/${RID}`,
+ {
+ version: 3,
+ currentVersion: 4,
+ byteLength: 25,
+ reader: 'text',
+ modelImageInput: 'unsupported',
+ },
+ ],
+ ])
})
it('pins an unavailable reference at its version without a current version', () => {
const pins = referencedFilePins([{ availability: 'denied', reference: gfsReference(3) }])
- expect([...pins]).toEqual([[`main/${RID}`, { version: 3 }]])
+ expect([...pins]).toEqual([
+ [
+ `main/${RID}`,
+ { version: 3, byteLength: 25, reader: 'text', modelImageInput: 'unsupported' },
+ ],
+ ])
})
it('pins nothing for a message without references', () => {
diff --git a/mcp-host/src/core/tools/__tests__/shellOutputBounds.test.ts b/mcp-host/src/core/tools/__tests__/shellOutputBounds.test.ts
new file mode 100644
index 000000000..ec5faae55
--- /dev/null
+++ b/mcp-host/src/core/tools/__tests__/shellOutputBounds.test.ts
@@ -0,0 +1,154 @@
+import { expect, it } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import type { GfsProcessingLeaseProvider } from '../../../internalTools/gfsProcessingLease'
+import type { ExecutionContext } from '../../interfaces'
+import { ShellTool } from '../shell'
+
+function executionContext(chunks: string[]): ExecutionContext {
+ return {
+ onOutput(chunk: string): void {
+ chunks.push(chunk)
+ },
+ }
+}
+
+it('bounds retained shell output and live progress independently', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-output-bounds-'))
+ const progress: string[] = []
+ const tool = new ShellTool(workspace, 10_000, ['PATH'])
+
+ try {
+ const result = await tool.execute(
+ {
+ command: 'node -e "process.stdout.write(\'x\'.repeat(1536 * 1024))"',
+ },
+ executionContext(progress)
+ )
+
+ expect(result.is_error).toBe(true)
+ expect(result.content).toContain('output_limit_exceeded')
+ expect(Buffer.byteLength(result.content, 'utf8')).toBeLessThanOrEqual(1024 * 1024)
+ expect(Buffer.byteLength(progress.join(''), 'utf8')).toBeLessThanOrEqual(64 * 1024)
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
+
+it('resets HOME to the caller workspace after environment merging', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-home-boundary-'))
+ const tool = new ShellTool(workspace, 5_000, ['PATH', 'HOME'], () => ({ HOME: '/tmp' }))
+
+ try {
+ const result = await tool.execute({ command: 'printf %s "$HOME"' })
+ expect(result.is_error).toBe(false)
+ expect(result.content).toBe(`stdout:\n${workspace}`)
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
+
+it('does not spawn when processing lease acquisition fails', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-lease-denied-'))
+ const marker = path.join(workspace, 'started')
+ const leases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => {
+ throw new Error('download expired')
+ },
+ releaseProcessingLease: async () => undefined,
+ }
+ const tool = new ShellTool(workspace, 5_000, ['PATH'], () => ({}), undefined, leases)
+
+ try {
+ const result = await tool.execute({ command: `touch ${JSON.stringify(marker)}` })
+ expect(result.is_error).toBe(true)
+ expect(result.content).toContain('download expired')
+ expect(fs.existsSync(marker)).toBe(false)
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
+
+it('releases the processing lease after command completion', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-lease-release-'))
+ const events: string[] = []
+ const leases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => {
+ events.push('acquire')
+ return { leaseId: 'lease-1', expiresAt: new Date(Date.now() + 60_000).toISOString() }
+ },
+ releaseProcessingLease: async () => {
+ events.push('release')
+ },
+ }
+ const tool = new ShellTool(workspace, 5_000, ['PATH'], () => ({}), undefined, leases)
+
+ try {
+ const result = await tool.execute({ command: 'printf done' })
+ expect(result.is_error).toBe(false)
+ expect(events).toEqual(['acquire', 'release'])
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
+
+it('does not report success when processing lease release fails', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-lease-failed-release-'))
+ const leases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => ({
+ leaseId: 'lease-1',
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ }),
+ releaseProcessingLease: async () => {
+ throw new Error('lease ledger locked')
+ },
+ }
+ const tool = new ShellTool(workspace, 5_000, ['PATH'], () => ({}), undefined, leases)
+
+ try {
+ const result = await tool.execute({ command: 'printf done' })
+ expect(result.is_error).toBe(true)
+ expect(result.content).toContain('processing_lease_release_failed')
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
+
+it('releases the processing lease only after process-group termination', async () => {
+ const workspace = fs.mkdtempSync(path.join(os.tmpdir(), 'shell-group-lease-'))
+ const marker = path.join(workspace, 'group-settled')
+ const script = path.join(workspace, 'spawn-group-child.js')
+ const releaseSawSettledMarker: boolean[] = []
+ fs.writeFileSync(
+ script,
+ `const cp = require('child_process')\n` +
+ `cp.spawn(process.execPath, ['-e', ${JSON.stringify(
+ `require('fs').writeFileSync(${JSON.stringify(marker)}, 'yes'); setInterval(() => {}, 1000)`
+ )}], { stdio: 'ignore' })\n` +
+ `const timer = setInterval(() => {\n` +
+ ` if (require('fs').existsSync(${JSON.stringify(marker)})) { clearInterval(timer); process.exit(0) }\n` +
+ `}, 5)\n`,
+ 'utf8'
+ )
+ const leases: GfsProcessingLeaseProvider = {
+ acquireProcessingLease: async () => ({
+ leaseId: 'lease-group',
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ }),
+ releaseProcessingLease: async () => {
+ releaseSawSettledMarker.push(fs.existsSync(marker))
+ },
+ }
+ const tool = new ShellTool(workspace, 5_000, ['PATH'], () => ({}), undefined, leases)
+
+ try {
+ const result = await tool.execute({ command: `node ${JSON.stringify(script)}` })
+ expect(result.is_error).toBe(false)
+ expect(result.content).not.toContain('process_group_termination_failed')
+ expect(fs.readFileSync(marker, 'utf8')).toBe('yes')
+ expect(releaseSawSettledMarker).toEqual([true])
+ } finally {
+ fs.rmSync(workspace, { recursive: true, force: true })
+ }
+})
diff --git a/mcp-host/src/core/tools/fileRead.ts b/mcp-host/src/core/tools/fileRead.ts
index ac80acb22..2cd6f25b5 100644
--- a/mcp-host/src/core/tools/fileRead.ts
+++ b/mcp-host/src/core/tools/fileRead.ts
@@ -1,5 +1,8 @@
import * as fs from 'fs/promises'
-import { isStateDbPath, stateDbProtectedMessage } from '../../workspace/service'
+import {
+ isProtectedWorkspacePath,
+ protectedWorkspacePathMessage,
+} from '../../workspace/protectedPaths'
import { Tool } from '../interfaces'
import { ToolOutput } from '../types'
import { validatePath } from './pathValidation'
@@ -47,11 +50,11 @@ export class FileReadTool implements Tool {
const filePath = params.path as string
const encoding = (params.encoding as BufferEncoding) || 'utf-8'
- // D3 — the session state database holds every user's conversations; the
- // agent must not read it (or its WAL laterals) through file tools.
- if (isStateDbPath(filePath)) {
+ // Platform-owned state and governed GFS downloads are not direct file-tool
+ // inputs. Large GFS content is processed through its governed transfer flow.
+ if (isProtectedWorkspacePath(filePath)) {
return {
- content: stateDbProtectedMessage(filePath),
+ content: protectedWorkspacePathMessage(filePath),
duration_ms: Date.now() - startTime,
is_error: true,
}
diff --git a/mcp-host/src/core/tools/fileWrite.ts b/mcp-host/src/core/tools/fileWrite.ts
index 693dcd4e5..d2affbe15 100644
--- a/mcp-host/src/core/tools/fileWrite.ts
+++ b/mcp-host/src/core/tools/fileWrite.ts
@@ -1,11 +1,10 @@
import * as fs from 'fs/promises'
import * as path from 'path'
import {
- adminManagedIdentityFileMessage,
- isLockedPath,
- isStateDbPath,
- stateDbProtectedMessage,
-} from '../../workspace/service'
+ isProtectedWorkspacePath,
+ protectedWorkspacePathMessage,
+} from '../../workspace/protectedPaths'
+import { adminManagedIdentityFileMessage, isLockedPath } from '../../workspace/service'
import { Tool } from '../interfaces'
import { ToolOutput } from '../types'
import { validatePath } from './pathValidation'
@@ -73,10 +72,11 @@ export class FileWriteTool implements Tool {
}
}
- // D3 — the session state database is platform state, never agent-writable.
- if (isStateDbPath(filePath)) {
+ // Platform-owned state and governed GFS downloads never enter through the
+ // generic write tool, even when an agent knows their relative path.
+ if (isProtectedWorkspacePath(filePath)) {
return {
- content: stateDbProtectedMessage(filePath),
+ content: protectedWorkspacePathMessage(filePath),
duration_ms: Date.now() - startTime,
is_error: true,
}
diff --git a/mcp-host/src/core/tools/memory.ts b/mcp-host/src/core/tools/memory.ts
index 37aa9cfc6..6d9d3479b 100644
--- a/mcp-host/src/core/tools/memory.ts
+++ b/mcp-host/src/core/tools/memory.ts
@@ -1,10 +1,10 @@
-import type { Workspace } from '../../workspace/service'
import {
- adminManagedIdentityFileMessage,
- isLockedPath,
- isStateDbPath,
- stateDbProtectedMessage,
-} from '../../workspace/service'
+ ProtectedWorkspacePathError,
+ isProtectedWorkspacePath,
+ protectedWorkspacePathMessage,
+} from '../../workspace/protectedPaths'
+import type { Workspace } from '../../workspace/service'
+import { adminManagedIdentityFileMessage, isLockedPath } from '../../workspace/service'
import { Tool } from '../interfaces'
import { ToolOutput } from '../types'
@@ -18,6 +18,13 @@ function todayDate(): string {
return new Date().toISOString().slice(0, 10) // YYYY-MM-DD
}
+function protectedPathOutput(err: unknown, startTime: number): ToolOutput | null {
+ if (err instanceof ProtectedWorkspacePathError) {
+ return { content: err.message, duration_ms: Date.now() - startTime, is_error: true }
+ }
+ return null
+}
+
export class MemorySearchTool implements Tool {
constructor(private readonly workspace: Workspace) {}
@@ -49,10 +56,14 @@ export class MemorySearchTool implements Tool {
async execute(params: Record<string, unknown>): Promise<ToolOutput> {
const startTime = Date.now()
- const results = await this.workspace.search(
- params.query as string,
- (params.limit as number) || 5
- )
+ let results
+ try {
+ results = await this.workspace.search(params.query as string, (params.limit as number) || 5)
+ } catch (err) {
+ const out = protectedPathOutput(err, startTime)
+ if (out) return out
+ throw err
+ }
return {
content: JSON.stringify({
query: params.query,
@@ -112,7 +123,7 @@ export class PersistentMemoryWriteTool implements Tool {
if (
err instanceof Error &&
(err.name === 'LockedFileError' ||
- err.name === 'StateDbPathError' ||
+ err instanceof ProtectedWorkspacePathError ||
err.name === 'MemoryScanRejectionError' ||
err.name === 'CrossUserAccessError')
) {
@@ -169,10 +180,10 @@ export class PersistentMemoryWriteTool implements Tool {
}
}
- // D3 — the session state database is platform state, never agent-writable.
- if (isStateDbPath(filePath)) {
+ // Platform-owned state and governed GFS downloads are not memory targets.
+ if (isProtectedWorkspacePath(filePath)) {
return {
- content: stateDbProtectedMessage(filePath),
+ content: protectedWorkspacePathMessage(filePath),
duration_ms: Date.now() - startTime,
is_error: true,
}
@@ -260,7 +271,14 @@ export class PersistentMemoryReadTool implements Tool {
is_error: true,
}
}
- const content = await this.workspace.read(path)
+ let content: string | null
+ try {
+ content = await this.workspace.read(path)
+ } catch (err) {
+ const out = protectedPathOutput(err, startTime)
+ if (out) return out
+ throw err
+ }
if (content === null) {
return {
content: `File not found: ${path}`,
@@ -307,7 +325,14 @@ export class MemoryTreeTool implements Tool {
async execute(params: Record<string, unknown>): Promise<ToolOutput> {
const startTime = Date.now()
- const entries = await this.workspace.list((params.path as string) || '')
+ let entries
+ try {
+ entries = await this.workspace.list((params.path as string) || '')
+ } catch (err) {
+ const out = protectedPathOutput(err, startTime)
+ if (out) return out
+ throw err
+ }
return {
content: JSON.stringify(entries),
duration_ms: Date.now() - startTime,
diff --git a/mcp-host/src/core/tools/nativeToolRegistry.ts b/mcp-host/src/core/tools/nativeToolRegistry.ts
index 368698bde..c965ab04d 100644
--- a/mcp-host/src/core/tools/nativeToolRegistry.ts
+++ b/mcp-host/src/core/tools/nativeToolRegistry.ts
@@ -12,6 +12,8 @@ import {
referencedFilePins,
} from '../../internalTools/gfs'
import { createGfscClient, getGfsToolScopes } from '../../internalTools/gfsClient'
+import type { GfsDownloadStore } from '../../internalTools/gfsDownloadStore'
+import type { GfsProcessingLeaseProvider } from '../../internalTools/gfsProcessingLease'
import type { LlmProvider } from '../../llm/registryCore'
import type { McpManager } from '../../mcp/manager'
import type { IncomingMessage } from '../../server'
@@ -183,15 +185,21 @@ export class NativeToolRegistry implements ToolRegistry {
// credential-slot stripping — only the active provider's credential env
// var survives into the child env. Appended as a trailing optional so
// existing positional call sites stay valid; only taskExecutor passes it.
- activeLlmProvider?: LlmProvider
+ activeLlmProvider?: LlmProvider,
+ gfsDownload?: {
+ store: GfsDownloadStore
+ callerIdentity: string
+ callerWorkspacePath: string
+ processingLeaseProvider?: GfsProcessingLeaseProvider
+ }
) {
// file_read/file_write are scoped to the per-user root when a ScopedWorkspace
// is wired (F1c) — they operate on a raw path string, not the Workspace
// interface, so we read the per-user root off it explicitly. Falls back to
// the shared root when there is no user context (e.g. the tool-name listing
- // registry in main.ts). ShellTool stays on the shared root: scoping its cwd
- // does not contain absolute-path access — shell isolation is a documented
- // residual (bundled with OS-level sandboxing).
+ // registry in main.ts). ShellTool uses the caller root when governed local
+ // processing is enabled. Its cwd/HOME boundary does not contain absolute-path
+ // access — shell isolation remains a documented residual.
const fileToolsRoot =
workspace instanceof ScopedWorkspace ? workspace.userRootPath : config.workspacePath
this.register(new FileReadTool(fileToolsRoot))
@@ -206,11 +214,12 @@ export class NativeToolRegistry implements ToolRegistry {
this.register(new FileWriteTool(fileToolsRoot))
this.register(
new ShellTool(
- config.workspacePath,
+ gfsDownload?.callerWorkspacePath ?? config.workspacePath,
config.shellTimeout,
config.envAllowlist,
dynamicEnvProvider,
- activeLlmProvider
+ activeLlmProvider,
+ gfsDownload?.processingLeaseProvider
)
)
this.register(new HttpRequestTool(config.httpAllowlist))
@@ -241,6 +250,9 @@ export class NativeToolRegistry implements ToolRegistry {
...(gfsScopes.has('gfs.read')
? buildGfsReadTools(gfsClient, {
referencedFiles: referencedFilePins(sourceMessage?.fileReferenceResolutions),
+ downloadStore: gfsDownload?.store,
+ callerIdentity: gfsDownload?.callerIdentity,
+ callerWorkspacePath: gfsDownload?.callerWorkspacePath,
})
: []),
...(gfsScopes.has('gfs.write') ? buildGfsWriteTools(gfsClient) : []),
diff --git a/mcp-host/src/core/tools/pathValidation.ts b/mcp-host/src/core/tools/pathValidation.ts
index 250124c25..0cf70f547 100644
--- a/mcp-host/src/core/tools/pathValidation.ts
+++ b/mcp-host/src/core/tools/pathValidation.ts
@@ -1,9 +1,10 @@
-import * as path from "path";
-import * as fs from "fs";
+import * as fs from 'fs'
+import * as path from 'path'
+import { isProtectedRealPath, isProtectedWorkspacePath } from '../../workspace/protectedPaths'
export function isWithinDirectory(child: string, parent: string): boolean {
- const relative = path.relative(parent, child);
- return !relative.startsWith("..") && !path.isAbsolute(relative);
+ const relative = path.relative(parent, child)
+ return !relative.startsWith('..') && !path.isAbsolute(relative)
}
/**
@@ -17,69 +18,92 @@ export function isWithinDirectory(child: string, parent: string): boolean {
*/
export function validatePath(
requestedPath: string,
- workspacePath: string,
+ workspacePath: string
): { valid: boolean; resolved: string; error?: string } {
// Reject null bytes (Risk 3.5.1)
- if (requestedPath.includes("\0")) {
- return { valid: false, resolved: "", error: "Path contains null bytes" };
+ if (requestedPath.includes('\0')) {
+ return { valid: false, resolved: '', error: 'Path contains null bytes' }
}
// Reject absolute paths (Risk 3.5.1)
if (path.isAbsolute(requestedPath)) {
- return { valid: false, resolved: "", error: "Absolute paths not allowed" };
+ return { valid: false, resolved: '', error: 'Absolute paths not allowed' }
}
// Reject parent directory traversal (Risk 3.5.1)
- if (requestedPath.includes("..")) {
+ if (requestedPath.includes('..')) {
return {
valid: false,
- resolved: "",
- error: "Directory traversal (..) not allowed",
- };
+ resolved: '',
+ error: 'Directory traversal (..) not allowed',
+ }
+ }
+
+ if (isProtectedWorkspacePath(requestedPath)) {
+ return {
+ valid: false,
+ resolved: '',
+ error: 'Reserved workspace path is not agent-accessible',
+ }
}
// Resolve within workspace
- const resolved = path.resolve(workspacePath, requestedPath);
+ const resolved = path.resolve(workspacePath, requestedPath)
// Verify resolved path is within workspace (symlink protection)
- const realWorkspace = fs.realpathSync(workspacePath);
- let realResolved: string;
+ const realWorkspace = fs.realpathSync(workspacePath)
+ let realResolved: string
try {
- realResolved = fs.realpathSync(resolved);
+ realResolved = fs.realpathSync(resolved)
} catch {
// File doesn't exist yet (for writes) — walk up to the first existing
// ancestor and verify it's within the workspace. This supports creating
// files in nested directories that don't exist yet (e.g., "mathlib/utils.js").
- let ancestor = path.dirname(resolved);
+ let ancestor = path.dirname(resolved)
while (ancestor !== path.dirname(ancestor)) {
try {
- const realAncestor = fs.realpathSync(ancestor);
+ const realAncestor = fs.realpathSync(ancestor)
if (!isWithinDirectory(realAncestor, realWorkspace)) {
return {
valid: false,
- resolved: "",
- error: "Path resolves outside workspace",
- };
+ resolved: '',
+ error: 'Path resolves outside workspace',
+ }
+ }
+ if (isProtectedRealPath(realAncestor, realWorkspace)) {
+ return {
+ valid: false,
+ resolved: '',
+ error: 'Reserved workspace path is not agent-accessible',
+ }
}
- return { valid: true, resolved };
+ return { valid: true, resolved }
} catch {
- ancestor = path.dirname(ancestor);
+ ancestor = path.dirname(ancestor)
}
}
return {
valid: false,
- resolved: "",
- error: "No valid ancestor directory found within workspace",
- };
+ resolved: '',
+ error: 'No valid ancestor directory found within workspace',
+ }
}
if (!isWithinDirectory(realResolved, realWorkspace)) {
return {
valid: false,
- resolved: "",
- error: "Path resolves outside workspace (symlink)",
- };
+ resolved: '',
+ error: 'Path resolves outside workspace (symlink)',
+ }
+ }
+
+ if (isProtectedRealPath(realResolved, realWorkspace)) {
+ return {
+ valid: false,
+ resolved: '',
+ error: 'Reserved workspace path is not agent-accessible',
+ }
}
- return { valid: true, resolved };
+ return { valid: true, resolved }
}
diff --git a/mcp-host/src/core/tools/shell.ts b/mcp-host/src/core/tools/shell.ts
index 15cf3daf8..94691f50a 100644
--- a/mcp-host/src/core/tools/shell.ts
+++ b/mcp-host/src/core/tools/shell.ts
@@ -1,4 +1,10 @@
import { spawn } from 'child_process'
+import { setTimeout as sleep } from 'node:timers/promises'
+import { recordGfsShellOutputLimit } from '../../internalTools/gfsDownloadMetrics'
+import type {
+ GfsProcessingLease,
+ GfsProcessingLeaseProvider,
+} from '../../internalTools/gfsProcessingLease'
import { ALL_PROVIDERS, LlmProvider, PROVIDERS } from '../../llm/registryCore'
import { ToolError, ToolErrorCode } from '../errors'
import { ExecutionContext, Tool } from '../interfaces'
@@ -14,6 +20,38 @@ import { ToolOutput } from '../types'
const STATE_DB_COMMAND_PATTERN =
/(^|[^A-Za-z0-9_.-])state\.db(-wal|-shm)?([^A-Za-z0-9_]|$)|\.clerum-state(\/|[^A-Za-z0-9_.-]|$)/
+function truncateUtf8Bytes(value: string, maximumBytes: number): string {
+ if (maximumBytes <= 0) return ''
+ const bytes = Buffer.from(value, 'utf8')
+ if (bytes.length <= maximumBytes) return value
+ return bytes
+ .subarray(0, maximumBytes)
+ .toString('utf8')
+ .replace(/\uFFFD$/u, '')
+}
+
+function processGroupExists(processGroupId: number): boolean {
+ try {
+ process.kill(-processGroupId, 0)
+ return true
+ } catch (error) {
+ return (error as NodeJS.ErrnoException).code === 'EPERM'
+ }
+}
+
+async function ensureProcessGroupTerminated(processGroupId: number): Promise<boolean> {
+ try {
+ process.kill(-processGroupId, 'SIGKILL')
+ } catch {
+ /* The group can exit between leader close and this signal. */
+ }
+ for (let attempt = 0; attempt < 100; attempt += 1) {
+ if (!processGroupExists(processGroupId)) return true
+ await sleep(10)
+ }
+ return !processGroupExists(processGroupId)
+}
+
/**
* ShellTool — runs an arbitrary shell command in the agent's workspace.
*
@@ -30,7 +68,10 @@ const STATE_DB_COMMAND_PATTERN =
*/
export class ShellTool implements Tool {
private static readonly MAX_TOTAL_BYTES = 1024 * 1024 // 1 MB — preserved from pre-spawn behavior
- private static readonly SIGKILL_GRACE_MS = 5000
+ /** Leaves room for status text, XML wrapping, and message serialization. */
+ private static readonly MAX_RESULT_BYTES = ShellTool.MAX_TOTAL_BYTES - 2048
+ private static readonly MAX_PROGRESS_BYTES = 64 * 1024
+ static readonly SIGKILL_GRACE_MS = 5000
/**
* `dynamicEnvProvider` returns operator-managed env vars from the
@@ -47,7 +88,9 @@ export class ShellTool implements Tool {
// stripping: only this provider's credential env var survives into the
// child env; every other provider slot is deleted. Undefined (tool-name
// listing registry, legacy tests) strips ALL slots — the secure default.
- private readonly activeLlmProvider?: LlmProvider
+ private readonly activeLlmProvider?: LlmProvider,
+ /** Caller-bound cleanup protection for retained GFS downloads. */
+ private readonly processingLeases?: GfsProcessingLeaseProvider
) {}
name() {
@@ -127,6 +170,9 @@ export class ShellTool implements Tool {
for (const [k, v] of Object.entries(dynamicEnv)) {
if (typeof v === 'string') safeEnv[k] = v
}
+ // Set HOME after operator/dynamic environment merging. A supplied HOME must
+ // never relocate the approved shell outside the caller's workspace root.
+ safeEnv.HOME = this.workspacePath
// §13 (stateless agents) — credential-slot stripping. The child env
// carries ONLY the ACTIVE provider's credential slots; every OTHER
@@ -145,6 +191,21 @@ export class ShellTool implements Tool {
}
}
+ let processingLease: GfsProcessingLease | undefined
+ if (this.processingLeases) {
+ try {
+ processingLease = await this.processingLeases.acquireProcessingLease({
+ durationMs: timeout + ShellTool.SIGKILL_GRACE_MS + 1_000,
+ })
+ } catch (err) {
+ return {
+ content: `Processing lease failed before command start: ${(err as Error).message}`,
+ duration_ms: Date.now() - startTime,
+ is_error: true,
+ }
+ }
+ }
+
return new Promise<ToolOutput>(resolve => {
// detached: true makes the child a process group leader so we can kill
// the whole group (shell + grandchildren like `sleep`) with -pid signal.
@@ -155,9 +216,10 @@ export class ShellTool implements Tool {
detached: true,
})
- const stdoutBuf: string[] = []
- const stderrBuf: string[] = []
+ const stdoutBuf: Buffer[] = []
+ const stderrBuf: Buffer[] = []
let totalBytes = 0
+ let progressBytes = 0
let killed: 'timeout' | 'maxbuffer' | 'cancelled' | null = null
let resolved = false
let killTimer: ReturnType<typeof setTimeout> | undefined
@@ -168,6 +230,25 @@ export class ShellTool implements Tool {
resolve(out)
}
+ const complete = async (
+ out: ToolOutput,
+ options: { releaseLease?: boolean } = {}
+ ): Promise<void> => {
+ if (processingLease && this.processingLeases && options.releaseLease !== false) {
+ try {
+ await this.processingLeases.releaseProcessingLease(processingLease)
+ } catch {
+ const status = '\n\n[processing_lease_release_failed]'
+ out.is_error = true
+ out.content = `${truncateUtf8Bytes(
+ out.content,
+ ShellTool.MAX_RESULT_BYTES - Buffer.byteLength(status, 'utf8')
+ )}${status}`
+ }
+ }
+ resolveOnce(out)
+ }
+
const forceKill = () => {
// Kill the entire process group so grandchildren (e.g., `sleep` inside a
// semi-colon chain) are also terminated and the stdout/stderr pipes close.
@@ -200,18 +281,41 @@ export class ShellTool implements Tool {
}
const onChunk = (chunk: Buffer, which: 'stdout' | 'stderr') => {
- const s = chunk.toString('utf8')
- const buf = which === 'stdout' ? stdoutBuf : stderrBuf
- buf.push(s)
- totalBytes += Buffer.byteLength(s, 'utf8')
- // Stream to watcher (if any). Interleaved stdout+stderr is fine for the preview.
- context?.onOutput(s)
+ // Bound both destinations before accepting the chunk. Bytes beyond the
+ // admission bound are counted only for the truthful limit decision and
+ // never enter retained output.
+ if (totalBytes >= ShellTool.MAX_TOTAL_BYTES) {
+ if (!killed) {
+ killed = 'maxbuffer'
+ recordGfsShellOutputLimit('output_limit_exceeded')
+ forceKill()
+ }
+ return
+ }
+ const retained = Math.min(chunk.length, ShellTool.MAX_TOTAL_BYTES - totalBytes)
+ if (retained > 0) stdoutOrStderr(which).push(chunk.subarray(0, retained))
+ totalBytes += chunk.length
+
+ // Progress is a live preview, not a second output channel. Stop it at
+ // its own fixed bound even while the final result is allowed to reach
+ // its larger admission bound.
+ const progressRemaining = ShellTool.MAX_PROGRESS_BYTES - progressBytes
+ if (progressRemaining > 0) {
+ const preview = chunk.subarray(0, progressRemaining)
+ progressBytes += preview.length
+ context?.onOutput(preview.toString('utf8'))
+ }
if (totalBytes > ShellTool.MAX_TOTAL_BYTES && !killed) {
killed = 'maxbuffer'
+ recordGfsShellOutputLimit('output_limit_exceeded')
forceKill()
}
}
+ function stdoutOrStderr(which: 'stdout' | 'stderr'): Buffer[] {
+ return which === 'stdout' ? stdoutBuf : stderrBuf
+ }
+
child.stdout?.on('data', c => onChunk(c, 'stdout'))
child.stderr?.on('data', c => onChunk(c, 'stderr'))
@@ -239,67 +343,79 @@ export class ShellTool implements Tool {
clearTimeout(killTimer)
context?.signal?.removeEventListener('abort', onAbort)
}
- child.on('close', exitCode => {
- // Leader/stdio close does not prove descendants exited. Complete group
- // termination now before dropping escalation, rather than leaving a
- // delayed signal that could target a reused process-group identifier.
- if (killed && typeof child.pid === 'number' && child.pid > 0) {
- try {
- process.kill(-child.pid, 'SIGKILL')
- } catch {
- /* Group already gone. */
- }
- }
+ const handleClose = async (exitCode: number | null): Promise<void> => {
+ // Leader/stdio close does not prove descendants exited. Always complete
+ // group termination before releasing a processing lease or resolving.
+ const processGroupId = child.pid
+ const groupTerminated =
+ typeof processGroupId === 'number' && processGroupId > 0
+ ? await ensureProcessGroupTerminated(processGroupId)
+ : true
cleanup()
- const stdout = stdoutBuf.join('')
- const stderr = stderrBuf.join('')
+ const stdout = Buffer.concat(stdoutBuf).toString('utf8')
+ const stderr = Buffer.concat(stderrBuf).toString('utf8')
const body =
[stdout ? `stdout:\n${stdout}` : '', stderr ? `stderr:\n${stderr}` : '']
.filter(Boolean)
.join('\n\n') || '(no output)'
+ const format = (status: string) => {
+ return `${truncateUtf8Bytes(
+ body,
+ ShellTool.MAX_RESULT_BYTES - Buffer.byteLength(status, 'utf8')
+ )}${status}`
+ }
+ let out: ToolOutput
if (killed === 'timeout') {
- resolveOnce({
- content: `${body}\n\n[Command killed after ${timeout}ms timeout — partial output above]`,
+ const status = `\n\n[Command killed after ${timeout}ms timeout — partial output above]`
+ out = { content: format(status), duration_ms: Date.now() - startTime, is_error: true }
+ } else if (killed === 'cancelled') {
+ const status = '\n\n[Command cancelled — partial output above]'
+ out = { content: format(status), duration_ms: Date.now() - startTime, is_error: true }
+ } else if (killed === 'maxbuffer') {
+ const status = `\n\n[output_limit_exceeded: command killed after more than ${ShellTool.MAX_TOTAL_BYTES} output bytes — bounded partial output above]`
+ out = { content: format(status), duration_ms: Date.now() - startTime, is_error: true }
+ } else if (exitCode !== 0 && exitCode !== null) {
+ const status = `Command failed (exit code ${exitCode}):\n`
+ out = {
+ content: `${status}${truncateUtf8Bytes(
+ body,
+ ShellTool.MAX_RESULT_BYTES - Buffer.byteLength(status, 'utf8')
+ )}`,
duration_ms: Date.now() - startTime,
is_error: true,
- })
- return
- }
- if (killed === 'cancelled') {
- resolveOnce({
- content: `${body}\n\n[Command cancelled — partial output above]`,
- duration_ms: Date.now() - startTime,
- is_error: true,
- })
- return
- }
- if (killed === 'maxbuffer') {
- resolveOnce({
- content: `${body}\n\n[Command killed: output exceeded ${ShellTool.MAX_TOTAL_BYTES} bytes — partial output above]`,
+ }
+ } else {
+ out = {
+ content: truncateUtf8Bytes(body, ShellTool.MAX_RESULT_BYTES),
duration_ms: Date.now() - startTime,
- is_error: true,
- })
- return
+ is_error: false,
+ }
}
- if (exitCode !== 0 && exitCode !== null) {
- resolveOnce({
- content: `Command failed (exit code ${exitCode}):\n${body}`,
+
+ if (!groupTerminated) {
+ const status = '\n\n[process_group_termination_failed]'
+ out = {
+ content: `${truncateUtf8Bytes(
+ out.content,
+ ShellTool.MAX_RESULT_BYTES - Buffer.byteLength(status, 'utf8')
+ )}${status}`,
duration_ms: Date.now() - startTime,
is_error: true,
- })
+ }
+ await complete(out, { releaseLease: false })
return
}
- resolveOnce({
- content: body,
- duration_ms: Date.now() - startTime,
- is_error: false,
- })
+ await complete(out)
+ }
+
+ child.on('close', exitCode => {
+ void handleClose(exitCode)
})
child.on('error', err => {
cleanup()
- resolveOnce({
+ void complete({
content: `Command failed to start: ${err.message}`,
duration_ms: Date.now() - startTime,
is_error: true,
diff --git a/mcp-host/src/core/types.ts b/mcp-host/src/core/types.ts
index 191244f45..0971987df 100644
--- a/mcp-host/src/core/types.ts
+++ b/mcp-host/src/core/types.ts
@@ -34,6 +34,9 @@ export type MessageContentPart =
data: string
/** Extra source copy, omitted from the legacy view while another current frame represents it. */
sourceIdentityOnly?: true
+ /** Producer-measured image shape; used by provider-attempt policy without redecoding. */
+ width?: number
+ height?: number
/**
* Optional so pre-#650 producers and provider translators keep compiling;
* `codexSubscription` rejects an image part without a usable source
diff --git a/mcp-host/src/internalTools/gfs.ts b/mcp-host/src/internalTools/gfs.ts
index 500d80d2b..1959b2b1f 100644
--- a/mcp-host/src/internalTools/gfs.ts
+++ b/mcp-host/src/internalTools/gfs.ts
@@ -1,3 +1,8 @@
+import { createHash } from 'node:crypto'
+import { constants as fsConstants } from 'node:fs'
+import * as fs from 'node:fs/promises'
+import * as path from 'node:path'
+import type { FileReferenceV1 } from '@clerum/gfs-interaction-policy'
import type { FileReferenceResolution } from '../agent/fileReferenceResolver'
import { inspectImage, validateImage } from '../visualInput/imageValidation'
import { VisualInputError } from '../visualInput/policy'
@@ -8,7 +13,17 @@ import type {
InternalToolResult,
} from '../workflow/types'
import { GfscHttpError } from './gfsClient'
-import { normalizeRid } from './gfsContentRead'
+import { GfsDownloadError } from './gfsContentDownload'
+import type { GfsDownloadOptions, GfsDownloadResult } from './gfsContentDownload'
+import { type GfsMetadataSnapshot, normalizeRid } from './gfsContentRead'
+import {
+ enterGfsDownloadTransfer,
+ exitGfsDownloadTransfer,
+ recordGfsDownloadAdmission,
+ recordGfsDownloadTransfer,
+} from './gfsDownloadMetrics'
+import { GfsDownloadStore, GfsDownloadStoreError } from './gfsDownloadStore'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
import type { GfsFileContent, GfsReadOptions } from './gfsReadTypes'
import { decodeTextContent } from './textContent'
@@ -41,6 +56,19 @@ export interface GfscReadClient {
args: { drive: string; resourceId: string },
options?: GfsReadOptions
): Promise<GfsFileContent>
+ readMetadata?(
+ args: { drive: string; resourceId: string },
+ options: {
+ signal?: AbortSignal
+ timeoutMs?: number
+ deadlineMs?: number
+ expectedVersion?: number
+ }
+ ): Promise<GfsMetadataSnapshot>
+ download?(
+ args: { drive: string; resourceId: string },
+ options: GfsDownloadOptions
+ ): Promise<GfsDownloadResult>
stat(args: { drive: string; resourceId: string }, call?: GfscCallOptions): Promise<unknown>
resolve(args: { uri: string }, call?: GfscCallOptions): Promise<unknown>
}
@@ -61,9 +89,18 @@ function ok(content: unknown): InternalToolResult {
// never the response body, which could carry paths or server internals.
function fail(error: unknown): InternalToolResult {
if (error instanceof VisualInputError) return { success: false, error: error.message }
+ if (error instanceof GfsDownloadError || error instanceof GfsDownloadStoreError)
+ return { success: false, error: error.message }
return redactedFail('GFS read failed', error)
}
+function versionConflict(error: unknown): boolean {
+ return (
+ (error instanceof VisualInputError && error.code === 'version_conflict') ||
+ (error instanceof GfsDownloadError && error.code === 'version_conflict')
+ )
+}
+
function fileReference(file: GfsFileContent, reason: string): InternalToolResult {
return ok({
resource: file.source,
@@ -95,6 +132,147 @@ function invalidArgs(message: string): InternalToolResult {
return { success: false, error: message }
}
+function workspaceFileUsage(visualDelivery: 'included' | 'not_included', visualReason?: string) {
+ return {
+ pathSemantics: 'relative-to-caller-workspace',
+ nextTool: 'shell_exec_when_local_processing_is_needed',
+ visualDelivery,
+ ...(visualReason === undefined ? {} : { visualReason }),
+ approval: 'user-approval-required',
+ writeOutputsTo: 'outputs/',
+ processLocally: true,
+ boundedOutputOnly: true,
+ wholeFileToContextAllowed: false,
+ }
+}
+
+async function resolveManagedImagePath(receipt: GfsDownloadResult, callerWorkspacePath: string) {
+ const realRoot = await fs.realpath(callerWorkspacePath)
+ const target = path.resolve(callerWorkspacePath, receipt.path)
+ const realTarget = await fs.realpath(target)
+ const managedRoot = path.join(realRoot, '.gfs-downloads')
+ const relativeToManagedRoot = path.relative(managedRoot, realTarget)
+ if (
+ relativeToManagedRoot.startsWith('..') ||
+ path.isAbsolute(relativeToManagedRoot) ||
+ !receipt.path.startsWith('.gfs-downloads/')
+ )
+ throw new VisualInputError('identity_mismatch')
+ return target
+}
+
+async function readManagedImageBytes(
+ receipt: GfsDownloadResult,
+ callerWorkspacePath: string
+): Promise<Buffer> {
+ const target = await resolveManagedImagePath(receipt, callerWorkspacePath)
+
+ const handle = await fs.open(target, fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW)
+ try {
+ const stat = await handle.stat()
+ if (
+ !stat.isFile() ||
+ stat.size !== receipt.sizeBytes ||
+ (stat.mode & 0o777) !== 0o600 ||
+ stat.nlink !== 1
+ )
+ throw new VisualInputError('identity_mismatch')
+ const bytes = await handle.readFile()
+ if (createHash('sha256').update(bytes).digest('hex') !== receipt.sha256)
+ throw new VisualInputError('identity_mismatch')
+ return bytes
+ } finally {
+ await handle.close()
+ }
+}
+
+async function readManagedImagePrefix(
+ receipt: GfsDownloadResult,
+ callerWorkspacePath: string
+): Promise<Buffer> {
+ const target = await resolveManagedImagePath(receipt, callerWorkspacePath)
+ const handle = await fs.open(target, fsConstants.O_RDONLY | fsConstants.O_NOFOLLOW)
+ try {
+ const prefix = Buffer.alloc(16)
+ const { bytesRead } = await handle.read(prefix, 0, prefix.length, 0)
+ return prefix.subarray(0, bytesRead)
+ } finally {
+ await handle.close()
+ }
+}
+
+async function projectManagedImage(
+ receipt: GfsDownloadResult,
+ callerWorkspacePath: string,
+ options: InternalToolExecutionOptions | undefined,
+ knownBytes?: Buffer
+): Promise<InternalToolResult> {
+ let visualReason = 'not_image'
+ try {
+ let bytes = knownBytes
+ if (bytes === undefined) {
+ const prefix = await readManagedImagePrefix(receipt, callerWorkspacePath)
+ const hasImageMagic =
+ prefix.length >= 3 &&
+ ((prefix[0] === 0x89 && prefix[1] === 0x50 && prefix[2] === 0x4e) ||
+ (prefix[0] === 0xff && prefix[1] === 0xd8 && prefix[2] === 0xff))
+ if (!hasImageMagic)
+ return {
+ success: true,
+ content: JSON.stringify({
+ delivery: 'workspace_file',
+ ...receipt,
+ visualDelivery: 'not_included',
+ visualReason,
+ usage: workspaceFileUsage('not_included', visualReason),
+ }),
+ }
+ bytes = await readManagedImageBytes(receipt, callerWorkspacePath)
+ }
+ const image = inspectImage(bytes)
+ if (image) {
+ const visualInput = options?.visualInput
+ if (!visualInput) visualReason = 'image_input_unavailable_in_this_execution'
+ else {
+ const capability = await visualInput.resolveCapability(options?.signal)
+ if (options?.signal?.aborted) throw new VisualInputError('cancelled')
+ if (capability.status !== 'supported')
+ visualReason = `model_image_input_${capability.status}`
+ else {
+ await validateImage(bytes, image, { signal: options?.signal, budget: visualInput.budget })
+ if (options?.signal?.aborted) throw new VisualInputError('cancelled')
+ const dataBase64 = visualInput.budget.encodeImage(bytes)
+ return {
+ success: true,
+ content: JSON.stringify({
+ delivery: 'workspace_file',
+ ...receipt,
+ visualDelivery: 'included',
+ usage: workspaceFileUsage('included'),
+ }),
+ images: [
+ { ...image, source: receipt.source, sizeBytes: receipt.sizeBytes, dataBase64 },
+ ],
+ }
+ }
+ }
+ }
+ } catch (error) {
+ if (error instanceof VisualInputError && error.code !== 'cancelled') visualReason = error.code
+ else throw error
+ }
+ return {
+ success: true,
+ content: JSON.stringify({
+ delivery: 'workspace_file',
+ ...receipt,
+ visualDelivery: 'not_included',
+ visualReason,
+ usage: workspaceFileUsage('not_included', visualReason),
+ }),
+ }
+}
+
const driveResourceParams = {
type: 'object',
required: ['drive', 'resourceId'],
@@ -111,6 +289,12 @@ const driveResourceParams = {
export interface ReferencedFilePin {
version: number
currentVersion?: number
+ /** Producer-classified payload size; retained without filename inference. */
+ byteLength?: FileReferenceV1['byteLength']
+ /** Producer byte classification: text, image, or binary. */
+ reader?: FileReferenceV1['reader']
+ /** Producer-declared visual candidacy, not a provider-capability claim. */
+ modelImageInput?: FileReferenceV1['modelImageInput']
}
/** Pins keyed by `${drive}/${rid}`, with the rid normalized as gfsc names it. */
@@ -133,6 +317,9 @@ export function referencedFilePins(
if (!key) throw new Error('A resolved GFS file reference must carry a 32-hex resourceId')
pins.set(key, {
version: source.version,
+ byteLength: reference.byteLength,
+ reader: reference.reader,
+ modelImageInput: reference.modelImageInput,
...(availability === 'stale' && resolvedVersion !== undefined
? { currentVersion: resolvedVersion }
: {}),
@@ -158,14 +345,62 @@ const LIVE_RESOURCE_NOTE =
export interface GfsReadToolOptions {
/** Files the turn's message referenced; an empty map when there is no message. */
referencedFiles: ReferencedFilePins
+ /** Host-owned durable store; task registries receive a caller-bound handle. */
+ downloadStore?: GfsDownloadStore
+ /** Trusted identity derived by the Host, never tool arguments. */
+ callerIdentity?: string
+ /** Caller workspace root derived by the Host, never tool arguments. */
+ callerWorkspacePath?: string
}
/** The five read tools, bound to a gfsc client. */
export function buildGfsReadTools(
client: GfscReadClient,
- { referencedFiles }: GfsReadToolOptions
+ { referencedFiles, downloadStore, callerIdentity, callerWorkspacePath }: GfsReadToolOptions
): InternalToolDefinition[] {
- return [
+ const canDownload = Boolean(
+ downloadStore && client.download && callerIdentity && callerWorkspacePath
+ )
+ const transferToWorkspace = async (
+ target: { drive: string; resourceId: string },
+ expectedVersion: number | undefined,
+ context?: InternalToolExecutionOptions
+ ): Promise<GfsDownloadResult> => {
+ if (!downloadStore || !client.download || !callerIdentity || !callerWorkspacePath)
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ const transferStartedAt = Date.now()
+ enterGfsDownloadTransfer()
+ try {
+ const receipt = await client.download(target, {
+ store: downloadStore,
+ callerIdentity,
+ callerWorkspacePath,
+ ...callOptions(context),
+ timeoutMs: context?.timeoutMs,
+ ...(expectedVersion === undefined ? {} : { expectedVersion }),
+ })
+ recordGfsDownloadTransfer('success', (Date.now() - transferStartedAt) / 1000)
+ return receipt
+ } catch (error) {
+ recordGfsDownloadTransfer('failure', (Date.now() - transferStartedAt) / 1000)
+ throw error
+ } finally {
+ exitGfsDownloadTransfer()
+ }
+ }
+ const downloadToWorkspace = async (
+ target: { drive: string; resourceId: string },
+ expectedVersion: number | undefined,
+ context?: InternalToolExecutionOptions
+ ): Promise<InternalToolResult> => {
+ const receipt = await transferToWorkspace(target, expectedVersion, context)
+ return ok({
+ delivery: 'workspace_file',
+ ...receipt,
+ usage: workspaceFileUsage('not_included'),
+ })
+ }
+ const tools: InternalToolDefinition[] = [
{
name: 'clerum__gfs_accessible',
description:
@@ -226,7 +461,7 @@ export function buildGfsReadTools(
{
name: 'clerum__gfs_read',
description:
- 'Read a GFS file by drive + resourceId. Returns UTF-8 text, or a bounded JPEG/PNG image when the active model supports image input. Other binary formats return a reference; malformed or unsupported JPEG/PNG returns an error. ' +
+ 'Read a GFS file by drive + resourceId. Returns bounded UTF-8 text inline; larger admitted sources are delivered as governed workspace files, and a valid JPEG/PNG may additionally become visual input after its workspace receipt is preserved. ' +
'The Host reads a referenced_file from the turn context at the version the reference names; pass expectedVersion only to read the current_version of a stale reference. If the file changed since it was referenced, the result has availability "stale" and no content.',
parameters: {
...driveResourceParams,
@@ -259,16 +494,63 @@ export function buildGfsReadTools(
return invalidArgs(pinnedVersionMessage(pin))
}
let file: GfsFileContent | undefined
+ let snapshot: GfsMetadataSnapshot | undefined
try {
+ if (client.readMetadata) {
+ const metadata = await client.readMetadata(
+ target as { drive: string; resourceId: string },
+ {
+ ...callOptions(options),
+ timeoutMs: options?.timeoutMs,
+ ...(expectedVersion === undefined ? {} : { expectedVersion }),
+ }
+ )
+ snapshot = metadata
+ if (metadata.size > GFS_FILE_LIMITS.inlineTextBytes) {
+ if (canDownload) {
+ recordGfsDownloadAdmission('workspace_attempt')
+ const receipt = await transferToWorkspace(
+ target as { drive: string; resourceId: string },
+ metadata.source.version,
+ options
+ )
+ return await projectManagedImage(receipt, callerWorkspacePath!, options)
+ } else {
+ recordGfsDownloadAdmission('workspace_unavailable')
+ expectedVersion ??= metadata.source.version
+ return ok({
+ availability: 'workspace_delivery_unavailable',
+ resource: metadata.source,
+ sizeBytes: metadata.size,
+ inlineTextBytes: GFS_FILE_LIMITS.inlineTextBytes,
+ reason: 'workspace_delivery_is_required_for_this_file_size',
+ })
+ }
+ recordGfsDownloadAdmission('inline_attempt')
+ if (expectedVersion === undefined) expectedVersion = metadata.source.version
+ }
+ }
file = await client.read(target as { drive: string; resourceId: string }, {
...callOptions(options),
timeoutMs: options?.timeoutMs,
budget: options?.readBudget ?? options?.visualInput?.budget,
- ...(expectedVersion === undefined ? {} : { expectedVersion }),
+ ...(snapshot === undefined ? {} : { metadataSnapshot: snapshot }),
+ ...((snapshot?.source.version ?? expectedVersion) === undefined
+ ? {}
+ : { expectedVersion: snapshot?.source.version ?? expectedVersion }),
})
if (options?.signal?.aborted) throw new VisualInputError('cancelled')
const image = inspectImage(file.bytes)
if (image) {
+ if (canDownload) {
+ recordGfsDownloadAdmission('workspace_attempt')
+ const receipt = await transferToWorkspace(
+ target as { drive: string; resourceId: string },
+ file.source.version,
+ options
+ )
+ return await projectManagedImage(receipt, callerWorkspacePath!, options, file.bytes)
+ }
const visualInput = options?.visualInput
if (!visualInput)
return fileReference(file, 'image_input_unavailable_in_this_execution')
@@ -297,21 +579,22 @@ export function buildGfsReadTools(
],
}
}
- if (/\.(?:png|jpe?g)$/i.test(file.source.name))
- throw new VisualInputError('invalid_image')
- // Same fileBytes cap as images (G0); same text rule as clerum__attachment_read.
+ // Inline text is bounded by the generic GFS policy; larger sources were
+ // routed to the workspace before this in-memory read.
const text = decodeTextContent(file.bytes)
if (text === null) return fileReference(file, 'unsupported_binary_format')
if (isSvgText(text)) return fileReference(file, 'svg_visual_input_not_supported')
return ok(text)
} catch (err) {
- // #666 — with an expected version, a version conflict is an answer about
- // the reference, not a read failure.
if (
- expectedVersion !== undefined &&
+ snapshot === undefined &&
err instanceof VisualInputError &&
- err.code === 'version_conflict'
+ err.code === 'limit_exceeded'
)
+ recordGfsDownloadAdmission('limit_exceeded')
+ // #666 — with an expected version, a version conflict is an answer about
+ // the reference, not a read failure.
+ if (expectedVersion !== undefined && versionConflict(err))
return ok({
availability: 'stale',
drive: target.drive,
@@ -363,6 +646,49 @@ export function buildGfsReadTools(
},
},
]
+ if (canDownload) {
+ tools.push({
+ name: 'clerum__gfs_download',
+ description:
+ 'Download a GFS file to this caller workspace without putting its contents in model context. Returns source metadata, SHA-256, exact size, version, expiry, and a relative workspace path for approved local processing.',
+ parameters: {
+ ...driveResourceParams,
+ properties: {
+ ...driveResourceParams.properties,
+ expectedVersion: {
+ type: 'integer',
+ minimum: 0,
+ description: 'Download only this version of the file.',
+ },
+ },
+ },
+ execute: async (args, _outputDir, options): Promise<InternalToolResult> => {
+ const { expectedVersion: requestedVersion, ...target } = args
+ if (requestedVersion !== undefined && !isValidIfMatch(requestedVersion))
+ return invalidArgs('expectedVersion must be a non-negative integer.')
+ const key = referencedFileKey(target.drive, target.resourceId)
+ const pin = key === null ? undefined : referencedFiles.get(key)
+ let expectedVersion = requestedVersion
+ if (pin) {
+ if (requestedVersion === undefined) expectedVersion = pin.version
+ else if (requestedVersion !== pin.version && requestedVersion !== pin.currentVersion)
+ return invalidArgs(pinnedVersionMessage(pin))
+ }
+ try {
+ return await downloadToWorkspace(
+ target as { drive: string; resourceId: string },
+ expectedVersion as number | undefined,
+ options
+ )
+ } catch (error) {
+ if (expectedVersion !== undefined && versionConflict(error))
+ return ok({ availability: 'stale', ...target, expectedVersion })
+ return fail(error)
+ }
+ },
+ })
+ }
+ return tools
}
/** The gfsc write surface (P4) — read + write. */
diff --git a/mcp-host/src/internalTools/gfsClient.test.ts b/mcp-host/src/internalTools/gfsClient.test.ts
index c9ff73a99..42e7aaa51 100644
--- a/mcp-host/src/internalTools/gfsClient.test.ts
+++ b/mcp-host/src/internalTools/gfsClient.test.ts
@@ -514,7 +514,7 @@ describe('GFS binary content snapshots', () => {
it('rejects an oversized snapshot before downloading it', async () => {
const { client, fetchFn, budget } = contentHarness(Buffer.alloc(0), {
- metadata: { bytes: VISUAL_INPUT_LIMITS.fileBytes + 1 },
+ metadata: { bytes: 16 * 1024 * 1024 + 1 },
})
await expect(client.read(READ_ARGS, { budget })).rejects.toMatchObject({
code: 'limit_exceeded',
diff --git a/mcp-host/src/internalTools/gfsClient.ts b/mcp-host/src/internalTools/gfsClient.ts
index ce6254e6c..4ef66d55e 100644
--- a/mcp-host/src/internalTools/gfsClient.ts
+++ b/mcp-host/src/internalTools/gfsClient.ts
@@ -3,7 +3,13 @@ import { readFile } from 'node:fs/promises'
import { withAbort } from '../core/adapters/abortableLlmPort'
import { decodeRuntimeJwtScopes } from '../workflow/mcpHostRuntimeJwt'
import type { GfscCallOptions, GfscWriteClient } from './gfs'
-import { boundedGfsErrorDetail, readGfsContent } from './gfsContentRead'
+import { downloadGfsContent } from './gfsContentDownload'
+import {
+ type GfsMetadataSnapshot,
+ boundedGfsErrorDetail,
+ readGfsContent,
+ readGfsMetadata,
+} from './gfsContentRead'
const DIRECT_KEY = 'MCP_HOST_GFS_TOKEN'
const FILE_KEY = 'MCP_HOST_GFS_TOKEN_FILE'
@@ -291,6 +297,32 @@ export function createGfscClient(env: GfsRuntimeEnv, options: GfscClientOptions)
args,
options
),
+ readMetadata: (args, options = {}) =>
+ readGfsMetadata(
+ (path, init, deadlineMs) =>
+ responseWithRetry(
+ readerBase,
+ path,
+ init,
+ { signal: init.signal as AbortSignal, deadlineMs },
+ true
+ ),
+ args,
+ options
+ ),
+ download: (args, options) =>
+ downloadGfsContent(
+ (path, init, deadlineMs) =>
+ responseWithRetry(
+ readerBase,
+ path,
+ init,
+ { signal: init.signal as AbortSignal, deadlineMs },
+ true
+ ),
+ args,
+ options
+ ),
stat: ({ drive, resourceId }, call) =>
request(
readerBase,
diff --git a/mcp-host/src/internalTools/gfsContentDownload.test.ts b/mcp-host/src/internalTools/gfsContentDownload.test.ts
new file mode 100644
index 000000000..bb5a227e0
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsContentDownload.test.ts
@@ -0,0 +1,274 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { createHash } from 'node:crypto'
+import * as fs from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import * as path from 'node:path'
+import { downloadGfsContent } from './gfsContentDownload'
+import { readGfsMetadata } from './gfsContentRead'
+import { GfsDownloadStore } from './gfsDownloadStore'
+
+const args = { drive: 'main', resourceId: 'a'.repeat(32) }
+const uri = `gfs://main/${args.resourceId}`
+let hostRoot: string
+let callerRoot: string
+let store: GfsDownloadStore
+
+function metadata(size: number, overrides: Record<string, unknown> = {}): Response {
+ return new Response(
+ JSON.stringify({
+ ok: true,
+ data: {
+ ...args,
+ rid: args.resourceId,
+ gfsUri: uri,
+ kind: 'file',
+ name: 'input.csv',
+ version: 4,
+ bytes: size,
+ ...overrides,
+ },
+ }),
+ { headers: { 'content-type': 'application/json' } }
+ )
+}
+
+function contentHeaders(size: number, overrides: Record<string, string | undefined> = {}) {
+ const defined = Object.fromEntries(
+ Object.entries(overrides).filter(([, value]) => value !== undefined)
+ ) as Record<string, string>
+ const headers = {
+ 'content-length': String(size),
+ 'x-gfs-uri': uri,
+ 'x-gfs-version': '4',
+ ...defined,
+ }
+ if ('content-length' in overrides && overrides['content-length'] === undefined)
+ delete (headers as { 'content-length'?: string })['content-length']
+ return headers
+}
+
+function harness(
+ bytes: Uint8Array,
+ changes: {
+ metadata?: Record<string, unknown>
+ headers?: Record<string, string | undefined>
+ body?: ReadableStream<Uint8Array>
+ } = {}
+) {
+ return vi.fn(async (url: string) =>
+ url.includes('/content?')
+ ? new Response(changes.body ?? new Uint8Array(bytes), {
+ headers: contentHeaders(bytes.byteLength, changes.headers),
+ })
+ : metadata(bytes.byteLength, changes.metadata)
+ )
+}
+
+function options(overrides: Record<string, unknown> = {}) {
+ return {
+ store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ ...overrides,
+ }
+}
+
+async function retainedDirectories(): Promise<string[]> {
+ return fs.readdir(path.join(callerRoot, '.gfs-downloads')).catch(() => [])
+}
+
+beforeEach(async () => {
+ hostRoot = await fs.mkdtemp(path.join(tmpdir(), 'gfs-download-test-'))
+ callerRoot = path.join(hostRoot, 'users', 'caller-a')
+ await fs.mkdir(callerRoot, { recursive: true, mode: 0o700 })
+ store = new GfsDownloadStore(hostRoot)
+ await store.initialize()
+})
+
+afterEach(async () => {
+ vi.useRealTimers()
+ vi.restoreAllMocks()
+ await fs.rm(hostRoot, { recursive: true, force: true })
+})
+
+describe('governed GFS content download', () => {
+ it('streams the exact incident-sized CSV and returns only a bounded receipt', async () => {
+ const bytes = Buffer.alloc(3_836_961, 65)
+ const result = await downloadGfsContent(harness(bytes), args, options())
+
+ expect(await fs.readFile(path.join(callerRoot, result.path))).toEqual(bytes)
+ expect(result).toMatchObject({
+ sizeBytes: 3_836_961,
+ sha256: createHash('sha256').update(bytes).digest('hex'),
+ })
+ expect(result.source.version).toBe(4)
+ expect(JSON.stringify(result)).not.toContain('AAAAAAA')
+ expect(JSON.stringify(result).length).toBeLessThan(4096)
+ expect((await fs.stat(path.join(callerRoot, result.path))).mode & 0o777).toBe(0o600)
+ expect((await fs.stat(path.dirname(path.join(callerRoot, result.path)))).mode & 0o777).toBe(
+ 0o700
+ )
+ }, 10_000)
+
+ it('waits for each partial write before pulling more content', async () => {
+ let finish!: () => void
+ const finalChunk = new Promise<void>(resolve => {
+ finish = resolve
+ })
+ let firstWritten!: () => void
+ const started = new Promise<void>(resolve => {
+ firstWritten = resolve
+ })
+ let pulls = 0
+ const body = new ReadableStream<Uint8Array>(
+ {
+ async pull(controller) {
+ if (pulls++ === 0) {
+ controller.enqueue(new Uint8Array([1, 2, 3]))
+ return
+ }
+ await finalChunk
+ controller.enqueue(new Uint8Array([4, 5, 6]))
+ controller.close()
+ },
+ },
+ { highWaterMark: 0 }
+ )
+ const probe = await fs.open(path.join(callerRoot, 'probe'), 'w')
+ const prototype = Object.getPrototypeOf(probe)
+ await probe.close()
+ const original = prototype.write
+ vi.spyOn(prototype, 'write').mockImplementation(async function (
+ this: fs.FileHandle,
+ ...values: unknown[]
+ ) {
+ const written = await original.apply(this, values)
+ firstWritten()
+ return written
+ })
+
+ const pending = downloadGfsContent(harness(new Uint8Array(6), { body }), args, options())
+ await started
+ const [directory] = await retainedDirectories()
+ expect(
+ await fs.readFile(path.join(callerRoot, '.gfs-downloads', directory, 'source.partial'))
+ ).toEqual(Buffer.from([1, 2, 3]))
+ finish()
+ const result = await pending
+ expect(await fs.readFile(path.join(callerRoot, result.path))).toEqual(
+ Buffer.from([1, 2, 3, 4, 5, 6])
+ )
+ })
+
+ it('rejects a source over the configured general limit before content', async () => {
+ const request = harness(new Uint8Array(0), {
+ metadata: { bytes: 16 * 1024 * 1024 + 1 },
+ })
+ await expect(downloadGfsContent(request, args, options())).rejects.toMatchObject({
+ code: 'limit_exceeded',
+ })
+ expect(request).toHaveBeenCalledTimes(1)
+ })
+
+ it.each([
+ [{ 'x-gfs-version': '5' }, 'version_conflict'],
+ [{ 'x-gfs-uri': 'gfs://main/other' }, 'identity_mismatch'],
+ [{ 'content-length': '4' }, 'incomplete_response'],
+ [{ 'content-encoding': 'gzip' }, 'invalid_response'],
+ [{ 'content-length': undefined }, 'incomplete_response'],
+ ])('does not publish inconsistent content headers: %j', async (changed, code) => {
+ await expect(
+ downloadGfsContent(
+ harness(new Uint8Array(3), {
+ headers: changed,
+ }),
+ args,
+ options()
+ )
+ ).rejects.toMatchObject({ code })
+ expect(await retainedDirectories()).toEqual([])
+ })
+
+ it.each([
+ [2, 'incomplete_response'],
+ [4, 'limit_exceeded'],
+ ])('removes truncated or oversized content (%i bytes)', async (length, code) => {
+ const request = vi.fn(async (url: string) =>
+ url.includes('/content?')
+ ? new Response(new Uint8Array(length), { headers: contentHeaders(3) })
+ : metadata(3)
+ )
+ await expect(downloadGfsContent(request, args, options())).rejects.toMatchObject({ code })
+ expect(await retainedDirectories()).toEqual([])
+ })
+
+ it('cancels an in-progress stream, removes the partial, and frees the slot', async () => {
+ const controller = new AbortController()
+ let pulled!: () => void
+ const started = new Promise<void>(resolve => {
+ pulled = resolve
+ })
+ let count = 0
+ const body = new ReadableStream<Uint8Array>(
+ {
+ pull(stream) {
+ if (count++ === 0) stream.enqueue(new Uint8Array([1]))
+ else pulled()
+ },
+ },
+ { highWaterMark: 0 }
+ )
+ const pending = downloadGfsContent(
+ harness(new Uint8Array(2), { body }),
+ args,
+ options({ signal: controller.signal })
+ )
+ const rejected = expect(pending).rejects.toMatchObject({ code: 'cancelled' })
+ await started
+ controller.abort()
+ await rejected
+ expect(await retainedDirectories()).toEqual([])
+ await downloadGfsContent(harness(new Uint8Array([1])), args, options())
+ })
+
+ it('enforces an absolute deadline when the request callback ignores abort', async () => {
+ vi.useFakeTimers()
+ const request = vi.fn(() => new Promise<Response>(() => undefined))
+ const rejected = expect(
+ downloadGfsContent(request, args, options({ timeoutMs: 10 }))
+ ).rejects.toMatchObject({ code: 'timeout' })
+ await vi.advanceTimersByTimeAsync(11)
+ await rejected
+ })
+
+ it('applies its own metadata deadline even without an external signal', async () => {
+ vi.useFakeTimers()
+ const request = vi.fn(() => new Promise<Response>(() => undefined))
+ const rejected = expect(
+ readGfsMetadata(request, args, { timeoutMs: 10 })
+ ).rejects.toMatchObject({ code: 'timeout' })
+ await vi.advanceTimersByTimeAsync(11)
+ await rejected
+ expect(request).toHaveBeenCalledTimes(1)
+ })
+
+ it('rejects redirects and preserves authoritative GFSC denials', async () => {
+ const redirected = metadata(0)
+ Object.defineProperty(redirected, 'redirected', { value: true })
+ await expect(
+ downloadGfsContent(
+ vi.fn(async () => redirected),
+ args,
+ options()
+ )
+ ).rejects.toMatchObject({ code: 'invalid_response' })
+ await expect(
+ downloadGfsContent(
+ vi.fn(async () => new Response('forbidden', { status: 403 })),
+ args,
+ options()
+ )
+ ).rejects.toThrow('gfsc 403')
+ expect(await retainedDirectories()).toEqual([])
+ })
+})
diff --git a/mcp-host/src/internalTools/gfsContentDownload.ts b/mcp-host/src/internalTools/gfsContentDownload.ts
new file mode 100644
index 000000000..90171e268
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsContentDownload.ts
@@ -0,0 +1,245 @@
+import { createHash } from 'node:crypto'
+import { constants } from 'node:fs'
+import * as fs from 'node:fs/promises'
+import * as path from 'node:path'
+import { withAbort } from '../core/adapters/abortableLlmPort'
+import { logger } from '../logger'
+import { type GfsImageSource, VisualInputError } from '../visualInput/policy'
+import {
+ type GfsContentRequest,
+ assertContentHeaders,
+ collect,
+ metadataSnapshot,
+ normalizeRid,
+ requireOk,
+} from './gfsContentRead'
+import {
+ type GfsDownloadReceipt,
+ GfsDownloadStore,
+ GfsDownloadStoreError,
+ type GfsDownloadTransfer,
+} from './gfsDownloadStore'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
+
+const MAX_DOWNLOAD_TIME_MS = 600_000
+
+export type GfsDownloadErrorCode =
+ | 'cancelled'
+ | 'identity_mismatch'
+ | 'incomplete_response'
+ | 'invalid_response'
+ | 'limit_exceeded'
+ | 'storage_write_failed'
+ | 'timeout'
+ | 'version_conflict'
+
+export class GfsDownloadError extends Error {
+ constructor(readonly code: GfsDownloadErrorCode) {
+ super(`GFS download failed (${code})`)
+ this.name = 'GfsDownloadError'
+ }
+}
+
+export interface GfsDownloadOptions {
+ store: GfsDownloadStore
+ callerIdentity: string
+ callerWorkspacePath: string
+ signal?: AbortSignal
+ timeoutMs?: number
+ deadlineMs?: number
+ expectedVersion?: number
+}
+
+export type GfsDownloadResult = GfsDownloadReceipt
+
+function transferError(error: VisualInputError): GfsDownloadError {
+ if (error.code === 'version_conflict') return new GfsDownloadError('version_conflict')
+ if (error.code === 'identity_mismatch') return new GfsDownloadError('identity_mismatch')
+ if (error.code === 'incomplete_response') return new GfsDownloadError('incomplete_response')
+ if (error.code === 'limit_exceeded') return new GfsDownloadError('limit_exceeded')
+ if (error.code === 'timeout') return new GfsDownloadError('timeout')
+ if (error.code === 'cancelled') return new GfsDownloadError('cancelled')
+ return new GfsDownloadError('invalid_response')
+}
+
+function requireDownloadContentHeaders(
+ response: Response,
+ source: GfsImageSource,
+ size: number
+): void {
+ assertContentHeaders(response, source, size)
+ const length = response.headers.get('content-length')
+ if (
+ length === null ||
+ !/^\d+$/.test(length) ||
+ !Number.isSafeInteger(Number(length)) ||
+ Number(length) !== size
+ )
+ throw new GfsDownloadError('incomplete_response')
+}
+
+/** Stream an authenticated, version-bound GFS resource into the Host-owned store. */
+export async function downloadGfsContent(
+ request: GfsContentRequest,
+ args: { drive: string; resourceId: string },
+ options: GfsDownloadOptions
+): Promise<GfsDownloadResult> {
+ if (typeof args.drive !== 'string' || !args.drive || !normalizeRid(args.resourceId))
+ throw new GfsDownloadError('invalid_response')
+ if (options.signal?.aborted) throw new GfsDownloadError('cancelled')
+
+ const startedAt = Date.now()
+ const timeoutMs = Math.min(
+ options.timeoutMs ?? MAX_DOWNLOAD_TIME_MS,
+ MAX_DOWNLOAD_TIME_MS,
+ options.deadlineMs === undefined ? Number.POSITIVE_INFINITY : options.deadlineMs - startedAt
+ )
+ if (!Number.isSafeInteger(timeoutMs) || timeoutMs <= 0) throw new GfsDownloadError('timeout')
+ const deadlineMs = startedAt + timeoutMs
+
+ const controller = new AbortController()
+ let expired = false
+ const abort = () => controller.abort()
+ options.signal?.addEventListener('abort', abort, { once: true })
+ const timer = setTimeout(() => {
+ expired = true
+ controller.abort()
+ }, timeoutMs)
+ const signal = controller.signal
+ const resourcePath = `/v1/resources/${encodeURIComponent(args.resourceId)}`
+ const query = `?drive=${encodeURIComponent(args.drive)}`
+ const init: RequestInit = {
+ signal,
+ redirect: 'error',
+ headers: { 'accept-encoding': 'identity' },
+ }
+
+ let transfer: GfsDownloadTransfer | undefined
+ let destination: fs.FileHandle | undefined
+ let response: Response | undefined
+ let published = false
+ try {
+ const metadata = await withAbort(
+ () => request(`${resourcePath}${query}`, init, deadlineMs),
+ signal
+ )
+ await requireOk(metadata, signal)
+ let source: GfsDownloadReceipt['source']
+ let sizeBytes: number
+ try {
+ const snapshot = metadataSnapshot(
+ await collect(metadata, GFS_FILE_LIMITS.metadataBytes, signal),
+ args,
+ options.expectedVersion
+ )
+ source = snapshot.source
+ sizeBytes = snapshot.size
+ } catch (error) {
+ if (error instanceof VisualInputError) throw transferError(error)
+ throw error
+ }
+
+ transfer = await options.store.createTransfer({
+ callerIdentity: options.callerIdentity,
+ callerWorkspacePath: options.callerWorkspacePath,
+ source,
+ sizeBytes,
+ expiresAt: new Date(Date.now() + GFS_FILE_LIMITS.retentionMs).toISOString(),
+ })
+ const partialPath = path.join(options.callerWorkspacePath, transfer.partialPath)
+ destination = await fs.open(partialPath, constants.O_WRONLY | constants.O_NOFOLLOW)
+ const partialInfo = await destination.stat()
+ if (
+ !partialInfo.isFile() ||
+ partialInfo.isSymbolicLink?.() ||
+ partialInfo.size !== 0 ||
+ partialInfo.mode & 0o077
+ )
+ throw new GfsDownloadError('storage_write_failed')
+
+ response = await withAbort(
+ () => request(`${resourcePath}/content${query}`, init, deadlineMs),
+ signal
+ )
+ await requireOk(response, signal)
+ requireDownloadContentHeaders(response, source, sizeBytes)
+
+ const digest = createHash('sha256')
+ let bytes = 0
+ if (response.body) {
+ const reader = response.body.getReader()
+ const cancel = () => {
+ void reader.cancel().catch(() => undefined)
+ }
+ signal.addEventListener('abort', cancel, { once: true })
+ try {
+ for (;;) {
+ if (signal.aborted) throw new GfsDownloadError('cancelled')
+ const next = await reader.read()
+ if (signal.aborted) throw new GfsDownloadError('cancelled')
+ if (next.done) break
+ if (next.value.byteLength > sizeBytes - bytes)
+ throw new GfsDownloadError('limit_exceeded')
+ let offset = 0
+ while (offset < next.value.byteLength) {
+ const written = await destination.write(
+ next.value,
+ offset,
+ next.value.byteLength - offset
+ )
+ if (written.bytesWritten <= 0) throw new GfsDownloadError('storage_write_failed')
+ offset += written.bytesWritten
+ }
+ digest.update(next.value)
+ bytes += next.value.byteLength
+ }
+ } finally {
+ signal.removeEventListener('abort', cancel)
+ await reader.cancel().catch(() => undefined)
+ reader.releaseLock()
+ }
+ }
+
+ if (bytes !== sizeBytes) throw new GfsDownloadError('incomplete_response')
+ if (signal.aborted) throw new GfsDownloadError('cancelled')
+ await destination.sync()
+ await destination.close()
+ destination = undefined
+ const receipt = await options.store.publish(
+ transfer.id,
+ options.callerIdentity,
+ digest.digest('hex')
+ )
+ published = true
+ return receipt
+ } catch (error) {
+ if (signal.aborted) throw new GfsDownloadError(expired ? 'timeout' : 'cancelled')
+ if (error instanceof GfsDownloadError) throw error
+ if (error instanceof VisualInputError) throw transferError(error)
+ if (error instanceof GfsDownloadStoreError) throw error
+ if (
+ (error as Error)?.name === 'GfscHttpError' ||
+ /^gfsc \d{3}:/.test((error as Error)?.message ?? '')
+ )
+ throw error
+ throw new GfsDownloadError('storage_write_failed')
+ } finally {
+ clearTimeout(timer)
+ options.signal?.removeEventListener('abort', abort)
+ await destination?.close().catch(() => undefined)
+ await response?.body?.cancel().catch(() => undefined)
+ if (transfer && !published) {
+ try {
+ await options.store.fail(transfer.id, options.callerIdentity)
+ } catch (cleanupError) {
+ logger.warn(
+ {
+ component: 'GfsDownload',
+ err: { code: (cleanupError as NodeJS.ErrnoException).code },
+ },
+ 'GFS partial cleanup failed'
+ )
+ }
+ }
+ }
+}
diff --git a/mcp-host/src/internalTools/gfsContentRead.expectedVersion.test.ts b/mcp-host/src/internalTools/gfsContentRead.expectedVersion.test.ts
index 7f061162e..e493bb750 100644
--- a/mcp-host/src/internalTools/gfsContentRead.expectedVersion.test.ts
+++ b/mcp-host/src/internalTools/gfsContentRead.expectedVersion.test.ts
@@ -5,6 +5,7 @@
import { describe, expect, it, vi } from 'vitest'
import { VISUAL_INPUT_LIMITS, VisualInputBudget } from '../visualInput/policy'
import { readGfsContent } from './gfsContentRead'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
const FILE_ID = '1234567890abcdef1234567890abcdef'
const FILE_URI = `gfs://main/${FILE_ID}`
diff --git a/mcp-host/src/internalTools/gfsContentRead.ts b/mcp-host/src/internalTools/gfsContentRead.ts
index 101b924f5..e9fba1738 100644
--- a/mcp-host/src/internalTools/gfsContentRead.ts
+++ b/mcp-host/src/internalTools/gfsContentRead.ts
@@ -1,3 +1,4 @@
+import { withAbort } from '../core/adapters/abortableLlmPort'
import {
type GfsImageSource,
type MemoryReservation,
@@ -5,6 +6,7 @@ import {
VisualInputBudget,
VisualInputError,
} from '../visualInput/policy'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
import type { GfsFileContent, GfsReadOptions } from './gfsReadTypes'
export type GfsContentRequest = (
@@ -18,7 +20,7 @@ function cancelBody(body: ReadableStream<Uint8Array> | null): void {
}
/** Bounded consumption also observes cancellation for injected/local streams. */
-async function collect(
+export async function collect(
response: Response,
maximum: number,
signal: AbortSignal,
@@ -68,7 +70,7 @@ export async function boundedGfsErrorDetail(
}
}
-async function requireOk(response: Response, signal: AbortSignal): Promise<void> {
+export async function requireOk(response: Response, signal: AbortSignal): Promise<void> {
if (response.redirected) {
cancelBody(response.body)
throw new VisualInputError('invalid_response')
@@ -89,11 +91,17 @@ export function normalizeRid(value: unknown): string | null {
return /^[a-f0-9]{32}$/.test(normalized) ? normalized : null
}
-function metadataSnapshot(
+export interface GfsMetadataSnapshot {
+ source: GfsImageSource
+ size: number
+}
+
+export function metadataSnapshot(
bytes: Buffer,
args: { drive: string; resourceId: string },
- expectedVersion?: number
-): { source: GfsImageSource; size: number } {
+ expectedVersion?: number,
+ maxBytes = GFS_FILE_LIMITS.maxFileBytes
+): GfsMetadataSnapshot {
let envelope: { ok?: unknown; data?: unknown }
try {
envelope = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes))
@@ -125,8 +133,7 @@ function metadataSnapshot(
if (expectedVersion !== undefined && data.version !== expectedVersion)
throw new VisualInputError('version_conflict')
if (data.kind !== 'file') throw new VisualInputError('unsupported_format')
- if ((data.bytes as number) > VISUAL_INPUT_LIMITS.fileBytes)
- throw new VisualInputError('limit_exceeded')
+ if ((data.bytes as number) > maxBytes) throw new VisualInputError('limit_exceeded')
return {
source: {
kind: 'gfs',
@@ -140,7 +147,11 @@ function metadataSnapshot(
}
}
-function assertContentHeaders(response: Response, source: GfsImageSource, size: number): void {
+export function assertContentHeaders(
+ response: Response,
+ source: GfsImageSource,
+ size: number
+): void {
// Content for another resource is not a newer version of this one.
if (response.headers.get('x-gfs-uri') !== source.gfsUri)
throw new VisualInputError('identity_mismatch')
@@ -196,18 +207,29 @@ export async function readGfsContent(
headers: { 'accept-encoding': 'identity' },
}
try {
- const metadataReservation = budget.reserve(2 * VISUAL_INPUT_LIMITS.metadataBytes)
let snapshot: ReturnType<typeof metadataSnapshot>
- try {
- const metadata = await request(`${path}${query}`, init, deadlineMs)
- await requireOk(metadata, signal)
- snapshot = metadataSnapshot(
- await collect(metadata, VISUAL_INPUT_LIMITS.metadataBytes, signal),
- args,
- options.expectedVersion
+ if (options.metadataSnapshot) {
+ snapshot = options.metadataSnapshot
+ if (
+ options.expectedVersion !== undefined &&
+ snapshot.source.version !== options.expectedVersion
)
- } finally {
- metadataReservation.release()
+ throw new VisualInputError('version_conflict')
+ if (snapshot.size > GFS_FILE_LIMITS.maxFileBytes) throw new VisualInputError('limit_exceeded')
+ } else {
+ const metadataReservation = budget.reserve(2 * VISUAL_INPUT_LIMITS.metadataBytes)
+ try {
+ const metadata = await request(`${path}${query}`, init, deadlineMs)
+ await requireOk(metadata, signal)
+ snapshot = metadataSnapshot(
+ await collect(metadata, VISUAL_INPUT_LIMITS.metadataBytes, signal),
+ args,
+ options.expectedVersion,
+ GFS_FILE_LIMITS.maxFileBytes
+ )
+ } finally {
+ metadataReservation.release()
+ }
}
if (signal.aborted) throw new VisualInputError('cancelled')
if (snapshot.size > budget.remainingReadBytes) throw new VisualInputError('limit_exceeded')
@@ -236,3 +258,56 @@ export async function readGfsContent(
if (!handedOff) reservation?.release()
}
}
+
+export async function readGfsMetadata(
+ request: GfsContentRequest,
+ args: { drive: string; resourceId: string },
+ options: {
+ signal?: AbortSignal
+ timeoutMs?: number
+ deadlineMs?: number
+ expectedVersion?: number
+ maxBytes?: number
+ }
+): Promise<GfsMetadataSnapshot> {
+ if (options.signal?.aborted) throw new VisualInputError('cancelled')
+ const timeoutMs = Math.min(
+ options.timeoutMs ?? VISUAL_INPUT_LIMITS.readTimeoutMs,
+ VISUAL_INPUT_LIMITS.readTimeoutMs,
+ options.deadlineMs === undefined ? Number.POSITIVE_INFINITY : options.deadlineMs - Date.now()
+ )
+ if (!Number.isSafeInteger(timeoutMs) || timeoutMs <= 0) throw new VisualInputError('timeout')
+ const controller = new AbortController()
+ let expired = false
+ const abort = () => controller.abort()
+ options.signal?.addEventListener('abort', abort, { once: true })
+ const timer = setTimeout(() => {
+ expired = true
+ controller.abort()
+ }, timeoutMs)
+ const signal = controller.signal
+ try {
+ const metadata = await withAbort(
+ () =>
+ request(
+ `/v1/resources/${encodeURIComponent(args.resourceId)}?drive=${encodeURIComponent(args.drive)}`,
+ { signal, redirect: 'error', headers: { 'accept-encoding': 'identity' } },
+ Date.now() + timeoutMs
+ ),
+ signal
+ )
+ await requireOk(metadata, signal)
+ return metadataSnapshot(
+ await collect(metadata, GFS_FILE_LIMITS.metadataBytes, signal),
+ args,
+ options.expectedVersion,
+ options.maxBytes
+ )
+ } catch (error) {
+ if (signal.aborted) throw new VisualInputError(expired ? 'timeout' : 'cancelled')
+ throw error
+ } finally {
+ clearTimeout(timer)
+ options.signal?.removeEventListener('abort', abort)
+ }
+}
diff --git a/mcp-host/src/internalTools/gfsDownloadMetrics.test.ts b/mcp-host/src/internalTools/gfsDownloadMetrics.test.ts
new file mode 100644
index 000000000..5411038c9
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsDownloadMetrics.test.ts
@@ -0,0 +1,46 @@
+import { describe, expect, it } from 'vitest'
+import { register } from 'prom-client'
+import {
+ enterGfsDownloadTransfer,
+ exitGfsDownloadTransfer,
+ recordGfsDownloadAdmission,
+ recordGfsDownloadExpiry,
+ recordGfsDownloadQuota,
+ recordGfsDownloadTransfer,
+ recordGfsShellOutputLimit,
+} from './gfsDownloadMetrics'
+
+describe('GFS download metrics', () => {
+ it('registers fixed-cardinality instruments on the global metrics endpoint', async () => {
+ const scraped = await register.metrics()
+ expect(scraped).toContain('clerum_gfs_download_admissions_total')
+ expect(scraped).toContain('clerum_gfs_download_transfers_total')
+ expect(scraped).toContain('clerum_gfs_download_duration_seconds')
+ expect(scraped).toContain('clerum_gfs_download_active')
+ expect(scraped).toContain('clerum_gfs_shell_output_limits_total')
+ expect(scraped).toContain('clerum_gfs_download_quota_total')
+ expect(scraped).toContain('clerum_gfs_download_expiry_total')
+ })
+
+ it('records admission, transfer, active count, and shell-limit outcomes', async () => {
+ recordGfsDownloadAdmission('workspace_attempt')
+ enterGfsDownloadTransfer()
+ recordGfsDownloadTransfer('success', 0.25)
+ exitGfsDownloadTransfer()
+ recordGfsShellOutputLimit('output_limit_exceeded')
+ recordGfsDownloadQuota('caller', 'storage_bytes')
+ recordGfsDownloadExpiry('cleanup_failed')
+
+ const scraped = await register.metrics()
+ expect(scraped).toContain('clerum_gfs_download_admissions_total{outcome="workspace_attempt"}')
+ expect(scraped).toContain('clerum_gfs_download_transfers_total{outcome="success"}')
+ expect(scraped).toContain('clerum_gfs_download_active 0')
+ expect(scraped).toContain(
+ 'clerum_gfs_shell_output_limits_total{outcome="output_limit_exceeded"}'
+ )
+ expect(scraped).toContain(
+ 'clerum_gfs_download_quota_total{scope="caller",reason="storage_bytes"}'
+ )
+ expect(scraped).toContain('clerum_gfs_download_expiry_total{outcome="cleanup_failed"}')
+ })
+})
diff --git a/mcp-host/src/internalTools/gfsDownloadMetrics.ts b/mcp-host/src/internalTools/gfsDownloadMetrics.ts
new file mode 100644
index 000000000..ea13ac94d
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsDownloadMetrics.ts
@@ -0,0 +1,125 @@
+import { Counter, Gauge, Histogram, register } from 'prom-client'
+
+function getOrCreateCounter<Label extends string>(options: {
+ name: string
+ help: string
+ labelNames: Label[]
+}): Counter<Label> {
+ const existing = register.getSingleMetric(options.name)
+ if (existing) return existing as Counter<Label>
+ return new Counter<Label>({ ...options, registers: [register] })
+}
+
+function getOrCreateHistogram<Label extends string>(options: {
+ name: string
+ help: string
+ labelNames: Label[]
+ buckets: number[]
+}): Histogram<Label> {
+ const existing = register.getSingleMetric(options.name)
+ if (existing) return existing as Histogram<Label>
+ return new Histogram<Label>({ ...options, registers: [register] })
+}
+
+function getOrCreateGauge(options: { name: string; help: string }): Gauge<string> {
+ const existing = register.getSingleMetric(options.name)
+ if (existing) return existing as Gauge<string>
+ return new Gauge<string>({ ...options, registers: [register] })
+}
+
+export type GfsDownloadAdmissionOutcome =
+ | 'inline_attempt'
+ | 'workspace_attempt'
+ | 'workspace_unavailable'
+ | 'limit_exceeded'
+
+export type GfsDownloadTransferOutcome = 'success' | 'failure'
+
+export type GfsDownloadShellLimitOutcome = 'output_limit_exceeded'
+
+export type GfsDownloadQuotaScope = 'host' | 'caller'
+
+export type GfsDownloadQuotaReason =
+ | 'storage_bytes'
+ | 'retained_files'
+ | 'active_downloads'
+ | 'free_space'
+
+export type GfsDownloadExpiryOutcome = 'expired_removed' | 'cleanup_failed' | 'sweep_failed'
+
+const gfsDownloadAdmissionsTotal = getOrCreateCounter({
+ name: 'clerum_gfs_download_admissions_total',
+ help: 'GFS source admission decisions by fixed outcome.',
+ labelNames: ['outcome'] as const as Array<'outcome'>,
+})
+
+const gfsDownloadTransfersTotal = getOrCreateCounter({
+ name: 'clerum_gfs_download_transfers_total',
+ help: 'Governed GFS workspace transfers by fixed outcome.',
+ labelNames: ['outcome'] as const as Array<'outcome'>,
+})
+
+const gfsDownloadDurationSeconds = getOrCreateHistogram({
+ name: 'clerum_gfs_download_duration_seconds',
+ help: 'Governed GFS workspace transfer duration.',
+ labelNames: [] as const as Array<never>,
+ buckets: [0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30],
+})
+
+const gfsDownloadActive = getOrCreateGauge({
+ name: 'clerum_gfs_download_active',
+ help: 'Governed GFS workspace transfers currently reported by the tool boundary.',
+})
+
+const gfsShellOutputLimitsTotal = getOrCreateCounter({
+ name: 'clerum_gfs_shell_output_limits_total',
+ help: 'Approved local GFS processing commands stopped by fixed output limits.',
+ labelNames: ['outcome'] as const as Array<'outcome'>,
+})
+
+const gfsDownloadQuotaTotal = getOrCreateCounter({
+ name: 'clerum_gfs_download_quota_total',
+ help: 'Governed GFS workspace transfer quota decisions by fixed scope and reason.',
+ labelNames: ['scope', 'reason'] as const as Array<'scope' | 'reason'>,
+})
+
+const gfsDownloadExpiryTotal = getOrCreateCounter({
+ name: 'clerum_gfs_download_expiry_total',
+ help: 'Governed GFS workspace-copy expiry and cleanup outcomes.',
+ labelNames: ['outcome'] as const as Array<'outcome'>,
+})
+
+export function recordGfsDownloadAdmission(outcome: GfsDownloadAdmissionOutcome): void {
+ gfsDownloadAdmissionsTotal.labels(outcome).inc()
+}
+
+export function recordGfsDownloadTransfer(
+ outcome: GfsDownloadTransferOutcome,
+ durationSeconds: number
+): void {
+ gfsDownloadTransfersTotal.labels(outcome).inc()
+ gfsDownloadDurationSeconds.observe(durationSeconds)
+}
+
+export function enterGfsDownloadTransfer(): void {
+ gfsDownloadActive.inc()
+}
+
+export function exitGfsDownloadTransfer(): void {
+ gfsDownloadActive.dec()
+}
+
+export function recordGfsShellOutputLimit(outcome: GfsDownloadShellLimitOutcome): void {
+ gfsShellOutputLimitsTotal.labels(outcome).inc()
+}
+
+export function recordGfsDownloadQuota(
+ scope: GfsDownloadQuotaScope,
+ reason: GfsDownloadQuotaReason
+): void {
+ gfsDownloadQuotaTotal.labels(scope, reason).inc()
+}
+
+export function recordGfsDownloadExpiry(outcome: GfsDownloadExpiryOutcome): void {
+ gfsDownloadExpiryTotal.labels(outcome).inc()
+}
diff --git a/mcp-host/src/internalTools/gfsDownloadStore.test.ts b/mcp-host/src/internalTools/gfsDownloadStore.test.ts
new file mode 100644
index 000000000..fc4011a5d
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsDownloadStore.test.ts
@@ -0,0 +1,287 @@
+import { afterEach, beforeEach, describe, expect, it } from 'vitest'
+import { createHash } from 'node:crypto'
+import * as fs from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import * as path from 'node:path'
+import type { GfsImageSource } from '../visualInput/policy'
+import { GfsDownloadStore } from './gfsDownloadStore'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
+
+const source: GfsImageSource = {
+ kind: 'gfs',
+ drive: 'main',
+ resourceId: 'a'.repeat(32),
+ gfsUri: `gfs://main/${'a'.repeat(32)}`,
+ name: 'input.csv',
+ version: 7,
+}
+
+let hostRoot: string
+let callerRoot: string
+let store: GfsDownloadStore
+
+beforeEach(async () => {
+ hostRoot = await fs.mkdtemp(path.join(tmpdir(), 'gfs-download-store-'))
+ callerRoot = path.join(hostRoot, 'users', 'caller-a')
+ await fs.mkdir(callerRoot, { recursive: true, mode: 0o700 })
+ store = new GfsDownloadStore(hostRoot)
+ await store.initialize()
+})
+
+afterEach(async () => {
+ await store.close().catch(() => undefined)
+ await fs.rm(hostRoot, { recursive: true, force: true })
+})
+
+describe('GFS download store', () => {
+ it('creates a caller-relative transfer and publishes an exact completed record', async () => {
+ const expiresAt = new Date(Date.now() + 60_000).toISOString()
+ const transfer = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 7,
+ expiresAt,
+ })
+
+ expect(transfer.path).toMatch(/^\.gfs-downloads\/input-[a-f0-9-]+\/source$/)
+ expect(path.isAbsolute(transfer.path)).toBe(false)
+ expect(await fs.realpath(path.dirname(path.join(callerRoot, transfer.path)))).toBe(
+ await fs.realpath(
+ path.join(callerRoot, '.gfs-downloads', path.basename(path.dirname(transfer.path)))
+ )
+ )
+ expect((await fs.stat(path.join(callerRoot, `${transfer.path}.partial`))).mode & 0o777).toBe(
+ 0o600
+ )
+
+ await fs.writeFile(path.join(callerRoot, `${transfer.path}.partial`), 'fixture')
+ const receipt = await store.publish(
+ transfer.id,
+ 'caller-a',
+ createHash('sha256').update('fixture').digest('hex')
+ )
+
+ expect(receipt).toMatchObject({
+ source,
+ path: transfer.path,
+ sizeBytes: 7,
+ sha256: createHash('sha256').update('fixture').digest('hex'),
+ expiresAt,
+ })
+ await expect(fs.stat(path.join(callerRoot, `${transfer.path}.partial`))).rejects.toMatchObject({
+ code: 'ENOENT',
+ })
+ expect((await fs.stat(path.join(callerRoot, transfer.path))).mode & 0o777).toBe(0o600)
+ })
+
+ it('binds receipts to the caller and fails closed on expiry or missing bytes', async () => {
+ const transfer = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ })
+ await fs.writeFile(path.join(callerRoot, `${transfer.path}.partial`), 'x')
+ const receipt = await store.publish(
+ transfer.id,
+ 'caller-a',
+ createHash('sha256').update('x').digest('hex')
+ )
+
+ await expect(store.inspect(receipt.path, 'caller-b')).rejects.toMatchObject({
+ code: 'caller_mismatch',
+ })
+ await expect(store.inspect(receipt.path, 'caller-a')).resolves.toMatchObject({
+ id: transfer.id,
+ })
+
+ await fs.writeFile(path.join(callerRoot, receipt.path), 'y')
+ await expect(store.inspect(receipt.path, 'caller-a')).rejects.toMatchObject({
+ code: 'download_missing',
+ })
+ await fs.writeFile(path.join(callerRoot, receipt.path), 'x')
+ await expect(store.inspect(receipt.path, 'caller-a')).resolves.toMatchObject({
+ id: transfer.id,
+ })
+
+ const record = store.debugRecord(transfer.id)!
+ record.expiresAt = new Date(Date.now() - 1).toISOString()
+ await store.debugPersist()
+ await expect(store.inspect(receipt.path, 'caller-a')).rejects.toMatchObject({
+ code: 'download_expired',
+ })
+
+ await fs.unlink(path.join(callerRoot, receipt.path))
+ record.expiresAt = new Date(Date.now() + 60_000).toISOString()
+ await store.debugPersist()
+ await expect(store.inspect(receipt.path, 'caller-a')).rejects.toMatchObject({
+ code: 'download_missing',
+ })
+ })
+
+ it('enforces caller and aggregate byte/file quotas from the durable ledger', async () => {
+ await expect(
+ store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: GFS_FILE_LIMITS.maxFileBytes + 1,
+ expiresAt: new Date().toISOString(),
+ })
+ ).rejects.toMatchObject({ code: 'host_quota_exceeded' })
+
+ const bytes = Buffer.alloc(1024)
+ for (let index = 0; index < GFS_FILE_LIMITS.callerRetainedFiles; index += 1) {
+ const transfer = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1024,
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ })
+ await fs.writeFile(path.join(callerRoot, transfer.partialPath), bytes)
+ await store.publish(transfer.id, 'caller-a', createHash('sha256').update(bytes).digest('hex'))
+ }
+
+ await expect(
+ store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1024,
+ expiresAt: new Date().toISOString(),
+ })
+ ).rejects.toMatchObject({ code: 'caller_quota_exceeded' })
+ expect(store.debugUsage()).toMatchObject({
+ bytes: GFS_FILE_LIMITS.callerRetainedFiles * 1024,
+ files: GFS_FILE_LIMITS.callerRetainedFiles,
+ callerBytes: { 'caller-a': GFS_FILE_LIMITS.callerRetainedFiles * 1024 },
+ callerFiles: { 'caller-a': GFS_FILE_LIMITS.callerRetainedFiles },
+ })
+ })
+
+ it('reconstructs completed charges after process restart', async () => {
+ const transfer = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1024,
+ expiresAt: new Date(Date.now() + 60_000).toISOString(),
+ })
+ await fs.writeFile(path.join(callerRoot, transfer.partialPath), Buffer.alloc(1024))
+ await store.publish(
+ transfer.id,
+ 'caller-a',
+ createHash('sha256').update(Buffer.alloc(1024)).digest('hex')
+ )
+
+ await store.close()
+ const reopened = new GfsDownloadStore(hostRoot)
+ await reopened.initialize()
+ expect(reopened.debugUsage()).toEqual({
+ bytes: 1024,
+ files: 1,
+ callerBytes: { 'caller-a': 1024 },
+ callerFiles: { 'caller-a': 1 },
+ })
+
+ reopened.debugRecord(transfer.id)!.sizeBytes = GFS_FILE_LIMITS.callerStorageBytes - 512
+ await reopened.debugPersist()
+ await expect(
+ reopened.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1024,
+ expiresAt: new Date().toISOString(),
+ })
+ ).rejects.toMatchObject({ code: 'caller_quota_exceeded' })
+ await reopened.close()
+ })
+
+ it('releases a failed transfer when its exact directory is removed', async () => {
+ const transfer = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1024,
+ expiresAt: new Date().toISOString(),
+ })
+ await store.fail(transfer.id, 'caller-a')
+ expect(store.debugUsage()).toEqual({ bytes: 0, files: 0, callerBytes: {}, callerFiles: {} })
+ await expect(fs.stat(path.join(callerRoot, transfer.path))).rejects.toMatchObject({
+ code: 'ENOENT',
+ })
+ })
+
+ it('reserves active-transfer slots atomically per caller and Host', async () => {
+ const first = await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date().toISOString(),
+ })
+
+ await expect(
+ store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date().toISOString(),
+ })
+ ).rejects.toMatchObject({ code: 'download_busy' })
+
+ const callerBRoot = path.join(hostRoot, 'users', 'caller-b')
+ await fs.mkdir(callerBRoot, { recursive: true, mode: 0o700 })
+ const second = await store.createTransfer({
+ callerIdentity: 'caller-b',
+ callerWorkspacePath: callerBRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date().toISOString(),
+ })
+
+ const callerCRoot = path.join(hostRoot, 'users', 'caller-c')
+ await fs.mkdir(callerCRoot, { recursive: true, mode: 0o700 })
+ await expect(
+ store.createTransfer({
+ callerIdentity: 'caller-c',
+ callerWorkspacePath: callerCRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date().toISOString(),
+ })
+ ).rejects.toMatchObject({ code: 'download_busy' })
+
+ await store.fail(first.id, 'caller-a')
+ await store.fail(second.id, 'caller-b')
+ await store.createTransfer({
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerRoot,
+ source,
+ sizeBytes: 1,
+ expiresAt: new Date().toISOString(),
+ })
+ })
+
+ it('rejects a second live writer and accepts initialization after close', async () => {
+ const second = new GfsDownloadStore(hostRoot)
+ await expect(second.initialize()).rejects.toMatchObject({ code: 'writer_locked' })
+ await store.close()
+ await second.initialize()
+ await second.close()
+ })
+
+ it('fails closed on a corrupt durable ledger', async () => {
+ await store.close()
+ await fs.writeFile(path.join(hostRoot, '.gfs-download-store', 'ledger-v1.json'), '{bad')
+ const reopened = new GfsDownloadStore(hostRoot)
+ await expect(reopened.initialize()).rejects.toMatchObject({
+ code: 'corrupt_store_ledger',
+ })
+ })
+})
diff --git a/mcp-host/src/internalTools/gfsDownloadStore.ts b/mcp-host/src/internalTools/gfsDownloadStore.ts
new file mode 100644
index 000000000..319a61885
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsDownloadStore.ts
@@ -0,0 +1,670 @@
+import { createHash, randomUUID } from 'node:crypto'
+import { constants } from 'node:fs'
+import * as fs from 'node:fs/promises'
+import * as path from 'node:path'
+import type { GfsImageSource } from '../visualInput/policy'
+import {
+ GFS_FILE_LIMITS,
+ GFS_HOST_ACTIVE_DOWNLOADS,
+ GFS_HOST_RETAINED_FILES,
+} from './gfsFilePolicy'
+
+export type GfsDownloadStoreErrorCode =
+ | 'caller_mismatch'
+ | 'caller_quota_exceeded'
+ | 'corrupt_store_ledger'
+ | 'download_expired'
+ | 'download_missing'
+ | 'download_busy'
+ | 'host_quota_exceeded'
+ | 'storage_write_failed'
+ | 'unsupported_store_schema'
+ | 'workspace_unavailable'
+ | 'writer_locked'
+
+export class GfsDownloadStoreError extends Error {
+ constructor(readonly code: GfsDownloadStoreErrorCode) {
+ super(`GFS download store failed (${code})`)
+ this.name = 'GfsDownloadStoreError'
+ }
+}
+
+export interface GfsDownloadRecord {
+ id: string
+ callerIdentity: string
+ source: GfsImageSource
+ directory: string
+ /** Caller-workspace-relative path safe for a model-visible receipt. */
+ path: string
+ /** Host-workspace-relative path used only inside the store. */
+ hostPath: string
+ sizeBytes: number
+ sha256?: string
+ createdAt: string
+ expiresAt: string
+ state: 'transferring' | 'completed' | 'missing' | 'cleanup_failed'
+}
+
+export interface GfsDownloadTransfer {
+ id: string
+ path: string
+ partialPath: string
+ sizeBytes: number
+ expiresAt: string
+}
+
+export interface GfsDownloadReceipt {
+ id: string
+ source: GfsImageSource
+ path: string
+ sizeBytes: number
+ sha256: string
+ expiresAt: string
+}
+
+interface StoreLedger {
+ schemaVersion: 1
+ records: Record<string, GfsDownloadRecord>
+}
+
+const UUID_RE = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i
+const SHA256_RE = /^[0-9a-f]{64}$/
+
+function parseLedger(raw: string): StoreLedger {
+ let parsed: unknown
+ try {
+ parsed = JSON.parse(raw)
+ } catch {
+ throw new GfsDownloadStoreError('corrupt_store_ledger')
+ }
+ if (
+ typeof parsed !== 'object' ||
+ parsed === null ||
+ (parsed as { schemaVersion?: unknown }).schemaVersion !== 1 ||
+ typeof (parsed as { records?: unknown }).records !== 'object' ||
+ (parsed as { records: unknown }).records === null
+ )
+ throw new GfsDownloadStoreError('corrupt_store_ledger')
+
+ const records = (parsed as { records: Record<string, unknown> }).records
+ const result: StoreLedger = { schemaVersion: 1, records: {} }
+ for (const [id, value] of Object.entries(records)) {
+ if (!UUID_RE.test(id) || typeof value !== 'object' || value === null)
+ throw new GfsDownloadStoreError('corrupt_store_ledger')
+ const record = value as GfsDownloadRecord
+ const source = record.source
+ if (
+ !UUID_RE.test(record.id) ||
+ record.id !== id ||
+ typeof record.callerIdentity !== 'string' ||
+ record.callerIdentity.length === 0 ||
+ typeof source?.kind !== 'string' ||
+ source.kind !== 'gfs' ||
+ typeof source.drive !== 'string' ||
+ typeof source.resourceId !== 'string' ||
+ typeof source.gfsUri !== 'string' ||
+ typeof source.name !== 'string' ||
+ !Number.isSafeInteger(source.version) ||
+ source.version < 0 ||
+ typeof record.directory !== 'string' ||
+ path.isAbsolute(record.directory) ||
+ record.directory.includes('..') ||
+ record.directory !==
+ path.join(
+ path.dirname(path.dirname(record.directory)),
+ '.gfs-downloads',
+ `input-${record.id}`
+ ) ||
+ typeof record.path !== 'string' ||
+ record.path !== path.join('.gfs-downloads', `input-${record.id}`, 'source') ||
+ typeof record.hostPath !== 'string' ||
+ record.hostPath !== path.join(record.directory, 'source') ||
+ !Number.isSafeInteger(record.sizeBytes) ||
+ record.sizeBytes < 0 ||
+ !['transferring', 'completed', 'missing', 'cleanup_failed'].includes(record.state) ||
+ (record.sha256 !== undefined && !SHA256_RE.test(record.sha256)) ||
+ (record.state === 'completed' && !SHA256_RE.test(record.sha256 ?? '')) ||
+ !Number.isFinite(Date.parse(record.createdAt)) ||
+ !Number.isFinite(Date.parse(record.expiresAt))
+ )
+ throw new GfsDownloadStoreError('corrupt_store_ledger')
+ result.records[id] = record
+ }
+ return result
+}
+
+interface Usage {
+ bytes: number
+ files: number
+ callerBytes: Record<string, number>
+ callerFiles: Record<string, number>
+}
+
+function isWithinDirectory(child: string, parent: string): boolean {
+ const relative = path.relative(parent, child)
+ return relative !== '' && !relative.startsWith('..') && !path.isAbsolute(relative)
+}
+
+function usageFor(records: Iterable<GfsDownloadRecord>): Usage {
+ const usage: Usage = { bytes: 0, files: 0, callerBytes: {}, callerFiles: {} }
+ for (const record of records) {
+ usage.bytes += record.sizeBytes
+ usage.files += 1
+ usage.callerBytes[record.callerIdentity] =
+ (usage.callerBytes[record.callerIdentity] ?? 0) + record.sizeBytes
+ usage.callerFiles[record.callerIdentity] = (usage.callerFiles[record.callerIdentity] ?? 0) + 1
+ }
+ return usage
+}
+
+async function syncDirectory(directory: string): Promise<void> {
+ const handle = await fs.open(directory, 'r')
+ try {
+ await handle.sync()
+ } finally {
+ await handle.close()
+ }
+}
+
+async function sha256File(path: string): Promise<string> {
+ const digest = createHash('sha256')
+ const handle = await fs.open(
+ path,
+ constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK
+ )
+ try {
+ const chunk = Buffer.alloc(64 * 1024)
+ let position = 0
+ for (;;) {
+ const { bytesRead } = await handle.read(chunk, 0, chunk.byteLength, position)
+ if (bytesRead === 0) break
+ digest.update(chunk.subarray(0, bytesRead))
+ position += bytesRead
+ }
+ } finally {
+ await handle.close()
+ }
+ return digest.digest('hex')
+}
+
+/** One Host/PVC-owned durable store; task registries receive bound handles. */
+export class GfsDownloadStore {
+ private hostRoot: string
+ private storeRoot: string
+ private ledgerPath: string
+ private lockPath: string
+ private writerLeaseId?: string
+ private ledger: StoreLedger = { schemaVersion: 1, records: {} }
+ private readonly activeByCaller = new Map<string, number>()
+ private readonly activeIds = new Set<string>()
+ private active = 0
+ private mutationTail = Promise.resolve()
+ private initialized = false
+ private closing = false
+ private unsafe = false
+ private static readonly writerOwners = new Map<string, string>()
+
+ constructor(hostRoot: string) {
+ this.hostRoot = path.resolve(hostRoot)
+ this.storeRoot = path.join(this.hostRoot, '.gfs-download-store')
+ this.ledgerPath = path.join(this.storeRoot, 'ledger-v1.json')
+ this.lockPath = path.join(this.storeRoot, 'writer.lock')
+ }
+
+ async initialize(): Promise<void> {
+ if (this.initialized) return
+ await fs.mkdir(this.hostRoot, { recursive: true, mode: 0o700 })
+ this.hostRoot = await fs.realpath(this.hostRoot)
+ this.storeRoot = path.join(this.hostRoot, '.gfs-download-store')
+ this.ledgerPath = path.join(this.storeRoot, 'ledger-v1.json')
+ const hostInfo = await fs.lstat(this.hostRoot)
+ if (!hostInfo.isDirectory() || hostInfo.isSymbolicLink())
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ await fs.mkdir(this.storeRoot, { recursive: true, mode: 0o700 })
+ const storeInfo = await fs.lstat(this.storeRoot)
+ if (!storeInfo.isDirectory() || storeInfo.isSymbolicLink() || storeInfo.mode & 0o077)
+ throw new GfsDownloadStoreError('workspace_unavailable')
+
+ await this.acquireWriterLease()
+ let raw = ''
+ try {
+ try {
+ raw = await fs.readFile(this.ledgerPath, 'utf8')
+ } catch (error) {
+ if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error
+ }
+ if (raw) this.ledger = parseLedger(raw)
+ await this.reconcile()
+ this.initialized = true
+ await this.cleanupExpired()
+ await this.persist()
+ } catch (error) {
+ this.initialized = false
+ await this.releaseWriterLease()
+ throw error
+ }
+ }
+
+ async createTransfer(input: {
+ callerIdentity: string
+ callerWorkspacePath: string
+ source: GfsImageSource
+ sizeBytes: number
+ expiresAt: string
+ }): Promise<GfsDownloadTransfer> {
+ this.assertInitialized()
+ if (this.closing) throw new GfsDownloadStoreError('download_busy')
+ if (
+ !Number.isSafeInteger(input.sizeBytes) ||
+ input.sizeBytes < 0 ||
+ input.sizeBytes > GFS_FILE_LIMITS.maxFileBytes
+ )
+ throw new GfsDownloadStoreError('host_quota_exceeded')
+
+ const callerRoot = await this.validateCallerRoot(input.callerWorkspacePath)
+ const id = randomUUID()
+ const relativeDirectory = path.join(
+ path.relative(this.hostRoot, callerRoot),
+ '.gfs-downloads',
+ `input-${id}`
+ )
+ const absoluteDirectory = path.join(this.hostRoot, relativeDirectory)
+ const hostPath = path.join(relativeDirectory, 'source')
+ const callerPath = path.relative(callerRoot, path.join(this.hostRoot, hostPath))
+ const record: GfsDownloadRecord = {
+ id,
+ callerIdentity: input.callerIdentity,
+ source: input.source,
+ directory: relativeDirectory,
+ path: callerPath,
+ hostPath,
+ sizeBytes: input.sizeBytes,
+ createdAt: new Date().toISOString(),
+ expiresAt: input.expiresAt,
+ state: 'transferring',
+ }
+
+ await this.serialize(async () => {
+ const usage = usageFor(Object.values(this.ledger.records))
+ const callerBytes = usage.callerBytes[input.callerIdentity] ?? 0
+ if (usage.bytes + input.sizeBytes > GFS_FILE_LIMITS.storageBytes)
+ throw new GfsDownloadStoreError('host_quota_exceeded')
+ if (callerBytes + input.sizeBytes > GFS_FILE_LIMITS.callerStorageBytes)
+ throw new GfsDownloadStoreError('caller_quota_exceeded')
+ if (usage.files >= GFS_HOST_RETAINED_FILES)
+ throw new GfsDownloadStoreError('host_quota_exceeded')
+ if ((usage.callerFiles[input.callerIdentity] ?? 0) >= GFS_FILE_LIMITS.callerRetainedFiles)
+ throw new GfsDownloadStoreError('caller_quota_exceeded')
+ if (this.active >= GFS_HOST_ACTIVE_DOWNLOADS) throw new GfsDownloadStoreError('download_busy')
+ if (
+ (this.activeByCaller.get(input.callerIdentity) ?? 0) >=
+ GFS_FILE_LIMITS.callerActiveDownloads
+ )
+ throw new GfsDownloadStoreError('download_busy')
+
+ const space = await fs.statfs(this.storeRoot, { bigint: true })
+ if (space.bavail * space.bsize < BigInt(input.sizeBytes + 16 * 1024 * 1024))
+ throw new GfsDownloadStoreError('host_quota_exceeded')
+
+ this.ledger.records[id] = record
+ await this.persist()
+ this.active += 1
+ this.activeIds.add(id)
+ this.activeByCaller.set(
+ input.callerIdentity,
+ (this.activeByCaller.get(input.callerIdentity) ?? 0) + 1
+ )
+ return undefined
+ })
+
+ try {
+ await fs.mkdir(path.dirname(path.join(this.hostRoot, record.hostPath)), {
+ recursive: true,
+ mode: 0o700,
+ })
+ await fs.mkdir(absoluteDirectory, { recursive: true, mode: 0o700 })
+ const partial = await fs.open(
+ path.join(this.hostRoot, `${record.hostPath}.partial`),
+ constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW,
+ 0o600
+ )
+ await partial.close()
+ } catch (error) {
+ try {
+ await this.serialize(async () => {
+ delete this.ledger.records[id]
+ await this.persist()
+ })
+ await fs.rm(absoluteDirectory, { recursive: true, force: true }).catch(() => undefined)
+ } finally {
+ this.releaseActive(record)
+ }
+ if (error instanceof GfsDownloadStoreError) throw error
+ throw new GfsDownloadStoreError('storage_write_failed')
+ }
+
+ const callerPartialPath = `${record.path}.partial`
+ return {
+ id,
+ path: record.path,
+ partialPath: callerPartialPath,
+ sizeBytes: input.sizeBytes,
+ expiresAt: input.expiresAt,
+ }
+ }
+
+ async publish(id: string, callerIdentity: string, sha256: string): Promise<GfsDownloadReceipt> {
+ this.assertInitialized()
+ const record = this.record(id)
+ const receipt = await this.serialize(async () => {
+ if (record.callerIdentity !== callerIdentity)
+ throw new GfsDownloadStoreError('caller_mismatch')
+ if (!SHA256_RE.test(sha256)) throw new GfsDownloadStoreError('corrupt_store_ledger')
+ if (record.state !== 'transferring') throw new GfsDownloadStoreError('download_missing')
+ if (Date.parse(record.expiresAt) <= Date.now())
+ throw new GfsDownloadStoreError('download_expired')
+ const partial = path.join(this.hostRoot, `${record.hostPath}.partial`)
+ const source = path.join(this.hostRoot, record.hostPath)
+ const info = await fs.lstat(partial).catch(() => undefined)
+ if (!info?.isFile() || info.isSymbolicLink() || info.size !== record.sizeBytes)
+ throw new GfsDownloadStoreError('download_missing')
+ if ((await sha256File(partial)) !== sha256)
+ throw new GfsDownloadStoreError('corrupt_store_ledger')
+ await fs.rename(partial, source)
+ await syncDirectory(path.dirname(source))
+ record.state = 'completed'
+ record.sha256 = sha256
+ await this.persist()
+ return {
+ id,
+ source: record.source,
+ path: record.path,
+ sizeBytes: record.sizeBytes,
+ sha256,
+ expiresAt: record.expiresAt,
+ }
+ })
+ this.releaseActive(record)
+ return receipt
+ }
+
+ async fail(id: string, callerIdentity: string): Promise<void> {
+ this.assertInitialized()
+ const record = this.record(id)
+ try {
+ await this.serialize(async () => {
+ if (record.callerIdentity !== callerIdentity)
+ throw new GfsDownloadStoreError('caller_mismatch')
+ const directory = path.join(this.hostRoot, record.directory)
+ try {
+ const info = await fs.lstat(directory)
+ if (!info.isDirectory() || info.isSymbolicLink())
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ await fs.rm(directory, { recursive: true, force: true })
+ } catch {
+ record.state = 'cleanup_failed'
+ await this.persist()
+ throw new GfsDownloadStoreError('storage_write_failed')
+ }
+ delete this.ledger.records[id]
+ await this.persist()
+ })
+ } finally {
+ this.releaseActive(record)
+ }
+ }
+ async inspect(
+ callerRelativePath: string,
+ callerIdentity: string
+ ): Promise<GfsDownloadReceipt & { id: string }> {
+ this.assertInitialized()
+ const record = Object.values(this.ledger.records).find(
+ item => item.callerIdentity === callerIdentity && item.path === callerRelativePath
+ )
+ if (!record) throw new GfsDownloadStoreError('caller_mismatch')
+ if (Date.parse(record.expiresAt) <= Date.now())
+ throw new GfsDownloadStoreError('download_expired')
+ if (record.state !== 'completed' || !record.sha256)
+ throw new GfsDownloadStoreError('download_missing')
+ const source = path.join(this.hostRoot, record.hostPath)
+ const info = await fs.lstat(source).catch(() => undefined)
+ if (
+ !info?.isFile() ||
+ info.isSymbolicLink() ||
+ info.size !== record.sizeBytes ||
+ (info.mode & 0o777) !== 0o600
+ )
+ throw new GfsDownloadStoreError('download_missing')
+ const digest = createHash('sha256')
+ const handle = await fs.open(
+ source,
+ constants.O_RDONLY | constants.O_NOFOLLOW | constants.O_NONBLOCK
+ )
+ try {
+ const chunk = Buffer.alloc(64 * 1024)
+ let position = 0
+ for (;;) {
+ const { bytesRead } = await handle.read(chunk, 0, chunk.byteLength, position)
+ if (bytesRead === 0) break
+ digest.update(chunk.subarray(0, bytesRead))
+ position += bytesRead
+ }
+ } finally {
+ await handle.close()
+ }
+ if (digest.digest('hex') !== record.sha256) throw new GfsDownloadStoreError('download_missing')
+ return {
+ id: record.id,
+ source: record.source,
+ path: record.path,
+ sizeBytes: record.sizeBytes,
+ sha256: record.sha256,
+ expiresAt: record.expiresAt,
+ }
+ }
+
+ debugRecord(id: string): GfsDownloadRecord | undefined {
+ return this.ledger.records[id]
+ }
+
+ debugUsage(): Usage {
+ return usageFor(Object.values(this.ledger.records))
+ }
+
+ async debugPersist(): Promise<void> {
+ await this.persist()
+ }
+
+ async close(drainTimeoutMs = 5_000): Promise<void> {
+ if (!this.initialized) return
+ this.closing = true
+ const deadline = Date.now() + drainTimeoutMs
+ while (this.activeIds.size > 0 && Date.now() < deadline) {
+ await new Promise(resolve => setTimeout(resolve, 25))
+ }
+ if (this.activeIds.size > 0) {
+ this.closing = false
+ throw new GfsDownloadStoreError('download_busy')
+ }
+ await this.mutationTail
+ this.initialized = false
+ try {
+ await this.serialize(async () => {
+ await this.persist()
+ })
+ await this.releaseWriterLease()
+ } finally {
+ this.closing = false
+ }
+ }
+
+ async cleanupExpired(now = Date.now()): Promise<void> {
+ if (!this.initialized && !this.closing) throw new GfsDownloadStoreError('workspace_unavailable')
+ await this.serialize(async () => {
+ const removed: string[] = []
+ let stateChanged = false
+ for (const record of Object.values(this.ledger.records)) {
+ if (Date.parse(record.expiresAt) > now) continue
+ if (this.activeIds.has(record.id)) continue
+ const directory = path.join(this.hostRoot, record.directory)
+ try {
+ const info = await fs.lstat(directory)
+ const realDirectory = await fs.realpath(directory)
+ if (!info.isDirectory() || info.isSymbolicLink() || realDirectory !== directory) {
+ if (record.state !== 'cleanup_failed') {
+ record.state = 'cleanup_failed'
+ stateChanged = true
+ }
+ continue
+ }
+ await fs.rm(directory, { recursive: true, force: true })
+ removed.push(record.id)
+ } catch {
+ if (record.state !== 'cleanup_failed') {
+ record.state = 'cleanup_failed'
+ stateChanged = true
+ }
+ }
+ }
+ if (removed.length > 0) for (const id of removed) delete this.ledger.records[id]
+ if (removed.length > 0 || stateChanged) await this.persist()
+ })
+ }
+
+ private async reconcile(): Promise<void> {
+ for (const record of Object.values(this.ledger.records)) {
+ const directory = path.join(this.hostRoot, record.directory)
+ if (!isWithinDirectory(directory, this.hostRoot))
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ const sourceInfo = await fs
+ .lstat(path.join(this.hostRoot, record.hostPath))
+ .catch(() => undefined)
+ if (
+ sourceInfo?.isFile() &&
+ !sourceInfo.isSymbolicLink() &&
+ sourceInfo.size === record.sizeBytes &&
+ SHA256_RE.test(record.sha256 ?? '')
+ ) {
+ record.state = 'completed'
+ continue
+ }
+ const partialInfo = await fs
+ .lstat(path.join(this.hostRoot, `${record.hostPath}.partial`))
+ .catch(() => undefined)
+ if (
+ (sourceInfo?.isFile() || partialInfo?.isFile()) &&
+ !sourceInfo?.isSymbolicLink() &&
+ !partialInfo?.isSymbolicLink()
+ ) {
+ record.state = 'transferring'
+ continue
+ }
+ record.state = 'missing'
+ }
+ }
+
+ private async persist(): Promise<void> {
+ try {
+ const temporary = `${this.ledgerPath}.tmp-${randomUUID()}`
+ const handle = await fs.open(
+ temporary,
+ constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL,
+ 0o600
+ )
+ try {
+ await handle.writeFile(JSON.stringify(this.ledger), 'utf8')
+ await handle.sync()
+ } finally {
+ await handle.close()
+ }
+ await fs.rename(temporary, this.ledgerPath)
+ await syncDirectory(this.storeRoot)
+ } catch (error) {
+ this.unsafe = true
+ throw error
+ }
+ }
+
+ private async acquireWriterLease(): Promise<void> {
+ const currentOwner = GfsDownloadStore.writerOwners.get(this.storeRoot)
+ if (currentOwner) throw new GfsDownloadStoreError('writer_locked')
+ const leaseId = randomUUID()
+ let handle: fs.FileHandle | undefined
+ try {
+ handle = await fs.open(
+ this.lockPath,
+ constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL,
+ 0o600
+ )
+ await handle.writeFile(
+ JSON.stringify({ pid: process.pid, leaseId, acquiredAt: new Date().toISOString() }),
+ 'utf8'
+ )
+ await handle.sync()
+ this.writerLeaseId = leaseId
+ GfsDownloadStore.writerOwners.set(this.storeRoot, leaseId)
+ } catch (error) {
+ await handle?.close().catch(() => undefined)
+ if ((error as NodeJS.ErrnoException).code === 'EEXIST')
+ throw new GfsDownloadStoreError('writer_locked')
+ throw error
+ } finally {
+ await handle?.close().catch(() => undefined)
+ }
+ }
+
+ private async releaseWriterLease(): Promise<void> {
+ if (!this.writerLeaseId) return
+ if (GfsDownloadStore.writerOwners.get(this.storeRoot) === this.writerLeaseId)
+ GfsDownloadStore.writerOwners.delete(this.storeRoot)
+ await fs.rm(this.lockPath, { force: true })
+ this.writerLeaseId = undefined
+ }
+
+ private async validateCallerRoot(callerRoot: string): Promise<string> {
+ const lexical = path.resolve(callerRoot)
+ const real = await fs.realpath(lexical).catch(() => {
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ })
+ if (!isWithinDirectory(real, await fs.realpath(this.hostRoot)))
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ const info = await fs.lstat(real)
+ if (!info.isDirectory() || info.isSymbolicLink())
+ throw new GfsDownloadStoreError('workspace_unavailable')
+ return real
+ }
+
+ private record(id: string): GfsDownloadRecord {
+ const record = this.ledger.records[id]
+ if (!record) throw new GfsDownloadStoreError('download_missing')
+ return record
+ }
+
+ private releaseActive(record: GfsDownloadRecord): void {
+ if (!this.activeIds.has(record.id)) return
+ this.active = Math.max(0, this.active - 1)
+ this.activeIds.delete(record.id)
+ const current = this.activeByCaller.get(record.callerIdentity) ?? 0
+ if (current <= 1) this.activeByCaller.delete(record.callerIdentity)
+ else this.activeByCaller.set(record.callerIdentity, current - 1)
+ }
+
+ private async serialize<T>(operation: () => Promise<T>): Promise<T> {
+ const previous = this.mutationTail
+ let unlock!: () => void
+ this.mutationTail = new Promise<void>(resolve => {
+ unlock = resolve
+ })
+ await previous
+ try {
+ return await operation()
+ } finally {
+ unlock()
+ }
+ }
+
+ private assertInitialized(): void {
+ if (!this.initialized) throw new GfsDownloadStoreError('workspace_unavailable')
+ if (this.unsafe) throw new GfsDownloadStoreError('storage_write_failed')
+ }
+}
diff --git a/mcp-host/src/internalTools/gfsFilePolicy.test.ts b/mcp-host/src/internalTools/gfsFilePolicy.test.ts
index 1e196c6d0..0d03ad1c7 100644
--- a/mcp-host/src/internalTools/gfsFilePolicy.test.ts
+++ b/mcp-host/src/internalTools/gfsFilePolicy.test.ts
@@ -79,6 +79,12 @@ describe('GFS file policy', () => {
'must not exceed the Host aggregate retained-storage budget'
)
+ vi.stubEnv('MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES', String(512 * 1024 * 1024 + 1))
+ vi.resetModules()
+ await expect(import('./gfsFilePolicy')).rejects.toThrow(
+ 'must leave aggregate storage for another caller'
+ )
+
vi.stubEnv('MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES', undefined)
vi.stubEnv('MCP_HOST_GFS_CALLER_DOWNLOAD_MAX_FILES', '65')
vi.resetModules()
diff --git a/mcp-host/src/internalTools/gfsFilePolicy.ts b/mcp-host/src/internalTools/gfsFilePolicy.ts
index 3846512e6..afd4a97f3 100644
--- a/mcp-host/src/internalTools/gfsFilePolicy.ts
+++ b/mcp-host/src/internalTools/gfsFilePolicy.ts
@@ -73,3 +73,8 @@ if (GFS_FILE_LIMITS.callerStorageBytes > GFS_FILE_LIMITS.storageBytes)
throw new Error(
'MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES must not exceed the Host aggregate retained-storage budget'
)
+
+if (GFS_FILE_LIMITS.callerStorageBytes * 2 > GFS_FILE_LIMITS.storageBytes)
+ throw new Error(
+ 'MCP_HOST_GFS_CALLER_DOWNLOAD_STORAGE_BYTES must leave aggregate storage for another caller'
+ )
diff --git a/mcp-host/src/internalTools/gfsLargeFile.test.ts b/mcp-host/src/internalTools/gfsLargeFile.test.ts
new file mode 100644
index 000000000..5dbcc2567
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsLargeFile.test.ts
@@ -0,0 +1,236 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { createHash } from 'node:crypto'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import type { InternalToolDefinition } from '../workflow/types'
+import { type GfscReadClient, buildGfsReadTools } from './gfs'
+import { GfsDownloadError } from './gfsContentDownload'
+import type { GfsMetadataSnapshot } from './gfsContentRead'
+import type { GfsDownloadStore } from './gfsDownloadStore'
+import { GFS_FILE_LIMITS } from './gfsFilePolicy'
+import type { GfsFileContent } from './gfsReadTypes'
+
+const target = { drive: 'main', resourceId: 'a'.repeat(32) }
+let callerWorkspace: string
+
+beforeEach(() => {
+ callerWorkspace = fs.mkdtempSync(path.join(os.tmpdir(), 'gfs-large-file-'))
+})
+
+afterEach(() => {
+ fs.rmSync(callerWorkspace, { recursive: true, force: true })
+})
+const source = {
+ kind: 'gfs' as const,
+ ...target,
+ gfsUri: `gfs://${target.drive}/${target.resourceId}`,
+ name: 'input.csv',
+ version: 7,
+}
+
+function content(bytes: Buffer): GfsFileContent {
+ return { source, bytes, reservation: { release: vi.fn() } }
+}
+
+function snapshot(sizeBytes: number): GfsMetadataSnapshot {
+ return { source, size: sizeBytes }
+}
+
+function client(
+ options: { read?: GfscReadClient['read']; download?: GfscReadClient['download'] } = {}
+) {
+ return {
+ accessible: vi.fn(),
+ list: vi.fn(),
+ read: options.read ?? vi.fn(async () => content(Buffer.from('text'))),
+ readMetadata: vi.fn(async (): Promise<GfsMetadataSnapshot> => snapshot(0)),
+ download: options.download,
+ stat: vi.fn(),
+ resolve: vi.fn(),
+ } satisfies GfscReadClient
+}
+
+function result(tool: InternalToolDefinition, args: Record<string, unknown>) {
+ return tool.execute(args, '/tmp').then(outcome => ({
+ outcome,
+ parsed: outcome.content?.startsWith('{') ? JSON.parse(outcome.content) : undefined,
+ }))
+}
+
+describe('GFS large-file tool routing', () => {
+ it('does not expose generic workspace delivery without a store', () => {
+ expect(
+ buildGfsReadTools(client(), { referencedFiles: new Map() }).map(tool => tool.name)
+ ).not.toContain('clerum__gfs_download')
+ })
+
+ it('routes the incident-sized CSV to a workspace receipt without inline content', async () => {
+ const read = vi.fn()
+ const download = vi.fn(async () => ({
+ id: 'download-1',
+ source,
+ path: '.gfs-downloads/input-download-1/source',
+ sizeBytes: 3_836_961,
+ sha256: 'a'.repeat(64),
+ expiresAt: '2026-10-08T00:00:00.000Z',
+ }))
+ const gfs = client({ read, download })
+ gfs.readMetadata = vi.fn(async () => snapshot(3_836_961))
+ const managedFile = path.join(callerWorkspace, '.gfs-downloads/input-download-1/source')
+ fs.mkdirSync(path.dirname(managedFile), { recursive: true })
+ fs.writeFileSync(managedFile, 'not-an-image-prefix', 'utf8')
+ const store = {} as GfsDownloadStore
+ const tool = buildGfsReadTools(gfs, {
+ referencedFiles: new Map(),
+ downloadStore: store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerWorkspace,
+ }).find(item => item.name === 'clerum__gfs_read')!
+
+ const { outcome, parsed } = await result(tool, target)
+
+ expect(outcome.success).toBe(true)
+ expect(parsed).toMatchObject({
+ delivery: 'workspace_file',
+ sizeBytes: 3_836_961,
+ sha256: 'a'.repeat(64),
+ })
+ expect(parsed.usage).toMatchObject({
+ pathSemantics: 'relative-to-caller-workspace',
+ nextTool: 'shell_exec_when_local_processing_is_needed',
+ wholeFileToContextAllowed: false,
+ })
+ expect(JSON.stringify(parsed)).not.toContain('AAAAAAA')
+ expect(read).not.toHaveBeenCalled()
+ expect(download).toHaveBeenCalledWith(
+ target,
+ expect.objectContaining({
+ store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerWorkspace,
+ expectedVersion: 7,
+ })
+ )
+ })
+
+ it('keeps exactly the inline threshold in memory and downloads one byte above it', async () => {
+ const download = vi.fn(async () => ({
+ id: 'download-2',
+ source,
+ path: '.gfs-downloads/input-download-2/source',
+ sizeBytes: 8193,
+ sha256: 'b'.repeat(64),
+ expiresAt: '2026-10-08T00:00:00.000Z',
+ }))
+ const gfs = client({ download })
+ gfs.readMetadata = vi.fn(async () => snapshot(8192))
+ const managedFile = path.join(callerWorkspace, '.gfs-downloads/input-download-2/source')
+ fs.mkdirSync(path.dirname(managedFile), { recursive: true })
+ fs.writeFileSync(managedFile, 'not-an-image-prefix', 'utf8')
+ const store = {} as GfsDownloadStore
+ const tool = buildGfsReadTools(gfs, {
+ referencedFiles: new Map(),
+ downloadStore: store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerWorkspace,
+ }).find(item => item.name === 'clerum__gfs_read')!
+
+ const { outcome } = await result(tool, target)
+ expect(outcome.content).toBe('text')
+ expect(download).not.toHaveBeenCalled()
+ expect(gfs.read).toHaveBeenCalledWith(
+ target,
+ expect.objectContaining({
+ expectedVersion: 7,
+ metadataSnapshot: snapshot(8192),
+ })
+ )
+
+ gfs.readMetadata = vi.fn(async () => snapshot(GFS_FILE_LIMITS.inlineTextBytes + 1))
+ const { parsed: large } = await result(tool, target)
+ expect(large).toMatchObject({ delivery: 'workspace_file' })
+ expect(download).toHaveBeenCalledTimes(1)
+ })
+
+ it('exposes an explicit generic download tool for admitted sources', async () => {
+ const download = vi.fn(async () => ({
+ id: 'download-3',
+ source,
+ path: '.gfs-downloads/input-download-3/source',
+ sizeBytes: 1,
+ sha256: 'c'.repeat(64),
+ expiresAt: '2026-10-08T00:00:00.000Z',
+ }))
+ const store = {} as GfsDownloadStore
+ const tool = buildGfsReadTools(client({ download }), {
+ referencedFiles: new Map(),
+ downloadStore: store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: '/tmp/caller-a',
+ }).find(item => item.name === 'clerum__gfs_download')!
+
+ const { parsed } = await result(tool, target)
+ expect(parsed).toMatchObject({ delivery: 'workspace_file', sizeBytes: 1 })
+ expect(download).toHaveBeenCalledWith(
+ target,
+ expect.objectContaining({ store, callerIdentity: 'caller-a' })
+ )
+ })
+
+ it('preserves a workspace receipt before projecting a byte-classified image', async () => {
+ const png = Buffer.from(
+ 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==',
+ 'base64'
+ )
+ const download = vi.fn(async () => ({
+ id: 'download-image',
+ source,
+ path: '.gfs-downloads/input-download-image/source',
+ sizeBytes: png.byteLength,
+ sha256: createHash('sha256').update(png).digest('hex'),
+ expiresAt: '2026-10-08T00:00:00.000Z',
+ }))
+ const gfs = client({ read: vi.fn(async () => content(png)), download })
+ gfs.readMetadata = vi.fn(async () => snapshot(png.byteLength))
+ const store = {} as GfsDownloadStore
+ const tool = buildGfsReadTools(gfs, {
+ referencedFiles: new Map(),
+ downloadStore: store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: callerWorkspace,
+ }).find(item => item.name === 'clerum__gfs_read')!
+
+ const { parsed } = await result(tool, target)
+ expect(parsed).toMatchObject({
+ delivery: 'workspace_file',
+ sizeBytes: png.byteLength,
+ visualDelivery: 'not_included',
+ visualReason: 'image_input_unavailable_in_this_execution',
+ })
+ expect(parsed.usage).toMatchObject({
+ visualDelivery: 'not_included',
+ wholeFileToContextAllowed: false,
+ })
+ expect(download).toHaveBeenCalledTimes(1)
+ })
+
+ it('reports a pinned explicit download as stale on version conflict', async () => {
+ const download = vi.fn(async () => {
+ throw new GfsDownloadError('version_conflict')
+ })
+ const store = {} as GfsDownloadStore
+ const tool = buildGfsReadTools(client({ download }), {
+ referencedFiles: new Map(),
+ downloadStore: store,
+ callerIdentity: 'caller-a',
+ callerWorkspacePath: '/tmp/caller-a',
+ }).find(item => item.name === 'clerum__gfs_download')!
+
+ const { parsed } = await result(tool, { ...target, expectedVersion: 7 })
+ expect(parsed).toMatchObject({
+ availability: 'stale',
+ expectedVersion: 7,
+ })
+ })
+})
diff --git a/mcp-host/src/internalTools/gfsProcessingLease.ts b/mcp-host/src/internalTools/gfsProcessingLease.ts
new file mode 100644
index 000000000..fcb6624b1
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsProcessingLease.ts
@@ -0,0 +1,21 @@
+/**
+ * Caller-bound cleanup protection for approved local processing.
+ *
+ * The provider is intentionally narrower than the download store: ShellTool can
+ * protect retained downloads without gaining store administration capability.
+ */
+export interface GfsProcessingLease {
+ leaseId: string
+ expiresAt: string
+}
+
+export interface GfsProcessingLeaseAcquisition {
+ /** Bounded lease duration in milliseconds. Implementations enforce a ceiling. */
+ durationMs?: number
+}
+
+export interface GfsProcessingLeaseProvider {
+ acquireProcessingLease(options?: GfsProcessingLeaseAcquisition): Promise<GfsProcessingLease>
+ releaseProcessingLease(lease: GfsProcessingLease): Promise<void>
+}
+// Processing leases protect retained downloads while a shell command runs.
diff --git a/mcp-host/src/internalTools/gfsReadTypes.ts b/mcp-host/src/internalTools/gfsReadTypes.ts
index bb89f6aca..404a7f43b 100644
--- a/mcp-host/src/internalTools/gfsReadTypes.ts
+++ b/mcp-host/src/internalTools/gfsReadTypes.ts
@@ -11,6 +11,8 @@ export interface GfsReadOptions {
* content is requested.
*/
expectedVersion?: number
+ /** Reuse a validated routing snapshot so inline classification cannot observe a second version. */
+ metadataSnapshot?: { source: GfsImageSource; size: number }
}
/** A validated metadata/content snapshot. This object is never a tool result. */
diff --git a/mcp-host/src/internalTools/gfsSurfaceCapability.test.ts b/mcp-host/src/internalTools/gfsSurfaceCapability.test.ts
new file mode 100644
index 000000000..533e71a9a
--- /dev/null
+++ b/mcp-host/src/internalTools/gfsSurfaceCapability.test.ts
@@ -0,0 +1,43 @@
+import { describe, expect, it, vi } from 'vitest'
+import { type GfscReadClient, buildGfsReadTools } from './gfs'
+import type { GfsMetadataSnapshot } from './gfsContentRead'
+
+const resourceId = '0123456789abcdef0123456789abcdef'
+const snapshot: GfsMetadataSnapshot = {
+ source: {
+ kind: 'gfs',
+ drive: 'main',
+ resourceId,
+ gfsUri: `gfs://main/${resourceId}`,
+ name: 'strong-fit.csv',
+ version: 7,
+ },
+ size: 3_836_961,
+}
+
+describe('GFS workspace delivery capability', () => {
+ it('does not read an over-inline body when workspace delivery is unavailable', async () => {
+ const read = vi.fn()
+ const client: GfscReadClient = {
+ accessible: vi.fn(),
+ list: vi.fn(),
+ read,
+ readMetadata: vi.fn(async () => snapshot),
+ stat: vi.fn(),
+ resolve: vi.fn(),
+ }
+ const tools = buildGfsReadTools(client, { referencedFiles: new Map() })
+ expect(tools.map(tool => tool.name)).not.toContain('clerum__gfs_download')
+
+ const readTool = tools.find(tool => tool.name === 'clerum__gfs_read')
+ expect(readTool).toBeDefined()
+ const result = await readTool!.execute({ drive: 'main', resourceId }, '/tmp/outputs')
+
+ expect(result.success).toBe(true)
+ expect(JSON.parse(result.content as string)).toMatchObject({
+ availability: 'workspace_delivery_unavailable',
+ sizeBytes: 3_836_961,
+ })
+ expect(read).not.toHaveBeenCalled()
+ })
+})
diff --git a/mcp-host/src/main.ts b/mcp-host/src/main.ts
index 6cff510ab..3ceeee2a8 100644
--- a/mcp-host/src/main.ts
+++ b/mcp-host/src/main.ts
@@ -63,6 +63,7 @@ import {
withResolvedHookDescriptors,
} from './guardrailHookResolver'
import { createGfscClient, inspectGfsToolScopes } from './internalTools/gfsClient'
+import { GfsDownloadStore } from './internalTools/gfsDownloadStore'
import { HostWatcher, LlmHookWatcher, getHost, getLlmHook } from './k8sClient'
import { StatelessHeartbeat } from './lifecycle/statelessHeartbeat'
import { TaskLifecycle } from './lifecycle/taskLifecycle'
@@ -232,6 +233,8 @@ let contextMapperClient: ContextMapperClient | null = null
let mcpAuthorityLastSuccessAt = 0
let activityHub: HostActivityHub | null = null
let workspaceProvider: ScopedWorkspaceProvider | null = null
+let gfsDownloadStore: GfsDownloadStore | null = null
+let gfsDownloadCleanupTimer: NodeJS.Timeout | null = null
let spilloverStorage: SpilloverStorage | null = null
let conversationStoreHandle: ConversationStoreHandle | null = null
let promptCache: PromptCache | null = null
@@ -1723,6 +1726,21 @@ async function initializeAgent(): Promise<void> {
agent.setDynamicEnvProvider(() => agentToolEnvProvider(configStore))
agent.setSecretEntriesProvider(() => configStore?.listSecretEntries() ?? [])
+ const gfsWorkspaceProvider = new ScopedWorkspaceProvider(config.nativeTool.workspacePath)
+ gfsDownloadStore = new GfsDownloadStore(config.nativeTool.workspacePath)
+ await gfsDownloadStore.initialize()
+ agent.setGfsWorkspaceProvider(gfsWorkspaceProvider)
+ agent.setGfsDownloadStore(gfsDownloadStore)
+ gfsDownloadCleanupTimer = setInterval(
+ () => {
+ void gfsDownloadStore?.cleanupExpired().catch(err => {
+ logger.warn({ err: err }, '[Main] GFS download cleanup failed:')
+ })
+ },
+ 60 * 60 * 1000
+ )
+ gfsDownloadCleanupTimer.unref?.()
+
// Phase 7–8: Create WorkspaceService when memory is enabled
const memoryCfg = currentHost?.spec.memory || config.memory
if (memoryCfg?.enabled) {
@@ -3054,6 +3072,15 @@ async function shutdown(signal: string): Promise<void> {
}
}
+ if (gfsDownloadStore) {
+ if (gfsDownloadCleanupTimer) clearInterval(gfsDownloadCleanupTimer)
+ try {
+ await gfsDownloadStore.close()
+ } catch (err) {
+ logger.warn({ err: err }, '[Main] GFS download store shutdown raised:')
+ }
+ }
+
process.exit(0)
}
diff --git a/mcp-host/src/server.ts b/mcp-host/src/server.ts
index a8420a6d2..34039c21b 100644
--- a/mcp-host/src/server.ts
+++ b/mcp-host/src/server.ts
@@ -10,6 +10,7 @@ import * as path from 'path'
import { register } from 'prom-client'
import { readOpenedArtifactBuffer, redactArtifactForDelivery } from './artifacts/artifactBytes'
import type { ArtifactSecretEntry } from './artifacts/artifactRedaction'
+import './internalTools/gfsDownloadMetrics'
import type { RuntimeLifecycleGate } from './lifecycle/statelessHeartbeat'
import './mcp/catalogBootstrapMetrics'
import './mcp/statusHeartbeatMetrics'
diff --git a/mcp-host/src/visualInput/deliveryLimits.test.ts b/mcp-host/src/visualInput/deliveryLimits.test.ts
index 1f914973b..7964fe50d 100644
--- a/mcp-host/src/visualInput/deliveryLimits.test.ts
+++ b/mcp-host/src/visualInput/deliveryLimits.test.ts
@@ -8,6 +8,7 @@ describe('effective visual delivery limits', () => {
maxTotalImageBytes: 16 * 1024 * 1024,
maxImages: 20,
maxVisualRequestBytes: 24 * 1024 * 1024,
+ maxNonImageRequestBytes: 8 * 1024 * 1024,
maxDimension: 2048,
maxPixels: 4_194_304,
})
diff --git a/mcp-host/src/visualInput/deliveryLimits.ts b/mcp-host/src/visualInput/deliveryLimits.ts
index 4201aa936..c1ff4043d 100644
--- a/mcp-host/src/visualInput/deliveryLimits.ts
+++ b/mcp-host/src/visualInput/deliveryLimits.ts
@@ -19,13 +19,16 @@ export interface VisualDeliveryLimits {
* metadata admission.
*/
export function resolveVisualDeliveryLimits(providerType: string): VisualDeliveryLimits | null {
- if (providerType !== 'codex-subscription') return null
- return {
- maxImageBytes: CODEX_VISUAL_LIMITS.maxImageBytes,
- maxTotalImageBytes: CODEX_VISUAL_LIMITS.maxTotalImageBytes,
- maxImages: CODEX_VISUAL_LIMITS.maxImages,
- maxVisualRequestBytes: CODEX_LIMITS.maxVisualRequestBodyBytes,
- maxDimension: CODEX_VISUAL_LIMITS.maxImageDimension,
- maxPixels: CODEX_VISUAL_LIMITS.maxImagePixels,
- }
+ if (providerType === 'codex-subscription')
+ return {
+ maxImageBytes: CODEX_VISUAL_LIMITS.maxImageBytes,
+ maxTotalImageBytes: CODEX_VISUAL_LIMITS.maxTotalImageBytes,
+ maxImages: CODEX_VISUAL_LIMITS.maxImages,
+ maxVisualRequestBytes: CODEX_LIMITS.maxVisualRequestBodyBytes,
+ maxNonImageRequestBytes: CODEX_LIMITS.maxRequestBodyBytes,
+ maxDimension: CODEX_VISUAL_LIMITS.maxImageDimension,
+ maxPixels: CODEX_VISUAL_LIMITS.maxImagePixels,
+ }
+
+ return null
}
diff --git a/mcp-host/src/workspace/__tests__/protectedWorkspacePaths.test.ts b/mcp-host/src/workspace/__tests__/protectedWorkspacePaths.test.ts
new file mode 100644
index 000000000..205c199a1
--- /dev/null
+++ b/mcp-host/src/workspace/__tests__/protectedWorkspacePaths.test.ts
@@ -0,0 +1,92 @@
+import { afterEach, beforeEach, describe, expect, it } from 'vitest'
+import * as fs from 'node:fs'
+import * as os from 'node:os'
+import * as path from 'node:path'
+import { FileReadTool } from '../../core/tools/fileRead'
+import { FileWriteTool } from '../../core/tools/fileWrite'
+import {
+ MemorySearchTool,
+ MemoryTreeTool,
+ PersistentMemoryReadTool,
+ PersistentMemoryWriteTool,
+} from '../../core/tools/memory'
+import { GfsDownloadPathError, WorkspaceService } from '../service'
+
+describe('protected workspace paths', () => {
+ let root: string
+
+ beforeEach(() => {
+ root = fs.mkdtempSync(path.join(os.tmpdir(), 'protected-workspace-'))
+ const downloadDir = path.join(root, '.gfs-downloads', 'input-1')
+ fs.mkdirSync(downloadDir, { recursive: true })
+ fs.writeFileSync(path.join(downloadDir, 'source.md'), 'xylophone quixotic zenith', 'utf-8')
+ fs.writeFileSync(path.join(root, 'note.md'), 'public-token', 'utf-8')
+ fs.symlinkSync(path.join(downloadDir, 'source.md'), path.join(root, 'alias.md'))
+ })
+
+ afterEach(() => {
+ fs.rmSync(root, { recursive: true, force: true })
+ })
+
+ it('blocks direct and symlinked access through WorkspaceService', async () => {
+ const workspace = new WorkspaceService(root)
+ await expect(workspace.read('.gfs-downloads/input-1/source.md')).rejects.toBeInstanceOf(
+ GfsDownloadPathError
+ )
+ await expect(workspace.read('alias.md')).rejects.toBeInstanceOf(GfsDownloadPathError)
+ await expect(workspace.exists('.gfs-downloads/input-1/source.md')).resolves.toBe(false)
+ await expect(workspace.list()).resolves.not.toContainEqual(
+ expect.objectContaining({ name: '.gfs-downloads' })
+ )
+ await expect(workspace.search('xylophone quixotic zenith')).resolves.toEqual([])
+ await expect(workspace.search('public-token')).resolves.toHaveLength(1)
+ })
+
+ it('blocks direct and symlinked access through file tools', async () => {
+ const read = new FileReadTool(root)
+ const direct = await read.execute({ path: '.gfs-downloads/input-1/source.md' })
+ const aliased = await read.execute({ path: 'alias.md' })
+
+ expect(direct.is_error).toBe(true)
+ expect(direct.content).toContain('governed GFS download')
+ expect(aliased.is_error).toBe(true)
+ expect(aliased.content).toContain('Reserved workspace path')
+
+ const write = new FileWriteTool(root)
+ const blockedWrite = await write.execute({
+ path: 'alias.md',
+ content: 'replacement',
+ })
+ expect(blockedWrite.is_error).toBe(true)
+ expect(
+ fs.readFileSync(path.join(root, '.gfs-downloads', 'input-1', 'source.md'), 'utf-8')
+ ).toBe('xylophone quixotic zenith')
+ })
+
+ it('blocks direct and symlinked access through memory tools without leaking search results', async () => {
+ const workspace = new WorkspaceService(root)
+ const read = new PersistentMemoryReadTool(workspace)
+ const write = new PersistentMemoryWriteTool(workspace)
+ const search = new MemorySearchTool(workspace)
+ const tree = new MemoryTreeTool(workspace)
+
+ await expect(read.execute({ path: 'alias.md' })).resolves.toMatchObject({
+ is_error: true,
+ content: expect.stringContaining('governed GFS download'),
+ })
+ await expect(
+ write.execute({ target: '.gfs-downloads/input-1/source.md', content: 'x' })
+ ).resolves.toMatchObject({
+ is_error: true,
+ content: expect.stringContaining('governed GFS download'),
+ })
+ await expect(search.execute({ query: 'xylophone quixotic zenith' })).resolves.toMatchObject({
+ is_error: false,
+ content: expect.stringContaining('"result_count":0'),
+ })
+ await expect(tree.execute({ path: '' })).resolves.toMatchObject({
+ is_error: false,
+ content: expect.not.stringContaining('.gfs-downloads'),
+ })
+ })
+})
diff --git a/mcp-host/src/workspace/protectedPaths.ts b/mcp-host/src/workspace/protectedPaths.ts
new file mode 100644
index 000000000..f0f8c5a0e
--- /dev/null
+++ b/mcp-host/src/workspace/protectedPaths.ts
@@ -0,0 +1,104 @@
+import * as path from 'path'
+
+export const PROTECTED_WORKSPACE_DIRS: ReadonlySet<string> = new Set([
+ '.clerum-state',
+ '.gfs-downloads',
+])
+
+export const PROTECTED_STATE_DB_FILES: ReadonlySet<string> = new Set([
+ 'state.db',
+ 'state.db-wal',
+ 'state.db-shm',
+])
+
+function normalizeWorkspacePath(relativePath: string): string {
+ let normalized = path.posix.normalize(relativePath || '')
+ if (normalized.startsWith('./')) normalized = normalized.slice(2)
+ if (normalized.startsWith('/')) normalized = normalized.slice(1)
+ if (normalized.endsWith('/')) normalized = normalized.slice(0, -1)
+ return normalized
+}
+
+/**
+ * True for platform-owned state paths that must never be exposed through the
+ * agent-facing workspace API. This is a lexical defense-in-depth check; callers
+ * that resolve symlinks must also use isProtectedRealPath.
+ */
+export function isProtectedWorkspacePath(relativePath: string): boolean {
+ if (typeof relativePath !== 'string' || relativePath.length === 0) return false
+ const segments = normalizeWorkspacePath(relativePath)
+ .split('/')
+ .filter(segment => segment.length > 0 && segment !== '.')
+ if (segments.some(segment => PROTECTED_WORKSPACE_DIRS.has(segment))) return true
+ const base = segments[segments.length - 1]
+ return base !== undefined && PROTECTED_STATE_DB_FILES.has(base)
+}
+
+/** Backward-compatible helper for callers that specifically report state-db access. */
+export function isStateDbPath(relativePath: string): boolean {
+ if (typeof relativePath !== 'string' || relativePath.length === 0) return false
+ const segments = normalizeWorkspacePath(relativePath)
+ .split('/')
+ .filter(segment => segment.length > 0 && segment !== '.')
+ if (segments.includes('.clerum-state')) return true
+ const base = segments[segments.length - 1]
+ return base !== undefined && PROTECTED_STATE_DB_FILES.has(base)
+}
+
+export function isGfsDownloadPath(relativePath: string): boolean {
+ if (typeof relativePath !== 'string' || relativePath.length === 0) return false
+ const segments = normalizeWorkspacePath(relativePath)
+ .split('/')
+ .filter(segment => segment.length > 0 && segment !== '.')
+ return segments.some(segment => segment === '.gfs-downloads')
+}
+
+export function stateDbProtectedMessage(filename: string): string {
+ return `${filename} is part of the session state database and cannot be accessed by the agent.`
+}
+
+export class ProtectedWorkspacePathError extends Error {
+ constructor(message: string) {
+ super(message)
+ this.name = 'ProtectedWorkspacePathError'
+ }
+}
+
+export class StateDbPathError extends ProtectedWorkspacePathError {
+ constructor(filename: string) {
+ super(stateDbProtectedMessage(filename))
+ this.name = 'StateDbPathError'
+ }
+}
+
+export class GfsDownloadPathError extends ProtectedWorkspacePathError {
+ constructor(filename: string) {
+ super(
+ `${filename} is a governed GFS download and cannot be accessed through workspace file tools.`
+ )
+ this.name = 'GfsDownloadPathError'
+ }
+}
+
+export function protectedWorkspacePathError(relativePath: string): ProtectedWorkspacePathError {
+ if (isGfsDownloadPath(relativePath)) return new GfsDownloadPathError(relativePath)
+ return new StateDbPathError(relativePath)
+}
+
+export function protectedWorkspacePathMessage(relativePath: string): string {
+ return protectedWorkspacePathError(relativePath).message
+}
+
+/**
+ * Check a fully resolved path against the real workspace root. This catches
+ * aliases whose lexical path looks harmless but whose symlink target is a
+ * protected platform directory or database lateral.
+ */
+export function isProtectedRealPath(realPath: string, realWorkspaceRoot: string): boolean {
+ const relative = path.relative(realWorkspaceRoot, realPath)
+ if (relative.startsWith('..') || path.isAbsolute(relative) || relative === '') return false
+ const segments = relative.split(path.sep).filter(segment => segment.length > 0)
+ if (segments.some(segment => PROTECTED_WORKSPACE_DIRS.has(segment))) return true
+ const base = segments[segments.length - 1]
+ return base !== undefined && PROTECTED_STATE_DB_FILES.has(base)
+}
diff --git a/mcp-host/src/workspace/search.ts b/mcp-host/src/workspace/search.ts
index 5b62dc7ce..4e2696688 100644
--- a/mcp-host/src/workspace/search.ts
+++ b/mcp-host/src/workspace/search.ts
@@ -12,6 +12,7 @@
*/
import * as fs from 'fs/promises'
import * as path from 'path'
+import { isProtectedRealPath, isProtectedWorkspacePath } from './protectedPaths'
import { SearchConfig, SearchResult } from './types'
const RECENCY_BOOST_FILES = /^daily\//
@@ -43,6 +44,7 @@ async function collectMarkdownFiles(
if (dir === base && excludeTopLevel?.has(entry.name)) continue
const full = path.join(dir, entry.name)
const rel = path.relative(base, full)
+ if (isProtectedWorkspacePath(rel)) continue
if (entry.isDirectory()) {
const sub = await collectMarkdownFiles(full, base, excludeTopLevel)
files.push(...sub)
@@ -74,6 +76,13 @@ export async function searchWorkspace(
for (const relPath of mdFiles) {
const absPath = path.join(workspacePath, relPath)
+ try {
+ const realPath = await fs.realpath(absPath)
+ const realRoot = await fs.realpath(workspacePath)
+ if (isProtectedRealPath(realPath, realRoot)) continue
+ } catch {
+ continue
+ }
let fileContent: string
try {
fileContent = await fs.readFile(absPath, 'utf-8')
diff --git a/mcp-host/src/workspace/service.ts b/mcp-host/src/workspace/service.ts
index 0a20f4d28..65c119420 100644
--- a/mcp-host/src/workspace/service.ts
+++ b/mcp-host/src/workspace/service.ts
@@ -8,6 +8,11 @@ import { Mutex } from 'async-mutex'
import * as fs from 'fs/promises'
import * as path from 'path'
import type { PersonalizationConfig } from '../types'
+import {
+ isProtectedRealPath,
+ isProtectedWorkspacePath,
+ protectedWorkspacePathError,
+} from './protectedPaths'
import { scanWriteContent } from './scanner'
import { searchWorkspace } from './search'
import { SearchConfig, SearchResult, WorkspaceEntry } from './types'
@@ -56,46 +61,13 @@ export function isLockedPath(relativePath: string): boolean {
return LOCKED_IDENTITY_FILES.has(normalized)
}
-// D3 (stateless-agents) §1.2 — the session database and its WAL laterals are
-// platform state, never agent-writable. The db may live at the workspace root
-// (workspace PVC) or under CLERUM_SESSION_DB_DIR; the guard rejects the
-// basenames at ANY depth (defense in depth — same trust model as the identity
-// files: POSIX perms are the OS backstop, this is the loud tool-level gate).
-export const PROTECTED_STATE_DB_FILES: ReadonlySet<string> = new Set([
- 'state.db',
- 'state.db-wal',
- 'state.db-shm',
-])
-
-/** Reserved directory for stateless-lifecycle runtime state. */
-export const PROTECTED_STATE_DIR = '.clerum-state'
-
-export function stateDbProtectedMessage(filename: string): string {
- return `${filename} is part of the session state database and cannot be accessed by the agent.`
-}
-
-export class StateDbPathError extends Error {
- constructor(filename: string) {
- super(stateDbProtectedMessage(filename))
- this.name = 'StateDbPathError'
- }
-}
-
-/**
- * True when — after normalization — any path segment is `.clerum-state` or
- * the basename is one of the protected state-db files, at any depth.
- *
- * Returns true for: "state.db", "./state.db", "state.db-wal", "a/state.db",
- * ".clerum-state/x". Returns false for: "state.db.bak", "notes/state.database".
- */
-export function isStateDbPath(relativePath: string): boolean {
- if (typeof relativePath !== 'string' || relativePath.length === 0) return false
- const normalized = path.posix.normalize(relativePath)
- const segments = normalized.split('/').filter(s => s.length > 0 && s !== '.')
- if (segments.includes(PROTECTED_STATE_DIR)) return true
- const base = segments[segments.length - 1]
- return base !== undefined && PROTECTED_STATE_DB_FILES.has(base)
-}
+export {
+ GfsDownloadPathError,
+ PROTECTED_STATE_DB_FILES,
+ StateDbPathError,
+ isStateDbPath,
+ stateDbProtectedMessage,
+} from './protectedPaths'
/**
* Agent-facing workspace surface, implemented by both {@link WorkspaceService}
@@ -170,6 +142,38 @@ export class WorkspaceService implements Workspace {
return resolved
}
+ /**
+ * Resolve an agent-requested path and reject aliases into platform-owned
+ * storage. The lexical guard blocks direct names; realpath checks block a
+ * symlink or missing file whose nearest existing ancestor points into the
+ * protected namespace or outside the workspace.
+ */
+ private async resolveAccessiblePath(relativePath: string): Promise<string> {
+ if (isProtectedWorkspacePath(relativePath)) {
+ throw protectedWorkspacePathError(relativePath)
+ }
+ const resolved = this.resolvePath(relativePath)
+ const realWorkspaceRoot = await fs.realpath(this.workspacePath)
+ let current = resolved
+ while (current !== path.dirname(current)) {
+ try {
+ const realPath = await fs.realpath(current)
+ const relative = path.relative(realWorkspaceRoot, realPath)
+ if (relative.startsWith('..') || path.isAbsolute(relative)) {
+ throw new Error('Path resolves outside workspace')
+ }
+ if (isProtectedRealPath(realPath, realWorkspaceRoot)) {
+ throw protectedWorkspacePathError(path.relative(realWorkspaceRoot, realPath))
+ }
+ break
+ } catch (err) {
+ if ((err as NodeJS.ErrnoException).code !== 'ENOENT') throw err
+ current = path.dirname(current)
+ }
+ }
+ return resolved
+ }
+
private mutexFor(absolutePath: string): Mutex {
let m = this.fileMutexes.get(absolutePath)
if (!m) {
@@ -182,7 +186,7 @@ export class WorkspaceService implements Workspace {
// ── Document CRUD ──────────────────────────────────────────────────────────
async read(relativePath: string): Promise<string | null> {
- const resolved = this.resolvePath(relativePath) // throws on invalid paths
+ const resolved = await this.resolveAccessiblePath(relativePath)
try {
return await fs.readFile(resolved, 'utf-8')
} catch {
@@ -194,10 +198,7 @@ export class WorkspaceService implements Workspace {
if (isLockedPath(relativePath)) {
throw new LockedFileError(relativePath)
}
- if (isStateDbPath(relativePath)) {
- throw new StateDbPathError(relativePath)
- }
- const resolved = this.resolvePath(relativePath)
+ const resolved = await this.resolveAccessiblePath(relativePath)
scanWriteContent(relativePath, content, Buffer.byteLength(content, 'utf-8'))
await this.mutexFor(resolved).runExclusive(async () => {
await this._atomicWrite(resolved, content)
@@ -217,10 +218,7 @@ export class WorkspaceService implements Workspace {
if (isLockedPath(relativePath)) {
throw new LockedFileError(relativePath)
}
- if (isStateDbPath(relativePath)) {
- throw new StateDbPathError(relativePath)
- }
- const resolved = this.resolvePath(relativePath)
+ const resolved = await this.resolveAccessiblePath(relativePath)
await this.mutexFor(resolved).runExclusive(async () => {
let existing: string | null
try {
@@ -241,10 +239,7 @@ export class WorkspaceService implements Workspace {
if (isLockedPath(relativePath)) {
throw new LockedFileError(relativePath)
}
- if (isStateDbPath(relativePath)) {
- throw new StateDbPathError(relativePath)
- }
- await fs.unlink(this.resolvePath(relativePath))
+ await fs.unlink(await this.resolveAccessiblePath(relativePath))
}
/**
@@ -278,7 +273,7 @@ export class WorkspaceService implements Workspace {
async exists(relativePath: string): Promise<boolean> {
try {
- await fs.access(this.resolvePath(relativePath))
+ await fs.access(await this.resolveAccessiblePath(relativePath))
return true
} catch {
return false
@@ -288,7 +283,7 @@ export class WorkspaceService implements Workspace {
// ── Directory listing ──────────────────────────────────────────────────────
async list(directory: string = ''): Promise<WorkspaceEntry[]> {
- const resolved = directory ? this.resolvePath(directory) : this.workspacePath
+ const resolved = directory ? await this.resolveAccessiblePath(directory) : this.workspacePath
let entries: { name: string; isDirectory(): boolean }[]
try {
entries = await fs.readdir(resolved, { withFileTypes: true })
@@ -298,6 +293,7 @@ export class WorkspaceService implements Workspace {
const results: WorkspaceEntry[] = []
for (const entry of entries) {
+ if (isProtectedWorkspacePath(entry.name)) continue
// At the root, skip excluded top-level dirs (e.g. `users/` on collective).
if (!directory && this.options?.excludeDirs?.includes(entry.name)) continue
const entryPath = path.join(directory, entry.name).replace(/^\//, '')