Fully autonomous AI Agents system capable of performing complex penetration testing tasks
Go ai-agentsai-security-toolanthropicautonomous-agentsgolanggptgraphqlmulti-agent-systemoffensive-securityopen-sourceopenaipenetration-testingpenetration-testing-toolsreactsecurity-automationsecurity-testingsecurity-toolsself-hosted
docs(rfc): propose MCP client integration design
mason5052ba8a687cb687
diff --git a/examples/proposals/mcp_client_integration.md b/examples/proposals/mcp_client_integration.mdnew file mode 100644index 00000000..80272818--- /dev/null+++ b/examples/proposals/mcp_client_integration.md@@ -0,0 +1,435 @@+# MCP Client Integration RFC++## Summary++Issue [#296](https://github.com/vxcontrol/pentagi/issues/296) proposes that+PentAGI add a generic Model Context Protocol (MCP) client so agents can use+external MCP-compatible security tools as first-class tools, with Burp Suite+Pro as the concrete motivating example. This RFC sketches a possible design+direction for that capability.++This document does not implement runtime behavior. It does not add Go or+JavaScript dependencies, change the tool registry, modify agent+orchestration, change the database schema or generated code, add REST+endpoints, or change Docker Compose or installer behavior. It is a design+surface for maintainers to push back on before any code lands.++The RFC is intentionally staged. MCP is powerful, and the issue's most+direct reading -- auto-discover every MCP tool from every configured+server and expose all of them to every agent -- would land near the+pattern PR [#268](https://github.com/vxcontrol/pentagi/pull/268) was+rejected for: implicit lifecycle state, weak operator visibility, and+weak operator control. The proposed v1 below is narrower on purpose so a+later implementation PR can be reviewed in small slices.++## Goals++- Connect configured MCP servers as explicit external tool sources.+- Discover tools from allowlisted MCP servers via the standard MCP+ `tools/list` method.+- Expose only approved MCP tools to agents, alongside native PentAGI+ tools, with a stable namespace.+- Preserve PentAGI's existing tool visibility and auditability+ expectations: operators can see which tools exist, which agent invoked+ which tool, with what inputs and what result.+- Support a staged implementation path that starts with a narrow,+ reviewable milestone (read-only discovery) and grows from there.+- Keep operators in control of network reachability, credentials, and+ tool exposure at every step.++## Non-Goals++- This RFC does not add MCP runtime code, dependencies, registry,+ orchestration, schema, GraphQL, REST, or installer changes.+- This RFC does not propose auto-trusting every tool advertised by a+ configured MCP server. Discovered tools are inert until an operator+ allowlists them.+- This RFC does not propose hidden background execution of MCP tool+ calls. Calls happen inside the same agent loop and are visible the+ same way native tool calls are. This is the explicit lesson from PR+ [#268](https://github.com/vxcontrol/pentagi/pull/268): no hidden+ in-memory queues, no implicit lifecycle state, no invisible background+ behavior.+- This RFC does not propose a hidden queue, retry buffer, or implicit+ per-tool state machine sitting between agents and MCP servers.+- This RFC does not grant MCP servers arbitrary host or network access+ by default. Reachability is opt-in per server.+- This RFC does not propose a generic plugin marketplace. MCP servers+ are configured by the operator, not browsed.+- This RFC does not guarantee that every MCP transport or server will+ be supported in v1. The proposed v1 is deliberately small.+- This RFC does not change any existing native tool. Native tools keep+ their current behavior.++## Design Principles++- **Explicit configuration over auto-discovery.** Operators name and+ enable each MCP server. Nothing is connected by default.+- **Least privilege and allowlists by default.** Discovered tools are+ inert until allowlisted. The default allowlist is empty.+- **Tool namespace isolation.** MCP tools appear to agents under a fixed+ prefix (proposed: `mcp.<server>.<tool>`) so they cannot collide with+ native tools or with tools from another MCP server.+- **User- and admin-visible tool inventory.** Operators can see every+ configured server, every discovered tool, every allowlisted tool, and+ the last successful discovery time, without reading raw logs.+- **Auditable execution records.** Each MCP tool invocation is logged+ with the same shape as native tool calls: who, when, which flow /+ task / subtask, input, result, and error.+- **Secrets redaction.** Server credentials, headers, and MCP tool+ arguments that look like secrets are never echoed to prompts, logs,+ or UI surfaces.+- **Container and host network boundary clarity.** It is explicit per+ server whether the MCP endpoint sits inside the PentAGI compose+ network, on the host (for example via `host.docker.internal`), or on+ a remote network the operator has chosen to expose.+- **Failure isolation.** A broken or unreachable MCP server surfaces as+ a tool error or a degraded discovery state. It must not crash the+ agent loop, the tool registry, or any native tool.++## Proposed v1 Design++### Configuration model++A future implementation should accept a list of named MCP servers, each+with explicit fields. Names are stable identifiers used in the tool+namespace and the audit trail. Server entries are off by default and+require an explicit `enabled: true` (or equivalent) before any+connection is attempted.++Each entry should describe at minimum:++- a stable server name (used as the namespace prefix segment);+- the transport (see below);+- transport-specific connection details (URL for HTTP/SSE, command and+ argv for stdio);+- optional headers, tokens, or other authentication material, sourced+ from existing secret patterns rather than hard-coded;+- a per-server tool allowlist;+- bounded timeouts and a maximum response size;+- an optional human-readable description shown to operators.++The exact storage shape (env, mounted YAML, DB row, GraphQL settings+mutation) is **not** chosen here. The only constraints this RFC+asserts are that the configuration must be inspectable to operators+and that secrets must not leak into prompts, logs, or UI surfaces.++### Transports++The MCP ecosystem currently uses three primary transports: HTTP, SSE,+and stdio. They have very different operational profiles:++- **HTTP and SSE** are remote-friendly. They fit the existing PentAGI+ deployment shape (containerized backend reaching a service over the+ network) and can target either a host-side endpoint via+ `host.docker.internal` or a remote endpoint chosen by the operator.+ These are the proposed first-class transports for v1.+- **stdio** assumes the MCP client can spawn a child process and pipe+ JSON-RPC over its stdin and stdout. In a containerized PentAGI+ deployment the question of *which* container that child runs in,+ with *which* binaries on PATH, and with *which* network egress, is+ non-trivial. v1 should not promise stdio support beyond the narrow+ case of a server-side managed command shipped or installed by the+ operator. A clean stdio story is a later milestone.++### Tool discovery++Discovery uses the standard MCP `tools/list` method. A successful+discovery records: the set of advertised tool names, their input+schemas, their human-readable descriptions, and a timestamp. A failed+discovery records the error (transport, authentication, schema parse,+timeout) without retrying in a tight loop. Discovery results are+visible to operators independently of agent behavior.++Discovery does not by itself expose any tool to agents. It only updates+the inventory operators can see and allowlist from.++### Tool naming and exposure++Allowlisted tools are presented to agents under a fixed prefix. This+RFC suggests `mcp.<server>.<tool>` -- for example,+`mcp.burp.start_scan` or `mcp.nuclei.run_template`. The prefix is+reserved so it cannot be confused with native tool names, which today+do not start with `mcp.`.++Exposure is allowlist-driven. An MCP tool is callable by agents only+if the operator has both (a) enabled the server and (b) added the+tool to that server's allowlist. The default allowlist is empty.++Per-server and per-tool metadata -- name, description, input schema,+last discovery time, allowlist status, last invocation, last error --+should be visible to operators in whatever surface PentAGI uses for+existing tool and provider configuration. The exact UI surface is+left to the implementation PR; this RFC only asserts that the+information must be visible somewhere operators already look.++### Invocation, audit, and failure surface++An MCP tool call should flow through the same broad permission and+audit mindset as a native tool call. At minimum each MCP invocation+should record the same fields as a native tool call (flow, task,+subtask, agent, tool name, input arguments after redaction, result+summary, error if any, duration).++Failure semantics:++- A transport error, server-side error, or schema mismatch is+ surfaced to the agent as a tool error. The agent loop continues.+- A timeout cancels the in-flight request and is logged as a tool+ error.+- A discovery failure marks the server as degraded and removes its+ previously-allowlisted tools from the agent-facing surface until+ rediscovery succeeds. It does not affect native tools or tools from+ other MCP servers.+- A misbehaving MCP server cannot crash the agent loop or block+ native tools.++### Bounded timeouts and payload limits++Every MCP request must have a per-call timeout and a maximum response+size. Both are configurable per server with conservative defaults. An+overlong response is truncated at the boundary, with a marker, and the+truncation event is recorded in the invocation log so operators can+see that the agent did not receive the full payload.++### Secrets++Server credentials, headers, tokens, and MCP tool arguments that match+known secret patterns must be sourced from existing PentAGI secret+storage where possible and must never appear verbatim in prompts,+agent context, audit logs displayed in the UI, or error messages+surfaced to agents. The exact secret-storage choice (existing env+handling, existing provider-credential patterns, a future dedicated+store) is a question for the implementation PR, not this RFC.++### Schema, API, and UI implications (deferred)++A future implementation may need new database rows for MCP server+records, new GraphQL types for the inventory and allowlist, new audit+log columns, and new UI surfaces. This RFC does **not** choose those+shapes. They are explicitly deferred to the implementation PR(s) so+that the design here can be reviewed without committing to a+migration.++## Burp Suite MCP Example++This section is illustrative. PentAGI does not implement Burp Suite+MCP support today, and shipping this RFC does not change that. The+example exists to make the v1 design concrete against a real MCP+server.++PortSwigger publishes an official Burp MCP server extension at+[`PortSwigger/mcp-server`](https://github.com/PortSwigger/mcp-server),+which the issue body references as the motivating use case.++### Possible operator setup++A pentester running Burp Suite Pro on their own analyst workstation,+or on a controlled internal host, enables the Burp MCP extension. The+extension exposes an MCP endpoint over HTTP on localhost. From a+PentAGI compose deployment running on the same workstation, that+endpoint is reachable as `http://host.docker.internal:<port>` from+the backend container, which is the existing pattern PentAGI already+uses for other host-reachable services.++The operator adds a single named server entry (call it `burp`),+explicitly enables it, supplies any required header or token, and+allowlists a small set of read-only tools. Nothing else changes.++### Candidate tool categories++The Burp MCP server, depending on version and configured access, can+expose a range of capabilities. From the issue's framing, useful+categories for PentAGI agents would include:++- reading the current Burp sitemap or list of discovered URLs;+- starting an active scan against a specific in-scope URL;+- retrieving structured findings (vulnerability type, severity,+ request and response evidence) for incorporation into the agent's+ reasoning and reports;+- retrieving Collaborator or other out-of-band events, if the MCP+ server exposes them, to detect blind classes of vulnerabilities+ that PentAGI's CLI-only toolset cannot detect today.++### Scope discipline++Active scan tools are not read-only. They send traffic, can be noisy,+and can be destructive against fragile targets. They must be:++- explicitly allowlisted per server;+- only invoked against targets the operator has already declared as+ in scope for the engagement;+- bounded by the same scope and approval mindset PentAGI applies to+ other intrusive tools.++PentAGI must not be inferred from this RFC to "freely scan arbitrary+targets" once Burp MCP is wired in. The agent only ever invokes a+Burp MCP tool that an operator has both configured and allowlisted,+against a target that already exists inside the engagement's scope.++### Illustrative configuration (non-final pseudocode)++The shape below is **non-final pseudocode** to make the design+tangible. It is not a proposed final environment-variable schema,+YAML schema, or database schema. The implementation PR may use any of+those, or something else.++```text+# illustrative only -- not a final config schema+mcp:+ servers:+ - name: burp+ enabled: true+ transport: http+ url: http://host.docker.internal:1337+ timeout_seconds: 30+ max_response_bytes: 1048576+ auth:+ header: X-API-Key+ secret_ref: burp_mcp_token+ allowlist:+ - get_sitemap+ - get_findings+ - start_active_scan+```++## Security and Safety++MCP support meaningfully expands PentAGI's attack surface and the+agent's reach. The following risks should be addressed by the v1+design rather than left for "later":++- **SSRF and internal network pivot.** A misconfigured `url` value+ can point an enabled MCP server entry at internal infrastructure.+ The implementation should validate URLs at configuration time and+ at request time, and operators should be made aware that an MCP+ server entry effectively grants the agent access to whatever+ network that endpoint sits on.+- **Prompt injection through tool descriptions and results.** MCP+ tool descriptions and results flow into the agent's context. A+ hostile or compromised MCP server can therefore attempt to inject+ instructions the same way a hostile web page can. Tool descriptions+ should be treated as untrusted text, not as instructions the agent+ must follow.+- **Destructive or high-impact MCP tools.** Some MCP tools can modify+ state (start scans, send messages, write to ticketing systems, log+ findings). The allowlist is the primary control here. A future+ per-tool approval step (see Open Questions) may be appropriate for+ the highest-impact tools.+- **Host access via `host.docker.internal`.** This is a real and+ useful capability, but it must be opt-in per server and clearly+ visible to operators. The operator should know that enabling a+ host-reachable MCP server gives the agent reach into the host's+ network namespace.+- **Credential handling and redaction.** Server credentials and any+ tool argument that matches a known secret pattern must be redacted+ in logs, redacted in any UI surface, and never echoed back into+ agent context. This includes tokens passed as headers, query+ parameters, or tool arguments.+- **Scope enforcement for scan targets.** When an MCP tool takes a+ target URL, hostname, or IP, the implementation should be able to+ reuse PentAGI's existing scope notion (target derived from the flow+ or engagement) to refuse out-of-scope inputs, rather than relying+ on the agent to self-police. The exact mechanism is open (see Open+ Questions).+- **Tool output size limits.** The bounded `max_response_bytes` from+ the configuration model also serves as a denial-of-context control:+ an MCP server cannot flood the agent's context with arbitrary+ payloads.+- **Audit trail.** Operators should be able to answer "who enabled+ this server" and "which agent invoked which MCP tool, when, against+ what target" without reading raw logs. Audit records must be tied+ to flow / task / subtask the same way native tool calls are.++Recommended safe defaults:++- All MCP servers disabled by default.+- All allowlists empty by default.+- All timeouts and response-size caps set to conservative values.+- Secrets never echoed into prompts, logs, or UI.+- Discovery never automatically promotes a tool into the allowlist.++## Observability and Auditability++For each configured MCP server, operators should be able to see:++- connection status (enabled / disabled, reachable / unreachable);+- discovery status (last successful discovery time, last error);+- the set of currently advertised tools and their schemas;+- the per-server allowlist and its diff against the advertised set;+- the most recent invocation outcomes, including errors and timeouts.++For each MCP tool invocation, the audit trail should record:++- the flow, task, and subtask the call belongs to;+- the agent that initiated the call;+- the tool name in its namespaced form (`mcp.<server>.<tool>`);+- the redacted input arguments and a redacted summary or hash of the+ output;+- the duration, error, and any truncation events.++MCP tool outputs should map into the existing flow / task / subtask+context the same way native tool outputs do, so that downstream+reporting can treat them uniformly. Future alignment with the evidence+and reporting direction discussed in+[issue #235](https://github.com/vxcontrol/pentagi/issues/235) (see+also the existing `examples/proposals/evidence_chain.md`) is desirable+but should not be a hard dependency for shipping the first MCP+milestone.++## Open Questions++- Which transports should ship first, and what is the minimum+ supported transport set for v1? (The leaning above is HTTP and SSE+ first; stdio later or only as a server-side managed command.)+- Should MCP servers be configured globally for the deployment,+ per-user, or per-flow? Each level has very different implications+ for audit, sharing, and isolation.+- Should MCP tool approval happen entirely in configuration, in the+ UI, or both? A pure-config flow is simpler; a UI flow allows+ per-tool review at first use.+- How should MCP tool output be summarized or truncated when it+ exceeds the response-size cap, both in the agent loop and in the+ audit log?+- How should MCP tool calls fit with PentAGI's existing permission+ and scope controls -- particularly target scope for scan-style+ tools? Should scope checks be enforced by the MCP layer, by the+ tool registry, or by a shared scope service?+- Should MCP tool calls become first-class report and evidence+ artifacts in their own right, beyond the per-call audit log? This+ overlaps with the evidence-chain RFC but does not have to be solved+ in the first milestone.+- How should Burp-specific scope (Burp's own in-scope and+ out-of-scope configuration) be represented to PentAGI without+ hard-coding Burp semantics into core? A generic "scope hint"+ attached to MCP tools is one possibility; a Burp-specific guide is+ another.++## Suggested First Milestone++The first MCP milestone should be deliberately narrow so each+lifecycle change is small enough to review in isolation:++1. **This RFC.** Land the design surface, gather maintainer feedback,+ and freeze the boundaries (namespace, allowlist-by-default, no+ hidden background execution).+2. **Read-only discovery for one configured HTTP or SSE MCP server.**+ The backend can connect to a single named server, run `tools/list`,+ and surface the resulting inventory plus discovery status to+ admins and operators. No agent execution yet, or at most a single+ allowlisted read-only tool whose output is bounded and audited the+ same way native tool calls are.+3. **Allowlisted execution with audit logs and target-scope checks.**+ Once read-only discovery is shipping, extend to allowlisted MCP+ tool invocation by agents, with full audit and with explicit+ handling of target scope for scan-style tools.+4. **Burp-specific operator guide.** After the generic MCP client+ behavior is proven, document the Burp Suite Pro setup as an+ operator guide (similar to+ `examples/proposals/osint-integration-scenarios.md` for OSINT+ providers) -- so Burp lands on top of generic MCP rather than as a+ bespoke integration.++Each milestone is intentionally self-contained: a maintainer can stop+the work after any step without leaving PentAGI in a half-shipped+state.