π OpenHands: AI-Driven Development
DRAFT: docs: update plugin launch flow with complete data flow (#14303)
John-Mason P. Shackelford2cc7e15aba4f
diff --git a/enterprise/doc/design-doc/plugin-launch-flow.md b/enterprise/doc/design-doc/plugin-launch-flow.mdindex fa30e3fa31..0ba5b47b61 100644--- a/enterprise/doc/design-doc/plugin-launch-flow.md+++ b/enterprise/doc/design-doc/plugin-launch-flow.md@@ -1,99 +1,555 @@# Plugin Launch Flow-This document describes how plugins are launched in OpenHands Saas / Enterprise, from the plugin directory through to agent execution.+This document traces the complete data flow for launching plugins in OpenHands, from the source marketplace through to agent execution. Each section shows the exact endpoints, payloads, and transformations.## Architecture Overview```-Plugin Directory βββΆ Frontend /launch βββΆ App Server βββΆ Agent Server βββΆ SDK- (external) (modal) (API) (in sandbox) (plugin loading)+Marketplace βββΆ Plugin Directory βββΆ Frontend /launch βββΆ App Server βββΆ Agent Server βββΆ SDK+ (GitHub) (Index + UI) (Modal) (API) (in sandbox) (plugin loading)```| Component | Responsibility ||-----------|---------------|-| **Plugin Directory** | Index plugins, present to user, construct launch URLs |+| **Marketplace** | Source of truth for plugin catalog (GitHub repo) |+| **Plugin Directory** | Index plugins from marketplace, serve browsing UI, construct launch URLs || **Frontend** | Display confirmation modal, collect parameters, call API |-| **App Server** | Validate request, pass plugin specs to agent server |+| **App Server** | Validate request, create conversation, pass plugin specs to agent server || **Agent Server** | Run inside sandbox, delegate plugin loading to SDK || **SDK** | Fetch plugins, load contents, merge skills/hooks/MCP into agent |-## User Experience+----### Plugin Directory+## Step 1: Marketplace (GitHub)-The plugin directory presents users with a catalog of available plugins. For each plugin, users see:-- Plugin name and description (from `plugin.json`)-- Author and version information-- A "Launch" button+**Source**: A GitHub repository (e.g., `github.com/OpenHands/plugin-marketplace`)-When a user clicks "Launch", the plugin directory:-1. Reads the plugin's `entry_command` to know which slash command to invoke-2. Determines what parameters the plugin accepts (if any)-3. Redirects to OpenHands with this information encoded in the URL+The marketplace is a GitHub repository containing a `marketplace.json` that indexes all available plugins.-### Parameter Collection+### marketplace.json-If a plugin requires user input (API keys, configuration values, etc.), the frontend displays a form modal before starting the conversation. Parameters are passed in the launch URL and rendered as form fields based on their type:+```json+{+ "name": "OpenHands Plugin Marketplace",+ "owner": {+ "name": "OpenHands",+ "email": "[email protected]"+ },+ "metadata": {+ "description": "Official OpenHands plugin marketplace",+ "pluginRoot": "plugins"+ },+ "plugins": [+ {+ "name": "city-weather",+ "source": "github:jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather",+ "description": "Get current weather for any city",+ "tags": ["weather", "utility"]+ }+ ]+}+```-- **String values** β Text input-- **Number values** β Number input-- **Boolean values** β Checkbox+### Plugin Source (`plugin.json`)-Only primitive types are supported. Complex types (arrays, objects) are not currently supported for parameter input.+Each plugin has a `plugin.json` in its `.claude-plugin/` directory. This file contains both official plugin manifest fields and optional directory-specific config fields:-The user fills in required values, then clicks "Start Conversation" to proceed.+```json+{+ "name": "city-weather",+ "description": "Get current weather for any city",+ "entry_command": "now",+ "parameters": {+ "city": {+ "type": "string",+ "description": "City name",+ "required": true,+ "default": "San Francisco"+ }+ },+ "examples": [+ {+ "title": "Check Tokyo weather",+ "prompt": "/city-weather:now Tokyo"+ }+ ]+}+```-## Launch Flow+**Output to Plugin Directory**: `marketplace.json` + individual `plugin.json` files-1. **Plugin Directory** (external) constructs a launch URL to the OpenHands app server when user clicks "Launch":- ```- /launch?plugins=BASE64_JSON&message=/city-weather:now%20Tokyo- ```+---++## Step 2: Plugin Directory Server++**Endpoints**:+- `GET /api/plugins` - List all plugins+- `GET /api/plugins/{id}` - Get plugin details+- `GET /api/plugins/{id}/config` - Get plugin config (entry_command, parameters, examples)- The `plugins` parameter includes any parameter definitions with default values:+### GET /api/plugins++Fetches and transforms the marketplace catalog.++**Request**: None (fetches from configured `MARKETPLACE_SOURCE`)++**Response**:+```json+{+ "plugins": [+ {+ "id": "city-weather",+ "name": "city-weather",+ "description": "Get current weather for any city",+ "source": {+ "source": "github",+ "repo": "jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather"+ },+ "tags": ["weather", "utility"]+ }+ ]+}+```++### GET /api/plugins/{id}/config++Fetches and returns the config fields from `plugin.json`.++**Request**: `GET /api/plugins/city-weather/config`++**Response** (200 OK):+```json+{+ "entry_command": "now",+ "parameters": {+ "city": {+ "type": "string",+ "description": "City name",+ "required": true,+ "default": "San Francisco"+ }+ },+ "examples": [+ {+ "title": "Check Tokyo weather",+ "prompt": "/city-weather:now Tokyo"+ }+ ]+}+```++**Output to Plugin Directory Client**: Plugin metadata + config++---++## Step 3: Plugin Directory Client++When user clicks "Launch", the client constructs a launch URL using `buildLaunchUrl()`.++### buildLaunchUrl() Input++From Plugin Directory Server APIs:+- **Plugin** (from `/api/plugins/{id}`):+ ```json+ {+ "name": "city-weather",+ "source": {+ "source": "github",+ "repo": "jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather"+ }+ }+ ```+- **PluginConfig** (from `/api/plugins/{id}/config`):+ ```json+ {+ "entry_command": "now",+ "parameters": {+ "city": { "type": "string", "required": true, "default": "San Francisco" }+ }+ }+ ```++### buildLaunchUrl() Transformation++1. **Build PluginSpec** from plugin source:+ - `source`: Convert to string format `"github:owner/repo"`+ - `ref`: Extract git ref if present+ - `repo_path`: Extract subdirectory path if present+ - `parameters`: Extract default values from parameter definitions++2. **Build message** using `buildEntrySlashCommand(pluginName, entryCommand)`:+ - Combines plugin name + entry_command β `"/city-weather:now"`+ - Does NOT include parameter values (App Server will add them later)++3. **Encode and construct URL**:+ - Base64-encode the PluginSpec array as `plugins` query param+ - Add slash command as `message` query param++### buildLaunchUrl() Output++**Launch URL**:+```+https://app.openhands.ai/launch?plugins=BASE64&message=%2Fcity-weather%3Anow+```++Where `plugins` (base64-decoded) contains:+```json+[{+ "source": "github:jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather",+ "parameters": {+ "city": "San Francisco"+ }+}]+```++And `message` (URL-decoded) is:+```+/city-weather:now+```++**Key point**: The `parameters` in the PluginSpec contain **default values** for pre-filling the launch modal form. The `message` contains only the slash commandβthe Frontend passes it through unchanged, and the App Server appends the parameter values as a formatted text block.++---++## Step 4: OpenHands Frontend (`/launch` Route)++**Route**: `/launch?plugins=BASE64&message=/city-weather:now`++[PR #12699](https://github.com/OpenHands/OpenHands/pull/12699)++### Input (from URL query params)++- `plugins`: Base64-encoded JSON array of PluginSpec+- `message`: Pre-filled slash command (no parameter values)++**Decoded**:+```json+{+ "plugins": [{+ "source": "github:jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather",+ "parameters": { "city": "San Francisco" }+ }],+ "message": "/city-weather:now"+}+```++### Modal Display++The frontend displays a confirmation modal:+1. Shows plugin info+2. Renders parameter form fields based on `plugins[].parameters`:+ - Text input for `city`, pre-filled with `"San Francisco"`+3. Shows message preview: `/city-weather:now`++### User Submits++When user clicks "Start Conversation":++1. **Collect final parameter values** from form inputs:+ - User changed `city` from `"San Francisco"` to `"Tokyo"`++2. **Update PluginSpec parameters** with user's values:```json- [{- "source": "github:owner/repo",- "repo_path": "plugins/my-plugin",- "parameters": {"api_key": "", "timeout": 30, "debug": false}- }]+ "parameters": { "city": "Tokyo" }```-2. **OpenHands Frontend** (`/launch` route, [PR #12699](https://github.com/OpenHands/OpenHands/pull/12699)) displays modal with parameter form, collects user input+3. **Pass message through unchanged**:+ - The message `/city-weather:now` is NOT modified by the Frontend+ - Parameter values are passed in `plugins[].parameters`, not in the message++### Output (API call to App Server)++```+POST /api/v1/app-conversations+Content-Type: application/json+Authorization: Bearer <user_token>++{+ "plugins": [{+ "source": "github:jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather",+ "parameters": {+ "city": "Tokyo"+ }+ }],+ "initial_message": {+ "role": "user",+ "content": [{"type": "text", "text": "/city-weather:now"}]+ }+}+```++**Summary of transformations**:+| Field | Input (from URL) | Output (to API) |+|-------|------------------|-----------------|+| `plugins[].parameters` | Default values (`"San Francisco"`) | User's values (`"Tokyo"`) |+| `initial_message.text` | Slash command (`/city-weather:now`) | Slash command unchanged (`/city-weather:now`) |++**Note**: The Frontend does NOT append parameter values to the message. Parameters are passed as structured data in `plugins[].parameters`. The App Server will append them to the message text (see Step 5).++----3. **OpenHands App Server** ([PR #12338](https://github.com/OpenHands/OpenHands/pull/12338)) receives the API call:+## Step 5: OpenHands App Server++**Endpoint**: `POST /api/v1/app-conversations`++[PR #12338](https://github.com/OpenHands/OpenHands/pull/12338)++### Input (API Request)++```json+{+ "plugins": [{+ "source": "github:jpshackelford/openhands-sample-plugins",+ "ref": "main",+ "repo_path": "plugins/city-weather",+ "parameters": { "city": "Tokyo" }+ }],+ "initial_message": {+ "role": "user",+ "content": [{"type": "text", "text": "/city-weather:now"}]+ }+}+```++Note: The `initial_message.text` contains only the slash commandβparameter values come separately in `plugins[].parameters`.++### Request Schema++```python+class PluginSpec(PluginSource):+ """Extends SDK's PluginSource with user-provided parameters."""+ parameters: dict[str, Any] | None = None # User-provided values++class AppConversationStartRequest(BaseModel):+ plugins: list[PluginSpec] | None = None+ initial_message: SendMessageRequest | None = None+ # ... other fields+```++### Processing & Transformation++**Call stack** in `LiveStatusAppConversationService`:++1. **`_construct_initial_message_with_plugin_params()`** - Appends parameters to message:+ ```python+ # Original message: "/city-weather:now"+ # Parameters: {"city": "Tokyo"}+ # Result: "/city-weather:now\n\nPlugin Configuration Parameters:\n- city: Tokyo"```- POST /api/v1/app-conversations- {- "plugins": [{"source": "github:owner/repo", "repo_path": "plugins/city-weather"}],- "initial_message": {"content": [{"type": "text", "text": "/city-weather:now Tokyo"}]}- }++2. **Convert PluginSpec β SDK PluginSource** (parameters are DROPPED):+ ```python+ sdk_plugins = [+ PluginSource(+ source=p.source, # "github:jpshackelford/openhands-sample-plugins"+ ref=p.ref, # "main"+ repo_path=p.repo_path # "plugins/city-weather"+ )+ # NOTE: p.parameters is NOT passed to SDK PluginSource!+ for p in plugins+ ]```- Call stack:- - `AppConversationRouter` receives request with `PluginSpec` list- - `LiveStatusAppConversationService._finalize_conversation_request()` converts `PluginSpec` β `PluginSource`- - Creates `StartConversationRequest(plugins=sdk_plugins, ...)` and sends to agent server+3. **Create StartConversationRequest** for agent server++### Output (to Agent Server)++```python+StartConversationRequest(+ plugins=[+ PluginSource(+ source="github:jpshackelford/openhands-sample-plugins",+ ref="main",+ repo_path="plugins/city-weather"+ # NO parameters field - SDK PluginSource doesn't have it+ )+ ],+ initial_message=SendMessageRequest(+ content=[+ TextContent(+ text="/city-weather:now\n\nPlugin Configuration Parameters:\n- city: Tokyo"+ )+ ]+ ),+ # ... other fields+)+```++**β οΈ CRITICAL**: Plugin parameters are passed to the agent via **message text**, not via the `PluginSource` object. The SDK's `PluginSource` class only has `source`, `ref`, and `repo_path` fields.++**Note on message construction**: The original slash command `/city-weather:now` does NOT include the parameter value "Tokyo" inline. The parameter appears only in the formatted "Plugin Configuration Parameters" block appended by the App Server.++---++## Step 6: Agent Server (in Sandbox)++**Entry point**: `ConversationService.start_conversation()`++[SDK PR #1651](https://github.com/OpenHands/software-agent-sdk/pull/1651)++### Input (`StartConversationRequest`)++```python+StartConversationRequest(+ plugins=[+ PluginSource(+ source="github:jpshackelford/openhands-sample-plugins",+ ref="main",+ repo_path="plugins/city-weather"+ )+ ],+ initial_message=SendMessageRequest(+ content=[+ TextContent(+ text="/city-weather:now\n\nPlugin Configuration Parameters:\n- city: Tokyo"+ )+ ]+ )+)+```++### Processing++**Call stack**:+1. `ConversationService.start_conversation(request)` receives `StartConversationRequest`+2. Creates `StoredConversation` with plugin specs persisted+3. Creates `LocalConversation(plugins=request.plugins, ...)`+4. Plugin loading deferred until first `run()` or `send_message()`++### Output (`LocalConversation`)++```python+LocalConversation(+ agent=agent,+ plugins=[PluginSource(...)], # Stored, not yet loaded+ workspace=workspace,+ # initial_message queued for processing+)+```++---++## Step 7: SDK Plugin Loading++**Trigger**: First `conversation.run()` or `conversation.send_message()`++[SDK PR #1647](https://github.com/OpenHands/software-agent-sdk/pull/1647)++### Input (`PluginSource` list)++```python+[+ PluginSource(+ source="github:jpshackelford/openhands-sample-plugins",+ ref="main",+ repo_path="plugins/city-weather"+ )+]+```++### Processing++**Call stack**:+1. `LocalConversation._ensure_plugins_loaded()` triggered+2. For each `PluginSource`:+ - `Plugin.fetch(source, ref, repo_path)` β clones/caches git repo+ - `Plugin.load(path)` β parses `plugin.json`, loads commands/skills/hooks+3. `plugin.add_skills_to(skill_context)` β merges skills into agent+4. `plugin.add_mcp_config_to(mcp_config)` β merges MCP servers++### Output (`Plugin` object)++```python+Plugin(+ name="city-weather",+ path="/tmp/plugins/city-weather",+ manifest=PluginManifest(+ name="city-weather",+ entry_command="now", # Read from plugin.json+ commands={"now": Command(...)},+ skills=[Skill(...)],+ hooks={...},+ mcp_servers={...}+ )+ # NOTE: No parameters field - parameters are in the message text+)+```++---++## Step 8: Agent Receives Message++The agent now has:+- Plugin skills merged into its skill context (including `/city-weather:now` command as a skill)+- MCP servers configured and running+- The initial message in its conversation++### Message Content++```+/city-weather:now++Plugin Configuration Parameters:+- city: Tokyo+```++### Processing++When the agent processes the message:+1. Recognizes `/city-weather:now` as a slash command (keyword trigger)+2. The `KeywordTrigger` activates the command skill+3. The agent reads the parameter value from the "Plugin Configuration Parameters" block+4. The skill executes with `city=Tokyo`++**Note**: Parameters are NOT passed as structured data to the plugin. The agent reads them from the message text in the formatted "Plugin Configuration Parameters" block appended by the App Server.-4. **Agent Server** (inside sandbox, [SDK PR #1651](https://github.com/OpenHands/software-agent-sdk/pull/1651)) stores specs, defers loading:+---- Call stack:- - `ConversationService.start_conversation()` receives `StartConversationRequest`- - Creates `StoredConversation` with plugin specs- - Creates `LocalConversation(plugins=request.plugins, ...)`- - Plugin loading deferred until first `run()` or `send_message()`+## Complete Data Flow Summary-5. **SDK** fetches and loads plugins on first use:+| Step | Component | Input | Output |+|------|-----------|-------|--------|+| 1 | Marketplace | - | `marketplace.json` + `plugin.json` files |+| 2 | Plugin Directory Server | Marketplace files | REST API responses with `entry_command`, `parameters` |+| 3 | Plugin Directory Client | Plugin + Config | Launch URL: `plugins` (with defaults) + `message` (slash command only) |+| 4 | OpenHands Frontend | URL query params | API call: `plugins` (with user values) + `message` (unchanged slash command) |+| 5 | App Server | API request | `StartConversationRequest`: `PluginSource` (no params) + message (params in text) |+| 6 | Agent Server | `StartConversationRequest` | `LocalConversation` with deferred plugins |+| 7 | SDK | `PluginSource` list | Loaded `Plugin` objects with skills/hooks/MCP |+| 8 | Agent | Initial message with params in text | Command execution |++### Parameter Journey++```+βββββββββββββββββββββββ βββββββββββββββββββββββ βββββββββββββββββββββββ+β Plugin Directory β β OpenHands Frontend β β App Server β+β β β β β β+β plugins[].params ββββββΆβ plugins[].params ββββββΆβ Appends params to β+β = defaults β β = user values β β message as text β+β β β (from form edit) β β block, then DROPS β+β β β β β from PluginSource β+β message = β β message = β β β+β /cmd:entry ββββββΆβ /cmd:entry ββββββΆβ Final message: β+β (no values) β β (unchanged!) β β /cmd:entry β+β β β β β + params block β+βββββββββββββββββββββββ βββββββββββββββββββββββ βββββββββββββββββββββββ+```- Call stack:- - `LocalConversation._ensure_plugins_loaded()` triggered by first message- - For each plugin spec:- - `Plugin.fetch(source, ref, repo_path)` β clones/caches git repo- - `Plugin.load(path)` β parses `plugin.json`, loads commands/skills/hooks- - `plugin.add_skills_to(context)` β merges skills into agent- - `plugin.add_mcp_config_to(config)` β merges MCP servers+**Key insight**: The Frontend does NOT modify the message. It passes the slash command through unchanged and sends parameters as structured data in `plugins[].parameters`. The App Server is responsible for formatting parameters into the message text.-6. **Agent** receives message, `/city-weather:now` triggers the skill+---## Key Design Decisions@@ -104,28 +560,39 @@ Plugins load **inside the sandbox** because:- MCP servers run inside the sandbox- Skills may reference sandbox filesystem-### Entry Command Handling+### Entry Command vs Full Message-The `entry_command` field in `plugin.json` allows plugin authors to declare a default command:+The `entry_command` field contains only the command name (e.g., `"now"`), not the full slash command. This separation allows:+- Plugin Directory to construct the slash command from plugin name + entry_command+- Frontend to collect user-provided parameter values via form UI+- App Server to format parameters into the message text+- Flexibility for the launch experience to differ from direct SDK usage-```json-{- "name": "city-weather",- "entry_command": "now"-}-```+### Parameter Flow (Important!)++Parameters travel through the system as **structured data until the App Server**, where they are converted to text:++1. **Structured data path** (`PluginSpec.parameters`):+ - Plugin Directory β Frontend β App Server API+ - Used for form rendering (pre-fill defaults, collect user edits)+ - **Formatted into message text by App Server**+ - **Then dropped** (not passed to SDK `PluginSource`)++2. **Message path**:+ - Plugin Directory sends slash command only (e.g., `/city-weather:now`)+ - Frontend passes it through **unchanged**+ - App Server appends formatted parameter block to the message+ - **This is how the agent receives parameter values**-This flows through the system:-1. Plugin author declares `entry_command` in plugin.json-2. Plugin directory reads it when indexing-3. Plugin directory includes `/city-weather:now` in the launch URL's `message` parameter-4. Message passes through to agent as `initial_message`+The SDK's `PluginSource` class intentionally does NOT have a `parameters` field. All parameter context is communicated to the agent via the initial message text, specifically in the "Plugin Configuration Parameters" block appended by the App Server.-The SDK exposes this field but does not auto-invoke itβcallers control the initial message.+----## Related+## Related PRs- [OpenHands PR #12338](https://github.com/OpenHands/OpenHands/pull/12338) - App server plugin support- [OpenHands PR #12699](https://github.com/OpenHands/OpenHands/pull/12699) - Frontend `/launch` route- [SDK PR #1651](https://github.com/OpenHands/software-agent-sdk/pull/1651) - Agent server plugin loading- [SDK PR #1647](https://github.com/OpenHands/software-agent-sdk/pull/1647) - Plugin.fetch() for remote plugin fetching+- [SDK PR #2230](https://github.com/OpenHands/software-agent-sdk/pull/2230) - entry_command field definition+- [Plugin Directory PR #84](https://github.com/OpenHands/plugin-directory/pull/84) - entry_command support in plugin directory