Skip to content

SmythOS Studio: Open-Source Visual AI Agent Builder and deployable runtime stack from SmythOS. Start with an intuitive drag-and-drop workspace, extend with custom code, and deploy your agents anywhere — local, cloud, or edge — with full governance and control.

update README.md

Forhad Hosain53f232521e86
diff --git a/packages/runtime/README.md b/packages/runtime/README.md
index 38722dab..35db993f 100644
--- a/packages/runtime/README.md
+++ b/packages/runtime/README.md
@@ -1,118 +1,108 @@
-# SRE Runtime Server
+# SmythOS Runtime Server
-A Runtime Server that combines the functionality of three separate SmythOS Runtime servers:
-
-- **sre-agent-server** (Agent Runner)
-- **sre-builder-debugger** (Debugger)
-- **sre-embodiment-server** (Embodiment)
+A unified runtime server that provides comprehensive execution, debugging, and integration capabilities for SmythOS agents.
## 📚 Table of Contents
-- [🚀 Quick Start - Smart Routing](#-quick-start---smart-routing)
-- [🚀 Smart Routing System](#-smart-routing-system) ⭐ **New!**
-- [Server Modes](#server-modes)
-- [Installation & Usage](#installation--usage)
-- [Configuration](#configuration)
-- [Health Check](#health-check)
+- [🚀 Quick Start](#-quick-start)
+- [💡 What is SmythOS Runtime?](#-what-is-smythos-runtime)
+- [🏗️ Architecture](#️-architecture)
+- [⚙️ Installation & Usage](#️-installation--usage)
+- [🔧 Configuration](#-configuration)
+- [🌐 API Modules](#-api-modules)
+- [🚀 Smart Routing](#-smart-routing)
+- [🌐 Deployment](#-deployment)
+- [🔍 Troubleshooting](#-troubleshooting)
-## 🚀 Quick Start - Smart Routing
+## 🚀 Quick Start
```bash
-# Default (Agent Runner)
-curl http://localhost:5000/api/test
-
-# Force Agent Runner
-curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5000/api/test
+# Install dependencies
+pnpm install
-# Force Debugger
-curl -H "X-FORCE-DEBUGGER: true" http://localhost:5000/api/test
+# Start development server
+pnpm run dev
-# Debug Session
-curl -H "X-DEBUG-RUN: true" http://localhost:5000/api/test
+# Health check
+curl http://localhost:5053/health
```
-> 📖 **[See full Smart Routing documentation below](#-smart-routing-system)**
+## 💡 What is SmythOS Runtime?
-## Server Modes
+The SmythOS Runtime Server is the execution engine for SmythOS agents. It provides:
-The server can run in different modes based on the `SERVER_MODE` environment variable:
+- **Agent Execution**: Run and manage SmythOS agents with OAuth authentication
+- **Debug Capabilities**: Debug agent workflows with real-time monitoring
+- **Integration Hub**: Connect agents to external services (ChatGPT, OpenAI, Alexa, etc.)
+- **API Gateway**: Unified API for all SmythOS runtime operations
-- `runtime` (default): All functionality enabled
-- `agent`: Only agent runner functionality
-- `debugger`: Only debugger functionality
-- `embodiment`: Only embodiment functionality
+## 🏗️ Architecture
-## Directory Structure
+The runtime server is organized into three core modules:
```
src/
-├── core/ # Shared functionality across all servers
-│ ├── services/ # Common services (Redis, logging, etc.)
-│ ├── middlewares/ # Common middlewares (CORS, rate limiting, etc.)
-│ ├── types/ # Shared TypeScript types
-│ └── utils/ # Utility functions
-├── agent-runner/ # Agent server specific code
-│ ├── routes/ # Agent routes (/oauth, /agent, etc.)
-│ └── middlewares/ # Agent-specific middlewares
-├── debugger/ # Debugger specific code
-│ ├── routes/ # Debugger routes (/models, etc.)
-│ └── services/ # Debugger services (component-async-ctx)
-└── embodiment/ # Embodiment specific code
- ├── modules/ # Embodiment modules (ChatGPT, OpenAI, etc.)
- ├── helpers/ # Helper functions
- └── routes/ # Embodiment routes
+├── core/ # Shared infrastructure
+│ ├── config/ # Configuration management
+│ ├── middlewares/ # Common middleware (CORS, auth, etc.)
+│ ├── services/ # Shared services
+│ ├── types/ # TypeScript definitions
+│ └── utils/ # Utility functions
+├── modules/ # Runtime modules
+│ ├── agent-runner/ # Agent execution & OAuth
+│ ├── debugger/ # Debug capabilities & monitoring
+│ └── embodiment/ # External integrations
+└── services/ # Additional services
+ └── code-sandbox.service.ts
```
-## Configuration
+### Core Components
-1. Copy `.env.example` to `.env` and configure your environment variables
-2. Set `PORT=5000` to run on port 5000
-3. Set `SERVER_MODE=runtime` to enable all functionality
+- **Agent Runner**: Executes SmythOS agents with authentication and rate limiting
+- **Debugger**: Provides debugging capabilities with component-level tracking
+- **Embodiment**: Handles integrations with external platforms and services
-## Installation & Usage
+## ⚙️ Installation & Usage
+
+### Development
```bash
# Install dependencies
-npm install
+pnpm install
-# Link smyth-runtime (if needed)
-npm link smyth-runtime
+# Start development server with auto-reload
+pnpm run dev
-# Development mode with auto-reload
-npm run dev
+# The server will start on http://localhost:5053
+```
-# Build for development
-npm run build:dev
+### Production
-# Build for production
-npm run build
+```bash
+# Build the application
+pnpm run build
-# Start the server
-npm run start
+# Start production server
+pnpm run start:prod
```
-## Available Scripts
+### Available Scripts
-### Core Scripts (Minimal)
-- `npm run dev` - Development mode with auto-reload using nodemon
-- `npm run build` - Build for production
-- `npm run start:prod` - Start server in production mode
+| Script | Description |
+|--------|-------------|
+| `pnpm run dev` | Development mode with auto-reload |
+| `pnpm run build` | Build for production |
+| `pnpm run start:prod` | Start production server |
+| `pnpm run test` | Run tests |
+| `pnpm run lint` | Lint TypeScript files |
+| `pnpm run clean` | Clean build directory |
-### Additional Scripts
-- `npm run test` - Run tests
-- `npm run clean` - Clean the dist directory
-- `npm run lint` - Lint TypeScript files
-- `npm run lint-fix` - Lint and auto-fix TypeScript files
-- `npm run ts-error-scan` - Check for TypeScript errors without emitting files
-- `npm run format` - Format TypeScript files with Prettier
-- `npm run prettier:format` - Format all supported files with Prettier
-- `npm run bump` - Bump patch version
+## 🔧 Configuration
-## 🔧 Environment Setup
+### Required Environment Variables
-### Required (for both development and production):
```bash
-# Authentication (get these from your LogTo setup)
+# Authentication (LogTo configuration)
LOGTO_M2M_APP_SECRET="your-m2m-app-secret"
LOGTO_M2M_APP_ID="your-m2m-app-id"
LOGTO_SERVER="https://auth.yourdomain.com"
@@ -120,13 +110,12 @@ LOGTO_API_RESOURCE="https://api.yourdomain.com"
MIDDLEWARE_API_BASE_URL="http://middleware.yourdomain.com"
```
-### Optional (development):
-```bash
-# Ports
-PORT="5053" # Runtime server port
-ADMIN_PORT="5054" # Admin interface port
+### Optional Configuration
-# URLs
+```bash
+# Server Configuration
+PORT="5053" # Runtime server port
+ADMIN_PORT="5054" # Admin interface port
BASE_URL="http://localhost:5053"
UI_SERVER="http://localhost:4000"
@@ -135,31 +124,70 @@ SESSION_SECRET="your-random-secret-here"
# Storage
DATA_PATH="/path/to/your/data"
+
+# Agent Domain Configuration
+DEFAULT_AGENT_DOMAIN="localagent.stage.yourdomain.ai"
+PROD_AGENT_DOMAIN="agentid.yourdomain.com"
```
-### Domain Configuration:
+> **⚠️ Local Development**: For local agent execution, ensure your domain contains `localagent`
-> **⚠️ WARNING**: To run the agent locally, the domain name must contain `localagent`
+## 🌐 API Modules
-When running agents in a local development environment, ensure your domain configuration includes the keyword `localagent`. This is required for proper local agent execution and routing.
+### Agent Runner (`/api/*`)
+- **OAuth Authentication**: `/oauth/*` - Handle OAuth flows
+- **Agent Execution**: `/agent/*` - Execute and manage agents
+- **File Operations**: File upload and processing capabilities
-```bash
-# Development/staging wildcard domain
-DEFAULT_AGENT_DOMAIN="localagent.stage.yourdomain.ai"
+### Debugger (`/models/*`)
+- **Model Management**: Team-based access control for AI models
+- **Debug Sessions**: Real-time agent debugging and monitoring
+- **Component Tracking**: Async component execution tracking
-# Production wildcard domain
-PROD_AGENT_DOMAIN="agentid.yourdomain.com"
-```
+### Embodiment (Various endpoints)
+- **ChatGPT Plugin**: `/chatgpt/*` - ChatGPT plugin integration
+- **OpenAI Proxy**: `/openai/*` - OpenAI API proxy
+- **Alexa Skills**: `/alexa/*` - Amazon Alexa integration
+- **MCP Protocol**: `/mcp/*` - Model Context Protocol support
+- **API Documentation**: `/swagger/*` - Interactive API docs
+- **Postman Collections**: `/postman/*` - Generate API collections
+
+## 🚀 Smart Routing
+
+The runtime server uses intelligent routing to direct requests to the appropriate module based on HTTP headers:
+
+### Quick Examples
-**Example:**
```bash
-# ✅ Correct - domain contains 'localagent'
-DEFAULT_AGENT_DOMAIN="localagent.stage.yourdomain.ai"
+# Default behavior (Agent Runner)
+curl http://localhost:5053/api/test
-# ❌ Incorrect - missing 'localagent' keyword
-DEFAULT_AGENT_DOMAIN="localhost:3000"
+# Force specific modules
+curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5053/api/test
+curl -H "X-FORCE-DEBUGGER: true" http://localhost:5053/api/test
+
+# Debug session
+curl -H "X-DEBUG-RUN: true" http://localhost:5053/api/test
```
+### Routing Headers
+
+| Header | Purpose | Routes To |
+|--------|---------|-----------|
+| `X-FORCE-AGENT-RUNNER` | Force agent execution | Agent Runner |
+| `X-FORCE-DEBUGGER` | Force debug mode | Debugger |
+| `X-DEBUG-RUN` | Start debug session | Debugger |
+| `X-MONITOR-ID` | Monitor agent execution | Debugger |
+
+### Server Modes
+
+Control server functionality with the `SERVER_MODE` environment variable:
+
+- `runtime` (default): Full functionality
+- `agent`: Agent runner only
+- `debugger`: Debugger only
+- `embodiment`: Integrations only
+
## 🌐 Deployment
### Environment Requirements
@@ -172,396 +200,106 @@ DEFAULT_AGENT_DOMAIN="localhost:3000"
**Production:**
- All development requirements
- Redis (for session management)
-- SSL certificates (for production domains)
+- SSL certificates
- Load balancer (recommended)
-### Quick Deploy
+### Production Deployment
-1. **Build everything:**
+1. **Build the application:**
```bash
- npm run build
+ pnpm run build
```
-2. **Set production environment:**
+2. **Set environment:**
```bash
NODE_ENV=production
PORT=5053
- # Add your production environment variables
+ # Add production environment variables
```
-3. **Start production server:**
+3. **Start server:**
```bash
- npm run start:prod
+ pnpm run start:prod
```
-### Production Checklist
+### Health Monitoring
-- [ ] Environment variables configured
-- [ ] Authentication server running
-- [ ] Database connected
-- [ ] Redis connected (for sessions)
-- [ ] SSL certificates installed
-- [ ] Domain routing configured
-- [ ] Health checks passing (`http://your-domain/health`)
+Check server health at: `http://localhost:5053/health`
+
+The health endpoint provides:
+- Server status and mode
+- Available modules
+- Version information
## 🔍 Troubleshooting
-### Common Issues:
+### Common Issues
-**"Agent not routing correctly"**
-```bash
-# Check routing with explicit headers
-curl -H "X-FORCE-DEBUGGER: true" http://localhost:5053/api/test
-```
-
-**"Authentication failing"**
+**Authentication Failing**
- Verify LogTo environment variables
-- Check LogTo server is accessible
-- Confirm M2M app is configured correctly
+- Ensure LogTo server is accessible
+- Check M2M app configuration
-**"Development server not starting"**
+**Routing Issues**
```bash
-# Clean and reinstall
-rm -rf node_modules
-npm install
+# Test with explicit headers
+curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5053/api/test
```
-**"Build failing"**
+**Build Problems**
```bash
# Increase memory for builds
export NODE_OPTIONS='--max-old-space-size=4096'
-npm run build
-```
-
-## Health Check
-
-Once running, check server health at:
-
-- `http://localhost:5000/health`
-
-The health endpoint shows:
-
-- Server mode
-- Available functionality
-- Version information
-
-## Conditional Routing
-
-The server automatically detects the mode and enables appropriate routes:
-
-### Agent Mode Routes
-
-- `/oauth/*` - OAuth authentication
-- `/agent/*` - Agent execution
-- `/debugger-proxy/*` - Debugger proxy
-
-### Debugger Mode Routes
-
-- `/models/*` - Model management with team access control
-- All agent routes (debugger extends agent functionality)
-
-### Embodiment Mode Routes
-
-- `/static/*` - Static file serving
-- `/swagger/*` - Swagger UI
-- Various embodiment modules:
- - `/chatgpt/*`
- - `/openai/*`
- - `/alexa/*`
- - `/mcp/*`
- - And more...
-
-## Mode Detection
-
-The server uses several methods to detect the appropriate functionality:
-
-1. **Environment Variable**: `SERVER_MODE` setting
-2. **Header Detection**: `x-smyth-debug: true` for debugger mode
-3. **Path Detection**: Certain paths automatically enable specific modes
-4. **User Agent**: SmythOS-Agent user agent for agent mode
-
-## Migration from Individual Servers
-
-This Runtime Server replaces the need for running three separate servers:
-
-- Port 5000 now handles all functionality
-- Environment variables from all three servers are supported
-- All routes and features are preserved
-
-## Features by Mode
-
-### Agent Runner Features
-
-- OAuth authentication
-- Agent execution and management
-- File uploads and processing
-- Rate limiting and usage tracking
-
-### Debugger Features
-
-- Model management with team access
-- Component async context tracking
-- Debug mode detection via headers
-- Extended agent functionality
-
-### Embodiment Features
-
-- Session management with Redis
-- Static file serving
-- Multiple integration modules:
- - ChatGPT Plugin support
- - OpenAI API proxy
- - Alexa Skills integration
- - MCP (Model Context Protocol)
- - Postman collection generation
- - Swagger documentation
- - Form preview functionality
- - Agent chat interface
-
-## 🚀 Smart Routing System
-
-The Runtime Server features an intelligent routing system that automatically directs requests to the appropriate service (Agent Runner or Debugger) based on HTTP headers. This provides **predictable, header-based routing** that works consistently across all environments.
-
-### 🎯 Quick Start
-
-**Default Behavior (No Headers)**
-
-```bash
-curl http://localhost:5000/api/test
-# → Routes to Agent Runner (production-safe default)
+pnpm run build
```
-**Force Agent Runner**
-
-```bash
-curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5000/api/test
-# → Always routes to Agent Runner
-```
-
-**Force Debugger**
-
-```bash
-curl -H "X-FORCE-DEBUGGER: true" http://localhost:5000/api/test
-# → Always routes to Debugger
-```
-
-**Debug Session**
-
+**Development Server Issues**
```bash
-curl -H "X-DEBUG-RUN: true" http://localhost:5000/api/test
-# → Routes to Debugger for debugging
-```
-
-### 📋 Routing Priority (Highest to Lowest)
-
-| Priority | Header | Routes To | Use Case |
-| -------- | --------------------------- | ------------ | ------------------------- |
-| 1 | `X-FORCE-AGENT-RUNNER` | Agent Runner | Force production behavior |
-| 2 | `X-DEBUG-*`, `X-MONITOR-ID` | Debugger | Debug sessions |
-| 3 | `X-FORCE-DEBUGGER` | Debugger | Force debug behavior |
-| 4 | Deployed versions | Agent Runner | Production safety |
-| 5 | `X-ROUTING-MODE` | As specified | Explicit routing |
-| 6 | No headers | Agent Runner | Safe default |
-
-### 🔧 Available Headers
-
-#### Force Routing Headers
-
-```bash
-# Force Agent Runner (highest priority)
-X-FORCE-AGENT-RUNNER: true
-
-# Force Debugger
-X-FORCE-DEBUGGER: true
-```
-
-#### Debug Session Headers
-
-```bash
-# Any of these will route to Debugger
-X-DEBUG-RUN: true
-X-DEBUG-STOP: true
-X-DEBUG-READ: true
-X-DEBUG-INJ: true
-X-DEBUG-SKIP: true
-X-MONITOR-ID: your-monitor-id
-```
-
-#### Explicit Mode Headers
-
-```bash
-# Explicit routing mode
-X-ROUTING-MODE: debugger # Routes to Debugger
-X-ROUTING-MODE: agent-runner # Routes to Agent Runner
-```
-
-### 🧪 Testing Different Environments
-
-#### Test Agent Runner from Localhost
-
-```bash
-# Without header (old behavior might route to debugger)
-curl http://localhost:5000/api/test
-
-# With header (guaranteed agent runner)
-curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5000/api/test
-```
-
-#### Test Debugger from Production
-
-```bash
-# Force debugger even on production domains
-curl -H "X-FORCE-DEBUGGER: true" https://your-agent.com/api/test
-```
-
-#### Debug Sessions
-
-```bash
-# Start a debug session
-curl -H "X-DEBUG-RUN: true" http://localhost:5000/api/test
-
-# Monitor agent execution
-curl -H "X-MONITOR-ID: session-123" http://localhost:5000/api/test
-```
-
-### 🛠️ For Developers
-
-#### Example Scripts
-
-We provide ready-to-use test scripts:
-
-```bash
-# Test routing logic programmatically
-node examples/test-smart-routing.js
-
-# Test routing with actual HTTP requests (server must be running)
-./examples/test-routing-http.sh
-```
-
-#### Testing Routing Logic
-
-```typescript
-import { testRoutingDecision, RoutingHeaders } from "./core/smart-agent-router";
-
-const agent = { agentVersion: "development" };
-
-// Test different scenarios
-const scenarios = [
- { headers: RoutingHeaders.FORCE_AGENT_RUNNER, expected: "agent-runner" },
- { headers: RoutingHeaders.DEBUG_RUN, expected: "debugger" },
- { headers: RoutingHeaders.NONE, expected: "agent-runner" },
-];
-
-scenarios.forEach(({ headers, expected }) => {
- const result = testRoutingDecision(agent, headers);
- console.log(`Headers: ${JSON.stringify(headers)}`);
- console.log(`Routes to: ${result.useDebugger ? "debugger" : "agent-runner"}`);
- console.log(`Reason: ${result.reason}\n`);
-});
-```
-
-#### Available Test Headers
-
-```typescript
-import { RoutingHeaders } from "./core/smart-agent-router";
-
-// Pre-defined header combinations
-RoutingHeaders.FORCE_AGENT_RUNNER; // { "X-FORCE-AGENT-RUNNER": "true" }
-RoutingHeaders.FORCE_DEBUGGER; // { "X-FORCE-DEBUGGER": "true" }
-RoutingHeaders.DEBUG_RUN; // { "X-DEBUG-RUN": "true" }
-RoutingHeaders.DEBUG_STOP; // { "X-DEBUG-STOP": "true" }
-RoutingHeaders.MODE_DEBUGGER; // { "X-ROUTING-MODE": "debugger" }
-RoutingHeaders.MODE_AGENT_RUNNER; // { "X-ROUTING-MODE": "agent-runner" }
-RoutingHeaders.NONE; // {} (no headers)
+# Clean reinstall
+rm -rf node_modules
+pnpm install
```
-### 🔍 Troubleshooting
-
-#### Request Not Routing as Expected?
-
-1. **Check your headers**: Use browser dev tools or `curl -v` to verify headers
-2. **Check the logs**: The server logs routing decisions with reasons
-3. **Test with explicit headers**: Use `X-FORCE-*` headers to override behavior
+### Debug Mode
-#### Common Issues
-
-**Issue**: "Agent always routes to debugger from localhost"
+Enable debug logging by setting routing headers:
```bash
-# Solution: Use explicit header
-curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5000/api/test
-```
+# Enable debug mode
+curl -H "X-DEBUG-RUN: true" http://localhost:5053/api/test
-**Issue**: "Can't debug from production domain"
-
-```bash
-# Solution: Use debug header
-curl -H "X-FORCE-DEBUGGER: true" https://production.com/api/test
+# Monitor specific session
+curl -H "X-MONITOR-ID: session-123" http://localhost:5053/api/test
```
-**Issue**: "Routing is inconsistent"
-
-```bash
-# Solution: Use explicit routing mode
-curl -H "X-ROUTING-MODE: agent-runner" http://your-domain.com/api/test
-```
-
-### 📊 Monitoring
-
-The server logs all routing decisions:
-
-```
-Routing decision: AGENT-RUNNER - X-FORCE-AGENT-RUNNER header present
-Routing decision: DEBUGGER - Debug headers present
-Routing decision: AGENT-RUNNER - Default production-safe routing
-```
-
-### 🎯 Best Practices
-
-1. **Production**: No headers = safe default (Agent Runner)
-2. **Development**: Use explicit headers for predictable behavior
-3. **Testing**: Use `X-FORCE-*` headers to test specific services
-4. **Debugging**: Use `X-DEBUG-*` headers for debug sessions
-5. **CI/CD**: Use `X-ROUTING-MODE` for explicit test scenarios
-
-### 🔄 Migration from Domain-Based Routing
-
-If you were relying on domain-based routing:
-
-**Old Way** (unreliable):
-
-```bash
-# Localhost = debugger, production = agent-runner (unpredictable)
-curl http://localhost:5000/api/test
-```
-
-**New Way** (explicit):
-
-```bash
-# Explicit header-based routing (predictable)
-curl -H "X-FORCE-AGENT-RUNNER: true" http://localhost:5000/api/test
-curl -H "X-FORCE-DEBUGGER: true" http://production.com/api/test
-```
+## 📊 Monitoring & Logging
----
+The server provides comprehensive logging for:
+- Request routing decisions
+- Authentication events
+- Agent execution status
+- Error tracking and debugging
-## 📁 Examples Directory
+Logs are formatted for easy consumption and include routing reasons for troubleshooting.
-The `examples/` directory contains helpful scripts for testing and understanding the Smart Routing system:
+## 🔐 Security Features
-- **`test-smart-routing.js`** - Programmatic testing of routing logic
-- **`test-routing-http.sh`** - HTTP request testing with curl
-- Run these scripts to validate your routing configuration
+- **Rate Limiting**: Built-in rate limiting for API endpoints
+- **CORS Protection**: Configurable CORS policies
+- **Authentication**: OAuth 2.0 with LogTo integration
+- **Input Validation**: Request validation with Joi schemas
+- **Error Handling**: Secure error responses without sensitive data exposure
## 🤝 Contributing
-When contributing to the Smart Routing system:
+When contributing to the runtime server:
-1. **Test all scenarios** using the provided example scripts
-2. **Update documentation** if adding new headers or routing logic
-3. **Maintain backward compatibility** with existing debug headers
-4. **Follow the priority system** when adding new routing rules
+1. **Follow TypeScript best practices**
+2. **Add tests for new functionality**
+3. **Update documentation for API changes**
+4. **Maintain backward compatibility**
+5. **Test routing scenarios thoroughly**
## 📄 License
-This project is part of the SmythOS Runtime system.
+This project is part of the SmythOS ecosystem.
\ No newline at end of file