π OpenHands: AI-Driven Development
SAAS: add local development helper scripts (#11459)
Rohit Malhotracd967ef4bce5
diff --git a/enterprise/enterprise_local/README.md b/enterprise/enterprise_local/README.mdnew file mode 100644index 0000000000..e756e6cb7e--- /dev/null+++ b/enterprise/enterprise_local/README.md@@ -0,0 +1,274 @@+# Instructions for developing SAAS locally++You have a few options here, which are expanded on below:++- A simple local development setup, with live reloading for both OSS and this repo+- A more complex setup that includes Redis+- An even more complex setup that includes GitHub events++## Prerequisites++Before starting, make sure you have the following tools installed:++### Required for all options:++- [gcloud CLI](https://cloud.google.com/sdk/docs/install) - For authentication and secrets management+- [sops](https://github.com/mozilla/sops) - For secrets decryption+ - macOS: `brew install sops`+ - Linux: `sudo apt-get install sops` or download from GitHub releases+ - Windows: Install via Chocolatey `choco install sops` or download from GitHub releases++### Additional requirements for enabling GitHub webhook events++- make+- Python development tools (build-essential, python3-dev)+- [ngrok](https://ngrok.com/download) - For creating tunnels to localhost++## Option 1: Simple local development++This option will allow you to modify the both the OSS code and the code in this repo,+and see the changes in real-time.++This option works best for most scenarios. The only thing it's missing is+the GitHub events webhook, which is not necessary for most development.++### 1. OpenHands location++The open source OpenHands repo should be cloned as a sibling directory,+in `../OpenHands`. This is hard-coded in the pyproject.toml (edit if necessary)++If you're doing this the first time, you may need to run++```+poetry update openhands-ai+```++### 2. Set up env++First run this to retrieve Github App secrets++```+gcloud auth application-default login+gcloud config set project global-432717+local/decrypt_env.sh+```++Now run this to generate a `.env` file, which will used to run SAAS locally++```+python -m pip install PyYAML+export LITE_LLM_API_KEY=<your LLM API key>+python enterprise_local/convert_to_env.py+```++You'll also need to set up the runtime image, so that the dev server doesn't try to rebuild it.++```+export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/all-hands-ai/runtime:main-nikolaik+docker pull $SANDBOX_RUNTIME_CONTAINER_IMAGE+```++By default the application will log in json, you can override.++```+export LOG_PLAIN_TEXT=1+```++### 3. Start the OpenHands frontend++Start the frontend like you normally would in the open source OpenHands repo.++### 4. Start the SaaS backend++```+make build++make start-backend+```++You should have a server running on `localhost:3000`, similar to the open source backend.+Oauth should work properly.++## Option 2: With Redis++Follow all the steps above, then setup redis:++```bash+docker run -p 6379:6379 --name openhands-redis -d redis+export REDIS_HOST=host.docker.internal # you may want this to be localhost+export REDIS_PORT=6379+```++## Option 3: Work with GitHub events++### 1. Setup env file++(see above)++### 2. Build OSS Openhands++Develop on [Openhands](https://github.com/All-Hands-AI/OpenHands) locally. When ready, run the following inside Openhands repo (not the Deploy repo)++```+docker build -f containers/app/Dockerfile -t openhands .+```++### 3. Build SAAS Openhands++Build the SAAS image locally inside Deploy repo. Note that `openhands` is the name of the image built in Step 2++```+docker build -t openhands-saas ./app/ --build-arg BASE="openhands"+```++### 4. Create a tunnel++Run in a separate terminal++```+ngrok http 3000+```++There will be a line++```+Forwarding https://bc71-2603-7000-5000-1575-e4a6-697b-589e-5801.ngrok-free.app+```++Remember this URL as it will be used in Step 5 and 6++### 5. Setup Staging Github App callback/webhook urls++Using the URL found in Step 4, add another callback URL (`https://bc71-2603-7000-5000-1575-e4a6-697b-589e-5801.ngrok-free.app/oauth/github/callback`)++### 6. Run++This is the last step! Run SAAS openhands locally using++```+docker run --env-file ./app/.env -p 3000:3000 openhands-saas+```++Note `--env-file` is what injects the `.env` file created in Step 1++Visit the tunnel domain found in Step 4 to run the app (`https://bc71-2603-7000-5000-1575-e4a6-697b-589e-5801.ngrok-free.app`)++### Local Debugging with VSCode++Local Development necessitates running a version of OpenHands that is as similar as possible to the version running in the SAAS Environment. Before running these steps, it is assumed you have a local development version of the OSS OpenHands project running.++#### Redis++A Local redis instance is required for clustered communication between server nodes. The standard docker instance will suffice.+`docker run -it -p 6379:6379 --name my-redis -d redis`++#### Postgres++A Local postgres instance is required. I used the official docker image:+`docker run -p 5432:5432 --name my-postgres -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres -e POSTGRES_DB=openhands -d postgres`+Run the alembic migrations:+`poetry run alembic upgrade head `++#### VSCode launch.json++The VSCode launch.json below sets up 2 servers to test clustering, running independently on localhost:3030 and localhost:3031. Running only the server on 3030 is usually sufficient unless tests of the clustered functionality are required. Secrets may be harvested directly from staging by connecting...+`kubectl exec --stdin --tty <POD_NAME> -n <NAMESPACE> -- /bin/bash`+And then invoking `printenv`. NOTE: _DO NOT DO THIS WITH PROD!!!_ (Hopefully by the time you read this, nobody will have access.)++```+{+ "configurations": [+ {+ "name": "Python Debugger: Python File",+ "type": "debugpy",+ "request": "launch",+ "program": "${file}"+ },+ {+ "name": "OpenHands Deploy",+ "type": "debugpy",+ "request": "launch",+ "module": "uvicorn",+ "args": [+ "saas_server:app",+ "--reload",+ "--host",+ "0.0.0.0",+ "--port",+ "3030"+ ],+ "env": {+ "DEBUG": "1",+ "FILE_STORE": "local",+ "REDIS_HOST": "localhost:6379",+ "OPENHANDS": "<YOUR LOCAL OSS OPENHANDS DIR>",+ "FRONTEND_DIRECTORY": "<YOUR LOCAL OSS OPENHANDS DIR>/frontend/build",+ "SANDBOX_RUNTIME_CONTAINER_IMAGE": "ghcr.io/all-hands-ai/runtime:main-nikolaik",+ "FILE_STORE_PATH": "<YOUR HOME DIRECTORY>>/.openhands-state",+ "OPENHANDS_CONFIG_CLS": "server.config.SaaSServerConfig",+ "GITHUB_APP_ID": "1062351",+ "GITHUB_APP_PRIVATE_KEY": "<GITHUB PRIVATE KEY>",+ "GITHUB_APP_CLIENT_ID": "Iv23lis7eUWDQHIq8US0",+ "GITHUB_APP_CLIENT_SECRET": "<GITHUB CLIENT SECRET>",+ "POSTHOG_CLIENT_KEY": "<POSTHOG CLIENT KEY>",+ "LITE_LLM_API_URL": "https://llm-proxy.staging.all-hands.dev",+ "LITE_LLM_TEAM_ID": "62ea39c4-8886-44f3-b7ce-07ed4fe42d2c",+ "LITE_LLM_API_KEY": "<LITE LLM API KEY>"+ },+ "justMyCode": false,+ "cwd": "${workspaceFolder}/app"+ },+ {+ "name": "OpenHands Deploy 2",+ "type": "debugpy",+ "request": "launch",+ "module": "uvicorn",+ "args": [+ "saas_server:app",+ "--reload",+ "--host",+ "0.0.0.0",+ "--port",+ "3031"+ ],+ "env": {+ "DEBUG": "1",+ "FILE_STORE": "local",+ "REDIS_HOST": "localhost:6379",+ "OPENHANDS": "<YOUR LOCAL OSS OPENHANDS DIR>",+ "FRONTEND_DIRECTORY": "<YOUR LOCAL OSS OPENHANDS DIR>/frontend/build",+ "SANDBOX_RUNTIME_CONTAINER_IMAGE": "ghcr.io/all-hands-ai/runtime:main-nikolaik",+ "FILE_STORE_PATH": "<YOUR HOME DIRECTORY>>/.openhands-state",+ "OPENHANDS_CONFIG_CLS": "server.config.SaaSServerConfig",+ "GITHUB_APP_ID": "1062351",+ "GITHUB_APP_PRIVATE_KEY": "<GITHUB PRIVATE KEY>",+ "GITHUB_APP_CLIENT_ID": "Iv23lis7eUWDQHIq8US0",+ "GITHUB_APP_CLIENT_SECRET": "<GITHUB CLIENT SECRET>",+ "POSTHOG_CLIENT_KEY": "<POSTHOG CLIENT KEY>",+ "LITE_LLM_API_URL": "https://llm-proxy.staging.all-hands.dev",+ "LITE_LLM_TEAM_ID": "62ea39c4-8886-44f3-b7ce-07ed4fe42d2c",+ "LITE_LLM_API_KEY": "<LITE LLM API KEY>"+ },+ "justMyCode": false,+ "cwd": "${workspaceFolder}/app"+ },+ {+ "name": "Unit Tests",+ "type": "debugpy",+ "request": "launch",+ "module": "pytest",+ "args": [+ "./tests/unit",+ //"./tests/unit/test_clustered_conversation_manager.py",+ "--durations=0"+ ],+ "env": {+ "DEBUG": "1"+ },+ "justMyCode": false,+ "cwd": "${workspaceFolder}/app"+ },+ // set working directory...+ ]+}+```diff --git a/enterprise/enterprise_local/convert_to_env.py b/enterprise/enterprise_local/convert_to_env.pynew file mode 100644index 0000000000..54e2c5d71b--- /dev/null+++ b/enterprise/enterprise_local/convert_to_env.py@@ -0,0 +1,127 @@+import base64+import os+import sys++import yaml+++def convert_yaml_to_env(yaml_file, target_parameters, output_env_file, prefix):+ """Converts a YAML file into .env file format for specified target parameters under 'stringData' and 'data'.++ :param yaml_file: Path to the YAML file.+ :param target_parameters: List of keys to extract from the YAML file.+ :param output_env_file: Path to the output .env file.+ :param prefix: Prefix for environment variables.+ """+ try:+ # Load the YAML file+ with open(yaml_file, 'r') as file:+ yaml_data = yaml.safe_load(file)++ # Extract sections+ string_data = yaml_data.get('stringData', None)+ data = yaml_data.get('data', None)++ if string_data:+ env_source = string_data+ process_base64 = False+ elif data:+ env_source = data+ process_base64 = True+ else:+ print(+ "Error: Neither 'stringData' nor 'data' section found in the YAML file."+ )+ return++ env_lines = []++ for param in target_parameters:+ if param in env_source:+ value = env_source[param]+ if process_base64:+ try:+ decoded_value = base64.b64decode(value).decode('utf-8')+ formatted_value = (+ decoded_value.replace('\n', '\\n')+ if '\n' in decoded_value+ else decoded_value+ )+ except Exception as decode_error:+ print(f"Error decoding base64 for '{param}': {decode_error}")+ continue+ else:+ formatted_value = (+ value.replace('\n', '\\n')+ if isinstance(value, str) and '\n' in value+ else value+ )++ new_key = prefix + param.upper().replace('-', '_')+ env_lines.append(f'{new_key}={formatted_value}')+ else:+ print(+ f"Warning: Parameter '{param}' not found in the selected section."+ )++ # Write to the .env file+ with open(output_env_file, 'a') as env_file:+ env_file.write('\n'.join(env_lines) + '\n')++ except Exception as e:+ print(f'Error: {e}')+++lite_llm_api_key = os.getenv('LITE_LLM_API_KEY')+if not lite_llm_api_key:+ print('Set the LITE_LLM_API_KEY environment variable to your API key')+ sys.exit(1)++yaml_file = 'github_decrypted.yaml'+target_parameters = ['client-id', 'client-secret', 'webhook-secret', 'private-key']+output_env_file = './enterprise/.env'++if os.path.exists(output_env_file):+ os.remove(output_env_file)+convert_yaml_to_env(yaml_file, target_parameters, output_env_file, 'GITHUB_APP_')+os.remove(yaml_file)++yaml_file = 'keycloak_realm_decrypted.yaml'+target_parameters = ['client-id', 'client-secret', 'provider-name', 'realm-name']+convert_yaml_to_env(yaml_file, target_parameters, output_env_file, 'KEYCLOAK_')+os.remove(yaml_file)++yaml_file = 'keycloak_admin_decrypted.yaml'+target_parameters = ['admin-password']+convert_yaml_to_env(yaml_file, target_parameters, output_env_file, 'KEYCLOAK_')+os.remove(yaml_file)++lines = []+lines.append('KEYCLOAK_SERVER_URL=https://auth.staging.all-hands.dev/')+lines.append('KEYCLOAK_SERVER_URL_EXT=https://auth.staging.all-hands.dev/')+lines.append('OPENHANDS_CONFIG_CLS=server.config.SaaSServerConfig')+lines.append(+ 'OPENHANDS_GITHUB_SERVICE_CLS=integrations.github.github_service.SaaSGitHubService'+)+lines.append(+ 'OPENHANDS_GITLAB_SERVICE_CLS=integrations.gitlab.gitlab_service.SaaSGitLabService'+)+lines.append(+ 'OPENHANDS_BITBUCKET_SERVICE_CLS=integrations.bitbucket.bitbucket_service.SaaSBitBucketService'+)+lines.append(+ 'OPENHANDS_CONVERSATION_VALIDATOR_CLS=storage.saas_conversation_validator.SaasConversationValidator'+)+lines.append('POSTHOG_CLIENT_KEY=test')+lines.append('ENABLE_PROACTIVE_CONVERSATION_STARTERS=true')+lines.append('MAX_CONCURRENT_CONVERSATIONS=10')+lines.append('LITE_LLM_API_URL=https://llm-proxy.eval.all-hands.dev')+lines.append('LITELLM_DEFAULT_MODEL=litellm_proxy/claude-sonnet-4-20250514')+lines.append(f'LITE_LLM_API_KEY={lite_llm_api_key}')+lines.append('LOCAL_DEPLOYMENT=true')+lines.append('DB_HOST=localhost')++with open(output_env_file, 'a') as env_file:+ env_file.write('\n'.join(lines))++print(f'.env file created at: {output_env_file}')diff --git a/enterprise/enterprise_local/decrypt_env.sh b/enterprise/enterprise_local/decrypt_env.shnew file mode 100644index 0000000000..e2988b4bd8--- /dev/null+++ b/enterprise/enterprise_local/decrypt_env.sh@@ -0,0 +1,27 @@+#!/bin/bash+set -euo pipefail++# Check if DEPLOY_DIR argument was provided+if [ $# -lt 1 ]; then+ echo "Usage: $0 <DEPLOY_DIR>"+ echo "Example: $0 /path/to/deploy"+ exit 1+fi++# Normalize path (remove trailing slash)+DEPLOY_DIR="${DEPLOY_DIR%/}"++# Function to decrypt and rename+decrypt_and_move() {+ local secret_path="$1"+ local output_name="$2"++ ${DEPLOY_DIR}/scripts/decrypt.sh "${DEPLOY_DIR}/${secret_path}"+ mv decrypted.yaml "${output_name}"+ echo "Moved decrypted.yaml to ${output_name}"+}++# Decrypt each secret file+decrypt_and_move "openhands/envs/feature/secrets/github-app.yaml" "github_decrypted.yaml"+decrypt_and_move "openhands/envs/staging/secrets/keycloak-realm.yaml" "keycloak_realm_decrypted.yaml"+decrypt_and_move "openhands/envs/staging/secrets/keycloak-admin.yaml" "keycloak_admin_decrypted.yaml"