Skip to content

armgit / cua

Public Imported from GitHub
0

Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.

feat(docs): add S3 sync to Modal pipeline for K8s MCP server (#1430)

r33drichardse5098af18e41
diff --git a/docs/scripts/HANDOFF-modal-s3-sync.md b/docs/scripts/HANDOFF-modal-s3-sync.md
new file mode 100644
index 0000000000..b7ab10b12c
--- /dev/null
+++ b/docs/scripts/HANDOFF-modal-s3-sync.md
@@ -0,0 +1,130 @@
+# Handoff: Deploy & Test Modal S3 Sync
+
+## Context
+
+PR #1430 (`feat/modal-s3-sync` branch in `trycua/cua`) adds a `sync_to_s3()` function to `docs/scripts/modal_app.py`. This function uploads generated documentation and code search databases from Modal volumes to S3, bridging the gap between Modal (which generates the DBs on a schedule) and a K8s-hosted MCP server (which serves them).
+
+### Architecture
+
+```
+Modal (generates DBs daily) → sync_to_s3() → S3 → K8s CronJob (pulls from S3) → PVC → MCP Server
+```
+
+### What was built
+
+1. **`sync_to_s3()` function** in `docs/scripts/modal_app.py`:
+ - Uploads SQLite + LanceDB files from two Modal volumes to `s3://trycua-docs-mcp-data/`
+ - Uses **Modal OIDC federation** (no static AWS keys) — Modal auto-injects `MODAL_IDENTITY_TOKEN`, which is exchanged via STS `AssumeRoleWithWebIdentity` for temporary credentials
+ - IAM role: `arn:aws:iam::296062593712:role/modal-docs-mcp-write-role` (already provisioned via Terraform in `trycua/cloud` repo)
+
+2. **Integration into scheduled pipelines**:
+ - `scheduled_crawl()` (6 AM UTC) — calls `sync_to_s3()` after regenerating docs databases
+ - `scheduled_code_index()` (5 AM UTC) — calls `sync_to_s3()` after aggregating code databases
+
+3. **S3 bucket** (`trycua-docs-mcp-data`) already exists with versioning, encryption, and cross-region replication. Currently has `docs_db/` data from a prior manual upload.
+
+### What needs testing
+
+The code has NOT been deployed yet. The previous environment could not reach PyPI (SSL handshake failures), so `modal` CLI was never installed.
+
+---
+
+## Your Task
+
+### 1. Install Modal CLI
+
+```bash
+pip install modal
+```
+
+### 2. Authenticate Modal
+
+The Modal token should be available in `~/.modal/config.toml`. If not, ask the user to provide it or run:
+
+```bash
+modal token set --token-id <ask-user> --token-secret <ask-user>
+```
+
+### 3. Checkout the branch
+
+```bash
+cd /path/to/cua # the trycua/cua repo
+git fetch origin
+git checkout feat/modal-s3-sync
+```
+
+### 4. Deploy the Modal app
+
+```bash
+modal deploy docs/scripts/modal_app.py
+```
+
+This should succeed. The app name is `cua-docs-mcp` — it will register all functions including the new `sync_to_s3`.
+
+### 5. Test `sync_to_s3` manually
+
+```bash
+modal run docs/scripts/modal_app.py::sync_to_s3
+```
+
+**Expected output:**
+- "Syncing databases to s3://trycua-docs-mcp-data/ ..."
+- Upload lines for docs SQLite, docs LanceDB directory, code SQLite, and code LanceDB directory
+- "S3 sync complete: N files uploaded to s3://trycua-docs-mcp-data/"
+
+**If OIDC auth fails** with an STS error about the role trust policy, check:
+- The Modal app name must be `cua-docs-mcp` (defined at top of `modal_app.py`)
+- The role trust policy restricts to `app_name:cua-docs-mcp` — if the app name changed, the OIDC claim won't match
+- Verify the role exists: `aws iam get-role --role-name modal-docs-mcp-write-role`
+
+**If volumes are empty** (no databases to upload), you'll need to generate them first:
+```bash
+# Generate docs databases (takes ~30 min)
+modal run docs/scripts/modal_app.py --skip-code
+
+# Generate code databases (takes ~1-2 hours)
+modal run docs/scripts/modal_app.py --skip-docs
+```
+
+### 6. Verify S3 contents
+
+```bash
+aws s3 ls s3://trycua-docs-mcp-data/ --recursive --region us-west-2
+```
+
+**Expected structure:**
+```
+docs_db/docs.sqlite
+docs_db/docs.lance/...
+code_db/code_index.sqlite
+code_db/code_index.lancedb/...
+```
+
+### 7. Report results
+
+Let the user know:
+- Whether `modal deploy` succeeded
+- Whether `sync_to_s3` ran and how many files were uploaded
+- The S3 listing output
+- Any errors encountered
+
+---
+
+## Key Files
+
+| File | Purpose |
+|------|---------|
+| `docs/scripts/modal_app.py` | Modal app — crawling, indexing, S3 sync, MCP server |
+| (cloud repo) `terraform/aws/docs-mcp-storage/main.tf` | S3 bucket, OIDC role, readonly IAM user |
+| (cloud repo) `terraform/aws/oidc/modal-oidc.tf` | Modal OIDC provider |
+
+## AWS Details
+
+| Resource | Value |
+|----------|-------|
+| S3 Bucket | `trycua-docs-mcp-data` |
+| Region | `us-west-2` |
+| Write Role ARN | `arn:aws:iam::296062593712:role/modal-docs-mcp-write-role` |
+| Readonly Creds Secret | `docs-mcp/readonly-credentials` (in AWS Secrets Manager) |
+| Modal App Name | `cua-docs-mcp` |
+| Modal Workspace ID | `ac-3LfmQEOVnLXl4ns0YBxNuA` |
diff --git a/docs/scripts/modal_app.py b/docs/scripts/modal_app.py
index cb3499d18d..cb317bfccb 100644
--- a/docs/scripts/modal_app.py
+++ b/docs/scripts/modal_app.py
@@ -29,6 +29,12 @@ code_volume = modal.Volume.from_name("cua-code-index", create_if_missing=True)
# GitHub token secret for cloning
github_secret = modal.Secret.from_name("github-secret", required_keys=["GITHUB_TOKEN"])
+# AWS IAM role ARN for OIDC-based S3 write access (no static keys needed)
+# Role created in cloud repo: terraform/aws/docs-mcp-storage/main.tf
+S3_WRITE_ROLE_ARN = "arn:aws:iam::296062593712:role/modal-docs-mcp-write-role"
+S3_BUCKET_NAME = "trycua-docs-mcp-data"
+S3_BUCKET_REGION = "us-west-2"
+
# Define the container image with all dependencies
image = (
modal.Image.debian_slim(python_version="3.12")
@@ -45,6 +51,7 @@ image = (
"pandas>=2.0.0",
"markdown-it-py>=3.0.0",
"markitdown>=0.0.1",
+ "boto3>=1.34.0",
)
.run_commands("playwright install --with-deps chromium")
)
@@ -320,6 +327,96 @@ async def crawl_docs():
return summary
+ image=image,
+ volumes={VOLUME_PATH: docs_volume, CODE_VOLUME_PATH: code_volume},
+ timeout=1800, # 30 minutes
+ cpu=1.0,
+ memory=2048,
+)
+def sync_to_s3(bucket: str = S3_BUCKET_NAME):
+ """Sync generated databases from Modal volumes to S3.
+
+ Uses Modal OIDC federation to assume an AWS IAM role for write access.
+ No static AWS credentials needed — the IAM role trust policy is scoped
+ to this Modal workspace and app (terraform/aws/docs-mcp-storage/main.tf).
+
+ Args:
+ bucket: S3 bucket name to upload to
+ """
+ import os
+
+ import boto3
+
+ print(f"Syncing databases to s3://{bucket}/ ...")
+
+ # Use Modal's OIDC token (auto-injected env var) to assume AWS IAM role
+ oidc_token = os.environ["MODAL_IDENTITY_TOKEN"]
+
+ sts = boto3.client("sts", region_name=S3_BUCKET_REGION)
+ creds = sts.assume_role_with_web_identity(
+ RoleArn=S3_WRITE_ROLE_ARN,
+ RoleSessionName="modal-docs-mcp-s3-sync",
+ WebIdentityToken=oidc_token,
+ DurationSeconds=3600,
+ )["Credentials"]
+
+ s3 = boto3.client(
+ "s3",
+ region_name=S3_BUCKET_REGION,
+ aws_access_key_id=creds["AccessKeyId"],
+ aws_secret_access_key=creds["SecretAccessKey"],
+ aws_session_token=creds["SessionToken"],
+ )
+
+ uploaded = 0
+
+ # --- docs databases ---
+ docs_db_dir = Path(DB_PATH)
+ if docs_db_dir.exists():
+ # SQLite
+ sqlite_path = docs_db_dir / "docs.sqlite"
+ if sqlite_path.exists():
+ key = "docs_db/docs.sqlite"
+ print(f" Uploading {sqlite_path} -> {key}")
+ s3.upload_file(str(sqlite_path), bucket, key)
+ uploaded += 1
+
+ # LanceDB directory
+ lance_dir = docs_db_dir / "docs.lance"
+ if lance_dir.exists():
+ for fpath in lance_dir.rglob("*"):
+ if fpath.is_file():
+ key = f"docs_db/docs.lance/{fpath.relative_to(lance_dir)}"
+ s3.upload_file(str(fpath), bucket, key)
+ uploaded += 1
+ print(f" Uploaded docs LanceDB directory")
+
+ # --- code databases ---
+ code_db_dir = Path(CODE_DB_PATH)
+ if code_db_dir.exists():
+ # Aggregated SQLite
+ code_sqlite = code_db_dir / "code_index.sqlite"
+ if code_sqlite.exists():
+ key = "code_db/code_index.sqlite"
+ print(f" Uploading {code_sqlite} -> {key}")
+ s3.upload_file(str(code_sqlite), bucket, key)
+ uploaded += 1
+
+ # Aggregated LanceDB directory
+ code_lance = code_db_dir / "code_index.lancedb"
+ if code_lance.exists():
+ for fpath in code_lance.rglob("*"):
+ if fpath.is_file():
+ key = f"code_db/code_index.lancedb/{fpath.relative_to(code_lance)}"
+ s3.upload_file(str(fpath), bucket, key)
+ uploaded += 1
+ print(f" Uploaded code LanceDB directory")
+
+ print(f"S3 sync complete: {uploaded} files uploaded to s3://{bucket}/")
+ return {"bucket": bucket, "files_uploaded": uploaded}
+
+
@app.function(
image=image,
volumes={VOLUME_PATH: docs_volume},
@@ -336,6 +433,11 @@ async def scheduled_crawl():
await generate_vector_db.remote.aio()
await generate_sqlite_db.remote.aio()
+ # Sync docs databases to S3
+ print("Syncing docs databases to S3...")
+ sync_result = sync_to_s3.remote()
+ print(f"S3 sync result: {sync_result}")
+
print(f"Scheduled crawl complete: {summary}")
return summary
@@ -1463,6 +1565,12 @@ async def scheduled_code_index():
print(f"Aggregation complete: {agg_result}")
result["aggregation"] = agg_result
+ # Sync code databases to S3
+ print("Syncing code databases to S3...")
+ sync_result = sync_to_s3.remote()
+ print(f"S3 sync result: {sync_result}")
+ result["s3_sync"] = sync_result
+
return result
except modal.exception.FunctionTimeoutError as e:
print(f"Code indexing timed out: {e}")