Skip to content

Self-improving AI Employees for everyone

22.4 KB
Raw Blame

GCP VM Deploy Runbook (released package via bun add -g, manual gcloud + cf)

Step-by-step runbook for deploying the released @zeenie-ai/opencompany package (from the npm registry, installed with bun add -g — no Node, no npm on the VM) on a fresh GCP VM behind a Cloudflare-proxied domain, with the single-owner login gate enabled. Written to be executable by an AI agent (or a human) with no other context.

This is the manual path — it does NOT use company deploy / Terraform (cli/commands/deploy/, cli/terraform/). It was derived from a real deployment to demo.opencompany.sh (June 2026, before the scoped-package cutover) and encodes every pitfall hit on the way. Re-validated July 2026 against 0.0.95 with an IP-only variant (no Cloudflare / domain — skip step 4): pitfalls 2 and 3 are fixed in >= 0.0.95 (annotated below), and pitfalls 11-13 were added from that run. Pitfall 14 affects 0.2.0 and 0.2.1.

The unscoped opencompany package belongs to a different publisher. Do not install or remove it while following this runbook.

Parameters

Decide these up front; they appear in commands below as <PLACEHOLDERS>.

Placeholder Example Notes
<PROJECT> zeenie GCP project id (gcloud config get-value project)
<VM_NAME> demo Instance name
<REGION> / <ZONE> asia-south1 / asia-south1-a Mumbai. Any region works
<IP_NAME> opencompany-india-ip Name for the reserved static IP
<DOMAIN> demo.opencompany.sh Must be a subdomain of a zone in the Cloudflare account
<SUBDOMAIN> demo The record name within the zone
<OWNER_EMAIL> [email protected] Login email for the owner account
<OWNER_PASSWORD> (generated) >= 8 chars; generate 20-char alphanumeric if not supplied
<JWT_KEY> <SECRET_KEY> <ENC_KEY> (generated) Three independent 48-hex secrets

Generate secrets (run locally):

python -c "import secrets,string; a=string.ascii_letters+string.digits; \
print('PW='+''.join(secrets.choice(a) for _ in range(20))); \
print('JWT='+secrets.token_hex(24)); print('SEC='+secrets.token_hex(24)); \
print('ENC='+secrets.token_hex(24))"

Prerequisites (verify before starting)

  1. gcloud installed and authenticated: gcloud config get-value project returns the project and gcloud config get-value account returns an account. On Windows, ignore the harmless Test-Path ... bundledpython ... is denied stderr noise — output after it is still valid.
  2. Only if doing step 4 (domain): cf (Cloudflare CLI, npm package) installed and authenticated: cf auth whoami shows "authenticated": true with dns_records:edit scope. IP-only deploys don't need cf at all.
  3. No Terraform, no ADC needed — plain gcloud credentials suffice.

Known pitfalls (read first — these each cost a redeploy or a debugging loop)

  1. Debian 12 fails. The package's provisioning step (scripts/install.js, run after bun add -g; it was the npm postinstall when this was first hit) hard-requires Python 3.12+; Debian 12 ships 3.11 and it exits 1 (ERROR: Python 3.12+ is required.). Use Ubuntu 24.04 (ubuntu-2404-lts-amd64 in ubuntu-os-cloud), which ships Python 3.12.
  2. company serve exists only in >= 0.0.95. Releases up to 0.0.88 shipped only start | dev | stop | build | clean | doctor | help | version; 0.0.95 added the single-port serve (API + WS + SPA, used by the Terraform path). This runbook runs company start behind an nginx reverse proxy on :80/:443. Up to 0.0.95, company start served the static client on :3000 and the backend on :3010. After 0.0.95 it is single-port itself (uvicorn serves the SPA) and the ports moved to the ones named in .env.template: PYTHON_BACKEND_PORT (the app), TEMPORAL_UI_PORT, TEMPORAL_FRONTEND_GRPC_PORT, NODEJS_EXECUTOR_PORT, WHATSAPP_RPC_PORT. Step 2 reads PYTHON_BACKEND_PORT from the installed package and points every nginx proxy_pass at it. On a 0.0.95-or-older release, use :3010 for /api/, /ws/, /webhook/ and /health, and :3000 for location /.
  3. Owner env seeding requires >= 0.0.95. 0.0.88 ignored MACHINA_OWNER_* entirely. Since 0.0.95 the backend seeds the owner at startup from OPENCOMPANY_OWNER_EMAIL + OPENCOMPANY_OWNER_PASSWORD (>= 8 chars; MACHINA_OWNER_* kept as legacy fallback) — verified present in the published package. You can add those two keys to the env file in step 2 instead of racing to register. The manual POST /api/auth/register in step 6 remains the fallback for older releases and still works as first-user-becomes-owner either way.
  4. Unknown env vars are fine, but don't set PORT=80. The backend requires PORT between 1024 and 65535 (Settings.port), so PORT=80 stops it at startup; nginx owns :80. Leave PORT at the template value: company start binds PYTHON_BACKEND_PORT while the backend dials itself on PORT, so the two must match. Keep the env file to the keys listed in step 2.
  5. Cloudflare proxied + "Full" SSL mode → 521 without origin TLS. If the zone forces HTTPS and SSL mode is Full, Cloudflare connects to origin :443. Fix: nginx listens on 443 with a self-signed cert (Full mode accepts it; only Full-strict would reject) and the GCP firewall must allow tcp:443 too.
  6. Startup script must have LF line endings. When authoring on Windows, strip CRLF before passing to --metadata-from-file or bash on the VM fails obscurely.
  7. GCP startup scripts re-run on every boot. Keep the script idempotent (it is, below). A stop/start re-provisions harmlessly.
  8. Renaming a VM requires stopping it (gcloud compute instances set-name), and an ephemeral IP would be released on recreate. Reserve the static IP first (step 3) so the DNS record never goes stale.
  9. PowerShell quoting eats | and \" in long ssh --command strings. Write a local .sh file, gcloud compute scp it, then ssh --command="sudo bash /tmp/x.sh".
  10. WebSocket 403 without a cookie is correct behavior, not a proxy bug. The backend rejects unauthenticated /ws/status upgrades; with a valid login cookie the same handshake returns 101 Switching Protocols.
  11. JWT_COOKIE_SECURE must match the transport or login silently bounces. With JWT_COOKIE_SECURE=true and the app accessed over plain http://<IP>, the login POST returns 200 + the cookie, but the browser drops the Secure cookie — the SPA loops straight back to the login page with valid credentials. Set true only behind Cloudflare/TLS (the step-4 path); for an IP-only deploy set JWT_COOKIE_SECURE=false in the env file (matches what the Terraform path's build_app_env does). Curl-based verification passes either way (curl ignores Secure), so test in a browser or check the set-cookie attributes explicitly.
  12. PowerShell splits unquoted comma lists into separate argv entries. --rules=tcp:80,tcp:443 reaches gcloud as [tcp:80 tcp:443] and fails with "Firewall rules must be of the form PROTOCOL[:PORT[-PORT]]". Quote the whole argument: "--rules=tcp:80,tcp:443" (same for --source-ranges, --target-tags).
  13. SSH to the VM: expect port-22 egress blocks, plink prompts, and root-only env. (a) If plain gcloud compute ssh times out, the local network is blocking outbound :22 — add --tunnel-through-iap (rides HTTPS; the default-allow-ssh rule already admits IAP's 35.235.240.0/20). (b) On Windows gcloud uses plink, which blocks non-interactively on first-connect host-key confirmation — prefix echo y | gcloud compute ssh .... (c) /etc/opencompany/opencompany.env is root-owned chmod 600: an unprivileged grep/cat on it fails and short-circuits any cmd1 && cmd2 chain (e.g. skipping a trailing systemctl restart). Use sudo on every command touching it, or chain with ; and verify state after.
  14. 0.2.0 and 0.2.1: company start stops with Project not built. Run "company build" first. (errors.md #25 and #26). The service then fails and restarts in a loop, and the package also lacks the JS executor sidecar bundle. #25 is fixed after 0.2.1; #26 is still open. On those releases, run sudo company build once after step 2 (it fixes both), then sudo systemctl restart opencompany.

Step 1 — Reserve a static IP

gcloud compute addresses create <IP_NAME> --region=<REGION> --project=<PROJECT>
gcloud compute addresses describe <IP_NAME> --region=<REGION> --project=<PROJECT> --format="value(address)"

Save the returned address as <STATIC_IP>.

(If the VM already exists with an ephemeral IP, promote it instead: gcloud compute addresses create <IP_NAME> --region=<REGION> --addresses=<CURRENT_IP>.)

Step 2 — Author the startup script (locally)

Write opencompany-startup.sh with the placeholders filled in. This is the complete, corrected, idempotent script (toolchain + release install + env + nginx with TLS + systemd):

#!/bin/bash
# OpenCompany VM startup script: released package via `company start`,
# nginx on :80/:443 (self-signed TLS for Cloudflare Full mode), login gate on.
# Toolchain block mirrors cli/terraform/gcp/startup.sh.tftpl (the bun channel):
# bun + uv, no Node, no npm.
set -euo pipefail
export DEBIAN_FRONTEND=noninteractive

echo "[opencompany] installing toolchain..."
apt-get update
apt-get install -y curl ca-certificates git unzip build-essential pkg-config libffi-dev libssl-dev nginx
# bun on a stable system path: the runtime for the `company` shim, the JS
# executor sidecar and the plugin CLIs, and the installer for the package.
export BUN_INSTALL=/usr/local
curl -fsSL https://bun.sh/install | bash
curl -LsSf https://astral.sh/uv/install.sh | env UV_INSTALL_DIR=/usr/local/bin sh

echo "[opencompany] installing OpenCompany released package..."
bun add -g @zeenie-ai/opencompany@latest
# bun runs no lifecycle scripts for a global package; provision the Python
# side now so the service's first start is warm. The shim lives in bun's
# global bin dir ($BUN_INSTALL/bin) and knows its own package root.
"$BUN_INSTALL/bin/company" provision
OPENCOMPANY_BIN=$(command -v company)
# The shim is a symlink into the package (bun keeps global packages wherever it
# likes); resolve it rather than spelling that directory.
OPENCOMPANY_PACKAGE_DIR="$(cd "$(dirname "$(readlink -f "$OPENCOMPANY_BIN")")/.." && pwd)"
test -f "$OPENCOMPANY_PACKAGE_DIR/package.json"

echo "[opencompany] writing login-gate env..."
mkdir -p /etc/opencompany
cat > /etc/opencompany/opencompany.env <<'OPENCOMPANY_ENV_EOF'
HOST=0.0.0.0
DATA_DIR=/var/lib/opencompany
WORKSPACE_BASE_DIR=workspaces
VITE_AUTH_ENABLED=true
DEPLOYMENT_MODE=cloud
AUTH_MODE=single
# true ONLY behind Cloudflare/TLS (step 4). IP-only deploy: false (pitfall 11).
JWT_COOKIE_SECURE=true
JWT_COOKIE_SAMESITE=lax
# true works on >= 0.0.95 (dev server ships in-package; the backend spawns it
# and connects on TEMPORAL_FRONTEND_GRPC_PORT — Temporal's stock 7233 on the
# 0.0.95-era releases this runbook was validated against, the serial-block
# value from that release's .env.template afterwards).
# false = in-process sequential execution.
TEMPORAL_ENABLED=false
REDIS_ENABLED=false
LOG_FORMAT=text
JWT_SECRET_KEY=<JWT_KEY>
SECRET_KEY=<SECRET_KEY>
API_KEY_ENCRYPTION_KEY=<ENC_KEY>
# >= 0.0.95 only (pitfall 3): seeds the owner at first boot, replacing step 6.
# OPENCOMPANY_OWNER_EMAIL=<OWNER_EMAIL>
# OPENCOMPANY_OWNER_PASSWORD=<OWNER_PASSWORD>
OPENCOMPANY_ENV_EOF
chmod 600 /etc/opencompany/opencompany.env
mkdir -p /var/lib/opencompany

echo "[opencompany] self-signed TLS cert (Cloudflare Full mode accepts it)..."
mkdir -p /etc/nginx/certs
if [ ! -f /etc/nginx/certs/origin.crt ]; then
  openssl req -x509 -nodes -newkey rsa:2048 -days 3650 \
    -keyout /etc/nginx/certs/origin.key -out /etc/nginx/certs/origin.crt \
    -subj "/CN=<DOMAIN>"
fi

echo "[opencompany] nginx reverse proxy (everything to the app port)..."
# company start serves API + WS + SPA on one port: PYTHON_BACKEND_PORT in the
# installed package's .env.template. The heredoc stays quoted so nginx's own
# $variables are not expanded; APP_PORT is filled in with sed below.
APP_PORT=$(grep -E '^PYTHON_BACKEND_PORT=' "$OPENCOMPANY_PACKAGE_DIR/.env.template" | cut -d= -f2)
test -n "$APP_PORT"
cat > /etc/nginx/sites-available/opencompany <<'NGINX_EOF'
server {
    listen 80 default_server;
    listen 443 ssl default_server;
    ssl_certificate /etc/nginx/certs/origin.crt;
    ssl_certificate_key /etc/nginx/certs/origin.key;
    server_name <DOMAIN> _;
    client_max_body_size 50m;

    location /api/ {
        proxy_pass http://127.0.0.1:APP_PORT;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
    }
    location /ws/ {
        proxy_pass http://127.0.0.1:APP_PORT;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
    }
    location /webhook/ {
        proxy_pass http://127.0.0.1:APP_PORT;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
    location = /health {
        proxy_pass http://127.0.0.1:APP_PORT;
    }
    location / {
        proxy_pass http://127.0.0.1:APP_PORT;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}
NGINX_EOF
sed -i "s/APP_PORT/$APP_PORT/g" /etc/nginx/sites-available/opencompany
rm -f /etc/nginx/sites-enabled/default
ln -sf /etc/nginx/sites-available/opencompany /etc/nginx/sites-enabled/opencompany
nginx -t
systemctl enable --now nginx
systemctl reload nginx

echo "[opencompany] installing systemd service (README usage: company start)..."
cat > /etc/systemd/system/opencompany.service <<SERVICE_EOF
[Unit]
Description=OpenCompany (released, company start)
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
EnvironmentFile=/etc/opencompany/opencompany.env
WorkingDirectory=$OPENCOMPANY_PACKAGE_DIR
ExecStart=$OPENCOMPANY_BIN start
Restart=on-failure
RestartSec=5
# Must outlast the CLI's backend shutdown allowance with Temporal on
# (backend_shutdown_grace_seconds in cli/_common.py), or systemd kills the
# backend before it records a clean shutdown.
TimeoutStopSec=150

[Install]
WantedBy=multi-user.target
SERVICE_EOF

systemctl daemon-reload
systemctl enable --now opencompany
systemctl restart opencompany
echo "[opencompany] done."

On Windows, force LF endings after writing:

$p = "$env:TEMP\opencompany-startup.sh"
[IO.File]::WriteAllText($p, [IO.File]::ReadAllText($p).Replace("`r`n","`n"))

Step 3 — Firewall + VM

gcloud compute firewall-rules create opencompany-allow-http \
  --project=<PROJECT> --direction=INGRESS --action=ALLOW \
  --rules=tcp:80,tcp:443 --source-ranges=0.0.0.0/0 --target-tags=opencompany
gcloud compute instances create <VM_NAME> \
  --project=<PROJECT> --zone=<ZONE> \
  --machine-type=e2-standard-2 \
  --image-family=ubuntu-2404-lts-amd64 --image-project=ubuntu-os-cloud \
  --boot-disk-size=30GB --boot-disk-type=pd-balanced \
  --tags=opencompany --address=<STATIC_IP> \
  --metadata-from-file=startup-script=opencompany-startup.sh

Notes: e2-standard-2 (2 vCPU / 8 GB) is comfortable; the old e2-micro is too small. Updating an existing firewall rule uses --rules= (not --allow=): gcloud compute firewall-rules update opencompany-allow-http --rules="tcp:80,tcp:443".

Step 4 — DNS via cf CLI (optional — skip for an IP-only deploy)

Skipping this step is fine: the app is fully reachable at http://<STATIC_IP> (and https://<STATIC_IP> with a browser cert warning). If you skip it, set JWT_COOKIE_SECURE=false in step 2 (pitfall 11) or browser login will bounce.

The cf CLI (1.0 and later) takes the account and zone from the environment — the old cf context set commands are gone — and records create takes a raw JSON --body (individual flags like --type/--name are NOT supported):

cf zones list --name <ZONE_DOMAIN>     # e.g. opencompany.sh; note the zone "id" and "account" id
export CLOUDFLARE_ACCOUNT_ID=<ACCOUNT_ID>   # needed when the login can see more than one account
export CLOUDFLARE_ZONE_ID=<ZONE_ID>
cf dns records list --name <SUBDOMAIN>.<ZONE_DOMAIN>   # check the name is free
cf dns records create --body '{"type":"A","name":"<SUBDOMAIN>","content":"<STATIC_IP>","proxied":true,"ttl":1,"comment":"OpenCompany VM <ZONE> (gcloud, opencompany@latest)"}'

Delete a record by id: cf dns records delete <RECORD_ID> (cf asks to confirm; in a script add --force, or cf prints "Aborted." and exits 0 without deleting).

proxied: true matches the zone's other app records; Cloudflare terminates public TLS and (in Full SSL mode) connects to the origin's self-signed :443.

Step 5 — Wait for provisioning

Provisioning takes ~5-8 minutes (apt + bun + bun add -g + the Python dep sync in install.js). Poll until healthy:

until curl -sf -m 5 http://<STATIC_IP>/health; do sleep 10; done

If it never comes up, debug in this order:

# startup script progress / errors:
gcloud compute instances get-serial-port-output <VM_NAME> --zone=<ZONE> | grep -E "opencompany|error"
# service state:
gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --command="systemctl is-active opencompany nginx; sudo journalctl -u opencompany -n 40 --no-pager"

Step 6 — Register the owner (this IS the login setup)

On >= 0.0.95 you can skip this step by uncommenting OPENCOMPANY_OWNER_EMAIL / OPENCOMPANY_OWNER_PASSWORD in the step-2 env file (pitfall 3) — the backend seeds the owner at first boot. Otherwise (or on older releases), create the owner via the API the moment health is green. First user in single mode becomes owner and closes registration:

curl -s -X POST http://<STATIC_IP>/api/auth/register \
  -H "Content-Type: application/json" \
  -d '{"email":"<OWNER_EMAIL>","password":"<OWNER_PASSWORD>","display_name":"Owner"}'
# expect: {"success":true,"user":{...,"is_owner":true}}

Do this promptly — until it runs, anyone reaching the IP could register as owner. (Optionally create the VM with --allow-cidr style source-range restriction on the firewall first, then widen after registering.)

Step 7 — Verify end-to-end

All of these must pass before declaring success:

# 1. health via domain (Cloudflare -> origin TLS):
curl -s https://<DOMAIN>/health                    # 200, {"status":"healthy",...}
# 2. SPA serves:
curl -s https://<DOMAIN>/ | grep -o "<title>[^<]*</title>"   # <title>OpenCompany</title>
# 3. auth gate on, registration closed:
curl -s https://<DOMAIN>/api/auth/status
# expect: {"auth_enabled":true,"auth_mode":"single","can_register":false,...}
# 4. login works (200 + HttpOnly cookie named by JWT_COOKIE_NAME — `opencompany_token`
#    today; `machina_token` on pre-rebrand releases):
curl -s -i -X POST https://<DOMAIN>/api/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"<OWNER_EMAIL>","password":"<OWNER_PASSWORD>"}' | head -12
# 5. WebSocket: 403 without cookie (correct), 101 with it:
cookie=$(curl -s -i -X POST https://<DOMAIN>/api/auth/login -H "Content-Type: application/json" \
  -d '{"email":"<OWNER_EMAIL>","password":"<OWNER_PASSWORD>"}' \
  | grep -i '^set-cookie' | sed 's/^[Ss]et-[Cc]ookie: //' | cut -d';' -f1)
curl -s -i -N -m 8 -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" -H "Sec-WebSocket-Version: 13" \
  -H "Cookie: $cookie" https://<DOMAIN>/ws/status | head -3   # HTTP/1.1 101

If https://<DOMAIN> returns 521: origin :443 is unreachable — confirm the firewall allows tcp:443 and nginx is listening (sudo ss -tlnp | grep :443). If 526: the zone is Full (strict); replace the self-signed cert with a Cloudflare Origin CA cert.

Step 8 — Hygiene

  • Delete any local temp scripts containing the secrets after the deploy.
  • Report the URL, owner email, and password to the operator once; the password is not recoverable from the VM (only its bcrypt hash is stored).

Teardown

gcloud compute instances delete <VM_NAME> --zone=<ZONE> --project=<PROJECT> --quiet
gcloud compute addresses delete <IP_NAME> --region=<REGION> --project=<PROJECT> --quiet
gcloud compute firewall-rules delete opencompany-allow-http --project=<PROJECT> --quiet
cf dns records list --zone <ZONE_ID>   # find the record id
cf dns records delete <RECORD_ID> --zone <ZONE_ID>

Operations cheat sheet

If plain ssh times out or plink prompts, use the pitfall-13 form: echo y | gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --tunnel-through-iap --command="...".

# logs:
gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --command="sudo journalctl -u opencompany -f"
# restart app:
gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --command="sudo systemctl restart opencompany"
# upgrade to a new release:
gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --command="sudo env BUN_INSTALL=/usr/local bun add -g @zeenie-ai/opencompany@latest && sudo env BUN_INSTALL=/usr/local /usr/local/bin/company provision && sudo systemctl restart opencompany"
# flip an env toggle (sudo on EVERY command touching the 600 env file - pitfall 13c):
gcloud compute ssh <VM_NAME> --zone=<ZONE> --quiet --command="sudo sed -i 's/TEMPORAL_ENABLED=false/TEMPORAL_ENABLED=true/' /etc/opencompany/opencompany.env; sudo systemctl restart opencompany"
# Temporal Web UI (localhost-bound on the VM; gRPC :7233 / UI :8080 are the
# release-pinned values for <= 0.0.95; later releases bind
# TEMPORAL_FRONTEND_GRPC_PORT / TEMPORAL_UI_PORT from that release's
# .env.template — substitute the UI port below accordingly):
gcloud compute ssh <VM_NAME> --zone=<ZONE> --tunnel-through-iap -- -L 8080:localhost:8080
#   then open http://localhost:8080. Note: the Temporal dev server is spawned by
#   the backend lifespan ONLY when TEMPORAL_ENABLED=true (`main.py` gates
#   `run_temporal_lifecycle` on `settings.temporal_enabled`); with the flag false
#   nothing listens on those ports and /health reports temporal.enabled=false.
# env (secrets live here):
#   /etc/opencompany/opencompany.env   (chmod 600; service reads it via EnvironmentFile)
# data:
#   /var/lib/opencompany           (DATA_DIR: DBs, workspaces, packages)