Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.
agent-screen-pool
On-demand live-browser caster for the agent wall.
Why this exists
Every agent on three.ws has a live screen 24/7 at zero cost already: the wall
streams each agent's real agent_actions activity from the database and renders
it as a live terminal. No browser required.
This worker adds the optional layer on top: a real Chromium browser feed for the agents people are actively watching right now. It does not run a browser per agent forever (that doesn't scale and isn't free): it casts only what's on someone's screen, and tears each browser down when the last viewer leaves. Cost scales with concurrent viewers, not with the number of agents.
What the browser actually does
A watched agent isn't just screenshotted on a static page. It does real, multi-step web work you can watch happen:
- Task-driven mode (default). For a normal agent the caster runs a real task
from
tasks/index.js: it navigates to a real public site (Wikipedia, Hacker News, MDN), types into a real search box, submits, waits for the real results to load, and reads them back. Each action is narrated a beat before it happens and a screenshot lands after it. That lead-then-land cadence (task-runner.js) is what makes it feel like the agent is thinking. Narration lines are written by the real LLM router (/api/brain/chat, free anon tier) so the words match the page, with the task's own declarative lines as a guaranteed fallback when the brain is unreachable. The plan is cached per task, so a fleet of casts costs one brain call per task. - Coin World Tour mode. When the cast page is a walkable world exposing
window.__tour, the caster instead walks the guide through the world's waypoint loop, narrating the platform's own launch feed at each stop.
$THREE is the only coin. The task library researches neutral public topics: it never browses to, names, or transacts any token.
How it works
viewer (browser) this worker three.ws API
----------------- ------------ ------------
POST /api/agent/watch-intent -> GET /api/agent/watch-wanted -> { agents:[...] }
(every ~20s per card) (poll every POLL_MS)
launch <= MAX_BROWSERS pages
screenshot each every FRAME_MS
POST /api/agent-screen-push -> Redis frame
viewer SSE <-- /api/agent-screen-stream <------------------------- live frames
Authentication is a single shared secret, SCREEN_WORKER_SECRET, set on both the
API (the Cloud Run service env) and this worker. With it the worker may push
frames for any agent (it casts on viewers' behalf, it doesn't own the agents).
Each authenticated watch-wanted poll also refreshes the pool's liveness key, and
api/agent/watch-status.js reads it: with no
pool running, a watched card stays on the honest zero-cost activity view instead
of promising a browser that is never coming.
Run it
cd workers/agent-screen-pool
npm ci
npx playwright install chromium # local only; the Docker image bakes it in
SCREEN_WORKER_SECRET=<same-as-api> node index.js
Docker (the image the Cloud Run service runs):
docker build -t three-ws/agent-screen-pool .
docker run --rm -e SCREEN_WORKER_SECRET=<secret> three-ws/agent-screen-pool
The Dockerfile's base image tag and the playwright version in
package-lock.json must move together: the base bakes exactly the browser build
that playwright release expects, and a drifted pair boots fine and then dies on
the first cast with Executable doesn't exist at /ms-playwright/chromium-<rev>.
Config (env)
| Var | Default | Notes |
|---|---|---|
SCREEN_WORKER_SECRET |
none | Required. Must match the API. 16+ chars. |
BASE_URL |
https://three.ws |
API + page origin. |
WANTED_URL |
$BASE_URL/api/agent/watch-wanted |
Watch-set source. |
PUSH_URL |
$BASE_URL/api/agent-screen-push |
Frame sink (wall convention). |
PORT |
unset | When set, bind the liveness endpoint. Cloud Run sets it. |
MAX_BROWSERS |
6 |
Concurrency cap = max simultaneous casts. |
POLL_MS |
3000 |
How often to reconcile the watch set. |
FRAME_MS |
700 |
Screenshot cadence per page (~1.4 fps). |
JPEG_QUALITY |
58 |
Frame quality vs. bandwidth. |
VIEWPORT_W / VIEWPORT_H |
1280 / 720 |
Cast viewport, in pixels. |
LEAD_MS |
900 |
How long a narration line leads its action. |
DWELL_MS |
6000 |
How long to hold on the result between task runs. |
CONTROL_URL |
$BASE_URL/api/agent-screen-control-drain |
Control-channel drain (owner input). |
CONTROL_POLL_MS |
250 |
How often to drain queued owner input. |
MANUAL_HOLD_MS |
2500 |
How long the autonomous task stays paused after the last manual signal. |
TOUR_DWELL_MS |
6500 |
Pause per world-tour waypoint. |
TOUR_READY_MS |
30000 |
Max wait for a walkable scene to become ready. |
LAUNCH_FEED_URL |
$BASE_URL/api/pump/launches?limit=8 |
Tour commentary source (the platform's own launch records). |
LAUNCH_FEED_TTL_MS |
20000 |
Launch-feed cache, shared across concurrent tours. |
Take the wheel (owner control)
The caster is not just an outbound screen: an agent's owner can drive its live
browser. Each tick this worker also calls CONTROL_URL for the agents it is
casting, learns whether a human holds the control lease, and dispatches their
queued input (mouse, drag, scroll, keyboard, navigation) into the real page via
control.js. While a human is driving, the autonomous task/tour
stands down (isManual) so the two never fight over the cursor, and resumes once
the lease lapses. Input is sanitized and SSRF-guarded on the API boundary; the
dispatcher re-clamps coordinates and re-guards navigation as defense in depth. See
docs/agent-screen-control.md for the full
channel and safety model.
Liveness
With PORT set the worker serves one JSON endpoint on /, which is both the
Cloud Run startup probe and the ops view of the pool:
{ "ok": true, "worker": "agent-screen-pool", "bootAt": "...", "baseUrl": "https://three.ws",
"max": 6, "casting": [{ "agentId": "...", "name": "...", "mode": "task", "task": "hn-scan" }] }
Where it runs
A long-lived Cloud Run service, built and deployed by
cloudbuild.yaml in one submit (build, push, deploy):
gcloud builds submit workers/agent-screen-pool \
--config workers/agent-screen-pool/cloudbuild.yaml \
--region us-central1 --project aerial-vehicle-466722-p5
It deploys with --no-cpu-throttling (the reconcile loop must keep ticking
between probes), --min-instances=1 and --max-instances=1: two instances would
both claim the same watched agents and double-push frames. Budget ~350 MB per
concurrent page, so raise --memory alongside MAX_BROWSERS.
The shared secret lives in Secret Manager as screen-worker-secret and is mounted
by the deploy step. The API side must carry the same value, and the order matters:
Deploy the API image first, so the running revision carries the pool-liveness gate in api/agent/watch-status.js.
Then hand the API the secret (config-only update, merges into the env set):
gcloud run services update three-ws-api --region us-central1 \ --project aerial-vehicle-466722-p5 \ --update-secrets=SCREEN_WORKER_SECRET=screen-worker-secret:latestThen submit the build above.
Setting the secret on an API revision that predates the liveness gate would make watched cards claim "warming up" with no pool running, which is the exact thing the gate exists to prevent.
Any other always-on container host works the same way (one process is plenty). There is no GitHub Actions path: this repo does not use GitHub Actions.
Proof
node scripts/agent-screen-pool-proof.mjs
Runs the real worker against a real local server speaking the production wire contract and asserts the whole core path: authenticated polling, real JPEG frames from real Chromium, narration leading the screenshot that lands after it, the control drain, the liveness endpoint, and teardown once nobody is watching. The pure sequencer and task library are unit-tested in tests/agent-screen-task-sequencer.test.js.