Skip to content

Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.

6.7 KB
Raw Blame
/**
 * Computer tables: agent computers, sessions, computer-use audit.
 *
 * Split by owner so two people can add tables all day without touching the same lines. Add tables
 * here; never edit core.ts or coworker.ts to do it.
 */
import {
  index,
  integer,
  pgTable,
  primaryKey,
  text,
  timestamp,
} from "drizzle-orm/pg-core";
import type { RuleLabel } from "../../computer/policy";
import { jsonb } from "./json";

/**
 * The boundary this deployment is enforcing, kept where a restart cannot lose it.
 *
 * This table keeps policy across restarts. The policy can be changed while running, and a restart
 * must not silently return to the default.
 *
 * One row, by construction. A deployment has one boundary, so the primary key is a constant and every
 * write is an upsert onto it. A table that can hold two policies is a table that will eventually hold
 * two and have to decide between them, and "which of these is in force" is not a question this should
 * ever be able to ask.
 *
 * Memory is still the cache. The gateway asks for the policy on every action, so it reads from
 * memory; this is the record that survives a restart, not something on the path of a click.
 */
export const actionPolicy = pgTable("action_policy", {
  /** Always `current`. See the note above on there being exactly one. */
  id: text("id").primaryKey(),
  /** `enforce` or `dry-run`. Not an enum: the policy module owns that vocabulary. */
  mode: text("mode").notNull(),
  deny: text("deny").array().notNull(),
  allow: text("allow").array().notNull(),
  /** Each rule's name and description, keyed by its text. Display only; see `ActionPolicy.labels`. */
  labels: jsonb("labels")
    .$type<Record<string, RuleLabel>>()
    .notNull()
    .default({}),
  /** Who last changed it, for the Admin page and the trail. */
  updatedBy: text("updated_by"),
  updatedAt: timestamp("updated_at", { withTimezone: true })
    .notNull()
    .defaultNow(),
});

/**
 * The last snapshot a computer produced, kept where every replica can read it.
 *
 * The gateway resolves the opaque ref in an acting call into the element it points at, and it must
 * resolve it against the snapshot the ref came from, never against a label the caller supplied. That
 * mapping used to live in a `Map` inside one process. OpenBot runs several processes behind a load
 * balancer, and the process that took the snapshot is rarely the one that handles the click that
 * follows it, so the mapping was absent exactly when a click arrived on another replica: the policy
 * then decided with no element in front of it and the audit row could not name what was touched. The
 * boundary the whole gateway exists for stopped applying, and nothing said so.
 *
 * One row per computer, upserted on every snapshot: the newest one wins, because a ref is only ever
 * resolved against the snapshot that is current. `snapshot_id` is the generation the far-side
 * computer stamped on it, so a ref carrying an older generation does not resolve to whatever now
 * holds that ref — the staleness a persisted cache is rightly warned about is answered by matching
 * the generation, not by keeping the cache in memory.
 *
 * Kept small on purpose: only the interactive elements a policy can match on, keyed by ref. It is a
 * resolution table for the boundary, not a history of pages, and the next snapshot replaces it.
 */
export const computerSnapshot = pgTable("computer_snapshot", {
  /** The computer these refs belong to. One live snapshot each, so it is the key. */
  computerId: text("computer_id").primaryKey(),
  /** The generation the computer stamped on this snapshot. A ref names the one it came from. */
  snapshotId: integer("snapshot_id").notNull(),
  /** The page it was taken on, so a rule about the host still has a page to match after a handover. */
  url: text("url").notNull(),
  /** The interactive elements, keyed by ref. The one thing resolve looks a ref up in. */
  elements: jsonb("elements").notNull(),
  takenAt: timestamp("taken_at", { withTimezone: true }).notNull().defaultNow(),
  /**
   * Which run of the computer took it.
   *
   * The generation only tells snapshots apart within one session: a replaced container counts from
   * one again, so a ref from a dead session matches a row the new one has not overwritten yet, and
   * the policy decides against an element from a page that no longer exists. Reset clears the row
   * for exactly that reason, but reset is not the only way a computer is replaced — an image change
   * does it too, and the server is never told. This is the session, so a stale row is recognisable
   * without anybody having to remember to delete it.
   *
   * Null where the provider cannot say, which is a deployment with one shared computer and no
   * supervisor. There the behaviour is what it was before this column existed.
   */
  session: text("session"),
});

/**
 * What a Bot's screen looked like on the turn that opened it.
 *
 * A conversation is a record, and a record must not change its mind. The transcript used to fetch
 * the live screen for every past turn, so an answer about one page sat under a picture of whichever
 * page the Bot had open by the time somebody read it back.
 *
 * KEYED ON THE TURN, which is the identity of the thing being remembered. Keying on the page instead
 * was a mistake with a plausible reason: two visits to one address collided, and letting the newer
 * win made a past turn's picture change under it, which is the exact mutability this table exists to
 * remove. A turn happens once and is then over for good, so the row is written once and never
 * updated.
 *
 * The computer is in the key as well as the turn, so a caller who may reach one Bot cannot read
 * another Bot's screen by naming a tool call.
 */
export const computerPageFrame = pgTable(
  "computer_page_frame",
  {
    /** Whose computer it was. */
    computerId: text("computer_id").notNull(),
    /** The turn that opened it. */
    toolCallId: text("tool_call_id").notNull(),
    /** The page, as the browser reported it after the navigation settled. */
    url: text("url").notNull(),
    title: text("title"),
    /**
     * The frame itself, base64 PNG.
     *
     * Bounded by the code that writes it rather than by the column, because the useful limit is "a
     * screenshot" and the honest failure is a refusal at the boundary rather than a database error.
     */
    frame: text("frame").notNull(),
    capturedAt: timestamp("captured_at", { withTimezone: true })
      .notNull()
      .defaultNow(),
  },
  (table) => [
    primaryKey({ columns: [table.computerId, table.toolCallId] }),
    // The reaper's query: everything older than the retention window, whoever it belongs to.
    index("computer_page_frame_captured_idx").on(table.capturedAt),
  ],
);