Skip to content

Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.

15.5 KB
Raw Blame
import { checkNavigationTarget } from "../computer/target";

/**
 * Where an external agent lives, and whether we are willing to talk to it.
 *
 * Registering an agent hands us a URL that this server will POST to on every single run. That is a
 * server-side request to an address a user chose,
 * which is the textbook shape of SSRF: without a check, `http://169.254.169.254/` is a registrable
 * "agent" and the runtime will happily fetch cloud credentials on a user's behalf, from inside the
 * deployment, on a schedule.
 *
 * Ordering is the control. If the private-host escape hatch is consulted before the
 * deny-list, the cloud metadata endpoint is reachable under the configuration every developer
 * actually runs. Never-allowed hosts are refused first, ahead of any opt-in, which is why this reuses
 * `checkNavigationTarget` rather than a second checker that would have to get the ordering right
 * again.
 *
 * A developer's own agent legitimately lives at
 * `http://localhost:4200/ag-ui`, so on a laptop the private-host opt-in is ON, which is the case where
 * this check is weakest. That is the same trade navigation makes, and the reason the never-allowed
 * list is checked ahead of it. In a hosted deployment the opt-in is off and a private address is
 * refused.
 */

/**
 * Is this address one the deployment named, and refused only for being private?
 *
 * The second half is the load-bearing half. Re-running the check with the floor down separates
 * "refused because it is inside the network", which naming may overrule, from "refused because it is
 * the metadata address or is not a web address at all", which nothing may. Deciding that by reading
 * the refusal text would break the first time somebody rephrased it.
 *
 * Matching is exact, on the host as written, and a name with a port pins that port. No suffixes and
 * no patterns: a pattern that widened by accident is how host checks usually fail, and an operator
 * naming three addresses can name three addresses.
 */
function namedAsAllowed(
  raw: string,
  allowedHosts: ReadonlySet<string> | undefined,
): boolean {
  if (!allowedHosts || allowedHosts.size === 0) return false;
  if (!checkNavigationTarget(raw, { allowPrivateHosts: true }).allowed) {
    return false;
  }
  let url: URL;
  try {
    url = new URL(raw);
  } catch {
    return false;
  }
  // As the parser spells them, brackets included: `[::1]:8443` names an address and a port and
  // `[::1:8443]` names an address, and with the brackets gone the two read as one. The list is
  // stored in the same spelling (see `normalizeAllowedHost` in config.ts).
  const hostname = url.hostname.toLowerCase();
  const host = url.port ? `${hostname}:${url.port}` : hostname;
  return allowedHosts.has(host) || allowedHosts.has(hostname);
}

export type EndpointVerdict =
  | { allowed: true; url: string }
  | { allowed: false; reason: string };

/**
 * Check an agent endpoint before it is stored.
 *
 * Refused at registration rather than at run time: a URL that must never be fetched should
 * never reach the database, because everything downstream treats a stored agent as trustworthy.
 */
export function checkAgentEndpoint(
  raw: unknown,
  options: {
    allowPrivateHosts?: boolean;
    allowedHosts?: ReadonlySet<string>;
  } = {},
): EndpointVerdict {
  if (typeof raw !== "string" || !raw.trim()) {
    return { allowed: false, reason: "An agent needs a web address." };
  }

  const verdict = checkNavigationTarget(raw.trim(), options);
  if (!verdict.allowed && namedAsAllowed(raw.trim(), options.allowedHosts)) {
    /*
     * Named, one host at a time, by whoever runs this deployment.
     *
     * The private-host opt-in is a floor: it permits this deployment's whole network, to browsing
     * and to agent endpoints alike, which is why it is refused in production. But a company's own
     * agent legitimately lives at an internal address, and telling them to drop the floor to reach
     * it is the advice that made the opt-in dangerous in the first place. So an address may be named
     * instead, and nothing else is opened.
     *
     * Only ever reached for an address the strict check refused *for being private*. Anything on the
     * never-allowed list, and anything that is not http or https, is refused before this and cannot
     * be named back in — see `namedAsAllowed`, which re-runs the check with the floor down to find
     * out which kind of refusal it was rather than pattern-matching the message.
     *
     * Agent endpoints only. Browsing is not widened by this: a page can steer a Bot somewhere, and
     * an operator naming an address they run is a different act from a Bot following a link to it.
     */
    return { allowed: true, url: new URL(raw.trim()).toString() };
  }
  if (!verdict.allowed) {
    // The navigation wording talks about "the Bot opening" a page, which is not what is
    // happening here, so the reason is restated for the form surface.
    return {
      allowed: false,
      reason: verdict.reason.replace(
        /the Bot can never open it\.|the Bot can't open it\./,
        "an agent may not live there.",
      ),
    };
  }

  return { allowed: true, url: verdict.url };
}

/**
 * How many redirects an agent is allowedbefore we stop believing it has somewhere to be.
 *
 * Three, which covers the ordinary shapes (`http` to `https`, a host rename, a trailing-slash
 * canonicalisation) and stops a chain that has no end.
 */
const MAX_REDIRECTS = 3;

/** An address this deployment will not dial, named so the person registering sees which hop. */
export class EndpointNotAllowedError extends Error {
  constructor(message: string) {
    super(message);
    this.name = "EndpointNotAllowedError";
  }
}

/** How a refusal of the stored address begins, whoever ends up holding its message. */
const NOT_DIALLED = "This deployment will not dial ";

/**
 * Whether a run failed because its stored address is one this deployment will not dial.
 *
 * By message as well as by type, because the AG-UI client turns a fetch that throws into a RUN_ERROR
 * event carrying only the message, so the error a caller finally holds is a plain `Error`. Only the
 * stored address counts: a redirect can stop redirecting, and that address cannot change on its own.
 */
export function isEndpointRefusal(error: unknown): boolean {
  return (
    error instanceof Error &&
    (error.name === "EndpointNotAllowedError" ||
      error.message.startsWith(NOT_DIALLED))
  );
}

const REDIRECT_STATUSES = new Set([301, 302, 303, 307, 308]);

/**
 * The headers an AG-UI POST needs to be an AG-UI POST. Everything else on one of these requests is
 * the registered agent's own configuration, which is to say its key.
 */
const PROTOCOL_HEADERS = new Set(["content-type", "accept"]);

/**
 * Whether a hop stays inside the authorisation the request was carrying credentials for.
 *
 * Host and port must match, because a different one is a different party however similar the name.
 * A scheme upgrade is the exception in the permissive direction: `http` to `https` on the same host
 * is the ordinary shape of a deployment behind a redirect, and the credential ends up somewhere
 * strictly better protected than where it started. The downgrade is not the same trade and is
 * treated as a different party.
 */
function sameCredentialScope(from: string, to: string): boolean {
  const a = new URL(from);
  const b = new URL(to);
  if (a.hostname !== b.hostname || a.port !== b.port) return false;
  return (
    a.protocol === b.protocol ||
    (a.protocol === "http:" && b.protocol === "https:")
  );
}

/** The request with everything that proves who we are taken out of it. */
function withoutCredentials(init: RequestInit | undefined): RequestInit {
  const kept = new Headers();
  for (const [name, value] of new Headers(init?.headers)) {
    if (PROTOCOL_HEADERS.has(name.toLowerCase())) kept.set(name, value);
  }
  return { ...init, headers: kept, ...strippedBody(init?.body) };
}

/**
 * The body with this deployment's own signed run taken out of it.
 *
 * The run assertion is a bearer capability: it names the Bot and the person, and whatever holds it
 * can call back and spend that person's grants. Stripping the headers and forwarding the body would
 * leave the more valuable of the two credentials travelling.
 *
 * A body this cannot read is not forwarded at all. A stream or a form is not a shape this deployment
 * sends here, so the choice is between refusing an unreachable case and forwarding something
 * unexamined to a host the request was not authorised for, and only one of those fails safely.
 */
function strippedBody(body: BodyInit | null | undefined): { body?: BodyInit } {
  if (body === null || body === undefined) return {};
  if (typeof body !== "string") {
    throw new EndpointNotAllowedError(
      "That address redirected to another host, and this deployment will not forward the run to it.",
    );
  }

  let parsed: unknown;
  try {
    parsed = JSON.parse(body);
  } catch {
    // Not ours to sanitise and not ours to leak. The same reasoning as the non-string case.
    throw new EndpointNotAllowedError(
      "That address redirected to another host, and this deployment will not forward the run to it.",
    );
  }
  if (parsed === null || typeof parsed !== "object") return { body };

  const run = parsed as { forwardedProps?: Record<string, unknown> };
  if (!run.forwardedProps || typeof run.forwardedProps !== "object") {
    return { body };
  }
  const { openbotRun: _dropped, ...rest } = run.forwardedProps;
  return { body: JSON.stringify({ ...run, forwardedProps: rest }) };
}

/**
 * `fetch`, with the endpoint check applied to every hop rather than only the address a person typed.
 *
 * Checking the URL once and then handing it to a fetch that follows redirects is a check with a hole
 * in it: `https://agent.example.com/ag-ui` passes, answers `307`, and the request lands wherever the
 * `Location` header says, which is how a registrable agent becomes a way to read the deployment's own
 * cloud metadata. The address that gets dialled is the one that must be allowed, and a redirect makes
 * those two different addresses.
 *
 * The stored address is checked too, not only the hops after it. A row written before this guard
 * existed, or under a rule that has since changed, is dialled on every run, and that is the one
 * address a check that only reads `Location` headers never looks at.
 *
 * Redirects are followed rather than refused, because a deployment that puts its agent behind one has
 * done nothing wrong and `http` to `https` is the common case. Each destination goes through
 * {@link checkAgentEndpoint} first, so following one can only ever reach somewhere registering it
 * directly would have been allowed to reach.
 *
 * A hop that leaves the host the request was authorised for arrives with nothing that proves who we
 * are: the customer's key is theirs and was given to us for their host, and the run assertion is
 * this deployment's own capability. Once dropped they stay dropped, so a chain that wanders off and
 * comes back does not collect them again.
 *
 * The method and body are carried across every hop. A browser turns a redirected `POST` into a `GET`;
 * doing that here would only ever produce a confusing "that is not an AG-UI endpoint" from an agent
 * that is one, because AG-UI is a POST protocol and this is a server talking to an API, not a person
 * following a link.
 */
export function createAgentFetch(
  options: {
    allowPrivateHosts?: boolean;
    /**
     * Carried to every hop, not only the first. An address the deployment named is reachable
     * wherever it appears, and one it did not name is refused wherever it appears — a redirect must
     * not be a way to arrive somewhere registration would have declined.
     */
    allowedHosts?: ReadonlySet<string>;
    fetchImpl?: typeof fetch;
    /**
     * Told about every address this refused to dial, and why.
     *
     * A refusal is the one thing on this path an operator cannot otherwise learn. The person who
     * registered the agent finds out immediately, because their run fails and says why; the
     * deployment finds out nothing, and a stored agent that has quietly begun redirecting to the
     * metadata address is precisely the event worth being able to count.
     *
     * A callback rather than an audit store, so this module keeps deciding and nothing else. It is
     * the same reason `checkAgentEndpoint` reuses the navigation target check instead of growing a
     * second one: a file that decides is testable without the machinery that records.
     *
     * Reporting must never be able to stop a refusal, so a throwing reporter is swallowed. The
     * refusal is the security property and the row is the record of it; losing the record is bad and
     * turning it into a dialled request would be worse.
     */
    onRefusal?: (refusal: { address: string; reason: string }) => void;
  } = {},
): (url: string, init?: RequestInit) => Promise<Response> {
  const doFetch = options.fetchImpl ?? fetch;
  const refuse = (address: string, reason: string) => {
    try {
      options.onRefusal?.({ address, reason });
    } catch {
      // See above: a reporter that throws must not become a request that succeeds.
    }
    return new EndpointNotAllowedError(reason);
  };
  const check = (address: string) =>
    checkAgentEndpoint(address, {
      ...(options.allowPrivateHosts !== undefined
        ? { allowPrivateHosts: options.allowPrivateHosts }
        : {}),
      ...(options.allowedHosts ? { allowedHosts: options.allowedHosts } : {}),
    });

  return async function guardedFetch(url: string, init?: RequestInit) {
    const stored = check(url);
    if (!stored.allowed) {
      throw refuse(
        url,
        `${NOT_DIALLED}${url}: ${stored.reason.charAt(0).toLowerCase()}${stored.reason.slice(1)}`,
      );
    }

    const origin = stored.url;
    let target = stored.url;
    let carried = init;

    for (let hop = 0; hop <= MAX_REDIRECTS; hop += 1) {
      // `manual` is what makes this a check rather than a comment: the caller sees the redirect, and
      // the underlying fetch cannot quietly follow one on its own.
      const response = await doFetch(target, {
        ...carried,
        redirect: "manual",
      });
      if (!REDIRECT_STATUSES.has(response.status)) return response;

      const location = response.headers.get("location");
      // A redirect status with nowhere to go is just an answer. Whatever it means, it is the
      // agent's own reply and not a hop.
      if (!location) return response;

      const next = new URL(location, target).toString();
      const verdict = check(next);
      if (!verdict.allowed) {
        throw refuse(
          next,
          `That address redirected to ${next}, and ${verdict.reason.charAt(0).toLowerCase()}${verdict.reason.slice(1)}`,
        );
      }
      if (!sameCredentialScope(origin, verdict.url)) {
        // A body this cannot strip refuses the hop rather than forwarding it, and that refusal is
        // worth the same row as any other: it means a run was carrying something unreadable to a
        // host it was not authorised for.
        try {
          carried = withoutCredentials(carried);
        } catch (error) {
          throw refuse(
            verdict.url,
            error instanceof Error ? error.message : String(error),
          );
        }
      }
      target = verdict.url;
    }

    // Counted with the rest. An agent that loops is not a trust decision, but it is an endpoint
    // failing in a way only the trail can show is happening repeatedly.
    throw refuse(
      target,
      `That address redirected more than ${MAX_REDIRECTS} times without arriving anywhere.`,
    );
  };
}