Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

10.2 KB
Raw Blame
/**
 * Git repository URL parsing — AUTHORITATIVE copy.
 *
 * Answers "is this URL a repository we can clone, and which host/owner/repo is
 * it?" for the hosts PortOS clones from: github.com and gitlab.com. Lives in
 * `lib/` (not `services/repoCloner.js`, which re-exports it) because the Brain
 * capture boxes have to preview the same answer BEFORE submitting: a capture
 * that is a bare repo URL gets cloned, and the client only offers the post-clone
 * agent options (malware scan / learn-from-repo) when it agrees with the server
 * about what counts as a repo.
 *
 * A pure leaf: `client/src/lib/repoUrl.js` re-exports it, so the Brain capture
 * boxes reveal the post-clone agent options for exactly the URLs the server will
 * clone. Import no Node built-in here, and nothing outside `server/lib`.
 */

// The host allowlist, and the two behaviors that actually differ between hosts.
// They live IN the table rather than as `host === 'github.com'` branches further
// down, so adding a host is one entry here and nothing else — and so each flag
// has to be decided deliberately for the new host rather than inherited from
// whichever existing host the branch happened to compare against.
//
//   provider           stable id stored on a link record (`repoHost` holds the
//                      hostname itself)
//   namespace.maxDepth how many path segments before the project may be the
//                      namespace. 1 = no subgroups (GitHub: anything past
//                      owner/repo is a deep link). GitLab nests subgroups, but
//                      the depth is CAPPED rather than unbounded: a bare GitLab
//                      URL carries no marker separating `group/sub/project` from
//                      `group/project/route/...`, so an unbounded walk reads a
//                      project asset URL as a deep namespace and manufactures
//                      directories INSIDE an existing clone. The cap also bounds
//                      the on-disk layout, which `repoCloner`'s staging sweep
//                      derives its recursion depth from.
//   namespace.allowDots whether a namespace segment may contain a dot. GitHub
//                      logins may not; GitLab group paths may. Keeping it off
//                      for the flat-clone host is also what makes a hostname
//                      collision impossible (`github.com/gitlab.com/x` cannot
//                      parse, so it cannot land on another host's clone root).
//   flatClonePath      LEGACY CARVE-OUT — clone to `<owner>/<repo>` with no
//                      hostname level, so clones made before PortOS supported a
//                      second host stay exactly where their link record says
//                      they are. Never set this for a newly added host.
export const REPO_HOSTS = Object.freeze({
  'github.com': { provider: 'github', namespace: { maxDepth: 1, allowDots: false }, flatClonePath: true },
  'gitlab.com': { provider: 'gitlab', namespace: { maxDepth: 3, allowDots: true }, flatClonePath: false },
});

/** The deepest `<host>/<namespace…>/<repo>` layout any host in the table produces. */
export const MAX_REPO_PATH_DEPTH = Object.values(REPO_HOSTS).reduce(
  (deepest, { namespace, flatClonePath }) => Math.max(deepest, (flatClonePath ? 0 : 1) + namespace.maxDepth + 1),
  0,
);

// The owner/repo pair is a PATH OPERAND, not just a label: the cloner clones
// into `join(reposDir, …owner, repo)` and the resulting `localPath` is later
// handed to an agent as the directory to scan/study. So every segment is
// matched against the character sets the hosts actually allow, NOT "anything
// but a slash" — the loose form parsed `https://github.com/../evil` as owner
// `..`, which resolves OUTSIDE the managed clone root.
//   owner: a login (or a GitLab group/subgroup). Both forms REQUIRE a leading
//          alphanumeric, which is what makes `.` and `..` unrepresentable; the
//          dotted form is gated per host (see `namespace.allowDots`).
//   repo:  alphanumerics plus `_`, `.`, `-` — a leading dot is legal (`.github`
//          is a real repository), so dot segments are rejected explicitly below.
const OWNER_RE = /^[A-Za-z0-9](?:[A-Za-z0-9-]*[A-Za-z0-9])?$/;
const DOTTED_OWNER_RE = /^[A-Za-z0-9][A-Za-z0-9_.-]*$/;
const REPO_RE = /^[A-Za-z0-9_.-]+$/;

// `REPO_RE`'s character class admits the dot segments `.` and `..`, which would
// escape (or collapse to) the clone root the same way a bad owner does. The
// owner pattern already rejects them by requiring a leading alphanumeric.
const DOT_SEGMENTS = new Set(['.', '..']);

// SSH remote: git@host:path (optionally `ssh://git@host/path`).
const SSH_RE = /^(?:ssh:\/\/)?git@([^:/\s]+)[:/](\S+)$/i;

// Any scheme (or none), optional userinfo: host[:port][/path]. Anchored at the
// host so `https://evil.com/github.com/o/r` is NOT read as a GitHub repo, and
// whitespace-free end-to-end so `github.com/a b/c` is rejected outright.
const HTTP_RE = /^(?:[A-Za-z][A-Za-z0-9+.-]*:\/\/)?(?:[^/@\s]+@)?([^/?#\s]+)(\/\S*)?$/;

// Path words that begin a host's own UI route rather than another namespace
// segment. GitLab's modern deep links use the `/-/` separator (handled
// separately), but its legacy links — and every GitHub deep link — put the
// route word directly after the project, so the segment walk stops here.
const RESERVED_PATH_SEGMENTS = new Set([
  'tree', 'blob', 'raw', 'commit', 'commits', 'compare', 'branches', 'tags',
  'releases', 'issues', 'pull', 'pulls', 'merge_requests', 'wiki', 'wikis',
  'actions', 'pipelines', 'settings', 'activity', 'network', 'graphs', 'blame',
  // GitLab serves these directly off the project path with no `/-/` marker, so
  // without them a badge or upload URL parses as a deeper namespace.
  'badges', 'uploads', 'archive', 'edit', 'forks', 'starrers', 'members',
  'artifacts', 'jobs', 'boards', 'milestones', 'labels', 'snippets',
  'environments', 'analytics', 'insights', 'hooks', 'container_registry',
]);

/**
 * Parse a repository URL into `{ host, provider, owner, repo }`, or null when
 * the URL isn't a repository on a supported host (or names a path-unsafe
 * owner/repo).
 *
 * `owner` is a single login on GitHub and may be a `group/subgroup` path on
 * GitLab; every one of its segments is validated, so it stays path-safe.
 *
 * @param {string} url
 * @returns {{ host: string, provider: string, owner: string, repo: string } | null}
 */
export function parseRepoUrl(url) {
  if (!url) return null;
  const normalized = String(url).trim();
  // A backslash defeats the host anchor: WHATWG maps `\` to `/` inside the
  // authority, so `https://evil.example.com\@github.com/o/r` resolves to
  // evil.example.com in a browser while the regex below reads github.com. The
  // clone would still go to the real github.com (repoCloneUrl rebuilds from the
  // parsed host), but the link would be STORED and rendered as a trusted repo
  // whose href points at the attacker.
  if (normalized.includes('\\')) return null;

  const match = normalized.match(SSH_RE) || normalized.match(HTTP_RE);
  if (!match) return null;

  const host = match[1].toLowerCase().replace(/^www\./, '').replace(/:\d+$/, '');
  const hostConfig = REPO_HOSTS[host];
  if (!hostConfig) return null;

  // Drop the query/hash, then GitLab's `/-/` deep-link separator. GitHub has a
  // fixed owner/repo pair, so identify those two operands before looking at any
  // later route words — a repository itself may be named `issues`, `settings`,
  // or `tree`, and arbitrary GitHub deep links are still valid repo URLs.
  let path = (match[2] || '').split(/[?#]/)[0].replace(/^\//, '');
  const dashIndex = path.indexOf('/-/');
  if (dashIndex !== -1) path = path.slice(0, dashIndex);

  const pathSegments = path.split('/').filter(Boolean);
  const { maxDepth, allowDots } = hostConfig.namespace;
  let segments = pathSegments;
  if (hostConfig.flatClonePath) {
    segments = pathSegments.slice(0, maxDepth + 1);
  } else {
    const routeIndex = pathSegments.findIndex((segment, index) =>
      index >= 2 && RESERVED_PATH_SEGMENTS.has(segment.toLowerCase()));
    if (routeIndex !== -1) segments = pathSegments.slice(0, routeIndex);
  }
  if (segments.length < 2) return null;

  // Past the cap the path is a deep link into the project, not a deeper
  // namespace — reading it as one would clone into (and create directories
  // inside) an existing checkout.
  if (segments.length > maxDepth + 1) return null;

  const ownerSegments = segments.slice(0, -1);
  const repo = segments[segments.length - 1].replace(/\.git$/i, '');

  const ownerPattern = allowDots ? DOTTED_OWNER_RE : OWNER_RE;
  if (!ownerSegments.every(segment => ownerPattern.test(segment))) return null;
  if (!repo || DOT_SEGMENTS.has(repo) || !REPO_RE.test(repo)) return null;

  return {
    host,
    provider: hostConfig.provider,
    owner: ownerSegments.join('/'),
    repo,
  };
}

/**
 * True when the URL points at a repository on a supported host.
 *
 * @param {string} url
 * @returns {boolean}
 */
export function isRepoUrl(url) {
  return parseRepoUrl(url) !== null;
}

/**
 * The `https` clone URL for a parsed repo.
 *
 * @param {{ host: string, owner: string, repo: string }} parsed
 * @returns {string}
 */
export function repoCloneUrl({ host, owner, repo }) {
  return `https://${host}/${owner}/${repo}.git`;
}

/**
 * The browsable web URL for a parsed repo — what to `href` when a record stores
 * the scp-style SSH remote a browser can't follow.
 *
 * @param {{ host: string, owner: string, repo: string }} parsed
 * @returns {string}
 */
export function repoBrowseUrl({ host, owner, repo }) {
  return `https://${host}/${owner}/${repo}`;
}

/**
 * Parse a URL only when it is a GitHub repository. The GitHub-only callers are
 * the ones whose downstream really is GitHub-specific (the Eidoverse worlds
 * repo, which is pushed to with a GitHub token), NOT the Brain's repo capture.
 *
 * @param {string} url
 * @returns {{ owner: string, repo: string, isGitHub: true } | null}
 */
export function parseGitHubUrl(url) {
  const parsed = parseRepoUrl(url);
  return parsed?.provider === 'github' ? { owner: parsed.owner, repo: parsed.repo, isGitHub: true } : null;
}

/**
 * True when the URL points at a GitHub repository specifically.
 *
 * @param {string} url
 * @returns {boolean}
 */
export function isGitHubRepoUrl(url) {
  return parseGitHubUrl(url) !== null;
}