The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.
TypeScript 3d-modelingagent-frameworkagent-harnessai-agentscadclaude-codecodexcoding-agentsdeveloper-toolsesp32game-developmentmcp-servermulti-agentopen-hardwareopencodeopenharnesspcbroboticsterminaltmux
feat(backend): the connector gateway, so Harness signs in to GitHub, Slack and Google without Grid
bubu-dudu76f8a2234a15
diff --git a/backend/README.md b/backend/README.mdindex 8c0aa8b98..2ab52ef0d 100644--- a/backend/README.md+++ b/backend/README.md@@ -97,6 +97,33 @@ Redis side set it to at least `64mb 16mb 60` and alert on `client_recent_max_outapp-proxy channels ride their own subscriber connection (`appSub` in `bus.ts`), so a disconnect therecannot take the chat / presence subscriptions down with it.+## Connector gateway++`routes/connectors.ts`, `lib/connectorGateway.ts`: sign-in to GitHub, Slack, Asana, HubSpot, Gmail,+Google Calendar, Google Drive and Figma for `harness connections` and the desktop app's Settings ▸+Connectors. These services let no computer register its own OAuth client, so the OAuth apps Autonomous+registered with them are used, with the same requests and answers as the Grid control plane+(`autonomous-grid-be` `grid_networks/connectors.py`). Every other service signs in from the computer.++To turn it on:++1. **The apps' config**, never in git: the Grid control plane's `config-connector-auth.json` shape+ (`{"connectors": {"github": {"auth_type": "app", "client_id", "client_secret", "auth_url",+ "token_url", "scopes", …}}}`), as a file (`CONNECTOR_AUTH_FILE`) or inline (`CONNECTOR_AUTH_JSON`,+ for a k8s secret). Only `auth_type: app` entries are used. Production's are the `app` entries of+ Grid prod's `/var/lib/grid-apis/config-connector-auth.json`; staging's, Grid dev's.+2. **`CONNECTOR_ENCRYPTION_KEY`**: 32 random bytes as base64 (`openssl rand -base64 32`), its own key.+ It seals the tokens kept per account (`ConnectorCredential`). Losing it means everyone connects again.+3. **`CONNECTOR_REDIRECT_URI`**: the page every one of those apps allows. Production keeps the default,+ `https://www.autonomous.ai/connector/callback`; staging,+ `https://staging.autonomousdev.xyz/connector/callback`.+4. **The index** for `ConnectorCredential` (`npx prisma db push` creates `connector_credentials` and its+ unique `userId + connector` index).+5. **The Autonomous web callback page** forwards a state that starts `harness_` to+ `POST /api/connectors/callback`, as it forwards `grid_` to Grid (see the PR that added this gateway).++Unset, the gateway lists no service and starts no sign-in; nothing else changes.+## PersistencePrisma over the **same MongoDB** as the agent-manager (no migration files — `prisma db push` atdiff --git a/backend/prisma/schema.prisma b/backend/prisma/schema.prismaindex 2c6db3c14..ade695387 100644--- a/backend/prisma/schema.prisma+++ b/backend/prisma/schema.prisma@@ -905,3 +905,21 @@ model HarnessLink {@@index([ownerId, machineId])@@map("harness_links")}++/// One account's sign-in to a service through the connector gateway (lib/connectorGateway.ts), as the+/// Grid control plane keeps it: the provider's tokens sealed with CONNECTOR_ENCRYPTION_KEY (AES-256-GCM),+/// so the gateway can renew them for any of the account's computers. Disconnecting deletes the row.+model ConnectorCredential {+ id String @id @default(uuid()) @map("_id")+ userId String+ connector String+ /// The token response, sealed: access and refresh token, type, scope, expiry.+ sealed String+ accountName String @default("")+ expiresAt Int @default(0)+ connectedAt DateTime @default(now())+ updatedAt DateTime @updatedAt++ @@unique([userId, connector])+ @@map("connector_credentials")+}diff --git a/backend/src/config/env.ts b/backend/src/config/env.tsindex 77bbf726d..57c0ad2f5 100644--- a/backend/src/config/env.ts+++ b/backend/src/config/env.ts@@ -89,6 +89,19 @@ const envSchema = z.object({// the backend authenticates the manager's dial-in socket (/api/manager-ws) against that apiKey and// picks a manager on create by capacity (machines.managerId → managers). See lib/managers.ts.+ // The connector gateway (lib/connectorGateway.ts): the OAuth apps Autonomous registered with GitHub,+ // Slack, Google and the others that let no computer register its own, in the Grid control plane's+ // config-connector-auth.json shape. A file path, or the JSON inline (a k8s secret). Neither → the+ // gateway answers that no service signs in through it. Never in git.+ CONNECTOR_AUTH_FILE: z.string().optional(),+ CONNECTOR_AUTH_JSON: z.string().optional(),+ // Seals each account's tokens kept for those services (ConnectorCredential): 32 random bytes as+ // base64 (`openssl rand -base64 32`), its own key. Unset → the gateway starts no sign-in.+ CONNECTOR_ENCRYPTION_KEY: z.string().optional(),+ // The redirect every one of those apps allows: the Autonomous web page, which forwards a `harness_`+ // state here (POST /api/connectors/callback). Staging: https://staging.autonomousdev.xyz/connector/callback.+ CONNECTOR_REDIRECT_URI: z.string().url().default('https://www.autonomous.ai/connector/callback'),+// Autonomous SSO (OAuth2 Authorization Code + PKCE). The backend exchanges the code, then hands// the SSO access token to the web. Control-plane REST + web WS validate that token through the// Autonomous profile API instead of minting a second backend-owned session JWT.diff --git a/backend/src/lib/connectorGateway.ts b/backend/src/lib/connectorGateway.tsnew file mode 100644index 000000000..8044a91ff--- /dev/null+++ b/backend/src/lib/connectorGateway.ts@@ -0,0 +1,333 @@+/**+ * The connector gateway: sign-in to the services whose MCP or API lets no computer register its own+ * OAuth client (GitHub, Slack, Asana, HubSpot, Google, Figma). Autonomous registered an OAuth app with+ * each; this server holds those apps' client secrets and does the parts that need them, the code+ * exchange and the refresh. Every other service signs in from the computer (cli/src/lib/connectors).+ *+ * computer → POST /api/connectors/start (signed in) → { authorize_url, pickup_code }+ * browser → the service's consent → https://www.autonomous.ai/connector/callback?code&state+ * web → POST /api/connectors/callback (no auth: the state is the ticket) → exchange here+ * computer → POST /api/connectors/poll (signed in) → the token, once+ * computer → POST /api/connectors/refresh (signed in) {connector} → a new token+ *+ * As the Grid control plane does it (autonomous-grid-be grid_networks/connectors.py), with the same+ * requests and answers: the account's tokens are kept here (ConnectorCredential, sealed with+ * CONNECTOR_ENCRYPTION_KEY), so any of its computers renews them by naming the service, and+ * disconnecting forgets them here. A sign-in in progress (its state, PKCE verifier and, once exchanged,+ * its one-time result for the computer) lives in Redis for ten minutes, under hashed keys.+ *+ * The apps' config is the Grid control plane's `config-connector-auth.json` shape, given as a file+ * (CONNECTOR_AUTH_FILE) or inline (CONNECTOR_AUTH_JSON): never in git.+ */+import { createCipheriv, createDecipheriv, createHash, randomBytes, timingSafeEqual } from 'crypto'+import { readFileSync } from 'fs'+import { pub } from './bus.js'+import { prisma } from './prisma.js'+import { env } from '../config/env.js'+import { logger } from '../utils/logger.js'++const TTL_SEC = 600+const POLL_INTERVAL_SEC = 2+const HTTP_TIMEOUT_MS = 15_000+/** Every state this gateway mints starts so; the Autonomous web callback forwards those here. */+export const STATE_PREFIX = 'harness_'++export class GatewayError extends Error {+ constructor(message: string, readonly code: string, readonly status = 400) { super(message) }+}++export interface ConnectorApp {+ code: string+ label: string+ description: string+ imageUrl: string+ clientId: string+ clientSecret: string+ authUrl: string+ tokenUrl: string+ refreshUrl: string+ userinfoUrl: string+ scopes: string[]+ /** `header`: client credentials as HTTP Basic; otherwise in the form body. */+ authStyle: string+ pkce: boolean+ refresh: boolean+ /** Provider-specific consent parameters (Google's access_type=offline). */+ authParams: Record<string, string>+ /** Where the access token is in the token response, dotted (Slack's authed_user.access_token). */+ tokenField: string+ mcpUrl: string+ /** `header:<Name>` sends the raw token under that name; anything else is `Authorization: Bearer`. */+ mcpAuthHeader: string+}++const str = (value: unknown): string => (typeof value === 'string' ? value : value == null ? '' : String(value))++/** The `app` entries of a config-connector-auth.json document; others sign in from the computer. */+export function parseConfig(raw: string): Record<string, ConnectorApp> {+ let document: unknown+ try { document = JSON.parse(raw) } catch { throw new Error('connector config is not valid JSON') }+ const entries = (document as { connectors?: unknown })?.connectors+ if (!entries || typeof entries !== 'object' || Array.isArray(entries)) throw new Error('connector config must have a "connectors" object')+ const apps: Record<string, ConnectorApp> = {}+ for (const [rawCode, value] of Object.entries(entries as Record<string, unknown>)) {+ if (!value || typeof value !== 'object') continue+ const entry = value as Record<string, unknown>+ const code = rawCode.trim().toLowerCase()+ if (!/^[a-z0-9][a-z0-9_-]{0,47}$/.test(code) || (str(entry.auth_type) || 'app').toLowerCase() !== 'app') continue+ if (!str(entry.client_id) || !str(entry.auth_url) || !str(entry.token_url)) continue+ const scopes = Array.isArray(entry.scopes) ? entry.scopes.map(str) : str(entry.scopes).split(/[\s,]+/)+ const extra = (entry.extra && typeof entry.extra === 'object' ? entry.extra : {}) as Record<string, unknown>+ apps[code] = {+ code, label: str(entry.label) || code, description: str(entry.description), imageUrl: str(entry.image_url),+ clientId: str(entry.client_id), clientSecret: str(entry.client_secret),+ authUrl: str(entry.auth_url), tokenUrl: str(entry.token_url), refreshUrl: str(entry.refresh_url), userinfoUrl: str(entry.userinfo_url),+ scopes: scopes.filter(Boolean), authStyle: str(entry.auth_style).toLowerCase(), pkce: entry.pkce === true, refresh: entry.refresh === true,+ authParams: Object.fromEntries(Object.entries((entry.auth_params ?? {}) as Record<string, unknown>).map(([k, v]) => [k, str(v)])),+ tokenField: str(entry.token_field), mcpUrl: str(entry.mcp_url) || str(extra.mcp_url), mcpAuthHeader: str(entry.mcp_auth_header) || str(extra.mcp_auth_header),+ }+ }+ return apps+}++let cached: { apps: Record<string, ConnectorApp> } | null = null++/** The configured apps; none (the gateway off) when no config is given. Read once per process. */+export function apps(): Record<string, ConnectorApp> {+ if (cached) return cached.apps+ let raw = env.CONNECTOR_AUTH_JSON+ if (!raw && env.CONNECTOR_AUTH_FILE) {+ try { raw = readFileSync(env.CONNECTOR_AUTH_FILE, 'utf8') } catch (error) {+ logger.error('connector gateway: cannot read CONNECTOR_AUTH_FILE', { error: String(error) })+ }+ }+ try { cached = { apps: raw ? parseConfig(raw) : {} } } catch (error) {+ logger.error('connector gateway: invalid connector config', { error: String(error) })+ cached = { apps: {} }+ }+ return cached.apps+}++/** For tests: forget the parsed config. */+export function resetConfig(): void { cached = null }++function app(code: unknown): ConnectorApp {+ const found = typeof code === 'string' ? apps()[code] : undefined+ if (!found) throw new GatewayError('This service does not sign in through Harness.', 'UNKNOWN_CONNECTOR', 404)+ return found+}++/** AES-256-GCM, version.iv.tag.ciphertext, as lib/machineCredential.ts seals, under its own key. */+function sealKey(): Buffer {+ const value = Buffer.from(env.CONNECTOR_ENCRYPTION_KEY ?? '', 'base64')+ if (value.length !== 32) throw new GatewayError('Connections through Harness are not set up on this server.', 'GATEWAY_NOT_CONFIGURED', 503)+ return value+}++export function seal(value: unknown): string {+ const iv = randomBytes(12)+ const cipher = createCipheriv('aes-256-gcm', sealKey(), iv)+ const body = Buffer.concat([cipher.update(JSON.stringify(value), 'utf8'), cipher.final()])+ return ['v1', iv.toString('base64url'), cipher.getAuthTag().toString('base64url'), body.toString('base64url')].join('.')+}++export function unseal(envelope: string): Record<string, unknown> {+ const [version, iv, tag, body, extra] = envelope.split('.')+ if (version !== 'v1' || !iv || !tag || !body || extra) throw new Error('Invalid connector credential')+ const decipher = createDecipheriv('aes-256-gcm', sealKey(), Buffer.from(iv, 'base64url'))+ decipher.setAuthTag(Buffer.from(tag, 'base64url'))+ return JSON.parse(Buffer.concat([decipher.update(Buffer.from(body, 'base64url')), decipher.final()]).toString('utf8'))+}++/** What the account's Connectors page lists for these services, with its own state. Never a secret. */+export async function list(userId: string): Promise<Record<string, unknown>[]> {+ const rows = await prisma.connectorCredential.findMany({ where: { userId }, select: { connector: true, accountName: true, expiresAt: true, connectedAt: true } })+ const mine = new Map(rows.map(row => [row.connector, row]))+ return Object.values(apps()).map(a => {+ const row = mine.get(a.code)+ return {+ code: a.code, label: a.label, description: a.description, image_url: a.imageUrl,+ auth_type: 'app', mcp_url: a.mcpUrl, refresh: a.refresh, scopes: a.scopes,+ status: row ? 'connected' : 'not_connected', account_name: row?.accountName ?? '',+ expires_at: row?.expiresAt ?? 0, connected_at: row ? Math.floor(row.connectedAt.getTime() / 1000) : 0,+ }+ })+}++async function keep(userId: string, connector: string, token: Record<string, unknown>): Promise<void> {+ const sealed = seal({ access_token: token.access_token, refresh_token: token.refresh_token, token_type: token.token_type, scope: token.scope, expires_at: token.expires_at })+ const fields = { sealed, accountName: str(token.account_name), expiresAt: Number(token.expires_at) || 0 }+ await prisma.connectorCredential.upsert({ where: { userId_connector: { userId, connector } }, create: { userId, connector, ...fields }, update: fields })+}++const sha = (value: string): string => createHash('sha256').update(value).digest('hex')+const stateKey = (state: string): string => `cgw:state:${sha(state)}`+const pickupKey = (pickup: string): string => `cgw:pickup:${sha(pickup)}`++interface FlowState { userId: string, connector: string, pickup: string, verifier: string }+interface PickupRecord { userId: string, connector: string, status: 'pending' | 'ready' | 'failed', error?: string, token?: Record<string, unknown> }++export async function start(userId: string, connector: unknown): Promise<Record<string, unknown>> {+ const config = app(connector)+ sealKey() // A sign-in this server could not keep is not started.+ const state = STATE_PREFIX + randomBytes(24).toString('base64url')+ const pickup = randomBytes(24).toString('base64url')+ const verifier = config.pkce ? randomBytes(48).toString('base64url') : ''+ const query = new URLSearchParams({ response_type: 'code', client_id: config.clientId, redirect_uri: env.CONNECTOR_REDIRECT_URI, state })+ if (config.scopes.length) query.set('scope', config.scopes.join(' '))+ for (const [key, value] of Object.entries(config.authParams)) query.set(key, value)+ if (verifier) {+ query.set('code_challenge', createHash('sha256').update(verifier).digest('base64url'))+ query.set('code_challenge_method', 'S256')+ }+ const flow: FlowState = { userId, connector: config.code, pickup, verifier }+ const record: PickupRecord = { userId, connector: config.code, status: 'pending' }+ await pub.multi().set(stateKey(state), JSON.stringify(flow), 'EX', TTL_SEC).set(pickupKey(pickup), JSON.stringify(record), 'EX', TTL_SEC).exec()+ return {+ connector: config.code, authorize_url: `${config.authUrl}${config.authUrl.includes('?') ? '&' : '?'}${query}`,+ pickup_code: pickup, poll_interval: POLL_INTERVAL_SEC, expires_in: TTL_SEC,+ }+}++function dig(payload: Record<string, unknown>, dotted: string): unknown {+ let node: unknown = payload+ for (const part of dotted.split('.')) node = node && typeof node === 'object' ? (node as Record<string, unknown>)[part] : undefined+ return node+}++async function tokenRequest(config: ConnectorApp, form: Record<string, string>, refresh: boolean): Promise<Record<string, unknown>> {+ const headers: Record<string, string> = { Accept: 'application/json', 'Content-Type': 'application/x-www-form-urlencoded' }+ const body = { ...form }+ if (config.authStyle === 'header') {+ headers.Authorization = 'Basic ' + Buffer.from(`${config.clientId}:${config.clientSecret}`).toString('base64')+ } else {+ body.client_id = config.clientId+ if (config.clientSecret) body.client_secret = config.clientSecret+ }+ let response: Response+ try {+ response = await fetch((refresh && config.refreshUrl) || config.tokenUrl, {+ method: 'POST', headers, body: new URLSearchParams(body).toString(), redirect: 'error', signal: AbortSignal.timeout(HTTP_TIMEOUT_MS),+ })+ } catch {+ throw new GatewayError('The service could not be reached. Try again.', 'PROVIDER_UNREACHABLE', 502)+ }+ const payload = await response.json().catch(() => null) as Record<string, unknown> | null+ // GitHub answers 200 with {"error": …}: the error key is checked whatever the status.+ const error = payload && typeof payload.error === 'string' ? payload.error : ''+ if (error === 'invalid_grant') throw new GatewayError('This account needs to be connected again.', 'INVALID_GRANT', 401)+ if (!response.ok || error || !payload) {+ logger.warn('connector gateway: token endpoint refused', { connector: config.code, status: response.status, error })+ throw new GatewayError('The service did not issue a token. Try again.', 'TOKEN_REFUSED', 502)+ }+ return payload+}++/** The token the computer stores (cli/src/lib/connectors/gateway.ts tokenFrom). */+function tokenFor(config: ConnectorApp, payload: Record<string, unknown>, previousRefresh = '', account = '', expiresAt = 0): Record<string, unknown> {+ const access = str(config.tokenField ? dig(payload, config.tokenField) : '') || str(payload.access_token)+ if (!access) throw new GatewayError('The service did not issue a token. Try again.', 'TOKEN_REFUSED', 502)+ const expiresIn = Number(payload.expires_in) || 0+ // Providers that do not rotate it (Google) leave the refresh token out of a refresh: keep the one held.+ const refreshToken = str(payload.refresh_token) || previousRefresh+ const token: Record<string, unknown> = {+ connector: config.code, access_token: access, refresh_token: refreshToken, token_type: str(payload.token_type) || 'Bearer',+ expires_at: expiresIn > 0 ? Math.floor(Date.now() / 1000) + expiresIn : expiresAt,+ scope: Array.isArray(payload.scope) ? payload.scope.join(' ') : str(payload.scope) || config.scopes.join(' '),+ account_name: account, refresh: config.refresh && Boolean(refreshToken),+ }+ if (config.mcpUrl) {+ const custom = config.mcpAuthHeader.startsWith('header:') ? config.mcpAuthHeader.slice('header:'.length).trim() : ''+ const headers = custom && custom.toLowerCase() !== 'authorization' ? { [custom]: access } : { Authorization: `Bearer ${access}` }+ token.mcp_entry = { url: config.mcpUrl, headers }+ }+ return token+}++/** Best effort: which account was connected, for the Connectors page. Never fails a sign-in. */+async function accountName(config: ConnectorApp, access: string): Promise<string> {+ if (!config.userinfoUrl) return ''+ try {+ const response = await fetch(config.userinfoUrl, { headers: { Authorization: `Bearer ${access}`, Accept: 'application/json' }, redirect: 'error', signal: AbortSignal.timeout(HTTP_TIMEOUT_MS) })+ if (!response.ok) return ''+ const profile = await response.json() as Record<string, unknown>+ for (const key of ['email', 'name', 'login', 'username']) if (typeof profile[key] === 'string' && profile[key]) return profile[key] as string+ } catch { /* The tokens are already good. */ }+ return ''+}++/**+ * The service's redirect, forwarded by the Autonomous web callback: exchanged once (the state is taken+ * from Redis as it is read, so a replay finds nothing), its result left for the computer's poll.+ */+export async function callback(body: { code?: unknown, state?: unknown, error?: unknown }): Promise<Record<string, unknown>> {+ const state = str(body.state)+ if (!state.startsWith(STATE_PREFIX)) throw new GatewayError('This sign-in is not Harness\'s.', 'UNKNOWN_STATE', 400)+ // GET and DEL in one transaction (GETDEL needs Redis 6.2): a replayed callback finds nothing.+ const taken = await pub.multi().get(stateKey(state)).del(stateKey(state)).exec()+ const raw = taken?.[0]?.[1] as string | null | undefined+ if (!raw) throw new GatewayError('This sign-in has ended. Start it again from Harness.', 'STATE_EXPIRED', 410)+ const flow = JSON.parse(raw) as FlowState+ const config = app(flow.connector)+ const save = (record: PickupRecord) => pub.set(pickupKey(flow.pickup), JSON.stringify(record), 'EX', TTL_SEC)+ if (body.error || !str(body.code)) {+ await save({ userId: flow.userId, connector: config.code, status: 'failed', error: str(body.error) === 'access_denied' ? 'The sign-in was cancelled.' : 'The service did not complete the sign-in.' })+ return { connector: config.code }+ }+ try {+ const form: Record<string, string> = { grant_type: 'authorization_code', code: str(body.code), redirect_uri: env.CONNECTOR_REDIRECT_URI }+ if (flow.verifier) form.code_verifier = flow.verifier+ const payload = await tokenRequest(config, form, false)+ const first = tokenFor(config, payload)+ const token = { ...first, account_name: await accountName(config, first.access_token as string) }+ await keep(flow.userId, config.code, token)+ await save({ userId: flow.userId, connector: config.code, status: 'ready', token })+ } catch (error) {+ await save({ userId: flow.userId, connector: config.code, status: 'failed', error: error instanceof GatewayError ? error.message : 'The sign-in did not finish.' })+ }+ // Never the token: the browser that carried the code is not the computer that asked.+ return { connector: config.code }+}++/** The sign-in's result, to the account that started it, exactly once. */+export async function poll(userId: string, pickupCode: unknown): Promise<Record<string, unknown>> {+ const pickup = str(pickupCode)+ const raw = pickup ? await pub.get(pickupKey(pickup)) : null+ if (!raw) return { status: 'expired' }+ const record = JSON.parse(raw) as PickupRecord+ const mine = Buffer.from(sha(record.userId)), asker = Buffer.from(sha(userId))+ if (!timingSafeEqual(mine, asker)) return { status: 'expired' }+ if (record.status === 'pending') return { status: 'pending', connector: record.connector }+ // One-time: taken as it is answered; a second poll finds it consumed.+ if (!(await pub.del(pickupKey(pickup)))) return { status: 'consumed' }+ return record.status === 'ready'+ ? { status: 'ready', ...record.token }+ : { status: 'failed', connector: record.connector, error: record.error ?? 'The sign-in did not finish.' }+}++/**+ * A new access token for the account's sign-in, renewed with the refresh token kept here (Grid's+ * `/connectors/refresh {connector}`). A provider without refresh tokens answers what is kept, until it+ * expires; a revoked one asks for connecting again and leaves the row for the next sign-in to replace.+ */+export async function refresh(userId: string, connector: unknown): Promise<Record<string, unknown>> {+ const config = app(connector)+ const row = await prisma.connectorCredential.findUnique({ where: { userId_connector: { userId, connector: config.code } } })+ if (!row) throw new GatewayError('This service is not connected. Connect it again.', 'NOT_CONNECTED', 404)+ const held = unseal(row.sealed)+ const refreshToken = str(held.refresh_token)+ if (!refreshToken || !config.refresh) {+ return tokenFor(config, { ...held, expires_in: undefined }, refreshToken, row.accountName, Number(held.expires_at) || 0)+ }+ const payload = await tokenRequest(config, { grant_type: 'refresh_token', refresh_token: refreshToken }, true)+ const token = tokenFor(config, payload, refreshToken, row.accountName)+ await keep(userId, config.code, token)+ return token+}++/** Forget the account's sign-in here. Revoking access at the service is the service's own setting. */+export async function disconnect(userId: string, connector: unknown): Promise<Record<string, unknown>> {+ const config = app(connector)+ const { count } = await prisma.connectorCredential.deleteMany({ where: { userId, connector: config.code } })+ return { connector: config.code, disconnected: count > 0 }+}diff --git a/backend/src/middlewares/authMiddleware.ts b/backend/src/middlewares/authMiddleware.tsindex abaaf3d76..dd3194ef1 100644--- a/backend/src/middlewares/authMiddleware.ts+++ b/backend/src/middlewares/authMiddleware.ts@@ -66,7 +66,11 @@ export function shouldSkipAuth(url: string): boolean {// The approve/deny/lookup half is NOT listed and stays SSO-gated — that is where the user's identity// and their machine list are involved. Listed one by one, like the cursor pair above.path === '/api/device-auth/start' ||- path === '/api/device-auth/poll'+ path === '/api/device-auth/poll' ||+ // The connector gateway's callback (routes/connectors.ts), posted by the Autonomous web callback+ // page for an anonymous browser: its one-time `harness_` state is the ticket, and it answers no token.+ // The rest of /api/connectors stays SSO-gated.+ path === '/api/connectors/callback')}diff --git a/backend/src/routes/connectors.test.ts b/backend/src/routes/connectors.test.tsnew file mode 100644index 000000000..ab5da16a4--- /dev/null+++ b/backend/src/routes/connectors.test.ts@@ -0,0 +1,182 @@+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'+import Fastify, { type FastifyInstance } from 'fastify'+import { createHash } from 'crypto'++const mocks = vi.hoisted(() => {+ // An in-memory Redis for the sign-ins in progress: set (with EX), get, del and multi.+ const store = new Map<string, string>()+ const ops = {+ set: (key: string, value: string) => { store.set(key, value); return 'OK' },+ get: (key: string) => store.get(key) ?? null,+ del: (key: string) => (store.delete(key) ? 1 : 0),+ }+ const pub = {+ set: vi.fn(async (key: string, value: string) => ops.set(key, value)),+ get: vi.fn(async (key: string) => ops.get(key)),+ del: vi.fn(async (key: string) => ops.del(key)),+ multi: vi.fn(() => {+ const queued: (() => unknown)[] = []+ const chain = {+ set: (key: string, value: string) => { queued.push(() => ops.set(key, value)); return chain },+ get: (key: string) => { queued.push(() => ops.get(key)); return chain },+ del: (key: string) => { queued.push(() => ops.del(key)); return chain },+ exec: async () => queued.map(run => [null, run()]),+ }+ return chain+ }),+ }+ // The account's kept sign-ins (ConnectorCredential), by userId + connector.+ type Row = { userId: string, connector: string, sealed: string, accountName: string, expiresAt: number, connectedAt: Date }+ const rows = new Map<string, Row>()+ const id = (where: { userId_connector: { userId: string, connector: string } }) => `${where.userId_connector.userId}/${where.userId_connector.connector}`+ const connectorCredential = {+ findMany: vi.fn(async ({ where }: { where: { userId: string } }) => [...rows.values()].filter(row => row.userId === where.userId)),+ findUnique: vi.fn(async ({ where }: { where: { userId_connector: { userId: string, connector: string } } }) => rows.get(id(where)) ?? null),+ upsert: vi.fn(async ({ where, create, update }: { where: { userId_connector: { userId: string, connector: string } }, create: Row, update: Partial<Row> }) => {+ const row = rows.has(id(where)) ? { ...rows.get(id(where))!, ...update } : { ...create, connectedAt: new Date() }+ rows.set(id(where), row as Row); return row+ }),+ deleteMany: vi.fn(async ({ where }: { where: { userId: string, connector: string } }) => {+ const had = rows.delete(`${where.userId}/${where.connector}`); return { count: had ? 1 : 0 }+ }),+ }+ return { store, pub, rows, prisma: { connectorCredential }, auth: vi.fn() }+})+vi.mock('../lib/bus.js', () => ({ pub: mocks.pub }))+vi.mock('../lib/prisma.js', () => ({ prisma: mocks.prisma }))+vi.mock('../lib/ssoAuth.js', async original => ({ ...await original<typeof import('../lib/ssoAuth.js')>(), authenticateAccessToken: mocks.auth }))+vi.mock('../config/env.js', () => {+ return { env: { NODE_ENV: 'test', CONNECTOR_REDIRECT_URI: 'https://www.autonomous.ai/connector/callback', CONNECTOR_AUTH_FILE: undefined,+ CONNECTOR_ENCRYPTION_KEY: Buffer.alloc(32, 7).toString('base64'),+ CONNECTOR_AUTH_JSON: JSON.stringify({ connectors: {+ github: { auth_type: 'app', label: 'GitHub', client_id: 'gh-client', client_secret: 'gh-secret', auth_url: 'https://github.com/login/oauth/authorize',+ token_url: 'https://github.com/login/oauth/access_token', scopes: ['repo', 'read:user'], userinfo_url: 'https://api.github.com/user', mcp_url: 'https://api.githubcopilot.com/mcp/' },+ gmail: { auth_type: 'app', label: 'Gmail', client_id: 'g-client', client_secret: 'g-secret', auth_url: 'https://accounts.google.com/o/oauth2/v2/auth',+ token_url: 'https://oauth2.googleapis.com/token', scopes: ['https://www.googleapis.com/auth/gmail.readonly'], refresh: true, auth_style: 'header',+ auth_params: { access_type: 'offline', prompt: 'consent' } },+ notion: { auth_type: 'app', client_id: 'n', auth_url: 'https://mcp.notion.com/authorize', token_url: 'https://mcp.notion.com/token', pkce: true },+ linear: { auth_type: 'dcr', mcp_url: 'https://mcp.linear.app/mcp' },+ } }) } }+})+import { connectorRoutes } from './connectors.js'+import { resetConfig } from '../lib/connectorGateway.js'+import { registerAuthMiddleware } from '../middlewares/authMiddleware.js'+import { errorHandler } from '../middlewares/errorHandler.js'++type Fetched = { url: string, init: RequestInit }++describe('the connector gateway', () => {+ let app: FastifyInstance+ let fetched: Fetched[]+ let answer: (url: string, init: RequestInit) => Response+ const owner = { authorization: 'Bearer owner' }++ beforeEach(async () => {+ vi.clearAllMocks(); mocks.store.clear(); mocks.rows.clear(); resetConfig(); fetched = []+ mocks.auth.mockImplementation(async (token: string) => ({ sub: token === 'stranger' ? 'stranger' : 'owner', email: '[email protected]', role: 'user', autonomousEnv: 'prod' }))+ answer = (url) => url.includes('/user')+ ? Response.json({ login: 'octo' })+ : Response.json({ access_token: 'gho-1', token_type: 'bearer', scope: 'repo,read:user' })+ vi.stubGlobal('fetch', vi.fn(async (url: string, init: RequestInit) => { fetched.push({ url: String(url), init }); return answer(String(url), init) }))+ app = Fastify(); app.setErrorHandler(errorHandler); registerAuthMiddleware(app, mocks.auth)+ await app.register(connectorRoutes); await app.ready()+ })+ afterEach(async () => { vi.unstubAllGlobals(); await app.close() })++ const post = (url: string, payload: unknown, headers: Record<string, string> = owner) => app.inject({ method: 'POST', url, payload: payload as object, headers })++ it('lists only the services that sign in here, never a secret', async () => {+ const res = await app.inject({ method: 'GET', url: '/api/connectors', headers: owner })+ const codes = res.json().data.connectors.map((row: { code: string }) => row.code)+ expect(codes).toEqual(['github', 'gmail', 'notion'])+ expect(res.body).not.toContain('secret')+ expect((await app.inject({ method: 'GET', url: '/api/connectors' })).statusCode).toBe(401)+ })++ it('signs in through the service, hands the token over once, to the account that asked', async () => {+ const started = (await post('/api/connectors/start', { connector: 'github' })).json().data+ const consent = new URL(started.authorize_url)+ expect(consent.origin + consent.pathname).toBe('https://github.com/login/oauth/authorize')+ const state = consent.searchParams.get('state')!+ expect(state.startsWith('harness_')).toBe(true)+ expect(consent.searchParams.get('redirect_uri')).toBe('https://www.autonomous.ai/connector/callback')+ expect(consent.searchParams.get('scope')).toBe('repo read:user')+ expect((await post('/api/connectors/poll', { pickup_code: started.pickup_code })).json().data.status).toBe('pending')++ // The Autonomous web callback page, for an anonymous browser: no token in the answer.+ const back = await post('/api/connectors/callback', { code: 'code-1', state }, {})+ expect(back.json().data).toEqual({ connector: 'github' })+ const exchange = fetched.find(call => call.url.includes('access_token'))!+ expect(String(exchange.init.body)).toContain('client_secret=gh-secret')+ expect(String(exchange.init.body)).toContain('redirect_uri=https%3A%2F%2Fwww.autonomous.ai%2Fconnector%2Fcallback')++ expect((await post('/api/connectors/poll', { pickup_code: started.pickup_code }, { authorization: 'Bearer stranger' })).json().data.status).toBe('expired')+ const ready = (await post('/api/connectors/poll', { pickup_code: started.pickup_code })).json().data+ expect(ready).toMatchObject({ status: 'ready', access_token: 'gho-1', account_name: 'octo',+ mcp_entry: { url: 'https://api.githubcopilot.com/mcp/', headers: { Authorization: 'Bearer gho-1' } } })+ expect(JSON.stringify(ready)).not.toContain('gh-secret')+ expect((await post('/api/connectors/poll', { pickup_code: started.pickup_code })).json().data.status).toBe('expired')+ // Nothing of the sign-in is left in Redis; the account's sign-in is kept, sealed.+ expect(mocks.store.size).toBe(0)+ const kept = mocks.rows.get('owner/github')!+ expect(kept.accountName).toBe('octo')+ expect(kept.sealed.startsWith('v1.')).toBe(true)+ expect(kept.sealed).not.toContain('gho-1')+ const listed = (await app.inject({ method: 'GET', url: '/api/connectors', headers: owner })).json().data.connectors+ expect(listed.find((row: { code: string }) => row.code === 'github')).toMatchObject({ status: 'connected', account_name: 'octo' })+ const theirs = (await app.inject({ method: 'GET', url: '/api/connectors', headers: { authorization: 'Bearer stranger' } })).json().data.connectors+ expect(theirs.find((row: { code: string }) => row.code === 'github').status).toBe('not_connected')+ })++ it('takes a callback once, refuses another\'s state, and says when the service said no', async () => {+ const started = (await post('/api/connectors/start', { connector: 'github' })).json().data+ const state = new URL(started.authorize_url).searchParams.get('state')!+ expect((await post('/api/connectors/callback', { code: 'x', state: 'grid_abc' }, {})).statusCode).toBe(400)+ expect((await post('/api/connectors/callback', { error: 'access_denied', state }, {})).statusCode).toBe(200)+ expect((await post('/api/connectors/callback', { code: 'again', state }, {})).statusCode).toBe(410)+ expect((await post('/api/connectors/poll', { pickup_code: started.pickup_code })).json().data)+ .toEqual({ status: 'failed', connector: 'github', error: 'The sign-in was cancelled.' })+ })++ it('sends Google\'s offline consent, PKCE where configured, and client credentials as Basic when asked', async () => {+ const google = new URL((await post('/api/connectors/start', { connector: 'gmail' })).json().data.authorize_url)+ expect(google.searchParams.get('access_type')).toBe('offline')+ expect(google.searchParams.get('code_challenge')).toBeNull()+ const notion = new URL((await post('/api/connectors/start', { connector: 'notion' })).json().data.authorize_url)+ expect(notion.searchParams.get('code_challenge_method')).toBe('S256')+ // Signed in to Gmail: a refresh token Google gave once, kept here.+ const gmail = (await post('/api/connectors/start', { connector: 'gmail' })).json().data+ answer = () => Response.json({ access_token: 'ya29-1', expires_in: 3600, refresh_token: 'refresh-held' })+ await post('/api/connectors/callback', { code: 'c', state: new URL(gmail.authorize_url).searchParams.get('state') }, {})+ answer = () => Response.json({ access_token: 'ya29-2', expires_in: 3600 })+ const renewed = (await post('/api/connectors/refresh', { connector: 'gmail' })).json().data+ expect(renewed).toMatchObject({ access_token: 'ya29-2', refresh_token: 'refresh-held', refresh: true })+ const call = fetched.at(-1)!+ expect((call.init.headers as Record<string, string>).Authorization).toBe('Basic ' + Buffer.from('g-client:g-secret').toString('base64'))+ expect(String(call.init.body)).toContain('refresh_token=refresh-held')+ expect(String(call.init.body)).not.toContain('g-secret')+ })++ it('a revoked refresh token asks for connecting again; disconnect forgets; an unknown service is refused', async () => {+ const gmail = (await post('/api/connectors/start', { connector: 'gmail' })).json().data+ answer = () => Response.json({ access_token: 'ya29-1', expires_in: 3600, refresh_token: 'r' })+ await post('/api/connectors/callback', { code: 'c', state: new URL(gmail.authorize_url).searchParams.get('state') }, {})+ answer = () => Response.json({ error: 'invalid_grant' }, { status: 400 })+ const revoked = await post('/api/connectors/refresh', { connector: 'gmail' })+ expect([revoked.statusCode, revoked.json().error.code]).toEqual([401, 'INVALID_GRANT'])+ expect((await post('/api/connectors/refresh', { connector: 'gmail' }, { authorization: 'Bearer stranger' })).json().error.code).toBe('NOT_CONNECTED')+ expect((await post('/api/connectors/disconnect', { connector: 'gmail' })).json().data).toEqual({ connector: 'gmail', disconnected: true })+ expect(mocks.rows.size).toBe(0)+ expect((await post('/api/connectors/start', { connector: 'linear' })).statusCode).toBe(404)+ expect((await post('/api/connectors/start', { connector: '../x' })).statusCode).toBe(404)+ })++ it('stores only hashes of the state and the pickup code', async () => {+ const started = (await post('/api/connectors/start', { connector: 'github' })).json().data+ const state = new URL(started.authorize_url).searchParams.get('state')!+ const keys = [...mocks.store.keys()].join()+ expect(keys).not.toContain(state)+ expect(keys).not.toContain(started.pickup_code)+ expect(keys).toContain(createHash('sha256').update(started.pickup_code).digest('hex'))+ })+})diff --git a/backend/src/routes/connectors.ts b/backend/src/routes/connectors.tsnew file mode 100644index 000000000..fe2f560ab--- /dev/null+++ b/backend/src/routes/connectors.ts@@ -0,0 +1,51 @@+import type { FastifyInstance, FastifyReply } from 'fastify'+import { callback, disconnect, GatewayError, list, poll, refresh, start } from '../lib/connectorGateway.js'+import { sendError, sendSuccess } from '../utils/response.js'+import { logger } from '../utils/logger.js'++/**+ * The connector gateway (lib/connectorGateway.ts): sign-in to GitHub, Slack, Google and the other+ * services that let no computer register its own OAuth client, for `harness connections` and the+ * desktop app's Settings ▸ Connectors.+ *+ * GET /api/connectors (signed in) the services that sign in here, with this account's state+ * POST /api/connectors/start (signed in) {connector} → {authorize_url, pickup_code, …}+ * POST /api/connectors/poll (signed in) {pickup_code} → {status, …token once ready}+ * POST /api/connectors/refresh (signed in) {connector} → a new token, renewed with the one kept here+ * POST /api/connectors/disconnect (signed in) {connector} → forget the account's sign-in here+ * POST /api/connectors/callback (no auth) {code, state} from the Autonomous web callback page+ */+export async function connectorRoutes(app: FastifyInstance): Promise<void> {+ const fail = (reply: FastifyReply, error: unknown) => {+ if (error instanceof GatewayError) return sendError(reply, error.message, error.code, error.status)+ logger.error('connector gateway failed', { error: String(error) })+ return sendError(reply, 'Connections are unavailable. Try again.', 'CONNECTORS_FAILED', 503)+ }+ type Body = Record<string, unknown> | undefined++ app.get('/api/connectors', async (req, reply) => {+ try { sendSuccess(reply, { connectors: await list(req.user!.sub) }) } catch (error) { fail(reply, error) }+ })++ app.post<{ Body: Body }>('/api/connectors/start', async (req, reply) => {+ try { sendSuccess(reply, await start(req.user!.sub, req.body?.connector)) } catch (error) { fail(reply, error) }+ })++ app.post<{ Body: Body }>('/api/connectors/poll', async (req, reply) => {+ try { sendSuccess(reply, await poll(req.user!.sub, req.body?.pickup_code)) } catch (error) { fail(reply, error) }+ })++ app.post<{ Body: Body }>('/api/connectors/refresh', async (req, reply) => {+ try { sendSuccess(reply, await refresh(req.user!.sub, req.body?.connector)) } catch (error) { fail(reply, error) }+ })++ app.post<{ Body: Body }>('/api/connectors/disconnect', async (req, reply) => {+ try { sendSuccess(reply, await disconnect(req.user!.sub, req.body?.connector)) } catch (error) { fail(reply, error) }+ })++ // Unauthenticated by design: the browser that brings the code is anonymous to Harness. The state+ // (minted by start, one use, ten minutes) is the ticket, and the answer never carries a token.+ app.post<{ Body: Body }>('/api/connectors/callback', async (req, reply) => {+ try { sendSuccess(reply, await callback(req.body ?? {})) } catch (error) { fail(reply, error) }+ })+}diff --git a/backend/src/server.ts b/backend/src/server.tsindex 2b2e14a2d..99ffc19b4 100644--- a/backend/src/server.ts+++ b/backend/src/server.ts@@ -15,6 +15,7 @@ import { communityRoutes } from './routes/community.js'import { deviceKeyRoutes } from './routes/deviceKeys.js'import { qrSignInRoutes } from './routes/qrSignIn.js'import { deskRoutes } from './routes/desk.js'+import { connectorRoutes } from './routes/connectors.js'import { tabChannelRoutes } from './routes/tabChannels.js'import { zooRoutes } from './routes/zoo.js'import { experimentalSettingsRoutes } from './routes/experimentalSettings.js'@@ -177,6 +178,7 @@ async function start(): Promise<void> {await app.register(qrSignInRoutes) // sign a computer in by scanning its QR with a signed-in phoneawait app.register(deviceKeyRoutes) // the account's device key log: signing in is what trusts a deviceawait app.register(deskRoutes) // the account's tabs, the same on every computer (lib/desk.ts)+ await app.register(connectorRoutes) // sign-in to GitHub, Slack, Google… for local agents (lib/connectorGateway.ts)if (env.HARNESS_CHANNELS) await app.register(tabChannelRoutes)await app.register(experimentalSettingsRoutes, { daemons: DAEMONS })// Availability is separate from opt-in: every account starts off, and its collection is retained.