Open-source SEO, GEO, and marketing skills for AI agents.
feat(notfair): centralized adaptive PR polling on the scheduler sweep
Yuting Zhonga104c611deb0
diff --git a/notfair/CHANGELOG.md b/notfair/CHANGELOG.mdindex 6950d2a2..ca3c1ac8 100644--- a/notfair/CHANGELOG.md+++ b/notfair/CHANGELOG.md@@ -6,7 +6,7 @@**Completed checks stop looking active.** A harness final event now renders as a static “Turn complete” state with no spinner, heartbeat, shimmer, or increasing timer. Read-only check logs also explain that follow-up belongs in the goal chat instead of implying the agent is still working forever.-**Code changes go through pull requests — and the loop tracks them natively.** Goals whose levers live in your website's code (SEO above all) now have a sanctioned, governed channel: the agent may change code only inside the workspace's **Codebase folder** (new Settings card, validated + editable any time), only on a `notfair/…` branch, and only via a PR it registers with the new `register_pull_request` tool — never a direct push, never merging its own PR. You review and merge on GitHub, exactly like any teammate's PR. The platform syncs PR state through your authenticated `gh` CLI on every check (and when you view the goal), so each tick brief carries the live truth: changes requested → the agent addresses your review comments that tick; awaiting review → your Goal tab and the goals index show a "PR needs your review" badge; merged → the observation window measures from there; closed unmerged → the agent scores the action as rejected and learns. Open PRs also pin the heartbeat to its normal cadence (no smart-sleep past a pending review). Without a codebase folder set, code stays off-limits and agents log recommendations instead.+**Code changes go through pull requests — and the loop tracks them natively.** Goals whose levers live in your website's code (SEO above all) now have a sanctioned, governed channel: the agent may change code only inside the workspace's **Codebase folder** (new Settings card, validated + editable any time), only on a `notfair/…` branch, and only via a PR it registers with the new `register_pull_request` tool — never a direct push, never merging its own PR. You review and merge on GitHub, exactly like any teammate's PR. PR state syncs through your authenticated `gh` CLI via a centralized freshness sweep on the 30-second scheduler: every open PR carries its own adaptive `next_sync_at` (checks every 2 minutes right after activity, decaying to hourly for long-quiet PRs, resetting on any observed change; merged/closed PRs leave the schedule forever), with tick-time and goal-page-view syncs kept as correctness/UX guarantees and a single-flight guard so triggers coalesce. Each tick brief therefore carries the live truth: changes requested → the agent addresses your review comments that tick; awaiting review → your Goal tab and the goals index show a "PR needs your review" badge; merged → the observation window measures from there; closed unmerged → the agent scores the action as rejected and learns. Open PRs also pin the heartbeat to its normal cadence (no smart-sleep past a pending review). Without a codebase folder set, code stays off-limits and agents log recommendations instead.**Fix: the START button now appears without a manual reload.** The goal page only auto-refreshed during `intake` and `active`, so once the metric verified (→ `proposed`) the page went static — the agent would say "press START" but the button never rendered until you reloaded. `proposed` goals now poll too, so the target recording and the START card surface live.diff --git a/notfair/next-env.d.ts b/notfair/next-env.d.tsindex 9edff1c7..c4b7818f 100644--- a/notfair/next-env.d.ts+++ b/notfair/next-env.d.ts@@ -1,6 +1,6 @@/// <reference types="next" />/// <reference types="next/image-types/global" />-import "./.next/types/routes.d.ts";+import "./.next/dev/types/routes.d.ts";// NOTE: This file should not be edited// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.diff --git a/notfair/src/lib/pr-poll-policy.test.ts b/notfair/src/lib/pr-poll-policy.test.tsnew file mode 100644index 00000000..cf2be217--- /dev/null+++ b/notfair/src/lib/pr-poll-policy.test.ts@@ -0,0 +1,49 @@+import { describe, expect, it } from "vitest";++import {+ computeNextSyncAt,+ nextSyncDelayMs,+ SYNC_ERROR_RETRY_MS,+} from "@/lib/pr-poll-policy";++const MIN = 60_000;+const HOUR = 3_600_000;++describe("nextSyncDelayMs", () => {+ it("polls fast right after activity, decaying to an hourly cap", () => {+ expect(nextSyncDelayMs(0)).toBe(2 * MIN);+ expect(nextSyncDelayMs(59 * MIN)).toBe(2 * MIN);+ expect(nextSyncDelayMs(2 * HOUR)).toBe(10 * MIN);+ expect(nextSyncDelayMs(23 * HOUR)).toBe(10 * MIN);+ expect(nextSyncDelayMs(48 * HOUR)).toBe(30 * MIN);+ expect(nextSyncDelayMs(100 * HOUR)).toBe(60 * MIN);+ expect(nextSyncDelayMs(1000 * HOUR)).toBe(60 * MIN);+ });+});++describe("computeNextSyncAt", () => {+ it("schedules from now, keyed off quietness", () => {+ const now = "2026-07-13T12:00:00.000Z";+ // Fresh activity → 2 minutes out.+ expect(computeNextSyncAt(now, "2026-07-13T11:59:00.000Z")).toBe(+ "2026-07-13T12:02:00.000Z",+ );+ // Two days quiet → 30 minutes out.+ expect(computeNextSyncAt(now, "2026-07-11T12:00:00.000Z")).toBe(+ "2026-07-13T12:30:00.000Z",+ );+ });++ it("treats a garbage activity timestamp as fresh (fast lane)", () => {+ const now = "2026-07-13T12:00:00.000Z";+ expect(computeNextSyncAt(now, "not-a-date")).toBe(+ "2026-07-13T12:02:00.000Z",+ );+ });+});++describe("SYNC_ERROR_RETRY_MS", () => {+ it("retries failures on a short fixed delay", () => {+ expect(SYNC_ERROR_RETRY_MS).toBe(5 * MIN);+ });+});diff --git a/notfair/src/lib/pr-poll-policy.ts b/notfair/src/lib/pr-poll-policy.tsnew file mode 100644index 00000000..b3d1e098--- /dev/null+++ b/notfair/src/lib/pr-poll-policy.ts@@ -0,0 +1,39 @@+/**+ * Adaptive polling policy for open pull requests — pure functions shared+ * by the db layer (which stamps next_sync_at on every write) and tests.+ *+ * The centralized 30s scheduler sweep syncs a PR when its own+ * `next_sync_at` arrives; this policy decides how far out that is. The+ * interval keys off time since the PR's last OBSERVED activity (state /+ * review decision / comment count change): hot PRs — just opened, just+ * reviewed — poll fast; a PR that has sat quiet for days decays toward+ * an hourly cap. Any observed change resets the clock to the fast lane.+ */++const MINUTE = 60_000;+const HOUR = 3_600_000;++/** How long to wait before the next GitHub check, given quietness. */+export function nextSyncDelayMs(msSinceActivity: number): number {+ if (msSinceActivity < 1 * HOUR) return 2 * MINUTE;+ if (msSinceActivity < 24 * HOUR) return 10 * MINUTE;+ if (msSinceActivity < 72 * HOUR) return 30 * MINUTE;+ return 60 * MINUTE;+}++/** Retry delay after a failed sync (gh missing, network, rate limit). */+export const SYNC_ERROR_RETRY_MS = 5 * MINUTE;++/**+ * Compute the next_sync_at for an OPEN PR from "now" and its last+ * activity timestamp. Terminal PRs never poll — callers store NULL.+ */+export function computeNextSyncAt(+ nowIso: string,+ lastActivityAtIso: string,+): string {+ const now = Date.parse(nowIso);+ const activity = Date.parse(lastActivityAtIso);+ const since = Number.isFinite(activity) ? Math.max(0, now - activity) : 0;+ return new Date(now + nextSyncDelayMs(since)).toISOString();+}diff --git a/notfair/src/server/db/goal-prs.test.ts b/notfair/src/server/db/goal-prs.test.tsindex 45227ca8..232282de 100644--- a/notfair/src/server/db/goal-prs.test.ts+++ b/notfair/src/server/db/goal-prs.test.ts@@ -15,6 +15,7 @@ import {applyGoalPrSync,createGoalPr,findGoalPrByUrl,+ listDuePrSyncs,listGoalPrs,listOpenGoalPrs,listPrsAwaitingReview,@@ -123,3 +124,73 @@ describe("goal_prs data layer", () => {expect(awaiting.map((p) => p.id)).toEqual([a.id]);});});++describe("adaptive sync scheduling", () => {+ it("a fresh registration is immediately due", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ expect(pr.next_sync_at).not.toBeNull();+ expect(pr.last_activity_at).not.toBeNull();+ expect(listDuePrSyncs().map((p) => p.id)).toContain(pr.id);+ });++ it("an observed change resets activity and schedules the fast lane", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ const synced = applyGoalPrSync(pr.id, {+ state: "open",+ review_decision: "CHANGES_REQUESTED",+ comment_count: 2,+ is_draft: false,+ merged_at: null,+ })!;+ expect(synced.last_activity_at).not.toBeNull();+ // Fast lane: ~2 minutes out — definitely not due now, well within 3 min.+ const dueIn = Date.parse(synced.next_sync_at!) - Date.now();+ expect(dueIn).toBeGreaterThan(60_000);+ expect(dueIn).toBeLessThan(3 * 60_000);+ expect(listDuePrSyncs().map((p) => p.id)).not.toContain(pr.id);+ });++ it("a no-change sync keeps the old activity timestamp (backoff grows)", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ const payload = {+ state: "open" as const,+ review_decision: "",+ comment_count: 0,+ is_draft: false,+ merged_at: null,+ };+ const first = applyGoalPrSync(pr.id, payload)!; // change (decision null → "")+ const second = applyGoalPrSync(pr.id, payload)!; // no change+ expect(second.last_activity_at).toBe(first.last_activity_at);+ });++ it("terminal PRs leave the schedule entirely", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ const merged = applyGoalPrSync(pr.id, {+ state: "merged",+ review_decision: "APPROVED",+ comment_count: 0,+ is_draft: false,+ merged_at: "2026-07-13T00:00:00Z",+ })!;+ expect(merged.next_sync_at).toBeNull();+ expect(listDuePrSyncs().map((p) => p.id)).not.toContain(pr.id);+ });++ it("a failed sync retries on the error delay instead of going dark", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ markGoalPrSyncError(pr.id, "gh: command not found");+ const row = findGoalPrByUrl(goalId, URL1)!;+ const dueIn = Date.parse(row.next_sync_at!) - Date.now();+ expect(dueIn).toBeGreaterThan(4 * 60_000);+ expect(dueIn).toBeLessThan(6 * 60_000);+ });++ it("open rows with NULL next_sync_at (pre-policy data) self-heal as due", () => {+ const pr = createGoalPr({ goal_id: goalId, url: URL1, title: "t" });+ getDb()+ .prepare("UPDATE goal_prs SET next_sync_at = NULL WHERE id = ?")+ .run(pr.id);+ expect(listDuePrSyncs().map((p) => p.id)).toContain(pr.id);+ });+});diff --git a/notfair/src/server/db/goal-prs.ts b/notfair/src/server/db/goal-prs.tsindex 735e427e..c5e74a1b 100644--- a/notfair/src/server/db/goal-prs.ts+++ b/notfair/src/server/db/goal-prs.ts@@ -1,4 +1,8 @@import { randomUUID } from "node:crypto";+import {+ computeNextSyncAt,+ SYNC_ERROR_RETRY_MS,+} from "@/lib/pr-poll-policy";import { getDb } from "./db";/**@@ -30,6 +34,12 @@ export type GoalPr = {merged_at: string | null;last_synced_at: string | null;sync_error: string | null;+ /** When the centralized sweep should next check GitHub. NULL for+ * terminal PRs (merged/closed) — they are never synced again. */+ next_sync_at: string | null;+ /** Last time a sync OBSERVED a change (state / review / comments).+ * Drives the adaptive poll interval in pr-poll-policy. */+ last_activity_at: string | null;created_at: string;updated_at: string;};@@ -51,8 +61,8 @@ export function createGoalPr(input: {const now = new Date().toISOString();const id = randomUUID();db.prepare(- `INSERT OR IGNORE INTO goal_prs (id, goal_id, action_id, url, title, branch, created_at, updated_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?)`,+ `INSERT OR IGNORE INTO goal_prs (id, goal_id, action_id, url, title, branch, next_sync_at, last_activity_at, created_at, updated_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,).run(id,input.goal_id,@@ -60,6 +70,9 @@ export function createGoalPr(input: {input.url,input.title,input.branch ?? null,+ // Immediately due: a fresh registration should reflect GitHub asap.+ now,+ now,now,now,);@@ -132,12 +145,29 @@ export function applyGoalPrSync(},): GoalPr | null {const db = getDb();+ const current = getGoalPr(id);+ if (!current) return null;const now = new Date().toISOString();++ // "Activity" = any observable change on GitHub since the last sync.+ // It resets the adaptive poll clock so hot PRs poll fast again.+ const changed =+ current.state !== sync.state ||+ current.review_decision !== sync.review_decision ||+ current.comment_count !== sync.comment_count ||+ current.is_draft !== sync.is_draft;+ const last_activity_at = changed+ ? now+ : (current.last_activity_at ?? current.created_at);+ const next_sync_at =+ sync.state === "open" ? computeNextSyncAt(now, last_activity_at) : null;+db.prepare(`UPDATE goal_prsSET state = ?, title = COALESCE(?, title), review_decision = ?,comment_count = ?, is_draft = ?, merged_at = ?,- last_synced_at = ?, sync_error = NULL, updated_at = ?+ last_synced_at = ?, sync_error = NULL,+ next_sync_at = ?, last_activity_at = ?, updated_at = ?WHERE id = ?`,).run(sync.state,@@ -147,6 +177,8 @@ export function applyGoalPrSync(sync.is_draft ? 1 : 0,sync.merged_at,now,+ next_sync_at,+ last_activity_at,now,id,);@@ -154,10 +186,28 @@ export function applyGoalPrSync(}export function markGoalPrSyncError(id: string, error: string): void {- const now = new Date().toISOString();+ const now = Date.now();+ const nowIso = new Date(now).toISOString();+ const retryIso = new Date(now + SYNC_ERROR_RETRY_MS).toISOString();getDb().prepare(- "UPDATE goal_prs SET sync_error = ?, last_synced_at = ?, updated_at = ? WHERE id = ?",+ "UPDATE goal_prs SET sync_error = ?, last_synced_at = ?, next_sync_at = ?, updated_at = ? WHERE id = ?",)- .run(error.slice(0, 500), now, now, id);+ .run(error.slice(0, 500), nowIso, retryIso, nowIso, id);+}++/**+ * Open PRs due for a freshness check, across every project — the+ * centralized sweep's work list. `next_sync_at IS NULL` on an open row+ * (pre-policy data) counts as due so it self-heals into the schedule.+ */+export function listDuePrSyncs(nowIso = new Date().toISOString()): GoalPr[] {+ const rows = getDb()+ .prepare(+ `SELECT * FROM goal_prs+ WHERE state = 'open' AND (next_sync_at IS NULL OR next_sync_at <= ?)+ ORDER BY next_sync_at ASC`,+ )+ .all(nowIso) as GoalPrRow[];+ return rows.map(mapRow);}diff --git a/notfair/src/server/db/schema.ts b/notfair/src/server/db/schema.tsindex 1a6f7966..0963f9e2 100644--- a/notfair/src/server/db/schema.ts+++ b/notfair/src/server/db/schema.ts@@ -205,11 +205,18 @@ CREATE TABLE IF NOT EXISTS goal_prs (merged_at TEXT,last_synced_at TEXT,sync_error TEXT,+ -- Centralized freshness sweep: the 30s scheduler syncs a PR when+ -- next_sync_at arrives. The interval adapts to activity (fast after+ -- last_activity_at, backing off toward an hourly cap); terminal PRs+ -- carry NULL and are never synced again.+ next_sync_at TEXT,+ last_activity_at TEXT,created_at TEXT NOT NULL,updated_at TEXT NOT NULL,UNIQUE(goal_id, url));CREATE INDEX IF NOT EXISTS idx_goal_prs_goal ON goal_prs(goal_id, state);+CREATE INDEX IF NOT EXISTS idx_goal_prs_due ON goal_prs(state, next_sync_at);CREATE TABLE IF NOT EXISTS goal_ticks (id TEXT PRIMARY KEY,diff --git a/notfair/src/server/goals/pr-sync.ts b/notfair/src/server/goals/pr-sync.tsindex 872756c0..ccd3a5e8 100644--- a/notfair/src/server/goals/pr-sync.ts+++ b/notfair/src/server/goals/pr-sync.ts@@ -3,6 +3,7 @@ import { promisify } from "node:util";import {applyGoalPrSync,+ listDuePrSyncs,listOpenGoalPrs,markGoalPrSyncError,type GoalPr,@@ -70,7 +71,14 @@ export function parseGhPrView(payload: unknown): {};}+// Single-flight per PR: the tick sync, a page-view sync, and the sweep+// can all fire around the same moment — the first one wins, the rest+// no-op instead of stacking duplicate gh subprocesses.+const inFlight = new Set<string>();+async function syncOnePr(pr: GoalPr): Promise<GoalPr | null> {+ if (inFlight.has(pr.id)) return null;+ inFlight.add(pr.id);try {const { stdout } = await execFileAsync("gh",@@ -86,6 +94,8 @@ async function syncOnePr(pr: GoalPr): Promise<GoalPr | null> {} catch (err) {markGoalPrSyncError(pr.id, err instanceof Error ? err.message : String(err));return null;+ } finally {+ inFlight.delete(pr.id);}}@@ -112,3 +122,27 @@ export function maybeSyncGoalPrs(goal_id: string, maxAgeMs = 120_000): void {console.error(`[pr-sync] background sync failed for goal ${goal_id}:`, err),);}++// The centralized sweep never overlaps itself: with many due PRs and a+// slow gh, one pass could outlive the 30s scheduler interval.+let sweepRunning = false;++/**+ * The centralized freshness pass, called from the 30s scheduler sweep.+ * One cheap SQL lists every open PR whose own `next_sync_at` has arrived+ * (the adaptive schedule stamped by the db layer on each sync); only+ * those hit GitHub. Returns how many PRs were checked.+ */+export async function syncDueGoalPrs(): Promise<number> {+ if (sweepRunning) return 0;+ sweepRunning = true;+ try {+ const due = listDuePrSyncs();+ for (const pr of due) {+ await syncOnePr(pr);+ }+ return due.length;+ } finally {+ sweepRunning = false;+ }+}diff --git a/notfair/src/server/scheduler/tick.ts b/notfair/src/server/scheduler/tick.tsindex 2e58e819..05e527d3 100644--- a/notfair/src/server/scheduler/tick.ts+++ b/notfair/src/server/scheduler/tick.ts@@ -1,29 +1,36 @@import { runDueGoalTicks } from "@/server/goals/tick";+import { syncDueGoalPrs } from "@/server/goals/pr-sync";/*** The heartbeat loop. Started once on boot (src/instrumentation.ts); a- * 30s interval sweeps active goals whose next_tick_at has passed and- * runs their ticks. This is the ONLY scheduler in the product — goals- * are the one thing that recurs.+ * 30s interval runs two due-work checks — both cheap local SQL reads+ * that only spawn real work when a per-row timestamp has arrived:+ *+ * - goal ticks (goals.next_tick_at — the goal's agreed cadence)+ * - open-PR freshness (goal_prs.next_sync_at — the adaptive poll+ * schedule from pr-poll-policy; GitHub is only called for due PRs)+ *+ * This is the ONLY scheduler in the product.*/let started = false;let timer: NodeJS.Timeout | null = null;const TICK_INTERVAL_MS = 30_000;+function sweep(): void {+ runDueGoalTicks().catch((err) =>+ console.error("[scheduler] goal tick sweep failed:", err),+ );+ syncDueGoalPrs().catch((err) =>+ console.error("[scheduler] PR freshness sweep failed:", err),+ );+}+export function ensureSchedulerRunning(): void {if (started) return;started = true;- timer = setInterval(() => {- runDueGoalTicks().catch((err) =>- console.error("[scheduler] goal tick sweep failed:", err),- );- }, TICK_INTERVAL_MS);+ timer = setInterval(sweep, TICK_INTERVAL_MS);// First sweep on the next event-loop turn so callers return immediately.- setImmediate(() =>- runDueGoalTicks().catch((err) =>- console.error("[scheduler] goal tick sweep failed:", err),- ),- );+ setImmediate(sweep);}export function stopScheduler(): void {