Open source alternative to Semrush and Ahrefs
TypeScript backlink-analysisgoogle-search-console-mcpkeyword-researchmcpseoseo-agentseo-auditseo-automationseo-skillsseo-toolssite-audit
Collect live answers when a user runs an AI visibility check (EVE-401) (#886)
Carter Smith9a0be3f8d072
diff --git a/.agents/skills/ai-visibility-audit/SKILL.md b/.agents/skills/ai-visibility-audit/SKILL.mdindex 99a5c1ae..5c109292 100644--- a/.agents/skills/ai-visibility-audit/SKILL.md+++ b/.agents/skills/ai-visibility-audit/SKILL.md@@ -37,9 +37,9 @@ The project-context tools are free and shared with the app and other agents.- **Tracker with a completed run**: use the latest completed baseline or scheduled run, or the latest completed manual check when `recentRuns` has no other. Do not buy new answers.- **Run in progress**: when `recentRuns` shows a pending or partially finished run, follow it with `get_ai_visibility_run`, respecting `pollAfterSeconds`. Do not start another. A failed or partial run still supplies whatever answers it completed; report its coverage.-- **Tracker with active prompts but no runs**: saved prompts that were never collected, for example because the schedule was never enabled. Estimate one check of the saved active prompts with `estimate_ai_visibility_cost`, get approval once, then call `run_ai_visibility_check` with the approved cost as `maxCostUsd`. Follow the returned run. Do not change the schedule.+- **Tracker with active prompts but no runs**: saved prompts that were never collected, for example because the schedule was never enabled. Estimate one check of the saved active prompts with `estimate_ai_visibility_cost`, get approval once, then call `run_ai_visibility_check` with the approved `runNowCostUsd` as `maxCostUsd`. Follow the returned run. Do not change the schedule.- **Tracker with no active prompts** (every topic or prompt paused or archived): ask whether to resume the existing prompts or add new ones, then take the matching path. Do not unpause anything without the user's direction.-- **No tracker**: propose 10–15 neutral prompts across two or three topics the business sells into, preferring real prompts from `research_ai_visibility_prompts`. Show one plan with the check cost and the monthly cost from `estimate_ai_visibility_cost`, and say that a new tracker runs weekly, with its first scheduled check a week later. Get approval once, save through `save_ai_visibility_tracker` with each prompt's `topic`, and run one check with the approved cost as `maxCostUsd`. If the user wants this audit only, pause the schedule with `set_ai_visibility_schedule` and `enabled: false`. If the user declines the spend, run the audit on prompt research alone and say the report has no observed answers.+- **No tracker**: propose 10–15 neutral prompts across two or three topics the business sells into, preferring real prompts from `research_ai_visibility_prompts`. Show one plan with the check cost and the monthly cost from `estimate_ai_visibility_cost`, and say that a new tracker runs weekly, with its first scheduled check a week later. Get approval once, save through `save_ai_visibility_tracker` with each prompt's `topic`, and run one check with the approved `runNowCostUsd` as `maxCostUsd`. If the user wants this audit only, pause the schedule with `set_ai_visibility_schedule` and `enabled: false`. If the user declines the spend, run the audit on prompt research alone and say the report has no observed answers.### 2. Read where the brand standsdiff --git a/docs/maintainers/specs/0002-hosted-dataforseo-metering-with-autumn.md b/docs/maintainers/specs/0002-hosted-dataforseo-metering-with-autumn.mdindex e9201681..f37b9842 100644--- a/docs/maintainers/specs/0002-hosted-dataforseo-metering-with-autumn.md+++ b/docs/maintainers/specs/0002-hosted-dataforseo-metering-with-autumn.md@@ -33,7 +33,7 @@ Estimate and deduction use one formula (`creditsForProviderUsd`), so they cannotA live rank check batch shares its holds instead of taking one per call. It takes at most one hold on each balance, and each call goes where a hold of its own would go: in order, on `usage_credits` while they cover it, else on `topup_credits`, else that call alone is refused. Each hold is settled once, with only its own calls. Every call is still converted to credits on its own, so a batch costs exactly what the same calls would cost one at a time.-AI visibility tracking posts each batch of prompts through the client like queued rank checks; collecting the answers is free. Admitted website/setup research instead checks for a positive balance once, finishes all research without another credit gate, and settles actual usage through `billResearchSpend`, allowing a negative balance; it owns metering for its raw provider calls.+Scheduled AI visibility checks post each batch of prompts through the client like queued rank checks; collecting the answers is free. A manual AI visibility check collects live answers instead, and each batch of live calls shares one hold and one settle, like a manual rank check. Admitted website/setup research instead checks for a positive balance once, finishes all research without another credit gate, and settles actual usage through `billResearchSpend`, allowing a negative balance; it owns metering for its raw provider calls.In non-hosted mode, the client skips Autumn and executes the DataForSEO call directly.diff --git a/plugins/openseo/skills/ai-visibility-audit/SKILL.md b/plugins/openseo/skills/ai-visibility-audit/SKILL.mdindex 99a5c1ae..5c109292 100644--- a/plugins/openseo/skills/ai-visibility-audit/SKILL.md+++ b/plugins/openseo/skills/ai-visibility-audit/SKILL.md@@ -37,9 +37,9 @@ The project-context tools are free and shared with the app and other agents.- **Tracker with a completed run**: use the latest completed baseline or scheduled run, or the latest completed manual check when `recentRuns` has no other. Do not buy new answers.- **Run in progress**: when `recentRuns` shows a pending or partially finished run, follow it with `get_ai_visibility_run`, respecting `pollAfterSeconds`. Do not start another. A failed or partial run still supplies whatever answers it completed; report its coverage.-- **Tracker with active prompts but no runs**: saved prompts that were never collected, for example because the schedule was never enabled. Estimate one check of the saved active prompts with `estimate_ai_visibility_cost`, get approval once, then call `run_ai_visibility_check` with the approved cost as `maxCostUsd`. Follow the returned run. Do not change the schedule.+- **Tracker with active prompts but no runs**: saved prompts that were never collected, for example because the schedule was never enabled. Estimate one check of the saved active prompts with `estimate_ai_visibility_cost`, get approval once, then call `run_ai_visibility_check` with the approved `runNowCostUsd` as `maxCostUsd`. Follow the returned run. Do not change the schedule.- **Tracker with no active prompts** (every topic or prompt paused or archived): ask whether to resume the existing prompts or add new ones, then take the matching path. Do not unpause anything without the user's direction.-- **No tracker**: propose 10–15 neutral prompts across two or three topics the business sells into, preferring real prompts from `research_ai_visibility_prompts`. Show one plan with the check cost and the monthly cost from `estimate_ai_visibility_cost`, and say that a new tracker runs weekly, with its first scheduled check a week later. Get approval once, save through `save_ai_visibility_tracker` with each prompt's `topic`, and run one check with the approved cost as `maxCostUsd`. If the user wants this audit only, pause the schedule with `set_ai_visibility_schedule` and `enabled: false`. If the user declines the spend, run the audit on prompt research alone and say the report has no observed answers.+- **No tracker**: propose 10–15 neutral prompts across two or three topics the business sells into, preferring real prompts from `research_ai_visibility_prompts`. Show one plan with the check cost and the monthly cost from `estimate_ai_visibility_cost`, and say that a new tracker runs weekly, with its first scheduled check a week later. Get approval once, save through `save_ai_visibility_tracker` with each prompt's `topic`, and run one check with the approved `runNowCostUsd` as `maxCostUsd`. If the user wants this audit only, pause the schedule with `set_ai_visibility_schedule` and `enabled: false`. If the user declines the spend, run the audit on prompt research alone and say the report has no observed answers.### 2. Read where the brand standsdiff --git a/src/client/features/ai-visibility/TrackingCostReview.tsx b/src/client/features/ai-visibility/TrackingCostReview.tsxindex cde9ec6c..2b9a8347 100644--- a/src/client/features/ai-visibility/TrackingCostReview.tsx+++ b/src/client/features/ai-visibility/TrackingCostReview.tsx@@ -120,6 +120,9 @@ export function TrackingCostReview({placeholderData: keepPreviousData,});const cost = estimate.data;+ // Run now collects live answers; scheduled runs use the cheaper queue.+ const runCostUsd =+ (mode === "check" ? cost?.runNowCostUsd : cost?.costUsd) ?? 0;const start = useMutation({mutationFn: async () => {if (!cost) throw new Error("Review the estimate before starting.");@@ -143,7 +146,7 @@ export function TrackingCostReview({return runAiVisibilityCheck({data: {projectId,- maxCostUsd: cost.costUsd,+ maxCostUsd: cost.runNowCostUsd,promptIds,},});@@ -220,10 +223,10 @@ export function TrackingCostReview({{mode === "schedule" ? "Cost per run" : "Cost"}</FieldLabel><span className="text-2xl font-semibold tabular-nums">- {aiMoney(cost.costUsd)}+ {aiMoney(runCostUsd)}</span><FieldDescription>- {`${cost.observations} ${cost.observations === 1 ? "answer" : "answers"} × ${aiMoney(cost.observations ? cost.costUsd / cost.observations : 0)}${mode === "schedule" ? ` · about ${aiMoney(cost.monthlyCostUsd)} a month (${cost.checksPerMonth} ${cost.checksPerMonth === 1 ? "run" : "runs"})` : ""}`}+ {`${cost.observations} ${cost.observations === 1 ? "answer" : "answers"} × ${aiMoney(cost.observations ? runCostUsd / cost.observations : 0)}${mode === "schedule" ? ` · about ${aiMoney(cost.monthlyCostUsd)} a month (${cost.checksPerMonth} ${cost.checksPerMonth === 1 ? "run" : "runs"})` : " · live answers, ready in a few minutes"}`}</FieldDescription></Field>{start.error && <AiQueryError error={start.error} />}diff --git a/src/server/features/ai-visibility/services/aiVisibilityCollection.test.ts b/src/server/features/ai-visibility/services/aiVisibilityCollection.test.tsindex d96f90d0..850a7538 100644--- a/src/server/features/ai-visibility/services/aiVisibilityCollection.test.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityCollection.test.ts@@ -2,9 +2,11 @@ import { beforeEach, describe, expect, it, vi } from "vitest";import { projectCompetitors } from "@/db/schema";import { AiVisibilityRepository as repo } from "../repositories/AiVisibilityRepository";import {+ collectAiLiveBatch,collectAiRound,finalizeAiRun,postAiBatch,+ prepareAiRun,} from "./aiVisibilityCollection";import { configuration, customer } from "./aiVisibilityTestFixtures";@@ -12,7 +14,11 @@ const { testDb, dataforseo } = await vi.hoisted(async () => {const { createAiVisibilityTestDb } = await import("../aiVisibilityTestDb");return {testDb: await createAiVisibilityTestDb(),- dataforseo: { trackingTaskPost: vi.fn(), fetchTaskResult: vi.fn() },+ dataforseo: {+ trackingTaskPost: vi.fn(),+ trackingLiveBatch: vi.fn(),+ fetchTaskResult: vi.fn(),+ },};});vi.mock("cloudflare:workers", () => ({ env: { DATABASE_PROVIDER: "d1" } }));@@ -24,7 +30,10 @@ vi.mock("@/server/lib/runtime-env", () => ({vi.mock("@/server/lib/dataforseo", () => ({MAX_TASKS_PER_POST: 100,createDataforseoClient: () => ({- aiSearch: { trackingTaskPost: dataforseo.trackingTaskPost },+ aiSearch: {+ trackingTaskPost: dataforseo.trackingTaskPost,+ trackingLiveBatch: dataforseo.trackingLiveBatch,+ },}),fetchAiTrackingTaskResult: dataforseo.fetchTaskResult,}));@@ -135,6 +144,58 @@ describe("AI answer collection", () => {});});+ it("collects a manual run live, saving answers and failing rejected calls", async () => {+ expect(await prepareAiRun(runId, customer)).toMatchObject({ live: true });+ dataforseo.trackingLiveBatch.mockResolvedValue([+ {+ status: "fulfilled",+ value: {+ status: "completed",+ result: { markdown: "OpenSEO", sources: [] },+ },+ },+ { status: "rejected", reason: new Error("Timed out") },+ ]);++ await collectAiLiveBatch(runId, customer, market, [+ { tag: "answer-0", prompt: "Which SEO tools?", engine: "chatgpt" },+ { tag: "answer-1", prompt: "Best rank tracker?", engine: "chatgpt" },+ ]);++ expect(await repo.getObservation("answer-0")).toMatchObject({+ status: "completed",+ });+ expect(await repo.getObservation("answer-1")).toMatchObject({+ status: "failed",+ error: "Timed out",+ });+ });++ it("saves the rest of a paid live batch when one answer cannot be saved", async () => {+ const answer = {+ status: "fulfilled",+ value: {+ status: "completed",+ result: { markdown: "OpenSEO", sources: [] },+ },+ };+ dataforseo.trackingLiveBatch.mockResolvedValue([answer, answer]);+ vi.spyOn(repo, "persistAnswer").mockRejectedValueOnce(new Error("D1"));+ vi.spyOn(console, "error").mockImplementation(() => {});++ await collectAiLiveBatch(runId, customer, market, [+ { tag: "answer-0", prompt: "Which SEO tools?", engine: "chatgpt" },+ { tag: "answer-1", prompt: "Best rank tracker?", engine: "chatgpt" },+ ]);++ expect(await repo.getObservation("answer-0")).toMatchObject({+ status: "pending",+ });+ expect(await repo.getObservation("answer-1")).toMatchObject({+ status: "completed",+ });+ });+it("fails answers that never arrived and finishes the run as partial", async () => {dataforseo.fetchTaskResult.mockResolvedValue({status: "completed",diff --git a/src/server/features/ai-visibility/services/aiVisibilityCollection.ts b/src/server/features/ai-visibility/services/aiVisibilityCollection.tsindex 931dcf88..fe94a8e4 100644--- a/src/server/features/ai-visibility/services/aiVisibilityCollection.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityCollection.ts@@ -4,11 +4,12 @@ import {createDataforseoClient,fetchAiTrackingTaskResult,MAX_TASKS_PER_POST,+ type AiTrackingAnswer,type PostedAiTrackingTask,} from "@/server/lib/dataforseo";import { AppError } from "@/server/lib/errors";import { isHostedServerAuthMode } from "@/server/lib/runtime-env";-import type { AiEngine } from "@/shared/ai-visibility";+import type { AiBrand, AiEngine } from "@/shared/ai-visibility";import { AiVisibilityRepository as repo } from "../repositories/AiVisibilityRepository";import { parseDataforseoAnswer } from "../providers/dataforseoEvidence";import { aiBrands } from "./aiVisibilityConfiguration";@@ -24,6 +25,12 @@ interface AiTaskBatch {/** Tag = answer row id. At most 100 per task_post. */tasks: { tag: string; prompt: string }[];}+/** One live answer. A live step mixes engines, so each task names its own. */+interface AiLiveTask {+ tag: string;+ prompt: string;+ engine: AiEngine;+}export type AiPendingTask = PostedAiTrackingTask & { engine: AiEngine };interface AiRunMarket {locationCode: number;@@ -31,21 +38,39 @@ interface AiRunMarket {}/**- * Marks the run running and groups its answers into task_post batches. Hosted- * runs first check that credits cover the whole check, like rank checks.+ * Live answers per workflow step. A Worker holds 6 connections open at once,+ * so a larger step only queues calls behind the first 6.+ */+const LIVE_ANSWERS_PER_STEP = 6;++/**+ * Marks the run running and groups its answers into batches. Manual runs+ * collect live answers so the user sees results sooner. Scheduled and+ * baseline runs use the cheaper standard queue. Hosted runs first check that+ * credits cover the whole check, like rank checks.*/export async function prepareAiRun(runId: string,customer: BillingCustomerContext,-): Promise<{ market: AiRunMarket; batches: AiTaskBatch[] }> {+): Promise<+ { market: AiRunMarket } & (+ | { live: false; batches: AiTaskBatch[] }+ | { live: true; batches: AiLiveTask[][] }+ )+> {const run = await repo.getRunInternal(runId);if (!run || (run.status !== "queued" && run.status !== "running"))throw new AppError("NOT_FOUND", `Run ${runId} is no longer active.`);const answers = (await repo.getObservations([runId])).filter((row) => row.status === "pending",);+ const live = run.trigger === "manual";if (await isHostedServerAuthMode()) {- const required = aiCostForCount(answers.length, true).costCredits;+ const required = aiCostForCount(+ answers.length,+ true,+ live ? "live" : "queued",+ ).costCredits;const credits = await getUsageCreditsRemaining(customer.organizationId);if (credits.monthlyRemaining + credits.topupRemaining < required)throw new AppError(@@ -54,6 +79,21 @@ export async function prepareAiRun();}await repo.updateRun(runId, { status: "running" });+ const market = {+ locationCode: run.locationCode,+ languageCode: run.languageCode,+ };+ if (live) {+ const tasks = answers.map((row) => ({+ tag: row.id,+ prompt: row.prompt,+ engine: row.engine,+ }));+ const batches: AiLiveTask[][] = [];+ for (let i = 0; i < tasks.length; i += LIVE_ANSWERS_PER_STEP)+ batches.push(tasks.slice(i, i + LIVE_ANSWERS_PER_STEP));+ return { market, live, batches };+ }const batches: AiTaskBatch[] = [];for (const engine of new Set(answers.map((row) => row.engine))) {const tasks = answers@@ -62,10 +102,7 @@ export async function prepareAiRun(for (let i = 0; i < tasks.length; i += MAX_TASKS_PER_POST)batches.push({ engine, tasks: tasks.slice(i, i + MAX_TASKS_PER_POST) });}- return {- market: { locationCode: run.locationCode, languageCode: run.languageCode },- batches,- };+ return { market, live, batches };}/**@@ -98,6 +135,51 @@ export async function postAiBatch(return posted.map((task) => ({ ...task, engine: batch.engine }));}+/**+ * Collects one batch of live answers through the metered client, which bills+ * them, and saves each one. Answers that fail now are not retried.+ */+export async function collectAiLiveBatch(+ runId: string,+ customer: BillingCustomerContext,+ market: AiRunMarket,+ tasks: AiLiveTask[],+) {+ const brands = await runBrands(runId);+ if (!brands) return;+ let settled: PromiseSettledResult<AiTrackingAnswer>[];+ try {+ settled = await createDataforseoClient(customer).aiSearch.trackingLiveBatch(+ tasks.map((task) => ({ engine: task.engine, task, ...market })),+ );+ } catch (error) {+ settled = tasks.map(() => ({ status: "rejected", reason: error }));+ }+ for (const [index, outcome] of settled.entries()) {+ const result: AiTrackingAnswer =+ outcome.status === "fulfilled"+ ? outcome.value+ : {+ status: "failed",+ message:+ outcome.reason instanceof Error+ ? outcome.reason.message+ : String(outcome.reason),+ };+ // Every answer here is already paid for, and the step cannot rerun, so+ // one failed write must not drop the rest. finalizeAiRun fails the+ // answer that stays pending.+ try {+ await saveAiOutcome(runId, tasks[index], result, brands);+ } catch (error) {+ console.error(+ `[ai-visibility] ${runId} could not save answer ${tasks[index].tag}:`,+ error,+ );+ }+ }+}+/** Concurrent task_get requests within a collect step. */const TASK_GET_CONCURRENCY = 25;@@ -109,11 +191,8 @@ export async function collectAiRound(runId: string,tasks: AiPendingTask[],): Promise<AiPendingTask[]> {- const run = await repo.getRunInternal(runId);- const project = run ? await repo.getProject(run.projectId) : null;- if (!run || !project) return [];- // Brands come from the project as each answer is matched.- const brands = aiBrands(project, await repo.listCompetitors(project.id));+ const brands = await runBrands(runId);+ if (!brands) return [];const stillPending: AiPendingTask[] = [];for (let i = 0; i < tasks.length; i += TASK_GET_CONCURRENCY) {const chunk = tasks.slice(i, i + TASK_GET_CONCURRENCY);@@ -127,46 +206,62 @@ export async function collectAiRound(stillPending.push(task);continue;}- if (outcome.value.status === "failed") {- await repo.failPendingObservations(runId, outcome.value.message, [- task.tag,- ]);- continue;- }- const answer = parseDataforseoAnswer(outcome.value.result, task.engine);- if (!answer) {- await repo.failPendingObservations(- runId,- "DataForSEO returned an answer we could not read.",- [task.tag],- );- continue;- }- await repo.persistAnswer({- observationId: task.tag,- values: {- status: "completed",- collectedAt: answer.collectedAt ?? new Date().toISOString(),- answerMarkdown: answer.answerMarkdown,- error: null,- },- sources: answer.citations.map((citation) => ({- id: crypto.randomUUID(),- observationId: task.tag,- ...citation,- })),- matches: brands.map((brand) => ({- id: crypto.randomUUID(),- observationId: task.tag,- ...brand,- ...matchAiBrand(answer, brand),- })),- });+ await saveAiOutcome(runId, task, outcome.value, brands);}}return stillPending;}+/** Brands come from the project as each answer is matched. */+async function runBrands(runId: string) {+ const run = await repo.getRunInternal(runId);+ const project = run ? await repo.getProject(run.projectId) : null;+ if (!run || !project) return null;+ return aiBrands(project, await repo.listCompetitors(project.id));+}++/** Saves a finished answer with its citations and brand results, or fails it. */+async function saveAiOutcome(+ runId: string,+ task: { tag: string; engine: AiEngine },+ outcome: AiTrackingAnswer,+ brands: AiBrand[],+) {+ if (outcome.status === "failed") {+ await repo.failPendingObservations(runId, outcome.message, [task.tag]);+ return;+ }+ const answer = parseDataforseoAnswer(outcome.result, task.engine);+ if (!answer) {+ await repo.failPendingObservations(+ runId,+ "DataForSEO returned an answer we could not read.",+ [task.tag],+ );+ return;+ }+ await repo.persistAnswer({+ observationId: task.tag,+ values: {+ status: "completed",+ collectedAt: answer.collectedAt ?? new Date().toISOString(),+ answerMarkdown: answer.answerMarkdown,+ error: null,+ },+ sources: answer.citations.map((citation) => ({+ id: crypto.randomUUID(),+ observationId: task.tag,+ ...citation,+ })),+ matches: brands.map((brand) => ({+ id: crypto.randomUUID(),+ observationId: task.tag,+ ...brand,+ ...matchAiBrand(answer, brand),+ })),+ });+}+/** Fails answers that never arrived and records how the run finished. */export async function finalizeAiRun(runId: string) {const run = await repo.getRunInternal(runId);diff --git a/src/server/features/ai-visibility/services/aiVisibilityCost.test.ts b/src/server/features/ai-visibility/services/aiVisibilityCost.test.tsindex be5ac3d9..8f8c32dd 100644--- a/src/server/features/ai-visibility/services/aiVisibilityCost.test.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityCost.test.ts@@ -8,19 +8,22 @@ vi.mock("./aiVisibilityMutation", () => ({}));// Real pricing functions and constants: only unused IO dependencies are mocked.describe("AI collection estimates", () => {it("quotes hosted markup and credit rounding per provider call", () => {- expect(aiCostForCount(12, true)).toEqual({+ expect(aiCostForCount(12, true, "queued")).toEqual({providerCostUsd: 0.0144,costCredits: 24,costUsd: 0.024,});- expect(aiCostForCount(0, true)).toEqual({+ expect(aiCostForCount(0, true, "queued")).toEqual({providerCostUsd: 0,costCredits: 0,costUsd: 0,});});+ it("quotes live answers for manual runs at the live price", () => {+ expect(aiCostForCount(3, false, "live").providerCostUsd).toBe(0.012);+ });it("quotes self-hosted provider cost without markup or credits", () => {- expect(aiCostForCount(3, false)).toEqual({+ expect(aiCostForCount(3, false, "queued")).toEqual({providerCostUsd: 0.0036,costCredits: 0,costUsd: 0.0036,diff --git a/src/server/features/ai-visibility/services/aiVisibilityCost.ts b/src/server/features/ai-visibility/services/aiVisibilityCost.tsindex b78ee3cd..1669a7bf 100644--- a/src/server/features/ai-visibility/services/aiVisibilityCost.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityCost.ts@@ -6,6 +6,7 @@ import {isHostedServerAuthMode,} from "@/server/lib/runtime-env";import {+ AI_LIVE_RECORD_COST_USD,AI_RECORD_COST_USD,type AiCostEstimate,} from "@/shared/ai-visibility";@@ -17,10 +18,19 @@ import { scheduledChecksPerMonth } from "@/shared/rank-tracking";import type { EstimateAiCostInput } from "@/types/schemas/ai-visibility";import { projectPatch } from "./aiVisibilityMutation";-/** The customer's cost for `count` answers: credits when hosted, else provider USD. */-export function aiCostForCount(count: number, hosted: boolean) {- const providerCostUsd = Math.round(count * AI_RECORD_COST_USD * 1e6) / 1e6;- const costCredits = count * creditsForProviderUsd(AI_RECORD_COST_USD);+/**+ * The customer's cost for `count` answers: credits when hosted, else provider+ * USD. Manual runs collect live answers; scheduled runs use the queue.+ */+export function aiCostForCount(+ count: number,+ hosted: boolean,+ mode: "live" | "queued",+) {+ const answerUsd =+ mode === "live" ? AI_LIVE_RECORD_COST_USD : AI_RECORD_COST_USD;+ const providerCostUsd = Math.round(count * answerUsd * 1e6) / 1e6;+ const costCredits = count * creditsForProviderUsd(answerUsd);return {providerCostUsd,costCredits: hosted ? costCredits : 0,@@ -45,7 +55,8 @@ export async function estimateCost(const scope = aiScope(config, input.promptIds);const observations = scope.prompts.length * scope.engines.length;const hosted = await isHostedServerAuthMode();- const cost = aiCostForCount(observations, hosted);+ const cost = aiCostForCount(observations, hosted, "queued");+ const runNow = aiCostForCount(observations, hosted, "live");const scheduleInterval =input.scheduleInterval ?? current?.tracker.scheduleInterval ?? "weekly";const checksPerMonth = scheduledChecksPerMonth(scheduleInterval);@@ -54,6 +65,8 @@ export async function estimateCost(engineCount: scope.engines.length,observations,...cost,+ runNowCostUsd: runNow.costUsd,+ runNowCostCredits: runNow.costCredits,scheduleInterval,checksPerMonth,monthlyCostUsd: Math.round(cost.costUsd * checksPerMonth * 1e6) / 1e6,diff --git a/src/server/features/ai-visibility/services/aiVisibilityResults.ts b/src/server/features/ai-visibility/services/aiVisibilityResults.tsindex 53f5f3b4..5cc51112 100644--- a/src/server/features/ai-visibility/services/aiVisibilityResults.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityResults.ts@@ -41,7 +41,8 @@ export function aiRunView(pending: observations.length - completed - failed,createdAt: run.createdAt,completedAt: run.completedAt,- pollAfterSeconds: 20,+ // Manual runs collect live answers, which arrive within seconds.+ pollAfterSeconds: run.trigger === "manual" ? 5 : 20,};}diff --git a/src/server/features/ai-visibility/services/aiVisibilityRuns.ts b/src/server/features/ai-visibility/services/aiVisibilityRuns.tsindex edafa6ce..71dc1ebc 100644--- a/src/server/features/ai-visibility/services/aiVisibilityRuns.ts+++ b/src/server/features/ai-visibility/services/aiVisibilityRuns.ts@@ -164,6 +164,7 @@ export async function runCheck(const { costUsd } = aiCostForCount(scope.prompts.length * scope.engines.length,await isHostedServerAuthMode(),+ "live",);if (Math.round(costUsd * 1e6) > Math.round(input.maxCostUsd * 1e6))throw new AiVisibilityError(diff --git a/src/server/lib/dataforseo/ai-tracking.ts b/src/server/lib/dataforseo/ai-tracking.tsindex 913ab776..206a5b71 100644--- a/src/server/lib/dataforseo/ai-tracking.ts+++ b/src/server/lib/dataforseo/ai-tracking.ts@@ -9,8 +9,9 @@ import {import { AppError } from "@/server/lib/errors";import type { AiEngine } from "@/shared/ai-visibility";-// Standard-queue tasks for AI visibility tracking: task_post, then a free-// task_get/advanced by task ID.+// AI visibility tracking endpoints. Scheduled runs use the standard queue:+// task_post, then a free task_get/advanced by task ID. Manual runs use+// live/advanced, one prompt per call.// https://docs.dataforseo.com/v3/ai_optimization/chat_gpt/llm_scraper/overview/// https://docs.dataforseo.com/v3/serp/google/organic/overview/const AI_TRACKING_ENDPOINTS: Record<AiEngine, string> = {@@ -25,6 +26,14 @@ interface AiTrackingTaskInput {prompt: string;}+interface AiTrackingMarket {+ locationCode: number;+ languageCode: string;+}++// Live answers can take up to 120 seconds.+const LIVE_REQUEST_TIMEOUT_MS = 150_000;+export interface PostedAiTrackingTask {tag: string;taskId: string;@@ -43,17 +52,36 @@ function providerLanguageCode(engine: AiEngine, languageCode: string) {return languageCode;}+function aiTrackingTaskBody(+ engine: AiEngine,+ market: AiTrackingMarket,+ task: AiTrackingTaskInput,+) {+ return {+ // DataForSEO decodes %-escapes in keywords; escape literal % and +.+ keyword: task.prompt.replace(/%/g, "%25").replace(/\+/g, "%2B"),+ location_code: market.locationCode,+ language_code: providerLanguageCode(engine, market.languageCode),+ // Loads an overview that Google renders after the page. DataForSEO+ // refunds the extra charge when no overview loads.+ ...(engine === "google_ai_overview"+ ? { depth: 10, load_async_ai_overview: true }+ : {}),+ tag: task.tag,+ };+}+/*** Posts up to 100 prompts for one engine. DataForSEO bills at task_post, so* this is the metered call. Rejected entries come back without a task; the* caller fails their answers.*/-export async function postAiTrackingTasks(input: {- engine: AiEngine;- tasks: AiTrackingTaskInput[];- locationCode: number;- languageCode: string;-}): Promise<DataforseoApiResponse<PostedAiTrackingTask[]>> {+export async function postAiTrackingTasks(+ input: AiTrackingMarket & {+ engine: AiEngine;+ tasks: AiTrackingTaskInput[];+ },+): Promise<DataforseoApiResponse<PostedAiTrackingTask[]>> {if (input.tasks.length === 0 || input.tasks.length > MAX_TASKS_PER_POST) {throw new AppError("INTERNAL_ERROR",@@ -65,18 +93,7 @@ export async function postAiTrackingTasks(input: {DataforseoTaskLike & { id?: string; data?: Record<string, unknown> }>(`${endpoint}/task_post`,- input.tasks.map((task) => ({- // DataForSEO decodes %-escapes in keywords; escape literal % and +.- keyword: task.prompt.replace(/%/g, "%25").replace(/\+/g, "%2B"),- location_code: input.locationCode,- language_code: providerLanguageCode(input.engine, input.languageCode),- // Loads an overview that Google renders after the page. DataForSEO- // refunds the extra charge when no overview loads.- ...(input.engine === "google_ai_overview"- ? { depth: 10, load_async_ai_overview: true }- : {}),- tag: task.tag,- })),+ input.tasks.map((task) => aiTrackingTaskBody(input.engine, input, task)),// A billed task_post must never be replayed on an ambiguous 5xx.{ maxServerErrorRetries: 0 },);@@ -110,11 +127,11 @@ export async function postAiTrackingTasks(input: {};}-type AiTrackingTaskOutcome =- | { status: "pending" }+export type AiTrackingAnswer =| { status: "failed"; message: string }/** `result` is null when Google returned no results page. */| { status: "completed"; result: unknown };+type AiTrackingTaskOutcome = { status: "pending" } | AiTrackingAnswer;/*** Collects one queued answer. Not metered, like fetchRankCheckTaskResult: the@@ -135,6 +152,12 @@ export async function fetchAiTrackingTaskResult(input: {);}if (isTaskInProgress(task)) return { status: "pending" };+ return aiTrackingAnswer(task);+}++function aiTrackingAnswer(+ task: DataforseoTaskLike & { result?: unknown[] | null },+): AiTrackingAnswer {if (task.status_code !== 20000) {if (isNoResultsTask(task)) return { status: "completed", result: null };return {@@ -145,3 +168,35 @@ export async function fetchAiTrackingTaskResult(input: {}return { status: "completed", result: task.result?.[0] ?? null };}++/**+ * Fetches one answer from the live endpoint. DataForSEO bills each live call,+ * so this is the metered call. A failed task returns a failed outcome with+ * its charge, so the caller fails only that answer.+ */+export async function fetchAiTrackingLiveAnswer(+ input: AiTrackingMarket & {+ engine: AiEngine;+ task: AiTrackingTaskInput;+ },+): Promise<DataforseoApiResponse<AiTrackingAnswer>> {+ const endpoint = `${AI_TRACKING_ENDPOINTS[input.engine]}/live/advanced`;+ const response = await dataforseoPost<+ DataforseoTaskLike & { result?: unknown[] | null }+ >(endpoint, [aiTrackingTaskBody(input.engine, input, input.task)], {+ // A billed live call must never be replayed on an ambiguous 5xx.+ maxServerErrorRetries: 0,+ signal: AbortSignal.timeout(LIVE_REQUEST_TIMEOUT_MS),+ });+ const task = response?.tasks?.[0];+ if (!response || response.status_code !== 20000 || !task) {+ throw new AppError(+ "INTERNAL_ERROR",+ response?.status_message || "DataForSEO live request failed",+ );+ }+ return {+ data: aiTrackingAnswer(task),+ billing: { path: endpoint.slice(1).split("/"), costUsd: task.cost ?? 0 },+ };+}diff --git a/src/server/lib/dataforseo/client.ts b/src/server/lib/dataforseo/client.tsindex 110f5618..f75d05b7 100644--- a/src/server/lib/dataforseo/client.ts+++ b/src/server/lib/dataforseo/client.ts@@ -59,7 +59,10 @@ import {fetchLlmResponse,fetchLlmTopPages,} from "@/server/lib/dataforseo/ai";-import { postAiTrackingTasks } from "@/server/lib/dataforseo/ai-tracking";+import {+ fetchAiTrackingLiveAnswer,+ postAiTrackingTasks,+} from "@/server/lib/dataforseo/ai-tracking";import { isHostedServerAuthMode } from "@/server/lib/runtime-env";import { AppError } from "@/server/lib/errors";@@ -296,6 +299,21 @@ export function createDataforseoClient(customer: BillingCustomerContext) {dataforseoPricing.aiSearch.trackingTaskPost,"ai_prompt_responses",),+ // A manual run's live answers share one credit hold and one settle (see+ // meterDataforseoCalls). Resolves per call, in order.+ trackingLiveBatch: (+ inputs: Parameters<typeof fetchAiTrackingLiveAnswer>[0][],+ ) =>+ meterDataforseoCalls(+ customer,+ inputs.map((input) => () => fetchAiTrackingLiveAnswer(input)),+ inputs.map((input) =>+ creditsForProviderUsd(+ dataforseoPricing.aiSearch.trackingLive(input),+ ),+ ),+ "ai_prompt_responses",+ ),},} as const;}diff --git a/src/server/lib/dataforseo/index.ts b/src/server/lib/dataforseo/index.tsindex d96971f7..01fa194f 100644--- a/src/server/lib/dataforseo/index.ts+++ b/src/server/lib/dataforseo/index.ts@@ -32,6 +32,7 @@ export {} from "@/server/lib/dataforseo/serp";export {fetchAiTrackingTaskResult,+ type AiTrackingAnswer,type PostedAiTrackingTask,} from "@/server/lib/dataforseo/ai-tracking";export {diff --git a/src/server/lib/dataforseo/pricing.ts b/src/server/lib/dataforseo/pricing.tsindex 381e602b..a20b4287 100644--- a/src/server/lib/dataforseo/pricing.ts+++ b/src/server/lib/dataforseo/pricing.ts@@ -47,8 +47,14 @@ import {resolveLlmMentionsLimit,} from "@/server/lib/dataforseo/ai";import type { LlmResponseModelSlug } from "@/server/lib/dataforseo/llm-models";-import { postAiTrackingTasks } from "@/server/lib/dataforseo/ai-tracking";-import { AI_RECORD_COST_USD } from "@/shared/ai-visibility";+import {+ fetchAiTrackingLiveAnswer,+ postAiTrackingTasks,+} from "@/server/lib/dataforseo/ai-tracking";+import {+ AI_LIVE_RECORD_COST_USD,+ AI_RECORD_COST_USD,+} from "@/shared/ai-visibility";import {costPerSerpAtDepth,serpKeywordCostMultiplier,@@ -296,6 +302,11 @@ export const dataforseoPricing = {postAiTrackingTasks,(input) => input.tasks.length * AI_RECORD_COST_USD,),+ // One live answer, on every tracked engine.+ trackingLive: priced(+ fetchAiTrackingLiveAnswer,+ () => AI_LIVE_RECORD_COST_USD,+ ),llmResponse: priced(fetchLlmResponse, (input) => {const price = LLM_RESPONSE_USD[input.modelSlug];return (input.webSearch ?? LLM_RESPONSE_WEB_SEARCH_DEFAULT)diff --git a/src/server/mcp/tools/ai-visibility-output-schemas.ts b/src/server/mcp/tools/ai-visibility-output-schemas.tsindex 7ada39c1..eeadd376 100644--- a/src/server/mcp/tools/ai-visibility-output-schemas.ts+++ b/src/server/mcp/tools/ai-visibility-output-schemas.ts@@ -70,6 +70,8 @@ export const aiCostOutput = z.looseObject({providerCostUsd: money,costUsd: money,costCredits: money,+ runNowCostUsd: money,+ runNowCostCredits: money,scheduleInterval: text,checksPerMonth: count,monthlyCostUsd: money,diff --git a/src/server/mcp/tools/ai-visibility-tools.ts b/src/server/mcp/tools/ai-visibility-tools.tsindex f1b92335..95c97959 100644--- a/src/server/mcp/tools/ai-visibility-tools.ts+++ b/src/server/mcp/tools/ai-visibility-tools.ts@@ -230,10 +230,10 @@ export const estimateAiVisibilityCostTool = defineAiTool({output: aiCostOutput,readOnly: true,description:- "Estimate one check and the monthly cost of a daily, weekly, or monthly schedule without collecting answers. Optional patch estimates proposed settings; promptIds estimates a check of those prompts. Show the customer cost and monthly estimate before enabling tracking or running a paid check, and pass the approved check cost to run_ai_visibility_check as maxCostUsd.",+ "Estimate one scheduled check, one-off check, and the monthly cost of a daily, weekly, or monthly schedule without collecting answers. Optional patch estimates proposed settings; promptIds estimates a check of those prompts. costUsd prices each scheduled check. runNowCostUsd prices a one-off check, which collects live answers within minutes at a higher price. Show the customer the relevant cost before enabling tracking or running a paid check, and pass the approved runNowCostUsd to run_ai_visibility_check as maxCostUsd.",execute: (s, a) => s.estimateCost(a),summarize: (d) =>- `${d.observations} answers: $${d.costUsd.toFixed(4)} (${d.costCredits} credits) per check. Estimate on the ${d.scheduleInterval} schedule: $${d.monthlyCostUsd.toFixed(2)}/month (${d.checksPerMonth} ${d.checksPerMonth === 1 ? "check" : "checks"}).${d.warnings.length ? ` Warnings: ${d.warnings.join("; ")}` : ""}`,+ `${d.observations} answers: $${d.costUsd.toFixed(4)} (${d.costCredits} credits) per scheduled check, $${d.runNowCostUsd.toFixed(4)} (${d.runNowCostCredits} credits) to run now. Estimate on the ${d.scheduleInterval} schedule: $${d.monthlyCostUsd.toFixed(2)}/month (${d.checksPerMonth} ${d.checksPerMonth === 1 ? "check" : "checks"}).${d.warnings.length ? ` Warnings: ${d.warnings.join("; ")}` : ""}`,});export const setAiVisibilityScheduleTool = defineAiTool({name: "set_ai_visibility_schedule",@@ -257,7 +257,7 @@ export const runAiVisibilityCheckTool = defineAiTool({output: aiRunOutput,readOnly: false,description:- "Start an explicitly approved one-off paid AI check, preserving the schedule. maxCostUsd is the cost the user approved from estimate_ai_visibility_cost; the check is refused if it now costs more. Only one check runs per project at a time. Returns a run immediately. Never call this tool to poll: use get_ai_visibility_run with the returned run ID.",+ "Start an explicitly approved one-off paid AI check, preserving the schedule. It collects live answers, so results arrive within minutes. maxCostUsd is the runNowCostUsd the user approved from estimate_ai_visibility_cost; the check is refused if it now costs more. Only one check runs per project at a time. Returns a run immediately. Never call this tool to poll: use get_ai_visibility_run with the returned run ID.",execute: (s, a, b) => s.runCheck(a, b),summarize: (d) =>`Run ${d.id}: ${d.status}, ${d.completed}/${d.expected} completed. ${runNextAction(d)}`,diff --git a/src/server/workflows/AiVisibilityWorkflow.test.ts b/src/server/workflows/AiVisibilityWorkflow.test.tsindex c1ec48c6..e143aab5 100644--- a/src/server/workflows/AiVisibilityWorkflow.test.ts+++ b/src/server/workflows/AiVisibilityWorkflow.test.ts@@ -5,6 +5,7 @@ import { AiVisibilityWorkflow } from "./AiVisibilityWorkflow";const mocks = vi.hoisted(() => ({prepare: vi.fn(),post: vi.fn(),+ live: vi.fn(),collect: vi.fn(),finalize: vi.fn(),markFailed: vi.fn(),@@ -20,6 +21,7 @@ vi.mock(() => ({prepareAiRun: mocks.prepare,postAiBatch: mocks.post,+ collectAiLiveBatch: mocks.live,collectAiRound: mocks.collect,finalizeAiRun: mocks.finalize,markAiRunFailed: mocks.markFailed,@@ -61,6 +63,7 @@ function execute() {beforeEach(() => {mocks.prepare.mockResolvedValue({market: { locationCode: 2840, languageCode: "en" },+ live: false,batches: [{ engine: "chatgpt", tasks: [] },{ engine: "gemini", tasks: [] },@@ -83,6 +86,19 @@ describe("AI visibility workflow", () => {expect(mocks.finalize).toHaveBeenCalledExactlyOnceWith("run");});+ it("collects a manual run's live batches without posting or polling", async () => {+ mocks.prepare.mockResolvedValue({+ market: { locationCode: 2840, languageCode: "en" },+ live: true,+ batches: [[]],+ });+ await execute();+ expect(mocks.live).toHaveBeenCalledOnce();+ expect(mocks.post).not.toHaveBeenCalled();+ expect(mocks.sleep).not.toHaveBeenCalled();+ expect(mocks.finalize).toHaveBeenCalledExactlyOnceWith("run");+ });+it("stops polling when the collection window closes and finalizes what arrived", async () => {mocks.collect.mockResolvedValue([task]);await execute();diff --git a/src/server/workflows/AiVisibilityWorkflow.ts b/src/server/workflows/AiVisibilityWorkflow.tsindex 636a1d3b..36b8491c 100644--- a/src/server/workflows/AiVisibilityWorkflow.ts+++ b/src/server/workflows/AiVisibilityWorkflow.ts@@ -5,6 +5,7 @@ import {} from "cloudflare:workers";import { pgStep } from "./pgStep";import {+ collectAiLiveBatch,collectAiRound,finalizeAiRun,markAiRunFailed,@@ -35,6 +36,9 @@ const COLLECT_STEP = {retries: { limit: 2, delay: "10 seconds" as const },timeout: "5 minutes" as const,};+// Live answers are billed per call, so a live step runs once too. It waits for+// up to 6 answers, each up to 120 seconds.+const LIVE_STEP = { ...SINGLE_ATTEMPT, timeout: "5 minutes" as const };// Standard-queue answers usually arrive within minutes. Cumulative waits are// 2 / 4 / 7 / 10 / 14 / 18 / 22 / 26 / 30 minutes; anything still missing then// fails as not collected. Each sleep stays under 5 minutes, like rank check's:@@ -83,18 +87,23 @@ export class AiVisibilityWorkflow extends WorkflowEntrypoint<};let pending: AiPendingTask[] = [];try {- const { market, batches } = await pgStep(- step,- "prepare",- SINGLE_ATTEMPT,- () => prepareAiRun(runId, customer),+ const plan = await pgStep(step, "prepare", SINGLE_ATTEMPT, () =>+ prepareAiRun(runId, customer),);- for (const [index, batch] of batches.entries())- pending.push(- ...(await pgStep(step, `post-${index}`, SINGLE_ATTEMPT, () =>- postAiBatch(runId, customer, market, batch),- )),- );+ // Manual runs save each live batch as it returns, so nothing is left+ // pending to poll.+ if (plan.live)+ for (const [index, tasks] of plan.batches.entries())+ await pgStep(step, `live-${index}`, LIVE_STEP, () =>+ collectAiLiveBatch(runId, customer, plan.market, tasks),+ );+ else+ for (const [index, batch] of plan.batches.entries())+ pending.push(+ ...(await pgStep(step, `post-${index}`, SINGLE_ATTEMPT, () =>+ postAiBatch(runId, customer, plan.market, batch),+ )),+ );} catch (error) {return failRun(error);}diff --git a/src/shared/ai-visibility.ts b/src/shared/ai-visibility.tsindex 2c33b7e6..d643f424 100644--- a/src/shared/ai-visibility.ts+++ b/src/shared/ai-visibility.ts@@ -14,6 +14,11 @@ export const AI_ENGINE_LABELS: Record<AiEngine, string> = {* organic SERP page ($0.0006) plus the async overview load ($0.0006).*/export const AI_RECORD_COST_USD = 0.0012;+/**+ * Raw DataForSEO USD for one live answer, used by manual runs. AI Overviews is+ * one live organic SERP page ($0.002) plus the async overview load ($0.002).+ */+export const AI_LIVE_RECORD_COST_USD = 0.004;export type AiScheduleInterval = "daily" | "weekly" | "monthly";export type AiObservationStatus = "pending" | "completed" | "failed";/** Prompts without a chosen topic are grouped here. */@@ -102,6 +107,9 @@ export interface AiCostEstimate {providerCostUsd: number;costUsd: number;costCredits: number;+ /** A one-off Run now check collects live answers, at the live price. */+ runNowCostUsd: number;+ runNowCostCredits: number;scheduleInterval: AiScheduleInterval;checksPerMonth: number;monthlyCostUsd: number;diff --git a/src/types/schemas/ai-visibility.ts b/src/types/schemas/ai-visibility.tsindex 0077cca1..c5edb7cd 100644--- a/src/types/schemas/ai-visibility.ts+++ b/src/types/schemas/ai-visibility.ts@@ -98,7 +98,7 @@ export const runAiCheckSchema = aiProjectSchema.extend({.positive().max(100).describe(- "The most the user approved for this one check, from estimate_ai_visibility_cost. The check is refused if its current cost is higher.",+ "The most the user approved for this one check: runNowCostUsd from estimate_ai_visibility_cost. The check is refused if its current cost is higher.",),promptIds: z.array(z.string().uuid()).min(1).max(100).optional(),});diff --git a/web/content/docs/ai-visibility.md b/web/content/docs/ai-visibility.mdindex f1aeead5..e1545aa4 100644--- a/web/content/docs/ai-visibility.md+++ b/web/content/docs/ai-visibility.md@@ -20,7 +20,7 @@ The trend chart includes manual and scheduled checks, using the latest finished## Credits and cost-Prompt research costs about $0.25 in credits per keyword. Explorer charges actual usage for uncached answers. Tracking shows a cost estimate before a check or schedule is started and bills each check as it collects answers. Checks stop when your credits run out.+Prompt research costs about $0.25 in credits per keyword. Explorer charges actual usage for uncached answers. Tracking shows a cost estimate before a check or schedule is started and bills each check as it collects answers. Run now collects live answers, which usually arrive within a minute and cost more per answer than scheduled checks. Checks stop when your credits run out.On hosted OpenSEO, Prompt Research and Prompt Explorer require the paid plan; prompt tracking only needs credits. Setup research uses available credits and is billed after completion.