Open-source agentic AI Search Execution Platform. Track AI visibility, find opportunities, and let agents turn signals into prioritized actions and drafted fixes across ChatGPT, Claude, Gemini, Google AI Overviews, Google AI Mode, Perplexity, Grok, and Microsoft Copilot.
TypeScript aeoai-agentai-searchanswer-engine-optimizationbrand-monitoringchatgptclaudecopilotdockergeminigeogoogle-ai-overviewsgrokmcpnextjsopen-sourceperplexityseosupabasetypescript
feat(action-center): judge an action sent from Mentions by its answers' positive sentiment (learning D1) (#1725)
Cihan55e17cbda615
diff --git a/docs/guides/action-center.mdx b/docs/guides/action-center.mdxindex 8edcbf77d..0fe0bf9d8 100644--- a/docs/guides/action-center.mdx+++ b/docs/guides/action-center.mdx@@ -195,7 +195,7 @@ The **History** tab lists actions once they are In Progress, On Hold, CompletedClick an action to see **What was the goal?**, **What happened?** (the signals it was built from), its affected items and tasks, and the **Result**:-- A completed action's metrics are read three times: over the week ending 7, 14 and 30 days after it closed, each against the 7 days before it was raised. Until the reading on day 30, which is final, the result reads *so far*, and the drawer lists each reading. The result reads **Improved**, **Mixed**, **No meaningful change** or **Declined**, judged by the action's main metric (the first one it names): **Mixed** when another metric it names moved the other way, or, without the main one measured, when the ones it names moved both ways. Other metrics show beside but don't count. It is measured after the action, not proven to be caused by it. An action that names its prompts or pages is measured on them, with the brand-wide figures over the same days beside; when they can't show its main metric (citations, for prompts), it is measured brand-wide. Its prompts' change is also shown net of the brand's prompts it didn't touch over the same days (the other prompts in its topics, or the brand's others), so what moved for every prompt, such as an engine's update, is told apart from what moved for its own (a count from fewer than 5 before isn't netted: one more would read as a big swing); the result itself stays as measured. A Site Audit fix is judged by its pages audited again since it closed: the check that raised it passing, or a low score past the line; with none audited again yet, by the metrics. When tracking missed nights in either week, the drawer says so, and mentions and citations are read at a full week's worth from the nights that ran. More than two in one week are too few to read them from: they're left out of the result, and an action whose main metric is one of them reads **Not measurable**. The History drawer also says what became of the signals that raised it by then: how many ended, turned into another, were dismissed, or are still open.+- A completed action's metrics are read three times: over the week ending 7, 14 and 30 days after it closed, each against the 7 days before it was raised. Until the reading on day 30, which is final, the result reads *so far*, and the drawer lists each reading. The result reads **Improved**, **Mixed**, **No meaningful change** or **Declined**, judged by the action's main metric (the first one it names): **Mixed** when another metric it names moved the other way, or, without the main one measured, when the ones it names moved both ways. Other metrics show beside but don't count. It is measured after the action, not proven to be caused by it. An action that names its prompts or pages is measured on them, with the brand-wide figures over the same days beside; when they can't show its main metric (citations, for prompts), it is measured brand-wide. Positive Sentiment on its prompts is the positive share of their answers naming the brand, as on the Insights card, and isn't read from fewer than 5 of them: one answer turning would swing it, so the other metrics it names decide. Its prompts' change is also shown net of the brand's prompts it didn't touch over the same days (the other prompts in its topics, or the brand's others), so what moved for every prompt, such as an engine's update, is told apart from what moved for its own (a count from fewer than 5 before isn't netted: one more would read as a big swing); the result itself stays as measured. A Site Audit fix is judged by its pages audited again since it closed: the check that raised it passing, or a low score past the line; with none audited again yet, by the metrics. When tracking missed nights in either week, the drawer says so, and mentions and citations are read at a full week's worth from the nights that ran. More than two in one week are too few to read them from: they're left out of the result, and an action whose main metric is one of them reads **Not measurable**. The History drawer also says what became of the signals that raised it by then: how many ended, turned into another, were dismissed, or are still open.- Only a completed action is measured: a dismissed one reads **Not measured**, dismissed. One with no metric to read is **Not measurable**.- An action sent from a prompt's Improve tab reports on how many platforms its position improved. It is read again daily while the action is open and for 30 days after it is completed.@@ -217,7 +217,7 @@ Most actions are generated from signals each night. You can also send work to th| From | How ||---|---|-| **Mentions** | Select answers on the Mentions tab, or open one, and click **Send to Action Center**. |+| **Mentions** | Select answers on the Mentions tab, or open one, and click **Send to Action Center**. Sent with negative answers among them, the action is judged by Positive Sentiment: the positive share of its prompts' answers naming the brand, as on the Insights card (an answer that only cites the brand doesn't count); otherwise by how often the brand is named. An action sent before already listed Positive Sentiment among its metrics: it is still judged by mentions, but its result can now read **Mixed** when sentiment moved the other way. || **Content** | **Send to Actions** on a content opportunity; **Assign to…** on a Content Plan row; or **Who does it → Team** when you start new content. See [Content Editor](/guides/content-editor). || **Site Audit** | Audit results become signals with the Site Audit source, such as low AEO scores or missing structured data, and turn into Fix actions overnight. To send one page now, click **Send to Action Center** on its audit report: the page and the checks it fails go as one Fix action. || **AI Traffic** | On the [AI Traffic Analytics](/guides/ai-traffic-analytics) pages list, select pages under *Cited, not clicked* or *No AI traffic* and click **Send to Action Center**. Traffic drops and strong or weak landing pages also arrive as signals. |diff --git a/server/src/lib/action-center/basket-metrics.js b/server/src/lib/action-center/basket-metrics.jsindex 4ac1f0f0d..581652c1d 100644--- a/server/src/lib/action-center/basket-metrics.js+++ b/server/src/lib/action-center/basket-metrics.js@@ -20,10 +20,21 @@ export const BASKET_KPIS = {mentions: 'count',citations: 'count',ai_referral_traffic: 'sessions',+ // The positive share of the answers naming the brand (learning D1, 00373),+ // as Insights' Positive Sentiment card reads it (00191): an answer only+ // citing the brand isn't in it.+ positive_sentiment: 'percent',};const round1 = (n) => Math.round(n * 10) / 10;+/**+ * Fewest answers naming the brand a basket's positive sentiment is read+ * from, as CONTROL_MIN_COUNT: from 2, one answer turning positive reads+ * +50 points.+ */+export const SENTIMENT_MIN_ANSWERS = 5;+/*** The pages as the three keys the metrics function matches on: URL without* scheme, www., query, fragment or trailing slash; host without www.; and@@ -70,6 +81,12 @@ export function basketValues(raw, { pageCount }) {mentions: cells ? Number(raw.mention_answers) || 0 : null,citations: pageCount ? Number(raw.page_citations) || 0 : null,ai_referral_traffic: pageCount && raw.ga_connected ? Number(raw.page_ai_sessions) || 0 : null,+ // Null from fewer than SENTIMENT_MIN_ANSWERS answers naming the brand,+ // or while the window's sentiment isn't counted yet.+ positive_sentiment:+ raw.named_answers != null && Number(raw.named_answers) >= SENTIMENT_MIN_ANSWERS+ ? round1(((Number(raw.positive_answers) || 0) / Number(raw.named_answers)) * 100)+ : null,};}diff --git a/server/src/lib/action-center/basket-metrics.test.js b/server/src/lib/action-center/basket-metrics.test.jsindex 96a98b1b3..62e7bacda 100644--- a/server/src/lib/action-center/basket-metrics.test.js+++ b/server/src/lib/action-center/basket-metrics.test.js@@ -1,3 +1,4 @@+import { readFileSync } from 'node:fs';import { describe, expect, it, vi } from 'vitest';const tables = {};@@ -8,6 +9,7 @@ vi.mock('../../config/supabase.js', () => ({const {BASKET_KPIS,CONTROL_MIN_COUNT,+ SENTIMENT_MIN_ANSWERS,basketScope,basketValues,controlScope,@@ -68,6 +70,8 @@ describe('basketValues', () => {page_citations: 7,ga_connected: true,page_ai_sessions: 55,+ named_answers: 16,+ positive_answers: 7,};it('measures the basket', () => {@@ -79,12 +83,69 @@ describe('basketValues', () => {mentions: 12,citations: 7,ai_referral_traffic: 55,+ // 7 of the 16 answers naming the brand were positive (learning D1).+ positive_sentiment: 43.8,});});+ it('reads no sentiment while the window isn’t counted, or no answer named the brand', () => {+ expect(basketValues({ ...raw, named_answers: null }, { pageCount: 1 }).positive_sentiment).toBe(+ null,+ );+ expect(basketValues({ ...raw, named_answers: 0 }, { pageCount: 1 }).positive_sentiment).toBe(+ null,+ );+ });++ it('reads no sentiment from fewer than five answers naming the brand', () => {+ expect(SENTIMENT_MIN_ANSWERS).toBe(5);+ // 3 of 4 is 75%, but one answer turning either way would move it 25 points.+ expect(+ basketValues({ ...raw, named_answers: 4, positive_answers: 3 }, { pageCount: 1 })+ .positive_sentiment,+ ).toBe(null);+ expect(+ basketValues({ ...raw, named_answers: 5, positive_answers: 3 }, { pageCount: 1 })+ .positive_sentiment,+ ).toBe(60);+ });++ /** As Insights' Positive Sentiment card (00191): an answer only citing the brand is always neutral. */+ it('reads sentiment over the answers naming the brand, never those only citing it', () => {+ // 16 answers named or cited it, 10 named it, 4 of those positive: 40%, not 25%.+ expect(+ basketValues(+ { ...raw, mentioning_answers: 16, named_answers: 10, positive_answers: 4 },+ { pageCount: 1 },+ ).positive_sentiment,+ ).toBe(40);+ // A window of cited-only answers has no sentiment to read.+ expect(+ basketValues(+ { ...raw, mentioning_answers: 16, named_answers: 0, positive_answers: 0 },+ { pageCount: 1 },+ ).positive_sentiment,+ ).toBe(null);+ });++ it('is fed the rollup’s answers naming the brand by 00373, not those naming or citing it', () => {+ const sql = readFileSync(+ new URL(+ '../../../../supabase/migrations/00373_basket_positive_sentiment.sql',+ import.meta.url,+ ),+ 'utf8',+ );+ expect(sql).toMatch(/coalesce\(sum\(d\.mention_answers\), 0\) as named_answers/);+ expect(sql).toMatch(/'named_answers', \(select case when unfilled = 0 then named_answers end/);+ // Unfilled by the column it divides by.+ expect(sql).toMatch(/filter \(where d\.mention_answers is null or d\.positive_count is null\)/);+ expect(sql).not.toMatch(/sum\((d\.)?mentioning_answers\)/);+ });+it('leaves unanswerable metrics unmeasured instead of zero', () => {const none = basketValues(- { ...raw, cells: 0, visible: 0, competitor_visible_total: 0 },+ { ...raw, cells: 0, visible: 0, competitor_visible_total: 0, named_answers: 0 },{pageCount: 0,},diff --git a/server/src/lib/action-center/from-mentions.js b/server/src/lib/action-center/from-mentions.jsindex ae84785b9..ce0fa52e7 100644--- a/server/src/lib/action-center/from-mentions.js+++ b/server/src/lib/action-center/from-mentions.js@@ -21,6 +21,15 @@ export const MENTION_ACTION_KIND = 'mention_review';const ACTIVE = ['new', 'in_progress', 'on_hold'];const KPI_KEYS = ['mentions', 'ai_visibility', 'positive_sentiment'];++/**+ * Its success metrics, its aim first (validate.js judges it by the first):+ * answers sent with negative ones in them are Protect work, to turn how the+ * brand is spoken of (learning D1); others, to be named more.+ */+export function mentionKpiKeys(negative) {+ return negative > 0 ? ['positive_sentiment', 'mentions', 'ai_visibility'] : KPI_KEYS;+}/** At most a page of the Mentions table at once. */export const MAX_MENTIONS = 100;/** The answers quoted on the action. */@@ -163,7 +172,7 @@ export async function sendMentionsToActionCenter(brandId, { resultIds, excerptsdefinition_version: 1,impact,payload,- kpi_keys: KPI_KEYS,+ kpi_keys: mentionKpiKeys(negative),baseline: mentionBaseline(answers),dedup_key: dedupKey,})diff --git a/server/src/lib/action-center/from-mentions.test.js b/server/src/lib/action-center/from-mentions.test.jsindex 22135ac77..42cb8087c 100644--- a/server/src/lib/action-center/from-mentions.test.js+++ b/server/src/lib/action-center/from-mentions.test.js@@ -52,6 +52,7 @@ vi.mock('./sources.js', () => ({ resolveBrandSources: async () => new Set(['tracconst {MENTION_REVIEW_TASKS,mentionActionWeight,+ mentionKpiKeys,mentionBaseline,mentionDedupKey,sendMentionsToActionCenter,@@ -101,7 +102,8 @@ describe('sendMentionsToActionCenter', () => {category: 'protect',kind: 'mention_review',impact: 'medium',- kpi_keys: ['mentions', 'ai_visibility', 'positive_sentiment'],+ // Negative answers in it: judged by its sentiment first (learning D1).+ kpi_keys: ['positive_sentiment', 'mentions', 'ai_visibility'],baseline: { mentions: 3, positive: 1, negative: 1, netSentiment: 0 },// r4 isn't a mention, so it doesn't change which action these answers find.dedup_key: mentionDedupKey(['r1', 'r2', 'r3']),@@ -177,3 +179,11 @@ describe('the action’s weight and baseline', () => {expect(mentionDedupKey(['a'])).not.toBe(mentionDedupKey(['a', 'b']));});});++/** Its aim first (learning D1): sentiment when it carries negative answers, else mentions. */+describe('mentionKpiKeys', () => {+ it('judges answers with negative ones by their sentiment, the others by mentions', () => {+ expect(mentionKpiKeys(2)[0]).toBe('positive_sentiment');+ expect(mentionKpiKeys(0)).toEqual(['mentions', 'ai_visibility', 'positive_sentiment']);+ });+});diff --git a/server/src/lib/action-center/validate.test.js b/server/src/lib/action-center/validate.test.jsindex 9e101e52b..ddf19ec7a 100644--- a/server/src/lib/action-center/validate.test.js+++ b/server/src/lib/action-center/validate.test.js@@ -380,6 +380,46 @@ describe('validateAction', () => {expect(patch.validation.beforeWindow.to < patch.validation.afterWindow.from).toBe(true);});+ /** Sent with negative answers, it's judged by how the brand is spoken of (learning D1). */+ it('judges answers sent from Mentions by their positive sentiment', async () => {+ const writes = mockWrites();+ let reads = 0;+ // Named as often after as before, but 2 of 10 positive became 6 of 10.+ rpc.mockImplementation(async (name) => {+ if (name !== 'opportunity_basket_metrics') return { data: {}, error: null };+ const after = reads++ % 2 === 1;+ return {+ data: {+ cells: 10,+ visible: 10,+ mention_answers: 10,+ named_answers: 10,+ positive_answers: after ? 6 : 2,+ },+ error: null,+ };+ });++ await validateAction(+ action({+ kind: 'mention_review',+ kpi_keys: ['positive_sentiment', 'mentions', 'ai_visibility'],+ payload: { promptIds: ['p1'] },+ }),+ { now: NOW },+ );++ const patch = writes.updated.find((w) => w.table === 'actions').patch;+ expect(patch.validation.primary).toBe('positive_sentiment');+ expect(patch.validation.metrics).toContainEqual({+ metric: 'positive_sentiment',+ unit: 'percent',+ before: 20,+ after: 60,+ });+ expect(patch.outcome).toBe('improved');+ });+it("measures answers sent from Mentions on their prompts' basket", async () => {const writes = mockWrites();rpc.mockResolvedValue({ data: {}, error: null });diff --git a/supabase/migrations/00373_basket_positive_sentiment.sql b/supabase/migrations/00373_basket_positive_sentiment.sqlnew file mode 100644index 000000000..09b8b9fd1--- /dev/null+++ b/supabase/migrations/00373_basket_positive_sentiment.sql@@ -0,0 +1,102 @@+-- ============================================================================+-- 00373: a basket's positive sentiment (learning D1)+-- ============================================================================+-- An action sent from Mentions (MS-08) is measured on its answers' prompts+-- (opportunity_basket_metrics, 00116), but the basket read mentions,+-- visibility and citations only: an action sent to turn negative answers+-- around was judged by how often the brand was named, not by how it was+-- spoken of. The basket now reads its prompts' answers naming the brand+-- (the rollup's mention_answers, 00130) and the positive ones among them;+-- the server turns them into the positive share (basket-metrics.js). That is+-- the base Insights' Positive Sentiment card divides by since 00191: an+-- answer that only cites the brand is always neutral, so counting it (the+-- rollup's mentioning_answers, naming OR citing) would pull the share down.+-- Null while a day of the window isn't rewritten with them yet. Otherwise+-- as 00116.++create or replace function public.opportunity_basket_metrics(+ p_brand_id uuid,+ p_prompt_ids uuid[],+ p_page_keys text[],+ p_page_domains text[],+ p_page_paths text[],+ p_from date,+ p_to date+)+returns jsonb+language sql+stable+set search_path = public+as $$+ with cells as (+ select count(*) as cells,+ count(*) filter (where has_mention or has_citation) as visible,+ coalesce(sum(answer_count) filter (where has_mention), 0) as mention_answers,+ -- Sentiment (00373): the answers naming the brand (the rollup's+ -- column, not the alias above; never the ones only citing it), and+ -- the positive ones among them, as Insights' Positive Sentiment+ -- card counts them (00191).+ coalesce(sum(d.mention_answers), 0) as named_answers,+ coalesce(sum(d.positive_count), 0) as positive_answers,+ count(*) filter (where d.mention_answers is null or d.positive_count is null) as unfilled+ from insights_prompt_daily d+ where brand_id = p_brand_id+ and day between p_from and p_to+ and prompt_id = any(p_prompt_ids)+ ),+ comp as (+ select d.competitor_id, count(*) as visible+ from insights_competitor_prompt_daily d+ join competitors c on c.id::text = d.competitor_id and c.brand_id = p_brand_id+ where d.brand_id = p_brand_id+ and d.day between p_from and p_to+ and d.prompt_id = any(p_prompt_ids)+ group by d.competitor_id+ ),+ urls as (+ select id+ from citation_urls+ where domain = any(p_page_domains)+ and rtrim(+ regexp_replace(split_part(split_part(lower(url), '#', 1), '?', 1), '^https?://(www\.)?', ''),+ '/'+ ) = any(p_page_keys)+ )+ select jsonb_build_object(+ 'cells', (select cells from cells),+ 'visible', (select visible from cells),+ 'mention_answers', (select mention_answers from cells),+ -- Null while a day of the window isn't rewritten with them yet (00130):+ -- a partial count would read as a change.+ 'named_answers', (select case when unfilled = 0 then named_answers end from cells),+ 'positive_answers', (select case when unfilled = 0 then positive_answers end from cells),+ 'competitor_visible_total', (select coalesce(sum(visible), 0) from comp),+ 'top_competitor_visible', (select coalesce(max(visible), 0) from comp),+ 'page_citations', (+ select count(*)+ from prompt_result_citations+ where brand_id = p_brand_id+ and created_at >= p_from+ and created_at < p_to + 1+ and url_id in (select id from urls)+ ),+ 'ga_connected', exists (+ select 1 from ga_ai_traffic_stats+ where brand_id = p_brand_id and date between p_from and p_to+ ),+ 'page_ai_sessions', (+ select coalesce(sum(sessions), 0)+ from ga_ai_traffic_stats+ where brand_id = p_brand_id+ and date between p_from and p_to+ and coalesce(nullif(rtrim(split_part(landing_page, '?', 1), '/'), ''), '/') = any(p_page_paths)+ )+ );+$$;++revoke all on function public.opportunity_basket_metrics(uuid, uuid[], text[], text[], text[], date, date)+ from public, anon, authenticated;+grant execute on function public.opportunity_basket_metrics(uuid, uuid[], text[], text[], text[], date, date)+ to service_role;++notify pgrst, 'reload schema';diff --git a/supabase/schema.sql b/supabase/schema.sqlindex b0a0ba078..c66fe44a9 100644--- a/supabase/schema.sql+++ b/supabase/schema.sql@@ -42998,6 +42998,112 @@ comment on function public.catalog_suggestion_categories(uuid, integer, integer)notify pgrst, 'reload schema';+-- ─────────────────────────────────────────────────────────────────────────+-- migrations/00373_basket_positive_sentiment.sql+-- ─────────────────────────────────────────────────────────────────────────+-- ============================================================================+-- 00373: a basket's positive sentiment (learning D1)+-- ============================================================================+-- An action sent from Mentions (MS-08) is measured on its answers' prompts+-- (opportunity_basket_metrics, 00116), but the basket read mentions,+-- visibility and citations only: an action sent to turn negative answers+-- around was judged by how often the brand was named, not by how it was+-- spoken of. The basket now reads its prompts' answers naming the brand+-- (the rollup's mention_answers, 00130) and the positive ones among them;+-- the server turns them into the positive share (basket-metrics.js). That is+-- the base Insights' Positive Sentiment card divides by since 00191: an+-- answer that only cites the brand is always neutral, so counting it (the+-- rollup's mentioning_answers, naming OR citing) would pull the share down.+-- Null while a day of the window isn't rewritten with them yet. Otherwise+-- as 00116.++create or replace function public.opportunity_basket_metrics(+ p_brand_id uuid,+ p_prompt_ids uuid[],+ p_page_keys text[],+ p_page_domains text[],+ p_page_paths text[],+ p_from date,+ p_to date+)+returns jsonb+language sql+stable+set search_path = public+as $$+ with cells as (+ select count(*) as cells,+ count(*) filter (where has_mention or has_citation) as visible,+ coalesce(sum(answer_count) filter (where has_mention), 0) as mention_answers,+ -- Sentiment (00373): the answers naming the brand (the rollup's+ -- column, not the alias above; never the ones only citing it), and+ -- the positive ones among them, as Insights' Positive Sentiment+ -- card counts them (00191).+ coalesce(sum(d.mention_answers), 0) as named_answers,+ coalesce(sum(d.positive_count), 0) as positive_answers,+ count(*) filter (where d.mention_answers is null or d.positive_count is null) as unfilled+ from insights_prompt_daily d+ where brand_id = p_brand_id+ and day between p_from and p_to+ and prompt_id = any(p_prompt_ids)+ ),+ comp as (+ select d.competitor_id, count(*) as visible+ from insights_competitor_prompt_daily d+ join competitors c on c.id::text = d.competitor_id and c.brand_id = p_brand_id+ where d.brand_id = p_brand_id+ and d.day between p_from and p_to+ and d.prompt_id = any(p_prompt_ids)+ group by d.competitor_id+ ),+ urls as (+ select id+ from citation_urls+ where domain = any(p_page_domains)+ and rtrim(+ regexp_replace(split_part(split_part(lower(url), '#', 1), '?', 1), '^https?://(www\.)?', ''),+ '/'+ ) = any(p_page_keys)+ )+ select jsonb_build_object(+ 'cells', (select cells from cells),+ 'visible', (select visible from cells),+ 'mention_answers', (select mention_answers from cells),+ -- Null while a day of the window isn't rewritten with them yet (00130):+ -- a partial count would read as a change.+ 'named_answers', (select case when unfilled = 0 then named_answers end from cells),+ 'positive_answers', (select case when unfilled = 0 then positive_answers end from cells),+ 'competitor_visible_total', (select coalesce(sum(visible), 0) from comp),+ 'top_competitor_visible', (select coalesce(max(visible), 0) from comp),+ 'page_citations', (+ select count(*)+ from prompt_result_citations+ where brand_id = p_brand_id+ and created_at >= p_from+ and created_at < p_to + 1+ and url_id in (select id from urls)+ ),+ 'ga_connected', exists (+ select 1 from ga_ai_traffic_stats+ where brand_id = p_brand_id and date between p_from and p_to+ ),+ 'page_ai_sessions', (+ select coalesce(sum(sessions), 0)+ from ga_ai_traffic_stats+ where brand_id = p_brand_id+ and date between p_from and p_to+ and coalesce(nullif(rtrim(split_part(landing_page, '?', 1), '/'), ''), '/') = any(p_page_paths)+ )+ );+$$;++revoke all on function public.opportunity_basket_metrics(uuid, uuid[], text[], text[], text[], date, date)+ from public, anon, authenticated;+grant execute on function public.opportunity_basket_metrics(uuid, uuid[], text[], text[], text[], date, date)+ to service_role;++notify pgrst, 'reload schema';+-- ─────────────────────────────────────────────────────────────────────────-- migrations/00374_menu_access.sql-- ─────────────────────────────────────────────────────────────────────────