Skip to content

AI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary

fix: Step 0.55 category-peer subreddit expansion (#305)

Matt Van Horn4e91f4e75448
diff --git a/SKILL.md b/SKILL.md
index 14ebd2b7..b7f03e4d 100644
--- a/SKILL.md
+++ b/SKILL.md
@@ -596,6 +596,43 @@ The first search finds subreddits. The second gives you current events context (
Extract 3-5 subreddit names from the results. Store as `RESOLVED_SUBREDDITS` (comma-separated, no r/ prefix).
+**2a. Category-peer expansion (MANDATORY for product topics).** If the topic is a product in a recognizable category (AI image generation, AI video generation, AI coding agents, AI music, AI chat models, SaaS screen recording, prediction markets, etc.), the brand-specific subreddits that WebSearch returned are INSUFFICIENT. Add 2-3 peer subreddits from the category. Peer subs are where cross-product technique discussion actually lives. Missing them is the 2026-04-22 `GPT Image 2` failure mode: the model resolved `r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering` (all OpenAI-brand) and missed `r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt` where prompting techniques are actually shared. The user had to manually prompt "check image generation reddits too" to get a usable run.
+
+Canonical category peers (single source of truth; `scripts/lib/categories.py` mirrors this for the `--auto-resolve` engine path):
+
+| Category | Trigger keywords | Peer subs (priority order) |
+|----------|------------------|---------------------------|
+| `ai_image_generation` | image generation, text to image, GPT Image, Nano Banana, Midjourney, Stable Diffusion, DALL-E, Flux.1, Imagen, Seedance, Ideogram, Recraft | `StableDiffusion, midjourney, dalle2, aiArt, PromptEngineering, MediaSynthesis` |
+| `ai_video_generation` | video generation, text to video, Sora, Veo 3, Runway Gen, Kling, Pika Labs, Luma Dream Machine, Hailuo | `aivideo, StableDiffusion, runwayml, singularity, MediaSynthesis` |
+| `ai_music_generation` | music generation, ai music, Suno, Udio, Riffusion, Stable Audio | `SunoAI, udiomusic, aimusic, artificial` |
+| `ai_coding_agent` | Claude Code, Cursor IDE, GitHub Copilot, Windsurf, Aider, Cline, OpenClaw, Hermes Agent, Continue.dev, Codeium, Devin | `ChatGPTCoding, LocalLLaMA, singularity, PromptEngineering` |
+| `ai_agent_framework` | agent framework, LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, DSPy, smolagents | `LangChain, LocalLLaMA, AI_Agents, MachineLearning` |
+| `ai_chat_model` | GPT-5/4, Claude Opus/Sonnet/Haiku, Gemini Pro/Flash, Llama 3/4, DeepSeek, Qwen, Mistral Large, Grok | `LocalLLaMA, ChatGPT, ClaudeAI, singularity, artificial` |
+| `saas_screen_recording` | screen recording, screen recorder, Loom video, Tella screen, Vidyard | `SaaS, screenrecording, productivity, Entrepreneur` |
+| `saas_productivity` | Notion app, Obsidian, Linear app, Asana, ClickUp, productivity app | `productivity, SaaS, ObsidianMD, Notion` |
+| `prediction_markets` | Polymarket, Kalshi, prediction market, event contracts, Manifold Markets | `Polymarket, Kalshi, predictionmarkets` |
+| `crypto_defi` | DeFi protocol, yield farming, liquidity pool, stablecoin, layer 2, L2 rollup | `defi, ethfinance, CryptoCurrency, ethereum` |
+
+**Merging rule.** Start with WebSearch-returned subs. Append 2-3 category peers in the priority order shown. Dedupe case-insensitively (don't list `midjourney` twice if WebSearch already returned it). Cap total at 10: if adding all peers would exceed the cap, keep every WebSearch-returned sub (they are the freshest signal) and drop peers from the end of the priority list.
+
+**Extrapolation.** If the topic is a product in a category NOT listed in the table (new AI tool, niche SaaS), use the same spirit: pick the 2-3 most active cross-product communities where technique discussion happens. A new image-gen tool still gets `r/StableDiffusion, r/midjourney, r/aiArt`. A new code editor still gets `r/ChatGPTCoding, r/LocalLLaMA`.
+
+**Worked example — the failing query.** Topic: `Prompting GPT Image 2`.
+
+Before (the 2026-04-22 failure mode):
+```
+Resolved:
+- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/artificial
+```
+
+After (with category-peer expansion):
+```
+Resolved:
+- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
+```
+
+The parenthetical `(+ ai_image_generation peers)` is the observable contract of the new Resolved block format. See Step 0.55 self-check below.
+
**3. TikTok hashtags + creators** - **INFER these from your topic knowledge. Do NOT WebSearch for "{PERSON} TikTok account" - most people/CEOs don't have TikTok, and the search is wasted.**
- **Hashtags:** Infer 2-3 from the topic name + category. Examples: "Kanye West" → `kanyewest,ye,bully`. "Claude Code" → `claudecode,aiagent,aicoding`. "Sam Altman" → `samaltman,openai,chatgpt`.
@@ -658,17 +695,19 @@ Passing the resolved block visibly (per-entity, all 4 types each) is the observa
**If you can't infer targeting for a platform, skip that flag -- the Python engine will fall back to keyword search.**
+**Step 0.55 self-check: category-peer coverage.** Before emitting the Resolved block, re-read your resolved subreddit list. Does the topic match any category in the Section 2a table (or fit the spirit of one — AI image gen, AI coding, AI music, etc.)? If YES: does your list include AT LEAST 2 peer subs from that category? If NO, widen the list NOW — do not run the engine yet. The observable contract is the `(+ {category_id} peers)` annotation on the Reddit line in the Resolved block. Its absence on a product-in-a-known-category topic is a Step 0.55 regression — the named 2026-04-22 failure mode. Person topics, music artists, news stories, and topics outside any category are exempt; omit the annotation.
+
**After resolving all handles and communities, display what you found before moving on.** This shows the user that intelligent pre-research happened:
```
Resolved:
- X: @{HANDLE} (+ @{COMPANY}, @{COMMENTATOR})
-- Reddit: r/{sub1}, r/{sub2}, r/{sub3}
+- Reddit: r/{sub1}, r/{sub2}, r/{sub3}, r/{peer1}, r/{peer2} (+ {category_id} peers)
- TikTok: #{hashtag1}, #{hashtag2}
- YouTube: {query1}, {query2}
```
-Only show lines for platforms where something was resolved. Skip empty lines. This display replaces the old "Parsed intent" block with something more useful.
+Only show lines for platforms where something was resolved. Skip empty lines. On the Reddit line, the trailing `(+ {category_id} peers)` annotation appears when Step 0.55 Section 2a added category-peer subs. Omit the annotation when the topic had no matching category. This display replaces the old "Parsed intent" block with something more useful.
---
diff --git a/scripts/lib/categories.py b/scripts/lib/categories.py
new file mode 100644
index 00000000..f21f07c1
--- /dev/null
+++ b/scripts/lib/categories.py
@@ -0,0 +1,283 @@
+"""Category-peer subreddit map for Step 0.55 community resolution.
+
+When a topic is a product in a known category (AI image generation, AI coding
+agents, SaaS screen recording, etc.), brand-specific subreddits returned by
+WebSearch are insufficient: cross-product technique discussion lives in
+category-peer subs. This module classifies a topic into a category by matching
+compound-term patterns against the lowercased topic string, then returns the
+priority-ordered peer subreddit list for that category.
+
+The map is intentionally small, curated, and code-reviewed. Adding a new
+category is a code change; there is no user-editable override surface.
+
+False-positive guard: every pattern is either a multi-word compound (e.g.
+"image generation", "text to image") or a domain-specific single word
+(e.g. "midjourney", "stablediffusion"). Bare common nouns like "image",
+"ai", or "model" are never used as patterns.
+
+First-match-wins: categories are evaluated in declared order. Entries are
+sorted from most-specific to least-specific so narrower categories claim a
+topic before broader ones. For example, `ai_image_generation` appears
+before `ai_chat_model` so "gpt image 2" matches the image-gen category.
+"""
+
+from __future__ import annotations
+
+from typing import List, Optional, TypedDict
+
+
+class _CategoryEntry(TypedDict):
+ patterns: List[str]
+ peer_subs: List[str]
+
+
+CATEGORY_PEERS: dict[str, _CategoryEntry] = {
+ "ai_image_generation": {
+ "patterns": [
+ "image generation",
+ "image gen",
+ "text to image",
+ "text-to-image",
+ "gpt image",
+ "gpt-image",
+ "nano banana",
+ "midjourney",
+ "stable diffusion",
+ "stablediffusion",
+ "dall-e",
+ "dalle",
+ "flux.1",
+ "flux schnell",
+ "imagen",
+ "seedance",
+ "ideogram",
+ "recraft",
+ ],
+ "peer_subs": [
+ "StableDiffusion",
+ "midjourney",
+ "dalle2",
+ "aiArt",
+ "PromptEngineering",
+ "MediaSynthesis",
+ ],
+ },
+ "ai_video_generation": {
+ "patterns": [
+ "video generation",
+ "text to video",
+ "text-to-video",
+ "sora",
+ "veo 3",
+ "veo3",
+ "runway gen",
+ "kling",
+ "pika labs",
+ "luma dream machine",
+ "hailuo",
+ ],
+ "peer_subs": [
+ "aivideo",
+ "StableDiffusion",
+ "runwayml",
+ "singularity",
+ "MediaSynthesis",
+ ],
+ },
+ "ai_music_generation": {
+ "patterns": [
+ "music generation",
+ "ai music",
+ "suno",
+ "udio",
+ "riffusion",
+ "stable audio",
+ ],
+ "peer_subs": [
+ "SunoAI",
+ "udiomusic",
+ "aimusic",
+ "artificial",
+ ],
+ },
+ "ai_coding_agent": {
+ "patterns": [
+ "claude code",
+ "cursor ide",
+ "github copilot",
+ "windsurf",
+ "aider",
+ "cline",
+ "openclaw",
+ "hermes agent",
+ "continue.dev",
+ "codeium",
+ "sweep ai",
+ "devin ai",
+ "coding agent",
+ "coding assistant",
+ ],
+ "peer_subs": [
+ "ChatGPTCoding",
+ "LocalLLaMA",
+ "singularity",
+ "PromptEngineering",
+ ],
+ },
+ "ai_agent_framework": {
+ "patterns": [
+ "agent framework",
+ "agentic framework",
+ "langchain",
+ "langgraph",
+ "crewai",
+ "autogen",
+ "llamaindex",
+ "dspy",
+ "smolagents",
+ ],
+ "peer_subs": [
+ "LangChain",
+ "LocalLLaMA",
+ "AI_Agents",
+ "MachineLearning",
+ ],
+ },
+ "ai_chat_model": {
+ "patterns": [
+ "gpt-5",
+ "gpt-4",
+ "claude opus",
+ "claude sonnet",
+ "claude haiku",
+ "gemini pro",
+ "gemini flash",
+ "llama 3",
+ "llama 4",
+ "deepseek",
+ "qwen",
+ "mistral large",
+ "grok",
+ ],
+ "peer_subs": [
+ "LocalLLaMA",
+ "ChatGPT",
+ "ClaudeAI",
+ "singularity",
+ "artificial",
+ ],
+ },
+ "saas_screen_recording": {
+ "patterns": [
+ "screen recording",
+ "screen recorder",
+ "loom video",
+ "tella screen",
+ "vidyard",
+ "screen capture tool",
+ ],
+ "peer_subs": [
+ "SaaS",
+ "screenrecording",
+ "productivity",
+ "Entrepreneur",
+ ],
+ },
+ "saas_productivity": {
+ "patterns": [
+ "notion app",
+ "obsidian plugin",
+ "obsidian app",
+ "linear app",
+ "asana",
+ "clickup",
+ "productivity app",
+ ],
+ "peer_subs": [
+ "productivity",
+ "SaaS",
+ "ObsidianMD",
+ "Notion",
+ ],
+ },
+ "prediction_markets": {
+ "patterns": [
+ "polymarket",
+ "kalshi",
+ "prediction market",
+ "event contracts",
+ "manifold markets",
+ ],
+ "peer_subs": [
+ "Polymarket",
+ "Kalshi",
+ "predictionmarkets",
+ ],
+ },
+ "crypto_defi": {
+ "patterns": [
+ "defi protocol",
+ "yield farming",
+ "liquidity pool",
+ "stablecoin",
+ "ethereum layer",
+ "layer 2",
+ "l2 rollup",
+ ],
+ "peer_subs": [
+ "defi",
+ "ethfinance",
+ "CryptoCurrency",
+ "ethereum",
+ ],
+ },
+ "dev_tool_cli": {
+ "patterns": [
+ "cli tool",
+ "command line tool",
+ "terminal app",
+ "dev tool",
+ ],
+ "peer_subs": [
+ "commandline",
+ "programming",
+ "webdev",
+ ],
+ },
+}
+
+
+def detect_category(topic: Optional[str]) -> Optional[str]:
+ """Classify a topic into a known category by compound-term match.
+
+ Returns the category id (e.g. "ai_image_generation") or None if no
+ category's patterns match. Matching is case-insensitive substring over
+ the lowercased topic. Declaration order wins (first-match-wins), so the
+ map is ordered from most-specific to least-specific.
+
+ A None or empty topic returns None. Classification never raises on
+ normal string inputs; callers do not need to wrap in try/except for
+ typical paths, though defensive callers may.
+ """
+ if not topic:
+ return None
+ lowered = topic.lower()
+ for category_id, entry in CATEGORY_PEERS.items():
+ for pattern in entry["patterns"]:
+ if pattern in lowered:
+ return category_id
+ return None
+
+
+def peer_subs_for(category_id: Optional[str]) -> List[str]:
+ """Return the priority-ordered peer subreddit list for a category.
+
+ Returns an empty list for None or unknown category ids. The returned
+ list is a fresh copy; callers may safely mutate it.
+ """
+ if not category_id:
+ return []
+ entry = CATEGORY_PEERS.get(category_id)
+ if not entry:
+ return []
+ return list(entry["peer_subs"])
diff --git a/scripts/lib/resolve.py b/scripts/lib/resolve.py
index 8e945ce9..6bc77486 100644
--- a/scripts/lib/resolve.py
+++ b/scripts/lib/resolve.py
@@ -11,14 +11,64 @@ import re
import sys
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import datetime, timezone
+from typing import Optional
-from . import dates, grounding
+from . import categories, dates, grounding
+
+MAX_SUBS = 10
def _log(msg: str) -> None:
print(f"[Resolve] {msg}", file=sys.stderr)
+def _merge_category_peers(topic: str, subreddits: list[str]) -> tuple[list[str], Optional[str]]:
+ """Extend the WebSearch-extracted subreddit list with category peers.
+
+ Classifies the topic, fetches the category's peer subs, dedupes
+ case-insensitively against the existing list, and appends missing
+ peers in priority order. Caps the final list at MAX_SUBS, preserving
+ every WebSearch-returned sub (they are the freshest signal) and
+ trimming from the peer-additions end.
+
+ Returns a tuple of (merged_subs, matched_category_id_or_None).
+ Emits a [Resolve] Matched category log line only when peers were
+ actually added (not when every peer was already in the WebSearch set).
+
+ Classification failures degrade to "no match" — the unwidened list
+ is returned and a warning is logged.
+ """
+ try:
+ category = categories.detect_category(topic)
+ except Exception as exc:
+ _log(f"Category classification failed: {exc}")
+ return list(subreddits)[:MAX_SUBS], None
+
+ if category is None:
+ return list(subreddits)[:MAX_SUBS], None
+
+ peers = categories.peer_subs_for(category)
+ if not peers:
+ return list(subreddits)[:MAX_SUBS], category
+
+ existing_lower = {s.lower() for s in subreddits}
+ merged = list(subreddits)
+ added: list[str] = []
+ for peer in peers:
+ if len(merged) >= MAX_SUBS:
+ break
+ if peer.lower() in existing_lower:
+ continue
+ merged.append(peer)
+ existing_lower.add(peer.lower())
+ added.append(peer)
+
+ if added:
+ _log(f"Matched category={category}, adding peers: {', '.join(added)}")
+
+ return merged, category
+
+
def _has_backend(config: dict) -> bool:
"""Check if any web search backend is available."""
return bool(
@@ -134,10 +184,19 @@ def auto_resolve(topic: str, config: dict) -> dict:
config: Dict with API keys (BRAVE_API_KEY, EXA_API_KEY, SERPER_API_KEY).
Returns:
- Dict with keys: subreddits, x_handle, context, searches_run.
- Returns empty result if no web search backend is available.
+ Dict with keys: subreddits, x_handle, github_user, github_repos,
+ context, category, searches_run. Returns empty result if no web
+ search backend is available.
"""
- empty = {"subreddits": [], "x_handle": "", "context": "", "searches_run": 0}
+ empty = {
+ "subreddits": [],
+ "x_handle": "",
+ "github_user": "",
+ "github_repos": [],
+ "context": "",
+ "category": None,
+ "searches_run": 0,
+ }
if not _has_backend(config):
_log("No web search backend available, skipping resolve")
@@ -184,7 +243,9 @@ def auto_resolve(topic: str, config: dict) -> dict:
github_repos = _extract_github_repos(results.get("github", []))
context = _build_context_summary(results.get("news", []))
- _log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}")
+ subreddits, category = _merge_category_peers(topic, subreddits)
+
+ _log(f"Resolved {len(subreddits)} subreddits, x_handle={x_handle!r}, github_user={github_user!r}, github_repos={github_repos!r}, context_len={len(context)}, category={category!r}")
return {
"subreddits": subreddits,
@@ -192,5 +253,6 @@ def auto_resolve(topic: str, config: dict) -> dict:
"github_user": github_user,
"github_repos": github_repos,
"context": context,
+ "category": category,
"searches_run": searches_run,
}
diff --git a/tests/fixtures/prompting-gpt-image-2-resolved-block.md b/tests/fixtures/prompting-gpt-image-2-resolved-block.md
new file mode 100644
index 00000000..8aaea277
--- /dev/null
+++ b/tests/fixtures/prompting-gpt-image-2-resolved-block.md
@@ -0,0 +1,60 @@
+# Fixture: `Prompting GPT Image 2` Resolved-block regression
+
+Documentation-grade fixture. Captures the pre-fix and post-fix shape of the
+Step 0.55 Resolved block for the topic `Prompting GPT Image 2`. Not parsed
+by test code — read by reviewers when evaluating regressions in
+`scripts/lib/categories.py` or the SKILL.md Step 0.55 block.
+
+The live assertion lives in `tests/test_category_integration.py`. This
+markdown fixture exists so reviewers can eyeball expected behavior without
+running pytest.
+
+## Failing run (2026-04-22, pre-fix)
+
+User ran `/last30days Prompting GPT Image 2`. Step 0.55 WebSearch returned
+OpenAI-brand communities. The model resolved exactly those.
+
+```
+Resolved:
+- X: @OpenAI (+ @sama, @openaidevs)
+- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/artificial, r/ChatGPTpromptengineering
+- TikTok: #gptimage2, #openai, #aiart
+```
+
+Engine run returned thin results. User manually intervened with "make sure
+to check image generatorion reddits too" and re-ran with the image-gen
+peer subs added.
+
+## Expected run (post-fix, no user intervention)
+
+After Step 0.55 Section 2a (category-peer expansion) and Unit 2's engine-side
+merge in `auto_resolve`, the same topic produces:
+
+```
+Resolved:
+- X: @OpenAI (+ @sama, @openaidevs)
+- Reddit: r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering, r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt (+ ai_image_generation peers)
+- TikTok: #gptimage2, #openai, #aiart
+```
+
+The peer subs (`StableDiffusion, midjourney, dalle2, aiArt`) appear alongside
+the WebSearch-returned brand subs. The `(+ ai_image_generation peers)`
+annotation is the observable contract — its absence on a product-in-a-known-
+category topic is a Step 0.55 regression.
+
+## Guards
+
+- `tests/test_categories.py::DetectCategoryHappyPath::test_prompting_gpt_image_2_matches_image_generation`
+- `tests/test_resolve.py::MergeCategoryPeersHappyPath::test_image_gen_topic_appends_peers`
+- `tests/test_resolve.py::AutoResolveCategoryIntegration::test_auto_resolve_returns_category_key`
+- `tests/test_category_integration.py` — end-to-end over `auto_resolve` with
+ a stubbed WebSearch that mimics the original failing response.
+
+## When to update this fixture
+
+- Category map changed (a peer sub was reordered, added, or removed).
+- The observable Resolved-block annotation format changed.
+- A new category was added that affects this topic.
+
+Do not update casually. This file is the pre/post record of the 2026-04-22
+failure.
diff --git a/tests/test_categories.py b/tests/test_categories.py
new file mode 100644
index 00000000..848abaad
--- /dev/null
+++ b/tests/test_categories.py
@@ -0,0 +1,154 @@
+"""Unit tests for scripts/lib/categories.py — the Step 0.55 category-peer map.
+
+Guards the 2026-04-22 `Prompting GPT Image 2` failure mode: the original bug
+was that Step 0.55 resolved only brand-adjacent subs (r/OpenAI, r/ChatGPT)
+and missed the category peers (r/StableDiffusion, r/midjourney, r/dalle2)
+where prompting techniques actually live.
+"""
+
+import re
+import sys
+import unittest
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
+
+from lib import categories
+from lib.categories import CATEGORY_PEERS, detect_category, peer_subs_for
+
+
+class DetectCategoryHappyPath(unittest.TestCase):
+ def test_prompting_gpt_image_2_matches_image_generation(self):
+ self.assertEqual(
+ detect_category("Prompting GPT Image 2"),
+ "ai_image_generation",
+ )
+
+ def test_claude_code_matches_coding_agent(self):
+ self.assertEqual(
+ detect_category("Claude Code skills"),
+ "ai_coding_agent",
+ )
+
+ def test_suno_matches_music_generation(self):
+ self.assertEqual(detect_category("Suno v4 review"), "ai_music_generation")
+
+ def test_polymarket_matches_prediction_markets(self):
+ self.assertEqual(
+ detect_category("Polymarket election odds"),
+ "prediction_markets",
+ )
+
+ def test_sora_matches_video_generation(self):
+ self.assertEqual(detect_category("Sora 2 prompts"), "ai_video_generation")
+
+
+class PeerSubsForHappyPath(unittest.TestCase):
+ def test_image_generation_peer_subs_priority_order(self):
+ subs = peer_subs_for("ai_image_generation")
+ self.assertIn("StableDiffusion", subs)
+ self.assertIn("midjourney", subs)
+ self.assertIn("dalle2", subs)
+ self.assertLess(subs.index("StableDiffusion"), subs.index("midjourney"))
+ self.assertLess(subs.index("midjourney"), subs.index("dalle2"))
+
+ def test_unknown_category_returns_empty_list(self):
+ self.assertEqual(peer_subs_for("unknown_category"), [])
+
+ def test_none_category_returns_empty_list(self):
+ self.assertEqual(peer_subs_for(None), [])
+
+ def test_returned_list_is_fresh_copy(self):
+ first = peer_subs_for("ai_image_generation")
+ first.append("MutatedSub")
+ second = peer_subs_for("ai_image_generation")
+ self.assertNotIn("MutatedSub", second)
+
+
+class DetectCategoryEdgeCases(unittest.TestCase):
+ def test_case_insensitive_match(self):
+ self.assertEqual(
+ detect_category("STABLE DIFFUSION walkthrough"),
+ "ai_image_generation",
+ )
+
+ def test_non_category_topic_returns_none(self):
+ self.assertIsNone(detect_category("Kanye West"))
+
+ def test_bare_image_word_does_not_trigger_image_generation(self):
+ # Compound-term guard: "image" alone is not a pattern; only
+ # multi-word compounds or domain-specific brand names match.
+ self.assertIsNone(detect_category("image editing on my phone"))
+
+ def test_bare_ai_word_does_not_trigger_any_category(self):
+ self.assertIsNone(detect_category("ai news today"))
+
+ def test_empty_topic_returns_none(self):
+ self.assertIsNone(detect_category(""))
+
+ def test_none_topic_returns_none(self):
+ self.assertIsNone(detect_category(None))
+
+ def test_first_match_wins_image_gen_before_chat_model(self):
+ # "gpt image 2" contains "gpt image" (ai_image_generation) and the
+ # substring "gpt" could resemble gpt-N chat-model patterns. The
+ # narrower category wins because it is declared earlier.
+ self.assertEqual(
+ detect_category("gpt image 2 review"),
+ "ai_image_generation",
+ )
+
+
+class CategoryMapInvariants(unittest.TestCase):
+ """Regression guards on the map itself — catch accidental bare-word patterns."""
+
+ # Common nouns that would produce false positives if used as bare patterns.
+ FORBIDDEN_BARE_PATTERNS = frozenset({
+ "image", "video", "music", "ai", "model", "agent", "chat",
+ "code", "cli", "app", "tool", "defi",
+ })
+
+ def test_no_category_has_a_bare_common_noun_pattern(self):
+ offenders = []
+ for category_id, entry in CATEGORY_PEERS.items():
+ for pattern in entry["patterns"]:
+ if pattern.strip() in self.FORBIDDEN_BARE_PATTERNS:
+ offenders.append((category_id, pattern))
+ self.assertEqual(
+ offenders,
+ [],
+ msg=(
+ "Bare common-noun patterns cause false positives. "
+ f"Offenders: {offenders}. Patterns must be compound "
+ "(e.g. 'image generation') or domain-specific "
+ "(e.g. 'midjourney')."
+ ),
+ )
+
+ def test_every_category_has_at_least_one_compound_or_brand_pattern(self):
+ multi_word_or_brand = re.compile(r"(\s|-|\.)|^[a-z][a-z0-9]{3,}$")
+ for category_id, entry in CATEGORY_PEERS.items():
+ patterns = entry["patterns"]
+ self.assertTrue(patterns, f"{category_id} has no patterns")
+ has_strong = any(multi_word_or_brand.search(p) for p in patterns)
+ self.assertTrue(
+ has_strong,
+ f"{category_id} needs at least one multi-word or brand pattern",
+ )
+
+ def test_every_category_has_at_least_two_peer_subs(self):
+ for category_id, entry in CATEGORY_PEERS.items():
+ self.assertGreaterEqual(
+ len(entry["peer_subs"]),
+ 2,
+ f"{category_id} should list at least 2 peer subs",
+ )
+
+ def test_category_count_is_in_expected_range(self):
+ # Sanity check: the map is intentionally small and curated.
+ self.assertGreaterEqual(len(CATEGORY_PEERS), 8)
+ self.assertLessEqual(len(CATEGORY_PEERS), 20)
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/tests/test_category_integration.py b/tests/test_category_integration.py
new file mode 100644
index 00000000..513c193a
--- /dev/null
+++ b/tests/test_category_integration.py
@@ -0,0 +1,145 @@
+"""End-to-end regression test for the 2026-04-22 `Prompting GPT Image 2` bug.
+
+Guards the failing run's Resolved-block shape end-to-end: stubs
+`grounding.web_search` to return the OpenAI-only subs that caused the
+original failure, then asserts that `auto_resolve` now returns the widened
+list and emits the expected stderr trace.
+
+If this test starts failing after a `scripts/lib/categories.py` edit, either
+the fix regressed or the map intentionally dropped the `ai_image_generation`
+category — update the test deliberately.
+
+Fixture reference: `tests/fixtures/prompting-gpt-image-2-resolved-block.md`.
+"""
+
+import io
+import sys
+import unittest
+from contextlib import redirect_stderr
+from pathlib import Path
+from unittest.mock import patch
+
+sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
+
+from lib import resolve
+
+
+OPENAI_BRAND_SUBREDDIT_RESULTS = [
+ {
+ "title": "r/OpenAI community hub",
+ "snippet": "Discussion at r/ChatGPT and r/singularity about GPT Image 2.",
+ "url": "https://reddit.com/r/OpenAI/",
+ },
+ {
+ "title": "r/ChatGPTpromptengineering prompt collection",
+ "snippet": "Also see r/artificial for broader AI chatter.",
+ "url": "",
+ },
+]
+
+EMPTY_RESULTS: list[dict] = []
+
+
+def _fake_websearch(label_to_items: dict[str, list[dict]]):
+ def _search(query, date_range, config):
+ if "subreddit" in query:
+ return label_to_items.get("subreddit", EMPTY_RESULTS), {}
+ if "news" in query:
+ return label_to_items.get("news", EMPTY_RESULTS), {}
+ if "handle" in query:
+ return label_to_items.get("x_handle", EMPTY_RESULTS), {}
+ if "github" in query:
+ return label_to_items.get("github", EMPTY_RESULTS), {}
+ return EMPTY_RESULTS, {}
+
+ return _search
+
+
+class PromptingGptImage2RegressionGuard(unittest.TestCase):
+ """The named 2026-04-22 failure mode. Resolved block must include peers."""
+
+ @patch("lib.resolve.grounding.web_search")
+ def test_auto_resolve_widens_to_image_gen_peers(self, mock_search):
+ mock_search.side_effect = _fake_websearch({
+ "subreddit": OPENAI_BRAND_SUBREDDIT_RESULTS,
+ })
+
+ result = resolve.auto_resolve(
+ "Prompting GPT Image 2",
+ {"BRAVE_API_KEY": "fake"},
+ )
+
+ subs_lower = [s.lower() for s in result["subreddits"]]
+
+ # Original WebSearch-returned brand subs preserved
+ self.assertIn("openai", subs_lower)
+ self.assertIn("chatgpt", subs_lower)
+ self.assertIn("singularity", subs_lower)
+
+ # At least three of the image-gen peers were added
+ expected_peers = {"stablediffusion", "midjourney", "dalle2", "aiart", "promptengineering"}
+ found_peers = expected_peers.intersection(subs_lower)
+ self.assertGreaterEqual(
+ len(found_peers),
+ 3,
+ f"Expected at least 3 image-gen peer subs, found: {found_peers}. "
+ f"Actual subs: {result['subreddits']}",
+ )
+
+ self.assertEqual(result["category"], "ai_image_generation")
+
+ @patch("lib.resolve.grounding.web_search")
+ def test_stderr_contains_category_match_log_line(self, mock_search):
+ mock_search.side_effect = _fake_websearch({
+ "subreddit": OPENAI_BRAND_SUBREDDIT_RESULTS,
+ })
+
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ resolve.auto_resolve(
+ "Prompting GPT Image 2",
+ {"BRAVE_API_KEY": "fake"},
+ )
+
+ self.assertIn("Matched category=ai_image_generation", buf.getvalue())
+
+ @patch("lib.resolve.grounding.web_search")
+ def test_cap_enforced_end_to_end(self, mock_search):
+ # Synthesize a subreddit response with 9 brand subs
+ many_subs_items = [
+ {"title": f"r/Brand{i}", "snippet": "", "url": ""}
+ for i in range(9)
+ ]
+ mock_search.side_effect = _fake_websearch({
+ "subreddit": many_subs_items,
+ })
+
+ result = resolve.auto_resolve(
+ "Prompting GPT Image 2",
+ {"BRAVE_API_KEY": "fake"},
+ )
+
+ self.assertLessEqual(len(result["subreddits"]), resolve.MAX_SUBS)
+ # The first WebSearch sub is still present (brand subs never evicted)
+ self.assertIn("Brand0", result["subreddits"])
+
+ @patch("lib.resolve.grounding.web_search")
+ def test_uncategorized_topic_does_not_inject_peers(self, mock_search):
+ mock_search.side_effect = _fake_websearch({
+ "subreddit": [{"title": "r/Kanye is wild", "snippet": "", "url": ""}],
+ })
+
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ result = resolve.auto_resolve(
+ "Kanye West latest album",
+ {"BRAVE_API_KEY": "fake"},
+ )
+
+ self.assertEqual(result["subreddits"], ["Kanye"])
+ self.assertIsNone(result["category"])
+ self.assertNotIn("Matched category=", buf.getvalue())
+
+
+if __name__ == "__main__":
+ unittest.main()
diff --git a/tests/test_resolve.py b/tests/test_resolve.py
index 76902632..f497fde5 100644
--- a/tests/test_resolve.py
+++ b/tests/test_resolve.py
@@ -1,11 +1,14 @@
+import io
import sys
import unittest
+from contextlib import redirect_stderr
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "scripts"))
from lib import resolve
+from lib.resolve import MAX_SUBS, _merge_category_peers
class TestHasBackend(unittest.TestCase):
@@ -137,8 +140,8 @@ class TestAutoResolve(unittest.TestCase):
self.assertEqual(result["subreddits"], ["technology", "gadgets"])
self.assertEqual(result["x_handle"], "techco")
self.assertIn("breakthrough", result["context"])
- self.assertEqual(result["searches_run"], 3)
- self.assertEqual(mock_search.call_count, 3)
+ self.assertEqual(result["searches_run"], 4)
+ self.assertEqual(mock_search.call_count, 4)
@patch("lib.resolve.grounding.web_search")
def test_search_failure_graceful(self, mock_search):
@@ -167,8 +170,180 @@ class TestAutoResolve(unittest.TestCase):
self.assertEqual(result["subreddits"], ["cooking"])
# News search failed, so context is empty
self.assertEqual(result["context"], "")
- # 2 out of 3 succeeded
- self.assertEqual(result["searches_run"], 2)
+ # 3 out of 4 succeeded (subreddit, x_handle, github; news failed)
+ self.assertEqual(result["searches_run"], 3)
+
+
+class MergeCategoryPeersHappyPath(unittest.TestCase):
+ def test_image_gen_topic_appends_peers(self):
+ merged, category = _merge_category_peers(
+ "Prompting GPT Image 2",
+ ["OpenAI", "ChatGPT", "singularity"],
+ )
+ self.assertEqual(category, "ai_image_generation")
+ self.assertIn("OpenAI", merged)
+ self.assertIn("ChatGPT", merged)
+ self.assertIn("singularity", merged)
+ self.assertIn("StableDiffusion", merged)
+ self.assertIn("midjourney", merged)
+ self.assertIn("dalle2", merged)
+
+ def test_preserves_websearch_order_then_appends_peers(self):
+ merged, _ = _merge_category_peers(
+ "Prompting GPT Image 2",
+ ["OpenAI", "ChatGPT"],
+ )
+ self.assertEqual(merged[0], "OpenAI")
+ self.assertEqual(merged[1], "ChatGPT")
+ self.assertEqual(merged[2], "StableDiffusion")
+
+ def test_emits_stderr_log_when_peers_added(self):
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ _merge_category_peers(
+ "Prompting GPT Image 2",
+ ["OpenAI", "ChatGPT"],
+ )
+ output = buf.getvalue()
+ self.assertIn("Matched category=ai_image_generation", output)
+ self.assertIn("StableDiffusion", output)
+
+
+class MergeCategoryPeersDedupe(unittest.TestCase):
+ def test_peer_already_in_websearch_not_duplicated(self):
+ merged, _ = _merge_category_peers(
+ "midjourney v7 prompts",
+ ["midjourney", "aiArt"],
+ )
+ self.assertEqual(
+ sum(1 for s in merged if s.lower() == "midjourney"),
+ 1,
+ )
+
+ def test_dedupe_is_case_insensitive(self):
+ merged, _ = _merge_category_peers(
+ "Prompting GPT Image 2",
+ ["STABLEDIFFUSION"],
+ )
+ lower = [s.lower() for s in merged]
+ self.assertEqual(lower.count("stablediffusion"), 1)
+
+ def test_no_log_when_all_peers_already_present(self):
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ _merge_category_peers(
+ "Prompting GPT Image 2",
+ [
+ "StableDiffusion",
+ "midjourney",
+ "dalle2",
+ "aiArt",
+ "PromptEngineering",
+ "MediaSynthesis",
+ ],
+ )
+ self.assertNotIn("Matched category=", buf.getvalue())
+
+
+class MergeCategoryPeersEdgeCases(unittest.TestCase):
+ def test_topic_with_no_category_returns_unchanged(self):
+ merged, category = _merge_category_peers(
+ "Kanye West",
+ ["Kanye", "hiphopheads"],
+ )
+ self.assertIsNone(category)
+ self.assertEqual(merged, ["Kanye", "hiphopheads"])
+
+ def test_empty_subreddit_list_with_category_still_adds_peers(self):
+ merged, category = _merge_category_peers("Prompting GPT Image 2", [])
+ self.assertEqual(category, "ai_image_generation")
+ self.assertIn("StableDiffusion", merged)
+
+ def test_empty_topic_returns_unchanged(self):
+ merged, category = _merge_category_peers("", ["foo", "bar"])
+ self.assertIsNone(category)
+ self.assertEqual(merged, ["foo", "bar"])
+
+ def test_none_topic_returns_unchanged(self):
+ merged, category = _merge_category_peers(None, ["foo", "bar"])
+ self.assertIsNone(category)
+ self.assertEqual(merged, ["foo", "bar"])
+
+ def test_no_log_when_topic_has_no_category(self):
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ _merge_category_peers("Kanye West", ["Kanye"])
+ self.assertNotIn("Matched category=", buf.getvalue())
+
+
+class MergeCategoryPeersCap(unittest.TestCase):
+ def test_cap_is_enforced_at_max_subs(self):
+ websearch_subs = [f"Sub{i}" for i in range(9)]
+ merged, _ = _merge_category_peers(
+ "Prompting GPT Image 2",
+ websearch_subs,
+ )
+ self.assertEqual(len(merged), MAX_SUBS)
+ for s in websearch_subs:
+ self.assertIn(s, merged)
+ self.assertEqual(len(merged) - len(websearch_subs), 1)
+ self.assertEqual(merged[9], "StableDiffusion")
+
+ def test_cap_preserves_highest_priority_peer_when_trimming(self):
+ websearch_subs = [f"Sub{i}" for i in range(8)]
+ merged, _ = _merge_category_peers(
+ "Prompting GPT Image 2",
+ websearch_subs,
+ )
+ self.assertEqual(len(merged), MAX_SUBS)
+ self.assertEqual(merged[8], "StableDiffusion")
+ self.assertEqual(merged[9], "midjourney")
+
+
+class MergeCategoryPeersClassificationFailure(unittest.TestCase):
+ def test_classification_error_returns_unwidened_list_and_logs(self):
+ original = resolve.categories.detect_category
+
+ def boom(_topic):
+ raise RuntimeError("synthetic classifier failure")
+
+ resolve.categories.detect_category = boom
+ try:
+ buf = io.StringIO()
+ with redirect_stderr(buf):
+ merged, category = _merge_category_peers(
+ "Prompting GPT Image 2",
+ ["OpenAI"],
+ )
+ self.assertEqual(merged, ["OpenAI"])
+ self.assertIsNone(category)
+ self.assertIn("Category classification failed", buf.getvalue())
+ finally:
+ resolve.categories.detect_category = original
+
+
+class AutoResolveCategoryIntegration(unittest.TestCase):
+ @patch("lib.resolve.grounding.web_search")
+ def test_auto_resolve_returns_category_key(self, mock_search):
+ def side_effect(query, date_range, config):
+ if "subreddit" in query:
+ return [
+ {"title": "r/OpenAI", "snippet": "r/ChatGPT r/singularity", "url": ""},
+ ], {}
+ return [], {}
+
+ mock_search.side_effect = side_effect
+ result = resolve.auto_resolve(
+ "Prompting GPT Image 2",
+ {"BRAVE_API_KEY": "fake"},
+ )
+ self.assertEqual(result["category"], "ai_image_generation")
+ self.assertIn("StableDiffusion", result["subreddits"])
+ self.assertIn("OpenAI", result["subreddits"])
+
+ def test_no_backend_returns_category_none(self):
+ result = resolve.auto_resolve("test topic", {})
+ self.assertIsNone(result["category"])
if __name__ == "__main__":