AI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary
Python ai-promptsai-skillblueskyclaudeclaude-codeclawhubdeep-researchhackernewsinstagramopenclawpolymarketrecencyredditresearchsocial-mediatiktoktrendstwitterweb-searchyoutube
feat(reddit): free discovery (dedicated subs + RSS) + arctic-shift scores + SC thinness floor (#696)
Matt Van Horn4bd20f3f4beb
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 7ef3b04d..933af6c6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+### Added++- Free Reddit gets dedicated-subreddit lanes: entity-home subs (e.g. r/Kanye for "Kanye West", via the new `--dedicated-subreddits` flag) are pulled in full from top+hot+new listings and exempt from the relevance floor, since the whole sub is the topic. Fixes the over-aggressive floor that dropped on-topic posts whose titles lacked the entity name.+- `reddit_arctic` resolves upvote counts for threads found only via RSS search (which carries no score) using the free, keyless arctic-shift archive — batched, paced, cached, and graceful-degrading. Reddit now gets headlines-with-points and best-comments-with-points entirely for free, at parity with ScrapeCreators.+- `LAST30DAYS_REDDIT_SC_MIN_ITEMS` (default 0 = unchanged empty-only behavior): set above 0 to let the ScrapeCreators backup backfill a thin free Reddit run instead of sitting idle. Backfilled items merge deduped by post id.++### Removed++- The permanently-403 `search.json` Tier 0 is gone from the keyless Reddit path; discovery is RSS breadth + shreddit listing partials (real scores) + the dedicated-sub lanes, with no wasted 403 calls.+## [3.8.2] - 2026-06-25### Addeddiff --git a/skills/last30days/SKILL.md b/skills/last30days/SKILL.mdindex 535b33ae..3d2dcaf9 100644--- a/skills/last30days/SKILL.md+++ b/skills/last30days/SKILL.md@@ -987,6 +987,11 @@ The first search finds subreddits. The second gives you current events context (Extract 3-5 subreddit names from the results. Store as `RESOLVED_SUBREDDITS` (comma-separated, no r/ prefix).+**Dedicated vs broad subreddits.** Split the resolved subs into two buckets:+- **Dedicated** = subreddits whose entire purpose IS the topic (the entity's home: `r/Kanye` / `r/WestSubEver` / `r/GoodAssSub` for "Kanye West", `r/OpenClaw` for OpenClaw). Every post there is on-topic. Store as `RESOLVED_DEDICATED_SUBREDDITS` and pass via `--dedicated-subreddits`. The engine pulls these in full (top+hot+new) and skips the relevance floor for them, so an on-topic post whose title lacks the entity name (a "BULLY Deluxe" thread in r/Kanye) is not dropped.+- **Broad** = mixed-content communities where the topic is only sometimes discussed (`r/hiphopheads`, `r/Music`, category peers from 2a). Store as `RESOLVED_SUBREDDITS` and pass via `--subreddits`. These stay relevance-floored.+Label conservatively: only a sub clearly named for / dedicated to the entity goes in the dedicated bucket. Most topics have 0-3 dedicated subs (people and products often have one; generic concepts have none). When unsure, treat it as broad.+**2a. Category-peer expansion (MANDATORY for product topics).** If the topic is a product in a recognizable category (AI image generation, AI video generation, AI coding agents, AI music, AI chat models, SaaS screen recording, prediction markets, etc.), the brand-specific subreddits that WebSearch returned are INSUFFICIENT. Add 2-3 peer subreddits from the category. Peer subs are where cross-product technique discussion actually lives. Missing them is the 2026-04-22 `GPT Image 2` failure mode: the model resolved `r/OpenAI, r/ChatGPT, r/singularity, r/ChatGPTpromptengineering` (all OpenAI-brand) and missed `r/StableDiffusion, r/midjourney, r/dalle2, r/aiArt` where prompting techniques are actually shared. The user had to manually prompt "check image generation reddits too" to get a usable run.Canonical category peers (single source of truth; `scripts/lib/categories.py` mirrors this for the `--auto-resolve` engine path):@@ -1252,7 +1257,8 @@ Then add to the engine command:- `--plan "$QUERY_PLAN_FILE"` (path to the file you just wrote)- `--x-handle={RESOLVED_HANDLE}` (from Step 0.5)-- `--subreddits={RESOLVED_SUBREDDITS}` (from Step 0.55)+- `--subreddits={RESOLVED_SUBREDDITS}` (broad/category subs, from Step 0.55)+- `--dedicated-subreddits={RESOLVED_DEDICATED_SUBREDDITS}` (entity-home subs, from Step 0.55; pulled in full + floor-exempt)- `--tiktok-hashtags={RESOLVED_HASHTAGS}` (from Step 0.55)- `--tiktok-creators={RESOLVED_TIKTOK_CREATORS}` (from Step 0.55)- `--ig-creators={RESOLVED_IG_CREATORS}` (from Step 0.55)diff --git a/skills/last30days/scripts/last30days.py b/skills/last30days/scripts/last30days.pyindex 0e57f0fb..3a05b448 100644--- a/skills/last30days/scripts/last30days.py+++ b/skills/last30days/scripts/last30days.py@@ -356,7 +356,8 @@ def build_parser() -> argparse.ArgumentParser:help="Analyze public jobs/careers postings as evidence-backed company focus signals.")parser.add_argument("--plan", help="JSON query plan (skips internal LLM planner). Can be a JSON string or a file path.")parser.add_argument("--save-suffix", help="Suffix for saved output filename (e.g., 'gemini' → kanye-west-raw-gemini.md)")- parser.add_argument("--subreddits", help="Comma-separated subreddit names to search (e.g., SaaS,Entrepreneur)")+ parser.add_argument("--subreddits", help="Comma-separated broad/category subreddit names to search (e.g., SaaS,Entrepreneur)")+ parser.add_argument("--dedicated-subreddits", help="Comma-separated entity-home subreddit names (e.g., Kanye,WestSubEver). Pulled in full (top+hot+new) and exempt from the relevance floor since the whole sub is the topic.")parser.add_argument("--tiktok-hashtags", help="Comma-separated TikTok hashtags without # (e.g., tella,screenrecording)")parser.add_argument("--tiktok-creators", help="Comma-separated TikTok creator handles (e.g., TellaHQ,taborplace)")parser.add_argument("--ig-creators", help="Comma-separated Instagram creator handles (e.g., tella.tv,laborstories)")@@ -1080,6 +1081,7 @@ def main() -> int:try:x_related = [h.strip() for h in args.x_related.split(",") if h.strip()] if args.x_related else Nonesubreddits = [s.strip().removeprefix("r/") for s in args.subreddits.split(",") if s.strip()] if args.subreddits else None+ dedicated_subreddits = [s.strip().removeprefix("r/") for s in args.dedicated_subreddits.split(",") if s.strip()] if args.dedicated_subreddits else Nonetiktok_hashtags = [h.strip().lstrip("#") for h in args.tiktok_hashtags.split(",") if h.strip()] if args.tiktok_hashtags else Nonetiktok_creators = [c.strip().lstrip("@") for c in args.tiktok_creators.split(",") if c.strip()] if args.tiktok_creators else Noneig_creators = [c.strip().lstrip("@") for c in args.ig_creators.split(",") if c.strip()] if args.ig_creators else None@@ -1193,6 +1195,12 @@ def main() -> int:f"{' vs '.join(vs_entities)}\n")+ # Dedicated subs ride the config dict (already threaded to every source+ # fetch) so the keyless Reddit path can pull them floor-exempt without+ # widening pipeline.run / _retrieve_stream signatures.+ if dedicated_subreddits:+ config["_dedicated_subreddits"] = dedicated_subreddits+def _main_runner() -> schema.Report:r = pipeline.run(topic=topic,diff --git a/skills/last30days/scripts/lib/env.py b/skills/last30days/scripts/lib/env.pyindex 7f208e2f..c19b2de3 100644--- a/skills/last30days/scripts/lib/env.py+++ b/skills/last30days/scripts/lib/env.py@@ -507,6 +507,7 @@ def get_config(policy: ConfigLoadPolicy | None = None) -> dict[str, Any]:('LAST30DAYS_X_MODEL', None),('LAST30DAYS_X_BACKEND', None),('LAST30DAYS_REDDIT_BACKEND', None),+ ('LAST30DAYS_REDDIT_SC_MIN_ITEMS', None),('LAST30DAYS_STORE', None),('LAST30DAYS_MEMORY_DIR', None),('OPENAI_MODEL_PIN', None),diff --git a/skills/last30days/scripts/lib/pipeline.py b/skills/last30days/scripts/lib/pipeline.pyindex 5c925cfe..5704378a 100644--- a/skills/last30days/scripts/lib/pipeline.py+++ b/skills/last30days/scripts/lib/pipeline.py@@ -1155,6 +1155,29 @@ def _fetch_x_backend(backend, subquery, from_date, to_date, depth, config):return items, (err or "")+def _reddit_post_key(item: dict) -> str:+ """Stable per-thread dedupe key (base36 post id from the url/permalink)."""+ url = item.get("url") or item.get("permalink") or ""+ m = re.search(r"/comments/([A-Za-z0-9]+)", url)+ return m.group(1) if m else url+++def _merge_reddit_items(free: list[dict], sc: list[dict]) -> list[dict]:+ """Merge free + ScrapeCreators Reddit items, free first, deduped by post id.++ Used when the thinness-floor trigger backfills a thin free run with SC, so a+ thread present in both is never double-listed.+ """+ merged = list(free)+ seen = {_reddit_post_key(it) for it in free}+ for it in sc:+ key = _reddit_post_key(it)+ if key and key not in seen:+ seen.add(key)+ merged.append(it)+ return merged++def _retrieve_stream(*,topic: str,@@ -1196,6 +1219,7 @@ def _retrieve_stream(# Use raw_topic so expand_reddit_queries() generates diverse variants# from the original user topic, not the planner's narrowed search_query.reddit_query = raw_topic or subquery.search_query+ dedicated_subreddits = config.get("_dedicated_subreddits") or Nonehas_sc_key = bool(config.get("SCRAPECREATORS_API_KEY"))sc_first = (has_sc_key@@ -1240,14 +1264,20 @@ def _retrieve_stream()return [], {}- # Default: public Reddit first (free, gets comments); SC as backup+ # Default: public Reddit first (free). ScrapeCreators backfills when the+ # free path is empty OR returns fewer than the configured thinness floor+ # (LAST30DAYS_REDDIT_SC_MIN_ITEMS, default 0 = empty-only — today's+ # behavior, no extra credit spend unless the user opts in).+ try:+ min_items = int(config.get("LAST30DAYS_REDDIT_SC_MIN_ITEMS") or 0)+ except (TypeError, ValueError):+ min_items = 0+ public_results: list[dict] = []try:public_results = reddit_public.search_reddit_public(reddit_query, from_date, to_date, depth=depth,- subreddits=subreddits,- )- if public_results:- return public_results, {}+ subreddits=subreddits, dedicated_subreddits=dedicated_subreddits,+ ) or []except Exception as exc:sys.stderr.write(f"[Reddit] Public search failed ({type(exc).__name__}: {exc})"@@ -1256,20 +1286,30 @@ def _retrieve_stream(sys.stderr.write("\n")return [], {}sys.stderr.write(", using ScrapeCreators backup\n")- if has_sc_key:- try:- result = reddit.search_and_enrich(- reddit_query, from_date, to_date, depth=depth,- token=config.get("SCRAPECREATORS_API_KEY"),- subreddits=subreddits,- )- return reddit.parse_reddit_response(result), {}- except Exception as exc:- sys.stderr.write(- f"[Reddit] ScrapeCreators backup also failed "- f"({type(exc).__name__}: {exc})\n"- )- return [], {}+ # Enough free results, or no key to backfill with -> done. max(min_items,+ # 1) keeps the default (min_items=0) as empty-only AND treats exactly+ # `min_items` results as acceptable (no backfill) for min_items > 0.+ if len(public_results) >= max(min_items, 1) or not has_sc_key:+ return public_results, {}+ if public_results:+ sys.stderr.write(+ f"[Reddit] Free path returned {len(public_results)} "+ f"(below the {min_items}-item floor); backfilling with ScrapeCreators\n"+ )+ try:+ result = reddit.search_and_enrich(+ reddit_query, from_date, to_date, depth=depth,+ token=config.get("SCRAPECREATORS_API_KEY"),+ subreddits=subreddits,+ )+ sc_items = reddit.parse_reddit_response(result)+ except Exception as exc:+ sys.stderr.write(+ f"[Reddit] ScrapeCreators backup also failed "+ f"({type(exc).__name__}: {exc})\n"+ )+ return public_results, {}+ return _merge_reddit_items(public_results, sc_items), {}if source == "x":# One X source, an ordered chain of interchangeable backends. Try the# primary; fall through to the next only if it returns nothing or errors.diff --git a/skills/last30days/scripts/lib/reddit_arctic.py b/skills/last30days/scripts/lib/reddit_arctic.pynew file mode 100644index 00000000..ebd8ee3c--- /dev/null+++ b/skills/last30days/scripts/lib/reddit_arctic.py@@ -0,0 +1,92 @@+"""Arctic-shift score resolver — post upvote counts by id, keyless and free.++``search.json`` and ``/comments/{id}.json`` are 403 keyless, and ``search.rss``+(used for discovery) carries titles but NO score; the shreddit listing partials+score only posts that appear in a pulled listing. For a thread found only via+global RSS search in a broad sub, the free score comes from arctic-shift+(https://arctic-shift.photon-reddit.com), a public Reddit archive whose+``/api/posts/ids`` returns the post object (score, num_comments, title) for a+batch of base36 post ids. Scores are point-in-time snapshots — slightly stale vs+live, which is fine for ranking and display.++Best-effort, never raises. On rate-limit (HTTP 422 "slow down"), error, or an+unreachable host it returns ``{}`` so the caller shows the thread without a point+count rather than failing the Reddit source.+"""++import sys+import time+from typing import Dict, List++from . import http++API = "https://arctic-shift.photon-reddit.com/api/posts/ids"+BATCH = 50 # ids per request+TIMEOUT = 15+MAX_BATCHES = 3 # cap total requests per run (bounds latency + rate-limit risk)+PACE_SECONDS = 0.4 # gap between batches; arctic-shift answers 422 "slow down"+CACHE_MAX = 4096 # hard size bound so the in-run memo can never grow unbounded+# In-run memo: base36 id -> {score, num_comments}. Module-level so repeated+# fetch_scores calls within one `/last30days` run (e.g. across subqueries) reuse+# results, but capped at CACHE_MAX entries (never reached in a normal CLI run).+# Tests clear it via reddit_arctic._cache.clear().+_cache: Dict[str, Dict[str, int]] = {}+++def _log(msg: str) -> None:+ sys.stderr.write(f"[ArcticShift] {msg}\n")+ sys.stderr.flush()+++def fetch_scores(post_ids: List[str]) -> Dict[str, Dict[str, int]]:+ """Return ``{base36_post_id: {"score", "num_comments"}}`` for the given ids.++ Batched, paced, in-run cached, and never raises. Ids that fail or are absent+ from the archive are simply missing from the result (caller degrades to no+ point count for those threads).+ """+ out: Dict[str, Dict[str, int]] = {}+ todo: List[str] = []+ for pid in post_ids:+ if not pid:+ continue+ if pid in _cache:+ out[pid] = _cache[pid]+ elif pid not in todo:+ todo.append(pid)++ batches = [todo[i:i + BATCH] for i in range(0, len(todo), BATCH)][:MAX_BATCHES]+ for n, batch in enumerate(batches):+ if n:+ time.sleep(PACE_SECONDS)+ try:+ data = http.get(+ f"{API}?ids={','.join(batch)}",+ headers={"User-Agent": http.BROWSER_USER_AGENT},+ timeout=TIMEOUT,+ )+ except Exception as e: # network error / non-200 — degrade, never raise+ _log(f"lookup failed ({e}); {len(batch)} ids left unscored")+ break+ rows = (data or {}).get("data")+ if not isinstance(rows, list):+ # arctic-shift returns {"error": "..."} on rate-limit / bad request.+ _log(f"unexpected response (rate-limited?): {str(data)[:80]}")+ break+ for row in rows:+ if not isinstance(row, dict):+ continue+ rid = str(row.get("id") or "").removeprefix("t3_")+ if not rid:+ continue+ try:+ entry = {+ "score": int(row.get("score") or 0),+ "num_comments": int(row.get("num_comments") or 0),+ }+ except (TypeError, ValueError):+ continue+ if len(_cache) < CACHE_MAX:+ _cache[rid] = entry+ out[rid] = entry+ return outdiff --git a/skills/last30days/scripts/lib/reddit_keyless.py b/skills/last30days/scripts/lib/reddit_keyless.pyindex 7050b165..a377f573 100644--- a/skills/last30days/scripts/lib/reddit_keyless.py+++ b/skills/last30days/scripts/lib/reddit_keyless.py@@ -1,17 +1,19 @@-"""Keyless Reddit pipeline: tiered free search + comment enrichment.+"""Keyless Reddit pipeline: free discovery + comment enrichment.-Replaces the dead ``.json`` free path. Discovery tiers, cheapest/most-likely-first; enrichment then runs on whatever was discovered:+``search.json`` is permanently 403/429 keyless, so it is not used. Discovery+runs on the surfaces that still serve data without a key, then enrichment runs+on whatever was discovered:- Tier 0 one-shot legacy ``.json`` search — demoted. Datacenter IPs get 403,- but a residential machine (where the skill usually runs) may still- get 200, so it is worth one cheap try. Honors the "brute-force .json"- intent without depending on it.- Tier 1 RSS discovery (reddit_rss) — keyless, robust, the load-bearing path.- Tier 2 shreddit comment + count enrichment (reddit_shreddit) for top posts.+ Dedicated lane entity-home subreddits (e.g. r/Kanye) pulled in full via the+ shreddit listing partials (top+hot+new, real scores), kept+ whole — floor-exempt — because the sub IS the topic.+ RSS lane reddit_rss breadth (incl. global keyword search) + broad-sub+ listing partials for real upvote scores. Relevance-floored.+ Enrichment shreddit comment + count enrichment (reddit_shreddit) for the+ top-ranked posts (author + score + text + permalink).Returns ``[]`` (never raises) so ``pipeline.py`` can fall through to the-ScrapeCreators backup when every keyless tier comes up empty.+ScrapeCreators backup when every keyless lane comes up empty."""import concurrent.futures@@ -22,7 +24,7 @@ from typing import Any, Dict, List, Optionalfrom collections import Counter-from . import reddit_rss, reddit_shreddit, reddit_listing+from . import reddit_rss, reddit_shreddit, reddit_listing, reddit_arctic# Scores are backfilled from popular derived subreddits, so an engagement-first# final sort buries on-topic RSS hits under viral off-topic posts. A relevance# floor + relevance-first final ranking keeps the section on-topic. Thresholds@@ -33,6 +35,10 @@ ENRICH_LIMITS = reddit_shreddit.ENRICH_LIMITSENRICH_BUDGET = 45 # seconds total across all enrichment threadsMAX_ENRICH_WORKERS = 4MAX_DERIVED_SUBS = 5 # subreddits derived from RSS results for score backfill+# Dedicated subreddits (the entity's home, e.g. r/Kanye for "Kanye West") are+# wholly on-topic, so pull top+hot+new — the top-of-month listing alone misses+# fresh threads — and keep every item (floor-exempt).+DEDICATED_SORTS = ["top", "hot", "new"]def _relevance_rank_key(post: Dict[str, Any]) -> float:@@ -52,16 +58,6 @@ def _log(msg: str) -> None:sys.stderr.flush()-def _tier0_json(topic: str, depth: str) -> List[Dict[str, Any]]:- """One cheap global ``.json`` discovery attempt. Returns [] on the 403 wall."""- try:- from . import reddit_public- return reddit_public.search(topic, depth=depth) or []- except Exception as e: # never let the demoted tier sink the run- _log(f"Tier 0 (.json) unavailable: {e}")- return []--def _top_subreddits(posts: List[Dict[str, Any]], limit: int = MAX_DERIVED_SUBS) -> List[str]:"""Most frequent subreddits across discovered posts (for score backfill)."""counts = Counter(p.get("subreddit", "") for p in posts if p.get("subreddit"))@@ -75,15 +71,27 @@ def _apply_scores(post: Dict[str, Any], scored: Dict[str, int]) -> None:post["engagement"]["num_comments"] = scored["num_comments"]-def _discover(topic: str, depth: str, subreddits: Optional[List[str]]) -> List[Dict[str, Any]]:- # Tier 0: demoted one-shot .json (dead for normal users too, but free to try).- posts = _tier0_json(topic, depth)- if posts:- _log(f"Tier 0 (.json) returned {len(posts)} posts")- return posts-- # Tier 1: keyless discovery. RSS gives breadth (incl. global keyword search);- # the listing partials give real upvote scores.+def _discover(+ topic: str,+ depth: str,+ subreddits: Optional[List[str]],+ dedicated_subreddits: Optional[List[str]] = None,+) -> List[Dict[str, Any]]:+ # Dedicated lane: the entity's home subs are wholly on-topic. Pull+ # top+hot+new (real scores from the listing) and mark them floor-exempt so+ # an on-topic post whose title lacks the entity name is never dropped.+ dedicated_posts: List[Dict[str, Any]] = []+ if dedicated_subreddits:+ dedicated_posts = reddit_listing.fetch_listings(+ dedicated_subreddits, depth=depth, query=topic, sorts=DEDICATED_SORTS+ )+ for p in dedicated_posts:+ p["dedicated"] = True+ _log(f"Dedicated lane: {len(dedicated_posts)} posts from {dedicated_subreddits}")++ # search.json is permanently 403/429 keyless (no Tier 0). Discovery is RSS+ # breadth (incl. global keyword search) + broad-sub listing partials for+ # real upvote scores.rss_posts = reddit_rss.search_rss(topic, depth=depth, subreddits=subreddits)if subreddits:@@ -112,11 +120,13 @@ def _discover(topic: str, depth: str, subreddits: Optional[List[str]]) -> List[Dif pid:score_map[pid] = {"score": p["score"], "num_comments": p["num_comments"]}- # Merge: scored listing posts first (targeted only), then RSS breadth,- # backfilled with real scores where the post appears in a listing.+ # Merge: dedicated-sub posts first (floor-exempt), then scored broad listing+ # posts (targeted only), then RSS breadth backfilled with real scores where+ # the post appears in a listing. First writer wins the dedupe, so a thread+ # in both the dedicated lane and a listing keeps its floor-exempt status.merged: List[Dict[str, Any]] = []seen: set = set()- for p in listing_posts:+ for p in dedicated_posts + listing_posts:if p["url"] not in seen:seen.add(p["url"])merged.append(p)@@ -128,6 +138,25 @@ def _discover(topic: str, depth: str, subreddits: Optional[List[str]]) -> List[D_apply_scores(p, score_map[pid])seen.add(p["url"])merged.append(p)++ # Backfill scores for RSS-only posts (no listing card scored them) from the+ # free arctic-shift archive. Posts already scored by a listing keep that+ # live score; arctic only fills the gap, and is best-effort (never raises).+ need = [pid for p in merged+ if not (p.get("engagement", {}).get("score"))+ for pid in [reddit_listing._post_id(p["url"])] if pid]+ if need:+ scores = reddit_arctic.fetch_scores(need)+ filled = 0+ for p in merged:+ if p.get("engagement", {}).get("score"):+ continue+ pid = reddit_listing._post_id(p["url"])+ if pid in scores:+ _apply_scores(p, scores[pid])+ filled += 1+ if filled:+ _log(f"arctic-shift backfilled {filled} post scores")return merged@@ -228,22 +257,25 @@ def search_and_enrich(to_date: str,depth: str = "default",subreddits: Optional[List[str]] = None,+ dedicated_subreddits: Optional[List[str]] = None,) -> List[Dict[str, Any]]:- """Full keyless Reddit pipeline: discover (Tier 0/1) then enrich (Tier 2).+ """Full keyless Reddit pipeline: discover then enrich.Args:topic: Search topicfrom_date: Start date (YYYY-MM-DD)to_date: End date (YYYY-MM-DD)depth: 'quick', 'default', or 'deep'- subreddits: Optional pre-resolved subreddit names (without r/)+ subreddits: Optional pre-resolved broad/category subreddit names (no r/)+ dedicated_subreddits: Optional entity-home subreddit names (no r/) pulled+ in full (top+hot+new) and exempt from the relevance floor.Returns:List of normalized item dicts matching the reddit_public output shape,with top_comments/comment_insights attached on enriched posts.Empty list when all keyless tiers fail (so SC backup can engage)."""- posts = _discover(topic, depth, subreddits)+ posts = _discover(topic, depth, subreddits, dedicated_subreddits)if not posts:return []@@ -258,11 +290,13 @@ def search_and_enrich(# backfilled high-upvote posts from popular subs can't bury on-topic RSS# hits. Keep all only when nothing scored above zero.before = len(posts)- on_topic = [p for p in posts if (p.get("relevance") or 0) >= RELEVANCE_FLOOR]+ # Dedicated-sub posts are floor-exempt: their whole subreddit is the topic,+ # so an on-topic post whose title lacks the entity name must not be dropped.+ on_topic = [p for p in posts if p.get("dedicated") or (p.get("relevance") or 0) >= RELEVANCE_FLOOR]if len(on_topic) >= MIN_ON_TOPIC:posts = on_topicelse:- nonzero = [p for p in posts if (p.get("relevance") or 0) > 0]+ nonzero = [p for p in posts if p.get("dedicated") or (p.get("relevance") or 0) > 0]if nonzero:posts = nonzeroif len(posts) < before:diff --git a/skills/last30days/scripts/lib/reddit_listing.py b/skills/last30days/scripts/lib/reddit_listing.pyindex f3b92d2b..5cda23a0 100644--- a/skills/last30days/scripts/lib/reddit_listing.py+++ b/skills/last30days/scripts/lib/reddit_listing.py@@ -140,15 +140,20 @@ def fetch_listings(subreddits: List[str],depth: str = "default",query: str = "",+ sorts: Optional[List[str]] = None,) -> List[Dict[str, Any]]:- """Fetch scored post cards across subreddits × depth-appropriate sorts.+ """Fetch scored post cards across subreddits × sorts.Returns deduped normalized posts (with real scores), unranked/unsliced —the caller merges these with other sources, ranks, and slices.++ ``sorts`` overrides the depth-derived sort set. Dedicated-subreddit lanes+ pass ``["top", "hot", "new"]`` so fresh threads (which the top-of-month+ listing misses) are caught with their scores regardless of depth."""if not subreddits:return []- sorts = LISTING_SORTS.get(depth, LISTING_SORTS["default"])+ sorts = sorts or LISTING_SORTS.get(depth, LISTING_SORTS["default"])jobs = [(sub, sort) for sub in subreddits for sort in sorts]all_posts: List[Dict[str, Any]] = []with ThreadPoolExecutor(max_workers=min(MAX_WORKERS, len(jobs)) or 1) as executor:diff --git a/skills/last30days/scripts/lib/reddit_public.py b/skills/last30days/scripts/lib/reddit_public.pyindex b6789b06..ec8b1db2 100644--- a/skills/last30days/scripts/lib/reddit_public.py+++ b/skills/last30days/scripts/lib/reddit_public.py@@ -242,18 +242,20 @@ def search_reddit_public(to_date: str,depth: str = "default",subreddits: Optional[List[str]] = None,+ dedicated_subreddits: Optional[List[str]] = None,) -> List[Dict[str, Any]]:"""High-level free Reddit search + enrichment (keyless).- Thin compatibility shim over the tiered keyless pipeline: the legacy- ``.json`` search/enrichment endpoints now return HTTP 403, so this delegates- to ``reddit_keyless.search_and_enrich`` (Tier 0 one-shot ``.json`` →- Tier 1 RSS discovery → Tier 2 shreddit comment enrichment). The name and- signature are preserved so ``pipeline.py`` and other callers need no change- and the ScrapeCreators backup still engages when this returns empty.+ Thin compatibility shim over the keyless pipeline: the legacy ``.json``+ search/enrichment endpoints now return HTTP 403, so this delegates to+ ``reddit_keyless.search_and_enrich`` (dedicated-sub listings + RSS discovery+ → shreddit comment enrichment; no ``.json`` search). The name and signature+ are preserved so ``pipeline.py`` and other callers need no change and the+ ScrapeCreators backup still engages when this returns empty.- The module-level ``search`` / ``_parse_posts`` helpers remain in use as the- keyless pipeline's demoted Tier 0 ``.json`` attempt.+ The module-level ``search`` / ``_parse_posts`` helpers remain as a+ standalone ``.json`` search utility (own test coverage), no longer wired+ into the keyless production path.Args:topic: Search topic@@ -268,5 +270,6 @@ def search_reddit_public("""from . import reddit_keylessreturn reddit_keyless.search_and_enrich(- topic, from_date, to_date, depth=depth, subreddits=subreddits+ topic, from_date, to_date, depth=depth, subreddits=subreddits,+ dedicated_subreddits=dedicated_subreddits,)diff --git a/tests/conftest.py b/tests/conftest.pyindex 14195d18..9dc63d7d 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -1,4 +1,16 @@import sysfrom pathlib import Path+from unittest import mock++import pytestsys.path.insert(0, str(Path(__file__).resolve().parent.parent / "skills" / "last30days" / "scripts"))++[email protected](autouse=True)+def _no_arctic_network():+ """Default the arctic-shift score lookup to a no-op so the suite never makes+ a real network call. test_reddit_arctic overrides this fixture (same name) to+ exercise the real lookup with http.get mocked."""+ with mock.patch("lib.reddit_arctic.fetch_scores", return_value={}):+ yielddiff --git a/tests/test_codex_host_contract.py b/tests/test_codex_host_contract.pyindex 4542500e..bfccdf29 100644--- a/tests/test_codex_host_contract.py+++ b/tests/test_codex_host_contract.py@@ -114,3 +114,13 @@ def test_plan_invocation_warns_against_bash_lc_apostrophe_wrapper():text = SKILL_MD.read_text(encoding="utf-8")assert "bash -lc '...'" in textassert "unmatched" in text+++def test_step055_documents_dedicated_vs_broad_subreddits():+ # Step 0.55 must instruct the model to split entity-home (dedicated) subs from+ # broad subs and pass them via --dedicated-subreddits, which the engine pulls+ # in full and exempts from the relevance floor.+ text = SKILL_MD.read_text(encoding="utf-8")+ assert "RESOLVED_DEDICATED_SUBREDDITS" in text+ assert "--dedicated-subreddits" in text+ assert "relevance floor" in textdiff --git a/tests/test_reddit_arctic.py b/tests/test_reddit_arctic.pynew file mode 100644index 00000000..c4d6bdac--- /dev/null+++ b/tests/test_reddit_arctic.py@@ -0,0 +1,87 @@+"""Tests for reddit_arctic — keyless post-score lookup via arctic-shift (U6)."""++from unittest import mock++import pytest++from lib import reddit_arctic++[email protected](autouse=True)+def _no_arctic_network():+ # Override conftest's stub so this module exercises the real fetch_scores;+ # network is mocked per-test at http.get. Clear the in-run cache each time.+ reddit_arctic._cache.clear()+ yield+ reddit_arctic._cache.clear()+++def _resp(rows):+ return {"data": rows}+++class TestFetchScores:+ def test_returns_scores_by_id(self):+ with mock.patch.object(reddit_arctic.http, "get",+ return_value=_resp([{"id": "abc", "score": 1531, "num_comments": 336}])):+ out = reddit_arctic.fetch_scores(["abc"])+ assert out == {"abc": {"score": 1531, "num_comments": 336}}++ def test_strips_t3_prefix(self):+ with mock.patch.object(reddit_arctic.http, "get",+ return_value=_resp([{"id": "t3_xyz", "score": 5, "num_comments": 2}])):+ out = reddit_arctic.fetch_scores(["xyz"])+ assert out["xyz"]["score"] == 5++ def test_rate_limit_response_degrades(self):+ # arctic-shift answers {"error": "...slow down"} with no data list.+ with mock.patch.object(reddit_arctic.http, "get",+ return_value={"error": "Timeout. Maybe slow down a bit"}):+ out = reddit_arctic.fetch_scores(["abc"])+ assert out == {}++ def test_network_error_degrades(self):+ with mock.patch.object(reddit_arctic.http, "get", side_effect=Exception("boom")):+ out = reddit_arctic.fetch_scores(["abc"])+ assert out == {}++ def test_in_run_cache_avoids_refetch(self):+ with mock.patch.object(reddit_arctic.http, "get",+ return_value=_resp([{"id": "abc", "score": 9, "num_comments": 1}])) as g:+ reddit_arctic.fetch_scores(["abc"])+ reddit_arctic.fetch_scores(["abc"]) # served from cache, no 2nd call+ assert g.call_count == 1++ def test_dedupes_into_single_batch(self):+ with mock.patch.object(reddit_arctic.http, "get",+ return_value=_resp([{"id": "a", "score": 1, "num_comments": 0},+ {"id": "b", "score": 2, "num_comments": 0}])) as g:+ out = reddit_arctic.fetch_scores(["a", "b", "a", ""])+ assert g.call_count == 1+ assert set(out) == {"a", "b"}++ def test_cache_is_size_bounded(self):+ # The in-run memo never grows past CACHE_MAX; scores are still returned+ # for the current call once the cache is full.+ with mock.patch.object(reddit_arctic, "CACHE_MAX", 1), \+ mock.patch.object(reddit_arctic.http, "get",+ return_value=_resp([{"id": "a", "score": 1, "num_comments": 0},+ {"id": "b", "score": 2, "num_comments": 0}])):+ out = reddit_arctic.fetch_scores(["a", "b"])+ assert set(out) == {"a", "b"} # both returned+ assert len(reddit_arctic._cache) <= 1 # cache stayed bounded++ def test_empty_input_makes_no_call(self):+ with mock.patch.object(reddit_arctic.http, "get") as g:+ out = reddit_arctic.fetch_scores([])+ assert out == {}+ g.assert_not_called()++ def test_malformed_rows_skipped(self):+ rows = [{"id": "ok", "score": 7, "num_comments": 3},+ {"id": "", "score": 1}, # no id+ "not-a-dict", # junk+ {"id": "bad", "score": "x"}] # unparseable score+ with mock.patch.object(reddit_arctic.http, "get", return_value=_resp(rows)):+ out = reddit_arctic.fetch_scores(["ok", "bad"])+ assert out == {"ok": {"score": 7, "num_comments": 3}}diff --git a/tests/test_reddit_dedicated.py b/tests/test_reddit_dedicated.pynew file mode 100644index 00000000..56fc20cf--- /dev/null+++ b/tests/test_reddit_dedicated.py@@ -0,0 +1,99 @@+"""Tests for the dedicated-subreddit lane in reddit_keyless (U1).++A dedicated subreddit (the entity's home, e.g. r/Kanye for "Kanye West") is+wholly on-topic: pulled in full via top+hot+new listings and exempt from the+relevance floor, so an on-topic post whose title lacks the entity name is kept.+"""++from unittest import mock++from lib import reddit_keyless+++def _post(i, date="2026-05-20", rel=0.0):+ url = f"https://www.reddit.com/r/test/comments/{i:06d}/post_{i}/"+ return {+ "id": "", "title": f"Post {i}", "url": url, "score": 0, "num_comments": 0,+ "subreddit": "test", "created_utc": None, "author": "u", "selftext": "",+ "date": date, "engagement": {"score": 0, "num_comments": 0, "upvote_ratio": None},+ "relevance": rel, "why_relevant": "Reddit RSS", "metadata": {},+ }+++def _listing_post(i, score, title, rel=0.0):+ p = _post(i, rel=rel)+ p["title"] = title+ p["score"] = score+ p["engagement"]["score"] = score+ p["why_relevant"] = "Reddit listing"+ p["metadata"] = {"post_id": f"{i:06d}"}+ return p+++def _no_enrich():+ return mock.patch.object(+ reddit_keyless.reddit_shreddit, "fetch_comments",+ return_value={"top_comments": [], "comment_insights": [], "num_comments": None},+ )+++class TestDedicatedLane:+ def test_dedicated_listings_pulled_with_top_hot_new_and_marked(self):+ ded = _listing_post(1, 2643, "What the actual fuck is this ye?")+ captured = {}++ def fake_fetch(subs, depth="default", query="", sorts=None):+ if sorts == reddit_keyless.DEDICATED_SORTS:+ captured["sorts"] = sorts+ captured["subs"] = subs+ return [ded]+ return []++ with mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",+ side_effect=fake_fetch), \+ mock.patch.object(reddit_keyless.reddit_rss, "search_rss", return_value=[]):+ out = reddit_keyless._discover("Kanye West", "default", None,+ dedicated_subreddits=["Kanye"])+ assert captured["sorts"] == ["top", "hot", "new"]+ assert captured["subs"] == ["Kanye"]+ assert len(out) == 1+ assert out[0]["dedicated"] is True++ def test_dedicated_post_survives_floor_without_entity_name(self):+ # On-topic dedicated post whose title lacks "Kanye" (relevance 0) must be+ # kept; off-topic non-dedicated posts (relevance 0) are floored out.+ ded = _listing_post(1, 2284, "There's 2 types of people", rel=0.0)+ ded["dedicated"] = True+ offtopic = [_post(10 + i, rel=0.0) for i in range(5)]+ with mock.patch.object(reddit_keyless, "_discover", return_value=[ded] + offtopic), \+ _no_enrich():+ out = reddit_keyless.search_and_enrich("Kanye West", "2026-05-01", "2026-05-31")+ urls = {p["url"] for p in out}+ assert ded["url"] in urls # dedicated kept despite relevance 0+ assert all(o["url"] not in urls for o in offtopic) # off-topic floored++ def test_dedicated_dedup_keeps_floor_exempt_status(self):+ # A thread present in both the dedicated lane and a broad listing keeps+ # its dedicated (floor-exempt) flag — dedicated is merged first.+ shared_ded = _listing_post(1, 500, "fresh thread", rel=0.0)+ shared_broad = _listing_post(1, 500, "fresh thread", rel=0.0) # same url/id++ def fake_fetch(subs, depth="default", query="", sorts=None):+ return [shared_ded] if sorts == reddit_keyless.DEDICATED_SORTS else [shared_broad]++ with mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",+ side_effect=fake_fetch), \+ mock.patch.object(reddit_keyless.reddit_rss, "search_rss", return_value=[]):+ out = reddit_keyless._discover("Kanye West", "default", ["hiphopheads"],+ dedicated_subreddits=["Kanye"])+ same = [p for p in out if p["url"] == shared_ded["url"]]+ assert len(same) == 1+ assert same[0].get("dedicated") is True++ def test_no_dedicated_subs_is_noop(self):+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss",+ return_value=[_post(1)]), \+ mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",+ return_value=[]):+ out = reddit_keyless._discover("topic", "default", ["test"])+ assert all(not p.get("dedicated") for p in out)diff --git a/tests/test_reddit_dispatch.py b/tests/test_reddit_dispatch.pynew file mode 100644index 00000000..073d2336--- /dev/null+++ b/tests/test_reddit_dispatch.py@@ -0,0 +1,109 @@+"""Pipeline Reddit dispatch: free-first, SC thinness-floor backfill (U7), and+that the keyless path never calls search.json (U2)."""++from unittest import mock++from lib import pipeline, reddit_keyless, schema+++def _subquery():+ return schema.SubQuery(label="t", search_query="kanye", ranking_query="kanye",+ sources=["reddit"])+++def _runtime():+ return schema.ProviderRuntime(reasoning_provider="mock", planner_model="mock",+ rerank_model="mock")+++def _item(rid):+ return {"url": f"https://www.reddit.com/r/x/comments/{rid}/t/", "title": rid}+++def _ids(items):+ return [pipeline._reddit_post_key(i) for i in items]+++class TestThinnessFloor:+ KEY = {"SCRAPECREATORS_API_KEY": "k"}++ def _run(self, config, public, sc_parsed):+ with mock.patch("lib.reddit_public.search_reddit_public", return_value=public), \+ mock.patch("lib.reddit.search_and_enrich", return_value={"raw": 1}) as sc, \+ mock.patch("lib.reddit.parse_reddit_response", return_value=sc_parsed):+ items, _ = pipeline._retrieve_stream(+ topic="kanye", subquery=_subquery(), source="reddit", config=config,+ depth="quick", date_range=("2026-05-26", "2026-06-25"),+ runtime=_runtime(), mock=False,+ )+ return items, sc++ def test_default_zero_does_not_call_sc_when_free_has_items(self):+ # min_items unset (0): today's empty-only behavior — free wins, no SC.+ items, sc = self._run(self.KEY, [_item("a"), _item("b"), _item("c")], [_item("z")])+ assert len(items) == 3+ sc.assert_not_called()++ def test_default_empty_free_falls_to_sc(self):+ items, sc = self._run(self.KEY, [], [_item("z")])+ assert _ids(items) == ["z"]+ sc.assert_called_once()++ def test_threshold_fires_on_thin_run_and_merges_deduped(self):+ cfg = {**self.KEY, "LAST30DAYS_REDDIT_SC_MIN_ITEMS": "5"}+ free = [_item("a"), _item("b")] # 2 < 5 -> backfill+ sc_parsed = [_item("b"), _item("c")] # overlaps "b"+ items, sc = self._run(cfg, free, sc_parsed)+ sc.assert_called_once()+ assert _ids(items) == ["a", "b", "c"] # free first, dedup b, append c++ def test_threshold_not_fired_when_free_above_floor(self):+ cfg = {**self.KEY, "LAST30DAYS_REDDIT_SC_MIN_ITEMS": "2"}+ items, sc = self._run(cfg, [_item("a"), _item("b"), _item("c")], [_item("z")])+ sc.assert_not_called()+ assert len(items) == 3++ def test_exactly_floor_is_acceptable_no_backfill(self):+ # MIN_ITEMS=N means N results are acceptable; only fewer than N backfills.+ cfg = {**self.KEY, "LAST30DAYS_REDDIT_SC_MIN_ITEMS": "3"}+ items, sc = self._run(cfg, [_item("a"), _item("b"), _item("c")], [_item("z")])+ sc.assert_not_called()+ assert len(items) == 3++ def test_no_key_never_calls_sc(self):+ items, sc = self._run({"LAST30DAYS_REDDIT_SC_MIN_ITEMS": "5"}, [_item("a")], [_item("z")])+ sc.assert_not_called()+ assert len(items) == 1++ def test_bad_threshold_value_defaults_to_empty_only(self):+ cfg = {**self.KEY, "LAST30DAYS_REDDIT_SC_MIN_ITEMS": "not-an-int"}+ items, sc = self._run(cfg, [_item("a")], [_item("z")])+ sc.assert_not_called() # falls back to 0 -> free (1 item) wins+ assert len(items) == 1++ def test_sc_failure_degrades_to_free(self):+ cfg = {**self.KEY, "LAST30DAYS_REDDIT_SC_MIN_ITEMS": "5"}+ with mock.patch("lib.reddit_public.search_reddit_public", return_value=[_item("a")]), \+ mock.patch("lib.reddit.search_and_enrich", side_effect=Exception("down")):+ items, _ = pipeline._retrieve_stream(+ topic="kanye", subquery=_subquery(), source="reddit", config=cfg,+ depth="quick", date_range=("2026-05-26", "2026-06-25"),+ runtime=_runtime(), mock=False,+ )+ assert _ids(items) == ["a"] # backup failed -> keep the free items+++class TestMergeHelper:+ def test_dedup_by_post_id_free_first(self):+ out = pipeline._merge_reddit_items([_item("a"), _item("b")], [_item("b"), _item("c")])+ assert _ids(out) == ["a", "b", "c"]+++class TestNoSearchJson:+ def test_keyless_discovery_never_calls_searchjson(self):+ # reddit_public.search (the .json caller) must never run in the keyless flow.+ with mock.patch("lib.reddit_public.search") as json_search, \+ mock.patch("lib.reddit_keyless.reddit_rss.search_rss", return_value=[]), \+ mock.patch("lib.reddit_keyless.reddit_listing.fetch_listings", return_value=[]):+ reddit_keyless._discover("topic", "default", ["test"])+ json_search.assert_not_called()diff --git a/tests/test_reddit_keyless.py b/tests/test_reddit_keyless.pyindex 1ffb52b0..161d8b6c 100644--- a/tests/test_reddit_keyless.py+++ b/tests/test_reddit_keyless.py@@ -26,22 +26,11 @@ def _scored(i, score, ncmt=0):return p-class TestDiscoveryTierOrder:- """Tier 0 (.json) is tried first; RSS + scored listings are the keyless path."""+class TestDiscovery:+ """RSS breadth + scored listings are the keyless discovery path (no .json)."""- def test_tier0_success_skips_keyless(self):- with mock.patch.object(reddit_keyless, "_tier0_json", return_value=[_post(1)]) as t0, \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss") as rss, \- mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings") as lst:- out = reddit_keyless._discover("topic", "default", None)- assert len(out) == 1- t0.assert_called_once()- rss.assert_not_called()- lst.assert_not_called()-- def test_tier0_empty_falls_to_keyless(self):- with mock.patch.object(reddit_keyless, "_tier0_json", return_value=[]), \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss",+ def test_keyless_path_runs_rss_and_listings(self):+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss",return_value=[_post(1), _post(2)]) as rss, \mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",return_value=[]):@@ -53,8 +42,7 @@ class TestDiscoveryTierOrder:# RSS finds post 1 (no score); listing card for post 1 carries the score.rss_post = _post(1)listing_post = _scored(1, score=52692, ncmt=1743)- with mock.patch.object(reddit_keyless, "_tier0_json", return_value=[]), \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss",+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss",return_value=[rss_post]), \mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",return_value=[listing_post]):@@ -69,8 +57,7 @@ class TestDiscoveryTierOrder:rss_post = _post(7) # url .../000007/...listing_post = _scored(7, score=999)listing_post["url"] = "https://www.reddit.com/r/test/comments/zzzzzz/other/"- with mock.patch.object(reddit_keyless, "_tier0_json", return_value=[]), \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss",+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss",return_value=[rss_post]), \mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",return_value=[listing_post]):@@ -85,8 +72,7 @@ class TestDiscoveryTierOrder:rss_post = _post(1) # on-topic keyword matchofftopic_listing = _scored(99, score=88888) # high score, unrelated subofftopic_listing["url"] = "https://www.reddit.com/r/random/comments/zzz999/x/"- with mock.patch.object(reddit_keyless, "_tier0_json", return_value=[]), \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss",+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss",return_value=[rss_post]), \mock.patch.object(reddit_keyless, "_top_subreddits", return_value=["random"]), \mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings",@@ -96,9 +82,8 @@ class TestDiscoveryTierOrder:assert rss_post["url"] in urlsassert offtopic_listing["url"] not in urls # not merged as discovery- def test_tier0_never_raises(self):- with mock.patch("lib.reddit_public.search", side_effect=Exception("boom")), \- mock.patch.object(reddit_keyless.reddit_rss, "search_rss", return_value=[]), \+ def test_discover_never_raises_returns_empty(self):+ with mock.patch.object(reddit_keyless.reddit_rss, "search_rss", return_value=[]), \mock.patch.object(reddit_keyless.reddit_listing, "fetch_listings", return_value=[]):assert reddit_keyless._discover("t", "default", None) == []diff --git a/tests/test_reddit_public.py b/tests/test_reddit_public.pyindex 10b91940..96154fe6 100644--- a/tests/test_reddit_public.py+++ b/tests/test_reddit_public.py@@ -278,25 +278,23 @@ class TestDepthLimits:class TestSearchRedditPublicHighLevel:"""Test the high-level search_reddit_public interface."""- @mock.patch("lib.reddit_public.urllib.request.urlopen")- def test_date_filtering(self, mock_urlopen):- listing = _make_reddit_listing([- {- "title": "In range",- "permalink": "/r/test/comments/aaa/in_range/",- "created_utc": 1711670400, # 2024-03-29- },- {- "title": "Out of range",- "permalink": "/r/test/comments/bbb/out_of_range/",- "created_utc": 1609459200, # 2021-01-01- },- ])- mock_urlopen.return_value = _mock_urlopen_ok(listing)-- results = reddit_public.search_reddit_public(- "test", "2024-03-01", "2024-03-31"- )+ def test_date_filtering(self):+ # search_reddit_public delegates to the keyless pipeline, which filters+ # discovered posts to the requested date window. (search.json is gone, so+ # the filter is verified at the keyless discovery seam, not via .json.)+ def _p(title, slug, date):+ return {+ "id": "", "title": title, "score": 0, "num_comments": 0,+ "url": f"https://www.reddit.com/r/test/comments/{slug}/x/",+ "subreddit": "test", "author": "u", "selftext": "", "date": date,+ "engagement": {"score": 0, "num_comments": 0}, "relevance": 0.0,+ "metadata": {},+ }+ posts = [_p("In range", "aaa", "2024-03-29"), _p("Out of range", "bbb", "2021-01-01")]+ with mock.patch("lib.reddit_keyless._discover", return_value=posts), \+ mock.patch("lib.reddit_keyless.reddit_shreddit.fetch_comments",+ return_value={"top_comments": [], "comment_insights": [], "num_comments": None}):+ results = reddit_public.search_reddit_public("test", "2024-03-01", "2024-03-31")titles = [r["title"] for r in results]assert "In range" in titles@@ -345,5 +343,5 @@ class TestSearchRedditPublicDelegatesToKeyless:assert results == [{"id": "R1", "title": "x"}]mock_keyless.assert_called_once_with("test", "2024-03-01", "2024-03-31",- depth="quick", subreddits=["ClaudeAI"],+ depth="quick", subreddits=["ClaudeAI"], dedicated_subreddits=None,)