AI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary
Python ai-promptsai-skillblueskyclaudeclaude-codeclawhubdeep-researchhackernewsinstagramopenclawpolymarketrecencyredditresearchsocial-mediatiktoktrendstwitterweb-searchyoutube
fix(youtube): exempt transcript-bearing items from relevance pruning (#468) (#577)
SAI MOHANEESH7a0800fd741e
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 650e8143..2ff7e3cb 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0### Fixed+- YouTube items with successfully extracted transcripts are no longer pruned by title-only relevance scoring; the transcript content proves substantive topical coverage even when the video title has low lexical overlap with the query ([#468](https://github.com/mvanhorn/last30days-skill/issues/468))- First-run setup wizard in SKILL.md now references the existing Python setup wizard (`last30days.py setup`) instead of the missing `nux-wizard.md` file, so first-run setup actually runs on new installs. ([#574](https://github.com/mvanhorn/last30days-skill/issues/574))- Entity-grounding rerank demotion now keys on the head token of the primary entity instead of requiring the full multi-word phrase as a contiguous substring. A high-engagement on-entity item (e.g. a 323-pt HN thread titled "Stripe is friendly to 'friendly fraud'") is no longer demoted to score 0 on a `Stripe payments` query just because it lacks the trailing search-hint word. The intended demotion still fires for items that never name the brand at all. The keyless Reddit comment-enrichment slot selection (`_slot_priority`), which mirrors this signal, was updated to the same head-token grounding so the two paths stay consistent.- `--plan` / `--competitors-plan` file reads now specify `encoding="utf-8"` and catch `UnicodeDecodeError`, preventing crashes on non-ASCII content like accented entity names on Windows (cp1252). `check_perms()` in `check-config.sh` now skips the POSIX 600-permission check on MSYS/MinGW/Cygwin where `stat` runs in noacl mode. `skill_meta.py` `read_skill_version()` now passes `encoding="utf-8"` so SKILL.md emoji doesn't break version detection on Windows. ([#549](https://github.com/mvanhorn/last30days-skill/issues/549))diff --git a/skills/last30days/scripts/lib/signals.py b/skills/last30days/scripts/lib/signals.pyindex 02eb7d3a..bde4a3d0 100644--- a/skills/last30days/scripts/lib/signals.py+++ b/skills/last30days/scripts/lib/signals.py@@ -239,6 +239,11 @@ def prune_low_relevance(sources_present = {item.source for item in items}def passes(item: schema.SourceItem) -> bool:+ # YouTube items with successfully extracted transcripts should not+ # be pruned by title-only relevance scoring — the transcript content+ # already proves substantive topical coverage.+ if item.source == "youtube" and item.snippet:+ return Truerel = item.local_relevance if item.local_relevance is not None else 0.0if rel < minimum:return Falsediff --git a/tests/test_signals_v3.py b/tests/test_signals_v3.pyindex bf7089ec..5ea65bdd 100644--- a/tests/test_signals_v3.py+++ b/tests/test_signals_v3.py@@ -702,5 +702,88 @@ class SignalsV3Tests(unittest.TestCase):aspire_ids = [item.item_id for item in pruned if item.item_id.startswith("aspire")]self.assertEqual(len(aspire_ids), 0, f"All @aspiresnippets items should be pruned, got {aspire_ids}")+ # -- Fix 468: YouTube items with transcripts survive relevance pruning --++ def test_youtube_with_transcript_survives_pruning_even_with_low_relevance(self):+ """A YouTube item with a non-empty snippet (transcript) should not be+ pruned even if its title-only relevance is below the threshold."""+ has_transcript = schema.SourceItem(+ item_id="yt-transcript",+ source="youtube",+ title="Short title",+ body="Short body",+ url="https://youtube.com/watch?v=abc",+ snippet="This is a detailed transcript about the topic with substantive discussion...",+ local_relevance=0.05,+ )+ strong = schema.SourceItem(+ item_id="yt-strong",+ source="youtube",+ title="Strong video",+ body="Detailed analysis of the topic",+ url="https://youtube.com/watch?v=strong",+ snippet="Detailed transcript content about the topic",+ local_relevance=0.6,+ )+ pruned = signals.prune_low_relevance([strong, has_transcript], minimum=0.15)+ ids = [item.item_id for item in pruned]+ self.assertIn("yt-transcript", ids,+ "YouTube item with transcript should survive pruning")+ self.assertIn("yt-strong", ids, "Strong item should survive")++ def test_youtube_without_transcript_is_pruned_normally(self):+ """A YouTube item with no transcript (empty snippet) and low relevance+ should still be pruned when stronger items exist."""+ no_transcript = schema.SourceItem(+ item_id="yt-no-transcript",+ source="youtube",+ title="Short title",+ body="Short body",+ url="https://youtube.com/watch?v=xyz",+ snippet="",+ local_relevance=0.05,+ )+ strong = schema.SourceItem(+ item_id="yt-strong",+ source="youtube",+ title="Strong video",+ body="Detailed analysis of the topic",+ url="https://youtube.com/watch?v=strong",+ snippet="Detailed transcript content about the topic",+ local_relevance=0.6,+ )+ pruned = signals.prune_low_relevance([strong, no_transcript], minimum=0.15)+ ids = [item.item_id for item in pruned]+ self.assertIn("yt-strong", ids, "Strong item should survive")+ self.assertNotIn("yt-no-transcript", ids,+ "YouTube item without transcript should be pruned normally")++ def test_youtube_transcript_exemption_does_not_affect_other_sources(self):+ """Non-YouTube items with low relevance are still pruned even if they+ have a non-empty snippet (the exemption is YouTube-specific)."""+ reddit_with_snippet = schema.SourceItem(+ item_id="reddit-snippet",+ source="reddit",+ title="Short title",+ body="Short body",+ url="https://reddit.com/r/test",+ snippet="Some snippet content",+ local_relevance=0.05,+ )+ strong = schema.SourceItem(+ item_id="strong",+ source="reddit",+ title="Strong post",+ body="Detailed analysis of the topic",+ url="https://reddit.com/r/strong",+ local_relevance=0.5,+ )+ pruned = signals.prune_low_relevance([strong, reddit_with_snippet], minimum=0.15)+ ids = [item.item_id for item in pruned]+ self.assertIn("strong", ids, "Strong item should survive")+ self.assertNotIn("reddit-snippet", ids,+ "Non-YouTube items should still be pruned by relevance threshold")++if __name__ == "__main__":unittest.main()