Skip to content

AI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary

fix(youtube): SSH transcript routing and surface SSH search errors (#422)

Trevin Chow91987294cda7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 8ca3f967..84b2d8ce 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,8 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
-### Added
-
+- **`LAST30DAYS_YOUTUBE_SSH_HOST` transcript routing** — yt-dlp transcript fetch runs on the remote SSH host via a mktemp + cat pipeline ([#422](https://github.com/mvanhorn/last30days-skill/pull/422)).
- Browser-cookie auth for X/Twitter now covers the full Chromium family on macOS - Brave, Microsoft Edge, Vivaldi, Opera, Arc, and Chromium - alongside the existing Chrome, Firefox, and Safari. They all share Chrome's v10 AES-128-CBC decryption, differing only in profile path and Keychain service name, so they run through one shared decryption core. The profile finder probes both the modern `Default/Network/Cookies` layout (Chromium >= 96) and the legacy flat `Default/Cookies`, and Chrome now resolves through that same finder so it picks up the modern layout too. Set `FROM_BROWSER=auto` to try every browser, or `FROM_BROWSER=<name>` (e.g. `brave`, `edge`, `arc`) to target one. Verified end-to-end on real Brave and Edge installs ([#572](https://github.com/mvanhorn/last30days-skill/pull/572)).
- **First-party positioning research + pitch-vs-pulse synthesis (company / product / service topics).** A new mandatory research step captures each entity's current stated positioning from first-party sources (homepage, docs, pricing) rather than from memory. The fetched pitch grounds `What it is` descriptions (entities described as they pitch themselves today), helps reject unrelated brand-name noise, and feeds an evidence-triggered prose beat: when the month's conversation directly supports a specific claim, cuts against one, or is squarely about the pitched ground, the synthesis says so anchored to the top thread — and stays silent when the pulse is orthogonal to the pitch, because a manufactured connection is worse than omission. Claims are tested at matched altitude (specific claims against specific threads; broad taglines are never graded against individual items), and statements stay windowed to the 30 days — no trend verdicts. Scoped to entities with an identifiable first party: people are always excluded (even founders whose companies qualify), as are events, abstract concepts, and ownerless topics like Bitcoin; the beat requires positioning fetched during the run, never from memory.
@@ -20,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
+- **SSH routing failures no longer present as "0 results"** — `search_youtube` surfaces non-zero SSH exit codes as an explicit `error` field ([#422](https://github.com/mvanhorn/last30days-skill/pull/422)).
- `extract_browser_credentials()` silently ignored Brave even though the lower-level `cookie_extract` layer already supported it: `FROM_BROWSER=brave` fell back to Firefox/Safari and `FROM_BROWSER=auto` never tried Brave. The env wiring now passes Brave - and the rest of the Chromium family - through to the extractor ([#572](https://github.com/mvanhorn/last30days-skill/pull/572)).
- Chromium cookie extraction now fetches the macOS Keychain key lazily - only when an encrypted cookie actually needs decrypting. Previously the key was fetched as soon as the cookie DB existed, so `FROM_BROWSER=auto` could trigger a Keychain prompt for every installed Chromium browser. Now only the browser that actually holds the requested cookie prompts ([#572](https://github.com/mvanhorn/last30days-skill/pull/572)).
- YouTube transcript budget prioritises recent videos (by a combination of views and recency) instead of views alone, preventing transcript slots from being consumed by old high-view-count videos that would be discarded by strict_recent freshness pruning ([#531](https://github.com/mvanhorn/last30days-skill/issues/531))
diff --git a/skills/last30days/scripts/lib/youtube_yt.py b/skills/last30days/scripts/lib/youtube_yt.py
index 33742d6f..05624ef4 100644
--- a/skills/last30days/scripts/lib/youtube_yt.py
+++ b/skills/last30days/scripts/lib/youtube_yt.py
@@ -313,6 +313,7 @@ def search_youtube(
"--no-download",
]
cmd = _wrap_ytdlp_cmd(cmd)
+ ssh_host = _ytdlp_ssh_host()
try:
result = subproc.run_with_timeout(cmd, timeout=120)
@@ -323,6 +324,17 @@ def search_youtube(
return {"items": [], "error": "yt-dlp not found"}
stdout = result.stdout
+ if ssh_host and result.returncode != 0 and not stdout.strip():
+ stderr_first = (result.stderr or "").strip().splitlines()
+ first_line = stderr_first[0] if stderr_first else "(no stderr)"
+ _log(
+ f"YouTube search via SSH host {ssh_host!r} failed "
+ f"(rc={result.returncode}): {first_line}"
+ )
+ return {
+ "items": [],
+ "error": f"SSH routing to {ssh_host!r} failed: {first_line}",
+ }
if not stdout.strip():
_log("YouTube search returned 0 results")
return {"items": []}
@@ -511,6 +523,45 @@ def _fetch_transcript_direct(
return vtt_text
+def _fetch_transcript_ytdlp_via_ssh(video_id: str, ssh_host: str) -> Optional[str]:
+ """Fetch transcript via yt-dlp on a remote SSH host (mktemp + cat pipeline)."""
+ if not _SSH_HOST_ALIAS_RE.match(ssh_host):
+ return None
+ url = f"https://www.youtube.com/watch?v={video_id}"
+ quoted_url = shlex.quote(url)
+ sub_langs = shlex.quote(_ytdlp_sub_langs())
+ remote_script = (
+ "set -e; "
+ "TMPD=$(mktemp -d); "
+ "yt-dlp --ignore-config --no-cookies-from-browser "
+ f"--write-auto-subs --sub-lang {sub_langs} --sub-format vtt "
+ "--skip-download --no-warnings "
+ f'-o "$TMPD/%(id)s" {quoted_url} >/dev/null 2>&1 || true; '
+ 'VTT=$(find "$TMPD" -maxdepth 1 -name "*.vtt" 2>/dev/null | head -1); '
+ '[ -n "$VTT" ] && cat "$VTT"; '
+ 'rm -rf "$TMPD"'
+ )
+ cmd = ["ssh", "-o", "BatchMode=yes", "--", ssh_host, remote_script]
+ try:
+ result = subproc.run_with_timeout(cmd, timeout=45)
+ except subproc.SubprocTimeout:
+ _log(f"SSH yt-dlp transcript timed out for {video_id} via {ssh_host!r}")
+ return None
+ except FileNotFoundError:
+ _log("ssh executable not found; cannot route transcript fetch")
+ return None
+ out = result.stdout or ""
+ if not out.strip().startswith("WEBVTT"):
+ if result.returncode != 0 and result.stderr:
+ first_line = result.stderr.strip().splitlines()[0]
+ _log(
+ f"SSH yt-dlp transcript via {ssh_host!r} failed for "
+ f"{video_id} (rc={result.returncode}): {first_line}"
+ )
+ return None
+ return out
+
+
def _ytdlp_sub_langs() -> str:
"""Caption languages to try, from LAST30DAYS_YT_SUB_LANGS (default en,es,pt)."""
raw = os.environ.get("LAST30DAYS_YT_SUB_LANGS", "").strip()
@@ -663,30 +714,23 @@ def fetch_transcript(
Plaintext transcript string, or None if no captions available.
"""
raw_vtt = None
- # When SSH-routing is on, the yt-dlp transcript path would write a VTT
- # file on the remote host that we can't easily read back. Skip it and
- # use the HTTP transcript fallback (different YouTube endpoint, less
- # bot-walled, works fine from datacenter IPs).
ssh_host = _ytdlp_ssh_host()
- use_ytdlp = is_ytdlp_installed() and not ssh_host
- if use_ytdlp:
+ if ssh_host and is_ytdlp_installed():
+ raw_vtt = _fetch_transcript_ytdlp_via_ssh(video_id, ssh_host)
+ if not raw_vtt:
+ _log(f"SSH yt-dlp transcript failed for {video_id}, trying direct HTTP fallback")
+ raw_vtt = _fetch_transcript_direct(video_id, status=status)
+ elif is_ytdlp_installed():
raw_vtt = _fetch_transcript_ytdlp(video_id, temp_dir, status=status)
if not raw_vtt:
ytdlp_error = (status or {}).get("ytdlp_error")
if ytdlp_error:
- # yt-dlp hit a real error (rate-limit / bot-check / timeout) —
- # captions likely exist. The direct-HTTP fallback hits the same
- # wall and returns an empty body that masquerades as "no
- # captions", so skip it and surface the real reason instead.
_log(f"Transcript fetch failed for {video_id}: {ytdlp_error}")
return None
_log(f"yt-dlp found no captions for {video_id}, trying direct HTTP fallback")
raw_vtt = _fetch_transcript_direct(video_id, status=status)
else:
- if ssh_host:
- _log("SSH-routing active, using direct HTTP transcript fetch")
- else:
- _log("yt-dlp not installed, using direct HTTP transcript fetch")
+ _log("yt-dlp not installed, using direct HTTP transcript fetch")
raw_vtt = _fetch_transcript_direct(video_id, status=status)
if not raw_vtt:
diff --git a/tests/test_youtube_yt.py b/tests/test_youtube_yt.py
index ab03d62d..9735473f 100644
--- a/tests/test_youtube_yt.py
+++ b/tests/test_youtube_yt.py
@@ -774,5 +774,60 @@ class TestYtdlpSSHRouting(unittest.TestCase):
self.assertIn("--ignore-config", cmd[5])
self.assertIn("--no-cookies-from-browser", cmd[5])
+ def test_search_surfaces_ssh_failure_as_error(self):
+ """SSH connection failures surface as an error, not silent '0 results'."""
+ os.environ["LAST30DAYS_YOUTUBE_SSH_HOST"] = "macmini"
+ from lib.subproc import SubprocResult
+ fake_result = SubprocResult(
+ returncode=255,
+ stdout="",
+ stderr="ssh: connect to host macmini port 22: Connection refused\n",
+ )
+ with mock.patch.object(youtube_yt.subproc, "run_with_timeout",
+ return_value=fake_result):
+ out = youtube_yt.search_youtube("test", "2026-02-01", "2026-03-01")
+ self.assertIn("error", out)
+ self.assertIn("Connection refused", out["error"])
+
+
+class TestTranscriptSSHRouting(unittest.TestCase):
+ """LAST30DAYS_YOUTUBE_SSH_HOST routes yt-dlp transcript fetches through SSH."""
+
+ def setUp(self):
+ self._saved_env = os.environ.pop("LAST30DAYS_YOUTUBE_SSH_HOST", None)
+
+ def tearDown(self):
+ os.environ.pop("LAST30DAYS_YOUTUBE_SSH_HOST", None)
+ if self._saved_env is not None:
+ os.environ["LAST30DAYS_YOUTUBE_SSH_HOST"] = self._saved_env
+
+ def test_ssh_helper_invokes_remote_mktemp_pipeline(self):
+ os.environ["LAST30DAYS_YOUTUBE_SSH_HOST"] = "macmini"
+ from lib.subproc import SubprocResult
+ fake_vtt = "WEBVTT\nKind: captions\nLanguage: en\n\n00:00:00.000 --> 00:00:02.000\nhi\n"
+ fake_result = SubprocResult(returncode=0, stdout=fake_vtt, stderr="")
+ with mock.patch.object(youtube_yt.subproc, "run_with_timeout",
+ return_value=fake_result) as run_mock:
+ out = youtube_yt._fetch_transcript_ytdlp_via_ssh("vid1", "macmini")
+ self.assertEqual(out, fake_vtt)
+ remote_script = run_mock.call_args.args[0][5]
+ self.assertIn("mktemp -d", remote_script)
+ self.assertIn("find ", remote_script)
+
+ def test_fetch_transcript_uses_ssh_helper_when_routing_on(self):
+ os.environ["LAST30DAYS_YOUTUBE_SSH_HOST"] = "macmini"
+ fake_vtt = "WEBVTT\n\n00:00:00.000 --> 00:00:02.000\nhello there friends\n"
+ with mock.patch.object(youtube_yt, "is_ytdlp_installed", return_value=True), \
+ mock.patch.object(youtube_yt, "_fetch_transcript_ytdlp_via_ssh",
+ return_value=fake_vtt) as ssh_mock, \
+ mock.patch.object(youtube_yt, "_fetch_transcript_ytdlp") as local_mock, \
+ mock.patch.object(youtube_yt, "_fetch_transcript_direct") as direct_mock:
+ result = youtube_yt.fetch_transcript("vidX", "/tmp/test")
+ ssh_mock.assert_called_once_with("vidX", "macmini")
+ local_mock.assert_not_called()
+ direct_mock.assert_not_called()
+ self.assertIn("hello there friends", result)
+
+
if __name__ == "__main__":
unittest.main()