Skip to content

Convert Claude Code or Codex into the most intelligent Deep Research Agent. Collect, search, and synthesize web research into a persistent, searchable wiki that builds on itself. Hosted API + MCP: hyperresearch.ai

fix: 3 bugs from look-back review

Jordanebe33edad25b
diff --git a/src/hyperresearch/cli/fetch.py b/src/hyperresearch/cli/fetch.py
index 6e0e91d..7a81521 100644
--- a/src/hyperresearch/cli/fetch.py
+++ b/src/hyperresearch/cli/fetch.py
@@ -439,23 +439,37 @@ def fetch(
conn.commit()
# Detect the race: if the committed row's note_id != ours, another
- # fetch won the race. Clean up our orphan .md file and return the
- # winner's note_id so the caller can treat this as an idempotent no-op.
+ # fetch won the race. Clean up our orphan .md file (and raw artifact,
+ # if any) and return the winner's note_id so the caller can treat
+ # this as an idempotent no-op.
winner_row = conn.execute(
"SELECT note_id FROM sources WHERE url = ?", (url,)
).fetchone()
if winner_row and winner_row["note_id"] != note_id:
winning_note_id = winner_row["note_id"]
- # Our write lost the race. Unlink the orphan file and re-sync.
+ # Our write lost the race. Unlink the orphan .md file.
if note_path.exists():
try:
note_path.unlink()
except OSError:
pass
- plan2 = compute_sync_plan(vault)
- if plan2.to_delete:
- execute_sync(vault, plan2)
+ # Also unlink the raw file we wrote earlier — the winner already
+ # has its own raw file under raw/<winner-id>.<ext>. Without this,
+ # the orphaned-raw-files lint flags raw/<our-id>.<ext> on the
+ # next run and disk leaks accumulate.
+ if raw_file_path:
+ orphan_raw = vault.root / "research" / raw_file_path
+ if orphan_raw.exists():
+ try:
+ orphan_raw.unlink()
+ except OSError:
+ pass
+ # Re-sync to drop the orphan note row from the DB.
+ plan2 = compute_sync_plan(vault)
+ if plan2.to_delete:
+ execute_sync(vault, plan2)
note_id = winning_note_id
+ raw_file_path = None # we no longer own a raw artifact for this fetch
winner_path_row = conn.execute(
"SELECT path FROM notes WHERE id = ?", (winning_note_id,)
).fetchone()
diff --git a/src/hyperresearch/cli/note.py b/src/hyperresearch/cli/note.py
index 80afdd6..e8de28c 100644
--- a/src/hyperresearch/cli/note.py
+++ b/src/hyperresearch/cli/note.py
@@ -581,8 +581,17 @@ def note_rm(
text = file_path.read_text(encoding="utf-8-sig")
meta, _ = parse_frontmatter(text)
if meta.raw_file:
- raw_path = vault.root / "research" / meta.raw_file
- if raw_path.exists():
+ # Resolve and guard against path traversal — a malicious or
+ # corrupted frontmatter could set raw_file to "../../etc/passwd".
+ # Refuse to unlink anything outside <vault>/research/.
+ research_root = (vault.root / "research").resolve()
+ raw_path = (vault.root / "research" / meta.raw_file).resolve()
+ try:
+ raw_path.relative_to(research_root)
+ inside_vault = True
+ except ValueError:
+ inside_vault = False
+ if inside_vault and raw_path.exists() and raw_path.is_file():
raw_path.unlink()
removed_raw = str(raw_path.relative_to(vault.root).as_posix())
except Exception:
diff --git a/src/hyperresearch/skills/research.md b/src/hyperresearch/skills/research.md
index 93b3f39..00d7cc0 100644
--- a/src/hyperresearch/skills/research.md
+++ b/src/hyperresearch/skills/research.md
@@ -237,7 +237,7 @@ For high-priority failures, work through this fallback chain in order — stop a
- SSRN, RePEc, NBER, OpenAlex, Semantic Scholar mirror
- Google Scholar's "All N versions" link
2. **Try the visible-browser flag** on the original URL: `$HPR fetch "<url>" --visible ...`. The `--visible` flag runs the browser non-headless; many sites that block headless requests succeed when the browser is visible. Cost: ~5-10s slower per fetch.
-3. **Try the user's authenticated profile** if one is configured: `$HPR fetch "<url>" --profile research ...`. Login profiles bypass paywalls on sites the user has logged into. If no profile is configured, the user can create one via `hyperresearch setup`.
+3. **If a login profile is configured for this vault** (`.hyperresearch/config.toml` has a `[web] profile = "<name>"` entry), crawl4ai uses it automatically — paywalled sites the user has logged into in that profile will succeed transparently. If profile-protected sites still fail, the profile may be expired; tell the user to re-run `hyperresearch setup` to refresh it. There is no `--profile` CLI flag — it's a per-vault config setting.
4. **As a last resort**, search for a **summary or review** of the paper that captures its key claims (e.g., a textbook chapter that summarizes the result). Note this in the source's frontmatter `summary:` field as `(via summary, original PDF unavailable)`.
5. **If all four fail and the source is genuinely irreplaceable**, surface the failure to the user — don't ship a draft that depends on a source you couldn't read.