Convert Claude Code or Codex into the most intelligent Deep Research Agent. Collect, search, and synthesize web research into a persistent, searchable wiki that builds on itself. Hosted API + MCP: hyperresearch.ai
Python agentsagentskillscitationsclaude-codecodexcodex-plugincodex-skilldeep-researchdeep-research-agentdeepresearchmcp-serverresearchresearch-agentresearch-tools
4-activity modality taxonomy + dispatcher rewrite
Jordanc039f2c70b49
diff --git a/src/hyperresearch/core/agent_docs.py b/src/hyperresearch/core/agent_docs.pyindex d49f8d7..a2ddf6a 100644--- a/src/hyperresearch/core/agent_docs.py+++ b/src/hyperresearch/core/agent_docs.py@@ -26,276 +26,11 @@ This project uses hyperresearch as an agent-driven research knowledge base. The### MANDATORY: How to do research-When the user asks you to research a topic, **you MUST follow this workflow**. Do NOT use WebFetch for source pages — use `{hpr} fetch` instead. It runs a real headless browser, handles JavaScript, bypasses bot detection, saves full content with screenshots and images, and indexes everything for future sessions.+When the user asks you to research a topic, follow the full step-by-step workflow in `.claude/skills/hyperresearch/SKILL.md`. **Do NOT use WebFetch for source pages** — use `{hpr} fetch` instead.-**This is a deep research tool.** Your job is to go down rabbit holes, cast a wide net, and collect enough raw source material to say something meaningful across it. **Over-collect, then prune.** It is better to have too many sources than too few — you can deprecate notes later, but you can't un-skip a source you never fetched.+SKILL.md classifies your request by cognitive activity (what the output needs to DO, not what the subject IS) and routes to one of 4 modality files: SKILL-collect.md (enumerative coverage with per-entity fields), SKILL-synthesize.md (defended thesis or mechanism explanation), SKILL-compare.md (per-entity evaluation with committed recommendation), or SKILL-forecast.md (forward-looking prediction with explicit time horizon). The dispatcher owns the shared protocol (discovery, fetch, guided reading loop, curation, scaffold, comparisons, draft, audit, synthesis). Each modality file encodes only the substance rules and conformance checks specific to that activity. Read SKILL.md first — it owns the process and will tell you which modality file to open next for substance guidance.-**But collection is not the goal.** Collection is a means to an argument. A report built from 30 sources that disagree, contextualize each other, and force you to take positions is worth more than a report built from 80 sources that each contribute one paragraph of description. Optimise for **depth of engagement** (each source read carefully, compared with 2–3 others, cited multiple times across the report) rather than **count of citations**. If you find yourself translating one source into one paragraph or one section, you are building a catalog — stop and restructure.--**There is no time limit.** Deep research can run for hours if the topic demands it. Do multiple rounds of research — as new subtopics, angles, and questions reveal themselves, launch new rounds of fetcher subagents in parallel to chase them down. The first round uncovers the landscape; the second round digs into what you found; the third round fills gaps. Keep going.--**CHECKPOINT: Before you start writing ANY draft, stop and take stock.** Run:-```bash-{hpr} note list -j-```-Don't collect sources just to hit a number. Instead, review what you have and ask:-- What angles, subtopics, or perspectives are NOT yet covered?-- Which existing notes reference papers, data, or sources I haven't fetched yet?-- Are there counterarguments, alternative viewpoints, or competing theories I'm missing?-- Do I have primary sources, or just secondhand summaries?--If the answer to any of these is "yes, there's more to get" — go back and get it. Spawn more fetcher agents, search new angles, follow links from existing notes. Keep going until you genuinely can't find anything new worth adding.--**Step 0: Classify the request and pick a protocol (MANDATORY)** — Different research requests need fundamentally different strategies. Source counts, primary/secondary mix, section structure, target length, analytical mode — all change based on request shape. A single universal protocol is wrong. **Before Step 1, classify the request and state your classification in writing** (in working memory or `/tmp/scaffold.md` — NOT a hyperresearch note).--The 7 request types:--| Type | Name | Use when |-|---|---|---|-| 1 | **Canonical Knowledge Retrieval** | Mature field with textbook chapters or canonical surveys; an expert would start by citing a classic source. |-| 2 | **Market / Landscape Mapping** | Enumerable competitive analysis — list of companies, products, vendors with attributes. |-| 3 | **Engineering / Technical How-To** | User is trying to DO something. Ideal answer is a procedure or method selection. |-| 4 | **Interpretive / Humanities Analysis** | Subject is a text, work, tradition, or cultural phenomenon. Good answer takes a position and defends it. |-| 5 | **Comparative Evaluation** | Prompt names multiple alternatives by name and expects a matrix + pick. |-| 6 | **Emerging / Cutting-Edge Research** | Most useful material is <2 years old; best sources are arXiv/bioRxiv/conference papers. |-| 7 | **Forecast / Strategy / Recommendation** | Asks what will happen or should be done; ideal answer commits to a prediction or prescription. |-| 0 | **General Research (fallback)** | Doesn't cleanly fit one type, or genuinely spans multiple. |--Overlap is expected. Classify a **primary** type and, if applicable, a **secondary** type. Apply the primary's parameter block; borrow special rules from the secondary.--**State your classification** before Step 1:-1. Primary type: #N — one-sentence justification tied to the prompt.-2. Secondary type: #N or "none".-3. Applied parameters (5 lines): source strategy, target length, opening-section shape, target H2 count, analytical mode.--**Request-type parameter blocks** — apply the block for your primary type. These overrides take precedence over the generic defaults referenced in Step 3 and Step 7.--**Type 1 — Canonical Knowledge Retrieval.** Sources: 5–15 primary (seminal papers, textbooks, canonical surveys) + 5–15 secondary. Primary-heavy. Target 6,000–10,000 words. Opening: historical/conceptual progression — how the field arrived at consensus. H2 count: 10–15, sequenced by dependency (concept → result → consequence). Analytical mode: explain WHY canonical results hold; compare competing formulations; signal which textbook to start with. Special rule: sequence by dependency, not by source.--**Type 2 — Market / Landscape Mapping.** Sources: 20–50 secondary (reports, press, official docs) + 5–10 primary (filings, specs, API docs). Secondary-heavy. Target 5,000–9,000 words. Opening: market definition + segmentation + the framework used to score vendors. H2 count: 8–14, one per vendor *cluster* or category — NEVER one per individual vendor. Analytical mode: take a position on who is winning and why. Special rule: at least one comparison matrix in the body is MANDATORY. Never list vendors in isolation. If the user asked for rankings, deliver with confidence.--**Type 3 — Engineering / Technical How-To.** Sources: 5–15 primary (papers, docs, GitHub, RFCs, standards) + 5–15 secondary (blog posts, tutorials, SO). Balanced. Target 4,000–8,000 words. Opening: problem statement + constraint landscape — what makes this hard. H2 count: 8–14. Analytical mode: pick a recommended approach; explain when to use alternatives. Special rule: if a method has a reference implementation, link and characterize it. Include a decision tree or selection matrix.--**Type 4 — Interpretive / Humanities Analysis.** Sources: 3–10 primary (the work itself, key scholarly monographs, canonical commentary) + 10–25 secondary (critical essays, interviews, reviews). Primary-heavy for depth. Target 7,000–11,000 words. Opening: thesis + the interpretive tradition this work sits in. H2 count: **6–10** (fewer, LONGER sections — 800–1500 words each). Sections are THEMATIC, not entity-cataloged. Analytical mode: take and defend an interpretation; engage dissenting readings. Special rule: NEVER one section per character / chapter / author. Special rule: at least 2 body sections must put sources in tension with each other. Special rule: this is the type where catalog-style reports fail hardest — double-check that no section is <400 words or entity-named.--**Type 5 — Comparative Evaluation.** Sources: 3–5 primary per comparand + 2–5 secondary per comparand (so N×5 to N×10 total). Depth scales with N. Target 5,000–10,000 words. Opening: the comparison framework — dimensions being scored and why they matter. H2 count: typically 1 per comparand + 2–4 synthesis sections (~8–12 total). Analytical mode: explicit scoring on each dimension + recommended pick with tradeoffs. Special rule: a comparison matrix in the body is MANDATORY. Special rule: equal depth across comparands — rebalance if uneven.--**Type 6 — Emerging / Cutting-Edge Research.** Sources: 5–15 primary (preprints, conference papers, authoritative reviews if any) + 3–8 secondary (news, blog posts from authors). Primary-dominant. Target 5,000–9,000 words. Opening: the open problem + why prior approaches fell short. H2 count: 8–12. Analytical mode: honest about disagreement; forecast 2–5 years out with caveats. Special rule: if a claim is from a 2024–2026 preprint, say so — don't assume consensus. Include an explicit "What we don't know yet" section. Do NOT pad with mature-field material to hit a source count.--**Type 7 — Forecast / Strategy / Recommendation.** Sources: 15–30 secondary (analyst reports, think-tanks, news) + 5–10 primary (official announcements, filings, datasets). Secondary-heavy. Target 6,000–10,000 words. Opening: the decision/question being answered + the forces shaping it. H2 count: 8–12. Analytical mode: confident prescription with explicit reasoning chain; probability language over hedging. Special rule: at least one body section must commit to a direction — not everything can be deferred to the Opinionated Synthesis. Acknowledge the time horizon explicitly.--**Type 0 — General Research (fallback).** Use universal defaults: 15–25 / 30–50 / 50–80 source floors (see below), framework opening, 12–20 H2 headings, 400–600 words per H2 (see Step 7).--**Source-count floors for Type 0 (fallback only):**-- Simple factual question: 15–25 sources-- Complex or technical topic: 30–50 sources-- PhD-level research question: 50–80 sources--**Source count is a function of request type, not topic complexity.** If you classified in Step 0, use that type's parameter block as primary guidance — fall back to the floors above only for Type 0.--**Primary-heavy vs. secondary-heavy is the axis that matters most — getting this wrong is a worse failure than getting the count wrong.** Types 1, 4, 5, 6 are primary-heavy — cite originals and engage with them deeply. Types 2, 7 are secondary-heavy — triangulate across many descriptions. Type 3 is balanced. Most sources should be cited multiple times across multiple sections. If a source earns only one citation, it's probably not pulling its weight — either deepen engagement or cut it.--Stop fetching when you can't find anything new worth adding, not when you hit a number. And be honest about the opposite: if you're still fetching past the type's range without a clear gap in mind, you're procrastinating on the writing.--**Step 1: Check what's already known**-```bash-{hpr} search "topic" --include-body -j-{hpr} sources list -j-```--**Step 2: Search broadly** — Use WebSearch with multiple queries. Don't stop at one search. Try different phrasings, related terms, and specific sub-topics. Cast a wide net.--**Step 3: Fetch EVERYTHING relevant** — A `hyperresearch-fetcher` subagent is installed in `.claude/agents/`. It uses a **cheap, fast model (Haiku)** to do the actual URL fetching. **Delegate all fetching to it** — do NOT fetch URLs yourself or use WebFetch.--For each URL or batch of URLs, spawn the fetcher agent:-```-Use the hyperresearch-fetcher agent to fetch these URLs with tag "<topic>":-- https://example.com/article1-- https://example.com/article2-```--**Spawn 10-20 fetcher agents in parallel per round** — give each agent 2-3 URLs max. They run on Haiku and cost fractions of a cent each. After each web search, immediately spawn fetchers for ALL promising URLs from the results — don't cherry-pick, fetch everything. Then do another round of searches and another round of fetchers. Repeat until you hit the source checkpoint.--If the `hyperresearch-fetcher` agent is not available (other platforms), fall back to running the command directly:-```bash-{hpr} fetch "<url>" --tag <topic> -j-```--**Step 4: Go down rabbit holes** — After fetching, read the notes:-```bash-{hpr} note show <note-id> -j-```-Look for links in the content that point to **primary sources, references, related papers, deeper material, or tangentially related topics**. Fetch those too. Then read THOSE notes and follow THEIR links.--**Keep going aggressively:**-- A blog post about a paper? Fetch the paper.-- A news article about a tool? Fetch the docs AND the GitHub repo.-- A paper cites 3 key references? Fetch all 3.-- A profile mentions a company? Fetch the company page.-- Found a related topic that might be relevant? Fetch it. You can always deprecate it later.-- A page links to raw data, CSV files, or datasets? Fetch those and analyze them.-- A source references specific statistics without citing them? Search for and fetch the primary data.--**Follow links deeply.** Don't stop at the first page. When a fetched page links to more detailed sources, primary research, raw data, or technical documentation — follow those links and fetch them too. The best insights come from primary sources, not secondhand summaries.--**Run analysis when needed.** If you find raw data (tables, CSVs, statistics), don't just quote numbers — write and run code to compute real figures, verify claims, calculate trends, and produce original analysis. Concrete numbers from actual data are worth more than vague summaries.--**The goal is exhaustive collection.** You are building a knowledge base that future agents will rely on. Missing a source is worse than having one extra note. Prune later during curation.--**Do NOT stop collecting until you are certain you have enough breadth.** After each round of fetching, ask yourself: "Are there major angles, subtopics, or perspectives I haven't covered yet?" If yes, search and fetch more. If a topic has 5 facets, you need sources on all 5 — not just the first 2 you found. Keep going until you can confidently say the collection covers the full scope of the topic. A half-covered topic is worse than no coverage at all.--**Step 4.5: Read notes efficiently — triage by frontmatter (MANDATORY)** — When inspecting notes, NEVER blindly `{hpr} note show` every note. Bodies can be 15,000+ words; frontmatter summaries are usually under 40. **Triage first, body-read only when earned.** This protocol applies throughout Steps 4, 6, and 7.--**Level 1 — Frontmatter sweep (cheap, always).** Start with:-```bash-{hpr} note list -j-```-Returns per note: `id`, `title`, `tags`, `summary`, `word_count`, `status` — **no bodies**. Scan this to build a relevance shortlist. For many claims the summary alone is enough — cite the source URL directly without re-reading the body.--**Level 2 — Frontmatter-only show.** For a single note whose summary hints at deeper value, pull full frontmatter without body:-```bash-{hpr} note show <id> --meta -j-```-Adds `source`, `parent`, `created`, `updated`, `raw_file` (if the note has a PDF). Still cheap.--**Level 3 — Inline body read (small notes only).** For `word_count < 2000` (~2700 tokens), read directly:-```bash-{hpr} note show <id> -j-```-Batch siblings: `{hpr} note show <id1> <id2> <id3> -j`.--**Level 4 — Targeted search with body, token-capped.** When pulling content from multiple notes at once:-```bash-{hpr} search "<precise question>" --include-body --max-tokens 6000 -j-```-`--max-tokens` truncates intelligently and flags `"truncated": true` on the last fitting result. Use this instead of N separate `note show` calls.--**Level 5 — Delegate heavy notes to a Sonnet subagent (MANDATORY for large notes).** For any note with `word_count > 6000` (~8000 tokens, often PDFs), do NOT dump it into your own context. Spawn a Sonnet subagent with a pointed extraction prompt:--> Fresh Sonnet subagent prompt:-> "Read `research/notes/<id>.md` (and its raw file at `research/raw/<id>.pdf` if the frontmatter has `raw_file` set). Extract ONLY what answers this specific question: '<your precise question>'. Return under 500 words, with direct quotes for any non-trivial claim and the source URL. If the note doesn't actually answer the question, say so and stop."--The subagent reads the full 20K+ tokens internally but returns only ~500 tokens to you — roughly a 40× context savings per large note, which compounds across a session.--**Level 6 — Raw PDF re-extraction (rare).** If a note has `raw_file: raw/<id>.pdf` AND the extracted markdown looks garbled or is missing content you need (specific figure, table, equation), read the raw PDF directly with the Read tool. Most of the time the extracted markdown is sufficient — only fall back here when it's clearly broken.--**Triage thresholds (guidance, not hard rules):**-- `word_count < 2000` → **Trivial**: read inline (Level 3), batch with siblings-- `word_count 2000–6000` → **Medium**: Level 2 frontmatter first; body only if summary suggests high value-- `word_count > 6000` → **Heavy**: Level 5 Sonnet delegation. Don't read inline.--Override with reason (e.g. a single 8000-word paper is your most important primary source — read it once, directly). But defaulting to "just `note show` everything" is how sessions run out of context and reports turn shallow.--**Step 5: Build the conceptual scaffold (MANDATORY — do this BEFORE writing)** — This is the single biggest lever for report quality. Strong reports are built from a scaffold, not assembled by translating notes into sections. Before you write a single paragraph of prose, answer these four questions.--**Where to put the scaffold:** keep it in your working memory, or dump it to a temp file like `/tmp/scaffold.md`. **DO NOT save it as a hyperresearch note.** The research base is for durable source material, not ephemeral pre-writing analysis. Notes are forever; scaffolds are for the next hour of writing.--1. **What is the hard question?** One sentence — not the surface query, the underlying difficulty. ("Why is eukaryotic HGT rare despite being evolutionarily important?" not "What is HGT?")-2. **What is the naive answer?** What a reader would guess before reading the sources. This is your foil — the thing the report has to explain away.-3. **What is the structural tension?** What makes this topic worth 10,000 words instead of 500? The constraint, the paradox, the thing sources disagree on, the reason the field hasn't converged.-4. **What is the progression?** Sketch 6–12 H2 section headings in **dependency order**. Each section must BUILD on the one before, not sit parallel to it. A reader should only be able to understand section 3 because they read sections 1 and 2. If your headings could be read in any order, that's a catalog, not a report.--The opening section of the final report MUST be a framework section — not "What is X." It should establish the hard question, the naive answer, and why the naive answer is insufficient. This is what separates analytical reports from catalog-style ones.--**Step 6: Compare sources across a common axis (MANDATORY)** — Before writing the body, find 3–5 places where your sources actually disagree, emphasize different things, or frame the same problem differently. Use search to locate them:-```bash-{hpr} search "topic" --include-body -j-```--For each disagreement, capture a short comparison block (in your working memory or in the same scratch file as the scaffold from Step 5 — **NOT as a hyperresearch note**; these are ephemeral analysis artifacts, not durable sources):-- what each source says, with URLs-- why they differ (different data? methodology? assumptions? era? scope?)-- your position on which is more credible, with reasoning--These comparisons become the backbone of body sections — they will end up as prose *inside* the final report, not as separate KB entries. Sources earn their citations by being **compared**, not by being listed. If every source is corroborating every other source, you didn't need many sources — go back and search for contrarian views, criticism, or alternative frameworks. Reference-quality reports always surface disagreement; catalog-style reports never do.--**Step 7: Write the draft** — First, pull up your Step 0 classification. The parameter block for your primary type sets target length, H2 count, opening shape, and analytical mode. The hard constraints below are Type 0 defaults — they apply EXCEPT where your type's block overrides them. Notable overrides:-- **Type 4 Humanities** targets 6–10 longer H2s (800–1500 words each), not 12–20.-- **Type 2 Market** wants one section per vendor *cluster*, not per individual vendor.-- **Type 5 Comparative** requires a mandatory comparison matrix in the body.-- **Type 6 Emerging** requires a "What we don't know yet" section.--Now, and only now, write the report. Follow these rules as hard constraints (under Type 0; type-specific overrides apply where listed above):--- **Target 400–600 words per H2 section.** If a section is under 200 words, merge it into a sibling. If over 800, split it — but only if the split sections stay above 300. (Type 4 Humanities: 800–1500 per section.)-- **Aim for 12–20 H2 headings on a 10K-word report; 8–15 on a 5K-word report.** If you're above 25, you are fragmenting. Merge. (Type 4 Humanities: 6–10.)-- **Never write one section per source or one section per named entity.** Sources serve sections, not the other way around. A single H2 section should weave 3–8 sources together. A character, vendor, or paper does not earn its own section unless it has >400 words of analysis attached.-- **Every body section ends with an analytical beat** — a 1–2 sentence "so what does this tell us" that integrates across the sources just cited. Not a summary; a step forward in the argument. The Opinionated Synthesis at the end is NOT supposed to carry the synthesis load alone.-- **Open with the framework, not a definition.** Section 1 = the hard question, the naive answer, the structural tension. Not "X is a Y that does Z." Definitions can go into a sub-section or a sidebar.-- **Cite inline with parenthetical URLs.** Every non-trivial claim needs one. Prefer primary sources over secondary summaries.-- **Use comparison tables, not fact tables.** A good table contrasts cases on a common axis (e.g., Recipient → Donor → Gene → Function). A bad table dumps attributes of a single entity (Name | Star | Technique | Fate). If your table is dumping facts about one thing, it probably belongs in prose.-- **Word count targets by topic difficulty.** Simple factual: 3,000–5,000. Complex or technical: 6,000–9,000. PhD-level: 9,000–12,000. Hitting these is easier when sections are 400–600 words each, not 80–150. Missing these is usually a signal that the scaffold from Step 5 is off — revisit it.--**Step 8: Gap analysis (MANDATORY)** — After writing your draft, stop and compare it against the original query. Re-read the user's request word by word. Ask yourself:-- What specific questions did they ask that I haven't fully answered?-- What subtopics or angles did I miss entirely?-- Where did I make claims without strong enough evidence?-- What would an expert in this field say is missing?--Then launch a new round of research to fill every gap — web search, fetch, follow links, the full workflow. Spawn fetcher subagents in parallel for the new URLs. Add the new material to your draft.--**Step 9: Adversarial audit (MANDATORY — do this twice)** — After the gap-filling round, launch two subagents in parallel to audit the revised draft. This runs up to 2 loops.--Spawn two agents in parallel:-- **Agent 1 — Comprehensiveness auditor:** "Read this report and search the research base (`{hpr} search` and `{hpr} note show`) to find gaps. What subtopics, angles, data points, or counterarguments are missing? What claims lack citations? What sections are shallow? Compare against the original query and be ruthless — list every gap."-- **Agent 2 — Logic and structure auditor:** "Read this report as a domain expert. The report should declare its Step 0 request-type classification and follow that type's parameter block. Check specifically: (1) Does section 1 establish a framework (hard question, naive answer, structural tension) or just define terms? (2) Are sections in dependency order — does each build on the previous, or could they be shuffled? (3) Short sections: are there any under 200 words that should be merged into siblings? (Type 4 Humanities floor is 400, not 200.) (4) Count H2 headings. Fragmentation threshold varies by type: Type 0 General >25 on a <12K-word report; Type 4 Humanities >12; Type 2 Market >15. (5) Does each section end with an analytical beat, or does it just stop? (6) Are sources compared against each other anywhere, or only listed in parallel? (7) Does the argument flow logically — leaps, unsupported claims, contradictions? (8) Does the report match its declared request type? If Type 4 Humanities, are sections thematic rather than entity-catalog? If Type 5 Comparative, is there a mandatory comparison matrix? If Type 6 Emerging, is there a 'What we don't know yet' section? If Type 2 Market, is there a position on who is winning? Flag every type violation. List every weakness with specific section names and quotes."--Pass your full draft report text to both agents. They search the research base independently and report back.--**After each audit round:**-1. Read both agents' feedback-2. If they identified missing topics or weak arguments — do another round of web search and fetching to fill the gaps-3. If the structure auditor flagged fragmentation, over-short sections, or a missing framework — **consolidate headings and rewrite, don't just patch.** Merging five 100-word sections into one 500-word section is a rewrite, not an edit. Do not skip this; it's where catalog-style drafts become argument-style drafts.-4. Rewrite the report incorporating new sources and fixing structural issues-5. Run the audit again (second loop) on the revised report--**You MUST do at least one audit loop. Do two if the first audit found significant gaps.** Only after the auditors have no major complaints should you finalize the report.--**While audit agents are running, don't idle.** Use the wait time productively:-- Improve summaries on notes that have weak or missing summaries-- Add more specific tags to notes that only have one generic tag-- Add `[[wiki-links]]` between related notes to build the knowledge graph-- Run `{hpr} lint -j` and fix any issues it finds-- Read notes you haven't read yet and look for links worth following in future rounds--**Step 10: Opinionated Synthesis (MANDATORY — end of report)** — A catalog of facts is not a report. You **MUST** append a section titled `## Opinionated Synthesis` at the very end of the report. This is where you stop reciting what sources said and start reasoning across them. **This section complements — it does not replace — the per-section analytical beats from Step 7.** If the body sections are purely descriptive and all the analysis is dumped in this section, the report is still a catalog with a synthesis bolted onto the end.--The synthesis section must include:--1. **Comparative analysis across categories** — don't just describe each concept/group/topic in isolation. Compare them. Use tables where useful.-2. **Thematic threads and progression** — what patterns cut across the topic? What's the narrative arc?-3. **Your reasoned position** — take a defensible stance, clearly labeled as your own analysis grounded in the sources.-4. **Open questions and limitations** — what did the sources NOT answer?-5. **Concluding thoughts** — one tight paragraph answering the original question.--Use these exact subheadings at the end of your report:--```markdown-## Opinionated Synthesis--### Comparative Analysis-<cross-cutting comparisons — tables where useful>--### Thematic Threads-<patterns, progressions, throughlines>--### My Reasoned Position-<your defensible stance, clearly labeled as analysis>--### Open Questions-<gaps in the research, what the sources didn't answer>--### Concluding Thoughts-<one tight paragraph, answers the original question>-```--**Do NOT skip this section.** Even a 9,000-word body of cataloged facts is incomplete without it. The synthesis is what separates a research report from a Wikipedia dump.--**Step 11: Prune** — After the final report is done, deprecate notes that turned out to be irrelevant:-```bash-{hpr} note update <id> --status deprecated -j-```-This keeps the KB clean without losing anything permanently.+**The user's verbatim prompt is gospel.** SKILL.md requires the prompt to be copied verbatim into the first section of `research/scaffold.md` — every subsequent step re-reads it from there. The adversarial audit at Step 11 grades the draft against the verbatim prompt, not against an abstract notion of quality.### Why {hpr} fetch, not WebFetch@@ -312,15 +47,46 @@ PDF extraction is fast and produces clean text from all pages. **Do not skip a sRaw files are automatically saved to `research/raw/<note-id>.pdf` and linked from the note's frontmatter (`raw_file: raw/<note-id>.pdf`). You can read the raw PDF directly if you need to verify content or extract figures.-### Use scholarly APIs when available+### MANDATORY: Academic API sweep before web search++For any topic with a research literature, **query academic APIs BEFORE running web searches.** Academic APIs return citation-ranked canonical papers. Web search returns derivative commentary. This order matters — getting it backwards biases your entire source set toward secondhand summaries.++**Semantic Scholar** — citation-count search + forward/backward citation chaining:+```python+import urllib.request, json, time, urllib.parse+q = urllib.parse.quote("your topic")+url = "https://api.semanticscholar.org/graph/v1/paper/search?query=" + q + "&fields=title,year,citationCount,externalIds&limit=10"+with urllib.request.urlopen(url) as r:+ papers = json.loads(r.read())["data"]+papers.sort(key=lambda p: p.get("citationCount", 0), reverse=True)+# Then citation chain: for top 3 papers, fetch references + citations+for paper in papers[:3]:+ pid = paper["paperId"]+ refs = "https://api.semanticscholar.org/graph/v1/paper/" + pid + "/references?fields=title,year,citationCount&limit=30"+ cits = "https://api.semanticscholar.org/graph/v1/paper/" + pid + "/citations?fields=title,year,citationCount,isInfluential&limit=50"+ # fetch both, sort results by citationCount, add top papers to fetch queue+ time.sleep(0.4)+```+Backward chaining finds the foundational canon; forward chaining finds everything built on it in the last 3 years.-For academic research, use APIs to find and access papers programmatically:-- **arXiv API** — `https://export.arxiv.org/api/query?search_query=...` returns XML with abstracts, authors, PDF links-- **Semantic Scholar API** — `https://api.semanticscholar.org/graph/v1/paper/search?query=...` returns structured paper data with citations-- **CrossRef API** — `https://api.crossref.org/works?query=...` for DOI lookups and metadata-- **PubMed/NCBI API** — for biomedical research+**arXiv API** — for cs.*, stat.*, q-bio.*, econ.*, math.*, physics.*:+```+https://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+all:<topic>&sortBy=relevance&max_results=25+```+Returns Atom XML with titles, abstracts, and direct PDF links. Feed the PDF links to `{hpr} fetch`.++**OpenAlex** — for humanities, social science, medicine, or non-arXiv disciplines:+```+https://api.openalex.org/works?search=<topic>&sort=cited_by_count:desc&per-page=15&[email protected]+```+Covers 250M+ works across all disciplines. `related_works` on a known paper finds its citation neighborhood.++**PubMed eutils** — for biomedical/clinical topics:+```+https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=<topic>&retmode=json&retmax=20+```-Write code to call these APIs when the topic is academic. The structured data (citation counts, related papers, abstracts) is often more useful than scraping a webpage. Fetch the actual papers via their PDF links after finding them through the API.+After the academic sweep, run web searches for context, news, and non-academic angles — including at least one adversarial search ("criticism of X", "limitations of X", "X does not work").### Raw content is kingdiff --git a/src/hyperresearch/skills/research-collect.md b/src/hyperresearch/skills/research-collect.mdnew file mode 100644index 0000000..d19cffd--- /dev/null+++ b/src/hyperresearch/skills/research-collect.md@@ -0,0 +1,127 @@+---+name: research-collect+description: >+ Enumerative coverage research. The user wants to know what exists, what+ happened, who did what — in full. Primary virtue is exhaustive per-entity+ coverage with every requested field. Sources are artifacts, official+ references, and anything that establishes ground truth on what's in the set.+---++# Collect Protocol++> You are in **collect** modality. The dispatcher routed you here because the output needs to enumerate a set of entities or facts, not advance a thesis. If the prompt said "for each X, provide Y / Z / W", the contract is clear: every X gets every field. If the prompt named a category without naming members, your job is to identify the significant members and cover them.++Read the dispatcher (`.claude/skills/hyperresearch/SKILL.md`) for the process. This file is the SUBSTANCE reference for collect.++---++## What collect is for++| Prompt shape | Why it's collect |+|---|---|+| "For each Napoleonic marshal, cover key campaigns, defeat, and death" | Per-entity field list |+| "Analyze the armor classes in Saint Seiya: for each significant character, provide power / technique / arc / fate" | Per-entity field list |+| "List the major infrastructure bills passed between 2020 and 2025 with their key provisions" | Enumerative scope |+| "What are all the endogenous opioid receptors and what does each do?" | Enumerative scope |+| "Survey the state capitals of India and their major industries" | Per-entity field list |++Collect is NOT for: "explain how X works" (synthesize), "which is best, X or Y" (compare), "will X happen in 2026" (forecast).++---++## Source strategy++Collect is **primary-source-heavy for the artifact itself, then secondary for the per-entity fields.**++**Ground truth (first priority):**+- Official / authoritative registries, directories, catalogs, databases — whatever lists the enumerable set+- The primary artifact itself (if the subject is a text, franchise, or body of work)+- For historical enumerations: primary documents, archival sources, official records+- For scientific enumerations: peer-reviewed classifications, reference databases (NCBI, ChEBI, PDB, etc.)++**Institutional signal (second priority):**+- Encyclopedia entries (Wikipedia, domain-specific wikis — these are load-bearing for collect because they often *are* the enumerable set)+- Reference books and field handbooks+- Academic reviews that enumerate and classify++**Practitioner triangulation (third priority):**+- Named practitioners who have catalogued the set (niche experts, fandom compilers, historical societies)+- Community-curated databases (TV Tropes, Fandom wikis, GitHub awesome-lists)++**Commentary (de-prioritized):**+- Listicles and "top N" posts are often derivative of the reference sources above. Use them only as leads, not as citations for per-entity facts.++### Academic sweep — only if peer-reviewed literature enumerates the set++Run Semantic Scholar / OpenAlex only if the enumerable set has a scholarly classification literature. For fictional / pop-culture / product / historical enumerations, skip it — the reference wikis and primary sources are the authoritative tier.++### Adversarial round++Still required. Run the dispatcher's adversarial searches to catch:+- Missing members the consensus list omits+- Disputed members (is this really in the set?)+- Members that get grouped differently in non-mainstream sources++A collect report with no engagement with classification disputes is a transcription, not research.++---++## Substance rules for Step 9 (writing)++These are the collect-specific rules that layer on top of the dispatcher's shared writing constraints.++### 1. The coverage contract is non-negotiable++If the prompt says "for each X, provide Y / Z / W", the draft must deliver every X with every field. If the prompt named 108 entities, the draft covers all 108. If the prompt said "each significant X", YOU define the significance threshold explicitly in the scaffold (e.g., "named in ≥2 primary sources" or "featured in a canonical arc"), state it in the opening, and then honor it without silent downgrades.++**Representative examples + thematic commentary is NOT a substitute for enumerative coverage.** If you cannot deliver the full set inside the draft, surface that to the user and ask — do not paper over it.++### 2. Proportionate depth across entities++Every entity the prompt asked about gets proportionate treatment. If Gold Saint #1 gets 200 words, Gold Saint #12 gets approximately 200 words. Deeply over-covering the famous entries and thin-covering the obscure ones is a violation — the prompt asked for "each", not "the ones you find interesting".++### 3. Structured entity treatment is allowed and expected++Collect is the one activity where **structured per-entity treatment** (including tight dossier-style subsections, roundup tables, or densely packed per-entity paragraphs) is the correct shape. You do NOT have to write full essayistic paragraphs for every entity. A per-entity subsection with clear field labels (technique, role, fate, outcome) and 2–4 sentences of prose per field is correct, efficient, and legible.++However: **dense roundup is not license for bullet-only listicles.** Each field still needs a sentence of context or consequence, not just a 3-word noun phrase. And the section as a whole still needs an opening orientation paragraph and a closing analytical beat — the per-entity dossier is flanked by interpretive framing, not presented raw.++### 4. Grouping is legitimate when the set is very large++For sets larger than you can cover with individual treatment (100+ entities), group by natural categorical structure (rank, era, region, function). Every entity still appears, but less-significant entities can share a subsection with shorter per-entity lines. The scaffold's coverage checklist records the grouping strategy; the auditor verifies every entity from the source set is accounted for.++### 5. Opening establishes the scope and the significance threshold++The opening answers: *what is the set you're enumerating, how did you define its boundaries, and what significance threshold did you apply*. A reader must understand in the first 200 words what "every X" means in this draft. If the set has disputed membership, the opening names the dispute and states your inclusion rule.++### 6. Closing synthesizes the set++The closing section (before the Opinionated Synthesis) draws out what the enumeration as a whole reveals. This is where a collect draft earns its analytical beat: what pattern does the full set expose? What does the coverage reveal that no single entity could? This is NOT a thesis replacement — the draft is primarily enumerative — but every corpus-level report should have one moment where the whole is more than the sum.++---++## Conformance checks (auditor reads this at Step 11, mode=conformance)++When the auditor runs in conformance mode on a collect draft, it applies these checks in order. Each is a PASS / FAIL with a one-line quote from the report or the vault.++(0) **Verbatim prompt check.** Open `research/scaffold.md` — does its first section contain the user's verbatim prompt (unchanged, not paraphrased)? FAIL if missing or modified.++(1) **Coverage contract check.** Read the scaffold's "What the user explicitly asked for" section. For each explicit entity or field requirement, verify the draft delivers it. If the user said "for each of 108 Specters provide techniques and fate", count the draft's Specter entries and verify every entry has both fields. FAIL if any entity or field is silently dropped.++(2) **Proportionate depth check.** Count substantive claims per entity across the draft (not raw word count — count assertions about the entity's attributes, actions, or fate). Flag any entity with less than ~50% the depth of the most-covered entity in the same category. FAIL if the draft over-develops the famous entries and thins the obscure ones.++(3) **Scaffold + comparisons + extract artifacts exist.** Run `$HPR search 'scaffold' -j`, `$HPR search 'comparisons' -j`, and `$HPR note list --tag extract -j`. All must return non-empty. FAIL otherwise.++(4) **Provenance chain.** Run `$HPR lint --rule provenance -j`. FAIL if any `--suggested-by` issues.++(5) **Analyst coverage.** Run `$HPR lint --rule analyst-coverage -j`. FAIL if the extract:source ratio is below 30%.++(6) **Opening scope + significance threshold.** Read the opening. Does it state what set is being enumerated, how boundaries were defined, and what significance threshold was applied? FAIL if any of the three is missing.++(7) **Corpus-level analytical beat.** Read the closing body section (not the Opinionated Synthesis). Does it draw out a pattern the full set reveals? FAIL if the closing is just a transition to synthesis without a substantive pattern claim.++(8) **Adversarial engagement.** Does the draft acknowledge at least one disputed / ambiguous / contested classification? FAIL if the enumeration is presented as uncontested when the adversarial search round found disputes.++(9) **Cross-source tension at least twice.** At least two body sections must put specific sources in tension with explicit "Source A says X; Source B says Y; I hold Z because..." structure. FAIL if the draft only cites, never contrasts.++Auditor output: list every violation with a one-line quote from the report. The parent agent applies fixes.diff --git a/src/hyperresearch/skills/research-compare.md b/src/hyperresearch/skills/research-compare.mdnew file mode 100644index 0000000..6ba509f--- /dev/null+++ b/src/hyperresearch/skills/research-compare.md@@ -0,0 +1,169 @@+---+name: research-compare+description: >+ Comparative evaluation research. The user wants to know which option is+ better for what, how alternatives differ, and which one to pick. Primary+ virtue is proportionate per-entity depth plus a committed recommendation.+ Covers vendor scans, tool evaluations, approach selection, and any+ prompt that names multiple alternatives.+---++# Compare Protocol++> You are in **compare** modality. The dispatcher routed you here because the output needs to evaluate N alternatives against each other and commit to a recommendation. The reader leaves knowing which option to pick and why, plus the conditions under which alternatives win.++Read the dispatcher (`.claude/skills/hyperresearch/SKILL.md`) for the process. This file is the SUBSTANCE reference for compare.++---++## What compare is for++| Prompt shape | Why it's compare |+|---|---|+| "TigerBeetle vs Postgres vs FoundationDB for write-heavy workloads" | Named alternatives, explicit comparison |+| "Which JavaScript framework should I pick for a new SaaS startup" | Unnamed set, evaluation ask |+| "How does Kubernetes compare to Nomad for stateful workloads" | Two named alternatives |+| "What's the best reinforcement learning approach for robotic manipulation" | Named alternatives, selection ask |+| "Compare TLS 1.3 with QUIC for mobile apps" | Two named alternatives, comparison ask |++Compare is NOT for: "what exists" (collect), "explain how X works" (synthesize), "will X win" (forecast — if the ask is future-tense).++**Two variants:**+1. **market-scan** — entity set is discovered through research (the user asked about a space, not specific alternatives)+2. **comparison** — entity set is named explicitly in the prompt++Both use this file. The variants differ only in how you acquire the entity list.++---++## Source strategy++Compare is a **3-tier source hierarchy** that privileges the artifact itself, then authoritative synthesis, then practitioner reality checks.++### Ground truth (first priority — fetch for every entity)++- Official product / project pages, feature lists, pricing pages, API documentation, release notes+- Company filings, official announcements, specs+- Benchmark results (both official and independent)+- Source code / README / architecture docs for open-source entities+- Standards documents (RFC, ISO, IEEE) when the entities implement a standard++**For comparison variant:** fetch ground-truth sources for EACH named entity before moving to Tier 2. Imbalanced ground-truth coverage (entity A has 5 primary sources, entity B has 1) propagates to imbalanced body sections later.++### Institutional signal (second priority)++- Authoritative analyst reports and research firm writing+- Named expert reviews and long-form evaluations by recognized practitioners+- Academic / technical papers when the comparison is between methods rather than products+- Industry benchmarking bodies and standards organizations++**Paywall fallback:** when reports are gated: (1) search for "[report title] key findings"; (2) look for press coverage of the report; (3) find earlier free editions; (4) check the organization's free-publications section.++### Practitioner triangulation (third priority)++- Named practitioner postmortems and migration reports (the most honest signal in the ecosystem)+- Conference talks with transcripts (by practitioners who built the thing, not evangelists)+- StackOverflow canonical answers (accepted + high-upvote)+- Community threads (Hacker News, Reddit, Lobste.rs) on specific failure modes+- User reviews on professional review sites (not single-line "it's great" reviews)++### Commentary (load-bearing only for reception, not facts)++- News articles about product launches+- Blog summaries by authors who haven't used the thing+- Listicles++Never use commentary as the sole source for a factual claim about an entity. Use it only to characterize positioning or reception.++### Adversarial round++Required — specifically for the market leader. A compare report without at least one source critical of the dominant entity is advocacy, not evaluation.++```+WebSearch("<market-leader entity> problems limitations")+WebSearch("<market-leader entity> alternatives why leave")+WebSearch("moving away from <market-leader entity>")+WebSearch("<method A> vs <method B> when <method A> wins")+```++### Academic sweep — only when comparing methods, not products++If the comparison is between academic methods, algorithms, or theoretical approaches (e.g., "PPO vs SAC for continuous control"), run Semantic Scholar first. If the comparison is between products or services, skip the academic sweep and go straight to vendor docs + analyst reports.++---++## Substance rules for Step 9 (writing)++These are the compare-specific rules that layer on top of the dispatcher's shared writing constraints.++### 1. Scoring dimensions declared upfront++The opening establishes WHICH dimensions the entities will be evaluated on — and WHY those dimensions matter for the user's question. "Throughput", "correctness guarantees", "operational complexity", "ecosystem maturity" are dimensions; "performance" alone is not. Dimensions should reflect where the entities actually differ, not generic attributes.++A compare draft with undeclared or arbitrary dimensions is a set of entity descriptions pretending to be an evaluation. Declare them first; score against them throughout.++### 2. Proportionate depth per entity is the PRIMARY substance rule++Every entity named in the prompt (or discovered during market-scan) gets proportionate body coverage. If entity A has a 1,200-word section and entity B has a 400-word section, rebalance — either expand B's coverage or trim A's.++**This rule holds regardless of section structure.** Whether the draft has per-entity sections, per-dimension sections, clustered sections, or prose-embedded comparisons, count substantive claims per entity across the whole body. If one entity ends up with half the coverage of another, the research was lopsided, not the draft.++### 3. A mandatory comparison matrix++Somewhere in the body there is a table (or equivalent structured view) that places every entity against every scoring dimension. This is non-negotiable — no compare draft is complete without it. If the prompt requests prose comparison over a table, the matrix can be rendered as a structured prose section ("On throughput: A leads with X; B follows at Y; C lags at Z because..."), but every entity must be parseable on every dimension in a single view.++If a dimension cannot be scored for an entity (e.g., a private company with no public pricing), note the gap and explain why. Do not silently omit cells.++### 4. A committed recommendation++Compare drafts MUST end with a committed pick: "For [primary use case], use X because [reason]. For [condition A], Y wins because [reason]. For [condition B], Z wins because [reason]." "It depends" alone is not acceptable — it must be followed by the conditions that determine the answer.++The recommendation is a claim the reader can act on, not a balanced list.++### 5. Entities in tension, not in isolation++Body sections must put entities in direct tension: "A and B both claim to handle C; the difference is that A does C1 well while B does C2 well." A draft that describes each entity in isolation and then synthesizes at the end is missing the comparative substance that justifies the modality.++### 6. Opening establishes the question the evaluation answers++What is the user trying to decide? The draft frames this explicitly in the opening, then every body section is a step toward that decision. The opening is NOT a description of the first entity — it is the problem statement + the dimensions the solution will be scored on.++### 7. Entity sections open with mechanism, then method++For each entity: first explain what it IS and HOW it works (mechanism), then what you DO with it (method / usage / interface). Readers evaluate alternatives best when they understand the underlying model, not just the surface API.++### 8. Honest about maturity asymmetry++If the entities have dramatically different maturity (A has 10 years of production use and benchmarks; B is a 2024 research prototype with no independent replication), flag this explicitly in the opening AND in each affected section. Treating asymmetric alternatives as if they were evenly characterized is a form of dishonesty — the reader needs to know the evidence bases are uneven.++---++## Conformance checks (auditor reads this at Step 11, mode=conformance)++(0) **Verbatim prompt check.** Open `research/scaffold.md` — does its first section contain the user's verbatim prompt (unchanged)? FAIL if missing or modified.++(1) **All prompt-named entities appear.** If the prompt named specific entities, every one must appear in the draft with substantive coverage. FAIL if any named entity is missing or relegated to a passing mention.++(2) **Scoring dimensions declared.** Read the opening. Are the comparison dimensions named and their relevance explained? FAIL if dimensions are undeclared or arbitrary.++(3) **Proportionate depth per entity.** Count substantive claims per entity across the whole body. Flag any entity with less than ~50% the depth of the most-covered entity. FAIL if the draft over-develops one entity and thin-covers others.++(4) **Comparison matrix present.** Find the matrix (table or structured prose). Verify every entity is scored on every declared dimension. FAIL if the matrix is missing, incomplete, or present only as a synthesis afterthought.++(5) **Committed recommendation.** Read the recommendation section. Does it commit to a specific pick for the primary use case? If it uses "it depends", does it immediately enumerate the conditions and their answers? FAIL if the draft hedges without committing.++(6) **Entities in tension.** Count body sections where entities are put in direct tension (not described in isolation). FAIL if fewer than two — isolation descriptions are not comparisons.++(7) **Adversarial engagement with the market leader.** Find sources critical of the dominant entity. Is the draft's body engaging with those critiques, or is the recommendation a consensus echo? FAIL if there is zero engagement with dissent on the leader.++(8) **Scaffold + comparisons + extract artifacts exist.** Run `$HPR search 'scaffold' -j`, `$HPR search 'comparisons' -j`, `$HPR note list --tag extract -j`. All non-empty. FAIL otherwise.++(9) **Provenance chain.** Run `$HPR lint --rule provenance -j`. FAIL if any issues.++(10) **Analyst coverage.** Run `$HPR lint --rule analyst-coverage -j`. FAIL if below 30%.++(11) **Mechanism-before-method per entity.** Sample 3 entity sections. Does each open with mechanism (what it is, how it works) before method (how to use it)? FAIL if sections jump straight to usage without the underlying model.++(12) **Maturity asymmetry flagged if present.** If the entities have dramatically different evidence bases, is this flagged in the opening and in affected sections? FAIL if the draft presents asymmetric alternatives as if they were evenly characterized.++Auditor output: list every violation with a one-line quote from the report.diff --git a/src/hyperresearch/skills/research-forecast.md b/src/hyperresearch/skills/research-forecast.mdnew file mode 100644index 0000000..6f6a45c--- /dev/null+++ b/src/hyperresearch/skills/research-forecast.md@@ -0,0 +1,163 @@+---+name: research-forecast+description: >+ Forward-looking research. The user wants to know what will happen or+ what should be done. Primary virtue is a committed prediction or+ recommendation grounded in past + current state with an explicit time+ horizon. Sources are ground-truth statistics, institutional analysis,+ policy documents, and practitioner positioning. NOT academic papers.+---++# Forecast Protocol++> You are in **forecast** modality. The dispatcher routed you here because the output needs a committed position on something that has not yet happened: a prediction, a recommendation, a strategy. The reader leaves knowing what you expect / recommend and the reasoning chain that got you there. **Do not hedge.** Use probability language, not "it is possible".++Read the dispatcher (`.claude/skills/hyperresearch/SKILL.md`) for the process. This file is the SUBSTANCE reference for forecast.++---++## What forecast is for++| Prompt shape | Why it's forecast |+|---|---|+| "Will US inflation stay above 3% through 2026" | Prediction |+| "What should the US Navy do about the China shipbuilding gap" | Strategy / recommendation |+| "How will LLM pricing evolve over the next 2 years" | Forward-looking |+| "Is the AI bubble going to burst and when" | Prediction |+| "What's the right architecture for building a trading firm in 2026" | Forward-looking recommendation |++Forecast is NOT for: "what exists now" (collect), "how does X work" (synthesize), "which option is best today" (compare).++---++## Source strategy++Forecast is **secondary-heavy** — triangulation across many institutional perspectives matters more than any single primary source. BUT the forecast must still be **anchored in ground-truth** (official stats, filings, policy text). The structure is:++**Ground truth (anchoring evidence):**+- Official policy documents, regulatory filings, legislative text+- Government statistics and economic data (central banks, national statistics offices, international bodies)+- Company earnings guidance, SEC filings, official roadmaps and announcements+- Primary datasets with time-series data for trend establishment++**Time-horizon rule:** for 6–12 month forecasts, prioritize data from the last 12 months. For 3–5 year forecasts, prioritize structural trend data spanning 5–20 years. A short-horizon forecast built on 20-year structural data will miss the current regime; a long-horizon forecast built only on last-quarter data is noise-chasing.++**Institutional signal (main triangulation layer):**+- Research organizations and think-tanks relevant to your domain+- Industry analysts and sector research firms+- International organizations (IMF, OECD, World Bank for macro; ITU for telecom; IEA for energy)+- Named economists, strategists, and domain experts in institutional roles++**Find the right institutional voice for the domain.** Do not default to generic management consultants — find the analyst body with domain authority. Energy forecasts come from the IEA and industry-specific firms, not from McKinsey. Tech forecasts come from Gartner / IDC / 451 Research and specialized analysts, not general news.++**Paywall fallback:** when reports are gated: (1) search for "[report title] key findings"; (2) press coverage of the report; (3) the organization's free-publications section; (4) earlier free editions.++**Practitioner triangulation:**+- Expert commentary — op-eds and essays by named practitioners (not anonymous)+- Practitioner forecasts and positioning signals (how people operating in this domain are actually betting)+- Conference keynotes with transcripts++**Contrarian and dissenting voices:** independent analysts who challenge the consensus. These are load-bearing for forecast — the contrarian case is not optional; it must be represented with substance.++### Do NOT run academic API sweeps++Academic papers are the wrong sources for forecasting. Citation-ranked scholarship describes what's known about the past, not what will happen. Skip Semantic Scholar / arXiv / OpenAlex / PubMed for forecast queries.++### Adversarial round — mandatory++Run these before proceeding past the seed fetch:++```+WebSearch("<topic> bear case downside risk")+WebSearch("<topic> contrarian view dissenting forecast")+WebSearch("why <dominant forecast> is wrong")+WebSearch("<topic> worst case scenario")+WebSearch("<topic> historical failure analogous case")+```++A forecast without explicit engagement with the failure case is not analysis — it is advocacy.++---++## Substance rules for Step 9 (writing)++These are the forecast-specific rules that layer on top of the dispatcher's shared writing constraints.++### 1. Commit to a position++The draft is built around a committed forecast or recommendation. Use probability language: "more likely than not", "substantial probability", "unlikely unless", "approaches certainty in the 12-month horizon". Do NOT use hedge language: "it is possible", "some experts believe", "time will tell". Probability language commits to a distribution; hedging commits to nothing.++### 2. Time horizon is explicit++Every forecast claim carries a time scope. If the user specified a horizon ("what happens by 2026"), deliver claims scoped to that horizon. If the prompt is silent on time, distinguish between:+- **Short-term** (6–12 months): near-certain / high-confidence dynamics+- **Medium-term** (1–3 years): trend-dependent, structural moves+- **Long-term** (3–5+ years): speculative, path-dependent++A forecast that conflates short- and long-term claims is incoherent. Either scope every claim or dedicate sections to each horizon.++### 3. Directional commitment in the body, not only synthesis++At least one substantive body section must take a clear directional stance — a claim about what will happen or what should be done. "The synthesis will take a position" is not sufficient. The reasoning has to commit along the way, not just at the end.++If the user requested scenario-neutral analysis (e.g., "outline optimistic, central, and bear cases"), directional commitment lives *within each scenario branch*. Each scenario has a named probability and its own committed path — the branching is not an escape from commitment.++### 4. Engage the contrarian case with substance++Find the strongest argument against your forecast / recommendation. Give it real engagement — at minimum a full paragraph, ideally a full section. Represent it at its strongest. Then explain why you reject it.++If the consensus view gets 3 sections of analysis, the bear case deserves comparable substantive coverage. A one-paragraph dismissal of dissent is strawmanning, not stress-testing.++### 5. Historical precedent grounds the prediction++Anchor forward-looking claims in at least one analogous historical case. "This looks like 1998-99 in tech valuations" is stronger than "valuations are high". If the domain is genuinely unprecedented (which is rare), say so explicitly and explain why past cases don't apply.++A forecast with zero historical grounding has no epistemic floor — it is a guess.++### 6. Tier weighting++Anchor the forecast in `ground_truth` sources (official statistics, filings, policy text). Use `institutional` (analyst reports, think-tanks) for consensus positioning. Use `practitioner` (expert commentary, positioning signals) and `commentary` (contrarian op-eds) for the bear case and dissent.++Do NOT build a forecast on `commentary` alone. A forecast grounded in news opinions is an opinion amplifier, not a forecast.++### 7. Opening frames the decision and the forces++The opening answers: *what decision or question is this answering, and what are the 3–5 forces that will determine the outcome*. The reader understands in the first 300 words what is at stake and what the draft is actually measuring.++### 8. Explicit "what would change my mind"++The draft must include — in the body or in the synthesis — a statement of what evidence would shift your position. "My forecast of X would flip if Y happens, because Z" is load-bearing: it separates a real forecast from a religious conviction.++---++## Conformance checks (auditor reads this at Step 11, mode=conformance)++(0) **Verbatim prompt check.** Open `research/scaffold.md` — does its first section contain the user's verbatim prompt (unchanged)? FAIL if missing or modified.++(1) **Committed position stated.** Read the opening and the synthesis. Is there a clear committed forecast / recommendation? FAIL if the draft hedges without committing or hides commitment behind "it depends" language alone.++(2) **Probability language, not hedge language.** Sample 10 predictive claims in the body. Do they use probability language ("more likely than not", "substantial probability", "unlikely unless") or hedge language ("it is possible", "some experts believe")? FAIL if hedge language dominates.++(3) **Time horizon explicit.** Every forecast claim carries a time scope — either inline or via sectioning (short-term / medium-term / long-term). FAIL if horizon is ambiguous or claims conflate time scales.++(4) **Directional commitment in the body, not only synthesis.** At least one substantive body section takes a clear directional stance, not just the Opinionated Synthesis. FAIL if commitment is wholly deferred to the closing synthesis.++(5) **Contrarian case engaged with substance.** Find the bear-case section or engagement. Is it a substantive paragraph (or more), representing the opposing view at its strongest? FAIL if dissent is dismissed in one sentence or strawmanned.++(6) **Historical precedent present.** Find at least one analogous historical case used to ground the prediction. FAIL if the draft makes forward-looking claims with zero historical anchoring AND does not explicitly justify why the domain is unprecedented.++(7) **Tier weighting.** Are forecast claims anchored in `ground_truth` tier sources? Is any load-bearing claim resting solely on `commentary`? FAIL if commentary is the sole support for a substantive prediction.++(8) **Scaffold + comparisons + extract artifacts exist.** Run `$HPR search 'scaffold' -j`, `$HPR search 'comparisons' -j`, `$HPR note list --tag extract -j`. All non-empty. FAIL otherwise.++(9) **Provenance chain.** Run `$HPR lint --rule provenance -j`. FAIL if any issues.++(10) **Analyst coverage.** Run `$HPR lint --rule analyst-coverage -j`. FAIL if below 30%.++(11) **Opening frames the decision + forces.** Does the opening state what is being forecast, why it is uncertain, and what 3–5 forces will determine the outcome? FAIL if the opening is a history lesson or a definition.++(12) **Explicit "what would change my mind" statement.** Find a statement of what evidence would flip the forecast. FAIL if no such statement exists.++(13) **No academic API sweep was run.** Confirm via vault tags — if the corpus is dominated by `paper` content_type sources, the agent ran the wrong source strategy. FAIL (non-critical — correct the source mix and re-run).++Auditor output: list every violation with a one-line quote from the report.diff --git a/src/hyperresearch/skills/research-synthesize.md b/src/hyperresearch/skills/research-synthesize.mdnew file mode 100644index 0000000..0b97ad5--- /dev/null+++ b/src/hyperresearch/skills/research-synthesize.md@@ -0,0 +1,216 @@+---+name: research-synthesize+description: >+ Interpretive and explanatory research. The user wants to know what+ something MEANS, how it WORKS, or WHY it is the way it is. Primary virtue+ is interpretation density — every paragraph fuses fact with an analytical+ claim. Covers everything from literary/humanities interpretation to+ conceptual explanation of scientific mechanisms.+---++# Synthesize Protocol++> You are in **synthesize** modality. The dispatcher routed you here because the output needs a defended thesis or a mechanism explanation — not enumeration, not comparison, not prediction. The reader leaves with an argument they can be persuaded by or disagree with.++Read the dispatcher (`.claude/skills/hyperresearch/SKILL.md`) for the process. This file is the SUBSTANCE reference for synthesize.++---++## What synthesize is for++| Prompt shape | Why it's synthesize |+|---|---|+| "What does Blood Meridian's violence mean" | Thesis-driven interpretation |+| "Explain how attention mechanisms work in transformers" | Mechanism explanation |+| "Why did the Roman Republic collapse" | Causal / explanatory |+| "What is Wittgenstein's private language argument and why does it matter" | Interpretive + conceptual |+| "How does CRISPR-Cas9 edit DNA and what are its limitations" | Mechanism + critique |++Synthesize is NOT for: "list all X" (collect), "which X is best" (compare), "will X happen" (forecast).++**Two legitimate sub-shapes:**+1. **Interpretive synthesis** — the subject is a text, work, tradition, cultural phenomenon, or philosophical position. The thesis is an interpretation. Sources are primary texts + established critical tradition + dissenting readings.+2. **Mechanism synthesis** — the subject is a process, system, or phenomenon. The thesis is a mechanism explanation with WHY it works the way it does. Sources are peer-reviewed literature + authoritative technical references.++The dispatcher picks synthesize when the output's primary virtue is *understanding*, regardless of which sub-shape applies.++---++## Source strategy++### If a peer-reviewed literature exists — academic API sweep BEFORE web search++For mechanism synthesis, or interpretive synthesis where scholarly criticism is the dominant tradition: run the academic APIs first. Web search returns derivative commentary; academic APIs return citation-ranked canonical papers.++**Semantic Scholar** — citation count + forward/backward chaining:++```python+import httpx, json, urllib.parse, os, time+SS_KEY = os.environ.get("SS_API_KEY", "")+HEADERS = {"x-api-key": SS_KEY} if SS_KEY else {}+q = urllib.parse.quote("<topic>")+url = f"https://api.semanticscholar.org/graph/v1/paper/search?query={q}&fields=title,year,citationCount,externalIds&limit=10"+r = httpx.get(url, headers=HEADERS, timeout=15).json()+papers = sorted(r["data"], key=lambda p: p.get("citationCount", 0), reverse=True)+# Top 3 → backward chain (references) AND forward chain (citations)+for p in papers[:3]:+ pid = p["paperId"]+ refs = httpx.get(f"https://api.semanticscholar.org/graph/v1/paper/{pid}/references?fields=title,year,citationCount&limit=30", headers=HEADERS, timeout=15).json()+ cits = httpx.get(f"https://api.semanticscholar.org/graph/v1/paper/{pid}/citations?fields=title,year,citationCount,isInfluential&limit=50", headers=HEADERS, timeout=15).json()+ time.sleep(0.4)+```++Backward chaining finds the foundational canon; forward chaining finds everything built on it in the last 3 years.++**OpenAlex** for humanities, social science, medicine, or non-arXiv disciplines:++```+https://api.openalex.org/works?search=<topic>&sort=cited_by_count:desc&per-page=15&[email protected]+```++**arXiv** for cs / stat / q-bio / econ / math / physics:++```+https://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+all:<topic>&sortBy=relevance&max_results=25+```++**PubMed eutils** for biomedical / clinical:++```+https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=pubmed&term=<topic>&retmode=json&retmax=20+```++Feed PDF links directly to `$HPR fetch` — the fetcher handles PDF extraction automatically.++### If the subject has no peer-reviewed literature — start with primary sources + guided reading loop++For interpretive synthesis of pop-culture / contemporary / niche subjects where scholarly criticism is thin or absent, skip the academic sweep. The source strategy is:++**Ground truth:** the primary work itself (if available online; substantial excerpts if not), author/creator interviews stating intent, official materials.++**Institutional signal:** named critical essays, longform analyses, scholarly monographs, Wikipedia references harvest (curated for notability).++**Practitioner triangulation:** professional reviews, interviews with domain experts, conference keynote transcripts.++**Commentary (de-prioritized):** listicles and YouTube reviews — lead-generation only, not load-bearing.++Use the **guided reading loop** from the dispatcher as the default discovery mechanism. The critical tradition for interpretive subjects is discoverable only by reading — the essay names three critics, those critics cite a 1998 monograph, the monograph references a specific passage.++### Non-English sources when the subject has non-English reception++If the subject has major non-English reception (a franchise bigger in France than the US, a philosopher with denser German scholarship, a movement with foundational French literature), run searches in those languages:++```+WebSearch("<topic> análise crítica") # Portuguese+WebSearch("<topic> critique analyse") # French+WebSearch("<topic> analisi critica") # Italian+WebSearch("<topic> Kritik Rezeption") # German+WebSearch("<topic> 批評 論文") # Japanese+```++Non-English sources count toward voice diversity at Checkpoint 1.++### Adversarial round++Required. Find the strongest dissenting reading / dissenting mechanism explanation / dissenting theoretical position. Your thesis must engage it with substance, not a one-sentence dismissal.++```+WebSearch("<topic> criticism limitations")+WebSearch("against <dominant reading> <topic>")+WebSearch("<topic> reappraisal new reading")+```++### Source diversity — at least one non-academic voice++If the subject has practitioners (applied economists, engineers, policy analysts, industry users, operators), **at least one non-academic voice is mandatory.** A synthesize corpus built entirely of peer-reviewed papers plus lecture notes is a monoculture — the thesis loses its grip on reality when nobody in the corpus has actually *used* the thing.++Examples of required non-academic voices:+- "Is there a general method for solving first-price auctions?" — the academic corpus says "here are 5 methods"; an applied voice (FTC economist blog, procurement handbook, named practitioner postmortem) tells you whether any of them is actually used in the wild.+- "How does CRISPR edit DNA?" — the academic papers explain the mechanism; a biotech industry voice (Moderna process description, patent litigation commentary) tells you which off-target effects matter in production.+- "What does Blood Meridian mean?" — the scholarly tradition sets the interpretive axes; a named critic blog or longform essay brings the thesis into the present.++Search for these explicitly:++```+WebSearch("<topic> in practice applied industry")+WebSearch("<topic> practitioner guide used by")+WebSearch("<topic> does anyone actually use")+```++If no such voice exists (genuinely novel topic, purely theoretical subject), state that explicitly in the scaffold and explain why — otherwise Checkpoint 1 flags the corpus as under-diverse.++---++## Substance rules for Step 9 (writing)++These are the synthesize-specific rules that layer on top of the dispatcher's shared writing constraints.++### 1. Commit to a thesis++The draft is built around ONE defensible claim: what you are arguing about the subject. State it clearly — ideally in a single short sentence in the opening — then spend the draft defending it.++A thesis is NOT a summary ("Blood Meridian is about violence"). A thesis is an arguable position ("Blood Meridian's violence is structural, not expressive — it enacts a theological claim that human agency is a ventriloquism of geology"). A neutral reader could read your thesis and disagree.++### 2. Interpretation density — every paragraph fuses fact with claim++Every body paragraph must do both: report a fact / quote / source AND make an interpretive claim about what the fact MEANS within your thesis. A paragraph that only reports facts is a textbook excerpt, not synthesis.++**Bad:** "Transformers use a self-attention mechanism. The mechanism computes a weighted sum where weights come from query-key similarity."++**Good:** "Transformers use self-attention — but the substantive move is not the attention itself, it is what attention *replaces*. Recurrence imposes a temporal bottleneck: position N can only reach position N-1 through the sequence of hidden states. Self-attention obliterates this bottleneck and treats the sequence as a set, which is why transformers scale but LSTMs don't. The mechanism is a structural concession, not just an engineering trick."++Every paragraph earns its place by advancing the thesis or complicating it — not by restating source material.++### 3. Engage the strongest dissent++Find the reading / explanation / theoretical position that most directly contradicts your thesis. Give it substantive engagement — at minimum a full paragraph, ideally a full section. Represent it at its strongest, not strawmanned. Then explain why you reject it.++A thesis that never engages its opposite is advocacy, not argument.++### 4. Sources in tension at least twice++At least two body sections must put specific sources in direct tension: "Smith reads this passage as X; Jones reads it as Y. The textual evidence supports Z because..." This is the mark of synthesis — the reader sees you weighing evidence, not just citing it.++### 5. Quote the primary source directly++When the synthesis is interpretive, quote the primary work. Claim → direct quote → analysis of what the quote means. Paraphrase only when quotation is impossible or redundant.++When the synthesis is mechanism-explanation, quote the canonical paper's own description of the mechanism before you extend it.++### 6. Opening establishes the thesis AND the core tension++The opening answers: *what am I claiming, and why is this claim non-trivial*. If the thesis is obvious, the draft has no reason to exist. Name the tension: the paradox, the received view you're pushing against, the open problem, the tradeoff. That's the engine that drives the rest of the draft.++### 7. Section progression tracks the argument++Sections build on each other. Section 2's claim depends on section 1's setup; section 3 complicates section 2; the dissenting-view section is placed where the argument is strongest, so the reader feels the weight of the opposition before they see your counter. A shuffled set of thematic sections is not synthesis — it is a collage.++---++## Conformance checks (auditor reads this at Step 11, mode=conformance)++(0) **Verbatim prompt check.** Open `research/scaffold.md` — does its first section contain the user's verbatim prompt (unchanged, not paraphrased)? FAIL if missing or modified.++(1) **Thesis exists and is arguable.** Read the opening. Is there a clear thesis sentence? Is it a position a neutral reader could disagree with (not a neutral summary)? FAIL if the draft never commits to a claim.++(2) **Interpretation density.** Sample 5 random body paragraphs. For each: does it fuse a factual / source claim with an interpretive claim about what the fact means? FAIL any paragraph that is pure description or pure source-citation without an analytical move.++(3) **Dissenting view engaged substantively.** Find the section that engages the strongest opposing reading / explanation. Is the opposing position given a substantive paragraph (or more)? Is it represented at its strongest? FAIL if dissent is dismissed in one sentence or strawmanned.++(4) **Sources in tension at least twice.** Count body sections where specific sources are put in direct tension. FAIL if fewer than two.++(5) **Primary sources cited directly.** For interpretive synthesis: are there at least 3 direct quotations from the primary work? For mechanism synthesis: is the canonical source's own description of the mechanism quoted before the explanation extends it? FAIL if the draft only paraphrases.++(6) **Section progression.** Can sections be shuffled without loss? If yes, the draft is a collage, not an argument. FAIL.++(7) **Scaffold + comparisons + extract artifacts exist.** Run `$HPR search 'scaffold' -j`, `$HPR search 'comparisons' -j`, `$HPR note list --tag extract -j`. All must return non-empty. FAIL otherwise.++(8) **Provenance chain.** Run `$HPR lint --rule provenance -j`. FAIL if any issues.++(9) **Analyst coverage.** Run `$HPR lint --rule analyst-coverage -j`. FAIL if below 30%.++(10) **Tier weighting.** Are substantive claims anchored in `ground_truth` or `institutional` sources? Are load-bearing claims ever supported only by `commentary`? FAIL if any load-bearing claim rests solely on derivative commentary.++(11) **Core tension present in opening.** Does the opening state what makes the thesis non-trivial (the paradox, received view being pushed against, open problem)? FAIL if the opening is a neutral setup.++Auditor output: list every violation with a one-line quote from the report.diff --git a/src/hyperresearch/skills/research.md b/src/hyperresearch/skills/research.mdindex 22129d4..5c89da5 100644--- a/src/hyperresearch/skills/research.md+++ b/src/hyperresearch/skills/research.md@@ -1,526 +1,715 @@---name: researchdescription: >- Deep research on any topic. Searches the web, fetches sources with a headless browser,- follows links to primary sources, builds a searchable knowledge base, and synthesizes- findings. Use when the user asks to research, investigate, or learn about a topic.+ Deep research on any topic. Classifies requests by cognitive activity+ (collect / synthesize / compare / forecast), not subject matter. Produces+ a sourced, adversarially-audited report with full data-flow provenance.Trigger: /research----# Deep Research Skill+# Research Dispatcher-## Step 0: Clarify the request+The research protocol has ONE process spine (this file) and FOUR modality files that encode substance differences. Read this file end-to-end. Then read the modality file the classifier routes you to. Both are required.-Before doing ANY research, evaluate the user's request. If it is vague, ambiguous, or could go in multiple directions, **ask clarifying questions first**. Do not guess — ask. Examples:+**There are no numeric targets in this protocol.** No word counts, no H2 counts, no words-per-section floors. A draft is as long as the substance demands and as short as the structure allows. The user's own scope cues (if any) are the only length guidance that applies.-- "Research AI models" → Ask: "Which aspect? Training techniques, specific architectures, benchmarks, a particular company's models, deployment strategies?"-- "Find out about this company" → Ask: "What specifically? Financials, leadership, products, tech stack, culture, recent news?"-- "Look into transformers" → Ask: "The neural network architecture, or a specific variant? Are you interested in the theory, practical implementation, or recent alternatives like Mamba?"+**The user's verbatim prompt is the only gospel in this session.** It gets copied into the scaffold at Step 7 and every subsequent step re-reads it. Whenever you are unsure what to do, re-read the prompt. Whenever the draft feels like it's drifting, re-read the prompt. The auditor at Step 11 grades the draft against the verbatim prompt — not against any abstract notion of quality.-Ask 1-3 focused questions. Once you understand the scope, proceed. If the request is already specific and clear, skip this step and start researching immediately.--## Step 0.5: Classify the request and pick a protocol (MANDATORY)+----Different research requests need fundamentally different strategies. Source counts, primary/secondary mix, section structure, target length, analytical mode — all change based on the request shape. A single universal protocol is wrong. **Before Step 1, classify the request into one of the 7 types below and state your classification in writing** (in working memory or `/tmp/scaffold.md` — NOT a hyperresearch note).+## The 4 activities-### The 7 request types+Research requests are classified by what the output needs to DO, not what the subject IS.-| Type | Name | Use when |-|---|---|---|-| 1 | **Canonical Knowledge Retrieval** | Mature field with textbook chapters or canonical survey papers; a domain expert would start by citing a classic source. |-| 2 | **Market / Landscape Mapping** | Enumerable competitive analysis — list of companies, products, vendors with attributes. |-| 3 | **Engineering / Technical How-To** | User is trying to DO something. Ideal answer is a procedure or method selection. |-| 4 | **Interpretive / Humanities Analysis** | Subject is a text, work, tradition, or cultural phenomenon. Good answer takes a position and defends it. |-| 5 | **Comparative Evaluation** | Prompt names multiple alternatives by name and expects a matrix + pick. |-| 6 | **Emerging / Cutting-Edge Research** | Most useful material is <2 years old; best sources are arXiv/bioRxiv/conference papers. |-| 7 | **Forecast / Strategy / Recommendation** | Asks what will happen or what should be done; ideal answer commits to a prediction or prescription. |-| 0 | **General Research (fallback)** | Doesn't cleanly fit one type, or genuinely spans multiple. |+| Activity | Question it answers | Primary virtue | Modality file |+|---|---|---|---|+| **collect** | "What exists / what happened / who did what" | Exhaustive enumerative coverage with per-entity fields | `.claude/skills/hyperresearch/SKILL-collect.md` |+| **synthesize** | "What does it mean / how does it work / why" | Interpretive density — a defended thesis grounded in evidence | `.claude/skills/hyperresearch/SKILL-synthesize.md` |+| **compare** | "Which is better for what / how do they differ" | Proportionate per-entity depth + a committed recommendation | `.claude/skills/hyperresearch/SKILL-compare.md` |+| **forecast** | "What will happen / what should we do" | Committed prediction from past + current state with explicit time horizon | `.claude/skills/hyperresearch/SKILL-forecast.md` |-Overlap is expected. Classify a **primary** type and, if applicable, a **secondary** type. Apply the parameter block for the primary; borrow special rules from the secondary.+**Subject matter does not decide the classification.** A query about a fictional franchise can be collect (per-character enumeration), synthesize (a thesis about what the franchise means), compare (this franchise vs another), or forecast (will this franchise's new arc succeed). A query about distributed databases can be any of the four. The classifier reads what the output should *do*, not what the subject *is*.-### State your classification+**A single prompt can blend activities** — that is normal. Pick ONE primary and optionally ONE secondary flavor. The primary determines the process and the structural backbone; the secondary informs the substance rules at Step 9. Example: "Analyze Saint Seiya's armor classes: for each significant character, cover techniques, story arcs, fate — and develop a thematic argument about what the armor means" is **collect** (primary — the per-character enumeration is the backbone) + **synthesize** (secondary — thematic beats layered into each entity section).-Before Step 1, write this down:+----1. **Primary type**: #N — one-sentence justification tied to the prompt.-2. **Secondary type**: #N or "none".-3. **Applied parameters** (5 lines): source strategy, target length, opening-section shape, target H2 count, analytical mode.+## Step 0: Clarify if vague-Refer back to this throughout the workflow. It's how you avoid drifting into one-size-fits-all defaults.+If the request is vague or ambiguous, ask 1–3 focused questions before proceeding. Don't guess.----## Request-type parameter blocks--Pick the block for your primary type and apply it. These **override** the generic defaults in Step 3 (source count) and Step 9 (writing constraints).--### Type 1 — Canonical Knowledge Retrieval--- **Source strategy:** 5–15 primary (seminal papers, textbooks, canonical surveys) + 5–15 secondary (recent reviews, textbook chapters). Primary-heavy. Read primary DEEP.-- **Target length:** 6,000–10,000 words-- **Opening:** historical / conceptual progression — how the field arrived at current consensus-- **H2 count:** 10–15, sequenced by dependency (concept → result → consequence)-- **Analytical mode:** explain WHY canonical results hold; compare competing formulations; signal which textbook to start with-- **Special rule:** sequence content by dependency, not by source. A reader should need sections 1 and 2 to understand section 3.--### Type 2 — Market / Landscape Mapping--- **Source strategy:** 20–50 secondary (analyst reports, press, official docs) + 5–10 primary (filings, product specs, API docs). Secondary-heavy — triangulation across many descriptions is correct here.-- **Target length:** 5,000–9,000 words-- **Opening:** market definition + segmentation + the framework used to score vendors-- **H2 count:** 8–14, one per vendor *cluster* or category — NEVER one per individual vendor-- **Analytical mode:** take a position on who is winning and why-- **Special rule:** at least one comparison matrix in the body, with common axes, is MANDATORY. Never list vendors in isolation.-- **Special rule:** if the user asked for rankings, deliver them with confidence. Rankings without commitment are worthless.--### Type 3 — Engineering / Technical How-To--- **Source strategy:** 5–15 primary (papers, docs, GitHub repos, RFCs, standards) + 5–15 secondary (blog posts, tutorials, StackOverflow). Roughly balanced.-- **Target length:** 4,000–8,000 words-- **Opening:** the problem statement + the constraint landscape — what makes this hard, why naive approaches fail-- **H2 count:** 8–14-- **Analytical mode:** pick a recommended approach; explain when to use alternatives-- **Special rule:** if a method has a reference implementation, link it and characterize it-- **Special rule:** include a decision tree or selection matrix ("use X when..., use Y when...")--### Type 4 — Interpretive / Humanities Analysis--- **Source strategy:** 3–10 primary (the work itself, key scholarly monographs, canonical commentary) + 10–25 secondary (critical essays, interviews, reviews). Primary-heavy for depth.-- **Target length:** 7,000–11,000 words-- **Opening:** thesis + the interpretive tradition this work sits in-- **H2 count:** **6–10** (fewer, LONGER sections — 800–1500 words each). Sections are THEMATIC, not entity-cataloged.-- **Analytical mode:** take and defend an interpretation; engage dissenting readings-- **Special rule:** NEVER one section per character / chapter / author. Sections must be thematic (e.g. "Sacrifice as cosmology" — not "Pegasus Seiya").-- **Special rule:** at least 2 body sections must put sources in tension with each other — find where scholars disagree and walk through it.-- **Special rule:** this is the type where catalog-style reports fail hardest. Before submitting, double-check that no section is <400 words or entity-named.--### Type 5 — Comparative Evaluation--- **Source strategy:** 3–5 primary per comparand + 2–5 secondary per comparand (so N×5 to N×10 total). Depth scales with N.-- **Target length:** 5,000–10,000 words-- **Opening:** the comparison framework itself — the dimensions being scored and why they matter-- **H2 count:** typically 1 per comparand + 2–4 synthesis sections (~8–12 total)-- **Analytical mode:** explicit scoring on each dimension + a recommended pick with tradeoffs-- **Special rule:** one comparison matrix in the body is MANDATORY (not optional)-- **Special rule:** equal depth across comparands — if one section is 400 words and another is 1200, rebalance--### Type 6 — Emerging / Cutting-Edge Research--- **Source strategy:** 5–15 primary (preprints, conference papers, authoritative reviews if any) + 3–8 secondary (news, blog posts from authors). Primary-dominant.-- **Target length:** 5,000–9,000 words-- **Opening:** the open problem + why prior approaches fell short-- **H2 count:** 8–12-- **Analytical mode:** honest about disagreement; forecast 2–5 years out with caveats-- **Special rule:** if a claim is from a 2024–2026 preprint, say so. Don't assume consensus in fast-moving fields.-- **Special rule:** include an explicit "What we don't know yet" section-- **Special rule:** do NOT pad with mature-field material to hit a source count. Fewer primary sources deeply engaged beats more sources shallowly skimmed.--### Type 7 — Forecast / Strategy / Recommendation--- **Source strategy:** 15–30 secondary (analyst reports, think-tanks, news) + 5–10 primary (official announcements, filings, datasets). Secondary-heavy.-- **Target length:** 6,000–10,000 words-- **Opening:** the decision / question being answered + the forces shaping it-- **H2 count:** 8–12-- **Analytical mode:** confident prescription with an explicit reasoning chain. Probability language over hedging.-- **Special rule:** at least one body section must commit to a direction — not everything can be deferred to the Opinionated Synthesis at the end.-- **Special rule:** acknowledge the time horizon explicitly — what's true in 6 months vs. 5 years.--### Type 0 — General Research (fallback)--Use the generic defaults: 15–25 / 30–50 / 50–80 source floors (see Step 3), framework opening, 12–20 H2 headings, 400–600 words per H2 (see Step 9).+## Step 0.5: Check existing vault coverage first++```bash+$HPR search "<topic>" -j+```++If the vault already has >10 relevant notes: tell the user, summarize what's there, and ask whether they want deeper research, a specific angle, or synthesis from existing notes. Don't re-fetch what's already curated.----When invoked, follow these steps **in order**. Do not skip steps. The CLI path will be provided by the CLAUDE.md — look for the "CLI path:" line in the Research Base section.+## Step 1: Classify + commit the verbatim prompt-If you can't find the CLI path, check for the hyperresearch executable:-```bash-which hyperresearch 2>/dev/null || where hyperresearch 2>/dev/null || find . -path "*/.venv/*/hyperresearch*" -type f 2>/dev/null | head -1+State your classification in writing BEFORE doing anything else. Copy the user's prompt verbatim into your working memory. This is the format you MUST output:++```markdown+## User Prompt (VERBATIM — gospel)+> [paste the entire user message here, character-for-character. Do not+> paraphrase. Do not summarize. Do not reformat. This text is the only+> authoritative source for what the output should contain and look like.]++## Primary activity+<collect | synthesize | compare | forecast>++## Secondary flavor (optional)+<collect | synthesize | compare | forecast | none>++## One-sentence justification+<why this primary activity matches what the output needs to DO>++## What the user explicitly asked for (extracted from the verbatim prompt)+- **Explicit deliverables:** <list every specific ask — "answer A, B, C", "include a recommendation", "for each X give Y" — or "none">+- **Explicit entities or items:** <every item named in the prompt, or "none">+- **Explicit per-item fields:** <every field the prompt demands per entity — "power level / technique / fate" — or "none">+- **Explicit structure:** <"one section per", "ranked list", "executive summary", etc., or "silent">+- **Explicit format:** <prose / bullets / table / code / FAQ, or "silent">+- **Explicit scope cues:** <"brief", "detailed", "deep dive", specific lengths, or "silent">+- **Explicit ordering:** <chronological / by importance / custom, or "silent">++## Core tension+<what makes this question non-trivial — a paradox, a disagreement, a tradeoff,+a constraint interaction, an open problem. One-to-three sentences. This is+what the draft must earn its way through.>```-Store it in a variable: `HPR="<path>"` for the rest of this workflow.-## Step 1: Check existing knowledge+Then **read the corresponding modality file with the Read tool.** The modality file tells you HOW to do the activity-specific parts (source strategy, substance rules, conformance checks). This dispatcher tells you WHAT steps to run and in what order.-Before searching the web, check what's already in the research base:+----```bash-$HPR search "<topic>" --include-body -j-$HPR sources list -j-```+## Tiebreaker questions-If there's already substantial coverage, tell the user what's known and ask if they want deeper research or a specific angle.+When two activities seem equally applicable:-## Step 2: Search broadly+**collect vs compare** — Does the user want coverage ("tell me everything") or evaluation ("which is best")? Recommendation expected or entities set against each other → **compare**. Coverage is the point and no recommendation is requested → **collect**.-Run **multiple web searches** with different phrasings. Don't stop at one query:+**collect vs synthesize** — Does the output need a defended thesis or just complete coverage? If the user explicitly asks for interpretation, meaning, or analysis alongside enumeration, the primary is usually **collect** (the enumeration is the dominant ask) and the secondary flavor is **synthesize**. Pure enumeration without thesis → **collect**. Pure thesis without per-entity coverage → **synthesize**.-```-WebSearch("<topic>")-WebSearch("<topic> research paper 2025 2026")-WebSearch("<topic> deep dive technical")-WebSearch("<related angle or subtopic>")-```+**synthesize vs forecast** — Is the claim about what IS (how it works, what it means) or what WILL BE (prediction, recommendation)? Past/present → **synthesize**. Future → **forecast**.-Collect ALL promising URLs — aim for 10-20 sources minimum. More is better. You'll prune later.+**compare vs forecast** — Is the user asking which option is best NOW or which will be best LATER? Now → **compare**. Later → **forecast** (often with compare flavor).-## Step 3: Fetch everything+**Any activity when the prompt contains "for each X, provide Y / Z / W"** — The explicit per-entity field list is a collection contract. Route to **collect** as the primary regardless of subject, with any interpretive / comparative / predictive framing as secondary flavor. The per-entity contract takes precedence because it is the most specific kind of user mandate.-Spawn `hyperresearch-fetcher` subagents to fetch URLs in parallel. They run on Haiku (cheap).+----```-Spawn hyperresearch-fetcher agents for each URL batch:-- Batch 1: [url1, url2, url3, url4, url5]-- Batch 2: [url6, url7, url8, url9, url10]-```+## Prompt fidelity — the gospel rule-Each fetcher will report back with note IDs and interesting links found in the content.+**Re-read the user's verbatim prompt at every checkpoint.** The prompt is inside the scaffold (Step 7). Every checkpoint re-opens the scaffold. Every time you re-open the scaffold, re-read the prompt first. Every structural decision must trace back to the prompt or a modality default that the prompt is silent on.-**Over-collect, then engage deeply.** It's better to have too many sources than too few — but collection is a means to an argument, not the goal. A report built from 30 sources that disagree and force you to take positions beats a report built from 80 sources that each contribute one bullet of description. Read each source carefully. If you find yourself translating one source into one paragraph or one section, stop — you are building a catalog.+- If the prompt specifies structure / format / entities / fields / ordering / scope: **the user's request is authoritative**. Modality defaults yield.+- If the prompt is silent on any of those: **the modality default applies** to what the prompt left open.+- The modality file is authoritative for SUBSTANCE (interpretation density, proportionate depth, probability language, mechanism clarity). Substance rules hold regardless of shape.-**There is no time limit.** Do multiple rounds of fetching — as new subtopics and questions reveal themselves, launch new rounds of fetcher subagents in parallel to chase them down. The first round uncovers the landscape, the second digs into what you found, the third fills gaps. Keep going until you've exhausted the topic. Do not rush.+**Never negotiate the user's contract.** If the user asked for 108 entities and 4 fields each, you owe 108 × 4 data points. You do NOT get to substitute "representative examples + thematic commentary". You do NOT get to silently downgrade "each" to "the most important". If the user's contract seems impossible to satisfy within a reasonable draft, surface that tension to the user and ask — do not paper over it by producing less than what was asked.-**CHECKPOINT: Before writing any draft, stop and take stock:**-```bash-$HPR note list -j-```-Review what you have. Don't chase a number — chase completeness:-- What angles or subtopics are NOT yet covered?-- Which notes reference sources you haven't fetched?-- Are there counterarguments or alternative viewpoints missing?-- Do you have primary sources or just secondhand summaries?+----If there's more to get, go get it. **Source count is a function of request type, not topic complexity** — use the parameter block from Step 0.5 (your classified primary type) as your primary guidance. Fall back to the universal floors below ONLY if you classified as Type 0 General Research:-- Simple factual question: ~15–25 sources-- Complex or technical topic: ~30–50 sources-- PhD-level research question: ~50–80 sources+## Process discipline (applies to every activity)-**Primary-heavy vs. secondary-heavy is the axis that matters most — getting this wrong is a worse failure than getting the count wrong.** "Primary" means the claim comes from first-hand: an original paper, an official document, a dataset, a product spec. "Secondary" means any description or discussion of a primary source. Types 1, 4, 5, 6 are primary-heavy — cite originals, engage with them deeply, prune irrelevant secondary coverage. Types 2, 7 are secondary-heavy — triangulate across many descriptions because no single secondary source is authoritative. Type 3 is balanced.+This protocol is **process-driven**. Each required step produces an **artifact** that a later step reads. Skipping a step does not save work — it breaks the dependency chain silently, and the later step operates on missing-but-pretended input. The draft then gets written without the tension analysis; the audit never fires; the report ships un-stress-tested.-Stop fetching when you can't find anything new worth adding, not when you hit a number. And be honest about the opposite failure: if you're still fetching past the type's range without a clear gap in mind, you're procrastinating on the writing.+**Definition of task failure:** if a required artifact does not exist on disk or in the vault at the checkpoint where the next step expects it, the task has FAILED at that checkpoint. Failure is not fatal — it means: STOP the current step, return to the step that produces the missing artifact, complete it, and re-run the checkpoint. You may NOT compensate for a missing artifact by doing extra work inside the step that follows it.-## Step 4: Go down rabbit holes+Read this section before doing anything else, then follow it literally. You will self-review four times in a single research session. If any review fails, you go back.-Read the fetched notes and follow links to deeper material:+### Required TODO list (build this immediately after Step 1 classification)-```bash-$HPR note show <note-id> -j-```+Before executing Step 2, create this exact TODO list with the TodoWrite tool. Do NOT fetch anything, search anything, or write anything until the list exists:-For each note, look for:-- **Papers cited** → fetch them (PDFs are fully supported — fetch directly)-- **Documentation linked** → fetch it-- **GitHub repos mentioned** → fetch the README-- **Related topics referenced** → search and fetch-- **People/companies mentioned** → fetch their pages-- **Raw data, CSVs, datasets linked** → fetch and analyze them-- **Statistics without primary sources** → search for and fetch the original data+1. Check existing vault coverage (Step 0.5)+2. Discover sources (Step 2)+3. Fetch source batch or open the guided reading loop (Step 3)+4. **Checkpoint 1: post-fetch review**+5. Rabbit-hole pass with `--suggested-by` provenance (Step 4)+6. Curate every source: tier + content_type + summary + analyst extract (Steps 5–6)+7. Run link / lint / graph hubs (Steps 5–6)+8. **Checkpoint 2: post-curate review**+9. Write scaffold note WITH verbatim user prompt as first section (Step 7)+10. Write comparisons note (Step 8)+11. **Checkpoint 3: pre-draft gate — CRITICAL**+12. Write `research/notes/final_report.md` (Step 9)+13. Gap analysis + adversarial search (Step 10)+14. Adversarial audit via `hyperresearch-auditor` — both modes in parallel (Step 11)+15. **Checkpoint 4: post-audit review**+16. Write Opinionated Synthesis section (Step 12)+17. Save synthesis note + health checks (Steps 13–14)-**PDFs work natively.** arXiv papers, NBER working papers, conference proceedings — fetch them directly. The tool auto-detects PDFs and extracts full text. Don't skip sources because they're PDFs.+Mark each item `completed` only after its artifact exists AND you have verified it exists with a command. You may NOT mark item 12 (the draft) `in_progress` until items 1–11 are all `completed`. The TODO list is the process ledger.-**Use scholarly APIs for academic topics.** When the research is academic, use APIs to find papers:-- arXiv API: `https://export.arxiv.org/api/query?search_query=...`-- Semantic Scholar API: `https://api.semanticscholar.org/graph/v1/paper/search?query=...`-- CrossRef API: `https://api.crossref.org/works?query=...`+----Write code to call these, get structured paper data (citations, related work, abstracts), then fetch the actual PDFs.+## Zettelkasten conventions (apply in every activity)-Follow links deeply — don't stop at the first page. When a source links to more detailed material, primary research, or raw data, follow those links and fetch them too.+All notes follow atomic Zettelkasten conventions. The CLI manages the index.-**Run analysis when needed.** If you find raw data (tables, CSVs, statistics), write and run code to compute real figures, verify claims, calculate trends, and produce original analysis. Concrete numbers from actual data beat vague summaries.+**Note `type:` field:** `source` | `synthesis` | `moc` | `scaffold` | `comparison`-Spawn more fetcher agents for these secondary sources.+**Status lifecycle:** `draft` → `review` → `evergreen` → `stale` → `deprecated` → `archive`-**Do NOT stop collecting until you are certain you have enough breadth.** After each round of fetching, ask yourself: "Are there major angles, subtopics, or perspectives I haven't covered yet?" If yes, search and fetch more. If a topic has 5 facets, you need sources on all 5 — not just the first 2 you found. Keep going until you can confidently say the collection covers the full scope of the topic.+**Every source note must have:** `title`, `source` (URL), at least one `tag`, `summary` (≤120 chars), `status`, `tier`, `content_type`.-## Step 4.5: Read notes efficiently — triage by frontmatter (MANDATORY)+**Epistemic metadata (required on every source note before Checkpoint 2):**-When inspecting notes in the research base, NEVER blindly `note show` every note. Bodies can be 15,000+ words; frontmatter summaries are usually under 40. **Triage first, body-read only when earned.** This protocol applies throughout the rest of the workflow — use it in Step 4 (rabbit holes), Step 8 (cross-source comparison), and Step 9 (draft writing).+- **`tier`** — what epistemic role this source plays:+ - `ground_truth` — the artifact itself: official filings, product specs, datasets, primary texts, peer-reviewed papers with original data, regulatory text+ - `institutional` — authoritative synthesizers: analyst reports, think-tanks, review/survey papers, textbooks, canonical criticism+ - `practitioner` — how domain participants actually experience it: user reviews, community threads, practitioner blogs, postmortems, migration guides+ - `commentary` — discussion without new signal: news summaries, opinion pieces, derivative explainers+ - `unknown` — unclassified (default at fetch time; must be replaced during curation)+- **`content_type`** — what kind of artifact this is: `paper` | `docs` | `article` | `blog` | `forum` | `dataset` | `policy` | `code` | `book` | `transcript` | `review` | `unknown`-### Level 1 — Frontmatter sweep (cheap, always start here)+Tier and content_type are orthogonal: a `paper` can be `ground_truth` (original data) or `institutional` (a review). A `blog` can be `practitioner` (named engineer's postmortem) or `commentary` (derivative explainer).-```bash-$HPR note list -j-```+**Tagging:** Check existing tags first (`$HPR tags -j`). Reuse existing tags. Apply multiple relevant tags per note. Do not invent synonyms.-Returns one object per note with `id`, `title`, `tags`, `summary`, `word_count`, `status` — **no bodies**. Scan this to build a relevance shortlist. For many claims the summary field is enough — a well-curated note's summary already tells you what the source claims, and you cite the source URL directly without re-reading the body.+**Wiki-links:** Connect related notes with `[[note-id]]`. Every note should link to at least one sibling.-### Level 2 — Frontmatter-only show (targeted)+----For a single note whose summary hints at deeper value, pull its full frontmatter without the body:+## Step 2: Source discovery++The modality file tells you the source strategy (what APIs to hit, what source types to prioritize, whether to run the academic sweep). Run it.++Then, regardless of modality, run an **adversarial search round** — searches that specifically look for the failure case, the dissenting view, the critique, the contrarian, the known limitations. Every activity needs this:-```bash-$HPR note show <id> --meta -j```+WebSearch("<topic> criticism limitations problems")+WebSearch("<topic> against the grain dissenting view")+WebSearch("why <dominant view> is wrong")+```++A corpus without dissent is advocacy, not research. If the adversarial round returns nothing substantive, note that explicitly — it is itself a finding.-Adds `source`, `parent`, `created`, `updated`, `raw_file` (if the note has a PDF). Still cheap. Decide from here whether the body is worth pulling.+----### Level 3 — Inline body read (only for small notes)+## Step 3: Fetch — guided reading loop (preferred) or batch-```bash-$HPR note show <id> -j+**Default: the guided reading loop** (described below). Seed with 3–5 high-signal URLs, then let each iteration's analyst subagents propose next targets based on what the prior sources cite.++**Batch fetch** only when the target set is already known and complete — e.g., the user handed you 10 URLs, or the modality file says to batch (compare activity with explicitly-named entities is the usual case).++### Guided reading loop protocol++The loop is a bounce between fetching and reading:++```+fetch 3-5 seeds → hyperresearch-analyst reads each with goal in hand →+ analyst returns extract + "next_targets" (URLs, queries, names, claims to verify) →+ main agent fetches next_targets with --suggested-by flag →+ hyperresearch-analyst reads those → ...```-Threshold: `word_count < 2000` (~2700 tokens). Small enough to read directly. Batch several at once: `note show <id1> <id2> <id3> -j`.+Each iteration is a research director: the analyst reads one source, proposes what to read next, and hands the decision back to the main agent. This discovers source trees you couldn't find by keyword search alone — the essay names three critics, those critics cite a 1998 monograph, the monograph references a specific passage in the primary work.-### Level 4 — Targeted search with body, token-capped+**Phase 1 — Seed fetch.** Pick the 3–5 highest-signal URLs:+- 1–2 canonical reference entries (Wikipedia, field survey, official docs)+- 1–2 analytical/critical sources visible in search results+- 0–1 adversarial/dissenting source-When you need content from multiple notes at once:+Spawn `hyperresearch-fetcher` on ONLY those 3–5. Do NOT batch-fetch 20 URLs upfront — half get wasted on sources the loop would tell you to skip.++**Phase 2 — Guided reading iterations.** For each fresh note, spawn the **`hyperresearch-analyst`** agent (Sonnet, registered at `.claude/agents/hyperresearch-analyst.md`) with `mode=guided`:-```bash-$HPR search "<precise question>" --include-body --max-tokens 6000 -j+```+research_goal: <the user's original verbatim prompt>+sub_goal: <what this specific source should contribute>+source_note_id: <the id of the note to read>+mode: guided+already_covered: <sub-topics prior iterations answered, or "none">+already_fetched_urls: <output of `$HPR sources list -j`>```-`--max-tokens` truncates intelligently and flags `"truncated": true` on the last fitting result. Prefer this over N separate `note show` calls.+Spawn multiple analysts in parallel — one per source in the current batch. Each returns: extract content + covered sub-topics + 2–5 next_targets + coverage status.-### Level 5 — Delegate heavy notes to a Sonnet subagent (MANDATORY for large notes)+**Phase 3 — Main agent orchestration.** After each iteration returns:-For any note with `word_count > 6000` (~8000 tokens, often PDFs), do NOT dump it into your own context. Spawn a Sonnet subagent with a pointed extraction prompt:+1. **Track the iteration in TodoWrite.** Add `Iteration N: fetch [targets] from [sources]`. After it completes, mark with the coverage delta ("added 4 sources, 2 sub-topics newly covered"). This makes stalls visible.+2. Collect every extract into a running knowledge accumulator.+3. Merge and dedupe next_targets across all analysts. If three subagents propose the same URL, fetch it once — pass all three suggesting source IDs via repeated `--suggested-by` flags.+4. Prioritize next_targets by: (a) which would most likely *change* the argument if they disagreed with current sources, (b) which sub-topics the corpus doesn't yet cover, (c) which tiers are missing.+5. **Fetch the top 3–5 next_targets WITH `--suggested-by` flag. This is mandatory:**-> Use a fresh Sonnet subagent:-> "Read `research/notes/<id>.md` (and its raw file at `research/raw/<id>.pdf` if the frontmatter has `raw_file` set). Extract ONLY what answers this specific question: '<your precise question>'. Return under 500 words, with direct quotes for any non-trivial claim and the source URL. If the note doesn't actually answer the question, say so and stop."+ ```bash+ $HPR fetch "<url>" \+ --tag <topic> \+ --suggested-by <source-note-id-1> \+ --suggested-by <source-note-id-2> \+ --suggested-by-reason "<the analyst's justification>" \+ -j+ ```-The subagent reads the full 20K+ tokens internally but returns only ~500 tokens to you — roughly a 40× context savings per large note, which compounds across a session.+ Without this flag the data-flow chain breaks — the fetched note has no link back to the source that found it, and you cannot trace how the research graph assembled itself.-### Level 6 — Raw PDF re-extraction (rare)+ **Graceful duplicate handling:** if the URL is already in the vault, fetch will NOT error. It appends the breadcrumb to the existing note and returns `{ok: true, duplicate: true, backlinks_added: N}`. Fetch every next_target; let fetch handle dedup.-If a note has `raw_file: raw/<id>.pdf` in its frontmatter AND the extracted markdown looks garbled or is missing content you need (specific figure, table, equation), read the raw PDF directly with the Read tool. Most of the time the extracted markdown is sufficient — only fall back here when it's clearly broken.+6. Refresh `already_fetched_urls` (`$HPR sources list -j`). Pass it to the next iteration.+7. Spawn `hyperresearch-analyst` on the new notes. Loop.-### Triage thresholds (guidance, not hard rules)+**Phase 4 — Termination.** Exit when ANY of these fires:+- **Coverage complete** — all analyst returns report `coverage: complete` for the sub-topics you need.+- **Cycle detected** — next_targets repeat or every proposed target's fetch returns `duplicate: true` with `backlinks_added: 0`.+- **Diminishing returns** — a full iteration adds zero new sub-topics. The TodoWrite delta tracker catches this.+- **Iteration cap: 5 full loops** — more than this and the loop is drifting, not converging. Stop and move on.-| word_count | Tier | Action |-|---|---|---|-| `< 2000` | Trivial | Read inline via Level 3. Batch with siblings. |-| `2000 – 6000` | Medium | Level 2 frontmatter first. Body only if the summary suggests high value. |-| `> 6000` | Heavy | Level 5 Sonnet delegation. Don't read inline. |+Then proceed to curation (Steps 5–6).-Override with reason (e.g. a single 8000-word paper is your most important primary source — read it directly, once). But defaulting to "just `note show` everything" is how sessions run out of context and reports turn shallow.+---++## Step 4: Rabbit holes — follow what sources point at-## Step 5: Auto-link and curate+Beyond the guided loop's automated next_targets, actively scan fresh notes for:-Run the auto-linker to connect related notes:+- **Cited datasets / primary data** → fetch the original, not the summary+- **Named scholars / practitioners / companies** → find their own writing+- **Referenced historical cases / precedents** → fetch the case itself+- **Interviews or quotes** → find the full transcript++Every manual rabbit-hole fetch MUST also pass `--suggested-by` naming the source that sent you there:```bash-$HPR link --auto -j+$HPR fetch "<url>" \+ --tag <topic> \+ --suggested-by <source-note-id> \+ --suggested-by-reason "<one-line reason>" \+ -j```-Check what tags exist and ensure consistency:+Claims carry more weight when the graph shows you traced them to primary sources.++---++## Step 4.5: Note triage (MANDATORY throughout)++Never blindly `note show` every note. Triage first.++| Level | Command | When |+|---|---|---|+| 1 — List | `$HPR note list -j` | Always start here. Summaries only, no bodies. |+| 2 — Meta | `$HPR note show <id> --meta -j` | When the summary hints at deeper value. |+| 3 — Inline | `$HPR note show <id> -j` | word_count < 2000. Batch small notes: `note show <id1> <id2> <id3> -j` |+| 4 — Search | `$HPR search "<q>" --include-body --max-tokens 6000 -j` | When you want content across multiple notes. |+| 5 — Subagent | Spawn `hyperresearch-analyst` (mode=extract) | word_count > 3000, OR whenever you need one specific answer from a big source. Never dump large notes inline. |++---++## Steps 5–6: Curate every source++Raw fetched notes arrive with `status=draft`, `tier=unknown`, auto-detected `content_type`, and no summary. Curation has two parallel jobs:++1. **Set metadata** — tier, content_type, summary, tags, status (you do this with `$HPR note update`)+2. **Extract relevant content** — spawn `hyperresearch-analyst` on the source so you have an extract note grounded in the research goal++**Both must happen for every fetched source.** The analyst extract is what gives you the substance to write deep prose about the source later; the metadata is what lets you query and filter the corpus during drafting.++### Step A: Spawn the analyst on every source++For research, the analyst is the default reading mechanism — not a fallback for big sources. Even small sources (1500 words) deserve analyst treatment because the analyst's URL-scan surfaces follow-up targets for the loop.-```bash-$HPR tags -j```+For each draft source note:+ Spawn hyperresearch-analyst with:+ research_goal: <the user's original verbatim prompt>+ sub_goal: "establish what this source contributes and what URLs / authors / claims warrant follow-up"+ source_note_id: <draft note id>+ mode: guided+ already_covered: <running list from prior analysts>+ already_fetched_urls: <output of `$HPR sources list -j`>+```++Spawn 3–5 in parallel (Sonnet, cheap). Each analyst reads the source, persists an extract note (`--add-tag extract --parent <source-id>`), and returns an extract + next_targets + coverage status.-Update any notes missing summaries (auto-curation handles most, but fix any gaps):+Collect every analyst's next_targets into TodoWrite. After each batch, fetch the top next_targets WITH `--suggested-by` so the loop iterates. **The fetch:extract ratio after curation should be at least 1:1** — every fetched source has a paired extract note. The `analyst-coverage` lint rule catches misses at Checkpoint 2.++### Step B: Set metadata from the analyst's findings++The analyst's return tells you the tier and content_type. After it returns:```bash-$HPR lint -j-$HPR repair -j+$HPR note update <source-id> \+ --tier <ground_truth|institutional|practitioner|commentary> \+ --content-type <paper|docs|article|blog|forum|dataset|policy|code|book|transcript|review> \+ --summary "<one-line description, can borrow from the analyst's extract>" \+ --add-tag <topic> --add-tag <subtopic> \+ --status review \+ -j```-## Step 6: Discover the knowledge graph+**Tier and content_type are PER-NOTE decisions.** Never apply one value to the whole batch. A fandom wiki is `docs`; a news article is `article`; a named-engineer blog post is `blog`; a reddit thread is `forum`; a GitHub README is `code`. An arxiv paper on theory is `institutional`; the same paper reporting experimental data is `ground_truth`; a conference keynote transcript is `commentary`. Read each analyst return and make the call per source.-Find the hub notes and key connections:+### Completion check```bash+$HPR lint --rule uncurated -j # zero uncurated+$HPR lint --rule provenance -j # zero provenance issues (--suggested-by chain exists)+$HPR lint --rule analyst-coverage -j # zero analyst-coverage issues+$HPR link --auto -j$HPR graph hubs -j-$HPR graph backlinks <most-linked-note-id> -j```-## Step 7: Build the conceptual scaffold (MANDATORY — before writing)+If any lint returns issues, curation is NOT done. Go back, run more analyst calls, fetch proposed next_targets with `--suggested-by`, and re-run the lints before Checkpoint 2.-This is the single biggest lever for report quality. Reports that look like references are built from a scaffold; reports that look like Wikipedia dumps are assembled by translating notes into sections. Before a single paragraph of prose, answer these four questions.+----**Where to put the scaffold:** keep it in your working memory, or dump it to a temp file like `/tmp/scaffold.md`. **DO NOT save it as a hyperresearch note.** The research base is for durable source material, not ephemeral pre-writing analysis. Notes are forever; scaffolds are for the next hour of writing.+## Step 7: Build the scaffold — with the verbatim user prompt as the first section-1. **What is the hard question?** One sentence — the underlying difficulty, not the surface query.-2. **What is the naive answer?** What a reader would guess before reading any sources. This is your foil — the thing the report has to explain away.-3. **What is the structural tension?** What makes this topic worth 10,000 words? The paradox, the constraint, the thing sources disagree on, the unsettled question.-4. **What is the progression?** Sketch 6–12 H2 section headings in **dependency order**. Each section must build on the previous one. If the headings could be shuffled, you have a catalog, not a report.+**Required artifact:** a note tagged `scaffold` whose FIRST section is the user's verbatim prompt.+**Verification:** `$HPR note list --tag scaffold -j` must return ≥1 entry after this step AND the body of that note must contain the verbatim user prompt.+**On failure:** the draft has no anchor. Return to Step 7 and write the scaffold.-The final report's **opening section MUST be a framework section**, not a definition section. Section 1 establishes the hard question and the structural tension. Not "X is a Y that does Z."+Using the **Write tool**, create `research/scaffold.md` with this exact structure:-## Step 8: Cross-source comparison (MANDATORY)+```markdown+## User Prompt (VERBATIM — gospel)+> [the user's entire message, copied character-for-character from the prompt.+> This is the authoritative source. Every subsequent step re-reads this+> section. Do not paraphrase. Do not truncate. Do not reformat.]++## What the user explicitly asked for+- **Explicit deliverables:** <list — or "none">+- **Explicit entities or items:** <list — or "none">+- **Explicit per-item fields:** <list — or "none">+- **Explicit structure:** <or "silent">+- **Explicit format:** <or "silent">+- **Explicit scope cues:** <or "silent">+- **Explicit ordering:** <or "silent">++## Primary activity and secondary flavor+- **Primary:** <collect | synthesize | compare | forecast>+- **Secondary (optional):** <or "none">+- **Why:** <one sentence>++## Core tension+<what makes this question non-trivial — the paradox, disagreement, tradeoff,+or open problem the draft has to earn its way through>++## The structural plan+<Ordered list of sections the draft will produce. Honor every explicit+structural mandate from "What the user explicitly asked for". Where the+prompt is silent, apply the primary modality's default structural guidance.+For each section, name it AND describe what it will contain in one line.>++## Where each source will land+<One line per curated source-id: which heading(s) it serves and in what role+(ground-truth evidence, institutional synthesis, dissenting voice, etc.).>++## Coverage checklist (the auditor will verify this)+<One line per explicit contract from "What the user explicitly asked for".+The auditor at Step 11 reads this checklist and verifies each item is+ticked off in the draft. If the user said "for each of 108 Specters provide+techniques and fate", there must be a coverage-checklist line that will be+checked against the draft. Do not write vague checks.>+```-Sources earn their citations by being compared against each other, not by being listed in parallel. Find 3–5 places where your sources actually disagree, emphasize different things, or frame the same problem differently.+Then run:```bash-$HPR search "<topic>" --include-body -j+$HPR note new "Scaffold: <topic>" --tag scaffold --status draft \+ --body-file research/scaffold.md --summary "Scaffold for <topic>" -j+```++**The verbatim user prompt goes FIRST.** This is non-negotiable. Every checkpoint re-opens the scaffold, and re-opening the scaffold means re-reading the prompt. The prompt lives inside the document so it cannot drift out of context.++---++## Step 8: Cross-source comparisons++**Required artifact:** a note tagged `comparison` containing 3–5 explicit source-vs-source disagreements with your position on each.+**Verification:** `$HPR note list --tag comparison -j` must return ≥1 entry after this step.+**On failure:** the thesis / recommendation / forecast has no stress test. "I did it in my head" is not acceptable.++Find 3–5 places where sources disagree. Read both at Level 3+ before comparing.++Using the **Write tool**, create `research/comparisons.md`:++```markdown+## Disagreement 1: <what axis are they disagreeing about>+- Source A ([URL]): <claim/reading> because <reasoning/evidence>+- Source B ([URL]): <different claim> because <different reasoning>+- Why they differ: <theoretical framework? different data? different time window? different scope?>+- My position: <which is more credible and why — take a side, with the evidence that tips the balance>```-For each disagreement, capture a short comparison block in the same scratch file as the scaffold from Step 7 — **NOT as a hyperresearch note**. These are ephemeral analysis artifacts that will end up as prose inside the final report, not durable KB entries:-- what each source says, with URLs-- why they differ (data, methodology, assumptions, era, scope)-- your position on which is more credible, with reasoning+Then run:-These comparisons become the backbone of body sections. If every source is corroborating every other source, you didn't need many sources — search again for contrarian views, criticism, or competing frameworks.+```bash+$HPR note new "Comparisons: <topic>" --tag comparison --status draft \+ --body-file research/comparisons.md --summary "Source comparison for <topic>" -j+```++---## Step 9: Write the draft-**First, pull up your classification from Step 0.5.** The parameter block for your primary type sets target length, H2 count, opening shape, and analytical mode. The hard constraints below are Type 0 defaults — they apply EXCEPT where your type's block overrides them. Notable overrides:-- **Type 4 Humanities** targets 6–10 longer H2s (800–1500 words each), not 12–20.-- **Type 2 Market** wants one section per vendor *cluster*, not per individual vendor.-- **Type 5 Comparative** requires a mandatory comparison matrix in the body.-- **Type 6 Emerging** requires a "What we don't know yet" section.+**Before writing anything:** re-open `research/scaffold.md`. Re-read the verbatim user prompt. Re-read the "What the user explicitly asked for" extraction. Re-read the coverage checklist. This is mandatory — every draft session must begin with re-encountering the prompt.-Now write the report. These are hard constraints (under Type 0; type-specific overrides apply where listed above):+Then read your modality file's substance rules. The modality file tells you HOW to write: interpretation density (synthesize), enumerative completeness (collect), proportionate depth + recommendation (compare), probability language + time horizon (forecast).-- **Target 400–600 words per H2 section.** Sections under 200 words get merged. Sections over 800 get split — only if both halves stay above 300. (Type 4 Humanities: 800–1500.)-- **Aim for 12–20 H2 headings on a 10K-word report; 8–15 on a 5K-word report.** Above 25 is fragmentation — merge. (Type 4 Humanities: 6–10.)-- **Never write one section per source or one section per named entity.** A single H2 section weaves 3–8 sources together. A character, vendor, or paper does not earn its own section unless it has >400 words of real analysis attached.-- **Every body section ends with an analytical beat** — a 1–2 sentence "so what does this tell us" that integrates across the sources just cited. The Opinionated Synthesis at the end is NOT supposed to carry the synthesis load alone.-- **Open with the framework from Step 7, not a definition.** Section 1 = hard question + naive answer + structural tension.-- **Cite inline with parenthetical URLs.** Every non-trivial claim needs one. Prefer primary sources.-- **Comparison tables over fact tables.** A good table contrasts cases on a common axis. A bad table dumps attributes of one entity.-- **Word count targets.** Simple factual: 3,000–5,000. Complex/technical: 6,000–9,000. PhD-level: 9,000–12,000. Missing these is usually a signal that the scaffold from Step 7 is wrong — go back and fix the scaffold before writing more prose.+**Write `research/notes/final_report.md`** as a normal markdown file (no frontmatter needed for the draft — final save happens at Step 13).-## Step 10: Gap analysis (MANDATORY)+### Shared writing constraints (apply to every activity)-After writing your draft, stop and compare it against the original query. Re-read the user's request word by word:-- What specific questions did they ask that I haven't fully answered?-- What subtopics or angles did I miss entirely?-- Where did I make claims without strong enough evidence?-- What would an expert in this field say is missing?+- **The opening must establish the core tension.** What makes this question non-trivial? What paradox, disagreement, tradeoff, or open problem is the draft earning its way through? If the user requested a specific opening shape (definitions, executive summary, narrative hook), honor that shape — then make it do tension-framing work.+- **Every body section must earn its analytical beat.** A section that just describes facts without making an interpretive / comparative / forward move is a catalog entry, not a research contribution. Each section ends with a so-what, a comparison, a tension, or a forward beat.+- **Sources in tension at least twice in the body.** Find two places where your sources disagree and walk the reader through the disagreement, naming both positions and explaining which is more defensible. Synthesis alone does not make up for this — the body itself must engage dissent.+- **Every claim has a citation.** Inline parenthetical URL, or `[[note-id]]` if you want the reader to trace it in the vault.+- **Tier weighting.** Anchor substantive claims in `ground_truth` sources. Use `institutional` for positioning. Use `practitioner` for reality checks. Use `commentary` only to characterize reception, never to establish a load-bearing claim.+- **No padding to hit a length.** There is no length target. If your substance fits in 3,000 words, write 3,000 words. If it needs 12,000, write 12,000. A draft should be as long as the content demands.-Launch a new round of research to fill every gap — web search, fetch, follow links, spawn fetcher subagents in parallel. Add the new material to your draft.+### Activity-specific substance rules-## Step 11: Adversarial audit (MANDATORY — do this twice)+Open your modality file and read its Step 9 section. Its substance rules layer on top of the shared constraints above — they do not replace them.-After the gap-filling round, launch two subagents in parallel to audit the revised draft:+### Secondary flavor layering (MANDATORY when your scaffold declared a secondary)-**Agent 1 — Comprehensiveness auditor:**-```-Read this report and search the research base ($HPR search, $HPR note show) to find gaps.-What subtopics, angles, data points, or counterarguments are missing? What claims lack-citations? What sections are shallow? Compare against the original query and be ruthless —-list every gap you find.-```+If your Step 1 classification declared a secondary flavor, your draft must satisfy BOTH modalities' substance rules — not just the primary's. Read the **secondary modality's Step 9 substance rules AND its conformance checks** before you start writing.-**Agent 2 — Logic and structure auditor:**-```-Read this report as a domain expert. The report should declare its request type-(Step 0.5 classification) and follow the matching parameter block. Check specifically:- (1) Does section 1 establish a framework — hard question, naive answer, structural- tension — or does it just define terms?- (2) Are sections in dependency order? Does each section build on the previous one, or- could they be shuffled without loss?- (3) Are there sections under 200 words that should be merged into siblings?- (Type 4 Humanities: <400 is the floor, not 200.)- (4) Count H2 headings. Fragmentation thresholds vary by type: Type 0 General is >25 on- a <12K-word report; Type 4 Humanities is >12; Type 2 Market is >15.- (5) Does each body section end with an analytical beat, or does it just stop?- (6) Are sources compared against each other anywhere, or only listed in parallel?- (7) Does the argument flow logically — leaps, unsupported claims, contradictions?- (8) Does the report match its declared request type? If Type 4 Humanities, are sections- thematic rather than entity-catalog? If Type 5 Comparative, is there a mandatory- comparison matrix? If Type 6 Emerging, is there a "What we don't know yet" section?- If Type 2 Market, is there a position on who is winning? Flag every type violation.-List every weakness with specific section names and direct quotes.-```+Conflict resolution:+- **Primary wins on structure.** Section shape, entity-vs-thematic sequencing, comparison-matrix presence, scaffold skeleton — all determined by the primary modality.+- **Secondary wins on substance within sections.** Per-paragraph density rules, tension requirements, interpretation/enumeration discipline — these layer on top of the primary's structure.-Pass your full draft report text to both agents.+Example (Q91-style: `primary=collect`, `secondary=synthesize`):+- Collect (primary) forces per-character coverage across every entity category in the prompt. The structure is entity-based.+- Synthesize (secondary) forces every paragraph to fuse fact with interpretive claim AND at least two body sections to put sources in tension.+- The draft has entity-named sections (primary structure) where every paragraph inside makes an interpretive claim about what that entity means (secondary substance).-**After each audit round:**-1. Read both agents' feedback-2. If they found missing topics or weak arguments — do another round of web search + fetching-3. If the structure auditor flagged fragmentation, short sections, or a missing framework — **consolidate and rewrite, don't just patch.** Merging five 100-word sections into one 500-word section is a rewrite, not an edit. This is where catalog-style drafts become argument-style drafts.-4. Rewrite the report with new sources and structural fixes-5. Run the audit again (second loop) on the revised report+The auditor applies BOTH check lists at Step 11. Secondary-flavor violations are **not waivable** — they are CRITICAL findings that block synthesis save via the `audit-gate` lint rule. If a prompt is genuinely 50/50 across two activities, plan to satisfy both; do not silently drop the secondary.-**Do at least one audit loop. Do two if the first found significant gaps.** Only finalize after auditors have no major complaints.+----**While audit agents are running, don't idle.** Use the wait time to:-- Improve weak or missing summaries on notes (`$HPR note update <id> --summary "..." -j`)-- Add more specific tags to under-tagged notes (`$HPR note update <id> --add-tag <tag> -j`)-- Add `[[wiki-links]]` between related notes-- Run `$HPR lint -j` and fix issues-- Read notes you haven't reviewed yet for links worth following+## Step 10: Gap analysis + adversarial search-## Step 12: Opinionated synthesis (MANDATORY — end of report)+After the first draft, run adversarial searches specific to your activity — the modality file names the queries. Ask yourself:-A comprehensive catalog of facts is not a report. Before finalizing, you **MUST** append a section titled **`## Opinionated Synthesis`** at the very end of the report. This is where you stop reciting what sources said and start **reasoning across them**.+- Did you engage with the strongest counter-position?+- Is every item in the scaffold's coverage checklist actually ticked off in the draft?+- Are there tiers you're missing (all commentary, no ground_truth)?+- Is there a named voice you should be citing that isn't in the vault yet?-This section **complements — it does not replace** — the per-section analytical beats from Step 9. If all the analysis is dumped in this appendix and the body sections are purely descriptive, the report is still a catalog with a synthesis bolted onto the end. Both are required.+Fetch what's missing. Update the draft.-**Your Opinionated Synthesis section MUST include:**+----1. **Comparative analysis across categories.** Don't just describe each group or concept in isolation — compare them. What do the groups have in common? Where do they differ? What patterns emerge when you look at them side by side? Build at least one comparison table if the topic supports it.+## Step 11: Adversarial audit — spawn `hyperresearch-auditor` twice-2. **Thematic threads and progression.** What themes cut across the whole topic? What's the narrative arc? How do things evolve over time, across versions, across authors, across schools of thought? Identify the throughlines.+**Required artifact:** text output from both `hyperresearch-auditor` runs in your context, plus every identified violation applied to the draft.+**Verification:** in writing, list each violation the auditors raised and whether it has been fixed or explicitly rejected.+**On failure:** an unapplied audit is equivalent to no audit. Task has FAILED at Checkpoint 4.-3. **Your reasoned position.** Take a defensible stance. What's the most important finding? What's over-hyped? What's underrated? What do you expect to change in the next 2–5 years? Label it clearly as your own analysis, grounded in the sources you fetched.+**Spawn `hyperresearch-auditor` twice — once per mode. Run them SEQUENTIALLY, comprehensiveness first then conformance.** The auditor is a registered Claude Code agent (`.claude/agents/hyperresearch-auditor.md`) running on Opus. Both runs MUST receive the user's original research query **verbatim** — the same prompt you pasted into the scaffold.-4. **Open questions and limitations.** What did the sources NOT answer? What would an expert say is still unclear? Where is the research thin? Name the gaps explicitly.+Run comprehensiveness first so its findings land in `research/audit_findings.json` without a write race, then run conformance which reads the file and appends its own entry.-5. **Concluding thoughts.** One tight paragraph that ties everything back to the user's original question. What's the takeaway they should remember?+```+First spawn (wait for completion before the second):+ research_query: <paste the user's verbatim prompt from scaffold §1>+ modality: <collect | synthesize | compare | forecast>+ mode: comprehensiveness+ final_report_path: research/notes/final_report.md+ scaffold_note_id: <scaffold note id>+ comparison_note_id: <comparison note id>++Second spawn (only AFTER the first has returned and written its entry):+ research_query: <paste the user's verbatim prompt from scaffold §1>+ modality: <collect | synthesize | compare | forecast>+ mode: conformance+ final_report_path: research/notes/final_report.md+ scaffold_note_id: <scaffold note id>+ comparison_note_id: <comparison note id>+```++After both have returned, verify `research/audit_findings.json` has TWO new entries (one per mode) before proceeding. If only one mode's entry is present, re-spawn the missing mode — the `audit-gate` lint rule will fail at Step 13 if either is missing.-**Structure of the synthesis section (use these subheadings exactly):**+The auditor reads your modality file's "Conformance checks" section, applies each check, persists its findings to `research/audit_findings.json`, and returns a text summary with `pass` / `needs_fixes` / `failed` status. Apply every CRITICAL finding before saving the synthesis, and mark each fixed finding's `fixed_at` field in the JSON file as you go. The auditor also reads the scaffold — so it re-encounters the verbatim prompt — and verifies the draft honors the scaffold's coverage checklist.++---++## Step 12: Opinionated synthesis++Append this to the end of the draft:```markdown## Opinionated Synthesis-### Comparative Analysis-<cross-cutting comparisons — tables where useful>+### <Activity-specific header — see modality file>+<the cross-cutting picture the modality demands: thematic threads for+synthesize, comparison matrix for compare, position-with-horizon for+forecast, coverage summary for collect>### Thematic Threads-<patterns, progressions, throughlines across the topic>+<patterns across the body sections that the per-section analysis couldn't+surface — what the whole corpus is saying that no single source said>### My Reasoned Position-<your defensible stance, clearly labeled as analysis>+<your defensible stance, explicitly labeled. Synthesize demands a thesis;+compare demands a recommendation; forecast demands a prediction with+probability language and a time horizon; collect demands a "what this whole+set amounts to" claim.>### Open Questions-<gaps in the research, what the sources didn't answer>+<what the sources didn't settle, what data is unavailable, what would+change your position>### Concluding Thoughts-<one tight paragraph, answers the original question>+<one tight paragraph: the single most important thing a reader should+take away — and why>```-**Do NOT skip this section.** A report without an Opinionated Synthesis is incomplete. Even 9,000 words of carefully cataloged facts is just reference material without it. The synthesis is what separates a research report from a Wikipedia dump.+----## Step 13: Save the final report+## Steps 13–14: Save and present-Save the final reviewed report as a synthesis note:+**Before saving the synthesis**, run the audit-gate lint to confirm the audit loop is closed. This is a hard gate — the synthesis cannot be saved until it returns zero error-severity issues:```bash-cat > /tmp/synthesis.md << 'SYNTHESIS'-# <Topic> — Research Synthesis+$HPR lint --rule audit-gate -j+```-## Key Findings-<your synthesis here>+The `audit-gate` rule checks three conditions against `research/audit_findings.json`:-## Sources-<wiki-links to all source notes: [[note-id-1]], [[note-id-2]], etc.>+1. **Both audit modes must have appended a run.** If the file has a comprehensiveness run but no conformance run (or vice versa), the gate fails with a missing-mode error — spawn the missing auditor mode and retry.+2. **No unresolved CRITICAL findings in the most recent conformance run.** Every CRITICAL must have a non-null `fixed_at` timestamp. Walk the unresolved list, apply the fix to `research/notes/final_report.md`, and mark `fixed_at: <current ISO timestamp>` on the finding in the JSON file. If a finding cannot be fixed (e.g. the auditor misread the draft), resolve it by marking `fixed_at` with a `notes` field explaining why the rejection is safe — but that is a last resort, not a bypass.+3. **IMPORTANT findings surface as `info` severity** (advisory, does not block save). They appear in the lint output so you see them before committing. You are strongly encouraged to patch them if they name real gaps — they often do. Mark each as `fixed_at` in the JSON once patched.++After applying fixes:+- If you patched the draft, re-spawn the conformance auditor (which will append a fresh run to the findings file) and re-run the lint.+- If `audit-gate` returns zero error-severity issues — proceed to save.+- If it still fails, diagnose which specific finding is blocking and address it. **Do NOT bypass the gate by editing `audit_findings.json` to clear findings without patching the draft first.**++This is how the audit loop actually closes: findings are persisted → fixes are applied → `fixed_at` is set → the lint passes → the synthesis saves. Each step leaves an artifact on disk.++Using the **Write tool**, create `research/synthesis.md`:+```markdown+# <Topic> — Synthesis+## Position+<your thesis / recommendation / forecast / coverage summary>+## Sources+<[[note-id-1]], [[note-id-2]], ...>## Open Questions-<what wasn't answered, what needs more research>-SYNTHESIS+<unresolved>+```-$HPR note new "Synthesis: <topic>" --tag <topic> --tag synthesis --type moc --status review --body-file /tmp/synthesis.md --summary "<one-line summary of findings>" -j+Then run:++```bash+$HPR note new "Synthesis: <topic>" --tag <topic> --tag synthesis --type moc \+ --status review --body-file research/synthesis.md \+ --summary "<one-line position>" -j```-## Step 14: Present to user+Present to the user: your position in 2–3 sentences, the key structural beats, sources collected, and what would change your position.++---++## Process review checkpoints++Each checkpoint is a STOP point. Run the commands, state results in writing, decide whether the task has failed.-Present your findings with:+### Checkpoint 1 — after Step 3 (post-fetch)-1. **Executive summary** — 2-3 sentence answer to their question-2. **Key findings table** — the most important discoveries-3. **Sources collected** — how many notes, from which domains-4. **Hub notes** — the most-referenced/important notes in the graph-5. **Open questions** — what you didn't find or what needs more research-6. **How to explore** — tell them they can ask follow-up questions and you'll search the KB+**Purpose:** verify the corpus has enough voice diversity before reading it.-Example closing:+```bash+$HPR note list --status draft -j | python -c "import sys,json; d=json.load(sys.stdin)['data']; print(f'draft notes: {len(d)}')"+$HPR sources list -j | python -c "+import sys, json+from collections import Counter+from urllib.parse import urlparse+d = json.load(sys.stdin)['data']+sources = d if isinstance(d, list) else d.get('sources', [])+total = len(sources)+domains = Counter(urlparse(s['url']).netloc.replace('www.','') for s in sources if s.get('url'))+print(f'total sources: {total}, unique domains: {len(domains)}')+for dom, count in domains.most_common(10):+ pct = count / max(total, 1) * 100+ flag = ' <-- OVER 30% (concentrated)' if pct > 30 else ''+ print(f' {count:>3} ({pct:>5.1f}%) {dom}{flag}')+"+```++**Pass conditions:**+- [ ] At least one note fetched from each mandatory adversarial search+- [ ] No single domain represents >30% of fetched sources (voice diversity)+- [ ] At least 5 unique non-reference voices (critical essays, named-author blog posts, peer-reviewed papers — NOT listicles or reference wikis)++**On failure:** run additional searches with diversified queries, vary the source types, seek named voices. Return to Step 2.++### Checkpoint 2 — after Steps 5–6 (post-curate)++**Purpose:** verify every source has been analyzed AND classified.++```bash+$HPR lint --rule uncurated -j+$HPR lint --rule provenance -j+$HPR lint --rule analyst-coverage -j+$HPR note list --all -j | python -c "import sys,json; from collections import Counter; notes=json.load(sys.stdin)['data']; t=Counter(n.get('tier') for n in notes if n.get('status')!='draft'); ct=Counter(n.get('content_type') for n in notes if n.get('status')!='draft'); print(f'tiers: {dict(t)}'); print(f'content_types: {dict(ct)}')"```-Research complete. 15 sources collected on "<topic>":-- 3 primary papers, 5 technical blogs, 4 documentation pages, 3 news articles-- Hub notes: [[key-note-1]], [[key-note-2]]-- All sources are saved and searchable across sessions.-Ask me anything about this topic — I'll search the research base first.+**Pass conditions:**+- [ ] Zero `uncurated` issues+- [ ] Zero `provenance` **errors** — the rule itself computes the non-seed ratio and errors when under 30% on a corpus >10, so if this lint returns zero errors the guided loop fired adequately. Read its output carefully; its message tells you `non_seeds / total` directly.+- [ ] Zero `analyst-coverage` issues (at least 33% of fetched sources have a paired extract note)+- [ ] Tier and content_type distributions are nuanced (not one dominant value)+- [ ] At least two tiers represented+- [ ] At least three content_types represented++**On failure:** re-classify per-note based on each analyst's return; spawn analysts on skipped sources; **run the guided reading loop**: spawn `hyperresearch-analyst` on existing sources, collect their `next_targets`, and fetch the top 3-5 WITH `--suggested-by <source-note-id>`. You cannot proceed to Step 7 (scaffold) until the guided loop has fired at least once — the provenance chain is a load-bearing invariant.++**Specifically when `provenance` errors:** the rule message will say something like *"Only 3/19 source notes (16%) have breadcrumbs — the guided reading loop did not fire"*. Read that number literally. If it's below 30%, you MUST return to Step 4 and run the guided loop: spawn an analyst on each existing source, collect `next_targets`, fetch each target with `--suggested-by <source-note-id> --suggested-by-reason "<why>"`, then re-run Checkpoint 2. Skipping this means the draft rests on seed fetches alone and the bouncing-loop-discovered critical voices never made it into the corpus.++### Checkpoint 3 — BEFORE Step 9 (pre-draft gate — CRITICAL)++**Purpose:** this is the gate that protects the draft from being written on a broken foundation.++```bash+$HPR note list --tag scaffold -j | python -c "import sys,json; d=json.load(sys.stdin)['data']; print(f'scaffold notes: {len(d)}')"+$HPR note list --tag comparison -j | python -c "import sys,json; d=json.load(sys.stdin)['data']; print(f'comparison notes: {len(d)}')"+$HPR note list --tag extract -j | python -c "import sys,json; d=json.load(sys.stdin)['data']; print(f'extract notes: {len(d)}')"+$HPR lint --rule scaffold-prompt -j+$HPR lint --rule uncurated -j+$HPR lint --rule provenance -j+$HPR lint --rule analyst-coverage -j+$HPR lint --rule workflow -j```-## Important rules+**Pass conditions:**+- [ ] Scaffold note count ≥ 1+- [ ] Zero `scaffold-prompt` issues — the scaffold body MUST contain the verbatim user prompt as its first section. This is machine-checked; the gospel rule is non-negotiable.+- [ ] Comparison note count ≥ 1+- [ ] Extract note count ≥ 30% of fetched source count+- [ ] Zero `uncurated`, `provenance`, `analyst-coverage`, `workflow` issues++**On failure:** you CANNOT write the draft. Return to the step that produces the missing artifact. If `scaffold-prompt` failed, return to Step 7 and re-write the scaffold with the user's verbatim prompt as its first section — do not fudge it.++### Checkpoint 4 — after Step 11 (post-audit)++**Purpose:** verify the audit fired and its findings were applied.-- **NEVER use WebFetch for source pages.** Always use `$HPR fetch` or the fetcher subagent.-- **Over-collect, then prune.** Fetch aggressively. Deprecate irrelevant notes after synthesis.-- **Save raw content.** Don't rewrite or summarize sources before saving. The full text is the value.-- **Follow links.** A blog post about a paper is not the paper. Fetch the paper.-- **Save your synthesis.** Every research session should produce a synthesis note that future agents can find.+**Pass conditions:**+- [ ] `hyperresearch-auditor` was spawned twice (comprehensiveness + conformance) and both returned+- [ ] Every violation flagged has been fixed in `research/notes/final_report.md` OR explicitly rejected in writing with a reason+- [ ] If the conformance auditor flagged missing scaffold / comparison / extract notes, you returned to the relevant checkpoint and fixed it++**On failure:** spawn the auditor (or re-spawn) and apply the findings. Do not save the synthesis note until this checkpoint passes.++---++## CLI path++Find the `hyperresearch` executable from CLAUDE.md ("CLI path:" line) and store it:++```bash+HPR="<path from CLAUDE.md>"+```