Guides and resources for AI engineers: an agents course, five handbooks, a hands-on second-brain project and a vetted reading list.
JavaScript agent-skillsai-agentsai-engineeringclaudeclaude-codeclaude-skillsknowledge-graphknowledge-managementllmmcpobsidianpersonal-knowledge-basepkmsecond-brain
resources: verified links, skills index
undefined-ui56cc1e395004
diff --git a/README.md b/README.mdindex 1d77b7a..a584dcb 100644--- a/README.md+++ b/README.md@@ -158,13 +158,14 @@ source, and tool listings need a reason to exist.## Credit-The LLM wiki pattern is Andrej Karpathy's, published as a gist on 2026-04-04.-This repo is one implementation of it, plus the parts the gist deliberately-leaves undefined.+The LLM wiki pattern is Andrej Karpathy's, published as a+[gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) on+4 April 2026. This repo is one implementation of it, plus the parts the gist+deliberately leaves undefined.<!-- TODO before publishing:- add the link to the original article- - add the direct Karpathy gist URL, verified by hand- - screenshots: graph view at 30 days, an example concept page -->+ - screenshots: graph view at 30 days, an example concept page+ - social preview image in Settings -> General -->MIT licensed.diff --git a/docs/01-concepts/llm-wiki-pattern.md b/docs/01-concepts/llm-wiki-pattern.mdindex 630eb14..2b7d315 100644--- a/docs/01-concepts/llm-wiki-pattern.md+++ b/docs/01-concepts/llm-wiki-pattern.md@@ -49,7 +49,7 @@ layers](two-layers.md) and [Agents](../06-agents/README.md).The pattern also spread quickly because the primitives are ordinary: anyone withan agent and a folder can run it, and the implementations are all readable-markdown. See [resources](../../resources/repos.md) for the ones worth reading.+markdown. See [resources](../../resources/skills.md) for the ones worth reading.## Nextdiff --git a/docs/02-setup/mcp-obsidian.md b/docs/02-setup/mcp-obsidian.mdindex 0597908..61256d3 100644--- a/docs/02-setup/mcp-obsidian.md+++ b/docs/02-setup/mcp-obsidian.md@@ -8,6 +8,18 @@ inside that directory.You do not need this to start. Add it when you want scheduled tasks, or whenyou are working from the desktop app rather than a terminal in the vault.+## Two routes++The [Local REST API+plugin](https://github.com/coddingtonbear/obsidian-local-rest-api) now ships a+built-in MCP server at `/mcp/`, so a separate server is no longer required.+Check its README first: if the built-in server covers what you need, skip+straight to registering it with your client and ignore step 2 below.++The third-party route described here, `mcp-obsidian` over the plugin's REST API,+is still the most widely documented and is what most guides describe. Both end+up in the same place.+## Step 1: enable the pluginIn Obsidian: **Settings** (gear, bottom left) → **Community plugins** → **Turn@@ -60,6 +72,14 @@ If the agent reads your notes back, the connection works.- **Plugin disabled.** Community plugins get turned off when restricted mode isre-enabled after an update.+## Do you need this at all++One maintained fork of `mcp-obsidian` was archived with the reasoning that+routing through a REST plugin adds complexity when a vault is just markdown+files on disk. That is the same argument this guide makes for starting on the+filesystem, and it is worth taking seriously: MCP earns its place when you need+the vault reachable from sessions not running inside it, and not before.+## SecurityThis opens a local HTTP server with a key that grants full read and write accessdiff --git a/resources/README.md b/resources/README.mdindex 31d505e..a61f67c 100644--- a/resources/README.md+++ b/resources/README.md@@ -1,12 +1,12 @@# Resources-Curated, not exhaustive. Everything here is something worth the time it takes-to read or install.+Curated, not exhaustive. Every link here was opened and checked, and anything+that exists only in secondary coverage is marked as such.-- [Tools](tools.md) - editors, capture, agents, graph and search-- [Repositories](repos.md) - open-source implementations worth reading-- [Reading](reading.md) - the ideas behind the pattern-- [Papers](papers.md) - retrieval, graphs and memory research+- [Tools](tools.md). Editors, capture, agents, graph and search+- [Skills and implementations](skills.md). Agent skills and open-source builds+ of this pattern+- [Reading](reading.md). The ideas behind the pattern+- [Papers](papers.md). Retrieval, graphs and long-context research-Links are checked before they go in. If something here has gone stale or a-better option exists, open a pull request.+If something here has gone stale or a better option exists, open a pull request.diff --git a/resources/papers.md b/resources/papers.mdindex 99b4017..a7fa005 100644--- a/resources/papers.md+++ b/resources/papers.md@@ -1,18 +1,61 @@# PapersResearch relevant to how a knowledge base retrieves and reasons. Read these if-you want to understand why the wiki approach works, or to decide whether you-need a vector database at all.--- **From Local to Global: A Graph RAG Approach to Query-Focused- Summarization** (Microsoft Research). Introduces GraphRAG: build an entity- graph from a corpus, summarise communities, answer global questions the- chunk-retrieval approach cannot.-- **HippoRAG**. Memory-inspired retrieval using a knowledge graph and- personalised PageRank, aimed at multi-hop questions.-- **Lost in the Middle: How Language Models Use Long Contexts**. Why dumping a- whole vault into context degrades answers, and the argument for structured- retrieval over long-context stuffing.--<!-- TODO before publishing: verify each citation against the arXiv listing and- add direct links. Nothing goes in this file from secondary coverage. -->+you want to decide for yourself whether you need a vector database.++## GraphRAG++**[From Local to Global: A Graph RAG Approach to Query-Focused+Summarization](https://arxiv.org/abs/2404.16130)** (arXiv:2404.16130). Edge,+Trinh, Cheng, Bradley, Chao, Mody, Truitt, Metropolitansky, Ness, Larson.+Microsoft Research. Implementation at+[microsoft/graphrag](https://github.com/microsoft/graphrag).++The paper that named the approach. Its argument is specific and worth getting+right: RAG fails on global questions about an entire corpus, such as what the+main themes are, because that is a summarization task rather than a retrieval+task. Their pipeline derives an entity graph from the documents, pregenerates+summaries for communities of closely related entities, and answers global+questions from those summaries.++Relevant to a second brain because the failure it targets, questions about the+whole corpus, is exactly what people want from one. See+[GraphRAG and where it fits](../docs/05-graphs/graphrag.md) for why you probably+do not need to run it at personal scale.++## HippoRAG++**[HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language+Models](https://arxiv.org/abs/2405.14831)** (arXiv:2405.14831). Gutiérrez, Shu,+Gu, Yasunaga, Su.++Retrieval over a knowledge graph using personalised PageRank, aimed at multi-hop+questions. The closest published analogue of what an agent does walking links+outward from a starting page, which makes it the more directly relevant of the+two graph papers here.++## Lost in the Middle++**[Lost in the Middle: How Language Models Use Long+Contexts](https://arxiv.org/abs/2307.03172)** (arXiv:2307.03172). Liu, Lin,+Hewitt, Paranjape, Bevilacqua, Petroni, Liang. Published in TACL.++The empirical basis for not pasting your whole vault into context. Performance+is highest when the relevant information sits at the beginning or end of the+input and degrades significantly when the model has to use something in the+middle of a long context, including for models built for long contexts.++This is why [progressive+disclosure](../docs/07-retrieval/context-budget.md) beats loading everything,+and why an answer citing five pages is more trustworthy than one assembled from+four hundred.++## How to read these++All three are specific about the conditions their results hold under. Those+conditions are what decide whether the finding transfers to a personal vault of+a few thousand curated pages, which is a very different setting from the+corpora they evaluate on.++Read the limitations sections. That is the habit this whole guide is trying to+encode.diff --git a/resources/reading.md b/resources/reading.mdindex 137928b..99477d7 100644--- a/resources/reading.md+++ b/resources/reading.md@@ -1,26 +1,43 @@# Reading-## The pattern+## The source document-- Karpathy's `llm-wiki` gist (2026-04-04). The spec everything else builds on:- the LLM maintains the wiki, the wiki is primary, raw sources are archive.+**[Karpathy's llm-wiki gist](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)**,+published 4 April 2026. The pattern everything in this repo builds on.++It is short, and it is written as an idea file meant to be pasted into your own+agent rather than as a spec. The core argument: most people's use of LLMs with+documents is RAG, where the model rediscovers knowledge from scratch on every+question and nothing accumulates.++Read it first. Everything else, including this guide, is one interpretation of+it.## The tradition it comes from-- **Building a Second Brain**, Tiago Forte. Where the term comes from, and the- PARA method. Written before agents existed, which is exactly what makes it- useful to read now: most of the book is about the maintenance work that an- agent removes.- **How to Take Smart Notes**, Sönke Ahrens. The Zettelkasten method. Theargument for atomic notes and dense linking still holds; the manual labour itdemands is the part that killed it for most people.-- **Andy Matuschak's evergreen notes**. The strongest case for notes written to- be reread, and a useful counterweight: not everything should be delegated to- a machine.+- **Building a Second Brain**, Tiago Forte. Where the term comes from, and the+ PARA method. Written before agents existed, which is what makes it useful now:+ most of the book is about the maintenance work an agent removes.+- **[Andy Matuschak's notes](https://notes.andymatuschak.org)**. Evergreen notes,+ written in public as a working example of the idea. The strongest case for+ notes written to be reread, and a useful counterweight to this whole approach:+ his position is that the writing is the thinking, so delegating it means not+ doing it.## Why the maintenance argument matters-Every note system in the list above works. They fail on the same thing, which-is that the person maintaining them gets tired. That is the entire reason this-pattern is different, and it is worth reading the originals to see how much of-the design was shaped by that constraint.+Every method above works. They fail on the same thing, which is that the person+maintaining them gets tired. That single constraint shaped all of their design+decisions, and it is the constraint that changed.++Reading the originals is worth it for that reason specifically: you can see+which parts were about knowledge and which were workarounds for human effort.++## Talks and videos++This section is deliberately empty rather than padded with whatever ranks well+on YouTube. If you have watched something on this topic that was worth the time,+open an issue or a pull request with the link and one line on why.diff --git a/resources/skills.md b/resources/skills.mdnew file mode 100644index 0000000..0b7e4ab--- /dev/null+++ b/resources/skills.md@@ -0,0 +1,52 @@+# Skills and implementations++Agent skills that run this pattern, and open-source builds worth reading before+writing your own.++## In this repo++Four skills in [`skills/`](../skills/README.md): ingest, lint, query, review.+Five commands in [`commands/`](../commands/README.md). Four subagent definitions+in [`agents/`](../agents/README.md), two of them read-only by design.++All plain `SKILL.md` files, so they work with any agent that reads the Agent+Skills format, not only Claude Code.++## Community implementations++Reading someone else's page contracts is the fastest way to improve your own.+These are worth opening even if you never install them.++- **[Astro-Han/karpathy-llm-wiki](https://github.com/Astro-Han/karpathy-llm-wiki)**.+ The pattern packaged as a single installable skill for Claude Code, Cursor and+ Codex. Ingest into `raw/`, compile into `wiki/`, cited answers, lint. Includes+ a design spec and a list of what the author deliberately did not build, which+ is the most useful section in it.+- **[micuintus/llm-wiki](https://github.com/micuintus/llm-wiki)**. A deliberately+ minimal skill: no dependencies, no JSON metadata, convention-based guardrails+ rather than code-based ones. Good counterpoint to the heavier+ implementations, and its README lists several others in the same family.+- **[NicholasSpisak/second-brain](https://github.com/NicholasSpisak/second-brain)**.+ Four skills installable through npm, close to the original gist, with a setup+ wizard.+- **[eugeniughelbur/obsidian-second-brain](https://github.com/eugeniughelbur/obsidian-second-brain)**.+ Around 43 ready commands, works across Claude, Codex and Gemini. The author's+ write-up of what the gist leaves undefined is worth reading alongside it.++Two more get recommended often and are worth a look if the above do not fit:+`AgriciDaniel/claude-obsidian`, which ships role presets, and+`coleam00/second-brain-starter`, which interviews you and generates a plan+before building anything.++## Writing your own++The format is a `SKILL.md` with frontmatter and a body. The four in this repo+follow a consistent shape: why the skill exists, one core rule, a workflow, an+output template, calibration, and a worked example.++Two authoring rules that matter more than the rest. Explain the reason behind+every rule, because rules cover the cases you thought of and reasons cover the+rest. And name the near-misses where the skill should not fire, because+over-triggering is what makes people uninstall skills.++Details in [skills and commands](../docs/06-agents/skills-and-commands.md).diff --git a/resources/tools.md b/resources/tools.mdindex 031c12c..0187f04 100644--- a/resources/tools.md+++ b/resources/tools.md@@ -2,41 +2,73 @@## The core stack-- **[Obsidian](https://obsidian.md)** - local markdown editor. The vault is a+- **[Obsidian](https://obsidian.md)**. Local markdown editor. The vault is aplain folder, which is the whole reason agents work well with it.-- **[Obsidian Web Clipper](https://obsidian.md/clipper)** - official browser- extension for saving articles into the vault. Built by the Obsidian team.-- **[Claude Code](https://claude.com/claude-code)** - the agent that maintains- the wiki. Runs in the vault folder and edits files directly.+- **[Obsidian Web Clipper](https://obsidian.md/clipper)**. Official browser+ extension for saving articles into the vault, built by the Obsidian team. Uses+ Mozilla Readability, the engine behind Firefox Reader View.+- **[Claude Code](https://code.claude.com/docs/en/setup)**. The agent that+ maintains the wiki. Runs in the vault folder and edits files directly.+ Requires a paid account; the free plan does not include access.## Obsidian plugins-- **Dataview** - queries over frontmatter. Useful for dashboards of what needs- review.-- **Templater** - templates with variables, for the page shapes in- `vault-template/templates/`.-- **Local REST API** - required only if you want the agent to reach the vault- over MCP rather than through the filesystem.+- **[Dataview](https://github.com/blacksmithgu/obsidian-dataview)**. A query+ language over your frontmatter, rendered inline. This is what answers+ structural questions like "which concept pages have not been touched in ninety+ days". Docs at+ [blacksmithgu.github.io/obsidian-dataview](https://blacksmithgu.github.io/obsidian-dataview/).+- **[Templater](https://github.com/SilentVoid13/Templater)**. Templates with+ variables and dates, for the page shapes in `vault-template/templates/`. The+ agent does not need it; you will, for pages you write by hand.+- **[Local REST API](https://github.com/coddingtonbear/obsidian-local-rest-api)**.+ Serves your vault over a local API. Worth reading the README before choosing+ an MCP route: the plugin now ships a **built-in MCP server** at `/mcp/`, which+ removes the need for a separate server in most setups.++## MCP++- **[mcp-obsidian](https://github.com/MarkusPfundstein/mcp-obsidian)**. The+ widely used third-party MCP server, run via `uvx mcp-obsidian`, talking to the+ Local REST API plugin. Still the most documented path, and the one most guides+ including this one describe.++Worth knowing before you pick: at least one maintained fork+([proofsh/obsidian-mcp](https://github.com/proofsh/obsidian-mcp)) has been+archived with the explicit reasoning that routing through a REST plugin adds+complexity when the vault is markdown files on disk. That is the same argument+this guide makes for starting on the filesystem.## Capture-- **yt-dlp** - subtitles and audio from video platforms.-- **youtube-transcript-api** - Python library for pulling transcripts directly.-- **Zotero** - reference manager with a markdown export path for papers.-- **Readwise** - highlights from Kindle and read-later apps, exports to- Obsidian.+- **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**. Subtitles and audio from video+ platforms. `--write-auto-sub --skip-download` is the flag pair you want.+- **[youtube-transcript-api](https://github.com/jdepoix/youtube-transcript-api)**.+ Python library for pulling transcripts directly.+- **[OCRmyPDF](https://github.com/ocrmypdf/OCRmyPDF)**. Adds a text layer to+ scanned PDFs without changing the visible document.+- **[Zotero](https://www.zotero.org)**. Reference manager with a markdown export+ path, worth it if you read papers regularly.+- **[Readwise](https://readwise.io)**. Kindle and read-later highlights, syncs+ into Obsidian. Paid.## Search and graph-- **ripgrep** - fast enough that most vaults never need an index.-- **NetworkX** - graph analysis in Python once you have exported an edge list.-- **Kuzu** - embedded graph database, no server to run.-- **Neo4j** - full graph database, worth it only at scale.-- **Gephi** - visual graph exploration beyond what Obsidian's graph view does.+- **[ripgrep](https://github.com/BurntSushi/ripgrep)**. Fast enough that most+ vaults never need an index.+- **[NetworkX](https://networkx.org)**. Graph analysis in Python once you have+ exported an edge list.+- **[Kuzu](https://kuzudb.com)**. Embedded graph database, no server to run.+- **[Neo4j](https://neo4j.com)**. The full version. Overkill unless you are+ building something on top.+- **[Gephi](https://gephi.org)**. Visual graph exploration beyond what Obsidian's+ graph view does.## Alternatives to Obsidian-- **Logseq** - outliner, also plain markdown, block-level references.-- **Foam** - the same pattern inside VS Code.-- **Plain folders plus git** - genuinely viable. Obsidian is a viewer for the+- **[Logseq](https://logseq.com)**. Outliner, also plain markdown, block-level+ references.+- **[Foam](https://foambubble.github.io/foam/)**. The same pattern inside VS+ Code.+- **Plain folders plus git.** Genuinely viable. Obsidian is a viewer for thegraph, not a requirement of it.