Skip to content

Fastest way to scaffold your LLM powered personal knowledge base - Obsidian, Claude Code, qmd, Github

feat: add /brain-refresh and /brain-rebuild skills

jessepinkman99008150df53a093
diff --git a/.claude/skills/pack-test/SKILL.md b/.claude/skills/pack-test/SKILL.md
index 9f4fea8..b7faa16 100644
--- a/.claude/skills/pack-test/SKILL.md
+++ b/.claude/skills/pack-test/SKILL.md
@@ -45,7 +45,7 @@ ls test-vault/wiki/
# Sources dirs
ls test-vault/sources/
-# Claude Code skills (expect: brain-ingest, brain-search, lint, qmd-cli, setup)
+# Claude Code skills (expect: brain-ingest, brain-search, brain-refresh, brain-rebuild, lint, qmd-cli, setup)
ls test-vault/.claude/skills/
# qmd scripts
@@ -60,9 +60,11 @@ grep "const DB" test-vault/scripts/qmd/setup.ts
# Global skills must be installed
ls ~/.claude/skills/brain-ingest/
ls ~/.claude/skills/brain-search/
+ls ~/.claude/skills/brain-refresh/
# Global skill content must have the absolute INDEX_PATH
grep "INDEX_PATH=" ~/.claude/skills/brain-ingest/SKILL.md
+grep "INDEX_PATH=" ~/.claude/skills/brain-refresh/SKILL.md
```
Report results against this checklist:
@@ -73,13 +75,14 @@ Report results against this checklist:
| `wiki/index.md`, `log.md`, `overview.md` | All present | |
| `wiki/sources/` and `wiki/qa/` | Present | |
| `sources/articles/`, `pdfs/`, `personal/` | Present | |
-| `.claude/skills/` — 5 subdirs | brain-ingest, brain-search, lint, qmd-cli, setup | |
+| `.claude/skills/` — 7 subdirs | brain-ingest, brain-search, brain-refresh, brain-rebuild, lint, qmd-cli, setup | |
| `scripts/qmd/setup.ts`, `reindex.ts`, `package.json` | All present | |
| `CLAUDE.md`, `mise.toml` | Present | |
| `CLAUDE.md` INDEX_PATH | Absolute path (not `qmd.sqlite`) | |
| `scripts/qmd/setup.ts` DB const | Hardcoded absolute path (not `join(VAULT, ...)`) | |
| `~/.claude/skills/brain-ingest/SKILL.md` | Present with absolute INDEX_PATH | |
| `~/.claude/skills/brain-search/SKILL.md` | Present with absolute INDEX_PATH | |
+| `~/.claude/skills/brain-refresh/SKILL.md` | Present with absolute INDEX_PATH | |
**Step 4 — Clean up**
@@ -88,10 +91,11 @@ rm -rf test-vault
rm claude-second-brain-$(node -p "require('./package.json').version").tgz
rm -rf ~/.claude/skills/brain-ingest
rm -rf ~/.claude/skills/brain-search
+rm -rf ~/.claude/skills/brain-refresh
gh repo delete test-vault --yes
```
-> **Note:** If you had pre-existing `brain-ingest` / `brain-search` global skills from a real vault, they were overwritten by this test. Re-run `/setup` inside your real vault to restore them.
+> **Note:** If you had pre-existing `brain-ingest` / `brain-search` / `brain-refresh` global skills from a real vault, they were overwritten by this test. Re-run `/setup` inside your real vault to restore them.
## What failure means
diff --git a/.github/workflows/pack-test.yml b/.github/workflows/pack-test.yml
index a99606d..c8d6cff 100644
--- a/.github/workflows/pack-test.yml
+++ b/.github/workflows/pack-test.yml
@@ -58,6 +58,8 @@ jobs:
check ".claude/skills/setup" test-vault/.claude/skills/setup
check ".claude/skills/brain-ingest" test-vault/.claude/skills/brain-ingest
check ".claude/skills/brain-search" test-vault/.claude/skills/brain-search
+ check ".claude/skills/brain-refresh" test-vault/.claude/skills/brain-refresh
+ check ".claude/skills/brain-rebuild" test-vault/.claude/skills/brain-rebuild
check ".claude/skills/lint" test-vault/.claude/skills/lint
check ".claude/skills/qmd-cli" test-vault/.claude/skills/qmd-cli
check "scripts/qmd/setup.ts" test-vault/scripts/qmd/setup.ts
@@ -65,8 +67,9 @@ jobs:
check "scripts/qmd/package.json" test-vault/scripts/qmd/package.json
check "CLAUDE.md" test-vault/CLAUDE.md
check "mise.toml" test-vault/mise.toml
- check "~/.claude/skills/brain-ingest/SKILL.md" "$HOME/.claude/skills/brain-ingest/SKILL.md"
- check "~/.claude/skills/brain-search/SKILL.md" "$HOME/.claude/skills/brain-search/SKILL.md"
+ check "~/.claude/skills/brain-ingest/SKILL.md" "$HOME/.claude/skills/brain-ingest/SKILL.md"
+ check "~/.claude/skills/brain-search/SKILL.md" "$HOME/.claude/skills/brain-search/SKILL.md"
+ check "~/.claude/skills/brain-refresh/SKILL.md" "$HOME/.claude/skills/brain-refresh/SKILL.md"
grep_check() {
local label="$1"
@@ -80,9 +83,11 @@ jobs:
fi
}
- grep_check "CLAUDE.md INDEX_PATH is absolute" "INDEX_PATH=/" test-vault/CLAUDE.md
- grep_check "setup.ts DB is hardcoded absolute path" 'const DB = "/' test-vault/scripts/qmd/setup.ts
- grep_check "global brain-ingest INDEX_PATH is absolute" "INDEX_PATH=/" "$HOME/.claude/skills/brain-ingest/SKILL.md"
+ grep_check "CLAUDE.md INDEX_PATH is absolute" "INDEX_PATH=/" test-vault/CLAUDE.md
+ grep_check "setup.ts DB is hardcoded absolute path" 'const DB = "/' test-vault/scripts/qmd/setup.ts
+ grep_check "global brain-ingest INDEX_PATH is absolute" "INDEX_PATH=/" "$HOME/.claude/skills/brain-ingest/SKILL.md"
+ grep_check "global brain-search INDEX_PATH is absolute" "INDEX_PATH=/" "$HOME/.claude/skills/brain-search/SKILL.md"
+ grep_check "global brain-refresh INDEX_PATH is absolute" "INDEX_PATH=/" "$HOME/.claude/skills/brain-refresh/SKILL.md"
if [ "$fail" -ne 0 ]; then
echo ""
diff --git a/CLAUDE.md b/CLAUDE.md
index 517c68b..04379c0 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -229,3 +229,21 @@ Add `--json` for structured output. Omit `-c wiki` to search all collections (wi
bun scripts/qmd/reindex.ts
```
Do NOT re-index after every single file edit.
+
+---
+
+## Repo Maintenance (dev-only, not shipped to user vaults)
+
+This section applies only when working on the `claude-second-brain` repo itself — it is not part of the template copied into user vaults.
+
+Whenever you **add, remove, or rename a skill** under `template/.claude/skills/`, you MUST update the following in the same change:
+
+1. **`README.md`** (root) — the "Claude Code skills included" section and the "Installing and updating skills" block. Bump the skill count ("N wiki skills") in the install commands.
+2. **`template/README.md`** — the "Your Claude Code skills" section and the "Installing and updating skills" block. Bump the skill count.
+3. **`bin/create.js`** — the `installGlobalSkills()` array (lines ~65) if the skill should be installed globally (`~/.claude/skills/`). `brain-ingest`, `brain-search`, `brain-refresh` are global today; `brain-rebuild`, `lint`, `setup`, `qmd-cli` are vault-local.
+4. **`.github/workflows/pack-test.yml`** — add a `check` line for the new vault-local skill directory, and if the skill is global, add a `check` line for `$HOME/.claude/skills/<name>/SKILL.md` plus a `grep_check` for `INDEX_PATH=/`.
+5. **`.claude/skills/pack-test/SKILL.md`** — update the expected-skills comment, the `ls` and `grep` commands in Step 3, the checklist table (including the "N subdirs" count), and the cleanup / overwrite-warning note in Step 4.
+
+Whenever you **change a qmd collection name, context path, or CLI invocation** in `scripts/qmd/setup.ts` or in skill workflows, you MUST also update every `-c <collection>` reference in `template/CLAUDE.md`, all `SKILL.md` files under `template/.claude/skills/`, and both READMEs.
+
+Use `__QMD_PATH__` as the placeholder in any template file that references the qmd index path — `bin/create.js::patchVault()` substitutes it at scaffold time.
diff --git a/README.md b/README.md
index 2b333a5..c671002 100644
--- a/README.md
+++ b/README.md
@@ -96,7 +96,9 @@ Open `my-brain/` as a vault in Obsidian — the folder is already a valid Obsidi
## Claude Code skills included
-The wiki ships with two global slash commands that cover the full workflow. No manual prompting, no copy-pasting.
+The wiki ships with a set of slash commands that cover the full workflow. No manual prompting, no copy-pasting.
+
+### Daily workflow
**`/brain-ingest`** — Drop a file into `sources/articles/`, `sources/pdfs/`, or `sources/personal/`. Run `/brain-ingest`. Claude summarizes the source, asks what matters most to you, creates a `wiki/sources/` page, updates or creates related topic pages, flags any contradictions with existing knowledge, and logs everything.
@@ -104,6 +106,16 @@ The wiki ships with two global slash commands that cover the full workflow. No m
**`/lint`** — Health-check the wiki. Surfaces orphan pages, broken links, unresolved contradictions, and data gaps. Reports findings and applies fixes where possible.
+### Maintenance
+
+**`/brain-refresh`** — Re-scan the vault for new or changed files and regenerate vector embeddings. Run after a bulk ingest session or manual edits. Pass `force` to re-embed every chunk (e.g. after changing the embedding model).
+
+**`/brain-rebuild`** — **Destructive.** Redesigns the qmd schema: analyzes the wiki, proposes new collections and contexts, waits for your approval, then patches `scripts/qmd/setup.ts`, drops the old index, and rebuilds embeddings from scratch. Use only when the current structure no longer fits how you search.
+
+### Setup
+
+**`/setup`** — First-time initialization. Registers the qmd collections and generates local vector embeddings. Run once after scaffolding.
+
---
## Access from anywhere
@@ -176,14 +188,14 @@ my-brain/
## Installing and updating skills
-Skills are slash commands Claude Code loads from `.claude/skills/[name]/SKILL.md`. `/brain-ingest` and `/brain-search` are installed globally to `~/.claude/skills/` during setup so they work in any Claude Code session. `/lint`, `/setup`, and `/qmd-cli` are installed into the vault itself.
+Skills are slash commands Claude Code loads from `.claude/skills/[name]/SKILL.md`. `/brain-ingest`, `/brain-search`, and `/brain-refresh` install globally to `~/.claude/skills/` during setup so they work in any Claude Code session. `/brain-rebuild`, `/lint`, `/setup`, and `/qmd-cli` live inside the vault at `.claude/skills/`.
### Update the built-in wiki skills
-The wiki's own skills (`/brain-ingest`, `/brain-search`, `/lint`, `/setup`, `/qmd-cli`) are scaffolded at creation time. To pull in improvements, use `npx skills` pointing to the template skills directory in this repo:
+The wiki's own skills are scaffolded at creation time. To pull in improvements, use `npx skills` pointing to the template skills directory in this repo:
```bash
-# Install or update all 5 wiki skills from the latest template
+# Install or update all 7 wiki skills from the latest template
npx skills add https://github.com/jessepinkman9900/claude-second-brain/tree/main/template/.claude/skills -a claude-code -y
# Or update a specific skill
@@ -200,7 +212,7 @@ npx skills update -a claude-code
## Roadmap
-- [ ] GitHub Actions for scheduled re-indexing
+- [ ] GitHub Actions to invoke `/brain-refresh` on a schedule
- [ ] GitHub agentic workflows — auto-ingest on push, scheduled lint, auto-summary on new sources
- [ ] More Claude Code skills (source discovery, topic clustering)
diff --git a/bin/create.js b/bin/create.js
index befff11..43094b9 100644
--- a/bin/create.js
+++ b/bin/create.js
@@ -62,7 +62,7 @@ async function patchVault(targetDir, qmdPath, brainName) {
async function installGlobalSkills(qmdPath) {
const globalSkillsDir = join(homedir(), ".claude", "skills")
- await Promise.all(["brain-ingest", "brain-search"].map(async skillName => {
+ await Promise.all(["brain-ingest", "brain-search", "brain-refresh"].map(async skillName => {
const srcFile = join(TEMPLATE, ".claude/skills", skillName, "SKILL.md")
const destDir = join(globalSkillsDir, skillName)
diff --git a/template/.claude/skills/brain-rebuild/SKILL.md b/template/.claude/skills/brain-rebuild/SKILL.md
new file mode 100644
index 0000000..ae5bf4c
--- /dev/null
+++ b/template/.claude/skills/brain-rebuild/SKILL.md
@@ -0,0 +1,119 @@
+---
+name: brain-rebuild
+description: "Redesign the qmd schema for this vault — analyze the current wiki, recommend a new set of collections and contexts, update scripts/qmd/setup.ts, then tear down and rebuild the index from scratch. Destructive. Trigger phrases: /brain-rebuild, rebuild the brain, restructure qmd, redesign collections, redo contexts, recompute schema."
+argument-hint: "Optional: a hint about the desired structure (e.g. 'split wiki by domain', 'add a separate collection for qa pages')"
+---
+
+# Brain Rebuild
+
+Redesigns the qmd schema based on what the wiki actually contains today, then rebuilds the index from scratch. **Destructive** — wipes existing collections/contexts and their embeddings. Always get explicit user approval before applying changes.
+
+## When to Use
+
+- The wiki has grown and the current single `wiki` collection no longer matches how the user searches
+- The user wants finer-grained contexts (e.g. distinct context descriptions per sub-folder)
+- After significant reorganization of `wiki/` or `sources/` folder structure
+- Never as a routine refresh — for that, use `/brain-refresh`
+
+## Procedure
+
+All commands run from the vault root.
+
+### Step 1 — Survey the current state
+
+- Read `scripts/qmd/setup.ts` to see the current collections and contexts
+- Read `CLAUDE.md` to understand the documented schema and any references to collection names
+- List current state from qmd:
+ ```bash
+ INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd collection list
+ INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd context list
+ INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd status
+ ```
+
+### Step 2 — Analyze the wiki
+
+- Glob `wiki/**/*.md` and `sources/**/*.md`
+- Read enough pages (especially `wiki/index.md` and `wiki/overview.md`) to understand actual topic clusters, page-type distribution, and folder structure
+- Identify natural groupings: by domain (e.g. ml, distributed-systems, finance), by page type (topics vs entities vs qa), by source provenance, etc.
+
+### Step 3 — Propose a new schema
+
+Produce a recommendation that includes:
+
+- **Collections** — name, root path (relative to vault), file glob pattern. Justify each split.
+- **Contexts** — global context, plus per-collection and per-subpath context descriptions. Each context should be a one-sentence description that helps a future search disambiguate between similar pages.
+- **Migration impact** — what changes for `/brain-ingest`, `/brain-search`, `/lint`, and `CLAUDE.md` (e.g. if collection name `wiki` becomes `wiki-topics`, all `-c wiki` invocations break)
+
+Present the recommendation as a clear diff against the current schema. Do not modify any files yet.
+
+### Step 4 — Get explicit approval
+
+Ask the user: "Apply this schema? This will drop existing collections and their embeddings, then rebuild from scratch. Re-embedding the full wiki may take significant time and download GGUF models if not cached."
+
+Do not proceed without an explicit yes.
+
+### Step 5 — Update `scripts/qmd/setup.ts`
+
+Edit `scripts/qmd/setup.ts` to reflect the new schema:
+
+- Update the `ensureCollection` calls (add, remove, or rename)
+- Update `setGlobalContext` and the `addContext` calls
+- Keep the `__QMD_PATH__` / `DB` line untouched — it was already substituted at install time
+- Keep the script idempotent
+
+### Step 6 — Update related references
+
+If collection names changed:
+
+- Update every occurrence in `CLAUDE.md` (e.g. `qmd query -c wiki` → new name)
+- Update `.claude/skills/brain-ingest/SKILL.md`, `.claude/skills/brain-search/SKILL.md`, `.claude/skills/lint/SKILL.md`, `.claude/skills/qmd-cli/SKILL.md` — anywhere collection names are hard-coded
+- Use Grep to find any remaining stale references before moving on
+
+### Step 7 — Tear down the old schema
+
+For each existing collection that no longer fits the new schema:
+
+```bash
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd collection remove <old-name>
+```
+
+For each obsolete context:
+
+```bash
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd context rm <path>
+```
+
+### Step 8 — Register the new schema
+
+Run:
+```bash
+bun scripts/qmd/setup.ts
+```
+
+This is idempotent — collections that already exist (e.g. ones you kept) are skipped; new ones are added; contexts are upserted.
+
+### Step 9 — Rebuild embeddings from scratch
+
+Run:
+```bash
+bun scripts/qmd/reindex.ts
+```
+
+This indexes all files under the new collections and generates fresh embeddings.
+
+### Step 10 — Verify and report
+
+```bash
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd collection list
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd context list
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd status
+```
+
+Confirm the new collections appear, contexts match the plan, and document/embedding counts are non-zero. Report a summary of what changed (collections added/removed/renamed, contexts updated, files re-indexed, references updated in CLAUDE.md and skills).
+
+## Hard Rules
+
+- Never apply schema changes without explicit user approval at Step 4
+- Never delete a collection without first showing the user what will be dropped
+- Always update `CLAUDE.md` and skill files in lockstep with collection renames — stale `-c <name>` references will silently break `/brain-search`
+- Do not touch `wiki/`, `sources/`, or any user content — this skill changes the index schema, not the data
diff --git a/template/.claude/skills/brain-refresh/SKILL.md b/template/.claude/skills/brain-refresh/SKILL.md
new file mode 100644
index 0000000..7689dea
--- /dev/null
+++ b/template/.claude/skills/brain-refresh/SKILL.md
@@ -0,0 +1,63 @@
+---
+name: brain-refresh
+description: "Refresh the qmd index for this vault — re-scans all collections for new/changed files and regenerates vector embeddings. Use after a bulk ingest session, after manual file edits, or whenever search feels stale. Trigger phrases: /brain-refresh, refresh the brain, refresh embeddings, re-embed wiki, update qmd index, refresh search index."
+argument-hint: "Optional: 'force' to force re-embedding of every chunk (slow, useful after embedding model changes)"
+---
+
+# Brain Refresh
+
+Refreshes the qmd index so search reflects the current state of the vault. Wraps `bun scripts/qmd/reindex.ts` (incremental) and the qmd CLI's force-embed flag.
+
+## When to Use
+
+- After a `/brain-ingest` session (or several) — batch the refresh, don't run after every file edit
+- After manual edits to `wiki/` or `sources/` files
+- When `/brain-search` results feel stale or miss recently added content
+- After upgrading `@tobilu/qmd` or changing the embedding model — use `force` mode
+
+## Procedure
+
+All commands run from the vault root.
+
+### Default — Incremental Refresh
+
+Run:
+```bash
+bun scripts/qmd/reindex.ts
+```
+
+This script does two things:
+1. **Update** — scans collections for new, changed, or deleted files and updates the index
+2. **Embed** — generates vector embeddings for any chunks that need them
+
+Only chunks that changed get re-embedded, so this is fast on subsequent runs.
+
+### Force Mode (`force` argument)
+
+When the user passes `force`, re-embed **every** chunk — not just the changed ones. Use after embedding-model changes or if embeddings appear corrupted.
+
+```bash
+# Step 1 — update the file index first
+bun scripts/qmd/reindex.ts
+
+# Step 2 — force re-embed everything
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd embed -f
+```
+
+Force mode is slow — confirm with the user before proceeding if the wiki is large.
+
+## Verify
+
+After refresh, confirm with:
+
+```bash
+INDEX_PATH=__QMD_PATH__ bunx @tobilu/qmd status
+```
+
+Document and embedding counts should be non-zero and reflect recent activity. If embeddings show as `0` or far below document count, re-run the refresh.
+
+## Notes
+
+- First run downloads ~2GB of GGUF models — expected, one-time
+- Do not run this after every single file edit — batch it after a session
+- This skill does **not** change the qmd schema (collections, contexts). For that, use `/brain-rebuild`
diff --git a/template/README.md b/template/README.md
index 6aea1ce..a3b59d4 100644
--- a/template/README.md
+++ b/template/README.md
@@ -34,12 +34,24 @@ Registers the qmd collections and generates local vector embeddings. First run d
## Your Claude Code skills
+### Daily workflow
+
**`/brain-ingest`** — Add a file to `sources/articles/`, `sources/pdfs/`, or `sources/personal/`, then run `/brain-ingest`. Claude summarizes the source, asks what aspects matter most, updates related wiki pages, flags contradictions, and logs everything.
**`/brain-search`** — Ask anything: `what do I know about [topic]?` Claude searches the wiki semantically and returns a cited answer. If it synthesizes multiple pages in a useful way, it offers to file it as a permanent `wiki/qa/` entry.
**`/lint`** — Health-check the wiki. Finds orphan pages, broken links, unresolved contradictions, and data gaps. Reports findings and fixes what it can.
+### Maintenance
+
+**`/brain-refresh`** — Re-scan the vault for new or changed files and regenerate vector embeddings. Run after a bulk ingest session or manual edits. Pass `force` to re-embed every chunk.
+
+**`/brain-rebuild`** — **Destructive.** Redesigns the qmd schema: analyzes the wiki, proposes new collections and contexts, waits for your approval, then patches `scripts/qmd/setup.ts`, drops the old index, and rebuilds embeddings from scratch.
+
+### Setup
+
+**`/setup`** — First-time initialization. Registers the qmd collections and generates local vector embeddings. Run once after scaffolding.
+
---
## Obsidian Mobile
@@ -86,14 +98,14 @@ claude-second-brain/
## Installing and updating skills
-Skills are slash commands Claude Code loads from `.claude/skills/[name]/SKILL.md` in this vault. The wiki ships with `/brain-ingest`, `/brain-search`, `/lint`, `/setup`, and `/qmd-cli` pre-installed.
+Skills are slash commands Claude Code loads from `.claude/skills/[name]/SKILL.md` in this vault. The wiki ships with `/brain-ingest`, `/brain-search`, `/brain-refresh`, `/brain-rebuild`, `/lint`, `/setup`, and `/qmd-cli` pre-installed.
### Update built-in wiki skills
-Pull the latest `/brain-ingest`, `/brain-search`, `/lint`, `/setup`, and `/qmd-cli` from the upstream template:
+Pull the latest skills from the upstream template:
```bash
-# Install or update all 5 wiki skills
+# Install or update all 7 wiki skills
npx skills add https://github.com/jessepinkman9900/claude-second-brain/tree/main/template/.claude/skills -a claude-code -y
# Or update a specific skill
@@ -112,11 +124,11 @@ npx skills update -a claude-code
After a bulk ingest session, re-index to keep search current:
-```bash
-bun scripts/qmd/reindex.ts
+```
+/brain-refresh
```
-Or run `/setup` again inside Claude Code.
+This wraps `bun scripts/qmd/reindex.ts` — you can also run that command directly if you're not inside Claude Code. Pass `force` to `/brain-refresh` to re-embed every chunk (e.g. after changing the embedding model).
---